I am currently working on building a new site which is migrating from an existing one.  Front-end work mostly complete, now to creating the admin.

The Problem

Today, a really quite annoying “feature” of PostgreSQL raises its head and takes up far too much time to resolve.  For some background, the data is from an old site built in PHP & MySQL, being migrated to Django & PostgreSQL. This has been working fine, the frontend works and that’s great.

To maintain the old database id’s we inserted everything en-mass into the new database. This in itself was not the problem.  The problem arose when starting to adding new row and are expected auto-increment to the end.

Just a simple query, nothing overly complicated here.

INSERT INTO table (title) VALUES ('my_title');

However, instead of actually inserting into a new row at the end, it started working from id 1. This throws up errors whenever the id already existed (this differs from MySQL). Worse still, on some occasions started to delete items from the end of the database.

The Solution

So, what’s going on? I was initially expecting it to be a problem with my code although but, after plenty of testing, that wasn’t the case.

The primary key wasn’t updating the auto-increment value as expected. Here was the simple fix (change field names where necessary). This will reset the internal counter to the next available number.

SELECT setval('"table_id_seq"', coalesce(max("id"), 1), max("id") IS NOT null) FROM "table";

If you are using Django  you can generate the SQL by running, it does not run the automatically but you can pipe the response directly into the psql tool.

python manage.py sqlsequencereset appname

Very simple, but this does not follow the MySQL (less control over increments and automatically finding the next available number). Then again SQLite also does things differently and no doubt others as well.

It really pays to be aware of some of the fundamental, often unexpected, differences between databases and frameworks. Embrace change, don’t fear it. Do your research, is it better?, does it make for a logical change? But, don’t embrace unnecessarily.

Cloud/VPS hosting is great, except when its not.  The idea is, you have dedicated hardware to take over if something fails and on smaller installations, is quite a bit cheaper.

I have been using it for a over 12 months and for the most part its been fine. There have been a few occasions where there has been a bit of downtime, but it was early days…..except today there has been some quite serious downtime.

A power outage takes down a large chunk of machines, both primary and backup machines.  Not much of a problem I hear you say…WRONG.

A conventional server

If a power outage takes out a datacenter, once its back remote start, manually pressing up to 42 buttons per rack boots servers, or servers that just reboot when power is back (this will be most I’d guess), a couple of minutes later everyone’s server us running with minimal issue.

The Cloud

When a VPS cloud is affected the process is very different.  The main servers are booted, the disk servers are booted at which point not a single “client” server has started.  Now we go through the large queue of servers starting them, good luck if your at the end, up to 4 hours it seems to recover from this one.

Is the Cloud worth it?

This is a difficult question to answer.  The cloud can handle multiple failures at the same time (or at least its meant to) however due to the many number of components, the chance of something failing is far greater.  Having your own machine, if your down due to a hardware fault…that’s it..your down.

I still like the Cloud scenario, almost instant deployment of new servers & more CPU than you are often going to get in a standalone box.  It really does depend on what your system requirements are.  I’m hosting a game server in the cloud.  It would cost me the same, if not more, to just get a game server from a game server provider – without the flexibility I have now, but a lot more to host 1 game server on the cheapest dedicated server you could fine.