PostgreSQL, how about you don’t make migration more complex

I am currently work on building a new site, from an existing one.  A lot of the front-end work is done, now to a lot of the admin.

Today a really quite annoying “feature” of PostgreSQL raises its head and takes up 3 hours of my time.  A lot of the data from the old site PHP/MySQL has been migrated to the new one Django/PostgreSQL which all worked fine, the front end works and its all lovely.  To maintain the old database id’s we inserted them all into the new database, this in itself is not the problem.  The problem arose what starting to adding new rows, which are expected to go to the end.

INSERT INTO table (title) VALUES ('my_title');

Instead of actually insert into a new row, this started working from id 1, throwing up errors whenever the id already existed and sometimes started to delete items from the end of the database.

Just what the hell.  After plenty of research and testing, initially expecting it to be a problem with my code and the problem is found.

Adding the Primary Key manually DOES NOT update the auto-increment value, and here is the fix (change field names where necessary)

SELECT SETVAL('sequence_name', (SELECT MAX(id) FROM table_name) + 1);

If you are using Django  you can generate the SQL by running, it does not run the automatically but you can pipe the response directly into the psql tool.

python manage.py sqlsequencereset appname

Very simple but this does not follow the MySQL style I am used to, then again SQLite also does things differently which had me stumped at first.

Argh….The Cloud

Cloud/VPS hosting is great, except when its not.  The idea is, you have dedicated hardware to take over if something fails and on smaller installations, is quite a bit cheaper.

I have been using it for a over 12 months and for the most part its been fine. There have been a few occasions where there has been a bit of downtime, but it was early days…..except today there has been some quite serious downtime.

A power outage takes down a large chunk of machines, both primary and backup machines.  Not much of a problem I hear you say…WRONG.

A conventional server

If a power outage takes out a datacenter, once its back remote start, manually pressing up to 42 buttons per rack boots servers, or servers that just reboot when power is back (this will be most I’d guess), a couple of minutes later everyone’s server us running with minimal issue.

The Cloud

When a VPS cloud is affected the process is very different.  The main servers are booted, the disk servers are booted at which point not a single “client” server has started.  Now we go through the large queue of servers starting them, good luck if your at the end, up to 4 hours it seems to recover from this one.

Is the Cloud worth it?

This is a difficult question to answer.  The cloud can handle multiple failures at the same time (or at least its meant to) however due to the many number of components, the chance of something failing is far greater.  Having your own machine, if your down due to a hardware fault…that’s it..your down.

I still like the Cloud scenario, almost instant deployment of new servers & more CPU than you are often going to get in a standalone box.  It really does depend on what your system requirements are.  I’m hosting a game server in the cloud.  It would cost me the same, if not more, to just get a game server from a game server provider – without the flexibility I have now, but a lot more to host 1 game server on the cheapest dedicated server you could fine.

Dansette