Now a word from Stefan Antonowicz, our Director of Technology regarding our recent attempt again to migrate to WordPress. – Natalie
We were so close! This time, everything booted up as expected and scaled nicely. I sat back in my chair, well pleased with myself, until I got an IM from the sysad helping me out:
“Error establishing a database connection”
“Uh oh,” I thought. But we persevered, or attempted to – I hoped that this was just a result of all the pages attempting to generate their cache files, but after ten minutes or so I saw that the load on the database server was unusually high. The error message was being thrown once every handful of connect attempts (your mileage may have varied, of course). Not OK. We rolled back the DNS changes and went back to make some tweaks.
My understanding at this point was that the AWS cloud instance for the database wasn’t beefy enough. We’re reconfiguring things to run on a bigger virtual machine, and I’m taking a look at our benchmarking application to see why it didn’t pick up the db load during testing. We’re looking to try again next week or two once we have the evening resources available again and get a few other projects off our plate.
Anyway, we’re looking at this as a partial victory – the application seemed to hold up well under the load, and when they couldn’t reliably connect to the database they started spawning new instances (exactly what we wanted). Not fully back to the drawing board – let’s call it back to the tweaking board. You never know when boingboing or slashdot will strike, after all, and I want to be sure everyone is getting the best experience possible.
Thanks for bearing with us. Third time’s a charm, right guys?