Monthly Archives: December 2014

How to Migrate 3 MySQL Databases to Amazon RDS in 5 Minutes

I had a LAMT (Linux-Apache-MySQL-Tomcat) on Amazon EC2 and I wanted to move all remaining MySQL databases (3) to an existing Amazon RDS instance. This would allow me to shutdown the MySQL instance on EC2, freeing RAM for Tomcat and leveraging RDS automated backups for those 3 databases in case of a disaster. The databases to migrate only contain low volume TEST data, but I already have that RDS instance, so why not use it?

I also have a SLA with my clients that allows me to perform “Standard daily maintenance“. Basically, theĀ “Standard daily maintenance” must be performed outside business hours and must last at most 15 minutes.

The key to the success of this migration was to prepare, prepare and prepare before the actual migration. So here is what I did before the migration:

  • Create 3 empty databases and users on the RDS instance
  • Prepare new configuration files with the new JDBC url pointing to the RDS instance
  • Prepare all the commands that must be executed
  • Review and test the commands as needed

Now that I am prepared I just need to wait for theĀ “Standard daily maintenance” time. Then, I just copy and paste the commands in a terminal. I prefer to copy/paste the commands one by one, so if any command fails (for any reason), it can be fixed right away before running the next commands. Here is a summary of the commands:

  1. Shutdown Tomcat
  2. MySQL dump/restore from EC2 to RDS (3 databases)
  3. Copy the configuration files with JDBC url pointing to RDS (3 files)
  4. Prevent MySQL from starting during boot: echo manual > /etc/init/mysql.override
  5. sudo reboot (I want to verify that MySQL on EC2 won’t start after a reboot)

Everything went fine. After reboot, the MySQL instance on EC2 was not started, as expected. The Tomcat webapps were fine as well and there is more free RAM for Tomcat.

For those webapps, I have uptime monitoring at one minute interval with Pingdom. I receive emails when a webapp go down and up again. Here is the “UP” email from Pingdom, showing 5 minutes of downtime for that migration:

OpCode is UP again at 12/22/2014 08:25:36PM, after 5m of downtime.

 

Sharing few thoughts on my startup experience

I am glad to learn that IO Road has been published on FeedMyApp. Lately, I’ve been doing some PR for ioroad.com and it is good to get a tiny bit of recognition from an independent organization. I believe I am very good at developing software, but launching a product like IO Road requires more than “begin good with software development”. So if you need a lesson of humility, you may consider launching a new product :) . I am learning a lot in this process, which is good because it was one of the intended goals (and I was prepared to that)

The biggest challenge for me is to choose where to put the resources (time and money, but mostly time). I am trying to apply some principles of the Lean Startup and I realize the more I apply them, the more I understand them. Which mean I did not fully grasp them in the first place. Like the ‘MVP‘ thing, I realize I could have been much further with the ‘M’ part :).

Also, sometimes it is tempting to fall to the dark side of SEO, but I don’t believe I can actually beat Google’s algorithm, lol. If I had to develop a search engine, after a few days of thinking, I believe I could come up with pretty clever ways of using machine learning (mostly clustering) to detect “unusual linking patterns”. So I don’t even want to imagine what top engineers at Google came up with after years and years of improvements to their algorithm. This keeps me on the safe side, or at least, that is what I believe. For that matter, I am wondering how Google actually sees this post. At least, it is an honest attempt to share few thoughts on my experience.