Live Cross-Cloud Migrations With Aeolus and Snap

In the past when evaluating cloud providers, one had to to make a critical decision right up front, to select the provider that offers the features needed at a price that is agreeable to the end-user. If the provider changed their terms of service and/or pricing scheme, the sysadmin/developer was more or less stuck with their choice, unless they made the painful effort of manually migrating all software and data to another provider. Well not anymore.

I'm pleased to announce that with the assistance of Aeolus, Red Hat's Open Source Cloud API, and Snap! a system snapshotter and restoration utility (started as my Masters project in '07), it is now possible to migrate cloud instances of any type across cloud providers (currently supported distros are Fedora, Ubuntu, Debian, RHEL, and CentOS with more and Windows support in development / coming soon).

For details, see the following step-by-step guide complete w/ screenshots of how to migrate instances between Amazon EC2 and Rackspace (this will work for migrating instances between any cloud providers supported by deltacloud, the core Aeolus component/api). Everything will be migrated from the source instance to destination on the fly (no instance downtime), from package repositories and installed software, to files modified outside the package management system, to services and their configurations, etc.

  • To start off we are assuming you have an instance running on EC2 which you can ssh into. Here you can see a Fedora 15 instance running a mediawiki install. (click to see bigger screenshots)

  • The machine I was orchestrating the migration from was my local F15 laptop. I setup environment variables in my bash profile corresponding to the following:

    • $EC2_USERNAME: my ec2 access key, retrieved from the AWS security credentials screen
    • $EC2_PASSWORD: my ec2 secret access key, also retrieved from the AWS security credentials screen
    • $RACKSPACE_USERNAME: my rackspace login name
    • $RACKSPACE_PASSWORD: my rackspace API key, retrieved by logging into rackspace and navigating to Your Account > API Access

  • On my local laptop, I installed the deltacloud drivers for ec2 and rackspace. Deltacloud implements a client/server architecture for remote management, so I also installed the deltacloud client

    $ sudo yum install deltacloud-core-ec2 deltacloud-core-rackspace rubygem-deltacloud-client

  • I then started deltacloud servers for both the ec2 and rackspace drivers (note it is now possible w/ the upstream deltacloud project to use the same server for multiple drivers)

    $ deltacloudd -i ec2 -p 3002

    $ deltacloudd -i rackspace -p 3003

  • Next I used the deltacloud client to retrieve information about my running EC2 instance

    $ deltacloudc instances index -u http://$EC2_USERNAME:$EC2_PASSWORD@localhost:3002/api

    Note in this screenshot, you can see the ec2 instance hostname which I can ssh to (due to the formatting of the field, '' has been truncated)

  • Now I ssh'd into the instance, specifying my keypair previously configured on Amazon when starting the instance

    $ ssh -i ~/.ssh/mmorsi-keypair.pem

  • On the cloud instance, I used yum to install Snap. Until the Snap package is accepted and avaiable in Fedora / Ubuntu / other mainstream distributions, I have hosted binary packages in my own yum and apt repositories

    $ sudo wget -O /etc/yum.repos.d/morsi.repo

    $ sudo yum install snap

  • Due to how Amazon handles its cloud security policy (eg through their security groups concept), iptables is not setup on EC2 instances. Snap will by default backup and restore iptables rules, so we have to edit the snap configuration file to remove this backup target

    $ sudo vim /etc/snap.conf

    Remove the hightlighted content

  • Before running the snapshot, lets take a look at the options Snap presents us. Note Snap provides both a command line tool and a graphical GTK based one (more on snap coming in a future blogpost)

    $ snaptool --help

  • Now we run the actual snapshot

    $ sudo snaptool --backup --log-level verbose --norepos --packages --nofiles --services

    We pass in --backup to perform a backup operation instead of a restoration one, specify verbose output with --log-level, and configure the snapshot targets. Snap offers a very modular / extensible plugin system in which targets to backup / restore can easily be defined, and operating system specific backends to those targets implemented. In our case we opt to backup the packages installed on the instance as well as the running services. We do not need to backup any package repositories as we are only using stock fedora software and no extra files are required (--services will backup the files required for those services to run)

  • This produces a snapshot in /tmp/snap-shot.tgz (the location of this can be configured by specifying --snapfile to snaptool). Back on our host system, we scp that file locally

    $ scp -i ~/.ssh/mmorsi-keypair.pem .

  • We can easily inspect the snapshot as it is a tarball (snap also supports snapshot encryption/decryption, protecting the contents of the tarball)

    $ tar tzvf snap-shot.tgz

  • Next we goto launch a new instance on our destination cloud, Rackspace. To do this we first view the images available to base our instance off of

    $ deltacloudc images index -u http://$RACKSPACE_USERNAME:$RACKSPACE_PASSWORD@localhost:3003/api

    We've highlighted an image matching the OS out our source instance (note it is theoretically possible to restore the snapshot on a completely different OS! I have yet to try this out yet though so no guarantees)

  • Now we spin up a new instance using the selected image as the basis

    $ deltacloudc instances create -u http://$RACKSPACE_USERNAME:$RACKSPACE_PASSWORD@localhost:3003/api --image-id 78

    We see the ip address of the instance, which we can SSH to, right from this output. Once the instance is started (usually within a minute from my findings) Rackspace sends an email containing the root password

  • Before logging in, we scp our snapshot over to the new instance

    $ scp snap-shot.tgz root@

  • We now ssh into the new instance

    $ ssh root@

  • We use the same wget and yum commands as above to install Snap

    # wget -O /etc/yum.repos.d/morsi.repo

    # yum install snap

  • Now we restore the snapshot, again using snaptool

    # snaptool --restore -l verbose --snapfile snap-shot.tgz

  • Finally since the EC2 security groups mechanism is different than the standard iptables solution, and we weren't able to simply backup/restore the firewall policy (as Snap does by default), we need to open port 80 for http in our firewall. We use the text based firewall manager that ships w/ RH based system to do so

    # system-config-firewall-tui

  • Bring up a web browser, point it at our new instance and WHALA! the world's first open source cross-cloud migration!

Stay tuned for more updates. To the cloud!