Backing-Up To EC2 From nld

time required = f(instance and/or volume assigned by amazon)
  1. browse to http://console.aws.amazon.com/, then EC2, then Snaphots
    • (The next volume creation step could be skipped with the new workflow in amazon, which allows for a volume to be created from a snapshot on launch of an instance, but if it ain't broke don't fix it. Although, the integrated volume creation on instance launch probably guarantees that the instance and volume will be in the same zone.)
  2. create a new volume of size 64GB from the existing non-root snapshot. The volume MUST be the same size as the previous back-up. If more space is needed:
    • create the volume with more space
    • attach it as /dev/sdf (which maps to /dev/xvdf) to a running instance
    • try running nxclient (see Aside below), installing gparted, and using it to resize
    • run resize2fs -f -p /dev/xvdf # without -f you have to run e2fsck which takes waaaaay too long; with it the resize takes 75 seconds; there's no need to specify the new size because the resize consumes all space by default
  3. note the availability zone of the volume
  4. on the "AMIs" page, launch a new instance with these values:
    • ami = ami-3b797252 (copied then customized image from https://console.aws.amazon.com/ec2/v2/home?region=us-east-1#LaunchInstanceWizard:ami=ami-182f5d71)
    • aki = aki-407d9529 (which is already associated with the ami)
    • ari = N/A
    • m1.medium instance type since the t1.micro can be a severe bottleneck; m1.medium is inexpensive as of 2014.03.01
    • availability zone as noted above
    • default (existing) security group
    • ec2-key key pair
      • This isn't necessary since /etc/cloud/cloud.cfg was hacked on creation of the AMI to allow login via password
      • The key is necessary to login as the default user fedora, eg ssh -i /home/dave/ec2-key.pem fedora@ec2
  5. wait for the machine to come to life, which can take a little while
  6. copy the machine's public ip address info into nld:/etc/hosts, specifically including ec2.anoxymous.com as an alias
  7. Aside: the machine has nx, vncserver, etc installed, so it could be used for something more useful than just backing-up
    • Start nxclient and login as dave@ec2
    • Start vncserver as dave on ec2 with vncserver -geometry 1360x768
    • Start vncviewer on ec2 with vncviewer localhost:1
    • VoilĂ  at your desktop in the cloud
  8. attach the storage volume to the instance
    • make the volume device /dev/sdf, which gets attached as /dev/xvdf
  9. ssh into the ec2 machine as root (fuck you) expecting the ssh warning
  10. change /home in /etc/fstab to /dev/xvdf1 /home ext4 noauto,noatime,defaults 0 0
  11. mount /home, which can take a while for some reason sometimes
  12. ln -s /home .unison
  13. copy the machine's public ip address info into ec2:/etc/hosts, specifically including ec2.anoxymous.com as an alias
  14. rsync heavy stuff from nld into ec2
    • check out nld:/root/.unison/home for what's ignored and/or considered heavy
    • /home/dave/journal is ignored, so it's heavy - rsync it
  15. nice unison (through vncviewer) on nld as root to take care of the rest
    • the archive info on ec2 is stored in /home/ar79f3a5a9be151ab0920ff7773d9873fa
      • ec2:/root/.unison is a symlink to /home, so ar79f3a5a9be151ab0920ff7773d9873fa is persisted; it's the key to the operation because the info is stored in the storage volume, not the root filesystem, which is not persisted from instance invocation to instance invocation.
  16. sync
  17. df -h # note the free space on /dev/xvdf
  18. umount /home (at this point a snapshot of the volume can be created, even without the instance running)
  19. shutdown the instance, which automagically detaches the volume (and takes a little while)
  20. create a snapshot of the volume and add the free space as a description (it took ~22 minutes as of 2010.01.29, but took a LOT longer previously)
  21. delete the volume and old snapshot
  22. pay a paltry 0.10$ per GB per month for wickedly redundant back-up :)