So Dreamhost, odd place they are, have apologized to their users for a long stint of server and network problems (which seems to be resolved, knock on wood) by doubling our monthly bandwidth and 10x-ifying our plans’ disk quotas. That means I’ve got 413GB of storage there now. I don’t have 413GB worth of website and mail. (That’s also $10/month for 200GB of data, which is $0.05/GB, or 1/3 of Amazon S3’s price!)
So since I recently lost 200GB of data at home when both disks in a RAID set failed at the same time*, I figure I might as well use that for backing up. I do have a 35/70 AIT drive at home, but the new server has 320GB total, and that sounds like a lot of work changing tapes compared to shoving it over the network.
But to back up there I have a few requirements, or at least I think they are requirements, and I want to poll the lazyweb to see if I’m way off base or if there’s something that will already do all this for me. So, my requirements:
- Incremental. I can’t upload all 320GB nightly or even weekly. It doesn’t have to be completely bandwidth-cheap like rsync, but it has to at least be incremental at the file level.
- Encrypted. I’m backing up things like tax records, so I don’t want anyone at Dreamhost to be able to see the data. I don’t even want them to know what it is, so both the file and directory names and the file contents need to be hidden from prying eyes. I’m not worried about the existence of directory structures; it’s ok to reproduce the tree exactly as long as the directory and filenames are unreadable.
- Individual files. Dreamhost themselves offer Netapp-style .snapshot directories, so I need to upload individual files and not, say, one giant encrypted cpio archive, in order to be able to take advantage of those .snapshot directories for restoring files from versions prior to the last backup.
- Metadata-friendly. I want to keep file ownerships and permissions so I can restore these files later without having to worry about fixing ownerships and permissions on hundreds of GB of data, but I only have a single userid on the far end with which to store them.
Is there anything out there that comes close to what I need, that’d be extensible in the way I describe, or do I have to write this all myself? Are there any other requirements I’m forgetting? If it helps, at the Dreamhost end I have a Linux shell account via SSH, plus FTP, DAV, and maybe SMB access.
(Brad, brackup does all of this, right? :-)
* Yeah, I should know better. But I had RAID-1 and an unused tape drive, what could go WRONG? What went wrong: A scheduled power outage at home that must have come up badly, that took out what was apparently already a marginal power supply in my file server, which in turn managed to let the magic smoke out of everything it could, including two identical hard drives. Incidentally, as of this server I don’t run identical hard drives in my software RAID sets now! The guy at the computer store was confused but once I explained he seemed as though he was going to start recommending the same to everyone.