The days of using a few floppy disks to backup important files are long gone. In going through my website I realized that the information about my backup strategy was a bit dated and didn’t reflect how I do things. Going through everything in order to write this article would also help me be sure I had a bulletproof backup strategy. The goal is to have a solid backup solution so I never have to use a file recovery tool like Data Rescue II. So here goes…
My Backup Philosophy
My backup philosophy still hasn’t changed:
- A file doesn’t exist unless it exists in at least two places
- RAID (or RAID like technologies such as Drobo or Windows Home Server) does not mean the file exists in two places.
- To be truly protected, the two places must be geographically separated.
- The backup has to be automatic and unobtrusive. I’m lazy, if I have to manually initiate the backup it won’t happen.
That third and fourth points can be a problem for large groups of files such as my music and video libraries which total over 5TB of data. But I’ll deal with that later.
I also like to have two copies of the file locally (including the “live” copy) to allow quick restores, although this isn’t possible for my large video library.
Backup Software (and Services) Used
I make use of the following software and services in my backup strategy:
Jungle Disk (with files saved to Amazon S3) – This is my primary offsite backup software. I like it because it’s cross platform (Windows, OS X, Linux). Since pricing is based on usage it costs me a bit more than $5/mth which is typical of many other backup plans. But it’s been reliable so I consider it money well spent.
Windows Home Server – I have file duplication enabled for all files on the server. The WHS software includes backing up PCs and this is my primary in-house backup for my Windows PCs.
Jungle disk and Windows Home Server are my primary backups tools but the following software also helps out.
Drop Box – Another free service (for 2GB of space). Drop Box allows files to be synced between PCs. I primarily use it to sync settings between computers along with some files but in addition to putting the files on multiple PCs copies are also save on the web so it counts as offsite backup. As an added bonus it saves archives of deleted and changed files for 30 days.
I use ChronoSync this to sync files to my Windows Home Server. I use it to move files between my Mac and Windows Home Server as a way of having local backups. On the Windows side I use Microsoft’s SyncToy to sync between my Windows PCs and my WHS.
MozyHome Free – I use the free version of Mozy to backup some Windows files. I used to use the paid version on my Mac but after some serious problems with Mac Mozy I dropped the subscription. I use the free service for three reasons: It’s been reliable on Windows, it’s free, and it keeps historical copied of changed/deleted files for 30 days. It’s that last item that’s important. There are a few files that I want to keep historical copies of for awhile. With Jungle Disk I’d be charged for the space these historical copies use and I’d have to set up a separate backup in order to treat the files differently. The reality is I’ll probably never restore archive copies from my offsite backup, after all I have Windows Home Server. But Mozy’s free and unobtrusive so I’ve kept it.
Jungle Disk and Mozy share a security feature I want in my offsite backup. I supply my own encryption key and the files are encrypted on my local computer before being sent to the “cloud”. Neither Jungle Disk or Mozy have copies of my encryption key so if anyone gets access to my files they still can’t decrypt them.
The offsite backups exist for those cases I really hope never happen – from natural disasters and fires to someone breaking in and taking my computer and nearby drives. Except for those cases I’ll be counting on the local backups.
My iMac (running Snow Leopard) has my iTunes music library and an iPhoto library although both are on a external Drobo drive connected to the iMac. While providing RAID like protection, as point 2 states above, it’s not a backup. Other data files are either synced via Drop Box or saved on my Windows Home Server. I really don’t save miscellaneous data to my iMac anymore, they get saved directly to the server. But any miscellaneous files would be saved to my documents folder which Time Machine would catch.
I use Time Machine to back up my iMac’s system drive. There’s not much that changes, mainly settings, so I created a 500 GB partition on my Drobo. This will limit Time Machine to 500 GB as a maximum but because of the way Drobo works with multiple partitions it only uses the space actually needed by the files. At this point that well under 100 GB.
Every night SuperDuper clones my system drive to a second partition on the Drobo. This is really a holdover from the days when it was cloned to an external drive that I could boot off of in an emergency. It did come in handy during my recent iMac rebuilds, making it easy to find and restore an apps settings one at a time. If disk space was an issue or I didn’t already have it I wouldn’t be doing this backup anymore.
As for my offsite backups, which are done using Jungle Disk, I just back up:
- Documents folder (which has very few files)
- A software archive folder which has copies of any Mac software I paid for or is critical even if it’s a free download.
- My Preferences folder
- My Application Support folder since some applications may put data in these folders
All of this totals about 2.5 GB, most of which is the software.
As for my music and photo libraries which are on my Drobo drive – ChronoSync runs every night and syncs the libraries up to a drive on my Windows Home Server so they’re duplicated. ChronoSync takes care of any adds/deletes/changes automatically.
None of this requires me to do anything beyond the occasional check to make sure this are still working, which is perfect.
Windows 7 Machine Backup
Back when my iMac issues started my Windows 7 PC became my primary PC so that I’d have time to rebuild and test my iMac. I’ve come to really like Windows 7 so my home-build Windows 7 machine has become a daily worker.
The PC is backed up daily using Windows Home Server backup. I have it set to keep the last 10 daily backups, the last 52 weekly backups and the last 12 monthly backups. Yea, there’s a lot of overlap there but since Windows Home Server is efficient about disk space usage it’s not a huge disk hog. I do exclude one directory that I use for temporary storage of large files that I don’t need backed up (such as DVD images). Windows Home Server backup keeps just one copy of the file and the other backup sets just point to the original. So while my backup may be 84 GB it’s not 84 GB for each of my 74 saved backups (10 + 52 + 12).
Restores are straight-forward through the explorer like interface. I’ve never done a full restore using a WHS backup since I’ve always wanted a fresh OS install but I’ve never had a problem restoring from the WHS backup.
I use Jungle Disk to backup important files on this PC along with network files that are important. These get saved on Amazon’s S3 service. I don’t have Jungle Disk set to save archive copies of changed/deleted files, that’s what WHS is for. One backup job is set to backup local files from the PC, another job is set to backup files from my Windows Home Server (via shares). Since I save most data (and anything important) to my WHS the network backup is by far the largest of my Jungle Disk backups and is currently about 20GB. This offsite backup includes my photos but doesn’t include music or videos.
There are backup solutions for Windows Home Server but I prefer to do it via share using my existing backup solution. I can manage backups from my PC and it takes the processing load off the server. The only issue I have with Jungle Disk is that every time I reboot my Windows PC I have to go into Jungle Disk and “test” the network connection, otherwise the backup won’t connect to the shares. It’s frustrating since I don’t actually have to change anything, just click the test button for one share and it fixes the problem for all shares.
I have Jungle Disk send me a daily status email with the backup results so if any backups are missing or show errors I can fix them.
In the past I had used Transmit to synchronize the files from the server to my iMac using SFTP. When my iMac went bad I switched to using WinSCP on my Windows 7 PC. The theory is the same on Windows as it is on my Mac, just different software. I already wrote about how I schedule the WordPress database backups on the server so the database gets backed up to my local PC too.
WinSCP syncs the files down to my Windows 7 PC. From there SyncToy is used to make a local copy to a Windows Home Server share. The Windows Home Server backup also backs them up as a part of the regular PC backup. This regular backup also keeps a years worth of archive copies.
Jungle Disk also includes the web files when it backs up the files from the PC, this gives me my offsite backup (although considering the web server itself is offsite this may be redundant – but I’m paranoid).
The Big Backup
My music and video libraries are just too big (over 6 TB) to backup over the Internet. So to back those up and keep them offsite I copy the files to older hard drives and keep them at the office.
I have a eSata docking bay attached to my Windows PC. I plug the bar SATA drives into this toaster like dock and use robocopy to copy files to the drives. Once a video is added to my iTunes library it doesn’t change when I watch it so these files only show as modified if I change the meta data or re-encode the video. I just created batch files that used robocopy to copy files within a date range that would fit on the drive. I have one batch file per drive so when it comes time to refresh the backup I just rerun the file. An example robocopy command line is (all one line):
robocopy \servervideos f: *.* /S /COPY:DAT /MINAGE:20081129 /MAXAGE:20080908 /PURGE /XA:H
This copies all files from my video share to the drive plugged into the dock. The /S copies subdirectories. The /COPY:DAT says to copy data, attributes and timestamps which are supposed to be the defaults but I had some issues in testing so I specify them. /MINAGE says not to copy files with this date or newer (so in the example, files dated Nov 29, 2008 are not copied). /MAXAGE says not to copy files older than this date (so in this example, files dated Sept 8, 2008 are copied). /PURGE will delete any files within the date range that are no longer on the server. The /XA:H switch says to exclude files with the hidden attribute, such as the Thumbs.db file Windows likes to create since I put DVD art in the directories.
One thing I need to keep in mind is I need to spin up the disks a couple times a year at least. I started off with 500 GB drives being used for the backups since that’s mostly what I had as spares or had the best price per GB at the time I bought them. After removing the drive I start the backup process over again and backup to all the drives again which spins them up and rewrites the data. I put the new drive at the beginning of the backup so a drive isn’t overwritten until it’s files were already backed up.
This process can take a couple of days. While the file copies do take a lot of time most of the delay is because the current drive never seems to finish when I’m around to swap it quickly. So there’s usually a few days when my backups are in the same location as the originals but there’s only so much I can do on a budget.
I keep the latest drive in the dock and backup new videos to it. At this point they aren’t offsite but I figure the risk is minor.
My music library gets backed up in a similar manner although since it easily fits on a 500 GB disk, and I now have two of them available, I just keep swapping the drives between home and the office every month or so. More often if I’ve added a lot of new music.
Due to the size of my video library I only have one copy of the files locally, but that’s on my Windows Home Server with file duplication enabled. I’ve considered turning off file duplication and moving some of the drives to a different computer so I could have a true backup. But I’d need to build a second server (or have multiple locations) to handle 6 TB and a restore would take a long time. I’d rather have file duplication on so as to handle a bad drive, rather than have to do a file restore when a drive goes bad. Especially since I couldn’t be sure what files were on that bad drive. If the entire server craters the delay needed to get the files from the office is small compared to the amount of time to rebuild and restore. I’ve already had one bad drive which had minimal impact thanks to file duplication.
My MSI Wind netbook runs Windows 7 and gets backed up using the Windows Home Server backup. I really don’t save files locally on the Netbook but if I do they get saved to my Drop Box folder so get synced to other PCs and saved in the Drop Box “cloud”.
Windows Home Server is the main component of my Backup Strategy. It handles the daily backups along with weekly and monthly archives. Although earlier versions were buggy the current version has never given me trouble when it’s time to restore a file. Jungle Disk handles my offsite backup needs.
I do have one hole in my backup strategy, namely I have no backup for my email which is through Google. At one time I simply copied information I needed and lost email meant nothing. Eventually I’ll have to plug that.