Cloudberry Continuous Data Protection

I enabled CDP for my “hourly” backup soon after installing the update. It didn’t work exactly as I expected but the differences didn’t affect the actual backups. I also did some additional stress testing to check out performance and this is what I found.

Backup Logo - Laptops connected to backupCloudberry recently added Continuous Data Protection (CDP) to their backup software, including Cloudberry Backup for Windows Home Server 2011. This seems like something I can use to replace my hourly backup so I decided to do some testing. I switched the hourly backup to use a CDP schedule instead. I use the hourly backup to move my most important files offsite (Amazon S3) soon after they’re created. I don’t use RAID so if I lose a disk the data needs to be restored from backup and the hourly backup was my solution. The new CDP option seems like a good fit.

I enabled CDP for my “hourly” backup soon after installing the update. It didn’t work exactly as I expected but the differences didn’t affect the actual backups. I also did some additional stress testing to check out performance and this is what I found.

Using CDP

The CloudBerry blog post said…

Under the hood the changes are captured instantly but the data is uploaded to Cloud storage every 10 minutes.

But I found that the backups occur or are checked for every minute. According to the logs they’re uploaded immediately.  The blog posts also mentions this is configurable although I’ve yet to find where this can be configured in the WHS add-in.

Using CDP takes some getting used to because it changes the way the add-in reports it’s status. It would be less of a problem for someone not used to a regular schedule or is less concerned with checking the status regularly.

The screenshot below shows the status screen for my “hourly” backup several days after CDP was enabled.

Cloudberry CDP status screen

 

Some items of note:

  • The job status is always “running”. The status message uses the term “instant backup” when waiting for files to be backed up.
  • The “files uploaded” only shows the status for the last “instant backup”. If there was nothing to do then the files uploaded is 0. Checking the history shows that file uploads and purges are taking place as required. So while disconcerting, it’s only a cosmetic problem.
  • Since the backup just never ends there’s no email updates for success or errors. I used email to let me know if there was a backup error. The emails only go at the end of a job so even if there are errors (such as an open file) I don’t get an email. In fact, trying to set up an email status report for a CDP backup resulted in a error. This error read more like a bug than a message saying the feature was unavailable.
  • Rather than a 10 minute interval, a 60 second countdown begins when a “instant backup” is completed. Any waiting files are then backed up and uploaded to Amazon S3.
  • If I stop the backup by clicking “Stop Backup” it doesn’t restart. Rebooting the server does restart the CDP backup jobs.
  • Error handling is inconsistent. In my testing the backup would typically ignore errors created by open/locked files. These were valid and when the files where closed they would be backed up. So it was good the job kept running. But there was one instance where a file was moved after being flagged for backup but before it was backed up. This was a valid condition (an iTunes podcast download which downloads to a temp directory and is then moved). The backup job recorded this as an error but then stopped any additional processing. Since CDP backup plans don’t seem to restart on their own this is a problem.

Stress Testing

There wasn’t any noticeable impact changing the my hourly backup to use CDP. The HP MicroServers are relatively low powered and not capable of doing many intense tasks at once. The only add-in I run is the Cloudberry Backup add-in so I was a bit concerned it would impact streaming or other activity. There’s no noticeable load on the server while it’s waiting/looking for updated files. When there’s files to back up the load isn’t any more than the hourly backup and in theory may be less since it spreads the backups out over the hour rather than all at once. Most of the files in this backup plan get updated overnight through automated jobs (website backups, etc…) while the rest of the changes are data file changes. Still I decided to do some load testing.

I copied 121,000 files totaling 60 GB to the same drive I would be streaming a video from. I also copied that set of test files to a second drive. As a control I watched a streaming video while the files were being copied. I RDP’d into the server to do the copies so they were all local drive to drive copies. The streaming worked for awhile until a certain point where it first became slightly annoying until it eventually became unwatchable. At this time there were two file copies going on. One was copying from a directory on the drive (where the files were streaming from) to a second directory on the same drive. Another copy was running from a second drive to the streaming drive.

I have 7 backup plans. A full description can be found in my recent backup review but for purposes of this test I set all backup plans to use CDP. There were three backup plans that matched my test files so they began backing up while the other 4 just watched for files. Each of the test drives had a backup plan dedicated to them and would be doing local backups to eSATA drives so the backup wouldn’t be hindered by network or other limitations. So each drive would be backing up as quickly as the data could be read and written to disk. The third plan included all four drives in the server and backed up to a NAS. So this would be reading from both test drives but only one at a time.

Like the file copies my video stream started off fine and ran for awhile but then it became annoying as it would frequently stop and need to catch up. So no worse than a comparable file copy although still too annoying to be acceptable (while subjective, I doubt anyone would be happy). Not surprising since the backup is not much more than a file copy.

Once the backups were done and the backup plans were just watching I didn’t have a problem streaming and reading files off the server.  Deleting the test files and then letting Cloudberry update their status (I save deleted files for several days so they weren’t actually purged) didn’t affect streaming.

Summary

The good news was that my testing showed that CDP didn’t add any significant overhead above the actual file copies. The bad news is my server isn’t designed to handle a lot of simultaneous activity or file copies. Because of the way I have the shares and rives set up and the way I use the server I may not notice an impact even with CDP set on all plans. Two of the plans go to destinations that aren’t always online so CDP isn’t a good option for them and the other plans rarely have simultaneous changes. Still, CDP is far from a universal solution for me.

I’ve left CDP enabled for what was my hourly backup to Amazon S3 but I’ve returned all the other backup plans to their previous schedule. A lot of times there’s no need for immediate backup and I’d rather wait until all updates are made or a set of files is fully processed. Because of what I send to Amazon S3 I’m less likely to have issues and it’s been fine since being enabled. I do fel I need to monitor it more than I did when I used a hourly backup, if only to make sure it’s still running and that may end up being enough to go back to an hourly schedule if I don’t become more comfortable with CDP’s reliability.

[Update Dec 30, 2011]: I was able to configure email notifications for one CDP plan and it did send a notification when that plan ended with a failure. Unfortunately the CDP plans don’t restart on their own when an error is encountered so I’ve gone back to an hourly schedule for critical backups.