I was a guest on The Home Server Show podcast #158 in what was my first ever podcast. It was a lot of fun talking with the HSS guys. We talked about backups along with my WHS, HP Microservers, pfsense, Untangle and other rigs. The show also included news and a discussion about a WHS file restore speed (or lack thereof) problem.
Back when I upgraded my web server I also implemented logrotate to save off the logs each day so they don’t grow too large. At the time it seemed like a good idea to just save the logs for a year since I had the disk space. In retrospect that was a mistake. Why?
Logrotate numbers each file as it’s rotated out. What probably should have been obvious in restospect was that each time logrotate rotates it increments each log file by 1, and that number is part of the name. So each time the logs were rotated every log file was changed. This didn’t seem to affect the server at all, but since each file was changed it got backed up each night. This made the backups take considerably longer and used more bandwidth than necessary.
So I changed the logrotate configuration to only keep a month of logs. I’ll still backup all 31 files each night, but that’s less than the 70+ I’m already doing, and less than the 365 once a year passes. This changed will give me a little over a month of logs and if I really want more history I can archive the new file off every week. The decision for 365 was more due to laziness than a justifiable decision.
I also found logrotate isn’t smart enough to know when it has extra files. When I reduced the files to keep from 365 to 31 logrotate just deleted file 32 and kept all the higher numbered logs so I’ll have to delete them manually. I’ll go back in and delete the files but since they’re not updated anymore they won’t be backed up each night.
Now the logrotate file for my web logs looks like this:
/usr/sbin/apache2ctl graceful > /dev/null
It’s not so much the compressed log archives that I want to keep small, but I want to keep the active log itself small enough so that I can easily open it and read through it when necessary. So while a weekly archive file would still be manageable, opening or copying the active log file.
I retrospect it was pretty obvious the way logrotate would work, I just didn’t think about it too much.Easy enough to fix.
CrashPlan recently ran a discount offer for their one computer, unlimited backup plan so I decided to give it a try. [While the email I received said the promotions will end mid-October, when I check today it’s still active. The URL is: www.crashplan.com/mobilize.]
I’ve been using CrashPlan to back up my parent’s PC and it’s been working well. My main reason for using it on their PC was the ability to backup to a local disk in addition to online storage. (It can also back up to other PCs over the internet but that wasn’t a factor in it’s choice.)
Install CrashPlan on the Windows Home Server
It is the same software no matter what CrashPlan subscription plan you have – start by installing the software as a trial install. Download the Windows 64-bit version for Windows Home Server 2011. RDP (Remote Desktop) into the server and run the installer locally. I accepted the defaults for the entire wizard. Nice and simple. At the end of the installation CrashPlan will start and you’ll either create a new account or link to an existing account.
Create New Account
When the setup wizard completes CrashPlan will start. I want to create a new account for this testing and since this is a one computer subscription there’s no reason to add it to the CrashPlan account I already have for my parents. I enter the information to create the account.
Setup Encryption Key
One of my requirements is that the backups be encrypted using my own encryption key which is not available to the backup provider. So I went into Settings and selected “Replace With Your Own Data key (Advanced)” so that I could enter my own key. The CrashPlan docs indicate this encryption key will also be used for any additional computers I add to the account.
I click the passphrase option and enter in a 63 character passphrase
Acknowledge The Risk
Using my own encryption key brings some warning.
Default Backup Settings
The default settings aren’t very appropriate for a server so I’ll be changing them. I also want to select what to be backed up in relatively small groups to avoid bumping up to my bandwidth cap. I click the “Change” button under Files so I can deselect the Administrator’s home directory and add my first group of files to backup.
Select Files to Back Up
I select the files I want to back up. They total about 12.5 GB.
Start The Backup
Click the “”Start Backup” button to, well, start backing up. It’s initial estimate is that it was take just under 4 days although this was soon cut in half. The backup will continue even if I shut down the GUI and log off the user.
Adjust Bandwidth Limit
By default the outbound bandwidth was limited to 300 kbps. I’m in no particular hurry to get the backup done and I don’t want to impact my other internet activity, including other backups. So while this is well under my upload bandwidth I still lower it 100 kbps to avoid impacting performance while I’m home. At night and when I’m out for work I’ll bump it back to 300 kbps. At 100 kbps CrashPlan estimates 4.5 days to upload the 12.5 GB.
While CrashPlan isn’t officially supported under Windows Home Server 2011, and I’m leery of using it because of that, CrashPlan is generally reviewed positively so I’m going to give it a shot. Initially I’ll back up some files that don’t already get backed up to the cloud. They’re relatively large files (music, video, archived software) that don’t change a lot. With a data cap from my ISP it’s not feasible to store terabytes of data offsite, even if I had the bandwidth I’d hit the cap. While pricey, CrashPlan does offer the ability to get a hard disk in the mail as a restore solution. It’s a bit pricey but something I’d only need to pay for in a pinch.
So I’ll be doing some testing to see how CrashPlan works with Windows Home Server 2011. Anybody already using CrashPlan with WHS 2011?
Lion has brought a frustrating new feature even if it does make sense. Since first using Lion I’ve noticed that moving the mouse doesn’t reactivate the screen if it’s in power saver mode. Since I first encountered this on MacBooks with trackpads I simply tapped the spacebar to wake the machine. Barely a stretch from the trackpad.
But then I put Lion on my desktop Mac Mini and the feature became a frustration. The Mini shares a keyboard and mouse with my Windows 7 machine via Synergy,. The mouse/keyboard are physically on the Windows 7 machine which is the Synergy server. So the problem was the shared mouse wouldn’t activate the Mac Mini screen and the Synergy keyboard was ignored with the display asleep. I do have a keyboard and mouse always attached to the Mini, but they’re tucked away so it was a pain to tap that keyboard.
I found that clicking the mouse (or trackpad) wakes the screen, at least when the mouse is physically attached. But Synergy is still a problem. Even though the mouse appears to go over to the Mini’s screen, tapping a key or clicking the mouse won’t wake the Mini. But a least it’s easier to keep the trackball under the monitor so I can click it rather than digging out the keyboard. So it’s a partial solution.
I certainly see the benefits of this behavior since it would prevent waking the computer with an accidental brush of the mouse. But I wish there was a way to turn this off since it’s the exact opposite of my usage as I’m more likely to want to nudge the mouse to activate the screen. Personally, I rarely nudge the mouse and accidentally wake the PC. In addition to the Synergy annoyance,
Another tip is to use the application Caffeine to disable automatic sleep for a specified time, or indefinitely. This doesn’t solve the problem, but it can avoid power saving while I’m working at my desk. Caffeine is freeware and available from the Mac App Store or from the website.
After upgrading my desktop Mac to Lion 10.7.2 and my iPhone and iPad to iOS 5 it was time to convert my MobileMe account to iCloud and see what all the excitement was about. I don’t use most of the MobileMe features and in fact the features I did use weren’t moved to iCloud so I had to make other arrangements, But once I was ready these were the steps to move to iCloud. I did have problems getting the iCloud move to work, but eventually had success.
Start iCloud from System Preferences
The 10.7.2 install will automatically bring up the iCloud preferences pane. But since I skipped it then I started it through preferences.
Start the Move
Start the move by clicking the “Move to iCloud” button. This will switch you to a browser page.
Start Again – This Time in the Browser
Click the “Get Started” button (you may need to logon with your MobileMe account).
Calendar and Email Will Be Moved
The first screen notifies about the e-mail and Calendar move.
Gallery, iDisk and iWeb Will Stay On MobileMe for Awhile
I don’t use any of these three apps but they’ll go away on Jun 30, 2012.
Not Everything Will Be Available
I don’t sync any of these items so their loss is not a concern to me.
Is Everything Ready?
I’ve already upgraded my desktop Mac and MacBook Air to Lion 10.7.2. My iPhone and iPad are on iOS 5, so I’m ready to upgrade. I check the confimation box then click Nect.
Confirm At Least Once Device Is Up To Date
I don’t sync bookmarks through MobileMe but my contacts are up to date on both Macs that use MobileMe.
Confirm There’s a Backup
I did a backup before beginning so I’m ready to go.
Agree to the Legal term
Oops – Apple Is Busy
This was a frequent message throughout the first day. I start from the beginning each time. Finally I had…
The messages popped up on my computers, iPhone and iPad so I acknowledged it and verified the iCloud settings in preferences. I didn’t have to change anything,
Once everything is moved performance seems fine, although I sync very little with iCloud and I’m hesitant to enable more just yet.
I’ve had iOS 5 on my iPhone and iPad 2 for just about a day now. Along with these I’ve had two Macs on Lion 10.7.2 ready to see what iCloud is all about. I did have a couple issues with the iOS 5 upgrades and I still have an older iPad 1 that hasn’t been able to upgrade yet. I’m not overly anxious with that last iPad upgrade so I give it a try every few hours or so if I’m by my desk. But I’ve been completely unable to move my MobileMe account to iCloud, with the message in that earlier image being the end result when I run through the move wizard. If I actually needed MobileMe/iCloud I’d give up trying to do the move since this would probably be just the beginning of the problems.
Likes, Dislikes and Frustrations With iOS 5
By far my biggest like with iOS 5 is wireless syncing. I had read that the iPhone (or iPad) had to be plugged in to power which would be a hassle and the iPhone screen themselves imply this. Except the word “Automatically” is key. It requires power to sync automatically but can be synced manually anytime. I do have a dock for each on the night stand so do expect the morning sync to be slightly easier, saving a trip to the desk. I’ve found the wireless sync to be fast enough (I have 802.11N wireless). At this point there hasn’t been a backup done over wireless but I’m assuming it’s because the time for the next backup hasn’t rolled around yet as the last one was less than 24 hours ago.
One frustration was the re-organization of my app icons on the devices. Some of this may have been due to the Reminders and Newstand apps being added to the home screen, which was already full for me, I’ve read that others have had apps or folders removed completely but I didn’t experience any of that. (Or the app was used so infrequently that I didn’t notice). It was easy enough to fix, but still annoying.
Related to the moved icons I found that using iTunes to rearrange icons using the wireless connection was painfully slow and I gave up. As was scrolling through the app list to check and uncheck ones to install. I ended up doing this on the device itself, although connecting to USB would have been fine I’m sure.
I like the new notification drop down so I may actually turn notifications on for more apps. It’s taken from Android but it’s one of the features I missed from my Android days so I’ve no complaints.
It’s probably a frustration caused by iTunes and not iOS 5, but several of my smart playlists contain names of podcasts long deleted. They matched at the time but they don’t exist either on disk or in the podcast section. The main problem here is dead links in the playlists and sync errors since the files aren’t found. I had to delete the playlists.
On the other hand, one podcast related frustration was turned into a like with iOS 5. Previously the iPad did not sync playlists where the contents are podcasts. The playlists were there and the podcasts were there, but the playlists were empty.
One dislike is that the playlist display on the iPad shows album covers. In my case at least, the album covers provide little visual clue as the the playlist contents so I’d prefer the list names only, sorted alphabetically. I’ve yet to find a way to change this.
Frustration with iCloud and Apple Servers
Passed on Apple’s track record with “the cloud” I feared the worst. So I consider it a positive that they didn’t seem to break MobileMe.I don’t give Apple any slack for “unexpected demand” or launch bugs. This is the third cloud service I’ve used from Apple and all have had problems. I had no problem downloading the iOS 5 upgrade files themselves from Apple. It was only when the upgrade tried to communicate back to the mothership that I had problems. This is all before a couple million new iPhones hit the streets. It’ll be a long time before I trust Apple with any sort of cloud service. I’ll use it for convenience, but I’ll avoid having to rely on it always being there. Once they get this sorted out they’ll still have to prove they can do upgrades and enhancements without bringing the system down.
It may be a cute marketing ploy, but bringing down the Apple store every time there’s an update could be viewed as a inability to handle updating a live web service. Hopefully iCloud will prove this wrong, but it’s not looking good so far.
I’ve yet to be able to try iCloud and the trouble getting going doesn’t inspire confidence. So far my one liner is “lot’s of promise, little delivered”. No doubt iCloud will grow and get better I just hope it won’t be too painful. Anyone else using iCloud yet? What’s your favorite iOS feature?
It seems like ever computer or computer-like device I have received significant updates in the past couple of days. The 8 security updates for my Windows 7 PCs and VMs were the least time-consuming updates, much to my surprise. Apple released iOS 5 today so the last couple of days have seen related updates from apps to operating systems. Let’s not forget all those iPhone/iPad apps that have been updated in the last few days. With new iPhones arriving on Friday (or before) I figured I better work on upgrading my phone as soon as possible, or wait a week. All this motivated to me finally upgrade my desktop Mac Mini to Lion.
Mac Mini To Lion
I’ll start with the quick and easy upgrade. My MacBook Air is already running Lion so I knew all the apps that mattered would work. I did allow time to get Synergy working again as I figured it would break. It kept right on working and Synergy was sharing my mouse/keyboard between my desktops – Mac Mini and Windows 7 PC. I installed Lion from the DMG file I extracted from my original Lion install rather than from the App Store.
The Lion install downgraded iTunes so I wouldn’t open after Lion was installed. I got an error that the library was an older version. Running software update allowed me to install the latest iTunes. It also presented the latest Lion update 10.7.2 which required a reboot. Once I installed those I was all set.
The iCloud configuration was presented after the 10.7.2 install but I ignored it for now since I wasn’t ready to tackle moving from MobileMe,
iPhone Upgrade to iOS 5
The upgrade to iOS 5 on my iPhone was more problematic. The upgrade sounded scary and complicated – a backup and restore that could take an hour or more according to the dialogs. The “upgrade” seemed more like a complete wipe and re-install as it included a backup and restore.
I upgraded shortly after I saw it was available (about 1PM ET) and received a “internal error” just before the restore part of the installation. The error implied old software or a security setting as the cause, but I suspected it was overloaded Apple servers. My software was already up to date so I removed backup encryption (since the error was just before the restore phase) and removed the passcode from the iPhone. It failed again one more time but then worked. At least it didn’t do the download each time but used the cache copy. Once the upgrade started it took just under 90 minutes to finish, including a very long sync after the restore.
iPad2 Upgrade to iOS 5
I tried the iPad 2 upgrade after the phone finished. I received a different error, but again right before the restore. Right after the progress message that it’s verifying the restore with Apple I received the error that the update server could not be contacted. By this time I’d seen enough blog posts and tweets to know it was in fact an Apple server load issue so I didn’t bother trying again. This is where I upgraded the Mac Mini to Lion, since it’s also my iTunes computer I wouldn’t be compelled to waste time trying the iPad upgrade.
I tried the upgrade several hours later and it went right through but also taking about 90 minutes. Although it seemed to be the luck of the draw as a second iPad upgrade after this failed with the same error.
Preparing For iCloud
All these updates were to enable iCloud in the Apple world. Despite that I’ve yet to enable iCloud on anything, First off, I have little confidence that the iCloud introduction will be problem free. Apple’s history with MobileMe and .Mac does little to inspire confidence, although the optimist would say they learned from their mistakes. I don’t use MobileMe for much, but what I do use it for doesn’t get migrated to iCloud. Namely syncing Transmit favorites, Text Expander snippets and OmniFocus data.
OmniFocus has their own sync service, OmniSync, which is in beta. I had used it months ago and had problems and OmniFocus syncing became my main driver for getting MobileMe. Despite this I decided to move back to it, mainly because I knew I had regular and reliable OmniFocus backups. This was actually more problematic than I thought. First off, I synced all my devices so they all had the same data. Then when I setup my first Mac to sync with OmniSync it failed silently so it was time to troubleshoot.
I went out to the OmniSync website and changed the password. The Mac setup didn’t prompt for a password so I though it might be wrong. I also deleted the registered devices (which hadn’t synced in months).This time it did prompt for the password and then it told me it was going to replace everything on the Mac with the server data – not at all what I wanted. I thought this might be a bad message so did a backup then did the sync. Yup, all my data was replaced with the really old data, So it was time for a restore. Then within the OmniFocus for Mac menu I found “File -> Replace Server Database” and that did the trick, uploading my data. For the other Macs and devices I let the server data update OmniFocus. Interestingly, the iPhone and iPad prompted to ask whether I wanted to use the local or server data, while the Mac software just takes the server data.
As for transmit and Text Expander I decided to go with Dropbox since both supported using it. I had cancelled Dropbox awhile back but signed back up for the free 2GB account. I don’t plan to expand my Dropbox usage, but for now this seems to be the best solution. The setup was simple enough as both apps have settings for Dropbox syncing, no hacking required.
It took about 8 hours (including breaks), but I got all my iDevices upgraded to iOS 5 and got my desktop Mac Mini up to Lion. Now that everything is upgraded I can finally start looking at what all these updates bring me. First off I’ll enable iCloud. While I don’t trust it will be smooth, I don’t have anything critical that will depend upon it. There’s still one older iPad left to upgrade but there’s no hurry and I’ll try again once things settle down.
Anyone actually using the new features? Is it going to be worth the trouble?
I’ve been using Untangle as my router since June and don’t have any complaints, It’s worked well as a router an unified threat manager (UTM). I also took the plunge and subscribed to Kapersky AV for enhanced anti-virus scanning. But pfSense had been my first choice as a router, although I had to abandon it since pfSense didn’t work with my DSL I stuck with Untangle. Now that DSL was gone and there was a new version of pfSense it was time to try again. This time the plan is to run pfSense and Untangle each on their own HP MicroServer.
Untangle Server – This will be the same hardware, minus the dual port NIC, that has been running Untangle. There’s 2GB of RAM, which has proven to be more than enough even when working as a router and UTM. While Untangle is a bit more resource intensive than other solutions I could probably get by with 1 GB. I have the additional 1 GB stick and no other use for it so I might as well use it. The only hard drive is the 160 GB drive that came with the MicroServer. In addition to the onboard NIC I’ve installed a second NIC which is a run of the mill Intel NIC.
pfSense Server – This will be the same hardware as the Untangle server, 2 GB of RAM and the standard 160 GB hard drive. Even though I initially only need 2 NICs I have a dual port NIC I’ll add to this server and I’ll disable the onboard NIC. This will allow me to add another network segment down the road without having to open the server up again. The 2 GB of memory is even more overkill here. The minimum requirements are just 128 MB of RAM, with 512 MB recommended if some of the larger add-on packages are installed. Again, since I already have the second stick, I decide to use it. The network card is the StarTech Dual Port Gigabit NIC.
Since I’ll be moving routers, and therefore DHCP servers since the routers did double duty, I’ll need to document the current scopes and address reservations. After that the plan is simple.
- Shut down the Untangle server and remove the dual port NIC, but leave the software untouched for now.
- Install the dual port NIC in the second MicroServer and install pfSense
- Once pfSense is running as a router reset Untangle to run in bridge mode
The end result will look like this:
pfSense – Initial Problems – Eventual Success
I had some problem right out of the gate.
I decided to try configuring RAID 0, again mainly because I already had a matching drive. But this didn’t work. With RAID 0 configured, the pfSense CD went into a never ending reboot cycle. As soon as it started loading it would reboot. I didn’t spend much time working on this since RAID wasn’t a priority for me on this box, especially RAID via BIOS which I’ve never really trusted.
I rebooted again after turning off RAID in the BUOS. This time I got as far as the menu to select what I wanted to do – continue the live CD boot or install to the hard drive. I let the live CD boot continue but then the startup simply stopped with a error. I booted again but this time during the boot I didn’t accept the default boot option but instead picked the “Boot From USB Device” option since it was a USB CD drive, This did the trick.
Now I was able to boot the live CD and get it running as a router, getting me back on the internet. But my problems weren’t over yet. When I selected the option to install to the hard drive I received an error code 11 during the file copy. Setting the drive controller to IDE mode, and trying a second hard drive resulted in the same error at the same time. Google and pfSense forum searches for the error didn’t provide any help. I skipped through the error and ended up with a working router, but the web interface didn’t work properly. Long story short, while researching the possibility of a bad CD I stumbled upon a pfSense 1.3 CD and accidentally booted from it. So I decided to keep going and sure enough after getting it working as a router it installed to the hard drive just fine.
After having pfSense 1.3 running from the hard drive I was able to upgrade to pfSense 2 through the pfSense console. The upgrade went just fine and I had the pfSense router working just fine from the hard drive. So it was on to Untangle.
Untangle – Easy Enough
Once pfSense was working I was comfortable tackling Untangle since I no longer needed it as a router. I needed to change it to bridge mode so it would no longer function as a router or DHCP server. I could do it by either disabling the unneeded services or reseting to the factory defaults and running the setup wizard again. I chose the factory reset option as the safest route. Since I removed the network card that had the LAN connection I attached a monitor and keyboard to the Untangle server and booted it up. I selected the factory reset option from the console.
After the factory reset I just had to run the setup wizard and select bridge mode.
- The factory reset preserved my Untangle license for Kaspersky so I didn’t have to go through any re-registration process.
- The setup wizard was a little confusing, The first screen required me to assign the NIC ports as external and internal and implies an internet connection. I assigned the external as the port connecting to pfSense and the internal as the one going to the switch.
- The next screen asked me to configure the WAN (internal port). I selected a static IP addresses and entered in 192.168.1.2 (the pfSense LAN port to Untangle is 192.168.1.1). I used the pfSense IP address as the router address. It wasn’t until the next screen where I was asked to select Bridge or Router mode. Once bridge was selected there wasn’t any option to configure the second port (since they both have the same IP address).
- Most current NICs automatically sense the connection type so I could use a regular ethernet cable to link the pfSense server to the Untangle server without needing a crossover cable or a switch.
- Selecting the appropriate pfSense CD to install was the hardest part. There are multiple selections with little guidance, I used fSense-2.0-RELEASE-i386.iso.gz.
- Since the HP MicroServer has a dual core CPU I selected the SMP kernel when asked during the pfSense installation.
- [Added Oct 10] – I needed to re-select the network type on all my Windows 7 computers as well as a Windows 2008 R2 server I was running.
A diagram showing the setup is below:
I haven’t installed any added pfSense packages and the rest of the settings are still using the defaults. I look forward to playing around with pfSense and it’s optional packages bit out of the box it seems to be working fine.
Wrapping Up & Additional Information
The HomeServerShow.com website and forums have a bunch of information, mainly around installing both pfSense and Untangle on the same hardware via virtual machines. Start with the Super Router article or search for “Super Router”, pfSense, or Untangle. Earlier in the year when I started looking at a software router I was able to install both pfSense and Untangle as virtual machines running on Citrix XenServer. But I decided to go with two dedicated computers as a less complicated, slightly more secure solution. Less secure in the sense that the VM host wasn’t on the internet side of the firewall and potentially vulnerable (although admittedly unlikely).
Neither pfSense or Untangle is targeted at home users. This is more noticeable in pfSense in the lack of tutorials for the basics. Right from the beginning it’s noticeable as there are a couple dozen files available to download with no real indication of which to use and when. But with that said, and despite my specific speed bumps, the pfSense install itself is straightforward and result in an out of the box install that exceeds the capabilities of any store bought router and does “just works”. There’s also a active forum.
Untangle provides a GUI interface so it has a friendlier face. The GUI does add to the overhead makes the settings that aren’t front and center harder to find.
Admittedly this is overkill for a home network. But after running up against my bandwidth cap a couple of months I want more control and visibility into the bandwidth. Untangle was a start but pfSense has more features and charts than I’ll ever need so if nothing else, it will be more to play with. While a VM solution makes it easy to swap test machines in and out, the swappable drives of the HP MicroServers give me the same flexibility. The swappable drives are another reason I haven’t looked for smaller form-factor PCs to run pfSense and Untangle.
Anyone else using software routers or custom firmware?
Yesterday I had a phone that had the latest technology. Today it’s now an old piece of crap. Well, not really and not for another couple of weeks. But the iPhone announcement had me thinking of two things.
First, it wash’t long ago when a phone was a phone and it would be a couple years before making a change. Now, even though there are two year contracts to get the best phone price, the iPhone is in a yearly upgrade cycle. While Android is a bit more fractured most of those phones also seem to be “obsolete” in about a year. I picked my iPhone up with the Verizon release so I’ve had it less than a year, making it even worse for me even though Apple was a little late for the yearly upgrade.
Even though I’m eligible for an upgrade in November (the complicated contracts working in my favor this time even though it’s been less than a year) I doubt I’ll upgrade. This brings up the second observation – while my phone may be technically obsolete it’s fine for me. Nothing in the new phone I want that I won’t get with the OS upgrade. Is that a sign that the phone OS’s are mature of that I have a problem by not automatically lusting after the latest tech?