Micro Router Project: From Clear to Untangle

I recently wrote about installing ClearOS as my firewall/UTM and it included the ability to run as a typical. Well, its life was short and it was replaced by Untangle just a few days later. ClearOS’s feature checklist  seemed to meet my needs while providing even more features. The initial install also went well and things seemed fine at first.  But then I started I having  problems with the software and there were a few things I didn’t like. I probably should have given it more of a chance, but since I had so little invested, I bailed quickly.

I had problems browsing the web along with inconsistent performance. Some of these seemed to be performance related, as tweaking the settings and turning off certain features helped performance although the problem never really went away. I say it seemed to be performance related because actually watching CPU and memory usage didn’t highlight any problems. Yet cycling the software off and on resolved the immediate problem (usually just cycling the web proxy was enough).

But the biggest con for me was that everything seemed to tie into the web proxy. So if I wanted to scan for viruses it was done through the web proxy. It’s the proxy configuration that gave me the most headaches trying to get a working configuration.  Hardware wise the HP MicroServer seemed fine. Even when I had browsing problems there was plenty of free memory and the CPU usage was low. It’s not like memory or CPU usage was significantly lower after cycling the proxy.

Untangle

In a bit of irony, one reason I passed on Untangle was that I read it was a resource hog. While it certainly needs more resources than something like pfSense it has run fine on the same hardware ClearOS was on. I haven’t had any performance problems running Untangle and haven’t rebooted or cycled the server except to move it to a UPS.

Untangle uses a rack metaphor where all the installed modules are shown in the rack and all traffic flows through the rack. My current rack is shown below (click for full size image):

Image of my Untangle Rak

All the modules shown, except the Kaspersky Anti Virus Blocker, are included in the free version of the software. I’ll probably subscribe to the Kaspersky virus blocker for some added protection but other than that I’ll stick to the free modules. I tried some of the other modules, such as WAN balancing but haven’t found anything I really want and would pay for. In addition to the modules shown the free modules include:  Protocol Control (block unwanted protocols), Spam Blocker, Captive Portal (screen new network users) , and OpenVPN.

Savings Tip: Towards the end of each trial I received a email with a coupon code for an additional 10% off an annual (or multi-year) subscription so even if you know you want the module go for the trial and get the coupon code.

As for the WAN balancing I was looking for –  it doesn’t technically balance traffic. But I do have it hooked to both a DSL and Cable connection and it’s been splitting the traffic between the two without causing any problems. While I was originally looking for something to manage both my broadband connections I’ve found just splitting the traffic works fine. I don’t have the ability to report how much traffic uses each connection without buying an add-in. While something I’d like to have, I ‘d probably opt for adding pfSense before buying an add-in as it’s not worth $10/mth to me.

One Untangle Problem

I have had one problem with Untangle. After switching to Untangle I was not longer able to stream Netflix videos to any computer/device except my iPad. Since it was unlikely that all those computers and devices broke at once I started turning off Untangle modules one at a time until I narrowed it down to the virus blockers. The only configurable item (that seemed remotely related) for it was “Disable HTTP Resume” which was enabled/checked per the Untangle recommendation. Turning this option off in both the standard a Kaspersky virus blockers resolved the issue and Untangle stopped blocking Netflix.

Now, not disabling HTTP resume could let viruses through. HTTP resume allows a browser to start downloading a file from anywhere in the file. For example, from where a download was interrupted. I can see why this feature would be useful for streaming. But if a file download starts mid-way in the file then Untangle won’t be scanning the entire file so it could miss a virus.

I’m not too concerned about this, especially short term, since I have local virus protection on PCs and I don’t frequent bad internet neighborhoods. I may look at ways to route Netflix traffic around Untangle or to a different Untangle server since my Netflix devices are very low risk since they are video only devices . (Although how long before we have a Roku/Blu-ray/TV virus?) It looks like I can simply fix the IP addresses for those Netflix devices and then setup a rule to bypass Untangle for traffic to those IP addresses. But that’s still on my to do list. Since I don’t use Netflix on a computer I won’t have to bypass Untangle for these.

Conclusion

The best thing I can say about Untangle is that it’s been running much longer than I ran ClearOS and I’ve spent much less time fiddling with it. I just sits there and works. Well, except for that pesky Netflix streaming problem which took a little while to track down. Untangle’s Unified Threat Manager features seem better than it’s abilities as a router, at least out of the box at the free software level.

I’d like to have pfSense in front of Untangle to handle the routing but I’m not sure the benefits are worth the effort. I’ll also look at bypassing Untangle for my media devices (Roku/Blu-Ray/TV) but that’s a low priority and it may be awhile before it bubbles to the top of my list. For now I’m happy with Untangle and the status-quo.

Home Server Show Meetup

One of the tech podcasts I listen to, The Home Server Show, is having a meet up this coming weekend. It’s in Atkinson, WI which is over a thousand miles from me so won’t be able to attend what seems destined to be a fun day. If your closer than me, or willing to travel farther than I am, you can fine more info at the Home Server Show.

While I can’t travel out there for the day, there will be a live feed that I’ll be sure to jump on through out the day.

The meet up isn’t specific to Microsoft Windows Home server but should be useful for anyone wanting to use computers in their daily home life.

Downsizing Windows Home Server 2011

In keeping with my recent obsession with the HP MicroServer I just migrated my WHS 2011 server from a case with dimensions that approach 2 feet for both length and height, to the diminutive HP MicroServer that keeps well below a foot for those dimensions. The table below lists the major differences:

Old HAF 32 Build HP MicroServer Build
CPU Intel Core i3-530 Clarkdale 2.93GHz (Dual Core) AMD Athlon II NEO N36L processor 1.3 GHz (Dual Core)
RAM 8 GB 8GB
# Disk Drives 12 5
RAW Disk Space 20.6 TB 12.2 TB
Usable Data Space 16.7 TB 11.2 TB
RAID Protected Space 5.5 TB None
Power Draw @ Idle 155 W 48 W
Location Lots of floor space Cabinet top

The actual parts are:

  1. HP MicroServer w/160GB HDD
  2. Kingston 8GB Memory Part # KVR1333D3E9SK2/8G
  3. Antec Easy SATA Drive Bay
  4. 4 X 3 TB Drive –  (1  Western Digital, 3 Hitachi) in the internal drive bays
  5. Added Cables – Molex to SATA power adapter & SATA cable for the Antec Drive Bay

The Antec drive bay was installed into the ODD bay and it’s what required the power adapter and SATA cable. There’s an internal SATA port intended for the ODD. There’s no BIOS settings related to this port so I assume it’s locked to IDE mode. Benchmarking the included 160 GB drive connected to the original bay and the ODD bay resulted in similar results.

The disk space is well above the 8 TB limit listed in the HP specs but its been working fine for me. The Antec bay lets me add a fifth drive. When I put the bay into the server I planned to use it as a way to swap backup drives in and out. But I changed my mind before the install and used all four of the 3 TB drives I had for the internal bays. I like the Easy Sata bays and they’ve been reliable, but the drive sticks out. I already had the bay from a previous build, but if I was buying a new one I’d look for one that keeps the drive entirely internal.

This is definitely a downsizing. File copies have been a little slower than to the old server. That’s just an observation, not formal tests. But the file copies have been fine for what I do and I’ve been able to play movies from the server.

There’s no RAID so I lose some redundancy and will experience downtime when I lose a drive. But what I lose in redundancy I gain in simplicity and space. Losing a drive has me down for whatever was on that drive until I do a restore. But since I have multiple MicroServers I have spares of everything ready to go. With the old server if I lose a RAID controller/motherboard/psu I have to get a replacement and (depending on the part) do a restore anyway. And frankly, my old server had more moving and electronic parts creating those single points of failure.

The CPU is less powerful but it’s been fine for me. I don’t run a lot of add-ins and I don’t run “desktop” software on it. If I was using a add-in like My Movies and wanted my server to do the actual DVD ripping and encoding I probably wouldn’t be happy. The memory is more than enough and can help make up for the deficiencies in the cpu by avoiding any swapping.

Backups & Future Expansion

I have a Icy Dock External 4-bay Drive Cage. It’s an older type with unique SATA port for each drive. Since the MicroServer only takes low-profile cards my 4-port card (or any other 4 port) wouldn’t fit I bought two HighPoint Rocket 622 dual eSata port cards. They work fine in the server and give me the four needed ports.

I have typically backed up to another server rather than local drives but since I had the gear I hooked up the external bay with four 2 TB drives and have been using those as my backup. I’m not doing any RAID on those backup drives since I want to be able to get at the drives in another computer should the drive cage or cards fail. So this way I can pop a drive into my PC and copy the files off.

This backup is a temporary solution while I rebuild some other servers since the math is not going to work. I have each internal 3 TB drive mapped to a different external 2 TB drive for backup. So as I add more data the math will catch up to me, but I’m safe for now.

Once my data needs really grow I’ll be able to use the external drives for data. The HighPoint cards are supposed to support 3 TB drives although I have no spare 3 TB to test with. The Icy Dock sits nicely on top of the MicroServer so it’s still smaller than the old server. But if I wanted to go really nuts, each of those HighPoint cards can support up to 10 drives with port multipliers but at that point it wouldn’t be downsizing.

Summary

This is a shift from how I used to do things. Rather than redundancy and performance I’ve gone with simplicity and “good enough”. What ended up happening is I reached my data storage capacity and needed to decide whether to add more drives or not. I chose not, and instead revisited what I needed always available. Once I did this my online data needs dropped drastically. It will continue to grow, but the change made sense. Adding to the simplicity, drivers for all the hardware is included in Windows Home Server 2011, no added drivers. Support for the 3 TB drives doesn’t require any software or jumper changes (I’m not using them as boot drives).

While it was only a $15 part and my server kept running, the recent broken latch on my OS Drive Bay made me more paranoid about single points of failure. It’s not so much those single points of failure, but the time and expense needed to recover from them.

I kept all my shares the same to maintain flexibility and minimize the changes I had to make to my desktops and software. I used robocopy to transfer from old to new and when that was done I turned off the old server so I wouldn’t accidentally save files to it. On the Mac I simply disconnected the old shares and reconnected to the new. Since the names were the same all the software and scripts just worked. For Windows I had to remap the drives. I do use UNC connections in scripts on Windows so I had to do a search and replace there.

I’m happy with the first few days of use. It’s not a powerhouse, but it does the job I want it to do. Now I have to do something with that old hardware.

Micro Router Project: New Software Router/Firewall

Tile announcing ClearOSAs I mentioned in my last Trail Log, I started researching a move to a software firewall/router, running on one of my new HP MicroServers. I ended up in a place I didn’t even know existed when the trip started.

I came up with a list of 11 possible software packages I could try. Basically I went to DistroWatch.com and searched for security and firewall distributions, of which there about two dozen.

Pretty much everything would have more features than my current SysWan hardware router. So I was primarily interested in multi-wan support (load balancing & failover) and ease of use. It also had to run on the HP MicroServer.

My hardware for this is:

  • HP MicroServer, currently $280 but watch for sales & rebates. I picked mine up for $255.
  • 8 GB of ECC RAM (Kingstom Part #KVR1333D3E9SK2/8G) . This is overkill for all the applications tested. I had originally considered running this as a XenServer virtual server so sized the memory for that. My final choice benefits from this so I’ll keep it and its $120 cost. Other options would have been fine with the stock 1 GB while other may have benefited from a second 1 GB stick. [Updated 6/5: I swapped the 8GB of RAM for 2GB so now the server runs on 2 GB]
  • In all cases the included 160 GB HDD was sufficient for the software I looked at but I ended up with two 160 GB and two 1 TB HDD for reasons I’ll explain later. The drives came from my parts shelf (or in the MicroServer) and I wouldn’t have added drives if I didn’t already have them.
  • Low-Profile dual port NIC (StarTech ST1000SPEXDP)  along with a Intel NIC from the parts bin were added to the on-board NIC to give me 4 ports and max out the expansion slots. The dual port Gigabit NIC was $105 while the Intel NIC runs about $25 but came from the spare parts bin. As a side note, that StarTech dual port nic was recognized by XenServer without needing additional drivers.

So the hardware ran about $500 [update 6/5: No longer using the 8 GB RAM so about $400]. This is about twice the cost of my existing hardware router but I ended up with gobs more features. And it’s certainly more interesting than a metal box with blinking lights. I could have cut costs by cutting back memory and sticking to 3 network ports but decided I wanted the flexibility.

Software Reviewed

Let’s be clear – when I say I “reviewed” the software it means I installed it, connected it to my DSL connection and looked around the interface. This was less than an hour per app. I figured if it took longer than an hour it failed my ease of use requirement. The software I reviewed (or at least tried to) was:

Astaro – I couldn’t install this on the XenServer in my first attempt and since I wasn’t overly impressed with what I read I didn’t try again when I switched to bare metal. While free, Astaro does have a license with limitations around it and was just purchased by Sophos.

pfSense – This (or its mOnOwall parent) was actually my expected choice going into this. Version 2 is in Release Candidate state but I could never get my AT&T DSL to work with it. So I tested and looked at version 1.2.3 which is the latest stable release. It’s lite weight and fast so it would easily run on less hardware than I was using. The interface wasn’t overly complicated but it wasn’t  intuitive either  and took some research and getting used to, but the more I played around with it the more I liked it.

Untangle – This is a much heavier application that requires more resources. It has a nice web GUI, which comes at the expense of heavy Java use. The software does require registration but the free subscription level has a good set of features. You can buy higher level subscriptions or order feature a-la-carte. As a home user I thought the prices were high. Some of the features I wanted (load-balancing & failover) required an additional subscription cost.

For more information on pfSense and Untangle visit HomeServerShow.com and search the website or forums. There’s posts and discussions about Geek-Accountant’s “Super Router” that uses both pfSense and Untangle and started me off on this router quest.

After the above three I looked at what ultimately became my choice and I liked it at first sight so I blew through the remaining software and nothing caught my eye as being better for me than even pfSense or Untangle.

So, my choice was…

ClearOS

I hadn’t even heard of ClearOS until I searched DistroWatch. In looking at their website I made a mental note that it was like Astaro in that it was a free version of a commercial product and kind of dismissed it (that mental note was wrong).  Then coincidentally this week’s FLOSS Weekly podcast was about ClearOS. Even after the podcast I was still skeptical and had low expectations, but it was enough to get me to install it.

In addition to typical router, firewall and threat management functions, it can also function as a file, print, web & database server. I installed all the modules but have only enabled a few. The screenshot below shows the available modules with the running ones identified.

ClearOS Service Listing

I am using the 15-day free trial for some of the add-ons that require payment. But overall I find the free level of service reasonable for a home user, and the costs for the add-on services realistic for a business that needs them. They were less expensive than comparable add-ons for Untangle.

I’m still trying to figure out the whole relationship between the Clear Foundation and ClearCenter but in my simplistic view ClearOS is a pretty robust Linux distro controlled by a commercial entity and targeted to businesses. That commercial entity survives with their own business model of selling add-on services. Those services are sold through ClearCenter.

The current version of ClearOS is based upon CentOS 5.2. In my review on potential server OS’s I mentioned there was some grumblings in the CentOS community. Well, for whatever reason ClearOS has decided that the next version of ClearOS will be based upon RHEL and not CentOS. Since CentOS is based upon RHEL anyway this isn’t too much of a change for end-users like me but does cut out the middle-man.

There’s also a lot of good information on the ClearOS’s websites and they have a pretty active forum with good info. I did find the ClearFoundation/ClearCenter pillars a bit confusing and annoying at times but there’s good info there.

I’m always hesitant to trust “next version” promises and I did pick ClearOS for what it does today. But I looked at their Roadmap and two features I would want are slated for the next release: Kaspersky Antivirus Integration and Google Apps Integration.

Why I Picked ClearOS

To be honest, a big reason is that I went into it with low expectations and was pleasantly surprised to see it is really a nice distribution. Who doesn’t like a come from behind finish?  Plus there’s just a lot to play around with.

It’s also a full fledged server similar to Microsoft Small Business Server. It’s targeted to business rather than home (although a home edition is on their roadmap for a unspecified future version). As I said in the beginning, any of these solutions will be better than my current hardware.

The interface is a little quirky but it’s generally well laid out and easy to use. I like that there’s a context sensitive help button on each page that opens the suitable online help page in a new browser tab.

While free is good and I like free, I also like to support products I like and use if for no other reason than to help them stick around. Their business model makes it realistic for me to do that. If this lives up to my new & improved expectations I could see spending the $100/yr for the lowest paid subscription to get both those benefits and support the product. Or I may wait for Kaspersky to show up and go a-la-carte. Of course, this all depends on how well it works, how much I use it, and how much I’d miss it if it went away.

The Hardware

I went with 4 NIC ports although I could get by with 3. I have one for DSL, one for Cable and one for the local LAN. The fourth dates back to when I considered running XenServer and would have used it for management. I’ve left it as it gives me the flexibility to add a test segment, wireless guest network or DMZ. For a simple home network you’d only need one WAN and one local port.

I’m using the motherboard RAID for this install. The two 160GB drives are mirrored and the two 1 TB drives are mirrored. So far it’s worked well. I should point out that Untangle couldn’t install the Grub boot loaded when I tried it’s install on the mirror. It was fine when I removed the mirror. It may have been unrelated coincidence.

The services I’m using now could easily live on the 160 GB drive mirror, but the additional space will allow me to try out the web server and other services.

It’s only been a day, but performance on the HP MicroServer has been fine.

Power Consumption

The server pulls 48 to 51 watts during typical usage, putting it at the top of the list of power consumption on my MicroServers. Guess those NICs and drives add up.

The hardware router it replace pulled 6 watts so there is a 8x increase in power usage which is a negative aspect of this project.

Summary

ClearOS does require a subscription to get updates and it includes some additional services. I’ve only been running it about a day so I’ll hold off any review.

The free subscription does include Dynamic DNS so it’s nice to get that free although I’ve yet to use it. Seems similar to the service HP (and others) offer with Windows Home Server in that it adds a subdomain to a domain Clear provides. You can also register your own domain through ClearCenter and use that. There’s no price list but I started the registration process and was told it would be $25. This is expensive for domain registration (although not the most expensive) but more reasonable when you consider it includes Dynamic DNS which can cost more than $25 from other places.

Overall I’m happy with the capabilities and ease of use with ClearOS. I seriously considered Untangle due to it’s ability to integrate Kaspersky Antivirus (but at $108/yr). ClearCenter does offer their own AV subscription for $50/yr but they are unclear who creates/tests the virus defs so there’s no way to assess quality or reliability.

Now it’s time to just let it sit there and do it’s thing. I’ve swung from low expectations to now really liking it. I’m hoping the pendulum doesn’t swing back the other way.

Anybody else using ClearOS or a similar product”?