[Updated – see below] Today I had an issue with Apache. I moved servers last weekend and was still doing some tweaking. The problem started when changes I was making to the Apache site file for one of my sites wasn’t changing and that site was loading the default site as if it wasn’t found. After considerable time trying to find the problem without stopping Apache. I finally restarted Apache, no change. More time hunting around and finally I stopped Apache. The only problem was Apache didn’t stop. So I rebooted the server and Apache didn’t start.
So now the problem was serious. Luckily my old server was still running so I switched DNS for the site back to it. At least the site would be up once that change propagated.
More searching a looking but no progress. Nothing at all was being logged when I tried to start Apache. I received a starting message but that was all.
To cut to the end, the problem seems to have been caused with a recent update to Apache. When I looked at the apache2ctl file in /usr/sbin directory. The file was 0 bytes in size and there was a apache2ctl.1 file. It looks like a recent update caused the problem. The zero byte file was dated May 13th.
So I deleted the zero byte file and renamed apache2ctl.1 to apache2ctl. All was fine.
I did some more checking and I’m note sure what actually caused the problem [Self-inflicted, see below]. My test server, running the same software didn’t get any update to apache2ctl since the initial Apache install. Not really sure how to prevent it or even detect it. At least in the future I’ll recognize the symptoms and it will be one of the first places I look since it’s easy to see.
[Update 5/28/2012] – I found the problem, it was self-inflicted. A badly formatted logrotate config file for my website logs. I left out a curly brace so it saw everything as a file to rotate rather than the postrotate command.