I was up very late this morning. A storm rolled in around 3am and I decided to go to bed. Just as I did, the power went out. Since I host our web site here, I have to be somewhat concerned. So I got up to see what I needed to do. First thing I noticed is that the server room was quickly getting warm without hvac. Data center servers dont like to be in warm environments. Luckily the storm brought with it cool air so I opened the windows and the door to let the breeze keep things cool.
I also manually shut down other servers that were not needed in order to keep the web server going as long as possible and to reduce heat output. I am now looking at a way to do this automatically in any power event. Problem is, once you shut off a server when on battery, the UPS generally needs to finally shut down and then power back up when the power is restored. The power reset generally boots up all of the servers your script turned off. But, if the UPS never power cycles but instead just comes back to line, there is never a power reset event to turn on the other boxes. So one needs to figure a way to script this scenario using wake on lan or some such device.
I have found some software that should be able to handle this but it has to exist on the last remaining server running (my web server) and it needs access to the UPS management cards and server nics (internal network)... So I have a few problems to figure out. I may have to keep an internal server running for this process.
At any rate, I have a battery bank of 48VDC at 170A. It runs things fine for awhile. This outage lasted three hours and the web server remained up and accessible with a small load including itself, the firewall, cisco switch, fiber switch and some other various network things.
08/15/2008 03:19:02 UPS: Switched to battery backup power; utility power failure.
08/15/2008 06:16:54 UPS: Returned from battery backup power; utility power restored.
I went to bed at 6:30am after closing the windows and restoring things.
We used to never ever have power issues here during even the worst storms. The day they were boring underground down the alley to lay the new fiber, the lights were flickering all day. This told me then that they were scraping along the 8000V underground power lines. So now, when we get heavy rain that soaks the ground, power shuts off as the transformers get overloaded due to the bare high voltage copper lines in the ground.
Nobody cares. It's scary and a shame. Anybody wanna donate a natural gas generator? Anything from 8,000 on up to 20,000 watt will do.
SEE ALSO:
http://www.sugarglider.com/glidergossip/topic.asp?TOPIC_ID=14274&whichpage=1#93076