Failures in data center engineering.

Today I’ve been viewing this video of the Vodafone data center flood and just shaking my head. How could things go so wrong from the start?

With all the focus on things like reliability, energy efficiency, redundancy, physical security, etc, there seems to be a blind spot a lot of the time for what is not disaster recovery (if you have to recover from it, it’s already too late to think about it) but what I call disaster prevention. It’s the pound of cure vs ounce of prevention formula again.

So what are the ounces of prevention? First and most obvious is not to place a data center in an area where there’s chance of flooding. Given the massive damage and cost of recovery, I have no idea what kind of tradeoff would ever make this acceptable.

Second is not to place critical infrastructure in vulnerable locations. This means no generators or electrical switchgear in basements (I’m reminded of a number of major buildings the past few years that lost backup generators to basement flooding). Also, consider special requirements for green infrastructure such as protecting solar panel arrays and other vulnerable exterior equipment from high wind conditions, for example.

Third is securing the facility itself from the effects. It’s far better to prevent water intrusion in the first place than it is to deal with it after the fact. Once again, green techniques such as free air cooling require extra consideration. Are the vents for the cooling arranged to exclude water and high wind?

Finally, don’t ignore the need to prepare for the unlikely. I’m not suggesting building a roof for high snow loads in the middle of the desert (I said unlikely, not impossible) but flood damage and high wind damage prevention won’t be wrong in most places. With weather patterns changing around the world, more and more extreme weather is showing up in places that it’s never been seen before.

Expect the unexpected, it will keep you from being tomorrow’s disaster news headline, not to mention cautionary YouTube video.

Vern, SwiftWater Telecom
Data center, web hosting, Internet engineering


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s