Tag Archives: fire prevention

The Omgeo data center fire : Transformer safety …

Today’s cautionary tale comes from the recent fire at Omgeo’s Boston data center. 6000 financial traders down, but it’s not as hard to prevent these things as you might think. Initial news reports are that this failure was a transformer fire and that the transformer kept them from reaching equipment required to switch to their backup data center. Where do I start on this one.

In general, most data center power equipment is pretty much non-flammable. Some things, such as wire insulation, will burn if flame is applied to them, but they won’t support flame by themselves. Remove the ignition source (ie an electrical arc) and the fire goes out. So, how does something like this happen?

The offender is usually an oil cooled utility style transformer. Overloading the secondary of this kind of transformer can cause the oil pressure inside to suddenly spike, leading to the oil tank or the case failing, a large fireball, and burning oil leaking. Overloads of this kind are usually caused by short circuits of the secondary cables or insulation failure inside the transformer.

This was the type of failure that killed part of one of The Planet’s data centers some time ago. These things have even killed utility workers in the process of installing them. You wouldn’t want a lit stick of dynamite in your data center, why would you let a ticking bomb of this sort in?

To make matters worse, transformer fires don’t just stay in the transformer. Burning oil can also run in to underground cable ducts, destroying the secondary cables and the power switchgear.

So how do we prevent this kind of catastrophe? First, don’t ever allow this type of transformer inside the data center. The only safe place for these is outside of the building, either above or below ground.

Second, protect the building from the effects of a nearby explosion. Guard the transformer with a blast shield of brick or reinforced concrete or bury the transformer in a reinforced concrete underground vault.

Third, don’t allow burning transformer oil to spread. Provide a catch basin under the transformer and don’t use cable ducts that are below the transformer. Instead, bring your cables out above the transformer where burning oil can’t penetrate. Also, seal at least the inner end of the cable duct against smoke and fire.

Finally, insist on replacement of any transformer 40 years or more old. Transformers of this vintage are approaching the end of their lifespan and have a much higher chance of insulation breakdown.

Large oil cooled transformers are a fact of life. Recognize the hazard, keep it out of your data center, and you won’t be the next fire news story.

Vern, SwiftWater Telecom
data center facilities engineering

Greening the data center: False economy …

Today, I’ve been reading about the new Intel experimental data center hereand here. The thing that really caught my eye is their method of fire protection.

Simply put, their answer to fire protection is to reduce oxygen level in the data center to 15%. At this level, fire won’t burn, eliminating the requirement for water or inert gas suppression systems and saving that money.

The first issue I have with this idea is how to seal a data center to maintain this kind of atmosphere. Leaks and even the normal ingress and egress of technicians would throw this for a loop, unless you added airlocks at all access points.

The second issue is the health and safety of the data center workers. 15% oxygen is right on the edge of causing immediate physical damage. OSHA lists the effects of lower than 19.5% oxygen as increased breathing rate, accelerated heartbeat, impaired thinking and coordination, dizziness, and nausea. Does this really sound like the way you’d like your data center workers to be functioning? In fact, 15% oxygen is right on the hairy edge of causing permanent physical injury or worse.

In 2008, OSHA soundly rejected an application from FirePASS Corp for a variance for just such a fire suppression system. Suggesting that anyone who experiences ill effects from the environment should just get out isn’t going to fly.

Finally, I have to question the economics of this. With the possible exception of oil filled transformers and very high voltage feeders and switchgear, there’s little to nothing in a modern data center that poses a significant combustion threat. So, effectively, they’re trading systems that have a higher up front cost and little to no operating cost for a system with lower up front cost, ongoing operating cost, and likely hazard to workers for microscopic gain at best. Doesn’t sound very green or very smart to me.

This needs to go wherever all the other boneheaded schemes go to die.

Vern, SwiftWater Telecom

data center, web hosting, Internet engineering