Today’s cautionary tale comes from the recent fire at Omgeo’s Boston data center. 6000 financial traders down, but it’s not as hard to prevent these things as you might think. Initial news reports are that this failure was a transformer fire and that the transformer kept them from reaching equipment required to switch to their backup data center. Where do I start on this one.
In general, most data center power equipment is pretty much non-flammable. Some things, such as wire insulation, will burn if flame is applied to them, but they won’t support flame by themselves. Remove the ignition source (ie an electrical arc) and the fire goes out. So, how does something like this happen?
The offender is usually an oil cooled utility style transformer. Overloading the secondary of this kind of transformer can cause the oil pressure inside to suddenly spike, leading to the oil tank or the case failing, a large fireball, and burning oil leaking. Overloads of this kind are usually caused by short circuits of the secondary cables or insulation failure inside the transformer.
This was the type of failure that killed part of one of The Planet’s data centers some time ago. These things have even killed utility workers in the process of installing them. You wouldn’t want a lit stick of dynamite in your data center, why would you let a ticking bomb of this sort in?
To make matters worse, transformer fires don’t just stay in the transformer. Burning oil can also run in to underground cable ducts, destroying the secondary cables and the power switchgear.
So how do we prevent this kind of catastrophe? First, don’t ever allow this type of transformer inside the data center. The only safe place for these is outside of the building, either above or below ground.
Second, protect the building from the effects of a nearby explosion. Guard the transformer with a blast shield of brick or reinforced concrete or bury the transformer in a reinforced concrete underground vault.
Third, don’t allow burning transformer oil to spread. Provide a catch basin under the transformer and don’t use cable ducts that are below the transformer. Instead, bring your cables out above the transformer where burning oil can’t penetrate. Also, seal at least the inner end of the cable duct against smoke and fire.
Finally, insist on replacement of any transformer 40 years or more old. Transformers of this vintage are approaching the end of their lifespan and have a much higher chance of insulation breakdown.
Large oil cooled transformers are a fact of life. Recognize the hazard, keep it out of your data center, and you won’t be the next fire news story.
Vern, SwiftWater Telecom
data center facilities engineering
The Server Room 