How to deal with a data center flood.


Next to fire, flooding is one of the most catastrophic things that can happen to a data center. In this post, I’ll give some tips on how to minimize the impact of data center flooding, before, during, and after the event.

BF (Before Flood)

1. Place all electrical entrances, transfer switches, UPS, generators, and batteries above grade. This pretty much goes without saying. The basement looks like a convenient place for infrastructure gear but it’s also dead center bulls eye for flooding.

2. Don’t bury transformers in below grade vaults.

3. No basement drains, sinks, or toilets.

4. Thoroughly seal any below grade utility entrances, such as fiber.

5. Insure that all cooling air intakes and generator air intakes and exhausts are well above any expected flood level. Suck contaminated flood water into a running generator and you’ve got a major problem.

6. Portable plank barriers to temporarily raise door sill levels.

DF (During Flood)

So, the flood turns out to be bad enough that you have to abandon the data center. How do you prepare the facility for the worst?

1. Shut down, lock off, and denergize all generator controls.

2. Lock off all battery strings and remove the intercell connectors. This lowers the chances of the battery discharging through the contaminated flood water.

3. Top all flooded cell batteries off with water. This helps prevent contaminated water from getting in.

4. Shut down the data center and pull the main AC power. Water and electricity do NOT mix.

5. Move the most vulnerable and portable things to a safe location, possibly on an upper floor. This includes spare rectifier modules, other spare equipment, paper records, backup media, and server hard drives.

AF (After Flood)

Now the worst is over, how do you mop up?

1. Pump out all contaminated flood water immediately.

2. Bring in temporary power sources, restore the ventilation systems, and get building and contents dried out as quickly as possible. Do not restore any equipment or attempt to operate any generator that has been under water until it’s thoroughly dried, inspected, and (in the case of a generator) rehabbed.

3. Disconnect all loads from the power equipment. Restore the rectifiers or UPS and the battery string.

4. Restore equipment in small steps, starting with the network connections and working back toward the servers. Inspect all restored equipment for catastrophic failures when power is reapplied (this is why you just don’t restore the whole thing at once).

5. Inspect equipment that has been under water on an ongoing basis for corrosion problems and replace as required.

Hopefully you’ve built your data center in a location that’s unlikely to flood, but, if the worst happens, you’re now prepared to deal with it.

Email or call me or visit the SwiftWater Telecom web site for green data center services today.

Vern

swiftwater telecom rcs cloud computing logo

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s