The ING Bank Data Center in Bucharest, Romania, was severely damaged over the weekend during a fire-fighting test. It appears that the loud sound of the inert fire-fighting gas being released destroyed dozens of hard drives.
The bank's website was offline and the bank relied for 10 hours exclusively on backup copies of the data center, which was located a few kilometers away.
The bank wanted to test the datacenter , but although it worked flawlessly, there were collateral losses. The datacenter's fire suppression system is based on an inert gas that is capable of cutting off oxygen from the fire and extinguishing it while not damaging the computer's electronic components.
The gas is stored in cylinders, and released at high velocity through nozzles that are evenly distributed throughout the data center. According to people familiar with the system, the pressure in the ING Bank data center was higher than expected, and produced a loud sound as the gas was rapidly released through tiny holes in the nozzles (think of the noise a steam locomotive makes).
The bank reported that the sound produced was very loud and estimated to have reached 130dB (the sound of an airplane engine). Sound means vibration, and this caused the damage to the hard drives. The HDDs began to vibrate, the vibration was transmitted to the read/write heads, resulting in them erasing the traces of data.
There is little known about how sound can damage a hard drive. One of the first experiments on this was done by engineer Brendan Gregg in 2008, while working for Sun's Fishworks team. He released a video explaining how shouting in a data center can cause hard drives to malfunction. Watch the video below:
Siemens also published a report a year ago showing that “excessive noise can have a negative impact on hard drive performance.” The researchers said that this negative impact can begin even below 110dB.
Finally, the ING bank in Bucharest took 10 hours to resume operations, due to the size and complexity of the damage.
