It’s been a busy couple of days rebuilding a 6TB file server. Unfortunately just over a week ago due to severely bad weather we had some power outages, even more unfortunate was at that time the file server was sitting on the workbench having some routine hardware maintenance performed on it. Sadly whilst on the workbench it was not being protected by a UPS, so the power outage caused a few things to break.
One in One hundred year flood:
Normally the power is reasonably stable, however the one in one hundred year flood caused a few issues for the power utility providers around the city. It was just bad luck that the file server was on the workbench unprotected by the UPS at the time.
The file server utilised storage spaces in Windows Server 2012 R2 with a multitude (16) of difference HDDs, and was configured as a mirror set so could handle a drive failing, however it was unable to handle 3 HDD’s falling over, so initially panic stations were manned as replacement hardware was sourced before the weekend.
During migration to the new file server I also noticed the battery backup unit on the raid card was looking a little worse for wear too. So in addition to the 3 HDD’s, the battery backup unit is stuffed and it looks like the power supply unit is also a bit dodgy.
YAY for Backups:
Fortunately backups were available and I was able to restore 99% of information (the other 1% was data that wasn’t backed up). Sadly with this amount of data, it can be quite time consuming restoring it, so the best part of the weekend was taken up with getting everything back in the right place.
Good news though, it’s all up and running again, and no information was lost as the 1% that was not backed up was able to be recovered from the HDD’s from the failed file server.