Life is full of unplanned activities, and nothing is more exciting than responding to events that are unplanned. This was the case during the evening of Saturday, October 14th, 2023. That was the night when all of our phones started to ring. All electrical power had unfortunately been lost to our flagship high performance computing datacenter, causing computing for each of the five MGHPCC member institutes to immediately, and somewhat abruptly, come to a halt.
As the machines sat quiet and alone in a now dark and silent room 88.2 miles from the ORCD offices, we knew we needed to build a plan to bring them back to life as quickly and safely as possible. Such unplanned situations are so rare—this was the first of its kind in over a decade—that forming good plans around them is unfortunately inordinately difficult. However, the ORCD team scrambled into action.
We knew what we needed to do.
