Unexpected Power Outage in Komas Data Center - November 1st, 2012 6:30 p.m. to November 2nd, 5:15 p.m.
Posted: November 1, 2012
***UPDATE*** As of 5:15 p.m. November 2nd, all clusters are backup up and scheduling work. We appreciate your patience as we worked through this failure.
***UPDATE*** 10:00 a.m. November 2nd Expect the clusters to remain down through the day today. Failed equipment is still undergoing repairs. Once repairs are complete there will be additional time validating infrastructure and bring networking and clusters back online. As we learn more we will keep you posted. We apologize for the inconvenience.
***UPDATE*** 9:00 p.m. November 1st CHPC systems ascertained that the power will not be restored before mid-morning Friday November 2nd. Another update will be posted with better estimates as they are known. We are hopeful to have power back sometime tomorrow.
Around 6:30 p.m. on November 1st, power was unexpectedly dropped in the CHPC Komas data center, down our main HPC clusters including ember, updraft and sanddunearch.
CHPC staff are onsite and working with Campus and Rocky Mountain Power to find out the cause and to effect repairs. We will keep you updated on the progress as we learn more.