Build Servers on Fire

Early this morning I woke up to the smell of burning insulation, turns out it was coming from the Ubiquity (formerly Raspbian) build farm.

After some investigation (and making sure everything/everyone else was not on fire), it looks like one of the disk drives failed catastrophically, causing a short that burned the wires going from it to the PSU, along with the wires it was bundled with.

I haven’t tried spinning up the drive again to verify its deadness, but the main board for the drive looks mostly healthy.

Pictures (sorry about the not great quality, but it was 3AM): alt text alt text alt text alt text

Welp, time to go back to sleep.