Backup generator fails under live load despite passing tests
Your team is post-mortem-ing a significant availability event in a US East data center. During a regional bad-weather incident, the primary power feed failed and the facility switched to backup generators as designed. One generator failed to sustain stable output once it was under real production load, taking down the servers it was feeding. The same generator had passed a full load test two months prior and had been passing weekly power-on checks without issue. No software change, no capacity change, and no configuration drift was detected in the interval. The outage window ran long enough to affect customer-facing services. Leadership wants to know what change to the validation and rollout process would have caught this before it mattered — what's your recommendation to take to the resilience review board?