Resolved
This incident has been resolved.
Monitoring
All failover infrastructure is live and job service is restored. We will continue to address the underlying cause.
Identified
AWS are experiencing broad capacity constraints around our preferred instance type, we are activating a fallback to secondary instance type to address.
Identified
Failover capacity is online and job throughput is increasing.
Identified
Errors seem to be caused in part by lack of instance capacity in our primary runner instance pool, we are activating a failover instance pool to unblock jobs.
Investigating
We are investigating reports of some GitHub runners failing to connect to GitHub