All systems operational

Show current status

Resolved
AWS us-east-1 outage

Started
August 03, 2023 at 8:15 PM
Status
Resolved after about 1 hour

Impact

Major outage
Affected
Builds
  • Resolved

    This incident has been resolved.

  • Monitoring

    All builds are now using primary infrastructure, we are monitoring to ensure that all service is restored.

  • Identified

    AWS is reporting that the us-east-1 outage has cleared, we are routing builds back to primary infrastructure.

  • Identified

    AWS is experiencing an outage in us-east-1 (https://health.aws.amazon.com/health/status). We are attempting a failover to backup infrastructure in another region.

  • Identified

    AWS is still experiencing issues in us-east-1, we are prepping to fail over to backup infra.

  • Identified

    AWS us-east-1 is currently rate-limiting infrastructure updates, we are monitoring.

  • Investigating

    We are investigating lag in acquiring build machines, causing some build requests to time out.