Depot - Increased queue time and failure rates for Actions jobs and container builds – Incident details

All systems operational

Increased queue time and failure rates for Actions jobs and container builds

Resolved
Major outage
Started about 1 month agoLasted about 2 hours

Affected

Container builds

Operational from 8:17 AM to 9:41 AM, Degraded performance from 9:41 AM to 10:06 AM

us-east-1

Operational from 8:17 AM to 9:41 AM, Degraded performance from 9:41 AM to 10:06 AM

eu-central-1

Operational from 8:17 AM to 9:41 AM, Degraded performance from 9:41 AM to 10:06 AM

us-west-2

Operational from 8:17 AM to 9:41 AM, Degraded performance from 9:41 AM to 10:06 AM

GitHub Actions

Degraded performance from 8:17 AM to 9:11 AM, Partial outage from 9:11 AM to 9:22 AM, Operational from 9:22 AM to 10:06 AM

Depot-managed Actions Runners

Degraded performance from 8:17 AM to 9:11 AM, Partial outage from 9:11 AM to 9:22 AM, Operational from 9:22 AM to 10:06 AM

Updates
  • Resolved
    Resolved

    We have resolved the latency issue that was impacting jobs and container builds from starting. We will continue to monitor the system.

  • Update
    Update

    We are continuing to monitor build service. We are aware of container builds being impacted as well and have rolled out a fix for those as well.

  • Monitoring
    Monitoring

    Build service is restored, we are currently monitoring the result.

  • Update
    Update

    We are currently investigating this incident.

  • Investigating
    Investigating

    We are currently investigating an increase in queue times for some Actions jobs.