Builds

99.96% uptime
Mar 2024 · 99.90%Apr · 100.0%May · 99.99%
Mar 202499.90% uptime
Apr 2024100.0% uptime
May 202499.99% uptime
API
99.95% uptime
Mar 2024 · 99.93%Apr · 99.93%May · 100.0%
Mar 202499.93% uptime
Apr 202499.93% uptime
May 2024100.0% uptime

Depot-managed GitHub Actions Runners

99.93% uptime
Mar 2024 · 99.93%Apr · 99.93%May · 99.93%
Mar 202499.93% uptime
Apr 202499.93% uptime
May 202499.93% uptime
Website
99.96% uptime
Mar 2024 · 99.95%Apr · 99.94%May · 100.0%
Mar 202499.95% uptime
Apr 202499.94% uptime
May 2024100.0% uptime
Github - Actions
Operational

Notice history

May 2024

Increased GitHub job queue time
  • Resolved
    Resolved

    GitHub have marked the incident as resolved.

  • Monitoring
    Update

    We are observing all jobs able to successfully start. GitHub is still reporting that status updates are delayed for all Actions customers. https://www.githubstatus.com/incidents/nj47vccwm2zj

  • Monitoring
    Monitoring

    While GitHub are still investigating the incident, we are seeing some jobs able to complete, though their status is often not reflected correctly in the GitHub UI. https://www.githubstatus.com/incidents/nj47vccwm2zj

  • Identified
    Update

    GitHub have identified the cause as network connectivity issues on their end, they are continuing to investigate: https://www.githubstatus.com/incidents/nj47vccwm2zj

  • Identified
    Update

    GitHub have just opened an incident of their own for this outage: https://www.githubstatus.com/incidents/nj47vccwm2zj

  • Identified
    Update

    This appears to be a GitHub Actions outage - both GitHub-hosted runners and Depot-hosted runners are currently experiencing issues starting jobs.

  • Identified
    Identified

    We are observing GitHub taking an increased amount of time to release Actions jobs to runners.

GitHub Actions outage
  • Resolved
    Resolved
    This incident has been resolved.
  • Monitoring
    Monitoring

    GitHub has opened an outage of its own that we are now monitoring https://www.githubstatus.com/incidents/2bglpcjvg7xb

  • Identified
    Identified

    We have identified the issue as being on GitHub's end. They are either not releasing jobs in a timely manner to online runners or not sending webhook events when jobs are in progress. We will continue to monitor queue times on our side.

  • Investigating
    Investigating
    We are currently investigating this incident.
Increased latency in picking up GitHub Actions jobs
  • Resolved
    Resolved

    All queue times have returned to normal, marking as resolved.

  • Monitoring
    Monitoring

    We are seeing increased queue times as GitHub processes through the backlog of API requests.

  • Resolved
    Resolved

    New GitHub jobs are operational. GitHub does not appear to be re-queuing jobs that were unable to start during their downtime.

  • Monitoring
    Update

    All new GitHub Actions jobs are starting successfully. Jobs older than 40 minutes ago are still waiting on GitHub to process the backlog of jobs from the outage.

  • Monitoring
    Update

    We are observing new jobs starting, though GitHub has not yet processed all older jobs.

  • Monitoring
    Monitoring

    GitHub has opened an incident of its own that we are monitoring: https://www.githubstatus.com/incidents/f7jl0mdd2jr5

  • Identified
    Update

    We've identified the issue as GitHub webhooks not being delivered. We are monitoring their recovery. Jobs continue to start, but you may see a delay if GitHub continues to lag in delivering webhooks.

  • Identified
    Identified

    We believe this is a potential outage at GitHub as we are observing jobs actually processing and running, but we are not receiving webhooks from GitHub to tell us that.

  • Investigating
    Investigating

    We are currently investigating longer queue times to pick up GHA jobs.

Apr 2024

Monitoring GitHub outage
  • Resolved
    Resolved

    GitHub has cleared their outage, but we are monitoring webhook delivery times from their side.

  • Monitoring
    Monitoring

    We are still monitoring webhook flakiness due to the ongoing GitHub outage: https://www.githubstatus.com/incidents/7txcg0j03kkg. This is surfacing as longer than normal queue times while we wait for GitHub to pass the job to an online runner.

  • Identified
    Identified

    We are actively monitoring the ongoing GitHub outage that is impacting GitHub Actions: https://www.githubstatus.com/incidents/7txcg0j03kkg

GitHub outage
  • Resolved
    Resolved
    This incident has been resolved.
  • Monitoring
    Monitoring

    We are monitoring the rollback GitHub has performed and are observing jobs being processed: https://www.githubstatus.com/

  • Investigating
    Investigating

    GitHub is currently experiencing a widespread outage that is impacting webhook delivery to our runners: https://www.githubstatus.com/

Mar 2024

Mar 2024 to May 2024