Depot - Notice history

All systems operational

100% - uptime

us-east-1 - Operational

100% - uptime
Sep 2024 · 100%Oct · 99.96%Nov · 99.98%

Previous page

Next page

Sep 2024
Oct 2024
Nov 2024

eu-central-1 - Operational

100% - uptime
Sep 2024 · 100%Oct · 99.97%Nov · 99.98%

Previous page

Next page

Sep 2024
Oct 2024
Nov 2024
100% - uptime

Depot-managed Actions Runners - Operational

100% - uptime
Sep 2024 · 100%Oct · 99.98%Nov · 99.91%

Previous page

Next page

Sep 2024
Oct 2024
Nov 2024

Github.com - Actions - Operational

API - Operational

100% - uptime
Sep 2024 · 100%Oct · 100.0%Nov · 100.0%

Previous page

Next page

Sep 2024
Oct 2024
Nov 2024

Website - Operational

100% - uptime
Sep 2024 · 100%Oct · 100.0%Nov · 100.0%

Previous page

Next page

Sep 2024
Oct 2024
Nov 2024

Notice history

Nov 2024

Increased queue times for GitHub Actions
  • Resolved
    Resolved

    We now have a workaround in place to force GitHub webhooks to send to Depot.

  • Monitoring
    Monitoring

    We are seeing recovery and are going to continue to watch the webhook delivery times.

  • Identified
    Identified

    We are seeing very slow delivery of GitHub Action webhooks but are beginning to see GitHub recover.

  • Investigating
    Investigating
    We are currently investigating this incident.
Increased queue times for GitHub Actions
  • Resolved
    Resolved

    We are no longer seeing GitHub networking issues.

  • Monitoring
    Monitoring

    We are seeing normal queue times now. We'll continue to monitor.

  • Identified
    Identified

    We have identified that this appears to be issues with GitHub's network ingress returning "HTTP Status: BadGateway."

    We'll continue to investigate.

  • Investigating
    Investigating

    We are currently investigating this incident.

Oct 2024

Increased queue times for GitHub Actions
  • Resolved
    Resolved

    GitHub are reporting that the incident is resolved.

  • Update
    Update

    GitHub is still working to resolve the incident: https://www.githubstatus.com/incidents/9yk1fbk0qjjc

  • Update
    Update

    GitHub has now opened an incident for Actions: https://www.githubstatus.com/incidents/9yk1fbk0qjjc

  • Monitoring
    Monitoring

    This is an outage inside of GitHub Actions where jobs are not being delivered to online runners. We are continuing to monitor the situation and waiting for an update from GitHub.

  • Identified
    Identified

    We have identified that this appears to be related to an issue with GitHub delivering jobs to online runners. We are continuing to see if we can work around their issue.

  • Investigating
    Investigating

    We are currently investigating increased queue times for GitHub Actions.

Container builds unable to start
  • Resolved
    Resolved

    The deploy has completed, build service is restored.

  • Identified
    Identified

    We have identified an issue preventing container builds from starting, a fix is deploying now.

Increased GitHub job queue time
  • Resolved
    Resolved
    This incident has been resolved.
  • Monitoring
    Monitoring

    We are starting to see some recovery, still monitoring

  • Identified
    Identified

    GitHub has discovered the increased queue times and are investigating.

  • Investigating
    Investigating

    We are currently observing increased delays in GitHub distributing jobs to Actions runners.

    This may be related to their current outage:

    https://www.githubstatus.com/incidents/myvz2tsj2dh8

Some API requests failing
  • Resolved
    Resolved

    This appears to have been a brief network disruption for our database provider that has since recovered, all jobs and builds are processing normally again.

  • Investigating
    Investigating

    We are currently investigating reports of API request failing, preventing container builds from starting and increasing queue times for GitHub Actions jobs.

Increased GitHub job queue time
  • Resolved
    Resolved
    This incident has been resolved.
  • Identified
    Identified

    We are starting to see some recovery, still monitoring

  • Investigating
    Investigating

    We are currently observing increased delays in GitHub distributing jobs to Actions runners.

Sep 2024

Container builds failing to pull images from DockerHub
  • Resolved
    Resolved
    This incident has been resolved.
  • Monitoring
    Monitoring

    Docker has reported that a fix has been applied and that error rates are returning to normal. We'll continue monitoring the issue.

  • Investigating
    Investigating

    Docker has reported an outage with Docker Hub Registry with degraded performance pulling images. Some container builds may fail to pull images from Docker Hub. Track Docker's ongoing incident here: https://www.dockerstatus.com/

DockerHub is down undergoing maintenance
  • Resolved
    Resolved

    We are now seeing consistent success from DockerHub.

  • Monitoring
    Monitoring

    DockerHub seems stable at the moment. We'll continue to monitor.

  • Identified
    Identified

    We are seeing many builds fail when trying to pull container images from DockerHub.

    DockerHub is undergoing planned maintenance for the next several hours:

    https://www.dockerstatus.com/pages/maintenance/533c6539221ae15e3f000031/66f3458fa451070c6a8b47a6

Container builds are not starting
  • Resolved
    Resolved
    This incident has been resolved.
  • Monitoring
    Monitoring

    The fix has deployed and builds should be processing, we are monitoring.

  • Identified
    Identified

    We've identified the issue and are rolling out a fix.

  • Investigating
    Investigating

    We are currently investigating an issue that is causing container image builders to not launch in a timely manner.

Subset of GitHub Actions jobs queuing longer than usual
  • Resolved
    Resolved
    This incident has been resolved.
  • Monitoring
    Monitoring

    A fix has been deployed, and we are monitoring queue time recovery for the impacted customers.

  • Identified
    Identified

    We've identified the root cause of the issue and are implementing a fix.

  • Investigating
    Investigating

    We are currently investigating an issue that is impacting a small subset of customers where jobs appear to be queuing for longer than usual.

Increased queue times for GitHub Actions jobs
  • Resolved
    Resolved
    This incident has been resolved.
  • Monitoring
    Monitoring

    Queue times have returned to normal for the vast majority of jobs. We'll continue to monitor, as Github is still reporting an outage.

  • Identified
    Identified

    GitHub has opened an incident that we are monitoring: https://www.githubstatus.com/incidents/69sb0f8lydp4

  • Investigating
    Investigating

    We are currently investigating reports of longer queue times for GitHub Actions jobs

Sep 2024 to Nov 2024

Next