Depot - Notice history

This incident has been resolved and queue times for -latest labels are in the normal ranges again.

We have identified the problem and implemented a fix for those runner labels. We are monitoring queue times to verify they are in the expected range.

We have identified additional latency in starting depot-ubuntu-latest and depot-ubuntu-24.04 GitHub Actions runners. We are investigating whether the same latency exists in container builds.

We have resolved the latency issue that was impacting jobs and container builds from starting. We will continue to monitor the system.

We are continuing to monitor build service. We are aware of container builds being impacted as well and have rolled out a fix for those as well.

Build service is restored, we are currently monitoring the result.

We are currently investigating this incident.

The fix is fully deployed across the container builder regions, and we're observing full recovery. We will continue to monitor.

We have deployed a fix and are monitoring the rollout across the container builder regions.

We've identified a potential fix for amd64 builds and are working on deploying it.

We've isolated the problem to Intel container builds (i.e. linux/amd64) in both regions. We are working to identify the root cause.

This incident has been resolved, CLI downloads are functional again.

We have deployed a workaround to bypass the GitHub outage / rate limit, we are monitoring.

We are seeing queue times resolved and normal processing again.

We are seeing queue times decreasing to normal levels. We are monitoring.