[2026-07-16] Incident Thread

[github-actions[bot]]

❗ An incident has been declared:

Copilot Coding Agent Degraded Job Completion

Subscribe to this Discussion for updates on this incident. Please upvote or emoji react instead of commenting +1 on the Discussion to avoid overwhelming the thread. Any account guidance specific to this incident will be shared in thread and on the Incident Status Page.

[pasha-omni]

running gh api repos/.../commits/main returns an HTML 503 error page

[PramaYudhistira]

seeing this issue too. smh this is why origin is gonna win bruh

[github-actions[bot]]

Update

We are aware of degraded REST API availability and are investigating

[github-actions[bot]]

Update

We are continuing to investigate an issue causing approximately 35% of REST API requests to fail.  Based on our current understanding, requests are not consistently reaching the application layer, resulting in failed requests returning HTML responses instead of the expected API response format. We are actively investigating the issue and will provide another update as soon as more information is available.

[JonnyPower]

something more deterministic than 35% - I've retried 20+ times ;)

[tby208]

same here

[danr-cova]

Would this issue by a reason why Co-Pilot PR Reviews are currently seeming to all fail with the following added as a comment to the PR:

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

This error may be related to your runner configuration. You can now configure runners for Copilot code review separately from Copilot cloud agent by creating a copilot-code-review.yml file with your setup steps. Read the docs for details.

We haven't needed to do this before and have had no indication it was about to stop working.

[JonnyPower]

it's much larger than copilot - I can't even clone via app token

[JonnyPower]

though switching to a user's SSH key auth for clone in jenkins allowed me to workaround that for now

[danr-cova]

Yep seems to be. Saying "Degraded REST API Availability" may be the underslying issue, but it's not mentioning the things that use the Rest API which are also seemingly borked. :/

[github-actions[bot]]

Update

As of 23:46 UTC, the REST API service is reachable and responding to requests normally.

[github-actions[bot]]

Incident Resolved

This incident has been resolved.

[github-actions[bot]]

Incident Summary

From 22:21 UTC - 23:50 UTC on July 16, 2026, the REST API experienced significant degradation. During this period, about 39% of REST API requests failed with HTTP 500 level responses, with the errors peaking at 44.3%.

We identified the issue as an infrastructure change that wrongly marked the majority of API backends in a single region as unhealthy. As a result, requests routed to those backends failed before reaching the application layer.

To prevent this from happening again, we're improving our systems to catch this kind of invalid configuration before it reaches production. We'll also audit the related systems to make them more resilient to future changes, and we're increasing our monitoring sensitivity so we're alerted to problems like this sooner.