Intermittent 502 HTML pages and SUPABASE_EDGE_RUNTIME_SERVICE_DEGRADED 503s on Edge Function calls in ap-southeast-1 #52616
Replies: 2 comments 2 replies
|
This behavior (intermittent Cloudflare 502 HTML error pages and When calling Edge Functions via 1. Increase
|
|
One reliability improvement I would prioritize independently of the regional outage investigation is making scheduled jobs recoverable.
For daily reminder emails, I would use a durable job record with a unique business key, such as recipient + reminder type + scheduled date. A reconciliation worker can detect failed or unresolved attempts, apply bounded retries with backoff, and prevent duplicate effects using idempotency at the receiving side. This does not explain the upstream 502/503 errors. It does prevent one transient gateway failure from permanently losing a scheduled operation. The regional failures still warrant escalation with UTC timestamps and CF-Ray identifiers. |
Uh oh!
There was an error while loading. Please reload this page.
We are seeing intermittent gateway failures on scheduled calls to Edge Functions across five hosted projects in
ap-southeast-1. The failures started around 2026-11-01 19:55 UTC and were still happening at 2026-11-02 03:00 UTC. The status page shows no incident for this period.Setup
Each project calls its Edge Functions from
pg_cronthroughpg_net. One of those calls runs every 5 minutes in every project.What we see
In the responses
pg_netkept (2026-11-01 21:10 UTC to 2026-11-02 03:08 UTC), 44 of 459 calls failed:There were no
pg_nettimeouts in this set.The two failure types look different:
<center><h1>502 Bad Gateway</h1></center>,server: cloudflareand acf-rayending inSIN. The response has nox-sb-edge-regionheader.{"code":"SUPABASE_EDGE_RUNTIME_SERVICE_DEGRADED","message":"Service is temporarily unavailable"}, withx-sb-edge-region: ap-southeast-1and acf-rayending inSIN.Successful calls return
x-sb-edge-region: ap-southeast-1through the same colo.Why this looks platform-side
ACTIVE_HEALTHY, and their databases have not restarted since 2026-10-02.A sample of 400 unauthenticated calls from outside Supabase, taken once around 2026-11-02 02:00 UTC, returned the expected 401 every time. 125 of them were pinned to
ap-southeast-1withx-region. The failures come in bursts, so this does not show that the region recovered.Impact
A daily job has no second attempt, so a dropped call loses that day's run. On 2026-11-02 this stopped one daily reminder job from sending its emails until we reran it by hand.
Questions
ap-southeast-1, and is the gateway 502 related to the degraded 503?SUPABASE_EDGE_RUNTIME_SERVICE_DEGRADEDis not listed on the error codes page. #45252 mentions a platform-level retry on degraded responses. Is that retry live, and does it cover calls that come frompg_net?Related
#52588 reports intermittent Cloudflare 502 HTML pages through Seoul on the REST API.
We can share project refs and individual
cf-rayvalues privately with Supabase staff.All reactions