Replies: 10 comments 5 replies
|
Your metrics investigation already found the exact smoking gun: Here is an architectural breakdown of what is happening under the hood and how to stabilize it: 1. Why Specific Requests Hang While Concurrent Identical Requests Take 50msOn Supabase hosted instances, your container runs not just PostgreSQL, but an entire sidecar suite:
On the Nano (Free) tier, physical RAM is capped at 1 GB. In recent platform releases (including Postgres 17 and updated sidecar services), the baseline footprint of these services combined sits right at 1.1 – 1.3 GB. The Linux kernel responds by invoking to page out idle memory regions to swap on disk.
2. Actionable Workarounds on the Free PlanWithout upgrading, you can reduce memory pressure and avoid swap thrashing through a few adjustments:
3. Would Micro (1 GB Dedicated / Pro) Fix This?Yes. Micro and Compute Add-ons on the Pro tier have dedicated memory headroom, higher IOPS burst allowances, and significantly relaxed swap aggression. However, applying the pooler and keep-alive optimizations above should bring your free Nano instance back into stable low-latency performance. |
|
Your metrics are really useful here. The swap activity, major page faults, and elevated iowait definitely make memory pressure worth investigating. One thing I'd be careful about is treating swap as the confirmed root cause without correlating it with individual slow requests. Since I'd compare the timestamps of the slow requests against the memory, swap-in, and iowait metrics, then include a few representative request IDs in a support ticket. That should help determine whether the delays are actually associated with memory reclaim or another shared service component. I also wouldn't rely on frequent keep-alive pings as a fix yet, since the symptoms aren't consistently tied to idle periods. The evidence is strong enough to investigate further, but I wouldn't consider the root cause confirmed. |
|
Thanks both! One extra data point that I think rules out "Nano + swap" as the whole story. I copied the same schema (no data) into another free Nano project (t3a, eu-west-2), pointed a copy of the app to it and ran the exact same load: 15 simulated clients answering every ~15 s, plus a teacher publishing questions back to back, for 6 minutes. That project had the same memory situation (committed ~1.21 GB at the commit limit, 265–314 MB of swap, 3–15 MB swapped in per minute, iowait up to 22 % in the same minutes), but zero requests over 1 s: 405 loads (max 359 ms), 43 writes (max 205 ms), 128 heartbeats (max 304 ms). My project (t4g, eu-west-1) had 3–6 s hangs under the same test, and up to 20 s at other times. Some details on the other suggestions: The app doesn't connect to Postgres directly: everything goes through the REST API (PostgREST), Auth and Realtime with supabase-js, so there are no direct 5432 connections to move to the transaction pooler. Is there any way for you to check whether my project's instance has a problem (host, EBS volume, t4g vs t3a), or to move it to another host without changing the compute size? |
|
Update: with no clients connected, a probe of 3 anonymous reads every 5 s ran for 60 min (21 Oct, 06:48–07:48 UTC). This project: 59 of 1,917 requests over 1 s, max 20.4 s; 80 % of them started in the first 15 s of a minute, and the longest at every 5-minute mark (07:00:04 20 s, 07:05:02 11.8 s, 07:20:00 11 s, 07:30:02 10 s, 07:35:03 11 s, 07:40:03 11.6 s, 07:45:01 14 s). In the same hour, another free Nano project of mine (tvmtmsqevisinsdkcywr) under ~160 req/s of constant load had no minute-aligned slow requests. Something periodic on this instance seems to stall it. |
|
Thanks for the quick answer! There’s no pg_cron or any scheduled job in my database (only supabase_vault among those extensions). I repeated the probe with random 2–8 s intervals for 30 minutes (21 Oct, 08:39–09:09 UTC), still with no clients connected: 46 of 813 requests over 1 s, 52 % of them starting in the first 15 s of a minute (25 % expected at random), and some much longer today: 148 s, 58 s, 49 s, 25 s. The other Nano project, probed at the same time, had none except a ~1 s blip at the same second on both (my network). I’ve added this to the support ticket (SU-517417). |
|
They’re server-side too. response.origin_time in edge_logs for the same
requests:
- 08:47:45.005 UTC, GET /rest/v1/rooms — *148,468 ms* (request_id
01a15da5-1f7d-72d8-bf71-585abe35f011). An identical GET /rest/v1/rooms 2 ms
later took 50 ms, and an RPC in the same batch 49 ms.
- 09:04:02.757 UTC, GET /rest/v1/rooms — *49,285 ms*
(01a15db4-0ad4-7cd7-866c-24626a7ccccf). The other requests in that batch
took 925 and 1,163 ms.
- 09:05:52.300 UTC, POST /rest/v1/rpc/estat_alumne — *58,559 ms*
(01a15db5-b6bb-7c1b-af22-1e6a78f11e70), while two GETs 4–7 ms earlier took
47–49 ms.
So the time is spent at the origin, on individual requests, while
concurrent identical requests are fast. I’ve added these request IDs to the
ticket. I hope they can figure it out, because is very strange
thank you very much
Missatge de irfan t. atsal ***@***.***> del dia dc., 21
d’oct. 2026 a les 12:24:
… 148 seconds is crazy
And the fact that it still happens with randomized intervals, while your
other project is fine, makes this even weirder.
Just curious, do those 148s and 58s delays also show up in
response.origin_time, or are those client-side timings?
Hopefully Supabase can figure out what's going on with your instance. I'm
really curious what they find.
—
Reply to this email directly, view it on GitHub
<#52039?email_source=notifications&email_token=B6P4CFKK5QMMLEGCRVFPKMD5VCFHTA5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCOJQHE2DMNJUUZZGKYLTN5XKMYLVORUG64VFMV3GK3TUVRTG633UMVZF6Y3MNFRWW#discussioncomment-19094654>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/B6P4CFPQFBXM6F4HOIY2LPT5VCFHTAVCNFSNUABIKJSXA33TNF2G64TZHMZDCNBVHA3TCOJTHNCGS43DOVZXG2LPNY5TCMJQHA4DSMBUUF3AE>
.
Triage notifications, keep track of coding agent tasks and review pull
requests on the go with GitHub Mobile for iOS
<https://github.com/notifications/mobile/ios/B6P4CFJQ25FKSGY4L34FQI35VCFHTA5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCOJQHE2DMNJUUZZGKYLTN5XKMYLVORUG64VFMV3GK3TUVJTG633UMVZF62LPOM>
and Android
<https://github.com/notifications/mobile/android/B6P4CFKHYWLNVSFIVFMFEJ35VCFHTA5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCOJQHE2DMNJUUZZGKYLTN5XKMYLVORUG64VFMV3GK3TUVZTG633UMVZF6YLOMRZG62LE>.
Download it today!
You are receiving this because you authored the thread.Message ID:
***@***.***>
--
*Albert Giralt Borrell*
[image: Logo escola]
*ESCOLA SANTA MARIA DELS APÒSTOLS*
Rambla Prim, 146-152, 08019, Barcelona — *Telèfon:* 93 314 90 11
[image: Web] www.smapostols.org [image: Facebook] Facebook
<https://www.facebook.com/escolasantamariadelsapostols> [image: Instagram]
Instagram <https://www.instagram.com/smapostols>
En compliment del que estableix el Reglament General de Protecció de Dades
i la resta de normativa vigent, el COL·LEGI SANTA MARIA DELS APÒSTOLS
garanteix la confidencialitat i la privacitat de les dades personals que
tracta. Podeu accedir a la nostra Política de Privacitat per ampliar la
informació a www.smapostols.org/politica-de-privacitat/ o bé contactar amb
el nostre Delegat/da de Protecció de Dades, per correu electrònic, a
***@***.*** El contingut d’aquest correu electrònic, inclosos
els arxius que pugui contenir, és confidencial i el seu ús és exclusiu de
la/s persona/es destinatària/es. Si no en sou la persona destinatària, no
esteu autoritzat/ada a utilitzar-lo, copiar-lo o difondre’l, ni total ni
parcialment, per cap mitjà. En conseqüència, procediu a retornar-lo a la
persona remitent i/o a eliminar-lo. En aquests casos, us recordem que la
lectura, publicació, ús, còpia o difusió dels seus continguts, o de
qualsevol document annex, sigui quin sigui el seu propòsit, total o
parcial, està estrictament prohibida. Si no sou la persona destinatària
d’aquest missatge, no en feu cap ús indegut. Gràcies per la vostra
col·laboració.
*Dret a la desconexió digital (art. 88 de la LOPDGDD): *“He enviat aquest
missatge d’acord amb els meus horaris i criteris d’organització del temps.
Si el rep durant el seu període de descans, li confirmo que no espero que
em respongui ara, sinó quan pugui, dins del seu horari laboral.”
|
|
This empirical test comparing your two projects ( As noted by @AndreyA, this matches the active platform issue tracked in Issue #52011 ("API gateway (sb_gateway_version 2) holds browser requests ~5 s while upstream answers in 3–5 ms"), which is currently being actively investigated by Supabase engineer @aantti: Why This Explains Every Symptom You Observed:
|
|
Thank you @ronitgupta (and @AndreyA) — that was exactly it. Grouping my edge_logs by
The upstream never goes above ~0.6 s; all the waiting happens in gateway v2, in front of it. On gateway v1 there isn't a single request over 1 s. A second project of mine in eu-west-2 also goes through both gateways and shows no stalls on gateway 2 (0 of 176,499 requests over 1 s), so it seems specific to gateway v2 in eu-west-1. I've added this to my support ticket and asked for the project to be pinned to gateway v1. Query used, in case it helps others: select log_attributes['response.headers.sb_gateway_version'] as gateway,
count() as requests,
countIf(toFloat64OrZero(log_attributes['response.origin_time']) > 1000) as over_1s,
max(toFloat64OrZero(log_attributes['response.origin_time'])) as max_origin_ms,
max(toFloat64OrZero(log_attributes['response.headers.x_envoy_upstream_service_time'])) as max_upstream_ms
from logs where source = 'edge_logs'
group by gateway order by requests desc
limit 10 |
|
Glad it helped pin down the root cause! Just a quick heads-up on the handle: it's @ronitgupta138 (not @ronitgupta) if anyone needs to ping me. Fingers crossed the infra team rolls out the gateway v2 remediation soon! |
Uh oh!
There was an error while loading. Please reload this page.
Hi! I run a small classroom quiz app on the free plan (project ref chngkpufyghrpimnjktk, Nano, eu-west-1, Postgres 17.6). It used to work fine with classes of 20–25 students. In the last few days, some requests started hanging for 3–20 seconds, even with almost no load. The database is tiny (40 MB) and usage hasn't grown.
What the edge logs show (Logs Explorer, edge_logs, response.origin_time): six GET requests from the same browser arrive at the same millisecond (16:20:03.97 UTC, 20 Oct). Three of them take 49–59 ms, and the other three take 17,573, 17,581 and 20,294 ms. During those 20 seconds, other requests to the same tables (from other clients) are served in ~50 ms. So it isn't the whole instance freezing; specific requests hang.
It also happens with no load at all: with nobody else connected, a page load fired 10 small queries, 9 took 0.1–0.2 s and one (rooms, 1 row) took 14.1 s. A duplicate request sent after 3.5 s was also stuck.
What I've checked:
Network: TCP/TLS connect is 20–40 ms; a control request to Cloudflare never exceeds 0.2 s. The time is spent at the origin (origin_time).
auth/v1/settings (doesn't touch Postgres) also hangs 3–13 s at times.
An empty Nano project of mine (eu-west-2): 0 of 25 slow requests (max 0.22 s). This project: 6 of 25 over 1 s (up to 6.5 s) under the same test.
Observability: CPU 1–4 %, Disk IO 1 %, 19–22/60 connections, no blocked queries. Memory: committed 1.2–1.5 GB (limit 1.2 GB), swap 230–440 MB.
Metrics endpoint (/customer/v1/privileged/metrics), with 15 simulated clients: every minute 4–22 MB swapped in, up to 52 MB swapped out, 3,000–12,000 major page faults, iowait 8–30 %. node_vmstat_oom_kill = 0, CPU steal 0 %.
Bursts of 10 parallel reads after 40 s idle vs. immediately after: the "cold" ones are fine; one "warm" burst had 2 of 10 at 4.5–4.7 s. So it's not new pool connections.
I already reduced the app's load a lot (one RPC per student load instead of 5–6, Realtime broadcast instead of postgres_changes for students, no overlapping refreshes). It helped (average student load went from 3.4 s to 150 ms), but the random 3–20 s hangs remain.
A "Restart project" lowered swap from ~380 to ~180 MB but didn't fix it.
My guess is that idle processes (Postgres backends, PostgREST, Auth) get swapped out on the Nano and the next request has to wait for them to be paged back in. But I don't understand why it started now with the same usage.
Questions:
Has anything changed recently on Nano instances (new services, memory usage of the platform services)?
Is this the expected behaviour of the free Nano with swap, and would Micro (1 GB) be expected to fix it?
Is there anything I can check or tune on my side?
Thanks!
All reactions