Free outside check that tests each Supabase table with the public key, read only #53285
Replies: 9 comments 5 replies
|
The approach is the right first check, and your framing is exactly right: the anon key is never the problem, the table rules behind it are. Having done this review work on client projects, one addition — an outside anon check can only prove one of the states a table can be in, so a clean result deserves an honest caveat next to it. What your check establishes: anonymous runtime reachability. That is the failure that leaks data to the open internet, and it is the correct first gate. What it cannot see, and what a review checks next:
On the design choices: blanking values past the first two characters and stopping at one sample row is the right proportionality, and "never writes, never signs in, only what a visitor's browser could" is the correct boundary for a check that runs on any address. The useful line to draw in the output is between "anon cannot read rows" and "only the owning user can read rows" — the first is a network test you already do, the second is a catalog plus rehearsal test no outside probe can close. |
|
Thank you, this is exactly the kind of review I was hoping for, and every point is right. You drew the line in the right place: the outside check proves that anonymous visitors cannot read rows, not that only the owning user can. A clean result should say that next to it, and I will change the wording so it does. On your four points:
Thanks again for taking the time to write this up. |
|
Thanks for making that change — dropping the "all 35 of your database tables" claim was the right call. A guessed list of common names is not a table inventory, and saying so plainly makes a clean result meaningful instead of flattering. Two things that would make the honest signal stronger, both cheap:
Happy to look at the new wording once it ships. And yes, I'll take you up on the offer: I'm building a Devvit app for Reddit's Game of the Day hackathon — a daily 90-second "find the line that leaks" puzzle built from real-world RLS and authz mistakes, with the same snippet for every player. It isn't playable yet; I'll share the link when it is. |
|
@Zohaib-Mughal — the sentence split is exactly right. "We tried 35 common table names the way a stranger would, and none of them let us in" and "tables with other names were not tested" carry the same fact, but the second one is the only place the boundary actually lives. Claim and scope in separate sentences is the way I'll write mine too. The fifty-app study is better raw material than anything I'd have made up. Those four patterns are the honest short list, and two of them (a policy that only checks the user is signed in, a table named something nobody probes) are ones a free outside check structurally can't see — which is the whole reason the connected check is worth building. On the puzzle: starting with the signed-in false negative. First question is a SELECT policy that reads Question two is the shape version: Taking you up on playtesting — I'll build question one as one page: the policy SQL, the role, three candidate answers (own rows only / all rows / nothing), then the actual read with the proof. When it's up I'll drop the link here. If you want to look at the study data patterns in the meantime, the ones I'd most want to see are how often |
|
@Zohaib-Mughal — question one is ready. Here it is in full so you can use the wording as-is; a one-page version follows when I can push. The setup. One table, one policy, RLS on: create table public.notes (
id uuid primary key default gen_random_uuid(),
user_id uuid not null,
body text not null
);
alter table public.notes enable row level security;
grant select on public.notes to anon, authenticated;
create policy notes_select on public.notes
for select to authenticated using (true);What the toolchain says about this table. The anon check finds no rows (anon has no policy, so it reads nothing). Supabase's Security Advisor skips always-true read policies by design (lint 0024) because they are sometimes deliberate. A policy-exists scanner reports RLS enabled and a SELECT policy present. Everything reads "protected". The puzzle. Signed in as a fresh user, the SELECT returns: A. own rows only The read, run for real. Two users seeded on the same table, then read as each role (PGlite, claims set the way PostgREST sets them per transaction — Answer: B — all rows. The policy is Why this one first. It is the mistake that survives the entire reassuring toolchain — anon check clean, Security Advisor silent by design, policy-exists scanners green — and it sits one line away from your study's second pattern (a policy that only checks the user is signed in), which looks like it checks something and doesn't. In effect they are the same row: every signed-in caller sees every row. The free outside check also structurally cannot see it, for the reason you already named in your connected-check notes: the check runs as a stranger, and this bug only exists for signed-in callers. If the three-answer shape works for you, question two is drafted and uses the same table: the policy changed to |
|
@Zohaib-Mughal — I read both runs and the report JSON, and the honest answer to my own question is that the study cannot produce the ratio, for a reason that is itself the argument. What the data can say. What it cannot say. The ratio I asked for — how often That distinction is the whole connected check in one line: the free outside check can certify the stranger layer and nothing else, and the study's own numbers show the stranger layer is the smaller half of the surface (1 of 6 reachable databases, and only the apps where a database could be reached at all — 6 of 50, which is its own finding about how often the check gets to speak). The number I would still want, if the study ever wants it: run one signed-in probe per reachable database — sign up as a test user for the apps that allow it, then read each table as that user and compare against the same read as anon. The delta (rows that appear only when signed in) is the always-true authenticated policy count. On your six, that would take one test account each. It is also exactly the check a founder cannot do to themselves, which is the gap my $299 service sits in — but that is an argument for the connected check, not against the free one, and the free one is what gets them to ask. One methodological thanks either way: publishing |
|
@Zohaib-Mughal — question two is ready, with your plain sentence added after the reveal. Same table, one phrase changed. The setup. Identical to question one, except the policy: create policy notes_select on public.notes
for select to authenticated using ((select auth.role()) = 'authenticated');Same three answers for a fresh signed-in user: A. own rows only The read, run for real (same harness, same claim setup, Answer: B — all rows. The policy no longer says What it means for the app owner: anyone who makes an account can read every note. Why question two follows one. Question one is the loud version — a policy that says Two notes on where this sits in your study data. First, an Second, since you mentioned the sample may be too small: if you publish the raw per-app policy lines alongside the count, other people can extend it without re-reading the repos, and a small sample stops being a small sample. Happy to help fold counts in if useful. Question three is the grant that never got revoked — a table where RLS, policy and anon reads are all correct, and an |
|
The agreement to state what an anon-only check cannot see is the right call. One addition from fixing these by hand: when the report tells someone to paste a fix into their AI tool, the common failure is that enabling RLS with no policies empties the app. A line in the report like 'add a read policy first, then test with two users' would prevent most of it. We keep a recipe for that case, CC BY 4.0, you are welcome to link it per finding: github.com/lokkky-explore/supabase-fix-recipes. And if a user would rather have someone make the change than paste it, we do that at a fixed price and pay 25% of the first order to whoever sends them. Disclosure: I run Lok & Co., which sells fixed-price fixes for this kind of finding. Everything above is free to use; the recipes are CC BY 4.0 at github.com/lokkky-explore/supabase-fix-recipes. |
|
Good addition, and it belongs in the same sentence as the finding: |
Uh oh!
There was an error while loading. Please reload this page.
Disclosure: I built this, it is my product (GuardCue). The check is free and needs no sign up.
Many apps built with Claude, Lovable, Bolt, Cursor or any other AI tool run on Supabase, and the people who built them often cannot tell whether their tables are protected. The anon key is meant to be public, so the key is never the problem. The table rules behind it are, and an AI tool will often leave them off because the app works without them.
Rather than only telling you to go and look at your RLS, the check tests it:
What it does not do: it never writes, never signs in, and never tries anything a visitor's browser could not. Because of that it runs on any address you give it, so please only check apps you own or have been asked to check. Deeper checks, such as what a signed in user can reach, only run after you prove the site is yours.
Each problem comes with the exact words to paste into the tool that built the app.
Try it on your own app: https://guardcue.com/check/open-database?utm_source=github&utm_medium=community&utm_campaign=supabase-show-and-tell
I also wrote a plain answer to the question we get most, whether the anon key is safe to expose: https://guardcue.com/supabase-anon-key-public
Feedback very welcome, especially where the check is wrong: a table it calls open that is not, or one it misses.
All reactions