Skip to content

feat: three-actor RLS + adversarial attack integration tests (Alice/Bob/Mallory)#1095

Merged
pyramation merged 10 commits intomainfrom
feat/rls-alice-bob-integration-test
May 10, 2026
Merged

feat: three-actor RLS + adversarial attack integration tests (Alice/Bob/Mallory)#1095
pyramation merged 10 commits intomainfrom
feat/rls-alice-bob-integration-test

Conversation

@pyramation
Copy link
Copy Markdown
Contributor

@pyramation pyramation commented May 10, 2026

Summary

Extends upload.integration.test.ts from a two-actor (Alice/Bob) test into a comprehensive three-actor adversarial security test. Adds Mallory as a third tenant with the strictest RLS policies, and exercises attack vectors for metadata tampering, cross-tenant header manipulation, and bucket/file mutation abuse.

Three-actor model

Actor RLS level Description
Alice None Baseline tenant — no RLS, all CRUD allowed
Bob Moderate RLS on app_files (anonymous: SELECT public-bucket files + INSERT only) and app_buckets (anonymous: SELECT public only). No UPDATE/DELETE for anonymous.
Mallory Strictest Anonymous can only SELECT — no INSERT, UPDATE, or DELETE on files or buckets

Test suites (8 describe blocks)

  1. Presigned URL uploads (Alice) — Public upload, private upload, deduplication (unchanged from original)
  2. Feature flag gatingdatabase_settings / api_settings cascade for Alice, Bob (2 APIs), and Mallory via X-Api-Name header
  3. Three-way tenant isolation — Bob uploads → Bob sees own → Mallory sees own → no cross-tenant leakage in any direction
  4. Bucket enumeration attacks — Alice sees all buckets (no RLS), Bob sees only public (RLS hides private), Mallory sees all (her policy allows all SELECT)
  5. File mutation attacksbucket_id update, is_public flip, delete, direct file injection, plus persistence check that seeded data survives all attacks
  6. Bucket mutation attacks — Create, update (public→private flip), delete — blocked for both Bob and Mallory anonymous roles
  7. Cross-tenant header manipulation — Mismatched X-Database-Id / X-Api-Name combos (5 scenarios) + X-Schemata cross-schema leakage (2 scenarios)
  8. RLS enforcement on Bob's schema — Anonymous sees only public-bucket files; private-bucket file invisible; public file visible

expectRlsDenied helper

The helper handles the three ways PostGraphile surfaces an RLS denial:

  1. Raw PG errors (dev mode) — message contains permission denied, new row violates row-level security, insufficient_privilege, or No values were
  2. Masked internal errors (production/CI mode) — PostGraphile masks PG errors with code: 'INTERNAL_SERVER_ERROR'; the raw message is only logged server-side
  3. Null data — RLS USING clause filters all rows, so the mutation returns null or an object with all-null fields (e.g. { appFile: null })

GraphQL validation errors (GRAPHQL_VALIDATION_FAILED) are explicitly rejected — if a mutation fails due to a wrong field name or missing type, the test fails rather than silently passing.

Seed data changes

  • schema.sql: Refactored into a _test_create_storage_schema() helper function that creates buckets + files tables for any schema name. Adds Bob's storage schema with RLS on both app_buckets and app_files; adds Mallory's schema with strictest RLS (anonymous SELECT-only on both tables).
  • test-data.sql: Seeds Mallory's database, API (mallory-app), storage_module, two buckets (public/private), two pre-seeded files (one per bucket), database_settings, and all required metaschema rows. Also adds a pre-seeded public file in Bob's schema for mutation attack testing.
  • setup.sql: Adds database_settings and api_settings DDL (mirrors production tables from constructive-db PR feat(node-type-registry): add DataLimitCounter, DataFeatureFlag, AuthzAppMembership #1060).

Performance

All tests share a single beforeAll with one getConnections() server instance — no beforeEach, no per-describe teardown. All three schemas are loaded once and reused across all 8 test suites.

Updates since last revision

  • Fixed update mutation variable shapes — All update mutations used patch but PostGraphile v5 expects table-specific names: appFilePatch for UpdateAppFileInput and appBucketPatch for UpdateAppBucketInput. Previously these tests were failing for schema validation, not RLS — they never actually reached the RLS layer.
  • Handled masked errors in expectRlsDenied — In CI, NODE_ENV is not development, so PostGraphile masks all PG errors. The client receives "An unexpected error occurred" with code: INTERNAL_SERVER_ERROR instead of the raw PG message. Added the error code check alongside the raw message patterns.
  • Handled nested null data pattern — When RLS USING clauses filter all rows, PostGraphile returns { updateAppFile: { appFile: null } } rather than a top-level null. The helper now accepts objects where all values are null.
  • Added GRAPHQL_VALIDATION_FAILED rejection — Prevents tests from silently passing when a mutation fails for the wrong reason (e.g. typo in a field name).
  • Removed "Supabase-style" label from the test describe block name.
  • Fixed invalid Mallory UUIDs — All Mallory UUIDs originally used m as a prefix (e.g. m1m1m1m1-...), but m is not a valid hexadecimal digit. Replaced with fa-prefix UUIDs.

Review & Testing Checklist for Human

  • Verify INTERNAL_SERVER_ERROR acceptance isn't too broad — In production mode, expectRlsDenied accepts any masked internal error as an RLS denial. If PostGraphile hits a real bug (unrelated to RLS), the test would falsely pass. The GRAPHQL_VALIDATION_FAILED rejection catches the most common false-pass case, but a non-RLS database error would still be accepted. Consider whether to set NODE_ENV=development in the CI test runner to get unmasked PG errors.
  • Verify cross-tenant X-Schemata tests aren't vacuously true — The two X-Schemata cross-schema tests use if (res.status === 200 && res.body.data) before asserting. If the server returns a non-200 status for a different reason (e.g., schema not found), the assertions are silently skipped and the test passes. Worth verifying these actually exercise the intended 200 path.
  • Spot-check UUID alignment across seed files and test constants — There are many hand-written UUIDs across test-data.sql and the test file. A mismatch between seed INSERT UUIDs and test constants would cause silent false-passes in mutation/isolation tests.
  • Confirm database_settings / api_settings DDL in setup.sql matches production — These table definitions are duplicated from constructive-db PR feat(node-type-registry): add DataLimitCounter, DataFeatureFlag, AuthzAppMembership #1060. If those schemas change, these test seeds must be updated to match.

Notes

  • All 52/52 CI checks pass, including the integration-tests (graphql/server-test) job that runs these tests against real PostgreSQL + MinIO.
  • Alice's API seed uses is_public = false (not true) to match the server's isPublic: false configuration. The original upload tests use X-Schemata which bypasses API lookup entirely, so this doesn't affect them.
  • Bob's RLS deliberately has no UPDATE/DELETE policies for anonymous — the absence of a policy means denial in PostgreSQL.
  • Mallory's RLS is the strictest possible for anonymous: SELECT-only on both tables, no write access at all.

Link to Devin session: https://app.devin.ai/sessions/94a2728a9c414500bead29cbbc829c15
Requested by: @pyramation

- Extend seed setup.sql with database_settings and api_settings tables
- Add Bob's storage schema with RLS policies (anonymous sees public-bucket files only)
- Seed Bob's tenant data: database, APIs, storage_module, buckets, settings
- Add feature flag tests: presigned uploads enabled/disabled via api_settings cascade
- Add tenant isolation tests: Alice and Bob cannot see each other's files
- Add RLS enforcement tests: anonymous role restricted to public-bucket files
@devin-ai-integration
Copy link
Copy Markdown
Contributor

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

@blacksmith-sh

This comment has been minimized.

…sive RLS/attack scenarios

- Add Mallory as adversarial third tenant with strictest RLS (SELECT-only for anonymous)
- Add RLS on Bob's app_buckets table (anonymous can only see public buckets)
- Add bucket enumeration attack tests
- Add Supabase-style file mutation attack tests (bucket_id tampering, is_public flipping, deletion)
- Add bucket mutation attack tests (create, update, delete)
- Add cross-tenant header manipulation attack tests (mismatched database_id/API name)
- Add X-Schemata cross-tenant leakage tests
- Add feature flag gating test for Mallory
- Pre-seed files in both Bob and Mallory schemas for mutation/RLS testing
- Single beforeAll setup — one server, one pool, all three schemas (stays fast)
@devin-ai-integration devin-ai-integration Bot changed the title feat: add Alice/Bob RLS + feature flag integration tests feat: three-actor RLS + adversarial attack integration tests (Alice/Bob/Mallory) May 10, 2026
@blacksmith-sh

This comment has been minimized.

@blacksmith-sh
Copy link
Copy Markdown
Contributor

blacksmith-sh Bot commented May 10, 2026

Found 10 test failures on Blacksmith runners:

Failures

Test View Logs
Integration tests (uploads, tenant isolation, RLS) › Bucket mutation attacks/
Bob: anonymous cannot create a bucket
View Logs
Integration tests (uploads, tenant isolation, RLS) › Bucket mutation attacks/
Bob: anonymous cannot delete a bucket
View Logs
Integration tests (uploads, tenant isolation, RLS) › Bucket mutation attacks/
Bob: anonymous cannot update a bucket (flip public to private)
View Logs
Integration tests (uploads, tenant isolation, RLS) › Bucket mutation attacks/
Mallory: anonymous cannot create a bucket
View Logs
Integration tests (uploads, tenant isolation, RLS) › File mutation attacks (Supabase-st
yle)/Bob: anonymous cannot delete a file
View Logs
Integration tests (uploads, tenant isolation, RLS) › File mutation attacks (Supabase-st
yle)/Bob: anonymous cannot flip is_public flag on a file
View Logs
Integration tests (uploads, tenant isolation, RLS) › File mutation attacks (Supabase-st
yle)/Bob: anonymous cannot update file bucket_id (move between buckets)
View Logs
Integration tests (uploads, tenant isolation, RLS) › File mutation attacks (Supabase-st
yle)/Mallory: anonymous cannot create a file directly (bypassing presigned URL)
View Logs
Integration tests (uploads, tenant isolation, RLS) › File mutation attacks (Supabase-st
yle)/Mallory: anonymous cannot delete a file
View Logs
Integration tests (uploads, tenant isolation, RLS) › File mutation attacks (Supabase-st
yle)/Mallory: anonymous cannot update a file
View Logs

Fix in Cursor

…erns

- Update mutations: patch → appFilePatch/appBucketPatch (PostGraphile v5 input types)
- Add 'No values were' pattern to expectRlsDenied (RLS USING-clause denials on delete/update)
…n-variables

fix: correct mutation variable shapes and expand expectRlsDenied patterns
- Add INTERNAL_SERVER_ERROR code check (PostGraphile masks PG errors in production)
- Add explicit GRAPHQL_VALIDATION_FAILED rejection to catch test bugs
- Handle nested null data pattern (e.g. { appFile: null }) from RLS USING clause
- Remove 'Supabase-style' from test describe name
@pyramation pyramation merged commit 8b46d44 into main May 10, 2026
54 checks passed
@pyramation pyramation deleted the feat/rls-alice-bob-integration-test branch May 10, 2026 10:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant