Skip to content

Build exact Graphile schemas from scoped introspection - #1657

Draft
yyyyaaa wants to merge 1 commit into
research/graphile-density-stack-03-cache-realtime-lifecyclefrom
research/graphile-density-stack-04-scoped-graphile-build
Draft

Build exact Graphile schemas from scoped introspection#1657
yyyyaaa wants to merge 1 commit into
research/graphile-density-stack-03-cache-realtime-lifecyclefrom
research/graphile-density-stack-04-scoped-graphile-build

Conversation

@yyyyaaa

@yyyyaaa yyyyaaa commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Context

Part 4 of 6 in the draft tenant-density stack: #1654#1655#1656#1657 (this PR)#1658#1659. This is the memory-reduction layer and the security-preserving replacement for #1332#1334.

The old stack filtered introspection through SQL text substitution, then reused a canonical blueprint by rewriting runtime SQL. This PR changes the algorithm: it parameterizes schema-scoped catalog introspection, computes the catalog dependency closure needed to build an equivalent schema, releases build-only state, and still compiles one Graphile instance against each tenant's exact physical schemas. Runtime SQL is never rewritten.

What changes

  • Preserves stock makeIntrospectionQuery() and adds a bind-parameter scoped query path; schema names travel as values, not interpolated SQL.
  • Adds stock|scoped-required service behavior. Stock remains the default, and scoped-required fails when a requested schema is missing or closure cannot be satisfied; there is no silent stock fallback.
  • Follows required FK, type, function, range, and child-to-parent inheritance dependencies while avoiding reverse partition traversal into unrelated tenant schemas.
  • Hashes the exact Graphile build contract: opaque pool identity, database/API IDs, ordered physical schemas, roles, resolved plugin/settings configuration, compute/storage bindings, surface flags, and introspection mode.
  • Releases introspection/build-only state after validation, bounds Grafast caches, and composes server-owned presets so production caller plugins cannot replace protected database, context, transport, error, or lifecycle policy.
  • Completes the Constructive storage/public-key preset integration audited in Isolate Graphile plugin and storage data paths #1655.

Measured result

Across three fresh-process runs on the 62,298-pg_class fixture, median retained heap fell from 449.42 MiB to 6.55 MiB, final RSS delta from 1,106.94 MiB to 47.77 MiB, and cold build time from 3,495.95 ms to 130.34 ms. That is 68.60× less retained heap, 23.17× less final RSS, and 26.82× faster cold build, with byte-identical GraphQL SDL and zero recorded mismatches in the comparison.

Those are one complete Graphile surface's instance measurements, not complete-customer tenants-per-GiB qualification.

Validation

  • Frozen lockfile installation and Graphile settings/schema/server builds passed.
  • Focused settings/build suites passed 55/55 and focused server scoped/build suites passed 40/40.
  • Small PostgreSQL fixtures proved stock-query byte stability, bind-only schema names, missing-schema failure, dependency closure, safe partition direction, and byte-identical SDL.

Why this remains a draft

Scoped mode depends on temporary package patches and must stay off in production until Graphile maintainers review the API and dependency semantics. The complete 1/2/4 GiB tenant ramp, repeated 15-minute mixed workloads, two-hour churn soak, and production-schema extension matrix remain open; #1659 records those gates rather than presenting the old blueprint numbers as current evidence.

@yyyyaaa
yyyyaaa force-pushed the research/graphile-density-stack-04-scoped-graphile-build branch from 151d669 to 6314199 Compare August 5, 2026 02:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant