BE-319: Seed the backend integration test graph once per run

[claude]

Requested by Tim Diekmann · Slack thread

🌟 What is the purpose of this PR?

Speed up the backend integration test suite by seeding the system graph once per run instead of once per test file.

Before: every one of the 21 test files ran the full system-graph seed in its beforeAll (system policies, system account, 28 ontology migrations over HTTP, system entities) and then wiped the entire graph again in its teardown via resetGraph() — so the next file had to pay the full seed again. The migration runner already knows how to skip previously-applied migrations (state stored on the HASH Instance entity), but the per-file wipe destroyed that state every time.

After: the graph is seeded once in a vitest globalSetup before any test file runs. The per-file ensureSystemGraphIsInitialized calls remain but now hit the migration-state fast path, becoming cheap idempotent checks. Files no longer wipe the graph; the destructive subgraph snapshot tests run last so they cannot destroy the shared seed mid-run, and the next run's globalSetup re-seeds from scratch.

This is phase A of BE-319 — test files still run serially (fileParallelism: false is untouched); parallelising files is a possible follow-up.

🔗 Related links

• Part of BE-319 (internal)

🔍 What does this change?

• vitest.config.ts: adds globalSetup (src/tests/global-setup.ts) and a custom DestructiveTestsLastSequencer which sorts the src/tests/subgraph/ files (they resetGraph() + restore standalone snapshots) after all other test files.
• src/tests/global-setup.ts (new): runs ensureSystemGraphIsInitialized once against the running graph. It executes in a separate process from the test files, so it builds its graph context inline (no vi mock available) and uses dynamic imports so that the environment module is guaranteed to load before any @apps/hash-api module reads env vars at module scope.
• Per-file teardowns: all resetGraph() calls in src/tests/graph/** are removed (including the two in user.test.ts). Kratos-identity cleanup in the same teardowns is kept. The per-file ensureSystemGraphIsInitialized beforeAll calls are deliberately kept — with the migration state intact they skip all 28 migrations and only perform idempotent existence checks, and they keep each file self-sufficient.
• Isolation fixes for the shared graph:
  • entity.test.ts: the "can read a multi-type entity" query used an unfiltered all: [] filter over the whole graph; it is now scoped to the test org's web.
  • user.test.ts: hardcoded emails (test-user@, bob@, case-test-user@, kratos-profile-update@example.com) are randomized like createTestUser does, and the shortname-casing test now deletes the Kratos identity it creates (previously leaked and would collide on reruns against a persistent graph). charlie@example.com is kept because the incomplete-user test requires an allow-listed email (USER_EMAIL_ALLOW_LIST in .env.test); it is already deleted via the admin API in-file.
• hash-instance.test.ts stays in the main phase: its instance-admins mutations are add-then-remove within the file.
• Deletes the unreferenced src/tests/setup.ts (recreateDbAndRunSchemaMigrations).

Pre-Merge Checklist 🚀

🚢 Has this modified a publishable library?

This PR:

• does not modify any publishable blocks or libraries, or modifications do not need publishing

📜 Does this require a change to the docs?

The changes in this PR:

• are internal and do not require a docs change

🕸️ Does this require a change to the Turbo Graph?

The changes in this PR:

• do not affect the execution graph

⚠️ Known issues

• Leftover web-scoped entities (test users/orgs) now accumulate in the graph over a run instead of being wiped between files. This is intentional — tests were audited for cross-file assumptions (unscoped queries, global counts) and fixed where found.
• After a run, the graph is left in whatever state the last subgraph snapshot test leaves it in; the next run's globalSetup re-seeds from scratch, matching today's behaviour where every file re-seeded anyway.

🐾 Next steps

• Phase B (out of scope here): investigate parallelising test files now that they share a seed.

🛡 What tests cover this?

The backend integration suite itself (yarn test:integration in tests/hash-backend-integration) exercises all of this.

Verification performed:

• tsc --noEmit and eslint for @tests/hash-backend-integration: the full dependency graph (wasm build, graph client codegen) cannot be built in my sandbox, so both were run on this branch and on main with identical unbuilt-deps state and the normalized outputs diffed — the error sets are identical (475 pre-existing resolution errors from unbuilt workspace deps on both; zero findings introduced by this change). The new/changed files themselves report no errors. oxfmt --check passes on all changed files.
• Confirmed in vitest 4.1.8 internals that sequence.sequencer is instantiated and its sort() is applied to the spec list before execution (with fileParallelism: false files then run serially in that order).
• Not verified locally: an actual run of the suite. Starting the stack was attempted but is blocked in my sandbox (cargo git dependency fetch for libp2p and quay.io/Maven downloads are denied by the network proxy), so the graph binary and external services could not be brought up. CI on this PR is the real verification — please treat a green integration job as the gate.

❓ How to test this?

1. Checkout the branch
2. Start external services and the graph as usual (docker compose --profile dev up -d, yarn start:graph, or rely on CI)
3. yarn workspace @tests/hash-backend-integration test:integration
4. Observe that the full migration run happens once up front (globalSetup), the per-file setups log "Skipping migration … already been processed", and the three subgraph/ files run last.

---

Generated by Claude Code

CI timing results

Before/after on genuine (uncached) executions of the Integration (@tests/hash-backend-integration) job's "Run tests" step:

	Duration
Before (per-file reset+reseed)	~21 min (BE-639 run; corroborated by other genuine executions at 12–23 min, e.g. run 28616887518 at 21m31s)
After (this PR, seed once per run)	2m 30s (run 29039353915 — cache miss by construction, since this PR changes the test package)

≈ 8–9× faster. Note for anyone comparing against typical recent runs: most "Run tests" steps on main show 62–82s because turbo replays a cached test:integration result (cache hit, replaying logs) — those are recordings, not executions, and are not a valid baseline.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

3 Skipped Deployments

Project	Deployment	Actions	Updated (UTC)
hash	Ignored	Preview	Jul 9, 2026 6:55pm
hashdotdesign-tokens	Ignored	Preview	Jul 9, 2026 6:55pm
petrinaut	Skipped		Jul 9, 2026 6:55pm

[cursor]

PR Summary

Medium Risk
Shared graph state across files increases cross-test coupling; mitigations (sequencer, scoped queries, unique fixtures) are in place but leftover entities can still affect poorly isolated tests.

Overview
Backend integration tests now seed the system graph once via vitest globalSetup (global-setup.ts) instead of paying a full seed plus resetGraph() wipe in every test file.

Per-file ensureSystemGraphIsInitialized calls stay but reuse migration state on the HASH Instance, so skipped migrations log at debug instead of info. resetGraph() is removed from all graph/** teardowns (Kratos cleanup remains). DestructiveTestsLastSequencer runs subgraph/ tests last so snapshot restores cannot break the shared seed mid-run.

Isolation fixes for a persistent graph: entity.test.ts scopes the multi-type entity query to the test org web; user.test.ts randomizes emails, cleans up Kratos on failed case-collision signup, and drops redundant afterAll resets. Unused setup.ts (recreateDbAndRunSchemaMigrations) is deleted.

^{Reviewed by Cursor Bugbot for commit 4836ae5. Bugbot is set up for automated code reviews on this repo. Configure here.}

[Copilot]

Pull request overview

This PR speeds up the @tests/hash-backend-integration suite by seeding the system graph once per run via a vitest globalSetup, instead of once per test file. The per-file ensureSystemGraphIsInitialized calls are kept but now hit the migration-state fast path, and the destructive subgraph/ tests are sorted to run last so they can't destroy the shared seed mid-run. Test files no longer wipe the graph in teardown, so several tests were hardened against a shared/persistent graph.

Changes:

• Add globalSetup (global-setup.ts) that seeds the system graph inline in a separate process, plus a DestructiveTestsLastSequencer that orders subgraph/ tests last.
• Remove all per-file resetGraph() teardowns (keeping Kratos-identity cleanup) and delete the unused setup.ts.
• Harden isolation: entity.test.ts scopes its multi-type read query to the test org's web, and user.test.ts randomizes previously-hardcoded emails and deletes the Kratos identity leaked by the shortname-casing test.

Reviewed changes

Copilot reviewed 21 out of 21 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
vitest.config.ts	Adds globalSetup and a sequencer sorting subgraph/ files last; updates the parallelism comment.
src/tests/global-setup.ts	New once-per-run seed; builds graph context inline with dynamic imports to guarantee env loads first.
src/tests/setup.ts	Deleted; the unreferenced recreateDbAndRunSchemaMigrations helper is removed.
graph/knowledge/primitive/entity.test.ts	Scopes the "read a multi-type entity" query to the test org's web instead of all: [].
graph/knowledge/system-types/user.test.ts	Randomizes hardcoded emails, adds Kratos cleanup for the casing test, fixes the allow-list comment, drops resetGraph teardowns.
graph/knowledge/system-types/hash-instance.test.ts	Removes the resetGraph teardown; stays in the main phase (add-then-remove admin mutations).
Other graph/** test files (org, org-membership, org-invitation-email-casing, page, file, block, comment, comment-notification, mention-notification, ai, link-entity, policy, data/entity/property-type)	Remove resetGraph import and teardown call; retain Kratos-identity cleanup where present.

[codecov]

Codecov Report

❌ Patch coverage is 0% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 59.74%. Comparing base (232f15c) to head (4836ae5).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
...tem-graph-is-initialized/migrate-ontology-types.ts	0.00%	1 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #9001   +/-   ##
=======================================
  Coverage   59.74%   59.74%           
=======================================
  Files        1370     1370           
  Lines      135170   135170           
  Branches     6067     6067           
=======================================
  Hits        80760    80760           
  Misses      53477    53477           
  Partials      933      933           

Flag	Coverage Δ
apps.hash-api	6.39% <0.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[thehabbos007]

LGTM. I suppose if something starts being flaky on the integration tests, we should rever/reconsider.

[thehabbos007]

Only thing i am a little unsure about. It looks fine, and probably needs a better way to register destrutive ops than checking the path prefix, and we may have other destructive tests down the line that need to be sequenced. But i think our current integration tests are eventually going to look different as move move more into the graph

[claude]

Agreed on both counts — the path prefix is the minimal mechanism that works today, and if more destructive suites appear an explicit registration (e.g. a destructive tag/annotation the sequencer reads) would be the cleaner shape. Given the suite's likely restructuring as more moves into the graph, keeping it minimal for now seemed right.