feat(ci): container-based integration tests + migration gate (closes #3, #5)#234
Merged
charlie83Gs merged 7 commits intomainfrom Apr 20, 2026
Merged
feat(ci): container-based integration tests + migration gate (closes #3, #5)#234charlie83Gs merged 7 commits intomainfrom
charlie83Gs merged 7 commits intomainfrom
Conversation
Closes #3 and #5. Split unit tests (infra-free, key-free) from integration tests (real postgres/redis/qdrant/hatchet + API secrets). Removes integration tests from lefthook pre-push for faster local feedback; CI runs them per-package on PRs only when relevant code changes. CI changes (.github/workflows/test.yml) - backend-test job: drop service containers + DB env vars. Unit tests now run hermetically (1240 passing in under 60s). - backend-integration-test job (new): matrix per-package, boots postgres, postgres-write, redis, qdrant service containers; injects SERPER_KEY, OPENROUTER_API_KEY, OPENAI_API_KEY secrets. No BRAVE_KEY - Serper is primary provider; Brave tests skip cleanly via existing pytest.skip guards. - migrations-check job (new): applies both alembic migrations from scratch, asserts exactly one head per DB, sanity-tests downgrade/upgrade roundtrip. Runs on every PR regardless of path filter. - Path filter: add plugin-search-providers; dedupe worker-nodes block. Conftest split (tests/conftest.py → tests/integration/conftest.py) - libs/kt-{db,facts,graph,models,providers}, services/{api,worker-nodes, worker-bottomup,worker-sync,mcp}, plugins/backend-engine-search-providers - Top-level conftests keep only env setup (USE_HATCHET, SKIP_AUTH) and lightweight mock-friendly fixtures (_ambient_test_expense in models/ worker-nodes/worker-bottomup). - worker-{bottomup,sync,ingest} set a fake HATCHET_CLIENT_TOKEN to let modules that import the Hatchet client at load time collect tests. Test relocations (tests/ → tests/integration/) - libs/kt-db/tests/test_write_{dimensions,seeds}.py - use write_db_session - libs/kt-facts/tests/test_tiered_dedup_experiment.py - connects to write-db directly Alembic fix (libs/kt-db/alembic/versions/) - 212efc51d897 chained off e3eb15f51d7c instead of the shared initial_schema parent. Prior divergence would have failed the new migrations-check job. lefthook.yml / justfile - pre-push runs `just test-unit` (infra-free) instead of `just test-all`. - New `test-unit` and `test-integration` recipes; `test-all` remains as an alias that chains both.
Embeddings go through OpenRouter, so OPENAI_API_KEY is not configured as a repo secret. Keep SERPER_KEY + OPENROUTER_API_KEY only.
…odes - qdrant/qdrant:v1.17.2 does not exist; use v1.17.1. - services/worker-nodes/tests/conftest.py: set dummy HATCHET_CLIENT_TOKEN because auto_build imports the Hatchet client at module load, same as worker-bottomup, worker-ingest, and worker-sync.
worker-bottomup and worker-nodes have tests/integration/ dirs with just a conftest+__init__ but no test files. Previously they'd exit 5 (no tests collected) and fail the job. Now the matrix builder checks for actual test_*.py files before adding a package.
Addresses PR #234 review feedback: - `kt_db.testing` pytest plugin exposes engine/db_session/write_engine/ write_db_session fixtures. 8 integration conftests reduced from ~60 lines of duplication to a single `pytest_plugins = ["kt_db.testing"]` line. kt-facts (shared graph+write schema) and worker-sync (renamed fixtures + sync_engine) keep their own — they diverge meaningfully. - `kt_hatchet.testing.install_fake_hatchet_env()` replaces the inline JWT literal repeated in 4 worker conftests (bottomup, nodes, ingest, sync). Single source of the fake token and the Hatchet env triad. - Remove legacy justfile aliases (test-libs/test-api/test-mcp/ test-workers). All collapsed to test-unit, leaving the names misleading. - test.yml comment explains that integration tests use Base.metadata.create_all on per-worker schemas for xdist isolation, so model/migration drift is caught by `migrations-check`, not the integration matrix.
OpenRouter returned 429 on kt-models integration run. Two mitigations: - Switch test model from openrouter/google/gemini-2.0-flash-001 to openrouter/google/gemma-4-26b-a4b-it:nitro (matches Settings.decomposition_model). - Add per-suite pytest -n override to the integration matrix so suites that hit rate-limited external APIs can run serially. kt-models pinned to -n 1; all others default to -n auto.
Flaky runs observed on kt-db integration came from session-scoped state leaking across test modules (e.g. write_seeds committing rows that test_trigram_dedup read). Move schema creation/teardown from session scope to module scope so every test file gets its own random schema that's dropped on module teardown. - kt_db.testing: engine/schema_name/write_engine/write_schema_name are now module-scoped; db_session/write_db_session remain function-scoped with per-test txn rollback. Extensions install once per session. - kt-facts and worker-sync conftests follow the same pattern for their divergent fixtures (shared schema / renamed graph_engine). - services/api/tests/integration/*.py: downgrade local session-scoped fixtures (api_app, wi_client, etc.) to module scope to match the new engine scope. Result: test_trigram_dedup.test_abbreviation_found no longer needs to be deselected — runs green alongside the full suite. 152 kt-db integration tests in 6.6s (session-scope was 3s; the cost of per-module teardown is modest). Each CI run uses randomly-named schemas so concurrent runs and leftover data never interfere.
|
I have read the CLA Document and I hereby sign the CLA You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
tests/integration/suites. Gated by path filter so it only runs when relevant code changes.tests/integration/conftest.py. The three tests that genuinely need DB moved intotests/integration/.migrations-checkCI job — required, runs on every PR. Applies both graph-db and write-db Alembic migrations from scratch, asserts exactly one head each, and sanity-tests a downgrade/re-upgrade roundtrip.just test-unit(infra-free) instead ofjust test-all.SERPER_KEYandOPENROUTER_API_KEYconfigured.BRAVE_KEYintentionally not set — Serper is the primary provider; Brave tests skip cleanly via existingpytest.skip()guards.212efc51d897ande3eb15f51d7cwere both rooted at489643109ccd. Chained the former off the latter; the new migrations-check job would have failed without this.Addresses
Test plan
backend-lintpassesmigrations-checkpasses (detects single head, applies both DBs, downgrade roundtrip succeeds)backend-testmatrix runs hermetically per changed package — unit tests (~1240) pass without anyservices:blockbackend-integration-testmatrix runs per changed package — integration tests pass with postgres/postgres-write/redis/qdrant service containers +SERPER_KEY/OPENROUTER_API_KEYenvSKIPPED ... BRAVE_KEY not setin theplugin-search-providersintegration runfrontend-testpasses