Skip to content

test: watch indexer/ingest.py in the semantics tripwire (#106) - #116

Merged
IceRhymers merged 1 commit into
integration/indexer-performancefrom
feat/106-ingest-semantics-tripwire
Jul 25, 2026
Merged

test: watch indexer/ingest.py in the semantics tripwire (#106)#116
IceRhymers merged 1 commit into
integration/indexer-performancefrom
feat/106-ingest-semantics-tripwire

Conversation

@IceRhymers

Copy link
Copy Markdown
Owner

Refs #106

Mandatory PR B of #106's two-PR delivery: PR A (merged at ad04748) added indexer/ingest.py as the new production file source but deliberately left it out of the semantics tripwire's SEMANTICS_PATHS, since arming it in that same diff would have made the diff its own offender (added files count as changed for git diff --name-only) and forced the very INDEX_SEMANTICS_VERSION bump #106's acceptance criteria forbid. This PR closes that gap.

Changes

  • tests/unit/test_semantics_version_tripwire.py: add "indexer/ingest.py" to SEMANTICS_PATHS; extend the module docstring with why, and with the expected false-positive behavior below.
  • docs/runbooks/indexing-parallelism.md §5: add indexer/ingest.py to the module list that requires a version bump if changed.
  • indexer/store.py: comment-only fix to the delta-gate's lang/size no-re-derive justification — it previously cited indexer/parse.py + indexer/languages.py as the (watched) modules that derive those columns; PR A moved the actual derivation into indexer/ingest.py, so the comment now names it (plus languages.py's EXT_TO_LANG, which ingest.py calls directly) so a reviewer verifying the invariant isn't pointed at dead code.

No code in indexer/parse.py, indexer/symbols.py, indexer/languages.py, or app/db/models.py is touched, so no INDEX_SEMANTICS_VERSION bump is triggered by this PR itself.

Expected false positive — not confined to the future #111 arc merge

Adding a new path to SEMANTICS_PATHS makes it an offender in any diff whose base predates that file's creation. indexer/ingest.py postdates master (added by #106's first PR, not yet merged past integration/indexer-performance), so every local run of this test on this branch or its descendants sees indexer/ingest.py as an offender today — including, later, the diff that folds integration/indexer-performance into master (#111). Locally _base_ref() falls back to origin/master (there is usually no GITHUB_BASE_REF), so this is the common case, not a rare one; CI is actually less likely to hit it, since both workflows checkout at default depth-1 and this test typically skips there instead of running.

This is expected, not a regression, and not a reason to disable the test. Resolve it via the tripwire's own documented escape ("if this change genuinely cannot alter extraction output, say so in the PR") — do not bump INDEX_SEMANTICS_VERSION, and do not remove indexer/ingest.py from SEMANTICS_PATHS, to silence it. The actual backing for "no output change" is #106 PR A's oracle parity test (tests/unit/test_ingest_parity.py), which pins ingest.py against parse.py field-for-field on the (path, lang, size, content) set.

Verification

  • make lint — green (ruff check, ruff format --check, mypy app indexer webui).
  • GITHUB_BASE_REF=integration/indexer-performance uv run pytest tests/unit/test_semantics_version_tripwire.py -v — 9 passed, none skipped, no bump required (this is the PR's actual base, matching the local sibling-tripwire rule from the indexer: single-pass in-memory tarball ingestion (drop extract-to-disk) #106 plan).
  • GITHUB_BASE_REF=integration/indexer-performance uv run pytest -m "unit or observability" — 1221 passed, 261 deselected.
  • git diff --stat origin/integration/indexer-performance...HEAD -- indexer/parse.py and the app/db/models.py INDEX_SEMANTICS_VERSION diff are both empty — the primary AC3 evidence gate.
  • make webui-verify-dist — green, no drift (no frontend change).
  • make test-integration — ran locally against the codesearch-pg container; 8 failed / 205 passed / 43 errors, all from missing Lakebase-only extensions (lakebase_tokenizer, lakebase_ann, lakebase_bm25, ...) that don't exist in stock Postgres. Confirmed byte-identical failure count on the unmodified base commit (ad04748) via git stash, so this is a pre-existing local-environment limitation documented in tests/integration/AGENTS.md ("this project is Lakebase-only... no local Postgres in the development loop"), not something this PR introduces. CI's own Lakebase integration gate is repo-wide unprovisioned per the indexer: single-pass in-memory tarball ingestion (drop extract-to-disk) #106 plan.
  • Independent code-reviewer pass (separate context) found 2 HIGH + 1 MEDIUM + 2 LOW prose-accuracy issues in the first draft (mis-scoped "one-time Indexing performance integration arc (#110) #111-only" false-positive framing, an incorrect "parse.py has no production caller" claim, and languages.py's EXT_TO_LANG mis-described as oracle-only) — all fixed; diff scope unchanged (3 files, no production code path affected — indexer/store.py's change is comment-only).

Arms tests/unit/test_semantics_version_tripwire.py's SEMANTICS_PATHS on
indexer/ingest.py, the production file source added by the prior #106 PR,
which was left unwatched to avoid a spurious INDEX_SEMANTICS_VERSION bump
in that PR's own diff. Also updates indexer/store.py's delta-gate
lang/size no-re-derive justification to name the module that now performs
that derivation, and adds indexer/ingest.py to the runbook's list of
version-bump-triggering modules.

Refs #106
@IceRhymers
IceRhymers merged commit 732a7d7 into integration/indexer-performance Jul 25, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant