Skip to content

v3 prerequisites: fixes that distort the v3 measurements (§6) #245

Description

@nikolay-e

Engineering fixes that distort exactly the v3 measurements. All land before the dcbench split is frozen; the Q-class ones force a full calibration rerun if they land later. State verified at eb8f1a4f.

  • 1. Truncated generated fragments priced at 0 — fixed in 0b787e09: fragmentation.rs prices truncated content + per-fragment overhead. Used-tokens / budget-compliance columns are honest again.
  • 2. BM25 cell has no admission gatescoring.rs BM25Scoring still builds with admissible_files: None (it builds no graph). 53ba7b64 compensates by raising τ to UNGATED_STOPPING_THRESHOLD (0.12) when τ is the default — but an explicitly passed lower τ in BM25 mode is still ungated. Decide for cell 3: either derive admission for BM25 from the discovery universe (the gate then changes this cell's numbers — hence "before, not after") or document cell 3 as ungated-by-construction. Q-class either way.
  • 3. Empty-selection disclosure lost — fixed in 84bb6a0d / 0b787e09: every early exit returns ChangeSet::Empty { lockfile_changes, ignored_changes, policy_excluded_count }; regression test in tests/test_diffctx_invariants.py. C5's coverage block can rely on honest empty paths.
  • 4. stand_in_ids over-wide / mis-scoped — settled in d9af5925 / 53ba7b64: every sig_lookup pairing, with the safety argument in select.rs; carries_change in types.rs consumes it. The narrower definition was tried and regressed the stub role.
  • 5. memory_pipeline vs git.rs parity on pure deletions — PARTIAL: memory_pipeline.rs has a_pure_deletion_anchors_where_git_anchors_it, but its git side is a hand-written DiffHunk literal, not git::parse_diff output; nothing forces it to track git.rs. Needed: one test that runs both pipelines over the same deletion and asserts identical hunks (this is the "calib data comes from the same system" guarantee). Overlaps memory_pipeline re-spells the heavy phase: the #149 anti-drift fix covered only the selection half #232 (score_from_fragments shared prologue) — do them together.
  • 6. GIL held in pybridge — PARTIAL: the six pipeline entry points release the GIL via detach_guarded; still held: build_project_graph (full walk + tree-sitter parse under the GIL — the standout), resolve_diff_range (git subprocess), count_tokens, and the graph analytics that borrow &PyProjectGraph. For honest harness wall-clock p50/p90, build_project_graph and resolve_diff_range must detach before the matrix run; the analytics can wait.

Also blocking budget-compliance claims (C4): #241 (--budget is not a hard cap — cores bypass it, 3.15× at 1000) and #238 (one generated file takes 88% of a 15-file range).

Gate for every item: scripts/bitcheck.sh for E-class, corpus + re-baseline for Q-class; then freeze.

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions