docs: seal the results headline on the v227 to v230 frame#707
Merged
Conversation
The rendered book led a stranger to the wrong headline. This corrects it without recomputing any number. - results.rst: replace the superseded Fmax/ESM-C chapter with a short, honest chapter that states the sealed board (f_micro_w on the leakage-free v227 to v230 frame, learned k-WTA retrieval encoder d8979601 plus a stacked per-category re-ranker) and cross-references the evaluation protocol and the reproduction path. - historical/: move the superseded results and the old reproduction guide verbatim under a dated "superseded, retained for provenance" banner, in a hidden toctree that does not route a new reader in. - operate/reproduce-0.4063.rst: honest reproduction stub that names each stage (stand up the stack, load the v227 snapshot, compute learned-encoder codes, retrieve, re-rank, score with cafaeval on the sealed settings) and states plainly which stages are not yet automated. It cites only verified operations, endpoints and runbooks; the retired POST /scoring/rerankers/train is dropped. - index.rst, abstract.rst, introduction.rst: correct the pre-pillars "LightGBM re-ranking over ESM-C KNN" framing to the learned retrieval encoder plus stacked per-category re-ranker on the leakage-free frame; route newcomers to the quickstart and the evidence, cross-reference the number rather than restating it. - CLAUDE.md: replace the non-existent scripts/run_one_job.py command with the real dispatch-plus-worker path.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## develop #707 +/- ##
===========================================
- Coverage 88.90% 88.85% -0.05%
===========================================
Files 237 245 +8
Lines 17692 18481 +789
===========================================
+ Hits 15729 16422 +693
- Misses 1963 2059 +96 🚀 New features to boost your workflow:
|
github-actions Bot
pushed a commit
that referenced
this pull request
Jul 10, 2026
…_job.py) (#709) ## Why PR #707 resealed the results docs on the sealed headline (`f_micro_w` 0.4063, first in 7 of 9 cells, frame v227 to v230). Two residuals survived it, both flagged in #707's own "known residuals" note. ## Defect 1: `architecture/evaluation.rst` **The stale 0.431 board is removed, not moved, because the record already exists.** The full superseded board (the k-ablation, scoring configs, the three re-ranker generations, the external-tool benchmark carrying the `0.431` Fmax, and the leakage analysis) is already preserved **verbatim** in `historical/pre-v227.rst` under a dated "superseded, retained for provenance only" banner created by #707. The copy in `evaluation.rst` was a duplicate of that record. Deleting the duplicate and pointing to the historical page preserves the record while removing the superseded number from a page a reader would read as current. The protocol page keeps the temporal-leakage caveat as a protocol-level point (why external comparisons leak, why references freeze at t0), minus the specific stale figures. **Discrepancy reported:** the task described *two* "Provisional" admonitions on this page. There is only **one** literal `.. admonition:: Provisional results, pending final recompute` (and #707's own PR body refers to it in the singular). I resolved that one and also corrected a related stale `seealso` that framed `/results` as the "GOA 220 to 229 temporal holdout"; `/results` is now the sealed v227 to v230 board, so that bullet was retargeted and a link to the historical figures added. The one admonition read: > **Provisional results, pending final recompute** (class: warning) > The Fmax values in the tables below were produced before the 2026-04-10 unification of the embedding-backend slicing convention (see operations, section Residue-tensor convention). They will be regenerated end-to-end for the Zenodo deposit accompanying the thesis. The experimental protocol is stable; only the numbers will change. See /results for the full provisional notice. **Resolution:** it is no longer "provisional pending recompute". That board is a *different metric on a different, superseded frame*; it was never going to become the sealed board, so there is nothing to recompute into it. The honest state is **superseded**, which is exactly how the historical page frames it. The admonition (and its dangling pointer to a "full provisional notice" on `/results`, which no longer exists) was removed together with the duplicated board, and the page now points to the sealed board and the retained historical record. ## Defect 2: `complexity/measuring.rst` `scripts/run_one_job.py` was cited three times as a profiling target. **It does not exist.** Git history: it was added in the initial commit, migrated in `b08f536`, and **deleted in `80ed10e`** ("feat: CI/CD, linting, ngrok expose, and test fixes"). Its old body called `worker.handle_job(job_id)` directly against a hardcoded 3-op registry, bypassing the queue, the exact ad-hoc pattern the house rules now forbid. There is **no single-job CLI wrapper**. The task named the internal method `execute_single(job_id)`; that name does not exist anywhere in the repo. The real internal entry point is `BaseWorker.handle_job(job_id)` in `protea/workers/base_worker.py`, which is what the page now points at. **What replaced the dead command (all verified to exist):** - Dispatch one job via `POST /jobs` with `{operation, queue_name, payload}` (the only sanctioned dispatch path; the request model and `{"id", "status"}` response were read from `protea/api/routers/jobs.py`). - Profile the worker that consumes it: `poetry run scalene ... scripts/worker.py --queue <queue>`. `scalene` is a real project dependency (`pyproject.toml`, locked at 2.3.0); `scripts/worker.py --queue` is the documented worker command; the worker catches `KeyboardInterrupt`, logs `Worker stopped.`, and exits cleanly (read in `scripts/worker.py`), so scalene's on-exit report is written after `Ctrl+C`. - Function-level fallback via stdlib `cProfile` + `pstats` wrapping the same worker. - **Removed** the `pyinstrument` and `snakeviz` commands: neither is a project dependency, so `poetry run pyinstrument` / `snakeviz` were unverifiable and I would not print them. - Added a **Known gap** rubric: no CLI runs a single job by id and exits; `scripts/run_one_job.py` was removed in `80ed10e`; the internal entry point is `BaseWorker.handle_job(job_id)`; until a wrapper lands, profile the continuous worker or read per-step timings from the JobEvent log (which needs no profiler). ## Validation - `scripts/check_no_em_dashes.py`: OK, 0 violations on both files. - `scripts/check_doc_refs.py`: 199 broken refs, **unchanged** from the clean-`develop` baseline. All 5 contributed by `evaluation.rst` are pre-existing autodoc `:module:`/`:func:` roles I did not touch; `measuring.rst` contributes 0. My new `:doc:` cross-refs (`/results`, `/historical/pre-v227`, `/operate/reproduce-0.4063`) all resolve. - Docs only; no code, no database. ## Things I was told that turned out false 1. "**Two** Provisional admonitions" in `evaluation.rst`: there is only one. 2. "`base_worker.py` has an internal **`execute_single(job_id)`** method": no such method exists; the real entry point is `handle_job(job_id)`. Documented the real one; fabricated nothing.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The rendered Sphinx book led a stranger to the wrong headline: the Results chapter sold a superseded Fmax board (ESM-C 300M, GOA 220 to 229, the lgbm_v1/v2/v3 progression), the landing page routed every new reader into it, and the reproduction guide drove a retired endpoint (
POST /scoring/rerankers/train, zero routes implement it). This corrects the framing without recomputing any number.What changed
results.rst: replaced with a short, honest chapter that states the sealed board once (f_micro_w on the leakage-free v227 to v230 frame, learned k-WTA retrieval encoderd8979601plus a stacked per-category re-ranker, first in 7 of 9 cells, the two BP cells evidence-bound) and cross-referencesarchitecture/evaluation(protocol home), ADR D43 (re-ranker), and the reproduction stub.historical/: moved the superseded results (historical/pre-v227.rst) and the old reproduction guide (historical/reproduction_guide.rst) verbatim under a dated "superseded, retained for provenance" banner, in a hidden toctree that does not route a new reader in.operate/reproduce-0.4063.rst(new): honest reproduction stub naming each stage (stand up the stack, load the v227 snapshot, compute learned-encoder codes, retrieve, re-rank, score with cafaeval on the sealed settings) and stating plainly which stages are not yet automated. Cites only verified operations, endpoints, and runbooks; the retired train endpoint is dropped.index.rst,abstract.rst,introduction.rst: corrected the pre-pillars "LightGBM re-ranking over ESM-C KNN" headline to the learned retrieval encoder plus stacked per-category re-ranker on the leakage-free frame; route newcomers to the quickstart and the evidence; cross-reference the number rather than restating it.CLAUDE.md: replaced the non-existentscripts/run_one_job.pycommand with the real dispatch-plus-worker path (POST /jobs+scripts/worker.py).Before / after grep counts
0.431inresults.rsthistorical/pre-v227.rst)f_micro_winresults.rstrerankers/train(active docs)appendix/reproduction_guide.rst)historical/reproduction_guide.rst, flagged as retired)run_one_jobinCLAUDE.mdValidation
scripts/check_no_em_dashes.py: OK (127 files, 0 violations).scripts/check_doc_refs.py: exit 0.sphinx-build -b dummy: no toctree / cross-reference / dangling-doc warnings introduced (only pre-existing autodoc import warnings from missing runtime deps in this environment).Known residuals (not in this PR's scope)
architecture/evaluation.rststill contains a stale0.431Fmax benchmark table with its own "Provisional results" self-admission. It was designated the protocol home and out of scope here; flagging for a follow-up.complexity/measuring.rststill citesscripts/run_one_job.pythree times as a profiling target. There is no single-job CLI to substitute (jobs dispatch viaPOST /jobsand a worker consumes them), so no command was fabricated; flagging for a follow-up.🤖 Generated with Claude Code
https://claude.ai/code/session_01M2HhDnUSRArSF7P6kSMnXB