fix: honeyslate's registry declared no blind spots, and had two - #65
Merged
Conversation
Closes #62. Found by the same structural sweep that found testgraph's own blind spot: seed every node of each product file, run the impact closure, intersect with the entry map. 17 of 38 honeyslate product files reached no journey. Most were expected — 6 alembic migrations, 6 frontend files TECHNICAL.md already excludes. Two were not. 1. `app/main.py` — `lifespan` calls `scheduler.start()`. Break it and the auto-scheduler never runs, yet nothing selected J8 for a change to it. That is a RECALL BUG, not a registration preference: the journey exists, its entries were just one hop short of the thing that starts it. `lifespan` and `create_app` are now J8 entries. 2. `app/routers/google.py` — a LIVE wired router (`app.include_router(google.router)`) serving `/api/google/status` and `/selftest`, with no journey at all. Its two operator diagnostics are now J7 entries: they belong to the same Google integration as `gcal_push` rather than to a journey of their own. Closed by extending the ENTRIES of existing journeys, deliberately not by adding journeys. A J9 would have changed the registry's journey set, which is what `labels_honeyslate.json` and the published "mean 3.33 of 8 journeys named" are both written against. `app/cli.py` stays uncovered and is now DECLARED, along with the migrations and the frontend. The registry previously declared nothing — and since it is `approved: true`, the UNAPPROVED REGISTRY banner that exists to say "a NONE may mean not-registered rather than not-affected" never fired for it. honeyslate is the only repo whose numbers are quoted as ACCURACY rather than sizes, so an undeclared gap costs more here than anywhere else. RE-MEASURED, because the registry changed. `harness/accuracy.py` reads the registry live, so an edit silently re-points every published number at a registry that no longer exists — the exact defect 94e29a9 corrected for signedintake. Rather than assert four added entries were harmless, both registries were run back to back: before 16 entries min recall 1.00 mean precision 0.68 after 20 entries min recall 1.00 mean precision 0.68 S1 PASS either way. The honest reason precision did not move: the five labeled commits barely touch the newly covered files, so this shows the entries cost nothing ON THE SCORED SET, not that widening entries is free in general. A future commit touching app/main.py will now select J8 — the point of the change, and exactly the case the labeled set does not contain. Verified after the edit: all 8 journeys resolve, 20 entry nodes, no unchecked_entries, no approval warning, and main.py -> [J8], routers/google.py -> [J7], cli.py -> NONE. 251 tests.
The HIGH one first: the shipped map was never regenerated, so the fix did not reach the tool's actual consumer. `maps/honeyslate.md` is what the testgraph-verify skill reads — not a live index — and it had no section for `app/main.py` or `routers/google.py` at all, with a legend still reading `J7 … entry: gcal_push`. An agent editing main.py would have gotten "unknown, escalate": the exact pre-fix behaviour, from a registry that had been fixed. My "verified after the edit" was true only for `select` against a live index, which does not exercise that path. Regenerated: 149 symbols, 23 files (was 21). `create_app` DROPPED from J8. Measured: it produces a fan-out identical to `lifespan`, so it bought nothing for selection while widening the ast_oracle. The J8 fan-out cost is now documented rather than discovered later. `impacted_closure` expands a file node into every symbol it contains, so `file:app/main.py` enters the closure of every router: five files that did not name J8 now do (routers/tasks.py, routers/auth.py, auth_service.py, schemas.py, scheduling.py) at confidence 0.9 — above LOW_CONFIDENCE, so NOT flagged verify_manually. A real precision cost paid for a real recall gap, recorded because nothing in the harness output surfaces it. `delete_task` added to J3. A live @router.delete handler that TECHNICAL.md already listed at line 504 as missed by this registry — and that the file-level sweep structurally could not see, since routers/tasks.py is covered by J1-J5 while that one symbol was not. Costs no new fan-out; the file is already reached. seed_regressions.py re-run, which I skipped the first time — the same defect this section cites 94e29a9 for, three lines above. A/B at one base: before min recall 1.00 worst rank 6.00 8.00/8 named PASS after min recall 1.00 worst rank 6.07 8.00/8 named PASS This change moves worst rank by 0.07. The A/B also exposed something unrelated to it: the published "3.33 of 8, worst rank 3.17" was measured at base 1cd0385 and the harness runs at honeyslate's current HEAD (3f46a54), where BOTH registries select 8 of 8. Those figures are stale because the TARGET REPO moved, not because the registry did — a second way for a number to describe a world that no longer exists, which no registry discipline would have caught. "Precision unchanged" restated as MASKED, not tested: J8 was already selected on every one of accuracy.py's five labeled commits, so that harness is structurally incapable of registering the fan-out above. Its agreement is not evidence. Also: blind spots moved into a structured `blind_spots` key instead of free-text note; frontend tally corrected to 8 files (svelte.config.js and vite.config.js were missing from a declaration whose whole purpose was completeness); route path corrected to /api/google/selftest. 251 tests.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #62.
Found by the same structural sweep that found testgraph's own blind spot in #59: seed every node of each product file, run the impact closure, intersect with the entry map. 17 of 38 honeyslate product files reached no journey. Most were expected — 6 alembic migrations, 6 frontend files TECHNICAL.md already excludes. Two were not.
The two real gaps
app/main.pylifespancallsscheduler.start(). Break it and the auto-scheduler never runs — yet nothing selected J8 for a change to it. A recall bug, not a registration preference: the journey existed, its entries were just one hop short of the thing that starts it.app/routers/google.pyapp.include_router(google.router)) serving/api/google/statusand/selftest, with no journey at allClosed by extending the entries of existing journeys, deliberately not by adding journeys —
lifespan/create_appjoin J8, the two Google diagnostics join J7 (same integration asgcal_push). A J9 would have changed the registry's journey set, which is whatlabels_honeyslate.jsonand the published "mean 3.33 of 8 journeys named" are both written against.app/cli.pystays uncovered and is now declared, along with the migrations and the frontend.Why an undeclared gap here costs more than elsewhere
The registry previously declared nothing — and because it is
approved: true, theUNAPPROVED REGISTRYbanner whose entire job is to say "a NONE may mean not-registered rather than not-affected" never fired for it. honeyslate is the only repo whose numbers this project quotes as accuracy rather than sizes.Re-measured, because the registry changed
harness/accuracy.pyreadsjourneys/honeyslate.jsonlive, so any edit silently re-points every published number at a registry that no longer exists — the exact defect94e29a9corrected for signedintake. Rather than assert that four added entries were harmless, both registries went through the harness back to back:S1 PASS either way.
The honest reason precision did not move: the five labeled commits barely touch the newly covered files. This shows the entries cost nothing on the scored set, not that widening entries is free in general. A future commit touching
app/main.pywill now select J8 — which is the point of the change, and exactly the case the labeled set does not contain.Verification
All 8 journeys resolve, no
unchecked_entries, no approval warning. 251 tests green.