Skip to content

fix: honeyslate's registry declared no blind spots, and had two - #65

Merged
inth3shadows merged 2 commits into
mainfrom
fix/honeyslate-registry-blind-spots
Aug 6, 2026
Merged

fix: honeyslate's registry declared no blind spots, and had two#65
inth3shadows merged 2 commits into
mainfrom
fix/honeyslate-registry-blind-spots

Conversation

@inth3shadows

Copy link
Copy Markdown
Owner

Closes #62.

Found by the same structural sweep that found testgraph's own blind spot in #59: seed every node of each product file, run the impact closure, intersect with the entry map. 17 of 38 honeyslate product files reached no journey. Most were expected — 6 alembic migrations, 6 frontend files TECHNICAL.md already excludes. Two were not.

The two real gaps

file why it is a real gap
app/main.py lifespan calls scheduler.start(). Break it and the auto-scheduler never runs — yet nothing selected J8 for a change to it. A recall bug, not a registration preference: the journey existed, its entries were just one hop short of the thing that starts it.
app/routers/google.py a live wired router (app.include_router(google.router)) serving /api/google/status and /selftest, with no journey at all

Closed by extending the entries of existing journeys, deliberately not by adding journeys — lifespan/create_app join J8, the two Google diagnostics join J7 (same integration as gcal_push). A J9 would have changed the registry's journey set, which is what labels_honeyslate.json and the published "mean 3.33 of 8 journeys named" are both written against.

app/cli.py stays uncovered and is now declared, along with the migrations and the frontend.

Why an undeclared gap here costs more than elsewhere

The registry previously declared nothing — and because it is approved: true, the UNAPPROVED REGISTRY banner whose entire job is to say "a NONE may mean not-registered rather than not-affected" never fired for it. honeyslate is the only repo whose numbers this project quotes as accuracy rather than sizes.

Re-measured, because the registry changed

harness/accuracy.py reads journeys/honeyslate.json live, so any edit silently re-points every published number at a registry that no longer exists — the exact defect 94e29a9 corrected for signedintake. Rather than assert that four added entries were harmless, both registries went through the harness back to back:

registry entries min recall mean precision (non-empty oracles)
before 16 1.00 0.68
after 20 1.00 0.68

S1 PASS either way.

The honest reason precision did not move: the five labeled commits barely touch the newly covered files. This shows the entries cost nothing on the scored set, not that widening entries is free in general. A future commit touching app/main.py will now select J8 — which is the point of the change, and exactly the case the labeled set does not contain.

Verification

entry nodes: 20 | unresolved: none
  backend/app/main.py            -> ['J8']
  backend/app/routers/google.py  -> ['J7']
  backend/app/cli.py             -> NONE   (declared)

All 8 journeys resolve, no unchecked_entries, no approval warning. 251 tests green.

Closes #62. Found by the same structural sweep that found testgraph's own blind
spot: seed every node of each product file, run the impact closure, intersect
with the entry map. 17 of 38 honeyslate product files reached no journey. Most
were expected — 6 alembic migrations, 6 frontend files TECHNICAL.md already
excludes. Two were not.

1. `app/main.py` — `lifespan` calls `scheduler.start()`. Break it and the
   auto-scheduler never runs, yet nothing selected J8 for a change to it. That is
   a RECALL BUG, not a registration preference: the journey exists, its entries
   were just one hop short of the thing that starts it. `lifespan` and
   `create_app` are now J8 entries.

2. `app/routers/google.py` — a LIVE wired router
   (`app.include_router(google.router)`) serving `/api/google/status` and
   `/selftest`, with no journey at all. Its two operator diagnostics are now J7
   entries: they belong to the same Google integration as `gcal_push` rather than
   to a journey of their own.

Closed by extending the ENTRIES of existing journeys, deliberately not by adding
journeys. A J9 would have changed the registry's journey set, which is what
`labels_honeyslate.json` and the published "mean 3.33 of 8 journeys named" are
both written against.

`app/cli.py` stays uncovered and is now DECLARED, along with the migrations and
the frontend. The registry previously declared nothing — and since it is
`approved: true`, the UNAPPROVED REGISTRY banner that exists to say "a NONE may
mean not-registered rather than not-affected" never fired for it. honeyslate is
the only repo whose numbers are quoted as ACCURACY rather than sizes, so an
undeclared gap costs more here than anywhere else.

RE-MEASURED, because the registry changed. `harness/accuracy.py` reads the
registry live, so an edit silently re-points every published number at a registry
that no longer exists — the exact defect 94e29a9 corrected for signedintake.
Rather than assert four added entries were harmless, both registries were run
back to back:

    before  16 entries   min recall 1.00   mean precision 0.68
    after   20 entries   min recall 1.00   mean precision 0.68

S1 PASS either way. The honest reason precision did not move: the five labeled
commits barely touch the newly covered files, so this shows the entries cost
nothing ON THE SCORED SET, not that widening entries is free in general. A future
commit touching app/main.py will now select J8 — the point of the change, and
exactly the case the labeled set does not contain.

Verified after the edit: all 8 journeys resolve, 20 entry nodes, no
unchecked_entries, no approval warning, and main.py -> [J8],
routers/google.py -> [J7], cli.py -> NONE.

251 tests.
The HIGH one first: the shipped map was never regenerated, so the fix did not
reach the tool's actual consumer. `maps/honeyslate.md` is what the
testgraph-verify skill reads — not a live index — and it had no section for
`app/main.py` or `routers/google.py` at all, with a legend still reading
`J7 … entry: gcal_push`. An agent editing main.py would have gotten "unknown,
escalate": the exact pre-fix behaviour, from a registry that had been fixed. My
"verified after the edit" was true only for `select` against a live index, which
does not exercise that path. Regenerated: 149 symbols, 23 files (was 21).

`create_app` DROPPED from J8. Measured: it produces a fan-out identical to
`lifespan`, so it bought nothing for selection while widening the ast_oracle.

The J8 fan-out cost is now documented rather than discovered later.
`impacted_closure` expands a file node into every symbol it contains, so
`file:app/main.py` enters the closure of every router: five files that did not
name J8 now do (routers/tasks.py, routers/auth.py, auth_service.py, schemas.py,
scheduling.py) at confidence 0.9 — above LOW_CONFIDENCE, so NOT flagged
verify_manually. A real precision cost paid for a real recall gap, recorded
because nothing in the harness output surfaces it.

`delete_task` added to J3. A live @router.delete handler that TECHNICAL.md
already listed at line 504 as missed by this registry — and that the file-level
sweep structurally could not see, since routers/tasks.py is covered by J1-J5
while that one symbol was not. Costs no new fan-out; the file is already reached.

seed_regressions.py re-run, which I skipped the first time — the same defect this
section cites 94e29a9 for, three lines above. A/B at one base:

    before  min recall 1.00  worst rank 6.00  8.00/8 named  PASS
    after   min recall 1.00  worst rank 6.07  8.00/8 named  PASS

This change moves worst rank by 0.07. The A/B also exposed something unrelated to
it: the published "3.33 of 8, worst rank 3.17" was measured at base 1cd0385 and
the harness runs at honeyslate's current HEAD (3f46a54), where BOTH registries
select 8 of 8. Those figures are stale because the TARGET REPO moved, not because
the registry did — a second way for a number to describe a world that no longer
exists, which no registry discipline would have caught.

"Precision unchanged" restated as MASKED, not tested: J8 was already selected on
every one of accuracy.py's five labeled commits, so that harness is structurally
incapable of registering the fan-out above. Its agreement is not evidence.

Also: blind spots moved into a structured `blind_spots` key instead of free-text
note; frontend tally corrected to 8 files (svelte.config.js and vite.config.js
were missing from a declaration whose whole purpose was completeness); route path
corrected to /api/google/selftest.

251 tests.
@inth3shadows
inth3shadows merged commit cd133df into main Aug 6, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

honeyslate's registry has an undeclared blind spot: routers/google.py is a live router with no journey

1 participant