Skip to content

docs(memory): record hub-degree kill-criterion re-measurement (spec-067 §7.1)#6345

Merged
bug-ops merged 1 commit into
mainfrom
feat/issue-5625/hub-degree-remeasure
Jul 16, 2026
Merged

docs(memory): record hub-degree kill-criterion re-measurement (spec-067 §7.1)#6345
bug-ops merged 1 commit into
mainfrom
feat/issue-5625/hub-degree-remeasure

Conversation

@bug-ops

@bug-ops bug-ops commented Jul 16, 2026

Copy link
Copy Markdown
Owner

Summary

  • Re-measures the spec-067 §7 hub-degree kill-criterion on the real 29-transcript subagent corpus, per the knowledge ingest: hub-degree kill-criterion fails in first live subagent extraction (Python 24.3%) #5467 follow-up recommendation to re-measure before a final graph-sink (epic implement knowledge ingest command (zeph knowledge ingest, spec 067) #5012) go/no-go call.
  • N=5 fixed dry-runs (knowledge ingest --source subagents --dry-run) yield a clean, decisive FAIL: top-entity share 25.8-29.3% (median 27.9%, 0/5 runs ≤15%). knowledge ingest: hub-degree kill-criterion fails in first live subagent extraction (Python 24.3%) #5467's Rule 3 language/tool anchoring fix measurably relocated degree away from Python (now mid-table, never top), but hub formation migrated to other incidentally-repeated command names (bash, curl) — exactly as that fix's own root-cause analysis predicted.
  • The conditional prompt.rs generalization (extending the anchoring rule to bare concept-typed nouns) was not applied: its precondition — a derivable signal-to-noise (S/N) re-measurement alongside hub-degree — is not met by the existing dry-run report (IngestReport carries only entity/degree counts, no edge-type breakdown). crates/zeph-memory/src/graph/ingest/prompt.rs is unchanged.
  • Recommendation recorded: defer the epic implement knowledge ingest command (zeph knowledge ingest, spec 067) #5012 graph-sink go/no-go call — no corpus available in this environment is production-representative (the 29-transcript corpus is dominated by small CI-test/smoke-test scenarios, not architecture/spec/code-review/debugging sessions).
  • The extraction provider (openai-stt/gpt-4o-mini, cloud) was unreachable this session (quota exhaustion); measurement used a local Ollama substitute instead, documented as an explicit caveat — the two measurement cycles (knowledge ingest: hub-degree kill-criterion fails in first live subagent extraction (Python 24.3%) #5467 cloud, this PR local) agree qualitatively that the gate fails, but are not directly numerically comparable.
  • Three follow-ups recorded (not filed as issues by this PR): (a) add S/N instrumentation to the dry-run report, (b) accumulate/construct a genuinely diverse production-representative corpus, (c) re-run this same corpus on the cloud provider once quota is restored to isolate the provider variable.

This PR went through a full architect → critic (2 rounds, verdict significant → minor) → developer → tester/perf/security/impl-critic → reviewer (1 fix-and-re-review round) chain. No source code was changed — this is a measurement/reporting-only PR.

Closes #5625

Test plan

  • N=5 fixed dry-run measurement protocol executed and recorded (no discarding/re-rolling)
  • crates/zeph-memory/src/graph/ingest/prompt.rs confirmed unchanged (git diff empty)
  • Existing dry_run_hub_top_n regression test (crates/zeph-memory/src/semantic/graph.rs) re-run, passes
  • cargo nextest run -p zeph -E 'test(knowledge)' (44/44) and -p zeph-memory (1659/1659) pass
  • gitleaks protect --staged clean
  • Reporting artifacts cross-checked for numeric consistency: spec.md §7.1, CHANGELOG.md, main-repo playbook (knowledge-ingest.md G3), main-repo coverage-status.md row 990
  • Rebased onto latest origin/main, no conflicts (docs-only diff)

…67 §7.1)

Re-measures the spec-067 §7 hub-degree kill-criterion on the real
29-transcript subagent corpus per the #5467 follow-up. N=5 fixed dry-runs
show a clean, decisive FAIL (25.8-29.3%, median 27.9%, 0/5 runs <=15%):
#5467's language/tool anchoring fix relocated degree away from Python, but
hub formation migrated to other incidentally-repeated command names (bash,
curl), exactly as that fix's own analysis predicted. The conditional
prompt.rs generalization was not applied because its precondition (a
derivable signal-to-noise re-measurement) is not met by the existing
dry-run report. Recommends deferring the epic #5012 graph-sink go/no-go
call until a production-representative corpus becomes available.

Closes #5625
@github-actions github-actions Bot added documentation Improvements or additions to documentation size/M Medium PR (51-200 lines) labels Jul 16, 2026
@bug-ops
bug-ops merged commit 87e91af into main Jul 16, 2026
23 checks passed
@bug-ops
bug-ops deleted the feat/issue-5625/hub-degree-remeasure branch July 16, 2026 23:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/M Medium PR (51-200 lines)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

knowledge ingest: re-measure hub-degree kill-criterion on a larger/diverse corpus before G2 go/no-go

1 participant