Skip to content

Count parse failures in enhance telemetry - #196

Merged
gadievron merged 1 commit into
masterfrom
pr-2b-enhancer-telemetry
Jul 26, 2026
Merged

Count parse failures in enhance telemetry#196
gadievron merged 1 commit into
masterfrom
pr-2b-enhancer-telemetry

Conversation

@gadievron

Copy link
Copy Markdown
Collaborator

Count parse failures in enhance telemetry

Branch: pr-2b-enhancer-telemetry (1 commit off master/d4caf8a) · Target: knostic/OpenAnt:master
Diff: 2 files, +42 (fix +4, test +38) · Not yet pushed — awaiting approval.

Problem

In the non-agentic enhancer (utilities/context_enhancer.py::ContextEnhancer.enhance_unit), when the LLM response can't be parsed, the else branch stores _get_default_context(error={"type":"parse_error"}) on the unit but does not increment self.stats["errors"] — while the exception branch already does. So the [Enhance] Errors: N summary under-reports parse failures.

This is telemetry-only: the unit still carries llm_context.error and is analyzed with default context (not dropped). Benign for coverage, wrong for the error tally.

Fix

Add self.stats["errors"] += 1 to the parse-failure else branch, mirroring the exception branch. One line, purely additive; no verdict/coverage behavior change.

Testing

  • tests/test_enhancer_parse_failure_counted.py (2 tests): a parse failure increments errors (and is not counted as units_enhanced); a successful parse does not. Passes on this d4caf8a base.

Blast radius (checked)

  • ContextEnhancer.stats["errors"] is the enhance-stage counter (surfaced in the [Enhance] Errors line). It is a different counter from the analysis-stage metrics["errors"] that core/reporter.py:442 reads ("units_analyzed": total_units - metrics.get("errors", 0), at the d4caf8a base) — so this fix does not change units_analyzed.

Provenance (direct re-derivation — NOT the KB grade-A record)

This fix stands on a direct re-derivation at the PR base d4caf8a: the parse-failure else branch stores
_get_default_context(error={"type":"parse_error"}) but does not self.stats["errors"] += 1, while the
adjacent except branch does — verified at d4caf8a, and the guard test fails on the pre-fix d4caf8a source.

Do NOT cite the KB enhancer-failed-context-no grade-A record as corroboration (per fable review): that
record describes a different symptom — "failed enhance stored with no error key" — which is the c81c0e1
state and is already fixed at d4caf8a (the else-branch there carries error={parse_error}). Its grade row's
stated basis ("byte-identical blob at c81c0e1") is also false — context_enhancer.py blobs differ (f90d29be5365606).
So the grade-A record neither matches this PR's target nor transfers by identity. (Both are KB-provenance defects
flagged for feedback in KB-STUDY.md.) This PR's narrower counter-defect is real and residual at d4caf8a.

Rollback

Git-revert-clean: one commit, context_enhancer.py (+ its test), disjoint from all other changes. Reverting
restores the prior (under-counting) telemetry with zero coverage/verdict impact.

…ors'])

In the non-agentic enhancer (ContextEnhancer.enhance_unit), the parse-failure
else-branch stored _get_default_context(error=...) on the unit but did NOT
increment self.stats['errors'] (the exception branch already does). So the
[Enhance] Errors telemetry under-reported parse failures. The unit still carries
the error and is analyzed with default context - this is a telemetry-honesty fix,
not a coverage change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@gadievron
gadievron merged commit e81d8b9 into master Jul 26, 2026
4 of 9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant