Skip to content

feat(collectors): report_collector_errors helper for Flow Doctor visibility (v0.6.2)#34

Merged
cipher813 merged 1 commit into
mainfrom
feat/collector-error-helper
May 9, 2026
Merged

feat(collectors): report_collector_errors helper for Flow Doctor visibility (v0.6.2)#34
cipher813 merged 1 commit into
mainfrom
feat/collector-error-helper

Conversation

@cipher813
Copy link
Copy Markdown
Owner

Summary

Adds alpha_engine_lib.collector_results.report_collector_errors() — a thin helper that surfaces collector-style return-dicts ({"status": "error", "error": "..."}) to Flow Doctor's ERROR-level logging handler.

Why

Surfaced 2026-05-09 from the Saturday SF DataPhase1 PARTIAL run. The arcticdb collector hit a backfill regression preflight failure and stored the error in its result dict, but the only logger.error() in the pipeline is a generic summary line at the end of weekly_collector.main():

"Weekly collection finished with non-ok status=partial — Per-collector statuses: {arcticdb: error, ...}"

Two problems with that:

  1. Dedup collision. Same string regardless of which collector failed → Flow Doctor's per-message dedup treats every partial run as the same alert. Subsequent partial runs within the cooldown window are suppressed even when they fail for completely different reasons.
  2. No diagnose context. The actual error text ("Backfill regression preflight failed: 38 symbols would regress...") lives only in the result dict. Flow Doctor's LLM diagnosis pipeline + GitHub-issue body has nothing specific to work with.

This pattern recurs across orchestrators (data weekly_collector, predictor training stages, research per-team collectors) — every case where a try/except writes {"status": "error", "error": str(e)} and continues.

Approach

Single helper in the lib (canonical-home pattern) so any orchestrator using the results-dict shape gets per-collector ERROR visibility with one line. Output: one logger.error("collector %s failed: %s", name, err) per error entry → distinct dedup signatures, real error text routed through the existing setup_logging() Flow Doctor handler.

Test plan

  • 7 new unit tests in tests/test_collector_results.py covering: distinct error logs per collector, no-op on all-ok, missing error-field placeholder, malformed entries skipped, custom logger support, empty dict
  • Full lib suite: 287 passed locally
  • Version bump 0.6.1 → 0.6.2 lockstep (pyproject + __init__.py) verified by existing test_version_pin.py

Follow-up

  • alpha-engine-data PR pins to v0.6.2 and calls the helper in weekly_collector._finalize. Will land separately referencing this PR.
  • Opportunistic adoption in research/predictor where the same shape exists.

🤖 Generated with Claude Code

…bility (v0.6.2)

Surfaces collector-style {"status": "error", "error": "..."} return-dicts
to Flow Doctor's ERROR-level logging handler. Orchestrators that catch
exceptions into per-collector status entries (alpha-engine-data
weekly_collector, predictor training stages, research per-team
collectors) currently report errors only via a generic summary line at
the end of run, which dedups all partial runs into one signature and
contains no specific error text for LLM diagnosis.

The new helper walks the collectors dict and emits one logger.error per
error entry with the collector name + original message. Each emitted
record carries a distinct dedup signature, restoring the per-failure
alert granularity that Flow Doctor's pipeline expects.

Bumps version 0.6.1 → 0.6.2 in lockstep (pyproject + __init__).
First consumer: alpha-engine-data weekly_collector (separate PR).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@cipher813 cipher813 merged commit e162766 into main May 9, 2026
@cipher813 cipher813 deleted the feat/collector-error-helper branch May 9, 2026 12:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant