Skip to content

feat(kt-config): validate_all_graph_types startup canary + API wiring#255

Merged
charlie83Gs merged 2 commits intomainfrom
feat/startup-composition-validation
Apr 20, 2026
Merged

feat(kt-config): validate_all_graph_types startup canary + API wiring#255
charlie83Gs merged 2 commits intomainfrom
feat/startup-composition-validation

Conversation

@charlie83Gs
Copy link
Copy Markdown
Contributor

Summary

Adds `PluginRegistry.validate_all_graph_types()` — walks every registered graph type's composition and returns `{graph_type_id: [CompositionIssue]}`, with clean types excluded so the map is directly truthy-testable as "is the registry valid?".

Wires it into `services/api/src/kt_api/main.py::_load_plugins()` so startup logs one WARNING per broken graph type before the first pipeline task runs. Tolerant, not fatal: a missing contribution is an operator-visible config bug (expected during phased plugin extraction while provider ids still point at inline implementations), not a reason to refuse to boot.

Log shape: one line per broken graph type, every unresolved id on that line, so ops can grep one string per plugin instead of spelunking through dozens of log entries.

Delegates per-type to the existing `validate_composition` so pair-wise agentic + concept_extractor + list semantics stay consistent in one place.

Test plan

  • 4 new tests in `test_validate_composition.py`: empty registry, clean registry returns `{}`, broken types grouped by id, mixed registry excludes clean types
  • `libs/kt-config`: 212 pass
  • `services/api`: 80 unit pass (startup path exercised by existing app import)
  • `ruff check`/`format` clean, pyright green
  • pre-commit + pre-push lefthook green

🤖 Generated with Claude Code

charlie83Gs and others added 2 commits April 20, 2026 12:01
Adds PluginRegistry.validate_all_graph_types() that walks every
registered graph type's composition and returns a
``{graph_type_id: [issues]}`` map — with clean types excluded so
the map is truthy-testable as "is the registry valid?".

Wires it into the API's ``_load_plugins()`` so startup logs one
WARNING per broken graph type before the first pipeline task
runs. Tolerant rather than raising: a missing contribution is
an operator-visible config bug (expected during phased plugin
extraction when provider ids still point at inline impls), not
a reason to refuse to boot the API.

Startup log shape keeps every unresolved provider id on a single
line per graph type so operators see "GraphTypePlugin 'default'
has N unresolved provider id(s): fact_decomposition='llm-default'
(no contribution registered), …" without needing to grep across
dozens of log lines.

Delegates per-type to the existing validate_composition so
pair-wise agentic + concept_extractor + list semantics stay
consistent in one place.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…r every process

Address PR #255 review: the composition canary only fired in the API,
but pipeline tasks run in Hatchet workers. If a worker booted without
the API, operators got zero signal on unresolved provider ids — the
whole point of the canary.

Extracts the bootstrap triad into one function in kt_providers.registry:

  load_default_plugins()            # register plugins
  bridge_plugin_search_providers()  # bridge search contributions
  validate_all_graph_types() + log  # canary

Replaces the identical 4-line copy-paste at 5 entry points (API +
worker-{search,bottomup,ingest,all}) with a single bootstrap_plugins()
call, so every process that might dispatch a pipeline task fires the
canary at boot.

Side effect: CompositionIssue.reason is still interpolated verbatim
into the WARNING, so ops can tell "empty provider_id" (code/config
bug) apart from "no contribution registered" (phase extraction still
in progress) — they need different fixes.

Also drops unnecessary forward-ref quotes on
PluginRegistry.validate_all_graph_types return annotation
(``dict[str, list[CompositionIssue]]``) — CompositionIssue is
defined above the method and ``from __future__ import annotations``
is active, so quoting was redundant with the body's usage.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@charlie83Gs
Copy link
Copy Markdown
Contributor Author

Addressed review feedback

Widened to cover the main gap — canary now fires from every process that dispatches pipeline tasks, not just the API.

#1 Canary wired to workers 🔴 — Extracted the three-step bootstrap triad (load → bridge → validate+log) into a single `bootstrap_plugins()` function in `kt_providers.registry`. Replaced the identical copy-paste at 5 entry points:

  • `services/api/src/kt_api/main.py`
  • `services/worker-search/src/kt_worker_search/main.py`
  • `services/worker-bottomup/src/kt_worker_bottomup/main.py`
  • `services/worker-ingest/src/kt_worker_ingest/main.py`
  • `services/worker-all/src/kt_worker_all/main.py`

Every process that might dispatch a pipeline task now fires the canary at boot. Side benefit: de-dupes the 5-way copy-paste the review flagged.

#2 `reason` preserved 🟡 — Kept `{i.reason}` verbatim in the log message, as you noted. Added a test assertion (`test_bootstrap_logs_canary_for_broken_graph_type`) that explicitly checks the reason string surfaces in the WARNING so this doesn't drift.

Nit: drop quotes on return annotation — Done. `PluginRegistry.validate_all_graph_types` now annotates `dict[str, list[CompositionIssue]]` unquoted; consistent with the body on L1071.

Nit: tests reuse helper — Considered, left as-is. The new `test_bootstrap_plugins.py` is a separate file using lightweight stubs; reusing `_FullPlugin` from `test_validate_composition.py` would create a cross-file dependency for minor LoC savings. Inline stubs keep the two test files self-contained.

Tests

  • 3 new tests in `libs/kt-providers/tests/test_bootstrap_plugins.py`: bridging end-to-end, canary logs for broken graph type (with reason preserved), empty registry is silent
  • `just test-unit` across all 24 packages: green (including the 4 worker unit test suites that newly import `bootstrap_plugins`)

@charlie83Gs charlie83Gs merged commit cbe4d29 into main Apr 20, 2026
27 checks passed
@charlie83Gs charlie83Gs deleted the feat/startup-composition-validation branch April 20, 2026 19:16
@github-actions
Copy link
Copy Markdown


Thank you for your submission, we really appreciate it. Like many open-source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution. You can sign the CLA by just posting a Pull Request Comment same as the below format.


I have read the CLA Document and I hereby sign the CLA


You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant