Skip to content

feat: per-stage model routing for the extraction pipeline#40

Merged
miguelgfierro merged 7 commits into
demofrom
feat/per-stage-models
Jul 9, 2026
Merged

feat: per-stage model routing for the extraction pipeline#40
miguelgfierro merged 7 commits into
demofrom
feat/per-stage-models

Conversation

@miguelgfierro

Copy link
Copy Markdown

What

Every LLM pipeline stage can now run on its own model, configured via .env / environment variables. Stages left unconfigured fall back to the global FLYDOCS_MODEL (or the request's options.model when the caller sets one).

Stage Variable Default in env_template
splitter FLYDOCS_SPLITTER_MODEL anthropic:claude-sonnet-4-6
classifier FLYDOCS_CLASSIFIER_MODEL anthropic:claude-sonnet-4-6
extract FLYDOCS_EXTRACT_MODEL anthropic:claude-sonnet-4-6
visual_authenticity FLYDOCS_VISUAL_AUTHENTICITY_MODEL anthropic:claude-sonnet-4-6
content_authenticity FLYDOCS_CONTENT_AUTHENTICITY_MODEL anthropic:claude-sonnet-4-6
judge FLYDOCS_JUDGE_MODEL anthropic:claude-opus-4-8
rules FLYDOCS_RULE_ENGINE_MODEL anthropic:claude-sonnet-4-6
transform FLYDOCS_TRANSFORM_MODEL anthropic:claude-sonnet-4-6
bbox LLM matcher FLYDOCS_BBOX_MATCHER_MODEL anthropic:claude-sonnet-4-6

Resolution precedence (per stage)

FLYDOCS_<STAGE>_MODEL > options.model (request) > FLYDOCS_MODEL.

A pinned stage wins over options.model so operator stage-tuning survives per-request overrides.

How

  • IDPSettings: nine new optional fields (None = inherit the global model).
  • PipelineOrchestrator: a _stage_models() resolver computes the per-stage map once per request, stashed as ctx.metadata["stage_models"]; the seven per-call sites read their stage entry instead of the shared model_id.
  • transform and the bbox LLM matcher receive their model at bean construction (core/configuration.py) since those services are not passed a model per pipeline call.
  • Escalation is untouched: FLYDOCS_ESCALATION_MODEL / options.escalation.model still drives the judge-triggered re-run.
  • Cost telemetry needs no change: pipeline.usage.by_model already splits usage per model id.

Tests

  • 7 new unit tests (tests/unit/test_per_stage_models.py): settings defaults + env binding, resolver fallback + precedence, end-to-end orchestrator routing (extract + judge receive their pinned models), and bean wiring for matcher/transformer.
  • Full unit suite: 305 passed. ruff clean; no new pyright errors (12 pre-existing on base).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant