feat: per-stage model routing for the extraction pipeline#40
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Every LLM pipeline stage can now run on its own model, configured via
.env/ environment variables. Stages left unconfigured fall back to the globalFLYDOCS_MODEL(or the request'soptions.modelwhen the caller sets one).env_templateFLYDOCS_SPLITTER_MODELanthropic:claude-sonnet-4-6FLYDOCS_CLASSIFIER_MODELanthropic:claude-sonnet-4-6FLYDOCS_EXTRACT_MODELanthropic:claude-sonnet-4-6FLYDOCS_VISUAL_AUTHENTICITY_MODELanthropic:claude-sonnet-4-6FLYDOCS_CONTENT_AUTHENTICITY_MODELanthropic:claude-sonnet-4-6FLYDOCS_JUDGE_MODELanthropic:claude-opus-4-8FLYDOCS_RULE_ENGINE_MODELanthropic:claude-sonnet-4-6FLYDOCS_TRANSFORM_MODELanthropic:claude-sonnet-4-6FLYDOCS_BBOX_MATCHER_MODELanthropic:claude-sonnet-4-6Resolution precedence (per stage)
FLYDOCS_<STAGE>_MODEL>options.model(request) >FLYDOCS_MODEL.A pinned stage wins over
options.modelso operator stage-tuning survives per-request overrides.How
IDPSettings: nine new optional fields (None= inherit the global model).PipelineOrchestrator: a_stage_models()resolver computes the per-stage map once per request, stashed asctx.metadata["stage_models"]; the seven per-call sites read their stage entry instead of the sharedmodel_id.transformand the bbox LLM matcher receive their model at bean construction (core/configuration.py) since those services are not passed a model per pipeline call.FLYDOCS_ESCALATION_MODEL/options.escalation.modelstill drives the judge-triggered re-run.pipeline.usage.by_modelalready splits usage per model id.Tests
tests/unit/test_per_stage_models.py): settings defaults + env binding, resolver fallback + precedence, end-to-end orchestrator routing (extract + judge receive their pinned models), and bean wiring for matcher/transformer.ruffclean; no newpyrighterrors (12 pre-existing on base).