Skip to content

Use the shared LLM abstraction for extraction #6

Description

@pudo

Kolkhoz currently calls the OpenAI Responses API directly for structured text-and-image extraction.

Adopt the shared provider abstraction proposed in operations#2512 so the extraction model and provider can be selected through configuration rather than code changes.

Kolkhoz requires:

  • Pydantic structured output.
  • Text and image inputs in one request.
  • Model and provider selection through one configuration value.
  • Reasoning-effort configuration where supported.
  • Usage and cost metadata stored with each extraction.
  • Provider-neutral error handling.
  • Test seams above the provider SDK.

Keep Kolkhoz’s extraction interface local to Kolkhoz. The shared abstraction should replace vendor calls, not own prompts, page preparation, screenshot tiling, or extraction schemas.

Questions

  • Should Kolkhoz wait for operations#2512 to establish LiteLLM as the stack-wide choice?
  • Does LiteLLM support the exact structured-output and multi-image behavior used by the current extractor?
  • Which provider-specific settings may pass through the abstraction?
  • How should unsupported options be handled when switching models?
  • Should extraction records store both the configured model string and the provider’s resolved model identifier?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions