Skip to content

Enrich discovered models with model info#3

Merged
yuhp merged 4 commits into
yuhp:mainfrom
xz-dev:litellm-model-info-discovery
May 25, 2026
Merged

Enrich discovered models with model info#3
yuhp merged 4 commits into
yuhp:mainfrom
xz-dev:litellm-model-info-discovery

Conversation

@xz-dev
Copy link
Copy Markdown
Contributor

@xz-dev xz-dev commented May 17, 2026

Summary

Fixes #2.

  • Query a LiteLLM-style model info endpoint by default, while gracefully continuing when unavailable.
  • Filter non-chat models by default when model_info.mode is present.
  • Skip embedding models detected from model ids so OpenCode never receives unsupported embedding modalities.
  • Populate OpenCode context limits from max_input_tokens, max_output_tokens, and max_tokens.
  • Populate reasoning and reasoning-effort variants from LiteLLM capability metadata.
  • Preserve provider model id casing while using case-insensitive metadata lookup fallback.
  • Document provider overrides for custom model info endpoints and disabling non-chat filtering.

Testing

  • npm run typecheck
  • npm test -- --run
  • /bin/opencode debug config --print-logs --log-level ERROR
  • /bin/opencode models litellm --print-logs --log-level ERROR
  • /bin/opencode run "Reply with OK only." --model litellm/deepseek-v3.2 --print-logs --log-level ERROR

Notes

npm run lint currently fails before linting because this repo uses ESLint 9 but only has .eslintrc.json, so ESLint cannot find eslint.config.js.

xz-dev added 3 commits May 17, 2026 14:14
Signed-off-by: Xiangzhe <xiangzhedev@gmail.com>
Signed-off-by: Xiangzhe <xiangzhedev@gmail.com>
Signed-off-by: Xiangzhe <xiangzhedev@gmail.com>
@xz-dev xz-dev force-pushed the litellm-model-info-discovery branch from 4aab662 to 952d18f Compare May 17, 2026 07:08
@xz-dev xz-dev marked this pull request as draft May 17, 2026 16:56
@xz-dev xz-dev marked this pull request as ready for review May 18, 2026 09:35
@yuhp
Copy link
Copy Markdown
Owner

yuhp commented May 22, 2026

Thanks for the implementation. I pushed a follow-up commit to keep the LiteLLM metadata enrichment while making the integration more provider-specific and extensible.

What Changed

  • Model info enrichment is now opt-in instead of querying /v1/model/info for every OpenAI-compatible provider by default.
  • Providers can enable it with both modelsDiscovery.modelInfoEndpoint and modelsDiscovery.modelInfoFormat:
{
  "modelsDiscovery": {
    "modelInfoEndpoint": "/v1/model/info",
    "modelInfoFormat": "litellm"
  }
}
  • The LiteLLM-specific parsing and enrichment logic was moved into src/utils/model-info/litellm.ts.
  • The main config enhancement flow now goes through a small model-info registry keyed by modelInfoFormat, so future formats can be added without growing provider-specific logic in enhance-config.ts.
  • The generic model info fetcher now returns raw data, and each format-specific enricher owns its response parsing.
  • Existing LiteLLM enrichment behavior is preserved for:
    • context/input/output limits
    • reasoning support
    • reasoning-effort variants
    • non-chat filtering
    • embedding model skipping

Why

/v1/model/info is a LiteLLM-specific extension rather than part of the generic OpenAI-compatible API contract. Querying it globally would add an extra request and send a non-standard endpoint hit to providers that are unlikely to support it.

Question

Could you check whether this still covers your LiteLLM usage?

In particular, is explicitly setting both of these acceptable for your setup?

{
  "modelInfoEndpoint": "/v1/model/info",
  "modelInfoFormat": "litellm"
}

@yuhp yuhp merged commit ae3cee5 into yuhp:main May 25, 2026
@yuhp
Copy link
Copy Markdown
Owner

yuhp commented May 25, 2026

I’m going to proceed with merging this since the follow-up commit keeps the LiteLLM enrichment behavior while making it opt-in and provider-specific.
The final behavior will be:

  • /v1/models remains the default discovery path for OpenAI-compatible providers.
  • LiteLLM model metadata enrichment is available when explicitly configured with:
    {
      "modelsDiscovery": {
        "modelInfoEndpoint": "/v1/model/info",
        "modelInfoFormat": "litellm"
      }
    }
  • Non-chat filtering, context limits, reasoning support, and reasoning-effort variants are preserved for LiteLLM metadata.
  • The model info enrichment path is now extensible through the modelInfoFormat registry, so we can add more provider-specific metadata formats later without changing the main discovery flow.
    If this does not cover your LiteLLM setup, we can adjust in a follow-up PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Enrich discovered models with model info and filter non-chat models

2 participants