Skip to content

fix(providers): parse self-hosted context metadata#1227

Merged
Aaronontheweb merged 2 commits into
netclaw-dev:devfrom
Aaronontheweb:feature/openai-compatible-context-metadata
May 29, 2026
Merged

fix(providers): parse self-hosted context metadata#1227
Aaronontheweb merged 2 commits into
netclaw-dev:devfrom
Aaronontheweb:feature/openai-compatible-context-metadata

Conversation

@Aaronontheweb
Copy link
Copy Markdown
Collaborator

@Aaronontheweb Aaronontheweb commented May 29, 2026

Original Problem

OpenAI-compatible self-hosted providers could successfully list models, but Netclaw did not surface the model context window during discovery. In the CLI this showed up as a missing context window, and the runtime could fall back to the default 32K context value even when the backend reported a much larger effective context.

The issue was caused by metadata shape differences between self-hosted backends:

  • vLLM reports context length as max_model_len on /v1/models entries.
  • llama.cpp reports runtime context as meta.n_ctx on /v1/models entries.
  • Recent llama.cpp /props responses can report runtime context as default_generation_settings.n_ctx instead of the older default_generation_settings.params.n_ctx shape.

Netclaw already had separate backend strategies for vLLM and llama.cpp, but the llama.cpp strategy only understood the older n_ctx_train and nested /props shape. n_ctx_train is the training context, not necessarily the runtime-effective context when the server splits capacity across slots.

What Changed

  • OpenAI-compatible model discovery now reads context metadata for self-hosted backends without adding those backend-specific fields to the generic OpenAI-style parser.
  • vLLM discovery reads max_model_len.
  • llama.cpp discovery and capability resolution prefer runtime n_ctx and fall back to n_ctx_train only when runtime context is unavailable.
  • llama.cpp /props parsing supports both current top-level default_generation_settings.n_ctx and the older nested params.n_ctx shape.
  • Existing strategy separation is preserved: vLLM and llama.cpp still parse capabilities through their own backend-specific paths.

Tests

  • dotnet test "src/Netclaw.Daemon.Tests/Netclaw.Daemon.Tests.csproj" --filter "FullyQualifiedName~OpenAiCompatibleCapabilityResolverTests|FullyQualifiedName~LlamaCppBackendStrategyTests"
  • dotnet test "src/Netclaw.Configuration.Tests/Netclaw.Configuration.Tests.csproj" --filter "FullyQualifiedName~ProbeHelpersTests"
  • pwsh "./scripts/Add-FileHeaders.ps1" -Verify
  • git diff --check

@Aaronontheweb Aaronontheweb added providers Provider integrations and capability detection across OpenAI-compatible backends. bug Something isn't working labels May 29, 2026
@Aaronontheweb Aaronontheweb marked this pull request as ready for review May 29, 2026 19:14
@Aaronontheweb Aaronontheweb enabled auto-merge (squash) May 29, 2026 19:14
@Aaronontheweb Aaronontheweb merged commit ca207cc into netclaw-dev:dev May 29, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working providers Provider integrations and capability detection across OpenAI-compatible backends.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant