Context limit from Provider API is discarded, causing fallback to 128k for local models

[sgjohnson1981]

Describe the bug

In crates/goose/src/providers/litellm.rs (and potentially openai.rs), Goose fetches supported models from the proxy's API and successfully parses the max_input_tokens / context_length. However, it explicitly discards this context length when returning the list of supported models from the provider.

This causes Goose's ModelConfig to fall back to checking its internal static canonical_models.json. When it fails to find a match for a custom local model (e.g., local/unsloth/Qwen3-8B-Q4_K_M.gguf), it blindly defaults to its hardcoded DEFAULT_CONTEXT_LIMIT of 128,000.

As a result:

1. Goose can feed too many tokens to local models with smaller context windows (e.g., 8k tokens for Qwen), causing them to crash or throw errors during long sessions unless the user manually overrides GOOSE_CONTEXT_LIMIT.
2. Features that depend on the percentage of context used, such as automatic context compaction, will not work correctly because the perceived context limit (128k) is drastically different from the actual limit.

---

To Reproduce
Steps to reproduce the behavior:

1. Start a local LiteLLM proxy (or any OpenAI-compatible server) that returns custom local models where max_input_tokens is smaller than 128k.
2. Configure Goose to use the litellm provider.
3. Start Goose without setting GOOSE_CONTEXT_LIMIT.
4. Check the parsed context limit for the model (or try a long session).
5. See error: Goose assumes a 128k context window and either overflows the model or fails to trigger context compaction at the correct threshold.

---

Expected behavior
Goose should retain and respect the context_length (or max_input_tokens) provided by the /model/info (or /v1/models) endpoint of the provider, rather than discarding it and defaulting to 128,000.

---

Screenshots
N/A

---

Please provide the following information

• OS & Arch: Ubuntu 24.04.4 x86_64
• Interface: CLI
• Version: Latest (main branch)
• Extensions enabled: Developer, Tom, Skills, Todo, Summon, Apps, Extension Manager
• Provider & Model: LiteLLM / local .gguf models

---

Additional context
The issue is clearly visible in the fetch_supported_models implementation.

In crates/goose/src/providers/litellm.rs (around line 111):

let context_length = model_info["max_input_tokens"].as_u64().unwrap_or(128000) as usize;
let mut model_info_obj = ModelInfo::new(model_name, context_length);
model_info_obj.supports_cache_control = supports_cache_control;
models.push(model_info_obj);

// ... Later in fetch_supported_models:
let models = self.fetch_models().await.map_err(|e| { /* ... */ })?;

// BUG: Goose correctly fetched and parsed it, but throws it away here!
Ok(models.into_iter().map(|m| m.name).collect())

Because the Provider::fetch_supported_models trait signature expects Result<Vec<String>, ProviderError>, the rich metadata (including the context limit) is lost in transit.

[sgjohnson1981]

Correction: I was using the desktop GUI not the CLI, but it's also incorrect in the CLI.

[DOsinga]

Thanks for the detailed report — this is a real bug. The fetch_supported_models() trait method returns only model names (Vec<String>), so the context limit that fetch_models() correctly parses from the LiteLLM /model/info endpoint gets discarded.

A relatively contained fix would be to have the LiteLLM provider query /model/info during from_env() construction, find the matching model, and set context_limit on the ModelConfig when it isn't already overridden by GOOSE_CONTEXT_LIMIT. This avoids widening the Provider trait.

Snoozing this for a few days to give the community a chance to pick it up or react. If anyone wants to take a crack at a PR, the relevant code is in crates/goose/src/providers/litellm.rs — specifically from_env() and fetch_models().