Bug
`oc-bench init --providers local --no-detect --vllm-base-url X --vllm-model Y` produces a profile that is missing the model-specific parameter shaping defined in `openclaw_bench/providers/vllm.py`. For GPT-OSS specifically, the generated profile lacks `extra_body.reasoning_effort="low"`, which STATUS.md documents as required — without it, GPT-OSS burns the token budget on reasoning and returns empty/truncated content.
Reproduction (verified live in oc-stack on 2026-05-02)
```bash
python3 -m openclaw_bench init --providers local --no-detect \
--vllm-base-url http://10.68.198.1:8000/v1 \
--vllm-model gpt-oss-20b \
--bench-root /tmp/oc-bench-live-demo/bench-root \
--openclaw-profile bench-live-demo \
--gateway-port 19350 \
--no-validate
```
Resulting `agents.defaults.models["vllm/gpt-oss-20b"].params`:
```json
{
"chatTemplateKwargs": {"enable_thinking": false},
"maxTokens": 512
}
```
Expected (per `openclaw_bench/providers/vllm.py` `parameter_shaping`):
```json
{
"chatTemplateKwargs": {"enable_thinking": false},
"maxTokens": 512,
"extra_body": {"reasoning_effort": "low"}
}
```
Root cause
`init_quickstart` in `openclaw_bench/quickstart.py` only invokes `providers.vllm.parameter_shaping` indirectly via `detected_candidate` (the detection-driven path). When `--no-detect` is passed, `init_quickstart` falls into its env-var branch which calls `quickstart._vllm_provider_config(VllmEndpoint(...))` directly. That helper writes the provider config and the agent-default params, but does not look up model-specific tweaks by name.
The `enable_thinking=false` agent default is applied uniformly because it lives in `_openclaw_config` itself — but any model-name-conditional shaping (today: `reasoning_effort="low"` for `gpt-oss*`; tomorrow: anything Ollama / llama.cpp / LM Studio specific once those generators land) silently doesn't apply.
Expected fix
`init_quickstart`'s env-var path should construct a synthetic `ProviderCandidate` from the explicit `--vllm-*` flags and run it through the same `providers/vllm.py` `generate_route_config` + `parameter_shaping` pipeline that the detection path uses. That keeps a single source of truth for model shaping rules and makes adding future tweaks one-place-to-edit.
Alternatively: extract `parameter_shaping` into a pure "shape params for this model id" function and call it from `_openclaw_config` regardless of which path triggered.
Severity
P1. The deployment surface generates a config that fails silently on the model the README explicitly tells users to point at. The detected-Ollama silent-fallback bug from issue #1 is the same class of problem (init silently produces a broken artifact); these should likely be fixed together since they share the same root cause: `init_quickstart` doing config generation without going through the provider modules.
Discovered
While attempting to set up a live tier-small run against GPT-OSS to prove the bench produces real model evaluations end-to-end. Hand-patched the resulting `openclaw.json` to add the missing `extra_body` block before stopping the session; no live run was actually attempted with the buggy config.
Bug
`oc-bench init --providers local --no-detect --vllm-base-url X --vllm-model Y` produces a profile that is missing the model-specific parameter shaping defined in `openclaw_bench/providers/vllm.py`. For GPT-OSS specifically, the generated profile lacks `extra_body.reasoning_effort="low"`, which STATUS.md documents as required — without it, GPT-OSS burns the token budget on reasoning and returns empty/truncated content.
Reproduction (verified live in oc-stack on 2026-05-02)
```bash
python3 -m openclaw_bench init --providers local --no-detect \
--vllm-base-url http://10.68.198.1:8000/v1 \
--vllm-model gpt-oss-20b \
--bench-root /tmp/oc-bench-live-demo/bench-root \
--openclaw-profile bench-live-demo \
--gateway-port 19350 \
--no-validate
```
Resulting `agents.defaults.models["vllm/gpt-oss-20b"].params`:
```json
{
"chatTemplateKwargs": {"enable_thinking": false},
"maxTokens": 512
}
```
Expected (per `openclaw_bench/providers/vllm.py` `parameter_shaping`):
```json
{
"chatTemplateKwargs": {"enable_thinking": false},
"maxTokens": 512,
"extra_body": {"reasoning_effort": "low"}
}
```
Root cause
`init_quickstart` in `openclaw_bench/quickstart.py` only invokes `providers.vllm.parameter_shaping` indirectly via `detected_candidate` (the detection-driven path). When `--no-detect` is passed, `init_quickstart` falls into its env-var branch which calls `quickstart._vllm_provider_config(VllmEndpoint(...))` directly. That helper writes the provider config and the agent-default params, but does not look up model-specific tweaks by name.
The `enable_thinking=false` agent default is applied uniformly because it lives in `_openclaw_config` itself — but any model-name-conditional shaping (today: `reasoning_effort="low"` for `gpt-oss*`; tomorrow: anything Ollama / llama.cpp / LM Studio specific once those generators land) silently doesn't apply.
Expected fix
`init_quickstart`'s env-var path should construct a synthetic `ProviderCandidate` from the explicit `--vllm-*` flags and run it through the same `providers/vllm.py` `generate_route_config` + `parameter_shaping` pipeline that the detection path uses. That keeps a single source of truth for model shaping rules and makes adding future tweaks one-place-to-edit.
Alternatively: extract `parameter_shaping` into a pure "shape params for this model id" function and call it from `_openclaw_config` regardless of which path triggered.
Severity
P1. The deployment surface generates a config that fails silently on the model the README explicitly tells users to point at. The detected-Ollama silent-fallback bug from issue #1 is the same class of problem (init silently produces a broken artifact); these should likely be fixed together since they share the same root cause: `init_quickstart` doing config generation without going through the provider modules.
Discovered
While attempting to set up a live tier-small run against GPT-OSS to prove the bench produces real model evaluations end-to-end. Hand-patched the resulting `openclaw.json` to add the missing `extra_body` block before stopping the session; no live run was actually attempted with the buggy config.