Fix infer CLI reliability gaps

[Spolen23]

Summary

Fix three openclaw infer reliability gaps that showed up in real use:

• isolate local infer model run calls onto their own infer session key instead of the shared main-session lane
• let web search fall through when an auto-selected provider returns a structured error payload like missing_brave_api_key
• align image describe parser/runtime behavior by registering --prompt, threading prompt overrides into media config, and passing agentDir
• add image autoPriority for openai-codex
• cover the behavior with targeted CLI/web/media tests

What changed

• src/cli/capability-cli.ts
  • local runModelRun(...) now sets sessionKey: agent:${agentId}:infer:model-run:${randomIdempotencyKey()}
  • image describe and image describe-many now accept --prompt <text>
  • prompt overrides are passed into tools.media.image.prompt and _requestPromptOverride
  • image describe now passes the default agentDir
• src/web-search/runtime.ts
  • when fallback is allowed and a provider returns an object with top-level string error, treat it as a failed attempt and continue fallback
• extensions/openai/media-understanding-provider.ts
• src/media-understanding/bundled-defaults.ts
  • add image autoPriority for openai-codex
• src/config/types.tools.ts
  • type the internal request-scoped media override fields used by the CLI/runtime wrappers
• tests updated in:
  • src/cli/capability-cli.test.ts
  • src/web-search/runtime.test.ts
  • src/media-understanding/defaults.test.ts

Verification

Targeted source tests passed on this branch:

npx pnpm exec node --no-maglev node_modules/vitest/vitest.mjs run --config vitest.cli.config.ts src/cli/capability-cli.test.ts
npx pnpm exec node --no-maglev node_modules/vitest/vitest.mjs run --config vitest.unit.config.ts src/web-search/runtime.test.ts
npx pnpm exec node --no-maglev node_modules/vitest/vitest.mjs run --config vitest.unit-fast.config.ts src/media-understanding/defaults.test.ts

Observed:

• CLI shard: 24 passed
• web-search shard: 15 passed
• media-defaults shard: 6 passed

Real-world verification

I also rebuilt and exercised the same logic on a live OpenClaw install while debugging infer on a host that had all three failures:

• openclaw infer model run --prompt "Reply with exactly: infer-ok" --local --json
• same local command twice concurrently
• openclaw infer model run --prompt "Reply with exactly: infer-ok" --gateway --json
• openclaw infer web search --query "OpenClaw infer CLI" --limit 3 --json
• openclaw infer web search --provider duckduckgo --query "OpenClaw infer CLI" --limit 3 --json
• openclaw infer image describe --file ... --json
• openclaw infer image describe --file ... --prompt "Describe this image in one sentence." --json

Observed:

• local and gateway model runs returned infer-ok
• concurrent local model runs both succeeded without colliding on the main-session lock
• default web search fell through to duckduckgo instead of dying on Brave missing-key
• image describe succeeded both with and without --prompt

[greptile-apps]

Greptile Summary

This PR fixes three openclaw infer reliability gaps: local model run calls now get their own unique session key to prevent concurrent call collisions, web search falls through to the next provider when a structured { error: string } payload is returned instead of dying, and image describe gains --prompt support with proper config threading and agentDir forwarding. The openai-codex provider also receives an autoPriority entry for image understanding in both the bundled defaults table and the extension provider export. All changes are well-covered by targeted tests.

Confidence Score: 5/5

Safe to merge — all remaining findings are P2 style suggestions.

The three bug fixes are logically correct and fully tested. The only finding is a duplicate import statement from the same module in runtime.ts, which is a pure style issue with no behavioral impact.

No files require special attention.

Vulnerabilities

No security concerns identified.

Comments Outside Diff (1)

1. src/web-search/runtime.ts, line 7-8 (link)

   Duplicate import from the same module

   Lines 7 and 8 both import from ../plugins/web-search-providers.runtime.js. These should be merged into a single statement to avoid the lint/format noise and the duplicated module specifier.

   Prompt To Fix With AI

   This is a comment left during a code review.
   Path: src/web-search/runtime.ts
   Line: 7-8
   
   Comment:
   **Duplicate import from the same module**
   
   Lines 7 and 8 both import from `../plugins/web-search-providers.runtime.js`. These should be merged into a single statement to avoid the lint/format noise and the duplicated module specifier.
   
   
   
   How can I resolve this? If you propose a fix, please make it concise.

Prompt To Fix All With AI

This is a comment left during a code review.
Path: src/web-search/runtime.ts
Line: 7-8

Comment:
**Duplicate import from the same module**

Lines 7 and 8 both import from `../plugins/web-search-providers.runtime.js`. These should be merged into a single statement to avoid the lint/format noise and the duplicated module specifier.

```suggestion
import {
  resolvePluginWebSearchProviders,
  resolveRuntimeWebSearchProviders,
} from "../plugins/web-search-providers.runtime.js";
```

How can I resolve this? If you propose a fix, please make it concise.

_{Reviews (1): Last reviewed commit: "Fix infer CLI durability gaps" | Re-trigger Greptile}

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b9fdef1f0e

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[chatgpt-codex-connector]

Limit structured-error fallback to availability/auth failures

In runWebSearch, any auto-selected provider response with a top-level string error now triggers fallback, but several providers use this same shape for user-input validation errors (for example invalid freshness/date combinations) rather than provider unavailability. In auto mode this can silently switch to a different provider and return results that ignore the original constraints, instead of surfacing an actionable validation error from the selected provider.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Ensure image --prompt overrides per-model prompts

The CLI --prompt wiring only injects tools.media.image.prompt, but image execution resolves prompt precedence as entry.prompt ?? config.prompt in the media runner. That means if a configured image model entry already has prompt, the new --prompt option is ignored, so users cannot reliably override prompts per request despite the command flag implying that behavior.

Useful? React with 👍 / 👎.

[clawsweeper]

Codex automated review: keeping this open.

Keep this PR open. Current main has superseded the local infer model-run session collision part by using the lean local completion path and already passes agentDir for image description, but main still does not implement the remaining user-visible fixes: image describe metadata advertises --prompt while the command/runtime do not accept or thread it, web_search still returns structured provider error payloads as successful results, and openai-codex still lacks image media-understanding autoPriority in current manifest/runtime metadata.

Best possible solution:

Keep this PR open as an implementation candidate. Rebase it onto current main, drop or adapt the obsolete local session-key change, register and thread image describe --prompt and image describe-many --prompt, make request-scoped image prompt overrides win over model-entry prompts, narrow web-search structured fallback to provider availability/auth failures, add openai-codex image autoPriority consistently in current manifest and runtime provider metadata if it should participate in auto fallback, then rerun the targeted CLI, web-search, and media-understanding tests before maintainer review.

What I checked:

• local_model_run_collision_superseded: Current main local runModelRun prepares the selected agent model and calls completeWithPreparedSimpleCompletionModel directly instead of opening a chat-agent session lane, so the PR's session-key change is obsolete on this part. (src/cli/capability-cli.ts:587, 5828dcdb05aa)
• local_model_run_test_coverage: The current CLI test asserts local model probes use the lean completion path and do not call the Gateway. (src/cli/capability-cli.test.ts:395, 5828dcdb05aa)
• image_describe_prompt_gap_remains: Capability metadata lists --prompt for image.describe and image.describe-many, but the actual Commander registrations only expose --file, --model, and --json; runImageDescribe also has no prompt parameter. (src/cli/capability-cli.ts:199, 5828dcdb05aa)
• image_agent_dir_already_on_main: Current runImageDescribe resolves the default agentDir and passes it into both automatic and explicit provider/model image description paths, so that PR sub-change is already present on main. (src/cli/capability-cli.ts:860, 5828dcdb05aa)
• structured_web_search_fallback_gap_remains: runWebSearch still returns await definition.execute(params.args) directly for the first candidate. It falls back on missing tool definitions and thrown errors, but not on structured result payloads such as { error: 'missing_brave_api_key' }. (src/web-search/runtime.ts:350, 5828dcdb05aa)
• web_search_fallback_needs_narrowing: Brave uses the same top-level { error, message, docs } shape for missing credentials and for caller/actionable validation errors such as invalid language, unsupported freshness, and conflicting time filters, so the PR's broad fallback remains behaviorally risky. (extensions/brave/src/brave-web-search-provider.runtime.ts:53, 5828dcdb05aa)

Remaining risk / open question:

• Merging the PR unchanged could silently fall through on Brave validation/user-input errors because missing credentials and invalid constraints share a top-level structured error shape.
• The media defaults portion must be rebased onto manifest-backed metadata; the old bundled defaults file no longer exists on current main.
• The image prompt implementation should make request-scoped CLI prompts override per-entry prompts, not only set the default media prompt.

Codex Review notes: model gpt-5.5, reasoning high; reviewed against 5828dcdb05aa.

[Takhoffman]

Merged in 12aaef9 after representative verification with pnpm install --frozen-lockfile, git diff --check, focused media/web-search tests, and pnpm tsgo:test:src. I also rebased the branch onto current origin/main, addressed the review findings around structured web-search fallback and request-scoped image prompts, and added the Unreleased changelog entry.