feat(openai): forward prompt_cache_key for server-side prefix cache routing

[kevincodex1]

Summary

Send the session ID as OpenAI's prompt_cache_key on every completion request so consecutive turns route to a backend replica that already holds the conversation prefix in its prompt cache, instead of re-billing the full ~11k-token system+tools prefix each turn on OpenAI-compatible endpoints (which get no cache_control breakpoints, unlike Anthropic).

Keyless requests serialize byte-identically to before, and ZERO_DISABLE_PROMPT_CACHE_KEY=1 suppresses the field for strict endpoints that reject unknown parameters. The Codex (Responses API) flavor is deliberately untouched: the ChatGPT backend is a separate, stricter surface.

Checklist

Checklist

• The linked issue already has the issue-approved label.
  No linked issue — core-team change, same waiver as feat: agent quality, caching, retry, and tooling upgrades #506.
• go build ./..., go vet ./..., and go test ./... pass locally.
  Build and vet clean. Full test suite passes except the 6 pre-existing
  internal/cli failures caused by a local opengateway provider profile
  ("openai provider opengateway requires official baseURL") — byte-identical
  failure set on main, unrelated to this change.
• gofmt clean.
• Tests added/updated for the change (and run under -race where relevant).
  TestOpenAIRequestPromptCacheKey covers key serialization, omission for
  keyless requests, and the ZERO_DISABLE_PROMPT_CACHE_KEY kill switch.
  internal/providers/openai, internal/zeroruntime, and internal/agent
  all pass under -race.
• UI changes include screenshots or a short recording where possible.
  N/A — no UI changes (request serialization only).

Summary by CodeRabbit

• New Features

  • Added support for session-scoped prompt caching across chat requests, including retries and final-answer flows.
  • OpenAI requests now include cache-routing information when available.

• Bug Fixes

  • Improved consistency so repeated or retried requests can reuse the same caching behavior.
  • Added a safeguard to disable cache-key forwarding via an environment setting when needed.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: b8ddd4c3-f7ec-4e4b-82d0-d4d443ad49e2

📥 Commits

Reviewing files that changed from the base of the PR and between c2b714c and 24e6dbc.

📒 Files selected for processing (2)

• internal/providers/openai/provider.go
• internal/providers/openai/provider_test.go

🚧 Files skipped from review as they are similar to previous changes (2)

• internal/providers/openai/provider_test.go
• internal/providers/openai/provider.go

---

Walkthrough

Adds a PromptCacheKey field to CompletionRequest and the OpenAI request type, sets it from options.SessionID across agent loop completion call sites, and forwards it in the OpenAI provider unless disabled by ZERO_DISABLE_PROMPT_CACHE_KEY, with tests covering the mapping.

Changes

Prompt cache key plumbing

Layer / File(s)	Summary
PromptCacheKey field definitions internal/zeroruntime/types.go, internal/providers/openai/types.go	Adds PromptCacheKey to CompletionRequest and chatCompletionRequest, with comments and JSON tagging on the OpenAI request shape.
Agent loop session ID propagation internal/agent/loop.go	Sets PromptCacheKey: options.SessionID on the per-turn, retry, and final-answer completion requests.
OpenAI provider forwarding and tests internal/providers/openai/provider.go, internal/providers/openai/provider_test.go	Conditionally forwards PromptCacheKey as prompt_cache_key unless it is empty or ZERO_DISABLE_PROMPT_CACHE_KEY is truthy; tests cover forwarding, omission, and disablement behavior.

Estimated code review effort: 2 (Simple) | ~10 minutes

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main change: forwarding OpenAI prompt_cache_key for prefix-cache routing.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/openai-prompt-cache-key

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@internal/providers/openai/provider.go`:
- Around line 425-431: The kill switch check in the provider mapping currently
disables prompt-cache-key forwarding for any non-empty
ZERO_DISABLE_PROMPT_CACHE_KEY value, which is broader than intended. Update the
logic in the prompt cache key handling inside the OpenAI provider mapping so
only truthy enablement values (for example “1” or equivalent) disable
forwarding, and treat “0”, “false”, or other non-truthy values as no-op. Keep
the behavior localized to the request.PromptCacheKey mapping path and preserve
the existing documented env var behavior.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 09a46d6f-c204-4c70-963d-aed99b2ea8d2

📥 Commits

Reviewing files that changed from the base of the PR and between 30bff28 and c2b714c.

📒 Files selected for processing (5)

• internal/agent/loop.go
• internal/providers/openai/provider.go
• internal/providers/openai/provider_test.go
• internal/providers/openai/types.go
• internal/zeroruntime/types.go

[github-actions]

Zero automated PR review

Verdict: No blockers found

Blockers

• None found.

Validation

• [pass] Diff hygiene: git diff --check
• [pass] Tests: go test ./...
• [pass] Build: go run ./cmd/zero-release build
• [pass] Smoke build: go run ./cmd/zero-release smoke

Scope

Head: 24e6dbc966a5
Changed files (5): internal/agent/loop.go, internal/providers/openai/provider.go, internal/providers/openai/provider_test.go, internal/providers/openai/types.go, internal/zeroruntime/types.go

This deterministic review checks validation status and basic diff hygiene. A human reviewer still owns product judgment and design quality.

[Vasanthdev2004]

Read the whole thing — clean, well-scoped win. Forwarding the session ID as prompt_cache_key so OpenAI-compatible endpoints (which get no cache_control breakpoints) can route consecutive turns to a warm replica is the right lever for that ~11k-prefix re-bill, and it's done thoroughly: all five request-construction paths carry the key, omitempty keeps keyless requests byte-identical to before, the Codex/Responses surface is left alone, and the test covers serialization, keyless omission, and the kill switch under -race.

Privacy is fine — I checked, the session ID is an opaque zero_<timestamp>_<nano>_<counter> with no path or username in it, so it's a routing key, not user data. And the 6 local internal/cli failures are the opengateway-profile artifact you called out, not this change — CI is green across all nine checks, which confirms it.

One real thing worth fixing before it lands (CodeRabbit flagged this too): the kill switch reads os.Getenv(...) == "", so any non-empty value disables forwarding — including ZERO_DISABLE_PROMPT_CACHE_KEY=0 and =false, which read as "keep it on" but would turn it off. Low-probability footgun on its own, but it's inconsistent with how the other boolean flag parses (ZERO_FORMAT_ON_WRITE treats 0/false as unset). Match that truthy parse so the two behave the same.

One design call I'd flag rather than block: it's default-on for every openai-compatible endpoint, so a strict one that rejects unknown params would 400 until the user finds the kill switch. Rare, recoverable, and prompt_cache_key is a standard param now, so opt-out is a defensible default — just noting it's the spot a user could get surprised.

Approving. Fold in the env-var parse and it's a clean merge.