fix(harvey): map provider env to the OpenAI-compatible adapter#888
Merged
Conversation
harvey-lab-harness ended every proxied run on turn 0 with zero LLM activity (suspected_api_error). Root cause: its env_mapping was empty, but Harvey LAB's OpenAI-compatible adapter (the one that serves every proxied provider — deepseek, the azure gpt-5.4-mini gateway, the benchflow-* proxy aliases) reads OPENAI_BASE_URL / OPENAI_API_KEY, which only exist as BENCHFLOW_PROVIDER_* on the proxy path. With neither set, the adapter hit api.openai.com unauthenticated and the harness loop made zero calls. Map BENCHFLOW_PROVIDER_BASE_URL -> OPENAI_BASE_URL and BENCHFLOW_PROVIDER_API_KEY -> OPENAI_API_KEY so the adapter points at the gateway. Direct ANTHROPIC/GEMINI runs are unaffected (those adapters read ANTHROPIC_API_KEY / GOOGLE_API_KEY directly and ignore OPENAI_*). Verified on daytona (deepseek): harvey now routes and issues requests (total_requests >= 1) — previously total_requests == 0. The remaining HTTP 500s are transient upstream provider errors (benchflow retries them), not the harness defect this fixes.
Yiminnn
temporarily deployed
to
pypi-internal-preview
July 3, 2026 23:13 — with 
GitHub Actions
Inactive
GitHub Actions
Inactive
Contributor
Author
|
The failing |
Yiminnn
added a commit
to benchflow-ai/agents
that referenced
this pull request
Jul 4, 2026
Mirrors benchflow-ai/benchflow#888, which adds the OpenAI-compatible env_mapping to harvey's core AgentConfig (so its adapter reaches the gateway instead of ending turn 0 with zero activity). env_mapping is a data field the agents repo owns, so the manifest must carry the same mapping or the byte-identical parity gate fails. Verified: test_manifest_parity.py::test_manifest_byte_identical_to_core passes for harvey-lab-harness with this manifest + the core change. Co-authored-by: symphony-bot <symphony@benchflow.ai>
Yiminnn
temporarily deployed
to
pypi-internal-preview
July 4, 2026 01:59 — with
GitHub Actions
Inactive
GitHub Actions
Inactive
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
harvey-lab-harnessended every proxied run on turn 0 with zero LLM activity (suspected_api_error). Root cause: itsenv_mappingwas empty, but Harvey LAB's OpenAI-compatible adapter — the one that serves every proxied provider (deepseek, the azure gpt-5.4-mini gateway, thebenchflow-*proxy aliases) — readsOPENAI_BASE_URL/OPENAI_API_KEY, which only exist asBENCHFLOW_PROVIDER_*on the proxy path. With neither set, the adapter hitapi.openai.comunauthenticated and the harness loop made zero calls.Map
BENCHFLOW_PROVIDER_BASE_URL → OPENAI_BASE_URLandBENCHFLOW_PROVIDER_API_KEY → OPENAI_API_KEYso the adapter points at the gateway. DirectANTHROPIC/GEMINIruns are unaffected (those adapters readANTHROPIC_API_KEY/GOOGLE_API_KEYdirectly and ignoreOPENAI_*).Verification (daytona, deepseek)
harvey now routes and issues requests (
total_requests >= 1) — previouslytotal_requests == 0/ zero activity. This corrects the long-standing "harvey is a harness-level zero-activity defect" classification: it was an unwired adapter, not a broken harness. The remaining HTTP 500s in the run are transient upstream provider errors (benchflow retries them), orthogonal to this fix.Tests
test_registry_invariants.py: newtest_harvey_maps_provider_env_to_openai_compatible_adapterguards the mapping. 184 pass; ruff clean.