fix(runtime): auto-derive AZURE_OPENAI_ENDPOINT from project endpoint#222
Merged
Conversation
CONTRIBUTING.md and the env-var docs both state that AZURE_OPENAI_ENDPOINT is auto-derived from the Foundry project endpoint when absent, but pipeline/runtime.py::_model_config only read AZURE_OPENAI_ENDPOINT with no fallback. A fresh workspace created by `agentops init` (which writes AZURE_AI_FOUNDRY_PROJECT_ENDPOINT but not AZURE_OPENAI_ENDPOINT) therefore always tripped 'Missing environment variables: AZURE_OPENAI_ENDPOINT' the first time AI-assisted evaluators tried to run locally. Adds derive_openai_endpoint_from_project() in utils/azure_endpoints.py that trims the trailing /api/projects/<name> segment to recover the AI Services account base URL. _model_config uses it as a fallback, restoring the documented behavior. The missing-deployment error also now nudges users toward the Foundry deployments list and the execution: cloud escape hatch so the next step is obvious without leaving the terminal. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
While recording the prompt-agent tutorial,
agentops eval runin localexecution mode failed on step 17 with::
The workspace had been created with
agentops initand had a validAZURE_AI_FOUNDRY_PROJECT_ENDPOINTin.azure/sandbox/.env— exactlythe case
CONTRIBUTING.mdand the env-var docs claim is supported via"auto-derived from the project endpoint when absent". The runtime never
honored that promise:
pipeline/runtime.py::_model_configonly read theexplicit env var with no fallback.
What changed
derive_openai_endpoint_from_project()inutils/azure_endpoints.py. The helper trims the trailing/api/projects/<name>segment from a Foundry project URL — coveringboth
services.ai.azure.comand the legacycognitiveservices.azure.comhosts — to recover the AI Servicesaccount base URL.
_model_confignow uses the derived value as a fallback wheneverAZURE_OPENAI_ENDPOINTis unset, finally matching the documentedbehavior.
AZURE_OPENAI_DEPLOYMENTis the only thing missing, the errormessage now points users at the deployment list in the Foundry portal
and mentions the
execution: cloudescape hatch inagentops.yamlso the next step is obvious without leaving theterminal.
Tests
New helper covered in
tests/unit/test_azure_endpoints.py(Foundryhost, legacy host, project name with hyphens, case-insensitive match,
pass-through cases, base URL that already carries the OpenAI suffix).
New
tests/unit/test_runtime_model_config.pycovers explicitendpoint wins, project-endpoint auto-derive, missing-deployment error
with hint, missing-both error, default and overridden API version.
Full suite::
python -m pytest tests/ -x -q
833 passed, 1 skipped in 96.06s
Follow-up (not in this PR)
The original tutorial failure escalated into a second issue: even after
unblocking the run via
execution: cloud, the Foundry grader returnedPermissionDeniedbecause creating a Foundry project does not grantthe caller the OpenAI data-plane RBAC on the AI Services account. Doctor
should detect that pre-run; tracking separately so this PR stays
focused on the documented-but-not-implemented auto-derive.