Problem
In CI environments (GitHub Actions), agentv eval YAML files use ${{ VAR_NAME }} template syntax to resolve variables like EVAL_CARGOWISE_REPO_PATH, COPILOT_GITHUB_TOKEN, etc. Currently these must be defined in a .env file — agentv doesn't fall back to reading process environment variables.
This means CI workflows need a dedicated "Configure environment" step that writes a .env file with values that are already available as environment variables:
- name: Configure environment
run: |
cat > .env <<EOF
COPILOT_GITHUB_TOKEN=${COPILOT_PAT}
EVAL_CARGOWISE_REPO_PATH=${WORKSPACE_ROOT}/CargoWise
...
EOFProposal
When resolving ${{ VAR_NAME }} in eval YAML, agentv should:
- Check
.env file first (current behavior)
- Fall back to
process.env[VAR_NAME] if not found in .env
This would eliminate the need for the .env file in CI, since GitHub Actions already sets secrets and vars as environment variables at the job/step level.
Context
This came up while building a PR review workflow (review.yml) and an evals workflow (run-evals.yml) for WiseTechGlobal/WTG.AI.Prompts. Both workflows currently need a configure-env composite action just to write a .env file from values already in the environment.
Problem
In CI environments (GitHub Actions), agentv eval YAML files use
${{ VAR_NAME }}template syntax to resolve variables likeEVAL_CARGOWISE_REPO_PATH,COPILOT_GITHUB_TOKEN, etc. Currently these must be defined in a.envfile — agentv doesn't fall back to reading process environment variables.This means CI workflows need a dedicated "Configure environment" step that writes a
.envfile with values that are already available as environment variables:Proposal
When resolving
${{ VAR_NAME }}in eval YAML, agentv should:.envfile first (current behavior)process.env[VAR_NAME]if not found in.envThis would eliminate the need for the
.envfile in CI, since GitHub Actions already sets secrets and vars as environment variables at the job/step level.Context
This came up while building a PR review workflow (
review.yml) and an evals workflow (run-evals.yml) for WiseTechGlobal/WTG.AI.Prompts. Both workflows currently need aconfigure-envcomposite action just to write a.envfile from values already in the environment.