Problem
Environment variable interpolation appears inconsistent across eval YAML fields.
In practice, values like ${VAR} / ${{ VAR }} are resolved in some contexts (for example, certain command/provider fields), but are treated literally in others such as:
workspace.repos[*].source.path
This blocks portable eval configs that need machine-specific local paths.
Repro
Given an eval file with:
workspace:
repos:
- path: ./RepoA
source:
type: local
path: ${{ REPO_A_PATH }}
and environment:
REPO_A_PATH=/abs/path/to/RepoA
running agentv eval ... attempts:
git clone --no-checkout ${{ REPO_A_PATH }} ...
instead of cloning /abs/path/to/RepoA.
Expected behavior
Interpolation should be supported uniformly across all string fields in eval/targets/workspace config (or at least all fields where string values are accepted), not only selected subsets.
At minimum, these should work consistently in all relevant config files:
Proposal
- Add a single, centralized interpolation pass for loaded config objects.
- Apply it recursively to all string-valued fields before validation/execution.
- Keep behavior backward compatible: strings without interpolation syntax remain unchanged.
- If a variable is missing, provide a clear error with key and field path.
Why this matters
- Portable eval definitions across dev machines and CI
- No hardcoded absolute paths in committed YAML
- Predictable behavior for users (same syntax everywhere)
Acceptance criteria
workspace.repos[*].source.path resolves env variables before clone.
- Interpolation works in nested objects/arrays of string fields.
- Both
${VAR} and ${{ VAR }} are supported (or one canonical form is documented + enforced with explicit error for the other).
- Missing variables produce actionable errors including field path.
- Existing configs without interpolation continue to work unchanged.
Problem
Environment variable interpolation appears inconsistent across eval YAML fields.
In practice, values like
${VAR}/${{ VAR }}are resolved in some contexts (for example, certain command/provider fields), but are treated literally in others such as:workspace.repos[*].source.pathThis blocks portable eval configs that need machine-specific local paths.
Repro
Given an eval file with:
and environment:
running
agentv eval ...attempts:git clone --no-checkout ${{ REPO_A_PATH }} ...instead of cloning
/abs/path/to/RepoA.Expected behavior
Interpolation should be supported uniformly across all string fields in eval/targets/workspace config (or at least all fields where string values are accepted), not only selected subsets.
At minimum, these should work consistently in all relevant config files:
${VAR}${{ VAR }}Proposal
Why this matters
Acceptance criteria
workspace.repos[*].source.pathresolves env variables before clone.${VAR}and${{ VAR }}are supported (or one canonical form is documented + enforced with explicit error for the other).