Overview
Docker Agent does not automatically forward GITHUB_TOKEN or GH_TOKEN into eval containers, even when defined in the shell or ~/.config/cagent/.env. This is intentional because GitHub tokens are broad credentials, but the current docs do not make it clear.
Add a warning to the evaluation and GitHub Copilot provider docs:
Eval cases run in isolated containers. For security reasons, GITHUB_TOKEN and GH_TOKEN are not forwarded automatically. Explicitly pass the required token by name:
docker agent eval agent.yaml ./evals -e GITHUB_TOKEN
When using a custom env file, both flags are required:
docker agent eval agent.yaml ./evals \
--env-from-file /path/to/secrets.env \
-e GITHUB_TOKEN
Also clarify that host-side judge validation may succeed while the evaluated agent fails if the token is not forwarded.
Motivation
Env vars for other providers are forwarded to eval containers automatically and it is confusing that this doesn't happen for github-copilot.
Use cases
No response
Proposed solution
No response
Alternatives
No response
Related issues
No response
Additional context
No response
Overview
Docker Agent does not automatically forward
GITHUB_TOKENorGH_TOKENinto eval containers, even when defined in the shell or~/.config/cagent/.env. This is intentional because GitHub tokens are broad credentials, but the current docs do not make it clear.Add a warning to the evaluation and GitHub Copilot provider docs:
Eval cases run in isolated containers. For security reasons, GITHUB_TOKEN and GH_TOKEN are not forwarded automatically. Explicitly pass the required token by name:
When using a custom env file, both flags are required:
Also clarify that host-side judge validation may succeed while the evaluated agent fails if the token is not forwarded.
Motivation
Env vars for other providers are forwarded to eval containers automatically and it is confusing that this doesn't happen for github-copilot.
Use cases
No response
Proposed solution
No response
Alternatives
No response
Related issues
No response
Additional context
No response