Skip to content

Allow passing more env to docker run when running evals #1585

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
dgageot merged 2 commits into from
Feb 4, 2026
Merged

Allow passing more env to docker run when running evals #1585

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
47970e3
Allow passing more env to docker run when running evals
dgageot Feb 4, 2026
9ec4cc7
Make reviewer happy
dgageot Feb 4, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions cmd/root/eval.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ func newEvalCmd() *cobra.Command {
cmd.Flags().StringSliceVar(&flags.Only, "only", nil, "Only run evaluations with file names matching these patterns (can be specified multiple times)")
cmd.Flags().StringVar(&flags.BaseImage, "base-image", "", "Custom base Docker image for running evaluations")
cmd.Flags().BoolVar(&flags.KeepContainers, "keep-containers", false, "Keep containers after evaluation (don't use --rm)")
cmd.Flags().StringSliceVarP(&flags.EnvVars, "env", "e", nil, "Environment variables to pass to container (KEY or KEY=VALUE)")

return cmd
}
Expand Down
12 changes: 12 additions & 0 deletions pkg/evaluation/eval.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -372,6 +372,18 @@ func (r *Runner) runCagentInContainer(ctx context.Context, imageID, question str
}
}

// Pass additional environment variables specified via -e flag
// Format: KEY or KEY=VALUE
for _, entry := range r.EnvVars {
if key, val, hasValue := strings.Cut(entry, "="); hasValue && key != "" {
args = append(args, "-e", key)
env = append(env, key+"="+val)
} else if val, ok := r.runConfig.EnvProvider().Get(ctx, entry); ok && entry != "" {
args = append(args, "-e", entry)
env = append(env, entry+"="+val)
}
}

args = append(args, imageID, "/configs/"+agentFile, question)

cmd := exec.CommandContext(ctx, "docker", args...)
Expand Down
1 change: 1 addition & 0 deletions pkg/evaluation/types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,7 @@ type Config struct {
Only []string // Only run evaluations matching these patterns
BaseImage string // Custom base Docker image for running evaluations
KeepContainers bool // If true, don't remove containers after evaluation (skip --rm)
EnvVars []string // Environment variables to pass: KEY (value from env) or KEY=VALUE (explicit)
}

// Session helper functions
Expand Down