Skip to content

make output_dir configurable in evals#1029

Merged
mikasenghaas merged 3 commits intomainfrom
configurable-output-dir
Mar 19, 2026
Merged

make output_dir configurable in evals#1029
mikasenghaas merged 3 commits intomainfrom
configurable-output-dir

Conversation

@mikasenghaas
Copy link
Member

@mikasenghaas mikasenghaas commented Mar 18, 2026

Adds an output_dir field to EvalConfig that overrides the default ./outputs fallback in path resolution. Can be set via output_dir in eval TOML configs.

Description

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Test improvement

Testing

  • All existing tests pass when running uv run pytest locally.
  • New tests have been added to cover the changes

Checklist

  • My code follows the style guidelines of this project as outlined in AGENTS.md
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

Additional Notes


Note

Low Risk
Low risk: adds an optional output_dir override to evaluation path resolution without changing default behavior when unset. Main risk is misconfiguration leading to results/resume lookup writing to or scanning an unexpected directory.

Overview
Adds a new optional output_dir field to EvalConfig and allows it to be set from eval TOML/CLI configs.

Updates output path resolution in path_utils so eval/GEPA results (and --resume auto-detection via find_latest_incomplete_eval_results_path) use output_dir when provided, otherwise preserving the existing environment-local outputs/ or ./outputs fallback behavior.

Written by Cursor Bugbot for commit 756059d. This will update automatically on new commits. Configure here.

mikasenghaas and others added 3 commits March 18, 2026 16:15
Adds an `output_dir` field to EvalConfig that overrides the default
`./outputs` fallback in path resolution. Can be set via `output_dir`
in eval TOML configs.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@mikasenghaas mikasenghaas marked this pull request as ready for review March 18, 2026 11:36
@mikasenghaas mikasenghaas merged commit 8bf38c8 into main Mar 19, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants