Skip to content

fix(core): honor reset-only workspace hooks#1060

Merged
christso merged 1 commit intomainfrom
fix/1058-before-each-reset
Apr 12, 2026
Merged

fix(core): honor reset-only workspace hooks#1060
christso merged 1 commit intomainfrom
fix/1058-before-each-reset

Conversation

@christso
Copy link
Copy Markdown
Collaborator

Closes #1058

Summary

  • honor reset-only before_each hooks during eval execution instead of silently ignoring them
  • allow reset hooks to operate on the shared/static workspace root when no nested repos are declared
  • refresh the file-change baseline after before_each setup scripts so setup artifacts do not leak into file_changes

Red/Green E2E

Red on main:

  • Command: bun apps/cli/src/cli.ts eval run D:\GitHub\christso\agentv-e2e-1058\repro.eval.yaml --output D:\GitHub\christso\agentv-e2e-1058-out-main --workers 1
  • Evidence: D:\GitHub\christso\agentv-e2e-1058-out-main\repro\case-1\grading.json reported workspace_changes.diff_summary for setup.txt, even though that file was written by the before_each hook rather than the agent.

Green on this branch:

  • Command: bun apps/cli/dist/cli.js eval run D:\GitHub\christso\agentv-e2e-1058\repro.eval.yaml --output D:\GitHub\christso\agentv-e2e-1058-out-fix --workers 1
  • Evidence: D:\GitHub\christso\agentv-e2e-1058-out-fix\repro\case-1\grading.json no longer contains workspace_changes; the before_each setup file is excluded from file_changes as intended.

Verification

  • bun test packages/core/test/evaluation/orchestrator.test.ts packages/core/test/evaluation/validation/eval-validator.test.ts
  • bun run build
  • bun run test still has one unrelated timeout in pipeline grade > writes code_grader_results/<name>.json with score

@cloudflare-workers-and-pages
Copy link
Copy Markdown

Deploying agentv with  Cloudflare Pages  Cloudflare Pages

Latest commit: 543b2e8
Status: ✅  Deploy successful!
Preview URL: https://3c0de713.agentv.pages.dev
Branch Preview URL: https://fix-1058-before-each-reset.agentv.pages.dev

View logs

@christso christso marked this pull request as ready for review April 12, 2026 13:24
@christso christso merged commit ff03404 into main Apr 12, 2026
4 checks passed
@christso christso deleted the fix/1058-before-each-reset branch April 12, 2026 13:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix: apply before_each reset-only hooks for static workspace evals

1 participant