feat(evals): Add run mode to API payload#249
Conversation
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughThe evaluation submission payload now always includes ChangesEvaluation run mode
Estimated code review effort: 2 (Simple) | ~5 minutes Possibly related PRs
Suggested reviewers: ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 ESLint
ESLint install timed out. The project may have too many dependencies for the sandbox. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
🎉 This PR is included in version 0.4.0-main.1 🎉 The release is available on GitHub release Your semantic-release bot 📦🚀 |
Summary
run_modewasbatch, so the frontend didn’t sendrun_modeexplicitly. We only sentrun_mode=fastwhen overriding the default behavior.fast,batchruns will no longer work unless the frontend explicitly sendsrun_mode=batch. This PR updates the frontend to send the appropriaterun_modewhen triggering batch evaluations.Checklist
Before submitting a pull request, please ensure that you mark these task.
npm run devandnpm run buildin the repository root and test.Summary by CodeRabbit