feat: Add per-input trialCount support to Eval()#1814
Merged
Luca Forstner (lforst) merged 3 commits intomainfrom Apr 14, 2026
Merged
feat: Add per-input trialCount support to Eval()#1814Luca Forstner (lforst) merged 3 commits intomainfrom
trialCount support to Eval()#1814Luca Forstner (lforst) merged 3 commits intomainfrom
Conversation
Co-authored-by: Daniel Flynn <djflynn15@gmail.com>
Abhijeet Prasad (AbhiPrasad)
approved these changes
Apr 14, 2026
Abhijeet Prasad (AbhiPrasad)
added a commit
to braintrustdata/braintrust-sdk-python
that referenced
this pull request
Apr 14, 2026
Allow each EvalCase to specify its own trial_count that overrides the global evaluator-level trial_count. Resolution order: datum.trial_count > evaluator.trial_count > 1 (default). python implementation of braintrustdata/braintrust-sdk-javascript#1814
Merged
Luca Forstner (lforst)
added a commit
that referenced
this pull request
Apr 16, 2026
This PR was opened by the [Changesets release](https://github.com/changesets/action) GitHub action. When you're ready to do a release, you can merge this and publish to npm yourself or [setup this action to publish automatically](https://github.com/changesets/action#with-publishing). If you're not ready to do a release yet, that's fine, whenever you add more changesets to main, this PR will be updated. # Releases ## braintrust@3.9.0 ### Minor Changes - [#1824](#1824) [`c980e07`](c980e07) Thanks [@lforst](https://github.com/lforst)! - feat: Add reranking instrumentation for AI SDK and Openrouter SDK - [#1738](#1738) [`25d25d5`](25d25d5) Thanks [@AbhiPrasad](https://github.com/AbhiPrasad)! - - feat: Add instrumentation for @huggingface/inference ([#1807](#1807)) - feat: Add cohere-ai instrumentation ([#1781](#1781)) - fix: Capture anthropic server tool use inputs for streaming APIs ([#1776](#1776)) - feat: Capture grounding metadata for Google GenAI ([#1773](#1773)) - fix(claude-agent-sdk): Don't drop tool spans for spawning subagents ([#1779](#1779)) - feat: Track server tool use metrics for anthropic SDK ([#1772](#1772)) - fix(openai): Collect logprob and refulsals output for streaming APIs ([#1774](#1774)) - perf: Remove zod from deepCopyEvent ([#1796](#1796)) - fix(test): Double timeout for slow OpenAI API tests ([#1794](#1794)) - feat(claude-agent-sdk): Improve task lifecycle and lifecycle details ([#1777](#1777)) - ci(deps): bump actions/github-script from 8.0.0 to 9.0.0 ([#1783](#1783)) - ci(deps): bump docker/setup-buildx-action from 3.12.0 to 4.0.0 ([#1782](#1782)) - chore: Don't use environment (ie. github deployments) for canary tests ([#1775](#1775)) - chore: Make dependabot less annoying ([#1778](#1778)) - fix(auto-instrumentation): Upgrade @apm-js-collab/code-transformer to v0.12.0 ([#1708](#1708)) - fix(auto-instrumentation): Use sync channel for AI SDK CJS streamText/streamObject in v4+ ([#1768](#1768)) - fix: Give AI SDK top-level api spans type function ([#1769](#1769)) - [#1814](#1814) [`d9b9923`](d9b9923) Thanks [@lforst](https://github.com/lforst)! - - feat: Add per-input trialCount support to Eval() - [#1821](#1821) [`1b5de11`](1b5de11) Thanks [@lforst](https://github.com/lforst)! - feat: Instrument Google GenAI embedContent for text ### Patch Changes - [#1825](#1825) [`f389036`](f389036) Thanks [@lforst](https://github.com/lforst)! - fix(ai-sdk): Restore prompt cache metrics - [#1813](#1813) [`2434a0e`](2434a0e) Thanks [@lforst](https://github.com/lforst)! - fix(openai-agents): End child spans on trace end - [#1836](#1836) [`5581357`](5581357) Thanks [@stretpjc](https://github.com/stretpjc)! - feat: Add x-bt-use-gateway header to allowed CORS headers --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Luca Forstner <luca.forstner@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Based on #1341