Skip to content

chore: Add more benchmarks#146

Merged
Abhijeet Prasad (AbhiPrasad) merged 2 commits intomainfrom
abhi-more-benchmarks
Mar 26, 2026
Merged

chore: Add more benchmarks#146
Abhijeet Prasad (AbhiPrasad) merged 2 commits intomainfrom
abhi-more-benchmarks

Conversation

@AbhiPrasad
Copy link
Copy Markdown
Member

Adding a few more benchmarks so I can burn anthropic tokens on pi auto research over night.

@viadezo1er
Copy link
Copy Markdown
Contributor

Not exactly this PR, but the previous performance improvement PR added sped up bt_json.py. However if a list subclass is given as argument this new fast version doesn't behave as expected:

import dataclasses, sys, os
from braintrust.bt_json import bt_safe_deep_copy

class TaggedList(list): pass

@dataclasses.dataclass
class Point: x: float; y: float

# Plain list: dataclass → dict. TaggedList: dataclass → str (regression).
print(bt_safe_deep_copy([Point(1, 2)]))            # [{'x': 1, 'y': 2}]  ✓
print(bt_safe_deep_copy(TaggedList([Point(1, 2)])))# ['Point(x=1, y=2)'] ✗

And it behaves correctly before the performance improvement change.
I think these functions are public and could be used by users?

-        elif tv is list or tv is tuple:
+        elif tv is list or tv is tuple or isinstance(v, (list, tuple)):

@AbhiPrasad
Copy link
Copy Markdown
Member Author

Not exactly this PR, but the previous performance improvement PR added sped up bt_json.py. However if a list subclass is given as argument this new fast version doesn't behave as expected:

yeah that's a regression. Feel free to open a gh issue, we can fix as a follow up

I think these functions are public and could be used by users?

nope, we set package_dir={"": "src"}, in setup.py so it shouldn't be exposed. We do expose our test files, but that is a separate issue.

batch_items(
_BATCH_STRINGS,
batch_max_num_bytes=_ITEM_SIZE * 15,
get_byte_size=lambda s: len(s.encode()),
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move the function definition outside the function body so function creating time isn't counted by pyperf?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Anyway good to merge this is just nitpicking

@AbhiPrasad Abhijeet Prasad (AbhiPrasad) merged commit 7a4328e into main Mar 26, 2026
34 checks passed
@AbhiPrasad Abhijeet Prasad (AbhiPrasad) deleted the abhi-more-benchmarks branch March 26, 2026 17:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants