Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add jsons benchmark #3881

Closed
wants to merge 2 commits into from
Closed

Add jsons benchmark #3881

wants to merge 2 commits into from

Conversation

brodmo
Copy link

@brodmo brodmo commented Mar 4, 2022

Not sure if I should have created an issue first but this is pretty straightforward. Please review

@brodmo
Copy link
Author

brodmo commented Mar 4, 2022

My benchmark results are somewhat different from those in the docs:

Package Version Relative Performance Mean validation time
attrs + cattrs 21.4.0 21.3μs
pydantic 1.9.0 1.2x slower 24.5μs
valideer 0.4.2 1.4x slower 28.9μs
marshmallow 3.14.1 3.3x slower 69.5μs
voluptuous 0.12.2 3.3x slower 69.9μs
trafaret 2.1.0 3.6x slower 77.1μs
jsons 1.6.1 10.9x slower 231.4μs
schematics 2.1.1 11.9x slower 253.2μs
django-rest-framework 3.13.1 14.5x slower 308.6μs
cerberus 1.3.4 27.8x slower 590.9μs

I've refrained from pushing them for this reason.

@samuelcolvin
Copy link
Member

Sorry but I can't accept this. See #1810 (comment)

I'll remove the benchmarks soon, they're wasting peoples time (like this, I'm so sorry). They need a lot of work and that's outside the bounds of pydantic.

@brodmo
Copy link
Author

brodmo commented Apr 1, 2022

I see, that's unfortunate. Looking further into it I think wyfo makes a lot of good points here #3264 and the benchmarks really ought to be removed, glad you agree. I originally became interested in the benchmarks because pydantic often didn't behave the way I wanted it to and its supposed performance edge and in particular the benchmarks were pointed to as the reason why, see #1157 (comment). I feel a little misled but the comment is admittedly pretty old and a lot of things seem to have changed since then (though the benchmarks page doesn't reflect that unfortunately)

@brodmo brodmo deleted the jsons-benchmark branch April 1, 2022 20:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants