Proposed benchmark infra, adds collision detection benchmark, experimental performance improvements #1504
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a draft meant to accompany Discord conversation, but is not meant to be merged. Individual commits will, I hope, be useful and can be cherry-picked into separate pull requests.
The reports are confusing, I need to clean them up. But basically, every commit on this branch was checked out, and then the benchmark was run. The results show most recent commits first, older commits at the bottom. If you see a noticeable change in performance between one commit and the previous commit, then you know it made things faster/slower, and you can look at the changes within that specific commit. We can choose the commits that were most beneficial and extract them into a separate pull request.
If you try to run these benchmarks yourself, you'll need to tweak
bench.shand installhyperfine. When I come back to this, I'll add proper instructions.Over two benchmark runs, these are my results.
The big improvements came from:
python -m benchmarks.collisions.bench d93ba54ace7feb49cd949856a28d4e88977b1265python -m benchmarks.collisions.bench 055c42413039aa40c458e93663c60053f1dd66e9python -m benchmarks.collisions.bench 8d5d889a0f0eb9cfcf1d61122b902b61bfdc7d46python -m benchmarks.collisions.bench 39633527f4a2d3ab8454a74dd284a89b9ceb2befpython -m benchmarks.collisions.bench 68a5c838dad6afd5e996ee8ea3a7d2c96d243333python -m benchmarks.collisions.bench 5fcbcb68600d07b4957a0f797351ba87deb16620python -m benchmarks.collisions.bench d5cbb312c4c8a83388acfb96f83a683a527d87fcpython -m benchmarks.collisions.bench e1aebe50f5f79908f361ce587736e4b5c3d09001python -m benchmarks.collisions.bench 89d03fd156376d9c1ff8772ce11d3c062f74c1depython -m benchmarks.collisions.bench d93ba54ace7feb49cd949856a28d4e88977b1265python -m benchmarks.collisions.bench 055c42413039aa40c458e93663c60053f1dd66e9python -m benchmarks.collisions.bench 8d5d889a0f0eb9cfcf1d61122b902b61bfdc7d46python -m benchmarks.collisions.bench 39633527f4a2d3ab8454a74dd284a89b9ceb2befpython -m benchmarks.collisions.bench 68a5c838dad6afd5e996ee8ea3a7d2c96d243333python -m benchmarks.collisions.bench 5fcbcb68600d07b4957a0f797351ba87deb16620python -m benchmarks.collisions.bench d5cbb312c4c8a83388acfb96f83a683a527d87fcpython -m benchmarks.collisions.bench e1aebe50f5f79908f361ce587736e4b5c3d09001python -m benchmarks.collisions.bench 89d03fd156376d9c1ff8772ce11d3c062f74c1de