-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Adds callgrind as additional benchmarking harness. #12811
base: main
Are you sure you want to change the base?
Adds callgrind as additional benchmarking harness. #12811
Conversation
[ | ||
{ | ||
"_id": "6623c00aa16be249c931db66", | ||
"index": 0, | ||
"guid": "67b63043-00ae-4dfc-9082-f1a994af30f3", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What is the provenance of this example file? Just to make sure we don't run into any license issues with it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am unable to find the source of the data as it was a few weeks ago, I took it from some example JSON site as I thought it looked quite realistic to how people use JSON. I'm unaware if there were any licenses on the data.
@@ -232,6 +232,7 @@ pretty_assertions = { workspace = true } | |||
rstest = { workspace = true, default-features = false } | |||
serial_test = "3.1" | |||
tempfile = { workspace = true } | |||
iai-callgrind = "0.10.2" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
With the limits on only having dev-dependencies
, how will this impact the test build times?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
comparing the timings from cargo test --no-run --timings
main: 28.5s
this-PR: 29.5s
991daef
to
d2ccc71
Compare
Description
This is a test in addition to a time-based measurement we would also be able to use callgrid, It runs on simplified software CPU to calculate metrics of instructions, L1, L2, and RAM access. It is not 100% representative to a real CPU, but can most of the time be a very good heuristic. SQLite uses it as its default benchmarking tool.
I created a few bench cases that instead of running the internal library part of nushell, instead run the binary directly hopefully allowing better and simpler implementations of future benchmarks.
User-Facing Changes
Tests + Formatting
After Submitting