-
Notifications
You must be signed in to change notification settings - Fork 121
[DO NOT MERGE] Rewrite the Vortex benchmarks website #6132
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: develop
Are you sure you want to change the base?
Conversation
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
CodSpeed Performance ReportMerging this PR will degrade performance by 46.94%Comparing
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ❌ | WallTime | u16_FoR[10M] |
6.2 µs | 11.6 µs | -46.94% |
| ⚡ | WallTime | u8_FoR[10M] |
71.7 µs | 11.2 µs | ×6.4 |
| ⚡ | Simulation | canonical_into_nullable[(10000, 100, 0.0)] |
4.9 ms | 4.1 ms | +19.79% |
| ⚡ | Simulation | canonical_into_nullable[(10000, 10, 0.0)] |
528.5 µs | 444.1 µs | +19.03% |
| ❌ | Simulation | into_canonical_non_nullable[(10000, 100, 0.0)] |
1.9 ms | 2.7 ms | -29.42% |
| ❌ | Simulation | into_canonical_non_nullable[(10000, 100, 0.1)] |
3.8 ms | 4.6 ms | -17.7% |
| ❌ | Simulation | into_canonical_non_nullable[(10000, 100, 0.01)] |
2.2 ms | 3 ms | -27.05% |
| ❌ | Simulation | into_canonical_nullable[(10000, 100, 0.0)] |
4.4 ms | 5.2 ms | -15.62% |
| ❌ | Simulation | canonical_into_non_nullable[(10000, 100, 0.0)] |
1.9 ms | 2.7 ms | -29.75% |
| ❌ | Simulation | canonical_into_non_nullable[(10000, 100, 0.1)] |
3.7 ms | 4.5 ms | -18.15% |
| ❌ | Simulation | canonical_into_non_nullable[(10000, 100, 0.01)] |
2.1 ms | 2.9 ms | -27.39% |
Footnotes
-
1290 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
|
|
||
| ```sql | ||
| -- clickbench table | ||
| CREATE TABLE clickbench ( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
would be nice to build in the cpu/io used. e.g. ec2 type and s3 or nvme.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That would need to be a separate chart(s) right? Otherwise I don't know how you would display all the information
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
A MINIMUM of 2 weeks to do this if someone (me) decides to do this, simply due to the amount of data migration, benchmarks + CI refactoring, and of course the website itself (actually getting it to work + making it look nice).
Maybe this is overkill but to be honest anything less than this that fulfills the same goals we want will still take forever so the opportunity cost is not actually that high imo.
You don't have to read the whole thing to get the idea, probably just the first 300 lines and then skim the rest of the document.