Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve benchmarks #7

Merged
merged 2 commits into from Dec 20, 2023
Merged

Improve benchmarks #7

merged 2 commits into from Dec 20, 2023

Conversation

nvzqz
Copy link
Contributor

@nvzqz nvzqz commented Dec 20, 2023

  • Add allocation profiling. I have carefully optimized it so it should not have a noticeable footprint.
  • Remove setup cost from benchmarks. Previously benchmarks would include the cost of initializing a once_cell::sync::Lazy instance. This reduces benchmarked times by 100-400µs.

Example output:

simple                  fastest       │ slowest       │ median        │ mean          │ samples │ iters
├─ alpha_x_20000        1.975 ms      │ 2.168 ms      │ 1.99 ms       │ 1.995 ms      │ 100     │ 100
│                       alloc:        │               │               │               │         │
│                         12          │ 12            │ 12            │ 12            │         │
│                         960.8 KB    │ 960.8 KB      │ 960.8 KB      │ 960.8 KB      │         │
│                       dealloc:      │               │               │               │         │
│                         5           │ 5             │ 5             │ 5             │         │
│                         1.579 MB    │ 1.579 MB      │ 1.579 MB      │ 1.579 MB      │         │
│                       grow:         │               │               │               │         │
│                         18          │ 18            │ 18            │ 18            │         │
│                         1.578 MB    │ 1.578 MB      │ 1.578 MB      │ 1.578 MB      │         │
├─ alpha_x_20000_heavy  2.02 ms       │ 2.338 ms      │ 2.038 ms      │ 2.047 ms      │ 100     │ 100
│                       alloc:        │               │               │               │         │
│                         12          │ 12            │ 12            │ 12            │         │
│                         960.8 KB    │ 960.8 KB      │ 960.8 KB      │ 960.8 KB      │         │
│                       dealloc:      │               │               │               │         │
│                         5           │ 5             │ 5             │ 5             │         │
│                         1.579 MB    │ 1.579 MB      │ 1.579 MB      │ 1.579 MB      │         │
│                       grow:         │               │               │               │         │
│                         18          │ 18            │ 18            │ 18            │         │
│                         1.578 MB    │ 1.578 MB      │ 1.578 MB      │ 1.578 MB      │         │
├─ alpha_x_40000        4.013 ms      │ 4.396 ms      │ 4.044 ms      │ 4.061 ms      │ 100     │ 100
│                       alloc:        │               │               │               │         │
│                         12          │ 12            │ 12            │ 12            │         │
│                         1.92 MB     │ 1.92 MB       │ 1.92 MB       │ 1.92 MB       │         │
│                       dealloc:      │               │               │               │         │
│                         5           │ 5             │ 5             │ 5             │         │
│                         3.152 MB    │ 3.152 MB      │ 3.152 MB      │ 3.152 MB      │         │
│                       grow:         │               │               │               │         │
│                         19          │ 19            │ 19            │ 19            │         │
│                         3.151 MB    │ 3.151 MB      │ 3.151 MB      │ 3.151 MB      │         │
├─ alpha_x_40000_heavy  4.021 ms      │ 4.307 ms      │ 4.116 ms      │ 4.124 ms      │ 100     │ 100
│                       alloc:        │               │               │               │         │
│                         12          │ 12            │ 12            │ 12            │         │
│                         1.92 MB     │ 1.92 MB       │ 1.92 MB       │ 1.92 MB       │         │
│                       dealloc:      │               │               │               │         │
│                         5           │ 5             │ 5             │ 5             │         │
│                         3.152 MB    │ 3.152 MB      │ 3.152 MB      │ 3.152 MB      │         │
│                       grow:         │               │               │               │         │
│                         19          │ 19            │ 19            │ 19            │         │
│                         3.151 MB    │ 3.151 MB      │ 3.151 MB      │ 3.151 MB      │         │
├─ slice_word           2.275 ms      │ 2.707 ms      │ 2.292 ms      │ 2.299 ms      │ 100     │ 100
│                       alloc:        │               │               │               │         │
│                         12          │ 12            │ 12            │ 12            │         │
│                         960.9 KB    │ 960.9 KB      │ 960.9 KB      │ 960.9 KB      │         │
│                       dealloc:      │               │               │               │         │
│                         5           │ 5             │ 5             │ 5             │         │
│                         1.579 MB    │ 1.579 MB      │ 1.579 MB      │ 1.579 MB      │         │
│                       grow:         │               │               │               │         │
│                         18          │ 18            │ 18            │ 18            │         │
│                         1.578 MB    │ 1.578 MB      │ 1.578 MB      │ 1.578 MB      │         │
╰─ sqrt_pattern         1.598 ms      │ 1.752 ms      │ 1.609 ms      │ 1.615 ms      │ 100     │ 100
                        alloc:        │               │               │               │         │
                          7523        │ 7523          │ 7523          │ 7523          │         │
                          1.021 MB    │ 1.021 MB      │ 1.021 MB      │ 1.021 MB      │         │
                        dealloc:      │               │               │               │         │
                          8           │ 8             │ 8             │ 8             │         │
                          793 KB      │ 793 KB        │ 793 KB        │ 793 KB        │         │
                        grow:         │               │               │               │         │
                          17          │ 17            │ 17            │ 17            │         │
                          792 KB      │ 792 KB        │ 792 KB        │ 792 KB        │         │

Previously benchmarks would include the cost of initializing a
`once_cell::sync::Lazy` instance. This reduces benchmarked times by
100-400µs.
@nvzqz nvzqz changed the title Remove setup cost from benchmarks Improve benchmarks Dec 20, 2023
@OrangeX4 OrangeX4 merged commit 4b0c3e5 into mitex-rs:main Dec 20, 2023
@OrangeX4
Copy link
Member

Thanks!

@nvzqz nvzqz deleted the improve-benchmarks branch December 20, 2023 12:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants