Skip to content

Conversation

@maksymar
Copy link
Contributor

@maksymar maksymar commented May 20, 2025

This PR introduces an optional bench_scope feature for stable-structures.

When enabled, it adds canbench_rs scopes to selected functions.
The feature is disabled by default to avoid overhead during benchmarking.
This allows benchmarking with or without internal measurement code.

CSV report in google sheets

@maksymar maksymar changed the title debug: base chore: base May 20, 2025
@maksymar maksymar changed the title chore: base chore: turn off btreemap bench scopes May 20, 2025
@github-actions
Copy link

github-actions bot commented May 20, 2025

canbench 🏋 (dir: .) 6dbf8cf 2025-05-20 13:20:21 UTC

./canbench_results.yml is up to date
📦 canbench_results_benchmark.csv available in artifacts

---------------------------------------------------

Summary:
  instructions:
    status:   Improvements detected! 🟢
    counts:   [total 304 | new 0 | improved 262 | regressed 0 | unchanged 42]
    change:   [min -303.39M | med -79.44M | max +914]
    change %: [min -21.06% | med -11.18% | max 0.00%]

  heap_increase:
    status:   Improvements detected! 🟢
    counts:   [total 304 | new 0 | improved 68 | regressed 0 | unchanged 236]
    change:   [min -53 | med 0 | max 0]
    change %: [min -100.00% | med 0.00% | max 0.00%]

  stable_memory_increase:
    status:   No significant changes detected 👍
    counts:   [total 304 | new 0 | improved 0 | regressed 0 | unchanged 304]
    change:   [min 0 | med 0 | max 0]
    change %: [min 0.00% | med 0.00% | max 0.00%]

---------------------------------------------------

Only significant changes:
| status | name                                         |     ins |  ins Δ% | HI |    HI Δ% | SMI |  SMI Δ% |
|--------|----------------------------------------------|---------|---------|----|----------|-----|---------|
|   -    | btreemap_v2_remove_vec_1024_128              |   5.02B |  -2.01% |  0 |    0.00% |   0 |   0.00% |
|   -    | btreemap_v2_contains_blob_512_128            |   2.62B |  -2.10% |  0 |    0.00% |   0 |   0.00% |
|   -    | btreemap_v2_get_blob_512_128                 |   2.66B |  -2.24% |  0 |    0.00% |   0 |   0.00% |
|   -    | btreemap_v2_get_vec_1024_128                 |   2.92B |  -2.41% |  0 |    0.00% |   0 |   0.00% |
|   -    | btreemap_v2_insert_blob_1024_128             |   5.10B |  -2.48% |  0 | -100.00% | 196 |   0.00% |
|   -    | btreemap_v2_mem_manager_get_blob512_u64      |   2.76B |  -2.79% |  0 |    0.00% |   0 |   0.00% |
|   -    | btreemap_v2_pop_last_blob_512_128            |   5.05B |  -2.82% |  0 |    0.00% |   0 |   0.00% |
|   -    | btreemap_v2_pop_first_blob_512_128           |   5.17B |  -2.97% |  0 |    0.00% |   0 |   0.00% |
|   -    | btreemap_v2_get_vec_512_128                  |   1.83B |  -3.02% |  0 |    0.00% |   0 |   0.00% |
|   -    | btreemap_v2_contains_vec_512_128             |   1.81B |  -3.04% |  0 |    0.00% |   0 |   0.00% |
|   -    | btreemap_v2_mem_manager_insert_blob512_u64   |   2.96B |  -3.08% |  0 | -100.00% |   0 |   0.00% |
|   -    | btreemap_v2_mem_manager_remove_blob512_u64   |   3.85B |  -3.15% |  0 |    0.00% |   0 |   0.00% |
|   -    | btreemap_v2_mem_manager_contains_blob512_u64 |   2.72B |  -3.21% |  0 |    0.00% |   0 |   0.00% |
|   -    | btreemap_v2_contains_vec_1024_128            |   2.88B |  -3.21% |  0 |    0.00% |   0 |   0.00% |
|   -    | btreemap_v2_contains_blob_256_128            |   1.48B |  -3.34% |  0 |    0.00% |   0 |   0.00% |
|   -    | btreemap_v2_remove_vec_512_128               |   3.27B |  -3.37% |  0 |    0.00% |   0 |   0.00% |
|   -    | btreemap_v2_get_blob_256_128                 |   1.51B |  -3.57% |  0 |    0.00% |   0 |   0.00% |
|   -    | btreemap_v2_remove_blob_512_128              |   3.60B |  -3.61% |  0 |    0.00% |   0 |   0.00% |
|   -    | btreemap_v2_insert_vec_1024_128              |   3.32B |  -3.94% |  0 |    0.00% | 193 |   0.00% |
|   -    | btreemap_v2_insert_blob_512_128              |   2.86B |  -3.94% |  0 | -100.00% | 111 |   0.00% |
|   -    | btreemap_v2_remove_vec_256_128               |   2.31B |  -4.06% |  0 |    0.00% |   0 |   0.00% |
|   -    | btreemap_v2_pop_first_blob_256_128           |   3.09B |  -4.15% |  0 |    0.00% |   0 |   0.00% |
|   -    | btreemap_v2_pop_first_vec_1024_128           |   5.76B |  -4.18% |  0 |    0.00% |   0 |   0.00% |
|   -    | btreemap_v2_get_vec_256_128                  |   1.23B |  -4.30% |  0 |    0.00% |   0 |   0.00% |
|   -    | btreemap_v2_pop_last_blob_256_128            |   3.02B |  -4.32% |  0 |    0.00% |   0 |   0.00% |
|  ...   | ... 212 rows omitted ...                     |         |         |    |          |     |         |
|   -    | btreemap_v2_insert_u64_u64                   | 431.51M | -15.80% |  0 | -100.00% |   6 |   0.00% |
|   -    | btreemap_v2_pop_last_blob_32_4               | 787.77M | -15.88% |  0 | -100.00% |   0 |   0.00% |
|   -    | btreemap_v2_get_blob_32_256                  | 351.58M | -15.90% |  0 |    0.00% |   0 |   0.00% |
|   -    | btreemap_v2_insert_u64_vec8                  | 429.93M | -16.02% |  0 | -100.00% |  21 |   0.00% |
|   -    | btreemap_v2_contains_blob8_u64               | 277.71M | -16.02% |  0 |    0.00% |   0 |   0.00% |
|   -    | btreemap_v2_contains_blob_8_128              | 268.36M | -16.07% |  0 |    0.00% |   0 |   0.00% |
|   -    | btreemap_v2_insert_u64_blob8                 | 422.66M | -16.13% |  0 | -100.00% |   5 |   0.00% |
|   -    | btreemap_v2_pop_first_u64_u64                | 717.36M | -16.33% |  0 | -100.00% |   0 |   0.00% |
|   -    | btreemap_v2_pop_first_u64_vec8               | 709.59M | -16.39% |  0 | -100.00% |   0 |   0.00% |
|   -    | btreemap_v2_pop_last_u64_u64                 | 704.61M | -16.41% |  0 | -100.00% |   0 |   0.00% |
|   -    | btreemap_v2_pop_first_u64_blob8              | 705.25M | -16.45% |  0 | -100.00% |   0 |   0.00% |
|   -    | btreemap_v2_get_u64_u64                      | 249.27M | -16.71% |  0 |    0.00% |   0 |   0.00% |
|   -    | btreemap_v2_get_blob_8_128                   | 281.64M | -17.14% |  0 |    0.00% |   0 |   0.00% |
|   -    | btreemap_v2_pop_last_u64_blob8               | 692.89M | -17.20% |  0 | -100.00% |   0 |   0.00% |
|   -    | btreemap_v2_pop_last_u64_vec8                | 696.00M | -17.23% |  0 | -100.00% |   0 |   0.00% |
|   -    | btreemap_v2_get_u64_vec8                     | 246.14M | -17.62% |  0 |    0.00% |   0 |   0.00% |
|   -    | btreemap_v2_get_u64_blob8                    | 245.36M | -17.78% |  0 |    0.00% |   0 |   0.00% |
|   -    | btreemap_v2_contains_u64_u64                 | 236.42M | -18.55% |  0 |    0.00% |   0 |   0.00% |
|   -    | btreemap_v2_scan_keys_rev_1k_0b              | 963.32K | -19.02% |  0 |    0.00% |   0 |   0.00% |
|   -    | btreemap_v2_contains_u64_vec8                | 234.90M | -19.20% |  0 |    0.00% |   0 |   0.00% |
|   -    | btreemap_v2_contains_u64_blob8               | 234.90M | -19.20% |  0 |    0.00% |   0 |   0.00% |
|   -    | btreemap_v2_scan_keys_1k_0b                  | 946.08K | -19.32% |  0 |    0.00% |   0 |   0.00% |
|   -    | btreemap_v2_range_value_sum_1k_0b            |  17.42K | -20.53% |  0 |    0.00% |   0 |   0.00% |
|   -    | btreemap_v2_range_key_sum_1k_0b              |  17.41K | -20.55% |  0 |    0.00% |   0 |   0.00% |
|   -    | btreemap_v2_range_count_1k_0b                |  16.87K | -21.06% |  0 |    0.00% |   0 |   0.00% |

---------------------------------------------------
CSV results saved to canbench_results.csv

@maksymar maksymar marked this pull request as ready for review May 20, 2025 12:29
@maksymar maksymar requested a review from a team as a code owner May 20, 2025 12:29
@maksymar maksymar requested a review from dsarlis May 20, 2025 12:33
Copy link
Member

@dsarlis dsarlis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would leave a comment why these are disabled for now so that others that stumble upon this code they're not wondering.

@maksymar maksymar changed the title chore: turn off btreemap bench scopes feat: add optional bench_scope feature for stable-structures May 20, 2025
@maksymar maksymar merged commit 5b0df69 into main May 20, 2025
10 checks passed
@maksymar maksymar deleted the maksym/base branch May 20, 2025 13:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants