Skip to content

Conversation

@maksymar
Copy link
Contributor

@maksymar maksymar commented May 27, 2025

This PR adds benchmarks for stable Vec to compare with other storage options.

It also introduces BoundedVec for use with stable Vec, which only supports bounded types. Now both bounded and unbounded vector types are available. Blob is not used for large data (e.g., 100 MB) due to canister heap limitations.

@github-actions
Copy link

github-actions bot commented May 27, 2025

canbench 🏋 (dir: ./benchmarks/memory_manager) 1e3dd6a 2025-05-27 16:39:45 UTC

./benchmarks/memory_manager/canbench_results.yml is up to date
📦 canbench_results_memory-manager.csv available in artifacts

---------------------------------------------------

Summary:
  instructions:
    status:   No significant changes detected 👍
    counts:   [total 3 | new 0 | improved 0 | regressed 0 | unchanged 3]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

  heap_increase:
    status:   No significant changes detected 👍
    counts:   [total 3 | new 0 | improved 0 | regressed 0 | unchanged 3]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

  stable_memory_increase:
    status:   No significant changes detected 👍
    counts:   [total 3 | new 0 | improved 0 | regressed 0 | unchanged 3]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

---------------------------------------------------
CSV results saved to canbench_results.csv

@github-actions
Copy link

github-actions bot commented May 27, 2025

canbench 🏋 (dir: ./benchmarks/vec) 1e3dd6a 2025-05-27 16:39:45 UTC

./benchmarks/vec/canbench_results.yml is up to date
📦 canbench_results_vec.csv available in artifacts

---------------------------------------------------

Summary:
  instructions:
    status:   No significant changes detected 👍
    counts:   [total 16 | new 0 | improved 0 | regressed 0 | unchanged 16]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

  heap_increase:
    status:   No significant changes detected 👍
    counts:   [total 16 | new 0 | improved 0 | regressed 0 | unchanged 16]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

  stable_memory_increase:
    status:   No significant changes detected 👍
    counts:   [total 16 | new 0 | improved 0 | regressed 0 | unchanged 16]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

---------------------------------------------------
CSV results saved to canbench_results.csv

@maksymar maksymar changed the title test: add benchmarks to compare stable-strucutre vec to stable memory test: add stable-structures Vec benchmarks May 27, 2025
@maksymar maksymar requested a review from Copilot May 27, 2025 11:41
Copilot

This comment was marked as outdated.

@github-actions
Copy link

github-actions bot commented May 27, 2025

canbench 🏋 (dir: ./benchmarks/compare) 1e3dd6a 2025-05-27 16:40:37 UTC

./benchmarks/compare/canbench_results.yml is up to date
📦 canbench_results_compare.csv available in artifacts

---------------------------------------------------

Summary:
  instructions:
    status:   No significant changes detected 👍
    counts:   [total 18 | new 6 | improved 0 | regressed 0 | unchanged 12]
    change:   [max 0 | p75 0 | median -517.50K | p25 -3.90M | min -95.26M]
    change %: [max 0.00% | p75 0.00% | median -0.04% | p25 -0.08% | min -0.11%]

  heap_increase:
    status:   No significant changes detected 👍
    counts:   [total 18 | new 6 | improved 0 | regressed 0 | unchanged 12]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

  stable_memory_increase:
    status:   No significant changes detected 👍
    counts:   [total 18 | new 6 | improved 0 | regressed 0 | unchanged 12]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

---------------------------------------------------

Only significant changes:
| status | name                | calls |   ins |  ins Δ% |    HI |  HI Δ% |   SMI |  SMI Δ% |
|--------|---------------------|-------|-------|---------|-------|--------|-------|---------|
|  new   | read_chunks_vec_1   |       | 1.36B |         | 3.20K |        | 1.67K |         |
|  new   | read_chunks_vec_1k  |       | 1.38B |         | 3.20K |        | 1.67K |         |
|  new   | read_chunks_vec_1m  |       | 4.75B |         | 3.78K |        | 1.67K |         |
|  new   | write_chunks_vec_1  |       | 1.26B |         | 3.20K |        | 1.67K |         |
|  new   | write_chunks_vec_1k |       | 1.27B |         | 3.20K |        | 1.67K |         |
|  new   | write_chunks_vec_1m |       | 3.74B |         | 3.78K |        | 1.67K |         |

ins = instructions, HI = heap_increase, SMI = stable_memory_increase, Δ% = percent change

---------------------------------------------------
CSV results saved to canbench_results.csv

@github-actions
Copy link

github-actions bot commented May 27, 2025

canbench 🏋 (dir: ./benchmarks/btreemap) 1e3dd6a 2025-05-27 16:41:31 UTC

./benchmarks/btreemap/canbench_results.yml is up to date
📦 canbench_results_btreemap.csv available in artifacts

---------------------------------------------------

Summary:
  instructions:
    status:   No significant changes detected 👍
    counts:   [total 285 | new 0 | improved 0 | regressed 0 | unchanged 285]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

  heap_increase:
    status:   No significant changes detected 👍
    counts:   [total 285 | new 0 | improved 0 | regressed 0 | unchanged 285]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

  stable_memory_increase:
    status:   No significant changes detected 👍
    counts:   [total 285 | new 0 | improved 0 | regressed 0 | unchanged 285]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

---------------------------------------------------
CSV results saved to canbench_results.csv

@maksymar maksymar changed the title test: add stable-structures Vec benchmarks test: add stable Vec benchmarks May 27, 2025
@maksymar maksymar marked this pull request as ready for review May 27, 2025 16:32
@maksymar maksymar requested a review from a team as a code owner May 27, 2025 16:32
@maksymar maksymar merged commit 7e884fb into main May 27, 2025
15 checks passed
@maksymar maksymar deleted the maksym/vec branch May 27, 2025 19:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants