Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(clarity): Replace Vec::new() and vec![] #4408

Merged
merged 2 commits into from Feb 27, 2024

Conversation

jbencin
Copy link
Contributor

@jbencin jbencin commented Feb 21, 2024

Description

Replace instances of Vec::new and vec![] with vec::with_capacity() or .map()/.collect() whenever possible in Clarity code. This should help reduce the number of calls to alloc and realloc

Currently showing about a 2% improvement:

hyperfine -w 3 -r 10 "stacks-core/target/release/stacks-inspect.next replay-block /home/jbencin/data/next/ range 99990 100000" "stacks-core/target/release/stacks-inspect.vec-capacity replay-block /home/jbencin/data/next/ range 99990 100000"
Benchmark 1: stacks-core/target/release/stacks-inspect.next replay-block /home/jbencin/data/next/ range 99990 100000
  Time (mean ± σ):     10.565 s ±  0.039 s    [User: 10.070 s, System: 0.450 s]
  Range (min … max):   10.522 s … 10.655 s    10 runs
 
Benchmark 2: stacks-core/target/release/stacks-inspect.vec-capacity replay-block /home/jbencin/data/next/ range 99990 100000
  Time (mean ± σ):     10.408 s ±  0.032 s    [User: 9.877 s, System: 0.478 s]
  Range (min … max):   10.369 s … 10.491 s    10 runs
 
Summary
  stacks-core/target/release/stacks-inspect.vec-capacity replay-block /home/jbencin/data/next/ range 99990 100000 ran
    1.02 ± 0.00 times faster than stacks-core/target/release/stacks-inspect.next replay-block /home/jbencin/data/next/ range 99990 100000

Applicable issues

Additional info (benefits, drawbacks, caveats)

===> PLEASE READ BEFORE REVIEWING <===

There are several places where, the loop over Vec::push() would exit early in the event of an error. Since we now allocate before the loop using Vec::with_capacity(), this can lead to allocating more memory than would otherwise have been allocated in an error condition. This would only be a problem if user input could somehow force arbitrarily large allocations, resulting in a DoS attack by OOM. I don't think that is the case for any of the code I have modified, but keep it in mind while reviewing

Also there may be some places where pushing to the Vec is an uncommon event, or only done in case of an error. In these cases maybe pre-allocating memory does not make sense

Checklist

  • Test coverage for new or modified code paths
  • Changelog is updated
  • Required documentation changes (e.g., docs/rpc/openapi.yaml and rpc-endpoints.md for v2 endpoints, event-dispatcher.md for new events)
  • New clarity functions have corresponding PR in clarity-benchmarking repo
  • New integration test(s) added to bitcoin-tests.yml

Copy link

codecov bot commented Feb 21, 2024

Codecov Report

Attention: Patch coverage is 93.42105% with 5 lines in your changes are missing coverage. Please review.

Project coverage is 71.26%. Comparing base (ee1f7ed) to head (9ff45c3).

Additional details and impacted files
@@             Coverage Diff             @@
##             next    #4408       +/-   ##
===========================================
- Coverage   83.41%   71.26%   -12.15%     
===========================================
  Files         448      448               
  Lines      323911   323925       +14     
===========================================
- Hits       270182   230850    -39332     
- Misses      53729    93075    +39346     
Files Coverage Δ
.../src/vm/analysis/contract_interface_builder/mod.rs 90.80% <100.00%> (-4.42%) ⬇️
clarity/src/vm/analysis/type_checker/v2_05/mod.rs 90.85% <100.00%> (ø)
.../src/vm/analysis/type_checker/v2_05/natives/mod.rs 91.08% <100.00%> (-0.38%) ⬇️
...m/analysis/type_checker/v2_05/natives/sequences.rs 91.72% <100.00%> (-2.62%) ⬇️
clarity/src/vm/analysis/type_checker/v2_1/mod.rs 88.91% <100.00%> (-0.79%) ⬇️
...y/src/vm/analysis/type_checker/v2_1/natives/mod.rs 91.13% <100.00%> (ø)
...vm/analysis/type_checker/v2_1/natives/sequences.rs 92.64% <100.00%> (-3.00%) ⬇️
clarity/src/vm/ast/definition_sorter/mod.rs 97.80% <100.00%> (-0.27%) ⬇️
clarity/src/vm/ast/parser/v2/mod.rs 53.75% <100.00%> (-2.31%) ⬇️
clarity/src/vm/ast/sugar_expander/mod.rs 94.20% <100.00%> (ø)
... and 9 more

... and 218 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ee1f7ed...9ff45c3. Read the comment docs.

Copy link
Contributor

@obycode obycode left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@jbencin jbencin merged commit d72ad44 into stacks-network:next Feb 27, 2024
1 of 2 checks passed
@jbencin jbencin deleted the chore/vec-with-capacity branch February 27, 2024 22:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants