Costs-3 contract #3267

pavitthrap · 2022-08-29T15:40:18Z

Description

This PR adds new cost functions for the Clarity2 functions, and adds the costs-3 contract.

For the most part, the new costs contract does not alter old costs. However, as part of this benchmarking effort, I revamped the mocked data used (database and MARF accesses are now much more realistic). I also caught a few errors related to the computation of the input size (which only significantly altered one cost function). These changes led to a few cost changes. I provide a list of the altered costs below:

cost_stx_balance (increased 7x) - 99% of the runtime is in get_stx_balance_snapshot (DB access)
cost_stx_transfer (increased 7x) - 99% of the runtime is in get_stx_balance_snapshot (DB access)
cost_at_block (increased 10x) - using a real block now, which explains cost increase
cost_ast_parse (decreased 6x) (improvements in v2)
cost_concat (decreased 2x). (fixed input size computation)

We are now using version 0.21.0 of the Rust secp256k1 library. When we benchmarked the costs-2 contract, the version of this library was at 0.19.0. Due to improvements in these libraries, the runtimes for the following functions changed.

cost_secp256k1recover (decreased 2x)
cost_secp256k1verify (decreased 2x)
cost_poison_microblock (decreased 2x) - this function recovers the public key (so it essentially does the same operation as secp256k1recover)

Applicable issues

fixes [Clarity] need a .costs-3 contract for Stacks 2.1 and associated benchmarking #3115

Additional info (benefits, drawbacks, caveats)

Not currently using new cost functions in the code base as that requires additional code change of loading the costs-3 contract (Load costs-3 contract #3266) - I can also address that in this PR if people feel that is appropriate.

codecov · 2022-08-29T15:43:14Z

Codecov Report

Merging #3267 (4156983) into next (557f184) will decrease coverage by 0.03%.
The diff coverage is 19.47%.

@@            Coverage Diff             @@
##             next    #3267      +/-   ##
==========================================
- Coverage   29.60%   29.56%   -0.04%     
==========================================
  Files         279      284       +5     
  Lines      251259   258262    +7003     
==========================================
+ Hits        74378    76354    +1976     
- Misses     176881   181908    +5027

Impacted Files	Coverage Δ
clarity/src/vm/functions/assets.rs	`26.96% <0.00%> (ø)`
clarity/src/vm/functions/conversions.rs	`14.56% <0.00%> (-0.40%)`	⬇️
clarity/src/vm/functions/principals.rs	`0.00% <0.00%> (ø)`
clarity/src/vm/functions/sequences.rs	`50.98% <0.00%> (-2.77%)`	⬇️
clarity/src/vm/types/mod.rs	`50.41% <0.00%> (-0.28%)`	⬇️
src/burnchains/tests/affirmation.rs	`0.00% <0.00%> (ø)`
src/burnchains/tests/burnchain.rs	`0.00% <0.00%> (ø)`
src/burnchains/tests/db.rs	`0.00% <0.00%> (ø)`
src/burnchains/tests/mod.rs	`0.00% <0.00%> (ø)`
.../chainstate/burn/operations/leader_key_register.rs	`15.39% <0.00%> (-0.03%)`	⬇️
... and 92 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

jcnelson · 2022-08-29T19:08:48Z

cost_stx_balance (increased 7x) - 99% of the runtime is in get_stx_balance_snapshot (DB access)
cost_stx_transfer (increased 7x) - 99% of the runtime is in get_stx_balance_snapshot (DB access)
cost_at_block (increased 10x) - using a real block now, which explains cost increase

Wow! 🤯 Do we know if there's a way to speed these operations up? Curious if this is due to a natural average MARF key-path lengthening that will naturally occur at a logarithmic scale as more data gets added, or if this is due to an artifact / regression of the reference implementation.

jcnelson · 2022-09-12T15:56:01Z

Notes from the blockchain meeting:

This PR tests runtimes against a real DB, so the runtime cost includes the I/O costs of hitting a real MARF. Before, we had determined the read_count and write_count values with a real MARF, but determined the runtime values with a mocked MARF.

kantai · 2022-09-12T15:59:50Z

@pavitthrap raised a good point about the advantages of using the real backend versus the mocked backend -- this kind of relates to a question about the size of the MARF used in the benchmarks.

Do you know approximately how many keys were in the MARF, and how many MARF "blocks" there are in the database?

pavitthrap · 2022-09-19T16:27:18Z

Update: will replace the MARF with a smaller one (up to 10 blocks).
Will re-benchmark the functions with the new MARF once the PR for the function replace-at is done.

gregorycoppola

Update: will replace the MARF with a smaller one (up to 10 blocks). Will re-benchmark the functions with the new MARF once the PR for the function replace-at is done.

Is this updated to use the mocked out MARF?

Is there a link to the data?

jcnelson · 2022-10-17T15:36:30Z

Need a function for replace-at and the new analysis function for trait-checking

a3slade

LGTM

igorsyl · 2022-10-19T15:28:07Z

This PR tests runtimes against a real DB, so the runtime cost includes the I/O costs of hitting a real MARF. Before, we had determined the read_count and write_count values with a real MARF, but determined the runtime values with a mocked MARF.

@pavitthrap does the benchmark with real MARF use HDD or SSD?

igorsyl · 2022-10-19T15:36:07Z

It appears Load costs-3 contract #3266 is needed for 2.1. Could we include it to this PR?
Do we have a programmatic check that the ClarityCostFunction enum is complete and not missing clarity functions?

pavitthrap · 2022-11-04T14:48:56Z

This PR tests runtimes against a real DB, so the runtime cost includes the I/O costs of hitting a real MARF. Before, we had determined the read_count and write_count values with a real MARF, but determined the runtime values with a mocked MARF.

@pavitthrap does the benchmark with real MARF use HDD or SSD?

It uses a standard persistent disk, so HDD.

…ty function to its cost function

jcnelson · 2022-11-04T17:41:56Z

This should also address #3380 and #3381.

pavitthrap · 2022-11-04T22:47:26Z

It appears Load costs-3 contract #3266 is needed for 2.1. Could we include it to this PR?

Do we have a programmatic check that the ClarityCostFunction enum is complete and not missing clarity functions?

I'll be addressing #3380 and #3381, so I'll see about adding the contract when I do that.
I added a programmatic check now.

jcnelson · 2022-11-07T16:50:06Z

N.B. the benchmarks for I/O should use an SSD, because we don't want seeks and filesystem implementation details to interfere with benchmarks.

…t we actually need

…l work

…'s instantiation

jcnelson · 2022-11-15T04:58:34Z

Okay @gregorycoppola @obycode @igorsyl @a3slade, I've made a few optimizations to this PR so that the cost functions for the stx-* functions are now only about 20% slower instead of over 8x slower. The reason for this is because we have to determine the current burnchain tip height in order to calculate the consolidated balance, which incurs some extra CPU cycles and I/O for loading up the data from the headers DB and parsing it out.

Once #3389 merges, I'll go and add cost functions for the bitwise operators.

This PR also addresses #3266. costs-3 is loaded as part of the epoch 2.1 instantiation.

In addition, this PR addresses #3381, #3380, and #3311.

… -- it's inconsistent with the old behavior

jcnelson · 2022-11-15T15:36:57Z

Okay, had to revert the implementation of ClarityDB::get_current_burnchain_block_height() because the new one was buggy. But, the other optimizations can remain in place. The stx-* methods are now 2x slower than they were in 2.05, due to the additional burnchain DB MARF query.

obycode

LGTM

igorsyl · 2022-11-15T20:25:55Z

transition_empty_blocks test failing

…so we can use newer cost functions in later epochs without having to make multiple blocks

…d for `as-contract`)

… with pox-2 and costs-3, and will have epoch 2.1 set

pavitthrap added 4 commits August 26, 2022 17:26

Added new cost functions

bd5ae34

Added costs-3 contract

18a1bd6

Updated some cost functions

39fe25c

Reverted to using Unimplemented cost functions

629c89c

Updated secp related cost functions

6be0ffb

jcnelson added the stacks-2.1 label Sep 8, 2022

saralab requested review from gregorycoppola, jcnelson, igorsyl and a3slade October 13, 2022 17:33

gregorycoppola reviewed Oct 14, 2022

View reviewed changes

Merge branch 'next' into next-with-cost-fns

d3c3abe

jcnelson self-assigned this Oct 17, 2022

a3slade reviewed Oct 18, 2022

View reviewed changes

Merge branch 'next' into next-with-cost-fns

552c001

Added replace-at cost function; added programmatic check to map clari…

8167a14

…ty function to its cost function

jcnelson added 2 commits November 8, 2022 14:46

Merge branch 'next' into next-with-cost-fns

1efe5bc

Merge branch 'next' into next-with-cost-fns

bb487a6

jcnelson added 6 commits November 14, 2022 23:40

feat: instantiate costs-3 at the epoch 2.1 boundary

ac91240

fix: remove unused import

7ac66b8

chore: refactor code to only load column data from the headers DB tha…

5e8e5d0

…t we actually need

fix: instantiate the epoch-specific costs functions so cost tests wil…

9b327c0

…l work

doc: comment on the necessity of having a free cost tracker for pox-2…

cbebef5

…'s instantiation

fix: rusqlite can't take column names as input

53c0285

jcnelson requested a review from obycode November 15, 2022 04:55

fix: found a better runtime for get-burn-block-info

93cfef7

jcnelson requested review from a3slade and gregorycoppola November 15, 2022 05:01

jcnelson mentioned this pull request Nov 15, 2022

[Stacks 2.1] test new get_current_burnchain_block_height() function #3395

Closed

fix: rollback new get_current_burnchain_block_height() implementation…

1b57cd5

… -- it's inconsistent with the old behavior

jcnelson added 2 commits November 15, 2022 10:40

fix: new numbers with the original get_current_burnchain_block_height()

ec39f8e

fix: construct VRFSeed from block header's VRFProof

9c7d732

obycode approved these changes Nov 15, 2022

View reviewed changes

chore: remove unused macro definition

261fc3b

igorsyl approved these changes Nov 15, 2022

View reviewed changes

jcnelson added 6 commits November 15, 2022 18:08

chore: expose some boot code consts

3178d0d

chore: add ways to begin a genesis block in epoch 2.05 or epoch 2.1, …

a8f83f9

…so we can use newer cost functions in later epochs without having to make multiple blocks

fix: fix failing tests caused by unavailability of costs-3 (now neede…

d2d48c7

…d for `as-contract`)

fix: bring clarity_cli up to 2.1 rules -- the DB will be instantiated…

2b0dbbc

… with pox-2 and costs-3, and will have epoch 2.1 set

feat: helper to begin a test genesis block in epoch 2.1

9f4fba1

chore: log temporary MARF paths

4156983

jcnelson mentioned this pull request Nov 15, 2022

Cylewitruk feat/bitwise ops v2 #3397

Merged

jcnelson merged commit 66f2d13 into next Nov 16, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Costs-3 contract #3267

Costs-3 contract #3267

pavitthrap commented Aug 29, 2022 •

edited

Loading

codecov bot commented Aug 29, 2022 •

edited

Loading

jcnelson commented Aug 29, 2022

jcnelson commented Sep 12, 2022

kantai commented Sep 12, 2022

pavitthrap commented Sep 19, 2022

gregorycoppola left a comment

jcnelson commented Oct 17, 2022

a3slade left a comment

igorsyl commented Oct 19, 2022

igorsyl commented Oct 19, 2022 •

edited

Loading

pavitthrap commented Nov 4, 2022

jcnelson commented Nov 4, 2022

pavitthrap commented Nov 4, 2022

jcnelson commented Nov 7, 2022

jcnelson commented Nov 15, 2022 •

edited

Loading

jcnelson commented Nov 15, 2022

obycode left a comment

igorsyl commented Nov 15, 2022

Costs-3 contract #3267

Costs-3 contract #3267

Conversation

pavitthrap commented Aug 29, 2022 • edited Loading

Description

Applicable issues

Additional info (benefits, drawbacks, caveats)

codecov bot commented Aug 29, 2022 • edited Loading

Codecov Report

jcnelson commented Aug 29, 2022

jcnelson commented Sep 12, 2022

kantai commented Sep 12, 2022

pavitthrap commented Sep 19, 2022

gregorycoppola left a comment

Choose a reason for hiding this comment

jcnelson commented Oct 17, 2022

a3slade left a comment

Choose a reason for hiding this comment

igorsyl commented Oct 19, 2022

igorsyl commented Oct 19, 2022 • edited Loading

pavitthrap commented Nov 4, 2022

jcnelson commented Nov 4, 2022

pavitthrap commented Nov 4, 2022

jcnelson commented Nov 7, 2022

jcnelson commented Nov 15, 2022 • edited Loading

jcnelson commented Nov 15, 2022

obycode left a comment

Choose a reason for hiding this comment

igorsyl commented Nov 15, 2022

pavitthrap commented Aug 29, 2022 •

edited

Loading

codecov bot commented Aug 29, 2022 •

edited

Loading

igorsyl commented Oct 19, 2022 •

edited

Loading

jcnelson commented Nov 15, 2022 •

edited

Loading