Skip to content

Conversation

@jberthold
Copy link
Member

@jberthold jberthold commented Jul 22, 2022

Relates to #3119

Scope:

A more integrated version of leavesM along its use in proveClaim, describing different return values as data instead of using exceptions.
We could further monomorphise it to use the simplifier monad (including logging) at the bottom of the monad stack.

Estimate:


Review checklist

The author performs the actions on the checklist. The reviewer evaluates the work and checks the boxes as they are completed.

  • Summary. Write a summary of the changes. Explain what you did to fix the issue, and why you did it. Present the changes in a logical order. Instead of writing a summary in the pull request, you may push a clean Git history.
  • Documentation. Write documentation for new functions. Update documentation for functions that changed, or complete documentation where it is missing.
  • Tests. Write unit tests for every change. Write the unit tests that were missing before the changes. Include any examples from the reported issue as integration tests.
  • Clean up. The changes are already clean. Clean up anything near the changes that you noticed while working. This does not mean only spatially near the changes, but logically near: any code that interacts with the changes!

jberthold and others added 23 commits July 20, 2022 10:59
…red-leaf-count-refactor-leavesM

* Simplifier as a base monad
Semantics of TraversalResult (failure cases):
- When `GotStuck` is returned, the returned results are considered
  stuck and thrown as an exception;
- when `Aborted` is returned, the returned results are _analysed_
  and their _next_ states (to enqueue) are considered stuck and thrown.
@jberthold jberthold changed the title 3119 output unexplored leaf count refactor leaves m EXPERIMENT: refactor graph traversal to replace leavesM Jul 22, 2022
@jberthold
Copy link
Member Author

Performance gain is currently around 1% across the kevm proofs.

Test newprovecode-2-newprovecode-2 time master20220722-master20220722 time (newprovecode-2-newprovecode-2/master20220722-master20220722) time
specs/benchmarks/address00-spec.k 86.72 89.4 0.9700223713646532
specs/benchmarks/staticloop00-a0lt10-spec.k 157.2 161.32 0.9744606992313414
specs/examples/solidity-code-spec.md 118.41 121.39 0.9754510256199027
erc20/hkg/transfer-success-2-spec.k 151.45 154.8 0.9783591731266148
specs/benchmarks/staticarray00-spec.k 123.5 126.06 0.9796922100587022
specs/benchmarks/requires01-a0gt0-spec.k 91.09 92.69 0.9827381594562521
specs/functional/merkle-spec.k 17.67 17.98 0.9827586206896552
specs/benchmarks/storagevar01-spec.k 73.9 75.18 0.9829741952646981
specs/mcd/vow-flog-fail-rough-spec.k 258.5 262.91 0.9832261990795328
specs/mcd/cat-exhaustiveness-spec.k 107.31 109.07 0.983863573851655
specs/mcd/vat-subui-fail-rough-spec.k 190.94 194.04 0.9840239125953412
specs/mcd/vow-fess-fail-rough-spec.k 281.64 286.1 0.9844110450891296
specs/examples/sum-to-n-spec.k 19.45 19.75 0.9848101265822784
specs/benchmarks/storagevar03-spec.k 55.18 56.01 0.9851812176397072
erc20/hkg/allowance-spec.k 89.5 90.76 0.9861172322609079
specs/benchmarks/ecrecover00-sigvalid-spec.k 240.58 243.93 0.9862665518796376
specs/mcd/dsvalue-read-pass-summarize-spec.k 90.69 91.95 0.9862969004893963
specs/bihu/forwardToHotWallet-failure-2-spec.k 200.25 203.02 0.9863560240370406
specs/mcd/flipper-ttl-pass-spec.k 117.37 118.99 0.9863854105387008
specs/benchmarks/overflow00-overflow-spec.k 88.6 89.79 0.9867468537699075
erc20/hkg/transfer-success-1-spec.k 256.88 260.27 0.9869750643562455
erc20/hkg/transferFrom-success-2-spec.k 365.28 369.9 0.9875101378751013
erc20/ds/balanceOf-spec.k 130.34 131.98 0.9875738748295197
specs/benchmarks/encodepacked-keccak01-spec.k 124.85 126.42 0.9875810789432051
specs/bihu/forwardToHotWallet-failure-4-spec.k 349.8 354.17 0.987661292599599
erc20/ds/allowance-spec.k 198.29 200.68 0.9880904923260913
erc20/ds/transfer-failure-1-c-spec.k 270.67 273.72 0.9888572263627063
specs/benchmarks/storagevar00-spec.k 65.89 66.61 0.9891908121903619
erc20/ds/approve-failure-spec.k 82.91 83.81 0.9892614246509962
erc20/ds/transfer-failure-2-b-spec.k 185.77 187.78 0.9892959846629035
specs/bihu/forwardToHotWallet-failure-3-spec.k 219.44 221.81 0.9893151796582661
erc20/hkg/transferFrom-failure-2-spec.k 278.88 281.89 0.9893220759870872
specs/mcd/vat-addui-fail-rough-spec.k 191.55 193.61 0.9893600537162337
specs/mcd/dsvalue-read-pass-spec.k 194.39 196.4 0.9897657841140528
specs/mcd/cat-file-addr-pass-rough-spec.k 167.8 169.46 0.9902041779771038
specs/mcd/dstoken-burn-self-fail-rough-spec.k 1324.3 1337.08 0.990441858377958
specs/benchmarks/storagevar02-nooverflow-spec.k 82.1 82.89 0.9904692966582217
specs/benchmarks/encode-keccak00-spec.k 242.51 244.8 0.9906454248366012
erc20/ds/transferFrom-failure-2-c-spec.k 211.39 213.23 0.9913708202410543
specs/mcd/vat-subui-pass-spec.k 266.4 268.7 0.9914402679568292
erc20/hkg/transfer-failure-2-spec.k 123.01 124.07 0.9914564358829694
specs/mcd/vat-mului-pass-spec.k 197.81 199.49 0.9915785252393603
erc20/ds/approve-success-spec.k 351.78 354.75 0.9916279069767441
specs/mcd/vat-deny-diff-fail-rough-spec.k 307.23 309.7 0.9920245398773007
specs/mcd/vat-move-diff-rough-spec.k 540.55 544.83 0.9921443386010314
specs/opcodes/evm-optimizations-spec.md 15.26 15.38 0.9921976592977892
erc20/hkg/totalSupply-spec.k 32.17 32.42 0.9922887106724244
specs/mcd/vat-subui-pass-rough-spec.k 259.93 261.88 0.9925538414541012
specs/benchmarks/storagevar02-overflow-spec.k 78.83 79.42 0.9925711407705867
erc20/hkg/balanceOf-spec.k 68.12 68.62 0.992713494607986
specs/benchmarks/requires01-a0le0-spec.k 67.97 68.45 0.9929875821767713
erc20/hkg/transfer-failure-1-spec.k 210.33 211.79 0.993106378960291
specs/mcd/dai-adduu-fail-rough-spec.k 59.57 59.96 0.9934956637758505
erc20/hkg/approve-spec.k 116.19 116.95 0.9935014963659683
erc20/ds/transfer-failure-2-a-spec.k 363.84 366.2 0.9935554341889677
specs/mcd/dai-symbol-pass-spec.k 156.01 157.01 0.9936309789185402
specs/mcd/end-subuu-pass-spec.k 111.18 111.84 0.994098712446352
specs/benchmarks/ecrecover00-siginvalid-spec.k 190.68 191.65 0.9949386903208974
specs/benchmarks/keccak00-spec.k 156.0 156.74 0.9952788056654331
specs/mcd/dsvalue-peek-pass-rough-spec.k 186.54 187.31 0.9958891676899257
specs/mcd/flipper-tau-pass-spec.k 117.37 117.81 0.996265172735761
erc20/ds/totalSupply-spec.k 82.22 82.49 0.9967268759849679
specs/bihu/forwardToHotWallet-failure-1-spec.k 148.14 148.62 0.9967702866370608
specs/functional/lemmas-no-smt-spec.k 13.15 13.19 0.99696739954511
specs/mcd/flopper-cage-pass-spec.k 172.8 173.28 0.997229916897507
erc20/ds/transferFrom-failure-1-d-spec.k 327.41 327.88 0.9985665487373431
specs/functional/storageRoot-spec.k 31.94 31.9 1.001253918495298
specs/benchmarks/bytes00-spec.k 149.66 149.36 1.0020085698982324
specs/mcd/flipper-addu48u48-fail-rough-spec.k 76.38 76.2 1.0023622047244094
specs/functional/infinite-gas-spec.k 19.01 18.95 1.003166226912929
specs/mcd/functional-spec.k 18.36 18.3 1.0032786885245901
specs/benchmarks/functional-spec.k 28.43 28.32 1.0038841807909604
specs/benchmarks/overflow00-nooverflow-spec.k 93.21 92.73 1.005176318343578
specs/functional/lemmas-spec.k 21.1 20.58 1.0252672497570459
specs/bihu/functional-spec.k 11.89 11.4 1.042982456140351
TOTAL 12691.76 12817.610000000002 0.9901814768899972

<$> runTransitionT (applyGroup is config)

applyGroup :: Step -> config -> TransitionT rule m config
applyGroup is c = foldM (flip applyPrim) c is
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to be a problem with cabal but not with stack - we should consolidate dependency versions more thoroughly IMHO

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added import Control.Monad(foldM) to make cabal happy again.

debugBeforeTransition,
debugFinalTransition,
)
import Kore.Log.InfoExecBreadth
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs to be added to the GraphTraversal where the queue operation now lives.

@jberthold jberthold force-pushed the 3119-output-unexplored-leaf-count-refactor-leavesM branch from 0ae648e to b79fa99 Compare July 25, 2022 02:30
@jberthold jberthold force-pushed the 3119-output-unexplored-leaf-count-refactor-leavesM branch from b08902c to 3719858 Compare July 25, 2022 03:31
@github-actions
Copy link

name diff_allocated_bytes diff_max_live_bytes
test/regression-evm/test-lemmas.sh -0.011989 -0.000682
test/regression-evm/test-dsvalue-peek-pass-rough.sh -0.011855 -0.005352
test/regression-evm/test-storagevar03.sh -0.016338 0.003555
test/regression-evm/test-totalSupply.sh -0.014611 -0.000056
test/regression-evm/test-addu48u48.sh -0.003540 0
test/regression-evm/test-sum-to-n.sh -0.014501 0.000423
test/regression-wasm/test-memory.sh -0.006686 0
test/regression-wasm/test-wrc20.sh -0.011594 0.004043
test/regression-wasm/test-simple-arithmetic.sh -0.002833 -0.000001
test/regression-wasm/test-locals.sh -0.000676 0
test/regression-wasm/test-loops.sh -0.007160 -0.000000

@jberthold
Copy link
Member Author

When using Simplifier instead of MonadSimplify simplifier we get a few more seconds speed-up, overall ~2.4%

Test newprovecode-3-newprovecode-3 time master20220722-master20220722 time (newprovecode-3-newprovecode-3/master20220722-master20220722) time
specs/examples/solidity-code-spec.md 110.82 121.39 0.9129252821484471
specs/functional/merkle-spec.k 16.58 17.98 0.9221357063403781
specs/functional/storageRoot-spec.k 29.61 31.9 0.9282131661442007
specs/benchmarks/address00-spec.k 83.93 89.4 0.9388143176733781
erc20/hkg/balanceOf-spec.k 65.32 68.62 0.9519090644127075
erc20/hkg/transfer-success-2-spec.k 147.97 154.8 0.9558785529715762
specs/benchmarks/storagevar02-nooverflow-spec.k 79.25 82.89 0.9560863795391482
specs/benchmarks/encode-keccak00-spec.k 234.28 244.8 0.9570261437908496
erc20/hkg/transfer-failure-2-spec.k 118.8 124.07 0.9575239783992907
specs/benchmarks/overflow00-overflow-spec.k 86.24 89.79 0.9604633032631695
specs/benchmarks/overflow00-nooverflow-spec.k 89.23 92.73 0.9622560120780761
specs/mcd/vow-flog-fail-rough-spec.k 252.99 262.91 0.962268456886387
erc20/hkg/allowance-spec.k 87.47 90.76 0.9637505509034816
specs/benchmarks/staticarray00-spec.k 121.5 126.06 0.9638267491670632
specs/benchmarks/storagevar03-spec.k 53.99 56.01 0.9639350116050706
specs/benchmarks/requires01-a0gt0-spec.k 89.4 92.69 0.9645053403819183
specs/mcd/cat-exhaustiveness-spec.k 105.22 109.07 0.9647015678004951
erc20/ds/balanceOf-spec.k 127.35 131.98 0.9649189271101682
erc20/hkg/totalSupply-spec.k 31.31 32.42 0.9657618753855644
specs/mcd/dsvalue-read-pass-summarize-spec.k 88.81 91.95 0.9658510059815116
specs/benchmarks/storagevar01-spec.k 72.71 75.18 0.9671455174248469
specs/benchmarks/requires01-a0le0-spec.k 66.24 68.45 0.9677136596055513
erc20/hkg/approve-spec.k 113.24 116.95 0.9682770414707139
specs/mcd/flipper-ttl-pass-spec.k 115.22 118.99 0.9683166652659888
specs/bihu/forwardToHotWallet-failure-2-spec.k 196.61 203.02 0.9684267559846321
specs/mcd/cat-file-addr-pass-rough-spec.k 164.25 169.46 0.9692552814823556
erc20/ds/allowance-spec.k 194.53 200.68 0.9693541957345027
specs/benchmarks/staticloop00-a0lt10-spec.k 156.4 161.32 0.9695016117034466
erc20/hkg/transferFrom-failure-2-spec.k 273.68 281.89 0.9708751640710916
specs/opcodes/evm-optimizations-spec.md 14.95 15.38 0.972041612483745
erc20/ds/approve-success-spec.k 344.95 354.75 0.9723749119097956
erc20/hkg/transfer-failure-1-spec.k 205.96 211.79 0.9724727324236273
erc20/ds/transfer-failure-2-b-spec.k 182.65 187.78 0.9726807966769624
specs/mcd/dai-symbol-pass-spec.k 152.75 157.01 0.9728679701929814
erc20/hkg/transfer-success-1-spec.k 253.21 260.27 0.9728743228186115
erc20/hkg/transferFrom-success-2-spec.k 360.36 369.9 0.9742092457420926
specs/bihu/forwardToHotWallet-failure-1-spec.k 144.89 148.62 0.9749024357421611
erc20/ds/totalSupply-spec.k 80.42 82.49 0.9749060492180871
specs/benchmarks/keccak00-spec.k 152.82 156.74 0.9749904300114839
erc20/ds/approve-failure-spec.k 81.74 83.81 0.9753012766972914
specs/mcd/vat-subui-fail-rough-spec.k 189.29 194.04 0.975520511234797
specs/mcd/dsvalue-read-pass-spec.k 191.66 196.4 0.9758655804480652
specs/bihu/forwardToHotWallet-failure-4-spec.k 345.81 354.17 0.9763955162774938
specs/mcd/vat-move-diff-rough-spec.k 531.99 544.83 0.9764330158030945
specs/mcd/vow-fess-fail-rough-spec.k 279.51 286.1 0.9769660957707095
specs/benchmarks/encodepacked-keccak01-spec.k 123.55 126.42 0.977297895902547
specs/mcd/flipper-tau-pass-spec.k 115.23 117.81 0.9781003310415075
specs/bihu/forwardToHotWallet-failure-3-spec.k 217.0 221.81 0.9783147739055948
erc20/ds/transfer-failure-1-c-spec.k 267.79 273.72 0.9783355253543767
specs/benchmarks/ecrecover00-siginvalid-spec.k 187.64 191.65 0.9790764414296894
specs/benchmarks/bytes00-spec.k 146.24 149.36 0.9791108730583824
erc20/ds/transfer-failure-2-a-spec.k 359.1 366.2 0.9806116876024031
specs/mcd/vat-deny-diff-fail-rough-spec.k 303.8 309.7 0.980949305779787
specs/mcd/flopper-cage-pass-spec.k 170.07 173.28 0.9814750692520775
specs/benchmarks/ecrecover00-sigvalid-spec.k 240.04 243.93 0.9840528020333702
specs/examples/sum-to-n-spec.k 19.44 19.75 0.9843037974683545
specs/mcd/dsvalue-peek-pass-rough-spec.k 184.37 187.31 0.9843040948160803
specs/mcd/vat-subui-pass-spec.k 264.58 268.7 0.9846669147748418
specs/mcd/end-subuu-pass-spec.k 110.18 111.84 0.9851573676680974
specs/functional/infinite-gas-spec.k 18.69 18.95 0.9862796833773088
specs/mcd/vat-addui-fail-rough-spec.k 191.0 193.61 0.9865192913589174
specs/mcd/dstoken-burn-self-fail-rough-spec.k 1320.02 1337.08 0.987240853202501
specs/mcd/vat-mului-pass-spec.k 197.23 199.49 0.9886711113339014
erc20/ds/transferFrom-failure-2-c-spec.k 210.83 213.23 0.9887445481405056
specs/benchmarks/functional-spec.k 28.06 28.32 0.990819209039548
specs/mcd/dai-adduu-fail-rough-spec.k 59.43 59.96 0.9911607738492328
specs/mcd/functional-spec.k 18.17 18.3 0.992896174863388
specs/mcd/flipper-addu48u48-fail-rough-spec.k 75.75 76.2 0.9940944881889764
specs/benchmarks/storagevar00-spec.k 66.32 66.61 0.9956462993544511
specs/mcd/vat-subui-pass-rough-spec.k 260.81 261.88 0.9959141591568658
erc20/ds/transferFrom-failure-1-d-spec.k 327.87 327.88 0.9999695010369648
0 17.84 17.79 1.0028105677346824
specs/benchmarks/storagevar02-overflow-spec.k 80.54 79.42 1.0141022412490557
specs/functional/lemmas-spec.k 21.23 20.58 1.0315840621963073
specs/bihu/functional-spec.k 12.06 11.4 1.0578947368421052
specs/functional/lemmas-no-smt-spec.k 14.66 13.19 1.11144806671721
TOTAL 12515.45 12817.610000000002 0.9764261824162226

@github-actions
Copy link

name diff_allocated_bytes diff_max_live_bytes
test/regression-evm/test-lemmas.sh -0.029535 0.278330
test/regression-evm/test-dsvalue-peek-pass-rough.sh -0.029151 0.003703
test/regression-evm/test-storagevar03.sh -0.040155 -0.001009
test/regression-evm/test-totalSupply.sh -0.036041 -0.000751
test/regression-evm/test-addu48u48.sh -0.008690 0
test/regression-evm/test-sum-to-n.sh -0.035685 0.003908
test/regression-wasm/test-memory.sh -0.016507 0
test/regression-wasm/test-wrc20.sh -0.028630 -0.015543
test/regression-wasm/test-simple-arithmetic.sh -0.006981 0
test/regression-wasm/test-locals.sh -0.001668 0
test/regression-wasm/test-loops.sh -0.017564 0

@github-actions
Copy link

name diff_allocated_bytes diff_max_live_bytes
test/regression-evm/test-lemmas.sh -0.029584 -0.216300
test/regression-evm/test-dsvalue-peek-pass-rough.sh -0.029148 0.002784
test/regression-evm/test-storagevar03.sh -0.040161 0.003418
test/regression-evm/test-totalSupply.sh -0.036049 -0.002435
test/regression-evm/test-addu48u48.sh -0.008691 0.010998
test/regression-evm/test-sum-to-n.sh -0.035686 -0.000470
test/regression-wasm/test-memory.sh -0.016493 0
test/regression-wasm/test-wrc20.sh -0.028619 -0.012112
test/regression-wasm/test-simple-arithmetic.sh -0.006978 0
test/regression-wasm/test-locals.sh -0.001669 0.000000
test/regression-wasm/test-loops.sh -0.017565 0

@github-actions
Copy link

name diff_allocated_bytes diff_max_live_bytes
test/regression-evm/test-lemmas.sh -0.029597 0.278338
test/regression-evm/test-dsvalue-peek-pass-rough.sh -0.029147 0.003582
test/regression-evm/test-storagevar03.sh -0.040159 0.002077
test/regression-evm/test-totalSupply.sh -0.036047 0.001837
test/regression-evm/test-addu48u48.sh -0.008695 -0.001542
test/regression-evm/test-sum-to-n.sh -0.035724 -0.002611
test/regression-wasm/test-memory.sh -0.016495 -0.000001
test/regression-wasm/test-wrc20.sh -0.028620 -0.006969
test/regression-wasm/test-simple-arithmetic.sh -0.006975 0
test/regression-wasm/test-locals.sh -0.001667 -0.000001
test/regression-wasm/test-loops.sh -0.017561 0

@github-actions
Copy link

name diff_allocated_bytes diff_max_live_bytes
test/regression-evm/test-lemmas.sh -0.028921 0.000439
test/regression-evm/test-dsvalue-peek-pass-rough.sh -0.029142 0.000711
test/regression-evm/test-storagevar03.sh -0.040152 -0.000242
test/regression-evm/test-totalSupply.sh -0.036045 0.001370
test/regression-evm/test-addu48u48.sh -0.008689 0.009522
test/regression-evm/test-sum-to-n.sh -0.034924 -0.000406
test/regression-wasm/test-memory.sh -0.016492 -0.000000
test/regression-wasm/test-wrc20.sh -0.028615 0.004354
test/regression-wasm/test-simple-arithmetic.sh -0.006977 0
test/regression-wasm/test-locals.sh -0.001668 -0.000000
test/regression-wasm/test-loops.sh -0.017538 0

@jberthold
Copy link
Member Author

Replaced by #3168

@jberthold jberthold closed this Jul 27, 2022
@jberthold jberthold deleted the 3119-output-unexplored-leaf-count-refactor-leavesM branch August 1, 2022 00:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants