Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(migration): Migrate contract slots from mono- to modular- in deterministic order #10310

Merged

Conversation

david-bakin-sl
Copy link
Member

@david-bakin-sl david-bakin-sl commented Dec 5, 2023

Description:

Nodes added to the merkle tree must be added in deterministic order.

Especially for the migrators, which add so many nodes they cross commit boundaries.

Remove the concurrent sinking of contract slots; replace with accumulating all fromState slots into a concurrent queue, then placing them in an array, and then sorting them.

Related issue(s):

Fixes #10293

Checklist

  • Documented (code comments)
  • Tested (via incorporation into contract store dumper)

Nodes added to the merkle tree _must be added in deterministic order_.

Especially for the migrators, which add so many nodes they cross _commit_ boundaries.

Remove the concurrent sinking of contract slots; replace with accumulating all `fromState`
slots into a concurrent queue, then placing them in an array, and then sorting them.

Fixes #10293

Signed-off-by: David S Bakin <117694041+david-bakin-sl@users.noreply.github.com>
@david-bakin-sl david-bakin-sl self-assigned this Dec 5, 2023
@david-bakin-sl david-bakin-sl requested review from a team as code owners December 5, 2023 23:35
@david-bakin-sl david-bakin-sl changed the title Migrate contract slots from mono- to modular- in deterministic order fix(migration): Migrate contract slots from mono- to modular- in deterministic order Dec 5, 2023
Copy link

github-actions bot commented Dec 6, 2023

Node: Unit Test Results

    2 293 files  ±0      2 293 suites  ±0   44m 57s ⏱️ - 1m 6s
118 431 tests +2  118 397 ✔️ +2  34 💤 ±0  0 ±0 
126 853 runs  +2  126 819 ✔️ +2  34 💤 ±0  0 ±0 

Results for commit fcd2182. ± Comparison against base commit ce367a4.

♻️ This comment has been updated with latest results.

Copy link

codecov bot commented Dec 6, 2023

Codecov Report

Attention: 55 lines in your changes are missing coverage. Please review.

Comparison is base (3b0ea4f) 63.51% compared to head (fcd2182) 63.51%.
Report is 10 commits behind head on develop.

Files Patch % Lines
...ce/mono/state/migration/ContractStateMigrator.java 0.00% 51 Missing ⚠️
...state/migration/BrokenTransformationException.java 0.00% 4 Missing ⚠️
Additional details and impacted files
@@              Coverage Diff              @@
##             develop   #10310      +/-   ##
=============================================
- Coverage      63.51%   63.51%   -0.01%     
- Complexity     30953    30989      +36     
=============================================
  Files           3337     3341       +4     
  Lines         134351   134558     +207     
  Branches       13965    13994      +29     
=============================================
+ Hits           85334    85464     +130     
- Misses         45661    45732      +71     
- Partials        3356     3362       +6     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link

github-actions bot commented Dec 6, 2023

Node: HAPI Test (Crypto) Results

211 tests  ±0   201 ✔️ ±0   20m 37s ⏱️ + 1m 22s
  22 suites ±0     10 💤 ±0 
  22 files   ±0       0 ±0 

Results for commit fcd2182. ± Comparison against base commit ce367a4.

♻️ This comment has been updated with latest results.

Copy link

github-actions bot commented Dec 6, 2023

Node: HAPI Test (Misc) Results

419 tests  ±0   316 ✔️ +3   25m 41s ⏱️ - 1m 1s
  73 suites ±0   103 💤  - 3 
  73 files   ±0       0 ±0 

Results for commit fcd2182. ± Comparison against base commit ce367a4.

♻️ This comment has been updated with latest results.

Copy link

github-actions bot commented Dec 6, 2023

Node: E2E Test Results

    1 files  ±    0      1 suites  ±0   23m 29s ⏱️ + 23m 29s
311 tests +310  311 ✔️ +311  0 💤 ±0  0  - 1 
333 runs  +332  333 ✔️ +333  0 💤 ±0  0  - 1 

Results for commit fcd2182. ± Comparison against base commit ce367a4.

This pull request removes 1 and adds 311 tests. Note that renamed tests count towards both.
EndToEndTests ‑ initializationError
EndToEndTests ‑ ADDRESS_BOOK_CONTROLCanUpdateADDRESS_BOOK
EndToEndTests ‑ ADDRESS_BOOK_CONTROLCanUpdateNODE_DETAILS
EndToEndTests ‑ AccountsGetPayerRecordsIfSoConfigured
EndToEndTests ‑ Acct57CanMakeSmallChanges
EndToEndTests ‑ Acct57CantMakeLargeChanges
EndToEndTests ‑ AddingSignaturesToExecutedTxFails
EndToEndTests ‑ AddingSignaturesToExecutedTxFailsWithLongTermEnabled
EndToEndTests ‑ AddingSignaturesToNonExistingTxFails
EndToEndTests ‑ AddingSignaturesToNonExistingTxFailsWithLongTermEnabled
EndToEndTests ‑ AddressAliasIdFuzzing
…

♻️ This comment has been updated with latest results.

Copy link

github-actions bot commented Dec 6, 2023

Node: HAPI Test (Time Consuming) Results

21 tests  ±0     9 ✔️ ±0   24m 45s ⏱️ -14s
  2 suites ±0   12 💤 ±0 
  2 files   ±0     0 ±0 

Results for commit fcd2182. ± Comparison against base commit ce367a4.

♻️ This comment has been updated with latest results.

Copy link

github-actions bot commented Dec 6, 2023

Node: Integration Test Results

279 tests  ±0   279 ✔️ ±0   28m 31s ⏱️ -3s
    5 suites ±0       0 💤 ±0 
    5 files   ±0       0 ±0 

Results for commit fcd2182. ± Comparison against base commit ce367a4.

♻️ This comment has been updated with latest results.

Copy link

github-actions bot commented Dec 6, 2023

Node: HAPI Test (Smart Contract) Results

396 tests  ±0   349 ✔️ +1   54m 55s ⏱️ + 6m 50s
  55 suites ±0     47 💤  - 1 
  55 files   ±0       0 ±0 

Results for commit fcd2182. ± Comparison against base commit ce367a4.

♻️ This comment has been updated with latest results.

Copy link

github-actions bot commented Dec 6, 2023

Node: HAPI Test (Token) Results

190 tests  ±0   188 ✔️ ±0   19m 48s ⏱️ - 1m 23s
  13 suites ±0       2 💤 ±0 
  13 files   ±0       0 ±0 

Results for commit fcd2182. ± Comparison against base commit ce367a4.

♻️ This comment has been updated with latest results.

(remove misleading no-longer-needed comment)

Signed-off-by: David S Bakin <117694041+david-bakin-sl@users.noreply.github.com>
@david-bakin-sl david-bakin-sl merged commit f8f0317 into develop Dec 6, 2023
26 of 28 checks passed
@david-bakin-sl david-bakin-sl deleted the 10293-contract-store-deterministic-migration branch December 6, 2023 23:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Contract store migrator needs to put stuff in the merkle tree _deterministically_
2 participants