Add a spec for Restart test #10105

Neeharika-Sompalli · 2023-11-27T04:47:31Z

Added a test suite that does submit mixed operations, freeze, stops nodes, restarts nodes, waits for ACTIVE status, and then submits some more mixed operations.
Updated all Service schemas to use Current Version instead of Genesis version

Testing:

In swirlds.log I could see
- Platform freezes with following log

2023-11-26 22:52:03.830 545      INFO  PLATFORM_STATUS  <<platform: status-state-machine>> PlatformStatusStateMachine: Platform spent 34.7 s in ACTIVE. Now in FREEZING {"oldStatus":"ACTIVE","newStatus":"FREEZING"} [com.swirlds.logging.legacy.payload.PlatformStatusPayload]
2023-11-26 22:52:03.841 547      INFO  FREEZE           <<platform-core: thread-cons 0>> DualStateImpl: setLastFrozenTimeToBeCurrentFreezeTime: 2023-11-27T04:52:03.597263Z
2023-11-26 22:52:03.855 548      INFO  FREEZE           <<platform-core: thread-cons 0>> DualStateImpl: Set last frozen time {"lastFrozenTime":"2023-11-27T04:52:03.597263Z"} [com.swirlds.logging.legacy.payload.SetLastFrozenTimePayload]
2023-11-26 22:52:03.898 550      INFO  STATE_TO_DISK    <<platform-core: thread-curr 0>> SavedStateController: Signed state from round 255 created, will eventually be written to disk, for reason: FREEZE_STATE
2023-11-26 22:52:03.902 551      INFO  STATE_TO_DISK    <<scheduler signed_state_file_manager>> SignedStateFileWriter: Started writing round 255 state to disk. Reason: FREEZE_STATE, directory: /Users/neeharikasompalli/Documents/Hedera/Repos/hedera-services/hedera-node/test-clients/build/hapi-test/node0/data/saved/com.hedera.services.ServicesMain/0/123/255
2023-11-26 22:52:06.302 560      INFO  STATE_TO_DISK    <<scheduler signed_state_file_manager>> SignedStateFileWriter: Information for state written to disk:

Then writes saved state and shuts down

2023-11-26 22:52:06.677 566      INFO  PLATFORM_STATUS  <<platform: status-state-machine>> PlatformStatusStateMachine: Platform spent 2.8 s in FREEZING. Now in FREEZE_COMPLETE {"oldStatus":"FREEZING","newStatus":"FREEZE_COMPLETE"} [com.swirlds.logging.legacy.payload.PlatformStatusPayload]
2023-11-26 22:52:08.931 567      INFO  STARTUP          <<browser: shutdown-hook>> Log4jSetup: JVM is shutting down.

Loads last written state

2023-11-26 22:52:39.783 25       INFO  STARTUP          <main> StartupStateUtils: Platform has loaded a saved state {"round":255,"consensusTimestamp":"2023-11-27T04:52:03.598460Z"} [com.swirlds.logging.legacy.payload.SavedStateLoadedPayload]
2023-11-26 22:52:39.807 26       INFO  STARTUP          <main> BootstrapUtils: Not upgrading software, current software is version HederaSoftwareVersion{hapiVersion=v0.44.0, servicesVersion=v0.45.0-SNAPSHOT}.
2023-11-26 22:52:39.808 27       INFO  STARTUP          <main> AddressBookInitializer: Using the loaded state's address book and weight values.

Turns ACTIVE

2023-11-26 22:52:53.040 50       INFO  PLATFORM_STATUS  <<platform: status-state-machine>> PlatformStatusStateMachine: Platform spent 868.0 ms in CHECKING. Now in ACTIVE {"oldStatus":"CHECKING","newStatus":"ACTIVE"} [com.swirlds.logging.legacy.payload.PlatformStatusPayload]

Client Log:
Submits all transactions after restart successfully

NOTE:

When the platform loads state, the BlockRecordService.BLOCKS hash is different from what is written while freezing. Still need to Debug why that happens.

Signed-off-by: Neeharika-Sompalli <neeharika.sompalli@swirldslabs.com>

github-actions · 2023-11-27T05:21:53Z

Node: HAPI Test (Misc) Results

418 tests ±0 286 ✔️ +3 25m 0s ⏱️ + 3m 29s
  73 suites ±0 132 💤 - 3
  73 files ±0     0 ❌ ±0

Results for commit 1611c44. ± Comparison against base commit eb581b42.

♻️ This comment has been updated with latest results.

github-actions · 2023-11-27T05:22:09Z

Node: HAPI Test (Crypto) Results

211 tests ±0 201 ✔️ ±0 21m 9s ⏱️ + 3m 40s
  22 suites ±0   10 💤 ±0
  22 files ±0     0 ❌ ±0

Results for commit 1611c44. ± Comparison against base commit eb581b42.

♻️ This comment has been updated with latest results.

github-actions · 2023-11-27T05:22:17Z

Node: HAPI Test (Token) Results

190 tests ±0 188 ✔️ ±0 22m 25s ⏱️ + 4m 53s
  13 suites ±0     2 💤 ±0
  13 files ±0     0 ❌ ±0

Results for commit 1611c44. ± Comparison against base commit eb581b42.

♻️ This comment has been updated with latest results.

github-actions · 2023-11-27T05:23:25Z

Node: Unit Test Results

    2 286 files ±0     2 286 suites ±0 50m 44s ⏱️ + 3m 39s
118 388 tests +2 118 354 ✔️ +2 34 💤 ±0 0 ❌ ±0
126 760 runs +2 126 726 ✔️ +2 34 💤 ±0 0 ❌ ±0

Results for commit 1611c44. ± Comparison against base commit eb581b42.

♻️ This comment has been updated with latest results.

codecov · 2023-11-27T05:24:30Z

Codecov Report

Attention: 32 lines in your changes are missing coverage. Please review.

Comparison is base (eb581b4) 63.12% compared to head (1611c44) 63.13%.
Report is 3 commits behind head on develop.

Files	Patch %	Lines
...main/java/com/hedera/node/app/fees/FeeService.java	0.00%	20 Missing ⚠️
.../service/networkadmin/impl/NetworkServiceImpl.java	0.00%	3 Missing ⚠️
.../java/com/hedera/node/app/ids/EntityIdService.java	50.00%	1 Missing and 1 partial ⚠️
...src/main/java/com/hedera/node/app/spi/Service.java	0.00%	1 Missing ⚠️
...om/hedera/node/app/records/BlockRecordService.java	88.88%	0 Missing and 1 partial ⚠️
...a/node/app/throttle/CongestionThrottleService.java	0.00%	1 Missing ⚠️
...p/service/consensus/impl/ConsensusServiceImpl.java	0.00%	1 Missing ⚠️
...p/service/networkadmin/impl/FreezeServiceImpl.java	83.33%	0 Missing and 1 partial ⚠️
.../node/app/service/token/impl/TokenServiceImpl.java	0.00%	1 Missing ⚠️
...de/app/service/token/impl/schemas/TokenSchema.java	83.33%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@              Coverage Diff              @@
##             develop   #10105      +/-   ##
=============================================
+ Coverage      63.12%   63.13%   +0.01%     
- Complexity     30689    30696       +7     
=============================================
  Files           3327     3327              
  Lines         133864   133875      +11     
  Branches       13873    13880       +7     
=============================================
+ Hits           84498    84523      +25     
+ Misses         46032    46014      -18     
- Partials        3334     3338       +4

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

github-actions · 2023-11-27T05:28:09Z

Node: E2E Test Results

    1 files ±    0     1 suites ±0 22m 5s ⏱️ + 22m 5s
310 tests +309 310 ✔️ +310 0 💤 ±0 0 ❌ - 1
332 runs +331 332 ✔️ +332 0 💤 ±0 0 ❌ - 1

Results for commit 1611c44. ± Comparison against base commit eb581b42.

This pull request removes 1 and adds 310 tests. Note that renamed tests count towards both.

EndToEndTests ‑ initializationError

EndToEndTests ‑ ADDRESS_BOOK_CONTROLCanUpdateADDRESS_BOOK
EndToEndTests ‑ ADDRESS_BOOK_CONTROLCanUpdateNODE_DETAILS
EndToEndTests ‑ AccountsGetPayerRecordsIfSoConfigured
EndToEndTests ‑ Acct57CanMakeSmallChanges
EndToEndTests ‑ Acct57CantMakeLargeChanges
EndToEndTests ‑ AddingSignaturesToExecutedTxFails
EndToEndTests ‑ AddingSignaturesToExecutedTxFailsWithLongTermEnabled
EndToEndTests ‑ AddingSignaturesToNonExistingTxFails
EndToEndTests ‑ AddingSignaturesToNonExistingTxFailsWithLongTermEnabled
EndToEndTests ‑ AddressAliasIdFuzzing
…

♻️ This comment has been updated with latest results.

github-actions · 2023-11-27T05:30:04Z

Node: HAPI Test (Time Consuming) Results

21 tests ±0   9 ✔️ ±0 26m 37s ⏱️ + 1m 19s
  2 suites ±0 12 💤 ±0
  2 files ±0   0 ❌ ±0

Results for commit 1611c44. ± Comparison against base commit eb581b42.

♻️ This comment has been updated with latest results.

github-actions · 2023-11-27T05:34:08Z

Node: Integration Test Results

279 tests ±0 279 ✔️ +1 28m 23s ⏱️ -3s
    5 suites ±0     0 💤 ±0
    5 files ±0     0 ❌ - 1

Results for commit 1611c44. ± Comparison against base commit eb581b42.

♻️ This comment has been updated with latest results.

github-actions · 2023-11-27T05:46:21Z

Node: HAPI Test (Smart Contract) Results

411 tests ±0 318 ✔️ ±0 42m 16s ⏱️ +30s
  56 suites ±0   93 💤 ±0
  56 files ±0     0 ❌ ±0

Results for commit 1611c44. ± Comparison against base commit eb581b42.

♻️ This comment has been updated with latest results.

Signed-off-by: Neeharika-Sompalli <neeharika.sompalli@swirldslabs.com>

hedera-node/hedera-app-spi/src/main/java/com/hedera/node/app/spi/Service.java

Signed-off-by: Neeharika-Sompalli <neeharika.sompalli@swirldslabs.com>

…4-D-restart-testing

Signed-off-by: Neeharika-Sompalli <neeharika.sompalli@swirldslabs.com>

hedera-node/hedera-app/src/main/java/com/hedera/node/app/records/BlockRecordService.java

tinker-michaelj

This all makes sense to me, ty @Neeharika-Sompalli ! 🥂

Signed-off-by: Neeharika-Sompalli <neeharika.sompalli@swirldslabs.com>

Neeharika-Sompalli added 6 commits November 21, 2023 15:43

add basic test for restart testing

a609605

Signed-off-by: Neeharika-Sompalli <neeharika.sompalli@swirldslabs.com>

add current version

33e492b

Signed-off-by: Neeharika-Sompalli <neeharika.sompalli@swirldslabs.com>

Merge branch 'develop' into 9964-D-restart-testing

67df70b

add current version

bc9704b

Signed-off-by: Neeharika-Sompalli <neeharika.sompalli@swirldslabs.com>

fix some other issues

d67a537

Signed-off-by: Neeharika-Sompalli <neeharika.sompalli@swirldslabs.com>

remove all logs

8fa80de

Signed-off-by: Neeharika-Sompalli <neeharika.sompalli@swirldslabs.com>

Neeharika-Sompalli requested a review from tinker-michaelj November 27, 2023 04:57

fix failing unit tests

ac1172f

Signed-off-by: Neeharika-Sompalli <neeharika.sompalli@swirldslabs.com>

Neeharika-Sompalli self-assigned this Nov 27, 2023

Merge branch 'develop' into 9964-D-restart-testing

3ab6fc1

Neeharika-Sompalli marked this pull request as ready for review November 27, 2023 15:36

Neeharika-Sompalli requested review from a team as code owners November 27, 2023 15:36

Neeharika-Sompalli requested review from mhess-swl, rbair23 and netopyr November 27, 2023 15:36

Neeharika-Sompalli added 4 commits November 27, 2023 10:16

remove logs

5399c70

Signed-off-by: Neeharika-Sompalli <neeharika.sompalli@swirldslabs.com>

remove NPE check

de2ab44

Signed-off-by: Neeharika-Sompalli <neeharika.sompalli@swirldslabs.com>

disable test running on CI

6ff3c42

Signed-off-by: Neeharika-Sompalli <neeharika.sompalli@swirldslabs.com>

disable test

8932928

Signed-off-by: Neeharika-Sompalli <neeharika.sompalli@swirldslabs.com>

mhess-swl reviewed Nov 28, 2023

View reviewed changes

hedera-node/hedera-app-spi/src/main/java/com/hedera/node/app/spi/Service.java Outdated Show resolved Hide resolved

Neeharika-Sompalli and others added 5 commits November 27, 2023 22:07

Merge branch 'develop' into 9964-D-restart-testing

1d09c1e

chnage to release 0.45 version

c5284f3

Signed-off-by: Neeharika-Sompalli <neeharika.sompalli@swirldslabs.com>

Merge remote-tracking branch 'origin/9964-D-restart-testing' into 996…

637dd1c

…4-D-restart-testing

spotless

b0fcbd8

Signed-off-by: Neeharika-Sompalli <neeharika.sompalli@swirldslabs.com>

spotless

1611c44

Signed-off-by: Neeharika-Sompalli <neeharika.sompalli@swirldslabs.com>

Neeharika-Sompalli requested a review from mhess-swl November 28, 2023 18:26

tinker-michaelj reviewed Nov 28, 2023

View reviewed changes

hedera-node/hedera-app/src/main/java/com/hedera/node/app/records/BlockRecordService.java Show resolved Hide resolved

tinker-michaelj approved these changes Nov 28, 2023

View reviewed changes

Neeharika-Sompalli merged commit d25c2a6 into develop Nov 28, 2023
23 of 25 checks passed

Neeharika-Sompalli deleted the 9964-D-restart-testing branch November 28, 2023 19:06

agadzhalov pushed a commit that referenced this pull request Dec 1, 2023

Add a spec for Restart test (#10105)

d73f22c

Signed-off-by: Neeharika-Sompalli <neeharika.sompalli@swirldslabs.com>

agadzhalov pushed a commit that referenced this pull request Dec 1, 2023

Add a spec for Restart test (#10105)

39a519c

Signed-off-by: Neeharika-Sompalli <neeharika.sompalli@swirldslabs.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add a spec for Restart test #10105

Add a spec for Restart test #10105

Neeharika-Sompalli commented Nov 27, 2023 •

edited

github-actions bot commented Nov 27, 2023 •

edited

github-actions bot commented Nov 27, 2023 •

edited

github-actions bot commented Nov 27, 2023 •

edited

github-actions bot commented Nov 27, 2023 •

edited

codecov bot commented Nov 27, 2023 •

edited

github-actions bot commented Nov 27, 2023 •

edited

github-actions bot commented Nov 27, 2023 •

edited

github-actions bot commented Nov 27, 2023 •

edited

github-actions bot commented Nov 27, 2023 •

edited

tinker-michaelj left a comment

Add a spec for Restart test #10105

Add a spec for Restart test #10105

Conversation

Neeharika-Sompalli commented Nov 27, 2023 • edited

github-actions bot commented Nov 27, 2023 • edited

Node: HAPI Test (Misc) Results

github-actions bot commented Nov 27, 2023 • edited

Node: HAPI Test (Crypto) Results

github-actions bot commented Nov 27, 2023 • edited

Node: HAPI Test (Token) Results

github-actions bot commented Nov 27, 2023 • edited

Node: Unit Test Results

codecov bot commented Nov 27, 2023 • edited

Codecov Report

github-actions bot commented Nov 27, 2023 • edited

Node: E2E Test Results

github-actions bot commented Nov 27, 2023 • edited

Node: HAPI Test (Time Consuming) Results

github-actions bot commented Nov 27, 2023 • edited

Node: Integration Test Results

github-actions bot commented Nov 27, 2023 • edited

Node: HAPI Test (Smart Contract) Results

tinker-michaelj left a comment

Choose a reason for hiding this comment

Neeharika-Sompalli commented Nov 27, 2023 •

edited

github-actions bot commented Nov 27, 2023 •

edited

github-actions bot commented Nov 27, 2023 •

edited

github-actions bot commented Nov 27, 2023 •

edited

github-actions bot commented Nov 27, 2023 •

edited

codecov bot commented Nov 27, 2023 •

edited

github-actions bot commented Nov 27, 2023 •

edited

github-actions bot commented Nov 27, 2023 •

edited

github-actions bot commented Nov 27, 2023 •

edited

github-actions bot commented Nov 27, 2023 •

edited