Integration tests staking v4 #3951

mariusmihaic · 2022-03-30T12:11:00Z

What's new

StakingV4DistributeAuctionToWaitingEpoch activation flag = epoch in which selected nodes from auction will be distributed to waiting list (used in node shuffler)
Added a new entry in MaxNodesChangeEnableEpoch config, which will have enable epoch = StakingV4DistributeAuctionToWaitingEpoch and will decrease the max nodes config with the number of shuffled out validators from all shards (e.g. from 3200 -> 2880, since we shuffle out 80 eligible validators per 4 shards)
Computation of available slots for auction nodes which will be selected based on topup as following:

availableSlots = maxNodes - currentNumberOfValidators - numOfNodesWhichWillBeShuffledOut

Integration tests

Added TestMetaProcessor struct which will create a predefined nodes configuration with provided params:

        numOfMetaNodes uint32,
	numOfShards uint32,
	numOfEligibleNodesPerShard uint32,
	numOfWaitingNodesPerShard uint32,
	numOfNodesToShufflePerShard uint32,
	shardConsensusGroupSize int,
	metaConsensusGroupSize int,
	numOfNodesInStakingQueue uint32,

All generated address have a fixed length which starts from index = 0 -> maxNumNodes (e.g. 00address-0 -> 0address-50)
Every node in waiting/eligible is associated with a different owner
All nodes in staking queue are owned by a single owner
Every node has the same top-up per node
It can Process n number of rounds by committing blocks in each of them and will display nodes config each time.

Added 2 integration tests:

TestStakingV4 which simulates a mainnet nodes configuration (400 meta nodes + 400 eligible and waiting/shard + 60 nodes in staking queue, each of them having the same topup per node) going through stakingV4Init, stakingV4Enable, stakingV4DistributeAuctionToWaiting epochs.
TestStakingV4MetaProcessor_ProcessMultipleNodesWithSameSetupExpectSameRootHash which simulates multiple TestMetaProcessor starting to process n epochs with the same initial setup. It is expected that there is not root hash mismatch at any epoch

Bugfixes

Deactivated flagCorrectNumNodesToStake flag in legacySystemSCs.go upon staking v4 enable epoch, since it used code related to staking queue (which is now obsolete)
Fixed a bug in which two nodes with the same top up would not be correctly sorted/compared by pubKey XOR randomness, resulting in root hash mismatches between meta shard nodes
Uint32 substraction underflow when computing available slots from auction list
setNodesPerShards in indexHashedNodesCoordinatorLite.go to also save shuffled out nodes
Race condition in already existing code from process/block/displayMetaBlock.go by adding a mutex for peakTPS

Refactor

Moved some testing code related to staking from systemSCs_test.go to testscommon/stakingCommon.go to be used in stakingV4 integration tests
NewFeeAccumulator constructor to return directly a new instance of *feeHandler instead of (*feeHandler, error) , since it could have never returned an error
Several stubs so that code in tests would not panic because of uninitialized memory

…epoch

sasurobert · 2022-04-15T09:22:38Z

integrationTests/vm/staking/componentsHolderCreator.go

+	factory2 "github.com/ElrondNetwork/elrond-go/factory"
+	"github.com/ElrondNetwork/elrond-go/integrationTests"
+	mock2 "github.com/ElrondNetwork/elrond-go/integrationTests/mock"
+	factory3 "github.com/ElrondNetwork/elrond-go/node/mock/factory"


rename factory 2, factory 3

Renamed them

sasurobert · 2022-04-15T09:29:46Z

integrationTests/vm/staking/metaBlockProcessorCreator.go

+	"github.com/ElrondNetwork/elrond-go/epochStart"
+	"github.com/ElrondNetwork/elrond-go/epochStart/metachain"
+	factory2 "github.com/ElrondNetwork/elrond-go/factory"
+	mock2 "github.com/ElrondNetwork/elrond-go/integrationTests/mock"


rename mock2 and factory2

Renamed them

sasurobert · 2022-04-15T09:29:59Z

integrationTests/vm/staking/nodesCoordiantorCreator.go

+	"github.com/ElrondNetwork/elrond-go/common"
+	"github.com/ElrondNetwork/elrond-go/config"
+	factory2 "github.com/ElrondNetwork/elrond-go/factory"
+	mock2 "github.com/ElrondNetwork/elrond-go/integrationTests/mock"


rename factory2 and mock2

Renamed them

sasurobert · 2022-04-15T09:30:28Z

integrationTests/vm/staking/systemSCCreator.go

+	"github.com/ElrondNetwork/elrond-go/config"
+	"github.com/ElrondNetwork/elrond-go/epochStart/metachain"
+	mock3 "github.com/ElrondNetwork/elrond-go/epochStart/mock"
+	factory2 "github.com/ElrondNetwork/elrond-go/factory"


rename factory2 mock3

Renamed them

sasurobert · 2022-04-15T09:30:55Z

integrationTests/vm/staking/testMetaProcessor.go

+	"github.com/ElrondNetwork/elrond-go/config"
+	"github.com/ElrondNetwork/elrond-go/dataRetriever"
+	"github.com/ElrondNetwork/elrond-go/epochStart/metachain"
+	factory2 "github.com/ElrondNetwork/elrond-go/factory"


rename factory2

Renamed them

…aking-v4

testscommon/rewardsCreatorStub.go

epochStart/metachain/systemSCs.go

testscommon/stakingcommon/stakingCommon.go

factory/coreComponents.go

epochStart/metachain/legacySystemSCs.go

integrationTests/vm/staking/stakingV4_test.go

epochStart/metachain/systemSCs.go

…aking-v4 # Conflicts: # process/block/postprocess/feeHandler.go

codecov · 2022-04-29T07:39:14Z

Codecov Report

Merging #3951 (a98dcee) into feat/liquid-staking (2df534f) will increase coverage by 0.00%.
The diff coverage is 86.66%.

@@                 Coverage Diff                  @@
##           feat/liquid-staking    #3951   +/-   ##
====================================================
  Coverage                75.38%   75.38%           
====================================================
  Files                      622      622           
  Lines                    83134    83177   +43     
====================================================
+ Hits                     62668    62701   +33     
- Misses                   15748    15756    +8     
- Partials                  4718     4720    +2

Impacted Files	Coverage Δ
epochStart/metachain/systemSCs.go	`71.65% <70.83%> (-1.75%)`	⬇️
epochStart/metachain/legacySystemSCs.go	`72.14% <100.00%> (+0.27%)`	⬆️
factory/blockProcessorCreator.go	`80.44% <100.00%> (+0.39%)`	⬆️
factory/coreComponents.go	`79.61% <100.00%> (-0.10%)`	⬇️
process/block/displayMetaBlock.go	`47.63% <100.00%> (+0.68%)`	⬆️
process/block/postprocess/feeHandler.go	`92.20% <100.00%> (ø)`
sharding/nodesCoordinator/hashValidatorShuffler.go	`93.02% <100.00%> (+0.11%)`	⬆️
...ng/nodesCoordinator/indexHashedNodesCoordinator.go	`82.20% <100.00%> (+0.02%)`	⬆️
...odesCoordinator/indexHashedNodesCoordinatorLite.go	`81.39% <100.00%> (ø)`
p2p/libp2p/netMessenger.go	`74.70% <0.00%> (-0.27%)`	⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2df534f...a98dcee. Read the comment docs.

sasurobert · 2022-05-02T13:07:48Z

epochStart/metachain/systemSCs.go

@@ -179,6 +203,14 @@ func (s *systemSCProcessor) selectNodesFromAuctionList(validatorsInfoMap state.S
 	return nil
 }

+// TODO: Move this in elrond-go-core
+func safeSub(a, b uint32) (uint32, error) {


we have this for Uint64

Right, but I didn't want to cast to uint64, but have a separate function for it. Will move it in elrond-go-core in a separate task

mariusmihaic added 4 commits March 30, 2022 15:09

FEAT: Add file placeholder

5133860

FEAT: Add intermediary code

8ebc25f

FEAT: Refactor

6cb1275

FEAT: Refactor 2

4ebd97e

mariusmihaic changed the title ~~FEAT: Add file placeholder~~ Integration tests staking v4 Mar 31, 2022

mariusmihaic added 13 commits March 31, 2022 17:45

FEAT: Refactor 3

fca992d

FEAT: Refactor 4

d4e9a1e

FEAT: Refactor 5

f3dbe32

FEAT: Ugly version with 2 committed blocks

1856d58

FEAT: Test with epoch start prepare

4ea2b9d

FEAT: Register bls keys + bugfixes

1449bcc

FEAT: Add Process for num of rounds

3c26053

FIX: Sub bug, add safeSub

53a59e0

FIX: Waiting list + stubs

65d9a69

FIX: Refactor 1

23407f8

FIX: Refactor 2

28b4285

FIX: Refactor 3

cda1ce3

FIX: Refactor 4

b37fc76

mariusmihaic self-assigned this Apr 7, 2022

mariusmihaic added 11 commits April 8, 2022 14:01

FIX: Refactor 5

da98d43

FIX: Refactor6

0869a57

FIX: Refactor 7

4226a2d

FIX: Refactor 8

16efa27

FIX: Refactor 9

11f7dc5

FIX: Refactor 10

cd02f3a

FIX: Refactor 11

f1bd22b

FIX: Import cycle

82a4a3a

FIX: Race condition + add StakingV4DistributeAuctionToWaiting enable …

a0e443a

…epoch

FIX: Staking v4 complete test

e0d68a7

FIX: Roothash mismatch

9d5cee2

FIX: Minor fixes

9de7aec

mariusmihaic marked this pull request as ready for review April 14, 2022 14:13

FIX: Rename StakingV4DistributeAuctionToWaiting epoch

149bd22

sasurobert reviewed Apr 15, 2022

View reviewed changes

mariusmihaic added 5 commits April 15, 2022 14:50

FIX: Package names

1cf4bb0

Merge branch 'feat/liquid-staking' into EN-11927-integration-tests-st…

b32c79f

…aking-v4

FIX: Delete error condition for maxNumNodes decrease

0f3e91a

Merge branch 'feat/liquid-staking' into EN-11927-integration-tests-st…

07ff0da

…aking-v4

FIX: Linter errors

e9b8e72

AdoAdoAdo requested changes Apr 21, 2022

View reviewed changes

FIX: Review findings

f3fe6c5

AdoAdoAdo requested changes Apr 28, 2022

View reviewed changes

epochStart/metachain/systemSCs.go Outdated Show resolved Hide resolved

mariusmihaic added 4 commits April 28, 2022 12:51

FIX: Simplify logic in calcNormRand

063a1c3

Merge branch 'feat/liquid-staking' into EN-11927-integration-tests-st…

2e921fa

…aking-v4 # Conflicts: # process/block/postprocess/feeHandler.go

FIX: Merge conflict

c1f8aec

FIX: Build after merge

a98dcee

AdoAdoAdo approved these changes Apr 29, 2022

View reviewed changes

sasurobert approved these changes May 2, 2022

View reviewed changes

mariusmihaic merged commit a1dad37 into feat/liquid-staking May 2, 2022

mariusmihaic deleted the EN-11927-integration-tests-staking-v4 branch May 2, 2022 13:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Integration tests staking v4 #3951

Integration tests staking v4 #3951

mariusmihaic commented Mar 30, 2022 •

edited

sasurobert Apr 15, 2022

mariusmihaic Apr 15, 2022

sasurobert Apr 15, 2022

mariusmihaic Apr 15, 2022

sasurobert Apr 15, 2022

mariusmihaic Apr 15, 2022

sasurobert Apr 15, 2022

mariusmihaic Apr 15, 2022

sasurobert Apr 15, 2022

mariusmihaic Apr 15, 2022

codecov bot commented Apr 29, 2022 •

edited

sasurobert May 2, 2022

mariusmihaic May 2, 2022

Integration tests staking v4 #3951

Integration tests staking v4 #3951

Conversation

mariusmihaic commented Mar 30, 2022 • edited

What's new

Integration tests

Bugfixes

Refactor

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Apr 29, 2022 • edited

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mariusmihaic commented Mar 30, 2022 •

edited

codecov bot commented Apr 29, 2022 •

edited