Adopt gitflow for Lodestar releases

[dapplion]

Motivation

Our release process is working okay, but has pain points that are causing trouble and more human error.

Current Lodestar flow

• Main branch = master
• Features merged to master
• To trigger rc (release candidate), branch from master to v0.37.x and tag v0.37.0-beta.x
• QA is done on v0.37.x branch
• Fixes on rc are done on v0.37.x, then re-tag
• Once released final v0.37.0 tag is on a branch that is never merged
• Hot-fixes are done on new or existing v0.36.x branch, never merged

However this had some issues:

• Aborted releases left master in awkward version 2 minors ahead of stable. When triggering the release again, we had to rollback master
• Almost all release tags ended in branches not part of the master tree. This caused issues and confusing in tooling since it's not straight-forward to compute the diff between commits that are not direct parents of each other

We can do some minor changes to our release flow to just do a generic gitflow strategy that fixes this points and fits well our needs.

Description

Switch to gitflow release strategy

Read the new RELEASE.md

➡️ I've marked this issue as discussion and only done the rationale part. Please voice your opinion now, if there's consensus I'll go ahead and update tooling and scripts to automate as much as possible.

Key relevant changes:

• Lodestar default branch will change from master to unstable. master will get retired. Lodestar contributors will use the unstable branch in the same way they do master now.

---

@q9f please comment your opinions as you gave excellent feedback on our current strategy ❤️

[dapplion]

Some notes from research on other strategies

• Typescript's workflow for nightly releases. They release once per day to NPM with next tag
  • workflow https://github.com/microsoft/TypeScript/blob/main/.github/workflows/nightly.yaml
  • script https://github.com/microsoft/TypeScript/blob/main/scripts/configurePrerelease.ts
  • npm tags https://www.npmjs.com/package/typescript
• Lighthouse uses a variation of gitflow, where master is stable and develop is unstable. Bors handle merges so things don't look that common on a git tree. Their release process:
  • Decide on release candidate and commit version bump + tag in unstable
  • If all good, merge to stable, tag and fast-forward unstable to stable head
  • To handle fixes during release testing LH establishes a merge freeze, to keep unstable clean. I don't think we should adopt this strategy since we want to release more often and need to merge faster.

[github-actions]

Performance Report

✔️ no performance regression detected

Full benchmark results

Benchmark suite	Current: 6cc6eef	Previous: -	Ratio
BeaconState.hashTreeRoot - No change	536.00 ns/op
BeaconState.hashTreeRoot - 1 full validator	74.076 us/op
BeaconState.hashTreeRoot - 32 full validator	773.87 us/op
BeaconState.hashTreeRoot - 512 full validator	7.9414 ms/op
BeaconState.hashTreeRoot - 1 validator.effectiveBalance	96.688 us/op
BeaconState.hashTreeRoot - 32 validator.effectiveBalance	1.3357 ms/op
BeaconState.hashTreeRoot - 512 validator.effectiveBalance	18.066 ms/op
BeaconState.hashTreeRoot - 1 balances	74.845 us/op
BeaconState.hashTreeRoot - 32 balances	635.98 us/op
BeaconState.hashTreeRoot - 512 balances	6.0870 ms/op
BeaconState.hashTreeRoot - 250000 balances	131.81 ms/op
processSlot - 1 slots	15.754 us/op
processSlot - 32 slots	2.2399 ms/op
getCommitteeAssignments - req 1 vs - 250000 vc	5.7275 ms/op
getCommitteeAssignments - req 100 vs - 250000 vc	8.2835 ms/op
getCommitteeAssignments - req 1000 vs - 250000 vc	8.9480 ms/op
computeProposers - vc 250000	21.003 ms/op
computeEpochShuffling - vc 250000	189.54 ms/op
getNextSyncCommittee - vc 250000	350.02 ms/op
altair processAttestation - 250000 vs - 7PWei normalcase	5.3388 ms/op
altair processAttestation - 250000 vs - 7PWei worstcase	6.6948 ms/op
altair processAttestation - setStatus - 1/6 committees join	238.76 us/op
altair processAttestation - setStatus - 1/3 committees join	448.70 us/op
altair processAttestation - setStatus - 1/2 committees join	642.57 us/op
altair processAttestation - setStatus - 2/3 committees join	810.20 us/op
altair processAttestation - setStatus - 4/5 committees join	1.1323 ms/op
altair processAttestation - setStatus - 100% committees join	1.4079 ms/op
altair processBlock - 250000 vs - 7PWei normalcase	33.140 ms/op
altair processBlock - 250000 vs - 7PWei normalcase hashState	43.285 ms/op
altair processBlock - 250000 vs - 7PWei worstcase	89.778 ms/op
altair processBlock - 250000 vs - 7PWei worstcase hashState	142.63 ms/op
altair processEth1Data - 250000 vs - 7PWei normalcase	1.1039 ms/op
altair processEpoch - mainnet_e81889	648.98 ms/op
mainnet_e81889 - altair beforeProcessEpoch	156.56 ms/op
mainnet_e81889 - altair processJustificationAndFinalization	68.470 us/op
mainnet_e81889 - altair processInactivityUpdates	11.925 ms/op
mainnet_e81889 - altair processRewardsAndPenalties	156.98 ms/op
mainnet_e81889 - altair processRegistryUpdates	13.976 us/op
mainnet_e81889 - altair processSlashings	4.3460 us/op
mainnet_e81889 - altair processEth1DataReset	4.7340 us/op
mainnet_e81889 - altair processEffectiveBalanceUpdates	3.6703 ms/op
mainnet_e81889 - altair processSlashingsReset	20.394 us/op
mainnet_e81889 - altair processRandaoMixesReset	24.705 us/op
mainnet_e81889 - altair processHistoricalRootsUpdate	4.4480 us/op
mainnet_e81889 - altair processParticipationFlagUpdates	11.902 us/op
mainnet_e81889 - altair processSyncCommitteeUpdates	4.4320 us/op
mainnet_e81889 - altair afterProcessEpoch	215.10 ms/op
altair processInactivityUpdates - 250000 normalcase	42.769 ms/op
altair processInactivityUpdates - 250000 worstcase	46.430 ms/op
altair processRewardsAndPenalties - 250000 normalcase	148.72 ms/op
altair processRewardsAndPenalties - 250000 worstcase	113.32 ms/op
altair processSyncCommitteeUpdates - 250000	354.01 ms/op
Tree 40 250000 create	1.0449 s/op
Tree 40 250000 get(125000)	338.69 ns/op
Tree 40 250000 set(125000)	3.1131 us/op
Tree 40 250000 toArray()	35.935 ms/op
Tree 40 250000 iterate all - toArray() + loop	35.928 ms/op
Tree 40 250000 iterate all - get(i)	136.66 ms/op
MutableVector 250000 create	18.011 ms/op
MutableVector 250000 get(125000)	14.303 ns/op
MutableVector 250000 set(125000)	930.43 ns/op
MutableVector 250000 toArray()	6.6172 ms/op
MutableVector 250000 iterate all - toArray() + loop	7.0217 ms/op
MutableVector 250000 iterate all - get(i)	3.4004 ms/op
Array 250000 create	7.0471 ms/op
Array 250000 clone - spread	3.9785 ms/op
Array 250000 get(125000)	1.9140 ns/op
Array 250000 set(125000)	1.9920 ns/op
Array 250000 iterate all - loop	152.74 us/op
effectiveBalanceIncrements clone Uint8Array 300000	234.88 us/op
effectiveBalanceIncrements clone MutableVector 300000	804.00 ns/op
effectiveBalanceIncrements rw all Uint8Array 300000	298.08 us/op
effectiveBalanceIncrements rw all MutableVector 300000	226.55 ms/op
aggregationBits - 2048 els - zipIndexesInBitList	30.331 us/op
regular array get 100000 times	58.911 us/op
wrappedArray get 100000 times	58.383 us/op
arrayWithProxy get 100000 times	42.385 ms/op
ssz.Root.equals	624.00 ns/op
byteArrayEquals	641.00 ns/op
phase0 processBlock - 250000 vs - 7PWei normalcase	4.6165 ms/op
phase0 processBlock - 250000 vs - 7PWei worstcase	58.522 ms/op
phase0 afterProcessEpoch - 250000 vs - 7PWei	200.11 ms/op
phase0 beforeProcessEpoch - 250000 vs - 7PWei	80.963 ms/op
phase0 processEpoch - mainnet_e58758	604.51 ms/op
mainnet_e58758 - phase0 beforeProcessEpoch	303.01 ms/op
mainnet_e58758 - phase0 processJustificationAndFinalization	65.292 us/op
mainnet_e58758 - phase0 processRewardsAndPenalties	83.882 ms/op
mainnet_e58758 - phase0 processRegistryUpdates	34.188 us/op
mainnet_e58758 - phase0 processSlashings	4.1360 us/op
mainnet_e58758 - phase0 processEth1DataReset	3.7870 us/op
mainnet_e58758 - phase0 processEffectiveBalanceUpdates	2.5311 ms/op
mainnet_e58758 - phase0 processSlashingsReset	16.384 us/op
mainnet_e58758 - phase0 processRandaoMixesReset	23.801 us/op
mainnet_e58758 - phase0 processHistoricalRootsUpdate	4.4220 us/op
mainnet_e58758 - phase0 processParticipationRecordUpdates	23.548 us/op
mainnet_e58758 - phase0 afterProcessEpoch	178.71 ms/op
phase0 processEffectiveBalanceUpdates - 250000 normalcase	2.9310 ms/op
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5	2.6877 ms/op
phase0 processRegistryUpdates - 250000 normalcase	27.597 us/op
phase0 processRegistryUpdates - 250000 badcase_full_deposits	530.40 us/op
phase0 processRegistryUpdates - 250000 worstcase 0.5	284.62 ms/op
phase0 getAttestationDeltas - 250000 normalcase	14.889 ms/op
phase0 getAttestationDeltas - 250000 worstcase	14.074 ms/op
phase0 processSlashings - 250000 worstcase	6.6442 ms/op
shuffle list - 16384 els	12.352 ms/op
shuffle list - 250000 els	181.06 ms/op
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei	440.87 us/op
pass gossip attestations to forkchoice per slot	3.8583 ms/op
computeDeltas	4.7657 ms/op
computeProposerBoostScoreFromBalances	926.92 us/op
getPubkeys - index2pubkey - req 1000 vs - 250000 vc	2.6424 ms/op
getPubkeys - validatorsArr - req 1000 vs - 250000 vc	92.901 us/op
BLS verify - blst-native	2.3689 ms/op
BLS verifyMultipleSignatures 3 - blst-native	4.8139 ms/op
BLS verifyMultipleSignatures 8 - blst-native	10.313 ms/op
BLS verifyMultipleSignatures 32 - blst-native	36.702 ms/op
BLS aggregatePubkeys 32 - blst-native	51.307 us/op
BLS aggregatePubkeys 128 - blst-native	196.73 us/op
getAttestationsForBlock	78.316 ms/op
isKnown best case - 1 super set check	536.00 ns/op
isKnown normal case - 2 super set checks	528.00 ns/op
isKnown worse case - 16 super set checks	514.00 ns/op
CheckpointStateCache - add get delete	13.944 us/op
validate gossip signedAggregateAndProof - struct	5.2218 ms/op
validate gossip attestation - struct	2.4638 ms/op
altair verifyImport mainnet_s3766816:31	7.4890 s/op
pickEth1Vote - no votes	2.6479 ms/op
pickEth1Vote - max votes	32.352 ms/op
pickEth1Vote - Eth1Data hashTreeRoot value x2048	16.412 ms/op
pickEth1Vote - Eth1Data hashTreeRoot tree x2048	31.834 ms/op
pickEth1Vote - Eth1Data fastSerialize value x2048	1.8777 ms/op
pickEth1Vote - Eth1Data fastSerialize tree x2048	17.377 ms/op
bytes32 toHexString	1.2940 us/op
bytes32 Buffer.toString(hex)	843.00 ns/op
bytes32 Buffer.toString(hex) from Uint8Array	1.1240 us/op
bytes32 Buffer.toString(hex) + 0x	837.00 ns/op
Object access 1 prop	0.44200 ns/op
Map access 1 prop	0.37000 ns/op
Object get x1000	17.400 ns/op
Map get x1000	1.0070 ns/op
Object set x1000	115.76 ns/op
Map set x1000	81.912 ns/op
Return object 10000 times	0.43890 ns/op
Throw Error 10000 times	7.0958 us/op
enrSubnets - fastDeserialize 64 bits	3.0990 us/op
enrSubnets - ssz BitVector 64 bits	1.0660 us/op
enrSubnets - fastDeserialize 4 bits	441.00 ns/op
enrSubnets - ssz BitVector 4 bits	1.0850 us/op
prioritizePeers score -10:0 att 32-0.1 sync 2-0	113.36 us/op
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25	162.90 us/op
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5	276.42 us/op
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75	616.74 us/op
prioritizePeers score 0:0 att 64-1 sync 4-1	603.03 us/op
RateTracker 1000000 limit, 1 obj count per request	227.03 ns/op
RateTracker 1000000 limit, 2 obj count per request	165.13 ns/op
RateTracker 1000000 limit, 4 obj count per request	137.41 ns/op
RateTracker 1000000 limit, 8 obj count per request	126.40 ns/op
RateTracker with prune	5.0120 us/op
array of 16000 items push then shift	5.9287 us/op
LinkedList of 16000 items push then shift	31.443 ns/op
array of 16000 items push then pop	263.65 ns/op
LinkedList of 16000 items push then pop	26.794 ns/op
array of 24000 items push then shift	8.8301 us/op
LinkedList of 24000 items push then shift	32.260 ns/op
array of 24000 items push then pop	242.68 ns/op
LinkedList of 24000 items push then pop	25.898 ns/op
intersect bitArray bitLen 8	13.062 ns/op
intersect array and set length 8	191.36 ns/op
intersect bitArray bitLen 128	76.670 ns/op
intersect array and set length 128	2.5314 us/op

by benchmarkbot/action

[dapplion]

From Cayman chat:

• Strongly in favor of changing to stable + unstable.
• Trigger release on tag to stable. But in CI check that: tag matches version in source + commit is latest in branch. This prevents bad rogue versions and errors
• Document pull request strategies also: how to trigger releases specifically, and which scripts to run
• Release branch should be named the exact semver (0.38.0), immediately after branching unstable commit bump to 0.38.0-rc.0. Then when ready, bump and commit version to 0.38.0 and merge to master. So final PR contains correct final version and all the changes.
• Add a step to merge stable into unstable in CI so it's done automatically. Either commit and push directly or create a PR

[philknows]

LGTM! I'm glad this was discussed and planned out. Any issues with having this set up before the first block audit on June 7th? It would be good to have this looked at, especially any parts of the build process that are automated.

[q9f]

LGTM

(didn't test the CI though)