refactor(reqresp)!: support byte based handlers

[nazarhussain]

Motivation

Update reqresp package to only support the bytes for handler.

Description

To make network more flexible to work form the worker, need to make the reqresp more consistent.

Closes #5378

Steps to test or reproduce

• Run all tests

[github-actions]

Performance Report

✔️ no performance regression detected

Full benchmark results

Benchmark suite	Current: ca96e86	Previous: 342eb17	Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc	906.83 us/op	774.49 us/op	1.17
getPubkeys - validatorsArr - req 1000 vs - 250000 vc	44.183 us/op	45.164 us/op	0.98
BLS verify - blst-native	1.1936 ms/op	1.2118 ms/op	0.98
BLS verifyMultipleSignatures 3 - blst-native	2.4081 ms/op	2.4591 ms/op	0.98
BLS verifyMultipleSignatures 8 - blst-native	5.1696 ms/op	5.2816 ms/op	0.98
BLS verifyMultipleSignatures 32 - blst-native	18.685 ms/op	19.178 ms/op	0.97
BLS aggregatePubkeys 32 - blst-native	25.034 us/op	25.499 us/op	0.98
BLS aggregatePubkeys 128 - blst-native	97.509 us/op	99.933 us/op	0.98
getAttestationsForBlock	59.283 ms/op	52.213 ms/op	1.14
isKnown best case - 1 super set check	256.00 ns/op	266.00 ns/op	0.96
isKnown normal case - 2 super set checks	255.00 ns/op	256.00 ns/op	1.00
isKnown worse case - 16 super set checks	248.00 ns/op	255.00 ns/op	0.97
CheckpointStateCache - add get delete	4.9260 us/op	4.8330 us/op	1.02
validate gossip signedAggregateAndProof - struct	2.7470 ms/op	2.7481 ms/op	1.00
validate gossip attestation - struct	1.3180 ms/op	1.3127 ms/op	1.00
pickEth1Vote - no votes	1.2155 ms/op	1.2229 ms/op	0.99
pickEth1Vote - max votes	10.511 ms/op	9.7636 ms/op	1.08
pickEth1Vote - Eth1Data hashTreeRoot value x2048	8.9589 ms/op	8.5919 ms/op	1.04
pickEth1Vote - Eth1Data hashTreeRoot tree x2048	18.687 ms/op	14.625 ms/op	1.28
pickEth1Vote - Eth1Data fastSerialize value x2048	634.00 us/op	621.67 us/op	1.02
pickEth1Vote - Eth1Data fastSerialize tree x2048	9.1933 ms/op	7.8334 ms/op	1.17
bytes32 toHexString	498.00 ns/op	473.00 ns/op	1.05
bytes32 Buffer.toString(hex)	362.00 ns/op	334.00 ns/op	1.08
bytes32 Buffer.toString(hex) from Uint8Array	554.00 ns/op	528.00 ns/op	1.05
bytes32 Buffer.toString(hex) + 0x	367.00 ns/op	339.00 ns/op	1.08
Object access 1 prop	0.17300 ns/op	0.16400 ns/op	1.05
Map access 1 prop	0.16600 ns/op	0.16600 ns/op	1.00
Object get x1000	6.2940 ns/op	6.5590 ns/op	0.96
Map get x1000	0.62700 ns/op	0.59600 ns/op	1.05
Object set x1000	51.817 ns/op	49.886 ns/op	1.04
Map set x1000	42.730 ns/op	41.565 ns/op	1.03
Return object 10000 times	0.23390 ns/op	0.22960 ns/op	1.02
Throw Error 10000 times	4.1534 us/op	4.1125 us/op	1.01
fastMsgIdFn sha256 / 200 bytes	3.4230 us/op	3.3220 us/op	1.03
fastMsgIdFn h32 xxhash / 200 bytes	284.00 ns/op	279.00 ns/op	1.02
fastMsgIdFn h64 xxhash / 200 bytes	401.00 ns/op	384.00 ns/op	1.04
fastMsgIdFn sha256 / 1000 bytes	11.486 us/op	11.294 us/op	1.02
fastMsgIdFn h32 xxhash / 1000 bytes	408.00 ns/op	398.00 ns/op	1.03
fastMsgIdFn h64 xxhash / 1000 bytes	469.00 ns/op	450.00 ns/op	1.04
fastMsgIdFn sha256 / 10000 bytes	103.03 us/op	100.54 us/op	1.02
fastMsgIdFn h32 xxhash / 10000 bytes	1.9040 us/op	1.8630 us/op	1.02
fastMsgIdFn h64 xxhash / 10000 bytes	1.3650 us/op	1.3190 us/op	1.03
enrSubnets - fastDeserialize 64 bits	1.2760 us/op	1.3030 us/op	0.98
enrSubnets - ssz BitVector 64 bits	495.00 ns/op	494.00 ns/op	1.00
enrSubnets - fastDeserialize 4 bits	179.00 ns/op	182.00 ns/op	0.98
enrSubnets - ssz BitVector 4 bits	504.00 ns/op	493.00 ns/op	1.02
prioritizePeers score -10:0 att 32-0.1 sync 2-0	127.05 us/op	107.99 us/op	1.18
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25	122.88 us/op	132.73 us/op	0.93
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5	158.42 us/op	160.18 us/op	0.99
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75	285.41 us/op	293.66 us/op	0.97
prioritizePeers score 0:0 att 64-1 sync 4-1	346.90 us/op	352.25 us/op	0.98
array of 16000 items push then shift	1.5541 us/op	1.6231 us/op	0.96
LinkedList of 16000 items push then shift	8.5430 ns/op	8.8630 ns/op	0.96
array of 16000 items push then pop	88.826 ns/op	76.180 ns/op	1.17
LinkedList of 16000 items push then pop	8.5870 ns/op	8.5300 ns/op	1.01
array of 24000 items push then shift	2.3855 us/op	2.3464 us/op	1.02
LinkedList of 24000 items push then shift	8.8330 ns/op	8.7710 ns/op	1.01
array of 24000 items push then pop	71.068 ns/op	76.576 ns/op	0.93
LinkedList of 24000 items push then pop	9.6110 ns/op	8.4790 ns/op	1.13
intersect bitArray bitLen 8	13.644 ns/op	13.357 ns/op	1.02
intersect array and set length 8	78.280 ns/op	77.615 ns/op	1.01
intersect bitArray bitLen 128	44.519 ns/op	43.977 ns/op	1.01
intersect array and set length 128	1.1102 us/op	1.0492 us/op	1.06
Buffer.concat 32 items	2.6360 us/op	2.8760 us/op	0.92
Uint8Array.set 32 items	2.2270 us/op	2.6700 us/op	0.83
pass gossip attestations to forkchoice per slot	2.9092 ms/op	2.6569 ms/op	1.09
computeDeltas	4.1401 ms/op	3.4910 ms/op	1.19
computeProposerBoostScoreFromBalances	1.7716 ms/op	1.7839 ms/op	0.99
altair processAttestation - 250000 vs - 7PWei normalcase	2.4206 ms/op	2.0570 ms/op	1.18
altair processAttestation - 250000 vs - 7PWei worstcase	3.7530 ms/op	3.2688 ms/op	1.15
altair processAttestation - setStatus - 1/6 committees join	143.33 us/op	139.85 us/op	1.02
altair processAttestation - setStatus - 1/3 committees join	283.65 us/op	267.91 us/op	1.06
altair processAttestation - setStatus - 1/2 committees join	393.48 us/op	364.82 us/op	1.08
altair processAttestation - setStatus - 2/3 committees join	478.66 us/op	458.17 us/op	1.04
altair processAttestation - setStatus - 4/5 committees join	670.95 us/op	631.12 us/op	1.06
altair processAttestation - setStatus - 100% committees join	780.34 us/op	730.55 us/op	1.07
altair processBlock - 250000 vs - 7PWei normalcase	19.405 ms/op	17.486 ms/op	1.11
altair processBlock - 250000 vs - 7PWei normalcase hashState	28.510 ms/op	27.499 ms/op	1.04
altair processBlock - 250000 vs - 7PWei worstcase	63.064 ms/op	45.173 ms/op	1.40
altair processBlock - 250000 vs - 7PWei worstcase hashState	82.569 ms/op	71.907 ms/op	1.15
phase0 processBlock - 250000 vs - 7PWei normalcase	2.0603 ms/op	1.9876 ms/op	1.04
phase0 processBlock - 250000 vs - 7PWei worstcase	28.683 ms/op	27.791 ms/op	1.03
altair processEth1Data - 250000 vs - 7PWei normalcase	476.71 us/op	468.15 us/op	1.02
vc - 250000 eb 1 eth1 1 we 0 wn 0 - smpl 15	8.4850 us/op	6.6330 us/op	1.28
vc - 250000 eb 0.95 eth1 0.1 we 0.05 wn 0 - smpl 219	21.070 us/op	19.366 us/op	1.09
vc - 250000 eb 0.95 eth1 0.3 we 0.05 wn 0 - smpl 42	11.999 us/op	8.3400 us/op	1.44
vc - 250000 eb 0.95 eth1 0.7 we 0.05 wn 0 - smpl 18	8.3900 us/op	6.5540 us/op	1.28
vc - 250000 eb 0.1 eth1 0.1 we 0 wn 0 - smpl 1020	114.12 us/op	74.439 us/op	1.53
vc - 250000 eb 0.03 eth1 0.03 we 0 wn 0 - smpl 11777	777.95 us/op	623.58 us/op	1.25
vc - 250000 eb 0.01 eth1 0.01 we 0 wn 0 - smpl 16384	1.0380 ms/op	844.03 us/op	1.23
vc - 250000 eb 0 eth1 0 we 0 wn 0 - smpl 16384	1.0165 ms/op	872.80 us/op	1.16
vc - 250000 eb 0 eth1 0 we 0 wn 0 nocache - smpl 16384	2.4278 ms/op	2.8998 ms/op	0.84
vc - 250000 eb 0 eth1 1 we 0 wn 0 - smpl 16384	2.9530 ms/op	1.5554 ms/op	1.90
vc - 250000 eb 0 eth1 1 we 0 wn 0 nocache - smpl 16384	3.9260 ms/op	3.5525 ms/op	1.11
Tree 40 250000 create	325.59 ms/op	288.96 ms/op	1.13
Tree 40 250000 get(125000)	195.33 ns/op	175.13 ns/op	1.12
Tree 40 250000 set(125000)	1.0073 us/op	890.95 ns/op	1.13
Tree 40 250000 toArray()	21.071 ms/op	16.969 ms/op	1.24
Tree 40 250000 iterate all - toArray() + loop	20.097 ms/op	17.092 ms/op	1.18
Tree 40 250000 iterate all - get(i)	70.162 ms/op	67.746 ms/op	1.04
MutableVector 250000 create	13.112 ms/op	10.670 ms/op	1.23
MutableVector 250000 get(125000)	6.5440 ns/op	6.2320 ns/op	1.05
MutableVector 250000 set(125000)	249.80 ns/op	250.34 ns/op	1.00
MutableVector 250000 toArray()	2.9734 ms/op	2.7143 ms/op	1.10
MutableVector 250000 iterate all - toArray() + loop	2.9774 ms/op	2.8700 ms/op	1.04
MutableVector 250000 iterate all - get(i)	1.5857 ms/op	1.5116 ms/op	1.05
Array 250000 create	3.4178 ms/op	2.5368 ms/op	1.35
Array 250000 clone - spread	1.2779 ms/op	1.3141 ms/op	0.97
Array 250000 get(125000)	0.63200 ns/op	0.63800 ns/op	0.99
Array 250000 set(125000)	0.72600 ns/op	0.72200 ns/op	1.01
Array 250000 iterate all - loop	93.735 us/op	97.724 us/op	0.96
effectiveBalanceIncrements clone Uint8Array 300000	51.711 us/op	32.449 us/op	1.59
effectiveBalanceIncrements clone MutableVector 300000	396.00 ns/op	427.00 ns/op	0.93
effectiveBalanceIncrements rw all Uint8Array 300000	171.65 us/op	168.95 us/op	1.02
effectiveBalanceIncrements rw all MutableVector 300000	93.881 ms/op	87.130 ms/op	1.08
phase0 afterProcessEpoch - 250000 vs - 7PWei	117.69 ms/op	112.90 ms/op	1.04
phase0 beforeProcessEpoch - 250000 vs - 7PWei	35.713 ms/op	33.190 ms/op	1.08
altair processEpoch - mainnet_e81889	348.95 ms/op	303.67 ms/op	1.15
mainnet_e81889 - altair beforeProcessEpoch	67.912 ms/op	67.461 ms/op	1.01
mainnet_e81889 - altair processJustificationAndFinalization	17.853 us/op	16.990 us/op	1.05
mainnet_e81889 - altair processInactivityUpdates	6.8549 ms/op	4.9499 ms/op	1.38
mainnet_e81889 - altair processRewardsAndPenalties	72.045 ms/op	51.466 ms/op	1.40
mainnet_e81889 - altair processRegistryUpdates	4.0270 us/op	2.4510 us/op	1.64
mainnet_e81889 - altair processSlashings	957.00 ns/op	525.00 ns/op	1.82
mainnet_e81889 - altair processEth1DataReset	633.00 ns/op	523.00 ns/op	1.21
mainnet_e81889 - altair processEffectiveBalanceUpdates	1.4973 ms/op	1.2673 ms/op	1.18
mainnet_e81889 - altair processSlashingsReset	4.3010 us/op	5.1430 us/op	0.84
mainnet_e81889 - altair processRandaoMixesReset	5.1370 us/op	4.2870 us/op	1.20
mainnet_e81889 - altair processHistoricalRootsUpdate	1.2870 us/op	576.00 ns/op	2.23
mainnet_e81889 - altair processParticipationFlagUpdates	3.6230 us/op	2.3880 us/op	1.52
mainnet_e81889 - altair processSyncCommitteeUpdates	661.00 ns/op	505.00 ns/op	1.31
mainnet_e81889 - altair afterProcessEpoch	132.62 ms/op	118.74 ms/op	1.12
phase0 processEpoch - mainnet_e58758	409.88 ms/op	323.47 ms/op	1.27
mainnet_e58758 - phase0 beforeProcessEpoch	168.21 ms/op	130.48 ms/op	1.29
mainnet_e58758 - phase0 processJustificationAndFinalization	19.277 us/op	17.506 us/op	1.10
mainnet_e58758 - phase0 processRewardsAndPenalties	66.973 ms/op	53.321 ms/op	1.26
mainnet_e58758 - phase0 processRegistryUpdates	9.7460 us/op	8.1060 us/op	1.20
mainnet_e58758 - phase0 processSlashings	928.00 ns/op	492.00 ns/op	1.89
mainnet_e58758 - phase0 processEth1DataReset	919.00 ns/op	524.00 ns/op	1.75
mainnet_e58758 - phase0 processEffectiveBalanceUpdates	1.2729 ms/op	1.0610 ms/op	1.20
mainnet_e58758 - phase0 processSlashingsReset	10.048 us/op	2.9120 us/op	3.45
mainnet_e58758 - phase0 processRandaoMixesReset	6.6700 us/op	4.2670 us/op	1.56
mainnet_e58758 - phase0 processHistoricalRootsUpdate	1.1050 us/op	563.00 ns/op	1.96
mainnet_e58758 - phase0 processParticipationRecordUpdates	4.1880 us/op	4.0910 us/op	1.02
mainnet_e58758 - phase0 afterProcessEpoch	105.44 ms/op	100.48 ms/op	1.05
phase0 processEffectiveBalanceUpdates - 250000 normalcase	1.2828 ms/op	1.3024 ms/op	0.98
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5	2.0123 ms/op	1.4412 ms/op	1.40
altair processInactivityUpdates - 250000 normalcase	28.766 ms/op	21.393 ms/op	1.34
altair processInactivityUpdates - 250000 worstcase	31.062 ms/op	20.680 ms/op	1.50
phase0 processRegistryUpdates - 250000 normalcase	7.8380 us/op	8.8090 us/op	0.89
phase0 processRegistryUpdates - 250000 badcase_full_deposits	350.65 us/op	276.40 us/op	1.27
phase0 processRegistryUpdates - 250000 worstcase 0.5	146.52 ms/op	115.15 ms/op	1.27
altair processRewardsAndPenalties - 250000 normalcase	72.003 ms/op	53.935 ms/op	1.34
altair processRewardsAndPenalties - 250000 worstcase	77.259 ms/op	50.708 ms/op	1.52
phase0 getAttestationDeltas - 250000 normalcase	9.5367 ms/op	8.7945 ms/op	1.08
phase0 getAttestationDeltas - 250000 worstcase	7.9464 ms/op	6.9157 ms/op	1.15
phase0 processSlashings - 250000 worstcase	3.6663 ms/op	3.7887 ms/op	0.97
altair processSyncCommitteeUpdates - 250000	187.18 ms/op	182.68 ms/op	1.02
BeaconState.hashTreeRoot - No change	294.00 ns/op	276.00 ns/op	1.07
BeaconState.hashTreeRoot - 1 full validator	54.750 us/op	51.719 us/op	1.06
BeaconState.hashTreeRoot - 32 full validator	584.05 us/op	579.33 us/op	1.01
BeaconState.hashTreeRoot - 512 full validator	6.0023 ms/op	5.3597 ms/op	1.12
BeaconState.hashTreeRoot - 1 validator.effectiveBalance	69.699 us/op	64.654 us/op	1.08
BeaconState.hashTreeRoot - 32 validator.effectiveBalance	929.02 us/op	906.26 us/op	1.03
BeaconState.hashTreeRoot - 512 validator.effectiveBalance	12.933 ms/op	11.535 ms/op	1.12
BeaconState.hashTreeRoot - 1 balances	56.583 us/op	49.910 us/op	1.13
BeaconState.hashTreeRoot - 32 balances	498.75 us/op	482.96 us/op	1.03
BeaconState.hashTreeRoot - 512 balances	5.0713 ms/op	4.5964 ms/op	1.10
BeaconState.hashTreeRoot - 250000 balances	80.434 ms/op	75.403 ms/op	1.07
aggregationBits - 2048 els - zipIndexesInBitList	21.498 us/op	16.587 us/op	1.30
regular array get 100000 times	36.441 us/op	33.254 us/op	1.10
wrappedArray get 100000 times	33.784 us/op	33.317 us/op	1.01
arrayWithProxy get 100000 times	16.576 ms/op	15.993 ms/op	1.04
ssz.Root.equals	590.00 ns/op	576.00 ns/op	1.02
byteArrayEquals	581.00 ns/op	566.00 ns/op	1.03
shuffle list - 16384 els	7.2289 ms/op	7.1177 ms/op	1.02
shuffle list - 250000 els	104.32 ms/op	102.44 ms/op	1.02
processSlot - 1 slots	9.5770 us/op	9.5860 us/op	1.00
processSlot - 32 slots	1.4687 ms/op	1.3944 ms/op	1.05
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei	38.891 ms/op	33.116 ms/op	1.17
getCommitteeAssignments - req 1 vs - 250000 vc	3.0131 ms/op	3.0009 ms/op	1.00
getCommitteeAssignments - req 100 vs - 250000 vc	4.2610 ms/op	4.2415 ms/op	1.00
getCommitteeAssignments - req 1000 vs - 250000 vc	4.6920 ms/op	4.5154 ms/op	1.04
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei	5.5500 ns/op	4.4800 ns/op	1.24
state getBlockRootAtSlot - 250000 vs - 7PWei	827.33 ns/op	677.54 ns/op	1.22
computeProposers - vc 250000	11.757 ms/op	11.440 ms/op	1.03
computeEpochShuffling - vc 250000	108.88 ms/op	107.43 ms/op	1.01
getNextSyncCommittee - vc 250000	185.24 ms/op	190.98 ms/op	0.97
computeSigningRoot for AttestationData	14.551 us/op	13.946 us/op	1.04
hash AttestationData serialized data then Buffer.toString(base64)	2.6323 us/op	2.4835 us/op	1.06
toHexString serialized data	1.3596 us/op	1.2654 us/op	1.07
Buffer.toString(base64)	389.73 ns/op	339.74 ns/op	1.15

by benchmarkbot/action

[nflaig]

Regarding the PR title, if a change is specific to a package, we could use the package name as scope e.g. refactor(reqresp). This allows to quickly search for commits for a specific subpackage.

[dapplion]

Pushed a significant refactor to align this to what is eventually needed to work within a worker thread. We really need to have this asap so please review correctness and not organizational approaches that we can review latter after landing the worker thread issue

[twoeths]

Generally this looks good to me, there's a typo and a question about the rate limiter change

Implemented changes requested

[wemeetagain]

🎉 This PR is included in v1.9.0 🎉