ShufflingRef approach to next-epoch validator duty calculation/prediction

[tersec]

No description provided.

[github-actions]

Unit Test Results

         9 files  ±0    1 089 suites  ±0   37m 55s ⏱️ -1s
  3 887 tests +1    3 590 ✔️ +1  297 💤 ±0  0 ❌ ±0 
15 958 runs  +3  15 635 ✔️ +3  323 💤 ±0  0 ❌ ±0 

Results for commit b04e02d. ± Comparison against base commit a4cf203.

♻️ This comment has been updated with latest results.

[tersec]

#5489 is a version of this where it checks during runtime, not only during unit tests, whether the ShufflingRef and EpochRef match, i.e. does not remove b04e02d

[tersec]

From a variation of that (FOO1 showing the EpochRef-based calculation and FOO2 showing the ShufflingRef-based calculation):

INF 2023-10-09 22:40:25.768+00:00 State replayed                             topics="chaindag" blocks=32 slots=32 current=83d2f2a3:7505503@7505504 ancestor=83d2f2a3:7505503@7505504 target=59d685ac:7505535@7505536 ancestorStateRoot=d7b08747 targetStateRoot=65bd896a found=true assignDur=26us790ns replayDur=9s20ms658us982ns
INF 2023-10-09 22:40:27.731+00:00 State replayed                             topics="chaindag" blocks=32 slots=32 current=59d685ac:7505535@7505536 ancestor=59d685ac:7505535@7505536 target=246b90ba:7505567@7505568 ancestorStateRoot=65bd896a targetStateRoot=35677ed9 found=true assignDur=12us94ns replayDur=1s83ms181us721ns
INF 2023-10-09 22:40:32.151+00:00 State replayed                             topics="chaindag" blocks=0 slots=32 current=8f011e7a:7505597@7505600 ancestor=8f011e7a:7505597@7505600 target=8f011e7a:7505597@7505632 ancestorStateRoot=ef90fc55 targetStateRoot=4599ea71 found=true assignDur=4us779ns replayDur=1s685ms370us813ns
NOT 2023-10-09 22:40:32.202+00:00 FOO1                                       topics="beacnde" epoch_Ref="[ok(953091), ok(222440), ok(193489), ok(939556), ok(612943), ok(872539), ok(944288), ok(176572), ok(825825), ok(475512), ok(396632), ok(288849), ok(599862), ok(932886), ok(918974), ok(627571), ok(206686), ok(698978), ok(589829), ok(753410), ok(138074), ok(715015), ok(224758), ok(307349), ok(166381), ok(796028), ok(890531), ok(601702), ok(746401), ok(406841), ok(309854), ok(22936)]"
NOT 2023-10-09 22:40:32.203+00:00 FOO2                                       topics="beacnde" shuffling="@[ok(953091), ok(222440), ok(193489), ok(939556), ok(612943), ok(872539), ok(944288), ok(176572), ok(825825), ok(475512), ok(396632), ok(288849), ok(599862), ok(932886), ok(918974), ok(627571), ok(206686), ok(698978), ok(589829), ok(753410), ok(138074), ok(715015), ok(224758), ok(307349), ok(166381), ok(796028), ok(890531), ok(601702), ok(746401), ok(406841), ok(309854), ok(22936)]"
NOT 2023-10-09 22:41:23.348+00:00 FOO1                                       topics="beacnde" epoch_Ref="[ok(937707), ok(810518), ok(15330), ok(121618), ok(786266), ok(642224), ok(301931), ok(781044), ok(338981), ok(786564), ok(605726), ok(197221), ok(178260), ok(237342), ok(100079), ok(439734), ok(677182), ok(477713), ok(132654), ok(176724), ok(113208), ok(258002), ok(25767), ok(198520), ok(745331), ok(378500), ok(784624), ok(348449), ok(826820), ok(904318), ok(414925), ok(522493)]"
NOT 2023-10-09 22:41:23.348+00:00 FOO2                                       topics="beacnde" shuffling="@[ok(937707), ok(810518), ok(15330), ok(121618), ok(786266), ok(642224), ok(301931), ok(781044), ok(338981), ok(786564), ok(605726), ok(197221), ok(178260), ok(237342), ok(100079), ok(439734), ok(677182), ok(477713), ok(132654), ok(176724), ok(113208), ok(258002), ok(25767), ok(198520), ok(745331), ok(378500), ok(784624), ok(348449), ok(826820), ok(904318), ok(414925), ok(522493)]"
INF 2023-10-09 22:46:59.979+00:00 State replayed                             topics="chaindag" blocks=0 slots=32 current=5221e710:7505631@7505632 ancestor=50ced3ea:7505632 target=50ced3ea:7505632@7505664 ancestorStateRoot=1f3b2557 targetStateRoot=d8493c2e found=true assignDur=67ms440us198ns replayDur=2s157ms938us603ns
NOT 2023-10-09 22:47:00.031+00:00 FOO1                                       topics="beacnde" epoch_Ref="[ok(695347), ok(287545), ok(492728), ok(840671), ok(813981), ok(741952), ok(696844), ok(798655), ok(665156), ok(585530), ok(820199), ok(562458), ok(600969), ok(805429), ok(331004), ok(187625), ok(111352), ok(761359), ok(289862), ok(734192), ok(3185), ok(905159), ok(896915), ok(193477), ok(439619), ok(6753), ok(432924), ok(715712), ok(698056), ok(338187), ok(6932), ok(873210)]"
NOT 2023-10-09 22:47:00.031+00:00 FOO2                                       topics="beacnde" shuffling="@[ok(695347), ok(287545), ok(492728), ok(840671), ok(813981), ok(741952), ok(696844), ok(798655), ok(665156), ok(585530), ok(820199), ok(562458), ok(600969), ok(805429), ok(331004), ok(187625), ok(111352), ok(761359), ok(289862), ok(734192), ok(3185), ok(905159), ok(896915), ok(193477), ok(439619), ok(6753), ok(432924), ok(715712), ok(698056), ok(338187), ok(6932), ok(873210)]"
INF 2023-10-09 22:53:23.780+00:00 State replayed                             topics="chaindag" blocks=0 slots=32 current=68d2e578:7505663@7505664 ancestor=a48bd094:7505664 target=a48bd094:7505664@7505696 ancestorStateRoot=743e18f6 targetStateRoot=afeedba5 found=true assignDur=17ms644us144ns replayDur=2s223ms60us413ns
NOT 2023-10-09 22:53:23.831+00:00 FOO1                                       topics="beacnde" epoch_Ref="[ok(620836), ok(359396), ok(489055), ok(413083), ok(252120), ok(687999), ok(830337), ok(449478), ok(258862), ok(673046), ok(246526), ok(288978), ok(215323), ok(359672), ok(581132), ok(742098), ok(883231), ok(778729), ok(500595), ok(610804), ok(900743), ok(906514), ok(174842), ok(742007), ok(332068), ok(487397), ok(713285), ok(276350), ok(488353), ok(743609), ok(476740), ok(621845)]"
NOT 2023-10-09 22:53:23.831+00:00 FOO2                                       topics="beacnde" shuffling="@[ok(620836), ok(359396), ok(489055), ok(413083), ok(252120), ok(687999), ok(830337), ok(449478), ok(258862), ok(673046), ok(246526), ok(288978), ok(215323), ok(359672), ok(581132), ok(742098), ok(883231), ok(778729), ok(500595), ok(610804), ok(900743), ok(906514), ok(174842), ok(742007), ok(332068), ok(487397), ok(713285), ok(276350), ok(488353), ok(743609), ok(476740), ok(621845)]"
INF 2023-10-09 22:59:46.698+00:00 State replayed                             topics="chaindag" blocks=0 slots=32 current=d172366a:7505695@7505696 ancestor=a3c1f5c5:7505696 target=a3c1f5c5:7505696@7505728 ancestorStateRoot=0ebc2c10 targetStateRoot=0f46e35a found=true assignDur=14ms611us307ns replayDur=1s451ms130us407ns
NOT 2023-10-09 22:59:46.744+00:00 FOO1                                       topics="beacnde" epoch_Ref="[ok(267207), ok(585738), ok(718282), ok(35254), ok(300176), ok(698628), ok(658230), ok(817033), ok(543385), ok(281677), ok(813546), ok(285403), ok(140281), ok(734882), ok(610379), ok(241536), ok(558926), ok(258587), ok(832618), ok(305176), ok(480909), ok(754290), ok(665732), ok(678148), ok(753526), ok(175610), ok(779447), ok(930284), ok(492593), ok(60413), ok(811387), ok(956058)]"
NOT 2023-10-09 22:59:46.744+00:00 FOO2                                       topics="beacnde" shuffling="@[ok(267207), ok(585738), ok(718282), ok(35254), ok(300176), ok(698628), ok(658230), ok(817033), ok(543385), ok(281677), ok(813546), ok(285403), ok(140281), ok(734882), ok(610379), ok(241536), ok(558926), ok(258587), ok(832618), ok(305176), ok(480909), ok(754290), ok(665732), ok(678148), ok(753526), ok(175610), ok(779447), ok(930284), ok(492593), ok(60413), ok(811387), ok(956058)]"
INF 2023-10-09 23:06:10.792+00:00 State replayed                             topics="chaindag" blocks=0 slots=32 current=3bf51600:7505727@7505728 ancestor=6065bad9:7505728 target=6065bad9:7505728@7505760 ancestorStateRoot=5fae1dbc targetStateRoot=bc89827f found=true assignDur=14ms742us528ns replayDur=1s444ms38us93ns
NOT 2023-10-09 23:06:10.845+00:00 FOO1                                       topics="beacnde" epoch_Ref="[ok(337463), ok(458288), ok(216492), ok(411740), ok(930994), ok(271217), ok(809622), ok(39192), ok(166428), ok(143096), ok(11993), ok(701459), ok(354792), ok(951912), ok(22622), ok(901813), ok(222540), ok(171557), ok(572312), ok(108523), ok(707964), ok(195288), ok(603650), ok(908572), ok(643546), ok(237979), ok(487423), ok(940172), ok(776415), ok(266988), ok(744591), ok(12817)]"
NOT 2023-10-09 23:06:10.845+00:00 FOO2                                       topics="beacnde" shuffling="@[ok(337463), ok(458288), ok(216492), ok(411740), ok(930994), ok(271217), ok(809622), ok(39192), ok(166428), ok(143096), ok(11993), ok(701459), ok(354792), ok(951912), ok(22622), ok(901813), ok(222540), ok(171557), ok(572312), ok(108523), ok(707964), ok(195288), ok(603650), ok(908572), ok(643546), ok(237979), ok(487423), ok(940172), ok(776415), ok(266988), ok(744591), ok(12817)]"
INF 2023-10-09 23:12:35.742+00:00 State replayed                             topics="chaindag" blocks=0 slots=32 current=d8e67f73:7505759@7505760 ancestor=a3e5d882:7505760 target=a3e5d882:7505760@7505792 ancestorStateRoot=7e3d9fe3 targetStateRoot=74e2f0d0 found=true assignDur=73ms707us253ns replayDur=2s2ms723us304ns
NOT 2023-10-09 23:12:35.823+00:00 FOO1                                       topics="beacnde" epoch_Ref="[ok(544751), ok(82325), ok(707949), ok(321492), ok(557852), ok(960138), ok(508165), ok(742863), ok(323644), ok(84138), ok(580373), ok(785309), ok(903450), ok(226392), ok(641900), ok(759033), ok(601255), ok(855624), ok(197567), ok(654421), ok(373208), ok(204879), ok(840359), ok(700627), ok(241119), ok(557205), ok(294556), ok(517776), ok(951405), ok(789960), ok(365379), ok(331524)]"
NOT 2023-10-09 23:12:35.823+00:00 FOO2                                       topics="beacnde" shuffling="@[ok(544751), ok(82325), ok(707949), ok(321492), ok(557852), ok(960138), ok(508165), ok(742863), ok(323644), ok(84138), ok(580373), ok(785309), ok(903450), ok(226392), ok(641900), ok(759033), ok(601255), ok(855624), ok(197567), ok(654421), ok(373208), ok(204879), ok(840359), ok(700627), ok(241119), ok(557205), ok(294556), ok(517776), ok(951405), ok(789960), ok(365379), ok(331524)]"
INF 2023-10-09 23:19:00.468+00:00 State replayed                             topics="chaindag" blocks=0 slots=32 current=b9757960:7505791@7505792 ancestor=77341cd7:7505792 target=77341cd7:7505792@7505824 ancestorStateRoot=d54dda89 targetStateRoot=6b7d9f77 found=true assignDur=15ms920us87ns replayDur=1s923ms279us876ns
NOT 2023-10-09 23:19:00.530+00:00 FOO1                                       topics="beacnde" epoch_Ref="[ok(857304), ok(898613), ok(653128), ok(557639), ok(821198), ok(348720), ok(786602), ok(477502), ok(387151), ok(374705), ok(617553), ok(785946), ok(747279), ok(325630), ok(361171), ok(54084), ok(721566), ok(156772), ok(496417), ok(193248), ok(944396), ok(802315), ok(397288), ok(899470), ok(796044), ok(603610), ok(108642), ok(591742), ok(232865), ok(835001), ok(78416), ok(86013)]"
NOT 2023-10-09 23:19:00.530+00:00 FOO2                                       topics="beacnde" shuffling="@[ok(857304), ok(898613), ok(653128), ok(557639), ok(821198), ok(348720), ok(786602), ok(477502), ok(387151), ok(374705), ok(617553), ok(785946), ok(747279), ok(325630), ok(361171), ok(54084), ok(721566), ok(156772), ok(496417), ok(193248), ok(944396), ok(802315), ok(397288), ok(899470), ok(796044), ok(603610), ok(108642), ok(591742), ok(232865), ok(835001), ok(78416), ok(86013)]"
INF 2023-10-09 23:25:24.167+00:00 State replayed                             topics="chaindag" blocks=0 slots=32 current=3469cbf1:7505823@7505824 ancestor=4294d1f6:7505824 target=4294d1f6:7505824@7505856 ancestorStateRoot=c28860b9 targetStateRoot=361e9121 found=true assignDur=16ms744us423ns replayDur=2s307ms329us769ns
NOT 2023-10-09 23:25:24.261+00:00 FOO1                                       topics="beacnde" epoch_Ref="[ok(417997), ok(505167), ok(230843), ok(677873), ok(552845), ok(19140), ok(955437), ok(726065), ok(900343), ok(897247), ok(479767), ok(885764), ok(854837), ok(327962), ok(166285), ok(247016), ok(396330), ok(432594), ok(608125), ok(759738), ok(640662), ok(278973), ok(782773), ok(167926), ok(431088), ok(436930), ok(536147), ok(455994), ok(93116), ok(857374), ok(169467), ok(626666)]"
NOT 2023-10-09 23:25:24.261+00:00 FOO2                                       topics="beacnde" shuffling="@[ok(417997), ok(505167), ok(230843), ok(677873), ok(552845), ok(19140), ok(955437), ok(726065), ok(900343), ok(897247), ok(479767), ok(885764), ok(854837), ok(327962), ok(166285), ok(247016), ok(396330), ok(432594), ok(608125), ok(759738), ok(640662), ok(278973), ok(782773), ok(167926), ok(431088), ok(436930), ok(536147), ok(455994), ok(93116), ok(857374), ok(169467), ok(626666)]"
INF 2023-10-09 23:31:46.823+00:00 State replayed                             topics="chaindag" blocks=0 slots=32 current=7459fc26:7505855@7505856 ancestor=dc86faf6:7505856 target=dc86faf6:7505856@7505888 ancestorStateRoot=34caf169 targetStateRoot=a05e0819 found=true assignDur=15ms2us608ns replayDur=1s401ms482us1ns
NOT 2023-10-09 23:31:46.863+00:00 FOO1                                       topics="beacnde" epoch_Ref="[ok(29730), ok(70834), ok(711354), ok(716988), ok(262388), ok(71946), ok(255969), ok(455941), ok(13024), ok(15340), ok(204468), ok(699731), ok(670745), ok(514347), ok(94454), ok(762851), ok(79419), ok(384210), ok(397038), ok(955712), ok(931504), ok(305339), ok(816778), ok(219926), ok(83043), ok(935614), ok(725736), ok(196307), ok(505412), ok(319546), ok(539725), ok(960284)]"
NOT 2023-10-09 23:31:46.864+00:00 FOO2                                       topics="beacnde" shuffling="@[ok(29730), ok(70834), ok(711354), ok(716988), ok(262388), ok(71946), ok(255969), ok(455941), ok(13024), ok(15340), ok(204468), ok(699731), ok(670745), ok(514347), ok(94454), ok(762851), ok(79419), ok(384210), ok(397038), ok(955712), ok(931504), ok(305339), ok(816778), ok(219926), ok(83043), ok(935614), ok(725736), ok(196307), ok(505412), ok(319546), ok(539725), ok(960284)]"
INF 2023-10-09 23:38:11.763+00:00 State replayed                             topics="chaindag" blocks=0 slots=32 current=f2065b37:7505887@7505888 ancestor=ee96c9e8:7505888 target=ee96c9e8:7505888@7505920 ancestorStateRoot=55460d47 targetStateRoot=dcf94cdd found=true assignDur=18ms213us949ns replayDur=2s414ms887us335ns
NOT 2023-10-09 23:38:11.857+00:00 FOO1                                       topics="beacnde" epoch_Ref="[ok(325340), ok(736137), ok(696310), ok(215389), ok(394091), ok(486078), ok(355452), ok(155509), ok(251932), ok(504212), ok(714257), ok(353357), ok(239638), ok(731966), ok(635633), ok(904945), ok(461809), ok(567137), ok(649421), ok(121011), ok(730682), ok(35246), ok(859263), ok(113062), ok(318166), ok(569276), ok(321726), ok(361296), ok(765764), ok(867557), ok(523083), ok(260239)]"
NOT 2023-10-09 23:38:11.857+00:00 FOO2                                       topics="beacnde" shuffling="@[ok(325340), ok(736137), ok(696310), ok(215389), ok(394091), ok(486078), ok(355452), ok(155509), ok(251932), ok(504212), ok(714257), ok(353357), ok(239638), ok(731966), ok(635633), ok(904945), ok(461809), ok(567137), ok(649421), ok(121011), ok(730682), ok(35246), ok(859263), ok(113062), ok(318166), ok(569276), ok(321726), ok(361296), ok(765764), ok(867557), ok(523083), ok(260239)]"
INF 2023-10-09 23:44:35.555+00:00 State replayed                             topics="chaindag" blocks=0 slots=32 current=db86df14:7505919@7505920 ancestor=ea3695aa:7505920 target=ea3695aa:7505920@7505952 ancestorStateRoot=89e7aa8a targetStateRoot=9e521bd8 found=true assignDur=15ms41us699ns replayDur=2s200ms850us81ns
NOT 2023-10-09 23:44:35.641+00:00 FOO1                                       topics="beacnde" epoch_Ref="[ok(455954), ok(508304), ok(291214), ok(507361), ok(751563), ok(627744), ok(564691), ok(391865), ok(64992), ok(257455), ok(211526), ok(747000), ok(166198), ok(289753), ok(691353), ok(227050), ok(258765), ok(630268), ok(607615), ok(282988), ok(591663), ok(756777), ok(260344), ok(878889), ok(76671), ok(800520), ok(58057), ok(743108), ok(885877), ok(268370), ok(268285), ok(892423)]"
NOT 2023-10-09 23:44:35.641+00:00 FOO2                                       topics="beacnde" shuffling="@[ok(455954), ok(508304), ok(291214), ok(507361), ok(751563), ok(627744), ok(564691), ok(391865), ok(64992), ok(257455), ok(211526), ok(747000), ok(166198), ok(289753), ok(691353), ok(227050), ok(258765), ok(630268), ok(607615), ok(282988), ok(591663), ok(756777), ok(260344), ok(878889), ok(76671), ok(800520), ok(58057), ok(743108), ok(885877), ok(268370), ok(268285), ok(892423)]"
INF 2023-10-09 23:51:00.133+00:00 State replayed                             topics="chaindag" blocks=0 slots=32 current=ffdf42b0:7505951@7505952 ancestor=110b0c0b:7505952 target=110b0c0b:7505952@7505984 ancestorStateRoot=bee8e069 targetStateRoot=a82a063f found=true assignDur=16ms860us604ns replayDur=2s654ms514us78ns
NOT 2023-10-09 23:51:00.229+00:00 FOO1                                       topics="beacnde" epoch_Ref="[ok(505595), ok(435568), ok(509861), ok(94703), ok(726714), ok(715218), ok(635536), ok(916512), ok(121608), ok(643599), ok(334892), ok(637601), ok(167897), ok(286883), ok(544009), ok(595123), ok(539788), ok(645564), ok(711581), ok(525440), ok(479319), ok(272131), ok(471803), ok(116130), ok(263697), ok(690947), ok(434100), ok(300371), ok(698840), ok(432158), ok(839190), ok(205617)]"
NOT 2023-10-09 23:51:00.229+00:00 FOO2                                       topics="beacnde" shuffling="@[ok(505595), ok(435568), ok(509861), ok(94703), ok(726714), ok(715218), ok(635536), ok(916512), ok(121608), ok(643599), ok(334892), ok(637601), ok(167897), ok(286883), ok(544009), ok(595123), ok(539788), ok(645564), ok(711581), ok(525440), ok(479319), ok(272131), ok(471803), ok(116130), ok(263697), ok(690947), ok(434100), ok(300371), ok(698840), ok(432158), ok(839190), ok(205617)]"

i.e. it matches exactly, in mainnet conditions, even without all the guard conditions, which are not at all tight bounds.

[tersec]

On linux-30.ih-eu-mda1.nimbus.holesky

$ zgrep -h "replayDur" ${BEACON_UNSTABLE_LOGS} | grep -F '"blocks":0,"slots":32' | wc -l; zgrep -h "replayDur" ${BEACON_STABLE_LOGS} | grep -F '"blocks":0,"slots":32' | wc -l
8
25

so, same network conditions, same host, over about 3 hours on Holesky even with the cautious guard/failsafe conditions, it removes 2/3 of the epoch-lookahead getEpochRefs and replaces them with getShufflingRefs.

Each had taken about 1.7 seconds:

$ zgrep -h "replayDur" ${BEACON_STABLE_LOGS} | grep -F '"blocks":0,"slots":32' | jq .replayDur.value
1717378708
1716295904
1708602513
1825814881
1751597065
1704157394
1723162272
1697904914
1708474943
1704757403
1710344872
1722362438
1709450567
1704300989
1712434896
1701290537
1706447440
1697090888
1713836254
1716878590
1704176283
1708887015
1699030574
1719330984
1715540117
1702522916

during which the main Nimbus event loop had been blocked, once per epoch.

It's possible to gradually loosen/broaden the conditions under which getShufflingRef is used, by more allowing some penalties/rewards so long as they don't exceed the anti-hysteresis thresholds, but it should already be more effective on mainnet (~100% participation) than Holesky (~87% participation).