fix(simulator): bump pre-init buffer 1.05x -> 2.0x to fix CI#192
Closed
meyer9 wants to merge 1 commit into
Closed
Conversation
The basic-benchmarks job in public-benchmarks.yaml has been failing on every push to main since cf47296 (PR #184, May 15) with: execution reverted: Not enough accounts to load/update Reproduces consistently for base-mainnet-simulation @ 25M gas (the first cell of the matrix, run on both reth and geth). Fails roughly 7-8 benchmark blocks into a 20-block run. Root mechanism: PR #184 changed targetCalls from ceil(numCallsPerBlock * scaleFactor) // pre-fix to numCallsPerBlock // post-fix to address an overshoot for scaleFactor > 1 (e.g. 200M gas case). For scaleFactor < 1 (small gas limits like 25M), this raised on-chain consumption within the existing 5% pre-init buffer. CI's base-mainnet-simulation @ 25M sits right at the boundary where rounding accumulation + per-field interaction overruns the 5% margin and trips the contract's current_address_index + load + update <= num_address_initialized require well before block 20. Bumping the multiplier to 2.0x removes the boundary entirely. Pre-init runs during setup (one-time cost, dominated by mineAndConfirm batching), so this adds maybe a few seconds of setup wall time per test cell — negligible vs the ~5 minutes per cell + 1h total job time. Out of scope for this PR but noted for follow-up: - OpcodeStats.Sub/Add iterate 'other' rather than the union of s and other, which causes per-tx blockCounts.Precompiles to always be empty in sendTxs. Result: per-tx actual gas is much lower than gas_per_call estimate, so the benchmark under-utilizes the gas budget. Not the cause of THIS CI failure, but should be fixed. - The contract require is a benchmark-fidelity check, not a protocol constraint; making the worker recalibrate numCallsPerBlock from observed on-chain consumption would be a more principled fix than relying on a static safety multiplier.
Collaborator
🟡 Heimdall Review Status
|
Collaborator
Author
|
Superseded by #194 which combines this buffer bump with the OpcodeStats Add/Sub fix and observed-gas recalibration into one coordinated change. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The
basic-benchmarksjob in.github/workflows/public-benchmarks.yamlhas been failing on every push to main since 2026-05-15 (PR #184). Symptom:Reproduces deterministically on
base-mainnet-simulation @ 25M gas(the first matrix cell, run against both reth and geth), about 7-8 benchmark blocks into a 20-block run. CI history:b5290a8(#191)4f98307(#186)cf47296(#184)cb5b9d6The contract revert is at Simulator.sol:82 — a benchmark-fidelity check that
current_address_index + load + update <= num_address_initialized.Mechanism
PR #184 (
cf47296) changedtargetCallsin the simulator worker:This was correct for
scaleFactor > 1(the 200M-gas case the PR was targeting). But forscaleFactor < 1(the 25M case frombase-mainnet-simulation, wherescaleFactor = 25M / 35M = 0.714), the change INCREASES per-block consumption fromceil(46 × 0.714) = 33calls/block to46calls/block.Pre-init is sized as:
This gives a uniform 5% buffer post-PR-#184. The buffer is mathematically sufficient for an ideal run, but in practice the boundary is too tight: rounding accumulation across multiple
Statsfields (per-txblockCounts.Round(),big.Int.Int64()truncation, etc.) plus the deterministic per-tx layout pushes consumption past the safety margin well before block 20.Fix
Bump the multiplier from
1.05to2.0. Pre-init now allocates ~2× the predicted consumption. Pre-init runs during setup as a one-time cost (dominated bymineAndConfirmRPC batching), so the wall-clock impact is a few seconds per cell — negligible against the ~5 min/cell, ~1 h/job total runtime.Verification
go build ./runner/...cleango vet ./runner/payload/simulator/...cleango test ./runner/payload/simulator/... -timeout 60sclean (existingmineAndConfirmBatchSizetests pass; no test coverstestForBlockspre-init sizing directly)InitializeAddressChunk/InitializeStorageChunkexactly the same way.Out of scope (follow-up)
While digging I noticed two real bugs worth separate PRs:
OpcodeStats.Sub/Additerateotherrather thans ∪ other(types.go:48,38) → per-txblockCounts.Precompilesis always empty insendTxs→ per-tx actual gas is much lower than thegasvaluecalcNumCallsuses → block utilization is far below the configured gas limit even when the require passes.numCallsPerBlockbased on observed on-chain consumption would be more principled than the static safety multiplier and would also fix thebase-mainnet-simulation @ 250Munder-fill I documented in the gas-limit investigation. Likely a one-method addition tosimulatorPayloadWorker.Both are outside the scope of "make CI green again", which this PR does.
Draft because end-to-end validation requires one CI run.