cmd/utils: expose executor perf toggles as --exec.* CLI flags#20797
Merged
cmd/utils: expose executor perf toggles as --exec.* CLI flags#20797
Conversation
AskAlexSharov
approved these changes
Apr 24, 2026
mh0lt
force-pushed
the
feat/exec-perf-flags
branch
from
ab07a50 to
50031d9
Compare
Promotes five existing env-var-only executor performance toggles to
first-class CLI flags so they're discoverable via --help and settable
without prefixing every invocation with env vars. Matches the surface
other EL clients (reth, besu) expose — e.g. reth's `--engine.disable-prewarming`
and besu's `--bonsai-parallel-tx-processing-enabled` are direct analogues
of --exec.batched-io.
New flags (all in --exec.* namespace):
--exec.batched-io bool, default true
Use BAL hints to pre-populate the parallel executor's version map.
Inverted alias for IGNORE_BAL: --exec.batched-io=false disables the
prefetch and forces runtime dependency tracking. Matches reth's
--engine.disable-prewarming semantically.
--exec.state-cache bool, default true
Executor domain-shared read cache (alias for USE_STATE_CACHE).
Distinct from the existing --state.cache (which sizes the
rpcdaemon's RPC-path cache — different cache, different code path).
--exec.workers int, default NumCPU/2
Parallel executor worker count (alias for EXEC3_WORKERS). Wired
into cfg.ExecWorkerCount directly plus dbg.Exec3Workers.
--exec.no-merge bool, default false
Disable state-aggregator file merges for Domain/History/II
(alias for NO_MERGE). Diagnostic / perf-comparison use.
--exec.no-prune bool, default false
Disable state-aggregator pruning of historical steps (alias for
NO_PRUNE). Diagnostic / perf-comparison use.
EXEC3_PARALLEL (default true) deliberately stays env-only — it's not a
knob users tune.
Env vars remain the source of truth when no CLI flag is explicitly set:
dbg reads them at package init; SetEthConfig overrides only on
ctx.IsSet(flag). Existing launch scripts (e.g. our bal-devnet-3 setup)
keep working unchanged.
Test: cmd/utils/flags_test.go TestExecPerfFlags_OverrideDbg covers the
no-flag-set path (dbg untouched) and each flag's override semantics.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
mh0lt
force-pushed
the
feat/exec-perf-flags
branch
from
f2b0bc1 to
c7e1211
Compare
mh0lt
added a commit
that referenced
this pull request
Apr 28, 2026
Follow-up to #20797. Adds a single-purpose boolean that forces serial execution by clamping the parallel executor's worker count to 1 — wins over both --exec.workers and EXEC3_WORKERS. Cleaner than `--exec.workers=1` because it states intent ("force serial") at the call site rather than relying on a magic value, and it can't be accidentally undone by a stale EXEC3_WORKERS env var that gets re-read by tooling. Use case: like-for-like baseline comparisons against the parallel executor (devnet A/B runs, profiling cold paths) where the operator explicitly wants serial irrespective of any other knob. Wiring: applied AFTER --exec.workers in SetEthConfig, so: --exec.serial=true → workers=1 --exec.workers=12 --exec.serial=true → workers=1 (serial wins) --exec.workers=12 → workers=12 --exec.serial=false (default) → no override Test: TestExecPerfFlags_OverrideDbg covers serial-true, serial-vs-workers precedence, and serial-false-leaves-untouched. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
mh0lt
added a commit
that referenced
this pull request
Apr 28, 2026
Sahil-4555
pushed a commit
to Sahil-4555/erigon
that referenced
this pull request
Apr 28, 2026
…0853) ## Summary Follow-up to erigontech#20797. Adds a single-purpose boolean — \`--exec.serial\` — that forces serial execution by clamping the parallel executor's worker count to 1. Wins over both \`--exec.workers\` and \`EXEC3_WORKERS\`. ## Why a separate flag rather than just \`--exec.workers=1\` - States intent at the call site (\"force serial\"), so logs / launch scripts / dashboards read clearly. - Can't be accidentally undone by a stale \`EXEC3_WORKERS\` env var picked up by a test harness or wrapper script. - Lets operators do like-for-like baseline comparisons against the parallel executor (A/B perf runs on bal-devnet-3, profiling cold paths) without juggling worker numbers. ## Precedence (resolved in \`SetEthConfig\`) | Flags / env | Result | |---|---| | \`--exec.serial=true\` | \`workers=1\` | | \`--exec.workers=12 --exec.serial=true\` | \`workers=1\` (serial wins) | | \`--exec.workers=12\` | \`workers=12\` | | \`--exec.serial=false\` (default) | no override; env var / default applies | ## Test plan - [x] \`go test -short ./cmd/utils/...\` — \`TestExecPerfFlags_OverrideDbg\` extended with three new subtests: serial=true clamps, serial=true wins over --exec.workers, serial=false leaves Exec3Workers untouched. - [x] \`make lint\` — 0 issues, twice. - [x] \`make erigon\` — \`--help\` shows the new flag with the correct usage and default. ## Files touched - \`cmd/utils/flags.go\` — flag definition + wiring in SetEthConfig (post --exec.workers so it overrides cleanly). - \`cmd/utils/flags_test.go\` — three new subtests. - \`node/cli/default_flags.go\` — register flag in DefaultFlags. ~35 LOC including tests. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Promotes executor performance toggles from env-var-only to first-class CLI flags so they're discoverable via
--helpand settable without prefixing every invocation with env vars. Matches the surface other EL clients expose: reth's--engine.disable-prewarmingand besu's--bonsai-parallel-tx-processing-enabledare direct analogues of--exec.batched-io.New flags (all in
--exec.*namespace)--exec.batched-iotruewarmBodyat blocks_read_ahead.go:111) and BAL-driven version-map pre-population (parallel executor seeds write-dependency map from BAL hints — exec3.go:612). Flipping this off disables both in lockstep for clean "BAL on" vs "BAL off" comparisons. Env aliases:READ_AHEAD,IGNORE_BAL(inverted).--exec.state-cachetrue--state.cache(which sizes the rpcdaemon RPC-path cache — different cache entirely). Env alias:USE_STATE_CACHE.--exec.workersNumCPU/2cfg.ExecWorkerCountanddbg.Exec3Workers. Env alias:EXEC3_WORKERS.--exec.no-mergefalseNO_MERGE.--exec.no-prunefalseNO_PRUNE.EXEC3_PARALLEL(defaulttrue) deliberately stays env-only — it's not a knob users tune.Why
--exec.batched-iocontrols bothBAL (EIP-7928) drives two different optimisations in the executor:
Both are BAL-driven optimisations that users either want together (normal perf mode) or disabled together (measuring "cold state + no BAL scheduling" baseline). One flag for both avoids users having to toggle two env vars in sync.
Semantics / backwards compatibility
Env vars remain the source of truth when no CLI flag is explicitly set:
dbgreads them at package init;SetEthConfigoverrides only onctx.IsSet(flag). Existing launch scripts (e.g. our bal-devnet-3 setup usingERIGON_EXEC3_WORKERS=12 USE_STATE_CACHE=false) keep working unchanged.Why now
We've been running bal-devnet-3 perf comparisons against reth, besu, geth and nethermind where all other clients expose equivalent knobs on the CLI. Having to preface erigon invocations with
ERIGON_EXEC3_WORKERS=N IGNORE_BAL=false READ_AHEAD=false USE_STATE_CACHE=false ./build/bin/erigon …hurts both discoverability and script parity — the launch scripts for other clients in ethpandaops'ethereum-packagepass perf knobs as flags, not env vars.Test plan
go test -short ./cmd/utils/...— newTestExecPerfFlags_OverrideDbgcovers: (a) no-flag-set path leavesdbguntouched so env vars still drive behaviour; (b)--exec.batched-io=falsedisables read-ahead and setsIgnoreBAL=truein one go; (c)--exec.batched-io=trueenables read-ahead and clearsIgnoreBAL; (d) each other flag's override semantics.make lint— 0 issues, verified twice (non-deterministic per CLAUDE.md).make erigon— binary builds,--helpshows all 5 flags with correct defaults and usage.Files touched
cmd/utils/flags.go— 5 flag definitions + wiring inSetEthConfig.cmd/utils/flags_test.go— new test.common/dbg/experiments.go— 6 setters + newReadAheadenv var (READ_AHEAD=truedefault).execution/exec/blocks_read_ahead.go— gatewarmBodyondbg.ReadAhead.node/cli/default_flags.go— register flags inDefaultFlags.~180 LOC including tests.