docs: w31 maintenance on main — stale flags, db.read.concurrency, new flag, disk sizes - #22799
Merged
Conversation
The 3.5 series removed several flag families that the docs never absorbed, so release/3.5 (the deployed branch) still tells users to pass flags that no longer parse. * --p2p.allowed-ports (8 occurrences / 4 files). Removed by #21335 "p2p/sentry: share one p2p.Server across all eth protocols" (in 3.5, not 3.4): all eth protocol versions now share a single p2p.Server, so one listening port set via --port is sufficient. The migrating-from-geth examples are updated accordingly. This also fixes a latent casing bug in that page, which wrote --P2P.allowed-ports; Go flags are case-sensitive. * --clique.checkpoint / .snapshots / .signatures / .datadir. Clique is gone from the tree entirely — no clique package remains. * --diagnostics.endpoint.port and the --diagnostics.* lines in the downloader / sentry / txpool / rpc-daemon --help blocks. Removed by #21351 "cmd/diag, go.mod: remove diag CLI". The "Diagnostics" section of default-ports.md documented only this flag and is dropped with it. * --polygon.sync in the rpc-daemon --help block. Removed by #16035 "Remove unused polygon sync flags"; the polygon flags that remain are polygon.pos.ssf, polygon.pos.ssf.block and polygon.wit-protocol. * --rpc.maxgetproofrewindblockcount.limit in the same block. MaxGetProofRewindBlockCount survives as an internal config field (http_cfg.go, default 100_000) but has no CLI registration, so it is not settable and must not be documented as a flag. Verified per family: no Go string literal and no corresponding identifier on release/3.5, release/3.6 or main, and present through release/3.4 where applicable. Also in this change: * --beacon.api.ide.timeout -> --beacon.api.idle.timeout in caplin.md. The table carried the note "flag name is `ide` not `idle` — typo in source". That was accurate for v3.4, where flags.go really did define "beacon.api.ide.timeout", but #20289 "cmd: fix typo in flag beacon.api.ide.timeout" corrected the source for 3.5 and the note was left behind. Every live branch now defines only beacon.api.idle.timeout, so the flag name is corrected and the note removed. Default (25s) is unchanged and matches Value: 25. * Fix a 404: the Ethereum on ARM GitHub link pointed at diglos/ethereum-on-arm; the project is at diglos/ethereumonarm. * Type two bare code fences (architecture.md, database.md) per the docs style rule, so Prism highlights them. The --help paste blocks had only their obsolete lines removed rather than being regenerated wholesale; regenerating them from freshly built downloader/sentry/txpool/rpcdaemon binaries is worth doing as a follow-up. Pre-push gate: npm ci && npm run build green (onBrokenLinks/onBrokenAnchors throw), generate-llms.py --check OK (74 pages), render-disk-sizes.py --check OK, editorial-artifact scan clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Follow-up to review of this PR (Copilot + two adversarial passes). Wrong information corrected: * caplin.md documented the three Beacon API timeouts as `5s` / `31536000s` / `25s`, but all three are cli.Uint64Flag taking a bare integer number of seconds (Value: 5 / 31536000 / 25). `--beacon.api.idle.timeout=25s` does not parse. Values are now plain integers with "in seconds" in the description. * The Ethereum on ARM repository was transferred: diglos/ethereumonarm now 301-redirects to EOA-Blockchain-Labs/ethereumonarm. Point at the canonical URL rather than relying on the redirect. * default-ports.md still described the pre-#21335 two-listener model — separate `30303` (eth/68) and `30304` (eth/69) Sentry rows, plus prose saying both are typically exposed. Since #21335 there is a single listener on `:30303` serving ProtocolVersion [ETH69, ETH70, ETH71] (nodecfg/defaults.go), and eth/68 is not in the defaults at all. Same stale pair fixed in the default port allocation table of multiple-instances.md. * configuring-erigon.mdx gave `--p2p.protocol` defaults as `68, 69`; the actual default is the full ProtocolVersion slice `69, 70, 71`. * sentry.mdx --help block had two wrong defaults: `--maxpeers` is 64, not 32 (nodecfg/defaults.go MaxPeers), and `--p2p.protocol` is [69,70,71], not 68. * migrating-from-geth.mdx had an unmatched closing parenthesis (Copilot). Note on the diagnostics attribution in the previous commit message: those four flags were NOT removed by #21351, which only deleted the cmd/diag CLI. They last exist on release/3.3 (diagnostics/setup.go) and are already absent on release/3.4, so those docs had been stale since the 3.4 series rather than the 3.5 one. The deletions themselves are unaffected. Pre-push gate re-run: npm ci && npm run build green, generate-llms.py --check OK (74 pages), render-disk-sizes.py --check OK. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
#22408 ("node, commitment: fix parallel exec deadlock on many-core machines") rewrote this flag's behaviour and its Usage string on main, and three pages now say the opposite of what the code does. Each parallel-execution worker holds a long-lived read transaction, so a ceiling below the worker count would deadlock. The value is therefore silently raised to the worker count, and lowering the flag does not reduce read concurrency at all — --exec.workers is the knob for that. The docs still advised the reverse: "Low values are fine for low read-concurrency nodes (for example, validators)" in configuring-erigon.mdx and "low values are fine for low-read-concurrency nodes such as validators" in database.md. Both are replaced with the clamp behaviour and a pointer to --exec.workers. The rpc-daemon --help block is refreshed to the current Usage text, which had not been updated since the change. Verified against the source rather than the commit message: the clamp is cmd/utils/flags.go:2041 (RoTxsLimit(c, cfg.ExecWorkerCount)), and the existing "HTTP/WebSocket fail fast with an overload response" wording is still accurate (rpc/http.go:241 httpOverloadedKey / kv.ErrReadTxLimitExceeded), so it is kept. Applies to main only — release/3.5 and release/3.6 predate #22408. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
main still carried the 2025-09-01 manual figures, because the release/3.5 refresh (#22640) was never ported under the dual-commit rule — roughly eleven months of drift on a page users size hardware from. Copies disk-sizes.json from release/3.5 (mainnet and gnosis, all three modes, measured 2026-07-19 and 2026-07-21) and re-renders the static markers in hardware-requirements.mdx with render-disk-sizes.py. These are the v3.5 measurements rather than fresh ones, which keeps a single consistent set of numbers across the live branches. For reference, the minimal-mode CI runs on main and release/3.6 currently measure higher (mainnet ~458-468 GB, gnosis ~256-257 GB against the 379 GB / 204 GB carried here), so these values are conservative for v3.6 and should be replaced once v3.6 has its own full-mode measurements. Note also that the full-mode qa-sync-from-scratch workflow has never uploaded disk-usage-* artifacts, so full and archive can only be measured manually today. render-disk-sizes.py --check and generate-llms.py --check both pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
#22384 ("rpc/jsonrpc: eager in-memory cache for debug_executionWitness") registered utils.WitnessCacheBlocksFlag in node/cli/default_flags.go earlier today, so the CLI reference was one flag behind again. It is operator-facing — it trades memory for debug_executionWitness latency on RPC nodes — so it belongs in the reference per the flag-coverage rule. Documented from the source Value/Usage: default 0 (cache disabled), clamped at 96, embedded RPC only, and dependent on commitment history. The Usage string names the --prune.experimental.include-commitment-history alias; the docs use the canonical --prune.include-commitment-history instead. After this, the Step-4c sweep on main reports 10 undocumented registered flags — the intentionally-undocumented internal/dev baseline shared with the other live branches. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Weekly documentation maintenance on main, focused on removing stale CLI flags, correcting flag semantics/defaults, documenting newly added operator-facing flags, and refreshing measured disk-usage figures used across the docs site.
Changes:
- Remove or correct documentation for flags that no longer exist (e.g.,
--p2p.allowed-ports,--diagnostics.*, clique flags,--polygon.sync,--rpc.maxgetproofrewindblockcount.limit) and fix several wrong defaults/links. - Update docs to reflect
--db.read.concurrency’s post-#22408 semantics (flooring to parallel exec worker count) and document--witness.cache.blocks. - Refresh disk usage figures via
disk-sizes.jsonand re-rendered static markers / LLM exports.
Reviewed changes
Copilot reviewed 15 out of 15 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| llms-full.txt | Updates generated “full” docs export: disk sizes, flag removals/updates, new flag doc. |
| docs/site/static/llms-full.txt | Same as above for the site’s static LLM export. |
| docs/site/src/data/disk-sizes.json | Refreshes measured disk usage values + dates used to render markers. |
| docs/site/docs/get-started/migrating-from-geth.mdx | Removes obsolete --p2p.allowed-ports guidance and updates examples. |
| docs/site/docs/get-started/installation/index.mdx | Fixes Ethereum-on-ARM GitHub link target. |
| docs/site/docs/get-started/hardware-requirements.mdx | Updates disk usage markers/dates rendered from disk-sizes.json. |
| docs/site/docs/fundamentals/multiple-instances.md | Updates default-port allocation to the single-listener model. |
| docs/site/docs/fundamentals/modules/txpool.md | Removes stale diagnostics flags from the help excerpt. |
| docs/site/docs/fundamentals/modules/sentry.mdx | Removes stale diagnostics/allowed-ports and updates defaults (e.g., --maxpeers, --p2p.protocol). |
| docs/site/docs/fundamentals/modules/rpc-daemon.md | Updates help excerpt for removed flags + --db.read.concurrency description. |
| docs/site/docs/fundamentals/modules/downloader.mdx | Removes stale diagnostics flags from the help excerpt. |
| docs/site/docs/fundamentals/default-ports.md | Updates default ports table/prose for the single listener and removes diagnostics section. |
| docs/site/docs/fundamentals/database.md | Corrects --db.read.concurrency semantics text. |
| docs/site/docs/fundamentals/configuring-erigon.mdx | Removes stale flags, corrects defaults, updates --db.read.concurrency, and documents --witness.cache.blocks. |
| docs/site/docs/fundamentals/caplin.md | Fixes Beacon API timeout units and corrects flag name (idle, not ide). |
Comments suppressed due to low confidence (1)
llms-full.txt:4099
- This
--db.read.concurrencydescription claims extra readers wait "rather than error", but some RPC paths fail fast when the MDBX read-tx semaphore is full (e.g., WebSocket uses kv.WithNonBlockingAcquire and returns kv.ErrReadTxLimitExceeded). Update the wording to reflect "wait by default" plus fail-fast overload behavior.
--db.read.concurrency int Ceiling on concurrent open DB read transactions (MDBX read-tx semaphore); extra readers wait for a slot rather than error. Default scales as min(max(10, GOMAXPROCS*64), 9000) — kept well above CPU count because reads are I/O-bound, and capped below Go's ~10K OS-thread limit. A value below the parallel-exec worker count is raised to it (each worker holds a long-lived read tx, so a lower ceiling would deadlock); to actually reduce read concurrency, lower --exec.workers instead
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
* sentry.mdx: my earlier correction over-shot. The erigon binary registers --p2p.protocol as a slice (Value: DefaultConfig.P2P.ProtocolVersion, so 69, 70, 71 in configuring-erigon.mdx is right), but the standalone sentry binary registers it as a single uint defaulting to Value[0] — cmd/sentry/main.go:68, UintVar(..., P2pProtocolVersionFlag.Value[0], ...) — which is ETH69 = 69 (p2p/protocols/eth/protocol.go:39). The sentry --help block therefore reads "(default 69)", not "[69,70,71]". (The --maxpeers 32 -> 64 correction stands: cmd/sentry/main.go:70 takes MaxPeersFlag.Value, which is nodecfg.DefaultConfig.P2P.MaxPeers = 64.) * rpc-daemon.md: the previous commit pasted the source Usage string verbatim, and that string says extra readers "wait for a slot rather than error" — dropping the fail-fast behaviour the docs had previously captured. There are explicit fail-fast paths: rpc/websocket.go:83 tags the connection context with kv.WithNonBlockingAcquire so BeginRo returns kv.ErrReadTxLimitExceeded. Restores "wait for a slot by default, though some RPC paths (HTTP/WebSocket) fail fast with an overload response" while keeping the new clamp sentence, so the block now matches the code rather than the source help text, and agrees with the prose pages. * configuring-erigon.mdx: --witness.cache.blocks notes that the runtime errors from debug_executionWitness and eth_getWitness (rpc/jsonrpc/debug_execution_witness.go:736, rpc/jsonrpc/eth_call.go:672) name the --prune.experimental.include-commitment-history alias, so operators copying from an error message find the same flag in the reference. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
AskAlexSharov
approved these changes
Jul 29, 2026
github-merge-queue
Bot
removed this pull request from the merge queue due to failed status checks
Jul 29, 2026
pull Bot
pushed a commit
to Dustin4444/erigon
that referenced
this pull request
Jul 29, 2026
…rigontech#22848) One-line change to the `--db.read.concurrency` help text. The flag claimed a behaviour the code does not have. ## The problem `cmd/utils/flags.go:407` said: > extra readers wait for a slot **rather than error** That is only true for callers that do not opt into fail-fast acquisition. The RPC layer does: * `rpc/websocket.go:83` tags every WebSocket connection context with `kv.WithNonBlockingAcquire`, unconditionally. * `node/rpcstack.go:87-90` tags HTTP requests whenever the admission handler is active — which is the default: `--rpc.max.concurrency` defaults to `0` (`cmd/utils/flags.go:410-414`), and `cmd/rpcdaemon/cli/config.go:764-775` resolves `0` to `db.read.concurrency`, always ≥ 10. * `db/kv/mdbx/kv_mdbx.go:700-705` then takes the `TryAcquire` branch and returns `kv.ErrReadTxLimitExceeded` immediately instead of blocking. * `rpc/handler.go:631-639` remaps that to JSON-RPC `-32005`, and `rpc/http.go:239-241` surfaces it as HTTP 503 with `Retry-After`. So under read-tx exhaustion an operator gets an overload response, not a stalled request — the opposite of what `--help` promised. ## The fix Only the second clause changes; the rest of the string is untouched: > extra readers wait for a slot **by default, though some RPC paths (HTTP/WebSocket) fail fast with an overload response** ## Why "some RPC paths" rather than naming HTTP outright Deliberate hedge. These HTTP-served paths do **not** fail fast: | Path | Where | Behaviour | |---|---|---| | Engine API (auth) HTTP | `config.go:1001-1005` — built `limit=0, tagAsRPC=false`, with a comment saying so | blocks | | GraphQL | `config.go:952-956` routes `/graphql` before the tagged handler | blocks | | Healthcheck | `config.go:958-961`, same | blocks | | Unix/TCP socket transport | `config.go:742-762`, no tag | blocks | | Public HTTP with `--rpc.max.concurrency=-1` | admission control disabled | blocks | A tighter phrasing such as *"JSON-RPC over HTTP/WebSocket fails fast"* was considered and rejected: the Engine API **is** JSON-RPC over HTTP and deliberately blocks, so that wording would be wrong in a new way. `some RPC paths` is vague but true; the exhaustive list belongs in prose docs, not a help string. ## Relationship to erigontech#22799 This closes a source↔docs divergence that exists on `main` today. `docs/site/docs/fundamentals/modules/rpc-daemon.md:51` already describes the fail-fast behaviour (merged in erigontech#22415) while the Go string still claimed the opposite. Open PR erigontech#22799 refines that same docs sentence; the string in this PR is **byte-identical (548 chars)** to what erigontech#22799 lands, so once both merge, `erigon --help` and the docs help block agree verbatim. The two PRs are independent and can merge in either order — no docs generator reads the Go source (`generate-llms.py` derives the exports from the markdown pages), so nothing needs regenerating here. ## Testing * `go build ./cmd/utils/` and `go vet ./cmd/utils/` — clean. * No test, golden file, fixture, or docs generator asserts on this string or on `--help` output, so nothing else changes. Verified across `cmd/utils/flags_test.go`, `node/cli/helpers_test.go`, and a tree-wide search for the literal. * `cmd/utils/flags.go:407` is the only definition and the only occurrence in Go sources. ## Two pre-existing issues noticed, not addressed here Both are out of scope for a help-string fix; happy to file issues or follow-up PRs if wanted. 1. **Engine API over WebSocket fails fast, contradicting the intent stated for Engine over HTTP.** `createEngineListener` deliberately leaves the engine HTTP stack untagged so CL↔EL calls block (`config.go:1001-1005`), but its WS handler goes through `engineSrv.WebsocketHandler` → the unconditional tag at `rpc/websocket.go:83`. A CL speaking engine-over-WS can receive `-32005` under read-tx exhaustion. Related: the comment at `config.go:1001` refers to "TxPriorityRPC", an identifier that no longer exists anywhere in the tree — the mechanism is `kv.WithNonBlockingAcquire`. 2. **The unchanged tail understates the floor.** It says a value below the parallel-exec worker count "is raised to it", but `RoTxsLimit` floors at `execWorkers + 23` (5 permanent + 2 read-ahead + 16 reserve — `cmd/rpcdaemon/cli/httpcfg/http_cfg.go:38-62`). erigontech#22799 repeats the same simplification in the docs, so correcting it means touching both. Co-authored-by: Bloxster <gianni.morselli@erigon.tech>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The
mainside of this week's documentation maintenance (w31). Carries the same stale-flag cleanup as #22793 (release/3.5) and #22798 (release/3.6) under the dual-commit rule, plus three findings that apply only tomain.1. Flags removed from the code but still documented
Identical to #22793 — see that PR for the per-family provenance:
--p2p.allowed-ports— 8 occurrences / 4 files, removed by p2p/sentry: share one p2p.Server across all eth protocols #21335 "share one p2p.Server across all eth protocols"; a single--portnow suffices. Also fixes the--P2P.allowed-portscasing bug.--clique.checkpoint/.snapshots/.signatures/.datadir— clique is gone from the tree.--diagnostics.*— last present onrelease/3.3, already absent fromrelease/3.4.--polygon.sync— removed by Remove unused polygon sync flags #16035.--rpc.maxgetproofrewindblockcount.limit— an internal config field with no CLI registration.2. Wrong information corrected
Also identical to #22793: Beacon API timeouts are bare integer seconds (
25, not25s— they arecli.Uint64Flag, so25sdoes not parse); the pre-#21335 two-listener port model removed fromdefault-ports.mdandmultiple-instances.md;--p2p.protocoldefault is69, 70, 71;--maxpeersdefault is 64; the Ethereum on ARM link now points at the canonicalEOA-Blockchain-Labs/ethereumonarm; one unmatched parenthesis.3.
--db.read.concurrencysemantics —mainonly#22408 "node, commitment: fix parallel exec deadlock on many-core machines" rewrote this flag's behaviour, and three pages said the opposite of what the code does.
Each parallel-execution worker holds a long-lived read transaction, so a ceiling below the worker count would deadlock. The value is silently raised to the worker count, and lowering the flag does not reduce read concurrency at all —
--exec.workersis the knob for that. The docs advised the reverse: "Low values are fine for low read-concurrency nodes (for example, validators)".Verified against source rather than the commit message: the clamp is
cmd/utils/flags.go:2041(RoTxsLimit(c, cfg.ExecWorkerCount)), and the existing "HTTP/WebSocket fail fast with an overload response" wording is still accurate (rpc/http.go:241,httpOverloadedKey/kv.ErrReadTxLimitExceeded), so it is kept rather than dropped along with the rest.Does not apply to
release/3.5orrelease/3.6, which predate #22408.4.
--witness.cache.blocks— new today#22384 "rpc/jsonrpc: eager in-memory cache for
debug_executionWitness" registeredutils.WitnessCacheBlocksFlagearlier today, leaving the CLI reference one flag behind. It trades memory fordebug_executionWitnesslatency, so it is operator-facing and in scope per the flag-coverage rule.Documented from the source
Value/Usage: default0(disabled), clamped at96, embedded RPC only, requires commitment history. The sourceUsagenames the--prune.experimental.include-commitment-historyalias; the docs use the canonical--prune.include-commitment-history.5. Disk sizes
disk-sizes.jsonis brought in line withrelease/3.5(mainnet and gnosis, all three modes, measured 2026-07-19 / 2026-07-21) and the static markers inhardware-requirements.mdxre-rendered withrender-disk-sizes.py, so the live branches carry one consistent set of numbers.Verification
npm ci && npm run build— green (onBrokenLinksandonBrokenAnchorsboththrow)generate-llms.py --check— OK, 4 files, 74 pagesrender-disk-sizes.py --check— OKallowed-ports/diagnostics./clique./beacon.api.ide/polygon.sync/maxgetproofrewind/eth/68references left indocs/site/docs