Skip to content

indexer: bump tx-indexer commit timeouts from 30s to 45s#679

Merged
barnabasbusa merged 2 commits into
ethpandaops:masterfrom
barnabasbusa:bbusa/txindexer-bump-commit-timeout
May 6, 2026
Merged

indexer: bump tx-indexer commit timeouts from 30s to 45s#679
barnabasbusa merged 2 commits into
ethpandaops:masterfrom
barnabasbusa:bbusa/txindexer-bump-commit-timeout

Conversation

@barnabasbusa
Copy link
Copy Markdown
Collaborator

Bumps the three tx-indexer-side context timeouts in indexer/execution/txindexer/process_blocks.go from 30s to 45s:

  • Main commit context (Phase 5)
  • Balance lookup context (Phase 4)
  • Balance-commit-on-error context

Why

Per-block tx-indexer commits can contain up to 8192 event-log inserts and ~70K internal-call inserts on high-deposit-density blocks (e.g. EIP-8254-style stress at 200M gas). On SQLite, even after the recent pragma tuning + chunking + driver-swap improvements (#677, #678), these blocks land in the 30-35s range — right at the existing 30s deadline. The deadline fires, the transaction is rolled back, and the indexer enters a rollback-and-retry loop instead of making forward progress.

Measured timings on a 200M-gas devnet running the spamoor geastx 16×512-deposit scenario (8192 logs/block):

Build per-block commit time outcome under 30s deadline
upstream glebarez + default pragmas 41–45s always exceeds → retry loop
glebarez + tuned pragmas 33–45s always exceeds → retry loop
mattn/go-sqlite3 + tuned pragmas + chunking (#677 + #678) 31–33s exceeds → retry loop

Bumping to 45s gives enough headroom for the 8192-log case to finish under the deadline once #678 lands.

What this isn't

A real fix for the underlying per-block insert volume — the deeper change would restructure the per-tx callbacks in processElBlock to collect rows and do one chunked INSERT per table per block (acknowledging the address-id-resolution dependency tangle pk910 already flagged). That's a much bigger refactor.

This is a band-aid: it stops the rollback-and-retry storm at high-deposit density, while leaving the structural work for a follow-up. Postgres-backed deployments are unaffected — their commit time on the same workload is sub-second.

Test plan

  • go build ./... clean
  • On a 200M-gas devnet with 8192 deposits/block, dora's tx-indexer no longer fails commits with context deadline exceeded and stays caught up

Per-block tx-indexer commits include up to 8192 event-log inserts and
up to ~70K internal-call inserts on high-deposit-density blocks (e.g.
EIP-8254-style stress at 200M gas). Even with WAL+NORMAL pragmas and
the mattn/go-sqlite3 driver, these blocks regularly land in the
30-35 second range under SQLite, which causes the existing 30s context
deadline to fire and rolls the transaction back, putting the indexer
into a retry loop instead of making forward progress.

Bumping the three tx-indexer-side timeouts (main commit, balance
lookup, balance-commit-on-error) to 45s lets these large blocks finish
under the deadline. The change is conservative: 45s is well below the
12s slot rate that an indexer falling behind on every block would
imply, and Postgres-backed deployments aren't affected since their
commit phase is sub-second on the same workload.

This is a band-aid for the underlying volume issue (a deeper fix would
restructure per-block inserts into one chunked INSERT per table), but
it stops the rollback-and-retry storm in the meantime.
@barnabasbusa barnabasbusa enabled auto-merge May 6, 2026 13:38
@barnabasbusa barnabasbusa merged commit 06964ea into ethpandaops:master May 6, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants