Aggregated Internal Transaction Index

[pk910]

Summary

Rewrites el_transactions_internal from a per-call-frame index (one row per sub-call) into a per-account aggregate (one row per touched account per transaction). Heavy-call transactions — e.g. a contract that fans 100 sub-calls into the same address, or a multisend with many recipients — drop from ~100 rows to a handful, eliminating the SQLite insert-pressure issue (~80k inserts/slot observed on busy devnets). The address-page "Internal Txs" tab now shows aggregated per-direction call counts and value/gas totals instead of one row per call frame.

Changes

Schema & Data Model

• el_transactions_internal rebuilt with new layout, primary key (tx_uid, account_id):

  Column	Type	Description
  tx_uid	INTEGER	block_uid << 16 | tx_index
  account_id	INTEGER	touched account (from, to, or both)
  in_count	SMALLINT	calls where account was the callee
  out_count	SMALLINT	calls where account was the caller
  call_type_mask	SMALLINT	bitmap of incoming call types — bit n = 1 << CallType_n (CALL=0, STATICCALL=1, DELEGATECALL=2, CREATE=3, CREATE2=4, SELFDESTRUCT=5)
  value_in	REAL	sum of ETH received (account was callee)
  value_out	REAL	sum of ETH sent (account was caller)
  gas_used	INTEGER	sum of frame-local gas across calls involving the account

• Single composite index (account_id ASC, tx_uid DESC) replaces the four old per-direction indexes (from_id + to_id, each with and without tx_uid).

• Migration (20260519000000_internal-tx-aggregate.sql, both engines): rebuilds the table from the old per-call layout. Historical data is aggregated in SQL:

  • From-side out_count = COUNT(*) per (tx_uid, from_id), contributes to value_out.
  • To-side in_count = COUNT(*) per (tx_uid, to_id), contributes to value_in.
  • call_type_mask reconstructed via SUM(DISTINCT 1 << call_type) — since each call type maps to a distinct power of two, summing distinct values is equivalent to bitwise OR (portable across SQLite and Postgres, no BIT_OR aggregate required).
  • gas_used zeroed for migrated rows (the old schema didn't store it); repopulates as the indexer touches new blocks.

Indexer (txindexer/process_transactions.go)

• pendingInternalCall (one struct per sub-call) replaced with pendingInternalAggregate (one struct per touched account, accumulated as the call tree is walked).
• processCallTrace now builds a map[*pendingAccount]*pendingInternalAggregate during the depth-first walk:
  • Non-self call A → B: A.outCount++, B.inCount++, B.callTypeMask |= 1<<type, value/gas attributed both sides.
  • Self call A → A: inCount++ and outCount++ on the single row, mask bit set, value/gas attributed once.
  • Top-level frame (idx 0) is skipped — it duplicates el_transactions.
• inCount / outCount clamped to 32767 (postgres SMALLINT signed max) on insert.
• Insert uses upsert (INSERT OR REPLACE for SQLite, ON CONFLICT (tx_uid, account_id) DO UPDATE for Postgres) so re-indexed blocks overwrite cleanly.

Frame-Local Gas Fix

debug_traceBlockByHash (callTracer) returns gasUsed per frame as a cumulative subtree total. Summing it across frames over-counts nested execution: a 3-deep chain attributing the same parent gas to every account it passes through. Fix:

selfGas := uint64(call.GasUsed)
for i := range call.Calls {
    childGas := uint64(call.Calls[i].GasUsed)
    if childGas >= selfGas { selfGas = 0; break } // saturating
    selfGas -= childGas
}

Applied to both:

• the gas_used written into el_transactions_internal (aggregation correctness).
• the GasUsed stored in the blockdb FlatCallFrame (so the tx-detail "Internal Txs" tab shows the gas each frame's own execution consumed, not the inflated subtree total).

The saturating subtract guards against non-Geth tracers that may round gasUsed inconsistently.

DB Layer (db/el_transactions_internal.go)

• InsertElTransactionsInternal rewritten to insert 8-field aggregate rows (was 7-field per-call rows).
• GetElTransactionsInternalByAccount is now a single direct index scan (WHERE account_id = ?) — the old query needed UNION ALL over from_id / to_id branches.
• HasElTransactionsInternalByAccount simplified to a single EXISTS.
• GetElTransactionsInternalCountByTxUid now counts distinct accounts touched by the tx (was: count of call frames). GetElTransactionsInternalByTxUid returns aggregate rows ordered by account_id.

Address Page (/address/{addr}?v=internaltxs)

New columns:

Tx Hash	Block	Age	Types	In	Out	Eth In	Eth Out	Gas Used

• In / Out = in_count / out_count — number of internal calls hitting the address in each direction. Em-dash when zero.
• Types = badges expanded from call_type_mask (incoming-only). Lets staticcall/precompile interactions show "STATICCALL" even when no value transferred.
• Eth In / Eth Out = value_in / value_out formatted as ETH.
• Gas Used = aggregated frame-local gas.

One row per tx (no more rowspan grouping). calculateInternalTxHashRowspans removed.

Transaction Page (/tx/{hash}?v=internaltxs)

• Primary path unchanged: rich per-frame call trace still rendered from blockdb (FlatCallFrame array).
• DB fallback removed: when blockdb is missing call-trace data (ElBlockDataCallTraces not set), the page now shows the "not available" archive notice instead of the old degraded per-call DB view — the DB index no longer holds per-call detail.
• Tab badge Internal Txs (N) shows account-count from GetElTransactionsInternalCountByTxUid on initial render; once the tab loads, blockdb data overwrites it with the actual call count.

Models (types/models/address.go)

AddressPageDataInternalTransaction reshaped to mirror the aggregate row (InCount, OutCount, CallTypes, ValueIn, ValueOut, GasUsed). Added AddressPageDataInternalTransactionCallType helper struct for pre-expanded badge rendering. TxHashRowspan and per-call fields (CallIndex, CallType, FromAddr, ToAddr, IsOutgoing, …) removed.

Impact

Scenario	Before	After
100-call loop to one contract	100 rows + 4 index updates per row	1–2 rows + 1 index update each
Multisend to 40 recipients	40 rows	41 rows (sender + 40 recipients), each with in_count=1 or out_count=40
Precompile (0x...0E) called once per tx	1 row per tx	1 row per tx, in_count=1
el_transactions_internal storage	~7 fields × N calls	~8 fields × M accounts (M ≪ N for fan-out txs)

[pk910]

handling 8k deposits with sqlite now: