Skip to content

Conversation

@Savid
Copy link
Member

@Savid Savid commented Dec 1, 2025

Add a new lightweight transaction processor that fetches basic transaction metadata via eth_getTransactionReceipt instead of expensive debug_traceTransaction calls. This processor stores transaction data (from/to, value, gas, nonce, etc.) to ClickHouse for analytics.

Key changes:

  • Add pkg/processor/transaction/simple/ package with full processor implementation
  • Migrate from ethkit to official go-ethereum ethclient for RPC calls
  • This fixes blob (type 3) and EIP-7702 (type 4) transactions being skipped (ethkit filtered out transaction types > 2)
  • Update execution node to use go-ethereum's ethclient.Client and rpc.Client
  • Add BlockReceipts batch fetching for improved performance
  • Register simple processor in manager with queue support

The simple processor provides:

  • Forwards and backwards block processing modes
  • Transaction count verification against ClickHouse
  • Batch receipt fetching with per-tx fallback
  • Full support for all transaction types (legacy, access list, EIP-1559, blob, EIP-7702)

Migrations

CREATE TABLE default.execution_transaction_local ON CLUSTER '{cluster}' (
    `updated_date_time` DateTime COMMENT 'Timestamp when the record was last updated' CODEC(DoubleDelta, ZSTD(1)),
    `block_number` UInt64 COMMENT 'The block number' CODEC(DoubleDelta, ZSTD(1)),
    `transaction_index` UInt64 COMMENT 'The transaction index' CODEC(DoubleDelta, ZSTD(1)),
    `transaction_hash` FixedString(66) COMMENT 'The transaction hash' CODEC(ZSTD(1)),
    `nonce` UInt64 COMMENT 'The transaction nonce' CODEC(ZSTD(1)),
    `from_address` String COMMENT 'The transaction from address' CODEC(ZSTD(1)),
    `to_address` Nullable(String) COMMENT 'The transaction to address' CODEC(ZSTD(1)),
    `value` UInt256 COMMENT 'The transaction value in wei' CODEC(ZSTD(1)),
    `input` Nullable(String) COMMENT 'The transaction input in hex' CODEC(ZSTD(1)),
    `gas_limit` UInt64 COMMENT 'The transaction gas limit' CODEC(ZSTD(1)),
    `gas_used` UInt64 COMMENT 'The transaction gas used' CODEC(ZSTD(1)),
    `gas_price` UInt64 COMMENT 'The transaction gas price' CODEC(ZSTD(1)),
    `transaction_type` UInt32 COMMENT 'The transaction type' CODEC(ZSTD(1)),
    `max_priority_fee_per_gas` UInt64 COMMENT 'The transaction max priority fee per gas' CODEC(ZSTD(1)),
    `max_fee_per_gas` UInt64 COMMENT 'The transaction max fee per gas' CODEC(ZSTD(1)),
    `success` Bool COMMENT 'The transaction success' CODEC(ZSTD(1)),
    `n_input_bytes` UInt32 COMMENT 'The transaction input bytes' CODEC(ZSTD(1)),
    `n_input_zero_bytes` UInt32 COMMENT 'The transaction input zero bytes' CODEC(ZSTD(1)),
    `n_input_nonzero_bytes` UInt32 COMMENT 'The transaction input nonzero bytes' CODEC(ZSTD(1)),
    `meta_network_id` Int32 COMMENT 'Ethereum network ID' CODEC(DoubleDelta, ZSTD(1)),
    `meta_network_name` LowCardinality(String) COMMENT 'Ethereum network name'
) ENGINE = ReplicatedReplacingMergeTree(
    '/clickhouse/{installation}/{cluster}/default/tables/{table}/{shard}',
    '{replica}',
    updated_date_time
) PARTITION BY intDiv(block_number, 201600) -- roughly 1 month of blocks
ORDER BY
    (
        meta_network_name,
        block_number,
        transaction_index,
        transaction_hash
    ) COMMENT 'Contains canonical execution transaction data.';

CREATE TABLE default.execution_transaction ON CLUSTER '{cluster}' AS default.execution_transaction_local ENGINE = Distributed(
    '{cluster}',
    default,
    execution_transaction_local,
    cityHash64(
        block_number,
        meta_network_name,
        transaction_hash
    )
);

TODO

  • Add block_root and parent_root to table
  • handle reorgs by matching roots, maybe causing an update to admin table to find which block to go all the way back to?
  • make sure verify uses a diff RPC to check total number of txns for a block (block number + block root) matches the count, verify job should include the block number and root

Add a new lightweight transaction processor that fetches basic transaction
metadata via eth_getTransactionReceipt instead of expensive debug_traceTransaction
calls. This processor stores transaction data (from/to, value, gas, nonce, etc.)
to ClickHouse for analytics.

Key changes:
- Add pkg/processor/transaction/simple/ package with full processor implementation
- Migrate from ethkit to official go-ethereum ethclient for RPC calls
- This fixes blob (type 3) and EIP-7702 (type 4) transactions being skipped
  (ethkit filtered out transaction types > 2)
- Update execution node to use go-ethereum's ethclient.Client and rpc.Client
- Add BlockReceipts batch fetching for improved performance
- Register simple processor in manager with queue support

The simple processor provides:
- Forwards and backwards block processing modes
- Transaction count verification against ClickHouse
- Batch receipt fetching with per-tx fallback
- Full support for all transaction types (legacy, access list, EIP-1559, blob, EIP-7702)
Add block_hash and parent_hash fields to track block lineage in the
transaction table, enabling reorg detection and chain verification.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@Savid Savid merged commit a26d535 into master Dec 2, 2025
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants