Skip to content

hakmarkdev/hyperevm-ingest

Repository files navigation

HyperEVM-ingest v1.0.0

Ingests HyperEVM blocks, transactions, and logs into PostgreSQL.

Built with Kotlin coroutines, ethers-kt, and Exposed.

What it does

  • Polls the HyperEVM JSON-RPC endpoint for new blocks (WebSocket optional, requires a third-party provider)
  • Fetches full block data + receipts + logs concurrently per block
  • Upserts everything into PostgreSQL — idempotent, safe to restart
  • Tracks a backfill cursor separately from the live cursor so both can run in parallel
  • Exposes /health and /metrics (Prometheus) on port 9090

Startup modes

Mode When
FullBackfillThenLive No checkpoint in DB — backfills from block 0 to tip, then goes live
ResumeLive Checkpoint exists — starts live immediately, resumes backfill in background
TargetedBackfill INGEST_END_BLOCK is set — backfills the given range and exits

Schema

Five Flyway migrations create:

  • blocks — one row per block
  • transactions — one row per transaction, FK to blocks
  • logs — one row per log, unique on (tx_hash, log_index)
  • ingest_state — single-row checkpoint for the live cursor
  • backfill_state — single-row checkpoint for the backfill cursor

Requirements

  • JDK 24
  • PostgreSQL 15+
  • Docker (optional, for make db / make up)

Quickstart

# Start PostgreSQL
make db

# Build and run
make run

Or run the full stack in Docker:

make up
make logs

Configuration

All settings have defaults and can be overridden via environment variables:

Variable Default Description
DB_URL jdbc:postgresql://localhost:5432/hyperevm JDBC connection URL
DB_USER ingest Database user
DB_PASSWORD (empty) Database password
RPC_HTTP_URL https://rpc.hyperliquid.xyz/evm HyperEVM HTTP RPC
RPC_WS_URL wss://rpc.hyperliquid.xyz/nanoreth WebSocket RPC (third-party only)
MAX_WS_RECONNECT_ATTEMPTS 0 Set > 0 to enable WebSocket mode
INGEST_START_BLOCK 0 Backfill start block
INGEST_END_BLOCK (unset) Set for targeted backfill
INGEST_BATCH_SIZE 50 Rows flushed per DB batch
INGEST_BACKFILL_CONCURRENCY 10 Concurrent RPC fetches during backfill
METRICS_PORT 9090 Port for /health and /metrics
LOG_LEVEL INFO Logback log level

Make targets

make build      build shadow JAR
make test       run tests
make run        build + run locally (requires make db first)
make up         start app + postgres in Docker
make down       stop Docker stack
make logs       tail app logs
make db         start postgres only (for local dev)
make db-shell   open psql in the postgres container
make health     curl /health

License

Apache License 2.0 — see LICENSE.

About

Ingests HyperEVM blocks, transactions, and logs into PostgreSQL

Topics

Resources

License

Stars

Watchers

Forks

Contributors

Languages