SPICE

SPICE is a temporal fee-timing research pipeline for EVM chains. It acquires canonical block data, builds feature tables, constructs temporal decision problems, trains neural models, tunes studies, stores artifacts, and evaluates fee-timing decisions under real delay budgets.

The project is organized around explicit seams: configs resolve into typed contracts, contracts compile concrete implementations, workflows orchestrate storage effects, and docs are split into generic architecture and concrete implementation guides.

Start Here

Read in this order if you are new to the codebase and the domain:

README.md
  -> ARCHITECTURE.md
  -> src/spice/ARCHITECTURE.md
  -> src/spice/conf/IMPLEMENTATIONS.md
  -> package ARCHITECTURE.md files
  -> package IMPLEMENTATIONS.md files

Use this rule:

File	Purpose
ARCHITECTURE.md	Generic structure, boundaries, data flow, dependency direction.
IMPLEMENTATIONS.md	Current concrete engines, families, algorithms, YAML Surfaces/Config Groups, math, and failure modes.

Architecture docs explain the shape of the system. Implementation docs explain what the current code actually runs.

Learning Paths

Beginner ML And Modeling Path

This path explains how raw block rows become model predictions:

features
  -> temporal problems
  -> dataset builders
  -> model families
  -> prediction families
  -> objectives
  -> evaluation

Read:

1. Feature implementations
2. Temporal compiler implementations
3. Execution policy implementation
4. Input normalization implementations
5. Dataset builder implementations
6. Model family implementations
7. Prediction family implementations
8. Objective implementations
9. Evaluator implementations

The key mental model:

canonical block rows
  -> feature matrix
  -> temporal problem store
  -> sequence batches
  -> neural model
  -> decoded offsets
  -> evaluator metrics

Data Acquisition And Storage Path

This path explains how chain data is downloaded, validated, committed, indexed, and transferred:

RPC acquisition
  -> Corpus Assembly
  -> corpus validation
  -> storage roots
  -> catalog
  -> transfer

Read:

1. RPC acquisition implementations
2. Corpus implementations
3. Storage implementations
4. Catalog implementations

The key mental model:

timestamp window
  -> block range
  -> RPC batches
  -> canonical parquet rows
  -> root state DB
  -> derived catalog row

Workflow And Operations Path

This path explains how users run work:

YAML Surfaces and Config Groups
  -> config resolution
  -> CLI command
  -> local workflow or remote submit
  -> storage effect

Read:

1. Config implementations
2. Config resolution implementations
3. Workflow implementations
4. Execution implementations
5. CLI command implementations

The key mental model:

surface name + overrides
  -> resolved workflow config
  -> acquire/train/tune/evaluate
  -> corpus/study/artifact state

Core Terms

Term	Meaning
Corpus	Stored canonical block data for one chain/dataset/evaluation date.
History rows	Rows before the evaluation window, used for training and warmup.
Evaluation rows	Rows in the evaluation day, used for diagnostic replay.
Feature	Numeric observable derived from block rows.
Sample	One temporal decision example.
Anchor row	Row representing the decision time for a sample.
Context rows	Past rows the model may observe.
Candidate window	Future row interval the model may choose from.
Candidate offset	Integer action relative to the candidate-window start.
Execution policy	Rule that maps decoded offsets to actual outcome rows.
Decoded Result ABI	Typed prediction output contract consumed by evaluators.
DecodedOffsets	Current candidate-offset decoded result ABI.
Artifact	Persisted trained model plus exact manifest and runtime state.
Study	Persisted tuning state and Optuna trial database.
Evaluator	Runtime scorer that turns decoded predictions into metrics.
Surface	High-level YAML recipe combining chain, dataset, model, features, problem, objective, and evaluation.

Stack

Tool	Use
Typer	CLI
Pydantic + PyYAML	Config models and YAML loading
SQLAlchemy Core	SPICE-owned SQLite state
Polars	Corpus IO and validation
PyTorch	Modeling
Optuna	Tuning
web3.py	RPC access

Setup

brew install uv
uv sync --extra dev
source .venv/bin/activate

uv manages the repo-local .venv/. Without activation, prefix commands with uv run.

Push to both remotes:

git push origin main
git push university main

CLI Quickstart

Local acquisition:

spice acquire --surface current_row_fee_dynamics

Remote train/tune/evaluate submission:

spice train --surface current_row_fee_dynamics --dataset-id cor_9a73b1e88edb488afb1e
spice tune --surface current_row_fee_dynamics --dataset-id cor_9a73b1e88edb488afb1e --trial-count 20
spice evaluate --artifact-id art_... --dataset-id cor_9a73b1e88edb488afb1e --evaluation poisson_replay

The CLI owns the default remote target, disi_l40. Train, tune, and evaluate submit remotely by default; Python workflow runners remain available for the remote runner and tests.

Config and storage inspection:

spice config list provider
spice config show dataset icdcs_2026
spice config edit problem current_row_nominal

spice show dataset
spice show artifact --artifact-id art_...
spice delete artifact --artifact-id art_...
spice refresh catalog

Transfer:

spice transfer push dataset --dataset-id cor_9a73b1e88edb488afb1e
spice transfer pull artifact --artifact-id art_...

Current Concrete IDs

Seam	Current ids
Features	core_fee_dynamics
Temporal compilers	observed_time_window
Execution policies	strict_deadline_miss
Input normalization	row_standard, window_weighted_standard
Dataset builders	fixed_sequence_temporal
Model families	lstm, transformer, transformer_lstm
Prediction families	min_block_fee_multitask
Evaluators	poisson_replay, full_temporal_replay
Remote target	disi_l40

Output Layout

outputs/
  .spice/catalog.sqlite
  corpora/<chain>/<corpus_id>/
    history/
    evaluation/
    .spice/state.sqlite
  studies/<chain>/<study_id>/
    .spice/state.sqlite
  artifacts/<chain>/<artifact_id>/
    model.pt
    .spice/state.sqlite

Root state DBs and manifests are source of truth. The catalog is derived and can be refreshed.

Verification

uv run ruff check .
uv run pyright
uv run vulture
uv run pytest -q

vulture runs at min_confidence = 90 from pyproject.toml. Treat its output as review input, not proof: manually verify every reported item before deleting code because dynamic Python usage can hide real references.

YAML specs can be validated through raw config group loading:

uv run python - <<'PY'
from spice.config.groups import load_named_group_payload, named_group_keys, list_group_names

count = 0
errors = []
for group in named_group_keys():
    for name in list_group_names(group):
        try:
            load_named_group_payload(name, group)
        except Exception as exc:
            errors.append((group, name, type(exc).__name__, str(exc)))
        count += 1
print(f"validated={count} errors={len(errors)}")
for error in errors:
    print(error)
PY