@subsquid/pipes v1.0.0-alpha.18
Pre-release
Pre-release
·
22 commits
to main
since this release
Prerelease. Install with npm i @subsquid/pipes@alpha.
Parquet target
- Pluggable segment-writer engine (#124).
settings.engineonparquetTargetnow accepts anyParquetEngineimplementation; the SDK shipsparquetjsEngineas the built-in, zero-config default. The target owns staging, coverage-window naming, output verification, publication and crash recovery; an engine owns only the rows→Parquet-file step, so alternative encoders (e.g. a DuckDB-backed engine) can plug in without touching durability or recovery. - Encoding options moved to the engine factory.
settings.rowGroupSizeandsettings.compressionare removed — pass them to the engine instead:parquetjsEngine({ rowGroupSize, compression }). Zero-config behaviour is unchanged (same 100k / SNAPPY defaults). - Files are named for the block window they cover (#123),
<from>-<to>.parquet, not for the min/max block of the rows inside them. A range absent from a table's filenames now means "not indexed" rather than "indexed, no data". Locate a block's rows via the row-group footer statistics, not the filename. This changes on-disk layout; existing datasets are read back correctly, but contiguity begins from the upgrade point.
EVM
- Portal trace shapes accept nulls that occur on-chain: a reverted
CREATEwhoseresult.addressis null, andsuicidetraces carrying a nulladdress/balance. Previously these made trace validation throw.
Solana
- A decoder now covers exactly one program/ABI, making multi-program decode wiring explicit.
Decoding (EVM + Solana)
- Unified decode-error hook policy across EVM and Solana, so both surface decode failures through the same mechanism.
Breaking changes
parquetTarget:settings.rowGroupSize/settings.compressionremoved — use the engine factory. The exportedParquetStoreconstructor now takes anengine.- Parquet on-disk file naming changed to coverage windows (see above).
What to test
Parquet backfills end to end — file naming, rollover, and crash-recovery — plus any EVM traces pipeline that touches reverted CREATE / self-destruct traces.