Skip to content

http-to-arrow v0.2.0

Choose a tag to compare

View all tags
@ProxayFox ProxayFox released this 05 Jun 05:57
http-to-arrow-v0.2.0
1344be0

What's Changed

  • Added async Arrow IPC streaming through ArrowIPCStream and lower-level IPCStreamSink batch serialization.
  • Added batch-draining helpers on ArrowRecordContainer for streaming and memory-constrained workflows: drain_batches(), iter_batches(), and flush_partial().
  • Improved streaming backpressure behavior with cooperative producer/consumer scheduling.
  • Refactored append, materialization, and container state internals into helper modules while keeping package-level public exports stable.
  • Added benchmark matrix artifacts and profiling workflows for comparing memory tuning options.
  • Expanded documentation and examples for streaming IPC and memory-conscious ingestion workflows.

Validation

  • just quality
  • uv build src/http_to_arrow --out-dir src/http_to_arrow/dist/
  • uvx twine check src/http_to_arrow/dist/*

Full Changelog: http-to-arrow-v0.1.3...http-to-arrow-v0.2.0

Assets 2
Loading