Skip to content

v0.1.2

Choose a tag to compare

@github-actions github-actions released this 23 Oct 00:02
· 29 commits to main since this release
v0.1.2
4874494
  • About 30% faster with plugins: the filters, the plugin calls and the rows of a batch run
    on a thread of their own while the next batch is read, as without plugins, with the same
    Parquet bytes as before. On a 2019 laptop, a 1 GB tall CSV with the examples/rates
    config 7.7 -> 5.2 seconds, the 2.5 GB wide CSV 5.8 -> 4.0, a 229 MB JSON 2.1 -> 1.5.
  • With plugins, a batch now ends early where the file's metadata changes, so each record
    has the metadata it was read with, and a parse error still has the records before it
    written.