Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Change list
AsyncParquetFile
toParquetFile
ParquetFile
API.SharedIO
trait, since now we only have a single struct.with_batch_size
andselect_columns
. I don't see a need for theParquetFile
struct itself to maintain any reader state. That information is only used in the read phase, not the constructor phase, and so I think it's fine to pass those options intoread
orstream
.readRowGroup
in favor ofread
.read
now has options including arowGroups
parameter, that takes a list of integers.ReaderOptions
struct for read options, used by both sync and async readers.I couldn't get the row group selection to work in
stream()
herehttps://github.com/kylebarron/parquet-wasm/pull/510/files#diff-e1a77beecd2634c6c0489c20cc3cae036ed6668d62c4d47f00760ab60b0d404eR188-R192
I was hitting lifetime errors with having a
Vec<usize>
there that wouldn't live long enough for the stream.@H-Plus-Time I'd like to get a release out in the next day or two, because otherwise I'll forget about it again and it'll never get released. I already did a lot of other cleanup, so I think it's just this and a little more README updates and then I'm ready to publish 0.6. I don't want to spend a lot more time on this. But I wanted to give you a heads up in case you wanted to make any more edits before the release!