ARROW-16187: [Go][Parquet] Properly utilize BufferedStream and buffer size when reading #12876
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently the BufferSize in the
ReaderProperties
isn't utilized properly when enabling BufferedStreams. This fixes that issue so that enablingBufferedStream
reading via the properties will correctly utilize the given buffer size when reading. The default buffersize is currently 16K, so reads that are larger than that will ignore the buffering and just pull directly from the underlying reader when BufferedStream is enabled, pulling the entire page or otherwise from the reader if BufferedStream is not enabled.The buffer size can be set larger so that controlled reads can improve performance on high-latency readers without having to use the memory to read the entire column/page/row group into memory.