FileStream: Buffer more than one FileOpenFuture #5209
Description
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
(This section helps Arrow developers understand the context and why for this feature, in addition to the what)
Follow up to #5161 (comment)
Describe the solution you'd like
A clear and concise description of what you want to happen.
Instead of prefetching just the next stream, buffer up to N (where N is configurable?) open files. So maybe break this into two pipelined streams FileOpenStream and FileDecodeStream then we can used StreamExt::buffered to parallelize the opening.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Do nothing
Additional context
Add any other context or screenshots about the feature request here.