Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[C++] Support prefetch in ReadRangeCache Lazy mode #36178

Closed
jp0317 opened this issue Jun 20, 2023 · 0 comments · Fixed by #36180
Closed

[C++] Support prefetch in ReadRangeCache Lazy mode #36178

jp0317 opened this issue Jun 20, 2023 · 0 comments · Fixed by #36180
Assignees
@jp0317
Labels
Component: C++ Type: enhancement
Milestone
13.0.0

Comments

@jp0317
Copy link
Contributor

jp0317 commented Jun 20, 2023 •
edited

Describe the enhancement requested

The current ReadRangeCache support a laze mode where each range is read upon an explicit request, and a non-lazy mode where all ranges are read concurrently at once. In some scenarios, it would help to support a prefetch mode (which naturally is bound to the lazy mode) such that explicitly reading one range will prefetching one or more following ranges (the number of prefetched ranges should be configurable via cache options).

Component(s)

C++
@github-actions github-actions bot mentioned this issue Jun 20, 2023
Merged
@lidavidm lidavidm changed the title Support prefetch in ReadRangeCache Lazy mode [C++] Support prefetch in ReadRangeCache Lazy mode Jun 28, 2023
lidavidm pushed a commit that referenced this issue Jun 28, 2023
@jp0317
GH-36178: [C++]support prefetching for ReadRangeCache lazy mode (#36180)
72ec35e 
### Rationale for this change

The current [ReadRangeCache](https://github.com/apache/arrow/blob/main/cpp/src/arrow/io/caching.h#L100) support a laze mode where each range is read upon an [explicit request](https://github.com/apache/arrow/blob/main/cpp/src/arrow/io/caching.cc#L255), and a non-lazy mode where all ranges are read concurrently [at once](https://github.com/apache/arrow/blob/main/cpp/src/arrow/io/caching.cc#L168). In some scenarios, it would help to support a prefetch mode (which naturally is bound to the lazy mode) such that  explicitly reading one range will prefetching one or more following ranges . 

### What changes are included in this PR?

Add a new cache option `prefetch_limit` to define the maximum number of ranges to be prefetched when reading one range in lazy mode.

### Are these changes tested?

Yes. A new unit test is included

### Are there any user-facing changes?

No.
* Closes: #36178

Authored-by: jp0317 <zjpzlz@gmail.com>
Signed-off-by: David Li <li.davidm96@gmail.com>
@lidavidm lidavidm added this to the 13.0.0 milestone Jun 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jp0317 jp0317

Labels
Component: C++ Type: enhancement
Projects
None yet
Milestone
13.0.0
Development

Successfully merging a pull request may close this issue.

GH-36178: [C++]support prefetching for ReadRangeCache lazy mode jp0317/arrow
2 participants
@lidavidm @jp0317