Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[C++] Specify IO executor in Acero QueryContext #35125

Closed
zifengyu opened this issue Apr 14, 2023 · 2 comments · Fixed by #35464
Closed

[C++] Specify IO executor in Acero QueryContext #35125

zifengyu opened this issue Apr 14, 2023 · 2 comments · Fixed by #35464
Labels
Component: C++ good-first-issue Type: enhancement
Milestone
13.0.0

Comments

@zifengyu
Copy link

Describe the enhancement requested

We want to use a dedicated IO executor other than arrow default in scheduling IO task. The QueryContext constructor only set the IOContext's memory pool. Could it provide a way to also specify the executor?

Component(s)

C++
@westonpace
Copy link
Member

I think it makes the most sense for the I/O executor to be determined by the filesystem. However, I don't think this can be done today. Can you explain why you want to control the I/O executor?

Otherwise, this seems like a reasonable ask.

@zifengyu
Copy link
Author

zifengyu commented Apr 23, 2023 •
edited

Thanks for the response. We are adapting Acero with our current system in which I/O executor wraps some additional information for tracking and configuration. We noticed QueryContext introduced; and wonder whether move to use it scheduling the I/O tasks. This should not be a blocking issue if could not be done currently.

@github-actions github-actions bot mentioned this issue May 6, 2023
Merged
westonpace pushed a commit that referenced this issue May 25, 2023
@mapleFU
GH-35125: [C++][Acero] Add a self-defined io-executor in QueryOptions (…
6bd31f3 
…#35464)

### Rationale for this change

Support a custom io-executor in Acero QueryOptions.

### What changes are included in this PR?

Allow user to define a customed io-executor.

### Are these changes tested?

Currently not

### Are there any user-facing changes?

User can specify an io-executor

* Closes: #35125

Authored-by: mwish <maplewish117@gmail.com>
Signed-off-by: Weston Pace <weston.pace@gmail.com>
@westonpace westonpace added this to the 13.0.0 milestone May 25, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Component: C++ good-first-issue Type: enhancement
Projects
None yet
Milestone
13.0.0
Development

Successfully merging a pull request may close this issue.

GH-35125: [C++][Acero] Add a self-defined io-executor in QueryOptions mapleFU/arrow
2 participants
@westonpace @zifengyu