Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ARROW-15262: [C++] Create a ToTable sink node #12275

Closed
wants to merge 8 commits into from

Conversation

vibhatha
Copy link
Collaborator

This PR includes an addition of a Sink node customized to obtain the output from an streaming execution.

@github-actions
Copy link

@github-actions
Copy link

⚠️ Ticket has no components in JIRA, make sure you assign one.

@github-actions
Copy link

⚠️ Ticket has not been started in JIRA, please click 'Start Progress'.

cpp/src/arrow/compute/exec/sink_node.cc Outdated Show resolved Hide resolved
cpp/src/arrow/compute/exec/sink_node.cc Outdated Show resolved Hide resolved
cpp/src/arrow/compute/exec/sink_node.cc Outdated Show resolved Hide resolved
cpp/src/arrow/compute/exec/sink_node.cc Outdated Show resolved Hide resolved
cpp/src/arrow/compute/exec/sink_node.cc Show resolved Hide resolved
cpp/src/arrow/compute/exec/options.h Outdated Show resolved Hide resolved
cpp/src/arrow/compute/exec/options.h Outdated Show resolved Hide resolved
cpp/src/arrow/compute/exec/plan_test.cc Outdated Show resolved Hide resolved
@vibhatha
Copy link
Collaborator Author

vibhatha commented Feb 1, 2022

@westonpace I updated the PR with review comments. But there is an issue when we remove the future from the Consuming sync node, the process hangs without finishing. Looking into it.

@lidavidm
Copy link
Member

lidavidm commented Feb 1, 2022

I don't think we want to remove the Future from ConsumingSinkNode, but we don't need it in TableSinkNodeOptions.

Copy link
Member

@lidavidm lidavidm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, just a couple nits

cpp/src/arrow/compute/exec/sink_node.cc Outdated Show resolved Hide resolved
cpp/src/arrow/compute/exec/sink_node.cc Outdated Show resolved Hide resolved
@lidavidm
Copy link
Member

lidavidm commented Feb 4, 2022

LGTM. I think the Dev failure is just because this branch needs to be rebased/it will go away once merged.

@lidavidm lidavidm closed this in 47d5575 Feb 4, 2022
@ursabot
Copy link

ursabot commented Feb 4, 2022

Benchmark runs are scheduled for baseline = 1ff2e19 and contender = 47d5575. 47d5575 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Finished ⬇️1.88% ⬆️0.17%] test-mac-arm
[Finished ⬇️0.36% ⬆️0.0%] ursa-i9-9960x
[Failed] ursa-thinkcentre-m75q
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

vibhatha added a commit to vibhatha/arrow that referenced this pull request Feb 4, 2022
This PR includes an addition of a Sink node customized to obtain the output from an streaming execution.

Closes apache#12275 from vibhatha/arrow-15262

Authored-by: Vibhatha Abeykoon <vibhatha@gmail.com>
Signed-off-by: David Li <li.davidm96@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants