ARROW-15016: [R] show_exec_plan for an arrow_dplyr_query

[dragosmg]

This PR adds show_exec_plan() will allow users to inspect the ExecPlan, in a similar way to dplyr's show_query().

mtcars %>% 
  arrow_table() %>% 
  filter(mpg > 20) %>% 
  mutate(x = gear/carb) %>%
  group_by(cyl) %>% 
  show_exec_plan()
#> ExecPlan with 3 nodes:
#> 2:ProjectNode{projection=[mpg, cyl, disp, hp, drat, wt, qsec, vs, am, gear, carb, "x": divide(cast(gear, {to_type=double, allow_int_overflow=false, allow_time_truncate=false, allow_time_overflow=false, allow_decimal_truncate=false, allow_float_truncate=false, allow_invalid_utf8=false}), cast(carb, {to_type=double, allow_int_overflow=false, allow_time_truncate=false, allow_time_overflow=false, allow_decimal_truncate=false, allow_float_truncate=false, allow_invalid_utf8=false}))]}
#>   1:FilterNode{filter=(mpg > 20)}
#>     0:TableSourceNode{}

Some design considerations are discussed in the design doc.

Summary of the approach:

• I opted for the show_exec_plan() name as I believe it aligns well with the purpose of this PR: to expose the ExecPlan (in its raw state).
• We could later extend, beautify the output (e.g. use {cli} and / or show_query() / explain() methods).

[github-actions]

https://issues.apache.org/jira/browse/ARROW-15016

[jonkeane]

I know this is a still a draft, but I was curious + saw a few things that looked comment-able that might help drive this forward

I appreciate how nice and targeted these changes are so far!

[wjones127]

I like this function! I made a few optional suggestions.

One potential future enhancement: Neal is adding this function query_can_stream() in #13563, which might be helpful info to know when looking at the query exec plan. Showing it next to the plan will help people understand what kinds of plans can and can't stream.

[dragosmg]

@wjones127 Thanks for the review. I definitely think we could expand this sort of functionality. Maybe we could keep show_exec_plan() as bare-bone and add an explain.arrow_dplyr_query() method with more bells and whistles (both in terms of information and formatting). What do you think?

[wjones127]

Maybe we could keep show_exec_plan() as bare-bone and add an explain.arrow_dplyr_query() method with more bells and whistles (both in terms of information and formatting). What do you think?

Yeah that makes sense to me 👍

[dragosmg]

In order to avoid repetition in the unit tests, I went for a combined approach:

• used expect_output() to check for specific pieces of information that we want / expect to be present for show_exec_plan()
• used expect_snapshot() to test show_query() and explain() which are just wrappers around show_exec_plan().

[paleolimbot]

Once the CI turns green, I'm good with this! Technically the ExecPlan_prepare() changes in compute-exec.cpp are no longer needed for this PR but I am also not worried about them (they're needed for the user-defined functions PR as well).

[nealrichardson]

Unless I'm missing something, this looks like a major rewrite of the PR that causes show_query() to actually run the query, which is not desirable.

[nealrichardson]

IIUC this starts evaluating the ExecPlan, which we don't want to do.

Suggested change

arrow::StopIfNotOk(prepared_plan.first->StartProducing());

[dragosmg]

@nealrichardson I think StartProducing() is central to this (let's call it the BuildAndShow) approach. I couldn't get it to work without it. Extracting the duplicated code in ExecPlan_prepare doesn't work without starting the plan.

My f40e5d2 commit, submitted 3 days ago - was AFAICT exactly what you suggest and it didn't work.

So was Dewey's fbb4c1e (submitted 2 days ago).

I will revert to this state minus the line you suggested we delete and see where that takes us.

[nealrichardson]

My recommendation was the latter, but I don't object to also having a standalone show_exec_plan()

[nealrichardson]

IIUC this is a really bad idea: you're evaluating the whole query just to print it.

[paleolimbot]

Do we know how much data gets pulled from an exec plan that is created and immediately deleted?

[nealrichardson]

As I understand it, Acero is a push model, not pull. So when you call start producing, it fires up, doesn't matter that you haven't pulled the first batch, it's pushing batches to the reader. I also wouldn't assume that just because the RBR object goes out of scope that that sends any signal to stop producing. Assumptions aside, we should be able to observe this if we test with real data.

But I would think that printing the query shouldn't trigger any evaluation on the data--anything >0 data being read is probably too much.

[westonpace]

Creating an exec plan: None
Calling StartProducing: All of it

You can call StopProducing after you call StartProducing but that isn't great right now (we still fully consume whatever files we happen to be reading at the moment) and will hopefully get better.

[nealrichardson]

@dragosmg check out #13397 (comment). I think this explains why "the filter node is gone" on the head queries--it gets evaluated 🙀 inside of Build(). For now we should probably not try to show the query if (is_collapsed(.data) && has_head_tail(.data$.data)) because that will cause the inner query to evaluate. We can back that out after ARROW-16628, I believe (should add a comment pointing to that issue too).

[dragosmg]

This ☝🏻 commit (5751543 ) includes the call to start the plan. Most tests pass (except the large memory test).

Commit e29a835 removes the call to _StartProducing => tests fail with

/arrow/cpp/src/arrow/compute/exec/exec_plan.cc:209: Check failed: started_ stopped an ExecPlan which never started

This makes me think, that, if we want to avoid starting the plan, we need to go back to a state where the "preparation" is done with duplicated code in both ExecPlan_run and ExecPlan_ToString/ExecPlan_BuildAndShow.

[dragosmg]

What have I done so far today:

• Reverted to an earlier version (without the ExecPlan_prepare C++ helper function).
• Removed unit tests for show_query() and explain().
• Improved the documentation for show_exec_plan() to reference dplyr::show_query() and dplyr::explain().
• Added comments indicating duplicate code that should be in sync, in:
  • ExecPlan_run and ExecPlan_BuildAndShow.
  • $Run() and $BuildAndShow()
• Created ARROW-17184 to investigate why some nodes are missing from the print out.

[dragosmg]

@dragosmg check out #13397 (comment). I think this explains why "the filter node is gone" on the head queries--it gets evaluated 🙀 inside of Build(). For now we should probably not try to show the query if (is_collapsed(.data) && has_head_tail(.data$.data)) because that will cause the inner query to evaluate. We can back that out after ARROW-16628, I believe (should add a comment pointing to that issue too).

Not sure adding a comment referencing ARROW-16628 is needed since I am no longer touching the R6 $Build() method. I added a comment on ARROW-17184.

[nealrichardson]

@dragosmg check out #13397 (comment). I think this explains why "the filter node is gone" on the head queries--it gets evaluated 🙀 inside of Build(). For now we should probably not try to show the query if (is_collapsed(.data) && has_head_tail(.data$.data)) because that will cause the inner query to evaluate. We can back that out after ARROW-16628, I believe (should add a comment pointing to that issue too).

Not sure adding a comment referencing ARROW-16628 is needed since I am no longer touching the R6 $Build() method. I added a comment on ARROW-17184.

But you're still calling $Build, and in this case, the inner query will be evaluated inside of that, so we should check before calling $Build.

[dragosmg]

I think this is ready for another review @nealrichardson & @paleolimbot. 😄

[paleolimbot]

I don't like the duplication but as long as its clearly marked and there is a followup JIRA to remove it I'm game. The fact that we have to copy so much code just to print out what's happening suggests to me that we need to improve our abstraction and the solution you've provided here are exactly what we need to do that.

Thank you for sticking with this! I will keep checking https://github.com/dragosmg/arrow/branches for green CI since that's likely to come well before the Arrow CI runs.

[dragosmg]

Thank you for sticking with this! I will keep checking https://github.com/dragosmg/arrow/branches for green CI since that's likely to come well before the Arrow CI runs.

Most checks are green - https://github.com/dragosmg/arrow/runs/7469491011?check_suite_focus=true - with the exception of the first one which has been failing for a few days now (OOM in the large memory tests).

Follow-up Jira: ARROW-17185.

[ursabot]

Benchmark runs are scheduled for baseline = 9ad2255 and contender = 9442e1c. 9442e1c is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Failed ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Failed ⬇️0.63% ⬆️0.0%] test-mac-arm
[Failed ⬇️0.0% ⬆️0.0%] ursa-i9-9960x
[Finished ⬇️0.25% ⬆️0.04%] ursa-thinkcentre-m75q
Buildkite builds:
[Failed] 9442e1ce ec2-t3-xlarge-us-east-2
[Failed] 9442e1ce test-mac-arm
[Failed] 9442e1ce ursa-i9-9960x
[Finished] 9442e1ce ursa-thinkcentre-m75q
[Failed] 9ad22551 ec2-t3-xlarge-us-east-2
[Failed] 9ad22551 test-mac-arm
[Failed] 9ad22551 ursa-i9-9960x
[Finished] 9ad22551 ursa-thinkcentre-m75q
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java