Add example for writing SQL analysis using DataFusion structures #10938
Conversation
|
Thank you @LorrensP-2158466 -- I am going to check this out later today |
There was a problem hiding this comment.
Thank you @LorrensP-2158466 -- this looks great. I left some small suggestions but I think this is basically a great example.
Do you potentially have time to finish it up? Otherwise I can help do so too
|
|
||
| //! This example shows how we can use the structures that DataFusion provide to perform | ||
| //! SQL Analysis. | ||
| //! We'll show how we can count the amount of join's in a query as well as how many |
| )?; | ||
| } | ||
|
|
||
| let query_88 = "datafusion/core/tests/tpc-ds/88.sql"; |
There was a problem hiding this comment.
This is very cool -- thank you
This example might actually be easier to understand if we inlined these queries below -- I can do it if you like -- so the example stands on its own
https://github.com/apache/datafusion/blob/main/datafusion/core/tests/tpc-ds/88.sql
|
It's currently midnight where I live, so I can do it tomorrow. If you have free time you're free to make these changes. |
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
No worries! In general we operate asynchronously, so there is no need to an immediate response. Thank you for making the changes -- I am taking a second look now |
There was a problem hiding this comment.
This is amazing -- thank you so much @LorrensP-2158466
It is really neat to see how to use the TreeNode API from @peter-toth and others to write non trivial analyses that walk the tree.
I took the liberty of merging this branch up to main and fixing the Cargo.toml formatting, as well doing some small textual cleanups
I plan to merge it once the CI checks complete.
Again, thank you, this is so cool
|
Thanks again @LorrensP-2158466 |
|
Glad I could help, it was a lot fun! |
Which issue does this PR close?
Closes #10871 .
Rationale for this change
What changes are included in this PR?
Added an example of an SQL analysis where we count joins in a SQL query.
Are these changes tested?
Ran it on some other queries and the join counts where right.
Are there any user-facing changes?
No