New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ARROW-17066: [C++][Python][Substrait] "ignore_unknown_fields" should be specified when converting JSON to binary #13605
Conversation
cc @westonpace Not exactly sure if this is the expected fix. I may need some help to figure out a better test case too. |
|
|
Need to figure out a better way to represent substrait plans in strings. |
I'll try to test it out using your branch |
I tried your branch, but since it also has the latest substrait now, i'm actually hitting some other issues, that Weston will be fixing. ARROW-15582/ARROW-15538 |
@vibhatha Can you update the PR description since it will end up as the commit message? |
@pitrou Sure I will update it. |
@pitrou Updated the description. |
@vibhatha Well, can you make the PR description actually descriptive, instead of merely repeting the PR title? Thanks :-) |
i.e., explain why this is needed. |
I will modify it 👍 |
@pitrou could you please check the updated description? Is it okay? |
Benchmark runs are scheduled for baseline = 32016b1 and contender = 791e5bd. 791e5bd is a master commit associated with this PR. Results will be available as each benchmark for each run completes. |
Substrait is continously changing and it introduces may unknown fields in the consumed plan. Since it is not practical to support all these fields simultaneously, we have to include a way to ignore such fields. To support that, the
ignore_unknown_fields
is added to theJsonParseOptions
.