Serialize nested columns as JSON strings #9007
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov Report
@@ Coverage Diff @@
## master #9007 +/- ##
=======================================
Coverage 59.16% 59.16%
=======================================
Files 367 367
Lines 11680 11680
Branches 2863 2863
=======================================
Hits 6910 6910
Misses 4591 4591
Partials 179 179Continue to review full report at Codecov.
|
nytai
approved these changes
Jan 24, 2020
| if pa.types.is_temporal(pa_data[i].type): | ||
| # TODO: revisit nested column serialization once Arrow 1.0 is released with: | ||
| # https://github.com/apache/arrow/pull/6199 | ||
| # Related issue: #8978 |
etr2460
pushed a commit
to etr2460/incubator-superset
that referenced
this pull request
Feb 1, 2020
This reverts commit 2fc3d84.
mistercrunch
added
🏷️ bot
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
CATEGORY
Choose one
SUMMARY
SQL Lab queries against databases with nested columns/results containing structs or maps are failing with the following error:
Serializing results to JSON is not the ideal solution, mainly for performance reasons, but there are pending fixes yet to be released in Arrow 1.0 that may improve the situation:
apache/arrow#6199
BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
TEST PLAN
Ensure queries against databases containing nested columns (particularly arrays containing maps) are successful. Per #8978, the following query should succeed and produce the proper results:
ADDITIONAL INFORMATION
REVIEWERS
@lxhoang97 @graceguo-supercat @john-bodley @michellethomas @villebro