Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: allow for multiple columns in pivot table report #17636

Merged
merged 3 commits into from
Dec 3, 2021

Conversation

eschutho
Copy link
Member

@eschutho eschutho commented Dec 2, 2021

SUMMARY

This fixes a 500 error when running a pivot table v2 report on a chart with multiple columns and no rows
On a pandas dataframe stack function:
if the columns have a single level, the output is a Series;
if the columns have multiple levels, the new index level(s) is (are) taken from the prescribed level(s) and the output is a DataFrame.

For a chart with multiple columns, a dataframe does not have the to_frame method, and was erroring.
This pr checks for the return type of stack before calling to_frame

Second fix: some charts that have null values were having problems when pivoting because pandas will drop some data and then later filtering and manipulation can cause errors. I resolved this by filling in null values with a "NULL" string.

*some work still needs to be done to improve the formatting of a pivot chart report with multiple columns.

TESTING INSTRUCTIONS

Create a chart with pivot v2 with multiple columns and no rows and make sure that a chart as both a csv and table can be sent as a report.
Create a pivot v2 chart with a column of all null values and select that as a chart column. Reports should send correctly for csv and test.

ADDITIONAL INFORMATION

  • Has associated issue:
  • Required feature flags:
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

@eschutho eschutho changed the title allow for multiple columns in pivot table report fix; allow for multiple columns in pivot table report Dec 2, 2021
@eschutho eschutho changed the title fix; allow for multiple columns in pivot table report fix: allow for multiple columns in pivot table report Dec 2, 2021
@codecov
Copy link

codecov bot commented Dec 2, 2021

Codecov Report

Merging #17636 (1e03ebd) into master (7353a2b) will decrease coverage by 0.07%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #17636      +/-   ##
==========================================
- Coverage   76.81%   76.73%   -0.08%     
==========================================
  Files        1049     1062      +13     
  Lines       56653    57005     +352     
  Branches     7847     7847              
==========================================
+ Hits        43517    43745     +228     
- Misses      12883    13007     +124     
  Partials      253      253              
Flag Coverage Δ
mysql 81.80% <100.00%> (-0.21%) ⬇️
postgres 81.81% <100.00%> (-0.21%) ⬇️
python 81.89% <100.00%> (-0.21%) ⬇️
sqlite 81.49% <100.00%> (-0.21%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
superset/charts/post_processing.py 68.21% <100.00%> (+1.01%) ⬆️
superset/utils/cache_manager.py 96.15% <0.00%> (-3.85%) ⬇️
superset/connectors/base/models.py 88.19% <0.00%> (-0.49%) ⬇️
superset/dashboards/api.py 91.86% <0.00%> (-0.29%) ⬇️
superset/charts/commands/export.py 94.11% <0.00%> (-0.17%) ⬇️
superset/cli.py 54.82% <0.00%> (-0.11%) ⬇️
superset/views/core.py 77.38% <0.00%> (-0.06%) ⬇️
superset/db_engine_specs/base.py 88.15% <0.00%> (-0.05%) ⬇️
superset/constants.py 100.00% <0.00%> (ø)
superset/db_engine_specs/hana.py 95.00% <0.00%> (ø)
... and 48 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7353a2b...1e03ebd. Read the comment docs.

Copy link
Member

@betodealmeida betodealmeida left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

6 lines of code, 583 of tests! <3

tests/unit_tests/charts/test_post_processing.py Outdated Show resolved Hide resolved
Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>
@eschutho
Copy link
Member Author

eschutho commented Dec 3, 2021

6 lines of code, 583 of tests! <3

Props to you for writing the original tests. :) This code was very easy to work with as a result!

@eschutho eschutho merged commit 13e1929 into apache:master Dec 3, 2021
@eschutho eschutho deleted the elizabeth/fix-pivot-report branch December 3, 2021 00:42
@mistercrunch mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 1.5.0 labels Mar 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels preset-io size/XL 🚢 1.5.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants