Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: send post-processed data in reports #15953

Merged
merged 4 commits into from
Jul 30, 2021

Conversation

betodealmeida
Copy link
Member

SUMMARY

For text-based chart reports (pivot tables and regular tables) we want to send the data transformed, so that the user sees the same data they would see on the chart. This PR changes reports to use the new ChartDataResultType.POST_PROCESSED result type for that.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

Here's a simple table:

Screenshot 2021-07-29 at 12-21-02  DEV  Top Timezones

Here's how the new "text" report data format looks like in Slack:

Screen Shot 2021-07-29 at 12 22 35 PM

Here's how it looks like in the email:

Screen Shot 2021-07-29 at 12 24 05 PM

Here's a simple pivot table:

Screenshot 2021-07-29 at 12-24-42  DEV  Pivot Table

Here's how it looks like in Slack as "text":

Screen Shot 2021-07-29 at 12 25 28 PM

And email:

Screen Shot 2021-07-29 at 12 26 25 PM

The CSV is also pivoted, here's an example in the email notification:

Screen Shot 2021-07-29 at 12 27 34 PM

Pivot v2 looks just like Pivot v1.

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

  • Has associated issue:
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

@@ -637,9 +632,7 @@ def get_data(self, pk: int) -> Response:
except (TypeError, json.decoder.JSONDecodeError):
form_data = {}

return self.get_data_response(
command, viz_type=chart.viz_type, form_data=form_data
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

viz_type is also a field in form_data, so we just need to pass form_data around.

@codecov
Copy link

codecov bot commented Jul 29, 2021

Codecov Report

Merging #15953 (e0ea3aa) into master (f4739f4) will increase coverage by 0.07%.
The diff coverage is 74.75%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #15953      +/-   ##
==========================================
+ Coverage   76.98%   77.05%   +0.07%     
==========================================
  Files         988      988              
  Lines       52378    52378              
  Branches     6622     6622              
==========================================
+ Hits        40322    40360      +38     
+ Misses      11833    11795      -38     
  Partials      223      223              
Flag Coverage Δ
hive 81.32% <74.75%> (-0.06%) ⬇️
mysql 81.58% <74.75%> (-0.06%) ⬇️
postgres 81.64% <74.75%> (-0.02%) ⬇️
presto 81.48% <74.75%> (?)
python 82.16% <74.75%> (+0.14%) ⬆️
sqlite 81.28% <74.75%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
superset/models/slice.py 86.18% <ø> (ø)
superset/reports/notifications/slack.py 86.95% <72.72%> (-3.21%) ⬇️
superset/charts/api.py 85.71% <75.00%> (+0.22%) ⬆️
superset/charts/post_processing.py 77.01% <75.00%> (-2.99%) ⬇️
superset/reports/commands/execute.py 92.50% <100.00%> (+0.03%) ⬆️
superset/models/core.py 89.87% <0.00%> (+0.25%) ⬆️
superset/connectors/sqla/models.py 89.83% <0.00%> (+1.40%) ⬆️
superset/db_engine_specs/presto.py 90.37% <0.00%> (+5.85%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f4739f4...e0ea3aa. Read the comment docs.

@@ -55,7 +55,7 @@

class Slice(
Model, AuditMixinNullable, ImportExportMixin
): # pylint: disable=too-many-public-methods
): # pylint: disable=too-many-public-methods, too-many-instance-attributes
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we could just update the max attributes
https://github.com/apache/superset/blob/master/.pylintrc#L385

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmmm, I'm not sure I want to change the settings for the whole codebase. I like being conservative.

@betodealmeida betodealmeida merged commit 2d61f15 into apache:master Jul 30, 2021
opus-42 pushed a commit to opus-42/incubator-superset that referenced this pull request Nov 14, 2021
* feat: send post-processed data in reports

* Fix tests and lint

* Use enums

* Limit Slack message to 4k chars
cccs-RyanS pushed a commit to CybercentreCanada/superset that referenced this pull request Dec 17, 2021
* feat: send post-processed data in reports

* Fix tests and lint

* Use enums

* Limit Slack message to 4k chars
QAlexBall pushed a commit to QAlexBall/superset that referenced this pull request Dec 29, 2021
* feat: send post-processed data in reports

* Fix tests and lint

* Use enums

* Limit Slack message to 4k chars
cccs-rc pushed a commit to CybercentreCanada/superset that referenced this pull request Mar 6, 2024
* feat: send post-processed data in reports

* Fix tests and lint

* Use enums

* Limit Slack message to 4k chars
@mistercrunch mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 1.3.0 labels Mar 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels size/XL 🚢 1.3.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants