Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(dashboard): commit update once #17781

Merged
merged 1 commit into from
Dec 16, 2021

Conversation

serenajiang
Copy link
Contributor

@serenajiang serenajiang commented Dec 16, 2021

SUMMARY

Recent changes to dashboard updates caused incorrect partial updates to be committed, resulting in malformed json in some db engines (mysql) and breaking dashboards that contained non ascii characters in headers or custom chart titles. This PR ensures that we only commit when all updates are complete.

This issue emerged in #17570. I verified that #17766 seemed to fix the problem in a test env, but it turns out this problem is likely db specific, so it did not actually address the root cause.

The root cause:

  • DashboardDAO.update updates all properties naively, including json_metadata, which includes positions
  • DashboardDAO.update_charts_owners commits the changes done in DashboardDAO.update. Something goes wrong when committing a string with emojis, resulting in a truncated dashboard.json_metadata for mysql.
  • DashboardDAO.set_dash_metadata fetches dashboard.params_dict, which fails to parse due to malformed json. Error is thrown.

Other locations where we save json with emojis inadvertently use json.dumps(json.loads(...)) to encode the emojis in a mysql-friendly way, so we didn't see this before.

TESTING INSTRUCTIONS

Verified change locally with mysql db.

ADDITIONAL INFORMATION

@geido @kgabryje @graceguo-supercat @etr2460

@codecov
Copy link

codecov bot commented Dec 16, 2021

Codecov Report

Merging #17781 (4746c99) into master (c18d486) will decrease coverage by 0.06%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #17781      +/-   ##
==========================================
- Coverage   67.78%   67.72%   -0.07%     
==========================================
  Files        1605     1605              
  Lines       64200    64208       +8     
  Branches     6790     6790              
==========================================
- Hits        43519    43482      -37     
- Misses      18825    18870      +45     
  Partials     1856     1856              
Flag Coverage Δ
hive 81.79% <100.00%> (+<0.01%) ⬆️
mysql 82.17% <100.00%> (+<0.01%) ⬆️
postgres 82.22% <100.00%> (+<0.01%) ⬆️
presto ?
python 82.56% <100.00%> (-0.15%) ⬇️
sqlite 81.90% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
superset/dashboards/commands/update.py 84.28% <100.00%> (+0.46%) ⬆️
superset/db_engine_specs/presto.py 84.34% <0.00%> (-6.06%) ⬇️
superset/connectors/sqla/models.py 87.07% <0.00%> (-1.35%) ⬇️
superset/dashboards/dao.py 95.55% <0.00%> (-0.75%) ⬇️
superset/models/core.py 89.26% <0.00%> (-0.74%) ⬇️
superset/databases/schemas.py 98.47% <0.00%> (+0.02%) ⬆️
superset/reports/commands/alert.py 96.51% <0.00%> (+0.04%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c18d486...4746c99. Read the comment docs.

Copy link
Member

@geido geido left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks for the fix!

@serenajiang serenajiang merged commit 3657cbe into apache:master Dec 16, 2021
eschutho pushed a commit that referenced this pull request Dec 21, 2021
shcoderAlex pushed a commit to casual-precision/superset that referenced this pull request Feb 7, 2022
bwang221 pushed a commit to casual-precision/superset that referenced this pull request Feb 10, 2022
@mistercrunch mistercrunch added 🍒 1.4.0 🍒 1.4.1 🍒 1.4.2 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 1.5.0 labels Mar 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels size/XS v1.4 🍒 1.4.0 🍒 1.4.1 🍒 1.4.2 🚢 1.5.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants