Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add dataset tagging to the back-end #20892

Merged
merged 40 commits into from
Sep 23, 2022

Conversation

cccs-Dustin
Copy link
Contributor

@cccs-Dustin cccs-Dustin commented Jul 27, 2022

SUMMARY

In the current version of the master branch, the back-end only supports tagging for 'Charts', 'Dashboards' and 'Saved Queries'. However, this PR would add the back-end support for tagging 'Datasets' as well. This is just the first of at least 2 PR's, the second of which will introduce new API endpoints as well as add dataset tags to the front-end.

This PR also incorporates a few changes to files for refactoring purposes:

  1. The model for tags was moved from superset/models/tags.py to superset/tags/models.py (all of the import statements have been updated to point to the new location). This was done so that going forward, the model will be in the same location as the API and command code, which will be added later on.
  2. The SQLAlchemy event listeners for adding, updating, and removing tags have been removed from each model, and were instead placed into a singular location: superset/tags/core.py. The event listeners are now registered from within the init_app_in_ctx method in the superset/initialization/__init__.py file, and are placed behind a check for the TAGGING_SYSTEM feature flag.

NOTE: If you are not starting from a 'fresh' database, be sure to run the following command: superset sync_tags. This will add tags to all of the existing datasets.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

N/A

TESTING INSTRUCTIONS

To manually verify the changes, first connect to the local database, then run the following:

Before Code Changes:

  1. SELECT * FROM tag; (you should not see a dataset entry)
  2. SELECT * FROM tagged_object; (you should not see any entry with the dataset tag/type)

After Code Changes:

  1. SELECT * FROM tag; (you should now see a dataset entry)
  2. SELECT * FROM tagged_object; (you should now see some entries with the dataset tag/type)

ADDITIONAL INFORMATION

  • Has associated issue:
  • Required feature flags: TAGGING_SYSTEM
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

@codecov
Copy link

codecov bot commented Jul 27, 2022

Codecov Report

Merging #20892 (3611579) into master (b739e27) will increase coverage by 0.01%.
The diff coverage is 53.04%.

@@            Coverage Diff             @@
##           master   #20892      +/-   ##
==========================================
+ Coverage   66.66%   66.68%   +0.01%     
==========================================
  Files        1793     1794       +1     
  Lines       68492    68562      +70     
  Branches     7277     7277              
==========================================
+ Hits        45663    45722      +59     
- Misses      20967    20978      +11     
  Partials     1862     1862              
Flag Coverage Δ
hive 52.89% <15.65%> (-0.19%) ⬇️
mysql 78.21% <53.04%> (+0.01%) ⬆️
postgres 78.28% <53.04%> (+0.01%) ⬆️
presto 52.79% <15.65%> (-0.19%) ⬇️
python 81.41% <53.04%> (+<0.01%) ⬆️
sqlite 76.78% <53.04%> (+0.01%) ⬆️
unit 50.91% <14.78%> (-0.15%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
superset/common/tags.py 0.00% <0.00%> (ø)
superset/models/dashboard.py 76.80% <ø> (+0.72%) ⬆️
superset/models/slice.py 86.20% <ø> (+1.11%) ⬆️
superset/models/sql_lab.py 76.51% <ø> (-0.38%) ⬇️
superset/views/tags.py 48.93% <40.00%> (-1.07%) ⬇️
superset/initialization/__init__.py 91.46% <66.66%> (-0.26%) ⬇️
superset/tags/models.py 88.42% <90.90%> (ø)
superset/models/core.py 90.60% <100.00%> (+0.41%) ⬆️
superset/tags/core.py 100.00% <100.00%> (ø)
superset/tasks/cache.py 61.15% <100.00%> (ø)
... and 8 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@cccs-Dustin cccs-Dustin changed the title [WIP] feat: Add dataset tagging to the back-end feat: Add dataset tagging to the back-end Aug 2, 2022
Copy link
Member

@AAfghahi AAfghahi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you add some unit tests to make sure that the commands are working as expected.

@cccs-Dustin
Copy link
Contributor Author

Closing this PR so that I can add unit tests to it. I will create a new PR once that is done.

@cccs-Dustin cccs-Dustin closed this Aug 8, 2022
@cccs-Dustin cccs-Dustin reopened this Aug 25, 2022
@cccs-Dustin cccs-Dustin reopened this Aug 26, 2022
@cccs-Dustin cccs-Dustin requested review from AAfghahi and removed request for john-bodley, eschutho and lyndsiWilliams August 26, 2022 14:26
@cccs-Dustin cccs-Dustin changed the title feat: Add dataset tagging to the back-end [WIP] feat: Add dataset tagging to the back-end Aug 26, 2022
@cccs-Dustin cccs-Dustin requested review from villebro and AAfghahi and removed request for AAfghahi, betodealmeida, rusackas and villebro September 20, 2022 13:47
Copy link
Member

@villebro villebro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! 👍 One minor comment regarding the tagging system fixture, but I don't consider it blocking, as flipping a default feature flag will always require some test refactoring anyway.

tests/integration_tests/fixtures/tags.py Outdated Show resolved Hide resolved
Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>
Copy link
Member

@dpgaspar dpgaspar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!
One question, why the move of /superset/models/tags.py to /superset/tags/models.py?

Note: We are currently moving and deprecating all endpoints from /supeset to /api/v1

@villebro villebro dismissed AAfghahi’s stale review September 23, 2022 08:00

The requested changes have been made

@villebro
Copy link
Member

Could you add some unit tests to make sure that the commands are working as expected.

FYI @AAfghahi I removed the change request from your review as the requested tests have been added to the PR

@villebro villebro merged commit 2e56489 into apache:master Sep 23, 2022
@cccs-Dustin cccs-Dustin deleted the Dataset-Tagging branch September 23, 2022 11:16
@mistercrunch mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 2.1.0 and removed 🚢 2.1.3 labels Mar 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels size/XL 🚢 2.1.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants