feat: filter parameters from DB API #21248

betodealmeida · 2022-08-30T01:29:12Z

SUMMARY

Filter out some of the parameters from the database connection in the API.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

Has associated issue:
Required feature flags:
Changes UI
Includes DB Migration (follow approval process in SIP-59)
- Migration is atomic, supports rollback & is backwards-compatible
- Confirm DB migration upgrade and downgrade tested
- Runtime estimates and downtime expectations provided
Introduces new feature or API
Removes existing feature or API

codecov · 2022-08-30T01:38:44Z

Codecov Report

Merging #21248 (3078aeb) into master (c3a00d4) will decrease coverage by 0.15%.
The diff coverage is 69.31%.

❗ Current head 3078aeb differs from pull request most recent head 7e87ac7. Consider uploading reports for the commit 7e87ac7 to get more accurate results

@@            Coverage Diff             @@
##           master   #21248      +/-   ##
==========================================
- Coverage   66.44%   66.29%   -0.16%     
==========================================
  Files        1786     1786              
  Lines       68267    68348      +81     
  Branches     7263     7264       +1     
==========================================
- Hits        45357    45308      -49     
- Misses      21041    21170     +129     
- Partials     1869     1870       +1

Flag	Coverage Δ
hive	`?`
mysql	`80.82% <69.23%> (?)`
postgres	`80.87% <69.23%> (-0.02%)`	⬇️
presto	`?`
python	`80.98% <69.23%> (-0.34%)`	⬇️
sqlite	`79.41% <69.23%> (-0.03%)`	⬇️
unit	`?`

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
superset-frontend/src/SqlLab/App.jsx	`0.00% <ø> (ø)`
...d/src/SqlLab/components/TabbedSqlEditors/index.jsx	`55.17% <0.00%> (ø)`
...ntend/src/SqlLab/components/TableElement/index.tsx	`72.30% <0.00%> (ø)`
superset-frontend/src/SqlLab/constants.ts	`100.00% <ø> (ø)`
superset-frontend/src/utils/localStorageHelpers.ts	`90.00% <ø> (ø)`
superset-frontend/src/views/App.tsx	`0.00% <0.00%> (ø)`
...aseModal/DatabaseConnectionForm/EncryptedField.tsx	`10.00% <0.00%> (ø)`
...c/views/CRUD/data/database/DatabaseModal/index.tsx	`32.11% <0.00%> (+0.07%)`	⬆️
superset/databases/api.py	`95.49% <ø> (ø)`
superset/models/helpers.py	`39.21% <0.00%> (ø)`
... and 55 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

betodealmeida · 2022-08-30T03:55:55Z

superset/models/core.py

@@ -339,14 +339,6 @@ def get_effective_user(self, object_url: URL) -> Optional[str]:
            else None
        )

-    @memoized(


This is not working for BigQuery, had to disable.

betodealmeida · 2022-08-30T03:56:04Z

superset/models/core.py

@@ -639,7 +631,6 @@ def db_engine_spec(self) -> Type[db_engine_specs.BaseEngineSpec]:
        return self.get_db_engine_spec(url)

    @classmethod
-    @memoized


Same here, not working for BigQuery.

villebro

LGTM

yousoph · 2022-08-30T06:11:41Z

/testenv up

github-actions · 2022-08-30T06:19:45Z

@yousoph Ephemeral environment spinning up at http://52.12.84.244:8080. Credentials are admin/admin. Please allow several minutes for bootstrapping and startup.

superset/db_engine_specs/bigquery.py

dpgaspar

tests are failing

superset/databases/dao.py

jinghua-qa · 2022-09-02T16:04:33Z

for the skip test, do we have a plan to get it fix and re-enable?

jinghua-qa · 2022-09-02T16:46:10Z

/testenv up

betodealmeida · 2022-09-02T16:47:21Z

for the skip test, do we have a plan to get it fix and re-enable?

I'm not sure why these were timing out, they are unrelated to this PR. I need to take a better look.

github-actions · 2022-09-02T17:32:36Z

@jinghua-qa Ephemeral environment spinning up at http://34.221.65.136:8080. Credentials are admin/admin. Please allow several minutes for bootstrapping and startup.

jinghua-qa

LGTM

github-actions · 2022-09-02T18:50:24Z

Ephemeral environment shutdown and build artifacts deleted.

sadpandajoe · 2022-09-02T19:39:17Z

🏷️ preset:2022.33

feat: filter parameters from DB API (apache#21248)

pull-request-size bot added the size/M label Aug 30, 2022

betodealmeida requested review from AAfghahi, eschutho, villebro and dpgaspar and removed request for villebro August 30, 2022 01:31

AAfghahi approved these changes Aug 30, 2022

View reviewed changes

pull-request-size bot added size/L and removed size/M labels Aug 30, 2022

betodealmeida commented Aug 30, 2022

View reviewed changes

eschutho approved these changes Aug 30, 2022

View reviewed changes

betodealmeida force-pushed the filter_parameters_from_db branch from 93c20a7 to ec01948 Compare August 30, 2022 05:35

villebro approved these changes Aug 30, 2022

View reviewed changes

dpgaspar approved these changes Aug 30, 2022

View reviewed changes

superset/db_engine_specs/bigquery.py Outdated Show resolved Hide resolved

dpgaspar reviewed Aug 30, 2022

View reviewed changes

betodealmeida force-pushed the filter_parameters_from_db branch from ec01948 to f28bfbc Compare August 30, 2022 14:11

hughhhh reviewed Aug 30, 2022

View reviewed changes

superset/databases/dao.py Outdated Show resolved Hide resolved

pull-request-size bot added size/XL and removed size/L labels Sep 1, 2022

betodealmeida force-pushed the filter_parameters_from_db branch 5 times, most recently from da55a5a to 6379dbc Compare September 1, 2022 06:13

betodealmeida added 2 commits September 1, 2022 09:14

feat: filter parameters from DB API

21373d4

Small fixes

175df9c

betodealmeida added 10 commits September 1, 2022 09:14

Add unit tests

bef55cd

Add tests

27e348f

Fix Postgres

50854ee

Wrap json decode

2a396da

Tentative fix for test

82ab28c

Skip test

af88aa1

Add mask_encrypted_extra

c918ba1

Mask encrypted_extra

0d2771f

Fix serialization

5279d7f

Fix tests

7c5fe05

betodealmeida force-pushed the filter_parameters_from_db branch from 6379dbc to 7c5fe05 Compare September 1, 2022 16:14

Skip tests that are timing out

7e87ac7

betodealmeida requested a review from jinghua-qa as a code owner September 2, 2022 14:38

jinghua-qa approved these changes Sep 2, 2022

View reviewed changes

betodealmeida merged commit 34a79ad into apache:master Sep 2, 2022

betodealmeida added a commit to betodealmeida/incubator-superset that referenced this pull request Sep 2, 2022

feat: filter parameters from DB API (apache#21248)

3f94ff1

betodealmeida added a commit to betodealmeida/incubator-superset that referenced this pull request Sep 2, 2022

feat: filter parameters from DB API (apache#21248)

358550f

sadpandajoe added a commit to preset-io/superset that referenced this pull request Sep 2, 2022

Merge pull request #266 from betodealmeida/cherry_pick_21248

d78e17c

feat: filter parameters from DB API (apache#21248)

sadpandajoe added a commit to preset-io/superset that referenced this pull request Sep 2, 2022

Merge pull request #267 from betodealmeida/cherry_pick_21248b

753371c

feat: filter parameters from DB API (apache#21248)

betodealmeida mentioned this pull request Oct 24, 2022

fix: accept old database payload #21923

Merged

9 tasks

mistercrunch added the 🚢 2.1.3 label Feb 18, 2024

mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 2.1.0 and removed 🚢 2.1.3 labels Mar 13, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: filter parameters from DB API #21248

feat: filter parameters from DB API #21248

betodealmeida commented Aug 30, 2022

codecov bot commented Aug 30, 2022 •

edited

betodealmeida Aug 30, 2022

betodealmeida Aug 30, 2022

villebro left a comment

yousoph commented Aug 30, 2022

github-actions bot commented Aug 30, 2022

dpgaspar left a comment •

edited

jinghua-qa commented Sep 2, 2022

jinghua-qa commented Sep 2, 2022

betodealmeida commented Sep 2, 2022

github-actions bot commented Sep 2, 2022

jinghua-qa left a comment

github-actions bot commented Sep 2, 2022

sadpandajoe commented Sep 2, 2022

feat: filter parameters from DB API #21248

feat: filter parameters from DB API #21248

Conversation

betodealmeida commented Aug 30, 2022

SUMMARY

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

codecov bot commented Aug 30, 2022 • edited

Codecov Report

betodealmeida Aug 30, 2022

Choose a reason for hiding this comment

betodealmeida Aug 30, 2022

Choose a reason for hiding this comment

villebro left a comment

Choose a reason for hiding this comment

yousoph commented Aug 30, 2022

github-actions bot commented Aug 30, 2022

dpgaspar left a comment • edited

Choose a reason for hiding this comment

jinghua-qa commented Sep 2, 2022

jinghua-qa commented Sep 2, 2022

betodealmeida commented Sep 2, 2022

github-actions bot commented Sep 2, 2022

jinghua-qa left a comment

Choose a reason for hiding this comment

github-actions bot commented Sep 2, 2022

sadpandajoe commented Sep 2, 2022

codecov bot commented Aug 30, 2022 •

edited

dpgaspar left a comment •

edited