allow cache and force refresh on table list #6078

youngyjd · 2018-10-11T23:44:26Z

This is a follow-up PR for #5933

Added table_cache_timeout settings to metadata_cache_timeout in database configuration. If table_cache_timeout is unset in the field, cache will not be enabled for the metadata fetch of this database. If it is set to 0, cache will never time out.
Added a button to sqllab UI which allows user to refetch table list metadata, or force refresh cache if cache is enabled.

youngyjd · 2018-10-12T17:56:29Z

superset/assets/src/SqlLab/components/SqlEditorLeftBar.jsx

-    if (!(db)) {
-      this.setState({ tableOptions: [] });
-    } else {
-      this.fetchTables(val, this.props.queryEditor.schema);


fetchTables here makes no sense because the database has been changed but this.props.queryEditor.schema is still the schema selected for the previously selected db.
And it is set null on line 44:
this.props.actions.queryEditorSetSchema(this.props.queryEditor, null);

so I think removing it is necessary.

codecov-io · 2018-10-12T19:06:30Z

Codecov Report

Merging #6078 into master will decrease coverage by 0.32%.
The diff coverage is 45.16%.

@@            Coverage Diff             @@
##           master    #6078      +/-   ##
==========================================
- Coverage   77.56%   77.23%   -0.33%     
==========================================
  Files          47       47              
  Lines        9484     9349     -135     
==========================================
- Hits         7356     7221     -135     
  Misses       2128     2128

Impacted Files	Coverage Δ
superset/views/core.py	`74.08% <0%> (-0.11%)`	⬇️
superset/models/core.py	`83.87% <36.36%> (-1.24%)`	⬇️
superset/cache_util.py	`48.14% <50%> (-6.14%)`	⬇️
superset/db_engine_specs.py	`56.1% <61.53%> (+0.23%)`	⬆️
superset/connectors/base/views.py	`71.42% <0%> (-10.39%)`	⬇️
superset/dict_import_export_util.py	`21.05% <0%> (-7.52%)`	⬇️
superset/legacy.py	`13.43% <0%> (-4.88%)`	⬇️
superset/stats_logger.py	`60.97% <0%> (-3.47%)`	⬇️
superset/translations/utils.py	`86.66% <0%> (-2.81%)`	⬇️
... and 40 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e37b83a...b858166. Read the comment docs.

youngyjd · 2018-10-12T19:33:54Z

Please take a look @betodealmeida

mistercrunch

Few minor changes requested, otherwise LGTM

mistercrunch · 2018-10-13T17:16:07Z

superset/assets/src/SqlLab/components/SqlEditorLeftBar.jsx

@@ -41,12 +41,10 @@ class SqlEditorLeftBar extends React.PureComponent {
  onDatabaseChange(db, force) {
    const val = db ? db.value : null;
    this.setState({ schemaOptions: [] });
+    this.setState({ tableOptions: [] });


A single setState call with ({ schemaOptions: [], tableOptions: [] }) would be better as it would trigger a single render call instead of 2.

mistercrunch · 2018-10-13T17:25:05Z

superset/assets/src/SqlLab/components/SqlEditorLeftBar.jsx

@@ -196,7 +195,7 @@ class SqlEditorLeftBar extends React.PureComponent {
              <RefreshLabel
                onClick={this.onDatabaseChange.bind(
                    this, { value: database.id }, true)}
-                tooltipContent="force refresh schema list"
+                tooltipContent="force refresh table list"


shouldn't this say schema?

Also, don't forget i18n as in tooltipContent={t('force refresh table list')}

woops. my bad.

mistercrunch · 2018-10-13T17:25:32Z

superset/assets/src/SqlLab/components/SqlEditorLeftBar.jsx

+              <RefreshLabel
+                onClick={this.changeSchema.bind(
+                    this, { value: this.props.queryEditor.schema }, true)}
+                tooltipContent="force refresh schema list"


shouldn't this say table list?

mistercrunch · 2018-10-13T17:34:15Z

superset/db_engine_specs.py

@@ -318,9 +318,25 @@ def get_schema_names(cls, inspector, db_id,
        return inspector.get_schema_names()

    @classmethod
-    def get_table_names(cls, schema, inspector):
+    @cache_util.memoized_func(
+        enable_cache=lambda *args, **kwargs: kwargs.get('enable_cache', False),


The lambda stuff looks a bit nasty to me, had to go read the decorator docstring to figure out what is going on here. I understand this is outside the scope of this PR so it's ok.

Refactored the decorator a little to avoid too many lambda functions passed to the decorator. Please let me know how I can improve it better.

youngyjd · 2018-10-15T23:26:16Z

@mistercrunch Please take another look.

* allow cache and force refresh on table list * wording * flake8 * javascript test * address comments * nit (cherry picked from commit 177bed3)

* allow cache and force refresh on table list * wording * flake8 * javascript test * address comments * nit (cherry picked from commit 177bed3) (cherry picked from commit 214edcc)

* allow cache and force refresh on table list * wording * flake8 * javascript test * address comments * nit

youngyjd force-pushed the allow-cache-and-force-refresh-on-table-list branch from d516d61 to 853d874 Compare October 11, 2018 23:49

youngyjd commented Oct 12, 2018

View reviewed changes

mistercrunch requested changes Oct 13, 2018

View reviewed changes

youngyjd added 6 commits October 15, 2018 14:39

allow cache and force refresh on table list

79705d2

wording

2d88ec3

flake8

69f131a

javascript test

0725e5f

address comments

488bdff

nit

c147497

youngyjd force-pushed the allow-cache-and-force-refresh-on-table-list branch from 59f5818 to c147497 Compare October 15, 2018 21:42

mistercrunch approved these changes Oct 16, 2018

View reviewed changes

betodealmeida merged commit 177bed3 into apache:master Oct 16, 2018

bipinsoniguavus pushed a commit to ThalesGroup/incubator-superset that referenced this pull request Dec 26, 2018

allow cache and force refresh on table list (apache#6078)

2bbf0ee

* allow cache and force refresh on table list * wording * flake8 * javascript test * address comments * nit

matrix-compute mentioned this pull request Dec 27, 2023

[Snyk] Security upgrade axios from 1.6.0 to 1.6.3 matrix-compute/superset#29

Open

laxmanraparthi mentioned this pull request Dec 27, 2023

[Snyk] Security upgrade axios from 0.25.0 to 1.6.3 ShareChat/superset#102

Open

laxmanraparthi mentioned this pull request Jan 5, 2024

[Snyk] Security upgrade axios from 0.25.0 to 1.6.4 ShareChat/superset#111

Open

mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 0.34.0 labels Feb 27, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

allow cache and force refresh on table list #6078

allow cache and force refresh on table list #6078

youngyjd commented Oct 11, 2018 •

edited

Loading

youngyjd Oct 12, 2018 •

edited

Loading

codecov-io commented Oct 12, 2018 •

edited

Loading

youngyjd commented Oct 12, 2018

mistercrunch left a comment

mistercrunch Oct 13, 2018

mistercrunch Oct 13, 2018

mistercrunch Oct 13, 2018

youngyjd Oct 15, 2018

mistercrunch Oct 13, 2018

mistercrunch Oct 13, 2018

mistercrunch Oct 13, 2018

youngyjd Oct 15, 2018

youngyjd commented Oct 15, 2018

allow cache and force refresh on table list #6078

allow cache and force refresh on table list #6078

Conversation

youngyjd commented Oct 11, 2018 • edited Loading

youngyjd Oct 12, 2018 • edited Loading

Choose a reason for hiding this comment

codecov-io commented Oct 12, 2018 • edited Loading

Codecov Report

youngyjd commented Oct 12, 2018

mistercrunch left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

youngyjd commented Oct 15, 2018

youngyjd commented Oct 11, 2018 •

edited

Loading

youngyjd Oct 12, 2018 •

edited

Loading

codecov-io commented Oct 12, 2018 •

edited

Loading