feat(config): Add none force-refresh option for auto refresh

[yongchand]

SUMMARY

Add alternative option for issue #16944
If people think this should be default, I may just remove from config and simply change force -> fetch

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

Before:
https://user-images.githubusercontent.com/43336767/136601879-c3ea7040-5944-43b6-bf05-15d7f29ef22a.mov

When DASHBOARD_FORCE_AUTO_REFRESH is True, chart will query every time when it auto-refreshes.

After:
https://user-images.githubusercontent.com/43336767/136601893-a876f406-6190-4d90-bffb-e78e4eaa8661.mov

When DASHBOARD_FORCE_AUTO_REFRESH is False, chart will use cache if it exist during auto-refresh.

TESTING INSTRUCTIONS

Since this PR requires to change superset-ui to change, you can test by following step instead.

1. Open dashboard with several charts with cache of certain period (Say 300s)
2. Check that chart will not use cache at all (like before video) during auto-refresh
3. Change fetchCharts(xx, True, xx, xx) to fetchCharts(xx, false, xx, xx)
4. Check that chart use cache during auto-refresh

ADDITIONAL INFORMATION

• Has associated issue:
• Required feature flags:
• Changes UI
• Includes DB Migration (follow approval process in SIP-59)
  • Migration is atomic, supports rollback & is backwards-compatible
  • Confirm DB migration upgrade and downgrade tested
  • Runtime estimates and downtime expectations provided
• Introduces new feature or API
• Removes existing feature or API

[yongchand]

Thanks @villebro, I just added PR :) I think it is better to add as a config option so users can choose - what do you think?

[codecov]

Codecov Report

Merging #17039 (27afd22) into master (2c8e06e) will decrease coverage by 0.26%.
The diff coverage is 60.97%.

❗ Current head 27afd22 differs from pull request most recent head 38c1a97. Consider uploading reports for the commit 38c1a97 to get more accurate results

@@            Coverage Diff             @@
##           master   #17039      +/-   ##
==========================================
- Coverage   76.92%   76.66%   -0.27%     
==========================================
  Files        1031     1031              
  Lines       55157    55200      +43     
  Branches     7501     7507       +6     
==========================================
- Hits        42430    42318     -112     
- Misses      12475    12630     +155     
  Partials      252      252              

Flag	Coverage Δ
hive	?
javascript	70.79% <54.92%> (-0.09%)	⬇️
presto	?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...frontend/src/dashboard/components/Header/index.jsx	68.71% <0.00%> (-0.78%)	⬇️
...frontend/src/profile/components/CreatedContent.tsx	87.50% <ø> (ø)
.../database/DatabaseModal/DatabaseConnectionForm.tsx	48.71% <ø> (ø)
...ontend/src/views/CRUD/data/dataset/DatasetList.tsx	66.29% <0.00%> (+0.37%)	⬆️
superset/db_engine_specs/ascend.py	100.00% <ø> (ø)
superset/db_engine_specs/athena.py	89.28% <ø> (ø)
superset/db_engine_specs/base.py	88.17% <ø> (-0.39%)	⬇️
superset/db_engine_specs/bigquery.py	85.97% <ø> (ø)
superset/db_engine_specs/clickhouse.py	66.03% <ø> (ø)
superset/db_engine_specs/crate.py	92.30% <ø> (ø)
... and 46 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2c8e06e...38c1a97. Read the comment docs.

[geido]

Hello @yongchand thanks for the contribution!

I am thinking that feature flags might not be the best place for this type of setting. @villebro correct me if I am wrong, but being a frontend-only constant, probably superset-frontend/src/constants.ts could be a better place for this one.

[yongchand]

@geido Thanks for your opinion!
I was thinking that user should choose whether to enable force-refresh (for auto refresh).
Therefore, config.py could be a good place to simply change the configuration.
In config.py, feature flag like ENABLE_DND_WITH_CLICK_UX is only used on frontend-only too as long as I know.
Please let me know if I am misunderstanding!

[villebro]

I agree with @geido that this should be a general config, not a feature flag. I would propose adding something similar to this to config.py:

DASHBOARD_AUTO_REFRESH_MODE: Literal["fetch", "force"] = "force"

Then that needs to be added to bootstrap data by adding the key to FRONTEND_CONF_KEYS on the backend. After this the value should be available on the dashboard on dashboardInfo.common.conf. But I may be oversimplifying things - let me know if you need help @yongchand!

[yongchand]

Tested locally :) Works totally fine! @geido @villebro
Feedback welcomed :)

[villebro]

Thanks @yongchand! I started CI and will test today

[yongchand]

Done with lint issues! All fixed! (I hope) :)
@villebro

[villebro]

Thanks @yongchand; CI started 👍

[yongchand]

@villebro sorry for keep pinging you! Just fixed isort issue. If this goes fine, please merge whenever you are available :)

[villebro]

sorry for keep pinging you! Just fixed isort issue. If this goes fine, please merge whenever you are available :)

No worries @yongchand, CI restarted 🙂

[yongchand]

sorry for keep pinging you! Just fixed isort issue. If this goes fine, please merge whenever you are available :)

No worries @yongchand, CI restarted 🙂

Everything seems fine except for cypress-matrix (which seemed to be fine on last run). Do you possibly know the reason? @villebro

[villebro]

Everything seems fine except for cypress-matrix (which seemed to be fine on last run). Do you possibly know the reason?

@yongchand seems to be a flaky test - I just restarted it, will try to monitor it today to make sure it passes

[yongchand]

Seems good now! Thanks :) @villebro

[villebro]

LGTM, thanks for all the iterations @yongchand !