fix: Remove`time_range_endpoints` from query context object #19423

hughhhh · 2022-03-30T01:28:43Z

SUMMARY

When running reports/alerts charts are getting the following 400 error, due deprecating time_range_endpoints field.

{"message":"Request is incorrect: {'queries': {0: {'extras': {'time_range_endpoints': ['Unknown field.']}}}}"}

To fix this we'll be running a migration to remove all the time_range_endpoints keys from the slice.query_context. We are only updating the query_context that are valid json

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

Has associated issue:
Required feature flags:
Changes UI
Includes DB Migration (follow approval process in SIP-59)
- Migration is atomic, supports rollback & is backwards-compatible
- Confirm DB migration upgrade and downgrade tested
- Runtime estimates and downtime expectations provided
Introduces new feature or API
Removes existing feature or API

Benchmark Results:
Current: 0.42 s
10+: 0.45 s
100+: 0.34 s
1000+: 0.33 s
Cleaning up DB

Downgrade:

INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
INFO  [alembic.runtime.migration] Running downgrade 2ed890b36b94 -> 58df9d617f14, rm_time_range_endpoints_from_qc

Upgrade:

INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
INFO  [alembic.runtime.migration] Running upgrade 58df9d617f14 -> 2ed890b36b94, rm_time_range_endpoints_from_qc

codecov · 2022-03-30T02:18:43Z

Codecov Report

Merging #19423 (238dceb) into master (8e29ec5) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master   #19423   +/-   ##
=======================================
  Coverage   66.57%   66.57%           
=======================================
  Files        1675     1675           
  Lines       64092    64092           
  Branches     6519     6519           
=======================================
  Hits        42672    42672           
  Misses      19729    19729           
  Partials     1691     1691

Flag	Coverage Δ
hive	`52.70% <ø> (ø)`
javascript	`51.25% <ø> (ø)`
mysql	`81.91% <ø> (ø)`
postgres	`81.95% <ø> (ø)`
presto	`52.55% <ø> (ø)`
python	`82.39% <ø> (ø)`
sqlite	`81.72% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8e29ec5...238dceb. Read the comment docs.

zhaoyongjie

add some codes suggestion.

superset/migrations/versions/2ed890b36b94_rm_time_range_endpoints_from_qc.py

…nts_from_qc.py Co-authored-by: Yongjie Zhao <yongjie.zhao@gmail.com>

github-actions · 2022-03-30T05:34:50Z

⚠️ @hughhhh Your base branch master has just also updated superset/migrations.

❗ Please consider rebasing your branch to avoid db migration conflicts.

eschutho

Thanks @hughhhh. LGTM.

eschutho · 2022-03-30T17:30:58Z

@hughhhh before merging, I would recommend also running through the db migration benchmark script and checkboxes in the PR description and add the results of the benchmark to the description.

eschutho · 2022-03-30T17:40:06Z

This PR is a follow up to #18936

yousoph · 2022-03-30T18:19:40Z

Before this fix, users were getting an error when requesting a CSV or text report. PNG reports were working as expected

Repro steps:

Create a chart of a Table or Pivot Table
Add a time range filter onto your chart
Create a report of your chart

Current behavior:
Report owner will receive an email with a 400 error instead of the report

betodealmeida

LGTM, just a suggestion on the try/except.

superset/migrations/versions/2ed890b36b94_rm_time_range_endpoints_from_qc.py

…nts_from_qc.py Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

…ime-range

sadpandajoe · 2022-03-31T00:28:07Z

🏷️ preset:2022.11

…9423) * template for remove time_range_endpoints from query context * Update superset/migrations/versions/2ed890b36b94_rm_time_range_endpoints_from_qc.py Co-authored-by: Yongjie Zhao <yongjie.zhao@gmail.com> * Update superset/migrations/versions/2ed890b36b94_rm_time_range_endpoints_from_qc.py Co-authored-by: Yongjie Zhao <yongjie.zhao@gmail.com> * fix reference * fix column reference * fix * Update superset/migrations/versions/2ed890b36b94_rm_time_range_endpoints_from_qc.py Co-authored-by: Beto Dealmeida <roberto@dealmeida.net> Co-authored-by: Yongjie Zhao <yongjie.zhao@gmail.com> Co-authored-by: Beto Dealmeida <roberto@dealmeida.net> (cherry picked from commit 129063d)

* template for remove time_range_endpoints from query context * Update superset/migrations/versions/2ed890b36b94_rm_time_range_endpoints_from_qc.py Co-authored-by: Yongjie Zhao <yongjie.zhao@gmail.com> * Update superset/migrations/versions/2ed890b36b94_rm_time_range_endpoints_from_qc.py Co-authored-by: Yongjie Zhao <yongjie.zhao@gmail.com> * fix reference * fix column reference * fix * Update superset/migrations/versions/2ed890b36b94_rm_time_range_endpoints_from_qc.py Co-authored-by: Beto Dealmeida <roberto@dealmeida.net> Co-authored-by: Yongjie Zhao <yongjie.zhao@gmail.com> Co-authored-by: Beto Dealmeida <roberto@dealmeida.net> (cherry picked from commit 129063d)

sadpandajoe · 2022-04-06T18:16:20Z

🏷️ preset:2022.13

…9423) * template for remove time_range_endpoints from query context * Update superset/migrations/versions/2ed890b36b94_rm_time_range_endpoints_from_qc.py Co-authored-by: Yongjie Zhao <yongjie.zhao@gmail.com> * Update superset/migrations/versions/2ed890b36b94_rm_time_range_endpoints_from_qc.py Co-authored-by: Yongjie Zhao <yongjie.zhao@gmail.com> * fix reference * fix column reference * fix * Update superset/migrations/versions/2ed890b36b94_rm_time_range_endpoints_from_qc.py Co-authored-by: Beto Dealmeida <roberto@dealmeida.net> Co-authored-by: Yongjie Zhao <yongjie.zhao@gmail.com> Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

template for remove time_range_endpoints from query context

581d90e

hughhhh requested a review from a team as a code owner March 30, 2022 01:28

pull-request-size bot added the size/M label Mar 30, 2022

zhaoyongjie reviewed Mar 30, 2022

View reviewed changes

superset/migrations/versions/2ed890b36b94_rm_time_range_endpoints_from_qc.py Outdated Show resolved Hide resolved

superset/migrations/versions/2ed890b36b94_rm_time_range_endpoints_from_qc.py Outdated Show resolved Hide resolved

Update superset/migrations/versions/2ed890b36b94_rm_time_range_endpoi…

d7f9d55

…nts_from_qc.py Co-authored-by: Yongjie Zhao <yongjie.zhao@gmail.com>

hughhhh changed the title ~~fix(wip): Removetime_range_endpoints from query context object~~ fix: Removetime_range_endpoints from query context object Mar 30, 2022

Update superset/migrations/versions/2ed890b36b94_rm_time_range_endpoi…

c3955b2

…nts_from_qc.py Co-authored-by: Yongjie Zhao <yongjie.zhao@gmail.com>

hughhhh force-pushed the rm-time-range branch from 75a761f to c3955b2 Compare March 30, 2022 05:33

fix reference

8bcacde

eschutho approved these changes Mar 30, 2022

View reviewed changes

eschutho requested a review from john-bodley March 30, 2022 17:21

fix column reference

98c73af

fix

5e22f58

betodealmeida approved these changes Mar 30, 2022

View reviewed changes

superset/migrations/versions/2ed890b36b94_rm_time_range_endpoints_from_qc.py Outdated Show resolved Hide resolved

hughhhh and others added 2 commits March 30, 2022 12:17

Update superset/migrations/versions/2ed890b36b94_rm_time_range_endpoi…

a2159fe

…nts_from_qc.py Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

Merge branch 'master' of https://github.com/apache/superset into rm-t…

238dceb

…ime-range

hughhhh merged commit 129063d into master Mar 31, 2022

superset-github-bot bot added the preset:2022.11 label Mar 31, 2022

villebro added the lts-v1 label Mar 31, 2022

ktmud mentioned this pull request Apr 1, 2022

perf: speed up db migration for deprecating time_range_endpoints #19495

Merged

9 tasks

villebro removed the lts-v1 label Apr 4, 2022

superset-github-bot bot added the preset:2022.13 label Apr 6, 2022

craig-rueda deleted the rm-time-range branch April 14, 2022 21:42

hughhhh mentioned this pull request Apr 14, 2022

fix: Removetime_range_endpoints from query context object pt 2 #19728

Merged

9 tasks

mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 2.0.0 labels Mar 13, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: Remove`time_range_endpoints` from query context object #19423

fix: Remove`time_range_endpoints` from query context object #19423

hughhhh commented Mar 30, 2022 •

edited

Loading

codecov bot commented Mar 30, 2022 •

edited

Loading

zhaoyongjie left a comment

github-actions bot commented Mar 30, 2022

eschutho left a comment

eschutho commented Mar 30, 2022

eschutho commented Mar 30, 2022

yousoph commented Mar 30, 2022

betodealmeida left a comment

sadpandajoe commented Mar 31, 2022

sadpandajoe commented Apr 6, 2022

fix: Removetime_range_endpoints from query context object #19423

fix: Removetime_range_endpoints from query context object #19423

Conversation

hughhhh commented Mar 30, 2022 • edited Loading

SUMMARY

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

codecov bot commented Mar 30, 2022 • edited Loading

Codecov Report

zhaoyongjie left a comment

Choose a reason for hiding this comment

github-actions bot commented Mar 30, 2022

eschutho left a comment

Choose a reason for hiding this comment

eschutho commented Mar 30, 2022

eschutho commented Mar 30, 2022

yousoph commented Mar 30, 2022

betodealmeida left a comment

Choose a reason for hiding this comment

sadpandajoe commented Mar 31, 2022

sadpandajoe commented Apr 6, 2022

fix: Remove`time_range_endpoints` from query context object #19423

fix: Remove`time_range_endpoints` from query context object #19423

hughhhh commented Mar 30, 2022 •

edited

Loading

codecov bot commented Mar 30, 2022 •

edited

Loading