chore: bump celery and Flask #19168

dpgaspar · 2022-03-16T12:49:20Z

SUMMARY

This PR bumps major versions on the following main packages:

Celery from 4.X to 5.X:

Breaking changes: https://docs.celeryq.dev/en/stable/history/whatsnew-5.0.html#step-5-upgrade-to-celery-5-0

Relevant CLI breaking changes, and a final "warning" to migrate configuration keys to the new standard introduced on celery 4:

https://docs.celeryq.dev/en/stable/userguide/configuration.html#conf-old-settings-map

Note: Celery is at 5.2.2 and not 5.2.3 because of celery/celery#7194 this can cause problems on newer OSes forcing setup's to pin down setuptools

Flask-AppBuilder 3.X to 4.X

Drops python 3.6 support
Removed config key AUTH_STRICT_RESPONSE_CODES, it's always strict now
Removes Flask-OpenID dependency (extra dependency now)
Major version bump on the packages bellow

Flask from 1.X to 2.X

Breaking changes: https://flask.palletsprojects.com/en/2.0.x/changes/#version-2-0-0

Relevant:

JSON support no longer uses simplejson. To use another JSON module, override app.json_encoder and json_decoder. #3555
send_file() raises a ValueError when passed an io object in text mode. Previously, it would respond with 200 OK and an empty file. #3358

No relevant changes for our users, unless using a customised/extended Superset.

flask-jwt-extended 3.X to 4.X:

Breaking changes: https://flask-jwt-extended.readthedocs.io/en/stable/v4_upgrade_guide/

No relevant changes for our users, unless using a customised Security Manager that uses this extension JWT claims or using JWT_CSRF_HEADER_NAME configuration key.

pyJWT 1.X to 2.X:

Breaking changes: https://pyjwt.readthedocs.io/en/stable/changelog.html#v2-0-0

No relevant changes for our users, unless using a customised/extended Superset.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

Has associated issue:
Required feature flags:
Changes UI
Includes DB Migration (follow approval process in SIP-59)
- Migration is atomic, supports rollback & is backwards-compatible
- Confirm DB migration upgrade and downgrade tested
- Runtime estimates and downtime expectations provided
Introduces new feature or API
Removes existing feature or API

dpgaspar · 2022-03-16T16:03:51Z

superset/dashboards/api.py

            response = send_file(
                buf,
                mimetype="application/zip",
                as_attachment=True,
-                attachment_filename=filename,
+                download_name=filename,


https://flask.palletsprojects.com/en/2.0.x/api/#flask.send_file
attachment_filename is deprecated

john-bodley · 2022-03-16T17:24:50Z

requirements/base.txt

@@ -64,7 +74,7 @@ dnspython==2.1.0
    # via email-validator
 email-validator==1.1.3
    # via flask-appbuilder
-flask==1.1.4
+flask==2.0.3


Per the changelog the only breaking change which may impact us is pallets/flask#3555.

codecov · 2022-03-16T17:41:51Z

Codecov Report

Merging #19168 (add9a2b) into master (a08f83b) will increase coverage by 0.03%.
The diff coverage is 88.88%.

@@            Coverage Diff             @@
##           master   #19168      +/-   ##
==========================================
+ Coverage   66.64%   66.67%   +0.03%     
==========================================
  Files        1674     1675       +1     
  Lines       64614    64612       -2     
  Branches     6502     6501       -1     
==========================================
+ Hits        43061    43080      +19     
+ Misses      19868    19848      -20     
+ Partials     1685     1684       -1

Flag	Coverage Δ
hive	`52.70% <55.55%> (+0.03%)`	⬆️
mysql	`81.65% <88.88%> (-0.01%)`	⬇️
postgres	`81.70% <88.88%> (-0.01%)`	⬇️
presto	`52.55% <55.55%> (+0.03%)`	⬆️
python	`82.11% <88.88%> (-0.01%)`	⬇️
sqlite	`81.47% <88.88%> (+<0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
superset/cli/importexport.py	`80.00% <50.00%> (ø)`
superset/config.py	`91.69% <100.00%> (-0.03%)`	⬇️
superset/tasks/async_queries.py	`91.02% <100.00%> (ø)`
superset/utils/async_query_manager.py	`81.18% <100.00%> (-1.06%)`	⬇️
superset/views/core.py	`75.87% <100.00%> (ø)`
...et-frontend/src/dashboard/actions/dashboardInfo.ts	`52.17% <0.00%> (-9.37%)`	⬇️
superset-frontend/src/components/Select/utils.ts	`52.94% <0.00%> (-4.96%)`	⬇️
...erset-frontend/src/SqlLab/components/App/index.jsx	`52.38% <0.00%> (-3.18%)`	⬇️
...perset-frontend/src/dashboard/containers/Chart.jsx	`72.72% <0.00%> (-2.28%)`	⬇️
...src/BigNumber/BigNumberWithTrendline/buildQuery.ts	`9.09% <0.00%> (-2.03%)`	⬇️
... and 115 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a08f83b...add9a2b. Read the comment docs.

craig-rueda · 2022-03-17T18:09:29Z

Should we be merging this into a 2.0 branch or something? This is gonna break a lot of folks :)

craig-rueda

LGTM

# Conflicts: # UPDATING.md # superset/cli/celery.py

villebro

A minor leftover from a merge conflict, other than that LGTM and looking forward to getting this in!

UPDATING.md

villebro

LGTM

garyng · 2022-03-25T17:37:57Z

Hi! I noticed flask_openid was removed in https://github.com/apache/superset/pull/19168/files#diff-0940fec0afe21798f617cc4522db676f15769ae28a72992b8d19216ef7b1c3b5R100-R101, but it is still being use in https://github.com/dpgaspar/Flask-AppBuilder/blob/release/4.0.0/flask_appbuilder/security/manager.py#L259.
Is the removal intentional?

dpgaspar · 2022-03-25T17:53:19Z

@garyng yes it's intentional, that dependency is optional on FAB 4.0.0: https://flask-appbuilder.readthedocs.io/en/latest/breaking.html#version-4-0-0

garyng · 2022-03-25T18:14:18Z

@dpgaspar okay, thanks!

* chore: bump celery, Flask, flask-jwt-extended, pyJWT * fix pyJWT breaking change * fix pyJWT breaking change 2 * test * fixed test * fixed test * fixed test * revert since mypy won't pick the correct signature * lint 1 * fix test * fix test * docs and celery config migration * bump FAB to 4.0.0rc3, remove AUTH_STRICT_RESPONSE_CODES * update docs for new celery config keys * downgrade celery to 5.2.2 * ref FAB to final 4.0.0 release * remove conflict left over

dpgaspar added 3 commits March 15, 2022 22:21

chore: bump celery, Flask, flask-jwt-extended, pyJWT

417c738

fix pyJWT breaking change

59f78af

fix pyJWT breaking change 2

a5dc103

superset-github-bot bot added the preset-io label Mar 16, 2022

pull-request-size bot added the size/L label Mar 16, 2022

dpgaspar changed the title ~~chore: bump celery and Flask~~ [WIP] chore: bump celery and Flask Mar 16, 2022

dpgaspar changed the title ~~[WIP] chore: bump celery and Flask~~ chore: bump celery and Flask Mar 16, 2022

dpgaspar changed the title ~~chore: bump celery and Flask~~ [WIP] chore: bump celery and Flask Mar 16, 2022

dpgaspar marked this pull request as ready for review March 16, 2022 13:03

test

59036e0

etr2460 added risk:breaking-change Issues or PRs that will introduce breaking changes v2.0 labels Mar 16, 2022

dpgaspar added 2 commits March 16, 2022 15:59

fixed test

57183e2

fixed test

a6c6aa9

dpgaspar commented Mar 16, 2022

View reviewed changes

john-bodley reviewed Mar 16, 2022

View reviewed changes

dpgaspar added 8 commits March 16, 2022 18:08

fixed test

95194ac

revert since mypy won't pick the correct signature

7a1459c

lint 1

d63e737

fix test

445cc5c

fix test

a2a46ef

docs and celery config migration

6d02e88

bump FAB to 4.0.0rc3, remove AUTH_STRICT_RESPONSE_CODES

1288403

update docs for new celery config keys

1fd4040

dpgaspar changed the title ~~[WIP] chore: bump celery and Flask~~ chore: bump celery and Flask Mar 17, 2022

dpgaspar requested review from john-bodley, villebro and craig-rueda March 17, 2022 17:59

craig-rueda approved these changes Mar 17, 2022

View reviewed changes

dpgaspar added 5 commits March 18, 2022 11:14

downgrade celery to 5.2.2

bac916f

Merge branch 'master' into chore/celery-5

d8cbcf2

Merge branch 'master' into chore/celery-5

5babab1

ref FAB to final 4.0.0 release

b9807ca

Merge branch 'master' into chore/celery-5

28c6f4c

# Conflicts: # UPDATING.md # superset/cli/celery.py

villebro requested changes Mar 23, 2022

View reviewed changes

UPDATING.md Outdated Show resolved Hide resolved

dpgaspar added 2 commits March 23, 2022 15:47

Merge branch 'master' into chore/celery-5

181eb41

remove conflict left over

add9a2b

villebro approved these changes Mar 23, 2022

View reviewed changes

villebro closed this Mar 24, 2022

villebro reopened this Mar 24, 2022

dpgaspar merged commit f37fc1a into apache:master Mar 24, 2022

dpgaspar deleted the chore/celery-5 branch March 24, 2022 09:16

dpgaspar mentioned this pull request May 10, 2022

Bump Celery to 5.2.2 #19238

Closed

dpgaspar mentioned this pull request May 25, 2022

Upgrade FAB to 4+ apache/airflow#23899

Closed

1 task

michael-s-molina removed the v2.0 label Jun 16, 2022

mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 2.0.0 labels Mar 13, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore: bump celery and Flask #19168

chore: bump celery and Flask #19168

dpgaspar commented Mar 16, 2022 •

edited

dpgaspar Mar 16, 2022

john-bodley Mar 16, 2022

codecov bot commented Mar 16, 2022 •

edited

craig-rueda commented Mar 17, 2022

craig-rueda left a comment

villebro left a comment

villebro left a comment

garyng commented Mar 25, 2022

dpgaspar commented Mar 25, 2022

garyng commented Mar 25, 2022

chore: bump celery and Flask #19168

chore: bump celery and Flask #19168

Conversation

dpgaspar commented Mar 16, 2022 • edited

SUMMARY

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

dpgaspar Mar 16, 2022

Choose a reason for hiding this comment

john-bodley Mar 16, 2022

Choose a reason for hiding this comment

codecov bot commented Mar 16, 2022 • edited

Codecov Report

craig-rueda commented Mar 17, 2022

craig-rueda left a comment

Choose a reason for hiding this comment

villebro left a comment

Choose a reason for hiding this comment

villebro left a comment

Choose a reason for hiding this comment

garyng commented Mar 25, 2022

dpgaspar commented Mar 25, 2022

garyng commented Mar 25, 2022

dpgaspar commented Mar 16, 2022 •

edited

codecov bot commented Mar 16, 2022 •

edited