Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Upgrade Flask ecosystem dependencies #838

Merged
merged 43 commits into from
Sep 22, 2023
Merged

Upgrade Flask ecosystem dependencies #838

merged 43 commits into from
Sep 22, 2023

Conversation

Flix6x
Copy link
Contributor

@Flix6x Flix6x commented Aug 30, 2023 •
edited
Loading

Description

Unpin several dependencies within the Flask ecosystem:

  • Flask
  • Flask-Classful (the culprit blocker)
  • Flask-Login
  • Flask-Marshmallow
  • Flask-Security-Too
  • Flask-SQLAlchemy
  • Werkzeug

Also update rq-dashboard and modernize how we integrate this dashboard into our Flask app.

Look & Feel

image

How to test

make test

Further Improvements

  • Still need to find out how to show all registered queues in the Tasks dashboard. Update: this worked out of the box already. Apparently, I was initially using a redis server with just one queue set up.

Related Items

Closes #595.

  • I agree to contribute to the project under Apache 2 License.
  • To the best of my knowledge, the proposed patch is not based on code under GPL or other license that is incompatible with FlexMeasures

@Flix6x
feat (dependencies): update dependencies previously blocked by Flask-…
8b56703 
…Classful

Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
feat: fix integration of rq-dashboard within Flask app
cc0fd98 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
chore: update app.txt for Python 3.9
f4e336a 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x Flix6x self-assigned this Aug 30, 2023
nhoening
nhoening requested changes Aug 30, 2023
View reviewed changes
Copy link
Contributor

@nhoening nhoening left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You have not upgraded dependencies after removing upper limits. I suspect that tests might not succeed then, e.g. with the problems in flask-login.

pip-installing would fail then.

requirements/app.in Outdated Show resolved Hide resolved
requirements/3.9/app.txt Outdated Show resolved Hide resolved
@Flix6x
Merge branch 'main' into upgrade-flask-ecosystem-dependencies
ba15ca3
@Flix6x
chore: update requirements for Python 3.9
c261a8a 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
fix (test): task lastrun
c13ccbb 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@nhoening
Copy link
Contributor

It seems we can also (since today) move to the real rq-dashboard's new version 0.6.6: Parallels/rq-dashboard#417 (comment)

This is the first release on Pypi in more than 3 years for them!

@Flix6x
Revert "feat (dependencies): update dependencies previously blocked b…
15eb5a7 
…y Flask-Classful"

This reverts commit 8b56703.
@Flix6x
chore: include Flask patch update in pin, and update requirements for…
e9fc659 
… Python 3.9

Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
chore: so far so good
8e9a5c6 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
chore: upgrade Flask-Login and fix API tests
db1e133 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
chore: upgrade deps for Python 3.9 again
c6845c2 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
style: flake8
8d0172e 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
chore: clean up pins and inline documentation
7f901a8 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
chore: clean up another pin, and update Flask-Marshmallow
6aab58f 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
fix: updated rq-dashboard uses different underscore.js
a8307ee 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
Copy link
Contributor Author

Flix6x commented Sep 3, 2023 •
edited
Loading

I think what happened is that Flask-Login==0.6.2 started keeping test users logged in using a session cookie, which looked like it had precedence over the token in the Authorization header. I went through the API tests and started using a new fixture that logs in a user for the scope of the test (i.e. logging out before moving on to the next test).

I managed to upgrade to Flask==2.2.5. This is as far as I can take it before being blocked by #754.

The PR is still in draft awaiting a new release of Flask-Classful.

@Flix6x
fix (UI): missing space
0ca6f5e 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
chore (deps): move pin to point to original repo
98294a3 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
chore: upgrade deps
1dd25b3 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
fix: favicon 500 error
46035e4 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
chore: upgrade deps, incl. Flask-Classful release
4743b91 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
docs: changelog entry
29e6d7f 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x Flix6x marked this pull request as ready for review September 8, 2023 09:04
@Flix6x Flix6x requested a review from nhoening September 8, 2023 09:04
@Flix6x Flix6x added this to the 0.16.0 milestone Sep 8, 2023
@nhoening
upgrade dependencies for Python3.10
7083f4b 
Signed-off-by: Nicolas Höning <nicolas@seita.nl>
@GustaafL
chore: upgrade dependencies py3.11 (#839)
8a21233 
Signed-off-by: GustaafL <guus@seita.nl>
nhoening
nhoening previously requested changes Sep 11, 2023
View reviewed changes
Copy link
Contributor

@nhoening nhoening left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great that you could resolve bottlenecks here!

I have a few smaller issues, and of course we miss the 3.8 pins. I'll talk to @Nischay-Pro , maybe his current work can be used here (dev script to update dependencies per Python version)

flexmeasures/api/conftest.py Outdated Show resolved Hide resolved
flexmeasures/api/v3_0/tests/test_sensors_api.py Outdated Show resolved Hide resolved
flexmeasures/api/v3_0/tests/test_api_v3_0_users.py Outdated Show resolved Hide resolved
flexmeasures/api/conftest.py Show resolved Hide resolved
flexmeasures/api/v3_0/tests/test_assets_api.py Outdated Show resolved Hide resolved
flexmeasures/ui/templates/base.html Outdated Show resolved Hide resolved
flexmeasures/ui/templates/base.html Outdated Show resolved Hide resolved
@Flix6x
refactor: use variable
964c0a6 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
refactor: remove obsolete util function
98bbc59 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
refactor: rename test
8e7499f 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
refactor: simplify test
21f3ac9 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
fix: logging out does not require auth token anymore; also test loggi…
613e509 
…ng in

Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
refactor: simplify check
439f54c 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
feat: derive tasks listing from queues registered in app
dea3961 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
fix: remove redundant use of auth token
60c15fa 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
flake8
c4608cf 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x Flix6x mentioned this pull request Sep 12, 2023
Merged
2 tasks
Flix6x and others added 9 commits September 12, 2023 09:53
@Flix6x
chore: use ci/update-packages.sh to upgrade dependencies for all supp…
36906b5 
…orted Python versions

Signed-off-by: F.N. Claessen <felix@seita.nl>
@nhoening
do not push application context, should be handled by Flask
e236d8a 
Signed-off-by: Nicolas Höning <nicolas@seita.nl>
@nhoening
remove outdated and not that helpful comment
c2949c3 
Signed-off-by: Nicolas Höning <nicolas@seita.nl>
@nhoening
add auth_required() decorator to all API endpoints explicitly
b1ebcb2 
Signed-off-by: Nicolas Höning <nicolas@seita.nl>
@nhoening
Add a test explicitly testing authenticating via auth token - using a…
65fdba4 
… patch in flask-security for now (this works without patch outside of testing).

Signed-off-by: Nicolas Höning <nicolas@seita.nl>
@nhoening
Move permission check for /GET and /POST sensor data into endpoint de…
6423c75 
…finition (was: schema), just like others. Became possible with recent enhancements of permission_required_for_context capabilities

Signed-off-by: Nicolas Höning <nicolas@seita.nl>
@nhoening
Merge branch 'main' into upgrade-flask-ecosystem-dependencies
206e219 
Signed-off-by: Nicolas Höning <nicolas@seita.nl>
@Flix6x
refactor: set auth decorator once for all endpoints in a FlaskView
b6ff4af 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
style: typos
fa953fc 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x Flix6x merged commit db9282f into main Sep 22, 2023
9 checks passed
@Flix6x Flix6x deleted the upgrade-flask-ecosystem-dependencies branch September 22, 2023 12:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@nhoening nhoening nhoening left review comments

@GustaafL GustaafL GustaafL left review comments

Assignees

@Flix6x Flix6x

Labels
dependency-hygiene
Projects
None yet
Milestone
0.16.0
Development

Successfully merging this pull request may close these issues.

Upgrade or replace Flask-Classful
3 participants
@Flix6x @nhoening @GustaafL