Support SQLAlchemy 2.0.x - #938

Merged

Flix6x merged 50 commits into

mainfrom

feature/update-sqlalchemy-2.0

Feb 15, 2024

Ahmad-Wahid commented Dec 19, 2023 •

edited

Loading

Contributor

Description

This PR updates SQLAlchemy dependency from version 1.4 to 2.0. The update underwent thorough testing to ensure compatibility, and all the tests passed successfully.

How to test

Run make install-for-dev to install the required packages. Do the testing using the UI and CLI.

Ahmad-Wahid added 9 commits

December 16, 2023 17:56


          import declarative object from orm

8a90135


          use db.session.get instead of query.get

2289efb


          refactor users to use session.get()

68adac5


          refactor assets and view_utils to use session.get and session.execute

8314f1c


          refactor generic_assets to use session.get and session.execute

401cce9


          refactor data-show to use session.get and session.execute

859092d


          remove commented in code

da190ee


          refactor account to use session.get()

c3423c2


          adopt sqlalchemy 2 features in data_add

8da4eef

Ahmad-Wahid self-assigned this

Ahmad-Wahid added the dependency-hygiene label

Ahmad-Wahid added 4 commits

December 19, 2023 16:41


          refactore cli scripts to support sqlalchemy 2.0

c6b0ea5


          update v3_0/assets

5a0483e

Signed-off-by: Ahmad Wahid <ahmedwahid16101@gmail.com>


          update api to use sqlalchemy 2.0

3a996d1

Signed-off-by: Ahmad Wahid <ahmedwahid16101@gmail.com>


          update models to support sqlalchemy 2.0

bac86d7

Ahmad-Wahid linked an issue

that may be closed by this pull request

Support SQLAlchemy 2 #607

Closed

Ahmad-Wahid and others added 14 commits

December 20, 2023 17:58


          update migration scripts

568f1f5


          update api


          update ui to use sqlalchemy 2.0

f2366e0


          update cli queries to sqlalchemy 2.0 version

20bb740

Signed-off-by: Ahmad Wahid <ahmedwahid16101@gmail.com>


          migrate queries to sqlalchemy 2.0

a3b98c7

Signed-off-by: Ahmad Wahid <ahmedwahid16101@gmail.com>


          Merge branch 'main' into feature/update-sqlalchemy-2.0

b06b5de


          update requirements

6083cbc


          Merge remote-tracking branch 'origin/feature/update-sqlalchemy-2.0' i…

57afa7d

…nto feature/update-sqlalchemy-2.0

# Conflicts:
#	requirements/3.10/app.txt


          use session.execute instead of engine.execute

83b450f


          Merge remote-tracking branch 'origin/main' into feature/update-sqlalc…

501c56b

…hemy-2.0


          first check for a table if exists then data

b1ec01d

Signed-off-by: Ahmad Wahid <ahmedwahid16101@gmail.com>


          add zone to session before returning it

0e97f52

Signed-off-by: Ahmad Wahid <ahmedwahid16101@gmail.com>


          update last value datatime query

07b866b

Signed-off-by: Ahmad Wahid <ahmedwahid16101@gmail.com>


          update data_gen.py queries

16e9774

Signed-off-by: Ahmad Wahid <ahmedwahid16101@gmail.com>

Flix6x added 2 commits

February 6, 2024 21:14


          fix: more type annotations for Python 3.8

14be2ff

Signed-off-by: F.N. Claessen <felix@seita.nl>


          Merge remote-tracking branch 'origin/main' into feature/update-sqlalc…

f8f1c8c

…hemy-2.0

nhoening commented Feb 6, 2024

Member

We can run the script @Nischay-Pro wrote to update the dependencies for all supported versions.


          update timerange query

ffb0fb5

Signed-off-by: Ahmad Wahid <ahmedwahid16101@gmail.com>

Flix6x mentioned this pull request

Chore: update dependencies for #938 #973

Merged

Flix6x added this to the 0.19.0 milestone

Flix6x reviewed

View reviewed changes

flexmeasures/data/models/time_series.py Outdated


          fix: typo

50d0abf

Signed-off-by: Ahmad Wahid <ahmedwahid16101@gmail.com>

Flix6x reviewed

View reviewed changes

flexmeasures/cli/data_delete.py Outdated


          revert nan and unchanged beleifs delete queries

a194d04

Signed-off-by: Ahmad Wahid <ahmedwahid16101@gmail.com>

Ahmad-Wahid requested a review from Flix6x

February 12, 2024 13:17

Flix6x requested changes

View reviewed changes

Flix6x left a comment •

edited

Loading

Member

Please don't be scared by the high number of comments I made. Most of them are just duplicate suggestions to reduce the verbosity of the queries where possible (I was just keeping track of occurrences).

The code would probably also become cleaner if we refactor a bit, but probably that would be better as a follow-up PR.

I did find a couple of potentially serious issues with some of the delete statements (those that filter by name rather than by ID). Here we should take a critical look, but I believe the fixes should be easy.

All in all I think you did a great job on this PR. This looked like hard work. Some of these queries weren't trivial (e.g. in the migration files).

flexmeasures/ui/crud/users.py Outdated

flexmeasures/api/dev/sensors.py Outdated

flexmeasures/api/dev/tests/conftest.py Outdated

flexmeasures/api/v3_0/assets.py Outdated

flexmeasures/api/v3_0/assets.py Outdated

flexmeasures/data/services/scheduling.py Outdated

flexmeasures/data/services/users.py Outdated

flexmeasures/data/tests/test_annotations.py

flexmeasures/data/tests/test_annotations.py

flexmeasures/data/tests/test_scheduling_repeated_jobs.py Outdated


          refactor changes

bf81165

Signed-off-by: Ahmad Wahid <ahmedwahid16101@gmail.com>

Ahmad-Wahid commented Feb 14, 2024

Contributor Author

@Flix6x Some tests are failing, please fix them if you can. I have tried but couldn't understand them.


          refactor epex_da sensor query

cf4b3da

Signed-off-by: Ahmad Wahid <ahmedwahid16101@gmail.com>

Ahmad-Wahid requested a review from Flix6x

February 14, 2024 18:13

Flix6x reviewed

View reviewed changes

flexmeasures/data/services/users.py Outdated

Ahmad-Wahid and others added 3 commits

February 15, 2024 09:08


          fix queries

1af5ff8

Signed-off-by: Ahmad Wahid <ahmedwahid16101@gmail.com>


          docs: changelog entry

ac04574

Signed-off-by: F.N. Claessen <felix@seita.nl>


          Merge remote-tracking branch 'origin/main' into feature/update-sqlalc…

2ee2ac8

…hemy-2.0

Flix6x added Data and removed Still Needs Changelog Entry labels

Flix6x and others added 4 commits

February 15, 2024 11:16


          fix: deletion of role account association for a given account

Signed-off-by: F.N. Claessen <felix@seita.nl>


          style: black

465c8f7

Signed-off-by: F.N. Claessen <felix@seita.nl>


          fix: test missed adding an object to session before flushing to assig…

c647c44

…n IDs

Signed-off-by: F.N. Claessen <felix@seita.nl>


          Chore: update dependencies for #938 (#973)

b9becd0

* style: fix typos

Signed-off-by: F.N. Claessen <felix@seita.nl>

* docs: add readme comment on updating requirements

Signed-off-by: F.N. Claessen <felix@seita.nl>

* chore: run ci/update-packages.sh

Signed-off-by: F.N. Claessen <felix@seita.nl>

---------

Signed-off-by: F.N. Claessen <felix@seita.nl>

Flix6x approved these changes

View reviewed changes

Flix6x merged commit 071a4de into main

Flix6x deleted the feature/update-sqlalchemy-2.0 branch

February 15, 2024 10:52

nhoening mentioned this pull request

flexmeasures delete prognoses fails with TypeError #1092

Closed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Data dependency-hygiene