🪛 Moving to SQLAlchemy 2.0 #540

tarsil · 2023-03-22T16:17:58Z

Added support for python 3.11
Added common and dialects packages to handle the new SQLAlchemy 2.0+

@aminalaee @Kludex this is a cleaner version of the previous PR with the changes passing the new SQLAlchemy 2.0 which is significantly different from the < 2.0

It is also good to mention that when I was doing this PR, I was reading some suggestions and the one from here #507 was good to add it as well. So I would like to thank @circlingthesun and @JGoutin for the suggestions

Edit: The asyncpg sometimes hangs during the tests. I forcibly close the connections during that test to make sure it doesn't get stuck

* Removed support for python 3.7 * Added common and dialects packages to handle the new SQLAlchemy 2.0+

* The connections were hanging sometimes during the tests.

tarsil · 2023-03-22T16:53:46Z

@aminalaee @Kludex maybe you could manually kill:

Please? This was the whole reason why the second commit. Quite a well known issue but that CI passed with flying colours, only these 2 are hanging

Kludex · 2023-03-22T18:24:37Z

Done

tarsil · 2023-03-22T18:26:33Z

Well, I hope this is ok. Quite nice and clean integration for SQLAlchemy 2.0

lucasguiss · 2023-03-22T18:51:06Z

I've forked the version of this PR and it's working fine in our apps, waiting for the release 😄

tarsil · 2023-03-22T19:02:18Z

@lucasguiss very happy to hear that :)

tarsil · 2023-03-22T22:03:47Z

@Kludex here the status is not "complete" because this PR also makes sure python 3.7 is no longer supported for many reasons but also in June the support will stop.

Kludex · 2023-03-22T22:05:28Z

@Kludex here the status is not "complete" because this PR also makes sure python 3.7 is no longer supported for many reasons but also in June the support will stop.

Why did you remove now?

tarsil · 2023-03-22T22:06:35Z

@Kludex here the status is not "complete" because this PR also makes sure python 3.7 is no longer supported for many reasons but also in June the support will stop.

Why did you remove now?

Well, I can always add and we remove after june, I would say. The initial tests were failing with some 3.7 issues but I will add it back. I think I solved them

Kludex · 2023-03-22T22:09:04Z

Can we remove all changes that are not related to the support of SQLAlchemy 2.0? You can create other PRs for them, but let's focus on a single thing here, please.

tarsil · 2023-03-22T22:13:58Z

Can we remove all changes that are not related to the support of SQLAlchemy 2.0? You can create other PRs for them, but let's focus on a single thing here, please.

@Kludex Done. It was mainly that, had initial configuration issues with python 3.7 but that was addressed.

The changes you see are the ones used for the move to SQLAlchemy 2.0+

Regarding other PRs. I can do that later once the time comes. This was already a good change that allow us to move forward.

CHANGELOG.md

.github/workflows/test-suite.yml

.github/workflows/publish.yml

README.md

mkdocs.yml

docs/index.md

* Revert changes automatically applied by IDE * Fix formatting of CI caused by IDE changes * Codebase cleanup for SQLAlchemy 2.0 * Fix EOL in mkdocs

tarsil · 2023-03-26T00:28:45Z

Hey @aminalaee . Any news about this? I know you are extremely busy

aminalaee · 2023-03-26T07:06:35Z

Hi @tarsil , I will take a look today or tomorrow. Sorry for the delay.

mathause · 2023-03-26T22:59:20Z

I think you'll have to update setup.py as well:

databases/setup.py

Line 50 in b6eba5f

install_requires=["sqlalchemy>=1.4.42,<1.5"],

tarsil · 2023-03-26T23:02:51Z

I think you'll have to update setup.py as well:

databases/setup.py

Line 50 in b6eba5f

install_requires=["sqlalchemy>=1.4.42,<1.5"],

I did update it as well before. Its there 👍🏼

mathause · 2023-03-27T08:59:18Z

pip complained when I installed the branch, but I see it now. Not sure why I missed it before - sorry for the noise.

tarsil · 2023-03-30T21:33:42Z

@aminalaee sorry to a bother but, any news?

tarsil · 2023-04-27T21:58:52Z

@Kludex this ia probably my fault. Since our last conversation unfortunately I didn't have enough time to address these comments. @rafalp in your code comment. I know SQLA stopped supporting that and that is the reason why I didn't change too much and I simple added the * as per normal python.

I promise I will try to address all the comments soon. Apologies

MAKMED1337 · 2023-04-30T15:40:55Z

Few days ago SQLAlchemy released 2.0.11 where they have changed Row class
Will it be supported ?

MAKMED1337 · 2023-04-30T22:05:13Z

databases/backends/common/records.py

+        return raw
+
+    def __iter__(self) -> typing.Iterator:
+        return iter(self._row.keys())


Because of this you can't unpack result anymore
Example:

class LastUsername(Base): __tablename__ = 'last_username' user_id = Column(BIGINT, primary_key=True) username = Column(TEXT, nullable=False) r = await db.fetch_one(select(LastUsername)) user_id, username = r assert user_id == 'user_id' #but it need to be at least int

stonebig · 2023-06-04T14:01:17Z

hum, any hope for this to be merged ?

Kludex · 2023-06-04T17:38:57Z

hum, any hope for this to be merged ?

If comments are addressed or answered, we can continue here

Feijo · 2023-06-04T23:09:16Z

Honestly I just decided to give up on this library altogether and use the new async features built-in into SA2.

stlomib · 2023-07-06T05:44:53Z

Any update?

POLIROID · 2023-08-08T16:48:42Z

Any update?

tarsil · 2023-08-08T17:26:45Z

Sorry, didn't have enough time to fully address this but I will soon

philipbjorge · 2023-09-20T23:20:20Z

@tarsil --
Thanks for putting this together --
We've put this in one of our projects and our test suite was 🟢.

We'll be kicking the tires more on this over the coming weeks and sharing any feedback or discoveries here.

(We did pin on SA 2.0.10 based on this comment)

ziglef · 2023-11-28T12:27:23Z

Any updates or roadmap for this PR?

philipbjorge · 2023-11-28T17:23:32Z

We just shipped this branch out to production and haven't noticed any problems yet.
We're a fairly low volume service, but overall feeling very good about this branch.
We're using this as a bridge while we port over to SA2.

adnam · 2023-12-05T11:03:44Z

I've tried out this branch by specifying the following in my requirements file:

databases @ git+https://github.com/tarsil/databases@feature/add_sqlalchemy_2

which resulted in pip-compile choosing sqlalchemy==2.0.23 - the latest version as of Nov 2 '23.

As previously mentioned here the Row class has changed in v2.0.11 resulting in an error like this when performing a query:

AttributeError: type object 'Row' has no attribute '_default_key_style'

I think it would be worth updating this PR to account for the new Row class before merging, otherwise projects using databases will be pinned to an sqlalchemy version from April '23.

@tarsil I'm not very familiar with SqlAlchemy internals but the solution would be something like this: adnam@96e15fd

pmdevita · 2024-02-20T20:44:16Z

@ansipunk solved this here tarsil#2 Just need to get this merged and we'll be back to 100% tests passing.
What else needs to be done for this PR to be merged?

Fix compatibility with SQLAlchemy>=2.0.11

tarsil · 2024-02-20T21:30:49Z

@ansipunk solved this here tarsil#2 Just need to get this merged and we'll be back to 100% tests passing. What else needs to be done for this PR to be merged?

It is merged and I do apologize for not receiving notifications and missing this.

Kludex · 2024-02-21T08:04:41Z

When @tarsil is happy (and this is ready to be merged...), I'll merge it and make a release. Ping me.

tarsil · 2024-02-21T09:02:25Z

@Kludex i do believe I just need to fix the conflicts and we should be good to go

Kludex · 2024-02-21T09:55:03Z

I've invited you to encode @tarsil . Ask me on PM if you have questions about workflows or anything.

Kludex

Merge when happy @tarsil 👍

tarsil · 2024-02-21T12:26:46Z

@Kludex , I just merged after fixing the minor conflicts (normal as it was an old branch) and passed 👍🏼

tarsil · 2024-02-21T12:27:29Z

I've invited you to encode @tarsil . Ask me on PM if you have questions about workflows or anything.

Will do but so far was just this simple merge conflict, I believe. Thank you for the invite.

tarsil added 2 commits March 22, 2023 16:15

🪛 Added support for SQLAlchemy 2.0

6e98885

* Removed support for python 3.7 * Added common and dialects packages to handle the new SQLAlchemy 2.0+

🪲 Fix specific asynpg oriented test

4cc05b9

* The connections were hanging sometimes during the tests.

tarsil changed the title ~~🪛 Added support for SQLAlchemy 2.0~~ 🪛 Moving to SQLAlchemy 2.0 Mar 22, 2023

tarsil mentioned this pull request Mar 22, 2023

Support for psycopg3? #535

Open

Kludex requested a review from aminalaee March 22, 2023 22:16

Kludex reviewed Mar 24, 2023

View reviewed changes

tarsil added 2 commits March 24, 2023 10:26

Add python 3.7 back to support

64dd1b3

* Revert changes automatically applied by IDE * Fix formatting of CI caused by IDE changes * Codebase cleanup for SQLAlchemy 2.0 * Fix EOL in mkdocs

Revert the previous changes to index.md

a703d54

tarsil force-pushed the feature/add_sqlalchemy_2 branch 2 times, most recently from 23223c9 to b271a54 Compare March 24, 2023 10:31

Update changelog and revert version

deedd13

tarsil force-pushed the feature/add_sqlalchemy_2 branch from b271a54 to deedd13 Compare March 24, 2023 10:32

This was referenced Mar 27, 2023

Support for SQL Server #543

Closed

Support for SQL Server #544

Closed

MAKMED1337 reviewed Apr 30, 2023

View reviewed changes

podsvirov mentioned this pull request Jun 4, 2023

python-databases: New package msys2/MINGW-packages#17437

Draft

2 tasks

pmdevita mentioned this pull request Aug 24, 2023

Upgrade to SQLAlchemy 2.0 collerek/ormar#1177

Open

Fix compatibility with SQLAlchemy>=2.0.11

b0ec5b3

Merge pull request #2 from ansipunk/feature/add_sqlalchemy_2

4665024

Fix compatibility with SQLAlchemy>=2.0.11

Kludex approved these changes Feb 21, 2024

View reviewed changes

tarsil added 2 commits February 21, 2024 12:04

Merge branch 'master' into feature/add_sqlalchemy_2

a2835c9

Fix linting and formatting

c5a5b19

tarsil merged commit 1e40ad1 into encode:master Feb 21, 2024
5 checks passed

tarsil deleted the feature/add_sqlalchemy_2 branch February 21, 2024 12:26

ansipunk mentioned this pull request Feb 22, 2024

SQLAlchemy 2.0 #530

Closed

🪛 Moving to SQLAlchemy 2.0 #540

🪛 Moving to SQLAlchemy 2.0 #540

Conversation

tarsil commented Mar 22, 2023 • edited

tarsil commented Mar 22, 2023 • edited

Kludex commented Mar 22, 2023

tarsil commented Mar 22, 2023

lucasguiss commented Mar 22, 2023

tarsil commented Mar 22, 2023

tarsil commented Mar 22, 2023

Kludex commented Mar 22, 2023

tarsil commented Mar 22, 2023

Kludex commented Mar 22, 2023

tarsil commented Mar 22, 2023 • edited

tarsil commented Mar 26, 2023

aminalaee commented Mar 26, 2023

mathause commented Mar 26, 2023

tarsil commented Mar 26, 2023

mathause commented Mar 27, 2023

tarsil commented Mar 30, 2023

tarsil commented Apr 27, 2023

MAKMED1337 commented Apr 30, 2023 • edited

MAKMED1337 Apr 30, 2023 • edited

Choose a reason for hiding this comment

stonebig commented Jun 4, 2023 • edited

Kludex commented Jun 4, 2023

Feijo commented Jun 4, 2023

stlomib commented Jul 6, 2023

POLIROID commented Aug 8, 2023

tarsil commented Aug 8, 2023

philipbjorge commented Sep 20, 2023

ziglef commented Nov 28, 2023

philipbjorge commented Nov 28, 2023

adnam commented Dec 5, 2023 • edited

pmdevita commented Feb 20, 2024

tarsil commented Feb 20, 2024

Kludex commented Feb 21, 2024 • edited

tarsil commented Feb 21, 2024

Kludex commented Feb 21, 2024

Kludex left a comment

Choose a reason for hiding this comment

tarsil commented Feb 21, 2024

tarsil commented Feb 21, 2024

tarsil commented Mar 22, 2023 •

edited

tarsil commented Mar 22, 2023 •

edited

tarsil commented Mar 22, 2023 •

edited

MAKMED1337 commented Apr 30, 2023 •

edited

MAKMED1337 Apr 30, 2023 •

edited

stonebig commented Jun 4, 2023 •

edited

adnam commented Dec 5, 2023 •

edited

Kludex commented Feb 21, 2024 •

edited