Add python 3.10 support #2885

PrettyWood · 2021-06-06T22:23:32Z

Change Summary

Python 3.10.0b2 is out. Maintainers are invited to try it out. I reckon adding it in the CI doesn't hurt!

ℹ️ I invite you to read this commit by commit, which should make the review trivial

Related issue number

closes #2607
closes #2786
closes #2787

Checklist

Unit tests for the changes exist
Tests pass on CI and coverage remains at 100%
Documentation reflects the changes where applicable
changes/<pull request or issue id>-<github username>.md file added describing change
(see changes/README.md for details)
My PR is ready to review, please add a comment including the phrase "please review" to assign reviewers

Error with 3.10 > DeprecationWarning: accessing one member from another is not supported

error with python 3.10 > DeprecationWarning: The distutils package is deprecated and slated > for removal in Python 3.12. > Use setuptools or check PEP 632 for potential alternatives

`|` operator has origin `types.Union` (and not `typing.Union`)

change from basic `__init__` to `test_hashable_required.<locals>.MyDataclass.__init__()` (with `__qualname__`)

With python 3.10, calling it results in > DeprecationWarning: There is no current event loop

For now it can not be compiled. Let's just skip the check on linux for now instead of tuning the CI pipeline

PrettyWood · 2021-06-07T09:14:54Z

Hey @samuelcolvin
Since 3.10 was possibly a game breaker for pydantic, I reckon it's a good idea to check everything works as expected as soon as possible.
Quoting the python release itself

We strongly encourage maintainers of third-party Python projects to test with 3.10 during the beta phase

Reading this PR commit by commit should be trivial. The main thing is that I extracted BaseConfig and Extra in a dedicated module to make checks on enum values clean (only import this module instead of the whole main one)
Please review and tell me what you think 🙇

samuelcolvin

This looks great, pretty much ready to merge.

Can you just confirm what's changed in config.py vs. main.py?

pydantic/config.py

samuelcolvin · 2021-07-19T13:23:12Z

Thanks so much.

Sorry for the slow responses. New baby, new job, no sleep, no open source for a little while.

tirkarthi · 2021-07-19T13:43:45Z

Just to add enum repr related changes were reverted in Python 3.10 after discussion : https://mail.python.org/archives/list/python-dev@python.org/thread/ZMC67QA2JVQJSWSFWRS6IM6ZX4EK277G/#LSTMFAPSPD3BGZ4D6HQFODXZVB3PLYKF

PrettyWood · 2021-07-19T13:47:33Z

Congrats @samuelcolvin for the baby I didnt' know!
And thanks @tirkarthi for the info.
We (you, I, or anyone else) can just open another PR with 3.10.0b4 and revert b1c8d9e.
I reckon the rest should not be impacted 🤔

Update: done in #3000

Add python 3.10 support (pydantic#2885)

yifeikong · 2021-08-27T06:37:23Z

pydantic/typing.py

+    import types
+
+    def is_union(tp: Type[Any]) -> bool:
+        return tp is Union or tp is types.Union


Is this a typo? shouldn't types.Union be types.UnionType?

https://docs.python.org/3.10/library/types.html#types.UnionType

They changed between last beta and first RC.
#3085

Thanks, hope this could be merged soon.

I will soon. But it will be part of v1.9 most likely
Adding python 3.10 was mostly for CI and to ensure it will still work once out.
But I don't expect people to use 3.10 for production already since it is not stable yet

yifeikong · 2021-08-27T06:38:10Z

tests/test_main.py

+
+@pytest.mark.skipif(sys.version_info < (3, 10), reason='need 3.10 version')
+def test_new_union_origin():
+    """On 3.10+, origin of `int | str` is `types.Union`, not `typing.Union`"""


rmzr7 · 2021-10-06T19:50:34Z

@samuelcolvin when will this be released?

samuelcolvin · 2021-10-06T21:20:42Z

Hopefully soon.

I have a lot on at the moment.

m10d · 2022-04-17T23:25:46Z

commenting here looking for advice from those familiar with the py3.10 changes. We recently had to upgrade from python3.8->3.10. However, we encountered a pydantic bug (#3896) that blocks us from upgrading pydantic to 1.9.0 - which I understand per the title here is the only version to officially support python3.10. remaining on pydantic==1.7.4 is the only workaround I am aware of. A comment from samuel on that bug indicates its not trivial / won't likely be resolved very soon.

We use pydantic for all serialization & deserialization for json input to and output from our api (and offline for schema & tool generation). While our automated testing IS somewhat basic, both our unit and api-level full system tests are passing with python3.10 + pydantic==1.7.4.

my question is - any advice on common edge cases we should be aware of or add testing for, to (attempt to) verify this pair of versions will not bite us? I've read the linked issues and I don't believe they affect our specific use of pydantic, but 3.10 has some significant changes from 3.8. Any advise or landmines would be greatly appreciated.

samuelcolvin · 2022-04-18T07:31:50Z

Best way to speedup a resolution and the release of 1.9.1 world be if you could submit a bugfix for #3896.

Otherwise I'll look at it soon.

m10d · 2022-04-18T21:23:55Z

could submit a bugfix for #3896.

You may have struck some fear into my heart with your comment on #3896 :)

"bug deep in schema.py I doubt there's an easy fix I'm afraid." -SC

If you take an initial look, and it's something suited for a new contributor, I might be able to dig in. A deep dive is not in the cards for me at this movement, unfortunately. Thank you for your replies, pydantic has long been a very helpful library.

PrettyWood added 10 commits June 6, 2021 23:26

refactor: extra BaseConfig and Extra in dedicated config module

6a243ae

refactor: clean useless #noqa: F401

e8fc3f1

refactor: clean useless #noqa: F811

98d7096

refactor: replace enum check

863be10

Error with 3.10 > DeprecationWarning: accessing one member from another is not supported

refactor: avoid using distutils directly

3877599

error with python 3.10 > DeprecationWarning: The distutils package is deprecated and slated > for removal in Python 3.12. > Use setuptools or check PEP 632 for potential alternatives

fix: __annotations__ always exists

944fb7e

fix: origin of typing.Hashable is not None

534774a

ci: add run with 3.10.0b2

d754b31

docs: add 3.10

f45a02f

feat: support | union operator properly

c9e0604

`|` operator has origin `types.Union` (and not `typing.Union`)

PrettyWood force-pushed the support-3.10 branch from fce4eb4 to 67a0623 Compare June 7, 2021 08:46

PrettyWood added 4 commits June 7, 2021 10:51

fix: enum repr is different with 3.10+

b1c8d9e

fix: error message changed a bit

f422277

change from basic `__init__` to `test_hashable_required.<locals>.MyDataclass.__init__()` (with `__qualname__`)

fix: always exists and is not inherited anymore

05c51bd

fix: avoid calling asyncio.get_event_loop directly

75174c2

With python 3.10, calling it results in > DeprecationWarning: There is no current event loop

PrettyWood force-pushed the support-3.10 branch from f6efe07 to bbb25df Compare June 7, 2021 08:51

fix(ci): do not run 3.10 on linux for now

8b79ceb

For now it can not be compiled. Let's just skip the check on linux for now instead of tuning the CI pipeline

PrettyWood force-pushed the support-3.10 branch 2 times, most recently from efad94a to 358ded3 Compare June 7, 2021 09:00

PrettyWood added 2 commits June 7, 2021 11:04

fix(ci): ignore DeprecationWarning raised by mypy on windows

2820a92

docs: add change file

5bf6f7c

PrettyWood force-pushed the support-3.10 branch from 358ded3 to 5bf6f7c Compare June 7, 2021 09:04

PrettyWood marked this pull request as ready for review June 7, 2021 09:15

github-actions bot added the ready for review label Jun 7, 2021

github-actions bot assigned PrettyWood and samuelcolvin Jun 7, 2021

hmvp approved these changes Jul 19, 2021

View reviewed changes

samuelcolvin reviewed Jul 19, 2021

View reviewed changes

pydantic/config.py Show resolved Hide resolved

samuelcolvin merged commit 4a54f39 into pydantic:master Jul 19, 2021

PrettyWood deleted the support-3.10 branch July 19, 2021 13:47

sthagen added a commit to sthagen/pydantic-pydantic that referenced this pull request Jul 19, 2021

Merge pull request #164 from samuelcolvin/master

45843b9

Add python 3.10 support (pydantic#2885)

PrettyWood mentioned this pull request Jul 19, 2021

chore(ci): update python 3.10 version #3000

Merged

graingert mentioned this pull request Jul 20, 2021

test on 3.10 tiangolo/fastapi#3558

Closed

yifeikong reviewed Aug 27, 2021

View reviewed changes

koxudaxi mentioned this pull request Nov 28, 2021

GitHub Actions: Add Python 3.10 to the testing koxudaxi/datamodel-code-generator#605

Merged

samuelcolvin mentioned this pull request Dec 31, 2021

Test error with Python3.10 #3604

Closed

Halkcyon mentioned this pull request Jan 25, 2022

Pydantic fails with Python 3.10 new UnionType #3300

Closed

3 tasks

m10d mentioned this pull request Apr 17, 2022

Pydantic changes json schema representation when extra keys are provided #3896

Closed

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add python 3.10 support #2885

Add python 3.10 support #2885

PrettyWood commented Jun 6, 2021 •

edited

PrettyWood commented Jun 7, 2021

samuelcolvin left a comment

samuelcolvin commented Jul 19, 2021

tirkarthi commented Jul 19, 2021

PrettyWood commented Jul 19, 2021 •

edited

yifeikong Aug 27, 2021

PrettyWood Aug 27, 2021

yifeikong Aug 27, 2021

PrettyWood Aug 27, 2021 •

edited

yifeikong Aug 27, 2021

rmzr7 commented Oct 6, 2021

samuelcolvin commented Oct 6, 2021

m10d commented Apr 17, 2022

samuelcolvin commented Apr 18, 2022

m10d commented Apr 18, 2022

Add python 3.10 support #2885

Add python 3.10 support #2885

Conversation

PrettyWood commented Jun 6, 2021 • edited

Change Summary

Related issue number

Checklist

PrettyWood commented Jun 7, 2021

samuelcolvin left a comment

Choose a reason for hiding this comment

samuelcolvin commented Jul 19, 2021

tirkarthi commented Jul 19, 2021

PrettyWood commented Jul 19, 2021 • edited

yifeikong Aug 27, 2021

Choose a reason for hiding this comment

PrettyWood Aug 27, 2021

Choose a reason for hiding this comment

yifeikong Aug 27, 2021

Choose a reason for hiding this comment

PrettyWood Aug 27, 2021 • edited

Choose a reason for hiding this comment

yifeikong Aug 27, 2021

Choose a reason for hiding this comment

rmzr7 commented Oct 6, 2021

samuelcolvin commented Oct 6, 2021

m10d commented Apr 17, 2022

samuelcolvin commented Apr 18, 2022

m10d commented Apr 18, 2022

PrettyWood commented Jun 6, 2021 •

edited

PrettyWood commented Jul 19, 2021 •

edited

PrettyWood Aug 27, 2021 •

edited