Upgrade to Pydantic v2 #35551
Upgrade to Pydantic v2 #35551
Conversation
|
Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst)
|
There was a problem hiding this comment.
NOTE! We only want to remove it when we know some important packages that are used by Airflow have already moved to Pydantic v2 . See this comment #34081 (comment) where we had to bring Pydantic 1 back after removing it (becasue Great Expections had not supported Pydantic v2)
@sinwoobang : Can you please take on the task on reviewing Great Expectations and possibly finding out the state of Pydantic v2 among important data engineering libraries? I am pretty sure someone somewhere did such a research - and maybe keeps the list updated - or maybe it can be asked in Pydantic discussion list.
As a follow-up after apache#35511 it turned out that we have THREE tests which are named Test_BranchDecoratedOperator - because they were copied originally from the "Python" one - there is a Python, ExternalPython and PythonVirtualenv tests :). The apache#35551 only added timeout in one of them - and in the least problematic one at that (Python). This PR also adds it in the other tests and renames the tests classes to distinguish between the different tests (to more easily see which one failed).
As a follow-up after #35511 it turned out that we have THREE tests which are named Test_BranchDecoratedOperator - because they were copied originally from the "Python" one - there is a Python, ExternalPython and PythonVirtualenv tests :). The #35551 only added timeout in one of them - and in the least problematic one at that (Python). This PR also adds it in the other tests and renames the tests classes to distinguish between the different tests (to more easily see which one failed).
|
@potiuk Thank you for the review. I looked around Great Expectations as per your request. Pydantic V2 has been supported on Great Expectations since 8th Sep according to this commit great-expectations/great_expectations@091fc1d, whereas the PR - bringing back Pydantic V1 compatibility - was created on 5th Sep. Facts
|
As a follow-up after apache#35511 it turned out that we have THREE tests which are named Test_BranchDecoratedOperator - because they were copied originally from the "Python" one - there is a Python, ExternalPython and PythonVirtualenv tests :). The apache#35551 only added timeout in one of them - and in the least problematic one at that (Python). This PR also adds it in the other tests and renames the tests classes to distinguish between the different tests (to more easily see which one failed).
|
I think it's about to time to resurrect that one. We are nearing 2.8.0 releease and I think it's good idea to move to Pydanticv2 only. Will you work on it @sinwoobang ? |
|
@potiuk I would be glad to do it. I will let you know after I take a look around the code again. Would you have any cautions or things I need to know? |
|
Nope. You've already checked great expectations and 5 months passed and Pydantic is already at 2.5 - there is no particular reason to stick to v1 any more. |
|
But let me know If I can help in any way. |
sinwoobang
requested review from
kaxil,
ashb and
bolkedebruin
as code owners
sinwoobang
changed the title
|
Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions. |
|
Looks good :) |
ephraimbuddy
added
the
type:misc/internal
* Replace deprecated Config with ConfigDict * Drop Pydantic v1 compatibility as bumping it to 2.3.0
* Replace deprecated Config with ConfigDict * Drop Pydantic v1 compatibility as bumping it to 2.3.0
[Airflow's upgraded to Pydantic 2.0 (!!)](apache/airflow#35551), so this removes the restriction on Cosmos to ensure it's compatible.
[Airflow's upgraded to Pydantic 2.0 (!!)](apache/airflow#35551), so this removes the restriction on Cosmos to ensure it's compatible.
This PR removes deprecated features from Pydantic v1 in preparation for the upcoming major release. The focus is on eliminating elements that are no longer supported in v2 or later, as outlined in the migration guide.
ConfigDictinstead ofConfig^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in newsfragments.