Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Restore Python 3.12 support for Databricks #38207

Merged
merged 1 commit into from
Mar 16, 2024

Conversation

potiuk
Copy link
Member

@potiuk potiuk commented Mar 16, 2024

We excluded Python 3.12 from Databricks provider, because it was failing our Python 3.12 tests intermittently (but often enough to make a difference). It turned out that this was caused by running the tests with coverage enabled and PEP 669 implementation in Python 3.12 impacting intermittently performance of tests run with coverage. However seems that experimenetal PEP 669 support implemented in coverage 7.4.0 is nicely handling the performance issues and after #38194 we shoudl be able to enable Python 3.12 for Databricks without impacting our tests.

Related: databricks/databricks-sql-python#369


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

@eladkal
Copy link
Contributor

eladkal commented Mar 16, 2024

Are we proceeding with disable of --cov or with COVERAGE_CORE=sysmon ?
nedbat/coveragepy#1665 (comment)

@eladkal
Copy link
Contributor

eladkal commented Mar 16, 2024

Are we proceeding with disable of --cov or with COVERAGE_CORE=sysmon ? nedbat/coveragepy#1665 (comment)

ah right
https://github.com/apache/airflow/pull/38194/files

Copy link
Contributor

@Taragolis Taragolis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW, maybe also bump coverage in devel-test extras to 7.4+ ?

"coverage>=7.2",

@potiuk
Copy link
Member Author

potiuk commented Mar 16, 2024

BTW, maybe also bump coverage in devel-test extras to 7.4+ ?

Good idea.

We excluded Python 3.12 from Databricks provider, because it was
failing our Python 3.12 tests intermittently (but often enough to
make a difference). It turned out that this was caused by running
the tests with coverage enabled and PEP 669 implementation in
Python 3.12 impacting intermittently performance of tests run
with coverage. However seems that experimenetal PEP 669 support
implemented in coverage 7.4.0 is nicely handling the performance
issues and after apache#38194 we shoudl be able to enable Python 3.12 for
Databricks without impacting our tests.

Related: databricks/databricks-sql-python#369
@potiuk potiuk force-pushed the restore-databricks-python-3-12-support branch from bf61df5 to 1ad0864 Compare March 16, 2024 10:55
@potiuk potiuk merged commit aa75fbb into apache:main Mar 16, 2024
85 of 86 checks passed
@potiuk potiuk deleted the restore-databricks-python-3-12-support branch March 16, 2024 11:45
@ephraimbuddy ephraimbuddy added the changelog:skip Changes that should be skipped from the changelog (CI, tests, etc..) label Mar 18, 2024
utkarsharma2 pushed a commit to astronomer/airflow that referenced this pull request Apr 22, 2024
We excluded Python 3.12 from Databricks provider, because it was
failing our Python 3.12 tests intermittently (but often enough to
make a difference). It turned out that this was caused by running
the tests with coverage enabled and PEP 669 implementation in
Python 3.12 impacting intermittently performance of tests run
with coverage. However seems that experimenetal PEP 669 support
implemented in coverage 7.4.0 is nicely handling the performance
issues and after apache#38194 we shoudl be able to enable Python 3.12 for
Databricks without impacting our tests.

Related: databricks/databricks-sql-python#369
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:providers changelog:skip Changes that should be skipped from the changelog (CI, tests, etc..) provider:databricks
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants