Skip to content

Commit

Permalink
Exclude Python 3.12 for Databricks provider (apache#38070)
Browse files Browse the repository at this point in the history
Databricks is excluded for Python 3.12 because running
databricks-sql-python imports on Python 3.12 Cause extremely long import
times: databricks/databricks-sql-python#369
and until the problem is fixed, we exclude Python 3.12 for Databricks
provider
  • Loading branch information
potiuk authored and howardyoo committed Mar 31, 2024
1 parent 073725d commit 573e454
Show file tree
Hide file tree
Showing 3 changed files with 14 additions and 6 deletions.
6 changes: 6 additions & 0 deletions airflow/providers/databricks/provider.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,12 @@ versions:
- 1.0.1
- 1.0.0


# Databricks is excluded for Python 3.12 because running databricks-sql-python imports on Python 3.12
# Cause extremely long import times https://github.com/databricks/databricks-sql-python/issues/369
# and until the problem is fixed, we exclude Python 3.12 for Databricks provider
excluded-python-versions: ['3.12']

dependencies:
- apache-airflow>=2.6.0
- apache-airflow-providers-common-sql>=1.10.0
Expand Down
4 changes: 3 additions & 1 deletion generated/provider_dependencies.json
Original file line number Diff line number Diff line change
Expand Up @@ -377,7 +377,9 @@
"cross-providers-deps": [
"common.sql"
],
"excluded-python-versions": [],
"excluded-python-versions": [
"3.12"
],
"state": "ready"
},
"datadog": {
Expand Down
10 changes: 5 additions & 5 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -662,12 +662,12 @@ common-sql = [ # source: airflow/providers/common/sql/provider.yaml
"sqlparse>=0.4.2",
]
databricks = [ # source: airflow/providers/databricks/provider.yaml
"aiohttp>=3.9.2, <4",
"apache-airflow[common_sql]",
"databricks-sql-connector>=2.0.0, <3.0.0, !=2.9.0",
"requests>=2.27.0,<3",
"aiohttp>=3.9.2, <4;python_version != \"3.12\"",
"apache-airflow[common_sql];python_version != \"3.12\"",
"databricks-sql-connector>=2.0.0, <3.0.0, !=2.9.0;python_version != \"3.12\"",
"requests>=2.27.0,<3;python_version != \"3.12\"",
# Devel dependencies for the databricks provider
"deltalake>=0.12.0",
"deltalake>=0.12.0;python_version != \"3.12\"",
]
datadog = [ # source: airflow/providers/datadog/provider.yaml
"datadog>=0.14.0",
Expand Down

0 comments on commit 573e454

Please sign in to comment.