Skip to content

Commit

Permalink
Exclude Python 3.12 for Databricks provider
Browse files Browse the repository at this point in the history
Databricks is excluded for Python 3.12 because running
databricks-sql-python imports on Python 3.12 Cause extremely long import
times: databricks/databricks-sql-python#369
and until the problem is fixed, we exclude Python 3.12 for Databricks
provider
  • Loading branch information
potiuk committed Mar 12, 2024
1 parent 1d3010c commit 12901b9
Show file tree
Hide file tree
Showing 3 changed files with 14 additions and 6 deletions.
6 changes: 6 additions & 0 deletions airflow/providers/databricks/provider.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,12 @@ versions:
- 1.0.1
- 1.0.0


# Databricks is excluded for Python 3.12 because running databricks-sql-python imports on Python 3.12
# Cause extremely long import times https://github.com/databricks/databricks-sql-python/issues/369
# and until the problem is fixed, we exclude Python 3.12 for Databricks provider
excluded-python-versions: ['3.12']

dependencies:
- apache-airflow>=2.6.0
- apache-airflow-providers-common-sql>=1.10.0
Expand Down
4 changes: 3 additions & 1 deletion generated/provider_dependencies.json
Original file line number Diff line number Diff line change
Expand Up @@ -377,7 +377,9 @@
"cross-providers-deps": [
"common.sql"
],
"excluded-python-versions": [],
"excluded-python-versions": [
"3.12"
],
"state": "ready"
},
"datadog": {
Expand Down
10 changes: 5 additions & 5 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -662,12 +662,12 @@ common-sql = [ # source: airflow/providers/common/sql/provider.yaml
"sqlparse>=0.4.2",
]
databricks = [ # source: airflow/providers/databricks/provider.yaml
"aiohttp>=3.9.2, <4",
"apache-airflow[common_sql]",
"databricks-sql-connector>=2.0.0, <3.0.0, !=2.9.0",
"requests>=2.27.0,<3",
"aiohttp>=3.9.2, <4;python_version != \"3.12\"",
"apache-airflow[common_sql];python_version != \"3.12\"",
"databricks-sql-connector>=2.0.0, <3.0.0, !=2.9.0;python_version != \"3.12\"",
"requests>=2.27.0,<3;python_version != \"3.12\"",
# Devel dependencies for the databricks provider
"deltalake>=0.12.0",
"deltalake>=0.12.0;python_version != \"3.12\"",
]
datadog = [ # source: airflow/providers/datadog/provider.yaml
"datadog>=0.14.0",
Expand Down

0 comments on commit 12901b9

Please sign in to comment.