Skip to content

Commit

Permalink
Set min pandas dependency to 1.2.5 for all providers and airflow (#36698
Browse files Browse the repository at this point in the history
)

We had some REALLY old minimum version of Pandas set for all our
pandas dependency - Pandas 0.17.1 has been released in 2015 (!)

Looking at the dependency tree - most of our dependencies had
> 1.2.5 set - which is more than reasonable limit as Pandas 1.2.5
had been released in June 2021 - so more than 2.5 years ago.

This limit bump further helps us to limit the pip backtracking
that starts happening in certain situations.

Extracted from: #36537
  • Loading branch information
potiuk committed Jan 9, 2024
1 parent 0d7bad2 commit ecb2c9f
Show file tree
Hide file tree
Showing 11 changed files with 17 additions and 17 deletions.
2 changes: 1 addition & 1 deletion airflow/providers/amazon/provider.yaml
Expand Up @@ -742,7 +742,7 @@ logging:
additional-extras:
- name: pandas
dependencies:
- pandas>=0.17.1
- pandas>=1.2.5
# There is conflict between boto3 and aiobotocore dependency botocore.
# TODO: We can remove it once boto3 and aiobotocore both have compatible botocore version or
# boto3 have native async support and we move away from aio aiobotocore
Expand Down
2 changes: 1 addition & 1 deletion airflow/providers/apache/hive/provider.yaml
Expand Up @@ -66,7 +66,7 @@ dependencies:
- apache-airflow>=2.6.0
- apache-airflow-providers-common-sql>=1.3.1
- hmsclient>=0.1.0
- pandas>=0.17.1
- pandas>=1.2.5
- pyhive[hive_pure_sasl]>=0.7.0
- thrift>=0.9.2

Expand Down
2 changes: 1 addition & 1 deletion airflow/providers/common/sql/provider.yaml
Expand Up @@ -54,7 +54,7 @@ dependencies:
additional-extras:
- name: pandas
dependencies:
- pandas>=0.17.1
- pandas>=1.2.5

integrations:
- integration-name: Common SQL
Expand Down
2 changes: 1 addition & 1 deletion airflow/providers/exasol/provider.yaml
Expand Up @@ -55,7 +55,7 @@ dependencies:
- apache-airflow>=2.6.0
- apache-airflow-providers-common-sql>=1.10.0
- pyexasol>=0.5.1
- pandas>=0.17.1
- pandas>=1.2.5

integrations:
- integration-name: Exasol
Expand Down
2 changes: 1 addition & 1 deletion airflow/providers/google/provider.yaml
Expand Up @@ -137,7 +137,7 @@ dependencies:
- json-merge-patch>=0.2
- looker-sdk>=22.2.0
- pandas-gbq
- pandas>=0.17.1
- pandas>=1.2.5
# A transient dependency of google-cloud-bigquery-datatransfer, but we
# further constrain it since older versions are buggy.
- proto-plus>=1.19.6
Expand Down
2 changes: 1 addition & 1 deletion airflow/providers/presto/provider.yaml
Expand Up @@ -57,7 +57,7 @@ dependencies:
- apache-airflow>=2.6.0
- apache-airflow-providers-common-sql>=1.3.1
- presto-python-client>=0.8.4
- pandas>=0.17.1
- pandas>=1.2.5

integrations:
- integration-name: Presto
Expand Down
2 changes: 1 addition & 1 deletion airflow/providers/salesforce/provider.yaml
Expand Up @@ -53,7 +53,7 @@ versions:
dependencies:
- apache-airflow>=2.6.0
- simple-salesforce>=1.0.0
- pandas>=0.17.1
- pandas>=1.2.5

integrations:
- integration-name: Salesforce
Expand Down
2 changes: 1 addition & 1 deletion airflow/providers/trino/provider.yaml
Expand Up @@ -57,7 +57,7 @@ versions:
dependencies:
- apache-airflow>=2.6.0
- apache-airflow-providers-common-sql>=1.3.1
- pandas>=0.17.1
- pandas>=1.2.5
- trino>=0.318.0

integrations:
Expand Down
2 changes: 1 addition & 1 deletion airflow/providers/weaviate/provider.yaml
Expand Up @@ -42,7 +42,7 @@ integrations:
dependencies:
- apache-airflow>=2.6.0
- weaviate-client>=3.24.2
- pandas>=0.17.1
- pandas>=1.2.5

hooks:
- integration-name: Weaviate
Expand Down
14 changes: 7 additions & 7 deletions generated/provider_dependencies.json
Expand Up @@ -125,7 +125,7 @@
"apache-airflow-providers-common-sql>=1.3.1",
"apache-airflow>=2.6.0",
"hmsclient>=0.1.0",
"pandas>=0.17.1",
"pandas>=1.2.5",
"pyhive[hive_pure_sasl]>=0.7.0",
"thrift>=0.9.2"
],
Expand Down Expand Up @@ -403,7 +403,7 @@
"deps": [
"apache-airflow-providers-common-sql>=1.10.0",
"apache-airflow>=2.6.0",
"pandas>=0.17.1",
"pandas>=1.2.5",
"pyexasol>=0.5.1"
],
"cross-providers-deps": [
Expand Down Expand Up @@ -509,7 +509,7 @@
"json-merge-patch>=0.2",
"looker-sdk>=22.2.0",
"pandas-gbq",
"pandas>=0.17.1",
"pandas>=1.2.5",
"proto-plus>=1.19.6",
"sqlalchemy-bigquery>=1.2.1",
"sqlalchemy-spanner>=1.6.2"
Expand Down Expand Up @@ -842,7 +842,7 @@
"deps": [
"apache-airflow-providers-common-sql>=1.3.1",
"apache-airflow>=2.6.0",
"pandas>=0.17.1",
"pandas>=1.2.5",
"presto-python-client>=0.8.4"
],
"cross-providers-deps": [
Expand All @@ -864,7 +864,7 @@
"salesforce": {
"deps": [
"apache-airflow>=2.6.0",
"pandas>=0.17.1",
"pandas>=1.2.5",
"simple-salesforce>=1.0.0"
],
"cross-providers-deps": [],
Expand Down Expand Up @@ -1007,7 +1007,7 @@
"deps": [
"apache-airflow-providers-common-sql>=1.3.1",
"apache-airflow>=2.6.0",
"pandas>=0.17.1",
"pandas>=1.2.5",
"trino>=0.318.0"
],
"cross-providers-deps": [
Expand All @@ -1033,7 +1033,7 @@
"weaviate": {
"deps": [
"apache-airflow>=2.6.0",
"pandas>=0.17.1",
"pandas>=1.2.5",
"weaviate-client>=3.24.2"
],
"cross-providers-deps": [],
Expand Down
2 changes: 1 addition & 1 deletion setup.py
Expand Up @@ -350,7 +350,7 @@ def write_version(filename: str = str(AIRFLOW_SOURCES_ROOT / "airflow" / "git_ve
leveldb = ["plyvel"]
otel = ["opentelemetry-exporter-prometheus"]
pandas = [
"pandas>=0.17.1",
"pandas>=1.2.5",
]
password = [
"bcrypt>=2.0.0",
Expand Down

0 comments on commit ecb2c9f

Please sign in to comment.