Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Version Incompatibility in 1.10.14 resulting from snowflake workaround #13093

Closed
r-richmond opened this issue Dec 15, 2020 · 2 comments
Closed
Assignees
Labels
kind:bug This is a clearly a bug

Comments

@r-richmond
Copy link
Contributor

Apache Airflow version: 1.10.14
Environment: Python 3.8

  • Install tools: pipenv

What happened:

So I appear to have stumbled across a version incompatibility for airflow-1.10.14
apache-airflow[aws] was modified with this pr to restrict boto to boto3~=1.10,<1.11 with the comment for snowflake
When attempting to install the latest amazon-provider backport apache-airflow-backport-providers-amazon==2020.11.23->-r I’m faced with an incompatible version in resolved dependancies as the backport requires boto3<2.0.0,>=1.12.0

attempting to install aws extra and amazon-provide backport results in a version incompatibility error.

There are incompatible versions in the resolved dependencies:
  boto3<1.11,~=1.10 (from apache-airflow[aws]==1.10.14->-r /var/folders/dw/8s4dltks7bg5ws8kht1wfbqw0000gp/T/pipenvtfsk4ltbrequirements/pipenv-ukucfwxw-constraints.txt (line 2))
  boto3<2.0.0,>=1.12.0 (from apache-airflow-backport-providers-amazon==2020.11.23->-r /var/folders/dw/8s4dltks7bg5ws8kht1wfbqw0000gp/T/pipenvtfsk4ltbrequirements/pipenv-ukucfwxw-constraints.txt (line 3))

What you expected to happen:

Airflow has compatible versions.

How to reproduce it:

1. Create Pipfile with the following contents
[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]

[packages]
apache-airflow = {extras = ["aws"],version = "*"}
# https://github.com/apache/airflow/blob/master/README.md#backport-packages
apache-airflow-backport-providers-amazon = "*"

[requires]
python_version = "3.8"
2. run `pipenv lock`

Anything else we need to know:

Suggested solution: move all snowflake version modifications out of related airflow extras and directly into the snowflake extra.
Slack Thread: https://apache-airflow.slack.com/archives/CCQ7EGB1P/p1608057981159400

@r-richmond r-richmond added the kind:bug This is a clearly a bug label Dec 15, 2020
@boring-cyborg
Copy link

boring-cyborg bot commented Dec 15, 2020

Thanks for opening your first issue here! Be sure to follow the issue template!

@potiuk potiuk added this to the Airflow 1.10.15 milestone Dec 15, 2020
@potiuk potiuk self-assigned this Dec 15, 2020
potiuk added a commit to PolideaInternal/airflow that referenced this issue Dec 23, 2020
potiuk added a commit that referenced this issue Dec 23, 2020
@potiuk
Copy link
Member

potiuk commented Dec 23, 2020

Closed by #13286

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind:bug This is a clearly a bug
Projects
None yet
Development

No branches or pull requests

2 participants