Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Airflow >= 2.2.0 execution date change is failing TaskInstance get_task_instances method and possibly others #21656

Closed
2 tasks done
TidesMind opened this issue Feb 17, 2022 · 1 comment · Fixed by #21705
Closed
2 tasks done
Labels
area:core kind:bug This is a clearly a bug

Comments

@TidesMind
Copy link

TidesMind commented Feb 17, 2022

Apache Airflow version

2.2.3 (latest released)

What happened

This is my first time reporting or posting on this forum. Please let me know if I need to provide any more information. Thanks for looking at this!

I have a Python Operator that uses the BaseOperator get_task_instances method and during the execution of this method, I encounter the following error:
Screen Shot 2022-02-17 at 2 28 48 PM

This error is from doing an upgrade from airflow 1.10.15 -> 2.2.3.

I am using SQLAlchemy version 1.2.24 but I also tried with version 1.2.23 and encountered the same error. However, I do not think this is a sqlAlchemy issue.

The issue seems to have been introduced with Airflow 2.2.0 (pr: https://github.com/apache/airflow/pull/17719/files), where the TaskInstance.execution_date changed from being a column to this association_proxy. I do not have deep knowledge of SQLAlchemny so I am not sure why this change was made, but it results in it the error I'm getting.

2.2 .0 +
Screen Shot 2022-02-17 at 2 41 00 PM

1.10.15
Screen Shot 2022-02-17 at 2 56 15 PM

if you follow the stack trace you will get to this chunk of code that leads to the error because the association_proxy has a 'clause_element' attr, but the attr raises the exception in the error when called.

Screen Shot 2022-02-17 at 2 43 51 PM

What you expected to happen

No response

How to reproduce

No response

Operating System

Linux from the official airflow helm chart docker image python version 3.7

Versions of Apache Airflow Providers

apache-airflow-providers-amazon 2.4.0
apache-airflow-providers-celery 2.1.0
apache-airflow-providers-cncf-kubernetes 2.2.0
apache-airflow-providers-databricks 2.2.0
apache-airflow-providers-docker 2.3.0
apache-airflow-providers-elasticsearch 2.1.0
apache-airflow-providers-ftp 2.0.1
apache-airflow-providers-google 6.2.0
apache-airflow-providers-grpc 2.0.1
apache-airflow-providers-hashicorp 2.1.1
apache-airflow-providers-http 2.0.1
apache-airflow-providers-imap 2.0.1
apache-airflow-providers-microsoft-azure 3.4.0
apache-airflow-providers-mysql 2.1.1
apache-airflow-providers-odbc 2.0.1
apache-airflow-providers-postgres 2.4.0
apache-airflow-providers-redis 2.0.1
apache-airflow-providers-sendgrid 2.0.1
apache-airflow-providers-sftp 2.3.0
apache-airflow-providers-slack 4.1.0
apache-airflow-providers-sqlite 2.0.1
apache-airflow-providers-ssh 2.3.0

Deployment

Official Apache Airflow Helm Chart

Deployment details

The only extra dependency I am using is awscli==1.20.65. I have changed very little with the deployment besides a few environments variables and some pod annotations.

Anything else

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@TidesMind TidesMind added area:core kind:bug This is a clearly a bug labels Feb 17, 2022
@boring-cyborg
Copy link

boring-cyborg bot commented Feb 17, 2022

Thanks for opening your first issue here! Be sure to follow the issue template!

@TidesMind TidesMind changed the title Airflow >= 2.1.0 execution date change is failing TaskInstance get_task_instances method and possibly others Airflow >= 2.2.0 execution date change is failing TaskInstance get_task_instances method and possibly others Feb 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:core kind:bug This is a clearly a bug
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant