Type annotation for `@task` errors when called with parentheses/arguments

[benbuckman]

Apache Airflow version

2.6.2

What happened

👋 If I define a task with @task (via from airflow.decorators import task) then mypy (type checker) is happy. If I add add parentheses to add arguments, like @task() or @task(queue='foo'), then mypy complains,

Untyped decorator makes function [name] untyped.

I see in airflow src, there are example DAGs with @task() (using parentheses). So I also wonder how this works internally.

Thank you!

What you think should happen instead

Mypy should understand any of,

@task
def foo():
    ...

@task()
def foo():
    ...

@task(queue='secondary')
def foo():
    ...

How to reproduce

• Python 3.10.8
• Airflow 2.6.2
• Mypy 1.0.0 (as defined by constraints file)
  • (Note: upgrading mypy to the latest 1.4.1 results in the same behavior.)

Place somewhere in <dags_folder>:

from datetime import datetime
from airflow.decorators import dag, task


@task
def task_without_parens_is_fine() -> None:
    ...


@task()
# ^ type error on this line: "Untyped decorator makes function "demo_type_error" untyped"
def demo_type_error() -> None:
    ...


@dag(schedule=None, start_date=datetime(2021, 1, 1), catchup=False)
def demo_dag():
    demo_type_error()


demo_dag()

Run mypy --strict . in the parent or application-root folder.
(edit: I initially omitted --strict from this snippet, but it's required to repro.)

Operating System

MacOS Ventura 13.4

Versions of Apache Airflow Providers

apache-airflow-providers-common-sql==1.5.1
apache-airflow-providers-ftp==3.4.1
apache-airflow-providers-google==10.1.1
apache-airflow-providers-http==4.4.1
apache-airflow-providers-imap==3.2.1
apache-airflow-providers-postgres==5.5.0
apache-airflow-providers-sendgrid==3.2.0
apache-airflow-providers-slack==7.3.0
apache-airflow-providers-sqlite==3.4.1

Deployment

Official Apache Airflow Helm Chart

Deployment details

The deployment is not relevant to this issue, as this can be reproduced locally in a mypy-enabled IDE or in a terminal running mypy <dir>.

Anything else

I tried to copy/use the dev/mypy/plugins/decorators.py mypy plugin in my application, but this did not help. (And anyway that file is not exported in the public apache-airflow package.)

Airflow Slack thread about this: https://apache-airflow.slack.com/archives/CCPRP7943/p1688571561287409

Tzu-ping Chung:
Sounds like our type annotation to @task is inaccurate. File an issue with a repro and I can try to fix that.

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[uranusjr]

Hmm, I can’t reproduce with a clean environment:

$ docker run -iqt --rm apache/airflow:2.6.2 bash

airflow@d1644eaceff2:/opt/airflow$ pip install mypy -q
airflow@d1644eaceff2:/opt/airflow$ printf 'from airflow.decorators import dag, task

@task
def task_without_parens_is_fine() -> None:
    ...

@task()
def demo_type_error() -> None:
    ...
' > /opt/airflow/dags/t.py
airflow@d1644eaceff2:/opt/airflow$ cd dags
airflow@d1644eaceff2:/opt/airflow$ cat t.py
from airflow.decorators import dag, task

@task
def task_without_parens_is_fine() -> None:
    ...

@task()
def demo_type_error() -> None:
    ...
airflow@d1644eaceff2:/opt/airflow$ mypy t.py
Success: no issues found in 1 source file

[benbuckman]

mypy needs --strict mode to see this:

host:

$ docker run -it --rm apache/airflow:2.6.2-python3.10 bash

container:

$ pip install 'mypy==1.0.0'

$ printf 'from datetime import datetime
from airflow.decorators import dag, task

@task()
def demo_type_error() -> None:
    ...

@dag(schedule=None, start_date=datetime(2021, 1, 1), catchup=False)
def demo_dag() -> None:
    demo_type_error()

demo_dag()' > /opt/airflow/dags/t.py

$ cd dags

# (this takes a while to build the cache the first time)
$ mypy --strict .

I see:

t.py:7: error: Untyped decorator makes function "demo_type_error" untyped  [misc]
Found 1 error in 1 file (checked 1 source file)

[kelechi2020]

@potiuk I want to work on this.

[potiuk]

Feel free. Not sure if this is absolutely necessary but if it can be improved, this is cool.

[github-actions]

This issue has been automatically marked as stale because it has been open for 365 days without any activity. There has been several Airflow releases since last activity on this issue. Kindly asking to recheck the report against latest Airflow version and let us know if the issue is reproducible. The issue will be closed in next 30 days if no further activity occurs from the issue author.