Skip to content

There's no way to pass params to dag.test() #35930

@getaaron

Description

@getaaron

Description

I'm writing a new dag which takes params and trying out calling dag.test() in my unit test. The basic dag could look like:

emotions = ['elated', 'ennui']

@task
def some_task(emotion: str, date: str):
    url = f"http://example.com/do_something/{date}?emotion={emotion}"
    response = requests.post(url)
    if response.status_code != 200:
        raise AirflowException(f"Request failed with status {response.status_code}")

@dag(default_args=default_args,
     schedule=None,
     params={"emotion": Param(None, enum=emotions),
             "date": Param(None, type="string", format="date")})
def process_my_emotion():
    some_task('{{ params.emotion }}', '{{ params.date }}')

In my test, I want to call dag.test() with an emotion and date and assert the correct URL is formed. I tried this:

@patch("requests.post")
@freeze_time("2024-01-01")
def test_process_my_emotion(mock_post, manual_dag):
    mock_post.return_value = Mock(status_code=200)
    manual_dag.test(execution_date=datetime(2024, 1, 1),
                    params={'emotion': 'ennui', 'date': '2024-01-01'})

    mock_post.assert_any_call(
        f"http://example.com/do_something/2024-01-01?emotion=ennui"
    )

This fails with:

TypeError: test() got an unexpected keyword argument 'params'

Looking at

airflow/airflow/models/dag.py

Lines 2769 to 2776 in f6962a9

def test(
self,
execution_date: datetime | None = None,
run_conf: dict[str, Any] | None = None,
conn_file_path: str | None = None,
variable_file_path: str | None = None,
session: Session = NEW_SESSION,
) -> DagRun:

I don't see a way to pass params in.

If I'm correct, the feature I'm requesting is a way to pass params in. If I'm incorrect, I'm requesting an improvement to the docs.

Use case/motivation

I would like to test dags that use params by calling dag.test()

Related issues

I didn't find any.

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions