Operator template fields via callable serialization causes unstable DAG serialization

[m8719-github]

Apache Airflow version

3.1.5

If "Other Airflow 3 version" selected, which one?

No response

What happened?

Operator template fields may be provided via callable of the form: def fn(context, jinja_env)
These template field callables are serialized via default str which includes current address of the callable: <function fn at 0x7592eb0cb380>
The address of the loaded callable changes between iterations of the dag-processor DAG refresh loop, and since DAG versioning depends on stable serialization, this leads to DAG version explosion.

What you think should happen instead?

Operator template fields serialization is treated separately here and serialize_template_field does not handle callable case.
Instead template field callables should be serialized via already existing logic here.

How to reproduce

Create a simple DAG with a single task with template fields, configure any of the template fields to be provided via callable: def fn(context, jinja_env)
Watch DAG versions increment indefinitely.

Operating System

Ubuntu 24.04.3 LTS

Versions of Apache Airflow Providers

apache-airflow-providers-amazon==9.18.1
apache-airflow-providers-cncf-kubernetes==10.11.1
apache-airflow-providers-common-compat==1.10.1
apache-airflow-providers-common-io==1.7.0
apache-airflow-providers-common-sql==1.30.1
apache-airflow-providers-databricks==7.8.1
apache-airflow-providers-datadog==3.10.1
apache-airflow-providers-ftp==3.14.0
apache-airflow-providers-http==5.6.1
apache-airflow-providers-mysql==6.4.0
apache-airflow-providers-openlineage==2.9.1
apache-airflow-providers-pagerduty==5.2.1
apache-airflow-providers-sftp==5.5.1
apache-airflow-providers-slack==9.6.1
apache-airflow-providers-smtp==2.4.1
apache-airflow-providers-ssh==4.2.1
apache-airflow-providers-standard==1.10.1
apache-airflow-providers-trino==6.4.1

Deployment

Virtualenv installation

Deployment details

No response

Anything else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[m8719-github]

@potiuk Could you please take a look at this issue as well as the PR. This is somewhat related to #59465
Much appreciated!