Document xcom clearing behaviour on task retries #18917
Description
Apache Airflow version
2.2.0 (latest released)
Operating System
Debian Buster (in Docker)
Versions of Apache Airflow Providers
n/a
Deployment
Astronomer
Deployment details
Dockerfile:
FROM quay.io/astronomer/ap-airflow-dev:2.2.0-buster-onbuild-44762
Launched via astro dev start
DAG
def fail_then_succeed(**kwargs):
val = kwargs["ti"].xcom_pull(key="foo")
if not val:
kwargs["ti"].xcom_push(key="foo", value="bar")
raise Exception("fail")
with DAG(
dag_id="fail_then_succeed",
start_date=days_ago(1),
schedule_interval=None,
) as dag:
PythonOperator(
task_id="succeed_second_try",
python_callable=fail_then_succeed,
retries=2,
retry_delay=timedelta(seconds=15)
)What happened
The pushed value appears in XCOM, but even on successive retries xcom_pull returned None
What you expected to happen
On the second try, the XCOM value pushed by the first try is available, so the task succeeds after spending some time in "up for retry".
How to reproduce
Run the dag shown above, notice that it fails.
Anything else
No response
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct