Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions providers/amazon/docs/operators/redshift/redshift_data.rst
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,13 @@ earlier Airflow versions the flag is a no-op and the operator always submits fre
exactly as before. If the task state store is unavailable at runtime, the operator logs that
crash recovery is disabled and behaves the same way.

Like the persisted state itself, the stored statement id isn't deleted automatically, that only
happens when someone runs ``airflow state-store clean``. If a task's ``retry_delay`` is longer
than ``[state_store] default_retention_days`` (30 days by default) and cleanup runs in between,
the statement id won't be there for the next retry, and the operator will submit the SQL fresh
instead of reconnecting. Avoid running cleanup on a schedule shorter than your longest
``retry_delay``.

To opt out and always submit fresh SQL on retry, set ``durable=False``:

.. code-block:: python
Expand Down
6 changes: 6 additions & 0 deletions providers/databricks/docs/operators/run_now.rst
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,12 @@ Airflow versions the flag is a no-op and the operator always triggers a fresh ru
exactly as before. If the task state store is unavailable at runtime, the operator logs that crash
recovery is disabled and behaves the same way.

Like the persisted state itself, the stored run id isn't deleted automatically, that only happens
when someone runs ``airflow state-store clean``. If a task's ``retry_delay`` is longer than
``[state_store] default_retention_days`` (30 days by default) and cleanup runs in between, the
run id won't be there for the next retry, and the operator will trigger a fresh run instead of
reconnecting. Avoid running cleanup on a schedule shorter than your longest ``retry_delay``.

To opt out and always trigger a fresh run on retry, set ``durable=False``:

.. code-block:: python
Expand Down
6 changes: 6 additions & 0 deletions providers/databricks/docs/operators/submit_run.rst
Original file line number Diff line number Diff line change
Expand Up @@ -184,6 +184,12 @@ earlier Airflow versions the flag is a no-op and the operator always submits a f
run on retry, exactly as before. If the task state store is unavailable at runtime, the
operator logs that crash recovery is disabled and behaves the same way.

Like the persisted state itself, the stored run id isn't deleted automatically, that only happens
when someone runs ``airflow state-store clean``. If a task's ``retry_delay`` is longer than
``[state_store] default_retention_days`` (30 days by default) and cleanup runs in between, the
run id won't be there for the next retry, and the operator will submit a fresh run instead of
reconnecting. Avoid running cleanup on a schedule shorter than your longest ``retry_delay``.

To opt out and always submit a fresh run on retry, set ``durable=False``:

.. code-block:: python
Expand Down
6 changes: 6 additions & 0 deletions providers/google/docs/operators/cloud/bigquery.rst
Original file line number Diff line number Diff line change
Expand Up @@ -408,6 +408,12 @@ exactly as before -- including the pre-existing ``reattach_states``/``Conflict``
unchanged. If the task state store is unavailable at runtime, the operator logs that crash
recovery is disabled and behaves the same way.

Like the persisted state itself, the stored job id isn't deleted automatically, that only happens
when someone runs ``airflow state-store clean``. If a task's ``retry_delay`` is longer than
``[state_store] default_retention_days`` (30 days by default) and cleanup runs in between, the
job id won't be there for the next retry, and the operator will submit a fresh job instead of
reconnecting. Avoid running cleanup on a schedule shorter than your longest ``retry_delay``.

To opt out and always submit a fresh job on retry, set ``durable=False``:

.. code-block:: python
Expand Down
7 changes: 7 additions & 0 deletions providers/snowflake/docs/operators/snowflake.rst
Original file line number Diff line number Diff line change
Expand Up @@ -182,6 +182,13 @@ earlier Airflow versions the flag is a no-op and the operator always submits fre
exactly as before. If the task state store is unavailable at runtime, the operator logs that crash
recovery is disabled and behaves the same way.

Like the persisted state itself, the stored statement handles aren't deleted automatically, that
only happens when someone runs ``airflow state-store clean``. If a task's ``retry_delay`` is
longer than ``[state_store] default_retention_days`` (30 days by default) and cleanup runs in
between, the handles won't be there for the next retry, and the operator will submit the SQL
fresh instead of reconnecting. Avoid running cleanup on a schedule shorter than your longest
``retry_delay``.

To opt out and always submit fresh SQL on retry, set ``durable=False``:

.. code-block:: python
Expand Down