Skip to content
This repository has been archived by the owner on Aug 4, 2023. It is now read-only.

Commit

Permalink
Log last query_params hit before AirflowTaskTimeout (#1058)
Browse files Browse the repository at this point in the history
* Log query_params before raising AirflowExceptions

* Handle datetimes in queryparams
  • Loading branch information
stacimc committed Mar 28, 2023
1 parent 40461cb commit 5c2d707
Showing 1 changed file with 7 additions and 2 deletions.
Expand Up @@ -237,8 +237,13 @@ def ingest_records(self, **kwargs) -> None:
should_continue = False

except AirflowException as error:
# AirflowExceptions should not be caught, as execution should not
# continue when the task is being stopped by Airflow.
# AirflowExceptions should not be caught or reraised as IngestionErrors,
# as execution should not continue when the task is being stopped by
# Airflow. However, we should still log the last query_params to be
# hit before the error was raised.
logger.info(
f"Last query_params used: {json.dumps(query_params, default=str)}"
)

# If errors have already been caught during processing, raise them
# as well.
Expand Down

0 comments on commit 5c2d707

Please sign in to comment.