[v3-3-test] Fix task callbacks being skipped when TriggerDagRunOperator gets a 404 (#70719)

[github-actions]

• Fix task callbacks being skipped when TriggerDagRunOperator gets a 404

run() maps a task's outcome through a flat chain of except clauses, and
several of those clauses do real work: they call the API server, or serialize
user-supplied values. Python does not offer an exception raised inside an
except clause to that clause's siblings, so when one of them raised, the
exception escaped run() entirely -- skipping the retry decision in
_handle_current_task_failed() and every callback, listener and failure email
in finalize().

The reported path: triggering a Dag that does not exist returns 404, which
DagRunOperations.trigger re-raises (it only special-cases the 409
already-exists case). The supervisor turns it into an API_SERVER_ERROR
response and CommsDecoder._from_frame raises AirflowRuntimeError -- from
inside except DagRunTriggerException, a few lines above the clause that
already handles AirflowRuntimeError. It is not limited to that path:
_defer_task and _await_input_task run serde_serialize over user-supplied
kwargs, which raises TypeError for any value serde has no serializer for.

Split the function at the point where deciding the outcome ends and reporting
it begins. _run_task_and_map_outcome() keeps the chain verbatim and returns
the outcome; run() calls it, and its except now covers the handlers too.
The chain and the terminal-state finally block are untouched, so this is a
behaviour change rather than a reshuffle of existing lines.

_handle_handler_failure() keeps the chain's own classifications instead of
routing everything through the retry-count check, so AirflowFailException,
AirflowSensorTimeout and AirflowTaskTerminated still fail without retrying
when they surface from a handler. It catches Exception, not BaseException,
so KeyboardInterrupt still reaches main()'s exit-code-2 path -- the
supervisor's default termination signal is SIGINT, so swallowing it would turn
an operator-initiated kill into an ordinary retry. If the failure path itself
raises, it fails closed on a plain FAILED state rather than re-entering the
code that just failed and escaping again.

• Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI 175728472+Copilot@users.noreply.github.com

---

(cherry picked from commit f0c13dc)

Co-authored-by: Kaxil Naik kaxilnaik@gmail.com
Co-authored-by: Copilot Autofix powered by AI 175728472+Copilot@users.noreply.github.com