fix(providers/standard): add response_timeout to HITLOperator to prevent race with execution_timeout#63475
Conversation
abb5062 to
9bf0d68
Compare
|
I'm not convinced this is required. From worker to triggerer should'nt take that long and a short execution_time usually does not make sense |
|
Thanks for the review, @Lee-W. You're right that worker-to-triggerer is normally fast, and short
When notifiers are involved (e.g., Even without notifiers, the current design means users can't independently control "how long my task is allowed to run" vs "how long to wait for human input." These are separate knobs that shouldn't be coupled. The fix is backward-compatible (auto-migrates Happy to discuss alternative approaches if you see a simpler path, but I believe the semantic separation is the right fix here. |
…ent race with execution_timeout HITLOperator reused BaseOperator's execution_timeout for two conflicting purposes: (1) as a task execution guard that kills the task, and (2) as the human response wait window passed to HITLTrigger. When short, the BaseOperator timeout fires before defer() is reached, killing the task instead of deferring. Add a dedicated response_timeout parameter for the trigger wait window. For backward compatibility, if execution_timeout is set without response_timeout, it is migrated with a deprecation warning and execution_timeout is cleared to prevent the race condition. Closes apache#55866 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…timeout The example DAG was using execution_timeout which triggers the new deprecation warning during DAG import, causing DagBag import errors. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
9bf0d68 to
20ef120
Compare
|
The task is basically waiting for a response, so I think it's okay to use |
Summary
response_timeoutparameter toHITLOperatorthat controls how long theHITLTriggerwaits for a human responseBaseOperator.execution_timeout, which wraps the entireexecute()method and can kill the task beforedefer()is reachedexecution_timeoutis set withoutresponse_timeout, it is automatically migrated with a deprecation warning andexecution_timeoutis clearedRoot Cause
HITLOperatorreusedBaseOperator.execution_timeoutfor two conflicting purposes:execute()in a timeout that kills the taskHITLTriggeras the human response wait windowWhen
execution_timeoutis short (or notifiers are slow), the BaseOperator timeout fires beforeself.defer()is reached, killing the task withAirflowTaskTimeoutinstead of deferring.Test Plan
response_timeoutcorrectly passed to triggerexecution_timeoutbackward compatibility emits deprecation warning and migratesexecution_timeout > response_timeoutresponse_timeoutprek)Closes #55866
🤖 Generated with Claude Code
Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com