Skip to content

Fix flaky test timeout test_end_drains_result_queue_to_avoid_join_deadlock on Python 3.14#69525

Closed
FrankYang0529 wants to merge 1 commit into
apache:mainfrom
FrankYang0529:airflow-flaky-test_end_drains_result_queue_to_avoid_join_deadlock
Closed

Fix flaky test timeout test_end_drains_result_queue_to_avoid_join_deadlock on Python 3.14#69525
FrankYang0529 wants to merge 1 commit into
apache:mainfrom
FrankYang0529:airflow-flaky-test_end_drains_result_queue_to_avoid_join_deadlock

Conversation

@FrankYang0529

@FrankYang0529 FrankYang0529 commented Jul 7, 2026

Copy link
Copy Markdown
Member

Why

  • Python 3.14 changed the default multiprocessing start method on Linux from fork to forkserver.

  • With fork, the child is "effectively identical to the parent process" and inherits the already-imported airflow modules. With forkserver it is forked from a clean server process instead. The child must import the test module and related airflow import chain.

  • From run 28801030578, both Postgres and Sqlite failed on the same test case test_end_drains_result_queue_to_avoid_join_deadlock with Python 3.14. Both took around 10 seconds to load modules.

    2026-07-06T15:37:46.9780300Z 2026-07-06T15:24:31.594329Z [info     ] Shutting down LocalExecutor; waiting for running tasks to finish.  Signal again if you don't want to wait. [airflow.executors.local_executor.LocalExecutor]
    2026-07-06T15:37:46.9782155Z 2026-07-06T15:24:38.789986Z [info     ] Unit test configuration loaded from 'unit_tests.cfg' [airflow.sdk.configuration] loc=configuration.py:247
    2026-07-06T15:37:46.9784010Z 2026-07-06T15:24:41.043595Z [warning  ] FileType is deprecated. Simply open files after parsing arguments. [py.warnings] category=PendingDeprecationWarning filename=/opt/airflow/airflow-core/src/airflow/cli/cli_config.py lineno=646
    2026-07-06T15:37:46.9786186Z 2026-07-06T15:24:41.045115Z [warning  ] FileType is deprecated. Simply open files after parsing arguments. [py.warnings] category=PendingDeprecationWarning filename=/opt/airflow/airflow-core/src/airflow/cli/cli_config.py lineno=868
    
    2026-07-06T16:23:12.8193184Z 2026-07-06T15:24:18.524457Z [info     ] Shutting down LocalExecutor; waiting for running tasks to finish.  Signal again if you don't want to wait. [airflow.executors.local_executor.LocalExecutor]
    2026-07-06T16:23:12.8195400Z 2026-07-06T15:24:26.279040Z [info     ] Unit test configuration loaded from 'unit_tests.cfg' [airflow.sdk.configuration] loc=configuration.py:247
    2026-07-06T16:23:12.8200174Z 2026-07-06T15:24:28.560555Z [warning  ] FileType is deprecated. Simply open files after parsing arguments. [py.warnings] category=PendingDeprecationWarning filename=/opt/airflow/airflow-core/src/airflow/cli/cli_config.py lineno=646
    2026-07-06T16:23:12.8203506Z 2026-07-06T15:24:28.562044Z [warning  ] FileType is deprecated. Simply open files after parsing arguments. [py.warnings] category=PendingDeprecationWarning filename=/opt/airflow/airflow-core/src/airflow/cli/cli_config.py lineno=868
    
  • Other failed run:

How

Change the test's execution_timeout from 10s to 30s. Align it with test_clean_stop_on_signal in the same file, which also spawns a subprocess.


Was generative AI tooling used to co-author this PR?
  • Yes - Claude Code

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

Signed-off-by: PoAn Yang <payang@apache.org>
@boring-cyborg boring-cyborg Bot added the area:Executors-core LocalExecutor & SequentialExecutor label Jul 7, 2026
@FrankYang0529 FrankYang0529 marked this pull request as ready for review July 7, 2026 10:09
@Lee-W

Lee-W commented Jul 8, 2026

Copy link
Copy Markdown
Member

Hey, I think it's been solved by #69538 instead. could you please check and confirm whehter my understanding is correct? Thanks!

@FrankYang0529

Copy link
Copy Markdown
Member Author

Yes, that is correct. Let me close my PR.

@FrankYang0529 FrankYang0529 deleted the airflow-flaky-test_end_drains_result_queue_to_avoid_join_deadlock branch July 8, 2026 11:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:Executors-core LocalExecutor & SequentialExecutor

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants