Skip to content

Give each mypy prek hook its own file list in CI - #70654

Draft
FrankYang0529 wants to merge 1 commit into
apache:mainfrom
FrankYang0529:airflow-fix-mypy-hook-concurrency
Draft

Give each mypy prek hook its own file list in CI#70654
FrankYang0529 wants to merge 1 commit into
apache:mainfrom
FrankYang0529:airflow-fix-mypy-hook-concurrency

Conversation

@FrankYang0529

@FrankYang0529 FrankYang0529 commented Jul 29, 2026

Copy link
Copy Markdown
Member

Why

  • prek runs hooks from different prek projects concurrently. Every one of them wrote its mypy file list to the same files/mypy_files.txt.
  • On v3-3-test run 30279407098, mypy-shared-configuration, mypy-shared-dagnode and mypy-shared-listeners all failed on non-existent path.

Their writing file time are close.

2026-07-27T17:28:58.9980347Z     Running mypy for folders: ['shared/configuration']
2026-07-27T17:28:58.9981475Z     You can check the list of files in: /home/runner/work/airflow/airflow/files/mypy_files.txt

2026-07-27T17:28:59.0832479Z     Running mypy for folders: ['shared/dagnode']
2026-07-27T17:28:59.0833606Z     You can check the list of files in: /home/runner/work/airflow/airflow/files/mypy_files.txt

2026-07-27T17:28:59.1323493Z     Running mypy for folders: ['shared/listeners']
2026-07-27T17:28:59.1324120Z     You can check the list of files in: /home/runner/work/airflow/airflow/files/mypy_files.txt
2026-07-27T17:28:59.0455195Z     mypy: Cannot read file 'shared/dagnode/tests/dagnode/__init__.pyteners/spec/taskinstance.py': No such file or directory

2026-07-27T17:28:59.1199107Z     mypy: Cannot read file 'shared/dagnode/tests/dagnode/__init__.pyteners/spec/taskinstance.py': No such file or directory

2026-07-27T17:28:59.1521972Z     mypy: Cannot read file 'shared/dagnode/tests/dagnode/__init__.pyteners/spec/taskinstance.py': No such file or directory

How

  • run_mypy_full_dist_local_venv_or_breeze_in_ci.py: Use suffix argument in write_file_list() to avoid overwriting same file.

Was generative AI tooling used to co-author this PR?
  • Yes - Claude Code

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

Signed-off-by: PoAn Yang <payang@apache.org>
@FrankYang0529
FrankYang0529 force-pushed the airflow-fix-mypy-hook-concurrency branch from 18c1b6b to 16e39f3 Compare July 29, 2026 08:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant