Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix issue in pre-importing modules in zipfile #31061

Merged
merged 1 commit into from
May 4, 2023

Conversation

ephraimbuddy
Copy link
Contributor

This fixes the scheduler crash due to reading a zipped file as source code when trying to pre-import airflow modules from the python modules in the zip files. The fix was to extract the python modules in the zip file and then pre-import airflow modules in them

closes: #31039

@boring-cyborg boring-cyborg bot added the area:Scheduler Scheduler or dag parsing Issues label May 4, 2023
@ephraimbuddy ephraimbuddy requested a review from ashb May 4, 2023 13:45
@ephraimbuddy ephraimbuddy added the type:bug-fix Changelog: Bug Fixes label May 4, 2023
@ephraimbuddy ephraimbuddy added this to the Airflow 2.6.1 milestone May 4, 2023
This fixes the scheduler crash due to reading a zipped file as source code
when trying to pre-import airflow modules from the python modules in the zip files.
The fix was to extract the python modules in the zip file and then pre-import airflow modules in them

apply suggestions from code review
@ephraimbuddy ephraimbuddy merged commit 34b6230 into apache:main May 4, 2023
42 checks passed
@ephraimbuddy ephraimbuddy deleted the fix-zip-file-issue branch May 4, 2023 17:18
ephraimbuddy added a commit that referenced this pull request May 8, 2023
This fixes the scheduler crash due to reading a zipped file as source code
when trying to pre-import airflow modules from the python modules in the zip files.
The fix was to extract the python modules in the zip file and then pre-import airflow modules in them

apply suggestions from code review

(cherry picked from commit 34b6230)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:Scheduler Scheduler or dag parsing Issues type:bug-fix Changelog: Bug Fixes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Packaged DAGs not getting loaded in Airflow 2.6.0 (ValueError: source code string cannot contain null bytes)
3 participants