Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bpo-46608: exclude marshalled-frozen data if deep-freezing to save 300 KB space #31074

Merged
merged 17 commits into from
Feb 4, 2022

Conversation

kumaraditya303
Copy link
Contributor

@kumaraditya303 kumaraditya303 commented Feb 2, 2022

This reduces the size of the data segment by 300 KB of the executable because if the modules are deep-frozen then the marshalled frozen data just wastes space. This was inspired by comment by @gvanrossum in #29118 (comment). Note: There is a new option --deepfreeze-only in freeze_modules.py to change this behavior, it is on be default to save disk space.

# du -s ./python before
27892   ./python
# du -s ./python after
27524   ./python

https://bugs.python.org/issue46608

Automerge-Triggered-By: GH:ericsnowcurrently

@kumaraditya303 kumaraditya303 changed the title exclude marshalled-frozen data if deep-freezing to save 300 KB space bpo-46608: exclude marshalled-frozen data if deep-freezing to save 300 KB space Feb 2, 2022
@kumaraditya303
Copy link
Contributor Author

kumaraditya303 commented Feb 2, 2022

I added is_package field to _frozen struct to differentiate between package and module as the struct is private (prefixed with underscore).

@gvanrossum
Copy link
Member

Hopefully @ericsnowcurrently can review this -- he should know about use cases where this might break.

Copy link
Member

@ericsnowcurrently ericsnowcurrently left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other than one small thing, this is looking good. (The PR is marked as "Draft". Are you planning on changing anything?)

Include/cpython/import.h Show resolved Hide resolved
Python/import.c Show resolved Hide resolved
@bedevere-bot
Copy link

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

@brettcannon brettcannon removed their request for review February 3, 2022 19:59
@bedevere-bot
Copy link

Thanks for making the requested changes!

@ericsnowcurrently: please review the changes made to this pull request.

Copy link
Member

@ericsnowcurrently ericsnowcurrently left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With a few small tweaks, this is ready to go.

Doc/c-api/import.rst Outdated Show resolved Hide resolved
Doc/whatsnew/3.11.rst Outdated Show resolved Hide resolved
@bedevere-bot
Copy link

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

kumaraditya303 and others added 3 commits February 4, 2022 22:52
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
Copy link
Member

@ericsnowcurrently ericsnowcurrently left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Thanks for working on this, Kumar!

@miss-islington
Copy link
Contributor

@kumaraditya303: Status check is done, and it's a failure ❌ .

2 similar comments
@miss-islington
Copy link
Contributor

@kumaraditya303: Status check is done, and it's a failure ❌ .

@miss-islington
Copy link
Contributor

@kumaraditya303: Status check is done, and it's a failure ❌ .

Copy link
Member

@gvanrossum gvanrossum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't follow the logic of the argument parsing, and I have one small grammar tweak. Otherwise LGTM!

Doc/whatsnew/3.11.rst Show resolved Hide resolved
Tools/scripts/freeze_modules.py Show resolved Hide resolved
@miss-islington
Copy link
Contributor

@kumaraditya303: Status check is done, and it's a success ✅ .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants