Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bpo-46476: Fix memory leak in code objects generated by deepfreeze #30853

Merged
merged 25 commits into from Jan 27, 2022

Conversation

kumaraditya303
Copy link
Contributor

@kumaraditya303 kumaraditya303 commented Jan 24, 2022

@kumaraditya303
Copy link
Contributor Author

Valgrind log after this PR:

==7131== HEAP SUMMARY:
==7131==     in use at exit: 0 bytes in 0 blocks
==7131==   total heap usage: 18,468 allocs, 18,468 frees, 2,631,556 bytes allocated
==7131== 
==7131== All heap blocks were freed -- no leaks are possible
==7131== 
==7131== Use --track-origins=yes to see where uninitialised values come from
==7131== For lists of detected and suppressed errors, rerun with: -s
==7131== ERROR SUMMARY: 81 errors from 16 contexts (suppressed: 0 from 0)

Programs/_freeze_module.c Outdated Show resolved Hide resolved
Programs/_freeze_module.c Outdated Show resolved Hide resolved
Programs/_bootstrap_python.c Outdated Show resolved Hide resolved
@bedevere-bot
Copy link

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

And if you don't make the requested changes, you will be put in the comfy chair!

@tiran
Copy link
Member

tiran commented Jan 24, 2022

The new code is causing a segfault.

@kumaraditya303 kumaraditya303 marked this pull request as draft January 24, 2022 17:09
Modules/main.c Outdated Show resolved Hide resolved
Modules/main.c Outdated Show resolved Hide resolved
Tools/scripts/deepfreeze.py Outdated Show resolved Hide resolved
Tools/scripts/deepfreeze.py Outdated Show resolved Hide resolved
Tools/scripts/deepfreeze.py Outdated Show resolved Hide resolved
@kumaraditya303 kumaraditya303 marked this pull request as ready for review January 25, 2022 05:46
Python/pylifecycle.c Outdated Show resolved Hide resolved
Python/pylifecycle.c Outdated Show resolved Hide resolved
Tools/scripts/deepfreeze.py Outdated Show resolved Hide resolved
@kumaraditya303
Copy link
Contributor Author

I have made the requested changes; please review again

@bedevere-bot
Copy link

Thanks for making the requested changes!

@tiran: please review the changes made to this pull request.

@vstinner
Copy link
Member

Before reviewing, I checked that the PR fix my issue: https://bugs.python.org/issue46476#msg411315

Yes, it does :-) Python no longer leaks any memory block at exit!

$ ./python -I -X showrefcount -c pass
[-5 refs, 0 blocks]

@vstinner
Copy link
Member

[dealloc_codeobject] can be easily moved when there is more need for such function as currently as I don't think there is any so I think this is good enough for now.

My concern is more that this C code is far from the code responsible to handle PyCodeObject, codeobject.c. I would prefer to make it close to code_dealloc() to ease its maintenance.

The python core currently does not have any immortal objects support so it isn't usable in there too.

Even if it's somehow "hidden", there is already _PyObject_IMMORTAL_INIT() in Include/internal/pycore_object.h.

@kumaraditya303
Copy link
Contributor Author

My concern is more that this C code is far from the code responsible to handle PyCodeObject, codeobject.c. I would prefer to make it close to code_dealloc() to ease its maintenance.

Ok, I moved it to the codeobject.c added a comment as it is for deepfreeze.

Even if it's somehow "hidden", there is already _PyObject_IMMORTAL_INIT() in Include/internal/pycore_object.h.

That is a partial implementation because if an object is immortal then Py_INCREF and Py_DECREF should be no-op for them but that's another topic.

Objects/codeobject.c Outdated Show resolved Hide resolved
Objects/codeobject.c Outdated Show resolved Hide resolved
@bedevere-bot
Copy link

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

@vstinner vstinner merged commit c7f810b into python:main Jan 27, 2022
@vstinner
Copy link
Member

Thanks @kumaraditya303 for the PR and the updates, the final change is even better! I merged your PR.

Ah right, I hesitated to complain about the braces / PEP 7, @tiran did it ;-)

@kumaraditya303 kumaraditya303 deleted the fix-code branch January 27, 2022 13:15
Copy link
Member

@gvanrossum gvanrossum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry I missed the review, but I have a question.

Objects/codeobject.c Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants