Destructor behavior faulty

[wrogner]

BPO	1717900
Nosy	@birkenfeld, @pitrou
Superseder	bpo-812369: module shutdown procedure based on GC

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2009-03-31.13:16:00.230>
created_at = <Date 2007-05-12.20:41:54.000>
labels = ['interpreter-core', 'type-bug']
title = 'Destructor behavior faulty'
updated_at = <Date 2009-03-31.13:17:18.826>
user = 'https://bugs.python.org/wrogner'

bugs.python.org fields:

activity = <Date 2009-03-31.13:17:18.826>
actor = 'pitrou'
assignee = 'none'
closed = True
closed_date = <Date 2009-03-31.13:16:00.230>
closer = 'pitrou'
components = ['Interpreter Core']
creation = <Date 2007-05-12.20:41:54.000>
creator = 'wrogner'
dependencies = []
files = []
hgrepos = []
issue_num = 1717900
keywords = []
message_count = 12.0
messages = ['31992', '31993', '31994', '31995', '31996', '31997', '31998', '31999', '32000', '32001', '84772', '84774']
nosy_count = 6.0
nosy_names = ['georg.brandl', 'sf-robot', 'alanmcintyre', 'ggenellina', 'pitrou', 'wrogner']
pr_nums = []
priority = 'normal'
resolution = 'duplicate'
stage = 'test needed'
status = 'closed'
superseder = '812369'
type = 'behavior'
url = 'https://bugs.python.org/issue1717900'
versions = ['Python 3.1', 'Python 2.7']

[wrogner]

I tried example 11.4. from bytesofpython (by C.H. Swaroop).
Example works fine.

Added a new Person instance 'wolf' -> Program terminated with:
Exception exceptions.AttributeError: "'NoneType' object has no attribute 'population'" in <bound method Person.__del__ of <main.Person instance at 0xb7d48d6c>> ignored

added print list(globals()) ->
['kalam', '__builtins__', '__file__', 'DBGPHideChildren', 'swaroop', 'Person', 'wolf', '__name__', '__doc__']

changed wolf to aaa:

print list(globals()) ->
['aaa', 'kalam', '__builtins__', '__file__', 'DBGPHideChildren', 'swaroop', 'Person', '__name__', '__doc__']

Please note the position of 'aaa' at the beginning of the list, before 'Person'. With 'wolf' being after 'Person'.

If the destructing code removes items in this order, no wonder I get an error.

Person should not get deleted if refcount is still > 0.

Wolf Rogner

[alanmcintyre]

Could you post the code for your entire script? It makes it a lot easier to figure out what's going on.

[alanmcintyre]

I took the example mentioned from here:
http://www.python.org/doc/essays/cleanup/

and added this line to the end:

wolf = Person('wolf')

and it gives the reported error. Here is a minimal snippet that produces the same error when executed as the top-level module:

class Person:
        population = 0
        def __del__(self):
                Person.population -= 1

wolf = Person()

This appears to be consistent with the behavior described here:
http://www.python.org/doc/essays/cleanup/

While I understand that cleaning up a module at exit time is probably not an easy thing to make arbitrarily smart, this behavior seems a little too not-smart to me. It seems like it's not all that hard to get bitten by it, and the error makes no sense unless you're familiar with the module cleanup algorithm.

For what it's worth, I offer to help make module cleanup a little smarter, although I may not be able to spend much time on it until I finish some things I'm already committed to do.

[ggenellina]

FWIW, the script name appears to be relevant as well. I were going to say that I could not reproduce it as it was; this same example saved as "a.py" doesn't show the error; "w.py" does.

To the OP: Module finalization is a fragile step; this is a long standing issue and could be improved, but anyway I don't think it can be made robust enough (this is just my opinion!). I usually try to *never* reference any globals in destructors. In this case, using self.__class__ instead of Person is safer and works fine; if other globals were needed they could be passed as default argument values.

[ggenellina]

FWIW, the script name appears to be relevant as well. I were going to say that I could not reproduce it as it was; this same example saved as "a.py" doesn't show the error; "w.py" does.

To the OP: Module finalization is a fragile step; this is a long standing issue and could be improved, but anyway I don't think it can be made robust enough (this is just my opinion!). I usually try to *never* reference any globals in destructors. In this case, using self.__class__ instead of Person is safer and works fine; if other globals were needed they could be passed as default argument values.

[ggenellina]

FWIW, the script name appears to be relevant as well. I were going to say that I could not reproduce it as it was; this same example saved as "a.py" doesn't show the error; "w.py" does.

To the OP: Module finalization is a fragile step; this is a long standing issue and could be improved, but anyway I don't think it can be made robust enough (this is just my opinion!). I usually try to *never* reference any globals in destructors. In this case, using self.__class__ instead of Person is safer and works fine; if other globals were needed they could be passed as default argument values.

[alanmcintyre]

I tried out a simple change (to the trunk) in _PyModule_Clear to prevent this particular problem. Between the "remove everything beginning with an underscore" and the "remove everything except __builtins__" steps, I added a step to remove all instance objects in the module's dictionary. It appears to stop this problem and passes the regression test suite. I can post it as a patch if this seems like a reasonable change to make.

Also, I noticed that earlier I posted the wrong link for the location of the example code; it should have been:
http://www.dpawson.co.uk/bop/byteofpython_120.txt

[birkenfeld]

Alan: you should bring that up on python-dev.

[sf-robot]

This Tracker item was closed automatically by the system. It was
previously set to a Pending status, and the original submitter
did not respond within 14 days (the time period specified by
the administrator of this Tracker).

[alanmcintyre]

Brought this up on python-dev: http://mail.python.org/pipermail/python-dev/2007-May/073329.html

Since there is some interest in making the change Armin suggested, I suggest re-opening this item so that it doesn't get overlooked/forgotten.

[pitrou]

Armin's proposal is in bpo-812369, closing this bug.
(it is not obvious Armin's patch is enough to solve the problem at hand,
but the problem is well-known anyway and there are certainly other bug
entries pointing to it :-))

[pitrou]

By the way, an easy way to fix it would probably to rewrite the
destructor in the following way (haven't tested):

    def __del__(self):
        self.__class__.population -= 1