Skip to content

gh-154594: Fix copy.deepcopy() re-copying objects whose __deepcopy__ returns None - #154819

Open
sreehariannam wants to merge 1 commit into
python:mainfrom
sreehariannam:gh-154594-deepcopy-none-memo
Open

gh-154594: Fix copy.deepcopy() re-copying objects whose __deepcopy__ returns None#154819
sreehariannam wants to merge 1 commit into
python:mainfrom
sreehariannam:gh-154594-deepcopy-none-memo

Conversation

@sreehariannam

Copy link
Copy Markdown

Fixes gh-154594.

copy.deepcopy() uses memo.get(d, None) to check the memo dict, treating None both as the "not cached" sentinel and as a value a __deepcopy__ can legitimately return. When an object's __deepcopy__ returns None, the memo hit is indistinguishable from a miss, so the object gets deep-copied again on every subsequent reference instead of once:

import copy

call_count = 0

class C:
    def __deepcopy__(self, memo):
        global call_count
        call_count += 1
        memo[id(self)] = None
        return None

obj = C()
copy.deepcopy([obj, obj, obj])
print(call_count)  # 3, expected 1

This was introduced by gh-132657 / GH-138429, which replaced the previous _nil = [] sentinel with None specifically because None is an immortal singleton in CPython, avoiding refcount contention under free-threading that a plain sentinel object would reintroduce.

Rather than reintroducing a sentinel object (which would bring back that contention) or special-casing None, this switches the lookup to memo[d] guarded by try/except KeyError, matching the pattern already used a few lines down in _deepcopy_tuple and _deepcopy_frozendict. This distinguishes every possible cached value (including None) from a genuine cache miss without any shared sentinel object.

Added a regression test and a changelog entry.

…opy__ returns None

deepcopy() used None both as the memo dict's "not cached" sentinel and
as a value callers can legitimately store there, so a __deepcopy__ that
returns None was indistinguishable from a cache miss and got invoked
again on every subsequent reference to the same object.

Use a direct memo[d] lookup (matching the existing pattern in
_deepcopy_tuple/_deepcopy_frozendict) instead of memo.get(d, None), so
every return value -- including None -- is memoized correctly. This
also avoids reintroducing the non-immortal-sentinel refcount
contention under free-threading that pythongh-132657/pythongh-138429 fixed, since
no sentinel object is used at all.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

copy.deepcopy memo lookup fails when __deepcopy__ returns None

1 participant