Fix obscure lru_cache reentrancy bug

[rhettinger]

BPO	28991
Nosy	@rhettinger, @serhiy-storchaka
PRs	[Do Not Merge] Convert Misc/NEWS so that it is managed by towncrier #552
Files	fix_lru_len_reentrancy.diff

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/rhettinger'
closed_at = <Date 2016-12-16.22:16:09.356>
created_at = <Date 2016-12-16.18:59:17.257>
labels = ['3.7', 'type-bug', 'library']
title = 'Fix obscure lru_cache reentrancy bug'
updated_at = <Date 2017-03-31.16:36:08.034>
user = 'https://github.com/rhettinger'

bugs.python.org fields:

activity = <Date 2017-03-31.16:36:08.034>
actor = 'dstufft'
assignee = 'rhettinger'
closed = True
closed_date = <Date 2016-12-16.22:16:09.356>
closer = 'rhettinger'
components = ['Library (Lib)']
creation = <Date 2016-12-16.18:59:17.257>
creator = 'rhettinger'
dependencies = []
files = ['45927']
hgrepos = []
issue_num = 28991
keywords = ['patch']
message_count = 6.0
messages = ['283426', '283427', '283438', '283444', '283445', '283447']
nosy_count = 3.0
nosy_names = ['rhettinger', 'python-dev', 'serhiy.storchaka']
pr_nums = ['552']
priority = 'normal'
resolution = 'fixed'
stage = 'commit review'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue28991'
versions = ['Python 3.5', 'Python 3.6', 'Python 3.7']

[rhettinger]

Call __len__ directly inside the LRU cache code rather than using len(). This helps further encapsulate the cache internals making it less dependent on parts of the environment that are subject to change.

[serhiy-storchaka]

Wrapping len() in an lru_cache is bad idea, because this brakes len() of lists, dicts and other non-hashable collections.

Should we support this doubtful case?

[rhettinger]

It is effortless to make this change and is consistent with all the other efforts taken to to protect against reentrancy, so yes, I think it should be done. It isn't so much that I want this to be a use case, I would just like the implementation to be as sound as reasonably possible so that we don't find the LRU cache to be at heart of some future hard-to-find bug in a complex application.

How does the patch look to you?

[serhiy-storchaka]

How does the patch look to you?

It looks ugly, but technically correct. Push it if you like.

[python-dev]

New changeset c23f8614151d by Raymond Hettinger in branch '3.5':
Issue bpo-28991: Fix obscure reentrancy bug in functools.lru_cache().
https://hg.python.org/cpython/rev/c23f8614151d

[python-dev]

New changeset 5ec5bfcf0089 by Raymond Hettinger in branch 'default':
Issue bpo-28991: Address comment that the __len__ call looked unattractive
https://hg.python.org/cpython/rev/5ec5bfcf0089