Skip to content

Fix obscure lru_cache reentrancy bug #73177

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
rhettinger opened this issue Dec 16, 2016 · 6 comments
Closed

Fix obscure lru_cache reentrancy bug #73177

rhettinger opened this issue Dec 16, 2016 · 6 comments
Assignees
Labels
3.7 (EOL) end of life stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@rhettinger
Copy link
Contributor

BPO 28991
Nosy @rhettinger, @serhiy-storchaka
PRs
  • [Do Not Merge] Convert Misc/NEWS so that it is managed by towncrier #552
  • Files
  • fix_lru_len_reentrancy.diff
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/rhettinger'
    closed_at = <Date 2016-12-16.22:16:09.356>
    created_at = <Date 2016-12-16.18:59:17.257>
    labels = ['3.7', 'type-bug', 'library']
    title = 'Fix obscure lru_cache reentrancy bug'
    updated_at = <Date 2017-03-31.16:36:08.034>
    user = 'https://github.com/rhettinger'

    bugs.python.org fields:

    activity = <Date 2017-03-31.16:36:08.034>
    actor = 'dstufft'
    assignee = 'rhettinger'
    closed = True
    closed_date = <Date 2016-12-16.22:16:09.356>
    closer = 'rhettinger'
    components = ['Library (Lib)']
    creation = <Date 2016-12-16.18:59:17.257>
    creator = 'rhettinger'
    dependencies = []
    files = ['45927']
    hgrepos = []
    issue_num = 28991
    keywords = ['patch']
    message_count = 6.0
    messages = ['283426', '283427', '283438', '283444', '283445', '283447']
    nosy_count = 3.0
    nosy_names = ['rhettinger', 'python-dev', 'serhiy.storchaka']
    pr_nums = ['552']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'commit review'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue28991'
    versions = ['Python 3.5', 'Python 3.6', 'Python 3.7']

    @rhettinger
    Copy link
    Contributor Author

    Call __len__ directly inside the LRU cache code rather than using len(). This helps further encapsulate the cache internals making it less dependent on parts of the environment that are subject to change.

    @rhettinger rhettinger added 3.7 (EOL) end of life stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error labels Dec 16, 2016
    @serhiy-storchaka
    Copy link
    Member

    Wrapping len() in an lru_cache is bad idea, because this brakes len() of lists, dicts and other non-hashable collections.

    Should we support this doubtful case?

    @rhettinger
    Copy link
    Contributor Author

    It is effortless to make this change and is consistent with all the other efforts taken to to protect against reentrancy, so yes, I think it should be done. It isn't so much that I want this to be a use case, I would just like the implementation to be as sound as reasonably possible so that we don't find the LRU cache to be at heart of some future hard-to-find bug in a complex application.

    How does the patch look to you?

    @serhiy-storchaka
    Copy link
    Member

    How does the patch look to you?

    It looks ugly, but technically correct. Push it if you like.

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Dec 16, 2016

    New changeset c23f8614151d by Raymond Hettinger in branch '3.5':
    Issue bpo-28991: Fix obscure reentrancy bug in functools.lru_cache().
    https://hg.python.org/cpython/rev/c23f8614151d

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Dec 16, 2016

    New changeset 5ec5bfcf0089 by Raymond Hettinger in branch 'default':
    Issue bpo-28991: Address comment that the __len__ call looked unattractive
    https://hg.python.org/cpython/rev/5ec5bfcf0089

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.7 (EOL) end of life stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    2 participants