Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-113028: correctly memoize str when escapes added #113436

Merged
merged 4 commits into from Dec 24, 2023

Conversation

jeff5
Copy link
Contributor

@jeff5 jeff5 commented Dec 23, 2023

Ensure that when pickling a str using the Python implementation in pickle.py, the original object is made the memo key (as in the C version), rather than the replacement, which when escape sequences are added. The result is that a second reference to a string in the input will un-pickle as a repeat reference, rather than as a duplicate object. The change also adds a test for this.

Fixes #113028 .

This fixes a divergence between the Python and C implementations of pickle
for protocol 0, such that it pickle.py fails to re-use the first pickled
representation of strings involving characters that have to be escaped.
@jeff5 jeff5 marked this pull request as draft December 23, 2023 17:09
@jeff5
Copy link
Contributor Author

jeff5 commented Dec 23, 2023

Making this draft while I blurb it.

@jeff5 jeff5 marked this pull request as ready for review December 23, 2023 18:16
Copy link
Member

@serhiy-storchaka serhiy-storchaka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@serhiy-storchaka serhiy-storchaka merged commit 0839863 into python:main Dec 24, 2023
36 checks passed
@miss-islington-app
Copy link

Thanks @jeff5 for the PR, and @serhiy-storchaka for merging it 🌮🎉.. I'm working now to backport this PR to: 3.11, 3.12.
🐍🍒⛏🤖

@bedevere-app
Copy link

bedevere-app bot commented Dec 24, 2023

GH-113448 is a backport of this pull request to the 3.12 branch.

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Dec 24, 2023
…ythonGH-113436)

This fixes a divergence between the Python and C implementations of pickle
for protocol 0, such that it pickle.py fails to re-use the first pickled
representation of strings involving characters that have to be escaped.
(cherry picked from commit 0839863)

Co-authored-by: Jeff Allen <ja.py@farowl.co.uk>
@bedevere-app bedevere-app bot removed the needs backport to 3.12 bug and security fixes label Dec 24, 2023
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Dec 24, 2023
…ythonGH-113436)

This fixes a divergence between the Python and C implementations of pickle
for protocol 0, such that it pickle.py fails to re-use the first pickled
representation of strings involving characters that have to be escaped.
(cherry picked from commit 0839863)

Co-authored-by: Jeff Allen <ja.py@farowl.co.uk>
@bedevere-app
Copy link

bedevere-app bot commented Dec 24, 2023

GH-113449 is a backport of this pull request to the 3.11 branch.

@bedevere-app bedevere-app bot removed the needs backport to 3.11 only security fixes label Dec 24, 2023
serhiy-storchaka pushed a commit that referenced this pull request Dec 24, 2023
…H-113436) (GH-113448)

This fixes a divergence between the Python and C implementations of pickle
for protocol 0, such that it pickle.py fails to re-use the first pickled
representation of strings involving characters that have to be escaped.
(cherry picked from commit 0839863)

Co-authored-by: Jeff Allen <ja.py@farowl.co.uk>
serhiy-storchaka pushed a commit that referenced this pull request Dec 24, 2023
…H-113436) (GH-113449)

This fixes a divergence between the Python and C implementations of pickle
for protocol 0, such that it pickle.py fails to re-use the first pickled
representation of strings involving characters that have to be escaped.
(cherry picked from commit 0839863)

Co-authored-by: Jeff Allen <ja.py@farowl.co.uk>
@jeff5 jeff5 deleted the gh113028-str-esc-pickle branch December 25, 2023 17:12
ryan-duve pushed a commit to ryan-duve/cpython that referenced this pull request Dec 26, 2023
…ythonGH-113436)

This fixes a divergence between the Python and C implementations of pickle
for protocol 0, such that it pickle.py fails to re-use the first pickled
representation of strings involving characters that have to be escaped.
kulikjak pushed a commit to kulikjak/cpython that referenced this pull request Jan 22, 2024
…ythonGH-113436)

This fixes a divergence between the Python and C implementations of pickle
for protocol 0, such that it pickle.py fails to re-use the first pickled
representation of strings involving characters that have to be escaped.
aisk pushed a commit to aisk/cpython that referenced this pull request Feb 11, 2024
…ythonGH-113436)

This fixes a divergence between the Python and C implementations of pickle
for protocol 0, such that it pickle.py fails to re-use the first pickled
representation of strings involving characters that have to be escaped.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Difference between pickle.py and _pickle for certain strings
3 participants