Skip to content

bpo-22005: Fixed unpickling instances of datetime classes pickled by Python 2. #11017

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 13 commits into from
Dec 7, 2018
Merged

bpo-22005: Fixed unpickling instances of datetime classes pickled by Python 2. #11017

merged 13 commits into from
Dec 7, 2018

Conversation

serhiy-storchaka
Copy link
Member

@serhiy-storchaka serhiy-storchaka commented Dec 7, 2018 •
edited by bedevere-bot
Loading

encoding='latin1' should be used for successful decoding.

(This is a copy of #794).

https://bugs.python.org/issue22005

@serhiy-storchaka
bpo-22005: Fixed unpickling instances of datetime classes pickled by …
caf031e 
…Python 2.

errors='surrogateescape' should be used for successful decoding.
@serhiy-storchaka
Merge branch 'master' into datetime-compat-unpickle
6bd9b2f
@serhiy-storchaka
Merge branch 'master' into datetime-compat-unpickle
531c5f6
@serhiy-storchaka
Update datetime.py
cb55d65
@serhiy-storchaka
Update _datetimemodule.c
f4c80ff
@serhiy-storchaka
Merge branch 'master' into datetime-compat-unpickle
447f137
@serhiy-storchaka
Move the NEWS entry to NEWS.d/.
985c923
@serhiy-storchaka
Merge branch 'master' into datetime-compat-unpickle
66bf2c2
@serhiy-storchaka
Merge branch 'master' into datetime-compat-unpickle
46b28a0
@serhiy-storchaka
Use encoding='latin1' instead of errors='surrogateescape'.
7d44e3a
@serhiy-storchaka
Merge branch 'master' into datetime-compat-unpickle
1f45085
@serhiy-storchaka
Merge branch 'master' into datetime-compat-unpickle
4e890c0
@serhiy-storchaka
Merge branch 'master' into datetime-compat-unpickle
93b65b0
@serhiy-storchaka serhiy-storchaka added type-bug An unexpected behavior, bug, or error needs backport to 3.6 labels Dec 7, 2018
@serhiy-storchaka serhiy-storchaka merged commit 8452ca1 into python:master Dec 7, 2018
@miss-islington
Copy link
Contributor

Thanks @serhiy-storchaka for the PR 🌮🎉.. I'm working now to backport this PR to: 3.6, 3.7.
🐍🍒⛏🤖

@miss-islington
Copy link
Contributor

Sorry, @serhiy-storchaka, I could not cleanly backport this to 3.7 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker 8452ca15f41061c8a6297d7956df22ab476d4df4 3.7

@miss-islington
Copy link
Contributor

Sorry, @serhiy-storchaka, I could not cleanly backport this to 3.6 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker 8452ca15f41061c8a6297d7956df22ab476d4df4 3.6

@serhiy-storchaka serhiy-storchaka mentioned this pull request Dec 7, 2018
Closed
@serhiy-storchaka serhiy-storchaka deleted the datetime-compat-unpickle2 branch December 7, 2018 11:43
serhiy-storchaka added a commit to serhiy-storchaka/cpython that referenced this pull request Dec 7, 2018
@serhiy-storchaka
[3.7] bpo-22005: Fixed unpickling instances of datetime classes pickl…
c40f3a4 
…ed by Python 2. (pythonGH-11017)

encoding='latin1' should be used for successful decoding..
(cherry picked from commit 8452ca1)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
@bedevere-bot
Copy link

GH-11022 is a backport of this pull request to the 3.7 branch.

serhiy-storchaka added a commit that referenced this pull request Dec 7, 2018
@serhiy-storchaka
[3.7] bpo-22005: Fixed unpickling instances of datetime classes pickl…
0d5730e 
…ed by Python 2. (GH-11017) (GH-11022)

encoding='latin1' should be used for successful decoding.
(cherry picked from commit 8452ca1)
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Dec 7, 2018
@serhiy-storchaka @miss-islington
[3.7] bpo-22005: Fixed unpickling instances of datetime classes pickl…
b8f228c 
…ed by Python 2. (pythonGH-11017) (pythonGH-11022)

encoding='latin1' should be used for successful decoding.
(cherry picked from commit 8452ca1)
(cherry picked from commit 0d5730e)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x SLES 3.7 has failed when building commit 0d5730e.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/122/builds/876) and take a look at the build logs.
  4. Check if the failure is related to this commit (0d5730e) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/122/builds/876

Click to see traceback logs 
fatal: unable to access 'https://github.com/python/cpython.git/': Unknown SSL protocol error in connection to github.com:443 

make: *** No rule to make target 'distclean'.  Stop.

serhiy-storchaka pushed a commit that referenced this pull request Dec 7, 2018
@miss-islington @serhiy-storchaka
bpo-22005: Fixed unpickling instances of datetime classes pickled by …
19f6e83 
…Python 2. (GH-11017) (GH-11022) (GH-11024)

encoding='latin1' should be used for successful decoding.
(cherry picked from commit 8452ca1)
(cherry picked from commit 0d5730e)
@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x RHEL 3.6 has failed when building commit 19f6e83.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/29/builds/758) and take a look at the build logs.
  4. Check if the failure is related to this commit (19f6e83) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/29/builds/758

Click to see traceback logs 
Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/test/test_robotparser.py", line 355, in test_read_404
    self.assertTrue(parser.allow_all)
AssertionError: False is not true

----------------------------------------------------------------------

Ran 40 tests in 5.037s

FAILED (failures=1)


Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/test/test_httplib.py", line 1605, in test_networked_good_cert
    h.request('GET', '/')
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 964, in send
    self.connect()
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1400, in connect
    server_hostname=server_hostname)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/ssl.py", line 407, in wrap_socket
    _context=self, _session=session)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/ssl.py", line 817, in __init__
    self.do_handshake()
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/ssl.py", line 1077, in do_handshake
    self._sslobj.do_handshake()
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/ssl.py", line 689, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:847)

======================================================================
ERROR: test_networked_noverification (test.test_httplib.HTTPSTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/test/test_httplib.py", line 1577, in test_networked_noverification
    h.request('GET', '/')
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 964, in send
    self.connect()
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1400, in connect
    server_hostname=server_hostname)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/ssl.py", line 407, in wrap_socket
    _context=self, _session=session)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/ssl.py", line 817, in __init__
    self.do_handshake()
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/ssl.py", line 1077, in do_handshake
    self._sslobj.do_handshake()
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/ssl.py", line 689, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:847)

======================================================================
ERROR: test_networked_trusted_by_default_cert (test.test_httplib.HTTPSTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/test/support/__init__.py", line 767, in dec
    f(*args, **kwargs)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/test/test_httplib.py", line 1589, in test_networked_trusted_by_default_cert
    h.request('GET', '/')
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 964, in send
    self.connect()
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1400, in connect
    server_hostname=server_hostname)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/ssl.py", line 407, in wrap_socket
    _context=self, _session=session)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/ssl.py", line 817, in __init__
    self.do_handshake()
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/ssl.py", line 1077, in do_handshake
    self._sslobj.do_handshake()
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/ssl.py", line 689, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:847)

======================================================================
FAIL: test_networked (test.test_httplib.HTTPSTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/test/test_httplib.py", line 1567, in test_networked
    self.assertEqual(exc_info.exception.reason, 'CERTIFICATE_VERIFY_FAILED')
AssertionError: None != 'CERTIFICATE_VERIFY_FAILED'

======================================================================
FAIL: test_networked_bad_cert (test.test_httplib.HTTPSTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/test/test_httplib.py", line 1623, in test_networked_bad_cert
    self.assertEqual(exc_info.exception.reason, 'CERTIFICATE_VERIFY_FAILED')
AssertionError: None != 'CERTIFICATE_VERIFY_FAILED'

----------------------------------------------------------------------

Ran 103 tests in 5.298s

FAILED (failures=2, errors=3)


Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/test/test_httplib.py", line 1605, in test_networked_good_cert
    h.request('GET', '/')
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 964, in send
    self.connect()
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1400, in connect
    server_hostname=server_hostname)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/ssl.py", line 407, in wrap_socket
    _context=self, _session=session)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/ssl.py", line 817, in __init__
    self.do_handshake()
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/ssl.py", line 1077, in do_handshake
    self._sslobj.do_handshake()
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/ssl.py", line 689, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:847)

======================================================================
ERROR: test_networked_noverification (test.test_httplib.HTTPSTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/test/test_httplib.py", line 1577, in test_networked_noverification
    h.request('GET', '/')
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 964, in send
    self.connect()
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1400, in connect
    server_hostname=server_hostname)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/ssl.py", line 407, in wrap_socket
    _context=self, _session=session)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/ssl.py", line 817, in __init__
    self.do_handshake()
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/ssl.py", line 1077, in do_handshake
    self._sslobj.do_handshake()
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/ssl.py", line 689, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:847)

======================================================================
ERROR: test_networked_trusted_by_default_cert (test.test_httplib.HTTPSTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/test/support/__init__.py", line 767, in dec
    f(*args, **kwargs)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/test/test_httplib.py", line 1589, in test_networked_trusted_by_default_cert
    h.request('GET', '/')
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 964, in send
    self.connect()
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/http/client.py", line 1400, in connect
    server_hostname=server_hostname)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/ssl.py", line 407, in wrap_socket
    _context=self, _session=session)
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/ssl.py", line 817, in __init__
    self.do_handshake()
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/ssl.py", line 1077, in do_handshake
    self._sslobj.do_handshake()
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/ssl.py", line 689, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:847)

======================================================================
FAIL: test_networked (test.test_httplib.HTTPSTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/test/test_httplib.py", line 1567, in test_networked
    self.assertEqual(exc_info.exception.reason, 'CERTIFICATE_VERIFY_FAILED')
AssertionError: None != 'CERTIFICATE_VERIFY_FAILED'

======================================================================
FAIL: test_networked_bad_cert (test.test_httplib.HTTPSTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.6.edelsohn-rhel-z/build/Lib/test/test_httplib.py", line 1623, in test_networked_bad_cert
    self.assertEqual(exc_info.exception.reason, 'CERTIFICATE_VERIFY_FAILED')
AssertionError: None != 'CERTIFICATE_VERIFY_FAILED'

----------------------------------------------------------------------

Ran 103 tests in 0.231s

FAILED (failures=2, errors=3)

@serhiy-storchaka serhiy-storchaka removed their assignment Dec 29, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
type-bug An unexpected behavior, bug, or error
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@serhiy-storchaka @miss-islington @bedevere-bot @the-knights-who-say-ni