PYTHON-3347 Test against Python 3.11 prerelease #1069

blink1073 · 2022-10-05T21:42:37Z

No description provided.

ShaneHarvey · 2022-10-05T21:56:13Z

Assuming the 3.11 tests pass let's add the 3.11 trove classifier too.

blink1073 · 2022-10-06T22:06:00Z

Failures that were pre-existing in recent build:

test_dns_record_lookup_empty (test_srv_polling.TestSrvPolling)
test_dns_failures (test_srv_polling.TestSrvPolling)
test_srv_polling.TestSrvPolling.test_dns_failures
test_srv_polling.TestSrvPolling.test_dns_record_lookup_empty
[2022/10/06 21:02:41.676] Running command 'shell.exec' in "run enterprise auth tests" (step 2 of 2). [2022/10/06 21:09:54.828] Command ''shell.exec' in "run enterprise auth tests"' failed: shell script encountered problem: exit code 1. src

New failures:

test_corpus (test.test_encryption.TestCorpus.test_corpus) src
ModuleNotFoundError: No module named 'gevent' src - due to Compatibility with Python 3.11 and greenlet 2.0.0 gevent/gevent#1867

ShaneHarvey · 2022-10-06T23:27:37Z

 [2022/10/06 21:40:11.633] ERROR: test_corpus (test.test_encryption.TestCorpus.test_corpus)
 [2022/10/06 21:40:11.633] ----------------------------------------------------------------------
 [2022/10/06 21:40:11.633] Traceback (most recent call last):
 [2022/10/06 21:40:11.633]   File "C:\data\mci\ccc3b9397cdce05218603e98ad57b081\src\pymongo\encryption.py", line 76, in _wrap_encryption_errors
 [2022/10/06 21:40:11.633]     yield
 [2022/10/06 21:40:11.633]   File "C:\data\mci\ccc3b9397cdce05218603e98ad57b081\src\pymongo\encryption.py", line 364, in encrypt
 [2022/10/06 21:40:11.633]     encrypted_cmd = self._auto_encrypter.encrypt(database, encoded_cmd)
 [2022/10/06 21:40:11.633]                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [2022/10/06 21:40:11.633]   File "Z:\data\mci\ccc3b9397cdce05218603e98ad57b081\src\venv-encryption\Lib\site-packages\pymongocrypt\auto_encrypter.py", line 44, in encrypt
 [2022/10/06 21:40:11.633]     return run_state_machine(ctx, self.callback)
 [2022/10/06 21:40:11.633]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [2022/10/06 21:40:11.633]   File "Z:\data\mci\ccc3b9397cdce05218603e98ad57b081\src\venv-encryption\Lib\site-packages\pymongocrypt\state_machine.py", line 150, in run_state_machine
 [2022/10/06 21:40:11.633]     callback.kms_request(kms_ctx)
 [2022/10/06 21:40:11.633]   File "C:\data\mci\ccc3b9397cdce05218603e98ad57b081\src\pymongo\encryption.py", line 142, in kms_request
 [2022/10/06 21:40:11.633]     data = conn.recv(kms_context.bytes_needed)
 [2022/10/06 21:40:11.633]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [2022/10/06 21:40:11.633]   File "C:\python\Python311\Lib\ssl.py", line 1263, in recv
 [2022/10/06 21:40:11.633]     return self.read(buflen)
 [2022/10/06 21:40:11.633]            ^^^^^^^^^^^^^^^^^
 [2022/10/06 21:40:11.633]   File "C:\python\Python311\Lib\ssl.py", line 1136, in read
 [2022/10/06 21:40:11.633]     return self._sslobj.read(len)
 [2022/10/06 21:40:11.633]            ^^^^^^^^^^^^^^^^^^^^^^
 [2022/10/06 21:40:11.633] TimeoutError: The read operation timed out

Interesting, is this a fluke or does it happen consistently?

blink1073 · 2022-10-07T15:21:24Z

I kicked the task and it passed

ShaneHarvey · 2022-10-07T18:24:49Z

test/test_bson.py

        )
+        # The repr() of an enum changed in Python 3.11.
+        if sys.version_info >= (3, 11):
+            r = r.replace("DatetimeConversion.DATETIME", str(DatetimeConversion.DATETIME))


We should fix CodecOptions.__repr__ to avoid changing behavior.

Also do you know if this was an intentional change? Could it be a python bug?

We're actually using the str() representation, and it is an intentional change:

IntEnum used to subclass int, Enum and now it subclasses int, ReprEnum.:

"Changed in version 3.11: str() is now int.str() to better support the replacement of existing constants use-case. format() was already int.format() for that same reason."

can we use format (f-string) instead?

I don't think so. If we had been going the other direction we could have specified int() formatting, but we'd have to basically recreate the old str() repr ourselves:

f'{DatetimeConversion.DATETIME_AUTO}' '4'

I think we can use class DatetimeConversion(int, enum.Enum) to keep the old behavior.

ShaneHarvey

One last question

pymongo/topology_description.py

ShaneHarvey · 2022-10-11T19:08:31Z

 [2022/10/11 16:54:43.381] FAIL [0.001s]: test_must_connect (test_client_context.TestClientContext.test_must_connect)
 [2022/10/11 16:54:43.381] ----------------------------------------------------------------------
 [2022/10/11 16:54:43.381] Traceback (most recent call last):
 [2022/10/11 16:54:43.381]   File "/data/mci/6d8fe5eaf124547d0611aa2dab642ffd/src/test/test_client_context.py", line 28, in test_must_connect
 [2022/10/11 16:54:43.381]     self.assertTrue(
 [2022/10/11 16:54:43.381] AssertionError: False is not true : client context must be connected when PYMONGO_MUST_CONNECT is set. Failed attempts:
 [2022/10/11 16:54:43.381] failed to connect client MongoClient(host=['localhost:27017'], document_class=dict, tz_aware=False, connect=True, serverselectiontimeoutms=5000): 'bytes' object has no attribute 'is_absolute', Timeout: 5.0s, Topology Description: <TopologyDescription id: 63459f235735d3dee7ba4219, topology_type: Unknown, servers: [<ServerDescription ('localhost', 27017) server_type: Unknown, rtt: None, error=AttributeError("'bytes' object has no attribute 'is_absolute'")>]>
 [2022/10/11 16:54:43.381] failed to connect client MongoClient(host=['localhost:27017'], document_class=dict, tz_aware=False, connect=True, serverselectiontimeoutms=5000, tls=True, tlscertificatekeyfile='/data/mci/6d8fe5eaf124547d0611aa2dab642ffd/src/test/certificates/client.pem', tlscafile='/data/mci/6d8fe5eaf124547d0611aa2dab642ffd/src/test/certificates/ca.pem'): 'bytes' object has no attribute 'is_absolute', Timeout: 5.0s, Topology Description: <TopologyDescription id: 63459f285735d3dee7ba421a, topology_type: Unknown, servers: [<ServerDescription ('localhost', 27017) server_type: Unknown, rtt: None, error=AttributeError("'bytes' object has no attribute 'is_absolute'")>]>

This error was fixed by PYTHON-3451.

…PYTHON-3347

ShaneHarvey

Could we merge this PR and push out the 3.11 gevent testing fix and the 3.11 macos wheels to a new ticket?

ShaneHarvey · 2022-10-12T21:46:05Z

.evergreen/run-tests.sh

+    # Install the framework from pip.
+    # Use the same flags as the ones used by gevent, in case we have to build the wheel.
+    $PYTHON -m pip install --upgrade pip wheel setuptools
+    $PYTHON -m pip install $GREEN_FRAMEWORK


Is this using a venv? If not we need to create one first before pip installing anything.

It was not, I'll add a note in the new ticket

blink1073 · 2022-10-12T21:50:15Z

Could we merge this PR and push out the 3.11 gevent testing fix and the 3.11 macos wheels to a new ticket?

Yep, sounds good

blink1073 · 2022-10-12T21:56:49Z

I opened https://jira.mongodb.org/browse/PYTHON-3471 and https://jira.mongodb.org/browse/PYTHON-3470

(cherry picked from commit df77653)

blink1073 added 3 commits September 27, 2022 21:39

PYTHON-3347 Test against Python 3.11 beta

d8ef911

debug print

2d7ad60

undo changes

2074a05

blink1073 requested review from juliusgeo and ShaneHarvey as code owners October 5, 2022 21:42

ShaneHarvey approved these changes Oct 5, 2022

View reviewed changes

blink1073 added 2 commits October 6, 2022 11:45

python 3.11 fixes

68c5e2e

address regression and add trove classifier

8db3bcb

ShaneHarvey requested changes Oct 7, 2022

View reviewed changes

blink1073 added 3 commits October 10, 2022 13:17

install green framework from pip

6b43084

address review

0ba77d6

try setting flags

dfdd7c5

ShaneHarvey reviewed Oct 10, 2022

View reviewed changes

pymongo/topology_description.py Show resolved Hide resolved

blink1073 added 6 commits October 10, 2022 16:43

run green framework on ubuntu 20.04

6796d43

fix matrix

a78200c

try adding deps

e0c7123

fix pip install

912309e

fix install

47558dd

try with less strict deps

98cca28

blink1073 marked this pull request as draft October 11, 2022 16:49

blink1073 added 2 commits October 11, 2022 12:43

try upgrading pip and wheel

2f2d429

try upgrading setuptools

c7b2aa6

blink1073 added 3 commits October 11, 2022 14:12

Merge branch 'master' of github.com:mongodb/mongo-python-driver into …

77a95d4

…PYTHON-3347

return to ubuntu 18

57954a1

add py311 to release

dfe75f1

blink1073 added 9 commits October 11, 2022 14:25

debug release mac

0d46073

debug release windows

f401735

cleanup

6b78490

try explicit deps

901645c

try installing newer greenlet

a582317

pin greenlet

00bc33a

try without pin in verbose mode

6779c48

build linux and windows wheels

25a3352

fix syntax

3808c6c

ShaneHarvey reviewed Oct 12, 2022

View reviewed changes

address review

fa3f39c

blink1073 marked this pull request as ready for review October 12, 2022 21:58

blink1073 requested a review from ShaneHarvey October 12, 2022 21:58

ShaneHarvey approved these changes Oct 12, 2022

View reviewed changes

blink1073 merged commit df77653 into mongodb:master Oct 13, 2022

blink1073 deleted the PYTHON-3347 branch October 13, 2022 11:09

blink1073 added a commit to blink1073/mongo-python-driver that referenced this pull request Oct 19, 2022

PYTHON-3347 Test against Python 3.11 prerelease (mongodb#1069)

fe25f15

(cherry picked from commit df77653)

PYTHON-3347 Test against Python 3.11 prerelease #1069

PYTHON-3347 Test against Python 3.11 prerelease #1069

Uh oh!

Conversation

blink1073 commented Oct 5, 2022

Uh oh!

ShaneHarvey commented Oct 5, 2022

Uh oh!

blink1073 commented Oct 6, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ShaneHarvey commented Oct 6, 2022

Uh oh!

blink1073 commented Oct 7, 2022

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ShaneHarvey left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ShaneHarvey commented Oct 11, 2022

Uh oh!

ShaneHarvey left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

blink1073 commented Oct 12, 2022

Uh oh!

blink1073 commented Oct 12, 2022

Uh oh!

Uh oh!

blink1073 commented Oct 6, 2022 •

edited

Loading