Python 3.10 #5952

graingert · 2022-03-16T17:58:26Z

Closes Python 3.10 support #5460
Tests added / passed
Passes pre-commit run --all-files

github-actions · 2022-03-16T21:22:00Z

Unit Test Results

      18 files +      18       18 suites +18 9h 25m 41s ⏱️ + 9h 25m 41s
  2 699 tests +  2 699   2 616 ✔️ +  2 616     81 💤 +    81 2 ❌ +2
24 127 runs +24 127 22 839 ✔️ +22 839 1 286 💤 +1 286 2 ❌ +2

For more details on these failures, see this check.

Results for commit b0308ac. ± Comparison against base commit b872d45.

♻️ This comment has been updated with latest results.

jakirkham

Thanks for working on this Thomas! 😄

Had a few questions below 🙂

.github/workflows/tests.yaml

distributed/tests/test_client.py

distributed/profile.py

distributed/tests/test_client.py

In 3.10, the _internal_ `_loop` attribute of several asyncio classes has been removed.

by interacting with the ac.condition we bind it to the loop on py3.10

ERROR asyncio:base_events.py:1729 Exception in callback BaseAsyncIOLoop._handle_events(18, 1) handle: <Handle BaseAsyncIOLoop._handle_events(18, 1)> Traceback (most recent call last): File "/home/graingert/miniconda3/envs/dask-distributed-310/lib/python3.10/asyncio/events.py", line 80, in _run self._context.run(self._callback, *self._args) File "/home/graingert/miniconda3/envs/dask-distributed-310/lib/python3.10/site-packages/tornado/platform/asyncio.py", line 189, in _handle_events handler_func(fileobj, events) File "/home/graingert/miniconda3/envs/dask-distributed-310/lib/python3.10/site-packages/tornado/netutil.py", line 276, in accept_handler callback(connection, address) File "/home/graingert/miniconda3/envs/dask-distributed-310/lib/python3.10/site-packages/tornado/tcpserver.py", line 288, in _handle_connection connection = ssl_wrap_socket( File "/home/graingert/miniconda3/envs/dask-distributed-310/lib/python3.10/site-packages/tornado/netutil.py", line 615, in ssl_wrap_socket return context.wrap_socket(socket, server_hostname=server_hostname, **kwargs) File "/home/graingert/miniconda3/envs/dask-distributed-310/lib/python3.10/ssl.py", line 512, in wrap_socket return self.sslsocket_class._create( File "/home/graingert/miniconda3/envs/dask-distributed-310/lib/python3.10/ssl.py", line 1061, in _create self._sslobj = self._context._wrap_socket( ssl.SSLError: Cannot create a server socket with a PROTOCOL_TLS_CLIENT context (_ssl.c:794)

…her_semaphore_loop skip reason

distributed/utils_test.py

jakirkham · 2022-03-24T21:59:42Z

Seeing 2 test failures. One with test_restart on Linux Python 3.9 and one with test_restart_sync_no_center on Windows Python 3.9.

The former seems to be due to a missing file and the latter due to contention around a file.

Not seeing other issues around these tests on the issue tracker (so don't know if they are known to be flaky). Though wouldn't be surprised if these are due to sporadic test failures. Did see PR ( #5994 ) modernizing test_restart_sync_no_center (maybe something similar is needed for test_restart?).

Idk if these should be blockers, but ok going ahead without them (if others agree) to get Python 3.10 included.

distributed/tests/test_profile.py

mrocklin · 2022-03-25T10:55:22Z

test_restart_sync_no_center can be nuked. That's happening at #5994 . Taking a look at the test_restart report now.

mrocklin · 2022-03-25T11:07:00Z

For test_restart I think it's not related to this PR. That being said, I think that folks in this PR might be able to help diagnose what's going on. I think that that failing test (and others) might boil down to this question: #6001

sjperkins

Some observations:

Pytorch is no longer tested on Python 3.10.
The default SSL Context purpose has changed from SERVER_AUTH to CLIENT_AUTH
Do the docs build OK?

continuous_integration/environment-3.10.yaml

distributed/tests/test_profile.py

distributed/node.py

jakirkham · 2022-03-25T16:29:28Z

For test_restart I think it's not related to this PR.

Yeah agree it is probably unrelated.

That being said, I think that folks in this PR might be able to help diagnose what's going on. I think that that failing test (and others) might boil down to this question: #6001

Maybe just wanted to avoid holding up Python 3.10 addition over this test failure, but if others want to debug and fix it that is fine as well

mrocklin · 2022-03-25T16:35:02Z

I don't think that this PR should block on that test

…

On Fri, Mar 25, 2022 at 11:29 AM jakirkham ***@***.***> wrote: For test_restart I think it's not related to this PR. Yeah agree it is probably unrelated. That being said, I think that folks in this PR might be able to help diagnose what's going on. I think that that failing test (and others) might boil down to this question: #6001 <#6001> Maybe just wanted to avoid holding up Python 3.10 addition over this test failure, but if others want to debug and fix it that is fine as well — Reply to this email directly, view it on GitHub <#5952 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACKZTBY5UE5NUDM4PI6BLDVBXSXFANCNFSM5Q4TV4TQ> . You are receiving this because you commented.Message ID: ***@***.***>

jakirkham · 2022-03-25T16:50:59Z

@fjetter did you have more thoughts here or is this good from your perspective? 🙂

fjetter · 2022-03-28T09:39:34Z

The environment.yaml still includes too many dependencies. All the "only tested here" dependencies should be removed since they are tested in the py3.9 build

graingert

remove "# Only tested here" deps

continuous_integration/environment-3.10.yaml

fjetter

I'm also wondering why we are testing

paramiko
scikit-learn/scipy
prometheus
netcdf4
h5py
...

in this build but we can clean up these build files in a follow up PR.

continuous_integration/environment-3.10.yaml

Co-authored-by: Florian Jetter <fjetter@users.noreply.github.com>

graingert force-pushed the python-3.10 branch from 972921f to 8965729 Compare March 16, 2022 19:49

jakirkham reviewed Mar 16, 2022

View reviewed changes

.github/workflows/tests.yaml Show resolved Hide resolved

distributed/tests/test_client.py Outdated Show resolved Hide resolved

distributed/tests/test_client.py Outdated Show resolved Hide resolved

distributed/tests/test_client.py Show resolved Hide resolved

graingert commented Mar 17, 2022

View reviewed changes

distributed/profile.py Outdated Show resolved Hide resolved

graingert force-pushed the python-3.10 branch 2 times, most recently from cb7548b to b3a5b38 Compare March 18, 2022 14:35

graingert commented Mar 18, 2022

View reviewed changes

distributed/profile.py Show resolved Hide resolved

graingert force-pushed the python-3.10 branch 5 times, most recently from 2423e95 to 090f5ee Compare March 22, 2022 14:20

graingert commented Mar 22, 2022

View reviewed changes

distributed/tests/test_client.py Show resolved Hide resolved

graingert marked this pull request as ready for review March 22, 2022 16:20

graingert requested a review from jakirkham March 22, 2022 16:20

graingert force-pushed the python-3.10 branch from 3d02874 to 7754b00 Compare March 23, 2022 10:19

jrbourbeau and others added 10 commits March 23, 2022 13:49

Add Python 3.10 build to CI

9c6a4c9

Stacktrace workaround

ea5557c

remove install stacktrace step

f449687

Remove explicit loop from asyncio tests

af3a2c0

In 3.10, the _internal_ `_loop` attribute of several asyncio classes has been removed.

fix test_as_completed_condition_loop on py3.10

a34b5ca

by interacting with the ac.condition we bind it to the loop on py3.10

work around frame.f_lineno being None sometimes

99ea0e1

avoid DeprecationWarning in rpc.__exit__ by using rpc.__aexit__

81f380f

Update distributed/tests/test_client.py

f6d2219

clarify test_client_connectionpool_semaphore_loop and test_client_gat…

2ca74e2

…her_semaphore_loop skip reason

graingert force-pushed the python-3.10 branch from 7754b00 to 2ca74e2 Compare March 23, 2022 13:52

fjetter requested a review from sjperkins March 24, 2022 16:18

jcrist approved these changes Mar 24, 2022

View reviewed changes

distributed/utils_test.py Show resolved Hide resolved

jcrist mentioned this pull request Mar 24, 2022

Add Python 3.10 support dask/dask#8566

Merged

3 tasks

update trove classifiers

47c5997

fjetter reviewed Mar 25, 2022

View reviewed changes

distributed/tests/test_profile.py Outdated Show resolved Hide resolved

document co_lnotab, co_lines_seq and co_code values in test_profile

d214ba8

graingert requested a review from fjetter March 25, 2022 12:07

sjperkins reviewed Mar 25, 2022

View reviewed changes

continuous_integration/environment-3.10.yaml Show resolved Hide resolved

distributed/tests/test_profile.py Show resolved Hide resolved

distributed/node.py Show resolved Hide resolved

graingert requested a review from sjperkins March 25, 2022 13:02

sjperkins approved these changes Mar 25, 2022

View reviewed changes

graingert commented Mar 28, 2022

View reviewed changes

continuous_integration/environment-3.10.yaml Outdated Show resolved Hide resolved

continuous_integration/environment-3.10.yaml Outdated Show resolved Hide resolved

continuous_integration/environment-3.10.yaml Outdated Show resolved Hide resolved

remove "Only tested here" deps from 3.10 environment

43cc4a6

fjetter reviewed Mar 28, 2022

View reviewed changes

Apply suggestions from code review

b0308ac

Co-authored-by: Florian Jetter <fjetter@users.noreply.github.com>

graingert requested a review from fjetter March 28, 2022 12:34

bnavigator mentioned this pull request Mar 28, 2022

Python 3.10 support #5460

Closed

hayesgb assigned graingert Mar 29, 2022

fjetter approved these changes Mar 30, 2022

View reviewed changes

fjetter merged commit 4c4df91 into dask:main Mar 30, 2022

graingert deleted the python-3.10 branch March 30, 2022 08:18

This was referenced Mar 30, 2022

[WIP] Python 3.10 #5778

Closed

Release 2022.04.0 dask/community#233

Closed

Python 3.10 support dask/dask#8288

Closed

mrocklin pushed a commit to mrocklin/distributed that referenced this pull request Mar 31, 2022

Python 3.10 (dask#5952)

caa852f

graingert mentioned this pull request May 25, 2022

Remove coverage pin from Python 3.10 environment #6439

Merged

2 tasks

QuLogic mentioned this pull request Nov 28, 2022

Remove explicit loop from asyncio tests #5353

Closed

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Python 3.10 #5952

Python 3.10 #5952

graingert commented Mar 16, 2022 •

edited

github-actions bot commented Mar 16, 2022 •

edited

jakirkham left a comment

jakirkham commented Mar 24, 2022

mrocklin commented Mar 25, 2022

mrocklin commented Mar 25, 2022

sjperkins left a comment

jakirkham commented Mar 25, 2022

mrocklin commented Mar 25, 2022 via email

jakirkham commented Mar 25, 2022

fjetter commented Mar 28, 2022

graingert left a comment

fjetter left a comment

Python 3.10 #5952

Python 3.10 #5952

Conversation

graingert commented Mar 16, 2022 • edited

github-actions bot commented Mar 16, 2022 • edited

Unit Test Results

jakirkham left a comment

Choose a reason for hiding this comment

jakirkham commented Mar 24, 2022

mrocklin commented Mar 25, 2022

mrocklin commented Mar 25, 2022

sjperkins left a comment

Choose a reason for hiding this comment

jakirkham commented Mar 25, 2022

mrocklin commented Mar 25, 2022 via email

jakirkham commented Mar 25, 2022

fjetter commented Mar 28, 2022

graingert left a comment

Choose a reason for hiding this comment

fjetter left a comment

Choose a reason for hiding this comment

graingert commented Mar 16, 2022 •

edited

github-actions bot commented Mar 16, 2022 •

edited