Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix consumer fetcher for python3.9 #672

Merged
merged 1 commit into from
Oct 27, 2020
Merged

Fix consumer fetcher for python3.9 #672

merged 1 commit into from
Oct 27, 2020

Conversation

dutradda
Copy link
Contributor

@dutradda dutradda commented Oct 24, 2020 •
edited
Loading

Changes

Fixes #

  • Fix consumer fetcher for python3.9

From python3.9 the asyncio.wait function iterates over futures two times.
The consumer fetcher gives an iterator to this function, so the second iteration is empty, that causes the error Set of Futures is empty.

python3.9 asyncio code:

    if any(coroutines.iscoroutine(f) for f in set(fs)):
        warnings.warn("The explicit passing of coroutine objects to "
                      "asyncio.wait() is deprecated since Python 3.8, and "
                      "scheduled for removal in Python 3.11.",
                      DeprecationWarning, stacklevel=2)

    fs = {ensure_future(f, loop=loop) for f in set(fs)}

Checklist

  • I think the code is well written
  • Unit tests for the changes exist
  • Documentation reflects the changes
  • Add a new news fragment into the CHANGES folder
    • name it <issue_id>.<type> (e.g. 588.bugfix)
    • if you don't have an issue_id change it to the pr id after creating the PR
    • ensure type is one of the following:
      • .feature: Signifying a new feature.
      • .bugfix: Signifying a bug fix.
      • .doc: Signifying a documentation improvement.
      • .removal: Signifying a deprecation or removal of public API.
      • .misc: A ticket has been closed, but it is not of interest to users.
    • Make sure to use full sentences with correct case and punctuation, for example: Fix issue with non-ascii contents in doctest text files.

@codecov
Copy link

codecov bot commented Oct 24, 2020 •
edited
Loading

Codecov Report

Merging #672 into master will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##           master     #672   +/-   ##
=======================================
  Coverage   97.95%   97.95%           
=======================================
  Files          29       29           
  Lines        5192     5192           
=======================================
  Hits         5086     5086           
  Misses        106      106
Flag Coverage Δ
#cext 87.63% <ø> (-0.04%) ⬇️
#integration 97.91% <ø> (ø)
#purepy 97.47% <ø> (+0.03%) ⬆️
#unit 51.48% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
aiokafka/consumer/fetcher.py 97.17% <ø> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 48f5df5...bc60559. Read the comment docs.

@dutradda dutradda mentioned this pull request Oct 24, 2020
Merged
@tvoinarovskyi
Copy link
Member

Wow, did not think of this problem. Thanks for looking into it!

@tvoinarovskyi tvoinarovskyi merged commit 1d2e02e into aio-libs:master Oct 27, 2020
jstasiak added a commit to jstasiak/cpython that referenced this pull request Nov 1, 2020
@jstasiak
bpo-42230: Improve asyncio documentation regarding accepting sets vs …
d26c968 
…iterables

People call wait() and as_completed() with various non-set iterables,
a list should be the most common but there are others as well[1].

Considering typeshed also documents wait()[2] and as_completed()[3]
as accepting arbitrary iterables I think it's a good idea to document
the status quo better.

[1] aio-libs/aiokafka#672
[2] https://github.com/python/typeshed/blob/620989bac572f30349b95590ebe81a73ce0fe862/stdlib/3/asyncio/tasks.pyi#L161
[3] https://github.com/python/typeshed/blob/620989bac572f30349b95590ebe81a73ce0fe862/stdlib/3/asyncio/tasks.pyi#L40
@jstasiak jstasiak mentioned this pull request Nov 1, 2020
Merged
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Nov 2, 2020
@jstasiak @miss-islington
bpo-42230: Improve asyncio documentation regarding accepting sets vs …
e6fe7f5 
…iterables (pythonGH-23073)

People call wait() and as_completed() with various non-set iterables,
a list should be the most common but there are others as well[1].

Considering typeshed also documents wait()[2] and as_completed()[3]
as accepting arbitrary iterables I think it's a good idea to document
the status quo better.

[1] aio-libs/aiokafka#672
[2] https://github.com/python/typeshed/blob/620989bac572f30349b95590ebe81a73ce0fe862/stdlib/3/asyncio/tasks.pyiGH-L161
[3] https://github.com/python/typeshed/blob/620989bac572f30349b95590ebe81a73ce0fe862/stdlib/3/asyncio/tasks.pyiGH-L40
(cherry picked from commit 3d86d09)

Co-authored-by: Jakub Stasiak <jakub@stasiak.at>
asvetlov pushed a commit to python/cpython that referenced this pull request Nov 2, 2020
@jstasiak
bpo-42230: Improve asyncio documentation regarding accepting sets vs …
3d86d09 
…iterables (GH-23073)

People call wait() and as_completed() with various non-set iterables,
a list should be the most common but there are others as well[1].

Considering typeshed also documents wait()[2] and as_completed()[3]
as accepting arbitrary iterables I think it's a good idea to document
the status quo better.

[1] aio-libs/aiokafka#672
[2] https://github.com/python/typeshed/blob/620989bac572f30349b95590ebe81a73ce0fe862/stdlib/3/asyncio/tasks.pyi#L161
[3] https://github.com/python/typeshed/blob/620989bac572f30349b95590ebe81a73ce0fe862/stdlib/3/asyncio/tasks.pyi#L40
@miss-islington miss-islington mentioned this pull request Nov 2, 2020
Merged
@jstasiak jstasiak mentioned this pull request Nov 2, 2020
Merged
jstasiak added a commit to jstasiak/cpython that referenced this pull request Nov 2, 2020
@jstasiak
[3.8] bpo-42230: Improve asyncio documentation regarding accepting se…
ba21031 
…ts vs iterables (pythonGH-23073)

People call wait() and as_completed() with various non-set iterables,
a list should be the most common but there are others as well[1].

Considering typeshed also documents wait()[2] and as_completed()[3]
as accepting arbitrary iterables I think it's a good idea to document
the status quo better.

[1] aio-libs/aiokafka#672
[2] https://github.com/python/typeshed/blob/620989bac572f30349b95590ebe81a73ce0fe862/stdlib/3/asyncio/tasks.pyiGH-L161
[3] https://github.com/python/typeshed/blob/620989bac572f30349b95590ebe81a73ce0fe862/stdlib/3/asyncio/tasks.pyiGH-L40.
(cherry picked from commit 3d86d09)

Co-authored-by: Jakub Stasiak <jakub@stasiak.at>
miss-islington added a commit to python/cpython that referenced this pull request Nov 2, 2020
@miss-islington @jstasiak
bpo-42230: Improve asyncio documentation regarding accepting sets vs …
ff852aa 
…iterables (GH-23073)

People call wait() and as_completed() with various non-set iterables,
a list should be the most common but there are others as well[1].

Considering typeshed also documents wait()[2] and as_completed()[3]
as accepting arbitrary iterables I think it's a good idea to document
the status quo better.

[1] aio-libs/aiokafka#672
[2] https://github.com/python/typeshed/blob/620989bac572f30349b95590ebe81a73ce0fe862/stdlib/3/asyncio/tasks.pyiGH-L161
[3] https://github.com/python/typeshed/blob/620989bac572f30349b95590ebe81a73ce0fe862/stdlib/3/asyncio/tasks.pyiGH-L40
(cherry picked from commit 3d86d09)

Co-authored-by: Jakub Stasiak <jakub@stasiak.at>
asvetlov pushed a commit to python/cpython that referenced this pull request Nov 2, 2020
@jstasiak
[3.8] bpo-42230: Improve asyncio documentation regarding accepting se…
ad37c66 
…ts vs iterables (GH-23073) (GH-23105)

People call wait() and as_completed() with various non-set iterables,
a list should be the most common but there are others as well[1].

Considering typeshed also documents wait()[2] and as_completed()[3]
as accepting arbitrary iterables I think it's a good idea to document
the status quo better.

[1] aio-libs/aiokafka#672
[2] https://github.com/python/typeshed/blob/620989bac572f30349b95590ebe81a73ce0fe862/stdlib/3/asyncio/tasks.pyiGH-L161
[3] https://github.com/python/typeshed/blob/620989bac572f30349b95590ebe81a73ce0fe862/stdlib/3/asyncio/tasks.pyiGH-L40.
(cherry picked from commit 3d86d09)

Co-authored-by: Jakub Stasiak <jakub@stasiak.at>
miss-islington pushed a commit to python/cpython that referenced this pull request Nov 10, 2020
@dutradda
bpo-42140: Improve asyncio.wait function (GH-22938)
7e5ef0a 
# Improve asyncio.wait function

The original code creates the futures set two times.
We can create this set before, avoiding the second creation.

This new behaviour [breaks the aiokafka library](aio-libs/aiokafka#672), because it gives an iterator to that function, so the second iteration become empty.

Automerge-Triggered-By: GH:1st1
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Nov 10, 2020
@dutradda @miss-islington
bpo-42140: Improve asyncio.wait function (pythonGH-22938)
a3e4788 
GH- Improve asyncio.wait function

The original code creates the futures set two times.
We can create this set before, avoiding the second creation.

This new behaviour [breaks the aiokafka library](aio-libs/aiokafka#672), because it gives an iterator to that function, so the second iteration become empty.

Automerge-Triggered-By: GH:1st1
(cherry picked from commit 7e5ef0a)

Co-authored-by: Diogo Dutra <diogodutradamata@gmail.com>
@miss-islington miss-islington mentioned this pull request Nov 10, 2020
Merged
miss-islington added a commit to python/cpython that referenced this pull request Nov 10, 2020
@miss-islington @dutradda
bpo-42140: Improve asyncio.wait function (GH-22938)
33922cb 
GH- Improve asyncio.wait function

The original code creates the futures set two times.
We can create this set before, avoiding the second creation.

This new behaviour [breaks the aiokafka library](aio-libs/aiokafka#672), because it gives an iterator to that function, so the second iteration become empty.

Automerge-Triggered-By: GH:1st1
(cherry picked from commit 7e5ef0a)

Co-authored-by: Diogo Dutra <diogodutradamata@gmail.com>
adorilson pushed a commit to adorilson/cpython that referenced this pull request Mar 13, 2021
@jstasiak @adorilson
bpo-42230: Improve asyncio documentation regarding accepting sets vs …
5e651d7 
…iterables (pythonGH-23073)

People call wait() and as_completed() with various non-set iterables,
a list should be the most common but there are others as well[1].

Considering typeshed also documents wait()[2] and as_completed()[3]
as accepting arbitrary iterables I think it's a good idea to document
the status quo better.

[1] aio-libs/aiokafka#672
[2] https://github.com/python/typeshed/blob/620989bac572f30349b95590ebe81a73ce0fe862/stdlib/3/asyncio/tasks.pyi#L161
[3] https://github.com/python/typeshed/blob/620989bac572f30349b95590ebe81a73ce0fe862/stdlib/3/asyncio/tasks.pyi#L40
adorilson pushed a commit to adorilson/cpython that referenced this pull request Mar 13, 2021
@dutradda @adorilson
bpo-42140: Improve asyncio.wait function (pythonGH-22938)
cf09d38 
# Improve asyncio.wait function

The original code creates the futures set two times.
We can create this set before, avoiding the second creation.

This new behaviour [breaks the aiokafka library](aio-libs/aiokafka#672), because it gives an iterator to that function, so the second iteration become empty.

Automerge-Triggered-By: GH:1st1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@dutradda @tvoinarovskyi