Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bpo-36373: Fix deprecation warnings #15889

Merged
merged 1 commit into from Sep 11, 2019

Conversation

@asvetlov
Copy link
Contributor

commented Sep 10, 2019

@asvetlov

This comment has been minimized.

Copy link
Contributor Author

commented Sep 10, 2019

@tirkarthi that's what I mean

async def _asyncioLoopRunner(self):
queue = self._asyncioCallsQueue
async def _asyncioLoopRunner(self, fut):
self._asyncioCallsQueue = queue = asyncio.Queue()

This comment has been minimized.

Copy link
@tirkarthi

tirkarthi Sep 10, 2019

Contributor

Just curious why we moved it here. Is there a difference in constructing it in _setupAsyncioLoop itself without passing loop and having it here?

This comment has been minimized.

Copy link
@asvetlov

asvetlov Sep 10, 2019

Author Contributor

Well, it is possible to just drop the loop parameter from Queue construction because we installed the default event loop a few lines above but I did two steps further.

Instantiation on asyncio loop coupled objects from an async function is more idiomatic, it always grabs a currently executed loop (the loop which is used to run a coroutine).
It prevents things like this:

import asyncio
queue = asyncio.Queue()

asyncio.run(...)  # a code that works with queue

In the example queue variable is instantiated with the default asyncio loop but run() creates a new loop version. So the queue doesn't work and hangs eventually because a future created by one loop cannot be awaited by another one.

In Python 3.9 I have a plan to add another deprecation warning if not self._loop.is_running() for queues and locks and eventually replace get_event_loop() with get_running_loop() as a final fix.

@tirkarthi
Copy link
Contributor

left a comment

LGTM. As I understand previously we were always passing self._loop which would always be initialized with a loop object and would mean we are passing the loop value to the other callers though the initial caller had loop=None only. This looks more correct and simple to me given asyncio.Queue() is a valid call. Thanks Andrew.

@1st1

This comment has been minimized.

Copy link
Member

commented Sep 10, 2019

I thought we were going to be more subtle about this.

We have two scenarios:

  1. Somebody creates an instance of asyncio.Lock outside of a coroutine, also manually creating the loop, etc. In this case the user has to pass the lock argument.

  2. Somebody creates an instance of asyncio.Lock in a coroutine (that is, wheh asyncio.get_running_loop() returns a loop). In this case passing the loop argument is an error.

The (1) approach is how people were used to writing asyncio programs before asyncio.run() (that was the only way actually).

The (2) approach is how we want people to write asyncio programs.

There's a subtle difference between things like asyncio.gather() and asyncio.Lock. Passing loop to the former is just nonsensical. Passing loop to the latter can be a valid thing, it's the (1).

If we remove the loop parameter entirely from classes like asyncio.Lock we're making (1) impossible. I'm not sure that is what we are ready to do now.

@asvetlov thoughts?

@1st1
Copy link
Member

left a comment

Please don't merge this until we discuss it.

@bedevere-bot

This comment has been minimized.

Copy link

commented Sep 10, 2019

When you're done making the requested changes, leave the comment: I have made the requested changes; please review again.

@asvetlov

This comment has been minimized.

Copy link
Contributor Author

commented Sep 10, 2019

asyncio.run() is a nice API, everybody should use it (and people do as I see).

On Python 3.6 where asyncio.run() doesn't exist the difference is not drastic though, instead of asyncio.run(main()) you can use loop.run_until_complete(main()).
The loop preparation and especially finalization is missing here but roughly the code is the same.

Also, please read comment above. In aiohttp we forbade the creation of ClientSession outside of async function context step by step because we have a long gory list of reports when "aiohttp hangs for some reason", and the reason was misusage of a loop and an object that belongs to a different loop by accident.

@1st1
1st1 approved these changes Sep 10, 2019
@1st1

This comment has been minimized.

Copy link
Member

commented Sep 10, 2019

Alright.

@miss-islington miss-islington merged commit 7264e92 into python:master Sep 11, 2019

5 checks passed

Azure Pipelines PR #20190910.129 succeeded
Details
bedevere/issue-number Issue number 36373 found
Details
bedevere/news "skip news" label found
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@miss-islington

This comment has been minimized.

Copy link

commented Sep 11, 2019

Thanks @asvetlov for the PR 🌮🎉.. I'm working now to backport this PR to: 3.8.
🐍🍒🤖 I'm not a witch! I'm not a witch!

@asvetlov asvetlov deleted the asvetlov:fix-deprecations branch Sep 11, 2019

@bedevere-bot

This comment has been minimized.

Copy link

commented Sep 11, 2019

GH-15896 is a backport of this pull request to the 3.8 branch.

@miss-islington

This comment has been minimized.

Copy link

commented Sep 11, 2019

Thanks @asvetlov for the PR 🌮🎉.. I'm working now to backport this PR to: 3.8.
🐍🍒🤖

@miss-islington

This comment has been minimized.

Copy link

commented Sep 11, 2019

Sorry @asvetlov, I had trouble checking out the 3.8 backport branch.
Please backport using cherry_picker on command line.
cherry_picker 7264e92b718d307cc499b2f10eab7644b00f0499 3.8

@miss-islington miss-islington self-assigned this Sep 11, 2019

@bedevere-bot

This comment has been minimized.

Copy link

commented Sep 11, 2019

GH-15901 is a backport of this pull request to the 3.8 branch.

asvetlov added a commit to asvetlov/cpython that referenced this pull request Sep 11, 2019
[3.8] bpo-36373: Fix deprecation warnings (pythonGH-15889)
https://bugs.python.org/issue36373
(cherry picked from commit 7264e92)

Co-authored-by: Andrew Svetlov <andrew.svetlov@gmail.com>
asvetlov added a commit that referenced this pull request Sep 11, 2019
[3.8] bpo-36373: Fix deprecation warnings (GH-15889) (GH-15901)
https://bugs.python.org/issue36373
(cherry picked from commit 7264e92)

Co-authored-by: Andrew Svetlov <andrew.svetlov@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants
You can’t perform that action at this time.