-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
PYTHON-4533 Convert essential tests to async for Async PyMongo beta release #1724
Changes from all commits
8c96646
17a6761
ad48086
73d3b19
46fbc97
d588167
427e4b5
9c1bf99
985c25c
665f730
0172ff3
a0652ce
c00c15b
2850ed7
6457f6c
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -138,18 +138,21 @@ | |
import collections | ||
import time | ||
import uuid | ||
from asyncio import iscoroutinefunction | ||
from collections.abc import Mapping as _Mapping | ||
from typing import ( | ||
TYPE_CHECKING, | ||
Any, | ||
AsyncContextManager, | ||
Callable, | ||
Coroutine, | ||
Mapping, | ||
MutableMapping, | ||
NoReturn, | ||
Optional, | ||
Type, | ||
TypeVar, | ||
Union, | ||
) | ||
|
||
from bson.binary import Binary | ||
|
@@ -598,7 +601,10 @@ def _inherit_option(self, name: str, val: _T) -> _T: | |
|
||
async def with_transaction( | ||
self, | ||
callback: Callable[[AsyncClientSession], _T], | ||
callback: Union[ | ||
Callable[[AsyncClientSession], Coroutine[Any, Any, _T]], | ||
Callable[[AsyncClientSession], _T], | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why are there two callables here? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. An artifact of the reasoning below, removed. |
||
], | ||
read_concern: Optional[ReadConcern] = None, | ||
write_concern: Optional[WriteConcern] = None, | ||
read_preference: Optional[_ServerMode] = None, | ||
|
@@ -693,7 +699,10 @@ async def callback(session, custom_arg, custom_kwarg=None): | |
read_concern, write_concern, read_preference, max_commit_time_ms | ||
) | ||
try: | ||
ret = callback(self) | ||
if not _IS_SYNC and iscoroutinefunction(callback): | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why check IS_SYNC or iscoroutinefunction here? Shouldn't it just be: ret = await callback(self) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I was trying to be more lenient and allow both async and sync callbacks, but it's better to require all asynchronous callbacks for the Async API. |
||
ret = await callback(self) # type: ignore[assignment] | ||
else: | ||
ret = callback(self) # type: ignore[assignment] | ||
except Exception as exc: | ||
if self.in_transaction: | ||
await self.abort_transaction() | ||
|
@@ -708,7 +717,7 @@ async def callback(session, custom_arg, custom_kwarg=None): | |
|
||
if not self.in_transaction: | ||
# Assume callback intentionally ended the transaction. | ||
return ret | ||
return ret # type: ignore[return-value] | ||
|
||
while True: | ||
try: | ||
|
@@ -730,7 +739,7 @@ async def callback(session, custom_arg, custom_kwarg=None): | |
raise | ||
|
||
# Commit succeeded. | ||
return ret | ||
return ret # type: ignore[return-value] | ||
|
||
async def start_transaction( | ||
self, | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -861,6 +861,10 @@ def __init__( | |
# This will be used later if we fork. | ||
AsyncMongoClient._clients[self._topology._topology_id] = self | ||
|
||
async def connect(self) -> None: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. "Connect" might not be the best name here since we're not actually connecting. I know we have the "connect=True" argument but I can't help but feel like "open()" would be more appropriate. If a user actually wanted to connect I would expect them to run something like "db.command('ping')". What do you think? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It feels a bit odd to me for the synchronous API to have a There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. In that case I think the docstring just needs to explain the concept more clearly. |
||
"""Explicitly connect asynchronously.""" | ||
await self._get_topology() | ||
|
||
def _init_background(self, old_pid: Optional[int] = None) -> None: | ||
self._topology = Topology(self._topology_settings) | ||
# Seed the topology with the old one's pid so we can detect clients | ||
|
@@ -1360,7 +1364,7 @@ async def __aexit__(self, exc_type: Any, exc_val: Any, exc_tb: Any) -> None: | |
__iter__ = None | ||
|
||
def __next__(self) -> NoReturn: | ||
raise TypeError("'MongoClient' object is not iterable") | ||
raise TypeError("'AsyncMongoClient' object is not iterable") | ||
|
||
next = __next__ | ||
|
||
|
@@ -1517,6 +1521,9 @@ async def close(self) -> None: | |
# TODO: PYTHON-1921 Encrypted MongoClients cannot be re-opened. | ||
await self._encrypter.close() | ||
|
||
async def aclose(self) -> None: | ||
await self.close() | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Public API should have a docstring. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why do we need aclose() when we already have close()? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. In that case should we remove "close()"? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think that's better, yeah. |
||
|
||
async def _get_topology(self) -> Topology: | ||
"""Get the internal :class:`~pymongo.topology.Topology` object. | ||
|
||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -138,6 +138,7 @@ | |
import collections | ||
import time | ||
import uuid | ||
from asyncio import iscoroutinefunction | ||
from collections.abc import Mapping as _Mapping | ||
from typing import ( | ||
TYPE_CHECKING, | ||
|
@@ -150,6 +151,7 @@ | |
Optional, | ||
Type, | ||
TypeVar, | ||
Union, | ||
) | ||
|
||
from bson.binary import Binary | ||
|
@@ -598,7 +600,10 @@ def _inherit_option(self, name: str, val: _T) -> _T: | |
|
||
def with_transaction( | ||
self, | ||
callback: Callable[[ClientSession], _T], | ||
callback: Union[ | ||
Callable[[ClientSession], _T], | ||
Callable[[ClientSession], _T], | ||
], | ||
Comment on lines
+603
to
+606
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It looks like the type definition is the same twice here. Is that needed? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. No, this is an artifact of supporting both kinds of callbacks in the async API. Will be resolved in the separate PR for |
||
read_concern: Optional[ReadConcern] = None, | ||
write_concern: Optional[WriteConcern] = None, | ||
read_preference: Optional[_ServerMode] = None, | ||
|
@@ -691,7 +696,10 @@ async def callback(session, custom_arg, custom_kwarg=None): | |
while True: | ||
self.start_transaction(read_concern, write_concern, read_preference, max_commit_time_ms) | ||
try: | ||
ret = callback(self) | ||
if not _IS_SYNC and iscoroutinefunction(callback): | ||
ret = callback(self) # type: ignore[assignment] | ||
blink1073 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
else: | ||
ret = callback(self) # type: ignore[assignment] | ||
except Exception as exc: | ||
if self.in_transaction: | ||
self.abort_transaction() | ||
|
@@ -706,7 +714,7 @@ async def callback(session, custom_arg, custom_kwarg=None): | |
|
||
if not self.in_transaction: | ||
# Assume callback intentionally ended the transaction. | ||
return ret | ||
return ret # type: ignore[return-value] | ||
|
||
while True: | ||
try: | ||
|
@@ -728,7 +736,7 @@ async def callback(session, custom_arg, custom_kwarg=None): | |
raise | ||
|
||
# Commit succeeded. | ||
return ret | ||
return ret # type: ignore[return-value] | ||
|
||
def start_transaction( | ||
self, | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -860,6 +860,10 @@ def __init__( | |
# This will be used later if we fork. | ||
MongoClient._clients[self._topology._topology_id] = self | ||
|
||
def connect(self) -> None: | ||
"""Explicitly connect synchronously.""" | ||
self._get_topology() | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We should avoid adding connect() and aclose() to the sync client. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What if we rename them to be private for the synchronous client? Like There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. How about something like this?: _IS_SYNC = False
class AsyncMongoClient:
if not _IS_SYNC:
async def aclose(self):
pass There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. A better solution for
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. aclose -> close SGTM. For aconnect() I would prefer the approach I outlined above so that the method is never added to the sync client. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Actually. renaming aconnect -> _aconnect is fine too. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The synchro process will still convert the There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm good with any approach as long as aconnect() or connect() isn't a public method on MongoClient. |
||
|
||
def _init_background(self, old_pid: Optional[int] = None) -> None: | ||
self._topology = Topology(self._topology_settings) | ||
# Seed the topology with the old one's pid so we can detect clients | ||
|
@@ -1512,6 +1516,9 @@ def close(self) -> None: | |
# TODO: PYTHON-1921 Encrypted MongoClients cannot be re-opened. | ||
self._encrypter.close() | ||
|
||
def aclose(self) -> None: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Same as the comment from @ShaneHarvey before. We can remove close |
||
self.close() | ||
|
||
def _get_topology(self) -> Topology: | ||
"""Get the internal :class:`~pymongo.topology.Topology` object. | ||
|
||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we use Coroutine here or Awaitable? Which is a better fit?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We've used
Coroutine
consistently so far,Awaitable
is a less-specific type hint that allows several other objects besides asynchronous functions.