Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 9 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,15 @@
- [I/O Multiplex (I/O多路复用) (Client)](https://leven-cn.github.io/python-cookbook/cookbook/core/net/io_multiplex_client)
- [Pack/Unpack Binary Data - `struct`](https://leven-cn.github.io/python-cookbook/cookbook/core/net/struct)

### Asynchronous I/O (异步 I/O)

- [Coroutine (协程)](https://leven-cn.github.io/python-cookbook/cookbook/core/asyncio/coroutine)
- [Run coroutines concurrently (并发执行协程)](https://leven-cn.github.io/python-cookbook/cookbook/core/asyncio/coroutine_concurrent)
- [Scheduled Tasks (调度任务)](https://leven-cn.github.io/python-cookbook/cookbook/core/asyncio/schedule)
- [Wait](https://leven-cn.github.io/python-cookbook/cookbook/core/asyncio/wait)
- [TCP Server](https://leven-cn.github.io/python-cookbook/cookbook/core/asyncio/tcp_server)
- [TCP Client](https://leven-cn.github.io/python-cookbook/cookbook/core/asyncio/tcp_client)

## Build (构建)

### Command-Line Arguments Parser
Expand Down Expand Up @@ -207,20 +216,13 @@
- [IPC - Socket Pair](https://leven-cn.github.io/python-cookbook/recipes/core/ipc_socketpair)
- [IPC - UNIX Domain Socket (UDS, UNIX 域套接字) Server and Client](https://leven-cn.github.io/python-cookbook/recipes/core/ipc_unix_domain_socket)
- Asynchronous I/O (异步 I/O)
- [Coroutine (协程)](https://leven-cn.github.io/python-cookbook/recipes/core/asyncio_coroutine)
- [Chain coroutines (串链协程)](https://leven-cn.github.io/python-cookbook/recipes/core/asyncio_coroutine_chain)
- [Run coroutines Concurrently (并发执行协程)](https://leven-cn.github.io/python-cookbook/recipes/core/asyncio_coroutine_chain)
- [Scheduled Tasks (调度任务)](https://leven-cn.github.io/python-cookbook/recipes/core/asyncio_schedule)
- [Wait](https://leven-cn.github.io/python-cookbook/recipes/core/asyncio_wait)
- [Nonblocking Main Thread](https://leven-cn.github.io/python-cookbook/recipes/core/asyncio_nonblocking)
- [Synchronization Primitives: Lock](https://leven-cn.github.io/python-cookbook/recipes/core/asyncio_synchronization_lock)
- [Synchronization Primitives: Event](https://leven-cn.github.io/python-cookbook/recipes/core/asyncio_synchronization_event)
- [Synchronization Primitives: Condition](https://leven-cn.github.io/python-cookbook/recipes/core/asyncio_synchronization_condition)
- [Synchronization Primitives: Semapore (信号量)](https://leven-cn.github.io/python-cookbook/recipes/core/asyncio_synchronization_semapore)
- [Queue (队列)](https://leven-cn.github.io/python-cookbook/recipes/core/asyncio_queue)
- [TCP Server (High-Level APIs)](https://leven-cn.github.io/python-cookbook/recipes/core/tcp_server_asyncio_high_api)
- [TCP Server (Low-Level APIs)](https://leven-cn.github.io/python-cookbook/recipes/core/tcp_server_asyncio_low_api)
- [TCP Client - (High-Level APIs)](https://leven-cn.github.io/python-cookbook/recipes/core/tcp_client_asyncio_high_api)
- [TCP Client - (Low-Level APIs)](https://leven-cn.github.io/python-cookbook/recipes/core/tcp_client_asyncio_low_api)
- [UDP Server](https://leven-cn.github.io/python-cookbook/recipes/core/udp_server_asyncio)
- [UDP Client](https://leven-cn.github.io/python-cookbook/recipes/core/udp_client_asyncio)
Expand Down
Original file line number Diff line number Diff line change
@@ -1,11 +1,8 @@
# Asynchronous I/O - Create coroutine
# Asynchronous I/O - Coroutine

## Solution
## Recipes

```python
"""Asynchronous I/O - coroutine.
"""

import asyncio
import logging
import sys
Expand All @@ -15,9 +12,21 @@ logging.basicConfig(
)


async def coroutine(arg: int):
async def coroutine(arg: int) -> tuple[int, str, str]:
"""Coroutine demo."""
logging.debug(f'run coroutine: {arg}')
return arg + 1

result_1: str = await task(1, 1.0)
result_2: str = await task(2, 1.5)

return arg + 1, result_1, result_2


async def task(num: int, wait: float) -> str:
"""Coroutine task demo."""
logging.debug(f'run task {num}, wait {wait} seconds')
await asyncio.sleep(wait)
return f'task {num} result'


coro = coroutine(1)
Expand All @@ -26,7 +35,9 @@ if sys.version_info >= (3, 7): # Python 3.7+
logging.debug(f'result: {result}')
else:
# Low-level APIs
loop = asyncio.get_event_loop()
# `get_running_loop()` has been added since Python 3.7.
# `get_event_loop()` has been deprecated since Python 3.10.
loop = asyncio.get_running_loop()
try:
logging.debug('starting event loop')
result = loop.run_until_complete(coro)
Expand All @@ -35,8 +46,6 @@ else:
loop.close()
```

See [source code](https://github.com/leven-cn/python-cookbook/blob/main/examples/core/asyncio_coroutine.py)

## References

- [Python - `asyncio` module](https://docs.python.org/3/library/asyncio.html)
Expand Down
Original file line number Diff line number Diff line change
@@ -1,11 +1,8 @@
# Asynchronous I/O - Run coroutines Concurrently

## Solution
## Recipes

```python
"""Asynchronous I/O - Run coroutines concurrently.
"""

import asyncio
import logging

Expand Down Expand Up @@ -97,8 +94,6 @@ result = asyncio.run(main()) # Python 3.7+
logging.debug(f'result: {result}')
```

See [source code](https://github.com/leven-cn/python-cookbook/blob/main/examples/core/asyncio_coroutine_concurrent.py)

## References

- [Python - `asyncio` module](https://docs.python.org/3/library/asyncio.html)
Expand Down
Original file line number Diff line number Diff line change
@@ -1,11 +1,8 @@
# Asynchronous I/O - Scheduled Tasks

## Solution
## Recipes

```python
"""Asynchronous I/O - Scheduled Tasks.
"""

import asyncio
import logging
import time
Expand Down Expand Up @@ -70,8 +67,6 @@ result = asyncio.run(main())
logging.debug(f'result: {result}')
```

See [source code](https://github.com/leven-cn/python-cookbook/blob/main/examples/core/asyncio_schedule.py)

## References

- [Python - `asyncio` module](https://docs.python.org/3/library/asyncio.html)
Expand Down
Original file line number Diff line number Diff line change
@@ -1,11 +1,8 @@
# TCP Client - Asynchronous I/O (High-Level APIs)
# TCP Client - Asynchronous I/O

## Solution

```python
"""TCP Client - Asynchronous I/O (High-Level APIs).
"""

import asyncio
import logging

Expand All @@ -32,8 +29,6 @@ async def tcp_echo_client(data: bytes):
asyncio.run(tcp_echo_client(b'Hello World!')) # Python 3.7+
```

See [source code](https://github.com/leven-cn/python-cookbook/blob/main/examples/core/tcp_client_asyncio_high_api.py)

## References

- [Python - `asyncio` module](https://docs.python.org/3/library/asyncio.html)
Expand Down
134 changes: 134 additions & 0 deletions cookbook/core/asyncio/tcp_server.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
# TCP Server - Asynchronous I/O

## Recipes

```python
import asyncio
import logging
import socket
import sys

logging.basicConfig(
level=logging.DEBUG, style='{', format='[{threadName} ({thread})] {message}'
)

recv_bufsize: int | None = None
send_bufsize: int | None = None


async def handle_echo(
reader: asyncio.StreamReader, writer: asyncio.StreamWriter
) -> None:
client_address = writer.get_extra_info('peername')
logging.debug(f'connected from {client_address}')

sock: socket.socket = writer.get_extra_info('socket')
assert sock.type is socket.SOCK_STREAM
assert sock.getpeername() == client_address
assert sock.getsockname() == writer.get_extra_info('sockname') # server address
assert sock.gettimeout() == 0
assert bool(sock.getsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY))
assert bool(sock.getsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR))
assert bool(sock.getsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT))
if hasattr(socket, 'TCP_QUICKACK'):
assert sys.platform == 'linux'
assert bool(sock.getsockopt(socket.IPPROTO_TCP, socket.TCP_QUICKACK))
assert bool(sock.getsockopt(socket.SOL_SOCKET, socket.SO_KEEPALIVE))
if sys.platform == 'linux': # Linux 2.4+
assert sock.getsockopt(socket.IPPROTO_TCP, socket.TCP_KEEPIDLE) == 1800
elif hasattr(socket, 'TCP_KEEPALIVE'): # macOS and Python 3.10+
assert sys.platform == 'darwin' and sys.version_info >= (3, 10)
assert not sock.getsockopt(socket.IPPROTO_TCP, socket.TCP_KEEPALIVE) == 1800
sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_KEEPALIVE, 1800)
assert sock.getsockopt(socket.IPPROTO_TCP, socket.TCP_KEEPCNT) == 5
assert sock.getsockopt(socket.IPPROTO_TCP, socket.TCP_KEEPINTVL) == 15

# logging.debug(dir(sock))

# Recv
data = await reader.read(100)
logging.debug(f'recv: {data!r}')

# Send
writer.write(data)
await writer.drain()
logging.debug(f'sent: {data!r}')

writer.close()


async def tcp_echo_server(
host: str,
port: int,
*,
accept_queue_size: int = socket.SOMAXCONN,
keep_alive_idle: int | None = None,
keep_alive_cnt: int | None = None,
keep_alive_intvl: int | None = None,
start_serving: bool = False,
) -> None:
# Low-level APIs: loop.create_server()
server = await asyncio.start_server(
handle_echo,
host,
port,
reuse_address=True,
reuse_port=True,
backlog=accept_queue_size,
start_serving=start_serving,
)

# Prior to Python 3.7 `asyncio.Server.sockets` used to return an internal list of
# server sockets directly.
# In 3.7 a copy of that list is returned.
server_addressess = ', '.join(str(sock.getsockname()) for sock in server.sockets)
logging.debug(f'Serving on {server_addressess}')

for sock in server.sockets:
# Issue: ? The socket option `TCP_NODELAY` is set by default in Python 3.6+
assert not bool(sock.getsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY))
sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)

# Keep-Alive
if (
keep_alive_cnt is not None
and keep_alive_intvl is not None
and keep_alive_idle is not None
):
sock.setsockopt(socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1)
if sys.platform == 'linux': # Linux 2.4+
sock.setsockopt(
socket.IPPROTO_TCP, socket.TCP_KEEPIDLE, keep_alive_idle
)
elif hasattr(socket, 'TCP_KEEPALIVE'):
assert sys.platform == 'darwin' and sys.version_info >= (3, 10)
sock.setsockopt(
socket.IPPROTO_TCP, socket.TCP_KEEPALIVE, keep_alive_idle
)
sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_KEEPCNT, keep_alive_cnt)
sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_KEEPINTVL, keep_alive_intvl)

# QUICK ACK
if hasattr(socket, 'TCP_QUICKACK'):
assert sys.platform == 'linux'
assert bool(sock.getsockopt(socket.IPPROTO_TCP, socket.TCP_QUICKACK))

# `asyncio.Server` object is an asynchronous context manager since Python 3.7.
if not start_serving:
async with server:
await server.serve_forever()


asyncio.run(
tcp_echo_server(
'127.0.0.1', 8888, keep_alive_idle=1800, keep_alive_cnt=5, keep_alive_intvl=15
)
) # Python 3.7+
```

## References

- [Python - `asyncio` module](https://docs.python.org/3/library/asyncio.html)
- [Python - `socket` module](https://docs.python.org/3/library/socket.html)
- [PEP 3156 – Asynchronous IO Support Rebooted: the "asyncio" Module](https://peps.python.org/pep-3156/)
- [PEP 3151 – Reworking the OS and IO exception hierarchy](https://peps.python.org/pep-3151/)
Original file line number Diff line number Diff line change
@@ -1,11 +1,8 @@
# Asynchronous I/O - Wait

## Solution
## Recipes

```python
"""Asynchronous I/O - Wait.
"""

import asyncio
import logging

Expand Down Expand Up @@ -50,8 +47,6 @@ async def main():
asyncio.run(main()) # Python 3.7+
```

See [source code](https://github.com/leven-cn/python-cookbook/blob/main/examples/core/asyncio_wait.py)

## References

- [Python - `asyncio` module](https://docs.python.org/3/library/asyncio.html)
Expand Down
15 changes: 13 additions & 2 deletions examples/core/asyncio_coroutine.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,21 @@
)


async def coroutine(arg: int) -> int:
async def coroutine(arg: int) -> tuple[int, str, str]:
"""Coroutine demo."""
logging.debug(f'run coroutine: {arg}')
return arg + 1

result_1: str = await task(1, 1.0)
result_2: str = await task(2, 1.5)

return arg + 1, result_1, result_2


async def task(num: int, wait: float) -> str:
"""Coroutine task demo."""
logging.debug(f'run task {num}, wait {wait} seconds')
await asyncio.sleep(wait)
return f'task {num} result'


coro = coroutine(1)
Expand Down
31 changes: 0 additions & 31 deletions examples/core/asyncio_coroutine_chain.py

This file was deleted.

Loading