1K Request can't be finished in a simultaneous way in aiohttp

[KevinZhou92]

Long story short

I set up a server for nlu service use Klein, and i was trying some pressure test using aiohttp. However, the maximum simultaneous seems to be under 100. If i have the number as 1000, then i will receive many ConnectionRefusedErrors.

I also tried another pressure test script using multiproccessing and it worked well, so i'm quite confusing about which part of aiohttp is restricting the performance?

aiohttp Code

import time
import asyncio, json
from aiohttp import ClientSession
from aiohttp.connector import TCPConnector, BaseConnector


payload = json.dumps({
    "model_name": ["NCC_NOSE_RS_WFSB_YM","PTP_NPN"],
    "text": "Hello",
    "version_id": "1"})
 
async def fetch(url):
    async with ClientSession() as session:
        # try:
        async with session.post(url, data=payload) as response:
            return await response.read()
        # except:   
        #     print('unable to connect to device')

async def run(loop,  r):
    url = "http://localhost:8081/predict"
    tasks = []
    for i in range(r):
        task = asyncio.ensure_future(fetch(url.format(i)))
        tasks.append(task)
    responses = await asyncio.gather(*tasks)
       # you now have all response bodies in this variable
    #print(responses)

def print_responses(result):
    print(result)


start = time.time()
loop = asyncio.get_event_loop()
future = asyncio.ensure_future(run(loop, 500))
loop.run_until_complete(future)
print(time.time() -start)

multiprocessing Code

import requests
import multiprocessing
import time
import json
import sys

def pressure_test():
    url = "http://localhost:8081/predict"
    payload = json.dumps({
        "model_name": ["NCC_NOSE_RS_WFSB_YM","PTP_NPN"],
        "text": "hello",
        "version_id": "1"})
    a = time.time()
    response = requests.post(url, data=payload)
    print(response.content)
    b = time.time()
    delta = b-a

if __name__ == "__main__": 
  pool = multiprocessing.Pool(processes=10)
  a = time.time()
  for i in range(500):
      pool.apply_async(pressure_test,())
  pool.close()
  pool.join()
  b = time.time()
  c = b-a
  print(str(c)) 

Error Log

Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/aiohttp/connector.py", line 822, in _wrap_create_connection
return await self._loop.create_connection(*args, **kwargs)
File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/base_events.py", line 778, in create_connection
raise exceptions[0]
File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/base_events.py", line 765, in create_connection
yield from self.sock_connect(sock, address)
File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/selector_events.py", line 450, in sock_connect
return (yield from fut)
File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/selector_events.py", line 480, in _sock_connect_cb
raise OSError(err, 'Connect call failed %s' % (address,))
ConnectionRefusedError: [Errno 61] Connect call failed ('::1', 8081)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "test/pressure_test.py", line 37, in
loop.run_until_complete(future)
File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/base_events.py", line 468, in run_until_complete
return future.result()
File "test/pressure_test.py", line 26, in run
responses = await asyncio.gather(*tasks)
File "test/pressure_test.py", line 15, in fetch
async with session.post(url, data=payload) as response:
File "/usr/local/lib/python3.6/site-packages/aiohttp/client.py", line 855, in aenter
self._resp = await self._coro
File "/usr/local/lib/python3.6/site-packages/aiohttp/client.py", line 370, in _request
timeout=timeout
File "/usr/local/lib/python3.6/site-packages/aiohttp/connector.py", line 445, in connect
proto = await self._create_connection(req, traces, timeout)
File "/usr/local/lib/python3.6/site-packages/aiohttp/connector.py", line 757, in _create_connection
req, traces, timeout)
File "/usr/local/lib/python3.6/site-packages/aiohttp/connector.py", line 879, in _create_direct_connection
raise last_exc
File "/usr/local/lib/python3.6/site-packages/aiohttp/connector.py", line 862, in _create_direct_connection
req=req, client_error=client_error)
File "/usr/local/lib/python3.6/site-packages/aiohttp/connector.py", line 829, in _wrap_create_connection
raise client_error(req.connection_key, exc) from exc
aiohttp.client_exceptions.ClientConnectorError: Cannot connect to host localhost:8081 ssl:None [Connect call failed ('::1', 8081)]

Your environment

aiohttp 3.4.4
python 3.6.5
MacOS 10.13.6

[asvetlov]

GitMate.io thinks possibly related issues are #2920 (AIOHttp failing after some requests), #660 (aiohttp.request hangs on some URLs), #206 (SSL issue with aiohttp.request), #2540 (Drop await aiohttp.request(...)), and #127 (verify_ssl for aiohttp.request).

[asvetlov]

Looking on the stack trace I see that on your box Python is compiled with IPv6, OS declared to support it but actual call fails.

Replacement localhost with 127.0.0.1 can help.

[KevinZhou92]

@asvetlov Thanks for replying Andrew. I changed the address to 127.0.0.1. Then i encountered a ConnectionResetError.

Error Log

Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/aiohttp/connector.py", line 822, in _wrap_create_connection
return await self._loop.create_connection(*args, **kwargs)
File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/base_events.py", line 778, in create_connection
raise exceptions[0]
File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/base_events.py", line 765, in create_connection
yield from self.sock_connect(sock, address)
File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/selector_events.py", line 450, in sock_connect
return (yield from fut)
File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/selector_events.py", line 480, in _sock_connect_cb
raise OSError(err, 'Connect call failed %s' % (address,))
ConnectionResetError: [Errno 54] Connect call failed ('127.0.0.1', 8081)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "test/pressure_test.py", line 37, in
loop.run_until_complete(future)
File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/base_events.py", line 468, in run_until_complete
return future.result()
File "test/pressure_test.py", line 26, in run
responses = await asyncio.gather(*tasks)
File "test/pressure_test.py", line 15, in fetch
async with session.post(url, data=payload) as response:
File "/usr/local/lib/python3.6/site-packages/aiohttp/client.py", line 855, in aenter
self._resp = await self._coro
File "/usr/local/lib/python3.6/site-packages/aiohttp/client.py", line 370, in _request
timeout=timeout
File "/usr/local/lib/python3.6/site-packages/aiohttp/connector.py", line 445, in connect
proto = await self._create_connection(req, traces, timeout)
File "/usr/local/lib/python3.6/site-packages/aiohttp/connector.py", line 757, in _create_connection
req, traces, timeout)
File "/usr/local/lib/python3.6/site-packages/aiohttp/connector.py", line 879, in _create_direct_connection
raise last_exc
File "/usr/local/lib/python3.6/site-packages/aiohttp/connector.py", line 862, in _create_direct_connection
req=req, client_error=client_error)
File "/usr/local/lib/python3.6/site-packages/aiohttp/connector.py", line 829, in _wrap_create_connection
raise client_error(req.connection_key, exc) from exc
aiohttp.client_exceptions.ClientConnectorError: Cannot connect to host 127.0.0.1:8081 ssl:None [Connect call failed ('127.0.0.1', 8081)]

[asvetlov]

The difference is: in the asyncio example you use 500 concurrent connections.

The multiprocessed code starts 10 processes for processing 500 requests each one-by-one. As the result, you have concurrency factor 10.

Looks like the server just doesn't support high parallelism.

[derlih]

@asvetlov should we close this issue?