switch to aiohttp post request mode

[leiwen83]

Why are these changes needed?

httpx performance is very poor in fastapi. use aiohttp is much better

Related issue number (if applicable)

#2256

Checks

• I've run format.sh to lint the changes in this PR.
• I've included any doc changes needed.
• I've made sure the relevant tests are passing (if applicable).

[merrymercy]

Thanks for the contribution!

1. Could you fix the lint errors?
2. Could you run these tests? https://github.com/lm-sys/FastChat/blob/main/docs/commands/test_process.md#test-openai-api-server

[leiwen83]

Thanks for the contribution!

1. Could you fix the lint errors?
2. Could you run these tests? https://github.com/lm-sys/FastChat/blob/main/docs/commands/test_process.md#test-openai-api-server

Both 1/2 are done.

[merrymercy]

Why is this still required?

[merrymercy]

I found one remaining usage of httpx. Should this be updated?

FastChat/fastchat/serve/openai_api_server.py

Line 580 in cbbd682

async with httpx.AsyncClient() as client:

[leiwen83]

I found one remaining usage of httpx. Should this be updated?

FastChat/fastchat/serve/openai_api_server.py

Line 580 in cbbd682

async with httpx.AsyncClient() as client:

Here httpx is used as stream, which is a different usage with other place.
I am not sure which aiohttp api should be used here, and how to verify it...

[merrymercy]

Okay. httpx is not a built-in library. Please move it to the second block of import statements.

[merrymercy]

1. Sort the packages alphabetically
2. Declare this dependency in pyproject.toml

[leiwen83]

1. Sort the packages alphabetically
2. Declare this dependency in pyproject.toml

Done

[merrymercy]

Do you have any performance numbers?

[leiwen83]

Do you have any performance numbers?

Hi,

please see the performance data below.
I have tested batching request 10/20/30/40 for comparing aiohttp and httpx.

request type \request batch	10	20	30	40
aiohttp	2.5s	4.5s	6.1s	8.5s
httpx	8s	17s	27s	httpx.ConnectTimeout

[merrymercy]

@leiwen83 Thanks! It is merged