Very slow response time

[umnovI]

Hi, I'm getting very low server response time compared to requests_cache.

Here are some screenshots from locust comparing requests_cache on the left and hishel on the right

I was testing response time also with code provided in documentation. It wasn't as fast as requests_cache but still not 3 RPS.

app.py

from api import afetch, fetch
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel

app = FastAPI()

async def make_request(endpoint: str, query: dict) -> dict:
    data = await afetch(endpoint, query)
    if "error" in data:
        raise HTTPException(status_code=int(data["error"]), detail=data["msg"])
    return data

@app.get("/api/pokemon/")
async def get_pokemon(offset: int | None = None, limit: int | None = None) -> dict:
    pagination: dict[str, int | None] = {"offset": offset, "limit": limit}
    return await make_request("pokemon/", pagination)

api.py

import hishel
from httpx import Limits
from requests_cache import CachedSession

API_URL: str = "https://pokeapi.co/api/v2/"
hishel_storage = hishel.AsyncFileStorage()
hishel_controller = hishel.Controller(allow_stale=True)


def fetch(endpoint: str, query: dict) -> dict:

    session = CachedSession(
        ".cache/http_cache",
        backend="filesystem",
        cache_control=True,
        stale_while_revalidate=True,
    )
    request_url: str = API_URL + endpoint
    response = session.get(request_url, timeout=3, params=query)
    if not response.ok:
        return {
            "msg": f"Error {response.status_code} occurred while requesting {request_url}",
            "error": response.status_code,
        }

    return response.json()


async def afetch(endpoint: str, raw_query: dict[str, int | None]) -> dict:

    # requests omits params whose values are None.
    # This is not supported by HTTPX.
    processed_query: dict = {}
    for data in raw_query:
        if raw_query[data] is not None:
            processed_query[data] = raw_query[data]

    async with hishel.AsyncCacheClient(
        storage=hishel_storage,
        base_url=API_URL,
        controller=hishel_controller,
        limits=Limits(max_connections=1000),
    ) as client:
        response = await client.get(endpoint, timeout=3, params=processed_query)
        if not response.is_success:
            return {
                "msg": f"Error {response.status_code} occurred while requesting {endpoint}",
                "error": response.status_code,
            }

        return response.json()

If I declare hishel.AsyncCacheClient outside a function and use without with then I get about 145 RPS.

[umnovI]

After a couple of more tests I've got better results than from requests_cache (on the left and hishel on the right).

But only if I don't close the connection. Don't I need to close it tho? Or when should I close it then?

import hishel
from httpx import Limits
from requests_cache import CachedSession

API_URL: str = "https://pokeapi.co/api/v2/"
hishel_storage = hishel.AsyncFileStorage()
hishel_controller = hishel.Controller(allow_stale=True)
hishel_client = hishel.AsyncCacheClient(
    storage=hishel_storage,
    base_url=API_URL,
    controller=hishel_controller,
    limits=Limits(max_connections=1000),
)

async def afetch(endpoint: str, raw_query: dict[str, int | None]) -> dict:

    processed_query: dict = {}
    for data in raw_query:
        if raw_query[data] is not None:
            processed_query[data] = raw_query[data]

    response = await hishel_client.get(endpoint, timeout=3, params=processed_query)
    if not response.is_success:
        return {
            "msg": f"Error {response.status_code} occurred while requesting {endpoint}",
            "error": response.status_code,
        }

    return response.json()

[karpetrosyan]

I ran these tests locally as well, and when I move httpx.Client outside of my endpoint, it runs in 0.0001 second, rather than the 0.05 that I get when creating a client for each HTTP request.

After a few tests, I discovered that httpx.HTTPTransport is very expensive, so creating 100 of them takes about 5 seconds.

Code:

import httpx

for i in range(100):
    httpx.HTTPTransport()

Result:

real	0m5.382s
user    0m5.358s
sys	0m0.024s

[karpetrosyan]

After a few more tests, I discovered that the load_verify_locations function takes that long, and creating an SSL Context for each transport is very expensive.

So the example above can be changed to this:

import ssl
import certifi

for i in range(100):
    context = ssl.SSLContext()
    context.load_verify_locations(certifi.where())

I'm not sure how urllib3 handles this problem.
I'm guessing it just configures SSL Context later, rather than when the pool is created, and since any subsequent responses requests_cache gets from the cache, there's no need to configure SSL Context each time.

[umnovI]

I've found this. So using it out of endpoint is a correct decision, I guess. It also mentions "explicit" closure. I guess client gets closed eventually (?)

[karpetrosyan]

I've found this. So using it out of endpoint is a correct decision, I guess. It also mentions "explicit" closure. I guess client gets closed eventually (?)

Simply use a single client for your application and don't close it. Your operating system will clean up everything after the programme has finished.

[umnovI]

I'll do that then. Thanks!