Native memory leak when not using `async with GitHub`

[galshi]

Context

Without async with GitHub() as github:, every async API call creates and closes a new httpx.AsyncClient. Each construction loads the CA bundle into native OpenSSL memory. In long-running processes this memory accumulates. It did stop growing at some point in production for unknown reasons; the only observed correlation was that the change appeared after making POST-style API calls (for example posting issue comments or pull request reviews), but there are no reliable reproduction steps beyond that. Still worth fixing because the behavior is undocumented and the memory growth is real and observable.

Less noticeable with GET requests because hishel's ETag caching short-circuits some of the client lifecycle. Most visible with GraphQL and POST requests which hishel cannot cache.

Not visible in tracemalloc or objgraph. Only shows up in memray --native. That is how the leak was found.

Root Cause

githubkit/core.py lines 289–298:

@asynccontextmanager
async def get_async_client(self) -> AsyncGenerator[httpx.AsyncClient, None]:
    if client := self.__async_client.get():
        print("Creating new async client")  # Added this
        yield client
    else:
        print("Reusing existing async client")  # Added this
        client = self._create_async_client()  # new SSL context every call
        try:
            yield client
        finally:
            await client.aclose()

self.__async_client is a ContextVar set only inside __aenter__. Without the context manager, every call hits the else branch. _create_async_client() calls ssl.create_default_context(cafile=certifi.where()) which loads the CA bundle into native heap.

This was verified by adding prints to both branches (see comments in the code sample). In reproduction the leak run repeatedly printed Creating new async client, while the async with run printed Reusing existing async client once and then reused the client.

Memray Trace

The native allocations trace through OpenSSL certificate parsing during the TLS handshake on each new client:

_ssl_MemoryBIO_write_impl  (_ssl.c)
SSL_read_ex
X509_new_ex
ASN1_item_d2i_ex
ASN1_item_new_ex
CRYPTO_malloc / CRYPTO_zalloc   <-- repeated allocations
EVP_CipherInit_ex
OSSL_DECODER_from_bio
d2i_EC_PUBKEY
EVP_PKEY_new
EC_GROUP_dup / EC_GROUP_copy
CRYPTO_zalloc

Each new client parses certificates and allocates EC key and ASN.1 structures in native heap that are not tracked by Python's memory management.

Reproduction

import asyncio
import resource
from pathlib import Path
from githubkit import GitHub

def rss_mb() -> float:
    pages = int(Path("/proc/self/statm").read_text().split()[1])
    return pages * resource.getpagesize() / 1024 / 1024

TOKEN = "<token>"
TURN = 20
async def memory_leak():
    github = GitHub(TOKEN)
    for i in range(TURN):
        await github.async_graphql("{ viewer { login } }")
        print(f"[leak]    turn={i} rss={rss_mb():.1f}MB")
        await asyncio.sleep(1)

async def no_leak():
    async with GitHub(TOKEN) as github:
        for i in range(TURN:
            await github.async_graphql("{ viewer { login } }")
            print(f"[no-leak] turn={i} rss={rss_mb():.1f}MB")
            await asyncio.sleep(1)

asyncio.run(memory_leak())
asyncio.run(no_leak())

Output

[leak]    turn=0 rss=72.4MB
[leak]    turn=1 rss=72.9MB
[leak]    turn=2 rss=73.8MB
[leak]    turn=3 rss=74.6MB
[leak]    turn=4 rss=75.4MB
[leak]    turn=5 rss=76.3MB
[leak]    turn=6 rss=77.1MB
[leak]    turn=7 rss=78.0MB
[leak]    turn=8 rss=78.9MB
[leak]    turn=9 rss=79.8MB
[leak]    turn=10 rss=80.6MB
[leak]    turn=11 rss=81.5MB
[leak]    turn=12 rss=82.3MB
[leak]    turn=13 rss=83.1MB
[leak]    turn=14 rss=84.0MB
[leak]    turn=15 rss=84.8MB
[leak]    turn=16 rss=85.6MB
[leak]    turn=17 rss=86.6MB
[leak]    turn=18 rss=87.4MB
[leak]    turn=19 rss=88.3MB
[no-leak] turn=0 rss=89.1MB
[no-leak] turn=1 rss=89.1MB
[no-leak] turn=2 rss=89.1MB
[no-leak] turn=3 rss=89.1MB
[no-leak] turn=4 rss=89.1MB
[no-leak] turn=5 rss=89.1MB
[no-leak] turn=6 rss=89.1MB
[no-leak] turn=7 rss=89.3MB
[no-leak] turn=8 rss=89.3MB
[no-leak] turn=9 rss=89.3MB
[no-leak] turn=10 rss=89.3MB
[no-leak] turn=11 rss=89.3MB
[no-leak] turn=12 rss=89.3MB
[no-leak] turn=13 rss=89.3MB
[no-leak] turn=14 rss=89.3MB
[no-leak] turn=15 rss=89.4MB
[no-leak] turn=16 rss=89.4MB
[no-leak] turn=17 rss=89.4MB
[no-leak] turn=18 rss=89.4MB
[no-leak] turn=19 rss=89.4MB

Why It's Hard To Notice

The quickstart and most examples show github = GitHub(token) with direct await calls. The async with pattern appears in some docs but with no explanation of why it matters. No warnings, no existing issues about this. A Python profiler will show nothing wrong.

Suggested Fixes

Emit a ResourceWarning in the else branch of get_async_client() when called without prior __aenter__.

Document that async with is required for connection reuse and that skipping it causes native memory growth on every uncached request.

Environment

Python	3.14
githubkit	0.15.0
OS	WSL2 (devcontainer), Amazon Linux (EC2)
Profiler	memray --native

[yanyongyu]

It seems this issue is caused by the repeated creation of httpx clients. Is it possible to reproduce this using only httpx without using githubkit? If so, I think this issue should also be reported to httpx.

[galshi]

I reproduced it with httpx only.

import asyncio
import resource
from pathlib import Path

import httpx


def rss_mb() -> float:
    pages = int(Path("/proc/self/statm").read_text().split()[1])
    return pages * resource.getpagesize() / 1024 / 1024

TURN = 10

async def httpx_leak_test():
    print("Testing: Create new AsyncClient per request")
    for i in range(TURN):
        client = httpx.AsyncClient(verify=True)
        try:
            resp = await client.get("https://api.github.com/")
            await resp.aread()
        finally:
            await client.aclose()

        print(f"[new-client] turn={i} rss={rss_mb():.1f}MB")
        await asyncio.sleep(0.1)

async def httpx_no_leak_test():
    print("\nTesting: Reuse single AsyncClient")
    async with httpx.AsyncClient(verify=True) as client:
        for i in range(TURN):
            resp = await client.get("https://api.github.com/")
            await resp.aread()

            print(f"[reuse-client] turn={i} rss={rss_mb():.1f}MB")
            await asyncio.sleep(0.1)

asyncio.run(httpx_leak_test())
asyncio.run(httpx_no_leak_test())

I found a Python 3.14 GC issue discussing a leak in SSL when creating multiple async connections, it was observed with httpx as well python/cpython#142516.
I also found an issue from 2023 (pre 3.14) in httpx (encode/httpx#2785) with a similar problem, httpx expects you to reuse the same client otherwise memory leaks.
Worth noting that both issues appear specifically with async client creation.
Seems to me like it's worth adding something to the documentation or a resource warning.

[galshi]

Hi, small nit regarding #288 . You added it to the rest api docs, but rest is cached by Hishel, so I rarely felt it there. I actually discovered it because of a leak in a graphql query.

Since this is a general httpx problem exacerbated by Python 3.14, maybe move it to a more central place in the docs?

[yanyongyu]

It seems better to move the Reusing Client section to a separate page.