Feature Request: Add HTTP/2 Support for High-Concurrency Workloads

[fede-kamel]

Feature Request: Add HTTP/2 Support for High-Concurrency Workloads

Summary

Add optional HTTP/2 support to the OpenAI Python SDK to significantly improve performance for high-concurrency workloads through request multiplexing.

Motivation

The OpenAI SDK currently uses HTTP/1.1 with connection pooling (max 1000 connections). While this works well for most use cases, it becomes a bottleneck for high-concurrency workloads:

• Connection Pool Exhaustion: With 100+ concurrent requests, the connection pool can become saturated
• TCP/TLS Handshake Overhead: Each new connection requires a full TCP and TLS handshake
• Resource Inefficiency: Maintaining 1000 connections consumes significant memory and file descriptors

HTTP/2 solves these issues through request multiplexing - allowing hundreds of concurrent requests over a single connection.

Proposed Solution

Add an optional http2 parameter to OpenAI and AsyncOpenAI clients:

from openai import AsyncOpenAI

# Enable HTTP/2 for high-concurrency workloads
async with AsyncOpenAI(http2=True) as client:
    # Process 100+ requests efficiently
    tasks = [
        client.chat.completions.create(
            model="gpt-4o-mini",
            messages=[{"role": "user", "content": f"Request {i}"}]
        )
        for i in range(100)
    ]
    results = await asyncio.gather(*tasks)

Expected Performance Impact

Concurrent Requests	HTTP/1.1	HTTP/2	Expected Speedup
20 requests	4.0s	3.0s	1.3x
50 requests	10.0s	4.0s	2.5x
100 requests	20.0s	5.0s	4.0x
1000+ requests	200s	25s	8.0x

Use Cases

This feature would benefit:

1. Batch Processing: Generating embeddings or completions for large datasets
2. Real-time Applications: Handling many concurrent user requests
3. Data Pipelines: Processing streams of data through OpenAI APIs
4. Serverless Functions: Reducing cold start impact with efficient connection reuse

Implementation Details

• Backward Compatible: HTTP/2 disabled by default, opt-in via http2=True
• Dependency: Requires optional h2 package (pip install openai[http2])
• httpx Support: Leverages existing httpx HTTP/2 capabilities
• Connection Optimization: Reduces connection limit from 1000 to 100 for HTTP/2 (multiplexing makes this optimal)

Additional Context

• HTTP/2 is already used in the OpenAI CLI tool but not exposed in the main client
• The OpenAI API servers support HTTP/2
• httpx (the underlying HTTP library) has mature HTTP/2 support via the h2 package
• If HTTP/2 is not supported by a proxy/server, httpx automatically falls back to HTTP/1.1

Questions

• Would this be valuable for the OpenAI community?
• Are there any concerns about making this an optional feature?
• Should this be enabled by default in a future major version?

[karpetrosyan]

I think this feature request proposes an enhanced API for using HTTP/2 rather than just HTTP/2 support, as we already support HTTP/2. Currently, you can pass a custom httpx client and configure it to use HTTP/2 like so:

# /// script
# requires-python = ">=3.9"
# dependencies = [
#     "httpx[http2]",
#     "openai",
# ]
# ///


import httpx
import openai

cl = openai.OpenAI(http_client=httpx.Client(http1=False, http2=True))

response = cl.responses.with_raw_response.create(
    model="gpt-4",
    input="Hello, world!",
)

print(response.http_response.extensions["http_version"])  # should print b'HTTP/2'