[ENHANCEMENT] Async client

[guipenedo]

The client code is currently fully sync. If we have n simultaneous rollouts, we are currently unable to submit n steps (one for each rollout) simultaneously, as the sync calls will block the main thread.

Any client code performing network calls (specially step which can take a while to give back a result) should be defined async.

[maneetbansal]

Hi @guipenedo,

I’m interested in contributing to OpenEnv and would like to help with this async client enhancement.

My understanding:

The current HTTPEnvClient is fully synchronous, so with n environments we end up stepping them sequentially and effectively serialize network latency.

The goal here is to expose an async client API so we can do things like:

results = await asyncio.gather(*[env.step(a) for env in envs])

and overlap the HTTP calls, especially for long-running step calls.

Proposed direction (happy to adjust):

• Introduce a new AsyncHTTPEnvClient alongside the existing sync client (to avoid breaking current users), with async def reset/step/state/close methods.
• Implement the async HTTP layer using an async HTTP client (e.g. httpx.AsyncClient), reusing a single session per env instance.
• Keep container startup (from_docker_image) synchronous for a first pass, and focus on making the HTTP calls themselves non-blocking. We could add a fully async startup factory in a follow-up if that’s desirable.
• Add pytest-asyncio tests that:
  • Exercise a basic async reset/step loop for a single env.
  • Demonstrate multiple environments being stepped concurrently with asyncio.gather.
• Update the docs with a short example showing how to use the async client for parallel rollouts.

Questions:

Before I start prototyping this, I’d love your thoughts on:

• Do you prefer a separate AsyncHTTPEnvClient class vs. evolving the existing client to be async-first with sync wrappers?
• Do you have a preferred async HTTP library (httpx vs aiohttp), or would you like to avoid adding new heavy dependencies?

If this direction sounds reasonable, I’d be happy to take a first stab at it.

P.S. To avoid code duplication between the sync and async clients, my plan would be to extract the shared request/response logic into a small base class or set of helpers, and have both HTTPEnvClient and AsyncHTTPEnvClient reuse that, so the only difference is the transport layer (sync requests.Session vs async HTTP client).