Summary
cleanup_routine() in airtbench/kernel.py lists all containers on the Docker daemon and force-deletes any in exited state, with no filter on image, label, name, or ownership.
# kernel.py:690-707
containers = await client.containers.list(all=True)
for container in containers:
container_info = await container.show()
if container_info.get("State", {}).get("Status") == "exited":
await container.delete(force=True)
This routine is called every 5 steps inside each concurrent challenge's loop (main.py:660-665, default concurrency=3). On a shared Docker host, any other workload's stopped container — another benchmark run, a developer's containers, CI sidecars — gets docker rm -f'd.
Reproduction
# Terminal 1: start an unrelated container
docker run -d alpine sleep 1
# Terminal 2: run AIRTBench
python -m airtbench --concurrency 3 ...
# Within 5 steps, the alpine container is force-deleted by cleanup_routine.
Impact
Cross-tenant container destruction on shared Docker hosts. AIRTBench is designed to run with concurrency=3 on a shared host; the cleanup routine destroys any unrelated exited container it finds, with no label/image/name filter to scope it to AIRTBench-owned containers.
Proposed fix
I have a fix on a branch at AUTHENSOR/AIRTBench-Code:fix/cleanup-routine-scope-to-airtbench-containers. (Opening as an issue rather than a PR because fork-based PRs appear to be restricted at the org level.)
The fix:
- Adds a
"Labels": {"airtbench": "true"} field to the container config in PythonKernel.__aenter__ (kernel.py container creation).
- Filters
cleanup_routine to only delete containers with that label (client-side label check on the already-fetched container_info, avoiding Docker API filter-param encoding subtleties).
The label check is done client-side rather than via the Docker API filters query param, to avoid query-param encoding subtleties in aiodocker's containers.list(**kwargs) passthrough.
# Fixed cleanup_routine
containers = await client.containers.list(all=True)
for container in containers:
container_info = await container.show()
labels = container_info.get("Config", {}).get("Labels") or {}
if labels.get("airtbench") != "true":
continue
if container_info.get("State", {}).get("Status") == "exited":
await container.delete(force=True)
ruff check introduces zero new errors (verified: 11 errors before and after, all pre-existing BLE001 in unchanged code).
Happy to open a PR directly if the org allows it, or feel free to cherry-pick from the branch.
Summary
cleanup_routine()inairtbench/kernel.pylists all containers on the Docker daemon and force-deletes any inexitedstate, with no filter on image, label, name, or ownership.This routine is called every 5 steps inside each concurrent challenge's loop (
main.py:660-665, defaultconcurrency=3). On a shared Docker host, any other workload's stopped container — another benchmark run, a developer's containers, CI sidecars — getsdocker rm -f'd.Reproduction
Impact
Cross-tenant container destruction on shared Docker hosts. AIRTBench is designed to run with
concurrency=3on a shared host; the cleanup routine destroys any unrelated exited container it finds, with no label/image/name filter to scope it to AIRTBench-owned containers.Proposed fix
I have a fix on a branch at
AUTHENSOR/AIRTBench-Code:fix/cleanup-routine-scope-to-airtbench-containers. (Opening as an issue rather than a PR because fork-based PRs appear to be restricted at the org level.)The fix:
"Labels": {"airtbench": "true"}field to the container config inPythonKernel.__aenter__(kernel.py container creation).cleanup_routineto only delete containers with that label (client-side label check on the already-fetchedcontainer_info, avoiding Docker API filter-param encoding subtleties).The label check is done client-side rather than via the Docker API
filtersquery param, to avoid query-param encoding subtleties inaiodocker'scontainers.list(**kwargs)passthrough.ruff checkintroduces zero new errors (verified: 11 errors before and after, all pre-existingBLE001in unchanged code).Happy to open a PR directly if the org allows it, or feel free to cherry-pick from the branch.