Okay to add arbitrary flags to docker create?

[galv]

Description

Hi all,

I used your project to create manylinux2014 wheels for a project that uses CUDA. This in itself is quite interesting because I needed to (1) keep a stable ABI on the pybind11 boundaries, (2) not link to any cuda shared libraries, (3) make sure that the wheel size stayed small, and (4) the version of gcc used in manylinux2014 has some incapabilities with. This workflow is definitely worth of sharing more broadly since a lot of CUDA projection adoption is hampered by a lack of pre-built binaries. But that is not the current topic.

Building wheels is perfectly fine for the build process, which uses only CPUs, but I noticed failures when I tried to test because GPUs (which were required by the test cases) on the host machine weren't accessible within the container. The fix, in short, was to add "---gpus all" to the end of the flags here:

cibuildwheel/cibuildwheel/oci_container.py

Lines 88 to 92 in 27fc88e

	"--env=CIBUILDWHEEL",
	f"--name={self.name}",
	"--interactive",
	"--volume=/:/host", # ignored on CircleCI
	*network_args,

I made a quick hack where I added an extra variable (CIBW_EXTRA_DOCKER_CREATE_ARGS, which I set to "--gpus all") to support my use case. Unfortunately, there's no clever way to make docker automatically see from what I've found other than changing the (I would be happy to be corrected). Is this a change that you would be interested in accepting? I can see how allowing users to add arbitrary commands to the docker create command could be a bit fishy (e.g., they could override --volume=/:/host if they were silly), but cibuildwheel does seem like an expert's tools anyway. I would greatly appreciate any feedback form the maintainers.

Build log

No response

CI config

No response

[joerick]

This seems like a reasonable request to me - I can't think of another good way to do this within the existing API.

Since these options will be very contingent on the docker engine, and rarely used, I'd be inclined to add these to the existing CIBW_CONTAINER_ENGINE option. For example, one could do CIBW_CONTAINER_ENGINE="docker,create_args=--gpus all", or in TOML,

container_engine = { name = "docker", create_args = ["--gpus", "all"] }

(This TOML would depend on how easy it is to implement!)

[bryandeng]

I encountered exactly the same situation as @galv. And my test cases include deep learning workloads, so some other flags like --ipc host need to be passed to the container engine.

[VDuchauffour]

Any news about this? Any other ideas on how to activate the GPU for cibuildwheel? Thanks

[joerick]

I'm working on a solution here, might have something ready in a couple of days!