How to run in a (Docker) container environment?

[iedmrc]

How can a project developed on top of the Mlx Python library be run in a container (preferably docker) environment without losing the metal framework and unified memory advantage on an Apple silicon chip device?

[zcbenz]

Docker runs Linux (even when used on macOS), which has no software support for Metal so it is impossible to make use of Apple's GPU in it. The only way to get there is to ask Apple to release drivers and SDKs for Linux, which I have no idea how to make it happen.

[andreyvelich]

For those who are interested in running MLX in Docker and Kubernetes environments on Linux, we maintain this container image:

docker pull ghcr.io/kubeflow/trainer/mlx-runtime

The image is built from this Dockerfile: https://github.com/kubeflow/trainer/blob/master/cmd/runtimes/mlx/Dockerfile

It comes preinstalled with mlx[cuda] and OpenMPI, enabling distributed MLX using TrainJob and NVIDIA GPUs in a Kubernetes cluster.

Docs can be found here: https://www.kubeflow.org/docs/components/trainer/user-guides/mlx/
Llama fine-tuning example: https://github.com/kubeflow/trainer/blob/master/examples/mlx/language-modeling/fine-tune-llama.ipynb