Skip to content
Container plugin for Slurm Workload Manager
C Makefile
Branch: master
Clone or download
lukeyeager and flx42 Overload the --container-image option to allow squashfs path (#11)
Co-authored-by: Felix Abecassis <>
Signed-off-by: Felix Abecassis <>
Latest commit 84a31ce Nov 22, 2019
Type Name Latest commit message Commit time
Failed to load latest commit information.
.gitignore Initial release Aug 22, 2019
Makefile More verbose logging (#7) Nov 20, 2019 Overload the --container-image option to allow squashfs path (#11) Dec 3, 2019
common.c Use strcat instead of strncat to remove a warning with gcc 9.2.1 Nov 20, 2019
pyxis_slurmd.c Fix capitalization of "Slurm" in code comments Oct 3, 2019
pyxis_slurmstepd.c Overload the --container-image option to allow squashfs path (#11) Dec 3, 2019
seccomp_filter.h Initial release Aug 22, 2019


Pyxis is a SPANK plugin for the Slurm Workload Manager. It allows unprivileged cluster users to run containerized tasks through the srun command.

A pyxis is an ancient small box or container.


  • Seamlessly execute the user's task in an unprivileged container.
  • Simple command-line interface.
  • Fast Docker image download with support for layers caching and layers sharing across users.
  • Supports multi-node MPI jobs through PMI2 or PMIx (requires Slurm support).
  • Allows users to install packages inside the container.
  • Works with shared filesystems.
  • Does not require cluster-wide management of subordinate user/group ids.


Pyxis requires the enroot container utility to be installed.

$ # Option 1: quick install from sources
$ sudo make install
$ sudo ln -s /usr/local/share/pyxis/pyxis.conf /etc/slurm-llnl/plugstack.conf.d/pyxis.conf
$ sudo systemctl restart slurmd

$ # Option 2: generate a deb package and install it
$ make orig
$ make deb
$ sudo dpkg -i ../nvslurm-plugin-pyxis_*_amd64.deb
$ sudo ln -s /usr/share/pyxis/pyxis.conf /etc/slurm-llnl/plugstack.conf.d/pyxis.conf
$ sudo systemctl restart slurmd


Pyxis being a SPANK plugin, the new command-line arguments it introduces are directly added to srun.

$ srun --help
                              [pyxis] the image to use for the container
                              filesystem. Can be either a docker image given as
                              an enroot URI, or a path to a squashfs file on the
                              remote host filesystem.

                              [pyxis] bind mount[s] inside the container. Mount
                              flags are separated with "+", e.g. "ro+rprivate"

                              [pyxis] working directory inside the container
      --container-name=NAME   [pyxis] name to use for saving and loading the
                              container on the host. Unnamed containers are
                              removed after the slurm task is complete; named
                              containers are not. If a container with this name
                              already exists, the existing container is used and
                              the import is skipped.
      --container-mount-home  [pyxis] bind mount the user's home directory.
                              System-level enroot settings might cause this
                              directory to be already-mounted.

                              [pyxis] do not bind mount the user's home
      --container-remap-root  [pyxis] ask to be remapped to root inside the
                              container. Does not grant elevated system
                              permissions, despite appearances. (default)

                              [pyxis] do not remap to root inside the container


$ # Run a command on a worker node
$ srun grep PRETTY /etc/os-release
PRETTY_NAME="Ubuntu 18.04.2 LTS"

$ # run the same command, but now inside of a container
$ srun --container-image=centos grep PRETTY /etc/os-release
PRETTY_NAME="CentOS Linux 7 (Core)"

$ # mount a file from the host and run the command on it, from inside the container
$ srun --container-image=centos --container-mounts=/etc/os-release:/host/os-release grep PRETTY /host/os-release
PRETTY_NAME="Ubuntu 18.04.2 LTS"

Copyright and License

This project is released under the Apache License Version 2.0.

Issues and Contributing

Reporting Security Issues

When reporting a security issue, do not create an issue or file a pull request.
Instead, disclose the issue responsibly by sending an email to psirt<at>

You can’t perform that action at this time.