Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Nomad exec driver does not support rootfs root partition #7136

th0m opened this issue Feb 12, 2020 · 4 comments · Fixed by #7149

Nomad exec driver does not support rootfs root partition #7136

th0m opened this issue Feb 12, 2020 · 4 comments · Fixed by #7149


Copy link

th0m commented Feb 12, 2020

Nomad version

Nomad v0.10.2

Operating system and Environment details

Debian Stretch
Root partition is a rootfs (ramdisk)

$ df -h /
Filesystem      Size  Used Avail Use% Mounted on
rootfs           63G  4.5G   59G   8% /


Any job using the exec driver fails to start which I believe because pivot_root does not work when the root partition is a ramdisk, see man 2 pivot_root

The rootfs (initial ramfs) cannot be pivot_root()ed.

I think Docker addressed this issue with the DOCKER_RAMDISK env variable which essentially disables pivot_root. I think what I need here is a way to disable pivot_root, probably as a configuration variable in the exec driver.

Reproduction steps

  1. Boot a Ubuntu live cd
  2. Run Nomad
  3. Plan a job using the exec driver

Job file (if appropriate)

$ cat /etc/nomad-jobs/job-hello_world.hcl 
job "hello_world" {
  datacenters = ["foo"]

    task "server" {
      driver = "exec"

      config {
        command = "/bin/true"

Nomad Client logs (if appropriate)

Feb 12 19:49:03 machine nomad-client[258433]:     2020-02-12T19:49:03.332Z [ERROR] client.alloc_runner.task_runner: running driver failed: alloc_id=fca82002-653b-33df-9324-4ba2bc1bbd15 task=server error="failed to launch command with executor: rpc error: code = Unknown desc = container_linux.go:346: starting container process caused "process_linux.go:449: container init caused \"rootfs_linux.go:109: jailing process inside rootfs caused \\\"pivot_root invalid argument\\\"\"""
@tgross tgross added this to the unscheduled milestone Feb 13, 2020
Copy link

tgross commented Feb 13, 2020

Hi @th0m! Thanks for opening this issue. I'm really curious to hear what you're doing with Nomad that you're running it from a rootfs!

As far as the feature request goes, that sounds reasonable. It looks like the underlying runc library supports this with the Config.NoPivot root field, so it'd be a matter of adding the config value to the exec stanza and then plumbing it through to runc. We'd love to get this as a contribution if you're interested.

Copy link
Contributor Author

th0m commented Feb 13, 2020

Hi @tgross thanks for the quick reply. About the rootfs, nothing fancy, just running Nomad on mostly stateless machines :)

If that sounds reasonable to you, I'll start working on a PR then!

Copy link

tgross commented Feb 13, 2020

Great! Looking forward to seeing the PR! 😀

Copy link

I'm going to lock this issue because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Oct 23, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
None yet

Successfully merging a pull request may close this issue.

2 participants