Using artifact stanza errors with relative paths require a module with a pwd

[jasonmccallister]

Nomad version

Nomad v0.12.5

Operating system and Environment details

• macOS 10.15.6
• Ubuntu 20.04

Issue

I'm new to Nomad and I believe this is more of a documentation issue and wanted to put this somewhere other people can find easily. I spent way too much time trying to resolve this and it was a little bit of a burden to getting started with Nomad.

When using the artifact stanza to clone a Git repository, I was getting the error relative paths require a module with a pwd. It took quite a long time and digging to find out how the artifact stanza really worked as well as how to resolve the error.

Reproduction steps

Create a Job file with an artifact and using the docker exec driver. See the missing

Job file (that causes the error)

job "example" {
  datacenters = [
    "dc1"
  ]

  type = "batch"

  group "web" {
    task "setup" {
      artifact {
        source = "git@github.com:username/repo.git"
        destination = "local/repository"
        options {
           sshkey = "<key-in-base64>"
         }
      }

      driver = "docker"

      config {
        image = "alpine"
        args = ["ls"]
      }
    }
  }
}

Since it was unclear how the artifact and paths works for the task, the jobs were failing with relative paths require a module with a pwd which really did not say anything that would help me resolve the issue. I even scoured the documentation on the go-getter repository.

The real issue is that I was not mounting the local directory into the container (I was going step by step to examine the process and filesystem). To resolve the issue I needed to mount the directory using the docker config.volume stanza and the job completed successfully.

Job file (that resolves the error)

job "example" {
  datacenters = [
    "dc1"
  ]

  type = "batch"

  group "web" {
    task "setup" {
      artifact {
        source = "git@github.com:username/repo.git"
        destination = "local/repository"
        options {
           sshkey = "<key-in-base64>"
         }
      }

      driver = "docker"

      config {
        image = "alpine"
        args = ["ls"]

        volumes = [
          "local/repository/:/path/on/container",
        ],
      }
    }
  }
}

I know Nomad is new and this is not an attempt to bash the product, just an illumination on the confusion on how jobs/documentation might need an extra set of eyes from the perspective of someone new to Nomad. More specifically, I think there should be example job files for lots of popular application stacks to make it easier to get started.

[tgross]

Hi @jasonmccallister! Thanks for opening this issue! The artifact stanza definitely has some documentation issues; we're punting to the go-getter docs and it would be really helpful to have better examples for sure.

More specifically, I think there should be example job files for lots of popular application stacks to make it easier to get started.

Big time agree with you here. We have some work underway to improve this but as you can imagine that's a bit of a lift.

---

Ok, onto the issue at hand. You definitely shouldn't need to mount a Docker volume to make artifact work with git URLs, and if we somehow do I'd call that a bug and not just a documentation issue! 😁

I looked up the error message you got and it only shows up for the "file" URL detector: detect_file.go#L21, which tells me that Nomad and go-getter are confused as to the URL we're getting.

Interestingly, there's a comment in the source that points to just this form of URL: getter.go#L63-L64 so this must be something we have specific handling for. Just for clarification, is the full error you're seeing in the logs failed to parse source URL: git@github.com:username/repo.git: relative paths require a module with a pwd?

[tgross]

The behavior for this has been improved in #9671 and we've provided some new documentation for the file system layout here: https://www.nomadproject.io/docs/internals/filesystem

I'm going to close this issue but please feel free to open a new one if you encounter new issues.

[undying]

Hello!
I faced the same problem.
Have read documentation and source code and still have no idea how to fix this problem.
I'm trying to clone source code from bitbucket repository and also get error relative paths require a module with a pwd

My artifact configuration is:

      artifact {
        source = "git::https://bitbucket.org:***/***.git"
        destination = "${NOMAD_TASK_DIR}/repo"

        options {
          sshkey = "***"
          depth = 1
        }
      }

[tgross]

Hi @undying! That should be working on any version of Nomad after 1.0.2. Can you open a new issue with the complete jobspec, Nomad version, and any debug logs?

[undying]

I got this error with version 1.0.3 from deb repository.
I can create a new task, what kind of logs will be required?

[tgross]

The client logs around the time the task is placed.

[undying]

Link to the new issue.
Hope this helps :)

[github-actions]

I'm going to lock this issue because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.