mix deps.get - Error fetching private repo on GitHub using :github option

[seantanly]

I'm using Elixir 1.0.4 and git 2.4.4.

mix deps.get is unable to fetch my private Github repo when it is specified using the :github option as follows in mix.exs

dep dep do
  [
    {:mylib, github: "seantanly/elixir-mylib", tag: "v0.1.0"}
  ]
end

The output

$ mix deps.get 
* Getting mylib (git@github.com/seantanly/elixir-mylib.git)
fatal: repository 'git@github.com/seantanly/elixir-mylib.git' does not exist
** (Mix) Command `git clone --no-checkout --progress "git@github.com/seantanly/elixir-mylib.git" "myprojpath/deps/mylib"` failed

However, it works perfectly when I specify the mixfile using the standard :git option as follows

dep dep do
  [
    {:mylib, git: "git@github.com:seantanly/elixir-mylib.git", tag: "v0.1.0"}
  ]
end

Am I missing something obvious?

[josevalim]

The github option only works with public repos. Private repos require the path as you have specified.

[seantanly]

Thanks for the fast reply!

Perhaps the example for the docs at http://elixir-lang.org/getting-started/mix-otp/dependencies-and-umbrella-apps.html#internal-dependencies should be updated from

def deps do
  [{:kv, git: "git://github.com/YOUR_ACCOUNT/kv.git"}]
end

to

def deps do
  [{:kv, git: "git@github.com:YOUR_ACCOUNT/kv.git"}]
end

Following the format of the original example results in the same error I have encountered.

[josevalim]

That would then only work for private repos and not public ones. :) We should add a note about private repos though.

[seantanly]

I just tried the following and I have no problems fetching from the public cmullaparthi/ibrowse GitHub repo. If this format works regardless of whether it's a public or private repo, perhaps the internally mix deps.get task should instead adopt this format for GitHub repos?

def deps do
  {:ibrowse, git: "git@github.com:cmullaparthi/ibrowse.git", tag: "v4.1.1"}
end

and here's the corresponding mix.lock entry

"ibrowse": {:git, "git@github.com:cmullaparthi/ibrowse.git", "d2e369ff42666c3574b8b7ec26f69027895c4d94", [tag: "v4.1.1"]},

[josevalim]

Ah, that's a good point. However, wouldn't still require you to have authentication on all of your servers? For example, once you deploy, you would need to have an SSH key that can access Github on the server. If that is the case, I still don't think it is worth doing this change. Just be explicit if you have a private repo.

[seantanly]

I see your point now, for the alternative format to work even for public repos, it needs an GitHub account/project associated SSH key being present.

I agree there isn't a need for mix to change, and yes, updating the docs example would help since that's the first resource I found on using private dependencies, and I couldn't get the example to work.

[gualopezb]

@josevalim I'm needing to grab a dependency from a private git repo but the ssh key pair includes a passphrase, how can I set that passphrase on the mix file?

[josevalim]

It is not recommended to put ssh passphrases in the mix.exs file. There are different ways you can configure ssh so it applies to a certain host.

However, if you really need to use ssh keys in the URL, then I recommend using url rewrite in your .gitconfig:

[url "ssh://key:pass@domain.com/"]
  insteadOf = https://domain.com/

So you will use git: "https://domain.com/foo/bar" in your mix.exs file and git will rewrite it to the proper "key:pass".

[NobbZ]

I think the proper solution were to use an ssh-agent of your choice and to unlock the key before fetching the deps. The alternative is of course to change the passphrase to an empty one, but that's probably not the thing you actually want to do.

…

---------- *edit* remove quoted mail text.

[tanweerdev]

There is a need just to give read access like using deployment keys of bitbucket. For this it would be great if there is some way that I can put my read only access deployment key inside project and specify path in mix file to fetch that dependency

[nacengineer]

Note: I was having this problem with bitbucket and windows. I had to switch to the https version of the repo url to get it working.

[i-n-g-m-a-r]

GitLab CI needs a (https) url containing a gitlab-ci-token to fetch private repositories.

Fetching a private repo like this only works locally, not in CI:
{:my_repo, git: "git@gitlab.com:my-group/my_repo.git"}

A workaround is to put the GitLab url in a variable in .gitlab-ci.yml:

  variables:
    MIX_ENV: "test"
    MIX_MY_REPO: "https://gitlab-ci-token:${CI_JOB_TOKEN}@gitlab.com/my-group/my_repo.git"

Then use the environment variable in mix.exs:

  defp deps do
    [
      ...
      {:my_repo, git: System.get_env("MIX_MY_REPO", "git@gitlab.com:my-group/my_repo.git")},
      ...
    ]
  end

Note that when a dependency also needs to fetch a (nested) dependency the parent .gitlab-ci.yml needs to define both variables.

[josevalim]

Git also knows how to manage credentials, so you can store the credentials on git directly and keep using bare URLs in your mix.exs: https://git-scm.com/docs/gitcredentials

[i-n-g-m-a-r]

That would mean not using a ssh key to fetch dependencies from GitLab right?
Storing username/password in git directly instead (on every machine).
I kind of like my ssh key and never having to deal with username/password.
It helps me to know exactly which machines have access by looking at the ssh keys settings in GitLab.

[josevalim]

You can use ssh hosts for that. For example, in your .ssh/config you could do:

Host orggitlab
  User git
  HostName ssh.github.com
  Port 443

And now you declare this as your git repo in mix.exs and everywhere: orggitlab:path/to/repo - Each person can define their own host, user and pass on their ssh config.

Not saying this is the way to go, just providing options. :)

[i-n-g-m-a-r]

That's pretty clever but it would probably not solve the fetching private repos in GitLab CI problem right?