Can't install package from private GitHub repository with Yarn but can with NPM

[adamwathan]

Do you want to request a feature or report a bug?

Bug

What is the current behavior?

Running yarn add git+ssh://git@github.com:adamwathan/my-package results in a "Permission denied (publickey)" error:

I can git clone this repo without any trouble, and adding via NPM works fine:

What is the expected behavior?

The package is installed without permission issues.

Please mention your node.js, yarn and operating system version.

Node 8.4.0
Yarn 1.0.1
macOS Sierra 10.12.4

[naganowl]

This seems related to #4302 and isn't a problem in 1.0.0 (at least not with my experience with the private package I'm working with)

Also, it looks like the pre-existing yarn modules cache might be related to it as well as I ran into this problem on 1.0.0 as soon as I tried an install after yarn cache clean

[scull7]

I'm also receiving this error. I've upgraded to the latest yarn and have ensured to add the proper key via ssh-add. I've also verified that npm correctly installs the packages.

Node 8.4.0
Yarn 1.2.1
macOS Sierra 10.12.6

[scull7]

diff --git a/package.json b/package.json
index 9ded77ada..2bdfc80f0 100644
--- a/package.json
+++ b/package.json
@@ -82,7 +82,7 @@
     "node-nconf-config": "^1.0.3",
     "node-notifier": "^4.1.1",
     "object-assign": "^2.0.0",
-    "our-node-stuff": "git+ssh://github.com/influentialpublishers/our-node-stuff.git#0d91067",
+    "our-node-stuff": "git+ssh://git@github.com/influentialpublishers/our-node-stuff.git#0d91067",
     "proxy-middleware": "^0.15.0",
     "pug": "^2.0.0-beta3",
     "pug-loader": "^2.3.0",

It seems that our git string was incorrectly configured. notice the lack of git@ in the deletion.

[rally25rs]

Are you able to re-test this on Yarn v1.2.1? I just tried a private Github repo using the same URL format on v1.2.1 and it worked. (on OSX)

[scull7]

Tried again and received the same failure.

[NewEvolution]

Same issue as well, though it's a private Bitbucket repo and yarn 1.3.2. Confirmed I can run the failing commands manually:

[rally25rs]

@scull7 seems to have found the issue. git@github.com works but github.com does not.

Git itself doesn't support the URL without the git@

$ git clone git+ssh://github.com/my-org/my-project
Cloning into 'my-project'...
Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

$ git clone git+ssh://git@github.com/my-org/my-project
Cloning into 'my-project'...
remote: Counting objects: 4501, done.
remote: Total 4501 (delta 0), reused 0 (delta 0), pack-reused 4501
Receiving objects: 100% (4501/4501), 6.23 MiB | 2.36 MiB/s, done.
Resolving deltas: 100% (2920/2920), done.

I suspect NPM must do something to "fix" the URLs for you, but using a proper git URL would be ideal here. I'm not sure if Yarn should attempt to fix an improper URL. More discussion needed here...

---

After some poking around in the NPM source, I think they use https://github.com/npm/normalize-git-url to "fix" their URLs, however:

$ node
> const ngu = require('normalize-git-url');
undefined
> ngu('git+ssh://github.com/my-org/my-project');
{ url: 'ssh://github.com/my-org/my-project', branch: 'master' }

So the normalization just removes the leading ssh+ which the git client also rejects:

git clone ssh://github.com/my-org/my-project
Cloning into 'my-project'...
Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

so I'm not really sure how NPM handles this... 😕

[scull7]

@rally25rs, I don’t believe this is a yarn issue. I think NPM is wrong in that it will accept an invalid configuration. Perhaps a highlighted note in the documentation, and a better error message would be the best approach?

[Daniel15]

Git itself doesn't support the URL without the git@

For what it's worth, without the git@ it'll likely use your current system username rather than a username of git, similar to how SSH works (ssh example.com will use your current username, ssh git@example.com will use a username of git). GitHub always uses a username of "git" to connect to Git via SSH.

[VictorGosse]

Hey !

I don't know if this can help but I had the same issue for few days now. To solve it, here is what I did:

• reconnect to the private gitlab (it's using LDAP)
• launched a new console and cloned another repo (I did that to make sure I still can access it)
• re-ran yarn install in my initial repo and it worked....

Looked like a weird connexion issue somewhere...

[BYK]

We already do some normalization/massaging in

yarn/src/util/git.js

Lines 86 to 122 in aa1e54d

	static npmUrlToGitUrl(npmUrl: string): GitUrl {
	// Expand shortened format first if needed
	let parsed = url.parse(npmUrl);
	const expander = parsed.protocol && SHORTHAND_SERVICES[parsed.protocol];
	if (expander) {
	parsed = expander(parsed);
	}
	if (parsed.protocol && parsed.protocol.startsWith(GIT_PROTOCOL_PREFIX)) {
	parsed.protocol = parsed.protocol.slice(GIT_PROTOCOL_PREFIX.length);
	}
	// Special case in npm, where ssh:// prefix is stripped to pass scp-like syntax
	// which in git works as remote path only if there are no slashes before ':'.
	// See #3146.
	if (
	parsed.protocol === SSH_PROTOCOL &&
	parsed.hostname &&
	parsed.path &&
	parsed.path.startsWith(SCP_PATH_PREFIX) &&
	parsed.port === null
	) {
	const auth = parsed.auth ? parsed.auth + '@' : '';
	const pathname = parsed.path.slice(SCP_PATH_PREFIX.length);
	return {
	hostname: parsed.hostname,
	protocol: parsed.protocol,
	repository: `${auth}${parsed.hostname}:${pathname}`,
	};
	}
	return {
	hostname: parsed.hostname || null,
	protocol: parsed.protocol || 'file:',
	repository: url.format({...parsed, hash: ''}),
	};
	}

I think it is reasonable to add well-known services there such as GitHub, BitBucket and GitLab and default to git@ as the auth part if it is missing. I don't think these services will ever support custom usernames anyway (well, maybe GitLab but definitely not others).

Thoughts @Daniel15 @rally25rs?

[BYK]

The only "catch" is when people actually want to omit the username/auth part but I've just checked url.parse and if you do ssh://@github.com/la/lo.git it returns auth: '' whereas if you omit @ completely, it returns auth: null so we have a way to distinguish the two if we need to.

[NewEvolution]

I can confirm that modifying the URL to include ssh@ and/or a leading git+ did not change the behavior.

However, the steps outlined in this comment resolved the issue for me: #686 (comment)

[rally25rs]

@BYK I'm not opposed to adding git@ automatically if we are already doing some URL manipulation. Otherwise I would just say "if it doesn't work on the Git command line client, why would it work in Yarn?" ... so I guess that's not really much of an answer :)

If we can add back in the git@ in a simple change then 👍 but if it adds much complexity at all 👎

[scull7]

@rally25rs, why not just add a better error message with a helpful link? Or if yarn adds the git@ and it succeeds then should it update the package.json? Or at least issue a warning?

[rally25rs]

@scull7 the output is the actual output from the Git command, so it's as clear (or not as clear) as they made it. It isn't Yarn's error message. We might be able to capture exit code 128 and substitute our own error message.

[scull7]

@rally25rs the git error is clear about the permissions failure. I’m suggesting an augmentation to point the user into a better why direction. So, yes, if capturing the error code is possible then it would be good to provide some solution direction similar to Elm’s compiler.

[wanghanlin]

solution mentioned in #3942 by using ssh-add -l -E md5 seems able to fix the problem

[s-lee-kwong]

I am getting this error and none of the solutions above have helped

My scenario might be different from others though as I am on engineyard and using rails webpacker and yarn

[UmamaheshMaxwell]

In my case, it was due to passphrase , my SSH Keys generated using passphrase and it was throwing "Permission Denied" (Public Key) error, I re-generated the ssh keys without passphrase and tried again and got it working.

[purplecabbage]

I am having this issue too.
If I use npm i I get multiple prompts for my passphrase, if I sudo npm i I only need my passphrase once.
If I sudo yarn install it gets blocked at the second passphrase prompt

[sp3c1]

Had, same problem. Keys were not added to do ssh-agent for the user i was running it with.

@UmamaheshMaxwell would not recommend running keys without password.

[oussamajabnouni]

ssh-add -K worked here

[NewEvolution]

Ran into this issue with a coworker, turned out it was a mismatch between his username on his machine, and his username in Bitbucket.

Solved via adding a User line to ~/.ssh/config as follows:

Host bitbucket.org
  User bitbucket_username

[AaronSoap]

I also ran into this issue trying to get a site running in homestead. For me it seemed to be the fact that there is no entry for bitbucket in my known hosts. I ran this command first and it seemed to fix the issue for me.

ssh-keyscan bitbucket.org >> ~/.ssh/known_hosts

Likely opens me up to some man in the middle or something but it seems to work.

[folmert]

Anyone has figured it out on Windows?

Git's SSH Agent seems to suffer from sclerosis. I have to run "C:\Program Files\Git\cmd\start-ssh-agent.cmd" every single time (on each terminal instance) before I run yarn install, otherwise I get the same error.

Edit: even in the same terminal it seems to forget the SSH key after ~30 minutes.

[andrii-lazarenko]

ssh-add -K worked here

ssh-add -k for windows 10. Thanks.
-k Load only keys and not certificates.

[mith495]

ssh-add -K worked here

This worked for me.

[sinneren]

It seems you try to add remote package via sudo. But you have key of your user. But by trying with sudo you need similar key of root user, not your.

[wildandickya]

ssh-add -K worked here

this worked for me too

[itsjavi]

I had the same issue.
In the URL, make sure you use / before your username instead of : like I see in the issue description.

[clearbrian]

just had it today. I moved from home mac to work mac. Suddenly got ssl errors on yarn start..
Caused by git path in package.json changed from https to generic format github: .
changing it back to git+https://github.com/USERNAME/REPONAME fixed it
-    "cordova-plugin-twilio-video": "github:SeaChatGit/cordova-plugin-twilio-video",
+    "cordova-plugin-twilio-video": "git+https://github.com/SeaChatGit/cordova-plugin-twilio-video.git",

[irinaivanovaalex]

This link in Github Docs is very helpful for me: https://docs.github.com/en/github/authenticating-to-github/error-permission-denied-publickey#make-sure-you-have-a-key-that-is-being-used

# start the ssh-agent in the background
eval "$(ssh-agent -s)"
ssh-add -l -E sha256

[vantientu1703]

Remove yarn.lock file after run yarn install

[Saddam-tech]

This can actually be a problem with yarn not being able to find your ssh public key, usually yarn uses your public key even to install packages so you need to add your ssh key with ssh-add "path-to-your-ssh-public-key" and then try to install it again.

[Saddam-tech]

This can actually be a problem with yarn not being able to find your ssh public key, usually yarn uses your public key even to install packages so you need to add your ssh key with ssh-add and then try to install it again

if this doesn't help try checking your public key with pbcopy < ~/.ssh/id_rsa.pub (which copies your ssh public key to clipboard) and pasting it somewhere and compare it with the one you pasted in your github account ssh public keys it should match and should have your email that you registered to your github account at the end after " = " sign for example "ADAQABAAABgQDGF0pWlJiovfnfDM3uk4jek1F1BXYy/0Kq+FqD24JIhn17bhk7MCAml4qoFpc3XtmAlRNDrtEeJ84KwP8WTOhs75ZnSfZvESf4YwOUM40GSymiHIN1BYapT+Po21c8eL3x2445v0LpACev1amFKewWqx5BHqXJ9HO1usCTiWo9m4/KfBHDa339G888zkniID14IdpkvsylH4tDTdYSv60Si4Xzw5XvMa80KY/GOvmxP4nmR+NVtwKCO4lfrOyADvy1NDuyF5M374zFN0vczdp4OWKen2ofpRJRMDdz8mC8F9JLGpLjM6iLSs7q4XCitaOhskLmDsdfaCdUce+kNEwasdfLUgP5hWasdfHZrL3/ncbuHqZuKqHJSnCBNbifyPidcz3HOOwOvts3cuf5JOkPmwAasdfNZs44GX4SIgZg7nzdLg2b9fQdY+rE2TarKLVnvOm+J4u1NQ3tATaiDnUbBGOkIEzKkrOmNIr6B3OQosSn8WJGzqVgIIrPSQc/QCEeKsd5xOzW8uGuVH0= youremail@test.com"

[repeat]

This link in Github Docs is very helpful for me: https://docs.github.com/en/github/authenticating-to-github/error-permission-denied-publickey#make-sure-you-have-a-key-that-is-being-used

In my case, my ssh key was generated years ago and is not "strong" enough nowadays, so I can git clone my private repository without any problem, but cannot yarn install it.
My solution was to generate a new ssh key with 4096 bits, upload new keys to GitHub, and then it worked. 🎉