This repository has been archived by the owner. It is now read-only.

Failed to install git packages using cygwin #7357

Closed
itsnotvalid opened this Issue Feb 14, 2015 · 25 comments

Comments

Projects
None yet
@itsnotvalid

Did a node.js upgrade and tried recompiling native modules (which is the usuals after each upgrade) but greeted with the following error:

Enviornment:
node.js v0.12.0 win64
cygwin win64
git: cygwin latest version (2.1.4.0)
npm: default version on node.js v0.12.0 (2.5.1)

55 verbose git remote.origin.url https://github.com/itsnotvalid/node-plist-native.git
<redacted>
57 info git [ 'fetch', '-a', 'origin' ]
58 verbose git fetch -a origin (https://github.com/itsnotvalid/node-plist-native.git)
59 silly verifyOwnership skipping for windows
60 info git [ 'rev-list', '-n1', 'plist-json-bridge' ]
61 verbose git rev-list -n1 plist-json-bridge 374dd978d1c6a895f9e45bff958ce4aef2ccbe44
62 verbose resolved git url git+https://github.com/itsnotvalid/node-plist-native.git#374dd978d1c6a895f9e45bff958ce4aef2ccbe44
63 info git [ 'clone',
63 info git   'C:\\Users\\itsnotvalid\\AppData\\Roaming\\npm-cache\\_git-remotes\\https-github-com-itsnotvalid-node-plist-native-git-4e9eff69',
63 info git   'C:\\cygwin64\\tmp\\npm-148168-d3eacb24\\git-cache-b4419446b15c\\374dd978d1c6a895f9e45bff958ce4aef2ccbe44' ]
64 error Failed to clone git+https://github.com/itsnotvalid/node-plist-native.git#374dd978d1c6a895f9e45bff958ce4aef2ccbe44 from https://github.com/itsnotvalid/node-plist-native.git Cloning into 'C:\cygwin64\tmp\npm-148168-d3eacb24\git-cache-b4419446b15c\374dd978d1c6a895f9e45bff958ce4aef2ccbe44'...
64 error Failed to clone git+https://github.com/itsnotvalid/node-plist-native.git#374dd978d1c6a895f9e45bff958ce4aef2ccbe44 from https://github.com/itsnotvalid/node-plist-native.git fatal: '/cygdrive/c/Users/itsnotvalid/AppData/Roaming/npm-cache/_git-remotes/https-github-com-itsnotvalid-node-plist-native-git-4e9eff69/C:\Users\itsnotvalid\AppData\Roaming\npm-cache\_git-remotes\https-github-com-itsnotvalid-node-plist-native-git-4e9eff69' does not appear to be a git repository
64 error Failed to clone git+https://github.com/itsnotvalid/node-plist-native.git#374dd978d1c6a895f9e45bff958ce4aef2ccbe44 from https://github.com/itsnotvalid/node-plist-native.git fatal: Could not read from remote repository.
64 error Failed to clone git+https://github.com/itsnotvalid/node-plist-native.git#374dd978d1c6a895f9e45bff958ce4aef2ccbe44 from https://github.com/itsnotvalid/node-plist-native.git
64 error Failed to clone git+https://github.com/itsnotvalid/node-plist-native.git#374dd978d1c6a895f9e45bff958ce4aef2ccbe44 from https://github.com/itsnotvalid/node-plist-native.git Please make sure you have the correct access rights
64 error Failed to clone git+https://github.com/itsnotvalid/node-plist-native.git#374dd978d1c6a895f9e45bff958ce4aef2ccbe44 from https://github.com/itsnotvalid/node-plist-native.git and the repository exists.
65 verbose stack Error: Command failed: git -c core.longpaths=true clone C:\Users\itsnotvalid\AppData\Roaming\npm-cache\_git-remotes\https-github-com-itsnotvalid-node-plist-native-git-4e9eff69 C:\cygwin64\tmp\npm-148168-d3eacb24\git-cache-b4419446b15c\374dd978d1c6a895f9e45bff958ce4aef2ccbe44
65 verbose stack Cloning into 'C:\cygwin64\tmp\npm-148168-d3eacb24\git-cache-b4419446b15c\374dd978d1c6a895f9e45bff958ce4aef2ccbe44'...
65 verbose stack fatal: '/cygdrive/c/Users/itsnotvalid/AppData/Roaming/npm-cache/_git-remotes/https-github-com-itsnotvalid-node-plist-native-git-4e9eff69/C:\Users\itsnotvalid\AppData\Roaming\npm-cache\_git-remotes\https-github-com-itsnotvalid-node-plist-native-git-4e9eff69' does not appear to be a git repository
65 verbose stack fatal: Could not read from remote repository.
65 verbose stack
65 verbose stack Please make sure you have the correct access rights
65 verbose stack and the repository exists.
65 verbose stack
65 verbose stack     at ChildProcess.exithandler (child_process.js:744:12)
65 verbose stack     at ChildProcess.emit (events.js:110:17)
65 verbose stack     at maybeClose (child_process.js:1008:16)
65 verbose stack     at Process.ChildProcess._handle.onexit (child_process.js:1080:5)
66 verbose cwd C:\Users\itsnotvalid\<redacted>
67 error Windows_NT 6.3.9600
68 error argv "C:\\Program Files\\nodejs\\\\node.exe" "C:\\Program Files\\nodejs\\node_modules\\npm\\bin\\npm-cli.js" "install"
69 error node v0.12.0
70 error npm  v2.5.1
71 error code 128
72 error Command failed: git -c core.longpaths=true clone C:\Users\itsnotvalid\AppData\Roaming\npm-cache\_git-remotes\https-github-com-itsnotvalid-node-plist-native-git-4e9eff69 C:\cygwin64\tmp\npm-148168-d3eacb24\git-cache-b4419446b15c\374dd978d1c6a895f9e45bff958ce4aef2ccbe44
72 error Cloning into 'C:\cygwin64\tmp\npm-148168-d3eacb24\git-cache-b4419446b15c\374dd978d1c6a895f9e45bff958ce4aef2ccbe44'...
72 error fatal: '/cygdrive/c/Users/itsnotvalid/AppData/Roaming/npm-cache/_git-remotes/https-github-com-itsnotvalid-node-plist-native-git-4e9eff69/C:\Users\itsnotvalid\AppData\Roaming\npm-cache\_git-remotes\https-github-com-itsnotvalid-node-plist-native-git-4e9eff69' does not appear to be a git repository
72 error fatal: Could not read from remote repository.

Not sure why there are tmp folders involved, but it just doesn't seem to fetch the repo, then of course it won't do the gyp part.

Thanks.

@smikes

This comment has been minimized.

Show comment
Hide comment
@smikes

smikes Feb 15, 2015

Contributor

It looks like you're running into a problem where cygwin paths and Windows paths collide:

/cygdrive/c/Users/itsnotvalid/AppData/Roaming/npm-cache/_git-remotes/https-github-com-itsnotvalid-node-plist-native-git-4e9eff69/C:\Users\itsnotvalid\AppData\Roaming\npm-cache\_git-remotes\https-github-com-itsnotvalid-node-plist-native-git-4e9eff69' does not appear to be a git repository 65 verbose stack fatal: Could not read from remote repository.

As for the tmp folders, when npm installs from a git repository, it first clones the repository to a tmp folder, then extracts the desired tag/branch/sha to the npm cache, and then installs from the cache to the desired location. So it can be substantially easier to publish an npm package than to use a git dependency.

I believe the official stance of the npm project is that cygwin is not supported. I use cygwin myself when I'm using Windows, so I'm sympathetic to the difficulties that you're seeing; but my advice would be to try using npm in a Command Prompt or other native Window shell, and to use the git-for-windows distributed as msysGit or the github windows distribution (which comes with a shortcut for git shell that can be set up to use bash)

Contributor

smikes commented Feb 15, 2015

It looks like you're running into a problem where cygwin paths and Windows paths collide:

/cygdrive/c/Users/itsnotvalid/AppData/Roaming/npm-cache/_git-remotes/https-github-com-itsnotvalid-node-plist-native-git-4e9eff69/C:\Users\itsnotvalid\AppData\Roaming\npm-cache\_git-remotes\https-github-com-itsnotvalid-node-plist-native-git-4e9eff69' does not appear to be a git repository 65 verbose stack fatal: Could not read from remote repository.

As for the tmp folders, when npm installs from a git repository, it first clones the repository to a tmp folder, then extracts the desired tag/branch/sha to the npm cache, and then installs from the cache to the desired location. So it can be substantially easier to publish an npm package than to use a git dependency.

I believe the official stance of the npm project is that cygwin is not supported. I use cygwin myself when I'm using Windows, so I'm sympathetic to the difficulties that you're seeing; but my advice would be to try using npm in a Command Prompt or other native Window shell, and to use the git-for-windows distributed as msysGit or the github windows distribution (which comes with a shortcut for git shell that can be set up to use bash)

@itsnotvalid

This comment has been minimized.

Show comment
Hide comment
@itsnotvalid

itsnotvalid Feb 15, 2015

@smikes what puzzled is that it worked perfectly before upgrading to v0.12 (I was living under v0.11 before that) so It just sounded like a bug than something on that path. Thanks for your advice though.

@smikes what puzzled is that it worked perfectly before upgrading to v0.12 (I was living under v0.11 before that) so It just sounded like a bug than something on that path. Thanks for your advice though.

@smikes

This comment has been minimized.

Show comment
Hide comment
@smikes

smikes Feb 15, 2015

Contributor

node 0.12 came with a newer version of npm, which has had some bugfixes regarding git on Windows. This may have caused your setup to break, unfortunately.

/cc @othiym23 any comment regarding the cygwin-git issue?

Contributor

smikes commented Feb 15, 2015

node 0.12 came with a newer version of npm, which has had some bugfixes regarding git on Windows. This may have caused your setup to break, unfortunately.

/cc @othiym23 any comment regarding the cygwin-git issue?

@othiym23

This comment has been minimized.

Show comment
Hide comment
@othiym23

othiym23 Feb 19, 2015

Contributor

As always – and I hate to be a broken record about this, but it does bear repeating – there is no support for npm on cygwin. You're free to get it running, and we won't ever do anything consciously or explicitly to break Cygwin, but you're on your own. Unfortunately, the rationale behind that lack of support appears to be germane in this case, because it looks like turning on core.longpaths in git (which is necessary to address a number of issues with git on Windows) breaks git on Cygwin. The underlying issue was a longstanding pain point for Windows users of npm, so unless a solution can be proposed that also supports that fix, we're unlikely to accept it.

If you want POSIX-like behavior on Windows, I suggest looking at either using Git Bash, or a full mingw installation. npm works fine under mingw, and is supported.

Contributor

othiym23 commented Feb 19, 2015

As always – and I hate to be a broken record about this, but it does bear repeating – there is no support for npm on cygwin. You're free to get it running, and we won't ever do anything consciously or explicitly to break Cygwin, but you're on your own. Unfortunately, the rationale behind that lack of support appears to be germane in this case, because it looks like turning on core.longpaths in git (which is necessary to address a number of issues with git on Windows) breaks git on Cygwin. The underlying issue was a longstanding pain point for Windows users of npm, so unless a solution can be proposed that also supports that fix, we're unlikely to accept it.

If you want POSIX-like behavior on Windows, I suggest looking at either using Git Bash, or a full mingw installation. npm works fine under mingw, and is supported.

@mathieucarbou

This comment has been minimized.

Show comment
Hide comment
@mathieucarbou

mathieucarbou May 11, 2015

FYI, I was able to make npm install works by patching the file node_modules\npm\lib\utils\git.js:

function execGit (args, options, cb) {
  if(args) {
      for(var i=0; i<args.length; i++) {
          if('C:\\'.indexOf(args[i]) != 0) {
              args[i] = args[i].replace(/\\/g, '/').replace(/C\:\//i, '/cygdrive/c/');
          }
      }
  }
  var fullArgs = prefixGitArgs().concat(args || [])
  log.info('git', fullArgs)
  return exec(git, fullArgs, options, cb)
}

The patch works for me but it could require some change to work for you. I am running node & npm on cygwin:

$ node -v && npm -v
v0.12.2
2.9.1

FYI, I was able to make npm install works by patching the file node_modules\npm\lib\utils\git.js:

function execGit (args, options, cb) {
  if(args) {
      for(var i=0; i<args.length; i++) {
          if('C:\\'.indexOf(args[i]) != 0) {
              args[i] = args[i].replace(/\\/g, '/').replace(/C\:\//i, '/cygdrive/c/');
          }
      }
  }
  var fullArgs = prefixGitArgs().concat(args || [])
  log.info('git', fullArgs)
  return exec(git, fullArgs, options, cb)
}

The patch works for me but it could require some change to work for you. I am running node & npm on cygwin:

$ node -v && npm -v
v0.12.2
2.9.1
@mathieucarbou

This comment has been minimized.

Show comment
Hide comment
@mathieucarbou

mathieucarbou May 11, 2015

update:

Here are a patched git.js plus the diff that should work regardless of the drive letter.

update:

Here are a patched git.js plus the diff that should work regardless of the drive letter.

@fsworld009

This comment has been minimized.

Show comment
Hide comment
@fsworld009

fsworld009 May 20, 2015

@mathieucarbou thanks for the effort, I'm using msys2 bash and msys2 git, I also had the same issue, so I forked your git.js and add support for msys2:
https://gist.github.com/fsworld009/bd8baaeccf884e776790

@mathieucarbou thanks for the effort, I'm using msys2 bash and msys2 git, I also had the same issue, so I forked your git.js and add support for msys2:
https://gist.github.com/fsworld009/bd8baaeccf884e776790

@amb26

This comment has been minimized.

Show comment
Hide comment
@amb26

amb26 Aug 5, 2015

Thanks, @mathieucarbou - your patch very nearly works for me, but the ORIGINAL_PATH variable isn't defined on my system, perhaps testing something like PWD would be more reliable.

amb26 commented Aug 5, 2015

Thanks, @mathieucarbou - your patch very nearly works for me, but the ORIGINAL_PATH variable isn't defined on my system, perhaps testing something like PWD would be more reliable.

@jesconstantine

This comment has been minimized.

Show comment
Hide comment
@jesconstantine

jesconstantine Aug 11, 2015

Thanks @mathieucarbou your patch worked for me as well!

Thanks @mathieucarbou your patch worked for me as well!

@xinshangshangxin

This comment has been minimized.

Show comment
Hide comment
@xinshangshangxin

xinshangshangxin Jan 6, 2016

Thanks @mathieucarbou your patch worked for me as well!

Thanks @mathieucarbou your patch worked for me as well!

@technopagan

This comment has been minimized.

Show comment
Hide comment
@technopagan

technopagan Jan 29, 2016

Thank you @mathieucarbou! After days of trying to debug this, I found your patch & it's working perfectly. You saved my dev-environment!

Thank you @mathieucarbou! After days of trying to debug this, I found your patch & it's working perfectly. You saved my dev-environment!

@ichadhr

This comment has been minimized.

Show comment
Hide comment
@ichadhr

ichadhr Jan 31, 2016

@mathieucarbou your patch worked for me !!

ichadhr commented Jan 31, 2016

@mathieucarbou your patch worked for me !!

@htaox

This comment has been minimized.

Show comment
Hide comment
@htaox

htaox Feb 27, 2016

@mathieucarbou you the man. The patch works great!

htaox commented Feb 27, 2016

@mathieucarbou you the man. The patch works great!

@DiThi

This comment has been minimized.

Show comment
Hide comment
@DiThi

DiThi Mar 26, 2016

@mathieucarbou 's patch worked perfectly.

DiThi commented Mar 26, 2016

@mathieucarbou 's patch worked perfectly.

@Doormouse2House

This comment has been minimized.

Show comment
Hide comment

+1 @mathieucarbou - ta muchly,

@bsclifton

This comment has been minimized.

Show comment
Hide comment
@bsclifton

bsclifton Apr 14, 2016

@othiym23 Any reason the patch by @mathieucarbou can't be incorporated?

It would be awesome to not have to switch between Cygwin and GitHub's shell

bsclifton commented Apr 14, 2016

@othiym23 Any reason the patch by @mathieucarbou can't be incorporated?

It would be awesome to not have to switch between Cygwin and GitHub's shell

@amb26

This comment has been minimized.

Show comment
Hide comment
@amb26

amb26 Apr 14, 2016

+1 from me, I've been manually patching each version of npm I upgrade to to incorporate it

amb26 commented Apr 14, 2016

+1 from me, I've been manually patching each version of npm I upgrade to to incorporate it

@bsclifton

This comment has been minimized.

Show comment
Hide comment
@bsclifton

bsclifton Apr 15, 2016

Created #12366 incorporating the patch by @mathieucarbou, ready for review 😄

Created #12366 incorporating the patch by @mathieucarbou, ready for review 😄

@mrboli

This comment has been minimized.

Show comment
Hide comment
@tw3

This comment has been minimized.

Show comment
Hide comment
@tw3

tw3 Apr 19, 2017

To be clear the full path to git.js on my Windows 7 machine (and possibly yours) is C:\Program Files\nodejs\node_modules\npm\lib\utils\git.js. Applying @mathieucarbou's fix worked for me.

tw3 commented Apr 19, 2017

To be clear the full path to git.js on my Windows 7 machine (and possibly yours) is C:\Program Files\nodejs\node_modules\npm\lib\utils\git.js. Applying @mathieucarbou's fix worked for me.

@vilemj-Viclick

This comment has been minimized.

Show comment
Hide comment
@vilemj-Viclick

vilemj-Viclick Aug 1, 2017

@mathieucarbou Thanks a lot. Applied the fix to npm@5.2.0. Still works.

@mathieucarbou Thanks a lot. Applied the fix to npm@5.2.0. Still works.

@geekyme

This comment has been minimized.

Show comment
Hide comment
@geekyme

geekyme Nov 20, 2017

I patched by directly editing the C:\Program Files\nodejs\node_modules\npm\lib\utils\git.js as suggested by @mathieucarbou, but it didn't work as shown by npm install. Do I have to refresh some cache?

$ npm install
npm ERR! code 128
npm ERR! Command failed: C:\workspace\ext\cygwin\bin\git.EXE clone --depth=1 -q -b 3.0.0 https://github.com/elm-node/core.git C:\Users\boo\AppData\Roaming\npm-cache\_cacache\tmp\git-clone-17ad0817 --config core.longpaths=true
npm ERR! fatal: Invalid path '/cygdrive/c/Users/boo/AppData/Roaming/npm-cache/_cacache/tmp/git-clone-17ad0817/C:\Users\boo\AppData\Roaming\npm-cache\_cacache\tmp\git-clone-17ad0817': No such file or directory
npm ERR!

npm ERR! A complete log of this run can be found in:
npm ERR!     C:\Users\boo\AppData\Roaming\npm-cache\_logs\2017-11-20T08_02_41_383Z-debug.log

geekyme commented Nov 20, 2017

I patched by directly editing the C:\Program Files\nodejs\node_modules\npm\lib\utils\git.js as suggested by @mathieucarbou, but it didn't work as shown by npm install. Do I have to refresh some cache?

$ npm install
npm ERR! code 128
npm ERR! Command failed: C:\workspace\ext\cygwin\bin\git.EXE clone --depth=1 -q -b 3.0.0 https://github.com/elm-node/core.git C:\Users\boo\AppData\Roaming\npm-cache\_cacache\tmp\git-clone-17ad0817 --config core.longpaths=true
npm ERR! fatal: Invalid path '/cygdrive/c/Users/boo/AppData/Roaming/npm-cache/_cacache/tmp/git-clone-17ad0817/C:\Users\boo\AppData\Roaming\npm-cache\_cacache\tmp\git-clone-17ad0817': No such file or directory
npm ERR!

npm ERR! A complete log of this run can be found in:
npm ERR!     C:\Users\boo\AppData\Roaming\npm-cache\_logs\2017-11-20T08_02_41_383Z-debug.log

@cyantree

This comment has been minimized.

Show comment
Hide comment
@cyantree

cyantree Jan 1, 2018

Git functionality apparently has moved to pacote. Applying the fix there seems to work (tested with NPM 5.6):
C:\Users\[USER]\AppData\Roaming\npm\node_modules\npm\node_modules\pacote\lib\util\git.js

function execGit (gitArgs, gitOpts, opts) {
  opts = optCheck(opts)
  if (gitArgs) {
    gitArgs = gitArgs.map(gitArg => gitArg.indexOf('C:\\') === -1 ? gitArg : gitArg.replace(/\\/g, '/').replace(/C\:\//i, '/cygdrive/c/'));
  }
  
  return checkGit().then(gitPath => {
    return execFileAsync(gitPath, gitArgs, mkOpts(gitOpts, opts))
  })
}

cyantree commented Jan 1, 2018

Git functionality apparently has moved to pacote. Applying the fix there seems to work (tested with NPM 5.6):
C:\Users\[USER]\AppData\Roaming\npm\node_modules\npm\node_modules\pacote\lib\util\git.js

function execGit (gitArgs, gitOpts, opts) {
  opts = optCheck(opts)
  if (gitArgs) {
    gitArgs = gitArgs.map(gitArg => gitArg.indexOf('C:\\') === -1 ? gitArg : gitArg.replace(/\\/g, '/').replace(/C\:\//i, '/cygdrive/c/'));
  }
  
  return checkGit().then(gitPath => {
    return execFileAsync(gitPath, gitArgs, mkOpts(gitOpts, opts))
  })
}
@lainlee

This comment has been minimized.

Show comment
Hide comment
@lainlee

lainlee Jan 15, 2018

Thank @mathieucarbou and @cyantree , the patch works for me

I met this error when I use cordova and add a plugin from git , it takes me whole day to find solution

By the way , my path of git.js is

C:\Program Files\nodejs\node_modules\npm\node_modules\pacote\lib\util\git.js

lainlee commented Jan 15, 2018

Thank @mathieucarbou and @cyantree , the patch works for me

I met this error when I use cordova and add a plugin from git , it takes me whole day to find solution

By the way , my path of git.js is

C:\Program Files\nodejs\node_modules\npm\node_modules\pacote\lib\util\git.js

@ashleyvidetto

This comment has been minimized.

Show comment
Hide comment
@ashleyvidetto

ashleyvidetto Apr 24, 2018

Just another person thanking both @mathieucarbou and @cyantree for creating the fix and the change w/ pacote. Able to use cygwin now :).

Just another person thanking both @mathieucarbou and @cyantree for creating the fix and the change w/ pacote. Able to use cygwin now :).

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.