Patch for issue 7357, courtesy of @mathieucarbou

[bsclifton]

See #7357 (comment) for the patch by @mathieucarbou. This PR is me manually applying the patch, which has slight changes.

I confirmed it works great by testing locally (before and after). Before npm install would fail, after it works great (see other comments in the issue for reports of +1s)

[othiym23]

As I said when I closed #7357:

As always – and I hate to be a broken record about this, but it does bear repeating – there is no support for npm on cygwin. You're free to get it running, and we won't ever do anything consciously or explicitly to break Cygwin, but you're on your own. Unfortunately, the rationale behind that lack of support appears to be germane in this case, because it looks like turning on core.longpaths in git (which is necessary to address a number of issues with git on Windows) breaks git on Cygwin. The underlying issue was a longstanding pain point for Windows users of npm, so unless a solution can be proposed that also supports that fix, we're unlikely to accept it.

If you want POSIX-like behavior on Windows, I suggest looking at either using Git Bash, or a full mingw installation. npm works fine under mingw, and is supported.

This is still true. npm explicitly doesn't offer support for Cygwin. For more context, this chunk from this week's npm CLI team meeting (link goes to correct spot in video) is relevant, where I discuss the fact that npm doesn't (yet) support being used in the Ubuntu compatibility environment on Windows. Because npm supports both Windows and Unix, and does so in different ways, it's tricky for it to support environments that try to be both. Cygwin makes it especially hard.

If you want to apply that patch to your own copy of npm, that's fine, but in the form it's in here, it's unlikely to be merged into npm proper. Either way, thanks for your time and your understanding!

[bsclifton]

Like your quoted post says (which I read beforehand), this PR keeps core.longpaths on for Windows users, so this solution supports that fix... which is why I thought that it might be considered.

core.longpaths has nothing to do with the actual fix, which is this line:
https://github.com/npm/npm/pull/12366/files#diff-7a3a2c594d26c88926c09e40c70f46ceR25

[mathieucarbou]

@othiym23 : sad to see that such little modification helping a lot of users is not welcomed.

I didn't see it as a request to support cygwin. I much more see it as making life better for a lot of people that will be happy to see this patch included and will be able to continue working with their environment they like, EVEN IF you do not officially support it.

In my opinion, it's not because something is not officially supported and documented that it shouldn't be there... It's up to the users to know that if they use an unsupported env, then it is up to them. But at least, it will make life easier for these people ;-)

I agree Cygwin is difficult to entirely support, and I fear it won't be possible. But there is a lot of small things like this one that would help solving 80% of the issues people have in Cygwin. So it's not a big demand ;-)

[othiym23]

In my opinion, it's not because something is not officially supported and documented that it shouldn't be there... It's up to the users to know that if they use an unsupported env, then it is up to them. But at least, it will make life easier for these people ;-)

If it's in code that's published and distributed by the project, it's de facto supported. We're responsible for dealing with the consequences of shipping changes, and although this does handle the case of longpaths, it also adds conditional code related to Cygwin, and based on the wide variety of failure modes that I've seen Cygwin users having on Windows (users using straight Cygwin, users using cmd.exe with Cygwin git, users using Git bash with Cygwin git (???)), I'm not comfortable taking that complexity into the CLI. The team is very close to having npm's test suite running on Windows (with CI), but even then, to properly ensure that this change didn't cause complications for Windows users, including making it more difficult them to figure out what's going on when they find themselves in one of the edge case situations above, we'd need to have an even more complicated test matrix that involves installing Cygwin and various other permutations. In my judgment, the change is not worth the cost.

Your patch is available for people who want to make it work for themselves, and I believe that the act of applying that patch signals a willingness to accept the complications that result far more effectively than the project landing it and then not documenting it.

[bsclifton]

Since this PR has a very explicit check for Cygwin, it's pretty safe to say there is not going to be any impact to non-Cygwin users. Win32 is even being detected the same way as the original npm code, via process.platform === 'win32' (it's just being stored in a variable, so it can be re-used for the Cygwin check; non-Cygwin shells should never have a /cygdrive/).

I am with @mathieucarbou in stating that this will help make lives easier. I really enjoy trying to make open source contributions, even if unsuccessful. And as a Windows dev, I can fully sympathize with folks that have a bad time, because I live that every day. Every step forward helps, even if it's a small one 😄

[bsclifton]

cc'ing the folks that also encountered the issue and were savvy enough to find the patch on GitHub
@amb26
@Doormouse2House
@DiThi
@htaox
@ichadhr
@technopagan
@xinshangshangxin
@jesconstantine
@fsworld009

[bsclifton]

I went back and fixed the errors reported by travis-ci.org (didn't see those until now, definitely my fault). Are there other repo owners / contributors that can consider re-opening this? Having this contribution rejected feels like an arbitrary company mandate by npm, Inc employees (@isaacs, @othiym23). Non-employees should have a say also, IMO.

[othiym23]

Having this contribution rejected feels like an arbitrary company mandate by npm, Inc employees (@isaacs, @othiym23).

I'll discuss this PR during the npm CLI team meeting on 2016-04-19, but to speak to this specific point, when @isaacs wrote that commit back in 2011, npm, Inc. wasn't even an idea yet. Back then, Isaac was more or less the sole developer and maintainer of npm, which in practice meant he was solely responsible for supporting its users. He passed responsibility for maintaining the CLI on to me after hiring me into the newly established npm, Inc., and similarly, I and my team have inherited the responsibility for support.

Supportability is a primary design criterion for the CLI, because npm is a very powerful, flexible, and complicated tool. (Supporting it is hard.) Most of the time this means putting thought into designing the user experience and effectively communicating to users what's happened (often, what's gone wrong). Occasionally, it means choosing between tradeoffs rooted in the conflicting needs of different segments of the user base. The team's decision to not support Cygwin is an example of the latter. I understand that it seems frustrating and unresponsive to you. It's a decision made with the best interests of the community as a whole in mind, though, and wasn't, and isn't, an arbitrary one.

[bsclifton]

I definitely do appreciate the work you all do; being able to revisit this topic (this thread and then the meeting tomorrow) is a great thing, regardless of the outcome, IMO 😄

My proposal for tomorrow's discussion would be:

changing from a stance of "No." to "Cygwin is not supported, use at your own risk."

meaning...

• Cygwin is not officially supported (ex: issues opened will be closed)
• Cygwin related changes, if accepted, must be completely isolated

  • A standardized test for Cygwin can be put in npm.js (or other suitable place)

    npm.isCygwin = function () {
      return process.platform === "win32" && 
      (process.env.ORIGINAL_PATH || '').indexOf('/cygdrive/') != -1
    }

  • Any PRs regarding Cygwin must use that test to isolate the changes

[Doormouse2House]

HIHIHIHIH

I sort of feel like I accidentally fell into this issue by nature of me stumbling across the Cygwin issue - as such, I have no real desire to see this patch added or not. Maybe, if you'll permit me, this allows me to comment on both sides of this discussion. Normally I wouldn't bother, however like @othiym23 I have spent years supporting and leading teams that support software. It's a tough job, with very little reward, and I have seen first-hand how fixing an issue in an unsupported area of the software can lead to the floodgates opening & every tom, dick & harry wanting a "free fix".

@othiym23, given @bsclifton's last post, and the commitment to a non-support of Cygwin, can you comment on what your concern is about allowing the community-driven fix? Is it the floodgates scenario I mentioned? If so, is there anything that can be said or done to reassure the core dev team that this change would not be a precursor to official Cygwin support?
From where I sit, it does feel as though @bsclifton's enthusiasm could be a positive thing to harness - maybe you want to support Cygwin Brian??? :)

Just my tuppence; feel free to dismiss me as an irrelevant mad-man...

[bsclifton]

I don't think it would ever make sense to have Cygwin officially supported. By that, I mean that contributors for this repo should close Cygwin-specific issues instead of actively looking for fixes to resolve them. Like @othiym23 has shown, there are already a large number of outstanding issues and this would blow up the scope.

If people found a resolution for a Cygwin-specific issue themselves and submitted it in the form of a PR though... I'd be more than happy to review it 😀

[DiThi]

@othiym23 Wouldn't it better to accept the patch but warn about the non supported situation? It's much better to have a giant message in red letters "CYGWIN NOT SUPPORTED, MAY BREAK AT ANY MOMENT, USE AT YOUR OWN RISK" than having random error messages sometimes? It did work for me without the patch for a while and if I was warned beforehand it would have made things easier.

@bsclifton @Doormouse2House What does imply supporting cygwin officially (or at least a working "run at your own risk" stance)? Running many tests locally for each alpha/beta release and fix it when it breaks? Does anyone else want to join efforts? @mathieucarbou ?

[mathieucarbou]

I don't think cygwin can be completely supported. There are some npm dependencies requiring some native compiling for example at npm install.
So a complete support would mean also to develop also some cygwin libraries perhaps. Which is another issue.

[DiThi]

@mathieucarbou that and many other issues may be worked around by calling CMD with the user's env variables instead of cygwin's. I.e. only using git from cygwin/msys/git bash, but using the rest of the supported tools (VS, python for windows...)

[demisx]

Cygwin is a very popular package on Windows and having npm not supporting it is a big deal. We've moved away from Git bash to Cygwin due to numerous issues. If major patches like that could be incorporated it would be a huge plus.

[amb26]

I don't care if npm continues to advertise that Cygwin is not supported, so long as they merge the patch. Given that the community widely reports that this patch is good and useful, one would hope that this would be evidence enough that it is a worthwhile addition to the codebase.
I'm perfectly happy for Cygwin to continue to have "run at your own risk" status. Everyone who uses Cygwin understands that anyway :)

[mathieucarbou]

@bsclifton : is it possible that you update the patch to fix the build syntax check ?

https://travis-ci.org/npm/npm/jobs/123364024

  /home/travis/build/npm/npm/lib/utils/git.js:14:34: Strings must use singlequote.
  /home/travis/build/npm/npm/lib/utils/git.js:15:79: Expected '!==' and instead saw '!='.
  /home/travis/build/npm/npm/lib/utils/git.js:18:19: Strings must use singlequote.
  /home/travis/build/npm/npm/lib/utils/git.js:18:25: Strings must use singlequote.
  /home/travis/build/npm/npm/lib/utils/git.js:22:3: Keyword "if" must be followed by whitespace.
  /home/travis/build/npm/npm/lib/utils/git.js:23:5: Keyword "for" must be followed by whitespace.
  /home/travis/build/npm/npm/lib/utils/git.js:23:14: Infix operators must be spaced.
  /home/travis/build/npm/npm/lib/utils/git.js:23:19: Infix operators must be spaced.
  /home/travis/build/npm/npm/lib/utils/git.js:24:7: Keyword "if" must be followed by whitespace.
  /home/travis/build/npm/npm/lib/utils/git.js:24:33: Expected '!==' and instead saw '!='.
  /home/travis/build/npm/npm/lib/utils/git.js:25:90: Extra semicolon.

Should be ok with:

var win32 = process.platform === 'win32'
var cygwin = win32 && (process.env.ORIGINAL_PATH || '').indexOf('/cygdrive/') !== -1

function prefixGitArgs () {
  return win32 ? ['-c', 'core.longpaths=true'] : []
}

function execGit (args, options, cb) {
  if (cygwin && args) {
    for (var i = 0; i < args.length; i++) {
      if (':\\'.indexOf(args[i]) !== 1) {
        args[i] = args[i].replace(/\\/g, '/').replace(/^([A-Za-z])\:\//, '/cygdrive/$1/')
      }
    }
  }
  log.info('git', args)
  var fullArgs = prefixGitArgs().concat(args || [])
  return exec(git, fullArgs, options, cb)
}

[bsclifton]

@mathieucarbou I already did actually- but since PR was marked as closed, it never picked it up 😢
https://travis-ci.org/bsclifton/npm/builds/123465028

[othiym23]

At today's CLI team meeting, I spent about 10 minutes addressing this issue. The summary is this: npm is not going to accept patches to support Cygwin, quietly or otherwise. This thread contains several examples of the hazards that can arise when doing so – it's impossible (and illogical) to completely disavow support for a set of features while still having code that conditionally takes advantages of those features.

I do think there are two solid avenues for those of you who want a usable version of npm in Cygwin, though:

1. Take this and other Cygwin-specific tweaks and turn them into a floating patch. Potentially take that patch and use it to produce a Cygwin package. This is far from unusual in the npm world – Homebrew has made many changes large and small to npm over the years to make npm work well within its constraints, and the other downstream distributions (such as Fedora and Debian) make other changes, for which they're responsible, to npm to make it fit the needs of their distributions.
2. Fork npm and maintain these patches there. At any point in time, there are 10-15 forks of npm with various tweaks and changes in them, and this is something we at the team acknowledge and encourage.

Either way, that (again) makes clear to consumers and potential contributors who to look to for help and support, and lets you make whatever changes you think desirable to make npm work better with Cygwin.

To be as explicit as possible: the CLI team isn't going to merge this patch, and won't merge any patch that's intended to solely change the behavior of npm for Cygwin. The video above explains our reasoning. I hope you find these explanations useful, and I thank you all for your understanding.

[silverwind]

Ultimately, I think a lot of these issues boil down to git being unable to handle Windows-style paths. Cygwin has some sort of compatibilty layer for certain binaries, as you can cat c:\\file.txt but you can't git add c:\\file.txt. As for alternatives to this patch, I see two possible approaches:

• Get a patch into Cygwin's git package.
• Get Node.js to return linux from process.platform on Cygwin. This would require it to be built on Cygwin's toolchain.

A wrapper script around git has also been suggested, but I think that's just another bandaid solution. The second approach will surely break a lot of things (native Node.js extensions come to mind), but it'd probably be the right thing to do given that Cygwin is trying so hard to appear like Linux.

Feel free to join the discussion on the Cygwin mailing list:

• https://cygwin.com/ml/cygwin/2016-04/msg00474.html
• https://cygwin.com/ml/cygwin/2016-04/msg00510.html

[bsclifton]

I don't really have any more comments regarding this PR or the issue (definitely appreciate other folks taking interest though 😃)- I'm taking part in the Cygwin mailing list to work on a resolution independent of npm.

But I am curious if npm's CLI collects platform/shell information when packages are requested. I'd be super interested to see an analytics breakdown of Windows users by shell. It could make a great blog post if that information is available.

I looked at the mingw project (which is a fork of Cygwin, BTW), but it hasn't been updated since 2013 and the docs are all from mid or early 2000's. I'm having a hard time believing that more people use mingw than Cygwin, which is why seeing data would be cool.

[silverwind]

Submitted a patch for Windows-style path compatibilty to Cygwin's git maintainer:

cygporter/git#25

[youurayy]

Cygwin is massive (see all the download mirrors) as Windows without a proper shell is much less usable for developers. MS tried to fix it with PowerShell and they failed, now they are going with full Linux shell support.

At the same time, having read all the threads,

1. it seems that this should be fixed in the Git upstream <insert Linus-never-gonna-happen song here:)> since it would be a pain for both NPM and Cygwin maintainers,
2. Linux is coming to Windows so this is potentially moot from then onwards (let's hope so).

Apart from installing from repo URLs, NPM works perfectly for me on Cygwin, and while installation of NPM under Cygwin is not supported (and doesn't need to be), the operation of NPM under Cygwin already works 99% so why not push it to full 100%, bend a rule here and there.

[ankostis]

Cygwin is massive

Indeed!
But I can also understand the npm developers wanting to avoid the burden of supporting one more git-platform.
We have to somehow offload this duty from them.

MS tried to fix it with PowerShell and they failed, now they are going with full Linux shell support.

Not surer if this will work at all.
The new "ubuntu" bash is an isolated "world", probably you would not be able to invoke non apt-get installed commands.
That means, invoking standalone npm from "ubuntu" bash will not work!

Will install ubuntu-bash later today when I get back home,. and check that for my self.

[technopagan]

@ankostis:

[ankostis]

Thanks for the quick feedback +1:

which npm ??
(the "ubuntu" one or the Windows release?)

[edit: ] Did it my self, indeed, you may run only the WSL binaries, not from the Windows releases; yet, reusing the same binaries from linux, is joyful & creepy at the same time :-)

[emigenix]

Jesus! I can't believe all these discussions over merging a trivial patch that would have helped all Cygwin users, using npm. Instead you will have to continue discussing these issues for the rest of the lifetime of npm, as Cygwin users will not disappear anytime soon, not even with the emerging native Ubuntu bash (which has it's own npm issues.) It's really sad to see the npm folks blatantly denying the wide use of Cygwin. After all, Cygwin is the only reason developers still bother to use Windows. ;)

[bsclifton]

@emigenix I agree that this patch could have helped. I championed it for a while and after attending their weekly meeting, it's safe to say Cygwin will never be supported by them.

In the meantime, besides applying the patch, you can try redefining git in your bashrc:

function git {
    declare -a ARGS
    for n in "$@" ; do ARGS+=("$(cygpath -u -- "${n}")") ; done
        command git "${ARGS[@]}"
    }
}

I didn't get a chance to finish it... it works as long as the paths are quoted. Shout-outs to Eliot Moss on the Cygwin mailing list for suggesting this and also giving me a rev1 to work with.

[demisx]

@bsclifton Being in Cygwin world and switching to Git Bash just to run npm install really sucks! Is this new git function supposed to be added to the ~/.bashrc file? I've tried it, but got the following error:

sh: /home/dmoore/.bashrc: line 7: syntax error near unexpected token `('
sh: /home/dmoore/.bashrc: line 7: `  for n in "$@" ; do ARGS +=("$(cygpath -u -- "${n}")") ; done'

My current ~/.bashrc:

$ cat ~/.bashrc                  
---                                                                                                              
source ~/.git-prompt.sh                                                             
export PS1='\e[0;32m\u@\h:\e[m\W\e[1;33m`__git_ps1 " (%s)"`\e[m\n$ '                

function git {                                                                      
   declare -a ARGS                                                                  
   for n in "$@" ; do ARGS +=("$(cygpath -u -- "${n}")") ; done                     
      command git "${ARGS[@]}"                                                      
   }                                                                                
}                                                                                   

Thanks.

[bsclifton]

@demisx yeah, it does suck having to switch back and forth. There are other times where you have no choice (for project to build using Visual Studio vs GCC)

Apologies for not finishing the method above. All these months later, I've switched to Mac as my primary OS. But you should be able test/tweak it using the npm install line that breaks (likely a clone)

[youurayy]

I don't know the details of the "npm problems under windows ubuntu native emulator" but I think all npm-on-windows efforts should now be focused there.
"If your quary goes to ground, leave no ground to go to."

[ankostis]

I don't know the details of the "npm problems under windows ubuntu native emulator" but...

WSL bash is a lightweight virtualized environment to execute ELF binaries (ie Ubuntu).
As such, it cannot interact with the host OS(!) except through network; in particular, it cannot launch any Windows executable, so you CANNOT use Visual Studio to compile native binaries.
Is that problem serious enough?

[youurayy]

Well that sucks royally. Count me back into the cygwin-for-the-masses camp.

[silverwind]

This works for me in zsh, I assume it should work in bash too:

function git {
  declare -a ARGS
  for n in "$@"; do ARGS+=("$(cygpath -u -- "${n}")"); done
  command git "${ARGS[@]}"
}

You can test it with a local clone:

$ git clone c:\\some\\repo somedir

[demisx]

@bsclifton Yeah, I am on the same bandwagon. My go-to OS for everything is Mac OSX. Unfortunately, we still have some client projects that require us to run Visual Studio, .NET and the whole myriad of other Microsoft goodies.

@silverwind Thanks, that made the error go away, but npm install still fails with the same problem as before:

...
npm ERR! git clone C:\Users\dmoore\AppData\Roaming\npm-cache\_git-remotes\git-github-com-gulpjs-gulp-git-4-0-ecf98f08 C:\cygwin64\tmp\npm-56
12-aedffa10\git-cache-8629f384\d8f5c90a0622d19ef1943a2a3d02dc50e3c853e7: fatal: '/cygdrive/c/Users/dmoore/AppData/Roaming/npm-cache/_git-rem
otes/git-github-com-gulpjs-gulp-git-4-0-ecf98f08/C:\Users\dmoore\AppData\Roaming\npm-cache\_git-remotes\git-github-com-gulpjs-gulp-git-4-0-e
cf98f08' does not appear to be a git repository
...

[silverwind]

npm install still fails

Yeah that's because npm does not see the shell function, you actually have to create a wrapper script. Now it would be handy if npm config --global set git somescript would work as expected, but because of some weird bug in npm or its dependencies, it can't handle a git binary that's not named git it seems.

Enter this hack:

$ mv /bin/git /bin/gitbin
$ vim /bin/git

#!/bin/bash
declare -a ARGS
for arg in "$@"; do
  if [[ $arg == *"\\"* ]]; then
    ARGS+=("$(cygpath "$arg")");
  else
    ARGS+=("$arg");
  fi
done
command /bin/gitbin "${ARGS[@]}"

[emigenix]

Hi Guys! (@silverwind, @bsclifton, @ankostis ) Thanks for looking into this. I've tried the WSL Ubuntu and not only are the node and npm packages way outdated, but it also introduce a lot of other weirdness that is gonna cause a whole bunch of issues to users. Among those are the default directories you get when using CMD from desktop link, vs cmd from Start button and also depending on if you're running as Admin or not... In addition the entire CMD often crashes without any visible reason. There is no future on WSL as it will never be able to run any x-windows or open additional command windows from within, like opening windows explorer windows from within bash etc...

$ node -v && npm -v
v0.10.25
1.3.10

To install and update do this:

#----------------------------------------------------------
# Using node and npm in W10 WSL (Ubuntu Bash)
#----------------------------------------------------------
# Updating Ubuntu on W10:
sudo apt dist-upgrade
sudo apt upgrade

# Installing node and npm on W10-WSL
sudo apt-get install nodejs
sudo ln -s /usr/bin/nodejs /usr/bin/node
sudo apt-get install npm 

# Updating npm on W10-WSL
npm install npm@latest -g

# Updating node on W10-WSL
sudo npm cache clean -f
sudo npm install -g n
sudo n stable

Now re-start bash shell to reload new environment

$ node -v && npm -v
v6.4.0
3.10.6

[emigenix]

Also, don't forget the shopt and set -o shell options may influence how things are quoted and expanded.

$ shopt
autocd          off
cdable_vars     off
cdspell         off
checkhash       off
checkjobs       off
checkwinsize    off
cmdhist         on
compat31        off
compat32        off
compat40        off
compat41        off
compat42        off
complete_fullquote      on
direxpand       off
dirspell        off
dotglob         off
execfail        off
expand_aliases  on
extdebug        off
extglob         off
extquote        on
failglob        off
force_fignore   on
globstar        off
globasciiranges off
gnu_errfmt      off
histappend      on
histreedit      off
histverify      off
hostcomplete    on
huponexit       off
interactive_comments    on
lastpipe        off
lithist         off
login_shell     on
mailwarn        off
no_empty_cmd_completion off
nocaseglob      off
nocasematch     off
nullglob        off
progcomp        on
promptvars      on
restricted_shell        off
shift_verbose   off
sourcepath      on
xpg_echo        off

$ set -o
allexport       off
braceexpand     on
emacs           on
errexit         off
errtrace        off
functrace       off
hashall         on
histexpand      on
history         on
igncr           off
ignoreeof       off
interactive-comments    on
keyword         off
monitor         on
noclobber       off
noexec          off
noglob          off
nolog           off
notify          off
nounset         off
onecmd          off
physical        off
pipefail        off
posix           off
privileged      off
verbose         off
vi              off
xtrace          off

[silverwind]

@emigenix why not just use nodesource repos?

https://github.com/nodesource/distributions#debian-and-ubuntu-based-distributions

[emigenix]

@silverwind I need the simple Cygwin out-of-the-box experience, for multiple Windows 8/10 installations.

[emigenix]

For all those of you who want to run npm on Cygwin, I've created a repo with clear working instructions in: https://github.com/emigenix/npm_on_cygwin. Works like a charm!