New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Node.js doesn't run as tty on windows / cygwin #3006

Closed
martinheidegger opened this Issue Sep 22, 2015 · 47 comments

Comments

Projects
None yet
@martinheidegger

martinheidegger commented Sep 22, 2015

Just had a report @ nodeschool that node -p -e "Boolean(process.stdout.isTTY)" on Windows 8.1 Pro with Node.js 4.1.0 on Cygwin 2.2.1 returns false. It does seem to work in the regular command prompt.

@bnoordhuis

This comment has been minimized.

Member

bnoordhuis commented Sep 22, 2015

IIRC, that's because the cygwin shell isn't really a tty, it's a pipe.

@martinheidegger

This comment has been minimized.

martinheidegger commented Sep 22, 2015

I am confused (general state of mind these days): Ain't I supposed to be able to do things like process.stdin.resume() on cygwin? Is process.stdin.resume() not tied to it being a tty? Am I not understanding something?

@bnoordhuis

This comment has been minimized.

Member

bnoordhuis commented Sep 22, 2015

process.stdin can be a tty, pipe, file, or even a socket. node.js detects what type of device it is and adjusts accordingly.

@martinheidegger

This comment has been minimized.

martinheidegger commented Sep 22, 2015

Let me rephrase the question: In my understanding tty is indicating that there is some user input loop that can be .resume()ed. Is that correct? A pipe would not indicate that there is ui attached to it?!

Note: clarification on Cygwin: I meant the Cygwin bash.

@bnoordhuis

This comment has been minimized.

Member

bnoordhuis commented Sep 22, 2015

You can .resume() either way, it's not specific to ttys. stdin being a pipe doesn't say anything about the other end having a UI.

@martinheidegger

This comment has been minimized.

martinheidegger commented Sep 22, 2015

I am sorry to ask differently yet again: Does this mean that there is no reliable way to figure out whether node (or any other app) is running inside a terminal?

(I am asking because http://stackoverflow.com/questions/4426280/what-do-pty-and-tty-mean tells me that tty means having a terminal...)

@bnoordhuis

This comment has been minimized.

Member

bnoordhuis commented Sep 22, 2015

If by 'terminal' you mean 'user-visible window', then no, not in general and not reliably.

@martinheidegger

This comment has been minimized.

martinheidegger commented Sep 22, 2015

I need to ask again to form a good question here (so sorry for taking your time - and anyone who reads this). I am still not clear what tty means in the node context. I am still assuming it has to do with "user input". As far as I read in the api documentation, depending on process.stdout.isTTY other parts of the stdio of node (tty.ReadStream) change. If I can't rely on isTTY to tell me if the terminal supports asks for user input or not, can I force node to use the tty.Read/WriteStream ?

@mcnameej

This comment has been minimized.

mcnameej commented Sep 22, 2015

Short version: Cygwin sucks. Don't use it. 😄

Medium version: Cygwin (and others) that attempt to provide "terminal windows" have a lot of problems with standard Windows programs (e.g. node).

Long version...

POSIX has pty's, which allows programs to work the same way on the console, serial terminals, ssh sessions, and GUI terminal windows (e.g. XTerm). The program sees a standard "tty" interface in all cases, and can easily differentiate this interface from a pipe or file redirection.

Windows has no equivalent feature. Windows consoles are a special facility built in to the OS, and generally aren't extensible. The Windows console can do things that no other application can do.

cygwin (and others) attempt to emulate POSIX pty's and provide "terminal windows" with varying degrees of success. These terminal windows are generally NOT Windows consoles, and can confuse standard Windows programs (e.g. node). Some of them use pipes, but pipes have problems. A program with a pipe for stdin or stdout has no way to know if it's being run in a traditional shell pipeline (e.g. "ls -l | head"), or if the other end of the pipe is a process attempting to emulate an interactive tty.

@martinheidegger

This comment has been minimized.

martinheidegger commented Sep 23, 2015

@mcnameej Thank you for your detailed explanation. The problem with this situation is that the bash of cygwin and the bash of github have been promoted widely in the nodeschool community. I also meet many windows developers that use it in their corporate computers.

I seem to have the following problems after processing all your great input:

  • In node.js 4 (probably before that in one of the iojs versions) something seemingly changed in the tty detection. The result is that now it is false where before it probably was true (have not tracked non-issues). Those problems have only been reported since the 4.0 release. Does anyone know what has changed? Have people similar issues?
  • Some logic inside of node.js seem to change when isTTY is detected but since we can't guarantee that it is detected properly I would like to be able to enforce that behaviour - making the code assume that it is a tty. Would that be possible? Is that even necessary? (Keep in mind that I don't know what the internals are about)
  • I have added an isTTY check to my code in response to some strange error that occurs when you try to use setRawMode and resume(). It seems I have used isTTY wrong. When and how is it appropriate to do a isTTY check?
@mcnameej

This comment has been minimized.

mcnameej commented Sep 23, 2015

bash itself is not the problem -- the problem is running node inside a "fake" console. If you run bash in a regular Windows console, it gets along fine with node. See screen shots below. .

In node.js 4 (probably before that in one of the iojs versions) something seemingly changed in the tty detection.

I don't know what changed. A review of the code (probably in libuv) could probably determine the cause, but IMHO, that's asking the wrong question. You're trying to do something that is inherently unreliable on Windows, and any "fix" made today may break again tomorrow.

Would it be possible to make the code assume that it is a tty?

I'm not the right person to answer that; perhaps @bnoordhuis will chime in. I have a lot of experience with Win32, and can talk about the OS-level issues, but I'm not an expert on node's internals.


screenshot2

screenshot1

@martinheidegger

This comment has been minimized.

martinheidegger commented Sep 24, 2015

@mcnameej Don't have time for a lengthy reply but there is one thing to point out: those screenshots are made with win 7, the problem has been reported with windows 8.1

@mcnameej

This comment has been minimized.

mcnameej commented Sep 24, 2015

I have the same result with Windows 8.1...

w81-screenshot2

w81-screenshot1

@LuigiR0jas

This comment has been minimized.

LuigiR0jas commented Sep 24, 2015

Hi! I'm the one who reported the issue, yesterday the problem was presented to me in my work's desktop, running win8. Later at home I tried it in my pc running Windows 10, same issue on cygwin, but on cmd worked perfectly. Now I'm at work again(win8) trying to make it through the tutorial, it opens OK in cmd but the key arrows doesn't work. Here's a screenshot I took yesterday in my work's desktop.
Please tell me if you need more info and I will see what can I do for you!
unnamed

@martinheidegger

This comment has been minimized.

martinheidegger commented Sep 27, 2015

(My lengthy reply)

bash itself is not the problem -- the problem is running node inside a "fake" console. If you run bash in a regular Windows console, it gets along fine with node. See screen shots below. .

It should but somehow it seems it doesn't as hinted by the screenshots of @LuigiR0jas above. Do you have an idea why his bash returns something different than yours? Is there a way to find out the difference between these two setups?

You're trying to do something that is inherently unreliable on Windows, and any "fix" made today may break again tomorrow.

It is not about trying to "fix" it windows for me. It is however important to me to write command line tools that work predictably and reliably on windows. I haven't had the chance to test it but it seems like the cli of other tools seems to work fine?! If the bash is up and running what is the practical difference between isTTY being true and false? (why does node need to adapt to it?)

@mcnameej

This comment has been minimized.

mcnameej commented Sep 27, 2015

@LuigiR0jas: Please try this...

Use Windows Explorer to navigate to wherever you have Cygwin installed. There should be a Cygwin.bat file in the base directory. Double click on that file to run it from Explorer. This will open Cygwin bash inside a standard Windows console.

Change to the node directory (e.g. cd "/cygdrive/c/Program Files/nodejs").

Run ./node -p -e "Boolean(process.stdout.isTTY)" from there.

It should print true.

I just did a fresh install of Cygwin on Windows 8.1 X64 and verified these results.

The "Cygwin Terminal" shortcut runs bash inside Cygwin's fake console. The fake console doesn't get along with node.

@mcnameej

This comment has been minimized.

mcnameej commented Sep 27, 2015

@martinheidegger:

Do you have an idea why his bash returns something different than yours? Is there a way to find out the difference between these two setups?

See my message above to LuigiR0jas. I suspect he was using Cygwin's fake console. The problem isn't bash -- it's the console.

It is however important to me to write command line tools that work predictably and reliably on windows

Unfortunately, the Cygwin people have different priorities. They want their environment to be as "POSIX-ish" as possible, even if that causes problems with some native Windows programs.

I haven't had the chance to test it but it seems like the cli of other tools seems to work fine?

Tools that work with pure ASCII input and output streams generally work fine inside fake consoles. They don't care if their stdin/stdout are pipes. Tools that do special input processing (e.g. read single characters rather than lines), or that colorize their output, will fail.

why does node need to adapt to it

The problem is that Window has console-specific API's, which node uses when run inside a real Windows console. Node falls back to generic I/O functions when stdin/stdout are anything other than a real console (e.g. pipe, redirected to file, etc.). Having console-specific I/O functions is a key difference between Windows and POSIX.

@LuigiR0jas

This comment has been minimized.

LuigiR0jas commented Sep 27, 2015

Tried it, it worked.
screenshot_1
I hope this is useful, i took it in my home PC with Windows 10 (it had the same problem), on the left the regular cygwin terminal (showing false), on the right the cygwin bash on cmd.

@martinheidegger

This comment has been minimized.

martinheidegger commented Sep 27, 2015

Short question: in the bash without false we can see colors, does vim or emacs work (keyword: stdin)

@mcnameej

This comment has been minimized.

mcnameej commented Sep 27, 2015

@LuigiR0jas:

on the left the regular cygwin terminal (showing false), on the right the cygwin bash on cmd.

Proves what I'm saying. The problem isn't bash -- it's the console.

@mcnameej

This comment has been minimized.

mcnameej commented Sep 27, 2015

@martinheidegger:

does vim or emacs work

I'm not sure what that test will prove. cygwin's own programs (bash, vim, emacs, etc.) know how to deal with both real Windows consoles and cygwin's terminal. They'll work either way. The problem is with standard Windows console applications (e.g. node) running inside cygwin's terminal.

@martinheidegger

This comment has been minimized.

martinheidegger commented Sep 27, 2015

@mcanameej I never doubted your word. Just need to go 100% clear in understanding to phrase it correctly when passing on. There is still some question open:

The problem is with standard Windows console applications (e.g. node) running inside cygwin's terminal.

Just for the sake of argument: could you compile nodejs with cygwin and it would work?
Could it be determined if you run bash in a "not-regular-console"?

Tools that do special input processing (e.g. read single characters rather than lines),

So it is possible to read lines and output text in those terminals?!
Does it allow other characters (like ignore the color on windows)?
Does this assumption exclude control characters like [;H or are only the colors a problem?

@mcnameej

This comment has been minimized.

mcnameej commented Sep 27, 2015

I went over and looked at workshopper/workshopper#120 and nodeschool/discussions#1448 that were mentioned by @SomeoneWeird.

I strongly suspect there is a bug in node v4 (probably in libuv) related to setting Windows 8/8.1/10 consoles into raw mode. I can reproduce the failure in a standard console. This is a completely separate issue from running node inside cygwin's fake console.

@mcnameej

This comment has been minimized.

mcnameej commented Sep 27, 2015

@martinheidegger:

Just for the sake of argument: could you compile nodejs with cygwin and it would work?

AFAIK, node doesn't support building under cygwin. Somebody might be able to hack it, but I'm afraid it won't be me.

Could it be determined if you run bash in a "not-regular-console"?

I don't understand the question.

So it is possible to read lines and output text in those terminals?!

Yes.

When running node in a fake console, it's useful to imagine that you're running it with input and output redirected to files (e.g. node foobar.js <infile.txt >outfile.txt). That's basically what node thinks is going on (a pipe and a file look more-or-less the same to node). Things that would work with file redirections will work in the fake console.

Does this assumption exclude control characters like [;H or are only the colors a problem?

Windows consoles don't implement ANY escape sequences. node (via libuv) emulates them when it detects itself running in a console. Behind the scenes, libuv parses the escape sequences and makes the appropriate Windows console API calls to implement them.

@clicktravel-jaroslav

This comment has been minimized.

clicktravel-jaroslav commented Aug 4, 2016

Yeah. Create git bash script and wrapped node into this script solves the problem.

@sebolio

This comment has been minimized.

sebolio commented Sep 28, 2016

Is this fixed in Node v6?

@Fishrock123

This comment has been minimized.

Member

Fishrock123 commented Sep 28, 2016

Looks like the solution was:

I don't know if this helps, but when installing Git for Windows 2.7.0 there is a prompt now (I don't recall seeing it in some fairly earlier version) that defaults to using MinTTY, but if you choose Use Windows' default console window, $ node -p -e "Boolean(process.stdout.isTTY)" prints true and I don't experience any of the problems that I do with MinTTY (some gulp hangs, inquirer prompts not working correctly, etc.)

I'm not sure we can actually fix this.

@yairEO

This comment has been minimized.

yairEO commented Nov 1, 2016

Is this fixed in Node v6?

or 7 ?

@sam-github

This comment has been minimized.

Member

sam-github commented Nov 2, 2016

No. I tried to fix it, but gave up, see #2908 (comment)

I just rebased my branch at https://github.com/sam-github/node/tree/spawn-detached-window-hide-option, feel free to adopt it if you want.

I completely failed to make the pop-up windows happen at all, so could not verify that my change was capable of hiding the pop-up... if you can build and verify that branch, I could re-PR it and I'm sure we could get it merged in short order.

@trevor-mink

This comment has been minimized.

trevor-mink commented Dec 2, 2016

I had this problem and I tried running a 'sub-shell' (mean I ran bash and the command line and created another shell). Then I ran node program < infile and I don't have the problem anymore

@thetrompf

This comment has been minimized.

thetrompf commented Dec 13, 2016

It is not a node problem, I investigated this in context of yarnpkg. The way git-bash "fixed" the problem is by wrapping the node process in a program called winpty, which emulates some/enough of the unix teletype to make it work: https://github.com/rprichard/winpty

If you have a "newer" version of git bash installed, you will find a hack in /etc/profile.d/aliases.sh wrapping common programs like node, php and python to run through winpty.

WinPTY works with cygwin as well, so if you introduce a similar "hack/fix" your bash profile for, it should work "just as well" in cygwin.

Checkout this pull-request for further details: yarnpkg/yarn#2230

@martinheidegger

This comment has been minimized.

martinheidegger commented Dec 13, 2016

@thetrompf I am sorry but that seems like it is a Node.js problem after all: If you have to wrap it with some other method that "does the compatibility" it means - by what I understand - that the Node.js compatibility is not given.

@thetrompf

This comment has been minimized.

thetrompf commented Dec 13, 2016

@martinheidegger Node.js works fine with the standard windows shells (cmd.exe and power shell), using cygwin/msys shells is the users who has decided to use shells that looks like teletype terminals but are not, and the common way to test if you are command line programs is running in an interactive shell is by checking for process.stdout.isTTY or in a unix shell script http://stackoverflow.com/questions/911168/how-to-detect-if-my-shell-script-is-running-through-a-pipe CygWin ans Msys terminals has implemented some of the teletype layer, but unfortunately not enough to work in all cases. If you ask me cygwin and msys/mingw should include whatever winpty is doing in their terminals so they actually work.

@martinheidegger

This comment has been minimized.

martinheidegger commented Dec 13, 2016

@thetrompf I do know that cygwin does not implement certain parts correctly. But: if cygwin would break interactive mode then it wouldn't be possible for winpty to "fix" it. In other words: if an application running in cygwin can fix it node - an application running in cygwin can fix that too. That being said Cygwin is afaik not a officially supported platform. It would be good if it became one though (since several tools on windows promote cygwin as "best shell for windows".

@pmalhaire

This comment has been minimized.

pmalhaire commented Feb 2, 2017

note : if it can help someone there is a workaround for this issue using msys2 I guess it would be the same for cygwin

the issue I face is how to write npm's output to a file using msys2.
it's not the exact same issue but it has the same root as this issue.

I am using msys2w64 under windows 10

the workarround is:
use npm.cmd instead of npm

my case is to run npm and gulp only
this fails

$npm > /tmp/test
stdout is not a tty
$

this works

$npm.cmd > tmp/test
$
@geonanorch

This comment has been minimized.

geonanorch commented Feb 14, 2017

@pmalhaire I use msys2 (from msys2-x86_64-20160205.exe) and do not seem to need the workaround, calling the npm shell script directly works fine.
I do need winpty for running node itself, though.

@pmalhaire

This comment has been minimized.

pmalhaire commented Feb 14, 2017

@geonanorch did you try to redirect it's output to a file as done in my example ?

$npm > /tmp/test
stdout is not a tty
$

@geonanorch

This comment has been minimized.

geonanorch commented Feb 15, 2017

@pmalhaire yes I did:

$/C/Program\ Files/nodejs/npm > test
$cat test

Usage: npm <command>
. . . .

(remark: I installed node/npm independently of MSYS2)

Note that things aren't perfect for me either though: when invoking npm init for instance I have to type CTRL-C to exit the program, it just hangs after generating package.json (and going through winpty first does not help in this case).

I wish there existed a 'standard' library which allowed to build windows console programs with dual support native-console/mintty...

@pmalhaire

This comment has been minimized.

pmalhaire commented Feb 15, 2017

@geonanorch it's a workaround
fixing the issue implies fixing mysys2 it self (not nodejs)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment