Unable to pipe Bash output to other programs #2

Open
raeesbhatti opened this Issue Apr 7, 2016 · 33 comments

Projects

None yet
@raeesbhatti

I'm unable to spawn Bash from NodeJS. If you try to, it will return this error: Error: 0x80070057

How to reproduce:

  • Install NodeJS
  • Run node in CMD
  • Type:
cp = require('child_process');
proc = cp.spawnSync('cmd', ['/c', 'bash -c "ls"'], {encoding: 'utf8'}).output

It outputs:

[ null, 'Error: 0x80070057\r\r\n', '' ]

ref: https://github.com/Microsoft/CommandLine-Documentation/issues/8

@patryk9200

Try by changing encoding. I don't have problem to run bash -c "ls" from CMD.
Try UTF16 or Latin II (852). I have in CMD Latin II encodig by default.

@raeesbhatti

NodesJS says unknown encoding for both UTF-16 and ISO-8859-2. And it works just find in the CMD. Just not in NodeJS

@raeesbhatti
raeesbhatti commented Apr 7, 2016 edited

This is a problem with Bash having problems with output piping. If I specify { stdio: 'inhert' }, it works but that way I then have no way of getting that data. { stdio: 'pipe' } produces the error above

@raeesbhatti raeesbhatti changed the title from Unable to spawn Bash from other programs to Unable to pipe Bash output to other programs Apr 7, 2016
@tkelman
tkelman commented Apr 11, 2016

Same from mintty (the terminal emulator used by cygwin and msys2), which uses named pipes for stdin/stdout:

$ /cygdrive/c/windows/system32/bash
Error: 0x80070057
@russalex
Collaborator

Sorry for the late comment on this one. Main reason for the delay is that I did not want to comment until I have tried it myself and I did not have node.exe on my system.

I still get an error here on the latest build, but it at least is a different error (no more helpful).

We will look into this one, but I can not promise anything.

@raeesbhatti

This is actually very crucial to many applications of WSL. For example in my case I have to make Atom-Hack (HackLang plugin for Atom Editor) work by executing a Hack Linter inside the WSL and pipe its output to Atom's linter. There are a lot of use cases for this.

You can actually try this on Python too as suggested here

@russalex
Collaborator

We have a known limitation where we only support the console for stdin and stdout. If either of those are not a console then we will fail. That said, I believe I'm telling you things you already know with your last comment in #20.

We do understand that this is a limitation and does block some scenarios. It is on the backlog. My suggestion at this point would be to add it to our User Voice page and go for the up votes to help raise the priority.

@wywzxxz
wywzxxz commented Apr 14, 2016 edited

node.js spawn have the same problem.
and popen in C++ have the problem(tested with Dev-c++).

however,If there is no output,then It works fine.
here is an example:
a.cpp

#include"stdio.h"
int main()
{
    FILE *fp=fopen("out.txt","w");
    fprintf(fp,"asdadasd\n");
    fclose(fp); 
}

main.cpp

#include"stdio.h"
#include"windows.h"
int main(int argc,char** argv)
{
    //if (argc<=1) return 0;
    system("bash -c \"./a\"");
    return 0;
}

g++ a.cpp -o a then compile and run main.cpp in windows. the out.txt file appear with the right content.

so here is a temporary solution:
1.use a wraper to write everything into a file
2.read that file in windows

2016/04/15 update:
It turn out system works just fine in C++(at least in Dev-C++/TDM-GCC),both bash -c "ls >out.txt" and bash -c "ls" works fine. However, when node.js called it, above problem appeared.

@raeesbhatti
raeesbhatti commented Apr 14, 2016 edited

I've just purposed a UserVoice Idea for this problem. Anyone who wants to see this fixed, can vote for it.

@benhillis
Member

Thanks for creating the UserVoice page. I wanted to be clear that this is definitely very high on our internal list of things we want to implement for future updates.

@russalex russalex added the feature label Apr 25, 2016
@mjoppich

Is there any quick fix around the corner ... i'd desperately love to grab the output from bash -c :S

As a first "quick fix" is it somehow possible to save the output from bash to file?

@aseering
Contributor

I haven't tried this, but if you want to write the output to a file, could you just do that within bash? bash -c 'ls > /mnt/c/Users/me/tmp.txt'

@wywzxxz
wywzxxz commented Apr 27, 2016 edited

"ls > file" doesn't work for node-webkit,and as I remember,doesn't work for node.js either.
But as I mentioned above,system("bash -c 'ls > file' ") in C++ works.

@russalex
Collaborator

Update on our oldest open issue.

The new Insider build, 14901, has some updates here. The following commands now work:

> C:\tmp>bash -c "ls -la" | findstr foo
> drwxrwxrwx 2 root root 0 Aug 12 12:01 foo
C:\tmp>dir | bash -c "grep foo"
08/12/2016  12:01 PM    <DIR>          foo

In Node on Windows:

cp = require('child_process');
proc = cp.spawnSync('cmd', ['/c', 'bash -c "ls"'], {encoding: 'utf8'}).output
outputs:
[ null,
'node_etw_provider.man\nnode.exe\nnode_modules\nnode_perfctr_provider.man\nnodevars.bat\nnpm\nnpm.cmd\n',
  '' ]

The following does not work.

C:\tmp>bash -c "ls -la" > out.txt

Out.txt only has the text: Error: 0x80070057. We are aware of this one and working on it.

@DanielRamosAcosta
DanielRamosAcosta commented Aug 23, 2016 edited

Hi guys, previor to this build, I couldn't spawn bash without getting the error. Now works almost fine, but I can't send delayed commands, try this:

const spawn = require('child_process').spawn;

const bash = spawn('/Windows/System32/bash.exe');

bash.stdout.on('data', (data) => {
  console.log(data.toString());
})

bash.stderr.on('data', (data) => {
  console.log(`stderr: ${data}`);
})

bash.on('close', (code) => {
  console.log(`child process exited with code ${code}`);
})

bash.stdin.setEncoding('utf-8');

bash.stdin.write('pwd\n');
bash.stdin.write('uptime\n');
bash.stdin.write('apt-get moo\n');
// All this executes fine

setTimeout(() => {
  bash.stdin.write('apt-get --version\n');
  bash.stdin.end();
}, 5000)
// But these don't

I'm at 14905 build.

@tjmonk15
tjmonk15 commented Sep 11, 2016 edited

I have been fighting this bug myself for a few hours and found this issue. I have managed to create a TEMPORARY workaround for devs fighting this bug. You can find the .Net project here and you can either use it as is, or adapt the code for other languages. This work-around/project/code is MIT licensed, so have at it.

Basically, it forwards any args passed to it to Bash and tells Bash to pipe its output (using /mnt/c/...) to a temp file and then after Bash exits the file is read and output. This output CAN be redirected using any of the standard methods.

PLEASE note, this is VERY quick and dirty. Feel free to submit pull requests as you see fit.

@raeesbhatti

I'm having some problems with Bash output in GoLang. I've tried three things:

cmd := exec.Command("bash", "-c", `"ls /"`)
stdout, err := cmd.StdoutPipe();
if err != nil {
    panic(err)
}
stderr, err := cmd.StderrPipe();
if err != nil {
    panic(err)
}
if err := cmd.Start(); err != nil {
    panic(err)
}
defer cmd.Wait()
go io.Copy(os.Stdout, stdout)
go io.Copy(os.Stderr, stderr)
cmd := exec.Command("bash", "-c", `"ls /"`)
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
err := cmd.Run()
if err != nil {
    panic(err)
}
cmd := exec.Command("bash", "-c", `"ls /"`)
var out bytes.Buffer
cmd.Stdout = &out
err := cmd.Run()
if err != nil {
    panic(err)
}
fmt.Println(out.String())

All of these result in Error: 0x80070006. @russalex Can you please take a look!

@benhillis
Member

@raeesbhatti We have some changes coming to Insider builds which will allow pipe and file redirection to and from bash.exe. Stay tuned to the release notes.

@rbohac
rbohac commented Oct 18, 2016

Same problem when calling bash from a batch file if I set it up under windows scheduler.

If I call bash from a batch file and run the batch file from the command prompt directly it works

If I call that same batch file (that calls bash from within it) from Windows Scheduler I receive E r r o r : 0 x 8 0 0 7 0 0 5 7

@mheden
mheden commented Oct 20, 2016

Is there any ETA when this fix will be rolled out to non-insiders?

@russalex
Collaborator

New features and bug fixes will roll out to non-insiders when the next version of Windows ships to the public. We do know this is not ideal but these changes have a strong dependency on the Windows kernel version which ships with the Insider program.

Some required fixes will be backported. Examples would be any security issues found and the bug which was blocking Java from running, but these are the exception and not the rule.

If at all possible I highly recommend jumping onto the insider program. We are rolling out new features and bug fixes all the time there, including our new interoperability functionality.

@pradyuman

Has this been fixed in an insider build? Bash still cannot run as a shell in emacs (I'm on build 14965): syl20bnr/spacemacs#6751

@felixfbecker

This unfortunately prevents me from integrating gcc under WSL as a task in Visual Studio Code to compile Unix programs right from VS Code under Windows, and get problems reported directly in VS Code as red squiggles. VS Code is Node-based and spawns bash as a child process as described in the issue and then displays the output in the output panel. I can of course just compile from a terminal, but then the errors won't get matched with a regexp to get red squiggles right in the editor. Please support this use case

@benhillis
Member

@felixfbecker - Are you running Anniversary Update or an insider build? This functionality is currently available via the Windows Insider program.

@felixfbecker

@benhillis I will update instantly. Slow or Fast ring?

@mjoppich

@felixfbecker they should be currently the same on 14965 or so...

@rogalmic
rogalmic commented Dec 11, 2016 edited

I am trying to implement bash debugger extension based on bashdb to VS Code.

Will there ever be a possibility to do following in nodejs/Windows?

var process = spawn("bash", ["-c", `echo whatever; echo whatever2 >&3`
		], {stdio: ["pipe", "pipe", "pipe", "pipe"]});
@felixfbecker

@rogalmic It's already possible in Insiders release

@rogalmic

@felixfbecker

I just tried with fresh W10 14965.1001 from ISO, but I am having some problems still (though the line in my previous post does not throw any error).

The problem occurs in write method plus the traces from output pipes do not appear:

		this.sendEvent(new OutputEvent(`STARTING\n`, 'stderr'));

		try
		{
			var process = spawn("bash", ["-c", `echo WHATEVER >&3`]
			, {stdio: ["pipe", "pipe", "pipe", "pipe"]}
			);

			process.stdin.write(`DATA DATA DATA \n`);

			process.stdio[3].on("data", (data) => {
					this.sendEvent(new OutputEvent(`${data}`, 'stderr'));
			});
		}
		catch(ex)
		{
			this.sendEvent(new OutputEvent(`EXCEPTION ${ex}\n ${ex.stack}\n`, 'stderr'));
		}

		this.sendEvent(new OutputEvent(`STARTED\n`, 'stderr'));

Output shows Error: write EPIPE in Socket._writeGeneric():

STARTING
EXCEPTION Error: write EPIPE
 Error: write EPIPE
    at exports._errnoException (util.js:1026:11)
    at Socket._writeGeneric (net.js:706:26)
    at Socket._write (net.js:725:8)
    at doWrite (_stream_writable.js:307:12)
    at writeOrBuffer (_stream_writable.js:293:5)
    at Socket.Writable.write (_stream_writable.js:220:11)
    at Socket.write (net.js:651:40)
    at BashDebugSession.launchRequest (bashDebug.js)
...
STARTED

Am I doing something wrong?

@felixfbecker

What is stdio[3] supposed to be? Maybe that's the issue, that there is only stdin/stdout/stderr? Anyway, Imo this is getting offtopic for this thread and pretty node-specific, you may want to open a new issue to not spam everyone. You also don't handle your errors correctly, those functions will never throw synchronously, you need to attach an error event handler.

@rogalmic
rogalmic commented Dec 11, 2016 edited

@felixfbecker Well, stdio[3] is just another (non-standard) pipe passed to child process. It works fine when using cmd.exe instead of bash.exe.

Anyway, this is off-topic, so I will end here (and check the error correctly).

...Thanks to proper error handling, the error turned out to be in spawn ("ENOENT"), which I was able to fix by providing full bash path ("C:\Windows\sysnative\bash.exe"). Thank you for help...

@wgma00 wgma00 referenced this issue in wgma00/PokemonShowdownBot Dec 12, 2016
Open

Windows compatibility #11

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment