plumbum hides stderr of pipes

[hashbrowncipher]

Type "help", "copyright", "credits" or "license" for more information.
>>> from plumbum.cmd import cat, head
>>> from plumbum import FG
>>> cat['/dev/urndom'] & FG(1)
/bin/cat: /dev/urndom: No such file or directory
>>> (cat['/dev/urndom'] | head['-c', '10']) & FG(retcode=1)
>>>

I would expect to see the standard error output of cat printed to the stderr of the Python process. For instance, zsh shows this:

❯❯❯ /bin/cat /dev/urndom | head -c10
/bin/cat: /dev/urndom: No such file or directory

[henryiii]

I'll try to look at these soon! Hoping it's an easy fix. Feel free to dig around a bit, would start in plumbum/commands.

[henryiii]

I've got a fix for this, will commit in a day or two.

[hashbrowncipher]

Hey, I don't see code for this in master. Did something prevent you from merging the change you had?

[henryiii]

Seems to work...

maya8:plumbum henryiii$ git status
On branch master
Your branch is up-to-date with 'origin/master'.
maya8:plumbum henryiii$ python
Python 3.5.0 |Anaconda 2.4.0 (x86_64)| (default, Oct 20 2015, 14:39:26) 
[GCC 4.2.1 (Apple Inc. build 5577)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from plumbum import FG
>>> from plumbum.cmd import cat, head
>>> cat['/dev/urndom'] & FG(1)
cat: /dev/urndom: No such file or directory
>>> (cat['/dev/urndom'] | head['-c', '10']) & FG(retcode=1)
cat: /dev/urndom: No such file or directory

[henryiii]

Should also be tested for, now, with py.test, using file descriptor capture:

    @skip_on_windows
    def test_pipe_stderr(self, capfd):
        from plumbum.cmd import cat, head
        cat['/dev/urndom'] & FG(1)
        assert 'urndom' in capfd.readouterr()[1]

        assert '' == capfd.readouterr()[1]

        (cat['/dev/urndom'] | head['-c', '10']) & FG(retcode=1)
        assert 'urndom' in capfd.readouterr()[1]

[thomasshanks]

@henryiii:
I'm getting this new error:
plumbum.machines.session.SSHCommsChannel2Error: No stderr result detected. Does the remote have Bash as the default shell?
EVERY time I attempt to run a command with a pipe in it, eg.
'echo {} | sudo tee /etc/hostname'.format(hostname)'. (I am also getting this for 'sudo hostname {}'.format(hostname), so I think that this might be an issue that goes beyond use of pipes.) What's the right way to use pipes after this command?

My attempt to work around this was to use "sudo dd of=/etc/hostname <<<'{}'".format(hostname). That didn't work, either. It was met with: plumbum.machines.session.SSHCommsError: No communication channel detected. Does the remote exist?. So was 'sudo sh -c "echo {} > /etc/hostname"'.format(hostname). Is there a right way to do that, as well?

[arunrubrik]

Any plan to fix this bug ?

[henryiii]

I'd be happy with a PR :) I won't have time to work on it for at least a couple of weeks or more.

[woky]

It seems it's not just pipes. I'd expect the following to print errors to Python process's sys.stderr:

from plumbum import local

bash = local['bash']['-c']
x = bash('echo foo >&2')
y = bash('ls /123456 || true')

It doesn't print anything though.

[hashbrowncipher]

@woky I looked up the docstring for __call__, and found that it says "A shortcut for run(args), returning only the process’ stdout". I was able to get output going to sys.stderr by doing:

>>> bash['ls /123456 || true'] & FG

If instead capturing stderr is preferred, one can do:

>>> bash.run('ls /123456 || true')
(0, '', "ls: cannot access '/123456': No such file or directory\n")

[woky]

I don't want to capture stderr. It should be printed to actual sys.stderr and not discarded. If you do in e.g. Bash

foo=$(ls /12345 || echo foo)

you get stdout into $foo but stderr will still be printed to shell's stderr. Plumbum seems to discard it for no reason.

[ThomasHickman]

One way to workaround this is to just add >= sys.stderr after any ... = local[...] assignments e.g.:

$ python3
Python 3.11.8 (main, Feb 25 2024, 16:39:33) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from plumbum import local
>>> import sys
>>> bash = local['bash']['-c'] >= sys.stderr
>>> x = bash('echo foo >&2')
foo
>>> y = bash('ls /123456 || true')
ls: cannot access '/123456': No such file or directory