BUG: Can't run subprocess.Popen(..., stdout=sys.stdout) in IPython window

[larsoner]

This fails in Jupyter notebook but passes in plain python and ipython:

import subprocess
import sys
subprocess.Popen(['who'], stdout=sys.stdout)

With output:

---------------------------------------------------------------------------
UnsupportedOperation                      Traceback (most recent call last)
<ipython-input-1-be12742210dd> in <module>()
      1 import subprocess
      2 import sys
----> 3 subprocess.Popen(['who'], stdout=sys.stdout)

~/miniconda3/envs/mne/lib/python3.6/subprocess.py in __init__(self, args, bufsize, executable, stdin, stdout, stderr, preexec_fn, close_fds, shell, cwd, env, universal_newlines, startupinfo, creationflags, restore_signals, start_new_session, pass_fds, encoding, errors)
    663         (p2cread, p2cwrite,
    664          c2pread, c2pwrite,
--> 665          errread, errwrite) = self._get_handles(stdin, stdout, stderr)
    666 
    667         # We wrap OS handles *before* launching the child, otherwise a

~/miniconda3/envs/mne/lib/python3.6/subprocess.py in _get_handles(self, stdin, stdout, stderr)
   1174             else:
   1175                 # Assuming file-like object
-> 1176                 c2pwrite = stdout.fileno()
   1177 
   1178             if stderr is None:

UnsupportedOperation: fileno

[Carreau]

Yes, the stdout and err are not files in the ipykernel as they are hooked to ZMQ.
You can use the ! who syntax or %sc, %sx magics to do this kind of things when using ipykernel.

[larsoner]

Unfortunately that workaround does not allow code to be written to work in both ipykernel and regular Python / IPython.

[takluyver]

You would need to use a pipe, pull data from it and write it to stdout yourself. This should work both in ipykernel and in a terminal. I wrote a little example of doing that here:

https://github.com/takluyver/rt2-workshop-jupyter/blob/e7fde6565e28adf31a0f9003094db70c3766bd6d/Subprocess%20output.ipynb

It gets trickier if you need to get stderr separately, or supply stdin at the same time, because then you need to avoid deadlocks (your code waiting to read/write one pipe while the subprocess is waiting to write/read a different one). But if you just want to display output live, it's quite easy.

(If the program will run briefly and it's OK to show output when it's finished, it's even easier:

from subprocess import run, PIPE
res = run(['who'], stdout=PIPE)
print(res.stdout.decode('utf-8'))

)

[takluyver]

Alternatively, Min's wurlitzer package might be able to help you.

[larsoner]

Okay, I'll see if I can work around the issue for now. Thanks for the tips