Output of C extensions is not shown in IPython console

[peendebak]

Description of your problem

The output of C extensions that print to stdout is not shown in the ipython console. For Spyder 2 the output was shown in the kernel window.

A minimal example:

import oapackage
T=oapackage.conference_transformation_t()
T.show()

When executed from the command line a message is shown, in the Spyder 3 ipython console no output is shown at all.

Note: oapackage can by installed from pypi: https://pypi.python.org/pypi/OApackage. The output desired is generated by C++ code wrapped using SWIG.

Note: the same problem is there when the code is executed in an ipython notebook.

[ccordoba12]

@minrk or @takluyver, is there a configuration setting we can change in qtconsole to redirect stdout/stderr of all C extensions to the zmq corresponding channel?

[takluyver]

There isn't a config setting, but you can use Min's package wurlitzer in the kernel.

Once we implement the kernel nanny proposal, we'll be able to capture this kind of output automatically.

[ccordoba12]

Thanks a lot Thomas! We'll use wurlitzer in the meantime :-)

Once we implement the kernel nanny proposal ...

This seems more and more important for Spyder. What are your plans about it? Do you have a time frame for it?

I say it because we have now 4 new junior developers working for Spyder, so I could assign some of them to work on this project (under your guidance and my supervision, of course :-).

[takluyver]

We discussed it again at the dev meeting, but there isn't an explicit time frame. The primary sticking point for me is that we still don't have a firm decision on whether we use one nanny process for one kernel, or one nanny process per system, managing multiple kernels.

[ccordoba12]

Ok, thanks. Please let me know if you need some help with that :-)

[peendebak]

Another example that does not generate output in the spyder3 with the ipython console

import os
os.system('ls')

[YWaller]

Any update on this issue? I'd like to get tensorflow logs in my console...

[nyngwang]

I need this feature, please?

[ccordoba12]

Sorry, we haven't worked on this one yet. We could work on it after we release 3.3 though.

[jitseniesen]

I had a quick try. It is easy to change spyder-kernels to load the wurlitzer extension on startup and this does the job:

Normally, the output "bla" of the echo command is lost. There is a slight difference with running python from the console, in that IPython run from Spyder forwards the output through Python's sys.stdout while when running python from the console, the output is interspersed with python's output without going through sys.stdout (if I understand it correctly). Nevertheless, I think this is an improvement to the current situation.

@ccordoba12 How to proceed?

1. We can add wurlitzer as a dependency to spyder-kernels. This is perhaps not so nice for a minor release. Additionally, wurlitzer is not available in the default conda channel, only in conda-forge and pip.
2. We can include the source code of wurlitzer in spyder-kernels. It is just one file and MIT licensed. The main disadvantage is that we would not profit from any improvements in the wurlitzer package.
3. As a middle way, we could do option 2 for spyder 3.x and spyder-kernels 0.x and option 1 for spyder 4.x and spyder-kernels 1.x.

[ccordoba12]

We can add wurlitzer as a dependency to spyder-kernels.

This is the option I was planning to implement in a couple of weeks.

This is perhaps not so nice for a minor release

We can add this minor improvement to spyder-kernels 0.3. After that, I don't plan to add more enhancements to that branch.

Additionally, wurlitzer is not available in the default conda channel, only in conda-forge and pip.

I know wurlitzer is not part of Anaconda, but since it's already part of conda-forge, the Anaconda guys can add it without problems.

[ccordoba12]

I said minor in the sense that it doesn't require too much work and it's not too disruptive. I know it's very important to work with certain libraries.

[jitseniesen]

I guess this should be moved to the spyder-kernels repository?

[ccordoba12]

We can close this issue with a PR from spyder-kernels, so let's not move it because it has all our discussion about this topic.

[ccordoba12]

This will be fixed once we release spyder-kernels 0.3.0, in a couple of weeks.

[Nicholaswogan]

Is it possible to turn off the C printing to the IPython console? I'd rather just see the C extension stuff in my terminal, which has the jupyter notebook opened.

[ccordoba12]

Sorry, now you can't opt-out of this behavior.

[minrk]

It can't be toggled at runtime (except with some private API calls), but you can disable it globally:

# ipython_config.py
c.IPKernelApp.capture_fd_output = False

[Nicholaswogan]

It works great! Thanks.