Skip to content


Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading… -> #397

merged 2 commits into from

2 participants


There are a variety of methods that depend on io.Term existing, and some of these can be called before io.Term is created, and none of them check for its existence.

Possible solutions are:

  1. ensure io.Term always exists, but may be overridden
  2. check for existence of io.Term before using it

This PR implements 1., by adding simple Term=IOTerm() in

An example of the failure: missing readline. init_readline() can display warnings via utils.warn, which depends on io.Term. However, init_io() must be called after init_readline(), so these warnings will always crash IPython.


I don't like this, but it is probably OK until we figure out a better way of initializing stuff like this. At some level, things like this and logging need to be setup at the Application level - IE, before things like InteractiveShell are created.


What is the argument against io.Term having default behavior of sys.stdin/out/err? In what cases would it be better for calls to utils.warn prior to io.Term configuration to raise an error of some kind (AttributeError at the moment) rather than just printing to stderr?

We need to ensure one of the following:

  1. the object is always valid
  2. the object is valid in the scopes from which it can be called

And we do neither, but code assumes we have done 2.

You are right that it should be configured at the application level like logging, but it should also be like logging and not raise an error if used prior to configuration. logging is silent if unconfigured, and I think sys.stdin/out/err is the most reasonable default for our object that's an abstraction of exactly those streams.


I think that stdout/stdin/stderr are fine defaults, I am just not excited about hardcoding defaults into a top-level module variable. Sort of goes against our model of defaults and configuration. I guess it would make sense to have a model that is similar to that of logging though, maybe have a get_term function in I do agree that it should work even before configuration like logging does. The most important aspect of Term is that other modules that need to use Term should get it in a way that is robust. The old, from io import Term is horrible because if io.Term changes, the code doesn't see that. That is why we removed io.Term in the first place. So maybe the logging model does make sense.


What about having top-level stdin/out/err like sys? (or cin/out/err, if that's preferable for some reason). Then it's quite familiar that the top-level object (io) is what you want to hold onto, and code can redirect io.stdout/err as it sees fit, just like it can with sys.stdout/err, and code that does one out=io.stdout will not follow redirection, just like you expect if you do out=sys.stdout.

io.Term looks like it might behave that way, but doesn't because we create a new Term object on reconfiguration. Note that we don't have to do this - there's no reason not to assign directly to io.Term.cout instead of creating a new Term object, and if we did that, then from import Term would still work.

Side question: Is there a reason that io.Term is capitalized?


I like the idea of echoing the design of sys.stdout and friends with io.stdout|stdin|stderr and I think this is probably the way to go. The naming of "Term" is historical and even the name term (which suggests "terminal" is horrible at this point. I don't see any reason to not move to this new API.


Okay, this has now been updated, such that io.Term.cin/out/err are no longer used, in favor of io.stdin/out/err. The behavior is now the same as sys.stdin/out/err, only IPython-wide, instead of sys-wide. The objects are also initialized to IOStream(stdin/out/err), so they are always defined.

The IOTerm class remains, for any cases where one wants to contain a triplet of stdin/out/err redirects in a single object, but it's used nowhere in the code.


This is wonderful. The changes are pretty simple. Have you tested the qt console with this branch as we point things to zmq enabled out streams? I don't see any issues though. Some things we should look at after this is merged:

  • Once the new config stuff is merged, we should get rid of IPython's custom warning stuff and just use the logging module.

Min, have you tested this on Windows? If not, could you do that (I don't have a Windows VM setup right now). Fernando wanted to make sure that the terminal still works with colors (try 1/0 to make sure colored exceptions are working) before merging. I need to get a VM setup for this stuff...


I didn't change how anything works on the inside of the IOStream class, so it should behave the same. I did test on Windows, and 1/0 does produce the expected colorful output. As there is no color related code in, I presume it is the linking of stdout/err to readline._outputfile that allows colors on Windows? That code is the same as before.

I'll just rebase on master, and it should be ready then.

minrk added some commits
@minrk minrk io.Term.cin/out/err replaced by io.stdin/out/err
Behavior is now the same as sys.stdin/out/err, and
defaults to those streams.
@minrk minrk Add StreamProxy soft link io.stdout/err to sys.stdout/err
This allows io.stdout/err to be captured during tests

rebased on master, tests passing, colors working.

@ellisonbg ellisonbg referenced this pull request from a commit
@ellisonbg ellisonbg Merged pull request #397 from minrk/ioterm. ->
@ellisonbg ellisonbg merged commit a34642c into from
@ellisonbg ellisonbg referenced this pull request from a commit
Commit has since been removed from the repository and is no longer available.
@mattvonrocketstein mattvonrocketstein referenced this pull request from a commit in mattvonrocketstein/ipython
@ellisonbg ellisonbg Merged pull request #397 from minrk/ioterm. ->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Apr 27, 2011
  1. @minrk

    io.Term.cin/out/err replaced by io.stdin/out/err

    minrk authored
    Behavior is now the same as sys.stdin/out/err, and
    defaults to those streams.
  2. @minrk

    Add StreamProxy soft link io.stdout/err to sys.stdout/err

    minrk authored
    This allows io.stdout/err to be captured during tests
This page is out of date. Refresh to see the latest.
9 IPython/core/
@@ -31,8 +31,7 @@
from IPython.utils import PyColorize
from IPython.core import ipapi
-from IPython.utils import coloransi
+from IPython.utils import coloransi, io
from IPython.core.excolors import exception_colors
# See if we can use pydb.
@@ -171,7 +170,7 @@ def __init__(self,color_scheme='NoColor',completekey=None,
# Parent constructor:
if has_pydb and completekey is None:
- OldPdb.__init__(self,stdin=stdin,
+ OldPdb.__init__(self,stdin=stdin,stdout=io.stdout)
@@ -279,7 +278,7 @@ def print_stack_trace(self):
def print_stack_entry(self,frame_lineno,prompt_prefix='\n-> ',
context = 3):
#frame, lineno = frame_lineno
- print >>, self.format_stack_entry(frame_lineno, '', context)
+ print >>io.stdout, self.format_stack_entry(frame_lineno, '', context)
# vds: >>
frame, lineno = frame_lineno
@@ -419,7 +418,7 @@ def print_list_lines(self, filename, first, last):
self.lineno = lineno
- print >>, ''.join(src)
+ print >>io.stdout, ''.join(src)
except KeyboardInterrupt:
17 IPython/core/
@@ -26,8 +26,7 @@
from IPython.config.configurable import Configurable
from IPython.core import prompts
-import IPython.utils.generics
+from IPython.utils import io
from IPython.utils.traitlets import Instance, List
from IPython.utils.warn import warn
@@ -178,13 +177,13 @@ def write_output_prompt(self):
"""Write the output prompt.
The default implementation simply writes the prompt to
- ``io.Term.cout``.
+ ``io.stdout``.
# Use write, not print which adds an extra space.
+ io.stdout.write(self.output_sep)
outprompt = str(self.prompt_out)
if self.do_full_cache:
+ io.stdout.write(outprompt)
def compute_format_data(self, result):
"""Compute format data of the object to be displayed.
@@ -219,7 +218,7 @@ def write_format_data(self, format_dict):
"""Write the format data dict to the frontend.
This default version of this method simply writes the plain text
- representation of the object to ``io.Term.cout``. Subclasses should
+ representation of the object to ``io.stdout``. Subclasses should
override this method to send the entire `format_dict` to the
@@ -244,7 +243,7 @@ def write_format_data(self, format_dict):
# But avoid extraneous empty lines.
result_repr = '\n' + result_repr
- print >>, result_repr
+ print >>io.stdout, result_repr
def update_user_ns(self, result):
"""Update user_ns with various things like _, __, _1, etc."""
@@ -287,8 +286,8 @@ def log_output(self, format_dict):
def finish_displayhook(self):
"""Finish up all displayhook activities."""
+ io.stdout.write(self.output_sep2)
+ io.stdout.flush()
def __call__(self, result=None):
"""Printing with history cache management.
4 IPython/core/
@@ -98,9 +98,9 @@ def publish(self, source, data, metadata=None):
the data.
from IPython.utils import io
- # The default is to simply write the plain text data using io.Term.
+ # The default is to simply write the plain text data using io.stdout.
if data.has_key('text/plain'):
- print(data['text/plain'], file=io.Term.cout)
+ print(data['text/plain'], file=io.stdout)
def publish_display_data(self, source, data, metadata=None):
7 IPython/core/
@@ -22,10 +22,9 @@
# Our own packages
from IPython.config.configurable import Configurable
from IPython.testing import decorators as testdec
-from import ask_yes_no
+from IPython.utils import io
from IPython.utils.traitlets import Bool, Dict, Instance, Int, List, Unicode
from IPython.utils.warn import warn
@@ -630,12 +629,12 @@ def _format_lineno(session, line):
outfname = opts['f']
except KeyError:
- outfile = # default
+ outfile = io.stdout # default
# We don't want to close stdout at the end!
close_at_end = False
if os.path.exists(outfname):
- if not ask_yes_no("File %r exists. Overwrite?" % outfname):
+ if not io.ask_yes_no("File %r exists. Overwrite?" % outfname):
1  IPython/core/
@@ -45,7 +45,6 @@ def calljed(self,filename, linenum):
import sys
from IPython.core.error import TryNext
# List here all the default hooks. For now it's just the editor functions
# but over time we'll move here all the public API for user-accessible things.
17 IPython/core/
@@ -511,14 +511,13 @@ def init_inspector(self):
def init_io(self):
# This will just use sys.stdout and sys.stderr. If you want to
# override sys.stdout and sys.stderr themselves, you need to do that
- # *before* instantiating this class, because Term holds onto
+ # *before* instantiating this class, because io holds onto
# references to the underlying streams.
if sys.platform == 'win32' and self.has_readline:
- Term = io.IOTerm(cout=self.readline._outputfile,
- cerr=self.readline._outputfile)
+ io.stdout = io.stderr = io.IOStream(self.readline._outputfile)
- Term = io.IOTerm()
- io.Term = Term
+ io.stdout = io.IOStream(sys.stdout)
+ io.stderr = io.IOStream(sys.stderr)
def init_prompts(self):
# TODO: This is a pass for now because the prompts are managed inside
@@ -1477,7 +1476,7 @@ def _showtraceback(self, etype, evalue, stb):
Subclasses may override this method to put the traceback on a different
place, like a side channel.
- print >> io.Term.cout, self.InteractiveTB.stb2text(stb)
+ print >> io.stdout, self.InteractiveTB.stb2text(stb)
def showsyntaxerror(self, filename=None):
"""Display the syntax error that just occurred.
@@ -1931,7 +1930,7 @@ def auto_rewrite_input(self, cmd):
# plain ascii works better w/ pyreadline, on some machines, so
# we use it and only print uncolored rewrite if we have unicode
rw = str(rw)
- print >>, rw
+ print >> io.stdout, rw
except UnicodeEncodeError:
print "------> " + cmd
@@ -2325,12 +2324,12 @@ def mktempfile(self, data=None, prefix='ipython_edit_'):
# TODO: This should be removed when Term is refactored.
def write(self,data):
"""Write a string to the default output"""
- io.Term.cout.write(data)
+ io.stdout.write(data)
# TODO: This should be removed when Term is refactored.
def write_err(self,data):
"""Write a string to the default error output"""
- io.Term.cerr.write(data)
+ io.stderr.write(data)
def ask_yes_no(self,prompt,default=True):
if self.quiet:
2  IPython/core/
@@ -10,8 +10,6 @@
import re
import sys
coding_declaration = re.compile(r"#\s*coding[:=]\s*([-\w.]+)")
class Macro(object):
1  IPython/core/
@@ -53,7 +53,6 @@
from IPython.external.Itpl import itpl, printpl
from IPython.testing import decorators as testdec
from import file_read, nlprint
from IPython.utils.path import get_py_filename
from IPython.utils.process import arg_split, abbrev_cwd
from IPython.utils.terminal import set_term_title
4 IPython/core/
@@ -31,7 +31,7 @@
from IPython.core import page
from IPython.external.Itpl import itpl
from IPython.utils import PyColorize
+from IPython.utils import io
from IPython.utils.text import indent
from IPython.utils.wildcard import list_namespace
from IPython.utils.coloransi import *
@@ -300,7 +300,7 @@ def pdef(self,obj,oname=''):
if output is None:
self.noinfo('definition header',oname)
- print >>, header,self.format(output),
+ print >>io.stdout, header,self.format(output),
def pdoc(self,obj,oname='',formatter = None):
"""Print the docstring for any object.
14 IPython/core/
@@ -36,7 +36,7 @@
from IPython.core.error import TryNext
from IPython.utils.cursesimport import use_curses
from import chop
+from IPython.utils import io
from IPython.utils.process import system
from IPython.utils.terminal import get_terminal_size
@@ -56,18 +56,18 @@ def page_dumb(strng, start=0, screen_lines=25):
out_ln = strng.splitlines()[start:]
screens = chop(out_ln,screen_lines-1)
if len(screens) == 1:
- print >>, os.linesep.join(screens[0])
+ print >>io.stdout, os.linesep.join(screens[0])
last_escape = ""
for scr in screens[0:-1]:
hunk = os.linesep.join(scr)
- print >>, last_escape + hunk
+ print >>io.stdout, last_escape + hunk
if not page_more():
esc_list = esc_re.findall(hunk)
if len(esc_list) > 0:
last_escape = esc_list[-1]
- print >>, last_escape + os.linesep.join(screens[-1])
+ print >>io.stdout, last_escape + os.linesep.join(screens[-1])
def page(strng, start=0, screen_lines=0, pager_cmd=None):
@@ -176,7 +176,7 @@ def page(strng, start=0, screen_lines=0, pager_cmd=None):
#print 'numlines',numlines,'screenlines',screen_lines # dbg
if numlines <= screen_lines :
#print '*** normal print' # dbg
- print >>, str_toprint
+ print >>io.stdout, str_toprint
# Try to open pager and default to internal one if that fails.
# All failure modes are tagged as 'retval=1', to match the return
@@ -282,13 +282,13 @@ def page_more():
@return: True if need print more lines, False if quit
-'---Return to continue, q to quit--- ')
+ io.stdout.write('---Return to continue, q to quit--- ')
ans = msvcrt.getch()
if ans in ("q", "Q"):
result = False
result = True
-"\b"*37 + " "*37 + "\b"*37)
+ io.stdout.write("\b"*37 + " "*37 + "\b"*37)
return result
def page_more():
1  IPython/core/
@@ -37,7 +37,6 @@
from IPython.core import page
from IPython.utils.traitlets import List, Int, Any, Str, CBool, Bool, Instance
from IPython.utils.text import make_quoted_expr
from IPython.utils.autoattr import auto_attr
8 IPython/core/
@@ -325,8 +325,8 @@ def __init__(self, color_scheme='NoColor', call_pdb=False, ostream=None):
# Output stream to write to. Note that we store the original value in
# a private attribute and then make the public ostream a property, so
- # that we can delay accessing io.Term.cout until runtime. The way
- # things are written now, the Term.cout object is dynamically managed
+ # that we can delay accessing io.stdout until runtime. The way
+ # things are written now, the io.stdout object is dynamically managed
# so a reference to it should NEVER be stored statically. This
# property approach confines this detail to a single location, and all
# subclasses can simply access self.ostream for writing.
@@ -349,12 +349,12 @@ def _get_ostream(self):
Valid values are:
- None: the default, which means that IPython will dynamically resolve
- to io.Term.cout. This ensures compatibility with most tools, including
+ to io.stdout. This ensures compatibility with most tools, including
Windows (where plain stdout doesn't recognize ANSI escapes).
- Any object with 'write' and 'flush' attributes.
- return io.Term.cout if self._ostream is None else self._ostream
+ return io.stdout if self._ostream is None else self._ostream
def _set_ostream(self, val):
assert val is None or (hasattr(val, 'write') and hasattr(val, 'flush'))
30 IPython/lib/
@@ -176,8 +176,8 @@
import sys
from IPython.utils.PyColorize import Parser
+from IPython.utils import io
from import file_read, file_readlines
from IPython.utils.text import marquee
__all__ = ['Demo','IPythonDemo','LineDemo','IPythonLineDemo','DemoError']
@@ -319,7 +319,7 @@ def _get_index(self,index):
if index is None:
if self.finished:
- print >>, 'Demo finished. Use <demo_name>.reset() if you want to rerun it.'
+ print >>io.stdout, 'Demo finished. Use <demo_name>.reset() if you want to rerun it.'
return None
index = self.block_index
@@ -388,9 +388,9 @@ def show(self,index=None):
if index is None:
- print >>, self.marquee('<%s> block # %s (%s remaining)' %
+ print >>io.stdout, self.marquee('<%s> block # %s (%s remaining)' %
- print >>,(self.src_blocks_colored[index])
+ print >>io.stdout,(self.src_blocks_colored[index])
def show_all(self):
@@ -403,12 +403,12 @@ def show_all(self):
marquee = self.marquee
for index,block in enumerate(self.src_blocks_colored):
if silent[index]:
- print >>, marquee('<%s> SILENT block # %s (%s remaining)' %
+ print >>io.stdout, marquee('<%s> SILENT block # %s (%s remaining)' %
- print >>, marquee('<%s> block # %s (%s remaining)' %
+ print >>io.stdout, marquee('<%s> block # %s (%s remaining)' %
- print >>, block,
+ print >>io.stdout, block,
def run_cell(self,source):
@@ -433,18 +433,18 @@ def __call__(self,index=None):
next_block = self.src_blocks[index]
self.block_index += 1
if self._silent[index]:
- print >>, marquee('Executing silent block # %s (%s remaining)' %
+ print >>io.stdout, marquee('Executing silent block # %s (%s remaining)' %
if self.auto_all or self._auto[index]:
- print >>, marquee('output:')
+ print >>io.stdout, marquee('output:')
- print >>, marquee('Press <q> to quit, <Enter> to execute...'),
+ print >>io.stdout, marquee('Press <q> to quit, <Enter> to execute...'),
ans = raw_input().strip()
if ans:
- print >>, marquee('Block NOT executed')
+ print >>io.stdout, marquee('Block NOT executed')
save_argv = sys.argv
@@ -462,10 +462,10 @@ def __call__(self,index=None):
if self.block_index == self.nblocks:
mq1 = self.marquee('END OF DEMO')
if mq1:
- # avoid spurious print >>,s if empty marquees are used
- print >>
- print >>, mq1
- print >>, self.marquee('Use <demo_name>.reset() if you want to rerun it.')
+ # avoid spurious print >>io.stdout,s if empty marquees are used
+ print >>io.stdout
+ print >>io.stdout, mq1
+ print >>io.stdout, self.marquee('Use <demo_name>.reset() if you want to rerun it.')
self.finished = True
# These methods are meant to be overridden by subclasses who may wish to
25 IPython/testing/
@@ -28,12 +28,33 @@
# our own
from . import tools
+from IPython.utils import io
from IPython.frontend.terminal.interactiveshell import TerminalInteractiveShell
# Functions
+class StreamProxy(io.IOStream):
+ """Proxy for sys.stdout/err. This will request the stream *at call time*
+ allowing for nose's Capture plugin's redirection of sys.stdout/err.
+ Parameters
+ ----------
+ name : str
+ The name of the stream. This will be requested anew at every call
+ """
+ def __init__(self, name):
+ @property
+ def stream(self):
+ return getattr(sys,
+ def flush(self):
# Hack to modify the %run command so we can sync the user's namespace with the
# test globals. Once we move over to a clean magic system, this will be done
# with much less ugliness.
@@ -191,5 +212,9 @@ def start_ipython():
get_ipython = _ip.get_ipython
__builtin__._ip = _ip
__builtin__.get_ipython = get_ipython
+ # To avoid extra IPython messages during testing, suppress io.stdout/stderr
+ io.stdout = StreamProxy('stdout')
+ io.stderr = StreamProxy('stderr')
return _ip
39 IPython/utils/
@@ -26,13 +26,21 @@
class IOStream:
- def __init__(self,stream,fallback):
+ def __init__(self,stream, fallback=None):
if not hasattr(stream,'write') or not hasattr(stream,'flush'):
- stream = fallback
+ if fallback is not None:
+ stream = fallback
+ else:
+ raise ValueError("fallback required, but not specified") = stream
self._swrite = stream.write
- self.flush = stream.flush
+ # clone all methods not overridden:
+ def clone(meth):
+ return not hasattr(self, meth) and not meth.startswith('_')
+ for meth in filter(clone, dir(stream)):
+ setattr(self, meth, getattr(stream, meth))
def write(self,data):
@@ -46,11 +54,21 @@ def write(self,data):
# if we get here, something is seriously broken.
print('ERROR - failed to write data to stream:',,
+ def writelines(self, lines):
+ if isinstance(lines, basestring):
+ lines = [lines]
+ for line in lines:
+ self.write(line)
# This class used to have a writeln method, but regular files and streams
# in Python don't have this method. We need to keep this completely
# compatible so we removed it.
+ @property
+ def closed(self):
+ return
def close(self):
@@ -65,10 +83,15 @@ class IOTerm:
# In the future, having IPython channel all its I/O operations through
# this class will make it easier to embed it into other environments which
# are not a normal terminal (such as a GUI-based shell)
- def __init__(self, cin=None, cout=None, cerr=None):
- self.cin = IOStream(cin, sys.stdin)
- self.cout = IOStream(cout, sys.stdout)
- self.cerr = IOStream(cerr, sys.stderr)
+ def __init__(self, stdin=None, stdout=None, stderr=None):
+ self.stdin = IOStream(stdin, sys.stdin)
+ self.stdout = IOStream(stdout, sys.stdout)
+ self.stderr = IOStream(stderr, sys.stderr)
+# setup stdin/stdout/stderr to sys.stdin/sys.stdout/sys.stderr
+stdin = IOStream(sys.stdin)
+stdout = IOStream(sys.stdout)
+stderr = IOStream(sys.stderr)
class Tee(object):
10 IPython/utils/tests/
@@ -15,6 +15,7 @@
import sys
from cStringIO import StringIO
+from subprocess import Popen, PIPE
import as nt
@@ -59,3 +60,12 @@ def test(self):
for chan in ['stdout', 'stderr']:
for check in ['close', 'del']:
yield self.tchan(chan, check)
+def test_io_init():
+ """Test that io.stdin/out/err exist at startup"""
+ for name in ('stdin', 'stdout', 'stderr'):
+ p = Popen([sys.executable, '-c', "from IPython.utils import io;print io.%s.__class__"%name],
+ stdout=PIPE)
+ p.wait()
+ classname =
+ nt.assert_equals(classname, '')
8 IPython/utils/
@@ -16,7 +16,7 @@
import sys
+from IPython.utils import io
# Code
@@ -25,7 +25,7 @@
def warn(msg,level=2,exit_val=1):
"""Standard warning printer. Gives formatting consistency.
- Output is sent to (sys.stderr by default).
+ Output is sent to io.stderr (sys.stderr by default).
@@ -41,9 +41,9 @@ def warn(msg,level=2,exit_val=1):
if level>0:
header = ['','','WARNING: ','ERROR: ','FATAL ERROR: ']
- print >>, '%s%s' % (header[level],msg)
+ print >> io.stderr, '%s%s' % (header[level],msg)
if level == 4:
- print >>,'Exiting.\n'
+ print >> io.stderr,'Exiting.\n'
Something went wrong with that request. Please try again.