Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Fix string input2 #265

Merged
5 commits merged into from

3 participants

@fperez
Owner

Completes the fix started by Robert Kern for the execution of naked strings. This fixes the case where the input is a multiline string.

The actual fix was a trivial one-liner, but I took the opportunity to clean up some aspects of our test system to simplify them and remove some odd special casing that made testing harder. This let me add some proper tests for this bug so it doesn't resurface, and there was a bit of cleanup in the process.

fperez added some commits
@fperez fperez Small cleanup of dead debug code 65f8567
@fperez fperez Improve docs and comments of some internal tools, and of testing code d42a12c
@fperez fperez Remove code that is not needed once python 2.6 is our minimal version. 346bc9b
@fperez fperez Limit special-casing of _ variable to doctests.
In doctests, _ *must* be special-cased and removed from the user's
namespace because otherwise Python won't set it.  But we were doing
this special-casing unconditionally, making it impossible to write
unittests that checked the _ variable after cell execution.

This commit makes the special-casing of _ apply only in the doctest
execution (where it's needed) and leaves normal unittests alone.
827b268
@fperez fperez Fix bug with execution of naked multiline strings.
The actual bug fix was a trivial one-line change, made here.  The rest
of the commits in this series improve our testing machinery and clean
up related code.

The actual fix was just calling the run_source instead of the run_code
method, which should only be called with compiled code objects.
de4ba65
@rkern

+1

@takluyver
Owner

Looks good to me.

@fperez
Owner

Thanks a lot for the reviews! Pushed and closed.

@damianavila damianavila referenced this pull request from a commit
Commit has since been removed from the repository and is no longer available.
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Feb 3, 2011
  1. @fperez
Commits on Feb 4, 2011
  1. @fperez
  2. @fperez
  3. @fperez

    Limit special-casing of _ variable to doctests.

    fperez authored
    In doctests, _ *must* be special-cased and removed from the user's
    namespace because otherwise Python won't set it.  But we were doing
    this special-casing unconditionally, making it impossible to write
    unittests that checked the _ variable after cell execution.
    
    This commit makes the special-casing of _ apply only in the doctest
    execution (where it's needed) and leaves normal unittests alone.
  4. @fperez

    Fix bug with execution of naked multiline strings.

    fperez authored
    The actual bug fix was a trivial one-line change, made here.  The rest
    of the commits in this series improve our testing machinery and clean
    up related code.
    
    The actual fix was just calling the run_source instead of the run_code
    method, which should only be called with compiled code objects.
This page is out of date. Refresh to see the latest.
View
5 IPython/core/displayhook.py
@@ -260,11 +260,14 @@ def update_user_ns(self, result):
self.flush()
# Don't overwrite '_' and friends if '_' is in __builtin__ (otherwise
# we cause buggy behavior for things like gettext).
+
if '_' not in __builtin__.__dict__:
self.___ = self.__
self.__ = self._
self._ = result
- self.shell.user_ns.update({'_':self._,'__':self.__,'___':self.___})
+ self.shell.user_ns.update({'_':self._,
+ '__':self.__,
+ '___':self.___})
# hackish access to top-level namespace to create _1,_2... dynamically
to_main = {}
View
37 IPython/core/interactiveshell.py
@@ -4,7 +4,7 @@
#-----------------------------------------------------------------------------
# Copyright (C) 2001 Janko Hauser <jhauser@zscout.de>
# Copyright (C) 2001-2007 Fernando Perez. <fperez@colorado.edu>
-# Copyright (C) 2008-2010 The IPython Development Team
+# Copyright (C) 2008-2011 The IPython Development Team
#
# Distributed under the terms of the BSD License. The full license is in
# the file COPYING, distributed as part of this software.
@@ -2113,19 +2113,6 @@ def run_cell(self, cell):
self.history_manager.store_inputs(ipy_cell, cell)
self.logger.log(ipy_cell, cell)
- # dbg code!!!
- if 0:
- def myapp(self, val): # dbg
- import traceback as tb
- stack = ''.join(tb.format_stack())
- print 'Value:', val
- print 'Stack:\n', stack
- list.append(self, val)
-
- import new
- self.history_manager.input_hist_parsed.append = types.MethodType(myapp,
- self.history_manager.input_hist_parsed)
- # End dbg
# All user code execution must happen with our context managers active
with nested(self.builtin_trap, self.display_trap):
@@ -2167,15 +2154,31 @@ def myapp(self, val): # dbg
self.execution_count += 1
def run_one_block(self, block):
- """Run a single interactive block.
+ """Run a single interactive block of source code.
If the block is single-line, dynamic transformations are applied to it
(like automagics, autocall and alias recognition).
+
+ If the block is multi-line, it must consist of valid Python code only.
+
+ Parameters
+ ----------
+ block : string
+ A (possibly multiline) string of code to be executed.
+
+ Returns
+ -------
+ The output of the underlying execution method used, be it
+ :meth:`run_source` or :meth:`run_single_line`.
"""
if len(block.splitlines()) <= 1:
out = self.run_single_line(block)
else:
- out = self.run_code(block)
+ # Call run_source, which correctly compiles the input cell.
+ # run_code must only be called when we know we have a code object,
+ # as it does a naked exec and the compilation mode may not be what
+ # we wanted.
+ out = self.run_source(block)
return out
def run_single_line(self, line):
@@ -2329,7 +2332,7 @@ def run_code(self, code_obj, post_execute=True):
try:
try:
self.hooks.pre_run_code_hook()
- #rprint('Running code') # dbg
+ #rprint('Running code', repr(code_obj)) # dbg
exec code_obj in self.user_global_ns, self.user_ns
finally:
# Reset our crash handler in place
View
5 IPython/core/tests/test_inputsplitter.py
@@ -1,5 +1,10 @@
# -*- coding: utf-8 -*-
"""Tests for the inputsplitter module.
+
+Authors
+-------
+* Fernando Perez
+* Robert Kern
"""
#-----------------------------------------------------------------------------
# Copyright (C) 2010 The IPython Development Team
View
37 IPython/core/tests/test_interactiveshell.py
@@ -0,0 +1,37 @@
+"""Tests for the key interactiveshell module.
+
+Historically the main classes in interactiveshell have been under-tested. This
+module should grow as many single-method tests as possible to trap many of the
+recurring bugs we seem to encounter with high-level interaction.
+
+Authors
+-------
+* Fernando Perez
+"""
+#-----------------------------------------------------------------------------
+# Copyright (C) 2011 The IPython Development Team
+#
+# Distributed under the terms of the BSD License. The full license is in
+# the file COPYING, distributed as part of this software.
+#-----------------------------------------------------------------------------
+
+#-----------------------------------------------------------------------------
+# Imports
+#-----------------------------------------------------------------------------
+# stdlib
+import unittest
+
+#-----------------------------------------------------------------------------
+# Tests
+#-----------------------------------------------------------------------------
+
+class InteractiveShellTestCase(unittest.TestCase):
+ def test_naked_string_cells(self):
+ """Test that cells with only naked strings are fully executed"""
+ ip = get_ipython()
+ # First, single-line inputs
+ ip.run_cell('"a"\n')
+ self.assertEquals(ip.user_ns['_'], 'a')
+ # And also multi-line cells
+ ip.run_cell('"""a\nb"""\n')
+ self.assertEquals(ip.user_ns['_'], 'a\nb')
View
121 IPython/testing/_doctest26.py
@@ -1,121 +0,0 @@
-"""Code taken from the Python2.6 standard library for backwards compatibility.
-
-This is just so we can use 2.6 features when running in 2.5, the code below is
-copied verbatim from the stdlib's collections and doctest modules.
-"""
-
-#-----------------------------------------------------------------------------
-# Copyright (C) 2009 The IPython Development Team
-#
-# Distributed under the terms of the BSD License. The full license is in
-# the file COPYING, distributed as part of this software.
-#-----------------------------------------------------------------------------
-
-#-----------------------------------------------------------------------------
-# Imports
-#-----------------------------------------------------------------------------
-
-from keyword import iskeyword as _iskeyword
-from operator import itemgetter as _itemgetter
-import sys as _sys
-
-def namedtuple(typename, field_names, verbose=False):
- """Returns a new subclass of tuple with named fields.
-
- >>> Point = namedtuple('Point', 'x y')
- >>> Point.__doc__ # docstring for the new class
- 'Point(x, y)'
- >>> p = Point(11, y=22) # instantiate with positional args or keywords
- >>> p[0] + p[1] # indexable like a plain tuple
- 33
- >>> x, y = p # unpack like a regular tuple
- >>> x, y
- (11, 22)
- >>> p.x + p.y # fields also accessable by name
- 33
- >>> d = p._asdict() # convert to a dictionary
- >>> d['x']
- 11
- >>> Point(**d) # convert from a dictionary
- Point(x=11, y=22)
- >>> p._replace(x=100) # _replace() is like str.replace() but targets named fields
- Point(x=100, y=22)
-
- """
-
- # Parse and validate the field names. Validation serves two purposes,
- # generating informative error messages and preventing template injection attacks.
- if isinstance(field_names, basestring):
- field_names = field_names.replace(',', ' ').split() # names separated by whitespace and/or commas
- field_names = tuple(map(str, field_names))
- for name in (typename,) + field_names:
- if not all(c.isalnum() or c=='_' for c in name):
- raise ValueError('Type names and field names can only contain alphanumeric characters and underscores: %r' % name)
- if _iskeyword(name):
- raise ValueError('Type names and field names cannot be a keyword: %r' % name)
- if name[0].isdigit():
- raise ValueError('Type names and field names cannot start with a number: %r' % name)
- seen_names = set()
- for name in field_names:
- if name.startswith('_'):
- raise ValueError('Field names cannot start with an underscore: %r' % name)
- if name in seen_names:
- raise ValueError('Encountered duplicate field name: %r' % name)
- seen_names.add(name)
-
- # Create and fill-in the class template
- numfields = len(field_names)
- argtxt = repr(field_names).replace("'", "")[1:-1] # tuple repr without parens or quotes
- reprtxt = ', '.join('%s=%%r' % name for name in field_names)
- dicttxt = ', '.join('%r: t[%d]' % (name, pos) for pos, name in enumerate(field_names))
- template = '''class %(typename)s(tuple):
- '%(typename)s(%(argtxt)s)' \n
- __slots__ = () \n
- _fields = %(field_names)r \n
- def __new__(_cls, %(argtxt)s):
- return _tuple.__new__(_cls, (%(argtxt)s)) \n
- @classmethod
- def _make(cls, iterable, new=tuple.__new__, len=len):
- 'Make a new %(typename)s object from a sequence or iterable'
- result = new(cls, iterable)
- if len(result) != %(numfields)d:
- raise TypeError('Expected %(numfields)d arguments, got %%d' %% len(result))
- return result \n
- def __repr__(self):
- return '%(typename)s(%(reprtxt)s)' %% self \n
- def _asdict(t):
- 'Return a new dict which maps field names to their values'
- return {%(dicttxt)s} \n
- def _replace(_self, **kwds):
- 'Return a new %(typename)s object replacing specified fields with new values'
- result = _self._make(map(kwds.pop, %(field_names)r, _self))
- if kwds:
- raise ValueError('Got unexpected field names: %%r' %% kwds.keys())
- return result \n
- def __getnewargs__(self):
- return tuple(self) \n\n''' % locals()
- for i, name in enumerate(field_names):
- template += ' %s = _property(_itemgetter(%d))\n' % (name, i)
- if verbose:
- print template
-
- # Execute the template string in a temporary namespace and
- # support tracing utilities by setting a value for frame.f_globals['__name__']
- namespace = dict(_itemgetter=_itemgetter, __name__='namedtuple_%s' % typename,
- _property=property, _tuple=tuple)
- try:
- exec template in namespace
- except SyntaxError, e:
- raise SyntaxError(e.message + ':\n' + template)
- result = namespace[typename]
-
- # For pickling to work, the __module__ variable needs to be set to the frame
- # where the named tuple is created. Bypass this step in enviroments where
- # sys._getframe is not defined (Jython for example).
- if hasattr(_sys, '_getframe'):
- result.__module__ = _sys._getframe(1).f_globals.get('__name__', '__main__')
-
- return result
-
-
-TestResults = namedtuple('TestResults', 'failed attempted')
View
21 IPython/testing/globalipapp.py
@@ -68,11 +68,19 @@ class ipnsdict(dict):
This subclass adds a simple checkpointing capability so that when testing
machinery clears it (we use it as the test execution context), it doesn't
get completely destroyed.
+
+ In addition, it can handle the presence of the '_' key in a special manner,
+ which is needed because of how Python's doctest machinery operates with
+ '_'. See constructor and :meth:`update` for details.
"""
def __init__(self,*a):
dict.__init__(self,*a)
self._savedict = {}
+ # If this flag is True, the .update() method will unconditionally
+ # remove a key named '_'. This is so that such a dict can be used as a
+ # namespace in doctests that call '_'.
+ self.protect_underscore = False
def clear(self):
dict.clear(self)
@@ -86,10 +94,15 @@ def update(self,other):
self._checkpoint()
dict.update(self,other)
- # If '_' is in the namespace, python won't set it when executing code,
- # and we have examples that test it. So we ensure that the namespace
- # is always 'clean' of it before it's used for test code execution.
- self.pop('_',None)
+ if self.protect_underscore:
+ # If '_' is in the namespace, python won't set it when executing
+ # code *in doctests*, and we have multiple doctests that use '_'.
+ # So we ensure that the namespace is always 'clean' of it before
+ # it's used for test code execution.
+ # This flag is only turned on by the doctest machinery, so that
+ # normal test code can assume the _ key is updated like any other
+ # key and can test for its presence after cell executions.
+ self.pop('_', None)
# The builtins namespace must *always* be the real __builtin__ module,
# else weird stuff happens. The main ipython code does have provisions
View
6 IPython/testing/ipunittest.py
@@ -39,11 +39,7 @@
import re
import sys
import unittest
-from doctest import DocTestFinder, DocTestRunner
-try:
- from doctest import TestResults
-except:
- from ._doctest26 import TestResults
+from doctest import DocTestFinder, DocTestRunner, TestResults
# We already have python3-compliant code for parametric tests
if sys.version[0]=='2':
View
7 IPython/testing/plugin/ipdoctest.py
@@ -273,6 +273,10 @@ def setUp(self):
# fills with the necessary info from the module being tested).
_ip.user_ns.update(self._dt_test.globs)
self._dt_test.globs = _ip.user_ns
+ # IPython must protect the _ key in the namespace (it can't exist)
+ # so that Python's doctest code sets it naturally, so we enable
+ # this feature of our testing namespace.
+ _ip.user_ns.protect_underscore = True
super(DocTestCase, self).setUp()
@@ -282,6 +286,9 @@ def tearDown(self):
# teardown doesn't destroy the ipython namespace
if isinstance(self._dt_test.examples[0],IPExample):
self._dt_test.globs = self._dt_test_globs_ori
+ # Restore the behavior of the '_' key in the user namespace to
+ # normal after each doctest, so that unittests behave normally
+ _ip.user_ns.protect_underscore = False
# XXX - fperez: I am not sure if this is truly a bug in nose 0.11, but
# it does look like one to me: its tearDown method tries to run
Something went wrong with that request. Please try again.