Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Execfile #896

Merged
merged 4 commits into from

3 participants

@jstenar
Collaborator

Work related to #818

J�rgen Stena... added some commits
J�rgen Stenarson fix for execfile that does not tolerate non-ascii characters in filen…
…ame.
26e9691
J�rgen Stenarson Fix for 2.x execfile that can not handle non-ascii filenames.
Special case fix for windows where we can not in general use
sys.getfilesystemencoding() to get a str that can be used to access
all files.
1b786a0
@fperez fperez commented on the diff
IPython/utils/py3compat.py
((7 lines not shown))
+ loc = loc if (loc is not None) else glob
+ scripttext = file(fname).read()
+ #compile converts unicode filename to str assuming
+ #ascii. Let's do the conversion before calling compile
+ if isinstance(fname, unicode):
+ filename = unicode_to_str(fname)
+ else:
+ filename = fname
+ exec compile(scripttext, filename, 'exec') in glob, loc
+ else:
+ def execfile(fname, glob, loc=None):
+ if isinstance(fname, unicode):
+ filename = fname.encode(sys.getfilesystemencoding())
+ else:
+ filename = fname
+ __builtin__.execfile(filename, glob, loc)
@fperez Owner
fperez added a note

I thought py3 had removed execfile from the language... Did you actually test this on py3/non-windows? Does it work?

@jstenar Collaborator
jstenar added a note
@fperez Owner
fperez added a note
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
IPython/utils/py3compat.py
@@ -149,3 +151,22 @@ else:
Accepts a string or a function, so it can be used as a decorator."""
return s.format(u='u')
+
+ if sys.platform == 'win32':
+ def execfile(fname, glob, loc=None):
+ loc = loc if (loc is not None) else glob
+ scripttext = file(fname).read()
@fperez Owner
fperez added a note

Should use open instead of file, it's the recommended form.

See http://stackoverflow.com/questions/112970/python-when-to-use-file-vs-open for details.

@jstenar Collaborator
jstenar added a note
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@fperez
Owner

@jstenar, unfortunately when merged against current master, this PR doesn't pass the test suite. Running iptest -vvs IPython.core gives one failure:

======================================================================
ERROR: Test safe_execfile with non-ascii path
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/fperez/usr/lib/python2.7/site-packages/IPython/core/tests/test_interactiveshell.py", line 222, in test_1
    _ip.shell.safe_execfile(self.fname, raise_exceptions=True)
  File "/home/fperez/usr/lib/python2.7/site-packages/IPython/core/interactiveshell.py", line 2284, in safe_execfile
    py3compat.execfile(fname,*where)
  File "/home/fperez/usr/lib/python2.7/site-packages/IPython/utils/py3compat.py", line 172, in execfile
    __builtin__.execfile(filename, glob, loc)
TypeError: must be dict, not None

----------------------------------------------------------------------
Ran 353 tests in 1.176s

FAILED (KNOWNFAIL=2, errors=1)

Do you see it also on Windows? If not, we may need a bit more work to track it down, but if you see the error on Windows, just go ahead and fix it and we'll continue reviewing. Let us know if you need a hand!

@takluyver
Owner

The builtin execfile doesn't use None as the defaults for the namespace arguments. I think we need to use the *where trick to pass it the correct number of arguments.

@fperez
Owner

If you see the fix right away, feel free to just add one extra commit on top of @jstenar's work and then merge it with the fix... You can just follow steps 1 and 2 of the PR manual merge instructions (click on the 'i' at the left of the green merge bar), and then git reset --hard 9403023 to unwind the merge. Then apply the fix, commit and merge to master/push.

@fperez
Owner

I do the above often with PRs where I see a tiny bit of manual cleanup at the end, which is quicker to do myself than to ask the submitter for another round of edits. I think that if the changes will teach the submitter something valuable of the development process (such as adding tests or docs that were missing) it's better to send it back. Similarly if the needed work is significant. But often there's one last bit of easy cleanup missing, and at that point it's just more work to go back and forth on the PR than to simply do it myself and push.

@takluyver
Owner

OK, I'll get on it.

@takluyver takluyver merged commit 9403023 into ipython:master
@takluyver
Owner

Done. For reference, the extra change I made is here: 4adf668.

@fperez
Owner

Thanks a ton, @takluyver! And also @jstenar, for the original work :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Oct 18, 2011
  1. fix for execfile that does not tolerate non-ascii characters in filen…

    J�rgen Stenarson authored
    …ame.
  2. Fix for 2.x execfile that can not handle non-ascii filenames.

    J�rgen Stenarson authored
    Special case fix for windows where we can not in general use
    sys.getfilesystemencoding() to get a str that can be used to access
    all files.
  3. Switching to builtin open instead of file

    J�rgen Stenarson authored
  4. Adding test for safe_execfile call with non-ascii path

    J�rgen Stenarson authored
This page is out of date. Refresh to see the latest.
View
8 IPython/core/interactiveshell.py
@@ -2257,8 +2257,12 @@ def safe_execfile(self, fname, *where, **kw):
exit_ignore : bool (False)
If True, then silence SystemExit for non-zero status (it is always
silenced for zero status, as it is so common).
+ raise_exceptions : bool (False)
+ If True raise exceptions everywhere. Meant for testing.
+
"""
kw.setdefault('exit_ignore', False)
+ kw.setdefault('raise_exceptions', False)
fname = os.path.abspath(os.path.expanduser(fname))
@@ -2288,9 +2292,13 @@ def safe_execfile(self, fname, *where, **kw):
# 0
# For other exit status, we show the exception unless
# explicitly silenced, but only in short form.
+ if kw['raise_exceptions']:
+ raise
if status.code not in (0, None) and not kw['exit_ignore']:
self.showtraceback(exception_only=True)
except:
+ if kw['raise_exceptions']:
+ raise
self.showtraceback()
def safe_execfile_ipy(self, fname):
View
27 IPython/core/tests/test_interactiveshell.py
@@ -1,3 +1,4 @@
+# -*- coding: utf-8 -*-
"""Tests for the key interactiveshell module.
Historically the main classes in interactiveshell have been under-tested. This
@@ -19,7 +20,11 @@
# Imports
#-----------------------------------------------------------------------------
# stdlib
+import os
+import shutil
+import tempfile
import unittest
+from os.path import join
from StringIO import StringIO
from IPython.testing import decorators as dec
@@ -193,3 +198,25 @@ def test_drop_by_id(self):
assert name not in ip.user_ns_hidden, name
assert ip.user_ns['b'] == 12
ip.reset()
+
+class TestSafeExecfileNonAsciiPath(unittest.TestCase):
+
+ def setUp(self):
+ self.BASETESTDIR = tempfile.mkdtemp()
+ self.TESTDIR = join(self.BASETESTDIR, u"åäö")
+ os.mkdir(self.TESTDIR)
+ with open(join(self.TESTDIR, u"åäötestscript.py"), "w") as sfile:
+ sfile.write("pass\n")
+ self.oldpath = os.getcwdu()
+ os.chdir(self.TESTDIR)
+ self.fname = u"åäötestscript.py"
+
+
+ def tearDown(self):
+ os.chdir(self.oldpath)
+ shutil.rmtree(self.BASETESTDIR)
+
+ def test_1(self):
+ """Test safe_execfile with non-ascii path
+ """
+ _ip.shell.safe_execfile(self.fname, raise_exceptions=True)
View
21 IPython/utils/py3compat.py
@@ -1,5 +1,6 @@
# coding: utf-8
"""Compatibility tricks for Python 3. Mainly to do with unicode."""
+import __builtin__
import functools
import sys
import re
@@ -142,6 +143,7 @@ def MethodType(func, instance):
def doctest_refactor_print(func_or_str):
return func_or_str
+
# Abstract u'abc' syntax:
@_modify_str_or_docstring
def u_format(s):
@@ -149,3 +151,22 @@ def u_format(s):
Accepts a string or a function, so it can be used as a decorator."""
return s.format(u='u')
+
+ if sys.platform == 'win32':
+ def execfile(fname, glob=None, loc=None):
+ loc = loc if (loc is not None) else glob
+ scripttext = __builtin__.open(fname).read()
+ #compile converts unicode filename to str assuming
+ #ascii. Let's do the conversion before calling compile
+ if isinstance(fname, unicode):
+ filename = unicode_to_str(fname)
+ else:
+ filename = fname
+ exec compile(scripttext, filename, 'exec') in glob, loc
+ else:
+ def execfile(fname, glob=None, loc=None):
+ if isinstance(fname, unicode):
+ filename = fname.encode(sys.getfilesystemencoding())
+ else:
+ filename = fname
+ __builtin__.execfile(filename, glob, loc)
@fperez Owner
fperez added a note

I thought py3 had removed execfile from the language... Did you actually test this on py3/non-windows? Does it work?

@jstenar Collaborator
jstenar added a note
@fperez Owner
fperez added a note
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.