Skip to content
This repository

Execfile #896

Merged
merged 4 commits into from over 2 years ago

3 participants

Jörgen Stenarson Fernando Perez Thomas Kluyver
Jörgen Stenarson
Collaborator

Work related to #818

Jörgen Stena... added some commits September 22, 2011
fix for execfile that does not tolerate non-ascii characters in filen…
…ame.
26e9691
Fix for 2.x execfile that can not handle non-ascii filenames.
Special case fix for windows where we can not in general use
sys.getfilesystemencoding() to get a str that can be used to access
all files.
1b786a0
Fernando Perez fperez commented on the diff October 18, 2011
IPython/utils/py3compat.py
((7 lines not shown))
  157
+            loc = loc if (loc is not None) else glob
  158
+            scripttext = file(fname).read()
  159
+            #compile converts unicode filename to str assuming
  160
+            #ascii. Let's do the conversion before calling compile
  161
+            if isinstance(fname, unicode):
  162
+                filename = unicode_to_str(fname)
  163
+            else:
  164
+                filename = fname
  165
+            exec compile(scripttext, filename, 'exec') in glob, loc
  166
+    else:
  167
+        def execfile(fname, glob, loc=None):
  168
+            if isinstance(fname, unicode):
  169
+                filename = fname.encode(sys.getfilesystemencoding())
  170
+            else:
  171
+                filename = fname
  172
+            __builtin__.execfile(filename, glob, loc)
3
Fernando Perez Owner
fperez added a note October 18, 2011

I thought py3 had removed execfile from the language... Did you actually test this on py3/non-windows? Does it work?

Jörgen Stenarson Collaborator
jstenar added a note October 18, 2011
Fernando Perez Owner
fperez added a note October 18, 2011
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
IPython/utils/py3compat.py
@@ -149,3 +151,22 @@ else:
149 151
         
150 152
         Accepts a string or a function, so it can be used as a decorator."""
151 153
         return s.format(u='u')
  154
+
  155
+    if sys.platform == 'win32':
  156
+        def execfile(fname, glob, loc=None):
  157
+            loc = loc if (loc is not None) else glob
  158
+            scripttext = file(fname).read()
2
Fernando Perez Owner
fperez added a note October 18, 2011

Should use open instead of file, it's the recommended form.

See http://stackoverflow.com/questions/112970/python-when-to-use-file-vs-open for details.

Jörgen Stenarson Collaborator
jstenar added a note October 18, 2011
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Fernando Perez
Owner

@jstenar, unfortunately when merged against current master, this PR doesn't pass the test suite. Running iptest -vvs IPython.core gives one failure:

======================================================================
ERROR: Test safe_execfile with non-ascii path
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/fperez/usr/lib/python2.7/site-packages/IPython/core/tests/test_interactiveshell.py", line 222, in test_1
    _ip.shell.safe_execfile(self.fname, raise_exceptions=True)
  File "/home/fperez/usr/lib/python2.7/site-packages/IPython/core/interactiveshell.py", line 2284, in safe_execfile
    py3compat.execfile(fname,*where)
  File "/home/fperez/usr/lib/python2.7/site-packages/IPython/utils/py3compat.py", line 172, in execfile
    __builtin__.execfile(filename, glob, loc)
TypeError: must be dict, not None

----------------------------------------------------------------------
Ran 353 tests in 1.176s

FAILED (KNOWNFAIL=2, errors=1)

Do you see it also on Windows? If not, we may need a bit more work to track it down, but if you see the error on Windows, just go ahead and fix it and we'll continue reviewing. Let us know if you need a hand!

Thomas Kluyver
Collaborator

The builtin execfile doesn't use None as the defaults for the namespace arguments. I think we need to use the *where trick to pass it the correct number of arguments.

Fernando Perez
Owner

If you see the fix right away, feel free to just add one extra commit on top of @jstenar's work and then merge it with the fix... You can just follow steps 1 and 2 of the PR manual merge instructions (click on the 'i' at the left of the green merge bar), and then git reset --hard 9403023 to unwind the merge. Then apply the fix, commit and merge to master/push.

Fernando Perez
Owner

I do the above often with PRs where I see a tiny bit of manual cleanup at the end, which is quicker to do myself than to ask the submitter for another round of edits. I think that if the changes will teach the submitter something valuable of the development process (such as adding tests or docs that were missing) it's better to send it back. Similarly if the needed work is significant. But often there's one last bit of easy cleanup missing, and at that point it's just more work to go back and forth on the PR than to simply do it myself and push.

Thomas Kluyver
Collaborator

OK, I'll get on it.

Thomas Kluyver takluyver merged commit 9403023 into from October 19, 2011
Thomas Kluyver takluyver closed this October 19, 2011
Thomas Kluyver
Collaborator

Done. For reference, the extra change I made is here: 4adf668.

Fernando Perez
Owner

Thanks a ton, @takluyver! And also @jstenar, for the original work :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Showing 4 unique commits by 1 author.

Oct 18, 2011
fix for execfile that does not tolerate non-ascii characters in filen…
…ame.
26e9691
Fix for 2.x execfile that can not handle non-ascii filenames.
Special case fix for windows where we can not in general use
sys.getfilesystemencoding() to get a str that can be used to access
all files.
1b786a0
Switching to builtin open instead of file c5e0bc1
Adding test for safe_execfile call with non-ascii path 9403023
This page is out of date. Refresh to see the latest.
8  IPython/core/interactiveshell.py
@@ -2257,8 +2257,12 @@ def safe_execfile(self, fname, *where, **kw):
2257 2257
         exit_ignore : bool (False)
2258 2258
             If True, then silence SystemExit for non-zero status (it is always
2259 2259
             silenced for zero status, as it is so common).
  2260
+        raise_exceptions : bool (False)
  2261
+            If True raise exceptions everywhere. Meant for testing.
  2262
+
2260 2263
         """
2261 2264
         kw.setdefault('exit_ignore', False)
  2265
+        kw.setdefault('raise_exceptions', False)
2262 2266
 
2263 2267
         fname = os.path.abspath(os.path.expanduser(fname))
2264 2268
 
@@ -2288,9 +2292,13 @@ def safe_execfile(self, fname, *where, **kw):
2288 2292
                 # 0
2289 2293
                 # For other exit status, we show the exception unless
2290 2294
                 # explicitly silenced, but only in short form.
  2295
+                if kw['raise_exceptions']:
  2296
+                    raise
2291 2297
                 if status.code not in (0, None) and not kw['exit_ignore']:
2292 2298
                     self.showtraceback(exception_only=True)
2293 2299
             except:
  2300
+                if kw['raise_exceptions']:
  2301
+                    raise
2294 2302
                 self.showtraceback()
2295 2303
 
2296 2304
     def safe_execfile_ipy(self, fname):
27  IPython/core/tests/test_interactiveshell.py
... ...
@@ -1,3 +1,4 @@
  1
+# -*- coding: utf-8 -*-
1 2
 """Tests for the key interactiveshell module.
2 3
 
3 4
 Historically the main classes in interactiveshell have been under-tested.  This
@@ -19,7 +20,11 @@
19 20
 # Imports
20 21
 #-----------------------------------------------------------------------------
21 22
 # stdlib
  23
+import os
  24
+import shutil
  25
+import tempfile
22 26
 import unittest
  27
+from os.path import join
23 28
 from StringIO import StringIO
24 29
 
25 30
 from IPython.testing import decorators as dec
@@ -193,3 +198,25 @@ def test_drop_by_id(self):
193 198
             assert name not in ip.user_ns_hidden, name
194 199
         assert ip.user_ns['b'] == 12
195 200
         ip.reset()
  201
+
  202
+class TestSafeExecfileNonAsciiPath(unittest.TestCase):
  203
+
  204
+    def setUp(self):
  205
+        self.BASETESTDIR = tempfile.mkdtemp()
  206
+        self.TESTDIR = join(self.BASETESTDIR, u"åäö")
  207
+        os.mkdir(self.TESTDIR)
  208
+        with open(join(self.TESTDIR, u"åäötestscript.py"), "w") as sfile:
  209
+            sfile.write("pass\n")
  210
+        self.oldpath = os.getcwdu()
  211
+        os.chdir(self.TESTDIR)
  212
+        self.fname = u"åäötestscript.py"
  213
+
  214
+
  215
+    def tearDown(self):
  216
+        os.chdir(self.oldpath)
  217
+        shutil.rmtree(self.BASETESTDIR)
  218
+
  219
+    def test_1(self):
  220
+        """Test safe_execfile with non-ascii path
  221
+        """
  222
+        _ip.shell.safe_execfile(self.fname, raise_exceptions=True)
21  IPython/utils/py3compat.py
... ...
@@ -1,5 +1,6 @@
1 1
 # coding: utf-8
2 2
 """Compatibility tricks for Python 3. Mainly to do with unicode."""
  3
+import __builtin__
3 4
 import functools
4 5
 import sys
5 6
 import re
@@ -142,6 +143,7 @@ def MethodType(func, instance):
142 143
     def doctest_refactor_print(func_or_str):
143 144
         return func_or_str
144 145
 
  146
+
145 147
     # Abstract u'abc' syntax:
146 148
     @_modify_str_or_docstring
147 149
     def u_format(s):
@@ -149,3 +151,22 @@ def u_format(s):
149 151
         
150 152
         Accepts a string or a function, so it can be used as a decorator."""
151 153
         return s.format(u='u')
  154
+
  155
+    if sys.platform == 'win32':
  156
+        def execfile(fname, glob=None, loc=None):
  157
+            loc = loc if (loc is not None) else glob
  158
+            scripttext = __builtin__.open(fname).read()
  159
+            #compile converts unicode filename to str assuming
  160
+            #ascii. Let's do the conversion before calling compile
  161
+            if isinstance(fname, unicode):
  162
+                filename = unicode_to_str(fname)
  163
+            else:
  164
+                filename = fname
  165
+            exec compile(scripttext, filename, 'exec') in glob, loc
  166
+    else:
  167
+        def execfile(fname, glob=None, loc=None):
  168
+            if isinstance(fname, unicode):
  169
+                filename = fname.encode(sys.getfilesystemencoding())
  170
+            else:
  171
+                filename = fname
  172
+            __builtin__.execfile(filename, glob, loc)
Commit_comment_tip

Tip: You can add notes to lines in a file. Hover to the left of a line to make a note

Something went wrong with that request. Please try again.