Skip to content

Loading…

Allow %loadpy to load remote URLs that don't end in .py #1161

Merged
merged 5 commits into from

2 participants

@fperez
IPython member

Will also fix same encoding bug we found for notebooks next.

@takluyver takluyver commented on an outdated diff
IPython/core/magic.py
@@ -2151,7 +2151,11 @@ def magic_loadpy(self, arg_s):
%loadpy http://www.example.com/myscript.py
"""
arg_s = unquote_filename(arg_s)
- if not arg_s.endswith('.py'):
+ if not arg_s.startswith(('http://', 'https://')) \
+ and not arg_s.endswith('.py'):
+ # Local files must be .py; for remote URLs it's possible that the
+ # fetch URL doesn't have a .py in it (many servers have an opaque
+ # URL, such as scipy-central.org).
raise ValueError('%%load only works with .py files: %s' % arg_s)
if arg_s.startswith('http'):
@takluyver IPython member

Don't forget to change this one as well (same issue with local files like 'httpx.py')

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@fperez
IPython member

Reviewed by @takluyver and @minrk on IRC in addition to what's here, merging now.

@fperez fperez closed this
@fperez fperez merged commit 3deb45e into master
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Dec 15, 2011
  1. @fperez
  2. @fperez

    Fix and clarify the logic for local/remote urls.

    fperez committed
    Use extra local variables that are easier to read than double-negatives.
  3. @takluyver
  4. @takluyver

    Fix %loadpy for Python 3.

    takluyver committed
Commits on Dec 16, 2011
  1. @fperez

    Add encoding explanatory comment

    fperez committed
Showing with 27 additions and 7 deletions.
  1. +27 −7 IPython/core/magic.py
View
34 IPython/core/magic.py
@@ -96,6 +96,9 @@ def needs_local_scope(func):
# Used for exception handling in magic_edit
class MacroToEdit(ValueError): pass
+# Taken from PEP 263, this is the official encoding regexp.
+_encoding_declaration_re = re.compile(r"^#.*coding[:=]\s*([-\w.]+)")
+
#***************************************************************************
# Main class implementing Magic functionality
@@ -2151,16 +2154,33 @@ def magic_loadpy(self, arg_s):
%loadpy http://www.example.com/myscript.py
"""
arg_s = unquote_filename(arg_s)
- if not arg_s.endswith('.py'):
+ remote_url = arg_s.startswith(('http://', 'https://'))
+ local_url = not remote_url
+ if local_url and not arg_s.endswith('.py'):
+ # Local files must be .py; for remote URLs it's possible that the
+ # fetch URL doesn't have a .py in it (many servers have an opaque
+ # URL, such as scipy-central.org).
raise ValueError('%%load only works with .py files: %s' % arg_s)
- if arg_s.startswith('http'):
+ if remote_url:
import urllib2
- response = urllib2.urlopen(arg_s)
- content = response.read()
+ fileobj = urllib2.urlopen(arg_s)
+ # While responses have a .info().getencoding() way of asking for
+ # their encoding, in *many* cases the return value is bogus. In
+ # the wild, servers serving utf-8 but declaring latin-1 are
+ # extremely common, as the old HTTP standards specify latin-1 as
+ # the default but many modern filesystems use utf-8. So we can NOT
+ # rely on the headers. Short of building complex encoding-guessing
+ # logic, going with utf-8 is a simple solution likely to be right
+ # in most real-world cases.
+ linesource = fileobj.read().decode('utf-8', 'replace').splitlines()
else:
- with open(arg_s) as f:
- content = f.read()
- self.set_next_input(content)
+ fileobj = linesource = open(arg_s)
+
+ # Strip out encoding declarations
+ lines = [l for l in linesource if not _encoding_declaration_re.match(l)]
+ fileobj.close()
+
+ self.set_next_input(os.linesep.join(lines))
def _find_edit_target(self, args, opts, last_call):
"""Utility method used by magic_edit to find what to edit."""
Something went wrong with that request. Please try again.