config.py uses imp.load_source which is deprecated in Python 3 #214

Closed
pfmoore opened this Issue Mar 5, 2015 · 7 comments

Projects

None yet

2 participants

@pfmoore
Contributor
pfmoore commented Mar 5, 2015

In Python 3, imp.load_source is undocumented (apparently it was deprecated and removed via http://bugs.python.org/issue14551 with some discussion of how it is "fundamentally broken" in https://mail.python.org/pipermail/python-ideas/2014-December/030265.html).

On Windows under Python 3.3, the invoke tests fail somewhere in the compatibility implementation of imp.load_source.

An alternative approach to loading config from Python files should be implemented for Python 3. Would something like the following work?

import tokenize

values = {}
with tokenize.open(filename) as f:
    exec(f.read(), values)
@bitprophet
Member

That's unfortunate, it's always worked fine under 3.3 and 3.4 on POSIX platforms for me.

The discussion thread linked implies the following could be the "correct" replacement for importing arbitrary files, and it works for me in a Python 3.4 shell:

from importlib.machinery import SourceFileLoader

mymodule = SourceFileLoader('modname', '/path/to/file.py').load_module()

So if that functions on your end I'd be +1 to a six.PY3 based branch which uses imp.load_source on Python 2 (importlib does not really exist in 2) and the above in Python 3.

@pfmoore
Contributor
pfmoore commented Mar 5, 2015

I'll give that a go - I must admit I was getting a bit befuddled at that point and couldn't quite follow what the suggested alternative was meant to be...

@pfmoore
Contributor
pfmoore commented Mar 5, 2015

Looks like SourceFileLoader doesn't exist on Python 3.2. So I've used it for 3.3 and later. Just pushed the change to my windows test fixes branch (because that one runs the tests on appveyor). I can make a separate PR of it once it's working.

@pfmoore
Contributor
pfmoore commented Mar 5, 2015

OK, the big issue seems to be that if you try to use any of the load functions on Windows under Python 3.2, it breaks if the file doesn't exist. I'm trying to avoid the error by checking for file existence first. This is so messy, but I'm not sure we can do anything better as it's a Python bug. Unless you feel like dropping support for Python 3.2 ;-)

@pfmoore
Contributor
pfmoore commented Mar 5, 2015

Yay! Working on Appveyor and Travis

@pfmoore
Contributor
pfmoore commented Mar 5, 2015

PR #215 created for this

@bitprophet
Member

Yaaay

@bitprophet bitprophet closed this Mar 9, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment