New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
imp.find_module reacts badly to iterator #78598
Comments
RuntimeError: sys.path must be a list of directory names Secondly, it would be helpful if one could pass an iterator. I'm thinking in particular of the situation where one wants to import something from a large folder tree, and the module in question is likely to be found early in the search process, so that it is more efficient to explore the folder tree incrementally. |
There are several issues at hand here, Phillip. I'll enumerate them below. Thanks for taking the time to let us know about this. However, I'm closing this issue since realistically the behavior of imp.find_module() isn't going to change, particularly in Python 2.7. Even though the issue is closed, feel free to reply, particularly about how you are using imp.find_module() (we may be able to point you toward how to use importlib instead). Also, I've changed this issue's type to "enhancement". imp.find_module() is working as designed, so what you are looking for is a feature request. Consequently there's a much higher bar for justifying a change. Here are reasons why the requested change doesn't reach that bar:
So imp.find_module() is not going to change.
We highly recommend that everyone move to Python 3 as soon as possible. Hopefully you are in a position to do so. If you're stuck on Python 2.7 then you miss the advantages of importlib, along with a ton of other benefits. If you are not going to be able to migrate before 2020 then send an email to python-list@python.org asking for recommendations on what to do.
"Deprecated since version 3.4: The imp package is pending deprecation in favor of importlib." [1] The importlib package should have everything you need. What are you using imp.find_module() for? We should be able to demonstrate the equivalent using importlib.
[1] https://docs.python.org/3/library/imp.html#module-imp |
It appears that the On Fri, Aug 17, 2018 at 9:05 AM Eric Snow <report@bugs.python.org> wrote:
|
Saying "the available functionality is massively inefficient" is unnecessarily hostile towards those of us who actually wrote and maintain that code. Without diving into the code, chances are that requirement is there so that the C code can use macros to access the list as efficiently as possible. Now if you want to propose specific changes to importlib's code for it to work with iterables instead of just lists then we would be happy to review the pull request. |
My apologies for the tone of my remark. I am grateful to you and others I'm attaching the wrapper code that I created to work around the problem. Phillip def expander(paths='./*'):
"""
OVERVIEW This function is a generator, i.e., creates an iterator that recursively INPUTS
Each path in RETURN VALUES The returned value is an iterator. Invoking the if isinstance(paths, basestring):
paths= paths.split(os.pathsep) elif not isinstance(paths, list): found= set()
for path in paths:
if path.endswith('/*') or path.endswith('\*'):
# A recursive search of subfolders is required:
for item in os.walk(path[:-2]):
base= os.path.abspath(item[0])
new= [os.path.join(base, nested) for nested in item[1]]
for item in new:
if not item in found:
found.add(item)
yield item
# No recursive search is required:
if not item in found:
found.add(item)
yield item # end for path in paths def find_module(module_name, in_folders=[]):
"""
This function finds a module and return the fully-qualified file name.
Folders from `in_folders`, if specified, are search first, followed by
folders in the global `import_path` list. If any folder name in EXPLICIT INPUTS
IMPLICIT INPUTS
RETURN VALUES If if isinstance(in_folders, basestring):
in_folders= [in_folders]
elif not isinstance(in_folders, list):
raise TypeError("If specified, `in_folders` must be either a string
or a "
"list of strings. (A string is wrapped to produce a length-1
list).") if any([item.endswith('*') for item in in_folders ]) or \ ex= None
for folder in itertools.chain(
expander(in_folders), expander(import_path)):
try:
return imp.find_module(module_name, in_folders + import_path)
except Exception as ex:
pass
if ex:
raise ex else: On Tue, Aug 21, 2018 at 10:32 AM Brett Cannon <report@bugs.python.org>
|
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: