"modernize" the modulefinder module

[ericsnowcurrently]

BPO	20020
Nosy	@theller, @ericsnowcurrently, @berkerpeksag, @matrixise
PRs	bpo-35936: Updates to modulefinder #11787 bpo-35936: Updates to modulefinder #11787

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2013-12-19.04:05:25.376>
labels = ['type-feature', 'library']
title = '"modernize" the modulefinder module'
updated_at = <Date 2019-02-08.03:24:32.555>
user = 'https://github.com/ericsnowcurrently'

bugs.python.org fields:

activity = <Date 2019-02-08.03:24:32.555>
actor = 'brandtbucher'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2013-12-19.04:05:25.376>
creator = 'eric.snow'
dependencies = []
files = []
hgrepos = []
issue_num = 20020
keywords = ['patch', 'patch']
message_count = 3.0
messages = ['206576', '206608', '229310']
nosy_count = 5.0
nosy_names = ['theller', 'Arfrever', 'eric.snow', 'berker.peksag', 'matrixise']
pr_nums = ['11787', '11787']
priority = 'low'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue20020'
versions = ['Python 3.5']

[ericsnowcurrently]

The modulefinder module (Lib/modulefinder.py) provides a ModuleFinder class (plus 2 helpers) you can use to see what modules a script imports (directly or indirectly). The module's implementation is centered on the old imp.find_/load_module() API (which has been deprecated). The implementation should be refactored to reflect/make use of the newer import-related APIs.

The documented API for ModuleFinder is very small. However, it has quite a few methods that are "public" in the sense that their names do not start with an underscore. This will make any kind of refactoring trickier.

[theller]

I have written a new modulefinder based on importlib. It is not a refactoring of the old one, so it is no plug-in replacement. Instead it has some new features:

• Better logging output
• collects dependencies (self._depgraph maps module names to callers)
• when run as script, the command line syntax are easier to understand (although parsing is still done by getopt; argparse would be event better)
• The Module proxies that modulefinder collects give better access to the module's attributes, including the byte code

It is not yet tested in the wild but I will use it for the Python3 py2exe implementation.

If anyone wants to take a look the current version is here:

http://code.google.com/p/ctypes-stuff/source/browse/trunk/mf/py2exe/mf3.py

[matrixise]

The code of Thomas has not the same API than the "modulefinder" module.

If you want a patch, we need to rework the code of Thomas and provide an equivalent API with the existing one.

What do you suggest?

[iritkatriel]

Fixed by #11787.