Error in doctest on namespace package with Python 3.7

[jaraco]

There seems to be an emergent issue with pytest when running --doctest-modules on Python 3.7.

Here's a run of jaraco.functools' tests:

$ tox
python develop-inst-noop: /Users/jaraco/Dropbox/code/main/jaraco.functools
python installed: attrs==17.4.0,backports.unittest-mock==1.3,collective.checkdocs==0.2,docutils==0.14,flake8==3.5.0,jaraco.classes==1.4.3,-e git+gh://jaraco/jaraco.functools@aaecb0c47110c38b697f6b14f0dd1d9ecc4d28be#egg=jaraco.functools,mccabe==0.6.1,more-itertools==4.1.0,pluggy==0.6.0,py==1.5.2,pycodestyle==2.3.1,pyflakes==1.6.0,pytest==3.4.1,pytest-flake8==0.9.1,pytest-sugar==0.9.1,six==1.11.0,termcolor==1.1.0
python runtests: PYTHONHASHSEED='479286674'
python runtests: commands[0] | py.test
Test session starts (platform: darwin, Python 3.7.0, pytest 3.4.1, pytest-sugar 0.9.1)
rootdir: /Users/jaraco/Dropbox/code/main/jaraco.functools, inifile: pytest.ini
plugins: sugar-0.9.1, flake8-0.9.1, backports.unittest-mock-1.3

―――――――――――――――――――――――――――――――――――― ERROR collecting jaraco/__init__.py ――――――――――――――――――――――――――――――――――――
/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/doctest.py:893: in find
    file = inspect.getsourcefile(obj)
/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/inspect.py:687: in getsourcefile
    if any(filename.endswith(s) for s in all_bytecode_suffixes):
/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/inspect.py:687: in <genexpr>
    if any(filename.endswith(s) for s in all_bytecode_suffixes):
E   AttributeError: 'NoneType' object has no attribute 'endswith'

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Results (0.36s):
ERROR: InvocationError: '/Users/jaraco/Dropbox/code/main/jaraco.functools/.tox/python/bin/py.test'
__________________________________________________ summary __________________________________________________
ERROR:   python: commands failed

At the point where inspect.getsourcefile(obj) is called, obj is <module 'jaraco' (namespace)>.

Perhaps all that's needed to be done here is have pytest ignore collection on namespace packages.

[jaraco]

Interestingly, I don't encounter the issue when running tests on backports.unittest_mock, which suggests the issue is likely related either to the pkg_resources-managed namespace or fact that jaraco.classes (another package in the namespace) is installed in the environment.

[jaraco]

Interesting - the tests are passing in Travis, so whatever is going on seems to be isolated to my environment.

[jaraco]

Also interesting - on my mac where the test is failing, the test also fails when testing jaraco.collections (which has a dependency on other jaraco.* packages) but not testing jaraco.classes (which does not).

[jaraco]

I just noticed the Travis build, although it's theoretically running a nightly build of Python, has Python 3.7.0a4+, while on my mac, I'm running 3.7.0b2.

[jaraco]

Uninstalling jaraco.classes from the environment before running the test bypasses the issue, so it is definitely related to having another package of the same namespace as the package under test present. Also, the issue seems only to affect pkg_resources-managed namespaces, as adding another 'backports' package to 'backports.unittest_mock' doesn't trigger the issue.

[jaraco]

I was able to replicate the issue without pytest, so it seems to be an upstream issue.

$ mkdir foo
$ python
Python 3.7.0b2 (tags/v3.7.0b2:b0ef5c979b, Feb 27 2018, 20:38:21)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import doctest
>>> import foo
>>> doctest.DocTestFinder().find(foo)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/doctest.py", line 893, in find
    file = inspect.getsourcefile(obj)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/inspect.py", line 687, in getsourcefile
    if any(filename.endswith(s) for s in all_bytecode_suffixes):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/inspect.py", line 687, in <genexpr>
    if any(filename.endswith(s) for s in all_bytecode_suffixes):
AttributeError: 'NoneType' object has no attribute 'endswith'

[nicoddemus]

Thanks @jaraco for the detailed investigation. Should we report this upstream?

[jaraco]

Yes, I have, and meant to link back here.

It's bpo-32991.

[nicoddemus]

Great, thanks!

[nicoddemus]

I see this has been fixed in 3.8 and backported into 3.7, so shall we close this issue?

[jaraco]

Yes.