Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

unsaved editor files cause cryptic crash #238

Closed
aspiers opened this issue Mar 19, 2019 · 11 comments
Closed

unsaved editor files cause cryptic crash #238

aspiers opened this issue Mar 19, 2019 · 11 comments

Comments

@aspiers
Copy link

aspiers commented Mar 19, 2019

Issue description

Steps to reproduce the problem

Edit an existing testcase file in emacs without saving it. emacs creates a lock symlink which is presumably the cause of the issue:

lrwxrwxrwx  1 adam users   25 Mar 19 22:10  .#test_utils.py -> '@pacific.11726:1552522564'

Expected behavior

stestr runs as normal.

Actual behavior

stestr crashes with a hopelessly cryptic error.

=========================
Failures during discovery
=========================
--- stdout ---
2019-03-19 22:18:23.437 20379 INFO oslo_service.periodic_task [req-f8f7c7a0-4c14-4d39-915f-045737444a0d - - - - -] Skipping periodic task _discover_hosts_in_cells because its interval is negative
2019-03-19 22:18:23.990 20379 CRITICAL nova [req-f8f7c7a0-4c14-4d39-915f-045737444a0d - - - - -] Unhandled error: TypeError: 'NoneType' object is not iterable
2019-03-19 22:18:23.990 20379 ERROR nova Traceback (most recent call last):
2019-03-19 22:18:23.990 20379 ERROR nova   File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main
2019-03-19 22:18:23.990 20379 ERROR nova     "__main__", mod_spec)
2019-03-19 22:18:23.990 20379 ERROR nova   File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code
2019-03-19 22:18:23.990 20379 ERROR nova     exec(code, run_globals)
2019-03-19 22:18:23.990 20379 ERROR nova   File "/home/adam/SUSE/cloud/OpenStack/git/nova/.tox/py36/lib/python3.6/site-packages/subunit/run.py", line 149, in <module>
2019-03-19 22:18:23.990 20379 ERROR nova     main()
2019-03-19 22:18:23.990 20379 ERROR nova   File "/home/adam/SUSE/cloud/OpenStack/git/nova/.tox/py36/lib/python3.6/site-packages/subunit/run.py", line 145, in main
2019-03-19 22:18:23.990 20379 ERROR nova     stdout=stdout, exit=False)
2019-03-19 22:18:23.990 20379 ERROR nova   File "/home/adam/SUSE/cloud/OpenStack/git/nova/.tox/py36/lib/python3.6/site-packages/testtools/run.py", line 171, in __init__
2019-03-19 22:18:23.990 20379 ERROR nova     self.parseArgs(argv)
2019-03-19 22:18:23.990 20379 ERROR nova   File "/home/adam/SUSE/cloud/OpenStack/git/nova/.tox/py36/lib/python3.6/site-packages/unittest2/main.py", line 113, in parseArgs
2019-03-19 22:18:23.990 20379 ERROR nova     self._do_discovery(argv[2:])
2019-03-19 22:18:23.990 20379 ERROR nova   File "/home/adam/SUSE/cloud/OpenStack/git/nova/.tox/py36/lib/python3.6/site-packages/testtools/run.py", line 211, in _do_discovery
2019-03-19 22:18:23.990 20379 ERROR nova     super(TestProgram, self)._do_discovery(argv, Loader=Loader)
2019-03-19 22:18:23.990 20379 ERROR nova   File "/home/adam/SUSE/cloud/OpenStack/git/nova/.tox/py36/lib/python3.6/site-packages/unittest2/main.py", line 223, in _do_discovery
2019-03-19 22:18:23.990 20379 ERROR nova     self.test = loader.discover(self.start, self.pattern, self.top)
2019-03-19 22:18:23.990 20379 ERROR nova   File "/home/adam/SUSE/cloud/OpenStack/git/nova/.tox/py36/lib/python3.6/site-packages/unittest2/loader.py", line 367, in discover
2019-03-19 22:18:23.990 20379 ERROR nova     tests = list(self._find_tests(start_dir, pattern))
2019-03-19 22:18:23.990 20379 ERROR nova   File "/home/adam/SUSE/cloud/OpenStack/git/nova/.tox/py36/lib/python3.6/site-packages/unittest2/loader.py", line 433, in _find_tests
2019-03-19 22:18:23.990 20379 ERROR nova     for test in path_tests:
2019-03-19 22:18:23.990 20379 ERROR nova   File "/home/adam/SUSE/cloud/OpenStack/git/nova/.tox/py36/lib/python3.6/site-packages/unittest2/loader.py", line 433, in _find_tests
2019-03-19 22:18:23.990 20379 ERROR nova     for test in path_tests:
2019-03-19 22:18:23.990 20379 ERROR nova   File "/home/adam/SUSE/cloud/OpenStack/git/nova/.tox/py36/lib/python3.6/site-packages/unittest2/loader.py", line 424, in _find_tests
2019-03-19 22:18:23.990 20379 ERROR nova     full_path, pattern, namespace)
2019-03-19 22:18:23.990 20379 ERROR nova TypeError: 'NoneType' object is not iterable
2019-03-19 22:18:23.990 20379 ERROR nova 

================================================================================
The above traceback was encountered during test discovery which imports all the found test modules in the specified test_path.

System information

stestr version (stestr --version): 2.2.0

Python release (python --version): 3.6.5

pip packages (pip freeze):

alembic==1.0.7
amqp==2.4.1
appdirs==1.4.3
asn1crypto==0.24.0
atomicwrites==1.3.0
attrs==18.2.0
automaton==1.15.0
Babel==2.6.0
bandit==1.5.1
bcrypt==3.1.6
cachetools==3.1.0
castellan==1.2.0
certifi==2018.11.29
cffi==1.12.1
chardet==3.0.4
cliff==2.14.0
cmd2==0.8.9
colorama==0.4.1
coverage==4.5.2
cryptography==2.5
cursive==0.2.2
ddt==1.2.0
debtcollector==1.20.0
decorator==4.3.2
dnspython==1.15.0
dogpile.cache==0.7.1
eventlet==0.24.1
extras==1.0.0
fancycompleter==0.9.dev2+g69f3876
fasteners==0.14.1
fixtures==3.0.0
flake8==2.5.5
future==0.17.1
futurist==1.8.0
gabbi==1.44.0
-e git://github.com/johnnadratowski/git-reviewers.git@d3678458708ce4a1b119709b788cfb8760ad7872#egg=git_reviewers
gitdb2==2.0.5
GitPython==2.1.11
greenlet==0.4.15
grpcio==1.15.0
hacking==0.12.0
idna==2.8
iso8601==0.1.12
Jinja2==2.10
jmespath==0.9.3
jsonpatch==1.23
jsonpath-rw==1.4.0
jsonpath-rw-ext==1.2.0
jsonpointer==2.0
jsonschema==2.6.0
keystoneauth1==3.11.2
keystonemiddleware==5.3.0
kombu==4.3.0
linecache2==1.0.0
lxml==4.3.1
Mako==1.0.7
MarkupSafe==1.1.0
mccabe==0.2.1
microversion-parse==0.2.1
mock==2.0.0
monotonic==1.5
more-itertools==6.0.0
mox3==0.26.0
msgpack==0.6.1
munch==2.3.2
netaddr==0.7.19
netifaces==0.10.9
networkx==2.2
-e git+https://github.com/openstack/nova.git@ea3de5ca023abf2a89d3a627b866af516318995c#egg=nova
numpy==1.16.1
openstacksdk==0.24.0
os-brick==2.7.0
os-client-config==1.31.2
os-resource-classes==0.2.0
os-service-types==1.5.0
os-traits==0.11.0
os-vif==1.14.0
os-win==4.2.0
os-xenapi==0.3.4
osc-lib==1.12.0
oslo.cache==1.33.0
oslo.concurrency==3.29.0
oslo.config==6.8.0
oslo.context==2.22.0
oslo.db==4.44.0
oslo.i18n==3.23.0
oslo.log==3.42.2
oslo.messaging==9.4.0
oslo.middleware==3.37.0
oslo.policy==2.1.0
oslo.privsep==1.32.0
oslo.reports==1.29.1
oslo.rootwrap==5.15.1
oslo.serialization==2.28.1
oslo.service==1.37.0
oslo.upgradecheck==0.2.0
oslo.utils==3.40.2
oslo.versionedobjects==1.35.0
oslo.vmware==2.32.1
oslotest==3.7.0
osprofiler==2.6.0
ovs==2.10.0
ovsdbapp==0.15.0
paramiko==2.4.2
Paste==3.0.6
PasteDeploy==2.0.1
pbr==5.1.2
pdbpp==0.9.12
pep8==1.5.7
pluggy==0.8.1
ply==3.11
prettytable==0.7.2
psutil==5.5.1
psycopg2==2.7.7
psycopg2-binary==2.7.7
py==1.7.0
pyasn1==0.4.5
pyasn1-modules==0.2.4
pycadf==2.9.0
pycparser==2.19
pydot==1.4.1
pyflakes==0.8.1
Pygments==2.3.1
pyinotify==0.9.6
PyJWT==1.7.1
PyMySQL==0.9.3
PyNaCl==1.3.0
pyOpenSSL==19.0.0
pyparsing==2.3.1
pyperclip==1.7.0
pypowervm==1.1.20
pyroute2==0.5.3
pytest==4.3.0
python-barbicanclient==4.8.1
python-cinderclient==4.1.0
python-dateutil==2.8.0
python-editor==1.0.4
python-glanceclient==2.15.0
python-ironicclient==2.6.0
python-keystoneclient==3.18.0
python-mimeparse==1.6.0
python-neutronclient==6.11.0
python-subunit==1.3.0
pytz==2018.9
PyYAML==3.13
repoze.lru==0.7
requests==2.21.0
requests-mock==1.5.2
requestsexceptions==1.4.0
retrying==1.3.3
rfc3986==1.2.0
Routes==2.4.1
simplejson==3.16.0
six==1.12.0
smmap2==2.0.5
sortedcontainers==2.1.0
SQLAlchemy==1.2.18
sqlalchemy-migrate==0.12.0
sqlparse==0.2.4
statsd==3.3.0
stestr==2.2.0
stevedore==1.30.0
suds-jurko==0.6
taskflow==3.4.0
Tempita==0.5.2
tenacity==5.0.3
testresources==2.0.1
testscenarios==0.5.0
testtools==2.3.0
tooz==1.64.1
traceback2==1.4.0
unittest2==1.1.0
urllib3==1.24.1
vine==1.2.0
voluptuous==0.11.5
warlock==1.3.0
wcwidth==0.1.7
WebOb==1.8.5
websockify==0.8.0
wmctrl==0.3
wrapt==1.11.1
wsgi-intercept==1.8.0
yappi==1.0
zVMCloudConnector==1.4.0
@masayukig
Copy link
Collaborator

I was able to reproduce this error in my stestr repo like following. It looks there's a bug in handling non-existent files.
If the symbolic link is correct, it works well.

$ ln -s test_foo.py stestr/tests/test_foo.py
$ stestr run
Traceback (most recent call last):
  File "/usr/lib64/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib64/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/masayuki/git/stestr/.tox/py37/lib/python3.7/site-packages/subunit/run.py", line 149, in <module>
    main()
  File "/home/masayuki/git/stestr/.tox/py37/lib/python3.7/site-packages/subunit/run.py", line 145, in main
    stdout=stdout, exit=False)
  File "/home/masayuki/git/stestr/.tox/py37/lib/python3.7/site-packages/testtools/run.py", line 171, in __init__
    self.parseArgs(argv)
  File "/home/masayuki/git/stestr/.tox/py37/lib/python3.7/site-packages/unittest2/main.py", line 113, in parseArgs
    self._do_discovery(argv[2:])
  File "/home/masayuki/git/stestr/.tox/py37/lib/python3.7/site-packages/testtools/run.py", line 211, in _do_discovery
    super(TestProgram, self)._do_discovery(argv, Loader=Loader)
  File "/home/masayuki/git/stestr/.tox/py37/lib/python3.7/site-packages/unittest2/main.py", line 223, in _do_discovery
    self.test = loader.discover(self.start, self.pattern, self.top)
  File "/home/masayuki/git/stestr/.tox/py37/lib/python3.7/site-packages/unittest2/loader.py", line 367, in discover
    tests = list(self._find_tests(start_dir, pattern))
  File "/home/masayuki/git/stestr/.tox/py37/lib/python3.7/site-packages/unittest2/loader.py", line 424, in _find_tests
    full_path, pattern, namespace)
TypeError: cannot unpack non-iterable NoneType object

=========================
Failures during discovery
=========================

================================================================================
The above traceback was encountered during test discovery which imports all the found test modules in the specified test_path.

@masayukig
Copy link
Collaborator

masayukig commented Mar 26, 2019

And, it looks this issue comes from unittest2.loader. So, I'm not sure it's good to fix it in stestr. Maybe, we should fix it in unittest2?

@mtreinish
Copy link
Owner

Hmm I'm curious if this bug also happens with unittest (not unittest2) too? Both are supposed to have a default discovery pattern of test*py. You can test this with python -m unittest discover -s $test_path -t ./ (so for nova unit tests it would be python -m unittest discover -s ./nova/tests/unit -t ./). You can test both stdlib unittest and unittest2 (just s/unittest/unittest2/g in the above command) in the failing environment.

If it is an issue with unittest2 we might have a hard time getting a fix merged that code base hasn't been touched in ages and was just a backport of newer stdlib unittest features.

@masayukig
Copy link
Collaborator

hmm, it doesn't happen with unittest but happens with unittests.

$ python -m unittest discover -s ./stestr/tests -t ./
................................................................................................................................ss..............
----------------------------------------------------------------------
Ran 144 tests in 142.536s

OK (skipped=2)
$ python -m unittest2 discover -s ./stestr/tests -t ./
Traceback (most recent call last):
  File "/usr/lib64/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib64/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/masayuki/git/stestr/.tox/py37/lib/python3.7/site-packages/unittest2/__main__.py", line 21, in <module>
    main_()
  File "/home/masayuki/git/stestr/.tox/py37/lib/python3.7/site-packages/unittest2/__main__.py", line 18, in main_
    main(module=None)
  File "/home/masayuki/git/stestr/.tox/py37/lib/python3.7/site-packages/unittest2/main.py", line 89, in __init__
    self.parseArgs(argv)
  File "/home/masayuki/git/stestr/.tox/py37/lib/python3.7/site-packages/unittest2/main.py", line 113, in parseArgs
    self._do_discovery(argv[2:])
  File "/home/masayuki/git/stestr/.tox/py37/lib/python3.7/site-packages/unittest2/main.py", line 223, in _do_discovery
    self.test = loader.discover(self.start, self.pattern, self.top)
  File "/home/masayuki/git/stestr/.tox/py37/lib/python3.7/site-packages/unittest2/loader.py", line 367, in discover
    tests = list(self._find_tests(start_dir, pattern))
  File "/home/masayuki/git/stestr/.tox/py37/lib/python3.7/site-packages/unittest2/loader.py", line 424, in _find_tests
    full_path, pattern, namespace)
TypeError: cannot unpack non-iterable NoneType object

@masayukig
Copy link
Collaborator

Can we try to import unittest first here?
https://github.com/testing-cabal/testtools/blob/master/testtools/run.py#L18

unittest = try_imports(['unittest2', 'unittest'])

@mtreinish
Copy link
Owner

Yeah, we'll have to raise this in testtools as a bug.

Honestly I really think testtools should drop the unittest2 usage now. It's not really needed anymore since unittest2 doesn't backport anything of note to supported python versions anymore. It introduces more bugs at this point than it solves.

mtreinish added a commit to mtreinish/testtools that referenced this issue May 3, 2019
Currently testtools bases all of it's unittest extensions off of
unittest2 instead of the stdlib unittest. At one point this made sense
since unittest2 provided a stable base as unittest in stdlib added
features. But it's been ~5 years since there was a unittest2 release
(or a patch merged) and things have changed since then. The best
example of this is of the supported python versions listed in the
unittest2 project description/README only one is still supported by
upstream python, 2.7, which goes end of life at the end of this year.

More specific to testtools the use of unittest2 causes a whole slew of
issues because of differences in behavior with stdlib unittest. For
example here a couple issues encountered:

https://bugs.launchpad.net/testtools/+bug/1467558
https://bugs.launchpad.net/testtools/+bug/1417803
mtreinish/stestr#238
testing-cabal#272

which are caused, at least in part, by unittest2. There are likely other
bugs related to it that haven't been reported (or I just missed/forgot
about). At this point it's better to remove the unittest2 usage and just
rely on the upstream stdlib unittest which if nothing else is actively
maintained. It'll improve compatibility using the testtools runner with
stdlib unittest test suites and removes the class of bugs caused by the differences in unittest2.

Fixes testing-cabal#263
@mtreinish
Copy link
Owner

I pushed up testing-cabal/testtools#277 we'll see how it goes...

@masayukig
Copy link
Collaborator

Oh, thanks!
yeah..

@mtreinish
Copy link
Owner

@masayukig @aspiers can you test stestr 2.5.0 to see if this still exists? We switched the internal test runner to use stdlib unittest directly (instead of testtools.run) in: #256 which I think should address this.

@aspiers
Copy link
Author

aspiers commented Aug 16, 2019

Upgrading to 2.5.0 does indeed seem to have fixed it! Many thanks :-)

@mtreinish
Copy link
Owner

Ok great, I thought it would. Abandoning testtools/unittest2 in the test execution path fixed a whole class of problems, unittest2 just doesn't work well with modern python 3. I'll go ahead and close this issue now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants