New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Invalid syntax exception not caught by pep257 runner #168

Closed
jayvdb opened this Issue Dec 16, 2015 · 2 comments

Comments

Projects
2 participants
@jayvdb
Copy link
Member

jayvdb commented Dec 16, 2015

When invoked directly on the command line to parse a module with invalid syntax, an exception is raised.
Not a big problem when only one module is being checked, but it causes a halt if many modules are being checked. It would be helpful for the exception to indicate which module failed, and skip it. It is possible to determine the failing module by using -v, however that requires re-running the job when an exception is encountered.

Each of the following are probably separate issues that can be solved by hardening the parser, but there will always be new syntax issues, and the pep257 runner should handle them more reasonably.

pep8 test file E90.py:

pep8$ pep257 testsuite/E90.py 
Traceback (most recent call last):
  File "/usr/bin/pep257", line 9, in <module>
    load_entry_point('pep257==0.7.1a0', 'console_scripts', 'pep257')()
  File "/.../pep257.py", line 1642, in main
    sys.exit(run_pep257())
  File "/.../pep257.py", line 1300, in run_pep257
    errors.extend(check((filename,), select=checked_codes))
  File "/.../pep257.py", line 1241, in check
    for error in PEP257Checker().check_source(source, filename):
  File "/.../pep257.py", line 1338, in check_source
    module = parse(StringIO(source), filename)
  File "/.../pep257.py", line 276, in __call__
    return self.parse_module()
  File "/.../pep257.py", line 428, in parse_module
    children = list(self.parse_definitions(Module, all=True))
  File "/.../pep257.py", line 370, in parse_definitions
    for definition in self.parse_definitions(class_):
  File "/.../pep257.py", line 370, in parse_definitions
    for definition in self.parse_definitions(class_):
  File "/.../pep257.py", line 378, in parse_definitions
    self.stream.move()
  File "/.../pep257.py", line 241, in move
    current = next(self._generator, None)
  File "/usr/lib64/python2.7/tokenize.py", line 357, in generate_tokens
    ("<tokenize>", lnum, pos, line))
  File "<tokenize>", line 9
    except:
    ^
IndentationError: unindent does not match any outer indentation level

Another one:

https://bitbucket.org/blais/snakefood/src/fa88cf9c38275362f13b208d1a7d88c4d4ab9313/test/data/simple/invalid.py

snakefood$ pep257 test/data/simple/invalid.py 
Traceback (most recent call last):
  File "/usr/bin/pep257", line 9, in <module>
    load_entry_point('pep257==0.7.1a0', 'console_scripts', 'pep257')()
  File "/.../pep257.py", line 1640, in main
    sys.exit(run_pep257())
  File "/.../pep257.py", line 1298, in run_pep257
    errors.extend(check((filename,), select=checked_codes))
  File "/.../pep257.py", line 1239, in check
    for error in PEP257Checker().check_source(source, filename):
  File "/.../pep257.py", line 1336, in check_source
    module = parse(StringIO(source), filename)
  File "/.../pep257.py", line 276, in __call__
    return self.parse_module()
  File "/.../pep257.py", line 428, in parse_module
    children = list(self.parse_definitions(Module, all=True))
  File "/.../pep257.py", line 367, in parse_definitions
    yield self.parse_definition(class_._nest(self.current.value))
  File "/.../pep257.py", line 466, in parse_definition
    assert self.current.kind != tk.INDENT
AttributeError: 'NoneType' object has no attribute 'kind'

Another one is running pep257 on Python 2 with code containing matmul ; e.g. cpython Lib/unittest/test/testmock/testmagicmethods.py

cpython$ pep257 Lib/unittest/test/testmock/testmagicmethods.py 
Checking file Lib/unittest/test/testmock/testmagicmethods.py.
Traceback (most recent call last):
  File "/usr/bin/pep257", line 9, in <module>
    load_entry_point('pep257==0.7.1a0', 'console_scripts', 'pep257')()
  File "/.../pep257.py", line 1642, in main
    sys.exit(run_pep257())
  File "/.../pep257.py", line 1300, in run_pep257
    errors.extend(check((filename,), select=checked_codes))
  File "/.../pep257.py", line 1241, in check
    for error in PEP257Checker().check_source(source, filename):
  File "/.../pep257.py", line 1338, in check_source
    module = parse(StringIO(source), filename)
  File "/.../pep257.py", line 276, in __call__
    return self.parse_module()
  File "/.../pep257.py", line 428, in parse_module
    children = list(self.parse_definitions(Module, all=True))
  File "/.../pep257.py", line 367, in parse_definitions
    yield self.parse_definition(class_._nest(self.current.value))
  File "/.../pep257.py", line 487, in parse_definition
    self.current.kind,
AttributeError: 'NoneType' object has no attribute 'kind'

@Nurdok Nurdok modified the milestone: pydocstyle 1.1.0 Apr 23, 2016

@Nurdok

This comment has been minimized.

Copy link
Member

Nurdok commented Sep 29, 2016

pydocstyle should continue parsing other files while emitting a warning.

@Nurdok

This comment has been minimized.

Copy link
Member

Nurdok commented Oct 19, 2017

This should be solved by now, so I'm closing this issue. Reopen if this isn't the case.

@Nurdok Nurdok closed this Oct 19, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment