Bblfsh fails to extract UAST from file

[zurk]

I have strange error trying to get UAST from this file: oo.py.zip

I run this code

from bblfsh.client import BblfshClient
bc = BblfshClient("0.0.0.0:9432")
res = bc.parse("./oo.py", language='Python')
print(res)

Output:

status: FATAL
errors: "expected object of type map[string]interface{}, got: \"NoneLiteral\""

but py file seems to be correct because I can run python3 ./oo.py without any problem

Also, bblfsh server is running in docker and output only

time="2017-08-08T20:34:55Z" level=info msg="parsing oo.py (34525 bytes)"

[zurk]

I get error output from bblfsh somehow:

time="2017-08-09T08:59:28Z" level=info msg="parsing oo.py (34525 bytes)"
time="2017-08-09T08:59:28Z" level=error msg="driver bblfsh/python-driver:latest (01BQ33CS23S3TSW2JSVD5ABPMX) stderr: ERROR:root:Filepath: , Errors: ['Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/python_driver/requestprocessor.py", line 139, in process_request
    '\n------ Python3 errors:\n%s' % codeinfo['py3_ast_errors']
Exception: Errors produced trying to get an AST for both Python versions
------ Python2 errors:
[b'Traceback (most recent call last):\n  File "<string>", line 1, in <module>\n  File "/usr/lib/python2.7/site-packages/pydetector/astexport.py", line 23, in export_dict\n    return visitor.parse()\n  File "/usr/lib/python2.7/site-packages/pydetector/astexport.py", line 536, in parse\n    tree = ast.parse(self.codestr, mode=\'exec\')\n  File "/usr/lib/python2.7/ast.py", line 37, in parse\n    return compile(source, filename, mode, PyCF_ONLY_AST)\n  File "<unknown>", line 231\n    print(ansi.CYAN, end=\'\')\n                        ^\nSyntaxError: invalid syntax\n']
------ Python3 errors:
['Traceback (most recent call last):\n  File "/usr/lib/python3.6/site-packages/pydetector/ast_checks.py", line 52, in check_ast\n    current_ast = export_dict(code)\n  File "/usr/lib/python3.6/site-packages/pydetector/astexport.py", line 23, in export_dict\n    return visitor.parse()\n  File "/usr/lib/python3.6/site-packages/pydetector/astexport.py", line 537, in parse\n    res = self.visit(tree, root=True)\n  File "/usr/lib/python3.6/site-packages/pydetector/astexport.py", line 555, in visit\n    visit_result = meth(node)\n  File "/usr/lib/python3.6/site-packages/pydetector/astexport.py", line 576, in visit_other\n    nodedict[field] = meth(getattr(node, field))\n  File "/usr/lib/python3.6/site-packages/pydetector/astexport.py", line 604, in visit_other_field\n    return [self.visit(x) for x in node]\n  File "/usr/lib/python3.6/site-packages/pydetector/astexport.py", line 604, in <listcomp>\n    return [self.visit(x) for x in node]\n  File "/usr/lib/python3.6/site-packages/pydetector/astexport.py", line 555, in visit\n    visit_result = meth(node)\n  File "/usr/lib/python3.6/site-packages/pydetector/astexport.py", line 576, in visit_other\n    nodedict[field] = meth(getattr(node, field))\n  File "/usr/lib/python3.6/site-packages/pydetector/astexport.py", line 604, in visit_other_field\n    return [self.visit(x) for x in node]\n  File "/usr/lib/python3.6/site-packages/pydetector/astexport.py", line 604, in <listcomp>\n    return [self.visit(x) for x in node]\n  File "/usr/lib/python3.6/site-packages/pydetector/astexport.py", line 555, in visit\n    visit_result = meth(node)\n  File "/usr/lib/python3.6/site-packages/pydetector/astexport.py", line 576, in visit_other\n    nodedict[field] = meth(getattr(node, field))\n  File "/usr/lib/python3.6/site-packages/pydetector/astexport.py", line 602, in visit_other_field\n    return self.visit(node)\n  File "/usr/lib/python3.6/site-packages/pydetector/astexport.py", line 555, in visit\n    visit_result = meth(node)\n  File "/usr/lib/python3.6/site-packages/pydetector/astexport.py", line 576, in visit_other\n    nodedict[field] = meth(getattr(node, field))\n  File "/usr/lib/python3.6/site-packages/pydetector/astexport.py", line 604, in visit_other_field\n    return [self.visit(x) for x in node]\n  File "/usr/lib/python3.6/site-packages/pydetector/astexport.py", line 604, in <listcomp>\n    return [self.visit(x) for x in node]\n  File "/usr/lib/python3.6/site-packages/pydetector/astexport.py", line 559, in visit\n    self.pos_sync.apply_fixes(visit_result)\n  File "/usr/lib/python3.6/site-packages/pydetector/astexport.py", line 236, in apply_fixes\n    self._sync_node_col(nodedict)\n  File "/usr/lib/python3.6/site-packages/pydetector/astexport.py", line 155, in _sync_node_col\n    if self._fix_virtualnode_col(nodedict):\n  File "/usr/lib/python3.6/site-packages/pydetector/astexport.py", line 129, in _fix_virtualnode_col\n    node_type = nodedict[\'ast_type\']\nTypeError: string indices must be integers\n']
']"

[juanjux]

I'll check this after #57.

[juanjux]

Fixed by #60 when merged.

[juanjux]

Version 0.6.2 of the Python driver fixes this (should be published in Dockerhub as latest soon), thanks for the excellent report.