import bblfsh fails on 2.8.0

[vmarkovtsev]

I installed the most recent version 2.8 and I get:

In [1]: import bblfsh
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-1-e99ccc08549f> in <module>()
----> 1 import bblfsh

/usr/local/lib/python3.5/dist-packages/bblfsh/__init__.py in <module>()
----> 1 from bblfsh.client import BblfshClient
      2 from bblfsh.pyuast import filter, iterator
      3 from bblfsh.aliases import *
      4 
      5 class TreeOrder:

/usr/local/lib/python3.5/dist-packages/bblfsh/client.py in <module>()
      4 import grpc
      5 
----> 6 from bblfsh.aliases import ParseRequest, NativeParseRequest, VersionRequest, ProtocolServiceStub
      7 from bblfsh.sdkversion import VERSION
      8 

/usr/local/lib/python3.5/dist-packages/bblfsh/aliases.py in <module>()
     24 
     25 ParseResponse = importlib.import_module(
---> 26     "bblfsh.gopkg.in.bblfsh.sdk.%s.protocol.generated_pb2" % VERSION).ParseResponse
     27 
     28 NativeParseResponse = importlib.import_module(

/usr/lib/python3.5/importlib/__init__.py in import_module(name, package)
    124                 break
    125             level += 1
--> 126     return _bootstrap._gcd_import(name[level:], package, level)
    127 
    128 

/usr/local/lib/python3.5/dist-packages/bblfsh/gopkg/in/bblfsh/sdk/v1/protocol/generated_pb2.py in <module>()
    101       message_type=None, enum_type=None, containing_type=None,
    102       is_extension=False, extension_scope=None,
--> 103       options=None, file=DESCRIPTOR),
    104     _descriptor.FieldDescriptor(
    105       name='language', full_name='gopkg.in.bblfsh.sdk.v1.protocol.NativeParseRequest.language', index=1,

TypeError: __new__() got an unexpected keyword argument 'file'

[vmarkovtsev]

Installing 2.6.1 works. v2.7.0 fails with an other error:

In [1]: import bblfsh
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-1-e99ccc08549f> in <module>()
----> 1 import bblfsh

/usr/local/lib/python3.5/dist-packages/bblfsh/__init__.py in <module>()
----> 1 from bblfsh.client import BblfshClient
      2 from bblfsh.pyuast import filter
      3 from bblfsh.aliases import *
      4 
      5 # "in" is a reserved keyword in Python thus can't be used as package name, so

/usr/local/lib/python3.5/dist-packages/bblfsh/client.py in <module>()
      4 import grpc
      5 
----> 6 from bblfsh.aliases import ParseRequest, NativeParseRequest, VersionRequest, ProtocolServiceStub
      7 from bblfsh.sdkversion import VERSION
      8 

/usr/local/lib/python3.5/dist-packages/bblfsh/aliases.py in <module>()
     18 
     19 DESCRIPTOR = importlib.import_module(
---> 20         "bblfsh.gopkg.in.bblfsh.sdk.%s.uast.generated_pb2" % VERSION).DESCRIPTOR
     21 
     22 Node = importlib.import_module(

/usr/lib/python3.5/importlib/__init__.py in import_module(name, package)
    124                 break
    125             level += 1
--> 126     return _bootstrap._gcd_import(name[level:], package, level)
    127 
    128 

ImportError: No module named 'bblfsh.gopkg'

[abeaumont]

I cannot reproduce it:

> sudo pip install --upgrade bblfsh
Collecting bblfsh
  Downloading bblfsh-2.8.0.tar.gz
Collecting grpcio (from bblfsh)
[...]
Successfully installed bblfsh-2.8.0 certifi-2017.11.5 docker-2.7.0 grpcio-1.8.2 grpcio-tools-1.8.2 protobuf-3.5.1 setuptools-38.2.5 websocket-client-0.46.0
> python
Python 3.6.3 (default, Oct 24 2017, 14:48:20) 
[GCC 7.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import bblfsh
>>> 

Looking at errors, it looks like you may be getting incorrect protobuf files... could you maybe try a fresh install to discard conflicts with previous installation issues?

[vmarkovtsev]

I did pip uninstall before installing a new version. What information do you need on my side to debug?

[abeaumont]

Hard to say, since it seems a protobuf related issue, but the generated python code is generated by your own protobuf compiler, which apparently is using some features that are not supported by that same protobuf...
The autogenerated code in my side seems to be the same, so may be some issue with python and/or protobuf versions?

[vmarkovtsev]

vadim@vadim-XPS-13-9360:~/Projects/client-go$ sudo rm -rf /usr/local/lib/python3.5/dist-packages/bblfsh
vadim@vadim-XPS-13-9360:~/Projects/client-go$ sudo pip3 uninstall bblfsh
The directory '/home/vadim/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Uninstalling bblfsh-2.8.0:
  /usr/local/lib/python3.5/dist-packages/bblfsh-2.8.0-py3.5.egg-info
Proceed (y/n)? y
  Successfully uninstalled bblfsh-2.8.0
The directory '/home/vadim/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
vadim@vadim-XPS-13-9360:~/Projects/client-go$ sudo pip3 install bblfsh
The directory '/home/vadim/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/vadim/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting bblfsh
  Downloading bblfsh-2.8.0.tar.gz
Requirement already satisfied: grpcio in /usr/local/lib/python3.5/dist-packages (from bblfsh)
Requirement already satisfied: grpcio-tools in /usr/local/lib/python3.5/dist-packages (from bblfsh)
Requirement already satisfied: docker in /usr/local/lib/python3.5/dist-packages (from bblfsh)
Requirement already satisfied: six>=1.5.2 in /usr/lib/python3/dist-packages (from grpcio->bblfsh)
Requirement already satisfied: protobuf>=3.2.0 in /usr/local/lib/python3.5/dist-packages (from grpcio->bblfsh)
Requirement already satisfied: websocket-client>=0.32.0 in /usr/local/lib/python3.5/dist-packages (from docker->bblfsh)
Requirement already satisfied: docker-pycreds>=0.2.1 in /usr/local/lib/python3.5/dist-packages (from docker->bblfsh)
Requirement already satisfied: requests!=2.11.0,!=2.12.2,>=2.5.2 in /usr/local/lib/python3.5/dist-packages (from docker->bblfsh)
Requirement already satisfied: setuptools in /usr/local/lib/python3.5/dist-packages (from protobuf>=3.2.0->grpcio->bblfsh)
Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python3.5/dist-packages (from requests!=2.11.0,!=2.12.2,>=2.5.2->docker->bblfsh)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python3.5/dist-packages (from requests!=2.11.0,!=2.12.2,>=2.5.2->docker->bblfsh)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python3.5/dist-packages (from requests!=2.11.0,!=2.12.2,>=2.5.2->docker->bblfsh)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.5/dist-packages (from requests!=2.11.0,!=2.12.2,>=2.5.2->docker->bblfsh)
Installing collected packages: bblfsh
  Running setup.py install for bblfsh ... done
Successfully installed bblfsh-2.8.0
vadim@vadim-XPS-13-9360:~/Projects/client-go$ python3 -m bblfsh 
Traceback (most recent call last):
  File "/usr/lib/python3.5/runpy.py", line 174, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "/usr/lib/python3.5/runpy.py", line 133, in _get_module_details
    return _get_module_details(pkg_main_name, error)
  File "/usr/lib/python3.5/runpy.py", line 109, in _get_module_details
    __import__(pkg_name)
  File "/usr/local/lib/python3.5/dist-packages/bblfsh/__init__.py", line 1, in <module>
    from bblfsh.client import BblfshClient
  File "/usr/local/lib/python3.5/dist-packages/bblfsh/client.py", line 6, in <module>
    from bblfsh.aliases import ParseRequest, NativeParseRequest, VersionRequest, ProtocolServiceStub
  File "/usr/local/lib/python3.5/dist-packages/bblfsh/aliases.py", line 26, in <module>
    "bblfsh.gopkg.in.bblfsh.sdk.%s.protocol.generated_pb2" % VERSION).ParseResponse
  File "/usr/lib/python3.5/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/usr/local/lib/python3.5/dist-packages/bblfsh/gopkg/in/bblfsh/sdk/v1/protocol/generated_pb2.py", line 103, in <module>
    options=None, file=DESCRIPTOR),
TypeError: __new__() got an unexpected keyword argument 'file'

Need to investigate this.

[vmarkovtsev]

Indeed, protobuf 3.4.0 or later is required.