Pysus not finding _readdbc

[LucasMMota]

I'm getting this error when trying to use function read_dbc():

Traceback (most recent call last):
  File "../download.py", line 3, in <module>
    from pysus.utilities.readdbc import read_dbc
  File "/Users/lucas/Monografia/datasus-data-extractor/src-virtualenv-datasus/lib/python3.7/site-packages/pysus/utilities/readdbc.py", line 12, in <module>
    from pysus.utilities._readdbc import ffi, lib
ModuleNotFoundError: No module named 'pysus.utilities._readdbc'

I've already installed cffi.
I also have the files _readdbc.c, _readdbc.cpython-37m-darwin.so and _readdbc.o on the pysus/utilities/.

What could it be?

[fccoelho]

hi @LucasMMota ,
how did you install PySUS?
When it cannot load it can be a failure to compile the _readdbc extension. Do you have libffi installed?
Please check the installation of cffi here, then completely uninsttall and reinstall PySUS, making sure there is no compilation error.

[higoramp]

Same error here. Already installed libffi-dev package.

[fccoelho]

Post the entire error here. It builds normally on Linux Em sáb, 23 de fev de 2019 16:37, higoramp <notifications@github.com> escreveu:

…

Same error here. Already installed libffi-dev package. — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#10 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAIjW7Vq9NBynMEstg4UXrae6paxbVNLks5vQZiKgaJpZM4bEQY8> .

[ArsCarlos]

same here, try in ubunto 19.04 and docker with pythonpython:3.5.7-stretch

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-3-ab3d6ae29c04> in <module>
----> 1 from pysus.utilities import readdbc

/usr/local/lib/python3.5/site-packages/pysus/utilities/readdbc.py in <module>
     10 from dbfread import DBF
     11 
---> 12 from pysus.utilities._readdbc import ffi, lib
     13 
     14 

ImportError: No module named 'pysus.utilities._readdbc'
``

[edinhodiluviano]

I just got the same error and, maybe, this is a hint to the problem.
I'm using Ubuntu 18.04 LTS. When I install pysus in a virtualenv with python3.6, it works correctly. However, in a env with python3.5 or pyhon3.7, it breaks (and it doesn't show compilation errors).
On both (3.5 and 3.7) I'm able to fix it by going to pysus/utilities, downloading the c-src folder (with its contents, of course) and running the _build_readdbc.py.

The problems seems to come from the _readdbc.cpython-36m-x86_64-linux-gnu.so file. When I run the above procedure, it creates one just like that but for the python version the env is using (for instance: _readdbc.cpython-37m-x86_64-linux-gnu.so)
However, I'm not sure why it doesn't creates the correct file since the beginning.

[fccoelho]

The error simply means that the C extension is not building under certain circumstances.
If it is failing on Linux, please check if you have have the latest version of CFFI pip3 install -U cffi installed. It should build normally on Python 3.5 and 3.7

[fccoelho]

A good way to test is to clone this repo and run

python3 setup.py build
python3 setup.py install

[edinhodiluviano]

"The error simply means that the C extension is not building under certain circumstances.
If it is failing on Linux, please check if you have have the latest version of CFFI pip3 install -U cffi installed. It should build normally on Python 3.5 and 3.7"

The OS is fine, all dependencies are met. I'm using the same OS for all python versions (just the environment changes)
Also, if it wasn't, then it would not work with python3.6, but it does.
When using pip, it always gives the binary for python3.6. Even the environment have a different python version.

python3 setup.py build
python3 setup.py install

These worked fine but missed the _readdbc.cpython-3Xm-x86_64-linux-gnu.so binary in all envs. When I run _build_readdbc.py manually, it builds with no problem. (the same result for all python versions)
Also, besides my machine, I tested this last approach on brand new Ubuntu 18.04 LTS on AWS just with dependencies installed. Same results (but just with python3.6 and 3.7).

Let me know if there is any log or output that I can provide to help.

[fccoelho]

build_readdbc.py should be called from setup.py according to CFFI documentation

I think this failure to build is due to some some issue with the way setuptools builds cffi extensions. I looked around for easy workarounds but could not find.

I think for now the workaround will have to be this:
After cloning the source code:

$ cd pysus/utilities
$ python3 _build_readdbc.py
$ cd ../..
$ python3 setup.py install

I'll keep looking for a solution if I find it I'll post it here an update the package on PyPI.
Meanwhile, I have updated the package because it was missing a required package.

[marceloandriolli]

@fccoelho I did exactly what you suggested before, but I still got the same error that @ArsCarlos reported.

[fccoelho]

This maybe due to the fact that virtualenvs are created by default with an old version of pip which is not compatible with cffi. I tested here, and the venv is created with pip 9.0 and the current version is 19.0.

after you activate your virtualenv, issue this command:

pip install -U pip

before installing pysus.
I hope it helps.

[kauly]

same error here in 2024

[vinmaga]

same error here