osx ModuleNotFoundError: No module named 'pyreadr.librdata'

[giko45]

Hi,

installed with: conda install -c conda-forge pyreadr

I get an import error on import.

Python 3.7.5 (default, Oct 25 2019, 10:52:18) 
[Clang 4.0.1 (tags/RELEASE_401/final)] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyreadr
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/xxx/pyreadr-master/pyreadr/__init__.py", line 1, in <module>
    from .pyreadr import read_r, list_objects, write_rds, write_rdata
  File "/Users/xxx/pyreadr-master/pyreadr/pyreadr.py", line 8, in <module>
    from ._pyreadr_parser import PyreadrParser, ListObjectsParser
  File "/Users/xxx/pyreadr-master/pyreadr/_pyreadr_parser.py", line 10, in <module>
    from .librdata import Parser
ModuleNotFoundError: No module named 'pyreadr.librdata'

osx 10.10

[ofajardo]

I think you have cloned the repo and python is trying to import from the cloned repo, where you have not compiled correctly the package.
It should import instead from the conda environment.
Maybe your working folder is the cloned repo? You have to move somewhere else to work with what you installed with conda.

[giko45]

Hi,

That is correct, when i had issues i tried compiling from source.

I now started off with a clean conda environnement.

conda create --name test4
conda install pandas libiconv
conda install -c conda-forge pyreadr

I still get an error but a different one: Symbol not found: _iconv

Python 3.7.5 (default, Oct 25 2019, 10:52:18) 
[Clang 4.0.1 (tags/RELEASE_401/final)] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyreadr
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/miniconda3/envs/test4/lib/python3.7/site-packages/pyreadr/__init__.py", line 1, in <module>
    from .pyreadr import read_r, list_objects, write_rds, write_rdata
  File "/opt/miniconda3/envs/test4/lib/python3.7/site-packages/pyreadr/pyreadr.py", line 8, in <module>
    from ._pyreadr_parser import PyreadrParser, ListObjectsParser
  File "/opt/miniconda3/envs/test4/lib/python3.7/site-packages/pyreadr/_pyreadr_parser.py", line 10, in <module>
    from .librdata import Parser
ImportError: dlopen(/opt/miniconda3/envs/test4/lib/python3.7/site-packages/pyreadr/librdata.cpython-37m-darwin.so, 2): Symbol not found: _iconv
  Referenced from: /opt/miniconda3/envs/test4/lib/python3.7/site-packages/pyreadr/librdata.cpython-37m-darwin.so
  Expected in: flat namespace
 in /opt/miniconda3/envs/test4/lib/python3.7/site-packages/pyreadr/librdata.cpython-37m-darwin.so

[ofajardo]

Somehow the libiconv version is not good. Try a new conda env without installing libiconv. What do you get now?

[giko45]

it gives the same error independent of if i use conda install pandas libiconv or conda install pandas. I read somwhere that there apperently are 2 versions of libiconv that are difficult to distinguish and i haven't found the correct one

[ofajardo]

Ok do

otool -L /opt/miniconda3/envs/test4/lib/python3.7/site-packages/pyreadr/librdata.cpython-37m-darwin.so

To check where it is loading libiconv from. My guess is that it is not loading it from conda. Try to force it to take it from conda by adding the folder to DYLD_LIBRARY_PATH

[giko45]

Hi,

got it working with:

conda create --name test_build2 pandas cython libiconv
conda activate test_build2
export LDFLAGS=-liconv
cd pyreadr-master
python setup.py install
cd # away 
python -c "import pyreadr"

i found
https://stackoverflow.com/questions/57734434/libiconv-or-iconv-undefined-symbol-on-mac-osx
also usefull.

Thanks for your help

[ofajardo]

Wow, amazing job!
Now I notice that in my other library pyreadstat I am linking iconv statically as you did, so I should include that in my setup.py for pyreadr as well.
Thanks!

[giko45]

:)

[ofajardo]

@giko45 The new version 0.2.2 has a static linking for iconv. If you happen to test it let me know if it works.

[giko45]

did you push it to conda-forge?

i still get 2.1
The following NEW packages will be INSTALLED:

pyreadr conda-forge/osx-64::pyreadr-0.2.1-py37hbbe82c9_0

[ofajardo]

Yeah I did, probably was on its way when you tried but now I can see that 0.2.2 is there for mac on conda. Could you please give it another try?

[giko45]

hmm, i still get 0.2.1

conda search pyreadr --channel conda-forge
Loading channels: done
# Name                       Version           Build  Channel             
pyreadr                        0.1.8  py36ha441bb4_0  conda-forge         
pyreadr                        0.1.8  py36ha441bb4_2  conda-forge         
pyreadr                        0.1.8  py37ha441bb4_0  conda-forge         
pyreadr                        0.1.8  py37ha441bb4_2  conda-forge         
pyreadr                        0.1.9  py36h2573ce8_0  conda-forge         
pyreadr                        0.1.9  py36ha441bb4_0  conda-forge         
pyreadr                        0.1.9  py37h2573ce8_0  conda-forge         
pyreadr                        0.1.9  py37ha441bb4_0  conda-forge         
pyreadr                        0.2.0  py36h2573ce8_0  conda-forge         
pyreadr                        0.2.0  py37h2573ce8_0  conda-forge         
pyreadr                        0.2.1  py36h2573ce8_0  conda-forge         
pyreadr                        0.2.1  py36hbbe82c9_0  conda-forge         
pyreadr                        0.2.1  py37h2573ce8_0  conda-forge         
pyreadr                        0.2.1  py37hbbe82c9_0  conda-forge         
pyreadr                        0.2.1  py38hbbe82c9_0  conda-forge         

[ofajardo]

Sorry, maybe it was still on its way, now I get 0.2.2

conda search pyreadr --channel conda-forge
Loading channels: done
# Name                       Version           Build  Channel
pyreadr                        0.1.8  py36h84994c4_0  conda-forge
pyreadr                        0.1.8  py36h84994c4_2  conda-forge
pyreadr                        0.1.8  py37h84994c4_0  conda-forge
pyreadr                        0.1.8  py37h84994c4_2  conda-forge
pyreadr                        0.1.9  py36h84994c4_0  conda-forge
pyreadr                        0.1.9  py36hed695b0_0  conda-forge
pyreadr                        0.1.9  py37h84994c4_0  conda-forge
pyreadr                        0.1.9  py37hed695b0_0  conda-forge
pyreadr                        0.2.0  py36hed695b0_0  conda-forge
pyreadr                        0.2.0  py37hed695b0_0  conda-forge
pyreadr                        0.2.1  py36hed695b0_0  conda-forge
pyreadr                        0.2.1  py37hed695b0_0  conda-forge
pyreadr                        0.2.1  py38hed695b0_0  conda-forge
pyreadr                        0.2.2  py36hed695b0_0  conda-forge
pyreadr                        0.2.2  py37hed695b0_0  conda-forge
pyreadr                        0.2.2  py38hed695b0_0  conda-forge

[giko45]

Yes it works directly from conda-forge now. Great work!

[ofajardo]

Great TEAM work! Thanks a lot for discovering the root cause of the issue and testing!

[nastaranmarzban]

hi, hope you're doing well, I have a file in format Rdata, I've searched and understood that I have to use pyreadr in order to read it in python... but after installing it when I import pyreadr I receive an error which says module 'pandas' has no attribute 'Float64Dtype'
Can anyone help me? I really don't know how to fix it
Thanks in advance

[ofajardo]

Please update pandas to the newest version

[nastaranmarzban]

Please update pandas to the newest version

got it, thanks

[ofajardo]

You need pandas 1.2.0 at least

[nastaranmarzban]

You need pandas 1.2.0 at least

hi, happy new year
thanks, I've fixed it by !pip install pandas==1.3.5