Skip to content
This repository has been archived by the owner on Apr 23, 2024. It is now read-only.

Import somehow fails #33

Closed
arrrrrmin opened this issue Nov 2, 2019 · 4 comments
Closed

Import somehow fails #33

arrrrrmin opened this issue Nov 2, 2019 · 4 comments
Labels
bug Something isn't working

Comments

@arrrrrmin
Copy link

arrrrrmin commented Nov 2, 2019

When i'm tryping

import youtokentome as yttm
  • numpy
  • pandas
  • youtokentome
  • sentencepiece

are installed as requirements via

pip install -r requirements.txt

within a virtual environment called devenv on a Python3.6

There is some symbol not found

    (devenv) machine:tokenizer-learning USER$ python tokenize.py
    Traceback (most recent call last):
      File "tokenize.py", line 1, in <module>
        import youtokentome as yttm
      File "./tokenizer-learning/devenv/lib/python3.6/site-packages/youtokentome/__init__.py", line 1, in <module>
        from .youtokentome import BPE
      File "./tokenizer-learning/devenv/lib/python3.6/site-packages/youtokentome/youtokentome.py", line 4, in <module>
        import _youtokentome_cython
    ImportError: dlopen(./tokenizer-learning/devenv/lib/python3.6/site-packages/_youtokentome_cython.cpython-36m-darwin.so, 2): Symbol not found: __ZNKSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE7compareEPKc
      Referenced from: ./tokenizer-learning/devenv/lib/python3.6/site-packages/_youtokentome_cython.cpython-36m-darwin.so
      Expected in: /usr/lib/libstdc++.6.dylib
     in ./tokenizer-learning/devenv/lib/python3.6/site-packages/_youtokentome_cython.cpython-36m-darwin.so
    (devenv) machine:tokenizer-learning USER$

Maybe cause I'v upgraded recently to 10.15.1 Catalina?

I'd be happy for some help, since SentencePiece is causing us some issues in our project and we want to tackle these.

Edit:

Python 3.7.3 (default, Apr  3 2019, 05:39:12) 
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import youtokentome as yttm
>>> x = yttm.BPE
>>> print(x)
<class 'youtokentome.youtokentome.BPE'>

Seems to work out fine. So I'll work on that system, but maybe this is still a somewhat valuable information.

@LarsHill
Copy link

LarsHill commented Nov 4, 2019

I have exactly the same issue.
Mac OS Sierra 10.12.6 and Python 3.7.
Youtokentome version 1.0.1 runs fine. The problem only occurs with the recent version 1.0.2

@yutkin yutkin added the bug Something isn't working label Nov 4, 2019
@xbelonogov
Copy link
Contributor

We are working on this issue.
Currently you can install YouTokenToMe for Mac OS from source.

git clone git@github.com:VKCOM/YouTokenToMe.git
cd YouTokenToMe
CFLAGS="-stdlib=libc++" pip install .

@yutkin
Copy link
Contributor

yutkin commented Nov 5, 2019

Fixed in https://github.com/VKCOM/YouTokenToMe/releases/tag/v1.0.3. Please, update package.

@yutkin yutkin closed this as completed Nov 7, 2019
@yilmazay74
Copy link

yilmazay74 commented Jul 8, 2022

I have youtokentome version 1.0.6, but still getting No module named youtokentome error message. I am sure it is installed.
And I am running a python script in the same environment. How come it cannot see the installed youtokentome module.
Is there any other workaround? (Windows 10, python 3.7, youtokentome 1.0.6)

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

5 participants