Segmentation fault error

[ZordoC]

Hello!

🐛 Bug

Whenever I am try to run your demo, when using :

from comet.models import download_model

I get:

Segmentation fault (core dumped)

To Reproduce

I'm using Amazon EC2 instances with ubuntu 18.04 and I've also tried with ubuntu 20.04 and got the same errors.
I log into the instance and do:

sudo su
apt-get update
apt-get install python3-pip
pip3 install unbabel-comet

If I'm on 18.04 I have to pip upgrade because otherwise sentence piece breaks.

I've tried building from source installing from pip nothing seems to be working it installs and I can even do:

import comet

But whenever :

from comet.models import download_model

segmentation fault, also if I use the cli command:

comet download -d apequest --saving_path data/

again the same error!

Environment

OS: Linux
Packaging pip
Version : pip 20.0.2

I've tried multiple instances, virtual environments but nothing seems to be effective! With ubuntu 20.04 I used python 3.8, I've seen that the recommendation is python 3.6, On ubuntu 18.04 I've used the default python3 which is 3.6.9 and still had the same issues!

Here is the output of my pip freeze

absl-py==0.11.0
cachetools==4.1.1
certifi==2020.11.8
cffi==1.14.3
chardet==3.0.4
click==7.1.2
Cython==0.29.15
fairseq==0.9.0
fastBPE==0.1.0
filelock==3.0.12
fsspec==0.8.4
future==0.18.2
google-auth==1.23.0
google-auth-oauthlib==0.4.2
grpcio==1.33.2
idna==2.10
joblib==0.17.0
Markdown==3.3.3
numpy==1.19.4
oauthlib==3.1.0
pandas==1.0.5
portalocker==2.0.0
protobuf==3.14.0
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycparser==2.20
python-dateutil==2.8.1
pytorch-lightning==1.0.7
pytorch-nlp==0.5.0
pytz==2020.4
PyYAML==5.3.1
regex==2020.11.13
requests==2.25.0
requests-oauthlib==1.3.0
rsa==4.6
sacrebleu==1.4.14
sacremoses==0.0.43
scikit-learn==0.23.1
scipy==1.5.4
sentencepiece==0.1.94
six==1.15.0
sphinx-markdown-tables==0.0.15
tensorboard==2.2.0
tensorboard-plugin-wit==1.7.0
threadpoolctl==2.1.0
tokenizers==0.7.0
torch==1.4.0
tqdm==4.52.0
transformers==2.10.0
-e git+https://github.com/Unbabel/COMET@c9ac4c9cbdb8484aa5ee286c9cbe13002c16a193#egg=unbabel_comet
urllib3==1.26.2
Werkzeug==1.0.1
wget==3.2

If you could help me figure this thing out it would be wonderful!

Thank you for your time and willingness to share your tool! I'm eager to try it :-)

[ricardorei]

Can you please try: pip install sentencepiece==0.1.91 ?

[ricardorei]

I managed to replicate the error you were getting and after installing sentencepiece 0.1.91 it worked

[ricardorei]

huggingface/transformers#4857

[ZordoC]

That seem to have solved my issue :-)
Obrigado!

[sammichenVV]

hi, have downgraded the sentencepiece version to 0.19.1, but still report this error.