RuntimeError: Internal: src/sentencepiece_processor.cc(1101) [model_proto->ParseFromArray(serialized.data(), serialized.size())]

[clabe45]

When applying these deltas to these base weights I get the following error:

$ python apply_delta.py --base-model-path ../../llama-7b-hf/ --target-model-path ../../gorilla-7b-hf-v0/ --delta-path ../../gorilla-7b-hf-delta-v0/
Loading the delta weights from ../../gorilla-7b-hf-delta-v0/
Traceback (most recent call last):
  File "/home/paperspace/projects/gorilla/gorilla/inference/apply_delta.py", line 167, in <module>
    apply_delta(args.base_model_path, args.target_model_path, args.delta_path)
  File "/home/paperspace/projects/gorilla/gorilla/inference/apply_delta.py", line 129, in apply_delta
    delta_tokenizer = AutoTokenizer.from_pretrained(delta_path, use_fast=False)
  File "/home/paperspace/.local/lib/python3.9/site-packages/transformers/models/auto/tokenization_auto.py", line 702, in from_pretrained
    return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
  File "/home/paperspace/.local/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 1811, in from_pretrained
    return cls._from_pretrained(
  File "/home/paperspace/.local/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 1965, in _from_pretrained
    tokenizer = cls(*init_inputs, **init_kwargs)
  File "/home/paperspace/.local/lib/python3.9/site-packages/transformers/models/llama/tokenization_llama.py", line 96, in __init__
    self.sp_model.Load(vocab_file)
  File "/home/paperspace/.local/lib/python3.9/site-packages/sentencepiece/__init__.py", line 905, in Load
    return self.LoadFromFile(model_file)
  File "/home/paperspace/.local/lib/python3.9/site-packages/sentencepiece/__init__.py", line 310, in LoadFromFile
    return _sentencepiece.SentencePieceProcessor_LoadFromFile(self, arg)
RuntimeError: Internal: src/sentencepiece_processor.cc(1101) [model_proto->ParseFromArray(serialized.data(), serialized.size())] 

Specs:

$ nvidia-smi
Thu Jun  1 17:50:22 2023       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 515.105.01   Driver Version: 515.105.01   CUDA Version: 11.7     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Quadro M4000        Off  | 00000000:00:05.0  On |                  N/A |
| 46%   32C    P8    16W / 120W |    189MiB /  8192MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A      1532      G   /usr/lib/xorg/Xorg                121MiB |
|    0   N/A  N/A      2011      G   /usr/bin/gnome-shell               59MiB |
|    0   N/A  N/A      2571      G   ...bexec/gnome-initial-setup        2MiB |
+-----------------------------------------------------------------------------+

$ LC_ALL=C lspci -v | grep -EA10 "3D|VGA" | grep 'prefetchable' 
	Memory at f4000000 (32-bit, prefetchable) [size=8M]
	Memory at f3000000 (32-bit, non-prefetchable) [size=16M]
	Memory at e0000000 (64-bit, prefetchable) [size=256M]
	Memory at f0000000 (64-bit, prefetchable) [size=32M]

$ free -h
              total        used        free      shared  buff/cache   available
Mem:           29Gi       1.2Gi       5.6Gi        13Mi        22Gi        27Gi
Swap:            0B          0B          0B

[ShishirPatil]

Hi @clabe45 can you share with us your python3 environment and make sure you have all the dependencies installed? If you can freeze your environment and share it with me, I can try to recreate the issue. Thanks!

[clabe45]

Hi, here's the output of pip freeze:

absl-py==1.4.0
accelerate==0.19.0
agate==1.6.0
agate-dbf==0.2.0
agate-excel==0.2.3
agate-sql==0.5.2
aiohttp==3.8.3
aiosignal==1.3.1
anyio==3.6.2
appdirs==1.4.3
apturl==0.5.2
argon2-cffi==21.3.0
argon2-cffi-bindings==21.2.0
asttokens==2.2.1
astunparse==1.6.3
async-timeout==4.0.2
attrs==18.2.0
Automat==0.8.0
awscli==1.18.69
Babel==2.11.0
backcall==0.2.0
beautifulsoup4==4.11.1
bleach==5.0.1
blinker==1.4
blis==0.7.9
boto3==1.24.66
botocore==1.27.96
Brlapi==0.7.0
cachetools==5.2.1
catalogue==2.0.8
certifi==2019.11.28
cffi==1.15.1
chardet==3.0.4
charset-normalizer==2.1.1
click==8.1.3
click-completion==0.5.2
click-didyoumean==0.3.0
click-help-colors==0.9.1
cloud-init==23.1.2
cloudpickle==2.1.0
cmake==3.26.3
colorama==0.4.3
comm==0.1.2
command-not-found==0.3
confection==0.0.4
configobj==5.0.6
constantly==15.1.0
cryptography==2.8
csvkit==1.0.2
cupshelpers==1.0
cycler==0.11.0
cymem==2.0.7
Cython==0.29.32
datasets==2.4.0
dbfread==2.0.7
dbus-python==1.2.16
debugpy==1.6.5
decorator==5.1.1
defer==1.0.6
defusedxml==0.7.1
dill==0.3.5.1
distlib==0.3.0
distro==1.4.0
distro-info===0.23ubuntu1
docker-pycreds==0.4.0
docutils==0.16
entrypoints==0.3
et-xmlfile==1.0.1
executing==1.2.0
fastjsonschema==2.16.2
filelock==3.9.0
flatbuffers==1.12
fonttools==4.38.0
frozenlist==1.3.3
fsspec==2022.11.0
future==0.18.2
gast==0.4.0
gdown==4.5.1
gitdb==4.0.10
GitPython==3.1.30
google-auth==2.16.0
google-auth-oauthlib==0.4.6
google-pasta==0.2.0
gql==3.0.0a6
gradient==2.0.6
gradient-utils==0.5.0
graphql-core==3.1.7
greenlet==2.0.1
grpcio==1.51.1
h5py==3.7.0
halo==0.0.31
httplib2==0.14.0
huggingface-hub==0.14.1
hyperlink==19.0.0
idna==2.8
imageio==2.24.0
importlib-metadata==6.0.0
incremental==16.10.1
ipykernel==6.15.2
ipython==8.5.0
ipython-genutils==0.2.0
ipywidgets==8.0.2
isodate==0.6.0
jax==0.4.8
jaxlib==0.4.7+cuda11.cudnn82
jdcal==1.0
jedi==0.18.2
Jinja2==3.1.2
jmespath==0.9.4
joblib==1.2.0
json5==0.9.11
jsonify==0.5
jsonpatch==1.22
jsonpointer==2.0
jsonschema==4.17.3
jupyter-contrib-core==0.4.2
jupyter-contrib-nbextensions==0.7.0
jupyter-highlight-selected-word==0.2.0
jupyter-nbextensions-configurator==0.6.1
jupyter-server==1.23.5
jupyter-server-mathjax==0.2.6
jupyter_client==7.4.8
jupyter_core==5.1.3
jupyterlab==3.4.6
jupyterlab-git==0.41.0
jupyterlab-pygments==0.2.2
jupyterlab-widgets==3.0.5
jupyterlab_server==2.18.0
keras==2.9.0
Keras-Preprocessing==1.1.2
keyring==18.0.1
kiwisolver==1.4.4
langcodes==3.3.0
language-selector==0.1
launchpadlib==1.10.13
lazr.restfulclient==0.14.2
lazr.uri==1.0.3
leather==0.3.3
libclang==15.0.6.1
lit==16.0.5
log-symbols==0.0.14
louis==3.12.0
lxml==4.5.0
macaroonbakery==1.3.1
Markdown==3.4.1
MarkupSafe==2.1.1
marshmallow==2.21.0
matplotlib==3.5.3
matplotlib-inline==0.1.6
mistune==2.0.4
ml-dtypes==0.1.0
more-itertools==4.2.0
mpmath==1.3.0
multidict==6.0.4
multiprocess==0.70.13
murmurhash==1.0.9
nbclassic==0.4.8
nbclient==0.7.2
nbconvert==7.2.7
nbdime==3.1.1
nbformat==5.7.3
nest-asyncio==1.5.6
netifaces==0.10.4
networkx==3.0
nltk==3.7
notebook==6.5.2
notebook_shim==0.2.2
numpy==1.23.2
nvidia-cublas-cu11==11.10.3.66
nvidia-cuda-cupti-cu11==11.7.101
nvidia-cuda-nvrtc-cu11==11.7.99
nvidia-cuda-runtime-cu11==11.7.99
nvidia-cudnn-cu11==8.5.0.96
nvidia-cufft-cu11==10.9.0.58
nvidia-curand-cu11==10.2.10.91
nvidia-cusolver-cu11==11.4.0.1
nvidia-cusparse-cu11==11.7.4.91
nvidia-nccl-cu11==2.14.3
nvidia-nvtx-cu11==11.7.91
oauthlib==3.1.0
olefile==0.46
opencv-python==4.6.0.66
openpyxl==3.0.3
opt-einsum==3.3.0
packaging==23.0
pandas==1.4.4
pandocfilters==1.5.0
parsedatetime==2.4
parso==0.8.3
pathtools==0.1.2
pathy==0.10.1
pexpect==4.6.0
pickleshare==0.7.5
Pillow==9.2.0
platformdirs==2.6.2
preshed==3.0.8
progressbar2==4.2.0
prometheus-client==0.9.0
promise==2.3
prompt-toolkit==3.0.38
protobuf==3.19.6
psutil==5.9.4
ptyprocess==0.7.0
pure-eval==0.2.2
pyarrow==10.0.1
pyasn1==0.4.2
pyasn1-modules==0.2.1
pycairo==1.16.2
pycparser==2.21
pycups==1.9.73
pydantic==1.9.2
Pygments==2.14.0
PyGObject==3.36.0
PyHamcrest==1.9.0
PyJWT==1.7.1
pymacaroons==0.13.0
pymongo==3.13.0
PyNaCl==1.3.0
pyOpenSSL==19.0.0
pyparsing==3.0.9
pyRFC3339==1.1
pyrsistent==0.15.5
pyserial==3.4
PySocks==1.7.1
python-apt==2.0.1+ubuntu0.20.4.1
python-dateutil==2.8.2
python-debian===0.1.36ubuntu1
python-distutils-extra==2.39
python-slugify==4.0.0
python-utils==3.4.5
pytimeparse==1.1.5
pytz==2022.7
PyWavelets==1.4.1
pyxdg==0.26
PyYAML==5.4.1
pyzmq==25.0.0
regex==2022.10.31
reportlab==3.5.34
requests==2.28.2
requests-oauthlib==1.3.1
requests-toolbelt==0.10.1
requests-unixsocket==0.2.0
responses==0.18.0
roman==2.0.0
rsa==4.0
s3transfer==0.6.0
scikit-image==0.19.3
scikit-learn==1.1.2
scipy==1.9.1
screen-resolution-extra==0.0.0
seaborn==0.12.0
SecretStorage==2.3.1
Send2Trash==1.8.0
sentence-transformers==2.2.2
sentencepiece==0.1.99
sentry-sdk==1.13.0
service-identity==18.1.0
setproctitle==1.3.2
shellingham==1.5.0.post1
shortuuid==1.0.11
simplejson==3.16.0
six==1.14.0
smart-open==6.3.0
smmap==5.0.0
sniffio==1.3.0
sos==4.4
soupsieve==2.3.2.post1
spacy==3.4.1
spacy-legacy==3.0.11
spacy-loggers==1.0.4
spinners==0.0.24
SQLAlchemy==1.4.40
srsly==2.4.5
ssh-import-id==5.10
stack-data==0.6.2
sympy==1.12
systemd-python==234
tabulate==0.8.10
tensorboard==2.9.1
tensorboard-data-server==0.6.1
tensorboard-plugin-wit==1.8.1
tensorflow==2.9.2
tensorflow-estimator==2.9.0
tensorflow-io-gcs-filesystem==0.29.0
termcolor==2.2.0
terminado==0.17.1
terminaltables==3.1.10
thinc==8.1.6
threadpoolctl==3.1.0
tifffile==2022.10.10
tinycss2==1.2.1
tokenizers==0.12.1
torch==2.0.1
torchaudio==0.12.1+cu116
torchvision==0.13.1+cu116
tornado==6.2
tqdm==4.65.0
traitlets==5.8.1
transformers==4.28.1
triton==2.0.0
Twisted==18.9.0
typer==0.4.2
typing_extensions==4.4.0
ubuntu-advantage-tools==8001
ubuntu-drivers-common==0.0.0
ufw==0.36
unattended-upgrades==0.1
Unidecode==1.1.1
urllib3==1.26.14
virtualenv==20.0.17
wadllib==1.3.3
wandb==0.13.4
wasabi==0.10.1
wcwidth==0.2.5
webencodings==0.5.1
websocket-client==0.57.0
Werkzeug==2.2.2
widgetsnbextension==4.0.5
wrapt==1.14.1
xgboost==1.6.2
xkit==0.0.0
xlrd==1.1.0
xxhash==3.2.0
yarl==1.8.2
zipp==1.0.0
zope.interface==4.7.1

[clabe45]

I'm using Python 3.8.10 too

[ShishirPatil]

This should be fixed with the latest releases! Please feel free to re-open this if you run into this issue again!