Getting very weird response from finetuned llama13b #18416
Replies: 1 comment
-
Hey there, @kaushalpowar! Long time no see - hope you've been doing well aside from this quirky issue you've run into. Based on the code and output you've shared, it seems like the model is stuck in a loop, generating the same token repeatedly. This could be due to a variety of reasons, including but not limited to, the model's configuration, the prompt you're using, or the parameters for the text generation. Here are a few things you could try:
If none of the above suggestions work, it would be helpful if you could provide more information about the issue. Specifically, it would be useful to know:
This additional information would help in diagnosing the issue more accurately. Sources
|
Beta Was this translation helpful? Give feedback.
-
Checked other resources
Commit to Help
Example Code
Description
I have used This script for finetuning.
When first finetuned llama13b it worked, all responses were coming normal.
But for past 2-3 weeks all the responses are giving garbage response. So for an alternative i am using same finetuned model from llamacpp. It is working fine over there.
I even tried finetunend Gemma 7b it 2 days ago, same weird responses were coming.
System Info
Package Version
absl-py 2.1.0
accelerate 0.28.0.dev0
aiohttp 3.9.2
aiosignal 1.3.1
annotated-types 0.6.0
anyio 4.2.0
appdirs 1.4.4
argon2-cffi 23.1.0
argon2-cffi-bindings 21.2.0
arrow 1.3.0
asttokens 2.4.1
async-lru 2.0.4
async-timeout 4.0.3
attrs 23.2.0
auto-gptq 0.6.0
azure-ai-formrecognizer 3.3.2
azure-cognitiveservices-vision-computervision 0.9.0
azure-common 1.1.28
azure-core 1.29.7
azure-identity 1.15.0
azure-storage-blob 12.19.0
Babel 2.14.0
backoff 2.2.1
beautifulsoup4 4.12.3
bitsandbytes 0.42.0
bleach 6.1.0
blinker 1.7.0
boto3 1.34.29
botocore 1.34.29
cachetools 5.3.2
certifi 2023.11.17
cffi 1.16.0
chardet 5.2.0
charset-normalizer 3.3.2
click 8.1.7
coloredlogs 15.0.1
comm 0.2.1
contourpy 1.2.0
cryptography 42.0.4
cycler 0.12.1
dataclasses-json 0.6.3
datasets 2.16.1
debugpy 1.8.0
decorator 5.1.1
defusedxml 0.7.1
Deprecated 1.2.14
deskew 1.5.1
dill 0.3.7
dirtyjson 1.0.8
diskcache 5.6.3
distro 1.9.0
docker-pycreds 0.4.0
einops 0.7.0
emoji 2.10.1
et-xmlfile 1.1.0
exceptiongroup 1.2.0
executing 2.0.1
fastapi 0.109.0
fastjsonschema 2.19.1
ffmpeg 1.4
filelock 3.13.1
filetype 1.2.0
fire 0.5.0
Flask 3.0.2
fonttools 4.47.2
fqdn 1.5.1
frozenlist 1.4.1
fsspec 2023.10.0
fuzzywuzzy 0.18.0
gekko 1.0.6
gguf 0.6.0
gitdb 4.0.11
GitPython 3.1.41
google 3.0.0
google-auth 2.27.0
google-auth-oauthlib 1.2.0
greenlet 3.0.3
grpcio 1.60.0
h11 0.14.0
httpcore 1.0.2
httpx 0.26.0
huggingface-hub 0.20.3
humanfriendly 10.0
idna 3.6
imageio 2.33.1
imutils 0.5.4
install 1.3.5
ipykernel 6.29.0
ipython 8.20.0
ipywidgets 8.1.1
isodate 0.6.1
isoduration 20.11.0
itsdangerous 2.1.2
jedi 0.19.1
Jinja2 3.1.3
jmespath 1.0.1
joblib 1.3.2
json5 0.9.14
jsonformer 0.12.0
jsonlines 4.0.0
jsonpatch 1.33
jsonpath-python 1.0.6
jsonpointer 2.4
jsonschema 4.21.1
jsonschema-specifications 2023.12.1
jupyter 1.0.0
jupyter_client 8.6.0
jupyter-console 6.6.3
jupyter_core 5.7.1
jupyter-events 0.9.0
jupyter-lsp 2.2.2
jupyter_server 2.12.5
jupyter_server_terminals 0.5.2
jupyterlab 4.0.12
jupyterlab_pygments 0.3.0
jupyterlab_server 2.25.2
jupyterlab-widgets 3.0.9
kiwisolver 1.4.5
kor 1.0.0
langchain 0.1.4
langchain-community 0.0.16
langchain-core 0.1.16
langdetect 1.0.9
langsmith 0.0.84
lazy_loader 0.3
llama_cpp_python 0.2.38
llama-index 0.9.40
loralib 0.1.2
lxml 5.1.0
Markdown 3.5.2
MarkupSafe 2.1.5
marshmallow 3.20.2
matplotlib 3.8.2
matplotlib-inline 0.1.6
mistune 3.0.2
more-itertools 10.2.0
mpmath 1.3.0
msal 1.26.0
msal-extensions 1.1.0
msrest 0.7.1
multidict 6.0.4
multiprocess 0.70.15
mypy-extensions 1.0.0
nbclient 0.9.0
nbconvert 7.14.2
nbformat 5.9.2
nest-asyncio 1.6.0
networkx 3.2.1
nltk 3.8.1
notebook 7.0.7
notebook_shim 0.2.3
numpy 1.24.4
nvidia-cublas-cu12 12.1.3.1
nvidia-cuda-cupti-cu12 12.1.105
nvidia-cuda-nvrtc-cu12 12.1.105
nvidia-cuda-runtime-cu12 12.1.105
nvidia-cudnn-cu12 8.9.2.26
nvidia-cufft-cu12 11.0.2.54
nvidia-curand-cu12 10.3.2.106
nvidia-cusolver-cu12 11.4.5.107
nvidia-cusparse-cu12 12.1.0.106
nvidia-nccl-cu12 2.18.1
nvidia-nvjitlink-cu12 12.3.101
nvidia-nvtx-cu12 12.1.105
oauthlib 3.2.2
openai 1.10.0
opencv-python 4.7.0.72
openpyxl 3.1.2
optimum 1.16.2
overrides 7.7.0
packaging 23.2
pandas 1.5.3
pandocfilters 1.5.1
parso 0.8.3
peft 0.8.2
pexpect 4.9.0
pillow 10.2.0
pip 24.0
platformdirs 4.1.0
portalocker 2.8.2
prometheus-client 0.19.0
prompt-toolkit 3.0.43
protobuf 4.25.2
psutil 5.9.8
ptyprocess 0.7.0
pure-eval 0.2.2
pyarrow 15.0.0
pyarrow-hotfix 0.6
pyasn1 0.5.1
pyasn1-modules 0.3.0
pybboxes 0.1.6
pycparser 2.21
pydantic 2.6.0
pydantic_core 2.16.1
Pygments 2.17.2
PyJWT 2.8.0
pynvml 11.5.0
pyparsing 3.1.1
pypdf 4.0.1
pytesseract 0.3.10
python-dateutil 2.8.2
python-dotenv 1.0.1
python-iso639 2024.1.2
python-json-logger 2.0.7
python-magic 0.4.27
python-multipart 0.0.9
pytz 2023.4
PyYAML 6.0.1
pyzmq 25.1.2
qtconsole 5.5.1
QtPy 2.4.1
rapidfuzz 3.6.1
referencing 0.33.0
regex 2023.12.25
requests 2.31.0
requests-oauthlib 1.3.1
rfc3339-validator 0.1.4
rfc3986-validator 0.1.1
rouge 1.0.1
rpds-py 0.17.1
rsa 4.9
s3transfer 0.10.0
safetensors 0.4.2
sahi 0.11.15
scikit-image 0.22.0
scikit-learn 1.4.0
scipy 1.12.0
seaborn 0.13.2
Send2Trash 1.8.2
sentence-transformers 2.3.1
sentencepiece 0.1.99
sentry-sdk 1.39.2
setproctitle 1.3.3
setuptools 69.1.0
shapely 2.0.2
six 1.16.0
smmap 5.0.1
sniffio 1.3.0
soundfile 0.12.1
soupsieve 2.5
SQLAlchemy 2.0.25
stack-data 0.6.3
starlette 0.35.1
sympy 1.12
tabulate 0.9.0
tenacity 8.2.3
tensorboard 2.15.1
tensorboard-data-server 0.7.2
termcolor 2.4.0
terminado 0.18.0
terminaltables 3.1.10
thop 0.1.1.post2209072238
threadpoolctl 3.2.0
tifffile 2023.12.9
tiktoken 0.5.2
tinycss2 1.2.1
tokenizers 0.15.1
tomli 2.0.1
torch 2.1.2
torchaudio 2.2.0
torchmetrics 0.6.2
torchvision 0.17.0
tornado 6.4
tqdm 4.66.1
traitlets 5.14.1
transformers 4.38.0.dev0
triton 2.1.0
types-python-dateutil 2.8.19.20240106
typing_extensions 4.9.0
typing-inspect 0.9.0
tzdata 2023.4
unstructured 0.12.3
unstructured-client 0.16.0
uri-template 1.3.0
urllib3 2.0.7
uvicorn 0.27.0.post1
wandb 0.16.2
wcwidth 0.2.13
webcolors 1.13
webencodings 0.5.1
websocket-client 1.7.0
Werkzeug 3.0.1
wheel 0.42.0
widgetsnbextension 4.0.9
wrapt 1.16.0
xxhash 3.4.1
yarl 1.9.2
yolov5 6.0.6
zipp 1.0.0
Beta Was this translation helpful? Give feedback.
All reactions