-
Notifications
You must be signed in to change notification settings - Fork 998
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
LoRA Adapter from local model are leading to error #1893
Comments
Issue is in Peft I believe ? |
It works when using
from peft import AutoPeftModelForCausalLM
m = AutoPeftModelForCausalLM.from_pretrained("./tmp") |
Your example doesn't showcase the issue since you're passing the HFValidator, try using |
Thanks this leads to the issue. Steps:
huggingface-cli download alignment-handbook/zephyr-7b-sft-qlora --exclude "*.bin" "*.pth" "*.gguf" --local-dir ./tmp
rm tmp/config.json
docker run --gpus all -it --rm \
-v $(pwd)/tmp/:/opt/ml/model \
-e HUGGING_FACE_HUB_TOKEN=$(cat ~/.cache/huggingface/token) \
-e HF_TOKEN=$(cat ~/.cache/huggingface/token) \
--entrypoint /bin/bash nvcr.io/nvidia/pytorch:24.01-py3
pip3 install peft && python3
from peft import AutoPeftModelForCausalLM
m = AutoPeftModelForCausalLM.from_pretrained("/opt/ml/model") Error >>> m = AutoPeftModelForCausalLM.from_pretrained("/opt/ml/model")
config.json: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 571/571 [00:00<00:00, 9.28MB/s]
model.safetensors.index.json: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 25.1k/25.1k [00:00<00:00, 19.0MB/s]
model-00001-of-00002.safetensors: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████| 9.94G/9.94G [00:18<00:00, 549MB/s]
model-00002-of-00002.safetensors: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4.54G/4.54G [00:08<00:00, 553MB/s]
Downloading shards: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:26<00:00, 13.22s/it]
Loading checkpoint shards: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:01<00:00, 1.33it/s]
generation_config.json: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 116/116 [00:00<00:00, 1.14MB/s]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.10/dist-packages/peft/auto.py", line 128, in from_pretrained
return cls._target_peft_class.from_pretrained(
File "/usr/local/lib/python3.10/dist-packages/peft/peft_model.py", line 430, in from_pretrained
model.load_adapter(model_id, adapter_name, is_trainable=is_trainable, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/peft/peft_model.py", line 984, in load_adapter
adapters_weights = load_peft_weights(model_id, device=torch_device, **hf_hub_download_kwargs)
File "/usr/local/lib/python3.10/dist-packages/peft/utils/save_and_load.py", line 415, in load_peft_weights
has_remote_safetensors_file = file_exists(
File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_validators.py", line 106, in _inner_fn
validate_repo_id(arg_value)
File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_validators.py", line 154, in validate_repo_id
raise HFValidationError(
huggingface_hub.errors.HFValidationError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': '/opt/ml/model'. Use `repo_type` argument if needed. Will open an issue in |
This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days. |
System Info
ghcr.io/huggingface/text-generation-inference:2.0.2
Information
Tasks
Reproduction
Expected behavior
TGI will load peft model from the local disk abd then the base model defined in the
adapter_config.json
from huggingface.Error
The text was updated successfully, but these errors were encountered: