First Example is not working #318
Assignees
Labels
bug
Something isn't working
Comments
|
I kind of ran into the same problem when trying it, after trying to debug it, I just changed the key to "llama2" after realizing that "llama" was not in the dictionary. |
|
Will open up a PR to update the readme and docs! |
This was referenced Oct 7, 2023
|
opened #319 to update the readme and anarchy-ai/docs#37 to update the docs |
|
@Bouscout @daspartho In [1]: # import our client
...: from llm_vm.client import Client
...:
...: # Select the LlaMA model
...: client = Client(big_model = 'llama2')
...:
...: # Put in your prompt and go!
...: response = client.complete(prompt = 'What is Anarchy?', context = '')
...: print(response)
/usr/lib/python3/dist-packages/scipy/__init__.py:146: UserWarning: A NumPy version >=1.17.3 and <1.25.0 is required for this version of SciPy (detected version 1.25.2
warnings.warn(f"A NumPy version >={np_minversion} and <{np_maxversion}"
/home/patel/.local/lib/python3.10/site-packages/torch/cuda/__init__.py:107: UserWarning: CUDA initialization: CUDA unknown error - this may be due to an incorrectly set up environment, e.g. changing env variable CUDA_VISIBLE_DEVICES after program start. Setting the available devices to be zero. (Triggered internally at ../c10/cuda/CUDAFunctions.cpp:109.)
return torch._C._cuda_getDeviceCount() > 0
Project Root: /media/patel/ubuntu/coding/learn/opensource/LLM-VM/src/llm_vm
---------------------------------------------------------------------------
HTTPError Traceback (most recent call last)
File ~/.local/lib/python3.10/site-packages/huggingface_hub/utils/_errors.py:261, in hf_raise_for_status(response, endpoint_name)
260 try:
--> 261 response.raise_for_status()
262 except HTTPError as e:
File ~/.local/lib/python3.10/site-packages/requests/models.py:1021, in Response.raise_for_status(self)
1020 if http_error_msg:
-> 1021 raise HTTPError(http_error_msg, response=self)
HTTPError: 404 Client Error: Not Found for url: https://huggingface.co/meta-llama/Llama-2-7b/resolve/main/config.json
The above exception was the direct cause of the following exception:
EntryNotFoundError Traceback (most recent call last)
File ~/.local/lib/python3.10/site-packages/transformers/utils/hub.py:429, in cached_file(path_or_repo_id, filename, cache_dir, force_download, resume_download, proxies, token, revision, local_files_only, subfolder, repo_type, user_agent, _raise_exceptions_for_missing_entries, _raise_exceptions_for_connection_errors, _commit_hash, **deprecated_kwargs)
427 try:
428 # Load from URL or cache if already cached
--> 429 resolved_file = hf_hub_download(
430 path_or_repo_id,
431 filename,
432 subfolder=None if len(subfolder) == 0 else subfolder,
433 repo_type=repo_type,
434 revision=revision,
435 cache_dir=cache_dir,
436 user_agent=user_agent,
437 force_download=force_download,
438 proxies=proxies,
439 resume_download=resume_download,
440 token=token,
441 local_files_only=local_files_only,
442 )
443 except GatedRepoError as e:
File ~/.local/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py:118, in validate_hf_hub_args.<locals>._inner_fn(*args, **kwargs)
116 kwargs = smoothly_deprecate_use_auth_token(fn_name=fn.__name__, has_token=has_token, kwargs=kwargs)
--> 118 return fn(*args, **kwargs)
File ~/.local/lib/python3.10/site-packages/huggingface_hub/file_download.py:1195, in hf_hub_download(repo_id, filename, subfolder, repo_type, revision, library_name, library_version, cache_dir, local_dir, local_dir_use_symlinks, user_agent, force_download, force_filename, proxies, etag_timeout, resume_download, token, local_files_only, legacy_cache_layout)
1194 try:
-> 1195 metadata = get_hf_file_metadata(
1196 url=url,
1197 token=token,
1198 proxies=proxies,
1199 timeout=etag_timeout,
1200 )
1201 except EntryNotFoundError as http_error:
1202 # Cache the non-existence of the file and raise
File ~/.local/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py:118, in validate_hf_hub_args.<locals>._inner_fn(*args, **kwargs)
116 kwargs = smoothly_deprecate_use_auth_token(fn_name=fn.__name__, has_token=has_token, kwargs=kwargs)
--> 118 return fn(*args, **kwargs)
File ~/.local/lib/python3.10/site-packages/huggingface_hub/file_download.py:1541, in get_hf_file_metadata(url, token, proxies, timeout)
1532 r = _request_wrapper(
1533 method="HEAD",
1534 url=url,
(...)
1539 timeout=timeout,
1540 )
-> 1541 hf_raise_for_status(r)
1543 # Return
File ~/.local/lib/python3.10/site-packages/huggingface_hub/utils/_errors.py:271, in hf_raise_for_status(response, endpoint_name)
270 message = f"{response.status_code} Client Error." + "\n\n" + f"Entry Not Found for url: {response.url}."
--> 271 raise EntryNotFoundError(message, response) from e
273 elif error_code == "GatedRepo":
EntryNotFoundError: 404 Client Error. (Request ID: Root=1-6522c4a6-2ffd0e5f1beba264165d0597;0e03c9d0-c0b4-4d6c-9e26-0ee65cc34e05)
Entry Not Found for url: https://huggingface.co/meta-llama/Llama-2-7b/resolve/main/config.json.
The above exception was the direct cause of the following exception:
OSError Traceback (most recent call last)
Cell In[1], line 5
2 from llm_vm.client import Client
4 # Select the LlaMA model
----> 5 client = Client(big_model = 'llama2')
7 # Put in your prompt and go!
8 response = client.complete(prompt = 'What is Anarchy?', context = '')
File /media/patel/ubuntu/coding/learn/opensource/LLM-VM/src/llm_vm/client.py:66, in Client.__init__(self, big_model, small_model, big_model_config, small_model_config)
52 def __init__(self, big_model = default_big_model, small_model =default_small_model,big_model_config={},small_model_config={}):
53 """
54 This __init__ function allows the user to specify which LLM they want to use upon instantiation.
55
(...)
64 >>> client = Client(big_model = 'neo')
65 """
---> 66 self.teacher = load_model_closure(big_model)(**big_model_config)
67 self.student = load_model_closure(small_model)(**small_model_config)
69 ## FIXME, do something like $HOME/.llm_vm/finetuned_models/
File /media/patel/ubuntu/coding/learn/opensource/LLM-VM/src/llm_vm/onsite_llm.py:89, in BaseOnsiteLLM.__init__(self, model_uri, tokenizer_kw_args, model_kw_args)
87 raise ValueError('model_uri not found')
88 self.model_name : str = self.model_uri.split('/')[-1] # our default for deriving model name
---> 89 self.model=self.model_loader(**model_kw_args)
90 self.tokenizer=self.tokenizer_loader(**tokenizer_kw_args)
92 # Move the model to the specified device(s)
File /media/patel/ubuntu/coding/learn/opensource/LLM-VM/src/llm_vm/onsite_llm.py:320, in SmallLocalLLama.model_loader(self)
319 def model_loader(self):
--> 320 return LlamaForCausalLM.from_pretrained(self.model_uri)
File ~/.local/lib/python3.10/site-packages/transformers/modeling_utils.py:2530, in PreTrainedModel.from_pretrained(cls, pretrained_model_name_or_path, config, cache_dir, ignore_mismatched_sizes, force_download, local_files_only, token, revision, use_safetensors, *model_args, **kwargs)
2528 if not isinstance(config, PretrainedConfig):
2529 config_path = config if config is not None else pretrained_model_name_or_path
-> 2530 config, model_kwargs = cls.config_class.from_pretrained(
2531 config_path,
2532 cache_dir=cache_dir,
2533 return_unused_kwargs=True,
2534 force_download=force_download,
2535 resume_download=resume_download,
2536 proxies=proxies,
2537 local_files_only=local_files_only,
2538 token=token,
2539 revision=revision,
2540 subfolder=subfolder,
2541 _from_auto=from_auto_class,
2542 _from_pipeline=from_pipeline,
2543 **kwargs,
2544 )
2545 else:
2546 model_kwargs = kwargs
File ~/.local/lib/python3.10/site-packages/transformers/configuration_utils.py:591, in PretrainedConfig.from_pretrained(cls, pretrained_model_name_or_path, cache_dir, force_download, local_files_only, token, revision, **kwargs)
587 kwargs["revision"] = revision
589 cls._set_token_in_kwargs(kwargs, token)
--> 591 config_dict, kwargs = cls.get_config_dict(pretrained_model_name_or_path, **kwargs)
592 if "model_type" in config_dict and hasattr(cls, "model_type") and config_dict["model_type"] != cls.model_type:
593 logger.warning(
594 f"You are using a model of type {config_dict['model_type']} to instantiate a model of type "
595 f"{cls.model_type}. This is not supported for all configurations of models and can yield errors."
596 )
File ~/.local/lib/python3.10/site-packages/transformers/configuration_utils.py:620, in PretrainedConfig.get_config_dict(cls, pretrained_model_name_or_path, **kwargs)
618 original_kwargs = copy.deepcopy(kwargs)
619 # Get config dict associated with the base config file
--> 620 config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, **kwargs)
621 if "_commit_hash" in config_dict:
622 original_kwargs["_commit_hash"] = config_dict["_commit_hash"]
File ~/.local/lib/python3.10/site-packages/transformers/configuration_utils.py:675, in PretrainedConfig._get_config_dict(cls, pretrained_model_name_or_path, **kwargs)
671 configuration_file = kwargs.pop("_configuration_file", CONFIG_NAME)
673 try:
674 # Load from local folder or from cache or download from model Hub and cache
--> 675 resolved_config_file = cached_file(
676 pretrained_model_name_or_path,
677 configuration_file,
678 cache_dir=cache_dir,
679 force_download=force_download,
680 proxies=proxies,
681 resume_download=resume_download,
682 local_files_only=local_files_only,
683 token=token,
684 user_agent=user_agent,
685 revision=revision,
686 subfolder=subfolder,
687 _commit_hash=commit_hash,
688 )
689 commit_hash = extract_commit_hash(resolved_config_file, commit_hash)
690 except EnvironmentError:
691 # Raise any environment error raise by `cached_file`. It will have a helpful error message adapted to
692 # the original exception.
File ~/.local/lib/python3.10/site-packages/transformers/utils/hub.py:480, in cached_file(path_or_repo_id, filename, cache_dir, force_download, resume_download, proxies, token, revision, local_files_only, subfolder, repo_type, user_agent, _raise_exceptions_for_missing_entries, _raise_exceptions_for_connection_errors, _commit_hash, **deprecated_kwargs)
478 if revision is None:
479 revision = "main"
--> 480 raise EnvironmentError(
481 f"{path_or_repo_id} does not appear to have a file named {full_filename}. Checkout "
482 f"'https://huggingface.co/{path_or_repo_id}/{revision}' for available files."
483 ) from e
484 except HTTPError as err:
485 # First we try to see if we have a cached version (not up to date):
486 resolved_file = try_to_load_from_cache(path_or_repo_id, full_filename, cache_dir=cache_dir, revision=revision)
OSError: meta-llama/Llama-2-7b does not appear to have a file named config.json. Checkout 'https://huggingface.co/meta-llama/Llama-2-7b/main' for available files. |
|
probably it was meant to use https://huggingface.co/meta-llama/Llama-2-7b/tree/main instead of https://huggingface.co/meta-llama/Llama-2-7b/main somewhere in the code |
|
https://huggingface.co/docs/transformers/v4.34.0/en/model_doc/llama2#overview this might be useful resource |
|
did some digging on the issue and found these relevant links
basically, it seems we should load model that is converted to HF LLM-VM/src/llm_vm/onsite_llm.py Line 464 in 7d2a52a so here instead of @VictorOdede @cartazio could you confirm this? |
|
@daspartho thanks for looking into this. Will make this fix asap |
|
Fixed in #366 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
After installing the LLM-VM i tried to run the first example in the README.md but it is giving me KeyError
The text was updated successfully, but these errors were encountered: