Unable to use converted Llama 3.3 instruct model

[SRGAnalytics-MD]

System Info

Hi, I'm having issues loading a model (downloaded from meta) and converted using convert_llama_weights_to_hf. This is the error I am receiving: cannot access local variable 'model' where it is not associated with a value

Output of transformers-cli env

- `transformers` version: 4.49.0
- Platform: Linux-5.15.0-1072-aws-x86_64-with-glibc2.35
- Python version: 3.11.10
- Huggingface_hub version: 0.29.2
- Safetensors version: 0.5.3
- Accelerate version: 1.4.0
- Accelerate config: 	not found
- DeepSpeed version: not installed
- PyTorch version (GPU?): 2.6.0+cu124 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: Using Databricks Notebook, serverless cluster

Who can help?

@ArthurZucker

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

Steps taken

1. Downloaded Llama3.3-70B-Instruct, llama model download --source meta --model-id Llama3.3-70B-Instruct

Files are:
checklist.chk
consolidated.00.pth
consolidated.01.pth
consolidated.02.pth
consolidated.03.pth
consolidated.04.pth
consolidated.05.pth
consolidated.06.pth
consolidated.07.pth
params.json
tokenizer.model

2. Downloaded and saved conversion script to local directory: https://github.com/huggingface/transformers/blob/main/src/transformers/models/llama/convert_llama_weights_to_hf.py

3. Convert files to Hugging Face, as per as per unable to convert llama 3.3 weights to hf.py #35326.

!python convert_llama_weights_to_hf.py --input_dir /Volumes/dev/models/llama3_3_70b/llama3-3-70b/ --model_size 70B --output_dir /Volumes/dev/models/llama3_3_70b_hf --llama_version 3 --instruct

Files in output_dir:
special_tokens_map.json
tokenizer_config.json
tokenizer.json

3. Attempt to load model, as per https://huggingface.co/docs/transformers/main/en/model_doc/llama3

from transformers import AutoModelForCausalLM, AutoTokenizer

hf_path = '/Volumes/dev/models/llama3_3_70b_hf/'

tokenizer = AutoTokenizer.from_pretrained(hf_path)
model = AutoModelForCausalLM.from_pretrained(hf_path)

Error:

cannot access local variable 'model' where it is not associated with a value

OSError: Error no file named pytorch_model.bin, model.safetensors, tf_model.h5, model.ckpt.index or flax_model.msgpack found in directory /Volumes/dev/models/llama3_3_70b_hf/.
File , line 4
1 from transformers import AutoModelForCausalLM, AutoTokenizer
3 tokenizer = AutoTokenizer.from_pretrained(hf_path)
----> 4 model = AutoModelForCausalLM.from_pretrained(hf_path)
File /local_disk0/.ephemeral_nfs/envs/pythonEnv-64f42d2b-5704-4f18-af79-66f0e0b3c167/lib/python3.11/site-packages/transformers/modeling_utils.py:3808, in PreTrainedModel.from_pretrained(cls, pretrained_model_name_or_path, config, cache_dir, ignore_mismatched_sizes, force_download, local_files_only, token, revision, use_safetensors, weights_only, *model_args, **kwargs)
3803 raise EnvironmentError(
3804 f"Error no file named {_add_variant(SAFE_WEIGHTS_NAME, variant)} found in directory"
3805 f" {pretrained_model_name_or_path}."
3806 )
3807 else:
-> 3808 raise EnvironmentError(
3809 f"Error no file named {_add_variant(WEIGHTS_NAME, variant)}, {_add_variant(SAFE_WEIGHTS_NAME, variant)},"
3810 f" {TF2_WEIGHTS_NAME}, {TF_WEIGHTS_NAME + '.index'} or {FLAX_WEIGHTS_NAME} found in directory"
3811 f" {pretrained_model_name_or_path}."
3812 )
3813 elif os.path.isfile(os.path.join(subfolder, pretrained_model_name_or_path)):
3814 archive_file = pretrained_model_name_or_path

Expected behavior

No error!

[Rocketknight1]

@SRGAnalytics-MD it seems like no model files exist in the converted directory - did the conversion script print any errors?

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.