gguf2safetensors: GGUF → HF Safetensors Converter

Author: Emmanuel Forgues • Year: 2025 • License: MIT

Convert a quantized Llama-compatible GGUF model into a standard Hugging Face (HF) folder with .safetensors shards. This allows using the model with HF pipelines, libraries, and tools (e.g., ComfyUI Transformers/LLM node).

The provided script upcasts GGUF weights to PyTorch tensors in memory and saves them as HF checkpoints. It disables torchvision import inside Transformers to avoid unnecessary dependency issues.

Note: Python 3.13 wheels are still incomplete for several libraries. Python 3.11 is recommended.

Contents

• convert.py — conversion script using Transformers to load from GGUF and save to HF
• FileIdol-...-fp32-hf/ — example output folder with safetensors shards and tokenizer files
• FileIdol-...IQ2_M.gguf — example input model file

What this does (need and approach)

• Need: Load a quantized GGUF model and produce a standard HF checkpoint (.safetensors) for broader compatibility.
• Approach: Use Transformers' AutoTokenizer/AutoModelForCausalLM with gguf_file to load the tokenizer and model, upcast weights to a desired dtype (default FP32), and save_pretrained() into an HF folder with sharded .safetensors.
• Extras: Torchvision import is disabled via TRANSFORMERS_NO_TORCHVISION=1 to avoid pulling the optional vision stack; device_map="cpu" ensures a stable, CPU-only conversion.

Requirements

• Windows with PowerShell
• Python 3.11 (recommended)
• Packages:
  • transformers >= 4.48
  • gguf
  • safetensors
  • accelerate
  • tokenizers
  • sentencepiece
  • torch (CPU build is sufficient for conversion)

Setup (PowerShell)

Create and activate a virtual environment, then install dependencies:

py -3.11 -m venv D:\Gguf2Safetensors\venv
D:\Gguf2Safetensors\venv\Scripts\Activate.ps1
pip install -U "transformers>=4.48" gguf safetensors accelerate tokenizers sentencepiece
pip install --index-url https://download.pytorch.org/whl/cpu torch

Usage

1. Place your .gguf file in this folder (or adjust the path in convert.py).
2. Edit the GGUF and OUT constants in convert.py if needed.
3. Run the converter:

python .\convert.py

Expected output

The script will create an HF-style directory (OUT) containing:

• model-00001-of-000NN.safetensors (sharded weights)
• model.safetensors.index.json
• tokenizer.json / tokenizer.model / tokenizer_config.json / special_tokens_map.json
• config.json / generation_config.json
• chat_template.jinja (if present in the GGUF)

By default, shards use max_shard_size="4GB" to keep each .safetensors file manageable.

Choosing dtype and disk footprint

• Default: FP32 (highest precision, largest disk/RAM usage)
• Optional: Save directly in BF16 to reduce size and memory pressure:

# After loading the model in convert.py
model.to(dtype=torch.bfloat16)
OUT = r"D:\\Gguf2Safetensors\\<your-model-name>-bf16-hf"
model.save_pretrained(OUT, safe_serialization=True, max_shard_size="4GB")

Tokenizer is shared; you can reuse the same saved tokenizer folder.

Memory notes

• A 8B model in FP32 can peak around ~30 GB RAM during conversion. Close heavy apps and ensure a sufficiently large pagefile if needed.
• BF16 significantly reduces memory and disk usage.

Troubleshooting

• Transformers tries to import torchvision
  • Ensure torchvision is not installed: pip show torchvision should show nothing.
  • You can also force-disable it for the process:

$env:TRANSFORMERS_NO_TORCHVISION = "1"
python .\convert.py

Chargement du tokenizer... Chargement du modèle (IQ2_M → upcast FP32 en mémoire)... Converting and de-quantizing GGUF tensors...: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 292/292 [03:51<00:00, 1.26it/s] Sauvegarde HF (.safetensors) vers : D:\Gguf2Safetensors\file-Llama-3.1-8B-Instruct-1.2-fp32-hf Conversion terminée ✅ (venv) PS D:\Gguf2Safetensors>

result : Mode LastWriteTime Length Name

---

-a---- 03/09/2025 19:33 575 chat_template.jinja -a---- 03/09/2025 19:36 831 config.json -a---- 03/09/2025 19:36 154 generation_config.json -a---- 03/09/2025 19:38 3946908152 model-00001-of-00009.safetensors -a---- 03/09/2025 19:40 3791786264 model-00002-of-00009.safetensors -a---- 03/09/2025 19:42 3959591416 model-00003-of-00009.safetensors -a---- 03/09/2025 19:44 3892449944 model-00004-of-00009.safetensors -a---- 03/09/2025 19:46 3959591456 model-00005-of-00009.safetensors -a---- 03/09/2025 19:49 3892449944 model-00006-of-00009.safetensors -a---- 03/09/2025 19:56 3959591456 model-00007-of-00009.safetensors -a---- 03/09/2025 20:01 2617363608 model-00008-of-00009.safetensors -a---- 03/09/2025 20:04 2101346432 model-00009-of-00009.safetensors -a---- 03/09/2025 20:04 24285 model.safetensors.index.json -a---- 03/09/2025 19:33 79 special_tokens_map.json -a---- 03/09/2025 19:36 17221647 tokenizer.json -a---- 03/09/2025 19:36 2948282048 tokenizer.model -a---- 03/09/2025 19:33 52804 tokenizer_config.json

• Progress stalls at "Converting and de-quantizing GGUF tensors"
  • This can take several minutes depending on CPU and disk. Let it complete; large models have hundreds of tensors.

ComfyUI

In the Transformers/LLM node, point to the output HF folder (OUT). If you saved in BF16, ensure the config has "torch_dtype": "bfloat16" so it will be picked up by default.

License

MIT License. See LICENSE for details. You are responsible for the licenses and usage terms of any models you convert.