ValueError: Tensor index in the file is empty

[mikob]

I'm trying to serialize the 'runwayml/stable-diffusion-inpainting' model using the hf_serialize.py script in examples/. First, it seems to happen way too fast - in a matter of seconds. How long should we expect serialization to take for a 5GB model? Getting the error below.

$ poetry run scripts/hf_tensorize.py 'runwayml/stable-diffusion-inpainting' tensorized2 --model-type diffusers --validate --force
/home/x/.cache/pypoetry/virtualenvs/x-SHelyAR8-py3.10/lib/python3.10/site-packages/diffusers/utils/outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
  torch.utils._pytree._register_pytree_node(
/home/x/.cache/pypoetry/virtualenvs/x-SHelyAR8-py3.10/lib/python3.10/site-packages/diffusers/utils/outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
  torch.utils._pytree._register_pytree_node(
2024-02-06 13:48:10,240 INFO hf_tensorize.py(113412) - MODEL PATH: runwayml/stable-diffusion-inpainting
2024-02-06 13:48:10,240 INFO hf_tensorize.py(113412) - OUTPUT PREFIX: tensorized2
unet/diffusion_pytorch_model.safetensors not found
Keyword arguments {'use_auth_token': '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'} are not expected by StableDiffusionPipeline and will be ignored.
Cannot initialize model with low cpu memory usage because `accelerate` was not found in the environment. Defaulting to `low_cpu_mem_usage=False`. It is strongly recommended to install `accelerate` for faster and less memory-intense model loading. You can do so with: 

pip install accelerate

.
Loading pipeline components...: 100%|███████████████████████████████████████████████████████████████████████████| 7/7 [00:10<00:00,  1.57s/it]
2024-02-06 13:48:21,749 INFO hf_tensorize.py(113412) - Serializing model
2024-02-06 13:48:21,749 INFO hf_tensorize.py(113412) - GPU: NVIDIA GeForce RTX 3070 Ti Laptop GPU
2024-02-06 13:48:21,898 INFO hf_tensorize.py(113412) - PYTHON USED RAM: CPU: (maxrss: 9,485MiB F: 375MiB) GPU: (U: 167MiB F: 7,806MiB T: 7,973MiB) TORCH: (R: 0MiB/0MiB, A: 0MiB/0MiB)
2024-02-06 13:48:21,899 INFO hf_tensorize.py(113412) - Writing config to tensorized2/encoder-config.json
2024-02-06 13:48:21,908 INFO hf_tensorize.py(113412) - Writing tensors to tensorized2/encoder.tensors
2024-02-06 13:48:21,914 INFO hf_tensorize.py(113412) - Writing config to tensorized2/vae-config.json
2024-02-06 13:48:21,914 INFO hf_tensorize.py(113412) - Writing tensors to tensorized2/vae.tensors
2024-02-06 13:48:21,923 INFO hf_tensorize.py(113412) - Writing config to tensorized2/unet-config.json
2024-02-06 13:48:21,923 INFO hf_tensorize.py(113412) - Writing tensors to tensorized2/unet.tensors
2024-02-06 13:48:22,062 INFO hf_tensorize.py(113412) - Writing tensorized2/tokenizer.zip
2024-02-06 13:48:22,182 INFO hf_tensorize.py(113412) - Writing tensorized2/scheduler.zip
2024-02-06 13:48:23,235 INFO hf_tensorize.py(113412) - Validating serialization
2024-02-06 13:48:23,235 INFO hf_tensorize.py(113412) - Loading tensorized2/vae-config.json
2024-02-06 13:48:23,258 INFO hf_tensorize.py(113412) - Loading tensorized2/vae.tensors, CPU: (maxrss: 9,485MiB F: 6,741MiB) GPU: (U: 5,481MiB F: 2,492MiB T: 7,973MiB) TORCH: (R: 5,314MiB/5,314MiB, A: 5,257MiB/5,257MiB)
Traceback (most recent call last):
  File "/home/x/workspace/x/scripts/hf_tensorize.py", line 579, in <module>
    main()
  File "/home/x/workspace/x/scripts/hf_tensorize.py", line 571, in main
    df_main(args)
  File "/home/x/workspace/x/scripts/hf_tensorize.py", line 414, in df_main
    vae = load_model(
  File "/home/x/workspace/x/scripts/hf_tensorize.py", line 307, in load_model
    with _read_stream(tensors_uri) as tensor_stream, TensorDeserializer(
  File "/home/x/.cache/pypoetry/virtualenvs/x-SHelyAR8-py3.10/lib/python3.10/site-packages/tensorizer/serialization.py", line 1549, in __init__
    raise ValueError("Tensor index in the file is empty")
ValueError: Tensor index in the file is empty

pyproject.toml

[tool.poetry.dependencies]
python = ">=3.10,<3.12"
distro-info = "1.0"
scikit-build = "^0.16.7"
pillow = "^10.2.0"
ninja = "^1.11.1"
diffusers = "^0.15.0"
triton = "^2.2.0"
transformers = "^4.37.2"
pynsq = "^0.9.1"
compel = "^2.0.2"
scipy = "^1.12.0"
torch = "^2.2.0"
xformers = "^0.0.24"
accelerate = "^0.26.1"
deepspeed = "^0.13.1"
tensorizer = "^2.7.2"

[mikob]

same issue with this set of deps too:

python = ">=3.10,<3.12"
tensorizer = "^2.7.2"
diffusers = "^0.26.1"
transformers = "^4.37.2"

[harubaru]

This has been fixed in 2.8.0 which will be released soon. Related PR: #75

[Eta0]

Version v2.8.0 has been released on PyPI with the fix for this issue.