safetensors

[yst-one]

Add safetensors to support transformers, enabling more convenient tokenization, encoding and decoding during ONNX inference.

https://pypi.org/project/safetensors/

[ndonkoHenri]

safetensors 0.8.0 is now supported on Android and iOS: https://pypi.flet.dev/safetensors

safetensors loads and saves model weights/embeddings as a safe, zero-copy file format — none of pickle's arbitrary-code-execution risk, and fast mmap reads that pull one tensor at a time.

Install

safetensors itself has no required dependencies, but the usable mobile path needs numpy, so add both:

dependencies = [
    "flet",
    "safetensors",
    "numpy",
]

Example

Save a couple of tensors, reload them via safe_open, and show the result on screen. Note the file goes into Flet's storage: the app bundle is read-only on device, so write to FLET_APP_STORAGE_TEMP/_DATA, not the working directory.

import os
import flet as ft
import numpy as np
from safetensors.numpy import save_file
from safetensors import safe_open


def main(page: ft.Page):
    path = os.path.join(os.environ.get("FLET_APP_STORAGE_TEMP", "."), "w.safetensors")
    save_file(
        {
            "emb": np.arange(6, dtype=np.float32).reshape(2, 3),
            "bias": np.ones(3, dtype=np.float32),
        },
        path,
    )

    rows = []
    with safe_open(path, framework="numpy") as f:  # mmap; one tensor at a time
        for k in f.keys():
            t = f.get_tensor(k)
            rows.append(ft.Text(f"{k}: shape {list(t.shape)}, sum {t.sum():.1f}"))

    page.add(*rows)


ft.run(main)

On device this shows emb: shape [2, 3], sum 15.0 / bias: shape [3], sum 3.0.

Recommendations

• On mobile the usable framework backend is numpy — torch / tensorflow / paddle aren't available as mobile wheels. Loading with framework="pt" fails at runtime because torch isn't installed; convert weights to the numpy view (or ONNX) ahead of time.
• Prefer safe_open + get_tensor over load_file for large files — it mmaps and reads tensors on demand instead of pulling everything into RAM at once.
• .safetensors is the weights/embeddings format for the on-device ML stack — pair it with onnxruntime + numpy for inference, or model2vec for static embeddings.