SeFi-Image

Official inference repository for SeFi-Image text-to-image generation.

       

SeFi-Image is a text-to-image model family built with Semantic-First Diffusion. It separates semantic and texture latent streams and denoises semantic structure slightly ahead of texture details, giving the texture stream a cleaner structural anchor during generation.

This repository provides command-line and Python inference for SeFi-Image Base, RL, and Turbo checkpoints. Use inference.py for generation scripts or the sefi package for integration in Python projects.

Semantic-first generation Semantic latents provide a cleaner structural anchor for image synthesis.

Faster training The 5B model reaches strong benchmark performance with about 125K A800 GPU hours.

Generation-reconstruction trade-off A high-fidelity texture latent preserves reconstruction detail while a compact semantic latent simplifies generation.

The checkpoint column is clickable and can be passed directly to --checkpoint or SEFIInferencePipeline.from_pretrained(...). The checkpoint name and sefi_config.yaml are used to infer model family, scale, default sampling steps, and guidance scale.

Run from the repository root so Python can import sefi.

python inference.py \
  --checkpoint SeFi-Image/SeFi-Image-5B-Base \
  --prompt "A red apple on a wooden table." \
  --output-dir outputs/inference/sefi_5b_base \
  --seed 42

Turbo checkpoints use the same interface with fewer denoising steps:

python inference.py \
  --checkpoint SeFi-Image/SeFi-Image-5B-turbo \
  --prompt "A blue ceramic mug on a white desk." \
  --steps 4 \
  --guidance-scale 1.0 \
  --output-dir outputs/inference/sefi_5b_turbo \
  --seed 42

If a checkpoint requires authentication, log in once before running inference:

huggingface-cli login

from sefi import SEFIInferencePipeline

pipe = SEFIInferencePipeline.from_pretrained(
    "SeFi-Image/SeFi-Image-5B-Base",
)

images = pipe(
    "A red apple on a wooden table.",
    seed=42,
)
images[0].save("sefi_5b_base.png")

Turbo checkpoints use the same API:

from sefi import SEFIInferencePipeline

pipe = SEFIInferencePipeline.from_pretrained(
    "SeFi-Image/SeFi-Image-5B-turbo",
)

images = pipe(
    "A blue ceramic mug on a white desk.",
    num_inference_steps=4,
    guidance_scale=1.0,
    seed=123,
)
images[0].save("sample.png")

For Base and RL checkpoints, omit num_inference_steps and guidance_scale to use the checkpoint defaults.

Create an environment and install the runtime dependencies. The PyTorch command below uses CUDA 12.6 wheels; choose the wheel index that matches your machine.

conda create -n sefi-infer python=3.11 -y
conda activate sefi-infer

pip install torch torchvision --index-url https://download.pytorch.org/whl/cu126
pip install diffusers transformers accelerate safetensors huggingface_hub omegaconf pillow

Generate from a text file with one prompt per line:

python inference.py \
  --checkpoint SeFi-Image/SeFi-Image-5B-Base \
  --prompt-file prompts.txt \
  --batch-size 2 \
  --num-images-per-prompt 1 \
  --output-dir outputs/inference/batch

Multi-GPU inference uses accelerate and shards prompts across processes:

accelerate launch --num_processes 8 inference.py \
  --checkpoint SeFi-Image/SeFi-Image-5B-Base \
  --prompt-file prompts.txt \
  --batch-size 1 \
  --output-dir outputs/inference/sefi_5b_multigpu

Images are saved as PNG files. Each rank writes metadata_rank*.jsonl; rank 0 also writes inference_manifest.json.

The figure below summarizes representative SeFi-Image-5B benchmark comparisons from the technical report.

Turbo checkpoints currently support 4, 8, or 10 denoising steps and should run with --guidance-scale 1.0.

Each checkpoint artifact should be self-contained and include sefi_config.yaml at the artifact root. Relative paths inside the config are resolved from that root.

sefi-model/
├── sefi_config.yaml
├── transformer/
│   ├── config.json
│   ├── diffusion_pytorch_model-00001-of-000xx.safetensors
│   └── diffusion_pytorch_model.safetensors.index.json
├── scheduler/
│   └── scheduler_config.json
├── vae/
│   └── ...
└── text_encoder/
    └── ...

The config tells the inference code where to find the final inference VAE, scheduler files, text encoder weights, and DiT transformer weights. Local paths and Hugging Face snapshots use the same layout.

SeFi-Image is released for research use and is not intended for direct product or service deployment. Responsible AI considerations were incorporated during development, including data selection, model training, and evaluation. The training data combines public, licensed, and internally curated sources, with processing intended to remove clearly identifiable personal information and reduce harmful content where possible.

Because web-scale image-text data can contain biases, uneven representation, and imperfect metadata, the model may produce outputs that are inaccurate, biased, inappropriate, misleading, or raise copyright and IP-related concerns under certain prompts. Use the model in controlled research settings with appropriate human oversight. Downstream users are responsible for applying additional safeguards, including content moderation, validation, and compliance checks, before broader use.

If you use SeFi-Image in your work, please cite the project report:

@misc{sefiteam2026sefiimagetexttoimagefoundationmodel,
      title={SeFi-Image: A Text-to-Image Foundation Model with Semantic-First Diffusion}, 
      author={SeFi-Team},
      year={2026},
      eprint={2606.22568},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2606.22568}, 
}

This project is released under the MIT License.