RulzTrainer Models and memory

Models and memory

Where the models come from

By default (model_source: files) the model you pick is downloaded as ComfyUI's single safetensors files from Comfy-Org's Hugging Face repackages on the first run, and reused after that. None of those repos is gated; FLUX 3 Klein's come from Black Forest Labs' own repo, which is (accept the license on the model page once). FLUX.2 Klein 9B's diffusion model is gated the same way, while its text encoder and VAE are not. You choose the format of the diffusion model and the text encoder separately; int8 ConvRot is the default (about half of bf16, trains as fast, under 1% error).

Model	Repo	Diffusion model	Text encoder
MOSS-Image Base / SFT	Comfy-Org/MOSS-Image	int8 5.4 GB, bf16 10.2 GB	Qwen3.5-9B int8 9.8 GB, bf16 19.3 GB
Krea 2 Raw / Turbo	Comfy-Org/Krea-2	int8 13.5 GB, fp8 13.1 GB, bf16 26.3 GB	Qwen3-VL-4B int8 4.9 GB, bf16 8.9 GB
FIBO 2.0 dev / Edit Turbo	Comfy-Org/Fibo-2.0	int8 8.4 GB, bf16 16.5 GB	Qwen3-VL-4B (Krea 2's file)
Qwen-Image 2.1 / Turbo	Comfy-Org/Qwen-Image-2.1	int8 7.3 GB, bf16 14.2 GB	Qwen3-VL-8B int8 9.4 GB, bf16 17.5 GB
FLUX 3 Klein Base / distilled	black-forest-labs/flux-3-image-klein-comfy (gated)	int8 5.8 GB, nvfp4 5.3 GB, bf16 11.1 GB	Qwen3-VL-4B int8 4.9 GB
FLUX.2 Klein 9B Base / distilled	black-forest-labs/FLUX.2-klein-base-9b-fp8, FLUX.2-klein-9b-fp8 (gated); text encoder from Comfy-Org/Flux-2-Klein-9B	fp8 8.9 GB, bf16 16.9 GB (no int8 release; auto quantizes on load: nvfp4 below 18 GB, int8 from 18 GB, bf16 from 30 GB)	Qwen3-8B fp8 or bf16
Ideogram 4	Comfy-Org/Ideogram-4	int8 9.6 GB, fp8 9.3 GB, nvfp4 5.5 GB (no bf16 release)	Qwen3-VL-8B (Qwen-Image 2.1's file)
MiniMax H3 FL2VA	Comfy-Org/MiniMax-H3	pruned int8 21.0 GB, pruned bf16 40.2 GB	Qwen3-VL-32B int8 27.1 GB, NVFP4 15.7 GB

Already have the files? Point the optional file fields (transformer_file, text_encoder_file, vae_file) at them; only the empty ones are downloaded. The original diffusers repos work too (model_source: hf).

Fitting a 16 GB card

From the cheapest change to the most involved:

1. Quantize. int8 ConvRot halves the model's memory; nvfp4 goes further and uses native FP4 on RTX 50-series cards. transformer_quant: auto keeps a quantized file's format where it fits.
2. Gradient checkpointing (on by default) recomputes activations instead of storing them. checkpoint_keep_blocks: -1 measures your card and keeps as many of the last blocks' activations as fit, which makes steps faster.
3. Batch size 1 with gradient accumulation for a bigger effective batch.
4. Block swap keeps the frozen weights of some blocks in system RAM and streams them to the GPU as they run. On Krea 2 int8, swapping 14 of 28 blocks cut the training step's peak from 10.8 to 7.7 GB at about the same speed. It needs gradient checkpointing and a single GPU, and not a fine-tune. The Wizard's Training page (Advanced) recommends a block count when the model plus about 4.5 GB for a step does not fit your card.
5. Lower the training resolution (0.25 to 1 megapixel).

Two GPUs

Mode	What it does
Data parallel	A model copy per GPU, each trains its own images, gradients are averaged. About 1.7 to 2x throughput. Each GPU must fit the whole model.
Model split	The blocks are divided across GPUs: half the memory each, not faster.

MiniMax H3 does not fit one 16 GB card whole: use model split on two cards (about 10.5 GB of weights each), or block swap on one card with about 25 of its 50 blocks in system RAM (64 GB of RAM advised). Both are estimates, not yet measured.

Samples and probes are rendered on both GPUs at once (parallel_jobs). If the two cards differ in speed, dp_balance gives the faster one more batches per step.

Speed

• torch.compile (compile_model: auto) pays off on runs of 100+ steps.
• Text caching (since 2.25.0) encodes up to 16 captions per pass of the text encoder instead of one at a time (fewer when captions are long), so the first run on a dataset, or a recache, prepares faster. Each caption gets exactly the features it would get alone. If a batch runs out of graphics memory it is split and retried by itself.
• cuDNN attention is picked automatically on RTX 40 (Ada) and Blackwell cards (5-9% faster per step on the RTX 4060 Ti, 15-22% on the RTX 5060 Ti).
• Compile cache (compile_cache: auto): compiled kernels are saved to models/.rulz_compile, one file per torch, triton, GPU model, Python and operating system. Put a file a vast.ai rental made (it ends in -linux.bin) in your model bucket's .rulz_compile folder and a new rental of the same GPU skips most of the first compile (Qwen-Image 2.1 to step 5 on Linux: 52-58 -> 30-37 s; on Windows 82 -> 32 s). A file from a Windows PC does not help on Linux.
• Native FP4 / W8A8 int8 matmuls are used where the card supports them.

Details and measurements for every item: the README sections Model sources and quantization, Using two GPUs, VRAM guidance and Block swap.