RulzTrainer Training options

Training options

A guide to the options beyond the basics. Every one is on the Train tab (pick its section in the list on the left; each setting is explained beside it) and in the YAML config; the README has the full detail.

Adapter

• LoRA (rank and alpha; rank 16 by default).
• LoKr (network_type: lokr): a Kronecker adapter with one dial, lokr_factor (8 is Fizgig's measured default: its best likeness and more natural skin, about 20% slower steps). Files load in ComfyUI.
• OFT (network_type: oft): rotates each layer's output channels in blocks (oft_block_size, 32) instead of adding to the weights, which keeps the base model's structure. The OFT paper found it held a subject's identity better than LoRA and stayed stable in long runs (on Stable Diffusion; not yet measured here). oft_constraint > 0 caps how far a layer rotates.
• DoRA (dora: true, on a LoRA or a LoKr): splits each layer's weight into a direction, which the adapter changes, and a magnitude per input channel that trains on its own.
• Measured on Krea 2 (0.25 MP, 16 GB card): a step took 3.5 s with LoKr 8, 2.6 s with OFT block 32 (a third of the weights) and 4.9 s with LoKr 8 + DoRA.
• OFT and DoRA files load in ComfyUI exactly as trained (checked against ComfyUI's own adapter code); diffusers cannot read them. Race them against LoKr 8 to see which gives your subject the best likeness.
• Training adapter (training_adapter, LoRA tab): a LoRA that stays on while your LoRA trains and off for samples and probes, and is never saved into your file. Qwen-Image 2.1 uses Fizgig's by default (auto, downloaded): Fizgig measured that Qwen-Image 2.1 LoRAs trained without one collapse into texture or wobble part-way, with nothing in the loss to show it (likeness 77 with it, 56 without). MiniMax H3 uses circlestone-labs' image training adapter by default: musubi-tuner saw one-frame H3 LoRAs trained without one break up within about 50 steps. The other models have none of their own; any LoRA file can be used, at training_adapter_strength.
• Full fine-tune, then extract a LoRA (finetune): trains the model's own weights one window of layers at a time and saves LoRAs extracted from the result (rank 64 by default). It needs a much lower learning rate (1e-5) and long runs, and has no race, data parallel or weight average. Fizgig found an extracted LoRA beats one trained at the same rank; not yet measured here. Its weights are held near the original model's (finetune_anchor, 0.01: after each step they move back a little, the bounded drift of Preserve and Personalize, ICLR 2026) instead of decaying toward zero.

Likeness-filtered self-training

selftrain renders selftrain_images (32) pictures with the LoRA at selftrain_at (0.5) of the steps, keeps the clean ones that score above selftrain_min_likeness (60) and the probes' smoothed score, up to selftrain_share (0.2) of the photos, and trains on them at selftrain_weight (0.5) in place of the last round's. Prompts: selftrain_prompts (empty: varied ones from the trigger word). Needs face or body references; not with race_parallel. See Character LoRAs.

copy_check (auto: on with self-training; on: also logged on every probe; off) compares pictures with the training photos using the SSCD copy detector. Self-training leaves out a picture above copy_max (0.5, or the photos' own typical match among themselves + 0.1 when higher) as "copies a photo"; each probe logs its closest photo and likeness.jsonl keeps copy, copy_of and copy_baseline.

probe_async (off by default; WebUI: Render probes on the second GPU) renders the likeness probes on the idle second GPU (probe_device, default the first of devices that is not the training GPU) in a separate process with its own copy of the model, while training goes on; each probe's decisions land when it is scored, up to probe_every steps later, and a best checkpoint still saves exactly the weights that were scored. The final probe renders on the training GPU. Measured on Qwen 2.1 (training on a 5060 Ti, probes on a 4060 Ti): 10% faster with small probes every 20 steps, 24% faster with 3 seeds at 1024 px every 80 steps; slower when a probe takes longer on the second GPU than probe_every steps take. Needs multi_gpu: off; not used in a race or a fine-tune.

Seed soup

soup_files (2+ LoRA / LoKr files, with steps: 0) averages their weight changes into one LoRA instead of training, picked greedily by the likeness probes at soup_strengths (1.0, 1.15, 1.3), saved at soup_rank (32; 0 = exact) as <job>_soup.safetensors. scripts/soup.py makes the plain average without the model. See Character LoRAs.

Second stage: likeness reward fine-tuning

reward_steps > 0 runs an online reinforcement-learning stage after the steps: groups of the LoRA's own pictures are rated by the likeness scorers and the better ones made more likely (Flow-GRPO). Settings: images per step (4), KL penalty (0.04), learning rate (2e-5), trained denoising steps (4), sampler noise (0.7), size and prompts. The probes keep running every probe_every reward steps and once more as the stage ends, so the final LoRA's likeness is always scored. Each picture's face and body scores are capped far below a stranger's, so a body that differs from the body references only in clothing or framing (clothed prompts, unclothed references) can't outweigh the face; reward prompts like the body references, or body weight 0, let the body count. See Character LoRAs.

Training-only losses inside the model

• Trigger attention on the subject (attn_localize, weight 0.05): in a few attention layers, the share of each picture region's attention to the caption that goes to the trigger word; the loss is the part of it off the person. Needs a trigger word; on FIBO 2.0 it is read in the injection blocks' cross-attention.
• Identity alignment (id_align, weight 0.05): the face tokens after one block, averaged and mapped by a small throwaway network onto the photo's ArcFace embedding (REPA-style representation alignment, aimed at identity).
• Both reuse the region maps and face analysis, change nothing in the saved file and cost about nothing in step time on Krea 2. They are race keys; the parallel race and a fine-tune run without them.

Noise levels (timestep sampling)

• logit_normal (the default), sigmoid, uniform, and the model families' own reference recipe.
• identity, content: more low-noise steps, where faces and detail form. style and logsnr: more very noisy steps, where an art style is learned (logsnr is the style-friendly sampler from a paper; for style LoRAs).
• Clean-end share (clean_share): sets what share of steps land in the clean half, and can change by epoch. Fizgig's Slide for MOSS-Image is 60:7:30:8:3: 60% until epoch 7, 30% at epoch 8, then 60% with every 3rd epoch at 30%. In Fizgig's test it held likeness about 5 points higher than the older 60:7:30:15:2. The fifth number means: after the slide, train at the higher share, with every Nth epoch at the lower one, at half the learning rate (those epochs reset the picture's structure without costing likeness). A full fine-tune counts rotations instead of epochs; its Slide FT is 11:4:30:8:2. The Slide and Slide FT buttons fill them in. Since 2.24.0, an older 5-number value follows these rules too.
• Timestep buckets (timestep_buckets, from musubi-tuner, since 2.26.0; 0 = off): with N buckets the noise levels are cut into N equal slices, and every N steps draw one level from each slice in shuffled order, so no slice is skipped by chance (10 free draws cover about 6.5 of 10 slices; bucketed, at least 9). The mix of noise levels is the same as without buckets, with every sampler, shift and clean-end share. A training state remembers the order, so a resume continues it. The reference sampler is not bucketed. It is a race key. Under Advanced > Timesteps, loss weighting and gradients on the Train tab.

Loss type (loss_type)

• mse (the default): the plain loss.
• band (experimental, from AI-Toolkit, since 2.25.0): splits each error into fine-to-coarse detail bands and drops the bands the noise has buried at that step, so very noisy steps stop pushing on details nobody could see. With almost no noise it is the same as mse; coarse shapes still train at high noise.
• Band's loss value runs lower than mse's (most at high noise), so the loss chart can't compare the two: judge by samples and likeness probes. It is a race key, so race band against mse on your own data. Under Advanced > Timesteps, loss weighting and gradients on the Train tab; it works with region weights and loss weighting.

Saving

• Save every N steps writes a checkpoint; Checkpoints to keep deletes the oldest beyond the newest N (0 keeps all). The final file, the best-likeness file and a race's dropped files are never deleted.
• Training states (save_state) let you resume exactly where a run stopped.
• Weight average (lora_ema) makes checkpoints and samples use a smoothed copy of the weights.

Post-hoc EMA

Since 2.26.0 (from musubi-tuner): a smoothed average made after the run from the checkpoints it saved, so you can try several blends without training again. The section is on the Train tab, after Seed soup.

• Use this job's checkpoints fills in the job's step checkpoints; any LoRA or LoKr files of one run work. They are put in step order.
• Power (EDM2) (the default) weights each checkpoint by how far into the run it was saved. sigma_rel sets how wide the average is: 0.05 to 0.2, smaller leans on the last checkpoints. At 0.15, four checkpoints give the newest about 69%.
• Decay per checkpoint is musubi-tuner's recipe: the oldest file starts as the whole average and each later one is mixed in at 1 - Decay. With few checkpoints a high decay leans on the oldest (0.9 over four puts 73% there), so the default here is 0.5. End decay slides the decay to another value over the files.
• Average now saves <job>_posthoc_ema_<settings>.safetensors in the job folder and says each file's weight and about which step the average sits at. Rank sets the result's rank (empty: the checkpoints'; 0: exact).
• scripts/posthoc_ema.py does the same from the command line (its --beta defaults to musubi-tuner's 0.95).

The Wizard's recommendations

Besides memory and speed settings, the Wizard starts a character from rank/alpha 32, AdamW 8-bit and a learning rate of 1e-4, and recommends: a learning rate of 1e-5 with adafactor for a full fine-tune (and no rate for automagic3, automagic4, keel or keel_fb, which set their own); LoRA for a character (it reached likeness sooner than LoKr in RulzTrainer's tests); no rank or alpha for LoKr or a fine-tune; the logsnr noise mode for a style; and a step count of about 100 epochs, counting repeats and crop images. A fine-tune or block swap is recommended one GPU, since neither runs data parallel.

Samples

Samples and probes use each model's own turbo LoRA by default so they render in 4 to 12 steps, at an aspect ratio and a resolution in megapixels that you choose. Turn the turbo LoRA off in the Samples settings to sample the plain model with its full steps and guidance. When you train on a Turbo model (Krea 2 Turbo, Qwen-Image 2.1 Turbo) or sample with one, no turbo LoRA is added: the model is already distilled, and a second distill would make the previews differ from how the LoRA is used.

A Sampling LoRA (sample_lora, since 2.27.0) is added to previews only; it is never trained or saved. On auto (the default), MOSS-Image Base previews get Kijai's SFT LoRA (Kijai/MOSS-image_experimental, moss_image_sft_lora_blocks_21_to_49_rank_512_bf16.safetensors, 1.16 GB): it is SFT minus Base, so at strength 1 the samples show how the LoRA looks on SFT. It is not added on SFT, when you sample with a separate model, on other models, or for a full fine-tune. Choose None to preview plain Base, or A LoRA file for another one (Kijai's full rank-512 version covers every block); Sampling LoRA strength sets its weight.

Each model's sampling files belong to it: when you switch models, the WebUI clears a sampling model, turbo LoRA, sampling LoRA or training adapter whose file name names another model, and Start refuses a run that still has one.

Learning rate and optimizers

AdamW, AdamW 8-bit, Adafactor, Automagic 3, Automagic 4, Keel and Keel (face + body). The two Automagics set their own rate from the weights' update directions (the rate you enter is the start; Automagic lr groups by_type gives the text-side, attention, feed-forward and other layers their own rate). Automagic 4 also takes a smaller step for the most sensitive weights (Sensitivity quantile) and holds its rate on a noisy loss instead of drifting up: ai-toolkit's v4 raised the rate 1.6% a step on gradient noise (doubling every 44 to 100 steps) until the LoRA turned to noise, which RulzTrainer's port fixes (Vote graded, Max lr change 5% a step). On a rank-16 LoRA its step is as fast as AdamW 8-bit's (about 3x slower at rank 64; Automagic 3's is about 10x slower than Automagic 4's), its state 1 byte per weight at the default Polarity history 8, and a full fine-tune's step no longer needs gigabytes of scratch memory. LoRA+ trains the up matrices faster (AdamW-style optimizers only). The likeness controller can hill-climb a rate multiplier for you. See also the per-image loss watch, which flags images that stop teaching the model and can throttle or exclude them. Which optimizer reaches likeness first is not measured yet: race them.

Keel (optimizer: keel, v2.22.0) is Automagic 4 built for character LoRAs: the same rate, groups and state, plus mechanisms aimed at keeping the identity. A gauge preconditioner scales each LoRA pair's two matrices so that neither takes the whole step; a cautious mask drops the elements where that preconditioned step disagrees with the raw gradient; noise bands give the quiet noise levels, where the face forms, steps as large as the loud ones (Band weights identity favours the middle levels); a consensus gate scales each element by how many of the step's photos agree on its direction (it needs a gradient accumulation of 2 or more); a probe-gated lookahead (off by default) anchors the weights at each probe that sets a new best likeness and, after a few probes without a gain, pulls the weights part of the way back toward the anchor and cuts the rate; and a face-token gain (off by default) moves the weights that serve the face's tokens more than the background's (it needs face detection). With every switch off it is Automagic 4 to the bit, and a Keel state resumes under Automagic 4 and the other way round. Every Keel setting is a race key. Whether it reaches likeness before Automagic 4 or AdamW 8-bit is not measured yet, so the Wizard still recommends AdamW 8-bit: race them. The design, the measurements to run and every option are in the README's Keel section and docs/keel-optimizer.md.

Keel (face + body) (optimizer: keel_fb, v2.23.0) is the same optimizer, one class, with the body treated as the face's equal. Two mechanisms differ. The consensus vote weighs each photo's gradient by its size (Consensus vote magnitude; auto picks it for keel_fb and the equal-vote sign count for keel), so a full-body photo outweighs a face close-up on the body's weights instead of each photo getting one vote on weights it knows nothing about. The token gain adds a body share next to the face's (Body weight, default 1): the person's tokens outside the face box, each region's share measured against its own typical value because the body covers far more tokens than the face, the two combined as a weighted geometric mean, and a step with no body (or no face) moving the gain by the other alone. It also brings the Band weights preset subject, a plateau over the middle noise levels, where the build is decided, instead of identity's drop at the loud ones. With the default settings the two differ by the vote alone, since the gain is off in both: set Face gain to 0.5 or 1 to bring the body share in. The body share needs person masks (SAM 3 or MediaPipe), so region maps cached before they were needed give a one-time note in the log and the face share alone until they are rebuilt. The vote costs 4 bytes per weight during the step instead of 1. The probes' combined likeness score that the lookahead and the best checkpoint follow is still weighted toward the face, so set the probes' face and body weights equal when you race it. Unmeasured, like Keel: race keel against keel_fb.

Hull (optimizer: hull, base rule only, v2.36.0) is an optimizer written from scratch, planned for body identity: build, proportions, structure and skin detail, with the face no worse. Only the base rule is built: it takes evenly sized steps on each layer, measured as a change in the layer's weights, however the LoRA's two matrices happen to be scaled. None of the body mechanisms exist yet, it has not trained a real model or been raced, and nothing shows it beats AdamW. Treat it as an experiment.

Hull step (hull_rho, on the Train tab under Optimization; the wizard does not offer Hull) is the size of a step, as a change relative to each layer's frozen weight. It has no default, and no recommended value yet, so you have to set one. The learning-rate schedule acts on it, and Learning rate and Max grad norm are not what sets its step. Hull trains plain LoRAs only: LoKr, OFT, DoRA and fine-tuning are refused. So is anything that could raise its step or train on a judge: the likeness learning-rate controller, the curriculum, a loss watch multiplier above 1, a FaceNet identity loss, reward fine-tuning and self-training. In a race it needs the gold face scorer on FaceNet and the copy check on. A step costs about 100 to 150 ms more than fused AdamW, around 6% of a typical training step.

docs/hull-optimizer.md describes the whole plan: the update rule, the body mechanisms, the judges, how it would be validated on synthetic characters and consenting volunteers, and the order it would be built and raced in, each part staying only if a race shows it helps.

Several datasets and resolutions

Train at several resolutions at once (the images are bucketed by shape), or step up from low to high resolution as the likeness plateaus. Each dataset can have its own resolutions and repeats.

Buckets keep the pixel count of the selected resolution (like ai-toolkit): 0.25 MP is ~512x512 of area, 4 MP ~2048x2048, whatever the aspect ratio. min_bucket_side / max_bucket_side only bite on extreme aspect ratios; the minimum is capped at half the resolution and the maximum raised to twice it.