v1.5.1

LoRA Forge 1.5.1

sd-scripts v0.12.2

• LoRA Forge is now tested with sd-scripts v0.12.2, and new installs clone that version. Existing installs keep their sd-scripts until you update it: .\run_gui.ps1 update-sd-scripts --branch v0.12.2 (Linux: bash run_gui.sh update-sd-scripts --branch v0.12.2). Until then, System info shows the installed sd-scripts as outdated.
• Short Anima and SDXL runs on v0.12.2 match v0.12.0: same loss, compiled graphs and speed.

Block swap

• The Anima and FLUX pages offer sd-scripts' new block_swap_h2d_only and use_pinned_memory_for_block_swap, shown under blocks_to_swap once blocks are swapped. Both are off by default: on Anima, H2D-only was about 7% faster with 26 of 28 blocks swapped and made no difference with fewer.
• With an sd-scripts older than v0.12.2, a run using them is refused with a message instead of the options being ignored silently.

MOSS-Image

• The text encoder now runs fully in bf16, like ComfyUI and OpenMOSS's own pipeline. Before, two kinds of its weights stayed in fp32, so the captions were encoded about 1.6% differently from what the LoRA gets at inference. Cached captions are encoded again once on the next run; cached images are kept.

v1.5.0

LoRA Forge 1.5.0

A new trainer page

• Options are rows: the name and description on the left, the control on the right. Sub-options slide in under the option that turns them on, so the option you switch never moves.
• Ten sections (Model, Dataset, Captions, Training, Timesteps and loss, Network, Saving and logging, Preview images, Precision and memory, Speed). Each shows the common options first and "+ more" for the rest; the Essentials / All options switch is gone, and search still covers every option.
• Hints next to the values: steps per epoch and in all, warmup steps, LoRA scale, megapixels, the model file checks and the dataset's folders. "Aim for steps" buttons set the epochs, and size buttons set the resolution.
• Problems show before you start: missing required files, a dataset folder without repeats_name subfolders, warmup values sd-scripts would misread, optimizers that cannot train with full-fp16 weights, shuffled or dropped captions with cached text encoder outputs, a fixed resolution the model cannot use. Start waits for the ones that would fail.
• When an option changes others (ProdigyPlus, Chroma, caption shuffling), the page says what it changed and offers Undo.
• A summary on the right: the run's steps, a time estimate from your last run of the same model, the open problems and every changed option.
• The sidebar and header follow the same design. A new Size setting (Auto / 100% / 125% / 150%) scales the whole UI; Auto makes it larger on 4K-wide windows.

Defaults and options

• Anima: attn_mode defaults to auto, which uses flash-attn when it is installed and can run.
• unet_lr and text_encoder_lr are empty by default, so learning_rate takes effect; cosine replaces a one-cycle cosine_with_restarts.
• FLUX: network_dim 16, flux_shift timestep sampling, previews at 1024 px without CFG (FLUX.1 dev is guidance-distilled), Chroma's values when it is chosen, and blocks_to_swap.
• lr_warmup_steps and the new lr_decay_steps (for warmup_stable_decay) take a number of steps or a fraction below 1, as sd-scripts and musubi-tuner read them.
• resolution accepts one number for a square and more ways of typing it; its description now explains what it does with and without buckets.
• MOSS-Image: caption dropout (off by default) and a cosine schedule.
• With two or more GPUs, the GPU picker is shown by default.

v1.4.0

LoRA-Forge 1.4.0

MOSS-Image previews fixed, attention-only LoRAs, and clearer system and VRAM information.

Fixed

• MOSS-Image previews: runs started from the web UI made no preview images in 1.3.0. They do now.

New

• lora_targets on the MOSS-Image page: attention trains only the attention layers (33M instead of 74M parameters at rank 32) and leaves the rest to the base model, which keeps its rendering style. In our tests it overtrained far more slowly than training all layers. Opt-in; the default is unchanged.
• VRAM spill notice (Windows): when a training needs more VRAM than the card has, Windows moves part of it to system RAM instead of failing, and every step gets much slower. The Runs page now says when this happens, how much moved, and what to change.
• GPU check in System info: each GPU with its VRAM, and a warning when a card is too old for this PyTorch build (GTX 10 series and older) or lacks bf16 and flash-attn (RTX 20 series).
• Versions in System info: LoRA Forge itself and musubi-tuner, next to sd-scripts, whose row now shows "(tested)" for the tested version; an outdated sd-scripts gets a warning with the update command.

Docs

• The README is rewritten for users, with screenshots, VRAM and GPU requirements, and a clearer Speed section. New logo.
• docs/moss-image.md (also in Chinese): what our first character trainings showed. Name everything that varies in every caption line; judge a LoRA on SFT rather than on the previews (they render with Base); the best results came around 500-600 steps at the default learning rate; and a descriptive prompt with the LoRA at 0.5-0.7 gave the cleanest images.

v1.3.0

LoRA-Forge 1.3.0

MOSS-Image training, a new home page, and help with model downloads.

New

• MOSS-Image LoRA page: OpenMOSS's new 5B model with strong text rendering and layout control, trained with LoRA Forge's own trainer (no upstream trainer supports it yet). It follows the MOSS-Image technical report and fits 16 GB cards; previews use OpenMOSS's sampler, and the LoRAs load in ComfyUI on both Base and SFT. Each line of a caption file is one caption of its image (for example a full description plus a short line with the name). See docs/moss-image.md.
• training_precision = fp16 on MOSS-Image: fp16 matrix multiplies with fp16 accumulation, about a quarter faster per step on GeForce RTX cards, with gradients as close to fp32 as bf16's. Opt-in.
• Model files panel on the Anima, Krea 2 and MOSS-Image pages: a download link for each file (through hf-mirror with --cn) and a live check of the path you enter: missing, quantized, Turbo, or a different model type. Fine-tunes and merges of the same model pass.
• Guide links next to the title of the Anima, Krea 2 and MOSS-Image pages.
• New home page: the main models as cards, the others in lists, your recent runs with their status, a quick start before your first run, and the live training status in the header.

Faster

• MOSS-Image: 2.93 s → 2.10 s per step with flash-attn and per-block compile (on by default), 1.54 s with fp16. Measured on an RTX 5070 Ti at 1 megapixel, rank 32.

Docs

• docs/moss-image.md (also in Chinese): files, captions, settings and the ComfyUI workflow. docs/moss-image-trainer.md: how the trainer works.

v1.2.0

LoRA-Forge 1.2.0

Krea 2 training, and much faster SDXL and Anima steps.

New

• Krea 2 LoRA page, trained with kohya-ss/musubi-tuner, which LoRA Forge now installs next to sd-scripts. It fits on 16 GB cards: int8 base weights (ComfyUI's int8_convrot file loads directly), and pinned, one-way block swap that costs almost no speed (a training that would not fit in free RAM is refused with a message). flash-attn is used automatically when installed. Preview images can use Krea 2 Turbo (the Turbo LoRA from Comfy-Org) for 8-step samples. See docs/krea2.md.
• training_precision on the Anima, SDXL and SD1.5 pages replaces mixed_precision + full_precision: one choice that says what runs in which precision (math and trained weights). The fp16 choices come with fp16_accumulation: on GeForce RTX cards it makes an Anima step about a quarter faster and an SDXL step about 8% faster, with gradients as close to fp32 as bf16's. Opt-in; the defaults train as before (the Dreambooth page keeps its fp16 default, without fp16_accumulation). Old configs import into the matching choice.
• SDXL compile (SDXL LoRA page, on by default): each transformer and resnet block is compiled, with one graph for all bucket sizes. Not used with LyCORIS and DyLoRA, which fail to compile.
• A Speed section in the README lists what LoRA Forge speeds up for each model.

Faster

Measured on an RTX 5070 Ti with rank-32 LoRAs at batch 2; every change trains with the same losses as before.

• SDXL LoRA: 1.51 s → 0.81 s per step (0.72 s with fp16), from per-block compile (−21%), flash-attn for the attention when it is installed (−17%; PyTorch's Windows builds have no flash kernels) and the faster ProdigyPlus step.
• ProdigyPlusScheduleFree updates all tensors of the same shape together instead of one by one, with the same math and the same saved state: −7% per step on Anima, −23% on SDXL. Its options use_bias_correction, use_adopt, use_cautious, use_grams, use_orthograd, use_focus and use_speed keep the original step.
• Anima activation_memory_budget now also keeps flash attention's outputs instead of always recomputing them (about 3% faster per step, slightly less memory).
• LORA_FORGE_PATCHES=0 runs sd-scripts without any of LoRA Forge's patches, for comparisons.

Changes

• SDPA attention is on by default on the SDXL, SDXL fine-tune, SD1.5 and Dreambooth pages. Before, both sdpa and xformers were off, so sd-scripts used its plain attention: about half as fast, and an SDXL LoRA at batch 2 did not fit in 16 GB. Saved configs keep their own value.
• save_precision defaults to fp16 on the Anima and Krea 2 pages: it keeps more detail of the LoRA weights than bf16 at the same size.
• SDXL with the fp16 training precisions always keeps the VAE in fp32 (no_half_vae): the SDXL VAE gives NaN in fp16.
• full-fp16 / full_fp16 (fp16 trained weights) refuses the optimizers that break with it (AdamW, RAdamScheduleFree, CAME, Prodigy, ProdigyPlus, DAdaptation) and warns for Lion and SGDNesterov; AdamW8bit, PagedAdamW8bit, Lion8bit, PagedLion8bit and AdaFactor work.

Fixes

• Krea 2: edited captions are re-encoded when caching; before, an existing cache was kept with the old caption.

v1.1.0

LoRA-Forge 1.1.0

Faster Anima training with spare VRAM, charts you can filter, and forms that stay filled in.

New

• Anima: activation_memory_budget (Memory section, needs compile): a faster replacement for gradient checkpointing that uses spare VRAM. Each compiled block keeps part of its activations for the backward pass instead of running its whole forward again. auto picks the amount from the free VRAM when training starts and keeps gradient checkpointing when too little is free; in our tests it was 10-20% faster per step with the same losses. Off by default. See docs/anima.md.
• Chart lines can be hidden: click a legend entry on the Runs page to hide or show its line, and the axis fits the lines still shown. The raw values under a smoothed line have their own entry. The choice is remembered in the browser.
• Forms stay filled in while the app is open: going to the Runs page or another model and back keeps what you entered and where you were on the page. A reload or Reset all starts over.
• A run that fails for lack of VRAM says so on the Runs page.

Changes

• DataLoader workers: an empty max_data_loader_n_workers now means none with cached latents and 2 otherwise. sd-scripts' default of 8 processes delayed the first step by about a minute on Windows.
• Training console: sd-scripts' colors and full-width progress bars are back, and network_reg_lrs / network_reg_dims log one line per pattern instead of one per module.
• The Anima LoRA page no longer offers self_attn_lr, cross_attn_lr, mlp_lr, mod_lr and llm_adapter_lr: sd-scripts only reads them for full fine-tuning. Use network_reg_lrs for per-module learning rates.

Fixes

• Anima with compile_shapes = bounded no longer fails in its first steps (ConstraintViolationError) when include_patterns adds LoRA to the embedders or the AdaLN modulation.

v1.0.0

LoRA-Forge 1.0.0

The first release of LoRA-Forge, a web UI for training LoRAs with kohya-ss/sd-scripts, continuing lora-scripts.

Highlights

• A new web UI (Vue 3) in English, 中文 and 日本語, with a dark mode. Each page has a section index, a search, an Essentials / All options switch and a "changed only" filter. Configs import and export as JSON or TOML, with a TOML preview.
• One page per model: Anima, SDXL, FLUX, SD3.5, SDXL fine-tune, SD1.5 and SD1.5 Dreambooth, each with its own defaults and the same sections in the same order. Old URLs still open the right page.
• Runs page: progress, speed and time left, loss and learning rate charts (one line per learning rate group, network_reg_lrs groups named by pattern), sample images, saved LoRA files and the console output. A failed run opens on its error. Runs stay listed after the GUI restarts. TensorBoard is optional.
• Anima: per-block torch.compile, on by default, with LoRA-Forge's compile_shapes = bounded, which compiles one graph for all bucket and batch sizes: 2 graphs instead of 28 on a 20-bucket dataset, 11-55% faster first epochs, same loss. No C++ compiler needed. Also per-subfolder timestep offsets, a timestep distribution preview and the 2D Qwen-Image VAE. See docs/anima.md.
• Setup with uv: run_gui.ps1 / run_gui.sh start everything through uv, which installs Python 3.12 and the dependencies on the first start (--cn for mirrors in mainland China). sd-scripts v0.12.0, the tested version, is cloned on first start and only updated when you run update-sd-scripts.

Changes since 1.0.0-rc1

• Anima: compile is on by default. compile_dynamic and compile_dynamic_seq are one option, compile_shapes (bounded, static, auto, dynamic); older configs import into it.
• Every trainer page uses the same sections (Model, Dataset, Captions, Network, Training, Learning rate and optimizer, Timesteps and loss, ..., Memory, Speed, torch.compile), and exported TOML files follow them.
• Learning rates and optimizer settings are number fields; number fields have sensible steps and limits.
• max_token_length (SD1.5 / SDXL) is a choice of 75, 150 or 225. The old default 255 is not a value sd-scripts accepts (it trained as 225); imported configs are corrected.
• min_bucket_reso and max_bucket_reso only show while bucket_no_upscale is off: sd-scripts ignores them otherwise. SDXL fine-tune and SD1.5 Dreambooth now have bucket_no_upscale too (off, as before).

Requirements

• Windows or Linux with uv and Git. Install uv with its official installer: powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex" on Windows, curl -LsSf https://astral.sh/uv/install.sh | sh on Linux, then open a new terminal. uv installs Python 3.12 by itself.
• An NVIDIA GPU with a driver that supports CUDA 13: the environment uses PyTorch 2.14 for CUDA 13.0. Other GPUs are not supported yet.
• The first start downloads Python, PyTorch and sd-scripts and takes a while.

Getting started

1. Download lora-forge-v1.0.0.zip below and extract it. The web UI is prebuilt, so no Node.js is needed.
2. Run run_gui.ps1 (Windows, or right-click → Run with PowerShell) or bash run_gui.sh (Linux).
3. Open http://127.0.0.1:28000. The README shows how to lay out a dataset and train a first LoRA.

Optional: uv sync --extra flash-attn in the LoRA-Forge folder installs a prebuilt flash-attn (Windows, Linux x86_64 and ARM) for attn_mode = flash.

Known limitations

• Maintained mainly for personal use; expect rough edges. Issues and PRs are welcome.
• SD1.5 pages are kept for older models and get less attention.
• PyTorch is capped below 2.15 until it has been tested.

v1.0.0-rc1

LoRA-Forge 1.0.0 release candidate 1

The first release of LoRA-Forge, a web UI for training LoRAs with kohya-ss/sd-scripts, continuing lora-scripts. This is a release candidate: please try the download on your machine and report what breaks. The final 1.0.0 will be the same code if nothing does.

Highlights

• A new web UI (Vue 3) in English, 中文 and 日本語, with a dark mode. Each page has a section index, a search, an Essentials / All options switch and a "changed only" filter. Configs import and export as JSON or TOML, with a TOML preview.
• One page per model: Anima, SDXL, FLUX, SD3.5, SDXL fine-tune, SD1.5 and SD1.5 Dreambooth, each with its own defaults. Old URLs still open the right page.
• Runs page: progress, speed and time left, loss and learning rate charts (one line per learning rate group, network_reg_lrs groups named by pattern), sample images, saved LoRA files and the console output. A failed run opens on its error. Runs stay listed after the GUI restarts. TensorBoard is optional.
• Anima: per-block torch.compile with compile_dynamic_seq (on by default), which compiles one graph for all bucket and batch sizes: 2 graphs instead of 28 on a 20-bucket dataset, 11-55% faster first epochs, same loss. Also per-subfolder timestep offsets, a timestep distribution preview and the 2D Qwen-Image VAE. See docs/anima.md.
• Setup with uv: run_gui.ps1 / run_gui.sh start everything through uv, which installs Python 3.12 and the dependencies on the first start (--cn for mirrors in mainland China). sd-scripts v0.12.0, the tested version, is cloned on first start and only updated when you run update-sd-scripts.

Requirements

• Windows or Linux with uv and Git. Install uv with its official installer: powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex" on Windows, curl -LsSf https://astral.sh/uv/install.sh | sh on Linux, then open a new terminal. uv installs Python 3.12 by itself.
• An NVIDIA GPU with a driver that supports CUDA 13: the environment uses PyTorch 2.14 for CUDA 13.0. Other GPUs are not supported yet.
• The first start downloads Python, PyTorch and sd-scripts and takes a while.

Getting started

1. Download lora-forge-v1.0.0-rc1.zip below and extract it. The web UI is prebuilt, so no Node.js is needed.
2. Run run_gui.ps1 (Windows, or right-click → Run with PowerShell) or bash run_gui.sh (Linux).
3. Open http://127.0.0.1:28000. The README shows how to lay out a dataset and train a first LoRA.

Optional: uv sync --extra flash-attn in the LoRA-Forge folder installs a prebuilt flash-attn (Windows, Linux x86_64 and ARM) for attn_mode = flash.

Known limitations

• Maintained mainly for personal use; expect rough edges. Issues and PRs are welcome.
• SD1.5 pages are kept for older models and get less attention.
• PyTorch is capped below 2.15 until it has been tested.