Skip to content

Minimax h3 lora training - #26

Merged
imprsnst merged 3 commits into
mainfrom
minimax-h3-lora-training
Aug 5, 2026
Merged

Minimax h3 lora training#26
imprsnst merged 3 commits into
mainfrom
minimax-h3-lora-training

Conversation

@imprsnst

@imprsnst imprsnst commented Aug 5, 2026

Copy link
Copy Markdown
Collaborator

Summary

MiniMax H3 LoRA training

Adds MiniMax H3 as a fourth trainable architecture. Trains on still images, and the adapter applies to video.

What's new

  • H3 arch entry in arch.py, sharing Z-Image's flow convention
  • Staged precache in h3.py, because the fp32 video VAE and the 32B conditioner cannot be resident together
  • 4-bit base loader: factorise the AdaLN branch, then quantise, streaming block by block
  • lora port on all four H3 nodes in runner.py, so the trained file is usable
  • Trainer UI picker, plus the model download popup wired to the H3 requirements

Measured on an L40S

Phase Peak
Latent caching 10.8GB
Caption caching 20.5GB
Training 11.7GB
  • Run peaks at 20.6GB, set by the caption pass, so 512px and 768px cost the same
  • 0.63s a step at 512px, 0.77s at 768px, about 7 minutes of startup
  • 62GB checkpoint down to 11.7GB, host RAM stays near 1.1GB
  • Full tables in TRAINING.md

Fixes two live bugs in the existing trainer

  • _to_device cast every non-mask tensor to the compute dtype, which would destroy index tensors
  • PEFT never saw is_loaded_in_4bit, so NF4 runs silently used the generic LoRA layer instead of the bitsandbytes one. Affects Krea 2 and FLUX.2 today

Verified

  • 771 Python tests, 205 TS tests, typecheck and lint clean
  • Trained a 400 step LoRA and rendered the same seed with and without it. Clips differ by 50.4 mean absolute out of 255, both coherent
  • Fuse checked arithmetically: all 313 adapted modules, including 13 outside the block stack that a per-block fuse would have silently missed

Known, not fixed here

  • A LoRA trained with alpha different from rank fuses at the wrong strength, on all four architectures. Hidden by the default of alpha equalling rank
  • 15 pre-existing UP042 lint errors in unrelated files, deferred deliberately

Also bumps to 1.2.62.
Closes #

Checklist

  • npm run typecheck && npm run lint && npm run test pass
  • cd core && ruff check . && uv run pytest -q pass (if the engine changed)
  • Commits use Conventional Commits and are signed off (git commit -s)
  • Change is small and focused, and matches the surrounding style

@imprsnst
imprsnst merged commit 2cb6e04 into main Aug 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant