Help wanted: training Level-of-Token for MiniMax H3

[johndpope]

Level-of-Token (LoT) runs the MiniMax H3 DiT on fewer, larger tokens where detail is low, then recovers the full-resolution velocity (paper). The speed is there: one DiT forward on a 24 GB card is 2.85× faster on a 124-frame 768×1344 clip, 2.55× on a 22-frame 512×768 clip, 2.15× on a 768×1152 still. The pictures are not there yet: coarse regions go soft, faces in talking-head shots distort in fixed layouts, and the LoRA can bend bodies even with LoT off.

This thread is for helping train it. Useful things to post here:

• GPU time: a longer run of the current recipe (train_h3.py --distill 1, warm-started from run 4) on more, and more varied, data.
• Data: 22-frame 24 fps clips with captions. lot/build_cache_h3.py media turns a folder of mp4s and .txt captions into a training cache.
• Ideas: a larger rank or nonlinear extent heads, a higher --dense-p, better layout families. The face-box layout in ComfyUI already comes close to dense quality.
• Results: a render (same seed, dense vs LoT) and the time_lot_h3.py lines from your card, so each new fit can be compared with the last.

Start here:

• Code and cheatsheet: https://github.com/scrya-com/lot-diffusion (start at lot/CHEATSHEET.md)
• Weights (runs 3 and 4): https://huggingface.co/johndpope/MiniMax-H3-LoT
• ComfyUI nodes: https://github.com/scrya-com/ComfyUI-MiniMax-H3-Image-Lane
• Renders, eval curves and run history so far: Level-of-Token: single-GPU synthetic training johndpope/MiniMax-H3#1

The weights are derived from MiniMax H3 and fall under the MiniMax H3 Community License: not for use in the EU, UK, South Korea or USA.