H3 minimax - level of token training - help with training wanted

[johndpope]

Model

H3-minimax

Weights and reference implementation

Na

What you plan to build

johndpope/MiniMax-H3#1

I’ve built level of token layout into / on top of h3 minimax -
Trained briefly on 24gb gpu for half a day.
I have some weights I will upload soon to huggingface / barely usable but can recreate images in ticket above.
I believe it requires some mod to Fizgig trainer

https://github.com/johndpope/MiniMax-H3/blob/main/scripts/lot/CHEATSHEET.md

https://github.com/johndpope/MiniMax-H3/blob/main/scripts/lot/README.md#training-h3-for-lot-from-your-own-mp4s

If anyone knows anyone training who has the compute for training - you can expect a 2.64x speed up on dit l.

Pull request

• I plan to build this and send it as a pull request.