Skip to content

It Works Well

Pre-release
Pre-release

Choose a tag to compare

View all tags
@SulRash SulRash released this 03 Feb 15:10
· 12 commits to main since this release
v0.1
61b5345
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.
  • Can pretrain an LLM from scratch given a config file.
  • Can train through Deepspeed.
  • Can train through Fully Sharded Data Parallel.
  • Code supports training through Megatron-LM.
  • Reliable checkpointing.
  • Implements dataset packing for efficient training.
  • Easy to understand code base.

A lot more but these are just the highlights.

Assets 2
Loading