v0.1.0

0.1.0 (2024-08-28)

Features

• add automatic mix precision (5c4831d)
• add crop block size of GPT model (3c66a57)
• add data distributed parallel training (f0bb8b0)
• add dataloader for .bin file - openwebtext dataset (2966136)
• add GPT2 config, weight converter, missing model file (c2ee7d7)
• add GPT2 model (bb53955)
• add log interval (6640c38)
• add loss function, learning rate schedular (4a5f625)
• add openwebtext preprocess (5d7daa9)
• add optimizers (634108f)
• add train config for openwebtext dataset, gpt2 model (92d20bd)
• add train script for GPT2, update config (13a3fb3)

Bug Fixes

• automatic mix precision for cpu (ffe3058)
• change cel to functional (66c25ef)
• duplicate log (a903e81)
• feed forward loss function, change variable name (6d799dd)
• gpt.py: remove model ff return loss (f12aa99)
• infinity validating step (a07d7b6)
• losses.py: missing parent init function (2b6a2df)
• openwebtext dataset link (4fe8add)
• same GPU in training multi GPU (b5ddc45)
• step calling in optimizer (4f1d309)
• x, y device in dataloader (9db7aad)

Documentation

• add training instruction (bcfc7b1)