v0.3.0

0.3.0 (2024-09-01)

Features

• add everyday converstation llama dataset (a27179c)
• add medical dataset (b79d723)
• add python_code_instructions dataset (540a33b)

Bug Fixes

• dataset name (0cdcee3)
• import lib (5b3cf2c)
• too small validation dataset (1498fad)

v0.2.0

0.2.0 (2024-08-30)

Features

• add change vocab size for model (bf1ff05)
• add custom dataset preparation (580e809)
• add hosting server (78f5ffb)
• add inference script (a5f177d)
• server_api: add optional checkpoint (4099e95)

Bug Fixes

• duplicate cfg.log file (960f530)
• duplicate log in mlflow (bd83877)
• eos_token, response message, wrong key json (f75d810)
• infer token of server api, add eot token to input (20d13fe)
• miss best ckpt, load config (7616f7c)
• not passing parameters in the config (59ed96a)
• server_api: bot repeat the same sentence (4f7c717)
• server_api: special token (02f9e4d)
• variable type in args (c164c7e)

Documentation

• add export weight (0438c89)
• fix username, add inference scripts (17ed15c)

v0.1.0

0.1.0 (2024-08-28)

Features

• add automatic mix precision (5c4831d)
• add crop block size of GPT model (3c66a57)
• add data distributed parallel training (f0bb8b0)
• add dataloader for .bin file - openwebtext dataset (2966136)
• add GPT2 config, weight converter, missing model file (c2ee7d7)
• add GPT2 model (bb53955)
• add log interval (6640c38)
• add loss function, learning rate schedular (4a5f625)
• add openwebtext preprocess (5d7daa9)
• add optimizers (634108f)
• add train config for openwebtext dataset, gpt2 model (92d20bd)
• add train script for GPT2, update config (13a3fb3)

Bug Fixes

• automatic mix precision for cpu (ffe3058)
• change cel to functional (66c25ef)
• duplicate log (a903e81)
• feed forward loss function, change variable name (6d799dd)
• gpt.py: remove model ff return loss (f12aa99)
• infinity validating step (a07d7b6)
• losses.py: missing parent init function (2b6a2df)
• openwebtext dataset link (4fe8add)
• same GPU in training multi GPU (b5ddc45)
• step calling in optimizer (4f1d309)
• x, y device in dataloader (9db7aad)

Documentation

• add training instruction (bcfc7b1)