New Models

[StellaAthena]

GPT-NeoX 20B is a new language model by EleutherAI trained on the Pile. It is a decoder model that is competent at both English and generating code.

The training code can be found here and the model weights will be released next week. It was trained using PyTorch and DeepSpeed on 96 A100 @ CoreWeave. You can find the compute states here... sorry, you'll have to do the math for the total compute yo