Use PyTorch Lightning to handle the training (free checkpointing + logging + 16-bit precision)

[williamFalcon]

Awesome repo!

However, not sure why go through the effort of implementing your own trainer again...

In lightning we already support:

• automatic checkpoint loading/saving
• multi-cpu
• multip-gpu
• multi-tpu core
• 16-bit precision (amp and native)
• accumulated gradients
• and about 40+ more features.

Not to mention it's maintained by a team of over 20+ fulltime engineers and 200+ open-source contributors and has been adopted by over 400 companies and research labs.

https://pytorch-lightning.readthedocs.io/en/latest/new-project.html

[ykumards]

minGPT, no? 😬

[karpathy]

@williamFalcon your link above doesn't work btw. Closing the Issue and will look at the PR shortly.

[williamFalcon]

fixed!

https://pytorch-lightning.readthedocs.io/en/latest/new-project.html