v1.4.0

What's new

Changed ⚠️

• Updated default layer norm epsilon for OLMo models from 1e-5 to 1e-6 to match latest model.
• Renamed FSLDataLoader to NumpyFSLDataLoader.
• Renamed VSLDataLoader to NumpyVSLDataLoader.
• The trainer now takes a data_loader: DataLoaderBase instead of a dataset: NumpyDatasetBase.

Commits

55343dd fix loading training state dict
b921299 Allow unknown number of batches with data loaders
87f1e89 fix restarts for custom data loader
767c550 Add example of custom data loader
6237f7d Trainer now takes a data loader instead of a dataset (#59)
f6fc369 update default LN eps to match latest OLMo model (#58)
db522d1 allow loading via pickling
7d26589 make VSL curr config more flexible