v0.10.0

What's Changed

• feat: disable progress bars, add hf hub as a direct dependency by @stephantul in #345
• feat: Replace pca by @stephantul in #344
• feat: export classifier and regressor pipelines to ONNX by @serhiizghama in #346
• feat: make onnx export a function by @stephantul in #347
• feat: add onnx model cards by @stephantul in #349
• fix: add distillation duplicate guard by @stephantul in #350
• fix: start of word tokens for added by @stephantul in #351
• fix: trainer uses best checkpoint by @stephantul in #352
• fix: onnx export correctly ignores unk by @stephantul in #353
• fix: unk token for unigrams by @stephantul in #354
• feat: new integration test by @stephantul in #355
• feat: make setters for max_length by @stephantul in #356
• chore: typing on loading should return a T, not a StaticModel by @stephantul in #357
• fix(train): pad training batches with the model's pad id by @serhiizghama in #358
• fix(train): drop unknown tokens during training and inference by @serhiizghama in #359
• docs: Update main readme by @Pringled in #361
• fix(inference): pass the token through when loading the encoder by @serhiizghama in #362
• feat: max length behavior by @stephantul in #363
• fix(onnx): export vocabulary-quantized models with their token mapping and weights by @serhiizghama in #364
• feat: small helper method to reduce overlap by @stephantul in #365
• docs: fix classifier docstring by @stephantul in #366
• feat: higher speed encoding by @stephantul in #367
• fix: norm underflow for float16 by @stephantul in #369
• feat: add allow list for paths by @stephantul in #370
• feat: remove loggit mapping from training by @stephantul in #371
• feat: add token dropout to training pipeline by @stephantul in #373
• feat: add pairwise trainer by @stephantul in #372
• fix(train): keep the trainer's max_length independent of the static model's tokenizer truncation by @serhiizghama in #374
• fix: disable padding during distill and inference by @stephantul in #376
• docs: Add Greptile acknowledgement by @Pringled in #377
• fix: divide by length 1 instead of 1e-16 by @stephantul in #380
• fix: stop the trainer's tokenizer from padding by @stephantul in #379
• feat: pair and similarity head with 0 layers have no layers by @stephantul in #383
• feat: train pair models with InfoNCE by @stephantul in #381
• feat: add other metrics to loop progress bar by @stephantul in #384
• feat: add contribution guidelines by @stephantul in #386
• feat: add support for datasets in training by @stephantul in #387
• feat: add explicit kwargs to all trainable by @stephantul in #388
• feat: make max length argument explicit by @stephantul in #389
• feat: speed up training, remove pad token requirement by @stephantul in #390
• feat: add focal loss by @stephantul in #391
• fix: download readme and respect subfolder by @stephantul in #392
• fix: add token dropout to pair trainer by @stephantul in #393
• remove tutorials by @stephantul in #394
• fix: random seed controls splitting by @stephantul in #396
• docs: update docs for training by @stephantul in #397
• chore: update version -> v0.10.0 by @stephantul in #398

New Contributors

• @serhiizghama made their first contribution in #346

Full Changelog: v0.9.0...v0.10.0