Skip to content

dl-translate v0.2.0rc1

Pre-release
Pre-release

Choose a tag to compare

View all tags
@xhluca xhluca released this 21 Mar 01:39
· 33 commits to main since this release
v0.2.0rc1
7ad1aa1
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Verified
Learn about vigilant mode.

Add m2m100 as an alternative to mbart50

m2m100 has more languages available (~110) and has also reported their absolute BLEU scores.

Added

  • dlt.lang.m2m100 module: Now has variables for over 100 languages, also auto-complete ready. Example: dlt.lang.m2m100.ENGLISH.
  • dlt.utils.available_languages, dlt.utils.available_codes: Now supports argument "m2m100"

Changed

  • [BREAKING] dlt.lang.TranslationModel: A new model parameter called model_family in the initialization function. Either "mbart50" or "m2m100". By default, it will be inferred based on model_or_path. Needs to be explicitly set if model_or_path is a path.
  • dlt.TranslationModel.translate: Improved docstring to be more general.
  • Tests pertaining to m2m100
  • scripts/generate_langs.py: Renamed, mechanism now changed to loading from json files

Fixed

  • dlt.TranslationModel.available_codes() was returning the languages instead of the codes. It will now correctly return the code.

Removed

  • Output type hints for TranslationModel.get_transformers_model and TranslationModel.get_tokenizer
  • [BREAKING] dlt.TranslationModel.bart_model and dlt.TranslationModel.tokenizer are no longer available to be used directly. Please use dlt.TranslationModel.get_transformers_model and dlt.TranslationModel.get_tokenizer instead.
Assets 2
Loading