v0.9.0

Fixed

• Training a m2m100 model on a language (code) not originally included in its tokenizer now works.

Changed

• Pytorch compatibility changed to >= 2.0, < 2.3
• 🤗 datasets compatibility changed to >= 2.18, < 2.19

Full Changelog: v0.8.0...v0.9.0

v0.8.0

Fixed

• Fixed multiple save when using step-save-period in conjunction with bach accumulation (close #30)

Changed

• Maximum Pyorch compatibility bumped to 2.1
• max_steps and max_epochs can now be set in the tuning config. Setting them via command line
  options is deprecated and will be removed in a future version.

v0.7.3 — Bug Fix

Fixed

• Behaviour when asking for denoising in mBART with a model that has no mask token.

v0.7.2 — Now with a doc??!?

Fixed

• In mBART training, loss scaling now works as it was supposed to.
• We have a documentation now! Check it out at https://zeldarose.readthedocs.io, it will get
  better over time (hopefully!).

v0.7.1 Bug fix

Fixed

• Translate loss logging is not always zero anymore.

Now with mBART translations!

The main highlight of this release is the addition of mBART training as a task, so far slightly different from the original one, but similar enough to work in our tests.

Added

• The --tf32-mode option allows to select the level of NVidia Ampère matmul otpimisations.
• The --seed option allows to fix a random seed.
• The mbart task allows training general seq2seq and translation models.
• A zeldarose command that serves as entry point for both tokenizer and transformer training.

Changed

• BREAKING --use-fp16 has been replaced by --precision, which allows to also use fp64 and
  bfloat. Previous behaviour can be emulated with --precision 16.
• Remove the GPU stats logging from the profile mode since Lightning stopped supporting it
• Switched TOML library from toml to
  tomli
• BREAKING Bumped the min version of several dependency
  • pytorch-lightning >= 1.8.0
  • torch >= 1.12
• Bumped max version of several dependency
  • datasets < 2.10
  • pytorch-lightning < 1.9
  • tokenizers < 0.14

v0.6.0 — Dependencies compatibilities

This one to fix compatibilities issues with our dependencies. Bumps minimal versions and add upper version limits.

Changed

• Bumped torchmetrics minimal version to 0.9
• Bumped datasets minimal version to 2.4
• Bumped torch max version to 1.12

Fixed

• Dataset fingerprinting/caching issues #31

Full Changelog: v0.5.0...v0.6.0

v0.5.0 — Housekeeping

The minor bump is because we have several new minimal version requirements (and to fairly recent versions with that). Otherwise, this is mostly internal stuff.

Added

• lint extra that install linting tools and plugins
• Config for flakeheaven
• Support for pytorch-lightning 1.6

Changed

• Move packaging config to pyproject.toml and require setuptools>=61.
• click_pathlib is no longer a dependency and click has a minimal version of 8.0.3

Full Changelog: v0.4.0...v0.5.0

v0.4.0 — experimental ELECTRA

Added

• Replaced Token Detection (ELECTRA-like) pretraining
  • Some of the API is still provisional, the priority was to get it out, a nicer interface will
    hopefully come later.
• --val-check-period and --step-save-period allowing to evaluate and save a model decoupled
  from epochs. This should be useful for training with very long epochs.
• The datasets path in zeldarose-transformer can now be 🤗 hub handles. See --help.

Changed

• The command line options have been changed to reflect change in Lightning
  • --accelerator is now used for devices, tested values are "cpu" and "gpu"
  • --strategy now specifies how to train, tested values are None (missing), "ddp",
    "ddp_sharded" "ddp_spawn" and"ddp_sharded_spawn".
  • No more option to select sharded training, use the strategy alias for that
  • --n-gpus has been renamed to --num-devices.
  • --n-workers and --n-nodes have been respectively renamed to --num-workers and
    --num-nodes.
• Training task configs now have a type config key to specify the task type
• Lightning progress bars are now provided by Rich
• Now supports Pytorch 1.11 and Python 3.10

Internal

• Tests now run in Pytest using the console-scripts
  plugin for smoke tests.
• Smoke tests now include ddp_spawn tests and tests on gpu devices if available.
• Some refactoring for better factorization of the common utilities for MLM and RTD.

v0.3.4 —Lightning bump

Just bumping pytorch-lightning to the current minor version.