Skip to content

v0.2.2

Latest

Choose a tag to compare

@github-actions github-actions released this 04 Nov 14:16
9a631ca

ReactantNitro v0.2.2

Diff since v0.2.1

strict for checkpoints with extra or missing leaves

Nitro(e; weights = source, strict = false) loads a source whose parameter tree differs from the model's, as PyTorch's load_state_dict(strict = False) does. Parameters are matched by keypath: one the source lacks keeps the model's fresh initialization, and one the model lacks is dropped. Both always warn, listing the leaves.

n = Nitro(MyExp(; head = true); weights = :latest => :best, strict = false)

strict = true is the default and refuses such a source, naming the leaves and pointing at strict = false. A parameter held at different sizes refuses either way. resume refuses strict = false, and a partial load refuses restore_optimizer = true, since the record's optimizer state follows its own leaves. strict applies to every weights source: a path, run => :best | :latest, run => Average(...), and another Nitro.

The nitro_train, nitro_validate, nitro_evaluate, nitro_predict and nitro_export tools take strict as well.

Changes

  • The layer state st is now compared with the model's when a checkpoint loads. A leaf that does not fit, or that only one side holds, warns, naming it as st.<keypath>, and keeps the model's fresh initialization. This applies under either strict setting. Before, st was replaced wholesale from the record without a check.
  • A weights = other::Nitro warm start now matches the two trees by keypath rather than by position.
  • check_record_tree_compatible is replaced by restore_trees.

Fixes

  • Manual mode now counts the step body in t_step, so data_wait_frac is no longer logged as 1.0 every epoch and the data wait warning no longer fires spuriously.