Skip to content

A new KD method, FSDP/FSDP2 support, and experiment tracking

Latest

Choose a tag to compare

@yoshitomo-matsubara yoshitomo-matsubara released this 05 Aug 05:40
· 5 commits to main since this release
6617f44

New method

New features

  • Extend data parallel support for FSDP/FSDP2 (PR #537)
  • Support trackio and wandb (PR #543)
  • Add accumulation mode with stack option (PR #536)

Improvements

  • Use a more robust eval for DDP (PR #532)
  • Remove teacher I/O caching (PR #539)
  • Extend example log_analyzer.py (PR #544)
  • Add yaml examples (PR #542)
  • Refactor (PR #526)

Tests and CI

  • Add loss test cases with fixtures (PR #535)
  • Replace setup.py with pyproject.toml (PR #534)
  • Add a Copilot instruction file and update GitHub Actions (PRs #528, #530, #531, #533)

Documentation

  • Update README's (PR #540)
  • Update acknowledgments (PRs #527, #529)
  • Update year (PR #523)

Misc

  • Update version and notebooks (PRs #522, #545)
  • Update dependencies (PR #539)