Skip to content

Tutel v0.1.5
Compare
Choose a tag to compare
@ghostplant ghostplant released this 26 Feb 07:19
· 25 commits to v0.1.x since this release
v0.1.5
bddc915
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

What's New in v0.1.5:

  1. Add 2D hierarchical a2a algorithm used for extremely-large scaling;
  2. Support different parallel_type for MoE computation: data, model, auto;
  3. Combine different expert granularities (e.g. normal, sharded experts, megatron dense ffn) into same programming interface & style;
  4. New features: is_postscore to specify whether gating scores are weighed during encoding or decoding;
  5. Enhance existing features: JIT compiler, a2a overlap with 2D.
How to Setup:
python3 -m pip install --user https://github.com/microsoft/tutel/archive/refs/tags/v0.1.5.tar.gz

Contributors: @abuccts, @yzygitzh, @ghostplant, @EricWangCN

Contributors

yzygitzh, ghostplant, and 2 other contributors
Assets 2
Loading