Skip to content

Commit

Permalink
[feature] update optimize readme;
Browse files Browse the repository at this point in the history
  • Loading branch information
duanjunwen committed Apr 8, 2024
1 parent f77ffc2 commit b75ac58
Showing 1 changed file with 1 addition and 3 deletions.
4 changes: 1 addition & 3 deletions colossalai/nn/optimizer/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,9 +89,7 @@ A series of optimizers have been optimized and integrated.

### Distributed Adafactor

Distributed Adafactor supports tensor parallelism and ZerO optimization. Here is a brief flowchart of how adafactor implements the tensor parallel:

[[Tensor Parallel Strategy in Distributed Adafactor]](adafactor_strategy.png)
Distributed Adafactor supports tensor parallelism and ZerO optimization.

### Performance
| Version | iter | Float Percision | Device Nums | weight shape | Avg runtime(ms) | Avg Speed Up Rate | Best Speed Up Rate |
Expand Down

0 comments on commit b75ac58

Please sign in to comment.