Skip to content

Commit

Permalink
[doc] add api reference
Browse files Browse the repository at this point in the history
  • Loading branch information
ver217 committed Mar 7, 2023
1 parent 5abf1d2 commit 89aa499
Show file tree
Hide file tree
Showing 2 changed files with 12 additions and 0 deletions.
6 changes: 6 additions & 0 deletions docs/source/en/features/nvme_offload.md
Original file line number Diff line number Diff line change
Expand Up @@ -247,3 +247,9 @@ Mem usage: 5298.344 MB
```

NVME offload saves about 294 MB memory. Note that enabling `pin_memory` of Gemini can accelerate training but increase memory usage. So this result also meets our expectation. If we disable `pin_memory`, we can aslo observe a memory usage drop about 900 MB.

## API Reference

{{ autodoc:colossalai.nn.optimizer.HybridAdam }}

{{ autodoc:colossalai.nn.optimizer.CPUAdam }}
6 changes: 6 additions & 0 deletions docs/source/zh-Hans/features/nvme_offload.md
Original file line number Diff line number Diff line change
Expand Up @@ -235,3 +235,9 @@ Mem usage: 5298.344 MB
```

NVME 卸载节省了大约 294 MB 内存。注意使用 Gemini 的 `pin_memory` 功能可以加速训练,但是会增加内存占用。所以这个结果也是符合我们预期的。如果我们关闭 `pin_memory`,我们仍然可以观察到大约 900 MB 的内存占用下降。

## API 参考

{{ autodoc:colossalai.nn.optimizer.HybridAdam }}

{{ autodoc:colossalai.nn.optimizer.CPUAdam }}

0 comments on commit 89aa499

Please sign in to comment.