Add async sgd document #8474
Add async sgd document #8474
Conversation
| @@ -365,7 +365,8 @@ def settings(batch_size, | |||
| regularization=None, | |||
| is_async=False, | |||
| model_average=None, | |||
| gradient_clipping_threshold=None): | |||
| gradient_clipping_threshold=None, | |||
There was a problem hiding this comment.
Seems these changes broke the CI unit tests
doc/howto/cluster/cmd_argument_cn.md
Outdated
|
|
||
| ## 异步 SGD 更新 | ||
|
|
||
| 您可以修改 `optimize` 的参数来支持异步SGD更新，示例如下： |
There was a problem hiding this comment.
下文里没有一个叫做 optimize 的参数呀？
There was a problem hiding this comment.
Done as comment #8474 (comment).
doc/howto/cluster/cmd_argument_en.md
Outdated
|
|
||
| ## Async SGD Update | ||
|
|
||
| You can modify the parameters of `optimize` to support Async SGD update, for example: |
There was a problem hiding this comment.
I don't see a parameter named optimize in the following code snippet. Do you mean
We can set some parameters of the optimizer to make it support async SGD update. For example, we can set the
is_asyncandasync_lagged_grad_discard_ratioof theAdaGradoptimizer:
There was a problem hiding this comment.
Thanks, this description is clearer.
| @@ -396,6 +397,10 @@ def settings(batch_size, | |||
| value larger than some value, will be | |||
| clipped. | |||
| :type gradient_clipping_threshold: float | |||
| :param async_lagged_grad_discard_ratio: async sgd gradient commit control, | |||
| @@ -396,6 +397,10 @@ def settings(batch_size, | |||
| value larger than some value, will be | |||
| clipped. | |||
| :type gradient_clipping_threshold: float | |||
| :param async_lagged_grad_discard_ratio: async sgd gradient commit control, | |||
| when async_lagged_grad_discard_ratio * num_gradient_servers commit passed, | |||
| current async gradient will be discard silently. | |||
There was a problem hiding this comment.
will be discard silently
==>
is discarded
| @@ -396,6 +397,10 @@ def settings(batch_size, | |||
| value larger than some value, will be | |||
| clipped. | |||
| :type gradient_clipping_threshold: float | |||
| :param async_lagged_grad_discard_ratio: async sgd gradient commit control, | |||
| when async_lagged_grad_discard_ratio * num_gradient_servers commit passed, | |||
| current async gradient will be discard silently. | |||
There was a problem hiding this comment.
current sgd gradient
==>
the current SGD gradient
Fixed #8268