Gluon trainer updates: add learning_rate and lr_scheduler properties and add setter for learning rate #7659
Conversation
t commit -m 'sync'
| @@ -113,6 +113,36 @@ def _init_kvstore(self): | |||
|
|
|||
| self._kv_initialized = True | |||
|
|
|||
|
|
|||
| @property | |||
| def learning_rate(self): | |||
There was a problem hiding this comment.
document this as
Properties
----------
in init doc
There was a problem hiding this comment.
Also report learning_rate when using lr_sheduler
python/mxnet/gluon/trainer.py
Outdated
|
|
||
|
|
||
| @property | ||
| def lr_scheduler(self): |
There was a problem hiding this comment.
don't expose this for now
python/mxnet/gluon/trainer.py
Outdated
|
|
||
| @property | ||
| def learning_rate(self): | ||
| return self._optimizer.lr |
There was a problem hiding this comment.
This kind of implementation couples the two classes together (i.e. Trainer must know the structure of optimizer). Instead add accessors in optimizer and make reading learning rate the concern of optimizer.
python/mxnet/gluon/trainer.py
Outdated
| "learning rate only when the LRScheduler of" | ||
| "the optimizer is undefined.") | ||
| else: | ||
| self._optimizer.lr = lr |
There was a problem hiding this comment.
Same as the accessor comment. Try make setting learning rate the concern of optimizer's.
|
Dudes, thank you for your comments. Code is updated. Let me know if you spot any further issues. |
python/mxnet/optimizer.py
Outdated
| @@ -191,6 +204,24 @@ def update(self, index, weight, grad, state): | |||
| """ | |||
| raise NotImplementedError() | |||
|
|
|||
| def set_learning_rate(self, lr): | |||
There was a problem hiding this comment.
@learning_rate.setter
def set_learning_rate
and then you can do
optimizer.learning_rate = 0.5
python/mxnet/gluon/trainer.py
Outdated
| raise UserWarning("Optimizer has to be defined before its learning" | ||
| "rate is mutated.") | ||
| else: | ||
| self._optimizer.set_learning_rate(lr) |
There was a problem hiding this comment.
Update according to the comment below
python/mxnet/gluon/trainer.py
Outdated
| ---------- | ||
| Learning_rate: float | ||
| The learning rate of the optimizer or the learning rate of the | ||
| LRScheduler of the optimizer if the LRScheduler is defined. |
There was a problem hiding this comment.
explain it can be accessed with trainer.learning_rate and set with trainer.learning_rate = xxx
astonzhang
changed the title
|
All tests passed except R GPU, @thirdwing is fixing this in #7686. |
tests/python/unittest/test_loss.py
Outdated
| @@ -70,15 +70,14 @@ def test_ce_loss(): | |||
| label = mx.nd.array(np.random.randint(0, nclass, size=(N,)), dtype='int32') | |||
There was a problem hiding this comment.
Submit test fix in a separate PR and revert the change on file property
100644 → 100755
astonzhang
changed the title
|
The unit test patch is in a separate pr as suggested by @piiswrong |
|
Closing this PR due to new PR at #7760 |
No description provided.