introduce gradient update handler to the base estimator

[liuzh47]

Description

This change add default gradient update handler to the base estimator class. The purpose of this change is to decouple the forward/backward computation with the gradient apply process. In some use cases such as gradient accumulation, gradient updates are not executed during each training batch. See issue description #16869.

[ptrendx]

@leezu What is the rationale behind marking this PR as 1.6? This looks like adding a new functionality.

[roywei]

Thank you for the improvement! 2 concerns.
Also could you point an example which require custom gradient handler? (gradient clipping or aggregation)

[leezu]

@ptrendx the estimator API is currently experimental and is not yet up to more complex real-world use-cases. @liuzh91 is currently converting GluonNLP training scripts to make use of the Estimator API and ran into several remaining shortcomings of the API.
We shouldn't release 1.6 with a known insufficient version of Estimator if fixes are available. Thus if this PR is approved and merged to master in time for the 1.6 release, it may be good to backport. Given the scope of the estimator API and the current shortcomings we can consider these PRs as bugfixes. If we don't fix the API with the 1.6 release, we may be bound to the current API due to backwards compatibility commitments.

Do you think backporting the fix would be reasonable or do you have concerns? When do you plan to tag a release candidate?

These extends the fixes commited started November 2019 https://github.com/apache/incubator-mxnet/commits/master/python/mxnet/gluon/contrib/estimator.

[liuzh47]

Thank you for the improvement! 2 concerns.
Also could you point an example which require custom gradient handler? (gradient clipping or aggregation)

Thank u for the review.

For the gradient update example, one use case of using gradient accumulation appears when training a transformer. (https://github.com/dmlc/gluon-nlp/blob/master/scripts/machine_translation/train_transformer.py#L320) Because the size of parameters in the transformer network is too large, we can compute gradient for a small batch of data examples during each iteration. In this case, the gradient is updated periodically on the weight parameters.

[codecov-io]

Codecov Report

Merging #16900 into master will decrease coverage by 0.11%.
The diff coverage is 80.95%.

@@            Coverage Diff             @@
##           master   #16900      +/-   ##
==========================================
- Coverage   67.06%   66.95%   -0.12%     
==========================================
  Files         271      264       -7     
  Lines       30173    29804     -369     
  Branches     4477     4400      -77     
==========================================
- Hits        20237    19954     -283     
+ Misses       8648     8563      -85     
+ Partials     1288     1287       -1

Impacted Files	Coverage Δ
python/mxnet/gluon/contrib/estimator/estimator.py	92.46% <80%> (-0.33%)	⬇️
...hon/mxnet/gluon/contrib/estimator/event_handler.py	71.96% <81.25%> (+0.1%)	⬆️
python/mxnet/rtc.py	29.03% <0%> (-58.07%)	⬇️
python/mxnet/ndarray/_internal.py	52.17% <0%> (-39.14%)	⬇️
python/mxnet/symbol/_internal.py	56% <0%> (-36%)	⬇️
python/mxnet/gluon/rnn/rnn_layer.py	72.78% <0%> (-10.66%)	⬇️
python/mxnet/context.py	85.71% <0%> (-7.15%)	⬇️
python/mxnet/module/bucketing_module.py	73.66% <0%> (-6.7%)	⬇️
python/mxnet/util.py	68.18% <0%> (-6.07%)	⬇️
python/mxnet/tvmop.py	84.21% <0%> (-5.27%)	⬇️
... and 21 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9b49cfe...d325e87. Read the comment docs.

[eric-haibin-lin]

1. Does this require the user to specify grad_req = 'add' for gradient accumulation? When is the gradient reset to zeros?
2. Now GradientUpdateHandler is added to the list of default handlers. If users explicitly create estimator with event_handlers=[metric, logging], will GradientUpdateHandler still be included in the estimator and performing updates correctly?

[liuzh47]

1. Does this require the user to specify grad_req = 'add' for gradient accumulation? When is the gradient reset to zeros?

2. Now GradientUpdateHandler is added to the list of default handlers. If users explicitly create estimator with `event_handlers=[metric, logging]`, will `GradientUpdateHandler` still be included in the estimator and performing updates correctly?

1. The gradient update handler is for general gradient update. When we implement gradient accumulation handler in the future, we will inherit this class.

2. Yes, it is added as a default handler. Please refer to (https://github.com/apache/incubator-mxnet/blob/5878451c4b476d8771d93000ff5d51b81071997d/tests/python/unittest/test_gluon_estimator.py#L369).

[roywei]

LGTM, re-triggered CI