fix adagrad error

[chengxiaohua1105]

fix the bug of adagrad algorithm.

The adagrad algorithm is implemented as follows:

for (size_t i = 0; i < param.size(); ++i) {
accum_g[i] += grad[i] * grad[i];
param[i] += learning_rate * grad[i] / std::sqrt(accum_g[i] + epsilon_) + learning_rate * decay_ * param[i];
}

it should be implemented as follows

for (size_t i = 0; i < param.size(); ++i) {
accum_g[i] += grad[i] * grad[i];
param[i] -= learning_rate * grad[i] / std::sqrt(accum_g[i] + epsilon_) + learning_rate * decay_ * param[i];
}

[tonyyang-svail]

@abhinavarora please confirm this. It looks suspicious to me.

[abhinavarora]

The correction from += to -= is definitely correct. However, the way the weight decay is being added is mathematically incorrect. We should be modifying the gradient first and then using the modified gradient for the updates. However, I checked that this implementation is incorrect in all optimizers. We can refer to http://pytorch.org/docs/0.1.12/optim.html for more details.
@chengxiaohua1105 Should we open an issue for all optimizers?

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.

---

litao seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[luotao1]

close due to paddle/optimizer/adagrad_optimizer.cc are not in the develop branch now.