Skip to content
wrapping a keras optimizer to implement gradient accumulation
Branch: master
Clone or download
Latest commit b898256 Jul 8, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
README.md Update README.md Jul 8, 2019
README_en.md Update README_en.md Jul 8, 2019
accum_optimizer.py Update accum_optimizer.py Jul 8, 2019
mnist_mlp_example.py Create mnist_mlp_example.py Jul 8, 2019

README.md

[中文|English]

为Keras实现梯度累积版优化器

特点

继承Optimizer类,包装原有优化器,实现梯度累积功能。能够无缝对接原有优化器,不需要重写优化器。

用法

如下例子等价于直接使用batch_size=100的Adam优化器(代价就是你跑了10个epoch,实际上只相当于batch_size=100跑了1个epoch):

opt = AccumOptimizer(Adam(), 10) # 10是累积步数
model.compile(loss='mse', optimizer=opt)
model.fit(x_train, y_train, epochs=10, batch_size=10)

读者也可以直接跑一跑mnist_mlp_example.py

链接

https://kexue.fm/archives/6794

You can’t perform that action at this time.