fluid下缺乏正则项示例

[jshower]

你好，我在进行fulid下的实验时，没有找到fulid下使用正则项的例子。能否提供一个使用正则项的例子，感觉还是很有必要的。
我试着用了一下正则项

 optimizer = fluid.optimizer.Momentum(
        momentum=0.0,
        learning_rate=2e-4,
        regularization=fluid.regularizer.L2DecayRegularizer(regularization_coeff=0.1)
        )

结果爆出了后面的错误。

Traceback (most recent call last):
  File "train.py", line 136, in <module>
    num_passes=1000)
  File "train.py", line 72, in main
    sgd_optimizer.minimize(avg_cost)
  File "/usr/local/lib/python2.7/dist-packages/paddle/v2/fluid/optimizer.py", line 234, in minimize
    self.regularization)
  File "/usr/local/lib/python2.7/dist-packages/paddle/v2/fluid/regularizer.py", line 65, in append_regularization_ops
    outputs={"Out": grad})
  File "/usr/local/lib/python2.7/dist-packages/paddle/v2/fluid/framework.py", line 801, in append_op
    op = Operator(self, op_desc, *args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/paddle/v2/fluid/framework.py", line 501, in __init__
    self.desc.infer_shape(self.block.desc)
paddle.v2.fluid.core.EnforceNotMet: enforce x_dim.size() >= y_dim.size() failed, 1 < 2

[chengduoZH]

我试着用了一下正则项
optimizer = fluid.optimizer.Momentum(
momentum=0.0,
learning_rate=2e-4,
regularization=fluid.regularizer.L2DecayRegularizer(regularization_coeff=0.1)
)

您好，这样写是可以的，看上去报错并不是因为使用了正则项

[jshower]

添加了regularization=fluid.regularizer.L2DecayRegularizer(regularization_coeff=0.1)这一行后就会报错，去掉就能正常运行。应该还是正则项的问题吧。能否提供一个包含正则项了fluid下的示例，这样也可以作为参考。

[chengduoZH]

@jshower 你跑的是哪个模型，我这边复现一下

[jshower]

我是在自己写的一个脚本上出现的这个问题的，但是我发现我在paddle给出的例子上添加正则项，也会出现同样的问题。比如说下面这个例子
Paddle/python/paddle/fluid/tests/book/test_machine_translation.py
在184行加上正则项，
optimizer = fluid.optimizer.Adagrad(learning_rate=1e-4, regularization=fluid.regularizer.L2DecayRegularizer(regularization_coeff=0.1))
结果也会报错，如下：

[[0L, 7L, 14L], [0L, 2L, 5L, 9L, 14L, 21L, 30L, 39L, 41L, 44L, 48L, 53L, 60L, 69L, 78L]]
.pass_id=0 batch=0 avg_cost=[10.308913]
pass_id=0 batch=1 avg_cost=[10.308919]
pass_id=0 batch=2 avg_cost=[10.308882]
pass_id=0 batch=3 avg_cost=[10.308773]
pass_id=0 batch=4 avg_cost=[10.308762]
.[[0L, 7L, 14L], [0L, 3L, 7L, 13L, 20L, 28L, 37L, 46L, 49L, 53L, 59L, 66L, 74L, 83L, 92L]]
.Es.s.
======================================================================
ERROR: test_cpu_sparse_train (__main__.TestMachineTranslation)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_machine_translation.py", line 309, in f
    train_main(use_cuda, is_sparse)
  File "test_machine_translation.py", line 185, in train_main
    optimize_ops, params_grads = optimizer.minimize(avg_cost)
  File "/usr/local/lib/python2.7/dist-packages/paddle/fluid/optimizer.py", line 227, in minimize
    self.regularization)
  File "/usr/local/lib/python2.7/dist-packages/paddle/fluid/regularizer.py", line 65, in append_regularization_ops
    outputs={"Out": grad})
  File "/usr/local/lib/python2.7/dist-packages/paddle/fluid/framework.py", line 815, in append_op
    op = Operator(block=self, desc=op_desc, *args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/paddle/fluid/framework.py", line 494, in __init__
    self.desc.infer_shape(self.block.desc)
EnforceNotMet: enforce x_dim.size() >= y_dim.size() failed, 1 < 2
Rank of first input must >= rank of second input. at [/paddle/Paddle/paddle/fluid/operators/elementwise_op.h:38]

错误类型和之前在我的脚本上直接改动是相同的。

[jshower]

这个问题有解决方案吗，我在Paddle/python/paddle/fluid/tests/book/test_label_semantic_roles.py发现存在同样的问题。

[chengduoZH]

@jshower 这里应该是有个bug，现在还在找，估计这两天能修复。

[chengduoZH]

@jshower 您好，问题已经定位到，原因就是test_machine_translation.py里面有的参数是sparse的，所以在做正则化的时候就会出问题。现在正在修复。

[jshower]

辛苦修复后在这个issue里告知一下，谢谢！

[chengduoZH]

@jshower 好的，已经开始review了

[chengduoZH]

@jshower 现在可以用了，test_machine_translation.py 里面也加了正则项

Paddle/python/paddle/fluid/tests/book/test_machine_translation.py

Lines 184 to 187 in 5a159f3

	optimizer = fluid.optimizer.Adagrad(
	learning_rate=1e-4,
	regularization=fluid.regularizer.L2DecayRegularizer(
	regularization_coeff=0.1))