regularization on biases

[GuangQin1995]

if regularizer is not None:
regularizers = sum([tf.nn.l2_loss(variable) for variable in self.variables])
loss += (regularizer * regularizers)
it seems like that you have regularization on biases, as the self.variables included the biases
` variables = []
for w1,w2 in weights:
variables.append(w1)
variables.append(w2)

for b1,b2 in biases:
    variables.append(b1)
    variables.append(b2)`

[jakeret]

Yes you're right. Have you checked if it makes a big difference?

According to deeplearning.stanford.edu/ :Applying weight decay to the bias units usually makes only a small difference to the final network.

But I might be worth investigating

[meijie0401]

I think when batchsize is not 1, we should divide 'regularizers' by 2batch size just like following. What's your idea?
regularizers = sum([tf.nn.l2_loss(variable) for variable in self.variables])//(2batch_size)

[jakeret]

Sorry for the very late reply.
The power of the regularizer is a hyperparameter like the batch size we can cover their relation in the param search instead of having a it implicitly in the code, don't we?