Small inconsistency between comment and code in mnist_1.0_softmax.py lines 65,66

[gammaguy]

I just watched your Tensorflow and deep learning - without a PhD by Martin Görner video. You are an amazing presenter and made a difficult topic much clearer for me. Thank you.

I was working through the code and noticed that in mnist_1.0_softmax.py lines 65 you multiply by 1000

cross_entropy = -tf.reduce_mean(Y_ * tf.log(Y)) * 1000.0 # normalized for batches of 100 images,

but in the comment on line 66 you have:

# *10 because "mean" included an unwanted division by 10

I believe you multiplied by 1000.0 instead of 10.0 to additionally make the error larger for the visual graphing but you didn't explicitly state that. For someone just beginning Tensorflow and is trying to make sure they really understand how it all works perhaps you could make the comment correspond to the code.

Also why didn't you use tf.reduce_sum then you would not need to multiply by 10?

Thank you again.

[martin-gorner]

indeed, this scaling factor is not important, and mainly used to display the training and test loss on the same scale.

reduce_mean computes an average. The population is the number of images in the batch * the number of elements in the tensor, that's 100 * 10 = 1000. So reduce_mean will divide by 1000. I multiply by 1000 to counter that effect and I get the sum of the components * 100, i.e. the loss for 100 images.
At test time, the population for the average is 10000*10 so that is what reduce_mean will divide by. I multiply by 10, to get the average loss per image and then 100 (total: 1000) to get the loss for 100 images.
Since both training and test loss are for 100 images, putting them on the same graph makes sense.