shouldn't we use moving_mean and moving_bar in training mode instead of batch mean and var？

[jonhe88]

            if 'train' in stats_mode:
                xn = tf.nn.batch_normalization(
                    list_input[i], mean, var, beta, gamma, bn_epsilon)
                if tf.get_variable_scope().reuse or 'gather' not in stats_mode:
                    list_output.append(xn)
                else:
                    # gather stats and it is the main gpu device.
                    xn = update_bn_ema(xn, mean, var, moving_mean, moving_var, bn_ema)
                    list_output.append(xn)
            else:
                xn = tf.nn.batch_normalization(
                    list_input[i], moving_mean, moving_var, beta, gamma, bn_epsilon)
                list_output.append(xn)

[holyseven]

What is your argument of using moving_mean and moving_bar?

[jonhe88]

In the third line of the pasted codes, the batch mean and var of the current batch are used for batchnorm computation. I think we should use moving mean and var in BN

[holyseven]

Why do you think we should use moving mean and var in BN during training?

[jonhe88]

You're right, this is the standard procedure, moving mean during training has defects