is_training=False

[wh0]

Thanks for releasing your pretrained weights checkpoint. I'm trying to use them to run the decompiler. When I set is_training=False in the call to tflib.network.im2latex_cnn, it outputs the wrong answer. Any advice on how to run it not in training mode?

For example, for 3e679e114e.png, I get

#START \begin{array} { l } & { \scriptscriptstyle } & { \bf } & { \bf } & { \bf } & { \bf } & { \bf } & { \bf } & { \bf } & { \bf } & { \bf } & { \bf } & { \bf } & { \bf } & { \bf } & { \bf } & { \bf } & { \bf } & { \bf } & { \bf } & { \bf } & { \bf } & { \bf } & { \bf } & { \bf } & { \bf } & { \bf } & { \bf } & { \bf } & { \bf } & { \bf } & { \bf } & { \bf } & { \bf } [truncated]

(originally was #START f _ { ( x , x _ { 0 } ) } ^ { c , L } \equiv f ^ { c } ( x , x _ { 0 } ) #END)

[ritheshkumar95]

Hey,

This issue seems peculiar to me.

Are you saying is_training=True returns the correct answer but is_training=False returns the wrong one?

[wh0]

Are you saying is_training=True returns the correct answer but is_training=False returns the wrong one?

Yes

[wh0]

I've found that the batch_norm wasn't working right in the training that went into the released weight files. The moving_mean and moving_variance tensors have initial values. We're going to try to update the code to carry out the batch_norm update operations during training. cc @mitar

[ritheshkumar95]

Right,

Sorry. I just noticed in tflib.contrib that batch_norm by default doesn't update moving means / averages. I thought it does by default.

Check this https://www.tensorflow.org/api_docs/python/tf/contrib/layers/batch_norm

[mingchen62]

I encountered the same issue with is_training=False. Not sure if @wh0 get a chance to update the code.

[wh0]

@mingchen62 I ended up not altering the training process at all.

[mingchen62]

All right. According to document, it looks like we will need to update "attention.py"
adding something like this:

update_ops = tf.get_collection(tf.GraphKeys.UPDATE_OPS)
with tf.control_dependencies(update_ops):
train_step =....

I will give it a try