Help!!! The different result between train_on_batch() and test_on_batch() with the same data

[XiaotaoChen]

Hi,
I get the different result by the two functions named "train_on_batch" and "test_on_batch" feed with the same data accidentally. And each layer.trainable was set by 'False'. the don't know why. the details as follows:

1. load trained model, by follow code.

def continue_train_model(model_path):
    model = load_model(model_path)
    for layer in model.layers:
        layer.trainable = False
    return model

2. use the two functions get results feed with the same data.

train_feature_batch = get_shuffled_data(x_train_new, index_for_shuffle, ran_train_sample, batch_size)
train_label_batch = get_shuffled_data(y_train_new, index_for_shuffle, ran_train_sample, batch_size)
ran_train_sample = ran_train_sample+batch_size
temp = model.test_on_batch(train_feature_batch, train_label_batch)
print('test batch:%d, loss:%.6f, mse:%.6f, acc:%6f'%(ran_train_sample, temp[0], temp[1], temp[2]))
temp = model.train_on_batch(train_feature_batch, train_label_batch)
print('train batch:%d, loss:%.6f, mse:%.6f, acc:%6f'%(ran_train_sample, temp[0], temp[1], temp[2]))

3. it's info printed as follows:

test batch:100, loss:0.032870, mse:0.006512, acc:1.000000
train batch:100, loss:3.948478, mse:0.657277, acc:0.290000
test batch:200, loss:0.010902, mse:0.000904, acc:1.000000
train batch:200, loss:3.740743, mse:0.727334, acc:0.180000
test batch:300, loss:0.023010, mse:0.007636, acc:0.990000
train batch:300, loss:3.480110, mse:0.707776, acc:0.170000
test batch:400, loss:0.104351, mse:0.035595, acc:0.940000
train batch:400, loss:3.038500, mse:0.671575, acc:0.210000
test batch:500, loss:0.028863, mse:0.004068, acc:1.000000
train batch:500, loss:2.745351, mse:0.704359, acc:0.150000

Can anyone explain it. Thanks.

[t-ae]

trainable change is not applied if you don't recompile the model.
https://keras.io/getting-started/faq/#how-can-i-freeze-keras-layers
And there are some layers (Dropout, BatchNormalization, and so on) which behave differently in train/test phase.
Check your model doesn't have these layers.

#!/usr/bin/env python

import numpy as np
from keras.models import Sequential
from keras.layers import Dense, InputLayer

model = Sequential([
    InputLayer([10]),
    Dense(1)
])
model.trainable = False
model.compile("adam", "mse")

x = np.random.uniform(-1, 1, [8, 10])
y = np.random.uniform(-1, 1, [8])
print(model.test_on_batch(x, y))
print(model.train_on_batch(x, y))

output:

0.753879
0.753879

[XiaotaoChen]

thanks. BN and Dropout layers are both on my model. Thanks very much. @t-ae

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 30 days if no further activity occurs, but feel free to re-open a closed issue if needed.

[florentguymartin]

While running the code of t-ae, I realized that the two outputs of

print(model.test_on_batch(x, y))
print(model.train_on_batch(x, y))

are the same even with
model.trainable = True
While normally, model.train_on_batch(x, y) should be smaller than model.test_on_batch(x, y).
It looks to me like when running
model.train_on_batch
keras outputs the loss before the training.
Does anyone has a clue on what I am getting wrong here?

[maym2104]

@flotinou Keras does output the loss before training. It represents the quantity it has used to compute the gradient on the parameters. If you do test_on_batch again right after train_on_batch you should normally see the loss dropping. If you set model.trainable = False the loss remains constant.