Fluctuating loss/accuracy

[raghuramdr]

• [ ✓] Check that you are up-to-date with the master branch of Keras. You can update with:
  pip install git+git://github.com/fchollet/keras.git --upgrade --no-deps

• If running on TensorFlow, check that you are up-to-date with the latest version. The installation instructions can be found here.

• [ ✓] If running on Theano, check that you are up-to-date with the master branch of Theano. You can update with:
  pip install git+git://github.com/Theano/Theano.git --upgrade --no-deps

• [ ✓] Provide a link to a GitHub Gist of a Python script that can reproduce your issue (or just copy the script here if it is short).

I am using Keras to build a convnet to perform a classification task on a set of medical images. There are two classes. When I run the network, the network runs but the loss and accuracy is really strange. The loss almost always increases, with the accuracy fluctuating wildly. My code is shown below

nb_epoch = 60
batch_size = 32
nb_train_samples = 1835
nb_validation_samples = 564
nb_classes = 2

model = Sequential()
model.add(Convolution2D(16,5,5, border_mode = 'valid', subsample = (1,1), init = 'glorot_uniform',input_shape = (3,100,100)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
print 'First layer setup'
    
###########################Second conv layer#################################
model.add(Convolution2D(32,3,3,border_mode = 'same', subsample = (1,1),init = 'glorot_uniform'))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size = (2,2),strides = None))
# Output is 24x24
    
print ' Second layer setup'
 
##########################Third conv layer###################################
model.add(Convolution2D(64,3,3, border_mode = 'same', subsample = (1,1), init = 'glorot_uniform'))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size = (2,2),strides = None))
#############################################################################
# Output is 12x12
print ' Third layer setup'
    
###############################Fourth conv layer#############################
model.add(Convolution2D(128,3,3, border_mode = 'same', subsample = (1,1), init = 'glorot_uniform'))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size = (2,2),strides = None))
############################################################################# 
print 'Fourth layer setup'   

model.add(Flatten()) 
model.add(Dense(128))
model.add(Activation('relu'))
model.add(Dropout(0.75))
model.add(Dense(1))
model.add(Activation('sigmoid'))

model.compile(loss = 'binary_crossentropy',optimizer = 'rmsprop',metrics = ['accuracy'])

train_datagen = ImageDataGenerator(
        rescale=1./255,
        rotation_range = 300,
        horizontal_flip=True,
        vertical_flip = True)

test_datagen = ImageDataGenerator(rescale=1./255)

train_generator = train_datagen.flow_from_directory(
        train_data_dir,
        target_size=(img_width, img_height),
        batch_size=32,
        class_mode='binary')

validation_generator = test_datagen.flow_from_directory(
        test_data_dir,
        target_size=(img_width, img_height),
        batch_size=32,
        class_mode='binary')

model.fit_generator(
        train_generator,
        samples_per_epoch=nb_train_samples,
        nb_epoch=nb_epoch,
        validation_data=validation_generator,
        nb_val_samples=nb_validation_samples)

I have also attached a screenshot of the loss and accuracy during the training.

[stepjam]

This looks like a problem with the learning rate.
Try 0.01, 0.001, 0.0001, and see if any of those cause the loss to drop. Otherwise, it could be a problem with your data-set.

[raghuramdr]

Thanks for the reply. I also thought it was a problem with the learning rate and have tried changing it, but no such luck. When you say it is a problem with the dataset, could you please elaborate?

[stepjam]

So I just meant that sometimes it can be beneficial to go through a sample of the data to make sure it's what you expect it to be.
The dropout looks quite high -- first try removing the dropout altogether and see if the model is able to fit the training data well. If it does, then reduce the dropout to a range between 20 and 50.