Binary_crossentropy producing negative loss values

[amjass12]

Hi all,

I have a built a classifier to learn various features of data. I am now using the functional API as i would also like to create a predictor of a class in my data at the same time. so as oppose to having 30 classes to classify (as in the normal sequential model) i would also like to use the same input to see if it can predict one class in the data. The code is as follows:

le = LabelEncoder()
metadataencode= metadata.apply(le.fit_transform)
metadataencode


##one-hot encode data
cat1=to_categorical(metadataencode['cat1'],num_classes=12)
cat2=to_categorical(metadataencode['cat2'],num_classes=7)
cat3= to_categorical(metadataencode['cat3'], num_classes=2)
cat4=to_categorical(metadataencode['cat4'], num_classes=5)
cat5=to_categorical(metadataencode['cat5'], num_classes=2)
cat6=to_categorical(metadataencode['cat6'],num_classes=3)
cat7=to_categorical(metadataencode['cat7'],num_classes=6)
cat8=to_categorical(metadataencode['cat8'],num_classes=2)

##remove unknown class categories and make them '0'--

cat5=np.delete(cat5, 4 ,axis=1)
cat6=np.delete(cat6, 0 ,axis=1)
cat7=np.delete(cat7, 0 ,axis=1)

#trainingtarget for classification
trainingtarget = np.concatenate((cat1,cat2,cat3,cat4,cat5,cat6,cat7, cat8), 
axis=1)
trainingtarget ##multi-class multi label array for classification


####create encoding for classifying ONE class based on all features in the data

encoder = LabelEncoder()
encoder.fit(metadata['cat5'])
encoded_Y = encoder.transform(metadata['cat5'])## there are 5 classes to chose from

Then build and compile model. one input with two outputs:: one for the classification, and one to predict based on the data if it can learn which class it belongs based on all the data (output for this is 1)

X_train, X_test, y_train, y_test, cat5train, cat5test = train_test_split(normCountsscale1,
trainingtarget, encoded_Y, train_size = 0.8)



##input

mainInput= tf.keras.Input([20501,], name= 'main_input')
mainInputlayer= tf.keras.layers.Dense(units=400, activation='relu', name='main_inputL1')(mainInput)
mainInputlayer= tf.keras.layers.Dense(units=250, activation= 'relu', name='main_inputL2')(mainInputlayer)
mainInputlayer= tf.keras.layers.Dense(units=500, activation= 'relu', name='main_inputL3')(mainInputlayer)

## branching in to different multi-task output --classification and regression

outputClass=tf.keras.layers.Dense(units=36, activation='sigmoid', name='outputClass')(mainInputlayer)
outputsingle=tf.keras.layers.Dense(units=1, activation='sigmoid', name='outputsingle')(mainInputlayer)


finalModel=tf.keras.Model(inputs=mainInput, outputs= [outputClass, outputsingle])

## model summary
finalModel.summary()

##plot model
tf.keras.utils.plot_model(finalModel, show_shapes=True)

#endregion

#region complile and fit model

finalModel.compile(optimizer='adam',
              loss='binary_crossentropy', metrics=['accuracy'])


finalModel.fit({'main_input': X_train},
              {'outputsingle': cat5train, 'outputClass': y_train},
              epochs=20, batch_size=32,
              verbose=1,
              validation_split=0.2,

i achieve good accuracy for both outputs, however, i am reaching negative loss values for the outputsingle class.. an example of the last line of epoch 20 shows:

outputClass_loss: 0.2093 - outputRegress_loss: -9.1634 - outputClass_accuracy: 0.9094 - outputRegress_accuracy: 0.659318/318 [==============================] - 1s 2ms/sample - loss: -9.2815 - outputClass_loss: 0.2075 - outputRegress_loss: -9.5086 - outputClass_accuracy: 0.9102 - outputRegress_accuracy: 0.6667 - val_loss: 9.5608 - val_outputClass_loss: 0.3463 - val_outputRegress_loss: 10.6915 - val_outputClass_accuracy: 0.8740 - val_outputRegress_accuracy: 0.6125

the loss values are all very odd. any help here would be appreciated! thanks!

[jvishnuvardhan]

@amjass12 Is this still an issue for you? Are you trying to use same loss function for both the outputs?

Please check functional API guide on keras website. Thanks!

[amjass12]

Hi @jvishnuvardhan ,

apologies - yes ths is all fine now!