Keras accuracy (metrics)

Keras accuracy (metrics)

A metric is a function that is used to judge the performance of your model. Metric functions are to be supplied in the metrics parameter when a model is compiled. - Keras documentation

Keras provides 5 different metrics for performance judging. These metrics use the following arguments:

• y_true: True labels
• y_pred: Predictions

Method explanation

These methods are defined in the Keras documentation

• K.mean(x, axis=None) -> returns the mean of a tensor alongside the specified axis
• K.cast(x, dtype) -> Casts a tensor to a different dtype (e.g. 'float64')
• K.equal(x,y) -> Compares two tensors on equality
• K.round(x) -> Rounds with "half to eleven" strategy (0.4 -> 0, 0.5 -> 1)
• K.argmax(x, axis=-1) -> Returns the index of the maximum value x along an axis.
• K.max(x, axis=None) -> Returns the maximum value of a tensor
• K.floatx -> Returns the default float type, as a string. (e.g. 'float16', 'float32', 'float64').
• K.in_top_k(predictions, targets, k) -> Returns whether the targets are in the top k predictions.

Binary accuracy:

In the following sourcecode from the github repository, we can see K.round(y_pred). The value of y_pred will be rounded up (to 1) if the y_pred is higher or equal 0.5. So we can say that the threshold is 0.5 and everthing above or equal this threshold will be considered as correct.

def binary_accuracy(y_true, y_pred):
return K.mean(K.equal(y_true, K.round(y_pred)), axis=-1)

Line 26-27 in Keras source (Github).

Categorical accuracy:

This metric is using the K.argmax method to compare the index of the maximal true value with the index of the maximal predicted value. In other words "how often predictions have maximum in the same spot as true values".

def categorical_accuracy(y_true, y_pred):
    return K.cast(K.equal(K.argmax(y_true, axis=-1),
                          K.argmax(y_pred, axis=-1)),
K.floatx())

Line 30-33 in Keras source (Github).

Sparse categorical accuracy:

This metric checks to see if the maximal true value is equal to the index of the maximal predicted value.

def sparse_categorical_accuracy(y_true, y_pred):
    return K.cast(K.equal(K.max(y_true, axis=-1),
                          K.cast(K.argmax(y_pred, axis=-1), K.floatx())),
                  K.floatx())

Line 36-39 in Keras source (Github).

Top-k categorical accuracy:

Top-k categorical accuracy is almost similar to categorical accuracy. Here we calculate how often target class is within the top-k predictions.

def top_k_categorical_accuracy(y_true, y_pred, k=5):
return K.mean(K.in_top_k(y_pred, K.argmax(y_true, axis=-1), k), axis=-1)

Line 42-43 in Keras source (Github).

Sparse_top_k categorical accuracy:

Another case for top_k accuracy is the sparse_top_k categorical accuracy. This is a mixture of top_k_categorical_accuracy and Sparse categorical accuracy. We calculate how often the index of the maximal true value is in the top k predictions.

def sparse_top_k_categorical_accuracy(y_true, y_pred, k=5):
return K.mean(K.in_top_k(y_pred, K.cast(K.max(y_true, axis=-1), 'int32'), k), axis=-1)

Line 46-47 in Keras source (Github).