ValueError: Rank mismatch: Rank of labels (received 2) should equal rank of logits minus 1 (received 2)

[Elysium1436]

I'm trying to make a custom loss function based on the sparse_bi_tempered_logistic_loss( ) function of this repository.

T_1 = 0.2
T_2 = 1.2
SMOOTH_FRACTION = 0.01
N_ITER = 5

def bi_tempered_loss(y_pred,y_true):
        return sparse_bi_tempered_logistic_loss(y_pred,y_true,T_1,T_2)

my_model.compile(loss=bi_tempered_loss, optimizer=keras.optimizers.Adam(lr=1e-4), metrics=['accuracy'])

The labels are integers from 0 to 4.

However, the following errors occurs when I fit my model:

ValueError: in user code:

    /opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py:806 train_function  *
        return step_function(self, iterator)
    <ipython-input-44-efd725700411>:12 bi_tempered_loss  *
        return sparse_bi_tempered_logistic_loss(y_pred,y_true,T_1,T_2)
    /kaggle/working/loss.py:421 sparse_bi_tempered_logistic_loss  *
        loss_values = tf.cond(
    /opt/conda/lib/python3.7/site-packages/tensorflow/python/util/dispatch.py:201 wrapper  **
        return target(*args, **kwargs)
    /opt/conda/lib/python3.7/site-packages/tensorflow/python/util/deprecation.py:507 new_func
        return func(*args, **kwargs)
    /opt/conda/lib/python3.7/site-packages/tensorflow/python/ops/control_flow_ops.py:1180 cond
        return cond_v2.cond_v2(pred, true_fn, false_fn, name)
    /opt/conda/lib/python3.7/site-packages/tensorflow/python/ops/cond_v2.py:85 cond_v2
        op_return_value=pred)
    /opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py:986 func_graph_from_py_func
        func_outputs = python_func(*func_args, **func_kwargs)
    /opt/conda/lib/python3.7/site-packages/tensorflow/python/util/dispatch.py:201 wrapper
        return target(*args, **kwargs)
    /opt/conda/lib/python3.7/site-packages/tensorflow/python/ops/nn_ops.py:4084 sparse_softmax_cross_entropy_with_logits
        (labels_static_shape.ndims, logits.get_shape().ndims))

    ValueError: Rank mismatch: Rank of labels (received 2) should equal rank of logits minus 1 (received 2).

I've tried using class-based custom loss implementation, but it gave the same error. Am I missing something?

[Witalia008]

I had a similar problem, and from what I was able to research online, this seems to do the trick:

def bi_tempered_loss(y_true, y_pred):
    return sparse_bi_tempered_logistic_loss(y_pred, K.cast(K.reshape(y_true, (-1,)), "int32"), 0.2, 1.2)

Seems like labels were of shape [None, 1] whereas the function was expecting shape [None].

However, I later get a different symptom of some issue:

Exception has occurred: InvalidArgumentError
 Incompatible shapes: [16,5] vs. [16]
	 [[{{node gradient_tape/bi_tempered_loss/sparse_bitempered_logistic/cond/StatelessIf/else/_15/gradient_tape/bi_tempered_loss/sparse_bitempered_logistic/cond/gradients/bi_tempered_loss/sparse_bitempered_logistic/cond/IdentityN_grad/Mul_2}}]] [Op:__inference_train_function_18841]

Not sure what to do about it... 5 is the number of classes, so somewhere the sparsity of labels is not handled or something..

[Witalia008]

Here's what I got working:

def bi_tempered_loss(y_true, y_pred):
    y_true = K.cast(K.reshape(y_true, (-1,)), "int64")
    labels = K.one_hot(y_true, N_CLASSES)
    return bi_tempered_logistic_loss(y_pred, labels, T1, T2)