Multiclass classification has one too many latent GPs?

[EvanBaker57]

In the multiclass classification model, using a robustmax likelihood, there are as many latent GPs (num_latent) as there are classes (C).
https://gpflow.github.io/GPflow/2.5.1/notebooks/advanced/multiclass_classification.html

When one of the latent GPs is the largest, then that class is predicted.
Shouldn't num_latent be equal to C-1 however? As one of the GPs can be set to 0 (or any other constant) instead, without loss of generalisability.

Is this not correct, and if it is correct, is there any easy way to implement this?

Thanks,
Evan

[st--]

Yes, technically with C latent GPs it's an overparametrized model, you can of course implement a likelihood yourself that just requires C-1 latent GPs (e.g. by simply adding a constant zero output as you suggest). I don't know if that necessarily makes for a better model though. If one of your classes is "don't know", then simply fixing the latent output for that to 0 might be fine. If all your classes are 'equally valid', then you might end up with a more easily interpretable model when you have one latent GP per class. Which one is easier to optimize might depend on the data.

In any case, I would recommend Softmax likelihood (with stochastic optimization) over Robustmax, it seems to work better in practice.

[st--]

(It's not a bug, it's a modelling choice.)