Skip to content

[ML] Increase the maximum number of classes we will support #2246

@tveasey

Description

@tveasey

Because we compute the Hessian of the cross-entropy, for classification we don't scale well to many classes and have limited the maximum number to 30.

We can likely already increase that limit as a result of performance enhancements. However, to increase it significantly we need to think about how to tackle this bottleneck. A reasonable option to explore is identifying classes which are well separated in feature space. For any given class pair (i,j) for which there aren't regions where you are choosing between i and j then one could safely zero {H(i,j), H(j,i)}. This would in effect create block diagonal Hessians and we can bound the maximum block size as we do now.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions