PyTorch/TensorFlow/scikit-learn implementations of Fenchel-Young losses
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Type Name Latest commit message Commit time
Failed to load latest commit information.


Fenchel-Young losses

NumPy, PyTorch, Tensorflow and scikit-learn compatible implementations of Fenchel-Young losses [1,2].

Tsallis losses

Supported losses

  • Multinomial logistic loss
  • One-vs-all logistic loss
  • Sparsemax loss
  • Tsallis losses


scikit-learn compatible classifier:

import numpy as np
from sklearn.datasets import make_classification
from fyl_sklearn import FYClassifier

X, y = make_classification(n_samples=10, n_features=5, n_informative=3,
                           n_classes=3, random_state=0)
clf = FYClassifier(loss="sparsemax"), y)

Drop-in replacement for PyTorch losses:

import torch
from fyl_pytorch import SparsemaxLoss

# integers between 0 and n_classes-1, shape = n_samples
y_true = torch.tensor([0, 2])
# model scores, shapes = n_samples x n_classes
theta = torch.tensor([[-2.5, 1.2, 0.5],
                      [2.2, 0.8, -1.5]])
loss = SparsemaxLoss()
# loss value (caution: reversed convention compared to numpy and tensorflow)
print(loss(theta, y_true))
# predictions (probabilities) are stored for convenience
# can also recompute them from theta
# label proportions are also allowed
y_true = torch.tensor([[0.8, 0.2, 0],
                       [0.1, 0.2, 0.7]])
print(loss(theta, y_true))

Drop-in replacement for tensorflow losses:

import tensorflow as tf
from fyl_tensorflow import sparsemax_loss, sparsemax_predict

# integers between 0 and n_classes-1, shape = n_samples
y_true = tf.constant([0, 2])
# model scores, shapes = n_samples x n_classes
theta = tf.constant([[-2.5, 1.2, 0.5],
                     [2.2, 0.8, -1.5]])
# loss value
print(sparsemax_loss(y_true, theta))
# predictions (probabilities)
# label proportions are also allowed
y_true = tf.constant([[0.8, 0.2, 0],
                      [0.1, 0.2, 0.7]])
print(sparsemax_loss(y_true, theta))


Simply copy relevant files to your project.


[1]SparseMAP: Differentiable Sparse Structured Inference. Vlad Niculae, André F. T. Martins, Mathieu Blondel, Claire Cardie. In Proc. of ICML 2018. [arXiv]
[2]Learning Classifiers with Fenchel-Young Losses: Generalized Entropies, Margins, and Algorithms. Mathieu Blondel, André F. T. Martins, Vlad Niculae. [arXiv]


  • Mathieu Blondel, 2018