loss.rst

Loss functions

Loss functions measure the disagreement between the true label y ∈ { − 1, 1} and the prediction.

Loss functions implement the following main methods:

value(l::Loss)

Compute the value of the loss.

gradient(l::Loss)

Compute the gradient of the loss.

The following loss functions are implemented:

Logistic(w::Vector, X::Matrix, y::Vector)

Return a vector of the logistic loss evaluated for all given training instances $\bf X$ and the labels $\bf y$

$$\ell({\bf w}, {\bf x}, y)&=\log(1+exp(-y{\bf x}^T{\bf w})),$$

where ${\bf w}$ is the weight vector of the decision function.

Note

The logistic loss corresponds to a likelihood function under an exponential family assumption of the class-conditional distributions $p({\bf x}|y;{\bf w})$.

Squared(w::Vector, X::Matrix, y::Vector)

Return a vector of the squared loss evaluated for all given training instances $\bf X$ and the labels $\bf y$

$$\ell({\bf w}, {\bf x}, y)&=(y-{\bf x}^T{\bf w})^2,$$

where ${\bf w}$ is the weight vector of the decision function.

Hinge(w::Vector, X::Matrix, y::Vector)

Return a vector of the hinge loss evaluated for all given training instances $\bf X$ and the labels $\bf y$

$$\ell({\bf w}, {\bf x}, y)&=\max(0, 1-y{\bf x}^T{\bf w}),$$

where ${\bf w}$ is the weight vector of the decision function.

Note

The hinge loss corresponds to a max-margin assumption.