Skip to content

Commit

Permalink
Merge pull request #127 from JuliaML/single-interface
Browse files Browse the repository at this point in the history 
Stick to a single interface
  • Loading branch information
@juliohm
juliohm committed Apr 10, 2020
2 parents 3ff02d4 + ec34ba8 commit d9a286a
Show file tree
Hide file tree
Showing 6 changed files with 7 additions and 380 deletions.
179 changes: 2 additions & 177 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -40,179 +40,6 @@ observations. In the case of arrays a user additionally has the
ability to define if and how element-wise results are averaged or
summed over.

## Example

The following code snippets show a simple "hello world" scenario
of how this package can be used to work with loss functions in
various ways.

```julia
using LossFunctions
```

All the concrete loss "functions" that this package provides are
actually defined as immutable types, instead of native Julia
functions. We can compute the value of some type of loss using
the function `value()`. Let us start with an example of how to
compute the loss for a group of three of observations. By default
the loss will be computed element-wise.

```julia
julia> true_targets = [ 1, 0, -2];

julia> pred_outputs = [0.5, 2, -1];

julia> value(L2DistLoss(), true_targets, pred_outputs)
# 3-element Array{Float64,1}:
# 0.25
# 4.0
# 1.0
```

Alternatively, one can also use an instance of a loss just like
one would use any other Julia function. This can make the code
significantly more readable while not impacting performance, as
it is a zero-cost abstraction (i.e. it compiles down to the same
code).

```julia
julia> loss = L2DistLoss()
# LossFunctions.LPDistLoss{2}()

julia> loss(true_targets, pred_outputs)
# 3-element Array{Float64,1}:
# 0.25
# 4.0
# 1.0

julia> loss(1, 0.5f0) # single observation
# 0.25f0
```

If you are not actually interested in the element-wise results
individually, but some accumulation of those (such as mean or
sum), you can additionally specify an average mode. This will
avoid allocating a temporary array and directly compute the
result.

```julia
julia> value(L2DistLoss(), true_targets, pred_outputs, AvgMode.Sum())
# 5.25

julia> value(L2DistLoss(), true_targets, pred_outputs, AvgMode.Mean())
# 1.75
```

Aside from these standard unweighted average modes, we also
provide weighted alternatives. These expect a weight-factor for
each observation in the predicted outputs and so allow to give
certain observations a stronger influence over the result.

```julia
julia> value(L2DistLoss(), true_targets, pred_outputs, AvgMode.WeightedSum([2,1,1]))
# 5.5

julia> value(L2DistLoss(), true_targets, pred_outputs, AvgMode.WeightedMean([2,1,1]))
# 1.375
```

We do not restrict the targets and outputs to be vectors, but
instead allow them to be arrays of any arbitrary shape. The shape
of an array may or may not have an interpretation that is
relevant for computing the loss. It is possible to explicitly
specify which dimension denotes the observations. This is
particularly useful for multivariate regression where one could
want to accumulate the loss per individual observation.

```julia
julia> A = rand(2,3)
# 2×3 Array{Float64,2}:
# 0.0939946 0.97639 0.568107
# 0.183244 0.854832 0.962534

julia> B = rand(2,3)
# 2×3 Array{Float64,2}:
# 0.0538206 0.77055 0.996922
# 0.598317 0.72043 0.912274

julia> value(L2DistLoss(), A, B, AvgMode.Sum())
# 0.420741920634

julia> value(L2DistLoss(), A, B, AvgMode.Sum(), ObsDim.First())
# 2-element Array{Float64,1}:
# 0.227866
# 0.192876

julia> value(L2DistLoss(), A, B, AvgMode.Sum(), ObsDim.Last())
# 3-element Array{Float64,1}:
# 0.1739
# 0.060434
# 0.186408
```

All these function signatures of `value` also apply for computing
the derivatives using `deriv` and the second derivatives using
`deriv2`.

```julia
julia> deriv(L2DistLoss(), true_targets, pred_outputs)
# 3-element Array{Float64,1}:
# -1.0
# 4.0
# 2.0

julia> deriv2(L2DistLoss(), true_targets, pred_outputs)
# 3-element Array{Float64,1}:
# 2.0
# 2.0
# 2.0
```

For computing the first and second derivatives we additionally
expose a convenience syntax which allows for a more math-like
look of the code.

```julia
julia> loss = L2DistLoss()
# LossFunctions.LPDistLoss{2}()

julia> loss'(true_targets, pred_outputs)
# 3-element Array{Float64,1}:
# -1.0
# 4.0
# 2.0

julia> loss''(true_targets, pred_outputs)
# 3-element Array{Float64,1}:
# 2.0
# 2.0
# 2.0
```

Additionally, we provide mutating versions for the subset of
methods that return an array. These have the same function
signatures with the only difference of requiring an additional
parameter as the first argument. This variable should always be
the preallocated array that is to be used as storage.

```julia
julia> buffer = zeros(3)
# 3-element Array{Float64,1}:
# 0.0
# 0.0
# 0.0

julia> deriv!(buffer, L2DistLoss(), true_targets, pred_outputs)
# 3-element Array{Float64,1}:
# -1.0
# 4.0
# 2.0
```

Note that this only shows a small part of the functionality this
package provides. For more information please have a look at
the documentation.

## Documentation

Check out the **[latest documentation](https://JuliaML.github.io/LossFunctions.jl/stable)**
Expand Down Expand Up @@ -254,12 +81,10 @@ search: HingeLoss L2HingeLoss L1HingeLoss SmoothedL1HingeLoss

## Installation

This package is registered in `METADATA.jl` and can be installed
as usual
Get the latest stable release with Julia's package manager:

```julia
import Pkg
Pkg.add("LossFunctions")
] add LossFunctions
```

## License
Expand Down
15 changes: 0 additions & 15 deletions docs/src/user/interface.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -282,21 +282,6 @@ deriv2(::SupervisedLoss, ::AbstractArray, ::AbstractArray)
deriv2!(::AbstractArray, ::SupervisedLoss, ::AbstractArray, ::AbstractArray)
```

## Function Closures

In some circumstances it may be convenient to have the loss
function or its derivative as a proper Julia function. Instead of
exporting special function names for every implemented loss (like
`l2distloss(...)`), we provide the ability to generate a true
function on the fly for any given loss.

```@docs
value_fun(::SupervisedLoss)
deriv_fun(::SupervisedLoss)
deriv2_fun(::SupervisedLoss)
value_deriv_fun(::SupervisedLoss)
```

## Properties of a Loss

In some situations it can be quite useful to assert certain
Expand Down
21 changes: 3 additions & 18 deletions src/LossFunctions.jl
View file
Original file line number Diff line number Diff line change
Expand Up @@ -33,11 +33,10 @@ export

value,
value!,
deriv,
deriv!,
deriv2,
deriv2!,
value_fun,
deriv_fun,
deriv2_fun,
value_deriv_fun,

ZeroOneLoss,
LogitMarginLoss,
Expand Down Expand Up @@ -92,18 +91,4 @@ include("supervised/io.jl")

include("deprecated.jl")

# allow using some special losses as function
(loss::ScaledSupervisedLoss)(args...) = value(loss, args...)
(loss::WeightedBinaryLoss)(args...) = value(loss, args...)

# allow using SupervisedLoss as function
for T in filter(isconcretetype, subtypes(SupervisedLoss))
@eval (loss::$T)(args...) = value(loss, args...)
end

# allow using MarginLoss and DistanceLoss as function
for T in union(subtypes(DistanceLoss), subtypes(MarginLoss))
@eval (loss::$T)(args...) = value(loss, args...)
end

end # module
99 changes: 2 additions & 97 deletions src/supervised/supervised.jl
View file
Original file line number Diff line number Diff line change
@@ -1,99 +1,5 @@
Base.Broadcast.broadcastable(l::SupervisedLoss) = Ref(l)

# --------------------------------------------------------------
# convenience closures

"""
value_fun(loss::SupervisedLoss) -> Function
Returns a new function that computes the [`value`](@ref) for the
given `loss`. This new function will support all the signatures
that [`value`](@ref) does.
```jldoctest
julia> f = value_fun(L2DistLoss());
julia> f(-1.0, 3.0) # computes the value of L2DistLoss
16.0
julia> f.([1.,2], [4,7])
2-element Array{Float64,1}:
9.0
25.0
```
"""
@inline function value_fun(l::SupervisedLoss)
_value(args...) = value(l, args...)
_value
end

"""
deriv_fun(loss::SupervisedLoss) -> Function
Returns a new function that computes the [`deriv`](@ref) for the
given `loss`. This new function will support all the signatures
that [`deriv`](@ref) does.
```jldoctest
julia> g = deriv_fun(L2DistLoss());
julia> g(-1.0, 3.0) # computes the deriv of L2DistLoss
8.0
julia> g.([1.,2], [4,7])
2-element Array{Float64,1}:
6.0
10.0
```
"""
@inline function deriv_fun(l::SupervisedLoss)
_deriv(args...) = deriv(l, args...)
_deriv
end

"""
deriv2_fun(loss::SupervisedLoss) -> Function
Returns a new function that computes the [`deriv2`](@ref) (i.e.
second derivative) for the given `loss`. This new function will
support all the signatures that [`deriv2`](@ref) does.
```jldoctest
julia> g2 = deriv2_fun(L2DistLoss());
julia> g2(-1.0, 3.0) # computes the second derivative of L2DistLoss
2.0
julia> g2.([1.,2], [4,7])
2-element Array{Float64,1}:
2.0
2.0
```
"""
@inline function deriv2_fun(l::SupervisedLoss)
_deriv2(args...) = deriv2(l, args...)
_deriv2
end

"""
value_deriv_fun(loss::SupervisedLoss) -> Function
Returns a new function that computes the [`value_deriv`](@ref)
for the given `loss`. This new function will support all the
signatures that [`value_deriv`](@ref) does.
```jldoctest
julia> fg = value_deriv_fun(L2DistLoss());
julia> fg(-1.0, 3.0) # computes the second derivative of L2DistLoss
(16.0, 8.0)
```
"""
@inline function value_deriv_fun(l::SupervisedLoss)
_value_deriv(args...) = value_deriv(l, args...)
_value_deriv
end

# --------------------------------------------------------------
# non-exported types

Expand Down Expand Up @@ -496,16 +402,15 @@ for (FUN, DESC, EXAMPLE) in (
numbers::AbstractArray{T,N},
::AggMode.Mean) where {T,N}
nrm = 1 / length(numbers)
S = typeof(($FUN)(loss, one(T)) * one(nrm))
mapreduce(x -> loss(x) * nrm, +, numbers)::S
mapreduce(x -> ($FUN)(loss, x) * nrm, +, numbers)
end

# Compute the sum (returns a Number)
function ($FUN)(
loss::$KIND,
numbers::AbstractArray{T,N},
::AggMode.Sum) where {T,N}
mapreduce(loss, +, numbers)
sum(x -> ($FUN)(loss, x), numbers)
end

# Compute the total weighted mean (returns a Number)
Expand Down
Loading

0 comments on commit d9a286a

Please sign in to comment.