Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 9 additions & 10 deletions docs/src/introduction/gettingstarted.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,10 +113,10 @@ avoid allocating a temporary array and directly compute the
result.

```julia-repl
julia> value(L2DistLoss(), true_targets, pred_outputs, AvgMode.Sum())
julia> value(L2DistLoss(), true_targets, pred_outputs, AggMode.Sum())
5.25

julia> value(L2DistLoss(), true_targets, pred_outputs, AvgMode.Mean())
julia> value(L2DistLoss(), true_targets, pred_outputs, AggMode.Mean())
1.75
```

Expand All @@ -126,10 +126,10 @@ each observation in the predicted outputs and so allow to give
certain observations a stronger influence over the result.

```julia-repl
julia> value(L2DistLoss(), true_targets, pred_outputs, AvgMode.WeightedSum([2,1,1]))
julia> value(L2DistLoss(), true_targets, pred_outputs, AggMode.WeightedSum([2,1,1]))
5.5

julia> value(L2DistLoss(), true_targets, pred_outputs, AvgMode.WeightedMean([2,1,1]))
julia> value(L2DistLoss(), true_targets, pred_outputs, AggMode.WeightedMean([2,1,1]))
1.375
```

Expand Down Expand Up @@ -157,7 +157,7 @@ julia> value(L2DistLoss(), A, B)
0.00161395 0.0423701 0.183882
0.172286 0.0180639 0.00252607

julia> value(L2DistLoss(), A, B, AvgMode.Sum())
julia> value(L2DistLoss(), A, B, AggMode.Sum())
0.420741920634
```

Expand All @@ -172,7 +172,7 @@ julia> value(L2DistLoss(), rand(2), rand(2,2))
0.228077 0.597212
0.789808 0.311914

julia> value(L2DistLoss(), rand(2), rand(2,2), AvgMode.Sum())
julia> value(L2DistLoss(), rand(2), rand(2,2), AggMode.Sum())
0.0860658081865589
```

Expand All @@ -182,18 +182,18 @@ multivariate regression where one could want to accumulate the
loss per individual observation.

```julia-repl
julia> value(L2DistLoss(), A, B, AvgMode.Sum(), ObsDim.First())
julia> value(L2DistLoss(), A, B, AggMode.Sum(), ObsDim.First())
2-element Array{Float64,1}:
0.227866
0.192876

julia> value(L2DistLoss(), A, B, AvgMode.Sum(), ObsDim.Last())
julia> value(L2DistLoss(), A, B, AggMode.Sum(), ObsDim.Last())
3-element Array{Float64,1}:
0.1739
0.060434
0.186408

julia> value(L2DistLoss(), A, B, AvgMode.WeightedSum([2,1]), ObsDim.First())
julia> value(L2DistLoss(), A, B, AggMode.WeightedSum([2,1]), ObsDim.First())
0.648608280735
```

Expand Down Expand Up @@ -287,4 +287,3 @@ If you encounter a bug or would like to participate in the
further development of this package come find us on Github.

- [JuliaML/LossFunctions.jl](https://github.com/JuliaML/LossFunctions.jl)

70 changes: 38 additions & 32 deletions docs/src/user/aggregate.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
```@meta
DocTestSetup = quote
using LossFunctions
end
```

# Efficient Sum and Mean

In many situations we are not really that interested in the
Expand Down Expand Up @@ -50,14 +56,14 @@ common accumulations efficiently without allocating temporary
arrays. These methods can be invoked using an additional
parameter which specifies how the values should be accumulated /
averaged. The type of this parameter has to be a subtype of
`AverageMode`.
`AggregateMode`.

## Average Modes

Before we discuss these memory-efficient methods, let us briefly
introduce the available average mode types. We provide a number
of different averages modes, all of which are contained within
the namespace `AvgMode`. An instance of such type can then be
the namespace `AggMode`. An instance of such type can then be
used as additional parameter to [`value`](@ref), [`deriv`](@ref),
and [`deriv2`](@ref), as we will see further down.

Expand All @@ -66,11 +72,11 @@ a short description of what their effect would be when used as an
additional parameter to the functions mentioned above.

```@docs
AvgMode.None
AvgMode.Sum
AvgMode.Mean
AvgMode.WeightedSum
AvgMode.WeightedMean
AggMode.None
AggMode.Sum
AggMode.Mean
AggMode.WeightedSum
AggMode.WeightedMean
```

## Unweighted Sum and Mean
Expand All @@ -82,15 +88,15 @@ broadcasted) results of [`value`](@ref), [`deriv`](@ref), and
temporary array and instead compute the result directly.

```@docs
value(::Loss, ::AbstractArray, ::AbstractArray, ::LossFunctions.AverageMode)
value(::Loss, ::AbstractArray, ::AbstractArray, ::LossFunctions.AggregateMode)
```

The exact same method signature is also implemented for
[`deriv`](@ref) and [`deriv2`](@ref) respectively.

```@docs
deriv(::Loss, ::AbstractArray, ::AbstractArray, ::LossFunctions.AverageMode)
deriv2(::Loss, ::AbstractArray, ::AbstractArray, ::LossFunctions.AverageMode)
deriv(::Loss, ::AbstractArray, ::AbstractArray, ::LossFunctions.AggregateMode)
deriv2(::Loss, ::AbstractArray, ::AbstractArray, ::LossFunctions.AggregateMode)
```

## Sum and Mean per Observation
Expand All @@ -110,7 +116,7 @@ that denotes the observations. For that purpose we provide the
types contained in the namespace `ObsDim`.

```@docs
value(::Loss, ::AbstractArray, ::AbstractArray, ::LossFunctions.AverageMode, ::LearnBase.ObsDimension)
value(::Loss, ::AbstractArray, ::AbstractArray, ::LossFunctions.AggregateMode, ::LearnBase.ObsDimension)
```

Consider the following two matrices, `targets` and `outputs`.
Expand All @@ -136,12 +142,12 @@ the observations. Thus this data would consist of two
observations with four variables each.

```jldoctest obsdim
julia> value(L1DistLoss(), targets, outputs, AvgMode.Sum(), ObsDim.First())
julia> value(L1DistLoss(), targets, outputs, AggMode.Sum(), ObsDim.First())
2-element Array{Float64,1}:
1.5
2.0

julia> value(L1DistLoss(), targets, outputs, AvgMode.Mean(), ObsDim.First())
julia> value(L1DistLoss(), targets, outputs, AggMode.Mean(), ObsDim.First())
2-element Array{Float64,1}:
0.375
0.5
Expand All @@ -152,14 +158,14 @@ second/last dimension denotes the observations. In that case our
data consists of four observations with two variables each.

```jldoctest obsdim
julia> value(L1DistLoss(), targets, outputs, AvgMode.Sum(), ObsDim.Last())
julia> value(L1DistLoss(), targets, outputs, AggMode.Sum(), ObsDim.Last())
4-element Array{Float64,1}:
0.125
0.625
1.125
1.625

julia> value(L1DistLoss(), targets, outputs, AvgMode.Mean(), ObsDim.Last())
julia> value(L1DistLoss(), targets, outputs, AggMode.Mean(), ObsDim.Last())
4-element Array{Float64,1}:
0.0625
0.3125
Expand All @@ -172,17 +178,17 @@ mutating version that can make use a preallocated vector to write
the results into.

```@docs
value!(::AbstractArray, ::Loss, ::AbstractArray, ::AbstractArray, ::LossFunctions.AverageMode, ::LearnBase.ObsDimension)
value!(::AbstractArray, ::Loss, ::AbstractArray, ::AbstractArray, ::LossFunctions.AggregateMode, ::LearnBase.ObsDimension)
```

Naturally we also provide both of these methods for
[`deriv`](@ref) and [`deriv2`](@ref) respectively.

```@docs
deriv(::Loss, ::AbstractArray, ::AbstractArray, ::LossFunctions.AverageMode, ::LearnBase.ObsDimension)
deriv!(::AbstractArray, ::Loss, ::AbstractArray, ::AbstractArray, ::LossFunctions.AverageMode, ::LearnBase.ObsDimension)
deriv2(::Loss, ::AbstractArray, ::AbstractArray, ::LossFunctions.AverageMode, ::LearnBase.ObsDimension)
deriv2!(::AbstractArray, ::Loss, ::AbstractArray, ::AbstractArray, ::LossFunctions.AverageMode, ::LearnBase.ObsDimension)
deriv(::Loss, ::AbstractArray, ::AbstractArray, ::LossFunctions.AggregateMode, ::LearnBase.ObsDimension)
deriv!(::AbstractArray, ::Loss, ::AbstractArray, ::AbstractArray, ::LossFunctions.AggregateMode, ::LearnBase.ObsDimension)
deriv2(::Loss, ::AbstractArray, ::AbstractArray, ::LossFunctions.AggregateMode, ::LearnBase.ObsDimension)
deriv2!(::AbstractArray, ::Loss, ::AbstractArray, ::AbstractArray, ::LossFunctions.AggregateMode, ::LearnBase.ObsDimension)
```

## Weighted Sum and Mean
Expand Down Expand Up @@ -233,7 +239,7 @@ each observation (which results in a vector), and then we compute
the weighted sum of all observations.

The following code snipped demonstrates how to compute the
`AvgMode.WeightedSum([2,1])` manually. This is **not** meant as
`AggMode.WeightedSum([2,1])` manually. This is **not** meant as
an example of how to do it, but simply to show what is happening
qualitatively. In this example we assume that we are working in a
multi-variable regression setting, in which our data set has four
Expand Down Expand Up @@ -261,7 +267,7 @@ julia> sum(tmp .* [2, 1]) # weigh 1st observation twice as high
5.0
```

To manually compute the result for `AvgMode.WeightedMean([2,1])`
To manually compute the result for `AggMode.WeightedMean([2,1])`
we follow a similar approach, but use the normalized weight
vector in the last step.

Expand All @@ -282,8 +288,8 @@ julia> sum(tmp .* [0.6666, 0.3333]) # weigh 1st observation twice as high
Note that you can specify explicitly if you want to normalize the
weight vector. That option is supported for computing the
weighted sum, as well as for computing the weighted mean. See the
documentation for [`AvgMode.WeightedSum`](@ref) and
[`AvgMode.WeightedMean`](@ref) for more information.
documentation for [`AggMode.WeightedSum`](@ref) and
[`AggMode.WeightedMean`](@ref) for more information.

The code-snippets above are of course very inefficient, because
they allocate (multiple) temporary arrays. We only included them
Expand All @@ -293,32 +299,32 @@ special methods for [`value`](@ref), [`deriv`](@ref),
[`deriv2`](@ref) and their mutating counterparts.

```jldoctest weight
julia> value(L1DistLoss(), [1.,2,3], [2,5,-2], AvgMode.WeightedSum([1,2,1]))
julia> value(L1DistLoss(), [1.,2,3], [2,5,-2], AggMode.WeightedSum([1,2,1]))
12.0

julia> value(L1DistLoss(), [1.,2,3], [2,5,-2], AvgMode.WeightedMean([1,2,1]))
julia> value(L1DistLoss(), [1.,2,3], [2,5,-2], AggMode.WeightedMean([1,2,1]))
3.0

julia> value(L1DistLoss(), targets, outputs, AvgMode.WeightedSum([2,1]), ObsDim.First())
julia> value(L1DistLoss(), targets, outputs, AggMode.WeightedSum([2,1]), ObsDim.First())
5.0

julia> value(L1DistLoss(), targets, outputs, AvgMode.WeightedMean([2,1]), ObsDim.First())
julia> value(L1DistLoss(), targets, outputs, AggMode.WeightedMean([2,1]), ObsDim.First())
0.4166666666666667
```

We also provide this functionality for [`deriv`](@ref) and
[`deriv2`](@ref) respectively.

```jldoctest weight
julia> deriv(L2DistLoss(), [1.,2,3], [2,5,-2], AvgMode.WeightedSum([1,2,1]))
julia> deriv(L2DistLoss(), [1.,2,3], [2,5,-2], AggMode.WeightedSum([1,2,1]))
4.0

julia> deriv(L2DistLoss(), [1.,2,3], [2,5,-2], AvgMode.WeightedMean([1,2,1]))
julia> deriv(L2DistLoss(), [1.,2,3], [2,5,-2], AggMode.WeightedMean([1,2,1]))
1.0

julia> deriv(L2DistLoss(), targets, outputs, AvgMode.WeightedSum([2,1]), ObsDim.First())
julia> deriv(L2DistLoss(), targets, outputs, AggMode.WeightedSum([2,1]), ObsDim.First())
10.0

julia> deriv(L2DistLoss(), targets, outputs, AvgMode.WeightedMean([2,1]), ObsDim.First())
julia> deriv(L2DistLoss(), targets, outputs, AggMode.WeightedMean([2,1]), ObsDim.First())
0.8333333333333334
```
6 changes: 6 additions & 0 deletions docs/src/user/interface.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
```@meta
DocTestSetup = quote
using LossFunctions
end
```

# Working with Losses

Even though they are called loss "functions", this package
Expand Down
2 changes: 1 addition & 1 deletion src/LossFunctions.jl
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ export

AggMode

include("common.jl")
include("devutils.jl")
include("aggregatemode.jl")

include("supervised/supervised.jl")
Expand Down
9 changes: 0 additions & 9 deletions src/common.jl

This file was deleted.

3 changes: 3 additions & 0 deletions src/devutils.jl
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
macro dimcheck(condition)
:(($(esc(condition))) || throw(DimensionMismatch("Dimensions of the parameters don't match: $($(string(condition)))")))
end
31 changes: 5 additions & 26 deletions src/supervised/io.jl
Original file line number Diff line number Diff line change
Expand Up @@ -33,33 +33,12 @@ Base.print(io::IO, loss::WeightedBinaryLoss{T,W}, args...) where {T,W} = print(i

_loss_xguide(loss::MarginLoss) = "y * h(x)"
_loss_xguide(loss::DistanceLoss) = "h(x) - y"
_loss_yguide(loss::SupervisedLoss) = "L("*_loss_xguide(loss)*")"

@recipe function plot(drv::Deriv, rng = -2:0.05:2)
xguide --> _loss_xguide(drv.loss)
yguide --> "L'(y, h(x))"
label --> string(drv.loss)
deriv_fun(drv.loss), rng
end

@recipe function plot(loss::SupervisedLoss, rng = -2:0.05:2)
@recipe function plot(loss::SupervisedLoss, range=-2:0.05:2; fun=value)
xguide --> _loss_xguide(loss)
yguide --> "L(y, h(x))"
yguide --> _loss_yguide(loss)
label --> string(loss)
value_fun(loss), rng
end

@recipe function plot(derivs::AbstractVector{T}, rng = -2:0.05:2) where T<:Deriv
for drv in derivs
@series begin
drv, rng
end
end
end

@recipe function plot(losses::AbstractVector{T}, rng = -2:0.05:2) where T<:SupervisedLoss
for loss in losses
@series begin
loss, rng
end
end
l(a) = fun(loss, a)
l, range
end
4 changes: 2 additions & 2 deletions src/supervised/sparse.jl
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,8 @@ end
) where {T,N,Q,Ti,M}
M > N && throw(ArgumentError("target has more dimensions than output; broadcasting not supported in this direction."))
quote
@_dimcheck size(buffer) == size(output)
@nexprs $M (n)->@_dimcheck(size(target,n) == size(output,n))
@dimcheck size(buffer) == size(output)
@nexprs $M (n)->@dimcheck(size(target,n) == size(output,n))
zeroQ = zero(Q)
negQ = Q(-1)
@simd for I in CartesianIndices(size(output))
Expand Down
Loading