Shifted Composite L1 Norm

[MaxenceGollier]

Implementation of the proximal operator of a composite term $||Ax+b||_1$. This is different from other operators of this library in the fact that shifts are no longer simply $\psi(x+s)$ for some non-differentiable function $\psi$.
Hence, some abstract types are added in ShiftedProximalOperators.jl.

In practice, such proximal operators are useful when we want to make a model of a penalty term $x \xrightarrow{} ||c(x)||_1$. The first-order model of the penalty term is then $s \xrightarrow{} ||c(x) + J(x)s||_1$.

As mentionned above, the shift is no longer simply $\psi(x+s)$. Hence, we add a CompositeNormL1 struct which mimics the unshifted function and implements $x\xrightarrow{} ||c(x)||_1$. On shifts, this transforms to a ShiftedCompositeNormL1 struct which implements $s\xrightarrow{} ||c(x)+J(x)s||_1$.

[dpo]

I think things would be clearer if you defined a type

abstract AbstractCompositeNorm <: CompositeProximableFunction end

and then

CompositeNormL1 <: AbstractCompositeNorm
CompositeNormL2 <: AbstractCompositeNorm

[dpo]

It would be useful to state how c!() and J!() are expected to be called and what they should return. In particular, is A expected to be dense or sparse? In NLPModels, there is no Jacobian method that fills in a matrix, i.e., something like jac!(nlp, x, A) does not exist. However, there is jac_coord!(nlp, x, vals), which fills in the array vals of nonzeros elements in the sparse coordinate representation of the Jacobian.

[MaxenceGollier]

I added a commit, see MaxenceGollier:affineNormL1@561c415
I just changed the documentation, do you expect to force A by changing the type in the constructor ? I don't know if it is necessary.

[dpo]

This needs some documentation to explain that $c$ will be linearized; otherwise, we would expect the shifted function to be $s \to \Vert c(x+s) \Vert_1$.

[MaxenceGollier]

See MaxenceGollier:affineNormL1@bb3faaa

[dpo]

Preallocate.

[MaxenceGollier]

If I add z as an argument, there will be problems later with R2 and overall most solvers compute $\psi(y)$ at some point without calling an extra argument.

[dpo]

Why does $\psi$ take an input of length 4 but $h$ takes an input of length 2 here?

[MaxenceGollier]

In our implementation, $\psi$ represents $\psi : \mathbb{R}^n \xrightarrow[]{} \mathbb{R}^m : s \xrightarrow[]{} ||c(x)+J(x)s||_1$. In this test, $n=4$ and $m = 2$.