decouple flattening and transformations

[tpapp]

Use separate building blocks for

1. splitting a vector into arrays (or other objects) of various sizes and dimensions, without transforming,
2. transformations.

Transformations should

1. have a dimension,
2. be able to perform log Jacobian calculations,
3. carry information (schemas, see below) that allows flattening and unflattening their arguments (ignoring extra payload, especially for user-defined transformations).

Transformations should be generic, eg going from StaticArray should also produce StaticArray.

Flattening/unflattening should use schemas, possibly nested.

1. make this work, define API.
2. see what's optimizable.

[tpapp]

Related work: https://github.com/rafaqz/Flatten.jl, https://github.com/simonbyrne/PackVec.jl

[tpapp]

Also related: https://github.com/oschulz/ShapesOfVariables.jl

[tkf]

Forgive me to suddenly drop a slightly ad-like comment here. I just thought it could be a useful information.

For nested (especially immutable) objects, I think Lens API from Setfield.jl is worth considering. Setfield.jl itself has no direct support for flattening (by that, I mean getting/setting multiple fields together) but I just created a package (Kaleido.jl) that does that. It also has a basic interop for TransformVariables.jl.

Setfield.jl/Kaleido.jl lack in-place operations at the moment so I guess it's probably not possible to directly cover all use-cases in TransformVariables.jl. But I think it is possible to implement "impure" lenses. See also jw3126/Setfield.jl#32

[tpapp]

@tkf: I appreciate the heads-up. I like your packages very much and I will look at Kaleido.jl.

[tpapp]

Another package for this is https://github.com/oschulz/ValueShapes.jl