Unit support for Axes & Recipes, a.k.a axis converts

[SimonDanisch]

Continuation of #1347.

Axis conversion

• Axis specifies types it can handle and creates objects that track global states for those types (e.g. categorical needs to track all categories of all plots in one axis)
• Axis compatible types => scene compatible types plus more types.
• Axis conversion can return axis parameters, which can be overridden step-by-step, similar to Plots.jl recipes.
• Scene conversions don't handle further axis-related tasks.
• For each plot command, we figure out the axis convert type, and then compare it to the existing axis conversion types in that axis. We then compare them by something like the Base.promotion trait system, and error if e.g. one tries to plot categorical values into a time axis.
• Each axis global (e.g. CategoricalConversion) will have an axis dimension associated and will be assigned to (dim)ticks of the axis, whcih then correctly creates axis labels etc.

We can implement float32 rescaling as a axis conversion:

• Update data rescaling only when axis limits change.
• The "limits" referred to in rescaling aren't the axis limits but the 2D scene limits.
  • Example: For user-facing limits from 1 to 100 and a log10 scale, scene limits would be 0 to 2. So, a real value of 1000
    would be transformed to 3 and then rescaled to (3 - 0) / (2 - 0) = 1.5.
• Example: Some type is converted by the axis to float128, which then gets converted + scaled for the scene to float32.
• The scene then converts float128 to float32.
• Transform functions need to occur before data rescaling
• We can optimize rescaling if inputs are already float32.

This PR right now ports all docs & examples and the actual implementation of unit conversions and bases it against #3113.
We still need to implement the actual axis conversion more cleanly, which was the Achilles heel for #1347 and the reason it wasn't merged yet. Since most work was getting units, printing and conversions to work, I hope that now with a better plan the axis conversion won't take that long to implement anymore.

[MakieBot]

Compile Times benchmark

Note, that these numbers may fluctuate on the CI servers, so take them with a grain of salt. All benchmark results are based on the mean time and negative percent mean faster than the base branch. Note, that GLMakie + WGLMakie run on an emulated GPU, so the runtime benchmark is much slower. Results are from running:

using_time = @ctime using Backend
# Compile time
create_time = @ctime fig = scatter(1:4; color=1:4, colormap=:turbo, markersize=20, visible=true)
display_time = @ctime Makie.colorbuffer(display(fig))
# Runtime
create_time = @benchmark fig = scatter(1:4; color=1:4, colormap=:turbo, markersize=20, visible=true)
display_time = @benchmark Makie.colorbuffer(display(fig))

	using	create	display	create	display
GLMakie	3.51s (3.49, 3.55) 0.02+-	106.78ms (104.22, 108.76) 1.88+-	520.35ms (515.45, 524.78) 3.18+-	7.69ms (7.64, 7.78) 0.05+-	25.59ms (25.46, 25.96) 0.17+-
sd/beta-20	3.40s (3.36, 3.44) 0.03+-	134.19ms (131.55, 135.96) 1.53+-	530.70ms (521.53, 537.84) 5.37+-	7.86ms (7.82, 7.91) 0.03+-	25.49ms (25.39, 25.61) 0.08+-
evaluation	0.97x slower X, 0.11s (4.53d, 0.00p, 0.02std)	1.26x faster✅, -27.41ms (-16.01d, 0.00p, 1.70std)	1.02x faster ✓, -10.35ms (-2.35d, 0.00p, 4.27std)	1.02x faster ✓, -0.17ms (-3.89d, 0.00p, 0.04std)	1.00x invariant, 0.1ms (0.80d, 0.17p, 0.12std)
CairoMakie	3.13s (3.12, 3.15) 0.01+-	106.18ms (105.38, 107.56) 0.90+-	127.03ms (125.89, 129.13) 1.28+-	7.42ms (7.34, 7.59) 0.08+-	968.16μs (958.99, 975.64) 5.12+-
sd/beta-20	3.05s (3.01, 3.09) 0.03+-	134.20ms (131.73, 135.65) 1.43+-	118.62ms (116.14, 121.68) 1.76+-	7.66ms (7.62, 7.73) 0.04+-	969.90μs (957.62, 1007.65) 17.00+-
evaluation	0.97x slower X, 0.08s (3.70d, 0.00p, 0.02std)	1.26x faster✅, -28.03ms (-23.37d, 0.00p, 1.17std)	0.93x slower❌, 8.41ms (5.46d, 0.00p, 1.52std)	1.03x faster ✓, -0.24ms (-3.77d, 0.00p, 0.06std)	1.00x invariant, -1.74μs (-0.14d, 0.80p, 11.06std)
WGLMakie	3.94s (3.80, 4.06) 0.10+-	110.78ms (106.86, 116.65) 3.41+-	8.79s (8.47, 9.08) 0.23+-	9.87ms (9.51, 10.30) 0.33+-	71.12ms (70.10, 72.15) 0.76+-
sd/beta-20	3.70s (3.59, 3.79) 0.08+-	137.51ms (134.27, 142.04) 2.96+-	8.74s (8.53, 8.91) 0.14+-	10.31ms (10.06, 10.52) 0.17+-	71.28ms (70.53, 72.22) 0.63+-
evaluation	0.94x slower❌, 0.23s (2.51d, 0.00p, 0.09std)	1.24x faster✅, -26.73ms (-8.36d, 0.00p, 3.19std)	1.00x invariant, 0.04s (0.22d, 0.70p, 0.19std)	1.04x faster ✓, -0.44ms (-1.68d, 0.01p, 0.25std)	1.00x invariant, -0.16ms (-0.23d, 0.67p, 0.70std)

[ffreyer]

I want to collect some information and thoughts on Float32.

Conversion Pipeline

Conversion steps (current):

1. convert_arguments normalizes type and structure of plot.input_args, generating plot.converted
2. transform_func applies any function to positional data in plot.converted
3. model moves the result into world space (translation + rotation + scale, technically arbitrary matrix transform)
4. view moves data into camera space (translation + basis change)
5. projection moves data into clip space, -1..1 range (perspective projection, scaling)
6. viewport/pixel_space moves data into pixel space, 0..w range (scaling, translation)

There might be some additional steps to deal with markerspace. All of those use matrices from steps 3-6.

Float issues:

1. Truncates to Float32, reducing precision
2. Can do arbitrary things such as moving data to a range with more float precision issues. (E.g. x -> x + 1e6, exp)
3. May introduce precision issues from translation
4. Same
5. Should be save since data gets scaled to -1..1 range before perspective projection?
6. Save

Steps 3-6 are Matrix operations and therefore can and should run on the GPU. We want to use Float32 here. (Note that Axis currently has translation in the projection matrix. We can easily move that to view. TODO: Check PolarAxis, Axis3)

Fixing things

To fix precision issues while keeping 3-6 on the GPU we need to remove translation partially or fully from view * model. Scaling shouldn't matter. For example 1e6 .+ collect(0:10) has precision issues. Removing translations would move the data into a -5:5 range, fixing the problem. Something like 10 .^ (1:10) does not have issues. On the full range of 10^10 float precision of e.g. 1 never becomes relevant. Zooming would result in a translation keeping precision high.

In 3D view = basis_transform * translationmatrix(-eyepositon) so extracting the translation is easy. (2D can be adjusted to match.) For a "normal" model matrix composed out of a translation, rotation and scale we have

$$\begin{align} &view \cdot model \cdot p \\ = &basis \cdot translationmatrix(-eyepos) \cdot (scale \cdot rotation \cdot p + translation) \\ = &basis \cdot (scale \cdot rotation \cdot p + translation - eyepostion) \\ = &basis \cdot scale \cdot rotation \cdot (p + rotation^{-1} \cdot scale^{-1} \cdot (translation - eyeposition)) \\ = &basis \cdot scale \cdot rotation \cdot (p - inv(model) \cdot eyeposition) \end{align}$$

For an arbitrary model matrix we could calculate M = translationmatrix(-eyepos) \cdot model, generate a matrix $M'$ with the translating part of M set to 0 and compute a translation matrix $T = inv(M') \cdot M$ such that $M = M' \cdot T$. It is not clear to me whether this fixes float precision issues. However this is also an edge case which we may just want to ignore or disallow for this purpose.

With this we have an extra conversion step p -> Float32(p + translation(view, model)) applying after step 2 and before step 3. Data in step 1 and 2 would need to be kept at it's original precision to remain accurate.

• Can't we just use eyeposition? - Technically no, as plots could be translated into positions that cause float precision issues
• Can we apply this earlier? - In most cases we can probably just inverse transform the offset, but there might be some transform functions that are not (locally) invertible or cause precision issues themselves. Things like log(0) or InversePolar(-5, 30°) could also cause issues

TODO

Following the above I think our Float32 TODO would be:

• remove Float32 conversion from convert_arguments (I guess this means keeping the user type and just adjusting the structure of data (e.g. 3x Vector{<: Real} -> Point3 Vector)?)
• generalize transform_funcs to take arbitrary numeric types
• remove translations from the view matrix (and projection) and keep track of them in camera.eyeposition
• add a conversion step between transfunc and model that applies an offset derived from camera.eyeposition and plot.model to transformed data
• And to allow non-Real types I think we'd need to extend convert_arguments accordingly

What to do with categorical data is still a question. In this structure they would have to fit in with convert_arguments, but they also need to talk to the axis/scene...

Tangent:
Why is our model matrix defined as scale_translation * rotation rather than rotation * scale_translation? This means that scale!(p, Vec3f(1,1,2)); rotate!(p, Vec3f(1,0,0), pi/4) scales the plot y and z coordinates (world z) rather than just plot z (world y and z). Do we want that?
We're also missing inv(::Quaternion).

[ffreyer]

I removed some things that seemed like forgotten comments/prototyping in the last commit. Also tried to rephrase some bits in the documentation.

[ffreyer]

Something like [1 2; 1.5 2.5] doesn't work for heatmaps since the cells are rectangular. So you can't use the extra freedom a Matrix input gives you. I think that's what the missing conversion/test is implying.