Add voxel plot type

[ffreyer]

Description

This is an experiment with a dedicated voxel rendering function using the meshing ideas presented here: https://www.youtube.com/watch?v=4xs66m1Of4A

The initial commit only implements plain voxel rendering, but can handle 100 Million Voxels without much of an issue for me.

scene = Scene()
cam3d!(scene)
p = voxel!(scene, rand(UInt8, 1024, 1024, 128)) # 134 217 728 Voxels
l = linesegments!(scene, boundingbox(p))
center!(scene)
display(scene)

TODO:

• add lighting
• replace Matrix{UInt8, 3} restriction with automatic compression
  • add colormaps (compressed id -> color)
  • add texturemaps (id, side -> uv -> texture fetch)
• fast update function (local update of chunk data)
• allow users to control what becomes air/invisible (currently NaN, nothing and missing)
• test actual textures
• improve render order
  • going front to back from eyeposition should improve performance
  • going back to front should enable correct transparency
• implement lowclip and highclip (and colorrange?) by adjusting voxel id generation and colormap
• WGLMakie (requires Add usampler and isampler SimonDanisch/ShaderAbstractions.jl#26)
  • fix rendering issues (moving planes and maybe z-fighting)
• maybe CairoMakie
  • enable tests
• maybe RPRMakie (TODO: try it)
• allow Colon() in local_update
• test alternating x/y/z planes before incrementing the plane index for transparency test some possible improvements to transparency
  • alternating x/y/z direction doesn't work doesn't (easily) work with varied sizes
  • adjusting dimensional order based on view direction results in very noticeable switches
  • shifting planes slightly closer to the viewer along the normal direction doesn't help
• px_per_unit doesn't apply
• test picking
• fix missing z plane when rendering without gap one weaker GPU (and with missing voxels)
• fix render order (consider scaling on voxels)

To discuss:

• Should resizing be possible? (Requires workaround for resize!(::Array{UInt8, 3}))
• Should the order of sides in uvmap change?

Type of change

• New feature (non-breaking change which adds functionality)

Checklist

• Added an entry in NEWS.md (for new features and breaking changes)
• Added or changed relevant sections in the documentation
• Added unit tests for new algorithms, conversion methods, etc.
• Added reference image tests for new plotting functions, recipes, visual options, etc.

[MakieBot]

Compile Times benchmark

Note, that these numbers may fluctuate on the CI servers, so take them with a grain of salt. All benchmark results are based on the mean time and negative percent mean faster than the base branch. Note, that GLMakie + WGLMakie run on an emulated GPU, so the runtime benchmark is much slower. Results are from running:

using_time = @ctime using Backend
# Compile time
create_time = @ctime fig = scatter(1:4; color=1:4, colormap=:turbo, markersize=20, visible=true)
display_time = @ctime Makie.colorbuffer(display(fig))
# Runtime
create_time = @benchmark fig = scatter(1:4; color=1:4, colormap=:turbo, markersize=20, visible=true)
display_time = @benchmark Makie.colorbuffer(fig)

	using	create	display	create	display
GLMakie	3.39s (3.28, 3.47) 0.05+-	166.53ms (158.74, 178.76) 6.56+-	627.08ms (610.15, 637.18) 8.76+-	7.44ms (7.25, 7.63) 0.12+-	25.97ms (25.57, 26.10) 0.18+-
master	3.41s (3.33, 3.45) 0.05+-	164.42ms (156.76, 169.18) 4.63+-	631.32ms (611.58, 647.28) 13.20+-	7.54ms (7.42, 7.70) 0.12+-	25.94ms (25.43, 26.15) 0.24+-
evaluation	1.01x invariant, -0.02s (-0.38d, 0.50p, 0.05std)	0.99x invariant, 2.11ms (0.37d, 0.50p, 5.60std)	1.01x invariant, -4.24ms (-0.38d, 0.49p, 10.98std)	1.01x invariant, -0.09ms (-0.77d, 0.18p, 0.12std)	1.00x invariant, 0.03ms (0.13d, 0.81p, 0.21std)
CairoMakie	2.98s (2.96, 3.01) 0.02+-	160.42ms (159.25, 162.20) 1.08+-	107.03ms (106.27, 108.48) 0.78+-	7.47ms (7.41, 7.62) 0.07+-	630.11μs (624.64, 635.39) 3.98+-
master	2.97s (2.96, 3.00) 0.02+-	160.79ms (158.66, 164.07) 2.07+-	106.63ms (105.65, 108.13) 0.88+-	7.45ms (7.37, 7.53) 0.05+-	625.09μs (621.09, 630.19) 3.05+-
evaluation	1.00x invariant, 0.01s (0.32d, 0.55p, 0.02std)	1.00x invariant, -0.37ms (-0.23d, 0.68p, 1.57std)	1.00x invariant, 0.41ms (0.49d, 0.38p, 0.83std)	1.00x invariant, 0.02ms (0.35d, 0.52p, 0.06std)	0.99x slower X, 5.02μs (1.42d, 0.02p, 3.51std)
WGLMakie	3.56s (3.52, 3.60) 0.03+-	159.55ms (157.18, 162.56) 2.09+-	8.74s (8.65, 8.87) 0.08+-	9.47ms (9.38, 9.58) 0.08+-	69.76ms (68.52, 70.34) 0.62+-
master	3.53s (3.50, 3.61) 0.04+-	160.37ms (156.77, 172.04) 5.37+-	8.73s (8.69, 8.79) 0.03+-	9.61ms (9.34, 10.09) 0.25+-	70.10ms (69.13, 72.68) 1.19+-
evaluation	0.99x invariant, 0.03s (0.84d, 0.15p, 0.03std)	1.01x invariant, -0.82ms (-0.20d, 0.72p, 3.73std)	1.00x invariant, 0.01s (0.17d, 0.76p, 0.06std)	1.01x invariant, -0.14ms (-0.77d, 0.19p, 0.16std)	1.00x invariant, -0.34ms (-0.36d, 0.52p, 0.91std)

[ffreyer]

Taking an example from the volume docs:

r = LinRange(-1, 1, 100)
cube = [(x.^2 + y.^2 + z.^2) for x = r, y = r, z = r]
cube_with_holes = cube .* (cube .> 1.4)
f, a, p = voxel(cube_with_holes, is_air = x -> !(1.65 <= x <= 1.75))

[ffreyer]

Texture mapping that needs more testing:

voxels = UInt8[
    1 0 2; 0 0 0; 3 0 4;;;
    0 0 0; 0 0 0; 0 0 0;;;
    5 0 6; 0 0 0; 7 0 8;;;
]

uv_map = [
    Vec4f(0.00, 0.25, 0.0, 0.5),
    Vec4f(0.00, 0.25, 0.5, 1.0),
    Vec4f(0.25, 0.50, 0.0, 0.5),
    Vec4f(0.25, 0.50, 0.5, 1.0),
    Vec4f(0.50, 0.75, 0.0, 0.5),
    Vec4f(0.50, 0.75, 0.5, 1.0),
    Vec4f(0.75, 1.00, 0.0, 0.5),
    Vec4f(0.75, 1.00, 0.5, 1.0),
]

texture = [
    RGBf(0, 0, 0) RGBf(1, 0, 0);
    RGBf(0, 1, 0) RGBf(0, 0, 1);
    RGBf(1, 1, 1) RGBf(1, 1, 0);
    RGBf(0, 1, 1) RGBf(1, 0, 1);
]

f, a, p = voxel(voxels, uvmap = uv_map, color = texture)

[ffreyer]

Using textures from https://www.kenney.nl/assets/voxel-pack for testing.

Texture test without orientations:

# 9 wide, 10 tall
texture = FileIO.load(Makie.assetpath("voxel_spritesheet.png"))

uv_map = [
    Vec4f(x, x+1/10, y, y+1/9)
    for x in range(0.0, 1.0, length = 11)[1:end-1]
    for y in range(0.0, 1.0, length = 10)[1:end-1]
]

voxels = UInt8[
    1 0 2; 0 0 0; 3 0 4;;;
    0 0 0; 0 0 0; 0 0 0;;;
    5 0 6; 0 0 0; 7 0 9;;;
]

f, a, p = voxel(voxels, uvmap = uv_map, color = texture)

With orientations:

# 9 wide, 10 tall
texture = FileIO.load(Makie.assetpath("voxel_spritesheet.png"))

uvs = [
    Vec4f(x, x+1/10, y, y+1/9)
    for y in range(0.0, 1.0, length = 10)[1:end-1]
    for x in range(0.0, 1.0, length = 11)[1:end-1]
]
# -x -y -z x y z
uv_map = Matrix{Vec4f}(undef, 4, 6)
uv_map[1, :] = [uvs[9],  uvs[9],  uvs[8],  uvs[9],  uvs[9],  uvs[8]]  # birch
uv_map[2, :] = [uvs[11], uvs[11], uvs[10], uvs[11], uvs[11], uvs[10]] # oak
uv_map[3, :] = [uvs[2],  uvs[2],  uvs[2],  uvs[2],  uvs[2],  uvs[18]] # crafting table
uv_map[4, :] = [uvs[1],  uvs[1],  uvs[1],  uvs[1],  uvs[1],  uvs[1]]  # planks

voxels = UInt8[
    1 0 1; 0 0 0; 1 0 1;;;
    0 0 0; 0 0 0; 0 0 0;;;
    2 0 2; 0 0 0; 3 0 4;;;
]

f, a, p = voxel(voxels, uvmap = uv_map, color = texture)

[ffreyer]

I added this in the lighting pr but didn't end up using it. Swapped it out for view_direction here, which is used to sort planes. Might also be useful in volume rendering?

[ffreyer]

WGLMakie still has some z-fighting artifacts

GLMakie doesn't have these. I wonder if that's a perhaps a different depth test (e.g. >= instead of >)?

Fixing this by shifting planes is not possible. To avoid the artifacts above, you'd want to shift planes in +- normal direction (whichever moves them closer to the camera). Doing that would however open up a gap for the front most voxels, so you'd have new artifacts.

[ffreyer]

I tried this pr on my Laptop and noticed a plane wasn't rendering on a 500x500x200 chunk. I fairly certain that was a float precision issue that got fixed by switching to integer modulo in the shaders. I wanted to use those in the first place but thought glsl only had float mods for some reason...

While working on that I also noticed the render order was not what I intended it to be so I fixed that too. Good render order should be beneficial for performance but I'm not sure how much it really is. If tests look good this should be ready to be merged again.

On a side note I can render a 500x500x200 table shape with no problems on an Intel Iris Xe. With transparency it gets a little sluggish, but still very usable.