Textured meshscatter

[ffreyer]

In response to https://discourse.julialang.org/t/3d-bars-makie-colors-and-meshes/70283.

This enables passing of textures and per instance/marker uv_scales to meshscatter with GLMakie. With that and the trick from #1368 you can do

using GLMakie, GeometryBasics, ColorSchemes
texture = reshape(get(colorschemes[:Spectral_11], 0:0.01:1), 1, 101)

rectMesh = FRect3D(Vec3f0(-0.5, -0.5, 0), Vec3f0(1, 1, 1))
recmesh = GeometryBasics.normal_mesh(rectMesh)
uvs = [Point2f(p[3], 0) for p in coordinates(recmesh)] # normalize this so zmax = 1
recmesh = GeometryBasics.Mesh(
    meta(coordinates(recmesh); normals=normals(recmesh), uv = uvs), 
    faces(recmesh)
)

pos = [Point3f(i, j, 0) for i in 1:10 for j in 1:10]
z = rand(10,10)

fig = Figure(resolution=(1200, 800), fontsize=26)
ax = Axis3(fig[1, 1]; aspect=(1, 1, 1), elevation=π / 6, perspectiveness=0.5)
meshscatter!(
    ax, pos, marker = recmesh, markersize = Vec3f.(1, 1, z[:]),
    uv_scale = Vec2f.(1, z[:]), color = texture, shading = false
)
ax.limits[] = ((0, 11), (0, 11), (0, 1.2))
  
fig

[lazarusA]

don't forget this PR, it's a nice addition 😄

[ffreyer]

#1436 would also add this but it'll take me some time to figure what each shader/plot primitive should include

[lazarusA]

since #1436 was close, any change of having this?

[asinghvi17]

Hey, just wanted to bump this - I'm working on some plots using meshscatter and this would definitely be nice to have!

Does this work in RPRMakie out of the box by any chance? (or can it be made to?)

[asinghvi17]

Hey, wanted to pick this up again. Is there anything stopping this from being merged?

[ffreyer]

I think this was in a working/mostly done state, so you probably just need to work out the merge conflicts.

[lazarusA]

bump

[ffreyer]

The branch should be up to date with master again.

It seems that something about coordinates(mesh) broke in GeometryBasics, so example now needs to be:

using GeometryBasics, ColorSchemes, GLMakie
texture = reshape(get(colorschemes[:Spectral_11], 0:0.01:1), 1, 101)
    
prim = FRect3D(Vec3f0(-0.5, -0.5, 0), Vec3f0(1, 1, 1))
rectmesh = GeometryBasics.normal_mesh(prim)
uvs = [Point2f(p[3], 0) for p in coordinates(rectmesh)] # normalize this so zmax = 1
rectmesh = GeometryBasics.Mesh(
    meta(collect(coordinates(prim)); normals=normals(rectmesh), uv = uvs), 
    faces(rectmesh)
)

pos = [Point3f(i, j, 0) for i in 1:10 for j in 1:10]
z = rand(10,10)

fig = Figure(resolution=(1200, 800), fontsize=26)
ax = Axis3(fig[1, 1]; aspect=(1, 1, 1), elevation=π / 6, perspectiveness=0.5)
meshscatter!(
    ax, pos, marker = rectmesh, markersize = Vec3f.(1, 1, z[:]),
    uv_scale = Vec2f.(1, z[:]), color = texture, shading = false
)
ax.limits[] = ((0, 11), (0, 11), (0, 1.2))
  
fig

@SimonDanisch Do you think it would be a good idea to generalize uv_scale to a uv_transform::Mat3f? That would encode rotation + translation + scale just like a model matrix, but for uv coordinates.

[MakieBot]

Compile Times benchmark

Note, that these numbers may fluctuate on the CI servers, so take them with a grain of salt. All benchmark results are based on the mean time and negative percent mean faster than the base branch. Note, that GLMakie + WGLMakie run on an emulated GPU, so the runtime benchmark is much slower. Results are from running:

using_time = @ctime using Backend
# Compile time
create_time = @ctime fig = scatter(1:4; color=1:4, colormap=:turbo, markersize=20, visible=true)
display_time = @ctime Makie.colorbuffer(display(fig))
# Runtime
create_time = @benchmark fig = scatter(1:4; color=1:4, colormap=:turbo, markersize=20, visible=true)
display_time = @benchmark Makie.colorbuffer(display(fig))

	using	create	display	create	display
GLMakie	10.45s (10.37, 10.58) 0.07+-	1.11s (1.08, 1.14) 0.02+-	794.73ms (772.98, 827.53) 21.01+-	10.45ms (10.25, 10.62) 0.13+-	89.56ms (88.39, 91.20) 1.02+-
master	10.58s (10.31, 10.84) 0.20+-	1.11s (1.07, 1.13) 0.02+-	786.66ms (769.70, 801.17) 11.54+-	10.54ms (10.41, 10.69) 0.14+-	89.73ms (88.13, 91.77) 1.44+-
evaluation	-1.26%, -0.13s invariant (-0.87d, 0.15p, 0.14std)	+0.07%, 0.0s invariant (0.04d, 0.95p, 0.02std)	+1.02%, 8.07ms invariant (0.48d, 0.40p, 16.27std)	-0.84%, -0.09ms invariant (-0.67d, 0.24p, 0.13std)	-0.18%, -0.16ms invariant (-0.13d, 0.81p, 1.23std)
CairoMakie	9.72s (9.60, 9.84) 0.09+-	1.26s (1.22, 1.28) 0.02+-	236.36ms (229.22, 241.36) 3.86+-	10.93ms (10.76, 11.08) 0.12+-	6.37ms (6.22, 6.49) 0.09+-
master	9.73s (9.62, 9.85) 0.08+-	1.27s (1.25, 1.30) 0.02+-	240.77ms (233.50, 252.22) 5.75+-	10.81ms (10.66, 10.94) 0.08+-	6.33ms (6.24, 6.38) 0.05+-
evaluation	-0.15%, -0.01s invariant (-0.17d, 0.76p, 0.09std)	-0.70%, -0.01s invariant (-0.42d, 0.44p, 0.02std)	-1.87%, -4.41ms invariant (-0.90d, 0.12p, 4.80std)	+1.04%, 0.11ms invariant (1.12d, 0.06p, 0.10std)	+0.65%, 0.04ms invariant (0.55d, 0.33p, 0.07std)
WGLMakie	14.73s (13.97, 15.81) 0.55+-	1.69s (1.54, 1.84) 0.09+-	15.89s (15.12, 16.79) 0.50+-	24.02ms (20.50, 32.52) 5.00+-	1.15s (1.10, 1.29) 0.07+-
master	14.78s (14.13, 15.24) 0.44+-	1.72s (1.63, 1.78) 0.05+-	15.60s (14.90, 15.90) 0.35+-	21.49ms (20.48, 22.46) 0.82+-	1.12s (1.07, 1.18) 0.04+-
evaluation	-0.34%, -0.05s invariant (-0.10d, 0.85p, 0.49std)	-1.61%, -0.03s invariant (-0.35d, 0.52p, 0.07std)	+1.83%, 0.29s invariant (0.68d, 0.23p, 0.42std)	+10.53%, 2.53ms noisy🤷‍♀️ (0.71d, 0.23p, 2.91std)	+2.41%, 0.03s invariant (0.50d, 0.37p, 0.05std)