Skip to content

Fix shaders failing to compile with Metal renderer#87

Merged
phandasm merged 1 commit into
phandasm:masterfrom
prime-optimal:fix/metal-scalar-distance
Jul 11, 2026
Merged

Fix shaders failing to compile with Metal renderer#87
phandasm merged 1 commit into
phandasm:masterfrom
prime-optimal:fix/metal-scalar-distance

Conversation

@prime-optimal

Copy link
Copy Markdown
Contributor

Fixes #85.

Root cause

The Metal Shading Language has no scalar overload of distance() — only float2/3/4 (and half variants). OBS's Metal backend (libobs-metal) transpiles effect-HLSL intrinsics verbatim, so PSGradient and PSRange fail to compile with:

program_source:19:30: error: call to 'distance' is ambiguous
    float lerp_t = saturate((distance(vert_in.tex.y, uniforms.grad_center) - uniforms.grad_offset) / uniforms.grad_height);

One failed shader makes gs_effect_create_from_file return NULL for the whole effect, so every technique (including Solid) stops rendering — matching the "no visual in any mode" symptom. The OBS log only shows the generic MTLDevice failed to create shader library and function; I recovered the real MTLCompiler error by instrumenting MetalShader.swift in a local libobs-metal build.

Fix

Replace scalar distance(a, b) with abs(a - b) in PSGradient and PSRange. Semantically identical for scalars and valid in HLSL, GLSL, and MSL, so D3D11/OpenGL behavior is unchanged.

Verification

macOS 26.4, Apple Silicon, OBS 32.1.2 with Renderer=Metal:

  • Before: 4× device_vertexshader_create: Device error compiling shader, Pass (0) <> missing pixel shader!, per-frame effect_setval_inline: invalid param spam, source screenshot has 0 painted pixels.
  • After: zero shader errors in the log; waveform renders correctly in mono and stereo (checked Gradient/Range paths; stereo draws both channels independently).

Everything else in gradient.effect transpiles cleanly on 32.1.2 (uniform default initializers, bool uniform, mul(), scalar saturate/lerp, all six techniques, GS_TRISTRIP/LINESTRIP/TRIS draws).

MSL has no scalar overload of distance(), so PSGradient and PSRange
failed to compile ('call to distance is ambiguous') under the Metal
backend, which made gs_effect_create return NULL and left all
waveform sources invisible on macOS with the Metal renderer.

Replace scalar distance(a, b) with abs(a - b), which is semantically
identical and valid in HLSL, GLSL, and MSL.

Fixes phandasm#85
@phandasm

Copy link
Copy Markdown
Owner

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Doesn't work with Metal rendering.

2 participants