Skip to content

Scene Renderer

Jump to bottom
David Goree edited this page Jun 6, 2026 · 2 revisions

Scene Renderer

Scene wallpapers are drawn by wallpaper-scene-renderer — an open-source Vulkan renderer for Wallpaper Engine wallpapers on Linux. It is pulled in as a Git submodule of this plugin (src/) and built alongside the plugin.

Open source Vulkan scene renderer for Wallpaper Engine wallpapers on Linux.

Core capabilities:

  • Vulkan 1.1
  • Render graph for automatic pass dependencies
  • HLSL-to-GLSL shader translation
  • Async shader compilation with disk cache

The rest of this page is a feature checklist — useful when you're trying to figure out why a particular Workshop wallpaper doesn't look quite right.

Layers

  • ✅ Image layers
    • autosize: true resolves size from the first texture's sprite frame dimensions (or mapWidth/mapHeight for non-sprite pictures).
    • Script-referenced layers stay in the main render graph even when initially invisible (so SceneScript visibility toggles actually render — e.g. dino_run's jump sprite).
    • Dynamic-asset pool for thisScene.createLayer(asset) — pre-allocated pool of hidden scene nodes per engine.registerAsset(path) for both image and particle assets. Burst particle FX rearm via ParticleSubSystem::Reset on each pool pop and auto-hide once all particles die.
  • ✅ Composition / Fullscreen layers
  • ✅ Text (static rasterization via FreeType)
  • ✅ Text (dynamic via SceneScript — clocks, dates, counters)
  • ✅ Sound objects (looping playback via miniaudio)
  • ✅ Particle systems — see Particle System
  • ✅ 3D models — see 3D Models
  • ✅ Light objects (point lights, skylight)

Effects (generic shader pipeline)

Built-in effects work via the same shader pipeline — any effect that compiles to a valid fragment shader will render.

  • ✅ Animation: Scroll, Spin, Shake, Pulse, Water Flow, Water Ripple, Water Waves, Foliage Sway, Swing, Twirl, Cloud Motion, Iris Movement
  • ✅ Blur: Blur, Blur Precise, Motion Blur, Radial Blur
  • ✅ Interactive: Cursor Ripple, Depth Parallax, X-Ray
  • ✅ Color: Blend, Blend Gradient, Chromatic Aberration, Clouds, Color Key, Film Grain, Shimmer, Glitter, Tint, Opacity, VHS, Fire, Light Shafts, Nitro, Reflection, Water Caustics
  • ✅ Distortion: Fisheye, Perspective, Refraction, Skew, Transform
  • ✅ Enhancement: Edge Detection, God Rays, Local Contrast, Shine
  • ❌ Simulation: Advanced Fluid Simulation
  • ✅ ColorBlendMode (effectpassthrough for complex blend modes)
  • ✅ Mouse position with delay (g_PointerPosition)
  • ✅ Parallax (mouse-reactive layer offset)
  • ✅ PBR lighting
  • ✅ Global bloom
  • ✅ Ping-pong FBO pipeline
  • ✅ Per-effect visibility control
  • ✅ HDR-capable effect rendering

Camera

  • ✅ Orthographic (2D scenes)
  • ✅ Perspective (3D scenes)
  • ✅ Zoom
  • ✅ Shake (sum-of-sinusoids)
  • ✅ Path animation (keyframe interpolation)
  • ✅ Parallax (mouse-reactive camera offset)
  • ✅ Fade (transitions between camera paths)

Audio

  • ✅ BGM playback (loop)
  • ✅ Spectrum analysis (KissFFT — 16/32/64 bands, L+R channels)
  • ✅ Shader uniforms (g_AudioSpectrum{16,32,64}{Left,Right}, std140 padded)
  • ✅ System audio capture (PipeWire/PulseAudio monitor source)
  • ✅ Playback tap fallback (when system capture unavailable)
  • ✅ Runtime toggle between system capture and playback tap
  • ✅ Random sound playback (random track selection + inter-track delay)

Particle System

  • ✅ Renderers: Sprite, Rope, Sprite Trail, Rope Trail
  • ✅ Geometry shader renderers: GS Sprite, GS Rope, GS Sprite Trail
  • ✅ Emitters: Box, Sphere — rate-based, burst, one-per-frame, periodic, duration-limited
  • ✅ Initializers: colorrandom, hsvcolorrandom, colorlist, lifetimerandom, sizerandom, alpharandom, velocityrandom, rotationrandom, angularvelocityrandom, turbulentvelocityrandom, positionoffsetrandom, box, sphere, remapinitialvalue, mapsequencebetweencontrolpoints, mapsequencearoundcontrolpoint, inheritcontrolpointvelocity, inheritinitialvaluefromevent
  • ✅ Operators: movement, angularmovement, sizechange, alphafade, alphachange, colorchange, oscillatealpha, oscillatesize, oscillateposition, turbulence, vortex, vortex_v2, controlpointattract, maintaindistancebetweencontrolpoints, reducemovementnearcontrolpoint, remapvalue, controlpointforce, capvelocity, maintaindistancetocontrolpoint, inheritvaluefromevent, boids (Reynolds 1987 flocking)
  • ✅ Collision: collisionplane, collisionsphere, collisionbox / collisionbounds, collisionquad, collisionmodel (with restitution)
  • ✅ Control Points (up to 8) — mouse-follow (link_mouse), worldspace mode, per-instance offset overrides
  • ✅ Child particle systems (static, eventfollow, eventspawn, eventdeath)
  • ✅ Instance overrides (color, alpha, brightness, count, lifetime, rate, speed, size)
  • ✅ Sprite sheet animation (sequential / random frames)
  • ✅ Perspective rendering flag
  • ❌ Real-time particle lighting (sample SceneLights in particle shader)
  • ✅ Audio-responsive emission — bass-band-driven emit-rate scaling (FFT bins 0..3 averaged, exponentially smoothed with fast attack / slow decay, ~0.4× at silence and ~2.0× at full).

Puppet Warp

  • ✅ Multi-bone skeletal animation
  • ✅ Play modes: Loop, Mirror, Single
  • ✅ Animation blending (blend weights)
  • ✅ Bone frame interpolation
  • ✅ Animation layer system (visibility, rate)
  • ✅ Per-vertex skinning (blend indices + weights)
  • ❌ Blend shapes / shape animations
  • ❌ Inverse kinematics
  • ❌ Texture channel animations
  • ❌ Interactive puppet warps (SceneScript-triggered)
  • ❌ 3D perspective extrusion

3D Models

  • ✅ MDL format parsing (v13+, multi-submesh)
  • ✅ Vertex attributes: position, normal, tangent, texcoord, texcoord1, blend indices/weights
  • ✅ Per-submesh materials
  • ✅ Camera path animation
  • ✅ Depth occlusion & transparency
  • ✅ Planar reflection (Y=0 mirror)
  • ✅ Normal mapping
  • ✅ Lightmap UVs (secondary UV channel)
  • ✅ Point lights & skylight
  • ✅ Skybox (spherical environment mapping)
  • ✅ MSAA x4
  • ✅ Anisotropic filtering (16x)
  • ✅ HDR rendering (RGBA16F internal RTs)
  • ✅ HDR texture formats (BC6H, RGBA16F, RG16F, R16F, RGB565, RGBA1010102)
  • ❌ Shadow mapping
  • ❌ HDR bloom post-process
  • ❌ Spot lights
  • ❌ Volumetric lighting

Blend Modes

  • ✅ Translucent (standard alpha)
  • ✅ Translucent_PA (premultiplied alpha)
  • ✅ Additive
  • ✅ Opaque (screen blend for passthrough compose)
  • ✅ Normal
  • ✅ Disable (no blending)
  • ✅ Complex colorBlendMode via effectpassthrough (BlendColor, etc.)

Texture Formats

  • ✅ BC1 (DXT1), BC2 (DXT3), BC3 (DXT5), BC7
  • ✅ BC6H (HDR compressed)
  • ✅ RGBA8, RGB8, RG8, R8
  • ✅ RGBA16F, RG16F, R16F
  • ✅ RGB565, RGBA1010102
  • ✅ TEXB container (v1–v4)
  • ✅ FreeImage formats (PNG, JPEG, TGA, BMP, GIF, DDS, PSD, HDR, EXR, etc.)

User Properties

  • ✅ Default values from project.json
  • ✅ Runtime visibility bindings (conditional show/hide)
  • ✅ Runtime uniform bindings (shader value overrides)
  • ✅ Per-wallpaper config persistence (JSON)
  • ✅ Combo, slider, color, bool, text property types
  • ❌ User-defined custom shaders

Shader Pipeline

  • ✅ HLSL-to-GLSL translation (XSC-based)
  • ✅ GLSL-to-SPIR-V compilation (glslang)
  • ✅ Geometry shader support (VS+GS+FS linked compilation)
  • ✅ Async compilation with disk caching
  • ✅ #define injection (log10, etc.)
  • ✅ Implicit conversion fixes (bool() wraps, cross-stage varying upgrades)
  • ✅ Effect alpha preservation (component-write shaders that omit .a)
  • ✅ flat interpolation qualifier support
  • ✅ 13 texture slots (g_Texture0–g_Texture12) with resolution/rotation/translation metadata

SceneScript

The renderer implements most of Wallpaper Engine's authoring-time JavaScript API.

  • ✅ engine global (IEngine) — registerAudioBuffers, screenResolution, canvasSize, timeOfDay, frametime, runtime, userProperties, setTimeout / setInterval, device detection
  • ✅ thisScene global (IScene)
    • Scene property control (bloom, clear color, camera, lighting)
      • ❌ bloomEnabled runtime toggle (requires render-graph rebuild)
    • getLayer(name), enumerateLayers()
    • createLayer(asset) / destroyLayer(layer) — pool-backed for image and particle assets
    • Camera transforms
  • ✅ thisLayer global (ILayer) — position, angles, scale, visibility, parent/child hierarchy (cycle-checked), animation-layer access
  • ✅ input global — cursor position, cursorEnter / cursorLeave / cursorClick / cursorDown / cursorUp / cursorMove
  • ✅ shared global (inter-script data sharing)
  • ✅ Events: init, update, destroy, applyUserProperties, resizeScreen
  • ✅ Media integration (MPRIS D-Bus → mediaPlaybackChanged, mediaPropertiesChanged, mediaThumbnailChanged, mediaTimelineChanged, mediaStatusChanged)
  • ✅ ILocalStorage (in-memory key-value store)
  • ✅ Text-layer dynamic content
  • ✅ Color scripts (colorScript field parsed, compiled, evaluated at 30 Hz)
  • ✅ WEMath, WEColor, WEVector utility modules
  • ✅ Vec2, Vec3, Vec4 factories (full method sets)
  • ✅ Mat3, Mat4 matrix classes (translation/rotation/scale; view + projection helpers)
  • ✅ engine.colorScheme — Vec3 from the schemecolor user property
  • ✅ engine.openUserShortcut(name) — dispatched to MPRIS and to in-scene scene.on('userShortcut', …) listeners
  • ✅ getEffect(name).visible runtime per-effect toggle (dirty-tracked)
  • ✅ scene.on(event, fn) / scene.off(event[, fn]) — in-scene event bus covering the lifecycle, media, and userShortcut events

Video texture

  • ✅ IVideoTexture — thisLayer.getVideoTexture() returns a proxy with getCurrentTime, setCurrentTime, duration, rate, play, pause, stop, isPlaying (mpv-backed decoder).

Material uniforms

  • ✅ IMaterial — thisLayer.getMaterial().setValue(name, value) / getValue(name); uniform writes go through a render-thread queue into customShader.constValues, dirty flag re-uploads on next frame. Reads are JS-side cached so they never race the renderer.

Sound

  • ✅ Sound layer control (enumerateLayers, play / stop / pause / isPlaying / volume)
  • ✅ console.log support (buffered → LOG_INFO flush)

Timeline Animations

Partly there. The renderer reads the keyframe tracks Wallpaper Engine stores on a layer's properties and plays them back every frame, so a scene that animates alpha, origin, scale, or angles with no script at all will still move. Loop, Mirror, and Single play modes all work, as do the relative flag (keyframe values are deltas off the layer's base value) and wraploop (the cycle interpolates back to the first keyframe instead of snapping).

Still missing:

  • ❌ Bezier / Hermite easing — interpolation between keyframes is linear only, so eased motion looks a little stiff.
  • ❌ Color and other non-transform property tracks.
  • ❌ Auto-keyframing and script-fired animation events.

Not yet investigated

  • ❌ RGB hardware lighting integration (OpenRGB / liquidctl)
  • ❌ IParticleSystem / IParticleSystemInstance SceneScript control

Roadmap — unimplemented features

What it would take to finish each unchecked item above, roughly ordered by scope (smallest → largest).

Easy / mostly plumbing

  • bloomEnabled runtime toggle — bloom passes are baked into the render graph at scene load. Cheapest implementation is a "bypass mode" branch in the combine pass; the cleaner path is a graph rebuild on toggle.
  • IParticleSystem / IParticleSystemInstance — expose a stable nodeId → ParticleSubSystem map through __sceneBridge, then thread emission rate / pause / per-instance CP overrides through the existing ParticleModify queue.
  • User-defined customshader user-properties — accept a customshader property type and swap the layer's shader pass at runtime through the async compile cache.
  • RGB hardware lighting — pure plumbing: forward audio-spectrum + dominant-color outputs to OpenRGB / liquidctl over D-Bus.

Medium

  • Real-time particle lighting — particles use a separate sprite/rope shader path that ignores SceneLights. Add per-particle sampling of nearby point lights, broadcast positions via a small uniform array, reuse the PBR shader prelude.
  • Spot lights — extend SceneLight.hpp with cone angle, inner/outer falloff, and direction; add cone culling in shaders.
  • HDR bloom post-process — the combine_hdr variant exists, but the per-mip bloom convolution chain that feeds it isn't built.
  • Puppet: blend shapes — linear morph-target blending. Adds vertex streams for shape deltas and per-shape weight uniforms.
  • Puppet: 3D perspective extrusion — synthesize Z + normals on a 2D puppet so it picks up lighting and parallax.
  • Puppet: texture channel animations — per-bone texcoord offsets, separate from skin matrices.
  • Interactive puppet warps (SceneScript-triggered) — expose Puppet bone manipulation through a thisLayer.getPuppet() JS proxy.
  • Timeline easing & extra tracks — keyframe playback already landed (linear interpolation, loop/mirror/single, parsed in WPSceneParser and run through the dirty-update queue). What's left is bezier/Hermite easing so motion isn't dead-linear, color and other non-transform tracks, and the editor's auto-keyframing plus script-fired animation events.

Large

  • Shadow mapping — depth-only render pass per light, cascade for the skylight, PCF/PCSS sampler, integration into the PBR prelude.
  • Volumetric lighting — ray-march from camera through participating media; pairs naturally with shadow mapping.
  • Advanced Fluid Simulation effect — the renderer has no compute pipeline today. Would need VkComputePipeline plumbing, pressure/velocity field textures, and a Navier-Stokes solver split across passes.
  • Puppet: inverse kinematics — FABRIK/CCD per chain. Limited value until interactive puppet warps land.

License

The scene renderer is GPL-2.0, same as the plugin.

Repo · Scene Renderer · GPL-2.0

Getting Started

Reference

Troubleshooting

Maintenance

About

Clone this wiki locally