Deckboy v0.87.0
Two decks, a crossfader and a tempo; a source you write instead of load, with
a real editor and a friend to explain it; an LFO on every effect parameter; and
eight new effects, six of which come out of physics rather than out of another
plugin.
VJ mode
A toggle. Off, Deckboy is a cue deck and every existing show renders exactly as
it did, through the same code path. On, two decks run at once and a crossfader
decides what the audience sees.
The decks were never the missing piece. Project::decks has always been a
vector and each deck has always had its own engine, playlist and transport --
what was missing is that an output could only ever be fed by ONE of them. So
this uses the layering hook that was already there and folds a mix gain into
the opacity each deck layer already carried, which means a deck faded down or
mid cue-fade stays faded down.
Both decks fade on a dissolve, not just the incoming one: they are drawn over
black, so holding A at full until B covered it would be a wipe. Add and
multiply are ways of combining two pictures, so there the base stays at
full and only the incoming deck rides the fader. Verified by recording the
composite with deck A solid red under deck B solid blue -- dissolve walks
250/0/0 to 0/0/253 through 64/0/127, add gives magenta, multiply gives black.
Colours in neither clip, which is the proof they are combined and not switched.
Tap tempo averages the recent taps rather than taking the last interval:
nobody taps evenly, and one interval makes the tempo jump on every beat. Taps
more than two seconds apart start again, because that is a person restarting
and not a 25bpm track. Quantised takes hold until the next beat -- the
point of tempo in a video mixer is not that anything moves by itself, it is
that what the operator does lands ON the music. Measured at 60bpm: unquantised
takes fire in 0.04s, quantised ones wait between 0.16s and 0.81s depending on
where in the beat they were asked for.
It announces itself. A mode you can enter without noticing is a mode that
ruins a show, so there are two signals: a band across the program column that
exists only in VJ mode and carries the controls rather than just announcing
itself, and the whole window edged in a colour used nowhere else -- for the
glance across a room before anyone touches the machine. Both playlists are on
screen side by side, each headed with which side of the crossfader it is,
because two lists both saying PLAYLIST is how the wrong clip reaches an
audience.
The animation carries information rather than decorating. The bar drops in over
a third of a second so the layout settles instead of jumping; the badge and the
frame breathe on the beat, which doubles as a tempo readout you can see without
looking at the number; and the fader handle leans the way it is travelling and
trails a wake that fades as it settles.
VJ ON|OFF | MIX <0-1> | BLEND <dissolve|add|multiply> | TAP | BPM <n> | QUANTISE <on|off> | DECKS <a> <b> | STATUS over the wire, because a crossfader
is a fader and a fader is the one control nobody wants to reach for with a
mouse.
Five bugs found building it. Two in the mix itself, both invisible from
outside: renderTextureWithCueGeometry overwrote the caller's blend mode,
silently discarding add and multiply while dissolve appeared to work, and the
crossfader had to be applied on the GPU zero-copy path as well as the CPU
bridge. Three in the look of it, all found by screenshotting the thing rather
than reasoning about it:
- The window edge was drawn AFTER
SDL_RenderPresent-- painting every frame,
perfectly, onto a back buffer nothing ever showed. The signal designed to be
impossible to miss had never once appeared. - VJ mode pushed the program monitor right to make room for the A preview by
advancing the column's own x, so the timeline lanes and the entire transport
row moved right with it while keeping the full column width, and ran off the
edge. TAKE-adjacent controls, clipped away, in the mode where the second deck
is live. - The bar was authored at fixed widths totalling 670px and VJ mode leaves the
program column around 500, so TAP and the tempo -- the two controls you reach
for on the beat -- were the two that fell off the end. The controls squeeze
toward a floor now, the fader takes what is left, and labels that cannot
survive the squeeze say less instead of being cut in half.
And the crossfader's own readout was the same colour as its handle, so the
handle ate a digit whenever it passed under the number. It sits in a dark well
now, legible at every position.
A code source you can write during a show
A pattern type called Code: the picture is an expression, evaluated per
pixel, edited while it runs.
sin(x*12+t)*0.5+0.5, sin(y*9-t)*0.5+0.5, r
One expression, or three separated by commas for red, green and blue.
Variables are x y (0-1 across the frame), cx cy (-1..1 from the
centre), r (radius), a (angle) and t (seconds), with sin cos tan abs floor fract sqrt min max mod pow atan2 step clamp mix to build from.
Why not GLSL. Deckboy draws through SDL_Renderer, whose backend is D3D11,
D3D12, Metal or OpenGL depending on the machine, and SDL's own shader path
wants SPIR-V, DXIL or MSL -- already compiled. Accepting GLSL at runtime on
every platform would mean bundling a shader compiler, tens of megabytes and a
per-backend translation step, to run arithmetic that fits in a few hundred
lines. So it is evaluated on the CPU, which is viable for the same reason the
effect stack is: the frame splits across cores.
The expression is compiled ONCE into a flat instruction list, cached against
its own text, and the inner loop sees only the instructions -- never a syntax
tree, which would spend its time chasing pointers instead of drawing.
A compile error does not black the output. The cue keeps drawing what it
last drew and the error appears in the inspector. Someone editing live is
mid-keystroke most of the time, and a source that goes black on every
half-typed function is unusable on a stage.
The language has its own test suite, and it earned its keep immediately:
multi-argument functions did not compile, unary minus bound so loosely that
-3+5 came out as -8, and ^ was left-associative. Division by zero, mod by
zero and the square root of a negative are all bounded rather than producing
infinities or NaN, because an operator typing at speed will produce all three.
The Windows CI gates were passing without checking anything
Deckboy is a GUI-subsystem binary on Windows, and PowerShell does not wait for
those: & .\Deckboy.exe --smoke returns immediately, $LASTEXITCODE is never
set from it, and the step passes whatever the app actually did. The Windows
--self-check and --smoke steps had been doing this, so on that platform
they had been reporting success without ever reading a result.
The tell was there in every log: the app's output appears AFTER the step that
was supposed to have run it. It was found by a new packaging gate failing with
no exit code in its message at all -- an empty value, rather than a number.
All three now use Start-Process -Wait -PassThru and read the real exit code.
Linux and macOS were never affected; their binaries are console subsystem and
the shell waits.
The Windows packager also runs the STAGED copy now, not just the one in the
build directory. The build tree has every DLL the build machine happens to
have; the staged tree is what people download, and it had never been started
before being zipped.
Releases now build and publish themselves
The scripts to build an installer and a portable package for all three
platforms have existed for a long time. CI built exactly one of the six --
the macOS .dmg -- and attached it to nothing; the Windows zip was made by hand
on a developer machine, and Linux shipped nothing at all. The README promised
an installer and a portable build for every platform, and only the macOS half
of that had ever been true. v0.86.0 was tagged and never released.
Tagging now produces all six and publishes them: -windows-x64-setup.exe and
-windows-x64.zip, -macos-arm64.dmg and .zip, .AppImage and
-linux-x86_64.tar.gz.
Nothing is built in the publish step. Every file is downloaded from the job
that already tested it, so what reaches the release page is the same file that
passed --smoke -- and both new packaging jobs unpack their own output and run
the binary from inside it before uploading, because an installer nobody has run
is a guess. The release refuses to publish unless all six are present: a
half-empty release page looks like a release.
The notes come from this changelog's own section for the version being tagged,
so the release page and CHANGES.md cannot drift apart.
Packaging runs on every push to main, not only on tags. Only the publishing is
tag-gated -- so the packaging is exercised continuously rather than discovered
to be broken at the moment somebody wants to ship.
The effect chain tells you what it costs
A cue has always been capped at twelve effects, but a cap only bounds the
damage — a dozen cheap ones are free and four expensive ones at 4K are not, so
the count an operator can already see is the wrong number.
The EFFECTS section now shows what the chain actually costs per frame,
measured on that machine at that raster while the cue is live, against the
16.7ms a 60fps frame allows. Over budget, it says so. That is the difference
between "it is stuttering, why" and "this chain costs 47ms".
Measured rather than predicted, and only once it has run: a figure added up
from per-effect benchmarks would be a guess about somebody else's hardware,
which is exactly what the number is there to avoid.
Going over is not a failure, and it is worth knowing what it does. Audio is
the master clock: sound continues in real time and the picture slaves to it,
so you lose frames rather than sync.
The Companion module knows about all of it
VJ mode, the effect stack and the code source were reachable from a control
surface only through the "custom command" box. There are now proper actions for
the crossfader, blend mode, tap tempo, BPM, quantised takes and deck
assignment; for adding an effect, its amount, its parameters and its LFOs; and
for setting a code-source expression.
The crossfader and the tap are the point of it. Those are precisely the two
controls nobody wants to reach for with a mouse, which is the whole argument
for having a surface at all.
Building it turned up a dead control before it shipped: a toggle button sent a
bare VJ, which reports STATUS rather than toggling. VJ TOGGLE exists now —
a button on a surface has one action and two meanings, and making it ask the
app which state it is in first defeats the object.
Six effects that are not in anything else
Each of these comes out of something real -- an instrument, a physical
experiment, a solid-state process, or your own retina -- rather than from
stacking two existing filters. All six fit inside a 60fps frame at 1080p.
Schlieren is how physicists photograph air. You cannot see a shockwave or
the heat off a road, but light bent by a density gradient can be passed or
blocked by a knife edge at the focus, which turns an invisible gradient into
brightness -- it is how every photograph of a bullet's shockwave was taken. Here
the picture is the density field, and what comes out is not the image and not
its edges but the RATE at which it is changing, in one chosen direction, with
everything flat left as mid-grey. Turning the knife changes which features exist
at all, because gradients running along the edge miss it entirely. 4.7ms.
Chladni is the shape a sound makes. Sand on a bowed metal plate runs away
from everything that is moving and piles up along the lines standing still;
Chladni catalogued those figures in 1787 and they are why violins are the shape
they are. Your picture is the sand. The two mode numbers are the note: whole
numbers give the clean classical figures, and between them the plate is being
driven at a frequency it does not want, which is exactly what a real plate does.
7.2ms.
Wavefront solves the actual wave equation, seeded from the picture's own
brightness -- so unlike every sine-based ripple in every video app, it has
INERTIA. Waves leave their source and keep going, pass through each other and
interfere, and reflect off the edges of the frame and come back. None of that
can be faked with a sine, and all of it is what a real surface does. 14.9ms.
Crystallise is grain growth, not a mosaic. A mosaic divides the frame into a
grid; metal does not solidify on a grid. Crystals nucleate at scattered points
and grow until they collide, so the cell a pixel lands in is the one whose seed
reached it first -- and because the seeds grow at DIFFERENT SPEEDS, the result
is the irregular shard structure of a polished metal section rather than a
honeycomb. Each grain takes a facet normal from the direction back to its own
seed, so the light catches it. 15.7ms.
Night eyes is your own retina. Rods are fast, sensitive and completely
colour-blind; cones see colour and are slow and need light. So the brightness
runs at full speed and the COLOUR LAGS BEHIND IT: move something and it goes
grey as it moves, its colour catching up a moment later. The purkinje control is
the other half -- as the rods take over, peak sensitivity slides toward blue,
which is the real reason night looks blue and moonlight photographs that way.
3.8ms.
Grain flow smears the picture along its own grain. Line integral convolution
is how a vector field is drawn in scientific visualisation; pointed at an image's
own structure it makes every stroke follow the direction that part of the picture
is already running -- along a hair, around a jaw, down the length of a shadow.
The direction comes from the structure tensor, the direction in which each
neighbourhood changes least, which a plain gradient cannot give you: a gradient
says which way is uphill, not which way the ridge runs. Flat areas are left
alone. 13.0ms.
Getting them inside the frame was most of the work, and the cost was never where
it looked. Grain flow started at 119ms -- it was converting each pixel's
thirty-six neighbours out of RGB after its neighbours had already done it, then
calling cos, sin and pow inside the pixel loop, and after both of those
were fixed it was still 20ms with the stroke length at zero, because what
actually costs is a scattered gather per pixel. It runs on a third-resolution
raster now, which for an effect whose job is to destroy detail along one axis is
indistinguishable. Chladni went from 24ms to 7ms when the plate equation was
written in its separable form and twenty sines per pixel became two tables and
none.
And wavefront made the same mistake twice in different clothes: first it
rendered as speckle, which looks exactly like an unstable solver and was
actually a displacement scale a hundred times too large; then, fixed, it showed
visible square blocks, which was the coarse field being read one cell at a time
instead of interpolated.
At 4K the budget is a different question, and it always has been. Measured
at 3840x2160: night eyes 14ms and schlieren 17ms still fit; chladni is 27ms,
grain flow 46ms, crystallise 54ms and wavefront 57ms -- one to three frames
each, alongside existing effects like caustics at 23ms. 1080p is the promise;
4K is one heavy effect at a time on a fast machine, and the app tells you what
any of them costs on YOUR machine with --effect-bench <token> 3840x2160.
An LFO on any effect parameter
Every parameter of every effect — and the effect's amount — can be handed to
an oscillator instead of a fixed number. A ~ sits at the right of each
parameter row; switch it on and the parameter starts moving, with its shape,
rate and depth on the line underneath.
Sine, triangle, saw, ramp, square, and sample-and-hold — one random value per
cycle, held, so it steps rather than fizzes. The held value is hashed from the
cycle number rather than drawn from a generator, so the same moment of the show
always gives the same value: a random that differs between the rehearsal and the
performance is not usable.
It can follow the tempo. VJ mode already has a tap tempo, so an LFO that
ignored it would be a second clock in a machine that already knows what the
music is doing. Locked, the cycle is measured in beats — a quarter of a beat up
to thirty-two — and it steps through musical lengths rather than by a fixed
amount, so every stop is a length someone would actually choose. Free-running,
the rate is multiplied rather than added: the useful range runs from one cycle
a minute to several a second, and a fixed step would take a hundred clicks at one
end and skip the whole interesting part at the other.
The swing is centred on the value you set, so switching an LFO on never jumps
the picture: it starts moving from where the parameter already was, and averages
back to it. It is clamped to the parameter's own 0–1, and near the ends the swing
goes lopsided rather than out of bounds.
The oscillators are evaluated outside the effects, into a modulated copy of
the stack. The effect code is unchanged and stays a pure function of its inputs —
which is what lets it be dumped headlessly, benched, and applied by the output
and the preview independently. Both paths read one clock sampled once per frame,
so the operator's monitor and the audience's screen are never at different
moments of the same oscillator. A cue with no LFO does not pay for the copy.
Saved on the end of each effect entry and only when armed, so every show ever
saved still loads — and a show saved here still loads in a build that predates
the feature, which stops at the last parameter and ignores the rest. That is
right: it has no oscillators to run.
FX LFO <n> <A-E> on|off|shape|rate|depth|phase|sync|beats [value] over the
wire, where E is the amount.
Effects were unreachable on every cue that is not a video
The EFFECTS section existed only in the video branch of the inspector, and the
engine has never cared: a pattern, a still, a camera, an NDI feed or a stream
carries an effect stack and renders it exactly as a clip does. Proven by adding
grain, caustics and a vignette to a colour-bar pattern over the wire and watching
all three come out. There was simply no way to reach any of it without a video
cue selected — a whole feature, applied to most of the cue kinds, with UI on one
of them.
The code source gets a real editor
The expression sat on one inline row in the inspector column, ellipsized. That
is the right widget for a number and the wrong one for a program: you could not
see the whole thing, the caret could only ever be at the end, and every
character was the same colour, so a mistyped function name looked exactly like a
correct one until the picture stopped changing.
It opens into a proper editor now. The text is syntax coloured — functions,
values, numbers, brackets, operators and the commas that split red from green
from blue each have their own colour, and a name the compiler will refuse is
red while you type it, before you find out by looking at the output. The
caret moves with the usual keys and you can click into the text to place it.
The colouring reads the compiler's own tables rather than keeping a copy, so it
cannot fall out of step with the language: "shown in red" means exactly "this
will not compile".
Every variable and function is a chip that inserts itself — a function
arrives with its brackets and the caret already inside them. The examples are
a picker rather than a button that cycled: ten presses to reach the tenth, no
way back, and no way to know what you were about to get.
And there is a friend in the corner, the same face that waits in an empty
program monitor, who tells you what the name under your pointer does — and reads
you the compile error when there is one. A syntax reference is a wall of names;
someone telling you what the one under your finger means is the same information
with a face on it.
CODE GET | CODE SET <expression> | CODE EDIT over the wire, so an expression
can come from a controller or a script and not only from typing.
Caustics: the light, not just the bend
Every "water" effect displaces the picture. This one also computes what the
water does to the LIGHT, which is the part the eye actually reads as water.
A refracting surface bends what you see through it and, in the same motion,
concentrates or spreads the rays doing so. Where neighbouring rays are pushed
toward each other the brightness piles up, and those bright filaments are
caustics -- the moving net of light on the floor of a swimming pool. The
focusing term is the DIVERGENCE of the displacement field: one finite
difference per cell, and it is the whole difference between this and a ripple.
Four crossed waves at different angles and rates, so it never reads as a grid.
Chop runs from long ocean swell to rain on a puddle, swell speed sets
the rate, focus how hard the light gathers -- through a tanh, so a strong
swell makes filaments instead of clipping to white. It saturated into hard
black and white bands on the first attempt; the curve is the fix. 1.9ms at
1080p.
Feedback that cannot run away
A camera pointed at its own monitor, except the transform between passes is
chosen rather than accidental -- and bounded, which is what makes it usable on
a stage.
Scale the echo slightly up and it walks toward you as a tunnel; scale it down
and it retreats; add a turn and the tunnel becomes a spiral; slide it and it
smears into a comet. Those are the four controls, because that is the loop:
zoom, spin, drift, and colour bleed for a trail that changes
colour as it fades rather than only going dim.
Real feedback blows out to white the moment the loop gain passes one, and there
is no getting it back during a show. Written the physical way -- add the echo
to the picture -- a colour bar went to clipped white in twenty frames, a third
of a second. So the echo LIGHTENS instead of adding: the brighter of the live
pixel and the decayed echo. Adding has a fixed point several times the input;
lightening has its fixed point at the input, so the picture can never come out
brighter than the picture went in. Measured over 120 passes it settles and
stops moving, to within three levels out of 255.
The loop is cleared at every take, so a new cue never opens with a ghost of the
last frame of the old one. Two outputs showing the same deck step it once
between them, not once each. 1.0ms at 1080p, after the source coordinate became
fixed-point stepping and the echo became three tables -- 5.4x faster than the
straightforward version, and byte for byte the same picture.
Motion puppetry has memory now
The puppet followed one frame of the driver's motion and let go of it
immediately, so it could only ever twitch. It now has a spring and an
accumulator: memory is how much each frame's motion adds to what is already
there, spring how fast it returns to rest. Both are needed -- memory alone
runs away, a return alone never builds. Measured on the same driver, the mean
displacement went from 10.5 to 25.9 grey levels per pixel.
memory 0 returns the raw per-frame field, which is exactly what it did before
and what every show saved until today carries, so none of them change.
Pixel sort rendered differently on macOS than on Windows
std::sort says nothing about how it orders elements the comparator considers
equal, and two standard libraries do not have to agree. Sorting a run by luma
alone left every equal-luma pixel free to land anywhere, so the same cue on the
same frame came out visibly different under libc++ than under MSVC -- 102 of
the sampled bytes differing, by as much as 226. The comparator now falls back
to the packed pixel value, which is a total order, and both platforms render
the same frame.
Time-based effects were frozen on stills
A still cue decodes exactly one frame, and both render paths skip re-applying
the effect stack when the source frame has not changed. That gate is right for
the effects it was written for and it cannot know about the ones that advance
with time -- so on a still, grain did not move, a ripple stood perfectly still,
and caustics and feedback, whose entire subject is motion, were one arbitrary
frame of themselves. Measured on a static colour-bar cue: 0.0% of the monitor
changed between two shots a second apart. It is now 9.7% with grain, 2.0% with
a ripple, 2.8% with caustics.
The stack knows which of its effects animate, so a still re-renders only when
one of them is present, and it is driven by the app's frame counter -- the only
clock available when the picture itself never moves. Video is untouched: the
look still follows the SOURCE frame, so a given frame of a clip always grades
the same way and a recording stays reproducible.
check_effects_offline.py --animation renders every effect at nine frame
indices and fails if the header's list disagrees with what the pixels do. This
is exactly the class of bug that hides: the effect renders correctly, once, and
every other check passes.
The preview sweep was checking effects against a test card
For however long it has existed, check_preview_effects.py rewrote the test
cue's media PATH but not its KIND. On a machine whose saved show happens to
start with a pattern cue, the path was simply ignored and every effect was
being verified against a generated test card rather than the clip it thought
it was using.
Most effects changed the card enough to pass anyway, which is why it went
unnoticed. Night eyes did not — an effect whose job is removing colour does
almost nothing visible to a near-monochrome test card — so it reported as
completely dead while being perfectly correct. Against the actual clip it
changes half the monitor.
A check that passes for the wrong reason is worse than no check, and this one
was passing for the wrong reason on every effect at once.
It was also blind to a whole CLASS of effect. Counting differing pixels
cannot see a smear: stroking along a feature preserves the local average
almost exactly, so grain flow read as 0.8% while visibly softening the
picture. The sweep now also measures the monitor's total edge energy, which
a smear unmistakably drops -- grain flow comes out at -11%. And the test
clip is one frame held for its whole length, half fractal and half colour
bars, so the result no longer depends on where the seek landed or on which
kind of content a given effect happens to need.
Smaller things
--effect-dumptakes a pass count, so an effect whose whole subject is what
happens across frames can be rendered headlessly. Feedback's first pass only
fills its buffer; there is nothing to look at until the second.tools/check_effects_offline.py --paramswas reporting motion puppet's two
parameters as dead on every run -- it has no motion vectors when called
directly, which the main sweep already knew and the parameter sweep did not.
A gate that always fails is a gate nobody reads.- The text timeline and the cue inspector sat hard against their margins.