Palinopsia v1.2.8 : sound in the dome

Palinopsia v1.2.8 : sound in the dome.

A release about where the sound is. Sonify's voices can now leave the stereo pair and
take their places in the room, the way the picture takes its place on the dome : each
voice sounds where it plays in the picture, on your own speakers or through SpatGRIS,
the renderer of the SAT Satosphère's 93 speakers. Their places can be played by hand,
by modulators, Meta knobs and OSC, heard on headphones anywhere, and the Collage can
sound piece by piece. Sonify's sound gets finished like a mix (an output EQ to tune
the room, one compressor that holds every film of a Collage at one level, a glue on the
whole), 18 scales, and a key sequencer whose Picture really follows the picture. The
projection warp gets a feather, and the Collage and Toile draw in about half the time.

Highlights

• Sound in the dome. Sonify's voices can now be placed in the room the way the
  picture is placed on the dome (Sonify → ◎ space). Each voice sounds where it plays in
  the picture, carried through the fulldome mapping : a voice at the zenith of the
  picture comes from above, and turns with the dome when it spins. Or put a voice at a
  fixed place, widen it, spread it, set it spinning. Two ways out : Palinopsia pans the
  voices onto your speakers itself (rings and domes built in, or a SpatGRIS speaker
  setup, up to 32 channels), or it sends each voice on its own channels, with its
  position over OSC, to SpatGRIS, the renderer the SAT Satosphère runs for its 93
  speakers and 5 subwoofers (ADM-OSC too). The SAT button sets it up for the
  Satosphère; recordings keep the stereo mix, which is also the stereo backup the
  venue asks for.

• Play the space. Every voice's place in the room can be moved by a modulator, a
  Meta knob or OSC : click a voice in the space view for its sliders and their M. A
  modulator on the azimuth spins a voice round and round the dome.

• Hear the dome on headphones. 🎧 renders the speaker layout (or SpatGRIS's objects)
  around your head in stereo, so a dome piece can be prepared anywhere.

• The Collage, piece by piece. The Collage voice's pieces can each be a sound of
  their own, coming from where the piece is on the dome : a wall of films becomes a
  room of films.

• Satosphère in one click (Output → Fulldome) : a 210° domemaster at 4096 × 4096,
  the whole picture over the whole dome, NDI at 60 fps, and the voices as objects for
  the venue's SpatGRIS, exactly as its guide asks.

• An output EQ for Sonify, to tune the room (Sonify → mixer, under the FX tail) :
  twelve bands after everything Sonify plays (low cut, low shelf, eight bells, high
  shelf, high cut), each with its own Q and a slope of 6, 12, 24 or 36 dB per octave.
  A small curve and two meters on the card; ⤢ edit opens a large editor with the
  live spectrum behind the curve and every point to drag. Input and output gain with
  meters, a soft ceiling under 0 dB. It plays to the speakers only (recordings stay
  as the piece), and presets and the sequencer leave it alone.

• A windy film, tamed (Sonify → Collage) : films in turns the films down before
  the resonators, hp and lp filter every film (the wind's rumble, the crackle),
  and ring in sets how hard they drive the Ring bank, so a hot recording rings
  without distorting.

• 12 or 24 dB/oct for the Collage voice's filters (the chip beside hp and lp), and
  the Prism reverb's crossover and the Collage Ring bank's cutoff read in Hz.

• One level for every film (Sonify → Collage → comp) : one compressor on all the
  films together, never one per film, so it costs next to nothing. A loud film comes
  down, a quiet one comes up a little; at full, two films 26 dB apart sit 6 dB apart.

• Glue for the whole mix (Sonify → mixer, under the EQ) : one knob, five voicings
  (clean, warm, punch, bite, slow), after the EQ and before its ceiling, with the
  reduction shown live.

• The Chord's number of voices can be modulated : an LFO or a Meta knob walks the
  chord from 2 notes to 16, re-spreading it at each new count.

• 18 scales for Sonify and its key sequencer, the full set of Berklee's scale
  reference : harmonic minor, mixolydian, locrian, the major pentatonic, the two blues
  and the two diminished join the others. Pivot now walks all seven modes of a key.

• Dial a Sonify slider finely : Shift-drag moves any of them ten times slower, and
  the Collage's filters and levels can be typed (4250, 4.2k). The Ring bank's
  controls grey out while its wet is 0, so it is clear they wait for it. The cheat-sheet
  (?) lists Sonify's gestures.

• Feather for the projection warp (Output → Mapping) : the warped frame's edges can
  fade into the black over up to 200 projector pixels, rounded at the corners.

Fixed

• The key sequencer's Picture really follows the picture. A frame's overall color
  is mostly its footage's cast and barely moves, so a piece used to sit on one or two
  roots (a warm one on C, G and D). Picture now reads the picture against the piece's
  own colors and light over the last 40 seconds or so, spreading them over every key
  and mood : the root walks the circle as the colors shift. The new range sets how
  far (0 = the plain color wheel, as before).

• The Chord's spread opens both ways. Raising it used to pull the chord to the left
  (a bright sky up to 19 dB); its notes now fan out in mirrored pairs and stay centred
  on whichever notes are loud.

• Switching Sonify off really stops it. Each off and on used to leave the previous
  engine running unheard in the audio thread, the whole last patch, until the app was
  closed. It now ends.

• Space on headphones no longer distorts. The binaural mix ran almost 8 dB hotter
  than the stereo one and the sound card clipped it (worst on dense films, like wind).
  It now plays at the stereo mix's level, with a limiter after it.

• The output EQ's points drag again, at any UI zoom : a point dragged across sets
  its frequency, up and down its gain, the wheel its Q. (Only the wheel
  worked when the interface was zoomed.)

• Space, easier to play : drag any number up or down, drag a voice's dot on the
  room map, see every Collage piece playing as a small pink dot, hear it on
  headphones from a choice right under the output, and find each voice's M under a ▸.

• A keystoned projector edge no longer steps. A nearly vertical side of a warped
  frame came out as a staircase of very long one-pixel steps, read on the wall as long
  lines down the sides. Every warped edge is now anti-aliased, crisp at feather 0.

• The iPhone feed costs a third as much : a portrait phone now arrives at the
  height of the picture it is drawn into (1080) instead of 1920.

• Sonify's two taps are read in one pass, which on a Mac halves the time the
  window spends waiting on them.

• The piled Collage walls draw in about half the GPU time (torn, shaped, a
  stacked mosaic or cut-up, FX before the shapes), bit for bit the same picture :
  shaped 10.2 to 5.9 ms a frame at 1080p, torn 14.0 to 7.6.

• Toile costs 60 % less at PAINT 0, bit for bit the same picture : its
  painterly pass no longer runs when it is mixed in at zero (7.9 to 3.1 ms a frame at
  1080p when Toile draws its line work alone).

• A Collage loads only the films a piece can show. films can be 30 or 55 for
  variety at no cost : every deal draws from the whole folder anyway, and a wall of
  10 pieces loads about 11 films. Pieces wait far less for a fresh film (0.25 s
  instead of 1.1 on an installation's wall, 0.5 instead of 10 at 55 films).

• Sonify's Collage voice reads a hundredth of the data : each film's sound now
  plays from a small audio-only copy of the original, identical sample for sample,
  instead of the whole iPhone video file.

• The Performance tab no longer blames Sonify for a full GPU. Waiting for the GPU
  to hand a picture back now has its own row, GPU wait; Sonify's own work on an
  installation's session is 0.3 to 0.6 ms a frame, not the 8 to 30 it showed.

• The Collage voice reuses its players : a film looping or re-cueing takes over
  its own spare player instead of opening and decoding its file again.

• The media keys and the Now Playing panel leave the instrument alone (the app no
  longer announces every Collage film to the system, and a play / pause key can no
  longer pause its sound).

Opening an older session

Everything opens as before : Space starts in stereo (the mix as always), the output EQ
is flat, the glue is off, the Collage's comp is at 0, the warp's feather is 0, and the
Collage and Toile draw the same picture bit for bit. A session saved now carries where
each voice sits in the room (spaceVoices); an older one starts the voices where they
begin. Three things sound different on purpose :

• The key sequencer's Picture reads the picture against the piece's own colors and
  light (range 100 %), so it walks more keys than before. Set range to 0 for the old
  plain color wheel.
• The Chord's spread stays centred : a chord that used to lean left now opens both
  ways (same width).
• "pentatonic" shows as the minor pentatonic it always was (the same notes). Over
  OSC, /opsia/sonify/scale keeps its first eight indices; a FLOAT sent there now
  spreads over the 18 scales.

Install

Platform	File
Windows	Palinopsia-1.2.8-win-x64.exe (installer) or Palinopsia-1.2.8-portable.exe
macOS (Apple Silicon)	Palinopsia-1.2.8-mac-arm64.dmg
Linux	Palinopsia-1.2.8-linux-x86_64.AppImage or Palinopsia-1.2.8-linux-amd64.deb

Builds are unsigned :

• Windows : SmartScreen may warn, click More info → Run anyway.
• macOS (Apple Silicon) : not notarized. Drag Palinopsia into Applications first
  (run from the disk image or Downloads, macOS runs a hidden temporary copy), then run
  xattr -cr /Applications/Palinopsia.app once in Terminal, or open it and click Open
  Anyway in System Settings → Privacy & Security. The first launch asks for the local
  network, and for the camera and the microphone when you use them : accept, or OSC, NDI
  and light output stay silent. Screen capture is the one permission macOS never prompts
  for : switch Palinopsia on under Privacy & Security → Screen & System Audio Recording,
  then start it again.
• Linux : chmod +x the AppImage, then run it.

Intel Macs (x64) : this release ships Apple Silicon only. On an Intel Mac, build from
source (npm install, then npm run build:mac).

Notes

• One audio output carries at most 32 channels (Chromium's limit). A studio ring or a
  dome up to 32 speakers plays from Palinopsia directly; a bigger system, such as the
  Satosphère's 93.5, plays through its own SpatGRIS, with the voices sent as objects.
• To reach a venue's Dante network, choose a Dante Virtual Soundcard as Sonify's
  output. Space was measured channel by channel inside the engine and over OSC; the
  first time on a real speaker rig or with SpatGRIS, allow time to check it, and line
  the sound's front up with the picture's with turn (and mirror).
• The SAT asks for multichannel performances to be prepared with its audio team, with
  a two-hour test on site and a stereo backup : the recordings keep the stereo mix.

For developers

• Space : shared/spatial.ts holds the conventions (SpatGRIS's : azimuth clockwise
  from the front, elevation above the horizon; x right, y front, z up), the SpatGRIS
  speaker-setup reader, the built-in layouts, VBAP with an MDAP span (written from the
  published method : no SpatGRIS code, which is GPL) and pictureToDirection, the
  exact inverse of engine/dome.ts in every mode. audio/space.ts runs at 30 Hz after
  Sonify : places, the gain matrix, OSC (/spat/serv deg, or ADM-OSC with the azimuth
  negated), the headphone directions. The soni worklet has a second output (32
  channels, SpaceBus : sources × 2 sides × channels, sub low-pass, one linked
  limiter); sonifyEngine.setSpace(msg) routes it to the device, or through HRTF
  PannerNodes for the headphones. A voice's place is a SonifyModParam
  (space.<voice>.<field>); the azimuth's matrix value is an unclamped turn the
  engine wraps.

• The Collage voice's output is a 32-channel ChannelMerger (CollageVoice.out) :
  the stereo bus on 0-1, Space's piece groups on 2-31; RingBank.process can hand its
  wet and the dry's share separately.

• The UI zoom is a CSS zoom on the app root : map a canvas's pointer through its
  box on screen (canvas.width / rect.width), never the device pixel ratio (the output
  EQ's points missed at any zoom but 100 %).

• Collage, LAZY FILMS (CollageSource : rankDecks, wants, release,
  realize) : the decks a wall of n pieces can show are the first n of the order
  rebuildCells deals films in; deal() cues every deck but loads only those.

• Collage, the piled pieces' handoff : the films pass writes the top piece (and a
  mosaic's runner-up and three nearest seeds) to an R32UI target attached as a second
  draw buffer for that one draw (oId), the shapes pass of the same frame reads it
  (uId, uHasId). The per-pixel tilt and lobe trigonometry stays (per-piece
  uniforms rounded differently : one boundary pixel a frame changed film).

• Toile at PAINT 0 skips its paint pass and binds a 1x1 black for uPaintT (no
  shader change).

• audioForCollage(src) (main, collage:audio IPC, window.api.collageAudio) :
  a film's audio-only copy in video-cache (<hash>.a.m4a, or an empty
  <hash>.a.none for a silent film). opsia-media:// serves .m4a. The Collage
  voice asks for the whole pool (CollageSoundSet.files) one file at a time.

• perfMeter.wait(ms) : a readback's wait for the GPU goes to its own gpuwait
  section and comes out of every section open around it. GridReadback.takeLatest
  and FrameCapture's harvest report theirs; a new readback should too.

• audio/eqMath.ts is the output EQ's maths (RBJ biquads, Butterworth Qs for the
  steep cuts, the curve the page draws); soni.worklet.js runs the same sections in
  a soni-eq worklet between the Sonify engine and the speakers (copied line for
  line : change one, change the other). recDest and the recording tap stay fed
  before it. sonifyEngine.eqSpectrum(post, out) reads the analysers either side.

• engine/warpQuad.ts is the one projection warp, shared by the compositor's present
  pass and the projector window (OutputPresenter). computeWarpVerts(corners, w, h)
  builds the quad 2 px larger than the frame, with the inverse perspective map at each
  corner (linear in screen space, divided in the shader), so its edge can be
  anti-aliased; it needs the drawn size, and both callers rebuild it when that changes.

• GridReadback.kickStrip(cellW, h, sources, ...) draws several sources side by side
  into one readback; Compositor.readSonifyGrids(taps, outs) replaces
  readSonifyGrid.

• The USB input addon's open(id, boxW, boxH) fits each frame inside a box (the old
  open(id, maxSide) still caps the long side).

• Sonify's dynamics : the Collage comp is one Leveler in the soni worklet on the
  films' bus before the Ring bank (its per-sample gain also scales Space's piece
  groups); the glue runs in SoniEqProc after the EQ's bands, before the output gain
  and the ceiling, with its config in SoniEq.glue (normalizeEq fills it). Both
  compute their gain every 16 / 8 samples. The worklets are benchmarked in Node by
  loading soni.worklet.js in a vm context with AudioWorkletProcessor /
  registerProcessor stubs.

• A stopped Sonify ends its worklets : stop() posts { t: 'end' } (their
  process then returns false) and drops their ports' handlers. A disconnected
  AudioWorkletNode otherwise keeps running while the context lives (it is kept warm
  between takes). End any new worklet the same way.

• Scales : SONI_SCALES keeps the old eight ids at indices 0-7 (OSC ints), the ten
  new ones after; menus use SCALE_ORDER + SCALE_LABEL (SCALE_SHORT in the bar,
  SCALE_ABBR on chips); sameNotes(a, b) treats major / ionian and minor / aeolian as
  one, and the key sequencer's pool and modes go by it. scaleFromName reads an id or a
  label.

• Picture (soniKeySeq.ts) : updatePicture keeps the palette memory every frame
  in picture mode, pictureView(range) gives the reading the key and the visual use.

• A modulated Chord count reaches the worklet as { t: 'chord', freqs }, sent only
  when the count changes (Sonify's modulation otherwise rides postMod's overlay, which
  never rebuilds a derived table).