The beat layer: tempo, its confidence, and downbeats

[robert-dean]

Status (2026-10-10)

This file began as "where per-track measurement comes from", and almost all of that has an answer.

• The measurement path is built. analysis/ is a Python sidecar answering POST /analyze,
  plugins/analyzer/ is the adapter, analysis is a capability of its own
  (packages/plugin-sdk/src/capabilities/analysis.ts), and deadair.track_analysis (migration
  0005) holds a schema version and a completeness flag. The four cue points come off an RMS envelope
  and the loudness layer is a BS.1770 implementation in analysis/loudness.py, read through
  playout/gain.ts. The sidecar runs under nice (Keep the stream on time: analyzer priority, repeated breaks, show overruns #223), measures held copies first and fetches
  nothing while the audience gate is shut (Analysis: no provider downloads while nobody is listening #399).
• The licence question is answered by analysis/README.md § "The rule, stated once": nothing
  copyleft or non-commercial enters the analysis path, weights included, and the sidecar exists to
  keep decoding out of Node and for no licence reason.
• The vocal fields came from somewhere else. Measuring them from audio was built and reverted on
  2026-08-30 (the negative result is kept below). They now come from synced lyrics (Lyrics: a store for the words of each record, and LRCLIB to fill it #411, Breaks: talk up to the post, from where the singing starts #412):
  lyrics/vocal.ranges.ts turns timed lines into stretches of singing, vocal.markers.service.ts
  publishes where the singing starts and stops, the talk-up limit reads the first, and an operator
  can override both (0064_track_vocal_overrides.sql). That is
  track-lyrics, and it is what this file's own
  conclusion asked for.
• The memory baseline exists. /health reports the worker's resident memory (_resident_mb in
  analysis/app.py), taken so there is a number from before the measurements get more expensive.
  Nothing acts on it yet.

What is left is the beat layer alone: bpm, beat_confidence and downbeats. None of them is in
analysis/*.py, and measure.py and ANALYSIS_SCHEMA_VERSION are both still 1. Its one consumer
is crossfades, "What the blend does inside
the buffer", which needs a beat grid and a tempo confidence on top of the four cue points. The
talk-up limit no longer waits on this file.

---

What has to be measured

The four cue points stand unchanged. What the transition work adds is a second layer:

Field	What it is	Who needs it
bpm	tempo, octave-ambiguous and stored as measured	the blend policy ladder
beat_confidence	0 to 1, how much the tempo is to be trusted	the ladder, decisively
downbeats	positions of bar ones, so an anchor can be quantised	phrase and downbeat snapping

beat_confidence is the load-bearing one, and it is the field a naive implementation omits. The
whole point of the policy ladder is that it refuses to beat-match on weak evidence, and a tempo
figure with no confidence attached cannot express weak. A measured BPM and a metadata BPM are also
different claims and should not be stored in one column: the rule the ladder enforces is that
metadata alone never authorises beat-matching, which is unstateable if the provenance is lost on
write.

Two guards from a comparable station's worst bugs in this exact layer, added 2026-08-28. Both are
the naive implementation rather than bad luck, and both are cheap to hold and expensive to retrofit:

• The confidence has to be WRONG-TEMPO-AWARE, not merely present. A tempo detector run without
  octave correction reports double-time on slow material and scores the wrong tempo as fully
  plausible. A number that says "this is definitely 140" when the record is 70 is worse than no
  number at all, because the ladder above is built to trust it. The row for bpm says
  octave-ambiguous; this is the same fact asked of the field beside it.
• A detector at its default threshold produced enough false positives to break the cue points
  that depended on it. That was their vocal detector, and the lesson is general: whatever threshold
  the beat pass ships with is a measured number on this library, not a default.
• The beat pass gets its own try, and leaves its fields out on failure rather than failing the
  whole measurement. bpm, beat_confidence and downbeats sit behind one try/except of
  their own; a beat-tracking failure on one record answers with the four cue points and the loudness
  figures still present and the beat fields simply absent, rather than the row failing outright and
  taking a working measurement of the rest of the track down with it. _analyze today has no reason
  to isolate a failure this way (cue points and loudness both come off the same decode, and either
  one raising is genuinely fatal to the row), but the beat layer is a second, riskier detector added
  later on top of a row that already works, and the negative result above is the reminder of how
  wrong a detector in this exact spot can be while still returning a number.

One operational note for the sidecar, same layer. A long-running Python process that decodes
whole records accumulates resident memory the allocator does not return, independent of any leak in
the code: their analysis worker grew unbounded over long uptime for exactly this reason. analysis/
has the same shape. It now REPORTS its resident memory, so there is a baseline; the cheap answers
(trim thresholds, or recycling the worker every N records) are worth having in hand before the beat
layer makes each measurement more expensive, and are now a decision against a number.

The vocal fields cannot be measured this way, and now do not need to be

Built and reverted 2026-08-30. The licence rule in analysis/README.md puts vocal_onset and
vocal_curve outside the licence question ("two of the five are not part of the problem at all",
computed from "band-limited energy, 200 Hz to 4 kHz"). That is correct about licensing and wrong
about feasibility, and the difference cost a day.

Band energy alone measures nothing new, which was caught at design time: 200 Hz to 4 kHz is the
band intro_end already uses, because it is where voices and lead instruments live. The
replacement was the standard speech/music discriminator: a voice is SYLLABIC, so measure the
envelope's own modulation at 2 to 8 Hz against the level it is modulating. Pure scipy.signal, no
model, no weights.

It works on synthetic signals and carries no vocal information on real music. Measured through
the sidecar against this library:

record	vocal enters	presence before	after	step
Rush, YYZ	never (instrumental)	19.5	14.9	−4.6
Kansas, Dust in the Wind	0:22	12.0	14.9	+2.9
Rolling Stones, Gimme Shelter	0:35	20.5	18.4	−2.0
AC/DC, Highway to Hell	0:11	22.9	19.3	−3.6
Black Sabbath, Paranoid	0:12	13.7	18.2	+4.6

The steps are ±5 with a sign unrelated to the truth, and the instrumental's own drift (−4.6) is as
large as any of them. Across twelve records the presence distribution was p50 17, p90 44 with no
bimodality for a threshold to cut, three of seven onsets landed on cue_in, and YYZ, an
instrumental, was given an onset at 0:00. A refinement contrasting the vocal band against a
60 to 160 Hz band (drums move both, a voice moves only the mid) collapsed to zero almost everywhere.

The reason, which is the part to keep: syllabic-rate modulation in the vocal band is not
specific to voice in rhythmic popular music, because drums, rhythm guitar and arpeggios all modulate
at 2 to 8 Hz. The synthetic negative control was a held tone, and no real instrumental is a held
tone, so the tests passed while the feature did not work. A detector whose negative case is
easier than reality is a detector that ships.

Three things worth having from it:

• The calibration pass is what caught it, exactly as the threshold guard above asks. It was worth building
  before the threshold was pinned rather than after, and a walk that had swept the library first
  would have written 762 confident wrong rows.
• to_mono rectifies, which is now documented in loudness.py and matters to anything spectral
  the beat layer adds. Folding as sqrt(mean(x²)) lifted a 60 Hz bassline into the mid band by
  64 dB. The cue points are calibrated against that signal; a filter must not read it.
• What would work is a model, and the honest next step for this field was not another DSP
  attempt. track-lyrics argued the better route, which needs no audio at
  all: a synced lyric's first timestamp is "a number somebody typed while listening". That route
  shipped (Breaks: talk up to the post, from where the singing starts #412), so this result is now the record of why it was taken.

For a record with no synced lyric there is still no vocal curve, and this result is the reason
not to try again from the signal. The coarse curve's two uses were filter depth during a blend and
outro talk; the lyric ranges answer the second wherever a lyric exists. If the blend ever needs the
first for records without one, the honest route is a permissively licensed model under the rule
above, not a third DSP attempt.

Bumping the schema version re-measures the library

The beat fields change the row's shape, so they come with SCHEMA_VERSION = 2 in measure.py and
ANALYSIS_SCHEMA_VERSION = 2 in the SDK, and the staleness rule this table was built on then reads
every existing row as stale. That is the design working: reanalysis is automatic rather than a
migration. It is also every record in the library fetched and decoded again, through a walk paced to
fifteen an hour and gated on the audience. Two things follow:

• Stale has to mean "re-measure", never "untrusted", and today it means both.
  AnalysisRepository.trustedAnalysisFor filters schemaVersion >= ANALYSIS_SCHEMA_VERSION, so a
  bump as the code stands drops every record's cue points at once, and the station gets them back over
  weeks as the walk grinds through. The fields a version-1 row carries are still right under
  version 2, so the readers of THOSE fields should accept any version that has them, and only a
  reader of the beat fields should require 2. Settle this in the same commit as the bump, not after.
• The beat pass's own failure must not count as the row's. The guard above (its own try, its
  fields simply absent) is what keeps a version-2 row from being worse than the version-1 row it
  replaces.

What this does not solve

Nobody sells these numbers. §3 checked this on 2026-08-10 for the four points and the answer is
the same for the beat layer: catalog upstreams return tempo, key and energy, none of them returns a
downbeat grid, a beat confidence or a vocal onset. The one large free corpus of computed descriptors
is a fixed 2022 dump, useful as a cold-start layer for back catalogue keyed by recording id and
nothing at all for recent releases. So this is measured locally or not at all, which is why the
sidecar was the whole question, and why the beat layer goes into it rather than beside it.

Analysis cannot be a precondition for airing. Whatever this becomes, an unanalysed track has to
play. Every consumer listed at the top degrades to a defined behaviour when the row is missing (plain
fade, no talk-up, original ordering), and that is a property to preserve deliberately rather than a
gap to close.

[robert-dean]

Status check, 2026-09-22 (main at 23b983e)

Done: the sidecar, plugins/analyzer, the analysis capability, the track_analysis table, the four cue points and loudness through playout/gain.ts. The analyzer now runs under nice (66c5a6b, #223).

Still open: the beat layer. No bpm, beat_confidence, downbeats or vocal_curve in analysis/*.py, and measure.py is still SCHEMA_VERSION = 1. vocal_onset was built and reverted; #47 is the suggested alternative. The sidecar worker recycling note is also not done.