v1.21.0

canticle v1.21.0

Highlights

A throughput collapse that got worse the longer the service ran. The Musixmatch client backs off when the provider signals throttling, but that back-off ratcheted up with no path down a real deployment could reach. A short burst of genuine 401s degraded throughput until restart. Where this was found, three throttle events had taken the effective interval from 15 seconds to 120 seconds and held it. Real work per item is about 0.2 seconds, so the cycle was almost entirely the service waiting on itself.

The recovery path existed but could not fire. Stepping down required five consecutive catalog hits. In the measured window, 52 items settled and 2 qualified; most were benign misses, which never reach the pacer. The audio detector made it worse: since v1.19.0 it settles instrumentals without going through Musixmatch, but its lane had no pacer, so that work stopped crediting recovery and nothing noticed.

Back-off now decays on time rather than on luck. One level per throttle-free interval, evaluated when the next request is paced. Twenty minutes per level, so a full unwind takes an hour: long enough that a brief lull does not erase the back-off and re-provoke the provider, short enough to recover within a day. Decay never goes below the configured minimum, and a fresh throttle signal re-applies immediately.

Throttle detection is unchanged, deliberately. Those signals were real provider push-back, so the fix is entirely on the recovery side.

The detector's statistics stop under-reporting. Instrumentals settled before the detector became a lane were never attributed to it, so the dashboard showed a fraction of its work. scan reconcile-detector-stats backfills them from recorded data, hits and misses together.

What's changed

Fixed

• The adaptive request pacer steps back down after a throttle-free interval, instead of holding an elevated interval until restart (#492)
• Tracks settled locally by the audio detector credit pacer recovery, without paying the provider pacing cost they never incurred (#550)
• A configuration read in the pacing path happened outside its lock (#494)

Added

• scan reconcile-detector-stats corrects the detector lane's undercounted statistics. Dry-run by default, --yes to apply, with a JSONL backup (#537)

Upgrading

No configuration changes are required.

Restarting alone gives immediate relief, because the back-off level lives in memory and a fresh process starts at the base interval. Throughput will jump on upgrade regardless of what this release contains. The signal that matters is whether it is still healthy hours later, after some genuine throttle signals have come and gone.

The backfill is optional and writes nothing until you pass --yes:

canticle scan reconcile-detector-stats          # preview
canticle scan reconcile-detector-stats --yes    # apply

Re-running it is a no-op. Rows with no recorded verdict are reported as an uncovered remainder rather than estimated.