Cap VBR frames by the buffer, not the rate

[thomas-vilte]

Description

In VBR, the encoder was still capping each frame at the nominal bitrate, so a difficult frame could only undershoot — it could never use extra bits and make them back later. libopus instead caps the frame using the buffer size provided by the caller:

nbAvailableBytes = (target+(1<<(BITRES+2)))>>(BITRES+3);
nbAvailableBytes = IMAX(min_allowed,nbAvailableBytes);
nbAvailableBytes = IMIN(nbCompressedBytes,nbAvailableBytes);

In VBR, nbCompressedBytes is max_data_bytes (capped at 1275), not vbr_rate. The reservoir keeps the long-term average in check rather than putting a hard ceiling on every frame.

I also ported min_allowed, which was missing:

min_allowed = ((tell+total_boost+(1<<(BITRES+3))-1)>>(BITRES+3)) + 2;

pion was using a fixed floor of 2 bytes, without accounting for what had already been written or the dynalloc boosts.

I checked what budget each stage sees in the reference before the VBR block to make sure this change didn't affect anything else. Instrumenting celt_encoder.c at 96 kbps VBR, effectiveBytes is 240 (the nominal rate) while nbCompressedBytes is 1275. The buffer only comes into play in the final clamp; dynalloc and the TF gate still see the nominal rate. This PR only changes that clamp.

The prefilter does see 1275 in the reference versus 239 in pion, but its thresholds (<25, <35, >12*C) don't change within that range. It would only matter below roughly 14 kbps, so I'm leaving that for now.

Measurement

Average packet size over 200 music frames:

bitrate	before	after	libopus
24000	54.30 (90.5%)	56.99 (95.0%)	53.50
32000	73.50 (91.9%)	77.58 (97.0%)	—
48000	111.97 (93.3%)	118.80 (99.0%)	113.71
64000	150.44 (94.0%)	160.00 (100.0%)	—
96000	227.51 (94.8%)	242.56 (101.1%)	244.03

VBR can now actually use the available headroom instead of being capped at the nominal frame size.

Round-trip SNR against libopus at the same bitrate:

bitrate	before	after
24000	+0.188	+0.286
32000	−0.043	+0.063
48000	−0.150	−0.055
64000	−0.124	+0.042
96000	−0.448	−0.292

Constrained VBR ends up at 97.6–98.5% of the target, with less variation than unconstrained VBR, as expected.

What doesn't match yet

At 96 kbps we're still about 0.29 dB behind libopus with roughly the same bit usage (242.6 vs 244.0 bytes), so the remaining difference is more about where the bits go than how many there are.

Looking at the distributions on the same clip, pion has more spread around the average (stddev 49.1 vs 37.6) but a smaller range (223/453 vs 206/477). I haven't isolated the cause yet.

The likely candidates are the remaining compute_vbr terms: stereo_saving and temporal_vbr are straightforward to port, while the tonality boost depends on analysis.c.

Tests

TestVBRUsesBufferHeadroom checks that a demanding frame can go above the nominal bitrate.

I deliberately didn't put an upper bound on the average in that test. On a sustained difficult signal there may not be easy frames to recover the bits, and the reference behaves the same way — with the same PCM, libopus averages 420.7 bytes against pion's 402.6 for a 240-byte target.

The long-term target is covered separately by TestVBRTracksTargetBitrate using material where the reservoir can actually recover.

Reference issue

Part of #34.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.08%. Comparing base (ad863fc) to head (be5a5cc).

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #203      +/-   ##
==========================================
- Coverage   93.09%   93.08%   -0.02%     
==========================================
  Files          58       58              
  Lines       10328    10337       +9     
==========================================
+ Hits         9615     9622       +7     
- Misses        503      504       +1     
- Partials      210      211       +1     

Flag	Coverage Δ
go	93.08% <100.00%> (-0.02%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.