RFC: Scaling output samples when switching audio configuration or CPU

[tfry-git]

I don't know how many people are playing / will be playing with Mozzi on different CPUs, but one of the annoying aspects while cross-testing is that different ports expect a different audio resolution to be returned from updateAudio(). The same problem applies when switching between available audio configurations on the same CPU.

An example written for StandardPlus (8.5bits or so) will be somewhat quiet on the STM32 (10 bits), very quiet on the Teensy (12 bits), definitely too quiet in HIFI mode (14 bits), and barely audible on the upcoming ESP8266 port (16 bits). That's rather sad, as everything else in a typical Mozzi sketch can be expected to work, unchanged. You would not believe how many times I went hunting for "what did I break now" while cross-testing, when the whole problem was that I forgot to adjust updateAudio() to my current configuration. I wonder how many users will experience similar woes when trying something new. Another symptom of this problem is "duplicate" examples for standard and HIFI modes.

So: Wouldn't it be cool, if Mozzi could do the necessary scaling all automatically? (Yeah!) But how?

1. A first, naive approach would be to make all ports assume the value returned by updateAudio() are 8 (or 9) bit resolution, these could then be scaled up to the output bits inside audioHook(), for those ports/configs that support more bits. But this approach has severe problems:

• Breaks existing sketches using higher resolution in updateAudio()
• Adjusts the volume, but limits the effective audio resolution to 8 bits, where some ports could offer much better quality.

2. A second approach would be to make all ports assume samples scaled to - say - 16 bits returned by updateAudio(), and scale down from this, automatically, as appropriate. As the intermediate resolution is high, each port can support maximum quality. However, this time, the problems are:

• Breaks the vast majority of existing sketches returning 8-9 bits in updateAudio()
• Places an additional (small) processing requirement on the lower end CPUs, where processing power is already most scarce.

3. A third approach will be to use an opt-in "best practice" solution to automatic scaling. This would specify the number of audio bits generated, and auto-magically scale those to the appropriate output resolution. To elaborate, this would rest on macros like this (where AUDIO_BITS is the desired output resolution):

// Assume that "value" is scaled to "bits" resolution, and scale it to the output resolution appropriate for the current configuration
// As long as bits is a compile time constant, the compiler should be smart enough to optimize this to a static shift operation (or nothing)
#define SCALE_AUDIO(bits, value) ((bits < AUDIO_BITS) ? (value << (AUDIO_BITS-bits)) : ((bits > AUDIO_BITS) ? (value >> (bits-AUDIO_BITS)) : value))
#define SCALE_AUDIO8(value) SCALE_AUDIO(8, value)
#define SCALE_AUDIO16(value) SCALE_AUDIO(16, value)

Then to "opt-in" to auto-scaling, users would be encouraged to wrap the return value in updateAudio() like this:

int updateAudio() {
   int ret = [terribly complicated maths resulting in 13 used bits];
   return SCALE_AUDIO(13, ret);
}

The third approach is much better, IMO:

• Being an opt-in solution, it cannot break any existing sketches, by definition.
• No need to limit the returned resolution, arbitrarily, but still any required up-or-down shifting will only need to be done once.

Now, that's all fine and dandy, but then there's the (important!) special case of "StandardPlus", where the supported audio output resolution is "almost, but not quite 9 bits". There, we cannot assume that we can safely shift samples to 9 bit resolution, but shifting everything to 8 bits would be wasteful. What to do?

Looking at the existing examples for StandardPlus, these seem to fall into two categories: a) One simply returns exactly 8 bits, assuming the full range will be used - e.g. direct output from an Oscil. b) The other category returns values that will not usually, but might reach 9 bits. E.g. "08.Samples/Samples_Tables_Arrays". Here, the return value is constrain()ed to the actual -243..244 range before returning.

My current idea is to for dealing with this is (I hope I'm explaining it well):

• No special handling for case a). This will simply use SCALE_AUDIO(8,...) or SCALE_AUDIO8(...), which will be set to not do any scaling for StandardPlus, or more precisely, this will shift to a target level of 8 bits in StandardPlus (if necessary). On other ports/configurations, the input will be scaled as shown above.
• For case b) add a second set of macros SCALE_CLIP_AUDIO(9,...) or SCALE_CLIP_AUDIO9(...). This will signify that the supplied values will have around 9 bits, possibly even more, but usually less. It will (not) shift to a target level of 9 bits in StandardPlus, but also add add a constrain(). On other ports/configurations, the input will be scaled as if it was 9 bits (and also constrained).

Your thoughts?