Optimize FlexASIO latency

[dechamps]

The current limitations around latency are described in the README:

FlexASIO has not been designed with latency in mind. That being said, the current version should not add any latency on top of PortAudio itself. The thing is, due to the way ASIO works (static buffer sizes), PortAudio sometimes has no choice but to add additional buffering (which adds latency) in order to meet the requirements of both FlexASIO and the system API it's using.

The way buffer sizes and latency are handled in FlexASIO is very naive, as it was not designed to optimize for latency. That said, there might be some low-hanging fruit there. For example, if we're smarter about the buffer sizes we suggest to the ASIO host application, we might avoid certain situations where PortAudio has to do additional buffering.

Looking at the current code, the main issue is that we're calling Pa_OpenStream() with a specific framesPerBuffer value (the one set in ASIO), which is not great, as the PortAudio documentation explains:

Note: With some host APIs, the use of non-zero framesPerBuffer for a callback stream may introduce an additional layer of buffering which could introduce additional latency. PortAudio guarantees that the additional latency will be kept to the theoretical minimum however, it is strongly recommended that a non-zero framesPerBuffer value only be used when your algorithm requires a fixed number of frames per stream callback.

[dechamps]

(One thing to note is that, AFAICT, the Windows audio engine uses 20 ms buffer sizes internally - or at least I've never seen it use any other buffer size. It might be that, for "shared" backends like WASAPI Shared and DirectSound, just telling PortAudio to use a 20 ms buffer size might be enough to make the latency optimal, since the ASIO buffers and the Windows audio buffers would then be aligned. We should experiment with that.)

[dechamps]

Experiments with 20 ms buffer sizes did not yield any interesting results, see #10.

For letting PortAudio choose the buffer size, see #20.

[dechamps]

Excellent news: after examining the PortAudio WASAPI backend code, I was able to find out how to nudge it into "low latency" mode, where it uses the smallest possible buffers and switches to event mode (instead of pull mode). With that, I was able to achieve extremely low latencies, on the order of 10 ms(!). The way to do that is trivial: just set PaStreamParameters::suggestedLatency to zero. That's it. It doesn't even require a matching buffer size. This will inform the work in #19 and #20.

[dechamps]

Leaving this issue open to continue the investigation into latency with other settings and backends other than WASAPI.

[dechamps]

Some notes from my investigation. In all tests, bufferSizeSamples was set to 48 in an attempt to force the backend into the lowest latency mode possible. Hardware is some mundane USB Audio device. Note that only output latency was evaluated.

Results

MME

Reported latency is fairly similar to suggested latency. However, it will happily report a latency of 1 ms, which MME clearly shouldn't be capable of. That's evidence that the reported latency is a lie.

DirectSound

Reported latency is fairly similar to suggested latency. However, reported latency bottoms out at 2 ms, which DirectSound clearly shouldn't be capable of. That's evidence that the reported latency is a lie.

WASAPI Shared

Reported latency is fairly similar to suggested latency. Reported latency bottoms out at 20 ms, which is consistent with what we would expect (the internal buffer size of the Windows audio engine is 20 ms).

WASAPI Exclusive

This is the only backend I've seen which interprets a suggested latency of 0.0 specially: it will switch into event mode (instead of poll mode) and will generally exhibit vastly better latency. A non-zero value, even a very small one like 0.001, will not trigger this. This behaviour is surprising and might be a PortAudio bug.

With a suggested latency of 0.0, reported latency bottoms out at around 4 ms, which is plausible (although with such insane timings it's unlikely the thread scheduling will be able to keep up).

With a suggested latency of 0.001, reported latency bottoms out at around 30 ms, which is quite bad.

With a suggested latency of 0.030, reported latency goes crazy at around 250 ms. (Shared mode does not exhibit this behaviour, so, ironically, it reports a much better latency in this case.)

WDM-KS

Appears to follow specified buffer size and suggested latency to the letter - what you see is what you get (at least as far as reported latency is concerned). Reported latency bottoms out at 10 ms, which seems plausible.

Action items

• The default suggested latency should probably be changed to simply 0.0. This will make WASAPI Exclusive perform way better, and other backends should be just fine.
• Add these results to the documentation.

[ImNoGuru]

Am I doing something wrong with FlexASIO? I see output latency: 85ms. 1024 samples.

However, no pops or clicks for now.

I have set WASAPI Exclusive and latency at 0 by default.

[dechamps]

What buffer size are you using? With WASAPI Exclusive and suggestedLatencySeconds already set to zero, buffer size should be the only thing left to improve latency.