Add AudioPlayoutStats interface #2645

Hernqvist · 2025-09-16T07:07:27Z

Integrates the Playout Statistics API for WebAudio spec into the WebAudio spec.

The following changes are made between the version incubated in WICG and this PR:

"fallback"/"fallbackFrames" terminology is replaced with "underrun".
Duration types are changed from DOMHighResTimestamp to double.
Time units are changed from milliseconds to seconds.

Related issue: #2642

Preview | Diff

hoch

Generally looks good with some nits.

Also could you wrap the text at the column 80?

index.bs

@hoch

From @hoch Co-authored-by: Hongchan Choi <hongchan.choi@gmail.com>

Hernqvist · 2025-09-18T09:13:10Z

Implemented your comments and wrapped the lines at 80 (except where it would have cut through a link, which seems to be the standard for the rest of the document), thanks!

hoch

LGTM. Thanks!

hoch · 2025-09-18T15:58:39Z

@padenot Could you take a look?

index.bs

ulfhammarqvist-st · 2025-09-22T11:27:42Z

index.bs

+1. Set {{[[underrun duration]]}} to the total duration of all
+[=underrun frames=] (in seconds) that
+{{[[audio context]]}} has played since its construction.
+1. Set {{[[underrun events]]}} to the number of times that {{[[audio context]]}}


I'm curious if this design holds up to distinguish these three cases

occasional spike

consistent overload -> consistent underruns (every quantum processed)

periodic overload. as an example, a misaligned block based computation that ends up processing every N frames

Hi!

An occasional spike will manifest as a single increase in underrunDuration, and an increase in underrunEvents by 1.

Consistent underruns will manifest as many small increases in underrunDuration, and many increases in underrunEvents.

Similar to 1, but periodic. Since the API updates at most once per second (for privacy reasons), this might not be possible to immediately detect if N is small enough that we get several underruns per second. If N is large, it should be possible to see that the underruns occur at regular intervals. Also (if my math is correct) underrunEvents / currentTime should converge steadily towards sampleRate / N, so you could also look for that.

Co-authored-by: Christoph Guttandin <chrisguttandin@media-codings.com>

padenot

Some comments.

padenot · 2025-09-30T11:59:54Z

index.bs

+Audio underruns (also commonly called glitches) are gaps in the audio
+playout which occur when the audio pipeline cannot deliver audio on time.
+Underruns (often manifesting as audible "clicks" in the playout) are bad
+for the user experience, so if any of these occur it
+can be useful for the application to be able to detect them and possibly
+take some action to improve the playout.


Suggested change

Audio underruns (also commonly called glitches) are gaps in the audio

playout which occur when the audio pipeline cannot deliver audio on time.

Underruns (often manifesting as audible "clicks" in the playout) are bad

for the user experience, so if any of these occur it

can be useful for the application to be able to detect them and possibly

take some action to improve the playout.

Audio underruns (also commonly called glitches) are gaps in the audio

playback that occur when an audio pipeline cannot deliver audio on time.

Underruns (often manifesting as audible "clicks" in the playout) are bad

for the user experience, so if any of these occur it

can be useful for the application to be able to detect them and possibly

take some action to improve the playout.

we probably need to rephrase this. When the audio isn't delievered on time, it causes an audio underrun, that causes a discontinuity, that causes and audible click, that can be called a glitch. This is because of the synchronous nature of the AudioContext, unlike for example a PeerConnection, that will do something when audio input is missing to cover it up.

index.bs

padenot · 2025-09-30T12:05:10Z

index.bs

+{{AudioContext's}} playout path via
+{{AudioDestinationNode}} and the associated output device. This allows


Suggested change

{{AudioContext's}} playout path via

{{AudioDestinationNode}} and the associated output device. This allows

{{AudioContext's}} playout path via the

{{AudioDestinationNode}} and its associated audio output device. This allows

padenot · 2025-09-30T12:07:44Z

index.bs

+applications to measure underruns occurring due to underperforming
+AudioWorklets as well as underruns and latency originating in the playout
+path between the {{AudioDestinationNode}} and the output device.


This is important, but unrelated to AudioWorklet. We want to point out two possible causes:

The audio graph itself is too expensive for the current setup -- latency is too low, computer is generally too slow, etc.

The audio graph itself would render fine, but an external factor causes issues: other audio program on the device, global system overload, overload because of thermal throttling, etc.

it is important to make it extra clear what is being discussed here (especially in relation to the other section).

padenot · 2025-09-30T12:45:23Z

index.bs

+    {{[[latency reset time]]}}.
+</div>
+
+<h4>Privacy considerations of glitch stats</h4>


Let's use the API name here, and not introduce another term.

padenot · 2025-09-30T12:49:03Z

index.bs

+covert channel between two cooperating sites.
+One site could transmit information by intentionally causing audio glitches
+(by causing very high CPU usage, for example) while the other site
+could detect these glitches.


Yes, but only if:

There is a linearization point somewhere on the system (typically the audio mixer, be it in the OS or in the browser)

The callbacks are effectively synchronous all the way from this linearization point, without a buffer in between that could flatten load spikes (that could be because of a different AudioContextLatencyCategory).

padenot · 2025-09-30T12:49:34Z

index.bs

+<h5 id="AudioPlayoutStats-mitigations">Mitigations</h5>
+To inhibit the usage of such a covert channel, the API implements these
+mitigations.
+- The values returned by the API should not be updated more than once per


why not MUST?

padenot · 2025-09-30T12:49:51Z

index.bs

+mitigations.
+- The values returned by the API should not be updated more than once per
+    second.
+- The API should be restricted to sites that fulfill at least one of the following


Also why not MUST? Is this normative at all?

padenot · 2025-09-30T12:52:10Z

index.bs

+if (playoutDelay > 0.2) {
+    // Do something to reduce latency, like suggesting alternative
+    // playout methods
+}


I think the current prose makes this example unnecessary, it's already quite clear.

Co-authored-by: Paul Adenot <paul@paul.cx>

Add AudioPlayoutStats interface to spec

66fa3b7

hoch requested changes Sep 17, 2025

View reviewed changes

index.bs Outdated Show resolved Hide resolved

index.bs Outdated Show resolved Hide resolved

index.bs Outdated Show resolved Hide resolved

index.bs Outdated Show resolved Hide resolved

index.bs Outdated Show resolved Hide resolved

index.bs Outdated Show resolved Hide resolved

Hernqvist and others added 2 commits September 18, 2025 10:39

Apply suggestions from code review

2aaed39

From @hoch Co-authored-by: Hongchan Choi <hongchan.choi@gmail.com>

Wrap text at column 80 and replace var with let in example code

32ca908

hoch approved these changes Sep 18, 2025

View reviewed changes

hoch requested a review from padenot September 18, 2025 15:58

chrisguttandin reviewed Sep 19, 2025

View reviewed changes

index.bs Outdated Show resolved Hide resolved

index.bs Outdated Show resolved Hide resolved

ulfhammarqvist-st reviewed Sep 22, 2025

View reviewed changes

Apply suggestions from code review

af18297

Co-authored-by: Christoph Guttandin <chrisguttandin@media-codings.com>

padenot requested changes Sep 30, 2025

View reviewed changes

Hernqvist and others added 3 commits September 30, 2025 16:40

Apply suggestions from code review

c39cef2

Co-authored-by: Paul Adenot <paul@paul.cx>

Apply suggestions from code review

73b7409

Co-authored-by: Paul Adenot <paul@paul.cx>

Work on addressing review comments

dd0b3a5

		{{AudioContext's}} playout path via
		{{AudioDestinationNode}} and the associated output device. This allows

Add AudioPlayoutStats interface #2645

Are you sure you want to change the base?

Add AudioPlayoutStats interface #2645

Conversation

Hernqvist commented Sep 16, 2025 • edited by pr-preview bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hoch left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Hernqvist commented Sep 18, 2025

Uh oh!

hoch left a comment

Choose a reason for hiding this comment

Uh oh!

hoch commented Sep 18, 2025

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

padenot left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Hernqvist commented Sep 16, 2025 •

edited by pr-preview bot

Loading