diff --git a/index.bs b/index.bs index 061124980..6c5db790d 100644 --- a/index.bs +++ b/index.bs @@ -663,6 +663,9 @@ The interfaces defined are: {{AudioNode}} which applies a non-linear waveshaping effect for distortion and other more subtle warming effects. +* An {{AudioPlaybackStats}} interface, which provides statistics about the audio + played from the {{AudioContext}}. + There are also several features that have been deprecated from the Web Audio API but not yet removed, pending implementation experience of their replacements: @@ -1488,6 +1491,7 @@ interface AudioContext : BaseAudioContext { [SecureContext] readonly attribute (DOMString or AudioSinkInfo) sinkId; attribute EventHandler onsinkchange; attribute EventHandler onerror; + [SameObject] readonly attribute AudioPlaybackStats playbackStats; AudioTimestamp getOutputTimestamp (); Promise resume (); Promise suspend (); @@ -1533,6 +1537,11 @@ and to allow it only when the {{AudioContext}}'s [=relevant global object=] has :: An ordered list to store pending {{Promise}}s created by {{AudioContext/resume()}}. It is initially empty. + + : [[playback stats]] + :: + A slot where an instance of {{AudioPlaybackStats}} can be stored. It is + initially null.

@@ -1769,6 +1778,22 @@ Attributes

the context is {{AudioContextState/running}}. * When the operating system reports an audio device malfunction. + : playbackStats + :: + An instance of {{AudioPlaybackStats}} for this {{AudioContext}}. + +
+ When accessing this attribute, run the + following steps: + + 1. If the {{[[playback stats]]}} slot is null, construct a new + {{AudioPlaybackStats}} object with [=this=] as the argument, and + store it in {{[[playback stats]]}}. + + 1. Return the value of the {{[[playback stats]]}} internal slot. +
+ +

@@ -11536,6 +11561,280 @@ context.audioWorklet.addModule('vumeter-processor.js').then(() => { }); +

+The {{AudioPlaybackStats}} Interface

+ +Provides audio underrun and latency statistics for audio played through the +{{AudioContext}}. + +When audio is not delivered to the playback device on time, this causes an +audio underrun. This causes a discontinuity in the played signal, which produces an +audible "click" which is commonly called a "glitch". These glitches are bad +for the user experience, so if any of these occur it +can be useful for the application to be able to detect them and possibly +take some action to improve the playback. + +{{AudioPlaybackStats}} is a dedicated object for audio statistics reporting; +it reports audio underrun and playback latency statistics for the +{{AudioContext's}} playback path via +{{AudioDestinationNode}} and the associated output device. This allows +applications to measure underruns underruns, which can occur due to the +following reasons: +- The audio graph is too complex for the system to generate audio on time, + causing underruns. +- There is some external problem causing issues. Examples of such problems are: + - Another program playing audio to the same playback device is malfunctioning. + - There is a global system CPU overload. + - The system is overloaded due to thermal throttling. + +Underruns are defined in terms of [=underrun frames=] and [=underrun events=]: +- An underrun frame is an audio frame played by the output device + that was not provided by the AudioContext. + This happens when the playback path fails to provide audio frames + to the output device on time, in which case it will still have to play something. + + NOTE: Underrun frames are typically silence. + + This typically only happens if the rendering graph is underperforming. + This includes underrun situations that happen for reasons unrelated to + WebAudio/{{AudioWorklet}}s. +- When an [=underrun frame=] is played after a non-underrun frame, we consider + this an underrun event. + That is, multiple consecutive [=underrun frames=] will count as a single + [=underrun event=]. + +
+[Exposed=Window, SecureContext]
+interface AudioPlaybackStats {
+    constructor (AudioContext context);
+    readonly attribute double underrunDuration;
+    readonly attribute unsigned long underrunEvents;
+    readonly attribute double totalDuration;
+    readonly attribute double averageLatency;
+    readonly attribute double minimumLatency;
+    readonly attribute double maximumLatency;
+    undefined resetLatency();
+    [Default] object toJSON();
+};
+
+ +{{AudioPlaybackStats}} has the following internal slots: + +
+ : [[audio context]] + :: + The {{AudioContext}} that this instance of {{AudioPlaybackStats}} is + associated with. + + : [[underrun duration]] + :: + The total duration in seconds of [=underrun frames=] that + {{[[audio context]]}} has played as of the last stat update, a double. + Initialized to 0. + + : [[underrun events]] + :: + The total number of [=underrun events=] that has occurred in playback by + {{[[audio context]]}} as of the last stat update, an int. Initialized + to 0. + + : [[total duration]] + :: + The total duration in seconds of all frames + (including [=underrun frames=]) that {{[[audio context]]}} has played + as of the last stat update, a double. Initialized to 0. + + : [[average latency]] + :: + The average audio output latency of the {{[[audio context]]}} over the currently tracked interval, a double. + + : [[minimum latency]] + :: + The minimum playback latency in seconds of frames played by + {{[[audio context]]}} over the currently tracked interval, a double. + Initialized to 0. + + : [[maximum latency]] + :: + The maximum playback latency in seconds of frames played by + {{[[audio context]]}} over the currently tracked interval, a double. + Initialized to 0. + + : [[latency reset time]] + :: + The time when the latency statistics were last reset, a + double. This is in the clock domain of {{BaseAudioContext/currentTime}}. +
+ +

+Constructors

+ +
+ : AudioPlaybackStats(context) + :: + Run the following steps: + 1. Set {{[[audio context]]}} to context. + 1. Set {{[[latency reset time]]}} to 0. + + 
+            context: The {{AudioContext}} this new {{AudioPlaybackStats}} will
+            be associated with.
+
+
+ +

+Attributes

+ +Note: These attributes update only once per second and under specific +conditions. See the update audio stats +algorithm and privacy mitigations +for details. + +
+ : underrunDuration + :: + Returns the duration of [=underrun frames=] played by the + {{AudioContext}}, in seconds. + NOTE: This metric can be used together with {{totalDuration}} to + calculate the percentage of played out media that was not provided by + the {{AudioContext}}. + + Returns the value of the {{[[underrun duration]]}} internal slot. + +
+ : underrunEvents + :: + Measures the number of [=underrun events=] that have occurred during + playback by the {{AudioContext}}. + + Returns the value of the {{[[underrun events]]}} internal slot. + +
+ : totalDuration + :: + Measures the total duration of all audio played by the {{AudioContext}}, + in seconds. + + Returns the value of the {{[[total duration]]}} internal slot. + +
+ : averageLatency + :: + The average playback latency, in seconds, for audio played since the + last call to {{resetLatency()}}, or since the creation of the + {{AudioContext}} if + {{resetLatency()}} has not been called. + + Returns the value of the {{[[average latency]]}} internal slot. + +
+ : minimumLatency + :: + The minimum playback latency, in seconds, for audio played since the + last call to {{resetLatency()}}, or since the creation of the + {{AudioContext}} if + {{resetLatency()}} has not been called. + + Returns the value of the {{[[minimum latency]]}} internal slot. + +
+ : maximumLatency + :: + The maximum playback latency, in seconds, for audio played since the + last call to {{resetLatency()}}, or since the creation of the + {{AudioContext}} if + {{resetLatency()}} has not been called. + + Returns the value of the {{[[maximum latency]]}} internal slot. + +

+Methods

+ +
+ : resetLatency() + :: + Sets the start of the interval that latency stats are tracked over to + the current time. + When {{resetLatency}} is called, run the following steps: + + 1. Set {{[[latency reset time]]}} to {{BaseAudioContext/currentTime}}. + 1. Let currentLatency be the playback latency of the last + frame played by {{[[audio context]]}}, or 0 if no frames have been + played out yet. + 1. Set {{[[average latency]]}} to currentLatency. + 1. Set {{[[minimum latency]]}} to currentLatency. + 1. Set {{[[maximum latency]]}} to currentLatency. + +

Updating the stats

+
+Once per second, execute the +update audio stats algorithm: +1. If {{[[audio context]]}} is not running, abort these steps. +1. Let canUpdate be false. +1. Let document be the current [=this=]'s + [=relevant global object=]'s [=associated Document=]. +If document is [=Document/fully active=] and document's + [=Document/visibility state=] is `"visible"`, set canUpdate to + true. +1. Let permission be the [=permission state=] for the permission + associated with [="microphone"=] access. +If permission is "granted", set canUpdate to true. +1. If canUpdate is false, abort these steps. +1. Set {{[[underrun duration]]}} to the total duration of all +[=underrun frames=] (in seconds) that +{{[[audio context]]}} has played since its construction. +1. Set {{[[underrun events]]}} to the number of times that {{[[audio context]]}} + has played an [=underrun frame=] after a non-underrun frame since its + construction. +1. Set {{[[total duration]]}} to the total duration of all frames (in seconds) + that {{[[audio context]]}} has played since its construction. +1. Set {{[[average latency]]}} to the average playback latency (in seconds) of + frames that {{[[audio context]]}} has played since + {{[[latency reset time]]}}. +1. Set {{[[minimum latency]]}} to the minimum playback latency (in seconds) of + frames that {{[[audio context]]}} has played since + {{[[latency reset time]]}}. +1. Set {{[[maximum latency]]}} to the maximum playback latency (in seconds) of + frames that {{[[audio context]]}} has played since + {{[[latency reset time]]}}. +
+ +

Privacy considerations of {{AudioPlayoutStats}}

+ +
Risk
+Audio underrun information could be used to form a cross-site +covert channel between two cooperating sites. +One site could transmit information by intentionally causing audio glitches +(by causing very high CPU usage, for example) while the other site +could detect these glitches. +
Mitigations
+To inhibit the usage of such a covert channel, the API implements these +mitigations. +- The values returned by the API MUST not be updated more than once per + second. +- The API MUST be restricted to sites that fulfill at least one of the following + criteria: + 1. The site has obtained + getUserMedia + permission. + + Note: The reasoning is that if a site has obtained + getUserMedia + permission, it can receive glitch information or communicate + efficiently through use of the microphone, making access to the + information provided by {{AudioPlaybackStats}} redundant. These options + include detecting glitches through gaps in the microphone signal, or + communicating using human-inaudible sine waves. If microphone access is + ever made safer in this regard, this condition should be reconsidered. + 1. The document is [=Document/fully active=] and its + [=Document/visibility state=] is `"visible"`. + + Note: Assuming that neither cooperating site has microphone permission, + this criterion ensures that the site that receives the covert signal + must be visible, restricting the conditions under which the covert + channel can be used. It makes it impossible for sites to communicate + with each other using the covert channel while not visible. +

Processing model