Exclusive access to audio hardware

[olivierthereaux]

Originally reported on W3C Bugzilla ISSUE-17320 Tue, 05 Jun 2012 11:08:32 GMT
Reported by Jer Noble
Assigned to

http://www.w3.org/2011/audio/track/issues/15

Raised by: Jer Noble
On product: Web Audio API

On the mailing list, Jer Noble proposed an addition to the WebAudio API which would allow the API to be implemented on platforms which allow exclusive access to audio hardware.

http://lists.w3.org/Archives/Public/public-audio/2012AprJun/0122.html

The proposed changes:

• Add the AudioContext.startRendering() method to the spec.
• Add a new AudioContext.renderState property to the spec.
  • Values:
    • RENDERING
    • INTERRUPTED
    • IDLE

Chris Rogers questioned why the startRendering() method was necessary:

http://lists.w3.org/Archives/Public/public-audio/2012AprJun/0124.html

Jer Noble replied with a use case demonstrating its use:

http://lists.w3.org/Archives/Public/public-audio/2012AprJun/0126.html

[olivierthereaux]

Original comment by Chris Rogers on W3C Bugzilla. Fri, 05 Oct 2012 21:35:42 GMT

Jer, is this something you still believe is important to consider? I know iOS has been shipping for a little while and didn't know if this was still an issue.

[olivierthereaux]

Original comment by Jer Noble on W3C Bugzilla. Fri, 05 Oct 2012 21:36:44 GMT

Yes I believe it is still an issue. It's unfortunate that we weren't able to resolve this prior to iOS 6, but I believe this is still important to resolve for future implementations.

[jernoble]

Updating my proposal:

Interface Changes

AudioContext

Add the concept of pausing and resuming to AudioContext.

enum AudioContextPlayState {
    "playing",
    "paused",
    "interrupted"
}

partial interface AudioContext {
    readonly attribute AudioContextPlayState playState;
    attribute bool resumable;

    void pause();
    void resume();
}

Algorithms

An AudioContext would start in the "playing" state. An audio hardware interruption would move the state from "playing" to "interrupted". When interruption ends, if resumable is set to tue, the state would move from "interrupted" to "playing". Otherwise, it would move from "interrupted" to "paused".

When the state is "playing", pause() will move the state from "playing" to "paused". When the state is "paused", calling play() will move the state from "paused" to "playing". If the audio hardware is unavailable, the state would move from "playing" to "interrupted".

When the state is "interrupted", both play() and pause() are no-ops.

When the state is "interrupted" or "paused", the context's currentTime does not advance, and no audio is generated from any node.

Attributes

playState

• A read-only attribute listing the current play state of the AudioContext. Valid values are "playing", "paused", and "interrupted".

resumable

• Defaults to true. Controls whether audio resumes automatically after a hardware interruption ends.

Methods

pause

• Changes the playState from "playing" to "paused".

resume

• Changes the playState from "paused" to "playing".

[joeberkovitz]

We'll come back to this later with a more complete proposal. Next version.

[mdjp]

This is no longer an issue now that we have implemented suspend and resume.

[jernoble]

Re-opening this issue. suspend() and resume() are indeed important parts of this proposal, but the "interrupted" state is important as well. During a phone call, (or other period of exclusive use of the audio hardware) requests to resume() the AudioContext will always fail. An "interrupted" state signals to the web page that, as opposed to merely "suspended", all requests to resume() will reject.

Additionally, "interrupted" is a state provided by the UA, rather than script running in the page. A UA could move the state from "running" to "interrupted" to "running" again (if the interruption allows normal audio playback to resume afterwards), or from "running" to "interrupted" to "suspended" (if audio playback is not allowed to resume automatically). In the first case, the page is free to react to the "interrupted" state by pausing (e.g.) the video game, and un-pause when the state moves to "running" after the interruption ends. In the second case, when the state moves to "suspended", the page can provide UI to restart playback (and the game) after the interruption. In either case, the page can choose to continue silently during the interruption.

"closed" is not a substitute here, as there is no mechanism for moving from "closed" to "running". "suspended" is not a substitute because there's no indication to the page why the state moved to "suspended" (did the user pause audio through UA provided hardware controls?) nor is there an indication when playback would be allowed again. The promise returned by "resume" is not a substitute, because if not immediately rejected, it would resolve at the exact same time as the UA would automatically move the state from "interrupted" to "running" in the first scenario, and would reject in the second without indicating to the page that, if they just tried again, things would work fine.

[padenot]

AudioWG call:

• This make sense and is a clean addition to the already existing set of state. The transition from and to this new state is also clear
• Some implementation already do something like this (but using "suspend"), but this proposal is better, because the meaning that is conveyed is clearer for authors, and there is a need to distinguish between a suspended context and an interrupted context

[rtoy]

@jernoble Can you clarify the state transitions here?

Assuming we're running, the only transitions are running -> interrupted -> running if allowed. And running -> interrupted -> suspended if the interruption is over, but we're not allowed to resume playing?

Calling suspend or resume in the interrupted state causes these promises to be rejected?

So once we're in the interrupted state, there's no way for the user to get out of that except wait for the OS to finish the interruption?

Presumably, we can still call close to close the context in the interrupted state?

What happens if we're suspended? Can we transition to interrupted by the OS? Seems like we might want to do that to match what running->interrupted does. I guess that means when the interruption ends, we go back to suspended instead of resuming?

Or maybe we don't go to interrupted state. When the user resumes(), we immediately go to interrupted if the OS says we were interrupted? Then the rest of the behavior is defined by the running->interrupted transitions above.

[rtoy]

Tel;econf: We'll leave this issue as is for discussing exclusive access to audio HW. The discussion about an interrupted state will be filed as a new issue in v2.

[rtoy]

Per #72 (comment), I'm closing this again.

Further discussions about the interrupted state will be handled in issue WebAudio/web-audio-api-v2#115

If at some later date we want to support exclusive access to HW, reopen this issue or, better yet, file a new issue for the V2 spec.