Conditional Focus (When Display-Capture Starts)

[eladalon1983]

Braw mornin' TAG!

I'm requesting a TAG review of Conditional Focus.

When an application starts capturing a display-surface, the user agent faces a decision - should the captured display-surface be brought to the forefront, or should the capturing application retain focus.

The user agent is mostly agnostic of the nature of the capturing and captured applications, and is therefore ill-positioned to make an informed decision.

In contrast, the capturing application is familiar with its own properties, and is therefore better suited to make this decision. Moreover, by reading displaySurface and/or using Capture Handle, the capturing application can learn about the captured display-surface, driving an even more informed decision.

• Explainer: https://github.com/WICG/conditional-focus/blob/main/README.md
• Specification URL: https://wicg.github.io/conditional-focus/
• Tests: N/A
• Security and Privacy self-review: TBD
• Primary contacts:
  • Elad Alon (@eladalon1983), Google
• Organization(s)/project(s) driving the specification: Google
• Key pieces of existing multi-stakeholder review or discussion of this specification:
  • Conditional Focus (When Display-Capture Starts) w3c/mediacapture-screen-share#190
  • Conditional Focus (When Display-Capture Starts) WICG/proposals#37
• External status/issue trackers for this specification (publicly visible, e.g. Chrome Status): https://www.chromestatus.com/feature/5646614535340032

Further details:

• I have reviewed the TAG's Web Platform Design Principles
• Relevant time constraints or deadlines: N/A
• The group where the work on this specification is currently being done: WebRTC WG
• The group where standardization of this work is intended to be done (if current group is a community group or other incubation venue): WebRTC WG and/or WICG
• Major unresolved issues with or opposition to this specification: N/A
• This work is being funded by: Google

You should also know that...

We'd prefer the TAG provide feedback as (please delete all but the desired option):

💬 leave review feedback as a comment in this issue and @-notify @eladalon1983

[torgo]

Hi @eladalon1983 - Looking at the thread in WebRTC, it looks like this is still under active discussion. If so, it seems like it might be premature for the TAG to be reviewing this? Is the plan to incubate this in WICG and then bring it to WebRTC for a future WebRTC rechartering?

[rhiaro]

Hello, @torgo and I are looking at this in our virtual face-to-face today. We noticed the problem statement in the explainer is from the perspective of the user agent, rather than the end user. Would you mind adding some examples/use cases/scenarios of how the end user of an application benefits or is affected by this feature? We appreciate that you have noted one security concern in the explainer; do you have an idea of when you will be able to complete the privacy and security questionnaire? Are you expecting to take this work to the WebRTC WG?

[eladalon1983]

Hi @eladalon1983 - Looking at the thread in WebRTC, it looks like this is still under active discussion. If so, it seems like it might be premature for the TAG to be reviewing this? Is the plan to incubate this in WICG and then bring it to WebRTC for a future WebRTC rechartering?

It's being incubated in the WICG at the same time as it's presented to the WebRTC WG. (It's also in origin trial in Chrome starting m96.) We've had some fruitful discussions in the WebRTC WG and I am hoping we'll reach a mutually agreed-upon API shape in January or possibly earlier.

Hello, @torgo and I are looking at this in our virtual face-to-face today. We noticed the problem statement in the explainer is from the perspective of the user agent, rather than the end user. Would you mind adding some examples/use cases/scenarios of how the end user of an application benefits or is affected by this feature?

I can rephrase this with the next revision, no problem. Could you share more of your thoughts about the phrasing? For example, I took a look at the Web Cryptography API and it looked to me to also be centered around the user agent and the applications, not the user. The way I see it, in both cases, the benefit to users is applications can serve them better. In the case of Conditional Focus, this better service is through less distractions thrown the user's way, and less clicks required of the user, when they are engaged in something as mentally-challenging as presenting to a live audience. Would adding an explicit shout-out of this fact at the end of the Problem Description section be sufficient?

We appreciate that you have noted one security concern in the explainer; do you have an idea of when you will be able to complete the privacy and security questionnaire? Are you expecting to take this work to the WebRTC WG?

Presently, I expect to do that after the final shape is agreed upon in WebRTC WG discussions.

[rhiaro]

this better service is through less distractions thrown the user's way, and less clicks required of the user, when they are engaged in something as mentally-challenging as presenting to a live audience

Thanks, that's exactly the kind of language we like to see in an explainer. It can also be helpful to walk us through how someone interacts with the feature from first principles, assuming little prior familiarity with the details of the work. (If you haven't already seen it, there are some links to good explainers in the explainer explainer.)

[torgo]

Hi @eladalon1983 - can you let us know status on the API shape (and security & privacy questionnaire response)? Also have you thought any more about documenting the user need - rephrasing and expanding the use case from an end-user perspective - as @rhiaro mentioned above?

[eladalon1983]

(Apologies for still not responding, but I am prioritizing other work. Please leave this open. I intend to get back to this in the next few weeks.)

[torgo]

Hi @eladalon1983 just pinging on this. Has there been any update we should be aware of?

[eladalon1983]

Sorry, still in the pipeline.

[torgo]

Hi @eladalon1983 we're going to close this for now just to keep our agendas in order. Can you ping us when you're ready for us to re-open and review?

[eladalon1983]

Will do.

[eladalon1983]

Can you ping us when you're ready for us to re-open and review?

Following discussions in the WebRTC WG, the API was reshaped. The new shape is visible in these PRs:

• Add CaptureController to the spec w3c/mediacapture-screen-share#235
• Define CaptureController.setFocusBehavior() w3c/mediacapture-screen-share#240

I'll be updating the explainer soon.
Please take a look.