-
Notifications
You must be signed in to change notification settings - Fork 28
/
index.html
1639 lines (1635 loc) · 75.9 KB
/
index.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<!DOCTYPE html>
<html lang="en-us">
<head>
<link href="screenshare.css" rel="stylesheet" type="text/css">
<title>
Screen Capture
</title>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
<script class="remove" src="screenshare.js" type="text/javascript"></script>
<script class="remove" src="https://www.w3.org/Tools/respec/respec-w3c"></script>
</head>
<body>
<section id="abstract">
<p>
This document defines how a user's display, or parts thereof, can be used as the source of
a media stream using {{MediaDevices/getDisplayMedia}}, an extension to the Media Capture API
[[GETUSERMEDIA]].
</p>
</section>
<section id="sotd">
<p>
This document is not complete. It is subject to major changes and, while early
experimentations are encouraged, it is therefore not intended for implementation.
</p>
</section>
<section class="informative" id="intro">
<h2>
Introduction
</h2>
<p>
This document describes an extension to the Media Capture API [[GETUSERMEDIA]] that enables
the acquisition of a user's display, or part thereof, in the form of a video track. In some
cases system, application or window audio is also captured which is presented in the form
of an audio track. This enables a number of applications, including screen sharing using
WebRTC [[WEBRTC]].
</p>
<p>
This feature has signficant security implications. Applications that use this API to access
information that is displayed to users could access confidential information from other
origins if that information is under the control of the application. This includes content
that would otherwise be inaccessible due to the protections offered by the user agent
sandbox.
</p>
<p>
This document concerns itself primarily with the capture of video and audio [[GETUSERMEDIA]],
but the general mechanisms defined here could be extended to other types of media, of which
depth [[MEDIACAPTURE-DEPTH]] is currently defined.
</p>
</section>
<section id="conformance">
<p>
This specification defines conformance criteria that apply to a single product: the
<dfn data-lt="user agents">user agent</dfn> that implements the interfaces that it
contains.
</p>
<p>
Implementations that use ECMAScript [[ECMA-262]] to implement the APIs defined in this
specification must implement them in a manner consistent with the ECMAScript Bindings
defined in the Web IDL specification [[!WEBIDL]], as this specification uses that
specification and terminology.
</p>
</section>
<section>
<h2>
Example
</h2>
<p>
The following example demonstrates a request for display capture using the
<code>navigator.mediaDevices.getDisplayMedia</code> method defined in this document.
</p>
<pre class="example highlight">try {
let mediaStream = await navigator.mediaDevices.getDisplayMedia({video:true});
videoElement.srcObject = mediaStream;
} catch (e) {
console.log('Unable to acquire screen capture: ' + e);
}</pre>
</section>
<section>
<h2>
Terminology
</h2>
<p>
This document uses the definition of {{MediaStream}}, {{MediaStreamTrack}},
{{MediaStreamConstraints}} and {{ConstrainablePattern}}
from [[!GETUSERMEDIA]].
</p>
<p>
Screen capture encompasses the capture of several different types of screen-based surfaces.
Collectively, these are referred to as <dfn data-lt="display surface">display
surfaces</dfn>, of which this document defines the following types:
</p>
<ul>
<li>A <dfn>monitor</dfn> <a>display surface</a> represents a physical display. Some systems
have multiple <a>monitor</a>s, which can be identified separately. Multiple <a>monitor</a>s
might also be aggregated into a single logical <a>monitor</a>. An aggregated <a>display
surface</a> is captured as a single {{MediaStreamTrack}}.
</li>
<li>A <dfn data-lt="windows">window</dfn> <a>display surface</a> is a single contiguous
surface that is used by a single application.
</li>
<li>A <dfn>browser</dfn> <a>display surface</a> is the rendered form of a
[=top-level browsing context=]'s <a data-cite="HTML#viewport">viewport</a>.
This is not strictly limited to HTML [[HTML]] documents, though the discussion in this
document will address some specific concerns with the capture of HTML.
</li>
</ul>
<p>
This document draws a distinction between two variants of each type of display surface:
</p>
<ul>
<li>A <dfn data-lt="logical display surfaces">logical display surface</dfn> is the surface
that an operating system makes available to an application for the purposes of rendering.
</li>
<li>a <dfn>visible display surface</dfn> is the portion of a [=logical display surface=]
that is rendered to a [=monitor=].
</li>
</ul>
<p>
Some operating systems permit windows from different applications to occlude other windows,
in whole or part, so the <a>visible display surface</a> is a strict subset of the
<a>logical display surface</a>.
</p>
<p>
The <dfn>source pixel ratio</dfn> of a <a>display surface</a> is 1/96th of 1 inch divided
by its vertical pixel size.
</p>
<p>The <dfn>devicechange</dfn> event is defined in [[GETUSERMEDIA]].</p>
</section>
<section>
<h2>
Capturing Displayed Media
</h2><!--
Interesting Links
https://bugzilla.mozilla.org/show_bug.cgi?id=742832
http://www.chromium.org/developers/design-documents/extensions/proposed-changes/apis-under-development/webrtc-tab-content-capture
https://docs.google.com/document/d/1-vFghorm8zDCeyg2Yk6kKFT-16GU1Ow1b1bor_jCqD8/edit
-->
<p>
Capture of displayed media is enabled through the addition of a new
{{MediaDevices/getDisplayMedia}} method on the {{MediaDevices}}
interface, that is similar to {{MediaDevices/getUserMedia()}}
, except that it acquires media from one display device
chosen by the end-user each time.
</p>
<section>
<h2>
<dfn>MediaDevices</dfn> Additions
</h2>
<pre class="idl"
>partial interface MediaDevices {
Promise<MediaStream> getDisplayMedia(optional DisplayMediaStreamOptions options = {});
};</pre>
<dl data-link-for="MediaDevices" data-dfn-for="MediaDevices" class="methods">
<dt>
<dfn>getDisplayMedia</dfn>
</dt>
<dd>
<p>Prompts the user for permission to live-capture their display.</p>
<p>
The user agent MUST let the end-user choose which [=display surface=] to share
out of all available choices every time, and MUST NOT use
any {{MediaTrackConstraints}} in <var>options</var>.<code>video</code>
or <var>options</var>.<code>audio</code> to limit that choice.
</p>
<p>
The user agent MAY use the presence of the {{displaySurface}} constraint and its value to
influence the presentation to the user of the sources to pick from. The user agent MUST still
offer the user unlimited choice of any [=display surface=].
The user agent is strongly recommended to steer users away from sharing a monitor,
as this poses
<a href="#security-and-permissions">risks to user privacy</a>.
</p>
<p>
Any {{MediaTrackConstraints}} in <var>options</var>.<code>video</code>
or <var>options</var>.<code>audio</code> MUST be applied
to the media chosen by the user only after the user has made their selection.
</p>
<p>
In the case of audio, the user agent MAY present the end-user with audio sources
to share. Which choices are available to choose from is up to the user agent, and
the audio source(s) are not necessarily the same as the video source(s). An audio
source may be a particular <a>window</a>, <a>browser</a>, the entire system audio
or any combination thereof.
Unlike {{MediaDevices/getUserMedia()}} with regards to audio+video, the user agent
is allowed not to return audio even if the audio constraint is present. If the user
agent knows no audio will be shared for the lifetime of the stream it MUST NOT
include an audio track in the resulting stream. The user agent MAY accept a
request for audio and video by only returning a video track in the resulting
stream, or it MAY accept the request by returning both an audio track and a video
track in the resulting stream. The user agent MUST reject audio-only requests.
</p>
<p>
In addition to drawing from a different set of sources and requiring user selection,
{{MediaDevices/getDisplayMedia}} also differs from
{{MediaDevices/getUserMedia()}} in that {{PermissionState/"granted"}} permissions cannot be persisted.
</p>
<p>When the {{MediaDevices/getDisplayMedia()}}
method is called, the user agent MUST run the following
steps:</p>
<ol>
<li>
<p>If the [=relevant global object=] of [=this=] does not have
[=transient activation=], return a promise <a>rejected</a>
with a {{DOMException}} object whose {{DOMException/name}}
attribute has the value {{InvalidStateError}}.</p>
</li>
<li>
<p>Let <var>options</var> be the method's first
argument.</p>
</li>
<li>
<p>
Let <var>controller</var> be <var>options</var>.<code>controller</code> if present,
or <code>null</code> otherwise.
</p>
</li>
<li>
<p>
If controller is not <code>null</code>, run the following steps:
</p>
<ol>
<li>
If <var>controller</var>.{{CaptureController/[[isBound]]}}
is <code>true</code>, return a promise [=reject|rejected=] with
a {{DOMException}} object whose {{DOMException/name}} attribute has
the value {{InvalidStateError}}.
</li>
<li>
Set <var>controller</var>.{{CaptureController/[[isBound]]}} to <code>true</code>.
</li>
</ol>
</li>
<li>
<p>Let <var>constraints</var> be
<code>[</code><var>options</var>.<code>audio</code>,
<var>options</var>.<code>video</code><code>]</code>.</p>
</li>
<li>
<p>If <code>constraints.video</code> is <code>false</code>,
return a promise [=reject|rejected=] with a newly
[=exception/created=] {{TypeError}}.</p>
</li>
<li>
<p>For each [= map/exist | existing =] member
in <var>constraints</var> whose value, <var>CS</var>, is
a dictionary, run the following steps:</p>
<ol>
<li>
<p>If <var>CS</var> contains a member named <code>advanced</code>,
return a promise [=reject|rejected=] with a newly
[=exception/created=] {{TypeError}}.</p>
</li>
<li>
<p>If <var>CS</var> contains a member whose name specifies a
constrainable property applicable to [=display surface=]s,
and whose value in turn is a dictionary containing a member
named either <code>min</code> or <code>exact</code>, return
a promise [=reject|rejected=] with a newly
[=exception/created=] {{TypeError}}.</p>
</li>
<li>
<p>If <var>CS</var> contains a member whose name specifies a
constrainable property applicable to [=display surface=]s,
and whose value in turn is a dictionary containing a member
named <code>max</code>, and that member's value in turn is
less than the constrainable property's [=floor value=], then let
<var>failedConstraint</var> be the name of the
member, let <var>message</var> be either
<code>undefined</code> or an informative human-readable
message, and return a promise [=reject|rejected=] with a new
<code>OverconstrainedError</code> created by calling
<code>OverconstrainedError(<var>failedConstraint</var>,
<var>message</var>)</code>.
</li>
</ol>
</li>
<li>
<p>Let <var>requestedMediaTypes</var> be the set of media
types in <var>constraints</var> with either a dictionary
value or a value of <code>true</code>.</p>
</li>
<li>
<p>If the <a>current settings object</a>'s [=relevant global object=]'s
[=associated `Document`=] is NOT [=Document/fully active=] or does NOT
<a data-cite="!HTML/#gains-focus">have focus</a>, return a promise
[=reject|rejected=] with a {{DOMException}} object whose {{DOMException/name}}
attribute has the value {{InvalidStateError}}.</p>
</li>
<li>
<p>Let <var>p</var> be a new promise.</p>
</li>
<li>
<p>Run the following steps in parallel:</p>
<ol>
<li>
<p>For each media type <var>T</var> in
<var>requestedMediaTypes</var>,</p>
<ol>
<li>
<p>If no sources of type <var>T</var> are available,
<a>reject</a> <var>p</var> with a new
{{DOMException}} object whose
{{DOMException/name}} attribute has the value
{{NotFoundError}}.</p>
</li>
<li>
<p>Read the current [= permission state=] for obtaining
sources of type <var>T</var> in the current browsing
context. If the permission state is {{PermissionState/"denied"}}, jump to
the step labeled <em>PermissionFailure</em> below.</p>
</li>
</ol>
</li>
<li>
<p>Optionally, e.g., based on a previously-established
user preference, for security reasons, or due to platform
limitations, jump to the step labeled <em>Permission Failure</em> below.</p>
</li>
<li>
<p>[=Prompt the user to choose=]
a display device, for a {{PermissionDescriptor}} with its
{{PermissionDescriptor/name}} set to "display-capture", resulting in a set of
provided media.</p>
<p>The provided media MUST include precisely one video track.</p>
<p>The provided media MUST include at most one audio track. This audio
track MUST NOT be included if audio was not specified in
<var>requestedMediaTypes</var>, or if it was specified as
<code>false</code>.</p>
<p>The devices chosen MUST be the ones determined by the user.
Once selected, the source of a {{MediaStreamTrack}} MUST NOT change,
unless the user permits it through their interaction with
the user agent.</p>
<p>User agents are encouraged to warn users against sharing
<a>browser</a> display devices as well as <a>monitor</a>
display devices where browser windows are visible, or
otherwise try to discourage their selection on the basis
that these represent a significantly higher risk when shared.</p>
<p>If the result of the request is {{PermissionState/"granted"}}, then for
each device that is sourcing the provided media, using
a stable and private id for the device, <var>deviceId</var>,
set [[\devicesLiveMap]]<var>[deviceId]</var> to
<code>true</code>, if it isn’t already <code>true</code>,
and set the
[[\devicesAccessibleMap]]<var>[deviceId]</var> to
<code>true</code>, if it isn’t already
<code>true</code>.</p>
<p>The user agent MUST NOT
store a {{PermissionState/"granted"}} permission entry.
</p>
<p>If the result is {{PermissionState/"denied"}}, jump to the step labeled
<em>Permission Failure</em> below. If the user never
responds, this algorithm stalls on this step.</p>
<p>If the user grants permission but a hardware error
such as an OS/program/webpage lock prevents access,
<a>reject</a> <var>p</var> with a new
{{DOMException}} object whose
{{DOMException/name}} attribute has the value
{{NotReadableError}} and abort these steps.</p>
<p>If the result is {{PermissionState/"granted"}} but device access fails for
any reason other than those listed above, <a>reject</a>
<var>p</var> with a new {{DOMException}}
object whose {{DOMException/name}} attribute has the
value {{AbortError}} and abort these steps.</p>
</li>
<li>
<p>Let <var>stream</var> be the
{{MediaStream}} object for which the user
granted permission.</p>
</li>
<li>
<p>Run the [=ApplyConstraints algorithm=] on all
tracks in <var>stream</var> with the appropriate
constraints. Should this fail, let <var>failedConstraint</var>
be the result of the algorithm that failed, and let
<var>message</var> be either <code>undefined</code> or an
informative human-readable message, and then <a>reject</a>
<var>p</var> with a new <code>OverconstrainedError</code>
created by calling
<code>OverconstrainedError(<var>failedConstraint</var>,
<var>message</var>)</code>.</p>
</li>
<li>
<p><a>Resolve</a> <var>p</var> with <var>stream</var> and
abort these steps. This invocation of {{MediaDevices/getDisplayMedia()}}
is now considered to have produced a new <dfn>capture-session</dfn></p>
</li>
<li>
<p><em>Permission Failure</em>: [=Reject=]
<var>p</var> with a new {{DOMException}}
object whose {{DOMException/name}} attribute has the
value {{NotAllowedError}}.</p>
</li>
</ol>
</li>
<li>
<p>Return <var>p</var>.</p>
</li>
</ol>
<p>
The <a>user agent</a> MUST NOT capture content that's behind a
partially transparent captured <a>display surface</a>.
</p>
<p>
For the newly created {{MediaStreamTrack}}, the <a>user agent</a> MUST NOT capture the
prompt that was shown to the user.
</p>
<p>
Information that is not currently rendered to the screen SHOULD be obscured in captures
unless the application has been specifically authorized to access that content (e.g.
through means such as <a>elevated permissions</a>).
</p>
<p>
The <a>user agent</a> MUST NOT share audio without <a>active user consent</a>, for example
if the capture of the video of a <a>window</a> is accompanied by capture of the audio of the
entire system, including applications unrelated to that window.
</p>
</dd>
</dl>
</section>
<section>
<h2 id="hidden-display-surfaces">
Closed and Minimized Display Surfaces
</h2>
<p>
A <a>display surface</a> that is being shared may temporarily or permanently become
inaccessible to the application because of actions taken by the operating system or user
agent. What makes a <a>display surface</a> considered inaccesible is outside the scope
of this specification, but examples MAY include a <a>monitor</a> disconnecting, a
<a>window</a> or <a>browser</a> closing or becoming minimized,
or due to an incoming call on a phone.
</p>
<p class="note">
User agents ultimately control what inaccesible means in this context, but are encouraged
to only fire mute and unmute events for interruptions that have external reasons.
</p>
<p>
When <a>display surface</a> enters an inaccessible state that is not necessarily
permanent, the user agent MUST queue a task that
[= set a track's muted state|sets the muted state =]
of the corresponding media track to <code>true</code>.
</p>
<p>
When <a>display surface</a> exits an inaccessible state and becomes accessible, the
user agent MUST queue a task that
[= set a track's muted state|sets the muted state =]
of the corresponding media track to <code>false</code>.
</p>
<p>
When a <a>display surface</a> enters an inaccessible state that is permanent (such as
the source <a>window</a> closing), the user agent MUST queue a task that
[= track/ended | ends =] the corresponding media track.
</p>
<p>
A stream that was just returned by {{MediaDevices/getDisplayMedia}} MAY contain
tracks that are muted by default. Audio and video tracks belonging to the same stream
MAY be muted/unmuted independently of one another.
</p>
</section>
<section>
<h2 id="constraints">
Unconstrained Display Surface Selection
</h2>
<p class="fingerprint">
Not accepting constraints for source selection means that
{{MediaDevices/getDisplayMedia}} only provides fingerprinting surface that exposes
whether audio, video or audio and video display sources are present. <img alt=
"(This is a fingerprinting vector.)" src="images/fingerprint.png" width="15" height="21">
</p>
<p>
Note that accepting the {{displaySurface}} constraint does not limit user selection.
</p>
</section>
<section>
<h2 id="constrainable-properties">
Constrainable Properties for Captured Display Surfaces
</h2>
<p>Constraints serve a different purpose in
{{MediaDevices/getDisplayMedia}} than they do in
{{MediaDevices/getUserMedia()}}.
They do not aid discovery, instead they are applied only after user-selection.</p>
<p>This section define which constraints apply to {{MediaDevices/getDisplayMedia}} tracks;
constraints defined for
{{MediaDevices/getUserMedia()}}
do not apply unless listed here.</p>
<p>Some of these constraints enable user agent processing like downscaling
and frame decimation, as well as display-specific features. Others enable
observation of inherent properties of a user-selected
<a>display surface</a>, as capabilities and settings.</p>
<p>The following new and existing {{MediaStreamTrack}}
<a data-cite="GETUSERMEDIA#constrainable-properties">
Constrainable Properties</a> are defined to apply to the user-selected
video <a>display surface</a>, with the following behavior:</p>
<table class="simple">
<thead>
<tr>
<th>Property Name</th>
<th>Type</th>
<th>Behavior</th>
</tr>
</thead>
<tbody>
<tr id="def-constraint-width">
<td><dfn>width</dfn></td>
<td>{{ConstrainULong}}</td>
<td>The width or width range, in pixels. As a capability, max MUST
reflect the <a>display surface</a>'s width, and min MUST reflect
the width of the smallest aspect-preserving representation
available through downscaling by the user agent.</td>
</tr>
<tr id="def-constraint-height">
<td><dfn>height</dfn></td>
<td>{{ConstrainULong}}</td>
<td>The height or height range, in pixels. As a capability, max
MUST reflect the <a>display surface</a>'s height, and min MUST
reflect the height of the smallest aspect-preserving
representation available through downscaling by the user agent.
</td>
</tr>
<tr id="def-constraint-frameRate">
<td><dfn>frameRate</dfn></td>
<td>{{ConstrainDouble}}</td>
<td>The frame rate (frames per second) or frame rate range. As a
capability, max MUST reflect the <a>display surface</a>'s frame
rate, and min MUST reflect the lowest frame rate available through
frame decimation by the user agent.</td>
</tr>
<tr id="def-constraint-aspect">
<td><dfn>aspectRatio</dfn></td>
<td>{{ConstrainDouble}}</td>
<td>The exact aspect ratio (width in pixels divided by height in
pixels, represented as a double rounded to the tenth decimal
place) or aspect ratio range. As a setting, represents
<code>width / height</code>. As a capability, min
and max both MUST be the current setting value, rendering this
property immutable from the application viewpoint.</td>
</tr>
<tr id="def-constraint-resizeMode">
<td><dfn>resizeMode</dfn></td>
<td>{{ConstrainDOMString}}</td>
<td>
This string (or each string, when a list) should be one of the members of
{{VideoResizeModeEnum}}.
As a setting, {{VideoResizeModeEnum/"none"}} means the {{MediaStreamTrack}}
contains all bits needed to render the display in full detail,
which if the <code><a>source pixel ratio</a> > 1</code>, means
<code>width</code> and <code>height</code> will be larger
than the display's appearance from an end-user viewpoint would
suggest, whereas {{VideoResizeModeEnum/"crop-and-scale"}} means the
{{MediaStreamTrack}} contains an
aspect-preserved representation of the <a>display surface</a>
that has been downscaled by the user agent, but not cropped. As
a capability, the values {{VideoResizeModeEnum/"none"}} and
{{VideoResizeModeEnum/"crop-and-scale"}} both MUST be present.
</td>
</tr>
<tr id="def-constraint-displaySurface">
<td><dfn>displaySurface</dfn></td>
<td>{{ConstrainDOMString}}</td>
<td>
<p>
This string (or each string, when a list) should be one of the
members of {{DisplayCaptureSurfaceType}}.
</p>
<p>
As a setting, indicates the type of [=display surface=] that
is being captured.
</p>
<p>
As a capability, the setting value MUST be
the lone value present, rendering this property immutable from
the application viewpoint.
</p>
<p>
As a constraint, the value signals the application's preference
of a particular [=display surface=] type to the user agent;
the user agent MAY reorder the options offered to the user
according to that preference. This constraint is ignored for all
other purposes, and can therefore not cause any side effects
(such as being the cause of <code>OverconstrainedError</code>).
</p>
</td>
</tr>
<tr id="def-constraint-logicalSurface">
<td><dfn>logicalSurface</dfn></td>
<td>{{ConstrainBoolean}}</td>
<td>
As a setting, a value of <code>true</code> indicates capture of
a [=logical display surface=], whereas a value of
<code>false</code> indicates a capture of a
[=visible display surface=]. As a capability, this same
value MUST be the lone value present, rendering this
property immutable from the application viewpoint.
</td>
</tr>
<tr id="def-constraint-cursor">
<td><dfn>cursor</dfn></td>
<td>{{ConstrainDOMString}}</td>
<td>
This string (or each string, when a list) should be one of the
members of {{CursorCaptureConstraint}}. As a
setting, indicates if and when the cursor is included in the
captured [=display surface=]. As a capability, the user
agent MUST include only the set of values from
{{CursorCaptureConstraint}} it is capable of
supporting for this [=display surface=].
</td>
</tr>
</tbody>
</table>
<p>The following new and existing {{MediaStreamTrack}}
<a data-cite="GETUSERMEDIA#constrainable-properties">
Constrainable Properties</a> are defined to apply to the user-selected
audio sources, with the following behavior:</p>
<table class="simple">
<thead>
<tr>
<th>Property Name</th>
<th>Type</th>
<th>Behavior</th>
</tr>
</thead>
<tbody>
<tr id="def-constraint-restrictOwnAudio">
<td><dfn>restrictOwnAudio</dfn></td>
<td>{{ConstrainBoolean}}</td>
<td>
<p>As a setting, this value indicates whether or not the user
agent is applying <a>own audio restriction</a> to the
source.</p>
<p>As a constraint, this property can be constrained resulting
in a source with <a>own audio restriction</a> enabled or
disabled.</p>
<p>When <dfn>own audio restriction</dfn> is applied, the user
agent MUST attempt to remove any audio from the audio being
captured that was produced by the document that performed
{{MediaDevices/getDisplayMedia}}. If the user agent is not able to
remove the audio through processing it SHOULD remove the audio
by excluding the document's audio from being captured. If
this results in no audio being captured, the user agent MUST
keep the track muted until it is able to capture audio
again.</p>
</td>
</tr>
<tr id="def-constraint-suppressLocalAudioPlayback">
<td><dfn>suppressLocalAudioPlayback</dfn></td>
<td>{{ConstrainBoolean}}</td>
<td>
<p>As a setting, this value indicates whether or not the user
agent is applying <a>local audio playback suppression</a> to
the source.</p>
<p>As a constraint, this value is only meaningful if the user
selects capturing a [=browser=] [=display surface=]. In that
case, a value of <code>true</code> indicates that the user
agent SHOULD perform <a>local audio playback suppression</a>
on the captured [=browser=] [=display surface=].</p>
<p>When <dfn>local audio playback suppression</dfn> is applied,
the user agent SHOULD stop relaying audio to the local speakers,
but that audio MUST still be captured by any ongoing audio-capturing
[=capture-sessions=]. This suppression MUST NOT be observable to the
captured document. Furthermore, the capturing document may only
observe whether it is applying <a>suppressLocalAudioPlayback</a>;
not whether that suppression is having an effect (i.e. can't
observe if the user is overriding this in the user agent).</p>
<p>When a [=browser=] [=display surface=] is subject to multiple
concurrent captures, <a>local audio playback suppression</a>
SHOULD be applied as long as at least one active audio-capturing
[=capture-session=] is constraining <a>suppressLocalAudioPlayback</a>
to <code>true</code>.</p>
</td>
</tr>
</tbody>
</table>
<p>When inherent properties of the underlying source of a user-selected
<a>display surface</a> change, for example in response to the end-user
resizing a captured window, and these changes render the capabilities
and/or settings of one or more constrainable properties outdated, the
user agent MUST queue a task to run the following step:</p>
<ol>
<li>
<p>Update all affected constrainable properties at the same time.</p>
<p>If this causes an "overconstrained" situation, then the user agent
MUST ignore the culprit constraints for as long as they
overconstrain. The user agent MUST NOT mute the track.</p>
</li>
</ol>
<div class="note">
<p>While min and exact constraints produce TypeError on
getDisplayMedia(), this specification does not alter the
track.applyConstraints() method. Therefore, they may instead produce
OverconstrainedError or succeed depending on values, and therefore
potentially be present to cause this "overconstrained" situation.
The max constraint may also cause this, e.g. with aspectRatio. This
spec considers these to be edge cases that aren't useful.</p>
</div>
<section>
<h2>
Downscaling and Frame Decimation
</h2>
<p>For the purposes of the
[=SelectSettings=]
algorithm, the user agent SHOULD consider all possible combinations of
downscaled dimensions that preserve the aspect ratio of the original
<a>display surface</a> (to the nearest pixel), and frame rates
available through frame decimation, as available
[= settings dictionaries =].
</p>
<p>The downscaling and decimation effects of constraints is then
effectively governed by the
[= fitness distance =] algorithm.</p>
<p>The intent is for the user agent to produce output that is close to
the ideal <code>width</code>, ideal <code>height</code>, and/or
ideal <code>frameRate</code> when these are specified, while at all
times preserving the aspect ratio of the original <a>display surface</a>.
</p>
<p>The user agent SHOULD downscale by the <a>source pixel ratio</a> by
default, unless otherwise directed by applied constraints.
</p>
<p>The user agent MUST NOT crop the captured output.</p>
<p>The user agent MUST NOT upscale the captured output, or create
additional frames, except as needed to preserve high resolutions and
frame rates in an aggregated <a>display surface</a>.</p>
<div class="note">
<p>The max constraint type lets a web application provide a maximum
envelope for constrainable properties like width and height. This is
helpful to limit extreme aspect ratios, should the end-user resize a
<a>window</a> or <a>browser</a> surface to such an extreme while
it is being captured.</p>
</div>
<p>For each constrainable property of positive numeric type in this
specification, the user agent MUST establish a <dfn>floor value</dfn>,
representing the smallest allowable value supported by the user agent
regardless of source. This value MUST be constant and MUST be greater
than <code>0</code>. The user agent is encouraged to support all
values above the <a>floor value</a> regardless of source.</p>
<div class="note">
<p>The purpose of the <a>floor value</a> is to help user agents
avoid failing {{MediaDevices/getDisplayMedia()}} with
<code>OverconstrainedError</code> after the user has already been
prompted, and avoid leaking information about the user's system.
</p>
</div>
</section>
<section>
<h2><dfn>CaptureController</dfn></h2>
<p>
A {{CaptureController}} object may be associated with a [=capture-session=].
It would be used to expose functionality that's associated with
the [=capture-session=] itself, rather than with the call
to {{MediaDevices/getDisplayMedia()}} or its resulting stream or tracks.
</p>
<p>
Any given [=capture-session=] is associated with at most one {{CaptureController}}.
</p>
<p>
At most one {{CaptureController}} is associated with any given [=capture-session=].
</p>
<pre class="idl">
[Exposed=Window, SecureContext]
interface CaptureController {
constructor();
// TODO: Add setFocusBehavior() in a separate PR.
};
</pre>
<dl data-link-for="CaptureController" data-dfn-for="CaptureController" class="methods">
<dt>
<dfn>constructor</dfn>
</dt>
<dd>
<p>
Create one internal slot: <dfn data-dfn-for="CaptureController">[[\isBound]]</dfn>,
initialized to <code>false</code>.
</p>
</dd>
</section>
<section>
<h2><dfn>SelfCapturePreferenceEnum</dfn></h2>
<p>
Describes the different hints an application can provide about whether the
[=display surface=] the application is in, should be among the choices
offered to the user.
</p>
<pre class="idl">
enum SelfCapturePreferenceEnum {
"include",
"exclude"
};
</pre>
<table data-dfn-for="SelfCapturePreferenceEnum" class="simple">
<caption>{{SelfCapturePreferenceEnum}} Enumeration description</caption>
<thead>
<tr>
<th>Enum value</th><th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td>
<dfn id="idl-def-SelfCapturePreferenceEnum.include">include</dfn>
</td>
<td>
The application prefers the surface be included
among the choices offered.
</td>
</tr>
<tr>
<td>
<dfn id="idl-def-SelfCapturePreferenceEnum.exclude">exclude</dfn>
</td>
<td>
The application prefers the surface be excluded
from the choices offered.
</td>
</tr>
</tbody>
</table>
</section>
<section>
<h2><dfn>SystemAudioPreferenceEnum</dfn></h2>
<p>
Describes whether an application invoking {{MediaDevices/getDisplayMedia()}}
would like the user agent to include system audio among the audio sources
offered to the user.
</p>
<pre class="idl">
enum SystemAudioPreferenceEnum {
"include",
"exclude"
};
</pre>
<table data-dfn-for="SystemAudioPreferenceEnum" class="simple">
<tbody>
<tr>
<th colspan="2">
Enumeration description
</th>
</tr>
<tr>
<td>
<dfn id="idl-def-SystemAudioPreferenceEnum.include">include</dfn>
</td>
<td>
The application prefers that options to share system audio be offered to the user
for [=monitor=] [=display surfaces=].
</td>
</tr>
<tr>
<td>
<dfn id="idl-def-SystemAudioPreferenceEnum.exclude">exclude</dfn>
</td>
<td>
The application prefers that options to share system audio not be offered to the user.
</td>
</tr>
</tbody>
</table>
</section>
<section>
<h2><dfn>SurfaceSwitchingPreferenceEnum</dfn></h2>
<p>
Describes whether an application invoking {{MediaDevices/getDisplayMedia()}}
would like the user agent to offer the user an option to dynamically switch
the source [=display surface=] during the capture.
</p>
<pre class="idl">
enum SurfaceSwitchingPreferenceEnum {
"include",
"exclude"
};
</pre>
<table data-dfn-for="SurfaceSwitchingPreferenceEnum" class="simple">
<tbody>
<tr>
<th colspan="2">
Enumeration description
</th>
</tr>
<tr>
<td>
<dfn id="idl-def-SurfaceSwitchingPreferenceEnum.include">include</dfn>
</td>
<td>
The application prefers that an option to dynamically switch the source
[=display surface=] during the capture be offered to the user.
</td>
</tr>
<tr>
<td>
<dfn id="idl-def-SurfaceSwitchingPreferenceEnum.exclude">exclude</dfn>
</td>
<td>
The application prefers that an option to dynamically switch the source
[=display surface=] during the capture NOT be offered to the user.
</td>
</tr>
</tbody>
</table>
</section>
<section>
<h2>DisplayMediaStreamOptions</h2>
<p>The <dfn>DisplayMediaStreamOptions</dfn> dictionary is used to
instruct the user agent what sort of {{MediaStreamTrack}}s may be
included in the {{MediaStream}} returned by
{{MediaDevices/getDisplayMedia}}.</p>
<div>
<pre class="idl">
dictionary DisplayMediaStreamOptions {
(boolean or MediaTrackConstraints) video = true;
(boolean or MediaTrackConstraints) audio = false;
CaptureController controller = null;
SelfCapturePreferenceEnum selfBrowserSurface;
SystemAudioPreferenceEnum systemAudio;
SurfaceSwitchingPreferenceEnum surfaceSwitching;
};</pre>
<section>
<h2>Dictionary <a class="idlType">DisplayMediaStreamOptions</a>
Members</h2>
<dl data-link-for="DisplayMediaStreamOptions" data-dfn-for=
"DisplayMediaStreamOptions" class="dictionary-members">
<dt><dfn><code>video</code></dfn> of type <code>(boolean or {{MediaTrackConstraints}})</code>,
defaulting to <code>true</code></dt>
<dd>
<p>If <code>true</code>, it requests that the returned
{{MediaStream}} contain a video track. If a <code>Constraints</code>
structure is provided, it further specifies desired processing
options to be applied to the video track rendition of the
display surface chosen by the user. If <code>false</code>, the
request will be [=rejected=] with a {{TypeError}}, as per the
<a href="#dom-mediadevices-getdisplaymedia">getDisplayMedia
algorithm</a>.</p>
</dd>
<dt><dfn><code>audio</code></dfn> of type <code>(boolean or {{MediaTrackConstraints}})</code>,
defaulting to <code>false</code></dt>
<dd>
<p>If <code>true</code>, it signals an interest that the
returned {{MediaStream}} contain an audio track, if
supported and audio is available for display surface chosen by
the user. If a <code>Constraints</code> structure is provided, it
further specifies desired processing options to be applied to
the audio track. If <code>false</code>, the {{MediaStream}}
will not contain an audio track.</p>
</dd>
<dt>
<dfn><code>controller</code></dfn> of type <code>{{CaptureController}})</code>,
defaulting to <code>null</code>
</dt>
<dd>
<p>
If not <code>null</code>, this {{CaptureController}} object will be
associated with the [=capture-session=]. Through the methods exposed
on this object, the [=capture-session=] can be manipulated.
</p>
</dd>
<dt>
<dfn><code>selfBrowserSurface</code></dfn> of type {{SelfCapturePreferenceEnum}}
</dt>
<dd>
If present, signals application preference for whether the
[=browser=] [=display surface=] which is associated with
[=this=]'s [=relevant global object=]'s [=associated Document=]'s
[=top-level browsing context=], should be among the choices
offered to the user. The user agent MAY ignore this hint.
</dd>
<dt>
<dfn><code>systemAudio</code></dfn> of type {{SystemAudioPreferenceEnum}}
</dt>
<dd>
If present, signals whether the application would like system audio
to be included among the possible audio sources offered to the user.
</dd>
<dt>
<dfn><code>surfaceSwitching</code></dfn> of type {{SurfaceSwitchingPreferenceEnum}}
</dt>
<dd>
If present, signals whether the application would like the
user agent to offer the user an option to dynamically switch
the captured [=display surface=].
</dd>
</dl>
</section>
</div>
</section>
<section>
<h2>
Extensions to {{MediaTrackSupportedConstraints}}
</h2>
<p>
{{MediaTrackSupportedConstraints}} is extended here with the list of
constraints that a user agent recognizes.
</p>
<pre class="idl"
>partial dictionary MediaTrackSupportedConstraints {
boolean displaySurface = true;
boolean logicalSurface = true;
boolean cursor = true;
boolean restrictOwnAudio = true;
boolean suppressLocalAudioPlayback = true;