Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Eye gaze correction #56

Merged
merged 1 commit into from
Nov 2, 2023
Merged

Eye gaze correction #56

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 62 additions & 0 deletions index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -801,6 +801,68 @@ <h3>Examples</h3>
// Consider falling back to some other method.
}

// Show to user.
const videoElement = document.querySelector("video");
videoElement.srcObject = stream;
&lt;/script&gt;
</pre>
</section>
</section>
<section>
<h2>Exposing MediaStreamTrack source eye gaze correction support</h2>
<p>Some platforms or User Agents may provide built-in support for human eye
gaze correction to make the eyes of faces appear to look at the camera,
in particular for camera video streams.
This may for instance allow the web application to update its UI or to
not apply human eye gaze correction on its own.
For that reason, we extend {{MediaStreamTrack}} with the following
properties.
</p>
<p>The WebIDL changes are the following:</p>
<pre class="idl"
>partial dictionary MediaTrackSupportedConstraints {
boolean eyeGazeCorrection = true;
};

partial dictionary MediaTrackCapabilities {
sequence&lt;boolean&gt; eyeGazeCorrection;
};

partial dictionary MediaTrackConstraintSet {
ConstrainBoolean eyeGazeCorrection;
};

partial dictionary MediaTrackSettings {
boolean eyeGazeCorrection;
};</pre>
<section>
<h3>Processing considerations</h3>
<p>When the "eyeGazeCorrection" setting is set to <code>true</code>,
the User Agent will attempt to correct human eye gaze so that the eyes
of faces appear to look at the camera.
</p>
<p>When the "eyeGazeCorrection" setting is set to <code>false</code>,
the User Agent will not correct human eye gaze.
</p>
</section>
<section>
<h3>Examples</h3>
<pre class="example">
&lt;video&gt;&lt;/video&gt;
&lt;script&gt;
// Open camera.
const stream = await navigator.mediaDevices.getUserMedia({video: true});
const [videoTrack] = stream.getVideoTracks();

// Try to correct eye gaze.
const videoCapabilities = videoTrack.getCapabilities();
if ((videoCapabilities.eyeGazeCorrection || []).includes(true)) {
await videoTrack.applyConstraints({eyeGazeCorrection: {exact: true}});
} else {
// Eye gaze correction is not supported by the platform or by the camera.
// Consider falling back to some other method.
}

Comment on lines +858 to +865
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
const videoCapabilities = videoTrack.getCapabilities();
if ((videoCapabilities.eyeGazeCorrection || []).includes(true)) {
await videoTrack.applyConstraints({eyeGazeCorrection: {exact: true}});
} else {
// Eye gaze correction is not supported by the platform or by the camera.
// Consider falling back to some other method.
}
const capabilities = videoTrack.getCapabilities();
if (("eyeGazeCorrection" in capabilities) {
await videoTrack.applyConstraints({eyeGazeCorrection: true});
} else {
// Eye gaze correction is not supported by the platform or by the camera.
// Consider falling back to some other method.
}

Copy link
Contributor Author

@eehakkin eehakkin Mar 31, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's not the same thing.

if ((videoCapabilities.eyeGazeCorrection || []).includes(true)) tests if eye gaze correction can be enabled and the else branch can then fall back to some other method.

if ("eyeGazeCorrection" in capabilities) tests only if the UA and the track know about eye gaze correction but the capability can be either [false], [false, true] or [true]. The await videoTrack.applyConstraints({eyeGazeCorrection: true}); will of course always succeed regardless the capability of the lack of it because only optional constraints are used. But now the else branch is reached only if there are no eye gaze correction capabilities but not if the eye gaze correction capability is [false] which contradicts the comment in and the intent of that else branch.

// Show to user.
const videoElement = document.querySelector("video");
videoElement.srcObject = stream;
Expand Down