Add experimental hand tracking support #26316

Manishearth · 2020-04-25T00:38:28Z

Adds support for the experimental hand tracking API (with some tweaks made that I intend to upstream).

This needs servo/webxr#163 to actually run on any backend, however that depends on some openxrs changes.

If folks want to try this out, patch in
servo/webxr#163 and run https://manishearth.net/sand/three.js/examples/webxr_vr_paint.html . You also need to toggle the dom.webxr.hand pref.

./mach build -d does not report any errors
./mach test-tidy does not report any errors
These changes fix #___ (GitHub issue number if applicable)

There are tests for these changes OR
These changes do not require tests because this is an experimental API

highfive · 2020-04-25T00:38:34Z

Heads up! This PR modifies the following files:

@asajeffrey: components/script/dom/webidls/XRFrame.webidl, components/script/dom/xrjointpose.rs, components/script/dom/webidls/XRHand.webidl, components/script/dom/webidls/XRJointPose.webidl, components/script/dom/xrspace.rs and 9 more
@KiChjang: components/script/dom/webidls/XRFrame.webidl, components/script/dom/xrjointpose.rs, components/script/dom/webidls/XRHand.webidl, components/script/dom/webidls/XRJointPose.webidl, components/script/dom/xrspace.rs and 9 more

highfive · 2020-04-25T00:38:37Z

Warning

These commits modify unsafe code. Please review it carefully!
These commits modify script code, but no tests are modified. Please consider adding a test!

Manishearth · 2020-04-25T00:38:39Z

r? @jdm @asajeffrey

bors-servo · 2020-04-25T01:28:27Z

☔ The latest upstream changes (presumably #26312) made this pull request unmergeable. Please resolve the merge conflicts.

Manishearth · 2020-04-25T04:47:24Z

This can be landed now. The actual feature is behind a pref, and it doesn't work anyway without the webxr openxr patches (servo/webxr#163), but we can get the impl in-tree at least.


        Update webxr

asajeffrey

Just minor nits. Feel free to ignore them and r=me.

asajeffrey · 2020-04-27T15:25:23Z

components/config/prefs.rs

@@ -299,6 +299,8 @@ mod gen {
                    test: bool,
                    #[serde(default)]
                    glwindow: bool,
+                    #[serde(default)]
+                    hand: bool,


should the pref be dom.webxr.hand or dom.webxr.hand.enabled?

done, dom.webxr.hands.enabled (the API is called webxr-hands-input, so I picked this option instead)

asajeffrey · 2020-04-27T15:25:23Z

components/script/dom/bindings/trace.rs

@@ -881,6 +882,58 @@ where
    }
 }

+unsafe impl<J> JSTraceable for Hand<J>


Hmm, it would be nice if we could derive this, it'll be very easy to mess this up down the road :(

Can't without introducing a mozjs dep to webxr

asajeffrey · 2020-04-27T15:25:23Z

components/script/dom/webidls/XRInputSource.webidl

@@ -24,4 +24,7 @@ interface XRInputSource {
  [SameObject] readonly attribute XRSpace? gripSpace;
  // [SameObject] readonly attribute Gamepad? gamepad;
  /* [SameObject] */ readonly attribute /* FrozenArray<DOMString> */ any profiles;
+
+  [Pref="dom.webxr.hand"]
+  readonly attribute XRHand? hand;


I guess this is a spec nit, but hand singular?

An input source only has one hand. If your input source has multiple hands, it might be time to call your doctor and/or plastic surgeon.

(This also matches with gamepad -- a single input source is a single controller in a single hand, and it can have one gamepad and one associated hand)

Manishearth · 2020-04-27T22:58:03Z

@bors-servo r=asajeffrey

bors-servo · 2020-04-27T22:58:04Z

📌 Commit 51f1c5e has been approved by asajeffrey

bors-servo · 2020-04-27T22:58:09Z

⌛ Testing commit 51f1c5e with merge 595160d...


        Auto merge of #26316 - Manishearth:hands, r=asajeffrey

Add experimental hand tracking support Depends on servo/webxr#162 Adds support for [the experimental hand tracking API](https://github.com/immersive-web/webxr-hands-input/blob/master/explainer.md) (with some tweaks made that I intend to upstream). This needs servo/webxr#163 to actually run on any backend, however that depends on some openxrs changes. If folks want to try this out, patch in servo/webxr#163 and run https://manishearth.net/sand/three.js/examples/webxr_vr_paint.html . You also need to toggle the `dom.webxr.hand` pref.  ---  - [x] `./mach build -d` does not report any errors - [x] `./mach test-tidy` does not report any errors - [ ] These changes fix #___ (GitHub issue number if applicable)  - [ ] There are tests for these changes OR - [x] These changes do not require tests because this is an experimental API

bors-servo · 2020-04-27T23:19:39Z

💔 Test failed - status-taskcluster

bors-servo · 2020-04-28T00:36:01Z

📌 Commit c30ad6c has been approved by asajeffrey

bors-servo · 2020-04-28T00:36:06Z

⌛ Testing commit c30ad6c with merge b4f382e...


        Auto merge of #26316 - Manishearth:hands, r=asajeffrey

Add experimental hand tracking support Depends on servo/webxr#162 Adds support for [the experimental hand tracking API](https://github.com/immersive-web/webxr-hands-input/blob/master/explainer.md) (with some tweaks made that I intend to upstream). This needs servo/webxr#163 to actually run on any backend, however that depends on some openxrs changes. If folks want to try this out, patch in servo/webxr#163 and run https://manishearth.net/sand/three.js/examples/webxr_vr_paint.html . You also need to toggle the `dom.webxr.hand` pref.  ---  - [x] `./mach build -d` does not report any errors - [x] `./mach test-tidy` does not report any errors - [ ] These changes fix #___ (GitHub issue number if applicable)  - [ ] There are tests for these changes OR - [x] These changes do not require tests because this is an experimental API

bors-servo · 2020-04-28T01:23:00Z

💔 Test failed - status-taskcluster

Manishearth · 2020-04-28T01:24:49Z

@bors-servo retry

#25513

  ▶ TIMEOUT [expected PASS] /html/infrastructure/common-microsyntaxes/colours/parsing-legacy-colour-value-ascii-case-insensitive.html
  │ 
  │ _RegisterApplication(), FAILED TO establish the default connection to the WindowServer, _CGSDefaultConnection() is NULL.
  │ Shutting down the Constellation after generating an output file or exit flag specified
  │ 24 threads are still running after shutdown (bad).
  └ _RegisterApplication(), FAILED TO establish the default connection to the WindowServer, _CGSDefaultConnection() is NULL.
  ▶ TIMEOUT [expected OK] /css/css-color/rgb-rounding-001.html
  ▶ TIMEOUT [expected OK] /css/css-text/white-space/seg-break-transformation-001.html
  ▶ TIMEOUT [expected OK] /css/css-text-decor/parsing/text-decoration-color-invalid.html
  ▶ TIMEOUT [expected OK] /css/css-text/inheritance.html

bors-servo · 2020-04-28T01:24:54Z

⌛ Testing commit c30ad6c with merge 970c0ed...


        Auto merge of #26316 - Manishearth:hands, r=asajeffrey

Add experimental hand tracking support Depends on servo/webxr#162 Adds support for [the experimental hand tracking API](https://github.com/immersive-web/webxr-hands-input/blob/master/explainer.md) (with some tweaks made that I intend to upstream). This needs servo/webxr#163 to actually run on any backend, however that depends on some openxrs changes. If folks want to try this out, patch in servo/webxr#163 and run https://manishearth.net/sand/three.js/examples/webxr_vr_paint.html . You also need to toggle the `dom.webxr.hand` pref.  ---  - [x] `./mach build -d` does not report any errors - [x] `./mach test-tidy` does not report any errors - [ ] These changes fix #___ (GitHub issue number if applicable)  - [ ] There are tests for these changes OR - [x] These changes do not require tests because this is an experimental API

bors-servo · 2020-04-28T01:25:27Z

⌛ Testing commit c30ad6c with merge 000fd93...


        Auto merge of #26316 - Manishearth:hands, r=asajeffrey

Add experimental hand tracking support Depends on servo/webxr#162 Adds support for [the experimental hand tracking API](https://github.com/immersive-web/webxr-hands-input/blob/master/explainer.md) (with some tweaks made that I intend to upstream). This needs servo/webxr#163 to actually run on any backend, however that depends on some openxrs changes. If folks want to try this out, patch in servo/webxr#163 and run https://manishearth.net/sand/three.js/examples/webxr_vr_paint.html . You also need to toggle the `dom.webxr.hand` pref.  ---  - [x] `./mach build -d` does not report any errors - [x] `./mach test-tidy` does not report any errors - [ ] These changes fix #___ (GitHub issue number if applicable)  - [ ] There are tests for these changes OR - [x] These changes do not require tests because this is an experimental API

bors-servo · 2020-04-28T01:44:46Z

💔 Test failed - status-taskcluster

Manishearth · 2020-04-28T01:53:17Z

@bors-servo retry

7 unexpected results that are NOT known-intermittents:
  ▶ TIMEOUT [expected FAIL] /css/css-text/shaping/shaping-007.html
  ▶ TIMEOUT [expected PASS] /css/css-text/shaping/shaping-003.html
  ▶ TIMEOUT [expected FAIL] /css/CSS2/css21-errata/s-11-1-1b-009.html
  ▶ TIMEOUT [expected FAIL] /css/css-text/shaping/shaping-005.html
  ▶ TIMEOUT [expected PASS] /css/css-text/shaping/shaping-002.html
  │ 
  │ Shutting down the Constellation after generating an output file or exit flag specified
  └ 24 threads are still running after shutdown (bad).
  ▶ TIMEOUT [expected PASS] /css/css-text/shaping/shaping-001.html
  │ 
  │ Shutting down the Constellation after generating an output file or exit flag specified
  └ 45 threads are still running after shutdown (bad).
  ▶ TIMEOUT [expected FAIL] /css/css-text/shaping/shaping-006.html
  │ 
  │ Shutting down the Constellation after generating an output file or exit flag specified
  └ 45 threads are still running after shutdown (bad).

bors-servo · 2020-04-28T01:53:22Z

⌛ Testing commit c30ad6c with merge 9ea4b10...


        Auto merge of #26316 - Manishearth:hands, r=asajeffrey

Add experimental hand tracking support Depends on servo/webxr#162 Adds support for [the experimental hand tracking API](https://github.com/immersive-web/webxr-hands-input/blob/master/explainer.md) (with some tweaks made that I intend to upstream). This needs servo/webxr#163 to actually run on any backend, however that depends on some openxrs changes. If folks want to try this out, patch in servo/webxr#163 and run https://manishearth.net/sand/three.js/examples/webxr_vr_paint.html . You also need to toggle the `dom.webxr.hand` pref.  ---  - [x] `./mach build -d` does not report any errors - [x] `./mach test-tidy` does not report any errors - [ ] These changes fix #___ (GitHub issue number if applicable)  - [ ] There are tests for these changes OR - [x] These changes do not require tests because this is an experimental API

bors-servo · 2020-04-28T11:53:22Z

💥 Test timed out

jdm · 2020-04-28T11:58:26Z

@bors-servo retry

bors-servo · 2020-04-28T12:33:21Z

⌛ Testing commit c30ad6c with merge 1f34c55...

bors-servo · 2020-04-28T12:34:09Z

☀️ Test successful - status-taskcluster
Approved by: asajeffrey
Pushing 1f34c55 to master...

highfive assigned jdm Apr 25, 2020

highfive added the S-awaiting-review label Apr 25, 2020

highfive added the S-needs-rebase label Apr 25, 2020

Manishearth force-pushed the Manishearth:hands branch from 50b311f to a1f941b Apr 25, 2020

Manishearth marked this pull request as ready for review Apr 25, 2020

Manishearth force-pushed the Manishearth:hands branch from a1f941b to bba2bfb Apr 25, 2020

Update webxr

d4a90b1

Manishearth force-pushed the Manishearth:hands branch from bba2bfb to 2a75d2e Apr 27, 2020

asajeffrey approved these changes Apr 27, 2020

View changes

Manishearth force-pushed the Manishearth:hands branch from 2a75d2e to 51f1c5e Apr 27, 2020

highfive assigned asajeffrey and unassigned jdm Apr 27, 2020

highfive added S-awaiting-merge and removed S-awaiting-review S-needs-rebase labels Apr 27, 2020

highfive added S-tests-failed and removed S-awaiting-merge labels Apr 27, 2020

Manishearth force-pushed the Manishearth:hands branch from 51f1c5e to 96eb24d Apr 27, 2020

highfive added S-awaiting-review and removed S-tests-failed labels Apr 27, 2020

highfive added S-awaiting-merge and removed S-awaiting-review labels Apr 28, 2020

highfive added S-tests-failed and removed S-awaiting-merge labels Apr 28, 2020

highfive added S-awaiting-merge and removed S-tests-failed labels Apr 28, 2020

highfive added S-tests-failed and removed S-awaiting-merge labels Apr 28, 2020

highfive added S-awaiting-merge and removed S-tests-failed labels Apr 28, 2020

bors-servo merged commit 1f34c55 into servo:master Apr 28, 2020
2 checks passed

2 checks passed

Community-TC (pull_request) TaskGroup: success
Details

homu Test successful
Details

highfive removed the S-awaiting-merge label Apr 28, 2020

servo / servo

Add experimental hand tracking support #26316

Add experimental hand tracking support #26316

Manishearth commented Apr 25, 2020 •

edited

highfive commented Apr 25, 2020

highfive commented Apr 25, 2020

Manishearth commented Apr 25, 2020

bors-servo commented Apr 25, 2020

Manishearth commented Apr 25, 2020 •

edited

asajeffrey left a comment

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Manishearth commented Apr 27, 2020

bors-servo commented Apr 27, 2020

bors-servo commented Apr 27, 2020

bors-servo commented Apr 27, 2020

bors-servo commented Apr 28, 2020

bors-servo commented Apr 28, 2020

bors-servo commented Apr 28, 2020

Manishearth commented Apr 28, 2020 •

edited

bors-servo commented Apr 28, 2020

bors-servo commented Apr 28, 2020

bors-servo commented Apr 28, 2020

Manishearth commented Apr 28, 2020

bors-servo commented Apr 28, 2020

bors-servo commented Apr 28, 2020

jdm commented Apr 28, 2020

bors-servo commented Apr 28, 2020

bors-servo commented Apr 28, 2020

servo / servo

Join GitHub today

Add experimental hand tracking support #26316

Add experimental hand tracking support #26316

Conversation

Manishearth commented Apr 25, 2020 • edited

highfive commented Apr 25, 2020

highfive commented Apr 25, 2020

Manishearth commented Apr 25, 2020

bors-servo commented Apr 25, 2020

Manishearth commented Apr 25, 2020 • edited

asajeffrey left a comment

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Manishearth commented Apr 27, 2020

bors-servo commented Apr 27, 2020

bors-servo commented Apr 27, 2020

bors-servo commented Apr 27, 2020

bors-servo commented Apr 28, 2020

bors-servo commented Apr 28, 2020

bors-servo commented Apr 28, 2020

Manishearth commented Apr 28, 2020 • edited

bors-servo commented Apr 28, 2020

bors-servo commented Apr 28, 2020

bors-servo commented Apr 28, 2020

Manishearth commented Apr 28, 2020

bors-servo commented Apr 28, 2020

bors-servo commented Apr 28, 2020

jdm commented Apr 28, 2020

bors-servo commented Apr 28, 2020

bors-servo commented Apr 28, 2020

Manishearth commented Apr 25, 2020 •

edited

Manishearth commented Apr 25, 2020 •

edited

Manishearth commented Apr 28, 2020 •

edited