New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add API for requesting permission to receive device motion / orientation events #57

Open
cdumez opened this Issue Dec 13, 2018 · 88 comments

Comments

Projects
None yet
@cdumez
Copy link

cdumez commented Dec 13, 2018

Due to the privacy issues related to this API (e.g. https://www.wired.com/story/mobile-websites-can-tap-into-your-phones-sensors-without-asking/ and https://arxiv.org/pdf/1605.08763.pdf), we are considering adding to WebKit a permission dialog in order to ask the user if they want to expose their device orientation/motion to the Website.

However, because trackers already register such event listeners on top sites, we are worried about the risk of over-prompting. We wouldn't want this prompt to start showing on a lot of top-sites.

To address the issue, we'd like to propose adding a new API allowing the page's script explicitly ask for permission to access the device / motion. We think we should also require a user-gesture to call this new API.

It could look something like this (similar to Notification API):

[Exposed=(Window)]
interface DeviceOrientation {
  [Exposed=Window] static Promise<boolean> requestPermission();
};

What are your thoughts?

@reillyeon

This comment has been minimized.

Copy link
Collaborator

reillyeon commented Dec 13, 2018

This specification is superseded by the Generic Sensors family of specifications, which include integration with the Permissions API. I think that this specification should be updated with a similar integration by requiring "accelerometer" and "gyroscope" permissions for deviceorientation and devicemotion events and additionally "magnetometer" permission for deviceorientationabsolute events. This would match the existing Orientation Sensor, Accelerometer, Gyroscope and Magnetometer specifications which provide access to the same or similar data.

Is there any plan to implement navigator.permissions.query() in WebKit?

Blink is currently considering similar changes to start notifying users or prompting when sites request access to device sensors. The plumbing is already implemented to manage these permissions however they are still granted by default.

@cdumez

This comment has been minimized.

Copy link
Author

cdumez commented Dec 13, 2018

@reillyeon I may be missing something but where is that API that JS actually calls to request permission for something? navigator.permissions.query() seems to only query, not request.

There is a section in the Permissions API spec about "requesting more permission" but it seems to merely be an algorithm, no actual API for it.

What we want in WebKit is really an API to ask for permission to access device orientation. Without such permission request, the content would not be getting the events at all.

@reillyeon

This comment has been minimized.

Copy link
Collaborator

reillyeon commented Dec 13, 2018

That is correct, I forgot that the navigator.permissions.request() method has been removed in favor of each specification integrating its own method for requesting more permission, but calling into the algorithm you mention. A cursory look at query() made me think that that had been integrated into it instead.

Given that the Generic Sensors specifications I linked to above should be updated to include a requestPermission() method similar to Notification.requestPermission().

As the DeviceOrientation API does not provide an object in the global scope like AbsoluteOrientationSensor or Notification (except perhaps DeviceOrientationEvent) what do you think about not adding a method specific to this API but instead relying on the ones that need to be added to the newer specifications?

My recollection is that WebKit does not implement the deviceorientationabsolute event and so implementing a single RelativeOrientationSensor.requestPermission() method would suffice for your purposes.

@cdumez

This comment has been minimized.

Copy link
Author

cdumez commented Dec 13, 2018

I think relying on a Permissions API from another spec would probably work (although, I'd have to see what it looks like first). That said, it would be bit confusing to use Permissions API from another (newer) API and predicate using the (older) API on it.

How about for the 'devicemotion' events, would it be a LinearAccelerationSensor.requestPermission() ?

Honestly, the fact that the names do not match would be super-confusing IMO.

I get that the DeviceOrientation API does not have a global scope object at the moment but it seems like it could have one. Or I actually like the idea of adding static requestPermission() methods to the existing DeviceOrientationEvent / DeviceMotionEvent interfaces.

@anssiko

This comment has been minimized.

Copy link
Member

anssiko commented Dec 13, 2018

navigator.permissions.request() is specified in https://wicg.github.io/permissions-request/

@jyasskin probably knows the status of that work.

@reillyeon

This comment has been minimized.

Copy link
Collaborator

reillyeon commented Dec 13, 2018

I think relying on a Permissions API from another spec would probably work (although, I'd have to see what it looks like first). That said, it would be bit confusing to use Permissions API from another (newer) API and predicate using the (older) API on it.

I believe that use of either API should be predicated on the same permission since they expose the same capability. Having two functions that request the same permission feels like unnecessary bloat but I'm willing to accept that in favor of not confusing developers.

How about for the 'devicemotion' events, would it be a LinearAccelerationSensor.requestPermission() ?

devicemotion events should require the "accelerometer" and "gyroscope" permissions which conveniently is the same as the ones I think should be required for deviceorientation events so a single call could provide the necessary permissions for either event type.

Honestly, the fact that the names do not match would be super-confusing IMO.

I don't disagree. It sounds like navigator.permissions.request() is not as abandoned as I thought it was. That API feels like a better solution.

I get that the DeviceOrientation API does not have a global scope object at the moment but it seems like it could have one. Or I actually like the idea of adding static requestPermission() methods to the existing DeviceOrientationEvent / DeviceMotionEvent interfaces.

That seems reasonable if doing this solely through the Permissions Request API is untenable.

@cdumez

This comment has been minimized.

Copy link
Author

cdumez commented Dec 13, 2018

So if I understand correctly, it would look like:

navigation.permissions.request({ name: “gyroscope” }); // For DeviceOrientationEvents
navigation.permissions.request({ name: “accelerometer” }); // For DeviceMotionEvents

This does not look bad actually.

@reillyeon

This comment has been minimized.

Copy link
Collaborator

reillyeon commented Dec 13, 2018

You need both "gyroscope" and "accelerometer" for both so it would look like:

Promise.all([navigation.permissions.request({ name: “accelerometer” }),
             navigation.permissions.request({ name: “gyroscope” })])
    .then(results => {
      ...
    });
@jyasskin

This comment has been minimized.

Copy link
Member

jyasskin commented Dec 13, 2018

I think w3c/permissions#158 is the most recent discussion about permissions.request(...), and it links to the older discussions. Firefox folks have generally been opposed, Chrome's been in favor, and Edge and Safari have ignored it. I haven't been paying a lot of attention in the last year so can't really help guide here.

@cdumez

This comment has been minimized.

Copy link
Author

cdumez commented Dec 13, 2018

@annevk Based on w3c/permissions#158, it looks like Mozilla considered gating the old deviceorientation / devicemotion events behind a permission as well. Did you end up doing it or do you still have plans to do so?

@annevk

This comment has been minimized.

Copy link
Member

annevk commented Dec 14, 2018

@cdumez we haven't made progress on this yet I believe, but I strongly suspect we're still interested. Perhaps not in a generic request() method though, but something specific for this API. Either that, or seeing if we can remove the API altogether (we don't expose it so widely I think).

cc @ehsan

@cdumez

This comment has been minimized.

Copy link
Author

cdumez commented Dec 14, 2018

Yes, I discussed it internally a bit and I do not think we are interested in the generic permissions.request().

A few things we could do:

  1. add a permission request api to this spec
  2. drop the api altogether but there are some legit use cases so it would be unfortunate.
  3. maybe require a user gesture when adding the event listener (and then we’d show a prompt). Avoids over prompting and adding any api.

I would have preferred option 1 I think but given the mess with permissions it seems, I am starting to think about option 3.

@richtr

This comment has been minimized.

Copy link
Member

richtr commented Dec 14, 2018

From the study referenced in that Wired article (PDF paper: https://sensor-js.xyz/webs-sixth-sense-ccs18.pdf) the recommendations given to browser vendors are as follows:

  • W3C’s recommendation for disabling sensor access on crossorigin
    iframes [80] will limit the access from untrusted thirdparty
    scripts and is a step in the right direction. However,
    Safari and Firefox are the only two browsers that follow this
    recommendation. Our measurements indicate that scripts
    that access sensor APIs are frequently embedded in crossorigin
    iframes (67.4% of the 31 444 cases). This shows that
    W3C’s mitigation would be effective at curbing the exposure
    to untrusted scripts. Allowing sensor access on insecure
    pages is another issue where browsers do not follow the
    W3C spec: all nine browsers we studied allowed access to
    sensors on insecure (HTTP) pages.

  • Feature Policy API [16], if deployed, will allow publishers
    to selectively disable JavaScript APIs. Publisher may disable
    sensor APIs using this API to prevent potential misuses by
    the third-party scripts they embed.

  • Provide low resolution sensor data by default, and require
    user permission for higher resolution sensor data.

  • To improve user awareness and curb surreptitious sensor
    access, provide users with a visual indication that the sensor
    data is being accessed.

  • Require user permission to access sensor data in private
    browsing mode, limit resolution, or disable sensor access all
    together.

[80] Rich Tibbett, Tim Volodine, Steve Block, and Andrei Popescu.
2018. DeviceOrientation event specification.
https://w3c.github.io/deviceorientation/

[16] Ian Clelland. 2017. Feature policy: Draft community group
report.
https://wicg.github.io/feature-policy/.

This specification could instead be updated to include normative text that limits the sampling rate for deviceorientation, deviceorientationabsolute and devicemotion events to less than or equal to 60 Hz as effective mitigation to fingerprinting and other security and privacy issues.

At that point, a permissions API could then be introduced for accessing higher frequency sensor data from devices. Or, just require higher frequency sensor data to be obtained only via the Generic Sensors API going forward.

Incremental improvements based on these research recommendations seem possible. Putting all access to these events behind a permission dialog seems like a Really Big Gun to start with. It may still be required to eventually look at that approach but for an API that has been available for 8+ years this could be extremely damaging for legitimate use cases given its widespread adoption.

@cdumez

This comment has been minimized.

Copy link
Author

cdumez commented Dec 14, 2018

This would not be acceptable on WebKit-side. Limiting sampling rate for those events would mitigate fingerprinting but not some of the other privacy issues (e.g. The site can tell if the user is standing or laying in bed).

We'd require a user gesture / permission for any of those events. We could also disable the API altogether as @annevk mentioned.

@richtr

This comment has been minimized.

Copy link
Member

richtr commented Dec 14, 2018

Determining which way the user is pointing their device is the entire purpose of this API.

Enabling usage for legitimate sites while preventing fingerprinting seems to be the objective of this proposal based on the contents and research in the cited studies.

The studies you referenced to justify this addition do not recommend putting this behind permission prompts as a first course of action. Those studies present many other options that could be considered.

Why the immediate and full rejection of all recommendations from your own references?

@cdumez cdumez closed this Dec 14, 2018

@cdumez cdumez reopened this Dec 14, 2018

@geoffreygaren

This comment has been minimized.

Copy link

geoffreygaren commented Dec 14, 2018

Why the immediate and full rejection of all recommendations from your own references?

Though the attached research paper investigated quantization,

(a) It acknowledged that quantization only reduced the fingerprinting surface area, and did not eliminate it;

(b) It acknowledged that quantization, though it did not break a toy game, might still break other important use cases;

(c) It did not address at all concerns like key logging and other forms of privacy invasion, which are mentioned in the Wired article;

(d) It did not address the unusually high power impact of the motion and orientation (which we probably should have mentioned earlier).

I don't think we reject quantization as an area of investigation. Rather, we acknowledge that quantization is an area of investigation, and we also acknowledge the need to solve this privacy and security problem without waiting for the conclusion of an open-ended investigation.

@reillyeon

This comment has been minimized.

Copy link
Collaborator

reillyeon commented Dec 14, 2018

I am in favor of adding requestPermission() methods to DeviceOrientationEvent and DeviceMotionEvent (as well as the equivalent Generic Sensors APIs). Acknowledging WebKit's concerns I think it is reasonable for an implementation to choose to completely deny permission for these APIs by default if they wish to. Blink's current plans are to,

  1. Implement a permissions API allowing a site to request a particular level of precision and/or frequency.
  2. Allow access to the existing level of precision and frequency by default.
  3. Notify the user that a site is using the device sensors and allow them to revoke permission.

(2) is necessary to maintain compatibility with existing sites and the existing information leaks are partially mitigated by (3). (1) provides a framework for enabling (a) future use cases that require more data and (b) a move towards reducing the amount of data available without a permission in a gradual fashion in order to allow sites to adapt.

@richtr

This comment has been minimized.

Copy link
Member

richtr commented Dec 19, 2018

@reillyeon If we must add permissions to this API then your approach seems like a least disruptive way to do that - progressively reducing the data available (assuming (2) continues to be available by default without permissions). We could also add a relevant console warning message during this process informing developers of the planned change to access in future releases.

What the research in https://arxiv.org/pdf/1605.08763.pdf suggests is that a combination of reducing the frequency of events + sensor obfuscation + sensor quantization would be enough to significantly fix security and privacy concerns in this thread without significantly disrupting web app UI and UX for legitimate actors. Permission dialogs or any changes that disrupt a user's flow while using a web app should be considered as a last resort (when all you have is a hammer...).

There is going to be a disruptive impact of adding permission requirements for sensor events on the nascent web VR/AR/360 ecosystem. e.g. Developers will need to start wrapping their calls for sensor events within user-gesture events IIUC. It is unclear if or how permission requests would display if called from within e.g. iframes (with e.g. correct allow attributes cross-origin).

Anything we can do to reduce the impact of blocked requests and disrupted UI/UX experiences within this ecosystem - gradually introducing such disruptive changes only if all other techniques have been tried and deemed unsatisfactory after further studies, making developers aware of these updates via e.g. console.warn before they land and not just landing these changes one day in mainline browsers without letting existing API consumers adapt - would be very welcome here.

@richtr

This comment has been minimized.

Copy link
Member

richtr commented Dec 19, 2018

In summary, changes could be made immediately to implementations by obfuscating, quantizing and/or reducing the rate of sensor event data emitted without impacting the existing API footprint or validity/conformity of implementations to this specification. We could also immediately add security and privacy considerations to this effect to this specification based on these emergent research studies since last publication. We can then continue to discuss additional requirements / API changes as required, hopefully with a view to reducing the impact of those changes on current legitimate consumers of this API.

@KenanSulayman

This comment has been minimized.

Copy link

KenanSulayman commented Dec 21, 2018

I would very much appreciate a more sober reflection on how this might possibly affect developers and users. The privacy concerns are perfectly valid. But removing the API, or locking it down, will wipe out a complete genre of interactive online games. Not only the ones on the web, but rather those distributed to mobile devices via Cordova (et al.).

Additionally, how do you intend to communicate this permission request to the users? What's the difference between device acceleration and orientation data for the end-user? What if the user only permits either of them? Based on what rationale? What if the user does not understand the requested sensor function?

Does requesting permissions protect users? Or does it simply limit the “tracking” to big platforms that provide so many features that there's at least one with a credible use case for one of these APIs, opening them for the whole rest of the platform?

I'm unconvinced these questions have sufficiently been answered.

@annevk

This comment has been minimized.

Copy link
Member

annevk commented Jan 2, 2019

@richtr you have two-out-of-three implementers saying that's not enough. That doesn't seem like a good summary to me.

@richtr

This comment has been minimized.

Copy link
Member

richtr commented Jan 7, 2019

There are many softer approaches to be explored here as opposed to implementing a black-and-white “on/off” switch straight out of the gate.

Following your current course will effectively destroy nascent VR/AR/360/gaming companies and industries and pretending otherwise is silly.

I want to avoid a cliff-edge scenario where access to these sensors would change overnight without proper planning — particularly in browsers where no platform-alternative browser engines exist.

For that purpose there are many many intermediate steps, recommended in the cited papers - before you need to implement on/off switches for an 8+ year old API. Jumping straight for that approach is incredibly hostile to web developers. Have you evaluated the full impact that would have?

That is the purpose of the summary above.

@cdumez

This comment has been minimized.

Copy link
Author

cdumez commented Jan 7, 2019

I believe @geoffreygaren’s comment above explains why we want a permission api and user consent in WebKit. For us, it is either this or disabling the API by default until we can make it safe. The other proposals do not address all our concerns as explained by Geoff.

@richtr

This comment has been minimized.

Copy link
Member

richtr commented Jan 9, 2019

For us, it is either this or disabling the API by default until we can make it safe.

All content relying on the API will break overnight. No replacement sensor access is provided on WebKit (Generic Sensors API). No alternative browser engines are allowed on iOS so there is no way to keep any applications relying on those APIs running. No deprecation strategy is planned warning developers of disruption or revocation of sensor data before they land in Safari. A cliff-edge backward-incompatible fall-off of usage is deemed the correct way to modify an extensively used 8+ year old web platform API.

Browser-based content and embedded web view content in native applications will cease operation in all of the following use cases:

  • Games and gaming
  • Navigation
  • Virtual Reality
  • Augmented Reality
  • 360 viewers (videos, panoramas)
  • 3D scanning
  • Gesture recognition
  • Step counters
  • Physical activity monitoring
  • Sleep monitors

Multiple proposals exist to make the emitted motion sensor data both a.) still useful for legitimate use cases and b.) resistant to raised security and privacy concerns by refining/limiting the data emitted in different ways. Those solutions will not be considered further because the effectiveness of those solutions needs further investigation/research even though they are cited as potentially resolving all current security and privacy concerns by the original investigators. Instead we will modify the API invocation to require permissions/user-gesture or just remove the whole API because it is easier, less implementation work and resolves all security and privacy concerns because then there are no use cases or usage anymore.

Now contrast this to the alternative approach/summary I posted above.

@zchrykng

This comment has been minimized.

Copy link

zchrykng commented Jan 9, 2019

Browser-based content and embedded web view content in native applications will cease operation in all of the following use cases:

  • Games and gaming
  • Navigation
  • Virtual Reality
  • Augmented Reality
  • 360 viewers (videos, panoramas)
  • 3D scanning
  • Gesture recognition
  • Step counters
  • Physical activity monitoring
  • Sleep monitors

How many of these things are implemented as web apps/views? And why would requiring them to ask permission be that big of a deal? There is exactly zero people who should be able to access any of this data without asking for permission, in my opinion. And I honestly couldn't care less how many business models or apps would be affected by having to ask permission. If the app is scared of asking for permission first, I don't want them having my data anyway.

@eligrey

This comment has been minimized.

Copy link

eligrey commented Jan 10, 2019

Companies like UnifyID use gyroscope data for user gait profiling. I would definitely prefer if gyroscope data required permission to access.

@Codeusa

This comment has been minimized.

Copy link

Codeusa commented Jan 10, 2019

Sacrificing UX in the name of a slight privacy increase is regressive at best. Not only will most end-users not understand what granting the permission actually entails, it just creates friction when trying to build solutions that “just work.”

In HTML5 eBook reader I created I actually leverage device orientation to determine when someone is laying down to suggest turning on night mode at certain hours. There are plenty of valid reasons not to put these standard features behind a permissions prompt.

Throttling polling rates is a decent mitigation.

@anssiko

This comment has been minimized.

Copy link
Member

anssiko commented Jan 10, 2019

A note on scoping with my chair hat on:

As @reillyeon pointed out earlier, DAS WG's position is this specification is superseded by the Generic Sensors family of specifications which include integration with the Permissions API. The DAS WG considers this spec to be in maintenance, and as such no new feature work per charter. The charter is not super clear on this, says "fix remaining known issues and to finish" so I'm open to discussion if someone feels strongly we should add new API surface to this spec.

An alternative transition plan to browsers who are planning to unship this legacy feature sometime in the future would be to first implement (applicable parts of) the new API surface (see Generic Sensor API and related concrete sensors at https://www.w3.org/das/roadmap), and deprecate the legacy API surface only after a reasonable grace period during which web developers are recommended to migrate to the new API that provides the permission hooks and other privacy enhancements.

@tangobravo

This comment has been minimized.

Copy link

tangobravo commented Feb 2, 2019

@othermaciej Thanks for the responses here.

Just to be clear about the current state of the beta: the feature is currently disabled by default everywhere unless the user manually turns on the user preference. If we had per-site permission, then that would take priority over the setting.

Thanks for clarifying the current state. For the full 12.2 release you have (at least) 3 options:

  1. Keep things as in the beta: many sites stop working, not many users would go through the hassle of turning on the option, horrible UX to talk users through the process.
  2. Switch the default setting to be enabled: gives time for more discussion whilst still allowing privacy-conscious users to turn it off. [I'd throw in implementing other less-impactful mitigations in this option too, but given timeframes I think that is unlikely.]
  3. Keep the default disabled but implement a permission request API: "progressive enhancement" use cases likely break, but those that require this API (360 photos etc) would be able to adapt.

I would strongly advocate for option 2 or 3.

Sorry for hijacking the bug for continuing this discussion. WebKit (rightly IMHO) don't consider it their bug, and the private nature of Apple's bug tracker shuts down discussion and prevents the web community from demonstrating the strength of feeling on this issue.

@geoffreygaren

This comment has been minimized.

Copy link

geoffreygaren commented Feb 2, 2019

Are you seriously suggesting that, returning e.g. integer values for deviceorientation is going to allow an attacker to identify a single device/user from a database of four billion smartphone devices?

"returning e.g. integer values for deviceorientation" is an incomplete specification.

What I'm suggesting is that you have provided no evidence that you could complete that specification in a meaningful way. And I have provided evidence that you can't.

If the solution is indeed trivial, you are welcome to end this debate at any time by specifying the solution and providing evidence that it works.

Everything being discussed is a shade of that argument that there is, with certainty, a level of default access available that provides enough privacy and security protections while not completely disabling current uses.

Indeed, you do keep asserting that you could solve the privacy and consent problem. But asserting something doesn't make it so. Evidence is how we evaluate truth, and you have provided none.

@tparisi

This comment has been minimized.

Copy link

tparisi commented Feb 4, 2019

Two questions: 1) is this same change going to be required of native apps going forward? If not, why? 2) has anyone considered all the non-WebXR content this will break, like HTML5 games or websites that use orientation to change layout?

@griffin2000

This comment has been minimized.

Copy link

griffin2000 commented Feb 4, 2019

Apple/WebKit aren't going to change their minds on this. iOS 12.2 is going to ship with device orientation/motion disabled by default. We need to expedite the process of formalizing an API so that future versions of iOS can ship with it. If we move quickly, maybe even iOS 12.2 could have this implemented.

This is by far the most important aspect of this discussion. The exact method that happens FAR is less important than coming up with a solution that can get this fixed for the 1.2.2 full release.

If that fails it will be really bad for web standards as a whole, and, speaking as a developer, make me much less likely try and support these kind of standards in the future.

@grorg

This comment has been minimized.

Copy link

grorg commented Feb 4, 2019

Two questions: 1) is this same change going to be required of native apps going forward? If not, why?

Possibly, yes. But it's a different situation - web sites are including third-party code that is violating privacy by using this API. Apps typically don't do that (as often).

And either way, the privacy situation exists on the Web, so it needs to be solved here.

  1. has anyone considered all the non-WebXR content this will break, like HTML5 games or websites that use orientation to change layout?

Websites shouldn't use the deviceorientation API to change layout. There are better ways to do that.

Yes, some games will be affected. As will things like panoramic photo viewers (not sure if they count as XR).

@reillyeon

This comment has been minimized.

Copy link
Collaborator

reillyeon commented Feb 4, 2019

With my working group chair hat on: To new folks coming to this issue, let's try to avoid piling on and making this an unfriendly place to collaborate. We have engagement from three implementers on this thread and a proposed specification change should be available soon.

@cdumez

This comment has been minimized.

Copy link
Author

cdumez commented Feb 5, 2019

Another thing that came up while discussing this internally is the need for a user gesture in order to call requestPermission(), similarly to what is done for getUserMedia. Any thoughts on this?

This would limit potential abuse.

@reillyeon

This comment has been minimized.

Copy link
Collaborator

reillyeon commented Feb 5, 2019

👍 Displaying a permission prompt should require a user gesture.

@KenanSulayman

This comment has been minimized.

Copy link

KenanSulayman commented Feb 5, 2019

Requiring a user gesture seems reasonable, although, does explicitly navigating to a site count as user gesture?

@machenmusik

This comment has been minimized.

Copy link

machenmusik commented Feb 5, 2019

A few thoughts

  • It seems clear that treating device motion and orientation events differently than other user inputs such as camera and microphone was unfortunate in hindsight.

  • It seems logical to effectively promote them to user input with respect to privacy considerations, and reuse the same affordances used for other user inputs such as camera and microphone here.

  • Ideally this would be a default stance taken to protect user privacy, as I am not sure exactly how/if this would impact generic sensors and other less established functionality at present.

@tobie

This comment has been minimized.

Copy link
Member

tobie commented Feb 5, 2019

We did a lot of work around these issues in the Generic Sensor API. It's worth having a look at the Security and Privacy section of the spec, and some specific conversations in the tracker.

@lknik and @maryammjd might have more to add.

@samuelweiler

This comment has been minimized.

Copy link
Member

samuelweiler commented Feb 5, 2019

I am glad to see that a rough consensus has emerged. If implementers want feedback on specific choices (e.g. what sort of text to include in a permission prompt, or whether to require a user gesture before prompt), you are welcome to ask the participants in the W3C Workshop on Permissions and User Consent held in September 2018. You can reach the lot of us at: public-2018-permissions-ws@w3.org

@erikmchut

This comment has been minimized.

Copy link

erikmchut commented Feb 5, 2019

Just some food for thought as progress is made here. Native apps and the web alike are moving towards more natural UX, where moving and pointing your phone is how you interact with the content. ARKit (native), ARCore (native), 8th Wall (web), and AR.js (web) are prominent examples of this.

Put another way, device orientation and motion are starting to replace user touches as input.

On native, Apple has made this very easy with ARKit.
On web, iOS Safari is about to make this harder or default-disabled.

Please consider the implications of a double-standard that indirectly benefits a closed app ecosystem.

Also, as you make changes here, do please consider the following:

  • Unlike other mobile browsers, Safari/WebKit doesn’t currently cache permissions per-domain, so any permission request occurs every single time you view the page. This contributes to permissions fatigue and users generally making less informed decisions about privacy.
  • For pages whose entire input is device motion / orientation, like many AR/VR pages, there is currently no user gesture (since canvas touches aren’t considered user gestures). Thus in the pathologically worst end-state, Safari might require a prompt for each of the following every pageload:
      1. Bespoke dialog stating ‘This page would like to use camera and device motion, click ‘here’ to request permission.
      1. Permission dialog, this page is requesting camera access (yes/no)
      1. Permission dialog, this page is requesting device motion (yes/no)
      1. Permission dialog, this page is requesting device orientation (yes/no)

While I expect that we end up with a much better state than the scenario above – If I could suggest anything, it would be to enable permissions caching at the domain level in Safari, perhaps combined with a check-box that says ‘don’t ask again for this site’. This would prevent permissions fatigue and give parity with native apps which do not need to request permission on every app load. Much of this may be addressed in future permissions APIs, but it would be helpful if Safari/WebKit teams would be open to mitigate these concerns before making changes to device motion/orientation permissions.

@lknik

This comment has been minimized.

Copy link

lknik commented Feb 6, 2019

Hi,

I wrote a bit on sensors a while ago (ambient light 1, 2).

Not sure if helpful in your conversation, but we also analysed the possible impact of changes 3 (PDF).

We all know well the debate on privacy vs usability/precision, and also the point of view of whether something is or is not a threat. In the end it boils down to individual beliefs. I've seen and shown some of the risks. While I'm not saying such attacks are always 100% reliable, I also know that quantization and frequency capping often don't work (see e.g. (1)).

And in that case I'm happy to hear the position of Apple as presented by @othermaciej
How to handle it with the "defaults" is another thing. I'm all for opt-in, the current model of the web is opt-out (with the exception of permission-gated things).

@richtr

This comment has been minimized.

Copy link
Member

richtr commented Feb 7, 2019

Replying to this comment from @geoffreygaren:

Everything being discussed is a shade of that argument that there is, with certainty, a level of default access available that provides enough privacy and security protections while not completely disabling current uses.

Can you cite some evidence to support your assertion that obfuscation, quantization, and frequency reduction solve the privacy and consent problem?

Empirical evidence for the case for obfuscation, quantization, and frequency is, I agree, important to proceed. If those approaches are capable of eliminating all privacy and security concerns then the consent problem is also taken care of implicitly. We are in the current predicament due to the lack of clarity within existing studies on the exact protections that would eliminate personally identifiable data concerns. Existing studies do not go far enough in evaluating the full effects that obfuscation, quantization, and frequency can have on eliminating personally-identifiable information from sensor data and, thus, eliminating fingerprinting. However, it remains equally disappointing to see a progressive, pro-active approach based on further investigation being dismissed in favor of a quick short-term solution.

In Bojinov-Boneh-Michalevsky-Nakibly, Section 6.1 states the following:

" In contrast to audio-based fingerprinting, there is no
good way to feed a signal into the accelerometer, namely
exert a known acceleration force; instead we take an
approach of performing background measurements and
waiting until there is enough data to estimate the accelerometer
calibration parameters. We perform a measurement
every time the phone is at a resting position, or
more precisely, the phone is at a constant velocity (no acceleration).
Note that in most reasonable cases it is very
unlikely that a phone will not be at rest for an extended
period of time. Detecting the phone is at rest is relatively
straightforward: the measured acceleration vector should
be static and its magnitude should roughly be equal to g
(gravity). "

Figure 6 in that same paper provides a scatter plot of all z-axis accelerometer data that is collected from 17 iOS devices when they are placed at rest on a table (i.e. the device has no velocity in any axis thus fulfilling the required calibration environment requirements as explained in the quote above).

All the iOS devices in that study present unique g values that fall within in the range [0.993, 1.004]. That study showed that devicemotion data is potentially fingerprintable based on those discrepancies between different devices (although many devices reported values do overlap with each other within that same range).

In a previous comment above I suggested obfuscation as follows should be added to emitted devicemotion data:

  • Add a random constant in the range ... [-0.005, 0.005] to all devicemotion data on each new invocation of the API.

By applying these obfuscation values to the data collected in that study, that data from 17 devices becomes indistinguishable from any subsequent invocation of the devicemotion API on the same device OR any invocation of the devicemotion API on any other device. Where the term "fingerprint" is defined as a means to identify and then re-identify a user on a subsequent visit due to consistencies in readings, this API would become useless for the purposes of ongoing identification of devices or users between visits.

Finally, in the same previous comment above, I continued that discussion around obfuscation with the following:

Optionally adjust that value during idle periods of inactivity for dynamic obfuscation (when e.g. the device is placed on a table).

Implementing this additional protection goes even further to prevent any fingerprinting from occurring within the same continuous, long-running stream of devicemotion data collected on a device. It does this by adjusting the obfuscation constant marginally (by a much smaller value) whenever the device is detected to be at rest (e.g. placed on a table or held still with no exerted velocity in any axis).

Finally, the presence or absence of any obfuscation constant in this data does not affect general calibration of devicemotion sensor data. Calibration of device sensors is performed under the same conditions as described above (when no velocity is present in any axis) - at which point we can estimate the calibration offset from earth's precise gravitational constant of 0.980665 (9.81 m/s^2) on each invocation.

In this comment, @geoffreygaren wrote:

Or, if the solution is as simple as you say it is, you might as well just code it up and we can test it empirically.

In WebKit, the most trivial code change would be to add a per-instance-bound obfuscation constant (CURRENT_OBFUSCATION_CONSTANT) in the range [-0.005, 0.005] to the collected gravitational values on these lines:

- totalAccel.x = userAccel.x + gravity.x;
+ totalAccel.x = userAccel.x + (gravity.x * (1 + CURRENT_OBFUSCATION_CONSTANT));
- totalAccel.y = userAccel.y + gravity.y;
+ totalAccel.y = userAccel.y + (gravity.y * (1 + CURRENT_OBFUSCATION_CONSTANT));
- totalAccel.z = userAccel.z + gravity.z;
+ totalAccel.z = userAccel.z + (gravity.z * (1 + CURRENT_OBFUSCATION_CONSTANT));

A basic patch of this type would provide enough initial protection to eliminate the majority of fingerprinting identified in existing privacy studies and specifically breaks the fingerprinting characteristics presented in the Bojinov-Boneh-Michalevsky-Nakibly paper.


By applying just 'obfuscation' initially would indicate effective removal of the ability to fingerprint devices/users via devicemotion if those fingerprinting attack vectors rely on any form of "at rest" calibration process. Other attack vectors (such as measuring the sensitivity between a series of sensor data readings), and again subject to ongoing study, could be further mitigated with quantization protections.

The entire concept of spatial computing on the web is at stake (i.e. motion as a basic input driver to rival mouse, keyboard and touch input drivers). Basic input drivers should not be subject to permission-based access controls (by all means, all other sensor APIs belong behind permissions but absolutely not coarse-grained motion and orientation sensor access).

Regarding battery life, a reduction in the frequency of event firing would materially reduce the amount of event collection and propagation required within a web browser's code. Execution of those code paths would be reduced by a factor of 50%+ (e.g. when moving from firing of events from 60 Hz to 30 Hz). Anything >= 15 Hz is still entirely acceptable for the use cases documented within this specification (i.e. non-VR/AR use cases). For use cases that require high-resolution data (at greater accuracy and delivered at higher frequencies) such as the use of motion and orientation sensor data within VR/AR scenes there are other, more suitable, APIs that should be implemented, gated with permissions and used by web developers (Generic Sensors API, WebVR API, WebXR API). Those APIs, obtained via permissions, should remain unencumbered with any forms of obfuscation, quantization, and frequency capping or limiting.

All of this is to say: we do still need more independent feedback on the effectiveness of these mitigations before pushing existing and future spatial computing use cases off a cliff. The most reasonable approach at the current time would be for Apple to implement a simple form of obfuscation as explained above in WebKit code and to also enable the "Motion and Orientation Sensors" setting by default with those protections implemented in an upcoming release. This would provide provable protections and mechanisms to eliminate the most egregious fingerprinting attack vectors, as identified in existing sensor privacy studies that require devices to be calibrated "at rest", while still allowing on-going research to continue in to additional protections that could be added on an ongoing basis in the future. This approach allows Apple and other browser vendors to continue to research, tweak and adapt sensor privacy protections in future releases while still actively preventing fingerprinting in the immediate short-term. In any other approach we are simply stating that spatial computing use cases on the web are immaterial to this discussion, spatial computing as a concept and basic input has no place or future on the web and spatial computing use cases will be eliminated as collateral damage when building more walls around access to this data instead of focussing on eliminating personally-identifiable features from the data itself. The stakes of this discussion, in the context of the ability for the web to support spatial computing, and the course of action being taken here to add permissions or otherwise restrict access to data warrants extreme caution. Nobody will stop browser vendors from their current course of action but I simply wish that the impact and loss to open motion access to the web was clearly understood in the context of the elimination of spatial computing uses (to the benefit of native applications).

@lknik

This comment has been minimized.

Copy link

lknik commented Feb 7, 2019

  • Add a random constant in the range ... [-0.005, 0.005] to all devicemotion data on each new invocation of the API.

Indeed, I'm not sure how long would that hold, considering the data range you stipulate. How many observations need to be made to discard the random noise?

@tangobravo

This comment has been minimized.

Copy link

tangobravo commented Feb 7, 2019

@richtr - Although I agree that fingerprinting concerns are likely to be solvable with some of the mitigations you mention that is not the only potential privacy concern.

To take one example; even heavily quantized, obfuscated, and reduced-frequency sensor data would give some information about whether the user is standing up or lying down. Whether that is a significant enough privacy concern to require explicit user consent is certainly debatable (most vendors appear not to think so at the moment), but it does appear Apple / WebKit have made their minds up here.

I believe spatial computing use cases on the web would survive a permissions prompt. It would be unfortunate however to have a widely-released iOS version with the API disabled by default and without a permissions API implemented.

So I still hope for the final iOS 12.2 release Apple will either have an opt-in permissions API implemented or will decide to flip the default setting until that permissions API is ready. To me that strikes a good balance between addressing privacy concerns but without being quite as hostile to web developers who depend on these features.

@richtr

This comment has been minimized.

Copy link
Member

richtr commented Feb 7, 2019

@lknik wrote:

  • Add a random constant in the range ... [-0.005, 0.005] to all devicemotion data on each new invocation of the API.

Indeed, I'm not sure how long would that hold, considering the data range you stipulate. How many observations need to be made to discard the random noise?

This is the purpose of quantization to sit above random noise thresholds. As I wrote:

Other attack vectors (such as measuring the sensitivity between a series of sensor data readings), and again subject to ongoing study, could be further mitigated with quantization protections.

@lknik

This comment has been minimized.

Copy link

lknik commented Feb 7, 2019

@richtr yep, it's catchy but choose the ransom noise level needed. And thanks @tangobravo for resurfacing that privacy is not merely about fingerprinting. Actually many of the risks I considered do not relate to fingerprinting at all. Sensors is such a rich data source...

@neave

This comment has been minimized.

Copy link

neave commented Feb 8, 2019

@cdumez @reillyeon @othermaciej @anssiko What is the process now for agreeing upon an API and adding it to the official spec? I think most of us would like to see this discussion moved on and to implementing a solution for developers.

@anssiko

This comment has been minimized.

Copy link
Member

anssiko commented Feb 8, 2019

Procedural steps need to be taken before a contribution can be added to a W3C WG spec from outside the group's current participants. This process is on the fast track and involved parties are informed.

The chairs will update this issue when we have new information to share. Meanwhile, technical discussion can continue on this issue. I can assure you everyone is highly motivated to resolve this issue. Thank you for your understanding.

@machenmusik

This comment has been minimized.

Copy link

machenmusik commented Feb 8, 2019

In reply to:

The entire concept of spatial computing on the web is at stake (i.e. motion as a basic input driver to rival mouse, keyboard and touch input drivers). Basic input drivers should not be subject to permission-based access controls (by all means, all other sensor APIs belong behind permissions but absolutely not coarse-grained motion and orientation sensor access).

My view is that the reason why we are looking for a different solution than default-enabled exposure is precisely because, for the browsers in question like Mobile Safari, the spatial computing aspect is not likely to be a primary function, and therefore this data is not likely to be a primary input source.

As an example, if Avaer were to decide that ExoKit is intended to be an immersive browser with spatial computing as a primary function, then ExoKit could decide to default to enabled for https sites or localhost (or perhaps to any site displayed in immersive mode, if that were too restrictive).

My intuition is that obfuscation of the data is at this moment an uncertain solution, and that a more intentional permission model will be needed, with browsers ideally remembering site permissions where appropriate to reduce user friction.

@nuthinking

This comment has been minimized.

Copy link

nuthinking commented Feb 11, 2019

With all due respect, it looks like a lazy solution to me. Please let’s stop showing people popups and let’s find instead ways to prevent this information to cause any harm. Haven’t we learnt anything from the cookies banners? Advertisers will track everything. If you don’t want websites to know your device orientation, stay away from the Web. MUCH WORSE THINGS HAPPEN OVER THERE! ✌️

@machenmusik

This comment has been minimized.

Copy link

machenmusik commented Feb 12, 2019

@nuthinking If you scroll upwards, you will see that there are several discussing work to mitigate potential harm, but also significant discussion that current techniques may not be sufficient. That discussion has been going on for some time, and appears likely to continue for some time.

In the meantime, I think it makes sense to do something additional to mitigate, and the obvious choices are to either disable by default with no interactive reprieve (that entirely kills the use cases, which is why folks are looking for other solutions here) or to treat like other sensitive inputs with interactive opt-in (camera, microphone). Those inputs cannot generally be degraded or rate limited without significantly impacting their primary use cases, and I suspect the same may prove true here.

@nuthinking

This comment has been minimized.

Copy link

nuthinking commented Feb 12, 2019

@machenmusic I am not a security expert and I appreciate being conservative when it comes to security. I would just hope the discussion was focused on the security aspect. Why having throttling, or a way to temporary disable the feature, for instance, while typing in a textfield marked as password, wouldn’t work? I don’t buy the privacy issues. That data Is not as exploitable as mic and camera. Should we ask permissions to get the mouse positions between clicks then? Can we clearly state the problem we want to solve? Not the solution.

@machenmusik

This comment has been minimized.

Copy link

machenmusik commented Feb 13, 2019

@nuthinking this has been discussed above - this is one of the longer threads that I can remember, actually - and I think the browser implementers (i.e. the ones with exposure risks and real skin in the game) have gotten at least as far as #57 (comment).

@nuthinking

This comment has been minimized.

Copy link

nuthinking commented Feb 13, 2019

Privacy hole... c’mon. You can probaly tell if someone is in bed by how they scroll the device. So what? Who defines the privacy line? Safari is disabling the feature and user will have to enable it in the settings (few gamers might do it).

@reillyeon

This comment has been minimized.

Copy link
Collaborator

reillyeon commented Feb 13, 2019

This discussion is becoming unproductive. I am locking the conversation until a PR for the proposed API changes has been posted.

@w3c w3c locked as too heated and limited conversation to collaborators Feb 13, 2019

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.