Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Detecting a mouse user #869

Closed
paulirish opened this Issue · 100 comments
@paulirish
Owner

Let's use this ticket just to explore the ideaspace. Basically... is there a way to detect a mouse user?

just random notes on detection in this area:

touch evts & input exclusivity

Can't make assumptions about input anymore...

  • Just because touch events are present doesn't mean the user is primarily using a touchscreen. Or that there is no mouse.
  • Just because there are touch events supported doesn't mean :hover styles will never be triggered.
  • Just because there is a mouse doesn't mean people can't also touch the screen.
  • ~33% of iPad users with VoiceOver enabled use an attached keyboard. (@stevefaulkner said this, iirc)

http://www.html5rocks.com/en/mobile/touchandmouse/ by @cwilso is a must read for this space.

(and when this ticket says "mouse user" it means currently using a mouse but may mid-session switch to touch)

mousemove

  • mousemove doesnt fire as a touch events user uses the page...
  • however it does fire in this sequence when a user taps the screen
  • so maybe we can test the difference between the two?
    • evt.isTrusted from DOM 4 events may be interesting. Implemented in FF, but not in Webkit. isTrusted is kinda untrusted right now though
      • hypothetically it could tell the diff between the simulated mousemove and real one
    • in my tests: evt.which was 0 in real mousemove and 1 in fake mousemove (part of the tap). Why? I have no idea.
    • the first time it fires, does mousedown fire <50ms after mousemove fired? (following the tap evt sequence) downsides galore: race condition. yucky detection, but .. might be worthwhile?

:hover

:hover styles are never triggered by a touch user except for a brief moment during a tap.

mouseenter / mouseleave


cc @robcolburn @roblarsen

Anyone please add any notes here that aid in the pursuit of detection of mouse users. Differences in event objects, user behavior heuristics, etc.

@paulirish paulirish referenced this issue
Closed

No-Touch #855

@dmethvin

in my tests: evt.which was 0 in real mousemove and 1 in fake mousemove (part of the tap). Why? I have no idea.

On Chrome, with a mouse, evt.which does report the active buttons during a mousemove. Firefox and IE always report 1 during real mouseover unfortunately, and IE10 reports 1 for its fake mouseover during touches as well. http://jsfiddle.net/jFAdL/

@neilcarpenter

Have been using presence of window.MouseEvent constructor to test for mouse input (not exclusively mouse input though, just the existence of mouse input), seeing as you don't mention this can I assume I am way off?

@ekrembk

Detecting a mouse user and changing the mouse user status mid-session will require listening to an event forever. I'm not sure if this is a good idea.

I think looking for event constructors (as @neilcarpenter said) and defining the status like 'only-touch', 'only-mouse' and 'touch-and-mouse' is a better idea.

@CLowbrow

@neilcarpenter If I pull up and run http://tinkerbin.com/IOeIeueT on my iPad with nothing paired to it, the window.MouseEvent object exists.

This feels like it gets really complicated when you start thinking about surface pros. When the page loads, you could potentially have no mouse input. But then you could attach the smart cover with the touchpad and start having it.

:hover feels like the right way to go here. Would be interesting to see how reliable it could get.

@roblarsen

With the Galaxy Note II you can see two different behaviors with :hover styles.

  1. With the pen out, :hover behaves exactly like a mouse as the pen (literally) hovers over the screen.
  2. With the pen in, using a finger, it behaves the way the HTML5Rocks article describes, triggering the :hover class until the next tap.

The Note II also returns true for window.MouseEvent with or without the pen detached.

I've got a Yoga coming this week and I've got the oddballest phone in the world, so I'm primed to play around with this stuff.

@CLowbrow

Ok, I have something pretty basic working on my pc/ipad. I fully expect this to break on other tablets, but I don't have those at home with me.

FIddle: http://jsfiddle.net/4NpV8/7/
For fullscreen: http://fiddle.jshell.net/4NpV8/7/show/

It's kind of crappy and you have to wait for someone to move their mouse or touch the page for it to work.

@frankmarineau

I know on Chrome for Mac, when I plug a mouse in my Macbook Pro, Chrome will make the scrollbar always visible, instead of having it look like an iOS scollbar. I don't know if this means there could be some way of finding out if the user uses a mouse from within the engine or if it is just is within Chrome itself, but nonetheless I think it would be a great feature. Were you thinking of including multi-touch trackpad users as "mouse" users or only users using a physical mouse ?

@SamuelEnglard
@fryn

mouseenter & mouseleave were implemented in Firefox 10: https://developer.mozilla.org/en-US/docs/DOM/DOM_event_reference/mouseenter

@mediastuttgart

what about mousewheel, delta & co.?

@SlexAxton
Owner

So far this has been working for me:

Modernizr.mouse = window.confirm("Click OK if you have a mouse, and CANCEL if you don't. No lying.");
@somatonic

Throw a cheese :P

Sorry, but this whole hover/click/touch issue is killing me currently on a project.

@matijs

Building on @SlexAxton's solution, how bad would it be assume a user wants to use touch when we find touch, but in addition to that, we allow them to override this with something like a "I have a mouse and prefer to use it" link/control in the same way mobile-only (yuck!) sites sometimes provide a "take me to the desktop version" link?

@stucox
Owner

@somatonic - if it's "killing" you, I suggest you look at a UX solution. To me, there are currently 3 ways to approach it:

  • Cater for all simultaneously: assume fat fingers, assume no hover, assume no touch gestures; treat each of those as an optional benefit/enhancement
  • Make assumptions which you believe represent your user base: e.g. that large-screen users will use a mouse, or that all users will be on a touch device — of course this option will exclude some users and probably isn't very future-proof
  • As @matijs suggests (although not as poetically as Alex), provide a UI control to switch — I showed a really simple demo of this in my recent article on the subject

But... less of the railroading. How about we keep this thread for discussing how we might be able to automatically detect? Rather than how to handle the UX if we can't.

@robcolburn

Questions (a testing todo list)

  • What is the behavior of :hover on the touch&mouse devices (Surface / Pixel)?
  • - Do they behave normally with a mouse?
  • - Do they behave quirky with a touch (like mobiles)?
  • Can you successfully browse the web on a touch device without touching (using the keyboard)?
  • Do you trigger mouseover/mousemove/mouseleave, when navigating the page with a keyboard? Is that consistent across mouse devices as well as touch&mouse?
  • Are there any giveaways in touch-based mouse-event? Perhaps, changedTouches is attached to the events? Do these quirks occur on touch&mosue?

Thoughts

  • Mouse users pretty much guarantee themselves a mouseover on the document, right?
  • Touch-only users will fire a touchstart before they ever fire a mouseover, right?

Theory

  • If PointerEvents exists, then "has-mouse" and "has-touch".
  • Else, Set up a listener on document for touchstart and mouseover
  • - If mouseover occurs first, flag "has-mouse", and unbind both listeners.
  • - If touchstart occurs first, flag "has-touch", and unbind both listeners.
  • Does that mess with scrolling?
@roblarsen

Can you successfully browse the web on a touch device without touching (using the keyboard)?

These devices are just laptops with a touch screen, so yeah, you can browse the web as far as the accessibility of the visited sites allows with no mouse and no touch.

@robcolburn
  • Can you successfully browse the web on a touch device without touching (using the keyboard)?

In this case, I'm curious about to iOS or Android devices with attached physical keyboard. I believe mobiles generally aren't designed to be accessible to low-vision users, but I'd be delighted to be wrong too.

@stucox
Owner

@robcolburn:

  • If PointerEvents exists, then "has-mouse" and "has-touch".

Surely IE10 for Windows Phone 8 supports PointerEvents, but will very rarely have a mouse? And if WebKit implements PointerEvents then they'd certainly like to leave the API enabled regardless what devices are available (this is what they did at first for TouchEvents, but too many developers had assumed that TouchEvents == touch device, so now only enable the API if a touchscreen is detected... which isn't ideal)

@CLowbrow

@robcolburn This sounds like what you are talking about: http://jsfiddle.net/4NpV8/55/show

  • It does not block scrolling because there is no e.preventDefault.
  • Touchmove needs to be bound to as well. Touchstart only triggers once in some cases.
  • Won't work in IE because those touch events have different names.
@stucox
Owner

@robcolburn - http://vimeo.com/51952723 - talk on mobile accessibility... apparently mobile devices are better for accessibility than most desktops. These stats claim that 32% of mobile users with disabilities use a connected keyboard (over 50% of the users surveyed identify as blind).

@stucox
Owner

@CLowbrow:

  • Won't work in IE because those touch events have different names.

Personal peev of mine :tongue: — it's a completely different spec (potentially superior), not just "different names".

@patrickhlauke

possibly a counter question to paul's thread starter: WHY would you want to detect if it's a mouse (rather than touch) user? the answer to that may help further shape the answers, i think.

is it primarily because of the "touch and :hover aren't best friends"? in that case it's worth noting that iOS Safari handles :hover stuff quite nicely in that it stops the event sequence (e.g. touch events > simulated mouse events culminating in a click) if it sees that there's been a change in the document or its presentation (e.g. if a :hover style kicked in and did something radical like display:none > display:block). see figure 6-4 on http://developer.apple.com/library/IOS/#documentation/AppleApplications/Reference/SafariWebContent/HandlingEvents/HandlingEvents.html

and if so, perhaps it's more a case of "how can we make sure :hover type stuff also works with touch and keyboard" rather than the other way around? i naively hacked something together for a presentation last year http://www.slideshare.net/redux/getting-touchy-an-introduction-to-touch-events-web-standards-days-moscow-24112012 (slide 33)

Pointer Events are perhaps nicer here, as the first time you handle an event you can work out what type of input generated it with the PointerEvent.pointerType attribute. But doing it before any event was fired at all may be tricky/inconclusive.

@matijs

@patrickhlauke the way iOS handles this was (is) mostly so existing things on the web don't break though, wasn't it?

@patrickhlauke

@matijs yes, and the potential breakage came from developers naively assuming that :hover style interactions were great in all situations (they weren't for keyboard users anyway, unless carefully doubled-up with :focus or extra JS, but devs didn't always care about accessibility in that case ... only once touch devices emerged did they start to realise the problem). to be clear: i'd favor being input-modality-agnostic in general, and then patching in extra stuff (like :hover-based things) as additional/alternative functionality/eye-candy...

@patrickhlauke

additional thought: the whole "simulated mouse events" part of event handling of touch events is arguably also done so "existing things on the web don't break".

@henriquea

I think @patrickhlauke made a good point about what is the problem we're trying to solve. If we're trying just to solve the :hover issue on touch devices, then should mobile web browsers just ignore the :hover styles?

@matijs

@henriquea that would ‘break’ stuff like li:hover > ul { display: block; } that depends on a :hover to show a nested <ul> that would otherwise be hidden by a display: none; for example.

@patrickhlauke

@matijs that's already bad practice for keyboard users, but yes we should aim to patch these issues (for both kbd and touch users) for better input-modality-indipendence.

@stucox
Owner

Could we better clarify exactly what we mean by "is a mouse user"? @paulirish said:

when this ticket says "mouse user" it means currently using a mouse but may mid-session switch to touch

... does this mean "the current/last event came from a mouse", or "user is likely to use a mouse for their next interaction"? I think the former is pretty much covered in the HTML5 Rocks article, the latter will inevitably take some guesswork.

@patrickhlauke said:

WHY would you want to detect if it's a mouse (rather than touch) user?

Maybe it doesn't matter. Sometimes giving creative people tools, even without full knowledge of how they're going to use them, can result in awesome things (it can also result in terrible things, of course).

That said, some examples of how you could take advantage of knowing if the user has the capabilities a mouse offers:

  • Hover:
    • No hover: put a help icon icon next to a UI element, which displays help text on click
    • Has hover: expose help text via a tooltip on hover and hide icon to reduce clutter
  • Accuracy:
    • Poor accuracy: finger-sized icons in a toolbar, maybe with multiple levels / menus
    • Good accuracy: make icons smaller so you fit more in a toolbar/menu → fewer clicks to do things
  • Multiple buttons: (alternative actions on a single contact point)
    • Single button: provide a button in the UI for e.g. a "drag and copy" action
    • Multiple buttons: hide the button and implement "drag and copy" via middle click

I appreciate the last case here is a bit of an unlikely one...

With any of these, you could "assume the worst" and cover all eventualities, but if they can be detected then there's the potential to provide a better, cleaner interface for users with these capabilities.

If we can detect these specific capabilities, rather than just "is a mouse", then that makes the designer's/developer's toolbox even more powerful and future-proof — a hover-capable stylus and probably Leap Motion would be able to take advantage of the "has hover" UI enhancements, a stylus would also be able to take advantage of the "good accuracy" UI etc.

There is of course still the multiple-input-device case to consider...

@patrickhlauke

@stucox great stuff, and just to be clear i'm not particularly against the idea of detecting particular inputs, just that i've seen enough bad usage in the past already regarding mouse-only, non-keyboard-accessible abominations that i just want to avoid a repeat here ;)

i like the examples you provide, but as you also note yourself, what happens if the user switches mid-session from one input method to another? difficult to determine a priori what they're going to use (which seems an issue the CSS Media Queries 4 peops are now bumping against as well...there, they seem to concentrate on "primary input device", which is a red herring in my view - what's the "primary" input on a win8 laptop with a touchscreen, trackpad, keyboard and possibly even an external mouse and a stylus for instance?)

@patrickhlauke

had to laugh, as i just now stumbled across http://www.stucox.com/blog/the-good-and-bad-of-level-4-media-queries/ ... so yes, teaching @stucox how to suck eggs, it seems...sorry ;)

@paulirish
Owner

@patrickhlauke
i know this might seem like solution in search of a problem. MAYBE it is..but i think we're close and have a possibility of figuring out a decent detect.
and I feel this thread is a necessary dependency to the "Definitive Guide to Customizing Web Experiences for Singular and Varied Input Devices", which is very necessary and partially still unwritten

@CLowbrow

@paulirish can you be more specific on what would make a detect good enough?

For example: does it have to be able to run without the user triggering events?

@stucox
Owner

Let's leave use cases for now then. I actually don't think they're going to make any difference, because we're so limited in what we can detect.

So to recap — (let me know if you disagree with any of these):

  1. We want to detect the presence of a mouse
  2. With the exception of @frank611's suggestion re. scrollbars, we probably can't detect before an event is fired
  3. As such, what we're detecting is if a mouse has been used in this session — it won't be immediately from page load
  4. We probably also can't detect that there isn't a mouse — it'd be undefined until true (I think this makes more sense than setting it false until proven)
  5. And we probably can't detect if a mouse is disconnected mid-session — that'll be indistinguishable from the user just giving up with their mouse

mousemove is definitely the starting point: it's pretty much always the first event fired on a document when a mouse is in use.

So I think out best option is to simply detect two mousemove events without a mousedown occurring between them:

// Do we want to do something dodgy like try and target window.top in case
// it's in an iframe?
var mightBeMouse = false;
window.addEventListener('mousedown', function () {
    mightBeMouse = false;
});
window.addEventListener('mousemove', function () {
    if(mightBeMouse) {
        Modernizr.addTest('mouse', true);
    }
    mightBeMouse = true;
});

@paulirish - can you give this jsFiddle a go on your Chromebook Pixel?

This may sounds like an odd choice, but a touchscreen tap fires mousedown immediately after mousemove, so in the case of a "false positive" will set mightBeMouse to false with practically no chance of simultaneous taps interfering (simultaneous taps seem to block the usual tap event sequence anyway and just fire a load of touchstart/touchend, on Android 2.x and Safari/Chrome on iPad at least).

This does mean the mouse needs to move enough to fire mousemove twice, but it's actually really difficult to get it to just fire once. I think we'll basically never miss it when a user's using a mouse normally.

Does this correspond to "has hover"? Probably... unless contact styluses (i.e. ones which can't hover) also fire consecutive mousemove events — anyone know? But whether that'll be remain the case for all possible future devices I have no idea.

@stucox
Owner

I'm interested to explore @frank611's idea though, in case we can get it to resolve before any events fire on some platforms...

@patrickhlauke

random thought: how about using a "if a mousedown/mousemove/mouseup/click type event happened less than a 500ms after a touch event, assume it's part of the simulated mouse events and leave mightBeMouse false, otherwise set it to true" approach? (sorry, my brain currently too frazzled for actual code)

this could still cause false positives/negatives of course.

@paulirish sure, i don't mind that we're trying to find a way for sites/apps to be more aware of their environment. just seems to me, though, that because users may switch inputs altogether anyway (what happens if halfway through using an app on an ostensibly "touch only" device i pair a mouse via bluetooth, or plug one in via USB), this stuff would be extremely brittle at best without an actual API (a la gamepad API perhaps?)

@stucox
Owner

@patrickhlauke - I think setting mightBeMouse to false when if a mousedown is encountered achieves the same as that, without the danger of the timeout... or have I misunderstood you?

I agree with you re. brittleness... but as I say, give people tools and see what they make with them. We'd certainly need to declare all of these caveats, probably with some blog posts, tutorials etc.

@paulirish
Owner
@paulirish
Owner
@roblarsen

IE10 on Windows 8 (with the exception of Modernizr being blocked for Mime Type issues) appears to work as expected.

I'm about 30 minutes into working with this machine so I'll check out some other browsers as I get them installed.

@stucox
Owner

Er yeah sorry, I'm cheekily serving it from a GitHub Gist there...

@roblarsen

no worries. I worked around it for now.

@stucox
Owner

Cross-browser-ified and fixed GitHub Gist link (with rawgithub.com - cheers @paulirish): http://jsfiddle.net/zUCBM/11/

@roblarsen

The above works as expected in Chrome and the Android browser on the Note II. The stylus is recognized as a mouse.

@stucox
Owner

@roblarsen - awesome, thanks. I'm really keen to see how contact styli react... you haven't got access to such a primitive piece of tech, have you? :)

@roblarsen

@stucox I don't. At one time I did (I was a Palm guy until right before they sold out) but they've all been recycled.

@patrickhlauke

@roblarsen what happens if you pair, say, a bluetooth mouse to the Note II ? from my early testing ages ago, it seemed that a mouse - though it brought up a visible mouse pointer on screen - still fired touch events first, then the simulated mouse events...interesting to read that stylus is indeed handled differently (unless that's a Note II specific enhancement?)

@roblarsen

@patrickhlauke I'm not sure. I'll try to get my hands on one to test it. I'm curious myself, since the concept of hover and mousemove on the Note is hovering mm in the air above the screen/moving he stylus about mm above the screen. A Bluetooth mouse might be treated differently.

@matijs

Just paired a bluetooth mouse with my Nexus 7 and a very quick test shows that this works as expected in Chrome. However :hover doesn't work… curious if it does on the Note II

@patrickhlauke

testing with a galaxy nexus phone and a bluetooth mouse, it seems that the android mouse (in chrome at least) fires lots of mousemoves (without a preceding mousedown). when actually pressing the button, it does a
touchstart > touchend > [mousemove]+ > mousedown > mouseup > click

and to be explicit: @stucox's test works fine in this situation

@stucox
Owner

@matijs - very helpful, thanks. Was the mouse connected before the browser started? I know Chrome does some enabling/disabling of APIs on start-up.

@patrickhlauke - does it trigger :hover styles?

@patrickhlauke

@stucox yes, it does (used a rather janky old test page for it https://dl.dropbox.com/u/48771/maus-und-touch-events.html and the :hover is tied to a div)

@matijs

@stucox loaded the fiddle with bluetooth turned off and did a bit of tapping without anything happening (as expected), switched to settings and turned on bluetooth, switched back to the browser, moved the mouse and BAM background: #0c0

@stucox
Owner

Updated fiddle which also reports if :hover styles are working and mouseover events are firing: http://jsfiddle.net/zUCBM/24/

You haven't got a stylus for your Nexus 7 have you @matijs ? I'm hoping that doesn't fire on this test (it shouldn't do, because it should just treat it like a finger touch).

@CLowbrow

@stucox that should be correct. The only styli that I can think of that would cause different behavior:

  • The ones on galaxy notes (previously mentioned in thread).
  • Microsoft surface.
  • This thing: http://tenonedesign.com/connect.php (my money is on browsers treating it like a finger, though)

Anyone besides Samsung/Microsoft putting styli on things?

@matijs

@stucox all true, true, true and a green background :/ and no stylus with the N7. Will test some more over the weekend.

@stucox
Owner

@matijs - that's cool, I think that's what we're after.

@CLowbrow - Wacom graphics slates etc?

@patrickhlauke

regarding stylus and nexus 7s...i suspect (though happy to be proven wrong) that after-market styluses simply act as a touch, rather than being recognised as some other type of input?

@patrickhlauke

@stucox the updated jsfiddle now reports true for :hover and for mouseover when just using touch. and as @matijs already confirmed, loading a fresh instance of chrome with no paired mouse, loading the page, THEN pairing the mouse also works fine, i.e. it doesn't adversely influence the test result.

@stucox
Owner

@patrickhlauke - yep, that's expected. I added those to ensure that the test wasn't giving a false-positive... i.e. if we're giving a positive result for "hover" (which is essentially what this test is now), I wanted to make sure mouseover events were actually being fired and :hover styles were being observed.

I don't think there can be such thing as a "negative" result with this test: just undefined and true... so mouseover and :hover occurring on touch shouldn't be considered a false positive.

I'll try and word this a bit better in my write-up — new discussion ticket incoming.

@stucox stucox referenced this issue from a commit in stucox/Modernizr
@stucox stucox Added detect for hover-capable input devices (see #869) a8127ba
@stucox
Owner

Thoughts on #873 please people...

@stucox
Owner

In the meantime... any other ideas of anything we could detect with a similar pattern, or ways we could improve this test at all?

Detecting touch has got to be pretty easy, now that we're considering "wait for an event" patterns... (although there may well be a longer delay before any events fire)

@patrickhlauke

"Detecting touch has got to be pretty easy" set it after the very first touchstart should be all that's required, no?

what about detecting keyboard use/support via keydown? or do virtual keyboards also generate keydown (sorry, lazy and haven't tested it myself)

@patrickhlauke

just tested the jsfiddle on an asus notebook with touchscreen and win8, and i'm getting some funky results. in both firefox and chrome, if i just gently tap into the results part, it works fine...but as soon as i slightly move the touch point, it registers it again as a mouselike. this may be due to some weird handling in win8?

@patrickhlauke

testing the events being fired a bit more, it seems that firefox fires a weird series of additional mousemoves (but not always consistently) in win8

touchstart > [touchmove]+ > mousemove > mousedown > [touchmove > mousemove]+ > touchend > mouseup > click

chrome is even more erratic, but i've seen stuff like

touchstart > touchmove > touchend > mousemove > mousemove > mousedown > mouseup > click

(note the two consecutive mousemoves)

@stucox
Owner

Interesting, thanks... looks like it needs some more thought there then.

@CLowbrow

Based on that: it seems like we are only safe to detect for a mouse after mouseup:

touchstart <- entering fake mouse event zone, stop listening for mousemove
touchmove <- stop listening for mousemove
touchend
mouseover <- can't trust
mousemove <- can't trust
mousedown <- can't trust
mouseup <- can't trust, can start listening for mousemove
click

@stucox based on your previous jsfiddle, would it work to default mightbemouse to true, and kill it on the first touch event?

I hope this makes sense: http://jsfiddle.net/vBstW/2/

The downside of this is that if a touchstart/touchmove event happens without triggering a click, we will never know it's safe to listen for mouse events until a mouseup happens somehow.

@CLowbrow

You're right. I think that's why I had it reset on click in previous fiddles. I need to take better notes. My ipad is consistently triggering mousedown, though.

It is a pretty likely case, unfortunately.

@stucox
Owner

I deleted my previous comment because it was a mess :) Patrick's test page fires all the usual mouse events on my iPad, so I think I must have made a mistake on the test page I was using.

Still, this case:

touchstart > touchmove > touchend > mousemove > mousemove > mousedown > mouseup > click

is indistinguishable from a touch with a very short drag, followed by a mouse move and click. Oh Windows.


Can we actually take a more straight-forward approach? "Hover" is moving without contact, right... so:

var hasContact = false
// Any of these indicate the start of contact
$(window).on('touchstart mousedown mspointerdown', function () {
    hasContact = true;
});
// Any of these indicate the end of contact
$(window).on('touchend mouseup mspointerup', function () {
    hasContact = false;
});
$(window).on('mousemove mspointerhover', function (evt) {
    // MSPointerHover categorically means a contactless interaction
    if(!hasContact || evt.type.toLowerCase() == 'mspointerhover') {
        Modernizr.addTest('hover', true);
    }
});

yetanothereffingjsfiddle.com

Looks like it would cover the Win 8 cases above. A contact stylus may well do something like this though:

mousemove > mousedown > mousemove+ > mouseup > click

which would false-positive — and I think the only way around that would be to ensure a qualifying mousemove isn't very shortly followed by a mousedown... might mean we miss a couple of triggers, but that's unlikely and it'll fire eventually.

Putting 2 fingers down then lifting one up would set hasContact = false even though there's still contact — although the touch devices I've tried (iPad + Android phones) don't fire a mousemove when multiple contact points are involved... not sure about IE10 with Pointer Events — this suggests it might still fire them:

Mouse compatibility
After firing pointer events, Internet Explorer 10 fires mouse events for the primary contact (for example, the first finger on the screen). This enables existing websites that are based on mouse events to continue to function.

Can anyone test this for me? Otherwise we might have to count pointers down and up...

@patrickhlauke

i know modernizr has so far avoided touching (hah) on the pointer events stuff in some of its tests, but for interest i hacked up https://dl.dropbox.com/u/48771/pointer-mouse-events.html and tested the sequence in which pointer events and simulated mouse events (unless prevented via CSS touch-action property) are fired:

IE10

mousemove > MSPointerOver > mouseover > MSPointerDown > mousedown > focus > MSPointerMove > mousemove > MSPointerUp > mouseup > MSPointerOut > mouseout > click

Chrome build with Pointer Events support (http://appendto.com/blog/2013/02/prototype-chromium-build-with-support-for-ms-pointer-events/)

pointerover > pointermove > mouseover > mousemove > mousedown > pointerdown > pointerup > mouseup > click

also, possibly of interest for our hover discussion: "IE10 adds new behavior to the existing aria-haspopup property to simulate hover [on touch]" http://msdn.microsoft.com/en-gb/library/ie/jj152141(v=vs.85).aspx (though i've not had a chance to test this)

@stucox
Owner

i know modernizr has so far avoided touching (hah) on the pointer events stuff in some of its tests

There's a detect for Pointer Events actually — what we avoided was including it in the Touch Events detect, because they're different things.

Thanks for the results. I take it IE10 false-positives for my last version then? How about this one (2 consecutive contactless mousemove events)?

Interesting about aria-haspopup. I wonder if that behaviour could be shimmed for other touch browsers? A topic for another day maybe...

@patrickhlauke

lost track a bit of what we're now actually testing, so my answer a bit more explicit:

http://jsfiddle.net/tB32y/10/ in IE10 with touchscreen: when i first touch the results bit, it turns green, hover:true, and contact:true when my finger is on the screen but stationary...as soon as i move, it goes to contact:false

http://jsfiddle.net/tB32y/13/ : doesn't turn green, hover remains undefined, contact:false unless i keep finger stationary (as above)

@stucox
Owner

That matches what I thought, thanks very much for your help.

To clarify: the latter test looks for 2 consecutive mousemove events which don't involve contact with the screen... so hopefully is a more reliable test for hover capability.

@patrickhlauke

for extra fun, dusted off my old blackberry playbook (updated to OS 2.1.0.1526 with browser based on AppleWebKit/536.2)

http://jsfiddle.net/tB32y/10/ - contact:true when touching screen, hover:true after a tap or two and turns green
http://jsfiddle.net/tB32y/13/ - contact:true when touching screen, hover:undefined and doesn't turn green

the events fired by the blackberry playbook browser seem a bit erratic at best. a clean tap gives me (most of the time):

touchstart > mouseover > mousemove > mousedown > touchend > mouseup > click

doing a small swipe usually results in:

touchstart > mousemove > mousedown > touchmove

without any touchend either. shrug

@RByers

See http://dev.w3.org/csswg/mediaqueries4/#pointer and http://dev.w3.org/csswg/mediaqueries4/#hover. Chrome has some support for this, and we're adding full support. Hopefully this API is part of the long-term solution for the problems raised here.

@patrickhlauke

In my view, CSS4's new media features are only part of the solution, since they rely on the "primary"/"least capable" device's capabilities (although, they still leave out keyboard users). This info can inform certain design decisions, certainly, but similar to the issues discussed here, the availability of some capability does not - in a multi-input situation - imply that users will actually use an input mechanism that can take advantage of them.

@Integralist

I'm not sure this has been mentioned yet (apologies if it has), but it seems like most people are focusing on the technical aspects of implementing a mouse detection after the page has loaded (e.g. should our event listener be set to listen for click, or touch or pointer) but it seems no one is considering the alternative scenario of why some devs want to test for mouse support...

For the responsive BBC News site our UX team want to load up styles that incorporate larger margins and button hit areas (amongst other styles) for users of touch based devices but for pointer based devices - which have more fine grain control over their clickable areas - load up tighter margins/padding & button click areas etc.

At the moment we're defaulting to loading up touch based styles but ultimately it would be good if we could detect if a user has a pointer device 'enabled' (not just supported, but actually a way to tell if it's in use or been in use) so we could decide (before the page renders) what stylesheets to load.

I think ultimately though most UX teams will need to consider the real possibility that even if we can detect that a mouse (or other pointer) is plugged into a device, that still doesn't mean the user is interacting with the page via the pointer but could very well be using touch gestures -> which means we'll need to have some kind of message area to let the user choose a style of page display they want.

It's all a bit sucky, but I think this is another area - much like responsive design - where we need to give up our (imagined) control.

@stucox
Owner

@Integralist — there are some thoughts on the why (rather than the how) on this thread: #873

The how we've arrived at on this thread does actually rely on user input, which to some extent implies the the user is using a mouse (well, actually a hover-capable device at the moment); but it's far from foolproof. If I'm on a Chromebook Pixel and accidentally nudge the trackpad, and my interface goes all fine-grained when I'd rather use the touchscreen, I'll be seriously pissed off.

It's all a bit sucky, but I think this is another area - much like responsive design - where we need to give up our (imagined) control.

+1000. Totally agree. Designers (and by extension their users) are going to have a much better time if they just accept that they don't know.

@Integralist

@stucox

accidentally nudge the trackpad, and my interface goes all fine-grained

...exactly what we want to avoid :-)

We want to load up the best possible experience from the get-go, if the user changes their interaction device after we've loaded up the UI then we wouldn't want to disrupt the experience by jump back and forth between fine-grain and more padded styles -> that would just be a bit silly :-)

@patrickhlauke

"I believe that the answer – as in so many other cases in web development – is to accept that we can’t fully detect or control how our users will interact with our web sites and applications, and to be input-agnostic."

https://hacks.mozilla.org/2013/04/detecting-touch-its-the-why-not-the-how/

@Paul-Browne

I'm a few months late to the party, but couldn't resist sharing my 'wisdom'

IMO one of the stumbling blocks is that even though we are only dealing with 2 inputs, mouse & touch, we actually have SEVEN combinations. And with a little help from a simple Venn diagram, I'll show you what I mean

venn diagram

Consider the above where;
A = mouse, touchpad, trackball, those Wacom Bamboo tablet things... pretty much anything that creates a cursor, which is controlled by something other than touching the screen.
B = Touchscreen; self explanatory, you touch the screen and something happens, other than leaving a dirty finger-print (also includes styluses).

Now, the 7 combo's and their device range would be

  1. A - laptops, desktops, chromebooks and tablets/smartphones with a USB mouse.
  2. B - smartphones, tablets, chromebooks and tablets/smartphones with a USB mouse.
  3. A but not A&B - laptops and desktops. Any device with even a hint of a touchscreen is excluded.
  4. B but not A&B - smartphones and tablets. No mice allowed.
  5. A + B - Everything, as long as it has a mouse or touchscreen or both.
  6. A + B but not A&B - laptops, desktops, smartphones and tablets, but not devices with both.
  7. A&B - Only chromebooks and tablets/smartphones with a USB mouse.

In a perfect world sigh to target these sections we would simply write...

  1. if ('MouseEvent' in window) { $('some').script(); }
  2. if ('ontouchstart' in window) { $('some').script(); }
  3. if (('MouseEvent' in window)&&!('ontouchstart' in window)) { $('some').script(); }
  4. if (('ontouchstart' in window)&&!('MouseEvent' in window)) { $('some').script(); }
  5. if (('MouseEvent' in window)||('ontouchstart' in window)) { $('some').script(); }
  6. if ((('MouseEvent' in window)||('ontouchstart' in window))&&!(('MouseEvent' in window)&&('ontouchstart' in window))) { $('some').script(); }
  7. if (('MouseEvent' in window)&&('ontouchstart' in window)) { $('some').script(); }

*5,6 and 7 would, in reality, never be used.

Unfortunately we don't live in a perfect world, and we get plenty of false-positives.

The trouble is, that people are "throwing the baby out with the bath water" when trying to avoid these false-positives. For example, avoiding the galaxy note II false-positive when targeting mouse users you might be tempted to use something like no.3 if (('MouseEvent' in window)&&!('ontouchstart' in window)) { $('some').script(); }, but in doing so you would exclude, en-masse, all those ipad-with-USB-mouse users.

A less "baby bathwater" solution would be to specifically target the galaxy note II device somehow, probably by using a combination of navigator.userAgent, window.screen.width etc. But I'm no doubt telling you stuff you already know...

@stucox
Owner

Superb post.

I'd actually say there are only 4 combos (A, A&B, B, neither), but 7 groupings (potential code forks?)... and that's just considering those 2 input devices.

My stance here is that it's best to detect (and hence target) capabilities rather than devices. If you detect (and fork) for hover, pointing ability, pointer accuracy, etc independently rather than "mouse", "touch" etc - you break it down to several detects with 2 or 3 possible outcomes each rather than one detect with loads of outcomes - which is way easier to manage and more future-proof IMO: It avoids assumptions like "touch means no hover" (which has already been proven wrong by the Galaxy S4).

So I think it's better if we treat this as a detect for hover capability rather than "a mouse user".

@patrickhlauke

As tempting as it is, let's avoid doing UA sniffing for specifics like Galaxy Note II...

I think it's fairly clear that a priori we can't detect anything:

  • it's a mouse-user when a specific mouse event (not the result of simulated/synthetic mouse events as a result of touch) is being received
  • it's a touch user when a touch event is being received
  • for pointer events, these two can be determined by the pointerType property of the events, and in addition we can determine if it's a stylus user

Anything else is guesswork / assumption. I'd defensively design with the idea that a user may be on a device that has all the input modalities, and that they could happen at any moment (as is indeed that case with touchscreen laptops, for instance), and the interface - if needed - should adapt to these modalities as they happen. For instance, MS Office 2013 on a touchscreen laptop running Win 8 has a nice way of showing big, finger-sized handles when touching a document to do text selection, but on the next mouse interaction those handles fade back out.

@benfrain

Tangentially related and apologies that this may subtract from the discussion: I'm very much a JS novice but in a very limited recent use case, I got around loading CSS hover styles in, only for devices that (seemingly) had a mouse, by checking for clientX. Only tried this on iOS devices mind so your mileage may vary:

var detectHover = function () {
    document.removeEventListener('mousemove', detectHover, false);
    console.log( 'Mouse moved (touch and mouse see this): ' + new Date() );
    currentMousePos = event.clientX;
    console.log(currentMousePos);
    if (currentMousePos) {
        console.log('I\'m a mouse - iOS devices don\'t get this');
    }
};

document.addEventListener('mousemove', detectHover, false);
@sbrunot sbrunot referenced this issue from a commit
Commit has since been removed from the repository and is no longer available.
@talamaska

Hello, I have read several posts around detecting and how and why the touch events. Looked at the Mozilla's post too. Also found a weird solution for my use case.

So it it seems there is window.navigator.pointerEnabled only in IE11, and window.navigator.msPointerEnabled in IE 10 and IE 11. Using this and MS proposed draft for pointers. Also I'm almost certain that android/ios devices are not used very often with a mouse (Assumption for my use case, I don't thing someone will use pen to play a game for 3 years old children).

In the end I want to have only one event name - tap or some variable that holds the correct name. I don't care about moving stuff, i use other 3rd party libraries that handle the drag N drop in IE 10.

so my solution is this one

clickhandler : (window.navigator.pointerEnabled ? 'pointerdown' : ( window.navigator.msPointerEnabled ? 'MSPointerDown' : ("ontouchstart" in window ? 'touchstart' : 'click')))

From my quick look of the pointer events draft, MS solution well elevated for the developers - you just bind to the pointer, and the browser knows what to do when you touch the screen or with pen or click with mouse. I would like to see that in all the browsers in the future. I guess ios and android always will be somewhat behind. Note that the last android 4.4 chrome webview removes the 300ms delay for the click events, my guess is that they are wrapping those events and they are assuming it will be a touch event, no matter what.

About handling the hover, well MS does it again with aria-haspopup, but I'm not sure does it work, and it's awkward to add it to all elements that I have hover.
http://msdn.microsoft.com/en-us/library/ie/jj152141(v=vs.85).aspx

I would rather use the proposed :focus rule duplicating the :hover, but I'm not sure what will happen, with the translations, do they trigger twice or something

Sources of reading:
https://coderwall.com/p/mfreca
http://www.w3.org/Submission/pointer-events/
http://msdn.microsoft.com/en-us/library/ie/hh673557(v=vs.85).aspx

@patrickhlauke

Note that window.navigator.pointerEnabled is going to be removed from the spec, as window.PointerEvent already covers the same ground - see https://www.w3.org/Bugs/Public/show_bug.cgi?id=22890

Also, of course that tells you nothing about whether or not a mouse or pen or touch are present, just that the browser supports the umbrella event model of pointer events that covers them all.

using your clickhandler ('pointerdown' if pointer events are supported, 'touchstart' if touch events are supported, or otherwise 'click') will kill keyboard interaction - and, because you're also using pointer events for your detection, you're also immediately killing keyboard for desktop users that only have a mouse and keyboard (rather than the old "'touchstart' or 'click'" approach which killed keyboard if users had a touchscreen and keyboard)...so it's even worse I'd say.

Doubling up :hover rule to also cover :focus works in general, but because of the different way keyboard focusing works (you can obviously only focus elements that are indeed focusable - links, form controls, etc - and as soon as you move away from the element - including to one of its children, as is sometimes the case with classic CSS dropdown menus - focus is lost again), this will not always give the desired result.

@talamaska

I'm not targeting the keyboard, the games are playable with mouse click or finger touch.
if the PointerEvent does what it has to in IE 10/11 Surface, I'm happy with it, why bother is it mouse or pen?
I don't see how this is killing the keyboard users. If you need to interact with the game using keyboard, shouldn't you bind on document/window to keyup/keydown events and check the key code? the tabulation is managed tab tabindex and tab button on your keyboard,

P.S. the title says 'detecting mouse user' not keyboard user.

P.S.2
window.PointerEvent is accessible only in IE 11. There are still users with IE 10 on older Surfaces,
But thanks for the note. I will have it in mind.

@patrickhlauke

"If you need to interact with the game using keyboard, shouldn't you bind on document/window to keyup,keydown events and check the key code?" depends on your explicit situation. if all your page needs to do is run code when the user activates something (where you would traditionally listen for 'click'), then no you shouldn't listen to keyup/keydown and check key code, but instead just hook into 'click'.

of course, if you're "not targeting the keyboard", and you're also advocating a pointer-agnostic approach because for your particular case it's irrelevant if it's a mouse or touch, then that's fine. but know that this discussion here goes beyond your very specific need and use case, which is why the discussed solutions here should really try not to break other things and be as general as possible.

and yes, thank you for pointing out the title of this discussion. but as you may have noticed by reading the lengthy back and forth, the discussion is much broader than that.

@talamaska

I'm still not understanding well the use case of keyboard. I have played a lot of games that supports both mouse click and keyboard button click (not js of course), and I think my logic is correct. I can't point to example where keyboard will be used to replace mouse events. And yes I read all the discussion down to the end.

Wouldn't be better for developer to not care about what moves the "pointer"?

Be more specific:
"if all your page needs to do is run code when the user activates something (where you would traditionally listen for 'click')"

@patrickhlauke

"I have played a lot of games"

This discussion isn't just about games.

"Wouldn't be better for developer to not care about what moves the "pointer"?"

sure, but then why come here and comment on a discussion that clearly started off with the intention of explicitly caring about what moves the pointer?

@patrickhlauke

"window.PointerEvent is accessible only in IE 11. There are still users with IE 10 on older Surfaces,"

IE10 still uses vendor prefix, so window.MSPointerEvent there

@talamaska

I'm not here for trolling or anything like that, or make you feel bad, just taking part in the discussion. Sorry about the games comment, but this use case is one of the most challenging, when you have bunch of interactions. I think everybody here knows how to make websites mobile interactive. For me the big confusion comes from the fact that mouse events are also triggered in the mobile safari. Like someone said above, you need to make assumptions for each use case. I don't think there is general solution. But IE can detect what's triggering the pointer event with ev.PointerType = 'pen,mouse,touch'. So there are moves in this direction - able to get what's moving the pointer.

@Elijen

Not sure if this issue is still active, but I think assuming what user wants to use is a bad idea. It's also a bad idea to change behaviour of you app mid-session depending on user input. I guarantee you user will get confused sooner or later.

The correct solution is to ask the user and respect his preference. This can be done by a settings option.

It would be interesting to detect change from mouse to touch and vice versa, though. This way we could display a pop-up where we could ask the user if he wants us to change app behaviour because we detected a change of his input. (Of course we wouldn't display it on every change)

@sompylasar

@Elijen A user with a touch-enabled laptop won't be happy answering whether he wants mouse or touch because his system does not ask and just handles it properly. The core issue of the current browser implementations is that browsers emulate mouse events in addition to touch events.

@patrickhlauke

basically, once you accept that "a priori" you can't know for sure what inputs the user has (until you encounter the first touch event, for instance, you can't know if the user does or doesn't have a touchscreen), this becomes less of a technical problem (basically, accept that you can't know for sure) and more of a UX one (what IF the user all of a sudden switches to keyboard, or touch, or stylus).

@patrickkettner

The end result of all of this is that you cannot detect a mouse use in a way that would conform to the level of reliability that Modernizr is credited with. For our intents and purposes, it is a undetectable.

If you, future traveler, wish to attempt to detect a mouse user, then the following is the best guide I can offer.

  1. Don't. Seriously. Just because a user has a "mouse" doesn't mean that they don't have multiple other forms of input. You should try really hard to avoid making any kind of UI/UX decision that changes based upon the idea of a mouse user being diametrically opposed to a touchscreen user (or any other kind, for that matter). Make things universal.
  2. If you have to, and only care about IE 10 and 11, then IE's PointerEvent would be worth checking out. Support is abysmal, outside of those two (and presumably future IE versions).
  3. You can attach a listener for a 'hover' event on the body, and if it is true, then the user probably has a mouse. The drawback with this approach include touch events briefly firing hover events on tap/touch, so you could get false positives.
  4. sniff for mobile user agents. This is a bad idea, and goes against the very core of Modernizr. Please don't do it.

But I urge you to question wether or not the reason you stumbled upon this page is actually an XY problem. This thread struggled to find a legitimate use case for forking logic based on the presence of a mouse. Make sure you are solving the problem, not the symptom. Design your code and UIs with all users in mind, support as many platforms as possible, godspeed, and excelsior!

@jaukia

There are a number of relevant use cases for this in interactive web apps. For example, the standard practice is to display zoom buttons on top of maps on desktop browsers but not on top of touch device maps (since they support pinch-zoom interaction). For this, detecting users with mouse would be practical.

@SlexAxton
Owner

I think the thread goes pretty deep into this above here, but there are many devices that are both touch and mouse capable. Detecting the wrong one is deeply frustrating, and we can't guarantee the correct one (consider the millions of microsoft surface and chromebook pixel users).

I agree there are relevant use cases for knowing how a user is interacting with your app, but it cannot currently be detected ahead of time. Instead, you can adapt your application based on the actual inputs as they happen.

@patrickhlauke

Also, remember that even on touch devices, some users don't want/can't pinch-zoom (e.g. holding mobile phone with one hand and interacting mostly with the thumb, blind user with VoiceOver/TalkBack navigating on a touchscreen device sequentially in a similar fashion to TAB/SHIFT+TAB on desktop, etc).

As @SlexAxton said, base your decisions on actual inputs (and consider that even though one moment a user may be firing touch events, the next interaction might instead come from a keyboard/mouse on multi-input devices, so make any interface/interaction changes "just in time" for the current interaction and don't then lock any other interaction methods out).

@roblarsen

As @SlexAxton said, base your decisions on actual inputs (and consider that even though one moment a user may be firing touch events, the next interaction might instead come from a keyboard/mouse on multi-input devices, so make any interface/interaction changes "just in time" for the current interaction and don't then lock any other interaction methods out).

I have two examples that I love to share. The one people can more readily imagine since it's been on TV ads is the case of a convertible Windows laptop like the Lenovo Yoga. It always has touch capability, but fine pointer (mouse-like) control comes and goes depending on the configuration of the device. A web page can be opened in laptop mode (mouse!) and then, before anything else happens, the device can be converted to table mode (no mouse!) And then, in tablet mode, you can add a peripheral pointer (mouse or stylus) and it all goes kooky once again.

The second is basically diabolical- I have a touchscreen laptop set up as a workstation with a second monitor. If I'm on the second monitor, I don't have touch capability, even though the OS and the browser think that I do. I often (including some very big sites that should know better) have to move sites from the second monitor to the laptop screen in order to activate controls.

@patrickhlauke

As a side note (only because @roblarsen mentioned "fine pointer"), the current implementation in browsers of the proposed CSS4 interaction media features is borked / does not dynamically change (in Chrome at the moment, not even on page reload) https://dev.opera.com/articles/media-features/

If it did work correctly, it would still be based on some assumptions/guesses on the part of the browser/OS, but would provide a starting point nonetheless.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.