-
Notifications
You must be signed in to change notification settings - Fork 48
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add gamepad input events #152
base: gh-pages
Are you sure you want to change the base?
Conversation
04884c3
to
e93ef99
Compare
I've updated the description to reference crbug/856290 as indication of implementation commitment from Chromium. Does this change completely replace #15? |
index.html
Outdated
|button|.{{GamepadButton/touched}}. | ||
</li> | ||
<li>[=Queue a task=] on the [=gamepad task source=] to [=fire | ||
an event=] named {{buttonchange}} at |gamepad| using |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should buttonchange
also fire when a button is touched but its value has not changed or do we want to save that for separate buttontouchdown
and buttontouchup
events?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think buttonchange
should only fire for value
changes and we should add touch events if needed.
The current gamepad market doesn't have many gamepads that detect touch separately from pressure so I think the potential for a gamepad to fire buttontouchdown
without buttonchange
is pretty low.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Makes sense to me. Do we want to specify that now or leave it as a TODO for later?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am happy either way. I would say let's do the thing that would be easiest from a standards approval perspective. I think that would be to add it here and get it all done in one go.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
index.html
Outdated
<dfn>axisIndex</dfn> member | ||
</dt> | ||
<dd> | ||
The button index for the event. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The button index for the event. | |
The axis index for the event. |
index.html
Outdated
<dfn>axisSnapshot</dfn> member | ||
</dt> | ||
<dd> | ||
A copy of the current button state. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A copy of the current button state. | |
A copy of the current axis state. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks good. However, I don't see anything about coalescing events as we had discussed. Are we still planning on adding that in?
Considering event coalescing does seem important because I am concerned that the current design will result in lots of events being fired for each "update" of the gamepad. For example, when a control stick is moved diagonally that will generate two axis events with identical timestamps. This seems like it will cause more work to be done on the main thread. It should be possible to handle these events simultaneously. Doing this actually seems like a slightly different problem than what is solved by APIs like PointerEvent.getCoalescedEvents() because in that case the coalesced events would otherwise be discarded while for the Gamepad API they would otherwise be fired next. I can see a couple ways to avoid this problem:
Maybe the issue only really affects axis (since movement of a joystick with X and Y axis means that there is a high likelihood of paired events) and so only applying proposal (1) to GamepadAxisEvents would be enough and GamepadButtonEvents can be left as specified here. This would also make GamepadAxisEvents similar to PointerEvents in having both X and Y axis (or an arbitrary number of axis) updates in the same event. |
I like proposal 1 (a single
When an input frame is received from the gamepad, if a When dispatching, dequeue all the queued instances. If there were no queued instances, don't dispatch an event. Otherwise, create a new GamepadChangeEvent and stash the queued instances in an internal slot so they can be returned by getCoalescedEvents(). Initialize its gamepadSnapshot attribute to the gamepadSnapshot of the most recent instance. Initialize the button and axis index arrays to the set union of the corresponding arrays in the dequeued instances. If a button or axis changed in any coalesced event then its index should be in the buttonsChanged or axesChanged arrays. If a button is pressed and released in separate coalesced events then its index should be in both buttonsDown and buttonsUp. |
I'm interested in the TAG's opinion on the best design here. I think you're about to submit a review request? A single event does seem more in line with the PointerEvent interface, which includes a number of different pointer properties (comparable to axis and buttons) in a single event. From that perspective coalescing events is an orthogonal concern and depends on whether we want to specify when the user agent may choose not to fire a GamepadChangeEvent. For example, it could discard events that arrive while the main thread is currently handling another event. (Actually implementing this in Chromium will be non-trivial but could be worthwhile.) As specified in this PR multiple input reports would be queued and delivered as separate events, while with the current polling-based system some input reports would simply be missed because the page was busy and didn't poll quickly enough. Coalescing events while the main thread is busy seems better than queuing them and delivering a cluster of events at a later time. |
Note also garbage collection text might be needed: whatwg/dom#1003 |
Co-authored-by: Marcos Cáceres <marcos@marcosc.com>
Did you folks see #4 (comment)? I propose an axischange event there. It feels like this matches pointermove, where you're generally interested in the latest value, sync'd to raf. Whereas button presses are more immediate (and must be preceded by events to update the axis state if changes are pending). Event coalescing can be used to get more detailed axis changes, but even then I wouldn't expect multiple events with the same type and timestamp. Having an equivalent of pointerrawupate also seems ok. |
Actually, this would be very common. Gamepads are usually implemented using the HID protocol or something similar where all button and axis values are delivered in a single packet. If you're moving a joystick then it's likely you'll see updated X and Y axis values in the same packet, which would result in two axischange events with the same timestamp (but different axis indices).
Perhaps there should be a gamepadchange event that fires at some reasonable time synchronized to rAF and provides coalesced events, and a rawgamepadchange event that fires for every change and doesn't coalesce? I'm looking at the pointermove spec and it looks there's no guidance on when the event should be fired if it's delayed. For gamepadchange I think it would be good to explicitly specify that if there's a pending gamepadchange event then it must be fired before rAF. That lets applications synchronize on rAF if they need to consume some inputs through getGamepads() polling and some through events. |
Could/should we combine those into a single event?
Yeah, that's what I'm thinking too. That seems to fit with pointer events nicely.
Yeah sigh I'm sad that they haven't done that.
Agreed! It'd be good to lead the way here and show pointerevents how it should be done 😄. https://html.spec.whatwg.org/multipage/webappapis.html#event-loop-processing-model steps 11.7-11.12 handle various events that shouls be sync'd to rAF, so I guess the gamepad stuff should go in there. What about button presses? Do you think those should happen immediately? Mouse clicks do. It's something like this: On mouse button down:
So the mouse-down flushes the queued |
The latest patch removes buttonchange/axischange and replaces them with gamepadchange/rawgamepadchange. I've added a "run the gamepad steps" algorithm to integrate with the event loop's pre-animation-frame steps which flushes any pending gamepadchange events.
Yes, they should happen immediately. I've added a step to flush gamepadchange before firing buttondown/buttonup. Currently it flushes on either event, maybe it should only flush for buttondown? |
Is this still happening? Is this something that mere mortals like myself can experiment with? Is this likely to happen any time soon, or should I continue to work on my half-@$$ed hack-job to shim an event? My (non-expert) two cents:
Ref: |
@jharris1993, this work is still in progress. If I'm interpreting your feedback correctly it sounds like this change is exactly what you're looking for. We would appreciate feedback on specific changes proposed in this PR which you can preview as a rendered specification here. Feedback from developers like you is what helps us prioritize our work and move designs in the right direction. Chromium is planning on implementing these changes but there isn't a prototype ready yet. If you subscribe to the Chromium issue you'll get updates when code for that lands. |
Ref: https://w3c.github.io/gamepad/#other-events My two kopeks: At least initially, a generic gamepad event that returns the entire gamepad data-frame (gamepad dictionary?) would be sufficient. (Ref: KISS rule) As far as data processing is concerned, we should be concentrating on getting the DATA to the developer/application. : Any additional data-massaging can be done to the returned data on the server or within the JavaScript on the client.. (i.e Qualifying axis inputs with a button press, normalizing or constraining data, etc.) After that has been implemented, the question of adding additional gamepad events can be addressed based on need. P.S. P.P.S. |
Thanks for this feedback. I will let @nondebug take it from here. I'm just managing things while he's out of the office.
"That specification" is this specification, and the issue tracker is here on GitHub.
You've already found the issue related to the Gamepad API in particular moving to a secure context requirement. In terms of the overall direction the web security model is moving I think commenting on https://github.com/w3c/webappsec-secure-contexts where "secure context" is defined. |
Is the event API meant to replace the polling API? Frankly this proposal doesn't make any sense, because gameplay isn't event driven, it's update driven. And it's often done in "phases", where input means different things at different times and in different contexts. So if I'm caching events to consume them during my game loop, that's actually worse than polling. What does an gamepad event handler look like in practice? Is my event handler supposed to contain game logic in it? "onJumpPressed, add velocity if Mario is not swimming and not on a trampoline on time 0-10 and did not just bounce off an enemy and does not have a a powerup"? And you'd have to do that for every single entity in the entire game for every single input... |
@sineprime, different games use different models for handling input. Polling input is very natural for games that consider controller state as part of their update step, but not every game works this way. Polling also means that it is possible for the game to miss inputs if it doesn't happen to poll the controller while a button is pressed. Input events are designed to enable developers that want to write logic like "when button A is pressed, start a jump animation" but also allows developers using the update model to modify state (e.g. "jump button was pressed") that is then checked as part of the main game loop. In other words, it is easy to polyfill a polling-based input model on top of an event-based one. Doing the reverse is inefficient (because you just keep having to poll faster and faster to avoid missing events) and so the platform should provide events and let developers use them however they'd like to. Events are also how the underlying hardware interfaces work, they generate interrupts when the controller state changes. Providing an event-based API allows developers to get those events as quickly as possible without wasting cycles polling. |
Ah, I thought HID was polling based and this API was an abstraction layer on top of polling. If it's event-driven, then this makes a lot more sense. |
For full disclosure HID over USB is a hybrid of polling and events. At the physical transport level (except for fancier USB 3.0 devices) the USB host controller constantly polls the devices on the bus for completion of "transfers" requested by the kernel. A HID device might respond to every request with a new HID report or it might only respond when something has changed. That's up to the device. When the host controller gets a response other than "I'm not ready, try again later" that completes the transfer and generates an interrupt. The kernel then submits a new transfer request to the host controller so that it will keep asking the device for more data. So the software is polling, but only gets interrupts when the hardware has something new to report. The kernel can forward those interrupts to listening applications or can simply update an internal record of the current controller state that can be queried by applications as part of their game loop. |
". . . different games use different models for handling input." . . . what do I have to do to get people to think of a joystick/gamepad as a generic HID, (in the same way a mouse is a kind of HID), instead of just going "game. . . game. . . game"? The biggest problem I have with all of these specifications, both this one and the security implementation, is that everyone is stuck on these items being "game" controllers. People see a joystick and the first thing they think of is some flight-sim game and that's the problem. At the risk of quoting an old Taco Bell commercial, people don't "think outside the bun". A joystick or game controller can be used for so many other things, like:
And using a joystick within a browser makes it location/OS agnostic which is a tremendous advantage! The uses are out there, it just remains to create an interface where people can use these devices without being confined to either a game loop or a thoughtless security model. Have any of you looked at the nipple.js library? Nipple.js uses a mouse to simulate a joystick. Why? Because using an actual joystick to do what a joystick does best is too much trouble and you have to jump through so many hoops. In my case I eventually got my project to work, (at least somewhat), but before it would work I had to load up my project with a separate full-blown nginx proxy, buy a domain certificate, and throw in a bunch of glue-logic just to use a joystick as a generic input device in a browser! And it just about doubled the footprint of my device's code - all to allow a joystick to just work. Please! Let's try to think of these devices as something other than game controllers. OK? |
@jharris1993, respectfully, @reillyeon was just making and illustrative example. We (Working Group, including @reillyeon) are well aware that gamepads are generic input devices. The fact remains that they are primarily used for gaming. |
Hey everyone, do we have any kind of a close date when this pr is going to be merged? or can you guys suggest any alternative to send button events? |
Closes #4
Closes #22 ?
The following tasks have been completed:
Implementation commitment:
Preview | Diff
Preview | Diff