Add native gyro support for the Steam Deck itself

[streetpea]

Add support for native gyro controls on the Steam Deck for games that have them implemented (i.e., paintbrush in Concrete Genie and flashlight in Last of Us).

[jbaiter]

So I ported all of the touchpad and sensor handling over to SDL (from Setsu, which only supports DS4 and DS5 controllers), in the hopes of getting it working on the Steam Deck.
The Good news: With the changes (to be upstreamed) any controller that SDL recognizes as having at least one touchpad, an accelerometer and a gyrosensor can be used.
The Bad news: None of these things are true for the Steam Deck controller :-( It's only visible as a very basic XBox 360 Controller to SDL, exposing neither of the touchpads and neither of the sensors. This is a known issue in SDL and looks like needs help from Valve to get working properly: libsdl-org/SDL#6068

One possible workaround could be to write custom sensor handling code that speaks directly to the device via hidapi/libusb, but that's going to require some reverse engineering.

--edit: Reverse engineering might not be so difficult, there's a hidapi driver for the Steam Controller in SDL itself that could serve as a good starting point, given that the Deck's controller is effectively a refinement of the Steam Controller: https://github.com/libsdl-org/SDL/blob/main/src/joystick/hidapi/SDL_hidapi_steam.c

[streetpea]

Hi @jbaiter. Yes, the SDL code doesn’t work for the Steam Deck controller gyro functionality. I tried that when working on the touchpad / touchscreen implementation, but was less ambitious than you and stopped when I couldn’t see anything with the SDL test program.

Having said that, switching to SDL should mean that using the functionality with the DualSense will no longer require a udev rules file as well as supporting other controllers with gyro/touchpad functionality such as the Switch Pro controller (I’d presume without having tried it personally) as you mentioned. Thus, it’s still a nice addition.

The touchpads and touchscreen are currently supported in chiaki4deck since the last release about a month ago using touchscreen support and mouse support within QT respectively. As far as the gyro support, in addition to not being available in SDL, QT sensors has very limited Linux functionality and doesn’t surface any sensors for the Steam Deck. The other sensor code I’ve used also doesn’t surface the Steam Deck’s gyro or accelerometer and neither does udev/evdev, only temperature sensors on the Deck and the ambient light sensor. Having said that, reading the Steam Deck usb device does provide the necessary data, so I see that as the way forward like you mentioned and plan to implement it this weekend when I’ll finally have some coding time again. You can see it using the gui program provided by the hidapi project. I don’t actually believe this will be very difficult I just haven’t had a chance to take a crack at it yet.

As far as implementations on the Steam Deck gyro outside of the Steam input api to date, the only functional tool using the gyro on the Deck I have seen is Steam Deck GyroDSU, which reads the hid device directly as seen here: https://github.com/kmicki/SteamDeckGyroDSU/blob/576275d19737257205fd6d711faf543bcf131aba/inc/sdgyrodsu/sdhidframe.h.

[streetpea]

@jbaiter I unfortunately didn’t make too much progress but here is the small update I have. I was able to write a simple program with hidapi to get the gyro data as well as produce haptic feedback similar to the Steam controller singer example via the Steam Deck controller. It works in the chiaki4deck flatpak (though isn’t integrated with Chiaki itself yet) and doesn’t require any sudo privileges / password on the deck since valve already installs the necessary udev rules by default for hidapi (via hidraw). It also provides access to a bunch of other Steam Deck controls but it probably makes the most sense to only use the gyro and haptics for chiaki4deck for now.

I have to clean up the code and separate some of the stuff out into more helper functions before moving it into the Chiaki4deck codebase. I can add a new branch when I’m done with that for you to plug your haptic stuff into if you want (once I’m done tidying up) or you could push a branch that has the haptic stuff and I could update that. I could also try my hand at getting the haptics to work with pcm files as input if you haven’t already figured that out / cracked that but I’m not sure where you’re at with that as of now.

[jbaiter]

I unfortunately didn’t make too much progress but here is the small update I have

That's not small, those are some pretty big advances 🤗

. I can add a new branch when I’m done with that for you to plug your haptic stuff into if you want (once I’m done tidying up) or you could push a branch that has the haptic stuff and I could update that.

I don't know how far the chiaki4deck codebase has deviated from upstream, but the branch with the haptics implemented via SDL is available at https://git.sr.ht/~jbaiter/chiaki/log/gui-haptics, specifically this commit: https://git.sr.ht/~jbaiter/chiaki/commit/92e23d362be713383204fb27f008b1f44d189840
I'm currently force-pushing often as I'm fighting with the Windows CI build, but if you rebase on top of that commit you should have a good basis to work from!

I could also try my hand at getting the haptics to work with pcm files as input if you haven’t already figured that out / cracked that but I’m not sure where you’re at with that as of now.

I have not done any work on mapping the haptics to the deck, so go for it if you want! If you want I can send you a PCM file of the haptics from the Astro Controller Demo intro, that's pretty good for testing.

One other thing that occurred to me the other night: Maybe we should map at least some of the trigger effects to the haptics as well. I'm specifically thinking of stuff like the 'gunfire' effect, which is felt as the main haptic effect on the DualSense when it's active, with the other haptics merely supporting it.
There's a very comprehensive overview of all possible effects available here: https://gist.github.com/Nielk1/6d54cc2c00d2201ccb8c2720ad7538db

[streetpea]

Rebasing from that branch works for me. I’ll start work on the haptic Steam Deck conversion once I finish the gyro (probably at least a week from now) unless you tell me otherwise. I have some questions but will post them in the haptic issue for clarity.

[streetpea]

Update on this is that I have this up and working now. It seems to work well after a bit of fine tuning and testing output as well as in-game testing but will continue to test against various games as I work on other issues (i.e., haptics) … so far I have just tested against Concrete Genie and Ghost of Tsushima (music tuning) as far as in-game tests.

[jbaiter]

If you get a chance, try testing against Astro's Playroom, the included Controller Demo is very useful for debugging sensor issues. Also, many of the included minigames rely on the sensors, so this can be a good opportunity as well.

[streetpea]

@jbaiter thx for the suggestion. Running against the controller demo in particular helped me realize there was a bit of input lag I was able to fix. Besides that, the gyro works well with the controller demo and Astro’s itself.

[lazzer2000]

As a curious party. When this is added will it work with PS4 also... And could this information be used in emulators that can use gyro?

[streetpea]

@lazzer2000 this should work for PS4 and PS5 though I have only tested it with a PS5 (using both PS4 and PS5 games). It provides an interface to the Steam Deck directly, so far only providing external interfaces for gyro and haptics though that could be theoretically expanded to other outputs with additional functions (the full input data packet is there and split up, I just am only sending gyro, accel, and orientation information since I am getting the rest via the virtual controller provided to SDL by Steam so it doesn't make sense to send it for this project).

As for if it will work with emulators, a given open-source emulator could use the HIDAPI (open source) library in a similar way that I am and/or adapt the steamdeck native interface (which leverages hidapi) in chiaki4deck to fit their emulator (this is also open source under GPL). As for if they would want to do that, it would be up to the individual project. If it's open source, it might make sense. If it's a closed source project, it might make more sense to just use Steam input directly (closed source projects also wouldn't be able to use the code from this GPL project but could use HIDAPI under one of its other licenses). Having said that, you are probably thinking of an open source emulator, since a closed source one probably would have already started going the Steam Input route.

The code will be added here (this GitHub repo) with the next release next week so at that point, the contributors to a given emulator could look at it and see if it would be of any help to them or not.