Addition of haptics for Steam Deck streaming from PS5 #14
Comments
|
Thanks for adding the issue. I will definitely look into this. |
|
This would be incredible, even if only for DualSense controllers connected to the deck. But I think it's going to require some reverse engineering, since the current Takion protocol definitions in chiaki have nothing related to haptics. Supporting it for attached DualSense controllers is probably going to require some additional work as well, since Sony's kernel driver does not support haptics, and neither does the general kernel API for controllers. However, from what I could find so far, Haptics on the DualSense seems to be implemented using audio data, i.e. the controller haptics function as a kind of speaker and haptic effects are sent as waveforms. See for example this PoC Windows implementation: So tasks involved in this seem to be:
Here's a talk from chiaki's original author, @thestr4ng3r, on how he reverse engineered the protocol: https://mdco2.mini.debconf.org/talks/9-chiaki-bringing-playstation-4-remote-play-to-linux/ -- edit: -- edit: |
|
Thanks for starting to look into this @jbaiter … as for steamworks that is something we would have to get permission from @thestr4ng3r to use because it violates the gpl license (see https://partner.steamgames.com/doc/sdk/uploading/distributing_opensource) so if there is a way to do this without using steamworks that would be preferred. |
|
I think there's a way, did some googling and it seems you can talk directly to the haptics without going through the steamworks library, e.g. https://github.com/Roboron3042/SteamControllerSinger. -- Update: Trigger effects are working, feels a bit weaker than in the official client, but the effects are all there. |
|
As far as the Takion side of things - I haven't really tested it much, but in theory pyremoteplay now supports haptics - ktnrg45/pyremoteplay@master...rumble As to Steamworks for haptics, a worst-case workaround might be to create a separate haptic daemon that links to the library and talks with chiaki using some form of IPC. But probably there's a sysfs trigger somewhere for the rumble? |
That's only for the DS4 rumble emulation, not for the 'real' haptics, Chiaki already supports that. As for the 'real' haptics, I managed to get them working with a connected DualSense just a few moments ago. Current requirement is using the
I think the way to go is to use The only remaining task now is to find a way to map the Haptics audio data (3kHz 16Bit Stereo PCM) to the Steam Deck's touchpad haptics in a way that feels good! From what I understand we need to convert our PCM samples into PWM that we can more easily translate to the Steam Deck's haptics parameters (pulse high, pulse low and repeat). |
|
Awesome work @jbaiter! I agree that libusb or hidapi is the way to go if not using SteamInput. It seems like it’s the only way at the moment based on what’s exposed outside of Steam Input so is the best path forward. Any software dependencies that need to be added (such as pipewire) if not already included in the flatpak can be added to the flatpak container during build. As far as haptics for the Steam Deck controller, how would it switch between simulated haptics for rumble (what it’s doing now) and “real” haptics or do fake haptics get sent for ps4 games too and we can still use those for ps4 games? As far as the pcm to pwm, Here is an example of someone trying this in a different project but I’m not sure how much help it will be. |
On the API level, rumble events are sent through SDL's gamepad routines to the HID device, while Haptics are sent as audio samples to the DualSense Audio device. When sending a rumble signal, SDL sets some bits to disable audio haptics, which are then cleared once the rumble effect is disabled. When emulating the rumble and haptics through the Steam Deck's touchpads, I think we can follow the same approach, use SDL for regular rumble, and direct USB access for the haptics. They will with large certainty never interfere with each other, since games that use rumble don't use haptics, and vice versa. |
|
Jumping back in only because I saw that @jbaiter is also contributing over on the main chiaki repo - in addition to this effort for the Deck, will haptics and adaptive trigger support for Dualsense be coming to the Win64 build of chiaki? Because holy crap, that would rule. |
|
It sure will! |
|
I absolutely will. 🙂 |
|
As per the discussion in #7, I have the some questions for when I start work on playing the files / inputs you get from the PS5 using the Steam Deck haptics using the play haptic functionality from my hidapi code. Once I start that effort, a sample haptic file would be nice for testing. As for the trigger effects, is that something that you know when you get it and can then play the appropriate haptic or does it work some other way? Also, how does this work with the sound output haptics? Are you guaranteed to only get one at a time or does one take precedence over another? Based on the files you provided, it seems like these effects would be something we would provide a specific function for and trigger at the appropriate time. Is this what you had in mind? |
Yes, it's simply a discrete packet sent over the Remote Play protocol that tells you which effect to send to the triggers, just like rumble. The data is basically sent as-is (with some rearranging) via HID (or rather, via SDL2, which has an API for that) to the DualSense.
Those two are completely independent. Trigger effects are discrete events, haptic effects are implemented as a semi-continuous audio stream, just like the regular video and audio streams. "Semi-continuous", because silence is not encoded explicitly, i.e. when there are no haptics, no data is sent. The handling of those two is completely separated as well, there's no relationship at the protocol or application level between them. Haptics are resampled from 3KHz to 48KHz and sent as raw PCM to the DualSense audio device, separate from the HID stack.
If we decide to map both the trigger effects and the regular haptics to the Steam Deck's haptics, we're going to need to interleave the two sources somehow, probably with a queue and some prioritization heuristics. It's probably wiser to leave that for later and start with mapping only the haptics first :-) Links to relevant code sections:
|
|
I have been working on the Steam Deck haptics. There are limitations to what the Steam Deck haptics can do as opposed to the DualSense (especially since we have to translate the pcm signal on the fly which leads to a trade off between resolution / minimum frequency and latency [though we can cheat the future a little bit]) but the majority of the haptics seem to be working reasonably well though are a bit noisy atm. One thing I ran into was the gpu jungle section of Astro’s playroom where you are the monkey seems to break when using the enable dualsense option. Specifically, when you are climbing it doesn’t let you grab one of the higher ledges. It seems that it’s waiting for a trigger to go to the next stage for the adaptive triggers or something. Not sure if you have seen this @jbaiter? Also, in general have you made any updates to the haptics since the link you posted before? I have been using that to start with but will want to add any changes before the next release. |
|
This was fixed in a recent commit on my branch, I had flipped the branches on a ternary by accident 🙈 The other update was support for hot plugging DS5 controllers, but that's not relevant for you, I think. The code is currently in review. |
|
@jbaiter as far as the hot plugging of the DualSense, I made 2 minor tweaks:
|
Notable Changes: - cli motion + haptics demos broken out from test code - updated cmake to require fftw for Steam Deck Native using pkgconfig - change chiaki4deck.yaml back to url for release - eliminate extraneous comments in Steam Deck native interface - updated Steam Deck dev doc with some env vars - updated preprocessor flags for Steam Deck interface to cover missing spots - update HIDAPI to latest release (0.13.0) in flatpak manifest Closes #14
Currently when streaming from PS5, there is no rumble feedback on the Steam Deck; if this is something that could be added, it would be a big boost to the gameplay feel for a lot of titles.
(As mentioned in #6 where I initially brought it up, there is also no haptic feedback on the DualSense controller when streaming from PS5, which is also true in the original chiaki on all platforms. Since this is a Steam Deck focused branch I didn't think about it, but hey, if it can be done? Rad. I'm sure folks playing on the Deck with a paired DualSense would appreciate it.)
The text was updated successfully, but these errors were encountered: