Support controlling Free Look via input bindings (motion controls, gamepad, etc!)

[iwubcode]

I've always thought Free Look was such a cool and underutilized feature. In an effort to make it more mainstream, I've removed the shackles of requiring to use the mouse. Now you can assign motion controls (or gamepad) to the Free Look rotation movement.

Rotation Free Look settings (mouse, gamepad, DSU or bluetooth emulated wiimote!)

[Techjar]

Yeah may as well change the names while we're at it, I doubt many people change the freelook bindings so them getting reset shouldn't be a big deal.

However, the default rotate bindings are not right. Right click should control pitch and yaw, while middle click should control roll. Turns out yaw and roll are swapped somewhere in the code.

Also, in testing I found the mouse rotation to be far too sensitive now, so much as to be completely unusable.

[iwubcode]

I took another stab at the freelook hotkey buttons in the UI. However, if that is too radical of a change, I can switch them back to a single group.

Thank you for reviewing @Techjar . About the mouse sensitivity, were you able to get an expression you liked? I need to retry the behavior on master but thought it was just as sensitive.

[jordan-woyak]

The poorly named (my fault) "Axis X/Y" (relative mouse movement) mappings all had os-specific smoothing worked into them to make them semi-usable for analog sticks.
We should probably eliminate this now that input expressions support a "smooth" function. Perhaps we can just add new "Relative Mouse X/Y" inputs and leave the "Axis" inputs as non-detectable just for backwards compatibility?

[Techjar]

To be clear I also tried binding my Wiimote gyro to the freelook rotation and it went absolutely nuts at the tiniest movements.

[Techjar]

Well RotateView is cumulative right? So velocity actually is an acceptable input for it, I think it just needs to be scaled since the arguments are radians. Though that wouldn't necessarily give you 1:1 mapping for phyiscal gyros. Hmm...

[iwubcode]

If someone is able to test the latest version with a real gyro, that'd be helpful. With OpenTrackDSUServer and a circular motion, I'm getting multiple arcs for in game rotations, despite PadTest showing a 180 turn.

Uncertain if this is normal behavior for Dolphin (I see the same thing in Citra with OOT's first-person view) or a bug in my DSU server. I was expecting 1:1 mapping.

At least in my mind, this is the only thing keeping this PR from being WIP still.

[mbc07]

I can give a try with the DualShock 4 motion sensors. Just to be clear, what is the expected behavior?

[iwubcode]

@mbc07 - I'm actually trying to figure out the expected behavior. I don't own / haven't setup any gyro based controllers so I don't know what the normal behavior is. If you move a DS4, do you get a 1:1 mapping in Dolphin?

For this freelook feature, when using my new DSU server, I see a 1:1 mapping in "PadTest" but not in Dolphin or Citra. I would assume "PadTest" is correct but I have no other controllers to confirm..

EDIT: To put it in a less verbose way, does this feature feel correct with a gyro-based controller?

[mbc07]

I gave it a test and apart from some inverted axes (more on that later) it seems to work as it should.

I used a DualShock 4 v2 (the one where the light bar is visible from the touchpad) with DS4Windows 2.0.16. I booted Zelda Wind Waker and started messing with the free look controls as soon as the player has control of Link (right after him waking up).

Up and down movement is working fine, however, roll and yaw movements are inverted. For yaw, assigning Yaw Right to Yaw Left and Yaw Left to Yaw Right on the hotkey settings was enough to correct that.

Roll, however, I couldn't figure out. With Roll Left assigned to Roll Left and Roll Right assigned to Roll Right, the free-look camera was inverted (it actually rotated left when you rolled the controller right and vice versa). Then I tried to invert the bindings, just like I did with Yaw, and while that works, some time later the Roll movements goes inverted again (I couldn't figure out what triggers that).

TL;DR it works as it should if you invert the Yaw assignments, except for Roll, which is prone to do the inverted movement even after trying to account for that by inverting the bindings...

[iwubcode]

Thanks for testing @mbc07 ! I actually forgot you can use a Wiimote and luckily had one to test and I'm seeing similar behavior.

I will need to look and see why that is..

[iwubcode]

@mbc07 - I've updated this to fix some things, drift should be gone and 1:1 motion is now as expected. It should also fix the axis inversion you were experiencing.

One thing I'm seeing is some odd roll rotation with the Wiimote. I don't think I'm adding it with my movement. Do you see any surprising behavior with your DS4?

---

As an aside, I need to tweak the mouse movements some more. If anyone gets some working defaults, let me know!

[mbc07]

Sorry for the delay. Tested again with the DualShock 4 and everything is working as it should, without needing to invert any axes. Hasn't tested mouse movement, though...

[iwubcode]

I followed @jordan-woyak 's recommendation and added a "RelativeMouse" input that doesn't do smoothing and deprecates the "Axis" for mouse movement.

Still trying to get mouse movement to match master. That's the only blocker at this point.

[iwubcode]

@mbc07 - if you don't mind, at your earliest convenience, please test this again. I rebased with master which required some changes due to the new freelook control types! From what I can tell, everything works appropriately but it'd be nice to get a second look.

[mbc07]

@iwubcode just gave a try and noticed the UI has some oddities:

• "Freelook Motion Simulation" tab has a "Gyro" visual indicator that doesn't seem to do anything? (but the default mouse bindings still works in-game) Disregard, false alarm.

• "Freelook Motion Input" tab has the Gyro bindings assigned by default (yay!), but they don't do anything unless you select the correct controller on the "Device" drop-down menu. But then, by doing that, all other hotkeys will stop working unless you reassign every one of them with the proper "DInput/Keyboard Mouse/" prefix (that's on Windows, it's likely something else on Linux/macOS).

  So, yeah, that's not trivial to resolve and I can't think of any reasonable compromise other than, perhaps, shipping with the Gyro bindings blank by default and requiring the user to configure them manually? At least it would be way easier than needing to reassign every single hotkey binding from all other tabs.

Other than that, everything is working as intended, just like the last time I tried this PR (tested with a DualShock 4 on Wind Waker)...

[iwubcode]

Thanks to @Filoppi for noticing that the GetTicks function shouldn't be ran on a separate thread.

I went back to the basics and now update the FreeLook input every 1/60th of a second. It seemed to work from my testing with Wiimote DSU. If we're dropping input, I can't notice, it still 'feels' fine.

I also made a change to have the movement consider the delta time. That will help if we ever move this to a separate thread that runs faster or slower (may also help with the accelerometer? I haven't tested yet). Because of this change, I also re-tested gamepad and keyboard/mouse. I believe I have things working very close to master.

@MayImilae / @mbc07 / @MichaelJW / @Miksel12 - sorry to nag you all again. But if some of you can do a sanity check, it would be very helpful. I really hope I didn't break anything!

[Filoppi]

I also made a change to have the movement consider the delta time. That will help if we ever move this to a separate thread that runs faster or slower (may also help with the accelerometer? I haven't tested yet). Because of this change, I also re-tested gamepad and keyboard/mouse. I believe I have things working very close to master.

Ideally you'd want to divide a fixed value by the emulation speed IMO, or the possible huge differences in timings between frames (e.g. one frame could take 16ms, the next one 3ms, while you still run at 60 overall) would add jerk to the camera movement. But of course it's fine as it is now and that change would require Freelook to update once per video frame independently of the emulation speed.

Great changes overall.

[Miksel12]

@iwubcode There seems to be quite a lot of drift now on my DS4 and it seems like it is harder to control. I did a sanity check on master which showed the correct behaviour.

On a somewhat related note, when I press on default with my DS4 as DSU selected input, it switches back to keyboard and mouse as selected input and configures the default mappings for keyboard. I would think it would be better if Dolphin didn't configure anything if the selected input doesn't have default mappings. I was half way into mapping my DS4 when I remembered I could just click on default, only to see my mappings replaced with keyboard mappings.

[Filoppi]

@iwubcode There seems to be quite a lot of drift now on my DS4 and it seems like it is harder to control. I did a sanity check on master which showed the correct behaviour.

On a somewhat related note, when I press on default with my DS4 as DSU selected input, it switches back to keyboard and mouse as selected input and configures the default mappings for keyboard. I would think it would be better if Dolphin didn't configure anything if the selected input doesn't have default mappings. I was half way into mapping my DS4 when I remembered I could just click on default, only to see my mappings replaced with keyboard mappings.

Off topic but "Default" restores the default profile, a profile also includes a specific default device, and in the case of the "Default" profile, the default device is the first one in the list of devices, which is always the keyboard and mouse device.
If we had a default mapping for a DSU device, that would be listed on the stock profiles on the right.

[Miksel12]

Off topic but "Default" restores the default profile, a profile also includes a specific default device, and in the case of the "Default" profile, the default device is the first one in the list of devices, which is always the keyboard and mouse device.
If we had a default mapping for a DSU device, that would be listed on the stock profiles on the right.

I see, misconception on my part.

[iwubcode]

There seems to be quite a lot of drift now on my DS4 and it seems like it is harder to control. I did a sanity check on master which showed the correct behaviour.

Drats, ok. Thank you for testing. It's gotta be the DS4 poll rate is higher than the Wiimote. I wasn't able to find a solution that I was happy with keeping FreeLook at 200hz without introducing another thread or spinning. Maybe I'll have to add another thread for now with a condition_variable to keep from spinning, still rather ugly..

What do you mean checked with master? You mean there's an issue with the mouse controls too?

[Filoppi]

Drats, ok. Thank you for testing. It's gotta be the DS4 poll rate is higher than the Wiimote. I wasn't able to find a solution that I was happy with keeping FreeLook at 200hz without introducing another thread or spinning. Maybe I'll have to add another thread for now with a condition_variable to keep from spinning, still rather ugly..

If you are going to add to this PR, I think it would be much better to just calculate the absolute rotation from within DSU, and add it as an input, then move FreeLook to the CPU thread and multiply the delta time you just added by the game speed.
That sounds better than a separate thread IMO, a separate thread won't ever be 100% aligned to the video frame rate, unless you implement some form of thread locking, which wouldn't great.

But really, as a simple solution you can just run the hotkeys at 200Hz, it's not gonna break anything.

[iwubcode]

If you are going to add to this PR, I think it would be much better to just calculate the absolute rotation from within DSU, and add it as an input, then move FreeLook to the CPU thread and multiply the delta time you just added by the game speed.
That sounds better than a separate thread IMO, a separate thread won't ever be 100% aligned to the video frame rate, unless you implement some form of thread locking, which wouldn't great.

Well, the problem seems that the input is only updated in UpdateInput and that happens at some rate. I imagine all devices have this problem (not just DSU).

However, assuming we could get around that something like what you said would be the ideal solution.

But really, as a simple solution you can just run the hotkeys at 200Hz, it's not gonna break anything.

I was thinking the thread would eat a lot more cycles but I guess I could try this. Key presses will occur more frequently which might effect sensitivity though

[Miksel12]

What do you mean checked with master? You mean there's an issue with the mouse controls too?

No, I haven't played around with gyro for some time so I wasn't sure if the behaviour I saw was purely from your update or maybe a faulty DS4.

[Filoppi]

Well, the problem seems that the input is only updated in UpdateInput and that happens at some rate. I imagine all devices have this problem (not just DSU).

Not exactly sure what you mean. DSU is the only device backend that has got a "velocity" input which you need to multiply by delta time and add to some total sum of a rotation.
When DSU updates, you can retrieve the whole list of packages that were sent over UDP from the previous update, so if you sum all all the velocity inputs there and multiply them by the delta time between packages, you are going to have the absolute angular rotation in the most accurate way you could. There is no other reason to update input more often that the game frame rate otherwise AFAIK.

[iwubcode]

@Miksel12 - at your earliest convenience, I updated the poll rate to 5ms. Please try again and see if it's any better?

[Miksel12]

Controls great again and no drift.

[Rumi-Larry]

Shouldn't "Other" be always the last one?

[MayImilae]

Gave it another try, and the drift problems are MUCH improved and the UI all still works and looks fine from the previous changes.

LGTM!

[JMC47]

Multiple Approvals and Tests. I also tested it and it was an improvement.