Raw input for mouse movement #304

Open
madeso opened this Issue Oct 18, 2012 · 35 comments

Comments

Projects
None yet
8 participants

madeso commented Oct 18, 2012

SFML currently uses WM_MOUSEMOVE and this smooths out the movement and is dependent on the resolution. Smoothing is preferred when the game integrates with the desktop. However when the mouse controls a 3d camera smoothing is less desired.

There should therefore be a option to enable non-smoothed "raw" input. The solution seems to be WM_INPUT http://msdn.microsoft.com/en-us/library/windows/desktop/ee418864%28v=vs.85%29.aspx

WM_INPUT also provides a resolution independent movement that might be good if the game needs to be run at low resolutions and/or sub-pixel movements are needed.

The forum thread where this issue originated from: http://en.sfml-dev.org/forums/index.php?topic=9426

@ghost ghost assigned LaurentGomila Oct 18, 2012

@LaurentGomila LaurentGomila removed their assignment May 19, 2014

hornta commented Jul 22, 2014

Since this is still open, I guess this feature is wanted(?).
I have managed to implement raw mouse input in the windows implementation of SFML. but I highly doubt that Laurent will accept it because everything in SFML must be cross platform, it must follow the architecture of SFML and it should also feature raw input for keyboard.
https://github.com/doodlemeat/SFML-2.1-RAWINPUT

madeso commented Jul 22, 2014

I switched over to SDL2 a while back as it already has this and more features that I want (screen queries + haptics among others), so while I don't need it anymore it still is a good thing for anyone who wants non accelerated mouse input.

Owner

eXpl0it3r commented Aug 11, 2014

@doodlemeat Thanks for the contribution, I'm sure if anyone wants to get around implementing this feature cross-platform, your code might be inspiring. 😃

@eXpl0it3r eXpl0it3r removed this from the 2.x milestone Nov 13, 2014

@madeso madeso removed this from the 2.x milestone Nov 13, 2014

@eXpl0it3r eXpl0it3r removed the s:unassigned label Jan 8, 2015

@JonnyPtn JonnyPtn referenced this issue Sep 28, 2017

Open

Initial Windows scancode implementation #1294

0 of 2 tasks complete
Contributor

JonnyPtn commented Sep 29, 2017

Using raw input for keyboard (and also joysticks I think) would be beneficial too

sro5h commented Jan 28, 2018

I'm also interested in this feature. I got a linux version and the windows implementation of hornta to work (link to my fork).
The linux version is using xinput to get the raw mouse data and is inspired by the implementation of SDL2. I basically took over hornta's MouseMotionEvent but changed the member method to setRawMouseEnabled.
I'm not that familiar with cmake and couldn't figure out how to add xinput as a dependency (You need to link the project using sfml on linux with -lXi for now). An example of how to use the feature:

#include <SFML/Graphics.hpp>
#include <iostream>

int main()
{
        sf::RenderWindow window(sf::VideoMode(400, 400), "Test");
        window.setRawMouseEnabled(true);

        while (window.isOpen())
        {
                sf::Event event;
                while (window.pollEvent(event))
                {
                        if (event.type == sf::Event::Closed)
                        {
                                window.close();
                        }
                        else if (event.type == sf::Event::MouseMotion)
                        {
                                std::cout << "Motion: x = ";
                                std::cout << event.mouseMotion.x;
                                std::cout << ", y = ";
                                std::cout << event.mouseMotion.y;
                                std::cout << std::endl;
                        }
                }

                window.clear();
                window.display();
        }

        return EXIT_SUCCESS;
}

Edit: Update repo link

sro5h commented Jan 29, 2018

I figured out how to add xinput as a dependency, you no longer need to link the final project with -lXi on Linux :)

Contributor

Foaly commented Jan 29, 2018

This is great! Thank you for contributing!

Maybe I missed something here, but I was wondering why did you chose to make another event? There is already an event for mouse motion sf::MouseMoved. I think it would be a better design to just have one event and use setRawMouseEnabled() to set the "resolution".

sro5h commented Jan 29, 2018

Thank you for your feedback.
I wasn't sure if it would make sense for some people to have access to both the raw mouse deltas and the pixel position. So i took over the the interface from hornta's Windows implementation. On the other hand one could still get the pixel position through sf::Mouse::getPosition.
I think you make a valid point and having both a MouseMotion and MouseMoved Event could lead to confusion.

Owner

mantognini commented Jan 29, 2018

Could you clarify what does setRawMouseEnabled exactly do? I haven't looked at your implementation but the header doesn't include the expected documentation. :)

sro5h commented Jan 29, 2018

Yes i haven't documented it yet, i will add some documentation comments later this evening :)
Essentially setRawMouseEnabled enables/disables the generation of MouseMoved events. In hornta's implementation for Windows it (un)registeres a RAWINPUTDEVICE and for Linux it (un)registers for the XI_RawMotion event. So you will only receive MouseMotion events if you want to.

Owner

mantognini commented Jan 29, 2018

Those are really technical details I don't understand (not being a Windows/Linux dev). ;)

So, does setRawMouseEnabled(true) mean the cursor doesn't move anymore on the screen and we get only relative movement? ("FPS-style cursor")

sro5h commented Jan 29, 2018

No it does not hide the cursor or lock it in a specific place. It gives you the relative mouse movement, even if your cursor hits the edge of your screen.
And it can be combined with setMouseCursorGrabbed and setMouseCursorVisible to hide the cursor and grab it :)

Owner

mantognini commented Jan 29, 2018

sro5h commented Jan 29, 2018

You mean i should add something like rawX and rawY to MouseMoved?
The problem is that MouseMoved uses a different event (MotionNotify vs XI_RawMotion on Linux and WM_MOUSEMOVE vs WM_INPUT on Windows) than MouseMotion. I think it would make sense to keep them seperated. Though i am not really happy with the name MouseMotion but can't think of a better one.

Owner

mantognini commented Jan 29, 2018

sro5h commented Jan 29, 2018

Ok, a MacOs implementation is still missing, do you know how to do it? I searched on the internet yesterday and found some hints but nothing concrete. But i haven't got access to a mac so i wouldn't be able to try it anyway

Yeah that sounds clearer than MouseMotion. Then i would also make sense to use a different name for the method in sf::Window. Maybe something like setMouseCursorRelative to follow the likes of setMouseCursorVisible and setMouseCursorGrabbed?

Owner

mantognini commented Jan 29, 2018

Don't worry too much about macOS, this feature is simple enough that I can provide the mac implementation when the API is stable. ;-)

About setMouseCursorRelative, I'm still wondering whether we should simply always have the feature on and remove this function altogether. What do you think?

sro5h commented Jan 29, 2018

Ok that's nice, then i can close all the chrome tabs i kept open ;D

About RelativeMouseMove: I think its kind of long and it would be nicer if it began with 'Mouse' like all other mouse related events, wouldn't it? MouseRelativeMove sounds kind of weird though

Sure, that would be the best way.If someone doesn't care about those events he can just ignore them.
If we can settle on a name for the Event i will update my fork with the points we discussed and add documentation comments :)

Owner

mantognini commented Jan 30, 2018

MouseRelativeMove or MouseMoveRelative? The latter seems more consistent with other event names but... What do you think @SFML/sfml?

Owner

eXpl0it3r commented Jan 30, 2018

Since this issue is about "raw input", is raw mouse data always relative?

I'd go for MouseMovedRelative so it's the same as MouseMoved except for the suffix. (Or would MouseMovedRaw make sense as well?

sro5h commented Jan 30, 2018

Yes it is relative

I think both would make sense.

hornta commented Jan 30, 2018

I like MouseMovedRelative. And I like to rename the event properties from x and y to something like deltaX and deltaY.

Another idea is to instead of introduction a new event, we could add two new properties to the MouseMoveEvent called deltaX and deltaY.
So when .setRawMouseEnabled(true) the MouseMoveEvent will be created from the WM_INPUT event and populate deltaX and deltaY, while still having the mouse position relative to upper left corner in x and y.

sro5h commented Jan 30, 2018

I like that too.

I'd like it more to separate MouseMoved and the raw data :)

hornta commented Jan 30, 2018

It makes sense to separate it because it is a completely different event behind, but wouldn't it be more understandable having a single MouseMove event that includes the change in the mouse movement?
I think it makes sense to abstract away the different events.

sro5h commented Jan 30, 2018

I think it does make sense to separate it because the data is different. The position is in pixel coords and the delta movement is just the raw data without any unit (am i right?). So in most cases the delta movement doesn't correspond to the new position in pixel coords due to acceleration and smoothing. That's why i think it is better to keep it separate.
Furthermore SFML stops emitting MouseMoved events if the cursor leaves the window, but (at least in my opinion) MouseMovedRaw/Relative events should still get emitted.

Owner

eXpl0it3r commented Jan 30, 2018

If the "delta" is not pixel based, then I'd go with MouseMovedRaw as it would otherwise be misleading.

Having it separated makes sense, since as said the data isn't the same and it quickly can lead to confusion.

sro5h commented Jan 30, 2018

No it gives the "logical coordinates for the mouse based on the mouse's native resolution" (stackoverflow question). For example on Windows i have mouse acceleration disabled and the mouse cursor matches my mouse movements. But if i enable it again it does not match anymore, the same is on Linux. So yes it can match but you need to explicitely disable acceleration and smoothing (and i don't know if even then it is garantueed to be the same because it could still be scaled by some amount). So yes i would definitely keep it separate to avoid confusion.

I'm gonna update my fork with MouseMovedRaw and will enable the generation of those events by default :)

sro5h commented Jan 30, 2018

Updated my fork
The event is now called MouseMovedRaw and raw mouse input will be enabled when the window is created. There is no more option to disable it. Usage example:

#include <SFML/Graphics.hpp>
#include <iostream>

int main()
{
    sf::RenderWindow window(sf::VideoMode(400, 400), "Raw Input Example");

    while (window.isOpen())
    {
        sf::Event event;
        while (window.pollEvent(event))
        {
            if (event.type == sf::Event::Closed)
                window.close();

            else if (event.type == sf::Event::MouseMovedRaw)
            {
                std::cout << "raw movement, x = " << event.mouseMoveRaw.dx;
                std::cout << " y = " << event.mouseMoveRaw.dy << std::endl;
            }
        }

        window.clear();
        window.display();
    }

    return EXIT_SUCCESS;
}
Owner

mantognini commented Jan 31, 2018

I don't know, raw doesn't really convey a meaning to me. It's used for so many things that it's too vague IMO. Relative, on the other hand, clearly identifies the data we're talking about. Maybe the data is unprocessed, but it still is relative, so I'd argue it is more appropriate here.

As for the data members, I'd either call them deltaX/Y, or simply x/y as the datatype (if we agree on relative) already holds the idea of relative distances. We haven't used abbreviations in SFML so far (right?), that's mainly why I'm objecting to dx/y.

The fact that the data is "raw", and therefore has no unit, might be a problem -- or not. I guess it all depends on how people want to use this feature, but the issue with unit-less distance is that we have no abstraction over hardware and therefore SFML cannot help developer correctly handle the data they receive. Maybe this is okay if people want to have "x/y sensibility factors" and let the users of their applications set its value for their own hardware. But maybe having a distance in actual pixel would be better?

In any case, I would include some explanation in the doc about what the data represents in an unambiguous manner.

sro5h commented Jan 31, 2018

Ok i can change it to Relative

I think so too, if we settle on MouseMovedRelative x/y should be enough.

Well i think that's exactly what this issue is about, having the mouse deltas without mouse acceleration etc. To convert it to a meaningful value in pixels you would have to take mouse acceleration into consideration again, but then this defeats the whole point of this feature. With mouse acceleration enabled you can't simply convert from the raw data to pixel values because it does not only depend on the distance the mouse covered but also the time it covered that distance in
But of course it has to be made clearer what kind of data you have to expect from MouseMoveRelative and that simply adding this relative data up might not corespond to the onscreen cursor movement.

sro5h commented Jan 31, 2018

My use case would be a fps like camera where i don't care about the unit of the data. But i would allow the user to specify a multiplier (sensitivity i guess) so he can adjust the speed of the movement to his liking.
I think that is how most games handle it, but i might be wrong.

Owner

eXpl0it3r commented Jan 31, 2018

I don't know, raw doesn't really convey a meaning to me. It's used for so many things that it's too vague IMO. Relative, on the other hand, clearly identifies the data we're talking about. Maybe the data is unprocessed, but it still is relative, so I'd argue it is more appropriate here.
The fact that the data is "raw", and therefore has no unit, might be a problem -- or not. I guess it all depends on how people want to use this feature, but the issue with unit-less distance is that we have no abstraction over hardware and therefore SFML cannot help developer correctly handle the data they receive. Maybe this is okay if people want to have "x/y sensibility factors" and let the users of their applications set its value for their own hardware. But maybe having a distance in actual pixel would be better?

If you read between the lines, you're actually saying that Raw is the better fitting word, because we do get raw, unit-less, unprocessed data.

Intuitive understanding of the property Raw Relative
Relative
Unprocessed
No specific unit
Not comparable with exiting positional data
More likely to make you check the docs to fully understand it
People who know what raw input means understand it
Same naming as the OS use

I don't see the point "normalizing" the raw data, just for the argument-sake that maybe someone doesn't know what to do with the raw data. Fact is, this is a feature that all OS (I assume) support and a feature people do want to use. It's an advanced feature for people that care about precision and continues data stream.

sro5h commented Jan 31, 2018

I totally agree, a normal user that wants relative cursor movement can store the position himself and calculate the difference since the last event.

What should we do with the members of the event? @mantognini mentioned dx doesn't go well with the existing api because it is an abbreviation, should we just call it x or go with something like deltaX to emphasize its relative nature?

Owner

mantognini commented Feb 1, 2018

I won't argue that the data need a unit -- mainly because I have no use case for that. Any sane application will let the user define their motion factor.

However, the name raw, really? I mean, if the majority wants that, sure, but...

More likely to make you check the docs to fully understand it

This is no argument for either direction. Everybody must read the documentation. Full stop. ;-)

Unprocessed

If one reads the doc, as one should, one would know what we're talking about. Raw is still ambiguous for me: it could simply ignore DPI scaling, or ignore y-axis flip, ...

People who know what raw input means understand it

People who know what relative motion in SFML is certainly understand it. This is not an argument either.

Same naming as the OS use

As far as I know, this is not a generality. And if we have a better name, we should use it instead of following head down what others did.

should we just call it x or go with something like deltaX to emphasize its relative nature?

I think the five little extra characters of delta don't hurt, but I don't have a strong opinion on this one.

[implementation details] BTW, on mac, I'm not sure we can get un-accelerated data. I've seen a few applications attempting to solve this at the system level (i.e. can work with any application), and all the graphics libraries I've looked at use the default value (i.e. probably always accelerated, unless the cursor is detached from the mouse motion, but I don't have confirmation for this). Maybe we could do something at the USB level, but it's not yet clear that it will work.

sro5h commented Feb 1, 2018

I agree with what @eXpl0it3r said but i wouldn't mind if you would better like to call it relative so i'm not going to comment on all of that :)

[implementation details] BTW, on mac, I'm not sure we can get un-accelerated data. ...

I got that feeling when i searched for it a little bit. If i am not mistaken SDL2 doesn't provide unaccelerated data on macOs either (only on Windows and Linux) 1. But there seem to be some ideas and manymouse might also be worth a look (don't know if it provides unaccelerated data on macOs, it might) :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment