Add hardware cursor support #9

Closed
thefiddler opened this Issue Nov 25, 2013 · 11 comments

Comments

Projects
None yet
3 participants
@thefiddler
Contributor

thefiddler commented Nov 25, 2013

Continuing from here: http://www.opentk.com/node/3443?page=1

This post describes how to implement hardware cursors on Windows: http://www.mimicprod.net/2010/09/hardware-cursor-in-opengl/

SDL provides a cursor API, but the documentation states that it is black-and-white only (it is possible that the documentation is out of date here.)

We still need to find out how this works on X11 and Carbon.

Ideally, our public API will take an array of 32bit RGBA values with a specific layout. This way we remain agnostic to any platform-specific cursor format (e.g. windows .ico files, .png images etc etc) and we avoid System.Drawing which is broken and not portable.

thefiddler added a commit that referenced this issue Feb 18, 2014

[Core] Added MouseCursor class
This branch marks the beginning of hardware cursor support (affects
issue #9)

thefiddler added a commit that referenced this issue Feb 18, 2014

@Frassle

This comment has been minimized.

Show comment Hide comment
@Frassle

Frassle Feb 21, 2014

Contributor

Did some work on Windows support. https://github.com/Frassle/opentk/tree/cursor
Not quite working yet, only seems to show the top scanline of the cursor and switches back when the mouse buttons are held down. But it has some of the infrastructure in place to support Windows cursors and I also added a simple example that just sets the cursor from a png. I'll pull request once I've fixed the bugs.

Contributor

Frassle commented Feb 21, 2014

Did some work on Windows support. https://github.com/Frassle/opentk/tree/cursor
Not quite working yet, only seems to show the top scanline of the cursor and switches back when the mouse buttons are held down. But it has some of the infrastructure in place to support Windows cursors and I also added a simple example that just sets the cursor from a png. I'll pull request once I've fixed the bugs.

@thefiddler

This comment has been minimized.

Show comment Hide comment
@thefiddler

thefiddler Mar 18, 2014

Contributor

I'm having some byte-order issues on the SDL2 implementation. I'm gradually adding the necessary byte swapping functionality to make this work.

Contributor

thefiddler commented Mar 18, 2014

I'm having some byte-order issues on the SDL2 implementation. I'm gradually adding the necessary byte swapping functionality to make this work.

thefiddler added a commit that referenced this issue Mar 19, 2014

thefiddler added a commit that referenced this issue Apr 28, 2014

[Core] Added MouseCursor class
This branch marks the beginning of hardware cursor support (affects
issue #9)

thefiddler added a commit that referenced this issue Apr 28, 2014

@thefiddler

This comment has been minimized.

Show comment Hide comment
@thefiddler

thefiddler Apr 28, 2014

Contributor

Initial support for switching between MouseCursor.Default and MouseCursor.Empty is now merged.

What we are now missing is support for byte-swapping (Rgba vs Bgra vs Argb) and a proper implementation for Cocoa. [NSCursor set] does not work until you move the mouse, so we probably need to override [NSView resetCursorRects]

@Ollhax any ideas?

Contributor

thefiddler commented Apr 28, 2014

Initial support for switching between MouseCursor.Default and MouseCursor.Empty is now merged.

What we are now missing is support for byte-swapping (Rgba vs Bgra vs Argb) and a proper implementation for Cocoa. [NSCursor set] does not work until you move the mouse, so we probably need to override [NSView resetCursorRects]

@Ollhax any ideas?

@Frassle

This comment has been minimized.

Show comment Hide comment
@Frassle

Frassle Apr 28, 2014

Contributor

I'm having some byte-order issues on the SDL2 implementation. I'm gradually adding the necessary byte swapping functionality to make this work.
support for byte-swapping (Rgba vs Bgra vs Argb)

What exactly is the issue here? Looking at the SDL docs color cursors are initialized by a surface that should define the pixel format.

Contributor

Frassle commented Apr 28, 2014

I'm having some byte-order issues on the SDL2 implementation. I'm gradually adding the necessary byte swapping functionality to make this work.
support for byte-swapping (Rgba vs Bgra vs Argb)

What exactly is the issue here? Looking at the SDL docs color cursors are initialized by a surface that should define the pixel format.

@Ollhax

This comment has been minimized.

Show comment Hide comment
@Ollhax

Ollhax Apr 28, 2014

Contributor

Haven't touched those parts, so no clue I'm afraid :(

Contributor

Ollhax commented Apr 28, 2014

Haven't touched those parts, so no clue I'm afraid :(

@thefiddler

This comment has been minimized.

Show comment Hide comment
@thefiddler

thefiddler Apr 28, 2014

Contributor

@Frassle if you use a System.Drawing.Bitmap with PixelFormat.Format32bppArgb, you get back pixels laid out as bgra. On Windows, CreateIconIndirect expects bgra so this works. Other platforms may expect different formats, so you need to know what the original format was and byte-swap as necessary (like SDL does in SDL_CreateSurface.)

We could always place this responsibility on the user ("cursors are argb little-endian, have fun figuring that out") or we could implement this internally. Seeing how much trouble I had getting the correct layout for SDL (it's still not correct in the repository), I'm inclined to go for the latter.

@Ollhax np, I've managed to get a custom cursor to saw up, but I didn't have any luck overriding resetCursorRects. I added:

Class.RegisterMethod(windowClass, new ResetCursorRectsDelegate(ResetCursorRects), "resetCursorRects", "v@:");

but this function was never called.

Edit: now that I see this, it could be that resetCursorRects is called on the contentView, not the window itself.

Edit 2: this is what I am trying to achieve:

- (void)resetCursorRects
{
    [super resetCursorRects];
    if(drag) {
        [self addCursorRect:self.bounds cursor:[NSCursor closedHandCursor]];
    }
}
Contributor

thefiddler commented Apr 28, 2014

@Frassle if you use a System.Drawing.Bitmap with PixelFormat.Format32bppArgb, you get back pixels laid out as bgra. On Windows, CreateIconIndirect expects bgra so this works. Other platforms may expect different formats, so you need to know what the original format was and byte-swap as necessary (like SDL does in SDL_CreateSurface.)

We could always place this responsibility on the user ("cursors are argb little-endian, have fun figuring that out") or we could implement this internally. Seeing how much trouble I had getting the correct layout for SDL (it's still not correct in the repository), I'm inclined to go for the latter.

@Ollhax np, I've managed to get a custom cursor to saw up, but I didn't have any luck overriding resetCursorRects. I added:

Class.RegisterMethod(windowClass, new ResetCursorRectsDelegate(ResetCursorRects), "resetCursorRects", "v@:");

but this function was never called.

Edit: now that I see this, it could be that resetCursorRects is called on the contentView, not the window itself.

Edit 2: this is what I am trying to achieve:

- (void)resetCursorRects
{
    [super resetCursorRects];
    if(drag) {
        [self addCursorRect:self.bounds cursor:[NSCursor closedHandCursor]];
    }
}
@Frassle

This comment has been minimized.

Show comment Hide comment
@Frassle

Frassle Apr 28, 2014

Contributor

We could always place this responsibility on the user ("cursors are argb little-endian, have fun figuring that out") or we could implement this internally. Seeing how much trouble I had getting the correct layout for SDL (it's still not correct in the repository), I'm inclined to go for the latter.

I'd be inclined to think putting this on the user is the correct thing to do. We don't do format conversions anywhere else, if a user asks for an RGBA GL texture and then passes in ABGR they get the wrong result. As long as we well document the MouseCursor constructor I don't think people will struggle too much, also if we set the expected format to be whatever the default Drawing.Bitmap format is then for most people it will just work. We can then handle converting from one defined format to a platform dependent format in the Cursor setter.

I also can't see any way that the library could be expected to work out the byte order given any arbitrary user input anyway.

Contributor

Frassle commented Apr 28, 2014

We could always place this responsibility on the user ("cursors are argb little-endian, have fun figuring that out") or we could implement this internally. Seeing how much trouble I had getting the correct layout for SDL (it's still not correct in the repository), I'm inclined to go for the latter.

I'd be inclined to think putting this on the user is the correct thing to do. We don't do format conversions anywhere else, if a user asks for an RGBA GL texture and then passes in ABGR they get the wrong result. As long as we well document the MouseCursor constructor I don't think people will struggle too much, also if we set the expected format to be whatever the default Drawing.Bitmap format is then for most people it will just work. We can then handle converting from one defined format to a platform dependent format in the Cursor setter.

I also can't see any way that the library could be expected to work out the byte order given any arbitrary user input anyway.

@thefiddler

This comment has been minimized.

Show comment Hide comment
@thefiddler

thefiddler Apr 28, 2014

Contributor

I was thinking about adding an extra parameter to the constructor similar to PixelFormat.Rgba / Bgra or, alternatively, exposing bitmasks similar to SDL. However you are probably right, matching System.Drawing.Bitmap would work perfectly for most users, and the rest would just have to match PixelFormat.Format32bppArgb, which is equivalent to OpenGL PixelFormat.Bgra, which is laid out as 0xAARRGGBB in memory. I think.

I must say that I do dislike System.Drawing and would like to get rid of its perma-broken libgdiplus dependency eventually - but that's a different discussion entirely.

Contributor

thefiddler commented Apr 28, 2014

I was thinking about adding an extra parameter to the constructor similar to PixelFormat.Rgba / Bgra or, alternatively, exposing bitmasks similar to SDL. However you are probably right, matching System.Drawing.Bitmap would work perfectly for most users, and the rest would just have to match PixelFormat.Format32bppArgb, which is equivalent to OpenGL PixelFormat.Bgra, which is laid out as 0xAARRGGBB in memory. I think.

I must say that I do dislike System.Drawing and would like to get rid of its perma-broken libgdiplus dependency eventually - but that's a different discussion entirely.

@Frassle

This comment has been minimized.

Show comment Hide comment
@Frassle

Frassle Apr 28, 2014

Contributor

I must say that I do dislike System.Drawing and would like to get rid of its perma-broken libgdiplus dependency eventually - but that's a different discussion entirely.

As part of OpenTK or as a separate library referenced by OpenTK? I saw there was some discussion on the MonoGame repo about something similar.

Contributor

Frassle commented Apr 28, 2014

I must say that I do dislike System.Drawing and would like to get rid of its perma-broken libgdiplus dependency eventually - but that's a different discussion entirely.

As part of OpenTK or as a separate library referenced by OpenTK? I saw there was some discussion on the MonoGame repo about something similar.

@thefiddler

This comment has been minimized.

Show comment Hide comment
@thefiddler

thefiddler Apr 28, 2014

Contributor

Either would be fine by me.

OpenTK already provides dummy implementations for System.Drawing Point, Size, Rectangle and Icon. You can add MINIMAL to your preprocessor definitions and compile OpenTK without System.Drawing. This will disable support for window icons, but should otherwise work.

In the mid-term, I would like to remove all System.Drawing calls from the platform backends. This is why, for example, OpenTK.MouseCursor inherits from OpenTK.WindowIcon rather than System.Drawing.Icon. The public API would remain as-is for compatibility, but with this in place, removing System.Drawing would become a matter of deciding when to flip the switch.

Contributor

thefiddler commented Apr 28, 2014

Either would be fine by me.

OpenTK already provides dummy implementations for System.Drawing Point, Size, Rectangle and Icon. You can add MINIMAL to your preprocessor definitions and compile OpenTK without System.Drawing. This will disable support for window icons, but should otherwise work.

In the mid-term, I would like to remove all System.Drawing calls from the platform backends. This is why, for example, OpenTK.MouseCursor inherits from OpenTK.WindowIcon rather than System.Drawing.Icon. The public API would remain as-is for compatibility, but with this in place, removing System.Drawing would become a matter of deciding when to flip the switch.

@thefiddler

This comment has been minimized.

Show comment Hide comment
@thefiddler

thefiddler May 2, 2014

Contributor

Closed via #106

Contributor

thefiddler commented May 2, 2014

Closed via #106

@thefiddler thefiddler closed this May 2, 2014

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment