x/exp/shiny: cursor API proposal

[sqweek]

The mouse cursor is a humble little pointer. It never needs to change for correct program operation, but the subtle feedback provided by doing so can translate to a great improvement to the user experience.

To start the conversation I propose a simple API along the lines;

// Cursor may identify a pre-defined system cursor or custom (user-created) cursor
type Cursor int

const (
    UnspecifiedCursor Cursor = iota
    NormalCursor
    CrosshairCursor
    IBeamCursor
    // you get the idea -- the full set of portable cursors is discussed below
}

interface CursorCreator {
    // CreateCursor gracefully degrades if eg. a coloured image is provided but the
    // driver only supports greyscale/monochrome cursors
    CreateCursor(img image.Image, hotspot imagePoint) Cursor
}

interface CursorSetter {
    SetCursor(c Cursor)
}

The underlying platforms do things a bit differently. In windows, changing the cursor affects the entire desktop, but users are advised to be courteous and "set the cursor shape only when the cursor is in its client area or when the window is capturing mouse input"[1]. In OSX, changing the cursor affects only the calling application (and all windows it has open). In X11, changing the cursor is a window-specific operation, and only affects the cursor while it is over the window in question. X11's API suggests SetCursor must be implemented by shiny.Window.

[1] https://msdn.microsoft.com/en-us/library/windows/desktop/ms648393(v=vs.85).aspx

It might be useful for SetCursor to return the previously active Cursor. But since the cursor is application-local in X11/OSX and best practice in windows is to try and avoid affecting other applications, in most circumstances the caller already knows the cursor's previous state (because it was responsible for the previous change, if any).

Portable cursors:

Name	WIN32	OSX	X11	Notes
Normal	👍	👍	👍
Crosshair	👍	👍	👍
IBeam	👍	👍	👍
Hand	👍	👍	👍	OSX has both open and closed variants
NotAllowed	👍	👍		(but X11 does provide a space shuttle!)
Busy	👍	[2]	👍
ResizeN	[3]	👍	👍
ResizeS	[3]	👍	👍
ResizeE	[3]	👍	👍
ResizeW	[3]	👍	👍
ResizeNS	👍	👍	👍
ResizeEW	👍	👍	👍
ResizeNE	[3]	[4]	👍
ResizeSE	[3]	[4]	👍
ResizeSW	[3]	[4]	👍
ResizeNW	[3]	[4]	👍
ResizeNESW	👍	[4]
ResizeNWSE	👍	[4]

[2] OSX's "busy" spinning wheel cursor is automatically displayed by the system if the application's main loop stops responding. There is no API allowing the application to set this cursor on request. Modern versions of windows have similar automatic behaviour which hints at consensus that setting the cursor to a busy icon has been decided to be a poor ux -- it may not be worth providing the busy cursor.

[3] Windows just has the four double-sided resize arrows - NS, EW, NESW, and NWSE. These can be reused to cover the N, S, E, W, NE, NW, SE, SW cases.

[4] OSX has diagonal resize cursors but they are not easily available to applications. possible workaround: http://stackoverflow.com/questions/10733228/native-osx-lion-resize-cursor-for-custom-nswindow-or-nsview . Alternately it may be worth automatically generating custom cursors to cover both these and X11's lack of NotAllowed cursor.

I gathered this information while implementing a cursor API for go.wde, primarily through API docs and explorative code. I do not have a wide experience with graphics APIs in general.

[sqweek]

Functions to hide/show the cursor would be straightforward to add.

In the interest of providing some context to the proposed API, there are some other ideas floating around this space:

• X11 allows you to specify a cursor to be used for the duration of a mouse grab
• Cocoa allows you to associate different cursors with different areas (rectangles) of the window's client area, alleviating the burden of watching mouse coordinates from the application programmer

I don't know how cursor changes interact with openGL, if at all...

[sqweek]

@nigeltao not sure if this is an inappropriate place for such a proposal or whether it simply hasn't been seen?

[nigeltao]

It simply hasn't been seen.

I am on extended leave, for non-work-related reasons, and am not reading my e-mail every day, let alone scanning the issues list regularly.

My thoughts are this are similar to that on issue #13339:

I'd like to avoid incrementally adding a dozen odd SetThis and SetThat methods to the Window interface, one by one in an ad hoc manner. The NewWindowOptions type already lists things like fullscreen, title, icon, cursorHidden in its TODO comments. There are undoubtedly more we could add. I'd like to have some sort of principled approach to where we draw the line.

One consideration is that not all systems support cursors, whether pre-defined system cursors or custom ones. For example, mobile does not.

[sqweek]

Thanks so much for taking time out of your leave to respond Nigel. I hope all is well!

Your concerns make a lot of sense - I'll revisit this proposal after looking into NewWindowOptions and giving cursorless systems some thought.

[bradfitz]

To @nigeltao for a decision.

[sqweek]

I'm yet to have a play with shiny but I remain interested in contributing, for the record :)

[nigeltao]

I interpret Brad's ping to mean that this formal Proposal needs a Decision on whether or not to proceed.

My decision is to reject the proposal as is, for the reasons I gave earlier: I'd like to avoid incrementally adding a dozen odd SetThis and SetThat methods to the Window interface, one by one in an ad hoc manner, and instead I'd like a bigger picture design. I'd also like some more thought for how a cursor API would work (or not work) on mobile.

I'm therefore closing this issue. Nonetheless, the ability to set a cursor is a legitimate feature request, albeit low priority, and probably lower the more prevalent touch screens become. Feel free to re-open the issue if you have an updated proposal that addresses those reasons for rejection.

[sqweek]

While I agree that the prominence of touch screens means cursors are less common, it remains a higher priority for me personally as when a cursor is present I consider its appearance a crucial part of the user experience. But I appreciate this may not lie within the primary goals of shiny.

Regarding a new proposal, is NewWindowOptions still the route you would recommend? Is there any other precedent in shiny for getting at backend-specific features? Although this doesn't quite fit there as eg. an android tablet will not usually have a cursor, but if you plug a usb mouse in suddenly it's there...

[nigeltao]

Well, I don't really have a concrete API suggestion. I was just saying that any cursor API proposal should probably be harmonious with any NewWindowOptions fields and the general concept of window attributes like full-screenness and titles. But that's a bigger (unsolved) problem. Sorry.