Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider support for different cursor shapes #1551

Closed
marmarek opened this issue Dec 27, 2015 · 18 comments
Closed

Consider support for different cursor shapes #1551

marmarek opened this issue Dec 27, 2015 · 18 comments
Labels
C: gui-virtualization help wanted P: major release notes T: enhancement ux
Milestone

Comments

@marmarek
Copy link
Member

marmarek commented Dec 27, 2015

https://groups.google.com/d/msgid/qubes-devel/20151226141511.GA95308%40cowbell.employees.org

Something I've noticed when using a web browser is that the cursor shape never changes,
always staying as a simple arrow pointer.

Having control over cursor shape it may mislead the user actions. While it shouldn't be a problem inside the same VM, it may be complex to do it right - to not interfere with windows of other VMs.
In short: without prior proper design, which would be simple and simple to implement, we don't want to have such feature, at least not by default.

But generally it is something to consider, because:

Cursor shape is a functional part of the UI, to the point of being critical
for certain apps like spreadsheets and drawing. If Qubes deems this a possible
source of confusion then it should explain the threat model for cursors
(having already done so for windows).

@marmarek marmarek added T: enhancement C: gui-virtualization P: major release notes labels Dec 27, 2015
@marmarek marmarek added this to the Release 3.2 milestone Dec 27, 2015
@bnvk bnvk mentioned this issue Dec 27, 2015
20 tasks
@mfc
Copy link
Member

mfc commented Jan 4, 2016

at congress it was suggested that cursor changing be allowed when a vm is in full-screen mode as a UX/security compromise.

@bnvk
Copy link

bnvk commented Jan 4, 2016

While it shouldn't be a problem inside the same VM, it may be complex to do it right - to not interfere with windows of other VMs.

Getting the cursor to change within the same VM will be a significant UX improvement. A whole range of GUI apps like graphic design programs, IDE's etc... all suffer from this fixed pointer cursor which make simple things like re-sizing a window pane hard to do!

at congress it was suggested that cursor changing be allowed when a vm is in full-screen mode as a UX/security compromise.

Disallowing in full-screen, but allowing in non-full-screen mode, is still a significant UX improvement. However, if we are able to (and proceed) in doing that, we should try to trigger an overlay that communicates this to user when toggling between full-screen and non mode!

@v6ak
Copy link

v6ak commented Jan 8, 2016

In-window cursors

What about just providing a way to hide the cursor when it is on app's window? Custom cursors would be then implemented on top of this feature in the VM agent.

  • + Simple design for dom0.
  • + Cursors do not open a way to show arbitrary content outside of the window, as they are shown within the window only.
  • - Having mouse at edge of some window would cause clipped cursor.

Cursor classes

Another option is having some predefined cursor types (e.g. hand, standard pointer, cross, …).

  • + Simple design for dom0.
  • + Cursors do not open a way to show arbitrary content outside of the window, as they come form dom0 and are trusted.
  • + Unifies cursor set.
  • + No issues with cursor at edge.
  • - No support for custom cursors.
  • - Might require some more effort for various systems in AppVMs.

@marmarek
Copy link
Member Author

marmarek commented Jan 8, 2016

On Fri, Jan 08, 2016 at 12:43:44PM -0800, Vít Šesták wrote:

In-window cursors

What about just providing a way to hide the cursor when it is on app's window? Custom cursors would be then implemented on top of this feature in the VM agent.

  • + Simple design for dom0.
  • + Cursors do not open a way to show arbitrary content outside of the window, as they are shown within the window only.
  • - Having mouse at edge of some window would cause clipped cursor.

From the security point of view, I like this idea. But from the
usability POV I'm afraid it would be horrible. Even small lag of cursor
position would be really frustrating. In early Windows Tools we had both
cursors visible (dom0 and windows) and it was bad - VM cursor was
lagging, even when click events were delivered correctly.

Also some technical problem would be that cursor is not part of the
window composition buffer (this is what is directly mapped as window
content in dom0). So it would be quite complex at VM side - not
affecting security, but most likely performance.

Cursor classes

Another option is having some predefined cursor types (e.g. hand, standard pointer, cross, …).

  • + Simple design for dom0.
  • + Cursors do not open a way to show arbitrary content outside of the window, as they come form dom0 and are trusted.
  • + Unifies cursor set.
  • + No issues with cursor at edge.
  • - No support for custom cursors.
  • - Might require some more effort for various systems in AppVMs.

I know exactly nothing about cursor shapes in X11 protocol. If it is
possible to extract such information, it would be good. If not, that
would require some guessing (OCSR - "optical cursor shape recognition"
;) ), not reliable. Even if that would be possible only at some higher
level (like for GTK based applications), that would be still an
improvement.

Since it is possible to set cursor theme globally, I guess it is
possible to extract abstract information about desired cursor shape.

Quick duckduckgo returns this:
http://stackoverflow.com/questions/6751928/set-x11-cursor-to-arrow

Looks promising. Especially the part about setting it as window property

  • it would mean that X server would handle switching its shape, so no
    need to do that manually on enter/leave events (which would be really
    fragile, probably a showstopper).

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

@marmarek marmarek added the help wanted label May 4, 2016
@marmarek marmarek modified the milestones: Release 4.0, Release 3.2 May 4, 2016
@andrewdavidwong
Copy link
Member

andrewdavidwong commented May 29, 2016

andrewdavidwong added a commit that referenced this issue May 31, 2016
andrewdavidwong added a commit that referenced this issue Jun 7, 2016
@andrewdavidwong andrewdavidwong added the ux label Mar 6, 2017
@marmarek marmarek modified the milestones: Release 4.1, Release 4.0 Jul 31, 2017
@Joeviocoe
Copy link

Joeviocoe commented Sep 20, 2017

Does this issue being labeled "Help Wanted" mean that it has the lowest priority and nobody from the dev team is working on it?

@v6ak
Copy link

v6ak commented Sep 20, 2017

@andrewdavidwong
Copy link
Member

andrewdavidwong commented Sep 20, 2017

Does this issue being labeled "Help Wanted" mean that it has the lowest priority and nobody from the dev team is working on it?

It does mean that the Qubes devs do not expect to have time to work on it in the foreseeable future. However, "priority" is somewhat orthogonal. An issue can have a high priority even if the Qubes devs can't work on it. In that case, we really want someone to help! 🙂

@t4777sd
Copy link

t4777sd commented Apr 19, 2019

Is there any plans to have this for 4.1? Would increase usability for a lot of apps and web pages.

@marmarek
Copy link
Member Author

marmarek commented Apr 19, 2019

It is something that indeed would align very well the goals of 4.1 (improving usability).

Right now, there is (at least) one technical difficulty: how to get current cursor shape. I can find a dozen of ways to set the cursor shape (libXcursor, XCreateWindow parameter, XSetWindowAttributes, ...), but it looks like none of them provide matching functionality to retrieve what was set.
I've also tried to look at X server sources, and it looks like very few places read this property and none of them are available for applications. While we have technical means to access X server internal structures, I'd really like to avoid going this way...

Any ideas?

@t4777sd
Copy link

t4777sd commented Apr 21, 2019

I spent a couple hours searching to see if I could find any projects that retrieved that information using some type of generalize-able methods and I could not identify any easy out of the box way to do it either, unfortunately.

@a-barinov
Copy link

a-barinov commented Oct 27, 2019

It is something that indeed would align very well the goals of 4.1 (improving usability).

Right now, there is (at least) one technical difficulty: how to get current cursor shape. I can find a dozen of ways to set the cursor shape (libXcursor, XCreateWindow parameter, XSetWindowAttributes, ...), but it looks like none of them provide matching functionality to retrieve what was set.
I've also tried to look at X server sources, and it looks like very few places read this property and none of them are available for applications. While we have technical means to access X server internal structures, I'd really like to avoid going this way...

Any ideas?

Since there is no obvious way to retrieve cursor shape the only option seems to be intercepting cursor-setting calls mentioned above using ld_preload. Pretty sure a few intercepts and qubes gui protocol extension can solve this.

@ninavizz
Copy link
Member

ninavizz commented Dec 19, 2019

Oyy, this is important. Flagging to possibly discuss in a forthcoming UX meeting #5523 .

@ninavizz
Copy link
Member

ninavizz commented Dec 19, 2019

Hey gang: So... I just had a lovely ice-bucket moment, realizing there is no multi-cursor state support within VMs, on Qubes. I'd created this issue for the SecureDrop workstation client, which I just learned we cannot implement. So: sad-panda for me, but I wanted to ask y'all... how feesable is it to do this, and might it be possible to prioritize in a forthcoming release?

As a global usability thing, it's a pretty big deal.

As with my other requests to date, purely making them as "nina," not as "person on SecureDrop team."

@ninavizz
Copy link
Member

ninavizz commented Dec 19, 2019

^ Within the linked issue there, is Mozilla's standard guide for how cursors should behave via CSS protocols. Those standards apply similarly to client apps, too, even though in Linux-land those standards are probably applied few and far between.

pwmarcz added a commit to pwmarcz/qubes-gui-common that referenced this issue Mar 27, 2020
See QubesOS/qubes-issues#1551.

The message contains a cursor ID to be set for a given window.
pwmarcz added a commit to pwmarcz/qubes-gui-daemon that referenced this issue Mar 27, 2020
pwmarcz added a commit to pwmarcz/qubes-gui-agent-linux that referenced this issue Mar 27, 2020
See QubesOS/qubes-issues#1551.

I use the Xfixes extension to subscribe to cursor change events.
See SelectCursorInput and CursorNotify in the Xfixes specification:

  https://cgit.freedesktop.org/xorg/proto/fixesproto/plain/fixesproto.txt

The returned "cursor name" can be empty, and in general doesn't
have to be meaningful. However, it practice the names are standard
X11 "cursor font" names:

  https://tronche.com/gui/x/xlib/appendix/b/

One exception to that I found is Chromium, which uses its own set
of cursor names. We can recognize these as well.
pwmarcz added a commit to pwmarcz/qubes-gui-common that referenced this issue Mar 27, 2020
See QubesOS/qubes-issues#1551.

The message contains a cursor ID to be set for a given window.
pwmarcz added a commit to pwmarcz/qubes-gui-daemon that referenced this issue Mar 27, 2020
pwmarcz added a commit to pwmarcz/qubes-gui-agent-linux that referenced this issue Mar 27, 2020
See QubesOS/qubes-issues#1551.

I use the Xfixes extension to subscribe to cursor change events.
See SelectCursorInput and CursorNotify in the Xfixes specification:

  https://cgit.freedesktop.org/xorg/proto/fixesproto/plain/fixesproto.txt

The returned "cursor name" can be empty, and in general doesn't
have to be meaningful. However, it practice the names are standard
X11 "cursor font" names:

  https://tronche.com/gui/x/xlib/appendix/b/

One exception that I found is Chromium, which uses its own set of
cursor names. We can recognize these as well.

Use binary search (bsearch() from stdlib) for looking up the cursor
by name, to avoid traversing ~100 entries one by one.
pwmarcz added a commit to pwmarcz/qubes-doc that referenced this issue Mar 27, 2020
@pwmarcz
Copy link

pwmarcz commented Mar 27, 2020

Peek 2020-03-27 16-10

@kajmagnus
Copy link

kajmagnus commented Sep 5, 2020

@pwmarcz — that looks nice :- ) does that (your screencast) mean that different cursor shapes are available today in Qubes 4.1?

Anyone knows if there're any plans to backported to 4.0 some day?

Or, would it be a crazy idea to upgrade to 4.1, just to get these custom cursors, although 4.1 it's not yet released? Maybe I could do that on 1 out of 2 laptops, hmm.

( Currently I try to resize different windows in an IDE, click and drag the border between the windows ... doesn't work, I try again and again, but fail to hit the tiny press-&-drag-mouse-resize area, ... I give up, and continue working although the inside-IDE window is too small. )

@pwmarcz
Copy link

pwmarcz commented Sep 5, 2020

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C: gui-virtualization help wanted P: major release notes T: enhancement ux
Projects
None yet
Development

No branches or pull requests