Knobs are erratic on macOS (and other systems)

[tresf]

As the title suggests, all knobs behave "wonky" on macOS.

Originally posted on Reddit, the OP (u/Coelacanth2) was kind enough to share a screen record. Holding Shift helps, but they're still way off.

I can reproduce this on my machine, macOS Mojave 10.14.6 using the latest tip of stable-1.2 and all dependencies updated. Note, the change in behavior from older versions may be Qt5 related.

[michaelgregorius]

Is this about the behavior that if you move the mouse faster then the values will increase in greater steps? Because if it is then it's not only a problem on macOS.

[tresf]

Is this about the behavior that if you move the mouse faster then the values will increase in greater steps? Because if it is then it's not only a problem on macOS.

@michaelgregorius it may be the same underlying issue, but on macOS, it's so sensitive that it makes the knobs practically unusable, similar to trying to draw 1 pixel at a time with Gimp zoomed all the way out gives almost no room for control. Worse, it continues goes in the wrong direction when switching.

[michaelgregorius]

Thanks for the clarification @tresf. I notice two things in the screen recording which might indicate differences in Qt's implementation on macOS compared to for example Linux:

1. The mouse cursor stays visible on macOS when changing the values. According to the implementation it should be hidden as can be seen in Knob::mousePressEvent where the method QApplication::setOverrideCursor( Qt::BlankCursor ) is called.
2. The mouse cursor changes its position when the value is changed on macOS. Normally the method Knob::mouseMoveEvent tries to suppress this cursor movement by calling QCursor::setPos( mapToGlobal( m_origMousePos ) ). I guess this is done so that the changing of the value is restricted by reaching a screen boundary.

So it seems that macOS restricts the capability for manipulation of the mouse cursor and perhaps this somehow breaks the implementation of the knob on macOS.

[PhysSong]

So it seems that macOS restricts the capability for manipulation of the mouse cursor and perhaps this somehow breaks the implementation of the knob on macOS.

It seems like macOS 10.14 requires the user to grant the "control the computer" permission to an app in order to use QCursor::setPos(). See a comment on QTBUG-75786 for details.

[tresf]

@PhysSong (as usual) good find. That fixes it.

Unfortunately, it's simply not going to be a long-term solution for macOS 10.14+. Can we compromise on a solution that doesn't require bumping up against accessibility options?

Here's some example of knob UIs that are what I would consider to be more standard. http://anthonyterrien.com/demo/knob/.

Although rewriting the knobs is too large of a task for our 1.2. branch, we need a way to make the UI usable on 10.14 without hidden settings, even if that means the mouse moves around when it's used.

[tresf]

So I've removed the calls to QCursor::setPos and QApplication::setOverrideCursor/restoreOverrideCursor but the bug still happens. Is there other code that interacts with the processing of drag events that's not obvious or not in Knob.cpp?

[tresf]

Ok, I see what's going on. The relative position is 100% reliant on the fact that the mouse keeps getting moved back.

Somewhat working example is to change this code:

- QCursor::setPos( mapToGlobal( m_origMousePos ) );
+ m_origMousePos = _me->pos();

The downside is the cursor eventually hits the edges of the screen requiring subsequent click/release.

This code also severely exposes @michaelgregorius' point about the small drags having nearly no effect on the knob.

It's arguably worse than driving into accessibility settings.

[PhysSong]

Here's some example of knob UIs that are what I would consider to be more standard. http://anthonyterrien.com/demo/knob/.

IIRC @gi0e5b06 made his fork use that kind of behavior, but I don't think that was better than the LMMS/lmms implementation(just a personal preference).

FYI, QCursor::setPos( mapToGlobal( m_origMousePos ) ); is also known to cause issues with Wacom tablets(see #4376).

[Spekular]

Having to "spin" the knob with your mouse seems gimmicky and far from standard to me, I can't recally any software I've used that behaves like that. Standard to me is a knob that reacts to up/right as increase, and down/left as decrease.

…

On Tue, Sep 24, 2019, 10:30 Hyunjin Song ***@***.***> wrote: Here's some example of knob UIs that are what I would consider to be more standard. http://anthonyterrien.com/demo/knob/. IIRC @gi0e5b06 <https://github.com/gi0e5b06> made his fork use that kind of behavior, but I don't think that was better than the LMMS/lmms implementation(just a personal preference). FYI, QCursor::setPos( mapToGlobal( m_origMousePos ) ); is also known to cause issues with Wacom tablets(see #4376 <#4376>). — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#5194?email_source=notifications&email_token=ACEBLGSKKVB76MV3JJAXIP3QLHF2XA5CNFSM4IXF6RR2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7NRSHY#issuecomment-534452511>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACEBLGQLSDJHOV5WHNU2UA3QLHF2XANCNFSM4IXF6RRQ> .

[michaelgregorius]

I agree with @Spekular. I have tried some VSTs in the past that implemented that "you have to move the mouse pointer in a circle exactly along the knob" behavior and they were very tedious to use, i.e. I did not use these plugins. 😄

If I remember correctly I even checked in the past whether it is possible to get "raw" mouse input, e.g. only relative movements a la "the mouse moved a delta of x upwards" but I did not find anything. Having such raw data would do away with the need to move the mouse pointer back during each event.

[tresf]

I'm testing out garageband and it's just up/down as we do now, not left/right.

Carla is up/down AND left/right.

@falkTX can we use your knob code? It feels fine to me and Carla works just fine with Mojave, Catalina, etc.

[PhysSong]

This is how LMMS' knobs calculate the value change:

lmms/src/gui/widgets/Knob.cpp

Lines 507 to 515 in 57a486c

	value = ( ( _p.y() + _p.y() * qMin( qAbs( _p.y() / 2.5f ), 6.0f ) ) ) / 12.0f;
	// if shift pressed we want slower movement
	if( gui->mainWindow()->isShiftPressed() )
	{
	value /= 4.0f;
	value = qBound( -4.0f, value, 4.0f );
	}
	return value * pageSize();

Here _p is the coordinate difference between two mouse move events, and +/-100 corresponds to the difference between the minimum value and the maximum value. I think the intention is to change the value depend on the speed of the mouse movement as well as the distance of the mouse movement. In contrast, Carla doesn't have such a feature.
Here's the graph of the function:

Also, I can't even change the value of a knob with the shift key pressed. I'm on Ubuntu 18.04 and using the builtin Synaptics touchpad.

[Spekular]

Not being able to move the knob when shift is pressed is probably not our bug. In my experience, laptops won't allow trackpad input while a key is held. If you can test with a USB mouse you'll likely find that shift + drag works.

[tresf]

In my experience, laptops won't allow trackpad input while a key is held. If you can test with a USB mouse you'll likely find that shift + drag works.

You have it backward. The "Palm Detect" feature of trackpads will temporarily disable the trackpad from moving the cursor when the keys are being pressed, not the other way around. I agree that @PhysSong's issues are probably environmental, but I strongly disagree that Shift modifier is expected to malfunction when using a trackpad.

[PhysSong]

Strangely, sliders work fine with the same modifier key. 🤔

[soundconjurer]

I am having this issue with Fedora 31 64-bit, with flatpak.

LMMS 1.2.1 from the Flatpak, the knobs just go all the way with one or the other with the tiniest move of the mouse.

LMMS 1.2.1 from the AppImage does not do this.

[JohannesLorenz]

Summary

By now, LMMS resets the cursor, which is relevant for the increasement speed formula. Tres suggested a patch where the cursor is moved and the formula is valid, but the disadvantage is that reaching the screen borders will limit you, so you need multiple drags to use a knob. This screen border limitation seems the main problem.

Proposal

Make one mouse direction invert the movement when a knob is close to the the top right or down left screen edges. E.g. if the knob is too much down left, a movement up to the top left corner could translate to "down", and a movement to the bottom right corner could translate to "up".

Proposed because I did not know how LMMS knobs worked until now and I never made any assumptions which direction does increase and which does decrease. I always moved the knob a bit, and if it worked, I moved the mouse on. So learning how to use a knob is individual for some users. It's similar to window context menus opening top left to the window if the window is almost completeley down right.

There may be other solutions like using some grain control (using keys 0-9 or arrow keys), but I'm against making control too grain, because many knobs control volume and you don't want to blow up your speakers. So if we want to stick to fine control, in the worst case, the described two long mouse directions is all we have.

[Spekular]

If we really are limited to screen bounds (which sounds strange to me, never had this issue in any software on windows nor linux), couldn't we also make the knob movement proportional to screen size?

I'm thinking

• Dragging the mouse up by one screen height moves the knob one full rotation
• Dragging the mouse right by one screen width moves the knob one half rotation

In this way, vertical movement serves as a fast adjustment (shorter distance* maps to big range) and horizontal serves as a fine adjust (wider distance* maps to smaller range).

A knob perfectly centered on the screen could be adjusted 75% in either direction in one go. This range gets skewed the further you get towards the top right/bottom left, but if this is unacceptable perhaps the ranges could be tweaked. If a vertical movement of one screen height translates to two knob rotations, and the knob isn't in the top or bottom 25% of the screen, you'd never need more than two adjustments to reach any point.

*Assuming wide monitor. Perhaps instead of the vertical/horizontal distinction, it should be based on the screens dimension. So, perpendicular to the long edge is fast adjust, parallel to the long edge is fine adjust.

[tresf]

Here's how GarageBand does it. Works fine.

The edges of the screen are annoying, but it's fine. People can click twice for those (metaphorical and literal) edge-cases.

[tresf]

Dragging the mouse up by one screen height moves the knob one full rotation
Dragging the mouse right by one screen width moves the knob one half rotation

I'd advocate that it's exactly consistent. Changing sensitivity on direction is clever, but not intuitive. GarageBand only allows up/down for knobs and it feels natural, I've never had an issue with it.

In regards to screen size, I'd vote against any screen magic. It should feel identically sensitive across all screens. If a screen is too small to move the knob a full rotation, the interface is suffering in many other ways already. 😄

[JohannesLorenz]

Zyn also only uses up/down. But if we offer additional left/right, that's no disadvantage.

I conclude if we take your suggested patch, tres, then the issue would be solved "well enough"? If a user still has too many "edge cases" they can simply move the window...

[tresf]

I conclude if we take your suggested patch, tres, then the issue would be solved "well enough"? If a user still has too many "edge cases" they can simply move the window...

Agreed although to be clear, I have provided no patch. I attempted to fix this using code, but was unable to, so I need help implementing this assuming it's to be expedited.

If the code changes are too large to introduce to stable-1.2, we can milestone it for master instead. In its current state, the knobs are unusable and macOS fails to raise the security dialog for the workaround, so if master is targeted, I'd still like to treat it as priority (as well as nightly downloads) so end-users have a quick fix which makes the software usable again.

[JohannesLorenz]

@tresf Can you please checkout branch #5194-setpos from my github (git@github.com:JohannesLorenz/lmms.git) and check

1. What the command line outputs?
2. If the behavior of setPos/hide looks right to you?

[tresf]

1. What the command line outputs?

setPos working? false
setPos working? false
setPos working? false
... (message shows exactly once per new knob turn)

2. If the behavior of setPos/hide looks right to you?

1. Well, when I added build/lmms to my list of trusted applications that can control the computer, the message didn't change, so I'm not sure if the testing technique is correct.
2. I wouldn't expect each knob to need it's own detection algorithm. It seems like one time would be sufficient.

[JohannesLorenz]

when I added build/lmms to my list of trusted applications that can control the computer, the message didn't change

Was the cursor back on the knob when you released the key?

I wouldn't expect each knob to need it's own detection algorithm.

Good point. I forgot to make the variable static.

[tresf]

Was the cursor back on the knob when you released the key?

No, it was working as intended, but when I enabled the option to control the cursor, it still detected as being off, so that's a bit concerning to me that your detection technique is not working.

[JohannesLorenz]

Sorry, I don't get what you say 😄 After you enabled the option to control the cursor, was the cursor back on the knob right after releasing?

[tresf]

After you enabled the option to control the cursor, was the cursor back on the knob right after releasing?

No, after enabling the option to control the cursor the behavior did not change.

[JohannesLorenz]

In the same LMMS process? On the same knobs?

[tresf]

I restarted LMMS and it made no difference.

[JohannesLorenz]

OK that would mean that you can tell MacOS to allow to set the cursor, but QCursor::setPos() does not change the position, at least not immediatelly. In this case, I think I must ask on SO.

[jcelerier]

I have the exact same problem on OSSIA/score - I'll try to reduce and make a bug report. The impression I get is that setPos is asynchronous on macOS and synchronous (or at least much less laggy) on other platforms.

[jcelerier]

looks a lot like this issue actually : https://bugreports.qt.io/browse/QTBUG-67639

[tresf]

looks a lot like this issue actually : https://bugreports.qt.io/browse/QTBUG-67639

Interesting, thanks for sharing. If that's the case, that would suggest that setPos() will work as a unit test so as long as we don't send it a float, no?

[jcelerier]

Can indeed fix my issue by calling the CG functions manually with floating point values - I do in the mousePressEvent

// m_viewport is a QGraphicsView, m_startPos a QPointF, ev a QGraphicsSceneMouseEvent*
m_startPos = m_viewport->mapToGlobal(QPoint{0,0}) + ev->pos();

and in the mouseMoveEvent

#if defined(__APPLE__)
    CGPoint ppos;
    ppos.x = m_startPos.x();
    ppos.y = m_startPos.y();

    CGEventRef e = CGEventCreateMouseEvent(nullptr, kCGEventMouseMoved, ppos, kCGMouseButtonLeft);
    CGEventPost(kCGHIDEventTap, e);
    CFRelease(e);
#else
    QCursor::setPos(m_startPos);
#endif

and things finally look less crazy - hopefully that can be useful to you !

[JohannesLorenz]

There is already an open PR #5360 . Unless anyone vetos in the next days, I'll close this issue to avoid confusion/redundancy.

[Spekular]

Aren't issues typically left open until their fixes are merged? If the fix PR fails to solve the issue, or gets closed without merging, you want the original issue open so the bug isn't incorrectly marked as fixed.

[JohannesLorenz]

Also true. I'll leave it open.

IIRC @tresf did that in the past a couple of times, so I thought it was common 😄

[tresf]

If I prematurely closed any bug reports because of active PRs, it was years ago -- and in hindsight -- premature. 😁