suggestion: knob keeps focus until mouse up / touch end

[hamoid]

Hi, thanks for sharing your knob :)

There's one feature that I find not very convenient: "Pulling mouse / touch outside the element before release restores back to old value.". The smaller your knob, the easier it is to leave the element by accident and revert the changes.

My plan is to change the behavior so the knob does NOT calculate the angle between the center and the mouse pointer / finger. Instead, clicking the knob sets it as focused, then I would just drag up / down to set the value (not caring about if the mouse / finger is inside the element or not). When releasing the value stays. A bit like with https://nexus-js.github.io/ui/

One drawback: there's no way to cancel (unless I detect the ESC to cancel).

What do you think of this approach? Would you use this as an alternative mode for pure-knob?

[hamoid]

Maybe it would be nice to have a boolean setting for the auto-restore value on mouse/finger exit, and another one to set the way it calculates the value (using polar coordinates or cartesian). Polar (current) is probably easy when the knob is large, but cartesian (y axis for instance) is easier with smaller knobs.

update: maybe https://github.com/nexus-js/ui/blob/master/lib/interfaces/dial.js#L23 can serve as an example, just found it.

[andrepxx]

Hey @hamoid!

Actually, there were a few reasons why the control is implemented this way, so please let me quickly elaborate on them.

1. "Angle for values" is intuitive: The reason why this control was incepted in the first place was to replace jQuery Knob by Anthony Terrien, which is no longer maintained. jQuery Knob uses the angle of the pointer, relative to the control's center, for values, so I wanted to do the same.

2. There is no (tidy) way to capture events outside your control: The event handlers are bound to the canvas element and will no longer fire as the cursor is moved outside the element. As a workaround, one might bind additional event handlers to the document root element, but this is problematic for several reasons.

2.1 - The document root element is normally "contended". Other code may bind event handlers at the document root element as well, which might interfere with our event handling. It might remove our event handlers from the document root or it might bind additional event handlers which interfere with ours by capturing events or "re-throwing" them or whetever.

2.2 - Also, events "bubble up" the DOM tree from the elements on which they were captured (the knob's canvas elements) up to the parent node and its parent and so on, until they arrive at the document root. Therefore, not only may another event handler at the document root capture events before we do it, but in fact any event handler at any node in the DOM tree in between the knob's canvas element and the document root may capture them.

2.3 - If we capture events at another DOM element (e. g. the document root), the coordinates will be relative to that particular element instead of our control. To get proper coordinates (relative to the actual knob element), we might have to perform lots of subtle coordinate transformations, which might be hard to get right across platforms and browsers.

2.4 - Therefore, capturing events "outside" our control is, technically, a really bad idea. And if we only capture events on our control, then we won't get notified about mouse movements that happen outside of our control. Period. There's no way around that. It's just how the event system works.

2.5 - Okay, so we definitely won't know what the mouse does when it's outside of the control. So what do we do if the cursor leaves the control? Well, originally, I did not reset the value, which led to the behaviour that, when the user enters a drag action inside the control, then moves the cursor out with the mouse button still depressed, the knob will change values (but not fire an event, since the mouse button was not released) until the cursor leaves the control and then ... well, the control will stop responding, since it no longer receives mouse move events, but it will not notify "upstream" code that the knob value was changed, since that only happens on mouse release, but the mouse was never released inside the control, which leads to a very confusing user experience, where the knob displays a value that the "upstream" code was never even notified about, so UI and "business logic" have become "out of sync". This was the actual reason why I introduced the "leaving the control resets the value to the last committed (i. e. the last one the business logic was informed about) value" behavior.

3. For "fine adjustment" of values, you can just use the scroll wheel (or, depending on your operating system or window manager, two-finger scrolling or similar gestures) or just double-click (this sets an "arbitrary" value first, since a double-click is also a single-click) or middle-click (this does not set an "arbitrary" value) or double-tap (sets "arbitrary value" since a double-tap is also a single-tap) the control and then use keyboard (on PCs) or on-screen keyboard (on mobile devices) to enter a precise value.

Especially the controls you mention I find quite "unintuitive" to use. I just tried them out and cursor movement and behaviour of the actual knob felt very "decoupled" a lot of times. I can figure out approximately what they're doing, but it certainly doesn't feel natural.

However, that is not to dismiss the idea. I just wanted to mention that we have to think thoroughly about what behaviour "makes sense" for the knob and that we should also get it to work well across platforms and on both mouse-operated devices (PCs) and touch-enabled devices (mobile phones / tablets).

I suppose if we were to use horizontal / vertical movement, we better not use the "absolute" cursor position (inside the knob) for setting a value, but we rather make it a thing like "every so many pixels dragged upwards increments the value by 1, every so many pixels dragged downwards decrements the value by 1". Just keep in mind that this is currently the behaviour which the knob shows for the scroll wheel (and emulated scroll events from trackpads on laptops, etc., though especially the direction of scroll is, in this case, implementation-dependant, especially depending on the window manager used).

Regards.

[hamoid]

Thank you, yes, I see this requires a lot of thinking and testing to get right. I also think that maybe there is no one-solution-fits-all.

My use case is real time controlling of graphics and sound. So I want to trigger updates continuously. When dragging near the center of the knob, the resolution is low and it's hard to set precisely. Maybe you go from 0 to 1 in 10 steps. But if you are far from that center, near the visible knob ring, then maybe you have 200 steps.

Another issue is, maybe moving the mouse in a circular fashion maps intuitively to a knob, but I suspect not every person is as capable of moving the mouse in a circle without leaving the canvas. Moving it straight up and down is much easier (or with the mousewheel, which works great!). I do see the complications of document-attached mouse events.

Currently the value is only set when releasing the mouse, so the lower resolution near the center is not a big deal, when compared to setting values in real time, in which case the value would jump greatly with single pixel motions.

With the vertical or horizontal motion approach (instead of in a circle) I only need to get the first click right (to target the knob) and then I could close my eyes and listen or look at the resulting effect while I drag up and down.

I'm writing this comment just to mention different use cases. Currently I have figured out how to make it work for me in my live performance. The only reason I'm not using nexusui at the moment is that they implement web audio to do precise timing in their step-sequencer widget, which produces high CPU usage. I don't need any timing, so I was looking at a minimal knob I could use in my personal project.

[andrepxx]

I also use the knob in an audio application, but my application is different. It is used to process signals from real instruments (e. g. electric guitar) in real-time. For this application, parameter update does not have to be real-time. A guitarist normally doesn't stand in front of his amp and turns knobs while playing. He plays the guitar, needs both hands for that - and, every now and then, adjusts his amp / effects to get a different sound. So "non-realtime control" is perfectly okay for that application. And it can't be real-time anyways, since I'm processing audio in blocks. The processing starts when a full block was captured and it processes the entire block as fast as it can. The processing is only "aligned" to the block boundary, but not to the sample clock within that block. The processing therefore happens in "burts".

1. Capturing a few hundred / thousand samples ...

2. Capturing is finished: Process all of them as quickly as possible!

3. Play back the result while capturing the next few hundred / thousand samples ...

4. Go back to step 2.

Because the control is not required to be in real-time, it is actually decoupled from the actual signal processing by putting the control into a different thread than the actual signal processing and by putting the UI into an entirely different process than the DSP application. The UI process communicates with the DSP process by exchanging JSON messages back and forth over a TLS connection. If I were to do "real-time" updates from the UI, I would literally "congest" the inter-process communication. Also, I think (though I'm not sure) that individual TLS messages do not even necessarily have to arrive in-order. (I don't think TLS guarentees order of messages.) So if I update too fast, it might be that the inter-process communication delivered my messages in a different order and the most recently sent value is not even the last that will be received and then, of course, things would get very messy. (I would have to include sequence numbers in the messages to make sure that at least the last value set prevails.)

Therefore, I deliberately chose NOT to update the values in real-time. I even have a delay for the scroll wheel event where the event subscriber is only notified of the value update after the scroll wheel "stands still" for a couple of milliseconds, so that I don't "spam" the inter-process communication too much.

However, I don't think that's a problem. If this is an "alternate mode" of the knob, upstream code just has to be aware of the fact that it will probably receive a lot of events in a short amount of time, and can then choose how to cope with that in the most appropriate manner. And if you have to do IPC, like I do, then you certainly don't want to notify the other process at every event, but if you do something else, then it may actually be feasible to take every value change into account.