ui controls

[felixroos]

as talked about in the chat, an easy API to add ui controls for params would be a lot of fun.
Here is a really basic prototype of how it could be possible to control params in real time:

https://strudel.tidalcycles.org?XliW8Yvcw99U (edit: see below for a better approach)

maybe the API could be made more logical / easier to use, something like:

note('c3').cutoff("2000".slider(200,800))

maybe this is only possible with a transpilation step.

The above link will control the params in a way that the new value takes effect on the next hap. maybe there is also a way to control params instantly while nodes are active. this would require plugging the dom events to the web audio nodes directly (and do all the plumbing in between)

[felixroos]

the ui controls could also be integrated into codemirror using decorations: https://codemirror.net/examples/decoration/#boolean-toggle-widgets

[felixroos]

another approach that works much better: https://strudel.tidalcycles.org/?STsblGtree5S

[felixroos]

"maybe there could even be helpers for each param, like lpf.slider where min, max are already set
s("bd").lpf.slider('kick_lpf', 500)` now imagine the 500 updating in the code when the slider is moved 🤯
should be possible with some ast wizardry"

[felixroos]

even simpler idea:

s("bd").lpf.ui(500)

so basically you just have to add .ui to any control.. could be tricky to get right though

[felixroos]

here's how dittytoy deals with input controls: https://dittytoy.net/ditty/80910da4d2#freq=209

[roipoussiere]

This is a very nice idea!

Among all the proposals, I personally prefer the syntax .lpf(slider('kick_lpf', { min: 200, max: 8000, init: 800 })).

I don't like the idea of updating the code when we touch the ui, it's really confusing: it's like playing a videogame that can change the code of the videogame. This circular behavior (code updates ui and ui updates code) will probably break things at some point. Also if I share my music in a repo I don't the files to be updated each time I play it.

.lpf.ui(500) seems too ambiguous: what if instead lpf we have a function that takes several parameters?

And parsing comments is not as powerful as having functions in the code, because it's not possible to have the slider itself defined dynamically.

I would also suggest to have a method on the slider object to define a range after its initialization, in order to have one slider updating several parameters. Something like that:

my_slider = slider('My slider') // default: min=0, max=1, init=0.5

s("bd sd, hh*8")
  .lpf(my_slider.range(200, 8000, 800))
  .lpq(my_slider.range(0, 20, 10))
  .gain(my_slider.range(0, 1, 0.5))

[felixroos]

i guess the comment "// default: min=0, max=1, init=0.5" is just for illustrative purposes and not a feature?
I like the idea of having one slider controlling multiple things. It also reminds me of how signals work, maybe they should share the same api:

signals:

s("bd sd, hh*8")
  .lpf(saw.range(200, 8000))
  .lpq(saw.range(0, 20))
  .gain(saw.range(0, 1))

sliders:

let my_slider = slider(.5).label('My Slider')

s("bd sd, hh*8")
  .lpf(my_slider.range(200, 8000))
  .lpq(my_slider.range(0, 20))
  .gain(my_slider.range(0, 1))

Besides getting the APIs the same, the third parameter of range also wouldn't work because you could set different values at the same time (at least if my_slider is really one slider and not multiple ones). The thing returned from slider(.5) could even be modified with pattern methods:

let my_slider = slider(0.5).add("0 .5")

So slider would behave like a signal that holds a primitive numerical value. You could maybe even do

let my_slider = slider("0 .5")

or

let my_slider = slider(saw)

in the latter 2 examples, the slider in the ui could actually move in real time

[roipoussiere]

i guess the comment "// default: min=0, max=1, init=0.5" is just for illustrative purposes and not a feature?

Yeah forget about this comment, doesn't really make sense. What I meant is that the slider object could return a value in the range [0, 1] and then this value can be scaled with .range().

let my_slider = slider(.5).label('My Slider')

Why is the advantage of chaining functions instead of using 2 arguments like slider('My Slider', .5)?

the third parameter of range also wouldn't work because you could set different values at the same time

Indeed!

in the latter 2 examples, the slider in the ui could actually move in real time

Is it just for fun or there are real use cases for updating ui controls in real time? It could be tricky to "catch" the slider with the mouse if it's moving every time. ^^

If the goal is to visualize values (ie. for debugging), I would prefer to have dedicated ui elements for that.

[felixroos]

Why is the advantage of chaining functions instead of using 2 arguments like slider('My Slider', .5)

label could be optional + the api is more similar to strudel in general

Is it just for fun or there are real use cases for updating ui controls in real time? It could be tricky to "catch" the slider with the mouse if it's moving every time. ^^

not sure if this is really needed. I was thinking of how it works in some daws with automations.
If they don't move, passing a pattern should probably be forbidden, otherwise the slider position would not represent the value that is used

[yaxu]

A vote for having UI integrated with the code. It's really nice when the code represents the whole state of what's running, there's less to remember and think about, and you don't have to context switch between the code and the 'UI pane'. After all in strudel, the code is the UI.
It would probably be nice if a widget stops moving on mouseover or other interaction though.

[roipoussiere]

I created a prototype for checkboxes, based on your previous prototype: https://strudel.tidalcycles.org/?6Tno91ORQaq4

We define the initial checkbox state, as well as values for both states:

.lpf(checkbox('kick_lpf', { val_off: 2000, val_on: 8000, init: true }))

[roipoussiere]

I think we should have something like pick or similar, that acts like range, but using predefined discrete values instead a range:

s("bd sd,hh*4").cutoff(sine.pick([500, 1000, 2000]).slow(4))

then use it with our checkbox:

s(my_cb.pick("bd sd", "hh*8"))
  .lpf(my_cb.pick(2000, 8000))

So we could eventually have multi-states buttons:

s(my_cb.pick("bd sd", "hh*8", "bd*4"))
  .lpf(my_cb.pick(2000, 4000, 8000))

There is probably a better word, make_discrete is more accurate but not as short.

[roipoussiere]

Oh I just found choose in the docs, pretty close to what we need no?

s(my_cb.choose("bd sd", "hh*8"))
  .lpf(my_cb.choose(2000, 8000))

[felixroos]

what does sine.pick([500, 1000, 2000]) do? how does a range work with 3 values?

[roipoussiere]

I suppose we can split the input range with several ranges of equal sizes, then we link those ranges to the predefined values.

So with 3 values, ranges are [0, 1/3[ , [1/3, 2/3[ , and [2/3, 1].
if n in [0, 1/3[, out = 500
if n in [1/3, 2/3[, out = 1000
if n in [2/3, 1], out = 2000

For a simple sine.pick([0, 1000]), it will be:
if n in [0-0.5[, out = 0
if n in [0.5-1], out = 1000

[yaxu]

In Tidal this is called select. There is also selectF for choosing a function to apply to a pattern.

selectF (slow 8 sine) [id, rev, fast 2] "bd sd"

I'm not sure how best to do that with method chaining, maybe:

"bd sd".selectF([id, rev, fast 2], sine.slow(8))

but then it looks a lot different from select/pick.. but these are a bit confusing:

sine.slow(8).selectF("bd sd", id, rev, fast 2)
sine.slow(8).selectF("bd sd", [id, rev, fast 2])

There's also pickF that works the same but indexing by int rather than float, which is also useful. pick was taken for something else..

[yaxu]

Some proposed live coding UI principles:

• The code is the user interface. Any graphical elements should be part of the code as the primary and only user interface.
• The user interface should be all about helping integrate the programmer into the program, so that they can live code.
• The user interface should be accessible from the keyboard, as much as possible.
• Debugging is a disaster - if you need to debug, it means you have to stop making music because the user interface is bad. Things should be simple/clear enough that any errors are more like 'bum notes', you can just embrace them by repeating them.
• The aim isn't to make correspondence between code and output totally clear - the great thing about patterns is that the code tends to be much simpler than the output. But it should be clear what can be changed next.

There could also be UI for manipulating generative music pieces, potentially with some shared resources, but I think it's best to treat that as a separate issue, as the design issues will be totally different.

[felixroos]

potentially related: #646

[felixroos]

https://github.com/ircam-ismm/sc-components

[felixroos]

here is a new version of the slider demo, using ref + scaling the value range as with any pattern: https://strudel.tidalcycles.org/?C6ybeBTc_DC9
I like the fact that slider just returns a pattern of numbers, which minimizes the number of things you need to know..
maybe the slider ids could also be implicit, though it wouldn't be possible to control multiple values with the same slider if there isn't a way to specify a key..

the arguments could also be flipped to slider(init, id) where id is optional. if id is not set, the slider could just receive a hash as id, although this might be tricky to retain between evaluations.

optional ids might be a bit too complicated, maybe id is just required and also acts as label?!

edit: using ids as labels seems to be a good idea: https://strudel.tidalcycles.org/?TkZDXKgvmMMy

although labels might be less relevant when using https://codemirror.net/examples/decoration/#boolean-toggle-widgets which are inline in the code, so a label might not be required

[felixroos]

another demo with more sliders: https://strudel.tidalcycles.org/?MVpaxiZ0LNDL

[roipoussiere]

Thanks for all those nice examples.

I wonder if it makes sense to have the ui elements in a CM gutter.
At least it's less disturbing for me because it doesn't mess the code and the ui elements.

Here is a mockup (made with Gimp, not coded):

So ui elements must be small to avoid having a large left bar (hence the use of a radial slider).

We also have to think about a behavior if there are multiple sliders on one line (the easiest is probably to enlarge the left bar).