dynamic global clock source

[catfact]

there are a number of musically useful ways that lua callbacks could be synchronized to various clock sources:

• internal, using the metro system; these are high-priority threads using nanosleep, should be maximally accurate

• MIDI clock (24 ppqn)

• sample clock; we could get a timestamped bundle from supercollider every N samples; this will have some jitter in the 100s of usecs range, but no drift

• ableton link (???)

create a Clock module or something, allowing scripts to conveniently select from these

[Dewb]

I took a stab at a lua-side clock here (in dust, maybe it should be here instead.)
monome/dust#135

Lua seems to work pretty well for 24ppq, but rebroadcasting external clock to other connected devices might be better done on the native side.

[Dewb]

Here's a draft straw-man proposal for a native clock system. This is going to be simultaneously inappropriately maximalist, poorly informed, and a little obvious, but just to push the conversation:

UI:

• add a System > Clock submenu with the following items:
  • clock master: one of "internal", "midi", or "ableton link"
  • tempo: master tempo bpm (ignored if clock master is midi)
  • running: true/false, to start and stop clock (ignored if clock master is midi)
  • reset phase: command to reset bar/beat phase
  • midi clock receive: submenu of connected midi devices, allowing one to be picked as 24ppq source (ignored if master is not "midi")
  • midi clock send: submenu of connected midi devices to toggle on/off for sending 24ppq clock regardless of master clock mode

Lua/script layer:

• Scripts can instantiate Lua clock objects based on the master clock, but with their own bar/beat quantum counter. Resetting the phase or adjusting the tempo of the master clock will apply to the phase and tempo of dependent lua clocks. Clock objects can trigger script callbacks on step, beat, bar divisions, and master start/stop/reset events. Clock objects can also be constructed with their own bpm to operate completely independently of the master clock.
• Global master clock APIs clock.setBPM, clock.start/stop, clock.reset (or similar) that scripts can use to control master bpm and reset beat/phase with knobs/buttons/params. (setBPM, start, and stop would be ignored if the master clock source is midi.)

matron/C layer:

• master clock object implemented in C
• state of master clock UI options are stored in and drives behavior of matron clock
• uses a 24ppq metro to schedule clock ticks. metro timing is adjusted in response to midi or link events. clock ticks are sent out to selected midi devices regardless of master mode.
• clock is informed of incoming hardware midi clock messages from devices. clock ignores midi clock messages from devices other than selected clock master (Should this use the event system or skip it?)
• clock registers callbacks with Ableton Link client instance and inspects Link session (if active) for accurate time/phase when triggering ticks
• clock pushes new event CLOCK_START when clock starts, affecting Lua clocks
• clock pushes new event CLOCK_STOP when clock stops, affecting Lua clocks
• clock pushes new event CLOCK_TICK (or CLOCK_UPDATE?) when clock ticks, containing a data structure with tempo, beat-phase, time information. This triggers an update function on Lua clock objects synced with master, allowing those clock objects to trigger the appropriate step/beat/bar callbacks in scripts.

SuperCollider layer:

• ???
• It would be useful if the timestamps generated by C clock (especially the ones generated by Ableton Link) could be carried through Lua layer into SC. Scripts could get timestamp information in their clock callbacks but I'm not clear on what the pattern would be for scripts to communicate that to engines
• It could also be useful for SC to be explicitly aware of bar/beat/phase info for things like tempo-synced delay etc. That may be the responsibility of an individual script, though.
• Midi clocking and Ableton Link could be implemented inside SC instead of the C layer, unsure what pros/cons would be (one pro is that timestamps might not have to round-trip through Lua, one con is that clocking would have to be reimplemented in hypothetical future non-SC engines.)
• Ableton Link can take advantage of information about the audio pipeline latency so audible events land in sync, rather than just syncing internal triggers. Would be fantastic if there's an automated way to estimate that, but could naturally be very tricky in a fully programmable system.

[Dewb]

Re: Ableton Link, after thinking about this some more, and catching up on the discussion on the SC Link pull request, I lean towards relying on the SC implementation of Link once it lands (hopefully in 3.11?) supercollider/supercollider#3351

As an experiment, though, I'm throwing the Link library into the C layer to see if I can work Link into the existing BeatClock system.

[okyeron]

pinging this thread regarding link support.

A report from a colleague:

SuperCollider 3.10.1 successfully built for RaspberryPi…
…and the still-in-development Ableton Link code, also working on the Pi!

[Dewb]

I've started some of the prep work for Link support, but it touches a lot of things that are changing in 2.0, so I've put that work on hold until after 2.0 is finished.

Still on the fence about whether it makes more long-term sense to use the SC Link implementation or do a direct integration of the Link C library into matron. In the current state of the system, timing is a responsibility of the scripting world.

[catfact]

fwiw, i think it would be better to put Link support in the C components if possible. we're moving further in the direction of making norns ecosystem independent of supercollider; SC becoming one of many options for DSP engine interfacing with the scripting / timing layer.

see also PR 712 #712

which makes steps to support lua code execution as coroutines, with worker threads under the hood.

[artfwo]

aye, the clock module support for syncing to external events is in progress. link should fit perfectly as long as it's implemented in matron.

[mimetaur]

I saw this pull request and from some recent experiments with particle synthesis I can see big benefits to an approach that gets clocks off the main lua thread.

I’m trying to run some metros in the 100+ calls per second range and it’s interfering with the menu system running smoothly and my ability to update the script via Maiden. (And I don’t think it’s what’s executing in that callback, because CPU is reporting ~17%.)

[catfact]

@NathanKoch while i agree that this is a good idea (servicing clock callbacks from worker thread), i also wonder if this isn't the event queue itself getting bogged down.

the CPU reporting on norns is average of all 4 cores, so isn't really granular enough to see if things are stalling in matron's event queue servicing. this is a general area that will get a lot more close attention for 2.1.

ii am surprised that maiden is affected though... what is in the callbacks? if you are printing lots and lots of stuff, then maiden will be having to deal with, i dunno, constantly updating the layout of the REPL or something... getting a bit OT there.)

[mimetaur]

I’ll leave my (terribly un-optimized, with a ton of garbage collection from these particles being spawned and killed) code here: https://github.com/nathankoch/NornsMachines/blob/master/grains/simplegrains.lua

(It’s basically a lua port of standard Processing/oF style particle systems as found in Daniel Shiffman’s Nature of Code.) What I’ll probably do is dig into this more, do some optimizations and isolate the problem better in fresh code - then create a separate issue.

(I don’t see any rogue print statements but I can’t rule out one slipping in. I know for a fact that tons of printing stalls out/overwhelms maiden‘s REPL but I’d call that expected behavior.)

Good to know also that the CPU is reporting across all cores.

[pq]

I know for a fact that tons of printing stalls out/overwhelms maiden‘s REPL but I’d call that expected behavior.

known (and unfortunate behavior). i could have sworn we had a bug tracking this but can't find it... @ngwese?

[mimetaur]

I wonder if it would be worth investigating a solution like what Unity 3D’s console does, where multiple identical prints to the console (or multiple identical errors getting thrown) just get an incrementing number tacked next to one line.

(But I’m getting way OT from this original issue here)

[Dewb]

@artfwo finally took a look at your coroutine clock PR #712. I'm about ready to start working on Ableton Link again. If I keep working on the plan described in #129 (comment), will that be compatible with what you're doing, or does the plan need to change? (Along with the obvious changes for post-2.0 UI.)

[artfwo]

@Dewb not exactly, the clock module already provides beat syncing coroutine execution scheduler with arbitrary multiplication and division. Currently it only has one clock source - tempo, MIDI and link are in progress.

To push the clock forward you'll need to update the master clock rather frequently using the "app" (thread-safe) link methods. And then call clock_update_reference_from(<beat>, <tempo>, CLOCK_SOURCE_LINK). The API will provide a method to select source eventually. I have also started some groundwork on the link bindings for C here (please feel free to send code): https://github.com/artfwo/link-c

And here's a crude matron clock source for link (not currently pushed because it's a horrible hack, please also feel free to contribute): https://gist.github.com/artfwo/2de822c5b6389d310299d6f29440a458

And here's a test script for the clock API:
https://gist.github.com/artfwo/115a7129b461f3f07fae5016ae15d048

[Dewb]

Awesome! Looks like you're well on the way, then, I'll try to build your WIP branch + gists.

[artfwo]

For building gists, add link-c and link as submodules to a subdirectory in norns and compile ableton_link.cpp as a static lib or just add it to the list of matron sources - waf will automatically build the object using c++ compiler and link it with the matron binary. Link.hpp should also be in the include path.

[Dewb]

Updating last year's straw man proposal to adapt to 2.0 changes and @artfwo's coroutine clock system.

UI:

• add a System > Clock submenu (or should these go in System > Audio?) with the following items:
  • clock master: one of "internal", "midi 1", "midi 2", "midi 3", "midi 4", or "ableton link"
  • tempo: master tempo bpm (read-only if clock master is midi)
  • running: true/false, to start and stop clock
  • reset phase: command to reset bar/beat phase
  • midi clock send: one of "off", "midi 1", "midi 2", "midi 3", "midi 4", or "all" for sending 24ppq clock regardless of master clock mode. (If master is not "internal" or "link", send "all" should not send to the clock source.)

(more to come)

[okyeron]

I would be excited about all of that.

What can I do to help? (probably in the testing and breaking things dept)

[artfwo]

stopping and resetting is currently not possible with the clock module, but is possible to implement in theory - it will require rescheduling sleeper threads when start/stop/reset command is received or there is a significant tempo change, for example.

[tehn]

@Dewb I agree, system wide clock is the future.

i am hoping to work on the param menu expansion soon but we can hack in the clock system ahead of that is needed.

[okyeron]

(scope creep) perhaps a "transport control" UI widget for start/stop/reset? I could see this being super useful with MIDI and Link when norns is the master clock.

[tehn]

looking at this again as i'm planning to integrate it into my in-process param menu overhaul.

i'll start work on a system-wide clock that (yikes) basically is feature-overblown (due to all of the sync possibilities i guess)

the param submenu system should make this clean and navigable though.

[Dewb]

Clock section in params sounds great.

I got a little ways towards prototyping a global clock before putting it on hold when the menu revamp started. Let me know if there’s a section of work that could be broken out that I could help with.

I think the biggest remaining issue in adopting the coroutine system for a global clock is having some sort of cancellation system to retrigger waiting coroutines when the master tempo changes. Right now going from a slow tempo to a fast tempo can give you events much later than expected.

[artfwo]

I think this is not a really big issue - abrupt tempo changes from 20 to 200 are rare and you still have enough flexibility from the script to keep the situation more or less under control.

For example, instead of calling clock.sync(100) it's possible to use a loop: for i=1,100 do clock.sync(1) end in scripts that expect the tempo to change quite often. Retriggering waiting coroutines is also possible from the script of course.

[tehn]

menu revamp shouldn't block progress on a clock class--- i'm just hoping to get config options for the clock in the menu. all of these should be configurable with class functions as well.

@Dewb would be happy to build upon what you've started

(and also, i'm looking to finish this menu revamp quite soon)

[tehn]

if anyone has begun work on this, could you make your branch visible?

there's interest from several people to work on this, but (my lack of) coordinating is probably confusing everyone. i am more than happy to have someone take the lead on this.

[tehn]

this is effectively in place. minor polish and fixes remain.