feature request: (semi) fixed latency

[magnetophon]

Currently, B.Schaffl updates the latency as you tweak the sliders.
That way it becomes impossible for the DAW to compensate while you tweak.
You have to restart the transport for the timing to be correct, preventing you from effectively tweaking timing related sliders while other tracks are playing.

It would be great if B.Schaffl had a fixed latency, but that would mean either the latency is too high, or the pattern can only be tweaked within a certain range.

A possible compromise would be for the user to set the maximum latency, preferably in bars/beats.
Ideally there would be a snapping effect of the max latency slider, when it is close to the latency needed for the current pattern.

I would like to stress again how much fun this thing is!!! 😄
Have you had a chance to give it a good workout yet, or am I taking up all of your free time with my requests and bugs? ;)

[sjaehn]

Thanks for your remarks :-).

Funny, I had the same idea before I started the project. I also thought about the problems of a user-defined latency. Such as invalid slider and marker settings. Anyway, I've no idea why I forgot to add this to TODO.

This feature will come ...

[sjaehn]

For the first, I added a user-defined latency option in 62b1704 . In frames. Other units will follow.

Attention, this feature reveals a bug. First note is played plus latency, last note is played minus latency. I've to check why.

[sjaehn]

First note bug fixed in 606ea2f . It was quite easy to find the cause. Latency was updated only once at the beginning for each run() call. If playback is halted, latency is 0. If the first note came in in the same run as playback starts, the latency still stuck at 0.

I'm thinking about the last note of the loop bug. There are no suspicious dump data. Looks like a host feature...

[magnetophon]

The above bugs are all fixed now, as far as I can tell, thank you!

Great progress also with the filtering, work wonderful.

In the latest version, I did find a different latency related bug.
When I turn on user defined latency, with a latency higher then what was calculated for the automatic setting,, the pattern changes, quite dramatically.

I have not been able to reproduce it in a simple demo setting yet, but I do have a fairly small Ardour6 project that reproduces it.

May I recommend you install Ardour 6 anyway?
It is so much improved over Ardour5. A joy to use.

BSchaffl.zip

[magnetophon]

I made a short video demonstrating the problem.
I think it's just that the first note of the loop is put in the wrong level bucket.
BSchafflVid.zip

[sjaehn]

Thanks for the video. I can hear the difference, but it's too fast for my ears to isolate the troublemaker. I'll check the dump to understand where the problem comes from.

[sjaehn]

Different tracks with different instances of B.Schaffl and different latencies. And a lot of notes ;-). Therefore I get a lot of console messages (also for the muted tracks of course) showing these different latencies. I don't know if this may caused the problem

I'll try to reproduce and analyze it in a simplified setting.

[sjaehn]

One step back. I tried to reproduce your observation with your SchafflDemo (#1).

In Ardour 6: If I set user latency to some 1 s and start playback, the playback line starts immediately to run. Sound appears after 1 s. Latency is shown in Ardour's plugin window frame. There apparently is no difference if I set latency compensation to zero in Ardour's plugin window frame.

In Ardour 5: If I set user latency to some 1 s and start playback, the playback line rests for some 1 s before it starts to run. Sound appears at the same time as the playback line starts to run. Latency is not shown in Ardour's plugin window frame.

And SchafflDemo sounds differently in Ardour 6 compared to Ardour 5.

It looks like Ardour 6 shows but doesn't handle MIDI latency?! I can't believe it. Did I miss something new in Ardour 6 ?

[magnetophon]

I had noticed the visual difference between A5 and A6, but I hadn't noticed the difference in sound, good catch!

There apparently is no difference if I set latency compensation to zero in Ardour's plugin window frame.

No difference where?

It looks like Ardour 6 shows but doesn't handle MIDI latency?! I can't believe it. Did I miss something new in Ardour 6 ?

Hmm, I hadn't noticed that eiter. I thought it was just a visual issue, but when I test with a click and a pattern that has all the level sliders down except the first, you clearly hear the click not eing synced.

@x42 Are we doing it wrong, or does Ardour6 have an issue with midi latency compensation?

[sjaehn]

A very simple example to show the differences. Play it in Ardour 5 and in Ardour 6 with and without user latency:

SchafflTestLab.zip

Edit: I only hear what I expect in Ardour 5

[sjaehn]

Let me correct. Ardour 6 does something with latency. I still don't understand what. :-(

[x42]

Let me correct. Ardour 6 does something with latency. I still don't understand what. :-(

Ardour compensates for it. If a plugin introduces latency during playback, the plugin receives future data, so that by the time the plugin has completed processing the result (delayed by the plugin) lines up with other non-latent tracks.

[sjaehn]

Ardour compensates for it. If a plugin introduces latency during playback, the plugin receives future data, so that by the time the plugin has completed processing the result (delayed by the plugin) lines up with other non-latent tracks.

Thank you, Robin. I red before about the major improvements in latency compensation in version 6. Therefore, I was surprised about my observation. By now (and a lot of atom data analysis) I think I understand what happened. And this makes it even more complicated:

Simplified, in Ardour 5:
Lets take my example. Track 1, latency L1 = 0. Track 2, reports latency L2 = an eighth note (1/8). The latency-introducing track starts at -1/8 (this means 1/8 notes in the background before track 1 starts to play and progression line starts to move) to send MIDI data to the plugin starting from position. The tracks reported time/position data (beat, bar, ...) start at 0 (= parallelized to the MIDI in data). This made it easy to handle. Too easy.

In Ardour 6:
The time/position data are corrected by latency. But by far not always from the start! Sometimes, the latency-introducing plugin starts at 0 to send future notes starting at some 1/8 (with a notable deviation) and the corrected tracks time/position at 0. As you would expect. Sometimes the plugin starts at 0 with notes starting from 0 and a time/position from 0. Upon time/position update by the host (less frequently than in Ardour 5) during playback, the time/position is then corrected by -1/8 (= parallelized to output).
Remark: No difference if the plugin dynamically calculates latency only during playback or if a fixed user latency of 1/8 note is used.

I see two problems:
(1) There is a time shift of 1/8 between notes in and time/position data in Ardour 6 compared to Ardour 5. This shift makes sense. But it also produces incompatibilities for plugins.
(2) The random behavior at the start in Ardour 6. No idea why this happens. But I can easily hear it and I can also see it on the stderr. I'd say, the random behavior comes from Ardour...

[x42]

(2) The random behavior at the start in Ardour 6. No idea why this happens

Can you elaborate? How does this manifest?

A shot in the dark:

Possibly the difference is by starting playback at zero (00:00:00:00) vs starting at a later position?
Or also when moving the first beat away from 0. There can be rubato-sections, where bar/beat is undefined. Transport moves, the sample-time count increases, but music-time stands still.

Sadly this is a bit of a gray area at the moment.

Moving towards Ardour 7 we are going to implement a proper rational-number based music-grid to prevent rounding errors. https://ardour.org/timing.html summarizes this.

This was originally planned to Ardour6 already, but there have been too many open issues to make it happen in time. One of the unsolved problems there is that LV2 uses floating point data for bar_beat position with limited resolution subject to rounding errors. -- If this is indeed the issue here, I'm sorry that I don't have better news than to wait, or help formalizing a good solution.

[x42]

Upon time/position update by the host (less frequently than in Ardour 5)

That is intentional. Ideally this should only be sent once at the start. re-transmission is only needed if there is drift (due to rounding errors) or when the transport locates.

Since this used to oscillate forth and back (depending on BPM and sample-rate) the condition was relaxed in Ardour/ardour@32cfed6

Sometimes the plugin starts at 0 with notes starting from 0 and a time/position from 0
[...]
during playback, the time/position is then corrected by -1/8 (= parallelized to output).

Aah, I see. Ardour 6 aligns the clock to output. when you see 00:00:00:00, you hear the corresponding audio. Ardour pre-rolls to fill the buffers up to that point. In theory we would have to inform the plugin about negative time, but that value is clamped to zero. This is not usually an issue, since there are also no MIDI events during that time, but I can see that some plugins that synchronize their internal clock might be have an issue with this non-linearity.

Strictly speaking this is a special case of the general rubato-sections.

[magnetophon]

@x42 Thanks for chipping in!

[sjaehn]

(2) The random behavior at the start in Ardour 6. No idea why this happens

Can you elaborate? How does this manifest?

I did a lot of testing using B.Schaffl and Ardour 5 or Ardour 6 within the last two days. I've to admit that I observed the problem in Ardour 5 too. And I think it is the plugin. Especially the latency calculation method. Fixed user latency produces rather reproducible results. Random behavior occurs (especially (?)) with plugin calculated latency. The plugin latency calculation method uses time#speed. But speed (and thus latency) is 0 if the plugin is stopped. The right plugin latency is reported if the plugin is running. Just after the start. Likely too late.

I'll try to fix it. Even if this may cause other problems with different playback speeds (!= 0.0 && != 1.0).

[sjaehn]

Aah, I see. Ardour 6 aligns the clock to output

The only way to make the plugin compatible to both, Ardour 5 and Ardour 6, would be an option to select between "Align position data to input" and "... to output" and to use this for calculation of the output signal position? Right ?

[sjaehn]

Fixed:

• Always report plugin-calculated latency assuming time#speed == 1.0 in 3335023
• Optional latency-compensation for time/position in 5e4e55d

@x42 Thanks for helping. Less frequent update of time/position by Ardour 6 is an improvement in most situations. I experienced a lot of jumps in B.SEQuencer due to discrepancies between internal end external time in Ardour 5. Therefore I removed sync in B.SEQuencer (where it's not really needed).

On my system and the experimental setting mentioned above, Ardour 6 starts with position 0.0 and reproducibly corrects the position after a half bar by the (now permanently reported) latency. But this means that the plugin run the first half bar with incorrect data.

You also talked about "negative time" and rubato (as it is). I also had it in my mind before. This is still an issue here for the plugin. Maybe I find a way to ship around.

@magnetophon The plugin should (hopefully) now also work in the expected way in Ardour 6. The new option "Latency-compensate time/position" must be switched off in Ardour 6 (and switched on in Ardour 5). Please keep in mind that there are still problems at the start of the track (vide supra). And I also experienced some discrepancies with user-defined latency (TODO fix).

[x42]

The situation at 0 is something that need to be resolved on Ardour's side.
Probably simply remaining at "stop" during pre-roll already does the trick.

[magnetophon]

Thank you!!! I ❤️ this plugin!

After a quick test in A6, I found the pattern to sound the same whether "Latency-compensate time/position" was on or off.
With it on, the visuals are correct as well: with it off, the play position indicator of the plugin does not sync up.

There are indeed some irregularities in the start of the song, so I hope Ardour will resolve those!

@x42 Thanks a gain. You're always most helpful!

[sjaehn]

@magnetophon In case of your examples I wouldn't expect to hear much difference as we talk about a latency of some 20 ms. And the 20 ms are maybe caught by the smart quantization option. Thus, there may really be no difference in this case.

In contrast, there are 250 ms latency in my example. 1/8 note. Results in a not-intended 1/8 rest after the C (control note in track 1) if compensation is on in Ardour 6. Not to overhear.

[sjaehn]

And I also experienced some discrepancies with user-defined latency (TODO fix).

Was only incorrect display. Fixed in f7a8d9d .

[magnetophon]

Perfect!
It all works very smooth and reliable now. 🥳

The only remaining issue afaik is the Ardour's bug, when it is at the start of the song.