Grand Staff Cursor Bar jumps around

[chris-saunders]

Your environment

• Version used: Develop
• Platform used: JavaScript
• Rendering engine used: SVG (default)
• Browser Name and Version: Any
• Operating System and version (desktop or mobile): Desktop

Expected Results

Grand staff cursor bar moves smoothly between notes on both the treble and bass staves.
May be a similar issue to #317

Observed Results

Currently there are periods where the cursor jumps much further than it should, and then snaps back as it hits a new note.

Steps to Reproduce (for bugs)

Having notes of different lengths on each staff can often throw the cursor off. Having a half note on the treble staff and a handful of 16th/8th notes/rests on the bass seems to make the cursor jump to the wrong location.
*Edit Created a JS fiddle to demonstrate the issue https://jsfiddle.net/rj63L2ey/5/
Sample AlphaTex:
\tempo 169 . \track \staff{score} \tuning piano \instrument acousticgrandpiano \ks Gb \ts 4 4 \clef G2 g#4.4 c5.4 g4.4 g#4.8{d} f4.16 | f4{t}.8{d} g4.8{d} d#4.4 r.16 d4.4 d4{t}.16 | d4{t}.8{d} r.8 (d4 f4).2 d4.8 | \staff{score} \tuning piano \instrument acousticgrandpiano \ks Gb \ts 4 4 \clef F4 f3.2 (g#3 c4).4{d} r.16 f3.16 | f3{t}.4 f3{t}.16 r.16 (g#3 c4).4{d} f3.8 a#3.8 | a#3{t}.2{d} a#3{t}.8 a#3.8

Possible Solution

May be due to the position setting of the seek bar only respecting the latest (bass staff) note, rather than taking into consideration both staves, and the next note between them.

Bounty

I'll donate some beers for the cause!

[gburlet]

I’ll also donate some beers for this fix!

[Danielku15]

The whole "cursor" system was initially developed with only one main staff in mind. Of course this causes issues on multi track and multi staff setups. This will likely need a whole redesign/-implementation of the cursor positioning system.

The main issue on this topic is performance. With the speed of playback updating the cursor too often is very costly. I have some idea in mind where I pre-build in best case when the cursor will be where. This will make it easier during actual playback.

[gburlet]

Looks like some of the data models are missing beats from staves other than the first, for example in the cursor cache. Agreed that some of the internal data models probably need an overhaul for the 1.0 milestone.

As for this specific issue, I did some digging around and one potential solution to this issue is to only update the cursor animation according to the notes on the first staff of multi-staff/multi-part music.

Specifically, in alphaTab.audio.MidiTickLookup.FindBeat, adding a filter in the master bar beat list to only look at beats on the first staff results in smooth cursor animations and doesn't have any additional performance hits relative to single-staff music:

while(b < beats.length) {
    var currentBeat = beats[b];
    var staff = currentBeat.Beat.Voice.Bar.Staff;
    if(!trackLookup.hasOwnProperty(currentBeat.Beat.Voice.Bar.Staff.Track.Index) || staff.Index != 0) {
        ++b;
	continue;
    }
...

^ this staff index filter would need to be done when searching for both the current beat, next beat, and next beat falling in the next bar in the FindBeat function

I'm going to test with a few more alphatex pieces, but I think this works as a solution because in multi-staff/multi-part music you'll always fill up each measure with notes or rests to match the time signature on all staves. Therefore, as far as the cursor is concerned, it doesn't really matter what's going on with all the staves. We could just follow notes on the first staff and disregard the rest.

[gburlet]

I just discovered how beat highlighting works. Basically, it will light up the current beat when FindBeat returns a beat that differs from the current note in the cache and a new cursor animation is issued. The above method doesn't work with the current method of highlighting because if FindBeat is only returning beats on the first staff, then we would never highlight notes the cursor passes on all staves. Will continue digging into the code a bit more and see if there's a better way to do highlighting.

[gburlet]

OK, I coded a temporary fix on the Javascript side that has smooth cursor scrolling and proper note highlighting for multi-staff or multi-part music. Attached is a zip with the JS patch and my final file for reference. I would offer a pull request, but I don't know Haxe or C#, which I think is the backend generating the Javascript.

Overview:

• Cursor: cursor now follows beats on the first staff only and uses that for CSS animations of the cursor. Modified alphaTab.audio.MidiTickLookup.FindBeat to only consider the first staff elements. As before, you still can not select beats on a staff other than the first. That is still not fixed.
• Beat highlighting: any beats that are active (sounding) at the current cursor location across all staves are highlighted correctly. Behind the scenes, this involved creating a Set cache of glyph group IDs (almost like the cursor cache) of beats that are currently active (updated each tick). Set difference is done to update glyph CSS when necessary.

I think to properly do this, we still need an overhaul of some data structures for multi-staff/part music (e.g., to enable the ability to select beats on any staff), but this is an OK fix for now so the cursor doesn't go insane and we have proper beat highlighting. I started looking into some of the data structures but I got real deep in the code and started getting a little lost.

I hope this helps in some way 🙂

multistaffcursorhighlightpatch.zip

[Danielku15]

I agree that only considering the first staff might be a nice temporary fix to avoid big jumping. Of course it will not fit all needs like:

• You want to have all note elements highlighted correctly as they are actually playing
• If the first staff has only a rest for the whole bar, but the second one has maybe some notes, the cursor position will likely not fit the playback position.

The build chain works like this: The main source is the C# codebase. The C# codebase uses my custom compiler "Phase" to translate C# to Haxe. Haxe I use to actually transform it to JavaScript. So Haxe and JS are both generated code bases. Its a bit complicated but it was quite an overkill to translate in my compiler C# to JavaScript directly. Spoiler: I started to investigate to move over to TypeScript for the main development codebase but its to early to say if it works out. People would fit more "home" there to contribute to alphaTab.