XML ID naming strategy

[WolfgangDrescher]

When a Humdrum file gets converted to MEI with Verovio the output of a note will look like this:

<note xml:id="note-L21F5" dur="1" oct="5" pname="d" accid.ges="n">

This can be read as note on line 21 field 5.

I would suggest that it could be useful to have a similar XML ID naming strategy for MEI within verovio when no xml:id is present. E.g.:

<note xml:id="note-M1S1L1N1" dur="1" oct="5" pname="d" accid.ges="n">

This can be read as note in measure 1 staff 1 layer 1 number 1. Where notes would simply count up within one measure for each staff layer.

I would also suggest to change --xml-id-seed to --xml-id-strategy and allow these values: seed, named/numbered/consecutively/sequentially.

I have an application where I need to map annotations to a specific note. Working with seeds will work as long as the score will not change. Once a score gets changed and re-exported and converted again from MusicXML to MEI with verovio, having a consecutively named XML ID strategy would keep most IDs with a re-exported score and only measures that have changes would be needed to be mapped again with the new ID to the corresponding annotation. But not all annotations as the current method with seeds would imply.

Any thoughts on this? Or is it best practice to already assign XML IDs in a notation software? But I could not find such a method in MuseScore 4.

Related issues: #1304 #274

[rettinghaus]

Personally I'm not a big fan of speaking IDs. If you want something like this, why not use XSLT?
The other thing: Where would you get the numbers from, especially for measures? Would you use the @n attribute? What if there is none, or two measures have the same value or a non-numeric value? Would you consider mdivs, sections and endings?

[ahankinson]

Yeah, I'm with @rettinghaus on this...

We had a very similar scheme in the Sibelius exporter in the first version, where the IDs were deterministic based on position. But we found that it was very hard to guarantee uniqueness without continually adding more "entropy" to the IDs... Humdrum can do this because it's a line-based format, and the IDs are tied to the specific input format, but for MEI the lines don't mean anything.

Additionally, if you add anything all the IDs afterwards change. So adding one line in your file will change all your IDs. Same with the deterministic ID outputs: adding one note changes all other IDs.

You could narrow it down to very specific things, so that your IDs are per-measure, so n1L1S1m1, n2L1S1m1, etc. But this is only valid for a single mdiv! If you have additional mdivs, you have to add that in... n1L1S1m1md1.... Your IDs quickly become unwieldy. chords, beams, etc. etc. means you need to have a whole set of ID schemas running.

[ahankinson]

Or is it best practice to already assign XML IDs in a notation software? But I could not find such a method in MuseScore 4.

You could ask the MuseScore folks to add it, but you would also need a way to guarantee that you're assigning unique IDs. If your IDs are deterministic, then when you edit the file you'll be changing the IDs of all other notes in the file.

There's also the larger data management problem of your workflow: making your annotations, storing them, then later editing the file, but, since your annotations and data are stored separately, it's likely that your annotations are now incorrect. IOW, your annotations still point to notes, but now they point to the WRONG notes.

I don't mean to sound dismissive -- you're not the first person I've heard to bring this up, and it is definitely an issue. But I don't think deterministic IDs are the correct solution.

The most obvious alternative way to solve the problem is to use and adapt something like EMA that seeks to provide a way of addressing content "musically" rather than "structurally". This abstracts away the underlying encoding, so that it doesn't matter if you're in Humdrum, MEI, or MusicXML, "measure 1, staff 1, note 1" is still a valid annotation address.

[WolfgangDrescher]

If you want something like this, why not use XSLT?

Extensible Stylesheet Language Transformation? Never heard of it before, but looks interesting. Is there an example in combination with MEI?

You could narrow it down to very specific things, so that your IDs are per-measure

Yes, I was thinking about doing this per measure. Did not clarify this in my initial post.

Your IDs quickly become unwieldy. chords, beams, etc. etc. means you need to have a whole set of ID schemas running.

I don't think unwieldy IDs are a problem as a UUID or the current shorter IDs are also not there the be understandable for humans.

You could ask the MuseScore folks to add it, but you would also need a way to guarantee that you're assigning unique IDs. If your IDs are deterministic, then when you edit the file you'll be changing the IDs of all other notes in the file.

That is a good idea. I will create a discussion on the MuseSocre repo. If it were possible give a unique ID in MuseScore, a re-export would also keep the ID (e.g. when you correct a wrong note, but probably only as long as you don't delete if and then add a new note instead of modifying it). However this will only solve this problem for one music notation software. I still think having something like my suggestion directly in verovio would be useful. Of course I'm not asking to make the the default id naming strategy but having it as an opt-in option.

IOW, your annotations still point to notes, but now they point to the WRONG notes.

Thats a good point.

The most obvious alternative way to solve the problem is to use and adapt something like EMA that seeks to provide a way of addressing content "musically" rather than "structurally". This abstracts away the underlying encoding, so that it doesn't matter if you're in Humdrum, MEI, or MusicXML, "measure 1, staff 1, note 1" is still a valid annotation address.

This is interesting, thank you for sharing. I could not get the docker image umdmith/omas running. How would I map the EMA address pointers to IDs in verovio (in MEI files)?

---

One simple alternative would be to write a small script the alters the xml:id of note elements the way I want it. So at least for my project where I know what scores I can expect and when there are no unknown variants with multiple mdivs this could be an easy solution.

Encoding the annotations directly in the MEI output could also be possible. But I'd prefer to keep the score clean in my case (annotations are also linked to audio files where you need to locate bad intonation in the recording and click on them in the score).

[ahankinson]

(annotations are also linked to audio files where you need to locate bad intonation in the recording and click on them in the score).

I don't know if this is useful, but MEI does have a way of encoding timepoints and linking them to audio files. <note@when> points to a <when> element, which gives a timepoint in relation to a <recording>, which can link to an <avFile> element.

The Facsimiles and Recordings chapter in the Guidelines has some examples.

[ahankinson]

I don't think unwieldy IDs are a problem

I meant unwieldy to generate in your code. You need to keep lots of context around on every object, and different schemes for each object to ensure you never generate a duplicate. You might, for example, accidentally use the letter "s" for both "staff" or "slur", and if you don't take care that you have a very specific pattern in place to ensure you never confuse the two, then it may happen that these two collide.

I could not get the docker image umdmith/omas running. How would I map the EMA address pointers to IDs in verovio (in MEI files)?

OMAS was the MEI-specific instance of using EMA (the addressing scheme) as a proof-of-concept. The point of EMA was that it was format-independent. You could give an EMA address like:

/foo.kern/1,10/1+2/@start-end/raw

And it would return the same "musical extract" from the file as doing

/foo.mei/1,10/1+2/@start-end/raw

Namely, from "foo.mei" or from "foo.kern", extract measures 1-10, staff 1 and 2, from the start to the end, as a "raw" extract. The point being that the "musical" identifiers is different from the "format" identifiers, since you can use multiple different formats, or even different methods with the same format type, to express the exact same musical content.