Fix: Dots valid position within MEI file.

[martha-thomae]

The position of the dots is problematic in mensural notation. Dots are allowed as children of note elements (which shouldn't be the case, they should be siblings). This behaviour is codified in the Shared module. Since we don't want to modify this behaviour for all <dot> elements, but only those used in mensural notation, I modified this in the customization rather than the source file. The other issue is that dots have to be allowed within ligatures. This PR solves both issues (which were originally pointed out in #691).

• Deleting model.noteModifierLike, made invalid the use of <dot> as a child of <note>.
• Adding model.eventLike.mensural, made valid the use of <dot> within a <ligature>.

Please, feel free to modify this PR if the changes are not made in the appropriate place (maybe this change was supposed to be done in the source file rather than the customization?) or if the @mode attribute was not used appropriately.

Right now, the customization works perfectly. It solves the two problems shown in the screenshot, where the original Mensural schema failed.

Also, in the second commit, removed the @dots attribute from the <note> element (this is the usual way of encoding dots of augmentation in CMN). In mensural notation, both dots of augmentation and dots of division are encoded within the stream of notes inside the <layer> element.

[ahankinson]

As I read it, <dot> is already allowed in layer: https://music-encoding.org/guidelines/v4/elements/dot.html

So dots as peers of notes is already possible.

Do you have a case to illustrate why dot should be allowed in ligature? Literally defined, a ligature would be what could be done with a single pen stroke, so a single dot cannot (technically) be part of a ligature. (I recognize that the technical description of a ligature and the actual practice of determining what is a ligature or not may differ, so I think it would be helpful to have an example.)

Putting too many customizations within the Mensural one will have some knock-on effects to the behaviour of the mensural notation in MEI-all. You should probably check to see if MEI-all behaves as expected. It would also be good to confirm that the RelaxNG from this customization contains the correct behaviour -- there have been problems in the past with using the 'change' mode, necessitating the use of 'delete' then 'add'.

[craigsapp]

Do you have a case to illustrate why dot should be allowed in ligature? Literally defined, a ligature would be what could be done with a single pen stroke, so a single dot cannot (technically) be part of a ligature. (I recognize that the technical description of a ligature and the actual practice of determining what is a ligature or not may differ, so I think it would be helpful to have an example.)

I have seen dots on internal notes of ligatures (they are placed above or below the notes in the ligature), but I don't remember where I saw them. A ligature is shorthand for a series of separated notes. If that separated set of notes contains a dot, then the ligature also needs the dot.

a ligature would be what could be done with a single pen stroke,

in white notation, they are not literally a single pen stroke but rather a set of connected boxes. You are thinking more in terms of neumes and black mensural notation... Ligatures function more like beams in modern music to group a set of notes together for efficient readability (and are derived from neumes). With beamed notes, it is possible to have grace notes inside of the beam group, which would be analogous to the dots in the case of ligatures.

[martha-thomae]

Hi @ahankinson !

As I read it, <dot> is already allowed in layer: https://music-encoding.org/guidelines/v4/elements/dot.html

So dots as peers of notes is already possible.

Yes, this is true. The PR is focusing on forbidding the use of <dot> as a child of <note> (and @dots as an attribute of <note>). When I was a first-MEI user, it was natural for me to encode <dot> as a child of <note>, and I have seen others make the same mistake when encoding mensural notation for the first time (then you realize that Verovio doesn't render them unless you encode them as siblings and that all examples in the guidelines are showing them as siblings, but there is nothing forbidding them to be encoded as children). So, why not removing these possibilities (encoding dots as a child or as an attribute of note elements) altogether?

Do you have a case to illustrate why dot should be allowed in ligature? Literally defined, a ligature would be what could be done with a single pen stroke, so a single dot cannot (technically) be part of a ligature. (I recognize that the technical description of a ligature and the actual practice of determining what is a ligature or not may differ, so I think it would be helpful to have an example.)

Yes, I do. Here are a few images taken from the first couple of folios of the Mellon Chansonnier (the ligatures with dots are on magenta boxes).

I also agree with everything @craigsapp said in the previous comment, including that the definition of ligature provided seems to be more in agreement with neumes, but not with white mensural notation. Ligatures in mensural notation are derived from modal notation (here having rhythmic implications), which were derived themselves from neumes.

[martha-thomae]

Putting too many customizations within the Mensural one will have some knock-on effects to the behaviour of the mensural notation in MEI-all. You should probably check to see if MEI-all behaves as expected. It would also be good to confirm that the RelaxNG from this customization contains the correct behaviour -- there have been problems in the past with using the 'change' mode, necessitating the use of 'delete' then 'add'.

I would actually love some guidance in this matter. I am still pretty new on modifying schema-related stuff and I learned by looking at how things are now implemented and some tips Johannes gave me at the very beginning. But I have so many questions. It might be better to ask these particular questions in slack (if you don't mind), but the ones that come to mind right now with the particular comment are:

Putting too many customizations within the Mensural one will have some knock-on effects to the behaviour of the mensural notation in MEI-all. You should probably check to see if MEI-all behaves as expected.

Why does the mei-Mensural.xml customization file would have an effect on mei-All? I am only modifying the customization for mensural, not touching anything in other customization files or in the source files.

It would also be good to confirm that the RelaxNG from this customization contains the correct behaviour -- there have been problems in the past with using the 'change' mode, necessitating the use of 'delete' then 'add'.

I am generating the RelaxNG using the MEI Customization Service in the MEI Garage. Selecting MEI v4.0.1 as the source file and the local customization file I am providing in this PR. When pointing to the generated RNG in the XML processing instructions of my MEI file, validation behaves as expected (<dot> is no longer valid as a child of <note>, @dots is invalid as an attribute of <note>, and <dot> is valid as a child of <ligature>, see image).

Here I am sending the RNG file generated and the MEI file I used for testing, in case it is useful.
dots_schema_changes.zip

[ahankinson]

You should probably change this description to reflect that this element cannot be used to encode a dot of augmentation any more.

[martha-thomae]

But yes, you would still use the <dot> element for encoding both dots of division and augmentation (using @form with the values "div" or "aug", respectively). What you would not use anymore is the @dots attribute included in that description.

What is changed by these few committed lines would be the place where this <dot> element is used—allowing it as a child of <ligature> and forbidding it as a child of <note>—not the features encoded by it.

So, I think that the description is still applicable. Isn't it?

[ahankinson]

That's great, @martha-thomae -- thank you.

When adding new functionality, it is relatively simple since you are not changing the validity of how others have encoded things. When removing capabilities -- in this case, dots as children of notes -- then the barrier to change should be higher, since you will be changing things that were valid before and potentially breaking things. This is one of the biggest challenges we have when developing MEI: when and how to introduce breaking changes, and what that means for schema development and for our user community who have built corpora around encoding in a particular way.

That's not to say that what you're doing here is not correct, it's just to say that these sorts of changes should seek and provide some sort of community consensus before being accepted, because they have the potential of breaking other people's encodings (and therefore costing money for them to fix!).

Why does the mei-Mensural.xml customization file would have an effect on mei-All? I am only modifying the customization for mensural, not touching anything in other customization files or in the source files.

MEI-all includes all of the source files, including the mensural module:

https://github.com/music-encoding/music-encoding/blob/develop/customizations/mei-all.xml#L86

Since it's including this, you will get all the mensural elements, but they won't behave in the same way as in the mensural module. So I guess what I was trying to say is that by using MEI-all you can still encode mensural notation where dots are available as a child of note. I think this is OK, since it is likely that you will, in the wide world of music notation, have a source where there are both dots of division AND dots of augmentation, so there should be some way of encoding them. But it does create some confusion why some behaviours are allowed, and others aren't, depending on the MEI schema that you use.

The reason for having MEI-all is partly historical, and partly "political." For a long time there was only a single MEI schema, and that always included everything. When we adopted the TEI ODD system, we gained the ability to have these customizations for individual types of repertoires, which were useful for specifying things that were incompatible between notation types (like the behaviour of dots).

After 10 years of trying to explain customizations, most people still don't understand the difference and just want to know if their file is "valid MEI." They will use the MEI-all schema to validate their files, and then call it a day. This has been slowly changing, but the idea that there are multiple MEI schemas, and some MEI may be valid with some schemas but not all MEI is valid with all schemas, has not quite caught on yet. They usually just want to check their files against "The MEI Schema" (if they even validate them at all!)

All that is to say: We can't really do a whole lot about what people think, but what we can do is try and ensure that MEI-all is a near-complete superset of the behaviours in the customizations. I'm not saying anything is wrong with this particular submission, but generally speaking, if a MEI document passes validation in MEI-all, it should also pass with the customization, and for that you usually just need to do a quick test to ensure the encoding you are expecting still works when validated with mei-all, even if it passes stuff that don't actually pass the customization. (Since you're not actually changing the mensural module, you should be fine. But it's still good practice to check.)

[martha-thomae]

Thank you very much for taking the time to give me all these details, @ahankinson! This helps me put everything in perspective and, believe it or not, has answered many of the questions whirling around my head.

It became clear to me why it is good practice always to check validation against MEI-all (because this has to be a super-set of what is valid on individual modules), and I will do this from now on. Thank you!

Now that I tried to validate against MEI-all, I can already see an issue: the <dot> within <ligature> is not valid. So, I guess that at least that part, the one that allows <dot> to be
a child of <ligature> (https://github.com/martha-thomae/music-encoding/blob/dot/customizations/mei-Mensural.xml#L121), should be moved into the actual Shared module rather than being in the customization, right?

I will work on the changes suggested :-)

[martha-thomae]

I am closing this PR (momentarily) because I agree with @ahankinson that the changes I did in the customization file (the ones that forbid the use of <dot> as a child of <note> and the use of @dots as an attribute of <note>) are removing some functionality, and I should consult with the IG to guarantee complete agreement before proceeding with these changes. I will re-open this PR after I have doubled-check this with the IG. I am substituting this pull request with the PR #701, which only deals with the encoding of <dot> elements within <ligature>.

[rettinghaus]

Removing <dot> from model.noteModifierLike doesn't removes it as child from <rest>, as it is a direct child here. So it should also be removed from rest to keep it consistent.

[martha-thomae]

Thank you for catching this, @rettinghaus! I will change that too.

[martha-thomae]

I have doubled-checked these changes with the IG.
So, I am re-opening this PR.

[martha-thomae]

@ahankinson and @rettinghaus, is there anything else you think should be corrected in this PR?

[kepper]

Regardig the outcomes, I think this is done. However, I'm not sure I like how this is implemented. For one element, it was necessary to remove the model.noteModifierLike class, and for the other it was necessary to replace the <elementSpec>, as the <dot> was inserted into both elements inconsistently. So, @martha-thomae helped to surface a deeper "problem" in the current setup of MEI. Wouldn't it be much more elegant if we'd use nested classes like model.noteModifierLike and model.noteModifierLike.cmn / model.noteModifierLike.mensural instead (same for rests), so that by turning off a module, the dot would go away automatically, without touching the <elementSpec>s individually? Again, this is totally unrelated to the purposes of this PR, but might be discussed in this context nevertheless…

[kepper]

At today's ODD Friday, we agreed to merge this, and to address the questions of different structures with rest and note independent of this later on.