Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix #23063 : Note on-time and off-time offsets #2132

Closed

Conversation

Projects
None yet
4 participants
@mgavioli
Copy link
Contributor

commented Jul 22, 2015

Fix #23063 : Note on-time and off-time offsets (mergeable, tests welcome!)

References:

The above links provide abundant discussions and details.

Architectural choices:

  • Note time offsets are visualized and stored internally only as offsets from default on-time and off-time in percent of total note duration. The other old notation "User" (in ticks) is no longer used and is converted into "Offset" when found in old scores. This spares the user from dealing with arcane (and under-documented) units like ticks.
  • I have kept the percent unit, instead of the more precise perthousand used during MIDI event rendering, because it keeps the backward compatibility with older file formats and I consider it precise enough (1% of a whole note is ca. 19 ticks or ca. 4 hundreds of second at 60 MM, which makes for a hardly audible difference; for shorter notes, 1% makes for less ticks and an even shorter difference).
  • On- and off-time offsets apply to a single note as a whole, not to the each of the (possibly multiple) events making the note up at MIDI rendering level.
  • They are stored in specific member variables Note::_onTimeOffset and Note::_offTimeOffset; as there is currently only one type of offset, the member variables Note::_onTimeType and Note::_offTimeType have been removed (as a consequence, the memory occupation of each Note instance did not change).
  • In these member variables, 0 means no difference from the defaults (whatever is mandated by the instrument characteristics and/or by the articulations like staccato, and so on).
  • A value other than 0 means an offset from the note nominal beginning or ending (i.e. an offset from time 0 or time 100% of the note duration). I.e., it is not possible to require an offset from the 'articulated' note beginning or ending (e.g.: 10% more than the staccato ending time). The rationale is that, when the user requires a custom time offset, he is more probably after a specific effect for the entire note, rather than trying to modify whatever 'time gating' is the the result of an articulation in effect by accident at that note.
  • Values other than 0 are written to score files, as sub-tags of the <Note> tag, using the <onTimeType>, <offTimeType>, <onTimeOffset> and <offTimeOffset> tags. Strictly speaking, <onTimeType> and <offTimeType> are redundant, as only 2 (= "Offset") is used as a value for them, but adding them makes the indications comprehensible to 2.0.x versions too (forward compatibility).
  • Note.onTimeOffset and Note.offTimeOffset properties are also exposed to the plug-in framework, both for reading and writing. These properties use the same percent unit. This addresses issue #25770 linked above.

Interaction with ornaments and articulations:

The current machinery for ornament realisation is almost unaffected, the only differences being:

  • The initial on-time offset and the final off-time offset are now used to calculate the whole duration of the event sequence (this required propagating the ontime and gateTime parameters to a number of low level functions in rendermidi.cpp).
  • The code which applied the articulation-specific (staccato, etc.) time gate to each event of an ornament has been removed and individual events of an ornament are now played legato (as they should in the great majority of cases).
  • All ontime and gateTime variables in ornament rendering functions of rendermidi.cpp have been normalized to use perthousand instead of the mix of percent and perthousand previously used.

These changes had some consequences:

  • Ornament events have a (slightly) different timing, invalidating most MIDI tests. I believe this to be an improvement, without unexpected side-effects, but more checks and tests are welcome.
  • Ornament which previously fitted tightly in the whole nominal duration of the note might not fit once the note is 'gated'; in particular 'long' ornaments like trills on short notes like 8th's with fast tempos. In this cases, the ornament is no longer played back (according to the current logic that an ornament which does not 'fit' is not played). This is kind of expected now and may be improved later if it will prove to be unconvenient.

Currently failing tests will be fixed as soon as the code proves to be reliable enough.

User interface:

  • Two new input boxes in the Note Inspector allow to enter the on-time and off-time offset for each note (in percent of the whole note duration).
  • If several notes are selected, the values are applied to all notes in a single stroke.
  • Default offset values are shown as "0%".
  • Modifying the offsets triggers a re-rendering of the play events of the involved chords (this required changing the Score::createPlayEvent() method from protected to public).

@mgavioli mgavioli force-pushed the mgavioli:Fix#23063_note_on-offtime_offsets branch from 564adb6 to 192e960 Jul 25, 2015

@mgavioli mgavioli changed the title #23063 : Note on-time and off-time offsets FOR TEST ONLY, DON'T MERGE #23063 : Note on-time and off-time offsets (TESTS WELCOME) Jul 26, 2015

@mgavioli mgavioli force-pushed the mgavioli:Fix#23063_note_on-offtime_offsets branch from ccc0f9d to 5720ac7 Aug 6, 2015

@mgavioli mgavioli changed the title #23063 : Note on-time and off-time offsets (TESTS WELCOME) Fix #23063 : Note on-time and off-time offsets (TESTS WELCOME) Aug 6, 2015

@mgavioli mgavioli changed the title Fix #23063 : Note on-time and off-time offsets (TESTS WELCOME) Fix #23063 : Note on-time and off-time offsets Aug 6, 2015

@mgavioli

This comment has been minimized.

Copy link
Contributor Author

commented Aug 6, 2015

Rebased, updated tests to lastest updates and squashed.

@orchestralmusic-net

This comment has been minimized.

Copy link

commented Aug 21, 2015

@mgavioli: Is relative measurement (%) really good idea here? Can you imagine what exactly 1 % from some note (e. g. 8th) is? It seems to me to be somehow unpractical. Moreover, if you want to shift two or more different notes by the same amount, then it is really hard to achieve.
Why not absolute measurement? I do understand that thick is not usable here for measurement. Hence, I would simply choose 64th note as “1 unit” of measurement. So “2” would mean 32th note, “4” would mean 16th note, “0.5” would mean 128th note etc. That’s all pretty clear.
What do you think about it?

@mgavioli

This comment has been minimized.

Copy link
Contributor Author

commented Aug 21, 2015

According to my personal experience, time offsets are mainly used to achieve a level of articulation / inégalité which is impossible / hard to achieve with the normal articulation signs and/or swing controls, more than to 'fake' a different rhythm easily expressible with 'regular' note durations.

In practice, the values I end up using often do not convert easily in regular notation (for instance, what about tuplet-based inégalité?).

So, I do not doubt there might be cases where the units you describe could be more convenient than percentage. But I suspect, in the majority of cases, they would not (how easy is to see that 2/3 of a quaver are 5.3 64ths or that 3/5 of a crochet are 4.8 64ths? Is it easier than 66% or 60% resp.?).

I know that, using percentage, expressing the same time shift for notes of different values is not overly simple. But I wonder how frequent this would be. Is it common that one needs to shorten a minim AND a quaver of the same absolute amount? Or to shift a quaver AND a semiquaver of the same absolute amount? According to my experience, the offset is more often related to the note duration.

I am open to more opinions, though.

@mgavioli mgavioli force-pushed the mgavioli:Fix#23063_note_on-offtime_offsets branch from 5720ac7 to e027308 Aug 29, 2015

@mgavioli

This comment has been minimized.

Copy link
Contributor Author

commented Aug 29, 2015

@lasconic : re-rebased

Fix #23063 - Note on-time and off-time offset
__References__:

- Latest forum post asking for this feature: https://musescore.org/en/node/58556 (a number of older posts also exist)

- Pending issues: https://musescore.org/en/node/23063 and https://musescore.org/en/node/25770

The above links provide abundant discussions and details.

__Architectural choices__:

- Note time offsets are visualized and stored internally only as offsets from default on-time and off-time in percent of total note duration. The other old notation "User" (in ticks) is no longer used and is converted into "Offset" when found in old scores. This spares the user from dealing with arcane (and under-documented) units like `ticks`.

- I have kept the percent unit, instead of the more precise perthousand used during MIDI event rendering, because it keeps the backward compatibility with older file formats and I consider it precise enough (1% of a whole note is ca. 19 ticks or ca. 4 hundreds of second at 60 MM, which makes for a hardly audible difference; for shorter notes, 1% makes for less ticks and an even shorter difference).

- On- and off-time offsets apply to a single note as a whole, not to the each of the (possibly multiple) events making the note up at MIDI rendering level.

- They are stored in specific member variables `Note::_onTimeOffset` and `Note::_offTimeOffset`; as there is currently only one type of offset, the member variables `Note::_onTimeType` and `Note::_offTimeType` have been removed (as a consequence, the memory occupation of each `Note` instance did not change).

- In these member variables, `0` means no difference from the defaults (whatever is mandated by the instrument characteristics and/or by the articulations like _staccato_, and so on).

- A value other than 0 means an offset from the note __nominal__ beginning or ending (i.e. an offset from time 0 or time 100% of the note duration). I.e., it is not possible to require an offset from the 'articulated' note beginning or ending (e.g.: 10% more than the _staccato_ ending time). The rationale is that, when the user requires a custom time offset, he is more probably after a specific effect for the entire note, rather than trying to modify whatever 'time gating' is the the result of an articulation in effect by accident at that note.

- Values other than 0 are written to score files, as sub-tags of the &lt;Note&gt; tag, using the &lt;onTimeType&gt;, &lt;offTimeType&gt;, &lt;onTimeOffset&gt; and &lt;offTimeOffset&gt; tags. Strictly speaking, &lt;onTimeType&gt; and &lt;offTimeType&gt; are redundant, as only `2` (= "Offset") is used as a value for them, but adding them makes the indications comprehensible to 2.0.x versions too (forward compatibility).

- `Note.onTimeOffset` and `Note.offTimeOffset` properties are also exposed to the plug-in framework, both for reading and writing. These properties use the same percent unit. This addresses issue #25770 linked above.

__Interaction with ornaments and articulations__:

The current machinery for ornament realisation is almost unaffected, the only differences being:

- The initial on-time offset and the final off-time offset are now used to calculate the whole duration of the event sequence (this required propagating the `ontime` and `gateTime` parameters to a number of low level functions in `rendermidi.cpp`).

- The code which applied the articulation-specific (_staccato_, etc.) time gate to each event of an ornament has been removed and individual events of an ornament are now played legato (as they should in the great majority of cases).

- All `ontime` and `gateTime` variables in ornament rendering functions of `rendermidi.cpp` have been normalized to use perthousand instead of the mix of percent and perthousand previously used.

These changes had some consequences:

- Ornament events have a (slightly) different timing, invalidating most MIDI tests. I believe this to be an improvement, without unexpected side-effects, but more checks and tests are welcome.

- Ornament which previously fitted tightly in the whole nominal duration of the note might not fit once the note is 'gated'; in particular 'long' ornaments like trills on short notes like 8th's with fast tempos. In this cases, the ornament is no longer played back (according to the current logic that an ornament which does not 'fit' is not played). This is kind of expected now and may be improved later if it will prove to be unconvenient.

Currently failing tests will be fixed as soon as the code proves to be reliable enough.

__User interface__:

- Two new input boxes in the Note Inspector allow to enter the on-time and off-time offset for each note (in percent of the whole note duration).

- If several notes are selected, the values are applied to all notes in a single stroke.

- Default offset values are shown as "0%".

- Modifying the offsets triggers a re-rendering of the play events of the involved chords (this required changing the `Score::createPlayEvent()` method from `protected` to `public`).

@mgavioli mgavioli force-pushed the mgavioli:Fix#23063_note_on-offtime_offsets branch from e027308 to 575aeca Oct 17, 2015

@mgavioli

This comment has been minimized.

Copy link
Contributor Author

commented Oct 17, 2015

@lasconic : re-re-rebased...

In case the high number of modified files is a source of concern, it should be noted that:

  • 1 file (rendermidi.cpp) is heavily changed;
  • 2 files (note.cpp/.h) have small changes;
  • 7 files only have trivial changes:
    • in instrument.cpp/.h changed 1 function prototype (used only in rendermidi.cpp),
    • in property.cpp/.h added 2 enum items,
    • in score.h 1 method changed from protected to public,
    • in inspectorNote.cpp and inspector_note.ui added UI input boxes for time offsets;
  • all the other 21 files are updated test output files.
@lasconic

This comment has been minimized.

Copy link
Member

commented Oct 28, 2015

Just discussed this with @wschweer .

The plan is still to make it possible to edit note events individually or several at the same time in the pianoroll. If I understand correctly, this PR adds note properties (offtime,ontime offsets) that will act on a full note, meaning on all the events of a note. It looks like this doesn't fit with the edition of events in the pianoroll dialog. It would add one more layer of abstraction on top of instrument gate time, and articulation gate time... while we would like to a much simpler approach of editing events one by one in the pianoroll.

If we had the pianoroll editing in place, we could expose in the inspector a way to change the ontime/offtime of a the single event of a note in the simplest case. If the note contains more than 1 event then these controls would be disabled. These controls would be like the one in the piano roll (ontime, length)

So this PR adds much complexity and I wouldn't prefer to not merge it.

@lasconic lasconic closed this Oct 28, 2015

@mgavioli

This comment has been minimized.

Copy link
Contributor Author

commented Oct 28, 2015

@lasconic : @wschweer : "this PR adds note properties (offtime,ontime offsets) that will act on a full note, meaning on all the events of a note_formulation)"; OK, I suspect I did not explain this stuff clearly enough. And I suspect all this attention to events is what is adding (too?) much complexity, in a way I still fail to understand. The changes I made seem to me rather simple conceptually and not more complex than before code-wise.

A note is a note, it is a primary musical concept; that, occasionally, a note may be made of several sound events (ornaments, glissandi, etc.), is purely accidental and does not change the fact that a note is a note.

Being able to articulate the individual events making an ornament up might be nice, vvvvvvvery rarely, so rarely that I fail to find any use case for this (my limitation, probably); articulating the note, as a whole, is much more relevant.

Articulating a note, to me, means defining its initial (occasionally) and final (much more often) time gate, regardless the note has an ornament or not.

Example: If a note has a trill and I would like to separate it from the resolution note following it (a rather common articulation), I do not want to fuss with the individual, multiple chunklets making the note up from the MIDI point of view; I want to be able to tell: "make this note 70% shorter" (or any equivalent formulation) and then the MIDI 'realizer' would generate as many trill alternations as they fit in the actual note duration (which is what my code does).

It is unclear to me how I could reach the same result by using the piano roll and meddling with the dozen(s) of trill events I would face (and which are totally irrelevant from this perspective): shall I delete trill events until I reach the amount of post-note silence I am after? Or shall I shorten each event and progressively shift them until I reach...? (which would alter the trill rate). This seems to me to add much complexity, rather than my approach, which is quite straightforward conceptually and not more complex than before in coding (yes, I already said this, but it is worth repeating).

In addition, the piano roll has a long way to go before being barely usable for this goal: it shows only one part at a time, it is not possible to select multiple separate notes (notes!!!!, not events!!!!), it always opens at the beginning of the score, and so on. Even assuming these limitations will be fixed in the future, the main point remains: a note is a note and the piano roll ignores the note level (being made for another purpose, I imagine)!

I would understand (perhaps not agree, but surely understand) an objection like: "this has to do with playback and we want to focus on notation", but objecting to a rather straightforward, musically meaningful, implementation because "in the future we will do this in a very complex, and largely counter-intuitive way", well, I frankly do not understand.

@lasconic

This comment has been minimized.

Copy link
Member

commented Oct 28, 2015

First, you're right that the pianoroll has a long way to go but we believe it's the way to go.

Adding more properties to note for playback is not viable and will only cover very specific use cases such as the one you highlight (trill + resolution) while the ability to edit the events of a note, if made right, will cover many different use cases at once. Your approach doesn't add complexity right now but we will possibly need other note properties in order to solve other use cases while a good event editor would solve them all.

I do agree that the most common use case is to articulate a note made of single event. The pianoroll approach would make that possible as easily than your approach and it would give visual feedback in addition to the audio feedback. For multiple events, I can imagine a function (and not a property), to extend a trill in the pianoroll without needing any new property in the note. This function would create events for this note, and these events would be stored in the mscx.

Another argument. Most of the changes to a note sound are visible in the score (stacatto, dynamics, turns etc...) or are a "style" (instrument gate time). I believe it's good to have a dedicated UI to manipulate all the other invisible changes like the total gatetime, or the ontime offset. Put differently, the pianoroll approach has the advantage to focus on notation in the notation editor, focus on sound in the pianoroll view.

@mgavioli

This comment has been minimized.

Copy link
Contributor Author

commented Oct 28, 2015

@lasconic @wschweer : Well, that my complete disagreement shall remain on the record: I do not agree that the piano roll is the way to go for this purpose.

Assuming that this discussion is meaningful only for ornamented notes (if a note has only one MIDI event, there is no real difference between manipulating the note or manipulating the event), my example can be extended to any ornamented note, as any note might be articulated or not and ornamented or not, independently. Time-gating an ornamented note does not seem to me "a very specific use case"; it seems to me the (almost?) only use case where time-gating a multi-event note may happen (time-gating a note with glissando should be very exceptional, I believe, but still would be quite cumbersome to achieve from the piano roll).

OTOH, I still fail to see any use case where time-gating individual MIDI events of a note could be necessary or useful (beyond the mere "I do it because I can do it") and then I fail to see what time-gating via the piano roll would add, in exchange for the inevitable complications or 'cumbersomeness' it would bring (even locating the note you want to act upon is difficult on the piano roll; now imagine time gating an orchestral chord via the piano roll, even in absence of any ornament).

I am not sure what you mean by "a function to extend a trill in the pianoroll"; AFAIU, this would be a very ad hoc solution for implementing what would boil down to an additional property (property!) of the trill: how much of the note does it 'cover', which would indeed be a very specific use case (so specific that I fail to see any actual use) and it is not connected with my specific example (it is in a sense the opposite of it). Your bringing such an example up let me suspect we are not really speaking of the same thing.

Time-gate is a property of the note (a property already present in the file structure, mind) and manipulating it via the piano roll makes little sense as the piano roll has (or at least presents to the user) no notion of 'notes', only of (some of the) MIDI events. Should the piano roll be extended to include the notion of 'note', it would become apt to deal with this concept; still, it would be slower and more cumbersome to use for this purpose than the score itself (see the orchestral example).

Lastly, time gate is not visible from the score, like note tuning, note play/not-play and note velocity are. This is why these properties are accessed from the inspector.

@MarcSabatella

This comment has been minimized.

Copy link
Contributor

commented Oct 28, 2015

FWIW, to me, whether a change is made in piano roll or inspector does not matter a whole lot except for one thing: the inspector allows you to change the easily property for multiple notes at once - even a discontiguus list selection - and the piano roll does not. If the piano roll could be made to operate on a selection like the inspector does, I think I would not care either way which solution is chosen. As it is, I am inclined to agree with @mgavioli that the inspector seems more logical for now, especially for this extremely common use case: make a selection notes and want to change the length of all by a certain amount. Right now I'm not sure how the piano roll might be extended to handle that use case well, but I concede it would be possible, and then @lasconic arguments about the extensibility of the two approaches starts to sway me more: the piano roll seems the right way to go once we look beyond these specific properties.

@mgavioli

This comment has been minimized.

Copy link
Contributor Author

commented Oct 28, 2015

@lasconic @MarcSabatella : Only another run, then I'll shut up, as apparently I am unable to explain myself clearly enough.

  1. Still no use case has been proposed in which time-gating of individual MIDI events would be useful / necessary, as opposed to time-gating the note as a whole. This would be the only case where using the piano roll would expose elements not accessible in other ways (the MIDI events themselves) and the only case where using the piano roll would be preferable to other ways.

By far the most common case is time-gating a note as a whole and the piano roll lacks (or hides from the user) the notion of note, only the MIDI events. The score exposes the notes as notes, in a clear and easily recognizable way, the piano roll does not. This is the main conceptual difference between working via the piano roll or via the inspector: the level at which each one works. And this is why note manipulation is better achieved via the score / inspector (actually only via the score / inspector).

  1. In addition to a disjoint selection of notes, there is at least another, musically quite relevant, use case in which the piano roll is clearly inferior to working via the inspector: time-gating an orchestral chord across several parts; the piano roll forces to access each part separately. Suppose someone want to articulate the 'chords' at meas. 19-21 of Beethoven's Fifth Symphony: he would have to bring the piano roll up twelve times, one for each part, and for each spend time to locate the right note(s). And Beethoven orchestra is not that big (but even a string quartet would already be a pain in the neck) !

  2. Which brings me to my last point: has anybody actually tried to use the piano roll for some meaningful time-gating task (i.e. not just to check if and how it works, but to achieve a specific goal)? I suspect not: I tried and I almost immediately gave up, because it is a surprisingly slow, cumbersome and counter-intuitive way of working, particularly if compared with the score + inspector way of working; not only because of the current limitations of the piano roll, but also because of the tenuous musical meaning of the drawings one looks at. Before expecting too much from (yet to come) piano roll improvements, I suggest to check if:

3a) these improvements can actually be implemented (for instance, having multiple parts shown in the piano roll would create big problems, but without multiple parts, any meaningful work is unacceptably slow and complex)

3b) the basic piano roll paradigm (lack of note concept, lack or obscurity of musical identity of the 'events') is compatible with this kind of operations.

If the above is not clear enough, then I have no idea of how to express myself better.

@MarcSabatella

This comment has been minimized.

Copy link
Contributor

commented Oct 29, 2015

I thin I do understand what you are saying, and as I said, with the piano roll as it exists currently, or even as I could imagine it looking without a major overhaul, I think you are right. In particular with respect to point 2 - it's really hard for me to imagine the piano roll working well with selections across staves etc. And of course point 3 - the piano roll is indeed currently quite cumbersome to use.

But if someone were to totally reinvent the piano roll in a way that somehow solved these problems, I'd be open to it, and can see reasons why that might turn out to be preferable long term. I do have my doubts about the feasiblity of this, though. Like you say, it's a tenuous relationship between the graphical display and what is actually selected in any but the simplest of cases.

@mgavioli

This comment has been minimized.

Copy link
Contributor Author

commented Apr 19, 2017

A year and a half has passed, several MuseScore versions went out and the most important (at least for me) missing aspect of the playback -- i.e. fine articulation -- is still missing; the piano roll made no progress and its expected magic is still to appear.

As my view remains that time gating is a property of a note and the piano roll is not the proper context to work on this property and as no use case has emerged which inherently requires the use of the piano roll for time gating, I wonder if it is not the case to re-evaluate this point.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.