Vec-less API for embedded usage?

[wyager]

I'm interested in using midi-msg for an embedded MIDI project, but I see that the core MidiMsg type relies on Vec<u8> for the internal representation of arbitrary-length message types, such as sysex messages.

Do you have any thoughts on what it might take to implement a type that has no_std support for all message types?

I could see two ways this might work:

1. Replace MidiMsg with MidiMsg<'a>, and any internal use of Vec<u8> would be changed to &'a [u8], referring to a slice of the input buffer. I'm not sure if this would work in general - are there cases where the output Vec<u8> is not a contiguous slice of the input buffer? If every output type is a contiguous sample of the input, this seems like the best approach - no memory overhead!
2. If the above won't work, replace MidiMsg with MidiMsg<Buffer>, where Buffer could be set to something like std::Vec<u8> or heapless::Vec<u8,N>, unified by a common (crate-specific) trait, and add a MessageTooLong constructor to the ParseError type (which would not ever occur while using std::Vec<u8>

I'm not sure if this would work with your design intentions for the library, but I'm probably going to have to add this functionality in any case, so figured I would check if there was some approach that would be upstreamable.

[wyager]

There are a few unfortunate constraints which make both of the above ideas difficult. Perhaps most salient is this:

MidiMsg contains a constructor with a SystemExclusiveMessage which contains a constructor with UniversalRealTime,UniversalNonRealTime which contain TimeCodeQueueingMessage which contains Event{Start,Stop},Cue which contain MidiMsgs.

Unfortunately, such a structure is not possible to represent in a bounded amount of memory. The recursive chain (as presented) could be unbounded in size, so any effort to make a fixed-size representation of a MidiMsg will fail without const-size-polymorphic type recursion (which I suspect Rust cannot do). I am not sure if this is an inherent property of the MIDI spec, or if there is a way to re-arrange the enums in this library such that there is no recursive path from a message type back to itself without violating the MIDI spec. Curious to hear if you have any thoughts on this @AlexCharlton.

[AlexCharlton]

I don't believe there's any way for to build sysex messages without managing memory. Anything else assumes there is some sort of contiguous buffer that contains all of a given message's data -- as you point out -- which is not a valid assumption according to the MIDI spec. The recursive nature of MIDI messages that you describe is also due to the spec: MIDI messages can indeed contain other MIDI messages.

That said no_std and sysex are not mutually exclusive, and Rust supports no_std Vecs. So I'd be interested to hear any reports of embedded usage in practice and if there are any practical performance implications to the API.