MIDI out regression. large amount of data can't be send in real time on OSX.

[ch-nry]

It's no more possible to send large amount of MIDI data using pd 0.49 on osx. everything is fine on linux, and I did not test windows. It works on OSX using an old pd-entended.

This is my test patch:
test_midi_out.pd.zip

I can tell with the led of my MIDI device when data are received. With Pd 0.49 on osX, it take 2 second to receive what I send in 1 second.
I know that all data are transferred (no data lost) on linux, because I use this to control DMX device.

[danomatika]

Is this happening only on macOS or other operating systems too?

[Lucarda]

On Windows (2 sound cards connected with a midi cable) "test_midi_out.pd" takes around 1 sec to stop.
I'm not sure if this is wrong considering the 1024 [until].

[Lucarda]

Changing [random 128] for a counter I can see that not all values are received:

[Lucarda]

Using a "virtual loopback MIDI-port" all messages are sent/received:

test patch with counter.
Windows 8.1

[ch-nry]

Test should be made only with USB MIDI : a midi cable is very slow so data can't pass at high speed. Pd is not the bottleneck when using a "real" midi cable.

[Lucarda]

Pd is not the bottleneck when using a "real" midi cable.

Yeah I know.
With Pd using an m-audio(fast track pro) not connected to anything if I do 1 second of the test patch I get the led shining around 2 seconds more.
Also when I click "off" the toggle it holds "on" for around 1 sec (the [until] thing).

[Lucarda]

With Pd-extended 0.44 I get the same behavior as above with the fast-track-pro.

[ch-nry]

Since the m-audio fast track did not know if a midi cable is plugged or not, this is normal : the midi data are send 1 messages every about 0.6ms, so it can't handle so much data.
For testing, I use a device that receive midi data but did not send them back, so it can handle them at usb speed.

so : to summarize :
buffer size : windows have a very small buffer and data are lost relatively fast.
linux have big buffer size.
osX have a buffer size that look unlimited (data are never lost)

about speed : windows : I don't know
Linux : everything is ok (every pd or linux version)
osx: works on extended, slower on pd 0.49 (tested using osX.12.3)

[Spacechild1]

I tested on Windows with a USB MIDI interface and turning on the metro gives me a constant ~1 second lag in the Pd GUI... so sending MIDI seems to be blocking Pd although portmidi uses a lockfree ringbuffer...

EDIT: turns out portmidi only uses a lockfree queue for receiving MIDI, so sending MIDI can indeed block Pd. I think this should be fixed by doing the actual sending in a worker thread.

[Spacechild1]

FWIW, on Windows I get the same blocking behavior on Pd extended 0.43.4 and on Pd vanilla 0.46 - 0.49 (I didn't check older releases).

@ch-nry did you try on macOS with older Pd vanilla releases? can you give us a Pd vanilla version which works for you?

[ch-nry]

I did not test other pd vanilla version. I did not have a mac at home, I may be able to test in few weeks.

[Spacechild1]

I think I've found the relevant difference: the portmidi version used by Pd vanilla throttles the data rate on macOS with a call to usleep while the version used by Pd extended 0.43 doesn't:

pure-data/portmidi/portmidi/pm_mac/pmmacosxcm.c

Line 492 in 7c27aa0

usleep((useconds_t)

this means that sending too much data in too short time will block Pd. check if your problem goes away if you spread the 1024 CC messages over several blocks and report back. If not, we can patch portmidi but generally it would be great to make sys_queuemidimess itself non-blocking.

[Spacechild1]

actually, I think its possible to disable the "speed limit" with #define LIMIT_RATE 0 (EDIT: actually we have comment the line out because the code uses #ifdef)

pure-data/portmidi/portmidi/pm_mac/pmmacosxcm.c

Line 66 in 7c27aa0

#define LIMIT_RATE 1

however, I think it's there for a reason, so I'm not sure if we should disable it by default...

[danomatika]

I will also try this on macOS with a hardware loop when I get the chance. Thanks for the test patch as previous issues about this were a bit more vague.

[ch-nry]

I test to split to 1024 CC message in 16 block of 64 CC. I did not see any difference (it did not work any better)
test_midi_out.pd.zip

[Spacechild1]

ok, can you try to build portmidi with #define LIMIT_RATE 0 and see if that helps?

[ch-nry]

Compiling Pd is not simple for me : It's not my computer, I never used a mac, I don't know how to compile pd on osX.
Can you easily compile it and send it to me?

[Spacechild1]

Here you are: Pd-0.49.0.zip

I haven't tested this build whatsoever. Let me know how it works for you.

BTW, I actually had to comment #define LIMIT_RATE 0 out because the code uses #ifdef LIMIT_RATE.

[ch-nry]

thanks for the compilation. It's working, and it solve my problem!
(I just did a quick test, I don't have the full hardware to do a deeper test).
I think this flag should be removed : midi buffer can be flood, but that's consistent with the rest of pd and with other OS.
thanks!

[danomatika]

Sounds good. Thanks for testing. We can look into fixing this via a poet midi patch or upstream issue, etc.

[Spacechild1]

I think in the portmidi code the #ifdefs should be replaced by #ifs and LIMIT_RATE should be made overridable by the build system:

#ifndef LIMIT_RATE
# define LIMIT_RATE 1
#endif

In Pd we would just have to pass -DLIMIT_RATE=0 to the compiler. I think this could go upstream... Who should do it?

[danomatika]

I can pass it on. There isn't really a formal way I could tell to do it so I just end an email...

[danomatika]

Should be fixed via 8304bde. I will send an email upstream.

[ch-nry]

thanks!

[danomatika]

It was indirectly my fault as I updated portmidi the last time. ;P

[rbdxup]

Sorry - I just found out about this thread and "fix" from Dan Wilcox. The reason OS X has rate limiting in PortMIDI is that without limiting, OS X will drop MIDI messages. At least that was the state of things some years ago. As far as I can tell, this is a design flaw in CoreMIDI that allows you to have a circular forwarding path among virtual (IAC Bus) MIDI devices. This would cause CoreMIDI to go compute bound, so the solution/hack in CoreMIDI is to drop messages. Unfortunately, CoreMIDI does not distinguish between true problems and simple cases where you try to send a lot of MIDI. I think dropping MIDI messages is unforgivable, so the work-around in PortMIDI is to restrict the MIDI data rate so that messages are not dropped by OS X.

I think if you want a real-time application to exceed the bandwidth of an output channel (MIDI), then there's something wrong with the application, so if PortMIDI stalls the application and you notice, that might be considered a feature.

If you really want your application to be allowed to send more data than can be reliably transmitted, I would suggest making that a special case, e.g. a "best effort" output that drops messages rather than calling usleep() -- this would require an extended PortMIDI API, and perhaps that should be provided. I think that would be much better than allowing OS X to arbitrarily drop messages.

Another alternative, suggested above, is to buffer the output. PortMIDI does not do this because PortMIDI is supposed to be a "thin" layer over existing API's and not "fat" system that's going to insert a thread and buffers between your output calls and the OS.

If OS X has changed it's implementation to where it does not drop messages, or if anyone can reverse engineer the message-dropping policy, which might allow more MIDI bandwidth without risking dropped messages, I'd be happy to learn more.

[ch-nry]

The test patch did not exceed midi (using usb) bandwith. I can attest that all message used to be send. Without this patch, the maximum bandwidth drop. With this patch, I can attest that there are no delayed message, but can't test that all messages are send.

If you exceed midi bandwidth, then there are no good solution : dropping message or delay them are both equally bad.

[rbdannenberg]

Unless OS X has changed, it will drop messages if the rate is too high. This was confirmed by Apple. I'm not sure how that rate compares to USB bandwidth, and Apple was not able to tell me in detail how to avoid dropped messages. E.g. if we knew what model they used (time between messages? issue a quota every 50ms? some kind of smoothing filter estimator?) maybe we could avoid exceeding the limit. Whatever you do, I think you should set up a loop-back test through IAC (which should be less rate-limiting than even USB) and see if anything is dropped.
(Just for the record, I used a different account -- rbdxup -- above by accident. rbdxup and rbdannenberg are both my (Roger Dannenberg's) accounts. Thanks for the discussion.

[Spacechild1]

@ch-nry

dropping message or delay them are both equally bad.

I disagree, dropping messages can be disastrous for Sysex, for example.

[Spacechild1]

@rbdxup

Another alternative, suggested above, is to buffer the output.

it's probably the better solution as it would generally prevent MIDI from blocking the audio thread. But I'm not sure if it would solve Cyrille's problem because at least in his test patch he tries to send more MIDI messages than what OSX is supposingly able to handle. But where do you get the "15000 bytes/second" number? Is it official?

EDIT: I'm talking about buffering/threading on the Pd side, just to be clear.

[Spacechild1]

please try that one: Pd-0.49.0-midi-io-fix3.zip

[ch-nry]

it's working!
thanks a lot!
sending time is about 0.1ms, so I receive the syncro on time and I can make the 1024 DMX chanel to change between 2 frames.

[Spacechild1]

Halleluja! The portmidi code missed some #ifdefs to actually turn off the rate limit. I'll patch it so we can properly enable/disable the speed limit per compile time flag. Then we just need to discuss what should be the default behavior.

[rbdannenberg]

I think audio and midi should always be in sync : if midi can't handle speed, then audio should stop and wait for midi, just like what we have with image using GEM.

I personally disagree. MIDI is way less time-sensitive than audio, and I think we shouldn't compromise audio, never.
Having audio struggling because of the MIDI bandwidth, can bring us scenarios where glitches are caused by data transfers. I don't like the idea.

Sometimes it's good to think about these problems in terms of policy and mechanism. Generally, mechanisms (e.g. MIDI buffers to avoid blocking audio or synchronization enforcement that might block audio) are good, but enforcing specific policies (such as audio should never block, or MIDI should never be dropped) should at least be considered carefully.

[ch-nry]

thanks for your work.
For the default behaviors, I think my pov is obvious. Let's see other pov.

[Spacechild1]

@rbdannenberg the portmidi code misses some more #ifdefs for LIMIT_RATE. I'll patch portmidi for Pd and send it upstream. Although you're right that CoreMIDI does drop messages (at least we could observe it via the IAC), it's good to also have an option (compile time flag) to disable it.

[ch-nry]

@rbdannenberg : the rate limit is safe when using a USB/MIDI device that can't send the data faster than the midi rate (31250kbps), but is over protective when using other device or virtual port.

[Spacechild1]

@ch-nry no, the 15 kB/s are CoreMIDI's hardcoded policy for feedback protection. that's why we lost data when sending via IAC without portmidi's rate limit.

Apple might have increased the limit, but we don't know... we have to test

[rbdannenberg]

Yes, 31250kbps is about 3KB/s which is way less that the PortMIDI/OS X rate limit. Has there been a careful test of high-speed MIDI over USB or virtual port other than IAC? If data is only dropped by IAC, then the next question would be is it possible to distinguish IAC ports that limit data from other ports.

[ch-nry]

I "think" we will experience data drop whenever the data go out slower than it goes in. With IAC or my DMX interface, data goes out very fast, so it's hard to see data drop.
With a usb/midi device, even with this limit (as you point out), we will probable experience data drop.
The good solution would be to have feedback from coremidi...

[ch-nry]

@Spacechild1 : ok, but my patch show that we can go lot's higher without data lost.

So, is the solution to measure with IAC the real coremidi feedback protection and adapt to this limit?

[Spacechild1]

The good solution would be to have feedback from coremidi...

but it's also the most unlikely solution :-).

So, is the solution to measure with IAC the real coremidi feedback protection and adapt to this limit?

maybe, but actually I tend to think that Pd should use portmidi without the rate limit and warn the user about sending too much data at once on macOS. it's trivial to implement the speed limit yourself in Pd, this is where it differs from other applications like DAWs.

EDIT: iemlib even has a [speedlim] object :-)

[ch-nry]

I also agree that the limit should be disable in pd!

[rbdannenberg]

I agree this is an OS X CoreMidi design flaw. The internet uses TTL (time to live) counters, which seems heavy-handed for MIDI messages, but it would be better than what we have now. But enough of wishful thinking.

I think we have a pretty good idea that the IAC limit is 15000 x 6 bytes over a 6-second window consisting of 6 1-second buckets. But this is hard to model since we don't know the bucket boundaries. Also it seems to depend on timestamps. I think we could at least allow bursts of MIDI data without any blocking (as apparently allowed in CoreMidi) without much work.

[Spacechild1]

I think we could at least allow bursts of MIDI data without any blocking (as apparently allowed in CoreMidi) without much work.

this sounds reasonable. would you still be willing to accept a patch for disabling portmidi's built-in rate limit via a compile time flag (e.g. -DNO_RATE_LIMIT)? I think this is what we would actually need in Pd because we rather want to handle the speed limit on the Pd side.

[rbdannenberg]

Yes, we can make this a compile time flag. I will try to figure out if this will result in unreliable MIDI and there will either be a comment saying this is a really, really bad idea or the comment will say if you disable rate limiting, PortMidi will not be reliable with IAC ports (also a bad idea). Upstream has gotten way behind: there's always too much to do (if you are in Pittsburgh in Feb, come hear my opera!), but I'm hoping to make a big pass of updates and testing sometime in July.

[danomatika]

From following this thread, I still don't quite understand how your patch works *perfectly* with Pd-extended. If the problem is in CoreMIDI, then it should *not* run perfectly in Pd-extended. This is on the same system, the same patch, the same macOS version with the same CoreMIDI. The only difference would then be the Pd and portmidi sources.

…

On May 22, 2019, at 12:57 PM, cyrille henry ***@***.*** ***@***.***>> wrote: If coremidi did not provide feedback on the % used of the buffer or the sending rate (that depend of the interface), then well.. we don't have good solution. - remove the rate limit and accept possible data loss - keep the limit rate (which is “safe“) and make it impossible to send lot's of midi data to interface that can receive them. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#581?email_source=notifications&email_token=AADVK7KBAY2RMAWN56WOAQTPWURI5A5CNFSM4HEZ2YQKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODV6VVEA#issuecomment-494754448>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AADVK7PC7E7PRQYVXACFHCTPWURI5ANCNFSM4HEZ2YQA>.

-------- Dan Wilcox @danomatika <http://twitter.com/danomatika> danomatika.com <http://danomatika.com/> robotcowboy.com <http://robotcowboy.com/>

[Spacechild1]

cool, I'll send you a patch then. And thanks a lot for taking the time to share your thoughts on this issue!

if you are in Pittsburgh in Feb, come hear my opera!

not quite close to Austria, but I have no plans already for February, so in case I happen to be in US... :-)

[Spacechild1]

@danomatika Pd extended uses an old portmidi version. the problem was merely in the updated portmidi in Pd 0.49 (the rate limit via usleep).

[Spacechild1]

disabling the rate limit can lead to dropped messages because of CoreMidis hard-coded feedback protection policy, but it might be that it only concerns the IAC and not when sending to a "real" device, like in the case of Cyrille's DMX device.

[danomatika]

Initial patch commit reverted in update/0.50 branch.

[danomatika]

I'm hoping to make a big pass of updates and testing sometime in July.

That would be helpful to us and the Pd release cycle revolves around Miller's teaching schedule, so naturally the 0.50 release could be some time late summer.

if you are in Pittsburgh in Feb, come hear my opera!

Sweet. Do you have any info on line?

[rbdannenberg]

Free tickets to anyone who comes from Austria! :-) No opera info online yet, but we're in Mexico City in late fall (dates TBD) and Pittsburgh, Feb 15-16, and the premiere is on youtube http://www.cs.cmu.edu/~rbd/videos.html#opera

[danomatika]

Thanks all for figuring this out. I think disabling the rate limit in Pd is the way to go for now. It never seemed to be a problem before with Pd-extended and most people can easily work around it within their patches. I have used the CoreMIDI IAC loop back for many years and never really ran into any dropped message problems, however there have been a few complaints with the latest Pd release over MIDI timing on macOS and I believe this is the culprit.

[Spacechild1]

Fixed in Pd 0.50.