Sporadic sysEx msg broadcast/capture

[bossredman]

Hi

I have a self built midi foot controller for my AxefX2 guitar processor.
It's Teensy 3.6 based & has both midi IN & OUT circuits.

One particular switch triggers the preset to increment by 1 via the following commands:

MIDI.sendControlChange(0,BANK,1);                   
MIDI.sendProgramChange( PRESET_NUM, 1 );            
MIDI.sendSysEx(8,EffectID_Sysex,true);

I call MIDI.read at the end of my void_Loop().

In turn my TFT display gets updated with PSet number, Bank Number, PSet name and all the Effect blocks in the Pset.

On the whole it works fine - but now and again I notice that the PSet name does not get up dated on the TFT - even though everything esle does.

I've noted the following whilst trouble shooting:
1- the CC cmd if sent on its own triggeres NO sysex msgs
2- the PC cmd if sent on its own triggeres the AFX to broadcast PSET NUMBER & PSET NAME
3- the EffectID_Sysex cmd if sent on its own triggeres the AFX to broadcast IA, Set SCENE & PARMAETR ID sysex msgs.

When all this happens - everything gets updated OK.
Occasionally when the name is not updated - it corresponds with the PSET name msg not being sent.
That makes sense I guess - but what I don't get is what would prevent this msg being recieved.

I'm not specifically requesting this msg - it's getting automatcially broadcast after I send the Program Change cmd.
I even tried adding a request sysex for the PSET name - but again only works sporadically.

Any ideas why this would be.
Dont know enough about sysex msgs to know if its a timing issue or whether by their nature they are flakey anyway OR whether its just bad code on my behalf.

[bossredman]

My sketch is very slow, what's going on ?

First, you need to call MIDI.read() frequently, to have the lowest latency possible. Raw MIDI data is stored in the Serial buffer, but the library can only extract it when you call MIDI.read().

Make sure that your callbacks are not blocking: they should be short and not induce delay (ie: they return as soon as possible), as they are launched from within MIDI.read (thus in the loop).

Just read this on the "callbacks section".
I have all my code for processing the various sysex ID types I'm interested in in one block/function (I'm guessing a Callback actually :) now I've read that).

One of the ID's block is quite detailed & also aCTUALLY sends another sysex msg request from within it.
Could this potentially be impacting things & causing me to miss a msg??
Not sure how else to do it though.

[bossredman]

Aplogies not too clued up on callbacks.

But does the callback run in parallel to the main Loop() code?

[bossredman]

Is there a detailed explanation of how the midi.read() & callback works anywhere pls.

Struggling to understand how this works to be able to trouble shoot my issue.

ie
1 - once a msg is read is it no longer readable (ie is it deleted/disgaurded) unless sent again?
2 - within the call back function code - is it best to use seperate "if" statements for each sysex msg ID or better to use if - else if . Reason I ask is it appears to me that the callback function only runs once per midi.read().

[franky47]

Here are a few pointers:

• The callbacks are called from within MIDI.read(), meaning that they will not run in parallel with the rest of your loop (hence why they must be lightweight in terms of processing, in order not to slow the loop too much, which would cause latency).
• When a message has finished parsing (ie: all the necessary bytes have been received), the corresponding callback (if any was attached with setHandleXXX) will be called with the message data, synchronously (ie: from within MIDI.read()). After MIDI.read() returns true, you can access the message data with getType, getChannel, getData1, getData2, getSysExArray and getSysExArrayLength. The information returned there will be valid until the next time a message has finished parsing (and therefore your callback has fired and MIDI.read() returned true).

[bossredman]

thanks - I'm obviously not an experienced/knowledgable programmer by my questions :) - but what difference does it make as to whether teh callback code runs long or the loop processes that code - if nothing is running in parallel

[franky47]

Nothing from your code runs in parallel, but the MIDI stream does not stop, and thus slowing down the loop will make the parsing slower, therefore lead to latency between the actual reception of the byte in the Arduino RX buffer and the action in your code. It's like a toll booth, if you want traffic to be fluid, you need incoming cars to exit as fast as possible to make space for the next incoming cars.

Obviously, this kind of reasoning assumes a continuous flow of MIDI data, if your application only uses event-based communications (sending/receiving messages when a discrete event happens every now and then), then this is much less of a constraint.

[bossredman]

Thanks again for your explanation.
Still a little confused though -sorry.
If nothing is running in parallel, then if the callback takes 3 secs to run (exagerated example), then it would still take 3 secs to run in the loop right?
Eitherway the loop is slowed down.

At this point, I'm only looking to read msgs after I send a request out to receive one, either:

• directly - ie I send a sysex msg request
• indirectly - ie I send a Program Change cmd (which then appears to auto generate a msg from my AxeFX).
  So there shouldn't be a much of problem - but there appears to be one - ie I keep missing teh name msg.

[bossredman]

When a message has finished parsing (ie: all the necessary bytes have been received), the corresponding callback (if any was attached with setHandleXXX) will be called with the message data, synchronously (ie: from within MIDI.read()). After MIDI.read() returns true, you can access the message data with getType, getChannel, getData1, getData2, getSysExArray and getSysExArrayLength. The information returned there will be valid until the next time a message has finished parsing (and therefore your callback has fired and MIDI.read() returned true).

Sorry for more questions.

So is only one msg availble to be read at any one time?
And if another msg comes along before the previous was "read" - then it gets over written/deleted?
Is there anyway to know how many msgs are waiting to be read?
Will teh callback code only run once per MIDI.read() = true call?

[franky47]

So is only one msg availble to be read at any one time?

Yes, the last one received.
For SysEx, if you plan on referencing the message data for longer than one loop, you should copy it as the array might be overwritten by incoming bytes of another SysEx message when calling MIDI.read().

And if another msg comes along before the previous was "read" - then it gets over written/deleted?

Yes.

Is there anyway to know how many msgs are waiting to be read?

Not until they've been parsed and therefore passed onto your program through callbacks or direct access.

Will the callback code only run once per MIDI.read() = true call?

Yes. Suppose that you have a callback for NoteOn, and the RX buffer contains two NoteOn messages. The first message will be parsed, recognized as NoteOn, the callback will be fired with this data, then MIDI.read() will return true, and the next message will be parsed and handled at the next round of the loop.

[bossredman]

Thanks again Franky.

This has helped me understand things a little better.
I've actually now changed my approach to sending & recieveing teh sysex msgs.
I'm now sending all CC, PC & sysex requests (required to update the info on my display) in one go.
ie
CC for Bank change
PC for Preset change
sysex for Scene umber chnage
sysex for Preset Name
sysex to get Instant Access switches & bypass states.

I've also added some code to wait for each returned msg before sending the next change/request.
(this has helped eliminate the missed msgs).
ie

while (MIDI.read() == false)
            {
              //Serial.println("...... waiting to read sysEx buffer [1]");
            }
Serial.println("Found something to read in sysEx buffer [1]");

It seems to be working alot better & more consistantly now.

I am however noticing one particular msg seems seems to take longer to "broadcast" (and thus read) than the others.
It's PSET NUMBER.
It varies from between 275 ms to 540 ms to be broadcast (ie send from the AFX).
This is after my code sends a Program Change (which seems to auto generate a response from teh AXEFX with the PSET NUMBER sysex.

Anything obvious on this one pls?
Realise it could simply be teh AFX be slow to send out the replay - but not sure how to prove that.
ALso any is PC's are generically slower than sysex requests.
If yes - then I could possibly look to make the PSET number change via sysex as opposed to PC.
(Just thinking out load now).
[edit] - actually dont think I can chnage Preset number via sysex - just get the current preset number - oh well :(

[Gabryxx]

You have try increase Sysex buff size?

struct MySettings : public midi::DefaultSettings{
static const unsigned SysExMaxSize = 1024; // Accept SysEx messages up to 1024 bytes long.
//static const bool UseRunningStatus = true; // My devices seem to be ok with it.
};
MIDI_CREATE_CUSTOM_INSTANCE(HardwareSerial, Serial, MIDI, MySettings);

Seem Arduino has many problems in dialog with peripherial (and bad isolation from external power: i have destroy a China Nano after 1 hour that work with a external powered servo), and also if i made a good midi interface with a good octopuler and square signal, is no way to receive all data transmitted from midi keyboard when i send all banks/patches/rithms (that normally work with a pseudo Sound Blaster with drives for Win7 64, also if is not certified and i need start Win without driver certified for use midi keyboard).
I have made a Sysex patch/Timbre editor for Roland D10 keyboard without any problem on Amiga Commodore that got a 8 Mhz processor and a external midi interface...favolous years '80....
Bye

[franky47]

ProgramChange messages are inherently shorter than SysEx, but still for your use case it should not matter, unless you plan to send a large amount of SysEx data to your AxeFX2.

Also using the Serial port for debugging slows the loop down, it's not an issue for sending data as there is no parallelism involved there, but if you find the reception on your controller to be lagging, that could be a cause.