8074211: javax.sound.midi: Error with send System Exclusive messages of different length

[AlecJY]

JVM attempts to reuse the buffer for sending MIDI out data when the buffer size is enough. It use dwBytesRecorded in MIDIHDR structure to indicate the actual size of the data. However, midiOutLongMsg() ignores dwBytesRecorded, although it did not mentioned in the documentation. I've tested on Windows 7, 10 and 11. All of them have the same behavior.

The bug cannot be easily reproduced because some MIDI drivers filter out any malformed MIDI data. The example code below create a special case to make sure all MIDI data are legally when the bug is triggered.

import javax.sound.midi.*;

public class MidiTest {
    public static class RawMidiMessage extends MidiMessage {
        public RawMidiMessage(byte[] data) {
            super(data);
        }

        @Override
        public Object clone() {
            return new RawMidiMessage(this.getMessage());
        }
    }

    public static void main(String[] args) {
        var deviceInfos = MidiSystem.getMidiDeviceInfo();
        for (var info : deviceInfos) {
            try (MidiDevice device = MidiSystem.getMidiDevice(info)) {
                if (device.getMaxReceivers() != 0) {
                    System.out.println("Open MIDI port: " + info.getName());
                    device.open();
                    Receiver receiver = device.getReceiver();
                    // Send two sysex messages at once
                    receiver.send(new RawMidiMessage(new byte[]{
                            (byte) 0xF0, 0x7D, 0x01, (byte) 0xF7,
                            (byte) 0xF0, 0x7D, 0x02, (byte) 0xF7
                        }), -1);
                    // Send another sysex message
                    receiver.send(new RawMidiMessage(new byte[]{(byte) 0xF0, 0x7D, 0x03, (byte) 0xF7}), -1);
                }
            } catch (MidiUnavailableException e) {
                e.printStackTrace();
            }
        }
    }
}

The expected messages received should be the following three messages

F0 7D 01 F7
F0 7D 02 F7
F0 7D 03 F7

But acually four messages was received with the second message repeated twice.

F0 7D 01 F7
F0 7D 02 F7
F0 7D 03 F7
F0 7D 02 F7

To resolve the issue, I add a new variable to backup the actual buffer size and set dwBufferLength of MIDIHDR structure to the size of MIDI data. After calling midiOutLongMsg(), I restore the original buffer size if the buffer hasn't been freed due to an error.

It seems that the patch may also resolve JDK-8250667. The extra bytes in the second sysex message is the same issue as JDK-8074211. I didn't figure out how the scrambled data generated in the third sysex message, but all the messages are correct after applying the patch.

---

Progress

• Change must be properly reviewed (1 review required, with at least 1 Reviewer)
• Change must not contain extraneous whitespace
• Commit message must refer to an issue

Issues

• JDK-8074211: javax.sound.midi: Error with send System Exclusive messages of different length (Bug - P4) ⚠️ Issue is not open.
• JDK-8250667: MIDI sysex over USB scrambled when reply length matches previous message (Bug - P4)

Reviewers

• Phil Race (@prrace - Reviewer)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/16399/head:pull/16399
$ git checkout pull/16399

Update a local copy of the PR:
$ git checkout pull/16399
$ git pull https://git.openjdk.org/jdk.git pull/16399/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 16399

View PR using the GUI difftool:
$ git pr show -t 16399

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/16399.diff

Webrev

Link to Webrev Comment

[bridgekeeper]

👋 Welcome back AlecJY! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk]

@AlecJY The following label will be automatically applied to this pull request:

• client

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

[AlecJY]

/issue add 8250667

[openjdk]

@AlecJY
Adding additional issue to issue list: 8250667: MIDI sysex over USB scrambled when reply length matches previous message.

[mlbridge]

Webrevs

• 04: Full - Incremental (18283c4a)
• 03: Full - Incremental (4ae4486b)
• 02: Full - Incremental (819e2cff)
• 01: Full - Incremental (a6b499f3)
• 00: Full (16918dd5)

[mlbridge]

Mailing list message from Florian Bomers on client-libs-dev:

Interesting!
I've reviewed this fix, it looks good to me.

For drivers that send Sys-Ex asynchronously, the dwBufferLength field
/might /be read after the code sets it back to the temporary variable
bufferLength. Trying to account for that would be much more work...
But this fix will improve matters in any case.

Florian

On 27.10.2023 13:51, null wrote:

--
Florian Bomers
Bome Software
/everything sounds./
https://www.bome.com
Bome Software GmbH & Co KG Gesellschafterin:
Petra-Kelly-Str. 15 Bome Komplement?r GmbH
80797 M?nchen, Germany Gesch?ftsf?hrung: Florian B?mers
Amtsgericht M?nchen HRA95502 Amtsgericht M?nchen HRB185574
LinkedIn <https://www.linkedin.com/company/bomesoftware/> YouTube
<https://youtube.com/bomesoftware> Facebook
<https://www.facebook.com/BomeSoftware/> Instagram
<https://www.instagram.com/bomesoftware/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/client-libs-dev/attachments/20231027/617c541b/attachment-0001.htm>

[mrserb]

I think the java code in the description can be added as a testcase, even if it works fine before the patch, it is useful that the affected code will be triggered.
As an example you can use the tests in the next folder: https://github.com/openjdk/jdk/tree/master/test/jdk/javax/sound

[openjdk]

@AlecJY Please do not rebase or force-push to an active PR as it invalidates existing review comments. Note for future reference, the bots always squash all changes into a single commit automatically as part of the integration. See OpenJDK Developers’ Guide for more information.

[AlecJY]

Thanks for the suggestions from Florian Bomers and @mrserb.

I’ve added a new structure to record the actual buffer length. Therefore the actual buffer length doesn’t need to write back to dwBufferLength. Additionally, The example code is added as a testcase.

Please let me know if there are any further improvements needed.

[prrace]

It use dwBytesRecorded in MIDIHDR structure to indicate the actual size of the data
However, midiOutLongMsg() ignores dwBytesRecorded, although it did not mentioned in the documentation.

This seems to be documented by Microsoft only in the most obscure fashion.

If you look at the doc for midiOutPrepareHeader
https://learn.microsoft.com/en-us/previous-versions/dd798477(v=vs.85)
it says "Before calling the function, set the lpData, dwBufferLength, and dwFlags members of the MIDIHDR structure"

So no mention of dwBytesRecorded.

WAVEHDR is a very similar struct to MIDIHDR and if you look at this WAVEHDR doc it says

https://learn.microsoft.com/en-us/previous-versions/dd743837(v=vs.85)

dwBytesRecorded
When the header is used in input, specifies how much data is in the buffer.

So I infer the same is true for MIDI

Therefore dwBytesRecorded is only used for input and dwBufferLength is what's important for output.

[prrace]

I ran this change through our CI testing system and it all looks good.

[openjdk]

@AlecJY This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8074211: javax.sound.midi: Error with send System Exclusive messages of different length
8250667: MIDI sysex over USB scrambled when reply length matches previous message

Reviewed-by: prr

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 27 new commits pushed to the master branch:

• 1588dd9: 8319567: Update java/lang/invoke tests to support vm flags
• 9727f4b: 8320199: Fix HTML 5 errors in java.math.BigInteger
• d6aa7c8: 8314621: ClassNotFoundException due to lambda reference to elided anonymous inner class
• 52e2878: 8319987: compilation of sealed classes leads to infinite recursion
• b05e69f: 8320209: VectorMaskGen clobbers rflags on x86_64
• f3ed275: 8319103: Popups that request focus are not shown on Linux with Wayland
• 9e7a3ae: 8319630: Monitor final audit log lacks separator
• 87be6b6: 8318757: VM_ThreadDump asserts in interleaved ObjectMonitor::deflate_monitor calls
• 9faead1: 8319927: Log that IEEE rounding mode was corrupted by loading a library
• f33c874: 8319764: C2 compilation asserts during incremental inlining because Phi input is out of bounds
• ... and 17 more: https://git.openjdk.org/jdk/compare/2e34a2ebf0f14043b129461b0397495e7e75a38b...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

As you do not have Committer status in this project an existing Committer must agree to sponsor your change. Possible candidates are the reviewers of this PR (@prrace) but any other Committer may sponsor as well.

➡️ To flag this PR as ready for integration with the above commit message, type /integrate in a new comment. (Afterwards, your sponsor types /sponsor in a new comment to perform the integration).

[AlecJY]

/integrate

[openjdk]

@AlecJY
Your change (at version 18283c4) is now ready to be sponsored by a Committer.

[prrace]

/sponsor

[openjdk]

Going to push as commit e47cf61.
Since your change was applied there have been 82 commits pushed to the master branch:

• 6d82436: 8320278: ARM32 build is broken after JDK-8301997
• f69e665: 8318364: Add an FFM-based implementation of harfbuzz OpenType layout
• 1c0bd81: 8319124: Update XML Security for Java to 3.0.3
• 61d81d6: 8317742: ISO Standard Date Format implementation consistency on DateTimeFormatter and String.format
• c4aba87: 8320272: Make method_entry_barrier address shared
• 9311749: 8320526: Use title case in building.md
• 9598ff8: 8315969: compiler/rangechecks/TestRangeCheckHoistingScaledIV.java: make flagless
• 53eb6f1: 8187591: -Werror turns incubator module warning to an error
• 570dffb: 8310807: java/nio/channels/DatagramChannel/Connect.java timed out
• 21a59b9: 8282726: java/net/vthread/BlockingSocketOps.java timeout/hang intermittently on Windows
• ... and 72 more: https://git.openjdk.org/jdk/compare/2e34a2ebf0f14043b129461b0397495e7e75a38b...master

Your commit was automatically rebased without conflicts.

[openjdk]

@prrace @AlecJY Pushed as commit e47cf61.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

[AlecJY]

@prrace @mrserb Thank you very much.