Managed debugger does not hit breakpoints on callbacks from native thread

[tristanlabelle]

Description

I'm P/Invoking the midiInOpen Win32 API with a callback delegate, but any breakpoints in the method are ignored when the callback happens on a midi user-mode driver thread (wdmaud.drv thread), whereas they work when the method is called on the main thread.

This reproes in a Packaged WinUI 3 project but not with a Console project nor with a WPF Core project. It also reproes with the waveInOpen APIs, see my comment below for that repro.

Reproduction Steps

Repro project: MidiBreakpointBugRepro.zip

Prerequisite: the computer must be connected to a MIDI device such as a digital piano, because the repro uses the midiInOpen API, which can only succeed in the presence of such a device.

• Open Visual Studio 2022
• Create a Blank App, Packaged (WinUI 3 in Desktop) project
• Add a NuGet reference to Microsoft.Windows.CsWin32
• Add a NativeMethods.txt file containing:

midiInOpen
midiInStart
LPMIDICALLBACK

• Modify the App class with the code below.
• Put a breakpoint at OnMidiMessage entry, where indicated by a comment.
• Start debugging.

using Microsoft.UI.Xaml;
using System;
using System.Diagnostics;
using System.Runtime.InteropService
using Windows.Win32.Media.Audio;
using Windows.Win32.Media.Multimedia;
using static Windows.Win32.PInvoke;

namespace MidiBreakpointBugRepro // Your namespace might be different
{
    public partial class App : Application
    {
        private static readonly LPMIDICALLBACK callback = OnMidiMessage;

        public App()
        {
            this.InitializeComponent();
        }

        protected override void OnLaunched(LaunchActivatedEventArgs args)
        {
            var result = midiInOpen(out HMIDIIN handle, uDeviceID: 0,
                dwCallback: (nuint)(nint)Marshal.GetFunctionPointerForDelegate(callback),
                dwInstance: 0, MIDI_WAVE_OPEN_TYPE.CALLBACK_FUNCTION);
            if (result != 0) throw new Exception("MIDI input device 0 could not be opened, is a midi device connected?");

            result = midiInStart(handle);
            if (result != 0) throw new Exception();
        }

        private static void OnMidiMessage(HDRVR hdrvr, uint uMsg, nuint dwUser, nuint dw1, nuint dw2)
        { // <-- Put a breakpoint here
            Debugger.Break();
        }
    }
}

Expected behavior

The breakpoint on OnMidiMessage entry and the Debugger.Break() both stop execution and highlight the line in source code, whether the callback is called within the midiInStart call, on the main thread, or later from the user-mode midi driver thread.

Actual behavior

When the callback is called within midiInStart on the main thread, both the breakpoint and Debugger.Break() stop execution and highlight the source code line.

When the callback is later called from the user-mode midi driver thread (wdmaud.drv thread), the method entry breakpoint is not hit and the Debugger.Break() stops execution with "Source not available" and a call stack showing:

 	ntdll.dll!NtWaitForSingleObject�()	Unknown
 	KernelBase.dll!WaitForSingleObjectEx�()	Unknown
 	ntdll.dll!RtlpCallVectoredHandlers()	Unknown
 	ntdll.dll!RtlDispatchException()	Unknown
	ntdll.dll!KiUserExceptionDispatch�()	Unknown

The net effect is that this function and all of its downstream calls cannot be debugged, except by using printf (Debug.WriteLine). It's like all int 3's had vanished on that thread.

Regression?

Regression from Console or WPF (.net core) app to WinUI 3.0 Packaged app.
Likely regression from .NET Framework.

Known Workarounds

At times, a combination of Debug.WriteLine and Debugger.Break() calls seem to revive the debugger on that thread, but it is inconsistent and not production-safe.

Configuration

.NET 6.0.1 (Visual Studio 2022 17.0.5 and 17.1.0 preview 5.0)
Windows 11, version 10.0.22000
Reproes when run as x86 and x64 both
Does not appear to be specific to that configuration

Other information

No response

[ghost]

Tagging subscribers to this area: @tommcdon
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Description

I'm P/Invoking the midiInOpen Win32 API with a callback delegate, and but any breakpoints in the method are ignored when the callback happens on a midi user-mode driver thread (wdmaud.drv thread). This reproes in a WinUI 3 project but not with a Console project.

Reproduction Steps

Prerequisite: the computer must be connected to a MIDI device such as a digital piano.

• Open Visual Studio 2022
• Create a Blank App, Packaged (WinUI 3 in Desktop) project
• Add a NuGet reference to Microsoft.Windows.CsWin32
• Add a NativeMethods.txt file containing:

MMSYSERR_NOERROR
midiInOpen
midiInStart
MM_MIM_OPEN
LPMIDICALLBACK

• Modify the App class with the code below.
• Put a breakpoint at OnMidiMessage entry, where indicated by a comment.
• Start debugging.

    public partial class App : Application
    {
        private static readonly LPMIDICALLBACK callback = OnMidiMessage;

        public App()
        {
            this.InitializeComponent();
        }

        protected override void OnLaunched(LaunchActivatedEventArgs args)
        {
            var result = midiInOpen(out HMIDIIN handle, uDeviceID: 0,
                dwCallback: (nuint)(nint)Marshal.GetFunctionPointerForDelegate(callback),
                dwInstance: 0, MIDI_WAVE_OPEN_TYPE.CALLBACK_FUNCTION);
            if (result != 0) throw new Exception("MIDI input device 0 could not be opened, is a midi device connected?");

            result = midiInStart(handle);
            if (result != 0) throw new Exception();
        }

        private static void OnMidiMessage(HDRVR hdrvr, uint uMsg, nuint dwUser, nuint dw1, nuint dw2)
        { // <-- Put a breakpoint here
            Debugger.Break();
        }
    }

Expected behavior

The breakpoint on OnMidiMessage entry and the Debugger.Break() both stop execution and highlight the line in source code, whether the callback is called within the midiInStart call, on the main thread, or later from the user-mode midi driver thread.

Actual behavior

When the callback is called within midiInStart on the main thread, both the breakpoint and Debugger.Break() stop execution and highlight the source code line.

When the callback is later called from the user-mode midi driver thread (wdmaud.drv thread), the method entry breakpoint is not hit and the Debugger.Break() stops execution with "Source not available" and a call stack showing ntdll.dll!_NtWaitForSingleObject@12�().

Regression?

Regression from Console to WinUI 3.0 app.
Likely regression from .NET Framework.

Known Workarounds

Inserting an artificial Debug.WriteLine calls seem to fix the debugger from that point on.

Configuration

.NET 6.0.1 (Visual Studio 2022 17.0.5 and 17.1.0 preview 5.0)
Windows 11, version 10.0.22000
Reproes when run as x86 and x64 both
Does not appear to be specific to that configuration

Other information

No response

Author:	tristanlabelle
Assignees:	-
Labels:	area-Diagnostics-coreclr, untriaged
Milestone:	-

[tommcdon]

@tristanlabelle Thanks for reporting this issue. I do not have a MIDI device readily available to reproduce the issue. On an initial look at the problem, the issue appears to be stack inbalance issue caused by incorrect PInvoke signatures.

The callback function signature for midiOpenIn is as follows:

void CALLBACK MidiInProc(
   HMIDIIN   hMidiIn,
   UINT      wMsg,
   DWORD_PTR dwInstance,
   DWORD_PTR dwParam1,
   DWORD_PTR dwParam2
);

CsWin32 code generation expects it to be:

[UnmanagedFunctionPointerAttribute(CallingConvention.Winapi)]
internal unsafe delegate void LPMIDICALLBACK(winmdroot.Media.Multimedia.HDRVR hdrvr, uint uMsg, nuint dwUser, nuint dw1, nuint dw2);

The expected parameter type for the first argument is HMIDIIN which I believe is typedef'd to HANDLE. Notice HDRVR is defined as a struct:

internal readonly partial struct HDRVR : IEquatable<HDRVR>
{
    internal readonly nint Value;
    internal HDRVR(nint value) => this.Value = value;
    public static implicit operator nint(HDRVR value) => value.Value;
    public static explicit operator HDRVR(nint value) => new HDRVR(value);
    public static bool operator ==(HDRVR left, HDRVR right) => left.Value == right.Value;
    public static bool operator !=(HDRVR left, HDRVR right) => !(left == right);

    public bool Equals(HDRVR other) => this.Value == other.Value;

    public override bool Equals(object obj) => obj is HDRVR other && this.Equals(other);

    public override int GetHashCode() => this.Value.GetHashCode();
}

A possible (untested) fix is to write a custom signature for the callback method, for example:

[UnmanagedFunctionPointerAttribute(CallingConvention.Winapi)]
internal unsafe delegate void My_LPMIDICALLBACK(IntPtr hdrvr, uint uMsg, nuint dwUser, nuint dw1, nuint dw2);

private static readonly My_LPMIDICALLBACK callback = OnMidiMessage;

Hope this helps!

[tristanlabelle]

Hi @tommcdon , we met at Microsoft :)! I tried your suggestion with no luck. I think a bad marshaling would be likely to break the callback no matter which thread it was on, and the code is able to go several frames deep without exploding due to corrupt register contents or something such in my real-life repro.

I created a new repro using the similar waveInOpen API, which should only require a microphone device. Could you give it a try?

WaveInBreakpointBugRepro.zip

var waveFormat = new WAVEFORMATEX
{
    cbSize = (ushort)Unsafe.SizeOf<WAVEFORMATEX>(),
    wFormatTag = (ushort)WAVE_FORMAT_PCM,
    nSamplesPerSec = 22050,
    nChannels = 2,
    wBitsPerSample = 16,
};

waveFormat.nBlockAlign = (ushort)(waveFormat.nChannels * waveFormat.wBitsPerSample / 8);
waveFormat.nAvgBytesPerSec = waveFormat.nBlockAlign * waveFormat.nSamplesPerSec;

HWAVEIN handle = default;
var result = waveInOpen(&handle, uDeviceID: WAVE_MAPPER,
    pwfx: in waveFormat,
    dwCallback: (nuint)(nint)Marshal.GetFunctionPointerForDelegate(callback),
    dwInstance: 0, MIDI_WAVE_OPEN_TYPE.CALLBACK_FUNCTION);
if (result != 0) throw new Exception("WAVE input device 0 could not be opened, is a microphone connected?");

var waveHdr = (WAVEHDR*)Marshal.AllocHGlobal(Unsafe.SizeOf<WAVEHDR>());
*waveHdr = new WAVEHDR(); // Zero out
waveHdr->dwBufferLength = waveFormat.nBlockAlign * 1024U;
waveHdr->lpData = (byte*)Marshal.AllocHGlobal((int)waveHdr->dwBufferLength);
result = waveInPrepareHeader(handle, waveHdr, (uint)Unsafe.SizeOf<WAVEHDR>());
if (result != 0) throw new Exception();

result = waveInAddBuffer(handle, waveHdr, (uint)Unsafe.SizeOf<WAVEHDR>());
if (result != 0) throw new Exception();

result = waveInStart(handle);
if (result != 0) throw new Exception();

NativeMethods.txt

waveInOpen
waveInPrepareHeader
waveInStart
waveInAddBuffer
LPWAVECALLBACK
WAVEHDR
WAVE_MAPPER
WAVE_FORMAT_PCM
MIDI_WAVE_OPEN_TYPE