Timeout messages can incorrectly end up in error queue after Bus restart

[chrisbednarski]

When the bus is restarted using Bus.Start(), Bus.Shutdown() then Bus.Start() timeout messages can throw an exception and end up in the error queue.

If a saga message handler requests any timeouts, the message being handled is rolled back and always ends up in error queue.

TimeoutPushed.BeginInvoke throws System.ArgumentException saying The delegate must have only one target. inside DefaultTimeoutManager.PushTimeout()

This is because TimeoutPersisterReceiver.Stop does not unsubscribe from TimeoutManager.TimeoutPushed event

Stack trace below:

NServiceBus.Core.dll!NServiceBus.Timeout.Core.DefaultTimeoutManager.PushTimeout(NServiceBus.Timeout.Core.TimeoutData timeout) Line 42 + 0x8a bytes  C#  
NServiceBus.Core.dll!NServiceBus.Timeout.Hosting.Windows.TimeoutMessageProcessor.HandleInternal(NServiceBus.TransportMessage message) Line 121  C#  
NServiceBus.Core.dll!NServiceBus.Timeout.Hosting.Windows.TimeoutMessageProcessor.Handle(NServiceBus.TransportMessage message) Line 52   C#
NServiceBus.Core.dll!NServiceBus.Satellites.SatelliteLauncher.HandleMessageReceived(object sender, NServiceBus.Unicast.Transport.TransportMessageReceivedEventArgs e, NServiceBus.Satellites.ISatellite satellite) Line 92 + 0x20 bytes C#
NServiceBus.Core.dll!NServiceBus.Satellites.SatelliteLauncher.StartSatellite.AnonymousMethod__7(object o, NServiceBus.Unicast.Transport.TransportMessageReceivedEventArgs e) Line 107 + 0x37 bytes  C#  
NServiceBus.Core.dll!NServiceBus.Unicast.Transport.TransportReceiver.OnTransportMessageReceived(NServiceBus.TransportMessage msg) Line 453 + 0x33 bytes C#  
NServiceBus.Core.dll!NServiceBus.Unicast.Transport.TransportReceiver.ProcessMessage(NServiceBus.TransportMessage message) Line 351 + 0xc bytes  C#  
NServiceBus.Core.dll!NServiceBus.Unicast.Transport.TransportReceiver.TryProcess(NServiceBus.TransportMessage message) Line 260  C#  
NServiceBus.Core.dll!NServiceBus.Transports.Msmq.MsmqDequeueStrategy.ProcessMessage(NServiceBus.TransportMessage message) Line 288 + 0x11 bytes C#  
NServiceBus.Core.dll!NServiceBus.Transports.Msmq.MsmqDequeueStrategy.Action() Line 239 + 0xc bytes  C#
 mscorlib.dll!System.Threading.Tasks.Task.InnerInvoke() + 0x49 bytes    
 mscorlib.dll!System.Threading.Tasks.Task.Execute() + 0x2e bytes    
// lines removed for brevity
 System.Threading.ContextCallback callback, object state) + 0x41 bytes  
 mscorlib.dll!System.Threading.ThreadHelper.ThreadStart(object obj) + 0x4e bytes    
    [Native to Managed Transition]  
 kernel32.dll!@BaseThreadInitThunk@12()  + 0x12 bytes   
 ntdll.dll!___RtlUserThreadStart@8()  + 0x27 bytes  
 ntdll.dll!__RtlUserThreadStart@8()  + 0x1b bytes   

[chrisbednarski]

related to #1885

[chrisbednarski]

Workaround: add code below after calling Bus.Shutdown()

var timeoutManager = Configure.Instance.Builder.Build<IManageTimeouts>();
var eventInfo = timeoutManager.GetType().GetEvent("TimeoutPushed");

var receiver = Configure.Instance.Builder.BuildAll<ISatellite>()
            .OfType<TimeoutDispatcherProcessor>().Single().TimeoutPersisterReceiver;
var handler = receiver.GetType().GetMethod("TimeoutsManagerOnTimeoutPushed",
                                  BindingFlags.NonPublic | BindingFlags.Instance);
var d = Delegate.CreateDelegate(eventInfo.EventHandlerType, receiver, handler);

object[] methodArgs = { d };
var removeHandler = eventInfo.GetRemoveMethod();
removeHandler.Invoke(timeoutManager, methodArgs);