New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Real exception is hidden when using Unobtrusive Conventions. #4000

Closed
jpv001 opened this Issue Aug 2, 2016 · 13 comments

Comments

Projects
None yet
5 participants
@jpv001
Copy link

jpv001 commented Aug 2, 2016

When sending a message to an endpoint that doesn't support that message type, if you're using unobtrusive conventions then the following exception hides the real error:

NServiceBus.ExceptionInfo.Message   Could not find metadata for 'Newtonsoft.Json.Linq.JObject'.
Ensure the following:
1. 'Newtonsoft.Json.Linq.JObject' is included in initial scanning. 
2. 'Newtonsoft.Json.Linq.JObject' implements either 'IMessage', 'IEvent' or 'ICommand' or alternatively, if you don't want to implement an interface, you can use 'Unobtrusive Mode'.

If you add a reference ICommand then the exception changes to:

System.InvalidOperationException: No handlers could be found for message type: Gateway.Twilio.Commands.AddCreditorTwilioIntegration

We see similar behavior when dealing with Timeouts, the timeout exception is masked by the unobtrusive conventions exception message.

This should be ideally be consistent regardless of the use of unobtrusive conventions as it makes the debugging process a lot harder. At the moment the decision to use unobtrusive conventions means that you mainly get that top level exception for any routing or timeout problems.

@timbussmann

This comment has been minimized.

Copy link
Member

timbussmann commented Aug 2, 2016

@jpv001 I assume you're using NSB v5, is that correct? Are the conventions of the receiving endpoint set up to identify the "failing" message correctly as a message or are you using different conventions on the receiver?

@jpv001

This comment has been minimized.

Copy link
Author

jpv001 commented Aug 2, 2016

Apologies, I'm using the newest beta 6.0.0-beta0004.

The following conventions are applied to all endpoints as part of endpoint configuration (so definitely run on both):

var conventionsBuilder = endpointConfiguration.Conventions();
conventionsBuilder.DefiningCommandsAs(t => t.Namespace != null && t.Namespace.EndsWith("Commands"));
conventionsBuilder.DefiningEventsAs(t => t.Namespace != null && t.Namespace.EndsWith("Events"));
conventionsBuilder.DefiningMessagesAs(t => t.Namespace != null && t.Namespace.EndsWith("Messages"));
conventionsBuilder.DefiningExpressMessagesAs(t => t.Name.EndsWith("Express"));

If the handler does exist it all works as expected, but if there is a missing handler or a problem with a timeout that top error is thrown which isn't the actual exception but rather a far broader one.

@timbussmann

This comment has been minimized.

Copy link
Member

timbussmann commented Aug 2, 2016

@jpv001 I wasn't able to reproduce the behavior you described locally as I got the expected No handlers could be found for message type exception on the receiver when using conventions.

My sample may misses some of your configurations or this may be fixed in more current versions of v6 which are to be released soon. If you're able to create a simple sample which reproduces the behavior you're experiencing I can take that and give it a closer look + run it against latest unstables.

@jpv001

This comment has been minimized.

Copy link
Author

jpv001 commented Aug 2, 2016

I won't be in the office for a few weeks, but I'll put a spike together when I get back. We're seeing this on all of our endpoints, they all work perfectly provided there is no mapping issue, as soon as there is we get this error. Simply add ICommand and a reference to the nsb binary in the message project and it disappears.

@timbussmann

This comment has been minimized.

Copy link
Member

timbussmann commented Aug 2, 2016

I just downgraded my sample to beta4 and I still see the expected exception. Can you share the code which sends the command and the exception stacktrace with us?

@timbussmann

This comment has been minimized.

Copy link
Member

timbussmann commented Aug 3, 2016

we were able to reproduce the issue whenwe do not include the message assembly. Marking this as bug.

@danielmarbach

This comment has been minimized.

Copy link
Member

danielmarbach commented Aug 4, 2016

@jpv001

I dug deeper into the root cause. The problem occurs only if the following points are fulfilled:

  • Conventions do not reference any base infrastructure from your message assemblies (let's call them stringified conventions)
  • Nothing on the receiving endpoint references anything from the message assembly.

Ultimately this means:

  • You have to copy paste the message conventions to each and every endpoint
  • Nothing references the message assembly, therefore the message assembly will never be loaded into the AppDomain. This behavior is controlled by the runtime and out of our control.

Under these circumstances, the exception you saw will hide the fact that a message handler was not present. The log actually contains two helpful warnings in that case

2016-08-04 13:52:00.641 WARN NServiceBus.Unicast.Messages.MessageMetadataRegistry Message header 'Messages.Crazy.IMyCommand' was mapped to type 'Messages.Crazy.IMyCommand' but that type was not found in the message registry, ensure the same message registration conventions are used in all endpoints, especially if using unobtrusive mode.

2016-08-04 13:52:00.755 WARN NServiceBus.RecoverabilityExecutor Second Level Retry will reschedule message 'c9baa450-c2e9-49cf-9f9e-a65800e46e22' after a delay of 00:00:20 because of an exception:
System.InvalidOperationException: No handlers could be found for message type: Messages.SomeCommand

I don't think it makes sense for us to optimize for such a rare case since the log file contains enough information to resolve it. You could easily mitigate it by

  • Define an extension method on EndpointConfiguration which lives in messages assembly and defines your stringified conventions and then write endpointConfiguration.RegisterMyCustomConventions(). Which would enforce that the messages assembly is loaded or
  • As soon as you have one handler which handles any message of that assembly the assembly is loaded an NSB can retrieve the type.

NSB v5 was extremely greedy in trying to load assemblies into the AppDomain. This caused all sorts of severe other problems. With v6 we no longer scanned assemblies that do not reference NSB (like said before). Which is in your special scenario a bit more inconvenient but in general still the better approach because it reduces grief with assembly scanning and increases startup time.

@danielmarbach

This comment has been minimized.

Copy link
Member

danielmarbach commented Aug 11, 2016

@jpv001 heads up I might have found a solution here which makes my previous statement mute. I couldn't let it go and find a solution to make your life easier ;)

@danielmarbach

This comment has been minimized.

Copy link
Member

danielmarbach commented Aug 11, 2016

@jpv001 Here is the PR #4029

@andreasohlund

This comment has been minimized.

Copy link
Member

andreasohlund commented Aug 11, 2016

PR is merged, will go out in beta7/RC1. Can we close this one?

@jpv001

This comment has been minimized.

Copy link
Author

jpv001 commented Aug 11, 2016

Thanks for all the work, happy to close and retest with the new beta.

@jpv001 jpv001 closed this Aug 11, 2016

@andreasohlund

This comment has been minimized.

Copy link
Member

andreasohlund commented Aug 11, 2016

Note: Beta6 which is going out in the next few days(already on myget) won't contain this.

What transport/persistence are you using? (If you want I could help you get going with the bleeding edge version)

@jpv001

This comment has been minimized.

Copy link
Author

jpv001 commented Aug 11, 2016

That's ok, it isn't a deal breaker for us as the work around is just being more explicit with our ICommand / IEvent markers and referencing NSB in our message assemblies, perfectly happy to wait for a future build.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment