Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When ingesting timeouts, TimeoutPersisterReceiver queries in tight loop #4015

Closed
DavidBoike opened this issue Aug 4, 2016 · 8 comments
Closed
Assignees
Labels
Milestone

Comments

@DavidBoike
Copy link
Member

DavidBoike commented Aug 4, 2016

In the TimeoutPersisterReceiver, when ingesting a large amount of timeouts into persistence, the value of timeoutPushed is held at true, which results in no code path where nextRetrieval is modified. This results in a tight loop where the timeout persistence is queries over and over, even if the timeouts being added are far into the future.

Who's affected

Users who register large amounts of saga timeouts or deferred messages.

Symptoms

When logging at INFO level or above, nothing would be evident in an endpoint's logs, because the TimeoutPersisterReceiver only logs at DEBUG level. At the DEBUG mode, rapid requerying of timeouts would be observed. Alternatively, you may notice undue load on the timeout persistence (SQL Server, RavenDB, etc.) when ingesting timeouts, although it would be easy to misattribute this to the actual writing of timeouts.

@timbussmann
Copy link
Contributor

is v6 affected by this too?

@DavidBoike
Copy link
Member Author

I don't know yet, will be checking that out for sure.

@DavidBoike
Copy link
Member Author

V6 is not affected by this. V6 does not have the timeoutPushed concept or the tricky if/then logic based on it. It only has NextRetrieval, which is simply backed up if a timeout comes in that's before the current value. Much better IMO, and my same test harness proves it out.

@danielmarbach
Copy link
Contributor

Nice @DavidBoike

@DavidBoike
Copy link
Member Author

Updated the description with Who's affected and Symptoms information for possible release notes.

@andreasohlund
Copy link
Member

Reading the updated description and @bording comment on the PR I'm 👍 to go ahead and release this

@DavidBoike
Copy link
Member Author

Will release on my Monday morning.

@DavidBoike
Copy link
Member Author

DavidBoike commented Aug 17, 2016

This is now released in NServiceBus 5.2.15.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants