-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
AbstractConfigurableMongoDbMessageStore's getNextId might overflow to long leading to "java.lang.Long cannot be cast to java.lang.Integer" #3336
Comments
Is this a real problem, or hypothetical? |
Does it work if you change it to We might not have it as back-ported to all supported versions because existing documents in DB might not be updated to a new type, but we definitely can consider it as a fix for the current version in progress. Thanks |
But my question is:
Is this even practical? How would such a message store perform? |
Or is this just like a sequence - it doesn't mean there would be that number of messages concurrently (in which case, yes, it should be changed - but I would still expect it to take a loooong time to hit that limit). |
yes, this is a real problem. we use the aggregation store to "aggregate" millions of messages sent by our customers and then release when they are complete. Most messages are single-messages but because we process 30+ (and growing) million every day we started seeing these errors starting on Friday last week For now, we are resetting that field back to 0 in MongoDB. |
yes it is a sequence that gets incremented with each message that ts "aggreagated". For us, it took several months to reach the problem and we have a workaround (reset in MongoDB to 0) but we'd like a long term solution. |
A clarification: The amount of messages that get aggregated are never beyond of the hundreds, and this issue is not related to the amount of messages in a group. The issue here is that each time a message is "aggregated", there is a mongodb-based sequence that gets incremented. You'd think that 2^31-1 would be a lot of messages to aggregate (and they are indeed a lot) but we process that much in weeks, so the use case is applicable and valid at least for us. |
So, yeah... Let's go ahead and fix the issue changing from
|
Fixes spring-projects#3336 Turns out there are some scenarios where too many messages are transferred through the message store, so `int` for sequence is not enough as a type * Change sequence to `long` to widen a sequence lifespan
* GH-3336: Change MongoDb Store sequence to long Fixes #3336 Turns out there are some scenarios where too many messages are transferred through the message store, so `int` for sequence is not enough as a type * Change sequence to `long` to widen a sequence lifespan * * Change MongoDb store to deal with `Number.longValue()` instead of casting which doesn't work from `Integer` to `Long`. This way we can keep an old sequence document with an `int` type for value * Documents with new `long` type for their sequence field are OK. The `NumberToNumberConverter` has an effect converting `int` to `long` properly.
In what version(s) of Spring Integration are you seeing this issue?
It happens from version 4.3.12.RELEASE up to current. But it is possible that it happens with prior/future versions as well.
Describe the bug
Because getNextId performs an "increment" (by 1) on the "sequence" field of the special "messagesSequence" document, after Integer.MAX_VALUE calls to getNextId, next value will overflow from Integer.MAX_VALUE to a Long and when trying to retrieve it, will get fail with:
Caused by: java.lang.ClassCastException: java.lang.Long cannot be cast to java.lang.Integer
To Reproduce
Add Integer.MAX_VALUE + 1 of messages in aggregation store.
Expected behavior
Well, getNextId returns an int which populates the MessageDocument's sequence which is an int, so most likely, the MessageDocument's sequence's type needs to be changed from int to long and then getNextId, needs to return a long.
The text was updated successfully, but these errors were encountered: