Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

What's not working? #44

Closed
lihongze opened this issue Nov 2, 2020 · 10 comments
Closed

What's not working? #44

lihongze opened this issue Nov 2, 2020 · 10 comments
Assignees
Labels
help wanted Extra attention is needed

Comments

@lihongze
Copy link

lihongze commented Nov 2, 2020

Describe the bug

My project is composed of a user-side module, a management-side module, and a public module. I created a mail entity class in the public module, and then put the mail entity into the queue on the management side, and then consume it. My client and management The end uses the same redis, and then there is a problem today. The mail entity message queued by the management end was successfully consumed on the management end, but an error was reported on the web end. String could not be converted to a mail entity, and the deserialization failed. Makes me very confused

exception:
ERROR rqueueMessageListenerContainer-6 com.github.sonus21.rqueue.listener.RqueueExecutor - [email-queue_low] Message consumer failed
org.springframework.messaging.MessagingException: An exception occurred while invoking the handler method; nested exception is org.springframework.messaging.converter.MessageConversionException: Cannot convert from [java.lang.String] to [com.xxxx.queue.EmailEntity] for GenericMessage
at com.github.sonus21.rqueue.listener.RqueueMessageHandler.processHandlerMethodException(RqueueMessageHandler.java:290) ~[rqueue-2.0.0-RELEASE.jar!/:?]
at org.springframework.messaging.handler.invocation.AbstractMethodMessageHandler.handleMatch(AbstractMethodMessageHandler.java:581) ~[spring-messaging-5.2.6.RELEASE.jar!/:5.2.6.RELEASE]
at org.springframework.messaging.handler.invocation.AbstractMethodMessageHandler.handleMessageInternal(AbstractMethodMessageHandler.java:520) ~[spring-messaging-5.2.6.RELEASE.jar!/:5.2.6.RELEASE]
at org.springframework.messaging.handler.invocation.AbstractMethodMessageHandler.handleMessage(AbstractMethodMessageHandler.java:454) ~[spring-messaging-5.2.6.RELEASE.jar!/:5.2.6.RELEASE]
at com.github.sonus21.rqueue.listener.RqueueExecutor.start(RqueueExecutor.java:377) [rqueue-2.0.0-RELEASE.jar!/:?]
at com.github.sonus21.rqueue.listener.MessageContainerBase.run(MessageContainerBase.java:90) [rqueue-2.0.0-RELEASE.jar!/:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_212]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_212]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_212]

How to Reproduce

  • Steps to reproduce the behavior
    Start two services, share a redis, and monitor the same queue. When one service generates a message, the other service sometimes reports an error

  • A sample reproducable code if possible.
    send code:
    here i user priorityCode is low
    private void sendEmailQueue(EmailEntity emailEntity,Integer priorityCode){
    String priority ;
    try {
    priority = EmailPriorityEnum.getDescription(priorityCode);

          rqueueMessageSender.enqueueWithPriority("email-queue", priority,emailEntity);
      }catch(Exception e ){
          log.error("邮箱打入队列失败",e);
      }
    

    }
    consume code:
    @RqueueListener(value = "email-queue",numRetries="3",priority="critical=10,high=8,medium=4,low=1")
    public void onMessage(EmailEntity emailEntity) {
    log.info("邮件开始消费{},{},{}",new Date(),emailEntity.getTitle(),emailEntity.getReceiver());
    try{
    if(!EmptyUtils.isEmpty(emailEntity.getReceiver())){
    log.info("发送html邮件至:{}", emailEntity.getReceiver());
    sendHtmlEmail(emailEntity.getTitle(),emailEntity.getText(),emailEntity.getReceiver());
    }
    }catch (EmailException e) {
    log.error("邮件发送失败 {} {} ",emailEntity.getReceiver(),emailEntity.getTitle());
    log.error("邮件内容为--------"+emailEntity.getText());
    }catch(Exception e ){
    log.error("邮件消费失败",e);
    }
    }

Library Dependencies

  • Spring Boot: 2.3.0.RELEASE
  • rquequ: 2.0.0-RELEASE
  • redis.clients.jedis: 2.8.2
  • Any other Spring library dependencies
@lihongze
Copy link
Author

lihongze commented Nov 2, 2020

and the serialization method of redis configuration on my client and management side is the same

@sonus21
Copy link
Owner

sonus21 commented Nov 2, 2020

It seems there's some issue with the entries in the Redis LIST. By any chance you had enqueued string then you have converted that to EmailEntity class, if you have done so then we need to ignore such entries as it can't be deserialised. As error is being reported on Web side, can you check classpath for EmailEntity, EmailEntity should be available on Web and Management end and both should have package name and class names.

Even if class paths are same than please share some entries of __rq::queue::email-queue_low List? You can do LRANGE __rq::queue::email-queue_low

@lihongze
Copy link
Author

lihongze commented Nov 2, 2020

The classpath is the same,and exception is from com.github.sonus21.rqueue.converter.GenericMessageConverter.fromMessage method,here return null

@sonus21
Copy link
Owner

sonus21 commented Nov 2, 2020

Have you added any other message converter? This should not returns null unless data is invalid in Redis LIST. If same message is working in the producer than it should be working in consumer as well unless there's some difference.

@lihongze
Copy link
Author

lihongze commented Nov 2, 2020

What puzzles me is that the entities and methods of the queue are all in the public package. The user side and the management side use a set of tools, and the timing task that reported the error has not had similar problems before, but today the user side A large number of errors have been reported. I can see the content of the emailentity entity in the error message. There is no problem. It is indeed an entity object in json format.

@lihongze
Copy link
Author

lihongze commented Nov 2, 2020

The only difference is that the redis transaction is enabled in the redis config configuration of the management side. There has been a problem that the redis data read is empty due to the redis transaction before, but here is the log, the read redis list content is no problem

@sonus21
Copy link
Owner

sonus21 commented Nov 2, 2020

Can you try to unmarshal json using com.github.sonus21.rqueue.converter.GenericMessageConverter.fromMessage?

@sonus21
Copy link
Owner

sonus21 commented Nov 2, 2020

Redis transaction should not effect message conversion. Removing/adding new fields from the entity can also cause this problem. In such cases add @JsonIgnoreProperties(ignoreUnknown = true) on the entities class.

@sonus21 sonus21 added the help wanted Extra attention is needed label Nov 6, 2020
@sonus21
Copy link
Owner

sonus21 commented Nov 16, 2020

This does not seem to be a Rqueue bug, closing this one. Please reopen as you see.

@sonus21 sonus21 closed this as completed Nov 16, 2020
@blacktide082
Copy link

I had this same issue while trying to migrate from ActiveMQ to Rqueue and spent hours trying to troubleshoot it. For anyone else that has this issue, it was caused by the class being sent via the queue did not have a default constructor. Adding a no-args constructor fixed the issue for me.

The exception looks like the following:

org.springframework.messaging.converter.MessageConversionException: Cannot convert from [java.lang.String] to [org.example.Email] for GenericMessage [payload={"msg":"{\"to\":\"to@test.com\",\"from\":\"from@test.com\",\"body\":\"This is a test\"}","name":"org.example.Email"}, headers={execution=Execution(super=com.github.sonus21.rqueue.models.db.Execution@9f22b00e, startTime=1690813731931, endTime=0, error=null, exception=null, status=IN_PROGRESS), destination=email-queue, messageId=8584ae39-b92b-41c2-bb11-0e53d643f8af, message=RqueueMessage(id=8584ae39-b92b-41c2-bb11-0e53d643f8af, queueName=email-queue, message={"msg":"{\"to\":\"to@test.com\",\"from\":\"from@test.com\",\"body\":\"This is a test\"}","name":"org.example.Email"}, retryCount=null, queuedTime=491407246113125, processAt=1690813731587, reEnqueuedAt=null, failureCount=0, sourceQueueFailureCount=0, sourceQueueName=null, period=0), job=com.github.sonus21.rqueue.listener.JobImpl@25885224, lookupDestination=email-queue}]
	at org.springframework.messaging.handler.annotation.support.PayloadMethodArgumentResolver.resolveArgument(PayloadMethodArgumentResolver.java:145) ~[spring-messaging-5.3.29.jar:5.3.29]
	at org.springframework.messaging.handler.invocation.HandlerMethodArgumentResolverComposite.resolveArgument(HandlerMethodArgumentResolverComposite.java:118) ~[spring-messaging-5.3.29.jar:5.3.29]
	at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.getMethodArgumentValues(InvocableHandlerMethod.java:147) ~[spring-messaging-5.3.29.jar:5.3.29]
	at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:115) ~[spring-messaging-5.3.29.jar:5.3.29]
	at org.springframework.messaging.handler.invocation.AbstractMethodMessageHandler.handleMatch(AbstractMethodMessageHandler.java:569) ~[spring-messaging-5.3.29.jar:5.3.29]
	at com.github.sonus21.rqueue.listener.RqueueMessageHandler.executeMatches(RqueueMessageHandler.java:255) ~[rqueue-core-2.13.1-RELEASE.jar:na]
	at com.github.sonus21.rqueue.listener.RqueueMessageHandler.handleMessageInternal(RqueueMessageHandler.java:227) ~[rqueue-core-2.13.1-RELEASE.jar:na]
	at org.springframework.messaging.handler.invocation.AbstractMethodMessageHandler.handleMessage(AbstractMethodMessageHandler.java:458) ~[spring-messaging-5.3.29.jar:5.3.29]
	at com.github.sonus21.rqueue.core.middleware.HandlerMiddleware.handle(HandlerMiddleware.java:50) ~[rqueue-core-2.13.1-RELEASE.jar:na]
	at com.github.sonus21.rqueue.listener.RqueueExecutor.callMiddlewares(RqueueExecutor.java:239) ~[rqueue-core-2.13.1-RELEASE.jar:na]
	at com.github.sonus21.rqueue.listener.RqueueExecutor.processMessage(RqueueExecutor.java:254) ~[rqueue-core-2.13.1-RELEASE.jar:na]
	at com.github.sonus21.rqueue.listener.RqueueExecutor.execute(RqueueExecutor.java:265) ~[rqueue-core-2.13.1-RELEASE.jar:na]
	at com.github.sonus21.rqueue.listener.RqueueExecutor.handleMessage(RqueueExecutor.java:287) ~[rqueue-core-2.13.1-RELEASE.jar:na]
	at com.github.sonus21.rqueue.listener.RqueueExecutor.handle(RqueueExecutor.java:360) ~[rqueue-core-2.13.1-RELEASE.jar:na]
	at com.github.sonus21.rqueue.listener.RqueueExecutor.start(RqueueExecutor.java:377) ~[rqueue-core-2.13.1-RELEASE.jar:na]
	at com.github.sonus21.rqueue.utils.RetryableRunnable.run(RetryableRunnable.java:49) ~[rqueue-core-2.13.1-RELEASE.jar:na]
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[na:na]
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[na:na]
	at java.base/java.lang.Thread.run(Thread.java:829) ~[na:na]

Exception in thread "rqueueMessageListenerContainer-2" java.lang.StackOverflowError
	at java.base/java.lang.Throwable.toString(Throwable.java:485)
	at org.springframework.messaging.MessagingException.toString(MessagingException.java:74)
	at java.base/java.lang.String.valueOf(String.java:2951)
	at java.base/java.lang.StringBuilder.append(StringBuilder.java:172)
	at org.springframework.core.NestedExceptionUtils.buildMessage(NestedExceptionUtils.java:51)
	at org.springframework.core.NestedRuntimeException.getMessage(NestedRuntimeException.java:77)
	at java.base/java.lang.Throwable.getLocalizedMessage(Throwable.java:396)
	at java.base/java.lang.Throwable.toString(Throwable.java:485)
	at org.springframework.messaging.MessagingException.toString(MessagingException.java:74)
	at java.base/java.lang.String.valueOf(String.java:2951)
	at java.base/java.lang.StringBuilder.append(StringBuilder.java:172)
	at com.github.sonus21.rqueue.models.db.Execution.toString(Execution.java:34)

<omitted over additional 1000 lines of StackOverflowException>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

3 participants