Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] SimpleConsumer message is lost and not received. #8133

Closed
3 tasks done
JanYork opened this issue May 14, 2024 · 7 comments
Closed
3 tasks done

[Bug] SimpleConsumer message is lost and not received. #8133

JanYork opened this issue May 14, 2024 · 7 comments

Comments

@JanYork
Copy link

JanYork commented May 14, 2024

Before Creating the Bug Report

  • I found a bug, not just asking a question, which should be created in GitHub Discussions.

  • I have searched the GitHub Issues and GitHub Discussions of this repository and believe that this is not a duplicate.

  • I have confirmed that this bug belongs to the current repository, not other repositories of RocketMQ.

Runtime platform environment

Macos

RocketMQ version

5.1.x

JDK Version

No response

Describe the Bug

Two questions:

  1. For the messages I produce, the consumer occasionally loses the first message.
  2. nodejs client threw an error: Cannot invoke "apache.rocketmq.v2.Settings.getSubscription()" because "settings" is null. NullPointerException. org.apache.rocketmq.proxy.grpc.v2.consumer.ReceiveMessageActivity .receiveMessage(ReceiveMessageActivity.java:63)

I used the SimpleConsumer of the nodejs client to receive messages and found that the messages I produced were not received.

This is my producer, it is sequential, and the topic type is also FIFO:

import { Producer } from 'rocketmq-client-nodejs';

const producer = new Producer({
    endpoints: '192.168.1.162:8081',
});

(async () => {
    await producer.startup();

    for (let i = 1; i <= 5; i++) {
        await producer.send({
            messageGroup: 'checkout-group',
            topic: 'checkout-topic-fifo',
            tag: 'checkout',
            keys: ['checkout-key'],
            body: Buffer.from("Hello World:"+i.toString())
        }).then(() => {
            console.log('message %d sent', i);
            console.log('checkout:send message success!');
        }).catch((e) => {
            console.error(e);
        });
    }

    process.exit(0);
})()

Here is my consumer, which simulates a situation where I am constantly listening for messages and receiving them:

import { SimpleConsumer } from 'rocketmq-client-nodejs';

const simpleConsumer = new SimpleConsumer({
    consumerGroup: 'checkout-group',
    endpoints: '127.0.0.1:8081',
    subscriptions: new Map().set('checkout-topic-fifo', '*'),
});

(async () => {
    await simpleConsumer.startup();

    async function consumeMessages(){
        try {
            const messages = await simpleConsumer.receive(1);

            if (messages.length > 0) {
                console.log('got %d messages', messages.length);

                for (const message of messages) {
                    console.log('body=%o', message.body.toString());
                    await simpleConsumer.ack(message);
                }
            } else {
                console.log('No messages received, waiting...');
            }
        } catch (error) {
            console.error('An error occurred:', error);
        } finally {
            console.log('waiting for messages...');
            await consumeMessages();
        }
    }

    await consumeMessages();
})()

I will only get one message at a time, that is to say, it should be received sequentially. When I sent 5 messages sequentially, my consumer did not receive the first piece of data, but 2-5 were successfully received. .
I need to wait for a long time to receive the first one (or the retry mechanism), and this is a random phenomenon, and the first data sent is lost inexplicably.

The second problem is that nodejs client will have an error for no reason in my above message listening code:

An error occurred: InternalErrorException: [request-id=undefined, response-code=50001] Cannot invoke "apache.rocketmq.v2.Settings.getSubscription()" because "settings" is null. NullPointerException. org.apache.rocketmq.proxy.grpc.v2.consumer.ReceiveMessageActivity.receiveMessage(ReceiveMessageActivity.java:63)
    at StatusChecker.check (/Users/muyouzhi/Code/demo/html-demo/node_modules/.pnpm/rocketmq-client-nodejs@1.0.0/node_modules/rocketmq-client-nodejs/dist/exception/StatusChecker.js:83:23)
    at SimpleConsumer.receiveMessage (/Users/muyouzhi/Code/demo/html-demo/node_modules/.pnpm/rocketmq-client-nodejs@1.0.0/node_modules/rocketmq-client-nodejs/dist/consumer/Consumer.js:64:35)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async SimpleConsumer.receive (/Users/muyouzhi/Code/demo/html-demo/node_modules/.pnpm/rocketmq-client-nodejs@1.0.0/node_modules/rocketmq-client-nodejs/dist/consumer/SimpleConsumer.js:95:16)
    at async consumeMessages (/Users/muyouzhi/Code/demo/html-demo/dist/test/c.js:13:30)
    at async consumeMessages (/Users/muyouzhi/Code/demo/html-demo/dist/test/c.js:30:13)
    at async consumeMessages (/Users/muyouzhi/Code/demo/html-demo/dist/test/c.js:30:13)
    at async consumeMessages (/Users/muyouzhi/Code/demo/html-demo/dist/test/c.js:30:13)
    at async consumeMessages (/Users/muyouzhi/Code/demo/html-demo/dist/test/c.js:30:13)
    at async consumeMessages (/Users/muyouzhi/Code/demo/html-demo/dist/test/c.js:30:13) {
  code: 50001
}

Steps to Reproduce

It's very simple. Use nodejs client to run a consumer to continuously listen for messages, and then run a producer to send multiple messages. The Topic type is FIFO.
If the problem does not occur, try closing and running it again. Sometimes it takes many attempts to reproduce the problem.

What Did You Expect to See?

I should receive the first piece of data.

What Did You See Instead?

The first piece of data is often lost.

Additional Context

No response

@CzyerChen
Copy link

RocketMQ 5.1.1
Send two messages, consume multiple messages at a time, and receive only one message rather than two occasionally.
I‘m not sure if this is the case.

@JanYork
Copy link
Author

JanYork commented May 15, 2024

RocketMQ 5.1.1
Send two messages, consume multiple messages at a time, and receive only one message rather than two occasionally.
I‘m not sure if this is the case.

I should say: My producer sends messages in order with content 1, 2, 3, 4, 5. However, after the consumer first starts, it only receives 2, 3, 4, 5, and then has to wait a long time to receive the missing message, which is the number 1.

I use this method to verify the consumption order of RocketMQ. A simple consumer's message receiving operation should be sequential and atomic, so I should receive number 1 first instead of 2. This is problematic.

The type of my topic is FIFO, and all messages are sent to the same group. My message sending operation is synchronous, which means my message producer ensures the order.

@zhaohai666
Copy link
Contributor

zhaohai666 commented May 16, 2024

这个客户端是使用的那个版本?

@JanYork
Copy link
Author

JanYork commented May 16, 2024

这个客户端是使用的那个版本?

nodejs使用的"rocketmq-client-nodejs": "^1.0.0"版本.

javago使用的最新版本cleint

@JanYork
Copy link
Author

JanYork commented May 16, 2024

这个客户端是使用的那个版本?

我想..这应该不是客户端的问题

@zhaohai666
Copy link
Contributor

nodejs使用的"rocketmq-client-nodejs": "^1.0.0"版本消费者暂时不能使用,有些问题需要修改发布新版本。

@JanYork
Copy link
Author

JanYork commented May 16, 2024

nodejs使用的"rocketmq-client-nodejs": "^1.0.0"版本消费者暂时不能使用,有些问题需要修改发布新版本。

可是我这个问题好像和客户端无关,我用java和go的client也一样复现,不只是在nodejs上

@JanYork JanYork closed this as completed May 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants