bug of retry consume message

[lixinwen]

The issue tracker is ONLY used for bug report(feature request need to follow RIP process). Keep in mind, please check whether there is an existing same report before your raise a new one.

Alternately (especially if your communication is not a bug report), you can send mail to our mailing lists. We welcome any friendly suggestions, bug fixes, collaboration and other improvements.

Please ensure that your bug report is clear and that it is complete. Otherwise, we may be unable to understand it or to reproduce it, either of which would prevent us from fixing the bug. We strongly recommend the report(bug report or feature request) could include some hints as the following:

BUG REPORT

1. Please describe the issue you observed:
   On the same machine, start two consumers, the same group name, the same instanceName, and then MessageListenerConcurrently return RECONSUME_LATER, will cause a large number of messages to be saved in the %RETRY% topic.

The reason is that the queued message will be consumed by two consumers. If it fails, the sendMessageBack will grow in doubles, and a message will be saved 2^16 times in the %RETRY% queue.

Although the production environment will not do this, I think that rocketmq should avoid this problem.

• What did you do (The steps to reproduce)?

1. On ConsumerManager registerConsumer, disconnect connection when found duplicated clientId.
2. clientId of client use 'clientIP@instanceName+pid. Keep pid even if the user has set instanceName.

• What did you expect to see?

• What did you see instead?

2. Please tell us about your environment:

3. Other information (e.g. detailed explanation, logs, related issues, suggestions how to fix, etc):

FEATURE REQUEST

1. Please describe the feature you are requesting.

2. Provide any additional detail on your proposed use case for this feature.

3. Indicate the importance of this issue to you (blocker, must-have, should-have, nice-to-have). Are you currently using any workarounds to address this issue?

4. If there are some sub-tasks using -[] for each subtask and create a corresponding issue to map to the sub task:

• sub-task1-issue-number: sub-task1 description here,
• sub-task2-issue-number: sub-task2 description here,
• ...

[duhenglucky]

@lixinwen the same group name, the same instanceName in the same machine will cause the consumer load balancing to not work properly, which will lead to a large amount of repeated consumption, so there will be so many messages sent back to the retry topic.
Since the server is difficult to perform repeated detections at a relatively low cost, so it is recommended to follow the best practices. If you have good suggestions, please leave a message or submit a PR.
I will turn off this issue first, but if you have any other questions about this issue, please feel free to reopen it.

[lixinwen]

I still recommend that rocketmq itself avoid this problem.
In our production environment, there was an accident.
In the docker environment, two tomcat processes are started without their knowledge.

Can be modified as follows：
ConsumerGroupInfo#updateChannel
synchronized (registerLock) {
......
ClientChannelInfo infoOld = this.channelInfoTable.get(infoNew.getChannel());
if (null == infoOld) {
for (ClientChannelInfo channelInfo : channelInfoTable.values()) {
if (clientId.equals(channelInfo.getClientId())) {
if (channel != channelInfo.getChannel()) {
log.error("channel[{}] found duplicated clientId[{}], exists consumer: {}", channel, clientId,
channelInfo);
// Catch the exception in ClientManageProcessor#headBeat() and close the channel, or close it directly here.
throw new ClientRegisterException("Duplicated clientId " + clientId);
}
}
}
}
......
}