Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RIP-28]Support light message queue (LMQ) multi-consumeQueue atomic dispatch #3666

Closed
tianliuliu opened this issue Dec 21, 2021 · 8 comments
Closed

Comments

@tianliuliu
Copy link
Contributor

Some messaging scenarios require Light message queue to support a large number of topics, such as MQTT, AMQP protocol, its multi-level topic sending and subscription can be set at will, and the number of multi-level topics is very large, the original RocketMQ topic It is resource-heavy, and it is difficult to support millions or more of lightweight multi-level topics.

Let's define Light message queue as (LMQ)

Light message queue (LMQ) needs to implement multi-dimensional dispatch of a message based on ConsumeQueue, and a message can support multiple lightweight queue consumption。

Light message queue (LMQ) needs to implement multi-level topic management, so topic metadata also needs adaptation management, such as topic verification, data cleaning and so on.

Light message queue (LMQ) needs to implement lightweight queue consumption offset management

@tianliuliu tianliuliu changed the title Support light message queue (LMQ) multi-consumeQueue atomic dispatch, at the same time support LMQ element & offset management Support light message queue (LMQ) multi-consumeQueue atomic dispatch, at the same time support LMQ metaData & offset management Dec 21, 2021
@RongtongJin
Copy link
Contributor

It seems to be a considerable change. I suggest you open a new RIP to introduce your design. You can refer to wiki-rip.

@duhenglucky duhenglucky modified the milestones: 5.0.0, 4.9.3 Dec 24, 2021
@vongosling
Copy link
Member

I could not figure out what you want to do from your description. You could arrange your sentences.

@tianliuliu
Copy link
Contributor Author

I could not figure out what you want to do from your description. You could arrange your sentences.

Some messaging scenarios require light message queue, Such as MQTT multi-level topic or AMQP lightweight queue can be set at will by users when sending and subscription message. Therefore, the implementation of LMQ's queue model is required

@Git-Yang
Copy link
Member

Looking forward to it very much. We have a big demand scenario here.

tianliuliu added a commit to tianliuliu/rocketmq that referenced this issue Jan 4, 2022
tianliuliu added a commit to tianliuliu/rocketmq that referenced this issue Jan 4, 2022
tianliuliu added a commit to tianliuliu/rocketmq that referenced this issue Jan 6, 2022
tianliuliu added a commit to tianliuliu/rocketmq that referenced this issue Jan 6, 2022
tianliuliu added a commit to tianliuliu/rocketmq that referenced this issue Jan 6, 2022
tianliuliu added a commit to tianliuliu/rocketmq that referenced this issue Jan 6, 2022
tianliuliu added a commit to tianliuliu/rocketmq that referenced this issue Jan 7, 2022
tianliuliu added a commit to tianliuliu/rocketmq that referenced this issue Jan 7, 2022
tianliuliu added a commit to tianliuliu/rocketmq that referenced this issue Jan 11, 2022
tianliuliu added a commit to tianliuliu/rocketmq that referenced this issue Jan 12, 2022
tianliuliu added a commit to tianliuliu/rocketmq that referenced this issue Jan 12, 2022
tianliuliu added a commit to tianliuliu/rocketmq that referenced this issue Jan 12, 2022
tianliuliu added a commit to tianliuliu/rocketmq that referenced this issue Jan 13, 2022
@RongtongJin
Copy link
Contributor

Merged

@ShannonDing
Copy link
Member

link #3694 #3753

@duhenglucky duhenglucky reopened this Jan 26, 2022
@duhenglucky duhenglucky changed the title Support light message queue (LMQ) multi-consumeQueue atomic dispatch, at the same time support LMQ metaData & offset management [RIP-28]Support light message queue (LMQ) multi-consumeQueue atomic dispatch Jan 26, 2022
yuz10 pushed a commit that referenced this issue Apr 8, 2022
Co-authored-by: hening <ninghe.hn@alibaba-inc.com>
GenerousMan pushed a commit to GenerousMan/rocketmq that referenced this issue Aug 12, 2022
Co-authored-by: hening <ninghe.hn@alibaba-inc.com>
@github-actions
Copy link

This issue is stale because it has been open for 365 days with no activity. It will be closed in 3 days if no further activity occurs.

@github-actions github-actions bot added the stale label Jan 27, 2023
@github-actions
Copy link

This issue was closed because it has been inactive for 3 days since being marked as stale.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

6 participants