Skip to content

fix(message): enumerate read queues when listing queue offsets - #3309

Open
zjncs wants to merge 1 commit into
apache:rocketmq-studiofrom
zjncs:fix/queue-offsets-read-queues
Open

fix(message): enumerate read queues when listing queue offsets#3309
zjncs wants to merge 1 commit into
apache:rocketmq-studiofrom
zjncs:fix/queue-offsets-read-queues

Conversation

@zjncs

@zjncs zjncs commented Sep 5, 2026

Copy link
Copy Markdown

Problem

RocketMQMessageProvider.getQueueOffsets lists the Queue Browser's queues with queueData.getWriteQueueNums(), but every other message-browse path is read-queue based:

  • the broker's PullMessageProcessor rejects any pull with queueId >= topicConfig.getReadQueueNums() (SYSTEM_ERROR "queueId is illegal");
  • fetchSubscribeMessageQueues — used by this provider's own queryByTopic and by the classic console's queryMessageByTopic — enumerates [0, readQueueNums).

So whenever a topic has readQueueNums != writeQueueNums (the standard expansion/shrink procedure sets the two counts independently, e.g. updateTopic -r 16 -w 8), the Queue Browser is wrong:

  • read > write (shrink draining): queues in [write, read) are still readable and still hold messages, but they never appear in the browser — the user cannot browse them at all, and the queue list disagrees with what queryByTopic scans.
  • write > read (queues not yet readable): queues in [read, write) are listed, but each one is a dead end — pulling from it fails with "queueId is illegal".

The writeQueueNums iteration was introduced incidentally by the admin-client pooling refactor b5a7d6c (#2544), not as a deliberate queue-selection choice.

Fix

Enumerate [0, queueData.getReadQueueNums()) in getQueueOffsets, matching the broker's pull validation and the read-queue enumeration used by the other browse paths. One-line change.

Verification

Base SHA: 3612602 (rocketmq-studio)

Fail-before / pass-after on the two new tests in RocketMQMessageProviderTest:

  • getQueueOffsetsListsReadQueuesWhenReadCountExceedsWriteCount (read=4, write=2): failed with [0, 1] vs expected [0, 1, 2, 3]; passes after the fix (also asserts min/max offsets are wired per queue).
  • getQueueOffsetsSkipsWriteOnlyQueuesWhenWriteCountExceedsReadCount (read=2, write=4): failed with [0, 1, 2, 3] vs expected [0, 1]; passes after the fix.

Full run after the fix:

  • RocketMQMessageProviderTest: 38/38 pass (36 pre-existing + 2 new)
  • whole org.apache.rocketmq.studio.provider.apache package: 256/256 pass

AI disclosure: This change was prepared with AI assistance (GitHub Copilot/Claude-style tooling guided by a human contributor).

getQueueOffsets iterated queueData.getWriteQueueNums() while every other
browse path is read-queue based: the broker's PullMessageProcessor
rejects queueId >= readQueueNums with SYSTEM_ERROR, and
fetchSubscribeMessageQueues (used by queryByTopic and the classic
console) enumerates [0, readQueueNums). When read != write:

- read > write (shrink draining): queues in [write, read) still hold
  browsable messages but were missing from the QueueBrowser
- write > read (queues not yet readable): queues in [read, write) were
  listed but every pull on them fails with "queueId is illegal"

Enumerate from readQueueNums to match the broker's pull validation.
@zjncs
zjncs marked this pull request as ready for review September 5, 2026 09:29
Copilot AI lite review requested due to automatic review settings September 5, 2026 09:29

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@RockteMQ-AI RockteMQ-AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Correct fix — getQueueOffsets should enumerate read queues, not write queues:

  • Consumers read from read queues
  • Broker rejects pull requests for queueId >= readQueueNums with SYSTEM_ERROR
  • When writeQueueNums > readQueueNums, old code listed non-browsable queues
  • When readQueueNums > writeQueueNums (queue shrinking), old code missed queues with messages

Tests validate both scenarios (read > write and write > read).

LGTM.


Automated review by github-manager-bot

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants