Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

parseMessageMetadata error when broker entry metadata enable #10967

Closed
aloyszhang opened this issue Jun 18, 2021 · 0 comments · Fixed by #10968
Closed

parseMessageMetadata error when broker entry metadata enable #10967

aloyszhang opened this issue Jun 18, 2021 · 0 comments · Fixed by #10968
Labels
type/bug The PR fixed a bug or issue reported a bug

Comments

@aloyszhang
Copy link
Contributor

aloyszhang commented Jun 18, 2021

Describe the bug
If broker entry metadata enabled, we may get the following exception in many cases :

Received error from server: Failed to get batch size for entry java.lang.IllegalArgumentException: Invalid unknonwn tag type: 6

this problem also reported in :
#10950
#9255
#10924

this is caused by #9046,

Before #9046 code is

public static MessageMetadata parseMessageMetadata(ByteBuf buffer) {
        try {
            // first, skip raw metadata if exist
            skipBrokerEntryMetadataIfExist(buffer);
            // initially reader-index may point to start_of_checksum : increment reader-index to start_of_metadata
            // to parse metadata
            skipChecksumIfPresent(buffer);
            int metadataSize = (int) buffer.readUnsignedInt();

            int writerIndex = buffer.writerIndex();
            buffer.writerIndex(buffer.readerIndex() + metadataSize);
            ByteBufCodedInputStream stream = ByteBufCodedInputStream.get(buffer);
            MessageMetadata.Builder messageMetadataBuilder = MessageMetadata.newBuilder();
            MessageMetadata res = messageMetadataBuilder.mergeFrom(stream, null).build();
            buffer.writerIndex(writerIndex);
            messageMetadataBuilder.recycle();
            stream.recycle();
            return res;
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
    }

After #9046,

 public static MessageMetadata parseMessageMetadata(ByteBuf buffer) {
        MessageMetadata md = LOCAL_MESSAGE_METADATA.get();
        parseMessageMetadata(buffer, md);
        return md;
    }

    public static void parseMessageMetadata(ByteBuf buffer, MessageMetadata msgMetadata) {
        // initially reader-index may point to start_of_checksum : increment reader-index to start_of_metadata
        // to parse metadata
        skipChecksumIfPresent(buffer);
        int metadataSize = (int) buffer.readUnsignedInt();

        msgMetadata.parseFrom(buffer, metadataSize);
    }

this PR removed code

           skipBrokerEntryMetadataIfExist(buffer);

So, this may cause exceptions when parseMessageMetadata

@aloyszhang aloyszhang added the type/bug The PR fixed a bug or issue reported a bug label Jun 18, 2021
BewareMyPower pushed a commit that referenced this issue Jun 18, 2021
#10968)

Fixes #10967

### Motivation
fix parseMessageMetadata error cause by not skip broker entry metadata 

### Modifications

skip broker entry metadata if exist before parsing message metadata
yangl pushed a commit to yangl/pulsar that referenced this issue Jun 23, 2021
apache#10968)

Fixes apache#10967

### Motivation
fix parseMessageMetadata error cause by not skip broker entry metadata 

### Modifications

skip broker entry metadata if exist before parsing message metadata
codelipenghui pushed a commit that referenced this issue Jun 25, 2021
#10968)

Fixes #10967

### Motivation
fix parseMessageMetadata error cause by not skip broker entry metadata 

### Modifications

skip broker entry metadata if exist before parsing message metadata

(cherry picked from commit 0774b5f)
eolivelli pushed a commit to datastax/pulsar that referenced this issue Sep 6, 2021
apache#10968)

Fixes apache#10967

### Motivation
fix parseMessageMetadata error cause by not skip broker entry metadata

### Modifications

skip broker entry metadata if exist before parsing message metadata

(cherry picked from commit 0774b5f)
bharanic-dev pushed a commit to bharanic-dev/pulsar that referenced this issue Mar 18, 2022
apache#10968)

Fixes apache#10967

### Motivation
fix parseMessageMetadata error cause by not skip broker entry metadata 

### Modifications

skip broker entry metadata if exist before parsing message metadata
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/bug The PR fixed a bug or issue reported a bug
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant