Skip to content

Auto-port 4.1: Fix MQTT decoder size check after variable header replay#16838

Merged
chrisvest merged 1 commit into
4.1from
auto-port-pr-16787-to-4.1
May 20, 2026
Merged

Auto-port 4.1: Fix MQTT decoder size check after variable header replay#16838
chrisvest merged 1 commit into
4.1from
auto-port-pr-16787-to-4.1

Conversation

@netty-project-bot
Copy link
Copy Markdown
Contributor

Auto-port of #16787 to 4.1
Cherry-picked commit: 72df658


Problem

The MQTT decoder can reject valid packets after the CVE-2026-44248 fix when several MQTT packets are present in the same cumulation buffer. If the current packet's variable header needs a replay, the decoder compares the total readable bytes in the buffer against maxBytesInMessage, so later packets can make the current in-limit packet look too large.

Root Cause

READ_VARIABLE_HEADER recorded buffer.readableBytes() before decoding the variable header. That value is the cumulation's total readable bytes, not the current MQTT packet's declared remaining length. When decodeVariableHeader throws Signal.REPLAY, the too-large decision must be based on bytesRemainingBeforeVariableHeader for the current packet.

Fix

  • Use bytesRemainingBeforeVariableHeader when deciding whether to swallow Signal.REPLAY and raise TooLongFrameException.
  • Continue replaying when the current packet is within maxBytesInMessage, even if the cumulation contains additional bytes for following packets.

Tests Added

Change Point Test
Variable-header replay uses the current packet size instead of total cumulation bytes for the too-long check testPublishMessageIncompleteVariableHeaderDoesNotUseCumulationSizeForTooLongCheck() verifies an incomplete in-limit PUBLISH variable header with extra cumulated PINGREQ packets does not emit an invalid message
Oversized current packets still fail during variable-header replay testPublishMessageIncompleteVariableHeaderStillFailsWhenCurrentPacketTooLarge() verifies an incomplete PUBLISH whose own remaining length exceeds maxBytesInMessage still emits a TooLongFrameException

Impact

This restores decoding for valid in-limit MQTT packets batched in the same buffer while preserving the CVE fix: packets whose declared remaining length exceeds maxBytesInMessage still fail with TooLongFrameException even if variable-header decoding requests replay.

Fixes #16776

## Problem

The MQTT decoder can reject valid packets after the CVE-2026-44248 fix
when several MQTT packets are present in the same cumulation buffer. If
the current packet's variable header needs a replay, the decoder
compares the total readable bytes in the buffer against
`maxBytesInMessage`, so later packets can make the current in-limit
packet look too large.

## Root Cause

`READ_VARIABLE_HEADER` recorded `buffer.readableBytes()` before decoding
the variable header. That value is the cumulation's total readable
bytes, not the current MQTT packet's declared remaining length. When
`decodeVariableHeader` throws `Signal.REPLAY`, the too-large decision
must be based on `bytesRemainingBeforeVariableHeader` for the current
packet.

## Fix

- Use `bytesRemainingBeforeVariableHeader` when deciding whether to
swallow `Signal.REPLAY` and raise `TooLongFrameException`.
- Continue replaying when the current packet is within
`maxBytesInMessage`, even if the cumulation contains additional bytes
for following packets.

## Tests Added

| Change Point | Test |
|-------------|------|
| Variable-header replay uses the current packet size instead of total
cumulation bytes for the too-long check |
`testPublishMessageIncompleteVariableHeaderDoesNotUseCumulationSizeForTooLongCheck()`
verifies an incomplete in-limit PUBLISH variable header with extra
cumulated PINGREQ packets does not emit an invalid message |
| Oversized current packets still fail during variable-header replay |
`testPublishMessageIncompleteVariableHeaderStillFailsWhenCurrentPacketTooLarge()`
verifies an incomplete PUBLISH whose own remaining length exceeds
`maxBytesInMessage` still emits a `TooLongFrameException` |

## Impact

This restores decoding for valid in-limit MQTT packets batched in the
same buffer while preserving the CVE fix: packets whose declared
remaining length exceeds `maxBytesInMessage` still fail with
`TooLongFrameException` even if variable-header decoding requests
replay.

Fixes #16776

(cherry picked from commit 72df658)
@chrisvest chrisvest added this to the 4.1.134.Final milestone May 20, 2026
@chrisvest chrisvest enabled auto-merge (squash) May 20, 2026 18:33
@chrisvest chrisvest merged commit 30f8f28 into 4.1 May 20, 2026
18 checks passed
@chrisvest chrisvest deleted the auto-port-pr-16787-to-4.1 branch May 20, 2026 20:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants