Skip to content

UDP message coalescing causing EVP_DecryptFinal_ex to fail? #4955

Open
@shit-lord

Description

@shit-lord

Describe the bug

Server and client are in the same LAN. Server sends 2.5mb data(the content of msquic.dll) with a single StreamSend call.

With the default UDP_RECV_MAX_COALESCED_SIZE enabled state, EVP_DecryptFinal_ex failed almost always occurs in client side when decrypting every sent and resent packets from the StreamSend.
Connection establishing packets are fine.

I noticed that the datagram sizes in quic.etl did not match captured packets using raw socket.
For example, an event in etl shows Recv 1 UDP datagrams, 2440 bytes but there is no packet larger than 1500 bytes from the capture. Not sure if this is abnormal though.

Nevertheless, when I turned off UDP_RECV_MAX_COALESCED_SIZE by changing Option = MAX_URO_PAYLOAD_LENGTH; to Option = 0; in SocketCreateUdp, the problem seemed to go away.

Affected OS

  • Windows
  • Linux
  • macOS
  • Other (specify below)

Additional OS information

server win11 23h2
client win11 24h2

MsQuic version

main

Steps taken to reproduce bug

  1. UDP_RECV_MAX_COALESCED_SIZE=MAX_URO_PAYLOAD_LENGTH
  2. server StreamSend 2.5mb
  3. client EVP_DecryptFinal_ex failed. SHUTDOWN_INITIATED_BY_TRANSPORT
  4. UDP_RECV_MAX_COALESCED_SIZE=0
  5. server StreamSend 2.5mb
  6. succeed

Expected behavior

the default UDP_RECV_MAX_COALESCED_SIZE causing no error

Actual outcome

EVP_DecryptFinal_ex failed in clientside

Additional details

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    Bug: PlatformA code bug in platform/TLS specific code.externalProposed by non-MSFT

    Type

    Projects

    Status

    No status

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions