MQTT 5.0 support — guidance on how to structure the PR(s)?

[nberlee]

Hi maintainers 👋

I've implemented experimental, opt-in MQTT 5.0 support for nats-server, layered on top of the existing 3.1.1 implementation, and I'd like your guidance on how you'd prefer to receive it before I open any PRs.

Full diff (all commits): main...nberlee:nats-server:mqtt5-shared-subs

Design principles

• Opt-in and off by default. A new mqtt.max_protocol_version defaults to 3.1.1; it must be set to 5 to enable v5. A v5 CONNECT to a non-enabled server is cleanly rejected with reason 0x84 (Unsupported Protocol Version) so clients fall back.
• No change to 3.1.1 behavior. Every v5 branch is gated on the per-connection negotiated protocol level, and a server-level flag lets v5-only machinery be skipped entirely on a 3.1.1-only server.
• Experimental. A [WRN] is logged at startup when enabled.
• Tested. Each commit ships with its own unit/conformance tests.

mqtt {
  port: 1883
  max_protocol_version: 5   # 0/4 = 3.1.1 only (default), 5 = enable MQTT 5.0
}

What's implemented (18 feature commits)

Foundation (prerequisite for everything else)

• Protocol negotiation, per-connection version tracking, the properties (variable-header) codec with strict validation, v5 reason codes across CONNACK/PUBACK/PUBREC/PUBREL/PUBCOMP/SUBACK/UNSUBACK/DISCONNECT, and v5 framing for CONNECT/CONNACK/PUBLISH/SUBSCRIBE/DISCONNECT.

PUBLISH properties

• Forward PUBLISH properties (Content Type, Response Topic, Correlation Data, User Properties, Payload Format Indicator) to subscribers.

Flow control & size limits

• Honor the client's Receive Maximum; advertise & enforce the server's Receive Maximum.
• Honor the client's Maximum Packet Size; enforce Maximum Packet Size on all server-generated packets.

Session / message lifecycle

• Message Expiry Interval
• Will Delay Interval
• Session Expiry Interval

Subscription options & properties

• Retain Handling, No Local, Retain As Published, Subscription Identifier

Topic Alias

• Honor the Topic Alias publish property (client→server); send Topic Alias on server→client PUBLISH.

Server niceties & the big one

• Server Keep Alive, Reason String on server-generated packets.
• Shared Subscriptions.

Combined this is ~15k lines including tests, so I don't think a single squashed PR is the right call, hence this question.

Note on 3 non-v5 commits

The compare link currently also contains 3 MQTT 3.1.1 conformance bugfixes (packet-identifier reuse, pending-delivery leak on QoS 0 downgrade, and prompt redelivery on reconnect). Those are independent of v5 and I'm submitting them as separate standalone PRs; they'd be dropped from the v5 series. (#8358, #8359, #8360)

How would you like this structured?

A few options I can see, happy to do whichever fits your review process:

1. One large PR with the full commit series (easy to see the whole design; heavy to review).
2. A stacked series of ~18 small PRs, each one feature, merged in dependency order (very reviewable; more PR overhead).
3. Foundation PR first (required), then a handful of themed batches (e.g. flow-control/limits, lifecycle/expiry, subscription options, topic alias, shared subscriptions).
4. Required core first, then each remaining feature as an optional independent PR. This is going to take a very long time, since i can only have one active PR at a time.

Dependencies to be aware of: the Foundation commit must land first; a few features build on earlier ones (e.g. Topic Alias server-side builds on the client-side property; server Receive Maximum builds on client Receive Maximum).

Is this something you'd want upstream at all, and if so, which shape works best for you? Could the tracking issue be the closed Support MQTT v5 #3369 (and be reopend) or the broad Improve MQTT #6562 ?

Thanks!

[neilalexander]

Appreciate the enthusiasm but I don't think we are likely to accept PRs for MQTT 5 support.

MQTT is not our primary API surface, our primary concern is and will continue to be the NATS protocol. We have supported MQTT 3.1.1 as a simple bridging mechanism to help people move towards a NATS-native system (and will happily accept PRs that fix up 3.1.1 conformance or bugs) but I don't think we want to own the ongoing maintenance cost for the massive API surface increase that comes with MQTT 5 at this time.

[cbrake]

@nberlee would it be possible to implement this in an external bridge process, or is the functionality too intertwined with the nats-server?

There is a lot of value in MQTT5 support, as industry is moving in that direction.

[nberlee]

Yes, that would be a next step looking into. No promises, because I would like to solve this in future proof, performant and not too hard way.

If that's possible with a bridge project, then I am going to make that.