MQTT Message Broker with Scalability written in Java.
MQTT is an open industry standard, specifying a light weight publish-subscribe messaging protocol. It is well suited for constrained devices on unreliable networks.
J1st-mqtt is an open source, distributed MQTT message broker for real world. It embraces the Microservices Architecture, and designed to fit into complex server-side application.
As a MQTT message broker, J1st-mqtt scales both horizontally and vertically on commodity hardware to support a large number of concurrent MQTT clients while maintaing low latency and fault tolerence.
As a Microservice, J1st-mqtt is small self contained with little external dependencies, using pluggable Communicator to coexist with other microservices.
- Fully compatible with MQTT v3.1.1 specification.
- Support QoS 0, QoS 1, QoS 2.
- Support session state and clean session.
- Support session keepalive.
- Support message delivery retry (when connect).
- Support retain message.
- Support topic name and topic filter (with wildcards).
- Strong message ordering for each session.
- Extensible authorization structure. J1st-mqtt can control operations like Connect Publish Subscribe Unsubscribe by providing authorization plugin.
- Distributed by design. J1st-mqtt is decentralized, can easily scale up and out. Nodes talking to each other via communicator.
- Fault tolerance. When used with load balancer, there will be no single point of failure.
- Redis storage. The only required external dependency is the Redis database, which J1st-mqtt used to store session state. Redis 2.8 and above is supported (include 3.x cluster).
- Communicator and $SYS topic. Communicator is a switchable internal implementation based on message queue or rpc. Normally MQTT brokers provide the $SYS topic for server side integration, J1st-mqtt uses communicator to pass messages to other microservices, which is more flexible and tied into your exist application. Communicator support Hazelcast, Kafka based implementation at the moment.
- RESTful HTTP interface. Although MQTT is a stateful protocol, J1st-mqtt provided a HTTP wrapper to MQTT operations. The HTTP server is also scalabe, and can be used both internally and publicly.
- Optinal InfluxDB based metrics. J1st-mqtt broker can gather MQTT related metrics and push into influxDB.
This is the high level architecture design for a typical application service using J1st-mqtt.
- User: Maybe a device or an app speaks MQTT.
- Load Balancer: TCP (HTTP) load balancer like Pound LVS HAproxy or the service provided by cloud.
- Communicator: J1st-mqtt internal commuication implmentation based on Hazelcast or Kafka.
- MQTT Broker: J1st-mqtt MQTT broker handle messages from User (through Load Balancer), redirect internal messages to Communicator.
- MQTT HTTP Interface: J1st-mqtt MQTT HTTP interface handle requests and transfer to internal messages, send to corresponding MQTT Broker via Communicator.
- Redis: The main storage for J1st-mqtt.
- InfluxDB: Optional storage for MQTT Broker and MQTT HTTP Interface metrics.
- Cloud Service: Application service which can receive inbound MQTT (Internal Format) messages from Communicator, and send outbound MQTT (Internal Format) messages from MQTT HTTP Interface.
J1st-mqtt broker is tested against Eclipse Paho's MQTT Conformance/Interoperability Testing.
- Basic Test
$ python client_test.py -z -d -s
hostname localhost port 1883
clean up starting
clean up finished
Basic test starting
Basic test succeeded
Retained message test starting
Retained message test succeeded
This server is not queueing QoS 0 messages for offline clients
Offline message queueing test succeeded
Will message test succeeded
Overlapping subscriptions test starting
This server is publishing one message for all matching overlapping subscriptions, not one for each.
Overlapping subscriptions test succeeded
Keepalive test starting
Keepalive test succeeded
Redelivery on reconnect test starting
Redelivery on reconnect test succeeded
Zero length clientid test starting
Zero length clientid test succeeded
Subscribe failure test starting
Subscribe failure test succeeded
topics test starting
topics test succeeded
test suite succeeded