Permalink
Switch branches/tags
Find file Copy path
16364b6 Dec 5, 2017
1 contributor

Users who have contributed to this file

107 lines (62 sloc) 9.36 KB

Understanding NATS Architecture

NATS is a publish/subscribe message oriented middleware with an emphasis on simplicity, performance, security, and scalability. It was built from the ground up to operate in the cloud.

NATS messaging is comprised of core NATS, and NATS streaming. Core NATS supports at-most-once delivery, is designed to be lightweight, performant, and always available. NATS Streaming supports log based persistence providing at-least-once delivery, replay of messages, and subscription continuity (durable subscribers).

Core NATS

Core NATS is the ideal messaging software for publish/subscribe, request/reply, and work queue messaging patterns.

NATS Server

The NATS server routes messages between NATS clients - applications that use the NATS protocol (usually via a NATS client library) to connect to the the NATS server (gnatsd). Logically, applications communicate over a message bus, but the network configuration is the standard TCP client-server model.

TCP NATS Client/Server

NATS clients send messages to the NATS server over TCP connections established by NATS client libraries. Published messages are delivered to clients based on subscriptions made to subjects.

The NATS server supports TLS and Authorization/Authentication.

Clustering

Running a single NATS server introduces a SPOF. In order to provide high availability and scalability, NATS servers support full mesh clustering. Each server is connected to all other servers in the cluster. There is a one-hop message routing maximum, ensuring messages will never loop througout a cluster. The servers communicate with each other using a server-to-server clustering protocol over a TCP connection. The protocol supports "discovery" to propogate topology information and changes in real-time with other members of the cluster and clients. Thus, servers can be dynamially added or removed from a cluster at runtime with zero downtime. Ideally, a client will have at least 2 addresses of "seed" servers.

NATS Server Cluster

It is important to note that from a client perspective, a NATS cluster is considered one entity. An officially supported NATS client only requires the address of one server in the cluster to connect, but will then receive the complete cluster topology. The client is able to fail over to other servers in the cluster in the event of a crash or network partition.

More information about clustering can be found here.

Subscriptions and routing

When a NATS client creates a subscription, it registers interest for a subject in the server. Subjects are discussed in the protocol conventions. The server maps interest in this subject to the particular subscription on the client. When the server receives a message, it inspects the subject, and routes the message to all subscriptions that have interest in the subject.

NATS Server Routing

When servers are clustered, they automatically register interest to other servers in the cluster on behalf of their clients, providing message delivery to clients regardless of which server in the cluster they are connected to.

NATS Server Routing Interest

Notably, messages only get routed to servers in the cluster with client interest, so are not unnecessarily propogated across a network.

NATS Server Routing Pruning

Core NATS client design and architecture

The NATS protocol is text based and simple, with only a handful of verbs. NATS Clients are fairly straightforward. Complexity typically falls into reconnection algorithms and the buffering of messages. Architecture varies based on the idiomatic features of the client language or platforms, although all officially maintained clients support the following features:

  • Allow credentials to be passed when connecting to a server
  • TLS support
  • Publishing of messages
  • Subscribing to subjects and receiving messages
  • Buffering messages for resiliency
  • Reconnection to servers on detecting broken connections
  • Update available servers via the discovery protocol

The typical flow of a NATS client is very straightforward:

  1. Establish a connection to a server and setup error/notification handlers.
  2. Optionally subscribe to subject(s) and setup handlers to process messages.
  3. Optionally publish messages.
  4. When finished, a client will disconnect from the NATS server.

Streaming server

The NATS streaming server and streaming clients are a different protocol than core NATS. Conceptually it's useful to consider NATS Streaming as a layer above NATS - streaming servers are actually core NATS clients. This offers flexibility in allowing NATS streaming servers to have dedicated hosts, distributing work.

When NATS streaming clients connect, they create a logical connection over core NATS to a streaming server; one might consider this a session established with the streaming server over core NATS connectivity. The NATS streaming server is associated with a streaming cluster-id, which alongside a unique client-id provided by a client is used to setup internal unique subjects for streaming clients to server communication. Clients then publish and subscribe to the NATS streaming server, receiving acknowledgements that their messages have been persisted to meet the at-least-once messaging guarantee.

The NATS streaming server requires a core NATS server to operate and defaults to using a side-car architecture by launching a NATS server instance in its process space. This is a convenience feature. While there is a single process, when NATS streaming clients connect, they are actually connecting to the internal NATS server. This internal NATS server is fully functional and can be configured to join an existing core NATS server cluster. The NATS streaming server can run stand-alone, and connect to an external NATS server cluster. Running stand-alone is slightly less convenient, but may yield better performance.

Regardless, from a network (TCP) standpoint the client and NATS streaming server look like this:

NATS Streaming Server and Clients

NATS Streaming high availability options

NATS streaming supports two methods to acheive fault tolerance / high availability:

Partitioning

Streaming servers can be partitioned to scale. Multiple streaming servers in the same cluster distribute work based on assigned channnels.

NATS Streaming Server Partitioning

Streaming client design and architecture

The NATS streaming protocol is more complex, as it requires a larger number of fields in the internal protocol messages. It is a binary protocol over the NATS protocol utlilizing protobuf for serialization. While NATS streaming clients use a different client API, many of the of the features found in core NATS are available to NATS streaming clients. However, streaming messages and core NATS messages are not interchangable. NATS streaming also uses a separate subject namespace than core NATS, so messages can not be published via streaming and subscribed via core NATS.

All officially supported clients provide the following:

  • A logical streaming connection with the NATS streaming server over core NATS.
  • Publishing of Messages
  • Subscribing to subjects to receive messages, supporting the various subscription options found here, as well as durable subscription support.
  • Queue group subscriptions.
  • Support for handling publish acknowledgements and acknowedging received messages.

The typical flow of a NATS streaming client is very similar to a core NATS client:

  1. Establish a connection to a streaming server
  2. Optionally subscribe to subject(s) and setup handlers to process messages. Messaged are acknowedged.
  3. Optionally publish messages, and handle publish acknowedgements from the server.
  4. When finished, a client will close its connection with the NATS streaming server.