Skip to content

Latest commit

 

History

History
311 lines (224 loc) · 11.5 KB

messaging.asciidoc

File metadata and controls

311 lines (224 loc) · 11.5 KB

Messaging layer

This section discusses the semantic meaning and layout of protocol messages. For details on how Neo4j types are represented in binary form, see serialization.

Clients may send request messages at any time after a session is initialized. Clients may pipeline requests, sending multiple requests together.

Servers must fully respond to each request before the next request is processed and processing of requests within a session must be done in the same order in which the requests.

Servers must ignore messages sent by the client after a failure occurs on the server, until the client has acknowledged the failure. See Failure & Acknowledgement.

For each request message sent, clients must anticipate receiving zero or more detail messages followed by exactly one summary message. The detail messages deliver the response content, while a summary message denotes the end of the response and any response metadata. Note that "detail" and "summary" are classifications of message, not specific message types. For example, RECORD messages are classed as detail messages and SUCCESS messages as summary messages.

The diagrams below illustrates a basic exchange wherein the client sends a request message and receives a series of response messages.

simple exchange

Initialization

Before a session can be used to run queries, it must be initialized. The INIT message should be sent by the client as the first message it sends after negotiating protocol version.

Pipelining

The client is not required to wait for a response before sending more messages. Sending multiple messages together like this is called pipelining:

pipelining

For performance reasons, it is recommended that clients use pipelining as much as possible. Through pipelining, multiple messages can be transmitted together in the same network package, significantly reducing latency and increasing throughput.

Tip
A common technique is to buffer outgoing messages on the client until the last possible moment, such as when a commit is issued or a result is read by the application, and then sending all messages in the buffer together.

Failure handling

Because the protocol leverages pipelining, the client and the server need to agree on what happens when a failure occurs, otherwise messages that were sent assuming no failure would occur might have unintended effects.

When requests fail on the server, the server will send the client a FAILURE message. The client must acknowledge the FAILURE message by sending an ACK_FAILURE message to the server. Until the server receives the ACK_FAILURE message, it will send an IGNORED message in response to any other message from by the client, including messages that were sent in a pipeline.

The diagram below illustrates a typical flow involving ACK_FAILURE messages:

failure ack

Here, the original failure is acknowledged immediately by the client, allowing the subsequent RUN to be actioned as expected.

This second diagram shows a sequence where a pair of request messages are sent together:

failure optimistic

Here, the client optimistically sends a pair of messages. The first of these fails and the second is consequently IGNORED. Once the client acknowledges the failure, it is then able to resend a corrected RUN message.

Messages

Protocol messages are represented as serialized structures.

INIT

The INIT message is a client message used once to initialize the session. This message is always the first message the client sends after negotiating protocol version. Sending any message other than INIT as the first message to the server will result in a FAILURE and the server closing the connection.

The server will reply with a SUCCESS message unless some parameter for initialization is malformed.

All parameters in the INIT message are required.

InitMessage (signature=0x01) {
    String clientName
}
Example
Value: INIT "MyClient/1.0"

B1 01 8C 4D  79 43 6C 69  65 6E 74 2F  31 2E 30
Table 1. Initialize message parameters
Parameter Description

clientName

A name and version for the client, if the user has allowed usage data collection, this is used to track popular clients. Example: "MyClient/1.0"

RUN

The RUN message is a client message used to start a new job on the server. It has the following structure:

RunMessage (signature=0x10) {
    String             statement
    Map<String,Value>  parameters
}

On receipt of a RUN message, the server will start a new job by executing the statement with the parameters supplied. If successful, the subsequent response will consist of a single SUCCESS message; if not, a FAILURE response will be sent instead. A successful job will always produce a result stream which must then be explicitly consumed (via PULL_ALL or DISCARD_ALL), even if empty.

Depending on the statement you are executing, additional metadata may be returned in both the SUCCESS message from the RUN, as well as in the final SUCCESS after the stream has been consumed. It is up to the statement you are running to determine what meta data to return. Notably, most queries will contain a 'fields' metadata section in the SUCCESS message for the RUN statement, which lists the result record field names. We list further examples of meta data in the examples section.

In the case where a previous result stream has not yet been fully consumed, an attempt to RUN a new job will trigger a FAILURE response.

If an unacknowledged failure is pending from a previous exchange, the server will immediately respond with a single IGNORED message and take no further action.

Example
Value: RUN "RETURN 1 AS num" {}

B2 10 8F 52  45 54 55 52  4E 20 31 20  41 53 20 6E  75 6D A0

DISCARD_ALL

The DISCARD_ALL message is a client message used to discard all remaining items from the active result stream. It has the following structure:

DiscardAllMessage (signature=0x2F) {
}

On receipt of a DISCARD_ALL message, the server will dispose of all remaining items from the active result stream, close the stream and send a single SUCCESS message to the client. If no result stream is currently active, the server will respond with a single FAILURE message.

If an unacknowledged failure is pending from a previous exchange, the server will immediately respond with a single IGNORED message and take no further action.

Example
Value: DISCARD_ALL

B0 7E

PULL_ALL

The PULL_ALL message is a client message used to retrieve all remaining items from the active result stream. It has the following structure:

PullAllMessage (signature=0x3F) {
}

On receipt of a PULL_ALL message, the server will send all remaining result data items to the client, each in a single RECORD message. The server will then close the stream and send a single SUCCESS message optionally containing summary information on the data items sent. If an error is encountered, the server must instead send a FAILURE message, discard all remaining data items and close the stream.

If an unacknowledged failure is pending from a previous exchange, the server will immediately respond with a single IGNORED message and take no further action.

Example
Value: PULL_ALL

B0 3F

ACK_FAILURE

The ACK_FAILURE message is a client message used to signal that a client has acknowledged a previous FAILURE . It has the following structure:

AcknowledgeFailureMessage (signature=0x0F) {
}

On receipt of an ACK_FAILURE message, the server will clear any pending failure state and respond with a single SUCCESS message. If no such failure state is pending, a FAILURE message will be sent instead.

An ACK_FAILURE will never be ignored by the server.

Example
Value: ACK_FAILURE

B0 0F

RECORD

The RECORD message is a server detail message used to deliver data from the server to the client. Each record message contains a single List, which in turn contains the fields of the record in order. It has the following structure:

RecordMessage (signature=0x71) {
    List<Value> fields
}
Example
Value: RECORD [1,2,3]

B1 71 93 01  02 03

SUCCESS

The SUCCESS message is a server summary message used to signal that a corresponding client message has been received and actioned as intended. The message contains a map of arbitrary meta-data, the contents of which depends on the original request. It has the following structure:

SuccessMessage (signature=0x70) {
    Map<String,Value> metadata
}
Example
Value: SUCCESS { "fields": ["name", "age"]}

B1 70 A1 86  66 69 65 6C  64 73 92 84  6E 61 6D 65
83 61 67 65

FAILURE

The FAILURE message is a server summary message used to signal that a corresponding client message has encountered an error while being processed. It has the following structure:

FailureMessage (signature=0x7F) {
    Map<String,Value> metadata
}

FAILURE messages contain metadata providing details regarding the primary failure that has occurred. This metadata is a simple map containing a code and a message. These codes map to the standard Neo4j status codes.

When a FAILURE occurs, in most cases any open transaction will be rolled back. However, if the FAILURE is classified as a client error, the transaction will be left open and can be used again after the FAILURE has been acknowledged. This is mainly to support user-driven queries, where a database administrator may have built up a large transaction, and we do not want a simple spelling mistake to roll it all back.

Example
Value: FAILURE { "code": "Neo.ClientError.Statement.InvalidSyntax", "message": "Invalid syntax." }

B1 7F A2 84  63 6F 64 65  D0 27 4E 65  6F 2E 43 6C
69 65 6E 74  45 72 72 6F  72 2E 53 74  61 74 65 6D
65 6E 74 2E  49 6E 76 61  6C 69 64 53  79 6E 74 61
78 87 6D 65  73 73 61 67  65 8F 49 6E  76 61 6C 69
64 20 73 79  6E 74 61 78  2E

IGNORED

The IGNORED message is a server summary message used to signal that a corresponding client message has been ignored and not actioned. It has the following structure:

IgnoredMessage (signature=0x7E) {
    Map<String,Value>  metadata
}

A client message will be ignored if an earlier failure has not yet been acknowledged by the client via an ACK_FAILURE message. For example, this will occur if the client optimistically sends a group of messages, one of which fails during execution: all subsequent messages in that group will then be ignored. Note that the original PULL_ALL message was never processed by the server.

Example
Value: IGNORED

B0 7E