diff --git a/docs/rfc/006-format-negotiation.md b/docs/rfc/006-format-negotiation.md new file mode 100644 index 000000000..f8d705ae3 --- /dev/null +++ b/docs/rfc/006-format-negotiation.md @@ -0,0 +1,172 @@ +# RFC: Conjure Format Negotiation + +2018-10-04 + +Today, Conjure supports one wire format: PLAIN for primitive data types in headers, path parameters, and query +parameters, and JSON for complex objects (and primitives) in HTTP bodies. In the future, we may want to introduce new +versions of this wire format, or entirely new wire formats like CBOR. This RFC defines a negotiation strategy by which +clients and servers agree on the wire format used for requests and responses. + +We have the following requirements for the format negotiation protocol: +- The protocol should in almost all cases not incur additional round-trips +- Clients and servers will eventually agree on the protocol that is (1) most preferred by the client and (2) + supported by the server. That, clients drive the negotiation, constrained by server capabilities. + +### Definitions + +- A *version* is a positive integer +- A *Conjure format identifier* is a string of the `application/; conjure=` where `` is a + non-empty string over `[a-z-]` (e.g., `json`) and `` is a version string (as above) +- A *Conjure format list* is a comma-separated, ordered list of Conjure format identifiers +- The `Accept` and `Content-Type` HTTP headers are defined as per the + [HTTP 1.1 spec](https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html). Note that `Accept: ` is a + valid HTTP 1.1 Accept header, and that `Content-Type: ` is a valid HTTP 1.1 Content-Type header. + +### Wire format versioning + +A Conjure wire format comprises specifications for Conjure parameters and return values. Today, the canonical format +uses JSON objects for `body` request and response objects and PLAIN for all other parameters. + +We propose that every revision of a Conjure wire format be labelled with a format identifier. The current PLAIN+JSON +format shall be labelled `application/json; conjure=1`. To allow for backwards compatibility, clients and servers +should interpret a `application/json` header without a `conjure=` parameter as the `application/json; +conjure=1` format. + +### Format capabilities + +We assume that clients and servers implement both serialization and deserialization for a given format: + +- A server can consume requests with a given format iff it can produce responses with the same format. +- A client can produce requests with a given format iff it can consume responses with the same format. + +### Negotiation protocol + +**Requests.** +Clients send the format identifier used to encode the request as a `Content-Type` header, and a format list as an +`Accept` header. The format indicates the preference-ordered list of formats that the client supports. The supported +formats must include the format use to encode the request, i.e., the format specified in the `Content-Type` request +header. + +**Responses.** +Servers that do not support the request format respond with HTTP status code `415 Unsupported Media Type`. Otherwise, if +the server does support the request format, it uses the most-preferred (as per `Accept` request header) format to encode +the response and advertise the chosen format in the response `Content-Type` header. Upon receiving a `415` response, +clients may choose to repeat the request encoded with a different (i.e., typically older) format. + +**Binary data** Conjure supports endpoints accepting `binary` request parameters or returning `binary` responses. In +accordance with the wire spec, a request carrying binary data in the request body must include a `Content-Type: +application/octet-stream` request header, and a server returning a `binary` response must include a `Content-Type: +application/octet-stream` response header. Further, when calling endpoints returning a `binary` response, clients must +include in the `Accept` format list: + +- `application/octet-stream` +- the list of supported Conjure formats and versions (e.g., `application/json; version=2`) + +The rationale for including both the `octet-stream` and the `json` (or `cbor`, etc.) formats is that the former is used +when returning binary data and the later for returning structured Conjure errors. The `application/octet-stream` format +is not versioned. + + +### Discussion + +**Clients in control.** Clients are in control of the format negotiation in the sense that rank the list of acceptable +protocols based on their preference and let the server merely "chooses" based on its support for the most preferred +formats. Further, clients are in control of the trade-off between choosing the newest or most preferred versus an older +or more widely supported format. The former approach unlocks new formats and features more quickly, but may result in an +additional round-trip when the client has to reissue the request encoded with a different and hopefully supported +format. A more complex negotiation mechanism (e.g., based on OPTIONS endpoints or `Accept-Post` response headers) is +possible, but outside the scope of this RFC. + +**OPTIONS.** This RFC does not propose to make the set of server-supported formats available as through an OPTIONS +request. A client receiving a `415` error from a server can instead retry the request with the maximal list of +client-supports formats and let the server pick the most preferred one. + +**Blue/green, server vs service.** This RFC does not propose any explicit mechanism for distinguishing between the +different servers making up a remote service. For example, a service undergoing a blue/green migration between versions +that support different sets of formats may pick different response formats depending on which server handles the +request. We submit that this will rarely cause issues since clients and servers would still be able to agree on a format +and even switching between different formats for subsequent requests would be acceptable. + +**Example: Conservative client.** The following sequence of two requests and corresponding responses are between a client +that supports CBOR version 2, CBOR version 1, and JSON version 1, and that prefers formats in that order, and a server +that supports CBOR version 1 and JSON version 1. To bootstrap a session, the client makes conservative assumptions about +the server's capabilities and encodes the first request with a widely supported JSON format, version 1. The servers +encodes the response with the format most preferred by the client that it also supports itself, CBOR version 1. The +second request is encoded with the format most preferred by the client that the server supports, CBOR version 1. + +```text +Request + Content-Type: application/json; conjure=1 + Accept: application/cbor; conjure=2, application/cbor; conjure=1, application/json; conjure=1 + +Response + Content-Type: application/cbor; conjure=1 + +Request + Content-Type: application/cbor; conjure=1 + Accept: application/cbor; conjure=2, application/cbor; conjure=1, application/json; conjure=1 + +Response + Content-Type: application/cbor; conjure=1 +``` + +**Example: Cutting edge client.** If a client accepts only the most cutting edge version of a format, an older server +may not be able to pick a supported format. The client may choose to error, or can alternatively retry the request +with an older format. + +```text +Request + Content-Type: application/json; conjure=2 + Accept: application/json; conjure=2 + +Response + Status: 415 + +Request + Content-Type: application/json; conjure=1 + Accept: application/json; conjure=1 + +Response + Content-Type: application/json; conjure=1 +``` + +**Example: Old server.** A server that predates this RFC may ignore the format instructions and decode the request +w.r.t. the canonical JSON format (i.e., `application/json; conjure=1`). The response should carry the appropriate +content type as per the wire spec. + +```text +Request + Content-Type: application/json; conjure=2 + Accept: application/json; conjure=2 + # Note that the server will interpret the version 2 request as if it were version 1 + +Response + Content-Type: application/json + # Clients may assume that application/json is the version 1 JSON format +``` + +**Example: JSON request, binary response.** A request to an endpoint returning `binary` data must include +both the `octet-stream` and a standard Conjure format list in the `Accept` header. + +```text +Request + Content-Type: application/json; conjure=2 + Accept: application/octet-stream, application/json; conjure=2 + +Response + Content-Type: application/octet-stream +``` + +In case the server returns a structured Conjure error, it formats the error according to the most prefered of the +accepted Conjure formats. + +```text +Request + Content-Type: application/json; conjure=2 + Accept: application/octet-stream, application/json; conjure=2 + +Response + Content-Type: application/json; conjure=2 + Status: 4xx or 5xx + # body contains Conjure error in JSON version 2 format +```