From 5ca3cd4b9fe289b90495367ac8e5308e5ec95326 Mon Sep 17 00:00:00 2001 From: Phil Hagelberg Date: Sun, 23 Nov 2025 10:40:34 -0800 Subject: [PATCH 1/7] Add specification document and makefile for HTML. --- Makefile | 4 + _foot.html | 2 + _head.html | 3 + spec.md | 457 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 466 insertions(+) create mode 100644 Makefile create mode 100644 _foot.html create mode 100644 _head.html create mode 100644 spec.md diff --git a/Makefile b/Makefile new file mode 100644 index 0000000..d80c012 --- /dev/null +++ b/Makefile @@ -0,0 +1,4 @@ +PANDOC ?= pandoc -f gfm -H _head.html -A _foot.html -T "nREPL Protocol" + +index.html: spec.md _head.html _foot.html + $(PANDOC) -i spec.md > $@ diff --git a/_foot.html b/_foot.html new file mode 100644 index 0000000..2c3ebe1 --- /dev/null +++ b/_foot.html @@ -0,0 +1,2 @@ +

© 2025 nREPL contributors | + site source

diff --git a/_head.html b/_head.html new file mode 100644 index 0000000..2c54109 --- /dev/null +++ b/_head.html @@ -0,0 +1,3 @@ + diff --git a/spec.md b/spec.md new file mode 100644 index 0000000..affd7c5 --- /dev/null +++ b/spec.md @@ -0,0 +1,457 @@ +# nREPL Protocol Specification + +The nREPL protocol is designed to solve the ["narrow waist" +problem][1] where you have N languages and M editors, and you want to +avoid having to create NxM adapters across them. In this way it shares +a lot with the [Language Server Protocol][2], though it predates LSP +by several years. However, the focus is different; LSP achieves its +integration using static analysis, while nREPL achieves it by running +project code. Writing an nREPL server is much easier than writing an +LSP server and can be done in a couple hundred lines. + +The nREPL protocol has its roots in the Clojure community; at the time +of this writing most nREPL users are Clojure users. But the design of +the protocol is language-agnostic and can be applied to any language +that can evaluate code at runtime. + +This document is an in-progress draft describing version 0.1.0 of the +nREPL protocol. + +## Protocol Description + +The nREPL protocol operates by default by exchanging [bencoded][3] +messages over a socket. Various implementations may also offer other +encodings and transports, such as JSON over stdio, but these are not +standardized. + +While it's somewhat unusual, bencode was chosen because it is a good +deal easier to implement than JSON; usually it can be done in under a +hundred lines of code. Unfortunately bencode is not particularly +readable, so examples in this document will show messages in JSON. + +Every message sent by the client must have an `op` field, for +operation. Every message except the first must have a `session` field +indicating which session it's part of. + +Every message sent by the client is a request. Requests can have one +or more response messages associated with them. Because a request can +have many responses, every request should be considered active until a +response is received with a `status` field which is a list that +contains the string `done`. Requests may remain active for a long time +before completing, so the responses should be handled asynchronously. + +Requests should have an `id` field, and responses to that request +should also include the same `id`. Depending on the concurrency +features of the language, it may be possible for multiple requests to +overlap. + +In this document, the examples use UUID strings for `session` and +`id`, but the only requirement is that they are unique strings. + +## Required Operations + +The absolute minimum a client or server needs to support are the first +three ops, `clone`, `eval`, and `stdin`. + +### `clone` op + +Messages are exchanged in sessions. In order to start a session, the +client sends a message with the `clone` op: + +```json +// client -> server +{"op": "clone", + "id": "01e0bbed-2819-41b8-9642-4c16a79f7efc", + "client-name": "nREPL documentation demo", + "client-version" "1.0.0"} +``` + +All fields except `op` are optional. Client information may be used +for debugging purposes. + +The response will contain a new `new-session` id which should be +included as the `session` for all subsequent requests: + +```json +// client <- server +{"new-session": "afd3c88e-707f-4169-a265-892f29476333", + "id": "01e0bbed-2819-41b8-9642-4c16a79f7efc", + "status": ["done"]} +``` + +[why is this an explicit op? why not just automatically register a +session any time a request comes in without a session attached?] + +### `eval` op + +This is the main workhorse operation where code gets run. The `code` +field contains the code to be run. + +```json +// client -> server +{"op": "eval", + "session": "afd3c88e-707f-4169-a265-892f29476333", + "id": "297a1dc1-e8ea-4a71-ac58-977841a301f4", + "code": "99 + 121"} +``` + +This should normally return a message with a `value` field containing +a string representation of the return value. + +```json +// client <- server +{"session": "afd3c88e-707f-4169-a265-892f29476333", + "id": "297a1dc1-e8ea-4a71-ac58-977841a301f4", + "value": "220", + "status": ["done"]} +``` + +In the case that evaluated code produces output, the server should +send messages that have `out` or `err` fields, for stdout and stderr +respectively. These may be sent in a separate message sent before the +"done" message that has `value` in it, or they may be present in that +message. The client should display these to the user in a way that +makes it clear they are part of the session; for example, in the editor +console right below the code was entered. + +```json +// client -> server +{"op": "eval", + "session": "afd3c88e-707f-4169-a265-892f29476333", + "id": "35e53d19-9f4a-4329-a820-d71481fdfec1", + "code": "print('hello, world')"} + +// client <- server +{"session": "afd3c88e-707f-4169-a265-892f29476333", + "id": "35e53d19-9f4a-4329-a820-d71481fdfec1", + "value": "nil", + "out": "hello world", + "status": ["done"]} +``` + +In the case that evaluated code encounters an error, the response +message should include an `ex` field instead of `value`. The format of +this field will vary depending on the way the language in question +represents errors. + +[should we go into detail about how to send stack traces?] + +```json +// client -> server +{"op": "eval", + "session": "afd3c88e-707f-4169-a265-892f29476333", + "id": "b9616f31-9fbd-4a76-b7d6-ab98eb9f7641", + "code": "client.connect(config.hostname, config.port)"} + +// client <- server +{"session": "afd3c88e-707f-4169-a265-892f29476333", + "id": "b9616f31-9fbd-4a76-b7d6-ab98eb9f7641", + "ex": "connection refused", + "status": ["done"]} +``` + +The `eval` message may also contain a `file` field indicating that the +code in question should be treated as if it came from a given file. If +this is not an absolute path, it should be interpreted as being +relative to the directory from which the server was started. The +`line` and `column` fields may include numbers indicating where in the +file the code was from. Lines start at 1 and columns start at 0. These +fields typically do not affect how the code is run, but they may help +improve stack traces if there is an error. + +In some languages, evaluation always happens in the context of a +specific namespace or module. For those languages, an `ns` field can +be included in the request which indicates the namespace to evaluate +the code in. + +```json +// client -> server +{"op": "eval", + "session": "afd3c88e-707f-4169-a265-892f29476333", + "id": "2102c017-2ff5-4ddd-9067-a54ec62fc0c8", + "file": "src/display/avatar.lua", + "line": 21, + "column": 8, + "ns": "display.avatar", + "code": "avatar.reload()"} +``` + +### `stdin` op + +In the case that evaluated code tries to read input from standard in, +the server will send a message to the client with a status of +`need-input`. When this happens, the client should accept input and +send what it receives using the `stdin` operation. + +```json +// client -> server +{"op": "eval", + "session": "afd3c88e-707f-4169-a265-892f29476333", + "id": "78f78353-c185-4211-a868-b19eaa85e054", + "code": "subsystem.activate()"} + +// client <- server +{"session": "afd3c88e-707f-4169-a265-892f29476333", + "id": "78f78353-c185-4211-a868-b19eaa85e054", + "out": "Username: ", + "status": ["need-input"]} + +// client -> server +{"op": "stdin", + "session": "afd3c88e-707f-4169-a265-892f29476333", + "id": "78f78353-c185-4211-a868-b19eaa85e054", + "stdin": "gorkon"} + +// client <- server +{"session": "afd3c88e-707f-4169-a265-892f29476333", + "id": "78f78353-c185-4211-a868-b19eaa85e054", + "output": "Activated.\n", + "value": "nil", + "status": ["done"]} + +``` + +## Optional operations + +Servers may choose to support these if they make sense. If a server +receives a request with an `op` it does not recognize, it must reply +with a message whose `status` contains `unknown-op` along with `done`. + +### `describe` op + +If a client wishes to know which operations are supported by a server, +it can query with the `describe` op, which takes no other parameters: + +```json +// client -> server +{"op": "describe", + "session": "afd3c88e-707f-4169-a265-892f29476333", + "id": "5d90576e-b5e1-4499-a43d-c75c60b579ff"} +``` + +The response must have a list of `ops` supported by the server. It may +also have a dictionary of `versions` for debugging purposes as well as +a dictionary of `features` describing additional extensions beyond the +nREPL specification. + +```json +// client <- server +{"session": "afd3c88e-707f-4169-a265-892f29476333", + "id": "5d90576e-b5e1-4499-a43d-c75c60b579ff", + "ops": ["clone", "eval", "stdin", "describe", "load-file"], + "features": {"encodings": ["bencode", "json"], + "transports": ["socket", "stdio"]}, + "versions": {"nrepl": "0.1.0", + "lua": "5.4"}, + "status": ["done"]} +``` + +[this one could really be required; it adds very little difficulty] + +### `interrupt` op + +For servers that support interrupting running code, the client may +send an `interrupt` op. Requests may optionally contain an +`interrupt-id` field which corresponds to the `id` of the request to +be interrupted. If this is omitted, it interrupts the most recent +request of the current session. + +```json +// client -> server +{"op": "eval", + "session": "afd3c88e-707f-4169-a265-892f29476333", + "id": "71629c7e-6c73-4dea-85f8-102d4b64c07f", + "code": "calculate_matrix()"} + +// client -> server +{"op": "interrupt", + "session": "afd3c88e-707f-4169-a265-892f29476333", + "id": "71629c7e-6c73-4dea-85f8-102d4b64c07f""} +``` + +The reply to this request should be a message with statuses +`interrupted` and `done` both: + +```json +// client <- server +{"session": "afd3c88e-707f-4169-a265-892f29476333", + "id": "78f78353-c185-4211-a868-b19eaa85e054", + "status": ["interrupted" "done"]} +``` + +### `lookup` op + +For servers that support providing documentation and reflective +information for functions and other values, the client may send a +`lookup` op containing a `sym` field for the item being looked up. + +```json +// client -> server +{"op": "lookup", + "session": "afd3c88e-707f-4169-a265-892f29476333", + "id": "d30f8bb9-4e6e-48a8-b0f8-58adf5b353a7", + "sym": "coroutine.create"} +``` + +The response will vary from one server to another due to language +variation, but all data should be under `info`. If documentation is +available, it should be under `doc`, and argument lists for functions +should be under `arglist`: + +```json +// client <- server +{"session": "afd3c88e-707f-4169-a265-892f29476333", + "id": "78f78353-c185-4211-a868-b19eaa85e054", + "info": {"doc": "Creates a new coroutine, with body `f`.", + "arglist": ["f"]}, + "status": ["done"]} +``` + +If the `sym` is not found, then the `info` field should be omitted. + +### `load-file` op + +A client may instruct the server to load an entire file instead of +sending its contents across the session. It may send a request with a +`load-file` op which has `file-path` indicating the path to the file +to load. Depending on the server, in some cases this may result in the +loaded code having better stack trace information. + +If `file-path` is not an absolute path, it should be interpreted as +being relative to the directory from which the server was started. + +```json +// client -> server +{"op": "load-file", + "session": "afd3c88e-707f-4169-a265-892f29476333", + "id": "b5c90973-bf4f-4626-a825-e493eed4759a", + "file-path": "src/utils.lua"} +``` + +The response should be interpreted similarly to the `eval` op: a +`value` or `err` may be included, but omitting both is also allowed. +It may also include `out` and/or `err`. + +```json +// client <- server +{"session": "afd3c88e-707f-4169-a265-892f29476333", + "id": "b5c90973-bf4f-4626-a825-e493eed4759a", + "value": "{debug=#}", + "status": ["done"]} +``` + +Compatibility note: older versions of nREPL required a `file` field +which contained code that was simply evaluated. This behavior is no +longer recommended. + +### `completions` op + +A client may request completions for a given input using the +`completions` op. The `prefix` field should be a string indicating the +input to be completed. For servers where completions may be specific +to a module or namespace context, an `ns` field may also be included +indicating this. + +```json +// client -> server +{"op": "completions", + "session": "afd3c88e-707f-4169-a265-892f29476333", + "id": "d862f516-a232-4e01-a4c1-1afb42e04637", + "prefix": "math.s"} +``` + +The response should contain a `completions` field with a list of +completion candidates: dictionaries with fields `candidate` with the +full text to complete, and `type` describing the candidate's type. + +```json +// client <- server +{"session": "afd3c88e-707f-4169-a265-892f29476333", + "id": "d862f516-a232-4e01-a4c1-1afb42e04637", + "completions": [{"candidate": "math.sqrt", "type": "function"}, + {"candidate": "math.sin", "type": "function"}, + {"candidate": "math.sinh", "type": "function"}], + "status": ["done"]} +``` + +### `find` op + +Clients may query the server to find the location where a given value +is defined using the `find` op with a `target` field containing the +value being found. + +```json +// client -> server +{"op": "find", + "session": "afd3c88e-707f-4169-a265-892f29476333", + "id": "99746b4f-a68e-4100-a56f-bb6fe63b0767", + "target": "mymodule.reloader"} +``` + +If the value is found, the response should contain a `file` and `line` +field. A `column` may also be provided. Line numbers are counted from +1, and column numbers are counted from 0. If it was found inside an +archive, for example a zip file or a jar file, it should also include +an `archive` field containing a path to the archive file. If `archive` +is present, then `file` is interpreted as being inside the archive; +otherwise it is either an absolute path or interpreted as being +relative to the directory in which the server was started. + +```json +// client <- server +{"session": "afd3c88e-707f-4169-a265-892f29476333", + "id": "b5c90973-bf4f-4626-a825-e493eed4759a", + "file": "src/mymodule.lua", + "line": 27, + "column": 2, + "archive": "lib/extras.zip", + "status": ["done"]} +``` + +### `close` op + +A client may send a `close` op to terminate the session. + +```json +// client -> server +{"op": "close", + "session": "afd3c88e-707f-4169-a265-892f29476333", + "id": "5da7c522-24eb-4a77-a133-63f08c4bdc1e"} + +// client <- server +{"session": "afd3c88e-707f-4169-a265-892f29476333", + "id": "5da7c522-24eb-4a77-a133-63f08c4bdc1e", + "status": ["done"]} +``` + +The server should close the socket after the reply is sent, if the +session is connected over a socket. The client may close the socket to +achieve the same effect. If the server was communicating over stdio, +it may exit, but if it was communicating over a socket, it should +remain running to accept sessions from other clients. + +## Extension operations + +Each server may include support for additional ops that are not part +of the protocol. Support for these ops should be indicated by +including them in the `ops` from the `describe` op, so clients can +discover them dynamically. + +If your client wants to perform some operation that is not part of the +spec and it only needs to support a single language, it can send the +implementation of this operation across the wire using `eval`, and +indeed, many clients have done this. For example, earlier versions of +the nREPL protocol did not have a `completions` operation, and so some +clients sent Clojure code to calculate completions and parsed the +`value` reply to determine what to display to the user. However, this +is not an ideal solution; it creates incompatibilities across +languages and puts server code in the client. + +Rather than sending code across the wire for an op, you may propose a +draft extension to the nREPL protocol so that clients can standardize +on it instead. Future versions of this document may link to a list of +proposed extensions. + +[1]: https://www.oilshell.org/blog/2022/02/diagrams.html +[2]: https://langserver.org +[3]: https://wiki.theory.org/index.php/BitTorrentSpecification#Bencoding From e586ea0f7862eb36d3e51335f3423df5f2d21da9 Mon Sep 17 00:00:00 2001 From: Phil Hagelberg Date: Sun, 23 Nov 2025 10:46:48 -0800 Subject: [PATCH 2/7] Combine find op into lookup; mention describe compatibility note. --- spec.md | 62 +++++++++++++++++++++------------------------------------ 1 file changed, 23 insertions(+), 39 deletions(-) diff --git a/spec.md b/spec.md index affd7c5..35da00c 100644 --- a/spec.md +++ b/spec.md @@ -238,7 +238,7 @@ nREPL specification. // client <- server {"session": "afd3c88e-707f-4169-a265-892f29476333", "id": "5d90576e-b5e1-4499-a43d-c75c60b579ff", - "ops": ["clone", "eval", "stdin", "describe", "load-file"], + "ops": ["clone", "eval", "stdin", "describe", "load-file", "sandbox"], "features": {"encodings": ["bencode", "json"], "transports": ["socket", "stdio"]}, "versions": {"nrepl": "0.1.0", @@ -246,6 +246,10 @@ nREPL specification. "status": ["done"]} ``` +Compatibility note: previous versions of the protocol had `ops` +defined as a dictionary with the operation names as the keys and an +unspecified dictionary as the values. This is no longer recommended. + [this one could really be required; it adds very little difficulty] ### `interrupt` op @@ -290,20 +294,34 @@ information for functions and other values, the client may send a {"op": "lookup", "session": "afd3c88e-707f-4169-a265-892f29476333", "id": "d30f8bb9-4e6e-48a8-b0f8-58adf5b353a7", - "sym": "coroutine.create"} + "sym": "mymodule.reloader"} ``` The response will vary from one server to another due to language variation, but all data should be under `info`. If documentation is available, it should be under `doc`, and argument lists for functions -should be under `arglist`: +should be under `arglist`. + +If the definition of the requested value can be traced to a file, then +a `file` and `line` field should be included. A `column` may also be +provided. Line numbers are counted from 1, and column numbers are +counted from 0. If it was found inside an archive, for example a zip +file or a jar file, it should also include an `archive` field +containing a path to the archive file. If `archive` is present, then +`file` is interpreted as being inside the archive; otherwise it is +either an absolute path or interpreted as being relative to the +directory in which the server was started. ```json // client <- server {"session": "afd3c88e-707f-4169-a265-892f29476333", "id": "78f78353-c185-4211-a868-b19eaa85e054", - "info": {"doc": "Creates a new coroutine, with body `f`.", - "arglist": ["f"]}, + "info": {"doc": "Reloads the configuration.", + "arglist": ["path", "restart"], + "file": "src/mymodule.lua", + "line": 27, + "column": 2, + "archive": "lib/extras.zip"}, "status": ["done"]} ``` @@ -374,40 +392,6 @@ full text to complete, and `type` describing the candidate's type. "status": ["done"]} ``` -### `find` op - -Clients may query the server to find the location where a given value -is defined using the `find` op with a `target` field containing the -value being found. - -```json -// client -> server -{"op": "find", - "session": "afd3c88e-707f-4169-a265-892f29476333", - "id": "99746b4f-a68e-4100-a56f-bb6fe63b0767", - "target": "mymodule.reloader"} -``` - -If the value is found, the response should contain a `file` and `line` -field. A `column` may also be provided. Line numbers are counted from -1, and column numbers are counted from 0. If it was found inside an -archive, for example a zip file or a jar file, it should also include -an `archive` field containing a path to the archive file. If `archive` -is present, then `file` is interpreted as being inside the archive; -otherwise it is either an absolute path or interpreted as being -relative to the directory in which the server was started. - -```json -// client <- server -{"session": "afd3c88e-707f-4169-a265-892f29476333", - "id": "b5c90973-bf4f-4626-a825-e493eed4759a", - "file": "src/mymodule.lua", - "line": 27, - "column": 2, - "archive": "lib/extras.zip", - "status": ["done"]} -``` - ### `close` op A client may send a `close` op to terminate the session. From 3198e64385125579244e069ef148a1971daa8930 Mon Sep 17 00:00:00 2001 From: Phil Hagelberg Date: Sun, 23 Nov 2025 10:52:37 -0800 Subject: [PATCH 3/7] License. --- README.md | 8 ++++++++ _foot.html | 4 ++++ 2 files changed, 12 insertions(+) diff --git a/README.md b/README.md index 1c6cf43..8fbcca4 100644 --- a/README.md +++ b/README.md @@ -1 +1,9 @@ # nREPL Protocol Spec + +This repository contains the documentation for the nREPL protocol. In +order to avoid breaking existing links, the top-level nrepl.org domain +documents the original Clojure server implementation, and all the +general cross-language details are meant to go here. + +Except where otherwise noted, nrepl.org is licensed under the Creative +Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0). diff --git a/_foot.html b/_foot.html index 2c3ebe1..b099d6a 100644 --- a/_foot.html +++ b/_foot.html @@ -1,2 +1,6 @@ +

Except where otherwise noted, nrepl.org is licensed under the + Creative + Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) + license.

© 2025 nREPL contributors | site source

From 275cb2e12b35ba94a7a9ca2a349b47750ebe0379 Mon Sep 17 00:00:00 2001 From: Phil Hagelberg Date: Sun, 23 Nov 2025 11:01:39 -0800 Subject: [PATCH 4/7] Make describe a required op. --- spec.md | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/spec.md b/spec.md index 35da00c..7e41c63 100644 --- a/spec.md +++ b/spec.md @@ -50,8 +50,9 @@ In this document, the examples use UUID strings for `session` and ## Required Operations -The absolute minimum a client or server needs to support are the first -three ops, `clone`, `eval`, and `stdin`. +The absolute minimum a server needs to support are the first four ops, +`clone`, `eval`, `stdin`, and `describe`. Clients may support +`describe` but this is not required. ### `clone` op @@ -211,12 +212,6 @@ send what it receives using the `stdin` operation. ``` -## Optional operations - -Servers may choose to support these if they make sense. If a server -receives a request with an `op` it does not recognize, it must reply -with a message whose `status` contains `unknown-op` along with `done`. - ### `describe` op If a client wishes to know which operations are supported by a server, @@ -250,7 +245,11 @@ Compatibility note: previous versions of the protocol had `ops` defined as a dictionary with the operation names as the keys and an unspecified dictionary as the values. This is no longer recommended. -[this one could really be required; it adds very little difficulty] +## Optional operations + +Servers may choose to support these if they make sense. If a server +receives a request with an `op` it does not recognize, it must reply +with a message whose `status` contains `unknown-op` along with `done`. ### `interrupt` op From d2c7a5c082b658ff97db6bb63487f4477419bca3 Mon Sep 17 00:00:00 2001 From: Phil Hagelberg Date: Sun, 23 Nov 2025 11:12:02 -0800 Subject: [PATCH 5/7] Clarify error and close behavior. Shouldn't close the socket on close op because one socket might have multiple sessions. Explicitly document multiple sessions on one socket. --- spec.md | 28 ++++++++++++++++++++-------- 1 file changed, 20 insertions(+), 8 deletions(-) diff --git a/spec.md b/spec.md index 7e41c63..3f2a0f9 100644 --- a/spec.md +++ b/spec.md @@ -30,8 +30,8 @@ hundred lines of code. Unfortunately bencode is not particularly readable, so examples in this document will show messages in JSON. Every message sent by the client must have an `op` field, for -operation. Every message except the first must have a `session` field -indicating which session it's part of. +operation. Every message except the first `clone` request must have a +`session` field indicating which session it's part of. Every message sent by the client is a request. Requests can have one or more response messages associated with them. Because a request can @@ -46,7 +46,14 @@ features of the language, it may be possible for multiple requests to overlap. In this document, the examples use UUID strings for `session` and -`id`, but the only requirement is that they are unique strings. +`id`, but the only requirement is that they are strings that are +unique to the life of the specific server process and all clients +connecting to it. + +When an error is encountered in the nREPL server itself (rather than +in the code that the client has sent) it should send a response with +`err` containing a message describing the error, and a `status` +containing `server-error` as well as `done`. ## Required Operations @@ -80,6 +87,8 @@ included as the `session` for all subsequent requests: "status": ["done"]} ``` +The server should support multiple sessions on a single socket. + [why is this an explicit op? why not just automatically register a session any time a request comes in without a session attached?] @@ -407,11 +416,14 @@ A client may send a `close` op to terminate the session. "status": ["done"]} ``` -The server should close the socket after the reply is sent, if the -session is connected over a socket. The client may close the socket to -achieve the same effect. If the server was communicating over stdio, -it may exit, but if it was communicating over a socket, it should -remain running to accept sessions from other clients. +The server may close the socket after the reply is sent, if the +session is connected over a socket and the socket is not being used +for other sessions. The client may close the socket to achieve the +same effect. + +If the server was communicating over stdio, it may exit if no other +sessions are active, but if it was communicating over a socket, it +should remain running to accept sessions from other clients. ## Extension operations From 3a11a9cd7abfc3583702895453e2447bac0d9097 Mon Sep 17 00:00:00 2001 From: Phil Hagelberg Date: Sun, 23 Nov 2025 11:19:40 -0800 Subject: [PATCH 6/7] Be more consistent with should/may language. --- spec.md | 71 +++++++++++++++++++++++++++++++-------------------------- 1 file changed, 39 insertions(+), 32 deletions(-) diff --git a/spec.md b/spec.md index 3f2a0f9..add7ec8 100644 --- a/spec.md +++ b/spec.md @@ -30,12 +30,12 @@ hundred lines of code. Unfortunately bencode is not particularly readable, so examples in this document will show messages in JSON. Every message sent by the client must have an `op` field, for -operation. Every message except the first `clone` request must have a +operation. Every message except the first `clone` request should have a `session` field indicating which session it's part of. Every message sent by the client is a request. Requests can have one or more response messages associated with them. Because a request can -have many responses, every request should be considered active until a +have many responses, every request must be considered active until a response is received with a `status` field which is a list that contains the string `done`. Requests may remain active for a long time before completing, so the responses should be handled asynchronously. @@ -77,7 +77,7 @@ client sends a message with the `clone` op: All fields except `op` are optional. Client information may be used for debugging purposes. -The response will contain a new `new-session` id which should be +The response should contain a new `new-session` id which should be included as the `session` for all subsequent requests: ```json @@ -89,7 +89,7 @@ included as the `session` for all subsequent requests: The server should support multiple sessions on a single socket. -[why is this an explicit op? why not just automatically register a +[TODO: why is this an explicit op? why not just automatically register a session any time a request comes in without a session attached?] ### `eval` op @@ -105,8 +105,9 @@ field contains the code to be run. "code": "99 + 121"} ``` -This should normally return a message with a `value` field containing -a string representation of the return value. +In the event that the code runs successfully this should return a +message with a `value` field containing a string representation of the +return value. ```json // client <- server @@ -144,7 +145,7 @@ message should include an `ex` field instead of `value`. The format of this field will vary depending on the way the language in question represents errors. -[should we go into detail about how to send stack traces?] +[TODO: should we go into detail about how to send stack traces?] ```json // client -> server @@ -165,12 +166,12 @@ code in question should be treated as if it came from a given file. If this is not an absolute path, it should be interpreted as being relative to the directory from which the server was started. The `line` and `column` fields may include numbers indicating where in the -file the code was from. Lines start at 1 and columns start at 0. These -fields typically do not affect how the code is run, but they may help -improve stack traces if there is an error. +file the code was from. Lines should start at 1 and columns start +at 0. These fields typically do not affect how the code is run, but +they may help improve stack traces if there is an error. In some languages, evaluation always happens in the context of a -specific namespace or module. For those languages, an `ns` field can +specific namespace or module. For those languages, an `ns` field may be included in the request which indicates the namespace to evaluate the code in. @@ -189,7 +190,7 @@ the code in. ### `stdin` op In the case that evaluated code tries to read input from standard in, -the server will send a message to the client with a status of +the server should send a message to the client with a status of `need-input`. When this happens, the client should accept input and send what it receives using the `stdin` operation. @@ -224,7 +225,7 @@ send what it receives using the `stdin` operation. ### `describe` op If a client wishes to know which operations are supported by a server, -it can query with the `describe` op, which takes no other parameters: +it may query with the `describe` op, which takes no other parameters: ```json // client -> server @@ -338,10 +339,12 @@ If the `sym` is not found, then the `info` field should be omitted. ### `load-file` op A client may instruct the server to load an entire file instead of -sending its contents across the session. It may send a request with a -`load-file` op which has `file-path` indicating the path to the file -to load. Depending on the server, in some cases this may result in the -loaded code having better stack trace information. +sending its contents across the session. This can be especially useful +if the server is running on a different machine from the client. It +may send a request with a `load-file` op which has `file-path` +indicating the path to the file to load. Depending on the server, in +some cases this may result in the loaded code having better stack +trace information. If `file-path` is not an absolute path, it should be interpreted as being relative to the directory from which the server was started. @@ -355,7 +358,7 @@ being relative to the directory from which the server was started. ``` The response should be interpreted similarly to the `eval` op: a -`value` or `err` may be included, but omitting both is also allowed. +`value` or `ex` may be included, but omitting both is also allowed. It may also include `out` and/or `err`. ```json @@ -374,9 +377,12 @@ longer recommended. A client may request completions for a given input using the `completions` op. The `prefix` field should be a string indicating the -input to be completed. For servers where completions may be specific -to a module or namespace context, an `ns` field may also be included -indicating this. +input to be completed. + +For servers where completions may be specific to a module or namespace +context, an `ns` field may also be included indicating this. The +client may also include `file`, `line`, and `column` which the server +may use to get better context to provide completions. ```json // client -> server @@ -387,8 +393,9 @@ indicating this. ``` The response should contain a `completions` field with a list of -completion candidates: dictionaries with fields `candidate` with the -full text to complete, and `type` describing the candidate's type. +completion candidates: dictionaries with a `candidate` field with the +full text to complete, and optionally a `type` field describing the +candidate's type. ```json // client <- server @@ -428,9 +435,9 @@ should remain running to accept sessions from other clients. ## Extension operations Each server may include support for additional ops that are not part -of the protocol. Support for these ops should be indicated by -including them in the `ops` from the `describe` op, so clients can -discover them dynamically. +of this protocol specification. Support for these ops should be +indicated by including them in the `ops` from the `describe` op so +clients can discover them dynamically. If your client wants to perform some operation that is not part of the spec and it only needs to support a single language, it can send the @@ -439,13 +446,13 @@ indeed, many clients have done this. For example, earlier versions of the nREPL protocol did not have a `completions` operation, and so some clients sent Clojure code to calculate completions and parsed the `value` reply to determine what to display to the user. However, this -is not an ideal solution; it creates incompatibilities across -languages and puts server code in the client. +is discouraged as it creates incompatibilities across languages and +puts functionality that belongs to the server in the client. -Rather than sending code across the wire for an op, you may propose a -draft extension to the nREPL protocol so that clients can standardize -on it instead. Future versions of this document may link to a list of -proposed extensions. +Rather than sending code across the wire for an op, implementers may +propose a draft extension to the nREPL protocol so that clients can +standardize on it instead. Future versions of this document may link +to a list of proposed extensions. [1]: https://www.oilshell.org/blog/2022/02/diagrams.html [2]: https://langserver.org From 882985d57e21a2045f29df7afcd609903734c6ef Mon Sep 17 00:00:00 2001 From: Phil Hagelberg Date: Fri, 28 Nov 2025 14:19:59 -0800 Subject: [PATCH 7/7] Address review feedback. * Make clone/session optional. * Clarify how sessions affect serialization and thread-local state. * Send `describe` immediately upon connecting. * Accept `client` in `describe` request. * Remove `features` from `describe` response. * Not all output messages must have `id`. --- spec.md | 206 +++++++++++++++++++++++++------------------------------- 1 file changed, 91 insertions(+), 115 deletions(-) diff --git a/spec.md b/spec.md index add7ec8..97c725a 100644 --- a/spec.md +++ b/spec.md @@ -29,9 +29,7 @@ deal easier to implement than JSON; usually it can be done in under a hundred lines of code. Unfortunately bencode is not particularly readable, so examples in this document will show messages in JSON. -Every message sent by the client must have an `op` field, for -operation. Every message except the first `clone` request should have a -`session` field indicating which session it's part of. +Every message sent by the client must have an `op` field, for operation. Every message sent by the client is a request. Requests can have one or more response messages associated with them. Because a request can @@ -41,14 +39,11 @@ contains the string `done`. Requests may remain active for a long time before completing, so the responses should be handled asynchronously. Requests should have an `id` field, and responses to that request -should also include the same `id`. Depending on the concurrency -features of the language, it may be possible for multiple requests to -overlap. +should also include the same `id`. -In this document, the examples use UUID strings for `session` and -`id`, but the only requirement is that they are strings that are -unique to the life of the specific server process and all clients -connecting to it. +In this document, the examples use UUID `id` strings, but the only +requirement is that they are strings that are unique to the life of +the specific server process and all clients connecting to it. When an error is encountered in the nREPL server itself (rather than in the code that the client has sent) it should send a response with @@ -57,62 +52,60 @@ containing `server-error` as well as `done`. ## Required Operations -The absolute minimum a server needs to support are the first four ops, -`clone`, `eval`, `stdin`, and `describe`. Clients may support -`describe` but this is not required. +The absolute minimum a server needs to support are the first three ops, +`describe`, `eval`, and `stdin`. -### `clone` op +### `describe` op -Messages are exchanged in sessions. In order to start a session, the -client sends a message with the `clone` op: +The client should send a `describe` request upon connecting so it +knows which ops the server supports. Servers should not require clients +to make this request first. -```json +```js // client -> server -{"op": "clone", - "id": "01e0bbed-2819-41b8-9642-4c16a79f7efc", - "client-name": "nREPL documentation demo", - "client-version" "1.0.0"} +{"op": "describe", + "id": "5d90576e-b5e1-4499-a43d-c75c60b579ff" + "client": "nREPL documentation demo 1.0.0"} ``` -All fields except `op` are optional. Client information may be used -for debugging purposes. +The `client` field may be used for debugging purposes, but the server +should not treat the client differently based on its name or version. -The response should contain a new `new-session` id which should be -included as the `session` for all subsequent requests: +The response must have a list of `ops` supported by the server. It may +also have a dictionary of `versions` for debugging purposes as well. -```json +```js // client <- server -{"new-session": "afd3c88e-707f-4169-a265-892f29476333", - "id": "01e0bbed-2819-41b8-9642-4c16a79f7efc", +{"id": "5d90576e-b5e1-4499-a43d-c75c60b579ff", + "ops": ["clone", "eval", "stdin", "describe", "load-file", "sandbox"], + "versions": {"nrepl": "0.1.0", + "lua": "5.4"}, "status": ["done"]} ``` -The server should support multiple sessions on a single socket. - -[TODO: why is this an explicit op? why not just automatically register a -session any time a request comes in without a session attached?] +Compatibility note: previous versions of the protocol had `ops` +defined as a dictionary with the operation names as the keys and an +unspecified dictionary as the values. This is no longer recommended. ### `eval` op This is the main workhorse operation where code gets run. The `code` field contains the code to be run. -```json +```js // client -> server {"op": "eval", - "session": "afd3c88e-707f-4169-a265-892f29476333", "id": "297a1dc1-e8ea-4a71-ac58-977841a301f4", "code": "99 + 121"} ``` In the event that the code runs successfully this should return a message with a `value` field containing a string representation of the -return value. +return value or values. -```json +```js // client <- server -{"session": "afd3c88e-707f-4169-a265-892f29476333", - "id": "297a1dc1-e8ea-4a71-ac58-977841a301f4", +{"id": "297a1dc1-e8ea-4a71-ac58-977841a301f4", "value": "220", "status": ["done"]} ``` @@ -125,38 +118,38 @@ message. The client should display these to the user in a way that makes it clear they are part of the session; for example, in the editor console right below the code was entered. -```json +```js // client -> server {"op": "eval", - "session": "afd3c88e-707f-4169-a265-892f29476333", "id": "35e53d19-9f4a-4329-a820-d71481fdfec1", "code": "print('hello, world')"} // client <- server -{"session": "afd3c88e-707f-4169-a265-892f29476333", - "id": "35e53d19-9f4a-4329-a820-d71481fdfec1", +{"id": "35e53d19-9f4a-4329-a820-d71481fdfec1", "value": "nil", "out": "hello world", "status": ["done"]} ``` +The server should make an effort to send `out` messages with an `id` +field where possible; however, it may be unavoidable to have some +output which is not tied to a specific request, so clients should be +prepared for output to arrive which is missing an `id` field or +contains an `id` for a request that is already considered done. + In the case that evaluated code encounters an error, the response message should include an `ex` field instead of `value`. The format of this field will vary depending on the way the language in question represents errors. -[TODO: should we go into detail about how to send stack traces?] - -```json +```js // client -> server {"op": "eval", - "session": "afd3c88e-707f-4169-a265-892f29476333", "id": "b9616f31-9fbd-4a76-b7d6-ab98eb9f7641", "code": "client.connect(config.hostname, config.port)"} // client <- server -{"session": "afd3c88e-707f-4169-a265-892f29476333", - "id": "b9616f31-9fbd-4a76-b7d6-ab98eb9f7641", +{"id": "b9616f31-9fbd-4a76-b7d6-ab98eb9f7641", "ex": "connection refused", "status": ["done"]} ``` @@ -175,10 +168,9 @@ specific namespace or module. For those languages, an `ns` field may be included in the request which indicates the namespace to evaluate the code in. -```json +```js // client -> server {"op": "eval", - "session": "afd3c88e-707f-4169-a265-892f29476333", "id": "2102c017-2ff5-4ddd-9067-a54ec62fc0c8", "file": "src/display/avatar.lua", "line": 21, @@ -194,70 +186,32 @@ the server should send a message to the client with a status of `need-input`. When this happens, the client should accept input and send what it receives using the `stdin` operation. -```json +```js // client -> server {"op": "eval", - "session": "afd3c88e-707f-4169-a265-892f29476333", "id": "78f78353-c185-4211-a868-b19eaa85e054", "code": "subsystem.activate()"} // client <- server -{"session": "afd3c88e-707f-4169-a265-892f29476333", - "id": "78f78353-c185-4211-a868-b19eaa85e054", +{"id": "78f78353-c185-4211-a868-b19eaa85e054", "out": "Username: ", "status": ["need-input"]} // client -> server {"op": "stdin", - "session": "afd3c88e-707f-4169-a265-892f29476333", "id": "78f78353-c185-4211-a868-b19eaa85e054", "stdin": "gorkon"} // client <- server -{"session": "afd3c88e-707f-4169-a265-892f29476333", - "id": "78f78353-c185-4211-a868-b19eaa85e054", +{"id": "78f78353-c185-4211-a868-b19eaa85e054", "output": "Activated.\n", "value": "nil", "status": ["done"]} - ``` -### `describe` op - -If a client wishes to know which operations are supported by a server, -it may query with the `describe` op, which takes no other parameters: - -```json -// client -> server -{"op": "describe", - "session": "afd3c88e-707f-4169-a265-892f29476333", - "id": "5d90576e-b5e1-4499-a43d-c75c60b579ff"} -``` - -The response must have a list of `ops` supported by the server. It may -also have a dictionary of `versions` for debugging purposes as well as -a dictionary of `features` describing additional extensions beyond the -nREPL specification. - -```json -// client <- server -{"session": "afd3c88e-707f-4169-a265-892f29476333", - "id": "5d90576e-b5e1-4499-a43d-c75c60b579ff", - "ops": ["clone", "eval", "stdin", "describe", "load-file", "sandbox"], - "features": {"encodings": ["bencode", "json"], - "transports": ["socket", "stdio"]}, - "versions": {"nrepl": "0.1.0", - "lua": "5.4"}, - "status": ["done"]} -``` - -Compatibility note: previous versions of the protocol had `ops` -defined as a dictionary with the operation names as the keys and an -unspecified dictionary as the values. This is no longer recommended. - ## Optional operations -Servers may choose to support these if they make sense. If a server +Clients and servers may choose to support these if they make sense. If a server receives a request with an `op` it does not recognize, it must reply with a message whose `status` contains `unknown-op` along with `done`. @@ -269,26 +223,23 @@ send an `interrupt` op. Requests may optionally contain an be interrupted. If this is omitted, it interrupts the most recent request of the current session. -```json +```js // client -> server {"op": "eval", - "session": "afd3c88e-707f-4169-a265-892f29476333", "id": "71629c7e-6c73-4dea-85f8-102d4b64c07f", "code": "calculate_matrix()"} // client -> server {"op": "interrupt", - "session": "afd3c88e-707f-4169-a265-892f29476333", "id": "71629c7e-6c73-4dea-85f8-102d4b64c07f""} ``` The reply to this request should be a message with statuses `interrupted` and `done` both: -```json +```js // client <- server -{"session": "afd3c88e-707f-4169-a265-892f29476333", - "id": "78f78353-c185-4211-a868-b19eaa85e054", +{"id": "78f78353-c185-4211-a868-b19eaa85e054", "status": ["interrupted" "done"]} ``` @@ -298,10 +249,9 @@ For servers that support providing documentation and reflective information for functions and other values, the client may send a `lookup` op containing a `sym` field for the item being looked up. -```json +```js // client -> server {"op": "lookup", - "session": "afd3c88e-707f-4169-a265-892f29476333", "id": "d30f8bb9-4e6e-48a8-b0f8-58adf5b353a7", "sym": "mymodule.reloader"} ``` @@ -321,10 +271,9 @@ containing a path to the archive file. If `archive` is present, then either an absolute path or interpreted as being relative to the directory in which the server was started. -```json +```js // client <- server -{"session": "afd3c88e-707f-4169-a265-892f29476333", - "id": "78f78353-c185-4211-a868-b19eaa85e054", +{"id": "78f78353-c185-4211-a868-b19eaa85e054", "info": {"doc": "Reloads the configuration.", "arglist": ["path", "restart"], "file": "src/mymodule.lua", @@ -339,7 +288,7 @@ If the `sym` is not found, then the `info` field should be omitted. ### `load-file` op A client may instruct the server to load an entire file instead of -sending its contents across the session. This can be especially useful +sending its contents across the wire. This can be especially useful if the server is running on a different machine from the client. It may send a request with a `load-file` op which has `file-path` indicating the path to the file to load. Depending on the server, in @@ -349,10 +298,9 @@ trace information. If `file-path` is not an absolute path, it should be interpreted as being relative to the directory from which the server was started. -```json +```js // client -> server {"op": "load-file", - "session": "afd3c88e-707f-4169-a265-892f29476333", "id": "b5c90973-bf4f-4626-a825-e493eed4759a", "file-path": "src/utils.lua"} ``` @@ -361,10 +309,9 @@ The response should be interpreted similarly to the `eval` op: a `value` or `ex` may be included, but omitting both is also allowed. It may also include `out` and/or `err`. -```json +```js // client <- server -{"session": "afd3c88e-707f-4169-a265-892f29476333", - "id": "b5c90973-bf4f-4626-a825-e493eed4759a", +{"id": "b5c90973-bf4f-4626-a825-e493eed4759a", "value": "{debug=#}", "status": ["done"]} ``` @@ -384,10 +331,9 @@ context, an `ns` field may also be included indicating this. The client may also include `file`, `line`, and `column` which the server may use to get better context to provide completions. -```json +```js // client -> server {"op": "completions", - "session": "afd3c88e-707f-4169-a265-892f29476333", "id": "d862f516-a232-4e01-a4c1-1afb42e04637", "prefix": "math.s"} ``` @@ -397,21 +343,51 @@ completion candidates: dictionaries with a `candidate` field with the full text to complete, and optionally a `type` field describing the candidate's type. -```json +```js // client <- server -{"session": "afd3c88e-707f-4169-a265-892f29476333", - "id": "d862f516-a232-4e01-a4c1-1afb42e04637", +{"id": "d862f516-a232-4e01-a4c1-1afb42e04637", "completions": [{"candidate": "math.sqrt", "type": "function"}, {"candidate": "math.sin", "type": "function"}, {"candidate": "math.sinh", "type": "function"}], "status": ["done"]} ``` +### `clone` op + +Some servers may wish to implement **sessions**. Requests and +responses may be sent inside a session, which has two effects: all +requests within one session are run in serial; that is, no request can +be accepted until the previous one has completed. In addition, +thread-local state (if applicable) should be kept consistent across a +session. + +In order to start a session, the client sends a message with the `clone` op: + +```js +// client -> server +{"op": "clone", + "id": "01e0bbed-2819-41b8-9642-4c16a79f7efc"} +``` + +If the request contains a `session` field, then that session is cloned +to make the requested session and all its state is passed on; if not +then the new session is not connected to an existing one. + +The response should contain a new `new-session` id which should be +included as the `session` for all subsequent requests: + +```js +// client <- server +{"new-session": "afd3c88e-707f-4169-a265-892f29476333", + "id": "01e0bbed-2819-41b8-9642-4c16a79f7efc", + "status": ["done"]} +``` + ### `close` op A client may send a `close` op to terminate the session. -```json +```js // client -> server {"op": "close", "session": "afd3c88e-707f-4169-a265-892f29476333",