Note: A red circle 🔴 marks internal notes which won't appear in the final version.
The Ride protocol is formed of messages sent in either direction over a TCP connection.
A message starts with a 4-byte big-endian total length field, followed by the ASCII bytes for "Ride"
and a
UTF-8-encoded payload:
8+len(payload) "Ride" magic number payload
┌───────────────────┬───────────────────┬─────~─────┐
│0x00 0x00 0x00 0x0b│0x52 0x49 0x44 0x45│ ... │
└───────────────────┴───────────────────┴─────~─────┘
Total length is 8 + the payload's length in bytes. The payload is almost always a 2-element JSON array consisting of a command name and arguments as key/value pairs:
["CommandName",{"key1":"value1","key2":222,"key3":[3,4,5]}]
The only exception are the first two messages that each side sends upon establishing a connection. These constitute the handshake and are not JSON-encoded. Their payloads are:
SupportedProtocols=2
UsingProtocol=2
Messages are independent and after the handshake can be sent/received in any order. Some messages infer that the other end will send a reply, but that reply may not be the next message to be received, or even ever be sent.
If the receiver of a message does not recognise it, it should respond with
["UnknownCommand",{"name":"Xyz"}]
Should the interpreter generate an error during the processing of an incoming Ride message it will respond with an InternalError message:
["InternalError",{"error":1,"error_text":"WS FULL","dmx":"","message":"Edit"}]
error
: aka ⎕ENerror_text
: aka ⎕EMdmx
: the DMX message for the error (currently always empty)message
: the name of the originating incoming Ride message
The connection may be closed at any time, leaving some messages undelivered or unprocessed.
Command names and their arguments are case-sensitive.
JSON booleans true
and false
can be freely substituted with and should be treated as equivalent to 1
and 0
.
After the connection has been established and a protocol agreed, both peers immediately send an Identify
message to indicate what type of application they are.
["Identify",{"apiVersion":1,"identity":1}]
The apiVersion
is introduced as a mechanism to handle breaking changes in the API.
Constants for identity
:
1
Ride,2
interpreter,3
process manager.
🔴 The interpreter sends an Identify
that means something else and has different args.
The interpreter responds with an Identify messsage containing details about the interpreter.
The interpreter responds with a ReplyIdentify
messsage containing details about the interpreter.
They should then check the type of application they are connected to, and if not happy to continue, close the connection. For instance, Ride may check that the application it's connected to is an interpreter or a process manager. If it finds the peer is another Ride, it should close the connection.
🔴 In reality Ride doesn't bother verifying that it's not talking to another Ride.
The interpreter responds with a ReplyIdentify
messsage containing details about the interpreter.
["ReplyIdentify",{ // Interpreter -> Ride
"apiVersion":1,
"Port":0,
"IPAddress":"",
"Vendor":"Dyalog Limited",
"Language":"APL",
"version":"18.2.46943",
"Machine":"HOSTNAME",
"arch":"Unicode/64",
"Project":"CLEAR WS",
"Process":"dyalog.exe",
"User":"username",
"pid":1000,
"token":"{F53F7747-B404-4F7A-893D-954F2764CC74}",
"date":"Created: Apr 10 2023 at 19:29:32",
"platform":"Windows-64"
}]
The apiVersion
specifies the negotiated API version accepted by the interpreter.
Request the session log from the interpreter.
["GetLog",{"format":"json","maxLines":100}] // Ride -> Interpreter
format
defaults to "json" if not specified.
maxLines
limits the number of lines returned (-1 = unlimited)
After it has received the Connect command the interpreter will send 0 or more "ReplyGetLog" messages containing the session log:
["ReplyGetLog",{"result":["line 1","line 2"]}] // Interpreter -> Ride
After it has received a "GetLog" command the interpreter will send 0 or more "ReplyGetLog" messages containing the session log in the format requested.
For text
format:
["ReplyGetLog",{"result":["line 1","line 2"]}] // Interpreter -> Ride
For json
format:
["ReplyGetLog",{"result":[ // Interpreter -> Ride
{
"group": 1,
"type": 1,
"text": "line 1",
},{
"group": 1,
"type": 1,
"text": "line 2",
},
]}]
If at any time the interpreter crashes with a syserror, it sends;
["SysError",{"text":"apl: sys error 123 errno 456","stack":""}] // Interpreter -> Ride
If the interpreter has been started by Ride, Ride should shut it down cleanly when the user closes the main application window (the session window):
["Exit",{"code":0}] // Ride -> Interpreter
Any echoed input or interpreter output are sent to Ride using either;
["EchoInput",{"input":" 1 2 3+4 5 6\n"}] // Interpreter -> Ride
["AppendSessionOutput",{"result":"5 7 9"}] // Interpreter -> Ride
These two perform essentially the same task except that AppendSessionOutput
doesn't necessarily have trailing "\n"
-s at the end of result
.
Any echoed input or interpreter output are sent to Ride using either;
["AppendSessionOutput",{"result":"5 7 9","type":1,"group":1}] // Interpreter -> Ride
type
specifies the source of the output:
type | description |
---|---|
0 | reserved |
1 | undetermined session output |
2 | default output e.g. not assigned to ⎕ or ⍞ |
3 | what would have been always sent to stderr |
4 | output from system commands |
5 | APL error message. e.g. ^ and DOMAIN ERROR |
6 | reserved |
7 | ⎕ output |
8 | ⍞ output |
9 | "information" that would have gone to the "status window" |
10 | reserved |
11 | echoed input |
12 | ⎕TRACE output |
13 | reserved |
14 | A “normal” input line |
The interpreter informs Ride about changes in its ability to accept user input with
["SetPromptType",{"type":5}] // Interpreter -> Ride
Constants for type
:
0
no prompt,1
the usual 6-space APL prompt (a.k.a. Descalc or "desktop calculator"),2
Quad(⎕
) input,3
line editor,4
Quote-Quad(⍞
) input,5
any prompt type unforeseen here.
🔴 These modes need explaining with expected behaviour.
When the user presses <ER>
(Enter) or <TC>
(Ctrl-Enter), Ride sends
["Execute",{"text":" 1 2 3+4 5 6","trace":1}] // Ride -> Interpreter
text
: the APL code to evaluatetrace
: 0 or 1, whether the expression should be evaluated in the tracer (<TC>
)
Note that Ride can't assume that everything entered in the session will be echoed, e.g. quote quad input (⍞
) doesn't
echo. Therefore, Ride should wait for the EchoInput
message.
If multiple lines have been modified in the session, Ride should queue them up and send them one by one, waiting for
a response of either SetPromptType
with type>0
or HadError.
Ride should clear its queue of pending lines on HadError
and focus the session.
["HadError",{}] // Interpreter -> Ride
Ride can optionally advise the interpreter about the session's width in characters with
["SetPW",{"pw":79}] // Ride -> Interpreter
Further output will wrap at that width (with a few exceptions).
See ⎕PW
.
When the user presses <ED>
(Shift-Enter), Ride should send;
["Edit",{"win":123,"text":"a←b+c×d","pos":4,"unsaved":{"124":"f"}}] // Ride -> Interpreter
to request opening an editor. pos
is the 0-based position of the cursor in text
.
unsaved
is a mapping from window ids to unsaved content.
🔴 "Edit" must be extended to submit the current content of all dirty windows, otherwise jumping from one method to another in a class will obliterate the current changes.
The interpreter will parse that and may respond later with one of;
["OpenWindow",{"name":"f","filename":"C:\\path\\to\\foo.txt","text":["r←f a","r←(+⌿÷≢)a"],"token":123,"currentRow":0,"debugger":false,
"entityType":1,"offset":0,"readOnly":false,"size":0,"stop":[1],
"tid":0,"tname":"Tid:0"}] // Interpreter -> Ride
["UpdateWindow",...] // Interpreter -> Ride (same args as OpenWindow)
It may also send these in response to )ed name
or
⎕ed'name'
, as well as when tracing into an
object that is not currently being traced.
Constants for entityType
:
entityType | Description | entityType | Description | |
---|---|---|---|---|
1 |
defined function | 128 |
simple character vector | |
2 |
simple character array | 256 |
APL namespace | |
4 |
simple numeric array | 512 |
APL class | |
8 |
mixed simple array | 1024 |
APL interface | |
16 |
nested array | 2048 |
APL session | |
32 |
⎕OR object |
4096 |
external function. | |
64 |
native file |
🔴 TODO: describe the other arguments
The interpreter can request transferring the focus to a particular window with;
["GotoWindow",{"win":123}] // Interpreter -> Ride
This could happen as a result of )ED
or ⎕ED
.
The interpreter may decide to change the type of a window (editor vs tracer) with;
["WindowTypeChanged",{"win":123,"tracer":true}] // Interpreter -> Ride
When the user presses <EP>
(Esc), Ride should request that the editor contents are fixed through;
["SaveChanges",{"win":123,"text":["r←avg a","s←+⌿a","n←≢a","r←s÷n"],"stop":[2,3]}] // Ride -> Interpreter
stop
is an array of 0-based line numbers.
["ReplySaveChanges",{"win":123,"err":0}] // Interpreter -> Ride
If err
is 0, save succeeded; otherwise it failed.
Ride can request that the intepreter reformat code:
["FormatCode",{"win":123,"text":["r←avg a","s←+⌿a","n ← ≢a","r←s÷n"]}] // Ride -> Interpreter
["ReplyFormatCode",{"win":123,"text":["r←avg a","s←+⌿a","n←≢a","r←s÷n"]}] // Interpreter -> Ride
win
: TENTATIVE: a window identifer. The interpreter needs a window in which to format the code (don't ask!). In the short term we'll insist that we can only format code in a window the interpreter is aware of.
When the user presses <EP>
(Esc) and saving is successful or presses <QT>
(Shift-Esc), Ride sends;
["CloseWindow",{"win":123}] // Ride -> Interpreter and Interpreter -> Ride
but does not close the UI window until the interpreter replies with the same message.
To close all windows, but leave the SIstack unchanged Ride can send the CloseAllWindows message.
["CloseAllWindows",{}] // Ride -> Interpreter
In response the interpreter will send a CloseWindow messsage for each window that it is aware of. The CloseAllWindows message will leave the SIStack unchanged, it will just close all (trace and edit) windows in the interpreter.
The following messages are used in relation to trace windows.
This tells Ride where the currently executed line is. Traditionally that's indicated by a red border around it.
["SetHighlightLine",{"win":123,"line":45}] // Interpreter -> Ride
Update the breakpoints.
["SetLineAttributes",{"win":123,"stop":[2,3,5]}] // Ride -> Interpreter or Interpreter -> Ride
stop
is an array of 0-based line numbers.
Request the current line in a trace window be moved back (skip back one line).
["TraceBackward",{"win":123}] // Ride -> Interpreter
Request it clears all traces, stops, and monitors in the active workspace. The reply says how many of each thing were cleared.
["ClearTraceStopMonitor",{"token":123}] // Ride -> Interpreter
["ReplyClearTraceStopMonitor",{"traces":0,"stops":0,"monitors":0,"token":123}] // Interpreter -> Ride
Request resume execution of the current thread.
["Continue",{"win":123}] // Ride -> Interpreter
Request resume execution of the current function, but stop on the next line of the calling function.
["ContinueTrace",{"win":123}] // Ride -> Interpreter
Request the stack is cut back one level. This is equivalent to returning to the caller without executing the rest of the current function.
["Cutback",{"win":123}] // Ride -> Interpreter
Request the current line in a trace window be moved forward (skip to next line).
["TraceForward",{"win":123}] // Ride -> Interpreter
Request resume execution of all threads.
["RestartThreads",{}] // Ride -> Interpreter
Request the current line in a trace window is executed. (Step over)
["RunCurrentLine",{"win":123}] // Ride -> Interpreter
["StepInto",{"win":123}] // Ride -> Interpreter
Request the current line in a trace window is executed. (Step into)
Ride requests status information from the interpreter to display in the status bar.
Ride 4.4 now uses the Subscribe method to retrieve information from the interpreter for the status bar. Protocol messages are sent by the interpreter when a change is detected, as opposed to polling at an interval.
["Subscribe", { // Ride -> Interpreter
"status": [ // 0 or more of the following values
// values are not saved between calls
"statusfields", // will result in InterpreterStatus messages
"stack" , // will result in ReplyGetSIStack messages
"threads" // will result in ReplyGetSIStack messages
],
"heartbeat": interval // interval is currently ignored
}]
There is no unsubscribe method, a new Subscribe message should be sent with the relevant fields removed.
["InterpreterStatus", { // Interpreter -> Ride
"IO": int, // current ⎕IO
"DQ": int, // length of current message queue
"WA": int, // current available workspace (not currently implemented)
"SI": int, // length of current SI stack
"TRAP": int, // Is there an active trap?
"ML": int, // // current ⎕ML
"NumThreads": int, // current number of threads
"TID": int, // current thread id
"CompactCount": int, // number of compactions so far
"GarbageCount": int // number of garbage collections so far (i.e. number of collections that have found garbage)
}]
["InterpreterHeartBeat", { // Interpreter -> Ride
"ping" : "ping" // maybe there will be additional reasons to ping
}]
Request information about the current stack.
["GetSIStack",{}] // Ride -> Interpreter
["ReplyGetSIStack",{"stack":[{"description":"#.f[12]*"},{"description":"#.g[34]"}],"tid":2}] // Interpreter -> Ride
Get information about the current threads.
["GetThreads",{}] // Ride -> Interpreter
["ReplyGetThreads",{"threads":[
{"description":"","state":"Session","tid":0,"flags":"Normal","Treq":""},
]}] // Interpreter -> Ride
description
: a text description of the thread. Derived from the Tid and ⎕TNAME for the threadstate
: a string indicating the current location of the threadtid
: the Tid (numeric)flags
: e.g. Normal, Paused or TerminatedTreq
: a string indicating any tokens that the thread is waiting for.
Request the interpreter focus a specific thread.
["SetThread", {"tid":123}] // Ride -> Interpreter
["ReplySetThread", {"tid":123, "rc":321, "message":"txt"}] // Interpreter -> Ride
tid
: the thread ID (numeric)rc
: Return code. TID of focused thread, or -1 if unsuccessful.message
: Empty, or text description of the result if unsuccessful.
Request attributes on multiple threads:
["GetThreadAttributes",{ // Ride -> Interpreter
"threads":[123 | -1]
}]
If first item is -1, return info for all threads and stop processing. If -1 is found after first, return non zero in rc for that element.
["ReplyGetThreadAttributes",{ // Interpreter -> Ride
"threads":[{
"tid": 123,
"rc": 321,
"paused": 1,
"noninterruptable": 2,
}]
}]
tid
: The thread ID (numeric)rc
: Return code. TID of thread, or -1 if unsuccessful.paused
: booleannoninterruptable
: int;0
: interruptable,1
: non-interruptable,2
: children will be created as non-interruptable
Set attributes on multiple threads.
["SetThreadAttributes",{ // Ride -> Interpreter
"threads":[{
"tid":123,
"paused":0,
"noninterruptable": 2,
}]
}]
If first item's tid is -1, set info for all threads and stop processing. The interpreter will respond with ReplySetThreadAttributes
["ReplySetThreadAttributes",{ // Interpreter -> Ride
"threads":[{
"tid":123,
"rc":0,
"paused":0,
"noninterruptable":2,
}]
}]
tid
: The thread ID (numeric)rc
: Return code. TID of thread, or -1 if unsuccessful.paused
: booleannoninterruptable
: int;0
: interruptable,1
: noninterruptable,2
: children will be created as non-interruptable
To pause all threads, pass a 1. To unpause all paused threads, pass a 0.
["PauseAllThreads", { "pause": 1 | 0}]
PauseAllThreads (pause=0) does not "restart" all threads, you'll need to send RestartThreads if that's what you want. PauseAllThreads does not send any response.
APL supports two kinds of interrupts;
["WeakInterrupt", {}] // Ride -> Interpreter
["StrongInterrupt",{}] // Ride -> Interpreter
The interpreter message queue should check for strong interrupts and handle them immediately without needing to fully parse messages.
🔴 I've no idea what the above sentence means -Nick
Ride can request autocompletion information from the interpreter.
["GetAutocomplete",{"line":"r←1+ab","pos":6,"token":234}] // Ride -> Interpreter
line
: text containing the name that's being completedpos
: position of cursor withinline
token
: is used byReplyGetAutocomplete
to identify which request it is a response to. Ride may send multipleGetAutocomplete
requests and the interpreter may only reply to some of them. Similarly, Ride may ignore some of the replies if the state of the editor has changed since theGetAutocomplete
request was sent. In order to remain responsive, Ride should throttle its autocompletion requests (no more than N per second) and it shouldn't block while it's waiting for the response.
🔴 The interpreter requires that "token" is the id of the window, so perhaps it should be renamed "win".
🔴 If Ride sends a different token, the interpreter doesn't respond.
["ReplyGetAutocomplete",{"skip":2,"options":["ab","abc","abde"],"token":234}] // Interpreter -> Ride
skip
: how many characters before the request'spos
to replace with an element ofoptions
When the user hovers a name with the mouse, Ride should ask for a short textual representation of the current value:
["GetValueTip",{"win":123,"line":"a←b+c","pos":2,"maxWidth":50,"maxHeight":20,"token":456}] // Ride -> Interpreter
["ValueTip",{"tip":["0 1 2","3 4 5"],"class":2,"startCol":2,"endCol":3,"token":456}] // Interpreter -> Ride
token
: is used to correlate requests and responses, and there is no guarantee that they will arrive in the same order, if ever (like with autocompletion).maxHeight
andmaxWidth
can be used to limit the number of lines and columns in the result.class
indicates the nameclass of the object. This information can be used to syntax-highlight the tooltip.startCol
andendCol
describe the position of the whole name to which the value tip pertains.startCol
is inclusive andendCol
is exclusive.
The interpreter can ask Ride to interact with the user by showing a modal dialog. Several kinds of dialogs are supported:
["OptionsDialog",{"title":"","text":"","type":1,"options":["Yes","No","Cancel"],"token":123}] // Interpreter -> Ride
["ReplyOptionsDialog",{"index":0,"token":123}] // Ride -> Interpreter
Constants for type:
1
warning,2
information,3
question,4
stop.
If the user closes the dialog without choosing an option, Ride responds with an index
of -1
.
["StringDialog",{"title":"Name","text":"Please enter a name:","initialValue":"abc","defaultValue":null,"token":123}] // Interpreter -> Ride
["ReplyStringDialog",{"value":"abcd","token":123}] // Ride -> Interpreter
A "task dialog" shows two sets of buttons -- vertically aligned buttonText
and below them the horizontally aligned options
.
["TaskDialog",{"title":"Save document","text":"Save document options",
"subtext":"Do you want to save the changes to the document?",
"buttonText":["Save in XML base format","Save in binary format"],
"options":["No","Cancel"],
"footer":"Note: If you don't choose to save, your changes will be lost",
"questionkey":"SaveFileOptionsExtension:.xml",
"questionlabel":"Save this response for all files with a \".xml\" extension"}] // Interpreter -> Ride
["ReplyTaskDialog",{"index":"101","token":123}] // Ride -> Interpreter
In the response index
can be:
100+i
wherei
is the index of abuttonText
button- the index of an
options
button -1
if the user closes the dialog
["NotificationMessage",{"message":"Object too large to edit","token":123}] // Interpreter -> Ride
Request Ride shows some HTML. See 3500⌶
.
["ShowHTML",{"title":"Example","html":"<i>Hello</i> <b>world</b>"}] // Interpreter -> Ride
This message is sent by the interpreter when WSID is changed.
["UpdateDisplayName",{"displayName":"CLEAR WS"}] // Interpreter -> Ride
Ride can use the display name as the title of its application window.
["UpdateSessionCaption",{"text":"CLEAR WS - Dyalog APL/W-64"}] // Interpreter -> Ride
Sent from any peer to shut down the connection cleanly.
["Disconnect",{"message":"..."}]
🔴 Why do we need "Disconnect"? Why not just close the TCP connection? That shouldn't be any less "clean".
Optionally, Ride can display a tree representing session content.
It can query information about the children of a particular node with TreeList.
["TreeList",{"nodeId":12}] // Ride -> Interpreter
["ReplyTreeList",{"nodeId":12,"nodeIds":[34,0],"names":["ab","cde"],
"classes":[9.4,3.2],"err":""}] // Interpreter -> Ride
The root of the tree is assumed to have a node id of 0.
nodeId
is the requested parent id.nodeIds
are the ids of the children; some of them can be 0 -- those children can't themselves have children.classes
are name classes that can be used to choose appropriate stylingerr
is non-empty only when an error has occurred in the interpreter, e.g. whennodeId
is no longer invalid
Ride should query information only about the visible parts of the tree as they get expanded.
When the user presses Enter or clicks on an editable node, Ride should use the Edit command to notify the interpreter. Then it can send back commands to open or focus an editor window.
The interpreter may request the display of messages in a separate "Status Output" window.
["StatusOutput",{"text":"some very important message\r\n","flags":2}] // Interpreter -> Ride
flags
(despite its name) is the message type, one of:
1
: info, usually shown in green2
: error, red4
: warning, blue8
: .Net function overload clash overload (red?)
🔴 As of April 2016 there is no process manager.
["GetAvailableConnections",{"connections":[c0,c1,...]}] // Ride or Interpreter -> PM
🔴 Specify what c0,c1,... look like
Request a connection to a specific item (Ride or interpreter).
["ConnectTo",{"remoteId":123}] // Ride or Interpreter -> PM
Tell the client that the ProcessManager is handing off the connection to a Ride or Interpreter (as requested). The process manager knows the supported protocols so it can pick a supported protocol for the clients to switch to. Once this is received the client is no longer connected to the PM, but rather is connected to the specified process.
["ConnectToSucceded",{"remoteId":123,"identity":1,"protocolNumber":...}] // PM -> Ride or Interpreter
identity
: see Identify
Tell the client that the attempt to connect to a particular process failed.
["ConnectToFailed",{"remoteId":123,"reason":""}] // PM -> Ride or Interpreter
["GetDetailedInformation",{"remoteId":[12,34,...]}] // anything -> anything
If sent to a Process manager, remoteId
is a list of remote IDs returned by
GetAvailableConnections
. Otherwise it's an empty list.
["ReplyGetDetailedInformation",{"information":[i0,i1,...]}] // anything -> anything
Ride can request help on current cursor position.
["GetHelpInformation",{"line":"r←1+ab","pos":4}] // Ride -> Interpreter
line
: text containing the name where help is requestedpos
: position of cursor withinline
(origin 0 is to the left of the first character)
["ReplyGetHelpInformation",{"url":"https://help.dyalog.com/18.1/#Language/Symbols/Plus%20Sign.htm"}] // Interpreter -> Ride
Ride can request Syntax information specific to the version of the interpreter being run.
["GetHelpInformation",{}] // Ride -> Interpreter
###⍕ ReplyGetSyntaxInformation
["ReplyGetSyntaxInformation",{"url":"https://help.dyalog.com/18.1/#Language/Symbols/Plus%20Sign.htm"}] // Interpreter -> Ride
Ride can request Language bar information specific to the version of the interpreter being run.
["GetLanguageBar",{}] // Ride -> Interpreter
["ReplyGetLanguageBar",{ // Interpreter -> Ride
"entries":[
{"name":"Left Arrow", "avchar":"←", "helptext":["...","...",]}
]}]
Configuration parameters can be queried using the GetConfiguration method
["GetConfiguration", {"names":["text"] // Ride -> Interpreter
["ReplyGetConfiguration", { // Interpreter -> Ride
"configurations":[
{"name":"string", "value":""},
]}]
Parameters can be set using the SetConfiguration method. [Currently only the AUTO_PAUSE_THREADS parameter is supported.]
["SetConfiguration", { // Ride -> Interpreter
"configurations": [
{"name":"", "value":""},
]}]
["ReplySetConfiguration", { // Interpreter -> Ride
"configurations": [
{"name":"", "rc":0123,},
]}]
name
: key of paramenter to set.value
: value to set it to.rc
: int, one of the following;0
: SO_OK1
: SO_BAD_NAME2
: SO_BAD_VALUE3
: SO_CANT_SET
- related to the process manager
AvailableConnection => [
int remoteID, // Unique ID
string displayName // Display name for the entry.
]
DetailedInformation => [
int remoteID, // Unique ID if sent from a Process Manager, otherwise 0
identity identity,
(DetailedRideInformation|DetailedInterpreterInformation|DetailedProcessInformation) Information
]
DetailedRideInformation => [] // Placeholder - add any more information
DetailedInterpreterInformation => [] // Placeholder - add any more information
DetailedProcessManagerInformation => [] // Placeholder - add any more information
⎕PFKEY
- programmatic access to "current object"
["SetCurrentObject",{"text":""}] // Ride -> Interpreter
- compiler information
- list of valid I-beams and their descriptions
ShowStack
andShowThreads
- drop workspace in Ride