Skip to content

Notes On The Design Of An Ocap Kernel

Jump to bottom
Chip Morningstar edited this page Aug 13, 2024 · 27 revisions

Notes On The Design Of An Ocap Kernel

@Chip Morningstar

Weaselly Disclaimer

This document has been posted primarily to make it available to the Ocap Kernel team, both to help my teammates stay aware of what I'm doing and to get their feedback. However, I still appreciate feedback from anybody else who has the time and energy to go down into the rabbit hole, regardless of who they are or what part of the organization they are in. That said, keep in mind that this is very much a work in progress. The content here is likely to vary wildly in form, substance, and coherence from one day to the next, as the writing here is not merely a record of my emerging design but very much a part of how I work out my thoughts as to what the design even is. If what you seek is some kind of global understanding of the design itself you might want to wait until it's more cooked; rest assured that I won't be quiet about it once some kind of good-enough-to-criticize threshold is reached (if you want to be actively notified at that point, just drop me a note). If what you seek is design details suitable for implementation or any kind of actual close scrutiny, you definitely should wait until it's more complete -- in its current state this is much more of a think piece than a specification, however much it might at times resemble the latter.

Names and Words

"The naming of names is a difficult matter, it isn't just one of your holiday games." -- not T. S. Eliot

We probably should pick a proper label for this thing we are creating. We've been referring to it as the "ocap kernel", but I think it's more than that. While our team is labeled "the ocap kernel team" (and I suspect the higher level management discussions that lead to our being organized and tasked with doing this probably used this language as well), there's a distinction to be made between the platform as a whole and the kernel per se. In particular, I expect we are headed for some kind of kernel+vats architecture. This suggests that there's almost certainly going to at least be a component that is the trusted supervisory code that lives inside the vat but above any user code. This component is clearly part of our platform but definitionally is not part of the kernel.

(Though I will note in passing that "ocap kernel" conveniently abbreviates as "OK", which might be a source of amusement as we contemplate the Naming Of Things.)

A further complication is that each user who is hosting some set of vats will be running a separate instance of our platform, and such instances probably also need a term to label them. An object inside a vat may exchange messages not only with objects in other vats on its same platform instance but also with objects in vats hosted by other users elsewhere on the internet. Consequently, we'll probably want a word meaning "all the vats hosted by some user". Also, I'm being a little loose here with the term "user". Our default mental model is of vats being hosted inside some person's web browser, and it's natural to think of that person as "the user", but of course we also want to consider vats being hosted by independent servers in some data center somewhere whose purpose is to provide services unrelated to any particular human -- it's the internet, after all. You could think of these servers (or their operators) as "users" in some sense, though at that point the language gets a little strained. And of course that's before we even start talking about message traffic exchanged with entities entirely outside our paradigm (via, say, HTTP or whatever), even if those entities remain part of the ecosystem we're concerned with.

So I'm just going to start inventing terminology and making up names for things. Don't get overly fixated on the language that I've chosen (despite the fact that I'm obsessing over it quite a bit here myself) but rather consider all these words provisional, subject to revision, elimination, or addition as we refine our design and get clearer in our own minds what we are doing. In this document I'm going to try to follow the convention writing any new bit of jargon in BOLD ALL CAPS the first time it appears (hopefully in the context of some kind of definition or explanation) and thereafter follow the Germanic convention of Capitalizing These Words when they're used.

Also, as I've been going I've been encountering unanswered design questions that I don't already have a strong opinion on. These are marked with OPEN DESIGN QUESTION in the text.

The Big Picture

Everything here takes as a given that we're building on a common base of Hardened JavaScript, the ocap security model, and various related concepts such as compartments, eventual messaging, and so on, as provided by the SES shim or the XS engine, plus various supporting packages in the Agoric endo repository for things like bundling or marshaling. For purposes of this discussion, I'm going to presume that everybody generally knows what these things are and is on board with this approach, rather than stopping to explain or justify these bits, except when there are nuances that merit deeper discussion (ultimately, of course, there will be more outward reaching documentation that explains All The Things, but that's for later).

At the bottom we have OBJECTS. Because we are building in Hardened JavaScript, these are all just JavaScript objects, but for our purposes we further divide them into three varieties: DATA OBJECTS, BEHAVIORAL OBJECTS, and ORDINARY OBJECTS. This distinction is relevant in the context of marshaling and serialization, and in the places those are used, namely messaging and persistence.

  • Data Objects are passed by value; they are selfless (that is, they have no identity) and are described entirely by the bits from which they are composed. All own properties of Data Objects must be either (a) JavaScript primitive values (e.g., number, string, boolean), (b) References to Behavioral Objects, or (c) recursively nested Data Objects. Since Data Objects are passed by value, they cannot have methods (i.e., own properties that are functions) because these can't be serialized. The prototype of a Data Object must be null or the JavaScript built-in object Object.

  • Behavioral Objects are passed by reference. They have identity and may have methods, but can have no visible state (i.e., no JavaScript own properties other than methods). An empty object (i.e., one with no own properties at all) is considered a Behavioral Object since References to empty object instances are extremely useful as passable tokens (i.e., things that have identity and essentially nothing else).

  • Ordinary Objects are regular JavaScript objects that remain inside the Vat. They may have any mixture of JavaScript properties whatsoever but are not passable.

Each Object resides within some VAT, an isolated process that can send and receive messages. When a message is sent from an Object in a Vat to the world outside that Vat, the message may contain OBJECT REFERENCES that designate Behavioral Objects (which are typically, but by no means necessarily, contained within the Vat sending the message). When a message is received by a Vat from the outside world, the message may contain Object References that designate Behavioral Objects contained by other Vats. Such References can be used by code within the Vat as targets for future messages it might send.

Broadly speaking, a Vat consists of two parts: a SUPERVISOR and USER CODE.

The Supervisor is trusted, privileged code that is part of our platform. It is responsible for:

  • receiving, deserializing, and queueing messages inbound from other Vats, then, when appropriate according to our code execution model, dequeueing, demarshaling, and dispatching these to User Code.

  • accepting, marshaling, and queueing messages sent from User Code to other Vats, then, when appropriate, dequeueing, serializing, and transmitting them to the outside world (specifically, transmitting them via the Kernel; more on this shortly).

  • overseeing persistent storage, both for the Supervisor's own use (e.g., holding the inbound and outbound message queues) and for the storage of persistent objects belonging to User Code.

  • providing other system level services to User Code as we see fit to provide, potentially including process termination, access to other communications channels (e.g., HTTP), clocks, timers, and so on. The list of possible services that might be included here is open ended and potentially large, though of course we'll want to constrain and thoroughly specify it in the final design.

User Code is a bundle of arbitrary, user-provided JavaScript that is executed within a compartment that the Supervisor maintains for that purpose. In general, User Code is untrusted, except insofar as we might have some kind of vetting and certification procedure in place for a limited number of special, privileged applications to be granted special endowments that provide capabilities to perform selected sensitive operations that are normally closely held (e.g., access to the clock or to the browser DOM).

The Vats running on a given host are managed by a KERNEL. The Kernel and Vats together are called a CLUSTER (note: of all the vocabulary I've been defining here, "Cluster" is the word I'd most like to have a better alternative for). The Kernel is responsible for:

  • creating, loading, starting, pausing, resuming, and stopping the Vats in its Cluster (note that a Vat might be created as the result of extrinsic user action or by the intrinsic action of another Vat via a service that the Kernel makes available to it for this purpose)

  • managing the granting of powers to specific Vats under the control of the user whose Cluster it is

  • providing a KERNEL MANAGEMENT API, enabling a user interface by which its associated user can control or configure the Kernel's operation and that of its Vats

  • conveying messages between the Vats that it is managing

  • receiving messages inbound from other Clusters and forwarding them to the appropriate Vats

  • accepting messages outbound from Vats in its Cluster that are targeted at objects hosted in other Clusters, and transmitting them over the network to those Clusters

  • translating inbound and outbound object references (contained in messages) between Vat-relative and global forms

  • Possibly: holding and managing the received message queues for any unsettled promises for which one of its Vats is the decider (OPEN DESIGN QUESTION: should responsibility for queueing messages to an unsettled promise be vested in the Kernel or in the deciding Vat? As a piece of Vat-associated state, there's obvious sense in having the Vat do it, but to the extent that promises can (and possibly often will) resolve to objects external to the deciding Vat, it might make sense to decouple this component of promise state from the Vat itself, in order to enable more efficient message forwarding, to limit certain kinds of potential Vat misbehavior, and ultimately to support automatic shortening of promise chains.)

A Cluster will be hosted either by a browser extension (where the primary UX will be presented in the browser itself via the web interface) or by a standalone Node executable (where the primary UX will most likely be command line arguments and configuration files). Whether it is presented as an interactive web page or as a CLI, we'll refer to the the controlling entity as the CONSOLE. The Console governs the operation of the Cluster via the Kernel. A Console itself consists of two parts:

  • A CONSOLE DAEMON, an independent process that maintains persistent console state independent of the execution state of the Kernel or any Vats. It interacts with the Kernel via the Kernel Management API. Note that the Console Daemon needs to be independent of the Kernel because one of its jobs will be launching the Cluster in the first place, prior to which there is no Kernel. The Console Daemon in turn presents an API to the:

  • A CONSOLE USER INTERFACE, a possibly ephemeral process (or a series of such processes over time) that actually implements the user facing controls for the cluster, whether via a web interface, a CLI, or something else.

One of the things that the Console Daemon maintains is the user's collection of PETNAMES, which are human readable, human meaningful, user assigned labels for those Objects within the Cluster's Vats that have been made available for the user's direct manipulation (using the Console User Interface) as the result of operations invoked from the Console.

We anticipate there will be a single Console Daemon implementation that serves for all purposes, but at least two and possibly more Console User Interface implementations that vary depending on the particulars of the type of control UX that is being realized.

Typically, a given machine will only host a single Cluster, with the notable exception of multiple Clusters on a machine in service of testing and debugging (i.e., not the normal use case but one that is still important to support as part of the DX).

Object Addressing

The ultimate affordance of an Object Reference is its use as the target destination for a message send, though the use of Object References as identity bearing tokens (i.e., values that can be compared with === and !==) is also supported.

At bottom, any given message is sent from one Object to another. However, to describe how things actually work we also need to consider the role of Vats and Clusters. In particular, a message sent to an Object in another Vat within the same Cluster as the sender needs to be handled rather differently from a message sent to an Object in a Vat in a different Cluster.

Within a Cluster, a message sent by an Object in Vat A to a different Object in Vat B is transferred from the sender to the receiver via the Kernel. The Kernel needs to be able to determine, based solely on the Object Reference itself, which Vat to deliver the message to, and in what form to deliver it such that the receiving Vat's Supervisor will understand which of its Objects is to be the actual recipient.

The primary way (aside from bootstrapping, about which more in a moment) that a Vat could know the identity of an Object outside itself is if the reference to that Object had previously been imported into the Vat as an argument of an earlier message, which in turn means that that Object Reference had to have been exported by the originating Vat as part of (or prior to) sending that earlier message. Export preceding import means that the Kernel is always able to associate any Object Reference with the Vat that exported it, and thus knows where to deliver any subsequent message addressed to that same Reference from a different Vat.

In order for this process of Reference begetting Reference not to lead to infinite regress, some kind of bootstrap mechanism is required, else there'd be no way for any Vat to ever know about anything outside itself in the first place and thus no way for it to ever send messages anywhere. To bootstrap the reference graph, whenever a Vat is created, its initialization creates within it an initial ROOT OBJECT, a Reference to which is exported to the creator as the result of the Vat creation operation. If the Vat was created by the action of User Code in another Vat, this Root Object Reference is delivered to the creator Object as the return value from its invocation of the Vat creation operation. If the Vat was created by an action of the user using the Console, the Root Object Reference is placed into the user's Petname table for potential future use in operations commanded via the Console (see the discussion of Petnames below).

The logic for messages between Objects in separate Clusters does not map directly to that for messages between Objects in Vats within the same Cluster. The first important difference is that an observer outside a given Cluster has no way to know which of that Cluster's Vats a given Object Reference feeds into. On the other hand, there's no particular reason for this outside observer to know or care about which specific Vat it is either. Instead, we consider the internal organization of a Cluster to be the Cluster's private business. Outside an Object's containing Cluster we only associate a Reference to it with the Cluster itself. Another way of putting this is: since you don't have visibility into the inner workings of a Cluster, you must treat all the Objects within the Cluster as if the Cluster was a single giant Vat, and that any finer-grained distinction within that Cluster that you might become aware of is an illusion presented to you by the Cluster's Kernel. The logic of References begetting References via message passing in the intra-Cluster case is recapitulated in the inter-Cluster case, but at the granularity of Clusters rather than the granularity of Vats.

A second, and probably more important, difference is that between Clusters there is no entity analogous to a Kernel possessing visibility onto all the Object Reference relationships that are in play between Clusters. Instead, Clusters connect to each other in an arbitrary peer-to-peer network rather than via the star topology that the Kernel organizes its Vats into. This has important implications with respect to how connectivity between Objects is bootstrapped across Cluster boundaries.

Whereas Vats are created by other Vats via the Kernel or by the user via the Console, new Clusters are created by the autonomous actions of users, external to any pre-existing Cluster (e.g., a user boots a new Cluster instance on their machine and connects it to the internet). In particular, the Cluster per se has no notion of a Root Object, since in its initial state a Cluster need not have any Vats at all. Once the Cluster does have Vats (either through later user action via the Console or as a consequence of configuration information provided as part of Cluster creation), these Vats (or rather, the Objects within them) have no a priori connection to anything, so initial connectivity between Clusters cannot be established purely by reference passing. Instead, we require some means for a Cluster's user, or software running on the user's behalf, to publish the equivalent of an Object Reference in a form that can be communicated out of band to the users of other Clusters, who in turn need a means to accept these and convert them into regular Object References of their own. To this end, the Kernel has the capability to translate between Object References, which have meaning in the context of messages between Objects as represented in the implementation of the inter-object message passing scheme, and OCAP URLS, which are text strings that can be communicated via pretty much any means you like (email, SMS, PostIt notes, QR codes on the sides of buses, etc.). The two directions of translation (from Ocap URL to Object Reference and vice versa) are treated as capabilities that are made available to the user via the Console. From the Console these capabilities may be used directly or granted to selected Objects to which the user has access.

Taken all together, the arrangements discussed above imply five different possible forms an Object Reference may take depending on context:

  • A VREF (short for "vat reference") designates an Object within the scope of the Objects known to a particular Vat. It is used across the Kernel/Vat boundary in the marshaling of messages delivered into or sent by that Vat. A Vref is generated and assigned by the Kernel when importing an Object Reference into a Vat for the first time and by the Vat when exporting an Object Reference from it for the first time.

  • A KREF (short for "kernel reference") designates an Object within the scope of the Kernel itself. It is used in the translation of References between one Vat and another. A Kref is generated and assigned by the Kernel whenever an Object reference is imported into or exported from a Vat for the first time. Krefs are strictly internal to the Kernel implementation. (Note: the differentiation between Vrefs and Krefs is important because it enables the Kernel to maintain 2N Reference mapping tables for N Vats rather than having to potentially maintain N2 mapping tables.)

  • An RREF (short for "remote reference") designates an object within the scope of an established point-to-point communications Channel between two Clusters (more on Channels below). An Rref does not survive the Channel it is associated with. An Rref is generated when the Kernel for one Cluster exports an Object Reference into the Channel connecting it to the Kernel for another Cluster.

  • An OCAP URL is an externalizable string that designates an object globally in a form suitable for transmission out of band. It incorporates addressing information that allows Kernel identify the Cluster containing the Object it refers to and create a Channel to it if one does not already exist or to obtain the existing Channel if one does. It designates an Object in a way that is independent of whether the Cluster containing the Object is currently available or not and independent of the Channel used to establish connectivity. It cannot be used to address messages directly, but can be used to establish connectivity and ultimately obtain an Rref which does permit messaging.

  • A PETNAME is a human readable, human meaningful, user assigned label that designates an Object within the scope of a given Cluster's Console. Internally, the Console maintains a table mapping these labels to Krefs. Petnames may manifest as textual names, such as a user might type or paste into a text input box on a web form, or draggable labels (with distinguishing text, icons, or both) that can be manipulated as part of an interactive Console GUI, or they might take some other form depending on the Console UX design, as long as they have the property of being directly meaningful to the user. One user's Petname for an Object is unrelated to another user's Petname for the same Object. (Note that the two users might happen to pick the same Petname string, perhaps deriving it from some functional role, known to both of them, that the Object plays in their world. However, such coincidences cannot be relied on and even when they do happen no deeper meaning can safely be inferred from them.)

In addition, this writeup will at times use EREF (short for "endpoint reference") as a generic term in contexts where we might be talking about either a Vref or an Rref, since it will be less cumbersome than repeating the phrase "Vref or Rref as appropriate" over and over again.

Inter-Cluster Communication and Ocap URLs

A CHANNEL is a communications connection between two Clusters that provides a context for message traffic between them. In particular, it establishes a scope for the interpretation of shared Rrefs. A Channel is a sturdy version of a TCP-like session connection. It may, in fact, be transported over an ordinary TCP (actually, more likely TLS) connection, but it can survive loss and reestablishment of the underlying transport session, as well as termination and restart of the Kernel or Vat processes at either end, while preserving all of the Object References stretched over the connection and preserving unbroken message streams back and forth between the two endpoints.

Connecting one Cluster to another requires solving two important sub-problems: (a) knowing or learning where to make a network connection (i.e., what IP address and port number to connect to) in order to interact with the Cluster, and, (b) once having connected, verifying that what one has connected to is, in fact, the Cluster one actually intended to connect to (or, indeed, if it is a Cluster at all).

In addition, the actual mechanism of network connectivity needs to account for our intention to host Clusters inside users' web browsers. This is significant because browsers commonly run on machines that aren't in a position to directly accept incoming network connections due to network address translation, dynamic IP address assignment, firewalls, proxies, and all the other myriad ways that the IT organizations of the world have gummed up the proper functioning of TCP/IP networks over the years. Unless and until proven otherwise, I'm going to assume we can use one of the several pre-existing solutions to this problem that clever people have devised rather than trying to invent our own workaround. However, we do need to acknowledge that whatever solution we adopt will most likely end up as a key part of our network stack, since it will almost certainly need to be baked into our design at a fairly fundamental level. Consequently, for the time being I'm just going to treat this as a solved problem, making the naive but useful presumption that one computer can make a connection to another at will, while handwaving away all the complications.

An Ocap URL contains several distinct pieces of information:

  • Network addressing information that indicates how to contact the Cluster that contains the object it designates.

  • Cluster identification information that enables the party establishing a Channel to verify that it is validly in contact with the proper Cluster.

  • An opaque encoding of the object's identity in a way that can be decoded by the target Cluster but which by itself does not uniquely reveal the Object's identity. In particular, multiple different Ocap URLs may all encode the same Object Reference, and it should generally be considered a best practice to vend a new, unique Ocap URL each time an Object Reference is externalized from its containing Cluster.

Verifying that a connection has been made to the proper endpoint as part of setting up a secure communications pathway is a job traditionally performed by TLS. However, in order to authenticate that a communications counterparty is the one expected, in normal operation TLS leverages DNS: when connecting to a server via TLS, the server provides a cryptographic certificate attesting to the association between the DNS name of the server and the public key used in the TLS handshake to generate the session key. This certificate is signed by a certificate authority (CA) whose validity is verified either by another certificate signed by a higher level CA (repeating this process recursively) or by being found in a collection of root CAs that the browser is pre-configured to automatically trust. The determination of the ultimate authenticity of the root CAs is highly centralized, leading to a number of points of vulnerability -- the authorities maintaining the root CA list could be compromised, individual CAs could be compromised without their parent CA's knowledge, revocation of a compromised CA's certificate might not have reached the browser (for many different possible reasons), or the root CA list held by the browser could itself be compromised (either within a given user's environment or at some point in the browser's implementation, build, and distribution path). Many of these failures have, in fact, occurred repeatedly in the history of the web. DNS itself is also a point of vulnerability: in the event that the operator of a server loses control over the DNS registry for its domain name, some other party may be positioned to substitute its own server address and corresponding TLS certificate chain. These sorts of compromises have also happened in practice. Because the root of the CA hierarchy is highly centralized, there remains a risk that some of these compromises could happen at scale. A further problem is that even without security failures, establishing the connection requires that the entity connected to possess the requisite certificates in the first place, which is itself administratively complex and highly error prone (and whether or not DNS is involved merely varies the modes of complexity and failure).

An alternative approach, which I want to advocate here, is for our Ocap URLs to employ something like Tyler Close's YURL scheme. While there are lots of details and possible variations, the basic idea behind them all is simple: embed the fingerprint of the contactee's public key into the URL itself, so that when a connection is made its authenticity can be validated directly. Tyler's documentation (just linked) shows how this can be accomplished with existing, mature TLS implementations, requiring only small and relatively simple changes to the way TLS is used. In the 20+ years since YURLs were first proposed they have seen only experimental adoption, in large part because of the huge entrenched installed base built around TLS, HTTPS, and the CA hierarchy, but also because YURLs and their brethren involve URLs that are large and unwieldy, giving them very poor ergonomics in many web use cases. However, because we are already handling Object References using Petnames combined with automation, our typical use cases do not generally require exposing representations of Object References to humans. YURLs do lend themselves very well to being handled via automation in such a fashion, which will allow us to have much better decentralization, avoid a large swath of potential security vulnerabilities, and bypass a lot of complicated, difficult to understand, and error prone administrative procedures (the latter being particularly important if we want this to be used by ordinary users in their browsers rather than just by professional IT people).

(OPEN DESIGN QUESTION: should we have a distinct Ocap URL scheme such as ocap for these URLs or are we better off sticking to some convention for using, say, https? Related: should we look into the cap URL scheme that Mark Miller registered years ago for E?)

(OPEN DESIGN QUESTION: should the host address in an Ocap URL be regarded as the actual site to contact to reach the Object in question or should it be regarded as a hint as to where one might look, with the URL providing entry into a directory service if it doesn't reach the Object directly? In some of the work with E we referred to this as a "redirectory", drawing analogy to a web server that might serve up a page or might serve up an HTTP redirect.)

Theory of Operation

With a couple of exceptions (which will be described when we get to them), we conceive of the interfaces between the various communicating components of the overall system in terms of asynchronous messaging between loosely coupled actors. Note that use of the term "message" here can be a source of confusion, since among the most common and important kinds of things being transported by messages between components are messages between Objects, but these are "messages" at two different levels of abstraction. When helpful for clarity I will sometimes refer to SYSTEM MESSAGES or OBJECT MESSAGES to disambiguate what is being talked about. All the traffic on the interfaces described in this section consists of System Messages; some of these System Messages will carry Object Messages in their payloads.

Typically, connections between processes on the same machine will be carried by interprocess pipes. Channels, which nominally connect across machine boundaries, will be carried via a network protocol layered over TCP/IP. In any case, these connections are always asynchronous and bidirectional. We will of course hide all that under an abstraction layer so that the components themselves are shielded from having to know the particulars of how they talk to each other.

Between any two kinds of components we define an interface that specifies what the allowed messages in each direction are and what operations they invoke. Every component implements some set of methods, some of which may be made available on more than one of its interfaces. In this sense, methods and interfaces are slightly orthogonal, despite most methods being interface-specific. Consequently, we will describe the some of the components below in terms of the methods they provide (and the operations they realize) and then separately indicate which interfaces those methods are exposed on.

Message Translation

One operation that is needed in various places is message translation, wherein a marshaled Object Message is converted from the form needed in one frame of reference to that needed in another. For example, transferring a message from the Kernel to some Vat requires converting all the Krefs it contains to that Vat's corresponding Vrefs. While this minimally includes simply mapping the various Object Reference strings through a table, importing an the Object into some context where it was previously unknown may also require generating a new Reference identifier for it in the receiving context.

Although this design is not yet committing to a specific scheme for marshaling messages and related data values, any acceptable scheme will need to make it easy and efficient to find and translate Object References embedded in a message's encoded representation. Agoric's capdata and smallcaps schemes both satisfy this requirement, as does OCapN. For expository purposes I'll adopt the approach (and accompanying terminology) used by both of Agoric's schemes. These schemes break the representation of a Data Object into two parts, labeled body and slots.

  • slots is an array of strings, each of which is one of the Object References found in the Data Object being marshaled (nominally deduplicated, though this is not strictly required).

  • body is an encoded representation of the structure and content of the Data Object itself, in which each of the embedded Object References is encoded by an index into the slots array.

With this arrangement, Object Reference translation can be performed by simply mapping over the slots array without having to parse or otherwise examine the potentially much more complicated and idiosyncratic body.

An Object Message is represented by a Data Object that contains all the message elements in a standardized form. For our purposes, a message consists of:

  • target - a singular Object Reference indicating the Object to which the message should be delivered
  • method - a string labeling the method to be invoked on the message target
  • arguments - an array of zero or more values that are the arguments to the method call. These values can be primitive data values, Object References, or more complex Data Objects that may have further Object References or Data Objects embedded within them.
  • result - a singular Object Reference indicating a promise that, when settled, will constitute the result of the method invoked

We will sometimes collectively refer to all the message elements except the target as the message payload. It is useful to distinguish this as its own abstraction because in some cases, a message may need to be handled or processed prior to the target being known. For example, we may want to reuse a given message payload in order to send the same message to multiple recipients.

Ken

The interactions between a Kernel and its Vats, and between Clusters (which is to say, between Kernels) are mediated by a protocol known as Ken. (The name itself doesn't stand for anything in particular; it's an allusion to Tyler Close's Waterken system in which the underlying principles were first articulated.) Ken is described in more detail in this paper from HP Labs and this paper given at Usenix 2012.

The purpose of many communications protocols, such as TCP/IP or X.25, is to compensate for the unreliability of the underlying data communications link, which can fail in a myriad of ways. Ken is not really one of those; its job is to compensate for the unreliability of the endpoints themselves. This is particularly important with systems of asynchronously interacting autonomous processes which are attempting to cooperate despite not being under a common administrative umbrella. In point of fact, Ken is not really a protocol as such but more of a design pattern that describes how communicating processes should behave and interact so as to ensure, as the papers describing it say, "output-valid rollback-recovery", a kind of fault tolerance that maintains global consistency of system state despite having autonomous components that can fail unpredictably.

Ken works particularly well in systems composed of event-loop driven processes, which is what this design is proposing. The basic outline is as follows:

  1. When a message is received, record its contents in a persistent medium before processing it.
  2. After it has been persisted, acknowledge its receipt to the sender.
  3. Process each received message to completion, one at at time. Only persist process state when completion is reached.
  4. Persistent process state must include the contents of any additional messages that were sent as part of this processing activity.
  5. These sent messages are only transmitted after they have been persisted.
  6. Only after a transmitted message has been acknowledged by its recipient can the message be removed from the sender's persistent record.

The idea is that all permanent external consequences of processing an event -- both state changes and message output -- manifest atomically. In particular, when an event handler sends a message, what it is really doing is adding that message to a queue of messages that will be released in a batch when it is done handling the event. If the handling of the event is aborted during processing, whether deliberately or as the consequence of system failure, it is as if the event processing never happened.

When an aborted process is restarted, part of its recovery procedure is to reconnect to any other processes it had been in communications with. Since it was persisting messages as they were received, it knows where it was in each incoming message stream and can request its counterpart on that stream to retransmit any messages that were missed.

Each ENDPOINT of a connection between two processes can be both the source and the destination of messages. In particular, in the Ken framework each Endpoint is the source of an ordered stream of messages to its counterpart and the destination of a stream of messages going in the opposite direction. Each message in one of these streams contains a sequence number, an ordinal indicating the message's position in the stream, counting upwards monotonically from 1. Sequence number counters in each direction are scoped to the Endpoint and last for the life of the connection. Note that when we talk about the "connection" between Endpoints, we are talking about the communications relationship between them that Ken provides. The scope of this connection potentially (depending on implementation) extends beyond the life of the raw, low-level communications link between the two parties that the Ken relationship is built on top of. In such cases, that low-level link can be broken (either deliberately or as the result of network errors or process crashes) and then resumed, while the message streams that the connection carries remain intact.

For purposes of explanation, it's useful to see things from the perspective of a single process. We'll refer to the process itself as the LOCAL ENDPOINT (or simply as LOCAL to avoid verbosity) and other processes that it has connections to as REMOTE ENDPOINTS (or REMOTES).

For each connection, Local maintains the following data:

  • recvCount -- The count of messages received from Remote
  • sendCount -- The count of messages sent to Remote
  • sendAckCount -- The sequence number of the last message sent to Remote for which acknowledgement has been received
  • sendQueue -- A queue of messages to Remote that have been produced by Local computational activity but not yet transmitted
  • sentMsgs -- A collection of messages that have been transmitted to Remote but not yet acknowledged
  • recvQueue -- A queue of messages that have been received from Remote but not yet processed

At the start of the connection, recvCount, sendCount, and sendAckCount are all 0, while sendQueue, sentMsgs, and recvQueue are empty.

In addition, for each incoming message, Local maintains a notional toSendQueue, an ordered collection of messages that begins empty and accumulates outgoing messages as the corresponding incoming message is being processed.

All of these values are kept in persistent storage (i.e., they will survive crash and restart of the Local process). This writeup will begin with the simplifying assumption that this data is always immediately and synchronously written to persistent storage whenever it is updated. In the interest of performance, in practice we don't actually need to persist this aggressively, as long as certain rules are followed. However, the particulars of those rules complicate the story, so in the interest of clarity we will describe the persistence rules and related complications separately.

The logic of message exchange with Remote rem goes like this:

When a message msg arrives from rem:

  1. If msg.seqnum is <= rem.recvCount, discard the message and take no further action. Otherwise:
  2. If msg.seqnum is != rem.recvCount + 1, raise a MessageOutOfOrder error
  3. Push the received message msg onto rem.recvQueue
  4. Increment rem.recvCount
  5. Transmit to rem an acknowledgement of rem.recvCount (Note: in practice this acknowledgement doesn't actually need to be transmitted eagerly but can be nagled. It can wait to piggyback on top of the next communication one would be sending to rem anyway, or simply be deferred for a time.)

When rem.sendQueue is not empty:

  1. Pop msg off rem.sendQueue
  2. Increment rem.sendCount
  3. Add a new entry rem.sentMsgs[rem.sendCount] = msg
  4. Transmit msg to rem

When rem.recvQueue is not empty:

  1. Pop rmsg off rem.recvQueue
  2. Set rmsg.toSendQueue to an empty collection
  3. Process rmsg; any messages sent during processing are pushed onto rmsg.toSendQueue
  4. (Atomically) While rmsg.toSendQueue is not empty:
    1. Pop smsg off rmsg.toSendQueue
    2. Push smsg onto smsg.target.sendQueue

When an acknowledgment of seqnum arrives from rem:

  1. Delete all entries rem.sentMsgs[n] where n <= seqnum
  2. Set rem.sendAckAcount to seqnum

When a retransmit request for seqnum arrives from rem:

  1. If seqnum <= rem.sendAckCount, raise a SequenceFailure error
  2. If rem.sendCount < seqnum, raise a SequenceFailure error
  3. If rem.sentMsgs[seqnum] does not exist, raise SequenceFailure error
  4. Transmit, to rem, each message rem.sentMsgs[n] where n >= seqnum

To re-establish connectivity with rem after an interruption in the underlying communications link or a restart of the Local process:

  1. Transmit to rem a retransmit request for rem.recvCount
  2. Compute lastSent, where rem.sentMsgs[lastSent] is the message most recently transmitted to rem
  3. Transmit a lastSent status update to rem

When a lastSent status update arrives from rem

  1. If lastSent > rem.recvCount, transmit to rem a retransmit request for rem.recvCount

Kernel

The Kernel interacts with three different (kinds of) entities via three different interfaces:

  • Console (via the Console Daemon) to Kernel: KERNEL MANAGEMENT API
  • Vats (via a Vat's Supervisor) to Kernel: SYSCALL API
  • Remote Cluster (via a Channel) to Kernel: REMOTE API

In the individual method descriptions, each method will be tagged with markers indicating which interface or interfaces expose it and how:

  • C - Console: the Kernel Management API
  • V - Vat: the Syscall API
  • R - Remote: the Remote API
  • I - internal, meaning that in addition to its appearance in one or more of the regular Kernel interfaces, the method is also employed as a compositional building block in the realization of other methods
  • O - Ocap, meaning the method may be made available to selected Vats as an endowment (i.e., provided to the Vat's User Code at the time the Vat is launched), in the form of a (potentially revocable) object capability that allows the method to be invoked from within the Vat to which the capability has been specifically granted

Inbound to the Kernel, messages are consumed eagerly (though subject to rate limiting to protect against attacks on availability) from the connection upon which they arrive, then queued within the Kernel itself. The Kernel maintains a separate queue for each connection to simplify prioritization of traffic handling. In general, the Console has priority over Vats which in turn have priority over Channels, while contending traffic from multiple Vats or multiple Channels are serviced round-robin. More complicated scheduling strategies are possible and might be worth exploring, but I think that at this stage in our design process we know too little about our requirements and our likely operating ecosystem to have actionable opinions about what, if any, alternatives might be desirable.

Outbound from the Kernel, messages are transmitted immediately, though subject to whatever buffering the host operating system might employ. In the fullness of time we probably need to consider how flow control might back up into actually blocking further processing on the sending end of an interface, but that seems like an advanced consideration that should not hold up engineering progress right now.

Kernel Methods

For expository purposes, the various Kernel methods are presented in functional groups. This grouping does not imply any deeper semantics.

Cluster Management

  • StartCluster(storage)* [C] -- Launch a new Kernel process for a Cluster

    1. Establish a connection to Kernel's Console Daemon.
    2. Interrogate Kernel's persistent storage to determine the state of its Cluster. If the Kernel is being launched as a part of the launch of a new Cluster (indicated by absence of such state), initialize persistent state with:
      1. New Cluster ID
      2. Empty extant Vats table
      3. Empty extant Channels table
    3. If there was already a running Kernel process with this Cluster's Cluster ID:
      1. Send error feedback to the Console
      2. Exit
    4. For each extant Vat vatID:
      1. If the corresponding Vat process is already running: connect to the running Vat process
      • If not: invoke RestartVat(vatID)
    5. For each extant Channel channelID:
      1. Invoke RestartChannel(channelID)
      2. If this succeeds: resynchronize the message streams to and from each of the corresponding remote Clusters according to the Ken protocol.
      • If this fails:
        1. Invoke TerminateChannel(channelID)
        2. Send appropriate error feedback to the Console

    * Note: this not actually a method, and it is not exposed on any Kernel interface, since it cannot be invoked by sending a message to the Kernel: by definition, at the time it is initiated there is no Kernel to send a message to. Instead, it is the operation implied by launching the Kernel application at the operating system level in the first place. Parameters governing its operation are passed as command line arguments rather than as message arguments.

  • StopCluster() [C] -- Shutdown all of a Cluster's processes in way that can be restarted

    1. For each extant Channel, stop that Channel
    2. For each extant Vat:
      1. Invoke StopVat() on that Vat
      2. Await ReceiveCrankCompletion resulting from the StopVat method
    3. Checkpoint Kernel state
    4. Exit

  • TerminateCluster() [C] -- Permanently shutdown all of a Clusters's processes and reclaim its resources

    1. For each extant Channel channelID, invoke TerminateChannel(channelID)
    2. For each extant Vat, invoke TerminateVat() for that Vat
    3. Delete Kernel's persistent state
    4. Exit
Vat Management

  • LaunchVat(isFirstTime, bundleSpec, config, endowments) [C, O] -- Create a new Vat and start it running in a process

    1. Start an instance of the Supervisor application, passing it:
      • a flag indicating that it should start a new Vat instance
      • a pathname or bundle that the Vat is to execute as its User Code
      • any additional configured initialization parameters for the Vat instance
      • any special ocaps that the party launching the Vat wishes it to have
    2. Add the newly launched Vat to the Kernel's extant Vats table

  • StopVat(vatID) [C] -- Shutdown a Vat process, retaining its persistent state

    1. Invoke StopVat() on the Vat
    2. Await ReceiveCrankCompletion resulting from the StopVat method
    3. Update the Vat's run state in the extant Vats table to note that it is stopped

  • UpdateVat(vatID, bundleSpec, updateConfig) [C, O] -- Replace a Vat's code without loss of persistent state

    1. Start an instance of the Supervisor application, passing it:
      • a flag indicating that it should restart an existing Vat instance
      • a pathname or bundle that the Vat is to execute as its User Code
      • any additional configured update parameters for the Vat instance
    2. Update the Vat's run state in the extant Vats table to note that it is running

  • RestartVat(vatID) [C, I] -- Restart a previously shutdown Vat from its persistent state

    1. Start an instance of the Supervisor application, passing it:
      • a flag indicating that it should restart an existing Vat instance
    2. Update the Vat's run state in the extant Vats table to note that it is running

  • TerminateVat(vatID) [C, I, O] -- Shutdown a Vat process and discard its persistent state

    1. Invoke StopVat() on the Vat
    2. For each Cluster the the Vat has exported remote References to: invoke DropExports([refs]) on all such References, via the corresponding Channel
    3. For each Cluster the the Vat has imported remote References to: invoke DropImports([refs]) on all such References, via the corresponding Channel
    4. Remove the Vat from the extant Vats table
Channel Management

  • StartChannel(clusterSpec) [I] -- Create a Channel to some remote Cluster

    1. If there is already a Channel to the Cluster, fail
    2. Open a connection to the Cluster
    3. If this succeeds:
      1. Construct a new Channel with the connection
      2. Add the new Channel to the Kernel's extant Channels table
    • If this fails, fail

  • StopChannel(channelID) [I] -- Disconnect a Channel, retaining the option to reconnect it

    1. Drop the Channel's connection
    2. Update the Channel's state in the extant Channels table to note that it is closed

  • RestartChannel(channelID) [I] -- Reconnect a stopped Channel

    1. Open a connection to the Cluster
    2. If this succeeds: update the Channel's state in the extant Channels table to note that it is open
    • If this fails, fail

  • TerminateChannel(channelID) [I] -- Permanently disconnect a Channel with no option to reconnect it

    1. Send a TerminateChannel message over the Channel to the remote Cluster
    2. Drop the Channel's connection
    3. For each Vat exporting References to the remote Cluster: invoke RetireExports([refs]) on all such exports from that Vat
    4. For each Vat importing References from the remote Cluster: invoke RetireImports([refs]) on all such imports from the Vat
    5. Remove the Channel from the extant Channels Table

    OPEN DESIGN QUESTION: Should these methods also be exposed on the Console interface (which in an earlier draft they were)? AGAINST: the management of Channels should be automatic and the Channels themselves should be transparent, so enabling them to be fiddled with externally seems questionable. FOR: being able to fiddle with them could be helpful in the context of development and debugging.

Code Execution

  • OcapURLToObjectReference(ref) [C, O] -- Given an Object Reference, provide an equivalent Ocap URL

  • ObjectReferenceToOcapURL(url) [C, O] -- Given an Ocap URL, provide the corresponding Object Reference

  • DeliverObjectMessage(targetRef, message) [C, V, R] -- Deliver an Object Message to the appropriate Vat

  • NotifyPromiseSettlements([[promiseRef, value], ...]) [C, V, R] -- Deliver a batch of promise resolutions/rejections to the appropriate Vat

  • ReceiveCrankCompletion(status, completionRecord) [V] -- Accept the results from a Vat of processing an Object Message or promise settlement, consisting of: result status, Object Message sends, and promise settlements

  • DropImports([refs]) [V, R, I] -- Note that a Vat will no longer Reference some Object

  • RetireImports([refs]) [V, R, I] -- Note that a Vat will no longer recognize some Object

  • DropExports([refs]) [V, R, I] -- Note that a Vat no longer contains some Object

  • RetireExports([refs]) [V, R, I] -- Note that a Vat may no longer Reference some Object

    Note: While the signature of each of these methods is the same regardless of which interface it appears on, Object References (which are simply strings) contained within arguments are interpreted in the context of that interface: - C (Kernel Management API): Krefs - V, O (Syscall API): Vrefs - R (Remote API): Rrefs - I (Internal): inherited from whoever called it

Debugging and Development
  • ListVats() [C] -- Provide a list of all extant Vats, with accompanying descriptive metadata
  • ListChannels() [C] -- Provide a list of all extant Channels, with accompanying descriptive metadata
  • DebugVat(vatID) [C] -- Command a Vat to make itself available to the/a JavaScript debugger
  • DumpVatInfo(vatID) [C] -- Provide an info record about a Vat, including its c-lists and persistent state
  • DumpKernelInfo() [C] -- Provide an info record about the Kernel, including its persistent state
  • DumpChannelInfo(channelID) [C] -- Provide an info record about a Channel, including its c-lists

Kernel Operation

The primary job of the kernel is routing Object Messages to and from Objects in its Vats. Each message has a source and a destination, either of which could be in one of its Vats or in some remote Cluster. (One might ask about the case where both source and destination are in remote Clusters and the Kernel is simply acting as a relay without any of its Vats actually participating. I believe the logic of the Kernel's operation will handle this case transparently and automatically. However, in the interest of not piling on complications, I'm not going to spend much energy right now analyzing this. Even so, this case could prove important in the future so, let's make a note not to forget about it.)

For purposes of this discussion we will treat notifications of promise resolutions and rejections as if they were Object Messages. The way these get handled within the Endpoints is quite different from how Object Messages get handled, but the logic of communications and routing is identical.

From the Kernel's perspective, there are three kinds of Endpoints we are concerned with:

  • remote Clusters, connected via Channels to the Kernel
  • local Vats, which the Kernel is managing
  • the Console, mostly trivial for purposes of this discussion, but needs to be accounted for

The Kernel manages its communications with these Endpoints following the rules of the Ken protocol, described above. However, note that for a remote Cluster, connections (and related data such as sequence numbers) are scoped to the Channel; the Channel wraps a communications link that can come and go, but if the Channel itself is closed or lost and then subsequently new Channel to the same Cluster is opened, the message streams and the associated sequence counters are reset.

In addition to the counters and queues required by Ken, for each Endpoint, the Kernel maintains the following data:

  • erefToKref -- A table mapping from the Remote Endpoint's Object Reference namespace to the Kernel's
  • krefToEref -- A table mapping from the Kernel's Object Reference namespace to the Remote Endpoint's

Initially, erefToKref and krefToEref are empty.

In addition, the Kernel maintains a single global table that is independent of Endpoint:

  • krefToEndpoint -- A table mapping each Kref to the Remote Endpoint from which the Object Reference it denotes was imported

The logic of message routing follows that of Ken generally, with a few important wrinkles:

When a message msg arrives from Endpoint ep:

  1. Validate the sequence number according to normal Ken rules
  2. Translate msg into kmsg by mapping it from the Endpoint namespace to the Kernel namespace, replacing each Reference r in msg (msg.target, msg.arguments.slots[], and msg.result) with ep.erefToKref[r]. If there is no mapping table entry for r then:
    • If r is msg.target, raise an UnknownMessageTarget error
    • Otherwise add a new export from ep to the Kernel:
      1. Generate a new Kref k
      2. Add a new entry krefToEndpoint[k] = ep
      3. Add a new entry ep.erefToKref[r] = k
  3. Push the translated message kmsg onto krefToEndpoint[kmsg.target].sendQueue
  4. Handle sequence numbers and acknowledgement according to normal Ken rules

When ep.sendQueue is not empty:

  1. Pop kmsg off ep.sendQueue
  2. Translate kmsg into msg by mapping it from the Kernel namespace to the Endpoint namespace, replacing each Reference r in kmsg (kmsg.target, kmsg.arguments.slots[], and kmsg.result) with ep.krefToEref[r]. If there is no mapping table entry for r then:
    • If r is kmsg.target, raise an InternalConsistencyFailure error
    • Otherwise add a new import from the Kernel to ep:
      1. Generate a new Eref e
      2. Add a new entry ep.krefToEref[r] = e
  3. Add msg to ep.SentMsgs and transmit according to normal Ken rules

The difference, in summary, is that the Object References embedded in Object Messages undergo namespace translation appropriate for the Remote Endpoint.

The Structure of Object References

The format of Object References proposed here is lifted wholesale from that used by Agoric's SwingSet. I don't believe our requirements introduce any new complications that we need to account for, nor have we proposed abandoning any functionality that might enable us to simplify the scheme. These patterns have had a lot of operational shakedown and are known to be robust, so there's no need to reinvent the wheel here.

Kernel Object References (Krefs)

Object references in the Kernel distinguish between regular objects from promises, since the latter require special handling. The pattern for Krefs will be kTN where k indicates this is a Kref, T is either o or p to indicate whether the referenced this is an object or a promise, and N is an ordinal integer assigned by the kernel from a counter. Thus, for example, ko47 is Kernel object number 47, while kp3 is Kernel promise number 3.

A single counter would suffice for both, so that the value of N would be sufficient to uniquely specify the thing referenced. However, by maintaining separate counters for promises and objects we halve the speed that Kref size grows. Note also that the k is technically redundant, but has historically proven extremely helpful in debugging to distinguish Krefs from other kinds of References appearing in log output and debug messages. (In this sense the p and o are also redundant, but eliminating them would require a table lookup any time we needed to know what something was, and since that will happen a lot it's more convenient to have it in the reference string directly. Also, it too is helpful in debugging.)

Vat Object References (Vrefs)

Vrefs are used for communicating across the Kernel/Vat boundary, so they need to be intelligible to both sides. Unlike Krefs, Vrefs can be generated on either side of this boundary. Vats generate them when exporting objects from the Vat to the Kernel, while the Kernel generates them when importing from the Kernel to the Vat. As with Krefs, these can be generated using counters, but we need a separate counter for each side and we need to be able to distinguish which is which. This leads to a slightly more complex scheme: vTSN, where v indicates this is a Vref, T is either o or p to indicate whether the reference is to an object or a promise, S is either + or - to indicate which party generated the identifier: + from the Vat and - from the Kernel, and N is an ordinal indicating which specific thing it is. Thus, for example, vp+12 is the 12th promise reference generated by the Vat, while vo-63 is the 63rd object reference generated by the Kernel (for that Vat). Similar to the story with Krefs, in some sense the v, o, and p are redundant, but regardless of that they're overwhelmingly helpful in practice.

Remote Object References (Rrefs)

Rrefs are used for communicating between Clusters over a Channel. Like Vrefs, references can be generated at either end, requiring support for separate allocation counters on each side. This leads to a scheme of the form rTSN, which is parallel to the pattern used for Vrefs except that we prefix the References with an r instead of a v. However, unlike the Vat/Kernel relationship, the relationship between the two ends of a Channel is symmetric, so there's no a priori basis for assigning the + to side A and the - to side B. There are various symmetry breaking schemes one might devise that would require some kind of handshake as part of connection setup, but that is not actually required. Instead, each side uses + for entities that exist on its local end of the Channel and - for entities that exist on the remote end. We then follow the principle that each side is very polite and will always encode any messages it sends using the form understood by its counterpart. This is accomplished via asymmetry in the erefToKref and krefToEref tables, where each + in a key in the erefToKref table is matched by a - in the corresponding value in the krefToEref table, and vice versa (whereas for Vrefs the erefToKref and krefToEref tables implement a pure bidirectional mapping). Thus when side A sends ro+47 to side B, it is understood to refer to object #47 existing on side B, which will then be referred to as ro-47 in any messages sent back from B to A that reference the same object.

Vats

A Vat is made up of the Supervisor and the User Code. Unlike most of the other components, the Supervisor and the User Code are together regarded as a single intentional unit, the Vat. While the User Code actually does whatever job that particular Vat has been put in place to do, the User Code itself is (usually) untrusted, since it can contain unknown code coming from anywhere. The role of the Supervisor is to provide a trusted intermediary between the Kernel and the User Code.

Operation of a Vat involves three interfaces:

  • Kernel to Supervisor: VAT MANAGEMENT API
  • Supervisor to User Code: USER CODE INTERFACE
  • User Code to Supervisor: VAT SERVICES API

The Vat Management API is how the outside world talks to a Vat. The Supervisor handles all interactions with the Kernel on behalf of its Vat. It receives System Messages from the Kernel via the Vat's Vat Management API, while it sends messages to the Kernel via the Kernel's Syscall API interface.

The User Code Interface and Vat Services API are strictly internal to the Vat.

Unlike almost all the other interfaces in this design, the User Code Interface is an asynchronous function call interface rather than a serialized data stream. The Supervisor dequeues an event from its run queue (put there by System Messages received via the Vat Management API) and dispatches it to the User Code, which then processes it to completion and returns to the Supervisor.

The Vat Services API is a synchronous function call interface that the Supervisor provides to the User Code as part of the Vat startup procedure. It is how the Supervisor provides various services to the User Code, such as the ability to send Object Messages.

All Supervisor activity aside from I/O (i.e., reading System Messages from the Kernel to Vat connection and capturing them in the Vat's run queue) happens when the Supervisor has agency, either between invocations of the User Code or during execution of one of the service functions in the Vat Services API. At any given time either the Supervisor has agency or the User Code does, but never both. This is important because while Supervisor code is running it cannot be interfered with by concurrent execution User Code. More importantly, once the User Code has completed processing an event the Supervisor can be assured that no additional User Code will run until such a time as the Supervisor chooses to allow it.

Unlike the Kernel's interfaces, the sets of methods exposed on the three Vat interfaces are disjoint. In other words, each interface presents a unique and distinct group of methods.

Kernel to Supervisor Methods: Vat Management API

  • LaunchVat(isFirstTime, storage, bundleSpec, config, endowments)* -- Start an instance of the Supervisor application, passing it:

    • a flag indicating whether it is launching a new Vat instance or restarting an existing Vat instance
    • access to the persistent storage that the running Vat will use
    • a pathname or bundle that the Vat is to execute as its User Code, or a null indicating that an existing Vat instance should deploy the User Code it already has.
    • any additional configured initialization or update parameters for the Vat instance
    • any special ocaps that the party launching the Vat wishes it to have

    * Note: this is not actually a method, since it cannot be invoked by sending a message to the Vat: by definition, at the time it is initiated there is no Vat to send a message to. Instead it is the operation implied by launching the Supervisor application at the operating system level. Parameters governing its operation are passed as command line arguments rather than as message arguments.

  • DeliverObjectMessage(targetRef, message) -- Deliver an Object Message to this Vat

  • NotifyPromiseSettlements([[promiseRef, value], ...]) -- Deliver a batch of promise resolutions/rejections to this Vat

  • StopVat() -- Shutdown this Vat's process, retaining its persistent state

  • TerminateVat() -- Shutdown this Vat's process and discard its persistent state

  • DumpVatInfo() -- Provide an info record about this Vat, including its persistent state

Supervisor to User Code Methods: User Code Interface

  • Init(startMode, syscall, configParameters) -- Initialize the User Code. The actual initialization that will be done is specific to whatever the User Code is implementing; the interface merely specifies a standard set of parameters including:

    • a flag indicating whether this is the initial launch, an ordinary restart, or a restart after upgrade
    • a reference to the Supervisor services (syscall) object
    • configured initialization/upgrade parameters (including endowments)

  • Invoke(target, args)* -- Invoke a function or method inside the User Code.

  • Settle(promise, successFlag, value)* -- Resolve or reject a promise contained within the User Code's state, possibly triggering execution of then handlers registered on that promise

    * Note: the Invoke and Settle are not actually methods exposed on the User Code Interface per se. Instead, they represent the actual direct method invocation, or promise resolution/rejection on the relevant targets. This is possible because the Supervisor and the User Code execute in the same address space. Externally initiated invocations or settlements are only possible for targets that have previously been exported by the User Code by earlier outbound messages or promise settlements.

User Code to Supervisor Methods: Vat Services API

  • SendObjectMessage(target, args) -- Send an Object Message to some target external to the Vat

  • SettlePromise(promise, successFlag, value) -- Notify the world outside the Vat of the settlement of some previously exported promise

  • Exit(status) -- Permanently shutdown this Vat

  • Persistent data store methods TBD

    In all cases the actions initiated by User Code invocations of Vat Services API methods do not take place immediately. Instead, they are accumulated into the crank completion record for whatever crank the Vat is currently executing, and will be acted on collectively when the User Code finishes and surrenders agency to the Supervisor. At that time the actions will be executed by the Supervisor or added to the completion record that will be relayed to the Kernel via the Kernel's ReceiveCrankCompletion method as part of executing the ken protocol on the Vat's behalf.

Console

As described above, the Console consists of two components, the Console Daemon and the Console UI. In terms of the overall system design, we have little to say in this section about the Console UI aside from the interface that the Console Daemon exposes to it. This is not to say the Console UI design is not vitally important, but in considering its design we are principally concerned with usability rather than fundamental semantics. Moreover, as previously explained, we expect there to be multiple flavors of Console UI for different kinds of users and for different purposes, so even in a mature system there's no definitive design to specify.

The Console Daemon exists mainly to mediate access to the Cluster in terms of Petnames rather than raw, low-level Object References. It runs as a separate process so that the mapping table can be managed independently from the comings and goings of Console UI instances. OPEN DESIGN QUESTION: Might the Console Daemon be better conceptualized not as a background process but as a Kernel interface library (to be linked directly into individual Console UI implementations) that accesses a shared Petname table database? This might simplify some things, though it would require adopting a database that supports concurrent access by multiple readers and writers. I have an intuition that this latter approach is a potential footgun, and in particular is risky from a security perspective, but I don't at this time feel like I have an articulable justification for that intuition. However, for the time being I'm going to proceed with the daemon process approach until we have a chance to discuss the question more deeply.

The Console Daemon nominally presents two interfaces, but really only one:

  • Console UI to Console Daemon CONSOLE SERVICES INTERFACE
  • Kernel to Console Daemon

The Console Services Interface is essentially a tamed version of the Kernel Management API, exposing those Kernel methods that are suitable for direct manipulation by a user in form adapted to such use.

The Kernel to Console Daemon interface isn't really an interface in the sense of all the other interfaces we're defining here. There are no actual methods, as all information that flows from the Kernel to the Console does so in the context of responses to Kernel Management API methods rather than the Kernel initiating an operation itself -- in principle the Kernel should be able run on its own for extended periods of time without any Console attached, so it only speaks to the Console when the Console speaks to it.

Console UI to Console Daemon Methods: Console Services Interface

Essentially all of the methods described above for the Kernel that are tagged with C are replicated on the Console Services Interface. The key difference is that Object References (as found, for example, in message deliveries) are expressed in terms of Petnames.

Note that most of these Kernel methods are quite inappropriate for direct exposure to human users, but are instead made available so that the Console UI can implement higher-level operations that are appropriate to give to people directly (without taking a position as to what those appropriate operations might or ought to be). The exception is debugging and development tools, which might provide affordances for doing all kinds of things manually that normally would only be done by code. However, since these are both powerful and potentially dangerous, the availability of such affordances should be gated by security checkpoints or at least buried deep in some menu, so that unsophisticated users (which is to say, most users) don't get entangled with them by accident when they don't know what they're doing, just as you wouldn't provide an ordinary user with a hex debugger that lets them poke directly into memory.

The remaining methods that are unique to the Console Services Interface are concerned with managing the Console's Petname table, which maps between user-provided Petnames and Krefs.

  • NameObject -- Assign a Petname (and optionally description) to an Object. Note that a given Object can have more than one Petname.
  • RenameObject -- Change the Petname of an Object to a different string
  • RemovePetname -- Remove an entry from the Petname table. Note that this does not remove the Object itself, simply makes it inaccessible to direct manipulation via the name being dropped.
  • ListPetnames -- Return a list of the currently extant Petnames (and optionally the descriptions that go with them)

[WIP todo notes]

get straight on import/export drop/retire logic

Channel and inter-Cluster connection logic

handling promises and eventual send

persistent storage services for kernel and vat

three party handoff - priority? necessary? threat or menace?

Clone this wiki locally