RFC 1010 Updated Implementation Details of the Protocol Changes

[fantix]

#3933 has been updated quite a few times, this is a summarized version of the results.

1. The Concept of State

State is defined as a payload of data that provides a context for EdgeQL commands to run with. Currently, state is consist of module aliases, "session" config and global values, for example:

{
    module := 'default',
    aliases := [ ( 'alias', 'module::target'), ... ],
    config := cfg::Config {
        session_idle_transaction_timeout: <duration>'0:05:00',
        query_execution_timeout: <duration>'0:00:00',
        allow_dml_in_functions: false,
        allow_bare_ddl: AlwaysAllow,
        apply_access_policies: true,
    },
    globals := { 'mod::key' := value, ... },
}

State is sent from the client with Parse and Execute messages, and returned from the server in CommandComplete messages. The server compiles and executes the given command(s) in the context of the given state, and command(s) in Execute may modify the state, then CommandComplete would include an updated state.

State is serialized in the same way as input/output arguments are serialized. A new type descriptor "input shape descriptor" is used to describe state data as sparse objects. This is similar to the object shape descriptor, only that sparse objects will skip serializing missing properties. For example, the state itself is a sparse object, values of its keys config and globals are also sparse objects.

2. On establishing connections

Once the server has confirmed successful authentication with AuthenticationOK, it then sends the following messages:

1. ServerKeyData
2. A new StateDataDescription message
3. ParameterStatus

The new StateDataDescription message contains an input shape descriptor of the state in the following format:

struct StateDataDescription {
  // Message type ('s').
  uint8           mtype = 0x73;

  // Length of message contents in bytes,
  // including self.
  uint32          message_length;

  // State descriptor ID.
  uuid            typedesc_id;

  // State descriptor.
  bytes           typedesc;
};

The client could build codecs from the type descriptor and cache them for future state (de-)serialization on demand.

3. Send command request with state

The Parse and Execute messages are updated to allow the client send states:

 struct Parse {
   // Message type ('P').
   uint8           mtype = 0x50;

   // Length of message contents in bytes,
   // including self.
   uint32          message_length;

   // A set of message headers.
   uint16          num_headers;
   Header          headers[num_headers];

   // A bit mask of allowed capabilities.
   uint64<Capability> allowed_capabilities;

   // A bit mask of query options.
   uint64<CompilationFlag> compilation_flags;

   // Implicit LIMIT clause on returned sets.
   uint64          implicit_limit;

   // Data output format.
   uint8<OutputFormat> output_format;

   // Expected result cardinality.
   uint8<Cardinality> expected_cardinality;

   // Command text.
   string          command_text;
+
+  // State data descriptor ID.
+  uuid            state_typedesc_id;
+
+  // Encoded state data.
+  bytes           state_data;
 };

 struct Execute {
   // Message type ('O').
   uint8           mtype = 0x4f;

   // Length of message contents in bytes,
   // including self.
   uint32          message_length;

   // A set of message headers.
   uint16          num_headers;
   Header          headers[num_headers];

   // A bit mask of allowed capabilities.
   uint64<Capability> allowed_capabilities;

   // A bit mask of query options.
   uint64<CompilationFlag> compilation_flags;

   // Implicit LIMIT clause on returned sets.
   uint64          implicit_limit;

   // Data output format.
   uint8<OutputFormat> output_format;

   // Expected result cardinality.
   uint8<Cardinality> expected_cardinality;

   // Command text.
   string          command_text;
+
+  // State data descriptor ID.
+  uuid            state_typedesc_id;
+
+  // Encoded state data.
+  bytes           state_data;

   // Argument data descriptor ID.
   uuid            input_typedesc_id;

   // Output data descriptor ID.
   uuid            output_typedesc_id;

   // Encoded argument data.
   bytes           arguments;
 };

If the client decides not to send state, the state_typedesc_id can be set to a special null type ID 00000000-0000-0000-0000-000000000000, the server will then use the default state to compile and/or execute the given commands accordingly.

Otherwise, the given state_typedesc_id is expected to match the current state type on the server-side. On mismatch, the server will return a StateDataDescription message with the current state type descriptor, immediately followed by a new error StateMismatchError in an ErrorResponse message.

4. Server returns state

On successful execution of EdgeQL commands, the server will return:

1. An optional CommandDataDescription if the output type identifier mismatches;
2. Zero or moreData messages.
3. An optional StateDataDescription if the given command(s) modified the state type (e.g. creating a new global);
4. An updated CommandComplete message with the state as follows:

 struct CommandComplete {
   // Message type ('C').
   uint8           mtype = 0x43;

   // Length of message contents in bytes,
   // including self.
   uint32          message_length;

   // A set of message headers.
   uint16          num_headers;
   Header          headers[num_headers];

   // A bit mask of allowed capabilities.
   uint64<Capability> capabilities;

   // Command status.
   string          status;
+
+  // State data descriptor ID.
+  uuid            state_typedesc_id;
+
+  // Encoded state data.
+  bytes           state_data;
 };

If the executed command didn't change the state values, the state_typedesc_id will be set to the same special null type ID 00000000-0000-0000-0000-000000000000, indicating the state was not changed. In this case, state_data is a zero-lengthed bytes. If the state values are changed by the command, the state_typedesc_id is guaranteed to be the same as it was in the Execute request message, even if the state schema is concurrently changed in other sessions, unless this Execute is also changing the state schema.

Please note, state DDL and mutation of the state values are transactional. That means, rollback command could return a StateDataDescription message and a CommandComplete message with concrete state data, if the state schema or state values were updated in the transaction to be rolled back.

5. Client bindings guideline

The client bindings is recommended to provide special APIs for users to set state. Particularly, client.with_config(), client.with_aliases() and client.with_globals(). Because client is assumed to be safely used in multithreading or different asynchronous tasks/coroutines, EdgeQL commands that modify the state like set global, set alias, set module, configure session and their reset variants would be racy. These commands are covered in the capability session config, so the client could either disable this capability in query() and execute(), or prevent the server-returned state from affecting other threads/tasks/coroutines.

6. State type changes while state is not default

If a global type is changed while its value is set, the DDL must use set global ... set type ... using ... to instruct the server to convert the value.

If the state type is concurrently modified by other clients, StateMismatchError will be raised from the server handling the next command with an outdated state type identifier. The client should be ready to handle this error, by e.g. re-encoding the state with the codecs built from the updated state type descriptors. If the state cannot be encoded, the client could drop the connection with a proper error.

[fantix]

CHANGELOG

2022-07-07

1. CommandComplete will return state_typedesc_id.
2. CommandComplete may return empty state_data.
3. Dropped the part that commit command will always get a StateDataDescription.
4. Instead, rollback command may now get a StateDataDescription, so that the client doesn't have to maintain states for transactions.
5. State schema is now frozen in the session after accepting an Execute message, and unfrozen after returning the CommandComplete message.