Commands' reply schema

[guybe7]

Background

In #9656 we add a lot of information about Redis commands, but we are missing information about the replies

Motivation

1. Documentation. This is the primary goal.
2. We would like, based on the output of COMMAND, to be able to generate client code in typed languages. In order to do that, we need Redis to tell us, in detail, how each reply looks like.
3. We would like to build a fuzzer that verifies the reply structure.

Schema

The idea is to supply some sort of schema for each reply of each command.
The schema will describe the conceptual structure of the reply (for generated clients), and, in case it differs from the actual (wire) reply, it will be mentioned.
null-reply is a special case since RESP2 has two types of null-reply (null-bulk-string and null-array), so for every command that may return null, we have to mention the wire-reply.

Example for ZSCORE:

{
  "oneOf": [
    {
      "description": "Score of the member",
      "type": "number",
      "wire": {
        "resp2": "string",
        "resp3": "double"
      }
    },
    {
      "description": "Key or member does not exist",
      "type": "null",
      "wire": {
        "resp2": "null-bulk-string",
        "resp3": "null"
      }
    }
  ]
}

Example for HGETALL:

{
  "type": "object",
  "additionalProperties": {
    "type": "string"
  }
}

Note that here we don't really have to mention the wire-reply (the type is "object" which is a map, which is the return type of the command in RESP3. the only way to represent a map in RESP2 is an array, so determining the wire-reply is implicit in that case)

Example for ZRANGE:

{
  "oneOf": [
    {
      "description": "A list of member elements",
      "type": "array",
      "resp": 2,
      "items": {
        "type": "string"
      },
      "uniqueItems": true,
      "minItems": 1
    },
    {
      "type": "array",
      "description": "Members and their scores",
      "items": {
        "type": "array",
        "items": [
          {
            "description": "Member",
            "type": "string"
          },
          {
            "description": "Score",
            "type": "number"
          }
        ],
        "additionalItems": false,
        "minItems": 1
      },
      "uniqueItems": true,
      "minItems": 1
    }
  ]
}

Here, both RESP2 and RESP3 will return an array, so no need to mention the wire-reply type, but RESP3 will return exactly the array described above, while RESP2 will return a flat array of member-score. The client needs to how to convert the actual wire-reply to the reply described in the schema (see "Conversion rules" below)

Conversion rules

In addition, we will have to supply general conversion rules in this structure:
If the reply structure is THIS but instead you got THAT, here's how to convert: ...

Examples (relevant for RESP2 only):
If the reply structure is an "object" but you got an "array" the way to convert is: {arr[i]: arr[i+1]} for each even i.
If the reply structure is a "number" but you got a "string" the way to convert is: cast the string into a double

There are more complex situations that require some thinking (For example ZRANGE WITHSCORES should return an array of (member, score) tuples, but in RESP2 it's a flat array)

Thoughts

1. the example for ZRANGE doesn't cover the case of two identical members with different scores (e.g. [["m1",6],["m1",7]]) because uniqueItems compares (member,score) tuples and not just the member name. is there a way to solve this?
2. the reply schema is gonna be based on a JSON scheme. the type of a string is "string" but even in RESP3 we have 2 strings: simple-string and bulk-string. do we want to reflect that somehow in the reply-schema?

[guybe7]

@yossigo @yoav-steinberg i'd appreciate your opinion here

[yoav-steinberg]

I'm leaning towards not providing any structured reply description for RESP2. RESP2 makes the description much more complicated and also makes the generated client parsing much more complex. I think there are also cases where supporting RESP2 in the reply actually prevents us from taking advantage of RESP3 features. For example if a command can return an integer or boolean, and we want to support RESP2 we just can't do so. So there needs to be some long-term process of discarding RESP2 otherwise the advantages of RESP3 become minimal.

Having said that, assuming all we care about is RESP3, why do we even need this schema? RESP3 is structured and clients can generate a typed response just by parsing the RESP3 protocol. Another concern is that in Redis we have different reply structures based on the command arguments (like WITHSOCRES) and not just the command itself. I can't think any way a client can query the server about what to expect for a given command without also passing all the arguments. And that would be very inefficient.

[guybe7]

@yoav-steinberg thank you

I like the idea of implementing the reply-schema just for RESP3 (it'll provide motivation to slowly deprecate RESP2).

I think we would still need this feature, even if we only provide data relevant for RESP3. IIUC the idea was that, given the reply of COMMAND, I can generate the code of a Redis client in a strongly-typed language.
Imagine we would like to use C: we would need to know the reply-schema the commands in order to generate the structures needed to hold the replies.

another aspect of this feature is the have the ability to build a testing fuzzer that validates the reply type (what we report in COMMAND vs. what we actually get)

[yoav-steinberg]

For a typed-dynamic language (java) it'll be possible to generate the APIs without this. For a static-typed language (c,rust) then it won't, and you're right. It's just that I don't see how to provide the reply schema for a command in Redis given that the schema is dependent on the arguments.

[guybe7]

yes, that's true.

for example, ZRANGE can return a list of strings or a list of (string, double) tuples.
in C, that would make the return type a union. maybe it's up to the caller to know which "side" of the union to use?

[zuiderkwast]

@yoav-steinberg

assuming all we care about is RESP3, why do we even need this schema? RESP3 is structured and clients can generate a typed response just by parsing the RESP3 protocol.

If RESP3 is like JSON, then the schema is like JSON-Schema. I think we need it to generate a C struct for each object.

It's just that I don't see how to provide the reply schema for a command in Redis given that the schema is dependent on the arguments.

@guybe7

maybe it's up to the caller to know which "side" of the union to use?

This seems dangerous, but I think we need to accept "oneof", i.e. a C union in some cases and force the users to know what they're doing. In C, it's easy to mess up anyway (but in Rust it shouldn't be possible). These are corner cases anyway. AFAIK most applications only use GET, SET, DEL, HGET, HSET, DEL and a few more...

ZRANGE is really bad for this feature. We get much nicer reply schemas if we deprecate the WITHSCORES option and add a separate command ZRANGEWITHSCORES, then both commands will have simple schemas. I'm not saying we must do it now, but we can keep it in mind for later.

[yoav-steinberg]

@zuiderkwast

but in Rust it shouldn't be possible

A note about rust: Unions in rust are unsafe, simply because there's no way to guarantee how the data will be accessed. Rust's solution for this is using an enum (with encapsulated data fields) but, again, there's no way for us to know how to match these enums to the response from the command alone (without understanding the args).

If RESP3 is like JSON, then the schema is like JSON-Schema.

I understand this, but in reality we can't match the response to a given struct. We can use unions in C/rust and fixed objects in other languages but it'll be up to the user to match the response to a give object type. Something like (will vary by language):

x = myredis.zrange(arags....)
print(type(x)) // UndefinedZrangeResponse
x = x.asWithScores() // User explicitly defines which ZrangeResponse they expect (may raise an exception)
print(type(x)) // ZrangeResponseWithScores
// Do something with x

The alternative would be just to return a dynamic JSON style structured response.

[zuiderkwast]

@yoav-steinberg I agree the value in return schemas is very limited. Most commands just return a string, an array of strings, a map of strings to strings (HGETALL) or simply "OK". A special struct provide no benefit in these cases. It just increases complexity.

Thanks for reminding me of Rust enums (AKA algebraic datatypes AKA tagged unions). It would be theoretically possible to provide some form of rules mapping commands to different return schemas depending on arguments so that clients could use some logic to pick the right schema, but it would be too complex. (Idea: A list of conditional schema-rules like [{"condition": ["has_arg", "WITHSCORES"], "schema": ...}, {"condition": ["not_has_arg", "WITHSCORES"], "schema": ...}].) I don't think it's worth it.

Parsing RESP3 directly to structs can potentially use less memory for responses to commands like COMMAND and CLUSTER SLOTS but these are not preformance critical anyway. I think the complexity outweights the benefit.

[guybe7]

@oranagra @yossigo and I had a meeting on the subject.
summary:

1. we feel ok with the fact that some commands' reply structure depends on the arguments and it's the caller's responsibility to know which is the relevant one. this comes after looking at other request-reply systems like OpenAPI, where the reply schema can also be oneOf and the caller is responsible to know which schema is the relevant one.
2. the reply schemas will describe RESP3 replies only. even though RESP3 is structured, we want to use reply schema for documentation (and possibly to create a fuzzer that validates the replies)
3. for documentation, the description field will include an explanation about the scenario in which the reply is sent, including any relation to arguments. for example, for ZRANGE's two schemas we will need to state that one is with WITHSCORES and the other is without.
4. for documentation, there will be another optional field "notes" in which we will add a short description about the representation in RESP2, in case it's not trivial (RESP3's ZRANGE's nested array vs. RESP2's flat array, for example)

given the above:

1. we can generate the "return" section of all commands in redis-doc (given that "description" and "notes" are comprehensive enough)
2. we can generate a client in a strongly typed language (but the return type could be a conceptual union and the caller needs to know which schema is relevant). see the section below for RESP2 support.
3. we can create a fuzzer for RESP3.

notes about RESP2:

1. we will not be able to use the fuzzer to verify RESP2 replies (we are ok with that, it's time to accept RESP3 as the future RESP)
2. to create generated clients we will need to know how to convert the actual reply to the one expected.
   • number and boolean are always strings in RESP2 so the conversion is easy
   • objects (maps) are always a flat array in RESP2
   • others (nested array in RESP3's ZRANGE and others) will need some special handling (so the client will not be totally auto-generated)

[guybe7]

if we only describe RESP3:

{
  "oneOf": [
    {
      "description": "Score of the member",
      "type": "number"
    },
    {
      "description": "Key or member does not exist",
      "type": "null"
    }
  ]
}

Example for HGETALL:

{
  "type": "object",
  "additionalProperties": {
    "type": "string"
  }
}

Example for ZRANGE:

{
  "oneOf": [
    {
      "description": "A list of member elements",
      "type": "array",
      "items": {
        "type": "string"
      },
      "uniqueItems": true,
      "minItems": 1
    },
    {
      "type": "array",
      "description": "Members and their scores. Returned in case WITHSCORES is given",
      "notes": "In RESP2 this is returned as a flat array",
      "items": {
        "type": "array",
        "items": [
          {
            "description": "Member",
            "type": "string"
          },
          {
            "description": "Score",
            "type": "number"
          }
        ],
        "additionalItems": false,
        "minItems": 1
      },
      "uniqueItems": true,
      "minItems": 1
    }
  ]
}