Support for Streams in Responses

[emmanuelproulx]

APIs that download binary data currently must be done by type: string, format: binary. This translates to byte arrays (in Java for example, anyway that's what swagger-ui and swagger-codegen do). But this consumes large amounts of memory if the data is very big, and easily gives out-of-memory errors.

Typically huge blocks of data are sent via streams instead of byte arrays in most remote APIs. This allows for loading only a block of data at a time in memory.

The workaround used to be the 'file' format but this was removed in version 3.

What I propose is a new type: stream (format: binary) which would allow this. As an example, this could map to Spring's ResponseEntity(inputStreamResource,...) on the server side, and a okhttp's ResponseBody.byteStream() on the client side. Lots of HTTP client & server frameworks support streams in API implementations so I'm surprised it doesn't already exist.

[handrews]

@emmanuelproulx type is from JSON Schema and only refers to the data type, not how it is produced or consumed (because JSON Schema is not aware of those things, although Hyper-Schema is).

If you're talking about HTTP chunked encoding then this should be done through documenting the use of the Transfer-Encoding header, which I believe is already possible.

[emmanuelproulx]

Right. I understand. But there are times when you don't want to respond with JSON. As an example there's this standard REST API to retrieve images, IIIF ( http://iiif.io/ ) and it returns images not just JSON. I'd like to come up with a Swagger description of it but I don't know what to put in the 200 response.

responses:
'200':
content:
'image/png':
???

Sure I could put a schema with type: string + format: binary there but in the end it'll keep the entire image in memory which for sure will blow up during performance testing. Anyway that's not how it's done normally in any API framework; they all use streams.

There should be a way (not sure what the best way is) to specify that the returned entity is not JSON and should be treated as a stream. In OpenAPI 2 there was the "file" type that was used for that purpose but it's gone now.

Would you have a good idea by any chance?

[handrews]

If your content is image/png then it's not JSON. Looking at the Media Type Object the schema field is not required. So I'm not sure you even need one (although I'd defer to the TSC members on this- my expertise is mostly JSON Schema/Hyper-Schema).

If you do need one, it would have to be {"type": "string", "format": "binary"} AFAICT. If that causes something to be held in memory then that sounds like a problem with whatever tooling you are using, and you'll need to open an issue with them.

[emmanuelproulx]

Yes, as I said, right now I'm using swagger-codegen and {"type": "string", "format": "binary"} maps to a byte array. The problem is, there's no way in swagger-codegen to specify a byte array or stream because it relies on the the OpenAPI standard definition which doesn't differentiate between the two. Which is exactly why I'm suggesting we do differentiate by introducing a new type.

The alternative is to add a non-standard feature only in swagger-codegen, which is not ideal.

[handrews]

@emmanuelproulx these are orthogonal concerns- the Media Type Object is just describing the response contents. Whether or not chunked encoding is used is set in headers in the Response Object.

[darrelmiller]

@emmanuelproulx My recommendation in the past has been to use a stream by default and a byte array only if there is a maxlength defined.

[handrews]

@darrelmiller why wouldn't this be indicated through headers? I'm apparently misunderstanding something here.

[darrelmiller]

@handrews You definitely can advertise that a server supports chunk encoding for a particular response using a header declaration. And I agree chunk-encoding and client side data representation are orthogonal concerns.

If the OpenAPI description wants to hint to the client that the result can be stored in a fixed size byte array, then they should indicate that the response has a limited size using maxLength.

Arguably, you can wait until you get the content-length response header until you allocate the byte array, but arguably HTTP responses are self descriptive so OpenAPI descriptions are optional in the first place :-).

My perspective is that the current behaviour of the Swagger tools is less than optimal and they should default to a Stream datatype if the OpenAPI description doesn't limit the response size.

[handrews]

@darrelmiller makes sense, thanks!

[darrelmiller]

We discussed this in the #1585 TSC meeting and the consensus was that tooling should interpret {"type": "string", "format": "binary"} as a stream of bytes and the maxLength schema property could be used to indicate that it can be loaded into a fixed length byte array. That doesn't preclude tooling from loading fixed length data into a stream, nor using the content-length header to load a chunked payload into a fixed length array.

[AtosNicoS]

Are there any updates on this? The spec representation seems to be clarified, are we just waiting for updated tooling?

[MikeRalphson]

@AtosNicoS that's my understanding. Unless you think something like raising a Technical Note or a blog posting would help clarify matters, then probably the way forward is raising issues on affected tools if they don't already exist, and referencing this issue.

[kylebevans]

@darrelmiller

Hi, sorry to swoop in here, but say that there is an http stream of JSON, should that still be coded as {"type": "string", "format": "binary"}? There's no way to code a stream of objects, right?

[handrews]

@kylebevans
This sounds mostly like a tooling problem. And I still think that if you want to indicate streamed processing you should do so with a header (that can be described in OAS), as that is how HTTP actually tells the client and server how to behave. Tooling should support common HTTP headers, I would think. But I am not an authority on tooling!

Also, in OAS 3.1 uses the newer JSON Schema keywords contentMediaType and contentEncoding (and potentially contentSchema although the use case for that is stuff like encoding a JSON document inside a JSON string which I have seen people do for whatever reason- but that's not the same as your response being streamed instead of all-at-once JSON).

contentMediaType and contentEncoding are more flexible than format: binary and format: byte. There are numerous examples in the sections on the Media Type Object and the Encoding Object.

These apply to either type: string (in which case you can use maxLength as is done now) or an untyped schema. Untyped schemas wouldn't be used for a stream of JSON, though. They are allowed under the forthcoming JSON Schema draft that should go out with OAS 3.1- an untyped schema that indicates a different media type through contentMediaType without any contentEncoding mostly means "don't try to interpret this according to the JSON data model, but here's some information that we hope the application can use." In this case, the "application" is the OAS tooling, which can document the non-JSON-ness of things, informing the code generator to not try to force things into JSON (raw unencoded binary is usually not a legal JSON string so that was always a bit of a stretch).

[kylebevans]

@handrews thanks for such a detailed response. So to signal to a code generator to create stream processing code, I should write the spec to use a type: string with no maxLength and specify a Transfer-Encoding header? Like this?

{
  "paths" : {
    "/stream" : {
      "get" : {
        "description" : "stream text.",
        "operationId" : "stream",
        "responses" : {
          "200" : {
            "description" : "The request was successful. Successful responses will return a stream of individual JSON Text payloads.",
            "headers" : {
              "Transfer-Encoding" : {
                "schema" : {
                  "type" : "string"
                },
                "description" : "chunked"
              }
            },
            "content" : {
              "text/plain" : {
                "schema" : {
                  "$ref" : "#/components/schemas/TextStream"
                }
              }
            }
          },
          "default" : {
            "$ref" : "#/components/responses/HttpErrorResponse"
          }
        }
      }
    }
  },
  "components" : {
    "schemas" : {
      "TextStream" : {
        "type" : "string",
        "description" : "A stream of text"
      }
    }
  }
}

I think I understand, but with this way then the user will have to write the JSON processing logic. I wish there was a way to model a stream of repeated objects with a content type and a delimiter. Thanks again for your help, though. I really appreciate it.

[handrews]

@kylebevans well of course it depends on what your tooling supports. But at a glance this looks like the right approach for the header to me. If the response will always set the header to "chunked", I would add "enum": ["chunked"] (in OAS 3.0) or "const": "chunked" (in OAS 3.1) to your header schema, which would signify that the code should always assume streaming.

Are you using text/plain because you have multiple JSON documents in the stream? If so, yeah I don't know how to indicate multiple documents. Would it make more sense to make a multipart response where the parts are all JSON? If you just have one JSON document in the stream, you'd use application/json rather than text/plain.

[handrews]

But keep in mind, with the multiple JSON documents thing, this is a limitation of HTTP and not simply OpenAPI. At some point, OAS cannot be expected to describe arbitrary application conventions across all media types. You could also get fancy and declare that your text/plain document consists of an array of strings of "contentMediaType": "application/json", but no tooling is going to know what to do with that (even in 3.1 I'd expect). That sort of thing would purely be for documentation purposes and you'd have to interpret it in your application code. (Why do I know it's not supported? Because I totally made it up just now while writing this)

[kylebevans]

@handrews thank you so much for such detailed replies. Yes, it's multiple JSON documents in a neverending stream. It's actually twitter's streaming api that I'm trying to work with, and basically it's a persistent http connection that just keeps sending tweets in JSON like this forever:

{"data":{"id":"1800000000000000000","text":"mmm i like a sandwich"},"matching_rules":[{"id":1800000000000000000,"tag":"\"sandwich\""}]}
{"data":{"id":"1800000000000000001","text":"lets have a sandwich"},"matching_rules":[{"id":1800000000000000001,"tag":"\"sandwich\""}]}

Since it goes on forever, I don't think a multipart will work. However, I do think the array of strings of "contentMediaType": "application/json" has potential. The twitter API doesn't send the enclosing brackets for an array, but maybe the resulting generated code would need the least amount of tweaks.

Twitter published an openapi spec for their v2 api (https://api.twitter.com/2/openapi.json), and they basically coded it as just one of the single objects instead of an array in a neverending stream. I was able to modify the generated code from openapi-generator (Go) to get streaming to work, but it was kind of a heavy mod, and i was hoping I could just fix the spec to get better generated code.

Thanks so much for your help, you provided a lot of info, and I learned a lot about how to specify things with openapi.

[whlavina]

If I may offer an opinion:

• HTTP is fundamentally streamed I/O, whether you send the content-length a priori, or you used chunked transfer encoding (which you may do if you don't yet know the content-length).
• JSON as media type, is necessarily exactly one element, per the specification. Multiple concatenated JSON (or an empty reply) is not formally valid as application/json.
• The decision for whether to process a response after receiving the complete content vs process in streaming fashion, is a client-side decision. A streaming JSON parser could accommodate processing a single, very large HTTP response even without chunked transfer encoding. The client can choose what constitutes "very large" - for a client with GBs of memory, a large maxLength is not an obstacle. Certainly, it is more convenient for a client to use a JSON object once it has been fully read, without error, parsed into memory, and passed back as e.g. a Python object (array, dict, or primitive type) rather than some streamed/iterable accessor object and the worries of getting half-way through the reply, before discovering there was an error in the reply and having to deal with the partial work.

Thus, the generated APIs should provide both options to the caller, fully parsed vs streamed access, regardless of the server's response. Whether to process as a stream or not, is really a coding/algorithm issue for the server and client to do, each independently; the fundamental protocol between them, is an octet stream.

[ePaul]

I think we have two issues here:

• If there is one JSON document which just won't ever be finished (i.e. new elements to an array, or new key-value pairs in an object are being added), the application/json content type applies and the usual JSON schema works, but it would be nice for a client-generator to know that a streaming JSON parser (and an API which provides access before parsing is done) needs to be used.

• I we have a series of JSON documents, in whatever framing format (line-delimited seems to be common), then describing it as application/json is wrong. And OpenAPI doesn't really provide a way to describe that it's a stream of X (where X matches a schema).

  As an example, Nakadi's API (Nakadi is my Company's internal event bus) just describes this behavior in the description field of the response (so in the generated documentation you don't even see the content schema).

  For this case, having something like type: stream analogously to type: array seems useful. A client could then parse the stream elements one-by-one using the schema, and pass them to the application as they come.

[chirino]

Another example of an API that sends a never ending stream of json objects (like the twitter api) is when you you watch Kubernetes resources. More details can be found at:
https://kubernetes.io/docs/reference/using-api/api-concepts/#efficient-detection-of-changes

This seems common enough that it should have a simple openapi way to describe it.

[dimaqq]

I too would like json-lines with typed documents.

Today the alternative is to implement pagination which is its own can of worms 🙈

[spacether]

Could this proposal work for streams of bytes?
#3024

In it I propose adding:
opeanpi-type: binary to store definition of bytes in a schema.

[handrews]

Note that there is now RFC 7464 JavaScript Object Notation (JSON) Text Sequences that defines application/json-seq for exactly this sort of use case.

With that in mind, is there anything that still needs to be clarified in the spec? I'm tagging things for a possible 3.0.4 release but I'm a little unclear on whether there's more to do here.

[nfroidure]

Anything is streamable, in fact, the question is do we buffer it or not. I think it could be a framework/sdk option to threat contents as streams or buffer or at least, be streams per default and provide helpers to buffer them. From the OpenAPI point of view, I don't think it is needed to overweight the spec with those concerns.

[handrews]

@darrelmiller I'm a bit confused on what does or does not need to be done with this issue. You made a comment about discussion in a TSC meeting about type: string, format: binary, but there was a subsequent question about doing such a thing with non-binary JSON data.

I'm trying to sort out various format/content* confusions for 3.0.4. Could you clarify if there is something to be done here?

In #1544 (comment) I propose noting that format: binary is meaningless except where unencoded binary data is possible. Of course we can treat streamed JSON (or JSON Text Sequences) as binary, but as the question above notes, that's losing information about the underlying media type.

[handrews]

I've filed:

• PR Clarify how to model streaming binary data #3729 (to address the original concern of this issue in a patch release- this issue will be closed when the PR is accepted)
• Issue Support application/json-seq and similar JSON-based sequential formats #3730 (to track JSON streaming, which will need to go in the next minor release)

[handrews]

PRs merged for 3.0.4, 3.1.1, and 3.2.0- closing!