Does OpenAPI define the outcome for requests with valid or invalid input?

[hudlow]

This topic spun out of a conversation with @mikekistler concerning the proposal for itemSchema in support of streaming media types.

Mike asked what I thought was a good question. If I have a schema and an itemSchema, the itemSchema will tell me in isolation whether an individual item in a stream is valid; but if I also include a schema does it necessarily imply that if, after the stream ends, the schema indicates that the collective content is not valid, does that mean that the service should not have processed any of the items? (Or maybe, should roll back any changes.)

I am not sure what the answer ought to be, but I think there's a broader question here that's interesting.

As a best practice, I would guide API authors/implementors to define and implement request schemas (for content, parameters, headers, whatever) such that if input is considered invalid by a schema, the request will fail with 4xx and without any side effects.

But I would also say that as a matter of pragmatism, it will likely be impossible for them to guarantee the converse: that if all the input validates against the corresponding schemas, the request will succeed with a 2xx status code (aside from unpredictable 5xx failures). For the APIs I work in every day, there are usually validation aspects that cannot be defined in the schema. Compatibility with existing resource state, for example. A basic example of this would be using If-Match where the success of the request is not determined by the schema of the value provided for the header, but by how it relates to the state on the server.

But I don't think OpenAPI actually says how a service implementation ought to behave with respect to request data validating or not validating against a schema. I think it just says "this is the schema for the input" and leaves the rest to the reader.

A similar question might be asked about whether OpenAPI specifically says what response schemas mean with respect to behavior.

I anticipate that the right thing to do here is to leave some flexibility for an author/implementor to define the specifics of how a schema is used, but I think we'd do better to define which parts are left to the author/implementor. I would also be supportive of nailing down some aspects of this with respect to what a schema validation failure means even if we can't nail down anything about what schema validation success means.

But someone might, even outside of a streaming use case, might say that they want to define a correct schema but they also want to have a service which does its best to fulfill a request that has invalid data. I don't like the idea but I can certainly imagine the argument and the desirability of keeping that use case under the OpenAPI tent.

[mikekistler]

Thank you for opening this discussion!

You make a great point that a request that satisfies the input schema is not guaranteed to succeed.

When a request fails to satisfy the input schema, it might be desirable for these to fail with a 400 status code, but I don't think we could ever require this, as I think that would violate the "Open World principle" that I believe is a foundation of OpenAPI. Which is why I tried to frame the discussion in the TDC today around "expectations". If it is okay for an operation to succeed, or partially succeed, even when it fails to satisfy the input schema, then I think the main value of the schema (to users of the API) is to convey how users should expect the operation to behave when validation succeeds or fails.

[karenetheridge]

Yes, I don't think we can prescribe that a particular response should be generated given a particular validation result. For instance, one might only be using the OpenAPI description as documentation, and not actually performing validation of the live request, or perhaps validation is only performed in a testing environment and not in production (or perhaps only generate log warnings in production but do not alter the response generated by the server).

There's also the question of validation of responses. Given that (in a server environment) the response is generated by the same system that is performing the validation, we should expect that the response should always validate. But we may still wish to perform this validation step in a testing environment, or again as above only generate logging errors on failure.

Also keep in mind that we may wish to perform validation on behalf of an HTTP client, not a server: in this situation, the scenarios are reversed from above: the request is generated by the local system and is always expected to validate, and the response is generated from the other side of the network connection and may need to be validated before the application processes it.

As for responses generated by validation failure, my implementation produces response codes corresponding to various error conditions -- not just HTTP 400 (indicating generic validation failure) is represented, but also:

• 404 Not Found (when an incoming request fails to match against any /paths entries)
• 411 Length Required (when the Content-Length header is missing and there is a request/response body)
• 500 (when the request/response cannot be fully parsed, i.e. it is RFC noncompliant), or an exception occurred in the validation process
• 405 Method Not Allowed (when a /paths entry matches, but there is no operation representing the request method)
• 415 Unsupported Media Type (an operation does not describe the request/response's media type)