New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal: x-oas-draft-alternativeSchemas #1532

Open
darrelmiller opened this Issue Apr 9, 2018 · 67 comments

Comments

Projects
None yet
10 participants
@darrelmiller
Member

darrelmiller commented Apr 9, 2018

A long standing request has been to add support for alternate schemas. See #764 #1443

There are many concerns about the effect this will have on the tooling ecosystem, however, there is also a strong desire from the community to support both more recent versions of JSON Schema and other formats such as XSD Schema, Protobuf Schema.

Instead of simply adding this feature to the specification and hoping that tooling vendors will implement it. I propose we use the draft feature (PR #1531 ) process to validate the value and feasibility of this feature.

openapi: 3.0.2
info:
  title: A sample using real JSON Schema and xsd
  version: 1.0.0
paths:
  /:
    get:
      responses:
        '200':
          description: Ok
          content:
            application/json:
              x-oas-draft-alternate-schema:
                type: json-schema
                externalValue: ./rootschema.json
            application/xml:
              x-oas-draft-alternate-schema:
                type: xml-schema
                externalValue: ./rootschema.xsd
  • The property is called alternate-schema because it can be used in instead of, or in addition to the OAS Schema object. If both are present then both schemas must be respected.
  • The type field is required and must match one of the values identified in an alternate schema registry which will be created as part of this proposal. The alternate schema registry will provide a link to a specification for the schema file.
  • It is recommended that tools provide users with warnings when they encounter an alternate schema type that they do not support. Tools should not fail to process a description, unless the schema is essential to the operation. Ideally they should continue as if the alternate-schema is not present in a form of graceful degradation.
  • The externalValue property is required and must be a relative or absolute URL.
  • The alternate-schema property can be used anywhere the schema property can be used.
@philsturgeon

This comment has been minimized.

Contributor

philsturgeon commented Apr 9, 2018

Where did we land on alternate schemas various versions? Saying "json-schema" here might be fine, as JSON Schema files contain "$schema": "http://json-schema.org/draft-XX/schema#", and Protobuf has syntax = "proto3";, but other formats might not?

It would be up to tool vendors to sniff out the right version, which might cause trouble.

Adding the version in the file does add a lot more work for everyone. When users upgrade their JSON Schema files, they would have to update all the OpenAPI references too, which doesn't sound ideal. Tool vendors would also need to update their registries of "supported alternate schemas" just to support a newer JSON Schema draft, which the underlying tools might already support just fine.

I'm conflicted. Probably leave them out and assume that formats declare their versions?

@handrews

This comment has been minimized.

handrews commented Apr 9, 2018

Please, please, PLEASE do not make people specify the JSON Schema draft version. Right now that's fairly straightforward, but as we move towards full vocabulary support including all of the things that will facilitate code generation, this will become far too complex. The whole point is to allow schema authors and vendors more flexibility in using and detecting support for vocabularies. Please don't duplicate the entire effort inside of OpenAPI.

@darrelmiller

This comment has been minimized.

Member

darrelmiller commented Apr 9, 2018

@philsturgeon Can we identify existing schema languages that are not clearly self-description as to version? I'm going to guess all the XML ones will use a namespace. JCR doesn't seem to have any way to version stuff, but then it is still draft.

Personally I'm a fan of formats that identify their own version. There is something icky about relying on external metadata. So, I wouldn't be opposed to just not offering a way to specify a version.

@philsturgeon

This comment has been minimized.

Contributor

philsturgeon commented Apr 9, 2018

I am no longer conflicted. 👍🏼

@handrews

This comment has been minimized.

handrews commented Apr 10, 2018

@darrelmiller do people ask about JCR in OpenAPI? It's on draft-09 but still lists only one complete implementation. No need for a big tangent here, just curious as to whether it comes up a lot or is just the first JSON-related example you thought of :-)

@darrelmiller

This comment has been minimized.

Member

darrelmiller commented Apr 10, 2018

@handrews It's the only other schema language in JSON that I know of. And no, nobody has ever asked for support.

@MikeRalphson

This comment has been minimized.

Member

MikeRalphson commented Apr 10, 2018

There is also schematype for Yaml which has come up before as a possible alternate schema-description language, though I'm not sure about requests for support.

@handrews

This comment has been minimized.

handrews commented Apr 10, 2018

the only other schema language in JSON

Technically "for JSON", as its syntax is not JSON, merely "JSON-like", possessing the conciseness and utility that has made JSON popular. (oh wait, I'm being pedantic again, oops...)

@MikeRalphson

This comment has been minimized.

Member

MikeRalphson commented Apr 10, 2018

As the guidance for the json schema $schema keyword is

The "$schema" keyword SHOULD be used in a root schema. It MUST NOT appear in subschemas.

I don't think we need any additional recommendation that people declare their json schema version. Presumably the $schema keyword is 'in scope' / visible the same way $refs outside the fragment being $ref'd by the OpenAPI document are resolvable.

Unless the fragment being $ref'd from the OpenAPI document counts as a "root schema"?

@handrews

This comment has been minimized.

handrews commented Apr 10, 2018

@MikeRalphson "root schema" is the root of a "schema document" which is a whole resource. If you are referencing a fragment other than "#" then you're referencing a subschema.

Certain concepts, most notably $schema and base URI, work based on the schema document scope, not how you got to a schema object.

@handrews

This comment has been minimized.

handrews commented Apr 10, 2018

I think with the vocabulary stuff in draft-08 we'll be clarifying the schema document / file-scope stuff a lot, including how $schema and the closely related proposed $vocabulary work.

@MikeRalphson

This comment has been minimized.

Member

MikeRalphson commented Apr 10, 2018

Thanks @handrews

@darrelmiller

This comment has been minimized.

Member

darrelmiller commented Apr 23, 2018

UPDATED Proposal based on feedback from TSC meeting of April 16th.

This proposal is to enable to use of schemas other than the native OAS Schema object for describing request and response payloads and complex parameter/header structures.

openapi: 3.0.2
info:
  title: A sample using real JSON Schema and xsd
  version: 1.0.0
paths:
  /cat:
    get:
      responses:
        '200':
          description: Ok
          content:
            application/json:
              schema:
                type: object
              x-oas-draft-alternateSchemas:
                - type: json-schema
                  externalValue: ./cat-schema-draft-08.json
                - type: json-schema
                  externalValue: ./cat-schema-draft-05.json
                - type: json-content-rules
                  externalValue: ./cat-JCRschema.json
            application/xml:
              x-oas-draft-alternateSchemas:
                - type: xml-schema
                  externalValue: ./cat-schema.xsd
                - type: relax-ng
                  externalValue: ./cat-schema.ng
                - type: schematron
                  externalValue: ./cat-schema.st
  • The property is called alternate-schemas because they can be used in instead of, or in addition to the OAS Schema object. If both OAS Schema and alternate schemas are present then both the OAS schema and one of the alternate schemas SHOULD be respected.
  • The alternate-schemas property is a list of schemas. They SHOULD be processed in the order included in the list. Tooling is expected to walk the list until it finds an alternate schema that it knows how to process. Multiple alternate-schemas are supported to enable a single specification be used to drive tools with different schema support capababilities.
  • The type field is required and must match one of the values identified in an alternate schema registry which will be created as part of this proposal. The alternate schema registry will provide a link to a specification for the schema file.
  • It is possible for the same schema type to appear multiple times in the list of supported schemas. This is to provide support for different versions of the same schema type.
  • It is recommended that tools provide users with warnings when they are unable to find an alternate schema that they support. Tools should not fail to process a description, unless the schema is essential to the operation. Ideally they should continue as if the alternate-schema is not present in a form of graceful degradation.
  • The externalValue property is required and must be a relative or absolute URL.
  • The alternate-schema property can be used anywhere the schema property can be used.
  • The use of an array of alternate schemas provides the ability for x- extensions to be included in the alternate schema information.
@cmheazel

This comment has been minimized.

cmheazel commented Apr 23, 2018

In this example shouldn't the externalValue property reference the schema for the returned content? For example, if I want to return GeoJSON (an IETF standard) then the externalValue could resolve to https://github.com/fge/sample-json-schemas/blob/master/geojson/geojson.json. The client would then validate the returned content against the GeoJSON schema.

@darrelmiller

This comment has been minimized.

Member

darrelmiller commented Apr 23, 2018

@cmheazel Yes you are correct. I see how my example file names are confusing. I'll fix that.

@cmheazel

This comment has been minimized.

cmheazel commented Apr 23, 2018

@darrelmiller here are some candidates (in addition to GeoJSON)

Geography Markup Language (GML) - MIME type application/gml+xml
Version 3.2.1 = http://schemas.opengis.net/gml/3.2.1/gml.xsd
Version 3.1.1 = http://schemas.opengis.net/gml/3.1.1/base/gml.xsd
Version 3.0.0 = http://schemas.opengis.net/gml/3.0.0/base/gml.xsd

And an in-development JSON encoding of the GML data model:
http://example.com/GML-SF0-JSON.json

@cmheazel

This comment has been minimized.

cmheazel commented Apr 23, 2018

Another question - XML validation can be a two step process; XML schema validation followed by XML content validation (Schematron). Would it be legal to have two externalValue properties? Validation would be the AND of the two.

@cmheazel

This comment has been minimized.

cmheazel commented May 7, 2018

I have some folks who want to use this feature in a RESTful Web Services standard. When can we lock this feature down and establish a timeline for adoption?

@darrelmiller darrelmiller changed the title from Proposal: x-oas-draft-alternate-schema to Proposal: x-oas-draft-alternateSchema May 13, 2018

@darrelmiller

This comment has been minimized.

Member

darrelmiller commented May 14, 2018

This a proposal to add a new field called alternativeSchemas to the Media Type Object. We are considering the possibility of also allowing it to be added to the Parameter Object for simple parameter definitions. However, we will wait for implementer feedback on that issue.

Media Type Object

Fixed Fields
Field Name Type Description
alternativeSchemas [alternative Schema Object] List of alternative schemas. These schemas can be used in addition to, or as an alternative to any existing schema property. If both OAS Schema and alternative schemas are present then both the OAS schema and one of the alternative schemas SHOULD be respected. Alternative schemas MUST be processed in order. It is sufficient for tooling to process just the first alternative schema they are capable of processing in order to consider the content as valid. If tooling cannot process any alternative schemas, then they MAY issue a warning, but MUST not report the OpenAPI description as invalid.

Alternative Schema Object

This object makes it possible to reference an external file that contains a schema that does not follow the OAS specification.

Fixed Fields
Field Name Type Description
type string REQUIRED. The value MUST match one of the values identified in the alternative Schema Registry name of the tag.
externalValue url REQUIRED. This is a absolute or relative reference to an external file containing a schema of a known type.

This object MAY be extended with Specification Extensions.

Alternative Schema Registry

The Alternative Schema Registry is located at https://spec.openapis.org/registries/alternative-schema. Inital contents of the registry include:

Name Link Description
jsonSchema
xmlSchema
@handrews

This comment has been minimized.

handrews commented May 18, 2018

@tedepstein

If the Schema Object does specify constraining properties, then the content must be JSON, and must conform to the native Schema Object constraints, as well as any alternative schemas.

JSON Schema does not require the content to be JSON, only that the content can be mapped into the data model, which is derived from JSON. Some media types, such as YAML as long as it doesn't use the more exotic features, are trivial to map, while others (XML) do not have an obvious mapping.

(Maybe) if the Schema Object specifies non-constraining properties like title, definition, and documentation those become part of the schema object definition, but do not require the content to be valid JSON.

This gets into the difference between assertions and annotations. How annotations get used is application-defined, so OAS can say anything it wants about that.

@handrews

This comment has been minimized.

handrews commented May 18, 2018

@tedepstein I am definitely against a "usage" concept. With multi-vocabulary support in draft-08, the vocabulary in use should indicate its purpose. Of course applications can attempt to do whatever they want with a schema (such as generate code from the validation vocabulary alone, even though it's not well-suited for such things), but the way you'd tell if a schema is intended for code generation is to check whether it uses the code generation vocabulary.

@darrelmiller

This comment has been minimized.

Member

darrelmiller commented May 18, 2018

@tedepstein Regarding this:

When we say both schemas must be respected,
Perhaps a better word would have been "processed" instead of "respected". If the tool doesn't understand an alternativeSchema, it cannot be expected to guarantee that it validates.

The idea of adding a usage property concerns me significantly. The application of schemas are currently vauge and I actually think that is a feature. I think any attempt to nail down an enumeration of usages would be both insanely challenging and ultimately restrictive.

The alternativeSchema feature is an escape hatch for people who know what they are doing, understand the limitations of OAS Schema and are aware of the limited tooling support for alternative schemas.

OAS schema objects are already used for non-JSON payloads, e.g. form encoding.

I believe if OAS schema constraints exist, their impact will be unaffected by the addition of the new alternativeSchema property. alternativeSchema is always in addtion to the OAS Schema object. This should never cause a problem because an empty OAS Schema Object allows everything.

If you look at the example snippets I gave in my last comment, the one labelled Layered Validation shows how allOf could be used to address the scenario @cmheazel brought up with Xml Schema and Schematron.

@darrelmiller

This comment has been minimized.

Member

darrelmiller commented May 18, 2018

So far, I like the idea of adding alternativeSchema to OAS Schema because:

  • It makes $refs just work
  • It makes alternativeSchema an object instead of a list, making the simple case even simpler
  • It enables allOf and anyOf scenarios covering both reasons for having multiple alternative schemas
  • It enables single representations to be validated by multiple schema languages. e.g. folks who embed encoded XML in JSON properties could actually validate that XML.

The one thing I'm not sure of is whether JSON Schema's rules for processing oneOf match what we were propsing. i.e. Walk the array in order and stop as soon as you find one that works.

I do think we should be explicit in indicating that externalValue can have a fragment identifier and therefore can point to a portion of a target document.

The only remaning issue I see is that not having a schema version property is going to force tools to load a schema to learn whether they support that version. That may be prohibitively expensive. So we probably need a generic way of identifying a version of a schema.

@tedepstein

This comment has been minimized.

Contributor

tedepstein commented May 18, 2018

The one thing I'm not sure of is whether JSON Schema's rules for processing oneOf match what we were propsing. i.e. Walk the array in order and stop as soon as you find one that works.

It sounds like anyOf would be a better fit for this.

@philsturgeon

This comment has been minimized.

Contributor

philsturgeon commented May 18, 2018

I'm worried that we're overthinking some of this, and this thread feels like we're jumping the shark. 🏄🦈

I really liked the initial proposal, as defined here my Darrel in this comment.

@darrelmiller

This comment has been minimized.

Member

darrelmiller commented May 18, 2018

@philsturgeon You don't see the $ref issue of having to keep the $ref'd schema in sync with alternate schema externalValue to be problematic? Especially if the $ref'd OAS schema is actually in another file.

@tedepstein

This comment has been minimized.

Contributor

tedepstein commented May 18, 2018

I’m really liking the level of integration afforded by making alternative schemas a property of the Schema Object, especially the possibilities of using Boolean assertions for composition: allOf, anyOf, oneOf, etc.

I posed bunch of questions and concerns in previous comments, before I had fully taken in this new idea. Now, I actually think I can go back and answer many of my own questions, using the new structure.

@handrews

This comment has been minimized.

handrews commented May 18, 2018

@darrelmiller oneOf cannot be evaluated in the way that you mention, as you need to verify that exactly one alternative works. anyOf can be short-circuited for assertion processing, but not for annotation processing. You know that the assertion result is true as soon as one subschema validates, but for applications that care about annotations (and code generation will be one such application), you need to examine all subschemas as all annotations from valid subschemas must be collected.

If you want a true "stop at first usable schema" approach then that needs to be an OAS extension keyword. Redefining the behavior of oneOf/anyOf would be very confusing.

You don't see the $ref issue of having to keep the $ref'd schema in sync with alternate schema externalValue to be problematic? Especially if the $ref'd OAS schema is actually in another file.

I'm confused on this- aren't all of the alternate schemas in external files? Why would the OAS schema be any harder to keep in sync? I may have missed something here...

@handrews

This comment has been minimized.

handrews commented May 18, 2018

The $ref would imply that it's a JSON Reference, which is not necessarily what we want in all cases

In draft-08 we are redefining $ref from "logically replace this with the target" to "this has the same results as evaluating the target", in part to ensure that you can $ref between schemas that use different processing rules. So OAS could adopt that definition for the OAS-specific schema when $ref is used with x-oas-draft-alternateSchema (to avoid breaking compatibility elsewhere).

@tedepstein

This comment has been minimized.

Contributor

tedepstein commented May 19, 2018

@handrews wrote:

OAS cannot define fragment syntax and semantics, in a registry or otherwise. Only the media type of the target representation can define those semantics. JSON Schema defines semantics for JSON Pointer and plain name fragment syntax. XML defines fragment based on the XPointer framework. These are not re-definable.

OK, makes perfect sense that it's defined in the media type. As long as fragment resolution is defined somewhere, I'm happy.

@tedepstein

This comment has been minimized.

Contributor

tedepstein commented May 20, 2018

Closing the loop on some of my earlier comments, to the extent that I think these are addressed by moving alternative schemas into Schema Object...

...can you include an OAS Schema Object (i.e. a "standard" OpenAPI schema) in alternateSchemas list, as a way to specify its order of preference with respect to other alternateSchemas?

Yes, using anyOf (or a new boolean logic assertion, TBD...)

anyOf:
  - alternative-schema:
      type:  jsonSchema 
      externalValue: ./myschema-draft-07.json
  - alternative-schema:
      type:  jsonSchema 
      externalValue: ./myschema-draft-06.json
  - type: object
      properties: 
        foo:
          type: string
     ...

If you don't do this, but you do specify both schema and alternativeSchemas, is the standard schema entry assumed to go at the end of the list, as the least preferred option?

IIUC, including an alternative schema in a Schema Object that also has assertions is equivalent to combining the two with allOf. These two Schema Objects should be equivalent:

Explicit allOf:

allOf:
  - type: object
    properties: 
      foo:
        type: string
     ...
  - alternative-schema:
      type:  jsonSchema 
      externalValue: ./myschema-draft-07.json

Implicit allOf:

  type: object
  properties: 
    foo:
      type: string
     ...
  alternative-schema:
      type:  jsonSchema 
      externalValue: ./myschema-draft-07.json

This was true before the refactoring of alternative schemas into Schema Object. The semantics of a Schema Object with assertions+alternative schema are the same as the previously specified semantics of using schema+x-oas-draft-alternativeSchemas.

But I think it's clearer now, for reasons I'll explain in the next point...

The property is called alternative-schema because it can be used in instead of, or in addition to the OAS Schema object. If both are present then both schemas must be respected.

When we say both schemas must be respected, that means the content must conform to both schemas in order to be valid. But it doesn't necessarily mean that implementations are required to validate against both schemas, or otherwise "use" both schemas. Is that right?

That's from the earlier design.

But now we have (or will have) a way to explicitly place a native OAS schema in a list, in preferential order relative to other schemas. The implicit and explicit allOf semantics are clearer now (to me, anyway) because they don't need to address the order-of-preference use case. That use case is handily covered by anyOf.

So it's clear that validators and other processors must always process all of the native OAS schema assertions, annotations, and/or alternative schema provided in an allOf scope, whether explicit or implicit.

@tedepstein

This comment has been minimized.

Contributor

tedepstein commented May 20, 2018

@handrews,

oneOf cannot be evaluated in the way that you mention, as you need to verify that exactly one alternative works. anyOf can be short-circuited for assertion processing, but not for annotation processing. You know that the assertion result is true as soon as one subschema validates, but for applications that care about annotations (and code generation will be one such application), you need to examine all subschemas as all annotations from valid subschemas must be collected.

OK, I see that requirement specified here, and I understand the need for it.

Also, anyOf doesn't imply an order of preference, so processors could evaluate the subschemas in any order, and short-circuit validation after evaluating a subschema that is not the most preferred.

If you want a true "stop at first usable schema" approach then that needs to be an OAS extension keyword. Redefining the behavior of oneOf/anyOf would be very confusing.

I think this is important, both to allow more efficient validation, and to avoid the potential complexities of handling conflicting annotations from multiple subschemas.

But these concerns don't seem unique to OpenAPI. If we introduced a new firstOf or firstOneOf that did exactly what we're describing here, would JSON Schema consider adding it to a future draft?

@tedepstein

This comment has been minimized.

Contributor

tedepstein commented May 20, 2018

Regarding usage: I'll withdraw this proposal.

I'm still not entirely comfortable that we have fully defined how native JSON Schema/Schema Object assertions and annotations are supposed to be applied to content that is not JSON, and does not have a well-defined, authoritative mapping to JSON (like YAML).

I'll try to describe those concerns in a later post or on Monday's call. For now, I just want to pull usage off the table, so @darrelmiller and @handrews can sleep better. ;-)

@darrelmiller

This comment has been minimized.

Member

darrelmiller commented May 20, 2018

UPDATED

This a proposal to add a new field called alternativeSchema to the OAS Schema Object. While still in draft, the field name will be prefixed with x-oas-draft-.

Schema Object

Fixed Fields
Field Name Type Description
x-oas-draft-alternativeSchema alternative Schema Object An external schema that participates in the validation of content along with other schema keywords.

Alternative Schema Object

This object makes it possible to reference an external file that contains a schema that does not follow the OAS specification. If tooling does not support the type, tooling MUST consider the content valid but SHOULD provide a warning that the alternative schema was not processed.

Fixed Fields
Field Name Type Description
type string REQUIRED. The value MUST match one of the values identified in the alternative Schema Registry.
location url REQUIRED. This is a absolute or relative reference to an external resource containing a schema of a known type. This reference may contain a fragment identifier to reference only a subset of an external document.

This object MAY be extended with Specification Extensions.

Examples

Minimalist usage of alternative schema:

schema:
  x-oas-draft-alternativeSchema:
    type: jsonSchema
    location: ./real-jsonschema.json

Combination of OAS schema and alternative:

schema:
  type: object
  nullable: true
  x-oas-draft-alternativeSchema:
    type: jsonSchema
    location: ./real-jsonschema.json

Multiple different versions of alternative schema:

schema:
  anyOf:
  - x-oas-draft-alternativeSchema:
      type: jsonSchema
      location: ./real-jsonschema-08.json
  - x-oas-draft-alternativeSchema:
      type: jsonSchema
      location: ./real-jsonschema-07.json

Combined alternative schemas:

schema:
  allOf:
  - x-oas-draft-alternativeSchema:
      type: xmlSchema
      location: ./xmlSchema.xsd
  - x-oas-draft-alternativeSchema:
      type: schematron
      location: ./schema.sch

Mixed OAS schema and alternative schema:

schema:
  type: array
  items:
    x-oas-draft-alternativeSchema:
        type: jsonSchema
        location: ./real-jsonschema.json

Alternative Schema Registry

*** Note this is a placeholder registry. Don't take the values seriously. ***
The Alternative Schema Registry is located at https://spec.openapis.org/registries/alternative-schema. Inital contents of the registry include:

Name Link Description
jsonSchema
xsdSchema
@earth2marsh

This comment has been minimized.

Member

earth2marsh commented May 20, 2018

Minor nit: does "Value" add anything in the name externalValue? Every name has a value, ideally, after all. :)

Suggest externalSchema to better align with externalDocs. WDYT?

@darrelmiller

This comment has been minimized.

Member

darrelmiller commented May 20, 2018

@earth2marsh externalValue was chosen to match the way example object works. ExternalDocs is the containing object that has a url property. We could take that approach, but I feel like externalValue is more explicit.

@tedepstein

This comment has been minimized.

Contributor

tedepstein commented May 21, 2018

@darrelmiller, in the example labeled "Combined alternative schemas", shouldn't it be allOf rather than anyOf?

I'm assuming this is supposed to work the same way as the previous "Layered Validation" example. The two schemas are complementary, covering different aspects of validation, and are intended to be used in combination.

I'm also assuming that the following rule would apply in allOf scenarios like this:

It is recommended that tools provide users with warnings when they are unable to find an alternate schema that they support. Tools should not fail to process a description, unless the schema is essential to the operation. Ideally they should continue as if the alternate-schema is not present in a form of graceful degradation.

So if the processor only supports some alternative schemas in an allOf group, it should process the ones it can, and warn on the others.

@earth2marsh

This comment has been minimized.

Member

earth2marsh commented May 21, 2018

@darrelmiller externalValue in examples complements the in-line value, plus it's actually an example value. In the case of alternativeSchema (an aside, wondering if altSchema would be worth considering?), the item being referenced, while it certainly has a value (as should any reference) it really is an external schema, hence my pref for externalSchema as the name.

@handrews

This comment has been minimized.

handrews commented May 23, 2018

and to avoid the potential complexities of handling conflicting annotations from multiple subschemas.

We are further formalizing how annotations are collected in draft-08, building on the work that is already present in draft-07. There are no problems with "conflicting" annotations, the collection process provides applications (in this case, OAS implementations) with enough information to disambiguate the values (or just keep them all, or whatever).

But these concerns don't seem unique to OpenAPI. If we introduced a new firstOf or firstOneOf that did exactly what we're describing here, would JSON Schema consider adding it to a future draft?

Unlikely. I'm not aware of this ever coming up before, and I have read a huge number of JSON Schema issue and feature requests. TBH I'm not even sold on the idea that OAS needs it. If you want to do validation only and not annotations, the spec already allows for short-circuit validation in such cases. If you want to do annotations, there is no conceivable way to short-circuit anyOf, contains, etc.

You can still short-circuit allOf on the first failure (as all annotations are dropped), and you can still short-circuit oneOf on the second successful validation (which fails overall, again meaning that all annotations are dropped). The annotation collection/dropping conditions will be more explicit in draft-08.

@tedepstein

This comment has been minimized.

Contributor

tedepstein commented May 23, 2018

TBH I'm not even sold on the idea that OAS needs it. If you want to do validation only and not annotations, the spec already allows for short-circuit validation in such cases. If you want to do annotations, there is no conceivable way to short-circuit anyOf, contains, etc.

That's true for the intended anyOf use case. The logic says, for each subschema, if the instance validates against that schema, then include its annotations.

In our firstOf case, we assume that each of the schemas in the list is sufficient; so once the processor finds a schema it can understand, it doesn't need to visit other schemas in the list.

Clearly our use case is different. JSON Schema's boolean assertions (at least through draft-07) naturally assume the processor understands the entire schema language, so the conditional logic is all about whether the instance validates. In our case, we're asking a different question - whether the subschema is usable by the processor. If so, we use that schema, and no others. I don't think it's our intent to proceed to the next schema, if we've already found a schema we can use, and the instance has already failed validation against that schema.

IMO, this justifies introducing a firstOf assertion, rather than trying to repurpose anyOf. But I can understand that this might not be of interest outside of OpenAPI.

@handrews

This comment has been minimized.

handrews commented May 23, 2018

@tedepstein the use case is so specific that I wouldn't even call it firstOf, as that sounds like a thing that people could use in general or could be in the regular schema vocabulary. I'd call it schemaList or something else very tied to how it is being used and why it behaves that specific way.

@zolyfarkas

This comment has been minimized.

zolyfarkas commented Sep 2, 2018

Can we also add avro(https://avro.apache.org) as alternative schema's, and also a way to provide the schema inline?

Here is some detail on my use case:

We currently use avro schemas to declare the payload of our REST services. Avro schemas are JSON and as such we currently inline them inside the swagger JSON via custom x-avro-schema field. Every avro REST endpoint we have, depending on the Accept provided by the client (application/json or application/octet-stream) will return the content either as JSON or as AVRO binary, this is very important where the content is large (array) (binary payload can be an order of magnitude smaller than the json payload). Although you can have a limited conversion of a avro schema into a swagger model, the 2 formats do not have feature parity and probably never will, it would be really useful to have a more "official" way of declaring different schema types...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment