Handling JSON Schema referencing in Moonwalk

[handrews]

Moonwalk imports are intended to drastically simplify OAD Document parsing and loading. JSON Schema referencing presents a challenge for this goal. This isn't so much a proposal as it is a listing of all the options I can think of. It assumes that #72 or something similar is used for imports.

NOTE: This proposal does not address any modularity that might support alternative content schema systems. Although fencing in the referencing/linking system of any format that we might embed in an OAD Document is something that we'll need to think of if we want to support multiple systems.

Importing separate JSON Schema resources

The easiest case is if all JSON Schemas are in separate documents of type application/schema+json (or some functional equivalent e.g. YAML). Since imports use IRIs, importing a JSON Schema document works just like importing an OAD Document, except with $id assigning the resource's URI instead of the OpenAPI Object's self field.

The challenge is then how to identify the exact schema to use, which may be the entire document or may be a subschema of some sort.

We'll use this schema for examples:

{
  "$schema": "https://json-schema.org/draft/2020-12/schema"
  "$id": "https://example.com/sharedSchema",
  "type": "object",
  "properties": {...},
  "$defs": {
    "FooDef": {
      "$anchor": "fooanchor",
      "type": "string"
    }
  }
}

and we'll assume our examples happen in an OAD that includes this:

openapi: 4.0.0
imports:
- namespace: shared
  href: https://example.com/sharedSchema

Accessing subschemas

For subschemas, I can think of three options for the name part of the <namespace>:<name> syntax. Two of them are based on URI fragments, and could both be used. That approach could then automatically extend to any other format that defines a URI fragment syntax.

Use names under "$defs"

  schema: shared:FooDef

PROS:

• Directly analogous to how OAD Document imports and components work

CONS:

• Not how JSON Schema works at all, producing divergent expectations and behavior depending on context
• It's not uncommon to nest multiple levels of "$defs" (we even have an issue that talks about it in the main OAS repo)
• Not obvious how to used an entire schema document with this syntax, which seems like it ought to be supported

Use names from plain name fragments as component names

  schema: shared:fooanchor

Note the lack of # as we are not using this in a URI context. For imports, it does not matter whether the fragment was created by \$anchor or \$dynamicAnchor. keywords.

PROS:

• This is the JSON Schema feature specifically intended for defining referencable location-independent schema names

CONS:

• People would have to use the keywords, which is an extra thing to do
• These keywords can appear in any subschema

Use JSON Pointers

  schema: shared:/$defs/FooDef

Again, note the plain-text, rather than fragment-encoded, JSON Pointer.

PROS:

• Familiar, flexible, and low-effort

CONS:

• Can access subschemas anywhere
• Have to deal with the potential ambiguity of $id-crossing JSON Pointers

Accessing the whole schema resource

If JSON Pointers are used, the empty string would indicate the whole document:

  schema: "shared:"

Note that the quotes are required or YAML will think it is an object key.

If plain name fragments are defined and are the only supported component name, authors can just define one in the document root schema.

If $defs names are used, using the namespace without a colon is the only idea I've had so far:

  schema: shared

Defining JSON Schemas inline

Inline schemas introduce two areas of complexity:

• how to handle same-document references from schemas
• the need to search all schema objects for $id, $anchor, and $dynamicAnchor

In particular, a same-document reference from an inline Schema Object that does not use $id can point anywhere in an OAD Document using JSON Pointer fragments, which contradicts the principle of only re-using components in the Components Object.

One thing we might want to do is place limits on inline schemas and require everything else to be placed in external documents. I can think of a few variations on this offhand:

Require a nonempty $id in the Schema Object root if $ref or $dynamicRef are present

References in inline Schema Objects that use \$id in the root object are resolved just as if the Schema Object were an external document.

PRO:

• Reduces the reference resolution problem to one that has to be solved anyway

CON:

• Need to come up with URIs for $id
• Easy to forget to add $id when adding $ref later
• Need to handle dialect switching with $schema

Forbid $ref and $dyanamicRef (and $id, $anchor, and $dynamicAnchor) in inline Schema Objects

This would enable using inline schemas for simple schemas, but push everything requiring referencing to external files and avoid the need to scan OpenAPI Documents for JSON Schema keywords that establish referencing targets.

PRO:

• Allows the sort of schemas that you really want to define inline
• Avoids having to scan OAD Documents for JSON Schema reference target-defining keywords
• Avoids the complexity of JSON Schema resources embedded in Schema Objects ($id in a subschema)
• Avoids needing to handle $schema as it is only meaningful alongside $id outside of schema document roots

CON:

• Some small schemas do benefit from referencing
• Pretty much every non-trivial OAD will need at least two documents: the entry OAD Document and a JSON Schema bundle of some sort

Different restrictions for schema components vs inline schemas elsewehre

This is kind of a combination of the previous two: forbid the referencing and identification keywords everywhere except under /components/schemas, and require $id in a schema component if $ref or $dynamicRef is used.

PRO:

• Forbidding those keywords outside of the components limits the places that have to be scanned for reference targets
• Requiring an $id keeps reference resolution equivalent to separate documents

CONS:

• Rules are more complex
• Still need to support embedded resources ($id in subschemas) and $schema... unless those are also forbidden / banished to schemas in external documents

[handrews]

Another question to consider if we allow "$ref" in inline Schema Objects (including under the Components Object): Should we require that the non-fragment part of any reference URIs match an import in the Imports Object?

PRO:

• This would ensure that the imports field lists all documents needed to resolve the current document (those documents may make additional imports or references, but the same process would apply to resolve them)

CON:

• It could be kind-of annoying and counter-intuitive
• Once you reference an external JSON Schema document, there's no way to control what it references, so imports wouldn't be able to guarantee that all necessary documents are listed

[gregsdennis]

For imports, it does not matter whether the fragment was created by \$anchor or \$dynamicAnchor. keywords.

Be aware that for the next JSON Schema version, we've planned to remove the non-dynamic behavior from $dynamicAnchor, which means it will no longer serve as a traditional anchor, effectively creating two disparate and disjoint referencing systems: one static and one dynamic. (Further reading)

While this technically wouldn't affect 2020-12 schemas that are included, this change in behavior should at least be considered for however OAS wants to use these reference targets.

[handrews]

The purpose of that sentence is to emphasize that it uses plain-name URI fragments and doesn't care what JSON Schema keywords are involved in how they are created. If a future version of JSON Schema changes which keywords create such fragments, it just uses whatever that is. If someone wants imports to support JSON Schema draft-07, then fragments are created with $id. No difference, they're just plain name fragments, that's all the imports system cares about.

[handrews]

Which also means that if you decided to make $defs names plain name fragments (although you'd have to figure out how to handle nested $defs - I'm not necessarily saying this is a good or simple idea) then that would also work. That particular option is independent of the internals of JSON Schema, and defined only in terms of URIs/IRIs.

[gregsdennis]

Different restrictions for schema components vs inline schemas elsewehre

Not sure what your feelings on implicit code are, but you could have the key under /components/schemas produce an implicit anchor that could be referenced from anywhere. That could alleviate the need to include $id or $anchor when using $ref, at least for referencing component schemas.

A pure JSON Schema implementation probably wouldn't like that, but I suppose an OAS implementation shouldn't have any issues.

[handrews]

I suspect we'll have components produce andhors like #schemas.whatever although with some syntax that is probably a bit more robust. But includes the section (semantic type) and the name. I just didn't want to clutter either #72 or this discussion with that part because both are long enough as it is. But I was planning to file it later.

I'd actually prefer to disallow JSON Pointer fragments, but there may be reasons to hold on to them.