Proposal: Vocabularies are Always Mandatory

[gregsdennis]

Previously, on JSON Schema Discussions:

• Dropping support for unknown keywords was proposed and mostly accepted
• Unknown keywords are still going to be kinda supported
• Other stuff that's irrelevant to this discussion

Currently, a vocabulary is listed in a meta-schema by URI ID along with a boolean value that indicates the required behavior for an implementation that does not understand the vocabulary:

• If the value for the vocabulary is true, then the implementation MUST refuse to process the schema.
• If the value for the vocabulary is false, then the implementation MAY continue to process the schema.

The second case, false, means the vocabulary is considered "optional" in that it's not required that the implementation understand the vocabulary.

However, now that we've said unknown keywords are disallowed, we have a conundrum:

The keywords in an unknown vocabulary are themselves unknown, and implementations MUST refuse to process them. But this means that implementations no longer have the option to continue processing. Therefore the false case now indirectly requires the same behavior as the true case.

Thus, if implementations must refuse to process schemas with unknown keywords, a vocabulary can't really be optional, and the true/false values are meaningless.

---

The fallout from this is that the syntax of the $vocabulary keyword

{
  "$vocabulary": {
    "https://my.schema.com/vocabs/myvocab": true, 
    ...
  }
}

doesn't really make sense anymore.

Should we change the keyword to just an array of URI IDs? Do something else?

The format of this keyword was never really solidified as vocabularies as a feature are still being developed, so I don't mind keeping its definition a bit fluid for now.

[jdesrosiers]

Whether optional vocabularies make sense or not depends on what "optional" means. I think we can define it a way where it makes sense, even if it's not quite as useful as it used to be. I would define an "optional" vocabulary to mean that implementations for the keywords in that library are optional. An implementation would need to "know" all vocabularies in $vocabularies, but doesn't need to "implement" keywords defined in optional vocabularies.

For example, my implementation doesn't support the "format-assertion" version of format. It knows that the "format-assertion" vocabulary defines the format keyword, but doesn't implement that keyword. If the dialect has ".../vocab/format-assertion": false and the schema has the format keyword, I can still evaluate that schema. However, if the dialect was ".../vocab/format-assertion": true, I wouldn't be able to evaluate the schema. In the future, if we add a vocabulary description format that includes what keywords the vocabulary defines, implementations could again support optional vocabularies they have no code for by looking up the vocabulary description.

This discussion is about reconciling optional vocabularies with forbidding unknown keywords. I think this approach does that in the least intrusive way. However, I think there's a less urgent discussion we should probably have about whether we think it even makes sense for the vocabulary system to have a concept of optional vocabularies.

[gregsdennis]

I thought so, too, until I started making the changes. The problem is that an unknown optional vocabulary produces unknown keywords, which are disallowed.

The only case where it doesn't is if someone explicitly writes code to "know" but not support a vocabulary, which

• produces inconsistent results (in general, "surprises")
• can still be done for required vocabs

I think this needs to be resolved before we can move on.

[jdesrosiers]

I think there's a miscommunication here. I'm not saying that there's not a problem here that needs to be addressed. I'm saying that we can address that problem without having to decide whether we should support optional vocabularies. With just a minor adjustment to the definition of what it means for a vocabulary to be "optional", we can avoid opening a huge can of worms.

Just so we're on the same page, setting aside the question of whether or not the concept of optional vocabularies is a good idea, and setting aside whether or not you like the solution I presented, do you agree that the solution I presented is a potential solution to the problem, or are you trying to tell me that my solution can't work?

[gregsdennis]

I think there's a miscommunication here. I'm not saying that there's not a problem here that needs to be addressed. I'm saying that we can address that problem without having to decide whether we should support optional vocabularies.

What's the problem you see that needs addressing?

For me, the problem (and the reason for opening the discussion) is that optional vocabularies have no utility and therefore should not be supported.

[SorinGFS]

@gregsdennis
As you said here $vocabulary is still a work in progress having multiple uncertainties. I think you should postpone the next release until all these uncertainties are resolved, or exclude the $vocabulary keyword from the next release.

[gregsdennis]

@SorinGFS thank you for your opinion, but that is part of a broader conversation on whether to include experimental features. It is off-topic for this discussion.

Please see Jason's SDLC proposal for a proposal on how experimental features could be included in the future. That conversation has been put on hold while we nail down some more immediate concerns.

[gregsdennis]

It's worth pointing out that the current document (in GitHub) also recognizes this relationship:

Implementations that do not support a vocabulary that is optionally used by a
schema SHOULD proceed with processing the schema. The keywords will be
considered to be unrecognized keywords as addressed by {{unrecognized}}.

The direct consequence of this is that if unsupported optional vocabularies create unrecognized keywords, and unrecognized keywords are forbidden, then optional vocabularies are actually required.

[karenetheridge]

I think the only way out here (to continue allowing a vocabulary to be specified as false) would be to explicitly list the keywords that such a vocabulary defines, so they can be treated as "ignored" (or annotation-only) rather than "error on first sight".

The other useful usecase for true/false vocabularies is where two vocabularies define the same set of keywords, and the schema has the option of deciding which to support - e.g. the two format vocabularies which both define format but in different ways. Because the format-annotation vocabulary is mandatory, we know that implementations will not treat the format keyword as unknown.

[gregsdennis]

I would like to point out that @Relequestual also (and independently) noticed this conflict when attempting to remove support for unknown keywords.

[jdesrosiers]

He's also expressed discomfort with dropping the concept of optional vocabularies. There's no question that there's a conflict that needs to be addressed. I still think there's a path forward that makes sense and allows for the concept of optional vocabularies to still exist.