Be more clear about format support (3.1.1)

[handrews]

This came up in conversation with @karenetheridge .

Most 2020-12 validators do not validate it, which is a regular point of confusion worth highlighting.

If accepted, I will port this to 3.2.0. It is not directly relevant to 3.0.4, as format was handled differently in that version of JSON Schema - still confusing and often not supporting useful validation, but it would require a different explanation.

[mikekistler]

I wonder if we should be adding observations about tooling to the specification, particularly when this may change in the future. What will we do in 5 years if all the common validators have corrected their behavior to handle format correctly? Can we at that time change old versions of the spec to remove these observations, or do we leave them in? And if we leave them, won't that be confusing to readers?

[mikekistler]

Suggested change

<a name="dataTypeFormat"></a>As defined by the [JSON Schema Validation specification](https://tools.ietf.org/html/draft-bhutton-json-schema-validation-00#section-7.3), data types can have an optional modifier property: `format`. Note that by default, JSON Schema draft 2020-12 validators _do not_ validate `format`, but instead treat it as an annotation. Support for `format` validation, either in best-effort form with the [default `format-annotation` vocabulary](https://datatracker.ietf.org/doc/html/draft-bhutton-json-schema-validation-00#section-7.2.1), or with the less-commonly-implemented [`format-assertion` vocabulary](https://datatracker.ietf.org/doc/html/draft-bhutton-json-schema-validation-00#section-7.2.2), varies among JSON Schema implementations.

<a name="dataTypeFormat"></a>As defined by the [JSON Schema Validation specification](https://tools.ietf.org/html/draft-bhutton-json-schema-validation-00#section-7.3), data types can have an optional modifier property: `format`. Note that by default, JSON Schema draft 2020-12 validators _do not_ validate `format`, but instead treat it as an annotation. Support for `format` validation, either in best-effort form with the [default `format-annotation` vocabulary](https://datatracker.ietf.org/doc/html/draft-bhutton-json-schema-validation-00#section-7.2.1), or with the less-commonly-implemented [`format-assertion` vocabulary](https://datatracker.ietf.org/doc/html/draft-bhutton-json-schema-validation-00#section-7.2.2), varies among JSON Schema validator implementations.

[ralfhandl]

If I understand the JSON Schema specification correctly, the OpenAPI 3.1 schema declares that it is based on https://json-schema.org/draft/2020-12/schema, which declares that format is just an annotation and not an assertion and can be ignored by validators:

    "https://json-schema.org/draft/2020-12/vocab/format-annotation": true,

So ignoring format is correct behavior for validators and explicitly allowed by how OpenAPI 3.1 uses JSON Schema.

As long as we do not change that part of OpenAPI 3.1, calling out that format is more of a comment and less of an instruction will also remain valid.

@handrews is my interpretation correct?

[handrews]

@ralfhandl Yes, that's correct. Perhaps it might be better to change the wording a bit as the wording I have here makes it sound like it's a tooling problem. It's actually that the spec changed between draft-04 and 2020-12 regarding how format is expected to be handled. Although it's slightly more complicated as there are two ways to get format validation: one is enabling a "best effort" validation mode that matches what has been historically implemented for the keyword, which can be done with the format-annotation vocabulary, but is done by the person running validation and can't be controlled by the schema or meta-schema. The other is the strict validation mode of the format-assertion vocabulary, which is not widely implemented in practice and requires making a meta-schema change that most people don't seem inclined to do (or support).

@mikekistler I think some wording changes here would make it more clear that this is not a transient thing, but the expected result of the specification.

[ralfhandl]

I wonder why we should worry about JSON Schema validators in the OpenAPI specification. The distinction between "annotation" and "assertion" is surely relevant for JSON Schema.

For API descriptions it seems irrelevant, especially as format is prominently mentionend in section Data Types as

OAS defines additional formats to provide fine detail for primitive data types.

If an API description declares that a property has type: 'string', format: 'date', this most likely means that the API implementation will refuse to accept any strings that do not contain a valid date. Ignoring the "fine detail" provided via the format keyword will result in 400 Bad Request.

[handrews]

I wonder why we should worry about JSON Schema validators in the OpenAPI specification.

Because the whole point of establishing JSON Schema compatibility in 3.1 was to offload JSON Schema support to JSON Schema tools. JSON Schema validators are effectively part of the OAS 3.1 tooling ecosystem.

If an API description declares that a property has type: 'string', format: 'date', this most likely means that the API implementation will refuse to accept any strings that do not contain a valid date. Ignoring the "fine detail" provided via the format keyword will result in 400 Bad Request.

In my experience people expect format to actually validate in JSON Schema, get really confused when it doesn't, and consider it a bug in something. If it's in the context of OpenAPI, they tend to blame OpenAPI.

I'll try re-writing this, but while I really try to avoid bloating the specification, I do think that addressing persistent problems is worth a sentence or two.

[handrews]

@ralfhandl @mikekistler I think I have now made this as minimal as possible while still setting user expectations correctly.

[ralfhandl]

Not sure which expectations we are setting here: "Don't use format because it may not be validated"?

Neither objecting nor agreeing to the change

[handrews]

Not sure which expectations we are setting here: "Don't use format because it may not be validated"?

"Don't expect format to be validated automatically."

It means what it says, no more, no less. Because I've spent nearly 10 years now answering this question, and @karenetheridge has dealt with it plenty as she's implemented both JSON Schema and OAS, and we both thought it would be helpful to head off some of that confusion.

[lornajane]

This is a good clarification, thanks

[mikekistler]

Looks good! 👍