-
Notifications
You must be signed in to change notification settings - Fork 12
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
User-friendly validation messages #9
Comments
We mentioned in our discussion that we could do that by forking the json-schema-validator repository. This repository is available here https://github.com/fge/json-schema-validator. Looks like there has not been much development on it and the owner is looking for a new maintainer. Should we fork it under ModelSolv? |
We could do it without having to fork this project, processing error messages should be possible to create more meaningful messages. # fixture 4
# invalid responses type, should be an object
swagger: '2.0'
info:
version: 0.0.0
title: Simple API
paths:
/:
get:
responses: 'Hello' This is the error object that result from the validation of the above yaml: {
"level": "error",
"schema": {
"loadingURI": "http://swagger.io/v2/schema.json#",
"pointer": "/definitions/responses"
},
"instance": {
"pointer": "/paths/~1/get/responses"
},
"domain": "validation",
"keyword": "type",
"message": "instance type (string) does not match any allowed primitive type (allowed: [\"object\"])",
"found": "string",
"expected": [
"object"
]
} The default message says that the value for But it should be doable to get a better message by getting meaningful information from the schema. The relevant part of the schema is given in the error field "responses": {
"type": "object",
"description": "Response objects names can either be any valid HTTP status code or 'default'.",
"minProperties": 1,
"additionalProperties": false,
"patternProperties": {
"^([0-9]{3})$|^(default)$": {
"$ref": "#/definitions/responseValue"
},
"^x-": {
"$ref": "#/definitions/vendorExtension"
}
},
"not": {
"type": "object",
"additionalProperties": false,
"patternProperties": {
"^x-": {
"$ref": "#/definitions/vendorExtension"
}
}
}
} |
A list of validation messages from JsonSchema Validation (provided by @ghillairet ) - https://github.com/fge/json-schema-validator/blob/master/src/main/resources/com/github/fge/jsonschema/validator/validation.properties |
We decided to create a suite of tests for validation messages. JSON Schema for Swagger is here - https://github.com/swagger-api/swagger-spec/blob/master/schemas/v2.0/schema.json . It can be used to retrieve additional information for the validation message. |
@tfesenko , I can provide better error messages, but think I will need some guidance as to how I can specify them, to make use of the information available in the schema. In some cases, I will need to use variables that don't seem to be defined yet. Let's talk after tomorrow's SUM. |
Add more tests showing the current messages for each type of errors.
…idation appears, it's for readability only, not processed by the tests
@tedepstein , @ghillairet started the "tests as documentation" in ValidationMessageTest. The expected message is placed above the model text. In the model text, the " #validation error marker" are placed right above the place where we expect to see a validation error. It's for human convenience only and will be ignored by the test. Does this format suit you? Do you have any suggestions to improve readability? |
@tfesenko , this looks fine to me. Want to complete the inventory of tests, and then I'll review? Or do you want my input now? |
@tedepstein , I don't plan to add more validation examples today. |
I've added my comments. See commit f2a852c . |
Updated some of the messages to be easier to understand by the user.
Some of the messages have been updated, as you can see in this commit 9988e42 It concerns the most common error messages. The most interesting would be that this previous kind of messages
will be replaced by more meaningful ones, such as
This excerpt represents in single message that replaces the previous one. It includes all the errors that apply at a position of the document. |
@ghillairet , w.r.t. your last comment, I opened #60, which I think is partially caused by the replacement of "instance failed to match exactly one schema" with a list of errors related to each failed match. We could try listing the schemas individually, like this:
It's not ideal, because we cannot easily name or describe the schemas. The user has to guess what the different schemas are looking for. But it's more informative than a single "failed to match", and it organizes the error messages in a way that might make it easier to interpret. Thoughts? |
Also, the example in #60 seems to show that schema validation failures can be recursive. We see "failed to match" errors, presumably arising from attempted schema validation of contained definitions. I don't think it's wise to try to "unwrap" schema validation errors recursively. We could get a really large tree, and could even encounter cycles, unless we're sure that this kind of validation is acyclic... Regardless, it seems like we should not try to take the unwrapping too far. |
Merge latest changes from open-sourceSwagEdit
Validation messages are composed by JSON-Schema validation and often times are not user (==human)-friendly and don't provide all necessary information.
We want to rephrase them and augment with information which can be obtained from JSON Schema. For example, if a value is enumeration, don't just say that it's not allowed, but also provide a list of available values.
Ted can provide or correct the exact text of the validation messages. We have to point him to the place it code or in tests where he can see them all.
We also would like to make the messages generic, not Swagger-specific and contribute it back to the validator if possible.
The text was updated successfully, but these errors were encountered: