-
Notifications
You must be signed in to change notification settings - Fork 2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[WIP] Try ajv instead of is-my-json-valid for JSON Schema validation #20145
Conversation
f88c8e1
to
15d17ee
Compare
I think this a worthy idea to pursue, especially if it's adopted in a way that allows us to consistently validate our forms. Furthermore, longer term, if it helps to share strict validation rules across front and backends all the better. Async validation looks tasty. JSON-Schema has some nice approaches to dependencies as well, and seems pretty flexible with regards to conditional validation. Do you see a 'per object/field' or 'per collection/form' schema as making more sense as a broad strategy? What I mean is, when building a form, do you think it makes more sense to run tests against a collection of individual schemata (such as Thanks for this! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This seems like a reasonable change to investigate. I see maybe five uses of is-my-json-valid
outside of test code. Could you spare the cycles to update those and see how the overall total build sizes change?
In Slack we discussed that this ended up as around 30 KB gzip'd. If we can eliminate is-my-json-valid
in the progress and cut down 15 KB or more or so of that size then I would imagine there would be very little opposition here.
I don't mind helping out with the data layer stuff because I don't think any code is yet relying on the failure messages from the validation. Pretty soon we'll have more momentum though as more people do.
Nice update.
15d17ee
to
ef5d5a0
Compare
I'm curious what that |
I had a quick look and it turns out that |
Good question! I don't know :) Some things are simple - Otherwise, we've got a lot of flexibility. With the way JSON Schema and Redux work, I suspect we'd get a better result if we focused on the state/data rather than form fields. The component already has to map the state to the fields, which seems like the same task/responsibility as mapping errors to fields. There's room to do some really clever things, but in the short term, we're probably going to start with validation that cover the same data as some component and be fast enough run in the
Thanks! I totally hacked through there, and made guesses about performance and side effects that definitely need checking if we end up going down this path. As an aside, ajv complained that a number of the schemas themselves are invalid (e.g. because |
I dug into It needs tests before it's ready to land, but it addresses several open issues and should have almost no impact on runtime validation performance, so with tests in place I think it will be an attractive PR for them. If they're slow to merge, we should move my fork (https://github.com/deBhal/is-my-json-valid/tree/add/schemaPath) to github.com/Automattic - I believe we're already forking a couple of repos like that already. The IMJV code is actually nice and easy to modify. I think I can see how to add the keyword, or get the errors out of references if we want that. I actually added the schema node itself to the error with a one-line change in the code. It ended up dumping potentially a lot of stringified schemas into the generated code, though, so I pulled it out, but now that I'm thinking about it a bit more, I can see a way to do it much more cleanly by adding the schema to the validation function and using these paths I've just added. As an aside, I noticed that the I'm going to go ahead and close this PR in favour of that one, but I'm happy to come back here if there's demand. |
What's wrong with
is-my-json-valid
?is-my-json-valid
is excellent in terms of pass/fail, but it's pretty terse with it's results. In particular, it would be super nice if we could reliably get back to the rule that failed, because that way we could add any meta-data we might need.I ran into this adding a rule to the
.fr
domain contact validation schema that says "if theregistrantType
isorganization
, then theorganization
field cannot be empty". That's no problem to validate with JSON schema, but we need to associate a failed result with the right field on the form.Up until now, all the validation I've done has been simple enough to fit into a
properties
that matches the data itself so it's been easy enough to put messages in the right place (I did have to rearrange rules a couple of times to disambiguate them, but nothing worth ripping out the library yet).However, once a rule involves more than one property it's intrinsically ambiguous which field is the wrong one (maybe both). In the
.fr
organization
case, I had to add logic in the validation result handling that says roughly "if it doesn't have a field, associate it with theorganization
field". That works, but only once, and it's an ugly hack.What I want to be able to do is specify in the rule itself which field it belongs to. If we can get back to the rule from the failure, this is as simple as adding another property to the rule itself, like
field: organization
, and taking that to mean "If this fails, show the error next to theorganization
field". ( We're already using this approach on the back end to facilitate i18n r165250-wpcom ).The awesome thing is that we can use that same trick for any extra data we turn out to need. We could add the user visible error message to the rule, or we could add a severity. We could even decide that we'd like a sort of inheritance for some properties, and search up the schema's tree for them.
But we can't do that with
is-my-json-valid
, because it doesn't tell us clearly where the problem occurred. There are several related issues (mafintosh/is-my-json-valid#38, mafintosh/is-my-json-valid#39, mafintosh/is-my-json-valid#22).I've put together a runnable example of the problem:
While this example is a little contrived, you can see that
is-my-json-valid
returns exactly the same resultimjvValidate( badRaven )
andimjvValidate( badDove )
, meaning we can't tell what failed.ajv
adds the criticial information we need in theschemaPath
to tell the two failures apart as well as find that rule in the schema.Why ajv?
There are a lot of other options, but ajv seems to be a pretty clear standout.
djv
has great stats too, but I found it a bit challenging to use. By default it returns a fairly uninformative string, and changing that involves passing in an error handling callback that seems to return a string to get evaluated in a context that contains the information you need, but the documentation on that aspect is not very helpful. I get the impression it would do what we need it to with some extra effort, but I don't see a compelling reason to follow the rougher path here.https://github.com/bugventure/jsen has good stats and good looking docs, but it performs worse than is-my-json-valid on the benchmarks, so not as attractive, but if we're going to do some rigorous testing we should probably include it.