Validation using JSON schema

[joshuatf]

Within various places around Woo, we have more and more cases where client-side validation proves beneficial.

Unfortunately, we don't have any real patterns for this so far and we seem to be taking a multitude of approaches to try and solve the same problem.

Current validation patterns

Typically our approaches when building out more modern user interfaces falls into one of two buckets; both of which have their pitfalls.

Custom client-side validation

The new product editor contains many fields which must be validated prior to saving products. Many of these fields implement custom blocks for the sake of being able to add their own validation logic within the block. Others are able to circumvent the need for custom blocks by using helper properties that handle validation within the templating API.

While this does seemingly offer a better experience, it can easily lead to the server and client becoming out of sync when it comes to those validation rules. As seen in the product editor, existing records have the potential to be completely valid in the previous experience, but can't be saved in the new experience given the introduction of client-side validation. (see pcQJnU-2nH-p2)

Server-side only validation

As an alternative to this, we can keep the single source of truth on the server and do validation there. The checkout block is extensible with additional fields from third parties. Currently, validation for these fields are handled on the server.

While this clearly leads to less errors and is a more stable approach, it also requires more round trips to the server to show validation errors and ultimately creates a much slower user experience for customers at a time when speed and reduction in friction is most important.

Why use JSON schema

Using JSON schema allows us to declare validation rules and properties to cover a large number of our validation cases. It is well-documented and allows us to easily extend our current schemas to support the necessary validation. But beyond that, it offers a solution to the bigger issues we're facing with validation right now:

1. Single source of truth

The biggest reason to use JSON schema is it gives us a sane way of understanding validation logic across both the client and server. This means that because they are using the same ruleset, there's little room for validation to become out of sync where server validation might pass, but client-side fails.

It also provides a self-documenting object that describes how data should be presented and allows easy reuse of validation rules going forward in different contexts.

2. Improved user experience

Because the validation can easily be used in the client, we don't need to call the server to check if a field is valid in many cases and can show errors in realtime as they are typed.

3. Easier third party integration

It allows for 3PDs to add validation logic in PHP via hooks that already exist. This reduces the need to spin up custom components or blocks in order to write validation logic in Javascript when generic block types already exist to extend existing experiences.

Proof of Concept

I put together a quick concept that shows how we can use JSON schema. The concept is pretty simple and piggybacks on our existing product endpoint schema. In brief:

1. We start by adding any validation rules to our existing schema. Third parties can and should extend this schema with their own properties and validation rules. This can already be achieved through existing hooks.

public function get_item_schema() {
	$schema = array(
		'$schema'    => 'http://json-schema.org/draft-04/schema#',
		'title'      => $this->post_type,
		'type'       => 'object',
		'properties' => array(
			...
			'stock_quantity' => array(
				'description' => __( 'Stock quantity.', 'woocommerce' ),
				'type'        => 'integer',
				'context'     => array( 'view', 'edit' ),
				'minimum'     => 0,
			),
			...
		),
	);

	return $this->add_additional_fields_schema( $schema );
}

2. We then pass the schema and record to an HOC that automatically handles validation and error creation based on the provided rules.

const [ productSchema, setProductSchema ] = useState< OptionsSchema | null >( null );

useEffect( () => {
	// Note that we could hydrate the page with the schema or preload this endpoint in some scenarios to improve page load times.
	apiFetch< OptionsResponse >( {
		path: 'wc/v3/products',
		method: 'OPTIONS',
	} ).then( ( result ) => setProductSchema( results.schema ) );
}, [] );

const { editedRecord } = useEntityRecord< Product >(
	'postType',
	'product',
	productId
);

<ValidationProvider
    schema={ productSchema }
    record={ editedProduct }
>
    { children }
</ValidationProvider>

3. In order to then use those validation errors somewhere, we simple grab the errors from the validation context, which contain everything from the error type, to the property, to the specific error message:

const { errors } = useContext( ValidationContext );

4. It is then up to the consumer to decide how to include the errors in components. Errors can exist in the initial state and the consumer decides whether or not to display them or wait until a field is touched/submitted before displaying them.

const [ isTouched, setIsTouched ] = useState( false );

<TextareaControl
	help={ isTouched && errors?.custom_field?.message }
	onBlur={ () => setIsTouched( true ) }
/>
) }

Issues that need more discussion

Despite all the positives that come with JSON schema, it is not perfect and there are areas that need more thought/discussion.

Complex validation use cases

While JSON Schema has come a long way and probably meets the needs of 95%+ of cases, there are some fields that are too complex to be handled purely in JSON or break the schema.

A field wanting to set a minimum amount based on another field can be handled by JSON schema, but a field wanting to set a limit based on a percentage of another field may be significantly more challenging to handle via pure JSON.

There are also cases that while possible to achieve, would be needlessly complex via JSON alone and validation on the server makes a lot more sense. For example, validating that a slug is unique is something we still want the server to handle as opposed to providing an irrationally long list of exclusions via JSON schema.

Related to the point below, but it is possible to declare custom keywords that may be able to handle this, but we would need to push towards having consistent dialect on both the client and server. Maintaining an organized package with all the validation logic would go a long way in making sure we can accomplish this.

Coerced types or non-conforming specs

Some fields, such as regular_price, would benefit from being a number type to allow usage of all the validation rules around numbers. However, WooCommerce treats prices as strings, meaning rules like exclusiveMinimum do not apply to this data type.

We can either set this type to number and attempt to coerce the string to a number at runtime during validation or we can create rules that aren't in the spec, such as exclusiveMinimumCoerced that allow keeping the type untouched while declaring a new rule to apply to this type.

On a side note, the Ajv package does offer an option to coerce data types but it's buggy when it comes to referencing other fields within the schema.

Error message consistency

The POC shows a way to provide error messages per each scenario as part of the schema. However, realistically we often have differing language when it comes to error messages on the client and server.

We need to pick between either providing consistent error messages or providing another API to intercept errors and provide messages based on error type and context.

Bundle size

Finally, it's also worth noting that package used for the client-side validation, Ajv, comes in at 119kb minified and 35kb when gzipped. While not extremely large, this is a non-insignificant amount. If we plan to use this on the frontend of the site, it may be worthwhile to consider creating a package that's a bit more stripped down and lighter weight to accomplish what we need.

Thoughts or concerns

Personally, I feel it's a pretty easy call to green-light this project since the benefits far outweigh the drawbacks. The only concerns I have currently are listed above and mostly just discussions that need decisions, not blockers.

But all feedback is welcome on this and if there are cases where this does not adequately fit project needs, I'd love to know what those are. Obviously JSON schema will not solve 100% of use cases, but if it can solve the overwhelming majority of cases, we should consider this approach to validation.

[mdperez86]

I like this approach to cover the validations and I do agree that this is not going to cover all possible use cases that we can have, but in my opinion, is a good way to follow. JSON schema has been widely used in micro service architectures as constraints/validations between API contracts. Also in some cases to build dynamic ui pages and forms.

One of the areas I faced and found not trivial to achieve is nesting schemas that are defined in different endpoints.
Lets say that there is a Product schema that contains a list of Attributes and both schemas belong to different APIs:

• In this case the schemas need to be loaded separately while building the ui page because they are not defined in one place but the validation manager should triggers the validations in both schemas in the same click event (well this always depends on the UI implementation) but you got my point.
• On the other hand schema definitions cannot handle complex and/or async validations like the slug one you just mentioned, but they can tell consumers how to do it, like hitting a specific endpoint that asynchronously manage the validation.
• When use it only for validations purposes, then how to manage the validations when the UI is presenting less fields than the ones defines in the schema? Consider the case where the schema validates the price field, but the UI does not render it.
• What about the error messages? Are them going to be translated server side and be the same in all contexts? Our experience tell us that error messages are always custom, even when they belongs to the same field, because the context/story telling can affect it.
• What happens when we have different templates representing different products with different validations, only one product schema will not scale to cover them all. Should the schema be defined in the product endpoint or somewhere else? Should it be just one definition or many? Should the schema be tied to the product and/or to the template?
• How can we link the data validation to the specific field that needs to be highlighted/focused? Consider that we need a reference to the field to focus it when the validation was triggered. Also how to keep the visual order so that the first visible field receives the focus.
• How all of this can be connected with the block system we have currently follow in the product editor?

Well these are some of my concerns about this approach (which I'm not seeking for answers) that of course they can be implemented, but I'm just put them on top of the table to be considered as part of the overall architecture of this solution.

[joshuatf]

Great thoughts all the way around, @mdperez86! Really appreciate you taking the time to leave this feedback.

In this case the schemas need to be loaded separately while building the ui page because they are not defined in one place but the validation manager should triggers the validations in both schemas in the same click event (well this always depends on the UI implementation) but you got my point.

Good point. This probably requires further investigation, but I wonder if we create a filter to autoload referenced schemas so when we retrieve, we have all the rules.

but they can tell consumers how to do it, like hitting a specific endpoint that asynchronously manage the validation.

💯 Creating definable passthrough endpoints that can be called and consistent responses will be key in doing this.

When use it only for validations purposes, then how to manage the validations when the UI is presenting less fields than the ones defines in the schema? Consider the case where the schema validates the price field, but the UI does not render it.

Ideally, the validation and the rendered fields should not be tied together. When doing validation we should always validate against the entire record and a record should always be valid regardless of what fields are shown.

If a field does not need to be entered when another is being used, the validation should represent this as "either property A or property B should be present." Writing this in JSON schema is arguably more complex than it needs to be, but we can discuss whether or not we add helper functions to assist in forming this JSON.

What about the error messages? Are them going to be translated server side and be the same in all contexts? Our experience tell us that error messages are always custom, even when they belongs to the same field, because the context/story telling can affect it.

Agreed on this point. We might not end up relying on the error messages sent from the server and instead rely on the error object that contains information about the error and offending keyword. This should be pretty trivial to do, but we'll of course need to make this extensible in the client if we choose to create custom errors there.

What happens when we have different templates representing different products with different validations, only one product schema will not scale to cover them all. Should the schema be defined in the product endpoint or somewhere else? Should it be just one definition or many? Should the schema be tied to the product and/or to the template?

Related to the point a few comments up, but the validation should happen at the record level and not the UI. It's the UIs responsibility to show those errors, but a record should always be valid and follow the same validation rules regardless of what template is being shown.

How can we link the data validation to the specific field that needs to be highlighted/focused? Consider that we need a reference to the field to focus it when the validation was triggered. Also how to keep the visual order so that the first visible field receives the focus.

Good question. I think we can solve this in a couple ways.

1. We can keep the errors in array format and rely on the first index allowing the component to handle grabbing focus:

useEffect( () => {
  if ( errors[0].this_field && formIsSubmitted ) {
    thisFieldRef.focus();
  }
}, [ errors, formIsSubmitted ] );

This of course won't focus a field if the first error isn't represented in the UI but that's a separate issue and we'll need to represent these errors somewhere either way.

2. If always focusing is a priority then we can register refs for errors and create a hook to handle this for us. It might make sense to keep this separate from the actual validation hooks.

function MyField() {
  useAutoFocusOnError( propertyToWatch, thisFieldRef );
}

How all of this can be connected with the block system we have currently follow in the product editor?

I think a lot of it we can rely on block context, but hooks are also an option. We'll probably need to evaluate this on a case by case basis.

[retrofox]

When use it only for validations purposes, then how to manage the validations when the UI is presenting less fields than the ones defines in the schema? Consider the case where the schema validates the price field, but the UI does not render it.

How can we link the data validation to the specific field that needs to be highlighted/focused? Consider that we need a reference to the field to focus it when the validation was triggered. Also how to keep the visual order so that the first visible field receives the focus.

These points are indeed valid, but they are not part of the validation process per se. It's essential to isolate this process from the UI entirely. Currently, it's mandatory to render all fields blocks, even those that are not visible, because of the form validation. The blocks should consume the Validation API instead of handling it.

With a Validation API, independent of the view, we can render errors and messages wherever we want. For instance, we can show a validation sidebar when the form contains errors, right after the user tries to save the post.

[senadir]

Another option would be to build something similar or reuse the Laravel validation system, which I found to be more approachable than a JSON schema

https://laravel.com/docs/11.x/validation#available-validation-rules

For Checkout additional fields, we're looking into adding visibility rules, (doc here). Ideally if we're going to come with a validation system, it should look slightly similar to that one and ideally can both be used together (under the same definition).

[joshuatf]

Nice! I like the idea of something like Laravel's validation system if it's acting as an ORM to generate JSON rules.

Correct me if I'm wrong, but Laravel's validation system does not generate any JSON and is only available server-side, right? We do need a reliable way to validate client-side as well, but I think there's a place for a Laravel-like system to help generate more complex JSON schema structure.

[senadir]

Correct me if I'm wrong, but Laravel's validation system does not generate any JSON and is only available server-side, right?

The basic one doesn't no, but it can be enabled via Precognition library: https://laravel.com/docs/11.x/precognition thou it seems to do that by doing automatic requests behind the scenes, so it's not a valid option to generate a JSON.

[senadir]

One main concern I have with JSON is the complexity of the system, and lack of ability to get dynamic-ish values. Store API Schema for example has a good amount of boilerplate in it, would it be possible that developers write a simpler version of the rules that gets translated down the road to JSON?

[joshuatf]

lack of ability to get dynamic-ish values

Do you have some examples of this we could use as a test case?

would it be possible that developers write a simpler version of the rules that gets translated down the road to JSON?

Mentioned in your above comment as well, but I definitely think there's some room to make writing complex validation rules easier. Could you provide an example here as well of something that feels complex to write in JSON schema so we can use that for the sake of discussion?

[senadir]

Do you have some examples of this we could use as a test case?

This is only valid if we're going to write straight out .json files, not a PHP one that generates the JSON.

When it comes to dynamic values, one thing to consider is the context those validation rules will run into, should they be expected to access existing fields, will they need to access merchant session or not.

This is a concern for us in Checkout and something we're still wondering about because we want to run validation in some places, like the Checkout itself, but also the my account screen, the order management screen, and possibly the mobile app. This is an issue we're currently facing with visiblity rules.

[senadir]

Could you provide an example here as well of something that feels complex to write in JSON schema so we can use that for the sake of discussion?

I find the schema needed to accept a field in REST API to be too complex and boilerplaty, an example:
https://github.com/woocommerce/wceu23-shipping-workshop-final/blob/main/shipping-workshop-extend-store-endpoint.php#L55-L82

[joshuatf]

@senadir I suppose those are a little boilerplatey, but I do see most of the options defined as necessary. Do you think it would be better if we had tools that handled most of the defaults? Or did you have something else in mind?

$schema[ 'myStringField' ] = SchemaTool::string( 'My description' );

[dmsnell]

Sharing some thoughts I had previously shared privately.

---

While JSON Schema does provide some level of validation, the schemas can get complicated to express and then become bloated themselves.

Depending on what you do, it might be worth checking into Zod, which @sirreal introduced into some other projects.

Zod won’t validate the types on the server, but it does provide an arguably more convenient way to express the types of data (because it’s composable and code-oriented) while providing auto-completion inside the JS/TS code. Once you parse a document with what Zod auto-generates, it’s guaranteed to be valid, vs. JSON Schema where it can be hard to properly validate certain constraints.

[joshuatf]

@mattsherman I'm running under the assumption that we already have a validation package of our own in place given it's going to need to be lightweight and capable of being bundled with frontend scripts.

I can see the argument for Zod if we want to use it solely in place of those validation scripts. It is smaller than some of the alternatives when minified and gzipped, but I'm also confident we could come up with a lighter weight solution with only the essentials if we choose to make that investment.

[reykjalin]

As someone with less first-hand knowledge around Zod, what are the (tangible) additional benefits that Zod provides us over generating TS type declarations if we do have other schema generation tools on the server? My gut feeling is leaning towards the TS type generation without Zod, but there may be other benefits that come with Zod that make bundling it worthwhile.

Like @mattsherman mentioned; Zod provides runtime guarantees that the data you're parsing is actually what you expect it to be, whereas TS types only help at compile time, which may not be correct in practice. Consider, for example, someone calling APIs from JS where they don't have the TS type guarantees. Zod will work regardless of whether the caller is using JS or TS (or ReScript or Fable or …).

In short: with TS you make assumptions and act based on those assumptions, with Zod you know exactly what you're working with.

There are other validations zod can make that are stricter than what TS can provide. As an example, you can verify that a string is a valid datetime string, something I don't think TS can do natively with it's type system. You can also do various number validations at runtime. And there are even more options zod can provide.

In general I'll caution projects wanting to generate types from PHP classes, reflection, or static analysis.

I think there's a time and a place for doing this. For example I wouldn't advise generating types based on something returned by a service class, or when a REST controller bundles together different data structures from different parts of the codebase. But I do think it's valid to do this for model classes, when those models are sent "as-is" to the frontend.

Even then, to do this properly you'd probably want to use something like attributes to add metadata for additional validations not covered by TS's or PHP's type systems. This would be a significant investment that may or may not pay off down the line. Especially when using 3rd party libraries.

But I find the idea interesting enough to at least explore, and I think there's a place for the approach even if it's strictly applied to model classes.

The problem here is that PHP often gets used in ways we don't expect, and our types misalign.

I actually see this as a reason to verify our assumptions in the front-end, regardless of whether or not the types/zod parsers are generated based on the PHP code or not 🙂

[joshuatf]

In short: with TS you make assumptions and act based on those assumptions, with Zod you know exactly what you're working with.

I see, thanks for the explanation @reykjalin!

We do have a lot of tooling in place (at least in Woo) that handles coercion to specific types (currencies, dates, etc), but this largely manual and I can see some advantages to using Zod in place of some of these tools.

I guess the 2 underlying questions are:

1. Do we try and come up with a creative solution mentioned above where Zod gets parsed to a schema and copied to the server (or the server schema dictates Zod in some way). Ideally the server remains the source of truth.
2. Whether or not this library is light enough to be considered for use on the frontend of the site versus creating our own.

I actually see this as a reason to verify our assumptions in the front-end, regardless of whether or not the types/zod parsers are generated based on the PHP code or not 🙂

➕ 1️⃣

[rjchow]

As I shared with @joshuatf in another discussion channel, I think that trying to mash the two concepts together is unlikely to result in a satisfactory outcome.

Zod's strength lies in its ergonomics (typedefs for static analysis) and ability to run code as validation. JSON schema is declarative, and has multiple implementations across platforms.

I do think that in the Wordpress regime, the backend remains the source of truth and that having a dependency cycle that generates types in the frontend repo then mirroring it to the backend to be used as the source of truth will be unwieldy and end up being less ergonomic on the whole. I've used Zod in various places in Woo Admin, but if there was an existing JSON schema validation library that was already in Woo Admin I probably wouldn't have introduced Zod.

I think that the most likely situation I would unarguably choose to use Zod would be with a mono-language TS stack where I can share the data definitions as a library throughout the stack.

[joshuatf]

Thanks for the thoughts, RJ! After experimenting with Zod a bit and attempting to make these two concepts fit, I agree with @rjchow.

Here's a try branch with Zod as the validation - #48017

Zod Pros

• The schema generation is extremely easy and comes with many tools out of the box
• The built-in coercion when parsing is very nice to have given we have this problem with a number of pieces of our data
• The ability to describe shapes of types within the environment (JS/TS) where we often use these types is unparalleled

Zod Cons

• While the JSON Schema generation tool for Zod is nifty, the comparison between fields (known as refine in Zod) is not accounted for. I believe a number of other issues may arise from trying to convert Zod to JSON Schema
• Since fields within WordPress are dynamic due to plugin extensibility, Schema is dynamic too. We could generate the JSON Schema in the client and send it to an endpoint for processing and persisting in a cache on the server, but there is always a chance that the server validations rules are out of date since we need to hit the client anytime fields change.

I think that if we were talking about a purely client-side tool, Zod would be an easy pick, but given we need server validation as well and this should remain the source of truth, I think it doesn't quite fit our needs here. According to the Zod repo owner:

Frankly I think generating Zod schemas from json-schema is a bit of an anti-pattern and you should just use a JSON-schema validation library in that case. The point of Zod is the type inference, and that's not possible with a JSON import.

I do agree with a lot of the philosophy in Zod, especially around "parse, don't validate." Luckily we can obtain the same functionality within AJV and it's trivial enough to convert JSON Schema to TypeScript types.

[ralucaStan]

The checkout block is extensible with additional fields from third parties. Currently, validation for these fields are handled on the server. While this clearly leads to less errors and is a more stable approach, it also requires more round trips to the server to show validation errors and ultimately creates a much slower user experience for customers at a time when speed and reduction in friction is most important.

Just pointing out that these fields are some exceptions to the other fields in the the checkout block. We have a mix of validation in the checkout block, some fields have front-end validations, some field combinations have additional back end validation, etc.

The Additional Checkout Fields is a PHP API, so it's expected that this is where the validation would happen as well. If we proceed to supporting merchant facing additional checkout fields functionality via the editor then the validation option we could provide would also be quite limited.

The JSON schema is an interesting approach, I've never worked with it.

In the checkout context extensions can add fields with heavy business logic behind.
My concern is the complex validation rules; while Ajv supports Compound keywords these might not support all cases. It also feels weird to me to keep logic in JSONs.

How would you allow 3PD to extend the schema?

[ralucaStan]

Just to double check- the core fields are mostly client-side validated at this point and the third party additional fields are validated entirely on the server, right?

Mostly yes, the core fields have FE validation in place. We do some server validation also after submitting the form.

An extension can also add a block and handle validation on the front-end. Like in this example This is a tutorial plugin

Having more of these fields offer client-side validation would create for better UX, but does not eliminate the need for server-side validation as well.

Thank you for mentioning this. It's good to have this context.

Do you have some examples of these heavier business logic cases? These would be good for vetting out more complex cases with JSON Schema.

EU VAT Number has some strong validation https://github.com/woocommerce/woocommerce-eu-vat-number/pull/256/files
I was thinking also about Conditional Shipping and Payments when I wrote that. Maybe @PanosSynetos can share more on different complex validations that hey run.

Does we need additional tooling to remove some of the boilerplate-like nature of Schema as @senadir mentioned #47277 (reply in thread).

+1 on this. This would improve the dev experience is always a plus and makes things more intuitive. Unrelated, but I was looking at some JS tools for validation schema creation and they have beautiful APIs (Joy and Yup)

Hope my answer was helpful

[ralucaStan]

Mostly yes, the core fields have FE validation in place. We do some server validation also after submitting the form.

For example we validate the address fields based against the locale information from WooCommerce.

[jimjasson]

I was thinking also about Conditional Shipping and Payments when I wrote that. Maybe @PanosSynetos can share more on different complex validations that hey run.

@ralucaStan

In the checkout, Conditional Shipping and Payments performs a number of validations in order to decide which Shipping Methods, Payment Gateways or Shipping Addresses are considered valid. These validations can be categorized as following:

• Based on the cart/package items, for example if they are on sale, or if they belong to a specific category or if they have a specific class.
• Based on the entire package/cart, for example based on the total.
• Based on the individual checkout fields, for example based on the selected shipping/billing country/state, postcode, selected Shipping Method or customer e-mail.
• Based on other factors, like the existence of coupons or gift cards, the active currency, the customer's role or the date/time of the order.

[joshuatf]

An extension can also add a block and handle validation on the front-end. Like in this example This is a tutorial plugin

Thanks for noting this example! This is a good example of something that's a bit of an anti-pattern. In general, we want to avoid letting UI components drive validation. Validity should be purely data-driven while the UI components like this block are presentational in nature and should only be responsible for displaying validation errors passed to them.

Implementing validation directly within blocks is also something that the new product editing experience does. I think in both its case and the one you linked to, it's understandable why we chose those routes since better options for validation don't yet exist.

EU VAT Number has some strong validation https://github.com/woocommerce/woocommerce-eu-vat-number/pull/256/files

Glancing over the rules in this PR, it looks like since both the IP Country and IP Address are known in advance prior to page load, we should be able to validate relatively easily with JSON Schema. Writing these rules out in JSON can be cumbersome though, so this is probably a strong case for the tooling we talked about above.

Hope my answer was helpful

Extremely! Thanks again for taking the time to note all these cases, @ralucaStan.

[joshuatf]

Based on the cart/package items, for example if they are on sale, or if they belong to a specific category or if they have a specific class.
Based on the entire package/cart, for example based on the total.
Based on the individual checkout fields, for example based on the selected shipping/billing country/state, postcode, selected Shipping Method or customer e-mail.
Based on other factors, like the existence of coupons or gift cards, the active currency, the customer's role or the date/time of the order.

These are all merchant described conditions, right, @jimjasson? On the surface, these look like the perfect case for JSON Schema as most of the rules are pretty straightforward comparisons between a variable and a user defined value.