Skip to content

Commit

Permalink
Add annotation support refactor keywords to use annotations implement…
Browse files Browse the repository at this point in the history
… output formats (#942)

* Support annotations

* Refactor recursive ref

* Fix IllegalStateException on recursive call in computeIfAbsent

* Refactor

* Refactor results

* Refactor

* Remove scope

* Refactor

* Add can short circuit logic

* Fix performance

* Fix

* Refactor

* Enable disabled tests

* Fix contains

* Fix fast fail

* Refactor

* Update docs

* Add schemaNode and instanceNode to validation message and configure serialization

* Fail fast based on execution context config

* Fix fail fast

* Refactor annotation config.

* Update doc

* Fix 939 and add 940 test

* Fix 936

* Fix 935

* Fix

* Add convenience method for schema loader

* Add convenience method for schema mappers

* Support annotation collection for reporting

* Support output formatting

* Refactor

* Refactor

* Collect format annotations

* Refactor

* Refactor

* Refactor

* Refactor

* Redesign and fix fail fast logic

* Support hierarchical output

* Update docs

* Throw specific exceptions if ref cannot be resolved

* Update docs

* Add 857 test

* Javadoc

* Add 927 test

* Move out non suite tests

* Update to latest test suite

* Fix os line ending difference

* Fix os line ending difference

* Fix javadoc

* Fix id handling

* Add tests for format output formatting

* Add test for type union
  • Loading branch information
justin-tay committed Feb 1, 2024
1 parent e95642c commit 9c95c06
Show file tree
Hide file tree
Showing 215 changed files with 7,013 additions and 2,429 deletions.
5 changes: 3 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
# Change Log
All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](http://keepachangelog.com/)
and this project adheres to [Semantic Versioning](http://semver.org/).
This format is based on [Keep a Changelog](http://keepachangelog.com/).

This project does not adhere to [Semantic Versioning](https://semver.org/) and minor version changes can have incompatible API changes. These incompatible API changes will largely affect those who have custom validator or walker implementations. Those who just use the library to validate using the standard JSON Schema Draft specifications may not need changes.

## [Unreleased]

Expand Down
286 changes: 264 additions & 22 deletions README.md

Large diffs are not rendered by default.

27 changes: 17 additions & 10 deletions doc/compatibility.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,20 @@
## Compatibility with JSON Schema versions

This implementation does not currently generate annotations.

The `pattern` validator by default uses the JDK regular expression implementation which is not ECMA-262 compliant and is thus not compliant with the JSON Schema specification. The library can however be configured to use a ECMA-262 compliant regular expression implementation.

Annotation processing and reporting are implemented. Note that the collection of annotations will have an adverse performance impact.

This implements the Flag, List and Hierarchical output formats defined in the [Specification for Machine-Readable Output for JSON Schema Validation and Annotation](https://github.com/json-schema-org/json-schema-spec/blob/8270653a9f59fadd2df0d789f22d486254505bbe/jsonschema-validation-output-machines.md).

### Known Issues
* The `anyOf` applicator currently returns immediately on matching a schema. This results in the `unevaluatedItems` and `unevaluatedProperties` keywords potentially returning an incorrect result as the rest of the schemas in the `anyOf` aren't processed.
* The `unevaluatedItems` keyword does not currently consider `contains`.

There are currently no known issues with the required functionality from the specification.

The following are the tests results after running the [JSON Schema Test Suite](https://github.com/json-schema-org/JSON-Schema-Test-Suite) as at 29 Jan 2024 using version 1.3.1. As the test suite is continously updated, this can result in changes in the results subsequently.

| Implementations | Overall | DRAFT_03 | DRAFT_04 | DRAFT_06 | DRAFT_07 | DRAFT_2019_09 | DRAFT_2020_12 |
|-----------------|-------------------------------------------------------------------------|-------------------------------------------------------------------|---------------------------------------------------------------------|--------------------------------------------------------------------|------------------------------------------------------------------------|----------------------------------------------------------------------|------------------------------------------------------------------------|
| NetworkNt | pass: r:4703 (100.0%) o:2369 (100.0%)<br>fail: r:0 (0.0%) o:1 (0.0%) | | pass: r:600 (100.0%) o:251 (100.0%)<br>fail: r:0 (0.0%) o:0 (0.0%) | pass: r:796 (100.0%) o:318 (100.0%)<br>fail: r:0 (0.0%) o:0 (0.0%) | pass: r:880 (100.0%) o:541 (100.0%)<br>fail: r:0 (0.0%) o:0 (0.0%) | pass: r:1201 (100.0%) o:625 (100.0%)<br>fail: r:0 (0.0%) o:0 (0.0%) | pass: r:1226 (100.0%) o:634 (99.8%)<br>fail: r:0 (0.0%) o:1 (0.2%) |

### Legend

Expand Down Expand Up @@ -79,15 +86,15 @@ The `pattern` validator by default uses the JDK regular expression implementatio

#### Content Encoding

Since Draft 2019-09, the `contentEncoding` keyword does not generate assertions. As the implementation currently does not collect annotations this only generates assertions in Draft 7.
Since Draft 2019-09, the `contentEncoding` keyword does not generate assertions.

#### Content Media Type

Since Draft 2019-09, the `contentMediaType` keyword does not generate assertions. As the implementation currently does not collect annotations this only generates assertions in Draft 7.
Since Draft 2019-09, the `contentMediaType` keyword does not generate assertions.

#### Content Schema

The `contentSchema` keyword does not generate assertions. As the implementation currently does not collect annotations this doesn't do anything.
The `contentSchema` keyword does not generate assertions.

#### Pattern

Expand All @@ -108,7 +115,7 @@ This also requires adding the `joni` dependency.
</dependency>
```

### Format
#### Format

Since Draft 2019-09 the `format` keyword only generates annotations by default and does not generate assertions.

Expand All @@ -119,7 +126,7 @@ This can be configured on a schema basis by using a meta schema with the appropr
| Draft 2019-09 | `https://json-schema.org/draft/2019-09/vocab/format` | `true` |
| Draft 2020-12 | `https://json-schema.org/draft/2020-12/vocab/format-assertion`| `true`/`false` |

This behavior can be overridden to generate assertions on a per-execution basis by setting the `setFormatAssertionsEnabled` to `true`.
This behavior can be overridden to generate assertions by setting the `setFormatAssertionsEnabled` option to `true`.

| Format | Draft 4 | Draft 6 | Draft 7 | Draft 2019-09 | Draft 2020-12 |
|:----------------------|:-------:|:-------:|:-------:|:-------------:|:-------------:|
Expand All @@ -143,7 +150,7 @@ This behavior can be overridden to generate assertions on a per-execution basis
| uri-template | 🚫 | 🟢 | 🟢 | 🟢 | 🟢 |
| uuid | 🚫 | 🚫 | 🟢 | 🟢 | 🟢 |

### Footnotes
##### Footnotes
1. Note that the validation are only optional for some of the keywords/formats.
2. Refer to the corresponding JSON schema for more information on whether the keyword/format is optional or not.

114 changes: 48 additions & 66 deletions doc/quickstart.md
Original file line number Diff line number Diff line change
@@ -1,83 +1,65 @@
## Quick Start

To use the validator, we need to have both the `JsonSchema` object and `JsonNode` object constructed.
There are many ways to do that.
Here is base test class, that shows several ways to construct these from `String`, `Stream`, `Url`, and `JsonNode`.
Please pay attention to the `JsonSchemaFactory` class as it is the way to construct the `JsonSchema` object.
To use the validator, we need to have the `JsonSchema` loaded and cached.

```java
public class BaseJsonSchemaValidatorTest {
For simplicity the following test loads a schema from a `String` or `JsonNode`. Note that loading a schema in this manner is not recommended as a relative `$ref` will not be properly resolved as there is no base IRI.

private ObjectMapper mapper = new ObjectMapper();
The preferred method of loading a schema is by using a `SchemaLocation` and by configuring the appropriate `SchemaMapper` and `SchemaLoader` on the `JsonSchemaFactory`.

protected JsonNode getJsonNodeFromClasspath(String name) throws IOException {
InputStream is1 = Thread.currentThread().getContextClassLoader()
.getResourceAsStream(name);
return mapper.readTree(is1);
}
```java
package com.example;

protected JsonNode getJsonNodeFromStringContent(String content) throws IOException {
return mapper.readTree(content);
}
import static org.junit.jupiter.api.Assertions.assertEquals;

protected JsonNode getJsonNodeFromUrl(String url) throws IOException {
return mapper.readTree(new URL(url));
}
import java.util.Set;

protected JsonSchema getJsonSchemaFromClasspath(String name) {
JsonSchemaFactory factory = JsonSchemaFactory.getInstance(SpecVersion.VersionFlag.V4);
InputStream is = Thread.currentThread().getContextClassLoader()
.getResourceAsStream(name);
return factory.getSchema(is);
}
import org.junit.jupiter.api.Test;

protected JsonSchema getJsonSchemaFromStringContent(String schemaContent) {
JsonSchemaFactory factory = JsonSchemaFactory.getInstance(SpecVersion.VersionFlag.V4);
return factory.getSchema(schemaContent);
}
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.JsonMappingException;
import com.fasterxml.jackson.databind.JsonNode;
import com.networknt.schema.*;
import com.networknt.schema.serialization.JsonMapperFactory;

protected JsonSchema getJsonSchemaFromUrl(String uri) throws URISyntaxException {
public class SampleTest {
@Test
void schemaFromString() throws JsonMappingException, JsonProcessingException {
JsonSchemaFactory factory = JsonSchemaFactory.getInstance(SpecVersion.VersionFlag.V4);
return factory.getSchema(SchemaLocation.of(uri));
/*
* This should be cached for performance.
*
* Loading from a String is not recommended as there is no base IRI to use for
* resolving relative $ref.
*/
JsonSchema schemaFromString = factory
.getSchema("{\"enum\":[1, 2, 3, 4],\"enumErrorCode\":\"Not in the list\"}");
Set<ValidationMessage> errors = schemaFromString.validate("7", InputFormat.JSON);
assertEquals(1, errors.size());
}

protected JsonSchema getJsonSchemaFromJsonNode(JsonNode jsonNode) {
@Test
void schemaFromJsonNode() throws JsonMappingException, JsonProcessingException {
JsonSchemaFactory factory = JsonSchemaFactory.getInstance(SpecVersion.VersionFlag.V4);
return factory.getSchema(jsonNode);
}

// Automatically detect version for given JsonNode
protected JsonSchema getJsonSchemaFromJsonNodeAutomaticVersion(JsonNode jsonNode) {
JsonSchemaFactory factory = JsonSchemaFactory.getInstance(SpecVersionDetector.detect(jsonNode));
return factory.getSchema(jsonNode);
}

}
```
And the following is one of the test cases in one of the test classes that extend from the above base class. As you can see, it constructs `JsonSchema` and `JsonNode` from `String`.

```java
class Sample extends BaseJsonSchemaValidatorTest {

void test() {
JsonSchema schema = getJsonSchemaFromStringContent("{\"enum\":[1, 2, 3, 4],\"enumErrorCode\":\"Not in the list\"}");
JsonNode node = getJsonNodeFromStringContent("7");
Set<ValidationMessage> errors = schema.validate(node);
assertThat(errors.size(), is(1));

// With automatic version detection
JsonNode schemaNode = getJsonNodeFromStringContent(
"{\"$schema\": \"http://json-schema.org/draft-06/schema#\", \"properties\": { \"id\": {\"type\": \"number\"}}}");
JsonSchema schema = getJsonSchemaFromJsonNodeAutomaticVersion(schemaNode);

schema.initializeValidators(); // by default all schemas are loaded lazily. You can load them eagerly via
// initializeValidators()

JsonNode node = getJsonNodeFromStringContent("{\"id\": \"2\"}");
Set<ValidationMessage> errors = schema.validate(node);
assertThat(errors.size(), is(1));
JsonNode schemaNode = JsonMapperFactory.getInstance().readTree(
"{\"$schema\": \"http://json-schema.org/draft-06/schema#\", \"properties\": { \"id\": {\"type\": \"number\"}}}");
/*
* This should be cached for performance.
*
* Loading from a JsonNode is not recommended as there is no base IRI to use for
* resolving relative $ref.
*
* Note that the V4 from the factory is the default version if $schema is not
* specified. As $schema is specified in the data, V6 is used.
*/
JsonSchema schemaFromNode = factory.getSchema(schemaNode);
/*
* By default all schemas are preloaded eagerly but ref resolve failures are not
* thrown. You check if there are issues with ref resolving using
* initializeValidators()
*/
schemaFromNode.initializeValidators();
Set<ValidationMessage> errors = schemaFromNode.validate("{\"id\": \"2\"}", InputFormat.JSON);
assertEquals(1, errors.size());
}

}

```
69 changes: 69 additions & 0 deletions doc/upgrading.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,76 @@
## Upgrading to new versions

This library can contain breaking changes in `minor` version releases.

This contains information on the notable or breaking changes in each version.

### 1.3.1

This does not contain any breaking changes from 1.3.0

* Annotation collection and reporting has been implemented
* Keywords have been refactored to use annotations for evaluation to improve performance and meet functional requirements
* The list and hierarchical output formats have been implemented as per the [Specification for Machine-Readable Output for JSON Schema Validation and Annotation](https://github.com/json-schema-org/json-schema-spec/blob/main/jsonschema-validation-output-machines.md).
* The fail fast evaluation processing has been redesigned and fixed. This currently passes the [JSON Schema Test Suite](https://github.com/json-schema-org/JSON-Schema-Test-Suite) with fail fast enabled. Previously contains and union type may cause incorrect results.
* This also contains fixes for regressions introduced in 1.3.0

The following keywords were refactored to improve performance and meet the functional requirements.

In particular this converts the `unevaluatedItems` and `unevaluatedProperties` validators to use annotations to perform the evaluation instead of the current mechanism which affects performance. This also refactors `$recursiveRef` to not rely on that same mechanism.

* `unevaluatedProperties`
* `unevaluatedItems`
* `properties`
* `patternProperties`
* `items` / `additionalItems`
* `prefixItems` / `items`
* `contains`
* `$recursiveRef`

This also fixes the issue where the `unevaluatedItems` keyword does not take into account the `contains` keyword when performing the evaluation.

This also fixes cases where `anyOf` short-circuits to not short-circuit the evaluation if a adjacent `unevaluatedProperties` or `unevaluatedItems` keyword exists.

This should fix most of the remaining functional and performance issues.

#### Functional

| Implementations | Overall | DRAFT_03 | DRAFT_04 | DRAFT_06 | DRAFT_07 | DRAFT_2019_09 | DRAFT_2020_12 |
|-----------------|-------------------------------------------------------------------------|-------------------------------------------------------------------|---------------------------------------------------------------------|--------------------------------------------------------------------|------------------------------------------------------------------------|----------------------------------------------------------------------|------------------------------------------------------------------------|
| NetworkNt | pass: r:4703 (100.0%) o:2369 (100.0%)<br>fail: r:0 (0.0%) o:1 (0.0%) | | pass: r:600 (100.0%) o:251 (100.0%)<br>fail: r:0 (0.0%) o:0 (0.0%) | pass: r:796 (100.0%) o:318 (100.0%)<br>fail: r:0 (0.0%) o:0 (0.0%) | pass: r:880 (100.0%) o:541 (100.0%)<br>fail: r:0 (0.0%) o:0 (0.0%) | pass: r:1201 (100.0%) o:625 (100.0%)<br>fail: r:0 (0.0%) o:0 (0.0%) | pass: r:1226 (100.0%) o:634 (99.8%)<br>fail: r:0 (0.0%) o:1 (0.2%) |

#### Performance

##### NetworkNT 1.3.1

```
Benchmark Mode Cnt Score Error Units
NetworkntBenchmark.testValidate thrpt 10 6776.693 ± 115.309 ops/s
NetworkntBenchmark.testValidate:·gc.alloc.rate thrpt 10 971.191 ± 16.420 MB/sec
NetworkntBenchmark.testValidate:·gc.alloc.rate.norm thrpt 10 165318.816 ± 0.459 B/op
NetworkntBenchmark.testValidate:·gc.churn.G1_Eden_Space thrpt 10 968.894 ± 51.234 MB/sec
NetworkntBenchmark.testValidate:·gc.churn.G1_Eden_Space.norm thrpt 10 164933.962 ± 8636.203 B/op
NetworkntBenchmark.testValidate:·gc.churn.G1_Survivor_Space thrpt 10 0.002 ± 0.001 MB/sec
NetworkntBenchmark.testValidate:·gc.churn.G1_Survivor_Space.norm thrpt 10 0.274 ± 0.218 B/op
NetworkntBenchmark.testValidate:·gc.count thrpt 10 89.000 counts
NetworkntBenchmark.testValidate:·gc.time thrpt 10 99.000 ms
```

###### Everit 1.14.1

```
Benchmark Mode Cnt Score Error Units
EveritBenchmark.testValidate thrpt 10 3719.192 ± 125.592 ops/s
EveritBenchmark.testValidate:·gc.alloc.rate thrpt 10 1448.208 ± 74.746 MB/sec
EveritBenchmark.testValidate:·gc.alloc.rate.norm thrpt 10 449621.927 ± 7400.825 B/op
EveritBenchmark.testValidate:·gc.churn.G1_Eden_Space thrpt 10 1446.397 ± 79.919 MB/sec
EveritBenchmark.testValidate:·gc.churn.G1_Eden_Space.norm thrpt 10 449159.799 ± 18614.931 B/op
EveritBenchmark.testValidate:·gc.churn.G1_Survivor_Space thrpt 10 0.001 ± 0.001 MB/sec
EveritBenchmark.testValidate:·gc.churn.G1_Survivor_Space.norm thrpt 10 0.364 ± 0.391 B/op
EveritBenchmark.testValidate:·gc.count thrpt 10 133.000 counts
EveritBenchmark.testValidate:·gc.time thrpt 10 148.000 ms
```

### 1.3.0

This adds support for Draft 2020-12
Expand Down
5 changes: 5 additions & 0 deletions src/main/java/com/networknt/schema/AbstractCollector.java
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,11 @@
*/
package com.networknt.schema;

/**
* Base collector.
*
* @param <E> the type
*/
public abstract class AbstractCollector<E> implements Collector<E> {

@Override
Expand Down
Loading

0 comments on commit 9c95c06

Please sign in to comment.