Skip to content

Commit

Permalink
Refactor walk (#986)
Browse files Browse the repository at this point in the history 
* Allow apply defaults to resolve ref and refactor walk listeners

* Refactor

* Refactor

* Refactor

* Refactor

* Refactor

* Refactor

* Refactor

* Update docs

* Refactor

* Refactor

* Refactor

* Add test
  • Loading branch information
@justin-tay
justin-tay committed Mar 12, 2024
1 parent eea61d6 commit 0f983b0
Show file tree
Hide file tree
Showing 28 changed files with 1,250 additions and 298 deletions.
25 changes: 24 additions & 1 deletion doc/upgrading.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,30 @@ This contains information on the notable or breaking changes in each version.

### 1.4.0

This contains breaking changes in how custom meta-schemas are created.
This contains breaking changes
- to those using the walk functionality
- in how custom meta-schemas are created

When using the walker with defaults the `default` across a `$ref` are properly resolved and used.

The behavior for the property listener is now more consistent whether or not validation is enabled. Previously if validation is enabled but the property is `null` the property listener is not called while if validation is not enabled it will be called. Now the property listener will be called in both scenarios.

The following are the breaking changes to those using the walk functionality.

`WalkEvent`
| Field | Change | Notes
|--------------------------|--------------|----------
| `schemaLocation` | Removed | For keywords: `getValidator().getSchemaLocation()`. For items and properties: `getSchema().getSchemaLocation()`
| `evaluationPath` | Removed | For keywords: `getValidator().getEvaluationPath()`. For items and properties: `getSchema().getEvaluationPath()`
| `schemaNode` | Removed | `getSchema().getSchemaNode()`
| `parentSchema` | Removed | `getSchema().getParentSchema()`
| `schema` | New | For keywords this is the parent schema of the validator. For items and properties this is the item or property schema being evaluated.
| `node` | Renamed | `instanceNode`
| `currentJsonSchemaFactory`| Removed | `getSchema().getValidationContext().getJsonSchemaFactory()`
| `validator` | New | The validator indicated by the keyword.


The following are the breaking changes in how custom meta-schemas are created.

`JsonSchemaFactory`
* The following were renamed on `JsonSchemaFactory` builder
Expand Down
85 changes: 40 additions & 45 deletions doc/walkers.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -38,65 +38,61 @@ public interface JsonSchemaWalker {
The JSONValidator interface extends this new interface thus allowing all the validator's defined in library to implement this new interface. BaseJsonValidator class provides a default implementation of the walk method. In this case the walk method does nothing but validating based on shouldValidateSchema parameter.

```java
/**
/**
* This is default implementation of walk method. Its job is to call the
* validate method if shouldValidateSchema is enabled.
*/
@Override
public Set<ValidationMessage> walk(ExecutionContext executionContext, JsonNode node, JsonNode rootNode,
JsonNodePath instanceLocation, boolean shouldValidateSchema);
Set<ValidationMessage> validationMessages = new LinkedHashSet<ValidationMessage>();
if (shouldValidateSchema) {
validationMessages = validate(executionContext, node, rootNode, instanceLocation);
}
return validationMessages;
default Set<ValidationMessage> walk(ExecutionContext executionContext, JsonNode node, JsonNode rootNode,
JsonNodePath instanceLocation, boolean shouldValidateSchema) {
return shouldValidateSchema ? validate(executionContext, node, rootNode, instanceLocation)
: Collections.emptySet();
}

```

A new walk method added to the JSONSchema class allows us to walk through the JSONSchema.

```java
public ValidationResult walk(JsonNode node, boolean shouldValidateSchema) {
// Create the collector context object.
CollectorContext collectorContext = new CollectorContext();
// Set the collector context in thread info, this is unique for every thread.
ThreadInfo.set(CollectorContext.COLLECTOR_CONTEXT_THREAD_LOCAL_KEY, collectorContext);
Set<ValidationMessage> errors = walk(node, node, AT_ROOT, shouldValidateSchema);
// Load all the data from collectors into the context.
collectorContext.loadCollectors();
// Collect errors and collector context into validation result.
ValidationResult validationResult = new ValidationResult(errors, collectorContext);
return validationResult;
public ValidationResult walk(JsonNode node, boolean validate) {
return walk(createExecutionContext(), node, validate);
}

@Override
public Set<ValidationMessage> walk(JsonNode node, JsonNode rootNode, String at, boolean shouldValidateSchema) {
Set<ValidationMessage> validationMessages = new LinkedHashSet<ValidationMessage>();
public Set<ValidationMessage> walk(ExecutionContext executionContext, JsonNode node, JsonNode rootNode,
JsonNodePath instanceLocation, boolean shouldValidateSchema) {
Set<ValidationMessage> errors = new LinkedHashSet<>();
// Walk through all the JSONWalker's.
for (Entry<String, JsonValidator> entry : validators.entrySet()) {
JsonWalker jsonWalker = entry.getValue();
String schemaPathWithKeyword = entry.getKey();
for (JsonValidator validator : getValidators()) {
JsonNodePath evaluationPathWithKeyword = validator.getEvaluationPath();
try {
// Call all the pre-walk listeners. If all the pre-walk listeners return true
// then continue to walk method.
if (keywordWalkListenerRunner.runPreWalkListeners(schemaPathWithKeyword, node, rootNode, at, schemaPath,
schemaNode, parentSchema)) {
validationMessages.addAll(jsonWalker.walk(node, rootNode, at, shouldValidateSchema));
// Call all the pre-walk listeners. If at least one of the pre walk listeners
// returns SKIP, then skip the walk.
if (this.validationContext.getConfig().getKeywordWalkListenerRunner().runPreWalkListeners(executionContext,
evaluationPathWithKeyword.getName(-1), node, rootNode, instanceLocation,
this, validator)) {
Set<ValidationMessage> results = null;
try {
results = validator.walk(executionContext, node, rootNode, instanceLocation, shouldValidateSchema);
} finally {
if (results != null && !results.isEmpty()) {
errors.addAll(results);
}
}
}
} finally {
// Call all the post-walk listeners.
keywordWalkListenerRunner.runPostWalkListeners(schemaPathWithKeyword, node, rootNode, at, schemaPath,
schemaNode, parentSchema, validationMessages);
this.validationContext.getConfig().getKeywordWalkListenerRunner().runPostWalkListeners(executionContext,
evaluationPathWithKeyword.getName(-1), node, rootNode, instanceLocation,
this, validator, errors);
}
}
return validationMessages;
return errors;
}
```
Following code snippet shows how to call the walk method on a JsonSchema instance.

```
ValidationResult result = jsonSchema.walk(data,false);
```java
ValidationResult result = jsonSchema.walk(data, false);

```

Expand All @@ -122,7 +118,7 @@ private static class PropertiesKeywordListener implements JsonSchemaWalkListener

@Override
public WalkFlow onWalkStart(WalkEvent keywordWalkEvent) {
JsonNode schemaNode = keywordWalkEvent.getSchemaNode();
JsonNode schemaNode = keywordWalkEvent.getSchema().getSchemaNode();
if (schemaNode.get("title").textValue().equals("Property3")) {
return WalkFlow.SKIP;
}
Expand All @@ -146,7 +142,7 @@ SchemaValidatorsConfig schemaValidatorsConfig = new SchemaValidatorsConfig();
schemaValidatorsConfig.addKeywordWalkListener(ValidatorTypeCode.PROPERTIES.getValue(),
new PropertiesKeywordListener());
final JsonSchemaFactory schemaFactory = JsonSchemaFactory
.builder(JsonSchemaFactory.getInstance(SpecVersion.VersionFlag.V201909)).addMetaSchema(metaSchema)
.builder(JsonSchemaFactory.getInstance(SpecVersion.VersionFlag.V201909)).metaSchema(metaSchema)
.build();
this.jsonSchema = schemaFactory.getSchema(getSchema(), schemaValidatorsConfig);

Expand All @@ -160,7 +156,7 @@ Both property walk listeners and keyword walk listener can be modeled by using t
SchemaValidatorsConfig schemaValidatorsConfig = new SchemaValidatorsConfig();
schemaValidatorsConfig.addPropertyWalkListener(new ExamplePropertyWalkListener());
final JsonSchemaFactory schemaFactory = JsonSchemaFactory
.builder(JsonSchemaFactory.getInstance(SpecVersion.VersionFlag.V201909)).addMetaSchema(metaSchema)
.builder(JsonSchemaFactory.getInstance(SpecVersion.VersionFlag.V201909)).metaSchema(metaSchema)
.build();
this.jsonSchema = schemaFactory.getSchema(getSchema(), schemaValidatorsConfig);

Expand All @@ -176,16 +172,15 @@ Following snippet shows the details captured by WalkEvent instance.

```java
public class WalkEvent {

private ExecutionContext executionContext;
private SchemaLocation schemaLocation;
private JsonNodePath evaluationPath;
private JsonNode schemaNode;
private JsonSchema parentSchema;
private JsonSchema schema;
private String keyword;
private JsonNode node;
private JsonNode rootNode;
private JsonNode instanceNode;
private JsonNodePath instanceLocation;
private JsonValidator validator;
...
}
```

### Sample Flow
Expand Down Expand Up @@ -285,7 +280,7 @@ But if we apply defaults while walking, then required validation passes, and the
JsonSchemaFactory schemaFactory = JsonSchemaFactory.getInstance(SpecVersion.VersionFlag.V4);
SchemaValidatorsConfig schemaValidatorsConfig = new SchemaValidatorsConfig();
schemaValidatorsConfig.setApplyDefaultsStrategy(new ApplyDefaultsStrategy(true, true, true));
JsonSchema jsonSchema = schemaFactory.getSchema(getClass().getClassLoader().getResourceAsStream("schema.json"), schemaValidatorsConfig);
JsonSchema jsonSchema = schemaFactory.getSchema(SchemaLocation.of("classpath:schema.json"), schemaValidatorsConfig);

JsonNode inputNode = objectMapper.readTree(getClass().getClassLoader().getResourceAsStream("data.json"));
ValidationResult result = jsonSchema.walk(inputNode, true);
Expand Down
5 changes: 2 additions & 3 deletions src/main/java/com/networknt/schema/ConstValidator.java
View file
Original file line number Diff line number Diff line change
Expand Up @@ -27,11 +27,10 @@
*/
public class ConstValidator extends BaseJsonValidator implements JsonValidator {
private static final Logger logger = LoggerFactory.getLogger(ConstValidator.class);
JsonNode schemaNode;

public ConstValidator(SchemaLocation schemaLocation, JsonNodePath evaluationPath, JsonNode schemaNode, JsonSchema parentSchema, ValidationContext validationContext) {
public ConstValidator(SchemaLocation schemaLocation, JsonNodePath evaluationPath, JsonNode schemaNode,
JsonSchema parentSchema, ValidationContext validationContext) {
super(schemaLocation, evaluationPath, schemaNode, parentSchema, ValidatorTypeCode.CONST, validationContext);
this.schemaNode = schemaNode;
}

public Set<ValidationMessage> validate(ExecutionContext executionContext, JsonNode node, JsonNode rootNode, JsonNodePath instanceLocation) {
Expand Down
2 changes: 1 addition & 1 deletion src/main/java/com/networknt/schema/DynamicRefValidator.java
View file
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@
public class DynamicRefValidator extends BaseJsonValidator {
private static final Logger logger = LoggerFactory.getLogger(DynamicRefValidator.class);

protected JsonSchemaRef schema;
protected final JsonSchemaRef schema;

public DynamicRefValidator(SchemaLocation schemaLocation, JsonNodePath evaluationPath, JsonNode schemaNode, JsonSchema parentSchema, ValidationContext validationContext) {
super(schemaLocation, evaluationPath, schemaNode, parentSchema, ValidatorTypeCode.DYNAMIC_REF, validationContext);
Expand Down
27 changes: 20 additions & 7 deletions src/main/java/com/networknt/schema/ItemsValidator.java
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.node.ArrayNode;
import com.networknt.schema.annotation.JsonNodeAnnotation;
import com.networknt.schema.utils.JsonSchemaRefs;
import com.networknt.schema.utils.SetView;

import org.slf4j.Logger;
Expand All @@ -35,7 +36,7 @@ public class ItemsValidator extends BaseJsonValidator {

private final JsonSchema schema;
private final List<JsonSchema> tupleSchema;
private Boolean additionalItems;
private final Boolean additionalItems;
private final JsonSchema additionalSchema;

private Boolean hasUnevaluatedItemsValidator = null;
Expand All @@ -47,6 +48,8 @@ public class ItemsValidator extends BaseJsonValidator {
public ItemsValidator(SchemaLocation schemaLocation, JsonNodePath evaluationPath, JsonNode schemaNode, JsonSchema parentSchema, ValidationContext validationContext) {
super(schemaLocation, evaluationPath, schemaNode, parentSchema, ValidatorTypeCode.ITEMS, validationContext);

Boolean additionalItems = null;

this.tupleSchema = new ArrayList<>();
JsonSchema foundSchema = null;
JsonSchema foundAdditionalSchema = null;
Expand All @@ -66,14 +69,15 @@ public ItemsValidator(SchemaLocation schemaLocation, JsonNodePath evaluationPath
if (addItemNode != null) {
additionalItemsSchemaNode = addItemNode;
if (addItemNode.isBoolean()) {
this.additionalItems = addItemNode.asBoolean();
additionalItems = addItemNode.asBoolean();
} else if (addItemNode.isObject()) {
foundAdditionalSchema = validationContext.newSchema(
parentSchema.schemaLocation.append(PROPERTY_ADDITIONAL_ITEMS),
parentSchema.evaluationPath.append(PROPERTY_ADDITIONAL_ITEMS), addItemNode, parentSchema);
}
}
}
this.additionalItems = additionalItems;
this.schema = foundSchema;
this.additionalSchema = foundAdditionalSchema;
this.additionalItemsEvaluationPath = parentSchema.evaluationPath.append(PROPERTY_ADDITIONAL_ITEMS);
Expand Down Expand Up @@ -205,7 +209,7 @@ public Set<ValidationMessage> walk(ExecutionContext executionContext, JsonNode n
JsonNode defaultNode = null;
if (this.validationContext.getConfig().getApplyDefaultsStrategy().shouldApplyArrayDefaults()
&& this.schema != null) {
defaultNode = this.schema.getSchemaNode().get("default");
defaultNode = getDefaultNode(this.schema);
}
int i = 0;
for (JsonNode n : arrayNode) {
Expand All @@ -222,6 +226,17 @@ public Set<ValidationMessage> walk(ExecutionContext executionContext, JsonNode n
return validationMessages;
}

private static JsonNode getDefaultNode(JsonSchema schema) {
JsonNode result = schema.getSchemaNode().get("default");
if (result == null) {
JsonSchemaRef schemaRef = JsonSchemaRefs.from(schema);
if (schemaRef != null) {
result = getDefaultNode(schemaRef.getSchema());
}
}
return result;
}

private void doWalk(ExecutionContext executionContext, HashSet<ValidationMessage> validationMessages, int i, JsonNode node,
JsonNode rootNode, JsonNodePath instanceLocation, boolean shouldValidateSchema) {
if (this.schema != null) {
Expand All @@ -247,14 +262,12 @@ private void doWalk(ExecutionContext executionContext, HashSet<ValidationMessage
private void walkSchema(ExecutionContext executionContext, JsonSchema walkSchema, JsonNode node, JsonNode rootNode,
JsonNodePath instanceLocation, boolean shouldValidateSchema, Set<ValidationMessage> validationMessages) {
boolean executeWalk = this.validationContext.getConfig().getItemWalkListenerRunner().runPreWalkListeners(executionContext, ValidatorTypeCode.ITEMS.getValue(),
node, rootNode, instanceLocation, walkSchema.getEvaluationPath(), walkSchema.getSchemaLocation(),
walkSchema.getSchemaNode(), walkSchema.getParentSchema(), this.validationContext, this.validationContext.getJsonSchemaFactory());
node, rootNode, instanceLocation, walkSchema, this);
if (executeWalk) {
validationMessages.addAll(walkSchema.walk(executionContext, node, rootNode, instanceLocation, shouldValidateSchema));
}
this.validationContext.getConfig().getItemWalkListenerRunner().runPostWalkListeners(executionContext, ValidatorTypeCode.ITEMS.getValue(), node, rootNode,
instanceLocation, this.evaluationPath, walkSchema.getSchemaLocation(),
walkSchema.getSchemaNode(), walkSchema.getParentSchema(), this.validationContext, this.validationContext.getJsonSchemaFactory(), validationMessages);
instanceLocation, walkSchema, this, validationMessages);

}

Expand Down
26 changes: 16 additions & 10 deletions src/main/java/com/networknt/schema/ItemsValidator202012.java
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.node.ArrayNode;
import com.networknt.schema.annotation.JsonNodeAnnotation;
import com.networknt.schema.utils.JsonSchemaRefs;
import com.networknt.schema.utils.SetView;

import org.slf4j.Logger;
Expand Down Expand Up @@ -119,7 +120,7 @@ public Set<ValidationMessage> walk(ExecutionContext executionContext, JsonNode n
JsonNode defaultNode = null;
if (this.validationContext.getConfig().getApplyDefaultsStrategy().shouldApplyArrayDefaults()
&& this.schema != null) {
defaultNode = this.schema.getSchemaNode().get("default");
defaultNode = getDefaultNode(this.schema);
}
for (int i = this.prefixCount; i < node.size(); ++i) {
JsonNode n = node.get(i);
Expand All @@ -139,6 +140,17 @@ public Set<ValidationMessage> walk(ExecutionContext executionContext, JsonNode n
return validationMessages;
}

private static JsonNode getDefaultNode(JsonSchema schema) {
JsonNode result = schema.getSchemaNode().get("default");
if (result == null) {
JsonSchemaRef schemaRef = JsonSchemaRefs.from(schema);
if (schemaRef != null) {
result = getDefaultNode(schemaRef.getSchema());
}
}
return result;
}

private void walkSchema(ExecutionContext executionContext, JsonSchema walkSchema, JsonNode node, JsonNode rootNode,
JsonNodePath instanceLocation, boolean shouldValidateSchema, Set<ValidationMessage> validationMessages) {
//@formatter:off
Expand All @@ -148,10 +160,7 @@ private void walkSchema(ExecutionContext executionContext, JsonSchema walkSchema
node,
rootNode,
instanceLocation,
walkSchema.getEvaluationPath(),
walkSchema.getSchemaLocation(),
walkSchema.getSchemaNode(),
walkSchema.getParentSchema(), this.validationContext, this.validationContext.getJsonSchemaFactory()
walkSchema, this
);
if (executeWalk) {
validationMessages.addAll(walkSchema.walk(executionContext, node, rootNode, instanceLocation, shouldValidateSchema));
Expand All @@ -162,11 +171,8 @@ private void walkSchema(ExecutionContext executionContext, JsonSchema walkSchema
node,
rootNode,
instanceLocation,
this.evaluationPath,
walkSchema.getSchemaLocation(),
walkSchema.getSchemaNode(),
walkSchema.getParentSchema(),
this.validationContext, this.validationContext.getJsonSchemaFactory(), validationMessages
walkSchema,
this, validationMessages
);
//@formatter:on
}
Expand Down

0 comments on commit 0f983b0

Please sign in to comment.