Is there a way to print a schema with references resolved?

[kelvinpho]

Hi,

I'm not having any issues with SchemaNode and validation, but I would like a way to print my SchemaNode with $ref resolved. An example of this is on http://www.jsonschema.net/ where they can pretty print the Json Schema in JSON format.

Is there any way to do this with json-schema-validator's SchemaNode or SchemaTree?

Thanks!

[fge]

If I understand correctly, say you have:

{
    "type": "array",
    "items": { "$ref": "foo://bar#" }
}

and at JSON Reference foo://bar# you have:

{
    "type": "string",
    "minLength": 2
}

then you would like to have:

{
    "type": "array",
    "items": {
        "type": "string",
        "minLength": 2
    }
}

and this, recursively?

[kelvinpho]

Yes, that is exactly right. Is this possible with the current API? (I'm using version 2.0.0)

[fge]

Well, by writing a processor, yes it is. However, it is quite delicate. Consider this schema:

{
    "type": "items",
    "minItems": 2,
    "maxItems": 2,
    "items": { "oneOf": [ { "type": "integer" }, { "$ref": "#" } ]
}

This is, by essence, a recursive schema -- it would lead to an infinite loop on expansion.

More generally, such a processor should fail if a resolved JSON Reference is contained within the schema, and the pointer in the schema at which this reference was found is a child of the pointer of the reference. Oh, and there is the case of enum for which we must not expand references.

And that is not all: if we expand a draft v4 schema, we don't want a draft v3 schema to come into the picture either.

As to other cases:

• reference loops, dangling references etc are handled by RefResolver;
• syntax validation is handled by a SyntaxProcessor;
• SchemaTree equivalence is handled by SchemaTreeEquivalence.getInstance().

Such a processor is writable, but not easy!

[fge]

Hmmno, this is not quite right, SchemaTreeEquivalence doesn't fit the bill since it checks for the current resolution context whereas here what is needed is equivalence of the loading URI and base node. So another equivalence needs to be written, but this is the easy part.

[fge]

Anyway, I think this is an interesting thing to have, so I'll give a go at it -- but not immediately: I have an Avro converter to write!

[fge]

There is also the solution that you give a go at it, of course. In this case, if you have questions, do not hesitate to ask ;)

[kelvinpho]

Thanks! This api is new to me but I'll give it a shot.

[fge]

You will need to have a good shot at the API of json-schema-core as well since this is what you will use to build your chains:

http://fge.github.com/json-schema-core/stable/index.html

In particular, look at the processing package.

[kelvinpho]

fge,

Perhaps I'm going about this the wrong way, but I basically need to "walk" the schema the same way that ValidationProcessor "walks" the schema. I see that you use the ArraySchemaSelector and ObjectSchemaSelector as helpers to cache/lookup things. Is there a reason that ObjectSchemaSelector, ObjectSchemaDigester, and ArraySchemaDigester are public, but ArraySchemaSelector has package access?

[fge]

Uhm, that is a visibility bug... The initial plan was to have them all package visible!

But now that you mention it, maybe they should not... Your need here, and another one I will have in the near future, will require that I walk a JSON Schema as well (note that syntax validation also walks it in some way), so maybe this could be factorized away and reused for both syntax and instance validation, and for your use case.

Out of curiosity, how did you plan to use these selectors? And would you mind sending a pull request making ArraySchemaSelector public? For the general case I'll have to think it out some more.

[kelvinpho]

Basically, I am doing deserialization/serialization of proprietary formats. Internally, we want our data representation and parsing to be very flexible.

For example, if I have a json schema:
{
"type" : "object",
"properties" : {
"name" : {
"type" : "string",
"required" : true
}
"age" : {
"type" : "number",
"required" : false
}
}
}

I can write a generic serializer/deserializer from this schema. I will "walk" this schema, then I can know that the first field is called "name" and the 2nd field is called "age", and also validate that the data received is in the appropriate format (i.e. required, string, date, number etc). Of course, any property can be a reference to another schema if the format is very complex.

When we need to adjust the schema, then all it will take is extending/modifying the json schema and our parsing logic can remain intact. Doing this with POJO's is too difficult to maintain. The way you are walking the schema and json value during validation is exactly what I need to do. But instead of a json value, I will be walking a proprietary serialization format.

[fge]

What do you mean by proprietary format? A "proprietary JSON Schema"?

If yes, there may be another solution: write a processor which converts this to a JSON Schema, and if some constraints are not enforceable by existing keywords, you can create your own.

This is what I am currently doing with Avro Schema (I am writing an Avro to JSON Schema translator at the moment).

[kelvinpho]

No, this is not a proprietary json schema, this is a format such as CSV, XML, fixed width, flat file, or some other proprietary format based on raw bytes.

[fge]

OK, and you need to "flatten" JSON Schemas for your particular use case?

The more I think of it, the more I think what is needed is this:

• generalize the KeywordValidator interface, so that its first and third argument can change;
• generalize ValidationProcessor to make its walking logic available to other processors.

Comments?

[kelvinpho]

My original work around was to create a flattened/resolved json schema, and stuff it into a JsonNode that I could walk.

After looking at the way ValidationProcessor is written, I think that your approach is much better. Making the walking logic generic would essentially remove the need for me to create a flattened json schema. I would basically take a set of bytes, and break it up into sub segments as I traversed down into the subfields of the JSON schema. Of course, as I walked through the bytes and schema, I would be building up a value to return to the user of my custom processor.

Theoretically, if the behavior was pulled out of ValidationProcessor, I could choose to walk the schema and build the flattened version, or walk the schema and parse my payload directly.