Implement JSON Schema inlining

[prashanthseshadri]

Hi,

Is it possible to get the expanded version of Json Schema (resolved along with references) in the Java code so that i get a flattened JSON version?
eg:
===== A.json =========
{
"type": "object",
"properties": {
"foo": {
"type": "string"
}
}
}
========= B.json ==============
{
"type": "object",
"properties": {
"bar": {
"type": "array",
"items": {
"$ref": "A.json#/properties/foo"
}
}
}
}

to give me the output as :
{
"type": "object",
"properties": {
"bar": {
"type": "array",
"items": {
"type": "string"
}
}
}
}

Also, is it possible to serialize this data? Jackson was throwing error on serializing it since JsonSchema itself does not have any field to be serialized.
Thanks in advance

[fge]

This is a very common request... This will be included in the next version.

It will be out by the end of the year.

[fge]

As to serialization, well, the original JSON is the serialization format ;)

[prashanthseshadri]

Yeah i get that - i mean the original JSON. But the ref resolution etc., might not happen out of the box via normal InputStream of the Json, which i think would be good to be included in the library in which case it would be good to serialize the JsonSchema object (along with the resolved URLs). :)

[prashanthseshadri]

Also, are you aware of the library (probably UI) which will do the resolution of the URLs? In that case, i can just give out the initial Schema from backend and from there on, UI would know to resolve it!

[fge]

Well, the plan is to have a class which you will get from a JsonSchemaFactory which will handle all the downloading of JSON. It will also perform inlining.

As to what resolves URIs, it is an interface: URIDownloader. You can implement your own for your own scheme. I do not use a dedicated library for that.

[prashanthseshadri]

Okay, thanks for the reply. Waiting for the flattening of Json work! is there any other issue link where i can track the progress of this?

[prashanthseshadri]

And just to clarfiy, i believe I would be able to get the Json Schema (after resolution) as a string some how at the end, right?

[fge]

As to tracking, I guess this issue will be the one (by the way, can you change the title to "Implement JSON Schema inlining"?).

If you want a JSON as a string you'll have to pass by an ObjectMapper. You can for instance do:

JacksonUtils.newMapper().writeValueAsString(theJsonNode);

[prashanthseshadri]

Yeah right, for which theJsonNode has to be got, some how from the JsonSchemaFactory object! So i would like you guys to ensure that the object obtained from JsonSchemaFactory (be JsonNode or any custom POJO) be actually serializable.

Thanks

[prashanthseshadri]

Hi,

Is there any update/progress on this?

[fge]

Not yet; recently I've been playing with another project. This is the next item on my todo list.

This turns out to be quite a complicated thing to do right... You have to account for ids, $schemas if any (what happens for instance if you try and inline a v3 schema into a v4 schema), "local" JSON References etc... It turns out to be quite a challenge.

[Simon-Payne]

Hi Francis,
I could definitely do with this functionality. In my case, I have a RAML API that references several JSON schemas that have quite complex structures, using some draft v4 features such as allOf[...], and JSON pointer references from the schema files to a separate file that has the definitions of various objects in it. The idea was to re-use object definitions in different schemas. I want to be able to generate POJOs from the object types that are being used in the schemas. Having schemas in a flattened format, like prashanthseshadri is asking, sounds like it is on the right lines. So I look forward to hearing more about this feature.
thanks
Simon

[fge]

@Simon-Payne I didn't even consider the deserialization problem to be honest...

Note also that there already exists a functionality whihc allows you to remap URIs; you can for instance tell that http://some.site/someschema is in fact located in resource:/somewhere/on/classpath/theschema. That is a cheap, and imho better, alternative to inlining.

But I'll do inlining anyway. I just have to decide on what to support. It is already a fact that inline addressing is a no. Also, if there are any "internal" JSON References in schemas, I'll have to transform them.

[aschrijver]

@fge I could live with an inlining implementation that has the constraint that it only imports referenced schema's with the same version number.

Supporting mixed version inlining could be a separate issue?

[aschrijver]

@fge Is it possible to already include the inlining code e.g. in a separate branch, or as a snapshot version?

[fge]

@aschrijver I can create a dedicated branch for that.

I am very, truthfully sorry, but I haven't found the time to work seriously on this yet. My main problem comes from the fact that I support so much that defining the context alone in which I may support inlining proves to be a chore in itself.

Right now I have set up the following constraints in order to provide a viable, tested implementation quickly:

• no inline addressing, only canonical;
• any ids present at any place other than at the root of a JSON Schema will throw an UnsupportedOperationException (therefore unchecked);
• JSON References will have to be valid, that is, if a fragment part is present, it will have to be a valid JSON Pointer.

Given these constraints I can come up with a tested inlining implementation. What I'm not sure about yet is the API...

[aschrijver]

And I truly appreciate the time and effort you spend on this great project!!

As far as I can see these are all logical constraints for an initial implementation. Can always further extend if there is a demand for it...

[prashanthseshadri]

Hi Francis,

Could you point us to the branch where you are working on so that we can see if we can use them or add value to it?

And also, how are we going to deal with anyOf ? How does it semantically map to JSON structure? allOf makes little sense to have. But anyOf/oneOf sounds more like tied to validation rather than inlining.

Thoughts?

Thanks

[aschrijver]

@prashanthseshadri : With the inlining I would expect those entries in allOf / anyOf / oneOf that refer to external schema documents to be added to a definitions section inline in the document.

[prashanthseshadri]

Okay. So the point i am asking is not about where we would store the references. its about in inlining, what does allOf actually mean. Normal $ref would mean that take all the properties from that ref and extend withe the current object. But i am not sure how does anyOf /allOf would mean.

[ddossot]

Just giving my 👍 to this: I'd like to inline my top level schemas to make them easier to read for end users (ie no need to navigate to other ref'ed schemas).

[uglyfigurine]

Has there been any progress on this?

[huggsboson]

There has not been any progress, @fge has found greener pastures. I'm happy to accept patches, but I'm unable to publish releases to com.github.fge as of yet, but am working on it.