support multiple json files

[2beers]

Does this library support multiple json files?

I'm having this schema:

{
  "$schema": "http://json-schema.org/draft-04/schema#",

  "definitions": {
    "address": {
      "type": "object",
      "properties": {
        "street_address": { "type": "string" },
        "city":           { "type": "string" },
        "state":          { "type": "string" }
      },
      "required": ["street_address", "city", "state"]
    }
  },

  "type": "object",

  "properties": {
    "billing_address": { "$ref": "file2.json#/address" },
    "shipping_address": { "$ref": "file2.json#/address" }
  }
}

and file2.json

 {
    "address": {
      "type": "object",
      "properties": {
        "street_address": { "type": "string" },
        "city":           { "type": "string" },
        "state":          { "type": "string" }
      },
      "required": ["street_address", "city", "state"]
    }
  }

and I'm getting this error:

Exception in thread "main" com.networknt.schema.JsonSchemaException: com.fasterxml.jackson.databind.JsonMappingException: No content to map due to end-of-input
 at [Source: UNKNOWN; line: 1, column: 0]
	at com.networknt.schema.JsonSchemaFactory.getSchema(JsonSchemaFactory.java:57)
	at com.networknt.schema.RefValidator.<init>(RefValidator.java:63)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
	at com.networknt.schema.JsonSchema.read(JsonSchema.java:120)
	at com.networknt.schema.JsonSchema.<init>(JsonSchema.java:54)
	at com.networknt.schema.PropertiesValidator.<init>(PropertiesValidator.java:36)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
	at com.networknt.schema.JsonSchema.read(JsonSchema.java:120)
	at com.networknt.schema.JsonSchema.<init>(JsonSchema.java:54)
	at com.networknt.schema.JsonSchema.<init>(JsonSchema.java:44)
	at com.networknt.schema.JsonSchemaFactory.getSchema(JsonSchemaFactory.java:64)
	at json.JsonTest.main(JsonTest.java:19)
Caused by: com.fasterxml.jackson.databind.JsonMappingException: No content to map due to end-of-input
 at [Source: UNKNOWN; line: 1, column: 0]
	at com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:261)
	at com.fasterxml.jackson.databind.ObjectMapper._initForReading(ObjectMapper.java:3829)
	at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3774)
	at com.fasterxml.jackson.databind.ObjectMapper.readTree(ObjectMapper.java:2312)
	at com.networknt.schema.JsonSchemaFactory.getSchema(JsonSchemaFactory.java:53)
	... 17 more

On single json file it works. Am I doing something wrong?

[stevehu]

Yes. It supports multiple files; however, you cannot refer to your local file system as in JSON schema spec, the $ref is a uri and it has to refer to a web url. You have the same limitation in swagger specification as well.

The following is one of my test cases and it gives you an example on how to use remote $ref. Hope it helps. Thanks.

https://github.com/networknt/json-schema-validator/blob/master/src/test/resources/tests/refRemote.json

[2beers]

Thanks. What I wanted was to load files relative to the main json file.
In the end I was able to achieve this using:

"$ref": "file:file2.json"

I hope you won't change this in the future, since this is very important to me.

Good job on library

[stevehu]

Yes. That is the right syntax for file system reference. It won't be changed:)

[henrik-io]

Hi there. I just started playing with this library today, and ran into the same issue. I can get it to work with the "$ref": "file:file2.json" syntax shown above, but my schemas will end up in a jar file, so I would need to load the relative schema files from the class path. The json-schema-validator library supports loading schemas from the class path through resource:... which is really handy. Does your library support something similar? Thanks.

[stevehu]

@henrik-io Could you please take a look at #45 and see if it works for you? There is an example to show you how to use it. Let me or @kenwa know if you have questions. Thanks.

[kenwa]

@henrik-io I see that the test case added in #45 only shows how to use the classpath prefix, but you can also use a resource prefix if you prefer that. Both prefixes work in the same way.

[henrik-io]

Hi there @stevehu and @kenwa, and thanks for the added test. This may be more of a JSON Schema newbie question, but I just can't seem to get this to work. I'm trying to validate an instance of the first schema, and I don't get any complaints from the validator, but when I run it in the debugger, it shows that it has no schema for the "url" property. These are my two schemas:

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "type": "object",
  "properties": {
    "url": {
      "id": "classpath:/json-schemas/url-schema.json",
      "type": "string",
      "$ref": "#/url"
    }
  },
  "required": ["url"]
}

URL schema:

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "type": "string",
  "pattern": "^\\s*https?://\\w+.\\w+(\\.\\w+)*.*$"
}

Both schemas reside in my project under src/main/resources/json-schemas. What am I doing wrong? Any help would be much appreciated.

[henrik-io]

I finally figured it out by reading this excellent guide. So now each of my documents have an id property that contains classpath:/.../blah.json (or whatever the name of the file is). So it now looks like this:

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "id": "classpath:/json-schemas/webhook-schema.json",
  "type": "object",
  "properties": {
    "url": {
      "$ref": "url-schema.json"
    },
  "required": ["url"]
}

and the url-schema.json file looks like this:

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "id": "classpath:/json-schemas/url-schema.json",
  "type": "string",
  "pattern": "^\\s*https?://\\w+.\\w+(\\.\\w+)*.*$"
}

This seems to work great. Thanks again for the test case example, it definitely put me on the right track!

[stevehu]

@henrik-io I am glad that you figured it out. The detailed documentation about your solution is very helpful and I am sure others will find it very helpful. Thanks.

[Whathecode]

I believe I struggled with a similar problem, and answering here for posterity. Perhaps some of this can be included in the documentation at some point in the future.

First of all, it is worth being aware about the advice on how to use $id. It should really be the identifier of the schema, and not the retrieval URI. (I believe henrik-io's solution goes counter that.) But it's relevant to be aware about how the retrieval URI is used to resolve $id when not present.

If you adopt a one file per schema approach, and match the .json files in the file system with that of relative $id paths, you get a lot of stuff "for free" during schema development. Therefore, I found out this is a good convention to follow.

First, I did include $id, and set it in each schema matching a fully qualified URI. I then used .uriFetcher to intercept requests to my namespace and swap them out with file: lookups.

val localSchemaFetcher =
    object : URIFetcher
    {
        private val uriFetcher = URLFetcher()

        override fun fetch( uri: URI ): InputStream
        {
            if ( uri.host == "carp.cachet.dk" )
            {
                val localPath = uri.path.replaceFirst( "/", "" )
                return uriFetcher.fetch( URI( "file:$localPath" ) )
            }

            return uriFetcher.fetch( uri )
        }
    }

val schemaFactory = JsonSchemaFactory.Builder()
    .addMetaSchema( metaSchemaVersion )
    .defaultMetaSchemaURI( metaSchemaVersion.uri )
    .uriFetcher( localSchemaFetcher, "https" )
    .build()

In fact, if you follow this convention (and you also include the .json file extension in the identifier, which I see no problem with), you can even remove all $ids. The file paths fully dictate the $id, and depending on the initial retrieval URL, everything will work. At the very least, this is great for development, but you may still want to transform the files when publishing, perhaps using a bundler (haven't used one yet).

[stevehu]

@Whathecode You have described a process I am following all the time to create big OpenAPI 3.0 specifications with tons of schemas. I also have a bundler to bundle all files into one openapi.yml for the light-codegen input.

I am wondering if you could create a document in the doc folder with the content above. It will help lots of other users. Thanks.