Nested Conditional Properties for Local file references not working in python-jsonschema

[dibdutta]

My base.schema.json looks like below

{"$schema": "http://json-schema.org/draft-07/schema#" }

My defination.schema.json looks like below

{
  "$id": "defination.schema.json",
  "type": "object",
  "properties": {
    "remote_os": {
      "default": "Linux",
      "enum": [
        "Linux",
        "Windows"
      ]
    }
  },
  "required": [
    "remote_os"
  ]
}

Now referenced the definition in update.schema.json

{
  "$id": "update.schema.json",
  "$schema": "http://json-schema.org/draft-07/schema",
  "if": {
    "type": "object",
    "required": [
      "common_data"
    ],
    "properties": {
      "common_data": {
        "$ref": "defination.schema.json",
        "properties": {
          "remote_os": {
            "const": "Linux"
          }
        }
      }
    }
  },
  "then": {
    "type": "object",
    "properties": {
      "dup_file": {
        "type": "string",
        "pattern": "^(.*.)(bin)$"
      }
    }
  },
  "else": {
    "type": "object",
    "properties": {
      "dup_file": {
        "type": "string",
        "pattern": "^(.*.)(exe)$"
      }
    }
  }
}

Basically added condition in JSON schema if {"common_data":{"remote_os":"Windows"}} then "dup_file" name should ended up with ".exe" else ".bin"
Now my RefResolver and Validator script is

from jsonschema import RefResolver
from jsonschema.validators import validator_for
import json

base = json.loads(open('base.schema.json', "r").read())
definitions =json.loads(open('defination.schema.json', "r").read()) # https://jsonschema.dev/s/FZDbO
schema =json.loads(open('update.schema.json', "r").read()) # https://jsonschema.dev/s/lvLFa

schema_store = {
  base.get('$id','base.schema.json') : base,
  definitions.get('$id','defination.schema.json') : definitions,
  schema.get('$id','update.schema.json') : schema,
}

resolver = RefResolver.from_schema(base, store=schema_store)
Validator = validator_for(base)
validator = Validator(schema, resolver=resolver)

data = {
    "common_data": {
        "remote_os": "Windows"
    },
    "dup_file": "ad7a.exe"
}
validator.validate(data)

Getting error : jsonschema.exceptions.ValidationError: 'ad7a.exe' does not match '^(.*.)(bin)$' ,which is wrong.
Since in my validation data {"common_data":{"remote_os":"Windows"}} and "dup_file": "ad7a.exe" , it should not raise any validation error ideally .

Same thing I have tested https://json-schema.hyperjump.io/ it working perfectly fine , suspecting some issue with python-jsonschema only. Any help on this highly appreciated .

[Julian]

Hi there. Please share a self contained (ideally simplified) version that exhibits your issue.

Edit: I see, you shared a few links at the bottom. But yeah simplifying your example would certainly help (and possibly reveal the issue).

[dibdutta]

@Julian I have updated the description and tried to simplify and self contain the issue. Please let me know what you think of the issue. I am trying something similar to https://jsonschema.dev/s/CEwMw but with local file references

[Julian]

Thanks, that's better, but in general a minimal example should be some small number of lines I can simply copy paste to run. In your example for instance, I can see quickly you're using some private helper load_schema. As another example of minimizing, does your issue still happen if you get rid of base schema? I suspect it does. So minimizing all the extra stuff is also a good way to get to the bottom of it.

If this is as far as you can go though I may at some point have some time to minimize it and diagnose.

It'd also be helpful for you to elaborate on why you think something's wrong -- I see your error says 'ad7a.exe' does not match '^(.*.)(bin)$' -- that string indeed does not match that regex (just try re.search('^(.*.)(bin)$', 'ad7a.exe')), so yeah saying something about why that's unexpected will also help.

[dibdutta]

@Julian I have figured out the issue

In my update.schema.json below schema looks like not working

"properties": {
      "common_data": {
        "$ref": "defination.schema.json",
        "properties": {
          "remote_os": {
            "const": "Linux"
          }
        }
      }
    }

But $ref if I include under "properties" then its working.

"properties": {
      "common_data": {
        "properties": {
          "$ref": "defination.schema.json",
          "remote_os": {
            "const": "Linux"
          }
        }
      }
     }

Not sure https://json-schema.hyperjump.io/ didn't complain about both .
I see one more issue if add any other required filed in either of defination.schema.json or update.schema.json and don't add it to the validation data, it throws same error : jsonschema.exceptions.ValidationError: 'ad7a.exe' does not match '^(.*.)(bin)$' ,which is wrong. It should be different validation error message .

[dibdutta]

Thanks, that's better, but in general a minimal example should be some small number of lines I can simply copy paste to run. In your example for instance, I can see quickly you're using some private helper load_schema. As another example of minimizing, does your issue still happen if you get rid of base schema? I suspect it does. So minimizing all the extra stuff is also a good way to get to the bottom of it.

If this is as far as you can go though I may at some point have some time to minimize it and diagnose.

It'd also be helpful for you to elaborate on why you think something's wrong -- I see your error says 'ad7a.exe' does not match '^(.*.)(bin)$' -- that string indeed does not match that regex (just try re.search('^(.*.)(bin)$', 'ad7a.exe')), so yeah saying something about why that's unexpected will also help.

@Julian I have updated the issue description to make it more generic and explanation of 'ad7a.exe' does not match '^(.*.)(bin)$' why wrong.

[Julian]

Thanks, for the future, here's an example of what a more minimal bug report looks like, which is super helpful to maintainers (and this isn't even fully minimal, but by the time I minimized to this point I see what your issue is):

from jsonschema import RefResolver
from jsonschema.validators import Draft7Validator

schema_store = {
  'defination.schema.json' : {
    "properties": {"remote_os": {"enum": ["Linux", "Windows"]}},
  },
  'update.schema.json' : {
  "if": {
    "properties": {
      "common_data": {
        "$ref": "defination.schema.json",
        "properties": {"remote_os": {"const": "Linux"}}
      }
    }
  },
  "then": {"properties": {"dup_file": {"pattern": "^(.*.)(bin)$"}}},
  "else": {"properties": {"dup_file": {"pattern": "^(.*.)(exe)$"}}}},
}

resolver = RefResolver.from_schema(
    schema_store["update.schema.json"], store=schema_store,
)
validator = Draft7Validator(schema_store["update.schema.json"], resolver=resolver)

data = {"common_data": {"remote_os": "Windows"}, "dup_file": "ad7a.exe"}
validator.validate(data)

Note that someone else (i.e. me) can copy paste that into a file and run it with no further work.

That being said, your issue is your use of $ref alongside remote_os in the update.schema.json file -- in the draft you're using, $ref causes all other keywords used alongside it to be ignored. The first draft that changed that was draft2019-09, so if some other validator you used has different behavior for your schema, it sounds like it has a bug perhaps.

The normal way to do this in previous drafts is to use allOf.