unprocessedProperties not populating for Fields referencing another definition

[NaveenGurram]

I was looking to leverege recenlty added "unprocessedProperties" but I see it is not loading for a field when it is referencing custom definition.

When I load below schema I was expecting "dateOfBirth" RefSchema has unprocessedProperties value of "unproc7".

is it possible to load unprocessedProperties for a field in properties when referencing another definition?

JSON Schema

{
    "title": "ttt",
    "description": "descr",
    "$id": "http://example.org",
    "default": 22,
    "nullable": false,
    "readOnly": false,
    "writeOnly": false,
    "definitions": {
      "monthDayYearDate": {
        "$comment": "Represents the MM/DD/YYYY date format.",
        "description": "(MM/dd/yyyy)",
        "type": "string",
        "maxLength": 10,
        "pattern": "^\\d{2}/\\d{2}/\\d{4}$"
      }
    },
    "type": "object",
    "unproc0": 1,
    "unproc1": "asdasd",
    "properties": {
      "prop": {
        "type": "string"
      },
      "prop2": {
        "unproc4": true,
        "unproc5": null
      },
      "prop3": {
        "$ref": "#",
        "unproc6": false
      },
      "dateOfBirth": {
        "title": "Date of Birth",
        "$ref": "#/definitions/monthDayYearDate",
        "unproc7": {
          "abc": "message 1"
        }
      }
    }

[erosb]

Hello,

the unprocessedProperties field contains only bare json fields, so they don't affect "$ref" processing in any way. Also the specification says that "All other properties in a "$ref" object MUST be ignored."

[NaveenGurram]

@erosb just curious how the "unproc6" having a ref "#" is different from "unproc7"

[erosb]

Good question :)
I think in both references the unprocessed properties should be present in the ReferenceSchema instance but not in the referred schema. I didn't try it out (though while reading SchemaLoader#loadSchemaObject() and SchemaLoader#runSchemaExtractors() I can't see where the control flow would be different).

[NaveenGurram]

@erosb if you think it's the right behavior I can look at the code and issue a PR if I can solve the problem.

[erosb]

@NaveenGurram please proceed (and include at least one test in the PR), thank you.

[NaveenGurram]

@erosb Thanks for providing the fix, but the fix is not working especially when there are more elements referring to same ref element. In the below example for "prop5" I was expecting "unproc09" in unprocessed properties but it is missing and addition to that it is showing the property from "prop4" in prop5.

    "title": "ttt",
    "description": "descr",
    "$id": "http://example.org",
    "default": 22,
    "nullable": false,
    "readOnly": false,
    "writeOnly": false,
    "definitions": {
      "dateMonthDayYear": {
        "description": "(MM/dd/yyyy)",
        "type": "string",
        "maxLength": 10,
        "pattern": "^\\d{2}/\\d{2}/\\d{4}$",
        "unproc8": false
      }
    },
    "type": "object",
    "unproc0": 1,
    "unproc1": "asdasd",
    "properties": {
      "prop": {
        "type": "string"
      },
      "prop2": {
        "unproc4": true,
        "unproc5": null
      },
      "prop3": {
        "$ref": "#",
        "unproc6": false
      },
      "prop4": {
        "$ref": "#/definitions/dateMonthDayYear",
        "unproc7": null
      },
      "prop5": {
        "$ref": "#/definitions/dateMonthDayYear",
        "unproc9": {
          "unproc9-01": false
        }
      }
    }
  }

[erosb]

At least we didn't release the current version :)
Not sure when will I have further time for this. Any attempts to fix the problem are welcome.
Thanks.

[ben-manes]

@erosb can you explain the problem and change here? This patch changed behavior by returning very different values, e.g. now #/allOf/0 instead of the schema's id in ReferenceSchema. That broke mapping the validation as the schemaLocation is no longer the actual schema URI. I cannot tell from this report or the commits what the expected behavior is supposed to be?

[ben-manes]

Okay, so it seems that the semantics of getLocation changed but the details are still available at getSchemaLocation, even though it references a deprecated field. Hopefully that will be a stable API method.

[erosb]

Hello @ben-manes , I'm sorry to see this change causes problems for you. I think the best possible migration plan is using refSchema.getReferredSchema().getSchemaLocation() instead of refSchema.getSchemaLocation() . Would it work for you?