New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Option to merge defaults #42

Closed
jonasfj opened this Issue Aug 30, 2015 · 20 comments

Comments

Projects
None yet
7 participants
@jonasfj
Contributor

jonasfj commented Aug 30, 2015

In JSON schema it's often useful for declare defaults, with the default value for an optional property. This is powerful for documentation, and many validators supports inserting the default into the validated object, this way you don't hardcode defaults into your code, but declare the defaults in your input schema.

Example A)

var Ajv = require('ajv');
var ajv = Ajv({mergeDefaults: true}); // Specify that we want defaults merged

var validate = ajv.compile({
  type: "object",
  properties: {
    myProp: { // optional property
      type: "string",
      default: "Hello World" // default value
    }
  }
});

var data = {
  someOtherProp: "another value"
};

validate(data); // true
console.log(data)
// {someOtherProp: "another value", myProp: "Hello World"}

This would be extremely useful, and doing this outside of the schema validation is hard, as you want to do this on all sub-objects as well, some of which may exist under an anyOf, so you can't insert the defaults until you're sure which anyOf branch is satisfied.

It's quite possible that this is a post processing step, as nested anyOf branches means you sometimes can't do this until after everything is validated. A possible work around might be to insert the defaults into a clone of the existing object, that way multiple anyOf branches shouldn't be a problem.

Note, this only really relevant for objects, but could also be done for arrays which has items: [{default: ...}, {default: ...}] (though this is a corner case). It doesn't really make sense to do this when the top-level schema type is a value type like integer as the input is either valid (ie. an integer) or invalid (ie. another type or null).

Remark, after inserting a default value, validator should validate the default value too. This should ideally be done a schema compile-time. It is important to do this, because the default value of a property which has type: 'object' may be {} and the schema for that object may specify properties for this that has additional default values. It would also be nice to get errors about inconsistencies at compile-time.

@epoberezkin

This comment has been minimized.

Owner

epoberezkin commented Aug 30, 2015

I agree that it's a useful feature. The reason it is absent is the number of complex and/or contradictory scenarios that should be covered. I am sure that there will be other complications that we don't immediately see. The additional problem with anyOf, for example, is that multiple branches can be satisfied. allOf is also not obvious. Post processing is not going to work, because I think that the expectation is that if default is provided and the value is missing, object with inserted default should be valid, not without. Pre-processing is difficult because it is not clear which branches are going to be valid. Some validators implement complex scenarios with changes and roll-backs...

I will probably implement it at some point, but it's not a simple task. PR is welcome of course :)

@jonasfj

This comment has been minimized.

Contributor

jonasfj commented Aug 31, 2015

One could probably forbid default under allOf branches... (I suspect allOf is rarely used too).
But to determine of anyOf can have multiple branches we would have to build static analysis tools for JSON schemas ... Wow, that got complex really fast :)

Or you define that you choose the first anyOf branch...

@epoberezkin

This comment has been minimized.

Owner

epoberezkin commented Sep 1, 2015

The first valid anyOf branch I assume. So I guess apply defaults first (if it wasn't applied in the previous branch), validate a branch, rollback if it wasn't valid...

@jonasfj

This comment has been minimized.

Contributor

jonasfj commented Sep 2, 2015

Roll back seems hard.. Perhaps it's simplest (and possibly cheapest) to create a new object and
return along with the result. That way you don't mutate the input object too...

@atrniv

This comment has been minimized.

atrniv commented Sep 5, 2015

@jonasfj you might be interested in taking a look at the defaults implementation in Themis. One of the main reasons I wrote it was because of the lack of support for defaults in most existing validators. We've been using it in production quite successfully for quite a while.

@epoberezkin, It is not a simple solution to implement and it does have a significant performance impact but ultimately it provides a level of convenience which makes up for the shortcomings.

@epoberezkin

This comment has been minimized.

Owner

epoberezkin commented Sep 5, 2015

@atrniv, thanks. I was actually thinking about implementation in themis.

@epoberezkin

This comment has been minimized.

Owner

epoberezkin commented Jan 9, 2016

The implementation uses default keywords only in properties and in items (when it's an array). It also ignores them if they are anywhere inside anyOf, oneOf and not as not only the required code is complex to implement but also because the meaning of these defaults is very ambiguous. I think the way it is done it covers the majority of use cases. The possible improvements to compilation are:

  • warn when default keyword is ignored
  • detect situations where minItems/maxItems (etc.) is guaranteed to pass (to simplify the compiled code) or to fail (to throw the exception during compilation).
  • detect situation when property dependency in dependencies keyword is guaranteed to pass.
  • validate defaults against the schema for the item/property (to throw compilation exception when it fails)
@paglias

This comment has been minimized.

paglias commented Jan 21, 2016

@epoberezkin does default support passing a function that returns a value?

For example:

  properties: {
    id: {
      type: 'uuid',
      default: function () { return uuid.v4() }
    },
}

If it doesn't then it would be very useful

@epoberezkin

This comment has been minimized.

Owner

epoberezkin commented Jan 21, 2016

JSON schema is a JSON-document. It should be serialisable. You can do it with custom keyword on the parent object though, e.g.:

{
  uuid: {
    property: 'id',
    version: 4,
    generate: true
  }
}
@paglias

This comment has been minimized.

paglias commented Jan 21, 2016

@epoberezkin thanks!

@epoberezkin

This comment has been minimized.

Owner

epoberezkin commented Jan 21, 2016

actually, with "inline" keyword you can do a more sane thing on the property itself as there you will have access to the parent object and the current property name (check the implementation of default for that):

{
  "properties": {
    "id": {
      "uuid": {
        "version": 4,
        "generate": true
      }
    }
  }
}
@paglias

This comment has been minimized.

paglias commented Jan 21, 2016

@epoberezkin would you mind pointing me to the implementation of default? I'm having some problem finding it,

I came up with this but it's probably wrong:

schemaValidator.addKeyword('uuid', {
  type: 'uuid',
  statements: true,
  inline (it, keyword, schema) {
    return `data || ${generateUUID()}`;
  },
});
@epoberezkin

This comment has been minimized.

Owner

epoberezkin commented Jan 21, 2016

file defaults.def

@epoberezkin

This comment has been minimized.

Owner

epoberezkin commented Jan 22, 2016

@ngryman

This comment has been minimized.

ngryman commented Aug 18, 2016

@paglias I have the exact same need to generate a uuid if not specified. Would you mind sharing your solution? Thanks!

@paglias

This comment has been minimized.

paglias commented Aug 18, 2016

@ngryman sorry but it's been a long time and I don't remember if I found the solution. I'm not using ajv now

@rondonjon

This comment has been minimized.

rondonjon commented Jun 21, 2017

Hello @epoberezkin,

I just ran into this problem ...

not only the required code is complex to implement but also because the meaning of these defaults is very ambiguous.

I don't quite understand.

I am dealing with a property that can either be false or an object. If it is an object, AJV should check its child properties and use a default in case that a child property is missing. This is currently not working.

Since AJV is treating both cases well during validation, the code that selects the correct schema subbranch must already be there and properly working. Shouldn't it then also be possible to load the default value from this particular schema subbranch and use it if no actual value is present?

From your comment, I conclude that defaults are currently implemented elsewhere, but then my naive assumption would be that the problem could be solved by moving the "default picking" code to run right after the branch selection?

@epoberezkin

This comment has been minimized.

Owner

epoberezkin commented Jun 21, 2017

@rondonjon that is not how Ajv works, it has no global awareness about branches, it just validates data against all of them and defaults have to be applied before validation. There is no way Ajv can decide which is the right branch without validating against it. But then if it is not valid, defaults have to be removed. So there is no "branch selection".

The suggestion to work around is exactly that - to explicitly select the branch using some kind of conditionals: "if/then/else" (will be in draft 7) or "select" (both are in ajv-keywords). It both solves defaults problem and excessive error reporting problem (in case of using oneOf/anyOf when all branches are invalid and Ajv has no idea what is the "right" branch).

@rondonjon

This comment has been minimized.

rondonjon commented Jun 21, 2017

Thanks for the clarification, @epoberezkin !

@mnpenner

This comment has been minimized.

mnpenner commented May 22, 2018

Would it not make more sense to implement this in a similar fashion to $ref? e.g.

"key": {
    "type": "string",
    "title": "Key",
    "default": {"$uuid": "v4"}
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment