Clone default values to avoid seemingly surprising behavior

[imbstack]

Currently if the default value inserted with useDefaults is an object and is subsequently modified on the validated data, the underlying default in the schema is actually modified. That is a confusing sentence, so it's probably just easier to see it in action. The test case in this PR shows an example of how this would occur. Without the change in this PR, it would fail on line 429 of options.spec.js as follows

 npm run build && mocha spec/options.spec.js -R spec -g 'underlying'

<...>

  Ajv Options
    useDefaults
      1) should not modify underlying defaults when modifying validated data


  0 passing (39ms)
  1 failing

  1) Ajv Options useDefaults should not modify underlying defaults when modifying validated data:

      AssertionError: expected [ 'a-default', 'another-value' ] to deeply equal [ 'a-default' ]
      + expected - actual

       [
         "a-default"
      -  "another-value"
       ]

      at test (spec/options.spec.js:429:29)
      at Context.<anonymous> (spec/options.spec.js:405:7)

I'm not 100% sure this is actually a bug, however it does seem to be surprising behavior. If this is not something we want to fix, perhaps the right thing to do is to just add something to the docs that makes mention of this.

Thanks!

[jonasfj]

Nice this looks good, if not a bug it's definitely a major footgun I would like to be without.

If not desired as default behavior for useDefault: true, it could be done with an option like useDefault: 'clone'. But it seems sane to have this as the only possible behavior.

[epoberezkin]

That's a valid concern. There is no need to clone the object at the validation time though, instead it can simply be inserted in the generated code: {{= it.useDefault($sch.default) }} should be replaced with {{= JSON.stringify($sch.default) }}, possibly with some conditional.

The reason it.useDefault was used in the first place is performance - it is faster to insert the reference than actual object, which is acceptable in cases you don't need to modify the data being validated.

So I think it should be based on some option, e.g. cloneDefaults: true (instantiating an object with an object literal is slower and is only necessary if you further modify the validated data), so the code will be:

{{= it.opts.cloneDefaults ? JSON.stringify($sch.default) : it.useDefault($sch.default) }}

[epoberezkin]

or probably better to use {{= it.opts.useDefaults == 'clone' ? JSON.stringify($sch.default) : it.useDefault($sch.default) }} as @jonasfj suggested - it is consistent with how other options are used.

[jonasfj]

+1 for useDefaults: 'clone' as having two options that can conflict is less fun...

[epoberezkin]

@jonasfj yes.

[imbstack]

That all seems very reasonable. Just have a couple questions:

Should we make the cloning behavior the default (occurs when the option is set to true) and make the faster version occur when the option is set to fast? That seems to align with other options in ajv currently and makes the default version do the less surprising thing.

Also, would you like it more if {{= it.opts.useDefaults == 'clone' ? JSON.stringify($sch.default) : it.useDefault($sch.default) }} gets wrapped into

    {{=$passData}} = {{= it.opts.useDefaults == 'clone'
      ? JSON.stringify($sch.default)
      : it.useDefault($sch.default) }};

in the actual template? It's seems like a rather long line otherwise, but I'm not sure what the best taste for this sort of file is.

[epoberezkin]

Given that it's all optimised for speed a bit surprising is ok. I think it's better if the current behaviour is the default. We will add to options and to filtering data section - it'll be clear enough.

Wrapping isn't that important, whatever you prefer, this is clearer though probably:

{{=$passData}} = {{? it.opts.useDefaults == 'clone' }}
                   {{= JSON.stringify($sch.default) }}
                 {{??}}
                   {{= it.useDefault($sch.default) }}
                 {{?}}

[epoberezkin]

On the other hand you are probably right and less surprising is better. It'll save somebody hours of debugging. In this case let's drop the option idea - nobody will use 'fast' option. And the whole useDefault business will be unused and can be removed.

[epoberezkin]

On the other hand:), if somebody used some dirty hack where you can modify defaults without recompiling the schema, because it's the same object, the hack would stop working. So let's stick with 'clone' option plan :)

[jonasfj]

I think almost anyone running into this is will call it a bug :)

Maintaining this obscure "feature" in the name of backwards compatibility, is a bit extreme.. Similar to how Microsoft maintains bugs in Excel because people rely on them :)

We could do the 'clone' option, and then make it default in the next major release. But I doubt anyone is intentionally relying on this :)
Especially, considering that useDefault was only recently introduced.

[epoberezkin]

Always liked that Microsoft's approach ;)

It only came to my mind only because of #103, which is a similar scenario - dynamically update schema dependency.

If, for example, you want to have the current timestamp in the default object (or anything else dynamically changing, e.g. fill in IP if the referrer of the request is not supplied), the fact that the default is by reference allows to do it efficiently without recompiling the schema, which would be slow.

Filling such dynamic defaults after validation is probably cleaner, I have suggested similar things many times, but in reality people like putting this sort of things in schemas because it allows to change the data structure without having to maintain property references in other places.

I agree that this "feature" is close enough to a "bug" :). I'll think whether to remove it completely or to switch the default by the next major version.

[imbstack]

Awesome. I've updated this with the "clone" option and added info to the docs about it.

[epoberezkin]

Great, thank you very much for spotting and fixing.

[epoberezkin]

@imbstack From v4.0 using object/array literal ("deep-clone") will be the default behaviour and useDefaults: "shared" will be inserting default using the reference (the default behaviour before v4.0).