Add stripUnknownKeys option to Joi.validate

[travisfischer]

Context

• node version: v5.10.1
• joi version: 8.1.1
• environment (node, browser): node
• used with (hapi, standalone, ...): standalone
• any other relevant information: NA

What I am trying to achieve

Several users want the ability to strip unknown object keys without also stripping array items which do not conform to the valid schema type.

This has been highlighted in #615 as well as #903

Example Scenario:

'use strict';

const Joi = require('joi');

const schema = Joi.object().keys({
  emails: Joi.array()
    .items(
      Joi.object().keys({
        value: Joi.string().email()
      })
    )
});

const dataWithUnknown = {
  unknownKey: 'value',
  emails: [{
      value: 'hagrid@potterworld.com'
  }, {
      value: 'hagrid@harrypotterworld.com'
  }],
};

const dataWithBadEmail = {
  unknownKey: 'value',
  emails: [{
      value: 'hagrid    @potterworld@@@'
  }, {
      value: 'hagrid@harrypotterworld.com'
  }],
};

let result = Joi.validate(dataWithUnknown, schema);
console.log('Case #1 | valid emails - stripUnknown: false\n', require('util').inspect(result, { depth: 3 }));

result = Joi.validate(dataWithUnknown, schema, { stripUnknown: true });
console.log('Case #2 | valid emails - stripUnknown: true\n', require('util').inspect(result, { depth: 3 }));

result = Joi.validate(dataWithBadEmail, schema);
console.log('Case #3 | invalid email - stripUnknown: false\n', require('util').inspect(result, { depth: 3 }));

result = Joi.validate(dataWithBadEmail, schema, { stripUnknown: true });
console.log('Case #4 | invalid email - stripUnknown: true\n', require('util').inspect(result, { depth: 3 }));

The result I had

Case #1 | valid emails - stripUnknown: false
 { error:
   { [ValidationError: "unknownKey" is not allowed]
     isJoi: true,
     name: 'ValidationError',
     details:
      [ { message: '"unknownKey" is not allowed',
          path: 'unknownKey',
          type: 'object.allowUnknown',
          context: [Object] } ],
     _object: { unknownKey: 'value', emails: [ [Object], [Object] ] },
     annotate: [Function] },
  value:
   { unknownKey: 'value',
     emails:
      [ { value: 'hagrid@potterworld.com' },
        { value: 'hagrid@harrypotterworld.com' } ] } }


Case #2 | valid emails - stripUnknown: true
 { error: null,
  value:
   { emails:
      [ { value: 'hagrid@potterworld.com' },
        { value: 'hagrid@harrypotterworld.com' } ] } }

Case #3 | invalid email - stripUnknown: false
 { error:
   { [ValidationError: child "emails" fails because ["emails" at position 0 fails because [child "value" fails because ["value" must be a valid email]]]]
     isJoi: true,
     name: 'ValidationError',
     details:
      [ { message: '"value" must be a valid email',
          path: 'emails.0.value',
          type: 'string.email',
          context: [Object] } ],
     _object: { unknownKey: 'value', emails: [ [Object], [Object] ] },
     annotate: [Function] },
  value:
   { unknownKey: 'value',
     emails:
      [ { value: 'hagrid    @potterworld@@@' },
        { value: 'hagrid@harrypotterworld.com' } ] } }

Case #4 | invalid email - stripUnknown: true
 { error: null,
  value: { emails: [ { value: 'hagrid@harrypotterworld.com' } ] } }

What I expected

In the scenario above cases 1, 2 and 3 work as expected. However, case 4 demonstrates the issue we would like a solution to. We need to be able to strip out unknown object keys while also validating the values of array items rather than stripping them from the array.

The resolution proposed in #615 was to add a new option flag.

I would argue that the least surprising behavior for an option named stripUnknown would be strip unknown keys but NOT strip items in an array which don't conform to the schema for items of that array. The term "unknown" is specifically misleading in this case because nothing is "unknown" about the invalid email value above.

The documentation is also misleading: https://github.com/hapijs/joi/blame/v9.0.0-2/API.md#L142

"when true, unknown keys are deleted (only when value is an object or an array). Defaults to false."

as it only references removing keys not array members.

However, as highlighted in #615 there is disagreement about that that particular option. We can agree to disagree.

Assuming that modifying the behavior of stripUnknown is off the table, I believe the best solution is to add an option named stripUnknownKeys which ONLY strips unknown object keys and still returns validation errors for items in an array which simply have invalid values. The correct behavior for this option would be to remove unknown keys from objects in the array but not the entire objects themselves.

Desired behavior would look something like:

'use strict';

const Joi = require('joi');

const schema = Joi.object().keys({
  emails: Joi.array()
    .items(
      Joi.object().keys({
        value: Joi.string().email()
      })
    )
});

const dataWithBadEmail = {
  unknownKey: 'value',
  emails: [{
      value: 'hagrid    @potterworld@@@'
  }, {
      value: 'hagrid@harrypotterworld.com'
  }],
};

const goodDataWithUnknowns = {
  unknownKey: 'value should be stripped',
  emails: [{
      value: 'hagrid@potterworld.info',
      extraInfo: 'should be stripped.
  }, {
      value: 'hagrid@harrypotterworld.com'
  }],
};

result = Joi.validate(dataWithBadEmail, schema, { stripUnknownKeys: true });
console.log('Invalid email - stripUnknownKeys: true\n', require('util').inspect(result, { depth: 3 }));

result = Joi.validate(goodDataWithUnknowns, schema, { stripUnknownKeys: true });
console.log('Good data - stripUnknownKeys: true\n', require('util').inspect(result, { depth: 3 }));

Output:

Invalid email - stripUnknown: true
 { error:
   { [ValidationError: child "emails" fails because ["emails" at position 0 fails because [child "value" fails because ["value" must be a valid email]]]]
     isJoi: true,
     name: 'ValidationError',
     details:
      [ { message: '"value" must be a valid email',
          path: 'emails.0.value',
          type: 'string.email',
          context: [Object] } ],
     _object: { unknownKey: 'value', emails: [ [Object], [Object] ] },
     annotate: [Function] },
  value:
   { emails:
      [ { value: 'hagrid    @potterworld@@@' },
        { value: 'hagrid@harrypotterworld.com' } ] } }

Good data - stripUnknown: true
 { error: null,
  value:
   { emails:
      [ { value:  'hagrid@potterworld.info' },
        { value: 'hagrid@harrypotterworld.com' } ] } }

Happy to discuss more and help with implementation if this proposal or something similar is accepted.

[Marsup]

As you said, breaking it is out of the table. Hopefully this will satisfy everyone.

[lock]

This thread has been automatically locked due to inactivity. Please open a new issue for related bugs or questions following the new issue template instructions.