Joi.validate is slow, Joi.assert is slower

[simlu]

This ticket came from #2253 (comment) and was prompted by @Marsup

Using the following bench code:

const Joi = require('@hapi/joi');
const Ajv = require('ajv');
const joiToJson = require('joi-to-json');

const ajv = new Ajv();

const data = true;
const bench = (name, fnUncompiled) => {
  const fn = fnUncompiled();
  // eliminate cold start bias
  for (let x = 0; x < 1000000; x += 1) {
    fn(data);
  }
  console.time(name);
  for (let x = 0; x < 1000000; x += 1) {
    fn(data);
  }
  console.timeEnd(name);
};

bench('Joi.assert', () => {
  const schema = Joi.boolean();
  return (d) => Joi.assert(d, schema);
});
bench('Joi.validate', () => {
  const schema = Joi.boolean();
  return (d) => schema.validate(d).error === undefined;
});
bench('ajv', () => {
  const jsonSchema = joiToJson(Joi.boolean());
  const validator = ajv.compile(jsonSchema);
  return (d) => validator(d) === true;
});
bench('vanilla', () => (d) => typeof d === 'boolean');

I get the output

Joi.assert: 573.636ms
Joi.validate: 99.948ms
ajv: 3.732ms
vanilla: 3.654ms

Asking the questions:

• Why is Joi.assert so much slower than Joi.validate?
• Why is ajv so much faster than Joi.validate?

[hueniverse]

Joi.assert() throws which has a significant cost. It also performs more copying of options because it allows for per-assert customization. While it could be optimized, I am not planning on spending any of my time on it because I would only user Joi.assert() in setup code, never in runtime hot-path code.

ajv works completely different than joi. It builds custom validation functions in runtime which means it can produce code that basically does exactly what your vanilla example does. The only overhead is an extra function call.

joi is never going to be as fast as ajv, unless we decide to rewrite the entire system for custom compiled functions which I doubt will ever happen. For super simple schemas, you are always going to pay an oversized price when using joi because it has a lot of setup per validation to support advanced features as well as composability. joi gives you significantly more validation power.

As for 100ms for validating a boolean, that does not look right.

[hueniverse]

I've ran the same tests with a different benchmark tool and got the same results but with more useful output:

joi.assert x 1,034,689 ops/sec ±2.47% (78 runs sampled)
joi.validate x 8,623,466 ops/sec ±2.00% (84 runs sampled)
ajv x 851,376,541 ops/sec ±0.84% (96 runs sampled)
node x 858,638,208 ops/sec ±1.01% (96 runs sampled)

Yes, ajv is 100x faster than joi for a schema as simple as this, but at 8m operations a second, joi would be perfectly fine in most real world use cases. When working on v16 we've done a lot of work to make things faster and v16 is faster than v15. I am not planning on doing any more performance work on joi for the foreseeable future (that is, in 2020).

At the end of the day, you need to pick the tool that is right for your application. If performance if your top priority and you need those fractional milliseconds, ajv is going to be a better fit at the cost of reduced functionality.

[Marsup]

I'll admit I kind of expected a full repro of the schema where you halved the response time, this benchmark doesn't really help as much as having a real life scenario.

[simlu]

@Marsup We went from using joi.assert to vanilla. We make thousands of calls with simple schema validation (a bit more complex than boolean schema but not much). The difference was a few hundred ms. Plain node assert doesn't seem to carry the overhead that joi.assert carries.

As for not using joi.assert in hot path.. Different use cases have different requirements. Who knew that it carried that much overhead.

[Marsup]

I know about slowness of assert and why that is, there might be something actionable there. So things are unclear for me right now, did you simply make the switch to validate or did you convert to json-schema?

[simlu]

@Marsup We ultimately switched to vanilla since joi-to-json had some limitations (which are now resolved btw). However if I had know about the significant difference between assert and valdiate it would have been enough (and much simpler!) to just switch that. /shrug