-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
any.label() significantly slows validation of large arrays #874
Comments
Can you use proper benchmarking tools to do that ? (eg. bench) |
'use strict';
const Benchmark = require('benchmark');
const Joi = require('joi');
const users = getUsers();
const schema = Joi.array().items(
Joi.object().keys({
username: Joi.string().required()
})
);
const schemaWithInnerLabel = Joi.array().items(
Joi.object().keys({
username: Joi.string().required()
}).label('User')
);
const schemaWithOuterLabel = Joi.array().items(
Joi.object().keys({
username: Joi.string().required()
})
).label('Users');
new Benchmark.Suite()
.add('no label', () => {
Joi.validate(users, schema);
})
.add('inner label', () => {
Joi.validate(users, schemaWithInnerLabel);
})
.add('outer label', () => {
Joi.validate(users, schemaWithOuterLabel);
})
.on('cycle', (event) => {
console.log(String(event.target));
})
.run();
///
function getUsers() {
const users = [];
for (var i=0; i<100; i++) {
users.push({username: 'example'});
}
return users;
} no label x 745 ops/sec ±2.24% (81 runs sampled)
inner label x 185 ops/sec ±2.23% (79 runs sampled)
outer label x 247 ops/sec ±2.26% (79 runs sampled) I believe the cause of the speed difference is in lib/object.js#L150 the // Children mustn't inherit the current label if it exists
const childOptions = options.language && options.language.label ?
Hoek.applyToDefaults(options, { language: { label: null } }, true) :
options; I'm not familiar enough with the code, but it seems strange to me that this must run for every element in the array when validating an array. |
Was just working on the same thing. Built a little github repo and used some more complex examples to try and see where the load is https://github.com/glennjones/benchmarking-joi-swagger. It does like adding any property |
I can pretty much guarantee meta won"t slow anything down in the validation process. The childOptions might be computed too many times but it's the only sane way to deal with the propagation of options. 1st step would be to try to optimize this removal of label with a simpler version of it, we probably don't need a full clone (I didn't expect it to be so consuming). If anyone feels like working on it... |
I have another idea (that involves breaking more stuff) that I'm going to try but don't let it stop you ;) |
Optimizing the line I referenced in |
My alternative strategy improves it a bit but still a 40% loss over baseline, I'm not yet explaining this. That's for another day if you don't mind. |
Context
What are you trying to achieve or the steps to reproduce?
Use
any.label()
on a schema within an array, or on the array schema directly.We use
Joi
to validate our API responses, so it's common that we are validating large arrays.Which result you had?
A significant impact on validation performance:
What did you expect?
No noticeable impact on performance.
The text was updated successfully, but these errors were encountered: