Windows: Maximum call stack size exceeded when number of items in anyOf is more than 1700

[anirudh-modi]

I have a schema where one of the field km04 is having 1700+ options, whenever I am trying to validate the object send from client, with 1700+ options for the km04 field, it is specifically throwing an error in windows for Maximum call stack, this error is not replicable in mac, or even in windows if we reduce the number of options available for the km04 field below 1700.

here is the sample schema,
schema.txt

The version of Ajv you are using
6.12.3

The environment you have the problem with
Windows

Your code (please make it as small as possible to reproduce the issue)
ajv.validate(schema, body);

If your issue is in the browser, please list the other packages loaded in the page in the order they are loaded. Please check if the issue gets resolved (or results change) if you move Ajv bundle closer to the top
It is not in browser

Results in node.js v8+
Maximum call stack size exceeded

Results and error messages in your platform

Maximum call stack size exceeded

\node_modules\\ajv\\lib\\compile\\index.js:120:26)
   at Ajv.compile (\node_modules\\ajv\\lib\\compile\\index.js:55:13)
   at Ajv._compile (\node_modules\\ajv\\lib\\ajv.js:348:27)
   at Ajv.validate (\node_modules\\ajv\\lib\\ajv.js:96:36)

[epoberezkin]

It’s the duplicate of #1581, there are several options suggested there - e.g. using allErrors option might work.

[epoberezkin]

Not closing yet, because it might be a bit different

[anirudh-modi]

Let me give the same a try and see if these are working appropriately or not..

Thank you for the suggestion

[epoberezkin]

It’s a fundamental call stack depth restriction given which code ajv has to generate without allErrors mode. I am quite reluctant to make compilation asynchronous to be honest…

One more idea is to simply increase the call stack size in nodejs (not 100% sure how many calls it makes per schema level as it compiles, but probably less than 10, so 10-20k should be enough) - please let me know if you try it what stack size makes it compile without any other changes.

[anirudh-modi]

I will be a bit skeptical of increasing stack, as the schema is dynamic for us, it changes across clients for us, where I am afraid if the number of options increases 2000+ we might hit the same block..

What issues do you see with the async compilation?

[epoberezkin]

I will be a bit skeptical of increasing stack, as the schema is dynamic for us, it changes across clients for us, where I am afraid if the number of options increases 2000+ we might hit the same block..

Worth trying - this limit is set arbitrarily low at 1000 I think and it has nothing to do with memory constraints, it’s been like that since the time when 1Gb was A LOT of memory, so at first I’d try finding out how much you need, and I don’t see why it’s bad to have it set at say 100k-1m if needed.

I wouldn’t do it in production code (to avoid locking the process in case of accidental unbounded recursion - that’s the main reason it’s low), but if compilation is at build time it should be ok.

What issues do you see with the async compilation?

1. Incidental complexity.
2. Effort to implement. Why do it at all if large stack size solves the problem.
3. Performance - asynchronous call dispatch will slow compilation down by some factor I think - but even if it’s 20%, 1 and 2 are good reasons not to.

[epoberezkin]

Re dynamic schemas - you can (and probably should) make compilation a queued task and save compiled code to avoid compiling it every time you need to use it.

[anirudh-modi]

Hi @epoberezkin my apologies for the delay, but as per your suggestion I tried these.

Enabling allErrors

const ajv = new Ajv({
  logger: false,
  allErrors: true
});
  
const isValid = ajv.validate(schema, {
 km01: '01'
});
console.log("Data is valid : ", isValid);

Which is still throwing an max call stack error in windows as compared to mac.

Increasing node stack-size

When I ran the above code by increasing the node stack size using the command
node --stack-size=65000 index.js

Weirdly, It is not throwing the error but for some reason it is not even returning anything. This is irrespective of whether I use allErrors or not, and silently exits.

Node version being used

12.8.3

[anirudh-modi]

Hi @epoberezkin Do you have any other suggestion which you think I can give a try?

[epoberezkin]

The problem is that even in allErrors mode, oneOf short-cirtcuits validation when 2 schemas succeed (it's a failure), and replacing it with anyOf, even though makes an execution more efficient (there is very rarely a reason to use oneOf, anyOf is almost always equivalent and much more efficient), it short-circuits on success.

This short-circuiting behaviour is achieved with nested ifs, irrespective of allErrors, makes the code tree deep, and requires as deep stack as many anyof/oneOf branches you have.

In your particular case, you could use enum and not many const's (and there is an option you might need to limit enum size that is compiled to expression - https://ajv.js.org/options.html#loopenum - without it it might require large stack too).

Ultimately it's the limitation that needs to worked around by modifying your schema, if increasing stack size does not work - by reducing the number of anyOf/oneOf branches, replacing multiple consts with enum, breaking schema into multiple files or in some other way.

[anirudh-modi]

Hi @epoberezkin Looks like converting oneOf/anyOf to enum did the trick for me...

Correct me if I am wrong, from the documentation it seems like oneOf and enum work in the same way right?

And I should not see any issue when I swap oneOf with enum right?

{
    type: string,
    oneOf: [{
        const: '01',
        value: 'Team'
    }, {
        const: '02',
        value: 'Model'
    }]
}

{
    type: string,
    enum: ['01', '02']
}

[epoberezkin]

yes, semantically they are equivalent in this case (value is noop, it's just metadata that can be provided some other way, e.g. as a map), but the generated code is very different (and enum is way more efficient).

[wiekern]

Hello, will be this bug fixed? In many cases, enum type is insufficient to display elaborate infos.

[arathore4]

Hello, Any update on fix for this bug?

[strooooke]

Stuck here as well. If only enumNames were still a thing (but seems like they aren't).