Maximum call stack size exceeded

[distortedsignal]

My team is changing over to Open API 3.0 documentation from RAML, and I'm trying to build up a linter from this tool. Our API is (fairly) complex and our data structures are deeply nested. I recently ran into an issue where one of our *.json files couldn't be linted because of a limit on the size of the callstack.

Local information

Installed via npm with the -g flag.

In the directory that I'm linting:

> lint-openapi -v
0.28.0

The call that makes the error happen, and output:

> # Source filename on following line changed to protect the guilty.
> lint-openapi --debug openapi.json

[Error] There is a problem with the Swagger.
Maximum call stack size exceeded

RangeError: Maximum call stack size exceeded
    at RegExp.exec (<anonymous>)
    at Url.parse (url.js:257:31)
    at urlParse (url.js:150:13)
    at Object.urlResolve [as resolve] (url.js:660:10)
    at dereference$Ref (/usr/local/lib/node_modules/ibm-openapi-validator/node_modules/json-schema-ref-parser/lib/dereference.js:99:22)
    at /usr/local/lib/node_modules/ibm-openapi-validator/node_modules/json-schema-ref-parser/lib/dereference.js:59:26
    at Array.forEach (<anonymous>)
    at crawl (/usr/local/lib/node_modules/ibm-openapi-validator/node_modules/json-schema-ref-parser/lib/dereference.js:52:24)
    at dereference$Ref (/usr/local/lib/node_modules/ibm-openapi-validator/node_modules/json-schema-ref-parser/lib/dereference.js:113:24)
    at /usr/local/lib/node_modules/ibm-openapi-validator/node_modules/json-schema-ref-parser/lib/dereference.js:59:26

I'm not totally sure how to debug this - do you have any advice? I'm more than happy to post my .validaterc file, but since the source is not open, I'm a little hesitant to provide the source files that are being used here.

[mkistler]

It looks like the error is in performing a regex on a URL, probably from a $ref (since the bottom of the stack is json-schema-ref-parser. So my guess is that you have a $ref somewhere in your doc that causes the regex engine to run off the rails. On that theory, I'd start by just examining all the $refs in the doc to see if any looked "strange". Or perhaps you can trap at the point of the failure to see what the value is that regex is failing on.

[jorge-ibm]

hey @distortedsignal - are you still having issues with this?

[distortedsignal]

Yeah - here's a recent run of the openapi linter to prove it:

docs > lint-openapi root_openapi_files_*.json

...

    Validation Results for root_openapi_files_one.json:

[Error] There is a problem with the Swagger.
Maximum call stack size exceeded


    Validation Results for root_openapi_files_two.json:

[Error] There was a problem with a validator.
Cannot read property 'type' of null

docs > lint-openapi --version
0.30.1

I think there is an infinite recursion in the openapi docs (something like "resource1 contains a resource1"), but I would prefer to have the linter tell me where it is, as opposed to having to hunt through the docs by hand. I think that's probably how this issue should be read?

[jorge-ibm]

thanks for clarifying @distortedsignal. Can you maybe provide the api definition files you encountered this issue in?

Edit: Or if you're hesitant to provide the api definition files, either way, we'll take a look at this in the near future just wanted to see if this was still a problem.

[distortedsignal]

They're in a proprietary codebase - let me ask my boss if I can ship the full files. If I can't, I'll try to provide a redacted version of the files.

[dpopp07]

@distortedsignal There is code in the linter to recognize circular references, report them, and continue without crashing. Perhaps you have a special case that breaks that code but I am guessing that this is something else. Any API definition you provide for us to reproduce the problem, even if heavily reduced or redacted, would be appreciated. If you need to send something to one of our emails, rather than posting on a public issue, that would be fine too.

[distortedsignal]

Thanks @dpopp07 - I've contacted people who can make a decision on this and I'll get back to you when I have their decision.

[distortedsignal]

Ok, I got permission from my boss to ship this code if all the fields were md5'ed. So here we go.

output_dir.tar.gz

My procedure here:

Unzip the tar'ed up dir:

/tmp > tar zxvf output_dir.tar.gz
x test_dir/a83fcef9ace0d75bab9602394f365f26.json
x test_dir/bad_second_half.json
/tmp > ls
...      output_dir.tar.gz            test_dir ...
/tmp > ls test_dir
a83fcef9ace0d75bab9602394f365f26.json bad_second_half.json

cd into the newly created test_dir and look around.

/tmp > cd test_dir
/tmp/test_dir > ls -laht
total 16
drwxr-xr-x   4 tom   wheel   128B Oct  8 13:07 .
drwxrwxrwt  15 root  wheel   480B Oct  8 13:07 ..
-rw-r--r--   1 tom   wheel   1.6K Oct  8 13:04 bad_second_half.json
-rw-r--r--   1 tom   wheel   1.5K Oct  8 13:04 a83fcef9ace0d75bab9602394f365f26.json

init a new openapi environment

/tmp/test_dir > lint-openapi init

[Success] '.validaterc' file created and set to defaults.

Run the linter.

/tmp/test_dir > lint-openapi bad_second_half.json

[Error] There is a problem with the Swagger.
Maximum call stack size exceeded

Interestingly, we don't hit the recursion limit if I just lint the a83fcef9ace0d75bab9602394f365f26.json file.

/tmp/test_dir > lint-openapi a83fcef9ace0d75bab9602394f365f26.json

errors

  Message :   `info` object must have a string-type `title` field
  Path    :   info.title
  Line    :   4
...

I tried to make this as minimal and readable as possible. If you would like clarification on anything, please feel free to ask.

I'm guessing there was a recursive call that was checked if everything was in one file but not if things were spread across files?