Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Maximum call stack size exceeded #179

Closed
distortedsignal opened this issue Jul 23, 2020 · 8 comments
Closed

Maximum call stack size exceeded #179

distortedsignal opened this issue Jul 23, 2020 · 8 comments

Comments

@distortedsignal
Copy link
Contributor

My team is changing over to Open API 3.0 documentation from RAML, and I'm trying to build up a linter from this tool. Our API is (fairly) complex and our data structures are deeply nested. I recently ran into an issue where one of our *.json files couldn't be linted because of a limit on the size of the callstack.

Local information

Installed via npm with the -g flag.

In the directory that I'm linting:

> lint-openapi -v
0.28.0

The call that makes the error happen, and output:

> # Source filename on following line changed to protect the guilty.
> lint-openapi --debug openapi.json

[Error] There is a problem with the Swagger.
Maximum call stack size exceeded

RangeError: Maximum call stack size exceeded
    at RegExp.exec (<anonymous>)
    at Url.parse (url.js:257:31)
    at urlParse (url.js:150:13)
    at Object.urlResolve [as resolve] (url.js:660:10)
    at dereference$Ref (/usr/local/lib/node_modules/ibm-openapi-validator/node_modules/json-schema-ref-parser/lib/dereference.js:99:22)
    at /usr/local/lib/node_modules/ibm-openapi-validator/node_modules/json-schema-ref-parser/lib/dereference.js:59:26
    at Array.forEach (<anonymous>)
    at crawl (/usr/local/lib/node_modules/ibm-openapi-validator/node_modules/json-schema-ref-parser/lib/dereference.js:52:24)
    at dereference$Ref (/usr/local/lib/node_modules/ibm-openapi-validator/node_modules/json-schema-ref-parser/lib/dereference.js:113:24)
    at /usr/local/lib/node_modules/ibm-openapi-validator/node_modules/json-schema-ref-parser/lib/dereference.js:59:26

I'm not totally sure how to debug this - do you have any advice? I'm more than happy to post my .validaterc file, but since the source is not open, I'm a little hesitant to provide the source files that are being used here.

@mkistler
Copy link
Contributor

It looks like the error is in performing a regex on a URL, probably from a $ref (since the bottom of the stack is json-schema-ref-parser. So my guess is that you have a $ref somewhere in your doc that causes the regex engine to run off the rails. On that theory, I'd start by just examining all the $refs in the doc to see if any looked "strange". Or perhaps you can trap at the point of the failure to see what the value is that regex is failing on.

@jorge-ibm
Copy link
Contributor

hey @distortedsignal - are you still having issues with this?

@distortedsignal
Copy link
Contributor Author

Yeah - here's a recent run of the openapi linter to prove it:

docs > lint-openapi root_openapi_files_*.json

...

    Validation Results for root_openapi_files_one.json:

[Error] There is a problem with the Swagger.
Maximum call stack size exceeded


    Validation Results for root_openapi_files_two.json:

[Error] There was a problem with a validator.
Cannot read property 'type' of null

docs > lint-openapi --version
0.30.1

I think there is an infinite recursion in the openapi docs (something like "resource1 contains a resource1"), but I would prefer to have the linter tell me where it is, as opposed to having to hunt through the docs by hand. I think that's probably how this issue should be read?

@jorge-ibm
Copy link
Contributor

jorge-ibm commented Oct 7, 2020

thanks for clarifying @distortedsignal. Can you maybe provide the api definition files you encountered this issue in?

Edit: Or if you're hesitant to provide the api definition files, either way, we'll take a look at this in the near future just wanted to see if this was still a problem.

@distortedsignal
Copy link
Contributor Author

They're in a proprietary codebase - let me ask my boss if I can ship the full files. If I can't, I'll try to provide a redacted version of the files.

@dpopp07
Copy link
Member

dpopp07 commented Oct 7, 2020

@distortedsignal There is code in the linter to recognize circular references, report them, and continue without crashing. Perhaps you have a special case that breaks that code but I am guessing that this is something else. Any API definition you provide for us to reproduce the problem, even if heavily reduced or redacted, would be appreciated. If you need to send something to one of our emails, rather than posting on a public issue, that would be fine too.

@distortedsignal
Copy link
Contributor Author

Thanks @dpopp07 - I've contacted people who can make a decision on this and I'll get back to you when I have their decision.

@distortedsignal
Copy link
Contributor Author

Ok, I got permission from my boss to ship this code if all the fields were md5'ed. So here we go.

output_dir.tar.gz

My procedure here:

Unzip the tar'ed up dir:

/tmp > tar zxvf output_dir.tar.gz
x test_dir/a83fcef9ace0d75bab9602394f365f26.json
x test_dir/bad_second_half.json
/tmp > ls
...      output_dir.tar.gz            test_dir ...
/tmp > ls test_dir
a83fcef9ace0d75bab9602394f365f26.json bad_second_half.json

cd into the newly created test_dir and look around.

/tmp > cd test_dir
/tmp/test_dir > ls -laht
total 16
drwxr-xr-x   4 tom   wheel   128B Oct  8 13:07 .
drwxrwxrwt  15 root  wheel   480B Oct  8 13:07 ..
-rw-r--r--   1 tom   wheel   1.6K Oct  8 13:04 bad_second_half.json
-rw-r--r--   1 tom   wheel   1.5K Oct  8 13:04 a83fcef9ace0d75bab9602394f365f26.json

init a new openapi environment

/tmp/test_dir > lint-openapi init

[Success] '.validaterc' file created and set to defaults.

Run the linter.

/tmp/test_dir > lint-openapi bad_second_half.json

[Error] There is a problem with the Swagger.
Maximum call stack size exceeded

Interestingly, we don't hit the recursion limit if I just lint the a83fcef9ace0d75bab9602394f365f26.json file.

/tmp/test_dir > lint-openapi a83fcef9ace0d75bab9602394f365f26.json

errors

  Message :   `info` object must have a string-type `title` field
  Path    :   info.title
  Line    :   4
...

I tried to make this as minimal and readable as possible. If you would like clarification on anything, please feel free to ask.

I'm guessing there was a recursive call that was checked if everything was in one file but not if things were spread across files?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants