Oatts throws an error while running mocha command

[Hiral-Dadhaniya]

Hi,

As given in the github documentation(https://github.com/google/oatts), following command generates the .js file for mentioned specification.
oatts generate -s ./path/to/openapi.yaml -w ./output/dir

For next steps, while running them using mocha (mocha --recursive (test directory)), this gives me following error for JS file.
"Error: Timeout of 2000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves."

Is there a way to resolve this? Which rules(validation) does Oatts uses?

Thanks.

[noahdietz]

Hi @Hiral-Dadhaniya thanks for using oatts and opening an issue.

oatts uses a rest testing framework called chakram to execute requests and assert on responses. If you take a look at your generated code, you'll notice a return chakram.wait() inside the callback for every it clause. As described here, this is returning a promise for mocha to wait for.

In this scenario it looks like the test timeout used is 2000ms (the default), and the test is being preempted. A couple of possible scenarios might be that the request is taking longer than the allotted timeout to complete, somehow the server is not reachable but the client's timeout for this is longer than the test's timeout, or there could even be an issue with how mocha is attempting to resolve the promise (i believe this would be out of the scope of this code base and we'd need to dig deeper to prove it).

I'd ask you to please raise the timeout by running the tests with a --timeout flag, and to ensure that the server you are targeting in the tests is reachable from the machine you are running the tests on (server is available on the host/port, firewalls aren't blocking, etc.)

Thank you!

[Hiral-Dadhaniya]

Hi @noahdietz thank you for quick response.
Yeh, looks like timeout issue. It works with --timeout flag. I was under impression oatts may use own rules to validate json/yaml files, but I think as you mentioned it's the chakram for executing requests and doing assert. So, is it possible to have own rules to validate json/yaml or .js(As it creates the JS files).

[noahdietz]

OK glad the timeout flag helped.

If I understand the question, you want to know how oatts validates the JS files it creates? Or the spec that it consumes? Or are you asking about the assertions done on the response as part of the tests that are run? Not sure I am following the question can you please explain in more detail? Thanks.

[Hiral-Dadhaniya]

Yes, Sure.
Basically i am interested in how Oatts validates spec that it consumes. But as it creates JS out of it and validates using chakram, am not sure specific validations of spec file for edit/update supported. (For example: json structure, schema(These two i think chakram itself does?), specific rules like 'parameter objects should have a description' or 'no comma in description' etc. ) Correct me if i am wrong

[noahdietz]

oatts uses a module called sway to consume & traverse the given specification. sway also validates the schema of the given OAS document, ensuring it conforms. However this is not the goal of oatts. oatts is meant to generate a test suite (of JS files) that would test the contract validity between the OAS document and the server that implements the API. It does not validate the schema of the specification specifically as a test.

I am going to close this issue as we have resolved the issue. Thanks!