Skip to content
This repository has been archived by the owner on Mar 26, 2021. It is now read-only.

Schema validation fails silently #97

Closed
finestructure opened this issue Nov 9, 2018 · 5 comments
Closed

Schema validation fails silently #97

finestructure opened this issue Nov 9, 2018 · 5 comments
Labels
bug Something isn't working
Projects

Comments

@finestructure
Copy link

finestructure commented Nov 9, 2018

Describe the bug
I've written a request that fails schema validation but the strest run does not fail.

To Reproduce

  • create a request yml that is invalid
  • run it
  • the run passes

Expected behavior

  • the run should fail with a non-zero exit code

Additional context

[ Strest ] Found 3 test file(s)
[ Strest ] Schema validation: 2 of 3 file(s) passed

Executing tests in ./
Executing tests in: /app/tests/
Executing tests in: /app/tests/tokens/
✔ Testing userToken succeeded (0.573s)

[ Strest ] ✨  Done in 0.634s

The only indication that something went wrong is 2 of 3 file(s) passed at the top, which is easy to miss, especially in longer scripts.

@eykrehbein eykrehbein added the bug Something isn't working label Nov 9, 2018
@jgroom33
Copy link
Collaborator

jgroom33 commented Nov 9, 2018

Agreed. It should also show details of the failure.

@finestructure
Copy link
Author

I've also just come across this one here, which is very similar and doesn't error out (although it's more obvious):

✔ Testing create_contract_1 succeeded (0s)

Status: undefined
Status Text: undefined

Headers:

undefined

Data:

undefined

✔ Testing create_contract_2 succeeded (0s)

I'm referencing a Value(...) that doesn't exist because I didn't run all the necessary tests in the batch.

@jgroom33
Copy link
Collaborator

jgroom33 commented Nov 9, 2018

That should probably be a new issue. Add an example yaml that is expected to pass but doesn't

@finestructure
Copy link
Author

Ok, I've added #99

And thanks for your swift reply. I'd take a look into this myself but JS is not my forte :)

@jgroom33
Copy link
Collaborator

Check out
CONTRIBUTING.md
This is the line that does the validation:
https://github.com/eykrehbein/strest/blob/master/src/yaml-parse.ts#L40

@jgroom33 jgroom33 mentioned this issue Nov 12, 2018
3 tasks
@jgroom33 jgroom33 added this to Done in Release 2.0 Nov 13, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Something isn't working
Projects
No open projects
Release 2.0
  
Done
Development

No branches or pull requests

3 participants