Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Assertion with contains and empty $output produces automatic pass #84

Closed
MichaelAquilina opened this issue Aug 28, 2017 · 5 comments
Closed

Comments

@MichaelAquilina
Copy link
Contributor

MichaelAquilina commented Aug 28, 2017

Simplest way to explain this is with a test example:

@test 'This test will pass no matter what' {
  assert "$output" contains "hello"
  assert $state equals 0
}

This also works with any other empty variable being tested - but output is usually the most likely case.

This is likely to do with the way assert has been written so that the assert operator is after the first operand. So zunit treats the the assert as assert contains "hello" since "$output" is empty.

@MichaelAquilina
Copy link
Contributor Author

I suspect that the only proper way to fix this is to change the way assert statements are run (change the ordering of the operator to be directly after the assert function

@molovo
Copy link
Member

molovo commented Aug 30, 2017

If it comes to it we can do that, but I'd rather keep it readable. The double quotes should prevent that from happening, so I'm inclined to think it's a bug

@molovo molovo added the bug label Aug 30, 2017
@MichaelAquilina
Copy link
Contributor Author

Nothing stands out as obvious from the source code 🤔 there are clearly quite a few things I need to learn about shell scripts

@molovo molovo modified the milestone: v0.9.0 Sep 5, 2017
molovo added a commit that referenced this issue Sep 5, 2017
Internally, the `assert` helper does not quote the value to be asserted
against when cascading the call to the relevant assertion function,
resulting in an implicit pass if the value is empty, since the
comparisons are shifted forward a step. Double quoting the vaue when
cascading prevents these false-positives from occurring.

- [ ] Write tests to capture these false-positives

Fix #84
molovo added a commit that referenced this issue Sep 5, 2017
Internally, the `assert` helper does not quote the value to be asserted
against when cascading the call to the relevant assertion function,
resulting in an implicit pass if the value is empty, since the
comparisons are shifted forward a step. Double quoting the vaue when
cascading prevents these false-positives from occurring.

- [ ] Write tests to capture these false-positives

Fix #84
@molovo molovo added this to In Progress in v0.9.0 Sep 13, 2017
@molovo molovo moved this from In Progress to Testing in v0.9.0 Sep 13, 2017
@MichaelAquilina
Copy link
Contributor Author

@molovo any chance we could get the relevant PR merged for this? I'm still affected by this bug right now :(

@molovo
Copy link
Member

molovo commented Jan 4, 2018

@MichaelAquilina whoops, thought I'd already merged it. I'll tag a new release tonight

molovo added a commit that referenced this issue Jan 4, 2018
Internally, the `assert` helper does not quote the value to be asserted
against when cascading the call to the relevant assertion function,
resulting in an implicit pass if the value is empty, since the
comparisons are shifted forward a step. Double quoting the vaue when
cascading prevents these false-positives from occurring.

- [ ] Write tests to capture these false-positives

Fix #84
@molovo molovo modified the milestones: v0.9.0, v0.8.2 Jan 4, 2018
@molovo molovo closed this as completed in c3d0f2f Jan 4, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
v0.9.0
Testing
Development

No branches or pull requests

2 participants