Command stdout/stderr don't appear test empty case correctly

[bryanlatten]

I've made the following passing gossfile, using two symlinked commands to the same binary. I'm trying to test that stderr remains empty during a command run.

command:
  php -v:
    exit-status: 0
    stderr: ["PHP Warning"]
  php7 -v:
    exit-status: 0
    stderr: ['']

Running it produces:

bash-4.3# goss goss.yaml validate
....

Total Duration: 0.030s
Count: 4, Failed: 0, Skipped: 0

I've made a failing condition, where the binary is currently reporting a warning over stderr, seen here:

bash-4.3# php -v 1>/dev/null
PHP Warning:  PHP Startup: Invalid library ...
bash-4.3# php7 -v 1>/dev/null
PHP Warning:  PHP Startup: Invalid library ...

But for some reason, goss is treating [''] as a passing string, at the exact same time, the actual string is encountered in stderr is also passing as a pattern. I've also tried [], [/^$/] (which doesn't seem to match on a good case), and others.

Any advice here? Am I doing something wrong?

[aelsabbahy]

Hmm.. you're not doing anything wrong, it's mostly an oversight on my side. I don't think there's a simple/elegant way to handle this currently at least none that come to mind.

The way the stdout/stderr check works is by checking if any line contains the provided pattern/string.
So your example above:

command:
  php7 -v:
    exit-status: 0
    stderr: ['']

Is checking if any stderr line contains "", which will always return true.

One solution might be to inverse the check. Basically check that no line contains any content:

command:
  php7 -v:
    exit-status: 0
    stderr: ['!/./']

I know this is probably not the most intuitive solution, but it might be the only one with the current limitation of processing files line by line.

In the future, I might add a special case for handling this scenario, or maybe provide a way for the user to run tests against the whole output as a string, rather than line by line.

Does this work for you for the time being? Also, if you have thoughts on your ideal syntax for this, I would love to hear it.

[bryanlatten]

@aelsabbahy, thanks for the reply! Should work just-fine. I wasn't as familiar with the gomega matchers, this makes total sense though. Not sure if there's any magic bullet on the syntax (esp between json + yaml), but maybe a quick note in the docs until we think of something.

[aelsabbahy]

Just to be clear, the stdout/stderr predate the gomega features, they support "patterns" but not the "Advanced Matchers" of gomega. All other checks support gomega though.

See: https://github.com/aelsabbahy/goss/blob/master/docs/manual.md#patterns

Feel free to submit a PR or suggest verbiage changes to that section. Apologies for any confusion this has caused you.

[aelsabbahy]

Feel free to re-open if you're still having this issue.