Expose actual command stdout / stderr on the goss test result

[colearendt]

It would be nice to be able to opt into exposing the stdout that is compared to a regex / etc. in a command resource. This is a super trivial example, but suppose I wanted to test how many files are in a folder, and ensure that the number is greater than 3:

  "NUMAPP=$(expr $(ls -l /apps | wc -l) - 1); if [ $NUMAPP -ge 3 ]; then echo success: $NUMAPP; else echo fail: $NUMAPP; fi":
    title: apps_deployed_gt_3
    exit-status: 0
    stdout: [
      "/^success/"
    ]
    stderr: []

It would be nice if the stdout (i.e. success: 4) and/or stderr could be returned in the lengthy / verbose test output (i.e. JSON, documentation, etc.).

[aelsabbahy]

I have a research item that may indirectly fix this, if my research is successful. I'll create a ticket for the actual implementation details and link this ticket to it.

Leaving this open for now, but not marking it as approved since the solution might be a consequence of another enhancement.

[shan100github]

May be #559 request is also similar ?

[pedroMMM]

Yes, #559 is pretty much the same, I didn't realize that an FR as already placed when I asked @shan100github to create one. Ups 😞

At least we have more attention to this FR now.

I have a research item that may indirectly fix this, if my research is successful. I'll create a ticket for the actual implementation details and link this ticket to it.

Leaving this open for now, but not marking it as approved since the solution might be a consequence of another enhancement.

@aelsabbahy did you made any progress there? If not we could move this discussion forward.

[aelsabbahy]

@pedroMMM I think this PR (when ready) will indirectly fix the issue: #576

With that, doing a string comparison will print out why the strings didn't match (expected vs result).

Only the io.reader I scan mode will not print out the contents.

On a side note, once that's implemented the test may be re-written as:

ls -l /apps | wc -l:
    title: apps_deployed_gt_3
    exit-status: 0
    stdout:
      ge: 3

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[colearendt]

Bump 😄

[aelsabbahy]

@colearendt How does #576 Look to you?

Once that's merged in, the functionality will look as follows:

command:
  echo "hello world":
    exit-status: 0
    stdout:
    - goodbye world

$ goss v
.F

Failures/Skipped:

Command: echo "hello world": stdout:
Expected
    "object: *bytes.Reader"
to contain patterns
    ["goodbye world"]
the missing elements were
    ["goodbye world"]

Total Duration: 0.005s
Count: 2, Failed: 1, Skipped: 0

The PR is still not ready yet, but it's the main thing I'm working on for goss v0.4.0

[aelsabbahy]

Oops, I realized I provided the wrong example in my last comment, lets try this again...

With goss v0.4.0 if you do a string match, the reader will be transformed into a string and compared. For example:

command:
  echo "hello world":
    exit-status: 0
    stdout: "goodbye world"

$ goss v
.F

Failures/Skipped:

Command: echo "hello world": stdout:
Expected
    "hello world\n"
to equal
    "goodbye world"

Total Duration: 0.004s
Count: 2, Failed: 1, Skipped: 0

Sorry for the confusion with the last comment.

[colearendt]

That looks great, thank you!! Very exciting! I have to admit, I was confused by the last one, but chalked it up to me not having read my request back into memory (hence my lack of response) 🙈 This looks great! The only feedback I have is that this line looks a little confusing:

Command: echo "hello world": stdout:

Lots of : and such. Not sure there is any way around that, or if that is the behavior today. Just initial thoughts - I am very excited about the expect vs. equal!!

[aelsabbahy]

There's an exec attribute that can be used for clarity (this exists today in goss doc), I think this is easier on the eyes:

command:
  hello_test:
    exec: echo "hello world"
    exit-status: 0

[jkroepke]

Any news here?

[Oloremo]

:-(