Expose specific reason for build failure

[eugeneshih]

Prior to V2, I used to use "$BITRISE_UNITTEST_STATUS" which would tell me whether unit tests passed or failed. It doesn't appear to be available anymore. When a build fails, I want to know which step caused the build to fail.

I used that status in curl-ping@2.0.0 when I report about successful / failing builds.

[viktorbenei]

hi @eugeneshih - wouldn't this be better solved by providing a generic solution, in bitrise (https://github.com/bitrise-io/bitrise). Something like $BITRISE_FIRST_FAILED_STEP_TITLE - to get the title of the first failed step, and maybe a combined $BITRISE_FAILED_STEP_TITLES which would be a list of the failed step titles (separated by a ; character or something similar).

What do you think?

[eugeneshih]

That would be fine. 👍

[viktorbenei]

@eugeneshih could you please describe what exactly would you like to get - for what you would like to use the information? This would help us to finalize the specification for the feature.

[eugeneshih]

Currently, my curl step takes the value of $BITRISE_UNITTEST_STATUS and makes a post to a special HTTP server. It includes the result of $BITRISE_UNITTEST_STATUS as a parameter, for example:

-u [destination] -d "subject=$BITRISE_APP_TITLE/$BITRISE_GIT_BRANCH" -d "unit tests: **$BITRISE_UNITTEST_STATUS**"

Both suggestions you made seem useful, but the first would be most useful for me.

[viktorbenei]

What would you like to get @eugeneshih - the Title of the failed step, or rather the "success" / "error" state/text of a specific step? Keep in mind that in general if a Step before the Test step fails the Test step would not even performed (unless you mark it as is_always_run), so you might get other states than just "success" and "error" (namely: "skipped").

It would help us if you could share what exactly do you want to use the environment for - do you want to just send a message to somewhere with the title of the failed step? Or you need the state of a specific step?

[eugeneshih]

I was pretty happy with the old variable. So, in answer to your questions, I would most like to know the "success" / "failure" state of the unit test step. This status is simply sent via a curl POST call as I show above.

I don't know if I can be any clearer.

If you want to provide a more general reporting mechanism which includes the title of the failed step, that's be great, but I don't need that. If you want to provide a dictionary of steps mapped to their success / failure state, that would be great too. Clearly, those would be more useful, but I don't need those features at this point.

[viktorbenei]

Thanks for your feedback, it really helps us to figure out a generic solution, so that the individual steps don't have to implement a mechanism like this!

One last question: would you like a textual status ("success"/"failure") or a binary one (1/0)?

[eugeneshih]

Textual is better for now.

[viktorbenei]

@eugeneshih this will be available in the next version of the step, but it'll require a new bitrise CLI version as well if you would like to run it on your own Mac, as the current version does not expose step outputs in case the step fails. This'll be fixed in a new patch release of bitrise, and the VMs will be patched as well asap.

We'll release the new version of this step once everything else is released & deployed.

[viktorbenei]

@eugeneshih the new version is now available, once again thank you for your feedback! https://github.com/bitrise-io/steps-xcode-test/releases/tag/1.3.0