Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFE: Disabling the exit failure when tests failing #57

Closed
ukulekek opened this issue Oct 14, 2019 · 5 comments

Comments

@ukulekek
Copy link

@ukulekek ukulekek commented Oct 14, 2019

When I'm running the rpminspect in a pipeline, the pipeline can't distinguish between failing tests and the failing execution of rpminspect. In both cases, rpminspect returns exit code 1.
I propose to add an additional tag for the threshold option to disable the exit failure(e.g. -t NO) or change the exit code for failing tests.

@thrix

This comment has been minimized.

Copy link

@thrix thrix commented Oct 14, 2019

@ukulekek can you post here an example pls? Feel free to share the NVR, should not be a problem ...

@ukulekek

This comment has been minimized.

Copy link
Author

@ukulekek ukulekek commented Oct 14, 2019

Example of valid execution of failing tests with "BAD" result:

$ rpminspect -o results symlinks-1.4-10.el7
$ echo $?
1

Example of invalid execution:

$ rpminspect not-existing-nvr.el7
*** Unable to find after build: none-existing-nvr.el7
*** Failed to gather specified builds.
$ echo $?
1
@dcantrell

This comment has been minimized.

Copy link
Collaborator

@dcantrell dcantrell commented Oct 14, 2019

I think this is a reasonable idea. For the example given, running rpminspect in Fedora on symlinks-1.4-10.el7 will fail unless you are providing an rpminspect.conf file which directs rpminspect to the internal brew instance so it can look up that build (the '-o results' captures the message, but that's my guess as to what's happening).

Rather than extending the -t option, I think there should be 3 exit codes for rpminspect. Similar to a program like grep. In the case of rpminspect:

0=passes inspection
1=fails inspection
2=program error occurred

Thoughts?

@ukulekek

This comment has been minimized.

Copy link
Author

@ukulekek ukulekek commented Oct 15, 2019

Yes, I like it. Moreover, this is the best option for our infrastructure.

dcantrell added a commit that referenced this issue Oct 15, 2019
Expand beyond EXIT_SUCCESS and EXIT_FAILURE.  Reserve code 0 for
inspection success, or all inspections passed per the result
threshold.  Code 1 means at least one inspection failed with regard to
the result threshold.  And code 2 means rpminspect errored in some way
or you messed up with your input.
@dcantrell

This comment has been minimized.

Copy link
Collaborator

@dcantrell dcantrell commented Oct 15, 2019

Just pushed a change for this. Exit 0 means inspections passed, exit 1 means inspections failed, and exit 2 means a program error occurred.

@dcantrell dcantrell closed this Oct 15, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.