-
Notifications
You must be signed in to change notification settings - Fork 38
feat(policy): policy eval should fail if no execution path is met #2341
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(policy): policy eval should fail if no execution path is met #2341
Conversation
Signed-off-by: Sylwester Piskozub <sylwesterpiskozub@gmail.com>
app/cli/cmd/policy_develop_eval.go
Outdated
|
||
// Check if any result was ignored | ||
for _, res := range result { | ||
if res.Ignored { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
but does this also include if it was ignored on purpose? I mean, would ignore mean only that no script was evaluated?
cc/ @jiparis
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
also do you think we could be more user friendly by including the kind, and the available kinds in the error output for example?
I'd like to make this tool super user friendly that would mean in general\
- removing technical boilerplate improve lint error message #2332 and error wrapping in general
- making errors self-explanatory
- showing useful if necessary
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ignored means: The script was evaluated, but it must be ignored because it doesn't apply, so it will count as it if wasn't evaluated at all.
Note that this is different to skipped
, where the evaluation still counts.
Ignored is for some cases where an input file has multiple purposes (CycloneDX for SBOMs and also for vulnerabilities report, for example)
So in this case, what we want to check whether result
is empty.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated to return error only when no eval results were returned. Included material kind in the error msg. Available kinds require some more work I can create another issue to address that.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can you please write the description of the PR with some examples of outputs so we can understand better what it solves? Thanks!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated the description
Signed-off-by: Sylwester Piskozub <sylwesterpiskozub@gmail.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
This PR adds error when no evaluations on provided material were performed.
Example
For policy with no defined path for kind
SBOM_CYCLONEDX_JSON
when validated against valid cyclonedx json file it outputs:Closes #2334