Consider: 'review' or 'audit' result level. and reconsider 'note'

[michaelcfanning]

The SARIF 'result' level doesn't provide for reporting on potentially extremely verbose/noisy issues that may still be relevant in circumstances when developers are motivated to provide a substantive audit or review.

It is helpful to separate these from 'note' which is currently an extremely ambiguous result type, that might flag a candidate pattern for change (or might represent any information at all).

So, a second change is to tighten up the meaning of 'note' or perhaps switch to 'suggestion' or 'information'. We could clarify that in all cases, resultLevel is intended to categorize a potential issue/candidate for code change. Simple observations about code, other logging, should be expressed as tool notifications, instead.

[ghost]

Per @michaelcfanning's comment in #280: We can also consider the question of a numeric severity property in this issue (#215).

[michaelcfanning]

See as well #59. A result could also flag that it is a valid result that occurs in dead code. I believe this is the remaining PolySpace designation not already covered by the format (since we added 'open' as an explicit result level.

[ghost]

@michaelcfanning We added result.kind value "review" as part of #317, "Consider splitting resultLevel into result.level and result.kind". I'd forgotten it was a separate issue.

We still have the "note" question to consider.

[michaelcfanning]

Nope, we considered this issue in the f2f and determined that that addition of 'review' (and now the addition of 'debug') obviates the need to reconsider 'note'. 'note' stands as defined, it is the least severe failure that may be returned by analysis (but it does, in fact, constitute a failure.

To get really picky, the concept we have stopped discussing is 'observation', that is, a format report of some kind that is perfectly neutral/factual, i.e., neither pass nor fail.

I am closing this. Please review. If you feel like observation is important, let's file a new issue.

[ghost]

I do not feel strongly enough about this to argue for adding "observation". It's non-breaking if we add it later.