Represent "triage status"

[ghost]

Do we need a field to represent an issue's triage status, e.g., "fixed", "false positive"?

One argument goes that tools would never emit this field, and we're defining a tool output format, so no. Another argument goes that the life cycle of an issue extends beyond the moment it is emitted by the tool, and we'd like to use the same format to represent the issue throughout its lifecycle, so yes.

If the second argument wins, we need to extend the introduction to say that we want the format to represent an issue throughout its lifecycle.

[zlandau]

If the format is meant to represent the lifecycle then there are definitely more things that come into play, like representing an instance of an issue (an issue as it exists in a specific version of a file) versus the issue itself, which could be carried across multiple versions of the code until it is fixed. Something like triage status would apply to the issue rather than the instance.

[ghost]

Thanks for the valuable feedback, Zachary.

We have thought some more about whether SARIF is a tool output format or whether it can represent an issue throughout its lifecycle. We think the answer is "both", but to make that work -- to make it clear which elements of SARIF are used for each of those purposes -- we are going to divide SARIF into a core format plus a collection of domain-specific profiles. The core format is a tool output format. The issue management profile defines additional information that defines the issue's life cycle status. Please see Issue #29 for more details on this.

I will address your other very good point -- that a triage status is attached to an issue "as a whole" rather than to a particular instance or occurrence of the issue -- in a separate comment, tomorrow, but now I have to run.

[ghost]

Re: issues vs. instances:

Yes, we have dealt with this problem in a previous system I worked on. Appendix A talks about how an issue management system can track an issue from run to run by computing a "fingerprint" for it (and how a static analysis tool can contribute to that fingerprint, if it wishes, by providing a value for the toolFingerprint property of the issue object).

In such an issue management system, the triage status -- along with other information, such as developer comments, which we refer to among ourselves as "issue metadata" -- is indeed associated with the fingerprint rather than with individual issue instances. In our implementation, the Issues database table had a column Fingerprint. Fingerprint was used as a key into a second table, IssueMetadata, which in turn had columns such as TriageStatus and Comments. So all issues with the same fingerprint shared the same triage status, which was only stored once.

So yes, conceptually, the triage status belongs to the issue "as a whole" rather than to a particular instance of it. The question now becomes, do we need to represent that in the serialization format?

To do so, we might imagine that we query our issue management system for some issues, and it returns something like this. This format would augment the core format with properties from the "issue management profile":

{
    "issues":
    [
        {
            "ruleId": "CA2100",
            "fingerprint": "0123abcd"
        }
    ],

    "issueMetadata":
    [
        {
            "fingerprint": "0123abcd",
            "triageStatus": "FalsePositive"
        }
    ]
}

This would directly represent the concept we're discussing.

However, in our system, we chose not to do that, and instead our issue management system emitted this:

{
    "issues":
    [
        {
            "ruleId": "CA2100",
            "fingerprint": "0123abcd",
            "triageStatus": "FalsePositive"
        }
    ]
}

Our reasoning was that this was easier for a client of the issue management system -- such as a web application, or an IDE such as Visual Studio, which would typically display the issues in a list box -- to consume the format if the issue metadata was "denormalized" and repeated in each issue (really, "issue instance") that referred to it. Otherwise, each consumer of the format would be required to perform the "join" in order to flatten the issues + metadata into a list.

That is what I would prefer to capture in the SARIF format. In the issue management profile, I would add properties fingerprint and triageStatus to the issue object, rather than introducing a new object such as issueMetadata to hold that information.

Please let me know your thoughts on this.

[zlandau]

We are also doing something similar with emitting fingerprints at the analysis level and using tooling further down the chain to determine the triage status. So even if the core format had the field we wouldn't be able to populate it until further down the chain. The addition of profiles makes sense for us.

Re: splitting issues and issueMetadata, I tend to agree that having the issue data updated with the extra information rather than putting it into a separate field makes sense. While some tools may care about the distinction between the issue and the occurrence, most will not and as you say, keeping them together will be easier. If any tools really do care about the distinction they can conceptually split them by 1) knowing which fields belong to an issue versus an occurrence and 2) looking at the history of an issue by using the fingerprint across multiple runs.

[ghost]

Yes, precisely! Thanks Zachary, glad we are on the same page on this. I will leave this issue open until I've reorganized the spec into "core + profiles", and have added triageStatus to the Issue Management Profile.

[michaelcfanning]

All this looks like great progress to me.

[codecurmudgeon]

I agree, that's an item I was going to ask about myself. There are multiple
states an issue might be in, like "fix later" or "never gonna fix it" etc.

On Tue, Sep 1, 2015 at 11:13 AM, Zachary P. Landau <notifications@github.com

wrote:

If the format is meant to represent the lifecycle then there are
definitely more things that come into play, like representing an instance
of an issue (an issue as it exists in a specific version of a file) versus
the issue itself, which could be carried across multiple versions of the
code until it is fixed. Something like triage status would apply to the
issue rather than the instance.

—
Reply to this email directly or view it on GitHub
#22 (comment)
.

Regards,

Arthur "Code Curmudgeon" Hicken
Office: 626-275-2445 (Los Angeles)
Cell: 909-343-0936
Google voice: 978-49-OWASP (in theory rings my mobile and my desk... in
theory)

[michaelcfanning]

This belongs in the result mgmt profile when we fill it out