Distinction between raw vs. hosted content

[michaelcfanning]

There is an interesting distinction between a URI that points to the raw content of a file that is referenced in a SARIF result vs. a URI that itself displays/hosts the file. We see this distinction in two different SARIF-driven user experiences we are building at Microsoft.

Within VS, a SARIF result URI of a file hosted on, say, GitHub, should result in a download of the file contents. The IDE itself will then load the file, its relevant language service, and handle region highlighting as usual.

Working with GitHub/VSTS however, the scenario is different. In this case, we want to generate a URL that allows the user to browse to the online-hosted content (provided by GitHub). An elaboration on this scenario is that we need to format the link to include the result region data (so that GitHub displays the source code with appropriate lines highlighted).

One solution here is to say producers must simply make a choice about what viewer experience is being supported (this is a pipeline scenario and so this may be reasonable).

But perhaps there is a way within SARIF where we can distinguish between a URI that retrieves raw file contents vs. a URI that itself has assumed file viewing responsibilities (for click-through scenarios).

[ghost]

TL;DR: Support for file download and online file viewing scenarios is not a format concern; it is a viewer/SDK concern. We propose a design for SDK support for these scenarios, and we will close this issue.

Philosophy

In general, when we have added properties to the format for the sake of a viewer, we have added information that

1. the viewer could not easily know on its own, and
2. differs from result to result or location to location.

For example, we define threadFlowLocation.indentLevel because a viewer can’t know which calls the end user might consider it useful to visualize (for example, a tool might exclude thunks from the display). And for that matter, there might be scenarios where the indentation should be based on something other than call depth. The tool knows this; the viewer doesn’t.

As another example, we add threadFlowLocation.kind because the viewer cannot otherwise know what is happening at this location (at least, not without being able to parse the language of the source code, and even then, context might matter), and so can’t otherwise decide which icon to display.

But consider the scenarios we are discussing here:

1. Downloading a file from a VCS so the user can examine it in the viewer’s UI.
2. Browsing to a file, or to a location within a file, so the user can view it in the VCS’s web UI.

Supporting those scenarios requires the following knowledge:

1. How to transform the information in a versionControlDetails object, together with information in a fileLocation object, into a download URI.
2. How to transform the information in a versionControlDetails object, together with the information in a physicalLocation object (that is, a fileLocation object + an optional region object) into a URI that can be used to browse to the file, or to a location within the file.

That knowledge does not vary from result to result, or even from run to run. For any given online VCS, such as GitHub, that information is well-known, and should exist in only one place; it should not be baked into every SARIF file.

Proposal:

In the SDK, define a configuration file format that describes how to construct the URLs for the file download and online viewing scenarios. It might look like this:

{
  "github": {
    "downloadUriTemplate": " {repoUri}/raw/{revisionId}/{path}",
    "fileViewingUriTemplate": " {repoUri}/blob/{revisionId}/{path}",
    "fileRegionViewingUriTemplate": "{repoUri}/blob/{revisionId}/{path}#L{startLine}-L{endLine}"
  },
  "vsts": {
    ...
  }
}

The SDK provides a sample configuration file for popular online VCSs like GitHub and VSTS; consumers can add their own entries.

NOTE 1: We have two different file viewing templates, one with and one without the region, because a viewer shouldn’t have to figure out how to strip out the region to get the “full file” URI. It might not be a fragment; it might not be all of the fragment.

NOTE 2: In the “template with region”, we recapitulate the entire “full file” URI because again, it might not always be the case the you form the region URI by just appending something to the “full file” URI.

We provide SDK helpers for parsing the configuration file, and for using the information to construct the URLs. Any viewer built on the SDK can use these helpers to implement their file download and online viewing experiences.

For example, the SDK might provide:

• A class OnlineUriTemplates with properties
  • DownloadUriTemplate
  • FileViewingUriTemplate
  • FileRegionViewingUriTemplate
• A class OnlineUriConfiguration as follows:
  • Method to parse the configuration file:
    public static OnlineUriConfiguration ParseFrom(string path);
  • Methods to construct URIs from the information in the configuration:
    • public Uri GetDownloadUri(string vcsKey, VersionControlDetails versionControlDetails, FileLocation fileLocation);
    • public Uri GetFileViewingUri(string vcsKey, VersionControlDetails versionControlDetails, FileLocation fileLocation);
    • public Uri GetFileRegionViewingUri(string vcsKey, VersionControlDetails versionControlDetails, PhysicalLocation physicalLocation);

Sample usage in a viewer:

OnlineUriConfiguration config = OnlineUriConfiguration.ParseFrom("onlineUriTemplates.config");
string vscKey = "github"; // TODO: How does the viewer know which entry to use? Selected in UI?
...
Uri downloadUri = config.GetDownloadUri("github", result.Locations[0].PhysicalLocation.FileLocation);
// Download the file and display it in the viewer.
...
Uri fileViewingUri = config.GetFileRegionViewingUri("github", result.Locations[0].PhysicalLocation);
// Browse to the location in the file.

All file formats, method signatures, and property names subject to change. This is just one way to do it.