Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support SARIF output #483

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Support SARIF output #483

wants to merge 1 commit into from

Conversation

mbg
Copy link

@mbg mbg commented Dec 22, 2022

Hi there!

Thank you for building this library and a happy holidays! This PR aims to resolve #467 and add support for outputting Stan's observations as SARIF files. To support this, I wrote a new library, sarif, which implements the relevant types to represent SARIF structures and to serialise/deserialise them to/from JSON.

The approach taken by my implementation is as follows:

  • I have added a dependency on the sarif library
  • There is a new command-line option, --sarif, which instructs Stan to produce output in the SARIF format
  • A new module, Stan.SARIF, contains functions to convert from Stan's representations to those used for SARIF
  • Once converted, the serialisation is handled by the sarif library

What's this good for?

SARIF files can be understood by other tools, such as GitHub Code Scanning. For example, I used the changes made in this PR to run Stan on itself and upload the results to Code Scanning, which will then show a list of all observations:

Screenshot 2022-12-22 at 14 47 39

These can be viewed in detail:

Screenshot 2022-12-22 at 14 47 53

How can I do this?

To reproduce this, simply run stan --sarif > stan.sarif and use the upload-sarif action to upload the file as part of a GitHub Actions workflow.

Questions / Points of note

  • The sarif library depends on aeson while stan depends on microaeson. If sarif is added as a dependency to stan, then aeson becomes a transitive dependency. Therefore, any benefits with respect to fewer dependencies obtained from using microaeson are lost. The different libraries in use here also lead to some minor complications in the integration between the two projects (e.g. with respect to writing custom properties, such as tags).
  • The --sarif option currently "overrides" the --json-output option (i.e. if both are specified, the output is formatted as SARIF, which sort of makes sense because SARIF is also JSON). I see a few options here:
    • Replace the --json-output option entirely in favour of SARIF output
    • Implement a check that both options are not active at the same time
    • Refactor the command-line options so that there are different sub-commands
    • ... something else entirely?

I'd appreciate a review of this and your thoughts on all of the above! Thanks!

@mbg mbg requested a review from vrom911 as a code owner December 22, 2022 17:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Supporting SARIF output
1 participant