The --sarif switch will swallow rule errors.

[squaresurf]

Describe the bug
Given that you have a broken rule, an error will be output when running semgrep. That is unless you add the --sarif switch.

To Reproduce
I created a SSCCE at https://gitlab.com/dsearles/semgrep-sarif-bug

Expected behavior
An error should be output when rules are broken no matter which output options are passed to semgrep.

What is the priority of the bug to you?
Is this a P0 (blocking your adoption of Semgrep or workflow), P1 (important to fix or quite annoying), P2 (regular bug that should get fixed)?

P1

Environment
If not using semgrep.dev: are you running off docker, an official binary, a local build?

The link to the example above uses docker in the example, but I was able to also reproduce it locally with the semgrep binary installed with homebrew.

[brendongo]

@mschwager this is probably because we only print errors when output format is text. https://github.com/returntocorp/semgrep/blob/develop/semgrep/semgrep/output.py#L440 in --json we append the error to the json output not sure if sarif has a similar place for errors. FWIW running with --sarif returns non-zero exit code.

[mschwager]

@mschwager this is probably because we only print errors when output format is text. https://github.com/returntocorp/semgrep/blob/develop/semgrep/semgrep/output.py#L440 in --json we append the error to the json output not sure if sarif has a similar place for errors. FWIW running with --sarif returns non-zero exit code.

I think adding errors to the structured output would be a good solution, similar to JSON. The best SARIF location I can find is the toolExecutionNotifications or toolConfigurationNotifications property. Does that seem reasonable?