Add ability to silence error reports via configuration comments

[Drvi]

Hi!
We're currently adding JET.report_package to our test suite to ensure we didn't introduce any detectable errors to our codebase.
But sometimes we encounter errors we're okay with, e.g.:

JET.report_text("""
    f(::Float64) = error("We got a Float64, this is bad, and here is why: ...")
    """,
    analyze_from_definitions=true,
)
#═════ 1 possible error found ═════
#┌ @ top-level:1 error("We got a Float64, this is bad, and here is why: ...")
#│┌ @ error.jl:33 error(::String)
#││ may throw: Base.throw(Base.ErrorException(s::String)::ErrorException)
#│└───────────────

Currently, I'm manually filtering the reports contained in the result object, which feels a bit hacky, as I'm touching the internals of VirtualFrames which I'm not super familiar with.

Generally, having more control over what appears in the reports would be great. Either an explicit API for filtering results beyond target_modules or, preferably, the ability to annotate code with comments which would tell JET to silence certain error reports for a given method/code segment.

I don't have a good idea of what that syntax should look like, but maybe something like this could work:

JET.report_text("""
    function f(::Float64) # JET ignore: UncaughtExceptionReport  
        error("We got a Float64, this is bad, and here is why: ...")
    end
    """,
    analyze_from_definitions=true,
)
# No errors detected

Ideally, one could condition the config on the method signature.

CC: @NHDaly

[NHDaly]

Just wanted to share an example of how this is configured via code comments in ESLint:
https://eslint.org/docs/user-guide/configuring/rules

[maxfreu]

Hi! @Drvi, can you share the code you currently use to filter out the errors?

[orenbenkiki]

Since it doesn't seem likely this will be fixed any time soon, I filter JET's output through the following. It not only skips errors on lines containing NOJET, but also breaks the output to a list of separate errors, with a line break between them, which IMVHO is more readable. Exit status is non-zero if there are any non-skipped errors.

import fileinput
import re
import sys

location_pattern = re.compile(r'(\S+):(\d+)$')

read_path = None
read_lines = None
bad_paths = set()

def is_disabled(path, line):
    if path in bad_paths:
        return False

    global read_path
    global read_lines

    if path != read_path:
        try:
            with open(path) as file:
                read_lines = list(file.readlines())
            read_path = path
        except:
            bad_paths.add(path)
            return False

    return "NOJET" in read_lines[int(line) - 1]

context_lines = []
context_disabled = []
context_changed = False

errors = 0
skipped = 0

for line in fileinput.input():
    if line.startswith("[toplevel-info]"):
        print(line[:-1])
        sys.stdout.flush()
        continue

    if line == "\n" or "[toplevel-info]" in line or "possible errors" in line:
        continue

    match = location_pattern.search(line)
    if match:
        context_changed = True
        depth = len(line.split(' ')[0])

        while len(context_lines) >= depth:
            context_lines.pop()
            context_disabled.pop()

        context_lines.append(line)
        context_disabled.append(is_disabled(*match.groups()))
        continue

    if any(context_disabled):
        if context_changed:
            skipped += 1
            context_changed = False
    else:
        if context_changed:
            context_changed = False
            errors += 1
            print("")
            for context_line in context_lines:
                print(context_line[:-1])
        print(line[:-1])

if errors > 0:
    if skipped > 0:
        print(f"\nJET: {errors} errors found, {skipped} errors skipped")
    else:
        print(f"\nJET: {errors} errors found")
    sys.exit(1)

if skipped > 0:
    print(f"\nJET: {skipped} errors skipped")
else:
    print(f"\nJET: clean!")