You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We compare the current version of an ATD file with past versions (using git difftool -x atddiff).
Each time something is reported, reviewed, but ignored, it continues to be reported by atddiff.
We want to ignore findings that we already reviewed. Our approach is to keep the atddiff output in git. Each new commit on the ATD file produces a report that diffed against the previous version of the report. We want this diff to be meaningful so as to show only the new findings or findings that disappeared.
Here's an example of a diff on the atddiff output:
Incompatibility in both directions:
File "semgrep_output_v1.atd", line 1071, characters 21-35
-File "semgrep_output_v1.atd", line 993, characters 21-28:
+File "semgrep_output_v1.atd", line 957, characters 21-28:
Incompatible kinds of types: option is now a string.
The following types are affected:
project_metadata
The - and + lines exist only because some code was inserted and shifted many of the definitions in the file without changing their contents.
Proposed solution
Support a --no-locations option that suppresses the output of the variable parts (e.g. line 1071, characters 21-35).
If possible (and reasonably easy), output a hash identifying each finding based on the structure of the types being compared.
Producing a stable hash for each pair of types being compared may be a lot of work. An approximate solution may be to identify findings only based on the error message (which may be sufficiently unique in practice) e.g. hash the following text:
Incompatible kinds of types: option is now a string.
The following types are affected:
project_metadata
When multiple findings end up having the same error message and the same identifier, we can report the number of such findings. It is only for diff purposes. For investigating the finding, the user would request the full output with locations anyway.
The text was updated successfully, but these errors were encountered:
CHANGES:
* atddiff: Breaking changes in the JSON output format of atddiff (ahrefs/atd#382)
* atddiff: Fix `atddiff --version` output (ahrefs/atd#379)
* atddiff: New experimental option `--no-locations` aimed at
producing more stable results that allow diffing successive
atddiff reports to spot new findings and ignore old ones (ahrefs/atd#377)
nberth
pushed a commit
to nberth/opam-repository
that referenced
this issue
Jun 18, 2024
CHANGES:
* atddiff: Breaking changes in the JSON output format of atddiff (ahrefs/atd#382)
* atddiff: Fix `atddiff --version` output (ahrefs/atd#379)
* atddiff: New experimental option `--no-locations` aimed at
producing more stable results that allow diffing successive
atddiff reports to spot new findings and ignore old ones (ahrefs/atd#377)
Problem
git difftool -x atddiff
).We want to ignore findings that we already reviewed. Our approach is to keep the atddiff output in git. Each new commit on the ATD file produces a report that diffed against the previous version of the report. We want this diff to be meaningful so as to show only the new findings or findings that disappeared.
Here's an example of a diff on the atddiff output:
The
-
and+
lines exist only because some code was inserted and shifted many of the definitions in the file without changing their contents.Proposed solution
--no-locations
option that suppresses the output of the variable parts (e.g.line 1071, characters 21-35
).Producing a stable hash for each pair of types being compared may be a lot of work. An approximate solution may be to identify findings only based on the error message (which may be sufficiently unique in practice) e.g. hash the following text:
When multiple findings end up having the same error message and the same identifier, we can report the number of such findings. It is only for diff purposes. For investigating the finding, the user would request the full output with locations anyway.
The text was updated successfully, but these errors were encountered: