added triage_version.py for running a code sample against a given amount of Cppcheck binaries

[firewave]

I have been using this for quite a while now and made adjustments so it will also work with commit hashes so I think this is ready for a review now.

Example output of --compare --compact on my current bisect binary folder with example from https://trac.cppcheck.net/ticket/11250:

'0005be1db' not a version - assuming commit hashes
f45cecc30 (1.87)
0
[input.cpp:7]: (warning) Redundant code: Found a statement that begins with numeric constant.
[input.cpp:8]: (warning) Redundant code: Found a statement that begins with numeric constant.
70e5fa359 (1.87)
0
[input.cpp:7]: (warning) Found suspicious operator ','
[input.cpp:8]: (warning) Found suspicious operator ','
247b2d8c8 (2.4)
0
input.cpp:7:10: warning: Found suspicious operator ',' [constStatement]
    x = 3,4 ; // x becomes 3
         ^
e02182bd8 (2.7)
0
input.cpp:7:10: warning: Found suspicious operator ',', result is not used. [constStatement]
    x = 3,4 ; // x becomes 3
         ^

And doing the same but with a collection of all official Windows release binaries:

1.30
0

1.57
0
[S:\___temp\input.cpp:7]: (warning) Redundant code: Found a statement that begins with numeric constant.
[S:\___temp\input.cpp:8]: (warning) Redundant code: Found a statement that begins with numeric constant.
1.88
0
[S:\___temp\input.cpp:7]: (warning) Found suspicious operator ','
[S:\___temp\input.cpp:8]: (warning) Found suspicious operator ','
1.89
0
S:\___temp\input.cpp:7:10: warning: Found suspicious operator ',' [constStatement]
    x = 3,4 ; // x becomes 3
         ^
S:\___temp\input.cpp:8:11: warning: Found suspicious operator ',' [constStatement]
    y = (3,4) ; // y becomes 4 --> results in out of bounds access
          ^
2.2
0
S:\___temp\input.cpp:7:10: warning: Found suspicious operator ',' [constStatement]
    x = 3,4 ; // x becomes 3
         ^
2.8
0
S:\___temp\input.cpp:7:10: warning: Found suspicious operator ',', result is not used. [constStatement]
    x = 3,4 ; // x becomes 3
         ^

[firewave]

There's still some polishing needed and I need to finish the version handling. Also some documentation is needed.

Integrating it better with the bisect script from #4286 also seems to make sense. As well as somehow integrating it with reduce.py or even daca comes to mind.

Also the name is totally wrong since it won't "diff" anything by default (and doesn't even diff anything yet). compare also seems wrong. Any suggestions are welcome.

[chrchr-github]

Maybe binbisect?

[firewave]

Maybe binbisect?

It does not bisect anything. It just iterated over a list of executables.

That would be a sub-functionality like actual diff. Finding out between which versions it changed and invoking the appropriate bisect - essentially automating what I do manually now.

[firewave]

I decided to name it triage.py which seems to make more sense given the future integration with things like bisect and reduce as well as daca.

I will also look into sharing some of the functionality with the triage GUI tool.

[firewave]

There's also some other tools in the tools (like the times stuff) folder which could probably be merged into this.

[firewave]

I think this is now fine for a first version to commit. So please review now.

I keep this as draft until it is approved since I currently removed all the GitHub actions to avoid unnecessary builds.

[firewave]

@chrchr-github @danmar
This appears to be quite stable now (I had not have to make adjustments for a week now) and I would like to get it merged - so an actual review would be appreciated. The documentation could probably be improved but I think it is mainly self-explanatory.

Also as mentioned before I will keep the GitHub Actions removed until the reviews are done to reduce the turnaround times and avoid unnecessary builds.

[pfultz2]

I decided to name it triage.py which seems to make more sense given the future integration with things like bisect and reduce as well as daca.

It would be nice to rename this to triage_version.py. As it would be nice to have a triage.py script in the future to handle the daca@home debugging as the GUI is not easy to use especially over remote machines. It would be nice to have a script to download the source files, run cppcheck, and open relevant files using $EDITOR.

[firewave]

It would be nice to rename this to triage_version.py. As it would be nice to have a triage.py script in the future to handle the daca@home debugging as the GUI is not easy to use especially over remote machines. It would be nice to have a script to download the source files, run cppcheck, and open relevant files using $EDITOR.

As mentioned in my comments above that's what I am planning to in the future. Integrating all of those parts/functionality together - even the GUI.

[firewave]

It would be nice to rename this to triage_version.py.

I am gonna rename this so we can get it pushed. After all it's just a name and we can still change it afterwards.