Do not abort repo scan if a commit fails to scan

[agateau-gg]

The problem

When ggshield secret scan repo fails to scan a commit, it stops. This is annoying because it forces users to restart the full scan.

Proposed solution

The command should note all failures and carry on. At the end it should summarize all failures and exit with an error.

To be decided: in the case of JSON output, should the failure summary be part of the JSON document or should it be sent directly to stderr?

[pierrelalanne]

@agateau-gg : I think we should prioritize this and work on it in the next sprint. Do you agree ?

[agateau-gg]

@agateau-gg : I think we should prioritize this and work on it in the next sprint. Do you agree ?

Sounds good 👍

[dghowlett]

Looking forward to this feature improvement. Ideally the JSON payload should include an error properties that includes details on each commit it failed to scan and the failure. That will allow the caller to programmatically decide what action to take further on those commits including driving that decision off the type of error for each commit, thx!

[agateau-gg]

Thanks for your feedback @dghowlett.

Note however that the failures this issue is talking about are bugs in ggshield preventing it from scanning a commit. They are not incidents: incidents are already included in the JSON output.

This means the failure is likely to be of little interest to users, except for opening an issue or working around a bug. In normal usage, ggshield should not fail to scan a commit.

There is the case of the network issue though, that would cause a failure to scan which would not be a ggshield bug 🤔.

[dghowlett]

Hi @agateau-gg

Correct, they aren't incidents/secret ids that generally map to the failure, but a result of failures from other areas which could include:
Read timed out
Remote end closed connection without response
etc....

Knowing which commit SHA failed to scan and just the associated error msg itself will be helpful. Ultimately though, not having ggshield abort is the key ask here especially when scanning repos with huge histories. Looking forward to this feature coming soon, thx!