PVF: Re-check file integrity before voting against; document

[mrcnski]

Before voting against a candidate, re-check the file integrity.

Document it: add an # Artifact integrity section to artifacts.rs. It could explain that artifacts must remain valid to satisfy the SAFETY constraints of execute_artifact. Therefore we do two things: check file integrity and version compatibility on host startup, and also before voting against a candidate we re-check the file integrity.

Originally posted by @mrcnski in #1918 (comment)

[Jpserrat]

@mrcnski Is this issue about re-checking the file integrity and adding the # Artifact integrity?
Why the re-checking is needed? Do we not vote against it in case of failure?

[mrcnski]

@mrcnski Is this issue about re-checking the file integrity and adding the # Artifact integrity?

Yes, exactly! We still need to do both, re-checking was not implemented yet.

Why the re-checking is needed? Do we not vote against it in case of failure?

We had the idea to re-check the file hash of the artifact before execution, to make sure it wasn't corrupted. However, this would add a few ms to each execution. As an optimization, we can only re-check after execution, if the execution failed. Reason being, that corruption should be very very rare, and if it happens then the execution will fail, so we only need to check it there. Hope that makes sense. Thanks for asking, for sure it's not obvious so should be documented. 👍

[mrcnski]

If we re-check and it's corrupted, we should return a specific error that indicates that the host should purge this artifact from the artifacts cache.

[eagr]

I'm about to take a shot at the integrity re-check. Wondering if you have started on this? @Jpserrat

[Jpserrat]

@eagr I haven't, you can take it

[eskimor]

Do we not vote against it in case of failure?

We do and this is precisely the problem. If the file got corrupted on disk, we would currently vote invalid on the candidate and raise a dispute, despite the issue being our hard disk and not the candidate. We would get slashed and disabled for raising a dispute on a valid candidate. Obviously there should be a big fat warning in the logs if a corruption is detected.