Write snapshots to a temporary file until the block they represent is final

[wanderingbort]

Change Description

This resolves #6984

When a request for a snapshot is received, nodeos will now write the snapshot data to a temporary file and create an internal tracking record. As this block becomes final OR finality passes it on a competing fork, nodeos will either rename the temporary file to the expected snapshot name OR delete the temporary file and return an error indicating that the snapshot was unusable.

Temporary files are written to the same directory as final snapshots with a different prefix. Notably this prefix contains a leading . so, on some OS's it they will be hidden.

This change will hold the RPC request (and its associated HTTP session) open until the temporary snapshot is either finalized or removed. Any subsequent requests for the same snapshot will be attached to the future response and no longer return an error code stating that the snapshot already exists.

This should eliminate situations where operators would rarely get snapshots that were invalid due to microforks and also prevent situations where the block.log and the snapshot were not immediately usable once the snapshot was written.

NOTES

Tracking of temporary snapshots is ephemeral, meaning that if the nodeos process is terminated while snapshots are outstanding it may leave the temporary files on disk and will not clean them up or promote them upon the next invocation. If this becomes a pain point, in the future we can consider durable ways of maintaining the tracking so that future invocations can resume the management of promoting snapshots produced from earlier invocations.

Consensus Changes

• Consensus Changes

API Changes

• API Changes

/v1/producer/create_snapshot

This endpoint used to return an error code if you made redundant requests for a snapshot before the head block had changed. Now, all requests for a snapshot at that head block will remain open pending the determination of that blocks finality. At that time, all pending requests for that snapshot's creation will receive the same affirmative or error response.

Documentation Additions

• Documentation Additions

[wanderingbort]

The latest commit allows the snapshot to be written to a temp file, promoted to a pending file and then finalized.

This means at rest you can determine the difference between an incomplete snapshot, a pending and potentially valid snapshot, and a certainly valid snapshot.