DRAFT feat: implement client-side remote state encryption (#297)

[StephanHCB]

This is not intended for merging as is, but is for the discussions on #297

Resolves parts of #297 (but does not meet all requirements)

Target Release

NOT APPLICABLE

Draft CHANGELOG entry

Add client side encryption of the entire state for all remote state storage backends except the enhanced backends.

ENHANCEMENTS

Summary:

Add client side encryption of the entire state for all remote state storage backends except the enhanced backends.

What gets sent to the remote state storage just looks like this:

{"crypted":"e93e3e7ad343405525...dda4fc061"}

The idea is that even the company that operates the remote state storage cannot read it. Of course, one should still configure all other protection mechanisms on the remote storage, this is just one layer of security, but I think it's a very important one.

Features:

• encryption algorithms are pluggable, I have implemented AES256 in CFB stream mode with a fresh random initialization vector generated each time and SHA256 hashing prior to encryption to check for payload integrity
• supports key rotation by giving an optional second key to try for decryption
• supports encryption of existing non-encrypted state
• supports decryption of existing encrypted state
• has unit tests that demonstrate how it works, as well as near complete code coverage
• includes documentation as an experimental feature

The change to existing code is minimal, see internal/states/remote/states.go, I have basically inserted the encryption/decryption at the point at which Client.Get / Client.Put are invoked. As the result of the encryption is again json, this should hopefully work with all Clients with zero changes.

Successfully tested with the azure backend ONLY, but there this code has seen more than a year of production use.

Limitations

I have marked this feature experimental in the documentation, I do not have the resources to test it with all remote backends.

I assume it will not work with enhanced backends because these need access to the state.

[github-actions]

Reminder for the PR assignee: If this is a user-visible change, please update the changelog as part of the PR.

[github-actions]

Please link the relevant issue that this PR handles using one of the following words

• close | closes | closed | fix | fixes | fixed | resolve | resolves | resolved if the Pull Request resolves the issue - more details can be found here
• relates to | related to | part of if the Pull Request is just part of the solution

[cam72cam]

Closing as Stephan has helped us build a variant of this feature.