Add encryption option using keyed-hash of existing digests

[tasket]

Proposal

For nonce generation, implement Hk( rnd || pthash ) as a higher-performance alternative to Hk( rnd || m ).

A subkey would be dedicated for this purpose, as in the alpha3 implementation of xchacha20-t3.

Rationale

Wyng already generates 256-bit cryptographic hashes of plaintext (meta)data for various uses such as deduplication. So, to attain both security and speed, it may be advantageous to use existing hashes to generate nonces in a reuse-resistant method that is similar to the libsodium recommendation used in #159.

HMAC itself demonstrates that hashing a hash with an additional factor (a key) can be robustly secure for generating nonces, as libsodium recommends HMAC for this purpose.

Further, it may be that libsodium did not recommend this method because it would be odd to assume an application would already have a supply of suitable hashes available. Most applications would only begin making cryptographic hashes of data when adding encryption, so Wyng is unusual in this respect.

Safeguards

• It may be necessary to disable encryption modes incorporating pthash when the user selects an unsuitable Wyng hash mode such as SHA256, or vice-versa.

• Using the same hash function as the pthash source may be best, i.e. BLAKE2b.

Other considerations

Having a deterministic mode (see #157) available may be desirable to some users, in which case another alternative could be to generate nonces similar to the above method but without the random prefix, for example Hk( pthash ).

---

CC: @DemiMarie

[DemiMarie]

This is fine with me!

[tasket]

Borrowing the test table from 159, I have a new benchmark to add...

Runtime (sec)	Wyng Encryption Mode	Nonce Method
68.7	xchacha20	XChaCha20 concat counter
83.8	xchacha20-t1	XChaCha20 BLAKE2b Hk(rnd||m)
68.0	xchacha20-t2	XChaCha20 rnd nonce
76.3	xchacha20-t3	XChaCha20 HMAC-SHA256 Hk(rnd||m)*
68.5	n/a	XChaCha20 BLAKE2b Hk(rnd||pthash)
60.0	n/a (-t4?)	XChaCha20 HMAC-SHA256 Hk(rnd||pthash) where pthash is also from HMAC-SHA256

Generating the manifest hashes with HMAC-SHA256 yields a 12.5% speed boost over the fastest BLAKE2b result (these figures do not include actually sending the data, just reading & encoding).

The question is whether HMAC can be securely used this way. Reading up on HMAC's properties, it seems like it would OK as long as the hashing key was kept secret. Also, some thought should be given to properly deriving the key.

[tasket]

This mode has been implemented as 'xchacha20-t4'. The resulting archive will be configured to use the -t4 handler which uses manifest hashes for data nonces, but uses the -t3 handler for metadata since the metadata is not speed-critical and -t3 allows simpler metadata code.

Further, the data hashing algorithm is now specifically using HMAC-SHA256 based on a secret subkey, and this applies across all the testing modes that are still user-selectable. As a result, if you created older -t2 or -t3 archives their data will no longer be recognized as valid! You are urged to create new -t2 and -t3 archives in that case.

(If you created archives under the old default xchacha20, they will still be handled the same way and data recognized as valid. A counter mode is still select-able as xchacha20-tc which now uses HMAC for data hashing.)

Switching the hash functions to HMAC has made send significantly faster for all select-able modes. Some current benchmark-mode times (12GB volume data):

Runtime (sec)	Wyng Encryption Mode	Nonce Method
59.3	xchacha20-tc	XChaCha20 counter nonce
58.4	xchacha20-t2	XChaCha20 rnd nonce
66.1	xchacha20-t3	XChaCha20 HMAC-SHA256 Hk(rnd||m)*
58.6	xchacha20-t4	XChaCha20 HMAC-SHA256 Hk(rnd||pthash)

This is a great pay-off for relatively few engineering changes that stay within safe, recommended norms. I don't see a need at this point to try any additional new modes.

[tasket]

An extra precaution was added to make the subkeys more robust. Their size has been increased to 512 bits and they now have independent 512 bit salts (slots 2 and 3) and separate contexts and separate iterations within those contexts. The subkey hash algorithm has also been changed to SHA-512. Primary keys (slots 0 and 1) now use 512 bit salts as well. This is being done to ensure ample key differentiation and strength.

This change also means if you were using any of the -tX encryption modes, those archives won't work with Wyng version 20230528 onward ...again. (Sorry about that!)

[tasket]

The encryption mode selection names have been decided. Here is a quick rundown of the changes:

Deprecated:

xchacha20 — This is the original counter mode, which is replaced by slightly better counter mode below. This mode will continue to function for existing archives.

Removed:

xchacha20-t1 and xchacha20-t2 — No longer present in code.

Accepted:

xchacha20-t3, xchacha20-t4 and xchacha20-tc — These modes have been renamed xchacha20-msr, xchacha20-dgr and xchacha20-ct, respectively. The last option is the name going forward for the protected counter mode.

Also, the default encryption setting will be xchacha20-dgr, which uses the 'Hk || rnd' amalgam for generating nonces under the data key and the simpler-to-implement but slower 'm || rnd' for metadata.