storage: roachtest coverage of v2 encryption

[sumeerbhola]

We would like to switch to v2 encryption which was developed multiple years ago, for efficiency reasons. See this thread as an example https://cockroachlabs.slack.com/archives/CAC6K3SLU/p1767969116561149?thread_ts=1767906320.923949&cid=CAC6K3SLU

But our roachtests don't metamorphically run with v2 encryption. That needs to change. Here is an AI summary.

Roachtest EAR coverage of fileCipherStreamV2

Summary

Almost no roachtests exercise encryption-at-rest with V2 cipher streams. The metamorphic encryption infrastructure generates V1-format keys exclusively.

Key generation for metamorphic encryption uses V1

The main path is generateKeyCmd() in pkg/roachprod/install/cockroach.go:1641, which generates keys via:

openssl rand -out <store_dir>/aes-128.key 48

This produces a 48-byte binary file (32-byte keyID + 16-byte key) — the V1 format. It does not use cockroach gen encryption-key, so the --version flag is irrelevant.

This path is used by all 41+ roachtests that enable EncryptionMetamorphic or EncryptionAlwaysEnabled (backup, kv, tpcc, import, restore, etc.).

Only one roachtest exercises V2

The encryption/rotation/nodes=1 test in pkg/cmd/roachtest/tests/encryption.go explicitly generates both V1 and V2 keys for all sizes (128/192/256) and rotates through them. This is the only roachtest that calls cockroach gen encryption-key --version=2.

Default in gen_encryption.go

In pkg/cli/gen_encryption.go:138:

genEncryptionKeyCmd.PersistentFlags().IntVar(&keyVersionFlag, "version", 1,
    "Encryption format version (1 or 2)")

Even manual cockroach gen encryption-key usage produces V1 keys unless --version=2 is explicitly passed.

V2 unit test coverage exists

pkg/storage/fs/ctr_stream_test.go has unit tests for fileCipherStreamV2, so the implementation is tested at that level. But integration-level roachtest coverage of V2 under realistic workloads (backup, kv, tpcc, etc.) is effectively zero.

Increasing V2 coverage

The simplest path would be to modify generateKeyCmd() in pkg/roachprod/install/cockroach.go to use cockroach gen encryption-key --version=2 instead of openssl rand, or to metamorphically choose between V1 and V2. The current openssl rand approach bypasses cockroach key generation entirely, so it can never produce V2 (JWK-format) keys.

Jira issue: CRDB-61785