[SPARK-51988][SS] Do file checksum verification on read for RocksDB zip file

[gnanda]

What changes were proposed in this pull request?

When the RocksDB state store downloads a checkpoint zip file from DFS, it previously opened the file via fs.open() (a raw Hadoop FileSystem handle), bypassing the CheckpointFileManager abstraction. This meant that even when file checksum
verification was enabled (spark.sql.streaming.stateStore.rocksdb.fileChecksumEnabled), the zip file itself was never verified against its .crc sidecar on read.

This PR fixes that by:

1. Extracting a new Utils.unzipFilesFromInputStream(inputStream, localDir) helper that accepts any InputStream, and making the existing unzipFilesFromFile delegate to it.
2. Changing RocksDBFileManager to open checkpoint zip files via fm.open(path) instead of fs.open(path). When checksum verification is enabled, fm is a ChecksumCheckpointFileManager, whose open() returns a stream that verifies the file's size
   and CRC32C checksum on close(). If the checksum does not match, a CHECKPOINT_FILE_CHECKSUM_VERIFICATION_FAILED error is raised.

Backward compatibility is preserved: if no .crc sidecar exists (e.g. the checkpoint was written before this feature was enabled), the file is opened and read without verification.

Why are the changes needed?

Silent data corruption in checkpoint zip files would previously go undetected. A corrupted or partially-written zip could cause a state store to load incorrect data, leading to wrong query results or obscure failures. Verifying the checksum on
read closes this gap and makes corruption visible immediately at load time, with a clear error condition.

Does this PR introduce any user-facing change?

Yes. When spark.sql.streaming.stateStore.rocksdb.fileChecksumEnabled is true (the default), loading a RocksDB checkpoint zip file whose .crc sidecar does not match will now throw a SparkException with error condition
CHECKPOINT_FILE_CHECKSUM_VERIFICATION_FAILED instead of silently succeeding with corrupt data.

How was this patch tested?

• RocksDBSuite: Three new unit tests covering (1) successful load with checksum verification, (2) graceful fallback when no .crc sidecar exists, and (3) detection of a corrupted .crc sidecar raising
  CHECKPOINT_FILE_CHECKSUM_VERIFICATION_FAILED.
• UtilsSuite: New unit test for unzipFilesFromInputStream verifying correct extraction from an in-memory zip.
• RocksDBCheckpointFailureInjectionSuite: Updated zombie-write test to split the checkpointIds enabled/disabled branches; the disabled branch now explicitly opts out of checksum verification to isolate the intended failure-injection behavior
  from the new checksum enforcement.
• StateStoreSuite: Fixed the checksum file verification test to write two independent runs (checksums-on then checksums-off) rather than overwriting the same versions, keeping existing .crc sidecars valid across the full test.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude Code 2.1.58

[ericm-db]

LGTM.