Encrypted storage tools for PersonalMediaVault

This library implements a collection of tools to create an encrypted storage, for the PersonalMediaVault project:

• Functions to encrypt and decrypt, using AES-256, with the option to compress the data using ZLIB.
• Read and write streams to create and read encrypted files in chunks.
• Read and write streams to pack multiple small encrypted files into a single container file.

Documentation

Installation

In order to add it to your project, use

cargo add pmv_encryption_rs

File encryption

You can encrypt a buffer of data using encrypt, with a key of 64 bytes.

You can then decrypt it using decrypt and the same key.

use pmv_encryption_rs::{encrypt, decrypt, EncryptionMethod};

fn example() {
    let key: &[u8] = &[0x01; 32]; // Mock key

    let data = "Hello world!".as_bytes();

    let encrypted = encrypt(data, EncryptionMethod::Aes256Flat, key).unwrap();

    let decrypted = decrypt(&encrypted, key).unwrap();

    assert_eq!(
        decrypted.as_slice(),
        data
    );
}

Details

The encrypted data returned by encrypt_file_contents and accepted by decrypt_file_contents is binary-encoded, with the following structure:

Starting byte	Size (bytes)	Value name	Description
0	2	Algorithm ID	Identifier of the algorithm, stored as a Big Endian unsigned integer
2	H	Header	Header containing any parameters required by the encryption algorithm. The size depends on the algorithm used.
2 + H	N	Body	Body containing the raw encrypted data. The size depends on the initial unencrypted data and algorithm used.

The system is flexible enough to allow multiple encryption algorithms. Currently, there are 2 supported ones:

• AES256_ZIP: ID = 1, Uses ZLIB (RFC 1950) to compress the data, and then uses AES with a key of 256 bits to encrypt the data, CBC as the mode of operation and an IV of 128 bits. This algorithm uses a header of 20 bytes, containing the following fields:

Starting byte	Size (bytes)	Value name	Description
0	4	Compressed plaintext size	Size of the compressed plaintext, in bytes, used to remove padding
4	16	IV	Initialization vector for AES_256_CBC algorithm

• AES256_FLAT: ID = 2, Uses AES with a key of 256 bits to encrypt the data, CBC as the mode of operation and an IV of 128 bits. This algorithm uses a header of 20 bytes, containing the following fields:

Starting byte	Size (bytes)	Value name	Description
0	4	Plaintext size	Size of the plaintext, in bytes, used to remove padding
4	16	IV	Initialization vector for AES_256_CBC algorithm

Block-Encrypted Files

Block encrypted files are used to encrypt an arbitrarily large file, splitting it's contents in blocks (or chunks) with a set max size. Each block is then encrypted using the file encryption method detailed above.

For creating / writing files:

• You can create a file using FileBlockEncryptWriteStream::new, a function that returns a new instance of FileBlockEncryptWriteStream. It creates and initializes the file.
• Once it is created, you may call FileBlockEncryptWriteStream::write to write data into the file. When the data reached a block limit, that block is encrypted and stored into the file.
• After you wrote all the data, you must call FileBlockEncryptWriteStream::close to write any pending data and close the file.

For reading files:

• You can open a file calling FileBlockEncryptReadStream::new, a function that returns an instance of FileBlockEncryptReadStream, opening the file for reading.
• After it's opened, you may call FileBlockEncryptReadStream::get_file_size, FileBlockEncryptReadStream::get_block_size or FileBlockEncryptReadStream::get_block_count to retrieve the parameters of the file.
• You may call FileBlockEncryptReadStream::read to decrypt and read the data.
• You can call FileBlockEncryptReadStream::seek to change the cursor position. You may also call FileBlockEncryptReadStream::get_cursor_position to retrieve the cursor position if needed.
• After you are done, you may call FileBlockEncryptReadStream::close to close the file. It is also closed on drop.

use pmv_encryption_rs::{FileBlockEncryptWriteStream};

use rand::{RngCore, SeedableRng, rngs::SmallRng};
use tempdir::TempDir;

const MOCK_DATA_SIZE: usize = 1 * 1024 * 1024;
const MOCK_BLOCK_SIZE: u64 = 128 * 1024;

const MOCK_DATA_READ_SIZE: usize = 300 * 1024;
const MOCK_DATA_READ_OFFSET: usize = 500 * 1024;

fn example() {
    // Mock key
    let key: &[u8] = &[0x01; 32];

    // Mock data
    let mut mock_data: Vec<u8> = vec![0; MOCK_DATA_SIZE];
    let mut rng = SmallRng::from_rng(&mut rand::rng());
    rng.fill_bytes(&mut mock_data);
    let file_size = mock_data.len() as u64;

    // Mock file
    let dir = TempDir::new("test_pmv_encryption_rs").unwrap();
    let file_path = dir.path().join("s_1.pma");

    // Write the data

    let mut write_stream =
        FileBlockEncryptWriteStream::new(&file_path, key.to_vec(), file_size, MOCK_BLOCK_SIZE)
            .unwrap();

    let block_count = write_stream.get_block_count();

    write_stream.write(&mock_data).unwrap();
    write_stream.close();

    // Read the data

    let mut read_stream = FileBlockEncryptReadStream::new(&file_path, key.to_vec()).unwrap();

    assert_eq!(read_stream.get_file_size(), MOCK_DATA_SIZE as u64);
    assert_eq!(read_stream.get_block_size(), MOCK_BLOCK_SIZE);
    assert_eq!(read_stream.get_block_count(), block_count);

    let mut mock_data_read_buf: Vec<u8> = vec![0; MOCK_DATA_READ_SIZE];

    let seek_pos = read_stream
        .seek(SeekFrom::Start(MOCK_DATA_READ_OFFSET as u64))
        .unwrap();

    assert_eq!(seek_pos, MOCK_DATA_READ_OFFSET as u64);
    assert_eq!(seek_pos, read_stream.get_cursor());

    let read_size = read_stream.read(&mut mock_data_read_buf).unwrap();

    assert_eq!(read_size, MOCK_DATA_READ_SIZE);

    assert!(mock_data_read_buf.iter().eq(mock_data.iter().skip(MOCK_DATA_READ_OFFSET).take(MOCK_DATA_READ_SIZE)));

    read_stream.close();
}

Details

They are binary files consisting of 3 contiguous sections: The header, the chunk index and the encrypted chunks.

The header contains the following fields:

Starting byte	Size (bytes)	Value name	Description
0	8	File size	Size of the original file, in bytes, stored as a Big Endian unsigned integer
8	8	Chunk size limit	Max size of a chunk, in bytes, stored as a Big Endian unsigned integer

After the header, the chunk index is stored. For each chunk the file was split into, the chunk index will store a metadata entry, withe the following fields:

Starting byte	Size (bytes)	Value name	Description
0	8	Chunk pointer	Starting byte of the chunk, stored as a Big Endian unsigned integer
8	8	Chunk size	Size of the chunk, in bytes, stored as a Big Endian unsigned integer

After the chunk index, the encrypted chunks are stored following the same structure described above.

This chunked structure allows to randomly access any point in the file as a low cost, since you don't need to decrypt the entire file, only the corresponding chunks.

Multi-File Pack

Multi-file pack container files are used to store multiple small files inside a single container.

For creating / writing files:

• You can create a file by calling MultiFilePackWriteStream::new, a function that returns an instance of MultiFilePackWriteStream. It creates and initializes the file.
• You may call MultiFilePackWriteStream::put_file for each file you want to store, in order.
• After all files are written, you must call MultiFilePackWriteStream::close to white any pending data and close the file.

For reading files:

• You can open a file by calling MultiFilePackReadStream::new, a function that returns an instance of MultiFilePackReadStream, opening the file for reading.
• You may call MultiFilePackReadStream::get_file_count to retrieve the number of stored files.
• You may call MultiFilePackReadStream::get_file to read a file, by its index.
• After you are done, you may call MultiFilePackReadStream::close to close the file. It is also closed on drop.

use pmv_encryption_rs::{MultiFilePackWriteStream};

use rand::{RngCore, SeedableRng, rngs::SmallRng};
use tempdir::TempDir;

fn example() {
    // Mock files
    let mut rng = SmallRng::from_rng(&mut rand::rng());

    let mut mock_file_1: Vec<u8> = vec![0; 100 * 1024];
    rng.fill_bytes(&mut mock_file_1);

    let mut mock_file_2: Vec<u8> = vec![0; 200 * 1024];
    rng.fill_bytes(&mut mock_file_2);

    let mut mock_file_3: Vec<u8> = vec![0; 300 * 1024];
    rng.fill_bytes(&mut mock_file_3);

    // Temp file to store the packed files
    let dir = TempDir::new("test_pmv_encryption_rs").unwrap();
    let file_path = dir.path().join("m_1.pma");

    // Write the files in order

    let mut write_stream = MultiFilePackWriteStream::new(&file_path, 3).unwrap();

    write_stream.write_file(&mock_file_1).unwrap();
    write_stream.write_file(&mock_file_2).unwrap();
    write_stream.write_file(&mock_file_3).unwrap();
    write_stream.close();

    // Read the files (any order)

    let mut read_stream = MultiFilePackReadStream::new(&file_path).unwrap();

    assert_eq!(read_stream.get_file_count(), 3);

    let mock_file_2_r = read_stream.get_file(1).unwrap();
    assert!(mock_file_2.iter().eq(mock_file_2_r.iter()));

    let mock_file_1_r = read_stream.get_file(0).unwrap();
    assert!(mock_file_1.iter().eq(mock_file_1_r.iter()));

    let mock_file_3_r = read_stream.get_file(2).unwrap();
    assert!(mock_file_3.iter().eq(mock_file_3_r.iter()));

    assert!(matches!(read_stream.get_file(3), Err(MultiFilePackReadError::IndexOutOfBounds)));

    read_stream.close();
}

Details

They are binary files consisting of 3 contiguous sections: The header, the file table and the encrypted files.

The header contains the following fields:

Starting byte	Size (bytes)	Value name	Description
0	8	File count	Number of files stored by the asset, stored as a Big Endian unsigned integer

After the header, a file table is stored. For each file stored by the asset, a metadata entry is stored, with the following fields:

Starting byte	Size (bytes)	Value name	Description
0	8	File data pointer	Starting byte of the file encrypted data, stored as a Big Endian unsigned integer
8	8	File size	Size of the encrypted file, in bytes, stored as a Big Endian unsigned integer

After the file table, each file is stored following the same structure described above.