Skip to content
This repository has been archived by the owner on Oct 30, 2018. It is now read-only.

Latest commit

 

History

History
114 lines (86 loc) · 4.73 KB

file-encryption.md

File metadata and controls

114 lines (86 loc) · 4.73 KB
Raw

This document serves to provide a detailed account of how files are encrypted by default by Storj Core to promote interoperability between different implementations of clients.

Sharding and Encryption Scheme

Before data is stored in the network, the Storj Core library automatically handles file encryption, sharding, and key management for you.

In order of operations:

  1. Complete file is encrypted with a unique key and initialization vector
  2. File is demultiplexed (or "sharded") into individual chunks
  3. Each shard is offered to the network and transferred
  4. Key and IV are encrypted with a passphrase and stored locally

Key and Initialization Vector Generation

Files are encrypted using AES-256-CTR. Use of CTR allows for random read-access to known indices. File keys are generated via one of two methods:

  • Local PBKDF2 key derivation (default)
  • Passphrase-based deterministic key derivation (recommended)
PBKDF2 Key Derivation

By default, the key for each file is the result of a standard key derivation function, PBKDF2. PBKDF2 generates a key from a password and a salt. These two values are randomly generated bytes:

  • Password: 512 random bytes
  • Salt: 32 random bytes

The key length of the PBKDF2 is 512 bytes, should use 25000 iterations, and use SHA-512 as the digest. To create the cipher/decipher and initialization vector, derive the SHA-256 hash of the resulting PBKDF2 for the cipher/decipher key and use the first 16 bytes of the original salt as the iv. The {@link DataCipherKeyIv} class represents these values, and provides methods for their generation.

The {@link KeyRing} class stores the original password and salt locally in an encrypted JSON document keyed by the ObjectId returned from Storj Bridge for the file object. The cleartext JSON document has two properties: pass and salt.

{
  "pass": "c7c311ee213d10baefd620a004d76485190d82...",
  "salt": "6d33490c999e9d613ccf4b146446763df15de2..."
}

This JSON string is encrypted with AES-256-CBC using a user-defined passphrase, and encoded as Base58. Each of those encrypted JSON documents is stored in a directory called key.ring/, where the file name is the ObjectId returned from Storj Bridge for the file object.

Portable Key Ring Format

Because the keys in the key ring are generated and stored locally, files encrypted with these keys cannot be accessed by other machines without first duplicating the key files onto that machine. To mitigate this, Storj Core can create a simple portable key ring archive.

A {@link KeyRing} created by Storj Core can be exported into a portable format, which is simply a gzipped tape archive (.tar.gz or .tgz). Importing this archive simply entails:

  1. Decompress and unpack the archive
  2. Decrypt each JSON document using the original passphrase
  3. Encrypt each document with the passphrase of the target keyring
  4. Move the files into the target keyring, optionally overwriting conflicts
Passphrase-based Key Derivation

Rather than relying on locally-generated keys, it is recommended that the user generate a high-entropy passphrase, and generate keys from it via a deterministic key derivation process. Using this method, the user may grant access to all encrypted files by simply transferring the passphrase. Storj Core offers tools for this based on Bitcoin's BIP39 Mnemonic passphrases.

{@link KeyRing} provides tools for generating a mnemonic passphrase and writing it to disk. If a mnemonic is found, Storj Core will prefer passphrase-based key derivation to PBKDF2 derivation. Storj clients should create a file key as follows:

  1. Prepend the passphrase to the Bucket ID
  2. Calculate the sha512 hash of the resulting string
  3. Take the first 64 bits of the resulting hash to make the Bucket Key
  4. Prepend the Bucket Key to the File ID
  5. Calculate the sha512 hash of the resulting string
  6. Take the first 64 bits of the resulting hash to make the File Key

As any client with access to the passphrase can easily generate file keys on demand, this provides additional portability of files between machines and applications.

It is strongly recommended that users write down the passphrase, and keep it in a secure place.

Public Bucket Key Storage

Public Buckets are a feature of Bridge, not a feature of Storj. When a Public Bucket is created, the Bucket Key is stored with Bridge. It is then provided to clients requesting PUSH or PULL tokens from that bucket.

References

  • {@link DataCipherKeyIv}
  • {@link DeterministicKeyIV}
  • {@link EncryptStream}
  • {@link DecryptStream}
  • {@link KeyRing}