syc encryption

syc encryption

Status: written and read by syc since v0.5.0; provisional, not frozen (an outside review of the design may still change it). Test vectors (enc-* in syc-vectors.zip) and the reference reader tools/syc_ref.py, which decrypts with libsodium and hashlib, not with syc's code. Encrypted archives are archive format 3 and encrypted streams stream version 5; everything not said here is as in syc archive format (format 2) and syc stream format (version 4), which do not change. Little-endian throughout. ‖ is concatenation.

What it protects, and what it does not

Against someone who has the file but not the password:

• Hidden: file contents. In the default mode also the index: names, sizes, dates, attributes, the directory tree, which fragments repeat. With --visible-names the index stays readable (syc l and syc i work without the password) and only the data is hidden.
• Detected (exit 3): any change to the data, the index or the key record; a block moved to another place, version or archive; a stream cut at a block boundary.
• Not hidden: the number of versions; each block's sizes (so roughly how much data each version added); the file's total size.
• Not detected: an archive cut back to the end of an earlier version reads as an archive with fewer versions (rollback). Keep a copy of the archive's size, or of its last commit, if that matters.
• Not a defence against: a weak password (Argon2id only slows guessing down), a compromised computer, or someone who saw the password.

The code is syc's own (no external library); the algorithms are standard and every primitive is checked against the RFCs' test vectors and against independent implementations (libsodium through PyNaCl, OpenSSL, Python's hashlib). An outside review is welcome before relying on it.

Algorithms

password → key	Argon2id version 1.3 (RFC 9106), output 32 bytes, no secret, no associated data
encryption	XChaCha20-Poly1305 (RFC 8439's ChaCha20-Poly1305 AEAD with HChaCha20's 24-byte nonce, draft-irtf-cfrg-xchacha), tag 16 bytes
subkeys, fragment ids	BLAKE2s-256 keyed (RFC 7693, key 32 bytes)
randomness	the system's CSPRNG (Linux getrandom, Windows BCryptGenRandom)

Keys

• K, the data key: 32 random bytes per archive or stream, made when it is created. Never derived from the password: the password only unwraps it, so changing the password rewrites ~100 bytes and nothing else.
• Subkeys: Kx = BLAKE2s-256(key = K, message = label) with the ASCII labels syc aead, syc fragment, syc cdc:
  • K_aead encrypts the blocks and the index;
  • K_fragment keys the fragments' hashes: a fragment's identity is BLAKE2s-256(key = K_fragment, message = fragment), so a readable index (--visible-names) does not reveal whether a known file is in the archive; deduplication works as before inside the archive;
  • K_cdc seeds the cutter: the gear table is made as in format 2 but from the state u64(K_cdc[0..8]) instead of 0x5359434143444331, so where fragments end does not depend on the contents alone.
• Password: its UTF-8 bytes as given (no normalization, no trailing newline), 1 to 1024 bytes.

The key record

Right after the header of an archive (8 bytes) or of a stream (7 bytes):

keyrec := "SYCK"  flags:u8  reserved:u8[3]  slot[2]                          216 bytes
slot   := gen:u32  kdf:u8  lanes:u8  reserved:u16  memKiB:u32  passes:u32    104 bytes
          salt[16]  nonce[24]  wrapped[48]

• flags bit 0: the archive's index is encrypted (the default mode); 0 with --visible-names, and always 0 in a stream. Other bits and reserved are zero.
• A slot with gen = 0 is empty (all its bytes zero). A reader tries the non-empty slots, the higher gen first, and takes the first that unwraps.
• kdf = 1 (Argon2id v1.3); lanes = Argon2's p (1–16), memKiB = m (8·p .. 4 194 304, i.e. up to 4 GiB), passes = t (1–256); reserved zero. syc writes p = 1, m = 65536 (64 MiB, --kdf-mem changes it), t = 3.
• KEK = Argon2id(password, salt, t, m, p, 32 bytes); wrapped = XChaCha20-Poly1305-Encrypt(key = KEK, nonce, plaintext = K, AAD) (32 bytes of ciphertext, then the 16-byte tag), where AAD = every byte of the file before the key record (the header) ‖ the key record's first 8 bytes ‖ the slot's first 16 bytes. The header, the flags and the Argon2 parameters cannot change without the unwrap failing.
• A new archive or stream fills slot 0 with gen = 1. Changing the password (syc k, since v0.5.3) opens a slot with the current password, writes the other slot with the same data key K at gen = (the higher of the two gens) + 1, flushes the file to the disk, then zeroes the opened slot and flushes again: if interrupted, the old password or the new one opens the file, never neither. Nothing else in the file changes. The new slot keeps the opened one's Argon2id parameters unless --kdf-mem sets the memory. Because K does not change, someone who learned it (or kept a copy of the file) while knowing the old password can still read the data: changing the password is not revoking access.
• No slot unwraps: "wrong password, or a damaged key record" (exit 3).

Archive format 3

header := "SYCA"  3  cdcMinLog  cdcAvgLog  cdcMaxLog  keyrec          8 + 216 bytes
block  := rawLen:u32  packed:u32  crc32:u32 (= 0)  nonce[24]  ciphertext[payLen]  tag[16]
index  := "SYCI"  rawLen:u32  packed:u32  crc32:u32 (= 0)  nonce[24]  ciphertext[payLen]  tag[16]   (flags bit 0 = 1)
        | as in format 2, with its crc32                                                         (flags bit 0 = 0)

• vhead as in format 2 (20 bytes), with time = 0: the time is in the index.
• Blocks: the payload (stored or a sylz block, as packed says, payLen = packed & 0x7FFFFFFF) is encrypted with K_aead and a random nonce; AAD = "SYCB" ‖ vnum:u32 ‖ k:u32 ‖ rawLen:u32 ‖ packed:u32, where k is the block's position in its version (0, 1, …). The tag replaces the CRC, which is zero (a CRC of the plaintext would let a guess be confirmed). A block's size in blockinfo.bytes is 12 + 24 + payLen + 16.
• Index: encrypted the same way with AAD = "SYCI" ‖ vnum:u32 ‖ rawLen:u32 ‖ packed:u32. Its plaintext (encrypted or not) is time:i64 ‖ indexdata, indexdata as in format 2, whose fragment hashes are the keyed ids above.
• end and commit as in format 2. Recovery and appending as in format 2; appending needs the password and keeps the archive's mode.

Stream version 5

stream := "SYZ0"  5  codec:u8  level:u8  keyrec  block*  end
block  := rawLen:u32  packed:u32  crc32:u32 (= 0)  nonce[24]  ciphertext[payLen]  tag[16]
end    := 0:u32  nonce[24]  tag[16]

• Blocks as in an archive, with AAD = "SYZB" ‖ k:u64 ‖ rawLen:u32 ‖ packed:u32, k the block's position from 0.
• end is the encryption of nothing with AAD = "SYZE" ‖ n:u64, n the number of blocks: a stream cut at a block boundary, or with blocks added, is refused. Nothing follows end.

Command line

• syc a -p ARCHIVE PATH..., syc c -p FILE: create encrypted (default: index too; --visible-names keeps it readable; --kdf-mem MIB).
• The password comes from, in order: -pPASSWORD / --password=PASSWORD (visible to other users in the process list: avoid it), --password-file FILE (its first line), the environment variable SYC_PASSWORD, else the terminal (without echo; twice when creating). Reading an encrypted file needs no -p: the same sources, or the terminal.
• Wrong password: exit 3. No password and no terminal: exit 2.
• Speed (silesia, 212 MB, level 1): Argon2id at 64 MiB takes ~0.24 s, once per command; the blocks are sealed and opened on the worker threads (XChaCha20-Poly1305 ~470 MB/s a thread on x86-64). syc a -t 4 0.73 → 0.93 s, syc x -t 4 0.37 → 0.63 s, syc t -t 4 0.32 → 0.48 s.