Skip to content
master
Switch branches/tags
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

ezio

'ezio' is a file archiver with modern cipher and compression.

Supported features:

  • Afio like archiving : each file is compressed and encrypted individually
  • Monolithic : no external tools required, such as gpg/openssl
  • Metadata encryption : filename, size, and so on
  • Authenticated encryption : AES256-GCM, CHACHA20-POLY1305
  • Public-key encryption : RSA key/certification
  • Electronic signature : RSA, ECDSA key/certification
  • Modern compression : zstd(default), lz4, xz, bzip2
  • Erasure code : aka recovery record, repairing damaged archive
  • Metadata list : quick access to file list in the archive
  • Extended Attribute : SElinux, capability, ACL
  • Easy to enhance : Go!
  • Easy to use : it's ez.

Status

  • Linux: fully supported
  • FreeBSD: Extended Attribute not supported yet
  • Windows: regular file and directory only. not fully tested

Todo

  • Windows support
  • zstd compression with dictionary
  • speed and memory optimization
  • better messages and error handling

Archive: ezio -a

basic usage

tar style:
ezio -af archive.aez srcdir
afio style:
find srcdir -print | ezio -a -f archive.aez
filter style:
find srcdir -print | ezio -a -c > archive.aez

typical usage

compress:
ezio -azf archive.aez srcdir srcdir2 file file2
compress and encrypt:
ezio -aez -f archive.aez srcdir
append metadata list and erasurecode:
ezio -azmr -f archive.aez srcdir

advanced usage

skip non-accessible files (to be accurate, ignore all errors
ezio -az --ignore-error -f archive.aez srcdir
xz -9 compress:
ezio -az -Zx -G9 -f archive.aez srcdir
password file redirection(read all as pass, including LF and other non-ascii characters):
ezio -ae -f archive.aez --pass-fd=3 srcdir <3 pass.txt
public-key encryption:
ezio -ae -f archive.aez --encrypt-key=pubkey.pem srcdir
signature:
ezio -aez -H5 -f archive.aez --sign-key=privkey.pem srcdir
too many options:
ezio -aezmrUW -f archive.aez -Zz -G11 -Ec -H5 --ignore-error --block-size=512 --block-data=64 --block-parity=3 --encrypt-key=cert.pem --sign-key=private.pem --sign-pass-file=pass.txt --exclude="bar/foo\.mp4$" --include="\.mp4$" srcdir

Extract: ezio -x

basic usage

tar style(extract to current dir):
ezio -xf archive.aez
zip style:
ezio -x -f archive.aez -d dstdir
pipe style:
cat archive.aez | ezio -x -d dstdir
filter style (without -p, extract only the first file):
cat archive.aez | ezio -x -c > dstfile

advanced usage

extract a file in the archive (ezio -l -L1 shows file position):
ezio -x -f archive.aez -p 11345
public-key decryption:
ezio -x -f archive.aez --decrypt-key=prvkey.pem -d dstdir
suppress warning:
ezio -x -O3 -f archive.aez
selective extract(regexp matching):
ezio -x -f archive.aez --include-"(foo|bar)\.dat"

List: ezio -l

If metadata list exists, ezio uses it.

path only:
ezio -l -L0 -f archive.aez
position and path:
ezio -l -L1 -f archive.aez
ls -l style:
ezio -l -L2 -f archive.aez
encrypted archive:
ezio -l -f archive.aez --pass-file=pass.txt

View: ezio -v

Ezio scans archive from the beginning, does not use metadata list.

stat style:
ezio -v -L3 -f archive.aez
json style:
ezio -v -L4 -f archive.aez

Test: ezio -t

extract to /dev/null:
ezio -t -O6 -otest.log -f archive.aez
verify signature:
ezio -t -f archive.aez --verify-key=cert.pem
repair if erasure code exists(generate archive.aez.rep):
ezio -t -r -f archive.aez

Compression algorithm

zstd -3: better size than gzip. much faster.

zstd -9..-11: good size. fast.

zstd -19: near to bzip2 -9 size, but slow compression.

lz4 : quite fast.

xz -9 : best compression rate, but extremely slow compression.

xz -1 .. -3 : good balance between size and speed.

bzip2 -9 : good size. accpetable speed.

sample comparison

centos7# ezio -az -Z? -G? -f usr.aez /usr

method compress(sec) extract size
lz4 36.4 26.1 938462472
xz -1 263.5 83.2 622824297
xz -9 1449.9 94.2 582330416
bzip -9 446.8 133.1 720979342
zstd -3 (default) 52.1 39.8 705216234
zstd -9 132.2 37.6 666123750
zstd -11 228.1 37.4 661068783
zstd -19 713.3 37.9 618484768

Encryption algorithm

If your CPU has AES accelarater, AES256-GCM is faster a bit.

Erasure Code

Ezio uses Reed-Solomon algorithm. You can specify 3 parameters: --block-size, --block-data, --block-parity.

RAID6 is a good analogy. Assume you have a big disk array consists of 130 HDDs, 128 data disks and 2 parity disks. Each HDD has capacity of 4096Bytes, so total size is 512kiB. This disk array is recoverable if 1 or 2 HDDs are damaged. That is, --block-size=4096 --block-data=128 --block-parity=2.

If archive size is over 512kiB, there are other disk arrays. Note that the parity disks in the first disk array are not useful for repairing the other disk array.