file archiver with modern cipher and compression
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
.zsh
LICENSES
test
LICENSE
Makefile
Makefile.win
README.md
aezio.go
archive.go
cipher.go
debug.go
decode.go
define.go
ec.go
element.go
encode.go
error.go
ezio.go
ezml.go
nodebug.go
option.go
path.go
sys_freebsd.go
sys_linux.go
sys_windows.go

README.md

ezio

'ezio' is a file archiver with modern cipher and compression.

Supported features:

  • Afio like archiving : each file is compressed and encrypted individually
  • Monolithic : no external tools required, such as gpg/openssl
  • Metadata encryption : filename, size, and so on
  • Authenticated encryption : AES256-GCM, CHACHA20-POLY1305
  • Public-key encryption : RSA key/certification
  • Electronic signature : RSA, ECDSA key/certification
  • Modern compression : zstd(default), lz4, xz, bzip2
  • Erasure code : aka recovery record, repairing damaged archive
  • Metadata list : quick access to file list in the archive
  • Extended Attribute : SElinux, capability, ACL
  • Easy to enhance : Go!
  • Easy to use : it's ez.

Status

  • Linux: fully supported
  • FreeBSD: Extended Attribute not supported yet
  • Windows: regular file and directory only. not fully tested

Todo

  • Windows support
  • zstd compression with dictionary
  • speed and memory optimization
  • better messages and error handling

Archive: ezio -a

basic usage

tar style:
ezio -af archive.aez srcdir
afio style:
find srcdir -print | ezio -a -f archive.aez
filter style:
find srcdir -print | ezio -a -c > archive.aez

typical usage

compress:
ezio -azf archive.aez srcdir srcdir2 file file2
compress and encrypt:
ezio -aez -f archive.aez srcdir
append metadata list and erasurecode:
ezio -azmr -f archive.aez srcdir

advanced usage

skip non-accessible files (to be accurate, ignore all errors
ezio -az --ignore-error -f archive.aez srcdir
xz -9 compress:
ezio -az -Zx -G9 -f archive.aez srcdir
password file redirection(read all as pass, including LF and other non-ascii characters):
ezio -ae -f archive.aez --pass-fd=3 srcdir <3 pass.txt
public-key encryption:
ezio -ae -f archive.aez --encrypt-key=pubkey.pem srcdir
signature:
ezio -aez -H5 -f archive.aez --sign-key=privkey.pem srcdir
too many options:
ezio -aezmrUW -f archive.aez -Zz -G11 -Ec -H5 --ignore-error --block-size=512 --block-data=64 --block-parity=3 --encrypt-key=cert.pem --sign-key=private.pem --sign-pass-file=pass.txt --exclude="bar/foo\.mp4$" --include="\.mp4$" srcdir

Extract: ezio -x

basic usage

tar style(extract to current dir):
ezio -xf archive.aez
zip style:
ezio -x -f archive.aez -d dstdir
pipe style:
cat archive.aez | ezio -x -d dstdir
filter style (without -p, extract only the first file):
cat archive.aez | ezio -x -c > dstfile

advanced usage

extract a file in the archive (ezio -l -L1 shows file position):
ezio -x -f archive.aez -p 11345
public-key decryption:
ezio -x -f archive.aez --decrypt-key=prvkey.pem -d dstdir
suppress warning:
ezio -x -O3 -f archive.aez
selective extract(regexp matching):
ezio -x -f archive.aez --include-"(foo|bar)\.dat"

List: ezio -l

If metadata list exists, ezio uses it.

path only:
ezio -l -L0 -f archive.aez
position and path:
ezio -l -L1 -f archive.aez
ls -l style:
ezio -l -L2 -f archive.aez
encrypted archive:
ezio -l -f archive.aez --pass-file=pass.txt

View: ezio -v

Ezio scans archive from the beginning, does not use metadata list.

stat style:
ezio -v -L3 -f archive.aez
json style:
ezio -v -L4 -f archive.aez

Test: ezio -t

extract to /dev/null:
ezio -t -O6 -otest.log -f archive.aez
verify signature:
ezio -t -f archive.aez --verify-key=cert.pem
repair if erasure code exists(generate archive.aez.rep):
ezio -t -r -f archive.aez

Compression algorithm

zstd -3: better size than gzip. much faster.

zstd -9..-11: good size. fast.

zstd -19: near to bzip2 -9 size, but slow compression.

lz4 : quite fast.

xz -9 : best compression rate, but extremely slow compression.

xz -1 .. -3 : good balance between size and speed.

bzip2 -9 : good size. accpetable speed.

sample comparison

centos7# ezio -az -Z? -G? -f usr.aez /usr

method compress(sec) extract size
lz4 36.4 26.1 938462472
xz -1 263.5 83.2 622824297
xz -9 1449.9 94.2 582330416
bzip -9 446.8 133.1 720979342
zstd -3 (default) 52.1 39.8 705216234
zstd -9 132.2 37.6 666123750
zstd -11 228.1 37.4 661068783
zstd -19 713.3 37.9 618484768

Encryption algorithm

If your CPU has AES accelarater, AES256-GCM is faster a bit.

Erasure Code

Ezio uses Reed-Solomon algorithm. You can specify 3 parameters: --block-size, --block-data, --block-parity.

RAID6 is a good analogy. Assume you have a big disk array consists of 130 HDDs, 128 data disks and 2 parity disks. Each HDD has capacity of 4096Bytes, so total size is 512kiB. This disk array is recoverable if 1 or 2 HDDs are damaged. That is, --block-size=4096 --block-data=128 --block-parity=2.

If archive size is over 512kiB, there are other disk arrays. Note that the parity disks in the first disk array are not useful for repairing the other disk array.