methods for hashing, compressing, and encoding bytes
Clojure Java
Switch branches/tags
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
src
test
.gitignore
.travis.yml
LICENSE.txt
NOTICE.txt
README.md
project.clj

README.md

This library helps you hash, compress, and encode streams of bytes. It contains the methods in the standard Java lib, as well as a curated collection of the best available methods.

Usage

[byte-transforms "0.1.4"]

All functions are in the byte-transforms namespace. There are five primary functions, hash, compress, decompress, encode, and decode. Each takes three arguments: the bytes, the method, and an (optional) options map. The bytes can be anything which is part of the byte-stream conversion graph.

byte-transforms> (hash "hello" :murmur64)
2191231550387646743

byte-transforms> (compress "hello" :snappy)
#<byte[] [B@7e0f980b>
byte-transforms> (byte-streams/to-string (decompress *1 :snappy))
"hello"

byte-transforms> (byte-streams/to-string (encode "hello" :base64 {:url-safe? false}))
"aGVsbG8"
byte-transforms> (byte-streams/to-string (decode *1 :base64))
"hello"

Note that Base64 encoding defaults to the URL-safe variant, which means that the output will not be padded. This can be disabled by passing in {:url-safe? false} to encode.

Available methods can be found via available-hash-functions, available-compressors, and available-encoders:

byte-transforms> (available-hash-functions)
(:sha384 :md2 :crc32 :crc64 :sha512 :sha1 :murmur32 :murmur128 :adler32 :sha256 :md5 :murmur64)
byte-transforms> (available-compressors)
(:lz4 :bzip2 :snappy :gzip)
byte-transforms> (available-encoders)
(:base64)

When choosing a compression algorithm, snappy is typically the fastest, bzip2 yields the highest compression, and lz4 provides a good balance between higher compression rate and fast decompression. All the compression algorithms except lz4 are concat-able; multiple compressed segments can be concatenated and decompressed as a single stream.

Full stats on all methods can be found by cloning the project and running lein test :benchmark.

License

Copyright © 2013 Zachary Tellman

Distributed under the Apache License 2.0.