LZ4 encoder plugin for Embulk
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
config/checkstyle
gradle/wrapper
lib/embulk/encoder
src
.gitignore
LICENSE.txt
README.md
build.gradle
gradlew
gradlew.bat

README.md

Lz4 encoder plugin for Embulk

This plugin is embulk file encoder for LZ4, a Extremely fast compression algorithm.

Caution! : This plugin supports LZ4 Frame Format only!

LZ4 Frame Format is the most famously format for compressing file of any size by LZ4. if you use Linux, lz4 program has generating a compressed file formatted by LZ4 Frame Format.

then, this plugin is not supported other formats based on LZ4 algorithm such as raw LZ4 Block Format, currently.

Overview

  • Plugin type: encoder

Configuration

  • block_size: Block Maximum Size for uncompressed buffer. you can choose from 65535, 262144, 1048576 or 4194304. (integer, default: 4194304)

Example

out:
  type: any output input plugin type
  encoders:
    - type: lz4
      block_size: 4194304

Build

$ ./gradlew gem  # -t to watch change of files and rebuild continuously