MessagePack parser plugin for Embulk
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
config/checkstyle
gradle/wrapper
lib/embulk
src
.gitignore
COPYING
ChangeLog
README.md
build.gradle
gradlew
gradlew.bat

README.md

MessagePack parser plugin for Embulk

Parses files encoded in MessagePack.

Overview

  • Plugin type: parser
  • Guess supported: yes

Configuration

  • row_encoding: type of a row. "array" or "map" (enum, default: map)
  • file_encoding: if a file includes a big array, set "array". Otherwise, if a file includes sequence of rows, set "sequence" (enum, default: sequence)
  • columns: description (schema, required)

Example

seed.yml:

in:
  # here can use any file input plugin type such as file, s3, gcs, etc.
  type: file
  path_prefix: /path/to/file/or/directory
  parser:
    type: msgpack

Command:

$ embulk gem install embulk-parser-msgpack
$ embulk guess -g msgpack seed.yml -o config.yml
$ embulk run config.yml

The guessed config.yml will include column settings:

in:
  type: any file input plugin type
  parser:
    type: msgpack
    row_encoding: map
    file_encoding: sequence
    columns:
    - {index: 0, name: a, type: long}
    - {index: 1, name: b, type: string}

Build

$ ./gradlew gem