No description, website, or topics provided.
Switch branches/tags
Nothing to show
Clone or download
Latest commit 6e95bad Mar 3, 2016
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
config/checkstyle first commit Mar 3, 2016
example Add: example Mar 4, 2016
gradle/wrapper first commit Mar 3, 2016
lib/embulk/filter first commit Mar 3, 2016
src Add: comment source code Mar 4, 2016
.gitignore Add: ignore intellij files Mar 3, 2016
CHANGELOG.md Add CHANGELOG Mar 4, 2016
LICENSE.txt first commit Mar 3, 2016
README.md Update README.md Mar 4, 2016
build.gradle Modify: dependencies and homepage Mar 3, 2016
gradlew first commit Mar 3, 2016
gradlew.bat first commit Mar 3, 2016

README.md

To Csv filter plugin for Embulk

Convert a record to CSV. This plugin is the filter-plugin just ported from embulk-formatter-csv.

Overview

  • Plugin type: filter

Configuration

cf. http://www.embulk.org/docs/built-in.html#csv-formatter-plugin

name type description required?
column_name string Column name used when converting to single value "payload" by default
delimiter string Delimiter character such as , for CSV, "\t" for TSV, `" "` or any single-byte character
quote string The character surrounding a quoted value " by default
quote_policy enum Policy for quote ( ALL, MINIMAL, NONE) (see below) MINIMAL by default
escape string Escape character to escape quote character same with quote default (*1)
header_line boolean If true, write the header line with column name at the first line false by default
null_string string Expression of NULL values empty by default
newline enum Newline character ( CRLF, LF or CR) CRLF by default
newline_in_field enum Newline character in each field ( CRLF, LF, CR) LF by default
charset enum Character encoding (eg. ISO-8859-1, UTF-8) UTF-8 by default
default_timezone string Time zone of timestamp columns. This can be overwritten for each column using column_options UTC by default
column_options hash See bellow optional

(*1): if quote_policy is NONE, quote option is ignored, and default escape is \.

The quote_policy option is used to determine field type to quote.

name description
ALL Quote all fields
MINIMAL Only quote those fields which contain delimiter, quote or any of the characters in lineterminator
NONE Never quote fields. When the delimiter occurs in field, escape with escape char

The column_options option is a map whose keys are name of columns, and values are configuration with following parameters:

name type description required?
timezone string Time zone if type of this column is timestamp. If not set, default_timezone is used. optional
format string Timestamp format if type of this column is timestamp. %Y-%m-%d %H:%M:%S.%6N %z by default

Why does this need?

Some output plugin cannot use a formatter plugin, because they are not inherited FileOutputPlugin, but sometimes they need formatters. In that case, this plugin is useful. For example, embulk-output-bigquery

Example

filters:
  - type: to_csv
    column_name: payload
    delimiter: "\t"
    newline: CRLF
    newline_in_field: LF
    charset: UTF-8
    quote_policy: MINIMAL
    quote: '"'
    escape: "\\"
    null_string: "\\N"
    default_timezone: 'UTC'
    column_options:
      time: {format: '%Y-%m-%d %H:%M:%S', timezone: 'America/Los_Angeles'}

Run the example

$ ./gradlew classpath
$ embulk run example/config.yml -Ilib

Build

$ ./gradlew gem  # -t to watch change of files and rebuild continuously