A filter plugin for Embulk to rearrange an input row data to multiple output row data.
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
gradle/wrapper
lib/embulk/filter
src/main/java/org/embulk/filter
.gitignore
LICENSE.txt
README.md
build.gradle
example.csv
example.yml
gradlew
gradlew.bat

README.md

Rearrange filter plugin for Embulk

A filter plugin for Embulk to rearrange an input row data to multiple output row data.

Overview

  • Plugin type: filter

Configuration

  • columns: Columns of output data. (array of hash, required)
    • name: Name of column.
    • type: Type of column.
  • output_data: Contents of output data. (array of array, required)
    • Each element is specified in column name of the input data.
    • Length of each line of output data is equal to or less than the length of the input data.
  • add_row_number: If true, add row number to output data. (boolean, optional, default: false)
    • Row number starts 0, and it increments in a same input data. If we read next input row data, row number resets to 0.
  • row_number_column_name: Column name of row number. (string, optional, default: row_number)

Example

sample input

date,item1,value1,item2,value2
2015-08-19 00:00:00,hoge1,100,fuga1,200
2015-08-20 00:00:00,hoge2,300,fuga2,400
2015-08-21 00:00:00,hoge3,500,fuga3,600

sample config

filters:
  - type: rearrange
    columns:
      - { name:date, type:timestamp }
      - { name:item, type:string }
      - { name:value, type:long }
    output_data:
      - ["date", "item1", "value1"]
      - ["date", "item2", "value2"]
    add_row_number: true
    row_number_column_name: "\"rownum\""

result

+-------------------------+-------------+------------+-------------+
|          date:timestamp | item:string | value:long | rownum:long |
+-------------------------+-------------+------------+-------------+
| 2015-08-19 00:00:00 UTC |       hoge1 |        100 |           0 |
| 2015-08-19 00:00:00 UTC |       fuga1 |        200 |           1 |
| 2015-08-20 00:00:00 UTC |       hoge2 |        300 |           0 |
| 2015-08-20 00:00:00 UTC |       fuga2 |        400 |           1 |
| 2015-08-21 00:00:00 UTC |       hoge3 |        500 |           0 |
| 2015-08-21 00:00:00 UTC |       fuga3 |        600 |           1 |
+-------------------------+-------------+------------+-------------+

TODO

  • Write test

Build

$ ./gradlew gem