digdag-sh-result-store is plugin storing output of shell to digdag store.
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.github
gradle/wrapper
src
.gitignore
.travis.yml
LICENSE
README.md
build.gradle
gradlew
gradlew.bat

README.md

digdag-plugin-shresult

JitPack Build Status Codacy Badge Coverage Status

digdag-plugin-shresult is plugin storing output of shell to digdag store.

Getting Started

setup digdag, see https://www.digdag.io/

create digdag workflow, following code is sample workflow.

sample.dig ... for each by resultset of BigQuery.

_export:
  plugin:
    repositories:
      - https://jitpack.io
    dependencies:
      - com.github.takemikami:digdag-plugin-shresult:0.0.2

+step1:
  sh_result>: |
    bq query --format=json 'select name from dataset1.table1 limit 10'
  destination_variable: resultset
  stdout_format: json-list-map

+step2:
  for_each>:
    rv: ${resultset}
  _parallel:
    true
  _do:
    echo>: ${rv.name}

run sample workflow.

digdag run sample.dig

Usage

sh_result>: Shell scripts and store output

sh_result> operator runs a shell scripts and set output to digdag store.

Options:

  • sh_result>: COMMAND [ARGS...]

    Name of the command to run.

  • destination_variable: Name

    Specifies a digdag-variable to store the standard output of script in.

  • stdout_format: Name

    Type of standard output format.

    • text
    • newline-delimited
    • space-delimited
    • json-list-map

Stdout Formats

  • text ... set each stdout to single value.

    example

    text message
    
  • newline-delimited ... set newline-delimited stdout to list.

    example

    item1
    item2
    item3
    
  • space-delimited ... set space-delimited stdout to list.

    example

    item1 item2 item3
    
  • json-list-map ... set json-based stdout to list of map.

    example

    [
      {
        "id": "001",
        "name": "item1"
      },
      {
        "id": "002",
        "name": "item2"
      }
    ]