Workload Automation System
Java JavaScript Python Shell Makefile Ruby Other
Latest commit 324c624 Jan 17, 2017 @frsyuki frsyuki committed on GitHub Merge pull request #450 from treasure-data/include-storage-s3-in-cli
Include digdag-storage-s3 in CLI
Permalink
Failed to load latest commit information.
ci td_load/embulk: allow secrets in config Jan 13, 2017
config unit tests use PostgreSQL if DIGDAG_TEST_POSTGRESQL is set Mar 25, 2016
digdag-cli Merge pull request #450 from treasure-data/include-storage-s3-in-cli Jan 17, 2017
digdag-client start 0.9.4 Jan 17, 2017
digdag-core Merge pull request #322 from treasure-data/dano/move-archiving-to-server Jan 17, 2017
digdag-docs release 0.9.3 Jan 17, 2017
digdag-guice-rs-server-undertow correctly handle OPTIONS and TRACE requests Nov 15, 2016
digdag-guice-rs-server remove unnecessary import Nov 9, 2016
digdag-guice-rs servlet name must be unique to bind multiple apps Dec 6, 2016
digdag-plugin-utils remove more dead secret access limit code Jan 16, 2017
digdag-server Merge pull request #322 from treasure-data/dano/move-archiving-to-server Jan 17, 2017
digdag-spi remove more dead secret access limit code Jan 16, 2017
digdag-standards Merge pull request #450 from treasure-data/include-storage-s3-in-cli Jan 17, 2017
digdag-storage-s3 fixed cross-project dependency conflict Jan 17, 2017
digdag-tests Merge pull request #450 from treasure-data/include-storage-s3-in-cli Jan 17, 2017
digdag-ui ui: timestamp tweaks Dec 16, 2016
docker Merge branch 'master' into dano/emr Dec 8, 2016
examples emr: download files to working directory Dec 6, 2016
gradle/wrapper Upgrade gradle to 3.2.1 Dec 6, 2016
.gitignore http_proxy env var support Aug 17, 2016
.travis.yml update docker build image Dec 8, 2016
LICENSE Apply Apache License, Version 2.0 Jun 8, 2016
NOTICE Bundle Moment.js Sep 27, 2016
README.md Merge pull request #346 from treasure-data/bintray Oct 25, 2016
appveyor.yml fixed npm build on Windows Dec 6, 2016
build.gradle Merge pull request #450 from treasure-data/include-storage-s3-in-cli Jan 17, 2017
circle.yml update docker build image Dec 8, 2016
gradlew Upgrade gradle to 3.2.1 Dec 6, 2016
gradlew.bat Upgrade gradle to 3.2.1 Dec 6, 2016
settings.gradle Include Digdag UI in the jar package Dec 6, 2016

README.md

Digdag

Circle CI

Travis CI

Documentation

Please check digdag.io for installation & user manual.

Development

Running tests

$ ./gradlew check

Test coverage report is generated at didgag-*/build/reports/jacoco/test/html/index.html. Findbugs report is generated at digdag-*/build/reports/findbugs/main.html.

Testing with PostgreSQL

Test uses in-memory H2 database by default. To use PostgreSQL, set following environment variables:

$ export DIGDAG_TEST_POSTGRESQL="$(cat config/test_postgresql.properties)"

Releasing a new version

You need to set Bintray user name and API key in BINTRAY_USER and BINTRAY_KEY environment variables.

  1. run ./gradlew setVersion -Pto=<version> command.
  2. write release notes to releases/release-<version>.rst file. It must include at least version (the first line) and release date (the last line).
  3. run ./gradlew clean cli check releaseCheck.
  4. if it succeeded, run ./gradlew release.

If major version is incremented, also update version = and release = at digdag-docs/src/conf.py.

Releasing a SNAPSHOT version

./gradlew releaseSnapshot

Building digdag-ui

$ cd digdag-ui/
$ npm install
$ npm run dev    # starts dev server on http://localhost:9000/
$ npm run build  # build files on public/

Updating documents

Documents are in digdag-docs/src directory. They're built using Sphinx.

Website is hosted on www.digdag.io using Github Pages. Pages are built using deployment step of circle.yml and automatically pushed to gh-pages branch of digdag-docs repository.

To build the pages and check them locally, run following command:

$ ./gradlew site

This might not always update all necessary files (Sphinx doesn't manage update dependencies well). In this case, run ./gradlew clean first.

It buids index.html at digdag-docs/build/html/index.html.