REST API document is available at docs.digdag.io/api.
The list of release note is here.
- JDK 8
- Node.js 12.x
Installing Node.js using nodebrew:
$ curl -L git.io/nodebrew | perl - setup $ echo 'export PATH=$HOME/.nodebrew/current/bin:$PATH' >> ~/.bashrc $ source ~/.bashrc $ nodebrew install-binary v12.x $ nodebrew use v12.x
Installing Node.js using Homebrew on Mac OS X:
$ brew install node
- Python 3
$ ./gradlew check
Test coverage report is generated at
Findbugs report is generated at
$ CI_ACCEPTANCE_TEST=true ./gradlew digdag-tests:test --info --tests acceptance.BuiltInVariablesIT
To execute tests in digdag-tests subproject locally,
tests option that is provided by Gradle is useful.
CI_ACCEPTANCE_TEST=true is needed to execute digdag-tests.
Testing with PostgreSQL
Test uses in-memory H2 database by default. To use PostgreSQL, set following environment variables:
$ export DIGDAG_TEST_POSTGRESQL="$(cat config/test_postgresql.properties)"
Building CLI executables
$ ./gradlew cli $ ./gradlew cli -PwithoutUi # build without integrated UI
(If the command fails during building UI due to errors from
node command, you can try to add
-PwithoutUi argument to exclude the UI from the package).
It makes an executable in
Node.js development server is useful because it reloads changes of digdag-ui source code automatically.
First, put following lines to ~/.config/digdag/config and start digdag server:
server.http.headers.access-control-allow-origin = http://localhost:9000 server.http.headers.access-control-allow-headers = origin, content-type, accept, authorization, x-td-account-override, x-xsrf-token, cookie server.http.headers.access-control-allow-credentials = true server.http.headers.access-control-allow-methods = GET, POST, PUT, DELETE, OPTIONS, HEAD server.http.headers.access-control-max-age = 1209600
Then, start digdag-ui development server:
$ cd digdag-ui/ $ npm install $ npm run dev # starts dev server on http://localhost:9000/
Updating REST API document
Run this command to update REST API document file at digdag-docs/src/api/swagger.yaml.
./gradlew swaggerYaml # dump swagger.yaml file
--enable-swagger option to check the current Digdag REST API.
$ ./gradlew cli $ ./pkg/digdag-<current version>.jar server --memory --enable-swagger # Run server with --enable-swagger option $ docker run -dp 8080:8080 swaggerapi/swagger-ui # Run Swagger-UI on different console $ open http://localhost:8080/?url=http://localhost:65432/api/swagger.json # Open api/swagger.json on Swagger-UI
Documents are in digdag-docs/src directory. They're built using Sphinx.
To build the pages and check them locally, follow this instruction.
Create a virtual environment of Python and install dependent Python libraries including Sphinx.
$ python3 -m venv .venv $ source .venv/bin/activate (.venv)$ pip install -r digdag-docs/requirements.txt -c digdag-docs/constraints.txt
After installation of Python libraries, You can build with running the following command:
(.venv)$ ./gradlew site
This might not always update all necessary files (Sphinx doesn't manage update dependencies well). In this case, run
./gradlew clean first.
It builds index.html at digdag-docs/build/html/index.html.
Development on IDEs
Digdag is using a Java annotation processor
org.immutables:value. The combination of Java annotation processing and Gradle on IntelliJ IDEA sometimes introduces some troubles. In Digdag's case, you may run into some compile errors like
cannot find symbol: class ImmutableRestWorkflowDefinitionCollection.
So we'd recommend the followings to avoid those compile errors if you want to develop Digdag one the IDE.
- There's an important configuration option to be enabled to fully have IntelliJ be fully integrated with an existing gradle build configuration:
Delegate IDE build/run actions to gradleneeds to be enabled.
Releasing a new version
This is for committers only.
Prerequisite: Sonatype OSSRH
You need an account in Sonatype OSSRH, and configure it in your
ossrhUsername=(your Sonatype OSSRH username) ossrhPassword=(your Sonatype OSSRH password)
Prerequisite: PGP signatures
You need your PGP signatures to release artifacts into Maven Central, and configure Gradle to use your key to sign.
Configure it in your
signing.gnupg.executable=gpg signing.gnupg.useLegacyGpg=false signing.gnupg.keyName=(the last 8 symbols of your keyId) signing.gnupg.passphrase=(the passphrase used to protect your private key)
As mentioned in the prerequirements, we need to build with JDK 8 in this procedure.
git pull upstream master --tags.
./gradlew setVersion -Pto=<version>command.
- write release notes to
releases/release-<version>.rstfile. It must include at least version (the first line) and release date (the last line).
./gradlew clean cli site check releaseCheck.
- make a release branch.
git checkout -b release_v<version>and commit.
- push the release branch to origin and create a PR.
- after the PR is merged to master, checkout master and pull latest upstream/master.
./gradlew clean cli site check releaseCheckagain.
- if it succeeded, run
- create a tag
git tag -a v<version>and push
git push upstream v<version>
- create a release in GitHub releases.
pkg/digdag-<version>.jarto the release
- a few minutes later, run
digdag selfupdateand confirm the version.
If major version is incremented, also update
version = and
release = at digdag-docs/src/conf.py.
If you are expert, skip 5. to 7. and directly update master branch.
Post-process of new release
You also need following steps after new version has been released.
- create next snapshot version, run
./gradlew setVersion -Pto=<next-version>-SNAPSHOT.
- push to master.
Releasing a SNAPSHOT version
Note Snapshot release is not supported currently.