Azkaban workflow manager.
chengren311 Refactor hadoop token fetch logic follow-up (#2028)
This PR:
1. adds more logging and standardize existing logging for each token prefetching methods so that we know prefetching from which service is stuck.

2. removes "synchronized" for doPrefetch(HadoopSecurityManager_H_2_0#doPrefetch). Current design makes it hard to debug which token service the job is stuck with fetching token. Since HadoopSecurityManager_H_2_0 is shared by all jobs in the executor, if one job is stuck with fetching token with a problematic token service, all other jobs will be blocked from entering into this synchronized method. It's impossible to infer which token service jobs are stuck with from job logs as they are just waiting for one job to finish fetching token.
Latest commit bcbb639 Nov 16, 2018
Permalink
Failed to load latest commit information.
az-core Improve error message for a cleaned project version (#1993) Nov 5, 2018
az-crypto reorganize the crypto module's directory structure to match with mave… Aug 10, 2018
az-examples write documents for Flow 2.0 user guide. (#1896) Aug 7, 2018
az-exec-util Move execute-as-user c program to its own module (#1450) Sep 11, 2017
az-flow-trigger-dependency-plugin move dependency plugin interface code to a separate module (#1632) Feb 7, 2018
az-flow-trigger-dependency-type Event trigger PR (#1858) Jul 25, 2018
az-hadoop-jobtype-plugin Only incorporate user pig additional jar setting or system's but not … Oct 31, 2018
az-hdfs-viewer remove goHomeDir (#1799) Jun 7, 2018
az-jobsummary Jobsummary lib should include both classes and velocity files (#1790) Jun 5, 2018
az-reportal Enhancement to reportal presto jobtype (#1977) Oct 15, 2018
azkaban-common Project dir cache enhancement (#2017) Nov 14, 2018
azkaban-db increase job name varchar size in Mysql table to allow long job name … May 31, 2018
azkaban-exec-server Project dir cache enhancement (#2017) Nov 14, 2018
azkaban-hadoop-security-plugin Refactor hadoop token fetch logic follow-up (#2028) Nov 16, 2018
azkaban-solo-server Fix single server startup (by activating the executor) (#2003) Oct 24, 2018
azkaban-spi Event reporting for azkaban events - No default implementation (#1524) Oct 5, 2017
azkaban-web-server New 'expand/collapse all flows' menu options on flow views (#2019) Nov 13, 2018
docs Always use multi-executor mode (#1986) Oct 18, 2018
gradle/wrapper Upgrade gradle to v4.6 (#1774) May 28, 2018
test Fix flaky test runFlowOnJobStatusConditionNull_2 (#1955) Sep 14, 2018
tools Update the create_release tool readme (#2008) Oct 31, 2018
.gitignore Move python related gitignore configs to the root level (#1898) Aug 7, 2018
.travis.yml Change `coveralls` logging level (#1983) Oct 18, 2018
CONTRIBUTING.md update save action guide (#1962) Oct 12, 2018
LICENSE Initial Commit Oct 18, 2012
NOTICE Updated the NOTICE file to fix typos and update links May 14, 2014
README.md move documentation build guide to the root readme (#1913) Aug 9, 2018
az-intellij-style.xml Customize code style template. Update README (#1406) Aug 28, 2017
build.gradle Revert "Add presto jdbc dependency for reportal module (#1959)" (#1963) Sep 24, 2018
gradle.properties Enable gradle build cache for faster build (#1309) Aug 6, 2017
gradlew Upgrade gradle wrapper to use version 4.0.2 (#1223) Aug 7, 2017
gradlew.bat Update gradle wrapper to use the 3.5 version (#1058) May 8, 2017
requirements.txt fix README (#1863) Jul 20, 2018
settings.gradle Move crypto plugin from azkaban-plugins repo to main azkaban repo. (#… Aug 9, 2018

README.md

Azkaban

Build Statuscodecov.ioJoin the chat at https://gitter.im/azkaban-workflow-engine/LobbyDocumentation Status

Build

Azkaban builds use Gradle and requires Java 8 or higher.

The following set of commands run on *nix platforms like Linux, OS X.

# Build Azkaban
./gradlew build

# Clean the build
./gradlew clean

# Build and install distributions
./gradlew installDist

# Run tests
./gradlew test

# Build without running tests
./gradlew build -x test

Build a release

Pick a release from the release page. Find the tag corresponding to the release.

Check out the source code corresponding to that tag. e.g.

git checkout 3.30.1

Build

./gradlew clean build

Documentation

The current documentation will be deprecated soon at azkaban.github.io. The new Documentation site is under development. The source code for the documentation is inside docs directory.

For help, please visit the Azkaban Google Group.

Developer Guide

See the contribution guide.

Documentation development

If you want to contribute to the documentation or the release tool (inside the tools folder), please make sure python3 is installed in your environment. python virtual environment is recommended to run these scripts.

To download the python3 dependencies, run

pip3 install -r requirements.txt

After, enter the documentation folder docs and make the build by running

cd docs
make html

[July, 2018] We are actively improving our documentation. Everyone in the AZ community is welcome to submit a pull request to edit/fix the documentation.