Skip to content

Multi-Step Flows
Compare
Choose a tag to compare
@paulhoule paulhoule released this 14 Oct 14:16
· 360 commits to master since this release
t20131013
5e91186

Issue fixed in this release are written up here

The major new feature is #32, which lets a number of Hadoop jobs be grouped into a "flow", named after the "job flow" concept in the Amazon EMR API. In the case of :BaseKB Now production, all of the steps are submitted as a unit to Amazon EMR so that a single cluster does all the work, rather than starting new clusters. This helps with speed, reliability, cost and all that.

Not using EMR? No problem. Haruhi will submit the jobs sequentially to your cluster.

This release has some minor bug fixes and also marks increasing process maturity because the maven-release plugin is integrated #44 and we are know using Travis CI to monitor build quality #45.

Assets 2
Loading