Skip to content
Cascading is a feature rich API for defining and executing complex and fault tolerant data processing flows on a Hadoop cluster. See for the release repository.
Latest commit 4454b44 Apr 13, 2016 @cwensel Updated jgrapht to 0.9.2.
Failed to load latest commit information.
cascading-core Updated jgrapht to 0.9.2. Apr 13, 2016
cascading-expression Created cascading-expression sub-project to isolate all 'expression' … Jul 16, 2015
cascading-hadoop if zero sinks, return null Feb 25, 2016
cascading-hadoop2-io Updated Apache Hadoop 2 based sub-projects to 2.7.1. Aug 7, 2015
cascading-hadoop2-mr1 allow slice attempts to return the hostname where the given attempt e… Jan 25, 2016
cascading-hadoop2-tez-stats allow slice attempts to return the hostname where the given attempt e… Jan 26, 2016
cascading-hadoop2-tez safely iterate all sinks on clean Jan 22, 2016
cascading-local fixed localization issues in tests, feature flags on TestPlatform int… Jan 19, 2016
cascading-platform fixed localization issues in tests, feature flags on TestPlatform int… Jan 19, 2016
cascading-xml merge wip-2.7 into wip-3.0 Mar 20, 2015
etc Created cascading-hadoop2-io sub-project to isolate Hadoop 2.x HDFS a… Jul 15, 2015
.gitignore eclipse specific ignores Mar 24, 2015
CHANGES.txt Updated jgrapht to 0.9.2. Apr 13, 2016 Added Sep 24, 2014
LICENSE.txt additional copyright updates Jan 15, 2015 cleanup build file, add allJavadoc task to aggregate all javadoc, upd… Jul 16, 2015
apl.txt Changed license from GPLv3 to APLv2. Oct 6, 2011
build.gradle Created cascading-hadoop2-tez-stats sub-project to isolate Tez/YARN t… Sep 24, 2015
settings.gradle Created cascading-hadoop2-tez-stats sub-project to isolate Tez/YARN t… Sep 24, 2015 update to version 3.1.0 Jun 25, 2015


Thanks for using Cascading.

Cascading 3.1

Cascading 3.1 includes a few major changes and additions from prior releases:

  • Complete re-write of the platform query planner and improvements to the planner API
  • Addition of Apache Tez as a supported runtime platform
  • Changes to the Tap/Scheme generic type signatures to support portability

These changes hope to simplify the creation of new bindings to new platform implementations and to improve the performance of resulting applications.

General Information:

For project documentation and community support, visit:

To download a pre-built distribution, visit, or use Maven (described below).

The project includes eight Cascading jar files:

  • cascading-core-x.y.z.jar - all Cascading Core class files
  • cascading-xml-x.y.z.jar - all Cascading XML operations class files
  • cascading-expression-x.y.z.jar - all Cascading Janino expression operations class files
  • cascading-local-x.y.z.jar - all Cascading Local in-memory mode class files
  • cascading-hadoop-x.y.z.jar - all Cascading Hadoop 1.x MapReduce mode class files
  • cascading-hadoop2-io-x.y.z.jar - all Cascading Hadoop 2.x HDFS and IO related class files
  • cascading-hadoop2-mr1-x.y.z.jar - all Cascading Hadoop 2.x MapReduce mode class files
  • cascading-hadoop2-tez-x.y.z.jar - all Cascading Hadoop 2.x Tez mode class files

These class jars, along with, tests, source and javadoc jars, are all available via the Maven repository.

Hadoop 1.x mode is where the Cascading application should run on a Hadoop MapReduce cluster.

Hadoop 2.x MR1 is the same as above but for Hadoop 2.x releases.

Hadoop 2.x Tez mode is where the Cascading application should run on an Apache Tez DAG cluster.

Local mode is where the Cascading application will run locally in memory without any Hadoop dependencies or cluster distribution. This implementation has minimal to no robustness in low memory situations, by design.

As of Cascading 3.x, all above jar files are built against Java 1.7. Prior versions of Cascading are built against Java 1.6.

Extensions, the SDK, and DSLs

There are a number of projects based on Cascading available. Visit the Cascading Extensions page for a current list.

Or download the Cascading SDK which includes pre-built binaries.

Of note are three top level projects:

  • Fluid - A fluent API for Cascading that is compatible with the default API.
  • Lingual - ANSI SQL and JDBC on Cascading
  • Pattern - Machine Learning scoring and PMML support with Cascading

And new languages:


Cascading stable releases are always of the form x.y.z, where z is the current maintenance release.

x.y.z releases are maintenance releases. No public incompatible API changes will be made, but in an effort to fix bugs, remediation may entail throwing new Exceptions.

x.y releases are minor releases. New features are added. No public incompatible API changes will be made on the core processing APIs (Pipes, Functions, etc), but in an effort to resolve inconsistencies, minor semantic changes may be necessary.

It is important to note that we do reserve to make breaking changes to the new query planner API through the 3.x releases. This allows us to respond to bugs and performance issues without issuing new major releases. Cascading 4.0 will keep the public query planner APIs stable.

The source and tags for all stable releases can be found here:

WIP (work in progress) releases are fully tested builds of code not yet deemed fully stable. On every build by our continuous integration servers, the WIP build number is increased. Successful builds are then tagged and published.

The WIP releases are always of the form x.y.z-wip-n, where x.y.z will be the next stable release version the WIP releases are leading up to. n is the current successfully tested build.

The source, working branches, and tags for all WIP releases can be found here:

Or downloaded from here:

When a WIP is deemed stable and ready for production use, it will be published as a x.y.z release, and made available from the page.

Writing and Running Tests

Comprehensive tests should be written against the cascading.PlatformTestCase.

When running tests built against the PlatformTestCase, the local cluster can be disabled (if enabled by the test) by setting:


From Gradle, to run a single test case:

> gradle :cascading-hadoop2-mr1:platformTest --tests=*.FieldedPipesPlatformTest -i 

or a single test method:

> gradle :cascading-hadoop2-mr1:platformTest --tests=*.FieldedPipesPlatformTest.testNoGroup -i 

Debugging the 3.x Planner

The new 3.0 planner has a much improved debugging framework.

When running tests, set the following


If you are on Mac OS X and have installed GraphViz, dot files can be converted to pdf on the fly. To enable, set:

Optionally, for stand alone applications, statistics and tracing can be enabled selectively with the following properties:

  • cascading.planner.stats.path - outputs detailed statistics on time spent by the planner
  • cascading.planner.plan.path - basic planner information
  • cascading.planner.plan.transforms.path - detailed information for each rule

Contributing and Reporting Issues

See at

Using with Maven/Ivy

It is strongly recommended developers pull Cascading from our Maven compatible jar repository

You can find the latest public and WIP (work in progress) releases here:

When creating tests, make sure to add any of the relevant above dependencies to your test scope or equivalent configuration along with the cascading-platform dependency.

Note the cascading-platform compile dependency has no classes, you must pull the tests dependency with the tests classifier.

See for example Maven pom dependency settings.

Source and Javadoc artifacts (using the appropriate classifier) are also available through Conjars.

Note that cascading-hadoop, cascading-hadoop2-mr1, and cascading-hadoop2-tez have a provided dependency on the Hadoop jars so that it won't get sucked into any application packaging as a dependency, typically.

Building and IDE Integration

For most cases, building Cascading is unnecessary as it has been pre-built, tested, and published to our Maven repository (above).

To build Cascading, run the following in the shell:

> git clone
> cd cascading
> gradle build

Cascading requires Gradle 1.12 and Java 1.7 to build.

To use an IDE like IntelliJ, run the following to create IntelliJ project files:

> gradle idea

Similarly for Eclipse:

> gradle eclipse

Using with Apache Hadoop

First confirm you are using a supported version of Apache Hadoop by checking the Compatibility page.

To use Cascading with Hadoop, we suggest stuffing cascading-core, cascading-hadoop2-mr1, (optionally) cascading-xml jar files and all third-party libs into the lib folder of your job jar and executing your job via $HADOOP_HOME/bin/hadoop jar your.jar <your args>.

For example, your job jar would look like this (via: jar -t your.jar)

/<all your class and resource files>
/lib/<cascading third-party jar files>

Hadoop will unpack the jar locally and remotely (in the cluster) and add any libraries in lib to the classpath. This is a feature specific to Hadoop.

Something went wrong with that request. Please try again.