- This template will point to a stable Spark and Hadoop-client version, potentially working-around some known bugs
src/main/scala/ignition/jobs/Runner.scalais a example of multi-setup project that can run using
core/tools/cluster.py jobs runcommand
scripts/job_runner.pyfor a full-featured, production-grade script-template that launchs and supervises clusters and jobs using AWS spot instances with automatic failure recovery
In general, check the source files for more information.
See http://monkeys.chaordic.com.br/start-using-spark-with-ignition/ for quick-start tutorial
- An example using Scalaz Validation
- Other examples, more documentation