Skip to content

Demonstration of how to build and deploy Scalding code using Leiningen

Notifications You must be signed in to change notification settings

vonwenm/scalding-on-leiningen

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

2 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Scalding build with Leiningen

This project demonstrates how to build and run a Scalding project using Leiningen. The job itself is ripped from https://github.com/twitter/scalding/blob/master/scalding-core/src/main/scala/com/twitter/scalding/examples/WordCountJob.scala.

Usage

Local mode:

lein run wordcount.WordCountJob --local --input resources/pg1140.txt --output counts.tsv

On Hadoop:

hadoop dfs -copyFromLocal resources/pg1140.txt pg1140.txt
lein uberjar
hadoop jar target/scalding-lein-0.1.0-SNAPSHOT-standalone.jar wordcount.WordCountJob --hdfs --input pg1140.txt --output counts.tsv
hadoop dfs -cat counts.tsv

About

Demonstration of how to build and deploy Scalding code using Leiningen

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published