Upgrading to 0.9.0
daisyzhou edited this page May 2, 2014
·
3 revisions
Pages 38
- Home
- Aggregation using Algebird Aggregators
- All about reducers count
- API Reference
- Automatic Orderings, Monoids and Arbitraries
- Building bigger platforms with scalding
- Calling Scalding from inside your application
- Common Exceptions and possible reasons
- Comparison to Scrunch and Scoobi
- Cookbook!
- Field rules
- Fields API: reduce functions of GroupBuilder
- Fields based API Reference
- Frequently asked questions
- Getting Started
- Intro to Scalding Jobs
- Introduction to Matrix Library
- Matrix API Reference
- Pig to Scalding
- Powered By
- REPL Reference
- Rosetta Code
- Run in Intellij IDEA
- Scala and sbt for Homebrew users
- Scala and sbt for MacPorts users
- Scald.rb
- Scalding Commons
- Scalding HBase
- Scalding on amazon elastic mapreduce
- Scalding REPL
- Scalding Sources
- Scalding with CDH3U2 in a Maven project
- SQL to Scalding
- Type safe api reference
- Upgrading to 0.9.0
- Using scalding with other versions of scala
- Using the distributed cache
- Why pack unpack and not toList[]
- Show 23 more pages…
Contents
Getting help
Documentation
- Scaladocs
- Getting Started
- Type-safe API Reference
- SQL to Scalding
- Building Bigger Platforms With Scalding
- Scalding Sources
- Scalding-Commons
- Rosetta Code
- Fields-based API Reference (deprecated)
Matrix API
Third Party Modules
Videos
- Scalding: Powerful & Concise MapReduce Programming
- Scalding lecture for UC Berkeley's Analyzing Big Data with Twitter class
- Scalding REPL with Eclipse Scala Worksheets
How-tos
- Scalding with CDH3U2 in a Maven project
- Running your Scalding jobs in Eclipse
- Running your Scalding jobs in IDEA intellij
- Running Scalding jobs on EMR
- Running Scalding with HBase support: Scalding HBase wiki
- Using the distributed cache
- Unit Testing Scalding Jobs
- TDD for Scalding
- Using counters
Tutorials
- Scalding for the impatient
- Movie Recommendations and more in MapReduce and Scalding
- Generating Recommendations with MapReduce and Scalding
- Poker collusion detection with Mahout and Scalding
- Portfolio Management in Scalding
- Find the Fastest Growing County in US, 1969-2011, using Scalding
- Mod-4 matrix arithmetic with Scalding and Algebird
- Dean Wampler's Scalding Workshop
- Typesafe's Activator for Scalding
Articles
- Hive, Pig, Scalding, Scoobi, Scrunch and Spark: A Comparison of Hadoop Frameworks
- Why Hadoop MapReduce needs Scala
- How Twitter is doing its part to democratize big data
- Meet the combo powering Hadoop at Etsy, Airbnb and Climate Corp.
- Scalding wins a Bossie award from InfoWorld
Other
Clone this wiki locally
-
def config
in Job does not accept a mode. Job hasJob.mode
use that if you need it.def listeners
in Job no longer accepts a mode either. -
sum
takes a type parameter in the fields API.sum[Double]
is equivalent to the old behavior, but you might wantsum[Long]
or any otherT
that has analgebird.Semigroup[T]
. Without the type parameter you get:diverging implicit expansion for type com.twitter.algebird.Semigroup[T]
- TypedSink and Mappable need setter and converter defined. Using TupleSetter.asSubSetter, or TupleConverter.asSuperConverter can help here. (add better docs, please if you get confused).
- RichDate parsing needs an implicit scalding.DateParser. Job has one in scope that follows the old rules (minus natty), but you may need to set an implicit DateParser outside of a Job. (See DateParser.default).
-
JsonLine
has been extracted intoscalding-json
module.
These sed rules may help.