Skip to content
Scala extensions for Storm
Scala Shell Java
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.
project
tormenta-avro/src
tormenta-core/src
tormenta-kafka/src/main/scala/com/twitter/tormenta/spout
tormenta-kestrel/src/main/scala/com/twitter/tormenta/spout
tormenta-twitter/src/main/scala/com/twitter/tormenta/spout
.gitignore
.travis.yml
CHANGES.md
LICENSE
NOTICE
README.md
build.sbt
sbt
version.sbt

README.md

Tormenta Build Status

Scala extensions for the Storm distributed computation system. Tormenta adds a type-safe wrapper over Storm's Kafka and Kestrel spouts. This type safety allows the user to push mapping and filtering transformations down to the level of the spout itself:

import com.twitter.tormenta.scheme._
import com.twitter.tormenta.spout._

// produces strings:
val scheme: Scheme[String] = Scheme { bytes => Some(new String(bytes)) }

// produces integers w/ string length:
val mappedScheme: Scheme[Int] = scheme.map(_.length)

// filters out all tuples less than 5:
val filteredScheme: Scheme[Int] = mappedScheme.filter(_ > 5)

// produces lengths for input strings > length of 5
val spout: KestrelSpout[Int] = new KestrelSpout(filteredScheme, hostSeq, "spout")

To use a Spout[T] in a Storm topology, call the getSpout method:

topologyBuilder.setSpout("spoutName", spout.getSpout, 10)

Now you're cooking with gas.

Community and Documentation

To learn more and find links to tutorials and information around the web, check out the Tormenta Wiki.

The latest ScalaDocs are hosted on Tormenta's Github Project Page.

Discussion occurs primarily on the Tormenta mailing list. Issues should be reported on the GitHub issue tracker.

Maven

Tormenta modules are available on Maven Central. The current groupid and version for all modules is, respectively, "com.twitter" and 0.11.1.

Current published artifacts are

  • tormenta-core_2.11
  • tormenta-core_2.10
  • tormenta-kafka_2.11
  • tormenta-kafka_2.10
  • tormenta-kestrel_2.11
  • tormenta-kestrel_2.10
  • tormenta-twitter_2.11
  • tormenta-twitter_2.10

The suffix denotes the scala version.

Authors

License

Copyright 2015 Twitter, Inc.

Licensed under the Apache License, Version 2.0: http://www.apache.org/licenses/LICENSE-2.0

Something went wrong with that request. Please try again.