Skip to content
Scala extensions for Storm
Scala Shell Java
Branch: develop
Clone or download
alanbato Merge pull request #77 from twitter/add_tls_to_urls
Use https instead of http in urls
Latest commit 988bbe5 Jun 7, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
project Replace http with https in files Jun 6, 2019
tormenta-avro/src Use Injection instead of Bufferable Nov 21, 2016
tormenta-core/src Minor: imports code style Jan 17, 2017
tormenta-kafka/src/main/scala/com/twitter/tormenta/spout Upgrade to Storm API 1.0.2 Nov 21, 2016
tormenta-twitter/src/main/scala/com/twitter/tormenta/spout Upgrade to Storm API 1.0.2 Nov 21, 2016
.gitignore Add IDEA's project files to .gitignore Jan 17, 2017
.travis.yml
CHANGES.md Update changes, readme and version Jan 17, 2017
COMMITTERS.md
LICENSE Get ready for release. Oct 16, 2012
NOTICE Get ready for release. Oct 16, 2012
README.md Update changes, readme and version Jan 17, 2017
build.sbt Setting version to 0.12.1-SNAPSHOT (#76) Jan 18, 2017
sbt Replace http with https in files Jun 6, 2019
version.sbt Setting version to 0.12.1-SNAPSHOT (#76) Jan 18, 2017

README.md

Tormenta

Build Status Codecov branch Latest version Chat

Scala extensions for the Storm distributed computation system. Tormenta adds a type-safe wrapper over Storm's Kafka spout. This type safety allows the user to push mapping and filtering transformations down to the level of the spout itself:

import com.twitter.tormenta.scheme._
import com.twitter.tormenta.spout._

// produces strings:
val scheme: Scheme[String] = Scheme { bytes => Some(new String(bytes)) }

// produces integers w/ string length:
val mappedScheme: Scheme[Int] = scheme.map(_.length)

// filters out all tuples less than 5:
val filteredScheme: Scheme[Int] = mappedScheme.filter(_ > 5)

// produces lengths for input strings > length of 5
val spout: KafkaSpout[Int] = new KafkaSpout(filteredScheme, zkHost, brokerZkPath, topic, appID, zkRoot)

To use a Spout[T] in a Storm topology, call the getSpout method:

topologyBuilder.setSpout("spoutName", spout.getSpout, 10)

Now you're cooking with gas.

Documentation

To learn more and find links to tutorials and information around the web, check out the Tormenta Wiki.

The latest ScalaDocs are hosted on Tormenta's Github Project Page.

Contact

Discussion occurs primarily on the Tormenta mailing list. Issues should be reported on the GitHub issue tracker.

Get Involved + Code of Conduct

Pull requests and bug reports are always welcome!

We use a lightweight form of project governence inspired by the one used by Apache projects. Please see Contributing and Committership for our code of conduct and our pull request review process. The TL;DR is send us a pull request, iterate on the feedback + discussion, and get a +1 from a Committer in order to get your PR accepted.

The current list of active committers (who can +1 a pull request) can be found here: Committers

A list of contributors to the project can be found here: Contributors

Maven

Tormenta modules are available on Maven Central. The current groupid and version for all modules is, respectively, "com.twitter" and 0.12.0.

Current published artifacts are

  • tormenta-core_2.12
  • tormenta-core_2.11
  • tormenta-core_2.10
  • tormenta-kafka_2.12
  • tormenta-kafka_2.11
  • tormenta-kafka_2.10
  • tormenta-twitter_2.12
  • tormenta-twitter_2.11
  • tormenta-twitter_2.10

The suffix denotes the scala version.

Authors

License

Copyright 2017 Twitter, Inc.

Licensed under the Apache License, Version 2.0.

You can’t perform that action at this time.