Permalink
Browse files

updated gem to work with sbt 0.11 and scala 2.9.1

  • Loading branch information...
danielschonfeld
danielschonfeld committed Mar 25, 2012
1 parent 4550688 commit 2f19693fc7c9fe0cf2ec56badbcebf28ff914317
View
@@ -1,3 +1,7 @@
+This gem produces a skeleton finagled service directory to be used with the new SBT 0.11 and Scala 2.9.1
+ :$ scala-bootstrapper -s birdname
+- - -
+
This creates a standard environment for your twitter-centric sbt/scala thrift service.
Building:
View
@@ -1 +1 @@
-0.9.2
+0.9.3
View
@@ -8,6 +8,7 @@ opts = Trollop::options do
opt :public, "Use the public twitter maven repo"
opt :namespace, "Use something besides com.twitter", :type => :string
opt :git, "Use Git to track updates to generated files"
+ opt :sbt11, "Setup an SBT 0.11 and Finagle2 skeleton"
end
@@ -55,6 +56,8 @@ project_name = ARGV.pop.camelize(true)
is_public = opts[:public]
namespace = opts[:namespace] || "com.twitter"
git = opts[:git]
+sbt11_requested = opts[:sbt11]
+
$overwrite_all = true if git
$ex_post_facto = false
$branch = 'master'
@@ -84,7 +87,11 @@ if git
end
end
-root = File.expand_path(File.dirname(__FILE__) + "/../lib/template")
+if sbt11_requested
+ root = File.expand_path(File.dirname(__FILE__) + "/../lib/template2")
+else
+ root = File.expand_path(File.dirname(__FILE__) + "/../lib/template")
+end
Dir["#{root}/**/*"].select{|path| File.file?(path)}.each do |path|
relative = path.sub("#{root}/", "")
@@ -95,12 +102,16 @@ Dir["#{root}/**/*"].select{|path| File.file?(path)}.each do |path|
print "File exists `#{relative}`, replace? ([Y]es, [N]o, [A]ll, [Q]uit)"
$stdout.flush
case STDIN.gets
- when /^y/i: # continue
- when /^n/i: next
- when /^a/i: $overwrite_all = true
- when /^q/i: exit(2)
+ when /^y/i
+ #continue
+ when /^n/i
+ next
+ when /^a/i
+ $overwrite_all = true
+ when /^q/i
+ exit(2)
else
- retry
+ redo
end
end
puts "writing #{target_path}"
@@ -0,0 +1,10 @@
+*.class
+*.log
+
+# sbt specific
+dist/*
+target/
+lib_managed/
+src_managed/
+project/boot/
+project/plugins/project/
View
@@ -0,0 +1,27 @@
+# Docs at http://confluence.local.twitter.com/display/RELEASE/Twitter-cap-utils+README
+begin
+ require 'rubygems'
+ require "bundler/setup"
+ require "railsless-deploy"
+ require 'twitter_cap_utils'
+rescue LoadError => e
+ puts e.message
+ abort "Please gem install twitter-cap-utils railsless-deploy"
+end
+
+set :user, :twitter
+set :application, "birdname"
+set :repository, "http://git.local.twitter.com/ro/#{application}"
+
+task :staging do
+ role :app, "server1", "server2", "etc"
+end
+
+task :canary do
+ role :app, "server1"
+end
+
+task :production do
+ role :app, "server1", "server2", "etc"
+end
+
View
@@ -0,0 +1,7 @@
+# use "bundle install" to update gems; "gem install bundler" to install bundler.
+source :rubygems
+source "http://gems.local.twitter.com"
+gem "thrift_client", "0.6.2"
+gem "thrift", "0.6"
+gem "railsless-deploy"
+gem "capistrano"
View
@@ -0,0 +1,24 @@
+# Project BirdName
+
+Welcome to your birdname project! To make sure things are working
+properly, you may want to:
+
+ $ sbt update test
+
+There is a tutorial for what to do next, which you can find in the
+scala-bootstrapper README.rdoc file.
+
+# Configuring Intellij
+
+If you want to setup Intellij, it has to happen off to the side:
+
+ $ sbt
+ > *sbtIdeaRepo at http://mpeltonen.github.com/maven/
+ > *idea is com.github.mpeltonen sbt-idea-processor 0.4.0
+ > update
+ > idea
+
+# Documenting your project
+
+Add documentation here! Eventually, you'll be able to publish this to
+a web site for the world to easily find and read.
View
@@ -0,0 +1,187 @@
+# Welcome to BirdName!
+
+## Setup
+
+Scala-bootstrapper has created a fully-functional Scala service for
+you. You can verify that things are set up correctly by doing:
+
+ $ sbt update test
+
+## Tutorial
+
+### Run your service!
+
+There are two ways to start your service. You can build a runnable
+jar and tell java to run it directly:
+
+ $ sbt package-dist
+ $ java -Dstage=development -jar ./dist/birdname/birdname-1.0.0-SNAPSHOT.jar
+
+or you can ask sbt to run your service:
+
+ $ sbt 'run -f config/development.scala'
+
+### Verify that the service is running
+
+The java/sbt command-lines will "hang" because the server is running in the
+foreground. (In production, we use libslack-daemon to wrap java processes into
+unix daemons.) Go to another terminal and check for a logfile. If your server
+is named "birdname", there should be a `birdname.log` with contents like this:
+
+ INF [20110615-14:05:41.656] stats: Starting JsonStatsLogger
+ INF [20110615-14:05:41.674] admin: Starting TimeSeriesCollector
+ DEB [20110615-14:05:41.792] nio: Using the autodetected NIO constraint level: 0
+
+That's your indication that the server is running. :)
+
+### View the Thrift IDL for your service
+
+The IDL for your service is in `src/main/thrift/birdname.thrift`. The
+Thrift compiler uses the IDL to generate bindings for various
+languages, making it easy for scripts in those languages to talk to
+your service. More information about Thrift and how to write an IDL
+for your service can be found [here](http://wiki.apache.org/thrift/Tutorial).
+
+### Call your service from ruby
+
+Your service implements simple get() and put() methods. Once you have
+your server running, as above, bring up a different shell and:
+
+ $ cd birdname
+ $ bundle install
+ $ ./dist/birdname/scripts/console
+ >> $client
+ >> $client.put("key1", "valueForKey")
+ >> $client.get("key1")
+
+### Look at the stats for your service
+
+By default, your project is configured to use
+[Ostrich](https://github.com/twitter/ostrich), a library for service
+configuration, administration, and stats reporting. Your config file
+in `config/development.scala` defines which port ostrich uses for admin
+requests. You can view the stats via that port:
+
+ $ curl localhost:9900/stats.txt
+ counters:
+ BirdName/connects: 1
+ BirdName/requests: 2
+ BirdName/success: 2
+ ...
+
+Ostrich also stores historial stats data and can build
+[graphs](http://localhost:9900/graph/) for you.
+
+### Stop the service
+
+You can ask the server to shutdown over the admin port also:
+
+ $ curl localhost:9900/shutdown.txt
+ ok
+
+### View the implementation of get() and put()
+
+In `src/main/scala`, take a look at `BirdNameServiceImpl.scala`. (This may
+have a different name, based on what you called your server.)
+
+The base interface is specified by thrift. Additionally, we're using Twitter's
+async I/O framework: finagle. Finagle (and a lot of great documentation about
+it) is hosted here: https://github.com/twitter/finagle
+
+### Try adding some timers and counters
+
+At the top of BirdNameServiceImpl.scala, add:
+
+ import com.twitter.ostrich.stats.Stats
+
+Then inside get():
+
+ Stats.incr("birdname.gets")
+
+and inside put():
+
+ Stats.incr("birdname.puts")
+
+Then restart your server, talk to the server via console, and check
+your stats:
+
+ $ curl localhost:9900/stats.txt
+ counters:
+ BirdName/connects: 1
+ BirdName/requests: 2
+ BirdName/success: 2
+ birdname.gets: 1
+ birdname.puts: 1
+
+You can also time various things that your server is doing, for
+example:
+
+ Stats.time("birdname.put.latency") {
+ Thread.sleep(10) // so you can see it
+ database(key) = value
+ }
+
+### Specs: let's add some tests
+
+[Specs](http://code.google.com/p/specs/) is a Behavior-Driven Design
+framework that allows you to write semi-human-readable descriptions of
+how your service should behave and test that those descriptions are
+valid. You already have some Specs code for your project in
+src/test/scala/com/twitter/birdname/BirdNameServiceSpec.scala. Check
+out the existing test and add a new one for the counter functionality
+we just added.
+
+ import com.twitter.ostrich.stats.Stats
+
+ ...
+
+ "verify stats" in {
+ val counters = Stats.getCounters
+ foofa.put("name", "bluebird")()
+ foofa.get("name")() mustEqual "bluebird"
+ counters.getOrElse("foofa.gets", 1) must_==1
+ counters.getOrElse("foofa.puts", 1) must_==1
+ }
+
+TODO: add link to scala school lesson on Specs
+
+### Automatically compile and test your server when you change code
+
+By now you've had to Ctrl-C your server and restart it to get changes
+to show up. This gets a little tiresome. The build tool we are
+using,
+[SBT (simple build tool)](http://code.google.com/p/simple-build-tool/)
+has a console that you can access by just running "sbt" from the
+command line.
+
+ $ sbt
+ [info] Standard project rules 0.11.4 loaded (2011-03-18).
+ [warn] No .svnrepo file; no svn repo will be configured.
+ [info] Building project birdname 1.0.0-SNAPSHOT against Scala 2.8.1
+ [info] using BirdNameProject with sbt 0.7.4 and Scala 2.7.7
+
+SBT has a wide array of features, but a useful one right now is to
+use the "~ test" command.
+
+ > ~ test
+
+The tilde tells SBT to look for changes to your source files and
+re-execute the command when it detects a change.
+
+TODO: add link to scala school lesson on SBT
+
+### Add an admin / dashboard page.
+
+### Add a new dependency to your project, perhaps twitter/util?
+
+### Take a tour of the logs our service is producing.
+
+### Add command-line parameters for your service.
+-D foo=bar
+runtime.arguments.get("foo")
+
+### Storage: let's persist the data in Cassandra!
+
+### Twitter API: let's listen to the Firehose!
+
+### Twitter API: let's fetch some statuses & users & stuff.
@@ -0,0 +1,22 @@
+import com.twitter.sbt._
+
+seq(CompileThriftScrooge.newSettings: _*)
+
+resolvers += "twitter-repo" at "http://maven.twttr.com"
+
+libraryDependencies ++= Seq(
+ "org.scala-lang" % "jline" % "2.9.1",
+ "thrift" % "libthrift" % "0.5.0" from "http://maven.twttr.com/thrift/libthrift/0.5.0/libthrift-0.5.0.jar",
+ "org.apache.thrift" % "libthrift" % "0.5.0" intransitive,
+ "com.twitter" %% "scrooge-runtime" % "1.1.3",
+ "com.twitter" %% "finagle-core" % "2.0.1",
+ "com.twitter" %% "finagle-thrift" % "2.0.1",
+ "com.twitter" %% "finagle-ostrich4" % "2.0.1"
+)
+
+mainClass in (Compile, run) := Some("com.twitter.birdname.Main")
+
+mainClass in (Compile, packageBin) := Some("com.twitter.birdname.Main")
+
+
+
@@ -0,0 +1,41 @@
+import com.twitter.conversions.time._
+import com.twitter.logging.config._
+import com.twitter.ostrich.admin.config._
+import com.twitter.birdname.config._
+
+// development mode.
+new BirdNameServiceConfig {
+
+ // Add your own config here
+
+ // Where your service will be exposed.
+ thriftPort = 9999
+
+ // Ostrich http admin port. Curl this for stats, etc
+ admin.httpPort = 9900
+
+ // End user configuration
+
+ // Expert-only: Ostrich stats and logger configuration.
+
+ admin.statsNodes = new StatsConfig {
+ reporters = new TimeSeriesCollectorConfig
+ }
+
+ loggers =
+ new LoggerConfig {
+ level = Level.DEBUG
+ handlers = new FileHandlerConfig {
+ filename = "birdname.log"
+ roll = Policy.SigHup
+ }
+ } :: new LoggerConfig {
+ node = "stats"
+ level = Level.INFO
+ useParents = false
+ handlers = new FileHandlerConfig {
+ filename = "stats.log"
+ formatter = BareFormatterConfig
+ }
+ }
+}
Oops, something went wrong.

0 comments on commit 2f19693

Please sign in to comment.