Create ThriftGemerator #7

Closed
wants to merge 9 commits into
from

Conversation

Projects
None yet
3 participants

This is incredibly ugly right now. I'm looking for feedback.

This is a gemerator for thrift gems. It'll create a gem directory structure and all the requisite files to get started on a gem. The thrift files will be ignored in git and regenerated every time the gem is built.

What's nice is, if you want to gemerate your service's thrift files just do the following:

class ThriftProject(info: ProjectInfo)
  extends StandardLibraryProject(info)
  with ThriftGemerator
{
  def gemFactory = new ThriftGemFactory(
    "gem-name",
    "GemNameSpace",
    "ThriftService",
    "Some nice description of what this is",
    Seq(("Your Name", "your@email.com")),
    "http://service-homepage.com"
  )
}

It also introduces the ability to exclude thrift IDLs from generation. Useful, for instance, when your service presents a scribe endpoint but the generated scribe files shouldn't be a part of the gem.

class ThriftProject(info: ProjectInfo)
  extends StandardLibraryProject(info)
  with ThriftGemerator
{
  def gemFactory = new ThriftGemFactory(
    "gem-name",
    "GemNameSpace",
    "ThriftService",
    "Some nice description of what this is",
    Seq(("Your Name", "your@email.com")),
    "http://service-homepage.com",
    Seq("scribe")
  )
}

Finally, there are two types of repositories that can be pushed to: Twitter internal and Rubygems. The default is Twitter interal. It can be changed simply by providing it:

class ThriftProject(info: ProjectInfo)
  extends StandardLibraryProject(info)
  with ThriftGemerator
{
  def gemFactory = new ThriftGemFactory(
    "gem-name",
    "GemNameSpace",
    "ThriftService",
    "Some nice description of what this is",
    Seq(("Your Name", "your@email.com")),
    "http://service-homepage.com",
    RubygemsGem
  )
}

Altogether now:

class ThriftProject(info: ProjectInfo)
  extends StandardLibraryProject(info)
  with ThriftGemerator
{
  def gemFactory = new ThriftGemFactory(
    "gem-name",
    "GemNameSpace",
    "ThriftService",
    "Some nice description of what this is",
    Seq(("Your Name", "your@email.com")),
    "http://service-homepage.com",
    Seq("scribe"),
    RubygemsGem
  )
}

@robey robey commented on an outdated diff Oct 4, 2011

src/main/scala/com/twitter/sbt/ThriftGemerator.scala
+ val url: String,
+ repository: GemRepository,
+ mainSourcePath: Path,
+ outputPath: Path,
+ generatedRubyFiles: scala.collection.Set[Path],
+ version: Version,
+ log: Logger
+) {
+ val basePath = mainSourcePath / "ruby" / name
+ val mainLibPath = basePath / "lib"
+ val libPath = mainLibPath / name
+ val thriftPath = libPath / "thrift"
+ val mainTestPath = basePath / "test"
+ val testPath = mainTestPath / name
+ val targetPath = outputPath / "gem"
+ val gemName = name+"-"+version.toString.replaceAll("-SNAPSHOT","")
@robey

robey Oct 4, 2011

(code style) and line 62, 110, 117, 225, 256, 264

@robey robey and 1 other commented on an outdated diff Oct 4, 2011

src/main/scala/com/twitter/sbt/ThriftGemerator.scala
+ FileUtilities.touch(mainTestPath / (name + "_test_helpers.rb"), log)
+ List("client_test.rb", "mock_service_test.rb") foreach { file =>
+ FileUtilities.touch(testPath / file, log)
+ }
+ }
+}
+
+trait GemRepository {
+ def release(gemName: String, path: Path, log: Logger): Option[String]
+}
+
+object TwitterGem extends GemRepository {
+ private[this] val uploadTemplate =
+ """require 'rubygems'
+ require 'ods_credentials'
+ = Class.new { extend ODSCredentials }.get_keychain_data.join(':').gsub('!', '\!')
@robey

robey Oct 4, 2011

what does it mean for a ruby line to begin with "="?

@sprsquish

sprsquish Oct 4, 2011

typo. I must've hit undo in my editor before committing.

@robey robey commented on an outdated diff Oct 4, 2011

src/main/scala/com/twitter/sbt/ThriftGemerator.scala
+
+object TwitterGem extends GemRepository {
+ private[this] val uploadTemplate =
+ """require 'rubygems'
+ require 'ods_credentials'
+ = Class.new { extend ODSCredentials }.get_keychain_data.join(':').gsub('!', '\!')
+ system("curl -H\"Expect: \" -F\"file=@#{ARGV[0]}\" -u#{creds} http://gems.local.twitter.com/upload")
+ """
+
+ def release(gemName: String, path: Path, log: Logger) = {
+ val uploadFile = (path / "upload.rb").asFile
+ if (!uploadFile.isFile)
+ FileUtilities.write(uploadFile, uploadTemplate, log)
+
+ val exitCode = (("ruby "+uploadFile+" "+(path / gemName).absolutePath) !)
+ if (exitCode == 0) None
@robey

robey Oct 4, 2011

if (_) {
...
} else {
...
}

also on line 265

@robey robey commented on an outdated diff Oct 4, 2011

src/main/scala/com/twitter/sbt/ThriftGemerator.scala
+trait GemRepository {
+ def release(gemName: String, path: Path, log: Logger): Option[String]
+}
+
+object TwitterGem extends GemRepository {
+ private[this] val uploadTemplate =
+ """require 'rubygems'
+ require 'ods_credentials'
+ = Class.new { extend ODSCredentials }.get_keychain_data.join(':').gsub('!', '\!')
+ system("curl -H\"Expect: \" -F\"file=@#{ARGV[0]}\" -u#{creds} http://gems.local.twitter.com/upload")
+ """
+
+ def release(gemName: String, path: Path, log: Logger) = {
+ val uploadFile = (path / "upload.rb").asFile
+ if (!uploadFile.isFile)
+ FileUtilities.write(uploadFile, uploadTemplate, log)
@robey

robey Oct 4, 2011

braces (or just put it on one line).

@robey robey commented on an outdated diff Oct 4, 2011

src/main/scala/com/twitter/sbt/ThriftGemerator.scala
@@ -0,0 +1,290 @@
+package com.twitter.sbt
+
+import _root_.sbt._
+import Process._
+
+class ThriftGemFactory(
@robey

robey Oct 4, 2011

there are so many ctor params, it's pretty foggy from context what they're used for.

i can think of a couple of ways to fix that:

  1. make a case class with good defaults and use that in the ctor, so most people will use the "name=value" form
  2. define these as undefined in the class, like "val service: String" so that they're required to be defined when constructed

robey commented Oct 4, 2011

i'm a big fan of this in general.

@sprsquish sprsquish cleanup based on feedback
ThriftGemFactory is now a trait with undefined vals
Fixed style issues
65ed949

Updated based on Robey's feedback. Used a trait with undefined vals for the factory.

class ThriftProject(info: ProjectInfo)
  extends StandardLibraryProject(info)
  with ThriftGemerator
{
  def gemFactory = new ThriftGemFactory {
    val name = "gem-name"
    val namespace = "GemNameSpace"
    val service = "ThriftService"
    val description = "Some nice description of what this is"
    val authors = Seq(("Your Name", "your@email.com"))
    val homepage = "http://service-homepage.com"
    override val thriftExclusions = Seq("scribe")
    override val repository = RubygemsGem
  }
}

sprsquish added some commits Oct 4, 2011

@sprsquish sprsquish add test-gem task b4adea5
@sprsquish sprsquish create a cleanup task for gemerator 2f8cf06
@sprsquish sprsquish Bugfix. ThriftGem was created with missing files
ThriftGem is instantiated before ruby files are generated
so the path was returning an empty set. Use PathFinder instead.
12918c6

9len commented Oct 4, 2011

how does this handle different thrift versions? The monorail is still on 0.2.0, but presumably we'd want to publish both 0.2.0 and 0.5.0 versions of things.

There are shims in the monorail so you can provide a thrift 5 gem. The thrift_client version is set as >= 5 which is bound to thrift 2 but wont get in your way if you use thrift_client 6. The downside to that is if you need to use the gem in a service that's bound to thrift 2 but doesn't have the shims. I'd be curious to know if there are other services aside from the monorail that are bound to thrift 2.

bump on this?

sprsquish closed this Sep 26, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment