Permalink
Switch branches/tags
Nothing to show
Find file
Fetching contributors…
Cannot retrieve contributors at this time
123 lines (95 sloc) 3.95 KB
layout title description date author keywords categories
post
Rails development with service fakes
For this blog post, we wanted to show how we swap out the real S3 tool for something that works better for local development.
2013-11-18
Ketan Padegaonkar
snap ci, continuous delivery, continuous integration, developer tools, github, heroku, rails, service fakes
hacks ruby

Snap talks to quite a few external services for its workings- GitHub, Heroku, S3 etc. Talking to all these external services is tricky while doing local development. They are slow even while on fast connections. You cannot work when disconnected or on a poor network connection. Plus- hammering them with unit tests is usually frowned upon by the service providers. ;)

Given these problems, most people try to stub these external services for local development. There are probably as many ways of stubbing external services as there are people. We wanted to discuss how we did this- just in case any of you found it useful- or wanted to tell us about other ways that you do it!

Stubbing S3

For this blog post, we wanted to show how we swap out the real S3 tool for something that works better for local development.

Snap uses S3 to store artifacts generated by your build.

For the most simple of cases -- we need to push a directory to S3 when your build completes and serve it out (through an authentication proxy) to users when they request for it.

For starters we need two different interfaces that can put and get from S3, so here's something that we put together:

Here's an implementation that talks to s3 via s3cmd

# Knows how to perform s3 operations against a real s3 bucket
module S3
  class RealS3
    class << self
      def get(src, dest)
        sh("s3cmd get #{Config.s3_bucket}/#{src} #{dest}")
      end

      def put(src, dest)
        sh("s3cmd put #{src} #{Config.s3_bucket}/#{dest}")
      end

      def rm(path)
        sh("s3cmd rm #{path}")
      end
    end  
  end
end

And here's a completely fake implementation that pretends like s3 is a local filesystem :)

# Pretends that the local file-system is s3
module S3
  class FakeS3
    class << self
      def get(src, dest)
        fake_src = File.join(Config.s3_fake_path, src)
        FileUtils.cp_rf(fake_src, dest)
      end

      def put(src, dest)
        fake_dest = File.join(Config.s3_fake_path, dest)
        FileUtils.cp_rf(src, fake_dest)
      end

      def rm(path)
        fake_path = File.join(Config.s3_fake_path, path)
        FileUtils.rm_rf(fake_path)
      end
    end
  end
end  

Now that we defined our two implementations -- one that talks to s3, and another that works off a file system. How do you load a particular implementation?

module S3
  # Since classes are constants, we define a constant that points to either RealS3 or FakeS3
  API = CustomDeletate.load('S3')
end

# To use the S3::API
S3::API.put('/path/to/local/artifact', 'location/in/s3')

# Knows how to load a constant specified in config/stubs.yml
class CustomDelegate
  class << self
    def load(delegate_name)
      all_stubs = YAML.load(File.read(File.join(Rails.root, 'config', 'stubs.yml')))
      stubs_for_current_environment = all_stubs[Rails.env]
      delegate_class_name = stubs_for_current_environment[delegate_name]
      delegate_class = delegate_class_name.constantize
      delegate_class
    end
  end
end

Our stubs.yml fle contains all the stubs that we would need to work with.

# config/stubs.yml
# Here we define the various stubs to 3rd party integrations and
# endpoints that we can now use for local development
production:
  "S3": "S3::RealS3"
  "Github": "Github::RealGithub"
development:
  "S3": "S3::FakeS3"
  "Github": "Github::FakeGithub"
test:
  "S3": "S3::FakeS3"
  "Github": "Github::FakeGithub"

Snap CI © 2017, ThoughtWorks