Browse files

ready for merge into master

Added Xvfb support.
Updated readme.
Set default instance type to c1.medium
  • Loading branch information...
1 parent 049a2b8 commit b2f56a7de9bf57db25e7cdb7f26974029465c991 Doc Ritezel & Matthew Kocher committed Oct 28, 2012
View
120 README.md
@@ -9,17 +9,17 @@ Lando Calrissian relies on Lobot to keep Cloud City afloat, and now you can rely
# What do I get?
-* Rake tasks for creating, starting, stopping, or destroying your CI server on EC2
-* Capistrano tasks for bootstrapping the Centos EC2 instance
-* Chef recipes for configuring the instance to run Jenkins and build Rails projects
+* Commands for creating, starting, stopping, or destroying your CI server on EC2
+* The full [Travis CI](http://travis-ci.org) environment on Ubuntu 12.04
+* A Jenkins frontend for monitoring your builds
-After you add lobot to your Gemfile, all you'll need to do is run the following commands:
+After you add `gem "lobot"` to your Gemfile, all you'll need to do is run the following commands:
- rails g lobot:install
- rails g lobot:config
- rake ci:create_server
- cap ci bootstrap
- cap ci chef
+ [lobot configure] - COMING SOON - See Setup for now.
+ lobot create
+ lobot bootstrap
+ lobot add_build BobTheBuild git@github.com:you/some_repo.git master script/ci_build.sh
+ lobot chef
Read on for an explanation of what each one of these steps does.
@@ -29,88 +29,68 @@ Add lobot to your Gemfile, in the development group:
gem "lobot", :group => :development
-### To Install
-
- rails g lobot:install
-
## Setup
-You can use a generator to interactively generate the config/ci.yml that includes some reasonable defaults and explanations.
-
- rails g lobot:config
-Alternatively, manually edit config/ci.yml
+Create a config/lobot.yml file in your project:
---
- app_name: # a short name for your application
- git_location: # The location of your remote git repository which Jenkins will poll and pull from on changes.
- basic_auth:
- - username: # The username you will use to access the Jenkins web interface
- password: # The password you will use to access the Jenkins web interface
- credentials:
- aws_access_key_id: # The Access Key for your Amazon AWS account
- aws_secret_access_key: The Secret Access Key for your Amazon AWS account
- provider: AWS # leave this one alone
- server:
- name: run 'rake ci:create_server to populate'
- instance_id: run 'rake ci:create_server to populate'
- build_command: ./script/ci_build.sh
- ec2_server_access:
- key_pair_name: myapp_ci
- id_rsa_path: ~/.ssh/id_rsa
- github_private_ssh_key_path: ~/.ssh/id_rsa
+ server_ssh_key: ~/.ssh/id_rsa
+ github_ssh_key: ~/.ssh/id_rsa
+ aws_key: <your AWS Key>
+ aws_secret: <your AWS Secret>
+ node_attributes:
+ nginx:
+ basic_auth_user: ci
+ basic_auth_password: password
+
+See [https://aws-portal.amazon.com/gp/aws/developer/account/index.html?ie=UTF8&action=access-key](https://aws-portal.amazon.com/gp/aws/developer/account/index.html?ie=UTF8&action=access-key) to generate AWS key/secret.
## Adjust Defaults (Optional)
-In your ci.yml, there are defaults set for values that have the recommened value. For example, the instance size used for EC2 is set to "c1.medium", which costs $118/month.
+
+In your config/lobot.yml, there are defaults set for values that have the recommened value. For example, the instance size used for EC2 is set to "c1.medium".
+
You can save on EC2 costs by using a tool like [projectmonitor](https://github.com/pivotal/projectmonitor) or ylastic to schedule when your instances are online.
## Commit and push your changes
At this point you will need to create a commit of the files generated or modified and push those changes to your remote git repository so Jenkins can execute the build script when it pulls down your repo for the first time.
-## Modify the soloistrc if necessary
+If you must, you can do this on a branch. Then later you can change the branch in lobot.yml later and rechef.
+
+## Modify recipe list
+
+You can modify the chef run list by setting the `recipes` key in config/lobot.yml. The default is:
+
+ ["pivotal_ci::jenkins", "pivotal_ci::limited_travis_ci_environment", "pivotal_ci"]`
-The soloistrc in root of your project is where you tell chef which recipes to run. Because we're using the cookbooks from Travis CI, you can look through [all the recipes Travis has available](https://github.com/travis-ci/travis-cookbooks/), and add any of them to your soloistrc.
+Because we're using the cookbooks from Travis CI, you can look through [all the recipes Travis has available](https://github.com/travis-ci/travis-cookbooks/), and add any that you need.
## Starting your lobot instance
-1. Launch an instance, allocate and associates an elastic IP and updates ci.yml:
+1. Launch an instance, allocate and associates an elastic IP and updates config/lobot.yml:
- rake ci:create_server
+ lobot create
-2. Bootstrap the instance using the boostrap_server.sh script generated by Lobot. The script installs ruby prerequisites, creates the app_user account, and installs RVM for that user:
+2. Bootstrap the instance using the boostrap_server.sh script. The script installs ruby prerequisites and installs RVM:
- cap ci bootstrap
+ lobot bootstrap
-3. Upload the contents of your chef/cookbooks/ directory, upload the soloistrc, and run chef:
+3. Upload the contents of Lobot's cookbooks, create a soloistrc, and run chef:
- cap ci chef
+ lobot chef
Your lobot instance should now be up and running. You will be able to access your CI server at: http://&lt;your instance address&gt;/ with the username and password you chose during configuration.
-For more information about Jenkins CI, see http://jenkins-ci.org/
+For more information about Jenkins CI, see [http://jenkins-ci.org](http://jenkins-ci.org).
## Troubleshooting
Shell access for your instance
- rake ci:ssh
+ lobot ssh
-Terminating your instance and deallocating the elastic IP
+Terminate all Lobot instances on your account and deallocate their elastic IPs
- rake ci:destroy_server
-
-Suspending your instance
-
- rake ci:stop_server
-
-Rstarting a server
-
- rake ci:start_server
-
-## Add your new CI instance to [projectmonitor](http://github.com/pivotal/projectmonitor) and CCMenu
-
-Lobot can generate the config for you, just run:
-
- rake ci:info
+ lobot destroy_ec2
## Color
@@ -125,17 +105,17 @@ end
## Dependencies
* fog
-* capistrano
-* capistrano-ext
-* rvm (the gem - it configures capistrano to use RVM on the server)
-* nokogiri
+* ci_reporter
+* thor
+* hashie
+* net-ssh
+
+## Testing
-# Tests
+Lobot is tested using rspec, vagrant and test kitchen. You will need to set environment variables with your AWS credentials to run tests which rely on ec2:
-Lobot is tested using rspec, generator_spec and cucumber. Cucumber provides a full integration test which can generate a rails application, push it to github, start a server and bring up CI for the generated project.
-You'll need a git repository (which should not have any code you care about) and an AWS account to run the whole test suite. It costs about $0.50 and takes about half an hour.
-It will attempt to terminate the server however you should verify this so you do not get charged additional money.
-Use the secrets.yml.example to create a secrets.yml file with your account information.
+ export EC2_KEY=FOO
+ export EC2_SECRET=BAR
# Contributing
View
4 chef/cookbooks/pivotal_ci/files/default/tests/minitest/default_test.rb
@@ -10,6 +10,10 @@
it "creates the jenkins user" do
user("jenkins").must_exist
end
+
+ it "configures the xvfb plugin" do
+ file("/var/lib/jenkins/org.jenkinsci.plugins.xvfb.XvfbBuildWrapper.xml").must_exist
+ end
end
describe_recipe "pivotal_ci::default" do
View
8 chef/cookbooks/pivotal_ci/recipes/jenkins_config.rb
@@ -15,12 +15,18 @@
jenkins_plugin.call(self, "lobot", "http://cheffiles.pivotallabs.com/lobot/lobot.hpi")
end
-['git', 'ansicolor'].each do |plugin|
+['git', 'ansicolor', 'xvfb'].each do |plugin|
execute "download #{plugin} plugin" do
jenkins_plugin.call(self, plugin, "http://mirrors.jenkins-ci.org/plugins/#{plugin}/latest/#{plugin}.hpi")
end
end
+template "#{node["jenkins"]["home"]}/org.jenkinsci.plugins.xvfb.XvfbBuildWrapper.xml" do
+ source "org.jenkinsci.plugins.xvfb.XvfbBuildWrapper.xml.erb"
+ owner "jenkins"
+ notifies :restart, "service[jenkins]"
+end
+
node["jenkins"]["builds"].each do |build|
directory "#{node["jenkins"]["home"]}/jobs/#{build['name']}" do
owner "jenkins"
View
8 chef/cookbooks/pivotal_ci/templates/default/jenkins-job-config.xml.erb
@@ -65,5 +65,13 @@
<hudson.plugins.ansicolor.AnsiColorBuildWrapper>
<colorMapName>xterm</colorMapName>
</hudson.plugins.ansicolor.AnsiColorBuildWrapper>
+ <org.jenkinsci.plugins.xvfb.XvfbBuildWrapper plugin="xvfb@1.0.4">
+ <installationName>system</installationName>
+ <screen>1024x768x24</screen>
+ <debug>false</debug>
+ <timeout>0</timeout>
+ <displayNameOffset>0</displayNameOffset>
+ <additionalOptions></additionalOptions>
+ </org.jenkinsci.plugins.xvfb.XvfbBuildWrapper>
</buildWrappers>
</project>
View
11 ...ookbooks/pivotal_ci/templates/default/org.jenkinsci.plugins.xvfb.XvfbBuildWrapper.xml.erb
@@ -0,0 +1,11 @@
+<?xml version='1.0' encoding='UTF-8'?>
+<org.jenkinsci.plugins.xvfb.XvfbBuildWrapper_-XvfbBuildWrapperDescriptor plugin="xvfb@1.0.4">
+ <helpRedirect/>
+ <installations>
+ <org.jenkinsci.plugins.xvfb.XvfbInstallation>
+ <name>system</name>
+ <home>/usr/bin</home>
+ <properties/>
+ </org.jenkinsci.plugins.xvfb.XvfbInstallation>
+ </installations>
+</org.jenkinsci.plugins.xvfb.XvfbBuildWrapper_-XvfbBuildWrapperDescriptor>
View
2 chef/cookbooks/pivotal_ci/test/kitchen/Kitchenfile
@@ -1,4 +1,4 @@
cookbook "pivotal_ci" do
configuration "default"
- lint(:ignore => ["FC007"])
+ lint(:ignore => ["FC007", "FC009"])
end
View
3 chef/cookbooks/pivotal_ci/test/kitchen/cookbooks/pivotal_ci_test/attributes/default.rb
@@ -1,2 +1,3 @@
node['nginx']['basic_auth_user'] = "wheelbarrow"
-node['nginx']['basic_auth_password'] = "full-of-garbage"
+node['nginx']['basic_auth_password'] = "full-of-garbage"
+node['jenkins']['builds'] = []
View
2 lib/lobot/config.rb
@@ -8,7 +8,7 @@ class Config < Hashie::Dash
property :instance_id
property :master
- property :instance_size, :default => 'm1.medium'
+ property :instance_size, :default => 'c1.medium'
property :ssh_port, :default => 22
property :server_ssh_key, :default => File.expand_path("~/.ssh/id_rsa")
property :github_ssh_key, :default => File.expand_path("~/.ssh/id_rsa")
View
6 lib/lobot/sobo.rb
@@ -15,6 +15,12 @@ def timeout
end
def exec(command)
+ Net::SSH.start(ip, "ubuntu", :keys => [key], :timeout => timeout) do |ssh|
+ ssh.exec(command)
+ end
+ end
+
+ def exec!(command)
output = nil
Net::SSH.start(ip, "ubuntu", :keys => [key], :timeout => timeout) do |ssh|
output = ssh.exec!(command)
View
12 spec/lib/lobot/cli_spec.rb
@@ -174,9 +174,9 @@ def action
it "installs all necessary packages, installs rvm and sets up the user" do
cli.bootstrap
- sobo.exec("dpkg --get-selections").should include("libncurses5-dev")
- sobo.exec("ls /usr/local/rvm/").should_not be_empty
- sobo.exec("groups ubuntu").should include("rvm")
+ sobo.exec!("dpkg --get-selections").should include("libncurses5-dev")
+ sobo.exec!("ls /usr/local/rvm/").should_not be_empty
+ sobo.exec!("groups ubuntu").should include("rvm")
end
end
@@ -199,9 +199,9 @@ def action
cli.lobot_config.recipes = ["pivotal_ci::jenkins", "pivotal_ci::id_rsa", "pivotal_ci::git_config", "sysctl", "pivotal_ci::jenkins_config"]
cli.chef
- sobo.exec("ls /var/lib/").should include "jenkins"
- sobo.exec("grep 'kernel.shmmax=' /etc/sysctl.conf").should_not be_empty
- sobo.exec("sudo cat /var/lib/jenkins/.ssh/id_rsa").should == File.read(lobot_config.github_ssh_key)
+ sobo.exec!("ls /var/lib/").should include "jenkins"
+ sobo.exec!("grep 'kernel.shmmax=' /etc/sysctl.conf").should_not be_empty
+ sobo.exec!("sudo cat /var/lib/jenkins/.ssh/id_rsa").should == File.read(lobot_config.github_ssh_key)
godot.wait!
godot.match!(/Bob/, 'api/json')
View
2 spec/lib/lobot/config_spec.rb
@@ -58,7 +58,7 @@
its(:github_ssh_key) { should =~ /id_rsa$/ }
its(:recipes) { should == ["pivotal_ci::jenkins", "pivotal_ci::limited_travis_ci_environment", "pivotal_ci"] }
its(:cookbook_paths) { should == ['./chef/cookbooks/', './chef/travis-cookbooks/ci_environment'] }
- its(:instance_size) { should == 'm1.medium' }
+ its(:instance_size) { should == 'c1.medium' }
describe "#node_attributes" do
it "defaults to overwriting the travis build environment" do

0 comments on commit b2f56a7

Please sign in to comment.