Continuous Integration Bootstrapper
Getting a fresh Jenkins server set up is kind of a pain and because of that, fewer projects have one than should. This project is designed to make it easy for people to fork and get set up with their very-own Jenkins server in the cloud because CI is awesome.
Right now it only supports Jenkins on Ubuntu 10.04 on Amazon EC2, but pull requests are welcome.
How does this work?
ci-infrastructure uses Chef driven via littlechef to make the setup as painless as possible. You fork this project, configure your EC2 credentials, configure your SSH access and then you're a couple of commands away from a maintanable CI setup that you can use to make your software more awesome.
1. Fork and Clone This Repository
It's not required, but it's recommended that you fork this repository. This allows you to make customizations and actually commit them in to source control, which is a good thing. It also means you're only a little bit of work from submitting pull requests to make this project better and remove some manual work.
$ git clone --recursive git://github.com/winhamwr/ci-infrastructure.git
The --recursive is important, you will regret not having done this later.
2. Install Chef
Because of some recent changes to the way Chef uses ruby vs json metadata files, we now need to install Chef and Knife locally for most cookbooks to work. It's annoying, but hopefully will be solved by littlechef another way. See this issue for details.
The official chef installation docs are the place to go if anything goes wrong, but on Ubuntu, assuming you already have rubygems, you can just run:
$ gem install chef --no-ri --no-rdoc
3. Install the Python Requirements
Littlechef is a python project that uses Fabric under the hood to allow most of the power that comes with a chef server only without the overhead of configuring a server. If you're not familiar with python, you'll want to read a quick tutorial on Pip and virtualenv to make your life easier (they're the rough python equivelents to RVM/Bundler/NPM etc). I recommend this pip + virtualenv introduction.
Alternatively, you can just run these commands.
$ easy_install pip $ pip install virtualenv $ mkdir ~/.virtualenvs $ virtualenv ~/.virtualenvs/ci-infrastructure $ pip install -r path/to/ci-infrastructure/requirements.txt
Note: if you close your shell, you must run the following command to switch to the ci-infrastructure virtual environment:
$ source ~/.virtualenvs/ci-infrastructure/bin/activate
4. Launch Your EC2 Node
Note: I'd prefer to make this possible with a one-command solution via this project, but for now, these are manual steps. Pull requests very welcome.
- Sign in to the AWS console
- Use the 32-bit instance root store Ubuntu 10.04 AMI in your preferred region from this list to launch your EC2 instance. Click the AMI links to redirect to the console launch screen for a handy shortcut. I recommend the High-CPU Medium instance.
- Use the console to create a new 20GB EBS volume in the same availability zone as your freshly-launched instance.
- Attach the new EBS volume to your instance as device
/dev/sdj(or whatever you'd like).
- Optionally, create and attach an elastic IP and point a DNS entry you control at that IP. eg. jenkins.yourproject.com
5. Configure SSH Authentication
Littlechef needs to know how to authenticate to your EC2 instance and gives you
the ability to set a username, password and private key. My preference though,
is to just use your existing
~/.ssh/config file since that puts connecting to
your instances on the same level as all of the other SSH work you do. It also
means that if you know how to use an SSH config, you can do other cool stuff.
Because of this, the auth.cfg that comes with this project just points to your
existing SSH configuration file.
There's an example bit of configuration located at
ssh_config.example to get
you started. You can concatenate the example to your
~/.ssh/config, edit a
few values and then keep rolling.
$ cat ssh_config.example >> ~/.ssh/config
Then open your
~/.ssh/config in your favorite text editor and change the
ec2_public_dnsshould be whatever URL you're using to reach your instance (for example, ec2-XX-XX-XX-XX.compute-1.amazonaws.com)
ec2_private_keyis the path to the AWS key pair file you used to create the instance (usually .pem file)
Ensure that your security group allows SSH access from 0.0.0.0. If your SSH wasn't open, you should also open port 80 so you can hit Jenkins from the internet.
When you've done this step successfully, you should be able to run:
$ ssh ec2-XX-XX-XX-XX.compute-1.amazonaws.com
6. Configure Littlechef for Your Instance
Now it's time to create a
node configuration for our EC2 instance so that
Littlechef knows where to go and what to do. Replace
ec2_public_dns with your
actual value of course:
$ cat nodes/example.json > nodes/ec2_public_dns.json
Now open that file and replace
ec2_public_dns with the appropriate value. If
you used something other than
/dev/sdj for your EBS volume mountpoint, you'll
want to change that also.
7. Put Littlechef to Work
Now it's time to let Littlechef mount your EBS volume, install and configure Jenkins and then put an Nginx reverse-proxy in front of Jenkins.
$ fix node:ec2_public_dns deploy_chef $ fix node:ec2_public_dns
Protip: If you get
Fatal error: Cookbook "jenkins" has no metadata.json
then you probably forgot to use the
--recursive flag when cloning the repo,
silly instruction skipper. You don't have the git submodules! Despair not,
as you can rectify this unfortunate situation with:
$ git submodule init $ git submodule update
8. Configure Jenkins
You're all set! Now go to the URL you've been using through your browser and
get your Jenkins instance configured to do what you need. Right off the bat,
I'd recommend going to
Manage Jenkins -
Configure and checking the
security box so that you can set up user accounts. Keeping your build server
open to the internet is kind of a security risk.
We've got plenty of things to improve, so pull requests are very welcome.
Future plans live at github issues, so feel free to file issues if you'd like to see something.