Permalink
Browse files

dhcplb virtual lab using vagrant and chef-solo

Summary:
One of the "problems" with the `dhcplb` project is that it's difficult to contribute it to it if you are not in that DC environment.

I have worked on setting up a virtual lab using VMs, it has been tested on my OSX laptop using `vagrant` and `chef-solo` cookbooks to set up the following:

* N`dhcpserver` VMs: N VMs running ISC `dhcpd` (both v4 and v6) configured with a subnet in the private network space. You can start as many as you want by changing the variable on top of the `Vagrantfile`.
* `dhcplb`: a VM running the `dhcplb` itself, configured to foward traffic to the above;
* `dhcprelay`: a VM running ISC `dhcrelay`, it intercepts broadcast/multicast traffic from the client below and relays traffic to the above, it simulates a top of rack switch relay in the DC;
* `dhcpclient`: a VM you can use to run `dhclient`, or `perfdhcp` manually to test things. It's DISCOVER/SOLICIT messages will be picked up by the `dhcprelay` instance

more information in the `README.md` file
Closes #12

Differential Revision: D4237057

Pulled By: pallotron

fbshipit-source-id: b99225409492a8b9b1a87f575a53e540d23b7079
  • Loading branch information...
pallotron authored and Facebook Github Bot committed Nov 28, 2016
1 parent 85c9ff9 commit cc038509d6cd2ed0984c2e12a5d5e5faf048edff
Showing with 488 additions and 0 deletions.
  1. +2 −0 .gitignore
  2. +154 −0 vagrant/README.md
  3. +83 −0 vagrant/Vagrantfile
  4. +18 −0 vagrant/chef/cookbooks/.gitignore
  5. +15 −0 vagrant/chef/cookbooks/.kitchen.yml
  6. +4 −0 vagrant/chef/cookbooks/Berksfile
  7. +2 −0 vagrant/chef/cookbooks/dhcpclient/README.md
  8. +7 −0 vagrant/chef/cookbooks/dhcpclient/metadata.rb
  9. +8 −0 vagrant/chef/cookbooks/dhcpclient/recipes/default.rb
  10. +1 −0 vagrant/chef/cookbooks/dhcplb/README.md
  11. +30 −0 vagrant/chef/cookbooks/dhcplb/files/default/dhcplb.config.json
  12. +8 −0 vagrant/chef/cookbooks/dhcplb/metadata.rb
  13. +24 −0 vagrant/chef/cookbooks/dhcplb/recipes/default.rb
  14. +1 −0 vagrant/chef/cookbooks/dhcplb/templates/default/dhcp-servers-v4.cfg.erb
  15. +1 −0 vagrant/chef/cookbooks/dhcprelay/README.md
  16. +7 −0 vagrant/chef/cookbooks/dhcprelay/metadata.rb
  17. +15 −0 vagrant/chef/cookbooks/dhcprelay/recipes/default.rb
  18. +15 −0 vagrant/chef/cookbooks/dhcprelay/templates/default/etc_default_isc-dhcp-relay.erb
  19. +1 −0 vagrant/chef/cookbooks/dhcpserver/README.md
  20. +21 −0 vagrant/chef/cookbooks/dhcpserver/files/default/etc_default_isc-dhcp-server
  21. +7 −0 vagrant/chef/cookbooks/dhcpserver/metadata.rb
  22. +29 −0 vagrant/chef/cookbooks/dhcpserver/recipes/default.rb
  23. +14 −0 vagrant/chef/cookbooks/dhcpserver/templates/default/dhcpd.conf.erb
  24. +4 −0 vagrant/chef/roles/base.rb
  25. +4 −0 vagrant/chef/roles/dhcpclient.rb
  26. +5 −0 vagrant/chef/roles/dhcplb.rb
  27. +4 −0 vagrant/chef/roles/dhcprelay.rb
  28. +4 −0 vagrant/chef/roles/dhcpserver.rb
View
@@ -0,0 +1,2 @@
.vagrant
*.swp
View
@@ -0,0 +1,154 @@
# How to setup your test environment with Vagrant
The instruction below will help you bringing up a virtual lab containing VMs
sharing their own private network(s).
This assumes you are somewhat familiar with
[`vagrant`](https://www.vagrantup.com/).
This has been tested under OSX but it should work find on Linux too.
Please provide feedback or PRs/patches if you find problems.
This instructions are for DHCPv4 only, DHCPv6 will follow soon.
## Install dependencies
First, install `chef-dk` from https://downloads.chef.io/chef-dk/ .
On OSX you can use `brew`:
```
$ brew cask install Caskroom/cask/chefdk
```
Install `vagrant-berkshelf` plugin:
```
$ vagrant plugin install vagrant-berkshelf
$ cd ${PROJECT_ROOT}/vagrant/
$ berk install
$ berks vendor chef/cookbooks
```
You might need to disable dhcpserver for `vboxnet0` in VirtualBox:
```
$ VBoxManage dhcpserver remove --netname HostInterfaceNetworking-vboxnet0
```
## Start VMs
To start all the vms:
```
$ cd ${PROJECT_ROOT}/vagrant/
$ vagrant up
```
This will bring up the following VMs:
* `dhcpserver`: a VM running ISC `dhcpd` (both v4 and v6) configured with a
subnet in the private network space. You can start as many as you want by
changing the variable on top of the `Vagrantfile`.
* `dhcplb`: a VM running the `dhcplb` itself, configured to foward traffic to
the above;
* `dhcprelay`: a VM running ISC `dhcrelay`, it intercepts broadcast/multicast
traffic from the client below and relays traffic to the above;
* `dhcpclient`: a VM you can use to run `dhclient`, or `perfdhcp` manually to
test things. It's DISCOVER/SOLICIT messages will be picked up by the
`dhcprelay` instance
You can ssh into VMs using `vagrant ssh ${vm_name}`. Destroy them with
`vagrant destrory ${vm_name}`. If you find bugs in the `chef` cookbooks or you
want to change something there you can test your `chef` changes using
`vagrant provision ${vm_name}` on a running VM.
## Development cycle
Just edit `dhcplb`'s code on your host machine (the machine running VirtualBox
or whatever VM solution you are using). The root directory of your github
checkout will be mounted into the `dhcplb` VM at
`~/go/src/github.com/facebookincubator/dhcplb`.
You can compile the binary using:
```
$ cd ~/go/src/github.com/facebookincubator/dhcplb
$ go build
$ sudo mv dhcplb $GOBIN
```
And restart it with:
```
# initctl restart dhcplb
```
Logs will be in `/var/log/upstart/dhcplb.log` (becuase the current Vagrant image
uses a version of Ubuntu using Upstart init replacement).
On the `dhcpclient` you can initiate dhcp requests using these commands:
```
# perfdhcp -R 1 -4 -r 1200 -p 30 -t 1 -i 192.168.51.104
# dhclient -d -1 -v -pf /run/dhclient.eth1.pid -lf /var/lib/dhcp/dhclient.eth1.leases eth1
```
You will see:
```
root@dhcpclient:~# dhclient -d -1 -v -pf /run/dhclient.eth1.pid -lf
/var/lib/dhcp/dhclient.eth1.leases eth1
Internet Systems Consortium DHCP Client 4.2.4
Copyright 2004-2012 Internet Systems Consortium.
All rights reserved.
For info, please visit https://www.isc.org/software/dhcp/
Listening on LPF/eth1/08:00:27:7b:79:94
Sending on LPF/eth1/08:00:27:7b:79:94
Sending on Socket/fallback
DHCPDISCOVER on eth1 to 255.255.255.255 port 67 interval 3 (xid=0xcd1fdb2d)
DHCPREQUEST of 192.168.51.152 on eth1 to 255.255.255.255 port 67
(xid=0x2ddb1fcd)
DHCPOFFER of 192.168.51.152 from 192.168.51.104
DHCPACK of 192.168.51.152 from 192.168.51.104
RTNETLINK answers: File exists
bound to 192.168.51.152 -- renewal in 227 seconds.
^C
```
And something in the dhcplb logs:
```
I1125 15:54:11.985895 12190 modulo.go:65] List of available stable servers:
I1125 15:54:11.985943 12190 modulo.go:67] 192.168.50.104:67
I1125 15:54:11.985953 12190 modulo.go:67] 192.168.50.105:67
I1125 15:54:16.532833 12190 glog_logger.go:91] client_mac: 08:00:27:7b:79:94, dhcp_server: 192.168.50.104, giaddr: 192.168.51.101, latency_us: 112, server_is_rc: false, source_ip: 192.168.50.101, success: true, type: Discover, version: 4, xid: 0xcd1fdb2d
I1125 15:54:16.534310 12190 glog_logger.go:91] client_mac: 08:00:27:7b:79:94, dhcp_server: 192.168.50.104, giaddr: 192.168.51.101, latency_us: 117, server_is_rc: false, source_ip: 192.168.50.101, success: true, type: Request, version: 4, xid: 0xcd1fdb2d
```
[ISC KEA's
perfdhcp](https://kea.isc.org/wiki/DhcpBenchmarking) utility comes handy so it's
installed for your convenience.
Should you need to change something in the `dhcprelay` here are some useful
commands:
```
# initctl list
# initctl (stop|start|restart) isc-dhcp-relay
# /usr/sbin/dhcrelay -d -4 -i eth1 -i eth2 192.168.50.104
```
The relay config is in `/etc/default/isc-dhcp-relay`.
In general you don't need to touch the `dhcpserver` but you need to restart it
you can use:
```
# /etc/init.d/isc-dhcp-server restart
```
The main config is in `/etc/dhcp/dhcpd.conf`.
Subnets are configured like this should you need to change them:
```
subnet 192.168.50.0 netmask 255.255.255.0 {}
subnet 192.168.51.0 netmask 255.255.255.0 {range 192.168.51.220 192.168.51.230;}
```
View
@@ -0,0 +1,83 @@
# If you use OSX and Virtual Box You need to run:
#
# $ VBoxManage dhcpserver remove --netname HostInterfaceNetworking-vboxnet0
#
# to remove the VirtualBox internal DHCP server... as it's going to interfeer
# with the your environment.
VAGRANTFILE_API_VERSION = "2"
NUM_DHCPSERVERS = 2
# external net => network where lb, relay and dhcp server sit.
EXT_NET_PREFIX = "192.168.50"
# internal net => network where only the client and relay sit.
INT_NET_PREFIX = "192.168.51"
# Following structure represents the list of nodes, please note that the "ips"
# array is in format [internal ip, external ip]. ORDER MATTERS.
nodes = {
'dhcprelay' =>
{'ips' => ["#{EXT_NET_PREFIX}.101", "#{INT_NET_PREFIX}.101"],
'roles' => ['role[dhcprelay]']},
'dhcpclient' =>
{'ips' => ["#{INT_NET_PREFIX}.102"],
'roles' => ['role[dhcpclient]']},
'dhcplb' =>
{'ips' => ["#{EXT_NET_PREFIX}.103"],
'roles' => ['role[dhcplb]']},
}
# list of dhcpservers, to be used to configure the dhcplb instance.
dhcpservers_ips = []
start_ip = 104
(1..NUM_DHCPSERVERS).each do |i|
int_ip = "#{INT_NET_PREFIX}.#{start_ip}"
ext_ip = "#{EXT_NET_PREFIX}.#{start_ip}"
start_ip += 1
nodes["dhcpserver#{i}"] =
{'ips' => [int_ip, ext_ip], 'roles' => ['role[dhcpserver]']}
i += 1
dhcpservers_ips.push(ext_ip)
end
nodes['dhcplb']['target_dhcp_servers'] = dhcpservers_ips
# the dhcplb ip the relay needs to point to
nodes['dhcprelay']['target_dhcplb'] = nodes['dhcplb']['ips'][0]
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
config.vm.box = "ubuntu/trusty64"
config.berkshelf.enabled = true
config.berkshelf.berksfile_path = "chef/cookbooks/Berksfile"
nodes.each do |name, node|
config.vm.define name do |vm|
vm.vm.hostname = name
ips = node["ips"]
ips.each do |ip|
vm.vm.network :private_network, ip: ip
end
if name == "dhcplb"
vm.vm.synced_folder "../",
"/home/vagrant/go/src/github.com/facebookincubator/dhcplb/"
end
vm.vm.provision :chef_solo do |chef|
chef.cookbooks_path = ["chef/cookbooks"]
chef.roles_path = "chef/roles"
chef.add_role("base")
chef.json = nodes
node['roles'].each do |role|
chef.add_role(role)
end
end
end
end
end
# -*- mode: ruby -*-
# vi: set ft=ruby :
@@ -0,0 +1,18 @@
*~
*#
.#*
\#*#
.*.sw[a-z]
*.un~
pkg/
# Berkshelf
.vagrant
/cookbooks
Berksfile.lock
# Bundler
Gemfile.lock
bin/*
.bundle/*
@@ -0,0 +1,15 @@
---
driver:
name: vagrant
provisioner:
name: chef_solo
platforms:
- name: ubuntu-14.04
- name: centos-7.2
suites:
- name: default
run_list:
attributes:
@@ -0,0 +1,4 @@
source "https://supermarket.chef.io/"
cookbook 'apt'
cookbook 'golang'
cookbook 'poise-service'
@@ -0,0 +1,2 @@
This cookbook configures a VM containing dhcp clients (like `dhclient` and
`perfdhcp`)
@@ -0,0 +1,7 @@
name 'dhcpclient'
maintainer 'Facebook'
maintainer_email 'pallotron@fb.com'
license 'All rights reserved'
description 'Installs/Configures the dhcp client VM'
long_description IO.read(File.join(File.dirname(__FILE__), 'README.md'))
version '0.1.0'
@@ -0,0 +1,8 @@
apt_repository 'kea-repo' do
uri 'ppa:xdeccardx/isc-kea'
end
# this contains perfdhcp utility
package 'kea-admin' do
action :install
end
@@ -0,0 +1 @@
This cookbook configures the dhcplb VM.
@@ -0,0 +1,30 @@
{
"v4": {
"version": 4,
"listen_addr": "0.0.0.0",
"port": 67,
"packet_buf_size": 1024,
"update_server_interval": 30,
"free_conn_timeout": 30,
"algorithm": "xid",
"host_sourcer": "file:/home/vagrant/dhcp-servers-v4.cfg",
"rc_ratio": 0,
"throttle_cache_size": 1024,
"throttle_cache_rate": 128,
"throttle_rate": 256
},
"v6": {
"version": 6,
"listen_addr": "::",
"port": 547,
"packet_buf_size": 1024,
"update_server_interval": 30,
"free_conn_timeout": 30,
"algorithm": "xid",
"host_sourcer": "file:/home/vagrant/dhcp-servers-v6.cfg",
"rc_ratio": 0,
"throttle_cache_size": 1024,
"throttle_cache_rate": 128,
"throttle_rate": 256
}
}
@@ -0,0 +1,8 @@
name 'dhcplb'
maintainer 'Facebook'
maintainer_email 'pallotron@fb.com'
license 'All rights reserved'
description 'Installs/Configures dhcplb'
long_description IO.read(File.join(File.dirname(__FILE__), 'README.md'))
version '0.1.0'
depends 'poise-service'
@@ -0,0 +1,24 @@
node.default['go']['version'] = '1.7'
node.default['go']['packages'] = ['github.com/facebookincubator/dhcplb']
poise_service 'dhcplb' do
command '/opt/go/bin/dhcplb -version 4 -config /home/vagrant/dhcplb.config.json'
end
directory '/home/vagrant/go' do
owner 'vagrant'
group 'vagrant'
recursive true
end
cookbook_file '/home/vagrant/dhcplb.config.json' do
source 'dhcplb.config.json'
notifies :restart, 'service[dhcplb]'
end
template '/home/vagrant/dhcp-servers-v4.cfg' do
source 'dhcp-servers-v4.cfg.erb'
# dhcplb will auto load files that change. no need to notify.
end
# Configure service via https://github.com/poise/poise-service
@@ -0,0 +1 @@
<%= node['dhcplb']['target_dhcp_servers'].join("\n") %>
@@ -0,0 +1 @@
This cookbook configures dhcrelay to point to dhcplb.
@@ -0,0 +1,7 @@
name 'dhcprelay'
maintainer 'Facebook'
maintainer_email 'pallotron@fb.com'
license 'All rights reserved'
description 'Installs/Configures isc-dhcp-relay'
long_description IO.read(File.join(File.dirname(__FILE__), 'README.md'))
version '0.1.0'
@@ -0,0 +1,15 @@
package 'isc-dhcp-relay' do
action :install
end
template '/etc/default/isc-dhcp-relay' do
source 'etc_default_isc-dhcp-relay.erb'
owner 'root'
group 'root'
mode '0644'
notifies :restart, 'service[isc-dhcp-relay]'
end
service 'isc-dhcp-relay' do
action [:enable, :start]
end
Oops, something went wrong.

0 comments on commit cc03850

Please sign in to comment.