This will be an attempt to document the vSphere Home Lab that I set up for deployment and testing of BOSH and Cloud Foundry.
I put together a spec for a decent-sized tower server from Dell:
-
PowerEdge T320
-
Intel Xeon E5-2440 2.40 GHz 6 Cores/12 Threads
-
64 GB 1333 MHz RAM
-
4 1 TB 7.2K RPM SATA 3Gbps Storage in RAID 0
The hope was that this would provide enough firepower so stand up reasonable vSphere and OpenStack environments. So far, so good. The only thing I would improve about this setup so far is to add an additional six core processor ($$$!), primarily to add additional virtualization capacity, and switch from spinners to solid-state storage to drive more IOPS.
One of the great desires I had for this environment was agility. It should be easy to set up and tear down environments so that I can rapidly switch between various vSphere and OpenStack configurations. To do this, I decided to go with nested virtualization, meaning I’d install the target hypervisor inside of another hypervisor. Interestingly enough, I could never get the T320 to boot from a USB stick, so I burned an Ubuntu 12.04 LTS Desktop ISO to a DVD and installed that. Next I installed a copy of VMware Workstation 9. Workstation 9 has built-in support for running VMware ESXi servers, and also has decent support for configuring virtual networks (although occasionally, the GUI does not present the full capabilities of the software and we have to drop down to low-level configuration files - more to come).
VMware Workstation has support for three types of virtual networks:
- Bridged
-
The virtual machine uses physical network adapters to connect a network, and the virtual machine appears as an additional computer on the same physical Ethernet network as the host system.
- NAT
-
The virtual machine is assigned an IP address on a separate network that is private to the host system. The VM accesses external networks via NAT, and traffic appears to have originated from the host system.
- Host-Only
-
The virtual machine is assigned an IP address on a separate network that is private to the host system. The VM is not visible outside the host system.
We’ll leverage each of these network types in our topology for the lab setup.
-
We’ll use a bridged network for components we’d like to be accessible from outside the host system. In this case, we’ll assign our vCenter Appliance to the bridged network, as we’d like to be able to access vCenter from machines other than the host server.
-
We’ll use a NAT network for components we’d like to place on a private network, but for which we also desire outbound Internet access. We’ll use a NAT network for our VM network, as we’d like our Cloud Foundry VM’s (e.g. the DEA) to be able to access the Internet (e.g. to pull in externally-defined buildpacks residing on GitHub).
-
We’ll use a Host-Only network for components we’d like to place on a private network, but which do not require outbound Internet access. Our management network, which we’ll use for connecting and administering our vCenter appliance, ESXi hosts, and Openfiler storage appliance, and our storage network, which we’ll use for providing iSCSI services from Openfiler to our ESXi hosts, will be Host-Only networks.
Things of which to take note:
-
When configuring a network (e.g. vmnet2), you must check the box labeled "Connect a host virtual adapter (vmnet2) to this network." This is the only way I was able to consistently access (e.g. ping) an IP address connected to one of these networks from the host machine.
-
For some reason, at least in my particular installation of VMware Workstation, you are unable to edit the subnet mask. Never fear. If you edit the file /etc/vmware/networking, you’ll find something similar to the following: ./etc/vmware/networking
VERSION=1,0 answer VNET_0_VIRTUAL_ADAPTER no answer VNET_1_DHCP yes answer VNET_1_DHCP_CFG_HASH E1650DC3EA2BB12C7BC5407F8180B7E6CEF48E00 answer VNET_1_HOSTONLY_NETMASK 255.255.255.0 answer VNET_1_HOSTONLY_SUBNET 192.168.85.0 answer VNET_1_VIRTUAL_ADAPTER yes answer VNET_2_HOSTONLY_NETMASK 255.0.0.0 answer VNET_2_HOSTONLY_SUBNET 10.0.0.0 answer VNET_2_VIRTUAL_ADAPTER yes answer VNET_3_DHCP no answer VNET_3_HOSTONLY_NETMASK 255.255.255.0 answer VNET_3_HOSTONLY_SUBNET 192.168.86.0 answer VNET_3_NAT yes answer VNET_3_VIRTUAL_ADAPTER yes answer VNET_4_HOSTONLY_NETMASK 255.255.255.0 answer VNET_4_HOSTONLY_SUBNET 192.168.87.0 answer VNET_4_VIRTUAL_ADAPTER yes answer VNET_8_DHCP yes answer VNET_8_DHCP_CFG_HASH 777F23EAC50D88F6E485C883930902A01191ACE8 answer VNET_8_HOSTONLY_NETMASK 255.255.255.0 answer VNET_8_HOSTONLY_SUBNET 172.16.110.0 answer VNET_8_NAT yes answer VNET_8_VIRTUAL_ADAPTER yes answer VNL_DEFAULT_BRIDGE_VNET 0 add_bridge_mapping eth0 0 add_bridge_mapping eth1 0
If you’ll notice answer VNET_2_HOSTONLY_NETMASK 255.0.0.0, this allows us to create 10.0.0.0/8 as a network. This blog entry sheds some additional light on the mechanics of manual network configuration in VMware Workstation.