OpenStack cluster with chef-provisioning
This is the testing framework for OpenStack and Chef. We leverage this to test against our changes to our cookbooks to make sure that you can still build a cluster from the ground up with any changes we push up.
This framework also gives us an opportunity to show different Reference Architectures and a sane example on how to start with OpenStack and Chef.
master branch of the cookbooks, which is currently tied to the base OpenStack Kilo release, this supports deploying to Ubuntu 14.04 and CentOS 7.1 for all-in-one with nova-network. Support for all-in-one with Neutron, and multi-node support, is a work in progress.
Support for CentOS 6.5 and Ubuntu 12.04 with Icehouse is available with the stable/icehouse branch of this project.
Initial Setup Steps
$ git clone https://github.com/openstack/openstack-chef-repo.git $ cd openstack-chef-repo $ chef exec rake berks_vendor
The OpenStack cookbooks by default use databags for configuring passwords. There are four
data_bags : user_passwords, db_passwords, service_passwords, secrets. I have a already created
data_bags/ directory, so you shouldn't need to make them, if you do something's broken.
See Databags section below for more details.
NOTE: If you are running Ubuntu 14.04 LTS and as your base compute machine, you should note that the shipped
3.13.0-24-generic has networking issues, and the best way to resolve this is
apt-get install linux-image-generic-lts-utopic. This will install at least
3.16.0 from the Utopic hardware enablement.
For each deployment model, there is a corresponding readme file in the doc/ directory. Please review that for specific details and additional setup that might be required before deploying the cloud.
Rake Deploy Commands
These commands will spin up various OpenStack cluster configurations, the simplest being the all-in-one controller with nova-network.
For CentOS, set the environment variable REPO_OS=centos7
$ chef exec rake aio_nova # All-in-one controller with nova-network $ chef exec rake aio_neutron # All-in-one controller with Neutron $ chef exec rake multi_nova # Multi-node controller with nova-network and 3 compute nodes $ chef exec rake multi_neutron # Multi-node controller with Neutron and 3 compute nodes
Access the Controller
$ cd vms $ vagrant ssh controller $ sudo su -
Testing the Controller
# Access the controller as noted above $ source openrc $ nova-manage version $ nova service-list && nova hypervisor-list $ glance image-list $ keystone user-list $ nova list
Working with Security Groups
To allow SSH access to instances, a nova security group is defined as follows:
$ nova secgroup-list $ nova secgroup-list-rules default $ nova secgroup-create allow_ssh "allow ssh to instance" $ nova secgroup-add-rule allow_ssh tcp 22 22 0.0.0.0/0 $ nova secgroup-list-rules allow_ssh
Working with keys
To allow SSH keys to be injected into instance, a nova keypair is defined as follows:
# Just press Enter to all the questions $ ssh-keygen $ nova keypair-add --pub-key=/root/.ssh/id_rsa.pub mykey
Booting up a cirros image on the Controller
$ nova boot test --image cirros --flavor 1 --security-groups=allow_ssh --key-name=mykey
Wait a few seconds and the run
nova list if Status is not Active, wait a few seconds and repeat.
Once status is active you should be able to log in via ssh to the listed IP.
$ ssh cirros@<ip address from nova list output>
Accessing the OpenStack Dashboard
If you would like to use the OpenStack dashboard you should go to https://localhost:9443 and the username and password is
Verifying stack with tempest
If you ssh into the
controller machine you can test via the most recent tempest release.
$ cd vms $ vagrant ssh controller $ sudo su - root@controller:~ cd /opt/tempest root@controller:/opt/tempest$ ./run_tests.sh -V [-- snip --] tempest.tests.test_wrappers.TestWrappers test_pretty_tox 1.68 test_pretty_tox_fails 1.03 test_pretty_tox_serial 0.61 test_pretty_tox_serial_fails 0.55 Ran 233 tests in 13.869s OK Running flake8 ... root@controller:/opt/tempest#
To remove all the nodes and start over again with a different environment or different environment attribute overrides, using the following rake command.
$ chef exec rake destroy_machines
To refresh all the cookbooks, use the following rake commands.
$ chef exec rake destroy_cookbooks $ chef exec rake berks_vendor
To cleanup everything, use the following rake command.
$ chef exec rake clean
See the doc/tools.md for more information.
Some basic information about the use of databags within this repo.
# Show the list of databags $ chef exec knife data bag list -z db_passwords secrets service_passwords user_passwords # Show the list of data bag items $ chef exec knife data bag show db_passwords -z ceilometer cinder dash glance heat horizon ironic keystone neutron nova # Show contents of data bag item $ chef exec knife data bag show db_passwords ceilometer -z Encrypted data bag detected, decrypting with provided secret. ceilometer: mypass id: ceilometer # Update contents of data bag item # set EDITOR env var to your editor. For PowerShell, I used nano $ chef exec knife data bag edit secrets dispersion_auth_user -z
Data bag default values
db_passwords are set to "mypass" secrets are set to "_token" service_passwords are set to "mypass" user_passwords are set to "mypass"
Default encrypted data bag secret
The default secret is stored here .chef/encrypted_data_bag_secret and referenced by .chef/knife.rb.
Known Issues and Workarounds
The ChefDK provides all the required level of gems this testing suite needs, but there exists a Gemfile-Provisioning file that can be used as well. You will need to replace the Gemfile with the Gemfile-Provisioning before running your gem bundling. Note: please ignore the Gemfile, as it is needed only to pass the existing gates with older levels of gems.
When using this on a Windows platform, here are some tweaks to make this work:
- In order to get SSH to work, you will need an SSL client installed. You can use the one that comes with Git for Windows. You will need to append
C:\Program Files (x86)\Git\bin;to the system PATH.
- Better instructions for multi-node network setup
- Better support for aio_neutron and multi-node tests
- Support for floating IPs
- Split out the
multi-neutron-network-nodecluster also so the network node is it's own machine
- Support for swift multi node test
- Easier debugging. Maybe a script to pull the logs from the controller.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.