VMPooler provides configurable 'pools' of instantly-available (pre-provisioned) and/or on-demand (provisioned on request) virtual machines.
At Puppet, Inc. we run acceptance tests on thousands of disposable VMs every day. VMPooler manages the life cycle of these VMs from request through deletion, with options available to pool ready instances, and provision on demand.
As of version 2.0.0, all providers other than the dummy one are now separate gems. Historically the vSphere provider was included within VMPooler itself. That code has been moved to the puppetlabs/vmpooler-provider-vsphere repository and the vmpooler-provider-vsphere gem. To migrate from VMPooler 1.x to 2.0 you will need to ensure that vmpooler-provider-vsphere is installed along side the vmpooler gem. See the Provider API docs for more information.
The recommended method of installation is via the Helm chart located in puppetlabs/vmpooler-deployment. That repository also provides Docker images of VMPooler.
VMPooler requires a Redis server. This is the data store used for VMPooler's inventory and queuing services.
VMPooler itself and the dev environment talked about below require additional Ruby gems to function. You can update the currently required ones for VMPooler by running ./update-gemfile-lock.sh. The gems for the dev environment can be updated by running ./docker/update-gemfile-lock.sh. These scripts will utilize the container on the FROM line of the Dockerfile to update the Gemfile.lock in the root of this repo and in the docker folder, respectively.
Configuration for VMPooler may be provided via environment variables, or a configuration file.
The provided configuration defaults are reasonable for small VMPooler instances with a few pools. If you plan to run a large VMPooler instance it is important to consider configuration values appropriate for the instance of your size in order to avoid starving the provider, or Redis, of connections.
VMPooler uses a connection pool for Redis to improve efficiency and ensure thread safe usage. At Puppet, we run an instance with about 100 pools at any given time. We have to provide it with 200 Redis connections to the Redis connection pool, and a timeout for connections of 40 seconds, to avoid timeouts. Because metrics are generated for connection available and waited, your metrics provider will need to be able to cope with this volume. Prometheus or StatsD is recommended to ensure metrics get delivered reliably.
Please see this configuration document for more details about configuring VMPooler via environment variables.
The following YAML configuration sets up two pools, debian-7-i386 and debian-7-x86_64, which contain 5 running VMs each:
---
:providers:
:vsphere:
server: 'vsphere.example.com'
username: 'vmpooler'
password: 'swimsw1msw!m'
:redis:
server: 'redis.example.com'
:config:
logfile: '/var/log/vmpooler.log'
:pools:
- name: 'debian-7-i386'
template: 'Templates/debian-7-i386'
folder: 'Pooled VMs/debian-7-i386'
pool: 'Pooled VMs/debian-7-i386'
datastore: 'vmstorage'
size: 5
provider: vsphere
- name: 'debian-7-x86_64'
template: 'Templates/debian-7-x86_64'
folder: 'Pooled VMs/debian-7-x86_64'
pool: 'Pooled VMs/debian-7-x86_64'
datastore: 'vmstorage'
size: 5
provider: vsphereSee the provided YAML configuration example, vmpooler.yaml.example, for additional configuration options and parameters or for supporting multiple providers.
VMPooler provides an API and web front-end (dashboard) on port :4567. See the provided YAML configuration example, vmpooler.yaml.example, to specify an alternative port to listen on.
VMPooler provides a REST API for VM management. See the API documentation for more information.
A dashboard is provided to offer real-time statistics and historical graphs. It looks like this:
Graphite is required for historical data retrieval. See the provided YAML configuration example, vmpooler.yaml.example, for details.
- vmfloaty is a ruby based CLI tool and scripting library. We consider it the primary way for users to interact with VMPooler.
- vagrant-vmpooler: Use Vagrant to create and manage your VMPooler instances.
A docker-compose file is provided to support running VMPooler and associated tools locally. This is useful for development because your local code is used to build the gem used in the docker-compose environment. The compose environment also pulls in the latest providers via git. Details of this setup are stored in the docker/ folder.
docker-compose -f docker/docker-compose.yml build && \
docker-compose -f docker/docker-compose.yml upA Vagrantfile is included in this repository so as to provide a reproducible development environment.
vagrant up
vagrant ssh
cd /vagrant
docker-compose -f docker/docker-compose.yml build && \
docker-compose -f docker/docker-compose.yml upThe Vagrant environment also contains multiple rubies you can utilize for spec test and the like. You can see a list of the pre-installed ones when you log in as part of the message of the day.
For more information about setting up a development instance of VMPooler or other subjects, see the docs/ directory.
| Endpoint | URL |
|---|---|
| Redis Commander | http://localhost:8079 |
| API | http://localhost:8080/api/v1 |
| Dashboard | http://localhost:8080/dashboard/ |
| Metrics (API) | http://localhost:8080/prometheus |
| Metrics (Manager) | http://localhost:8081/prometheus |
| Jaeger | http://localhost:8082 |
Additionally, the Redis instance can be accessed at localhost:6379.
VMPooler is distributed under the Apache License, Version 2.0. See the LICENSE file for more details.