(SUP-2007) Add local import and viewing of metrics

[jarretlavallee]

This PR ports the functionality of https://github.com/puppetlabs/puppet-metrics-viewer into this repository in an effort to archive https://github.com/puppetlabs/puppet-metrics-viewer. The viewer functionality is provided by new rake tasks which use Litmus and bolt to spin up a local docker container with the dashboard. The metrics from a local directory are then imported into the docker dashboard instance.

This PR adds a new module to the fixtures to be able to import the data into the local dashboard container.

There are some additional benefits from this functionality

• View a temporary copy of the 'puppet_metrics' dashboards
• Develop dashboards on an ephemeral instance
• Import offline data into the ephemeral instance

Usage

There 4 new rake tasks to perform tasks

rake viewer[metrics_location,retention_days]                                                               # Provisions a local dashboard instance and imports the metrics data
rake viewer:destroy                                                                                        # Destroys metrics dashboard instance
rake viewer:import[metrics_location,retention_days]                                                        # Imports metrics data into a local metrics dashboard instance
rake viewer:provision                                                                                      # Provisions a local metrics dashboard instance

Spin up a new instance and import 10 days of metrics from a path.

bundle exec rake 'viewer[/path/to/metrics,10]'

Destroy the instance

bundle exec rake viewer:destroy

Spin up a new instance with the dashboard

bundle exec rake viewer:provision

Import metrics into an existing instance

bundle exec rake viewer:import[/path/to/metrics,30]

[gguillotte]

prereqs: docker installed and running, bundle install

Ran:

bundle exec rake 'viewer[/Users/gguillotte/Cases/.../puppet_enterprise_support_.../metrics,10]'

This script's metrics had 1.6G of data.

Got:

Provisioning the node
HEAD is now at 9972009 Merge pull request #222 from tuxmea/rel/6.1.1
HEAD is now at b071cc8 Merge pull request #152 from tuxmea/rel/3.0.1
[✔] Provisioning litmusimage/centos:7 using docker provisioner.
localhost:2222, litmusimage/centos:7
install_agent
Building '/Users/gguillotte/git/puppet_metrics_dashboard' into '/Users/gguillotte/git/puppet_metrics_dashboard/pkg'
Built '/Users/gguillotte/git/puppet_metrics_dashboard/pkg/puppetlabs-puppet_metrics_dashboard-2.3.1.tar.gz'
Installed '/Users/gguillotte/git/puppet_metrics_dashboard/pkg/puppetlabs-puppet_metrics_dashboard-2.3.1.tar.gz' on
Preparing the node
Applying the dashboard class. This may take a while
... (about 4 minutes) ...
The dashboard is available on http://localhost:3000 and data can be imported in http://localhost:8086
The default login is admin:admin
Extracting tarballs
Deleted 26478 files older than 10 days
Importing 37485 metrics will take a while. Only STDERR will be displayed
... (about 8 minutes) ...
Metrics will populate the in the dashboard during this time.
WARNING: STDIN provided but not specified via '-'
All metrics have been imported. They should be accessible at http://localhost:3000  with the admin:admin credentials

The dashboard worked and the results were the same as in puppet-metrics-viewer.

Ran several sets of metrics through, most runs took 2-5 minutes from end to end. Shortest were 16 seconds (no records were imported because all were more than 10 days old), longest took 11 minutes (5.6GB set covering 90 days).

[jarretlavallee]

CI is failing due to puppetlabs/litmusimage#25. CentOS 6 went EOL and the repos are no longer available and causing the acceptance tests on CentOS6 to fail. As this PR does not touch any functionality that would be covered by the CI tests, I am merging it with the approval that was added.