README.md

@@ -1,68 +1,16 @@
 # Puppet Enterprise Extra Large
 
-This Puppet module contains profile classes used to deploy an at-scale Puppet Enterprise architecture.
+This Puppet module contains Puppet Task Plans used to deploy an at-scale Puppet Enterprise architecture.
 
-It SHOULD contain instructions for how to set that all up, too. Right now it doesn't. Big To-do.
-
-Note pending more detailed instructions:
+Use this module to deploy Puppet Enterprise 2019.0.x Extra Large architecture.
 
 * This deployment depends on and assumes the use of trusted facts. Specifically, `pp_role` and `pp_environment`.
 * This deployment assumes that at least for PE infrastructure nodes, Puppet certnames are correct, resolvable FQDNs.
-* This deployment assumes the control repository to manage PE is independent of the normal "customer" control-repo.
 
 ## Documentation
 
-See this README file and any documents in the [docs](docs) directory.
+See this README file and any documents in the [documentation](documentation) directory.
 
 ## Architecture
 
-![architecture](docs/images/architecture.png)
-
-## Installation
-
-These are just sketched out instructions right now. It's likely there are big gaps still.
-
-### Prepare the Control Repositories
-
-You'll need two control repositories configured. One dedicated to managing Puppet Enterprise nodes (consider it kinda like an appliance), and another for your regular Puppet code used to manage your infrastructure.
-
-### Installing the Primary Master
-
-1. Download and extract the Puppet Enterprise installer
-2. Place the csr\_attributes.yaml file from installer/primary-master in /etc/puppetlabs/puppet/csr\_attributes.yaml
-3. Place the pe.conf file from installer/primary-master in the working directory, and edit it to fill in required values
-4. Run the installer, passing the appropriate flags to use the prepared pe.conf file
-5. Set up the ssh private keys needed to access the configured control repositories
-6. For each environment configured (however many you want to initially deploy), run e.g.
-
-        puppet code deploy production --wait
-        puppet code deploy pe_production --wait
-
-7. Using the same list of environments deployed above, run e.g.
-
-        puppet apply --environment pe_production --exec '
-          class { "pe_xl::node_manager":
-            environments => ["production", "pe_production"],
-          }
-        '
-
-5. Perform the PuppetDB Database installation (described below)
-6. Run `puppet agent -t`
-
-### Installing the PuppetDB Database
-
-1. Download and extract the Puppet Enterprise installer
-2. Place the csr\_attributes.yaml file from installer/puppetdb-database in /etc/puppetlabs/puppet/csr\_attributes.yaml
-3. Place the pe.conf file from installer/puppetdb-database in the working directory, and edit it to fill in required values
-4. Run the installer, passing the appropriate flags to use the prepared pe.conf file
-5. Finish the Primary Master installation (described above)
-6. Run `puppet agent -t`
-
-### Installing a Compile Master
-
-```
-curl -k https://primary-master.example.com:8140/packages/current/install.bash | sudo bash -s \
-  main:certname=<certname> \
-  extension_requests:pp_role="pe_xl::compile_master" \
-  extension_requests:pp_environment="pe_production"
-```
+![architecture](documentation/images/architecture.png)

docs/basic_usage.md → documentation/basic_usage.md

@@ -15,14 +15,14 @@ These are still a work in progress and lack documentation. Reading through them
 Besides getting everything installed the key configuration that makes this work is laid out in four classification groups. Links provided below to a Markdown document that describes the groups, and also to the Puppet manifest that actually configures them:
 
 * [classification.md](https://github.com/reidmv/reidmv-pe_xl/blob/master/docs/classification.md)
-* [configure\_node\_groups.pp](https://github.com/reidmv/reidmv-pe_xl/blob/master/tasks/configure_node_groups.pp)
+* [pe\_xl::node\_manager class](https://github.com/reidmv/reidmv-pe_xl/blob/master/manifests/node_manager.pp)
 
-The reference implementation uses trusted facts to put nodes in the right groups. Because the important puppet\_enterprise::\* class parameters and data are specified in the console, it should also be safe to have a pe.conf present on both the primary master, and the primary master replica nodes.
+The reference implementation uses trusted facts to put nodes in the right groups. Because the important puppet\_enterprise::\* class parameters and data are specified in the console, it should also be safe to have a pe.conf present on both the master, and the master replica nodes.
 
 ## Basic usage instructions
 
 1. Ensure the hostname of each system is set correctly, to the same value that will be used to connect to the system, and refer to the system as. If the hostname is not set as expected the installation plan will refuse to continue.
-2. Install Bolt on a jumphost. This can be the primary master, or any other system.
+2. Install Bolt on a jumphost. This can be the master, or any other system.
 3. Download or git clone the pe\_xl module and put it somewhere on the jumphost, e.g. ~/modules/pe\_xl.
 4. Create an inventory file with connection information. Example included below. Available Bolt configuration options are documented here.
 5. Create a parameters file. Example included below. Note at the top of the file are arguments which dictate which plans should be run, such as install+configure.
@@ -51,8 +51,8 @@ groups:
       - pe-xl-core-1.lab1.puppet.vm
       - pe-xl-core-2.lab1.puppet.vm
       - pe-xl-core-3.lab1.puppet.vm
-      - pe-xl-compilemaster-0.lab1.puppet.vm
-      - pe-xl-compilemaster-1.lab1.puppet.vm
+      - pe-xl-compiler-0.lab1.puppet.vm
+      - pe-xl-compiler-1.lab1.puppet.vm
 ```
 
 Example params.json Bolt parameters file:
@@ -63,18 +63,18 @@ Example params.json Bolt parameters file:
   "configure": true,
   "upgrade": false,
 
-  "primary_master_host": "pe-xl-core-0.lab1.puppet.vm",
+  "master_host": "pe-xl-core-0.lab1.puppet.vm",
   "puppetdb_database_host": "pe-xl-core-1.lab1.puppet.vm",
-  "primary_master_replica_host": "pe-xl-core-2.lab1.puppet.vm",
+  "master_replica_host": "pe-xl-core-2.lab1.puppet.vm",
   "puppetdb_database_replica_host": "pe-xl-core-3.lab1.puppet.vm",
-  "compile_master_hosts": [
-    "pe-xl-compilemaster-0.lab1.puppet.vm",
-    "pe-xl-compilemaster-1.lab1.puppet.vm"
+  "compiler_hosts": [
+    "pe-xl-compiler-0.lab1.puppet.vm",
+    "pe-xl-compiler-1.lab1.puppet.vm"
   ],
 
   "console_password": "puppetlabs",
   "dns_alt_names": [ "puppet", "puppet.lab1.puppet.vm" ],
-  "compile_master_pool_address": "puppet.lab1.puppet.vm",
+  "compiler_pool_address": "puppet.lab1.puppet.vm",
   "version": "2018.1.4"
 }
 ```

docs/classification.md → documentation/classification.md

@@ -14,8 +14,8 @@ The new groups are:
 
 * PE Master A
 * PE Master B
-* PE Compile Master Group A
-* PE Compile Master Group B
+* PE Compiler Group A
+* PE Compiler Group B
 
 The configuration applied in each group looks as follows:
 
@@ -25,9 +25,9 @@ The configuration applied in each group looks as follows:
 
 Notes for PE Master A:
 
-* The (initial) Primary Master is the only member of this node group
+* The (initial) Master is the only member of this node group
 * Sets as data two parameters
-    * `puppet_enterprise::profile::primary_master_replica::database_host_puppetdb`
+    * `puppet_enterprise::profile::master_replica::database_host_puppetdb`
     * `puppet_enterprise::profile::puppetdb::database_host`
 * Sets both parameters to the name of the PuppetDB PostgreSQL node paired with this master
 * Uses a different PuppetDB PostgreSQL node than PE Master B
@@ -37,39 +37,39 @@ Notes for PE Master A:
 
 Notes for PE Master B:
 
-* The (initial) Primary Master Replica is the only member of this node group
+* The (initial) Master Replica is the only member of this node group
 * Sets as data two parameters
-    * `puppet_enterprise::profile::primary_master_replica::database_host_puppetdb`
+    * `puppet_enterprise::profile::master_replica::database_host_puppetdb`
     * `puppet_enterprise::profile::puppetdb::database_host`
 * Sets both parameters to the name of the PuppetDB PostgreSQL node paired with this master
 * Uses a different PuppetDB PostgreSQL node than PE Master A
 
-### PE Compile Master Group A
-![PE Compile Master Group A group](images/pe-compile-master-group-a.png)
+### PE Compiler Group A
+![PE Compiler Group A group](images/pe-compiler-group-a.png)
 
-Notes for PE Compile Master Group A:
+Notes for PE Compiler Group A:
 
-* Half of the compile masters are members of this group
+* Half of the compilers are members of this group
 * Applies the `puppet_enterprise::profile::puppetdb` class
 * Sets the `puppet_enterprise::profile::puppetdb::database_host` parameter
-    * Should be set to `"pdb-pg-a"`, where "pdb-pg-a" is the name of the PuppetDB PostgreSQL database host paired with the (initial) Primary Master
+    * Should be set to `"pdb-pg-a"`, where "pdb-pg-a" is the name of the PuppetDB PostgreSQL database host paired with the (initial) Master
 * Modifies the `puppet_enterprise::profile::master::puppetdb_host` parameter
-    * Should be set to `[${clientcert}, "master-b"]`, where "master-b" is the name of the (initial) Primary Master Replica.
-    * If you have a load balancer for the compile masters in PE Compile Master Group B port 8081, you should use that load balancer address instead of "master-b"
+    * Should be set to `[${clientcert}, "master-b"]`, where "master-b" is the name of the (initial) Master Replica.
+    * If you have a load balancer for the compilers in PE Compiler Group B port 8081, you should use that load balancer address instead of "master-b"
 * Modifies the `puppet_enterprise::profile::master::puppetdb_port` parameter
     * Should be set to `[8081]`
 
-### PE Compile Master Group B
-![PE Compile Master Group B group](images/pe-compile-master-group-b.png)
+### PE Compiler Group B
+![PE Compiler Group B group](images/pe-compiler-group-b.png)
 
-Notes for PE Compile Master Group B:
+Notes for PE Compiler Group B:
 
-* The other half of the compile masters (those not in the PE Compile Master Group A node group) are members of this group
+* The other half of the compilers (those not in the PE Compiler Group A node group) are members of this group
 * Applies the `puppet_enterprise::profile::puppetdb` class
 * Sets the `puppet_enterprise::profile::puppetdb::database_host` parameter
-    * Should be set to `"pdb-pg-b"`, where "pdb-pg-b" is the name of the PuppetDB PostgreSQL database host paired with the (initial) Primary Master Replica
+    * Should be set to `"pdb-pg-b"`, where "pdb-pg-b" is the name of the PuppetDB PostgreSQL database host paired with the (initial) Master Replica
 * Modifies the `puppet_enterprise::profile::master::puppetdb_host` parameter
-    * Should be set to `[${clientcert}, "master-a"]`, where "master-a" is the name of the PuppetDB PostgreSQL node paired with the (initial) Primary Master Replica.
-    * If you have a load balancer for the compile masters in PE Compile Master Group A port 8081, you should use that load balancer address instead of "master-a"
+    * Should be set to `[${clientcert}, "master-a"]`, where "master-a" is the name of the PuppetDB PostgreSQL node paired with the (initial) Master Replica.
+    * If you have a load balancer for the compilers in PE Compiler Group A port 8081, you should use that load balancer address instead of "master-a"
 * Modifies the `puppet_enterprise::profile::master::puppetdb_port` parameter
-    * Should be set to `[8081]`
+    * Should be set to `[8081]`

functions/flatten_compact.pp

@@ -1,11 +1,7 @@
 function pe_xl::flatten_compact (
   Array $input,
 ) {
-  $input.flatten.reduce([]) |$output, $value| {
-    if ($value == undef) {
-      $output
-    } else {
-      $output << $value
-    }
+  $input.flatten.filter |$value| {
+    $value != undef
   }
 }

functions/print_apply_result.pp

@@ -0,0 +1,20 @@
+function pe_xl::print_apply_result(
+  Variant[ApplyResult, ResultSet] $result,
+) {
+  $enumerable = $result ? {
+    ResultSet => $result,
+    default   => [$result],
+  }
+
+  $enumerable.each |ApplyResult $apply| {
+    $apply.report['logs'].each |$log| {
+      # TODO: include file and line number, if present
+      notice("${log['time']} ${log['level'].upcase} ${log['source']} ${log['message']}")
+    }
+
+    $status = $apply.report['status']
+    $message = $apply.message
+    $target = $apply.target.name
+    notice("\"${status}\" on ${target}: ${message}")
+  }
+}

installer/primary-master/csr_attributes.yaml

@@ -1,4 +1,4 @@
 ---
 extension_requests:
-  pp_role: "pe_xl::primary_master"
+  pp_role: "pe_xl::master"
   pp_environment: "pe_production"

installer/primary-master/pe.conf

@@ -1,15 +1,15 @@
 {
-  # PRIMARY MASTER
+  # MASTER
   # Required
   "console_admin_password": "<<CONSOLE-ADMIN-PASSWORD>>"
 
   # Basic Puppet Enterprise configuration
-  "puppet_enterprise::puppet_master_host": "<<PRIMARY-MASTER-FQDN>>"
+  "puppet_enterprise::puppet_master_host": "<<MASTER-FQDN>>"
   "pe_install::puppet_master_dnsaltnames": [ <<DNS-ALT-NAMES>> ]
 
   # PuppetDB Database configuration
-  # This parameter does not change the "role" of the primary master; the
-  # primary master still considers itself a database host. It will reconfigure
+  # This parameter does not change the "role" of the master; the
+  # master still considers itself a database host. It will reconfigure
   # PuppetDB though to use a different PostgreSQL host for that database.
   "puppet_enterprise::profile::puppetdb::database_host": "<<PUPPETDB-DATABASE-FQDN>>"
 

installer/puppetdb-database/pe.conf

@@ -12,7 +12,7 @@
   "console_admin_password": "not used"
 
   # Basic Puppet Enterprise configuration
-  "puppet_enterprise::puppet_master_host": "<<PRIMARY-MASTER-FQDN>>"
+  "puppet_enterprise::puppet_master_host": "<<MASTER-FQDN>>"
 
   # This parameter changes the role of this host so that it considers itself a
   # database host. It will therefore install and configure PostgreSQL.

manifests/compile_master.pp

@@ -1,4 +1,4 @@
-class pe_xl::compile_master {
+class pe_xl::compiler {
   include pe_xl::agent
 
   @@haproxy::balancermember { "${::clientcert}_puppetserver_listener":