1 change: 1 addition & 0 deletions .fixtures.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ fixtures:
forge_modules:
ruby_task_helper: "puppetlabs/ruby_task_helper"
service: "puppetlabs/service"
package: "puppetlabs/package"
repositories:
facts: 'https://github.com/puppetlabs/puppetlabs-facts.git'
puppet_agent: 'https://github.com/puppetlabs/puppetlabs-puppet_agent.git'
Expand Down
3 changes: 2 additions & 1 deletion .vscode/settings.json
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
{
"python.linting.pylintEnabled": true,
"python.linting.enabled": true
"python.linting.enabled": true,
"git.ignoreLimitWarning": true
}
17 changes: 16 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,29 @@

All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) and this project adheres to [Semantic Versioning](http://semver.org).

## [v3.8.0](https://github.com/puppetlabs/puppetlabs-peadm/tree/v3.8.0) (2022-06-21)

[Full Changelog](https://github.com/puppetlabs/puppetlabs-peadm/compare/v3.7.0...v3.8.0)

### Added

- Fix classification when adding some components [\#258](https://github.com/puppetlabs/puppetlabs-peadm/pull/258) ([ody](https://github.com/ody))
- Add task and update configure plan to allow for ldap configuration on… [\#253](https://github.com/puppetlabs/puppetlabs-peadm/pull/253) ([bwilcox](https://github.com/bwilcox))

### Fixed

- Restart compiler services [\#271](https://github.com/puppetlabs/puppetlabs-peadm/pull/271) ([ody](https://github.com/ody))
- Set additional rules on replica when adding compiler [\#270](https://github.com/puppetlabs/puppetlabs-peadm/pull/270) ([ody](https://github.com/ody))
- Updates documentation [\#269](https://github.com/puppetlabs/puppetlabs-peadm/pull/269) ([ody](https://github.com/ody))

## [v3.7.0](https://github.com/puppetlabs/puppetlabs-peadm/tree/v3.7.0) (2022-05-11)

[Full Changelog](https://github.com/puppetlabs/puppetlabs-peadm/compare/v3.6.0...v3.7.0)

### Added

- Add support for PE 2021.6 and 2019.8.11 [\#260](https://github.com/puppetlabs/puppetlabs-peadm/pull/260) ([reidmv](https://github.com/reidmv))
- Add optional `pe_installer_source` parameter [\#261](https://github.com/puppetlabs/puppetlabs-peadm/pull/261) ([reidmv](https://github.com/reidmv))
- Support PE 2021.6 and 2019.8.11 [\#260](https://github.com/puppetlabs/puppetlabs-peadm/pull/260) ([reidmv](https://github.com/reidmv))
- Add experimental restore plan [\#250](https://github.com/puppetlabs/puppetlabs-peadm/pull/250) ([reidmv](https://github.com/reidmv))

## [v3.6.0](https://github.com/puppetlabs/puppetlabs-peadm/tree/v3.6.0) (2022-05-04)
Expand Down
2 changes: 1 addition & 1 deletion Gemfile
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ group :development do
gem "puppet-module-win-dev-r#{minor_version}", '~> 1.0', require: false, platforms: [:mswin, :mingw, :x64_mingw]
gem "puppet-debugger", '>= 0.18.0', require: false
gem "bolt", '>= 3.17.0', require: false
gem "github_changelog_generator", require: false
gem "github_changelog_generator", '>= 1.16.4', require: false
gem "octokit", '4.21.0', require: false
end
group :system_tests do
Expand Down
84 changes: 84 additions & 0 deletions REFERENCE.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@

### Data types

* [`Peadm::Ldap_config`](#peadmldap_config)
* [`Peadm::Pe_version`](#peadmpe_version)
* [`Peadm::Pem`](#peadmpem)
* [`Peadm::Recovery_opts`](#peadmrecovery_opts)
Expand All @@ -51,6 +52,7 @@
* [`agent_upgrade`](#agent_upgrade): Upgrade the target system using upgrade.bash from a master
* [`backup_classification`](#backup_classification): A task to call the classification api and write to file
* [`cert_data`](#cert_data): Return certificate data related to the Puppet agent
* [`cert_valid_status`](#cert_valid_status): Check primary for valid state of a certificate
* [`code_manager`](#code_manager): Perform various code manager actions
* [`code_sync_status`](#code_sync_status): A task to confirm code is in sync accross the cluster for clusters with code manager configured
* [`divert_code_manager`](#divert_code_manager): Divert the code manager live-dir setting
Expand All @@ -62,6 +64,7 @@
* [`mkdir_p_file`](#mkdir_p_file): Create a file with the specified content at the specified location
* [`mv`](#mv): Wrapper task for mv command
* [`pe_install`](#pe_install): Install Puppet Enterprise from a tarball
* [`pe_ldap_config`](#pe_ldap_config): Set the ldap config in the PE console
* [`pe_uninstall`](#pe_uninstall): Uninstall Puppet Enterprise
* [`precheck`](#precheck): Return pre-check information about a system
* [`provision_replica`](#provision_replica): Execute the replica provision puppet command
Expand Down Expand Up @@ -111,6 +114,7 @@ Supported use cases:
* `peadm::util::insert_csr_extension_requests`
* `peadm::util::retrieve_and_upload`
* `peadm::util::sanitize_pg_pe_conf`
* `peadm::util::sync_global_hiera`
* `peadm::util::update_classification`: Configure classification
* `peadm::util::update_db_setting`: Make updates to PuppetDB database settings

Expand Down Expand Up @@ -770,6 +774,40 @@ Data type: `TargetSpec`

## Data types

### <a name="peadmldap_config"></a>`Peadm::Ldap_config`

The Peadm::Ldap_config data type.

Alias of

```puppet
Struct[{
base_dn => String,
connect_timeout => Integer,
disable_ldap_matching_rule_in_chain => Boolean,
display_name => String,
group_lookup_attr => String,
group_member_attr => String,
group_name_attr => String,
group_object_class => String,
Optional[group_rdn] => Optional[String],
Optional[help_link] => Optional[String],
hostname => String,
Optional[login] => Optional[String],
Optional[password] => Optional[String],
port => Integer,
search_nested_groups => Boolean,
ssl => Boolean,
ssl_hostname_validation => Boolean,
ssl_wildcard_validation => Boolean,
start_tls => Boolean,
user_display_name_attr => String,
user_email_attr => String,
user_lookup_attr => String,
Optional[user_rdn] => Optional[String],
}]
```

### <a name="peadmpe_version"></a>`Peadm::Pe_version`

The Peadm::Pe_version data type.
Expand Down Expand Up @@ -877,6 +915,20 @@ Return certificate data related to the Puppet agent

**Supports noop?** false

### <a name="cert_valid_status"></a>`cert_valid_status`

Check primary for valid state of a certificate

**Supports noop?** false

#### Parameters

##### `certname`

Data type: `String`

The certifcate name to check validation of

### <a name="code_manager"></a>`code_manager`

Perform various code manager actions
Expand Down Expand Up @@ -1093,6 +1145,26 @@ Data type: `Optional[Enum['stopped']]`

If 'stopped', ensure the Puppet agent is not running when install completes

### <a name="pe_ldap_config"></a>`pe_ldap_config`

Set the ldap config in the PE console

**Supports noop?** false

#### Parameters

##### `ldap_config`

Data type: `Peadm::Ldap_config`

The hash of options for ldap.

##### `pe_main`

Data type: `String`

The PE Main server

### <a name="pe_uninstall"></a>`pe_uninstall`

Uninstall Puppet Enterprise
Expand Down Expand Up @@ -1465,6 +1537,7 @@ The following parameters are available in the `peadm::install` plan:
* [`internal_compiler_a_pool_address`](#internal_compiler_a_pool_address)
* [`internal_compiler_b_pool_address`](#internal_compiler_b_pool_address)
* [`pe_installer_source`](#pe_installer_source)
* [`ldap_config`](#ldap_config)
* [`primary_host`](#primary_host)
* [`replica_host`](#replica_host)
* [`compiler_hosts`](#compiler_hosts)
Expand Down Expand Up @@ -1524,6 +1597,17 @@ URL given.

Default value: ``undef``

##### <a name="ldap_config"></a>`ldap_config`

Data type: `Optional[Peadm::Ldap_config]`

If specified, configures PE RBAC DS with the supplied configuration hash.
The parameter should be set to a valid set of connection settings as
documented for the PE RBAC /ds endpoint. See:
https://puppet.com/docs/pe/latest/rbac_api_v1_directory.html#put_ds-request_format

Default value: ``undef``

##### <a name="primary_host"></a>`primary_host`

Data type: `Peadm::SingleTargetSpec`
Expand Down
42 changes: 39 additions & 3 deletions documentation/automated_recovery.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,29 @@

These instructions provide automated procedures for recovering from select failures of PE components which are managed by PEADM.

Additional manual procedures are documented in [recovery.md](recovery.md)
Manual procedures are documented in [recovery.md](recovery.md)

## Recover from failed primary Puppet server

1. Promote the replica ([official docs](https://puppet.com/docs/pe/2019.8/dr_configure.html#dr-promote-replica))
2. [Replace missing or failed replica Puppet server](#replace-missing-or-failed-replica-puppet-server)

## Replace missing or failed replica Puppet server

This procedure uses the following placeholder references.

* _\<primary-server-fqdn\>_ - The FQDN and certname of the primary Puppet server
* _\<replica-postgres-server-fqdn\>_ - The FQDN and certname of the PE-PostgreSQL server which resides in the same availability group as the replacement replica Puppet server
* _\<replacement-replica-fqdn\>_ - The FQDN and certname of the replacement replica Puppet server

1. Run `peadm::add_replica` plan to deploy replacement replica Puppet server
1. For Standard and Large deployments

bolt plan run peadm::add_replica primary_host=<primary-server-fqdn> replica_host=<replacement-replica-fqdn>

2. For Extra Large deployments

bolt plan run peadm::add_replica primary_host=<primary-server-fqdn> replica_host=<replacement-replica-fqdn> replica_postgresql_host=<replica-postgres-server-fqdn>

## Replace failed PE-PostgreSQL server (A or B side)

Expand All @@ -22,7 +44,7 @@ Procedure:

2. Temporarily set both primary and replica server nodes so that they use the remaining healthy PE-PostgreSQL server

bolt plan run peadm::util::update_db_setting --target <primary-server-fqdn>,<replica-server-fqdn> primary_postgresql_host=<working-postgres-server-fqdn>
bolt plan run peadm::util::update_db_setting --target <primary-server-fqdn>,<replica-server-fqdn> postgresql_host=<working-postgres-server-fqdn> override=true

3. Restart `pe-puppetdb.service` on Puppet server primary and replica

Expand All @@ -34,4 +56,18 @@ Procedure:

5. Run `peadm::add_database` plan to deploy replacement PE-PostgreSQL server

bolt plan run peadm::add_database -t <replacement-postgres-server-fqdn> primary_host=<primary-server-fqdn>
bolt plan run peadm::add_database -t <replacement-postgres-server-fqdn> primary_host=<primary-server-fqdn>

## Replace failed replica puppet server AND failed replica pe-postgresql server

This procedure uses the following placeholder references.

* _\<primary-server-fqdn\>_ - The FQDN and certname of the primary Puppet server
* _\<failed-replica-fqdn\>_ - The FQDN and certname of the failed replica Puppet server

1. Ensure the old replica server is forgotten.

bolt command run "/opt/puppetlabs/bin/puppet infrastructure forget <failed-replica-fqdn>" --targets <primary-server-fqdn>

2. [Replace failed PE-PostgreSQL server (A or B side)](#replace-failed-pe-postgresql-server-a-or-b-side)
3. [Replace missing or failed replica Puppet server](#replace-missing-or-failed-replica-puppet-server)
14 changes: 13 additions & 1 deletion documentation/install.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,19 @@ Example params.json Bolt parameters file (shown: Extra Large with DR):
}
```

Review the [peadm::install plan](../plans/install.pp) to learn about more advanced installation options. It is possible to supply an ssh private key and git clone URL for a control-repo as part of installation, for example.
Example params.json Bolt parameters file (shown: Standard):

```json
{
"primary_host": "pe-xl-core-0.lab1.puppet.vm",

"console_password": "puppetlabs",
"dns_alt_names": [ "puppet", "puppet.lab1.puppet.vm" ],
"version": "2021.5.0",
}
```

Review the [peadm::install plan](../plans/install.pp) to learn about more advanced installation options. For example, it is possible to: supply an ssh private key and git clone URL for a control-repo as part of installation; supply the LDAP configuration data for PE; and similar complete automation tie-ins.

## Offline usage

Expand Down
6 changes: 5 additions & 1 deletion metadata.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "puppetlabs-peadm",
"version": "3.7.0",
"version": "3.8.0",
"author": "puppetlabs",
"summary": "Bolt plans used to deploy an at-scale Puppet Enterprise architecture",
"license": "Apache-2.0",
Expand Down Expand Up @@ -32,6 +32,10 @@
"name": "puppetlabs/service",
"version_requirement": ">= 1.3.0 < 3.0.0"
},
{
"name": "puppetlabs/package",
"version_requirement": ">= 2.1.0 < 3.0.0"
},
{
"name": "puppetlabs/inifile",
"version_requirement": ">= 5.2.0 < 6.0.0"
Expand Down
37 changes: 34 additions & 3 deletions plans/add_compiler.pp
Original file line number Diff line number Diff line change
Expand Up @@ -14,13 +14,35 @@
Peadm::SingleTargetSpec $compiler_host,
Peadm::SingleTargetSpec $primary_host,
Peadm::SingleTargetSpec $primary_postgresql_host,
){
) {
$compiler_target = peadm::get_targets($compiler_host, 1)
$primary_target = peadm::get_targets($primary_host, 1)
$primary_postgresql_target = peadm::get_targets($primary_postgresql_host, 1)

# Get current peadm config to determine where to setup additional rules for
# compiler's secondary PuppetDB instances
$peadm_config = run_task('peadm::get_peadm_config', $primary_target).first.value

# Return the opposite server than the compiler to be added so it can be
# configured with the appropriate rules for Puppet Server access from
# compiler
$replica_avail_group_letter = $avail_group_letter ? { 'A' => 'B', 'B' => 'A' }
$replica_puppetdb = $peadm_config['role-letter']['server'][$replica_avail_group_letter]

$replica_puppetdb_target = peadm::get_targets($replica_puppetdb, 1)

# Stop puppet.service
run_command('systemctl stop puppet.service', $primary_postgresql_target)
run_command('systemctl stop puppet.service', peadm::flatten_compact([
$primary_postgresql_target,
$replica_puppetdb_target
]))

apply($replica_puppetdb_target) {
file_line { 'pe-puppetdb-compiler-cert-allow':
path => '/etc/puppetlabs/puppetdb/certificate-allowlist',
line => $compiler_target.peadm::certname(),
}
}

# Add the following two lines to /opt/puppetlabs/server/data/postgresql/11/data/pg_ident.conf
#
Expand Down Expand Up @@ -85,15 +107,24 @@
},
)

# Source the global hiera.yaml from Primary and synchronize to new compiler
run_plan('peadm::util::sync_global_hiera', $compiler_target,
primary_host => $primary_target
)

# On <compiler-host>, run the puppet agent
run_task('peadm::puppet_runonce', $compiler_target)

# On <primary_postgresql_host> run the puppet agent
run_task('peadm::puppet_runonce', $primary_postgresql_target)
run_task('peadm::puppet_runonce', peadm::flatten_compact([
$primary_postgresql_target,
$replica_puppetdb_target
]))

# On <primary_postgresql_host> start puppet.service
run_command('systemctl start puppet.service', peadm::flatten_compact([
$primary_postgresql_target,
$replica_puppetdb_target,
$compiler_target,
]))

Expand Down
Loading