34 changes: 34 additions & 0 deletions .devcontainer/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# devcontainer


For format details, see https://aka.ms/devcontainer.json.

For config options, see the README at:
https://github.com/microsoft/vscode-dev-containers/tree/v0.140.1/containers/puppet

``` json
{
"name": "Puppet Development Kit (Community)",
"dockerFile": "Dockerfile",

// Set *default* container specific settings.json values on container create.
"settings": {
"terminal.integrated.shell.linux": "/bin/bash"
},

// Add the IDs of extensions you want installed when the container is created.
"extensions": [
"puppet.puppet-vscode",
"rebornix.Ruby"
]

// Use 'forwardPorts' to make a list of ports inside the container available locally.
"forwardPorts": [],

// Use 'postCreateCommand' to run commands after the container is created.
"postCreateCommand": "pdk --version",
}
```



16 changes: 5 additions & 11 deletions .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
@@ -1,23 +1,17 @@
// For format details, see https://aka.ms/devcontainer.json. For config options, see the README at:
// https://github.com/microsoft/vscode-dev-containers/tree/v0.140.1/containers/puppet
{
"name": "Puppet Development Kit (Community)",
"dockerFile": "Dockerfile",

// Set *default* container specific settings.json values on container create.
"settings": {
"terminal.integrated.shell.linux": "/bin/bash"
"terminal.integrated.profiles.linux": {
"bash": {
"path": "bash",
}
}
},

// Add the IDs of extensions you want installed when the container is created.
"extensions": [
"puppet.puppet-vscode",
"rebornix.Ruby"
]

// Use 'forwardPorts' to make a list of ports inside the container available locally.
// "forwardPorts": [],

// Use 'postCreateCommand' to run commands after the container is created.
// "postCreateCommand": "pdk --version",
}
16 changes: 8 additions & 8 deletions .fixtures.yml
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
---
fixtures:
repositories:
cron_core: "git://github.com/puppetlabs/puppetlabs-cron_core"
provision: 'https://github.com/puppetlabs/provision.git'
bootstrap: "https://github.com/puppetlabs/puppetlabs-bootstrap"
puppet_conf: "https://github.com/puppetlabs/puppetlabs-puppet_conf"
deploy_pe: 'git://github.com/jarretlavallee/puppet-deploy_pe.git'
ruby_task_helper: "https://git@github.com/puppetlabs/puppetlabs-ruby_task_helper"
stdlib: "https://github.com/puppetlabs/puppetlabs-stdlib.git"
facts: 'https://github.com/puppetlabs/puppetlabs-facts.git'
cron_core: 'https://github.com/puppetlabs/puppetlabs-cron_core'
provision: 'https://github.com/puppetlabs/provision'
bootstrap: 'https://github.com/puppetlabs/puppetlabs-bootstrap'
puppet_conf: 'https://github.com/puppetlabs/puppetlabs-puppet_conf'
deploy_pe: 'https://github.com/jarretlavallee/puppet-deploy_pe'
ruby_task_helper: 'https://git@github.com/puppetlabs/puppetlabs-ruby_task_helper'
stdlib: 'https://github.com/puppetlabs/puppetlabs-stdlib'
facts: 'https://github.com/puppetlabs/puppetlabs-facts'
25 changes: 25 additions & 0 deletions .github/workflows/main_pr_testing.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
name: Call main PR testing workflows

on:
workflow_dispatch:
pull_request:
paths-ignore:
- '**.md'
- 'examples/**'
- 'CODEOWNERS'
- 'LICENSE'

jobs:
spec_testing:
name: "Spec testing"
uses: ./.github/workflows/spec.yml

lts_testing:
name: "PE LTS Testing"
needs: spec_testing
uses: ./.github/workflows/pe_lts_testing.yml

latest_testing:
name: "PE latest Testing"
needs: spec_testing
uses: ./.github/workflows/pe_latest_testing.yml
24 changes: 22 additions & 2 deletions .github/workflows/pe_latest_testing.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ name: "PE Latest Acceptance Testing"

on:
workflow_dispatch:
pull_request:
workflow_call:

env:
HONEYCOMB_WRITEKEY: 7f3c63a70eecc61d635917de46bea4e6
Expand Down Expand Up @@ -50,14 +50,34 @@ jobs:
buildevents step $TRACE_ID $STEP_ID $STEP_START 'Setup Environment'
echo STEP_ID=Setup-Acceptance-Test-Matrix >> $GITHUB_ENV
echo STEP_START=$(date +%s) >> $GITHUB_ENV
# store the result of the curl call in $forge_response
- name: Curl Forge for PE versions
id: curl_forge
run: |
echo "::set-output name=forge_response::$(curl https://forge.puppet.com/private/versions/pe)"
- name: Set latest release
id: latest_release
run: |
out=$(jq -c '[.[] | select(.lts == false)][0].latest | {"collection": [.]}' <<<'${{ steps.curl_forge.outputs.forge_response }}')
echo "::set-output name=latest::$out"
- name: Setup Acceptance Test Matrix
id: get-matrix
run: |
echo "::set-output name=matrix::$(cat spec/fixtures/matrix/latest.json)"
if [[ -e spec/fixtures/matrix/latest.json ]]; then
out=$(jq -c '. + ($matrix | .[])' --slurpfile matrix spec/fixtures/matrix/latest.json <<<'${{ steps.latest_release.outputs.latest }}')
echo "::set-output name=matrix::$out"
else
echo "::set-output name=matris::{}"
fi
- name: "Honeycomb: Record Setup Test Matrix time"
if: ${{ always() }}
run: |
buildevents step $TRACE_ID $STEP_ID $STEP_START 'Setup Test Matrix'
Acceptance:
name: "${{matrix.platforms.label}}, ${{matrix.collection}}"
needs:
Expand Down
24 changes: 22 additions & 2 deletions .github/workflows/pe_lts_testing.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ name: "PE LTS Acceptance Testing"

on:
workflow_dispatch:
pull_request:
workflow_call:

env:
HONEYCOMB_WRITEKEY: 7f3c63a70eecc61d635917de46bea4e6
Expand Down Expand Up @@ -50,14 +50,34 @@ jobs:
buildevents step $TRACE_ID $STEP_ID $STEP_START 'Setup Environment'
echo STEP_ID=Setup-Acceptance-Test-Matrix >> $GITHUB_ENV
echo STEP_START=$(date +%s) >> $GITHUB_ENV
# Store the result of the curl call in $forge_response
- name: Curl Forge for PE versions
id: curl_forge
run: |
echo "::set-output name=forge_response::$(curl https://forge.puppet.com/private/versions/pe)"
- name: Set latest release
id: latest_release
run: |
out=$(jq -c '[.[] | select(.lts == true)][0].latest | {"collection": [.]}' <<<'${{ steps.curl_forge.outputs.forge_response }}')
echo "::set-output name=latest::$out"
- name: Setup Acceptance Test Matrix
id: get-matrix
run: |
echo "::set-output name=matrix::$(cat spec/fixtures/matrix/lts.json)"
if [[ -e spec/fixtures/matrix/lts.json ]]; then
out=$(jq -c '. + ($matrix | .[])' --slurpfile matrix spec/fixtures/matrix/lts.json <<<'${{ steps.latest_release.outputs.latest }}')
echo "::set-output name=matrix::$out"
else
echo "::set-output name=matris::{}"
fi
- name: "Honeycomb: Record Setup Test Matrix time"
if: ${{ always() }}
run: |
buildevents step $TRACE_ID $STEP_ID $STEP_START 'Setup Test Matrix'
Acceptance:
name: "${{matrix.platforms.label}}, ${{matrix.collection}}"
needs:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/spec.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ name: "Spec Tests"

on:
workflow_dispatch:
pull_request:
workflow_call:

env:
HONEYCOMB_WRITEKEY: 7f3c63a70eecc61d635917de46bea4e6
Expand Down
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,15 @@

All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) and this project adheres to [Semantic Versioning](http://semver.org).

## [v2.3.0](https://github.com/puppetlabs/puppetlabs-pe_databases/tree/v2.3.0) (2022-07-21)

[Full Changelog](https://github.com/puppetlabs/puppetlabs-pe_databases/compare/v2.2.0...v2.3.0)

### Added

- \(SUP-3409\) Update puppet strings comments [\#104](https://github.com/puppetlabs/puppetlabs-pe_databases/pull/104) ([elainemccloskey](https://github.com/elainemccloskey))
- \(SUP-3408\) Remove deprecated backup functionality [\#103](https://github.com/puppetlabs/puppetlabs-pe_databases/pull/103) ([elainemccloskey](https://github.com/elainemccloskey))

## [v2.2.0](https://github.com/puppetlabs/puppetlabs-pe_databases/tree/v2.2.0) (2021-10-06)

[Full Changelog](https://github.com/puppetlabs/puppetlabs-pe_databases/compare/v2.1.1...v2.2.0)
Expand Down
53 changes: 2 additions & 51 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,9 @@ Table of Contents
- [What does this module provide?](#what-does-this-module-provide)
- [Usage](#usage)
- [Items you may want to configure](#items-you-may-want-to-configure)
- [Backup Schedule](#backup-schedule)
- [Backup Retention Policy](#backup-retention-policy)
- [Disable Maintenance](#disable-maintenance)
- [General PostgreSQL Recommendations](#general-postgresql-recommendations)
- [Tuning](#tuning)
- [Backups](#backups)
- [Maintenance](#maintenance)
- [PostgreSQL Settings](#postgresql-settings)
- [maintenance_work_mem](#maintenance_work_mem)
Expand All @@ -23,17 +20,14 @@ Table of Contents

# Overview

This module provides tuning, maintenance, and backups for PE PostgreSQL.
This module provides tuning, and maintenance for PE PostgreSQL.

## What does this module provide?

This module provides the following functionaility

1. Customized settings for PE PostgreSQL
1. Maintenance to keep the `pe-puppetdb` database lean and fast
1. Backups for all PE PostgreSQL databases, disabled by default
- The `pe-puppetdb` database is backed up every week
- Other databases are backed up every night

## Usage

Expand All @@ -46,34 +40,12 @@ It is not recommended to classify using a pre-existing node group in the PE Cons

## Items you may want to configure

### Backup Schedule

> WARNING: The backup functionality in this module has been deprecated and will be removed in a future release.
> WARNING: The backup functionality in this module has been removed.
Please refer to the [PE Backup and Restore documentation](https://puppet.com/docs/pe/latest/backing_up_and_restoring_pe.html) for details on how to backup.
You should ensure the parameter `pe_databases::manage_database_backups` and any parameters from the `pe_databases::backup` class are removed from classification or hiera.
You should also clean up associated crontab entries.

Backups are not activated by default but can be enabled by setting the following parameter:

Hiera classification example

```
pe_databases::manage_database_backups:: true
```

You can modify the default backup schedule by provide an array of hashes that describes the databases to backup and their backup schedule.
Please refer to the [hieradata_examples](https://github.com/puppetlabs/puppetlabs-pe_databases/tree/main/hieradata_examples) directory of this repository for examples.

> IMPORTANT NOTE: If you change the default schedule, it will stop managing the associated crontab entries, and there's not a clean way to automatically remove unmanaged crontab entries.
So you should delete all pe-postgres crontab entries via `crontab -r -u pe-postgres` and let Puppet repopulate them if you change the default schedule.
Otherwise, you will create duplicate backups.

### Backup Retention Policy

By default, the backup script will retain two backups for each database.
When the backup script runs, it will remove the older of the two backups before starting the backup itself.
You can configure the retention policy by setting `pe_databases::backup::retention_policy: <NUMBER_OF_BACKUPS_TO_RETAIN>`.

### Disable Maintenance

The maintenance systemd timers will perform a `pg_repack` on various `pe-puppetdb` tables to keep them lean and fast.
Expand All @@ -93,27 +65,6 @@ If you are using a dedicated node for PE PostgreSQL, then some of the settings c
This module attempts to provide default settings that accommodate both a Monolithic install and a dedicated PE PostgreSQL instance.
Those defaults change based on the `$all_in_one_pe` parameter.

## Backups

> WARNING: The backup functionality in this module has been deprecated and will be removed in a future release.
Please refer to the [PE Backup and Restore documentation](https://puppet.com/docs/pe/latest/backing_up_and_restoring_pe.html) for details on how to backup.
You should ensure the parameter `pe_databases::manage_database_backups` and any parameters from the `pe_databases::backup` class are removed from classification or hiera.
You should also clean up associated crontab entries.

This is the documentation for Pupet Enterprise backups:

https://puppet.com/docs/pe/latest/backing_up_and_restoring_pe.html

This module provides an alternative to backup just the PE PostgreSQL databases.

It is recommended that you backup each database individually so that if you have an issue with one database you do not have to restore all databases.

Under ideal conditions you would backup all databases daily, however, backing up large databases such as `pe-puppetdb`, results in excessive disk I/O so you may prefer to backup `pe-puppetdb` weekly while backing up the rest of the smaller databases daily.

The choice to backup `pe-puppetdb` more frequently should be based on the business needs.

This module provides a script for backing up PE PostgreSQL databases and two default cron jobs: one weekly to back up the `pe-puppetdb` database, and one daily to backup every database except `pe-puppetdb`.

## Maintenance

This module provides systemd timers to pg_repack tables in the `pe-puppetdb` database:
Expand Down
Loading