Skip to content

PMM 15014 Upgrade internal PostgreSQL version from 14 to 18 - #5295

Open
talhabinrizwan wants to merge 21 commits into
mainfrom
PMM-15014-upgrade-to-pg18
Open

PMM 15014 Upgrade internal PostgreSQL version from 14 to 18#5295
talhabinrizwan wants to merge 21 commits into
mainfrom
PMM-15014-upgrade-to-pg18

Conversation

@talhabinrizwan

@talhabinrizwan talhabinrizwan commented Apr 27, 2026

Copy link
Copy Markdown
Contributor

PMM-15014

Link to the Feature Build: SUBMODULES-4324


Important

Two values are temporarily set to keep CI green and must be changed before merging.

Where Now Before merge
minPGVersion, managed/models/database.go 14 15
commonExpectedFiles, managed/services/server/logs_test.go postgresql14.log postgresql.log

Both are marked with a TODO in the code.

Why. The Unit tests job runs against the prebuilt ghcr.io/percona/pmm:3-dev-container image, which is built from main and still ships PostgreSQL 14.23 with a supervisord config that writes /srv/logs/postgresql14.log. With the real values in place, checkVersion rejects the server on the first database-backed test and every later test cascades on the leftover connection, and TestFiles compares against a log name the old container does not produce. The image only picks up PostgreSQL 18 once this branch's server image is published as 3-dev-latest, which is circular until merge.

15 rather than 18 for the floor: the bundled server is 18 either way, so standalone is unaffected, while a floor of 18 needlessly rejects externally hosted PostgreSQL 15/16/17 for HA and BYO-database setups. PostgreSQL 14 reaches end of life this autumn, so 14 is not viable as a final value. See the comment below for the measured compatibility data.

Summary by CodeRabbit

  • New Features
    • Upgraded bundled PostgreSQL to version 18.
    • Added automatic PostgreSQL 14-to-18 data migration with safer staging and recovery.
    • Improved database ownership, authentication, and extension setup.
  • Bug Fixes
    • Updated backup and restore workflows for PostgreSQL 18.
    • Standardized PostgreSQL log naming.
  • Tests
    • Updated log download expectations for the new log format.
  • Documentation
    • Added PostgreSQL 14-to-18 migration guidance.
    • Updated external PostgreSQL prerequisites to version 18 or later.

  PostgreSQL 15+ revoked the default CREATE privilege on the public schema
  from all users. Explicit grants are now required in the build-time Ansible
  role, the initialization role, and the pg14→pg18 migration script.
  PostgreSQL 15+ revoked the default CREATE privilege on the public schema
  from all users. Add an explicit grant in initWithRoot() after provisioning
  the database and role.
  PostgreSQL 15+ revoked the default CREATE privilege on the public schema
  from all users. Add ensureSchemaGrant() called unconditionally in SetupDB
  before migrateDB, covering both fresh installs and existing containers
  where the role was provisioned without the grant.

  Also fix a pre-existing bug in initWithRoot where GRANT ALL PRIVILEGES used
  $1/$2 placeholders, which are not supported for identifiers in PostgreSQL.
   Add back the scram-sha-256 authentication option alongside trust in the
   dev container's pg_hba.conf configuration, preserving the comments
   explaining the dual-auth setup for convenience in dev environments.
@codecov

codecov Bot commented Apr 27, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 0% with 16 lines in your changes missing coverage. Please review.
✅ Project coverage is 45.43%. Comparing base (31318c7) to head (d4e5517).
⚠️ Report is 92 commits behind head on main.

Files with missing lines Patch % Lines
managed/models/database.go 0.00% 16 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5295      +/-   ##
==========================================
+ Coverage   43.59%   45.43%   +1.84%     
==========================================
  Files         415      418       +3     
  Lines       43134    43332     +198     
==========================================
+ Hits        18804    19690     +886     
+ Misses      22454    21701     -753     
- Partials     1876     1941      +65     
Flag Coverage Δ
admin 34.96% <ø> (+0.17%) ⬆️
agent 51.12% <ø> (+2.07%) ⬆️
managed 45.00% <0.00%> (+2.02%) ⬆️
vmproxy 72.22% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@JiriCtvrtka

Copy link
Copy Markdown
Contributor

@talhabinrizwan It still needs some changes in managed to pass tests there properly.

  The Go template for the PostgreSQL supervisord config had an extra
  -c max_connections=2000 flag that was not present in the original PG14
  template and does not match the test fixture (pmm-db_enabled.ini),
  causing TestSavePMMConfig to fail. Removed to restore parity with
  prior behavior.
Comment thread build/ansible/roles/postgres/files/postgres-migration Outdated
fi
}

upgrade_pg14_to_pg18

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this func uses /srv/.pgpassword file but is called before ensure_postgres_password that creates this file in case it is absent

Comment thread managed/models/database.go Outdated
Comment thread managed/models/database.go Outdated
Comment thread managed/models/database.go Outdated
Conflict resolutions:
- .devcontainer/setup.py: deleted in favour of main's setup.sh; the
  PG18 data-dir rename ported over to it.
- build/ansible/roles/grafana/files/datasources.yml: took main's removal
  of the PostgreSQL datasource, dropping the postgresVersion bump.
- build/ansible/roles/postgres/tasks/main.yml: kept the PG18 rename and
  the public-schema grant, dropped the grafana data-migration task
  removed on main.
- managed/models/database.go: kept the ensureSchemaGrant call, adapted
  to main's non-inline error handling.
- managed/testdata/supervisord.d/pmm-ch_low_memory.ini: new fixture on
  main, renamed to PG18 to match the supervisord template.
Add a section describing the automatic pg14 to pg18 upgrade performed by
build/ansible/roles/postgres/files/postgres-migration on the first start
of a PMM Server that ships PostgreSQL 18: the trigger conditions, the
dump and restore steps, the role and schema grants required by
PostgreSQL 15+, and the rollback path.

Also scope the existing section to the v11 to v14 upgrade it describes
and fix a few typos in it.
Since PostgreSQL 15 the public schema belongs to pg_database_owner, and
GRANT ALL PRIVILEGES ON DATABASE does not confer CREATE on it. The branch
worked around that by granting CREATE on public in five separate places:
in Go on every pmm-managed start, in three ansible tasks and inline in the
upgrade script.

Create each database with its role as the owner instead, which confers the
privilege implicitly and leaves one mechanism:

- initWithRoot creates the role before the database and passes OWNER; the
  now-redundant GRANT ALL PRIVILEGES and the whole ensureSchemaGrant
  function are gone, so SetupDB no longer reads the superuser password and
  opens a second connection pool on every start.
- The postgresql_privs tasks become owner: on the existing postgresql_db
  tasks, and priv: ALL on the user tasks is dropped as redundant.
- The upgrade script already created the databases with OWNER, so its two
  GRANT statements are dropped.

Also thread ctx through initWithRoot to use the Context query variants,
collapse the duplicated grafana restore block into a loop over both
databases, drop a redundant PGPASSWORD unset/re-read cycle, and hoist the
repeated PostgreSQL bin path into PG_BIN.

The external PostgreSQL setup instructions had the same problem and would
leave PMM with a database it cannot create tables in, so they now use
OWNER too.
The entrypoint created the PostgreSQL cluster on a fresh installation and
the migration script created one for the pg14 upgrade, so the data dir
creation, the initdb flag set and the pg_stat_statements call existed
twice. A change to either had to be mirrored, and a divergence would only
show up as an auth failure on one installation flavour.

Give the script sole ownership of the embedded cluster. It gains
init_postgres_cluster for the fresh case, and both paths now share
initdb_cluster, create_pg_stat_statements and store_postgres_password.
The entrypoint just invokes the script, which it already did.

init_postgres_cluster runs after upgrade_pg14_to_pg18, so an upgradable
data directory can no longer be replaced by an empty one; previously that
ordering rested on /srv/pmm-distribution being present. Creation also
happens after /run/postgresql is created rather than before it.
update_pg_hba_auth was added alongside the switch to scram (f2d3c3a) to
retrofit clusters created earlier with initdb --auth=trust. Those clusters
are PostgreSQL 14 ones, and every PostgreSQL 18 data directory is created
by initdb_cluster or the ansible role with --auth-host=scram-sha-256, so
the condition it looks for can no longer arise: pg_hba.conf is not carried
across the dump and restore, and upgrading an --auth=trust cluster already
yields a scram-only one.

Its remaining effect was to revert the trust rule the dev container adds,
on every container start.
@ademidoff

ademidoff commented Jul 30, 2026

Copy link
Copy Markdown
Member

Note on minPGVersion

Superseding my earlier version of this comment: the pre-merge action is the one in the PR description — restore the constant to 18. It is temporarily 14 so the Unit tests job can run against the prebuilt dev container, which is built from main and still embeds PostgreSQL 14.23. With the floor at 18 that job fails on the first database-backed test and the remaining ~147 failures are all cascade from the leftover connection (pq: database "pmm-managed-dev" is being accessed by other users).

The test image only gets PostgreSQL 18 once this branch's server image is published as 3-dev-latest, which is circular until merge — hence the temporary floor rather than a workflow change.

Worth deciding separately: 18 or 15?

Leaving this here as data, not as an instruction. minPGVersion conflates the version PMM bundles (18 either way) with the minimum external server PMM accepts. At 18 we reject externally hosted PostgreSQL 15, 16 and 17, which affects HA and BYO-database deployments.

Measured by running the full migration set (118 migrations) as the pmm-managed role against real servers:

minPGVersion PG 14.22 PG 16.14 PG 18.3
18 rejected rejected 118/118
15 rejected 118/118 118/118
14 118/118 118/118 118/118

Nothing in the schema requires 15+, so the floor is purely a support-policy choice. 15 is a defensible line because it is where public stopped being writable by PUBLIC and became owned by pg_database_owner — the ownership model this PR adopts for provisioning. PostgreSQL 14 reaches end of life this autumn, so 14 is not a viable final value either way.

The unit test job runs against a prebuilt dev container image that is built
from main and still embeds PostgreSQL 14.23, so a floor of 18 makes
checkVersion reject it on the first database-backed test and every later
test cascades on the leftover connection.

Lower the floor so the suite can run. This must be restored before merging;
the constant is marked and the pull request description says so.
Same reason as the minPGVersion floor: the unit test job runs against a
prebuilt dev container built from main, whose supervisord still writes
/srv/logs/postgresql14.log. Restore before merging.
/srv/logs/postgresql18.log becomes /srv/logs/postgresql.log, so the name
survives future major upgrades instead of changing with each one. This also
matches what the troubleshooting documentation already describes.

Covers the supervisord template, the ansible supervisord config and log file
task, both supervisord test fixtures and the api-tests expectation. The unit
test in managed/services/server/logs_test.go still expects the old name
because it runs against the prebuilt dev container; its TODO now points at
postgresql.log.

Also corrects the minPGVersion marker to name 15 as the pre-merge target.
countDatabases and countRoles become dbCount and roleCount.
CREATE USER, ALTER USER and CREATE DATABASE were built with fmt.Sprintf,
interpolating params.Username and params.Password into a double-quoted
identifier and a single-quoted literal. Either value could close its quote
and append further statements. initWithRoot connects as the postgres
superuser, and these calls pass no bind arguments, so lib/pq uses the simple
query protocol, which accepts several statements in one Exec: a username of

    inj" LOGIN PASSWORD 'x'; CREATE ROLE pwned SUPERUSER; --

creates a superuser role, and the equivalent payload in the password does so
while initWithRoot still returns nil. gosec does not catch this because G201
only matches DML keywords, not CREATE.

These statements take no bind parameters, so quote the values with
pq.QuoteIdentifier and pq.QuoteLiteral. This also fixes ordinary passwords
containing a quote, which previously failed with a syntax error.

The values come from PMM_POSTGRES_USERNAME and PMM_POSTGRES_DBPASSWORD, so
this is not remotely reachable; the CREATE USER site predates the PostgreSQL
18 work, while the ALTER USER one was added by it.
@coderabbitai

coderabbitai Bot commented Jul 30, 2026

Copy link
Copy Markdown

Review Change Stack

Walkthrough

This change upgrades embedded PostgreSQL from version 14 to 18. It adds staged migration and fresh-cluster initialization, updates runtime paths and ownership provisioning, and revises documentation and log expectations.

Changes

PostgreSQL 18 upgrade

Layer / File(s) Summary
Migration and cluster initialization
build/ansible/roles/postgres/files/postgres-migration, build/docker/server/entrypoint.sh
Adds staged PostgreSQL 14-to-18 migration, fresh PostgreSQL 18 initialization, password storage, pg_stat_statements setup, and revised entrypoint ordering.
PostgreSQL 18 deployment and runtime wiring
build/ansible/roles/postgres/*, build/ansible/roles/supervisord/*, managed/services/supervisord/*, managed/testdata/supervisord.d/*, .devcontainer/setup.sh
Switches packages, binaries, data directories, supervisor commands, backup and restore paths, and log paths to PostgreSQL 18 conventions.
Database ownership provisioning
build/ansible/roles/initialization/tasks/main.yml, build/ansible/roles/postgres/tasks/main.yml, managed/models/database.go, documentation/docs/reference/third-party/postgresql.md
Creates pmm-managed and grafana databases with explicit owners. Updates HA root initialization to use context-aware database operations.
Migration and prerequisite documentation
build/docs/MIGRATION.md, documentation/docs/reference/third-party/postgresql.md
Documents corrected PostgreSQL 11-to-14 paths and the PostgreSQL 14-to-18 migration, restoration, publication, rollback, and startup flow.
Log and configuration validation
api-tests/server/logs_test.go, managed/services/server/logs_test.go, managed/models/database.go
Updates downloaded-log expectations and records temporary test compatibility for versioned PostgreSQL log output.

Sequence Diagram(s)

sequenceDiagram
  participant ServerEntrypoint
  participant PostgresMigration
  participant PostgreSQL14
  participant PostgreSQL18
  ServerEntrypoint->>PostgresMigration: run embedded database setup
  PostgresMigration->>PostgreSQL14: dump available databases
  PostgresMigration->>PostgreSQL18: initialize staged cluster
  PostgresMigration->>PostgreSQL18: recreate roles, restore dumps, enable pg_stat_statements
  PostgresMigration->>ServerEntrypoint: publish PostgreSQL 18 cluster
Loading
🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely states that the internal PostgreSQL version is upgraded from 14 to 18.
Description check ✅ Passed The description includes the ticket, feature build, compatibility rationale, temporary CI values, and required pre-merge changes.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🧹 Nitpick comments (6)
build/docker/server/entrypoint.sh (1)

9-9: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

POSTGRES_DATA_DIR is hardcoded here but overridable in postgres-migration.

build/ansible/roles/postgres/files/postgres-migration declares this as "${POSTGRES_DATA_DIR:-/srv/postgres18}" (override-friendly), while this file hardcodes /srv/postgres18. If anything (e.g. tests) relies on overriding this path, the two scripts would silently disagree on the effective data directory.

♻️ Proposed fix
-declare POSTGRES_DATA_DIR="/srv/postgres18"
+declare POSTGRES_DATA_DIR="${POSTGRES_DATA_DIR:-/srv/postgres18}"
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@build/docker/server/entrypoint.sh` at line 9, Update the POSTGRES_DATA_DIR
declaration in the entrypoint script to use the existing environment value when
provided, falling back to /srv/postgres18 otherwise, matching the
override-friendly behavior of postgres-migration.
build/docs/MIGRATION.md (2)

65-68: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Add a language tag to this fenced code block.

Static analysis (MD040) flags this fence as missing a language identifier.

📝 Proposed fix
-3. Move the database directory to /srv/backup/postgres14
-```
+3. Move the database directory to /srv/backup/postgres14
+```sh
   mv /srv/postgres14 /srv/backup/
</details>

<details>
<summary>🤖 Prompt for AI Agents</summary>

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @build/docs/MIGRATION.md around lines 65 - 68, Update the fenced code block
under “Move the database directory to /srv/backup/postgres14” by adding the sh
language tag to its opening fence, leaving the command content unchanged.


</details>

<!-- cr-comment:v1:5a875c40fe6d38f0d3c89043 -->

_Source: Linters/SAST tools_

---

`99-152`: _📐 Maintainability & Code Quality_ | _🔵 Trivial_ | _⚡ Quick win_

**New v14→v18 migration section is accurate against the script; only missing code-fence languages.**

The described trigger conditions, dump/restore mechanics, role/database recreation, and rollback steps all match `postgres-migration` and `entrypoint.sh` as reviewed. Static analysis (MD040) flags four fences in this section (lines 111, 124, 131, 141) missing a language identifier — worth adding `sh`/`bash`/`sql` tags for consistency with the rest of the doc.

<details>
<summary>🤖 Prompt for AI Agents</summary>

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @build/docs/MIGRATION.md around lines 99 - 152, Add language identifiers to
the four code fences in the PostgreSQL v14-to-v18 migration section: use shell
syntax tags for command blocks and the appropriate SQL tag for SQL statements,
matching the existing documentation convention and satisfying MD040.


</details>

<!-- cr-comment:v1:d556a1a4c4b77d3e9ad54555 -->

_Source: Linters/SAST tools_

</blockquote></details>
<details>
<summary>build/ansible/roles/postgres/files/postgres-migration (3)</summary><blockquote>

`11-14`: _🔒 Security & Privacy_ | _🔵 Trivial_ | _⚡ Quick win_

**Brief permission-gap window when writing the password file.**

`echo -n ... > file` creates the file with default (umask-derived) permissions before `chmod 600` narrows them, leaving a short window where the password file is world/group-readable. Consider writing under a restrictive `umask` or using `install -m 600 /dev/null "$POSTGRES_PASSWORD_FILE"` first.

<details>
<summary>🔒️ Proposed fix</summary>

```diff
 store_postgres_password() {
-    echo -n "$1" > "$POSTGRES_PASSWORD_FILE"
-    chmod 600 "$POSTGRES_PASSWORD_FILE"
+    install -m 600 /dev/null "$POSTGRES_PASSWORD_FILE"
+    echo -n "$1" > "$POSTGRES_PASSWORD_FILE"
 }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@build/ansible/roles/postgres/files/postgres-migration` around lines 11 - 14,
Update store_postgres_password so the password file is created with restrictive
permissions before its contents are written, using a temporary umask or
pre-creating it with mode 600 via install. Preserve the existing password value
and final file location.

57-95: 🔒 Security & Privacy | 🔵 Trivial | ⚡ Quick win

Restrict permissions on the backup directory holding plain-text SQL dumps.

mkdir -p "$BACKUP_DIR" and pg_dump -f ... don't set restrictive permissions; the resulting dumps of pmm-managed/grafana may contain credentials or other sensitive data and end up readable per the container's default umask.

🔒️ Suggested fix
-    mkdir -p "$BACKUP_DIR"
+    install -d -m 700 "$BACKUP_DIR"
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@build/ansible/roles/postgres/files/postgres-migration` around lines 57 - 95,
Update the backup-directory setup around mkdir and the pg_dump calls to ensure
the directory and generated plain-text SQL dumps are restricted to the intended
owner, independent of the container umask. Apply restrictive permissions when
creating or preparing BACKUP_DIR and ensure each dump file is created with
owner-only access before restoration.

148-161: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Inconsistent binary path: /usr/bin/psql instead of $PG_BIN/psql.

Every other database connection in this file now goes through "$PG_BIN/..." (the earlier reviewer's suggestion to centralize binary paths), but line 153 still hardcodes /usr/bin/psql. If that path resolves to a different major-version client than the pg18 server, this is the one place the refactor was missed.

♻️ Proposed fix
-    PGPASSWORD="$POSTGRES_PASSWORD" /usr/bin/psql -U postgres -h /run/postgresql -d postgres \
+    PGPASSWORD="$POSTGRES_PASSWORD" "$PG_BIN/psql" -U postgres -h /run/postgresql -d postgres \
         -c "ALTER USER postgres WITH PASSWORD '${POSTGRES_PASSWORD}'"
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@build/ansible/roles/postgres/files/postgres-migration` around lines 148 -
161, Update the psql invocation in the migration function to use the centralized
"$PG_BIN/psql" path instead of hardcoded /usr/bin/psql, while preserving its
existing arguments and password-setting behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@build/ansible/roles/postgres/files/postgres-migration`:
- Around line 38-51: Update upgrade_pg14_to_pg18 to validate that
POSTGRES_PASSWORD_FILE exists and is readable before reading it or invoking
initdb_cluster. If it is missing, emit a clear migration-specific diagnostic and
return or fail explicitly, preventing initdb --pwfile from receiving an invalid
path; keep the existing password-loading flow for valid files.

In `@build/docker/server/entrypoint.sh`:
- Line 9: Update POSTGRES_DATA_DIR in build/docker/server/entrypoint.sh to use
the "${POSTGRES_DATA_DIR:-/srv/postgres18}" defaulting pattern, matching the
reference in build/ansible/roles/postgres/files/postgres-migration (lines 5-8),
which requires no change.

In `@documentation/docs/reference/third-party/postgresql.md`:
- Line 13: Update the PostgreSQL Docker examples in the PMM configuration guide
to use a PostgreSQL 18+ image tag instead of postgres:14, and change the pg_data
volume mount to the PostgreSQL 18 data directory layout. Apply both changes
together in every affected snippet.

In `@managed/services/supervisord/pmm_config.go`:
- Line 135: Update the PostgreSQL log filename expectation in logs_test.go from
postgresql14.log to postgresql.log to match the supervisord template, and
restore the temporary minPGVersion value in the database model as required by
the PR objective.

---

Nitpick comments:
In `@build/ansible/roles/postgres/files/postgres-migration`:
- Around line 11-14: Update store_postgres_password so the password file is
created with restrictive permissions before its contents are written, using a
temporary umask or pre-creating it with mode 600 via install. Preserve the
existing password value and final file location.
- Around line 57-95: Update the backup-directory setup around mkdir and the
pg_dump calls to ensure the directory and generated plain-text SQL dumps are
restricted to the intended owner, independent of the container umask. Apply
restrictive permissions when creating or preparing BACKUP_DIR and ensure each
dump file is created with owner-only access before restoration.
- Around line 148-161: Update the psql invocation in the migration function to
use the centralized "$PG_BIN/psql" path instead of hardcoded /usr/bin/psql,
while preserving its existing arguments and password-setting behavior.

In `@build/docker/server/entrypoint.sh`:
- Line 9: Update the POSTGRES_DATA_DIR declaration in the entrypoint script to
use the existing environment value when provided, falling back to
/srv/postgres18 otherwise, matching the override-friendly behavior of
postgres-migration.

In `@build/docs/MIGRATION.md`:
- Around line 65-68: Update the fenced code block under “Move the database
directory to /srv/backup/postgres14” by adding the sh language tag to its
opening fence, leaving the command content unchanged.
- Around line 99-152: Add language identifiers to the four code fences in the
PostgreSQL v14-to-v18 migration section: use shell syntax tags for command
blocks and the appropriate SQL tag for SQL statements, matching the existing
documentation convention and satisfying MD040.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 12dce150-fd4c-4375-b68d-348d09a0c4ee

📥 Commits

Reviewing files that changed from the base of the PR and between 65be72e and 6326518.

📒 Files selected for processing (17)
  • .devcontainer/setup.sh
  • api-tests/server/logs_test.go
  • build/ansible/roles/initialization/tasks/main.yml
  • build/ansible/roles/pmm-images/tasks/main.yml
  • build/ansible/roles/postgres/files/postgres-migration
  • build/ansible/roles/postgres/tasks/backup.yml
  • build/ansible/roles/postgres/tasks/main.yml
  • build/ansible/roles/postgres/tasks/restore.yml
  • build/ansible/roles/supervisord/files/pmm.ini
  • build/docker/server/entrypoint.sh
  • build/docs/MIGRATION.md
  • documentation/docs/reference/third-party/postgresql.md
  • managed/models/database.go
  • managed/services/server/logs_test.go
  • managed/services/supervisord/pmm_config.go
  • managed/testdata/supervisord.d/pmm-ch_low_memory.ini
  • managed/testdata/supervisord.d/pmm-db_enabled.ini

Comment thread build/ansible/roles/postgres/files/postgres-migration
Comment thread build/docker/server/entrypoint.sh

## Prerequisites
Before configuring PMM with an external PostgreSQL database, ensure you have a PostgreSQL 14+ server accessible from your PMM Server.
Before configuring PMM with an external PostgreSQL database, ensure you have a PostgreSQL 18+ server accessible from your PMM Server.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
rg -n 'postgres:14|/var/lib/postgresql/data' documentation/docs/reference/third-party/postgresql.md

Repository: percona/pmm

Length of output: 320


🏁 Script executed:

#!/bin/bash
set -euo pipefail
sed -n '1,220p' documentation/docs/reference/third-party/postgresql.md | cat -n

Repository: percona/pmm

Length of output: 14943


🌐 Web query:

PostgreSQL 18 official Docker image default data directory PGDATA /var/lib/postgresql/18/data

💡 Result:

For the official PostgreSQL 18 Docker image, the default PGDATA environment variable is set to /var/lib/postgresql/18/docker [1][2]. This represents a change from previous versions of the official image. Starting with PostgreSQL 18, the image's PGDATA path was updated to be version-specific to match the pg_ctlcluster standard directory structure [1][3]. Additionally, the volume defined in the Dockerfile for PostgreSQL 18 and later is /var/lib/postgresql [1][2], rather than /var/lib/postgresql/data used in earlier releases [1][4]. Users are advised to target their mounts and volumes at /var/lib/postgresql to facilitate easier upgrades between PostgreSQL major releases using features like pg_upgrade --link [1][5]. For users who prefer to maintain the older directory structure or who are migrating existing data, PGDATA can be set explicitly via environment variables [1][3].

Citations:


Update the PostgreSQL example to match the 18+ requirement The Docker snippets still use postgres:14 and -v pg_data:/var/lib/postgresql/data, so the guide currently tells users to provision an unsupported version and the wrong volume layout for PostgreSQL 18. Update the image tag and mount path together.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@documentation/docs/reference/third-party/postgresql.md` at line 13, Update
the PostgreSQL Docker examples in the PMM configuration guide to use a
PostgreSQL 18+ image tag instead of postgres:14, and change the pg_data volume
mount to the PostgreSQL 18 data directory layout. Apply both changes together in
every affected snippet.

Comment thread managed/services/supervisord/pmm_config.go
The Helm chart test that installs the last released PMM 2, upgrades to v3
and uninstalls left the pmm container in CrashLoopBackOff:

    PostgreSQL 14 data found at /srv/postgres14. Migrating to PostgreSQL 18...
    cat: /srv/.postgres_password: No such file or directory

/srv/.postgres_password only arrived in PMM 3.7, so a data directory from
PMM 2 does not have one, and reading it unconditionally under errexit killed
the script and, with it, the entrypoint. Generate a password when the file is
missing, since initdb seeds the new cluster's postgres role from it; an
existing file is left alone so the superuser password is preserved. The dump
itself needs no password because it connects over the socket, which pg_hba
matches with a local trust rule.

Dumping also assumed both databases exist. A PMM 2 installation need not have
kept Grafana in PostgreSQL, and pg_dump of a missing database aborted the
upgrade the same way, so dump only the databases that are present. This
mirrors the restore loop, which already skips absent dumps.

Verified against real clusters for a PMM 2 directory with both databases, one
without grafana, a PMM 3.7+ directory with a password file, and an empty
/srv: all complete, restore the data with the right owners, produce a
scram-only pg_hba, and are no-ops on re-run.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (3)
build/ansible/roles/postgres/files/postgres-migration (2)

89-93: 🩺 Stability & Availability | 🟠 Major | 🏗️ Heavy lift

Make failed migrations retryable.

initdb_cluster creates /srv/postgres18 before restoration completes. Any failure or interruption before Line 120 leaves that directory behind; the next run returns at Line 43 and can start an incomplete pg18 cluster while pg14 remains. Stage the new cluster in a temporary directory and rename it into place only after a successful restore, or clean up the newly-created pg18 directory on failure.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@build/ansible/roles/postgres/files/postgres-migration` around lines 89 - 93,
Update the migration flow around initdb_cluster and the pg_ctl startup/restore
sequence so a failed or interrupted restoration cannot leave /srv/postgres18 as
a reusable partial cluster. Stage initialization in a temporary directory and
atomically rename it into the final location only after restoration succeeds, or
ensure failure cleanup removes only the newly created cluster while preserving
existing pg14 data.

109-109: 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

Abort on the first SQL restore error.
psql -f can continue after SQL errors unless ON_ERROR_STOP is set, which risks a partial restore before the pg14 directory is retired. Add -v ON_ERROR_STOP=1.

Proposed fix
-        "$PG_BIN/psql" -h /run/postgresql -U postgres -d "$db" -f "$dump"
+        "$PG_BIN/psql" -v ON_ERROR_STOP=1 -h /run/postgresql -U postgres -d "$db" -f "$dump"
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@build/ansible/roles/postgres/files/postgres-migration` at line 109, Update
the psql restore invocation to pass the ON_ERROR_STOP variable set to 1,
ensuring the dump process aborts on the first SQL error while preserving the
existing host, user, database, and dump-file arguments.
build/docs/MIGRATION.md (1)

88-89: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Restore these SQL dumps with psql. pg_dump writes plain-text .sql files here, so pg_restore --file=... will not load them. Use psql -X --dbname=pmm-managed -f /srv/backup/pmm-managed.sql and psql -X --dbname=grafana -f /srv/backup/grafana.sql instead.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@build/docs/MIGRATION.md` around lines 88 - 89, Replace the two pg_restore
commands in the migration instructions with psql commands using -X, the
respective pmm-managed and grafana databases via --dbname, and the matching SQL
dump via -f; preserve the existing PGPASSWORD and connection context.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@build/docs/MIGRATION.md`:
- Around line 122-123: Update the migration instructions so the missing
/srv/.postgres_password generation condition is performed before Step 1, before
the pg14 startup and dump commands read it. Keep the existing-password path
unchanged and remove the duplicated or later-only generation step from Step 2,
ensuring pre-PMM 3.7 volumes follow the documented sequence.

---

Outside diff comments:
In `@build/ansible/roles/postgres/files/postgres-migration`:
- Around line 89-93: Update the migration flow around initdb_cluster and the
pg_ctl startup/restore sequence so a failed or interrupted restoration cannot
leave /srv/postgres18 as a reusable partial cluster. Stage initialization in a
temporary directory and atomically rename it into the final location only after
restoration succeeds, or ensure failure cleanup removes only the newly created
cluster while preserving existing pg14 data.
- Line 109: Update the psql restore invocation to pass the ON_ERROR_STOP
variable set to 1, ensuring the dump process aborts on the first SQL error while
preserving the existing host, user, database, and dump-file arguments.

In `@build/docs/MIGRATION.md`:
- Around line 88-89: Replace the two pg_restore commands in the migration
instructions with psql commands using -X, the respective pmm-managed and grafana
databases via --dbname, and the matching SQL dump via -f; preserve the existing
PGPASSWORD and connection context.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: dbb68baa-6c36-4248-a37a-4e70e5649abe

📥 Commits

Reviewing files that changed from the base of the PR and between 6326518 and 7e21ec6.

📒 Files selected for processing (2)
  • build/ansible/roles/postgres/files/postgres-migration
  • build/docs/MIGRATION.md

Comment thread build/docs/MIGRATION.md Outdated
The renamed /srv/postgres14.old is the rollback artifact and is a complete
copy of the old cluster, so keeping the plain-text dumps as well leaves a
second copy of every database sitting in /srv for no benefit.

Delete them after the rename. Only the pg18-upgrade-*.sql files written by
this script are removed, since /srv/backup also holds user backups taken
through the backup playbook. The rename is the last step before this, and
errexit aborts earlier on any failure, so a failed upgrade still leaves its
dumps behind to inspect.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@build/ansible/roles/postgres/files/postgres-migration`:
- Around line 125-126: Restrict the cleanup in the postgres-migration upgrade
flow to dump files created by the current invocation instead of using the
unscoped "${BACKUP_DIR}"/pg18-upgrade-*.sql wildcard. Track each exact dump path
or place upgrade artifacts in a dedicated temporary directory, then remove only
those paths after the rename while leaving unrelated user backups untouched.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: c570f345-b1bc-4570-b1da-85dceebf3052

📥 Commits

Reviewing files that changed from the base of the PR and between 7e21ec6 and d82c74c.

📒 Files selected for processing (2)
  • build/ansible/roles/postgres/files/postgres-migration
  • build/docs/MIGRATION.md
🚧 Files skipped from review as they are similar to previous changes (1)
  • build/docs/MIGRATION.md

Comment thread build/ansible/roles/postgres/files/postgres-migration
Both cluster-creation paths wrote straight into /srv/postgres18, so an
interrupted attempt left a half-built directory behind. Since the upgrade is
guarded on that directory being absent and the fresh install on it being
present, neither would run again: the next start came up on an empty cluster
while the PostgreSQL 14 data sat untouched, with no retry and no complaint.

Create the cluster in /srv/postgres18.new and move it into place once it is
finished, which makes the directory's existence mean "completed" rather than
"attempted". The move is a rename within /srv, so it is atomic, and a stale
staging directory from an earlier attempt is discarded before retrying.

initdb_cluster now takes the target directory, and publish_cluster performs
the move. Verified by failing the upgrade just after initdb: nothing is
published, and the next run completes and restores the data.
@ademidoff

Copy link
Copy Markdown
Member

@coderabbitai configuration

@coderabbitai

coderabbitai Bot commented Jul 31, 2026

Copy link
Copy Markdown

Configuration used: Organization UI

CodeRabbit configuration
# Source: Organization UI (base)
language: en-US
# Source: Organization UI (base)
tone_instructions: ''
# Source: Organization UI (base)
early_access: false
# Source: Organization UI (base)
enable_free_tier: true
# Source: defaults
inheritance: false
reviews:
  # Source: Organization UI (base)
  profile: chill
  # Source: Organization UI (base)
  request_changes_workflow: false
  # Source: Organization UI (base)
  high_level_summary: true
  # Source: defaults
  high_level_summary_instructions: ''
  # Source: Organization UI (base)
  high_level_summary_placeholder: '`@coderabbitai` summary'
  # Source: Organization UI (base)
  high_level_summary_in_walkthrough: false
  # Source: Organization UI (base)
  auto_title_placeholder: '`@coderabbitai`'
  # Source: Organization UI (base)
  auto_title_instructions: ''
  # Source: Organization UI (base)
  review_status: true
  # Source: defaults
  review_details: false
  # Source: defaults
  review_progress: true
  # Source: Organization UI (base)
  commit_status: true
  # Source: Organization UI (base)
  fail_commit_status: false
  # Source: Organization UI (base)
  collapse_walkthrough: false
  # Source: Organization UI (base)
  changed_files_summary: true
  # Source: Organization UI (base)
  sequence_diagrams: true
  # Source: defaults
  estimate_code_review_effort: true
  # Source: Organization UI (base)
  assess_linked_issues: true
  # Source: Organization UI (base)
  related_issues: true
  # Source: Organization UI (base)
  related_prs: true
  # Source: Organization UI (base)
  suggested_labels: true
  # Source: defaults
  labeling_instructions: []
  # Source: defaults
  mutually_exclusive_groups: {}
  # Source: Organization UI (base)
  auto_apply_labels: false
  # Source: Organization UI (base)
  suggested_reviewers: true
  # Source: Organization UI (base)
  auto_assign_reviewers: false
  # Source: defaults
  suggested_reviewers_instructions: []
  # Source: defaults
  in_progress_fortune: true
  # Source: Organization UI (base)
  poem: true
  # Source: defaults
  enable_prompt_for_ai_agents: true
  # Source: defaults
  path_filters: []
  # Source: defaults
  path_instructions: []
  # Source: Organization UI (base)
  abort_on_close: true
  # Source: Organization UI (base)
  disable_cache: false
  slop_detection:
    # Source: defaults
    enabled: true
  auto_review:
    # Source: Organization UI (base)
    enabled: true
    # Source: defaults
    description_keyword: ''
    # Source: Organization UI (base)
    auto_incremental_review: true
    # Source: defaults
    auto_pause_after_reviewed_commits: 5
    # Source: defaults
    ignore_title_keywords: []
    # Source: defaults
    labels: []
    # Source: Organization UI (base)
    drafts: false
    # Source: defaults
    base_branches: []
    # Source: defaults
    ignore_usernames: []
  finishing_touches:
    docstrings:
      # Source: Organization UI (base)
      enabled: true
    unit_tests:
      # Source: Organization UI (base)
      enabled: true
    simplify:
      # Source: defaults
      enabled: false
    autofix:
      # Source: defaults
      enabled: true
    fix_ci:
      # Source: defaults
      enabled: true
    resolve_merge_conflict:
      # Source: defaults
      enabled: true
    # Source: defaults
    custom: []
  pre_merge_checks:
    # Source: defaults
    override_requested_reviewers_only: false
    docstrings:
      # Source: defaults
      mode: warning
      # Source: defaults
      threshold: 80
    title:
      # Source: defaults
      mode: warning
      # Source: defaults
      requirements: ''
    description:
      # Source: defaults
      mode: warning
    issue_assessment:
      # Source: defaults
      mode: warning
    # Source: defaults
    custom_checks: []
  # Source: defaults
  post_merge_actions: []
  tools:
    ast-grep:
      # Source: defaults
      rule_dirs: []
      # Source: defaults
      util_dirs: []
      # Source: Organization UI (base)
      essential_rules: true
      # Source: defaults
      packages: []
    shellcheck:
      # Source: Organization UI (base)
      enabled: true
    ruff:
      # Source: Organization UI (base)
      enabled: true
    markdownlint:
      # Source: Organization UI (base)
      enabled: true
    github-checks:
      # Source: Organization UI (base)
      enabled: true
      # Source: Organization UI (base)
      timeout_ms: 90000
    languagetool:
      # Source: Organization UI (base)
      enabled: true
      # Source: defaults
      enabled_rules: []
      # Source: defaults
      disabled_rules: []
      # Source: defaults
      enabled_categories: []
      # Source: defaults
      disabled_categories: []
      # Source: Organization UI (base)
      enabled_only: false
      # Source: Organization UI (base)
      level: default
    biome:
      # Source: Organization UI (base)
      enabled: true
    hadolint:
      # Source: Organization UI (base)
      enabled: true
    swiftlint:
      # Source: Organization UI (base)
      enabled: true
    phpstan:
      # Source: Organization UI (base)
      enabled: true
      # Source: Organization UI (base)
      level: default
    phpmd:
      # Source: defaults
      enabled: true
    phpcs:
      # Source: defaults
      enabled: true
    golangci-lint:
      # Source: Organization UI (base)
      enabled: true
    yamllint:
      # Source: Organization UI (base)
      enabled: true
    gitleaks:
      # Source: Organization UI (base)
      enabled: true
    trufflehog:
      # Source: defaults
      enabled: true
    checkov:
      # Source: Organization UI (base)
      enabled: true
    tflint:
      # Source: defaults
      enabled: true
    detekt:
      # Source: Organization UI (base)
      enabled: true
    eslint:
      # Source: Organization UI (base)
      enabled: true
      e18e:
        # Source: defaults
        enabled: true
    flake8:
      # Source: defaults
      enabled: true
    fbinfer:
      # Source: defaults
      enabled: true
      # Source: defaults
      enable_java: false
    fortitudeLint:
      # Source: defaults
      enabled: true
    rubocop:
      # Source: Organization UI (base)
      enabled: true
    buf:
      # Source: Organization UI (base)
      enabled: true
    regal:
      # Source: Organization UI (base)
      enabled: true
    actionlint:
      # Source: Organization UI (base)
      enabled: true
    zizmor:
      # Source: defaults
      enabled: true
    pmd:
      # Source: Organization UI (base)
      enabled: true
    clang:
      # Source: defaults
      enabled: true
    cppcheck:
      # Source: Organization UI (base)
      enabled: true
    opengrep:
      # Source: defaults
      enabled: true
    semgrep:
      # Source: Organization UI (base)
      enabled: true
    circleci:
      # Source: Organization UI (base)
      enabled: true
    clippy:
      # Source: Organization UI (base)
      enabled: true
    sqlfluff:
      # Source: Organization UI (base)
      enabled: true
    squawk:
      # Source: defaults
      enabled: true
    trivy:
      # Source: defaults
      enabled: true
    prismaLint:
      # Source: Organization UI (base)
      enabled: true
    pylint:
      # Source: Organization UI (base)
      enabled: true
    oxc:
      # Source: Organization UI (base)
      enabled: true
    shopifyThemeCheck:
      # Source: Organization UI (base)
      enabled: true
    luacheck:
      # Source: Organization UI (base)
      enabled: true
    brakeman:
      # Source: Organization UI (base)
      enabled: true
    dotenvLint:
      # Source: defaults
      enabled: true
    htmlhint:
      # Source: defaults
      enabled: true
    stylelint:
      # Source: defaults
      enabled: true
    checkmake:
      # Source: defaults
      enabled: true
    osvScanner:
      # Source: defaults
      enabled: true
    oasdiff:
      # Source: defaults
      enabled: true
    reactDoctor:
      # Source: defaults
      enabled: true
    presidio:
      # Source: defaults
      enabled: true
    blinter:
      # Source: defaults
      enabled: true
    smartyLint:
      # Source: defaults
      enabled: true
    emberTemplateLint:
      # Source: defaults
      enabled: true
    skillspector:
      # Source: defaults
      enabled: true
    psscriptanalyzer:
      # Source: defaults
      enabled: true
chat:
  # Source: defaults
  art: true
  # Source: defaults
  allow_non_org_members: true
  # Source: Organization UI (base)
  auto_reply: true
  integrations:
    jira:
      # Source: Organization UI (base)
      usage: auto
    linear:
      # Source: Organization UI (base)
      usage: auto
knowledge_base:
  # Source: Organization UI (base)
  opt_out: false
  web_search:
    # Source: Organization UI (base)
    enabled: true
  code_guidelines:
    # Source: defaults
    enabled: true
    # Source: defaults
    filePatterns: []
  learnings:
    # Source: Organization UI (base)
    scope: auto
    # Source: defaults
    approval_delay: 0
  issues:
    # Source: Organization UI (base)
    scope: auto
  jira:
    # Source: Organization UI (base)
    usage: auto
    # Source: defaults
    project_keys: []
    # Source: defaults
    excluded_project_keys: []
  linear:
    # Source: Organization UI (base)
    usage: auto
    # Source: defaults
    team_keys: []
  pull_requests:
    # Source: Organization UI (base)
    scope: auto
  mcp:
    # Source: defaults
    usage: auto
    # Source: defaults
    disabled_servers: []
  # Source: defaults
  automatic_repository_linking: false
  # Source: defaults
  linked_repositories: []
code_generation:
  docstrings:
    # Source: Organization UI (base)
    language: en-US
    # Source: defaults
    path_instructions: []
  unit_tests:
    # Source: defaults
    path_instructions: []
issue_enrichment:
  auto_enrich:
    # Source: defaults
    enabled: false
  planning:
    # Source: defaults
    enabled: true
    auto_planning:
      # Source: defaults
      enabled: true
      # Source: defaults
      labels: []
  labeling:
    # Source: defaults
    labeling_instructions: []
    # Source: defaults
    auto_apply_labels: false

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (3)
build/ansible/roles/postgres/files/postgres-migration (2)

85-98: 🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift

Do not reuse retained dumps on a retry.

When grafana is absent or uses an external database, the dump loop skips the current file. The restore loop still consumes any existing pg18-upgrade-grafana.sql. A failed earlier attempt can therefore restore stale data into the new cluster. Keep dumps in an invocation-specific directory, or restore only files created by the current invocation. This must be corrected before merge.

Also applies to: 115-125

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@build/ansible/roles/postgres/files/postgres-migration` around lines 85 - 98,
The migration script’s dump and restore flows can reuse stale retained database
dumps across retries, especially for the skipped grafana database. Update the
dump/restore logic around the database loop and restore section to use an
invocation-specific backup directory or track only files created during the
current invocation, ensuring skipped databases are never restored from prior
attempts.

121-125: 🗄️ Data Integrity & Integration | 🔴 Critical | ⚡ Quick win

Make the migration fail closed on SQL errors.

Without ON_ERROR_STOP, psql -f continues after SQL errors and does not return the script-error status required by errexit. The migration can then publish a partial staged cluster at postgres-migration#L135.

  • Add -v ON_ERROR_STOP=1 to the role/database setup and dump restore commands.
  • Apply the same option to both psql commands in build/docs/MIGRATION.md#L133-L136.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@build/ansible/roles/postgres/files/postgres-migration` around lines 121 -
125, Add -v ON_ERROR_STOP=1 to both psql commands in
build/ansible/roles/postgres/files/postgres-migration, covering role/database
setup and dump restoration. Apply the same option to both corresponding psql
commands in build/docs/MIGRATION.md. Ensure all four commands stop and propagate
SQL errors.
build/docs/MIGRATION.md (1)

88-89: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Restore the plain-text dumps with psql.

pg_dump without -F creates plain-text SQL files. pg_restore cannot read them. Use psql --dbname=<database> --file=<dump> --set=ON_ERROR_STOP=on for both databases. Make it so.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@build/docs/MIGRATION.md` around lines 88 - 89, Replace both pg_restore
commands in the migration instructions with psql commands, targeting the
postgres database and grafana database respectively. Use the existing dump files
as input via --file, preserve the PostgreSQL connection settings and PGPASSWORD
environment variable, and enable --set=ON_ERROR_STOP=on for each restore.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@build/docs/MIGRATION.md`:
- Line 105: Update the migration retry description near the one-time upgrade
behavior to state that an interrupted attempt can leave both the staging
directory and previously written dumps under /srv/backup, rather than saying it
leaves only a staging directory. Preserve the existing retry and cleanup
behavior described elsewhere.
- Around line 124-127: Update the opening fences for all three command blocks in
MIGRATION.md, including the blocks around the shown install/initdb commands and
the sections at the referenced locations, from untyped fences to ```bash so
markdownlint MD040 passes.

---

Outside diff comments:
In `@build/ansible/roles/postgres/files/postgres-migration`:
- Around line 85-98: The migration script’s dump and restore flows can reuse
stale retained database dumps across retries, especially for the skipped grafana
database. Update the dump/restore logic around the database loop and restore
section to use an invocation-specific backup directory or track only files
created during the current invocation, ensuring skipped databases are never
restored from prior attempts.
- Around line 121-125: Add -v ON_ERROR_STOP=1 to both psql commands in
build/ansible/roles/postgres/files/postgres-migration, covering role/database
setup and dump restoration. Apply the same option to both corresponding psql
commands in build/docs/MIGRATION.md. Ensure all four commands stop and propagate
SQL errors.

In `@build/docs/MIGRATION.md`:
- Around line 88-89: Replace both pg_restore commands in the migration
instructions with psql commands, targeting the postgres database and grafana
database respectively. Use the existing dump files as input via --file, preserve
the PostgreSQL connection settings and PGPASSWORD environment variable, and
enable --set=ON_ERROR_STOP=on for each restore.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 88aeeb40-a701-4db0-b2c7-5d541ffdee7f

📥 Commits

Reviewing files that changed from the base of the PR and between d82c74c and d4e5517.

📒 Files selected for processing (2)
  • build/ansible/roles/postgres/files/postgres-migration
  • build/docs/MIGRATION.md
🔗 Linked repositories identified

CodeRabbit considers these linked repositories for cross-repo context during reviews:

  • percona/pmm-qa (manual)
  • percona/pmm (manual)

Comment thread build/docs/MIGRATION.md

The upgrade is performed automatically on the first start of a PMM Server that ships PostgreSQL 18, so no manual intervention is required. It is driven by `build/docker/server/entrypoint.sh`, which runs `build/ansible/roles/postgres/files/postgres-migration` before supervisord starts. The steps below document what that script does.

The upgrade runs only when `/srv/postgres14` exists and `/srv/postgres18` does not, which makes it a one-time operation. The new cluster is built in `/srv/postgres18.new` and moved into place only once it is fully restored, so `/srv/postgres18` existing always means a finished cluster. An interrupted attempt leaves nothing but a staging directory, which the next start discards before retrying. It is skipped entirely when the embedded PostgreSQL is not in use, i.e. when `PMM_HA_ENABLE` or `PMM_DISABLE_BUILTIN_POSTGRES` is enabled. In those cases the external database has to be upgraded by its owner.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Describe retained dumps in the retry behavior.

An interrupted run can leave the staging directory and dumps already written under /srv/backup. Line 152 confirms that failed dumps remain. Replace “nothing but a staging directory” with wording that names both retained artifacts.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@build/docs/MIGRATION.md` at line 105, Update the migration retry description
near the one-time upgrade behavior to state that an interrupted attempt can
leave both the staging directory and previously written dumps under /srv/backup,
rather than saying it leaves only a staging directory. Preserve the existing
retry and cleanup behavior described elsewhere.

Comment thread build/docs/MIGRATION.md
Comment on lines +124 to +127
```
install -d -m 750 /srv/postgres18.new
/usr/pgsql-18/bin/initdb -D /srv/postgres18.new --auth-host=scram-sha-256 --auth-local=trust --username=postgres --pwfile=/srv/.postgres_password
```

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Specify the language for each new fenced command block.

markdownlint-cli2 reports MD040 at Lines 124, 131, and 148. Change each opening fence to ```bash.

Also applies to: 131-137, 148-150

🧰 Tools
🪛 markdownlint-cli2 (0.23.2)

[warning] 124-124: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@build/docs/MIGRATION.md` around lines 124 - 127, Update the opening fences
for all three command blocks in MIGRATION.md, including the blocks around the
shown install/initdb commands and the sections at the referenced locations, from
untyped fences to ```bash so markdownlint MD040 passes.

Source: Linters/SAST tools

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Documentation changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants