feat(go.d.plugin/postgres): add running-queries and pg_stat_monitor support

[ktsaou]

Summary

• Add running-queries function for PostgreSQL using pg_stat_activity

  • Shows currently executing queries in real-time
  • 24 columns with 12 visible by default (PID, database, user, application, query, duration, state, wait events)

• Add pg_stat_monitor auto-detection for top-queries

  • Automatically uses pg_stat_monitor when available (Percona distribution)
  • Falls back to pg_stat_statements when pg_stat_monitor is not installed
  • 12 additional columns with pg_stat_monitor: application name, client IP, CPU time, error info (elevel, sqlcode, message), command type, relations, comments

Test plan

• E2E tests pass for pg_stat_statements (PostgreSQL 14, 15, 16, 17, 18)
• E2E tests pass for pg_stat_monitor (Percona 16, 17)
• Unit tests pass
• Manual verification in Netdata dashboard

---

Summary by cubic

Adds a new PostgreSQL running-queries function (pg_stat_activity) and auto-detection of pg_stat_monitor for top-queries, with fallback to pg_stat_statements. Unlocks richer top-queries data when pg_stat_monitor is present.

• New Features

  • Running Queries: real-time active queries from pg_stat_activity with duration, state, wait events, and query text (version-aware).
  • Top Queries: auto-detects pg_stat_monitor and uses it when available; falls back to pg_stat_statements.
    • Extra pg_stat_monitor columns: application name, client IP, CPU user/system time, error info (elevel, sqlcode, message), command type, relations, comments, toplevel, bucket start time.

• Migration

  • To use top-queries:
    • pg_stat_monitor (recommended): enable shared_preload_libraries=pg_stat_monitor, restart, then CREATE EXTENSION pg_stat_monitor;
    • or pg_stat_statements: enable shared_preload_libraries=pg_stat_statements, restart, then CREATE EXTENSION pg_stat_statements.

^{Written for commit 40e2bbd. Summary will update on new commits.}

[ilyam8]

@ktsaou dont merge until I update config options

[cubic-dev-ai]

1 issue found across 12 files

Confidence score: 3/5

• There is a concrete regression risk: buildDynamicSQL in src/go/plugin/go.d/collector/postgres/func_top_queries.go still rewrites total_time/mean_time to total_exec_time for PG 13+, which will break pg_stat_monitor queries now routed through this function.
• Given the medium severity (6/10) and direct impact on query metrics collection, this PR carries some user-facing risk and warrants caution.
• Pay close attention to src/go/plugin/go.d/collector/postgres/func_top_queries.go - ensure pg_stat_monitor column names are preserved for correct SQL generation.

Prompt for AI agents (all issues)

Check if these issues are valid — if so, understand the root cause of each and fix them.


<file name="src/go/plugin/go.d/collector/postgres/func_top_queries.go">

<violation number="1" location="src/go/plugin/go.d/collector/postgres/func_top_queries.go:284">
P2: pg_stat_monitor uses total_time/mean_time column names, but buildDynamicSQL still remaps them to total_exec_time for PG 13+. Now that pg_stat_monitor is routed through this function, those rewritten column names won’t exist and the query will fail on PG 13+. Skip the pg_stat_statements rename logic when source is pg_stat_monitor or map to the correct pg_stat_monitor column names.</violation>
</file>

Architecture diagram

sequenceDiagram
    participant UI as Client
    participant Func as Collector (Go)
    participant DB as PostgreSQL

    Note over UI,DB: CHANGED: Top Queries (Auto-detection Strategy)

    UI->>Func: Call "top-queries"
    
    Func->>Func: Check cached stats source
    
    opt Source not yet detected
        Func->>DB: NEW: Check for 'pg_stat_monitor' extension
        alt pg_stat_monitor available (Percona)
            DB-->>Func: true
            Func->>Func: Set source = pg_stat_monitor
            Func->>DB: NEW: Detect available monitor columns
        else Standard Fallback
            DB-->>Func: false
            Func->>DB: Check for 'pg_stat_statements' extension
            Func->>Func: Set source = pg_stat_statements
        end
    end

    alt source == pg_stat_monitor
        Func->>DB: NEW: SELECT ... FROM pg_stat_monitor
        Note right of DB: Fetch extra cols: Client IP, App Name,<br/>CPU time, Error codes, Relations
    else source == pg_stat_statements
        Func->>DB: SELECT ... FROM pg_stat_statements
    end
    
    DB-->>Func: Statistics Rows
    Func-->>UI: Return Data

    Note over UI,DB: NEW: Running Queries Feature

    UI->>Func: Call "running-queries"
    Func->>DB: NEW: SELECT ... FROM pg_stat_activity
    Note right of DB: Fetch active PIDs, state,<br/>wait events, durations
    DB-->>Func: Active Sessions
    Func-->>UI: Return Data

Loading

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}

[Copilot]

Pull request overview

This PR extends the PostgreSQL go.d collector with richer query introspection: it adds a new running-queries function backed by pg_stat_activity, and enhances top-queries to auto-detect and prefer pg_stat_monitor (with fallback to pg_stat_statements), plus wiring for E2E validation across multiple Postgres distributions.

Changes:

• Add a new running-queries function and handler for PostgreSQL that exposes live query/session data from pg_stat_activity, with version-aware columns and configurable sorting.
• Enhance top-queries to auto-detect pg_stat_monitor vs pg_stat_statements, dynamically detect per-source columns (including pg_stat_monitor-only fields like application, client IP, CPU time, errors), and update metadata/docs accordingly.
• Extend the functions-validation tooling (docker-compose, seed SQL, matrix scripts, E2E harness) to support testing both pg_stat_statements and pg_stat_monitor on upstream PostgreSQL and Percona images, and to validate the new running-queries method.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
src/go/tools/functions-validation/seed/postgres/init-pgsm.sql	New seed SQL to create the pg_stat_monitor extension and generate sample workload for top-queries E2E tests.
src/go/tools/functions-validation/e2e/postgres.sh	Adds env-driven selection of query-stats extension (pg_stat_statements vs pg_stat_monitor), chooses init script accordingly, verifies the chosen extension via returned columns, and exercises the new running-queries function.
src/go/tools/functions-validation/e2e/postgres-matrix.sh	Expands the test matrix to run E2E tests for multiple vanilla PostgreSQL versions with pg_stat_statements and Percona images with pg_stat_monitor.
src/go/tools/functions-validation/docker-compose.yml	Makes the Postgres container’s shared_preload_libraries configurable via POSTGRES_PRELOAD_LIBRARIES to support extension selection in tests.
src/go/plugin/go.d/collector/postgres/metadata.yaml	Updates the Postgres function metadata to describe dual pg_stat_monitor/pg_stat_statements support, documents new pg_stat_monitor-only columns and behavior, and adds full docs for the new running-queries function.
src/go/plugin/go.d/collector/postgres/functions_test.go	Adjusts tests for the expanded method list, new function behavior signatures, and the relaxed label expectations for sort options across methods.
src/go/plugin/go.d/collector/postgres/functions.go	Adds detection, column discovery, and caching for pg_stat_monitor and a generic query-stats source selector shared by top-queries.
src/go/plugin/go.d/collector/postgres/func_top_queries.go	Extends top-queries to support pg_stat_monitor-specific columns, dynamic SQL and column selection per source, auto-detected sort options, and improved chart/label metadata.
src/go/plugin/go.d/collector/postgres/func_running_queries.go	Implements the running-queries method for PostgreSQL, including column metadata, query construction against pg_stat_activity, result scanning, and formatting.
src/go/plugin/go.d/collector/postgres/func_router.go	Wires the new running-queries handler into the Postgres function router and method list.
src/go/plugin/go.d/collector/postgres/collector.go	Extends the collector state with caches and mutex protection for pg_stat_monitor and the selected query-stats source.
docs/functions/databases.md	Updates the cross-database functions documentation to mark PostgreSQL as supporting Running Queries and Error Info (with pg_stat_monitor).

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

src/go/plugin/go.d/collector/postgres/func_top_queries.go:666

• The source parameter on topQueriesSortParam is never used, even though all callers now pass it, which makes the signature misleading and slightly harder to maintain. Consider removing the unused parameter (and its argument at call sites) or using it to vary sort options by source if that was the intent.

func (f *funcTopQueries) topQueriesSortParam(queryCols []pgColumn, source queryStatsSourceName) (funcapi.ParamConfig, []funcapi.ParamOption) {
	sortOptions := f.buildDynamicSortOptions(queryCols)
	sortParam := funcapi.ParamConfig{
		ID:         paramSort,
		Name:       "Filter By",
		Help:       "Select the primary sort column",
		Selection:  funcapi.ParamSelect,
		Options:    sortOptions,
		UniqueView: true,

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated 1 comment.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[ilyam8]

@cubic-dev-ai review this PR

[cubic-dev-ai]

@cubic-dev-ai review this PR

@ilyam8 I have started the AI code review. It will take a few minutes to complete.

[cubic-dev-ai]

1 issue found across 14 files

Confidence score: 3/5

• Potential runtime failure: src/go/plugin/go.d/collector/postgres/func_top_queries.go uses s.datname for pg_stat_monitor even though the view lacks that column, which would break the query after dropping the pg_database join.
• Severity is moderate (6/10) and impacts query execution, so there’s some risk to user-facing metrics behavior.
• Pay close attention to src/go/plugin/go.d/collector/postgres/func_top_queries.go - pg_stat_monitor query may fail due to missing datname.

Prompt for AI agents (all issues)

Check if these issues are valid — if so, understand the root cause of each and fix them.


<file name="src/go/plugin/go.d/collector/postgres/func_top_queries.go">

<violation number="1" location="src/go/plugin/go.d/collector/postgres/func_top_queries.go:611">
P2: pg_stat_monitor does not expose a `datname` column; switching the database field to `s.datname` (and dropping the pg_database join) will make the pg_stat_monitor query fail at runtime. Keep using `d.datname` and join `pg_database` on `s.dbid` for the pg_stat_monitor path.</violation>
</file>

Architecture diagram

sequenceDiagram
    *   participant UI as Netdata UI / Cloud
    *   participant Manager as Agent Function Manager
    *   participant Coll as Postgres Collector
    *   participant DB as PostgreSQL Database
    *   participant Ext as PG Extensions (monitor/statements)

    Note over UI, Ext: Runtime Request Flow for Database Functions

    UI->>Manager: Request Function (top-queries OR running-queries)
    
    Manager->>Coll: Invoke Method Handler
    
    alt Method: top-queries
        Coll->>Coll: NEW: getQueryStatsSource()
        opt source not cached
            Coll->>DB: NEW: Check for pg_stat_monitor extension
            DB-->>Coll: Extension exists/not
            alt pg_stat_monitor exists
                Coll->>Coll: Set source = pg_stat_monitor
            else Fallback
                Coll->>DB: Check for pg_stat_statements extension
                DB-->>Coll: Extension exists/not
                Coll->>Coll: Set source = pg_stat_statements
            end
        end

        Coll->>Coll: NEW: detectColumns()
        opt columns not cached
            Coll->>DB: CHANGED: Query information_schema.columns for active source
            DB-->>Coll: List of available columns
        end

        Coll->>DB: CHANGED: Execute dynamic SQL on detected source/columns
        DB-->>Coll: Aggregated Query Stats
        
    else NEW Method: running-queries
        Coll->>DB: NEW: Query pg_stat_activity view
        Note right of DB: Fetches PID, query text, state,<br/>wait events, duration
        DB-->>Coll: Snapshot of currently active queries
    end

    Coll-->>Manager: Return Function Response (Status, Columns, Data)
    
    Manager->>Manager: CHANGED: Inject update_every into response JSON
    Manager-->>UI: 200 OK (Table Data)

    Note over Coll,DB: Auto-detection results & column metadata cached in-memory

Loading

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}

[Copilot]

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated 3 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 15 out of 15 changed files in this pull request and generated 1 comment.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.