Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions documentation/components/bridges/symfony-postgresql-bundle.md
Original file line number Diff line number Diff line change
Expand Up @@ -135,9 +135,22 @@ flow_postgresql:
profiler:
enabled: ~
include_parameters: true # show bound query parameters in the panel
max_queries: 1000 # retained queries; oldest dropped first
max_retained_parameters: 100 # queries binding more than this keep no parameters
max_query_length: 1000 # retained statements longer than this are truncated
migrations: true # show the Flow Migrations panel (requires migrations enabled)
```

The log is bounded: at most `max_queries` entries are retained and the panel notes when older ones were
dropped. Because bound parameters dominate memory, a query binding more than `max_retained_parameters`
values keeps none of them — such queries show "parameters omitted" and cannot be explained from the panel.
The all-or-nothing rule is deliberate, and the reason this option is not named like the per-connection
`telemetry.max_parameters` (which keeps the first N): the panel re-binds the retained parameters to run
EXPLAIN, and a partial list is not a valid query. Statements themselves are bounded by
`max_query_length` — a batch `INSERT` with a thousand `VALUES` tuples is over 100 KB of SQL, so retaining
`max_queries` of them uncut would cost hundreds of megabytes. Truncated statements are marked in the panel
and cannot be explained from it.

Recording is dev-only and adds nothing in production: when the profiler is disabled — or
WebProfilerBundle is absent in `enabled: ~` mode — no connection is decorated. A single connection
can opt out while the profiler is on:
Expand All @@ -150,6 +163,11 @@ flow_postgresql:
profiler: false # do not record queries in selected connection (default: true)
```

Recording follows `kernel.debug`, like DoctrineBundle's `dbal.profiling`. With `enabled: ~` the panel is
wired only when WebProfilerBundle is registered **and** the kernel runs in debug mode, so
`bin/console --no-debug` — which compiles a separate container — decorates no connection and records
nothing. `enabled: true` forces recording on regardless of debug mode.

When `migrations` are enabled, a separate **Flow Migrations** panel reports the migrations connection's
executed, pending and unavailable migrations (with execution time) — like the Doctrine Migrations
bundle's panel. It queries the database on every profiled request; set `profiler.migrations: false`
Expand Down
15 changes: 15 additions & 0 deletions documentation/upgrading.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,21 @@ rank()->over(window()->partitionBy(ref('dept'))->orderBy(ref('salary')->desc()))

Implementations must no longer sort; `$window->partition()` and `$window->frame()` are already ordered.

### 7) `flow-php/postgresql` - query recording moved to `flow-php/symfony-postgresql-bundle`

| Before | After |
|--------------------------------------------------|----------------------------------------------------------------------|
| `Flow\PostgreSql\Client\Debug\RecordingClient` | `Flow\Bridge\Symfony\PostgreSqlBundle\Profiler\ProfilerClient` |
| `Flow\PostgreSql\Client\Debug\QueryLog` | `Flow\Bridge\Symfony\PostgreSqlBundle\Profiler\QueryRecorder` |
| `Flow\PostgreSql\Client\Debug\QueryLogOptions` | `Flow\Bridge\Symfony\PostgreSqlBundle\Profiler\QueryRecorderOptions` |
| `Flow\PostgreSql\Client\Debug\RecordedQuery` | `Flow\Bridge\Symfony\PostgreSqlBundle\Profiler\RecordedQuery` |
| `QueryLogOptions::$maxParameters` | `QueryRecorderOptions::$maxRetainedParameters` |
| `QueryLogOptions::maxParameters()` | `QueryRecorderOptions::maxRetainedParameters()` |
| service `flow.postgresql.profiler.query_log` | `flow.postgresql.profiler.query_recorder` |
| config `flow_postgresql.profiler.max_parameters` | `flow_postgresql.profiler.max_retained_parameters` |

Applies to `flow-php/postgresql` users only through the bundle; `Client\Telemetry` is unchanged.

---

## Upgrading from 0.41.x to 0.42.x
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
use Flow\Bridge\Symfony\PostgreSqlBundle\DependencyInjection\Compiler\CommandLocatorPass;
use Flow\Bridge\Symfony\PostgreSqlBundle\Generator\TwigMigrationGenerator;
use Flow\Bridge\Symfony\PostgreSqlBundle\Messenger\FlowPostgreSqlTransportFactory;
use Flow\Bridge\Symfony\PostgreSqlBundle\Profiler\ProfilerClient;
use Flow\Bridge\Symfony\PostgreSqlBundle\Profiler\ProfilerController;
use Flow\Bridge\Symfony\PostgreSqlBundle\Repository\FilesystemMigrationRepository;
use Flow\Bridge\Symfony\PostgreSQLCache\CacheCatalogProvider;
Expand All @@ -25,7 +26,6 @@
use Flow\PostgreSql\Client\Client;
use Flow\PostgreSql\Client\ConnectionParameters;
use Flow\PostgreSql\Client\Context;
use Flow\PostgreSql\Client\Debug\RecordingClient;
use Flow\PostgreSql\Client\DsnParser;
use Flow\PostgreSql\Client\Infrastructure\PgSql\PgSqlClient;
use Flow\PostgreSql\Client\Telemetry\PostgreSqlTelemetryConfig;
Expand Down Expand Up @@ -477,7 +477,7 @@ public function configure(DefinitionConfigurator $definition): void
->addDefaultsIfNotSet()
->children()
->variableNode('enabled')
->info('null (default) = auto-enable iff WebProfilerBundle is registered; true/false to force')
->info('null (default) = auto-enable iff WebProfilerBundle is registered and the kernel runs in debug mode (so bin/console --no-debug records nothing); true/false to force')
->defaultNull()
->validate()
->ifTrue(static fn (mixed $v): bool => $v !== null && !is_bool($v))
Expand All @@ -488,6 +488,21 @@ public function configure(DefinitionConfigurator $definition): void
->info('Show bound query parameters in the panel')
->defaultTrue()
->end()
->integerNode('max_queries')
->info('Maximum number of recorded queries retained for the panel; oldest are dropped first')
->defaultValue(1000)
->min(1)
->end()
->integerNode('max_query_length')
->info('Retained statements longer than this are truncated; truncated statements cannot be explained from the panel')
->defaultValue(1000)
->min(1)
->end()
->integerNode('max_retained_parameters')
->info('Queries binding more parameters than this keep none of them, to bound profiler memory. Unlike connections.<name>.telemetry.max_parameters, which keeps the first N, this is all-or-nothing: the panel re-binds the retained list to run EXPLAIN, and a partial list is not a valid query.')
->defaultValue(100)
->min(0)
->end()
->booleanNode('migrations')
->info('Show the Flow Migrations panel (executed/pending/unavailable status). Queries the database on every profiled request; set false to disable. Requires migrations to be enabled.')
->defaultTrue()
Expand All @@ -498,7 +513,7 @@ public function configure(DefinitionConfigurator $definition): void
}

/**
* @param array{connections: array<string, array{dsn: string, dbname: ?string, host: ?string, port: ?int, user: ?string, password: ?string, dbname_suffix: string, test_transaction_rollback: bool, lazy: bool, context?: array<string, mixed>, telemetry?: array{service_id: string, clock_service_id: ?string, trace_queries: bool, transaction_spans: 'grouped'|'per_operation'|'off', collect_metrics: bool, log_queries: bool, max_query_length: int, include_parameters: bool, max_parameters: int, max_parameter_length: int}, profiler?: bool}>, messenger: array{enabled: bool, table_name: string, schema: string}, cache: array{pools?: array<string, array{connection: ?string, table_name: string, schema: string, id_col: string, data_col: string, lifetime_col: string, time_col: string, namespace: string, default_lifetime: int, marshaller_service_id: ?string, share_connection: bool}>}, session: array{enabled: bool, connection: ?string, table_name: string, schema: string, id_col: string, data_col: string, lifetime_col: string, time_col: string, lock_mode: string, ttl: ?int, share_connection: bool}, migrations: array{enabled: bool, connection: ?string, directory: string, namespace: string, table_name: string, table_schema: string, migration_file_name: string, rollback_file_name: string, all_or_nothing: bool, generate_rollback: bool, drop_if_exists: bool, context?: array<string, mixed>, exclude?: list<array{schema: ?string, table: ?string, exact: ?string, starts_with: ?string, ends_with: ?string, pattern: ?string, policy_id: ?string, type: ?string, for_schema: ?string}>}, catalog_providers: list<array{catalog_provider_id: ?string, catalog: ?array<string, mixed>}>, profiler?: array{enabled?: bool|null, include_parameters?: bool, migrations?: bool}} $config
* @param array{connections: array<string, array{dsn: string, dbname: ?string, host: ?string, port: ?int, user: ?string, password: ?string, dbname_suffix: string, test_transaction_rollback: bool, lazy: bool, context?: array<string, mixed>, telemetry?: array{service_id: string, clock_service_id: ?string, trace_queries: bool, transaction_spans: 'grouped'|'per_operation'|'off', collect_metrics: bool, log_queries: bool, max_query_length: int, include_parameters: bool, max_parameters: int, max_parameter_length: int}, profiler?: bool}>, messenger: array{enabled: bool, table_name: string, schema: string}, cache: array{pools?: array<string, array{connection: ?string, table_name: string, schema: string, id_col: string, data_col: string, lifetime_col: string, time_col: string, namespace: string, default_lifetime: int, marshaller_service_id: ?string, share_connection: bool}>}, session: array{enabled: bool, connection: ?string, table_name: string, schema: string, id_col: string, data_col: string, lifetime_col: string, time_col: string, lock_mode: string, ttl: ?int, share_connection: bool}, migrations: array{enabled: bool, connection: ?string, directory: string, namespace: string, table_name: string, table_schema: string, migration_file_name: string, rollback_file_name: string, all_or_nothing: bool, generate_rollback: bool, drop_if_exists: bool, context?: array<string, mixed>, exclude?: list<array{schema: ?string, table: ?string, exact: ?string, starts_with: ?string, ends_with: ?string, pattern: ?string, policy_id: ?string, type: ?string, for_schema: ?string}>}, catalog_providers: list<array{catalog_provider_id: ?string, catalog: ?array<string, mixed>}>, profiler?: array{enabled?: bool|null, include_parameters?: bool, max_queries?: int, max_retained_parameters?: int, max_query_length?: int, migrations?: bool}} $config
*/
#[Override]
public function loadExtension(array $config, ContainerConfigurator $configurator, ContainerBuilder $container): void
Expand Down Expand Up @@ -568,7 +583,7 @@ private function registerMigrationsProfiler(

$hasWebProfiler = $this->isWebProfilerBundleRegistered($container);

if ($enabled === null && !$hasWebProfiler) {
if ($enabled === null && (!$hasWebProfiler || !$this->isDebug($container))) {
return;
}

Expand Down Expand Up @@ -602,11 +617,22 @@ private function registerProfiler(
);
}

if ($enabled === null && !$hasWebProfiler) {
// Recording follows kernel.debug like DoctrineBundle's dbal profiling, so "bin/console --no-debug"
// (a different compiled container) does not decorate any connection.
if ($enabled === null && (!$hasWebProfiler || !$this->isDebug($container))) {
return;
}

$container->setParameter('flow.postgresql.profiler.include_parameters', $includeParameters);
$container->setParameter('flow.postgresql.profiler.max_queries', (int) ($profilerConfig['max_queries'] ?? 1000));
$container->setParameter(
'flow.postgresql.profiler.max_retained_parameters',
(int) ($profilerConfig['max_retained_parameters'] ?? 100),
);
$container->setParameter(
'flow.postgresql.profiler.max_query_length',
(int) ($profilerConfig['max_query_length'] ?? 1000),
);
$configurator->import(__DIR__ . '/Resources/config/profiler.php');

$connections = is_array($config['connections'] ?? null) ? $config['connections'] : [];
Expand All @@ -619,9 +645,9 @@ private function registerProfiler(
continue;
}

$recordingDefinition = new Definition(RecordingClient::class, [
$recordingDefinition = new Definition(ProfilerClient::class, [
new Reference("flow.postgresql.{$name}.client.profiler.inner"),
new Reference('flow.postgresql.profiler.query_log'),
new Reference('flow.postgresql.profiler.query_recorder'),
$name,
]);
$recordingDefinition->setDecoratedService("flow.postgresql.{$name}.client", null, 10);
Expand All @@ -644,6 +670,11 @@ private function registerProfiler(
$container->setDefinition(ProfilerController::class, $controller);
}

private function isDebug(ContainerBuilder $container): bool
{
return $container->hasParameter('kernel.debug') && $container->getParameter('kernel.debug') === true;
}

private function isWebProfilerBundleRegistered(ContainerBuilder $container): bool
{
if (!$container->hasParameter('kernel.bundles')) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,6 @@

namespace Flow\Bridge\Symfony\PostgreSqlBundle\Profiler;

use Flow\PostgreSql\Client\Debug\QueryLog;
use Flow\PostgreSql\Client\Debug\RecordedQuery;
use Symfony\Component\HttpFoundation\Request;
use Symfony\Component\HttpFoundation\Response;
use Symfony\Component\HttpKernel\DataCollector\DataCollector;
Expand All @@ -21,22 +19,21 @@
use function strtoupper;

/**
* @phpstan-type QueryRow array{statement: string, parameters: array<int, mixed>, returnedRows: null|int, durationMs: float, failed: bool, error: null|string, connection: string, caller: null|string, explainable: bool, runCount: int, isDuplicate: bool}
* @phpstan-type QueryRow array{statement: string, parameters: array<int, mixed>, returnedRows: null|int, durationMs: float, failed: bool, error: null|string, connection: string, caller: null|string, explainable: bool, runCount: int, isDuplicate: bool, parametersTruncated: bool, statementTruncated: bool}
*/
final class FlowPostgreSqlDataCollector extends DataCollector implements LateDataCollectorInterface
{
private const array EXPLAINABLE_KEYWORDS = ['SELECT', 'WITH', 'INSERT', 'UPDATE', 'DELETE', 'VALUES', 'TABLE'];

public function __construct(
private readonly QueryLog $queryLog,
private readonly bool $includeParameters,
private readonly QueryRecorder $recorder,
) {}

public function collect(Request $request, Response $response, ?Throwable $exception = null): void {}

public function lateCollect(): void
{
$queries = $this->queryLog->queries();
$queries = $this->recorder->queries();

$runCounts = [];

Expand All @@ -45,44 +42,40 @@ public function lateCollect(): void
}

$byConnection = [];
$failedCount = 0;
$totalDurationMs = 0.0;

foreach ($queries as $query) {
$totalDurationMs += $query->durationMs;

if ($query->failed) {
$failedCount++;
}

$byConnection[$query->connection][] = [
'statement' => $query->sql,
'parameters' => $this->includeParameters ? $query->parameters : [],
'parameters' => $query->parameters,
'returnedRows' => $query->rowCount,
'durationMs' => $query->durationMs,
'failed' => $query->failed,
'error' => $query->error,
'connection' => $query->connection,
'caller' => $query->caller,
'explainable' => $this->isExplainable($query),
'explainable' =>
$this->isExplainable($query) && !$query->parametersTruncated && !$query->statementTruncated,
'runCount' => $runCounts[$query->sql],
'isDuplicate' => $runCounts[$query->sql] > 1,
'parametersTruncated' => $query->parametersTruncated,
'statementTruncated' => $query->statementTruncated,
];
}

$this->data = [
'queries' => $byConnection,
'queryCount' => count($queries),
'failedCount' => $failedCount,
'totalDurationMs' => $totalDurationMs,
'duplicateCount' => count($queries) - count($runCounts),
'queryCount' => $this->recorder->recordedCount(),
'retainedCount' => $this->recorder->retainedCount(),
'failedCount' => $this->recorder->failedCount(),
'totalDurationMs' => $this->recorder->totalDurationMs(),
'duplicateCount' => $this->recorder->retainedCount() - count($runCounts),
];
}

public function reset(): void
{
$this->data = [];
$this->queryLog->reset();
$this->recorder->reset();
}

public function getName(): string
Expand Down Expand Up @@ -112,6 +105,11 @@ public function getQueryCount(): int
return (int) ($this->data['queryCount'] ?? 0);
}

public function getRetainedCount(): int
{
return (int) ($this->data['retainedCount'] ?? 0);
}

public function getFailedCount(): int
{
return (int) ($this->data['failedCount'] ?? 0);
Expand Down
Loading
Loading