Skip to content

v0.9.6

Choose a tag to compare

@github-actions github-actions released this 07 Oct 21:38

Release v0.9.6

📦 NPM packages

🐳 Docker image

docker pull ms2data/malloy-publisher:0.9.6
docker pull ms2data/malloy-publisher:latest

ℹ️ Details

Server: per-connection sslmode and statement timeout for Postgres

A direct (non-proxied) Postgres connection, a DuckDB attachedDatabases Postgres entry, and a federated Postgres source now accept two settings that only a proxied connection, or nothing, accepted before.

  • sslmode (disable, no-verify, verify-ca, verify-full) applies against the configured host in place of the deployment's PGSSLMODE. A connection that sets a connectionString keeps that string's own sslmode, and the field is ignored with a warning. A direct connection that sets sslmode was rejected before and is now accepted.
  • statementTimeoutMilliseconds (new; 1 to 2147483647) makes the database cancel any statement that runs longer, on these connections and on a proxied one. It also bounds persisted-source builds on the connection, so size it for the longest build. When a connectionString already carries options, the timeout is merged into them.
  • Behaviour change on attached databases. An attachedDatabases Postgres entry that already carried sslmode had it silently ignored in favour of PGSSLMODE; it now applies. What to check: an entry with sslmode: disable under PGSSLMODE=require now connects in plaintext, and one with verify-ca needs the server's CA in NODE_EXTRA_CA_CERTS.
  • Persisted sources. On a connection without a fingerprint, setting or changing either field gives that connection's persisted sources new identities, so they are rebuilt. A connection with a fingerprint, or one that sets neither field, is unaffected.

A DuckLake catalog connection is unchanged and applies neither setting.


Server: plain Postgres connections cap their open sessions

A plain (non-proxied) Postgres connection used to open a new database session for every query with no limit, so concurrent queries across a fleet could exhaust a role's CONNECTION LIMIT. It now runs through a pool that holds at most 5 open sessions per connection per process. Each query still gets a fresh session that is closed when it finishes, so session state (SET, SET ROLE, an open BEGIN) never carries over to another caller.

  • New setting. PUBLISHER_POSTGRES_POOL_MAX sets the cap (default 5). Across a fleet the ceiling is roughly the cap times the pods serving the environment, so size it against the role's CONNECTION LIMIT.
  • New failure mode. When every session is busy, a query waits up to 30 s for one and then fails with HTTP 502 and a message saying the connection had no free session. It never reached the database, so retrying once other queries finish is safe. The same condition is logged as Postgres connection pool exhausted.
  • New application_name. These sessions show up as malloy-publisher in the database's pg_stat_activity.

What's Changed

  • ci: run the server unit and integration suites side by side by @sagarswamirao in #1326
  • fix(server): pool and cap plain Postgres connections, configurable with PUBLISHER_POSTGRES_POOL_MAX by @sagarswamirao in #1262
  • feat(server): per-connection sslmode and statement timeout for Postgres by @sagarswamirao in #1292
  • chore(release): stamp 0.9.5 on main by @housejester in #1344

Full Changelog: v0.9.5...v0.9.6