pgcat-0.1.0

A Helm chart for PgCat a PostgreSQL pooler and proxy (like PgBouncer) with support for sharding, load balancing, failover and mirroring.

v1.1.1

What's Changed

• Fixed performance regression in prepared statements issuing a Flush to Postgres on every iteration of the cache maintenance loop @levkk
• Restore the ability to filter spammy log messages by @Spindel in #530
• Fix typo in the config documentation by @bpaquet in #532

New Contributors

• @Spindel made their first contribution in #530
• @bpaquet made their first contribution in #532

Full Changelog: v1.1.0...v1.1.1

v1.1.0

Big Features

• Support for prepared statements
• Auth passthrough for MD5 (Postgres <14)
• Support for server TLS connections

Improvements

• Stats use atomics instead of channels, much faster and more reliable.
• Keep-alives for client connections help with reliability
• Clients and servers don't have to have the same password to connect to Postgres, allowing for password rotations on the server
• Can deploy pooler without connecting to servers
• Better logging using tracing
• LIFO and FIFO options for pool connection handling
• Started the plugins interface with a few example applications
• Stats using atomics instead of channels for better performance and reliability
• Many bug fixes and other improvements

Changelog

• Refactor stats to use atomics by @magec in #375
• Auth passthrough (auth_query) by @magec in #266
• Protocol-level test helpers by @drdrsh in #393
• Fix port; add user pool mode by @levkk in #395
• A couple things by @levkk in #397
• Fix typos by @kianmeng in #398
• Load config on client connect by @levkk in #401
• feat: set keepalive for pgcat server itself by @Cluas in #402
• Separate server and client passwords optionally by @levkk in #407
• Fix CONFIG by @levkk in #408
• README updates by @levkk in #409
• fix typo by @NOBLES5E in #415
• Add more pool settings by @levkk in #416
• Server TLS by @levkk in #417
• Readme update by @levkk in #418
• Add dns_cache for server addresses as in pgbouncer by @magec in #249
• Plugins! by @levkk in #420
• Actually plugins by @levkk in #421
• lowercase config query by @levkk in #422
• Optionally validate config on boot by @levkk in #423
• Accurate log messages by @levkk in #425
• First try with role reset by @atanner27 in #427
• Calculate averages correctly and preserve totals like before by @levkk in #429
• Fix docker-compose dev stack for Apple silicon by @drdrsh in #432
• Fixes average stats bug by @zainkabani in #436
• Improve logging for connection cleanup by @zainkabani in #428
• Prewarmer by @levkk in #435
• Allow to deploy pools without checking by @levkk in #438
• Fix time based average stats by @zainkabani in #442
• Optionally clean up server connections by @levkk in #444
• Ensure data makes it to the client by @levkk in #446
• Refactor Pool Stats to be based off of Server/Client stats by @drdrsh in #445
• Use latest bb8 and use Lifo as the queue strategy in the pool by @zainkabani in #455
• Change default server lifetime and bump bb8 version to use LIFO correctly by @zainkabani in #462
• Make queue strategy configurable and default to Fifo by @zainkabani in #463
• Report min_pool_size correctly by @jchen1 in #471
• Support for prepared statements by @levkk in #474
• Log not implemented messages as debug in prometheus metrics. by @jbub in #477
• fmt by @levkk in #480
• Fix build by @levkk in #481
• Implement Close for prepared statements by @levkk in #482
• Fixes the default for round robing in General by @zainkabani in #488
• [BUG] Fix binding of NULL value parameters in prepared statements by @voldemarich in #496
• docs: fix general.autoreload description by @Cluas in #491
• Add support for tcp_user_timeout by @drdrsh in #503
• add "show help" command by @sebastianwebber in #505
• push and build only in main and tags by @sebastianwebber in #508
• Add cmd line parser by @sebastianwebber in #512
• add support for multiple log formats by @sebastianwebber in #517
• add --no-color option to disable colors in the terminal by @sebastianwebber in #518
• Fix COPY FROM and add tests by @drdrsh in #522
• Release 1.1 by @levkk in #526

New Contributors

• @kianmeng made their first contribution in #398
• @NOBLES5E made their first contribution in #415
• @atanner27 made their first contribution in #427
• @jchen1 made their first contribution in #471
• @jbub made their first contribution in #477
• @voldemarich made their first contribution in #496
• @sebastianwebber made their first contribution in #505

Full Changelog: v1.0.0...v1.1.0

v1 - Stable

Announcing v1

After over a year of development, we are proud to announce that we made it to v1! PgCat is stable, running in production and serving hundreds of thousands of queries.

Many thanks and congratulations to our amazing contributors for this effort. A special thank you to @drdrsh for pioneering PgCat at Instacart and implementing many critical features and bug fixes.

What's Changed

• A bit faster get_pool by @drdrsh in #187
• Use Jemalloc by @drdrsh in #189
• Don't send discard all when state is changed in transaction by @zainkabani in #186
• Re-enable query parser and parse multiple statements by @levkk in #191
• Fix docker-compose by @levkk in #193
• Automatic sharding: part one of many by @levkk in #194
• Dont change shard unless you know by @levkk in #195
• Add dependabot for keeping dependencies up-to-date by @chhetripradeep in #196
• chore(deps): bump regex from 1.5.5 to 1.6.0 by @dependabot in #197
• chore(deps): bump serde_derive from 1.0.136 to 1.0.147 by @dependabot in #198
• chore(deps): bump sha2 from 0.10.2 to 0.10.5 by @dependabot in #199
• chore(deps): bump async-trait from 0.1.52 to 0.1.58 by @dependabot in #200
• chore(deps): bump bytes from 1.1.0 to 1.2.1 by @dependabot in #206
• chore(deps): bump rustls-pemfile from 1.0.0 to 1.0.1 by @dependabot in #205
• chore(deps): bump base64 from 0.13.0 to 0.13.1 by @dependabot in #203
• chore(deps): bump arc-swap from 1.5.0 to 1.5.1 by @dependabot in #202
• chore(deps): bump rand from 0.8.4 to 0.8.5 by @dependabot in #201
• chore(deps): bump tokio from 1.16.1 to 1.19.2 by @dependabot in #210
• chore(deps): bump once_cell from 1.9.0 to 1.16.0 by @dependabot in #209
• chore(deps): bump log from 0.4.14 to 0.4.17 by @dependabot in #216
• chore(deps): bump serde from 1.0.136 to 1.0.147 by @dependabot in #215
• chore(deps): bump md-5 from 0.10.0 to 0.10.4 by @dependabot in #214
• chore(deps): bump sha2 from 0.10.5 to 0.10.6 by @dependabot in #213
• chore(deps): bump env_logger from 0.9.0 to 0.9.1 by @dependabot in #212
• chore(deps): bump chrono from 0.4.19 to 0.4.22 by @dependabot in #211
• chore(deps): bump toml from 0.5.8 to 0.5.9 by @dependabot in #207
• Fix for warnings about avg_errors not implemented by @chhetripradeep in #220
• Fix dependabot labels for pull-requests by @chhetripradeep in #219
• chore(deps): bump jemallocator from 0.3.2 to 0.5.0 by @dependabot in #218
• chore(deps): bump md-5 from 0.10.4 to 0.10.5 by @dependabot in #217
• chore(deps): bump num_cpus from 1.13.1 to 1.14.0 by @dependabot in #221
• chore(deps): bump regex from 1.6.0 to 1.7.0 by @dependabot in #224
• chore(deps): bump env_logger from 0.9.1 to 0.9.3 by @dependabot in #223
• chore(deps): bump hyper from 0.14.20 to 0.14.23 by @dependabot in #222
• chore: make clippy lint happy by @Cluas in #225
• chore(deps): bump chrono from 0.4.22 to 0.4.23 by @dependabot in #230
• chore(deps): bump sqlparser from 0.26.0 to 0.27.0 by @dependabot in #229
• Default to using username when database isn't present on startup by @zainkabani in #234
• Adds health check setting to pool and avoids get_config in hotpath by @zainkabani in #235
• Adds configuration for logging connections and removes get_config from entrypoint by @zainkabani in #236
• Move ClientBadStartup error log to debug by @zainkabani in #237
• Move get_config in startup to admin branch to scope down usage by @zainkabani in #238
• Adds details to errors and fixes error propagation bug by @zainkabani in #239
• chore(deps): bump bytes from 1.2.1 to 1.3.0 by @dependabot in #240
• chore(deps): bump env_logger from 0.9.3 to 0.10.0 by @dependabot in #241
• chore(deps): bump serde from 1.0.147 to 1.0.148 by @dependabot in #242
• chore(deps): bump serde_derive from 1.0.147 to 1.0.148 by @dependabot in #243
• chore(deps): bump sha-1 from 0.10.0 to 0.10.1 by @dependabot in #244
• chore(deps): bump async-trait from 0.1.58 to 0.1.59 by @dependabot in #245
• chore(deps): bump sqlparser from 0.27.0 to 0.28.0 by @dependabot in #248
• chore(deps): bump serde_derive from 1.0.148 to 1.0.149 by @dependabot in #247
• chore(deps): bump serde from 1.0.148 to 1.0.149 by @dependabot in #246
• chore(deps): bump base64 from 0.13.1 to 0.20.0 by @dependabot in #250
• chore(deps): bump serde_derive from 1.0.149 to 1.0.150 by @dependabot in #251
• chore(deps): bump serde from 1.0.149 to 1.0.150 by @dependabot in #252
• chore(deps): bump toml from 0.5.9 to 0.5.10 by @dependabot in #256
• Allow setting idle_timeout for server connections. by @magec in #257
• Allow setting the number of runtime workers to be used. by @magec in #258
• chore(deps): bump serde from 1.0.150 to 1.0.151 by @dependabot in #260
• chore(deps): bump async-trait from 0.1.59 to 0.1.60 by @dependabot in #259
• chore(deps): bump serde_derive from 1.0.150 to 1.0.151 by @dependabot in #261
• chore(deps): bump num_cpus from 1.14.0 to 1.15.0 by @dependabot in #264
• Buffer copy data messages by @zainkabani in #265
• Remove logo, pending new logo by @levkk in #267
• Fix typo in README by @eoinkelly in #272
• chore(deps): bump arc-swap from 1.5.1 to 1.6.0 by @dependabot in #273
• Fix tests (use sudo) by @levkk in #276
• chore(deps): bump sqlparser from 0.28.0 to 0.30.0 by @dependabot in #275
• chore(deps): bump once_cell from 1.16.0 to 1.17.0 by @dependabot in #270
• chore(deps): bump serde_derive from 1.0.151 to 1.0.152 by @dependabot in #269
• chore(deps): bump serde from 1.0.151 to 1.0.152 by @dependabot in #268
• chore(deps): bump async-trait from 0.1.60 to 0.1.61 by @dependabot in #278
• chore(deps): bump base64 from 0.20.0 to 0.21.0 by @dependabot in #279
• chore(deps): bump regex from 1.7.0 to 1.7.1 by @dependabot in #280
• Update cargo lock file by @zainkabani in #281
• Write messages directly onto message buffer instead of allocating on own buffer by @zainkabani in #283
• Introduce least-outstanding-connections load balancing by @drdrsh in #282
• Buffer client CopyData messages by @zainkabani in #284
• chore(deps): bump tokio from 1.24.1 to 1.24.2 by @dependabot in #286
• chore(deps): bump activerecord from 7.0.3.1 to 7.0.4.1 in /tests/ruby by @dependabot in #287
• Log error messages for network failures by @drdrsh in #289
• Removes message cloning operation required for query router by @zainkabani in #285
• Add more metrics to prometheus endpoint by @magec in #263
• Refactors is_banned logic and forces health check on unban by @zainkabani in #288
• chore(deps): bump toml from 0.5.10 to 0.5.11 by @dependabot in #290
• chore(deps): bump async-t...

Lots of Good Stuff

Summary

Lots of bug fixes and optimizations.

What's Changed

• Live reloading entire config and bug fixes by @levkk in #84
• Fix panic & query router bug by @levkk in #85
• Automatically reload config every seconds (disabled by default) by @levkk in #86
• Fix stats dymanic reload by @levkk in #87
• Support for TLS by @levkk in #91
• Bump activerecord from 7.0.2.2 to 7.0.3.1 in /tests/ruby by @dependabot in #94
• Add support for multi-database / multi-user pools by @drdrsh in #96
• Minor fix for some stats by @chhetripradeep in #97
• Add Serialize trait to configs by @drdrsh in #99
• Slightly more light weight health check by @drdrsh in #100
• Avoid ValueAfterTable when serializing configs by @drdrsh in #101
• Add test for config Serializer by @drdrsh in #102
• Send proper server parameters to clients using admin db by @drdrsh in #103
• Sync pgcat config for docker-compose by @chhetripradeep in #104
• Fix Python tests and remove CircleCI-specific path by @drdrsh in #106
• Add user to SHOW STATS query by @drdrsh in #108
• Report banned addresses as disabled by @drdrsh in #111
• Generate test coverage report in CircleCI by @drdrsh in #110
• Fix local dev by @drdrsh in #112
• Implementing graceful shutdown by @zainkabani in #105
• Prevent clients from sticking to old pools after config update by @drdrsh in #113
• create a prometheus exporter on a standard http port by @dat2 in #107
• Validates pgcat is closed after shutdown python tests by @zainkabani in #116
• fix docker compose port allocation for local dev by @dat2 in #117
• Health check delay by @zainkabani in #118
• Speed up CI a bit by @levkk in #119
• Fix debug log by @levkk in #120
• Make prometheus port configurable by @chhetripradeep in #121
• Statement timeout + replica imbalance by @levkk in #122
• Add cl_idle to SHOW POOLS by @drdrsh in #124
• Fix missing statistics by @levkk in #125
• Minor cleanup in admin command by @chhetripradeep in #126
• Add pool name and username to address object by @drdrsh in #128
• Minor Refactoring of re-used code and server stat reporting by @zainkabani in #129
• Random instance selection by @drdrsh in #136
• Random lb by @levkk in #138
• Fix incorrect routing for replicas by @levkk in #139
• Fix too many idle servers by @levkk in #140
• Really fix idle servers by @levkk in #141
• Avoid sending Z packet in the middle of extended protocol packet sequence if we fail to get connection form pool by @drdrsh in #137
• Graceful shutdown and refactor by @levkk in #144
• Exit with failure codes if configs are bad by @drdrsh in #146
• Move autoreloader to own tokio task by @zainkabani in #148
• Ruby integration tests by @drdrsh in #147
• Allow running integration tests with coverage locally by @drdrsh in #151
• Log Address information in connection create/drop by @drdrsh in #154
• Better handling extended protocol messages in the event of busy pool by @drdrsh in #155
• Send DISCARD ALL even if client is not in transaction by @drdrsh in #152
• Patch graceful shutdown bug by @zain-kabani in #157
• Main Thread Panic when swarmed with clients by @drdrsh in #158
• Adds microsecond logging and also reformats duration to include milliseconds by @zainkabani in #156
• Avoid reporting ProtocolSyncError when admin session disconnects by @drdrsh in #160
• Better logging for failure to get connection from pool by @drdrsh in #161
• Send signal even if process is gone by @levkk in #162
• Clean connection state up after protocol named prepared statement by @drdrsh in #163
• Add Discord link by @levkk in #164
• Add SHOW CLIENTS / SHOW SERVERS + Stats refactor and tests by @drdrsh in #159
• Report Query times by @drdrsh in #166
• Export pgcat objects in lib by @zainkabani in #169
• Update to latest library versions by @zainkabani in #170
• Minor refactor for configs by @zainkabani in #172
• Add defaults for configs by @zainkabani in #174
• Log failed client logins by @drdrsh in #173
• Don't drop connections if DB hasn't changed by @levkk in #175
• Fix the pool fix by @levkk in #176
• Set client state to idle after error by @drdrsh in #179
• Change sharding config to enum and move validation of configs into public functions by @zainkabani in #178
• Replace a few types with more developer-friendly names by @levkk in #182
• Fix maxwait metric by @drdrsh in #183

New Contributors

• @drdrsh made their first contribution in #96
• @chhetripradeep made their first contribution in #97
• @dat2 made their first contribution in #107
• @zain-kabani made their first contribution in #157

Full Changelog: v0.2.1-beta1...v0.3.0

Client MD5 authentication

Clients are now required to authenticate with MD5.

Scram kitties, scram!

Implemented SCRAM-SHA-256 authentication made compulsory in Postgres 14. This is for PgCat --> server authentication only.

Cat shower

Plugged a memory leak in client -> server mapping used for query cancellation that happened with clients that disconnected mid-transaction.

Cat on wheels

Fix for rolling back (resetting) servers that were left idle in transaction by clients that disconnected mid-transaction.

Ready for meowtime

This is a beta release. The feature set is mature enough that it can be used in low risk environments, dev, staging, pre-prod and low traffic production.

The admin database supports monitoring with Datadog.