Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/administration/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ The admin database name is [configurable](../configuration/pgdog.toml/admin.md).
| [`SHOW SERVERS`](servers.md) | Server connections made by PgDog to PostgreSQL. |
| [`SHOW POOLS`](pools.md) | Connection pools used to multiplex clients and servers. |
| [`SHOW CONFIG`](config.md) | Currently loaded values from `pgdog.toml`. |
| `SHOW STATS` | Connection pools statistics. |
| `SHOW PEERS` | List of PgDog processes running on the same network. Requires service discovery to be enabled. |
| `RELOAD` | Reload configuration from disk. See [pgdog.toml](../configuration/pgdog.toml/general.md) and [users.toml](../configuration/users.toml/users.md) for which options can be changed at runtime. |
| `RECONNECT` | Re-create all server connections using existing configuration. |
Expand Down
18 changes: 13 additions & 5 deletions docs/configuration/pgdog.toml/general.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ Default: **none**

### `tls_client_required`

Reject clients that connect without TLS. Consider setting this to `true` when using the `enabled_plain` mode of [passthrough_auth](#passthrough_auth).
Reject clients that connect without TLS. Consider setting this to `true` when using the `enabled_plain` mode of [`passthrough_auth`](#passthrough_auth).

Default: **`false`** (disabled)

Expand Down Expand Up @@ -234,9 +234,9 @@ Available options:
- `exclude_primary`
- `include_primary_if_replica_banned`

Include primary uses the primary database as well as the replicas to serve read queries. Exclude primary will send all read queries to replicas, leaving the primary to serve only writes.
`include_primary` uses the primary database as well as the replicas to serve read queries. `exclude_primary` will send all read queries to replicas, leaving the primary to serve only writes.

Include primary if replica banned strategy will only send reads to the primary if one or more replicas have been banned. This is useful in case you want to use the primary as a failover for reads.
`include_primary_if_replica_banned` strategy will only send reads to the primary if one or more replicas have been banned. This is useful in case you want to use the primary as a failover for reads.

Default: **`include_primary`**

Expand Down Expand Up @@ -315,8 +315,8 @@ Enables support for prepared statements. Available options are:
- `extended_anonymous`
- `full`

Full enables support for rewriting prepared statements sent over the simple protocol. Extended handles prepared statements sent normally
using the extended protocol. `full` attempts to rewrite prepared statements sent using the simple protocol. `extended_anonymous` caches and rewrites unnamed prepared statements, which is useful for some legacy client drivers.
`full` enables support for rewriting prepared statements sent over the simple protocol. `extended` handles prepared statements sent normally
using the extended protocol. `extended_anonymous` caches and rewrites unnamed prepared statements, which is useful for some legacy client drivers.

Default: **`extended`**

Expand Down Expand Up @@ -383,3 +383,11 @@ Default: **`true`** (enabled)
If enabled, log every time a user disconnects from PgDog.

Default: **`true`** (enabled)

## Statistics

### `stats_period`

How often to calculate averages shown in `SHOW STATS` [admin](../../administration/index.md) command and the [Prometheus](../../features/metrics.md) metrics.

Default: **`15_000`** (15s)
2 changes: 1 addition & 1 deletion docs/configuration/pgdog.toml/network.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ PgDog speaks the Postgres protocol which, underneath, uses TCP. Optimal TCP sett

```toml
[tcp]
keepalives = true
keepalive = true
time = 60_000
interval = 60_000
retries = 3
Expand Down
62 changes: 62 additions & 0 deletions docs/configuration/pgdog.toml/sharded_schemas.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
---
icon: material/set-split
---

# Sharded schemas

Schema-based sharding places data from tables in different Postgres schemas on their own shards.

## Example

Each schema needs to have a shard mapping in the config, for example:

```toml
[[sharded_schemas]]
database = "prod"
name = "customer_a"
shard = 0

[[sharded_schemas]]
database = "prod"
name = "customer_b"
shard = 1
```

All queries that fully qualify the table names will be routed correctly, for example:

```postgresql
SELECT * FROM customer_a.users -- Prefixed with the schema name.
WHERE admin = true;
```

## Default shard

For queries that don't specify a schema or for which a mapping doesn't exist, the default behavior is to send it to all shards. If this is not desirable, you can configure a default shard, like so:

```toml
[[sharded_schemas]]
database = "prod"
shard = 0
```

For example, the following queries will be sent to shard zero:

```postgresql
-- No schema specified.
SELECT * FROM pg_stat_activity;

-- Schema isn't mapped in the config.
SELECT * FROM customer_c.users
WHERE admin = true;
```

## Manual routing

If you need to query a specific shard or can't specify the schema name in the query, you can add it to a comment, for example:

```postgresql
/* pgdog_sharding_key: "customer_a" */
SELECT * FROM pg_stat_activity;
```

This will send the query to the shard mapped to the `customer_a` schema.
61 changes: 61 additions & 0 deletions docs/features/sharding/sharding-functions.md
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,67 @@ PostgreSQL has dozens of data types. PgDog supports a subset of those for shardi
| `VARCHAR` / `TEXT` | :material-check-circle-outline: | :material-check-circle-outline: | No |
| `UUID` | :material-check-circle-outline: | :material-check-circle-outline: | No |

## Schema-based sharding

In addition to splitting the tables themselves, PgDog can shard Postgres databases by placing different schemas on different shards. This is useful for multi-tenant applications that have stricter separation between their users' data.

When enabled, PgDog will route queries that fully qualify tables based on their respective schema names.

### Schema-to-shard mapping

Schemas are mapped to their shards in [pgdog.toml], for example:

```toml
[[sharded_schemas]]
database = "prod"
name = "customer_a"
shard = 0

[[sharded_schemas]]
database = "prod"
name = "customer_b"
shard = 1
```

Queries that include the schema name in the tables they are referring to can be routed to the right shard. For example:

```postgresql
SELECT * FROM customer_a.users WHERE email = $1;
```

Since the `users` table is fully qualified as `customer_a.users`, the query will be routed to shard zero.

### DDL

Unlike other sharding functions, schema-based sharding will also route DDL (e.g., `CREATE TABLE`, `CREATE INDEX`, etc.) queries to their respective shard, as long as the entity name is fully qualified.

For example:

```postgresql
CREATE TABLE customer_b.users (
id BIGSERIAL PRIMARY KEY,
email VARCHAR NOT NULL
);

CREATE UNIQUE INDEX ON customer_b.users USING btree(email);
```

Both of these DDL statements will be sent to shard one, because they explicitly refer to tables in schema `customer_b`, which is mapped to shard one.

### Default routing

If a schema isn't mapped to a shard number, PgDog will fallback to using other configured sharding functions. If none are set, the query will be sent to all shards.

To avoid this behavior and send all other queries to a particular shard, you can add a default schema mapping:

```toml
[[sharded_schemas]]
database = "prod"
shard = 0
```

This will send all queries that don't specify a schema or use a schema without a mapping to shard zero.

## Read more

- [COPY command](copy.md)
Expand Down
Loading