Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion installation/app-backend-setup.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ When using PowerSync, your app backend is responsible for the following:

1. Authenticating app users
2. [Generating JWTs](/installation/authentication-setup) that allows the PowerSync Client SDK to authenticate users against the server-side [PowerSync Service](/architecture/powersync-service).
3. [Writing client-side changes](/installation/app-backend-setup/writing-client-changes) to the backend database (Postgres or MongoDB)
3. [Writing client-side changes](/installation/app-backend-setup/writing-client-changes) to the backend database (Postgres, MongoDB or MySQL)

<Frame caption="An overview of how PowerSync interacts with your backend application.">
<img src="/images/powersync-docs-app-backend-setup-diagram (2).png"/>
Expand Down
2 changes: 1 addition & 1 deletion installation/app-backend-setup/writing-client-changes.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ In other words, don't place writes into something like a queue for processing la
</Warning>

<Accordion title="Why must my write endpoint be synchronous?">
Client applications advance to a write checkpoint after uploads have been processed, so if the client believes that the server has written changes into your backend database (Postgres or MongoDB), but the next checkpoint does not contain your uploaded changes, those changes will be removed from the client. This could manifest as UI glitches for your end-users, where the changes disappear from the device for a few seconds and then re-appear.
Client applications advance to a write checkpoint after uploads have been processed, so if the client believes that the server has written changes into your backend database (Postgres, MongoDB or MySQL), but the next checkpoint does not contain your uploaded changes, those changes will be removed from the client. This could manifest as UI glitches for your end-users, where the changes disappear from the device for a few seconds and then re-appear.
</Accordion>

### Changes recorded on the client
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ description: "The PowerSync 'backend connector' provides the connection between
It is used to:

1. Retrieve a JWT token which can be used by the PowerSync Client SDK to authenticate against your [PowerSync Service](/architecture/powersync-service) instance.
2. Upload writes to your backend: Writes that are made to the local SQLite database are sent to your backend application, where you control how they're applied to your backend database (Postgres or MongoDB)
2. Upload writes to your backend: Writes that are made to the local SQLite database are sent to your backend application, where you control how they're applied to your backend database (Postgres, MongoDB or MySQL)

Accordingly, the connector must implement two methods:

Expand Down
5 changes: 3 additions & 2 deletions resources/demo-apps-example-projects.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -94,10 +94,11 @@ Example projects are listed under backend they use, but you can easily wire up y

<Accordion title="Self-Hosting" icon="desktop">
* [To-Do List App with Docker Compose](https://github.com/powersync-ja/self-host-demo)
* [Postgres + Node.js Backend](https://github.com/powersync-ja/self-host-demo/tree/main/demos/nodejs)
* [Postgres + Node.js Backend](https://github.com/powersync-ja/self-host-demo/tree/main/demos/nodejs-postgres-bucket-storage)
* [Postgres + Postgres sync bucket storage + Node.js Backend](https://github.com/powersync-ja/self-host-demo/tree/main/demos/nodejs)
* [MongoDB + Node.js Backend](https://github.com/powersync-ja/self-host-demo/tree/main/demos/nodejs-mongodb)
* [MySQL + Node.js Backend](https://github.com/powersync-ja/self-host-demo/tree/main/demos/nodejs-mysql)
* [Supabase Backend + Local Development Example](https://github.com/powersync-ja/self-host-demo/tree/main/demos/supabase)
* [Supabase Backend (Postgres) + Local Development Example](https://github.com/powersync-ja/self-host-demo/tree/main/demos/supabase)
* [Django Backend](https://github.com/powersync-ja/self-host-demo/tree/main/demos/django)
</Accordion>

Expand Down
2 changes: 1 addition & 1 deletion self-hosting/installation.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ In order to run the PowerSync Service, the following activities are required:
<CardGroup>
<Card title="Configure source database" icon="database" href="/installation/database-setup"horizontal />
<Card title="Configure PowerSync Service" icon="gears" href="/self-hosting/installation/powersync-service-setup"horizontal />
<Card title="Configure MongoDB" icon="database" href="/self-hosting/installation/powersync-service-setup#configure-mongodb" horizontal/>
<Card title="Configure sync bucket storage" icon="database" href="/self-hosting/installation/powersync-service-setup#configure-sync-bucket-storage" horizontal/>
<Card title="Connect PowerSync to database" icon="plug" href="/self-hosting/installation/powersync-service-setup#database-connection" horizontal/>
<Card title="Define sync rules" icon="list-check" href="/self-hosting/installation/powersync-service-setup#sync-rules" horizontal/>
<Card title="Import PowerSync client SDK" icon="file-import" href="/installation/client-side-setup"horizontal />
Expand Down
65 changes: 55 additions & 10 deletions self-hosting/installation/powersync-service-setup.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,15 @@ title: "PowerSync Service Setup"
description: "Configuration details for connecting the PowerSync Service to your database"
---

After configuring your Postgres database for PowerSync, you'll setup your [PowerSync Service](/architecture/powersync-service).
After configuring your source database for PowerSync, you'll need to setup your [PowerSync Service](/architecture/powersync-service).

This entails:

1. Configuring MongoDB (if required)
1. Configuring sync bucket storage

2. Defining your PowerSync config

1. Defining connections to Postgres and MongoDB
1. Defining connections to your source database, and sync bucket storage database

2. Defining your [Sync Rules](/usage/sync-rules)

Expand All @@ -23,9 +23,13 @@ Examples of the above can be found in our demo application [here](https://github
**Deploy PowerSync on Coolify:** See our [integration guide](/integration-guides/coolify) for deploying the PowerSync Service on Coolify. This can simplify the setup and management of the deployment.
</Tip>

## Configure MongoDB
## Configure Sync Bucket Storage

The PowerSync Service uses [MongoDB](https://www.mongodb.com/) under the hood and requires at least one replica set node. A single node is fine for development/staging environments, but a 3-node replicas set is recommended [for production](/self-hosting/lifecycle-maintenance/server-specs).
The PowerSync Service requires a storage backend for sync buckets. You can use either MongoDB or Postgres for this purpose. The storage backend is separate from your source database.

### MongoDB Storage

MongoDB requires at least one replica set node. A single node is fine for development/staging environments, but a 3-node replica set is recommended [for production](/self-hosting/lifecycle-maintenance/server-specs).

[MongoDB Atlas](https://www.mongodb.com/products/platform/atlas-database) enables replica sets by default for new clusters.

Expand All @@ -50,6 +54,36 @@ If you are rolling your own Docker environment, you can include this init script
- 'sleep 10 && mongosh --host mongo:27017 --eval ''try{rs.status().ok && quit(0)} catch {} rs.initiate({_id: "rs0", version: 1, members: [{ _id: 0, host : "mongo:27017" }]})'''
```

### Postgres Storage (Beta)

Available since version 1.3.8 of [`journeyapps/powersync-service`](https://hub.docker.com/r/journeyapps/powersync-service), you can use Postgres as an alternative storage backend for sync buckets. This feature is currently in [beta](/resources/feature-status).

#### Database Setup

You'll need to create a dedicated user and schema for PowerSync storage. You can either:

1. Let PowerSync create the schema (recommended):

```sql
CREATE USER powersync_storage_user WITH PASSWORD 'secure_password';
-- The user should only have access to the schema it created
GRANT CREATE ON DATABASE postgres TO powersync_storage_user;
```

2. Or manually create the schema:

```sql
CREATE USER powersync_storage_user WITH PASSWORD 'secure_password';
CREATE SCHEMA IF NOT EXISTS powersync AUTHORIZATION powersync_storage_user;
GRANT CONNECT ON DATABASE postgres TO powersync_storage_user;
GRANT USAGE ON SCHEMA powersync TO powersync_storage_user;
GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA powersync TO powersync_storage_user;
```

<Tip>
**Demo app:** A demo app with Postgres bucket storage is available [here](https://github.com/powersync-ja/self-host-demo/tree/main/demos/nodejs-postgres-bucket-storage).
</Tip>

## PowerSync Configuration

The PowerSync Service is configured using key/value pairs in a config file, and supports the following configuration methods:
Expand All @@ -70,7 +104,7 @@ The config file schema is also available here:

<Card title="self-host-demo/schema/schema.json at main · powersync-ja/self-host-demoGitHub" icon="github" href="https://github.com/powersync-ja/self-host-demo/blob/main/schema/schema.json" horizontal />

Below is a skeleton config file that you can copy/paste and edit locally:
Below is a skeleton config file you can copy and paste to edit locally:

```yaml
# config.yaml
Expand All @@ -90,16 +124,23 @@ replication:

# SSL settings
sslmode: disable # 'verify-full' (default) or 'verify-ca' or 'disable'
# 'disable' is OK for local/private networks, not for public networks
# Note: 'disable' is only suitable for local/private networks, not for public networks

# Connection settings for sync bucket storage
# Connection settings for sync bucket storage (MongoDB and Postgres are supported)
storage:
# Option 1: MongoDB Storage
type: mongodb
uri: mongodb://mongo:27017/powersync_demo
# use these if authentication is required
# Use these if authentication is required. The user should have `readWrite` and `dbAdmin` roles
# username: myuser
# password: mypassword

# Option 2: Postgres Storage
# type: postgresql
# This accepts the same parameters as a Postgres replication source connection
# uri: postgresql://powersync_storage_user:secure_password@storage-db:5432/postgres
# sslmode: disable

# The port which the PowerSync API server will listen on
port: 80

Expand Down Expand Up @@ -132,6 +173,10 @@ client_auth:

```

<Info>
**Important:** When using Postgres for sync bucket storage, a separate server is currently required for replication connections (if using Postgres for replication) and storage. Using the same server might cause unexpected results.
</Info>

Specify the connection to Postgres in the `replication` section. Retrieving your database connection string / individual parameters differs by database hosting provider. See [Database Connection](/self-hosting/appendix/database-connection) for further details.

<Info>
Expand All @@ -142,7 +187,7 @@ Specify the connection to Postgres in the `replication` section. Retrieving your
This is because Supabase only allows direct database connections over IPv6 — PowerSync cannot connect using the connection pooler.
</Info>

Specify the connection to MongoDB in the `storage` section.
Specify the connection to your sync bucket storage provider (Postgres or MongoDB) in the `storage` section.

### Environment Variables

Expand Down
10 changes: 5 additions & 5 deletions self-hosting/lifecycle-maintenance.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -12,14 +12,14 @@ Migrations run automatically by default.
# powersync.yaml

migrations:
# Setting this to true will skip automatic migrations.
# Migrations can be triggered externally by altering the container `command`.
# Setting this to false (default) enables automatic migrations on startup.
# When set to true, migrations must be triggered manually by modifying the container `command`.
disable_auto_migration: true
```
MongoDB locks are used to ensure that migrations are only executed once in the case where multiple containers can be started simultaneously.
MongoDB locks ensure migrations are executed exactly once, even when multiple containers start simultaneously.

## Backups

We recommend using git to backup your configuration files.
We recommend using Git to backup your configuration files.

Your MongoDB database (for sync bucket storage) does not need to be backed up since it's easily recreated from Postgres.
The sync bucket storage database doesn't require backups as it can be easily reconstructed.
2 changes: 1 addition & 1 deletion usage/sync-rules.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ The remainder of these docs dive further into the details.

Each [PowerSync Service](/architecture/powersync-service) instance has a Sync Rules configuration where Sync Rules are defined using SQL-like queries (limitations and more info [here](/usage/sync-rules/operators-and-functions)) combined together in a YAML file.

This SQL-like syntax is used when connecting to either Postgres or MongoDB as the backend source database.
This SQL-like syntax is used when connecting to either Postgres, MongoDB or MySQL as the backend source database.

The [PowerSync Service](/architecture/powersync-service) uses these SQL-like queries to group data into "sync buckets" when replicating data to client devices.

Expand Down
2 changes: 1 addition & 1 deletion usage/sync-rules/advanced-topics/sharded-databases.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ In the case of Postgres, PowerSync cannot replicate Postgres [foreign tables](ht
However, PowerSync does have options available to support sharded databases in general.

<Info>
When using MongoDB as the backend source database, PowerSync does not currently support connecting to sharded clusters.
When using MongoDB or MySQL as the backend source database, PowerSync does not currently support connecting to sharded clusters.
</Info>

The primary options are:
Expand Down
Loading