Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 8 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,30 +4,30 @@
![NPM Version](https://img.shields.io/npm/v/%40supabase%2Fstripe-sync-engine)
![Docker Image Version](https://img.shields.io/docker/v/supabase/stripe-sync-engine?label=Docker)

This monorepo contains two packages for synchronizing your Stripe account with a Postgres database:
This monorepo contains two packages for synchronizing your Stripe account with a PostgreSQL database:

- [`@supabase/stripe-sync-engine`](./packages/sync-engine/README.md): A TypeScript library for syncing Stripe data to Postgres, designed for integration into your own Node.js backend or serverless environment.
- [`stripe-sync-fastify`](./packages/fastify-app/README.md): A Fastify-based server and Docker image that exposes a `/webhooks` endpoint for Stripe, providing a ready-to-run service for real-time Stripe-to-Postgres sync.
- [`@supabase/stripe-sync-engine`](./packages/sync-engine/README.md): A TypeScript library for syncing Stripe data to PostgreSQL, designed for integration into your own Node.js backend or serverless environment.
- [`stripe-sync-fastify`](./packages/fastify-app/README.md): A Fastify-based server and Docker image that exposes a `/webhooks` endpoint for Stripe, providing a ready-to-run service for real-time Stripe-to-PostgreSQL sync.

![Sync Stripe with Postgres](./docs/stripe-sync-engine.jpg)
![Sync Stripe with PostgreSQL](./docs/stripe-sync-engine.jpg)

---

## Motivation

Sometimes you want to analyze your billing data using SQL. Even more importantly, you want to join your billing data to your product/business data.

This project synchronizes your Stripe account to a Postgres database. It can be a new database, or an existing Postgres database.
This project synchronizes your Stripe account to a PostgreSQL database. It can be a new database, or an existing PostgreSQL database.

---

## How it works

![How it works](./docs/sync-engine-how.png)

- Creates a new schema `stripe` in a Postgres database, with tables & columns matching Stripe.
- Creates a new schema `stripe` in a PostgreSQL database, with tables and columns matching Stripe.
- Exposes a `/webhooks` endpoint that listens to any Stripe webhooks (via the Fastify app).
- Inserts/updates/deletes changes into the tables whenever there is a change to Stripe.
- Inserts, updates, or deletes changes into the tables whenever there is a change to Stripe.

---

Expand All @@ -42,7 +42,7 @@ Each package has its own README with installation, configuration, and usage inst

## Supabase Edge Function

To deploy the sync-engine to a Supabase edge function, follow this [guide](./edge-function.md).
To deploy the sync-engine to a Supabase Edge Function, follow this [guide](./docs/edge-function.md).

## Webhook Support

Expand Down
10 changes: 5 additions & 5 deletions docs/contributing.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Contributing

Contributions are welcome.
Contributions are welcome! This document provides guidelines for contributing to the Stripe Sync Engine project.

## Contributing to the Docs

Expand All @@ -10,7 +10,7 @@ Building documentation requires Python 3.8+ and uv.

Create a virtual environment and install mkdocs, themes, and extensions using uv.

```shell
```sh
source .venv/bin/activate # On Windows: .venv\Scripts\activate
uv pip install -r docs/requirements_docs.txt
```
Expand All @@ -19,7 +19,7 @@ uv pip install -r docs/requirements_docs.txt

To serve the documentation locally, make sure your virtual environment is activated and run:

```shell
```sh
source .venv/bin/activate # On Windows: .venv\Scripts\activate
mkdocs serve
```
Expand All @@ -28,8 +28,8 @@ and visit the docs at [http://127.0.0.1:8000/](http://127.0.0.1:8000/)

### Deploying

If you have write access to the repo, docs can be updated using
If you have write access to the repository, documentation can be updated using:

```
```sh
mkdocs gh-deploy
```
98 changes: 76 additions & 22 deletions docs/docker.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,11 @@
![GitHub License](https://img.shields.io/github/license/supabase/stripe-sync-engine)
![Docker Image Version](https://img.shields.io/docker/v/supabase/stripe-sync-engine?label=Docker)

A Fastify-based server for syncing your Stripe account to a Postgres database in real time. Built on top of the Stripe Sync Engine.
A Fastify-based server for syncing your Stripe account to a PostgreSQL database in real time. Built on top of the Stripe Sync Engine.

## Features

- Exposes a `/webhooks` endpoint to receive Stripe webhooks and sync data to Postgres
- Exposes a `/webhooks` endpoint to receive Stripe webhooks and sync data to PostgreSQL
- Supports syncing customers, invoices, products, subscriptions, and more
- Runs as a lightweight Docker container
- Designed for easy deployment to any cloud or self-hosted environment
Expand Down Expand Up @@ -38,26 +38,31 @@ Set your webhook endpoint in the Stripe dashboard to point to your server’s `/

## Environment Variables

| Variable | Description | Required |
| ---------------------------------- | ------------------------------------------------------------------- | -------- |
| `DATABASE_URL` | Postgres connection string (with `search_path=stripe`) | Yes |
| `STRIPE_WEBHOOK_SECRET` | Stripe webhook signing secret | Yes |
| `API_KEY` | API key for admin endpoints (backfilling, etc.) | Yes |
| `SCHEMA` | Database schema name (default: `stripe`) | No |
| `STRIPE_SECRET_KEY` | Stripe secret key (needed for active sync/backfill) | No |
| `PORT` | Port to run the server on (default: 8080) | No |
| `STRIPE_API_VERSION` | Stripe API version (default: `2020-08-27`) | No |
| `AUTO_EXPAND_LISTS` | Fetch all list items from Stripe (default: false) | No |
| `BACKFILL_RELATED_ENTITIES` | Backfill related entities for foreign key integrity (default: true) | No |
| `MAX_POSTGRES_CONNECTIONS` | Max Postgres connection pool size (default: 10) | No |
| `REVALIDATE_OBJECTS_VIA_STRIPE_API` | Always fetch latest entity from Stripe (default: false) | No |
| `DISABLE_MIGRATIONS` | Disable the automated database migrations on app startup (default: false) | No |
| Variable | Description | Required |
| -------- | ----------- | -------- |
| `DATABASE_URL` | PostgreSQL connection string (with `search_path=stripe`) | Yes |
| `STRIPE_WEBHOOK_SECRET` | Stripe webhook signing secret | Yes |
| `API_KEY` | API key for admin endpoints (backfilling, etc.) | Yes |
| `SCHEMA` | Database schema name (default: `stripe`) | No |
| `STRIPE_SECRET_KEY` | Stripe secret key (needed for active sync/backfill) | No |
| `PORT` | Port to run the server on (default: 8080) | No |
| `STRIPE_API_VERSION` | Stripe API version (default: `2020-08-27`) | No |
| `AUTO_EXPAND_LISTS` | Fetch all list items from Stripe (default: false) | No |
| `BACKFILL_RELATED_ENTITIES` | Backfill related entities for foreign key integrity (default: true) | No |
| `MAX_POSTGRES_CONNECTIONS` | Max PostgreSQL connection pool size (default: 10) | No |
| `REVALIDATE_OBJECTS_VIA_STRIPE_API` | Always fetch latest entity from Stripe (default: false) | No |
| `DISABLE_MIGRATIONS` | Disable the automated database migrations on app startup (default: false) | No |
| `PG_SSL_CONFIG_ENABLED` | Enables SSL configuration. Set to `true` to enable | No |
| `PG_SSL_REJECT_UNAUTHORIZED` | Rejects unauthorized SSL connections. Set to `true` to enforce | No |
| `PG_SSL_CA` | Base64-encoded CA certificate for SSL connections | No |
| `PG_SSL_CERT` | Certificate chain in PEM format for SSL connections | No |
| `PG_SSL_REQUEST_CERT` | Requests a certificate from clients and attempts to verify it. Set to `true` to enable | No |

## Endpoints

- `POST /webhooks` — Receives Stripe webhook events and syncs data to Postgres
- `POST /webhooks` — Receives Stripe webhook events and syncs data to PostgreSQL
- `GET /health` — Health check endpoint
- `POST /sync` — Backfill Stripe data to Postgres (API key required)
- `POST /sync` — Backfill Stripe data to PostgreSQL (API key required)
- `POST /sync/single/:stripeId` — Backfill or update a single Stripe entity by ID (API key required)
- `POST /daily` — Backfill data from the last 24 hours (API key required)
- `POST /weekly` — Backfill data from the last 7 days (API key required)
Expand Down Expand Up @@ -114,13 +119,13 @@ body: {
- `object` **all** | **charge** | **customer** | **dispute** | **invoice** | **payment_method** | **payment_intent** | **plan** | **price** | **product** | **setup_intent** | **subscription** | **early_fraud_warning** | **refund** | **credit_note** | **tax_id** | **subscription_schedules**
- `created` is Stripe.RangeQueryParam. It supports **gt**, **gte**, **lt**, **lte**

#### Alternative routes to sync `daily/weekly/monthly` data
### Alternative routes to sync `daily/weekly/monthly` data

```
POST /sync/daily
```

---

```
POST /sync/daily
body: {
"object": "product"
Expand All @@ -129,10 +134,59 @@ body: {

### Syncing single entity

To backfill/update a single entity, you can use
To backfill/update a single entity, you can use:

```
POST /sync/single/cus_12345
```

The entity type is recognized automatically, based on the prefix.

### SSL CA Certificate in Base64 Format

To pass an SSL CA certificate in base64 format for the Dockerized application, follow these steps:

1. Obtain the CA certificate file (e.g., `prod-ca-2021.crt`).
2. Encode it in base64 format using the following command on Unix-based systems:

```sh
base64 -i prod-ca-2021.crt -o CA.base64
```

3. Open the `CA.base64` file and copy its contents.
4. Add the base64 string to your environment variables (e.g., `PG_SSL_CA`).
5. Pass the environment variable to the Docker container:

```sh
docker run -d \
-e DATABASE_URL=postgres://postgres:postgres@localhost:5432/postgres \
-e PG_SSL_CA="$(cat CA.base64)" \
-e STRIPE_SECRET_KEY=sk_test_... \
-e STRIPE_WEBHOOK_SECRET=... \
-e API_KEY="my-secret" \
-p 8080:8080 \
supabase/stripe-sync-engine:latest
```

> **Note:** The `PG_SSL_CA` environment variable should contain the base64-encoded CA certificate. The application will decode and use it automatically during runtime.

### Example Usage

To pass these variables to the Docker container, use the following command:

```sh
docker run -d \
-e DATABASE_URL=postgres://postgres:postgres@localhost:5432/postgres \
-e PG_SSL_CONFIG_ENABLED=true \
-e PG_SSL_REJECT_UNAUTHORIZED=true \
-e PG_SSL_CA="$(cat CA.base64)" \
-e PG_SSL_CERT="$(cat cert.pem)" \
-e PG_SSL_REQUEST_CERT=true \
-e STRIPE_SECRET_KEY=sk_test_... \
-e STRIPE_WEBHOOK_SECRET=... \
-e API_KEY="my-secret" \
-p 8080:8080 \
supabase/stripe-sync-engine:latest
```

> **Note:** Ensure the `PG_SSL_CA` and `PG_SSL_CERT` variables contain valid base64-encoded or PEM-formatted certificates as required.
39 changes: 32 additions & 7 deletions docs/edge-function.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# With Supabase Edge Functions

Create a new [Supabase](https://supabase.com) and create a new [Edge Function](https://supabase.com/docs/guides/functions/quickstart).
Create a new [Supabase](https://supabase.com) project and create a new [Edge Function](https://supabase.com/docs/guides/functions/quickstart).

## Prepare your database

Make sure to run the [migrations](./packages/sync-engine/src/database/migrations/), either by executing them manually, adding them into your CI or running this locally once:
Make sure to run the [migrations](../packages/sync-engine/src/database/migrations/), either by executing them manually, adding them into your CI, or running this locally once:

```ts
import { runMigrations } from '@supabase/stripe-sync-engine'
Expand All @@ -27,13 +27,20 @@ import 'jsr:@supabase/functions-js/edge-runtime.d.ts'
import { StripeSync } from 'npm:@supabase/stripe-sync-engine@0.37.2'

// Load secrets from environment variables
const databaseUrl = Deno.env.get('DATABASE_URL')!
const stripeWebhookSecret = Deno.env.get('STRIPE_WEBHOOK_SECRET')!
const stripeSecretKey = Deno.env.get('STRIPE_SECRET_KEY')!

// Initialize StripeSync
const stripeSync = new StripeSync({
databaseUrl,
poolConfig: {
connectionString: Deno.env.get('DATABASE_URL')!,
max: 20,
keepAlive: true,
// optional SSL configuration
ssl: {
ca: Buffer.from(Deno.env.get('DATABASE_SSL_CA')!).toString('utf-8'),
},
},
stripeWebhookSecret,
stripeSecretKey,
backfillRelatedEntities: false,
Expand All @@ -58,18 +65,36 @@ Deno.serve(async (req) => {

Deploy your Edge Function initially.

Set up a Stripe webhook with the newly deployed Supabase Edge Function url.
Set up a Stripe webhook with the newly deployed Supabase Edge Function URL.

Create a new .env file in the `supabase` directory.

```.env
DATABASE_URL="postgresql://postgres:..@db.<ref>.supabase.co:5432/postgres"
DATABASE_SSL_CA="<base64-encoded-ca>"
STRIPE_WEBHOOK_SECRET="whsec_"
STRIPE_SECRET_KEY="sk_test_..."
```

Load the secrets:

`sh
```sh
supabase secrets set --env-file ./supabase/.env
`.
```

> **Note:**
> Replace `<base64-encoded-ca>` with your actual base64-encoded certificate.

### Generating Base64 from CA Certificate

To generate a base64-encoded CA certificate, follow these steps:

1. Obtain the CA certificate file (e.g., `prod-ca-2021.crt`).
2. Use the following command on Unix-based systems:

```sh
base64 -i prod-ca-2021.crt -o CA.base64
```

3. Open the `CA.base64` file and copy its contents.
4. Use the base64 string in your configuration or environment variables.
8 changes: 4 additions & 4 deletions docs/index.md
Original file line number Diff line number Diff line change
@@ -1,20 +1,20 @@
![Sync Stripe with Postgres](./stripe-sync-engine.jpg)
![Sync Stripe with PostgreSQL](./stripe-sync-engine.jpg)

# Stripe Sync Engine

Sometimes you want to analyze your billing data using SQL. Even more importantly, you want to join your billing data to your product/business data.

This project synchronizes your Stripe account to a Postgres database. It can be a new database, or an existing Postgres database.
This project synchronizes your Stripe account to a PostgreSQL database. It can be a new database, or an existing PostgreSQL database.

---

## How it works

![How it works](./sync-engine-how.png)

- Creates a new schema `stripe` in a Postgres database, with tables & columns matching Stripe.
- Creates a new schema `stripe` in a PostgreSQL database, with tables and columns matching Stripe.
- Exposes a `/webhooks` endpoint that listens to any Stripe webhooks (via the Fastify app).
- Inserts/updates/deletes changes into the tables whenever there is a change to Stripe.
- Inserts, updates, or deletes changes into the tables whenever there is a change to Stripe.

## Webhook Support

Expand Down
Loading
Loading