Node.js React Native SQLite Seeder

This monorepo demonstrates how to generate seeded SQLite database files using the PowerSync Node.js SDK. These SQLite database files can then be uploaded to a storage provider, such as S3 and downloaded by clients, effectively bypassing the need to perform an initial sync on the client side. This repo also includes a React Native Expo demo client, that downloads and prepares the SQLite files for use by the PowerSync React Native SDK.

Prerequisites

• Docker (for local testing)
• Node.js > 20.x.x, or use the version specified in the .nvmrc in the /backend directory
• AWS account and a S3 bucket configured and ready to use

Note

This demo does not depend on a PowerSync Cloud instance and can run locally with no sign-up required. If you want to use a PowerSync Cloud instance, see details in Setting up the React Native demo app section.

Project Structure

• backend
  • This directory contains the Node.js application which runs a cron job (using node-cron) to schedule a job to pre-seed the database files, and starts an HTTP server which serves the endpoints needed for the React Native client demo app.
  • See src/Seeder.ts for details on how the PowerSync Node.js SDK is used to create and back-up the pre-seeded SQLite databases.
  • In the src/index.ts the app will start two API endpoints:
    • /token - This route is used to generate a JWT that's used by the React Native client to connect to the local PowerSync instance.
    • /database - This route is used to generate a signed S3 URL that is used to download the pre-seeded database.
  • docker
    • This contains the self-hosted PowerSync Service, which includes the following
      • init-scripts - This is a SQL script that runs after the Postgres database has started and seeds the database with sample data that's used as part of the pre-seeded databases.
      • compose.yaml - Defines all the services used for the local stack.
      • postgres.yaml - Defines the source database config.
      • powersync.yaml - Defines the PowerSync Service config and Sync Rules.
• react-native-client
  • This contains the source code for the React Native Expo app which; checks if the SQLite DB exists, downloads the SQLite file from S3, and connects to the local PowerSync Service, after the SQLite database has been prepared.

Note

When the Node.js application connects to the PowerSync Service to create the pre-seeded databases, it will store the SQLite files in the databases directory. Once the initial sync is completed, the application will remove the client_id key/value from the ps_kv table and will execute the VACUMM INTO '<path>' SQLite command to create a backup SQLite file. The backed-up SQLite file is then uploaded to S3 and downloaded to the client.

Usage

Follow the sections below, in order, to set up the project and test it out.

Setting up the local PowerSync Service

1. Navigate to the backend directory and set up the .env:

cd backend && cp .env.template .env

2. Start the PowerSync Service stack:

npm run dev:powersync

This starts a local PowerSync Service instance on port 8080 and two separate Postgres databases. The database listening on 5432 will replicate to PowerSync and contains the source data. The other database listening on 5431 is used as bucket storage, required by the PowerSync Service.

Setting up the backend app

1. Update the .env and set your AWS Secret Key, Access Key ID and Region:

AWS_SECRET_KEY=
AWS_ACCESS_KEY_ID=
AWS_REGION=

2. Open a new terminal window to build and run the backend app:

cd backend
npm run build
npm run start

This starts the HTTP server on port 3000 and starts the cron schedule, which runs every 30 seconds. You can adjust this as needed for your use case.

Setting up the React Native demo app

Note

This demo will not work with Expo Go, because we're using op-sqlite.

1. In a new terminal window, navigate to the react-native-client directory:

cd react-native-client 

2. Install dependencies:

npm install

3. Run the app:

npm run ios
# OR
npm run android

With the app launched, here's what's going to happen:

1. The app will check if the SQLite file exists in the file system.
   1. For demo purposes, it will delete the already existing SQLite file.
2. The app will request a signed URL from the Node.js backend app to download the SQLite file from S3
3. The SQLite file will download from S3 and be written to the Path.document directory on the client. This is a place to store files that are safe from being deleted by the system.
4. After the file is downloaded, the PowerSyncDatabase class is instantiated and points to the newly downloaded SQLite file in the directory specified above.
5. The client will insert a new client_id key/value pair in the ps_kv table.
6. The client will call .connect() and start the stream with the PowerSync instance.

Note

If you want to connect to a PowerSync Cloud instance, simply update the fetchCredentials() to point to your PowerSync Cloud instance. Also keep in mind that you will need to fetch a token from your auth provider if you don't want to the client to connect to the local instance.