Skip to content
master
Switch branches/tags
Code

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 
src
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Offline First Database Comparison

In this project I have implemented the exact same chat application with different database technologies. You can use it to compare metrics and learn about the differences. The chat app is a web based angular application, with functionality similar to Whatsapp Web.

chat app

Implemented Databases:

  • AWS Amplify Datastore
  • Firebase Firestore
  • PouchDB with IndexedDB adapter & CouchDB replication
  • RxDB with PouchDB Storage & GraphQL replication
  • RxDB with LokiJS Storage & GraphQL replication
  • WatermelonDB with LokiJS adapter (no backend sync atm)

Metrics

All metrics are measured automatically via code in a browser tests (chrome:headless). The results heavily depend on the developers device. You should compare the values relative to another and not as absolute values. Also you might want to create new metrics that better represent how you would use the respective database.

You can reproduce these values by running sh measure-metrics.sh in the root folder.

Metric \ Project aws firebase pouchdb rxdb-lokijs rxdb-pouchdb watermelondb
First angular component render 211ms 276ms 193ms 209ms 212ms 244ms
Page load time 299ms 187ms 281ms 295ms 297ms 336ms
First full render 452ms 1264ms 1173ms 643ms 1171ms 382ms
Insert one message 19ms 304ms 19ms 18ms 36ms 8ms
Inserting 20 messages one after another 646ms 6067ms 305ms 265ms 354ms 31ms
Inserting 20 messages in parallel 184ms 5353ms 101ms 191ms 329ms 175ms
Message insert to message list change 45ms 35ms 170ms 12ms 23ms 3ms
Message search query time 456ms 307ms 260ms 28ms 64ms 30ms
First full render with many messages 413ms 1712ms 1649ms 571ms 1630ms 250ms
Storage usage 200kb 2186kb 1174kb 1209kb 1039kb 953kb
Bundle size, plain JavaScript 1773kb 912kb 801kb 1027kb 1043kb 852kb
Bundle size, minified+gzip 408kb 223kb 192kb 251kb 265kb 199kb

Metrics Explanation

  • Page load time: How long does it take to download and parse the JavaScript bundle.
  • First angular component render: How long does it take for the first angular component to be rendered.
  • First full render: How long does it take until all relevant data is displayed for the first time.
  • Insert one message: How long does it take to insert a single message.
  • Inserting 20 messages one after another: How long does it take to insert 20 messages in serial.
  • Inserting 20 messages in parallel: How long does it take to insert 20 messages in parallel.
  • Message insert to message list change: How long does it take until a new message is rendered to the dom.
  • User change to message list change: How long does it take from changing the user to the displaying of the new messages list.
  • Message search query time: How long does it take to search for a given message by regex/like-operator.
  • First full render with many messages: Time to first full render when many messages exist.
  • Storage usage: Size of the stored IndexedDB database after inserting the full test dataset.
  • Bundle size, plain JavaScript: The full JavaScript bundle size, without minification or gzip.
  • Bundle size, minified+gzip: The full JavaScript bundle size after minification and gzip compression.

Investigations

Why is LokiJS so much faster?

WatermelonDB and the RxDB-LokiJS project use the LokiJS database as storage, which is an in memory database that regularly persists the data to IndexedDB either on interval, or when the browser tab is closed. Keeping and processing the data in memory has the benefit of being much faster, but it also has its downsides:

  • Data can be lost when the JavaScript process is killed ungracefully like when the browser crashes or the power of the PC is terminated.
  • There is no multi-tab-support with plain LokiJS. The data is not shared between multiple browser tabs of the same origin. RxDB handles that by adding its own multi tab handling.

Why is Firebase so slow on first render?

On the first page load, Firebase ensures that the local data is equal to the server side state. This means that the client has to be online at application startup which is the reason why Firebase is not completely offline first. To ensure the equalness of client side data, Firebase has to perform several requests to the backend, before the database will respond to queries. This makes the inital page load slow, and it becomes even more slower, the more data exists and has to be validated.

Why is PouchDB so slow?

For the PouchDB and RxDB (with PouchDB storage) I used the old Indexeddb adapter. It is much less optimized than the new adapter, but the new one made problems with returning the correct query results. Theses problems have been fixed on the PouchDB master branch, but I have to wait for the next PouchDB release. I will update the repo when this change can be done.

Why does AWS Datastore need so much less storage space?

AWS Datastore does not save any metadata together with the documents. Instead only the plain documents are stored in IndexedDB. They can do this because they only allow simple queries and do not keep a local version history.

Feature Map

Feature \ Project aws firebase pouchdb rxdb-pouchdb rxdb-lokijs watermelondb
Offline First No, login required Partially, must be online on first page load Yes Yes Yes Yes
Realtime Replication Yes Yes Yes Yes Yes Partially, must be implemented by hand
Multi Tab Support Yes Yes No Yes Yes Partially, relies on online sync
Observable Queries No Yes No Yes Yes Yes
Complex Queries No Yes Yes Yes Yes Yes
Client Side Encryption No No Yes Yes Yes No
Schema Support Yes No No Yes Yes Yes
Custom Backend No No No Yes Yes Yes
Custom Conflict Handling Yes No Yes Yes No No

Starting the projects

All sub-projects use the same port and so can not be started in parallel.

Installation

  • You must have installed Node.js
  • Clone this project
  • In the root folder, run npm install to install the dependencies.
  • In the root folder, run npm run build to build all projects.

Firebase Firestore

  • Run npm run start:firebase to start the mock server and the production build frontend.

  • Or run npm run dev:firebase to start the mock server and the development frontend server.

  • Open http://localhost:3000/ to browse the frontend.

AWS Amplify & Datastore

The official AWS mock does not allow a live replication at this point. So you first have to setup an amplify project in the ./projects/aws folder by using this tutorial

  • Run npm run start:aws to start the mock server and the production build frontend.

  • Or run npm run dev:aws to start the mock server and the development frontend server.

  • Open http://localhost:3000/ to browse the frontend.

PouchDB

  • Run npm run start:pouchdb to start the mock server and the production build frontend.

  • Or run npm run dev:pouchdb to start the mock server and the development frontend server.

  • Open http://localhost:3000/ to browse the frontend.

RxDB

  • Run npm run start:rxdb to start the mock server and the production build frontend.

  • Or run npm run dev:rxdb to start the mock server and the development frontend server.

  • Open http://localhost:3000/ to browse the frontend.

WatermelonDB

  • Run npm run start:watermelondb to start the mock server and the production build frontend.

  • Or run npm run dev:watermelondb to start the mock server and the development frontend server.

  • Open http://localhost:3000/ to browse the frontend.

TODOs

Pull requests are welcomed. Please help implementing more examples:

  • Meteor (with the IndexedDB offline first plugin).
  • WatermelonDB backend replication.
  • AWS Ampflify local backend mock with realtime replication.
  • GunDB.