Simple document db made for infinitely scalable globally distributed reads.
Clone or download
Latest commit 84ecd0b Dec 11, 2018

README.md

NearDB is a simple database that leverages cloud infrastructure like document storage and CDN to deliver an inexpensive unbelievably scalable document database optimized for reads and perfect for edge applications.

Codacy Badge Build Status Coverage Status

Motivation

While working on building edge applications for higher performance and lower latency there is a need store persistent data also on edge.

There are multiple distributed database solutions but they are very involved and costly while having a much lower global footprint than a CDN.

The idea came up to leverage ubiquitous and mature infrastructure like cloud storage and CDNs to deliver a persistent data solution from the edge.

Use with Edge Apps/Functions

Works with the following for database storage

Who is this for

This is perfect for persistent data that is read frequently and needs to be avaialble on the edge application to deliver dynamic data while keeping the costs low. Some examples of the best uses are:

  • Key-value
  • Configuration
  • Cached data

Probably not for you if

  • You plan on using this as your primary database for an app that has complex data needs.
  • You need transactions. (I have some ideas on how to accomplish this, but its currently not implemented.)
  • Do many writes/sec in the same document. Reads are incredibly efficient, fast and inexpensive; however, writes are always at the origin.

Installing

npm install neardb
# or
yarn add neardb

Usage

import Neardb from 'neardb'

const config = {
    database: 'bucketName'
    //...
}

const neardb = NearDB.database(config);

Config Options

Property Description Type
database Bucket name that is used to store the data on object storage string
cdn CDN configurations for get requests. object
- url Http endpoint of the CDN that has the bucket as the origin string
- headers Configure headers needed for your CDN here. Cache rules, or tokens can be set up through object
cacheExpiration Number in milliseconds which you would like to keep the data cached locally number
storage Object storage configuration object
- endpoint Endpoint to object storage string
- useSSL Set this if you are using a secure connection bool
- accessKeyId Access Key to object storage service string
- secretAccessKey Secret Key to object storage service string
signatureVersion Identifies the version of signature that you want to support for authenticated requests string
S3ForcePathStyle Whether to force path style URLs for S3 objects. bool
indices Create a collection index when document is stored (experimental) bool

References

You are able to store the reference of a collection or document, and use the reference when interacting with them.

const statesRef = nearDB.collection('states');

const nyRef = nearDB.collection('states').doc('ny')

Add a Document

By using set for document creation allows you to set the document id

nearDB.collection('states').doc('ny').set({
    name: 'New York',
    population: 19849399,
    largestCity: 'New York City'
})

By calling add on the collection a document id is auto-generated

nearDB.collection('states').add({
    name: 'New York',
    population: 19849399,
    largestCity: 'New York City'
})

Update a Document

By using set if the document does not exist it will create it. If it does exist you can use set to overwrite the whole document.

nearDB.collection('states').doc('ny').set({
    name: 'New York',
    population: 19849399,
    largestCity: 'New York City',
    eastCoast: true
})

If you wish to update fields within a document without overwriting all the data you should use update

nearDB.collection('states').doc('ny').update({
    eastCoast: true
})

To delete a value without overwriting the whole document you the following helper constant

nearDB.collection('states').doc('ny').update({
    eastCoast: NearDB.field.deleteValue
})

Delete a Document

By using delete the whole document will be deleted from the bucket

nearDB.collection('states').doc('ny').delete()

Get a Document

You can get the content of a single document by using get

nearDB.collection('states').doc('ny').get()

There a few options that you are able to pass on get depending where you want to get the data from.

By default get will try to retrieve the document the following way.

  1. Get local data if it exists and has not expired
  2. If CDN is configured will get from there
  3. If there is no local cache and CDN is not configured, it will get from the origin.

There are a few options you are able to pass to force where you get the data from.

const options = {
    // Gets data from origin even if 
    // there is local cache and a cdn configured
    source: 'origin' 

    // Gets data from edge even if 
    // there is local cache and a cdn configured
    // source: 'edge' 
}
nearDB.collection('states').doc('ny').get(options)

Running the tests

npm run test

Documentation

See documentation generated from TypeDoc here

Dependencies

  • aws sdk - AWS SDK for JavaScript in the browser and node.js
  • axios - Promise based HTTP client for the browser and node.js

Inspiration

The design of NearDBs API is heavily inspired by Firestore.

Contributing

Please read CONTRIBUTING.md for details on our code of conduct, and the process for submitting pull requests to us.

License

This project is licensed under the MIT License - see the LICENSE file for details