Skip to content

songzcn/webdb

BranchesTags
 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

143 Commits

lib

lib

 
 

test

test

 
 

.gitignore

.gitignore

 
 

README.md

README.md

 
 

index.js

index.js

 
 

package-lock.json

package-lock.json

 
 

package.json

package.json

 
 

Repository files navigation

WebDB

A database that reads and writes records on dat:// websites. How it works

Example

Instantiate:

// in the browser
const WebDB = require('@beaker/webdb')
var webdb = new WebDB()

// in nodejs
const DatArchive = require('node-dat-archive')
const WebDB = require('@beaker/webdb')
var webdb = new WebDB('./webdb', {DatArchive})

Define your schema:

webdb.define('people', {
  // uses JSONSchema v6
  schema: {
    type: 'object',
    properties: {
      firstName: {
        type: 'string'
      },
      lastName: {
        type: 'string'
      },
      age: {
        description: 'Age in years',
        type: 'integer',
        minimum: 0
      }
    },
    required: ['firstName', 'lastName']
  },

  // secondary indexes for fast queries (optional)
  index: ['lastName', 'lastName+firstName', 'age'],

  // files to index
  filePattern: [
    '/person.json',
    '/people/*.json'
  ]
})

Then open the DB:

await webdb.open()

Next we add source archives to be indexed. The source archives are persisted in IndexedDB/LevelDB, so this doesn't have to be done every run.

await webdb.addSource('dat://alice.com')
await webdb.addSource(['dat://bob.com', 'dat://carla.com'])

Now we can begin querying the database for records.

// get any person record where lastName === 'Roberts'
var mrRoberts = await webdb.people.get('lastName', 'Roberts')

// response attributes:
console.log(mrRoberts.lastName)          // => 'Roberts'
console.log(mrRoberts)                   // => {lastName: 'Roberts', ...}
console.log(mrRoberts.getRecordURL())    // => 'dat://foo.com/bar.json'
console.log(mrRoberts.getRecordOrigin()) // => 'dat://foo.com'
console.log(mrRoberts.getIndexedAt())    // => 1511913554723

// get any person record named Bob Roberts
var mrRoberts = await webdb.people.get('lastName+firstName', ['Roberts', 'Bob'])

// get all person records with the 'Roberts' lastname
var robertsFamily = await webdb.people
  .where('lastName')
  .equalsIgnoreCase('roberts')
  .toArray()

// get all person records with the 'Roberts' lastname
// and a firstname that starts with 'B'
// - this uses a compound index
var robertsFamilyWithaBName = await webdb.broadcasts
  .where('lastName+firstName')
  .between(['Roberts', 'b'], ['Roberts', 'b\uffff'])
  .toArray()

// get all person records on a given origin
// - `:origin` is an auto-generated attribute
var personsOnBobsSite = await webdb.people
  .where(':origin')
  .equals('dat://bob.com')
  .toArray()

// get the 30 oldest people indexed
var oldestPeople = await webdb.people
  .orderBy('age')
  .reverse() // oldest first
  .limit(30)
  .toArray()

// count the # of young people
var oldestPeople = await webdb.people
  .where('age')
  .belowOrEqual(18)
  .count()

We can also use WebDB to create, modify, and delete records (and their matching files).

// set the record
await webdb.people.put('dat://bob.com/person.json', {
  firstName: 'Bob',
  lastName: 'Roberts',
  age: 31
})

// update the record if it exists
await webdb.people.update('dat://bob.com/person.json', {
  age: 32
})

// update or create the record
await webdb.people.upsert('dat://bob.com/person.json', {
  age: 32
})

// delete the record
await webdb.people.delete('dat://bob.com/person.json')

// update the spelling of all Roberts records
await webdb.people
  .where('lastName')
  .equals('Roberts')
  .update({lastName: 'Robertos'})

// increment the age of all people under 18
var oldestPeople = await webdb.people
  .where('age')
  .belowOrEqual(18)
  .update(record => {
    record.age = record.age + 1
  })

// delete the 30 oldest people
var oldestPeople = await webdb.people
  .orderBy('age')
  .reverse() // oldest first
  .limit(30)
  .delete()

Table of Contents

How to use WebDB

Table definitions

Use the define() method to define your tables, and then call webdb.open() to create them. Schemas are defined using JSON Schema v6.

Indexing sites

Use addSource() and removeSource() to control which sites will be indexed. Sources are stored in metadata and so addSource() does not have to be called for each site on load (but it doesn't cause harm if you do).

Creating queries

Queries are created with a chained function API. You can create a query from the table object using .query(), .where(), or .orderBy(). The where() method returns an object with multiple filter functions that you can use.

var myQuery = webdb.query().where('foo').equals('bar')
var myQuery = webdb.where('foo').equals('bar') // equivalent
var myQuery = webdb.where('foo').startsWith('ba')
var myQuery = webdb.where('foo').between('bar', 'baz', {includeLower: true, includeUpper: false})

Each query has a primary key. By default, this is the url attribute, but it can be changed using .where() or .orderBy(). In this example, the primary key becomes 'foo':

var myQuery = webdb.orderBy('foo')

At this time, the primary key must be one of the indexed attributes. There are 2 indexes created automatically for every record: url and origin. The other indexes are specified in your table's define() call using the index option.

Applying linear-scan filters

After the primary key index is applied, you can apply additional filters using filter(fn) and until(fn). These methods are called "linear scan" filters because they require each record to be loaded and then filtered out. (Until stops when it hits the first false response.)

var myQuery = webdb.query()
  .where('foo').equals('bar')
  .filter(record => record.beep == 'boop') // additional filter

Applying query modifiers

You can apply the following modifiers to your query to alter the output:

Executing 'read' queries

Once your query has been defined, you can execute and read the results using one of these methods:

Executing 'write' queries

Once your query has been defined, you can execute and modify the results using one of these methods:

Table helper methods

The following methods exist on the table object for query reads and writes:

Record methods

record.getRecordURL()    // => 'dat://foo.com/bar.json'
record.getRecordOrigin() // => 'dat://foo.com'
record.getIndexedAt()    // => 1511913554723

Every record is emitted in a wrapper object with the following methods:

  • getURL() The URL of the record.
  • getOrigin() The URL of the site the record was found on.
  • getIndexedAt() The timestamp of when the record was indexed.

These attributes can be used in indexes with the following IDs:

  • :url
  • :origin
  • :indexedAt

For instance:

webdb.define('things', {
  // ...
  index: [
    ':indexedAt', // ordered by time the record was indexed
    ':origin+createdAt' // ordered by origin and declared create timestamp (a record attribute)
  ],
  // ...
})
await webdb.open()
webdb.things.where(':indexedAt').above(Date.now() - ms('1 week'))
webdb.things.where(':origin+createdAt').between(['dat://bob.com', 0], ['dat://bob.com', Infinity])

Handling multiple schemas

Since the Web is a complex place, you'll frequently have to deal with multiple schemas which are slightly different. This is solved in a two-step process.

Step 1, use JSON Schema's support for multiple definitions to define as many schemas as you want to match:

  schema: {
    anyOf: [
      { 
        type: 'object',
        properties: {
          name: {type: 'string'},
          zip_code: {type: 'string'}
        },
        required: ['name', 'zip_code']
      },
      { 
        type: 'object',
        properties: {
          name: {type: 'string'},
          zipCode: {type: 'string'}
        },
        required: ['name', 'zipCode']
      }
    ]
  }

Step 2, in your index, use a definition object to support multiple attribute names under one index.

webdb.define('places', {
  // ...

  index: [
    // a simple index definition:
    'name',

    // an object index definition:
    {name: 'zipCode', def: ['zipCode', 'zip_code']}
  ]

  // ...
})

Now, when you run queries on the 'zipCode' key, you will search against both 'zipCode' and 'zip_code'. Note, however, that the records emitted from the query will not be changed by WebDB and so they may differ.

For example:

webdb.places.where('zipCode').equals('78705').each(record => {
  console.log(record.zipCode) // may be '78705' or undefined
  console.log(record.zip_code) // may be '78705' or undefined
})

To solve this, you can preprocess records.

Preprocessing records

Sometimes, you need to modify records before they're stored in the database. This can be for a number of reasons:

  • Normalization. Small differences in accepted record schemas may need to be merged (see handling multiple schemas).
  • Indexing. WebDB's index spec only supports toplevel attributes. If the data is embedded in a sub-object, you'll need to place the data at the top-level.
  • Computed attributes.

For these cases, you can use the preprocess(record) function in the table definition:

webdb.define('places', {
  // ...

  preprocess(record) {
    // normalize zipCode and zip_code
    if (record.zip_code) {
      record.zipCode = record.zip_code
    }

    // move an attribute to the root object for indexing
    record.title = record.info.title

    // compute an attribute
    record.location = `${record.address} ${record.city}, ${record.state} ${record.zipCode}`

    return record
  }

  // ...
})

These attributes will be stored in the WebDB table.

Serializing records

When records are updated by WebDB, they are published to a Dat site as a file. Since these files are distributed on the Web, it's wise to avoid adding noise to the record.

To control the exact record that will be published, you can set the serialize(record) function in the table definition:

webdb.define('places', {
  // ...

  serialize(record) {
    // write the following object to the dat site:
    return {
      info: record.info,
      city: record.city,
      state: record.state,
      zipCode: record.zipCode
    }
  }

  // ...
})

Class: WebDB

new WebDB([name])

var webdb = new WebDB('mydb')
  • name String. Defaults to 'webdb'. If run in the browser, this will be the name of the IndexedDB instance. If run in NodeJS, this will be the path of the LevelDB folder.

Create a new WebDB instance. The given name will control where the indexes are saved. You can specify different names to run multiple WebDB instances at once.

WebDB.delete([name])

await WebDB.delete('mydb')
  • name String. Defaults to 'webdb'. If run in the browser, this will be the name of the IndexedDB instance. If run in NodeJS, this will be the path of the LevelDB folder.
  • Returns Promise<Void>.

Deletes the indexes and metadata for the given WebDB.

Instance: WebDB

webdb.open()

await webdb.open()
  • Returns Promise<Object>.
    • rebuilds Array<String>. The tables which were built or rebuilt during setup.

Runs final setup for the WebDB instance. This must be run after .define() to create the table instances.

webdb.close()

await webdb.close()
  • Returns Promise<Void>.

Closes and deconstructs the WebDB instance.

webdb.define(name, definition)

  • name String. The name of the table.
  • definition Object.
    • schema Object. A JSON Schema v6 definition.
    • index Array<String or Object>. A list of attributes which should have secondary indexes produced for querying. Each index value is a keypath (see https://www.w3.org/TR/IndexedDB/#dfn-key-path) or an object definition (see below).
    • filePattern String or Array<String>. An anymatch list of files to index.
    • preprocess Function. A method to modify the record after read from the dat site. See preprocessing records.
      • record Object.
      • Returns Object.
    • serialize Function. A method to modify the record before write to the dat site. See serializing records.
      • record Object.
      • Returns Object.
  • Returns Void.

Creates a new table on the webdb object. The table will be set at webdb.{name} and be the WebDBTable type. This method must be called before open()

Indexes may either be defined as a keypath string or an object definition. The object definition has the following values:

  • name String. The name of the index.
  • def String or Array<String>. The definition of the index.

If the value of def is an array, it supports each definition. This is useful when supporting multiple schemas (learn more here).

In the index definition, you can specify compound indexes with a + separator in the keypath. You can also index each value of an array using the * sigil at the start of the name. Some example index definitions:

a simple index           - 'firstName'
as an object def         - {name: 'firstName', def: 'firstName'}
a compound index         - 'firstName+lastName'
index an array's values  - '*favoriteFruits'
many keys                - {name: 'firstName', def: ['firstName', 'first_name']}
many keys, compound      - {name: 'firstName+lastName', def: ['firstName+lastName', 'first_name+last_name']}

You can specify which files should be processed into the table using the filePattern option. If unspecified, it will default to all json files on the site ('*.json').

Example:

webdb.define('people', {
  // (uses JSONSchema v6)
  schema: {
    type: 'object',
    properties: {
      firstName: {
        type: 'string'
      },
      lastName: {
        type: 'string'
      },
      age: {
        description: 'Age in years',
        type: 'integer',
        minimum: 0
      }
    },
    required: ['firstName', 'lastName']
  },
  index: ['lastName', 'lastName+firstName', 'age'],
  filePattern: [
    '/person.json',
    '/people/*.json'
  ]
})

await webdb.open()
// the new table will now be defined at webdb.people

webdb.addSource(url)

await webdb.addSource('dat://foo.com')
  • url String or DatArchive or Array<String or DatArchive>. The sites to index.
  • Returns Promise<Void>.

Add one or more dat:// sites to be indexed. The method will return when the site has been fully indexed. The added sites are saved, and therefore only need to be added once.

webdb.removeSource(url)

await webdb.removeSource('dat://foo.com')
  • url String or DatArchive. The site to deindex.
  • Returns Promise<Void>.

Remove a dat:// site from the dataset. The method will return when the site has been fully de-indexed.

webdb.listSources()

var urls = await webdb.listSources()
  • Returns Promise<String>.

Lists the URLs of the dat:// sites which are included in the dataset.

Event: 'open'

webdb.on('open', () => {
  console.log('WebDB is ready for use')
})

Emitted when the WebDB instance has been opened using open().

Event: 'open-failed'

webdb.on('open-failed', (err) => {
  console.log('WebDB failed to open', err)
})
  • error Error.

Emitted when the WebDB instance fails to open during open().

Event: 'versionchange'

webdb.on('versionchange', () => {
  console.log('WebDB detected a change in schemas and rebuilt all data')
})

Emitted when the WebDB instance detects a change in the schemas and has to reindex the dataset.

Event: 'indexes-updated'

webdb.on('indexes-updated', (url, version) => {
  console.log('Tables were updated for', url, 'at version', version)
})
  • url String. The site that was updated.
  • version Number. The version which was updated to.

Emitted when the WebDB instance has updated the stored data for a site.

Event: 'source-missing'

webdb.on('source-missing', (url) => {
  console.log('WebDB couldnt find', url, '- now searching')
})

Emitted when a source's data was not locally available or found on the network. When this occurs, WebDB will continue searching for the data, and emit 'source-found' on success.

Event: 'source-found'

webdb.on('source-found', (url) => {
  console.log('WebDB has found and indexed', url)
})

Emitted when a source's data was found after originally not being found during indexing. This event will only be emitted after 'source-missing' is emitted.

Event: 'source-error'

webdb.on('source-error', (url, err) => {
  console.log('WebDB failed to index', url, err)
})

Emitted when a source fails to load.

Instance: WebDBTable

table.count()

var numRecords = await webdb.mytable.count()
  • Returns Promise<Number>.

Count the number of records in the table.

table.delete(url)

await webdb.mytable.delete('dat://foo.com/bar.json')
  • Returns Promise<Void>.

Delete the record at the given URL.

table.each(fn)

await webdb.mytable.each(record => {
  console.log(record)
})
  • fn Function.
    • record Object.
    • Returns Void.
  • Returns Promise<Void>.

Iterate over all records in the table with the given function.

table.filter(fn)

var records = await webdb.mytable.filter(record => {
  return (record.foo == 'bar')
})
  • fn Function.
    • record Object.
    • Returns Boolean.
  • Returns WebDBQuery.

Start a new query and apply the given filter function to the resultset.

table.get(url)

var record = await webdb.mytable.get('dat://foo.com/myrecord.json')
  • url String. The URL of the record to fetch.
  • Returns Promise<Object>.

Get the record at the given URL.

table.get(key, value)

var record = await webdb.mytable.get('foo', 'bar')
  • key String. The keyname to search against.
  • value Any. The value to match against.
  • Promise<Object>.

Get the record first record to match the given key/value query.

table.isRecordFile(url)

var isRecord = webdb.mytable.isRecordFile('dat://foo.com/myrecord.json')
  • url String.
  • Returns Boolean.

Tells you whether the given URL matches the table's file pattern.

table.limit(n)

var query = webdb.mytable.limit(10)
  • n Number.
  • Returns WebDBQuery.

Creates a new query with the given limit applied.

table.listRecordFiles(url)

var recordFiles = await webdb.mytable.listRecordFiles('dat://foo.com')
  • url String.
  • Returns Promise<Array<Object>>. On each object:
    • recordUrl String.
    • table WebDBTable.

Lists all files on the given URL which match the table's file pattern.

table.name

  • String.

The name of the table.

table.offset(n)

var query = webdb.mytable.offset(5)
  • n Number.
  • Returns WebDBQuery.

Creates a new query with the given offset applied.

table.orderBy(key)

var query = webdb.mytable.orderBy('foo')
  • key String.
  • Returns WebDBQuery.

Creates a new query ordered by the given key.

table.put(url, record)

await webdb.mytable.put('dat://foo.com/myrecord.json', {foo: 'bar'})
  • url String.
  • record Object.
  • Returns Promise<Void>.

Replaces or creates the record at the given URL with the record.

table.query()

var query = webdb.mytable.query()
  • Returns WebDBQuery.

Creates a new query.

table.reverse()

var query = webdb.mytable.reverse()
  • Returns WebDBQuery.

Creates a new query with reverse-order applied.

table.schema

  • Object.

The schema definition for the table.

table.toArray()

var records = await webdb.mytable.toArray()
  • Returns Promise<Array>.

Returns an array of all records in the table.

table.update(url, updates)

var wasUpdated = await webdb.mytable.update('dat://foo.com/myrecord.json', {foo: 'bar'})
  • url String. The record to update.
  • updates Object. The new values to set on the record.
  • Returns Promise<Boolean>.

Updates the target record with the given key values, if it exists. Returns false if the target record did not exist.

table.update(url, fn)

var wasUpdated = await webdb.mytable.update('dat://foo.com/myrecord.json', record => {
  record.foo = 'bar'
  return record
})
  • url String. The record to update.
  • fn Function. A method to modify the record.
    • record Object. The record to modify.
    • Returns Object.
  • Returns Promise<Boolean>.

Updates the target record with the given function, if it exists. Returns false if the target record did not exist.

table.upsert(url, updates)

var didCreateNew = await webdb.mytable.upsert('dat://foo.com/myrecord.json', {foo: 'bar'})
  • url String. The record to update.
  • updates Object. The new values to set on the record.
  • Returns Promise<Boolean>.

If a record exists at the target URL, will update it with the given key values. If a record does not exist, will create the record. Returns true if the target record was created.

table.upsert(url, fn)

var didCreateNew = await webdb.mytable.upsert('dat://foo.com/myrecord.json', record => {
  if (record) {
    // update
    record.foo = 'bar'
    return record
  }
  // create
  return {foo: 'bar'}
})
  • url String. The record to update.
  • fn Function. A method to modify the record.
    • record Object. The record to modify. Will be falsy if the record does ot previously exist
    • Returns Object.
  • Returns Promise<Boolean>.

Updates the target record with the given function, if it exists. If a record does not exist, will give a falsy value to the method. Returns true if the target record was created.

table.where(key)

var whereClause = webdb.mytable.where('foo')
  • key String.
  • Returns WebDBWhereClause.

Creates a new where-clause using the given key.

Event: 'index-updated'

webdb.mytable.on('index-updated', (url, version) => {
  console.log('Table was updated for', url, 'at version', version)
})
  • url String. The site that was updated.
  • version Number. The version which was updated to.

Emitted when the table has updated the stored data for a site.

Instance: WebDBQuery

query.clone()

var query = webdb.mytable.query().clone()
  • Returns WebDBQuery.

Creates a copy of the query.

query.count()

var numRecords = await webdb.mytable.query().count()
  • Returns Promise<Number>. The number of found records.

Gives the count of records which match the query.

query.delete()

var numDeleted = await webdb.mytable.query().delete()
  • Returns Promise<Number>. The number of deleted records.

Deletes all records which match the query.

query.each(fn)

await webdb.mytable.query().each(record => {
  console.log(record)
})
  • fn Function.
    • record Object.
    • Returns Void.
  • Returns Promise<Void>.

Calls the given function with all records which match the query.

query.eachKey(fn)

await webdb.mytable.query().eachKey(url => {
  console.log('URL =', url)
})
  • fn Function.
    • key String.
    • Returns Void.
  • Returns Promise<Void>.

Calls the given function with the value of the query's primary key for each matching record.

The key is determined by the index being used. By default, this is the url attribute, but it can be changed by using where() or orderBy().

Example:

await webdb.mytable.orderBy('age').eachKey(age => {
  console.log('Age =', age)
})

query.eachUrl(fn)

await webdb.mytable.query().eachUrl(url => {
  console.log('URL =', url)
})
  • fn Function.
    • url String.
    • Returns Void.
  • Returns Promise<Void>.

Calls the given function with the URL of each matching record.

query.filter(fn)

var query = webdb.mytable.query().filter(record => {
  return record.foo == 'bar'
})
  • fn Function.
    • record Object.
    • Returns Boolean.
  • Returns WebDBQuery.

Applies an additional filter on the query.

query.first()

var record = await webdb.mytable.query().first()
  • Returns Promise<Object>.

Returns the first result in the query.

query.keys()

var keys = await webdb.mytable.query().keys()
  • Returns Promise<Array<String>>.

Returns the value of the primary key for each matching record.

The key is determined by the index being used. By default, this is the url attribute, but it can be changed by using where() or orderBy().

var ages = await webdb.mytable.orderBy('age').keys()

query.last()

var record = await webdb.mytable.query().last()
  • Returns Promise<Object>.

Returns the last result in the query.

query.limit(n)

var query = webdb.mytable.query().limit(10)
  • n Number.
  • Returns WebDBQuery.

Limits the number of matching record to the given number.

query.offset(n)

var query = webdb.mytable.query().offset(10)
  • n Number.
  • Returns WebDBQuery.

Skips the given number of matching records.

query.orderBy(key)

var query = webdb.mytable.query().orderBy('foo')
  • key String.
  • Returns WebDBQuery.

Sets the primary key and sets the resulting order to match its values.

query.put(record)

var numWritten = await webdb.mytable.query().put({foo: 'bar'})
  • record Object.
  • Returns Promise<Number>. The number of written records.

Replaces each matching record with the given value.

query.urls()

var urls = await webdb.mytable.query().urls()
  • Returns Promise<Array<String>>.

Returns the url of each matching record.

query.reverse()

var query = webdb.mytable.query().reverse()
  • Returns WebDBQuery.

Reverses the order of the results.

query.toArray()

var records = await webdb.mytable.query().toArray()
  • Returns Promise<Array<Object>>.

Returns the value of each matching record.

query.uniqueKeys()

var keys = await webdb.mytable.query().uniqueKeys()
  • Returns Promise<Array<String>>.

Returns the value of the primary key for each matching record, with duplicates filtered out.

The key is determined by the index being used. By default, this is the url attribute, but it can be changed by using where() or orderBy().

Example:

var ages = await webdb.mytable.orderBy('age').uniqueKeys()

query.until(fn)

var query = webdb.mytable.query().until(record => {
  return record.foo == 'bar'
})
  • fn Function.
    • record Object.
    • Returns Boolean.
  • Returns WebDBQuery.

Stops emitting matching records when the given function returns true.

query.update(updates)

var numUpdated = await webdb.mytable.query().update({foo: 'bar'})
  • updates Object. The new values to set on the record.
  • Returns Promise<Number>. The number of updated records.

Updates all matching record with the given values.

query.update(fn)

var numUpdated = await webdb.mytable.query().update(record => {
  record.foo = 'bar'
  return record
})
  • fn Function. A method to modify the record.
    • record Object. The record to modify.
    • Returns Object.
  • Returns Promise<Number>. The number of updated records.

Updates all matching record with the given function.

query.where(key)

var whereClause = webdb.mytable.query().where('foo')
  • key String. The attribute to query against.
  • Returns WebDBWhereClause.

Creates a new where clause.

Instance: WebDBWhereClause

where.above(value)

var query = webdb.mytable.query().where('foo').above('bar')
var query = webdb.mytable.query().where('age').above(18)
  • value Any. The lower bound of the query.
  • Returns WebDBQuery.

where.aboveOrEqual(value)

var query = webdb.mytable.query().where('foo').aboveOrEqual('bar')
var query = webdb.mytable.query().where('age').aboveOrEqual(18)
  • value Any. The lower bound of the query.
  • Returns WebDBQuery.

where.anyOf(values)

var query = webdb.mytable.query().where('foo').anyOf(['bar', 'baz'])
  • values Array<Any>.
  • Returns WebDBQuery.

Does not work on compound indexes.

where.anyOfIgnoreCase(values)

var query = webdb.mytable.query().where('foo').anyOfIgnoreCase(['bar', 'baz'])
  • values Array<Any>.
  • Returns WebDBQuery.

Does not work on compound indexes.

where.below(value)

var query = webdb.mytable.query().where('foo').below('bar')
var query = webdb.mytable.query().where('age').below(18)
  • value Any. The upper bound of the query.
  • Returns WebDBQuery.

where.belowOrEqual(value)

var query = webdb.mytable.query().where('foo').belowOrEqual('bar')
var query = webdb.mytable.query().where('age').belowOrEqual(18)
  • value Any. The upper bound of the query.
  • Returns WebDBQuery.

where.between(lowerValue, upperValue[, options])

var query = webdb.mytable.query().where('foo').between('bar', 'baz', {includeUpper: true, includeLower: true})
var query = webdb.mytable.query().where('age').between(18, 55, {includeLower: true})
  • lowerValue Any.
  • upperValue Any.
  • options Object.
    • includeUpper Boolean.
    • includeLower Boolean.
  • Returns WebDBQuery.

where.equals(value)

var query = webdb.mytable.query().where('foo').equals('bar')
  • value Any.
  • Returns WebDBQuery.

where.equalsIgnoreCase(value)

var query = webdb.mytable.query().where('foo').equalsIgnoreCase('bar')
  • value Any.
  • Returns WebDBQuery.

Does not work on compound indexes.

where.noneOf(values)

var query = webdb.mytable.query().where('foo').noneOf(['bar', 'baz'])
  • values Array<Any>.
  • Returns WebDBQuery.

Does not work on compound indexes.

where.notEqual(value)

var query = webdb.mytable.query().where('foo').notEqual('bar')
  • value Any.
  • Returns WebDBQuery.

Does not work on compound indexes.

where.startsWith(value)

var query = webdb.mytable.query().where('foo').startsWith('ba')
  • value Any.
  • Returns WebDBQuery.

Does not work on compound indexes.

where.startsWithAnyOf(values)

var query = webdb.mytable.query().where('foo').startsWithAnyOf(['ba', 'bu'])
  • values Array<Any>.
  • Returns WebDBQuery.

Does not work on compound indexes.

where.startsWithAnyOfIgnoreCase(values)

var query = webdb.mytable.query().where('foo').startsWithAnyOfIgnoreCase(['ba', 'bu'])
  • values Array<Any>.
  • Returns WebDBQuery.

Does not work on compound indexes.

where.startsWithIgnoreCase(value)

var query = webdb.mytable.query().where('foo').startsWithIgnoreCase('ba')
  • value Any.
  • Returns WebDBQuery.

Does not work on compound indexes.

How it works

WebDB abstracts over the DatArchive API to provide a simple database-like interface. It is inspired by Dexie.js and built using LevelDB. (In the browser, it runs on IndexedDB using level.js.

WebDB scans a set of source Dat archives for files that match a path pattern. Those files are indexed so that they can be queried easily. WebDB also provides a simple interface for adding, editing, and removing records from archives.

WebDB sits on top of Dat archives. It duplicates ingested data into IndexedDB, which acts as a throwaway cache. The cached data can be reconstructed at any time from the source Dat archives.

WebDB treats individual files in the Dat archive as individual records in a table. As a result, there's a direct mapping for each table to a folder of JSON files. For instance, if you had a posts table, it might map to the /posts/*.json files. WebDB's mutators, e.g., put, add, update, simply writes records as JSON files in the posts/ directory. WebDB's readers and query-ers, like get() and where(), read from the IndexedDB cache.

WebDB watches its source archives for changes to the JSON files that compose its records. When the files change, it syncs and reads the changes, then updates IndexedDB, keeping query results up-to-date. Roughly, the flow is: put() -> archive/posts/12345.json -> indexer -> indexeddb -> get().

About

A database that reads and writes records on dat:// websites.

Resources

Readme
Activity

Stars

0 stars

Watchers

3 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

No packages published

Languages

  • JavaScript 100.0%