New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow expressions in addition to keyPaths when creating indexes #33

Open
inexorabletash opened this Issue Aug 10, 2015 · 6 comments

Comments

Projects
None yet
2 participants
@inexorabletash

This comment has been minimized.

Show comment
Hide comment
@inexorabletash

inexorabletash Aug 10, 2015

Member

Lots of stuff to work through here, e.g. do we make expressions side-effect free by running in isolated execution environments? (VM implementers say: that's not cheap!) Also, loading libraries (e.g. text segmentation, ...) means these are probably non-trivial expressions... so do we abandon the notion that these are pure functions over the values and give sites the ability to do dumb (dangerous?) things?

Member

inexorabletash commented Aug 10, 2015

Lots of stuff to work through here, e.g. do we make expressions side-effect free by running in isolated execution environments? (VM implementers say: that's not cheap!) Also, loading libraries (e.g. text segmentation, ...) means these are probably non-trivial expressions... so do we abandon the notion that these are pure functions over the values and give sites the ability to do dumb (dangerous?) things?

@inexorabletash

This comment has been minimized.

Show comment
Hide comment
@inexorabletash

inexorabletash Apr 15, 2016

Member

Worklets might be the way to go for this. They do require developers to be consistent, but that seems like an acceptable constraint.

Very rough sketch: I haven't thought this through in detail, but something like:

  • IDBDatabase gets a worklet property; code can be loaded in via db.worklet.import(...)
  • IDBWorklet exposes a registerIndexExpression(name, func) global function
  • When creating an index, you can specify name() as the keyPath

Developers need to remember to call db.worklet.import(...) on every connection to the DB that might alter an index (create or add/update values). This import is also async, so maybe a new step in the open() phase. So the boilerplate would look like:

var open = indexedDB.open('stuff');
// new "registerworklet" phase always precedes "upgradeneeded" (if needed) and "success"
open.onregisterworklet = function() {
  var db = open.result;
  db.worklet.import('my_worklet_code.js');
};
// "upgradeneeded" implicitly delayed until all worklet imports are complete
open.onupgradeneeded = function() {
  var db = open.result;
  var store = db.createObjectStore('records');
  store.createIndex('freeText', 'words()', {multiEntry: true});
  store.createIndex('area', 'area()');
};
open.onsuccess = function() {
  var db = open.result;
  ...
};

And my_worklet_code.js would look like:

registerIndexExpression('area', r => r.width * r.height);
registerIndexExpression('words', r => r.body.split(/\s+/));
Member

inexorabletash commented Apr 15, 2016

Worklets might be the way to go for this. They do require developers to be consistent, but that seems like an acceptable constraint.

Very rough sketch: I haven't thought this through in detail, but something like:

  • IDBDatabase gets a worklet property; code can be loaded in via db.worklet.import(...)
  • IDBWorklet exposes a registerIndexExpression(name, func) global function
  • When creating an index, you can specify name() as the keyPath

Developers need to remember to call db.worklet.import(...) on every connection to the DB that might alter an index (create or add/update values). This import is also async, so maybe a new step in the open() phase. So the boilerplate would look like:

var open = indexedDB.open('stuff');
// new "registerworklet" phase always precedes "upgradeneeded" (if needed) and "success"
open.onregisterworklet = function() {
  var db = open.result;
  db.worklet.import('my_worklet_code.js');
};
// "upgradeneeded" implicitly delayed until all worklet imports are complete
open.onupgradeneeded = function() {
  var db = open.result;
  var store = db.createObjectStore('records');
  store.createIndex('freeText', 'words()', {multiEntry: true});
  store.createIndex('area', 'area()');
};
open.onsuccess = function() {
  var db = open.result;
  ...
};

And my_worklet_code.js would look like:

registerIndexExpression('area', r => r.width * r.height);
registerIndexExpression('words', r => r.body.split(/\s+/));
@inexorabletash

This comment has been minimized.

Show comment
Hide comment
@inexorabletash

inexorabletash Jun 20, 2016

Member

Whether the binding (name -> function) should occur in the owning context or the worklet context should be consistent with other uses of worklets.

Member

inexorabletash commented Jun 20, 2016

Whether the binding (name -> function) should occur in the owning context or the worklet context should be consistent with other uses of worklets.

@bfgeek

This comment has been minimized.

Show comment
Hide comment
@bfgeek

bfgeek Jun 21, 2016

This looks pretty fun :)

Looks like result of an index can only be primitives + array + binary?

Is it possible for a index to have multiple types? I.e.

registerIndexExpression('foo', r => { return r.bool ? r.str : r.arrayType });

bfgeek commented Jun 21, 2016

This looks pretty fun :)

Looks like result of an index can only be primitives + array + binary?

Is it possible for a index to have multiple types? I.e.

registerIndexExpression('foo', r => { return r.bool ? r.str : r.arrayType });
@inexorabletash

This comment has been minimized.

Show comment
Hide comment
@inexorabletash

inexorabletash Jun 21, 2016

Member

The "result" is a key, which (in Indexed DB) is a number, date, string, binary (proposed!), or array-of-key. See https://w3c.github.io/IndexedDB/#key-construct. Boolean is not supported (but see #76).

There's no requirement that keys within a store or index have the same type or "shape", so your example is valid.

Member

inexorabletash commented Jun 21, 2016

The "result" is a key, which (in Indexed DB) is a number, date, string, binary (proposed!), or array-of-key. See https://w3c.github.io/IndexedDB/#key-construct. Boolean is not supported (but see #76).

There's no requirement that keys within a store or index have the same type or "shape", so your example is valid.

@inexorabletash

This comment has been minimized.

Show comment
Hide comment
@inexorabletash

inexorabletash Sep 28, 2016

Member

If you're watching this issue...

One of the primitives necessary for building full text search on top of indexes is text segmentation. @littledan has a proposal for adding this to ECMA-402 (the ECMAScript Intl object). He'd love feedback:

Member

inexorabletash commented Sep 28, 2016

If you're watching this issue...

One of the primitives necessary for building full text search on top of indexes is text segmentation. @littledan has a proposal for adding this to ECMA-402 (the ECMAScript Intl object). He'd love feedback:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment