Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use IndexedDB indexes on gatherLocalChanges() #315

Merged
merged 2 commits into from Feb 2, 2016
Merged

Conversation

leplatrem
Copy link
Contributor

  • Take advantage of index if available for filtered field
  • Optimized gatherLocalChanges()
  • Move reduceRecords to IDB and Firefox storage (long term: replace it with generated sql)

Small benchmark: 4500 records in local store. None of them changed.

Before (1000+):
capture d ecran de 2016-01-29 14-38-28

After (<50):

capture d ecran de 2016-02-01 11-32-58

@@ -205,12 +205,30 @@ export default class IDB extends BaseAdapter {
* @override
* @return {Promise}
*/
list() {
list(params={}) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should provide default property values for the params object; eg Object.keys(params.filters) can fail when no filters object is provided.

@leplatrem
Copy link
Contributor Author

Ok, I just realized we were interating on the whole database even when no change was fetched from server!

Sync() is now 33 msec when no change is fetched (=138-105 network) :)

capture d ecran de 2016-02-01 11-32-58

return Promise.all([
this.list({order: "", filters: {_status: "deleted"}}, {includeDeleted: true}),
this.list({order: "", filters: {_status: "synced"}}, {includeDeleted: true}),
])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: refactoring suggestion:

const listStatus = (_status) => {
  return this.list({order: "", filters: {_status}}, {includeDeleted: true});
};
return Promise.all([listStatus("deleted"), listStatus("synced")])

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As we're using the same query type later, we could even extract the function as something reusable:

function listStatus(coll, status, options={includeDeleted: true}) {
  return coll.list({order: "", filters: {_status: status}}, options);
};

Or if we want to pick all records matching provided statuses:

function listStatuses(coll, statuses, options={includeDeleted: true}) {
  return Promise.all(statuses.map(status => {
    return coll.list({order: "", filters: {_status: status}}, options);
  })).then((...results) => [].concat.apply([], results));
};

Usage:

return listStatuses(this, ["deleted", "synced"]);

@n1k0 n1k0 added this to the 2.0.0 milestone Feb 1, 2016
Optimized synchronization at several places:
- during scan of un-synced records
- when no change was made on server

In terms of API, the adapters are now in charge of
adapting #list() parameters (filters and sort).

This PR does not break any API, the filtering and sorting
is done like before using a reduce from utils.js
@leplatrem leplatrem changed the title WIP Use IndexedDB indexes on gatherLocalChanges() Use IndexedDB indexes on gatherLocalChanges() Feb 1, 2016
@leplatrem
Copy link
Contributor Author

Ready for review/merge.

@@ -205,12 +206,33 @@ export default class IDB extends BaseAdapter {
* @override
* @return {Promise}
*/
list() {
list(params={filters: {}, order: ""}) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't we simply avoid passing order here?

@leplatrem
Copy link
Contributor Author

r+

n1k0 added a commit that referenced this pull request Feb 2, 2016
Use IndexedDB indexes on gatherLocalChanges()
@n1k0 n1k0 merged commit f04bfaf into master Feb 2, 2016
@n1k0 n1k0 deleted the use-idb-keyrange branch February 2, 2016 08:59
@n1k0 n1k0 removed the in progress label Feb 2, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants