Skip to content

Commit

Permalink
Merge branch 'release/1.1.0'
Browse files Browse the repository at this point in the history
  • Loading branch information
@Akeri
Akeri committed Mar 23, 2019
2 parents 796457e + b8f7d49 commit cc49a04
Show file tree
Hide file tree
Showing 10 changed files with 978 additions and 784 deletions.
74 changes: 59 additions & 15 deletions README.md
View file
Expand Up @@ -7,9 +7,12 @@

Give models the ability to query native **MongoDB aggregates** and **build instances** from results.

**Highlights**

* Accepts both Loopback filter's features and pipeline stages, it will merge in a single parsed pipeline to aggregate.
* Accepts relations' fields within the root where, it will be handled as $lookup stages.
* Refactor the logic from Loopback which is responsible for building the model instances and take advantage of it.
* Supports both callbacks and promises.

This Loopback mixin is intended to be used together with MongoDB connector.
Works for Loopback 2 and 3.
Expand Down Expand Up @@ -74,7 +77,7 @@ Invoke `aggregate` method passing either:

### Basic example

Find a random sample of 3 persons born after 1980:
> Find a random sample of 3 persons born after 1980:
```js
app.models.Person.aggregate({
Expand All @@ -93,7 +96,7 @@ $lookup stages will be automatically generated to reach those relations and filt
it works like a "LEFT JOIN" feature, however it's still necessary to add the "include" filter
if you require the relation to be hydrated.

Example: Bring persons who are part of a team in which there is some person who is born after 2001
> Example: Bring persons who are part of a team in which there is some person who is born after 2001
```js
app.models.Person.aggregate({
Expand All @@ -104,14 +107,36 @@ app.models.Person.aggregate({
});
```

Note: It works for hasOne, belongsTo and hasMany. Filtering by embedded properties is not affected and continues to work as usual.
Note: It works for hasOne, belongsTo and hasMany. Filtering by embedded properties is not affected and continues to work as usual.

### Do not build instances

Some queries are intended to retrieve data that can not be transformed into model instances.
`aggregate` method will attempt to build instances by default, but this behavior can be disabled
passing an options object `{build: false}` as second argument.

> Example: Bring count of persons by company
```js
app.models.Person.aggregate({
aggregate: [{
$group: {
_id: '$companyId',
total: {$sum: 1},
},
}],
}, {build: false}, (err, groups) => {
if (err) return done(err);
// Each group should be a plain object with just 'id' and 'total' attributes
});
```

### Build instances on demand

The aggregate result often needs some processing before building the model instances.
It's possible to postpone the build phase until the models' data are resolved.

Example: Bring the persons count together with a specific page
> Example: Bring the persons count together with a specific page
```js
Person.aggregate([{
Expand All @@ -125,25 +150,24 @@ Person.aggregate([{
total: 1,
items: {$slice: ['$objects', pageStart, pageLength]},
},
}], {build: false}, (err, data, build) => {
}], {buildLater: true}, (err, [data, build]) => {
if (err) return next(err);
// data is a plan structure {total, items} where items is a array of plain objects, not model instances.
// data is a plain structure {total, items} where items is an array of documents, not model instances.
build(data.items, (err, persons) => {
if (err) return next(err);
// now you got persons as Peron model instances
// now you got persons as Person model instances
});
});
```

Note: Pipeline array can be directly passed as argument. Also stage names can obviate "$" character.

Few more things worth commenting here:

* In this case, model documents are not brought as root result,
so we need to disable the automatic build through an options object `{build: false}`.
* When automatic build is disabled, we'll be provided with a build function as a third argument of the callback.
Person instances are finally obtained calling build function with `data.items`.
* Build on demand feature will be always available as a model static method `Model.buildResult`.
so we could disable the automatic building by just passing the option `{build: false}`,
but in this case, what we really need is the option `{buildLater: true}`.
* The difference is that `buildLater` will provide us a build function (together with native documents) to invoke by our hand .
Person instances will be finally obtained by calling such function passing `data.items`.
* Build on demand feature it's available as a model static method `Model.buildResult`.

Note: Pipeline array can be directly passed as argument. Also stage names can obviate "$" character.

### GeoNear example

Expand All @@ -167,6 +191,26 @@ app.models.Company.aggregate({
});
```

### Promise support

Methods `aggregae` and `buildResult` support either callback or promise usage.
All the examples above are made with callbacks.
Below it's shown how it's made with promise style.

> Example: Find a random sample of 3 persons born after 1980:
```js
app.models.Person.aggregate({
where: {birthDate: {gt: new Date('1980')}},
aggregate: [{$sample: {size: 3}}],
}).then((persons) => {
// persons are Person model instances
}).catch((err) => {
// handle an error
});
```


## Advanced configuration

Enable the mixin passing an options object instead of just true.
Expand Down
166 changes: 94 additions & 72 deletions lib/aggregate.js
View file
@@ -1,6 +1,7 @@
'use strict';

const _ = require('lodash');
const pg = require('polygoat');
const Aggregation = require('./aggregation');
const rewriteId = require('./rewrite-id');
const debug = require('debug')('loopback:mixins:aggregate');
Expand All @@ -10,7 +11,8 @@ module.exports = function (Model, options) {
buildBehavior(Model);

const settings = _.merge({
build: true, // Build model instances right after getting results from MongoDB
build: true, // Automatically build model instances right after getting results from MongoDB
buildLater: false, // Disable automatic build and provide a function to build on demand
buildOptions: { // Options that will be passed to build process
notify: true, // Notify model operation hooks on build
},
Expand All @@ -23,85 +25,105 @@ module.exports = function (Model, options) {

/**
* Perform MongoDB native aggregate.
* @param {Object} filter Loopback query filter.
* @param {Object} options Loopback query options.
* @param {Function} next Callback.
* @param {Object} [filter] Loopback query filter.
* @param {Object} [options] Loopback query options.
* @param {Function} [next] Callback.
* @returns {Promise<*>} Only if callback is not passed.
*/
Model.aggregate = function (filter, options, next) {
if (options === undefined && next === undefined) {
next = filter;
filter = null;
} else if (next === undefined) {
next = options;
options = {};
}
options = _.merge({}, settings, options);
this.applyScope(filter);
const modelName = this.modelName;
const connector = this.getConnector();
const aggregation = this.getAggregation();
if (!_.isEmpty(options.mongodbArgs)) {
if (options.mongodbArgs.explain === true) {
options.build = false;
// Process arguments
const args = [].slice.call(arguments);
const params = [];
while (args.length) {
const arg = args.shift();
if (!_.isFunction(arg)) {
params.push(arg);
} else {
next = arg;
}
aggregation.setOptions(options.mongodbArgs);
}
if (_.isPlainObject(filter)) {
if (filter.near) {
aggregation.near(filter.near);
[filter, options] = params;
// Make logic work either with callback or promise
return pg((done) => {
options = _.merge({}, settings, options);
if (options.buildLater === true) {
options.build = false;
}
if (filter.where) {
const relationalFields = this.whichFieldsAreRelational(filter.where);
const directFields = _.difference(_.keys(filter.where), relationalFields);
if (directFields.length) {
const directWhere = _.pick(filter.where, directFields);
const where = connector.buildWhere(modelName, directWhere);
aggregation.match(where);
}
if (relationalFields.length) {
const relationalWhere = _.pick(filter.where, relationalFields);
buildLookup(aggregation, relationalWhere);
aggregation.coalesce(relationalWhere);
aggregation.match(relationalWhere);
this.applyScope(filter);
const modelName = this.modelName;
const connector = this.getConnector();
const aggregation = this.getAggregation();
if (!_.isEmpty(options.mongodbArgs)) {
if (options.mongodbArgs.explain === true) {
options.build = false;
}
aggregation.setOptions(options.mongodbArgs);
}
if (filter.aggregate) {
aggregation.append(filter.aggregate);
}
if (filter.fields) {
aggregation.project(filter.fields);
}
if (filter.order) {
aggregation.sort(connector.buildSort(modelName, filter.order));
}
if (filter.skip || filter.offset) {
aggregation.skip(filter.skip);
}
if (filter.limit) {
aggregation.limit(filter.limit);
}
if (filter.postAggregate) {
aggregation.append(filter.postAggregate);
}
} else if (_.isArray(filter)) {
aggregation.append(filter);
} else {
return next(new Error('Filter must be plain object or array'));
}
debug('Exec pipeline', JSON.stringify(aggregation.pipeline));
const cursor = aggregation.exec(connector.collection(modelName));
cursor.toArray((err, data) => {
if (err) return next(err);
const build = _.get(options, 'build', true);
const docs = data.map(rewriteId);
if (build === true) {
aggregateCallback(docs, filter, options.buildOptions, next);
} else {
next(null, docs, (objects, buildCallback) => {
aggregateCallback(objects, filter, options.buildOptions, buildCallback);
});
try {
if (_.isPlainObject(filter)) {
if (filter.near) {
aggregation.near(filter.near);
}
if (filter.where) {
const relationalFields = this.whichFieldsAreRelational(filter.where);
const directFields = _.difference(_.keys(filter.where), relationalFields);
if (directFields.length) {
const directWhere = _.pick(filter.where, directFields);
const where = connector.buildWhere(modelName, directWhere);
aggregation.match(where);
}
if (relationalFields.length) {
const relationalWhere = _.pick(filter.where, relationalFields);
buildLookup(aggregation, relationalWhere);
aggregation.coalesce(relationalWhere);
aggregation.match(relationalWhere);
}
}
if (filter.aggregate) {
aggregation.append(filter.aggregate);
}
if (filter.fields) {
aggregation.project(filter.fields);
}
if (filter.order) {
aggregation.sort(connector.buildSort(modelName, filter.order));
}
if (filter.skip || filter.offset) {
aggregation.skip(filter.skip);
}
if (filter.limit) {
aggregation.limit(filter.limit);
}
if (filter.postAggregate) {
aggregation.append(filter.postAggregate);
}
} else if (_.isArray(filter)) {
aggregation.append(filter);
} else {
return done(new Error('Filter must be plain object or array'));
}
} catch (err) {
return done(err);
}
});
debug('Exec pipeline', JSON.stringify(aggregation.pipeline));
const cursor = aggregation.exec(connector.collection(modelName));
cursor.toArray((err, data) => {
if (err) return done(err);
const build = _.get(options, 'build', true);
const docs = data.map(rewriteId);
if (build === true) {
aggregateCallback(docs, filter, options.buildOptions, done);
} else {
if (options.buildLater !== true) {
done(null, docs);
} else {
done(null, [docs, (objects, buildCallback) => {
aggregateCallback(objects, filter, options.buildOptions, buildCallback);
}]);
}
}
});
}, next);
};

/**
Expand Down
1 change: 1 addition & 0 deletions lib/aggregation.js
View file
Expand Up @@ -16,6 +16,7 @@ module.exports = class Aggregation {
* Append pipeline stages.
* @param {Object|Object[]} stages One or several stages.
* @returns {module.Aggregation}
* @throws Error
*/
append (stages) {
if (!_.isArray(stages)) {
Expand Down

0 comments on commit cc49a04

Please sign in to comment.