ReactiveAggregate with angular-meteor

[jbx-alex]

hello,

Is it possible to use this library with angular-meteor?

I have this code:

validations.publisher.ts

Meteor.publish('validationsByCustomFind', function () {
    ReactiveAggregate(this, Validations, [
        { $match: { 'valStatus.valstaId':  6 }},
        { $addFields: {
            totalLeads: { '$size': '$valleaLead.leaId' }
        }},
    ]);
});

validations-list.component.ts

if (this._validationsSub)
    this._validationsSub.unsubscribe();
this._validationsSub = MeteorObservable.subscribe('validationsByCustomFind').subscribe(() => {
    this.validations = Validations.find();
});

I do not have any error in terminal or in client, but MiniMongo does not subscribe to any data in Validations.

I have tested the query in RoboMongo and it returns the data well.

Thanks in advance.
Regards

[robfallows]

@jbx-alex : Disclaimer: I've never used Angular. Having said that, ReactiveAggregate is client agnostic. As long as you can subscribe to a Meteor publication, it will work.

I notice you're using the same collection reference (Validations) for the client and for the aggregation source. If this collection reference is shared between client and server (for example by importing the same export), then you may have conflicting pub/sub between the actual collection and the aggregation. That will depend on if you are also publishing that collection (explicitly, or by having autopublish enabled).

I would suggest you check and eliminate that possibility by using a guaranteed client-only collection for the aggregation results (the README shows an example of doing that).

If you are still having problems, please provide a minimal reproduction that I can test. Thanks 🙂

[jbx-alex]

Hello @robfallows,

I have created a new collection for the client and so if it has worked for me.
I add my solution in case someone else uses angular-meteor:

validations.publisher.ts

Meteor.publish('validationsByCustomFind', function (find: object, numPage: number, perPage: number,
                                                    sortField: string, sortOrder: number) {
    ReactiveAggregate(this, Validations, [
        { $match: find},
        { $addFields: {
            valTotalLeads: { '$size': '$valleaLead.leaId' }
        }},
    ], { clientCollection: 'clientValidations' });
});

validations-list.component.ts

import { MeteorObservable, MongoObservable } from 'meteor-rxjs';

const clientValidations: MongoObservable.Collection<ClientValidation> = new MongoObservable.Collection<ClientValidation>('clientValidations');

...

private searchValidation(numPage: number): void {
    ...

    if (this._validationsSub)
        this._validationsSub.unsubscribe();
    this._validationsSub = MeteorObservable.subscribe('validationsByCustomFind', this.find,
        this.pagerObj.currentPage, this.pagerObj.pageSize, this.sortField, this.sortOrder).subscribe(() => {
            this.validations = clientValidations.find(this.find);

            const validations: Array<ClientValidation> = clientValidations.find(this.find).fetch();
                
            ...

        });
}

But now I have another problem, and that is when I try to paginate this listing:

Meteor.publish('validationsByCustomFind', function (find: object, numPage: number, perPage: number,
                                                    sortField: string, sortOrder: number) {
    const sort: any = {};
    if (sortField)
        sort[sortField] = sortOrder;

    ReactiveAggregate(this, Validations, [
        { $match: find},
        { $addFields: {
            valTotalLeads: { '$size': '$valleaLead.leaId' }
        }},
        { $sort: sort },
        { $limit: perPage },
        { $skip: perPage * (numPage - 1) }
    ], { clientCollection: 'clientValidations' });
});

The first page returns data, but if I click on any other it returns empty

Thanks in advance.
Regards

[jbx-alex]

Hello @robfallows,

Sorry for writing so many messages when the error is not from your library.

I have solved the pagination, it seems that it was the order of the parameters:

ReactiveAggregate(this, Validations, [
    { $match: find},
    { $addFields: {
        valTotalLeads: { '$size': '$valleaLead.leaId' }
    }},
    { $sort: sort },
    { $skip: perPage * (numPage - 1) },
    { $limit: perPage }
], { clientCollection: 'clientValidations' }, { allowDiskUse: true });

but now I get a console error when I delete an observable item from the list in the client, that is, the validations have a status and there are different lists according to their status, if the validation changes state I get an error when disappearing from that list.
This is the error message:

(node:5469) UnhandledPromiseRejectionWarning: Error: Removed nonexistent document 6732
    at SessionCollectionView.removed (packages/ddp-server/livedata_server.js:202:17)
    at Session.removed (packages/ddp-server/livedata_server.js:394:10)
    at Subscription.removed (packages/ddp-server/livedata_server.js:1292:19)
    at Object.keys.forEach.id (packages/tunguska:reactive-aggregate/aggregate.js:33:15)
    at Array.forEach (<anonymous>)
    at Promise.asyncApply (packages/tunguska:reactive-aggregate/aggregate.js:30:29)
    at /home/dev/.meteor/packages/promise/.0.11.2.19j3zmn.3ol4++os+web.browser+web.browser.legacy+web.cordova/npm/node_modules/meteor-promise/fiber_pool.js:43:40
(node:5469) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated
    either by throwing inside of an async function without a catch block, or by rejecting a promise which 
    was not handled with .catch(). (rejection id: 1)
(node:5469) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated.
    In the future, promise rejections that are not handled will terminate the Node.js process with
    a non-zero exit code.

Thanks in advance.
Regards

[robfallows]

@jbx-alex : Are you using string _ids or ObjectIds?

[jbx-alex]

I do not understand what you want to say to me with ObjectIds, _id we do not use it, we create our own id that they share in the same value, in the mongo collection if there is the _id of the table

validations collections:

_id: 5
valId: 5
valDescription: null
valInitialDate: 2018-02-01T00:00:00+0100
....

[robfallows]

Please provide a minimal reproduction in a GitHub repo I can clone and test.

[jbx-alex]

Hello @robfallows,

I have created a repository so you can test, but I could not replicate the error:
https://github.com/jbx-alex/reactive-aggregate-test

Our project does not use string _ids or ObjectIds

We add documents from our API, which is built in symfony, with int ids.

I do not know if you know how we can solve this, since we can not change our collections to use string _ids or ObjectIds.

Thank you so much.

[robfallows]

@jbx-alex : thanks for the repro. I'm struggling to build this, as it's eating all my CPU.

I think int _ids should be fine with reactive-aggregate, but Meteor only specifies using ObjectIds or strings. I will need to do some testing to confirm ints are ok.

However, if you cannot replicate the error with your repro, the issue may be elsewhere in your code.

I will close this issue for now. If you find you can reproduce the error, please feedback and I'll re-open it.

[jbx-alex]

@robfallows Perfect, I will try to insert in mongo int _ids to reproduce the error.

Thanks in advance.
Regards

[jbx-alex]

Hello @robfallows,

I have reproduced your code in my project to debug and I found the problem.

const update = async () => {
    if (initializing) return;
    // add and update documents on the client
    try {
        const docs = await collection.rawCollection().aggregate(pipeline, options.aggregationOptions).toArray();
        console.log(docs);
        docs.forEach(doc => {
            if (!sub._ids[doc._id]) {
                console.log(doc._id);
                console.log(typeof doc._id);
                sub.added(options.clientCollection, doc._id, doc);
            } else {
                sub.changed(options.clientCollection, doc._id, doc);
            }
                sub._ids[doc._id] = sub._iteration;
        });

        console.log(sub);

        // remove documents not in the result anymore
        Object.keys(sub._ids).forEach(id => {
            if (sub._ids[id] !== sub._iteration) {
                delete sub._ids[id];
                sub.removed(options.clientCollection, id);
            }
        });
        sub._iteration++;
    } catch (err) {
        throw err;
    }
 }

the conse.logs that I have added show this by console:

console.log(docs);

[ { _id: 6735,
    valDescription: null,
    ...
    valTotalLeads: 0 },
  { _id: 6734,
...
} ]

console.log(doc._id);
console.log(typeof doc._id);

6735
number
6732
number

console.log(sub);

 Subscription {
    _session:
    ...
    _documents:
        { clientValidations: { '~6735': true, '~6734': true, '~6732': true, '~6737': true } },
   ...
}

here sub.added(options.clientCollection, doc._id, doc);,in the doc._id wait for a string and it is passing an int and meteor converts it to '~6735', so this line fails sub.removed(options.clientCollection, id);

The only thing that has occurred to me is to force it to convert it to string in this way:
sub.added(options.clientCollection, String(doc._id), doc);
sub.changed(options.clientCollection, String(doc._id), doc);

I do not know if you have a better solution.
Thanks in advance.

[jbx-alex]

Please, can you tell me something about the issue

[robfallows]

@jbx-alex : Meteor adds the ~ prefix for the benefit of minimongo and pub/sub. As you may be aware, JavaScript objects with keys which look like integers will be sorted, which loses the expected ordering of object properties.

const x = { x:1, c:2, a:3 }
console.log(x) # {x: 1, c: 2, a: 3}

const y = { x:1, c:2, a:3, 11:11, "99": 99 }
console.log(y) # {11: 11, 99: 99, x: 1, c: 2, a: 3}

Adding a non-numeric character to anything looking like a number forces JavaScript to preserve the order of keys by preserving the String type. Even coercing a String type, as you've suggested, will not change this behaviour.

Short answer: you can't do what you want to do by using integers for _ids.

Longer answer: if you want to forego pub/sub and reactive aggregations, you can call a Meteor method to do a non-reactive aggregation and return the results as an array of documents. Those can have any _id type - or even no _id at all.

[jbx-alex]

hello @robfallows,

Thanks, finally I have changed the way in setting the _id from the API to be a ObjectId since there is not a good solution from Meteor.
Regards