Make Minimongo cursor iterable

[mitar]

This pull requests makes Minimongo cursors iterable using the iteration protocol.

This means that somebody can do [...cursor] to get all values in the cursor (instead of cursor.fetch()). Or used inside for..of loop.

Not sure what do we want to do about compatibility. Maybe we can define it only if Symbol.iterator exists and is available. Or, if we are good with having it always, we could then change forEach to use it internally (instead of having code duplicated a bit). And then fetch could really just be [...cursor] and maybe that will be even faster then our current version.

BTW, this currently is not very optimized because it creates internally the whole array and then iterates over. It would be great to replace _getRawObjects with something which is returning object by object by itself. And then we would be creating arrays of all objects only when really requested (in fetch, and not in map or forEach).

[mitar]

cc @zimme, @radekmie

[zimme]

As ecmascript is available in this package and this is a completely new function maybe it's ok that we use new syntax here instead? const/let, arrow functions, etc.

[mitar]

I just wanted to keep it similar to the rest of the code. Especially because code is very similar to forEach. I do not really care though. :-)

[radekmie]

As I said in already mentioned issue, I'm planning to submit a PR with making minimongo more ES5/6.

[radekmie]

To be more compliant with the standard, done should be set to false here.

[mitar]

Not necessary:

Has the value false if the iterator was able to produce the next value in the sequence. This is equivalent of not specifying the done property altogether.

[radekmie]

OK, ECMA-262 6th Edition, Section 25.1.1.3:

done
This is the result status of an iterator next method call. If the end of the iterator was reached done is true. If the end was not reached done is false and a value is available. If a done property (either own or inherited) does not exist, it is consider to have the value false.

[radekmie]

As I said in already mentioned issue, I'm planning to submit a PR with making minimongo more ES5/6.

[zimme]

I would prefer the more optimized version, but I think this would be nice until we can make that happen.

[radekmie]

I'm already preparing an issue with a plan for modernization...

[mitar]

I would prefer the more optimized version, but I think this would be nice until we can make that happen.

Yes, I think it is useful to expose the API now. With old JS code. And then we decide on how to change internals to use ES6 and that can then be a separate PR changing everything.

[benjamn]

Maybe we can define it only if Symbol.iterator exists and is available.

I think this is a good idea!

[hwillson]

@mitar - given the 👍 on checking if Symbol.iterator exists, do you have any other questions or need anything else to help get this one wrapped up?

[mitar]

I made it conditional. But one thing to consider maybe instead is that we could just poly-fill Symbol, or just leave to Babel to do it for us?

Anyway, I think this is it then for this iteration of code. So I would then say this is ready. Probably we should add tests and documentation and history entry? Anything else?

[hwillson]

Actually, it looks like we already have Symbol poly-fill's in place, since minimongo is using ecmascript:

meteor/packages/ecmascript-runtime-client/runtime.js

Line 25 in 59a052e

Symbol = exports.Symbol = core.Symbol;

meteor/packages/ecmascript-runtime-server/runtime.js

Line 10 in 59a052e

Symbol = exports.Symbol = core.Symbol;

Maybe we don't need the conditional? I guess it doesn't hurt to have it though - thanks again!

[mitar]

OK, so just add tests and documentation then?

[hwillson]

Yes tests / documentation would be great @mitar - thanks!

[hwillson]

Hi @mitar - any luck with tests / docs? Thanks!

[daniel-mf]

What about using this approach:

LocalCollection.Cursor.prototype[Symbol.iterator] = function* () {
    yield* this.fetch();
};

[mitar]

Oh, this pull request became now very much conflicting with just merged #8893.

@radekmie, did you implement this as well? Do you want to do a take on this using the modernized approach you just did?

[radekmie]

@mitar no, I did not. You can update it - only few minor changes are needed. If you want, I can do it.

[mitar]

@daniel-mf, does that work? I am not familiar with this syntax and how it would work.

[mitar]

I am currently a bit overwhelmed with other work, so if you could do it, that would be great. I added you as a collaborator to my repo.

[radekmie]

@mitar: don't you mind if I use .shift() instead of such lazy iteration?

EDIT: Maybe in the future - I'll leave it for now.

[mitar]

shift? You think that would be performant? Modifying array?

But it is sad that we have to generate whole array just to then iterate over it. This should really be improved.

[daniel-mf]

@mitar I've tested, it works.

[radekmie]

@mitar: I thought about .shifting it to make it smaller in every step but I tested it and it's not a big difference on small (<1k) arrays. Yeah, I'd love to make it really iterable (that was the first purpose of my optimisations) but it is impossible (without quite a lot of additional code) to do it because of... Sorting. Sort makes it impossible (at least as long as Minimongo selectors works as filters on all docs).

[daniel-mf]

Since we can't make it really iterable, I guess there's no point in writing a fetch like function once we could just yield it from a generator function, like the example I gave above.

[hwillson]

Hi all - it looks like the code changes are done, so I believe we're just waiting on tests, doc updates and a History.md entry? @radekmie @mitar Are either one of you available to wrap this up?

[radekmie]

I won't be able to get to it before next week. @mitar?

[mitar]

@hwillson: I looked at this but I am not sure how to document this? It is not a method. So how does one document this with current documentation syntax?

[mitar]

I added tests and history entry.

[hwillson]

Thanks for these changes @mitar. Maybe adding a small mention of cursors now being iterable would make sense in the following docs locations:

1. In the Mongo.Collection#find() section, where we first mention that find returns a cursor. So in https://github.com/meteor/docs/blob/b3b9c81a4b40a48dcb24597e7fc4d39d5fecb7a9/source/api/collections.md, search for "find returns a cursor". Adding a quick mention that the returned cursor is iterable here would be great.

2. In the Cursors section of that same doc page. Maybe mention that cursors are now iterable in the first paragraph somewhere.

If you don't have time to get a https://github.com/meteor/docs PR ready for these changes let me know, and I'll make it happen. As far as things go with this PR however, I think we're all set. Thanks all - LGTM!

[mitar]

I am slightly under time pressure at the moment, so if you could update the docs, that would be great. Sorry for putting more on your plate. :-( Otherwise I will try, but not sure when.

(You do not think there is a place in autogenerated code to document iterator?)

[hwillson]

No problem at all, I'll take care of it. No, I didn't see a good place to mention it in the jsdoc's (but if anyone reading this can think of another good place, we'll add it there as well).