How to process transformations on a container

[matthieu-fesselier]

For the needs of startinblox, we need to process some transformations on a container to sort, filter, group... resources.

To do this, we recreate an array of resources from the AsyncIterator returned by query-ldflex to perform the transformations.
We have two main concerns with this approach:

• it's complicated to use native javascript functions on the arrays like sort, filter because of the asynchronous proxies
• it's probably not a good practice to recreate an object from an already existing one.

Is there a smarter way to handle this?

[RubenVerborgh]

You will probably like https://www.npmjs.com/package/iter-tools 🙂

[matthieu-fesselier]

Thanks, it sounds interesting! I'll take a look ;)

[RubenVerborgh]

Feel free to reopen if you don't find what you need!

[matthieu-fesselier]

I added iter-tools to startinblox and it fits almost all my needs. The only problem I face is that I can't sort a list of resource, because of the asynchronicity of the proxies (see iter-tools/iter-tools#286)
Should the sort feature be added to query-ldflex as a parameter? I guess it make sense to sort resources before creating an iterator?

[rubensworks]

Sorting may indeed be valuable, as this could be pushed down to the SPARQL level by LDflex.

[RubenVerborgh]

Good idea; tracking sorting in LDflex/LDflex#41.

Note however that, in several case, sorting makes operations blocking, i.e., you receive all of them at the end instead of incrementally. So this is why we, by default, prefer to leave it to the consumer to perform sorting. For instance, if you are feeding a UI, you could take unsorted results and then insert them in the right place as they arrive. But that all depends on the use case.

As iter-tools also says:

For example: sorting, shuffling and in general all operations that rely on having the full array at your disposal. In that case the way to go is to convert the iterable in an array and use that.

So you could do https://www.npmjs.com/package/iter-tools#to-array and take it from there for now?

Or, for your specific case, map to sorting keys (asynchronously) and then sort?

[matthieu-fesselier]

Yes the solution would be to re-create an array with the sorting key and then sort the resources. This was our last choice but I think this is the only one possible for now

[RubenVerborgh]

Sorting landed via LDflex/LDflex#43.