std.range.chunks for InputRanges (buffered)

[wilzbach]

Hey all,

this is very vague proposal to allow chunks to be used on transient InputRanges that can't be saved (e.g. byLine). The current implementation has (at least) one major flaw, but I wanted to put this initial version up so we can start a discussion. Therefore I would like to know where you stand on such addition (if properly done)?

Moreover there a couple of things that I would like your feedback on:

• proper by reference buffering (maybe there is a better way to have such a buffering - AFAIK currently primitive values and structs are copied)
• conflicts with chunks from std.stdio (that's why it is named chunkss - for some reason the overloading doesn't work so nicely as for the normal chunks in Range)
• is it recommended to put the struct into a function or not (I have seen both styles in Phobos)
• I have added a template flag that users can use if they want to obtain a non-transient buffer - is such a flag common in phobos and do you also regard it as useful?

Thanks for your input,

---

Here's a copy of the post from the forum, to give you the motivation
http://forum.dlang.org/post/oidujhauyroygigrqddx@forum.dlang.org

tl;dr: It would be nice if chunks would accept only an InputRange too.

Let me show you a common pattern that I often use:

We want to iterate over two lines in a pair and compute the difference between them, e.g.

1 2
2 3

=> 2

stdin.byLine.map!(x => x.split(' ').map!(to!int)).chunks(2)).map(x =>
abs(x.front.sum - x.dropOne.front.sum);
);

This currently doesn't compile, because chunks doesn't buffer, it saves the state of the forward range. However adding support for normal Input Ranges can easily done using a buffer - it will of course require more memory as the .save implementation, however handling of ForwardRanges won't be affected.

I have seen this has been discussed more often [2,3]. The only argument against it was in [3] the missing manpower.
However I have already written such a simple buffering [4] in the past and would be happy to wrap this up nicely and (hopefully) bring it into a "phobos" shape - any thoughts/concerns?

[1] https://issues.dlang.org/show_bug.cgi?id=15759
[2] https://issues.dlang.org/show_bug.cgi?id=6621
[3] http://forum.dlang.org/post/omkpvigktgrgbzemhhuc@forum.dlang.org
[4] https://github.com/greenify/d-itertools/blob/4afbc804e8b50c797fa206969dc3b4934911a0b9/source/splitwise.d

[MetaLang]

Are you familiar with cache and cacheBidirectional? It looks like they could be improved to do what you want.

[wilzbach]

I will open a new PR once I have time to work on cache - thanks @MetaLang