Iterator Chunking

A TC39 proposal to add a method to iterators for producing an iterator of its subsequences.

Stage: 0

See the January 2024 presentation to committee.

motivation

It can be useful to consume a stream by more than one value at a time. For example, certain algorithms require looking at adjacent elements.

A common solution for this is a chunks method that works like the following:

let digits = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9].values();

let chunksOf2 = Array.from(digits.chunks(2));
// [ [0, 1], [2, 3], [4, 5], [6, 7], [8, 9] ]

let chunksOf3 = Array.from(digits.chunks(3));
// [ [0, 1, 2], [3, 4, 5], [6, 7, 8], [9] ]

let chunksOf4 = Array.from(digits.chunks(4));
// [ [0, 1, 2, 3], [4, 5, 6, 7], [8, 9] ]

A more flexible solution is a sliding window method, usually named windows:

let windowsOf3 = Array.from(digits.windows(3));
// [ [0, 1, 2], [1, 2, 3], [2, 3, 4], [3, 4, 5], [4, 5, 6], [5, 6, 7], [6, 7, 8], [7, 8, 9] ]

let windowsOf2AdvancingBy3 = Array.from(digits.windows(2, 3));
// [ [0, 1], [3, 4], [6, 7], [9] ]

chunks is just a specialisation of windows where chunks(n) is equivalent to windows(n, n).

let chunksOf4 = Array.from(digits.windows(4, 4));
// [ [0, 1, 2, 3], [4, 5, 6, 7], [8, 9] ]

design space

sliding windows sliding in/out past ends instead of sliding within boundaries?
default chunk size to 2?
chunking by 0 length?
windows advancing by 0?
should truncated chunks be returned?
- filler/padding elements?
should windows ever be truncated?
- should preserve equivalence with chunks, so answer should match chunks
- only when full iterator doesn't fill a single window?
- just lock step to 1 for windowing?

prior art

other languages

language	library	chunks	windows	chunks of 0?	truncates windows?
Clojure	core	`partition`	`partition`	infinite empty lists	when insufficient padding; terminates after 1
Elm	List.Extra	`groupsOf`	`groupsOfWithStep`	empty list	no
Haskell	split	`chunksOf`	`divvy`	infinite empty lists	yes
Java	Stream	--	--	N/A	N/A
Kotlin	Iterable	`chunked`	`windowed`	throws	configurable via parameter
.NET	System.Linq	`Enumerable.Chunk`	--	throws	N/A
PHP	array	`array_chunk`	--	throws	N/A
Python	itertools (3.12)	`batched`	--	??	N/A
Python	more-itertools	`grouper`	`windowed`	empty iterator	no, mandatory fill value
Ruby	Enumerable	`each_slice`	`each_cons`	throws	no, step not configurable
Rust	Iterator	`array_chunks`	`map_windows`	panics	no, step not configurable
Rust	slice	`chunks`	`windows`	panics	no, step not configurable
Scala	Seq	`grouped`	`sliding`	throws	yes
Swift	Sequence	--	--	N/A	N/A

JS libraries

library	chunks	windows	chunks of 0?	truncates windows?
`chunk`	`chunk`	--	coerces 0 to false 😞	N/A
`extra-iterable`	`chunk`	`chunk`	infinite empty arrays	yes
`iter-ops`	`page`	--	throws	N/A
`iter-tools`	`batch`	`window`, `windowAhead`, `windowBehind`	throws	optionally
`iterablefu`	`chunk`	--	collects everything into a single array	N/A
`itertools-ts`	`chunkwise`	`chunkwiseOverlap`	throws	yes
Lodash / Underscore	`chunk`	--	infinite empty arrays	N/A
Ramda	`splitEvery`	`aperture`	infinite empty arrays	no
sequency	`chunk`	--	throws	N/A
wu	`chunk`	--	collects everything into a single array	N/A

bakkot/proposal-iterator-chunking

ErrorLooks like something went wrong!

About

Resources

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Languages

Error
Looks like something went wrong!

Packages