sort

sort

Disk-backed external (k-way) merge sort for object streams of any size. Items are pulled into in-memory batches, each batch is sorted and flushed to a temporary file (a run), and the runs are then k-way-merged. Sorts data far larger than memory; when the whole input fits in one batch it never touches disk.

import sort from 'stream-sorting/sort.js';

for await (const item of sort(input, {compare: (a, b) => a.id - b.id, tmpDir: '/var/sort'})) {
  // items in ascending id order
}

input is AsyncIterable<T> | Iterable<T> — a Node Readable, a Web ReadableStream, a generator, an array, etc. The result is an AsyncIterable<T>; convert at the boundary you need:

import readableFrom from 'stream-chain/utils/readableFrom.js';

readableFrom(sort(input, opts)).pipe(downstream); // Node Readable
ReadableStream.from(sort(input, opts)).pipeTo(webDestination); // Web ReadableStream

Options

Pass a comparator (compare or lessFn) and a storage target (tmpDir or createWrapper).

Option	Type	Default	Description
compare	(a, b) => number	—	Comparator, Array.prototype.sort semantics. Provide this or lessFn.
lessFn	(a, b) => boolean	—	Strict-less comparator. Provide this or compare.
tmpDir	string	—	Directory for the built-in LocalFileWrapper run files. Provide this or createWrapper. No default — Linux /tmp is commonly RAM-backed (tmpfs) and would defeat a disk-backed sort.
createWrapper	(runIndex) => ObjectStreamWrapper	—	Factory producing one wrapper per run, for custom backing storage. Provide this or tmpDir.
batchSize	number	10000	Soft cap on in-memory items per run.
stable	boolean	true	Keep input order for equal items. Set false if your comparator already breaks ties.
onProgress	(stats) => void	—	Progress callback (see below).
keepTempFiles	boolean	false	Keep run files instead of deleting them.

Progress

onProgress(stats) fires at run boundaries and at merge start:

interface SortProgressStats {
  phase: 'pre-sort' | 'final-merge';
  itemsRead: number;
  itemsWritten: number;
  runsCreated: number;
}

Algorithm

1. Pull items into a buffer of up to batchSize, sort with Array.prototype.sort, and flush as a sorted run to a fresh wrapper.
2. Repeat until the input is exhausted.
3. K-way-merge all runs (via stream-join's mergeSorted) and emit in order.

When the input fits in a single batch it is sorted in memory and emitted directly — no wrapper, no disk. Stability uses an internal monotonic sequence tag as the tiebreaker, stripped before emission.

Prefer sort on modern SSDs: file handles are cheap and a single-pass k-way merge is the optimal I/O strategy. When the number of open files must be bounded, or storage is spread across expensive/remote backends, use polyphaseSort.

See also: polyphaseSort, ObjectStreamWrapper.