Home

Dashboard

About

stream-sorting is a streaming sort plus a suite of operations on sorted object-mode Readable streams. The headline operation is a disk-backed external merge sort designed to sort a billion records on a laptop without OOMing. On top of the sort, the package builds key-based join, set operations (union / intersection / difference), and k-way sorted merge.

Scope of the package: anything that requires or preserves sortedness. Combinators that know nothing about sortedness live in stream-join; 1→N split / route / filter lives in stream-fork. Built on stream-chain and stream-join — its only runtime dependencies. Node-only (server / CLI); browsers are out of scope. ESM only. Distributed under the New BSD license.

Status

Early development. The object-stream wrapper protocol and both sort algorithms (sort, polyphaseSort) are implemented. The sorted-stream operations below are still planned.

Documentation

Sorting

• sort — disk-backed external (k-way) merge sort. The headline operation.
• polyphaseSort — polyphase merge sort; fixed file budget, for bounded or heterogeneous storage.

Storage

• ObjectStreamWrapper — the storage abstraction the algorithms read and write through (also documents ItemWriter and consume).
• MemoryWrapper — in-memory wrapper for tests, small data, and benchmarks.
• LocalFileWrapper — local-file wrapper (line-delimited; JSON by default).

Sorted-stream operations (planned)

• mergeJoin / joinBy — SQL-style key-based join of two sorted streams (inner / left / right / full).
• union — sorted merge with cross-stream deduplication.
• intersection — values present in all input streams.
• difference — values in one stream not present in the others.
• mergeSorted — k-way sorted merge without deduplication; the suite's foundational op.

Other

• Release notes — detailed change history.

Installation

npm i stream-sorting