@padmaj/batchify

Process large arrays in batches with concurrency control. Zero dependencies. TypeScript-first.

Perfect for rate-limited APIs, bulk database inserts, or any operation where you can't process everything at once.

Install

npm install @padmaj/batchify

Usage

Basic

import { batchify } from '@padmaj/batchify'

const results = await batchify(userIds, {
  batchSize: 10,
  handler: (batch) => fetchUsers(batch),
})

With concurrency

const results = await batchify(userIds, {
  batchSize: 10,
  concurrency: 3,  // run 3 batches in parallel
  handler: (batch) => fetchUsers(batch),
})

Log progress with onBatch

const results = await batchify(records, {
  batchSize: 50,
  concurrency: 2,
  handler: (batch) => insertToDb(batch),
  onBatch: (batch, index) => {
    console.log(`Processing batch ${index + 1}...`)
  },
})

API

batchify(items, options)

Option	Type	Default	Description
batchSize	number	10	Number of items per batch
concurrency	number	1	Max batches processed in parallel
handler	(batch: T[]) => Promise<R[]>	required	Function to process each batch
onBatch	(batch: T[], index: number) => void	—	Called before each batch is processed

Returns Promise<R[]> — all results flattened into a single array, in the same order as the input.

Notes

• Results are always returned in the same order as the input, even with concurrency > 1.
• If handler throws, the error propagates immediately and remaining batches are not processed.
• batchSize and concurrency must be at least 1, otherwise a RangeError is thrown.

License

MIT