Skip to content
Very simple batching API for Go.
Go
Branch: master
Clone or download

Latest commit

Latest commit d6a9a4d Feb 13, 2020

Files

Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.gitignore Initial commit Nov 28, 2019
LICENSE Update LICENSE Feb 13, 2020
README.md Update README.md Feb 13, 2020
batch.go initial implementation Feb 13, 2020
batch_test.go update given new home Feb 13, 2020
go.mod update given new home Feb 13, 2020
go.sum initial implementation Feb 13, 2020

README.md

batch

Very simple batching API for Go.

This repo came out of a tutorial blog post.

Install

You might as well just copy the batch.go file into your own project (and the batch_test.go while you're at it for future generations) rather than adding a dependency.

But it is maintained as a Go module which you can get with:

go get github.com/pacedotdev/batch

Usage

Import it:

import (
	"github.com/pacedotdev/batch"
)

If we wanted to perform an operation in batches of ten, regardless of how many items were in the slice, we could use the batch.All function by passing in the total number of items len(items), and the batchSize. The function is a callback that gets called for each batch, with start and end indexes which you can use to re-slice the items:

items, err := getAllItemsFromRequest(r)
if err != nil {
	return errors.Wrap(err, "getAllItemsFromRequest")
}
batchSize := 10
err := batch.All(len(items), batchSize, func(start, end int) error {
	batchItems := items[start:end]
	if err := performSomeRemoteThing(ctx, batchItems); err != nil {
		return errors.Wrap(err, "performSomeRemoteThing")
	}
})
if err != nil {
	return err
}

In this example, if we got 105 items, the performSomeRemoteThing function would get called eleven times, each time with a different page of 10 items (the batchSize) except the last time, when it would be a slice of the remaining five items.

The mechanics are fairly simple, but the code is encapsulated and well tested.

Using context for cancellation

You can use a context.Context for cancellation by adding a check into the body of the BatchFunc. The following code will cancel batching if the user aborts the HTTP request.

ctx := r.Context()
items, err := getAllItemsFromRequest(r)
if err != nil {
	return errors.Wrap(err, "getAllItemsFromRequest")
}
batchSize := 10
err := batch.All(len(items), batchSize, func(start, end int) error {
	if err := ctx.Err(); err != nil {
		return err
	}
	batchItems := items[start:end]
	if err := performSomeRemoteThing(ctx, batchItems); err != nil {
		return errors.Wrap(err, "performSomeRemoteThing")
	}
})
if err != nil {
	return err
}
You can’t perform that action at this time.