Skip to content
A serverless cluster computing system for the Go programming language
Go
Branch: master
Clone or download
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
archive/tarslice bigslice/archive/tarslice: package to read tar files in bigslice Oct 1, 2019
cmd bigslice: close machine readers used in result scanning Nov 15, 2019
docs docs: prefer -apple-system font when available (#19) Nov 18, 2019
exec exec: improve error reporting (#21) Nov 19, 2019
frame bigslice: support prefixes for cogrouping Oct 7, 2019
internal bigslice: reduce spill size and canary size Oct 1, 2019
sliceconfig sliceconfig: reimport net/http which somehow got lost (#11) Nov 10, 2019
slicefunc slicefunc: package to abstract user-provided funcs (#20) Nov 19, 2019
sliceio exec: improve error reporting (#21) Nov 19, 2019
slicetest bigslice: close machine readers used in result scanning Nov 15, 2019
slicetype bigslice/slicetype: include the type's prefix Sep 10, 2019
sortio slicefunc: package to abstract user-provided funcs (#20) Nov 19, 2019
stats bigslice/exec: remove misleading stats Sep 10, 2019
typecheck slicefunc: package to abstract user-provided funcs (#20) Nov 19, 2019
.gitignore Initial commit Sep 10, 2019
.travis.yml .travis.yml: turn GOPROXY, GOSUMDB on again Oct 14, 2019
CNAME Create CNAME Sep 23, 2019
LICENSE Initial commit Sep 10, 2019
README.md bigslice: add simple README, try to fix image on bigslice.io Oct 2, 2019
accum.go slicefunc: package to abstract user-provided funcs (#20) Nov 19, 2019
accum_test.go slicefunc: package to abstract user-provided funcs (#20) Nov 19, 2019
cache.go slicefunc: package to abstract user-provided funcs (#20) Nov 19, 2019
cache_test.go bigslice: close machine readers used in result scanning Nov 15, 2019
cogroup.go slicefunc: package to abstract user-provided funcs (#20) Nov 19, 2019
cogroup_test.go bigslice: support prefixes for cogrouping Oct 7, 2019
doc.go fix misspells and ineffectual assignments (#15) Nov 11, 2019
func.go bigslice: Remove useless type annotation in FuncLocationsDiff Sep 10, 2019
func_test.go bigslice: Verify common funcs across workers Sep 10, 2019
go.mod go.mod: update base, bigmachine dependencies Oct 23, 2019
go.sum go.sum: remove testutil Oct 23, 2019
name_test.go bigslice: add bigslice.Helper, to denote helper slice constructors Oct 1, 2019
reduce.go slicefunc: package to abstract user-provided funcs (#20) Nov 19, 2019
reduce_test.go bigslice: implement user-defined prefixes Sep 10, 2019
reshard.go slicefunc: package to abstract user-provided funcs (#20) Nov 19, 2019
reshard_test.go bigslice: rename bigslice.Reshard to bigslice.Reshuffle; add bigslice… Oct 30, 2019
reshuffle.go slicefunc: package to abstract user-provided funcs (#20) Nov 19, 2019
reshuffle_test.go bigslice: rename bigslice.Reshard to bigslice.Reshuffle; add bigslice… Oct 30, 2019
scan.go bigslice: add ScanReader op Oct 1, 2019
scan_test.go bigslice: add ScanReader op Oct 1, 2019
slice.go slicefunc: package to abstract user-provided funcs (#20) Nov 19, 2019
slice_test.go exec: improve error reporting (#21) Nov 19, 2019

README.md

Bigslice

Bigslice is a serverless cluster data processing system for Go. Bigslice exposes composable API that lets the user express data processing tasks in terms of a series of data transformations that invoke user code. The Bigslice runtime then transparently parallelizes and distributes the work, using the Bigmachine library to create an ad hoc cluster on a cloud provider.

Developing Bigslice

Bigslice uses Go modules to capture its dependencies; no tooling other than the base Go install is required.

$ git clone https://github.com/grailbio/bigslice
$ cd bigslice
$ GO111MODULE=on go test
You can’t perform that action at this time.