A columnar dataframe engine for Go.
kuma is what you reach for when you have a few hundred million rows, you would rather write Go than Python, and you do not want to find out about a typo in a column name ten seconds into a job.
Status: early. Nothing here is usable yet. The design is finished and written down in docs, the work is broken into milestones in the issue tracker, and M0 is in progress. Star the repo if you want to know when it does something.
t := TradeCols
bars, err := kuma.ScanParquet[Trade]("trades/*.parquet").
Filter(t.Price.Gt(100).And(t.Side.Eq("BUY"))).
GroupBy(t.Symbol, t.TS.Trunc(time.Minute)).
Agg[Bar](
t.Price.Mul(t.Qty).Sum().To(BarCols.Volume),
t.Price.Quantile(0.99).To(BarCols.P99),
kuma.Count().To(BarCols.N),
).
SortDesc(BarCols.Volume).
Head(20).
Rows(ctx)bars is a []Bar. Not a bag of any, not a map, not something you have to type assert your way out of.
Everything in that query is checked when you compile it:
t.Prive.Gt(100) // does not compile, no field called Prive
t.Side.Gt(100) // does not compile, StrCol has no Gt(float64)
t.Price.Contains("x") // does not compile, F64Col has no Contains
t.Price.Sum().To(BarCols.Symbol) // does not compile, float64 into a string columnTradeCols is fifteen lines generated by kumagen from the struct. The struct itself is generated too when the data came first: kumagen -from trades.parquet -type Trade reads the schema of a Parquet file, an Arrow file or a sample of a CSV, TSV or NDJSON one and writes the Go struct for it. You can also write both by hand. Details are in docs/03-typed-api.md.
Every dataframe library identifies columns by string. df["price"], pl.col("price"). It is the obvious design and everyone copies it, and it is also the single biggest source of avoidable pain in day to day data work. A typo is a runtime error. A rename means grepping for a string literal and hoping. There is no autocomplete, no jump to definition, no find all references.
Python has an excuse, because there is no compile step to check anything at. Go does not have that excuse. A stringly typed dataframe API in Go is a slower pandas with worse ergonomics, and there would be no reason for anyone to use it.
So typed columns are the default here and strings are the escape hatch, rather than the other way around.
The other reasons to want this: a single static binary with no Python runtime to install, context.Context cancellation so a query dies when its HTTP request does, and an optimizer you can inspect with Explain instead of guessing.
The full design is eleven documents in docs. The short version:
Arrow memory layout, our own kernels. The layout buys zero copy interop with pyarrow, Polars and DuckDB. The kernels are the thing this project exists to provide.
StringView by default. A 16 byte view with an inline prefix, so most string comparisons never touch the heap.
Lazy by default, eager when you want it. Same verbs on both, with projection pushdown, predicate pushdown, join reordering and partition pruning in between. Explain and Profile are shipping features, not debug tooling.
Parallel execution. Morsel driven, work stealing, partitioned hash aggregation, sized to GOMAXPROCS.
SIMD, as an optimization and never a dependency. The library builds, tests and works correctly on stock Go with no experiment flag. Every vectorized kernel has a scalar twin, and the twin is the specification.
Full pandas coverage. docs/06-pandas-parity.md is a checklist of the entire pandas API with a Go idiomatic equivalent for each entry. There are five genuine omissions and each one says what to use instead.
Bindings. A C ABI, a Python package and a TypeScript package for Node, Bun and Deno. Data crosses the boundary as Arrow and queries cross as a serialized plan, so a query over a hundred million rows makes about four foreign calls.
kuma/ipc reads and writes the Arrow IPC file and stream formats and implements the Arrow C Data Interface in both directions, which is how a column reaches pyarrow, Polars or DuckDB without being copied. It is part of this module and it needs nothing outside the standard library.
If you are already using apache/arrow-go, kuma/arrowgo converts between its types and kuma's directly. It is a module of its own, so go get github.com/tamnd/kuma/arrowgo brings arrow-go in and go get github.com/tamnd/kuma does not.
table, err := arrowgo.ImportRecordBatch(rec) // shares the buffers, copies nothing
rec, err := arrowgo.ExportRecordBatch(table) // and back againGo 1.27 or newer. This is not negotiable and it is not conservatism about new releases. The typed API is built on generic methods, which arrived in 1.27, and it cannot be expressed at all in 1.26.
GOEXPERIMENT=simd enables the vectorized kernels. Without it you get the scalar path, which is correct and tested and simply slower. Both are supported configurations and both run in CI.
Nothing else. The module depends on the standard library and nothing more, and the one package that needs a dependency is the arrow-go bridge above, which is a module of its own so that it stays out of your build unless you ask for it.
| 00-README | index, the decisions already made, prerequisites |
| 01-research-2026 | the state of Go, pandas, Polars and Arrow, and what each fact forces |
| 02-architecture | layers, memory, types, plan, optimizer, execution |
| 03-typed-api | typed columns and kumagen, read this one first |
| 04-high-level-api | what using it feels like, errors, Explain, display |
| 05-simd-kernels | the kernel layer and what to vectorize in what order |
| 06-pandas-parity | the conformance checklist |
| 07-bindings | C, Python, TypeScript |
| 08-milestones | M0 through M10 and the bindings track |
| 09-quality-bar | what standard library quality means here |
| 10-benchmarks | how the numbers get made |
| 11-package-layout | the tree and the stability tiers |
They live in tamnd/kuma-bench, against pandas and Polars, on db-benchmark and TPC-H, running from the first milestone rather than assembled at the end to support an announcement.
Two rules there. Every timed query forces materialization, because benchmarking a lazy engine without materializing the result measures nothing. And we publish the ones we lose, because a suite that only reports wins is an advertisement and readers correctly discount everything in it.
Read CONTRIBUTING.md. The short version is that the quality bar is docs/09-quality-bar.md, the work is in the issue tracker grouped by milestone, and anything labelled good first issue is genuinely one.
The parts that parallelize well across contributors are the pandas surface area in M7 and M8. The engine work in M3 and M4 does not parallelize and is best left to fewer hands.
Bear, in Japanese. It follows the pandas and Polars lineage without being a pun about it.
There is a name collision worth knowing about. kumahq/kuma is the Kong service mesh and it is well known in its own corner of the world. It is not a dataframe library and nobody is going to confuse the two in context.
Apache 2.0. See LICENSE.