Skip to content

feat: introduce go binding#132

Merged
JingsongLi merged 6 commits intoapache:mainfrom
luoyuxia:introduce-go-binding
Mar 16, 2026
Merged

feat: introduce go binding#132
JingsongLi merged 6 commits intoapache:mainfrom
luoyuxia:introduce-go-binding

Conversation

@luoyuxia
Copy link
Contributor

@luoyuxia luoyuxia commented Mar 14, 2026

Purpose

Linked issue: close #121

Brief change log

Introduce a Go binding for Apache Paimon Rust. Uses purego + libffi to call the paimon-c shared library without CGO, and the Arrow C Data Interface for zero-copy data exchange.

Brief change log

  • Add bindings/go/ package: catalog, table, scan, read, plan, identifier, error handling, FFI framework, embedded library loading (zstd-compressed, platform-specific)
  • TableRead.ToArrow(plan) returns a lazy RecordBatchReader with Next()/Close(), streaming Arrow batches one at a time instead of loading all into memory
  • Add integration test and Makefile
  • Add Go integration test step to CI workflow

Tests

IT

API and Format

TBD

Documentation

Will add in next following pr since the api may change during review

}
defer p.Close()

catalog, err := p.NewFileSystemCatalog(warehouse)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will introduce some thing like catalog factory to create filesystem catalog instead of hard coding to NewFileSystemCatalog.
Now, let's just make go works.

// The table was populated by Docker provisioning with:
//
// (1, 'alice'), (2, 'bob'), (3, 'carol')
func TestReadLogTable(t *testing.T) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the current api design follow python & java client. Myabe in the future we can consider have a simpler(but may not flexiable) api like:

tbl, err := paimon.OpenTable(path, paimon.NewIdentifier("default", "t1"))
  if err != nil { return err }
  defer tbl.Close()

  reader, err := tbl.NewRecordReader(ctx)
  if err != nil { return err }
  defer reader.Close()

  for {
      rec, err := reader.NextRecord()
      ...
  }

@luoyuxia
Copy link
Contributor Author

I'll add build & reelase pipline in the next follow pr, and it only support mac/linux, if needed, I can support window in next pr.

@luoyuxia luoyuxia force-pushed the introduce-go-binding branch from f6bd026 to 46a0607 Compare March 15, 2026 04:26
@XiaoHongbo-Hope
Copy link

+1

@JingsongLi
Copy link
Contributor

+1

@JingsongLi JingsongLi merged commit 598a2c3 into apache:main Mar 16, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

add go binding

3 participants