Skip to content
šŸ‘¾ Gearman in Golang
Go Python Shell Makefile Smarty
Branch: master
Clone or download
Pull request Compare This branch is 19 commits ahead, 1 commit behind appscode:master.
toli-belo Sync/Wait for processing handler (#14)
Use a key-specific wait for delayed additions into the handler map.
Latest commit fb8c16d Aug 16, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
chart/g2 switch ssmccoy to quantcast. Feb 8, 2019
client Sync/Wait for processing handler (#14) Aug 16, 2019
cmd/gearmand switch ssmccoy to quantcast. Feb 8, 2019
example Fix client tests (#8) Apr 23, 2019
gearadmin switch ssmccoy to quantcast. Feb 8, 2019
hack switch ssmccoy to quantcast. Feb 8, 2019
pkg switch ssmccoy to quantcast. Feb 8, 2019
vendor g2: Fixed broken imports Jan 17, 2019
worker Enable worker to reconnect to gearman indefinitely (#7) Apr 17, 2019
.gitignore g2: converted to go.mod Jan 17, 2019
.travis.yml Add travis yaml (appscode#36) Mar 16, 2018
LICENSE Use Apache 2 License. Jan 26, 2017
README.md Add travis yaml (appscode#36) Mar 16, 2018
doc.go switch ssmccoy to quantcast. Feb 8, 2019
glide-slow
glide.lock Revendor (appscode#43) Aug 15, 2018
glide.yaml Fix path for github.com/appscode/log pkg (appscode#34) Nov 17, 2017
go.mod Sync/Wait for processing handler (#14) Aug 16, 2019

README.md

Go Report Card Build Status codecov Docker Pulls Slack Twitter

G2

G2 is a modern implementation of Gearman in Go Programming Language. G2 includes various improvements in retry and connection logic for using in Kubernetes. It comes with built-in Prometheus ready metrics. G2 also implements scheduled jobs via cron expressions.

The client package is used for sending jobs to the Gearman job server and getting responses from the server.

"github.com/appscode/g2/client"

The worker package will help developers in developing Gearman worker service easily.

"github.com/appscode/g2/worker"

The gearadmin package implements a client for the gearman admin protocol.

"github.com/appscode/g2/gearadmin"

GoDoc

Install

Install the client package:

$ go get github.com/appscode/g2/client

Install the worker package:

$ go get github.com/appscode/g2/worker

Both of them:

$ go get github.com/appscode/g2

G2 gearman server can be installed via pre-built Docker images mentioned in the Github releases. G2 gearman server can also be installed in Kubernetes via Helm using the chart included in this repository or from official charts repository. To install the chart with the release name my-release:

$ helm install chart/g2 --name my-release

To see the detailed configuration options, visit here.

Usage

Server

how to start gearmand?

./gearmand run --v=3 --addr="0.0.0.0:4730"

how to specify leveldb location?

./gearmand run --v=3 --storage-dir=/my-dir --addr="0.0.0.0:4730"

how to export metrics to Prometheus:

http://localhost:3000/metrics

how to list all workers ?

http://localhost:3000/workers

how to list workers by "cando" ?

http://localhost:3000/workers/<function>

how to list all jobs ?

http://localhost:3000/jobs

how to query job status ?

http://localhost:3000/jobs/<jobhandle>

how to change monitor address ?

./gearmand run --v=3 --web.addr=:4567

Worker

// Limit number of concurrent jobs execution.
// Use worker.Unlimited (0) if you want no limitation.
w := worker.New(worker.OneByOne)
w.ErrorHandler = func(e error) {
	log.Println(e)
}
w.AddServer("tcp4", "127.0.0.1:4730")
// Use worker.Unlimited (0) if you want no timeout
w.AddFunc("ToUpper", ToUpper, worker.Unlimited)
// This will give a timeout of 5 seconds
w.AddFunc("ToUpperTimeOut5", ToUpper, 5)

if err := w.Ready(); err != nil {
	log.Fatal(err)
	return
}
go w.Work()

Client

c, err := client.New("tcp4", "127.0.0.1:4730")
defer c.Close()
//error handling
c.ErrorHandler = func(e error) {
	log.Println(e)
}
echo := []byte("Hello\x00 world")
echomsg, err := c.Echo(echo)
log.Println(string(echomsg))
jobHandler := func(resp *client.Response) {
	log.Printf("%s", resp.Data)
}
handle, err := c.Do("ToUpper", echo, runtime.JobNormal, jobHandler)

Gearman Admin Client

Package gearadmin provides simple bindings to the gearman admin protocol: http://gearman.org/protocol/. Here's an example program that outputs the status of all worker queues in gearman:

c, err := net.Dial("tcp", "localhost:4730")
if err != nil {
	panic(err)
}
defer c.Close()
admin := gearadmin.NewGearmanAdmin(c)
status, _ := admin.Status()
fmt.Printf("%#v\n", status)

Build Instructions

# dev build
./hack/make.py

# Install/Update dependency (needs glide)
glide slow

# Build Docker image
./hack/docker/setup.sh

# Push Docker image (https://hub.docker.com/r/appscode/gearmand/)
./hack/docker/setup.sh push

# Deploy to Kubernetes (one time setup operation)
kubectl run gearmand --image=appscode/gearmand:<tag> --replica=1

# Deploy new image
kubectl set image deployment/gearmand tc=appscode/gearmand:<tag>

Acknowledgement

License

Apache 2.0. See LICENSE.

  • Copyright (C) 2016-2017 by AppsCode Inc.
  • Copyright (C) 2016 by Clever.com (portions of gearadmin client)
  • Copyright (c) 2014 ngaut (portions of gearmand)
  • Copyright (C) 2011 by Xing Xing (portions of client and worker)
You canā€™t perform that action at this time.