A plugin-based worker system implimented in Golang. APIs subject to change.
- Make sure the GOPATH environment variable is set.
- Run
go get github.com/barracudanetworks/GoWorker/...
From the project root, run go test ./... to test all packages. To test a single package, run go test from within the package's directory.
To run benchmarks for a package, run go test --bench=.
Read more about Golang testing
You may either run the program with go run goworker.go or build a binary with go build.
GoWorker is comprised of three main components:
- Provider: Provides jobs to the manager from an external source (e.g. Redis)
- Worker: Executes a job and returns results and stats to the manager.
- Manager: Manages the work pipline. Requests jobs from providers and routes them to the workers.
Provider -> Manager -> Worker -> Manager
There are three main stages in the worker pipeline.
- The Manager requests a job from a provider.
- The Manager hands the job off to a worker.
- The Worker returns the job back to the Manager after execution, with additional information including the result of the job. Jobs may optionally be set to retry in the case of failure until a maximum number of retries has been exhausted, or until the job succeeds. If all retries have been exhausted, the job is passed to the Manager's failure handlers, if any have been configured.
Workers and Providers are implemented as plugins. Because Go can compile down to a static binary, you must recompile GoWorker when you add new plugins.
To load your plugin into the binary, add a blank import to your package in the main package (gowork.go by default), located inside the cmd directory. Your package must have an init() function that calls either provider.LoadProvider(YourProviderFactory) or worker.LoadWorker(YourWorkerFactory).
Once your worker or provider has been loaded, it will be accessible in the config file by its name, in all lowercase.
You may learn more by reading the GoDoc entries for workers and providers.