Cmdstalk is a unix-process-based beanstalkd queue broker.
Each job is passed as stdin to a new instance of the configured worker command.
exit(0) the job is deleted. On
exit(1) (or any non-zero status) the job
is released with an exponential-backoff delay (releases^4), up to 10 times.
If the worker has not finished by the time the job TTR is reached, the worker
is killed (SIGTERM, SIGKILL) and the job is allowed to time out. When the
job is subsequently reserved, the
timeouts: 1 will cause it to be buried.
In this way, job workers can be arbitrary commands, and queue semantics are reduced down to basic unix concepts of exit status and signals.
# Make sure you have a sane $GOPATH go get github.com/99designs/cmdstalk
cmdstalk -help # Usage of ./cmdstalk: # -address="127.0.0.1:11300": beanstalkd TCP address. # -all=false: Listen to all tubes, instead of -tubes=... # -cmd="": Command to run in worker. # -per-tube=1: Number of workers per tube. # -tubes=[default]: Comma separated list of tubes. # Watch three specific tubes. cmdstalk -cmd="/path/to/your/worker --your=flags --here" -tubes="one,two,three" # Watch all current and future tubes, four workers per tube. cmdstalk -all -cmd="cat" -per-tube=4
gpm is used to track dependencies. Make sure
you have a sane
$GOPATH etc, then
brew install gpm and
# Run all tests, with minimal/buffered output. go test ./... # Run tests in the broker package with steaming output. (cd broker && go test -v) # Run cmdstalk from source. go run cmdstalk.go -cmd='hexdump -C' -tubes="default,another" # Build and run a binary. go build file cmdstalk # cmdstalk: Mach-O 64-bit executable x86_64
# Set up cross-compiling tool. go get github.com/mitchellh/gox gox -build-toolchain -os="darwin linux" -arch="amd64" # Compile for various systems. gox -os="darwin linux" -arch="amd64" gzip cmdstalk_*_* # Create a release. open https://github.com/99designs/cmdstalk/releases/new
- Graceful shutdown.
- SIGKILL recalcitrant worker processes.
- Logging improvements; stdout/stderr, concurrency-safety.
- Interactive mode; single-concurrency, prompt for action for each job.
© Copyright 2014 99designs Inc.