A skeletal guide of one way to implement a nsq consumer
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
nsqworker
.gitignore
LICENSE
Makefile
README.md
setup.py

README.md

Overview

nsqworker is a skeletal code of one way to implement nsq consumer(worker) in Python base on pynsq.

Installation

pip install nsqworker

Demo

Go to nsqworker root folder and run:

python bootstrap.py

Publish a message to topic demo:

curl -d '{"clicks": 10}' 'http://127.0.0.1:4151/put?topic=demo'

then the demo worker will output:

demo: demo/pageview, clicks 10

If run bootstrap.py in command like below:

python bootstrap.py --topic=newtopic --channel=newchannel

It will try to run NewtopicNewchannelWorker from workers/newtopic/newtopic_newchannel_worker.py which subscribes messages of newtopic/newchannel, of course, you need implement this worker.

Conventions

  1. Keep all of nsq workers of a topic in a sub-folder named the same as the topic name, eg. nsqworker/workers/demo/
  2. Create a channel's worker in a file named {topic}/{topic}_{channel}_worker.py, eg. workers/demo/demo_pageview_worker.py
  3. Implement the channel worker class, it should inherit from nsqworker.Worker, eg. DemoPageviewWorker(Worker)
  4. Add a functional endswith "_handler"(eg. demo_handler) in the worker to handle messages, this functional will be treated as a message_handler in pynsq automatically, you can add more than one handlers.

It is easy to implement a customize consumer follow this conventions.

Customize your bootstrap.py

from nsqworker.bootstrap import run
# Customize workers_module which default value is nsqworker.workers(The root module of demo workers)
# You should set it to yourpackage.workers.
run("yourpackage.workers")

Then it will load and run workers under yourpackage.workers like nsqworker.bootstrap.py