Google App Engine (GAE) heartbeat
api and jobqueue
api for workers to pull.
I design this heartbeat
module to solve this problem:
- Server publish a
heartbeat
info, which containsname
,is_on
,come_back
. is_on
indicates that if the worker shall wake up and feth jobs or not.come_back
indicates the minutes interval worker shall come back to check the heartbeat.name
is the name of theheartbeat
.
How this will work in real life:
- I have a
heartbeat
whichname
is cat. - At 8:00 I adjust it to
is_on=True
,come_back=5
. So the worker come back every 5mins and do the job. - At 19:00 I adjust it to
is_on=False
,come_back=20
, so the worker is going idle and come back every 10 mins. - At 7:30 I adjust it again to
is_on=False
,come_back=5
, so the worker is idle but come back more frequently. - You can adjust the above paremeters through web page api or GAE cron jobs.
I design this jobqueue
module to solve this problem:
- Admin User adds jobs onto the server.
- Workers, if wake up, pull the job queue via http. Get a list of jobs.
- Workers get the job excecuted, this process may take up to 10mins, or even weeks.
- Workers report the success/failure of each job, via http. This is called "update job status".
main.py
the main entrance of whole GAE application./heartbeat
the source code ofheartbeat
database structure and http json web handlers./jobqueue
the source code ofjobqueue
database structure and http json web handlers./human
the source code of http web handlers, for admins to call to visualize the status./html
where the html, javascript, and css files are.
You can copy-paste the /heartbeat
, /jobqueue
folder into your project, they doesn't depend on other folders. :D