Skip to content

Commit

Permalink
working on asynchronous tasks
Browse files Browse the repository at this point in the history
  • Loading branch information
matthiaskoenig committed Oct 20, 2016
1 parent 506ef23 commit 931077e
Show file tree
Hide file tree
Showing 3 changed files with 41 additions and 8 deletions.
7 changes: 0 additions & 7 deletions README.md
Expand Up @@ -33,13 +33,6 @@ given.
* django
* python web framework & python backend code

### Asynchronous task queue
Celery is a powerful, production-ready asynchronous job queue, which allows
you to run time-consuming Python functions in the background. A Celery powered
application can respond to user requests quickly, while long-running tasks
are passed onto the queue.

http://michal.karzynski.pl/blog/2014/05/18/setting-up-an-asynchronous-task-queue-for-django-using-celery-redis/


### Datastorage layer
Expand Down
39 changes: 39 additions & 0 deletions docs/AsynchronousTask.md
@@ -0,0 +1,39 @@
### Asynchronous task queue


Required to run the tellurium integrations in a process outside of the request.

The absolute simplest way to do this (on a Unix-like operating system, at least) is to fork():

if os.fork() == 0:
do_long_thing()
sys.exit(0)
… continue with request …

This has some downsides, though (ex, if the server crashes, the “long thing” will be lost)… Which is where, ex, Celery can come in handy. It will keep track of the jobs that need to be done, the results of jobs (success/failure/whatever) and make it easy to run the jobs on other machines.

Using Celery with a Redis backend (see Kombu's Redis transport) is very simple, so I would recommend looking there first.


Celery is a powerful, production-ready asynchronous job queue, which allows
you to run time-consuming Python functions in the background. A Celery powered
application can respond to user requests quickly, while long-running tasks
are passed onto the queue.

http://michal.karzynski.pl/blog/2014/05/18/setting-up-an-asynchronous-task-queue-for-django-using-celery-redis/

## Celery
Celery is a task queue with batteries included.
Celery uses “brokers” to pass messages between a Django Project and the Celery workers.

http://docs.celeryproject.org/en/latest/getting-started/first-steps-with-celery.html#first-steps
http://docs.celeryproject.org/en/latest/django/first-steps-with-django.html

pip install celery

Message broker
* RabbitMQ
sudo apt-get install rabbitmq-server
* Redis

* Database
3 changes: 2 additions & 1 deletion requirements.txt
@@ -1,3 +1,4 @@
# requirements for web app
django==1.10.2
django-debug-toolbar==1.6
django-debug-toolbar==1.6
celery==3.1.18

0 comments on commit 931077e

Please sign in to comment.