Skip to content

Commit

Permalink
Describe the Router.
Browse files Browse the repository at this point in the history
  • Loading branch information
thedrow committed Jun 30, 2020
1 parent 0c27e14 commit cdbeed6
Show file tree
Hide file tree
Showing 2 changed files with 32 additions and 2 deletions.
20 changes: 20 additions & 0 deletions draft/high-level-architecture.rst
Original file line number Diff line number Diff line change
Expand Up @@ -273,6 +273,26 @@ The Publisher will later retry publishing the message.
Router
------

In Celery 4.x there was a master process which consumes :term:`Messages <Message>`
and workers which process them.

In Celery we now decouple the execution of the tasks completely from the routing of :term:`Messages <Message>`
to the `Execution Platform`_.

The Router is a combination of the following architectural patterns:
- :term:`Event Driven Consumer`
- :term:`Message Dispatcher`
- :term:`Service Activator`
- :term:`Process Manager`.

The Router consumes :term:`Messages <Message>` from a :term:`Message Broker` cluster or multiple clusters,
dispatches the consumed :term:`Messages <Message>` to the `Execution Platform`_ for processing and awaits
for the results of the task and stores them in the appropriate `Data Sink(s) <Data Sinks>`_.

When the execution of a task requires coordination in case of a workflow, an :term:`Event Message` or an incoming
:term:`Document Message` from a `Data Source <Data Sources>`_ the Router is responsible for the order of the execution
of the task(s).

Execution Platform
------------------

Expand Down
14 changes: 12 additions & 2 deletions glossary.rst
Original file line number Diff line number Diff line change
Expand Up @@ -41,11 +41,19 @@ Glossary
but will produce the same side effects when facing duplicated messages.

Message Dispatcher

`Enterprise Integration Patterns`_ defines a `Message Dispatcher`_ as a
component that will consume messages from a channel and distribute them to
performers.

Process Manager
`Enterprise Integration Patterns`_ defines a `Process Manager`_ as a component
that maintains the state of the sequence and determines the next processing step
based on intermediate results.

Event Driven Consumer
`Enterprise Integration Patterns`_ defines an `Event Driven Consumer`_ as a component
that consumes a message as soon as it is delivered.

Circuit Breaker
Martin Fowler defines a `Circuit Breaker`_ in the following fashion:

Expand Down Expand Up @@ -297,4 +305,6 @@ CC0 1.0 Universal license (https://creativecommons.org/publicdomain/zero/1.0/dee
.. _Inter-process Communication: https://en.wikipedia.org/wiki/Inter-process_communication
.. _Domain Event: https://martinfowler.com/eaaDev/DomainEvent.html
.. _Domain Model: https://martinfowler.com/eaaCatalog/domainModel.html
.. _GIL: https://realpython.com/python-gil/
.. _GIL: https://realpython.com/python-gil/
.. _Process Manager: https://www.enterpriseintegrationpatterns.com/patterns/messaging/ProcessManager.html
.. _Event Driven Consumer: https://www.enterpriseintegrationpatterns.com/patterns/messaging/EventDrivenConsumer.html

0 comments on commit cdbeed6

Please sign in to comment.