Skip to content

Commit

Permalink
[docs] update docs
Browse files Browse the repository at this point in the history
  • Loading branch information
junjuew committed Apr 21, 2020
1 parent 897d8da commit b3188c4
Showing 1 changed file with 68 additions and 20 deletions.
88 changes: 68 additions & 20 deletions docs/tutorial.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,20 +4,44 @@ Tutorial
**********************

We will create a wearable cognitive assistant that recognize a person or a chair
in this tutorial. The `complete code
<https://github.com/cmusatyalab/OpenWorkflow/blob/master/examples/gabriel_example.py>`_
contains a few more use cases of gabrieltool package. We will focus on creating
a gabriel server in this tutorial. We will create the application logic as a FSM
using gabrieltool python library below. You can also build the FSM from the web
GUI. In general, the web GUI is good for simple application logic while the
python library provides more flexibility and supports more complicated
application logic.

To recognize a person or a chair, we will use a SSD MobileNet v2 object detector
network from Tensorflow. Download and decompress the detector from
`here <https://storage.cmusatyalab.org/openworkflow/ssd_mobilenet_v2_saved_model.zip>`_.

Next, let's build a two-state FSM. The first state is *st_start*. We want to
in this tutorial. First, let's get the example code running before going into
its implementation.

1. `Install gabrieltool <https://openworkflow.readthedocs.io/en/latest/installation.html>`_.
2. Download `gabriel_example.py <https://github.com/cmusatyalab/OpenWorkflow/blob/master/examples/gabriel_example.py>`_
and the `object detector <https://storage.cmusatyalab.org/openworkflow/ssd_mobilenet_v2_saved_model.zip>`_
into the same directory. This object detector is the SSD MobileNet v2 DNN from `Tensorflow <https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md>`_.
In this example, we will use this object detector to detect people and chairs.
3. Unzip the downloaded object detector into the same directory.
4. Launch the gabriel server.

.. code-block:: console
$ ./gabriel_example run_gabriel_server
5. In the console, you should see log messages of building the FSM, starting
gabriel server, and launching a docker container.
6. You should also be able to see the container started using docker commands.
Note that it may take a few minutes to download the container image before the container is started.

.. code-block:: console
$ docker ps -a --filter="name=GABRIELTOOL"
7. Once you see the container is up, the server is ready for connection.
Download `Gabriel client <https://play.google.com/store/apps/details?id=edu.cmu.cs.gabrielclient>`_
from Android Play Store to connect to it and try it out.

Now you've gotten the code running, let see what is happening under the hood. We
will focus on explaining how to create a gabriel server in this tutorial while
the example code contains a few more use cases of gabrieltool package.

The application logic is created as a FSM using gabrieltool python library
below. You can also build the same FSM from the web GUI. In general, the web GUI
is good for simple application logic while the python library provides more
flexibility and supports more complicated application logic.

The FSM has two states. The first state is *st_start*. We want to
send a welcome message when a user first connects. Therefore, *st_start* doesn't
have any processing involved and will always transition immediately to the next
state and return a welcome message to the client.
Expand All @@ -35,6 +59,12 @@ an instruction of 'Found Person' to the Gabriel client.
.. code-block:: python
:linenos:
import cv2
import fire
from logzero import logger
from gabriel_server.local_engine import runner as gabriel_runner
from gabrieltool.statemachine import fsm, predicate_zoo, processor_zoo, runner
def _build_fsm():
"""Build an example FSM for detecting a person or a chair.
Expand Down Expand Up @@ -95,7 +125,29 @@ an instruction of 'Found Person' to the Gabriel client.
st_tf.transitions[1].next_state = st_tf
return st_start
Now, let's launch a gabriel server using this FSM.
The *st_tf* state uses a custom transition predicate defined by the following
function. To learn more about the how to use and create FSM components, see its
`API documentation <https://openworkflow.readthedocs.io/en/latest/source/gabrieltool.statemachine.html#module-gabrieltool.statemachine.fsm>`_.

.. code-block:: python
:linenos:
def _add_custom_transition_predicates():
"""Here is how you can add a custom transition predicate to the predicate zoo
See _build_fsm to see how this custom transition predicate is used
"""
from gabrieltool.statemachine import callable_zoo
class HasChairClass(callable_zoo.CallableBase):
def __call__(self, app_state):
# id 62 is chair
return '62' in app_state
predicate_zoo.HasChairClass = HasChairClass
The gabriel cognitive engine is created using a `FSM cognitive engine runner <https://openworkflow.readthedocs.io/en/latest/source/gabrieltool.statemachine.html#module-gabrieltool.statemachine.runner>`_.

.. code-block:: python
:linenos:
Expand All @@ -121,8 +173,4 @@ Now, let's launch a gabriel server using this FSM.
input_queue_maxsize=60,
port=9099,
num_tokens=1
)
Now, call the run_gabriel_server method and the server will be started. Download `Gabriel client
<https://play.google.com/store/apps/details?id=edu.cmu.cs.gabrielclient>`_ from
Android Play Store to connect to it and try it out.
)

0 comments on commit b3188c4

Please sign in to comment.