started to rewrite featurization docs

docs/featurisation.rst

@@ -5,13 +5,7 @@ Featurization
 
 In order to apply machine learning algorithms to conversational AI, we need to build up vector representations of conversations.
 
-
-We use the ``X, y`` notation that's common for supervised learning, where ``X`` is a matrix of shape ``(num_data_points, data_dimension)``, and ``y`` is a 1D array of length ``num_data_points`` containing the target class labels.
-
-The target labels correspond to actions taken by the bot.
-If the domain defines the possible ``actions`` ``[ActionGreet, ActionListen]`` then a label 0 indicates a greeting and 1 indicates a listen.
-
-The rows in ``X`` correspond to the state of the conversation just before the action was taken.
+Each story corresponds to a tracker which consists of the states of the conversation just before each action was taken.
 
 Featurising a single state works like this:
 the tracker provides a bag of ``active_features`` comprising:
@@ -21,17 +15,46 @@ the tracker provides a bag of ``active_features`` comprising:
 - features indicating which slots are currently defined, e.g. ``slot_location`` if the user previously mentioned the area they're searching for restaurants.
 - features indicating the results of any API calls stored in slots, e.g. ``slot_matches``
 
-All of these features are represented in a binary vector which just indicates if they're present.
+
+Single State Featurizers
+^^^^^^^^^^^^^^^^^^^^^^^^
+
+``SingleStateFeaturizer`` converts all of these states into numeric vectors ``X, y``.
+
+We use the ``X, y`` notation that's common for supervised learning,
+where ``X`` is a matrix of shape ``(num_data_points, time_dimension, num_features)``,
+and ``y`` is an array of length ``num_data_points`` containing the target class labels encoded as one-hot vectors.
+
+The target labels correspond to actions taken by the bot.
+.. note::
+    If the domain defines the possible ``actions``: ``[ActionGreet, ActionGoodbye]``,
+    two additional default actions are added: ``[ActionListen, ActionRestart]``.
+    Therefore, label ``0`` indicates default action listen, label ``1`` default restart,
+    label ``2`` a greeting and ``3`` indicates goodbye.
+
+Binary
+------
+``BinarySingleStateFeaturizer`` converts all of these features to a binary vectors ``X, y`` which just indicates if they're present.
  e.g. ``[0 0 1 1 0 1 ...]``
 
-To recover the bag of features from a vector ``vec``, you can call ``domain.reverse_binary_encoded_features(vec)``.
-This is very useful for debugging.
+Label Tonizer
+-------------
+
+
+Tracker Featurizers
+^^^^^^^^^^^^^^^^^^^
+
+
+Full Dialogue
+-------------
+
+
 
-History
--------
+Max History
+-----------
 
 It's often useful to include a bit more history than just the current state in memory.
 The parameter ``max_history`` defines how many states go into defining each row in ``X``. 
 
-Hence the statement above that ``X`` is 2D is actually false, it has shape ``(num_states, max_history, num_features)``.
-For most algorithms you want a flat feature vector, so you will have to reshape this to ``(num_states, max_history * num_features)``.
+Hence ``X`` has shape ``(num_states, max_history, num_features)``.
+For some algorithms you want a flat feature vector, so you will have to reshape this to ``(num_states, max_history * num_features)``.

rasa_core/policies/embedding_policy.py

@@ -499,9 +499,9 @@ def _create_tf_graph(self, a_in, b_in, c_in, b_prev_in):
         b_prev = self._create_tf_nn(b_prev_in,
                                     self.num_hidden_layers_b,
                                     self.hidden_layer_size_b,
-                                    self.droprate['b'],
+                                    self.droprate['a'],
                                     name=name_b)
-        b_prev = tf.layers.dropout(b_prev, rate=self.droprate['b'],
+        b_prev = tf.layers.dropout(b_prev, rate=self.droprate['a'],
                                    training=self.is_training)
         emb_prev_act = self._create_embed(b_prev, name=name_b)