Merge pull request #1013 from RasaHQ/forms_v2

Forms v2

.gitignore

@@ -42,6 +42,10 @@ tokens.dat
 graph.png
 graph.html
 debug.md
+examples/restaurantbot/models*
+examples/moodbot/*.png
+examples/moodbot/errors.json
+examples/formbot/models*
 examples/concertbot/data*
 examples/concertbot/models*
 examples/moodbot/models*

CHANGELOG.rst

@@ -41,6 +41,17 @@ Added
 - environment variables specified with ``${env_variable}`` in a yaml
   configuration file are now replaced with the value of the environment variable
 - detailed documentation on how to deploy Rasa with Docker
+- add ``FormPolicy`` to handle form action prediction
+- add ``ActionExecutionRejection`` exception and ``ActionExecutionRejected`` event
+- add default action ``ActionDeactivateForm()``
+- add ``formbot`` example
+- add ability to turn off auto slot filling with entity for each slot in domain.yml
+- add ``InvalidDomain`` exception
+- add ``active_form_...`` to state dictionary
+- add ``active_form`` and ``latest_action_name`` properties to ``DialogueStateTracker``
+- add ``Form`` and ``FormValidation`` events
+- add ``REQUESTED_SLOT`` constant
+- add ability to read ``action_listen`` from stories
 
 Changed
 -------
@@ -50,7 +61,10 @@ Changed
 - the core container does not load the nlu model by default anymore.
   Instead it can be connected to a nlu server.
 - stories are now visualized as ``.html`` page instead of an image
-
+- move and deduplicate restaurantbot nlu data from ``franken_data.json`` to ``nlu_data.md``
+- forms were completely reworked, see changelog in ``rasa_core_sdk``
+- state featurization if some form is active changed
+- ``Domain`` raises ``InvalidDomain`` exception
 
 Removed
 -------

data/test_domains/form.yml

@@ -0,0 +1,35 @@
+intents:
+ - greet
+ - default
+ - goodbye
+ - start_form
+ - stop
+ - affirm
+ - deny
+ - inform
+
+slots:
+  cuisine:
+    type: text
+  location:
+    type: text
+
+entities:
+ - name
+
+templates:
+  utter_greet:
+    - hey there!
+  utter_goodbye:
+    - goodbye :(
+  utter_default:
+    - default message
+
+actions:
+  - utter_default
+  - utter_greet
+  - utter_goodbye
+  - utter_ask_continue
+
+forms:
+  - some_form

data/test_stories/stories_form.md

@@ -0,0 +1,61 @@
+## simple_story_with_multiple_turns
+* greet
+    - utter_greet
+* default
+    - utter_default
+* goodbye
+    - utter_goodbye
+
+## simple_story_with_form_happy
+* greet
+    - utter_greet
+* start_form
+    - some_form
+    - form{"name": "some_form"}
+* form: inform
+    - form: some_form
+    - form{"name": null}
+* default
+    - utter_default
+* goodbye
+    - utter_goodbye
+
+## simple_story_with_form_unhappy
+* greet
+    - utter_greet
+* start_form
+    - some_form
+    - form{"name": "some_form"}
+* default
+    - utter_default
+    - some_form
+    - form{"name": null}
+* goodbye
+    - utter_goodbye
+
+## simple_story_with_form_stop_continue
+* greet
+    - utter_greet
+* start_form
+    - some_form
+    - form{"name": "some_form"}
+* stop
+    - utter_ask_continue
+* affirm
+    - some_form
+    - form{"name": null}
+* goodbye
+    - utter_goodbye
+
+## simple_story_with_form_stop_deactivate
+* greet
+    - utter_greet
+* start_form
+    - some_form
+    - form{"name": "some_form"}
+* stop
+    - utter_ask_continue
+* deny
+    - action_deactivate_form
+    - form{"name": null}
+    - utter_goodbye

data/test_trackers/tracker_moodbot.json

@@ -22,6 +22,8 @@
     ]
   },
   "sender_id": "mysender",
+  "latest_action_name": "action_listen",
+  "active_form": {},
   "paused": false,
   "latest_event_time": 1517821726.211042,
   "followup_action": null,

docs/api/featurizer.rst

@@ -68,8 +68,9 @@ of the tracker has a couple steps:
 
     If the domain defines the possible ``actions``,
     ``[ActionGreet, ActionGoodbye]``,
-    two additional default actions are added:
-    ``[ActionListen, ActionRestart]``.
+    ``4`` additional default actions are added:
+    ``[ActionListen(), ActionRestart(),
+    ActionDefaultFallback(), ActionDeactivateForm()]``.
     Therefore, label ``0`` indicates default action listen, label ``1``
     default restart, label ``2`` a greeting and ``3`` indicates goodbye.
 

docs/api/slots_api.rst

@@ -99,6 +99,8 @@ List Slot
        set value, the feature will be ``0``. The **length of the list stored in
        the slot does not influence the dialogue**.
 
+Unfeaturized Slot
+-----------------
 
 .. option:: unfeaturized
 

docs/interactive_learning.rst

@@ -6,21 +6,21 @@ Interactive Learning
 ====================
 
 
-Interactive learning means giving feedback to your bot while you talk
-to it. It is a powerful tool! Interactive learning is a powerful way
+In interactive learning mode, you provide feedback to your bot while you talk
+to it. This is a powerful way
 to explore what your bot can do, and the easiest way to fix any mistakes
-it makes. One advantage of machine learning based dialogue is that when
+it makes. One advantage of machine learning-based dialogue is that when
 your bot doesn't know how to do something yet, you can just teach it!
-Some people are calling this `Software 2.0 <https://medium.com/@karpathy/software-2-0-a64152b37c35>`_.
+Some people call this `Software 2.0 <https://medium.com/@karpathy/software-2-0-a64152b37c35>`_.
 
 
 Load up an existing bot
 ^^^^^^^^^^^^^^^^^^^^^^^
 
-We have a basic working bot, and want to teach it by providing
-feedback on mistakes it makes.
+We have created some initial stories, and now want to improve our bot 
+by providing feedback on mistakes it makes.
 
-Run the following to start interactive learning:
+Run the following command to start interactive learning:
 
 .. code-block:: bash
 
@@ -29,15 +29,15 @@ Run the following to start interactive learning:
    python -m rasa_core.train \
      --interactive -o models/dialogue \
      -d domain.yml -s stories.md \
-     --endpoints endpoints.yml 
-     
-To include an existing model to identify intents use --nlu models/current/nlu in the above command. Else interactive learning will use a default REGEX to intentify default intents from the user input text. 
+     --nlu models/current/nlu \
+     --endpoints endpoints.yml
 
 The first command starts the action server (see :ref:`customactions`).
 
 The second command starts the bot in interactive mode.
-In interactive mode, the bot will ask you to confirm it has chosen
-the right action before proceeding:
+In interactive mode, the bot will ask you to confirm every prediction 
+made by NLU and Core before proceeding.
+Here's an example:
 
 
 .. code-block:: text
@@ -62,12 +62,15 @@ the right action before proceeding:
     ? The bot wants to run 'utter_greet', correct?  (Y/n)
 
 
-This gives you all the info you should hopefully need to decide
-what the bot *should* have done. In this case, the bot chose the
+The chat history and slot values are printed to the screen, which 
+should be all the information your need to decide what the correct 
+next action is.
+
+In this case, the bot chose the
 right action (``utter_greet``), so we type ``y``.
-Then we type ``y`` again, because 'action_listen' is the correct
-action after greeting. We continue this loop until the bot chooses
-the wrong action.
+Then we type ``y`` again, because ``action_listen`` is the correct
+action after greeting. We continue this loop, chatting with the bot, 
+until the bot chooses the wrong action.
 
 Providing feedback on errors
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -135,26 +138,131 @@ reviews!) so we select that action.
 
 Now we can keep talking to the bot for as long as we like to create a longer
 conversation. At any point you can press ``Ctrl-C`` and the bot will
-provide you with exit options, e.g. writing the created conversations as
-stories to a file. Make sure to combine the dumped story with your original
-training data for the next training.
+provide you with exit options. You can write your newly-created stories and NLU 
+data to files. You can also go back a step if you made a mistake when providing 
+feedback.
 
+Make sure to combine the dumped stories and NLU examples with your original
+training data for the next training.
 
 Visualization of conversations
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 During the interactive learning, Core will plot the current conversation
-and close surrounding conversations from the training data to help you
+and a few similar conversations from the training data to help you
 keep track of where you are.
 
 You can view the visualization at http://localhost:5005/visualization.html
-as soon as you have started the interactive learning.
+as soon as you've started interactive learning.
 
 To skip the visualization, pass ``--skip_visualization`` to the training
 script.
 
 .. image:: _static/images/interactive_learning_graph.gif
 
-.. include:: feedback.inc
+.. _section_interactive_learning_forms:
+
+Interactive Learning with Forms
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+If you're using a FormAction, there are some additional things to keep in mind 
+when using interactive learning.
+
+The ``form:`` prefix
+~~~~~~~~~~~~~~~~~~~~
+
+The form logic is described by your ``FormAction`` class, and not by the stories.
+The machine learning policies should not have to learn this behavior, and should 
+not get confused if you later change your form action, for example by adding or 
+removing a required slot. 
+When you user interactive learning to generate stories containing a form, 
+the conversation steps handled by the form 
+get a :code:`form:` prefix. This tells Rasa Core to ignore these steps when training 
+your other policies. There is nothing special you have to do here, all of the form's 
+happy paths are still covered by the basic story given in :ref:`section_form_basics`.
+
+Here is an example:
+
+.. code-block:: story
+
+    * request_restaurant
+        - restaurant_form
+        - form{"name": "restaurant_form"}
+        - slot{"requested_slot": "cuisine"}
+    * form: inform{"cuisine": "mexican"}
+        - slot{"cuisine": "mexican"}
+        - form: restaurant_form
+        - slot{"cuisine": "mexican"}
+        - slot{"requested_slot": "num_people"}
+    * form: inform{"number": "2"}
+        - form: restaurant_form
+        - slot{"num_people": "2"}
+        - form{"name": null}
+        - slot{"requested_slot": null}
+        - utter_slots_values
+
+
+Input validation
+~~~~~~~~~~~~~~~~
+
+Every time the user responds with something *other* than the requested slot or 
+any of the required slots, 
+you will be asked whether you want the form action to try and extract a slot 
+from the user's message when returning to the form. This is best explained with 
+and example:
+
+.. code-block:: text
+
+     7    restaurant_form 1.00
+          slot{"num_people": "3"}
+          slot{"requested_slot": "outdoor_seating"}
+          do you want to sit outside?
+          action_listen 1.00
+    ─────────────────────────────────────────────────────────────────────────────────────
+     8                                                                             /stop
+                                                                       intent: stop 1.00
+    ─────────────────────────────────────────────────────────────────────────────────────
+     9    utter_ask_continue 1.00
+          do you want to continue?
+          action_listen 1.00
+    ─────────────────────────────────────────────────────────────────────────────────────
+     10                                                                          /affirm
+                                                                     intent: affirm 1.00
 
 
+    Current slots:
+    	cuisine: greek, feedback: None, num_people: 3, outdoor_seating: None,
+      preferences: None, requested_slot: outdoor_seating
+
+    ------
+    2018-11-05 21:36:53 DEBUG    rasa_core.tracker_store  - Recreating tracker for id 'default'
+    ? The bot wants to run 'restaurant_form', correct?  Yes
+    2018-11-05 21:37:08 DEBUG    rasa_core.tracker_store  - Recreating tracker for id 'default'
+    ? Should 'restaurant_form' validate user input to fill the slot 'outdoor_seating'?  (Y/n)
+
+Here the user asked to stop the form, and the bot asks the user whether they're sure
+they don't want to continue. The user says they want to continue (the ``/affirm`` intent). 
+Here ``outdoor_seating`` has a ``from_intent`` slot mapping (mapping
+the ``/affirm`` intent to ``True``), so this user input could be used to fill
+that slot. However, in this case the user is just responding to the
+"do you want to continue?" question and so you select ``n``, the user input
+should not be validated. The bot will then continue to ask for the
+``outdoor_seating`` slot again.
+
+.. warning::
+
+    If there is a conflicting story in your training data, i.e. you just chose
+    to validate the input (meaning it will be printed with the ``forms:`` prefix),
+    but your stories file contains the same story where you don't validate
+    the input (meaning it's without the ``forms:`` prefix), you will need to make
+    sure to remove this conflicting story. When this happens, there is a warning
+    prompt that reminds you to do this:
+
+    **WARNING: FormPolicy predicted no form validation based on previous training
+    stories. Make sure to remove contradictory stories from training data**
+
+    Once you've removed that story, you can press enter and continue with
+    interactive learning
+
+
+.. include:: feedback.inc