Merge branch 'master' into e2e_entity_fix

.github/ISSUE_TEMPLATE/feature_request.md

@@ -27,5 +27,6 @@ Or might there be any potential blockers on the way? -->
 **Definition of Done**:
 <!-- What needs to be there to consider this feature as done?
 - [ ] Tests are added
-- [ ] Feature is documented
+- [ ] Feature described the docs
+- [ ] Feature mentioned in the changlog
 - [ ] ... -->

.gitignore

@@ -81,4 +81,5 @@ docs/core/key
 docs/core/key.pub
 failed_stories.md
 errors.json
+pip-wheel-metadata/*
 events.db

CHANGELOG.rst

@@ -16,14 +16,30 @@ Changed
 -------
 - messages with multiple entities are now handled properly with e2e evaluation
 - ``data/test_evaluations/end_to_end_story.md`` was re-written in the restaurantbot domain
+- `FallbackPolicy` can now be configured to trigger when the difference between confidences of two predicted intents is too narrow
+- throw error during training when triggers are defined in the domain without
+  ``MappingPolicy`` being present in the policy ensemble
+- experimental training data importer which supports training with data of multiple
+  sub bots. Please see the
+  `docs <https://rasa.com/docs/rasa/api/training-data-importers/>`_ for more
+  information.
+- The tracker is now available within the interpreter's ``parse`` method, giving the ability to create interpreter classes that 
+  use the tracker state (eg. slot values) during the parsing of the message. More details on motivation of this change see issues/3015
 
-Removed
+Changed
 -------
+- added character-level ``CountVectorsFeaturizer`` with empirically found parameters 
+  into the ``supervised_embeddings`` NLU pipeline template
+- bot messages contain the `timestamp` of the `BotUttered` event, which can be used in channels
+- NLU evaluations now also stores its output in the output directory like the core evaluation
+- show warning in case a default path is used instead of a provided, invalid path
 
+Removed
+-------
 
 Fixed
 -----
-
+- ``rasa test nlu`` with a folder of configuration files
 
 [1.2.2] - 2019-08-07
 ^^^^^^^^^^^^^^^^^^^^

data/test_multi_domain/config.yml

@@ -7,7 +7,7 @@ policies:
   - name: KerasPolicy
 
 importers:
-  - name: SkillSelector
+  - name: MultiProjectImporter
 
 imports:
   - data/MoodBot

docs/_static/spec/rasa.yml

@@ -364,9 +364,9 @@ paths:
       - Model
       summary: Train a Rasa model
       description: >-
-        Trains a Rasa model. Depending on the data given only a dialogue model,
+        Trains a new Rasa model. Depending on the data given only a dialogue model,
         only a NLU model, or a model combining a trained dialogue model with an
-        NLU model will be trained. The trained model is not loaded by default.
+        NLU model will be trained. The new model is not loaded by default.
       requestBody:
         required: true
         content:

docs/api/training-data-importers.rst

@@ -8,6 +8,9 @@ Training Data Importers
 
 .. edit-link::
 
+.. contents::
+   :local:
+
 By default, you can use command line arguments to specify where Rasa should look
 for training data on your disk. Rasa then loads any potential training files and uses
 them to train your assistant.
@@ -51,6 +54,89 @@ configuration file:
    importers:
    - name: "RasaFileImporter"
 
+MultiProjectImporter (experimental)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. warning::
+
+    This feature is currently experimental and might change or be removed in the future.
+    Please share your feedback on it in the `forum <https://forum.rasa.com>`_ to help
+    us making this feature ready for production.
+
+With this importer you can build a contextual AI assistant by combining multiple
+reusable Rasa projects.
+You might, for example, handle chitchat with one project and greet your users with
+another. These projects can be developed in isolation, and then combined at train time
+to create your assistant.
+
+An example directory structure could look like this:
+
+.. code-block:: bash
+
+    .
+    ├── config.yml
+    └── projects
+        ├── GreetBot
+        │   ├── data
+        │   │   ├── nlu.md
+        │   │   └── stories.md
+        │   └── domain.yml
+        └── ChitchatBot
+            ├── config.yml
+            ├── data
+            │   ├── nlu.md
+            │   └── stories.md
+            └── domain.yml
+
+In this example the contextual AI assistant imports the ``ChitchatBot`` project which in turn
+imports the ``GreetBot`` project. Project imports are defined in the configuration files of
+each project.
+To instruct Rasa to use the ``MultiProjectImporter`` module, put this section in the config
+file of your root project:
+
+.. code-block:: yaml
+
+    importers:
+    - name: MultiProjectImporter
+
+
+Then specify which projects you want to import.
+In our example, the ``config.yml`` in the root project would look like this:
+
+.. code-block:: yaml
+
+    imports:
+    - projects/ChitchatBot
+
+The configuration file of the ``ChitchatBot`` in turn references the ``GreetBot``:
+
+.. code-block:: yaml
+
+    imports:
+    - ../GreetBot
+
+The ``GreetBot`` project does not specify further projects so the ``config.yml`` can be
+omitted.
+
+Rasa uses relative paths from the referencing configuration file to import projects.
+These can be anywhere on your file system as long as the file access is permitted.
+
+During the training process Rasa will import all required training files, combine
+them, and train a unified AI assistant. The merging of the training data happens during
+runtime, so no additional files with training data are created or visible.
+
+.. note::
+
+    Rasa will use the policy and NLU pipeline configuration of the root project
+    directory during training. **Policy or NLU configurations of imported projects
+    will be ignored.**
+
+.. note::
+
+    Equal intents, entities, slots, templates, actions and forms will be merged,
+    e.g. if two projects have training data for an intent ``greet``,
+    their training data will be combined.
+
 Writing a Custom Importer
 ~~~~~~~~~~~~~~~~~~~~~~~~~
 If you are writing a custom importer, this importer has to implement the interface of

docs/conf.py

@@ -87,7 +87,6 @@
     "Thumbs.db",
     ".DS_Store",
     # ignore doc pages that we don't show to appease keep_warnings
-    "multi-skill-assistants.rst",
     "core/old-core-change-log.rst",
     "core/old-core-migration-guide.rst",
     "nlu/old-nlu-change-log.rst",

docs/core/domains.rst

@@ -277,6 +277,8 @@ multiple responses and Rasa will randomly pick one of them, e.g.:
     - text: "Hey, {name}. How are you?"
     - text: "Hey, {name}. How is your day going?"
 
+.. _use_entities:
+
 Ignoring entities for certain intents
 -------------------------------------
 

docs/core/policies.rst

@@ -406,9 +406,11 @@ Fallback Policy
 ^^^^^^^^^^^^^^^
 
 The ``FallbackPolicy`` invokes a :ref:`fallback action
-<fallback-actions>` if the intent recognition
-has a confidence below ``nlu_threshold`` or if none of the dialogue
-policies predict an action with confidence higher than ``core_threshold``.
+<fallback-actions>` if at least one of the following occurs:
+1. The intent recognition has a confidence below ``nlu_threshold``.
+2. The highest ranked intent differs in confidence with the second highest ranked intent
+by less than ``ambiguity_threshold``.
+3. None of the dialogue policies predict an action with confidence higher than ``core_threshold``.
 
 **Configuration:**
 
@@ -420,13 +422,18 @@ policies predict an action with confidence higher than ``core_threshold``.
       policies:
         - name: "FallbackPolicy"
           nlu_threshold: 0.3
+          ambiguity_threshold: 0.1
           core_threshold: 0.3
           fallback_action_name: 'action_default_fallback'
 
     +----------------------------+---------------------------------------------+
     | ``nlu_threshold``          | Min confidence needed to accept an NLU      |
     |                            | prediction                                  |
     +----------------------------+---------------------------------------------+
+    | ``ambiguity_threshold``    | Min amount by which the confidence of the   |
+    |                            | top intent must exceed that of the second   |
+    |                            | highest ranked intent.                      |
+    +----------------------------+---------------------------------------------+
     | ``core_threshold``         | Min confidence needed to accept an action   |
     |                            | prediction from Rasa Core                   |
     +----------------------------+---------------------------------------------+
@@ -446,7 +453,8 @@ policies predict an action with confidence higher than ``core_threshold``.
 
        fallback = FallbackPolicy(fallback_action_name="action_default_fallback",
                                  core_threshold=0.3,
-                                 nlu_threshold=0.3)
+                                 nlu_threshold=0.3,
+                                 ambiguity_threshold=0.1)
 
        agent = Agent("domain.yml", policies=[KerasPolicy(), fallback])
 
@@ -462,7 +470,8 @@ Two-Stage Fallback Policy
 The ``TwoStageFallbackPolicy`` handles low NLU confidence in multiple stages
 by trying to disambiguate the user input.
 
-- If a NLU prediction has a low confidence score, the user is asked to affirm
+- If an NLU prediction has a low confidence score or is not significantly higher
+  than the second highest ranked prediction, the user is asked to affirm
   the classification of the intent.
 
     - If they affirm, the story continues as if the intent was classified
@@ -494,6 +503,7 @@ by trying to disambiguate the user input.
         policies:
           - name: TwoStageFallbackPolicy
             nlu_threshold: 0.3
+            ambiguity_threshold: 0.1
             core_threshold: 0.3
             fallback_core_action_name: "action_default_fallback"
             fallback_nlu_action_name: "action_default_fallback"
@@ -503,6 +513,10 @@ by trying to disambiguate the user input.
     | ``nlu_threshold``             | Min confidence needed to accept an NLU   |
     |                               | prediction                               |
     +-------------------------------+------------------------------------------+
+    | ``ambiguity_threshold``       | Min amount by which the confidence of the|
+    |                               | top intent must exceed that of the second|
+    |                               | highest ranked intent.                   |
+    +--------------------------------------------------------------------------+
     | ``core_threshold``            | Min confidence needed to accept an action|
     |                               | prediction from Rasa Core                |
     +-------------------------------+------------------------------------------+