Merge pull request #4194 from RasaHQ/multi-skill-docs

move multi skill docs to data importer docs

CHANGELOG.rst

@@ -14,16 +14,18 @@ Added
 - `FallbackPolicy` can now be configured to trigger when the difference between confidences of two predicted intents is too narrow
 - throw error during training when triggers are defined in the domain without
   ``MappingPolicy`` being present in the policy ensemble
-- The tracker is now avaialble within the interpreter's ``parse`` method, giving the ability to create interpreter classes that use the tracker state (eg. slot values) during the parsing of the message. More details on motivation of this change see issues/3015
+- experimental training data importer which supports training with data of multiple
+  sub bots. Please see the
+  `docs <https://rasa.com/docs/rasa/api/training-data-importers/>`_ for more
+  information.
+- The tracker is now available within the interpreter's ``parse`` method, giving the ability to create interpreter classes that 
+  use the tracker state (eg. slot values) during the parsing of the message. More details on motivation of this change see issues/3015
 
 Changed
 -------
 - added character-level ``CountVectorsFeaturizer`` with empirically found parameters 
   into the ``supervised_embeddings`` NLU pipeline template
 - bot messages contain the `timestamp` of the `BotUttered` event, which can be used in channels
-
-Changed
--------
 - NLU evaluations now also stores its output in the output directory like the core evaluation
 
 Removed

data/test_multi_domain/config.yml

@@ -7,7 +7,7 @@ policies:
   - name: KerasPolicy
 
 importers:
-  - name: SkillSelector
+  - name: MultiProjectImporter
 
 imports:
   - data/MoodBot

docs/api/training-data-importers.rst

@@ -8,6 +8,9 @@ Training Data Importers
 
 .. edit-link::
 
+.. contents::
+   :local:
+
 By default, you can use command line arguments to specify where Rasa should look
 for training data on your disk. Rasa then loads any potential training files and uses
 them to train your assistant.
@@ -51,6 +54,89 @@ configuration file:
    importers:
    - name: "RasaFileImporter"
 
+MultiProjectImporter (experimental)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. warning::
+
+    This feature is currently experimental and might change or be removed in the future.
+    Please share your feedback on it in the `forum <https://forum.rasa.com>`_ to help
+    us making this feature ready for production.
+
+With this importer you can build a contextual AI assistant by combining multiple
+reusable Rasa projects.
+You might, for example, handle chitchat with one project and greet your users with
+another. These projects can be developed in isolation, and then combined at train time
+to create your assistant.
+
+An example directory structure could look like this:
+
+.. code-block:: bash
+
+    .
+    ├── config.yml
+    └── projects
+        ├── GreetBot
+        │   ├── data
+        │   │   ├── nlu.md
+        │   │   └── stories.md
+        │   └── domain.yml
+        └── ChitchatBot
+            ├── config.yml
+            ├── data
+            │   ├── nlu.md
+            │   └── stories.md
+            └── domain.yml
+
+In this example the contextual AI assistant imports the ``ChitchatBot`` project which in turn
+imports the ``GreetBot`` project. Project imports are defined in the configuration files of
+each project.
+To instruct Rasa to use the ``MultiProjectImporter`` module, put this section in the config
+file of your root project:
+
+.. code-block:: yaml
+
+    importers:
+    - name: MultiProjectImporter
+
+
+Then specify which projects you want to import.
+In our example, the ``config.yml`` in the root project would look like this:
+
+.. code-block:: yaml
+
+    imports:
+    - projects/ChitchatBot
+
+The configuration file of the ``ChitchatBot`` in turn references the ``GreetBot``:
+
+.. code-block:: yaml
+
+    imports:
+    - ../GreetBot
+
+The ``GreetBot`` project does not specify further projects so the ``config.yml`` can be
+omitted.
+
+Rasa uses relative paths from the referencing configuration file to import projects.
+These can be anywhere on your file system as long as the file access is permitted.
+
+During the training process Rasa will import all required training files, combine
+them, and train a unified AI assistant. The merging of the training data happens during
+runtime, so no additional files with training data are created or visible.
+
+.. note::
+
+    Rasa will use the policy and NLU pipeline configuration of the root project
+    directory during training. **Policy or NLU configurations of imported projects
+    will be ignored.**
+
+.. note::
+
+    Equal intents, entities, slots, templates, actions and forms will be merged,
+    e.g. if two projects have training data for an intent ``greet``,
+    their training data will be combined.
+
 Writing a Custom Importer
 ~~~~~~~~~~~~~~~~~~~~~~~~~
 If you are writing a custom importer, this importer has to implement the interface of

docs/conf.py

@@ -87,7 +87,6 @@
     "Thumbs.db",
     ".DS_Store",
     # ignore doc pages that we don't show to appease keep_warnings
-    "multi-skill-assistants.rst",
     "core/old-core-change-log.rst",
     "core/old-core-migration-guide.rst",
     "nlu/old-nlu-change-log.rst",

rasa/data.py

@@ -64,8 +64,6 @@ def get_core_nlu_files(
 
     Args:
         paths: List of paths to training files or folders containing them.
-        skill_imports: `SkillSelector` instance which determines which files
-                        should be loaded.
 
     Returns:
         Tuple of paths to story and NLU files.

rasa/importers/importer.py

@@ -148,14 +148,14 @@ def _importer_from_dict(
         domain_path: Optional[Text] = None,
         training_data_paths: Optional[List[Text]] = None,
     ) -> Optional["TrainingDataImporter"]:
-        from rasa.importers.skill import SkillSelector
+        from rasa.importers.multi_project import MultiProjectImporter
         from rasa.importers.rasa import RasaFileImporter
 
         module_path = importer_config.pop("name", None)
         if module_path == RasaFileImporter.__name__:
             importer_class = RasaFileImporter
-        elif module_path == SkillSelector.__name__:
-            importer_class = SkillSelector
+        elif module_path == MultiProjectImporter.__name__:
+            importer_class = MultiProjectImporter
         else:
             try:
                 importer_class = common_utils.class_from_module_path(module_path)

rasa/importers/skill.py → rasa/importers/multi_project.py

@@ -12,11 +12,12 @@
 from rasa.importers import utils
 from rasa.nlu.training_data import TrainingData
 from rasa.core.training.structures import StoryGraph
+import rasa.utils.common
 
 logger = logging.getLogger(__name__)
 
 
-class SkillSelector(TrainingDataImporter):
+class MultiProjectImporter(TrainingDataImporter):
     def __init__(
         self,
         config_file: Text,
@@ -44,11 +45,15 @@ def __init__(
         self._nlu_paths += list(extra_nlu_files)
 
         logger.debug(
-            "Selected skills: {}".format(
+            "Selected projects: {}".format(
                 "".join(["\n-{}".format(i) for i in self._imports])
             )
         )
 
+        rasa.utils.common.mark_as_experimental_feature(
+            feature_name="MultiProjectImporter"
+        )
+
     def _init_from_path(self, path: Text) -> None:
         if os.path.isfile(path):
             self._init_from_file(path)

rasa/utils/common.py

@@ -247,3 +247,14 @@ def not_found():
         return c[name]
     else:
         return not_found()
+
+
+def mark_as_experimental_feature(feature_name: Text) -> None:
+    """Warns users that they are using an experimental feature."""
+
+    logger.warning(
+        "The {} is currently experimental and might change or be "
+        "removed in the future 🔬 Please share your feedback on it in the "
+        "forum (https://forum.rasa.com) to help us make this feature "
+        "ready for production.".format(feature_name)
+    )