Merge branch 'master' into typechecked

CHANGELOG.rst

@@ -6,7 +6,7 @@ Rasa Change Log
 All notable changes to this project will be documented in this file.
 This project adheres to `Semantic Versioning`_ starting with version 1.0.
 
-[Unreleased 1.1.1] - `master`_
+[Unreleased 1.1.3] - `master`_
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Added
@@ -21,6 +21,13 @@ Removed
 Fixed
 -----
 
+[1.1.2] - 2019-06-13
+^^^^^^^^^^^^^^^^^^^^
+
+Fixed
+-----
+- fixed duplicate events appearing in tracker when using a PostgreSQL tracker store
+
 [1.1.1] - 2019-06-13
 ^^^^^^^^^^^^^^^^^^^^
 

docs/api/featurization.rst

@@ -43,7 +43,7 @@ of the tracker has a couple steps:
 
         The target labels correspond to actions taken by the bot.
         To convert the features into vector format, there are different
-        feaurizers available:
+        featurizers available:
 
         - ``BinarySingleStateFeaturizer`` creates a binary one-hot encoding:
             The vectors ``X, y`` indicate a presence of a certain intent,

docs/core/about.rst

@@ -51,5 +51,5 @@ The Rasa Core Dialogue Engine
    :sender: user
 
 .. chat-bubble::
-   :text: Head over to the <a href="../user-guide/rasa-tutorial.html">Rasa Tutorial</a> for an interactive example.
+   :text: Head over to the <a href="../../user-guide/rasa-tutorial">Rasa Tutorial</a> for an interactive example.
    :sender: user

docs/core/domains.rst

@@ -157,7 +157,7 @@ You can also send any arbitrary output to the output channel using the
 payload should first be converted to yaml format.
 
 For example, although date pickers are not a defined parameter in utterance
-templates because they are not supported by most channels, a slack date picker
+templates because they are not supported by most channels, a Slack date picker
 can be sent like so:
 
 .. code-block:: yaml
@@ -195,7 +195,7 @@ output payloads that will only work in certain channels.
     - text: "Which game would you like to play?"
       channel: "slack"
       custom:
-        - # payload for slack dropdown menu to choose a game
+        - # payload for Slack dropdown menu to choose a game
     - text: "Which game would you like to play?"
       buttons:
       - title: "Chess"

docs/core/forms.rst

@@ -70,7 +70,7 @@ happy paths.
 In this story the user intent is ``request_restaurant``, which is followed by
 the form action ``restaurant_form``. With ``form{"name": "restaurant_form"}`` the
 form is activated and with ``form{"name": null}`` the form is deactivated again.
-As shown in the section :ref:`section_unhappy` the the bot can execute any kind of
+As shown in the section :ref:`section_unhappy` the bot can execute any kind of
 actions outside the form while the form is still active. On the "happy path",
 where the user is cooperating well and the system understands the user input correctly,
 the form is filling all requested slots without interruption.

docs/core/interactive-learning.rst

@@ -20,8 +20,7 @@ Some people call this `Software 2.0 <https://medium.com/@karpathy/software-2-0-a
 .. note::
 
     Rasa X provides a UI for interactive learning, and you can use any user conversation
-    as a starting point. See `Copy to Interactive Learning
-    <../../rasa-x/docs/annotate-conversations/>`_ in the Rasa X docs.
+    as a starting point. See `Annotate Conversations <https://rasa.com/docs/rasa-x/annotate-conversations/>`_ in the Rasa X docs.
 
 
 Running Interactive Learning

docs/nlu/choosing-a-pipeline.rst

@@ -38,7 +38,7 @@ A Longer Answer
 
 The two most important pipelines are ``supervised_embeddings`` and ``pretrained_embeddings_spacy``.
 The biggest difference between them is that the ``pretrained_embeddings_spacy`` pipeline uses pre-trained
-word vectors from either GloVe or fastText. Instead, the supervised embeddings pipeline
+word vectors from either GloVe or fastText. The ``supervised embeddings`` pipeline, on the other hand,
 doesn't use any pre-trained word vectors, but instead fits these specifically for your dataset.
 
 The advantage of the ``pretrained_embeddings_spacy`` pipeline is that if you have a training example like:

docs/nlu/components.rst

@@ -539,7 +539,7 @@ MitieEntityExtractor
         }
 
 :Description:
-    This uses the MITIE entitiy extraction to find entities in a message. The underlying classifier
+    This uses the MITIE entity extraction to find entities in a message. The underlying classifier
     is using a multi class linear SVM with a sparse linear kernel and custom features.
     The MITIE component does not provide entity confidence values.
 :Configuration:

docs/nlu/training-data-format.rst

@@ -9,8 +9,8 @@ Training Data Format
 Data Format
 ~~~~~~~~~~~
 
-You can provide training data as markdown or as json, as a single file or as a directory containing multiple files.
-Note that markdown is usually easier to work with.
+You can provide training data as Markdown or as JSON, as a single file or as a directory containing multiple files.
+Note that Markdown is usually easier to work with.
 
 
 Markdown Format
@@ -19,7 +19,7 @@ Markdown Format
 Markdown is the easiest Rasa NLU format for humans to read and write.
 Examples are listed using the unordered
 list syntax, e.g. minus ``-``, asterisk ``*``, or plus ``+``.
-Examples are grouped by intent, and entities are annotated as markdown links,
+Examples are grouped by intent, and entities are annotated as Markdown links,
 e.g. ``[entity](entity name)``.
 
 .. code-block:: md
@@ -233,10 +233,9 @@ Lookup Tables
 -------------
 Lookup tables in the form of external files or lists of elements may also be specified in the training data.
 The externally supplied lookup tables must be in a newline-separated format.
-For example, ``data/test/lookup_tables/plates.txt`` may contain
+For example, ``data/test/lookup_tables/plates.txt`` may contain:
 
-# TODO: This is broken
-.. include:: ../../data/test/lookup_tables/plates.txt
+.. literalinclude:: ../../data/test/lookup_tables/plates.txt
 
 And can be loaded as:
 

docs/user-guide/command-line-interface.rst

@@ -234,8 +234,8 @@ This command will attempt to keep the proportions of intents the same in train a
 Convert Data Between Markdown and JSON
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-To convert NLU data from LUIS data format, WIT data format, Dialogflow data format, json, or Markdown
-to json or Markdown, run:
+To convert NLU data from LUIS data format, WIT data format, Dialogflow data format, JSON, or Markdown
+to JSON or Markdown, run:
 
 .. code:: bash
 

docs/user-guide/evaluating-models.rst

@@ -199,7 +199,7 @@ For each policy configuration provided, Rasa Core will be trained multiple times
 with 0, 5, 25, 50, 70 and 95% of your training stories excluded from the training
 data. This is done for multiple runs to ensure consistent results.
 
-Once this script has finished, you can use the evaluate script in compare
+Once this script has finished, you can use the evaluate script in ``compare``
 mode to evaluate the models you just trained:
 
 .. code-block:: bash

docs/user-guide/rasa-tutorial.rst

@@ -118,7 +118,7 @@ The ``policies`` key defines the :ref:`policies <policies>` that the Core model
 4. Write Your First Stories
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-At this stage, you will teach your assistant to how respond to your messages.
+At this stage, you will teach your assistant how to respond to your messages.
 This is called dialogue management, and is handled by your Core model.
 
 Core models learn from real conversational data in the form of training "stories".

rasa/core/tracker_store.py

@@ -100,14 +100,18 @@ def retrieve(self, sender_id: Text) -> Optional[DialogueStateTracker]:
         raise NotImplementedError()
 
     def stream_events(self, tracker: DialogueStateTracker) -> None:
-        old_tracker = self.retrieve(tracker.sender_id)
-        offset = len(old_tracker.events) if old_tracker else 0
+        offset = self.number_of_existing_events(tracker.sender_id)
         evts = tracker.events
         for evt in list(itertools.islice(evts, offset, len(evts))):
             body = {"sender_id": tracker.sender_id}
             body.update(evt.as_dict())
             self.event_broker.publish(body)
 
+    def number_of_existing_events(self, sender_id: Text) -> int:
+        """Return number of stored events for a given sender id."""
+        old_tracker = self.retrieve(sender_id)
+        return len(old_tracker.events) if old_tracker else 0
+
     def keys(self) -> Iterable[Text]:
         raise NotImplementedError()
 
@@ -472,23 +476,13 @@ def save(self, tracker: DialogueStateTracker) -> None:
             "stored to database".format(tracker.sender_id)
         )
 
-    def _additional_events(self, tracker: DialogueStateTracker) -> Iterator:
-        """Return events from the tracker which aren't currently stored."""
-
-        from sqlalchemy import func
-
-        query = self.session.query(func.max(self.SQLEvent.timestamp))
-        max_timestamp = query.filter_by(sender_id=tracker.sender_id).scalar()
-
-        if max_timestamp is None:
-            max_timestamp = 0
+    def number_of_existing_events(self, sender_id: Text) -> int:
+        """Return number of stored events for a given sender id."""
 
-        latest_events = []
+        query = self.session.query(self.SQLEvent.sender_id)
+        return query.filter_by(sender_id=sender_id).count() or 0
 
-        for event in reversed(tracker.events):
-            if event.timestamp > max_timestamp:
-                latest_events.append(event)
-            else:
-                break
-
-        return reversed(latest_events)
+    def _additional_events(self, tracker: DialogueStateTracker) -> Iterator:
+        """Return events from the tracker which aren't currently stored."""
+        n_events = self.number_of_existing_events(tracker.sender_id)
+        return itertools.islice(tracker.events, n_events, len(tracker.events))

rasa/version.py

@@ -1 +1 @@
-__version__ = "1.1.1"
+__version__ = "1.1.2"