Skip to content
This repository has been archived by the owner on Aug 22, 2019. It is now read-only.

Evaluation improvements #1225

Merged
merged 8 commits into from Oct 25, 2018
Merged

Evaluation improvements #1225

merged 8 commits into from Oct 25, 2018

Conversation

ricwo
Copy link
Contributor

@ricwo ricwo commented Oct 24, 2018 •
edited

Proposed changes:

  • fixes #1221
  • partial tracker uses correct user utterance and correct action, regardless of prediction
  • entities are excluded from evaluation metrics
  • the failed stories output prints user messages in end-to-end form, regardless of whether prediction was correct or not

Status (please check what you already did):

  • made PR ready for code review
  • added some tests for the functionality
  • updated the documentation
  • updated the changelog

@ricwo ricwo requested a review from tmbo October 24, 2018 20:57
tmbo
tmbo approved these changes Oct 25, 2018
View reviewed changes
Copy link
Member

@tmbo tmbo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like it, nice job 🚀 - would keep the timestamp property though

rasa_core/evaluate.py Show resolved Hide resolved
@ricwo ricwo changed the base branch from eval_endpoint to master October 25, 2018 09:05
@ricwo ricwo merged commit fc6340d into master Oct 25, 2018
@tmbo tmbo deleted the eval_improvements branch October 27, 2018 15:45
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@tmbo tmbo tmbo approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
2 participants
@ricwo @tmbo