Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Handle multiple entities in e2e testing #4033

Merged
merged 34 commits into from Aug 15, 2019
Merged

Handle multiple entities in e2e testing #4033

merged 34 commits into from Aug 15, 2019

Conversation

MetcalfeTom
Copy link
Contributor

@MetcalfeTom MetcalfeTom commented Jul 17, 2019 •
edited

Entities were padded for sklearn metrics too early, which caused errors in examples with multiple entities.
Also, we were passing the correct entities as the predicted ones during construction of WronglyClassifiedUserUtterance, so the results were incorrect
Fixes #4017

Proposed changes:

  • Use padded/serialised results for sklearn only
  • Change WronglyClassifiedUserUtterance
  • Use restaurantbot it for the e2e testing (since it has entities)

Status (please check what you already did):

  • made PR ready for code review
  • added some tests for the functionality
  • updated the changelog
  • reformat files using black (please check Readme for instructions)

@MetcalfeTom MetcalfeTom mentioned this pull request Jul 17, 2019
Closed
tabergma
tabergma reviewed Jul 18, 2019
View reviewed changes
Copy link
Contributor

@tabergma tabergma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a couple of comments. Also the tests are failing.

examples/restaurantbot/README.md Outdated Show resolved Hide resolved
rasa/core/test.py Outdated Show resolved Hide resolved
rasa/core/test.py Outdated Show resolved Hide resolved
rasa/core/test.py Outdated Show resolved Hide resolved
rasa/core/test.py Show resolved Hide resolved
rasa/core/test.py Outdated Show resolved Hide resolved
rasa/core/test.py Show resolved Hide resolved
@MetcalfeTom MetcalfeTom changed the title Handle multiple entities in e2e learning Handle multiple entities in e2e testing Jul 22, 2019
@MetcalfeTom
Copy link
Contributor Author

@tabergma had a couple issues on python3.5 because of the way the entities were being sorted when serialising the EvaluationStore(). For this reason I've opted to use the markdown format.

Now it's ready for another review

tabergma
tabergma reviewed Aug 5, 2019
View reviewed changes
CHANGELOG.rst Show resolved Hide resolved
CHANGELOG.rst Outdated Show resolved Hide resolved
rasa/core/test.py Show resolved Hide resolved
rasa/core/test.py Outdated Show resolved Hide resolved
tabergma
tabergma approved these changes Aug 5, 2019
View reviewed changes
Copy link
Contributor

@tabergma tabergma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Thanks for fixing this. Looks good!

@MetcalfeTom MetcalfeTom mentioned this pull request Aug 7, 2019
Merged
4 tasks
@tabergma
Copy link
Contributor

@MetcalfeTom Is this ready to be merged once the tests pass?

@MetcalfeTom
Copy link
Contributor Author

Yes - not sure how I overlooked this

@tabergma tabergma merged commit 63be756 into master Aug 15, 2019
@tabergma tabergma deleted the e2e_entity_fix branch August 15, 2019 11:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tabergma tabergma tabergma approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Errors using rasa test
2 participants
@MetcalfeTom @tabergma