Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

confusing about threshold setting #1

Closed
yawenouyang opened this issue Nov 1, 2019 · 1 comment
Closed

confusing about threshold setting #1

yawenouyang opened this issue Nov 1, 2019 · 1 comment

Comments

@yawenouyang
Copy link

In our evaluation, the out-of-scope threshold was chosen to be the value which yielded the highest validation score across all intents, treating out-of-scope as its own intent.

I am a little confused by this sentence. Does it mean that we select oos's highest score on the known intention as the threshold in the validation set? If so, isn't oos's recall equal to 1 in each epoch of validation set, how do we early stop and select hyper-parameters?

@gxlarson
Copy link
Contributor

Thanks for your interest.
We selected the out-of-scope threshold by maximizing the accuracy on all intents (including out-of-scope) on the validation set.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants