Docs: Fix and supplement answer relevancy description #705
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The docs for the answer relevancy metric are wrong in that it is claimed that the score ranges between 0 and 1. As it is written in the calculation section an averaged cosine similarity is used to calculate the metric. The cosine similarity can range between -1 and 1. Of course, most of the time the original question and generated questions will be kind of similar in practice, leading to scores between 0 and 1, but this is not mathematically guaranteed and depends on the used embedder and how it stretches the embedding space.
The following example shows this:
I updated the description and added a formula to make it clearer.