Evaluation of word2vec models against semantic similarity datasets

[akutuzov]

We long had analogy evaluation of wor2vec models in Gensim (also known as analogical inference). However, another type of evaluation is widespread in distributional semantics world, that is using word pairs ranked by their semantic similarity (see SimLex-999 and other datasets), and the correlation of these similarities to those produced by the model.

This PR adds the self.evaluation function to perform such evaluation against arbitrary datasets.

[piskvorky]

Thanks @akutuzov , that looks useful!

But what's with all those commits? Most look unrelated, and some look downright scary (like f3f2a52).

Also, we'll have to change the name evaluate to something more specific -- how about evaluate_word_pairs?

[akutuzov]

Thanks @piskvorky
I am certainly not against renaming, done.
As for extra commits, I am trying to understand why Github has bound this PR with my previous one (#538).
Only the last few commits starting from e11909f make sense in the context of this PR. Only two files are changed, in fact.

[akutuzov]

This is crazy.
So, can you squash all these commits into one or probably I should just start another PR from scratch?

[tmylk]

@akutuzov thanks for the feature. Could we please add some simple unit tests for this new feature?

[akutuzov]

@tmylk what can those be? Evaluating against a toy dataset? Should it follow the same structure as testAccuracy in https://github.com/RaRe-Technologies/gensim/blob/develop/gensim/test/test_word2vec.py#L370?

Also, what should we do with the old unneeded commits in this PR? As I've said, I can probably start a new one from scratch, if it is not possible to just squash them all into one on Gensim side.

[tmylk]

That test is not a good example. It is not a test of accuracy but a test of KeyedVectors. A good test is when a model trained on Lee corpus being given a single pair to evaluate, like in the sanity test

There is another point. Having the small and canonical questions-words.txt in the repo helps a lot of people to test accuracy of their models. So we should add a semantic similarity dataset it is less than 1Mb .

Don't worry about commits, I will squash them.

[akutuzov]

OK, I will add a test, then.

[tmylk]

Thanks for the PR! Merging to add it to this year's release. Tests and a dataset should be in a separate PR.

[akutuzov]

Cool, thanks!
I will implement tests in a few days.