negative sampling in Word2Vec tutorial

[bizzyvinci]

URL(s) with the issue:

negative sampling
generate training data

Description of issue (what needs changing):

I was going through the tutorial on skipgram word2vec and I noticed that positive sample candidates are also negative sample candidates too.

Clear description

For example, we have sentence = "The wide road shimmered in the hot sun" and window_size = 2 for tf.keras.preprocessing.sequence.skipgrams. Therefore positive skip-grams for include (road, the), (road, wide), (road, shimmered), & (road, in).

I guess the, wide, shimmered, & in should not be later labeled as negative skip-grams for road, right?.

PS: I'm a newbie

[Saduf2019]

@bizzyvinci
Thank you for the request, we will update you on the earliest on this.

[gqqnbig]

The incorrect negative samples are calculated from tf.random.log_uniform_candidate_sampler, which may have bugs. See #44758.

[8bitmp3]

Thank you @bizzyvinci and @gqqnbig. We'll look into this cc @MarkDaoust

[wangpengmit]

Please see my comment #44758 (comment) . The phrase "negative samples" may have different meanings in different candidate-sampling algorithms. For some algorithms "negative samples" can overlap with true classes.

We do need to mention this caveat in the tutorial though.