Skip to content
NEG loss implemented in pytorch
Python
Branch: master
Clone or download
kefirski Update neg.py
Sampling fix
Latest commit 9da4e76 Jul 7, 2017
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
NEG_loss Update neg.py Jul 7, 2017
images add module Mar 4, 2017
.gitignore add module Mar 4, 2017
LICENSE Initial commit Mar 4, 2017
README.md window support added May 10, 2017

README.md

Pytorch Negative Sampling Loss

Negative Sampling Loss implemented in PyTorch.

NEG Loss Equation

Usage

neg_loss = NEG_loss(num_classes, embedding_size)
    
optimizer = SGD(neg_loss.parameters(), 0.1)
    
for i in range(num_iterations):
    ''' 
    input is [batch_size] shaped tensors of Long type
    while target has shape of [batch_size, window_size]
    '''
    input, target = next_batch(batch_size)
        
    loss = neg_loss(input, target, num_sample)
    
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()
    
word_embeddings = neg_loss.input_embeddings()        
You can’t perform that action at this time.