Using Theano in Word Embedding Method

[lucas0]

Hello,

I'm trying to use your pre-processing methods to feed my LSTM RNN, and I cannot find a way to use the method for creating the Word Embeddings matrixes:

import theano, numpy
from theano import tensor as T

# nv :: size of our vocabulary
# de :: dimension of the embedding space
# cs :: context window size

nv, de, cs = 1000, 50, 5
embeddings = theano.shared(0.2 * numpy.random.uniform(-1.0, 1.0, \
(nv+1, de)).astype(theano.config.floatX)) # add one for PADDING at the end
idxs = T.imatrix() # as many columns as words in the context window and as many lines as words in the sentence
x = self.emb[idxs].reshape((idxs.shape[0], de*cs))

I can do the context window, but the code provided for the Word Embedding generation gives me this error:

NameError: name 'self' is not defined

I've tried to put the code in another .py file but is of no use, I'm sure it has a simple solution but I can't figure it out. Would you please give me a hand in this?

[mesnilgr]

https://en.wikipedia.org/wiki/Object-oriented_programming

[lucas0]

Seems like

x = self.emb[idxs].reshape((idxs.shape[0], de*cs))

should be:

x = embeddings[idxs].reshape((idxs.shape[0], de*cs))

Am I right? @mesnilgr

[Helicqin]

@lucas0 i met the same question.i want to run the code:

import theano, numpy
from theano import tensor as T

# nv :: size of our vocabulary
# de :: dimension of the embedding space
# cs :: context window size

nv, de, cs = 1000, 50, 5
emb = theano.shared(0.2 * numpy.random.uniform(-1.0, 1.0, \
(nv+1, de)).astype(theano.config.floatX)) # add one for PADDING at the end
idxs = T.imatrix() # as many columns as words in the context window and as many lines as words in the sentence
x = self.emb[idxs].reshape((idxs.shape[0], de*cs))

but errors happened in emb[idxs].i do not understand why emb[idxs] is correct?i think emb is a matrix,idxs is also a matrix.So why?

I would appreciate it if you could give me some tips.thanks.

[lucas0]

Im not sure, but idxs is a matrix of indexes, so it can be used as index of another matrix, like: a = [7,6,5,4,3,2,1] //a list b = [1, 3, 4] //a list of idxs a[b] = [6, 4, 3] hope it helps a bit.

…

On May 11, 2017, at 1:50 PM, Helicqin ***@***.***> wrote: @lucas0 <https://github.com/lucas0> i met the same question.i want to run the code: import theano, numpy from theano import tensor as T # nv :: size of our vocabulary # de :: dimension of the embedding space # cs :: context window size nv, de, cs = 1000, 50, 5 emb = theano.shared(0.2 * numpy.random.uniform(-1.0, 1.0, \ (nv+1, de)).astype(theano.config.floatX)) # add one for PADDING at the end idxs = T.imatrix() # as many columns as words in the context window and as many lines as words in the sentence x = self.emb[idxs].reshape((idxs.shape[0], de*cs)) but errors happened in emb[idxs].i do not understand why emb[idxs] is correct?i think emb is a matrix,idxs is also a matrix.So why? I would appreciate it if you could give me some tips.thanks. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#19 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAr0PGt4jqIZ282ltddECL1yhL9m7OHYks5r4wQtgaJpZM4GcZ4z>.