-
Notifications
You must be signed in to change notification settings - Fork 19.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Incorporating Word Vectors rather than using an Embedding class #853
Comments
UPDATED NOV 13, 2017 You have to pass a weight matrix to the Let's say So, an example
And you also have a dictionary called
The following code should do what you want
Note that this kind of setup will result in your embeddings being trained from their initial point! If you want them fixed, then you have to set |
No need of skipping the embedding layer.. setting word vectors as the initial weights of embedding layer is a valid approach. The word vectors will get fine tuned for the specific NLP task during training. |
Has anybody else attempted to embedded the word vectors into a model? I've managed to create the model however Im not able to achieve a worthwhile level of accuracy yet. I've used the "20 newsgroups dataset" from scikit-learn to test this model, with my own w2v vectors. The best accuracy I've achieved so far is 28%, over 5 epochs, which is not great (scikit script best - 85%). I intend to continue experimenting with the network configuration (inner dimensions and epochs initially). I do suspect that the number of dimentions is too high for such a small dataset (1000 samples). Will update again if the results improve |
From what I've seen, training your own vectors on top of a custom dataset has given me much better accuracy within that domain. That said - any updates on this? |
There are 3 approaches:
The third one is the best option(Assuming the word vectors were obtained from the same domain as the inputs to your models. For e.g, if you are doing sentiment analysis on tweets, you should use GloVe vectors trained on tweets). In the first option, everything has to be learned from scratch. You dont need it unless you have a rare scenario. The second one is good, but, your model will be unnecessarily big with all the word vectors for words that are not frequently used. |
@farizrahman4u agreed on those counts. The domains are a bit more specific and I have a lot more luck with option (2) than with (1) or (3) so far. An easy way to address the size problem with (2) is to prune out the vocabulary itself to the top k words. |
@farizrahman4u Thanks for sharing the ideas. I have a question on your first approach. Learning embeddings from scratch: each word in the dictionary is represented as one-hot vector, and then this vector is embedded as a continuous vector after applying embedding layer. Is that right? |
@MagicYoung Yes. |
To follow up from @sergeyf suggestions... @viksit @MagicYoung @vindiesel Take a look at my attempt (Script and Data) - it uses Gensim. This could easily all be done in Keras using the skipgram of course to remove dependencies... |
Looks cool! Are the results to your liking? |
Results after 2 epochs: |
@dandxy89 |
@liyi193328 are you using keras' sequence.pad_sequences(myseq, maxlen=maxlen)? Looks like your padding is to the right of the vectors where as it should be to the left.
Secondly, you should be using categorical_crossentropy as your loss as opposed to mean_squared_error. See #321 |
@viksit Thanks. |
You need to pad before you convert the words to vectors (presumably you have a step where you have only word indexes). |
@viksit Thanks. [ [ He, like, keras], [learning], [like, keras] ]so Index 2D array is [ [1,2,3], [4], [2,3] ] ---padding---> [ [1,2,3], [0,0,4], [0,2,,3] ](index of each is given) He -> [1,1,1,1], like->[2,2,2,2], keras->[3,3,3,3], learning->[5,5,5,5]then after padding ,the 3D array shape is (2,3,4), like: [[ [1,1,1,1], [2,2,2,2], [3,3,3,3] ],[ [0,0,0,0], [0,0,0,0], [5,5,5,5] ],[ [0,0,0,0], [2,2,2,2], [3,3,3,3] ]]if the specific example right? |
Thats correct. |
@viksit Thanks. |
@dandxy89 @farizrahman4u @viksit Do you mean, when initialize Embedding Layer with weights learned for other corpus calculated word2vec, the LSTM model can quickly convergence? Here is my code, just iter 2-3 times will get high score. https://github.com/taozhijiang/chinese_nlp/blob/master/DL_python/dl_segment_v2.py |
Thanks for sharing. I benefit a lot. @ |
Follow @liyi193328 comment: If this is my X_train: How should I structure my Y_train given each word (word-level training) will have it's own tag(Y and is also multi class). Is it like: [ Because I am having an error: Thank you very much! |
@ngopee |
Thank you very much for your reply! Yes, I realised that later on but now having another issue which I'm not sure how to fix: "if every token has a label and has y classes, then all the labels's(Y_train) shape is (x,y), 2D array". I'm not sure I follow this part. But below is what I have so far. So, in my case, each token(Word vector) has is own tag. Here is a sample of my input:
I am getting this error:
Here is my code: vocab_dim = 300
Thank you very much! |
@ngopee |
Thank you very much for pointing that out. That would have been yet another mistake. However this did not seem to fix the issue I previously had. Any insight on what I could be doing wrong? Thanks! |
@ngoee |
I still do not understand can I use Embedding layer and NOT TUNE IT AT ALL just as the saving memory mechanism? |
An Embedding Layer has weights, and they need to be tuned: https://github.com/fchollet/keras/blob/master/keras/layers/embeddings.py#L95 There are, however, pretrained word2vec models, which are already trained and hence need not to be retrained if they fit your needs. |
What should be the padding vector if i am using pretrained word2vec from google? Should i use word like 'stop' and transform it to vector with google word2vec model or should i use just vector of zeros? |
What happened to the weights argument to the Embedding function? I am following the tutorial on the blog here: https://blog.keras.io/using-pre-trained-word-embeddings-in-a-keras-model.html where we pass in the pre-trained embedding matrix. However, in the latest keras version, the weights function has been deleted. The PR that deleted it is here: 023331e |
@prinsherbert This approach may lead to over-fitting. For example you have 100000 words (pretty small vocabulary) all with 100 vector size if they are trainable you will get 10 millions free parameters "from nothing". |
@MaratZakirov Yes, it is wise to pretrain or just train with a large corpus such as Wikipedia to prevent overfitting. And yes, you need a high document to word ratio. |
From what I understand, the Embedding layer in Keras performs a lookup for a word index present in an input sequence, and replaces it with the corresponding vector through an embedding matrix. However, what I am confused about it what happens when we want to test/apply a model on unknown data? For example, if there is a word in the test document which is not present in the training vocabulary, we could compute the corresponding vectors using character n-grams using pre-trained Fast-text model. However, this term would not be present in the word_index that was generated while training the model, and a lookup in the embedding matrix would fail. One possible solution can be to create a word_index from the entire dataset, including the test data, as done for this example: https://github.com/fchollet/keras/blob/master/examples/pretrained_word_embeddings.py Any suggestion for workarounds for that? |
I think unknown words in general are mapped to random vectors, but this is not what the Embedding-layer does. In text processing people often consider the vocabulary prior knowledge. And if your model is word based, you will unlikely learn about words not seen in the training data anyway. Tweets are examples of data with many words that you'll likely will not encounter during training, but that do appear in testing, because people make typos. More importantly, they also add (hash) tags, which are typically some concatenation of words. If you search the literature, you'll notice many tweet-classifiers use a character level convolutional layer, and than some classifier on top of that (like LSTM). So if you want to generalize to new words, consider character level features/classifiers. |
It seems like these responses ignore the main issue, is if weights argument was removed in Keras 2.0? |
@BrianMiner Yes, it seems like embedding weights have indeed been removed. My guess is it can still be used via the option 'embeddings_initializer' and coding a custom initializer which returns embedding weights. @vijay120 @prinsherbert Yes, I am aware of such techniques. I was only wondering of an elegant way to add embeddings of OOVs based on character n-gram embeddings using Fast-text during test to avoid completely ignoring OOV terms in a word based model. |
Hello @Madhumita-Git @prinsherbert I also have a similar scenario, where I want to use a character-based model for another NLP task. So that my model starts with: Any idea about how I could use an |
If you are working on large data, it is recommended to directly use word vectors as an input to LSTM layer rather than having an embedding layer. This avoids matrix multiplication which takes a lot of time if there are more sequences. |
Hi! Is it just me or would your Since you've already set the width to be But in your embedding_weights it's
|
I think that +2 thing only made sense in an old version of Keras, and I
absolutely can't remember why anymore. But I do remember being annoyed.
I should probably update this comment, huh? It's like documentation by now!
…On Sep 2, 2017 4:50 PM, "Eric" ***@***.***> wrote:
@sergeyf <https://github.com/sergeyf>
Hi! Is it just me or would your embedding_weights numpy zeros array be a
+1 too wide?
Since you've already set the width to be n_symbols = len(index_dict) + 1
to account for 0th index
But in your embedding_weights it's embedding_weights =
np.zeros((n_symbols+1,vocab_dim)), which would be the same as the
original n_symbols = len(index_dict) + 2? Why the extra +1 to length?
vocab_dim = 300 # dimensionality of your word vectors
n_symbols = len(index_dict) + 1 # adding 1 to account for 0th index (for masking)
embedding_weights = np.zeros((n_symbols+1,vocab_dim))
for word,index in index_dict.items():
embedding_weights[index,:] = word_vectors[word]
# assemble the model
model = Sequential() # or Graph or whatever
model.add(Embedding(output_dim=rnn_dim, input_dim=n_symbols + 1, mask_zero=True, weights=[embedding_weights])) # note you have to put embedding weights in a list by convention
model.add(LSTM(dense_dim, return_sequences=False))
model.add(Dropout(0.5))
model.add(Dense(n_symbols, activation='softmax')) # for this is the architecture for predicting the next word, but insert your own here
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#853 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/ABya7HryYQV4Ygn9iVChsKgLxS33U9i8ks5sec2hgaJpZM4GRCom>
.
|
Hello! Following this: From what I see Keras 2 + is not supporting embedding weights (yes?). Anyone can advise whether I am doing something wrong? Thanks!
|
Have you run this and made sure it fails?
https://blog.keras.io/using-pre-trained-word-embeddings-in-a-keras-model.html
Says updated for keras 2
…On Nov 13, 2017 9:33 AM, "DomHudson" ***@***.***> wrote:
Is there any concensus of whether @sergeyf <https://github.com/sergeyf>'s
approach still works? It does indeed appear that the weights argument has
been removed, but it's still being used in the examples here
<https://github.com/fchollet/keras/blob/master/examples/pretrained_word_embeddings.py#L122>
..
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#853 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AHKCOfLkfQgjIc873-xQZm1gVJ9UEWj5ks5s2FMegaJpZM4GRCom>
.
|
Hi everyone, Looks like this is still a references for some people. Here is what I do now with Keras 2.0.8:
Note! This version assumes that the Hope that helps. |
Thanks for the reply both! @AllardJM It doesn't error no. I've done a little investigation and with the following code I've come to the conclusion that it is being utilised and in fact, the original solution (using a import numpy as np
from keras import initializers
from keras.layers import Embedding, LSTM, Dense
from keras.models import Sequential weights = np.concatenate([
np.zeros((1, 100)), # Masking row: all zeros.
np.ones((1, 100)) # First word: all weights preset to 1.
])
print(weights)
layer = Embedding(
output_dim = 100,
input_dim = 2,
mask_zero = True,
weights = [weights],
)
model = Sequential([
layer,
LSTM(2, dropout = 0.2, activation = 'tanh'),
Dense(1, activation = 'sigmoid')
])
model.compile(
optimizer = 'adam',
loss = 'binary_crossentropy',
metrics = []
) print(layer.get_weights())
The recent solution above also appears to work, but is probably more inefficient as it's initialising the weights and then overwriting them. This can be seen as shown: layer = Embedding(
output_dim = 100,
input_dim = 2,
mask_zero = True
)
layer.build((None,))
print(layer.get_weights())
layer.set_weights([weights])
print(layer.get_weights())
|
Great! This is what I found as well, the example code still works fine to pass in weights to initialize the embedding.
Cheers!
On 11/15/2017 08:05 AM, DomHudson wrote:
Thanks for the reply both!
@AllardJM<https://github.com/allardjm> It doesn't error no. I've done a little investigation and with the following code I've come to the conclusion that it is being utilised and in fact, the original solution (using a weights kwarg) that @sergeyf<https://github.com/sergeyf> posted does still work.
import numpy as np
from keras import initializers
from keras.layers import Embedding, LSTM, Dense
from keras.models import Sequential
weights = np.concatenate([
np.zeros((1, 100)), # Masking row: all zeros.
np.ones((1, 100)) # First word: all weights preset to 1.
])
print(weights)
array([[ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0.],
[ 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1., 1., 1., 1.]])
layer = Embedding(
output_dim = 100,
input_dim = 2,
mask_zero = True,
weights = [weights],
)
model = Sequential([
layer,
LSTM(2, dropout = 0.2, activation = 'tanh'),
Dense(1, activation = 'sigmoid')
])
model.compile(
optimizer = 'adam',
loss = 'binary_crossentropy',
metrics = []
)
print(layer.get_weights())
[array([[ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0.],
[ 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1., 1., 1., 1.]], dtype=float32)]
…________________________________
The recent solution above also appears to work, but is probably more inefficient as it's initialising the weights and then overwriting them. This can be seen as shown:
layer = Embedding(
output_dim = 100,
input_dim = 2,
mask_zero = True
)
layer.build((None,))
print(layer.get_weights())
[array([[ -2.64064074e-02, -4.05902900e-02, -1.71032399e-02,
6.36395207e-03, 4.03554030e-02, -2.91514937e-02,
-3.05371974e-02, 1.60062015e-02, -4.58858572e-02,
-2.71607353e-03, -6.45029533e-04, -3.60430926e-02,
-4.47065122e-02, -4.46958952e-02, 8.49759020e-03,
-2.07597855e-02, -4.63474654e-02, -4.47412431e-02,
-7.12857256e-03, 3.30050252e-02, -1.70418713e-02,
-3.46117802e-02, 1.63293723e-02, -3.06463335e-02,
-3.92450131e-02, 2.13836078e-02, 3.40061374e-02,
3.08677852e-02, 4.10322733e-02, 3.48727070e-02,
3.77323031e-02, 4.75023203e-02, -4.60593663e-02,
4.89875488e-02, -1.86587516e-02, 1.37329465e-02,
-1.24689462e-02, -2.74951141e-02, -2.39574052e-02,
-4.11705412e-02, -2.67224889e-02, -1.86454095e-02,
9.51218046e-03, 1.30047565e-02, -1.28185125e-02,
1.50464000e-02, -3.25884894e-02, 1.06664898e-03,
3.91772352e-02, -4.15717773e-02, 3.98341119e-02,
1.08094336e-02, -2.93221492e-02, -3.67895775e-02,
-1.90059599e-02, 3.34730162e-03, -2.74142250e-02,
-4.06444333e-02, -4.97532897e-02, -4.81352210e-04,
-2.15560924e-02, 4.51278277e-02, -2.36345585e-02,
4.39978205e-02, 2.73948014e-02, 4.52689640e-02,
1.53716626e-02, -2.49101524e-03, -8.96360632e-03,
3.06243300e-02, 4.95609641e-02, 2.66981137e-04,
3.92680196e-03, -2.85005327e-02, 4.53012399e-02,
3.41285653e-02, 4.43088599e-02, -1.21087050e-02,
-3.81706282e-02, -3.51855792e-02, 3.59421670e-02,
-3.01210601e-02, -3.23626027e-02, 4.94807661e-02,
-1.53903933e-02, 9.66792088e-03, 1.23059156e-03,
-1.84051401e-03, -3.88573073e-02, -3.77015956e-02,
4.48914282e-02, 3.49486731e-02, -4.73317020e-02,
1.45648129e-03, -2.16338988e-02, -3.01712025e-02,
-4.01302688e-02, 1.65192429e-02, -3.59362774e-02,
2.93326676e-02],
[ -4.46196795e-02, -2.18213685e-02, 2.72371471e-02,
4.23214212e-02, -3.41014937e-02, 4.29243445e-02,
3.27980518e-03, -4.80787531e-02, 3.40308845e-02,
-6.82551879e-03, 4.03380400e-04, 4.45233956e-02,
4.18974236e-02, -1.88305825e-02, 7.91913306e-04,
4.96885180e-03, -1.89449489e-02, 3.14035825e-02,
4.15420346e-02, -3.21644135e-02, -3.54666486e-02,
-3.17389816e-02, 2.59683859e-02, -3.76684554e-02,
4.51624401e-05, 4.44507562e-02, -4.96175438e-02,
-4.82493974e-02, 4.00636811e-03, -4.86469679e-02,
-2.88026463e-02, -4.70020436e-02, -1.23844091e-02,
-1.96035542e-02, -4.45893705e-02, -2.10967846e-02,
4.90186326e-02, -1.49804656e-03, -3.46895168e-03,
3.20515819e-02, -3.41350446e-03, 3.22987102e-02,
-3.75118107e-02, 3.66315842e-02, 6.32166862e-03,
-2.67616995e-02, -2.28005182e-02, 3.59728225e-02,
-1.14186527e-02, 6.25128765e-03, -1.01642106e-02,
1.16781592e-02, -3.82909179e-03, 3.07524931e-02,
-3.32702114e-03, 1.29272817e-02, -4.88508958e-03,
4.88356426e-02, 3.67677584e-02, -1.22928238e-02,
-5.73384156e-03, 2.96543725e-02, 4.05017398e-02,
-9.28649586e-03, -2.95463633e-02, -4.89737280e-02,
-4.42623487e-03, -4.81910333e-02, 8.44216347e-03,
-4.26033465e-03, 2.13968400e-02, -2.50094850e-02,
-4.68100868e-02, -1.76477917e-02, -1.68486964e-02,
1.41983628e-02, -3.38780954e-02, -3.14644054e-02,
4.16858196e-02, 4.50237580e-02, -6.27965620e-03,
5.43129456e-04, 3.63374949e-02, -1.94281098e-02,
-3.25115174e-02, 3.43143530e-02, 4.91250828e-02,
-4.51278165e-02, 4.32032421e-02, 3.06754243e-02,
-1.41283274e-02, 4.49896120e-02, -3.07326354e-02,
4.95368838e-02, -8.92946147e-04, 3.42890918e-02,
-1.97444838e-02, 3.26766376e-03, 3.58569697e-02,
4.43595164e-02]], dtype=float32)]
layer.set_weights([weights])
print(layer.get_weights())
[array([[ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0.],
[ 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1., 1., 1., 1.]], dtype=float32)]
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub<#853 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AIZkgnZmiRcmk9KcVAniAyspRySVNzz7ks5s2uG2gaJpZM4GRCom>.
|
Is there standard code or a function that takes a model built in gensim word2vec and converts it into the dictionary format's (i.e. index_dict and word_vectors the first comment above)? Otherwise I will write my code for this but that seems much less efficient. Thanks! -- { { |
@aksg87 You could use the The model = Word2Vec.load(fname)
embedding_layer = model.wv.get_keras_embedding(train_embeddings=True) Source: https://github.com/RaRe-Technologies/gensim/blob/develop/gensim/models/keyedvectors.py#L1048 |
Thank you so much for your reply. I ended up finding some examples and wrote it out: I'll have to try the version you provided. Also, if you set train_embeddings=True the weights in the layer will change from the word2vect output is this advisable in general? The code I wrote to do this: load the whole embedding into memoryembeddings_index = dict() dim_len = len(coefs) create a weight matrix for words in training docsembedding_matrix = zeros((vocab_size, dim_len))
|
Another question I have is my final output is a softmax prediction on several classes (396 to be exact). The output vector is messy (see below). Is their a clean way to both 1) convert this into the top 3 labels predicted and 2) write a custom accuracy function which checks how often the softmax predicts the top 3? array([ 2.74735111e-22, 0.00000000e+00, 0.00000000e+00, |
I don't think there's a 'correct' answer to this - it's up to you and the problem you're modelling. By having a trainable embeddings layer the weights will be tuned for the model's NLP task. This will give you more domain specific weights at the cost of increased training time. It's quite common to train initial weights on a large corpus (or to use a pre-trained third party model) and then use that to seed your embedding layer. In this case you will likely find benefit if you do train the embeddings layer with the model. However, if you've trained your own Word2Vec model on exactly the domain you're modelling, you may find that the difference in results is negligible and that training the layer is not preferential over a shorter training time.
To do this you could use numpy's >>> predictions = np.array([0.1, 0.3, 0.2, 0.4, 0.5])
>>> top_three_classes = np.argpartition(predictions, -3)[-3:]
>>> top_three_classes
array([1, 3, 4])
Yes this should be fairly straightforward utilising the above logic and a custom metric class or function. |
To calculate accuracy, I created a few functions and used Map to apply them on my prediction which essentially tell me how often my model's 'Top 3' prediction contains the true answer. (At the very end I basically counted 'True' vs 'False' to arrive at a percentage. I thought Keras might have a way to overwrite their Accuracy function but didn't see a way.)
|
I am solving an NLP task and I am trying to model it directly as a sequence using different RNN flavors. How can I use my own Word Vectors rather than using an instance of layers.embeddings.Embedding?
The text was updated successfully, but these errors were encountered: