Can't do multi label classification using LSTM

[12gjang]

from keras.layers.core import Dropout, Dense
from keras.layers.normalization import BatchNormalization
from keras.layers.recurrent import LSTM
from keras.models import Sequential
import numpy as np
import random

time_window_size = 50
nb_training_data = 100
data_dim = 25
nb_label = 2
train_x = np.zeros((nb_training_data, time_window_size+1, data_dim), dtype=bool)

for i in range(len(train_x)):
for j in range(len(train_x[i])):
for k in range(nb_label):
train_x[i][j][random.randint(0, data_dim - 1)] = 1

train_x = np.array(train_x, dtype=np.bool)

model = Sequential()
dropout = 0.25

model.add(LSTM(100, return_sequences=False, input_shape=(time_window_size, data_dim)))
model.add(BatchNormalization())
model.add(Dropout(dropout))

model.add(Dense(data_dim, activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

model.fit(train_x[:, 0:time_window_size, :], train_x[:,time_window_size, :], batch_size=50, nb_epoch=1000, shuffle=True)

---

That is my test code for multi-label classification.

I have searched method to solve multi label classification.

Many people recommended to use sigmoid + binary_crossentropy or relu (tanh) + mse

But, both of them doesn't work well on multi label classification.

Always stuck in around 0.6 accuracy. (in case of two label classification)

Is there some thing wrong in my approach ?

Please help me, I have too much struggled in this problem..

[phdowling]

Well you are using binary_crossentropy, you should probably be using categorical_crossentropy.

[jskDr]

I think Philipp's comment is reasonable since 'binary_crossentropy' must be
used for dual level classification, not the multilevel one.

By the way, is it not affordable to make it automatic in Keras, such as
loss = 'auto_classification'?
To my memory, Sklearn supports automatic mode in which library detects the
maximum level from the data.
http://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html

Also, is 'input_data.dense_to_one_hot()' provided in Keras, to transform
dense integer scalar to a one hot vector?

Best,

On Fri, May 27, 2016 at 9:59 AM Philipp Dowling notifications@github.com
wrote:

Well you are using binary_crossentropy, you should probably be using
categorical_crossentropy.

—
You are receiving this because you are subscribed to this thread.

Reply to this email directly, view it on GitHub
#2826 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/ACchnbghnkiM5JtYKN9KmXy6mrjqXC7Jks5qFvitgaJpZM4Inzl5
.

Sungjin (James) Kim, PhD

Postdoc, CCB in Harvard

sungjinkim@fas.harvard.edu

[Web] http://aspuru.chem.harvard.edu/james-sungjin-kim/

[Linkedin] https://www.linkedin.com/in/jamessungjinkim

[Facebook] https://www.facebook.com/jamessungjin.kim

[alternative email] jamessungjin.kim@gmail.com

[12gjang]

Thanks, Kim.

In Keras, Embedding layer is doing that task.

[jskDr]

Hi gjang.

Could you let me know how to use the embedding layer for auto adjusting of
classification level? For example, the classification level should be
automatically 5 if my labels have sequence of a set (0, 1, 2, 3, 4, 5). I
am much customized to Keras yet while I have been using Sklearn for years.

Best,

On Sat, May 28, 2016 at 10:28 AM 12gjang notifications@github.com wrote:

Thanks, Kim.

In Keras, Embedding layer is doing that task.

—
You are receiving this because you commented.

Reply to this email directly, view it on GitHub
#2826 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/ACchnejtQUs3aSK2NPCE-48l_bhG0Kg0ks5qGFEFgaJpZM4Inzl5
.

Sungjin (James) Kim, PhD

Postdoc, CCB in Harvard

sungjinkim@fas.harvard.edu

[Web] http://aspuru.chem.harvard.edu/james-sungjin-kim/

[Linkedin] https://www.linkedin.com/in/jamessungjinkim

[Facebook] https://www.facebook.com/jamessungjin.kim

[alternative email] jamessungjin.kim@gmail.com