Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

benchmark: update on GRU+SVM with Dropout #50

Closed
AFAgarap opened this issue Sep 1, 2017 · 2 comments
Closed

benchmark: update on GRU+SVM with Dropout #50

AFAgarap opened this issue Sep 1, 2017 · 2 comments

Comments

@AFAgarap
Copy link
Contributor

AFAgarap commented Sep 1, 2017

Hey @hanxiao , it's me again. I saw an update in the dataset, regarding duplicate samples. I did another training using my GRU+SVM (with Dropout) model (from #8 ) on the updated dataset. Here's the result:

Epoch : 0 completed out of 100, loss : 316.9036560058594, accuracy : 0.734375
Epoch : 1 completed out of 100, loss : 201.2646026611328, accuracy : 0.83984375
Epoch : 2 completed out of 100, loss : 253.3709259033203, accuracy : 0.796875
Epoch : 3 completed out of 100, loss : 257.7744140625, accuracy : 0.8359375
Epoch : 4 completed out of 100, loss : 179.52682495117188, accuracy : 0.8671875
Epoch : 5 completed out of 100, loss : 224.97421264648438, accuracy : 0.83984375
Epoch : 6 completed out of 100, loss : 212.19381713867188, accuracy : 0.859375
Epoch : 7 completed out of 100, loss : 200.80978393554688, accuracy : 0.859375
Epoch : 8 completed out of 100, loss : 187.77052307128906, accuracy : 0.85546875
Epoch : 9 completed out of 100, loss : 190.96389770507812, accuracy : 0.86328125
Epoch : 10 completed out of 100, loss : 185.72314453125, accuracy : 0.85546875
Epoch : 11 completed out of 100, loss : 189.3765411376953, accuracy : 0.8515625
Epoch : 12 completed out of 100, loss : 130.086669921875, accuracy : 0.89453125
Epoch : 13 completed out of 100, loss : 151.38232421875, accuracy : 0.8828125
Epoch : 14 completed out of 100, loss : 159.71595764160156, accuracy : 0.88671875
Epoch : 15 completed out of 100, loss : 218.80592346191406, accuracy : 0.84375
Epoch : 16 completed out of 100, loss : 131.5895233154297, accuracy : 0.9140625
Epoch : 17 completed out of 100, loss : 162.96995544433594, accuracy : 0.8671875
Epoch : 18 completed out of 100, loss : 155.52630615234375, accuracy : 0.890625
Epoch : 19 completed out of 100, loss : 159.76901245117188, accuracy : 0.88671875
Epoch : 20 completed out of 100, loss : 137.74642944335938, accuracy : 0.890625
Epoch : 21 completed out of 100, loss : 162.48875427246094, accuracy : 0.890625
Epoch : 22 completed out of 100, loss : 179.6526336669922, accuracy : 0.8828125
Epoch : 23 completed out of 100, loss : 127.58981323242188, accuracy : 0.8984375
Epoch : 24 completed out of 100, loss : 185.6982421875, accuracy : 0.8671875
Epoch : 25 completed out of 100, loss : 159.8983612060547, accuracy : 0.8828125
Epoch : 26 completed out of 100, loss : 160.69525146484375, accuracy : 0.89453125
Epoch : 27 completed out of 100, loss : 173.42813110351562, accuracy : 0.859375
Epoch : 28 completed out of 100, loss : 166.0702667236328, accuracy : 0.87890625
Epoch : 29 completed out of 100, loss : 157.59085083007812, accuracy : 0.87109375
Epoch : 30 completed out of 100, loss : 127.72993469238281, accuracy : 0.9140625
Epoch : 31 completed out of 100, loss : 136.65415954589844, accuracy : 0.90234375
Epoch : 32 completed out of 100, loss : 172.4806365966797, accuracy : 0.8515625
Epoch : 33 completed out of 100, loss : 139.81488037109375, accuracy : 0.8984375
Epoch : 34 completed out of 100, loss : 144.55099487304688, accuracy : 0.85546875
Epoch : 35 completed out of 100, loss : 122.90949249267578, accuracy : 0.8984375
Epoch : 36 completed out of 100, loss : 150.0441131591797, accuracy : 0.890625
Epoch : 37 completed out of 100, loss : 153.2085723876953, accuracy : 0.88671875
Epoch : 38 completed out of 100, loss : 143.91455078125, accuracy : 0.8984375
Epoch : 39 completed out of 100, loss : 117.63712310791016, accuracy : 0.91796875
Epoch : 40 completed out of 100, loss : 93.80998229980469, accuracy : 0.92578125
Epoch : 41 completed out of 100, loss : 136.52537536621094, accuracy : 0.87109375
Epoch : 42 completed out of 100, loss : 137.24530029296875, accuracy : 0.90625
Epoch : 43 completed out of 100, loss : 108.73893737792969, accuracy : 0.921875
Epoch : 44 completed out of 100, loss : 106.48686218261719, accuracy : 0.9296875
Epoch : 45 completed out of 100, loss : 104.41219329833984, accuracy : 0.92578125
Epoch : 46 completed out of 100, loss : 101.19454956054688, accuracy : 0.94140625
Epoch : 47 completed out of 100, loss : 127.536376953125, accuracy : 0.91015625
Epoch : 48 completed out of 100, loss : 109.94172668457031, accuracy : 0.9296875
Epoch : 49 completed out of 100, loss : 85.25288391113281, accuracy : 0.94140625
Epoch : 50 completed out of 100, loss : 112.01800537109375, accuracy : 0.91796875
Epoch : 51 completed out of 100, loss : 107.6760482788086, accuracy : 0.91015625
Epoch : 52 completed out of 100, loss : 121.9848403930664, accuracy : 0.921875
Epoch : 53 completed out of 100, loss : 101.01953887939453, accuracy : 0.9375
Epoch : 54 completed out of 100, loss : 69.95838165283203, accuracy : 0.94921875
Epoch : 55 completed out of 100, loss : 119.3257827758789, accuracy : 0.91796875
Epoch : 56 completed out of 100, loss : 102.73481750488281, accuracy : 0.921875
Epoch : 57 completed out of 100, loss : 89.11821746826172, accuracy : 0.94921875
Epoch : 58 completed out of 100, loss : 110.71992492675781, accuracy : 0.9140625
Epoch : 59 completed out of 100, loss : 105.85194396972656, accuracy : 0.9375
Epoch : 60 completed out of 100, loss : 114.6805648803711, accuracy : 0.921875
Epoch : 61 completed out of 100, loss : 99.33323669433594, accuracy : 0.92578125
Epoch : 62 completed out of 100, loss : 128.26809692382812, accuracy : 0.90625
Epoch : 63 completed out of 100, loss : 117.59638214111328, accuracy : 0.9140625
Epoch : 64 completed out of 100, loss : 86.27313995361328, accuracy : 0.9453125
Epoch : 65 completed out of 100, loss : 114.16581726074219, accuracy : 0.92578125
Epoch : 66 completed out of 100, loss : 102.78227233886719, accuracy : 0.94921875
Epoch : 67 completed out of 100, loss : 88.23193359375, accuracy : 0.9375
Epoch : 68 completed out of 100, loss : 60.24769592285156, accuracy : 0.953125
Epoch : 69 completed out of 100, loss : 97.67103576660156, accuracy : 0.94140625
Epoch : 70 completed out of 100, loss : 86.58494567871094, accuracy : 0.91796875
Epoch : 71 completed out of 100, loss : 98.33272552490234, accuracy : 0.921875
Epoch : 72 completed out of 100, loss : 77.44849395751953, accuracy : 0.94921875
Epoch : 73 completed out of 100, loss : 114.52888488769531, accuracy : 0.9296875
Epoch : 74 completed out of 100, loss : 94.6647720336914, accuracy : 0.9453125
Epoch : 75 completed out of 100, loss : 106.62199401855469, accuracy : 0.921875
Epoch : 76 completed out of 100, loss : 116.0970230102539, accuracy : 0.91015625
Epoch : 77 completed out of 100, loss : 78.5435791015625, accuracy : 0.953125
Epoch : 78 completed out of 100, loss : 125.43787384033203, accuracy : 0.91796875
Epoch : 79 completed out of 100, loss : 112.84344482421875, accuracy : 0.9296875
Epoch : 80 completed out of 100, loss : 65.7440185546875, accuracy : 0.95703125
Epoch : 81 completed out of 100, loss : 115.66653442382812, accuracy : 0.91796875
Epoch : 82 completed out of 100, loss : 76.14566040039062, accuracy : 0.9375
Epoch : 83 completed out of 100, loss : 72.91943359375, accuracy : 0.95703125
Epoch : 84 completed out of 100, loss : 56.55884552001953, accuracy : 0.95703125
Epoch : 85 completed out of 100, loss : 87.09599304199219, accuracy : 0.93359375
Epoch : 86 completed out of 100, loss : 80.97771453857422, accuracy : 0.93359375
Epoch : 87 completed out of 100, loss : 94.14187622070312, accuracy : 0.9453125
Epoch : 88 completed out of 100, loss : 80.44708251953125, accuracy : 0.94140625
Epoch : 89 completed out of 100, loss : 52.18363952636719, accuracy : 0.96875
Epoch : 90 completed out of 100, loss : 93.15214538574219, accuracy : 0.9296875
Epoch : 91 completed out of 100, loss : 97.51387023925781, accuracy : 0.9296875
Epoch : 92 completed out of 100, loss : 82.44243621826172, accuracy : 0.9375
Epoch : 93 completed out of 100, loss : 60.52445983886719, accuracy : 0.96484375
Epoch : 94 completed out of 100, loss : 57.100406646728516, accuracy : 0.96484375
Epoch : 95 completed out of 100, loss : 89.62207794189453, accuracy : 0.94140625
Epoch : 96 completed out of 100, loss : 86.14447784423828, accuracy : 0.9375
Epoch : 97 completed out of 100, loss : 75.90823364257812, accuracy : 0.953125
Epoch : 98 completed out of 100, loss : 65.80587768554688, accuracy : 0.9609375
Epoch : 99 completed out of 100, loss : 114.98580169677734, accuracy : 0.92578125
Accuracy : 0.897300124168396

The hyper-parameters used were as follows:

BATCH_SIZE = 256
CELL_SIZE = 256
DROPOUT_P_KEEP = 0.85
EPOCHS = 100
LEARNING_RATE = 1e-3
SVM_C = 1

Trained using tf.train.AdamOptimizer(), with tf.nn.dynamic_rnn(). The source may still be found here.

The graph from TensorBoard, tracking the training (accuracy at the top, loss at the bottom):

screenshot from 2017-09-01 22-54-52

The improved accuracy may not be too much, but I suppose it's still a considerable difference, i.e. ~85.5% v. ~89.7%.

@hanxiao
Copy link
Collaborator

hanxiao commented Sep 1, 2017

Thanks again for contributing the benchmark, I saw three PRs but they all look same to me.

@hanxiao hanxiao closed this as completed in 9d9915f Sep 1, 2017
@AFAgarap
Copy link
Contributor Author

AFAgarap commented Sep 1, 2017

Thank you as well. :) I updated the READMEs myself.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants