Permalink
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
4557 lines (4556 sloc) 282 KB
Namespace(batch_size=50, data_name='Subj', dropout=0.5, epochs=60, gpu=0, log_interval=30, lr=0.0001, model_mode='non-static', save_prefix='sa-model')
Use gpu0
3413
120
Done! Tokenizing Time=1.07s, #Sentences=10000
SentimentNet(
(embedding): Embedding(21326 -> 300, float32)
(encoder): ConvolutionalEncoder(
(_convs): HybridConcurrent(
(0): HybridSequential(
(0): Conv1D(300 -> 100, kernel_size=(3,), stride=(1,))
(1): Activation(relu)
(2): HybridLambda(<lambda>)
)
(1): HybridSequential(
(0): Conv1D(300 -> 100, kernel_size=(4,), stride=(1,))
(1): Activation(relu)
(2): HybridLambda(<lambda>)
)
(2): HybridSequential(
(0): Conv1D(300 -> 100, kernel_size=(5,), stride=(1,))
(1): Activation(relu)
(2): HybridLambda(<lambda>)
)
)
)
(output): HybridSequential(
(0): Dropout(p = 0.5, axes=())
(1): Dense(None -> 2, linear)
)
)
9000 1000
[Epoch 0 Batch 30/162] avg loss 0.0148511, throughput 3.67834K wps
[Epoch 0 Batch 60/162] avg loss 0.0139595, throughput 5.98653K wps
[Epoch 0 Batch 90/162] avg loss 0.0134982, throughput 5.99197K wps
[Epoch 0 Batch 120/162] avg loss 0.0133243, throughput 5.9919K wps
[Epoch 0 Batch 150/162] avg loss 0.0127033, throughput 5.98547K wps
Begin Testing...
[Epoch 0] train avg loss 0.0135663, test acc 0.7044, test avg loss 0.589011, throughput 5.36448K wps
Observed Improvement.
Begin Testing...
[Epoch 1 Batch 30/162] avg loss 0.0121692, throughput 6.13231K wps
[Epoch 1 Batch 60/162] avg loss 0.0115759, throughput 5.98403K wps
[Epoch 1 Batch 90/162] avg loss 0.0109777, throughput 5.99254K wps
[Epoch 1 Batch 120/162] avg loss 0.0109995, throughput 5.98859K wps
[Epoch 1 Batch 150/162] avg loss 0.0111181, throughput 5.9898K wps
Begin Testing...
[Epoch 1] train avg loss 0.0112938, test acc 0.7833, test avg loss 0.51791, throughput 6.01365K wps
Observed Improvement.
Begin Testing...
[Epoch 2 Batch 30/162] avg loss 0.0104517, throughput 6.14064K wps
[Epoch 2 Batch 60/162] avg loss 0.00954018, throughput 5.98975K wps
[Epoch 2 Batch 90/162] avg loss 0.00948983, throughput 5.98206K wps
[Epoch 2 Batch 120/162] avg loss 0.0092314, throughput 5.9865K wps
[Epoch 2 Batch 150/162] avg loss 0.00865087, throughput 5.98544K wps
Begin Testing...
[Epoch 2] train avg loss 0.00941552, test acc 0.8667, test avg loss 0.42779, throughput 6.01425K wps
Observed Improvement.
Begin Testing...
[Epoch 3 Batch 30/162] avg loss 0.00829809, throughput 6.12853K wps
[Epoch 3 Batch 60/162] avg loss 0.00773297, throughput 5.97935K wps
[Epoch 3 Batch 90/162] avg loss 0.00750403, throughput 5.98398K wps
[Epoch 3 Batch 120/162] avg loss 0.00721211, throughput 5.97833K wps
[Epoch 3 Batch 150/162] avg loss 0.0071458, throughput 5.97538K wps
Begin Testing...
[Epoch 3] train avg loss 0.00754613, test acc 0.8967, test avg loss 0.348069, throughput 6.00653K wps
Observed Improvement.
Begin Testing...
[Epoch 4 Batch 30/162] avg loss 0.00661584, throughput 6.13441K wps
[Epoch 4 Batch 60/162] avg loss 0.00623737, throughput 5.984K wps
[Epoch 4 Batch 90/162] avg loss 0.00583603, throughput 5.98067K wps
[Epoch 4 Batch 120/162] avg loss 0.00569099, throughput 5.9858K wps
[Epoch 4 Batch 150/162] avg loss 0.0054304, throughput 5.97918K wps
Begin Testing...
[Epoch 4] train avg loss 0.00589697, test acc 0.8967, test avg loss 0.297476, throughput 6.00925K wps
Observed Improvement.
Begin Testing...
[Epoch 5 Batch 30/162] avg loss 0.00502297, throughput 6.12659K wps
[Epoch 5 Batch 60/162] avg loss 0.00505266, throughput 5.98009K wps
[Epoch 5 Batch 90/162] avg loss 0.00480866, throughput 5.97774K wps
[Epoch 5 Batch 120/162] avg loss 0.00467858, throughput 5.98679K wps
[Epoch 5 Batch 150/162] avg loss 0.00467072, throughput 5.98475K wps
Begin Testing...
[Epoch 5] train avg loss 0.00480382, test acc 0.9078, test avg loss 0.259937, throughput 6.00895K wps
Observed Improvement.
Begin Testing...
[Epoch 6 Batch 30/162] avg loss 0.00411757, throughput 6.12048K wps
[Epoch 6 Batch 60/162] avg loss 0.00411875, throughput 5.9731K wps
[Epoch 6 Batch 90/162] avg loss 0.00388708, throughput 5.96631K wps
[Epoch 6 Batch 120/162] avg loss 0.00388419, throughput 5.96465K wps
[Epoch 6 Batch 150/162] avg loss 0.00397215, throughput 5.95875K wps
Begin Testing...
[Epoch 6] train avg loss 0.00395787, test acc 0.9233, test avg loss 0.233139, throughput 5.99318K wps
Observed Improvement.
Begin Testing...
[Epoch 7 Batch 30/162] avg loss 0.00317561, throughput 6.11068K wps
[Epoch 7 Batch 60/162] avg loss 0.00334908, throughput 5.96658K wps
[Epoch 7 Batch 90/162] avg loss 0.00358387, throughput 5.96814K wps
[Epoch 7 Batch 120/162] avg loss 0.00291912, throughput 5.96185K wps
[Epoch 7 Batch 150/162] avg loss 0.00343882, throughput 5.96105K wps
Begin Testing...
[Epoch 7] train avg loss 0.00327052, test acc 0.9244, test avg loss 0.216116, throughput 5.99079K wps
Observed Improvement.
Begin Testing...
[Epoch 8 Batch 30/162] avg loss 0.0026954, throughput 6.11523K wps
[Epoch 8 Batch 60/162] avg loss 0.00265825, throughput 5.96585K wps
[Epoch 8 Batch 90/162] avg loss 0.00272734, throughput 5.96886K wps
[Epoch 8 Batch 120/162] avg loss 0.0028447, throughput 5.95229K wps
[Epoch 8 Batch 150/162] avg loss 0.00269444, throughput 5.96523K wps
Begin Testing...
[Epoch 8] train avg loss 0.00271098, test acc 0.9244, test avg loss 0.205859, throughput 5.9909K wps
Observed Improvement.
Begin Testing...
[Epoch 9 Batch 30/162] avg loss 0.00228392, throughput 6.12104K wps
[Epoch 9 Batch 60/162] avg loss 0.00219503, throughput 5.96398K wps
[Epoch 9 Batch 90/162] avg loss 0.00237159, throughput 5.96571K wps
[Epoch 9 Batch 120/162] avg loss 0.00250481, throughput 5.96037K wps
[Epoch 9 Batch 150/162] avg loss 0.0023406, throughput 5.94911K wps
Begin Testing...
[Epoch 9] train avg loss 0.00233163, test acc 0.9267, test avg loss 0.194939, throughput 5.98785K wps
Observed Improvement.
Begin Testing...
[Epoch 10 Batch 30/162] avg loss 0.00181935, throughput 6.12625K wps
[Epoch 10 Batch 60/162] avg loss 0.00201857, throughput 5.96518K wps
[Epoch 10 Batch 90/162] avg loss 0.00213821, throughput 5.96741K wps
[Epoch 10 Batch 120/162] avg loss 0.00190838, throughput 5.97212K wps
[Epoch 10 Batch 150/162] avg loss 0.00201291, throughput 5.96379K wps
Begin Testing...
[Epoch 10] train avg loss 0.00197262, test acc 0.9289, test avg loss 0.188062, throughput 5.99644K wps
Observed Improvement.
Begin Testing...
[Epoch 11 Batch 30/162] avg loss 0.0017659, throughput 6.12056K wps
[Epoch 11 Batch 60/162] avg loss 0.0015223, throughput 5.9659K wps
[Epoch 11 Batch 90/162] avg loss 0.00163504, throughput 5.95793K wps
[Epoch 11 Batch 120/162] avg loss 0.00171471, throughput 5.97599K wps
[Epoch 11 Batch 150/162] avg loss 0.0016471, throughput 5.97842K wps
Begin Testing...
[Epoch 11] train avg loss 0.00162396, test acc 0.9256, test avg loss 0.184605, throughput 5.99719K wps
[Epoch 12 Batch 30/162] avg loss 0.00141116, throughput 6.12831K wps
[Epoch 12 Batch 60/162] avg loss 0.0013464, throughput 5.97828K wps
[Epoch 12 Batch 90/162] avg loss 0.00139033, throughput 5.9768K wps
[Epoch 12 Batch 120/162] avg loss 0.00144433, throughput 5.97571K wps
[Epoch 12 Batch 150/162] avg loss 0.00147812, throughput 5.96856K wps
Begin Testing...
[Epoch 12] train avg loss 0.00141669, test acc 0.9311, test avg loss 0.180423, throughput 6.00221K wps
Observed Improvement.
Begin Testing...
[Epoch 13 Batch 30/162] avg loss 0.00113866, throughput 6.11193K wps
[Epoch 13 Batch 60/162] avg loss 0.00120773, throughput 5.95342K wps
[Epoch 13 Batch 90/162] avg loss 0.00104799, throughput 5.95811K wps
[Epoch 13 Batch 120/162] avg loss 0.00115915, throughput 5.96476K wps
[Epoch 13 Batch 150/162] avg loss 0.00122908, throughput 5.94958K wps
Begin Testing...
[Epoch 13] train avg loss 0.00117842, test acc 0.9244, test avg loss 0.17807, throughput 5.9858K wps
[Epoch 14 Batch 30/162] avg loss 0.000952974, throughput 6.10322K wps
[Epoch 14 Batch 60/162] avg loss 0.00110234, throughput 5.9508K wps
[Epoch 14 Batch 90/162] avg loss 0.00101482, throughput 5.96667K wps
[Epoch 14 Batch 120/162] avg loss 0.00100553, throughput 5.78686K wps
[Epoch 14 Batch 150/162] avg loss 0.000822527, throughput 5.95779K wps
Begin Testing...
[Epoch 14] train avg loss 0.00098258, test acc 0.9256, test avg loss 0.178264, throughput 5.95303K wps
[Epoch 15 Batch 30/162] avg loss 0.000944725, throughput 6.10593K wps
[Epoch 15 Batch 60/162] avg loss 0.000814463, throughput 5.95661K wps
[Epoch 15 Batch 90/162] avg loss 0.000878198, throughput 5.95949K wps
[Epoch 15 Batch 120/162] avg loss 0.000832476, throughput 5.95362K wps
[Epoch 15 Batch 150/162] avg loss 0.000851743, throughput 5.96544K wps
Begin Testing...
[Epoch 15] train avg loss 0.000856954, test acc 0.9278, test avg loss 0.175523, throughput 5.9843K wps
[Epoch 16 Batch 30/162] avg loss 0.000675976, throughput 6.10126K wps
[Epoch 16 Batch 60/162] avg loss 0.000803733, throughput 5.96252K wps
[Epoch 16 Batch 90/162] avg loss 0.000721911, throughput 5.95789K wps
[Epoch 16 Batch 120/162] avg loss 0.000688669, throughput 5.94048K wps
[Epoch 16 Batch 150/162] avg loss 0.000734368, throughput 5.96635K wps
Begin Testing...
[Epoch 16] train avg loss 0.000731528, test acc 0.9267, test avg loss 0.174136, throughput 5.98306K wps
[Epoch 17 Batch 30/162] avg loss 0.000573989, throughput 6.09485K wps
[Epoch 17 Batch 60/162] avg loss 0.000563867, throughput 5.94892K wps
[Epoch 17 Batch 90/162] avg loss 0.000547938, throughput 5.96123K wps
[Epoch 17 Batch 120/162] avg loss 0.000767132, throughput 5.95554K wps
[Epoch 17 Batch 150/162] avg loss 0.000483308, throughput 5.94571K wps
Begin Testing...
[Epoch 17] train avg loss 0.000586954, test acc 0.9289, test avg loss 0.17431, throughput 5.97897K wps
[Epoch 18 Batch 30/162] avg loss 0.000438376, throughput 6.10923K wps
[Epoch 18 Batch 60/162] avg loss 0.000427731, throughput 5.96394K wps
[Epoch 18 Batch 90/162] avg loss 0.000475297, throughput 5.96222K wps
[Epoch 18 Batch 120/162] avg loss 0.000545973, throughput 5.9554K wps
[Epoch 18 Batch 150/162] avg loss 0.000449114, throughput 5.95816K wps
Begin Testing...
[Epoch 18] train avg loss 0.000470932, test acc 0.9289, test avg loss 0.176377, throughput 5.9878K wps
[Epoch 19 Batch 30/162] avg loss 0.000381926, throughput 6.09663K wps
[Epoch 19 Batch 60/162] avg loss 0.000363285, throughput 5.95588K wps
[Epoch 19 Batch 90/162] avg loss 0.000424124, throughput 5.94552K wps
[Epoch 19 Batch 120/162] avg loss 0.000427665, throughput 5.94413K wps
[Epoch 19 Batch 150/162] avg loss 0.000549958, throughput 5.94646K wps
Begin Testing...
[Epoch 19] train avg loss 0.000425333, test acc 0.9289, test avg loss 0.177385, throughput 5.97438K wps
[Epoch 20 Batch 30/162] avg loss 0.000367171, throughput 6.11214K wps
[Epoch 20 Batch 60/162] avg loss 0.000427197, throughput 5.95344K wps
[Epoch 20 Batch 90/162] avg loss 0.000353453, throughput 5.95224K wps
[Epoch 20 Batch 120/162] avg loss 0.000326675, throughput 5.95081K wps
[Epoch 20 Batch 150/162] avg loss 0.000353028, throughput 5.95034K wps
Begin Testing...
[Epoch 20] train avg loss 0.000365795, test acc 0.9311, test avg loss 0.179965, throughput 5.98047K wps
Observed Improvement.
Begin Testing...
[Epoch 21 Batch 30/162] avg loss 0.000291702, throughput 6.08891K wps
[Epoch 21 Batch 60/162] avg loss 0.000298514, throughput 5.94923K wps
[Epoch 21 Batch 90/162] avg loss 0.00027348, throughput 5.95896K wps
[Epoch 21 Batch 120/162] avg loss 0.00029203, throughput 5.95491K wps
[Epoch 21 Batch 150/162] avg loss 0.000262523, throughput 5.95522K wps
Begin Testing...
[Epoch 21] train avg loss 0.000286074, test acc 0.9311, test avg loss 0.182445, throughput 5.97887K wps
Observed Improvement.
Begin Testing...
[Epoch 22 Batch 30/162] avg loss 0.00027019, throughput 6.10595K wps
[Epoch 22 Batch 60/162] avg loss 0.00025544, throughput 5.96157K wps
[Epoch 22 Batch 90/162] avg loss 0.000286547, throughput 5.94374K wps
[Epoch 22 Batch 120/162] avg loss 0.000248973, throughput 5.95132K wps
[Epoch 22 Batch 150/162] avg loss 0.000237931, throughput 5.96623K wps
Begin Testing...
[Epoch 22] train avg loss 0.000257256, test acc 0.9322, test avg loss 0.186346, throughput 5.98253K wps
Observed Improvement.
Begin Testing...
[Epoch 23 Batch 30/162] avg loss 0.000235487, throughput 6.09891K wps
[Epoch 23 Batch 60/162] avg loss 0.000216955, throughput 5.94767K wps
[Epoch 23 Batch 90/162] avg loss 0.000198475, throughput 5.96145K wps
[Epoch 23 Batch 120/162] avg loss 0.000198744, throughput 5.95703K wps
[Epoch 23 Batch 150/162] avg loss 0.000211186, throughput 5.95196K wps
Begin Testing...
[Epoch 23] train avg loss 0.000211378, test acc 0.9300, test avg loss 0.18505, throughput 5.97992K wps
[Epoch 24 Batch 30/162] avg loss 0.00019322, throughput 6.10102K wps
[Epoch 24 Batch 60/162] avg loss 0.00017557, throughput 5.96525K wps
[Epoch 24 Batch 90/162] avg loss 0.000179715, throughput 5.95649K wps
[Epoch 24 Batch 120/162] avg loss 0.000159544, throughput 5.95299K wps
[Epoch 24 Batch 150/162] avg loss 0.000209984, throughput 5.94569K wps
Begin Testing...
[Epoch 24] train avg loss 0.000189022, test acc 0.9289, test avg loss 0.18618, throughput 5.9805K wps
[Epoch 25 Batch 30/162] avg loss 0.000147594, throughput 6.09753K wps
[Epoch 25 Batch 60/162] avg loss 0.000210433, throughput 5.95095K wps
[Epoch 25 Batch 90/162] avg loss 0.000174914, throughput 5.95132K wps
[Epoch 25 Batch 120/162] avg loss 0.000169691, throughput 5.95374K wps
[Epoch 25 Batch 150/162] avg loss 0.000196552, throughput 5.95003K wps
Begin Testing...
[Epoch 25] train avg loss 0.000178062, test acc 0.9322, test avg loss 0.193996, throughput 5.97793K wps
Observed Improvement.
Begin Testing...
[Epoch 26 Batch 30/162] avg loss 0.000145236, throughput 6.09794K wps
[Epoch 26 Batch 60/162] avg loss 0.000138669, throughput 5.93771K wps
[Epoch 26 Batch 90/162] avg loss 0.00011052, throughput 5.94682K wps
[Epoch 26 Batch 120/162] avg loss 0.000140511, throughput 5.94459K wps
[Epoch 26 Batch 150/162] avg loss 0.000122762, throughput 5.95213K wps
Begin Testing...
[Epoch 26] train avg loss 0.000131296, test acc 0.9289, test avg loss 0.19369, throughput 5.97373K wps
[Epoch 27 Batch 30/162] avg loss 0.00011948, throughput 6.09762K wps
[Epoch 27 Batch 60/162] avg loss 0.000133627, throughput 5.94187K wps
[Epoch 27 Batch 90/162] avg loss 0.000141896, throughput 5.95848K wps
[Epoch 27 Batch 120/162] avg loss 0.000113107, throughput 5.96446K wps
[Epoch 27 Batch 150/162] avg loss 0.000124005, throughput 5.96421K wps
Begin Testing...
[Epoch 27] train avg loss 0.000125024, test acc 0.9300, test avg loss 0.198264, throughput 5.98312K wps
[Epoch 28 Batch 30/162] avg loss 0.000107032, throughput 6.10651K wps
[Epoch 28 Batch 60/162] avg loss 9.94556e-05, throughput 5.95443K wps
[Epoch 28 Batch 90/162] avg loss 9.98618e-05, throughput 5.95837K wps
[Epoch 28 Batch 120/162] avg loss 0.000104087, throughput 5.93936K wps
[Epoch 28 Batch 150/162] avg loss 0.000130415, throughput 5.97495K wps
Begin Testing...
[Epoch 28] train avg loss 0.000108015, test acc 0.9311, test avg loss 0.19885, throughput 5.98423K wps
[Epoch 29 Batch 30/162] avg loss 9.76407e-05, throughput 6.10262K wps
[Epoch 29 Batch 60/162] avg loss 9.7073e-05, throughput 5.95131K wps
[Epoch 29 Batch 90/162] avg loss 8.70635e-05, throughput 5.94372K wps
[Epoch 29 Batch 120/162] avg loss 9.5305e-05, throughput 5.95564K wps
[Epoch 29 Batch 150/162] avg loss 9.2118e-05, throughput 5.9418K wps
Begin Testing...
[Epoch 29] train avg loss 9.37369e-05, test acc 0.9300, test avg loss 0.202662, throughput 5.97665K wps
[Epoch 30 Batch 30/162] avg loss 7.08983e-05, throughput 6.09625K wps
[Epoch 30 Batch 60/162] avg loss 6.64687e-05, throughput 5.96344K wps
[Epoch 30 Batch 90/162] avg loss 7.96989e-05, throughput 5.95215K wps
[Epoch 30 Batch 120/162] avg loss 7.72881e-05, throughput 5.95089K wps
[Epoch 30 Batch 150/162] avg loss 7.15027e-05, throughput 5.95658K wps
Begin Testing...
[Epoch 30] train avg loss 7.47268e-05, test acc 0.9289, test avg loss 0.206, throughput 5.98007K wps
[Epoch 31 Batch 30/162] avg loss 6.53025e-05, throughput 6.09291K wps
[Epoch 31 Batch 60/162] avg loss 7.24442e-05, throughput 5.96714K wps
[Epoch 31 Batch 90/162] avg loss 5.49221e-05, throughput 5.95271K wps
[Epoch 31 Batch 120/162] avg loss 6.49292e-05, throughput 5.9598K wps
[Epoch 31 Batch 150/162] avg loss 5.98095e-05, throughput 5.94371K wps
Begin Testing...
[Epoch 31] train avg loss 6.32211e-05, test acc 0.9322, test avg loss 0.207601, throughput 5.97906K wps
Observed Improvement.
Begin Testing...
[Epoch 32 Batch 30/162] avg loss 5.12301e-05, throughput 6.08462K wps
[Epoch 32 Batch 60/162] avg loss 8.95378e-05, throughput 5.95058K wps
[Epoch 32 Batch 90/162] avg loss 6.1442e-05, throughput 5.95562K wps
[Epoch 32 Batch 120/162] avg loss 5.50545e-05, throughput 5.94592K wps
[Epoch 32 Batch 150/162] avg loss 5.31053e-05, throughput 5.95611K wps
Begin Testing...
[Epoch 32] train avg loss 6.31393e-05, test acc 0.9311, test avg loss 0.212667, throughput 5.97661K wps
[Epoch 33 Batch 30/162] avg loss 5.46469e-05, throughput 6.10831K wps
[Epoch 33 Batch 60/162] avg loss 4.89296e-05, throughput 5.96396K wps
[Epoch 33 Batch 90/162] avg loss 4.94582e-05, throughput 5.94646K wps
[Epoch 33 Batch 120/162] avg loss 5.21592e-05, throughput 5.94368K wps
[Epoch 33 Batch 150/162] avg loss 5.10227e-05, throughput 5.96334K wps
Begin Testing...
[Epoch 33] train avg loss 5.11216e-05, test acc 0.9300, test avg loss 0.211789, throughput 5.98168K wps
[Epoch 34 Batch 30/162] avg loss 5.12613e-05, throughput 6.10783K wps
[Epoch 34 Batch 60/162] avg loss 4.65867e-05, throughput 5.95141K wps
[Epoch 34 Batch 90/162] avg loss 4.3913e-05, throughput 5.96018K wps
[Epoch 34 Batch 120/162] avg loss 6.1189e-05, throughput 5.95572K wps
[Epoch 34 Batch 150/162] avg loss 4.14755e-05, throughput 5.94763K wps
Begin Testing...
[Epoch 34] train avg loss 4.79303e-05, test acc 0.9322, test avg loss 0.21915, throughput 5.98094K wps
Observed Improvement.
Begin Testing...
[Epoch 35 Batch 30/162] avg loss 3.67393e-05, throughput 6.10577K wps
[Epoch 35 Batch 60/162] avg loss 4.49783e-05, throughput 5.94556K wps
[Epoch 35 Batch 90/162] avg loss 4.67867e-05, throughput 5.95718K wps
[Epoch 35 Batch 120/162] avg loss 3.64948e-05, throughput 5.9403K wps
[Epoch 35 Batch 150/162] avg loss 4.41019e-05, throughput 5.94011K wps
Begin Testing...
[Epoch 35] train avg loss 4.17669e-05, test acc 0.9322, test avg loss 0.221758, throughput 5.97452K wps
Observed Improvement.
Begin Testing...
[Epoch 36 Batch 30/162] avg loss 3.46075e-05, throughput 6.09488K wps
[Epoch 36 Batch 60/162] avg loss 4.43862e-05, throughput 5.95149K wps
[Epoch 36 Batch 90/162] avg loss 4.00549e-05, throughput 5.95164K wps
[Epoch 36 Batch 120/162] avg loss 4.02688e-05, throughput 5.9644K wps
[Epoch 36 Batch 150/162] avg loss 3.62446e-05, throughput 5.95515K wps
Begin Testing...
[Epoch 36] train avg loss 3.87508e-05, test acc 0.9300, test avg loss 0.222303, throughput 5.98087K wps
[Epoch 37 Batch 30/162] avg loss 2.62034e-05, throughput 6.08658K wps
[Epoch 37 Batch 60/162] avg loss 3.16566e-05, throughput 5.94976K wps
[Epoch 37 Batch 90/162] avg loss 3.94312e-05, throughput 5.95309K wps
[Epoch 37 Batch 120/162] avg loss 3.5279e-05, throughput 5.94707K wps
[Epoch 37 Batch 150/162] avg loss 3.31437e-05, throughput 5.94569K wps
Begin Testing...
[Epoch 37] train avg loss 3.29384e-05, test acc 0.9322, test avg loss 0.226268, throughput 5.97432K wps
Observed Improvement.
Begin Testing...
[Epoch 38 Batch 30/162] avg loss 3.21402e-05, throughput 6.09982K wps
[Epoch 38 Batch 60/162] avg loss 2.86965e-05, throughput 5.95737K wps
[Epoch 38 Batch 90/162] avg loss 3.52172e-05, throughput 5.94998K wps
[Epoch 38 Batch 120/162] avg loss 2.74696e-05, throughput 5.95415K wps
[Epoch 38 Batch 150/162] avg loss 2.66803e-05, throughput 5.96047K wps
Begin Testing...
[Epoch 38] train avg loss 3.049e-05, test acc 0.9300, test avg loss 0.231828, throughput 5.98055K wps
[Epoch 39 Batch 30/162] avg loss 2.26027e-05, throughput 6.09547K wps
[Epoch 39 Batch 60/162] avg loss 2.49652e-05, throughput 5.94437K wps
[Epoch 39 Batch 90/162] avg loss 2.58957e-05, throughput 5.93901K wps
[Epoch 39 Batch 120/162] avg loss 2.35916e-05, throughput 5.93942K wps
[Epoch 39 Batch 150/162] avg loss 2.52175e-05, throughput 5.9482K wps
Begin Testing...
[Epoch 39] train avg loss 2.49742e-05, test acc 0.9311, test avg loss 0.2337, throughput 5.97097K wps
[Epoch 40 Batch 30/162] avg loss 2.90263e-05, throughput 6.1051K wps
[Epoch 40 Batch 60/162] avg loss 3.05461e-05, throughput 5.95291K wps
[Epoch 40 Batch 90/162] avg loss 2.12375e-05, throughput 5.94513K wps
[Epoch 40 Batch 120/162] avg loss 2.37668e-05, throughput 5.94451K wps
[Epoch 40 Batch 150/162] avg loss 2.30785e-05, throughput 5.94312K wps
Begin Testing...
[Epoch 40] train avg loss 2.51892e-05, test acc 0.9311, test avg loss 0.238128, throughput 5.97403K wps
[Epoch 41 Batch 30/162] avg loss 3.34884e-05, throughput 6.08716K wps
[Epoch 41 Batch 60/162] avg loss 1.83006e-05, throughput 5.94669K wps
[Epoch 41 Batch 90/162] avg loss 2.15677e-05, throughput 5.937K wps
[Epoch 41 Batch 120/162] avg loss 2.00261e-05, throughput 5.91421K wps
[Epoch 41 Batch 150/162] avg loss 2.22158e-05, throughput 5.94049K wps
Begin Testing...
[Epoch 41] train avg loss 2.32496e-05, test acc 0.9333, test avg loss 0.241269, throughput 5.9644K wps
Observed Improvement.
Begin Testing...
[Epoch 42 Batch 30/162] avg loss 1.85581e-05, throughput 6.0975K wps
[Epoch 42 Batch 60/162] avg loss 1.81096e-05, throughput 5.95133K wps
[Epoch 42 Batch 90/162] avg loss 1.88573e-05, throughput 5.95536K wps
[Epoch 42 Batch 120/162] avg loss 2.12554e-05, throughput 5.94831K wps
[Epoch 42 Batch 150/162] avg loss 1.86156e-05, throughput 5.95263K wps
Begin Testing...
[Epoch 42] train avg loss 1.87477e-05, test acc 0.9322, test avg loss 0.246021, throughput 5.97772K wps
[Epoch 43 Batch 30/162] avg loss 1.7297e-05, throughput 6.08819K wps
[Epoch 43 Batch 60/162] avg loss 1.90735e-05, throughput 5.92877K wps
[Epoch 43 Batch 90/162] avg loss 1.73233e-05, throughput 5.93345K wps
[Epoch 43 Batch 120/162] avg loss 1.57851e-05, throughput 5.94132K wps
[Epoch 43 Batch 150/162] avg loss 1.53249e-05, throughput 5.94398K wps
Begin Testing...
[Epoch 43] train avg loss 1.74427e-05, test acc 0.9322, test avg loss 0.245878, throughput 5.96389K wps
[Epoch 44 Batch 30/162] avg loss 1.46984e-05, throughput 6.07827K wps
[Epoch 44 Batch 60/162] avg loss 1.57596e-05, throughput 5.93707K wps
[Epoch 44 Batch 90/162] avg loss 1.99001e-05, throughput 5.94883K wps
[Epoch 44 Batch 120/162] avg loss 1.34666e-05, throughput 5.95263K wps
[Epoch 44 Batch 150/162] avg loss 1.10531e-05, throughput 5.94043K wps
Begin Testing...
[Epoch 44] train avg loss 1.49056e-05, test acc 0.9322, test avg loss 0.250814, throughput 5.96922K wps
[Epoch 45 Batch 30/162] avg loss 1.53128e-05, throughput 6.07555K wps
[Epoch 45 Batch 60/162] avg loss 1.08353e-05, throughput 5.92789K wps
[Epoch 45 Batch 90/162] avg loss 1.50722e-05, throughput 5.93351K wps
[Epoch 45 Batch 120/162] avg loss 1.48012e-05, throughput 5.94145K wps
[Epoch 45 Batch 150/162] avg loss 1.5387e-05, throughput 5.93645K wps
Begin Testing...
[Epoch 45] train avg loss 1.43371e-05, test acc 0.9256, test avg loss 0.250696, throughput 5.95998K wps
[Epoch 46 Batch 30/162] avg loss 1.35553e-05, throughput 6.0851K wps
[Epoch 46 Batch 60/162] avg loss 1.08345e-05, throughput 5.94578K wps
[Epoch 46 Batch 90/162] avg loss 1.53067e-05, throughput 5.95439K wps
[Epoch 46 Batch 120/162] avg loss 1.6858e-05, throughput 5.95423K wps
[Epoch 46 Batch 150/162] avg loss 1.52229e-05, throughput 5.95355K wps
Begin Testing...
[Epoch 46] train avg loss 1.39403e-05, test acc 0.9300, test avg loss 0.258201, throughput 5.97578K wps
[Epoch 47 Batch 30/162] avg loss 1.69273e-05, throughput 6.08185K wps
[Epoch 47 Batch 60/162] avg loss 1.22272e-05, throughput 5.92674K wps
[Epoch 47 Batch 90/162] avg loss 9.60368e-06, throughput 5.94488K wps
[Epoch 47 Batch 120/162] avg loss 1.70839e-05, throughput 5.94843K wps
[Epoch 47 Batch 150/162] avg loss 1.266e-05, throughput 5.95061K wps
Begin Testing...
[Epoch 47] train avg loss 1.36213e-05, test acc 0.9289, test avg loss 0.258901, throughput 5.96746K wps
[Epoch 48 Batch 30/162] avg loss 1.38953e-05, throughput 6.0891K wps
[Epoch 48 Batch 60/162] avg loss 1.17013e-05, throughput 5.95726K wps
[Epoch 48 Batch 90/162] avg loss 1.1564e-05, throughput 5.94816K wps
[Epoch 48 Batch 120/162] avg loss 9.92422e-06, throughput 5.95313K wps
[Epoch 48 Batch 150/162] avg loss 9.74795e-06, throughput 5.94746K wps
Begin Testing...
[Epoch 48] train avg loss 1.11654e-05, test acc 0.9278, test avg loss 0.262303, throughput 5.97666K wps
[Epoch 49 Batch 30/162] avg loss 7.47194e-06, throughput 6.08647K wps
[Epoch 49 Batch 60/162] avg loss 1.00349e-05, throughput 5.94835K wps
[Epoch 49 Batch 90/162] avg loss 8.92189e-06, throughput 5.94386K wps
[Epoch 49 Batch 120/162] avg loss 7.04314e-06, throughput 5.937K wps
[Epoch 49 Batch 150/162] avg loss 9.26589e-06, throughput 5.94211K wps
Begin Testing...
[Epoch 49] train avg loss 8.35227e-06, test acc 0.9278, test avg loss 0.262647, throughput 5.9686K wps
[Epoch 50 Batch 30/162] avg loss 6.26426e-06, throughput 6.09547K wps
[Epoch 50 Batch 60/162] avg loss 6.62392e-06, throughput 5.95058K wps
[Epoch 50 Batch 90/162] avg loss 7.63582e-06, throughput 5.95342K wps
[Epoch 50 Batch 120/162] avg loss 6.55499e-06, throughput 5.957K wps
[Epoch 50 Batch 150/162] avg loss 8.20306e-06, throughput 5.94873K wps
Begin Testing...
[Epoch 50] train avg loss 7.34583e-06, test acc 0.9300, test avg loss 0.272521, throughput 5.9787K wps
[Epoch 51 Batch 30/162] avg loss 7.1681e-06, throughput 6.11295K wps
[Epoch 51 Batch 60/162] avg loss 7.81347e-06, throughput 5.94211K wps
[Epoch 51 Batch 90/162] avg loss 8.49518e-06, throughput 5.94953K wps
[Epoch 51 Batch 120/162] avg loss 8.28537e-06, throughput 5.94575K wps
[Epoch 51 Batch 150/162] avg loss 6.32891e-06, throughput 5.94697K wps
Begin Testing...
[Epoch 51] train avg loss 7.73428e-06, test acc 0.9289, test avg loss 0.270386, throughput 5.97644K wps
[Epoch 52 Batch 30/162] avg loss 5.52383e-06, throughput 6.08413K wps
[Epoch 52 Batch 60/162] avg loss 7.06775e-06, throughput 5.94898K wps
[Epoch 52 Batch 90/162] avg loss 5.49608e-06, throughput 5.9513K wps
[Epoch 52 Batch 120/162] avg loss 9.28387e-06, throughput 5.95114K wps
[Epoch 52 Batch 150/162] avg loss 7.10179e-06, throughput 5.95362K wps
Begin Testing...
[Epoch 52] train avg loss 6.67358e-06, test acc 0.9289, test avg loss 0.274122, throughput 5.97503K wps
[Epoch 53 Batch 30/162] avg loss 4.59926e-06, throughput 6.08006K wps
[Epoch 53 Batch 60/162] avg loss 6.01056e-06, throughput 5.93984K wps
[Epoch 53 Batch 90/162] avg loss 5.85443e-06, throughput 5.94614K wps
[Epoch 53 Batch 120/162] avg loss 6.13339e-06, throughput 5.9482K wps
[Epoch 53 Batch 150/162] avg loss 5.80566e-06, throughput 5.94391K wps
Begin Testing...
[Epoch 53] train avg loss 5.75271e-06, test acc 0.9311, test avg loss 0.277628, throughput 5.96941K wps
[Epoch 54 Batch 30/162] avg loss 4.53747e-06, throughput 6.10719K wps
[Epoch 54 Batch 60/162] avg loss 6.29102e-06, throughput 5.95339K wps
[Epoch 54 Batch 90/162] avg loss 5.4417e-06, throughput 5.94448K wps
[Epoch 54 Batch 120/162] avg loss 5.77367e-06, throughput 5.93256K wps
[Epoch 54 Batch 150/162] avg loss 8.84968e-06, throughput 5.94925K wps
Begin Testing...
[Epoch 54] train avg loss 6.10246e-06, test acc 0.9300, test avg loss 0.276599, throughput 5.97601K wps
[Epoch 55 Batch 30/162] avg loss 4.95241e-06, throughput 6.09477K wps
[Epoch 55 Batch 60/162] avg loss 4.77615e-06, throughput 5.95007K wps
[Epoch 55 Batch 90/162] avg loss 8.77102e-06, throughput 5.9366K wps
[Epoch 55 Batch 120/162] avg loss 6.35025e-06, throughput 5.94696K wps
[Epoch 55 Batch 150/162] avg loss 5.05123e-06, throughput 5.9581K wps
Begin Testing...
[Epoch 55] train avg loss 6.13849e-06, test acc 0.9289, test avg loss 0.282286, throughput 5.97521K wps
[Epoch 56 Batch 30/162] avg loss 4.31623e-06, throughput 6.09765K wps
[Epoch 56 Batch 60/162] avg loss 4.22796e-06, throughput 5.94488K wps
[Epoch 56 Batch 90/162] avg loss 5.59047e-06, throughput 5.94346K wps
[Epoch 56 Batch 120/162] avg loss 4.73081e-06, throughput 5.95787K wps
[Epoch 56 Batch 150/162] avg loss 5.06023e-06, throughput 5.95128K wps
Begin Testing...
[Epoch 56] train avg loss 5.0399e-06, test acc 0.9267, test avg loss 0.286593, throughput 5.97487K wps
[Epoch 57 Batch 30/162] avg loss 4.55627e-06, throughput 6.08843K wps
[Epoch 57 Batch 60/162] avg loss 4.34908e-06, throughput 5.94782K wps
[Epoch 57 Batch 90/162] avg loss 3.70256e-06, throughput 5.95624K wps
[Epoch 57 Batch 120/162] avg loss 6.41807e-06, throughput 5.94783K wps
[Epoch 57 Batch 150/162] avg loss 5.16301e-06, throughput 5.9503K wps
Begin Testing...
[Epoch 57] train avg loss 4.72939e-06, test acc 0.9289, test avg loss 0.285997, throughput 5.97482K wps
[Epoch 58 Batch 30/162] avg loss 3.73365e-06, throughput 6.08865K wps
[Epoch 58 Batch 60/162] avg loss 4.823e-06, throughput 5.939K wps
[Epoch 58 Batch 90/162] avg loss 4.70248e-06, throughput 5.94231K wps
[Epoch 58 Batch 120/162] avg loss 4.30351e-06, throughput 5.94243K wps
[Epoch 58 Batch 150/162] avg loss 4.62603e-06, throughput 5.95573K wps
Begin Testing...
[Epoch 58] train avg loss 4.4241e-06, test acc 0.9311, test avg loss 0.281892, throughput 5.97253K wps
[Epoch 59 Batch 30/162] avg loss 3.31475e-06, throughput 6.09082K wps
[Epoch 59 Batch 60/162] avg loss 3.56012e-06, throughput 5.95585K wps
[Epoch 59 Batch 90/162] avg loss 3.05715e-06, throughput 5.9539K wps
[Epoch 59 Batch 120/162] avg loss 2.84767e-06, throughput 5.95786K wps
[Epoch 59 Batch 150/162] avg loss 5.60371e-06, throughput 5.95884K wps
Begin Testing...
[Epoch 59] train avg loss 3.62998e-06, test acc 0.9267, test avg loss 0.288632, throughput 5.98013K wps
Test loss 0.296791, test acc 0.9080
Total time cost 342.13s
9000 1000
[Epoch 0 Batch 30/162] avg loss 0.0154891, throughput 5.67201K wps
[Epoch 0 Batch 60/162] avg loss 0.0142749, throughput 5.92701K wps
[Epoch 0 Batch 90/162] avg loss 0.0137888, throughput 5.9319K wps
[Epoch 0 Batch 120/162] avg loss 0.0134484, throughput 5.93103K wps
[Epoch 0 Batch 150/162] avg loss 0.0130521, throughput 5.93712K wps
Begin Testing...
[Epoch 0] train avg loss 0.0139161, test acc 0.6956, test avg loss 0.588623, throughput 5.88172K wps
Observed Improvement.
Begin Testing...
[Epoch 1 Batch 30/162] avg loss 0.0122352, throughput 6.09968K wps
[Epoch 1 Batch 60/162] avg loss 0.0120234, throughput 5.95212K wps
[Epoch 1 Batch 90/162] avg loss 0.0116036, throughput 5.95095K wps
[Epoch 1 Batch 120/162] avg loss 0.0110907, throughput 5.95318K wps
[Epoch 1 Batch 150/162] avg loss 0.0109659, throughput 5.95724K wps
Begin Testing...
[Epoch 1] train avg loss 0.0115238, test acc 0.7778, test avg loss 0.524253, throughput 5.98052K wps
Observed Improvement.
Begin Testing...
[Epoch 2 Batch 30/162] avg loss 0.0102144, throughput 6.10202K wps
[Epoch 2 Batch 60/162] avg loss 0.00988012, throughput 5.95242K wps
[Epoch 2 Batch 90/162] avg loss 0.00963291, throughput 5.95937K wps
[Epoch 2 Batch 120/162] avg loss 0.00921373, throughput 5.94709K wps
[Epoch 2 Batch 150/162] avg loss 0.00923887, throughput 5.95109K wps
Begin Testing...
[Epoch 2] train avg loss 0.00959241, test acc 0.8522, test avg loss 0.435106, throughput 5.9791K wps
Observed Improvement.
Begin Testing...
[Epoch 3 Batch 30/162] avg loss 0.00821314, throughput 6.10024K wps
[Epoch 3 Batch 60/162] avg loss 0.00805927, throughput 5.93875K wps
[Epoch 3 Batch 90/162] avg loss 0.00768125, throughput 5.95038K wps
[Epoch 3 Batch 120/162] avg loss 0.00754657, throughput 5.95561K wps
[Epoch 3 Batch 150/162] avg loss 0.00721204, throughput 5.95341K wps
Begin Testing...
[Epoch 3] train avg loss 0.00767953, test acc 0.8789, test avg loss 0.359105, throughput 5.97668K wps
Observed Improvement.
Begin Testing...
[Epoch 4 Batch 30/162] avg loss 0.006653, throughput 6.09537K wps
[Epoch 4 Batch 60/162] avg loss 0.00612625, throughput 5.95326K wps
[Epoch 4 Batch 90/162] avg loss 0.00610774, throughput 5.94322K wps
[Epoch 4 Batch 120/162] avg loss 0.00596284, throughput 5.94926K wps
[Epoch 4 Batch 150/162] avg loss 0.00583468, throughput 5.95745K wps
Begin Testing...
[Epoch 4] train avg loss 0.00611273, test acc 0.9044, test avg loss 0.298843, throughput 5.97767K wps
Observed Improvement.
Begin Testing...
[Epoch 5 Batch 30/162] avg loss 0.00520781, throughput 6.10356K wps
[Epoch 5 Batch 60/162] avg loss 0.00501935, throughput 5.95873K wps
[Epoch 5 Batch 90/162] avg loss 0.00474933, throughput 5.95407K wps
[Epoch 5 Batch 120/162] avg loss 0.00489081, throughput 5.95041K wps
[Epoch 5 Batch 150/162] avg loss 0.00462712, throughput 5.95426K wps
Begin Testing...
[Epoch 5] train avg loss 0.00488486, test acc 0.9067, test avg loss 0.264349, throughput 5.9815K wps
Observed Improvement.
Begin Testing...
[Epoch 6 Batch 30/162] avg loss 0.0043522, throughput 6.08719K wps
[Epoch 6 Batch 60/162] avg loss 0.00414096, throughput 5.94418K wps
[Epoch 6 Batch 90/162] avg loss 0.00409525, throughput 5.94582K wps
[Epoch 6 Batch 120/162] avg loss 0.00394868, throughput 5.95815K wps
[Epoch 6 Batch 150/162] avg loss 0.00385838, throughput 5.9492K wps
Begin Testing...
[Epoch 6] train avg loss 0.00406673, test acc 0.9211, test avg loss 0.243066, throughput 5.9745K wps
Observed Improvement.
Begin Testing...
[Epoch 7 Batch 30/162] avg loss 0.00366403, throughput 6.09475K wps
[Epoch 7 Batch 60/162] avg loss 0.00332394, throughput 5.94523K wps
[Epoch 7 Batch 90/162] avg loss 0.00354354, throughput 5.95062K wps
[Epoch 7 Batch 120/162] avg loss 0.00321995, throughput 5.95815K wps
[Epoch 7 Batch 150/162] avg loss 0.0031355, throughput 5.94025K wps
Begin Testing...
[Epoch 7] train avg loss 0.00337345, test acc 0.9200, test avg loss 0.229754, throughput 5.97494K wps
[Epoch 8 Batch 30/162] avg loss 0.00304875, throughput 6.09519K wps
[Epoch 8 Batch 60/162] avg loss 0.0028098, throughput 5.94447K wps
[Epoch 8 Batch 90/162] avg loss 0.00281048, throughput 5.94698K wps
[Epoch 8 Batch 120/162] avg loss 0.0028645, throughput 5.94549K wps
[Epoch 8 Batch 150/162] avg loss 0.00297454, throughput 5.93776K wps
Begin Testing...
[Epoch 8] train avg loss 0.00286903, test acc 0.9178, test avg loss 0.218072, throughput 5.97077K wps
[Epoch 9 Batch 30/162] avg loss 0.00252038, throughput 6.10226K wps
[Epoch 9 Batch 60/162] avg loss 0.00234564, throughput 5.94523K wps
[Epoch 9 Batch 90/162] avg loss 0.0023716, throughput 5.93895K wps
[Epoch 9 Batch 120/162] avg loss 0.00217271, throughput 5.9438K wps
[Epoch 9 Batch 150/162] avg loss 0.00227966, throughput 5.9487K wps
Begin Testing...
[Epoch 9] train avg loss 0.00232198, test acc 0.9178, test avg loss 0.211471, throughput 5.97237K wps
[Epoch 10 Batch 30/162] avg loss 0.00202154, throughput 6.09856K wps
[Epoch 10 Batch 60/162] avg loss 0.00198218, throughput 5.95571K wps
[Epoch 10 Batch 90/162] avg loss 0.00223152, throughput 5.94103K wps
[Epoch 10 Batch 120/162] avg loss 0.00209095, throughput 5.93763K wps
[Epoch 10 Batch 150/162] avg loss 0.00181649, throughput 5.9407K wps
Begin Testing...
[Epoch 10] train avg loss 0.00200634, test acc 0.9178, test avg loss 0.208227, throughput 5.97074K wps
[Epoch 11 Batch 30/162] avg loss 0.00195586, throughput 6.08866K wps
[Epoch 11 Batch 60/162] avg loss 0.00188429, throughput 5.93286K wps
[Epoch 11 Batch 90/162] avg loss 0.00154179, throughput 5.94459K wps
[Epoch 11 Batch 120/162] avg loss 0.00151675, throughput 5.95015K wps
[Epoch 11 Batch 150/162] avg loss 0.00160691, throughput 5.93376K wps
Begin Testing...
[Epoch 11] train avg loss 0.00168507, test acc 0.9222, test avg loss 0.20694, throughput 5.96679K wps
Observed Improvement.
Begin Testing...
[Epoch 12 Batch 30/162] avg loss 0.00144891, throughput 6.09676K wps
[Epoch 12 Batch 60/162] avg loss 0.001416, throughput 5.94382K wps
[Epoch 12 Batch 90/162] avg loss 0.00132311, throughput 5.93985K wps
[Epoch 12 Batch 120/162] avg loss 0.00136499, throughput 5.95711K wps
[Epoch 12 Batch 150/162] avg loss 0.00145854, throughput 5.94781K wps
Begin Testing...
[Epoch 12] train avg loss 0.00141128, test acc 0.9267, test avg loss 0.210357, throughput 5.97414K wps
Observed Improvement.
Begin Testing...
[Epoch 13 Batch 30/162] avg loss 0.00127315, throughput 6.09793K wps
[Epoch 13 Batch 60/162] avg loss 0.00110775, throughput 5.9317K wps
[Epoch 13 Batch 90/162] avg loss 0.00106129, throughput 5.92148K wps
[Epoch 13 Batch 120/162] avg loss 0.00112778, throughput 5.9467K wps
[Epoch 13 Batch 150/162] avg loss 0.00116968, throughput 5.94396K wps
Begin Testing...
[Epoch 13] train avg loss 0.00115994, test acc 0.9178, test avg loss 0.202322, throughput 5.96475K wps
[Epoch 14 Batch 30/162] avg loss 0.00102452, throughput 6.09709K wps
[Epoch 14 Batch 60/162] avg loss 0.000948119, throughput 5.94648K wps
[Epoch 14 Batch 90/162] avg loss 0.00115958, throughput 5.96534K wps
[Epoch 14 Batch 120/162] avg loss 0.000945313, throughput 5.93964K wps
[Epoch 14 Batch 150/162] avg loss 0.00101972, throughput 5.94872K wps
Begin Testing...
[Epoch 14] train avg loss 0.00102055, test acc 0.9244, test avg loss 0.2047, throughput 5.97738K wps
[Epoch 15 Batch 30/162] avg loss 0.000773485, throughput 6.09904K wps
[Epoch 15 Batch 60/162] avg loss 0.0008829, throughput 5.94989K wps
[Epoch 15 Batch 90/162] avg loss 0.000801319, throughput 5.96108K wps
[Epoch 15 Batch 120/162] avg loss 0.000760051, throughput 5.95379K wps
[Epoch 15 Batch 150/162] avg loss 0.000809382, throughput 5.93954K wps
Begin Testing...
[Epoch 15] train avg loss 0.000801972, test acc 0.9267, test avg loss 0.205478, throughput 5.97795K wps
Observed Improvement.
Begin Testing...
[Epoch 16 Batch 30/162] avg loss 0.000762704, throughput 6.08185K wps
[Epoch 16 Batch 60/162] avg loss 0.000656609, throughput 5.95157K wps
[Epoch 16 Batch 90/162] avg loss 0.000703933, throughput 5.95879K wps
[Epoch 16 Batch 120/162] avg loss 0.00060985, throughput 5.95317K wps
[Epoch 16 Batch 150/162] avg loss 0.000719739, throughput 5.94049K wps
Begin Testing...
[Epoch 16] train avg loss 0.000691236, test acc 0.9233, test avg loss 0.214014, throughput 5.97437K wps
[Epoch 17 Batch 30/162] avg loss 0.000642518, throughput 6.10575K wps
[Epoch 17 Batch 60/162] avg loss 0.000581463, throughput 5.9509K wps
[Epoch 17 Batch 90/162] avg loss 0.000562359, throughput 5.96167K wps
[Epoch 17 Batch 120/162] avg loss 0.000522018, throughput 5.95091K wps
[Epoch 17 Batch 150/162] avg loss 0.000637737, throughput 5.95158K wps
Begin Testing...
[Epoch 17] train avg loss 0.000585856, test acc 0.9267, test avg loss 0.212806, throughput 5.98145K wps
Observed Improvement.
Begin Testing...
[Epoch 18 Batch 30/162] avg loss 0.000502585, throughput 6.08068K wps
[Epoch 18 Batch 60/162] avg loss 0.000465533, throughput 5.94622K wps
[Epoch 18 Batch 90/162] avg loss 0.00050191, throughput 5.94904K wps
[Epoch 18 Batch 120/162] avg loss 0.000459877, throughput 5.94131K wps
[Epoch 18 Batch 150/162] avg loss 0.000476007, throughput 5.95579K wps
Begin Testing...
[Epoch 18] train avg loss 0.000476856, test acc 0.9244, test avg loss 0.214834, throughput 5.97261K wps
[Epoch 19 Batch 30/162] avg loss 0.000369025, throughput 6.11182K wps
[Epoch 19 Batch 60/162] avg loss 0.00036476, throughput 5.96001K wps
[Epoch 19 Batch 90/162] avg loss 0.000444969, throughput 5.94555K wps
[Epoch 19 Batch 120/162] avg loss 0.000429584, throughput 5.95853K wps
[Epoch 19 Batch 150/162] avg loss 0.000411469, throughput 5.95337K wps
Begin Testing...
[Epoch 19] train avg loss 0.000410349, test acc 0.9289, test avg loss 0.214104, throughput 5.98147K wps
Observed Improvement.
Begin Testing...
[Epoch 20 Batch 30/162] avg loss 0.000394032, throughput 6.08854K wps
[Epoch 20 Batch 60/162] avg loss 0.000369971, throughput 5.93599K wps
[Epoch 20 Batch 90/162] avg loss 0.000282095, throughput 5.94488K wps
[Epoch 20 Batch 120/162] avg loss 0.000388065, throughput 5.95469K wps
[Epoch 20 Batch 150/162] avg loss 0.000335028, throughput 5.94214K wps
Begin Testing...
[Epoch 20] train avg loss 0.000350028, test acc 0.9289, test avg loss 0.216231, throughput 5.96988K wps
Observed Improvement.
Begin Testing...
[Epoch 21 Batch 30/162] avg loss 0.000273442, throughput 6.09448K wps
[Epoch 21 Batch 60/162] avg loss 0.000334369, throughput 5.93249K wps
[Epoch 21 Batch 90/162] avg loss 0.000288574, throughput 5.94659K wps
[Epoch 21 Batch 120/162] avg loss 0.000258549, throughput 5.93676K wps
[Epoch 21 Batch 150/162] avg loss 0.000266604, throughput 5.94647K wps
Begin Testing...
[Epoch 21] train avg loss 0.000284764, test acc 0.9278, test avg loss 0.219245, throughput 5.96864K wps
[Epoch 22 Batch 30/162] avg loss 0.000245959, throughput 6.08935K wps
[Epoch 22 Batch 60/162] avg loss 0.000338585, throughput 5.94621K wps
[Epoch 22 Batch 90/162] avg loss 0.000232398, throughput 5.94962K wps
[Epoch 22 Batch 120/162] avg loss 0.000246913, throughput 5.93669K wps
[Epoch 22 Batch 150/162] avg loss 0.000218818, throughput 5.92671K wps
Begin Testing...
[Epoch 22] train avg loss 0.000256773, test acc 0.9233, test avg loss 0.224524, throughput 5.96804K wps
[Epoch 23 Batch 30/162] avg loss 0.000201531, throughput 6.09838K wps
[Epoch 23 Batch 60/162] avg loss 0.000208554, throughput 5.95286K wps
[Epoch 23 Batch 90/162] avg loss 0.000254908, throughput 5.94421K wps
[Epoch 23 Batch 120/162] avg loss 0.000210098, throughput 5.94541K wps
[Epoch 23 Batch 150/162] avg loss 0.000250751, throughput 5.9516K wps
Begin Testing...
[Epoch 23] train avg loss 0.000227934, test acc 0.9267, test avg loss 0.225468, throughput 5.97625K wps
[Epoch 24 Batch 30/162] avg loss 0.000189051, throughput 6.10091K wps
[Epoch 24 Batch 60/162] avg loss 0.000170713, throughput 5.95676K wps
[Epoch 24 Batch 90/162] avg loss 0.000169855, throughput 5.94932K wps
[Epoch 24 Batch 120/162] avg loss 0.000204405, throughput 5.94709K wps
[Epoch 24 Batch 150/162] avg loss 0.000146182, throughput 5.95441K wps
Begin Testing...
[Epoch 24] train avg loss 0.000178889, test acc 0.9244, test avg loss 0.232557, throughput 5.9792K wps
[Epoch 25 Batch 30/162] avg loss 0.00017165, throughput 6.092K wps
[Epoch 25 Batch 60/162] avg loss 0.000140042, throughput 5.93981K wps
[Epoch 25 Batch 90/162] avg loss 0.000176213, throughput 5.95911K wps
[Epoch 25 Batch 120/162] avg loss 0.000149368, throughput 5.95613K wps
[Epoch 25 Batch 150/162] avg loss 0.000151499, throughput 5.95797K wps
Begin Testing...
[Epoch 25] train avg loss 0.000157506, test acc 0.9233, test avg loss 0.2376, throughput 5.97766K wps
[Epoch 26 Batch 30/162] avg loss 0.000107234, throughput 6.10061K wps
[Epoch 26 Batch 60/162] avg loss 0.00012831, throughput 5.94391K wps
[Epoch 26 Batch 90/162] avg loss 0.00014874, throughput 5.94282K wps
[Epoch 26 Batch 120/162] avg loss 0.000118935, throughput 5.9437K wps
[Epoch 26 Batch 150/162] avg loss 0.000125312, throughput 5.95159K wps
Begin Testing...
[Epoch 26] train avg loss 0.000124463, test acc 0.9233, test avg loss 0.240925, throughput 5.97321K wps
[Epoch 27 Batch 30/162] avg loss 0.000119145, throughput 6.10004K wps
[Epoch 27 Batch 60/162] avg loss 0.000124791, throughput 5.94381K wps
[Epoch 27 Batch 90/162] avg loss 9.26316e-05, throughput 5.95962K wps
[Epoch 27 Batch 120/162] avg loss 0.000147297, throughput 5.94026K wps
[Epoch 27 Batch 150/162] avg loss 9.90262e-05, throughput 5.9407K wps
Begin Testing...
[Epoch 27] train avg loss 0.000115413, test acc 0.9233, test avg loss 0.243673, throughput 5.9733K wps
[Epoch 28 Batch 30/162] avg loss 0.000108608, throughput 6.08261K wps
[Epoch 28 Batch 60/162] avg loss 0.000111927, throughput 5.94497K wps
[Epoch 28 Batch 90/162] avg loss 9.20503e-05, throughput 5.93765K wps
[Epoch 28 Batch 120/162] avg loss 9.58228e-05, throughput 5.94648K wps
[Epoch 28 Batch 150/162] avg loss 0.000107395, throughput 5.9535K wps
Begin Testing...
[Epoch 28] train avg loss 0.000102243, test acc 0.9244, test avg loss 0.245968, throughput 5.97145K wps
[Epoch 29 Batch 30/162] avg loss 7.69179e-05, throughput 6.10147K wps
[Epoch 29 Batch 60/162] avg loss 7.6141e-05, throughput 5.96266K wps
[Epoch 29 Batch 90/162] avg loss 9.088e-05, throughput 5.96083K wps
[Epoch 29 Batch 120/162] avg loss 9.37181e-05, throughput 5.95222K wps
[Epoch 29 Batch 150/162] avg loss 8.02882e-05, throughput 5.93332K wps
Begin Testing...
[Epoch 29] train avg loss 8.37743e-05, test acc 0.9244, test avg loss 0.25144, throughput 5.978K wps
[Epoch 30 Batch 30/162] avg loss 7.65928e-05, throughput 6.09801K wps
[Epoch 30 Batch 60/162] avg loss 7.36518e-05, throughput 5.94981K wps
[Epoch 30 Batch 90/162] avg loss 7.9989e-05, throughput 5.94659K wps
[Epoch 30 Batch 120/162] avg loss 6.86571e-05, throughput 5.9622K wps
[Epoch 30 Batch 150/162] avg loss 6.35704e-05, throughput 5.94519K wps
Begin Testing...
[Epoch 30] train avg loss 7.31104e-05, test acc 0.9267, test avg loss 0.258144, throughput 5.97728K wps
[Epoch 31 Batch 30/162] avg loss 6.13658e-05, throughput 6.09915K wps
[Epoch 31 Batch 60/162] avg loss 5.97432e-05, throughput 5.93956K wps
[Epoch 31 Batch 90/162] avg loss 5.87628e-05, throughput 5.9385K wps
[Epoch 31 Batch 120/162] avg loss 7.47454e-05, throughput 5.94228K wps
[Epoch 31 Batch 150/162] avg loss 7.62794e-05, throughput 5.94831K wps
Begin Testing...
[Epoch 31] train avg loss 6.58116e-05, test acc 0.9256, test avg loss 0.267144, throughput 5.97113K wps
[Epoch 32 Batch 30/162] avg loss 7.47449e-05, throughput 6.09893K wps
[Epoch 32 Batch 60/162] avg loss 6.22346e-05, throughput 5.94524K wps
[Epoch 32 Batch 90/162] avg loss 6.80543e-05, throughput 5.95807K wps
[Epoch 32 Batch 120/162] avg loss 6.39973e-05, throughput 5.95636K wps
[Epoch 32 Batch 150/162] avg loss 4.79392e-05, throughput 5.95552K wps
Begin Testing...
[Epoch 32] train avg loss 6.33887e-05, test acc 0.9244, test avg loss 0.266649, throughput 5.97911K wps
[Epoch 33 Batch 30/162] avg loss 5.12643e-05, throughput 6.08558K wps
[Epoch 33 Batch 60/162] avg loss 6.21436e-05, throughput 5.95158K wps
[Epoch 33 Batch 90/162] avg loss 5.92493e-05, throughput 5.94722K wps
[Epoch 33 Batch 120/162] avg loss 4.82256e-05, throughput 5.94006K wps
[Epoch 33 Batch 150/162] avg loss 3.5005e-05, throughput 5.96218K wps
Begin Testing...
[Epoch 33] train avg loss 5.13732e-05, test acc 0.9256, test avg loss 0.270456, throughput 5.97549K wps
[Epoch 34 Batch 30/162] avg loss 4.05145e-05, throughput 6.0957K wps
[Epoch 34 Batch 60/162] avg loss 4.17323e-05, throughput 5.95769K wps
[Epoch 34 Batch 90/162] avg loss 4.05586e-05, throughput 5.95084K wps
[Epoch 34 Batch 120/162] avg loss 5.52379e-05, throughput 5.93786K wps
[Epoch 34 Batch 150/162] avg loss 4.10221e-05, throughput 5.94589K wps
Begin Testing...
[Epoch 34] train avg loss 4.60096e-05, test acc 0.9256, test avg loss 0.273178, throughput 5.97523K wps
[Epoch 35 Batch 30/162] avg loss 3.76541e-05, throughput 6.09745K wps
[Epoch 35 Batch 60/162] avg loss 4.16699e-05, throughput 5.94516K wps
[Epoch 35 Batch 90/162] avg loss 3.92795e-05, throughput 5.93843K wps
[Epoch 35 Batch 120/162] avg loss 4.62361e-05, throughput 5.94268K wps
[Epoch 35 Batch 150/162] avg loss 3.48446e-05, throughput 5.94228K wps
Begin Testing...
[Epoch 35] train avg loss 3.97761e-05, test acc 0.9244, test avg loss 0.277848, throughput 5.97042K wps
[Epoch 36 Batch 30/162] avg loss 3.05198e-05, throughput 6.10087K wps
[Epoch 36 Batch 60/162] avg loss 3.73226e-05, throughput 5.95815K wps
[Epoch 36 Batch 90/162] avg loss 2.93943e-05, throughput 5.95925K wps
[Epoch 36 Batch 120/162] avg loss 3.26884e-05, throughput 5.95038K wps
[Epoch 36 Batch 150/162] avg loss 3.28973e-05, throughput 5.93968K wps
Begin Testing...
[Epoch 36] train avg loss 3.35613e-05, test acc 0.9244, test avg loss 0.286596, throughput 5.9775K wps
[Epoch 37 Batch 30/162] avg loss 3.29627e-05, throughput 6.08431K wps
[Epoch 37 Batch 60/162] avg loss 2.69156e-05, throughput 5.96268K wps
[Epoch 37 Batch 90/162] avg loss 3.38217e-05, throughput 5.951K wps
[Epoch 37 Batch 120/162] avg loss 3.21306e-05, throughput 5.95473K wps
[Epoch 37 Batch 150/162] avg loss 3.3873e-05, throughput 5.9569K wps
Begin Testing...
[Epoch 37] train avg loss 3.17616e-05, test acc 0.9267, test avg loss 0.284406, throughput 5.97821K wps
[Epoch 38 Batch 30/162] avg loss 3.33462e-05, throughput 6.10237K wps
[Epoch 38 Batch 60/162] avg loss 2.65022e-05, throughput 5.95874K wps
[Epoch 38 Batch 90/162] avg loss 3.02897e-05, throughput 5.94818K wps
[Epoch 38 Batch 120/162] avg loss 2.62676e-05, throughput 5.9553K wps
[Epoch 38 Batch 150/162] avg loss 2.83297e-05, throughput 5.951K wps
Begin Testing...
[Epoch 38] train avg loss 2.87062e-05, test acc 0.9244, test avg loss 0.288191, throughput 5.97824K wps
[Epoch 39 Batch 30/162] avg loss 1.96317e-05, throughput 6.08507K wps
[Epoch 39 Batch 60/162] avg loss 2.42906e-05, throughput 5.94409K wps
[Epoch 39 Batch 90/162] avg loss 2.38093e-05, throughput 5.9485K wps
[Epoch 39 Batch 120/162] avg loss 2.46421e-05, throughput 5.94855K wps
[Epoch 39 Batch 150/162] avg loss 2.38772e-05, throughput 5.93888K wps
Begin Testing...
[Epoch 39] train avg loss 2.38467e-05, test acc 0.9233, test avg loss 0.293716, throughput 5.9706K wps
[Epoch 40 Batch 30/162] avg loss 2.79648e-05, throughput 6.10252K wps
[Epoch 40 Batch 60/162] avg loss 2.18179e-05, throughput 5.94562K wps
[Epoch 40 Batch 90/162] avg loss 2.19072e-05, throughput 5.94308K wps
[Epoch 40 Batch 120/162] avg loss 1.80377e-05, throughput 5.94618K wps
[Epoch 40 Batch 150/162] avg loss 1.91977e-05, throughput 5.95039K wps
Begin Testing...
[Epoch 40] train avg loss 2.12833e-05, test acc 0.9244, test avg loss 0.294065, throughput 5.97462K wps
[Epoch 41 Batch 30/162] avg loss 1.96768e-05, throughput 6.09746K wps
[Epoch 41 Batch 60/162] avg loss 2.01792e-05, throughput 5.93843K wps
[Epoch 41 Batch 90/162] avg loss 2.39704e-05, throughput 5.94122K wps
[Epoch 41 Batch 120/162] avg loss 2.32999e-05, throughput 5.94869K wps
[Epoch 41 Batch 150/162] avg loss 2.39639e-05, throughput 5.94708K wps
Begin Testing...
[Epoch 41] train avg loss 2.15633e-05, test acc 0.9278, test avg loss 0.294162, throughput 5.9725K wps
[Epoch 42 Batch 30/162] avg loss 1.75615e-05, throughput 6.09207K wps
[Epoch 42 Batch 60/162] avg loss 2.00973e-05, throughput 5.94912K wps
[Epoch 42 Batch 90/162] avg loss 1.80363e-05, throughput 5.95824K wps
[Epoch 42 Batch 120/162] avg loss 1.54647e-05, throughput 5.95165K wps
[Epoch 42 Batch 150/162] avg loss 2.70873e-05, throughput 5.94207K wps
Begin Testing...
[Epoch 42] train avg loss 1.93297e-05, test acc 0.9267, test avg loss 0.299863, throughput 5.97652K wps
[Epoch 43 Batch 30/162] avg loss 1.73536e-05, throughput 6.09414K wps
[Epoch 43 Batch 60/162] avg loss 1.82342e-05, throughput 5.94941K wps
[Epoch 43 Batch 90/162] avg loss 1.82578e-05, throughput 5.93411K wps
[Epoch 43 Batch 120/162] avg loss 1.85381e-05, throughput 5.94162K wps
[Epoch 43 Batch 150/162] avg loss 1.56829e-05, throughput 5.95939K wps
Begin Testing...
[Epoch 43] train avg loss 1.78497e-05, test acc 0.9278, test avg loss 0.302456, throughput 5.9741K wps
[Epoch 44 Batch 30/162] avg loss 1.31199e-05, throughput 6.09133K wps
[Epoch 44 Batch 60/162] avg loss 1.61189e-05, throughput 5.93786K wps
[Epoch 44 Batch 90/162] avg loss 1.81079e-05, throughput 5.94534K wps
[Epoch 44 Batch 120/162] avg loss 1.54704e-05, throughput 5.93867K wps
[Epoch 44 Batch 150/162] avg loss 1.30822e-05, throughput 5.93733K wps
Begin Testing...
[Epoch 44] train avg loss 1.50399e-05, test acc 0.9256, test avg loss 0.305596, throughput 5.96721K wps
[Epoch 45 Batch 30/162] avg loss 2.02955e-05, throughput 6.09333K wps
[Epoch 45 Batch 60/162] avg loss 1.51281e-05, throughput 5.91982K wps
[Epoch 45 Batch 90/162] avg loss 1.34552e-05, throughput 5.93439K wps
[Epoch 45 Batch 120/162] avg loss 1.41094e-05, throughput 5.94825K wps
[Epoch 45 Batch 150/162] avg loss 1.07652e-05, throughput 5.94922K wps
Begin Testing...
[Epoch 45] train avg loss 1.44412e-05, test acc 0.9267, test avg loss 0.312371, throughput 5.96605K wps
[Epoch 46 Batch 30/162] avg loss 1.02526e-05, throughput 6.09715K wps
[Epoch 46 Batch 60/162] avg loss 1.4618e-05, throughput 5.95688K wps
[Epoch 46 Batch 90/162] avg loss 1.38568e-05, throughput 5.94278K wps
[Epoch 46 Batch 120/162] avg loss 1.41652e-05, throughput 5.9419K wps
[Epoch 46 Batch 150/162] avg loss 1.15145e-05, throughput 5.95297K wps
Begin Testing...
[Epoch 46] train avg loss 1.2897e-05, test acc 0.9256, test avg loss 0.313432, throughput 5.97477K wps
[Epoch 47 Batch 30/162] avg loss 8.52852e-06, throughput 6.09038K wps
[Epoch 47 Batch 60/162] avg loss 1.2697e-05, throughput 5.94702K wps
[Epoch 47 Batch 90/162] avg loss 1.07641e-05, throughput 5.94905K wps
[Epoch 47 Batch 120/162] avg loss 1.06574e-05, throughput 5.95784K wps
[Epoch 47 Batch 150/162] avg loss 1.00937e-05, throughput 5.95399K wps
Begin Testing...
[Epoch 47] train avg loss 1.03503e-05, test acc 0.9256, test avg loss 0.314494, throughput 5.97645K wps
[Epoch 48 Batch 30/162] avg loss 9.24433e-06, throughput 6.09563K wps
[Epoch 48 Batch 60/162] avg loss 1.06302e-05, throughput 5.95546K wps
[Epoch 48 Batch 90/162] avg loss 1.06436e-05, throughput 5.95327K wps
[Epoch 48 Batch 120/162] avg loss 7.09691e-06, throughput 5.94021K wps
[Epoch 48 Batch 150/162] avg loss 1.0287e-05, throughput 5.94431K wps
Begin Testing...
[Epoch 48] train avg loss 9.77681e-06, test acc 0.9289, test avg loss 0.32297, throughput 5.975K wps
Observed Improvement.
Begin Testing...
[Epoch 49 Batch 30/162] avg loss 1.0139e-05, throughput 6.08474K wps
[Epoch 49 Batch 60/162] avg loss 9.85914e-06, throughput 5.93481K wps
[Epoch 49 Batch 90/162] avg loss 8.17125e-06, throughput 5.94655K wps
[Epoch 49 Batch 120/162] avg loss 8.35631e-06, throughput 5.93975K wps
[Epoch 49 Batch 150/162] avg loss 8.6403e-06, throughput 5.94085K wps
Begin Testing...
[Epoch 49] train avg loss 9.33695e-06, test acc 0.9278, test avg loss 0.319334, throughput 5.96659K wps
[Epoch 50 Batch 30/162] avg loss 8.04357e-06, throughput 6.10641K wps
[Epoch 50 Batch 60/162] avg loss 8.26962e-06, throughput 5.95684K wps
[Epoch 50 Batch 90/162] avg loss 1.25331e-05, throughput 5.95095K wps
[Epoch 50 Batch 120/162] avg loss 1.15461e-05, throughput 5.94335K wps
[Epoch 50 Batch 150/162] avg loss 1.15946e-05, throughput 5.93935K wps
Begin Testing...
[Epoch 50] train avg loss 1.00566e-05, test acc 0.9300, test avg loss 0.324627, throughput 5.97513K wps
Observed Improvement.
Begin Testing...
[Epoch 51 Batch 30/162] avg loss 7.68678e-06, throughput 6.09186K wps
[Epoch 51 Batch 60/162] avg loss 5.51246e-06, throughput 5.94573K wps
[Epoch 51 Batch 90/162] avg loss 8.4499e-06, throughput 5.94618K wps
[Epoch 51 Batch 120/162] avg loss 6.29628e-06, throughput 5.93297K wps
[Epoch 51 Batch 150/162] avg loss 6.50681e-06, throughput 5.94284K wps
Begin Testing...
[Epoch 51] train avg loss 6.90984e-06, test acc 0.9300, test avg loss 0.329597, throughput 5.97003K wps
Observed Improvement.
Begin Testing...
[Epoch 52 Batch 30/162] avg loss 6.01607e-06, throughput 6.08441K wps
[Epoch 52 Batch 60/162] avg loss 5.74323e-06, throughput 5.94378K wps
[Epoch 52 Batch 90/162] avg loss 6.09327e-06, throughput 5.94982K wps
[Epoch 52 Batch 120/162] avg loss 1.05884e-05, throughput 5.93338K wps
[Epoch 52 Batch 150/162] avg loss 5.8592e-06, throughput 5.94101K wps
Begin Testing...
[Epoch 52] train avg loss 6.72424e-06, test acc 0.9289, test avg loss 0.334344, throughput 5.9677K wps
[Epoch 53 Batch 30/162] avg loss 5.23558e-06, throughput 6.08538K wps
[Epoch 53 Batch 60/162] avg loss 4.76622e-06, throughput 5.94601K wps
[Epoch 53 Batch 90/162] avg loss 5.3064e-06, throughput 5.93226K wps
[Epoch 53 Batch 120/162] avg loss 5.26662e-06, throughput 5.9485K wps
[Epoch 53 Batch 150/162] avg loss 4.2623e-06, throughput 5.94485K wps
Begin Testing...
[Epoch 53] train avg loss 5.02667e-06, test acc 0.9289, test avg loss 0.336161, throughput 5.97024K wps
[Epoch 54 Batch 30/162] avg loss 5.83464e-06, throughput 6.09333K wps
[Epoch 54 Batch 60/162] avg loss 5.27409e-06, throughput 5.93292K wps
[Epoch 54 Batch 90/162] avg loss 6.08424e-06, throughput 5.94449K wps
[Epoch 54 Batch 120/162] avg loss 4.72961e-06, throughput 5.95035K wps
[Epoch 54 Batch 150/162] avg loss 5.15802e-06, throughput 5.94371K wps
Begin Testing...
[Epoch 54] train avg loss 5.5065e-06, test acc 0.9278, test avg loss 0.337755, throughput 5.9704K wps
[Epoch 55 Batch 30/162] avg loss 4.89215e-06, throughput 6.08369K wps
[Epoch 55 Batch 60/162] avg loss 5.34897e-06, throughput 5.94863K wps
[Epoch 55 Batch 90/162] avg loss 4.71668e-06, throughput 5.9388K wps
[Epoch 55 Batch 120/162] avg loss 4.13579e-06, throughput 5.9354K wps
[Epoch 55 Batch 150/162] avg loss 3.70172e-06, throughput 5.94093K wps
Begin Testing...
[Epoch 55] train avg loss 4.68912e-06, test acc 0.9289, test avg loss 0.342309, throughput 5.96774K wps
[Epoch 56 Batch 30/162] avg loss 5.58611e-06, throughput 6.10129K wps
[Epoch 56 Batch 60/162] avg loss 3.91528e-06, throughput 5.95256K wps
[Epoch 56 Batch 90/162] avg loss 6.66622e-06, throughput 5.94519K wps
[Epoch 56 Batch 120/162] avg loss 4.96579e-06, throughput 5.94253K wps
[Epoch 56 Batch 150/162] avg loss 3.18673e-06, throughput 5.92951K wps
Begin Testing...
[Epoch 56] train avg loss 4.8234e-06, test acc 0.9289, test avg loss 0.347305, throughput 5.97011K wps
[Epoch 57 Batch 30/162] avg loss 4.38531e-06, throughput 6.0948K wps
[Epoch 57 Batch 60/162] avg loss 3.66971e-06, throughput 5.93684K wps
[Epoch 57 Batch 90/162] avg loss 4.43699e-06, throughput 5.95803K wps
[Epoch 57 Batch 120/162] avg loss 3.40277e-06, throughput 5.95742K wps
[Epoch 57 Batch 150/162] avg loss 3.31978e-06, throughput 5.94741K wps
Begin Testing...
[Epoch 57] train avg loss 3.73609e-06, test acc 0.9311, test avg loss 0.3523, throughput 5.97595K wps
Observed Improvement.
Begin Testing...
[Epoch 58 Batch 30/162] avg loss 4.47332e-06, throughput 6.09181K wps
[Epoch 58 Batch 60/162] avg loss 2.65952e-06, throughput 5.94098K wps
[Epoch 58 Batch 90/162] avg loss 2.99977e-06, throughput 5.95512K wps
[Epoch 58 Batch 120/162] avg loss 3.64261e-06, throughput 5.9434K wps
[Epoch 58 Batch 150/162] avg loss 4.94249e-06, throughput 5.94808K wps
Begin Testing...
[Epoch 58] train avg loss 3.81481e-06, test acc 0.9289, test avg loss 0.356724, throughput 5.97209K wps
[Epoch 59 Batch 30/162] avg loss 2.69131e-06, throughput 6.1026K wps
[Epoch 59 Batch 60/162] avg loss 2.85773e-06, throughput 5.93746K wps
[Epoch 59 Batch 90/162] avg loss 3.2502e-06, throughput 5.94089K wps
[Epoch 59 Batch 120/162] avg loss 3.00895e-06, throughput 5.95412K wps
[Epoch 59 Batch 150/162] avg loss 3.05382e-06, throughput 5.94289K wps
Begin Testing...
[Epoch 59] train avg loss 3.01395e-06, test acc 0.9278, test avg loss 0.359333, throughput 5.97226K wps
Test loss 0.410946, test acc 0.8950
Total time cost 339.82s
9000 1000
[Epoch 0 Batch 30/162] avg loss 0.0153461, throughput 5.69237K wps
[Epoch 0 Batch 60/162] avg loss 0.0143308, throughput 5.93072K wps
[Epoch 0 Batch 90/162] avg loss 0.0133773, throughput 5.93291K wps
[Epoch 0 Batch 120/162] avg loss 0.0130774, throughput 5.93783K wps
[Epoch 0 Batch 150/162] avg loss 0.0127226, throughput 5.93807K wps
Begin Testing...
[Epoch 0] train avg loss 0.0136555, test acc 0.6978, test avg loss 0.586236, throughput 5.88883K wps
Observed Improvement.
Begin Testing...
[Epoch 1 Batch 30/162] avg loss 0.0122011, throughput 6.09647K wps
[Epoch 1 Batch 60/162] avg loss 0.0118689, throughput 5.94987K wps
[Epoch 1 Batch 90/162] avg loss 0.0112804, throughput 5.95455K wps
[Epoch 1 Batch 120/162] avg loss 0.0114704, throughput 5.94574K wps
[Epoch 1 Batch 150/162] avg loss 0.0110373, throughput 5.95797K wps
Begin Testing...
[Epoch 1] train avg loss 0.0115254, test acc 0.7900, test avg loss 0.516923, throughput 5.97922K wps
Observed Improvement.
Begin Testing...
[Epoch 2 Batch 30/162] avg loss 0.0102947, throughput 6.09144K wps
[Epoch 2 Batch 60/162] avg loss 0.0101197, throughput 5.95415K wps
[Epoch 2 Batch 90/162] avg loss 0.00976938, throughput 5.94195K wps
[Epoch 2 Batch 120/162] avg loss 0.00944748, throughput 5.94636K wps
[Epoch 2 Batch 150/162] avg loss 0.00906518, throughput 5.96527K wps
Begin Testing...
[Epoch 2] train avg loss 0.00966553, test acc 0.8578, test avg loss 0.434688, throughput 5.97764K wps
Observed Improvement.
Begin Testing...
[Epoch 3 Batch 30/162] avg loss 0.00825368, throughput 6.08416K wps
[Epoch 3 Batch 60/162] avg loss 0.00813457, throughput 5.94292K wps
[Epoch 3 Batch 90/162] avg loss 0.00755755, throughput 5.94925K wps
[Epoch 3 Batch 120/162] avg loss 0.0076197, throughput 5.92822K wps
[Epoch 3 Batch 150/162] avg loss 0.0069886, throughput 5.94651K wps
Begin Testing...
[Epoch 3] train avg loss 0.00763647, test acc 0.8956, test avg loss 0.355947, throughput 5.96917K wps
Observed Improvement.
Begin Testing...
[Epoch 4 Batch 30/162] avg loss 0.0066927, throughput 6.06937K wps
[Epoch 4 Batch 60/162] avg loss 0.00596002, throughput 5.92744K wps
[Epoch 4 Batch 90/162] avg loss 0.00611136, throughput 5.94125K wps
[Epoch 4 Batch 120/162] avg loss 0.00573211, throughput 5.94294K wps
[Epoch 4 Batch 150/162] avg loss 0.00585165, throughput 5.94398K wps
Begin Testing...
[Epoch 4] train avg loss 0.00606515, test acc 0.9011, test avg loss 0.299955, throughput 5.96312K wps
Observed Improvement.
Begin Testing...
[Epoch 5 Batch 30/162] avg loss 0.00495351, throughput 6.09528K wps
[Epoch 5 Batch 60/162] avg loss 0.00511755, throughput 5.95638K wps
[Epoch 5 Batch 90/162] avg loss 0.00501238, throughput 5.94171K wps
[Epoch 5 Batch 120/162] avg loss 0.00476335, throughput 5.95394K wps
[Epoch 5 Batch 150/162] avg loss 0.00484618, throughput 5.95718K wps
Begin Testing...
[Epoch 5] train avg loss 0.00492339, test acc 0.9044, test avg loss 0.271224, throughput 5.97855K wps
Observed Improvement.
Begin Testing...
[Epoch 6 Batch 30/162] avg loss 0.00439239, throughput 6.09217K wps
[Epoch 6 Batch 60/162] avg loss 0.00388624, throughput 5.92584K wps
[Epoch 6 Batch 90/162] avg loss 0.00401279, throughput 5.93451K wps
[Epoch 6 Batch 120/162] avg loss 0.00411827, throughput 5.93979K wps
[Epoch 6 Batch 150/162] avg loss 0.00393802, throughput 5.93546K wps
Begin Testing...
[Epoch 6] train avg loss 0.00406114, test acc 0.9056, test avg loss 0.247546, throughput 5.96384K wps
Observed Improvement.
Begin Testing...
[Epoch 7 Batch 30/162] avg loss 0.00362023, throughput 6.10291K wps
[Epoch 7 Batch 60/162] avg loss 0.0036038, throughput 5.95155K wps
[Epoch 7 Batch 90/162] avg loss 0.00313056, throughput 5.95943K wps
[Epoch 7 Batch 120/162] avg loss 0.0031137, throughput 5.94566K wps
[Epoch 7 Batch 150/162] avg loss 0.00346939, throughput 5.94862K wps
Begin Testing...
[Epoch 7] train avg loss 0.00338913, test acc 0.9089, test avg loss 0.231269, throughput 5.97891K wps
Observed Improvement.
Begin Testing...
[Epoch 8 Batch 30/162] avg loss 0.00292303, throughput 6.08385K wps
[Epoch 8 Batch 60/162] avg loss 0.0028989, throughput 5.9395K wps
[Epoch 8 Batch 90/162] avg loss 0.00290651, throughput 5.94204K wps
[Epoch 8 Batch 120/162] avg loss 0.00281305, throughput 5.95565K wps
[Epoch 8 Batch 150/162] avg loss 0.00288774, throughput 5.95073K wps
Begin Testing...
[Epoch 8] train avg loss 0.00286433, test acc 0.9133, test avg loss 0.221754, throughput 5.97205K wps
Observed Improvement.
Begin Testing...
[Epoch 9 Batch 30/162] avg loss 0.00255665, throughput 6.09849K wps
[Epoch 9 Batch 60/162] avg loss 0.00246655, throughput 5.94007K wps
[Epoch 9 Batch 90/162] avg loss 0.00214924, throughput 5.94164K wps
[Epoch 9 Batch 120/162] avg loss 0.00218595, throughput 5.93666K wps
[Epoch 9 Batch 150/162] avg loss 0.00246182, throughput 5.93003K wps
Begin Testing...
[Epoch 9] train avg loss 0.0023501, test acc 0.9144, test avg loss 0.216732, throughput 5.96518K wps
Observed Improvement.
Begin Testing...
[Epoch 10 Batch 30/162] avg loss 0.00188464, throughput 6.09864K wps
[Epoch 10 Batch 60/162] avg loss 0.00220303, throughput 5.94145K wps
[Epoch 10 Batch 90/162] avg loss 0.00191049, throughput 5.9488K wps
[Epoch 10 Batch 120/162] avg loss 0.0020251, throughput 5.94166K wps
[Epoch 10 Batch 150/162] avg loss 0.00203611, throughput 5.94956K wps
Begin Testing...
[Epoch 10] train avg loss 0.0020063, test acc 0.9156, test avg loss 0.213243, throughput 5.97269K wps
Observed Improvement.
Begin Testing...
[Epoch 11 Batch 30/162] avg loss 0.0017743, throughput 6.07468K wps
[Epoch 11 Batch 60/162] avg loss 0.00186856, throughput 5.94992K wps
[Epoch 11 Batch 90/162] avg loss 0.0016661, throughput 5.94055K wps
[Epoch 11 Batch 120/162] avg loss 0.00156271, throughput 5.94054K wps
[Epoch 11 Batch 150/162] avg loss 0.00152984, throughput 5.94358K wps
Begin Testing...
[Epoch 11] train avg loss 0.00167203, test acc 0.9133, test avg loss 0.20816, throughput 5.96805K wps
[Epoch 12 Batch 30/162] avg loss 0.00128914, throughput 6.08738K wps
[Epoch 12 Batch 60/162] avg loss 0.00144704, throughput 5.94373K wps
[Epoch 12 Batch 90/162] avg loss 0.00141326, throughput 5.93337K wps
[Epoch 12 Batch 120/162] avg loss 0.00130916, throughput 5.92584K wps
[Epoch 12 Batch 150/162] avg loss 0.00147934, throughput 5.92826K wps
Begin Testing...
[Epoch 12] train avg loss 0.00137728, test acc 0.9178, test avg loss 0.205564, throughput 5.96012K wps
Observed Improvement.
Begin Testing...
[Epoch 13 Batch 30/162] avg loss 0.00105444, throughput 6.09114K wps
[Epoch 13 Batch 60/162] avg loss 0.00123667, throughput 5.94802K wps
[Epoch 13 Batch 90/162] avg loss 0.00104548, throughput 5.93577K wps
[Epoch 13 Batch 120/162] avg loss 0.001222, throughput 5.94745K wps
[Epoch 13 Batch 150/162] avg loss 0.00119065, throughput 5.94585K wps
Begin Testing...
[Epoch 13] train avg loss 0.00114575, test acc 0.9211, test avg loss 0.211215, throughput 5.9718K wps
Observed Improvement.
Begin Testing...
[Epoch 14 Batch 30/162] avg loss 0.00109861, throughput 6.10063K wps
[Epoch 14 Batch 60/162] avg loss 0.000947021, throughput 5.94817K wps
[Epoch 14 Batch 90/162] avg loss 0.000990956, throughput 5.94087K wps
[Epoch 14 Batch 120/162] avg loss 0.000948343, throughput 5.93265K wps
[Epoch 14 Batch 150/162] avg loss 0.000889691, throughput 5.93058K wps
Begin Testing...
[Epoch 14] train avg loss 0.000965044, test acc 0.9244, test avg loss 0.21033, throughput 5.96633K wps
Observed Improvement.
Begin Testing...
[Epoch 15 Batch 30/162] avg loss 0.000777009, throughput 6.087K wps
[Epoch 15 Batch 60/162] avg loss 0.000874197, throughput 5.93743K wps
[Epoch 15 Batch 90/162] avg loss 0.000848351, throughput 5.95212K wps
[Epoch 15 Batch 120/162] avg loss 0.000839651, throughput 5.94706K wps
[Epoch 15 Batch 150/162] avg loss 0.000815674, throughput 5.94387K wps
Begin Testing...
[Epoch 15] train avg loss 0.000811106, test acc 0.9211, test avg loss 0.216101, throughput 5.97133K wps
[Epoch 16 Batch 30/162] avg loss 0.000706166, throughput 6.10033K wps
[Epoch 16 Batch 60/162] avg loss 0.000709945, throughput 5.94275K wps
[Epoch 16 Batch 90/162] avg loss 0.000667265, throughput 5.94611K wps
[Epoch 16 Batch 120/162] avg loss 0.00064103, throughput 5.92408K wps
[Epoch 16 Batch 150/162] avg loss 0.000697471, throughput 5.8785K wps
Begin Testing...
[Epoch 16] train avg loss 0.000678258, test acc 0.9200, test avg loss 0.213647, throughput 5.95632K wps
[Epoch 17 Batch 30/162] avg loss 0.000602745, throughput 6.09104K wps
[Epoch 17 Batch 60/162] avg loss 0.000547989, throughput 5.93733K wps
[Epoch 17 Batch 90/162] avg loss 0.00057807, throughput 5.94324K wps
[Epoch 17 Batch 120/162] avg loss 0.0005116, throughput 5.9415K wps
[Epoch 17 Batch 150/162] avg loss 0.000573119, throughput 5.94472K wps
Begin Testing...
[Epoch 17] train avg loss 0.000566044, test acc 0.9189, test avg loss 0.214492, throughput 5.9689K wps
[Epoch 18 Batch 30/162] avg loss 0.000497523, throughput 6.08162K wps
[Epoch 18 Batch 60/162] avg loss 0.00053971, throughput 5.9455K wps
[Epoch 18 Batch 90/162] avg loss 0.000450971, throughput 5.93376K wps
[Epoch 18 Batch 120/162] avg loss 0.00044113, throughput 5.94943K wps
[Epoch 18 Batch 150/162] avg loss 0.000513467, throughput 5.92848K wps
Begin Testing...
[Epoch 18] train avg loss 0.000489311, test acc 0.9189, test avg loss 0.220107, throughput 5.96473K wps
[Epoch 19 Batch 30/162] avg loss 0.000446427, throughput 6.08825K wps
[Epoch 19 Batch 60/162] avg loss 0.000361909, throughput 5.93706K wps
[Epoch 19 Batch 90/162] avg loss 0.000400204, throughput 5.93726K wps
[Epoch 19 Batch 120/162] avg loss 0.000380204, throughput 5.9366K wps
[Epoch 19 Batch 150/162] avg loss 0.000456784, throughput 5.94333K wps
Begin Testing...
[Epoch 19] train avg loss 0.000411293, test acc 0.9211, test avg loss 0.223406, throughput 5.96474K wps
[Epoch 20 Batch 30/162] avg loss 0.000331682, throughput 6.08916K wps
[Epoch 20 Batch 60/162] avg loss 0.00034552, throughput 5.94376K wps
[Epoch 20 Batch 90/162] avg loss 0.000277785, throughput 5.94036K wps
[Epoch 20 Batch 120/162] avg loss 0.000355645, throughput 5.94396K wps
[Epoch 20 Batch 150/162] avg loss 0.00037405, throughput 5.93923K wps
Begin Testing...
[Epoch 20] train avg loss 0.000334772, test acc 0.9189, test avg loss 0.225063, throughput 5.96775K wps
[Epoch 21 Batch 30/162] avg loss 0.000291872, throughput 6.08345K wps
[Epoch 21 Batch 60/162] avg loss 0.000310596, throughput 5.94K wps
[Epoch 21 Batch 90/162] avg loss 0.000290671, throughput 5.93457K wps
[Epoch 21 Batch 120/162] avg loss 0.000283658, throughput 5.93657K wps
[Epoch 21 Batch 150/162] avg loss 0.000269056, throughput 5.95138K wps
Begin Testing...
[Epoch 21] train avg loss 0.000286696, test acc 0.9178, test avg loss 0.227687, throughput 5.96707K wps
[Epoch 22 Batch 30/162] avg loss 0.00025286, throughput 6.08438K wps
[Epoch 22 Batch 60/162] avg loss 0.000247902, throughput 5.94217K wps
[Epoch 22 Batch 90/162] avg loss 0.000235173, throughput 5.9314K wps
[Epoch 22 Batch 120/162] avg loss 0.000234036, throughput 5.9448K wps
[Epoch 22 Batch 150/162] avg loss 0.00027244, throughput 5.94535K wps
Begin Testing...
[Epoch 22] train avg loss 0.000251499, test acc 0.9256, test avg loss 0.230764, throughput 5.96655K wps
Observed Improvement.
Begin Testing...
[Epoch 23 Batch 30/162] avg loss 0.00022942, throughput 6.07349K wps
[Epoch 23 Batch 60/162] avg loss 0.000209048, throughput 5.93201K wps
[Epoch 23 Batch 90/162] avg loss 0.000204363, throughput 5.9479K wps
[Epoch 23 Batch 120/162] avg loss 0.000192921, throughput 5.94268K wps
[Epoch 23 Batch 150/162] avg loss 0.00018293, throughput 5.9493K wps
Begin Testing...
[Epoch 23] train avg loss 0.000203639, test acc 0.9167, test avg loss 0.232994, throughput 5.96747K wps
[Epoch 24 Batch 30/162] avg loss 0.000215175, throughput 6.09539K wps
[Epoch 24 Batch 60/162] avg loss 0.00017487, throughput 5.93676K wps
[Epoch 24 Batch 90/162] avg loss 0.000178153, throughput 5.93407K wps
[Epoch 24 Batch 120/162] avg loss 0.000156485, throughput 5.94084K wps
[Epoch 24 Batch 150/162] avg loss 0.000164395, throughput 5.92773K wps
Begin Testing...
[Epoch 24] train avg loss 0.000180272, test acc 0.9244, test avg loss 0.24235, throughput 5.96349K wps
[Epoch 25 Batch 30/162] avg loss 0.000166871, throughput 6.0946K wps
[Epoch 25 Batch 60/162] avg loss 0.000183119, throughput 5.938K wps
[Epoch 25 Batch 90/162] avg loss 0.000140952, throughput 5.94287K wps
[Epoch 25 Batch 120/162] avg loss 0.00017781, throughput 5.94378K wps
[Epoch 25 Batch 150/162] avg loss 0.000156107, throughput 5.94209K wps
Begin Testing...
[Epoch 25] train avg loss 0.000163585, test acc 0.9222, test avg loss 0.242136, throughput 5.96921K wps
[Epoch 26 Batch 30/162] avg loss 0.000142402, throughput 6.08084K wps
[Epoch 26 Batch 60/162] avg loss 0.000126461, throughput 5.93133K wps
[Epoch 26 Batch 90/162] avg loss 0.000121351, throughput 5.94038K wps
[Epoch 26 Batch 120/162] avg loss 0.000119474, throughput 5.93094K wps
[Epoch 26 Batch 150/162] avg loss 0.000156867, throughput 5.94001K wps
Begin Testing...
[Epoch 26] train avg loss 0.00013314, test acc 0.9211, test avg loss 0.24405, throughput 5.96311K wps
[Epoch 27 Batch 30/162] avg loss 0.000108076, throughput 6.09183K wps
[Epoch 27 Batch 60/162] avg loss 0.000108054, throughput 5.93745K wps
[Epoch 27 Batch 90/162] avg loss 0.000115928, throughput 5.93688K wps
[Epoch 27 Batch 120/162] avg loss 0.00011234, throughput 5.93319K wps
[Epoch 27 Batch 150/162] avg loss 0.000103894, throughput 5.94117K wps
Begin Testing...
[Epoch 27] train avg loss 0.000107986, test acc 0.9200, test avg loss 0.252, throughput 5.96525K wps
[Epoch 28 Batch 30/162] avg loss 0.000119552, throughput 6.08628K wps
[Epoch 28 Batch 60/162] avg loss 9.32931e-05, throughput 5.94174K wps
[Epoch 28 Batch 90/162] avg loss 9.25973e-05, throughput 5.95594K wps
[Epoch 28 Batch 120/162] avg loss 0.000129243, throughput 5.94066K wps
[Epoch 28 Batch 150/162] avg loss 0.000111542, throughput 5.93064K wps
Begin Testing...
[Epoch 28] train avg loss 0.00010901, test acc 0.9144, test avg loss 0.251214, throughput 5.96834K wps
[Epoch 29 Batch 30/162] avg loss 8.38437e-05, throughput 6.08493K wps
[Epoch 29 Batch 60/162] avg loss 8.33076e-05, throughput 5.94256K wps
[Epoch 29 Batch 90/162] avg loss 7.53501e-05, throughput 5.93632K wps
[Epoch 29 Batch 120/162] avg loss 8.45741e-05, throughput 5.93863K wps
[Epoch 29 Batch 150/162] avg loss 8.52743e-05, throughput 5.94128K wps
Begin Testing...
[Epoch 29] train avg loss 8.18077e-05, test acc 0.9156, test avg loss 0.254205, throughput 5.96704K wps
[Epoch 30 Batch 30/162] avg loss 8.10482e-05, throughput 6.07379K wps
[Epoch 30 Batch 60/162] avg loss 7.43258e-05, throughput 5.94652K wps
[Epoch 30 Batch 90/162] avg loss 7.61833e-05, throughput 5.94622K wps
[Epoch 30 Batch 120/162] avg loss 7.30069e-05, throughput 5.94134K wps
[Epoch 30 Batch 150/162] avg loss 7.61496e-05, throughput 5.94461K wps
Begin Testing...
[Epoch 30] train avg loss 7.48989e-05, test acc 0.9189, test avg loss 0.259581, throughput 5.96779K wps
[Epoch 31 Batch 30/162] avg loss 5.92592e-05, throughput 6.08534K wps
[Epoch 31 Batch 60/162] avg loss 8.98615e-05, throughput 5.93543K wps
[Epoch 31 Batch 90/162] avg loss 6.55695e-05, throughput 5.9371K wps
[Epoch 31 Batch 120/162] avg loss 6.66219e-05, throughput 5.94239K wps
[Epoch 31 Batch 150/162] avg loss 5.36936e-05, throughput 5.93652K wps
Begin Testing...
[Epoch 31] train avg loss 6.6455e-05, test acc 0.9178, test avg loss 0.262956, throughput 5.96502K wps
[Epoch 32 Batch 30/162] avg loss 4.585e-05, throughput 6.09319K wps
[Epoch 32 Batch 60/162] avg loss 4.87233e-05, throughput 5.93206K wps
[Epoch 32 Batch 90/162] avg loss 5.98463e-05, throughput 5.94239K wps
[Epoch 32 Batch 120/162] avg loss 6.7086e-05, throughput 5.93271K wps
[Epoch 32 Batch 150/162] avg loss 6.59368e-05, throughput 5.93239K wps
Begin Testing...
[Epoch 32] train avg loss 5.70102e-05, test acc 0.9156, test avg loss 0.267246, throughput 5.96308K wps
[Epoch 33 Batch 30/162] avg loss 5.77894e-05, throughput 6.0794K wps
[Epoch 33 Batch 60/162] avg loss 5.57502e-05, throughput 5.93388K wps
[Epoch 33 Batch 90/162] avg loss 4.53184e-05, throughput 5.94719K wps
[Epoch 33 Batch 120/162] avg loss 4.89252e-05, throughput 5.9324K wps
[Epoch 33 Batch 150/162] avg loss 5.86369e-05, throughput 5.92798K wps
Begin Testing...
[Epoch 33] train avg loss 5.29952e-05, test acc 0.9178, test avg loss 0.274482, throughput 5.96179K wps
[Epoch 34 Batch 30/162] avg loss 5.16955e-05, throughput 6.08204K wps
[Epoch 34 Batch 60/162] avg loss 4.67304e-05, throughput 5.93477K wps
[Epoch 34 Batch 90/162] avg loss 4.12065e-05, throughput 5.93937K wps
[Epoch 34 Batch 120/162] avg loss 6.68333e-05, throughput 5.93212K wps
[Epoch 34 Batch 150/162] avg loss 4.4894e-05, throughput 5.92936K wps
Begin Testing...
[Epoch 34] train avg loss 4.89368e-05, test acc 0.9144, test avg loss 0.278378, throughput 5.96038K wps
[Epoch 35 Batch 30/162] avg loss 4.3655e-05, throughput 6.08923K wps
[Epoch 35 Batch 60/162] avg loss 3.47534e-05, throughput 5.94691K wps
[Epoch 35 Batch 90/162] avg loss 3.90361e-05, throughput 5.92732K wps
[Epoch 35 Batch 120/162] avg loss 3.82688e-05, throughput 5.94495K wps
[Epoch 35 Batch 150/162] avg loss 3.94545e-05, throughput 5.94574K wps
Begin Testing...
[Epoch 35] train avg loss 3.88177e-05, test acc 0.9144, test avg loss 0.282054, throughput 5.96809K wps
[Epoch 36 Batch 30/162] avg loss 3.42051e-05, throughput 6.09614K wps
[Epoch 36 Batch 60/162] avg loss 3.51061e-05, throughput 5.94412K wps
[Epoch 36 Batch 90/162] avg loss 3.61515e-05, throughput 5.95007K wps
[Epoch 36 Batch 120/162] avg loss 3.87184e-05, throughput 5.93655K wps
[Epoch 36 Batch 150/162] avg loss 3.4817e-05, throughput 5.93699K wps
Begin Testing...
[Epoch 36] train avg loss 3.60161e-05, test acc 0.9144, test avg loss 0.283023, throughput 5.96996K wps
[Epoch 37 Batch 30/162] avg loss 2.81171e-05, throughput 6.08881K wps
[Epoch 37 Batch 60/162] avg loss 3.32286e-05, throughput 5.94321K wps
[Epoch 37 Batch 90/162] avg loss 3.59506e-05, throughput 5.95112K wps
[Epoch 37 Batch 120/162] avg loss 2.85843e-05, throughput 5.95031K wps
[Epoch 37 Batch 150/162] avg loss 3.55178e-05, throughput 5.92377K wps
Begin Testing...
[Epoch 37] train avg loss 3.18139e-05, test acc 0.9156, test avg loss 0.290099, throughput 5.96801K wps
[Epoch 38 Batch 30/162] avg loss 2.89398e-05, throughput 6.07876K wps
[Epoch 38 Batch 60/162] avg loss 3.17123e-05, throughput 5.93684K wps
[Epoch 38 Batch 90/162] avg loss 2.93296e-05, throughput 5.93912K wps
[Epoch 38 Batch 120/162] avg loss 2.9012e-05, throughput 5.95374K wps
[Epoch 38 Batch 150/162] avg loss 2.83338e-05, throughput 5.93623K wps
Begin Testing...
[Epoch 38] train avg loss 2.96051e-05, test acc 0.9156, test avg loss 0.292984, throughput 5.96648K wps
[Epoch 39 Batch 30/162] avg loss 3.03162e-05, throughput 6.08937K wps
[Epoch 39 Batch 60/162] avg loss 2.49036e-05, throughput 5.94276K wps
[Epoch 39 Batch 90/162] avg loss 2.08985e-05, throughput 5.94627K wps
[Epoch 39 Batch 120/162] avg loss 2.48857e-05, throughput 5.94022K wps
[Epoch 39 Batch 150/162] avg loss 2.38092e-05, throughput 5.939K wps
Begin Testing...
[Epoch 39] train avg loss 2.48122e-05, test acc 0.9156, test avg loss 0.295594, throughput 5.97001K wps
[Epoch 40 Batch 30/162] avg loss 2.56611e-05, throughput 6.08811K wps
[Epoch 40 Batch 60/162] avg loss 2.29398e-05, throughput 5.93841K wps
[Epoch 40 Batch 90/162] avg loss 2.15402e-05, throughput 5.93339K wps
[Epoch 40 Batch 120/162] avg loss 2.3694e-05, throughput 5.92561K wps
[Epoch 40 Batch 150/162] avg loss 2.26885e-05, throughput 5.94078K wps
Begin Testing...
[Epoch 40] train avg loss 2.29026e-05, test acc 0.9144, test avg loss 0.301501, throughput 5.96091K wps
[Epoch 41 Batch 30/162] avg loss 1.85924e-05, throughput 6.087K wps
[Epoch 41 Batch 60/162] avg loss 1.87471e-05, throughput 5.94266K wps
[Epoch 41 Batch 90/162] avg loss 2.60335e-05, throughput 5.95002K wps
[Epoch 41 Batch 120/162] avg loss 2.10866e-05, throughput 5.93295K wps
[Epoch 41 Batch 150/162] avg loss 2.05203e-05, throughput 5.92894K wps
Begin Testing...
[Epoch 41] train avg loss 2.15283e-05, test acc 0.9144, test avg loss 0.301873, throughput 5.96443K wps
[Epoch 42 Batch 30/162] avg loss 1.51698e-05, throughput 6.07311K wps
[Epoch 42 Batch 60/162] avg loss 2.04728e-05, throughput 5.94382K wps
[Epoch 42 Batch 90/162] avg loss 1.84745e-05, throughput 5.94277K wps
[Epoch 42 Batch 120/162] avg loss 1.85282e-05, throughput 5.94307K wps
[Epoch 42 Batch 150/162] avg loss 2.57577e-05, throughput 5.94778K wps
Begin Testing...
[Epoch 42] train avg loss 1.92978e-05, test acc 0.9122, test avg loss 0.303382, throughput 5.96836K wps
[Epoch 43 Batch 30/162] avg loss 1.42532e-05, throughput 6.08236K wps
[Epoch 43 Batch 60/162] avg loss 1.38676e-05, throughput 5.95005K wps
[Epoch 43 Batch 90/162] avg loss 1.29936e-05, throughput 5.9471K wps
[Epoch 43 Batch 120/162] avg loss 1.8666e-05, throughput 5.9233K wps
[Epoch 43 Batch 150/162] avg loss 1.79068e-05, throughput 5.92794K wps
Begin Testing...
[Epoch 43] train avg loss 1.51551e-05, test acc 0.9167, test avg loss 0.30869, throughput 5.96256K wps
[Epoch 44 Batch 30/162] avg loss 2.43257e-05, throughput 6.07364K wps
[Epoch 44 Batch 60/162] avg loss 1.25655e-05, throughput 5.92975K wps
[Epoch 44 Batch 90/162] avg loss 1.19889e-05, throughput 5.94538K wps
[Epoch 44 Batch 120/162] avg loss 1.62231e-05, throughput 5.92928K wps
[Epoch 44 Batch 150/162] avg loss 1.51696e-05, throughput 5.92366K wps
Begin Testing...
[Epoch 44] train avg loss 1.66273e-05, test acc 0.9144, test avg loss 0.312875, throughput 5.95809K wps
[Epoch 45 Batch 30/162] avg loss 1.56564e-05, throughput 6.09145K wps
[Epoch 45 Batch 60/162] avg loss 1.36536e-05, throughput 5.94034K wps
[Epoch 45 Batch 90/162] avg loss 1.08608e-05, throughput 5.92207K wps
[Epoch 45 Batch 120/162] avg loss 1.063e-05, throughput 5.93022K wps
[Epoch 45 Batch 150/162] avg loss 1.41872e-05, throughput 5.93193K wps
Begin Testing...
[Epoch 45] train avg loss 1.27413e-05, test acc 0.9133, test avg loss 0.317034, throughput 5.95963K wps
[Epoch 46 Batch 30/162] avg loss 8.91444e-06, throughput 6.08947K wps
[Epoch 46 Batch 60/162] avg loss 1.12846e-05, throughput 5.93225K wps
[Epoch 46 Batch 90/162] avg loss 1.07385e-05, throughput 5.93594K wps
[Epoch 46 Batch 120/162] avg loss 1.08454e-05, throughput 5.93817K wps
[Epoch 46 Batch 150/162] avg loss 1.20607e-05, throughput 5.92364K wps
Begin Testing...
[Epoch 46] train avg loss 1.079e-05, test acc 0.9122, test avg loss 0.321192, throughput 5.96076K wps
[Epoch 47 Batch 30/162] avg loss 8.32173e-06, throughput 6.07966K wps
[Epoch 47 Batch 60/162] avg loss 1.06856e-05, throughput 5.92714K wps
[Epoch 47 Batch 90/162] avg loss 1.42733e-05, throughput 5.9375K wps
[Epoch 47 Batch 120/162] avg loss 9.05749e-06, throughput 5.94487K wps
[Epoch 47 Batch 150/162] avg loss 8.78076e-06, throughput 5.9437K wps
Begin Testing...
[Epoch 47] train avg loss 1.01596e-05, test acc 0.9144, test avg loss 0.322707, throughput 5.96496K wps
[Epoch 48 Batch 30/162] avg loss 1.1095e-05, throughput 6.08659K wps
[Epoch 48 Batch 60/162] avg loss 7.05347e-06, throughput 5.94173K wps
[Epoch 48 Batch 90/162] avg loss 1.1811e-05, throughput 5.93971K wps
[Epoch 48 Batch 120/162] avg loss 7.64699e-06, throughput 5.92755K wps
[Epoch 48 Batch 150/162] avg loss 9.80996e-06, throughput 5.91295K wps
Begin Testing...
[Epoch 48] train avg loss 9.46753e-06, test acc 0.9167, test avg loss 0.328446, throughput 5.95947K wps
[Epoch 49 Batch 30/162] avg loss 1.00984e-05, throughput 6.10782K wps
[Epoch 49 Batch 60/162] avg loss 1.31831e-05, throughput 5.94665K wps
[Epoch 49 Batch 90/162] avg loss 7.02739e-06, throughput 5.95772K wps
[Epoch 49 Batch 120/162] avg loss 7.10885e-06, throughput 5.94372K wps
[Epoch 49 Batch 150/162] avg loss 9.57626e-06, throughput 5.96243K wps
Begin Testing...
[Epoch 49] train avg loss 9.39946e-06, test acc 0.9133, test avg loss 0.325912, throughput 5.98161K wps
[Epoch 50 Batch 30/162] avg loss 7.10914e-06, throughput 6.08522K wps
[Epoch 50 Batch 60/162] avg loss 7.49718e-06, throughput 5.93724K wps
[Epoch 50 Batch 90/162] avg loss 6.21535e-06, throughput 5.94264K wps
[Epoch 50 Batch 120/162] avg loss 8.20591e-06, throughput 5.94807K wps
[Epoch 50 Batch 150/162] avg loss 9.01662e-06, throughput 5.94264K wps
Begin Testing...
[Epoch 50] train avg loss 7.77204e-06, test acc 0.9144, test avg loss 0.332995, throughput 5.96933K wps
[Epoch 51 Batch 30/162] avg loss 9.57471e-06, throughput 6.07828K wps
[Epoch 51 Batch 60/162] avg loss 6.9418e-06, throughput 5.94358K wps
[Epoch 51 Batch 90/162] avg loss 5.95158e-06, throughput 5.95053K wps
[Epoch 51 Batch 120/162] avg loss 8.1186e-06, throughput 5.94959K wps
[Epoch 51 Batch 150/162] avg loss 7.11041e-06, throughput 5.94077K wps
Begin Testing...
[Epoch 51] train avg loss 7.79111e-06, test acc 0.9156, test avg loss 0.336047, throughput 5.97005K wps
[Epoch 52 Batch 30/162] avg loss 6.63172e-06, throughput 6.07214K wps
[Epoch 52 Batch 60/162] avg loss 6.18217e-06, throughput 5.95751K wps
[Epoch 52 Batch 90/162] avg loss 5.92353e-06, throughput 5.95579K wps
[Epoch 52 Batch 120/162] avg loss 6.29251e-06, throughput 5.95391K wps
[Epoch 52 Batch 150/162] avg loss 7.12092e-06, throughput 5.93932K wps
Begin Testing...
[Epoch 52] train avg loss 6.35115e-06, test acc 0.9144, test avg loss 0.339633, throughput 5.9725K wps
[Epoch 53 Batch 30/162] avg loss 6.50104e-06, throughput 6.09771K wps
[Epoch 53 Batch 60/162] avg loss 5.3688e-06, throughput 5.94928K wps
[Epoch 53 Batch 90/162] avg loss 5.99349e-06, throughput 5.94441K wps
[Epoch 53 Batch 120/162] avg loss 4.86936e-06, throughput 5.94458K wps
[Epoch 53 Batch 150/162] avg loss 8.66447e-06, throughput 5.94325K wps
Begin Testing...
[Epoch 53] train avg loss 6.363e-06, test acc 0.9133, test avg loss 0.341265, throughput 5.97334K wps
[Epoch 54 Batch 30/162] avg loss 8.25036e-06, throughput 6.09609K wps
[Epoch 54 Batch 60/162] avg loss 6.04198e-06, throughput 5.94508K wps
[Epoch 54 Batch 90/162] avg loss 5.72059e-06, throughput 5.93626K wps
[Epoch 54 Batch 120/162] avg loss 5.79561e-06, throughput 5.93604K wps
[Epoch 54 Batch 150/162] avg loss 4.78139e-06, throughput 5.93416K wps
Begin Testing...
[Epoch 54] train avg loss 5.96524e-06, test acc 0.9133, test avg loss 0.347957, throughput 5.96688K wps
[Epoch 55 Batch 30/162] avg loss 4.29101e-06, throughput 6.09165K wps
[Epoch 55 Batch 60/162] avg loss 5.62965e-06, throughput 5.94473K wps
[Epoch 55 Batch 90/162] avg loss 3.43353e-06, throughput 5.95584K wps
[Epoch 55 Batch 120/162] avg loss 4.40024e-06, throughput 5.95897K wps
[Epoch 55 Batch 150/162] avg loss 4.37587e-06, throughput 5.94036K wps
Begin Testing...
[Epoch 55] train avg loss 4.36072e-06, test acc 0.9133, test avg loss 0.349695, throughput 5.97539K wps
[Epoch 56 Batch 30/162] avg loss 5.81815e-06, throughput 6.10423K wps
[Epoch 56 Batch 60/162] avg loss 5.54651e-06, throughput 5.94661K wps
[Epoch 56 Batch 90/162] avg loss 3.48782e-06, throughput 5.94946K wps
[Epoch 56 Batch 120/162] avg loss 3.82818e-06, throughput 5.95464K wps
[Epoch 56 Batch 150/162] avg loss 4.74293e-06, throughput 5.934K wps
Begin Testing...
[Epoch 56] train avg loss 4.58794e-06, test acc 0.9122, test avg loss 0.35087, throughput 5.9744K wps
[Epoch 57 Batch 30/162] avg loss 3.67007e-06, throughput 6.10647K wps
[Epoch 57 Batch 60/162] avg loss 3.28193e-06, throughput 5.95621K wps
[Epoch 57 Batch 90/162] avg loss 2.90653e-06, throughput 5.9487K wps
[Epoch 57 Batch 120/162] avg loss 5.66228e-06, throughput 5.95003K wps
[Epoch 57 Batch 150/162] avg loss 4.50464e-06, throughput 5.94662K wps
Begin Testing...
[Epoch 57] train avg loss 3.90209e-06, test acc 0.9122, test avg loss 0.352299, throughput 5.9795K wps
[Epoch 58 Batch 30/162] avg loss 3.42662e-06, throughput 6.08707K wps
[Epoch 58 Batch 60/162] avg loss 9.2783e-06, throughput 5.93923K wps
[Epoch 58 Batch 90/162] avg loss 7.92404e-06, throughput 5.94668K wps
[Epoch 58 Batch 120/162] avg loss 3.51429e-06, throughput 5.95543K wps
[Epoch 58 Batch 150/162] avg loss 3.98631e-06, throughput 5.95867K wps
Begin Testing...
[Epoch 58] train avg loss 5.75793e-06, test acc 0.9144, test avg loss 0.356029, throughput 5.97392K wps
[Epoch 59 Batch 30/162] avg loss 3.06548e-06, throughput 6.07689K wps
[Epoch 59 Batch 60/162] avg loss 3.49611e-06, throughput 5.9465K wps
[Epoch 59 Batch 90/162] avg loss 3.88108e-06, throughput 5.94419K wps
[Epoch 59 Batch 120/162] avg loss 4.28951e-06, throughput 5.95503K wps
[Epoch 59 Batch 150/162] avg loss 2.608e-06, throughput 5.93076K wps
Begin Testing...
[Epoch 59] train avg loss 3.63713e-06, test acc 0.9167, test avg loss 0.360175, throughput 5.96775K wps
Test loss 0.202495, test acc 0.9270
Total time cost 339.76s
9000 1000
[Epoch 0 Batch 30/162] avg loss 0.0153885, throughput 5.69346K wps
[Epoch 0 Batch 60/162] avg loss 0.0143946, throughput 5.92724K wps
[Epoch 0 Batch 90/162] avg loss 0.0138652, throughput 5.93743K wps
[Epoch 0 Batch 120/162] avg loss 0.013374, throughput 5.94094K wps
[Epoch 0 Batch 150/162] avg loss 0.0127184, throughput 5.94266K wps
Begin Testing...
[Epoch 0] train avg loss 0.0138678, test acc 0.7144, test avg loss 0.571652, throughput 5.89143K wps
Observed Improvement.
Begin Testing...
[Epoch 1 Batch 30/162] avg loss 0.0122858, throughput 6.10092K wps
[Epoch 1 Batch 60/162] avg loss 0.0116918, throughput 5.94671K wps
[Epoch 1 Batch 90/162] avg loss 0.0116402, throughput 5.95447K wps
[Epoch 1 Batch 120/162] avg loss 0.0114414, throughput 5.95621K wps
[Epoch 1 Batch 150/162] avg loss 0.0109733, throughput 5.93475K wps
Begin Testing...
[Epoch 1] train avg loss 0.0115352, test acc 0.8044, test avg loss 0.501565, throughput 5.97559K wps
Observed Improvement.
Begin Testing...
[Epoch 2 Batch 30/162] avg loss 0.0104058, throughput 6.09292K wps
[Epoch 2 Batch 60/162] avg loss 0.00990563, throughput 5.93866K wps
[Epoch 2 Batch 90/162] avg loss 0.00964629, throughput 5.92646K wps
[Epoch 2 Batch 120/162] avg loss 0.00928483, throughput 5.94329K wps
[Epoch 2 Batch 150/162] avg loss 0.00894901, throughput 5.93724K wps
Begin Testing...
[Epoch 2] train avg loss 0.00956537, test acc 0.8644, test avg loss 0.425113, throughput 5.96476K wps
Observed Improvement.
Begin Testing...
[Epoch 3 Batch 30/162] avg loss 0.00833236, throughput 6.09173K wps
[Epoch 3 Batch 60/162] avg loss 0.00804251, throughput 5.92754K wps
[Epoch 3 Batch 90/162] avg loss 0.00789487, throughput 5.94713K wps
[Epoch 3 Batch 120/162] avg loss 0.00737418, throughput 5.9548K wps
[Epoch 3 Batch 150/162] avg loss 0.00684163, throughput 5.94078K wps
Begin Testing...
[Epoch 3] train avg loss 0.0076627, test acc 0.8911, test avg loss 0.34564, throughput 5.96953K wps
Observed Improvement.
Begin Testing...
[Epoch 4 Batch 30/162] avg loss 0.00673086, throughput 6.08807K wps
[Epoch 4 Batch 60/162] avg loss 0.00613294, throughput 5.93401K wps
[Epoch 4 Batch 90/162] avg loss 0.0059191, throughput 5.94476K wps
[Epoch 4 Batch 120/162] avg loss 0.00605336, throughput 5.95383K wps
[Epoch 4 Batch 150/162] avg loss 0.00558074, throughput 5.94228K wps
Begin Testing...
[Epoch 4] train avg loss 0.00607109, test acc 0.9044, test avg loss 0.294231, throughput 5.96971K wps
Observed Improvement.
Begin Testing...
[Epoch 5 Batch 30/162] avg loss 0.00489092, throughput 6.10052K wps
[Epoch 5 Batch 60/162] avg loss 0.00504407, throughput 5.93986K wps
[Epoch 5 Batch 90/162] avg loss 0.00463182, throughput 5.92579K wps
[Epoch 5 Batch 120/162] avg loss 0.00483392, throughput 5.92636K wps
[Epoch 5 Batch 150/162] avg loss 0.00472981, throughput 5.93173K wps
Begin Testing...
[Epoch 5] train avg loss 0.00483901, test acc 0.9122, test avg loss 0.262906, throughput 5.96249K wps
Observed Improvement.
Begin Testing...
[Epoch 6 Batch 30/162] avg loss 0.00408423, throughput 6.08959K wps
[Epoch 6 Batch 60/162] avg loss 0.0040187, throughput 5.94323K wps
[Epoch 6 Batch 90/162] avg loss 0.00401831, throughput 5.94046K wps
[Epoch 6 Batch 120/162] avg loss 0.00413243, throughput 5.94212K wps
[Epoch 6 Batch 150/162] avg loss 0.00373338, throughput 5.94001K wps
Begin Testing...
[Epoch 6] train avg loss 0.00399029, test acc 0.9122, test avg loss 0.238111, throughput 5.96876K wps
Observed Improvement.
Begin Testing...
[Epoch 7 Batch 30/162] avg loss 0.00342325, throughput 6.07468K wps
[Epoch 7 Batch 60/162] avg loss 0.00324082, throughput 5.94707K wps
[Epoch 7 Batch 90/162] avg loss 0.00322428, throughput 5.94918K wps
[Epoch 7 Batch 120/162] avg loss 0.00344883, throughput 5.93718K wps
[Epoch 7 Batch 150/162] avg loss 0.0035217, throughput 5.94191K wps
Begin Testing...
[Epoch 7] train avg loss 0.00334861, test acc 0.9156, test avg loss 0.223907, throughput 5.96793K wps
Observed Improvement.
Begin Testing...
[Epoch 8 Batch 30/162] avg loss 0.00287248, throughput 6.10167K wps
[Epoch 8 Batch 60/162] avg loss 0.00262071, throughput 5.93115K wps
[Epoch 8 Batch 90/162] avg loss 0.00303506, throughput 5.94642K wps
[Epoch 8 Batch 120/162] avg loss 0.002799, throughput 5.93459K wps
[Epoch 8 Batch 150/162] avg loss 0.00271069, throughput 5.92888K wps
Begin Testing...
[Epoch 8] train avg loss 0.00279358, test acc 0.9122, test avg loss 0.216372, throughput 5.96557K wps
[Epoch 9 Batch 30/162] avg loss 0.00256002, throughput 6.08845K wps
[Epoch 9 Batch 60/162] avg loss 0.00241918, throughput 5.93538K wps
[Epoch 9 Batch 90/162] avg loss 0.00244562, throughput 5.93984K wps
[Epoch 9 Batch 120/162] avg loss 0.00233862, throughput 5.94155K wps
[Epoch 9 Batch 150/162] avg loss 0.00211703, throughput 5.93379K wps
Begin Testing...
[Epoch 9] train avg loss 0.002373, test acc 0.9133, test avg loss 0.206738, throughput 5.96424K wps
[Epoch 10 Batch 30/162] avg loss 0.00213926, throughput 6.10093K wps
[Epoch 10 Batch 60/162] avg loss 0.00187399, throughput 5.93957K wps
[Epoch 10 Batch 90/162] avg loss 0.00203293, throughput 5.94314K wps
[Epoch 10 Batch 120/162] avg loss 0.00196963, throughput 5.93482K wps
[Epoch 10 Batch 150/162] avg loss 0.00218295, throughput 5.93454K wps
Begin Testing...
[Epoch 10] train avg loss 0.00202407, test acc 0.9156, test avg loss 0.203738, throughput 5.96646K wps
Observed Improvement.
Begin Testing...
[Epoch 11 Batch 30/162] avg loss 0.00173428, throughput 6.07367K wps
[Epoch 11 Batch 60/162] avg loss 0.0015524, throughput 5.94509K wps
[Epoch 11 Batch 90/162] avg loss 0.00152228, throughput 5.94822K wps
[Epoch 11 Batch 120/162] avg loss 0.00189975, throughput 5.94609K wps
[Epoch 11 Batch 150/162] avg loss 0.00167757, throughput 5.93999K wps
Begin Testing...
[Epoch 11] train avg loss 0.00169739, test acc 0.9189, test avg loss 0.200659, throughput 5.9679K wps
Observed Improvement.
Begin Testing...
[Epoch 12 Batch 30/162] avg loss 0.00144712, throughput 6.06468K wps
[Epoch 12 Batch 60/162] avg loss 0.00141937, throughput 5.94996K wps
[Epoch 12 Batch 90/162] avg loss 0.00138005, throughput 5.93408K wps
[Epoch 12 Batch 120/162] avg loss 0.00140547, throughput 5.93836K wps
[Epoch 12 Batch 150/162] avg loss 0.00142766, throughput 5.93157K wps
Begin Testing...
[Epoch 12] train avg loss 0.00141052, test acc 0.9200, test avg loss 0.200583, throughput 5.96138K wps
Observed Improvement.
Begin Testing...
[Epoch 13 Batch 30/162] avg loss 0.00129098, throughput 6.0861K wps
[Epoch 13 Batch 60/162] avg loss 0.0012017, throughput 5.92922K wps
[Epoch 13 Batch 90/162] avg loss 0.00113356, throughput 5.94416K wps
[Epoch 13 Batch 120/162] avg loss 0.00119714, throughput 5.95673K wps
[Epoch 13 Batch 150/162] avg loss 0.00117394, throughput 5.95003K wps
Begin Testing...
[Epoch 13] train avg loss 0.00120481, test acc 0.9111, test avg loss 0.201909, throughput 5.96967K wps
[Epoch 14 Batch 30/162] avg loss 0.00100036, throughput 6.09912K wps
[Epoch 14 Batch 60/162] avg loss 0.0010362, throughput 5.96219K wps
[Epoch 14 Batch 90/162] avg loss 0.000959311, throughput 5.95996K wps
[Epoch 14 Batch 120/162] avg loss 0.000787188, throughput 5.95143K wps
[Epoch 14 Batch 150/162] avg loss 0.000907301, throughput 5.95899K wps
Begin Testing...
[Epoch 14] train avg loss 0.000972434, test acc 0.9089, test avg loss 0.204951, throughput 5.98352K wps
[Epoch 15 Batch 30/162] avg loss 0.000829349, throughput 6.10017K wps
[Epoch 15 Batch 60/162] avg loss 0.000808268, throughput 5.94379K wps
[Epoch 15 Batch 90/162] avg loss 0.000861701, throughput 5.95237K wps
[Epoch 15 Batch 120/162] avg loss 0.000864447, throughput 5.95891K wps
[Epoch 15 Batch 150/162] avg loss 0.000854766, throughput 5.9581K wps
Begin Testing...
[Epoch 15] train avg loss 0.000828983, test acc 0.9056, test avg loss 0.207966, throughput 5.98026K wps
[Epoch 16 Batch 30/162] avg loss 0.000742777, throughput 6.09051K wps
[Epoch 16 Batch 60/162] avg loss 0.000672351, throughput 5.95786K wps
[Epoch 16 Batch 90/162] avg loss 0.0007035, throughput 5.94525K wps
[Epoch 16 Batch 120/162] avg loss 0.000756539, throughput 5.95591K wps
[Epoch 16 Batch 150/162] avg loss 0.000665033, throughput 5.94597K wps
Begin Testing...
[Epoch 16] train avg loss 0.000706618, test acc 0.9100, test avg loss 0.208418, throughput 5.97624K wps
[Epoch 17 Batch 30/162] avg loss 0.000533665, throughput 6.10626K wps
[Epoch 17 Batch 60/162] avg loss 0.00063879, throughput 5.94368K wps
[Epoch 17 Batch 90/162] avg loss 0.000539104, throughput 5.94593K wps
[Epoch 17 Batch 120/162] avg loss 0.000587054, throughput 5.9356K wps
[Epoch 17 Batch 150/162] avg loss 0.000604405, throughput 5.95652K wps
Begin Testing...
[Epoch 17] train avg loss 0.000583733, test acc 0.9144, test avg loss 0.209822, throughput 5.97491K wps
[Epoch 18 Batch 30/162] avg loss 0.000490538, throughput 6.09406K wps
[Epoch 18 Batch 60/162] avg loss 0.000496974, throughput 5.94469K wps
[Epoch 18 Batch 90/162] avg loss 0.000520746, throughput 5.95085K wps
[Epoch 18 Batch 120/162] avg loss 0.000484213, throughput 5.94098K wps
[Epoch 18 Batch 150/162] avg loss 0.000549017, throughput 5.95583K wps
Begin Testing...
[Epoch 18] train avg loss 0.000508598, test acc 0.9067, test avg loss 0.216709, throughput 5.97437K wps
[Epoch 19 Batch 30/162] avg loss 0.000455171, throughput 6.09053K wps
[Epoch 19 Batch 60/162] avg loss 0.000422574, throughput 5.93851K wps
[Epoch 19 Batch 90/162] avg loss 0.000541472, throughput 5.96335K wps
[Epoch 19 Batch 120/162] avg loss 0.000345523, throughput 5.93899K wps
[Epoch 19 Batch 150/162] avg loss 0.000460613, throughput 5.93614K wps
Begin Testing...
[Epoch 19] train avg loss 0.000438409, test acc 0.9056, test avg loss 0.222816, throughput 5.97131K wps
[Epoch 20 Batch 30/162] avg loss 0.000347192, throughput 6.08701K wps
[Epoch 20 Batch 60/162] avg loss 0.000446231, throughput 5.92729K wps
[Epoch 20 Batch 90/162] avg loss 0.00037083, throughput 5.88166K wps
[Epoch 20 Batch 120/162] avg loss 0.000292695, throughput 5.92026K wps
[Epoch 20 Batch 150/162] avg loss 0.000371413, throughput 5.94719K wps
Begin Testing...
[Epoch 20] train avg loss 0.000369523, test acc 0.9133, test avg loss 0.226629, throughput 5.94979K wps
[Epoch 21 Batch 30/162] avg loss 0.000280284, throughput 6.09916K wps
[Epoch 21 Batch 60/162] avg loss 0.000310572, throughput 5.9428K wps
[Epoch 21 Batch 90/162] avg loss 0.000320457, throughput 5.9402K wps
[Epoch 21 Batch 120/162] avg loss 0.000306482, throughput 5.94704K wps
[Epoch 21 Batch 150/162] avg loss 0.000342964, throughput 5.95296K wps
Begin Testing...
[Epoch 21] train avg loss 0.000313373, test acc 0.9044, test avg loss 0.228795, throughput 5.97436K wps
[Epoch 22 Batch 30/162] avg loss 0.000258743, throughput 6.08591K wps
[Epoch 22 Batch 60/162] avg loss 0.000256165, throughput 5.9344K wps
[Epoch 22 Batch 90/162] avg loss 0.000287589, throughput 5.94753K wps
[Epoch 22 Batch 120/162] avg loss 0.00030438, throughput 5.94766K wps
[Epoch 22 Batch 150/162] avg loss 0.00027574, throughput 5.95181K wps
Begin Testing...
[Epoch 22] train avg loss 0.000270575, test acc 0.9089, test avg loss 0.234216, throughput 5.97217K wps
[Epoch 23 Batch 30/162] avg loss 0.000251649, throughput 6.08664K wps
[Epoch 23 Batch 60/162] avg loss 0.000239694, throughput 5.93584K wps
[Epoch 23 Batch 90/162] avg loss 0.000233956, throughput 5.93509K wps
[Epoch 23 Batch 120/162] avg loss 0.000246855, throughput 5.9482K wps
[Epoch 23 Batch 150/162] avg loss 0.000271645, throughput 5.94584K wps
Begin Testing...
[Epoch 23] train avg loss 0.000243616, test acc 0.9000, test avg loss 0.241097, throughput 5.96725K wps
[Epoch 24 Batch 30/162] avg loss 0.000217112, throughput 6.08427K wps
[Epoch 24 Batch 60/162] avg loss 0.000153437, throughput 5.93867K wps
[Epoch 24 Batch 90/162] avg loss 0.000189887, throughput 5.95302K wps
[Epoch 24 Batch 120/162] avg loss 0.00017286, throughput 5.94791K wps
[Epoch 24 Batch 150/162] avg loss 0.000214225, throughput 5.93679K wps
Begin Testing...
[Epoch 24] train avg loss 0.000188978, test acc 0.8989, test avg loss 0.244575, throughput 5.96956K wps
[Epoch 25 Batch 30/162] avg loss 0.000162445, throughput 6.08696K wps
[Epoch 25 Batch 60/162] avg loss 0.000167069, throughput 5.93706K wps
[Epoch 25 Batch 90/162] avg loss 0.000171273, throughput 5.94828K wps
[Epoch 25 Batch 120/162] avg loss 0.000184307, throughput 5.95138K wps
[Epoch 25 Batch 150/162] avg loss 0.000176354, throughput 5.94632K wps
Begin Testing...
[Epoch 25] train avg loss 0.0001707, test acc 0.9089, test avg loss 0.248018, throughput 5.97171K wps
[Epoch 26 Batch 30/162] avg loss 0.000124177, throughput 6.08874K wps
[Epoch 26 Batch 60/162] avg loss 0.000136705, throughput 5.9458K wps
[Epoch 26 Batch 90/162] avg loss 0.000135791, throughput 5.92637K wps
[Epoch 26 Batch 120/162] avg loss 0.00014232, throughput 5.93415K wps
[Epoch 26 Batch 150/162] avg loss 0.000148954, throughput 5.92376K wps
Begin Testing...
[Epoch 26] train avg loss 0.000136176, test acc 0.9089, test avg loss 0.251992, throughput 5.96043K wps
[Epoch 27 Batch 30/162] avg loss 0.000119034, throughput 6.08891K wps
[Epoch 27 Batch 60/162] avg loss 0.000129178, throughput 5.93868K wps
[Epoch 27 Batch 90/162] avg loss 0.00010413, throughput 5.92403K wps
[Epoch 27 Batch 120/162] avg loss 0.000135458, throughput 5.93621K wps
[Epoch 27 Batch 150/162] avg loss 0.00012974, throughput 5.94869K wps
Begin Testing...
[Epoch 27] train avg loss 0.000125798, test acc 0.9044, test avg loss 0.255797, throughput 5.96476K wps
[Epoch 28 Batch 30/162] avg loss 9.78328e-05, throughput 6.08102K wps
[Epoch 28 Batch 60/162] avg loss 0.000111919, throughput 5.92827K wps
[Epoch 28 Batch 90/162] avg loss 9.66563e-05, throughput 5.93985K wps
[Epoch 28 Batch 120/162] avg loss 0.000103082, throughput 5.94152K wps
[Epoch 28 Batch 150/162] avg loss 0.000141219, throughput 5.93072K wps
Begin Testing...
[Epoch 28] train avg loss 0.000110617, test acc 0.9078, test avg loss 0.256935, throughput 5.96206K wps
[Epoch 29 Batch 30/162] avg loss 9.64437e-05, throughput 6.07232K wps
[Epoch 29 Batch 60/162] avg loss 0.000119967, throughput 5.92769K wps
[Epoch 29 Batch 90/162] avg loss 7.70633e-05, throughput 5.92553K wps
[Epoch 29 Batch 120/162] avg loss 0.000100557, throughput 5.92887K wps
[Epoch 29 Batch 150/162] avg loss 9.853e-05, throughput 5.92611K wps
Begin Testing...
[Epoch 29] train avg loss 9.6595e-05, test acc 0.9000, test avg loss 0.267072, throughput 5.95363K wps
[Epoch 30 Batch 30/162] avg loss 8.1951e-05, throughput 6.08475K wps
[Epoch 30 Batch 60/162] avg loss 7.85755e-05, throughput 5.93267K wps
[Epoch 30 Batch 90/162] avg loss 8.91637e-05, throughput 5.92347K wps
[Epoch 30 Batch 120/162] avg loss 8.00666e-05, throughput 5.94101K wps
[Epoch 30 Batch 150/162] avg loss 7.55003e-05, throughput 5.94909K wps
Begin Testing...
[Epoch 30] train avg loss 8.10157e-05, test acc 0.9022, test avg loss 0.269529, throughput 5.96306K wps
[Epoch 31 Batch 30/162] avg loss 6.85484e-05, throughput 6.08793K wps
[Epoch 31 Batch 60/162] avg loss 7.3752e-05, throughput 5.94306K wps
[Epoch 31 Batch 90/162] avg loss 6.65765e-05, throughput 5.95328K wps
[Epoch 31 Batch 120/162] avg loss 7.64321e-05, throughput 5.93608K wps
[Epoch 31 Batch 150/162] avg loss 8.87312e-05, throughput 5.93094K wps
Begin Testing...
[Epoch 31] train avg loss 7.44565e-05, test acc 0.9089, test avg loss 0.276285, throughput 5.96776K wps
[Epoch 32 Batch 30/162] avg loss 7.90853e-05, throughput 6.08843K wps
[Epoch 32 Batch 60/162] avg loss 6.84136e-05, throughput 5.93979K wps
[Epoch 32 Batch 90/162] avg loss 6.61475e-05, throughput 5.93904K wps
[Epoch 32 Batch 120/162] avg loss 5.9191e-05, throughput 5.9327K wps
[Epoch 32 Batch 150/162] avg loss 5.84655e-05, throughput 5.93964K wps
Begin Testing...
[Epoch 32] train avg loss 6.49319e-05, test acc 0.9067, test avg loss 0.279974, throughput 5.96621K wps
[Epoch 33 Batch 30/162] avg loss 4.8765e-05, throughput 6.09449K wps
[Epoch 33 Batch 60/162] avg loss 6.03645e-05, throughput 5.92444K wps
[Epoch 33 Batch 90/162] avg loss 5.97251e-05, throughput 5.94379K wps
[Epoch 33 Batch 120/162] avg loss 4.74553e-05, throughput 5.9485K wps
[Epoch 33 Batch 150/162] avg loss 5.40687e-05, throughput 5.9413K wps
Begin Testing...
[Epoch 33] train avg loss 5.32534e-05, test acc 0.9033, test avg loss 0.285124, throughput 5.96614K wps
[Epoch 34 Batch 30/162] avg loss 4.63048e-05, throughput 6.08153K wps
[Epoch 34 Batch 60/162] avg loss 5.0729e-05, throughput 5.92375K wps
[Epoch 34 Batch 90/162] avg loss 4.86178e-05, throughput 5.94223K wps
[Epoch 34 Batch 120/162] avg loss 4.66503e-05, throughput 5.9475K wps
[Epoch 34 Batch 150/162] avg loss 5.42705e-05, throughput 5.9345K wps
Begin Testing...
[Epoch 34] train avg loss 4.9783e-05, test acc 0.9000, test avg loss 0.291474, throughput 5.96389K wps
[Epoch 35 Batch 30/162] avg loss 4.11271e-05, throughput 6.09091K wps
[Epoch 35 Batch 60/162] avg loss 5.1002e-05, throughput 5.93661K wps
[Epoch 35 Batch 90/162] avg loss 4.74133e-05, throughput 5.93197K wps
[Epoch 35 Batch 120/162] avg loss 3.92489e-05, throughput 5.94154K wps
[Epoch 35 Batch 150/162] avg loss 4.07532e-05, throughput 5.94573K wps
Begin Testing...
[Epoch 35] train avg loss 4.42619e-05, test acc 0.9044, test avg loss 0.293945, throughput 5.96706K wps
[Epoch 36 Batch 30/162] avg loss 3.75595e-05, throughput 6.08782K wps
[Epoch 36 Batch 60/162] avg loss 3.75913e-05, throughput 5.94377K wps
[Epoch 36 Batch 90/162] avg loss 4.734e-05, throughput 5.94178K wps
[Epoch 36 Batch 120/162] avg loss 3.20043e-05, throughput 5.93537K wps
[Epoch 36 Batch 150/162] avg loss 4.05323e-05, throughput 5.93225K wps
Begin Testing...
[Epoch 36] train avg loss 3.82958e-05, test acc 0.8989, test avg loss 0.30062, throughput 5.96674K wps
[Epoch 37 Batch 30/162] avg loss 3.13626e-05, throughput 6.0898K wps
[Epoch 37 Batch 60/162] avg loss 3.20466e-05, throughput 5.95195K wps
[Epoch 37 Batch 90/162] avg loss 3.778e-05, throughput 5.94752K wps
[Epoch 37 Batch 120/162] avg loss 3.33936e-05, throughput 5.9509K wps
[Epoch 37 Batch 150/162] avg loss 3.37018e-05, throughput 5.95336K wps
Begin Testing...
[Epoch 37] train avg loss 3.42687e-05, test acc 0.9033, test avg loss 0.30283, throughput 5.97525K wps
[Epoch 38 Batch 30/162] avg loss 2.92412e-05, throughput 6.09214K wps
[Epoch 38 Batch 60/162] avg loss 2.27574e-05, throughput 5.93308K wps
[Epoch 38 Batch 90/162] avg loss 2.60602e-05, throughput 5.93687K wps
[Epoch 38 Batch 120/162] avg loss 4.15462e-05, throughput 5.93225K wps
[Epoch 38 Batch 150/162] avg loss 2.59838e-05, throughput 5.93848K wps
Begin Testing...
[Epoch 38] train avg loss 2.86612e-05, test acc 0.9056, test avg loss 0.304505, throughput 5.96402K wps
[Epoch 39 Batch 30/162] avg loss 2.5335e-05, throughput 6.07717K wps
[Epoch 39 Batch 60/162] avg loss 2.36942e-05, throughput 5.93235K wps
[Epoch 39 Batch 90/162] avg loss 2.87576e-05, throughput 5.93303K wps
[Epoch 39 Batch 120/162] avg loss 3.31468e-05, throughput 5.93233K wps
[Epoch 39 Batch 150/162] avg loss 3.18003e-05, throughput 5.95422K wps
Begin Testing...
[Epoch 39] train avg loss 2.83319e-05, test acc 0.9022, test avg loss 0.310734, throughput 5.96321K wps
[Epoch 40 Batch 30/162] avg loss 2.21495e-05, throughput 6.09223K wps
[Epoch 40 Batch 60/162] avg loss 2.45264e-05, throughput 5.92642K wps
[Epoch 40 Batch 90/162] avg loss 2.88629e-05, throughput 5.9373K wps
[Epoch 40 Batch 120/162] avg loss 2.52293e-05, throughput 5.94315K wps
[Epoch 40 Batch 150/162] avg loss 2.16928e-05, throughput 5.93676K wps
Begin Testing...
[Epoch 40] train avg loss 2.46784e-05, test acc 0.9067, test avg loss 0.312415, throughput 5.96374K wps
[Epoch 41 Batch 30/162] avg loss 2.29629e-05, throughput 6.07897K wps
[Epoch 41 Batch 60/162] avg loss 2.42151e-05, throughput 5.94881K wps
[Epoch 41 Batch 90/162] avg loss 2.0505e-05, throughput 5.94203K wps
[Epoch 41 Batch 120/162] avg loss 2.90552e-05, throughput 5.94363K wps
[Epoch 41 Batch 150/162] avg loss 2.1541e-05, throughput 5.94948K wps
Begin Testing...
[Epoch 41] train avg loss 2.37405e-05, test acc 0.9044, test avg loss 0.316624, throughput 5.9706K wps
[Epoch 42 Batch 30/162] avg loss 2.08691e-05, throughput 6.09761K wps
[Epoch 42 Batch 60/162] avg loss 2.02679e-05, throughput 5.94422K wps
[Epoch 42 Batch 90/162] avg loss 1.95946e-05, throughput 5.94333K wps
[Epoch 42 Batch 120/162] avg loss 1.95568e-05, throughput 5.9447K wps
[Epoch 42 Batch 150/162] avg loss 1.82418e-05, throughput 5.94471K wps
Begin Testing...
[Epoch 42] train avg loss 1.98176e-05, test acc 0.9067, test avg loss 0.32075, throughput 5.97005K wps
[Epoch 43 Batch 30/162] avg loss 2.63845e-05, throughput 6.10066K wps
[Epoch 43 Batch 60/162] avg loss 1.55413e-05, throughput 5.93262K wps
[Epoch 43 Batch 90/162] avg loss 1.83272e-05, throughput 5.9405K wps
[Epoch 43 Batch 120/162] avg loss 1.56739e-05, throughput 5.9382K wps
[Epoch 43 Batch 150/162] avg loss 1.61263e-05, throughput 5.95265K wps
Begin Testing...
[Epoch 43] train avg loss 1.81039e-05, test acc 0.9056, test avg loss 0.325986, throughput 5.96988K wps
[Epoch 44 Batch 30/162] avg loss 1.4479e-05, throughput 6.0967K wps
[Epoch 44 Batch 60/162] avg loss 1.44698e-05, throughput 5.93191K wps
[Epoch 44 Batch 90/162] avg loss 2.27356e-05, throughput 5.94263K wps
[Epoch 44 Batch 120/162] avg loss 1.98871e-05, throughput 5.94451K wps
[Epoch 44 Batch 150/162] avg loss 1.55021e-05, throughput 5.95011K wps
Begin Testing...
[Epoch 44] train avg loss 1.76392e-05, test acc 0.9044, test avg loss 0.330397, throughput 5.96906K wps
[Epoch 45 Batch 30/162] avg loss 1.16576e-05, throughput 6.07155K wps
[Epoch 45 Batch 60/162] avg loss 1.37108e-05, throughput 5.94748K wps
[Epoch 45 Batch 90/162] avg loss 1.86163e-05, throughput 5.95664K wps
[Epoch 45 Batch 120/162] avg loss 1.27485e-05, throughput 5.94773K wps
[Epoch 45 Batch 150/162] avg loss 1.35507e-05, throughput 5.95137K wps
Begin Testing...
[Epoch 45] train avg loss 1.405e-05, test acc 0.9022, test avg loss 0.333607, throughput 5.97217K wps
[Epoch 46 Batch 30/162] avg loss 1.24472e-05, throughput 6.08028K wps
[Epoch 46 Batch 60/162] avg loss 1.4075e-05, throughput 5.95717K wps
[Epoch 46 Batch 90/162] avg loss 1.27315e-05, throughput 5.94695K wps
[Epoch 46 Batch 120/162] avg loss 1.06195e-05, throughput 5.95709K wps
[Epoch 46 Batch 150/162] avg loss 1.06869e-05, throughput 5.95854K wps
Begin Testing...
[Epoch 46] train avg loss 1.19692e-05, test acc 0.9044, test avg loss 0.336616, throughput 5.97701K wps
[Epoch 47 Batch 30/162] avg loss 1.12904e-05, throughput 6.08599K wps
[Epoch 47 Batch 60/162] avg loss 1.12981e-05, throughput 5.93355K wps
[Epoch 47 Batch 90/162] avg loss 1.62694e-05, throughput 5.95142K wps
[Epoch 47 Batch 120/162] avg loss 1.43113e-05, throughput 5.95015K wps
[Epoch 47 Batch 150/162] avg loss 1.27269e-05, throughput 5.95787K wps
Begin Testing...
[Epoch 47] train avg loss 1.31747e-05, test acc 0.9078, test avg loss 0.33969, throughput 5.97371K wps
[Epoch 48 Batch 30/162] avg loss 8.58868e-06, throughput 6.09944K wps
[Epoch 48 Batch 60/162] avg loss 1.7805e-05, throughput 5.95036K wps
[Epoch 48 Batch 90/162] avg loss 9.86689e-06, throughput 5.94402K wps
[Epoch 48 Batch 120/162] avg loss 9.22692e-06, throughput 5.94443K wps
[Epoch 48 Batch 150/162] avg loss 1.27848e-05, throughput 5.94555K wps
Begin Testing...
[Epoch 48] train avg loss 1.16149e-05, test acc 0.9033, test avg loss 0.345095, throughput 5.9735K wps
[Epoch 49 Batch 30/162] avg loss 7.50119e-06, throughput 6.07729K wps
[Epoch 49 Batch 60/162] avg loss 9.86599e-06, throughput 5.93852K wps
[Epoch 49 Batch 90/162] avg loss 1.14484e-05, throughput 5.93775K wps
[Epoch 49 Batch 120/162] avg loss 9.58262e-06, throughput 5.93115K wps
[Epoch 49 Batch 150/162] avg loss 8.76503e-06, throughput 5.95618K wps
Begin Testing...
[Epoch 49] train avg loss 9.42851e-06, test acc 0.9022, test avg loss 0.350823, throughput 5.96688K wps
[Epoch 50 Batch 30/162] avg loss 8.74088e-06, throughput 6.08422K wps
[Epoch 50 Batch 60/162] avg loss 8.35579e-06, throughput 5.94766K wps
[Epoch 50 Batch 90/162] avg loss 1.42916e-05, throughput 5.94888K wps
[Epoch 50 Batch 120/162] avg loss 8.2699e-06, throughput 5.94732K wps
[Epoch 50 Batch 150/162] avg loss 7.48254e-06, throughput 5.93837K wps
Begin Testing...
[Epoch 50] train avg loss 9.39472e-06, test acc 0.9044, test avg loss 0.352013, throughput 5.97149K wps
[Epoch 51 Batch 30/162] avg loss 1.17361e-05, throughput 6.08535K wps
[Epoch 51 Batch 60/162] avg loss 6.53145e-06, throughput 5.9425K wps
[Epoch 51 Batch 90/162] avg loss 7.64105e-06, throughput 5.93594K wps
[Epoch 51 Batch 120/162] avg loss 7.00927e-06, throughput 5.93173K wps
[Epoch 51 Batch 150/162] avg loss 1.04611e-05, throughput 5.94161K wps
Begin Testing...
[Epoch 51] train avg loss 8.57544e-06, test acc 0.9000, test avg loss 0.360032, throughput 5.96394K wps
[Epoch 52 Batch 30/162] avg loss 8.30445e-06, throughput 6.09473K wps
[Epoch 52 Batch 60/162] avg loss 7.13437e-06, throughput 5.94192K wps
[Epoch 52 Batch 90/162] avg loss 6.35939e-06, throughput 5.92458K wps
[Epoch 52 Batch 120/162] avg loss 6.53065e-06, throughput 5.92611K wps
[Epoch 52 Batch 150/162] avg loss 6.86704e-06, throughput 5.95105K wps
Begin Testing...
[Epoch 52] train avg loss 7.07254e-06, test acc 0.9067, test avg loss 0.358519, throughput 5.96426K wps
[Epoch 53 Batch 30/162] avg loss 5.71501e-06, throughput 6.0813K wps
[Epoch 53 Batch 60/162] avg loss 5.27478e-06, throughput 5.92878K wps
[Epoch 53 Batch 90/162] avg loss 8.31036e-06, throughput 5.95131K wps
[Epoch 53 Batch 120/162] avg loss 6.03204e-06, throughput 5.94347K wps
[Epoch 53 Batch 150/162] avg loss 7.443e-06, throughput 5.93047K wps
Begin Testing...
[Epoch 53] train avg loss 6.47883e-06, test acc 0.9033, test avg loss 0.366191, throughput 5.96419K wps
[Epoch 54 Batch 30/162] avg loss 5.93493e-06, throughput 6.07451K wps
[Epoch 54 Batch 60/162] avg loss 1.11661e-05, throughput 5.94203K wps
[Epoch 54 Batch 90/162] avg loss 5.5749e-06, throughput 5.94212K wps
[Epoch 54 Batch 120/162] avg loss 7.13425e-06, throughput 5.94741K wps
[Epoch 54 Batch 150/162] avg loss 5.29118e-06, throughput 5.94003K wps
Begin Testing...
[Epoch 54] train avg loss 7.38292e-06, test acc 0.9044, test avg loss 0.370054, throughput 5.9672K wps
[Epoch 55 Batch 30/162] avg loss 6.02864e-06, throughput 6.09271K wps
[Epoch 55 Batch 60/162] avg loss 4.13013e-06, throughput 5.93645K wps
[Epoch 55 Batch 90/162] avg loss 4.68883e-06, throughput 5.93983K wps
[Epoch 55 Batch 120/162] avg loss 5.40548e-06, throughput 5.9467K wps
[Epoch 55 Batch 150/162] avg loss 6.72596e-06, throughput 5.95038K wps
Begin Testing...
[Epoch 55] train avg loss 5.31272e-06, test acc 0.9078, test avg loss 0.371219, throughput 5.96986K wps
[Epoch 56 Batch 30/162] avg loss 4.42385e-06, throughput 6.1083K wps
[Epoch 56 Batch 60/162] avg loss 5.29783e-06, throughput 5.94743K wps
[Epoch 56 Batch 90/162] avg loss 3.43552e-06, throughput 5.94371K wps
[Epoch 56 Batch 120/162] avg loss 7.3051e-06, throughput 5.9378K wps
[Epoch 56 Batch 150/162] avg loss 5.72966e-06, throughput 5.93497K wps
Begin Testing...
[Epoch 56] train avg loss 5.26185e-06, test acc 0.9067, test avg loss 0.374117, throughput 5.97045K wps
[Epoch 57 Batch 30/162] avg loss 4.78426e-06, throughput 6.10228K wps
[Epoch 57 Batch 60/162] avg loss 6.52593e-06, throughput 5.95933K wps
[Epoch 57 Batch 90/162] avg loss 5.31508e-06, throughput 5.95647K wps
[Epoch 57 Batch 120/162] avg loss 4.47035e-06, throughput 5.94691K wps
[Epoch 57 Batch 150/162] avg loss 4.96308e-06, throughput 5.94319K wps
Begin Testing...
[Epoch 57] train avg loss 5.17841e-06, test acc 0.9078, test avg loss 0.381356, throughput 5.97784K wps
[Epoch 58 Batch 30/162] avg loss 3.43909e-06, throughput 6.09882K wps
[Epoch 58 Batch 60/162] avg loss 4.23203e-06, throughput 5.96004K wps
[Epoch 58 Batch 90/162] avg loss 4.66542e-06, throughput 5.94929K wps
[Epoch 58 Batch 120/162] avg loss 3.80046e-06, throughput 5.94608K wps
[Epoch 58 Batch 150/162] avg loss 2.94723e-06, throughput 5.9424K wps
Begin Testing...
[Epoch 58] train avg loss 3.81512e-06, test acc 0.9056, test avg loss 0.384096, throughput 5.97445K wps
[Epoch 59 Batch 30/162] avg loss 3.28402e-06, throughput 6.08795K wps
[Epoch 59 Batch 60/162] avg loss 4.35333e-06, throughput 5.93087K wps
[Epoch 59 Batch 90/162] avg loss 5.25511e-06, throughput 5.93384K wps
[Epoch 59 Batch 120/162] avg loss 4.3124e-06, throughput 5.92564K wps
[Epoch 59 Batch 150/162] avg loss 4.54204e-06, throughput 5.92862K wps
Begin Testing...
[Epoch 59] train avg loss 4.18955e-06, test acc 0.9044, test avg loss 0.387987, throughput 5.95991K wps
Test loss 0.172208, test acc 0.9290
Total time cost 339.02s
9000 1000
[Epoch 0 Batch 30/162] avg loss 0.0147687, throughput 5.68848K wps
[Epoch 0 Batch 60/162] avg loss 0.0144886, throughput 5.93436K wps
[Epoch 0 Batch 90/162] avg loss 0.0140481, throughput 5.93693K wps
[Epoch 0 Batch 120/162] avg loss 0.01321, throughput 5.9221K wps
[Epoch 0 Batch 150/162] avg loss 0.0130591, throughput 5.92675K wps
Begin Testing...
[Epoch 0] train avg loss 0.0138287, test acc 0.6900, test avg loss 0.588198, throughput 5.88419K wps
Observed Improvement.
Begin Testing...
[Epoch 1 Batch 30/162] avg loss 0.0122873, throughput 6.07328K wps
[Epoch 1 Batch 60/162] avg loss 0.0120782, throughput 5.92526K wps
[Epoch 1 Batch 90/162] avg loss 0.0114297, throughput 5.94421K wps
[Epoch 1 Batch 120/162] avg loss 0.0114863, throughput 5.93511K wps
[Epoch 1 Batch 150/162] avg loss 0.0110059, throughput 5.92873K wps
Begin Testing...
[Epoch 1] train avg loss 0.0116094, test acc 0.7967, test avg loss 0.514319, throughput 5.95804K wps
Observed Improvement.
Begin Testing...
[Epoch 2 Batch 30/162] avg loss 0.0102986, throughput 6.08066K wps
[Epoch 2 Batch 60/162] avg loss 0.00990402, throughput 5.93012K wps
[Epoch 2 Batch 90/162] avg loss 0.00981768, throughput 5.94435K wps
[Epoch 2 Batch 120/162] avg loss 0.00917646, throughput 5.93632K wps
[Epoch 2 Batch 150/162] avg loss 0.00890325, throughput 5.94453K wps
Begin Testing...
[Epoch 2] train avg loss 0.00954583, test acc 0.8689, test avg loss 0.435639, throughput 5.96555K wps
Observed Improvement.
Begin Testing...
[Epoch 3 Batch 30/162] avg loss 0.00800932, throughput 6.10043K wps
[Epoch 3 Batch 60/162] avg loss 0.00797318, throughput 5.94257K wps
[Epoch 3 Batch 90/162] avg loss 0.00752344, throughput 5.94332K wps
[Epoch 3 Batch 120/162] avg loss 0.00740282, throughput 5.94116K wps
[Epoch 3 Batch 150/162] avg loss 0.00733281, throughput 5.94468K wps
Begin Testing...
[Epoch 3] train avg loss 0.00760588, test acc 0.8767, test avg loss 0.370836, throughput 5.97183K wps
Observed Improvement.
Begin Testing...
[Epoch 4 Batch 30/162] avg loss 0.00655618, throughput 6.09054K wps
[Epoch 4 Batch 60/162] avg loss 0.00652205, throughput 5.94963K wps
[Epoch 4 Batch 90/162] avg loss 0.00606718, throughput 5.95256K wps
[Epoch 4 Batch 120/162] avg loss 0.00550101, throughput 5.93964K wps
[Epoch 4 Batch 150/162] avg loss 0.00576142, throughput 5.95926K wps
Begin Testing...
[Epoch 4] train avg loss 0.0060322, test acc 0.8956, test avg loss 0.3106, throughput 5.97585K wps
Observed Improvement.
Begin Testing...
[Epoch 5 Batch 30/162] avg loss 0.00514021, throughput 6.091K wps
[Epoch 5 Batch 60/162] avg loss 0.00487253, throughput 5.95006K wps
[Epoch 5 Batch 90/162] avg loss 0.00491185, throughput 5.93761K wps
[Epoch 5 Batch 120/162] avg loss 0.00458486, throughput 5.93999K wps
[Epoch 5 Batch 150/162] avg loss 0.00501984, throughput 5.95712K wps
Begin Testing...
[Epoch 5] train avg loss 0.00487827, test acc 0.9022, test avg loss 0.276755, throughput 5.97144K wps
Observed Improvement.
Begin Testing...
[Epoch 6 Batch 30/162] avg loss 0.00420435, throughput 6.09798K wps
[Epoch 6 Batch 60/162] avg loss 0.00408665, throughput 5.95882K wps
[Epoch 6 Batch 90/162] avg loss 0.00421699, throughput 5.94226K wps
[Epoch 6 Batch 120/162] avg loss 0.00401117, throughput 5.94165K wps
[Epoch 6 Batch 150/162] avg loss 0.00402332, throughput 5.9524K wps
Begin Testing...
[Epoch 6] train avg loss 0.00407888, test acc 0.9067, test avg loss 0.252911, throughput 5.97645K wps
Observed Improvement.
Begin Testing...
[Epoch 7 Batch 30/162] avg loss 0.0034426, throughput 6.08929K wps
[Epoch 7 Batch 60/162] avg loss 0.00360089, throughput 5.93561K wps
[Epoch 7 Batch 90/162] avg loss 0.00346011, throughput 5.93511K wps
[Epoch 7 Batch 120/162] avg loss 0.00344827, throughput 5.95127K wps
[Epoch 7 Batch 150/162] avg loss 0.00312527, throughput 5.94023K wps
Begin Testing...
[Epoch 7] train avg loss 0.00339845, test acc 0.9144, test avg loss 0.236596, throughput 5.96647K wps
Observed Improvement.
Begin Testing...
[Epoch 8 Batch 30/162] avg loss 0.00282611, throughput 6.07696K wps
[Epoch 8 Batch 60/162] avg loss 0.00283622, throughput 5.92759K wps
[Epoch 8 Batch 90/162] avg loss 0.00311086, throughput 5.94356K wps
[Epoch 8 Batch 120/162] avg loss 0.00258546, throughput 5.9506K wps
[Epoch 8 Batch 150/162] avg loss 0.00281245, throughput 5.95236K wps
Begin Testing...
[Epoch 8] train avg loss 0.0028436, test acc 0.9211, test avg loss 0.222726, throughput 5.96865K wps
Observed Improvement.
Begin Testing...
[Epoch 9 Batch 30/162] avg loss 0.00242795, throughput 6.09614K wps
[Epoch 9 Batch 60/162] avg loss 0.00238096, throughput 5.93798K wps
[Epoch 9 Batch 90/162] avg loss 0.00245697, throughput 5.94096K wps
[Epoch 9 Batch 120/162] avg loss 0.00235903, throughput 5.93384K wps
[Epoch 9 Batch 150/162] avg loss 0.00211498, throughput 5.94021K wps
Begin Testing...
[Epoch 9] train avg loss 0.00234385, test acc 0.9244, test avg loss 0.211156, throughput 5.96732K wps
Observed Improvement.
Begin Testing...
[Epoch 10 Batch 30/162] avg loss 0.00212031, throughput 6.09086K wps
[Epoch 10 Batch 60/162] avg loss 0.00189372, throughput 5.94197K wps
[Epoch 10 Batch 90/162] avg loss 0.0018783, throughput 5.93987K wps
[Epoch 10 Batch 120/162] avg loss 0.00215974, throughput 5.93465K wps
[Epoch 10 Batch 150/162] avg loss 0.00198964, throughput 5.93315K wps
Begin Testing...
[Epoch 10] train avg loss 0.00201582, test acc 0.9289, test avg loss 0.204609, throughput 5.9655K wps
Observed Improvement.
Begin Testing...
[Epoch 11 Batch 30/162] avg loss 0.00152964, throughput 6.09941K wps
[Epoch 11 Batch 60/162] avg loss 0.00170731, throughput 5.94413K wps
[Epoch 11 Batch 90/162] avg loss 0.00165012, throughput 5.93021K wps
[Epoch 11 Batch 120/162] avg loss 0.00159365, throughput 5.94793K wps
[Epoch 11 Batch 150/162] avg loss 0.00172677, throughput 5.94678K wps
Begin Testing...
[Epoch 11] train avg loss 0.00166711, test acc 0.9211, test avg loss 0.20158, throughput 5.97057K wps
[Epoch 12 Batch 30/162] avg loss 0.00141181, throughput 6.10502K wps
[Epoch 12 Batch 60/162] avg loss 0.00126036, throughput 5.9519K wps
[Epoch 12 Batch 90/162] avg loss 0.00137437, throughput 5.94339K wps
[Epoch 12 Batch 120/162] avg loss 0.00145585, throughput 5.94642K wps
[Epoch 12 Batch 150/162] avg loss 0.00143004, throughput 5.93793K wps
Begin Testing...
[Epoch 12] train avg loss 0.0013997, test acc 0.9289, test avg loss 0.195715, throughput 5.97307K wps
Observed Improvement.
Begin Testing...
[Epoch 13 Batch 30/162] avg loss 0.00129845, throughput 6.0996K wps
[Epoch 13 Batch 60/162] avg loss 0.00114564, throughput 5.94821K wps
[Epoch 13 Batch 90/162] avg loss 0.00113832, throughput 5.947K wps
[Epoch 13 Batch 120/162] avg loss 0.0010974, throughput 5.94271K wps
[Epoch 13 Batch 150/162] avg loss 0.00136035, throughput 5.93818K wps
Begin Testing...
[Epoch 13] train avg loss 0.00120341, test acc 0.9244, test avg loss 0.195854, throughput 5.97232K wps
[Epoch 14 Batch 30/162] avg loss 0.0010438, throughput 6.08099K wps
[Epoch 14 Batch 60/162] avg loss 0.000959138, throughput 5.94432K wps
[Epoch 14 Batch 90/162] avg loss 0.000928444, throughput 5.92763K wps
[Epoch 14 Batch 120/162] avg loss 0.000901453, throughput 5.93539K wps
[Epoch 14 Batch 150/162] avg loss 0.00112755, throughput 5.9409K wps
Begin Testing...
[Epoch 14] train avg loss 0.000992321, test acc 0.9256, test avg loss 0.190285, throughput 5.96458K wps
[Epoch 15 Batch 30/162] avg loss 0.000837323, throughput 6.09805K wps
[Epoch 15 Batch 60/162] avg loss 0.00093009, throughput 5.95066K wps
[Epoch 15 Batch 90/162] avg loss 0.000780232, throughput 5.9443K wps
[Epoch 15 Batch 120/162] avg loss 0.000857145, throughput 5.9397K wps
[Epoch 15 Batch 150/162] avg loss 0.000840524, throughput 5.94306K wps
Begin Testing...
[Epoch 15] train avg loss 0.000848365, test acc 0.9300, test avg loss 0.18588, throughput 5.97209K wps
Observed Improvement.
Begin Testing...
[Epoch 16 Batch 30/162] avg loss 0.000694814, throughput 6.08494K wps
[Epoch 16 Batch 60/162] avg loss 0.000634377, throughput 5.94507K wps
[Epoch 16 Batch 90/162] avg loss 0.000749888, throughput 5.92639K wps
[Epoch 16 Batch 120/162] avg loss 0.000769148, throughput 5.9323K wps
[Epoch 16 Batch 150/162] avg loss 0.000662887, throughput 5.94176K wps
Begin Testing...
[Epoch 16] train avg loss 0.000694368, test acc 0.9211, test avg loss 0.189556, throughput 5.96282K wps
[Epoch 17 Batch 30/162] avg loss 0.000560667, throughput 6.0691K wps
[Epoch 17 Batch 60/162] avg loss 0.000667798, throughput 5.92312K wps
[Epoch 17 Batch 90/162] avg loss 0.000547406, throughput 5.93172K wps
[Epoch 17 Batch 120/162] avg loss 0.00054156, throughput 5.92854K wps
[Epoch 17 Batch 150/162] avg loss 0.000622537, throughput 5.92811K wps
Begin Testing...
[Epoch 17] train avg loss 0.00058837, test acc 0.9289, test avg loss 0.189674, throughput 5.95237K wps
[Epoch 18 Batch 30/162] avg loss 0.000417923, throughput 6.08045K wps
[Epoch 18 Batch 60/162] avg loss 0.000503363, throughput 5.93725K wps
[Epoch 18 Batch 90/162] avg loss 0.000546557, throughput 5.93014K wps
[Epoch 18 Batch 120/162] avg loss 0.000432967, throughput 5.93146K wps
[Epoch 18 Batch 150/162] avg loss 0.000503114, throughput 5.93612K wps
Begin Testing...
[Epoch 18] train avg loss 0.000482085, test acc 0.9233, test avg loss 0.18798, throughput 5.9606K wps
[Epoch 19 Batch 30/162] avg loss 0.000383755, throughput 6.09461K wps
[Epoch 19 Batch 60/162] avg loss 0.000465612, throughput 5.94166K wps
[Epoch 19 Batch 90/162] avg loss 0.00042155, throughput 5.94025K wps
[Epoch 19 Batch 120/162] avg loss 0.00048824, throughput 5.94142K wps
[Epoch 19 Batch 150/162] avg loss 0.000416781, throughput 5.92839K wps
Begin Testing...
[Epoch 19] train avg loss 0.000434835, test acc 0.9211, test avg loss 0.187551, throughput 5.96687K wps
[Epoch 20 Batch 30/162] avg loss 0.000344372, throughput 6.0934K wps
[Epoch 20 Batch 60/162] avg loss 0.00043418, throughput 5.94269K wps
[Epoch 20 Batch 90/162] avg loss 0.000358256, throughput 5.93702K wps
[Epoch 20 Batch 120/162] avg loss 0.000361056, throughput 5.94086K wps
[Epoch 20 Batch 150/162] avg loss 0.00032483, throughput 5.9326K wps
Begin Testing...
[Epoch 20] train avg loss 0.00036017, test acc 0.9222, test avg loss 0.190501, throughput 5.96633K wps
[Epoch 21 Batch 30/162] avg loss 0.000331792, throughput 6.08744K wps
[Epoch 21 Batch 60/162] avg loss 0.000288104, throughput 5.95195K wps
[Epoch 21 Batch 90/162] avg loss 0.000302532, throughput 5.95118K wps
[Epoch 21 Batch 120/162] avg loss 0.000251499, throughput 5.93901K wps
[Epoch 21 Batch 150/162] avg loss 0.000272297, throughput 5.94695K wps
Begin Testing...
[Epoch 21] train avg loss 0.00029203, test acc 0.9222, test avg loss 0.192536, throughput 5.97157K wps
[Epoch 22 Batch 30/162] avg loss 0.000222955, throughput 6.08978K wps
[Epoch 22 Batch 60/162] avg loss 0.000227876, throughput 5.93016K wps
[Epoch 22 Batch 90/162] avg loss 0.000250658, throughput 5.94513K wps
[Epoch 22 Batch 120/162] avg loss 0.000312544, throughput 5.95679K wps
[Epoch 22 Batch 150/162] avg loss 0.000213512, throughput 5.94828K wps
Begin Testing...
[Epoch 22] train avg loss 0.000249276, test acc 0.9256, test avg loss 0.192366, throughput 5.97162K wps
[Epoch 23 Batch 30/162] avg loss 0.000206311, throughput 6.09798K wps
[Epoch 23 Batch 60/162] avg loss 0.000232659, throughput 5.94622K wps
[Epoch 23 Batch 90/162] avg loss 0.000261155, throughput 5.95115K wps
[Epoch 23 Batch 120/162] avg loss 0.000189619, throughput 5.95825K wps
[Epoch 23 Batch 150/162] avg loss 0.00024438, throughput 5.93882K wps
Begin Testing...
[Epoch 23] train avg loss 0.000227496, test acc 0.9211, test avg loss 0.200061, throughput 5.97535K wps
[Epoch 24 Batch 30/162] avg loss 0.000167819, throughput 6.06993K wps
[Epoch 24 Batch 60/162] avg loss 0.000162672, throughput 5.93282K wps
[Epoch 24 Batch 90/162] avg loss 0.000171761, throughput 5.95413K wps
[Epoch 24 Batch 120/162] avg loss 0.000220103, throughput 5.94738K wps
[Epoch 24 Batch 150/162] avg loss 0.000191082, throughput 5.94705K wps
Begin Testing...
[Epoch 24] train avg loss 0.000181847, test acc 0.9211, test avg loss 0.200188, throughput 5.96813K wps
[Epoch 25 Batch 30/162] avg loss 0.000177353, throughput 6.09229K wps
[Epoch 25 Batch 60/162] avg loss 0.000161777, throughput 5.93103K wps
[Epoch 25 Batch 90/162] avg loss 0.000163703, throughput 5.9442K wps
[Epoch 25 Batch 120/162] avg loss 0.000157565, throughput 5.93065K wps
[Epoch 25 Batch 150/162] avg loss 0.00016533, throughput 5.94186K wps
Begin Testing...
[Epoch 25] train avg loss 0.000167746, test acc 0.9211, test avg loss 0.199208, throughput 5.96691K wps
[Epoch 26 Batch 30/162] avg loss 0.000167773, throughput 6.0936K wps
[Epoch 26 Batch 60/162] avg loss 0.000157617, throughput 5.94762K wps
[Epoch 26 Batch 90/162] avg loss 0.000126849, throughput 5.94326K wps
[Epoch 26 Batch 120/162] avg loss 0.000144889, throughput 5.94571K wps
[Epoch 26 Batch 150/162] avg loss 0.00014841, throughput 5.9478K wps
Begin Testing...
[Epoch 26] train avg loss 0.0001484, test acc 0.9178, test avg loss 0.205351, throughput 5.97276K wps
[Epoch 27 Batch 30/162] avg loss 0.000129934, throughput 6.1013K wps
[Epoch 27 Batch 60/162] avg loss 0.000126755, throughput 5.95414K wps
[Epoch 27 Batch 90/162] avg loss 8.93367e-05, throughput 5.93854K wps
[Epoch 27 Batch 120/162] avg loss 0.000102948, throughput 5.95434K wps
[Epoch 27 Batch 150/162] avg loss 0.000112431, throughput 5.93991K wps
Begin Testing...
[Epoch 27] train avg loss 0.000113475, test acc 0.9178, test avg loss 0.207176, throughput 5.97342K wps
[Epoch 28 Batch 30/162] avg loss 0.000100978, throughput 6.10643K wps
[Epoch 28 Batch 60/162] avg loss 0.000103326, throughput 5.94507K wps
[Epoch 28 Batch 90/162] avg loss 0.000101222, throughput 5.93202K wps
[Epoch 28 Batch 120/162] avg loss 8.76479e-05, throughput 5.93686K wps
[Epoch 28 Batch 150/162] avg loss 9.34214e-05, throughput 5.94175K wps
Begin Testing...
[Epoch 28] train avg loss 9.94038e-05, test acc 0.9178, test avg loss 0.211847, throughput 5.97002K wps
[Epoch 29 Batch 30/162] avg loss 7.90473e-05, throughput 6.09736K wps
[Epoch 29 Batch 60/162] avg loss 9.00775e-05, throughput 5.94998K wps
[Epoch 29 Batch 90/162] avg loss 0.000109287, throughput 5.93988K wps
[Epoch 29 Batch 120/162] avg loss 8.26618e-05, throughput 5.94337K wps
[Epoch 29 Batch 150/162] avg loss 8.66091e-05, throughput 5.94468K wps
Begin Testing...
[Epoch 29] train avg loss 8.94862e-05, test acc 0.9189, test avg loss 0.211927, throughput 5.97277K wps
[Epoch 30 Batch 30/162] avg loss 5.95738e-05, throughput 6.07746K wps
[Epoch 30 Batch 60/162] avg loss 6.51067e-05, throughput 5.93981K wps
[Epoch 30 Batch 90/162] avg loss 9.041e-05, throughput 5.93483K wps
[Epoch 30 Batch 120/162] avg loss 7.63315e-05, throughput 5.93895K wps
[Epoch 30 Batch 150/162] avg loss 7.07746e-05, throughput 5.94829K wps
Begin Testing...
[Epoch 30] train avg loss 7.34437e-05, test acc 0.9189, test avg loss 0.214356, throughput 5.96576K wps
[Epoch 31 Batch 30/162] avg loss 6.63227e-05, throughput 6.08775K wps
[Epoch 31 Batch 60/162] avg loss 7.39736e-05, throughput 5.93485K wps
[Epoch 31 Batch 90/162] avg loss 7.86225e-05, throughput 5.92815K wps
[Epoch 31 Batch 120/162] avg loss 5.27462e-05, throughput 5.94882K wps
[Epoch 31 Batch 150/162] avg loss 6.64539e-05, throughput 5.93045K wps
Begin Testing...
[Epoch 31] train avg loss 6.78115e-05, test acc 0.9200, test avg loss 0.215011, throughput 5.96262K wps
[Epoch 32 Batch 30/162] avg loss 6.17356e-05, throughput 6.08728K wps
[Epoch 32 Batch 60/162] avg loss 5.34166e-05, throughput 5.94644K wps
[Epoch 32 Batch 90/162] avg loss 6.84751e-05, throughput 5.94901K wps
[Epoch 32 Batch 120/162] avg loss 6.04631e-05, throughput 5.94687K wps
[Epoch 32 Batch 150/162] avg loss 6.38262e-05, throughput 5.94852K wps
Begin Testing...
[Epoch 32] train avg loss 6.14965e-05, test acc 0.9222, test avg loss 0.215249, throughput 5.97247K wps
[Epoch 33 Batch 30/162] avg loss 5.0542e-05, throughput 6.09053K wps
[Epoch 33 Batch 60/162] avg loss 5.10216e-05, throughput 5.93966K wps
[Epoch 33 Batch 90/162] avg loss 4.4096e-05, throughput 5.92787K wps
[Epoch 33 Batch 120/162] avg loss 5.56689e-05, throughput 5.93174K wps
[Epoch 33 Batch 150/162] avg loss 5.7503e-05, throughput 5.94534K wps
Begin Testing...
[Epoch 33] train avg loss 5.30883e-05, test acc 0.9222, test avg loss 0.219795, throughput 5.96546K wps
[Epoch 34 Batch 30/162] avg loss 4.66857e-05, throughput 6.09679K wps
[Epoch 34 Batch 60/162] avg loss 4.8041e-05, throughput 5.94785K wps
[Epoch 34 Batch 90/162] avg loss 4.25803e-05, throughput 5.92681K wps
[Epoch 34 Batch 120/162] avg loss 4.63805e-05, throughput 5.93935K wps
[Epoch 34 Batch 150/162] avg loss 3.9322e-05, throughput 5.93625K wps
Begin Testing...
[Epoch 34] train avg loss 4.49317e-05, test acc 0.9200, test avg loss 0.223314, throughput 5.96624K wps
[Epoch 35 Batch 30/162] avg loss 3.92963e-05, throughput 6.08373K wps
[Epoch 35 Batch 60/162] avg loss 3.97182e-05, throughput 5.93716K wps
[Epoch 35 Batch 90/162] avg loss 3.74942e-05, throughput 5.92462K wps
[Epoch 35 Batch 120/162] avg loss 5.98008e-05, throughput 5.94879K wps
[Epoch 35 Batch 150/162] avg loss 4.26695e-05, throughput 5.92572K wps
Begin Testing...
[Epoch 35] train avg loss 4.3109e-05, test acc 0.9167, test avg loss 0.226728, throughput 5.96032K wps
[Epoch 36 Batch 30/162] avg loss 3.82807e-05, throughput 6.09202K wps
[Epoch 36 Batch 60/162] avg loss 3.50793e-05, throughput 5.95245K wps
[Epoch 36 Batch 90/162] avg loss 3.05124e-05, throughput 5.93305K wps
[Epoch 36 Batch 120/162] avg loss 3.14073e-05, throughput 5.9374K wps
[Epoch 36 Batch 150/162] avg loss 3.20039e-05, throughput 5.94446K wps
Begin Testing...
[Epoch 36] train avg loss 3.43551e-05, test acc 0.9167, test avg loss 0.229465, throughput 5.96917K wps
[Epoch 37 Batch 30/162] avg loss 3.2121e-05, throughput 6.09688K wps
[Epoch 37 Batch 60/162] avg loss 3.26087e-05, throughput 5.94587K wps
[Epoch 37 Batch 90/162] avg loss 3.01287e-05, throughput 5.94469K wps
[Epoch 37 Batch 120/162] avg loss 3.0705e-05, throughput 5.94497K wps
[Epoch 37 Batch 150/162] avg loss 3.12214e-05, throughput 5.9201K wps
Begin Testing...
[Epoch 37] train avg loss 3.10018e-05, test acc 0.9178, test avg loss 0.23221, throughput 5.96681K wps
[Epoch 38 Batch 30/162] avg loss 3.03119e-05, throughput 6.09167K wps
[Epoch 38 Batch 60/162] avg loss 2.35888e-05, throughput 5.9544K wps
[Epoch 38 Batch 90/162] avg loss 3.04639e-05, throughput 5.94796K wps
[Epoch 38 Batch 120/162] avg loss 3.7186e-05, throughput 5.94881K wps
[Epoch 38 Batch 150/162] avg loss 2.98846e-05, throughput 5.94247K wps
Begin Testing...
[Epoch 38] train avg loss 2.96765e-05, test acc 0.9178, test avg loss 0.235411, throughput 5.97346K wps
[Epoch 39 Batch 30/162] avg loss 2.55119e-05, throughput 6.08602K wps
[Epoch 39 Batch 60/162] avg loss 3.13424e-05, throughput 5.93931K wps
[Epoch 39 Batch 90/162] avg loss 2.7954e-05, throughput 5.92766K wps
[Epoch 39 Batch 120/162] avg loss 2.75877e-05, throughput 5.94186K wps
[Epoch 39 Batch 150/162] avg loss 2.90898e-05, throughput 5.94977K wps
Begin Testing...
[Epoch 39] train avg loss 2.79269e-05, test acc 0.9156, test avg loss 0.234824, throughput 5.96654K wps
[Epoch 40 Batch 30/162] avg loss 2.34082e-05, throughput 6.08546K wps
[Epoch 40 Batch 60/162] avg loss 2.00279e-05, throughput 5.94085K wps
[Epoch 40 Batch 90/162] avg loss 2.54307e-05, throughput 5.92275K wps
[Epoch 40 Batch 120/162] avg loss 2.18351e-05, throughput 5.93687K wps
[Epoch 40 Batch 150/162] avg loss 2.41593e-05, throughput 5.92593K wps
Begin Testing...
[Epoch 40] train avg loss 2.28617e-05, test acc 0.9156, test avg loss 0.237237, throughput 5.95933K wps
[Epoch 41 Batch 30/162] avg loss 2.04544e-05, throughput 6.08901K wps
[Epoch 41 Batch 60/162] avg loss 1.9377e-05, throughput 5.94436K wps
[Epoch 41 Batch 90/162] avg loss 1.58691e-05, throughput 5.94183K wps
[Epoch 41 Batch 120/162] avg loss 2.09297e-05, throughput 5.94013K wps
[Epoch 41 Batch 150/162] avg loss 2.42392e-05, throughput 5.94169K wps
Begin Testing...
[Epoch 41] train avg loss 2.0938e-05, test acc 0.9156, test avg loss 0.238592, throughput 5.96822K wps
[Epoch 42 Batch 30/162] avg loss 1.8498e-05, throughput 6.07174K wps
[Epoch 42 Batch 60/162] avg loss 1.62441e-05, throughput 5.93964K wps
[Epoch 42 Batch 90/162] avg loss 2.16927e-05, throughput 5.94102K wps
[Epoch 42 Batch 120/162] avg loss 2.24146e-05, throughput 5.93937K wps
[Epoch 42 Batch 150/162] avg loss 1.83926e-05, throughput 5.95148K wps
Begin Testing...
[Epoch 42] train avg loss 2.28875e-05, test acc 0.9133, test avg loss 0.244854, throughput 5.96649K wps
[Epoch 43 Batch 30/162] avg loss 1.92434e-05, throughput 6.08843K wps
[Epoch 43 Batch 60/162] avg loss 1.63616e-05, throughput 5.94281K wps
[Epoch 43 Batch 90/162] avg loss 1.67937e-05, throughput 5.94095K wps
[Epoch 43 Batch 120/162] avg loss 1.93267e-05, throughput 5.94993K wps
[Epoch 43 Batch 150/162] avg loss 1.73034e-05, throughput 5.94721K wps
Begin Testing...
[Epoch 43] train avg loss 1.7318e-05, test acc 0.9167, test avg loss 0.24568, throughput 5.97075K wps
[Epoch 44 Batch 30/162] avg loss 1.59992e-05, throughput 6.08979K wps
[Epoch 44 Batch 60/162] avg loss 1.31645e-05, throughput 5.93648K wps
[Epoch 44 Batch 90/162] avg loss 1.79735e-05, throughput 5.94369K wps
[Epoch 44 Batch 120/162] avg loss 1.52975e-05, throughput 5.9323K wps
[Epoch 44 Batch 150/162] avg loss 1.9461e-05, throughput 5.93172K wps
Begin Testing...
[Epoch 44] train avg loss 1.62596e-05, test acc 0.9144, test avg loss 0.249924, throughput 5.96334K wps
[Epoch 45 Batch 30/162] avg loss 1.36713e-05, throughput 6.10165K wps
[Epoch 45 Batch 60/162] avg loss 1.32745e-05, throughput 5.94144K wps
[Epoch 45 Batch 90/162] avg loss 1.10544e-05, throughput 5.94995K wps
[Epoch 45 Batch 120/162] avg loss 1.28818e-05, throughput 5.94936K wps
[Epoch 45 Batch 150/162] avg loss 1.74888e-05, throughput 5.94435K wps
Begin Testing...
[Epoch 45] train avg loss 1.36359e-05, test acc 0.9189, test avg loss 0.254948, throughput 5.9738K wps
[Epoch 46 Batch 30/162] avg loss 1.30979e-05, throughput 6.09379K wps
[Epoch 46 Batch 60/162] avg loss 1.28252e-05, throughput 5.95193K wps
[Epoch 46 Batch 90/162] avg loss 1.06739e-05, throughput 5.95248K wps
[Epoch 46 Batch 120/162] avg loss 1.10018e-05, throughput 5.94721K wps
[Epoch 46 Batch 150/162] avg loss 1.35418e-05, throughput 5.94529K wps
Begin Testing...
[Epoch 46] train avg loss 1.21703e-05, test acc 0.9133, test avg loss 0.253661, throughput 5.97322K wps
[Epoch 47 Batch 30/162] avg loss 1.08674e-05, throughput 6.09749K wps
[Epoch 47 Batch 60/162] avg loss 1.04296e-05, throughput 5.92898K wps
[Epoch 47 Batch 90/162] avg loss 1.32936e-05, throughput 5.94182K wps
[Epoch 47 Batch 120/162] avg loss 8.74062e-06, throughput 5.93032K wps
[Epoch 47 Batch 150/162] avg loss 1.00143e-05, throughput 5.94203K wps
Begin Testing...
[Epoch 47] train avg loss 1.07072e-05, test acc 0.9167, test avg loss 0.257163, throughput 5.96583K wps
[Epoch 48 Batch 30/162] avg loss 8.40829e-06, throughput 6.08167K wps
[Epoch 48 Batch 60/162] avg loss 8.37061e-06, throughput 5.94459K wps
[Epoch 48 Batch 90/162] avg loss 8.67216e-06, throughput 5.94045K wps
[Epoch 48 Batch 120/162] avg loss 1.00499e-05, throughput 5.9315K wps
[Epoch 48 Batch 150/162] avg loss 9.7358e-06, throughput 5.95802K wps
Begin Testing...
[Epoch 48] train avg loss 9.33889e-06, test acc 0.9167, test avg loss 0.263135, throughput 5.96851K wps
[Epoch 49 Batch 30/162] avg loss 9.0545e-06, throughput 6.08101K wps
[Epoch 49 Batch 60/162] avg loss 8.53756e-06, throughput 5.93222K wps
[Epoch 49 Batch 90/162] avg loss 1.00373e-05, throughput 5.92792K wps
[Epoch 49 Batch 120/162] avg loss 8.45909e-06, throughput 5.9407K wps
[Epoch 49 Batch 150/162] avg loss 1.07433e-05, throughput 5.93329K wps
Begin Testing...
[Epoch 49] train avg loss 9.07663e-06, test acc 0.9178, test avg loss 0.260926, throughput 5.96062K wps
[Epoch 50 Batch 30/162] avg loss 6.6981e-06, throughput 6.09649K wps
[Epoch 50 Batch 60/162] avg loss 7.26993e-06, throughput 5.93373K wps
[Epoch 50 Batch 90/162] avg loss 5.56917e-06, throughput 5.95K wps
[Epoch 50 Batch 120/162] avg loss 6.96049e-06, throughput 5.9409K wps
[Epoch 50 Batch 150/162] avg loss 8.21464e-06, throughput 5.94947K wps
Begin Testing...
[Epoch 50] train avg loss 6.93177e-06, test acc 0.9178, test avg loss 0.26441, throughput 5.97183K wps
[Epoch 51 Batch 30/162] avg loss 5.8846e-06, throughput 6.10452K wps
[Epoch 51 Batch 60/162] avg loss 2.03288e-05, throughput 5.94568K wps
[Epoch 51 Batch 90/162] avg loss 1.07978e-05, throughput 5.93361K wps
[Epoch 51 Batch 120/162] avg loss 1.00268e-05, throughput 5.96077K wps
[Epoch 51 Batch 150/162] avg loss 8.22448e-06, throughput 5.95016K wps
Begin Testing...
[Epoch 51] train avg loss 1.09626e-05, test acc 0.9144, test avg loss 0.270045, throughput 5.97643K wps
[Epoch 52 Batch 30/162] avg loss 7.27899e-06, throughput 6.0956K wps
[Epoch 52 Batch 60/162] avg loss 6.75724e-06, throughput 5.9626K wps
[Epoch 52 Batch 90/162] avg loss 7.26439e-06, throughput 5.95603K wps
[Epoch 52 Batch 120/162] avg loss 7.32217e-06, throughput 5.94265K wps
[Epoch 52 Batch 150/162] avg loss 7.86038e-06, throughput 5.94512K wps
Begin Testing...
[Epoch 52] train avg loss 7.24692e-06, test acc 0.9156, test avg loss 0.273597, throughput 5.97651K wps
[Epoch 53 Batch 30/162] avg loss 1.09822e-05, throughput 6.08416K wps
[Epoch 53 Batch 60/162] avg loss 5.72536e-06, throughput 5.94035K wps
[Epoch 53 Batch 90/162] avg loss 6.73269e-06, throughput 5.95191K wps
[Epoch 53 Batch 120/162] avg loss 6.31478e-06, throughput 5.9545K wps
[Epoch 53 Batch 150/162] avg loss 7.91641e-06, throughput 5.94923K wps
Begin Testing...
[Epoch 53] train avg loss 7.27023e-06, test acc 0.9144, test avg loss 0.274235, throughput 5.97449K wps
[Epoch 54 Batch 30/162] avg loss 7.65056e-06, throughput 6.08605K wps
[Epoch 54 Batch 60/162] avg loss 6.33201e-06, throughput 5.96071K wps
[Epoch 54 Batch 90/162] avg loss 3.95271e-06, throughput 5.93803K wps
[Epoch 54 Batch 120/162] avg loss 6.18845e-06, throughput 5.94065K wps
[Epoch 54 Batch 150/162] avg loss 5.35985e-06, throughput 5.94727K wps
Begin Testing...
[Epoch 54] train avg loss 5.8717e-06, test acc 0.9144, test avg loss 0.276535, throughput 5.97238K wps
[Epoch 55 Batch 30/162] avg loss 4.14813e-06, throughput 6.11015K wps
[Epoch 55 Batch 60/162] avg loss 4.41167e-06, throughput 5.95179K wps
[Epoch 55 Batch 90/162] avg loss 4.1662e-06, throughput 5.9428K wps
[Epoch 55 Batch 120/162] avg loss 4.91971e-06, throughput 5.94679K wps
[Epoch 55 Batch 150/162] avg loss 6.83482e-06, throughput 5.95723K wps
Begin Testing...
[Epoch 55] train avg loss 4.98632e-06, test acc 0.9144, test avg loss 0.280389, throughput 5.97972K wps
[Epoch 56 Batch 30/162] avg loss 3.36827e-06, throughput 6.06769K wps
[Epoch 56 Batch 60/162] avg loss 3.99452e-06, throughput 5.92486K wps
[Epoch 56 Batch 90/162] avg loss 4.91829e-06, throughput 5.95569K wps
[Epoch 56 Batch 120/162] avg loss 4.67076e-06, throughput 5.94835K wps
[Epoch 56 Batch 150/162] avg loss 8.00457e-06, throughput 5.95405K wps
Begin Testing...
[Epoch 56] train avg loss 4.92646e-06, test acc 0.9156, test avg loss 0.277166, throughput 5.9678K wps
[Epoch 57 Batch 30/162] avg loss 4.91432e-06, throughput 6.09791K wps
[Epoch 57 Batch 60/162] avg loss 3.42642e-06, throughput 5.95252K wps
[Epoch 57 Batch 90/162] avg loss 3.58783e-06, throughput 5.95423K wps
[Epoch 57 Batch 120/162] avg loss 5.60009e-06, throughput 5.95091K wps
[Epoch 57 Batch 150/162] avg loss 4.42792e-06, throughput 5.95534K wps
Begin Testing...
[Epoch 57] train avg loss 4.42158e-06, test acc 0.9167, test avg loss 0.28097, throughput 5.9797K wps
[Epoch 58 Batch 30/162] avg loss 3.58249e-06, throughput 6.10531K wps
[Epoch 58 Batch 60/162] avg loss 3.98991e-06, throughput 5.96175K wps
[Epoch 58 Batch 90/162] avg loss 2.80694e-06, throughput 5.94962K wps
[Epoch 58 Batch 120/162] avg loss 3.79104e-06, throughput 5.94062K wps
[Epoch 58 Batch 150/162] avg loss 2.98808e-06, throughput 5.94995K wps
Begin Testing...
[Epoch 58] train avg loss 3.32813e-06, test acc 0.9178, test avg loss 0.288121, throughput 5.97676K wps
[Epoch 59 Batch 30/162] avg loss 2.80784e-06, throughput 6.09765K wps
[Epoch 59 Batch 60/162] avg loss 3.79304e-06, throughput 5.94191K wps
[Epoch 59 Batch 90/162] avg loss 2.6795e-06, throughput 5.95793K wps
[Epoch 59 Batch 120/162] avg loss 4.00984e-06, throughput 5.95032K wps
[Epoch 59 Batch 150/162] avg loss 3.29603e-06, throughput 5.95628K wps
Begin Testing...
[Epoch 59] train avg loss 3.21787e-06, test acc 0.9167, test avg loss 0.293838, throughput 5.97813K wps
Test loss 0.203455, test acc 0.9190
Total time cost 339.35s
9000 1000
[Epoch 0 Batch 30/162] avg loss 0.015364, throughput 5.70042K wps
[Epoch 0 Batch 60/162] avg loss 0.0141227, throughput 5.94873K wps
[Epoch 0 Batch 90/162] avg loss 0.0136678, throughput 5.95407K wps
[Epoch 0 Batch 120/162] avg loss 0.0130597, throughput 5.95775K wps
[Epoch 0 Batch 150/162] avg loss 0.0127844, throughput 5.94932K wps
Begin Testing...
[Epoch 0] train avg loss 0.0137409, test acc 0.7411, test avg loss 0.575786, throughput 5.90404K wps
Observed Improvement.
Begin Testing...
[Epoch 1 Batch 30/162] avg loss 0.0119948, throughput 6.08337K wps
[Epoch 1 Batch 60/162] avg loss 0.0113798, throughput 5.96111K wps
[Epoch 1 Batch 90/162] avg loss 0.0114968, throughput 5.94872K wps
[Epoch 1 Batch 120/162] avg loss 0.0111801, throughput 5.94426K wps
[Epoch 1 Batch 150/162] avg loss 0.0106343, throughput 5.95555K wps
Begin Testing...
[Epoch 1] train avg loss 0.0113105, test acc 0.8033, test avg loss 0.51215, throughput 5.9765K wps
Observed Improvement.
Begin Testing...
[Epoch 2 Batch 30/162] avg loss 0.00993976, throughput 6.09791K wps
[Epoch 2 Batch 60/162] avg loss 0.00964038, throughput 5.93959K wps
[Epoch 2 Batch 90/162] avg loss 0.00923678, throughput 5.95253K wps
[Epoch 2 Batch 120/162] avg loss 0.00890388, throughput 5.94748K wps
[Epoch 2 Batch 150/162] avg loss 0.00880738, throughput 5.95785K wps
Begin Testing...
[Epoch 2] train avg loss 0.00927084, test acc 0.8611, test avg loss 0.425701, throughput 5.97684K wps
Observed Improvement.
Begin Testing...
[Epoch 3 Batch 30/162] avg loss 0.00789913, throughput 6.09906K wps
[Epoch 3 Batch 60/162] avg loss 0.00759253, throughput 5.94759K wps
[Epoch 3 Batch 90/162] avg loss 0.00698195, throughput 5.93563K wps
[Epoch 3 Batch 120/162] avg loss 0.00697272, throughput 5.94391K wps
[Epoch 3 Batch 150/162] avg loss 0.00688037, throughput 5.95226K wps
Begin Testing...
[Epoch 3] train avg loss 0.00720814, test acc 0.8944, test avg loss 0.348252, throughput 5.9721K wps
Observed Improvement.
Begin Testing...
[Epoch 4 Batch 30/162] avg loss 0.00627963, throughput 6.09463K wps
[Epoch 4 Batch 60/162] avg loss 0.00571153, throughput 5.95227K wps
[Epoch 4 Batch 90/162] avg loss 0.00573653, throughput 5.94869K wps
[Epoch 4 Batch 120/162] avg loss 0.005687, throughput 5.9464K wps
[Epoch 4 Batch 150/162] avg loss 0.00526449, throughput 5.94245K wps
Begin Testing...
[Epoch 4] train avg loss 0.00571755, test acc 0.9089, test avg loss 0.29571, throughput 5.97516K wps
Observed Improvement.
Begin Testing...
[Epoch 5 Batch 30/162] avg loss 0.00488096, throughput 6.09633K wps
[Epoch 5 Batch 60/162] avg loss 0.00451491, throughput 5.94774K wps
[Epoch 5 Batch 90/162] avg loss 0.00465276, throughput 5.93905K wps
[Epoch 5 Batch 120/162] avg loss 0.00449404, throughput 5.95206K wps
[Epoch 5 Batch 150/162] avg loss 0.0046169, throughput 5.9495K wps
Begin Testing...
[Epoch 5] train avg loss 0.0046158, test acc 0.9111, test avg loss 0.271942, throughput 5.97431K wps
Observed Improvement.
Begin Testing...
[Epoch 6 Batch 30/162] avg loss 0.00380153, throughput 6.10299K wps
[Epoch 6 Batch 60/162] avg loss 0.00394322, throughput 5.95986K wps
[Epoch 6 Batch 90/162] avg loss 0.00403953, throughput 5.95725K wps
[Epoch 6 Batch 120/162] avg loss 0.00365536, throughput 5.95483K wps
[Epoch 6 Batch 150/162] avg loss 0.00375677, throughput 5.95979K wps
Begin Testing...
[Epoch 6] train avg loss 0.00382917, test acc 0.9167, test avg loss 0.249601, throughput 5.98357K wps
Observed Improvement.
Begin Testing...
[Epoch 7 Batch 30/162] avg loss 0.00311634, throughput 6.09814K wps
[Epoch 7 Batch 60/162] avg loss 0.00335295, throughput 5.94489K wps
[Epoch 7 Batch 90/162] avg loss 0.00315963, throughput 5.94973K wps
[Epoch 7 Batch 120/162] avg loss 0.00315118, throughput 5.94415K wps
[Epoch 7 Batch 150/162] avg loss 0.00314465, throughput 5.95908K wps
Begin Testing...
[Epoch 7] train avg loss 0.00318764, test acc 0.9111, test avg loss 0.240087, throughput 5.9752K wps
[Epoch 8 Batch 30/162] avg loss 0.00271322, throughput 6.09127K wps
[Epoch 8 Batch 60/162] avg loss 0.00265217, throughput 5.93459K wps
[Epoch 8 Batch 90/162] avg loss 0.00292585, throughput 5.95522K wps
[Epoch 8 Batch 120/162] avg loss 0.00271223, throughput 5.95538K wps
[Epoch 8 Batch 150/162] avg loss 0.00252666, throughput 5.95666K wps
Begin Testing...
[Epoch 8] train avg loss 0.0026957, test acc 0.9144, test avg loss 0.230243, throughput 5.9755K wps
[Epoch 9 Batch 30/162] avg loss 0.00230187, throughput 6.08321K wps
[Epoch 9 Batch 60/162] avg loss 0.00251913, throughput 5.93531K wps
[Epoch 9 Batch 90/162] avg loss 0.00232905, throughput 5.95635K wps
[Epoch 9 Batch 120/162] avg loss 0.00217252, throughput 5.95989K wps
[Epoch 9 Batch 150/162] avg loss 0.00223133, throughput 5.94511K wps
Begin Testing...
[Epoch 9] train avg loss 0.00225719, test acc 0.9167, test avg loss 0.219858, throughput 5.97305K wps
Observed Improvement.
Begin Testing...
[Epoch 10 Batch 30/162] avg loss 0.00174534, throughput 6.08455K wps
[Epoch 10 Batch 60/162] avg loss 0.00206348, throughput 5.94478K wps
[Epoch 10 Batch 90/162] avg loss 0.00202327, throughput 5.95434K wps
[Epoch 10 Batch 120/162] avg loss 0.00198602, throughput 5.94061K wps
[Epoch 10 Batch 150/162] avg loss 0.00176125, throughput 5.94647K wps
Begin Testing...
[Epoch 10] train avg loss 0.0019069, test acc 0.9122, test avg loss 0.220727, throughput 5.97087K wps
[Epoch 11 Batch 30/162] avg loss 0.00155778, throughput 6.09418K wps
[Epoch 11 Batch 60/162] avg loss 0.00163548, throughput 5.94157K wps
[Epoch 11 Batch 90/162] avg loss 0.00139704, throughput 5.95422K wps
[Epoch 11 Batch 120/162] avg loss 0.00175769, throughput 5.95623K wps
[Epoch 11 Batch 150/162] avg loss 0.00159069, throughput 5.94414K wps
Begin Testing...
[Epoch 11] train avg loss 0.00160699, test acc 0.9156, test avg loss 0.215203, throughput 5.97456K wps
[Epoch 12 Batch 30/162] avg loss 0.00139324, throughput 6.08521K wps
[Epoch 12 Batch 60/162] avg loss 0.00133171, throughput 5.93211K wps
[Epoch 12 Batch 90/162] avg loss 0.00153666, throughput 5.9532K wps
[Epoch 12 Batch 120/162] avg loss 0.00128252, throughput 5.96127K wps
[Epoch 12 Batch 150/162] avg loss 0.00129411, throughput 5.94858K wps
Begin Testing...
[Epoch 12] train avg loss 0.00135852, test acc 0.9100, test avg loss 0.218219, throughput 5.97309K wps
[Epoch 13 Batch 30/162] avg loss 0.00111588, throughput 6.10601K wps
[Epoch 13 Batch 60/162] avg loss 0.0012567, throughput 5.95393K wps
[Epoch 13 Batch 90/162] avg loss 0.00123134, throughput 5.94346K wps
[Epoch 13 Batch 120/162] avg loss 0.000987889, throughput 5.94409K wps
[Epoch 13 Batch 150/162] avg loss 0.00103681, throughput 5.9406K wps
Begin Testing...
[Epoch 13] train avg loss 0.00111613, test acc 0.9067, test avg loss 0.227238, throughput 5.97495K wps
[Epoch 14 Batch 30/162] avg loss 0.000942637, throughput 6.10503K wps
[Epoch 14 Batch 60/162] avg loss 0.000947401, throughput 5.95585K wps
[Epoch 14 Batch 90/162] avg loss 0.000943927, throughput 5.94463K wps
[Epoch 14 Batch 120/162] avg loss 0.00104356, throughput 5.94739K wps
[Epoch 14 Batch 150/162] avg loss 0.000838159, throughput 5.94674K wps
Begin Testing...
[Epoch 14] train avg loss 0.000925384, test acc 0.9022, test avg loss 0.223191, throughput 5.97595K wps
[Epoch 15 Batch 30/162] avg loss 0.000797472, throughput 6.09521K wps
[Epoch 15 Batch 60/162] avg loss 0.000901168, throughput 5.93974K wps
[Epoch 15 Batch 90/162] avg loss 0.000802024, throughput 5.94105K wps
[Epoch 15 Batch 120/162] avg loss 0.000691595, throughput 5.94771K wps
[Epoch 15 Batch 150/162] avg loss 0.000813835, throughput 5.94089K wps
Begin Testing...
[Epoch 15] train avg loss 0.00079916, test acc 0.8978, test avg loss 0.226558, throughput 5.97018K wps
[Epoch 16 Batch 30/162] avg loss 0.000707631, throughput 6.10033K wps
[Epoch 16 Batch 60/162] avg loss 0.00070288, throughput 5.95673K wps
[Epoch 16 Batch 90/162] avg loss 0.000629153, throughput 5.94534K wps
[Epoch 16 Batch 120/162] avg loss 0.000597828, throughput 5.93939K wps
[Epoch 16 Batch 150/162] avg loss 0.000716725, throughput 5.94332K wps
Begin Testing...
[Epoch 16] train avg loss 0.00066551, test acc 0.9044, test avg loss 0.22764, throughput 5.97416K wps
[Epoch 17 Batch 30/162] avg loss 0.000585557, throughput 6.08188K wps
[Epoch 17 Batch 60/162] avg loss 0.000488464, throughput 5.93807K wps
[Epoch 17 Batch 90/162] avg loss 0.000565556, throughput 5.93499K wps
[Epoch 17 Batch 120/162] avg loss 0.000625915, throughput 5.95392K wps
[Epoch 17 Batch 150/162] avg loss 0.000527539, throughput 5.95046K wps
Begin Testing...
[Epoch 17] train avg loss 0.000555554, test acc 0.8989, test avg loss 0.236592, throughput 5.97055K wps
[Epoch 18 Batch 30/162] avg loss 0.000490953, throughput 6.10392K wps
[Epoch 18 Batch 60/162] avg loss 0.000477139, throughput 5.94197K wps
[Epoch 18 Batch 90/162] avg loss 0.000466607, throughput 5.93754K wps
[Epoch 18 Batch 120/162] avg loss 0.000420234, throughput 5.93887K wps
[Epoch 18 Batch 150/162] avg loss 0.000432273, throughput 5.94133K wps
Begin Testing...
[Epoch 18] train avg loss 0.00046854, test acc 0.9000, test avg loss 0.232118, throughput 5.97131K wps
[Epoch 19 Batch 30/162] avg loss 0.00045843, throughput 6.09685K wps
[Epoch 19 Batch 60/162] avg loss 0.000360488, throughput 5.95927K wps
[Epoch 19 Batch 90/162] avg loss 0.000412833, throughput 5.94512K wps
[Epoch 19 Batch 120/162] avg loss 0.000430232, throughput 5.94464K wps
[Epoch 19 Batch 150/162] avg loss 0.000362555, throughput 5.95247K wps
Begin Testing...
[Epoch 19] train avg loss 0.000407017, test acc 0.9033, test avg loss 0.235788, throughput 5.97642K wps
[Epoch 20 Batch 30/162] avg loss 0.000311969, throughput 6.10613K wps
[Epoch 20 Batch 60/162] avg loss 0.000345213, throughput 5.95081K wps
[Epoch 20 Batch 90/162] avg loss 0.000339924, throughput 5.95879K wps
[Epoch 20 Batch 120/162] avg loss 0.00032543, throughput 5.94843K wps
[Epoch 20 Batch 150/162] avg loss 0.000330138, throughput 5.94947K wps
Begin Testing...
[Epoch 20] train avg loss 0.000323841, test acc 0.9000, test avg loss 0.239455, throughput 5.97897K wps
[Epoch 21 Batch 30/162] avg loss 0.000254116, throughput 6.09443K wps
[Epoch 21 Batch 60/162] avg loss 0.000297417, throughput 5.95374K wps
[Epoch 21 Batch 90/162] avg loss 0.000283799, throughput 5.94348K wps
[Epoch 21 Batch 120/162] avg loss 0.000297791, throughput 5.95645K wps
[Epoch 21 Batch 150/162] avg loss 0.000348769, throughput 5.93567K wps
Begin Testing...
[Epoch 21] train avg loss 0.000298861, test acc 0.8967, test avg loss 0.247077, throughput 5.97391K wps
[Epoch 22 Batch 30/162] avg loss 0.000252568, throughput 6.08924K wps
[Epoch 22 Batch 60/162] avg loss 0.000229865, throughput 5.94521K wps
[Epoch 22 Batch 90/162] avg loss 0.000234826, throughput 5.95133K wps
[Epoch 22 Batch 120/162] avg loss 0.000275163, throughput 5.9456K wps
[Epoch 22 Batch 150/162] avg loss 0.000271125, throughput 5.94744K wps
Begin Testing...
[Epoch 22] train avg loss 0.000250308, test acc 0.8956, test avg loss 0.250937, throughput 5.97301K wps
[Epoch 23 Batch 30/162] avg loss 0.000176361, throughput 6.09938K wps
[Epoch 23 Batch 60/162] avg loss 0.000286787, throughput 5.94066K wps
[Epoch 23 Batch 90/162] avg loss 0.000206434, throughput 5.94357K wps
[Epoch 23 Batch 120/162] avg loss 0.000175751, throughput 5.95016K wps
[Epoch 23 Batch 150/162] avg loss 0.000202908, throughput 5.95206K wps
Begin Testing...
[Epoch 23] train avg loss 0.000210189, test acc 0.8944, test avg loss 0.256238, throughput 5.97552K wps
[Epoch 24 Batch 30/162] avg loss 0.000153796, throughput 6.10418K wps
[Epoch 24 Batch 60/162] avg loss 0.000238514, throughput 5.94844K wps
[Epoch 24 Batch 90/162] avg loss 0.00020036, throughput 5.94598K wps
[Epoch 24 Batch 120/162] avg loss 0.000186056, throughput 5.95302K wps
[Epoch 24 Batch 150/162] avg loss 0.000165898, throughput 5.9604K wps
Begin Testing...
[Epoch 24] train avg loss 0.000185384, test acc 0.9011, test avg loss 0.2547, throughput 5.97988K wps
[Epoch 25 Batch 30/162] avg loss 0.000155537, throughput 6.10411K wps
[Epoch 25 Batch 60/162] avg loss 0.000140153, throughput 5.94334K wps
[Epoch 25 Batch 90/162] avg loss 0.000139182, throughput 5.93843K wps
[Epoch 25 Batch 120/162] avg loss 0.000139991, throughput 5.95442K wps
[Epoch 25 Batch 150/162] avg loss 0.000153814, throughput 5.96159K wps
Begin Testing...
[Epoch 25] train avg loss 0.000142281, test acc 0.8933, test avg loss 0.264994, throughput 5.97776K wps
[Epoch 26 Batch 30/162] avg loss 0.000141623, throughput 6.08673K wps
[Epoch 26 Batch 60/162] avg loss 9.64985e-05, throughput 5.94078K wps
[Epoch 26 Batch 90/162] avg loss 0.000126205, throughput 5.95002K wps
[Epoch 26 Batch 120/162] avg loss 0.000125102, throughput 5.95405K wps
[Epoch 26 Batch 150/162] avg loss 0.000148535, throughput 5.954K wps
Begin Testing...
[Epoch 26] train avg loss 0.000130549, test acc 0.8989, test avg loss 0.265469, throughput 5.97543K wps
[Epoch 27 Batch 30/162] avg loss 0.00013405, throughput 6.10137K wps
[Epoch 27 Batch 60/162] avg loss 0.000111354, throughput 5.95968K wps
[Epoch 27 Batch 90/162] avg loss 0.000109176, throughput 5.958K wps
[Epoch 27 Batch 120/162] avg loss 0.000121357, throughput 5.94602K wps
[Epoch 27 Batch 150/162] avg loss 9.91434e-05, throughput 5.9365K wps
Begin Testing...
[Epoch 27] train avg loss 0.000115226, test acc 0.8978, test avg loss 0.273571, throughput 5.9765K wps
[Epoch 28 Batch 30/162] avg loss 8.80507e-05, throughput 6.06366K wps
[Epoch 28 Batch 60/162] avg loss 0.000110491, throughput 5.94078K wps
[Epoch 28 Batch 90/162] avg loss 0.000126241, throughput 5.94855K wps
[Epoch 28 Batch 120/162] avg loss 0.000110232, throughput 5.96237K wps
[Epoch 28 Batch 150/162] avg loss 9.23979e-05, throughput 5.94582K wps
Begin Testing...
[Epoch 28] train avg loss 0.000105884, test acc 0.9011, test avg loss 0.274515, throughput 5.96868K wps
[Epoch 29 Batch 30/162] avg loss 9.33844e-05, throughput 6.08446K wps
[Epoch 29 Batch 60/162] avg loss 8.8965e-05, throughput 5.94886K wps
[Epoch 29 Batch 90/162] avg loss 8.12455e-05, throughput 5.92883K wps
[Epoch 29 Batch 120/162] avg loss 0.000100001, throughput 5.94025K wps
[Epoch 29 Batch 150/162] avg loss 7.89826e-05, throughput 5.95381K wps
Begin Testing...
[Epoch 29] train avg loss 8.74542e-05, test acc 0.8978, test avg loss 0.275283, throughput 5.96855K wps
[Epoch 30 Batch 30/162] avg loss 7.17655e-05, throughput 6.10479K wps
[Epoch 30 Batch 60/162] avg loss 5.93791e-05, throughput 5.93993K wps
[Epoch 30 Batch 90/162] avg loss 7.0827e-05, throughput 5.95441K wps
[Epoch 30 Batch 120/162] avg loss 7.16696e-05, throughput 5.95221K wps
[Epoch 30 Batch 150/162] avg loss 8.08765e-05, throughput 5.94187K wps
Begin Testing...
[Epoch 30] train avg loss 7.22852e-05, test acc 0.9000, test avg loss 0.280826, throughput 5.97518K wps
[Epoch 31 Batch 30/162] avg loss 6.00427e-05, throughput 6.08638K wps
[Epoch 31 Batch 60/162] avg loss 6.29897e-05, throughput 5.93176K wps
[Epoch 31 Batch 90/162] avg loss 5.97136e-05, throughput 5.93803K wps
[Epoch 31 Batch 120/162] avg loss 7.4189e-05, throughput 5.94795K wps
[Epoch 31 Batch 150/162] avg loss 8.99497e-05, throughput 5.94925K wps
Begin Testing...
[Epoch 31] train avg loss 6.92226e-05, test acc 0.8989, test avg loss 0.287577, throughput 5.96815K wps
[Epoch 32 Batch 30/162] avg loss 5.49553e-05, throughput 6.08132K wps
[Epoch 32 Batch 60/162] avg loss 5.34877e-05, throughput 5.95672K wps
[Epoch 32 Batch 90/162] avg loss 5.70107e-05, throughput 5.94247K wps
[Epoch 32 Batch 120/162] avg loss 6.36547e-05, throughput 5.94502K wps
[Epoch 32 Batch 150/162] avg loss 6.08478e-05, throughput 5.95537K wps
Begin Testing...
[Epoch 32] train avg loss 5.70696e-05, test acc 0.8944, test avg loss 0.296597, throughput 5.97409K wps
[Epoch 33 Batch 30/162] avg loss 5.48733e-05, throughput 6.10499K wps
[Epoch 33 Batch 60/162] avg loss 7.07993e-05, throughput 5.94525K wps
[Epoch 33 Batch 90/162] avg loss 4.54839e-05, throughput 5.9398K wps
[Epoch 33 Batch 120/162] avg loss 4.34242e-05, throughput 5.94742K wps
[Epoch 33 Batch 150/162] avg loss 5.02402e-05, throughput 5.95348K wps
Begin Testing...
[Epoch 33] train avg loss 5.22549e-05, test acc 0.8967, test avg loss 0.297754, throughput 5.97557K wps
[Epoch 34 Batch 30/162] avg loss 5.4004e-05, throughput 6.09543K wps
[Epoch 34 Batch 60/162] avg loss 5.27051e-05, throughput 5.94265K wps
[Epoch 34 Batch 90/162] avg loss 4.10727e-05, throughput 5.95221K wps
[Epoch 34 Batch 120/162] avg loss 5.18556e-05, throughput 5.94708K wps
[Epoch 34 Batch 150/162] avg loss 4.97222e-05, throughput 5.94117K wps
Begin Testing...
[Epoch 34] train avg loss 4.88877e-05, test acc 0.8989, test avg loss 0.300389, throughput 5.97281K wps
[Epoch 35 Batch 30/162] avg loss 4.45056e-05, throughput 6.09994K wps
[Epoch 35 Batch 60/162] avg loss 3.49787e-05, throughput 5.95143K wps
[Epoch 35 Batch 90/162] avg loss 3.67914e-05, throughput 5.955K wps
[Epoch 35 Batch 120/162] avg loss 4.38234e-05, throughput 5.94859K wps
[Epoch 35 Batch 150/162] avg loss 3.92848e-05, throughput 5.93689K wps
Begin Testing...
[Epoch 35] train avg loss 3.99504e-05, test acc 0.8933, test avg loss 0.312371, throughput 5.97599K wps
[Epoch 36 Batch 30/162] avg loss 4.07025e-05, throughput 6.10032K wps
[Epoch 36 Batch 60/162] avg loss 3.22427e-05, throughput 5.93966K wps
[Epoch 36 Batch 90/162] avg loss 3.04078e-05, throughput 5.94055K wps
[Epoch 36 Batch 120/162] avg loss 3.47606e-05, throughput 5.93678K wps
[Epoch 36 Batch 150/162] avg loss 4.54123e-05, throughput 5.93802K wps
Begin Testing...
[Epoch 36] train avg loss 3.69751e-05, test acc 0.8944, test avg loss 0.309139, throughput 5.96789K wps
[Epoch 37 Batch 30/162] avg loss 3.51576e-05, throughput 6.09734K wps
[Epoch 37 Batch 60/162] avg loss 3.02721e-05, throughput 5.94767K wps
[Epoch 37 Batch 90/162] avg loss 3.42785e-05, throughput 5.93945K wps
[Epoch 37 Batch 120/162] avg loss 2.97658e-05, throughput 5.95538K wps
[Epoch 37 Batch 150/162] avg loss 3.01174e-05, throughput 5.95036K wps
Begin Testing...
[Epoch 37] train avg loss 3.1524e-05, test acc 0.8956, test avg loss 0.318415, throughput 5.97584K wps
[Epoch 38 Batch 30/162] avg loss 3.04566e-05, throughput 6.10737K wps
[Epoch 38 Batch 60/162] avg loss 3.26715e-05, throughput 5.95882K wps
[Epoch 38 Batch 90/162] avg loss 2.80162e-05, throughput 5.95748K wps
[Epoch 38 Batch 120/162] avg loss 2.65645e-05, throughput 5.95667K wps
[Epoch 38 Batch 150/162] avg loss 2.82637e-05, throughput 5.94452K wps
Begin Testing...
[Epoch 38] train avg loss 2.92637e-05, test acc 0.8933, test avg loss 0.324926, throughput 5.98057K wps
[Epoch 39 Batch 30/162] avg loss 3.27674e-05, throughput 6.09847K wps
[Epoch 39 Batch 60/162] avg loss 2.70631e-05, throughput 5.9458K wps
[Epoch 39 Batch 90/162] avg loss 4.77135e-05, throughput 5.94846K wps
[Epoch 39 Batch 120/162] avg loss 2.10195e-05, throughput 5.94114K wps
[Epoch 39 Batch 150/162] avg loss 2.70207e-05, throughput 5.93458K wps
Begin Testing...
[Epoch 39] train avg loss 3.0395e-05, test acc 0.8911, test avg loss 0.326059, throughput 5.96966K wps
[Epoch 40 Batch 30/162] avg loss 2.81105e-05, throughput 6.09298K wps
[Epoch 40 Batch 60/162] avg loss 2.20498e-05, throughput 5.9593K wps
[Epoch 40 Batch 90/162] avg loss 2.03262e-05, throughput 5.93349K wps
[Epoch 40 Batch 120/162] avg loss 2.13928e-05, throughput 5.94105K wps
[Epoch 40 Batch 150/162] avg loss 1.91694e-05, throughput 5.95068K wps
Begin Testing...
[Epoch 40] train avg loss 2.1972e-05, test acc 0.8956, test avg loss 0.334969, throughput 5.97438K wps
[Epoch 41 Batch 30/162] avg loss 2.06586e-05, throughput 6.10825K wps
[Epoch 41 Batch 60/162] avg loss 1.8993e-05, throughput 5.93815K wps
[Epoch 41 Batch 90/162] avg loss 1.98258e-05, throughput 5.95756K wps
[Epoch 41 Batch 120/162] avg loss 1.98584e-05, throughput 5.94793K wps
[Epoch 41 Batch 150/162] avg loss 1.8877e-05, throughput 5.93038K wps
Begin Testing...
[Epoch 41] train avg loss 2.00743e-05, test acc 0.8967, test avg loss 0.337453, throughput 5.97277K wps
[Epoch 42 Batch 30/162] avg loss 1.63989e-05, throughput 6.09605K wps
[Epoch 42 Batch 60/162] avg loss 2.84867e-05, throughput 5.95598K wps
[Epoch 42 Batch 90/162] avg loss 1.46282e-05, throughput 5.95813K wps
[Epoch 42 Batch 120/162] avg loss 2.25895e-05, throughput 5.95505K wps
[Epoch 42 Batch 150/162] avg loss 1.89422e-05, throughput 5.94681K wps
Begin Testing...
[Epoch 42] train avg loss 2.07278e-05, test acc 0.8956, test avg loss 0.341565, throughput 5.97971K wps
[Epoch 43 Batch 30/162] avg loss 1.45259e-05, throughput 6.08491K wps
[Epoch 43 Batch 60/162] avg loss 1.82268e-05, throughput 5.94258K wps
[Epoch 43 Batch 90/162] avg loss 2.929e-05, throughput 5.94553K wps
[Epoch 43 Batch 120/162] avg loss 1.76691e-05, throughput 5.95243K wps
[Epoch 43 Batch 150/162] avg loss 1.72717e-05, throughput 5.9529K wps
Begin Testing...
[Epoch 43] train avg loss 1.90448e-05, test acc 0.8944, test avg loss 0.346701, throughput 5.97267K wps
[Epoch 44 Batch 30/162] avg loss 1.6554e-05, throughput 6.10301K wps
[Epoch 44 Batch 60/162] avg loss 1.28873e-05, throughput 5.95152K wps
[Epoch 44 Batch 90/162] avg loss 1.47503e-05, throughput 5.95607K wps
[Epoch 44 Batch 120/162] avg loss 1.47863e-05, throughput 5.95514K wps
[Epoch 44 Batch 150/162] avg loss 1.58612e-05, throughput 5.93627K wps
Begin Testing...
[Epoch 44] train avg loss 1.45698e-05, test acc 0.8922, test avg loss 0.355457, throughput 5.97722K wps
[Epoch 45 Batch 30/162] avg loss 2.16147e-05, throughput 6.10127K wps
[Epoch 45 Batch 60/162] avg loss 1.34901e-05, throughput 5.93872K wps
[Epoch 45 Batch 90/162] avg loss 1.16908e-05, throughput 5.94263K wps
[Epoch 45 Batch 120/162] avg loss 1.38018e-05, throughput 5.93847K wps
[Epoch 45 Batch 150/162] avg loss 1.36727e-05, throughput 5.92957K wps
Begin Testing...
[Epoch 45] train avg loss 1.44867e-05, test acc 0.8956, test avg loss 0.357999, throughput 5.96749K wps
[Epoch 46 Batch 30/162] avg loss 1.14042e-05, throughput 6.09865K wps
[Epoch 46 Batch 60/162] avg loss 1.2804e-05, throughput 5.94517K wps
[Epoch 46 Batch 90/162] avg loss 8.94353e-06, throughput 5.94421K wps
[Epoch 46 Batch 120/162] avg loss 1.47742e-05, throughput 5.95535K wps
[Epoch 46 Batch 150/162] avg loss 7.74459e-06, throughput 5.95316K wps
Begin Testing...
[Epoch 46] train avg loss 1.12989e-05, test acc 0.8944, test avg loss 0.360846, throughput 5.97734K wps
[Epoch 47 Batch 30/162] avg loss 1.28797e-05, throughput 6.09334K wps
[Epoch 47 Batch 60/162] avg loss 1.21943e-05, throughput 5.94151K wps
[Epoch 47 Batch 90/162] avg loss 8.25142e-06, throughput 5.95776K wps
[Epoch 47 Batch 120/162] avg loss 1.0184e-05, throughput 5.96008K wps
[Epoch 47 Batch 150/162] avg loss 1.1769e-05, throughput 5.94434K wps
Begin Testing...
[Epoch 47] train avg loss 1.09423e-05, test acc 0.8944, test avg loss 0.364803, throughput 5.97667K wps
[Epoch 48 Batch 30/162] avg loss 7.60774e-06, throughput 6.10532K wps
[Epoch 48 Batch 60/162] avg loss 8.29239e-06, throughput 5.94609K wps
[Epoch 48 Batch 90/162] avg loss 1.0166e-05, throughput 5.95543K wps
[Epoch 48 Batch 120/162] avg loss 1.02558e-05, throughput 5.93973K wps
[Epoch 48 Batch 150/162] avg loss 1.06905e-05, throughput 5.93899K wps
Begin Testing...
[Epoch 48] train avg loss 9.40408e-06, test acc 0.8944, test avg loss 0.373367, throughput 5.97317K wps
[Epoch 49 Batch 30/162] avg loss 9.2238e-06, throughput 6.10321K wps
[Epoch 49 Batch 60/162] avg loss 8.30609e-06, throughput 5.93881K wps
[Epoch 49 Batch 90/162] avg loss 9.17462e-06, throughput 5.93936K wps
[Epoch 49 Batch 120/162] avg loss 8.18535e-06, throughput 5.93982K wps
[Epoch 49 Batch 150/162] avg loss 9.3484e-06, throughput 5.93337K wps
Begin Testing...
[Epoch 49] train avg loss 8.60557e-06, test acc 0.8944, test avg loss 0.378022, throughput 5.96852K wps
[Epoch 50 Batch 30/162] avg loss 7.03331e-06, throughput 6.09608K wps
[Epoch 50 Batch 60/162] avg loss 7.6209e-06, throughput 5.96038K wps
[Epoch 50 Batch 90/162] avg loss 9.42304e-06, throughput 5.95605K wps
[Epoch 50 Batch 120/162] avg loss 6.3262e-06, throughput 5.95378K wps
[Epoch 50 Batch 150/162] avg loss 1.04077e-05, throughput 5.95949K wps
Begin Testing...
[Epoch 50] train avg loss 8.06074e-06, test acc 0.8989, test avg loss 0.374748, throughput 5.98181K wps
[Epoch 51 Batch 30/162] avg loss 6.80242e-06, throughput 6.08923K wps
[Epoch 51 Batch 60/162] avg loss 1.00851e-05, throughput 5.93859K wps
[Epoch 51 Batch 90/162] avg loss 7.06396e-06, throughput 5.94634K wps
[Epoch 51 Batch 120/162] avg loss 6.06008e-06, throughput 5.95368K wps
[Epoch 51 Batch 150/162] avg loss 1.08063e-05, throughput 5.95649K wps
Begin Testing...
[Epoch 51] train avg loss 7.99476e-06, test acc 0.8878, test avg loss 0.39243, throughput 5.97453K wps
[Epoch 52 Batch 30/162] avg loss 6.44247e-06, throughput 6.10354K wps
[Epoch 52 Batch 60/162] avg loss 5.37491e-06, throughput 5.95816K wps
[Epoch 52 Batch 90/162] avg loss 5.7199e-06, throughput 5.95508K wps
[Epoch 52 Batch 120/162] avg loss 6.89857e-06, throughput 5.95066K wps
[Epoch 52 Batch 150/162] avg loss 6.16441e-06, throughput 5.94387K wps
Begin Testing...
[Epoch 52] train avg loss 6.24545e-06, test acc 0.8944, test avg loss 0.393696, throughput 5.97775K wps
[Epoch 53 Batch 30/162] avg loss 6.34496e-06, throughput 6.09658K wps
[Epoch 53 Batch 60/162] avg loss 5.43423e-06, throughput 5.95233K wps
[Epoch 53 Batch 90/162] avg loss 6.08934e-06, throughput 5.95079K wps
[Epoch 53 Batch 120/162] avg loss 6.61795e-06, throughput 5.94523K wps
[Epoch 53 Batch 150/162] avg loss 5.68465e-06, throughput 5.9459K wps
Begin Testing...
[Epoch 53] train avg loss 6.11581e-06, test acc 0.8911, test avg loss 0.403507, throughput 5.97638K wps
[Epoch 54 Batch 30/162] avg loss 5.80398e-06, throughput 6.10345K wps
[Epoch 54 Batch 60/162] avg loss 6.35809e-06, throughput 5.94468K wps
[Epoch 54 Batch 90/162] avg loss 3.47575e-06, throughput 5.94689K wps
[Epoch 54 Batch 120/162] avg loss 5.81367e-06, throughput 5.96804K wps
[Epoch 54 Batch 150/162] avg loss 4.26133e-06, throughput 5.96254K wps
Begin Testing...
[Epoch 54] train avg loss 5.11032e-06, test acc 0.8889, test avg loss 0.40722, throughput 5.98187K wps
[Epoch 55 Batch 30/162] avg loss 5.50654e-06, throughput 6.09949K wps
[Epoch 55 Batch 60/162] avg loss 4.32634e-06, throughput 5.95793K wps
[Epoch 55 Batch 90/162] avg loss 4.24711e-06, throughput 5.9547K wps
[Epoch 55 Batch 120/162] avg loss 5.18776e-06, throughput 5.95311K wps
[Epoch 55 Batch 150/162] avg loss 5.18582e-06, throughput 5.94086K wps
Begin Testing...
[Epoch 55] train avg loss 4.79224e-06, test acc 0.8911, test avg loss 0.405844, throughput 5.97804K wps
[Epoch 56 Batch 30/162] avg loss 4.88765e-06, throughput 6.09407K wps
[Epoch 56 Batch 60/162] avg loss 8.49987e-06, throughput 5.95567K wps
[Epoch 56 Batch 90/162] avg loss 5.35424e-06, throughput 5.93046K wps
[Epoch 56 Batch 120/162] avg loss 3.80757e-06, throughput 5.93624K wps
[Epoch 56 Batch 150/162] avg loss 3.8125e-06, throughput 5.95396K wps
Begin Testing...
[Epoch 56] train avg loss 5.2265e-06, test acc 0.8911, test avg loss 0.411983, throughput 5.97254K wps
[Epoch 57 Batch 30/162] avg loss 5.05964e-06, throughput 6.08988K wps
[Epoch 57 Batch 60/162] avg loss 3.88838e-06, throughput 5.94006K wps
[Epoch 57 Batch 90/162] avg loss 3.24812e-06, throughput 5.94442K wps
[Epoch 57 Batch 120/162] avg loss 3.71941e-06, throughput 5.95475K wps
[Epoch 57 Batch 150/162] avg loss 3.81373e-06, throughput 5.94897K wps
Begin Testing...
[Epoch 57] train avg loss 3.9064e-06, test acc 0.8878, test avg loss 0.414978, throughput 5.97196K wps
[Epoch 58 Batch 30/162] avg loss 6.32949e-06, throughput 6.07833K wps
[Epoch 58 Batch 60/162] avg loss 5.74716e-06, throughput 5.93984K wps
[Epoch 58 Batch 90/162] avg loss 4.61181e-06, throughput 5.94326K wps
[Epoch 58 Batch 120/162] avg loss 3.14292e-06, throughput 5.9464K wps
[Epoch 58 Batch 150/162] avg loss 4.75696e-06, throughput 5.95252K wps
Begin Testing...
[Epoch 58] train avg loss 4.85736e-06, test acc 0.8967, test avg loss 0.40654, throughput 5.9703K wps
[Epoch 59 Batch 30/162] avg loss 4.30368e-06, throughput 6.09214K wps
[Epoch 59 Batch 60/162] avg loss 3.56733e-06, throughput 5.93491K wps
[Epoch 59 Batch 90/162] avg loss 3.38218e-06, throughput 5.95373K wps
[Epoch 59 Batch 120/162] avg loss 3.54349e-06, throughput 5.95294K wps
[Epoch 59 Batch 150/162] avg loss 3.43839e-06, throughput 5.9551K wps
Begin Testing...
[Epoch 59] train avg loss 3.63534e-06, test acc 0.8867, test avg loss 0.420868, throughput 5.97182K wps
Test loss 0.187393, test acc 0.9330
Total time cost 338.15s
9000 1000
[Epoch 0 Batch 30/162] avg loss 0.0148564, throughput 5.58578K wps
[Epoch 0 Batch 60/162] avg loss 0.0144516, throughput 5.93337K wps
[Epoch 0 Batch 90/162] avg loss 0.0137556, throughput 5.94494K wps
[Epoch 0 Batch 120/162] avg loss 0.0130124, throughput 5.94395K wps
[Epoch 0 Batch 150/162] avg loss 0.013088, throughput 5.93157K wps
Begin Testing...
[Epoch 0] train avg loss 0.0137285, test acc 0.7144, test avg loss 0.580954, throughput 5.86937K wps
Observed Improvement.
Begin Testing...
[Epoch 1 Batch 30/162] avg loss 0.0118733, throughput 6.09069K wps
[Epoch 1 Batch 60/162] avg loss 0.0116125, throughput 5.95159K wps
[Epoch 1 Batch 90/162] avg loss 0.0116456, throughput 5.95372K wps
[Epoch 1 Batch 120/162] avg loss 0.0110691, throughput 5.9376K wps
[Epoch 1 Batch 150/162] avg loss 0.0104765, throughput 5.92967K wps
Begin Testing...
[Epoch 1] train avg loss 0.0113075, test acc 0.8111, test avg loss 0.503361, throughput 5.96991K wps
Observed Improvement.
Begin Testing...
[Epoch 2 Batch 30/162] avg loss 0.0100297, throughput 6.08724K wps
[Epoch 2 Batch 60/162] avg loss 0.00993511, throughput 5.93734K wps
[Epoch 2 Batch 90/162] avg loss 0.0095301, throughput 5.94178K wps
[Epoch 2 Batch 120/162] avg loss 0.00912946, throughput 5.93601K wps
[Epoch 2 Batch 150/162] avg loss 0.00872607, throughput 5.93587K wps
Begin Testing...
[Epoch 2] train avg loss 0.00939683, test acc 0.8578, test avg loss 0.422413, throughput 5.9633K wps
Observed Improvement.
Begin Testing...
[Epoch 3 Batch 30/162] avg loss 0.00797214, throughput 6.09146K wps
[Epoch 3 Batch 60/162] avg loss 0.00772673, throughput 5.94204K wps
[Epoch 3 Batch 90/162] avg loss 0.00749868, throughput 5.94149K wps
[Epoch 3 Batch 120/162] avg loss 0.00730468, throughput 5.94578K wps
[Epoch 3 Batch 150/162] avg loss 0.00705666, throughput 5.9493K wps
Begin Testing...
[Epoch 3] train avg loss 0.0074591, test acc 0.8822, test avg loss 0.348224, throughput 5.97181K wps
Observed Improvement.
Begin Testing...
[Epoch 4 Batch 30/162] avg loss 0.00625222, throughput 6.09565K wps
[Epoch 4 Batch 60/162] avg loss 0.00607516, throughput 5.93874K wps
[Epoch 4 Batch 90/162] avg loss 0.00576924, throughput 5.93323K wps
[Epoch 4 Batch 120/162] avg loss 0.0058439, throughput 5.93937K wps
[Epoch 4 Batch 150/162] avg loss 0.00579986, throughput 5.93874K wps
Begin Testing...
[Epoch 4] train avg loss 0.00594253, test acc 0.9056, test avg loss 0.295433, throughput 5.96728K wps
Observed Improvement.
Begin Testing...
[Epoch 5 Batch 30/162] avg loss 0.00543842, throughput 6.09225K wps
[Epoch 5 Batch 60/162] avg loss 0.00476476, throughput 5.94522K wps
[Epoch 5 Batch 90/162] avg loss 0.00462811, throughput 5.9506K wps
[Epoch 5 Batch 120/162] avg loss 0.00439624, throughput 5.94539K wps
[Epoch 5 Batch 150/162] avg loss 0.0047454, throughput 5.94462K wps
Begin Testing...
[Epoch 5] train avg loss 0.00478955, test acc 0.8989, test avg loss 0.271973, throughput 5.97167K wps
[Epoch 6 Batch 30/162] avg loss 0.00415351, throughput 6.09233K wps
[Epoch 6 Batch 60/162] avg loss 0.00408172, throughput 5.93168K wps
[Epoch 6 Batch 90/162] avg loss 0.0040524, throughput 5.95007K wps
[Epoch 6 Batch 120/162] avg loss 0.00393228, throughput 5.94668K wps
[Epoch 6 Batch 150/162] avg loss 0.00359451, throughput 5.93958K wps
Begin Testing...
[Epoch 6] train avg loss 0.00396448, test acc 0.9133, test avg loss 0.241799, throughput 5.96988K wps
Observed Improvement.
Begin Testing...
[Epoch 7 Batch 30/162] avg loss 0.00346406, throughput 6.09682K wps
[Epoch 7 Batch 60/162] avg loss 0.00334957, throughput 5.93555K wps
[Epoch 7 Batch 90/162] avg loss 0.00330568, throughput 5.9444K wps
[Epoch 7 Batch 120/162] avg loss 0.00312642, throughput 5.93871K wps
[Epoch 7 Batch 150/162] avg loss 0.00335285, throughput 5.95137K wps
Begin Testing...
[Epoch 7] train avg loss 0.00334417, test acc 0.9189, test avg loss 0.225328, throughput 5.96995K wps
Observed Improvement.
Begin Testing...
[Epoch 8 Batch 30/162] avg loss 0.00265244, throughput 6.08326K wps
[Epoch 8 Batch 60/162] avg loss 0.00298674, throughput 5.95138K wps
[Epoch 8 Batch 90/162] avg loss 0.00291746, throughput 5.9582K wps
[Epoch 8 Batch 120/162] avg loss 0.0027675, throughput 5.94508K wps
[Epoch 8 Batch 150/162] avg loss 0.00286948, throughput 5.94467K wps
Begin Testing...
[Epoch 8] train avg loss 0.00284467, test acc 0.9178, test avg loss 0.21379, throughput 5.97342K wps
[Epoch 9 Batch 30/162] avg loss 0.00249915, throughput 6.07251K wps
[Epoch 9 Batch 60/162] avg loss 0.00222431, throughput 5.94877K wps
[Epoch 9 Batch 90/162] avg loss 0.00240178, throughput 5.94962K wps
[Epoch 9 Batch 120/162] avg loss 0.00248715, throughput 5.94236K wps
[Epoch 9 Batch 150/162] avg loss 0.00220051, throughput 5.9448K wps
Begin Testing...
[Epoch 9] train avg loss 0.00236554, test acc 0.9222, test avg loss 0.209968, throughput 5.96979K wps
Observed Improvement.
Begin Testing...
[Epoch 10 Batch 30/162] avg loss 0.00182793, throughput 6.09983K wps
[Epoch 10 Batch 60/162] avg loss 0.00207996, throughput 5.95438K wps
[Epoch 10 Batch 90/162] avg loss 0.00200692, throughput 5.93729K wps
[Epoch 10 Batch 120/162] avg loss 0.00199903, throughput 5.92754K wps
[Epoch 10 Batch 150/162] avg loss 0.00214045, throughput 5.94807K wps
Begin Testing...
[Epoch 10] train avg loss 0.0020145, test acc 0.9244, test avg loss 0.20117, throughput 5.96955K wps
Observed Improvement.
Begin Testing...
[Epoch 11 Batch 30/162] avg loss 0.00158791, throughput 6.08775K wps
[Epoch 11 Batch 60/162] avg loss 0.001963, throughput 5.9416K wps
[Epoch 11 Batch 90/162] avg loss 0.00149651, throughput 5.93916K wps
[Epoch 11 Batch 120/162] avg loss 0.00181905, throughput 5.93288K wps
[Epoch 11 Batch 150/162] avg loss 0.00157116, throughput 5.93376K wps
Begin Testing...
[Epoch 11] train avg loss 0.00165476, test acc 0.9278, test avg loss 0.197573, throughput 5.96321K wps
Observed Improvement.
Begin Testing...
[Epoch 12 Batch 30/162] avg loss 0.00148, throughput 6.08201K wps
[Epoch 12 Batch 60/162] avg loss 0.0014973, throughput 5.92943K wps
[Epoch 12 Batch 90/162] avg loss 0.0013228, throughput 5.92593K wps
[Epoch 12 Batch 120/162] avg loss 0.00145619, throughput 5.94437K wps
[Epoch 12 Batch 150/162] avg loss 0.00129526, throughput 5.94813K wps
Begin Testing...
[Epoch 12] train avg loss 0.00140813, test acc 0.9289, test avg loss 0.193635, throughput 5.96283K wps
Observed Improvement.
Begin Testing...
[Epoch 13 Batch 30/162] avg loss 0.00115199, throughput 6.08631K wps
[Epoch 13 Batch 60/162] avg loss 0.00124425, throughput 5.93718K wps
[Epoch 13 Batch 90/162] avg loss 0.00121926, throughput 5.94193K wps
[Epoch 13 Batch 120/162] avg loss 0.00105635, throughput 5.95653K wps
[Epoch 13 Batch 150/162] avg loss 0.00118269, throughput 5.94252K wps
Begin Testing...
[Epoch 13] train avg loss 0.00116596, test acc 0.9300, test avg loss 0.192592, throughput 5.96859K wps
Observed Improvement.
Begin Testing...
[Epoch 14 Batch 30/162] avg loss 0.00102734, throughput 6.08436K wps
[Epoch 14 Batch 60/162] avg loss 0.00103388, throughput 5.94547K wps
[Epoch 14 Batch 90/162] avg loss 0.000989279, throughput 5.94974K wps
[Epoch 14 Batch 120/162] avg loss 0.000901945, throughput 5.95829K wps
[Epoch 14 Batch 150/162] avg loss 0.00095212, throughput 5.96239K wps
Begin Testing...
[Epoch 14] train avg loss 0.00097858, test acc 0.9300, test avg loss 0.193851, throughput 5.9783K wps
Observed Improvement.
Begin Testing...
[Epoch 15 Batch 30/162] avg loss 0.000880911, throughput 6.10147K wps
[Epoch 15 Batch 60/162] avg loss 0.00085948, throughput 5.9515K wps
[Epoch 15 Batch 90/162] avg loss 0.00072099, throughput 5.94756K wps
[Epoch 15 Batch 120/162] avg loss 0.00076892, throughput 5.94542K wps
[Epoch 15 Batch 150/162] avg loss 0.000814982, throughput 5.95106K wps
Begin Testing...
[Epoch 15] train avg loss 0.000810801, test acc 0.9289, test avg loss 0.19139, throughput 5.97566K wps
[Epoch 16 Batch 30/162] avg loss 0.000640797, throughput 6.08867K wps
[Epoch 16 Batch 60/162] avg loss 0.000648465, throughput 5.94977K wps
[Epoch 16 Batch 90/162] avg loss 0.000741615, throughput 5.94832K wps
[Epoch 16 Batch 120/162] avg loss 0.000652161, throughput 5.94178K wps
[Epoch 16 Batch 150/162] avg loss 0.000641872, throughput 5.94757K wps
Begin Testing...
[Epoch 16] train avg loss 0.000661751, test acc 0.9267, test avg loss 0.194958, throughput 5.9734K wps
[Epoch 17 Batch 30/162] avg loss 0.000555216, throughput 6.09429K wps
[Epoch 17 Batch 60/162] avg loss 0.000456908, throughput 5.9426K wps
[Epoch 17 Batch 90/162] avg loss 0.000582049, throughput 5.94366K wps
[Epoch 17 Batch 120/162] avg loss 0.000485186, throughput 5.95977K wps
[Epoch 17 Batch 150/162] avg loss 0.000633665, throughput 5.9548K wps
Begin Testing...
[Epoch 17] train avg loss 0.000546066, test acc 0.9311, test avg loss 0.194231, throughput 5.97645K wps
Observed Improvement.
Begin Testing...
[Epoch 18 Batch 30/162] avg loss 0.000464511, throughput 6.09646K wps
[Epoch 18 Batch 60/162] avg loss 0.000474063, throughput 5.94235K wps
[Epoch 18 Batch 90/162] avg loss 0.000517729, throughput 5.93913K wps
[Epoch 18 Batch 120/162] avg loss 0.000481431, throughput 5.94123K wps
[Epoch 18 Batch 150/162] avg loss 0.000503262, throughput 5.96218K wps
Begin Testing...
[Epoch 18] train avg loss 0.000486152, test acc 0.9311, test avg loss 0.196388, throughput 5.97407K wps
Observed Improvement.
Begin Testing...
[Epoch 19 Batch 30/162] avg loss 0.000430909, throughput 6.09472K wps
[Epoch 19 Batch 60/162] avg loss 0.000409093, throughput 5.94838K wps
[Epoch 19 Batch 90/162] avg loss 0.000418242, throughput 5.958K wps
[Epoch 19 Batch 120/162] avg loss 0.000382859, throughput 5.95613K wps
[Epoch 19 Batch 150/162] avg loss 0.000389322, throughput 5.94795K wps
Begin Testing...
[Epoch 19] train avg loss 0.000405899, test acc 0.9333, test avg loss 0.197532, throughput 5.97793K wps
Observed Improvement.
Begin Testing...
[Epoch 20 Batch 30/162] avg loss 0.000362396, throughput 6.08255K wps
[Epoch 20 Batch 60/162] avg loss 0.000343549, throughput 5.94842K wps
[Epoch 20 Batch 90/162] avg loss 0.000314247, throughput 5.93871K wps
[Epoch 20 Batch 120/162] avg loss 0.000312419, throughput 5.95393K wps
[Epoch 20 Batch 150/162] avg loss 0.000344484, throughput 5.94129K wps
Begin Testing...
[Epoch 20] train avg loss 0.000340388, test acc 0.9300, test avg loss 0.20122, throughput 5.97046K wps
[Epoch 21 Batch 30/162] avg loss 0.000261032, throughput 6.09155K wps
[Epoch 21 Batch 60/162] avg loss 0.000323644, throughput 5.93598K wps
[Epoch 21 Batch 90/162] avg loss 0.000289389, throughput 5.94315K wps
[Epoch 21 Batch 120/162] avg loss 0.000276854, throughput 5.94094K wps
[Epoch 21 Batch 150/162] avg loss 0.000288679, throughput 5.93613K wps
Begin Testing...
[Epoch 21] train avg loss 0.0002885, test acc 0.9311, test avg loss 0.203236, throughput 5.96628K wps
[Epoch 22 Batch 30/162] avg loss 0.000290246, throughput 6.0938K wps
[Epoch 22 Batch 60/162] avg loss 0.000258699, throughput 5.94076K wps
[Epoch 22 Batch 90/162] avg loss 0.000217699, throughput 5.93962K wps
[Epoch 22 Batch 120/162] avg loss 0.000272597, throughput 5.9416K wps
[Epoch 22 Batch 150/162] avg loss 0.000277902, throughput 5.93536K wps
Begin Testing...
[Epoch 22] train avg loss 0.00026188, test acc 0.9278, test avg loss 0.20428, throughput 5.96657K wps
[Epoch 23 Batch 30/162] avg loss 0.000209538, throughput 6.09123K wps
[Epoch 23 Batch 60/162] avg loss 0.000193991, throughput 5.9457K wps
[Epoch 23 Batch 90/162] avg loss 0.00018984, throughput 5.9424K wps
[Epoch 23 Batch 120/162] avg loss 0.000226331, throughput 5.95088K wps
[Epoch 23 Batch 150/162] avg loss 0.0002342, throughput 5.93985K wps
Begin Testing...
[Epoch 23] train avg loss 0.000209389, test acc 0.9333, test avg loss 0.205922, throughput 5.97031K wps
Observed Improvement.
Begin Testing...
[Epoch 24 Batch 30/162] avg loss 0.000160679, throughput 6.08691K wps
[Epoch 24 Batch 60/162] avg loss 0.000200429, throughput 5.93574K wps
[Epoch 24 Batch 90/162] avg loss 0.000194789, throughput 5.9432K wps
[Epoch 24 Batch 120/162] avg loss 0.0002146, throughput 5.94054K wps
[Epoch 24 Batch 150/162] avg loss 0.000164045, throughput 5.93526K wps
Begin Testing...
[Epoch 24] train avg loss 0.000186606, test acc 0.9300, test avg loss 0.210317, throughput 5.96516K wps
[Epoch 25 Batch 30/162] avg loss 0.000179275, throughput 6.1001K wps
[Epoch 25 Batch 60/162] avg loss 0.000157482, throughput 5.94633K wps
[Epoch 25 Batch 90/162] avg loss 0.000163168, throughput 5.93424K wps
[Epoch 25 Batch 120/162] avg loss 0.000135079, throughput 5.94269K wps
[Epoch 25 Batch 150/162] avg loss 0.000122707, throughput 5.94123K wps
Begin Testing...
[Epoch 25] train avg loss 0.00015266, test acc 0.9289, test avg loss 0.216181, throughput 5.97043K wps
[Epoch 26 Batch 30/162] avg loss 0.000133571, throughput 6.10688K wps
[Epoch 26 Batch 60/162] avg loss 0.000126751, throughput 5.93984K wps
[Epoch 26 Batch 90/162] avg loss 0.000123006, throughput 5.94297K wps
[Epoch 26 Batch 120/162] avg loss 0.000161425, throughput 5.94663K wps
[Epoch 26 Batch 150/162] avg loss 0.000134854, throughput 5.93543K wps
Begin Testing...
[Epoch 26] train avg loss 0.000133772, test acc 0.9267, test avg loss 0.220642, throughput 5.97031K wps
[Epoch 27 Batch 30/162] avg loss 0.000102091, throughput 6.08597K wps
[Epoch 27 Batch 60/162] avg loss 0.000102969, throughput 5.94408K wps
[Epoch 27 Batch 90/162] avg loss 0.000138541, throughput 5.93051K wps
[Epoch 27 Batch 120/162] avg loss 0.000106635, throughput 5.92918K wps
[Epoch 27 Batch 150/162] avg loss 0.0001144, throughput 5.93041K wps
Begin Testing...
[Epoch 27] train avg loss 0.00011281, test acc 0.9300, test avg loss 0.221514, throughput 5.96114K wps
[Epoch 28 Batch 30/162] avg loss 0.000116608, throughput 6.08276K wps
[Epoch 28 Batch 60/162] avg loss 9.97452e-05, throughput 5.93974K wps
[Epoch 28 Batch 90/162] avg loss 8.9756e-05, throughput 5.94701K wps
[Epoch 28 Batch 120/162] avg loss 8.96988e-05, throughput 5.93624K wps
[Epoch 28 Batch 150/162] avg loss 0.000108171, throughput 5.94183K wps
Begin Testing...
[Epoch 28] train avg loss 9.93135e-05, test acc 0.9289, test avg loss 0.22274, throughput 5.96605K wps
[Epoch 29 Batch 30/162] avg loss 9.07383e-05, throughput 6.08302K wps
[Epoch 29 Batch 60/162] avg loss 8.75971e-05, throughput 5.94166K wps
[Epoch 29 Batch 90/162] avg loss 7.43868e-05, throughput 5.94297K wps
[Epoch 29 Batch 120/162] avg loss 8.90187e-05, throughput 5.93949K wps
[Epoch 29 Batch 150/162] avg loss 8.04944e-05, throughput 5.95636K wps
Begin Testing...
[Epoch 29] train avg loss 8.33784e-05, test acc 0.9244, test avg loss 0.225994, throughput 5.97098K wps
[Epoch 30 Batch 30/162] avg loss 8.06925e-05, throughput 6.09299K wps
[Epoch 30 Batch 60/162] avg loss 6.95356e-05, throughput 5.94735K wps
[Epoch 30 Batch 90/162] avg loss 6.77089e-05, throughput 5.94754K wps
[Epoch 30 Batch 120/162] avg loss 7.30032e-05, throughput 5.93985K wps
[Epoch 30 Batch 150/162] avg loss 7.26334e-05, throughput 5.94517K wps
Begin Testing...
[Epoch 30] train avg loss 7.83612e-05, test acc 0.9244, test avg loss 0.231001, throughput 5.97136K wps
[Epoch 31 Batch 30/162] avg loss 6.68546e-05, throughput 6.10056K wps
[Epoch 31 Batch 60/162] avg loss 6.92857e-05, throughput 5.95647K wps
[Epoch 31 Batch 90/162] avg loss 6.31049e-05, throughput 5.92861K wps
[Epoch 31 Batch 120/162] avg loss 6.77303e-05, throughput 5.92422K wps
[Epoch 31 Batch 150/162] avg loss 6.57842e-05, throughput 5.94912K wps
Begin Testing...
[Epoch 31] train avg loss 6.69994e-05, test acc 0.9233, test avg loss 0.23194, throughput 5.96742K wps
[Epoch 32 Batch 30/162] avg loss 6.08537e-05, throughput 6.09365K wps
[Epoch 32 Batch 60/162] avg loss 6.13079e-05, throughput 5.95593K wps
[Epoch 32 Batch 90/162] avg loss 5.879e-05, throughput 5.95504K wps
[Epoch 32 Batch 120/162] avg loss 5.67554e-05, throughput 5.95856K wps
[Epoch 32 Batch 150/162] avg loss 6.00079e-05, throughput 5.96084K wps
Begin Testing...
[Epoch 32] train avg loss 5.90607e-05, test acc 0.9233, test avg loss 0.2394, throughput 5.98301K wps
[Epoch 33 Batch 30/162] avg loss 4.76975e-05, throughput 6.08838K wps
[Epoch 33 Batch 60/162] avg loss 5.50438e-05, throughput 5.93528K wps
[Epoch 33 Batch 90/162] avg loss 5.06056e-05, throughput 5.94307K wps
[Epoch 33 Batch 120/162] avg loss 5.91386e-05, throughput 5.93762K wps
[Epoch 33 Batch 150/162] avg loss 5.06525e-05, throughput 5.89789K wps
Begin Testing...
[Epoch 33] train avg loss 5.25662e-05, test acc 0.9222, test avg loss 0.24221, throughput 5.95777K wps
[Epoch 34 Batch 30/162] avg loss 5.00169e-05, throughput 6.086K wps
[Epoch 34 Batch 60/162] avg loss 3.4838e-05, throughput 5.93867K wps
[Epoch 34 Batch 90/162] avg loss 4.41201e-05, throughput 5.9435K wps
[Epoch 34 Batch 120/162] avg loss 4.7115e-05, throughput 5.94197K wps
[Epoch 34 Batch 150/162] avg loss 4.15604e-05, throughput 5.93626K wps
Begin Testing...
[Epoch 34] train avg loss 4.3272e-05, test acc 0.9222, test avg loss 0.246089, throughput 5.96643K wps
[Epoch 35 Batch 30/162] avg loss 3.85249e-05, throughput 6.10041K wps
[Epoch 35 Batch 60/162] avg loss 4.16121e-05, throughput 5.94358K wps
[Epoch 35 Batch 90/162] avg loss 4.04669e-05, throughput 5.94191K wps
[Epoch 35 Batch 120/162] avg loss 3.16493e-05, throughput 5.93692K wps
[Epoch 35 Batch 150/162] avg loss 3.96116e-05, throughput 5.9509K wps
Begin Testing...
[Epoch 35] train avg loss 3.90466e-05, test acc 0.9222, test avg loss 0.248781, throughput 5.97094K wps
[Epoch 36 Batch 30/162] avg loss 3.27131e-05, throughput 6.09567K wps
[Epoch 36 Batch 60/162] avg loss 3.66603e-05, throughput 5.93778K wps
[Epoch 36 Batch 90/162] avg loss 3.93583e-05, throughput 5.95091K wps
[Epoch 36 Batch 120/162] avg loss 2.79246e-05, throughput 5.94869K wps
[Epoch 36 Batch 150/162] avg loss 3.56323e-05, throughput 5.94787K wps
Begin Testing...
[Epoch 36] train avg loss 3.43548e-05, test acc 0.9200, test avg loss 0.253539, throughput 5.97329K wps
[Epoch 37 Batch 30/162] avg loss 2.7152e-05, throughput 6.0989K wps
[Epoch 37 Batch 60/162] avg loss 3.00788e-05, throughput 5.95518K wps
[Epoch 37 Batch 90/162] avg loss 3.61342e-05, throughput 5.95838K wps
[Epoch 37 Batch 120/162] avg loss 3.25302e-05, throughput 5.96038K wps
[Epoch 37 Batch 150/162] avg loss 3.29249e-05, throughput 5.94696K wps
Begin Testing...
[Epoch 37] train avg loss 3.15556e-05, test acc 0.9211, test avg loss 0.25799, throughput 5.98023K wps
[Epoch 38 Batch 30/162] avg loss 2.68754e-05, throughput 6.09821K wps
[Epoch 38 Batch 60/162] avg loss 2.93919e-05, throughput 5.93852K wps
[Epoch 38 Batch 90/162] avg loss 2.87374e-05, throughput 5.94302K wps
[Epoch 38 Batch 120/162] avg loss 3.12027e-05, throughput 5.94741K wps
[Epoch 38 Batch 150/162] avg loss 2.73029e-05, throughput 5.94269K wps
Begin Testing...
[Epoch 38] train avg loss 2.85108e-05, test acc 0.9178, test avg loss 0.258786, throughput 5.97032K wps
[Epoch 39 Batch 30/162] avg loss 2.37876e-05, throughput 6.09876K wps
[Epoch 39 Batch 60/162] avg loss 2.70792e-05, throughput 5.94985K wps
[Epoch 39 Batch 90/162] avg loss 2.31576e-05, throughput 5.95278K wps
[Epoch 39 Batch 120/162] avg loss 2.01324e-05, throughput 5.94513K wps
[Epoch 39 Batch 150/162] avg loss 2.53738e-05, throughput 5.9519K wps
Begin Testing...
[Epoch 39] train avg loss 2.35445e-05, test acc 0.9200, test avg loss 0.262126, throughput 5.97534K wps
[Epoch 40 Batch 30/162] avg loss 2.43725e-05, throughput 6.11079K wps
[Epoch 40 Batch 60/162] avg loss 1.97417e-05, throughput 5.94577K wps
[Epoch 40 Batch 90/162] avg loss 2.39819e-05, throughput 5.94768K wps
[Epoch 40 Batch 120/162] avg loss 2.10085e-05, throughput 5.95664K wps
[Epoch 40 Batch 150/162] avg loss 2.00011e-05, throughput 5.94456K wps
Begin Testing...
[Epoch 40] train avg loss 2.19926e-05, test acc 0.9211, test avg loss 0.264563, throughput 5.97805K wps
[Epoch 41 Batch 30/162] avg loss 1.85141e-05, throughput 6.10234K wps
[Epoch 41 Batch 60/162] avg loss 2.44035e-05, throughput 5.95495K wps
[Epoch 41 Batch 90/162] avg loss 2.2207e-05, throughput 5.93662K wps
[Epoch 41 Batch 120/162] avg loss 1.74796e-05, throughput 5.95438K wps
[Epoch 41 Batch 150/162] avg loss 1.76358e-05, throughput 5.94575K wps
Begin Testing...
[Epoch 41] train avg loss 1.99409e-05, test acc 0.9200, test avg loss 0.267578, throughput 5.97603K wps
[Epoch 42 Batch 30/162] avg loss 1.44446e-05, throughput 6.09793K wps
[Epoch 42 Batch 60/162] avg loss 1.48621e-05, throughput 5.94475K wps
[Epoch 42 Batch 90/162] avg loss 1.8576e-05, throughput 5.94475K wps
[Epoch 42 Batch 120/162] avg loss 1.90386e-05, throughput 5.95985K wps
[Epoch 42 Batch 150/162] avg loss 2.87813e-05, throughput 5.94488K wps
Begin Testing...
[Epoch 42] train avg loss 1.89888e-05, test acc 0.9222, test avg loss 0.272842, throughput 5.97672K wps
[Epoch 43 Batch 30/162] avg loss 1.70782e-05, throughput 6.08288K wps
[Epoch 43 Batch 60/162] avg loss 1.34473e-05, throughput 5.95469K wps
[Epoch 43 Batch 90/162] avg loss 1.94236e-05, throughput 5.95554K wps
[Epoch 43 Batch 120/162] avg loss 1.58159e-05, throughput 5.94795K wps
[Epoch 43 Batch 150/162] avg loss 1.6094e-05, throughput 5.95127K wps
Begin Testing...
[Epoch 43] train avg loss 1.64218e-05, test acc 0.9200, test avg loss 0.272876, throughput 5.97607K wps
[Epoch 44 Batch 30/162] avg loss 1.41862e-05, throughput 6.09432K wps
[Epoch 44 Batch 60/162] avg loss 1.4352e-05, throughput 5.93014K wps
[Epoch 44 Batch 90/162] avg loss 1.45356e-05, throughput 5.92149K wps
[Epoch 44 Batch 120/162] avg loss 1.19468e-05, throughput 5.93384K wps
[Epoch 44 Batch 150/162] avg loss 1.48166e-05, throughput 5.93315K wps
Begin Testing...
[Epoch 44] train avg loss 1.3779e-05, test acc 0.9222, test avg loss 0.278104, throughput 5.96021K wps
[Epoch 45 Batch 30/162] avg loss 1.74744e-05, throughput 6.08046K wps
[Epoch 45 Batch 60/162] avg loss 1.29679e-05, throughput 5.94174K wps
[Epoch 45 Batch 90/162] avg loss 1.6252e-05, throughput 5.92678K wps
[Epoch 45 Batch 120/162] avg loss 1.43706e-05, throughput 5.93613K wps
[Epoch 45 Batch 150/162] avg loss 1.30732e-05, throughput 5.94436K wps
Begin Testing...
[Epoch 45] train avg loss 1.48413e-05, test acc 0.9167, test avg loss 0.279475, throughput 5.96213K wps
[Epoch 46 Batch 30/162] avg loss 1.06753e-05, throughput 6.08968K wps
[Epoch 46 Batch 60/162] avg loss 1.52525e-05, throughput 5.94113K wps
[Epoch 46 Batch 90/162] avg loss 1.32434e-05, throughput 5.93278K wps
[Epoch 46 Batch 120/162] avg loss 1.36233e-05, throughput 5.94666K wps
[Epoch 46 Batch 150/162] avg loss 1.358e-05, throughput 5.93778K wps
Begin Testing...
[Epoch 46] train avg loss 1.31045e-05, test acc 0.9189, test avg loss 0.285122, throughput 5.96695K wps
[Epoch 47 Batch 30/162] avg loss 9.02102e-06, throughput 6.09084K wps
[Epoch 47 Batch 60/162] avg loss 8.76854e-06, throughput 5.94479K wps
[Epoch 47 Batch 90/162] avg loss 1.04521e-05, throughput 5.94621K wps
[Epoch 47 Batch 120/162] avg loss 9.4209e-06, throughput 5.94618K wps
[Epoch 47 Batch 150/162] avg loss 9.67976e-06, throughput 5.93517K wps
Begin Testing...
[Epoch 47] train avg loss 9.98653e-06, test acc 0.9200, test avg loss 0.287035, throughput 5.96971K wps
[Epoch 48 Batch 30/162] avg loss 7.91041e-06, throughput 6.08581K wps
[Epoch 48 Batch 60/162] avg loss 1.06106e-05, throughput 5.93741K wps
[Epoch 48 Batch 90/162] avg loss 1.07869e-05, throughput 5.94042K wps
[Epoch 48 Batch 120/162] avg loss 1.05735e-05, throughput 5.92698K wps
[Epoch 48 Batch 150/162] avg loss 9.84516e-06, throughput 5.9463K wps
Begin Testing...
[Epoch 48] train avg loss 9.78339e-06, test acc 0.9167, test avg loss 0.289355, throughput 5.96443K wps
[Epoch 49 Batch 30/162] avg loss 8.6625e-06, throughput 6.08982K wps
[Epoch 49 Batch 60/162] avg loss 7.61184e-06, throughput 5.94546K wps
[Epoch 49 Batch 90/162] avg loss 9.61472e-06, throughput 5.94389K wps
[Epoch 49 Batch 120/162] avg loss 9.85527e-06, throughput 5.9302K wps
[Epoch 49 Batch 150/162] avg loss 8.42068e-06, throughput 5.93542K wps
Begin Testing...
[Epoch 49] train avg loss 8.64149e-06, test acc 0.9178, test avg loss 0.294443, throughput 5.96592K wps
[Epoch 50 Batch 30/162] avg loss 9.07376e-06, throughput 6.0751K wps
[Epoch 50 Batch 60/162] avg loss 7.61868e-06, throughput 5.93834K wps
[Epoch 50 Batch 90/162] avg loss 8.55146e-06, throughput 5.93616K wps
[Epoch 50 Batch 120/162] avg loss 8.74379e-06, throughput 5.94774K wps
[Epoch 50 Batch 150/162] avg loss 7.28107e-06, throughput 5.94485K wps
Begin Testing...
[Epoch 50] train avg loss 8.02912e-06, test acc 0.9189, test avg loss 0.296711, throughput 5.96649K wps
[Epoch 51 Batch 30/162] avg loss 5.87287e-06, throughput 6.11197K wps
[Epoch 51 Batch 60/162] avg loss 8.899e-06, throughput 5.94692K wps
[Epoch 51 Batch 90/162] avg loss 6.58866e-06, throughput 5.94073K wps
[Epoch 51 Batch 120/162] avg loss 7.46555e-06, throughput 5.94407K wps
[Epoch 51 Batch 150/162] avg loss 8.10426e-06, throughput 5.94884K wps
Begin Testing...
[Epoch 51] train avg loss 7.19503e-06, test acc 0.9200, test avg loss 0.30345, throughput 5.9759K wps
[Epoch 52 Batch 30/162] avg loss 5.84918e-06, throughput 6.10124K wps
[Epoch 52 Batch 60/162] avg loss 6.60038e-06, throughput 5.94123K wps
[Epoch 52 Batch 90/162] avg loss 6.06874e-06, throughput 5.93238K wps
[Epoch 52 Batch 120/162] avg loss 6.13313e-06, throughput 5.93826K wps
[Epoch 52 Batch 150/162] avg loss 7.62899e-06, throughput 5.93341K wps
Begin Testing...
[Epoch 52] train avg loss 6.43097e-06, test acc 0.9189, test avg loss 0.309335, throughput 5.96664K wps
[Epoch 53 Batch 30/162] avg loss 6.34057e-06, throughput 6.08456K wps
[Epoch 53 Batch 60/162] avg loss 4.12036e-06, throughput 5.94664K wps
[Epoch 53 Batch 90/162] avg loss 4.94743e-06, throughput 5.94595K wps
[Epoch 53 Batch 120/162] avg loss 6.38551e-06, throughput 5.9474K wps
[Epoch 53 Batch 150/162] avg loss 5.94391e-06, throughput 5.95551K wps
Begin Testing...
[Epoch 53] train avg loss 5.48064e-06, test acc 0.9156, test avg loss 0.308805, throughput 5.97254K wps
[Epoch 54 Batch 30/162] avg loss 7.99301e-06, throughput 6.08528K wps
[Epoch 54 Batch 60/162] avg loss 5.01763e-06, throughput 5.94321K wps
[Epoch 54 Batch 90/162] avg loss 4.03876e-06, throughput 5.92932K wps
[Epoch 54 Batch 120/162] avg loss 3.9692e-06, throughput 5.93999K wps
[Epoch 54 Batch 150/162] avg loss 5.15023e-06, throughput 5.95019K wps
Begin Testing...
[Epoch 54] train avg loss 5.12502e-06, test acc 0.9144, test avg loss 0.309178, throughput 5.96777K wps
[Epoch 55 Batch 30/162] avg loss 4.60716e-06, throughput 6.09873K wps
[Epoch 55 Batch 60/162] avg loss 4.48662e-06, throughput 5.95475K wps
[Epoch 55 Batch 90/162] avg loss 5.38581e-06, throughput 5.94024K wps
[Epoch 55 Batch 120/162] avg loss 5.79877e-06, throughput 5.94434K wps
[Epoch 55 Batch 150/162] avg loss 5.63503e-06, throughput 5.95097K wps
Begin Testing...
[Epoch 55] train avg loss 5.55747e-06, test acc 0.9178, test avg loss 0.319414, throughput 5.97608K wps
[Epoch 56 Batch 30/162] avg loss 4.34978e-06, throughput 6.09382K wps
[Epoch 56 Batch 60/162] avg loss 5.48921e-06, throughput 5.93884K wps
[Epoch 56 Batch 90/162] avg loss 3.85167e-06, throughput 5.95874K wps
[Epoch 56 Batch 120/162] avg loss 3.91695e-06, throughput 5.95622K wps
[Epoch 56 Batch 150/162] avg loss 8.05671e-06, throughput 5.95252K wps
Begin Testing...
[Epoch 56] train avg loss 5.43852e-06, test acc 0.9167, test avg loss 0.317209, throughput 5.97722K wps
[Epoch 57 Batch 30/162] avg loss 4.67997e-06, throughput 6.09814K wps
[Epoch 57 Batch 60/162] avg loss 3.63611e-06, throughput 5.95502K wps
[Epoch 57 Batch 90/162] avg loss 6.00367e-06, throughput 5.94724K wps
[Epoch 57 Batch 120/162] avg loss 4.77744e-06, throughput 5.95593K wps
[Epoch 57 Batch 150/162] avg loss 3.25383e-06, throughput 5.95471K wps
Begin Testing...
[Epoch 57] train avg loss 4.39716e-06, test acc 0.9167, test avg loss 0.320097, throughput 5.98008K wps
[Epoch 58 Batch 30/162] avg loss 3.69767e-06, throughput 6.07436K wps
[Epoch 58 Batch 60/162] avg loss 3.34763e-06, throughput 5.92874K wps
[Epoch 58 Batch 90/162] avg loss 3.72922e-06, throughput 5.93945K wps
[Epoch 58 Batch 120/162] avg loss 3.85014e-06, throughput 5.93358K wps
[Epoch 58 Batch 150/162] avg loss 2.50297e-06, throughput 5.95909K wps
Begin Testing...
[Epoch 58] train avg loss 3.45374e-06, test acc 0.9167, test avg loss 0.324902, throughput 5.96511K wps
[Epoch 59 Batch 30/162] avg loss 3.3888e-06, throughput 6.08856K wps
[Epoch 59 Batch 60/162] avg loss 2.44253e-06, throughput 5.94534K wps
[Epoch 59 Batch 90/162] avg loss 3.77167e-06, throughput 5.95353K wps
[Epoch 59 Batch 120/162] avg loss 2.50076e-06, throughput 5.95399K wps
[Epoch 59 Batch 150/162] avg loss 4.27164e-06, throughput 5.95307K wps
Begin Testing...
[Epoch 59] train avg loss 3.29605e-06, test acc 0.9178, test avg loss 0.327231, throughput 5.97616K wps
Test loss 0.189944, test acc 0.9380
Total time cost 340.00s
9000 1000
[Epoch 0 Batch 30/162] avg loss 0.0148804, throughput 5.70562K wps
[Epoch 0 Batch 60/162] avg loss 0.0144705, throughput 5.94763K wps
[Epoch 0 Batch 90/162] avg loss 0.0133856, throughput 5.946K wps
[Epoch 0 Batch 120/162] avg loss 0.0130104, throughput 5.94861K wps
[Epoch 0 Batch 150/162] avg loss 0.0130255, throughput 5.96173K wps
Begin Testing...
[Epoch 0] train avg loss 0.0136653, test acc 0.6933, test avg loss 0.600245, throughput 5.90346K wps
Observed Improvement.
Begin Testing...
[Epoch 1 Batch 30/162] avg loss 0.0123332, throughput 6.10002K wps
[Epoch 1 Batch 60/162] avg loss 0.0118282, throughput 5.94562K wps
[Epoch 1 Batch 90/162] avg loss 0.011733, throughput 5.95261K wps
[Epoch 1 Batch 120/162] avg loss 0.0113792, throughput 5.94869K wps
[Epoch 1 Batch 150/162] avg loss 0.0111513, throughput 5.95596K wps
Begin Testing...
[Epoch 1] train avg loss 0.0116166, test acc 0.7844, test avg loss 0.528182, throughput 5.97863K wps
Observed Improvement.
Begin Testing...
[Epoch 2 Batch 30/162] avg loss 0.0101121, throughput 6.09545K wps
[Epoch 2 Batch 60/162] avg loss 0.00987455, throughput 5.951K wps
[Epoch 2 Batch 90/162] avg loss 0.00977541, throughput 5.95747K wps
[Epoch 2 Batch 120/162] avg loss 0.00919132, throughput 5.95014K wps
[Epoch 2 Batch 150/162] avg loss 0.00863486, throughput 5.94978K wps
Begin Testing...
[Epoch 2] train avg loss 0.00947123, test acc 0.8522, test avg loss 0.448617, throughput 5.97769K wps
Observed Improvement.
Begin Testing...
[Epoch 3 Batch 30/162] avg loss 0.00797904, throughput 6.09098K wps
[Epoch 3 Batch 60/162] avg loss 0.00787465, throughput 5.94767K wps
[Epoch 3 Batch 90/162] avg loss 0.00755399, throughput 5.93486K wps
[Epoch 3 Batch 120/162] avg loss 0.00723578, throughput 5.93137K wps
[Epoch 3 Batch 150/162] avg loss 0.00720321, throughput 5.95146K wps
Begin Testing...
[Epoch 3] train avg loss 0.00756314, test acc 0.8811, test avg loss 0.372347, throughput 5.9687K wps
Observed Improvement.
Begin Testing...
[Epoch 4 Batch 30/162] avg loss 0.00638271, throughput 6.10095K wps
[Epoch 4 Batch 60/162] avg loss 0.0062083, throughput 5.94569K wps
[Epoch 4 Batch 90/162] avg loss 0.0058072, throughput 5.95163K wps
[Epoch 4 Batch 120/162] avg loss 0.00618569, throughput 5.94415K wps
[Epoch 4 Batch 150/162] avg loss 0.00544401, throughput 5.95279K wps
Begin Testing...
[Epoch 4] train avg loss 0.00594772, test acc 0.8911, test avg loss 0.322075, throughput 5.97654K wps
Observed Improvement.
Begin Testing...
[Epoch 5 Batch 30/162] avg loss 0.00517744, throughput 6.0973K wps
[Epoch 5 Batch 60/162] avg loss 0.00518176, throughput 5.94502K wps
[Epoch 5 Batch 90/162] avg loss 0.00506906, throughput 5.95116K wps
[Epoch 5 Batch 120/162] avg loss 0.0046787, throughput 5.95474K wps
[Epoch 5 Batch 150/162] avg loss 0.00445595, throughput 5.94154K wps
Begin Testing...
[Epoch 5] train avg loss 0.00489565, test acc 0.8900, test avg loss 0.292875, throughput 5.97661K wps
[Epoch 6 Batch 30/162] avg loss 0.00402665, throughput 6.09069K wps
[Epoch 6 Batch 60/162] avg loss 0.00409109, throughput 5.95436K wps
[Epoch 6 Batch 90/162] avg loss 0.00412809, throughput 5.95367K wps
[Epoch 6 Batch 120/162] avg loss 0.00403438, throughput 5.95273K wps
[Epoch 6 Batch 150/162] avg loss 0.00389215, throughput 5.94734K wps
Begin Testing...
[Epoch 6] train avg loss 0.00401641, test acc 0.9000, test avg loss 0.271385, throughput 5.97822K wps
Observed Improvement.
Begin Testing...
[Epoch 7 Batch 30/162] avg loss 0.0034254, throughput 6.09508K wps
[Epoch 7 Batch 60/162] avg loss 0.00346582, throughput 5.94077K wps
[Epoch 7 Batch 90/162] avg loss 0.00355468, throughput 5.933K wps
[Epoch 7 Batch 120/162] avg loss 0.00328284, throughput 5.95602K wps
[Epoch 7 Batch 150/162] avg loss 0.00335427, throughput 5.95276K wps
Begin Testing...
[Epoch 7] train avg loss 0.00340616, test acc 0.9044, test avg loss 0.246875, throughput 5.97312K wps
Observed Improvement.
Begin Testing...
[Epoch 8 Batch 30/162] avg loss 0.00296529, throughput 6.09161K wps
[Epoch 8 Batch 60/162] avg loss 0.00301915, throughput 5.93622K wps
[Epoch 8 Batch 90/162] avg loss 0.00271227, throughput 5.93259K wps
[Epoch 8 Batch 120/162] avg loss 0.00275021, throughput 5.94121K wps
[Epoch 8 Batch 150/162] avg loss 0.00297225, throughput 5.94546K wps
Begin Testing...
[Epoch 8] train avg loss 0.00285774, test acc 0.9056, test avg loss 0.234583, throughput 5.9679K wps
Observed Improvement.
Begin Testing...
[Epoch 9 Batch 30/162] avg loss 0.00250269, throughput 6.08766K wps
[Epoch 9 Batch 60/162] avg loss 0.00237651, throughput 5.94497K wps
[Epoch 9 Batch 90/162] avg loss 0.00235397, throughput 5.94958K wps
[Epoch 9 Batch 120/162] avg loss 0.00228928, throughput 5.94032K wps
[Epoch 9 Batch 150/162] avg loss 0.00231399, throughput 5.9436K wps
Begin Testing...
[Epoch 9] train avg loss 0.00237268, test acc 0.9067, test avg loss 0.22396, throughput 5.96881K wps
Observed Improvement.
Begin Testing...
[Epoch 10 Batch 30/162] avg loss 0.00211187, throughput 6.09813K wps
[Epoch 10 Batch 60/162] avg loss 0.00200299, throughput 5.95088K wps
[Epoch 10 Batch 90/162] avg loss 0.00202318, throughput 5.94978K wps
[Epoch 10 Batch 120/162] avg loss 0.00219998, throughput 5.94766K wps
[Epoch 10 Batch 150/162] avg loss 0.00183623, throughput 5.94264K wps
Begin Testing...
[Epoch 10] train avg loss 0.00202597, test acc 0.9067, test avg loss 0.220482, throughput 5.97485K wps
Observed Improvement.
Begin Testing...
[Epoch 11 Batch 30/162] avg loss 0.00171615, throughput 6.08453K wps
[Epoch 11 Batch 60/162] avg loss 0.00175078, throughput 5.94744K wps
[Epoch 11 Batch 90/162] avg loss 0.0016118, throughput 5.93483K wps
[Epoch 11 Batch 120/162] avg loss 0.00162155, throughput 5.94092K wps
[Epoch 11 Batch 150/162] avg loss 0.00173365, throughput 5.94336K wps
Begin Testing...
[Epoch 11] train avg loss 0.00169535, test acc 0.9056, test avg loss 0.217728, throughput 5.96821K wps
[Epoch 12 Batch 30/162] avg loss 0.00145254, throughput 6.09217K wps
[Epoch 12 Batch 60/162] avg loss 0.00150712, throughput 5.95346K wps
[Epoch 12 Batch 90/162] avg loss 0.00154904, throughput 5.94064K wps
[Epoch 12 Batch 120/162] avg loss 0.00131472, throughput 5.94708K wps
[Epoch 12 Batch 150/162] avg loss 0.0014441, throughput 5.94771K wps
Begin Testing...
[Epoch 12] train avg loss 0.00143826, test acc 0.9078, test avg loss 0.212682, throughput 5.97354K wps
Observed Improvement.
Begin Testing...
[Epoch 13 Batch 30/162] avg loss 0.00135208, throughput 6.10618K wps
[Epoch 13 Batch 60/162] avg loss 0.00126878, throughput 5.96341K wps
[Epoch 13 Batch 90/162] avg loss 0.00131114, throughput 5.94197K wps
[Epoch 13 Batch 120/162] avg loss 0.00121379, throughput 5.93734K wps
[Epoch 13 Batch 150/162] avg loss 0.00114892, throughput 5.95514K wps
Begin Testing...
[Epoch 13] train avg loss 0.0012351, test acc 0.9078, test avg loss 0.211701, throughput 5.9778K wps
Observed Improvement.
Begin Testing...
[Epoch 14 Batch 30/162] avg loss 0.00108657, throughput 6.0934K wps
[Epoch 14 Batch 60/162] avg loss 0.000980759, throughput 5.94023K wps
[Epoch 14 Batch 90/162] avg loss 0.000897704, throughput 5.94491K wps
[Epoch 14 Batch 120/162] avg loss 0.000988816, throughput 5.94349K wps
[Epoch 14 Batch 150/162] avg loss 0.00100407, throughput 5.95008K wps
Begin Testing...
[Epoch 14] train avg loss 0.000986657, test acc 0.9089, test avg loss 0.208553, throughput 5.97209K wps
Observed Improvement.
Begin Testing...
[Epoch 15 Batch 30/162] avg loss 0.000864659, throughput 6.09701K wps
[Epoch 15 Batch 60/162] avg loss 0.000712581, throughput 5.95122K wps
[Epoch 15 Batch 90/162] avg loss 0.000931489, throughput 5.94702K wps
[Epoch 15 Batch 120/162] avg loss 0.000761852, throughput 5.94685K wps
[Epoch 15 Batch 150/162] avg loss 0.000891315, throughput 5.93591K wps
Begin Testing...
[Epoch 15] train avg loss 0.000833725, test acc 0.9067, test avg loss 0.216796, throughput 5.97252K wps
[Epoch 16 Batch 30/162] avg loss 0.000766368, throughput 6.08618K wps
[Epoch 16 Batch 60/162] avg loss 0.0007091, throughput 5.92469K wps
[Epoch 16 Batch 90/162] avg loss 0.000859636, throughput 5.94306K wps
[Epoch 16 Batch 120/162] avg loss 0.000723256, throughput 5.93082K wps
[Epoch 16 Batch 150/162] avg loss 0.000625937, throughput 5.92915K wps
Begin Testing...
[Epoch 16] train avg loss 0.000726568, test acc 0.9122, test avg loss 0.207923, throughput 5.95953K wps
Observed Improvement.
Begin Testing...
[Epoch 17 Batch 30/162] avg loss 0.000674744, throughput 6.08741K wps
[Epoch 17 Batch 60/162] avg loss 0.000535354, throughput 5.94199K wps
[Epoch 17 Batch 90/162] avg loss 0.000651463, throughput 5.93249K wps
[Epoch 17 Batch 120/162] avg loss 0.000597696, throughput 5.94469K wps
[Epoch 17 Batch 150/162] avg loss 0.000560919, throughput 5.94345K wps
Begin Testing...
[Epoch 17] train avg loss 0.000600835, test acc 0.9100, test avg loss 0.212176, throughput 5.96756K wps
[Epoch 18 Batch 30/162] avg loss 0.000431558, throughput 6.10115K wps
[Epoch 18 Batch 60/162] avg loss 0.000540177, throughput 5.94343K wps
[Epoch 18 Batch 90/162] avg loss 0.00053482, throughput 5.95237K wps
[Epoch 18 Batch 120/162] avg loss 0.000451558, throughput 5.95239K wps
[Epoch 18 Batch 150/162] avg loss 0.00056989, throughput 5.94439K wps
Begin Testing...
[Epoch 18] train avg loss 0.000504682, test acc 0.9111, test avg loss 0.213736, throughput 5.9755K wps
[Epoch 19 Batch 30/162] avg loss 0.000471183, throughput 6.08057K wps
[Epoch 19 Batch 60/162] avg loss 0.000514755, throughput 5.95005K wps
[Epoch 19 Batch 90/162] avg loss 0.000377327, throughput 5.93983K wps
[Epoch 19 Batch 120/162] avg loss 0.000486415, throughput 5.94617K wps
[Epoch 19 Batch 150/162] avg loss 0.000401923, throughput 5.92921K wps
Begin Testing...
[Epoch 19] train avg loss 0.00045242, test acc 0.9144, test avg loss 0.2155, throughput 5.96701K wps
Observed Improvement.
Begin Testing...
[Epoch 20 Batch 30/162] avg loss 0.000368865, throughput 6.09533K wps
[Epoch 20 Batch 60/162] avg loss 0.000317339, throughput 5.9488K wps
[Epoch 20 Batch 90/162] avg loss 0.000335726, throughput 5.94731K wps
[Epoch 20 Batch 120/162] avg loss 0.000335313, throughput 5.94355K wps
[Epoch 20 Batch 150/162] avg loss 0.000402138, throughput 5.9431K wps
Begin Testing...
[Epoch 20] train avg loss 0.000349768, test acc 0.9122, test avg loss 0.216425, throughput 5.97237K wps
[Epoch 21 Batch 30/162] avg loss 0.00030468, throughput 6.08667K wps
[Epoch 21 Batch 60/162] avg loss 0.000283471, throughput 5.93346K wps
[Epoch 21 Batch 90/162] avg loss 0.000273972, throughput 5.93482K wps
[Epoch 21 Batch 120/162] avg loss 0.000313741, throughput 5.93827K wps
[Epoch 21 Batch 150/162] avg loss 0.000280868, throughput 5.9402K wps
Begin Testing...
[Epoch 21] train avg loss 0.000291931, test acc 0.9122, test avg loss 0.220214, throughput 5.96324K wps
[Epoch 22 Batch 30/162] avg loss 0.000229884, throughput 6.10116K wps
[Epoch 22 Batch 60/162] avg loss 0.000254659, throughput 5.92541K wps
[Epoch 22 Batch 90/162] avg loss 0.000250429, throughput 5.94344K wps
[Epoch 22 Batch 120/162] avg loss 0.000237031, throughput 5.95322K wps
[Epoch 22 Batch 150/162] avg loss 0.000256653, throughput 5.93898K wps
Begin Testing...
[Epoch 22] train avg loss 0.000243021, test acc 0.9133, test avg loss 0.222223, throughput 5.96995K wps
[Epoch 23 Batch 30/162] avg loss 0.000185652, throughput 6.09618K wps
[Epoch 23 Batch 60/162] avg loss 0.000210627, throughput 5.9409K wps
[Epoch 23 Batch 90/162] avg loss 0.000212767, throughput 5.93483K wps
[Epoch 23 Batch 120/162] avg loss 0.000223685, throughput 5.94327K wps
[Epoch 23 Batch 150/162] avg loss 0.000232467, throughput 5.91848K wps
Begin Testing...
[Epoch 23] train avg loss 0.000212064, test acc 0.9111, test avg loss 0.238403, throughput 5.96459K wps
[Epoch 24 Batch 30/162] avg loss 0.000185607, throughput 6.08389K wps
[Epoch 24 Batch 60/162] avg loss 0.000194368, throughput 5.9421K wps
[Epoch 24 Batch 90/162] avg loss 0.000197624, throughput 5.94367K wps
[Epoch 24 Batch 120/162] avg loss 0.000185971, throughput 5.93643K wps
[Epoch 24 Batch 150/162] avg loss 0.000241361, throughput 5.961K wps
Begin Testing...
[Epoch 24] train avg loss 0.000201171, test acc 0.9122, test avg loss 0.232704, throughput 5.97018K wps
[Epoch 25 Batch 30/162] avg loss 0.000150024, throughput 6.10473K wps
[Epoch 25 Batch 60/162] avg loss 0.000138601, throughput 5.946K wps
[Epoch 25 Batch 90/162] avg loss 0.000198892, throughput 5.95005K wps
[Epoch 25 Batch 120/162] avg loss 0.000154499, throughput 5.94889K wps
[Epoch 25 Batch 150/162] avg loss 0.000139833, throughput 5.94763K wps
Begin Testing...
[Epoch 25] train avg loss 0.000158467, test acc 0.9133, test avg loss 0.240297, throughput 5.97527K wps
[Epoch 26 Batch 30/162] avg loss 0.000149549, throughput 6.09295K wps
[Epoch 26 Batch 60/162] avg loss 0.00013223, throughput 5.95518K wps
[Epoch 26 Batch 90/162] avg loss 0.000129454, throughput 5.94666K wps
[Epoch 26 Batch 120/162] avg loss 0.000138108, throughput 5.95136K wps
[Epoch 26 Batch 150/162] avg loss 0.000142227, throughput 5.95705K wps
Begin Testing...
[Epoch 26] train avg loss 0.000139261, test acc 0.9122, test avg loss 0.244073, throughput 5.97761K wps
[Epoch 27 Batch 30/162] avg loss 0.000136954, throughput 6.09087K wps
[Epoch 27 Batch 60/162] avg loss 0.000118807, throughput 5.94141K wps
[Epoch 27 Batch 90/162] avg loss 0.000109637, throughput 5.94085K wps
[Epoch 27 Batch 120/162] avg loss 0.000134082, throughput 5.94066K wps
[Epoch 27 Batch 150/162] avg loss 0.000128803, throughput 5.94204K wps
Begin Testing...
[Epoch 27] train avg loss 0.000124012, test acc 0.9133, test avg loss 0.245978, throughput 5.96935K wps
[Epoch 28 Batch 30/162] avg loss 0.000106293, throughput 6.0986K wps
[Epoch 28 Batch 60/162] avg loss 0.000112704, throughput 5.95002K wps
[Epoch 28 Batch 90/162] avg loss 0.000115569, throughput 5.93811K wps
[Epoch 28 Batch 120/162] avg loss 8.59446e-05, throughput 5.93852K wps
[Epoch 28 Batch 150/162] avg loss 0.000104334, throughput 5.95688K wps
Begin Testing...
[Epoch 28] train avg loss 0.000103689, test acc 0.9122, test avg loss 0.252085, throughput 5.97238K wps
[Epoch 29 Batch 30/162] avg loss 9.40115e-05, throughput 6.10529K wps
[Epoch 29 Batch 60/162] avg loss 9.31049e-05, throughput 5.95143K wps
[Epoch 29 Batch 90/162] avg loss 9.31762e-05, throughput 5.93965K wps
[Epoch 29 Batch 120/162] avg loss 8.26643e-05, throughput 5.95349K wps
[Epoch 29 Batch 150/162] avg loss 9.65096e-05, throughput 5.95616K wps
Begin Testing...
[Epoch 29] train avg loss 8.98377e-05, test acc 0.9100, test avg loss 0.255059, throughput 5.97824K wps
[Epoch 30 Batch 30/162] avg loss 7.12763e-05, throughput 6.09443K wps
[Epoch 30 Batch 60/162] avg loss 7.40649e-05, throughput 5.94235K wps
[Epoch 30 Batch 90/162] avg loss 7.32383e-05, throughput 5.95886K wps
[Epoch 30 Batch 120/162] avg loss 7.63402e-05, throughput 5.94847K wps
[Epoch 30 Batch 150/162] avg loss 8.56537e-05, throughput 5.93843K wps
Begin Testing...
[Epoch 30] train avg loss 7.67382e-05, test acc 0.9144, test avg loss 0.258315, throughput 5.97182K wps
Observed Improvement.
Begin Testing...
[Epoch 31 Batch 30/162] avg loss 6.25586e-05, throughput 6.09677K wps
[Epoch 31 Batch 60/162] avg loss 6.89513e-05, throughput 5.94188K wps
[Epoch 31 Batch 90/162] avg loss 7.45715e-05, throughput 5.94155K wps
[Epoch 31 Batch 120/162] avg loss 7.60328e-05, throughput 5.94396K wps
[Epoch 31 Batch 150/162] avg loss 7.29656e-05, throughput 5.93549K wps
Begin Testing...
[Epoch 31] train avg loss 7.0903e-05, test acc 0.9156, test avg loss 0.260248, throughput 5.97033K wps
Observed Improvement.
Begin Testing...
[Epoch 32 Batch 30/162] avg loss 6.09507e-05, throughput 6.09596K wps
[Epoch 32 Batch 60/162] avg loss 5.9677e-05, throughput 5.95006K wps
[Epoch 32 Batch 90/162] avg loss 6.04043e-05, throughput 5.94152K wps
[Epoch 32 Batch 120/162] avg loss 6.31264e-05, throughput 5.95041K wps
[Epoch 32 Batch 150/162] avg loss 7.24417e-05, throughput 5.93392K wps
Begin Testing...
[Epoch 32] train avg loss 6.22279e-05, test acc 0.9133, test avg loss 0.264549, throughput 5.97184K wps
[Epoch 33 Batch 30/162] avg loss 5.60543e-05, throughput 6.09556K wps
[Epoch 33 Batch 60/162] avg loss 4.41704e-05, throughput 5.95717K wps
[Epoch 33 Batch 90/162] avg loss 6.18957e-05, throughput 5.96834K wps
[Epoch 33 Batch 120/162] avg loss 4.71287e-05, throughput 5.95285K wps
[Epoch 33 Batch 150/162] avg loss 5.00915e-05, throughput 5.94201K wps
Begin Testing...
[Epoch 33] train avg loss 5.3872e-05, test acc 0.9156, test avg loss 0.265247, throughput 5.98047K wps
Observed Improvement.
Begin Testing...
[Epoch 34 Batch 30/162] avg loss 4.61926e-05, throughput 6.10081K wps
[Epoch 34 Batch 60/162] avg loss 5.19e-05, throughput 5.93599K wps
[Epoch 34 Batch 90/162] avg loss 4.81845e-05, throughput 5.93333K wps
[Epoch 34 Batch 120/162] avg loss 4.82171e-05, throughput 5.95164K wps
[Epoch 34 Batch 150/162] avg loss 4.59441e-05, throughput 5.94908K wps
Begin Testing...
[Epoch 34] train avg loss 4.8051e-05, test acc 0.9144, test avg loss 0.270932, throughput 5.967K wps
[Epoch 35 Batch 30/162] avg loss 4.85244e-05, throughput 6.07341K wps
[Epoch 35 Batch 60/162] avg loss 3.41767e-05, throughput 5.94191K wps
[Epoch 35 Batch 90/162] avg loss 4.64036e-05, throughput 5.9478K wps
[Epoch 35 Batch 120/162] avg loss 3.99853e-05, throughput 5.95138K wps
[Epoch 35 Batch 150/162] avg loss 3.62084e-05, throughput 5.94827K wps
Begin Testing...
[Epoch 35] train avg loss 4.09024e-05, test acc 0.9133, test avg loss 0.271811, throughput 5.97022K wps
[Epoch 36 Batch 30/162] avg loss 3.65123e-05, throughput 6.08191K wps
[Epoch 36 Batch 60/162] avg loss 4.38497e-05, throughput 5.95773K wps
[Epoch 36 Batch 90/162] avg loss 3.40502e-05, throughput 5.93648K wps
[Epoch 36 Batch 120/162] avg loss 4.1928e-05, throughput 5.95415K wps
[Epoch 36 Batch 150/162] avg loss 3.04818e-05, throughput 5.93851K wps
Begin Testing...
[Epoch 36] train avg loss 3.71254e-05, test acc 0.9167, test avg loss 0.281351, throughput 5.97155K wps
Observed Improvement.
Begin Testing...
[Epoch 37 Batch 30/162] avg loss 2.9442e-05, throughput 6.10066K wps
[Epoch 37 Batch 60/162] avg loss 2.92191e-05, throughput 5.94549K wps
[Epoch 37 Batch 90/162] avg loss 3.9798e-05, throughput 5.95683K wps
[Epoch 37 Batch 120/162] avg loss 3.18373e-05, throughput 5.94947K wps
[Epoch 37 Batch 150/162] avg loss 2.72835e-05, throughput 5.95357K wps
Begin Testing...
[Epoch 37] train avg loss 3.15109e-05, test acc 0.9156, test avg loss 0.286606, throughput 5.97935K wps
[Epoch 38 Batch 30/162] avg loss 2.72933e-05, throughput 6.10232K wps
[Epoch 38 Batch 60/162] avg loss 2.7627e-05, throughput 5.94416K wps
[Epoch 38 Batch 90/162] avg loss 3.13335e-05, throughput 5.94298K wps
[Epoch 38 Batch 120/162] avg loss 2.7098e-05, throughput 5.93783K wps
[Epoch 38 Batch 150/162] avg loss 2.49972e-05, throughput 5.94608K wps
Begin Testing...
[Epoch 38] train avg loss 2.73328e-05, test acc 0.9133, test avg loss 0.295049, throughput 5.97101K wps
[Epoch 39 Batch 30/162] avg loss 2.83185e-05, throughput 6.1013K wps
[Epoch 39 Batch 60/162] avg loss 2.74801e-05, throughput 5.94316K wps
[Epoch 39 Batch 90/162] avg loss 2.18148e-05, throughput 5.93226K wps
[Epoch 39 Batch 120/162] avg loss 2.61982e-05, throughput 5.94349K wps
[Epoch 39 Batch 150/162] avg loss 2.71085e-05, throughput 5.94254K wps
Begin Testing...
[Epoch 39] train avg loss 2.66145e-05, test acc 0.9156, test avg loss 0.295614, throughput 5.96912K wps
[Epoch 40 Batch 30/162] avg loss 2.01851e-05, throughput 6.08757K wps
[Epoch 40 Batch 60/162] avg loss 2.26059e-05, throughput 5.94895K wps
[Epoch 40 Batch 90/162] avg loss 2.38064e-05, throughput 5.94949K wps
[Epoch 40 Batch 120/162] avg loss 1.82962e-05, throughput 5.93703K wps
[Epoch 40 Batch 150/162] avg loss 1.90431e-05, throughput 5.93918K wps
Begin Testing...
[Epoch 40] train avg loss 2.17101e-05, test acc 0.9144, test avg loss 0.295005, throughput 5.97096K wps
[Epoch 41 Batch 30/162] avg loss 1.94749e-05, throughput 6.07451K wps
[Epoch 41 Batch 60/162] avg loss 2.03502e-05, throughput 5.94475K wps
[Epoch 41 Batch 90/162] avg loss 2.26649e-05, throughput 5.93788K wps
[Epoch 41 Batch 120/162] avg loss 1.91254e-05, throughput 5.93605K wps
[Epoch 41 Batch 150/162] avg loss 2.20902e-05, throughput 5.94591K wps
Begin Testing...
[Epoch 41] train avg loss 2.07191e-05, test acc 0.9144, test avg loss 0.302795, throughput 5.96571K wps
[Epoch 42 Batch 30/162] avg loss 2.06281e-05, throughput 6.09671K wps
[Epoch 42 Batch 60/162] avg loss 1.88214e-05, throughput 5.93836K wps
[Epoch 42 Batch 90/162] avg loss 1.70371e-05, throughput 5.93758K wps
[Epoch 42 Batch 120/162] avg loss 1.88542e-05, throughput 5.94356K wps
[Epoch 42 Batch 150/162] avg loss 1.51782e-05, throughput 5.94298K wps
Begin Testing...
[Epoch 42] train avg loss 1.83007e-05, test acc 0.9156, test avg loss 0.312477, throughput 5.96977K wps
[Epoch 43 Batch 30/162] avg loss 1.79221e-05, throughput 6.09973K wps
[Epoch 43 Batch 60/162] avg loss 1.55914e-05, throughput 5.93951K wps
[Epoch 43 Batch 90/162] avg loss 1.40794e-05, throughput 5.94115K wps
[Epoch 43 Batch 120/162] avg loss 1.45597e-05, throughput 5.94307K wps
[Epoch 43 Batch 150/162] avg loss 1.56555e-05, throughput 5.95075K wps
Begin Testing...
[Epoch 43] train avg loss 1.66051e-05, test acc 0.9178, test avg loss 0.315117, throughput 5.97215K wps
Observed Improvement.
Begin Testing...
[Epoch 44 Batch 30/162] avg loss 1.48269e-05, throughput 6.0896K wps
[Epoch 44 Batch 60/162] avg loss 1.87863e-05, throughput 5.94382K wps
[Epoch 44 Batch 90/162] avg loss 1.38077e-05, throughput 5.94961K wps
[Epoch 44 Batch 120/162] avg loss 1.66172e-05, throughput 5.95004K wps
[Epoch 44 Batch 150/162] avg loss 1.78191e-05, throughput 5.93986K wps
Begin Testing...
[Epoch 44] train avg loss 1.63135e-05, test acc 0.9133, test avg loss 0.315079, throughput 5.97149K wps
[Epoch 45 Batch 30/162] avg loss 1.06597e-05, throughput 6.09636K wps
[Epoch 45 Batch 60/162] avg loss 1.08859e-05, throughput 5.93853K wps
[Epoch 45 Batch 90/162] avg loss 1.44128e-05, throughput 5.94999K wps
[Epoch 45 Batch 120/162] avg loss 1.32672e-05, throughput 5.95305K wps
[Epoch 45 Batch 150/162] avg loss 1.09753e-05, throughput 5.95242K wps
Begin Testing...
[Epoch 45] train avg loss 1.19641e-05, test acc 0.9133, test avg loss 0.319701, throughput 5.9744K wps
[Epoch 46 Batch 30/162] avg loss 1.26026e-05, throughput 6.0941K wps
[Epoch 46 Batch 60/162] avg loss 1.32032e-05, throughput 5.94838K wps
[Epoch 46 Batch 90/162] avg loss 1.40637e-05, throughput 5.94879K wps
[Epoch 46 Batch 120/162] avg loss 1.144e-05, throughput 5.94732K wps
[Epoch 46 Batch 150/162] avg loss 9.8652e-06, throughput 5.94788K wps
Begin Testing...
[Epoch 46] train avg loss 1.21673e-05, test acc 0.9133, test avg loss 0.318061, throughput 5.97343K wps
[Epoch 47 Batch 30/162] avg loss 9.9909e-06, throughput 6.09909K wps
[Epoch 47 Batch 60/162] avg loss 1.16158e-05, throughput 5.93497K wps
[Epoch 47 Batch 90/162] avg loss 9.77969e-06, throughput 5.92555K wps
[Epoch 47 Batch 120/162] avg loss 9.83516e-06, throughput 5.94117K wps
[Epoch 47 Batch 150/162] avg loss 1.62787e-05, throughput 5.94632K wps
Begin Testing...
[Epoch 47] train avg loss 1.17886e-05, test acc 0.9122, test avg loss 0.350343, throughput 5.9671K wps
[Epoch 48 Batch 30/162] avg loss 1.13358e-05, throughput 6.09852K wps
[Epoch 48 Batch 60/162] avg loss 9.51139e-06, throughput 5.93383K wps
[Epoch 48 Batch 90/162] avg loss 1.18022e-05, throughput 5.9389K wps
[Epoch 48 Batch 120/162] avg loss 7.29654e-06, throughput 5.95917K wps
[Epoch 48 Batch 150/162] avg loss 8.04835e-06, throughput 5.94059K wps
Begin Testing...
[Epoch 48] train avg loss 9.49563e-06, test acc 0.9111, test avg loss 0.331466, throughput 5.97206K wps
[Epoch 49 Batch 30/162] avg loss 1.02295e-05, throughput 6.10449K wps
[Epoch 49 Batch 60/162] avg loss 1.11142e-05, throughput 5.95844K wps
[Epoch 49 Batch 90/162] avg loss 9.16395e-06, throughput 5.95035K wps
[Epoch 49 Batch 120/162] avg loss 1.60801e-05, throughput 5.955K wps
[Epoch 49 Batch 150/162] avg loss 1.55208e-05, throughput 5.95751K wps
Begin Testing...
[Epoch 49] train avg loss 1.22547e-05, test acc 0.9156, test avg loss 0.325708, throughput 5.9815K wps
[Epoch 50 Batch 30/162] avg loss 8.37759e-06, throughput 6.08093K wps
[Epoch 50 Batch 60/162] avg loss 8.55633e-06, throughput 5.94489K wps
[Epoch 50 Batch 90/162] avg loss 9.30513e-06, throughput 5.94557K wps
[Epoch 50 Batch 120/162] avg loss 8.0491e-06, throughput 5.94977K wps
[Epoch 50 Batch 150/162] avg loss 9.51656e-06, throughput 5.95469K wps
Begin Testing...
[Epoch 50] train avg loss 8.63859e-06, test acc 0.9111, test avg loss 0.334304, throughput 5.9737K wps
[Epoch 51 Batch 30/162] avg loss 6.26837e-06, throughput 6.0956K wps
[Epoch 51 Batch 60/162] avg loss 9.19922e-06, throughput 5.95591K wps
[Epoch 51 Batch 90/162] avg loss 8.38522e-06, throughput 5.94742K wps
[Epoch 51 Batch 120/162] avg loss 7.32922e-06, throughput 5.96075K wps
[Epoch 51 Batch 150/162] avg loss 7.16683e-06, throughput 5.95173K wps
Begin Testing...
[Epoch 51] train avg loss 7.55117e-06, test acc 0.9111, test avg loss 0.332906, throughput 5.97762K wps
[Epoch 52 Batch 30/162] avg loss 7.80238e-06, throughput 6.10497K wps
[Epoch 52 Batch 60/162] avg loss 6.91214e-06, throughput 5.9518K wps
[Epoch 52 Batch 90/162] avg loss 8.57747e-06, throughput 5.94606K wps
[Epoch 52 Batch 120/162] avg loss 6.21037e-06, throughput 5.9558K wps
[Epoch 52 Batch 150/162] avg loss 9.15228e-06, throughput 5.95768K wps
Begin Testing...
[Epoch 52] train avg loss 7.705e-06, test acc 0.9078, test avg loss 0.339287, throughput 5.97927K wps
[Epoch 53 Batch 30/162] avg loss 9.42971e-06, throughput 6.08058K wps
[Epoch 53 Batch 60/162] avg loss 5.65483e-06, throughput 5.94947K wps
[Epoch 53 Batch 90/162] avg loss 4.4841e-06, throughput 5.93999K wps
[Epoch 53 Batch 120/162] avg loss 7.40264e-06, throughput 5.93614K wps
[Epoch 53 Batch 150/162] avg loss 6.70989e-06, throughput 5.93638K wps
Begin Testing...
[Epoch 53] train avg loss 6.62472e-06, test acc 0.9089, test avg loss 0.343199, throughput 5.96574K wps
[Epoch 54 Batch 30/162] avg loss 7.41293e-06, throughput 6.09141K wps
[Epoch 54 Batch 60/162] avg loss 4.68536e-06, throughput 5.94579K wps
[Epoch 54 Batch 90/162] avg loss 4.46961e-06, throughput 5.95295K wps
[Epoch 54 Batch 120/162] avg loss 3.94136e-06, throughput 5.94838K wps
[Epoch 54 Batch 150/162] avg loss 6.64148e-06, throughput 5.94228K wps
Begin Testing...
[Epoch 54] train avg loss 5.37844e-06, test acc 0.9100, test avg loss 0.348332, throughput 5.97233K wps
[Epoch 55 Batch 30/162] avg loss 4.62347e-06, throughput 6.0895K wps
[Epoch 55 Batch 60/162] avg loss 3.91585e-06, throughput 5.94111K wps
[Epoch 55 Batch 90/162] avg loss 4.50588e-06, throughput 5.95325K wps
[Epoch 55 Batch 120/162] avg loss 4.20566e-06, throughput 5.95021K wps
[Epoch 55 Batch 150/162] avg loss 6.36089e-06, throughput 5.94999K wps
Begin Testing...
[Epoch 55] train avg loss 4.97282e-06, test acc 0.9122, test avg loss 0.344846, throughput 5.97477K wps
[Epoch 56 Batch 30/162] avg loss 8.70622e-06, throughput 6.09222K wps
[Epoch 56 Batch 60/162] avg loss 5.99852e-06, throughput 5.94901K wps
[Epoch 56 Batch 90/162] avg loss 4.84665e-06, throughput 5.94616K wps
[Epoch 56 Batch 120/162] avg loss 4.74852e-06, throughput 5.94786K wps
[Epoch 56 Batch 150/162] avg loss 3.75039e-06, throughput 5.94164K wps
Begin Testing...
[Epoch 56] train avg loss 5.42525e-06, test acc 0.9100, test avg loss 0.363636, throughput 5.97262K wps
[Epoch 57 Batch 30/162] avg loss 5.37541e-06, throughput 6.09716K wps
[Epoch 57 Batch 60/162] avg loss 3.03214e-06, throughput 5.94271K wps
[Epoch 57 Batch 90/162] avg loss 4.03702e-06, throughput 5.94345K wps
[Epoch 57 Batch 120/162] avg loss 3.45301e-06, throughput 5.94914K wps
[Epoch 57 Batch 150/162] avg loss 3.70051e-06, throughput 5.93998K wps
Begin Testing...
[Epoch 57] train avg loss 3.98502e-06, test acc 0.9089, test avg loss 0.369598, throughput 5.97192K wps
[Epoch 58 Batch 30/162] avg loss 4.60673e-06, throughput 6.09986K wps
[Epoch 58 Batch 60/162] avg loss 3.8582e-06, throughput 5.94488K wps
[Epoch 58 Batch 90/162] avg loss 3.84173e-06, throughput 5.95556K wps
[Epoch 58 Batch 120/162] avg loss 4.35688e-06, throughput 5.96338K wps
[Epoch 58 Batch 150/162] avg loss 3.98463e-06, throughput 5.96168K wps
Begin Testing...
[Epoch 58] train avg loss 4.09763e-06, test acc 0.9078, test avg loss 0.380735, throughput 5.98075K wps
[Epoch 59 Batch 30/162] avg loss 3.28217e-06, throughput 6.08793K wps
[Epoch 59 Batch 60/162] avg loss 7.48443e-06, throughput 5.95367K wps
[Epoch 59 Batch 90/162] avg loss 3.76961e-06, throughput 5.95264K wps
[Epoch 59 Batch 120/162] avg loss 3.44499e-06, throughput 5.9592K wps
[Epoch 59 Batch 150/162] avg loss 4.50126e-06, throughput 5.95245K wps
Begin Testing...
[Epoch 59] train avg loss 4.37784e-06, test acc 0.9100, test avg loss 0.380683, throughput 5.9772K wps
Test loss 0.284764, test acc 0.9230
Total time cost 340.40s
9000 1000
[Epoch 0 Batch 30/162] avg loss 0.0147857, throughput 5.68923K wps
[Epoch 0 Batch 60/162] avg loss 0.0140017, throughput 5.92675K wps
[Epoch 0 Batch 90/162] avg loss 0.0135454, throughput 5.95541K wps
[Epoch 0 Batch 120/162] avg loss 0.0129757, throughput 5.93514K wps
[Epoch 0 Batch 150/162] avg loss 0.0128077, throughput 5.95034K wps
Begin Testing...
[Epoch 0] train avg loss 0.0135348, test acc 0.7144, test avg loss 0.574337, throughput 5.89299K wps
Observed Improvement.
Begin Testing...
[Epoch 1 Batch 30/162] avg loss 0.0118293, throughput 6.0859K wps
[Epoch 1 Batch 60/162] avg loss 0.0114933, throughput 5.93671K wps
[Epoch 1 Batch 90/162] avg loss 0.0114569, throughput 5.9345K wps
[Epoch 1 Batch 120/162] avg loss 0.0108241, throughput 5.94829K wps
[Epoch 1 Batch 150/162] avg loss 0.0106889, throughput 5.95963K wps
Begin Testing...
[Epoch 1] train avg loss 0.0111748, test acc 0.8289, test avg loss 0.492291, throughput 5.9716K wps
Observed Improvement.
Begin Testing...
[Epoch 2 Batch 30/162] avg loss 0.00969193, throughput 6.08682K wps
[Epoch 2 Batch 60/162] avg loss 0.00961349, throughput 5.94642K wps
[Epoch 2 Batch 90/162] avg loss 0.00905287, throughput 5.95199K wps
[Epoch 2 Batch 120/162] avg loss 0.00898996, throughput 5.94713K wps
[Epoch 2 Batch 150/162] avg loss 0.00881467, throughput 5.94398K wps
Begin Testing...
[Epoch 2] train avg loss 0.0091769, test acc 0.8833, test avg loss 0.404548, throughput 5.97267K wps
Observed Improvement.
Begin Testing...
[Epoch 3 Batch 30/162] avg loss 0.00785728, throughput 6.09226K wps
[Epoch 3 Batch 60/162] avg loss 0.00767662, throughput 5.95063K wps
[Epoch 3 Batch 90/162] avg loss 0.0075392, throughput 5.95935K wps
[Epoch 3 Batch 120/162] avg loss 0.00706946, throughput 5.96122K wps
[Epoch 3 Batch 150/162] avg loss 0.00685133, throughput 5.95471K wps
Begin Testing...
[Epoch 3] train avg loss 0.00734053, test acc 0.9000, test avg loss 0.328544, throughput 5.97959K wps
Observed Improvement.
Begin Testing...
[Epoch 4 Batch 30/162] avg loss 0.00607512, throughput 6.10178K wps
[Epoch 4 Batch 60/162] avg loss 0.00637369, throughput 5.94813K wps
[Epoch 4 Batch 90/162] avg loss 0.00597694, throughput 5.95021K wps
[Epoch 4 Batch 120/162] avg loss 0.00550879, throughput 5.95457K wps
[Epoch 4 Batch 150/162] avg loss 0.00534453, throughput 5.95269K wps
Begin Testing...
[Epoch 4] train avg loss 0.00577406, test acc 0.9167, test avg loss 0.27663, throughput 5.97777K wps
Observed Improvement.
Begin Testing...
[Epoch 5 Batch 30/162] avg loss 0.00491778, throughput 6.0797K wps
[Epoch 5 Batch 60/162] avg loss 0.00485356, throughput 5.93704K wps
[Epoch 5 Batch 90/162] avg loss 0.00496553, throughput 5.93226K wps
[Epoch 5 Batch 120/162] avg loss 0.00476432, throughput 5.95248K wps
[Epoch 5 Batch 150/162] avg loss 0.0044604, throughput 5.93942K wps
Begin Testing...
[Epoch 5] train avg loss 0.0047537, test acc 0.9189, test avg loss 0.243902, throughput 5.96576K wps
Observed Improvement.
Begin Testing...
[Epoch 6 Batch 30/162] avg loss 0.00403981, throughput 6.10389K wps
[Epoch 6 Batch 60/162] avg loss 0.00405943, throughput 5.95671K wps
[Epoch 6 Batch 90/162] avg loss 0.00419626, throughput 5.96304K wps
[Epoch 6 Batch 120/162] avg loss 0.00397774, throughput 5.95403K wps
[Epoch 6 Batch 150/162] avg loss 0.00361919, throughput 5.92982K wps
Begin Testing...
[Epoch 6] train avg loss 0.00394185, test acc 0.9222, test avg loss 0.223222, throughput 5.97619K wps
Observed Improvement.
Begin Testing...
[Epoch 7 Batch 30/162] avg loss 0.00337212, throughput 6.07936K wps
[Epoch 7 Batch 60/162] avg loss 0.00318647, throughput 5.93984K wps
[Epoch 7 Batch 90/162] avg loss 0.00324115, throughput 5.94431K wps
[Epoch 7 Batch 120/162] avg loss 0.00314129, throughput 5.94228K wps
[Epoch 7 Batch 150/162] avg loss 0.00347108, throughput 5.95286K wps
Begin Testing...
[Epoch 7] train avg loss 0.00328197, test acc 0.9289, test avg loss 0.206624, throughput 5.9699K wps
Observed Improvement.
Begin Testing...
[Epoch 8 Batch 30/162] avg loss 0.0027017, throughput 6.09129K wps
[Epoch 8 Batch 60/162] avg loss 0.00295808, throughput 5.92433K wps
[Epoch 8 Batch 90/162] avg loss 0.00288159, throughput 5.93227K wps
[Epoch 8 Batch 120/162] avg loss 0.00260046, throughput 5.95469K wps
[Epoch 8 Batch 150/162] avg loss 0.00262146, throughput 5.94599K wps
Begin Testing...
[Epoch 8] train avg loss 0.00274782, test acc 0.9356, test avg loss 0.196326, throughput 5.96821K wps
Observed Improvement.
Begin Testing...
[Epoch 9 Batch 30/162] avg loss 0.00240513, throughput 6.09923K wps
[Epoch 9 Batch 60/162] avg loss 0.00232577, throughput 5.95385K wps
[Epoch 9 Batch 90/162] avg loss 0.00238735, throughput 5.94775K wps
[Epoch 9 Batch 120/162] avg loss 0.00227903, throughput 5.93919K wps
[Epoch 9 Batch 150/162] avg loss 0.00202507, throughput 5.936K wps
Begin Testing...
[Epoch 9] train avg loss 0.00233588, test acc 0.9344, test avg loss 0.191532, throughput 5.97159K wps
[Epoch 10 Batch 30/162] avg loss 0.00211805, throughput 6.09557K wps
[Epoch 10 Batch 60/162] avg loss 0.00211964, throughput 5.94408K wps
[Epoch 10 Batch 90/162] avg loss 0.00193365, throughput 5.94267K wps
[Epoch 10 Batch 120/162] avg loss 0.00192101, throughput 5.95679K wps
[Epoch 10 Batch 150/162] avg loss 0.00196054, throughput 5.93615K wps
Begin Testing...
[Epoch 10] train avg loss 0.00199763, test acc 0.9333, test avg loss 0.188139, throughput 5.97165K wps
[Epoch 11 Batch 30/162] avg loss 0.00160477, throughput 6.08859K wps
[Epoch 11 Batch 60/162] avg loss 0.00175096, throughput 5.9404K wps
[Epoch 11 Batch 90/162] avg loss 0.00163755, throughput 5.95428K wps
[Epoch 11 Batch 120/162] avg loss 0.00171823, throughput 5.94151K wps
[Epoch 11 Batch 150/162] avg loss 0.00136189, throughput 5.95579K wps
Begin Testing...
[Epoch 11] train avg loss 0.00162668, test acc 0.9344, test avg loss 0.18567, throughput 5.97408K wps
[Epoch 12 Batch 30/162] avg loss 0.00141671, throughput 6.10376K wps
[Epoch 12 Batch 60/162] avg loss 0.00136302, throughput 5.95559K wps
[Epoch 12 Batch 90/162] avg loss 0.00146258, throughput 5.95509K wps
[Epoch 12 Batch 120/162] avg loss 0.00145541, throughput 5.94413K wps
[Epoch 12 Batch 150/162] avg loss 0.00138156, throughput 5.9449K wps
Begin Testing...
[Epoch 12] train avg loss 0.00139138, test acc 0.9344, test avg loss 0.18315, throughput 5.97688K wps
[Epoch 13 Batch 30/162] avg loss 0.00111071, throughput 6.0912K wps
[Epoch 13 Batch 60/162] avg loss 0.0011935, throughput 5.94436K wps
[Epoch 13 Batch 90/162] avg loss 0.00113644, throughput 5.94287K wps
[Epoch 13 Batch 120/162] avg loss 0.00106034, throughput 5.9493K wps
[Epoch 13 Batch 150/162] avg loss 0.00126069, throughput 5.94062K wps
Begin Testing...
[Epoch 13] train avg loss 0.00114786, test acc 0.9356, test avg loss 0.182018, throughput 5.96944K wps
Observed Improvement.
Begin Testing...
[Epoch 14 Batch 30/162] avg loss 0.000946357, throughput 6.08939K wps
[Epoch 14 Batch 60/162] avg loss 0.000966157, throughput 5.93486K wps
[Epoch 14 Batch 90/162] avg loss 0.000961942, throughput 5.93343K wps
[Epoch 14 Batch 120/162] avg loss 0.00123058, throughput 5.93502K wps
[Epoch 14 Batch 150/162] avg loss 0.000853471, throughput 5.93356K wps
Begin Testing...
[Epoch 14] train avg loss 0.00097461, test acc 0.9322, test avg loss 0.183085, throughput 5.96287K wps
[Epoch 15 Batch 30/162] avg loss 0.000833884, throughput 6.09374K wps
[Epoch 15 Batch 60/162] avg loss 0.000878475, throughput 5.95992K wps
[Epoch 15 Batch 90/162] avg loss 0.000684824, throughput 5.94123K wps
[Epoch 15 Batch 120/162] avg loss 0.00070504, throughput 5.95307K wps
[Epoch 15 Batch 150/162] avg loss 0.000865131, throughput 5.9515K wps
Begin Testing...
[Epoch 15] train avg loss 0.000798661, test acc 0.9311, test avg loss 0.183934, throughput 5.9766K wps
[Epoch 16 Batch 30/162] avg loss 0.000634231, throughput 6.08479K wps
[Epoch 16 Batch 60/162] avg loss 0.000674814, throughput 5.95271K wps
[Epoch 16 Batch 90/162] avg loss 0.000733905, throughput 5.95538K wps
[Epoch 16 Batch 120/162] avg loss 0.000773599, throughput 5.94886K wps
[Epoch 16 Batch 150/162] avg loss 0.000594539, throughput 5.95474K wps
Begin Testing...
[Epoch 16] train avg loss 0.000683311, test acc 0.9333, test avg loss 0.184236, throughput 5.97657K wps
[Epoch 17 Batch 30/162] avg loss 0.000517342, throughput 6.09109K wps
[Epoch 17 Batch 60/162] avg loss 0.0005733, throughput 5.94774K wps
[Epoch 17 Batch 90/162] avg loss 0.000635944, throughput 5.94891K wps
[Epoch 17 Batch 120/162] avg loss 0.000491244, throughput 5.96052K wps
[Epoch 17 Batch 150/162] avg loss 0.000550104, throughput 5.95572K wps
Begin Testing...
[Epoch 17] train avg loss 0.000547491, test acc 0.9322, test avg loss 0.185843, throughput 5.97818K wps
[Epoch 18 Batch 30/162] avg loss 0.000458516, throughput 6.10084K wps
[Epoch 18 Batch 60/162] avg loss 0.000481001, throughput 5.94888K wps
[Epoch 18 Batch 90/162] avg loss 0.000431477, throughput 5.96077K wps
[Epoch 18 Batch 120/162] avg loss 0.000515998, throughput 5.96394K wps
[Epoch 18 Batch 150/162] avg loss 0.000439054, throughput 5.95728K wps
Begin Testing...
[Epoch 18] train avg loss 0.00046641, test acc 0.9378, test avg loss 0.18771, throughput 5.98312K wps
Observed Improvement.
Begin Testing...
[Epoch 19 Batch 30/162] avg loss 0.000435312, throughput 6.09458K wps
[Epoch 19 Batch 60/162] avg loss 0.000405147, throughput 5.95482K wps
[Epoch 19 Batch 90/162] avg loss 0.000389022, throughput 5.95743K wps
[Epoch 19 Batch 120/162] avg loss 0.000394379, throughput 5.9444K wps
[Epoch 19 Batch 150/162] avg loss 0.000392269, throughput 5.95408K wps
Begin Testing...
[Epoch 19] train avg loss 0.000406887, test acc 0.9367, test avg loss 0.190217, throughput 5.97854K wps
[Epoch 20 Batch 30/162] avg loss 0.000328864, throughput 6.09846K wps
[Epoch 20 Batch 60/162] avg loss 0.000326709, throughput 5.95112K wps
[Epoch 20 Batch 90/162] avg loss 0.000367562, throughput 5.94903K wps
[Epoch 20 Batch 120/162] avg loss 0.000372362, throughput 5.9422K wps
[Epoch 20 Batch 150/162] avg loss 0.000307062, throughput 5.96736K wps
Begin Testing...
[Epoch 20] train avg loss 0.00033914, test acc 0.9367, test avg loss 0.192626, throughput 5.9787K wps
[Epoch 21 Batch 30/162] avg loss 0.000283602, throughput 6.08905K wps
[Epoch 21 Batch 60/162] avg loss 0.000293632, throughput 5.93836K wps
[Epoch 21 Batch 90/162] avg loss 0.00026648, throughput 5.93961K wps
[Epoch 21 Batch 120/162] avg loss 0.000272926, throughput 5.94184K wps
[Epoch 21 Batch 150/162] avg loss 0.000312499, throughput 5.95388K wps
Begin Testing...
[Epoch 21] train avg loss 0.000285145, test acc 0.9344, test avg loss 0.195977, throughput 5.97046K wps
[Epoch 22 Batch 30/162] avg loss 0.00023661, throughput 6.09178K wps
[Epoch 22 Batch 60/162] avg loss 0.000247274, throughput 5.94659K wps
[Epoch 22 Batch 90/162] avg loss 0.000286298, throughput 5.9342K wps
[Epoch 22 Batch 120/162] avg loss 0.000253297, throughput 5.93525K wps
[Epoch 22 Batch 150/162] avg loss 0.000223026, throughput 5.93087K wps
Begin Testing...
[Epoch 22] train avg loss 0.000246528, test acc 0.9400, test avg loss 0.1993, throughput 5.96512K wps
Observed Improvement.
Begin Testing...
[Epoch 23 Batch 30/162] avg loss 0.000205728, throughput 6.09966K wps
[Epoch 23 Batch 60/162] avg loss 0.000227146, throughput 5.95292K wps
[Epoch 23 Batch 90/162] avg loss 0.000195874, throughput 5.94592K wps
[Epoch 23 Batch 120/162] avg loss 0.000211419, throughput 5.93366K wps
[Epoch 23 Batch 150/162] avg loss 0.000182124, throughput 5.92699K wps
Begin Testing...
[Epoch 23] train avg loss 0.000204996, test acc 0.9400, test avg loss 0.203008, throughput 5.96849K wps
Observed Improvement.
Begin Testing...
[Epoch 24 Batch 30/162] avg loss 0.00017804, throughput 6.08304K wps
[Epoch 24 Batch 60/162] avg loss 0.000220781, throughput 5.9327K wps
[Epoch 24 Batch 90/162] avg loss 0.000164801, throughput 5.9306K wps
[Epoch 24 Batch 120/162] avg loss 0.000171682, throughput 5.94731K wps
[Epoch 24 Batch 150/162] avg loss 0.000187217, throughput 5.95633K wps
Begin Testing...
[Epoch 24] train avg loss 0.000184909, test acc 0.9378, test avg loss 0.207998, throughput 5.96732K wps
[Epoch 25 Batch 30/162] avg loss 0.000138961, throughput 6.08625K wps
[Epoch 25 Batch 60/162] avg loss 0.000163315, throughput 5.95304K wps
[Epoch 25 Batch 90/162] avg loss 0.000151644, throughput 5.9472K wps
[Epoch 25 Batch 120/162] avg loss 0.000161385, throughput 5.9399K wps
[Epoch 25 Batch 150/162] avg loss 0.000141066, throughput 5.94305K wps
Begin Testing...
[Epoch 25] train avg loss 0.000151272, test acc 0.9378, test avg loss 0.210866, throughput 5.97204K wps
[Epoch 26 Batch 30/162] avg loss 0.000140895, throughput 6.08861K wps
[Epoch 26 Batch 60/162] avg loss 0.000119645, throughput 5.95107K wps
[Epoch 26 Batch 90/162] avg loss 0.000120264, throughput 5.93925K wps
[Epoch 26 Batch 120/162] avg loss 0.000106743, throughput 5.93324K wps
[Epoch 26 Batch 150/162] avg loss 0.000148726, throughput 5.93693K wps
Begin Testing...
[Epoch 26] train avg loss 0.000126347, test acc 0.9356, test avg loss 0.212426, throughput 5.96787K wps
[Epoch 27 Batch 30/162] avg loss 9.94179e-05, throughput 6.08892K wps
[Epoch 27 Batch 60/162] avg loss 0.000126853, throughput 5.94542K wps
[Epoch 27 Batch 90/162] avg loss 0.000123419, throughput 5.94964K wps
[Epoch 27 Batch 120/162] avg loss 0.000124355, throughput 5.95709K wps
[Epoch 27 Batch 150/162] avg loss 0.000100594, throughput 5.94931K wps
Begin Testing...
[Epoch 27] train avg loss 0.000115734, test acc 0.9356, test avg loss 0.216025, throughput 5.97533K wps
[Epoch 28 Batch 30/162] avg loss 7.842e-05, throughput 6.081K wps
[Epoch 28 Batch 60/162] avg loss 0.000105965, throughput 5.92906K wps
[Epoch 28 Batch 90/162] avg loss 9.21683e-05, throughput 5.93506K wps
[Epoch 28 Batch 120/162] avg loss 0.000108168, throughput 5.94462K wps
[Epoch 28 Batch 150/162] avg loss 0.000104958, throughput 5.95586K wps
Begin Testing...
[Epoch 28] train avg loss 9.90236e-05, test acc 0.9367, test avg loss 0.218837, throughput 5.96742K wps
[Epoch 29 Batch 30/162] avg loss 7.95311e-05, throughput 6.0987K wps
[Epoch 29 Batch 60/162] avg loss 9.41486e-05, throughput 5.9533K wps
[Epoch 29 Batch 90/162] avg loss 9.063e-05, throughput 5.93939K wps
[Epoch 29 Batch 120/162] avg loss 9.42854e-05, throughput 5.9378K wps
[Epoch 29 Batch 150/162] avg loss 7.9319e-05, throughput 5.93746K wps
Begin Testing...
[Epoch 29] train avg loss 8.9541e-05, test acc 0.9378, test avg loss 0.221873, throughput 5.97044K wps
[Epoch 30 Batch 30/162] avg loss 8.84279e-05, throughput 6.10295K wps
[Epoch 30 Batch 60/162] avg loss 8.74364e-05, throughput 5.94743K wps
[Epoch 30 Batch 90/162] avg loss 7.74372e-05, throughput 5.95157K wps
[Epoch 30 Batch 120/162] avg loss 7.79437e-05, throughput 5.95482K wps
[Epoch 30 Batch 150/162] avg loss 7.27083e-05, throughput 5.94332K wps
Begin Testing...
[Epoch 30] train avg loss 8.01799e-05, test acc 0.9378, test avg loss 0.223028, throughput 5.97726K wps
[Epoch 31 Batch 30/162] avg loss 7.09236e-05, throughput 6.08622K wps
[Epoch 31 Batch 60/162] avg loss 7.24164e-05, throughput 5.9441K wps
[Epoch 31 Batch 90/162] avg loss 7.74315e-05, throughput 5.94313K wps
[Epoch 31 Batch 120/162] avg loss 6.19815e-05, throughput 5.95152K wps
[Epoch 31 Batch 150/162] avg loss 6.41087e-05, throughput 5.94865K wps
Begin Testing...
[Epoch 31] train avg loss 6.78793e-05, test acc 0.9356, test avg loss 0.22773, throughput 5.97335K wps
[Epoch 32 Batch 30/162] avg loss 6.55362e-05, throughput 6.09416K wps
[Epoch 32 Batch 60/162] avg loss 5.94451e-05, throughput 5.95766K wps
[Epoch 32 Batch 90/162] avg loss 7.11297e-05, throughput 5.96471K wps
[Epoch 32 Batch 120/162] avg loss 6.44161e-05, throughput 5.9574K wps
[Epoch 32 Batch 150/162] avg loss 6.28045e-05, throughput 5.9498K wps
Begin Testing...
[Epoch 32] train avg loss 6.46861e-05, test acc 0.9356, test avg loss 0.232587, throughput 5.98255K wps
[Epoch 33 Batch 30/162] avg loss 4.60645e-05, throughput 6.09573K wps
[Epoch 33 Batch 60/162] avg loss 5.20811e-05, throughput 5.93441K wps
[Epoch 33 Batch 90/162] avg loss 5.18573e-05, throughput 5.94772K wps
[Epoch 33 Batch 120/162] avg loss 4.55941e-05, throughput 5.94643K wps
[Epoch 33 Batch 150/162] avg loss 5.3705e-05, throughput 5.9578K wps
Begin Testing...
[Epoch 33] train avg loss 4.97405e-05, test acc 0.9367, test avg loss 0.237047, throughput 5.9737K wps
[Epoch 34 Batch 30/162] avg loss 5.9292e-05, throughput 6.09385K wps
[Epoch 34 Batch 60/162] avg loss 3.96023e-05, throughput 5.957K wps
[Epoch 34 Batch 90/162] avg loss 4.84593e-05, throughput 5.94936K wps
[Epoch 34 Batch 120/162] avg loss 4.28541e-05, throughput 5.96443K wps
[Epoch 34 Batch 150/162] avg loss 4.01337e-05, throughput 5.94359K wps
Begin Testing...
[Epoch 34] train avg loss 4.55592e-05, test acc 0.9311, test avg loss 0.240122, throughput 5.97889K wps
[Epoch 35 Batch 30/162] avg loss 3.85775e-05, throughput 6.09412K wps
[Epoch 35 Batch 60/162] avg loss 4.28456e-05, throughput 5.93982K wps
[Epoch 35 Batch 90/162] avg loss 4.93602e-05, throughput 5.95269K wps
[Epoch 35 Batch 120/162] avg loss 4.61061e-05, throughput 5.94869K wps
[Epoch 35 Batch 150/162] avg loss 4.00936e-05, throughput 5.93619K wps
Begin Testing...
[Epoch 35] train avg loss 4.26534e-05, test acc 0.9311, test avg loss 0.244098, throughput 5.97231K wps
[Epoch 36 Batch 30/162] avg loss 4.38773e-05, throughput 6.09634K wps
[Epoch 36 Batch 60/162] avg loss 2.8583e-05, throughput 5.94437K wps
[Epoch 36 Batch 90/162] avg loss 3.469e-05, throughput 5.95701K wps
[Epoch 36 Batch 120/162] avg loss 3.35531e-05, throughput 5.94555K wps
[Epoch 36 Batch 150/162] avg loss 5.14774e-05, throughput 5.93129K wps
Begin Testing...
[Epoch 36] train avg loss 3.78021e-05, test acc 0.9333, test avg loss 0.245238, throughput 5.97162K wps
[Epoch 37 Batch 30/162] avg loss 3.2323e-05, throughput 6.10074K wps
[Epoch 37 Batch 60/162] avg loss 4.27026e-05, throughput 5.96028K wps
[Epoch 37 Batch 90/162] avg loss 3.58514e-05, throughput 5.95878K wps
[Epoch 37 Batch 120/162] avg loss 3.90022e-05, throughput 5.9576K wps
[Epoch 37 Batch 150/162] avg loss 3.60293e-05, throughput 5.94081K wps
Begin Testing...
[Epoch 37] train avg loss 3.64397e-05, test acc 0.9333, test avg loss 0.250174, throughput 5.9811K wps
[Epoch 38 Batch 30/162] avg loss 3.68916e-05, throughput 6.10118K wps
[Epoch 38 Batch 60/162] avg loss 3.12745e-05, throughput 5.9599K wps
[Epoch 38 Batch 90/162] avg loss 2.74474e-05, throughput 5.95165K wps
[Epoch 38 Batch 120/162] avg loss 2.62877e-05, throughput 5.93613K wps
[Epoch 38 Batch 150/162] avg loss 2.7262e-05, throughput 5.9236K wps
Begin Testing...
[Epoch 38] train avg loss 2.95647e-05, test acc 0.9311, test avg loss 0.251124, throughput 5.97116K wps
[Epoch 39 Batch 30/162] avg loss 2.15107e-05, throughput 6.10481K wps
[Epoch 39 Batch 60/162] avg loss 2.87334e-05, throughput 5.947K wps
[Epoch 39 Batch 90/162] avg loss 2.24303e-05, throughput 5.94364K wps
[Epoch 39 Batch 120/162] avg loss 2.52013e-05, throughput 5.96473K wps
[Epoch 39 Batch 150/162] avg loss 2.97298e-05, throughput 5.96219K wps
Begin Testing...
[Epoch 39] train avg loss 2.51022e-05, test acc 0.9322, test avg loss 0.252944, throughput 5.98217K wps
[Epoch 40 Batch 30/162] avg loss 2.49034e-05, throughput 6.11021K wps
[Epoch 40 Batch 60/162] avg loss 2.52469e-05, throughput 5.95541K wps
[Epoch 40 Batch 90/162] avg loss 2.26088e-05, throughput 5.95059K wps
[Epoch 40 Batch 120/162] avg loss 2.18767e-05, throughput 5.94391K wps
[Epoch 40 Batch 150/162] avg loss 2.70267e-05, throughput 5.94418K wps
Begin Testing...
[Epoch 40] train avg loss 2.41456e-05, test acc 0.9322, test avg loss 0.256621, throughput 5.97648K wps
[Epoch 41 Batch 30/162] avg loss 2.09145e-05, throughput 6.10918K wps
[Epoch 41 Batch 60/162] avg loss 2.63157e-05, throughput 5.95497K wps
[Epoch 41 Batch 90/162] avg loss 1.99244e-05, throughput 5.95269K wps
[Epoch 41 Batch 120/162] avg loss 2.02384e-05, throughput 5.93266K wps
[Epoch 41 Batch 150/162] avg loss 2.24512e-05, throughput 5.9489K wps
Begin Testing...
[Epoch 41] train avg loss 2.15101e-05, test acc 0.9333, test avg loss 0.260317, throughput 5.97637K wps
[Epoch 42 Batch 30/162] avg loss 1.64286e-05, throughput 6.09774K wps
[Epoch 42 Batch 60/162] avg loss 1.79169e-05, throughput 5.94173K wps
[Epoch 42 Batch 90/162] avg loss 1.78394e-05, throughput 5.95712K wps
[Epoch 42 Batch 120/162] avg loss 2.02686e-05, throughput 5.95039K wps
[Epoch 42 Batch 150/162] avg loss 1.58959e-05, throughput 5.9474K wps
Begin Testing...
[Epoch 42] train avg loss 1.78083e-05, test acc 0.9356, test avg loss 0.264406, throughput 5.97552K wps
[Epoch 43 Batch 30/162] avg loss 1.89607e-05, throughput 6.10183K wps
[Epoch 43 Batch 60/162] avg loss 1.56694e-05, throughput 5.94835K wps
[Epoch 43 Batch 90/162] avg loss 2.03303e-05, throughput 5.9518K wps
[Epoch 43 Batch 120/162] avg loss 1.78624e-05, throughput 5.94248K wps
[Epoch 43 Batch 150/162] avg loss 1.71721e-05, throughput 5.94044K wps
Begin Testing...
[Epoch 43] train avg loss 1.75216e-05, test acc 0.9356, test avg loss 0.268137, throughput 5.97347K wps
[Epoch 44 Batch 30/162] avg loss 1.26283e-05, throughput 6.08622K wps
[Epoch 44 Batch 60/162] avg loss 1.2579e-05, throughput 5.9552K wps
[Epoch 44 Batch 90/162] avg loss 1.59009e-05, throughput 5.95371K wps
[Epoch 44 Batch 120/162] avg loss 1.27192e-05, throughput 5.94951K wps
[Epoch 44 Batch 150/162] avg loss 1.50972e-05, throughput 5.95064K wps
Begin Testing...
[Epoch 44] train avg loss 1.33735e-05, test acc 0.9344, test avg loss 0.270773, throughput 5.97602K wps
[Epoch 45 Batch 30/162] avg loss 1.27873e-05, throughput 6.09852K wps
[Epoch 45 Batch 60/162] avg loss 1.16234e-05, throughput 5.9598K wps
[Epoch 45 Batch 90/162] avg loss 1.76196e-05, throughput 5.96065K wps
[Epoch 45 Batch 120/162] avg loss 1.51716e-05, throughput 5.94598K wps
[Epoch 45 Batch 150/162] avg loss 1.20172e-05, throughput 5.95702K wps
Begin Testing...
[Epoch 45] train avg loss 1.37856e-05, test acc 0.9333, test avg loss 0.272793, throughput 5.98055K wps
[Epoch 46 Batch 30/162] avg loss 1.1689e-05, throughput 6.09909K wps
[Epoch 46 Batch 60/162] avg loss 1.00888e-05, throughput 5.9516K wps
[Epoch 46 Batch 90/162] avg loss 1.39427e-05, throughput 5.95548K wps
[Epoch 46 Batch 120/162] avg loss 1.03938e-05, throughput 5.94945K wps
[Epoch 46 Batch 150/162] avg loss 1.10584e-05, throughput 5.95048K wps
Begin Testing...
[Epoch 46] train avg loss 1.13436e-05, test acc 0.9322, test avg loss 0.275588, throughput 5.97812K wps
[Epoch 47 Batch 30/162] avg loss 1.66354e-05, throughput 6.08712K wps
[Epoch 47 Batch 60/162] avg loss 9.62755e-06, throughput 5.95251K wps
[Epoch 47 Batch 90/162] avg loss 1.59549e-05, throughput 5.95411K wps
[Epoch 47 Batch 120/162] avg loss 8.85996e-06, throughput 5.94861K wps
[Epoch 47 Batch 150/162] avg loss 1.05766e-05, throughput 5.95898K wps
Begin Testing...
[Epoch 47] train avg loss 1.32137e-05, test acc 0.9333, test avg loss 0.281097, throughput 5.97662K wps
[Epoch 48 Batch 30/162] avg loss 1.10028e-05, throughput 6.09582K wps
[Epoch 48 Batch 60/162] avg loss 9.80811e-06, throughput 5.94551K wps
[Epoch 48 Batch 90/162] avg loss 7.9741e-06, throughput 5.93814K wps
[Epoch 48 Batch 120/162] avg loss 8.3568e-06, throughput 5.95347K wps
[Epoch 48 Batch 150/162] avg loss 1.06774e-05, throughput 5.95015K wps
Begin Testing...
[Epoch 48] train avg loss 9.40863e-06, test acc 0.9311, test avg loss 0.284928, throughput 5.97261K wps
[Epoch 49 Batch 30/162] avg loss 1.26068e-05, throughput 6.10416K wps
[Epoch 49 Batch 60/162] avg loss 1.06123e-05, throughput 5.9657K wps
[Epoch 49 Batch 90/162] avg loss 1.05695e-05, throughput 5.94004K wps
[Epoch 49 Batch 120/162] avg loss 9.15067e-06, throughput 5.948K wps
[Epoch 49 Batch 150/162] avg loss 9.46966e-06, throughput 5.95709K wps
Begin Testing...
[Epoch 49] train avg loss 1.01748e-05, test acc 0.9311, test avg loss 0.289116, throughput 5.97933K wps
[Epoch 50 Batch 30/162] avg loss 7.6634e-06, throughput 6.09407K wps
[Epoch 50 Batch 60/162] avg loss 1.00307e-05, throughput 5.94656K wps
[Epoch 50 Batch 90/162] avg loss 7.77368e-06, throughput 5.94764K wps
[Epoch 50 Batch 120/162] avg loss 7.29663e-06, throughput 5.95309K wps
[Epoch 50 Batch 150/162] avg loss 1.8332e-05, throughput 5.94238K wps
Begin Testing...
[Epoch 50] train avg loss 1.01136e-05, test acc 0.9333, test avg loss 0.292321, throughput 5.97364K wps
[Epoch 51 Batch 30/162] avg loss 7.6909e-06, throughput 6.10329K wps
[Epoch 51 Batch 60/162] avg loss 6.40239e-06, throughput 5.95013K wps
[Epoch 51 Batch 90/162] avg loss 6.90354e-06, throughput 5.95497K wps
[Epoch 51 Batch 120/162] avg loss 8.79632e-06, throughput 5.95215K wps
[Epoch 51 Batch 150/162] avg loss 5.33294e-06, throughput 5.93841K wps
Begin Testing...
[Epoch 51] train avg loss 6.97876e-06, test acc 0.9333, test avg loss 0.295321, throughput 5.9768K wps
[Epoch 52 Batch 30/162] avg loss 7.47571e-06, throughput 6.09619K wps
[Epoch 52 Batch 60/162] avg loss 7.23054e-06, throughput 5.96629K wps
[Epoch 52 Batch 90/162] avg loss 5.89909e-06, throughput 5.94716K wps
[Epoch 52 Batch 120/162] avg loss 5.70724e-06, throughput 5.94876K wps
[Epoch 52 Batch 150/162] avg loss 6.4296e-06, throughput 5.94631K wps
Begin Testing...
[Epoch 52] train avg loss 6.58961e-06, test acc 0.9311, test avg loss 0.299903, throughput 5.97819K wps
[Epoch 53 Batch 30/162] avg loss 5.64293e-06, throughput 6.10326K wps
[Epoch 53 Batch 60/162] avg loss 5.29588e-06, throughput 5.96035K wps
[Epoch 53 Batch 90/162] avg loss 5.79045e-06, throughput 5.9352K wps
[Epoch 53 Batch 120/162] avg loss 6.36299e-06, throughput 5.94155K wps
[Epoch 53 Batch 150/162] avg loss 4.84149e-06, throughput 5.94504K wps
Begin Testing...
[Epoch 53] train avg loss 5.61698e-06, test acc 0.9311, test avg loss 0.302246, throughput 5.97548K wps
[Epoch 54 Batch 30/162] avg loss 4.63668e-06, throughput 6.09491K wps
[Epoch 54 Batch 60/162] avg loss 4.95234e-06, throughput 5.94542K wps
[Epoch 54 Batch 90/162] avg loss 6.35915e-06, throughput 5.93651K wps
[Epoch 54 Batch 120/162] avg loss 1.02259e-05, throughput 5.93306K wps
[Epoch 54 Batch 150/162] avg loss 5.3952e-06, throughput 5.94263K wps
Begin Testing...
[Epoch 54] train avg loss 6.20009e-06, test acc 0.9300, test avg loss 0.307187, throughput 5.96837K wps
[Epoch 55 Batch 30/162] avg loss 5.27031e-06, throughput 6.08377K wps
[Epoch 55 Batch 60/162] avg loss 4.91854e-06, throughput 5.93627K wps
[Epoch 55 Batch 90/162] avg loss 3.95844e-06, throughput 5.93925K wps
[Epoch 55 Batch 120/162] avg loss 5.57823e-06, throughput 5.95809K wps
[Epoch 55 Batch 150/162] avg loss 6.68843e-06, throughput 5.95252K wps