Permalink
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
6423 lines (6422 sloc) 416 KB
Namespace(batch_size=50, data_name='SST-1', dropout=0.5, epochs=40, gpu=0, log_interval=30, lr=0.0001, model_mode='multichannel', save_prefix='sa-model')
Use gpu0
1704
53
Done! Tokenizing Time=9.49s, #Sentences=237107
Done! Tokenizing Time=0.85s, #Sentences=2125
SentimentNet(
(embedding): Embedding(18077 -> 300, float32)
(embedding_extend): Embedding(18077 -> 300, float32)
(encoder): ConvolutionalEncoder(
(_convs): HybridConcurrent(
(0): HybridSequential(
(0): Conv1D(600 -> 100, kernel_size=(3,), stride=(1,))
(1): Activation(relu)
(2): HybridLambda(<lambda>)
)
(1): HybridSequential(
(0): Conv1D(600 -> 100, kernel_size=(4,), stride=(1,))
(1): Activation(relu)
(2): HybridLambda(<lambda>)
)
(2): HybridSequential(
(0): Conv1D(600 -> 100, kernel_size=(5,), stride=(1,))
(1): Activation(relu)
(2): HybridLambda(<lambda>)
)
)
)
(output): HybridSequential(
(0): Dropout(p = 0.5, axes=())
(1): Dense(None -> 5, linear)
)
)
[Epoch 0 Batch 30/4268] avg loss 0.0319093, throughput 2.58143K wps
[Epoch 0 Batch 60/4268] avg loss 0.0301246, throughput 4.00468K wps
[Epoch 0 Batch 90/4268] avg loss 0.0288076, throughput 4.00625K wps
[Epoch 0 Batch 120/4268] avg loss 0.0286332, throughput 4.00806K wps
[Epoch 0 Batch 150/4268] avg loss 0.0276482, throughput 4.01748K wps
[Epoch 0 Batch 180/4268] avg loss 0.0277437, throughput 4.01059K wps
[Epoch 0 Batch 210/4268] avg loss 0.0269818, throughput 4.00774K wps
[Epoch 0 Batch 240/4268] avg loss 0.0264934, throughput 4.00798K wps
[Epoch 0 Batch 270/4268] avg loss 0.0265925, throughput 4.0044K wps
[Epoch 0 Batch 300/4268] avg loss 0.0261564, throughput 4.00405K wps
[Epoch 0 Batch 330/4268] avg loss 0.0254922, throughput 4.01124K wps
[Epoch 0 Batch 360/4268] avg loss 0.0252011, throughput 4.01023K wps
[Epoch 0 Batch 390/4268] avg loss 0.0251762, throughput 4.00029K wps
[Epoch 0 Batch 420/4268] avg loss 0.0252237, throughput 4.0071K wps
[Epoch 0 Batch 450/4268] avg loss 0.0261139, throughput 4.00248K wps
[Epoch 0 Batch 480/4268] avg loss 0.0249588, throughput 4.00653K wps
[Epoch 0 Batch 510/4268] avg loss 0.0247631, throughput 4.01155K wps
[Epoch 0 Batch 540/4268] avg loss 0.0246218, throughput 4.00543K wps
[Epoch 0 Batch 570/4268] avg loss 0.0241072, throughput 4.00953K wps
[Epoch 0 Batch 600/4268] avg loss 0.0245117, throughput 4.00893K wps
[Epoch 0 Batch 630/4268] avg loss 0.0242917, throughput 4.00392K wps
[Epoch 0 Batch 660/4268] avg loss 0.0243415, throughput 4.0045K wps
[Epoch 0 Batch 690/4268] avg loss 0.0245975, throughput 4.00303K wps
[Epoch 0 Batch 720/4268] avg loss 0.0236891, throughput 4.01146K wps
[Epoch 0 Batch 750/4268] avg loss 0.0234121, throughput 4.01748K wps
[Epoch 0 Batch 780/4268] avg loss 0.0233141, throughput 4.01286K wps
[Epoch 0 Batch 810/4268] avg loss 0.0231562, throughput 4.00419K wps
[Epoch 0 Batch 840/4268] avg loss 0.0235688, throughput 4.00476K wps
[Epoch 0 Batch 870/4268] avg loss 0.0229115, throughput 4.0065K wps
[Epoch 0 Batch 900/4268] avg loss 0.023924, throughput 4.00525K wps
[Epoch 0 Batch 930/4268] avg loss 0.0237515, throughput 4.00494K wps
[Epoch 0 Batch 960/4268] avg loss 0.0232803, throughput 4.00493K wps
[Epoch 0 Batch 990/4268] avg loss 0.0232432, throughput 4.00608K wps
[Epoch 0 Batch 1020/4268] avg loss 0.023187, throughput 4.00213K wps
[Epoch 0 Batch 1050/4268] avg loss 0.0228561, throughput 4.00209K wps
[Epoch 0 Batch 1080/4268] avg loss 0.0222405, throughput 4.00186K wps
[Epoch 0 Batch 1110/4268] avg loss 0.0226303, throughput 3.99981K wps
[Epoch 0 Batch 1140/4268] avg loss 0.0224608, throughput 4.00206K wps
[Epoch 0 Batch 1170/4268] avg loss 0.0220748, throughput 4.00065K wps
[Epoch 0 Batch 1200/4268] avg loss 0.0217241, throughput 4.00076K wps
[Epoch 0 Batch 1230/4268] avg loss 0.0221922, throughput 3.99892K wps
[Epoch 0 Batch 1260/4268] avg loss 0.0213874, throughput 3.99863K wps
[Epoch 0 Batch 1290/4268] avg loss 0.0227785, throughput 3.99759K wps
[Epoch 0 Batch 1320/4268] avg loss 0.0219528, throughput 3.99839K wps
[Epoch 0 Batch 1350/4268] avg loss 0.0214891, throughput 3.99448K wps
[Epoch 0 Batch 1380/4268] avg loss 0.0223048, throughput 3.99423K wps
[Epoch 0 Batch 1410/4268] avg loss 0.0214066, throughput 3.9969K wps
[Epoch 0 Batch 1440/4268] avg loss 0.0221984, throughput 3.99484K wps
[Epoch 0 Batch 1470/4268] avg loss 0.0210407, throughput 3.99397K wps
[Epoch 0 Batch 1500/4268] avg loss 0.0213873, throughput 3.99638K wps
[Epoch 0 Batch 1530/4268] avg loss 0.0213937, throughput 3.99601K wps
[Epoch 0 Batch 1560/4268] avg loss 0.0215415, throughput 3.99567K wps
[Epoch 0 Batch 1590/4268] avg loss 0.0212881, throughput 3.99414K wps
[Epoch 0 Batch 1620/4268] avg loss 0.0209996, throughput 3.99806K wps
[Epoch 0 Batch 1650/4268] avg loss 0.0210397, throughput 3.99998K wps
[Epoch 0 Batch 1680/4268] avg loss 0.020978, throughput 3.99131K wps
[Epoch 0 Batch 1710/4268] avg loss 0.020368, throughput 3.9946K wps
[Epoch 0 Batch 1740/4268] avg loss 0.021045, throughput 3.99763K wps
[Epoch 0 Batch 1770/4268] avg loss 0.0205233, throughput 3.99812K wps
[Epoch 0 Batch 1800/4268] avg loss 0.0212591, throughput 3.99826K wps
[Epoch 0 Batch 1830/4268] avg loss 0.021272, throughput 3.99822K wps
[Epoch 0 Batch 1860/4268] avg loss 0.0206846, throughput 4.00167K wps
[Epoch 0 Batch 1890/4268] avg loss 0.0208212, throughput 3.99565K wps
[Epoch 0 Batch 1920/4268] avg loss 0.0204632, throughput 3.99786K wps
[Epoch 0 Batch 1950/4268] avg loss 0.0214006, throughput 3.99613K wps
[Epoch 0 Batch 1980/4268] avg loss 0.0199903, throughput 3.998K wps
[Epoch 0 Batch 2010/4268] avg loss 0.0203639, throughput 3.99656K wps
[Epoch 0 Batch 2040/4268] avg loss 0.0202343, throughput 3.99723K wps
[Epoch 0 Batch 2070/4268] avg loss 0.0196967, throughput 3.99752K wps
[Epoch 0 Batch 2100/4268] avg loss 0.0200091, throughput 4.00224K wps
[Epoch 0 Batch 2130/4268] avg loss 0.0204447, throughput 4.00119K wps
[Epoch 0 Batch 2160/4268] avg loss 0.0204016, throughput 4.00335K wps
[Epoch 0 Batch 2190/4268] avg loss 0.0205389, throughput 4.00061K wps
[Epoch 0 Batch 2220/4268] avg loss 0.0198988, throughput 3.99335K wps
[Epoch 0 Batch 2250/4268] avg loss 0.0205971, throughput 3.9932K wps
[Epoch 0 Batch 2280/4268] avg loss 0.0196765, throughput 3.99687K wps
[Epoch 0 Batch 2310/4268] avg loss 0.0199075, throughput 3.99562K wps
[Epoch 0 Batch 2340/4268] avg loss 0.01948, throughput 3.99631K wps
[Epoch 0 Batch 2370/4268] avg loss 0.0190003, throughput 3.99755K wps
[Epoch 0 Batch 2400/4268] avg loss 0.0194136, throughput 3.99508K wps
[Epoch 0 Batch 2430/4268] avg loss 0.0196197, throughput 3.99729K wps
[Epoch 0 Batch 2460/4268] avg loss 0.0197033, throughput 3.99909K wps
[Epoch 0 Batch 2490/4268] avg loss 0.0196837, throughput 3.99792K wps
[Epoch 0 Batch 2520/4268] avg loss 0.0194788, throughput 3.99265K wps
[Epoch 0 Batch 2550/4268] avg loss 0.0199946, throughput 3.9932K wps
[Epoch 0 Batch 2580/4268] avg loss 0.0198282, throughput 3.98782K wps
[Epoch 0 Batch 2610/4268] avg loss 0.0200358, throughput 3.99576K wps
[Epoch 0 Batch 2640/4268] avg loss 0.0192022, throughput 3.99328K wps
[Epoch 0 Batch 2670/4268] avg loss 0.0188367, throughput 3.99815K wps
[Epoch 0 Batch 2700/4268] avg loss 0.0191476, throughput 3.99499K wps
[Epoch 0 Batch 2730/4268] avg loss 0.0191252, throughput 3.99512K wps
[Epoch 0 Batch 2760/4268] avg loss 0.0199275, throughput 3.99956K wps
[Epoch 0 Batch 2790/4268] avg loss 0.0194863, throughput 3.99923K wps
[Epoch 0 Batch 2820/4268] avg loss 0.0187689, throughput 4.00085K wps
[Epoch 0 Batch 2850/4268] avg loss 0.0189598, throughput 3.9997K wps
[Epoch 0 Batch 2880/4268] avg loss 0.0191961, throughput 3.99381K wps
[Epoch 0 Batch 2910/4268] avg loss 0.0188584, throughput 3.99984K wps
[Epoch 0 Batch 2940/4268] avg loss 0.019082, throughput 3.99754K wps
[Epoch 0 Batch 2970/4268] avg loss 0.0189097, throughput 3.99592K wps
[Epoch 0 Batch 3000/4268] avg loss 0.0194218, throughput 3.9955K wps
[Epoch 0 Batch 3030/4268] avg loss 0.0189257, throughput 3.99168K wps
[Epoch 0 Batch 3060/4268] avg loss 0.0182218, throughput 3.99395K wps
[Epoch 0 Batch 3090/4268] avg loss 0.0189154, throughput 3.99735K wps
[Epoch 0 Batch 3120/4268] avg loss 0.0191777, throughput 3.99764K wps
[Epoch 0 Batch 3150/4268] avg loss 0.0188215, throughput 3.99612K wps
[Epoch 0 Batch 3180/4268] avg loss 0.0187971, throughput 3.99633K wps
[Epoch 0 Batch 3210/4268] avg loss 0.0191461, throughput 4.00091K wps
[Epoch 0 Batch 3240/4268] avg loss 0.0178672, throughput 3.99633K wps
[Epoch 0 Batch 3270/4268] avg loss 0.0188441, throughput 3.99207K wps
[Epoch 0 Batch 3300/4268] avg loss 0.018954, throughput 3.9952K wps
[Epoch 0 Batch 3330/4268] avg loss 0.0189964, throughput 3.99493K wps
[Epoch 0 Batch 3360/4268] avg loss 0.0190383, throughput 3.99315K wps
[Epoch 0 Batch 3390/4268] avg loss 0.01835, throughput 3.99306K wps
[Epoch 0 Batch 3420/4268] avg loss 0.018571, throughput 3.99025K wps
[Epoch 0 Batch 3450/4268] avg loss 0.0183478, throughput 3.99657K wps
[Epoch 0 Batch 3480/4268] avg loss 0.0182331, throughput 3.98556K wps
[Epoch 0 Batch 3510/4268] avg loss 0.0190004, throughput 3.98943K wps
[Epoch 0 Batch 3540/4268] avg loss 0.0191645, throughput 3.99803K wps
[Epoch 0 Batch 3570/4268] avg loss 0.018398, throughput 3.99321K wps
[Epoch 0 Batch 3600/4268] avg loss 0.0187314, throughput 3.99328K wps
[Epoch 0 Batch 3630/4268] avg loss 0.0194213, throughput 3.99226K wps
[Epoch 0 Batch 3660/4268] avg loss 0.0183958, throughput 3.99831K wps
[Epoch 0 Batch 3690/4268] avg loss 0.018822, throughput 3.99294K wps
[Epoch 0 Batch 3720/4268] avg loss 0.0189085, throughput 3.99404K wps
[Epoch 0 Batch 3750/4268] avg loss 0.0182285, throughput 3.99603K wps
[Epoch 0 Batch 3780/4268] avg loss 0.0188626, throughput 3.99454K wps
[Epoch 0 Batch 3810/4268] avg loss 0.0186517, throughput 3.99452K wps
[Epoch 0 Batch 3840/4268] avg loss 0.0190066, throughput 3.99622K wps
[Epoch 0 Batch 3870/4268] avg loss 0.0182641, throughput 3.99595K wps
[Epoch 0 Batch 3900/4268] avg loss 0.0185768, throughput 3.99711K wps
[Epoch 0 Batch 3930/4268] avg loss 0.0189368, throughput 3.9935K wps
[Epoch 0 Batch 3960/4268] avg loss 0.0186714, throughput 3.99397K wps
[Epoch 0 Batch 3990/4268] avg loss 0.0192578, throughput 3.99359K wps
[Epoch 0 Batch 4020/4268] avg loss 0.0178813, throughput 3.99439K wps
[Epoch 0 Batch 4050/4268] avg loss 0.0189348, throughput 3.99627K wps
[Epoch 0 Batch 4080/4268] avg loss 0.0184024, throughput 4.00238K wps
[Epoch 0 Batch 4110/4268] avg loss 0.0175316, throughput 4.00421K wps
[Epoch 0 Batch 4140/4268] avg loss 0.0190466, throughput 4.00209K wps
[Epoch 0 Batch 4170/4268] avg loss 0.0178051, throughput 4.00162K wps
[Epoch 0 Batch 4200/4268] avg loss 0.0186432, throughput 3.99201K wps
[Epoch 0 Batch 4230/4268] avg loss 0.0186447, throughput 3.98993K wps
[Epoch 0 Batch 4260/4268] avg loss 0.0185279, throughput 3.99155K wps
Begin Testing...
[Batch 30/475] elapsed 0.46 s
[Batch 60/475] elapsed 0.43 s
[Batch 90/475] elapsed 0.43 s
[Batch 120/475] elapsed 0.43 s
[Batch 150/475] elapsed 0.43 s
[Batch 180/475] elapsed 0.43 s
[Batch 210/475] elapsed 0.43 s
[Batch 240/475] elapsed 0.43 s
[Batch 270/475] elapsed 0.43 s
[Batch 300/475] elapsed 0.43 s
[Batch 330/475] elapsed 0.43 s
[Batch 360/475] elapsed 0.43 s
[Batch 390/475] elapsed 0.43 s
[Batch 420/475] elapsed 0.43 s
[Batch 450/475] elapsed 0.43 s
[Epoch 0] train avg loss 0.0211216, test acc 0.6346, test avg loss 0.877471, throughput 3.97351K wps
Observed Improvement.
Begin Testing...
[Batch 30/43] elapsed 0.43 s
[Epoch 1 Batch 30/4268] avg loss 0.0180353, throughput 4.07794K wps
[Epoch 1 Batch 60/4268] avg loss 0.0172721, throughput 3.99526K wps
[Epoch 1 Batch 90/4268] avg loss 0.0176705, throughput 3.996K wps
[Epoch 1 Batch 120/4268] avg loss 0.0183346, throughput 3.99364K wps
[Epoch 1 Batch 150/4268] avg loss 0.0165192, throughput 3.99493K wps
[Epoch 1 Batch 180/4268] avg loss 0.0175136, throughput 3.99658K wps
[Epoch 1 Batch 210/4268] avg loss 0.0167714, throughput 3.99555K wps
[Epoch 1 Batch 240/4268] avg loss 0.0170326, throughput 3.9922K wps
[Epoch 1 Batch 270/4268] avg loss 0.0178138, throughput 3.99582K wps
[Epoch 1 Batch 300/4268] avg loss 0.0179069, throughput 3.99604K wps
[Epoch 1 Batch 330/4268] avg loss 0.0184897, throughput 3.99415K wps
[Epoch 1 Batch 360/4268] avg loss 0.0172815, throughput 3.99658K wps
[Epoch 1 Batch 390/4268] avg loss 0.0181465, throughput 3.99559K wps
[Epoch 1 Batch 420/4268] avg loss 0.0176417, throughput 3.98856K wps
[Epoch 1 Batch 450/4268] avg loss 0.0176325, throughput 3.99205K wps
[Epoch 1 Batch 480/4268] avg loss 0.0183391, throughput 3.99578K wps
[Epoch 1 Batch 510/4268] avg loss 0.018008, throughput 3.99511K wps
[Epoch 1 Batch 540/4268] avg loss 0.017673, throughput 3.9952K wps
[Epoch 1 Batch 570/4268] avg loss 0.0168965, throughput 3.99383K wps
[Epoch 1 Batch 600/4268] avg loss 0.0173008, throughput 3.9956K wps
[Epoch 1 Batch 630/4268] avg loss 0.0174191, throughput 3.99624K wps
[Epoch 1 Batch 660/4268] avg loss 0.0173753, throughput 3.99711K wps
[Epoch 1 Batch 690/4268] avg loss 0.0170699, throughput 3.99706K wps
[Epoch 1 Batch 720/4268] avg loss 0.0177196, throughput 4.00226K wps
[Epoch 1 Batch 750/4268] avg loss 0.0178194, throughput 3.99862K wps
[Epoch 1 Batch 780/4268] avg loss 0.0182387, throughput 4.00082K wps
[Epoch 1 Batch 810/4268] avg loss 0.0179251, throughput 3.99991K wps
[Epoch 1 Batch 840/4268] avg loss 0.017264, throughput 3.99013K wps
[Epoch 1 Batch 870/4268] avg loss 0.0177291, throughput 3.99507K wps
[Epoch 1 Batch 900/4268] avg loss 0.0171564, throughput 3.99359K wps
[Epoch 1 Batch 930/4268] avg loss 0.0176471, throughput 3.99385K wps
[Epoch 1 Batch 960/4268] avg loss 0.0167575, throughput 3.99309K wps
[Epoch 1 Batch 990/4268] avg loss 0.0169316, throughput 3.99419K wps
[Epoch 1 Batch 1020/4268] avg loss 0.0171024, throughput 3.99155K wps
[Epoch 1 Batch 1050/4268] avg loss 0.017186, throughput 3.99255K wps
[Epoch 1 Batch 1080/4268] avg loss 0.0174575, throughput 3.99357K wps
[Epoch 1 Batch 1110/4268] avg loss 0.0182468, throughput 3.99747K wps
[Epoch 1 Batch 1140/4268] avg loss 0.0178165, throughput 3.99033K wps
[Epoch 1 Batch 1170/4268] avg loss 0.0171878, throughput 3.99584K wps
[Epoch 1 Batch 1200/4268] avg loss 0.0173125, throughput 3.99529K wps
[Epoch 1 Batch 1230/4268] avg loss 0.0178102, throughput 3.9929K wps
[Epoch 1 Batch 1260/4268] avg loss 0.0178213, throughput 3.99563K wps
[Epoch 1 Batch 1290/4268] avg loss 0.0173595, throughput 3.99187K wps
[Epoch 1 Batch 1320/4268] avg loss 0.0171387, throughput 3.99528K wps
[Epoch 1 Batch 1350/4268] avg loss 0.0179064, throughput 3.99382K wps
[Epoch 1 Batch 1380/4268] avg loss 0.0176121, throughput 3.99724K wps
[Epoch 1 Batch 1410/4268] avg loss 0.0169312, throughput 3.99013K wps
[Epoch 1 Batch 1440/4268] avg loss 0.0172881, throughput 3.99187K wps
[Epoch 1 Batch 1470/4268] avg loss 0.0181806, throughput 3.9945K wps
[Epoch 1 Batch 1500/4268] avg loss 0.0169012, throughput 3.99455K wps
[Epoch 1 Batch 1530/4268] avg loss 0.0171371, throughput 3.99035K wps
[Epoch 1 Batch 1560/4268] avg loss 0.017625, throughput 3.9946K wps
[Epoch 1 Batch 1590/4268] avg loss 0.0167643, throughput 4.00595K wps
[Epoch 1 Batch 1620/4268] avg loss 0.0172211, throughput 3.99113K wps
[Epoch 1 Batch 1650/4268] avg loss 0.0172481, throughput 3.99826K wps
[Epoch 1 Batch 1680/4268] avg loss 0.0164739, throughput 3.99263K wps
[Epoch 1 Batch 1710/4268] avg loss 0.0163178, throughput 3.99141K wps
[Epoch 1 Batch 1740/4268] avg loss 0.0173534, throughput 3.98989K wps
[Epoch 1 Batch 1770/4268] avg loss 0.0174815, throughput 3.99312K wps
[Epoch 1 Batch 1800/4268] avg loss 0.0173623, throughput 3.99278K wps
[Epoch 1 Batch 1830/4268] avg loss 0.0178077, throughput 3.99348K wps
[Epoch 1 Batch 1860/4268] avg loss 0.017562, throughput 3.99576K wps
[Epoch 1 Batch 1890/4268] avg loss 0.0176552, throughput 4.00221K wps
[Epoch 1 Batch 1920/4268] avg loss 0.0171392, throughput 3.99078K wps
[Epoch 1 Batch 1950/4268] avg loss 0.0174184, throughput 3.99142K wps
[Epoch 1 Batch 1980/4268] avg loss 0.0174039, throughput 3.99052K wps
[Epoch 1 Batch 2010/4268] avg loss 0.0170933, throughput 3.99404K wps
[Epoch 1 Batch 2040/4268] avg loss 0.0175623, throughput 3.99648K wps
[Epoch 1 Batch 2070/4268] avg loss 0.0179331, throughput 3.99266K wps
[Epoch 1 Batch 2100/4268] avg loss 0.016881, throughput 3.99354K wps
[Epoch 1 Batch 2130/4268] avg loss 0.0171998, throughput 3.99986K wps
[Epoch 1 Batch 2160/4268] avg loss 0.0174842, throughput 3.99555K wps
[Epoch 1 Batch 2190/4268] avg loss 0.0177367, throughput 3.99618K wps
[Epoch 1 Batch 2220/4268] avg loss 0.0175165, throughput 3.99484K wps
[Epoch 1 Batch 2250/4268] avg loss 0.0161795, throughput 3.98744K wps
[Epoch 1 Batch 2280/4268] avg loss 0.0179868, throughput 3.99867K wps
[Epoch 1 Batch 2310/4268] avg loss 0.0172498, throughput 3.99873K wps
[Epoch 1 Batch 2340/4268] avg loss 0.0173779, throughput 3.99349K wps
[Epoch 1 Batch 2370/4268] avg loss 0.0173686, throughput 3.99571K wps
[Epoch 1 Batch 2400/4268] avg loss 0.0173637, throughput 3.9937K wps
[Epoch 1 Batch 2430/4268] avg loss 0.0161648, throughput 3.99969K wps
[Epoch 1 Batch 2460/4268] avg loss 0.0170334, throughput 3.99435K wps
[Epoch 1 Batch 2490/4268] avg loss 0.0164087, throughput 3.99181K wps
[Epoch 1 Batch 2520/4268] avg loss 0.0180019, throughput 3.99772K wps
[Epoch 1 Batch 2550/4268] avg loss 0.0173222, throughput 3.99651K wps
[Epoch 1 Batch 2580/4268] avg loss 0.0175359, throughput 3.99383K wps
[Epoch 1 Batch 2610/4268] avg loss 0.0176465, throughput 3.99382K wps
[Epoch 1 Batch 2640/4268] avg loss 0.0173771, throughput 3.97674K wps
[Epoch 1 Batch 2670/4268] avg loss 0.0169482, throughput 3.99167K wps
[Epoch 1 Batch 2700/4268] avg loss 0.016423, throughput 3.99133K wps
[Epoch 1 Batch 2730/4268] avg loss 0.017482, throughput 3.98873K wps
[Epoch 1 Batch 2760/4268] avg loss 0.0170274, throughput 3.99553K wps
[Epoch 1 Batch 2790/4268] avg loss 0.0161201, throughput 3.99103K wps
[Epoch 1 Batch 2820/4268] avg loss 0.0174822, throughput 3.9907K wps
[Epoch 1 Batch 2850/4268] avg loss 0.017181, throughput 3.99155K wps
[Epoch 1 Batch 2880/4268] avg loss 0.0171174, throughput 3.9915K wps
[Epoch 1 Batch 2910/4268] avg loss 0.0182153, throughput 3.99207K wps
[Epoch 1 Batch 2940/4268] avg loss 0.016979, throughput 3.99117K wps
[Epoch 1 Batch 2970/4268] avg loss 0.0168823, throughput 3.99239K wps
[Epoch 1 Batch 3000/4268] avg loss 0.0170879, throughput 3.99859K wps
[Epoch 1 Batch 3030/4268] avg loss 0.0167677, throughput 3.99217K wps
[Epoch 1 Batch 3060/4268] avg loss 0.0165963, throughput 3.99007K wps
[Epoch 1 Batch 3090/4268] avg loss 0.0165482, throughput 3.99204K wps
[Epoch 1 Batch 3120/4268] avg loss 0.0164381, throughput 3.99304K wps
[Epoch 1 Batch 3150/4268] avg loss 0.0164623, throughput 3.99476K wps
[Epoch 1 Batch 3180/4268] avg loss 0.0163521, throughput 3.99409K wps
[Epoch 1 Batch 3210/4268] avg loss 0.0169532, throughput 3.99229K wps
[Epoch 1 Batch 3240/4268] avg loss 0.0168184, throughput 3.99578K wps
[Epoch 1 Batch 3270/4268] avg loss 0.0170617, throughput 3.99433K wps
[Epoch 1 Batch 3300/4268] avg loss 0.0168618, throughput 3.99438K wps
[Epoch 1 Batch 3330/4268] avg loss 0.0171418, throughput 3.99618K wps
[Epoch 1 Batch 3360/4268] avg loss 0.0163664, throughput 3.99447K wps
[Epoch 1 Batch 3390/4268] avg loss 0.0166129, throughput 3.993K wps
[Epoch 1 Batch 3420/4268] avg loss 0.016888, throughput 3.99952K wps
[Epoch 1 Batch 3450/4268] avg loss 0.0169406, throughput 4.00028K wps
[Epoch 1 Batch 3480/4268] avg loss 0.0170717, throughput 4.00085K wps
[Epoch 1 Batch 3510/4268] avg loss 0.0166115, throughput 3.99462K wps
[Epoch 1 Batch 3540/4268] avg loss 0.0163537, throughput 3.99455K wps
[Epoch 1 Batch 3570/4268] avg loss 0.0172919, throughput 3.9966K wps
[Epoch 1 Batch 3600/4268] avg loss 0.0168166, throughput 3.99414K wps
[Epoch 1 Batch 3630/4268] avg loss 0.0167569, throughput 3.98937K wps
[Epoch 1 Batch 3660/4268] avg loss 0.0169347, throughput 3.99153K wps
[Epoch 1 Batch 3690/4268] avg loss 0.0165531, throughput 3.9915K wps
[Epoch 1 Batch 3720/4268] avg loss 0.0168965, throughput 3.99539K wps
[Epoch 1 Batch 3750/4268] avg loss 0.0174273, throughput 3.99959K wps
[Epoch 1 Batch 3780/4268] avg loss 0.016198, throughput 4.00192K wps
[Epoch 1 Batch 3810/4268] avg loss 0.0164831, throughput 3.99636K wps
[Epoch 1 Batch 3840/4268] avg loss 0.0167459, throughput 3.99439K wps
[Epoch 1 Batch 3870/4268] avg loss 0.0173207, throughput 3.99424K wps
[Epoch 1 Batch 3900/4268] avg loss 0.0170859, throughput 3.99841K wps
[Epoch 1 Batch 3930/4268] avg loss 0.0170998, throughput 4.00166K wps
[Epoch 1 Batch 3960/4268] avg loss 0.0167234, throughput 3.99963K wps
[Epoch 1 Batch 3990/4268] avg loss 0.0166111, throughput 3.99857K wps
[Epoch 1 Batch 4020/4268] avg loss 0.0172906, throughput 3.99599K wps
[Epoch 1 Batch 4050/4268] avg loss 0.0172607, throughput 3.99517K wps
[Epoch 1 Batch 4080/4268] avg loss 0.0179173, throughput 3.99981K wps
[Epoch 1 Batch 4110/4268] avg loss 0.0165471, throughput 4.0061K wps
[Epoch 1 Batch 4140/4268] avg loss 0.0172145, throughput 3.99422K wps
[Epoch 1 Batch 4170/4268] avg loss 0.01753, throughput 4.00226K wps
[Epoch 1 Batch 4200/4268] avg loss 0.0167929, throughput 3.99546K wps
[Epoch 1 Batch 4230/4268] avg loss 0.0164515, throughput 3.99987K wps
[Epoch 1 Batch 4260/4268] avg loss 0.0162194, throughput 3.99529K wps
Begin Testing...
[Batch 30/475] elapsed 0.47 s
[Batch 60/475] elapsed 0.43 s
[Batch 90/475] elapsed 0.43 s
[Batch 120/475] elapsed 0.43 s
[Batch 150/475] elapsed 0.43 s
[Batch 180/475] elapsed 0.43 s
[Batch 210/475] elapsed 0.43 s
[Batch 240/475] elapsed 0.43 s
[Batch 270/475] elapsed 0.43 s
[Batch 300/475] elapsed 0.43 s
[Batch 330/475] elapsed 0.43 s
[Batch 360/475] elapsed 0.43 s
[Batch 390/475] elapsed 0.43 s
[Batch 420/475] elapsed 0.43 s
[Batch 450/475] elapsed 0.43 s
[Epoch 1] train avg loss 0.0172177, test acc 0.6617, test avg loss 0.823489, throughput 3.99532K wps
Observed Improvement.
Begin Testing...
[Batch 30/43] elapsed 0.43 s
[Epoch 2 Batch 30/4268] avg loss 0.0162876, throughput 4.08149K wps
[Epoch 2 Batch 60/4268] avg loss 0.0156635, throughput 3.99807K wps
[Epoch 2 Batch 90/4268] avg loss 0.0153209, throughput 4.00026K wps
[Epoch 2 Batch 120/4268] avg loss 0.0156707, throughput 3.99914K wps
[Epoch 2 Batch 150/4268] avg loss 0.0155036, throughput 4.00544K wps
[Epoch 2 Batch 180/4268] avg loss 0.0159855, throughput 3.99635K wps
[Epoch 2 Batch 210/4268] avg loss 0.0163363, throughput 3.99484K wps
[Epoch 2 Batch 240/4268] avg loss 0.0162584, throughput 3.99546K wps
[Epoch 2 Batch 270/4268] avg loss 0.0151098, throughput 3.99572K wps
[Epoch 2 Batch 300/4268] avg loss 0.0154073, throughput 3.99609K wps
[Epoch 2 Batch 330/4268] avg loss 0.0154843, throughput 3.99764K wps
[Epoch 2 Batch 360/4268] avg loss 0.0163793, throughput 3.99821K wps
[Epoch 2 Batch 390/4268] avg loss 0.0168758, throughput 4.00424K wps
[Epoch 2 Batch 420/4268] avg loss 0.0164964, throughput 3.99529K wps
[Epoch 2 Batch 450/4268] avg loss 0.0155236, throughput 3.9989K wps
[Epoch 2 Batch 480/4268] avg loss 0.0154526, throughput 3.99096K wps
[Epoch 2 Batch 510/4268] avg loss 0.0164051, throughput 3.99632K wps
[Epoch 2 Batch 540/4268] avg loss 0.0163277, throughput 3.99819K wps
[Epoch 2 Batch 570/4268] avg loss 0.0157675, throughput 3.99589K wps
[Epoch 2 Batch 600/4268] avg loss 0.0163372, throughput 3.99205K wps
[Epoch 2 Batch 630/4268] avg loss 0.0155693, throughput 3.99487K wps
[Epoch 2 Batch 660/4268] avg loss 0.0159922, throughput 3.99837K wps
[Epoch 2 Batch 690/4268] avg loss 0.0159705, throughput 3.99805K wps
[Epoch 2 Batch 720/4268] avg loss 0.0157451, throughput 3.99134K wps
[Epoch 2 Batch 750/4268] avg loss 0.01596, throughput 3.99212K wps
[Epoch 2 Batch 780/4268] avg loss 0.0160573, throughput 3.99086K wps
[Epoch 2 Batch 810/4268] avg loss 0.0161489, throughput 3.99341K wps
[Epoch 2 Batch 840/4268] avg loss 0.0165521, throughput 4.00055K wps
[Epoch 2 Batch 870/4268] avg loss 0.0156946, throughput 3.99434K wps
[Epoch 2 Batch 900/4268] avg loss 0.0163384, throughput 3.99915K wps
[Epoch 2 Batch 930/4268] avg loss 0.0163206, throughput 3.99339K wps
[Epoch 2 Batch 960/4268] avg loss 0.0163253, throughput 3.9985K wps
[Epoch 2 Batch 990/4268] avg loss 0.0166686, throughput 3.99692K wps
[Epoch 2 Batch 1020/4268] avg loss 0.0159315, throughput 3.99454K wps
[Epoch 2 Batch 1050/4268] avg loss 0.0160078, throughput 3.99314K wps
[Epoch 2 Batch 1080/4268] avg loss 0.0164787, throughput 3.99732K wps
[Epoch 2 Batch 1110/4268] avg loss 0.0161389, throughput 3.99619K wps
[Epoch 2 Batch 1140/4268] avg loss 0.0157833, throughput 3.99474K wps
[Epoch 2 Batch 1170/4268] avg loss 0.0166795, throughput 3.99261K wps
[Epoch 2 Batch 1200/4268] avg loss 0.0165758, throughput 3.9967K wps
[Epoch 2 Batch 1230/4268] avg loss 0.0164338, throughput 3.99281K wps
[Epoch 2 Batch 1260/4268] avg loss 0.0159377, throughput 3.99035K wps
[Epoch 2 Batch 1290/4268] avg loss 0.0161897, throughput 3.99296K wps
[Epoch 2 Batch 1320/4268] avg loss 0.0160197, throughput 3.99007K wps
[Epoch 2 Batch 1350/4268] avg loss 0.015698, throughput 3.98943K wps
[Epoch 2 Batch 1380/4268] avg loss 0.0165046, throughput 3.99424K wps
[Epoch 2 Batch 1410/4268] avg loss 0.016296, throughput 3.99129K wps
[Epoch 2 Batch 1440/4268] avg loss 0.0157812, throughput 3.99099K wps
[Epoch 2 Batch 1470/4268] avg loss 0.01592, throughput 3.99131K wps
[Epoch 2 Batch 1500/4268] avg loss 0.0162367, throughput 3.99669K wps
[Epoch 2 Batch 1530/4268] avg loss 0.0156004, throughput 3.99797K wps
[Epoch 2 Batch 1560/4268] avg loss 0.0161122, throughput 3.99504K wps
[Epoch 2 Batch 1590/4268] avg loss 0.0157878, throughput 3.9976K wps
[Epoch 2 Batch 1620/4268] avg loss 0.016013, throughput 3.99439K wps
[Epoch 2 Batch 1650/4268] avg loss 0.016646, throughput 3.99495K wps
[Epoch 2 Batch 1680/4268] avg loss 0.0164882, throughput 3.9945K wps
[Epoch 2 Batch 1710/4268] avg loss 0.0159942, throughput 3.99395K wps
[Epoch 2 Batch 1740/4268] avg loss 0.016788, throughput 3.99436K wps
[Epoch 2 Batch 1770/4268] avg loss 0.0153908, throughput 3.99362K wps
[Epoch 2 Batch 1800/4268] avg loss 0.0159535, throughput 3.9925K wps
[Epoch 2 Batch 1830/4268] avg loss 0.0165674, throughput 3.97874K wps
[Epoch 2 Batch 1860/4268] avg loss 0.0152746, throughput 3.99298K wps
[Epoch 2 Batch 1890/4268] avg loss 0.016877, throughput 3.99148K wps
[Epoch 2 Batch 1920/4268] avg loss 0.0162957, throughput 4.00021K wps
[Epoch 2 Batch 1950/4268] avg loss 0.0163238, throughput 3.99457K wps
[Epoch 2 Batch 1980/4268] avg loss 0.0160452, throughput 3.99458K wps
[Epoch 2 Batch 2010/4268] avg loss 0.0158672, throughput 3.99517K wps
[Epoch 2 Batch 2040/4268] avg loss 0.0163038, throughput 3.99604K wps
[Epoch 2 Batch 2070/4268] avg loss 0.0162412, throughput 3.99572K wps
[Epoch 2 Batch 2100/4268] avg loss 0.015499, throughput 3.99565K wps
[Epoch 2 Batch 2130/4268] avg loss 0.015358, throughput 3.9932K wps
[Epoch 2 Batch 2160/4268] avg loss 0.0157656, throughput 3.99308K wps
[Epoch 2 Batch 2190/4268] avg loss 0.0160567, throughput 3.99336K wps
[Epoch 2 Batch 2220/4268] avg loss 0.0160417, throughput 3.99805K wps
[Epoch 2 Batch 2250/4268] avg loss 0.0157524, throughput 3.99369K wps
[Epoch 2 Batch 2280/4268] avg loss 0.0161934, throughput 3.99968K wps
[Epoch 2 Batch 2310/4268] avg loss 0.0164933, throughput 3.99292K wps
[Epoch 2 Batch 2340/4268] avg loss 0.0160345, throughput 3.99464K wps
[Epoch 2 Batch 2370/4268] avg loss 0.0156135, throughput 3.99744K wps
[Epoch 2 Batch 2400/4268] avg loss 0.0152922, throughput 3.9972K wps
[Epoch 2 Batch 2430/4268] avg loss 0.016587, throughput 3.99456K wps
[Epoch 2 Batch 2460/4268] avg loss 0.0160537, throughput 3.99251K wps
[Epoch 2 Batch 2490/4268] avg loss 0.0152419, throughput 3.99583K wps
[Epoch 2 Batch 2520/4268] avg loss 0.0165574, throughput 3.99194K wps
[Epoch 2 Batch 2550/4268] avg loss 0.0162984, throughput 3.99753K wps
[Epoch 2 Batch 2580/4268] avg loss 0.0159595, throughput 3.99459K wps
[Epoch 2 Batch 2610/4268] avg loss 0.0153002, throughput 4.00081K wps
[Epoch 2 Batch 2640/4268] avg loss 0.0153984, throughput 4.00425K wps
[Epoch 2 Batch 2670/4268] avg loss 0.0160057, throughput 3.99204K wps
[Epoch 2 Batch 2700/4268] avg loss 0.0158105, throughput 3.99377K wps
[Epoch 2 Batch 2730/4268] avg loss 0.0157446, throughput 3.99207K wps
[Epoch 2 Batch 2760/4268] avg loss 0.0150213, throughput 3.99601K wps
[Epoch 2 Batch 2790/4268] avg loss 0.0157095, throughput 3.99606K wps
[Epoch 2 Batch 2820/4268] avg loss 0.0164952, throughput 3.99671K wps
[Epoch 2 Batch 2850/4268] avg loss 0.0164952, throughput 3.99432K wps
[Epoch 2 Batch 2880/4268] avg loss 0.01624, throughput 3.99495K wps
[Epoch 2 Batch 2910/4268] avg loss 0.0165459, throughput 4.00253K wps
[Epoch 2 Batch 2940/4268] avg loss 0.0160772, throughput 4.00522K wps
[Epoch 2 Batch 2970/4268] avg loss 0.0158294, throughput 3.99879K wps
[Epoch 2 Batch 3000/4268] avg loss 0.0167549, throughput 3.99708K wps
[Epoch 2 Batch 3030/4268] avg loss 0.0164203, throughput 3.99719K wps
[Epoch 2 Batch 3060/4268] avg loss 0.0161464, throughput 3.99898K wps
[Epoch 2 Batch 3090/4268] avg loss 0.0160735, throughput 3.99401K wps
[Epoch 2 Batch 3120/4268] avg loss 0.016516, throughput 3.99473K wps
[Epoch 2 Batch 3150/4268] avg loss 0.0161896, throughput 3.99856K wps
[Epoch 2 Batch 3180/4268] avg loss 0.0167055, throughput 3.99594K wps
[Epoch 2 Batch 3210/4268] avg loss 0.0165209, throughput 3.99719K wps
[Epoch 2 Batch 3240/4268] avg loss 0.0169349, throughput 3.99712K wps
[Epoch 2 Batch 3270/4268] avg loss 0.0155145, throughput 3.99817K wps
[Epoch 2 Batch 3300/4268] avg loss 0.0154237, throughput 3.99338K wps
[Epoch 2 Batch 3330/4268] avg loss 0.0156293, throughput 4.00126K wps
[Epoch 2 Batch 3360/4268] avg loss 0.0159518, throughput 3.99596K wps
[Epoch 2 Batch 3390/4268] avg loss 0.0159955, throughput 3.994K wps
[Epoch 2 Batch 3420/4268] avg loss 0.0163128, throughput 3.99305K wps
[Epoch 2 Batch 3450/4268] avg loss 0.0160637, throughput 3.99957K wps
[Epoch 2 Batch 3480/4268] avg loss 0.0166178, throughput 3.99029K wps
[Epoch 2 Batch 3510/4268] avg loss 0.0161038, throughput 3.99634K wps
[Epoch 2 Batch 3540/4268] avg loss 0.0160967, throughput 4.00181K wps
[Epoch 2 Batch 3570/4268] avg loss 0.0162295, throughput 4.00266K wps
[Epoch 2 Batch 3600/4268] avg loss 0.0162439, throughput 3.99868K wps
[Epoch 2 Batch 3630/4268] avg loss 0.0160493, throughput 3.99911K wps
[Epoch 2 Batch 3660/4268] avg loss 0.0151987, throughput 3.9972K wps
[Epoch 2 Batch 3690/4268] avg loss 0.0157996, throughput 3.99416K wps
[Epoch 2 Batch 3720/4268] avg loss 0.0163178, throughput 3.99852K wps
[Epoch 2 Batch 3750/4268] avg loss 0.0155522, throughput 3.99748K wps
[Epoch 2 Batch 3780/4268] avg loss 0.0164103, throughput 3.99617K wps
[Epoch 2 Batch 3810/4268] avg loss 0.0162742, throughput 3.99721K wps
[Epoch 2 Batch 3840/4268] avg loss 0.0158914, throughput 3.99597K wps
[Epoch 2 Batch 3870/4268] avg loss 0.0158712, throughput 3.99574K wps
[Epoch 2 Batch 3900/4268] avg loss 0.0162558, throughput 3.99955K wps
[Epoch 2 Batch 3930/4268] avg loss 0.0164355, throughput 3.99749K wps
[Epoch 2 Batch 3960/4268] avg loss 0.0158502, throughput 4.00007K wps
[Epoch 2 Batch 3990/4268] avg loss 0.0168948, throughput 3.9963K wps
[Epoch 2 Batch 4020/4268] avg loss 0.0153601, throughput 3.99588K wps
[Epoch 2 Batch 4050/4268] avg loss 0.015763, throughput 3.99318K wps
[Epoch 2 Batch 4080/4268] avg loss 0.0163214, throughput 3.99172K wps
[Epoch 2 Batch 4110/4268] avg loss 0.0155036, throughput 3.9947K wps
[Epoch 2 Batch 4140/4268] avg loss 0.0161906, throughput 3.99087K wps
[Epoch 2 Batch 4170/4268] avg loss 0.0154027, throughput 3.99247K wps
[Epoch 2 Batch 4200/4268] avg loss 0.0159504, throughput 3.99342K wps
[Epoch 2 Batch 4230/4268] avg loss 0.016096, throughput 3.99131K wps
[Epoch 2 Batch 4260/4268] avg loss 0.0156606, throughput 3.99884K wps
Begin Testing...
[Batch 30/475] elapsed 0.46 s
[Batch 60/475] elapsed 0.43 s
[Batch 90/475] elapsed 0.43 s
[Batch 120/475] elapsed 0.43 s
[Batch 150/475] elapsed 0.43 s
[Batch 180/475] elapsed 0.43 s
[Batch 210/475] elapsed 0.43 s
[Batch 240/475] elapsed 0.43 s
[Batch 270/475] elapsed 0.43 s
[Batch 300/475] elapsed 0.43 s
[Batch 330/475] elapsed 0.43 s
[Batch 360/475] elapsed 0.43 s
[Batch 390/475] elapsed 0.43 s
[Batch 420/475] elapsed 0.43 s
[Batch 450/475] elapsed 0.43 s
[Epoch 2] train avg loss 0.016042, test acc 0.6637, test avg loss 0.808088, throughput 3.99624K wps
Observed Improvement.
Begin Testing...
[Batch 30/43] elapsed 0.43 s
[Epoch 3 Batch 30/4268] avg loss 0.0154424, throughput 4.08034K wps
[Epoch 3 Batch 60/4268] avg loss 0.015075, throughput 3.99354K wps
[Epoch 3 Batch 90/4268] avg loss 0.0156092, throughput 3.99243K wps
[Epoch 3 Batch 120/4268] avg loss 0.0154055, throughput 3.99503K wps
[Epoch 3 Batch 150/4268] avg loss 0.0151009, throughput 3.99221K wps
[Epoch 3 Batch 180/4268] avg loss 0.0155677, throughput 3.99375K wps
[Epoch 3 Batch 210/4268] avg loss 0.0150275, throughput 3.99489K wps
[Epoch 3 Batch 240/4268] avg loss 0.0158164, throughput 3.99813K wps
[Epoch 3 Batch 270/4268] avg loss 0.0151677, throughput 3.99209K wps
[Epoch 3 Batch 300/4268] avg loss 0.015643, throughput 3.9911K wps
[Epoch 3 Batch 330/4268] avg loss 0.0149809, throughput 3.99581K wps
[Epoch 3 Batch 360/4268] avg loss 0.0150907, throughput 3.9964K wps
[Epoch 3 Batch 390/4268] avg loss 0.0147628, throughput 3.99745K wps
[Epoch 3 Batch 420/4268] avg loss 0.0152015, throughput 3.99574K wps
[Epoch 3 Batch 450/4268] avg loss 0.0157182, throughput 3.99436K wps
[Epoch 3 Batch 480/4268] avg loss 0.0152753, throughput 3.99645K wps
[Epoch 3 Batch 510/4268] avg loss 0.0155374, throughput 3.99432K wps
[Epoch 3 Batch 540/4268] avg loss 0.0156053, throughput 3.99397K wps
[Epoch 3 Batch 570/4268] avg loss 0.0156729, throughput 3.99561K wps
[Epoch 3 Batch 600/4268] avg loss 0.0148992, throughput 3.99776K wps
[Epoch 3 Batch 630/4268] avg loss 0.0154884, throughput 3.99532K wps
[Epoch 3 Batch 660/4268] avg loss 0.0148909, throughput 3.99495K wps
[Epoch 3 Batch 690/4268] avg loss 0.0150869, throughput 3.99476K wps
[Epoch 3 Batch 720/4268] avg loss 0.0149024, throughput 3.99381K wps
[Epoch 3 Batch 750/4268] avg loss 0.01543, throughput 3.99381K wps
[Epoch 3 Batch 780/4268] avg loss 0.0155658, throughput 3.99314K wps
[Epoch 3 Batch 810/4268] avg loss 0.0155299, throughput 3.99217K wps
[Epoch 3 Batch 840/4268] avg loss 0.0159088, throughput 3.99473K wps
[Epoch 3 Batch 870/4268] avg loss 0.01515, throughput 3.99442K wps
[Epoch 3 Batch 900/4268] avg loss 0.0157224, throughput 3.99278K wps
[Epoch 3 Batch 930/4268] avg loss 0.0145551, throughput 3.99897K wps
[Epoch 3 Batch 960/4268] avg loss 0.01494, throughput 3.99612K wps
[Epoch 3 Batch 990/4268] avg loss 0.015278, throughput 3.97932K wps
[Epoch 3 Batch 1020/4268] avg loss 0.0147967, throughput 3.97118K wps
[Epoch 3 Batch 1050/4268] avg loss 0.0155913, throughput 3.99096K wps
[Epoch 3 Batch 1080/4268] avg loss 0.0150017, throughput 3.98854K wps
[Epoch 3 Batch 1110/4268] avg loss 0.0158338, throughput 3.99568K wps
[Epoch 3 Batch 1140/4268] avg loss 0.0149914, throughput 3.99262K wps
[Epoch 3 Batch 1170/4268] avg loss 0.0152158, throughput 3.99216K wps
[Epoch 3 Batch 1200/4268] avg loss 0.0151239, throughput 3.99189K wps
[Epoch 3 Batch 1230/4268] avg loss 0.0150485, throughput 3.99244K wps
[Epoch 3 Batch 1260/4268] avg loss 0.015462, throughput 3.99166K wps
[Epoch 3 Batch 1290/4268] avg loss 0.0152078, throughput 3.99358K wps
[Epoch 3 Batch 1320/4268] avg loss 0.0148878, throughput 3.99359K wps
[Epoch 3 Batch 1350/4268] avg loss 0.0145966, throughput 3.99129K wps
[Epoch 3 Batch 1380/4268] avg loss 0.0149422, throughput 3.98682K wps
[Epoch 3 Batch 1410/4268] avg loss 0.0158071, throughput 3.99623K wps
[Epoch 3 Batch 1440/4268] avg loss 0.0148311, throughput 3.99406K wps
[Epoch 3 Batch 1470/4268] avg loss 0.015217, throughput 3.99234K wps
[Epoch 3 Batch 1500/4268] avg loss 0.0150473, throughput 3.99611K wps
[Epoch 3 Batch 1530/4268] avg loss 0.0149054, throughput 3.99293K wps
[Epoch 3 Batch 1560/4268] avg loss 0.0155042, throughput 3.99133K wps
[Epoch 3 Batch 1590/4268] avg loss 0.0149905, throughput 3.99216K wps
[Epoch 3 Batch 1620/4268] avg loss 0.0155915, throughput 3.99421K wps
[Epoch 3 Batch 1650/4268] avg loss 0.0147868, throughput 3.99718K wps
[Epoch 3 Batch 1680/4268] avg loss 0.0156214, throughput 3.9942K wps
[Epoch 3 Batch 1710/4268] avg loss 0.0149783, throughput 3.99407K wps
[Epoch 3 Batch 1740/4268] avg loss 0.015464, throughput 3.99222K wps
[Epoch 3 Batch 1770/4268] avg loss 0.0162623, throughput 3.99076K wps
[Epoch 3 Batch 1800/4268] avg loss 0.0152282, throughput 3.99428K wps
[Epoch 3 Batch 1830/4268] avg loss 0.0153585, throughput 3.9873K wps
[Epoch 3 Batch 1860/4268] avg loss 0.0150987, throughput 3.99242K wps
[Epoch 3 Batch 1890/4268] avg loss 0.0162872, throughput 3.99177K wps
[Epoch 3 Batch 1920/4268] avg loss 0.0158571, throughput 3.99884K wps
[Epoch 3 Batch 1950/4268] avg loss 0.0148929, throughput 3.99249K wps
[Epoch 3 Batch 1980/4268] avg loss 0.0151334, throughput 3.99527K wps
[Epoch 3 Batch 2010/4268] avg loss 0.0159462, throughput 3.99494K wps
[Epoch 3 Batch 2040/4268] avg loss 0.0156777, throughput 3.99723K wps
[Epoch 3 Batch 2070/4268] avg loss 0.0152323, throughput 3.99638K wps
[Epoch 3 Batch 2100/4268] avg loss 0.0148155, throughput 3.9937K wps
[Epoch 3 Batch 2130/4268] avg loss 0.0149493, throughput 3.99211K wps
[Epoch 3 Batch 2160/4268] avg loss 0.0157381, throughput 3.99519K wps
[Epoch 3 Batch 2190/4268] avg loss 0.0162519, throughput 3.99698K wps
[Epoch 3 Batch 2220/4268] avg loss 0.0152943, throughput 3.99288K wps
[Epoch 3 Batch 2250/4268] avg loss 0.0154581, throughput 3.99859K wps
[Epoch 3 Batch 2280/4268] avg loss 0.0150938, throughput 3.99374K wps
[Epoch 3 Batch 2310/4268] avg loss 0.0156554, throughput 3.99732K wps
[Epoch 3 Batch 2340/4268] avg loss 0.0152159, throughput 3.99399K wps
[Epoch 3 Batch 2370/4268] avg loss 0.0151852, throughput 3.99K wps
[Epoch 3 Batch 2400/4268] avg loss 0.0155268, throughput 3.99107K wps
[Epoch 3 Batch 2430/4268] avg loss 0.0155535, throughput 3.99433K wps
[Epoch 3 Batch 2460/4268] avg loss 0.0158771, throughput 3.99399K wps
[Epoch 3 Batch 2490/4268] avg loss 0.0153642, throughput 3.99428K wps
[Epoch 3 Batch 2520/4268] avg loss 0.0155152, throughput 3.99591K wps
[Epoch 3 Batch 2550/4268] avg loss 0.0156773, throughput 3.99397K wps
[Epoch 3 Batch 2580/4268] avg loss 0.0147424, throughput 3.99559K wps
[Epoch 3 Batch 2610/4268] avg loss 0.0155484, throughput 3.99548K wps
[Epoch 3 Batch 2640/4268] avg loss 0.0150079, throughput 3.99258K wps
[Epoch 3 Batch 2670/4268] avg loss 0.0160003, throughput 3.99118K wps
[Epoch 3 Batch 2700/4268] avg loss 0.0152731, throughput 3.9985K wps
[Epoch 3 Batch 2730/4268] avg loss 0.0153532, throughput 3.99109K wps
[Epoch 3 Batch 2760/4268] avg loss 0.014964, throughput 3.99644K wps
[Epoch 3 Batch 2790/4268] avg loss 0.0149595, throughput 3.99266K wps
[Epoch 3 Batch 2820/4268] avg loss 0.0153459, throughput 3.9974K wps
[Epoch 3 Batch 2850/4268] avg loss 0.0152822, throughput 3.99629K wps
[Epoch 3 Batch 2880/4268] avg loss 0.0152975, throughput 3.99289K wps
[Epoch 3 Batch 2910/4268] avg loss 0.0143591, throughput 3.99745K wps
[Epoch 3 Batch 2940/4268] avg loss 0.0159568, throughput 3.98979K wps
[Epoch 3 Batch 2970/4268] avg loss 0.0154808, throughput 3.9974K wps
[Epoch 3 Batch 3000/4268] avg loss 0.0148423, throughput 3.99626K wps
[Epoch 3 Batch 3030/4268] avg loss 0.0155265, throughput 4.00154K wps
[Epoch 3 Batch 3060/4268] avg loss 0.0146178, throughput 3.99233K wps
[Epoch 3 Batch 3090/4268] avg loss 0.0148045, throughput 3.99767K wps
[Epoch 3 Batch 3120/4268] avg loss 0.0158027, throughput 3.99402K wps
[Epoch 3 Batch 3150/4268] avg loss 0.0158097, throughput 3.99684K wps
[Epoch 3 Batch 3180/4268] avg loss 0.0152467, throughput 3.99689K wps
[Epoch 3 Batch 3210/4268] avg loss 0.015478, throughput 3.99282K wps
[Epoch 3 Batch 3240/4268] avg loss 0.0156159, throughput 3.99035K wps
[Epoch 3 Batch 3270/4268] avg loss 0.0156874, throughput 3.9918K wps
[Epoch 3 Batch 3300/4268] avg loss 0.0161664, throughput 3.99152K wps
[Epoch 3 Batch 3330/4268] avg loss 0.0152301, throughput 3.99639K wps
[Epoch 3 Batch 3360/4268] avg loss 0.0151935, throughput 3.99827K wps
[Epoch 3 Batch 3390/4268] avg loss 0.0147728, throughput 3.99893K wps
[Epoch 3 Batch 3420/4268] avg loss 0.015741, throughput 3.9901K wps
[Epoch 3 Batch 3450/4268] avg loss 0.0154223, throughput 3.99128K wps
[Epoch 3 Batch 3480/4268] avg loss 0.0152737, throughput 3.99531K wps
[Epoch 3 Batch 3510/4268] avg loss 0.0154967, throughput 3.99388K wps
[Epoch 3 Batch 3540/4268] avg loss 0.0155315, throughput 3.99179K wps
[Epoch 3 Batch 3570/4268] avg loss 0.0152428, throughput 3.99203K wps
[Epoch 3 Batch 3600/4268] avg loss 0.0153065, throughput 3.99337K wps
[Epoch 3 Batch 3630/4268] avg loss 0.0154588, throughput 4.00213K wps
[Epoch 3 Batch 3660/4268] avg loss 0.0150731, throughput 3.99849K wps
[Epoch 3 Batch 3690/4268] avg loss 0.0150875, throughput 3.99806K wps
[Epoch 3 Batch 3720/4268] avg loss 0.0153087, throughput 4.00485K wps
[Epoch 3 Batch 3750/4268] avg loss 0.0154085, throughput 3.99535K wps
[Epoch 3 Batch 3780/4268] avg loss 0.0155141, throughput 3.99193K wps
[Epoch 3 Batch 3810/4268] avg loss 0.0156663, throughput 3.98989K wps
[Epoch 3 Batch 3840/4268] avg loss 0.0158138, throughput 3.99747K wps
[Epoch 3 Batch 3870/4268] avg loss 0.0146269, throughput 3.99405K wps
[Epoch 3 Batch 3900/4268] avg loss 0.0150001, throughput 3.99054K wps
[Epoch 3 Batch 3930/4268] avg loss 0.0156591, throughput 3.99278K wps
[Epoch 3 Batch 3960/4268] avg loss 0.0151518, throughput 3.99472K wps
[Epoch 3 Batch 3990/4268] avg loss 0.0160789, throughput 3.99315K wps
[Epoch 3 Batch 4020/4268] avg loss 0.016168, throughput 3.99289K wps
[Epoch 3 Batch 4050/4268] avg loss 0.015075, throughput 3.99737K wps
[Epoch 3 Batch 4080/4268] avg loss 0.0158316, throughput 3.99806K wps
[Epoch 3 Batch 4110/4268] avg loss 0.0154877, throughput 3.99423K wps
[Epoch 3 Batch 4140/4268] avg loss 0.015389, throughput 3.99487K wps
[Epoch 3 Batch 4170/4268] avg loss 0.0147884, throughput 3.99542K wps
[Epoch 3 Batch 4200/4268] avg loss 0.0151834, throughput 3.99669K wps
[Epoch 3 Batch 4230/4268] avg loss 0.0159708, throughput 3.99882K wps
[Epoch 3 Batch 4260/4268] avg loss 0.0151439, throughput 3.9927K wps
Begin Testing...
[Batch 30/475] elapsed 0.46 s
[Batch 60/475] elapsed 0.43 s
[Batch 90/475] elapsed 0.43 s
[Batch 120/475] elapsed 0.43 s
[Batch 150/475] elapsed 0.43 s
[Batch 180/475] elapsed 0.43 s
[Batch 210/475] elapsed 0.43 s
[Batch 240/475] elapsed 0.43 s
[Batch 270/475] elapsed 0.43 s
[Batch 300/475] elapsed 0.43 s
[Batch 330/475] elapsed 0.43 s
[Batch 360/475] elapsed 0.43 s
[Batch 390/475] elapsed 0.43 s
[Batch 420/475] elapsed 0.43 s
[Batch 450/475] elapsed 0.43 s
[Epoch 3] train avg loss 0.0153383, test acc 0.6762, test avg loss 0.794013, throughput 3.99465K wps
Observed Improvement.
Begin Testing...
[Batch 30/43] elapsed 0.43 s
[Epoch 4 Batch 30/4268] avg loss 0.0144923, throughput 4.08144K wps
[Epoch 4 Batch 60/4268] avg loss 0.0150506, throughput 3.9987K wps
[Epoch 4 Batch 90/4268] avg loss 0.0144079, throughput 3.99498K wps
[Epoch 4 Batch 120/4268] avg loss 0.0144163, throughput 3.99665K wps
[Epoch 4 Batch 150/4268] avg loss 0.015535, throughput 3.99522K wps
[Epoch 4 Batch 180/4268] avg loss 0.0148225, throughput 3.98668K wps
[Epoch 4 Batch 210/4268] avg loss 0.0145585, throughput 3.98932K wps
[Epoch 4 Batch 240/4268] avg loss 0.0152747, throughput 3.98771K wps
[Epoch 4 Batch 270/4268] avg loss 0.0142531, throughput 3.99067K wps
[Epoch 4 Batch 300/4268] avg loss 0.0147363, throughput 4.00038K wps
[Epoch 4 Batch 330/4268] avg loss 0.0151503, throughput 4.00015K wps
[Epoch 4 Batch 360/4268] avg loss 0.014325, throughput 3.99359K wps
[Epoch 4 Batch 390/4268] avg loss 0.0148825, throughput 3.99539K wps
[Epoch 4 Batch 420/4268] avg loss 0.0142009, throughput 3.99579K wps
[Epoch 4 Batch 450/4268] avg loss 0.0144552, throughput 3.9951K wps
[Epoch 4 Batch 480/4268] avg loss 0.0143193, throughput 3.99666K wps
[Epoch 4 Batch 510/4268] avg loss 0.0149806, throughput 3.99427K wps
[Epoch 4 Batch 540/4268] avg loss 0.0144718, throughput 3.99539K wps
[Epoch 4 Batch 570/4268] avg loss 0.0143651, throughput 3.99324K wps
[Epoch 4 Batch 600/4268] avg loss 0.0140942, throughput 3.99828K wps
[Epoch 4 Batch 630/4268] avg loss 0.0145647, throughput 4.00146K wps
[Epoch 4 Batch 660/4268] avg loss 0.0151735, throughput 3.99722K wps
[Epoch 4 Batch 690/4268] avg loss 0.0140058, throughput 3.99632K wps
[Epoch 4 Batch 720/4268] avg loss 0.0151559, throughput 3.9957K wps
[Epoch 4 Batch 750/4268] avg loss 0.0153974, throughput 3.99607K wps
[Epoch 4 Batch 780/4268] avg loss 0.0147807, throughput 3.99694K wps
[Epoch 4 Batch 810/4268] avg loss 0.0150088, throughput 3.99751K wps
[Epoch 4 Batch 840/4268] avg loss 0.0144578, throughput 3.99308K wps
[Epoch 4 Batch 870/4268] avg loss 0.0145734, throughput 3.99262K wps
[Epoch 4 Batch 900/4268] avg loss 0.0153468, throughput 3.99637K wps
[Epoch 4 Batch 930/4268] avg loss 0.0144472, throughput 3.9935K wps
[Epoch 4 Batch 960/4268] avg loss 0.0144786, throughput 3.99708K wps
[Epoch 4 Batch 990/4268] avg loss 0.0149159, throughput 3.99449K wps
[Epoch 4 Batch 1020/4268] avg loss 0.0151143, throughput 3.99659K wps
[Epoch 4 Batch 1050/4268] avg loss 0.0145081, throughput 3.99313K wps
[Epoch 4 Batch 1080/4268] avg loss 0.0136854, throughput 3.99821K wps
[Epoch 4 Batch 1110/4268] avg loss 0.0146998, throughput 3.99749K wps
[Epoch 4 Batch 1140/4268] avg loss 0.0153797, throughput 3.99866K wps
[Epoch 4 Batch 1170/4268] avg loss 0.014176, throughput 3.99664K wps
[Epoch 4 Batch 1200/4268] avg loss 0.0147649, throughput 3.99362K wps
[Epoch 4 Batch 1230/4268] avg loss 0.0142245, throughput 3.99025K wps
[Epoch 4 Batch 1260/4268] avg loss 0.0145298, throughput 3.99455K wps
[Epoch 4 Batch 1290/4268] avg loss 0.015045, throughput 3.99257K wps
[Epoch 4 Batch 1320/4268] avg loss 0.0144808, throughput 3.99861K wps
[Epoch 4 Batch 1350/4268] avg loss 0.0152659, throughput 3.99765K wps
[Epoch 4 Batch 1380/4268] avg loss 0.0143544, throughput 3.99232K wps
[Epoch 4 Batch 1410/4268] avg loss 0.0147027, throughput 3.99039K wps
[Epoch 4 Batch 1440/4268] avg loss 0.0147175, throughput 3.99333K wps
[Epoch 4 Batch 1470/4268] avg loss 0.0144855, throughput 3.99255K wps
[Epoch 4 Batch 1500/4268] avg loss 0.0141232, throughput 3.99054K wps
[Epoch 4 Batch 1530/4268] avg loss 0.0138193, throughput 3.99486K wps
[Epoch 4 Batch 1560/4268] avg loss 0.0146553, throughput 3.99439K wps
[Epoch 4 Batch 1590/4268] avg loss 0.0145888, throughput 3.99753K wps
[Epoch 4 Batch 1620/4268] avg loss 0.0150388, throughput 3.99038K wps
[Epoch 4 Batch 1650/4268] avg loss 0.0148683, throughput 3.99684K wps
[Epoch 4 Batch 1680/4268] avg loss 0.0139725, throughput 3.99511K wps
[Epoch 4 Batch 1710/4268] avg loss 0.0152613, throughput 3.99548K wps
[Epoch 4 Batch 1740/4268] avg loss 0.014771, throughput 3.99075K wps
[Epoch 4 Batch 1770/4268] avg loss 0.0140132, throughput 3.99972K wps
[Epoch 4 Batch 1800/4268] avg loss 0.0148667, throughput 3.99293K wps
[Epoch 4 Batch 1830/4268] avg loss 0.0145144, throughput 3.99673K wps
[Epoch 4 Batch 1860/4268] avg loss 0.0147884, throughput 3.99645K wps
[Epoch 4 Batch 1890/4268] avg loss 0.0146939, throughput 3.99446K wps
[Epoch 4 Batch 1920/4268] avg loss 0.0149302, throughput 3.99368K wps
[Epoch 4 Batch 1950/4268] avg loss 0.0149845, throughput 3.99397K wps
[Epoch 4 Batch 1980/4268] avg loss 0.0146147, throughput 3.99335K wps
[Epoch 4 Batch 2010/4268] avg loss 0.0146332, throughput 3.99675K wps
[Epoch 4 Batch 2040/4268] avg loss 0.0151368, throughput 3.99922K wps
[Epoch 4 Batch 2070/4268] avg loss 0.015167, throughput 3.99847K wps
[Epoch 4 Batch 2100/4268] avg loss 0.0149427, throughput 3.99633K wps
[Epoch 4 Batch 2130/4268] avg loss 0.0147354, throughput 3.99461K wps
[Epoch 4 Batch 2160/4268] avg loss 0.0142974, throughput 3.99927K wps
[Epoch 4 Batch 2190/4268] avg loss 0.0144031, throughput 3.99802K wps
[Epoch 4 Batch 2220/4268] avg loss 0.0150246, throughput 3.99512K wps
[Epoch 4 Batch 2250/4268] avg loss 0.0142309, throughput 3.99657K wps
[Epoch 4 Batch 2280/4268] avg loss 0.0147124, throughput 3.99549K wps
[Epoch 4 Batch 2310/4268] avg loss 0.0153694, throughput 3.99412K wps
[Epoch 4 Batch 2340/4268] avg loss 0.0146041, throughput 3.99529K wps
[Epoch 4 Batch 2370/4268] avg loss 0.0145502, throughput 3.99618K wps
[Epoch 4 Batch 2400/4268] avg loss 0.0147134, throughput 3.99503K wps
[Epoch 4 Batch 2430/4268] avg loss 0.0148961, throughput 4.00193K wps
[Epoch 4 Batch 2460/4268] avg loss 0.0146537, throughput 3.99256K wps
[Epoch 4 Batch 2490/4268] avg loss 0.0151822, throughput 3.99288K wps
[Epoch 4 Batch 2520/4268] avg loss 0.0156139, throughput 3.99456K wps
[Epoch 4 Batch 2550/4268] avg loss 0.014769, throughput 3.99691K wps
[Epoch 4 Batch 2580/4268] avg loss 0.0145301, throughput 3.99236K wps
[Epoch 4 Batch 2610/4268] avg loss 0.0145797, throughput 3.9947K wps
[Epoch 4 Batch 2640/4268] avg loss 0.0143914, throughput 3.992K wps
[Epoch 4 Batch 2670/4268] avg loss 0.0148456, throughput 3.99267K wps
[Epoch 4 Batch 2700/4268] avg loss 0.0150778, throughput 3.99673K wps
[Epoch 4 Batch 2730/4268] avg loss 0.0149476, throughput 3.99778K wps
[Epoch 4 Batch 2760/4268] avg loss 0.0143963, throughput 3.99268K wps
[Epoch 4 Batch 2790/4268] avg loss 0.0153933, throughput 3.99937K wps
[Epoch 4 Batch 2820/4268] avg loss 0.0151293, throughput 3.99668K wps
[Epoch 4 Batch 2850/4268] avg loss 0.0155387, throughput 3.99815K wps
[Epoch 4 Batch 2880/4268] avg loss 0.0149335, throughput 3.99829K wps
[Epoch 4 Batch 2910/4268] avg loss 0.0147753, throughput 3.99774K wps
[Epoch 4 Batch 2940/4268] avg loss 0.0153132, throughput 3.99509K wps
[Epoch 4 Batch 2970/4268] avg loss 0.0150638, throughput 3.99661K wps
[Epoch 4 Batch 3000/4268] avg loss 0.0137953, throughput 3.99188K wps
[Epoch 4 Batch 3030/4268] avg loss 0.0150508, throughput 3.99302K wps
[Epoch 4 Batch 3060/4268] avg loss 0.0153714, throughput 3.9914K wps
[Epoch 4 Batch 3090/4268] avg loss 0.0147712, throughput 3.99318K wps
[Epoch 4 Batch 3120/4268] avg loss 0.015233, throughput 3.99684K wps
[Epoch 4 Batch 3150/4268] avg loss 0.0145596, throughput 3.9939K wps
[Epoch 4 Batch 3180/4268] avg loss 0.0148979, throughput 3.99464K wps
[Epoch 4 Batch 3210/4268] avg loss 0.0142263, throughput 3.98988K wps
[Epoch 4 Batch 3240/4268] avg loss 0.0145425, throughput 3.99522K wps
[Epoch 4 Batch 3270/4268] avg loss 0.0147337, throughput 3.99956K wps
[Epoch 4 Batch 3300/4268] avg loss 0.0156344, throughput 3.997K wps
[Epoch 4 Batch 3330/4268] avg loss 0.0149874, throughput 3.99225K wps
[Epoch 4 Batch 3360/4268] avg loss 0.0153801, throughput 3.9868K wps
[Epoch 4 Batch 3390/4268] avg loss 0.0144288, throughput 3.99206K wps
[Epoch 4 Batch 3420/4268] avg loss 0.0145819, throughput 3.99229K wps
[Epoch 4 Batch 3450/4268] avg loss 0.0152806, throughput 3.99672K wps
[Epoch 4 Batch 3480/4268] avg loss 0.0144588, throughput 3.99545K wps
[Epoch 4 Batch 3510/4268] avg loss 0.0142753, throughput 3.99533K wps
[Epoch 4 Batch 3540/4268] avg loss 0.0146801, throughput 3.99397K wps
[Epoch 4 Batch 3570/4268] avg loss 0.0146263, throughput 3.998K wps
[Epoch 4 Batch 3600/4268] avg loss 0.0154021, throughput 3.99967K wps
[Epoch 4 Batch 3630/4268] avg loss 0.0157417, throughput 3.99752K wps
[Epoch 4 Batch 3660/4268] avg loss 0.0150747, throughput 4.00097K wps
[Epoch 4 Batch 3690/4268] avg loss 0.0150221, throughput 3.99753K wps
[Epoch 4 Batch 3720/4268] avg loss 0.0153862, throughput 3.99337K wps
[Epoch 4 Batch 3750/4268] avg loss 0.0148428, throughput 3.98944K wps
[Epoch 4 Batch 3780/4268] avg loss 0.0147747, throughput 3.98624K wps
[Epoch 4 Batch 3810/4268] avg loss 0.0150211, throughput 3.99174K wps
[Epoch 4 Batch 3840/4268] avg loss 0.0144941, throughput 3.99554K wps
[Epoch 4 Batch 3870/4268] avg loss 0.0152685, throughput 3.99766K wps
[Epoch 4 Batch 3900/4268] avg loss 0.0147589, throughput 3.99486K wps
[Epoch 4 Batch 3930/4268] avg loss 0.0153842, throughput 3.9948K wps
[Epoch 4 Batch 3960/4268] avg loss 0.014591, throughput 3.99223K wps
[Epoch 4 Batch 3990/4268] avg loss 0.0153732, throughput 3.99561K wps
[Epoch 4 Batch 4020/4268] avg loss 0.0144986, throughput 3.99546K wps
[Epoch 4 Batch 4050/4268] avg loss 0.0152262, throughput 3.99228K wps
[Epoch 4 Batch 4080/4268] avg loss 0.0153326, throughput 4.0001K wps
[Epoch 4 Batch 4110/4268] avg loss 0.0153506, throughput 3.99479K wps
[Epoch 4 Batch 4140/4268] avg loss 0.0152999, throughput 3.98963K wps
[Epoch 4 Batch 4170/4268] avg loss 0.014766, throughput 3.99519K wps
[Epoch 4 Batch 4200/4268] avg loss 0.0151646, throughput 3.99856K wps
[Epoch 4 Batch 4230/4268] avg loss 0.0152095, throughput 3.99617K wps
[Epoch 4 Batch 4260/4268] avg loss 0.0148187, throughput 3.99738K wps
Begin Testing...
[Batch 30/475] elapsed 0.47 s
[Batch 60/475] elapsed 0.43 s
[Batch 90/475] elapsed 0.43 s
[Batch 120/475] elapsed 0.43 s
[Batch 150/475] elapsed 0.43 s
[Batch 180/475] elapsed 0.43 s
[Batch 210/475] elapsed 0.43 s
[Batch 240/475] elapsed 0.43 s
[Batch 270/475] elapsed 0.43 s
[Batch 300/475] elapsed 0.43 s
[Batch 330/475] elapsed 0.43 s
[Batch 360/475] elapsed 0.43 s
[Batch 390/475] elapsed 0.43 s
[Batch 420/475] elapsed 0.43 s
[Batch 450/475] elapsed 0.43 s
[Epoch 4] train avg loss 0.0147924, test acc 0.6766, test avg loss 0.792526, throughput 3.99565K wps
Observed Improvement.
Begin Testing...
[Batch 30/43] elapsed 0.43 s
[Epoch 5 Batch 30/4268] avg loss 0.0146083, throughput 4.07736K wps
[Epoch 5 Batch 60/4268] avg loss 0.0134817, throughput 3.99177K wps
[Epoch 5 Batch 90/4268] avg loss 0.0137027, throughput 3.9933K wps
[Epoch 5 Batch 120/4268] avg loss 0.0137037, throughput 3.99192K wps
[Epoch 5 Batch 150/4268] avg loss 0.0137751, throughput 3.99425K wps
[Epoch 5 Batch 180/4268] avg loss 0.014499, throughput 3.99455K wps
[Epoch 5 Batch 210/4268] avg loss 0.0134762, throughput 4.00048K wps
[Epoch 5 Batch 240/4268] avg loss 0.0139084, throughput 3.99013K wps
[Epoch 5 Batch 270/4268] avg loss 0.0139312, throughput 3.99391K wps
[Epoch 5 Batch 300/4268] avg loss 0.0145907, throughput 3.9938K wps
[Epoch 5 Batch 330/4268] avg loss 0.014179, throughput 3.99681K wps
[Epoch 5 Batch 360/4268] avg loss 0.013859, throughput 3.99551K wps
[Epoch 5 Batch 390/4268] avg loss 0.0143427, throughput 3.99578K wps
[Epoch 5 Batch 420/4268] avg loss 0.0147008, throughput 3.99124K wps
[Epoch 5 Batch 450/4268] avg loss 0.014103, throughput 3.9935K wps
[Epoch 5 Batch 480/4268] avg loss 0.0139522, throughput 3.99677K wps
[Epoch 5 Batch 510/4268] avg loss 0.0143203, throughput 3.99507K wps
[Epoch 5 Batch 540/4268] avg loss 0.0144614, throughput 3.99344K wps
[Epoch 5 Batch 570/4268] avg loss 0.0139805, throughput 3.99306K wps
[Epoch 5 Batch 600/4268] avg loss 0.0143428, throughput 3.99435K wps
[Epoch 5 Batch 630/4268] avg loss 0.0137798, throughput 3.99464K wps
[Epoch 5 Batch 660/4268] avg loss 0.0138821, throughput 4.00024K wps
[Epoch 5 Batch 690/4268] avg loss 0.0142315, throughput 3.99436K wps
[Epoch 5 Batch 720/4268] avg loss 0.0140958, throughput 3.99464K wps
[Epoch 5 Batch 750/4268] avg loss 0.0142539, throughput 3.99382K wps
[Epoch 5 Batch 780/4268] avg loss 0.0143103, throughput 3.99412K wps
[Epoch 5 Batch 810/4268] avg loss 0.0141822, throughput 3.99636K wps
[Epoch 5 Batch 840/4268] avg loss 0.0140594, throughput 3.99702K wps
[Epoch 5 Batch 870/4268] avg loss 0.0142948, throughput 3.99632K wps
[Epoch 5 Batch 900/4268] avg loss 0.0143512, throughput 3.99634K wps
[Epoch 5 Batch 930/4268] avg loss 0.0140044, throughput 3.99552K wps
[Epoch 5 Batch 960/4268] avg loss 0.0138869, throughput 3.99701K wps
[Epoch 5 Batch 990/4268] avg loss 0.0145283, throughput 3.99352K wps
[Epoch 5 Batch 1020/4268] avg loss 0.0142872, throughput 3.99685K wps
[Epoch 5 Batch 1050/4268] avg loss 0.0142034, throughput 3.99676K wps
[Epoch 5 Batch 1080/4268] avg loss 0.014423, throughput 3.99094K wps
[Epoch 5 Batch 1110/4268] avg loss 0.0148146, throughput 3.99445K wps
[Epoch 5 Batch 1140/4268] avg loss 0.0147143, throughput 3.99589K wps
[Epoch 5 Batch 1170/4268] avg loss 0.0145345, throughput 3.99869K wps
[Epoch 5 Batch 1200/4268] avg loss 0.0138544, throughput 3.99625K wps
[Epoch 5 Batch 1230/4268] avg loss 0.0140211, throughput 4.00074K wps
[Epoch 5 Batch 1260/4268] avg loss 0.0143195, throughput 3.99722K wps
[Epoch 5 Batch 1290/4268] avg loss 0.014407, throughput 3.99591K wps
[Epoch 5 Batch 1320/4268] avg loss 0.0141178, throughput 3.99874K wps
[Epoch 5 Batch 1350/4268] avg loss 0.0142552, throughput 3.99882K wps
[Epoch 5 Batch 1380/4268] avg loss 0.0141115, throughput 3.99444K wps
[Epoch 5 Batch 1410/4268] avg loss 0.0142606, throughput 3.9965K wps
[Epoch 5 Batch 1440/4268] avg loss 0.0137411, throughput 3.99633K wps
[Epoch 5 Batch 1470/4268] avg loss 0.0145011, throughput 3.9994K wps
[Epoch 5 Batch 1500/4268] avg loss 0.0145123, throughput 3.99794K wps
[Epoch 5 Batch 1530/4268] avg loss 0.0139625, throughput 3.99936K wps
[Epoch 5 Batch 1560/4268] avg loss 0.0138586, throughput 3.99794K wps
[Epoch 5 Batch 1590/4268] avg loss 0.014991, throughput 3.99743K wps
[Epoch 5 Batch 1620/4268] avg loss 0.0150048, throughput 4.00032K wps
[Epoch 5 Batch 1650/4268] avg loss 0.0145999, throughput 3.99662K wps
[Epoch 5 Batch 1680/4268] avg loss 0.0145717, throughput 4.00045K wps
[Epoch 5 Batch 1710/4268] avg loss 0.0143963, throughput 3.99566K wps
[Epoch 5 Batch 1740/4268] avg loss 0.0142019, throughput 4.0018K wps
[Epoch 5 Batch 1770/4268] avg loss 0.0146255, throughput 3.99885K wps
[Epoch 5 Batch 1800/4268] avg loss 0.0143001, throughput 3.99548K wps
[Epoch 5 Batch 1830/4268] avg loss 0.0142085, throughput 3.99547K wps
[Epoch 5 Batch 1860/4268] avg loss 0.0138208, throughput 3.99569K wps
[Epoch 5 Batch 1890/4268] avg loss 0.0146114, throughput 3.99664K wps
[Epoch 5 Batch 1920/4268] avg loss 0.0146368, throughput 3.99099K wps
[Epoch 5 Batch 1950/4268] avg loss 0.0144849, throughput 3.99656K wps
[Epoch 5 Batch 1980/4268] avg loss 0.015366, throughput 3.99681K wps
[Epoch 5 Batch 2010/4268] avg loss 0.0145578, throughput 3.99974K wps
[Epoch 5 Batch 2040/4268] avg loss 0.0143208, throughput 3.99368K wps
[Epoch 5 Batch 2070/4268] avg loss 0.013713, throughput 3.99741K wps
[Epoch 5 Batch 2100/4268] avg loss 0.0150038, throughput 3.9964K wps
[Epoch 5 Batch 2130/4268] avg loss 0.013778, throughput 3.99653K wps
[Epoch 5 Batch 2160/4268] avg loss 0.0141916, throughput 3.99531K wps
[Epoch 5 Batch 2190/4268] avg loss 0.0145734, throughput 3.99314K wps
[Epoch 5 Batch 2220/4268] avg loss 0.0140958, throughput 3.99446K wps
[Epoch 5 Batch 2250/4268] avg loss 0.0138695, throughput 3.99563K wps
[Epoch 5 Batch 2280/4268] avg loss 0.0146077, throughput 3.99872K wps
[Epoch 5 Batch 2310/4268] avg loss 0.0150432, throughput 4.0005K wps
[Epoch 5 Batch 2340/4268] avg loss 0.0136898, throughput 3.99554K wps
[Epoch 5 Batch 2370/4268] avg loss 0.0142583, throughput 3.99881K wps
[Epoch 5 Batch 2400/4268] avg loss 0.0142827, throughput 3.99127K wps
[Epoch 5 Batch 2430/4268] avg loss 0.0142631, throughput 3.99507K wps
[Epoch 5 Batch 2460/4268] avg loss 0.0138408, throughput 3.99966K wps
[Epoch 5 Batch 2490/4268] avg loss 0.0144235, throughput 3.99906K wps
[Epoch 5 Batch 2520/4268] avg loss 0.0144207, throughput 4.00027K wps
[Epoch 5 Batch 2550/4268] avg loss 0.0138678, throughput 3.99715K wps
[Epoch 5 Batch 2580/4268] avg loss 0.013935, throughput 3.99455K wps
[Epoch 5 Batch 2610/4268] avg loss 0.0145057, throughput 4.00063K wps
[Epoch 5 Batch 2640/4268] avg loss 0.0140202, throughput 3.99458K wps
[Epoch 5 Batch 2670/4268] avg loss 0.0141558, throughput 3.99671K wps
[Epoch 5 Batch 2700/4268] avg loss 0.0146026, throughput 4.00484K wps
[Epoch 5 Batch 2730/4268] avg loss 0.0141984, throughput 4.00141K wps
[Epoch 5 Batch 2760/4268] avg loss 0.014698, throughput 3.9914K wps
[Epoch 5 Batch 2790/4268] avg loss 0.0149114, throughput 3.99553K wps
[Epoch 5 Batch 2820/4268] avg loss 0.0136165, throughput 3.99626K wps
[Epoch 5 Batch 2850/4268] avg loss 0.0144639, throughput 3.9921K wps
[Epoch 5 Batch 2880/4268] avg loss 0.0140364, throughput 3.99755K wps
[Epoch 5 Batch 2910/4268] avg loss 0.0145588, throughput 3.998K wps
[Epoch 5 Batch 2940/4268] avg loss 0.0148514, throughput 3.99464K wps
[Epoch 5 Batch 2970/4268] avg loss 0.0144159, throughput 3.984K wps
[Epoch 5 Batch 3000/4268] avg loss 0.0147083, throughput 3.99272K wps
[Epoch 5 Batch 3030/4268] avg loss 0.0142666, throughput 3.99462K wps
[Epoch 5 Batch 3060/4268] avg loss 0.0144089, throughput 3.99288K wps
[Epoch 5 Batch 3090/4268] avg loss 0.0149315, throughput 3.99558K wps
[Epoch 5 Batch 3120/4268] avg loss 0.0145915, throughput 3.99603K wps
[Epoch 5 Batch 3150/4268] avg loss 0.014088, throughput 3.99519K wps
[Epoch 5 Batch 3180/4268] avg loss 0.0145855, throughput 3.9941K wps
[Epoch 5 Batch 3210/4268] avg loss 0.0147308, throughput 3.99406K wps
[Epoch 5 Batch 3240/4268] avg loss 0.0147586, throughput 3.99699K wps
[Epoch 5 Batch 3270/4268] avg loss 0.0148013, throughput 4.00664K wps
[Epoch 5 Batch 3300/4268] avg loss 0.0146878, throughput 4.00191K wps
[Epoch 5 Batch 3330/4268] avg loss 0.0141403, throughput 4.00418K wps
[Epoch 5 Batch 3360/4268] avg loss 0.0145649, throughput 3.99285K wps
[Epoch 5 Batch 3390/4268] avg loss 0.0138604, throughput 3.99746K wps
[Epoch 5 Batch 3420/4268] avg loss 0.0143141, throughput 3.9966K wps
[Epoch 5 Batch 3450/4268] avg loss 0.0139438, throughput 3.99631K wps
[Epoch 5 Batch 3480/4268] avg loss 0.0145049, throughput 3.99578K wps
[Epoch 5 Batch 3510/4268] avg loss 0.0146746, throughput 3.99465K wps
[Epoch 5 Batch 3540/4268] avg loss 0.0141507, throughput 3.99957K wps
[Epoch 5 Batch 3570/4268] avg loss 0.014587, throughput 3.99575K wps
[Epoch 5 Batch 3600/4268] avg loss 0.0143474, throughput 4.00022K wps
[Epoch 5 Batch 3630/4268] avg loss 0.0141308, throughput 3.99662K wps
[Epoch 5 Batch 3660/4268] avg loss 0.014101, throughput 3.99374K wps
[Epoch 5 Batch 3690/4268] avg loss 0.0143131, throughput 3.99159K wps
[Epoch 5 Batch 3720/4268] avg loss 0.0145463, throughput 3.9949K wps
[Epoch 5 Batch 3750/4268] avg loss 0.0144442, throughput 3.99255K wps
[Epoch 5 Batch 3780/4268] avg loss 0.0149337, throughput 3.99571K wps
[Epoch 5 Batch 3810/4268] avg loss 0.0144962, throughput 3.99043K wps
[Epoch 5 Batch 3840/4268] avg loss 0.0138181, throughput 3.99414K wps
[Epoch 5 Batch 3870/4268] avg loss 0.0147111, throughput 3.99749K wps
[Epoch 5 Batch 3900/4268] avg loss 0.0134119, throughput 4.00256K wps
[Epoch 5 Batch 3930/4268] avg loss 0.0145511, throughput 4.00469K wps
[Epoch 5 Batch 3960/4268] avg loss 0.0149792, throughput 4.00175K wps
[Epoch 5 Batch 3990/4268] avg loss 0.0149511, throughput 4.0063K wps
[Epoch 5 Batch 4020/4268] avg loss 0.0145881, throughput 4.00255K wps
[Epoch 5 Batch 4050/4268] avg loss 0.0142529, throughput 3.99508K wps
[Epoch 5 Batch 4080/4268] avg loss 0.0143181, throughput 3.99342K wps
[Epoch 5 Batch 4110/4268] avg loss 0.0146611, throughput 3.99083K wps
[Epoch 5 Batch 4140/4268] avg loss 0.014327, throughput 3.99637K wps
[Epoch 5 Batch 4170/4268] avg loss 0.0148083, throughput 3.9898K wps
[Epoch 5 Batch 4200/4268] avg loss 0.0143658, throughput 3.9931K wps
[Epoch 5 Batch 4230/4268] avg loss 0.0141072, throughput 3.99212K wps
[Epoch 5 Batch 4260/4268] avg loss 0.0145474, throughput 3.99834K wps
Begin Testing...
[Batch 30/475] elapsed 0.46 s
[Batch 60/475] elapsed 0.43 s
[Batch 90/475] elapsed 0.43 s
[Batch 120/475] elapsed 0.43 s
[Batch 150/475] elapsed 0.43 s
[Batch 180/475] elapsed 0.43 s
[Batch 210/475] elapsed 0.43 s
[Batch 240/475] elapsed 0.43 s
[Batch 270/475] elapsed 0.43 s
[Batch 300/475] elapsed 0.43 s
[Batch 330/475] elapsed 0.43 s
[Batch 360/475] elapsed 0.43 s
[Batch 390/475] elapsed 0.43 s
[Batch 420/475] elapsed 0.43 s
[Batch 450/475] elapsed 0.43 s
[Epoch 5] train avg loss 0.0143192, test acc 0.6753, test avg loss 0.790622, throughput 3.99673K wps
[Epoch 6 Batch 30/4268] avg loss 0.0137334, throughput 4.07742K wps
[Epoch 6 Batch 60/4268] avg loss 0.0138199, throughput 3.99401K wps
[Epoch 6 Batch 90/4268] avg loss 0.0139144, throughput 3.99669K wps
[Epoch 6 Batch 120/4268] avg loss 0.013213, throughput 3.99246K wps
[Epoch 6 Batch 150/4268] avg loss 0.0139379, throughput 3.99507K wps
[Epoch 6 Batch 180/4268] avg loss 0.0134963, throughput 3.99469K wps
[Epoch 6 Batch 210/4268] avg loss 0.0132736, throughput 4.00167K wps
[Epoch 6 Batch 240/4268] avg loss 0.0137497, throughput 3.99791K wps
[Epoch 6 Batch 270/4268] avg loss 0.0141524, throughput 3.99397K wps
[Epoch 6 Batch 300/4268] avg loss 0.0136515, throughput 3.99702K wps
[Epoch 6 Batch 330/4268] avg loss 0.0132596, throughput 3.99807K wps
[Epoch 6 Batch 360/4268] avg loss 0.0132509, throughput 3.99598K wps
[Epoch 6 Batch 390/4268] avg loss 0.0144446, throughput 3.99617K wps
[Epoch 6 Batch 420/4268] avg loss 0.0137831, throughput 3.99923K wps
[Epoch 6 Batch 450/4268] avg loss 0.014065, throughput 3.99847K wps
[Epoch 6 Batch 480/4268] avg loss 0.0138081, throughput 3.99905K wps
[Epoch 6 Batch 510/4268] avg loss 0.0145227, throughput 3.99352K wps
[Epoch 6 Batch 540/4268] avg loss 0.0140512, throughput 4.00071K wps
[Epoch 6 Batch 570/4268] avg loss 0.0135175, throughput 3.99516K wps
[Epoch 6 Batch 600/4268] avg loss 0.0138297, throughput 3.99691K wps
[Epoch 6 Batch 630/4268] avg loss 0.0140131, throughput 3.99388K wps
[Epoch 6 Batch 660/4268] avg loss 0.0137426, throughput 3.99268K wps
[Epoch 6 Batch 690/4268] avg loss 0.0132208, throughput 3.99448K wps
[Epoch 6 Batch 720/4268] avg loss 0.0134827, throughput 3.99592K wps
[Epoch 6 Batch 750/4268] avg loss 0.0132359, throughput 3.99193K wps
[Epoch 6 Batch 780/4268] avg loss 0.0130324, throughput 3.99547K wps
[Epoch 6 Batch 810/4268] avg loss 0.0133163, throughput 3.99342K wps
[Epoch 6 Batch 840/4268] avg loss 0.0134923, throughput 3.99722K wps
[Epoch 6 Batch 870/4268] avg loss 0.013884, throughput 3.99899K wps
[Epoch 6 Batch 900/4268] avg loss 0.0139796, throughput 3.99555K wps
[Epoch 6 Batch 930/4268] avg loss 0.013549, throughput 3.99543K wps
[Epoch 6 Batch 960/4268] avg loss 0.0137765, throughput 3.99693K wps
[Epoch 6 Batch 990/4268] avg loss 0.0138295, throughput 3.99638K wps
[Epoch 6 Batch 1020/4268] avg loss 0.0138369, throughput 3.99249K wps
[Epoch 6 Batch 1050/4268] avg loss 0.0138978, throughput 3.99625K wps
[Epoch 6 Batch 1080/4268] avg loss 0.0138565, throughput 3.99554K wps
[Epoch 6 Batch 1110/4268] avg loss 0.0134879, throughput 3.99366K wps
[Epoch 6 Batch 1140/4268] avg loss 0.0140172, throughput 3.99562K wps
[Epoch 6 Batch 1170/4268] avg loss 0.0137205, throughput 3.99556K wps
[Epoch 6 Batch 1200/4268] avg loss 0.0134083, throughput 3.99253K wps
[Epoch 6 Batch 1230/4268] avg loss 0.0140113, throughput 3.99336K wps
[Epoch 6 Batch 1260/4268] avg loss 0.0143545, throughput 4.00041K wps
[Epoch 6 Batch 1290/4268] avg loss 0.0137026, throughput 4.00159K wps
[Epoch 6 Batch 1320/4268] avg loss 0.0144217, throughput 3.99681K wps
[Epoch 6 Batch 1350/4268] avg loss 0.0139697, throughput 3.99355K wps
[Epoch 6 Batch 1380/4268] avg loss 0.0138513, throughput 3.99914K wps
[Epoch 6 Batch 1410/4268] avg loss 0.0136747, throughput 3.99523K wps
[Epoch 6 Batch 1440/4268] avg loss 0.014209, throughput 3.99516K wps
[Epoch 6 Batch 1470/4268] avg loss 0.0139969, throughput 3.99719K wps
[Epoch 6 Batch 1500/4268] avg loss 0.0139369, throughput 3.99503K wps
[Epoch 6 Batch 1530/4268] avg loss 0.0142314, throughput 3.99521K wps
[Epoch 6 Batch 1560/4268] avg loss 0.0141409, throughput 3.99866K wps
[Epoch 6 Batch 1590/4268] avg loss 0.0137897, throughput 3.99808K wps
[Epoch 6 Batch 1620/4268] avg loss 0.0143064, throughput 3.99509K wps
[Epoch 6 Batch 1650/4268] avg loss 0.0137631, throughput 3.99388K wps
[Epoch 6 Batch 1680/4268] avg loss 0.0138113, throughput 3.99238K wps
[Epoch 6 Batch 1710/4268] avg loss 0.0141229, throughput 3.99522K wps
[Epoch 6 Batch 1740/4268] avg loss 0.014454, throughput 3.99667K wps
[Epoch 6 Batch 1770/4268] avg loss 0.0138287, throughput 4.00097K wps
[Epoch 6 Batch 1800/4268] avg loss 0.013254, throughput 3.9977K wps
[Epoch 6 Batch 1830/4268] avg loss 0.0138155, throughput 4.00057K wps
[Epoch 6 Batch 1860/4268] avg loss 0.0141025, throughput 3.9936K wps
[Epoch 6 Batch 1890/4268] avg loss 0.0125322, throughput 3.99597K wps
[Epoch 6 Batch 1920/4268] avg loss 0.0129311, throughput 3.99618K wps
[Epoch 6 Batch 1950/4268] avg loss 0.0142569, throughput 3.98936K wps
[Epoch 6 Batch 1980/4268] avg loss 0.0137458, throughput 3.99341K wps
[Epoch 6 Batch 2010/4268] avg loss 0.013692, throughput 3.99723K wps
[Epoch 6 Batch 2040/4268] avg loss 0.0134205, throughput 3.99848K wps
[Epoch 6 Batch 2070/4268] avg loss 0.0136817, throughput 3.99487K wps
[Epoch 6 Batch 2100/4268] avg loss 0.0140855, throughput 3.99746K wps
[Epoch 6 Batch 2130/4268] avg loss 0.0142239, throughput 3.99194K wps
[Epoch 6 Batch 2160/4268] avg loss 0.0142763, throughput 3.98675K wps
[Epoch 6 Batch 2190/4268] avg loss 0.0137808, throughput 4.00015K wps
[Epoch 6 Batch 2220/4268] avg loss 0.0142551, throughput 3.99652K wps
[Epoch 6 Batch 2250/4268] avg loss 0.014247, throughput 4.00056K wps
[Epoch 6 Batch 2280/4268] avg loss 0.0141497, throughput 3.999K wps
[Epoch 6 Batch 2310/4268] avg loss 0.0142502, throughput 3.99709K wps
[Epoch 6 Batch 2340/4268] avg loss 0.0138665, throughput 3.99815K wps
[Epoch 6 Batch 2370/4268] avg loss 0.0139152, throughput 3.99558K wps
[Epoch 6 Batch 2400/4268] avg loss 0.0137723, throughput 3.99534K wps
[Epoch 6 Batch 2430/4268] avg loss 0.0141217, throughput 3.995K wps
[Epoch 6 Batch 2460/4268] avg loss 0.0137416, throughput 3.99583K wps
[Epoch 6 Batch 2490/4268] avg loss 0.0140337, throughput 3.99677K wps
[Epoch 6 Batch 2520/4268] avg loss 0.0137413, throughput 3.99842K wps
[Epoch 6 Batch 2550/4268] avg loss 0.013909, throughput 4.00035K wps
[Epoch 6 Batch 2580/4268] avg loss 0.0144971, throughput 3.99167K wps
[Epoch 6 Batch 2610/4268] avg loss 0.0136666, throughput 3.99133K wps
[Epoch 6 Batch 2640/4268] avg loss 0.0143886, throughput 3.98969K wps
[Epoch 6 Batch 2670/4268] avg loss 0.0134894, throughput 3.99679K wps
[Epoch 6 Batch 2700/4268] avg loss 0.0135789, throughput 3.99252K wps
[Epoch 6 Batch 2730/4268] avg loss 0.0137533, throughput 3.99403K wps
[Epoch 6 Batch 2760/4268] avg loss 0.0137145, throughput 3.99488K wps
[Epoch 6 Batch 2790/4268] avg loss 0.0142281, throughput 3.99407K wps
[Epoch 6 Batch 2820/4268] avg loss 0.0137636, throughput 3.99691K wps
[Epoch 6 Batch 2850/4268] avg loss 0.0136531, throughput 3.99525K wps
[Epoch 6 Batch 2880/4268] avg loss 0.0134786, throughput 3.99795K wps
[Epoch 6 Batch 2910/4268] avg loss 0.0134108, throughput 3.99562K wps
[Epoch 6 Batch 2940/4268] avg loss 0.0145646, throughput 3.99655K wps
[Epoch 6 Batch 2970/4268] avg loss 0.0144088, throughput 3.99428K wps
[Epoch 6 Batch 3000/4268] avg loss 0.0135304, throughput 3.99439K wps
[Epoch 6 Batch 3030/4268] avg loss 0.0147995, throughput 3.9901K wps
[Epoch 6 Batch 3060/4268] avg loss 0.0141241, throughput 3.99525K wps
[Epoch 6 Batch 3090/4268] avg loss 0.0137915, throughput 3.99366K wps
[Epoch 6 Batch 3120/4268] avg loss 0.0138645, throughput 3.99575K wps
[Epoch 6 Batch 3150/4268] avg loss 0.0139379, throughput 3.99746K wps
[Epoch 6 Batch 3180/4268] avg loss 0.0140354, throughput 3.99903K wps
[Epoch 6 Batch 3210/4268] avg loss 0.0141859, throughput 3.99743K wps
[Epoch 6 Batch 3240/4268] avg loss 0.0140436, throughput 3.99547K wps
[Epoch 6 Batch 3270/4268] avg loss 0.0142786, throughput 3.99547K wps
[Epoch 6 Batch 3300/4268] avg loss 0.0143681, throughput 3.99686K wps
[Epoch 6 Batch 3330/4268] avg loss 0.0138302, throughput 3.99536K wps
[Epoch 6 Batch 3360/4268] avg loss 0.0137366, throughput 4.00055K wps
[Epoch 6 Batch 3390/4268] avg loss 0.0138241, throughput 3.99134K wps
[Epoch 6 Batch 3420/4268] avg loss 0.0134218, throughput 3.98964K wps
[Epoch 6 Batch 3450/4268] avg loss 0.0147688, throughput 3.99526K wps
[Epoch 6 Batch 3480/4268] avg loss 0.0139433, throughput 3.98889K wps
[Epoch 6 Batch 3510/4268] avg loss 0.0135802, throughput 3.99289K wps
[Epoch 6 Batch 3540/4268] avg loss 0.0142557, throughput 3.99393K wps
[Epoch 6 Batch 3570/4268] avg loss 0.0141576, throughput 3.99509K wps
[Epoch 6 Batch 3600/4268] avg loss 0.0142775, throughput 3.99444K wps
[Epoch 6 Batch 3630/4268] avg loss 0.0140222, throughput 3.99625K wps
[Epoch 6 Batch 3660/4268] avg loss 0.014403, throughput 3.99683K wps
[Epoch 6 Batch 3690/4268] avg loss 0.014332, throughput 3.99699K wps
[Epoch 6 Batch 3720/4268] avg loss 0.0143668, throughput 3.99474K wps
[Epoch 6 Batch 3750/4268] avg loss 0.0144145, throughput 3.9977K wps
[Epoch 6 Batch 3780/4268] avg loss 0.0140634, throughput 3.99412K wps
[Epoch 6 Batch 3810/4268] avg loss 0.0142587, throughput 3.99117K wps
[Epoch 6 Batch 3840/4268] avg loss 0.0137139, throughput 4.00015K wps
[Epoch 6 Batch 3870/4268] avg loss 0.0131339, throughput 3.98781K wps
[Epoch 6 Batch 3900/4268] avg loss 0.0146959, throughput 3.99305K wps
[Epoch 6 Batch 3930/4268] avg loss 0.0140441, throughput 3.99579K wps
[Epoch 6 Batch 3960/4268] avg loss 0.0136252, throughput 3.99714K wps
[Epoch 6 Batch 3990/4268] avg loss 0.0145301, throughput 3.99148K wps
[Epoch 6 Batch 4020/4268] avg loss 0.0147354, throughput 3.99505K wps
[Epoch 6 Batch 4050/4268] avg loss 0.0142351, throughput 3.99223K wps
[Epoch 6 Batch 4080/4268] avg loss 0.0144668, throughput 3.9972K wps
[Epoch 6 Batch 4110/4268] avg loss 0.0138734, throughput 3.9933K wps
[Epoch 6 Batch 4140/4268] avg loss 0.01366, throughput 3.99723K wps
[Epoch 6 Batch 4170/4268] avg loss 0.0147273, throughput 3.99485K wps
[Epoch 6 Batch 4200/4268] avg loss 0.0144584, throughput 3.99325K wps
[Epoch 6 Batch 4230/4268] avg loss 0.0148114, throughput 3.99762K wps
[Epoch 6 Batch 4260/4268] avg loss 0.0142756, throughput 4.00008K wps
Begin Testing...
[Batch 30/475] elapsed 0.46 s
[Batch 60/475] elapsed 0.43 s
[Batch 90/475] elapsed 0.43 s
[Batch 120/475] elapsed 0.43 s
[Batch 150/475] elapsed 0.43 s
[Batch 180/475] elapsed 0.43 s
[Batch 210/475] elapsed 0.43 s
[Batch 240/475] elapsed 0.43 s
[Batch 270/475] elapsed 0.43 s
[Batch 300/475] elapsed 0.43 s
[Batch 330/475] elapsed 0.43 s
[Batch 360/475] elapsed 0.43 s
[Batch 390/475] elapsed 0.43 s
[Batch 420/475] elapsed 0.43 s
[Batch 450/475] elapsed 0.43 s
[Epoch 6] train avg loss 0.013923, test acc 0.6773, test avg loss 0.789684, throughput 3.99611K wps
Observed Improvement.
Begin Testing...
[Batch 30/43] elapsed 0.43 s
[Epoch 7 Batch 30/4268] avg loss 0.0126257, throughput 4.07816K wps
[Epoch 7 Batch 60/4268] avg loss 0.013253, throughput 3.99347K wps
[Epoch 7 Batch 90/4268] avg loss 0.0127538, throughput 3.99525K wps
[Epoch 7 Batch 120/4268] avg loss 0.0128709, throughput 3.99498K wps
[Epoch 7 Batch 150/4268] avg loss 0.013665, throughput 3.99055K wps
[Epoch 7 Batch 180/4268] avg loss 0.0136022, throughput 3.9939K wps
[Epoch 7 Batch 210/4268] avg loss 0.0126814, throughput 3.99419K wps
[Epoch 7 Batch 240/4268] avg loss 0.0140266, throughput 3.9934K wps
[Epoch 7 Batch 270/4268] avg loss 0.012899, throughput 3.99412K wps
[Epoch 7 Batch 300/4268] avg loss 0.0134628, throughput 3.99326K wps
[Epoch 7 Batch 330/4268] avg loss 0.0136375, throughput 3.99201K wps
[Epoch 7 Batch 360/4268] avg loss 0.0135435, throughput 3.9932K wps
[Epoch 7 Batch 390/4268] avg loss 0.0132147, throughput 3.99828K wps
[Epoch 7 Batch 420/4268] avg loss 0.0129464, throughput 3.99587K wps
[Epoch 7 Batch 450/4268] avg loss 0.0132824, throughput 3.99399K wps
[Epoch 7 Batch 480/4268] avg loss 0.0129522, throughput 3.99268K wps
[Epoch 7 Batch 510/4268] avg loss 0.013035, throughput 3.99803K wps
[Epoch 7 Batch 540/4268] avg loss 0.0143468, throughput 3.99295K wps
[Epoch 7 Batch 570/4268] avg loss 0.0131768, throughput 3.99738K wps
[Epoch 7 Batch 600/4268] avg loss 0.0137869, throughput 3.99432K wps
[Epoch 7 Batch 630/4268] avg loss 0.0129679, throughput 3.99513K wps
[Epoch 7 Batch 660/4268] avg loss 0.0137911, throughput 3.9935K wps
[Epoch 7 Batch 690/4268] avg loss 0.0130706, throughput 3.9929K wps
[Epoch 7 Batch 720/4268] avg loss 0.013669, throughput 3.99511K wps
[Epoch 7 Batch 750/4268] avg loss 0.0133994, throughput 3.99559K wps
[Epoch 7 Batch 780/4268] avg loss 0.0126038, throughput 3.99479K wps
[Epoch 7 Batch 810/4268] avg loss 0.0137521, throughput 3.99559K wps
[Epoch 7 Batch 840/4268] avg loss 0.013159, throughput 3.99708K wps
[Epoch 7 Batch 870/4268] avg loss 0.0132689, throughput 3.99273K wps
[Epoch 7 Batch 900/4268] avg loss 0.0134033, throughput 3.99643K wps
[Epoch 7 Batch 930/4268] avg loss 0.0135901, throughput 3.99496K wps
[Epoch 7 Batch 960/4268] avg loss 0.0126903, throughput 3.99473K wps
[Epoch 7 Batch 990/4268] avg loss 0.0136851, throughput 3.99275K wps
[Epoch 7 Batch 1020/4268] avg loss 0.0135852, throughput 3.99405K wps
[Epoch 7 Batch 1050/4268] avg loss 0.0125491, throughput 3.99757K wps
[Epoch 7 Batch 1080/4268] avg loss 0.0133634, throughput 3.9916K wps
[Epoch 7 Batch 1110/4268] avg loss 0.0139339, throughput 3.99843K wps
[Epoch 7 Batch 1140/4268] avg loss 0.0137394, throughput 3.99341K wps
[Epoch 7 Batch 1170/4268] avg loss 0.0135584, throughput 3.98967K wps
[Epoch 7 Batch 1200/4268] avg loss 0.0133683, throughput 3.99535K wps
[Epoch 7 Batch 1230/4268] avg loss 0.0138885, throughput 3.99514K wps
[Epoch 7 Batch 1260/4268] avg loss 0.0139725, throughput 3.99669K wps
[Epoch 7 Batch 1290/4268] avg loss 0.0137725, throughput 3.99324K wps
[Epoch 7 Batch 1320/4268] avg loss 0.013707, throughput 3.97483K wps
[Epoch 7 Batch 1350/4268] avg loss 0.012954, throughput 3.98312K wps
[Epoch 7 Batch 1380/4268] avg loss 0.0134878, throughput 3.99297K wps
[Epoch 7 Batch 1410/4268] avg loss 0.0128353, throughput 3.99538K wps
[Epoch 7 Batch 1440/4268] avg loss 0.0127091, throughput 3.99562K wps
[Epoch 7 Batch 1470/4268] avg loss 0.0132692, throughput 3.99648K wps
[Epoch 7 Batch 1500/4268] avg loss 0.013896, throughput 3.99649K wps
[Epoch 7 Batch 1530/4268] avg loss 0.0138046, throughput 4.00061K wps
[Epoch 7 Batch 1560/4268] avg loss 0.0138043, throughput 3.99649K wps
[Epoch 7 Batch 1590/4268] avg loss 0.0128794, throughput 4.00042K wps
[Epoch 7 Batch 1620/4268] avg loss 0.0137277, throughput 4.00188K wps
[Epoch 7 Batch 1650/4268] avg loss 0.0132773, throughput 3.99571K wps
[Epoch 7 Batch 1680/4268] avg loss 0.0141157, throughput 3.99695K wps
[Epoch 7 Batch 1710/4268] avg loss 0.0135097, throughput 3.99515K wps
[Epoch 7 Batch 1740/4268] avg loss 0.0130494, throughput 3.99531K wps
[Epoch 7 Batch 1770/4268] avg loss 0.0131968, throughput 3.99523K wps
[Epoch 7 Batch 1800/4268] avg loss 0.0135092, throughput 3.99503K wps
[Epoch 7 Batch 1830/4268] avg loss 0.0134072, throughput 3.99832K wps
[Epoch 7 Batch 1860/4268] avg loss 0.0131823, throughput 3.99766K wps
[Epoch 7 Batch 1890/4268] avg loss 0.0128395, throughput 3.99493K wps
[Epoch 7 Batch 1920/4268] avg loss 0.0139044, throughput 3.99347K wps
[Epoch 7 Batch 1950/4268] avg loss 0.013546, throughput 3.99738K wps
[Epoch 7 Batch 1980/4268] avg loss 0.0134528, throughput 3.99519K wps
[Epoch 7 Batch 2010/4268] avg loss 0.0130771, throughput 3.99678K wps
[Epoch 7 Batch 2040/4268] avg loss 0.0133957, throughput 3.99328K wps
[Epoch 7 Batch 2070/4268] avg loss 0.0140313, throughput 3.99597K wps
[Epoch 7 Batch 2100/4268] avg loss 0.0139404, throughput 3.99611K wps
[Epoch 7 Batch 2130/4268] avg loss 0.0137461, throughput 3.99397K wps
[Epoch 7 Batch 2160/4268] avg loss 0.0134336, throughput 3.99734K wps
[Epoch 7 Batch 2190/4268] avg loss 0.0137783, throughput 4.00071K wps
[Epoch 7 Batch 2220/4268] avg loss 0.0132966, throughput 4.00013K wps
[Epoch 7 Batch 2250/4268] avg loss 0.0132749, throughput 4.00064K wps
[Epoch 7 Batch 2280/4268] avg loss 0.0133058, throughput 3.99855K wps
[Epoch 7 Batch 2310/4268] avg loss 0.0146796, throughput 4.00776K wps
[Epoch 7 Batch 2340/4268] avg loss 0.0143432, throughput 4.01005K wps
[Epoch 7 Batch 2370/4268] avg loss 0.0143224, throughput 4.00265K wps
[Epoch 7 Batch 2400/4268] avg loss 0.0134866, throughput 3.99371K wps
[Epoch 7 Batch 2430/4268] avg loss 0.0135215, throughput 3.99475K wps
[Epoch 7 Batch 2460/4268] avg loss 0.0126783, throughput 3.99457K wps
[Epoch 7 Batch 2490/4268] avg loss 0.0135141, throughput 3.9972K wps
[Epoch 7 Batch 2520/4268] avg loss 0.0141917, throughput 3.99827K wps
[Epoch 7 Batch 2550/4268] avg loss 0.0138863, throughput 4.0016K wps
[Epoch 7 Batch 2580/4268] avg loss 0.0136954, throughput 3.9952K wps
[Epoch 7 Batch 2610/4268] avg loss 0.0135775, throughput 3.99666K wps
[Epoch 7 Batch 2640/4268] avg loss 0.0140802, throughput 3.99602K wps
[Epoch 7 Batch 2670/4268] avg loss 0.0133483, throughput 3.99757K wps
[Epoch 7 Batch 2700/4268] avg loss 0.0142317, throughput 3.99481K wps
[Epoch 7 Batch 2730/4268] avg loss 0.0140916, throughput 3.99638K wps
[Epoch 7 Batch 2760/4268] avg loss 0.0138879, throughput 4.00061K wps
[Epoch 7 Batch 2790/4268] avg loss 0.0137461, throughput 3.99833K wps
[Epoch 7 Batch 2820/4268] avg loss 0.0134391, throughput 3.99633K wps
[Epoch 7 Batch 2850/4268] avg loss 0.013963, throughput 3.99743K wps
[Epoch 7 Batch 2880/4268] avg loss 0.0127323, throughput 3.99823K wps
[Epoch 7 Batch 2910/4268] avg loss 0.0138485, throughput 3.99929K wps
[Epoch 7 Batch 2940/4268] avg loss 0.0139313, throughput 3.99885K wps
[Epoch 7 Batch 2970/4268] avg loss 0.0134365, throughput 3.99536K wps
[Epoch 7 Batch 3000/4268] avg loss 0.0142458, throughput 3.99148K wps
[Epoch 7 Batch 3030/4268] avg loss 0.0134023, throughput 3.99634K wps
[Epoch 7 Batch 3060/4268] avg loss 0.0132104, throughput 3.99647K wps
[Epoch 7 Batch 3090/4268] avg loss 0.0139117, throughput 3.9974K wps
[Epoch 7 Batch 3120/4268] avg loss 0.0137777, throughput 3.99713K wps
[Epoch 7 Batch 3150/4268] avg loss 0.0141175, throughput 3.99185K wps
[Epoch 7 Batch 3180/4268] avg loss 0.0136105, throughput 3.99397K wps
[Epoch 7 Batch 3210/4268] avg loss 0.0135371, throughput 3.9939K wps
[Epoch 7 Batch 3240/4268] avg loss 0.0136119, throughput 3.99583K wps
[Epoch 7 Batch 3270/4268] avg loss 0.0130144, throughput 3.9926K wps
[Epoch 7 Batch 3300/4268] avg loss 0.0128367, throughput 3.99744K wps
[Epoch 7 Batch 3330/4268] avg loss 0.0141562, throughput 3.99505K wps
[Epoch 7 Batch 3360/4268] avg loss 0.0139011, throughput 3.99476K wps
[Epoch 7 Batch 3390/4268] avg loss 0.0141423, throughput 3.99485K wps
[Epoch 7 Batch 3420/4268] avg loss 0.0138461, throughput 3.99446K wps
[Epoch 7 Batch 3450/4268] avg loss 0.0141277, throughput 3.995K wps
[Epoch 7 Batch 3480/4268] avg loss 0.0130726, throughput 3.9996K wps
[Epoch 7 Batch 3510/4268] avg loss 0.0134685, throughput 3.99495K wps
[Epoch 7 Batch 3540/4268] avg loss 0.0141079, throughput 3.99865K wps
[Epoch 7 Batch 3570/4268] avg loss 0.0140886, throughput 3.99613K wps
[Epoch 7 Batch 3600/4268] avg loss 0.0135674, throughput 3.99566K wps
[Epoch 7 Batch 3630/4268] avg loss 0.0137435, throughput 3.99294K wps
[Epoch 7 Batch 3660/4268] avg loss 0.0134974, throughput 3.99602K wps
[Epoch 7 Batch 3690/4268] avg loss 0.0130581, throughput 3.99607K wps
[Epoch 7 Batch 3720/4268] avg loss 0.0136463, throughput 3.99206K wps
[Epoch 7 Batch 3750/4268] avg loss 0.0139748, throughput 3.99719K wps
[Epoch 7 Batch 3780/4268] avg loss 0.0134877, throughput 3.99404K wps
[Epoch 7 Batch 3810/4268] avg loss 0.0136609, throughput 3.99331K wps
[Epoch 7 Batch 3840/4268] avg loss 0.0141366, throughput 3.99767K wps
[Epoch 7 Batch 3870/4268] avg loss 0.014406, throughput 4.00387K wps
[Epoch 7 Batch 3900/4268] avg loss 0.0140962, throughput 3.9994K wps
[Epoch 7 Batch 3930/4268] avg loss 0.0131979, throughput 3.99488K wps
[Epoch 7 Batch 3960/4268] avg loss 0.0137449, throughput 3.99431K wps
[Epoch 7 Batch 3990/4268] avg loss 0.012828, throughput 3.99928K wps
[Epoch 7 Batch 4020/4268] avg loss 0.0139068, throughput 3.99546K wps
[Epoch 7 Batch 4050/4268] avg loss 0.0139592, throughput 3.99013K wps
[Epoch 7 Batch 4080/4268] avg loss 0.0135277, throughput 3.99619K wps
[Epoch 7 Batch 4110/4268] avg loss 0.0132011, throughput 3.99248K wps
[Epoch 7 Batch 4140/4268] avg loss 0.0141574, throughput 3.99468K wps
[Epoch 7 Batch 4170/4268] avg loss 0.0139096, throughput 3.99524K wps
[Epoch 7 Batch 4200/4268] avg loss 0.0142201, throughput 3.99811K wps
[Epoch 7 Batch 4230/4268] avg loss 0.0142374, throughput 3.99306K wps
[Epoch 7 Batch 4260/4268] avg loss 0.0139277, throughput 3.99077K wps
Begin Testing...
[Batch 30/475] elapsed 0.46 s
[Batch 60/475] elapsed 0.43 s
[Batch 90/475] elapsed 0.43 s
[Batch 120/475] elapsed 0.43 s
[Batch 150/475] elapsed 0.43 s
[Batch 180/475] elapsed 0.43 s
[Batch 210/475] elapsed 0.43 s
[Batch 240/475] elapsed 0.43 s
[Batch 270/475] elapsed 0.43 s
[Batch 300/475] elapsed 0.43 s
[Batch 330/475] elapsed 0.43 s
[Batch 360/475] elapsed 0.43 s
[Batch 390/475] elapsed 0.43 s
[Batch 420/475] elapsed 0.43 s
[Batch 450/475] elapsed 0.43 s
[Epoch 7] train avg loss 0.0135559, test acc 0.6772, test avg loss 0.794825, throughput 3.99618K wps
[Epoch 8 Batch 30/4268] avg loss 0.0129934, throughput 4.08525K wps
[Epoch 8 Batch 60/4268] avg loss 0.0124846, throughput 3.9951K wps
[Epoch 8 Batch 90/4268] avg loss 0.012317, throughput 3.99376K wps
[Epoch 8 Batch 120/4268] avg loss 0.0126362, throughput 3.99831K wps
[Epoch 8 Batch 150/4268] avg loss 0.0131466, throughput 3.99226K wps
[Epoch 8 Batch 180/4268] avg loss 0.0133246, throughput 3.99674K wps
[Epoch 8 Batch 210/4268] avg loss 0.0129614, throughput 4.00017K wps
[Epoch 8 Batch 240/4268] avg loss 0.013041, throughput 4.00043K wps
[Epoch 8 Batch 270/4268] avg loss 0.012739, throughput 3.99594K wps
[Epoch 8 Batch 300/4268] avg loss 0.0128847, throughput 4.0006K wps
[Epoch 8 Batch 330/4268] avg loss 0.0129769, throughput 4.00738K wps
[Epoch 8 Batch 360/4268] avg loss 0.0127581, throughput 3.99587K wps
[Epoch 8 Batch 390/4268] avg loss 0.0126947, throughput 3.99507K wps
[Epoch 8 Batch 420/4268] avg loss 0.0122828, throughput 3.99721K wps
[Epoch 8 Batch 450/4268] avg loss 0.0125328, throughput 3.99568K wps
[Epoch 8 Batch 480/4268] avg loss 0.0130073, throughput 3.9955K wps
[Epoch 8 Batch 510/4268] avg loss 0.0130486, throughput 3.98405K wps
[Epoch 8 Batch 540/4268] avg loss 0.0124753, throughput 3.98654K wps
[Epoch 8 Batch 570/4268] avg loss 0.0127551, throughput 4.00185K wps
[Epoch 8 Batch 600/4268] avg loss 0.0136902, throughput 4.00283K wps
[Epoch 8 Batch 630/4268] avg loss 0.0129727, throughput 4.00452K wps
[Epoch 8 Batch 660/4268] avg loss 0.0129229, throughput 4.00265K wps
[Epoch 8 Batch 690/4268] avg loss 0.0126298, throughput 3.99439K wps
[Epoch 8 Batch 720/4268] avg loss 0.0124935, throughput 3.99538K wps
[Epoch 8 Batch 750/4268] avg loss 0.0124557, throughput 3.99304K wps
[Epoch 8 Batch 780/4268] avg loss 0.013229, throughput 3.99438K wps
[Epoch 8 Batch 810/4268] avg loss 0.0133728, throughput 3.99456K wps
[Epoch 8 Batch 840/4268] avg loss 0.0131495, throughput 3.99461K wps
[Epoch 8 Batch 870/4268] avg loss 0.0125618, throughput 3.99531K wps
[Epoch 8 Batch 900/4268] avg loss 0.0125591, throughput 3.99591K wps
[Epoch 8 Batch 930/4268] avg loss 0.0121402, throughput 3.98893K wps
[Epoch 8 Batch 960/4268] avg loss 0.0127207, throughput 3.99502K wps
[Epoch 8 Batch 990/4268] avg loss 0.0128491, throughput 3.99481K wps
[Epoch 8 Batch 1020/4268] avg loss 0.013429, throughput 3.99911K wps
[Epoch 8 Batch 1050/4268] avg loss 0.0124315, throughput 3.99328K wps
[Epoch 8 Batch 1080/4268] avg loss 0.0128656, throughput 3.99505K wps
[Epoch 8 Batch 1110/4268] avg loss 0.0134632, throughput 3.99349K wps
[Epoch 8 Batch 1140/4268] avg loss 0.0130921, throughput 3.99609K wps
[Epoch 8 Batch 1170/4268] avg loss 0.0132091, throughput 3.99179K wps
[Epoch 8 Batch 1200/4268] avg loss 0.0131074, throughput 3.99366K wps
[Epoch 8 Batch 1230/4268] avg loss 0.0135167, throughput 3.99535K wps
[Epoch 8 Batch 1260/4268] avg loss 0.0137288, throughput 3.99391K wps
[Epoch 8 Batch 1290/4268] avg loss 0.0131021, throughput 3.99616K wps
[Epoch 8 Batch 1320/4268] avg loss 0.012795, throughput 3.99503K wps
[Epoch 8 Batch 1350/4268] avg loss 0.012955, throughput 3.99312K wps
[Epoch 8 Batch 1380/4268] avg loss 0.0139132, throughput 3.99431K wps
[Epoch 8 Batch 1410/4268] avg loss 0.0123623, throughput 3.99401K wps
[Epoch 8 Batch 1440/4268] avg loss 0.0136392, throughput 3.992K wps
[Epoch 8 Batch 1470/4268] avg loss 0.013228, throughput 3.993K wps
[Epoch 8 Batch 1500/4268] avg loss 0.0133115, throughput 3.99836K wps
[Epoch 8 Batch 1530/4268] avg loss 0.0137581, throughput 3.99263K wps
[Epoch 8 Batch 1560/4268] avg loss 0.0135971, throughput 3.99434K wps
[Epoch 8 Batch 1590/4268] avg loss 0.0135529, throughput 3.99688K wps
[Epoch 8 Batch 1620/4268] avg loss 0.0137412, throughput 3.99373K wps
[Epoch 8 Batch 1650/4268] avg loss 0.0128765, throughput 3.9948K wps
[Epoch 8 Batch 1680/4268] avg loss 0.0132653, throughput 3.9941K wps
[Epoch 8 Batch 1710/4268] avg loss 0.0127784, throughput 3.99167K wps
[Epoch 8 Batch 1740/4268] avg loss 0.0133897, throughput 3.99122K wps
[Epoch 8 Batch 1770/4268] avg loss 0.0130068, throughput 3.99467K wps
[Epoch 8 Batch 1800/4268] avg loss 0.0133768, throughput 3.99749K wps
[Epoch 8 Batch 1830/4268] avg loss 0.0132112, throughput 3.9925K wps
[Epoch 8 Batch 1860/4268] avg loss 0.0136889, throughput 3.99078K wps
[Epoch 8 Batch 1890/4268] avg loss 0.0140242, throughput 3.98905K wps
[Epoch 8 Batch 1920/4268] avg loss 0.0133166, throughput 3.99087K wps
[Epoch 8 Batch 1950/4268] avg loss 0.0128666, throughput 3.99951K wps
[Epoch 8 Batch 1980/4268] avg loss 0.0133868, throughput 3.99778K wps
[Epoch 8 Batch 2010/4268] avg loss 0.0129918, throughput 3.99539K wps
[Epoch 8 Batch 2040/4268] avg loss 0.0137603, throughput 3.99608K wps
[Epoch 8 Batch 2070/4268] avg loss 0.0137213, throughput 3.99456K wps
[Epoch 8 Batch 2100/4268] avg loss 0.0123471, throughput 3.99403K wps
[Epoch 8 Batch 2130/4268] avg loss 0.0134968, throughput 3.99276K wps
[Epoch 8 Batch 2160/4268] avg loss 0.0134032, throughput 3.99319K wps
[Epoch 8 Batch 2190/4268] avg loss 0.0135703, throughput 3.99137K wps
[Epoch 8 Batch 2220/4268] avg loss 0.013431, throughput 3.99745K wps
[Epoch 8 Batch 2250/4268] avg loss 0.013731, throughput 4.00695K wps
[Epoch 8 Batch 2280/4268] avg loss 0.0138002, throughput 3.99894K wps
[Epoch 8 Batch 2310/4268] avg loss 0.0135947, throughput 4.00274K wps
[Epoch 8 Batch 2340/4268] avg loss 0.0130593, throughput 4.00084K wps
[Epoch 8 Batch 2370/4268] avg loss 0.0133517, throughput 3.99598K wps
[Epoch 8 Batch 2400/4268] avg loss 0.0129283, throughput 3.99698K wps
[Epoch 8 Batch 2430/4268] avg loss 0.0126722, throughput 3.9921K wps
[Epoch 8 Batch 2460/4268] avg loss 0.0131835, throughput 3.98987K wps
[Epoch 8 Batch 2490/4268] avg loss 0.0129987, throughput 3.99313K wps
[Epoch 8 Batch 2520/4268] avg loss 0.0131914, throughput 3.99268K wps
[Epoch 8 Batch 2550/4268] avg loss 0.0126157, throughput 3.99599K wps
[Epoch 8 Batch 2580/4268] avg loss 0.0129159, throughput 3.99714K wps
[Epoch 8 Batch 2610/4268] avg loss 0.0124769, throughput 3.99355K wps
[Epoch 8 Batch 2640/4268] avg loss 0.0133775, throughput 3.99749K wps
[Epoch 8 Batch 2670/4268] avg loss 0.0132594, throughput 3.99749K wps
[Epoch 8 Batch 2700/4268] avg loss 0.0129929, throughput 4.0001K wps
[Epoch 8 Batch 2730/4268] avg loss 0.0135407, throughput 4.00173K wps
[Epoch 8 Batch 2760/4268] avg loss 0.0127519, throughput 4.00051K wps
[Epoch 8 Batch 2790/4268] avg loss 0.0142959, throughput 4.00261K wps
[Epoch 8 Batch 2820/4268] avg loss 0.0134316, throughput 3.99798K wps
[Epoch 8 Batch 2850/4268] avg loss 0.0133579, throughput 3.9988K wps
[Epoch 8 Batch 2880/4268] avg loss 0.0140899, throughput 3.99479K wps
[Epoch 8 Batch 2910/4268] avg loss 0.0142588, throughput 3.99463K wps
[Epoch 8 Batch 2940/4268] avg loss 0.0132849, throughput 3.99681K wps
[Epoch 8 Batch 2970/4268] avg loss 0.0131362, throughput 3.99933K wps
[Epoch 8 Batch 3000/4268] avg loss 0.0130009, throughput 3.99742K wps
[Epoch 8 Batch 3030/4268] avg loss 0.0128569, throughput 3.9969K wps
[Epoch 8 Batch 3060/4268] avg loss 0.0140559, throughput 3.99394K wps
[Epoch 8 Batch 3090/4268] avg loss 0.0134825, throughput 3.99447K wps
[Epoch 8 Batch 3120/4268] avg loss 0.0137916, throughput 4.00152K wps
[Epoch 8 Batch 3150/4268] avg loss 0.013862, throughput 3.99658K wps
[Epoch 8 Batch 3180/4268] avg loss 0.013985, throughput 3.99535K wps
[Epoch 8 Batch 3210/4268] avg loss 0.0134807, throughput 3.99615K wps
[Epoch 8 Batch 3240/4268] avg loss 0.013556, throughput 3.99806K wps
[Epoch 8 Batch 3270/4268] avg loss 0.0135652, throughput 3.99495K wps
[Epoch 8 Batch 3300/4268] avg loss 0.0128907, throughput 3.99451K wps
[Epoch 8 Batch 3330/4268] avg loss 0.0129432, throughput 3.99849K wps
[Epoch 8 Batch 3360/4268] avg loss 0.0131416, throughput 3.99494K wps
[Epoch 8 Batch 3390/4268] avg loss 0.0132099, throughput 3.99611K wps
[Epoch 8 Batch 3420/4268] avg loss 0.013321, throughput 3.99453K wps
[Epoch 8 Batch 3450/4268] avg loss 0.0135078, throughput 3.99407K wps
[Epoch 8 Batch 3480/4268] avg loss 0.0133345, throughput 3.99409K wps
[Epoch 8 Batch 3510/4268] avg loss 0.0129214, throughput 3.99653K wps
[Epoch 8 Batch 3540/4268] avg loss 0.0134873, throughput 3.9943K wps
[Epoch 8 Batch 3570/4268] avg loss 0.0136953, throughput 3.99912K wps
[Epoch 8 Batch 3600/4268] avg loss 0.0138182, throughput 3.99702K wps
[Epoch 8 Batch 3630/4268] avg loss 0.0139178, throughput 3.99691K wps
[Epoch 8 Batch 3660/4268] avg loss 0.0139797, throughput 3.99262K wps
[Epoch 8 Batch 3690/4268] avg loss 0.0137003, throughput 3.99881K wps
[Epoch 8 Batch 3720/4268] avg loss 0.0133363, throughput 3.99655K wps
[Epoch 8 Batch 3750/4268] avg loss 0.0135571, throughput 3.99418K wps
[Epoch 8 Batch 3780/4268] avg loss 0.0138197, throughput 3.99774K wps
[Epoch 8 Batch 3810/4268] avg loss 0.0135915, throughput 3.99545K wps
[Epoch 8 Batch 3840/4268] avg loss 0.0129806, throughput 3.99916K wps
[Epoch 8 Batch 3870/4268] avg loss 0.0123381, throughput 3.99785K wps
[Epoch 8 Batch 3900/4268] avg loss 0.0135782, throughput 4.00671K wps
[Epoch 8 Batch 3930/4268] avg loss 0.0134178, throughput 3.99835K wps
[Epoch 8 Batch 3960/4268] avg loss 0.013608, throughput 3.99472K wps
[Epoch 8 Batch 3990/4268] avg loss 0.0126392, throughput 4.0018K wps
[Epoch 8 Batch 4020/4268] avg loss 0.0137764, throughput 3.9952K wps
[Epoch 8 Batch 4050/4268] avg loss 0.0142361, throughput 3.99924K wps
[Epoch 8 Batch 4080/4268] avg loss 0.0131958, throughput 3.9978K wps
[Epoch 8 Batch 4110/4268] avg loss 0.0133917, throughput 3.98395K wps
[Epoch 8 Batch 4140/4268] avg loss 0.0136513, throughput 3.98826K wps
[Epoch 8 Batch 4170/4268] avg loss 0.0127403, throughput 3.99561K wps
[Epoch 8 Batch 4200/4268] avg loss 0.0138409, throughput 3.99751K wps
[Epoch 8 Batch 4230/4268] avg loss 0.0133635, throughput 3.99471K wps
[Epoch 8 Batch 4260/4268] avg loss 0.0130479, throughput 3.99921K wps
Begin Testing...
[Batch 30/475] elapsed 0.46 s
[Batch 60/475] elapsed 0.43 s
[Batch 90/475] elapsed 0.43 s
[Batch 120/475] elapsed 0.43 s
[Batch 150/475] elapsed 0.43 s
[Batch 180/475] elapsed 0.43 s
[Batch 210/475] elapsed 0.43 s
[Batch 240/475] elapsed 0.43 s
[Batch 270/475] elapsed 0.43 s
[Batch 300/475] elapsed 0.43 s
[Batch 330/475] elapsed 0.43 s
[Batch 360/475] elapsed 0.43 s
[Batch 390/475] elapsed 0.43 s
[Batch 420/475] elapsed 0.43 s
[Batch 450/475] elapsed 0.43 s
[Epoch 8] train avg loss 0.0132228, test acc 0.6767, test avg loss 0.804526, throughput 3.9965K wps
[Epoch 9 Batch 30/4268] avg loss 0.0125655, throughput 4.08654K wps
[Epoch 9 Batch 60/4268] avg loss 0.0120446, throughput 4.00308K wps
[Epoch 9 Batch 90/4268] avg loss 0.0134673, throughput 3.9951K wps
[Epoch 9 Batch 120/4268] avg loss 0.0131105, throughput 4.00518K wps
[Epoch 9 Batch 150/4268] avg loss 0.0127293, throughput 3.99686K wps
[Epoch 9 Batch 180/4268] avg loss 0.0131693, throughput 3.99142K wps
[Epoch 9 Batch 210/4268] avg loss 0.0126002, throughput 3.99777K wps
[Epoch 9 Batch 240/4268] avg loss 0.012847, throughput 4.00337K wps
[Epoch 9 Batch 270/4268] avg loss 0.0123349, throughput 4.00216K wps
[Epoch 9 Batch 300/4268] avg loss 0.0118982, throughput 3.99448K wps
[Epoch 9 Batch 330/4268] avg loss 0.0123834, throughput 3.99638K wps
[Epoch 9 Batch 360/4268] avg loss 0.0123104, throughput 3.99778K wps
[Epoch 9 Batch 390/4268] avg loss 0.0120363, throughput 3.99794K wps
[Epoch 9 Batch 420/4268] avg loss 0.0125617, throughput 3.99633K wps
[Epoch 9 Batch 450/4268] avg loss 0.0131123, throughput 3.99693K wps
[Epoch 9 Batch 480/4268] avg loss 0.0126965, throughput 3.99885K wps
[Epoch 9 Batch 510/4268] avg loss 0.0131392, throughput 3.9975K wps
[Epoch 9 Batch 540/4268] avg loss 0.0120904, throughput 3.99785K wps
[Epoch 9 Batch 570/4268] avg loss 0.0123517, throughput 3.9939K wps
[Epoch 9 Batch 600/4268] avg loss 0.0130732, throughput 3.99303K wps
[Epoch 9 Batch 630/4268] avg loss 0.0132971, throughput 3.9978K wps
[Epoch 9 Batch 660/4268] avg loss 0.0129501, throughput 3.99712K wps
[Epoch 9 Batch 690/4268] avg loss 0.0127162, throughput 3.99829K wps
[Epoch 9 Batch 720/4268] avg loss 0.0131411, throughput 3.99565K wps
[Epoch 9 Batch 750/4268] avg loss 0.0119508, throughput 3.99941K wps
[Epoch 9 Batch 780/4268] avg loss 0.0129995, throughput 3.99841K wps
[Epoch 9 Batch 810/4268] avg loss 0.0122033, throughput 3.9997K wps
[Epoch 9 Batch 840/4268] avg loss 0.0124029, throughput 4.00339K wps
[Epoch 9 Batch 870/4268] avg loss 0.0124273, throughput 3.99878K wps
[Epoch 9 Batch 900/4268] avg loss 0.0128396, throughput 3.9958K wps
[Epoch 9 Batch 930/4268] avg loss 0.0121917, throughput 3.99644K wps
[Epoch 9 Batch 960/4268] avg loss 0.0128877, throughput 3.99656K wps
[Epoch 9 Batch 990/4268] avg loss 0.0129055, throughput 3.9945K wps
[Epoch 9 Batch 1020/4268] avg loss 0.0130604, throughput 3.99574K wps
[Epoch 9 Batch 1050/4268] avg loss 0.0124535, throughput 3.99224K wps
[Epoch 9 Batch 1080/4268] avg loss 0.0125109, throughput 3.99591K wps
[Epoch 9 Batch 1110/4268] avg loss 0.0133232, throughput 3.99677K wps
[Epoch 9 Batch 1140/4268] avg loss 0.0128804, throughput 3.99375K wps
[Epoch 9 Batch 1170/4268] avg loss 0.0123826, throughput 3.9912K wps
[Epoch 9 Batch 1200/4268] avg loss 0.0131121, throughput 3.99753K wps
[Epoch 9 Batch 1230/4268] avg loss 0.0124377, throughput 3.99893K wps
[Epoch 9 Batch 1260/4268] avg loss 0.0128782, throughput 3.99343K wps
[Epoch 9 Batch 1290/4268] avg loss 0.013316, throughput 3.99613K wps
[Epoch 9 Batch 1320/4268] avg loss 0.0131493, throughput 3.99578K wps
[Epoch 9 Batch 1350/4268] avg loss 0.0125649, throughput 3.99582K wps
[Epoch 9 Batch 1380/4268] avg loss 0.0134121, throughput 4.00551K wps
[Epoch 9 Batch 1410/4268] avg loss 0.0126833, throughput 4.00366K wps
[Epoch 9 Batch 1440/4268] avg loss 0.0126393, throughput 3.99844K wps
[Epoch 9 Batch 1470/4268] avg loss 0.0126076, throughput 3.99679K wps
[Epoch 9 Batch 1500/4268] avg loss 0.0128232, throughput 3.99552K wps
[Epoch 9 Batch 1530/4268] avg loss 0.0126056, throughput 3.99355K wps
[Epoch 9 Batch 1560/4268] avg loss 0.012714, throughput 3.99704K wps
[Epoch 9 Batch 1590/4268] avg loss 0.0133495, throughput 4.00104K wps
[Epoch 9 Batch 1620/4268] avg loss 0.0125254, throughput 4.00206K wps
[Epoch 9 Batch 1650/4268] avg loss 0.0132541, throughput 3.99371K wps
[Epoch 9 Batch 1680/4268] avg loss 0.0126174, throughput 4.00193K wps
[Epoch 9 Batch 1710/4268] avg loss 0.012479, throughput 4.00245K wps
[Epoch 9 Batch 1740/4268] avg loss 0.0125894, throughput 3.99802K wps
[Epoch 9 Batch 1770/4268] avg loss 0.0126567, throughput 3.99829K wps
[Epoch 9 Batch 1800/4268] avg loss 0.0123562, throughput 3.99925K wps
[Epoch 9 Batch 1830/4268] avg loss 0.0128688, throughput 3.99707K wps
[Epoch 9 Batch 1860/4268] avg loss 0.0127391, throughput 3.99289K wps
[Epoch 9 Batch 1890/4268] avg loss 0.0129715, throughput 3.99725K wps
[Epoch 9 Batch 1920/4268] avg loss 0.0129556, throughput 3.99884K wps
[Epoch 9 Batch 1950/4268] avg loss 0.0131042, throughput 4.00195K wps
[Epoch 9 Batch 1980/4268] avg loss 0.0128847, throughput 3.99852K wps
[Epoch 9 Batch 2010/4268] avg loss 0.0129218, throughput 3.99839K wps
[Epoch 9 Batch 2040/4268] avg loss 0.0124547, throughput 4.00007K wps
[Epoch 9 Batch 2070/4268] avg loss 0.012441, throughput 3.9993K wps
[Epoch 9 Batch 2100/4268] avg loss 0.0133198, throughput 3.99896K wps
[Epoch 9 Batch 2130/4268] avg loss 0.0131899, throughput 3.99668K wps
[Epoch 9 Batch 2160/4268] avg loss 0.0137414, throughput 4.00355K wps
[Epoch 9 Batch 2190/4268] avg loss 0.0129337, throughput 3.99806K wps
[Epoch 9 Batch 2220/4268] avg loss 0.0129185, throughput 3.99295K wps
[Epoch 9 Batch 2250/4268] avg loss 0.0126449, throughput 4.00082K wps
[Epoch 9 Batch 2280/4268] avg loss 0.0134199, throughput 3.99889K wps
[Epoch 9 Batch 2310/4268] avg loss 0.0126658, throughput 4.00422K wps
[Epoch 9 Batch 2340/4268] avg loss 0.0123617, throughput 3.99454K wps
[Epoch 9 Batch 2370/4268] avg loss 0.012756, throughput 3.99317K wps
[Epoch 9 Batch 2400/4268] avg loss 0.0128237, throughput 3.99771K wps
[Epoch 9 Batch 2430/4268] avg loss 0.0131535, throughput 3.99797K wps
[Epoch 9 Batch 2460/4268] avg loss 0.013044, throughput 3.99518K wps
[Epoch 9 Batch 2490/4268] avg loss 0.0129285, throughput 3.99879K wps
[Epoch 9 Batch 2520/4268] avg loss 0.0127389, throughput 4.00042K wps
[Epoch 9 Batch 2550/4268] avg loss 0.0124734, throughput 3.99678K wps
[Epoch 9 Batch 2580/4268] avg loss 0.0131901, throughput 3.99422K wps
[Epoch 9 Batch 2610/4268] avg loss 0.0131343, throughput 3.99492K wps
[Epoch 9 Batch 2640/4268] avg loss 0.0138309, throughput 3.99367K wps
[Epoch 9 Batch 2670/4268] avg loss 0.0134, throughput 3.99676K wps
[Epoch 9 Batch 2700/4268] avg loss 0.0126982, throughput 3.99646K wps
[Epoch 9 Batch 2730/4268] avg loss 0.0126871, throughput 3.99591K wps
[Epoch 9 Batch 2760/4268] avg loss 0.0122484, throughput 3.99504K wps
[Epoch 9 Batch 2790/4268] avg loss 0.0138408, throughput 3.9961K wps
[Epoch 9 Batch 2820/4268] avg loss 0.0128321, throughput 3.99769K wps
[Epoch 9 Batch 2850/4268] avg loss 0.0130642, throughput 3.99561K wps
[Epoch 9 Batch 2880/4268] avg loss 0.012917, throughput 3.99642K wps
[Epoch 9 Batch 2910/4268] avg loss 0.0128737, throughput 3.99431K wps
[Epoch 9 Batch 2940/4268] avg loss 0.0135411, throughput 3.99834K wps
[Epoch 9 Batch 2970/4268] avg loss 0.01245, throughput 3.99349K wps
[Epoch 9 Batch 3000/4268] avg loss 0.0142876, throughput 3.99869K wps
[Epoch 9 Batch 3030/4268] avg loss 0.0131339, throughput 3.99744K wps
[Epoch 9 Batch 3060/4268] avg loss 0.0124807, throughput 3.99729K wps
[Epoch 9 Batch 3090/4268] avg loss 0.0133159, throughput 3.99319K wps
[Epoch 9 Batch 3120/4268] avg loss 0.0137573, throughput 3.99652K wps
[Epoch 9 Batch 3150/4268] avg loss 0.0134318, throughput 3.99708K wps
[Epoch 9 Batch 3180/4268] avg loss 0.0138491, throughput 3.99425K wps
[Epoch 9 Batch 3210/4268] avg loss 0.0129754, throughput 3.99761K wps
[Epoch 9 Batch 3240/4268] avg loss 0.0133551, throughput 3.99479K wps
[Epoch 9 Batch 3270/4268] avg loss 0.0131201, throughput 3.99937K wps
[Epoch 9 Batch 3300/4268] avg loss 0.0129624, throughput 3.98593K wps
[Epoch 9 Batch 3330/4268] avg loss 0.0131713, throughput 3.9913K wps
[Epoch 9 Batch 3360/4268] avg loss 0.0131384, throughput 4.00196K wps
[Epoch 9 Batch 3390/4268] avg loss 0.0129806, throughput 3.99685K wps
[Epoch 9 Batch 3420/4268] avg loss 0.0127669, throughput 3.99552K wps
[Epoch 9 Batch 3450/4268] avg loss 0.0135214, throughput 3.99675K wps
[Epoch 9 Batch 3480/4268] avg loss 0.0125871, throughput 3.99615K wps
[Epoch 9 Batch 3510/4268] avg loss 0.013698, throughput 3.99129K wps
[Epoch 9 Batch 3540/4268] avg loss 0.0129631, throughput 3.99718K wps
[Epoch 9 Batch 3570/4268] avg loss 0.013859, throughput 3.99182K wps
[Epoch 9 Batch 3600/4268] avg loss 0.0131261, throughput 3.99818K wps
[Epoch 9 Batch 3630/4268] avg loss 0.0129121, throughput 3.99871K wps
[Epoch 9 Batch 3660/4268] avg loss 0.0130433, throughput 3.99489K wps
[Epoch 9 Batch 3690/4268] avg loss 0.012717, throughput 3.99363K wps
[Epoch 9 Batch 3720/4268] avg loss 0.0124928, throughput 3.9963K wps
[Epoch 9 Batch 3750/4268] avg loss 0.0125944, throughput 3.99823K wps
[Epoch 9 Batch 3780/4268] avg loss 0.0131676, throughput 3.99843K wps
[Epoch 9 Batch 3810/4268] avg loss 0.0130826, throughput 3.99907K wps
[Epoch 9 Batch 3840/4268] avg loss 0.0131188, throughput 3.99446K wps
[Epoch 9 Batch 3870/4268] avg loss 0.0129367, throughput 3.99723K wps
[Epoch 9 Batch 3900/4268] avg loss 0.0127361, throughput 3.99203K wps
[Epoch 9 Batch 3930/4268] avg loss 0.0134876, throughput 3.99387K wps
[Epoch 9 Batch 3960/4268] avg loss 0.0134881, throughput 3.99783K wps
[Epoch 9 Batch 3990/4268] avg loss 0.0136386, throughput 3.99703K wps
[Epoch 9 Batch 4020/4268] avg loss 0.013196, throughput 3.99468K wps
[Epoch 9 Batch 4050/4268] avg loss 0.0126169, throughput 3.99555K wps
[Epoch 9 Batch 4080/4268] avg loss 0.0127764, throughput 3.99588K wps
[Epoch 9 Batch 4110/4268] avg loss 0.0126643, throughput 3.99653K wps
[Epoch 9 Batch 4140/4268] avg loss 0.0133282, throughput 3.99265K wps
[Epoch 9 Batch 4170/4268] avg loss 0.0130274, throughput 3.99536K wps
[Epoch 9 Batch 4200/4268] avg loss 0.0128413, throughput 3.98904K wps
[Epoch 9 Batch 4230/4268] avg loss 0.0129723, throughput 3.99507K wps
[Epoch 9 Batch 4260/4268] avg loss 0.0127852, throughput 3.99428K wps
Begin Testing...
[Batch 30/475] elapsed 0.46 s
[Batch 60/475] elapsed 0.43 s
[Batch 90/475] elapsed 0.43 s
[Batch 120/475] elapsed 0.43 s
[Batch 150/475] elapsed 0.43 s
[Batch 180/475] elapsed 0.43 s
[Batch 210/475] elapsed 0.43 s
[Batch 240/475] elapsed 0.43 s
[Batch 270/475] elapsed 0.43 s
[Batch 300/475] elapsed 0.43 s
[Batch 330/475] elapsed 0.43 s
[Batch 360/475] elapsed 0.43 s
[Batch 390/475] elapsed 0.43 s
[Batch 420/475] elapsed 0.43 s
[Batch 450/475] elapsed 0.43 s
[Epoch 9] train avg loss 0.0129031, test acc 0.6786, test avg loss 0.803311, throughput 3.99745K wps
Observed Improvement.
Begin Testing...
[Batch 30/43] elapsed 0.43 s
[Epoch 10 Batch 30/4268] avg loss 0.011978, throughput 4.08434K wps
[Epoch 10 Batch 60/4268] avg loss 0.0121643, throughput 3.99488K wps
[Epoch 10 Batch 90/4268] avg loss 0.0118236, throughput 3.99677K wps
[Epoch 10 Batch 120/4268] avg loss 0.0122533, throughput 3.99794K wps
[Epoch 10 Batch 150/4268] avg loss 0.0124397, throughput 4.00301K wps
[Epoch 10 Batch 180/4268] avg loss 0.0125254, throughput 4.00063K wps
[Epoch 10 Batch 210/4268] avg loss 0.0122921, throughput 4.00209K wps
[Epoch 10 Batch 240/4268] avg loss 0.0115399, throughput 4.00929K wps
[Epoch 10 Batch 270/4268] avg loss 0.0117972, throughput 4.0087K wps
[Epoch 10 Batch 300/4268] avg loss 0.0124752, throughput 3.99657K wps
[Epoch 10 Batch 330/4268] avg loss 0.0116904, throughput 4.00132K wps
[Epoch 10 Batch 360/4268] avg loss 0.0121894, throughput 3.99442K wps
[Epoch 10 Batch 390/4268] avg loss 0.011804, throughput 3.99332K wps
[Epoch 10 Batch 420/4268] avg loss 0.0117828, throughput 3.99506K wps
[Epoch 10 Batch 450/4268] avg loss 0.0126175, throughput 3.99425K wps
[Epoch 10 Batch 480/4268] avg loss 0.0124125, throughput 3.99452K wps
[Epoch 10 Batch 510/4268] avg loss 0.0130763, throughput 3.99554K wps
[Epoch 10 Batch 540/4268] avg loss 0.0128908, throughput 3.99516K wps
[Epoch 10 Batch 570/4268] avg loss 0.0122358, throughput 3.9951K wps
[Epoch 10 Batch 600/4268] avg loss 0.0123093, throughput 3.99295K wps
[Epoch 10 Batch 630/4268] avg loss 0.0122202, throughput 3.98919K wps
[Epoch 10 Batch 660/4268] avg loss 0.0124842, throughput 3.99254K wps
[Epoch 10 Batch 690/4268] avg loss 0.0127629, throughput 3.99875K wps
[Epoch 10 Batch 720/4268] avg loss 0.0118071, throughput 4.00122K wps
[Epoch 10 Batch 750/4268] avg loss 0.0126214, throughput 3.99975K wps
[Epoch 10 Batch 780/4268] avg loss 0.0122573, throughput 3.99386K wps
[Epoch 10 Batch 810/4268] avg loss 0.0127114, throughput 3.99732K wps
[Epoch 10 Batch 840/4268] avg loss 0.0124917, throughput 3.99534K wps
[Epoch 10 Batch 870/4268] avg loss 0.0122531, throughput 3.99689K wps
[Epoch 10 Batch 900/4268] avg loss 0.0125896, throughput 3.99244K wps
[Epoch 10 Batch 930/4268] avg loss 0.0121333, throughput 3.99221K wps
[Epoch 10 Batch 960/4268] avg loss 0.0130628, throughput 3.99826K wps
[Epoch 10 Batch 990/4268] avg loss 0.0122332, throughput 3.99648K wps
[Epoch 10 Batch 1020/4268] avg loss 0.0127321, throughput 4.00384K wps
[Epoch 10 Batch 1050/4268] avg loss 0.0120026, throughput 3.995K wps
[Epoch 10 Batch 1080/4268] avg loss 0.0122261, throughput 4.00023K wps
[Epoch 10 Batch 1110/4268] avg loss 0.0122716, throughput 3.99545K wps
[Epoch 10 Batch 1140/4268] avg loss 0.0128804, throughput 3.99611K wps
[Epoch 10 Batch 1170/4268] avg loss 0.0123755, throughput 3.99598K wps
[Epoch 10 Batch 1200/4268] avg loss 0.0120125, throughput 3.99772K wps
[Epoch 10 Batch 1230/4268] avg loss 0.0125874, throughput 3.99543K wps
[Epoch 10 Batch 1260/4268] avg loss 0.0119908, throughput 3.99412K wps
[Epoch 10 Batch 1290/4268] avg loss 0.0126508, throughput 3.99642K wps
[Epoch 10 Batch 1320/4268] avg loss 0.012132, throughput 3.99444K wps
[Epoch 10 Batch 1350/4268] avg loss 0.0123174, throughput 3.99809K wps
[Epoch 10 Batch 1380/4268] avg loss 0.0130318, throughput 3.98981K wps
[Epoch 10 Batch 1410/4268] avg loss 0.012733, throughput 3.9942K wps
[Epoch 10 Batch 1440/4268] avg loss 0.0120172, throughput 3.99336K wps
[Epoch 10 Batch 1470/4268] avg loss 0.0131626, throughput 3.9911K wps
[Epoch 10 Batch 1500/4268] avg loss 0.012083, throughput 3.99731K wps
[Epoch 10 Batch 1530/4268] avg loss 0.0129934, throughput 3.99311K wps
[Epoch 10 Batch 1560/4268] avg loss 0.0124081, throughput 3.99235K wps
[Epoch 10 Batch 1590/4268] avg loss 0.0121695, throughput 3.99435K wps
[Epoch 10 Batch 1620/4268] avg loss 0.0121597, throughput 3.99898K wps
[Epoch 10 Batch 1650/4268] avg loss 0.0120637, throughput 3.99313K wps
[Epoch 10 Batch 1680/4268] avg loss 0.0129466, throughput 3.99608K wps
[Epoch 10 Batch 1710/4268] avg loss 0.0125501, throughput 3.99501K wps
[Epoch 10 Batch 1740/4268] avg loss 0.0131153, throughput 4.00049K wps
[Epoch 10 Batch 1770/4268] avg loss 0.0129399, throughput 4.00234K wps
[Epoch 10 Batch 1800/4268] avg loss 0.012371, throughput 3.99858K wps
[Epoch 10 Batch 1830/4268] avg loss 0.0125924, throughput 3.99795K wps
[Epoch 10 Batch 1860/4268] avg loss 0.0124236, throughput 3.99883K wps
[Epoch 10 Batch 1890/4268] avg loss 0.0129163, throughput 3.98909K wps
[Epoch 10 Batch 1920/4268] avg loss 0.0131968, throughput 4.00072K wps
[Epoch 10 Batch 1950/4268] avg loss 0.0127441, throughput 4.0047K wps
[Epoch 10 Batch 1980/4268] avg loss 0.0127723, throughput 4.0069K wps
[Epoch 10 Batch 2010/4268] avg loss 0.0128075, throughput 3.99651K wps
[Epoch 10 Batch 2040/4268] avg loss 0.0127911, throughput 3.99421K wps
[Epoch 10 Batch 2070/4268] avg loss 0.0122667, throughput 3.99443K wps
[Epoch 10 Batch 2100/4268] avg loss 0.012136, throughput 3.99621K wps
[Epoch 10 Batch 2130/4268] avg loss 0.0119459, throughput 3.99612K wps
[Epoch 10 Batch 2160/4268] avg loss 0.0126645, throughput 3.99487K wps
[Epoch 10 Batch 2190/4268] avg loss 0.0125864, throughput 3.99541K wps
[Epoch 10 Batch 2220/4268] avg loss 0.0126791, throughput 3.99896K wps
[Epoch 10 Batch 2250/4268] avg loss 0.0131326, throughput 3.99825K wps
[Epoch 10 Batch 2280/4268] avg loss 0.0132221, throughput 4.00033K wps
[Epoch 10 Batch 2310/4268] avg loss 0.0131252, throughput 4.00279K wps
[Epoch 10 Batch 2340/4268] avg loss 0.0124818, throughput 3.99967K wps
[Epoch 10 Batch 2370/4268] avg loss 0.0128986, throughput 3.99844K wps
[Epoch 10 Batch 2400/4268] avg loss 0.0128071, throughput 3.99324K wps
[Epoch 10 Batch 2430/4268] avg loss 0.0130998, throughput 3.99295K wps
[Epoch 10 Batch 2460/4268] avg loss 0.0122234, throughput 3.99334K wps
[Epoch 10 Batch 2490/4268] avg loss 0.0132154, throughput 3.98674K wps
[Epoch 10 Batch 2520/4268] avg loss 0.0135724, throughput 3.99186K wps
[Epoch 10 Batch 2550/4268] avg loss 0.012464, throughput 3.99392K wps
[Epoch 10 Batch 2580/4268] avg loss 0.0124109, throughput 3.99621K wps
[Epoch 10 Batch 2610/4268] avg loss 0.013298, throughput 3.99722K wps
[Epoch 10 Batch 2640/4268] avg loss 0.0119533, throughput 4.00117K wps
[Epoch 10 Batch 2670/4268] avg loss 0.0132936, throughput 3.9927K wps
[Epoch 10 Batch 2700/4268] avg loss 0.0126106, throughput 3.99631K wps
[Epoch 10 Batch 2730/4268] avg loss 0.0128553, throughput 3.99543K wps
[Epoch 10 Batch 2760/4268] avg loss 0.0118519, throughput 3.99979K wps
[Epoch 10 Batch 2790/4268] avg loss 0.0129658, throughput 3.99702K wps
[Epoch 10 Batch 2820/4268] avg loss 0.0132204, throughput 3.9992K wps
[Epoch 10 Batch 2850/4268] avg loss 0.0126647, throughput 3.99683K wps
[Epoch 10 Batch 2880/4268] avg loss 0.0130106, throughput 3.99384K wps
[Epoch 10 Batch 2910/4268] avg loss 0.0121736, throughput 3.99444K wps
[Epoch 10 Batch 2940/4268] avg loss 0.012868, throughput 3.99495K wps
[Epoch 10 Batch 2970/4268] avg loss 0.012344, throughput 3.99069K wps
[Epoch 10 Batch 3000/4268] avg loss 0.0127328, throughput 4.00015K wps
[Epoch 10 Batch 3030/4268] avg loss 0.0130535, throughput 3.99245K wps
[Epoch 10 Batch 3060/4268] avg loss 0.012762, throughput 3.99646K wps
[Epoch 10 Batch 3090/4268] avg loss 0.0117434, throughput 3.99991K wps
[Epoch 10 Batch 3120/4268] avg loss 0.0124022, throughput 3.99823K wps
[Epoch 10 Batch 3150/4268] avg loss 0.0126119, throughput 3.99979K wps
[Epoch 10 Batch 3180/4268] avg loss 0.0136481, throughput 3.99737K wps
[Epoch 10 Batch 3210/4268] avg loss 0.0126437, throughput 3.99858K wps
[Epoch 10 Batch 3240/4268] avg loss 0.0121935, throughput 3.99846K wps
[Epoch 10 Batch 3270/4268] avg loss 0.012685, throughput 3.99666K wps
[Epoch 10 Batch 3300/4268] avg loss 0.0129825, throughput 3.99582K wps
[Epoch 10 Batch 3330/4268] avg loss 0.0127289, throughput 4.00088K wps
[Epoch 10 Batch 3360/4268] avg loss 0.0128732, throughput 3.99342K wps
[Epoch 10 Batch 3390/4268] avg loss 0.0128085, throughput 3.99429K wps
[Epoch 10 Batch 3420/4268] avg loss 0.0128017, throughput 3.99975K wps
[Epoch 10 Batch 3450/4268] avg loss 0.0130129, throughput 3.99865K wps
[Epoch 10 Batch 3480/4268] avg loss 0.0127429, throughput 3.99622K wps
[Epoch 10 Batch 3510/4268] avg loss 0.0118856, throughput 4.00017K wps
[Epoch 10 Batch 3540/4268] avg loss 0.0124044, throughput 3.99853K wps
[Epoch 10 Batch 3570/4268] avg loss 0.0124847, throughput 3.99459K wps
[Epoch 10 Batch 3600/4268] avg loss 0.0130171, throughput 3.99777K wps
[Epoch 10 Batch 3630/4268] avg loss 0.0128807, throughput 4.00065K wps
[Epoch 10 Batch 3660/4268] avg loss 0.0123169, throughput 3.99641K wps
[Epoch 10 Batch 3690/4268] avg loss 0.0128266, throughput 3.99956K wps
[Epoch 10 Batch 3720/4268] avg loss 0.0130144, throughput 3.99895K wps
[Epoch 10 Batch 3750/4268] avg loss 0.0122036, throughput 3.99629K wps
[Epoch 10 Batch 3780/4268] avg loss 0.0122433, throughput 3.99554K wps
[Epoch 10 Batch 3810/4268] avg loss 0.013357, throughput 3.99984K wps
[Epoch 10 Batch 3840/4268] avg loss 0.013121, throughput 3.99226K wps
[Epoch 10 Batch 3870/4268] avg loss 0.0128198, throughput 3.99399K wps
[Epoch 10 Batch 3900/4268] avg loss 0.0125393, throughput 3.99651K wps
[Epoch 10 Batch 3930/4268] avg loss 0.0122568, throughput 3.99922K wps
[Epoch 10 Batch 3960/4268] avg loss 0.0129914, throughput 3.99601K wps
[Epoch 10 Batch 3990/4268] avg loss 0.0132349, throughput 3.99515K wps
[Epoch 10 Batch 4020/4268] avg loss 0.0135982, throughput 3.99412K wps
[Epoch 10 Batch 4050/4268] avg loss 0.012768, throughput 3.99517K wps
[Epoch 10 Batch 4080/4268] avg loss 0.0127267, throughput 3.99695K wps
[Epoch 10 Batch 4110/4268] avg loss 0.0129894, throughput 3.99609K wps
[Epoch 10 Batch 4140/4268] avg loss 0.0129464, throughput 3.99538K wps
[Epoch 10 Batch 4170/4268] avg loss 0.0130836, throughput 3.99731K wps
[Epoch 10 Batch 4200/4268] avg loss 0.0132559, throughput 3.99449K wps
[Epoch 10 Batch 4230/4268] avg loss 0.012559, throughput 3.99821K wps
[Epoch 10 Batch 4260/4268] avg loss 0.0120976, throughput 3.99607K wps
Begin Testing...
[Batch 30/475] elapsed 0.47 s
[Batch 60/475] elapsed 0.43 s
[Batch 90/475] elapsed 0.43 s
[Batch 120/475] elapsed 0.43 s
[Batch 150/475] elapsed 0.43 s
[Batch 180/475] elapsed 0.43 s
[Batch 210/475] elapsed 0.43 s
[Batch 240/475] elapsed 0.43 s
[Batch 270/475] elapsed 0.43 s
[Batch 300/475] elapsed 0.43 s
[Batch 330/475] elapsed 0.43 s
[Batch 360/475] elapsed 0.43 s
[Batch 390/475] elapsed 0.43 s
[Batch 420/475] elapsed 0.43 s
[Batch 450/475] elapsed 0.43 s
[Epoch 10] train avg loss 0.012587, test acc 0.6791, test avg loss 0.810256, throughput 3.99724K wps
Observed Improvement.
Begin Testing...
[Batch 30/43] elapsed 0.43 s
[Epoch 11 Batch 30/4268] avg loss 0.0121466, throughput 4.09083K wps
[Epoch 11 Batch 60/4268] avg loss 0.0121762, throughput 4.00559K wps
[Epoch 11 Batch 90/4268] avg loss 0.0114233, throughput 3.99771K wps
[Epoch 11 Batch 120/4268] avg loss 0.0122265, throughput 4.007K wps
[Epoch 11 Batch 150/4268] avg loss 0.0119604, throughput 4.00019K wps
[Epoch 11 Batch 180/4268] avg loss 0.0119876, throughput 3.99355K wps
[Epoch 11 Batch 210/4268] avg loss 0.0115626, throughput 3.99485K wps
[Epoch 11 Batch 240/4268] avg loss 0.0121301, throughput 3.99769K wps
[Epoch 11 Batch 270/4268] avg loss 0.0118817, throughput 3.99707K wps
[Epoch 11 Batch 300/4268] avg loss 0.0121795, throughput 3.99152K wps
[Epoch 11 Batch 330/4268] avg loss 0.0116009, throughput 3.99366K wps
[Epoch 11 Batch 360/4268] avg loss 0.011804, throughput 3.99358K wps
[Epoch 11 Batch 390/4268] avg loss 0.0121538, throughput 3.9947K wps
[Epoch 11 Batch 420/4268] avg loss 0.0120415, throughput 3.99832K wps
[Epoch 11 Batch 450/4268] avg loss 0.0129142, throughput 3.99377K wps
[Epoch 11 Batch 480/4268] avg loss 0.0127267, throughput 3.99676K wps
[Epoch 11 Batch 510/4268] avg loss 0.0120321, throughput 3.9935K wps
[Epoch 11 Batch 540/4268] avg loss 0.0119772, throughput 3.99798K wps
[Epoch 11 Batch 570/4268] avg loss 0.0122046, throughput 3.99426K wps
[Epoch 11 Batch 600/4268] avg loss 0.011668, throughput 4.00055K wps
[Epoch 11 Batch 630/4268] avg loss 0.0113578, throughput 3.99499K wps
[Epoch 11 Batch 660/4268] avg loss 0.0122248, throughput 3.99592K wps
[Epoch 11 Batch 690/4268] avg loss 0.0124614, throughput 3.99776K wps
[Epoch 11 Batch 720/4268] avg loss 0.0123242, throughput 3.99747K wps
[Epoch 11 Batch 750/4268] avg loss 0.0116951, throughput 3.99681K wps
[Epoch 11 Batch 780/4268] avg loss 0.0120633, throughput 3.99991K wps
[Epoch 11 Batch 810/4268] avg loss 0.012358, throughput 3.99716K wps
[Epoch 11 Batch 840/4268] avg loss 0.0118761, throughput 3.99527K wps
[Epoch 11 Batch 870/4268] avg loss 0.0119977, throughput 3.9993K wps
[Epoch 11 Batch 900/4268] avg loss 0.0119909, throughput 3.99971K wps
[Epoch 11 Batch 930/4268] avg loss 0.0121603, throughput 3.99921K wps
[Epoch 11 Batch 960/4268] avg loss 0.0121838, throughput 3.99777K wps
[Epoch 11 Batch 990/4268] avg loss 0.0120452, throughput 4.00184K wps
[Epoch 11 Batch 1020/4268] avg loss 0.0122195, throughput 3.9989K wps
[Epoch 11 Batch 1050/4268] avg loss 0.0127846, throughput 4.00401K wps
[Epoch 11 Batch 1080/4268] avg loss 0.0122319, throughput 3.99663K wps
[Epoch 11 Batch 1110/4268] avg loss 0.0119351, throughput 3.99538K wps
[Epoch 11 Batch 1140/4268] avg loss 0.0131348, throughput 3.99729K wps
[Epoch 11 Batch 1170/4268] avg loss 0.0114464, throughput 3.99678K wps
[Epoch 11 Batch 1200/4268] avg loss 0.0118857, throughput 3.99424K wps
[Epoch 11 Batch 1230/4268] avg loss 0.0110152, throughput 3.99824K wps
[Epoch 11 Batch 1260/4268] avg loss 0.0117797, throughput 4.00026K wps
[Epoch 11 Batch 1290/4268] avg loss 0.0114445, throughput 3.99173K wps
[Epoch 11 Batch 1320/4268] avg loss 0.0117854, throughput 3.99266K wps
[Epoch 11 Batch 1350/4268] avg loss 0.0118248, throughput 3.99255K wps
[Epoch 11 Batch 1380/4268] avg loss 0.0119178, throughput 3.9939K wps
[Epoch 11 Batch 1410/4268] avg loss 0.0120236, throughput 3.9971K wps
[Epoch 11 Batch 1440/4268] avg loss 0.0122324, throughput 3.99588K wps
[Epoch 11 Batch 1470/4268] avg loss 0.0123689, throughput 4.00076K wps
[Epoch 11 Batch 1500/4268] avg loss 0.0123392, throughput 3.99928K wps
[Epoch 11 Batch 1530/4268] avg loss 0.0121987, throughput 4.00177K wps
[Epoch 11 Batch 1560/4268] avg loss 0.0128404, throughput 3.9966K wps
[Epoch 11 Batch 1590/4268] avg loss 0.0126006, throughput 3.99923K wps
[Epoch 11 Batch 1620/4268] avg loss 0.0128075, throughput 3.99881K wps
[Epoch 11 Batch 1650/4268] avg loss 0.0124081, throughput 3.99383K wps
[Epoch 11 Batch 1680/4268] avg loss 0.0127309, throughput 3.98101K wps
[Epoch 11 Batch 1710/4268] avg loss 0.012724, throughput 3.99974K wps
[Epoch 11 Batch 1740/4268] avg loss 0.0120282, throughput 3.99899K wps
[Epoch 11 Batch 1770/4268] avg loss 0.0118485, throughput 3.99515K wps
[Epoch 11 Batch 1800/4268] avg loss 0.0117808, throughput 3.99727K wps
[Epoch 11 Batch 1830/4268] avg loss 0.0118044, throughput 3.99498K wps
[Epoch 11 Batch 1860/4268] avg loss 0.0115909, throughput 3.99248K wps
[Epoch 11 Batch 1890/4268] avg loss 0.0115497, throughput 3.9966K wps
[Epoch 11 Batch 1920/4268] avg loss 0.0123424, throughput 3.99648K wps
[Epoch 11 Batch 1950/4268] avg loss 0.0109838, throughput 3.99561K wps
[Epoch 11 Batch 1980/4268] avg loss 0.0132007, throughput 3.99544K wps
[Epoch 11 Batch 2010/4268] avg loss 0.0124139, throughput 3.99289K wps
[Epoch 11 Batch 2040/4268] avg loss 0.0125721, throughput 4.002K wps
[Epoch 11 Batch 2070/4268] avg loss 0.0118914, throughput 4.00011K wps
[Epoch 11 Batch 2100/4268] avg loss 0.0128003, throughput 3.99783K wps
[Epoch 11 Batch 2130/4268] avg loss 0.012519, throughput 3.99647K wps
[Epoch 11 Batch 2160/4268] avg loss 0.0122161, throughput 3.99527K wps
[Epoch 11 Batch 2190/4268] avg loss 0.0121243, throughput 3.99501K wps
[Epoch 11 Batch 2220/4268] avg loss 0.0128099, throughput 3.99479K wps
[Epoch 11 Batch 2250/4268] avg loss 0.0121131, throughput 3.99725K wps
[Epoch 11 Batch 2280/4268] avg loss 0.0133564, throughput 3.99311K wps
[Epoch 11 Batch 2310/4268] avg loss 0.012229, throughput 3.99391K wps
[Epoch 11 Batch 2340/4268] avg loss 0.0121913, throughput 3.99463K wps
[Epoch 11 Batch 2370/4268] avg loss 0.0125622, throughput 3.99243K wps
[Epoch 11 Batch 2400/4268] avg loss 0.0127384, throughput 3.99565K wps
[Epoch 11 Batch 2430/4268] avg loss 0.0124333, throughput 3.99693K wps
[Epoch 11 Batch 2460/4268] avg loss 0.0129835, throughput 3.99622K wps
[Epoch 11 Batch 2490/4268] avg loss 0.0121399, throughput 3.99771K wps
[Epoch 11 Batch 2520/4268] avg loss 0.0124907, throughput 3.99543K wps
[Epoch 11 Batch 2550/4268] avg loss 0.0125204, throughput 3.99661K wps
[Epoch 11 Batch 2580/4268] avg loss 0.013017, throughput 3.99426K wps
[Epoch 11 Batch 2610/4268] avg loss 0.012931, throughput 3.98953K wps
[Epoch 11 Batch 2640/4268] avg loss 0.0123585, throughput 3.99318K wps
[Epoch 11 Batch 2670/4268] avg loss 0.0125531, throughput 3.99856K wps
[Epoch 11 Batch 2700/4268] avg loss 0.012091, throughput 4.00231K wps
[Epoch 11 Batch 2730/4268] avg loss 0.0120599, throughput 3.99975K wps
[Epoch 11 Batch 2760/4268] avg loss 0.0120308, throughput 4.0005K wps
[Epoch 11 Batch 2790/4268] avg loss 0.0126807, throughput 3.99677K wps
[Epoch 11 Batch 2820/4268] avg loss 0.0118609, throughput 3.993K wps
[Epoch 11 Batch 2850/4268] avg loss 0.0134864, throughput 3.99052K wps
[Epoch 11 Batch 2880/4268] avg loss 0.0131195, throughput 3.99412K wps
[Epoch 11 Batch 2910/4268] avg loss 0.0123697, throughput 3.99922K wps
[Epoch 11 Batch 2940/4268] avg loss 0.0129089, throughput 4.00676K wps
[Epoch 11 Batch 2970/4268] avg loss 0.012657, throughput 3.99806K wps
[Epoch 11 Batch 3000/4268] avg loss 0.0126698, throughput 3.99478K wps
[Epoch 11 Batch 3030/4268] avg loss 0.012732, throughput 3.99932K wps
[Epoch 11 Batch 3060/4268] avg loss 0.012825, throughput 3.99808K wps
[Epoch 11 Batch 3090/4268] avg loss 0.0125398, throughput 3.99885K wps
[Epoch 11 Batch 3120/4268] avg loss 0.0124321, throughput 3.99762K wps
[Epoch 11 Batch 3150/4268] avg loss 0.0126456, throughput 3.99756K wps
[Epoch 11 Batch 3180/4268] avg loss 0.0123579, throughput 3.9984K wps
[Epoch 11 Batch 3210/4268] avg loss 0.0131131, throughput 3.99816K wps
[Epoch 11 Batch 3240/4268] avg loss 0.0122362, throughput 3.99807K wps
[Epoch 11 Batch 3270/4268] avg loss 0.0122721, throughput 3.99343K wps
[Epoch 11 Batch 3300/4268] avg loss 0.012802, throughput 3.99526K wps
[Epoch 11 Batch 3330/4268] avg loss 0.013494, throughput 3.99706K wps
[Epoch 11 Batch 3360/4268] avg loss 0.0127771, throughput 4.00326K wps
[Epoch 11 Batch 3390/4268] avg loss 0.0130169, throughput 4.00067K wps
[Epoch 11 Batch 3420/4268] avg loss 0.012346, throughput 3.99585K wps
[Epoch 11 Batch 3450/4268] avg loss 0.0119764, throughput 3.99523K wps
[Epoch 11 Batch 3480/4268] avg loss 0.0131574, throughput 3.99396K wps
[Epoch 11 Batch 3510/4268] avg loss 0.0123472, throughput 3.99628K wps
[Epoch 11 Batch 3540/4268] avg loss 0.0131243, throughput 3.99661K wps
[Epoch 11 Batch 3570/4268] avg loss 0.0123872, throughput 3.99384K wps
[Epoch 11 Batch 3600/4268] avg loss 0.0129874, throughput 3.99411K wps
[Epoch 11 Batch 3630/4268] avg loss 0.0129441, throughput 3.99557K wps
[Epoch 11 Batch 3660/4268] avg loss 0.0126842, throughput 3.99674K wps
[Epoch 11 Batch 3690/4268] avg loss 0.0126023, throughput 3.99797K wps
[Epoch 11 Batch 3720/4268] avg loss 0.0122391, throughput 3.99697K wps
[Epoch 11 Batch 3750/4268] avg loss 0.0125435, throughput 3.99663K wps
[Epoch 11 Batch 3780/4268] avg loss 0.0128674, throughput 3.9989K wps
[Epoch 11 Batch 3810/4268] avg loss 0.0121828, throughput 3.99684K wps
[Epoch 11 Batch 3840/4268] avg loss 0.0130234, throughput 3.99565K wps
[Epoch 11 Batch 3870/4268] avg loss 0.0123775, throughput 3.99726K wps
[Epoch 11 Batch 3900/4268] avg loss 0.0134661, throughput 3.99755K wps
[Epoch 11 Batch 3930/4268] avg loss 0.0125273, throughput 3.99442K wps
[Epoch 11 Batch 3960/4268] avg loss 0.0127141, throughput 3.99425K wps
[Epoch 11 Batch 3990/4268] avg loss 0.0127824, throughput 3.99588K wps
[Epoch 11 Batch 4020/4268] avg loss 0.0122756, throughput 3.99693K wps
[Epoch 11 Batch 4050/4268] avg loss 0.0129043, throughput 3.99576K wps
[Epoch 11 Batch 4080/4268] avg loss 0.0124369, throughput 3.99705K wps
[Epoch 11 Batch 4110/4268] avg loss 0.0136191, throughput 3.99513K wps
[Epoch 11 Batch 4140/4268] avg loss 0.0126068, throughput 3.99496K wps
[Epoch 11 Batch 4170/4268] avg loss 0.0127537, throughput 3.9964K wps
[Epoch 11 Batch 4200/4268] avg loss 0.012692, throughput 3.99538K wps
[Epoch 11 Batch 4230/4268] avg loss 0.0127242, throughput 3.99674K wps
[Epoch 11 Batch 4260/4268] avg loss 0.0119344, throughput 3.9928K wps
Begin Testing...
[Batch 30/475] elapsed 0.46 s
[Batch 60/475] elapsed 0.43 s
[Batch 90/475] elapsed 0.43 s
[Batch 120/475] elapsed 0.43 s
[Batch 150/475] elapsed 0.43 s
[Batch 180/475] elapsed 0.43 s
[Batch 210/475] elapsed 0.43 s
[Batch 240/475] elapsed 0.43 s
[Batch 270/475] elapsed 0.43 s
[Batch 300/475] elapsed 0.43 s
[Batch 330/475] elapsed 0.43 s
[Batch 360/475] elapsed 0.43 s
[Batch 390/475] elapsed 0.43 s
[Batch 420/475] elapsed 0.43 s
[Batch 450/475] elapsed 0.43 s
[Epoch 11] train avg loss 0.0123643, test acc 0.6776, test avg loss 0.818379, throughput 3.99727K wps
[Epoch 12 Batch 30/4268] avg loss 0.0111271, throughput 4.08503K wps
[Epoch 12 Batch 60/4268] avg loss 0.0117921, throughput 3.99297K wps
[Epoch 12 Batch 90/4268] avg loss 0.0111926, throughput 3.9971K wps
[Epoch 12 Batch 120/4268] avg loss 0.011981, throughput 3.9961K wps
[Epoch 12 Batch 150/4268] avg loss 0.0111922, throughput 3.98993K wps
[Epoch 12 Batch 180/4268] avg loss 0.0118564, throughput 3.99302K wps
[Epoch 12 Batch 210/4268] avg loss 0.0122898, throughput 3.99143K wps
[Epoch 12 Batch 240/4268] avg loss 0.0118254, throughput 3.99208K wps
[Epoch 12 Batch 270/4268] avg loss 0.0116372, throughput 3.99684K wps
[Epoch 12 Batch 300/4268] avg loss 0.010885, throughput 3.99209K wps
[Epoch 12 Batch 330/4268] avg loss 0.0116242, throughput 3.99111K wps
[Epoch 12 Batch 360/4268] avg loss 0.0122494, throughput 3.9945K wps
[Epoch 12 Batch 390/4268] avg loss 0.0116943, throughput 3.99041K wps
[Epoch 12 Batch 420/4268] avg loss 0.0117908, throughput 3.99724K wps
[Epoch 12 Batch 450/4268] avg loss 0.0115242, throughput 3.99824K wps
[Epoch 12 Batch 480/4268] avg loss 0.0116974, throughput 3.99515K wps
[Epoch 12 Batch 510/4268] avg loss 0.0116689, throughput 3.99225K wps
[Epoch 12 Batch 540/4268] avg loss 0.011367, throughput 3.99279K wps
[Epoch 12 Batch 570/4268] avg loss 0.0121756, throughput 3.99648K wps
[Epoch 12 Batch 600/4268] avg loss 0.0118855, throughput 3.99722K wps
[Epoch 12 Batch 630/4268] avg loss 0.0120539, throughput 3.99828K wps
[Epoch 12 Batch 660/4268] avg loss 0.0123665, throughput 3.99929K wps
[Epoch 12 Batch 690/4268] avg loss 0.0119417, throughput 3.99968K wps
[Epoch 12 Batch 720/4268] avg loss 0.0117772, throughput 4.00047K wps
[Epoch 12 Batch 750/4268] avg loss 0.0120543, throughput 4K wps
[Epoch 12 Batch 780/4268] avg loss 0.0120199, throughput 3.9978K wps
[Epoch 12 Batch 810/4268] avg loss 0.0112713, throughput 3.99341K wps
[Epoch 12 Batch 840/4268] avg loss 0.0119454, throughput 3.99376K wps
[Epoch 12 Batch 870/4268] avg loss 0.0116545, throughput 3.97479K wps
[Epoch 12 Batch 900/4268] avg loss 0.0111419, throughput 3.99894K wps
[Epoch 12 Batch 930/4268] avg loss 0.0116433, throughput 3.99964K wps
[Epoch 12 Batch 960/4268] avg loss 0.0115424, throughput 3.99492K wps
[Epoch 12 Batch 990/4268] avg loss 0.0119373, throughput 3.99295K wps
[Epoch 12 Batch 1020/4268] avg loss 0.0115213, throughput 3.99952K wps
[Epoch 12 Batch 1050/4268] avg loss 0.0119026, throughput 4.0006K wps
[Epoch 12 Batch 1080/4268] avg loss 0.0121437, throughput 3.99566K wps
[Epoch 12 Batch 1110/4268] avg loss 0.012205, throughput 3.996K wps
[Epoch 12 Batch 1140/4268] avg loss 0.0117316, throughput 3.99444K wps
[Epoch 12 Batch 1170/4268] avg loss 0.012281, throughput 3.99825K wps
[Epoch 12 Batch 1200/4268] avg loss 0.0124646, throughput 3.99724K wps
[Epoch 12 Batch 1230/4268] avg loss 0.0118505, throughput 3.99403K wps
[Epoch 12 Batch 1260/4268] avg loss 0.0115469, throughput 3.99834K wps
[Epoch 12 Batch 1290/4268] avg loss 0.0117142, throughput 3.99592K wps
[Epoch 12 Batch 1320/4268] avg loss 0.0115275, throughput 3.99473K wps
[Epoch 12 Batch 1350/4268] avg loss 0.0119657, throughput 3.99382K wps
[Epoch 12 Batch 1380/4268] avg loss 0.0115282, throughput 3.99189K wps
[Epoch 12 Batch 1410/4268] avg loss 0.012571, throughput 3.99945K wps
[Epoch 12 Batch 1440/4268] avg loss 0.0122305, throughput 3.99404K wps
[Epoch 12 Batch 1470/4268] avg loss 0.0122324, throughput 4.00157K wps
[Epoch 12 Batch 1500/4268] avg loss 0.0125522, throughput 3.99892K wps
[Epoch 12 Batch 1530/4268] avg loss 0.0115586, throughput 3.99406K wps
[Epoch 12 Batch 1560/4268] avg loss 0.0120805, throughput 4.00241K wps
[Epoch 12 Batch 1590/4268] avg loss 0.0122266, throughput 3.9974K wps
[Epoch 12 Batch 1620/4268] avg loss 0.0121408, throughput 3.99447K wps
[Epoch 12 Batch 1650/4268] avg loss 0.012726, throughput 3.99663K wps
[Epoch 12 Batch 1680/4268] avg loss 0.0119497, throughput 3.9924K wps
[Epoch 12 Batch 1710/4268] avg loss 0.0124853, throughput 3.99219K wps
[Epoch 12 Batch 1740/4268] avg loss 0.0120828, throughput 3.99945K wps
[Epoch 12 Batch 1770/4268] avg loss 0.011831, throughput 3.99373K wps
[Epoch 12 Batch 1800/4268] avg loss 0.0118025, throughput 3.99756K wps
[Epoch 12 Batch 1830/4268] avg loss 0.0121235, throughput 4.00349K wps
[Epoch 12 Batch 1860/4268] avg loss 0.0128575, throughput 4.00014K wps
[Epoch 12 Batch 1890/4268] avg loss 0.0117146, throughput 4.00154K wps
[Epoch 12 Batch 1920/4268] avg loss 0.0128442, throughput 4.00275K wps
[Epoch 12 Batch 1950/4268] avg loss 0.0120671, throughput 4.00061K wps
[Epoch 12 Batch 1980/4268] avg loss 0.0119916, throughput 3.99478K wps
[Epoch 12 Batch 2010/4268] avg loss 0.0120212, throughput 3.99634K wps
[Epoch 12 Batch 2040/4268] avg loss 0.0119796, throughput 3.99798K wps
[Epoch 12 Batch 2070/4268] avg loss 0.0124222, throughput 3.99257K wps
[Epoch 12 Batch 2100/4268] avg loss 0.012222, throughput 3.99629K wps
[Epoch 12 Batch 2130/4268] avg loss 0.0119196, throughput 3.99498K wps
[Epoch 12 Batch 2160/4268] avg loss 0.0119105, throughput 3.99723K wps
[Epoch 12 Batch 2190/4268] avg loss 0.0116692, throughput 3.9937K wps
[Epoch 12 Batch 2220/4268] avg loss 0.0121475, throughput 3.9922K wps
[Epoch 12 Batch 2250/4268] avg loss 0.0119164, throughput 3.99875K wps
[Epoch 12 Batch 2280/4268] avg loss 0.0113975, throughput 3.99638K wps
[Epoch 12 Batch 2310/4268] avg loss 0.012075, throughput 3.99846K wps
[Epoch 12 Batch 2340/4268] avg loss 0.0121454, throughput 3.99638K wps
[Epoch 12 Batch 2370/4268] avg loss 0.0116707, throughput 3.99628K wps
[Epoch 12 Batch 2400/4268] avg loss 0.0129924, throughput 3.99589K wps
[Epoch 12 Batch 2430/4268] avg loss 0.0121369, throughput 3.99667K wps
[Epoch 12 Batch 2460/4268] avg loss 0.0118623, throughput 3.99242K wps
[Epoch 12 Batch 2490/4268] avg loss 0.0123294, throughput 3.99352K wps
[Epoch 12 Batch 2520/4268] avg loss 0.0119388, throughput 3.99319K wps
[Epoch 12 Batch 2550/4268] avg loss 0.0124979, throughput 3.994K wps
[Epoch 12 Batch 2580/4268] avg loss 0.0118323, throughput 3.99233K wps
[Epoch 12 Batch 2610/4268] avg loss 0.0129143, throughput 3.99429K wps
[Epoch 12 Batch 2640/4268] avg loss 0.0120855, throughput 3.99393K wps
[Epoch 12 Batch 2670/4268] avg loss 0.0122099, throughput 3.99487K wps
[Epoch 12 Batch 2700/4268] avg loss 0.0126509, throughput 4.00641K wps
[Epoch 12 Batch 2730/4268] avg loss 0.0124862, throughput 4.00216K wps
[Epoch 12 Batch 2760/4268] avg loss 0.01235, throughput 3.99842K wps
[Epoch 12 Batch 2790/4268] avg loss 0.0120128, throughput 4.00444K wps
[Epoch 12 Batch 2820/4268] avg loss 0.0125727, throughput 4.00015K wps
[Epoch 12 Batch 2850/4268] avg loss 0.0123619, throughput 3.99623K wps
[Epoch 12 Batch 2880/4268] avg loss 0.0123103, throughput 3.99202K wps
[Epoch 12 Batch 2910/4268] avg loss 0.0120163, throughput 3.99761K wps
[Epoch 12 Batch 2940/4268] avg loss 0.0124138, throughput 3.99827K wps
[Epoch 12 Batch 2970/4268] avg loss 0.0123179, throughput 3.99319K wps
[Epoch 12 Batch 3000/4268] avg loss 0.0120793, throughput 3.99797K wps
[Epoch 12 Batch 3030/4268] avg loss 0.0116812, throughput 3.99384K wps
[Epoch 12 Batch 3060/4268] avg loss 0.0124783, throughput 3.9988K wps
[Epoch 12 Batch 3090/4268] avg loss 0.0126216, throughput 3.99531K wps
[Epoch 12 Batch 3120/4268] avg loss 0.0124899, throughput 3.99341K wps
[Epoch 12 Batch 3150/4268] avg loss 0.0125144, throughput 3.99467K wps
[Epoch 12 Batch 3180/4268] avg loss 0.0123569, throughput 3.99412K wps
[Epoch 12 Batch 3210/4268] avg loss 0.0120644, throughput 4.00031K wps
[Epoch 12 Batch 3240/4268] avg loss 0.0117236, throughput 3.99839K wps
[Epoch 12 Batch 3270/4268] avg loss 0.0120527, throughput 3.99778K wps
[Epoch 12 Batch 3300/4268] avg loss 0.0120381, throughput 3.99893K wps
[Epoch 12 Batch 3330/4268] avg loss 0.0123762, throughput 3.99798K wps
[Epoch 12 Batch 3360/4268] avg loss 0.01211, throughput 4.00455K wps
[Epoch 12 Batch 3390/4268] avg loss 0.0122182, throughput 4.00158K wps
[Epoch 12 Batch 3420/4268] avg loss 0.0123538, throughput 4.00381K wps
[Epoch 12 Batch 3450/4268] avg loss 0.0127883, throughput 3.99552K wps
[Epoch 12 Batch 3480/4268] avg loss 0.0124204, throughput 3.99213K wps
[Epoch 12 Batch 3510/4268] avg loss 0.0122229, throughput 3.99549K wps
[Epoch 12 Batch 3540/4268] avg loss 0.0120762, throughput 3.99374K wps
[Epoch 12 Batch 3570/4268] avg loss 0.012766, throughput 3.98971K wps
[Epoch 12 Batch 3600/4268] avg loss 0.0121712, throughput 3.99551K wps
[Epoch 12 Batch 3630/4268] avg loss 0.0112717, throughput 3.99923K wps
[Epoch 12 Batch 3660/4268] avg loss 0.0125115, throughput 3.99799K wps
[Epoch 12 Batch 3690/4268] avg loss 0.0121902, throughput 3.99961K wps
[Epoch 12 Batch 3720/4268] avg loss 0.012558, throughput 3.99144K wps
[Epoch 12 Batch 3750/4268] avg loss 0.0122427, throughput 3.99506K wps
[Epoch 12 Batch 3780/4268] avg loss 0.0132468, throughput 3.99921K wps
[Epoch 12 Batch 3810/4268] avg loss 0.0119047, throughput 3.99083K wps
[Epoch 12 Batch 3840/4268] avg loss 0.0126561, throughput 3.99269K wps
[Epoch 12 Batch 3870/4268] avg loss 0.0119978, throughput 3.996K wps
[Epoch 12 Batch 3900/4268] avg loss 0.0126447, throughput 3.99397K wps
[Epoch 12 Batch 3930/4268] avg loss 0.0122678, throughput 3.9944K wps
[Epoch 12 Batch 3960/4268] avg loss 0.0125732, throughput 3.99382K wps
[Epoch 12 Batch 3990/4268] avg loss 0.0120134, throughput 3.99295K wps
[Epoch 12 Batch 4020/4268] avg loss 0.0117732, throughput 3.99799K wps
[Epoch 12 Batch 4050/4268] avg loss 0.0126307, throughput 3.9953K wps
[Epoch 12 Batch 4080/4268] avg loss 0.0124515, throughput 3.99646K wps
[Epoch 12 Batch 4110/4268] avg loss 0.0119824, throughput 3.99911K wps
[Epoch 12 Batch 4140/4268] avg loss 0.0116164, throughput 3.99669K wps
[Epoch 12 Batch 4170/4268] avg loss 0.0117837, throughput 3.99729K wps
[Epoch 12 Batch 4200/4268] avg loss 0.0124101, throughput 3.99556K wps
[Epoch 12 Batch 4230/4268] avg loss 0.0122676, throughput 3.99419K wps
[Epoch 12 Batch 4260/4268] avg loss 0.0123578, throughput 3.99377K wps
Begin Testing...
[Batch 30/475] elapsed 0.46 s
[Batch 60/475] elapsed 0.43 s
[Batch 90/475] elapsed 0.43 s
[Batch 120/475] elapsed 0.43 s
[Batch 150/475] elapsed 0.43 s
[Batch 180/475] elapsed 0.43 s
[Batch 210/475] elapsed 0.43 s
[Batch 240/475] elapsed 0.43 s
[Batch 270/475] elapsed 0.43 s
[Batch 300/475] elapsed 0.43 s
[Batch 330/475] elapsed 0.43 s
[Batch 360/475] elapsed 0.43 s
[Batch 390/475] elapsed 0.43 s
[Batch 420/475] elapsed 0.43 s
[Batch 450/475] elapsed 0.43 s
[Epoch 12] train avg loss 0.0120709, test acc 0.6785, test avg loss 0.828516, throughput 3.99671K wps
[Epoch 13 Batch 30/4268] avg loss 0.0116006, throughput 4.07668K wps
[Epoch 13 Batch 60/4268] avg loss 0.0108582, throughput 3.97729K wps
[Epoch 13 Batch 90/4268] avg loss 0.0116548, throughput 3.99263K wps
[Epoch 13 Batch 120/4268] avg loss 0.01086, throughput 3.99591K wps
[Epoch 13 Batch 150/4268] avg loss 0.011318, throughput 3.99695K wps
[Epoch 13 Batch 180/4268] avg loss 0.0116203, throughput 3.99548K wps
[Epoch 13 Batch 210/4268] avg loss 0.011862, throughput 3.99683K wps
[Epoch 13 Batch 240/4268] avg loss 0.0113698, throughput 3.99384K wps
[Epoch 13 Batch 270/4268] avg loss 0.0111196, throughput 3.9924K wps
[Epoch 13 Batch 300/4268] avg loss 0.0113637, throughput 3.99675K wps
[Epoch 13 Batch 330/4268] avg loss 0.0109972, throughput 3.99619K wps
[Epoch 13 Batch 360/4268] avg loss 0.0120591, throughput 3.9958K wps
[Epoch 13 Batch 390/4268] avg loss 0.0117811, throughput 3.99271K wps
[Epoch 13 Batch 420/4268] avg loss 0.0116542, throughput 3.99733K wps
[Epoch 13 Batch 450/4268] avg loss 0.0111006, throughput 3.99873K wps
[Epoch 13 Batch 480/4268] avg loss 0.011029, throughput 4.00062K wps
[Epoch 13 Batch 510/4268] avg loss 0.0107907, throughput 3.99885K wps
[Epoch 13 Batch 540/4268] avg loss 0.0118417, throughput 3.99683K wps
[Epoch 13 Batch 570/4268] avg loss 0.0114357, throughput 3.99931K wps
[Epoch 13 Batch 600/4268] avg loss 0.0122884, throughput 4.00131K wps
[Epoch 13 Batch 630/4268] avg loss 0.0120294, throughput 3.99808K wps
[Epoch 13 Batch 660/4268] avg loss 0.0117254, throughput 3.99819K wps
[Epoch 13 Batch 690/4268] avg loss 0.0118319, throughput 4.00432K wps
[Epoch 13 Batch 720/4268] avg loss 0.0113247, throughput 4.00315K wps
[Epoch 13 Batch 750/4268] avg loss 0.0114148, throughput 3.99924K wps
[Epoch 13 Batch 780/4268] avg loss 0.0112362, throughput 4.00008K wps
[Epoch 13 Batch 810/4268] avg loss 0.0118244, throughput 4.00049K wps
[Epoch 13 Batch 840/4268] avg loss 0.0120824, throughput 3.99011K wps
[Epoch 13 Batch 870/4268] avg loss 0.0116289, throughput 3.99435K wps
[Epoch 13 Batch 900/4268] avg loss 0.0113214, throughput 3.99203K wps
[Epoch 13 Batch 930/4268] avg loss 0.0121095, throughput 3.99094K wps
[Epoch 13 Batch 960/4268] avg loss 0.0112264, throughput 3.99837K wps
[Epoch 13 Batch 990/4268] avg loss 0.0110659, throughput 3.99492K wps
[Epoch 13 Batch 1020/4268] avg loss 0.0115078, throughput 4.00044K wps
[Epoch 13 Batch 1050/4268] avg loss 0.0118136, throughput 3.99316K wps
[Epoch 13 Batch 1080/4268] avg loss 0.0115328, throughput 3.99402K wps
[Epoch 13 Batch 1110/4268] avg loss 0.0117543, throughput 3.99716K wps
[Epoch 13 Batch 1140/4268] avg loss 0.0121693, throughput 3.99452K wps
[Epoch 13 Batch 1170/4268] avg loss 0.0114762, throughput 3.99348K wps
[Epoch 13 Batch 1200/4268] avg loss 0.0112898, throughput 3.99057K wps
[Epoch 13 Batch 1230/4268] avg loss 0.0118282, throughput 3.99103K wps
[Epoch 13 Batch 1260/4268] avg loss 0.0116682, throughput 3.99506K wps
[Epoch 13 Batch 1290/4268] avg loss 0.0116367, throughput 3.99336K wps
[Epoch 13 Batch 1320/4268] avg loss 0.0114033, throughput 3.99815K wps
[Epoch 13 Batch 1350/4268] avg loss 0.0120698, throughput 3.99581K wps
[Epoch 13 Batch 1380/4268] avg loss 0.0111223, throughput 4.00476K wps
[Epoch 13 Batch 1410/4268] avg loss 0.0115265, throughput 3.99827K wps
[Epoch 13 Batch 1440/4268] avg loss 0.0115453, throughput 3.99339K wps
[Epoch 13 Batch 1470/4268] avg loss 0.0121112, throughput 3.99758K wps
[Epoch 13 Batch 1500/4268] avg loss 0.0120641, throughput 3.99608K wps
[Epoch 13 Batch 1530/4268] avg loss 0.0116578, throughput 3.99981K wps
[Epoch 13 Batch 1560/4268] avg loss 0.0113873, throughput 3.99903K wps
[Epoch 13 Batch 1590/4268] avg loss 0.0117497, throughput 4.00419K wps
[Epoch 13 Batch 1620/4268] avg loss 0.0110803, throughput 3.9942K wps
[Epoch 13 Batch 1650/4268] avg loss 0.0123464, throughput 3.99449K wps
[Epoch 13 Batch 1680/4268] avg loss 0.0117235, throughput 3.99246K wps
[Epoch 13 Batch 1710/4268] avg loss 0.0115129, throughput 3.99566K wps
[Epoch 13 Batch 1740/4268] avg loss 0.0112299, throughput 3.99907K wps
[Epoch 13 Batch 1770/4268] avg loss 0.0122104, throughput 3.99628K wps
[Epoch 13 Batch 1800/4268] avg loss 0.0115581, throughput 3.99669K wps
[Epoch 13 Batch 1830/4268] avg loss 0.0124701, throughput 3.99967K wps
[Epoch 13 Batch 1860/4268] avg loss 0.0120491, throughput 4.00003K wps
[Epoch 13 Batch 1890/4268] avg loss 0.011146, throughput 3.99768K wps
[Epoch 13 Batch 1920/4268] avg loss 0.0113058, throughput 3.9987K wps
[Epoch 13 Batch 1950/4268] avg loss 0.012648, throughput 3.99722K wps
[Epoch 13 Batch 1980/4268] avg loss 0.0118137, throughput 3.995K wps
[Epoch 13 Batch 2010/4268] avg loss 0.0126354, throughput 3.99771K wps
[Epoch 13 Batch 2040/4268] avg loss 0.0121957, throughput 3.99736K wps
[Epoch 13 Batch 2070/4268] avg loss 0.011103, throughput 3.9946K wps
[Epoch 13 Batch 2100/4268] avg loss 0.0110907, throughput 3.99576K wps
[Epoch 13 Batch 2130/4268] avg loss 0.0120198, throughput 3.99635K wps
[Epoch 13 Batch 2160/4268] avg loss 0.0119288, throughput 3.99824K wps
[Epoch 13 Batch 2190/4268] avg loss 0.0117453, throughput 3.9978K wps
[Epoch 13 Batch 2220/4268] avg loss 0.0121064, throughput 3.99874K wps
[Epoch 13 Batch 2250/4268] avg loss 0.0120862, throughput 3.9969K wps
[Epoch 13 Batch 2280/4268] avg loss 0.0114351, throughput 3.99362K wps
[Epoch 13 Batch 2310/4268] avg loss 0.0121907, throughput 3.9968K wps
[Epoch 13 Batch 2340/4268] avg loss 0.0126103, throughput 3.99649K wps
[Epoch 13 Batch 2370/4268] avg loss 0.0125424, throughput 3.99761K wps
[Epoch 13 Batch 2400/4268] avg loss 0.0115518, throughput 3.99651K wps
[Epoch 13 Batch 2430/4268] avg loss 0.012159, throughput 3.99654K wps
[Epoch 13 Batch 2460/4268] avg loss 0.0113011, throughput 3.99739K wps
[Epoch 13 Batch 2490/4268] avg loss 0.0121583, throughput 3.9999K wps
[Epoch 13 Batch 2520/4268] avg loss 0.0117733, throughput 3.99498K wps
[Epoch 13 Batch 2550/4268] avg loss 0.0121775, throughput 3.9943K wps
[Epoch 13 Batch 2580/4268] avg loss 0.0120131, throughput 3.99763K wps
[Epoch 13 Batch 2610/4268] avg loss 0.0118423, throughput 3.99579K wps
[Epoch 13 Batch 2640/4268] avg loss 0.0127791, throughput 3.99688K wps
[Epoch 13 Batch 2670/4268] avg loss 0.0115494, throughput 3.99824K wps
[Epoch 13 Batch 2700/4268] avg loss 0.0121371, throughput 3.99849K wps
[Epoch 13 Batch 2730/4268] avg loss 0.0124531, throughput 3.99715K wps
[Epoch 13 Batch 2760/4268] avg loss 0.0126308, throughput 3.99552K wps
[Epoch 13 Batch 2790/4268] avg loss 0.0119297, throughput 4.00019K wps
[Epoch 13 Batch 2820/4268] avg loss 0.0120594, throughput 3.99647K wps
[Epoch 13 Batch 2850/4268] avg loss 0.0121029, throughput 3.99266K wps
[Epoch 13 Batch 2880/4268] avg loss 0.0123848, throughput 3.9988K wps
[Epoch 13 Batch 2910/4268] avg loss 0.0114812, throughput 3.99625K wps
[Epoch 13 Batch 2940/4268] avg loss 0.0124453, throughput 3.99768K wps
[Epoch 13 Batch 2970/4268] avg loss 0.0122411, throughput 3.99797K wps
[Epoch 13 Batch 3000/4268] avg loss 0.0121184, throughput 3.99355K wps
[Epoch 13 Batch 3030/4268] avg loss 0.0120914, throughput 3.99682K wps
[Epoch 13 Batch 3060/4268] avg loss 0.01164, throughput 3.99761K wps
[Epoch 13 Batch 3090/4268] avg loss 0.0120886, throughput 3.99722K wps
[Epoch 13 Batch 3120/4268] avg loss 0.0123183, throughput 3.99622K wps
[Epoch 13 Batch 3150/4268] avg loss 0.0125072, throughput 3.99459K wps
[Epoch 13 Batch 3180/4268] avg loss 0.0114636, throughput 3.99918K wps
[Epoch 13 Batch 3210/4268] avg loss 0.0119648, throughput 3.99796K wps
[Epoch 13 Batch 3240/4268] avg loss 0.012207, throughput 3.99483K wps
[Epoch 13 Batch 3270/4268] avg loss 0.011465, throughput 3.99607K wps
[Epoch 13 Batch 3300/4268] avg loss 0.0118765, throughput 3.99575K wps
[Epoch 13 Batch 3330/4268] avg loss 0.0122086, throughput 3.99677K wps
[Epoch 13 Batch 3360/4268] avg loss 0.0114587, throughput 3.9894K wps
[Epoch 13 Batch 3390/4268] avg loss 0.0123368, throughput 3.99224K wps
[Epoch 13 Batch 3420/4268] avg loss 0.0121087, throughput 3.99442K wps
[Epoch 13 Batch 3450/4268] avg loss 0.0118323, throughput 3.99701K wps
[Epoch 13 Batch 3480/4268] avg loss 0.0116439, throughput 3.99997K wps
[Epoch 13 Batch 3510/4268] avg loss 0.0122607, throughput 3.99316K wps
[Epoch 13 Batch 3540/4268] avg loss 0.0118223, throughput 3.99304K wps
[Epoch 13 Batch 3570/4268] avg loss 0.0121466, throughput 3.99517K wps
[Epoch 13 Batch 3600/4268] avg loss 0.0119248, throughput 3.99171K wps
[Epoch 13 Batch 3630/4268] avg loss 0.0121606, throughput 3.98206K wps
[Epoch 13 Batch 3660/4268] avg loss 0.0119401, throughput 3.98438K wps
[Epoch 13 Batch 3690/4268] avg loss 0.0124757, throughput 3.99404K wps
[Epoch 13 Batch 3720/4268] avg loss 0.0117024, throughput 3.99526K wps
[Epoch 13 Batch 3750/4268] avg loss 0.0122978, throughput 3.9984K wps
[Epoch 13 Batch 3780/4268] avg loss 0.0117963, throughput 3.99807K wps
[Epoch 13 Batch 3810/4268] avg loss 0.0120494, throughput 4K wps
[Epoch 13 Batch 3840/4268] avg loss 0.0116782, throughput 3.99457K wps
[Epoch 13 Batch 3870/4268] avg loss 0.0117345, throughput 3.99676K wps
[Epoch 13 Batch 3900/4268] avg loss 0.0127647, throughput 3.99904K wps
[Epoch 13 Batch 3930/4268] avg loss 0.0117516, throughput 3.99531K wps
[Epoch 13 Batch 3960/4268] avg loss 0.0122156, throughput 4.00024K wps
[Epoch 13 Batch 3990/4268] avg loss 0.0125367, throughput 3.99687K wps
[Epoch 13 Batch 4020/4268] avg loss 0.0122241, throughput 3.99551K wps
[Epoch 13 Batch 4050/4268] avg loss 0.0117736, throughput 3.99557K wps
[Epoch 13 Batch 4080/4268] avg loss 0.0129807, throughput 3.99392K wps
[Epoch 13 Batch 4110/4268] avg loss 0.012128, throughput 3.9921K wps
[Epoch 13 Batch 4140/4268] avg loss 0.0124467, throughput 3.99526K wps
[Epoch 13 Batch 4170/4268] avg loss 0.0121168, throughput 3.99322K wps
[Epoch 13 Batch 4200/4268] avg loss 0.0125379, throughput 3.99442K wps
[Epoch 13 Batch 4230/4268] avg loss 0.0124469, throughput 3.99549K wps
[Epoch 13 Batch 4260/4268] avg loss 0.0125627, throughput 3.99983K wps
Begin Testing...
[Batch 30/475] elapsed 0.46 s
[Batch 60/475] elapsed 0.43 s
[Batch 90/475] elapsed 0.43 s
[Batch 120/475] elapsed 0.43 s
[Batch 150/475] elapsed 0.43 s
[Batch 180/475] elapsed 0.43 s
[Batch 210/475] elapsed 0.43 s
[Batch 240/475] elapsed 0.43 s
[Batch 270/475] elapsed 0.43 s
[Batch 300/475] elapsed 0.43 s
[Batch 330/475] elapsed 0.43 s
[Batch 360/475] elapsed 0.43 s
[Batch 390/475] elapsed 0.43 s
[Batch 420/475] elapsed 0.43 s
[Batch 450/475] elapsed 0.43 s
[Epoch 13] train avg loss 0.0118484, test acc 0.6768, test avg loss 0.831977, throughput 3.99667K wps
[Epoch 14 Batch 30/4268] avg loss 0.0112168, throughput 4.07715K wps
[Epoch 14 Batch 60/4268] avg loss 0.0114081, throughput 3.99499K wps
[Epoch 14 Batch 90/4268] avg loss 0.0113365, throughput 3.99688K wps
[Epoch 14 Batch 120/4268] avg loss 0.0111181, throughput 3.99761K wps
[Epoch 14 Batch 150/4268] avg loss 0.0108789, throughput 3.99852K wps
[Epoch 14 Batch 180/4268] avg loss 0.011074, throughput 3.99539K wps
[Epoch 14 Batch 210/4268] avg loss 0.0112495, throughput 3.99673K wps
[Epoch 14 Batch 240/4268] avg loss 0.0109567, throughput 3.99834K wps
[Epoch 14 Batch 270/4268] avg loss 0.0112736, throughput 3.99559K wps
[Epoch 14 Batch 300/4268] avg loss 0.011052, throughput 3.99781K wps
[Epoch 14 Batch 330/4268] avg loss 0.011038, throughput 3.99163K wps
[Epoch 14 Batch 360/4268] avg loss 0.0112755, throughput 3.99797K wps
[Epoch 14 Batch 390/4268] avg loss 0.0112936, throughput 3.99583K wps
[Epoch 14 Batch 420/4268] avg loss 0.0106958, throughput 3.99629K wps
[Epoch 14 Batch 450/4268] avg loss 0.011844, throughput 3.99821K wps
[Epoch 14 Batch 480/4268] avg loss 0.0120592, throughput 3.99695K wps
[Epoch 14 Batch 510/4268] avg loss 0.0109518, throughput 4.00686K wps
[Epoch 14 Batch 540/4268] avg loss 0.0110055, throughput 3.99884K wps
[Epoch 14 Batch 570/4268] avg loss 0.0109607, throughput 3.99498K wps
[Epoch 14 Batch 600/4268] avg loss 0.0110964, throughput 3.99575K wps
[Epoch 14 Batch 630/4268] avg loss 0.011873, throughput 3.99709K wps
[Epoch 14 Batch 660/4268] avg loss 0.0116016, throughput 3.99817K wps
[Epoch 14 Batch 690/4268] avg loss 0.0112328, throughput 3.99869K wps
[Epoch 14 Batch 720/4268] avg loss 0.0112061, throughput 3.99752K wps
[Epoch 14 Batch 750/4268] avg loss 0.0123038, throughput 3.99917K wps
[Epoch 14 Batch 780/4268] avg loss 0.0107836, throughput 3.99874K wps
[Epoch 14 Batch 810/4268] avg loss 0.0115719, throughput 3.99436K wps
[Epoch 14 Batch 840/4268] avg loss 0.010829, throughput 3.99627K wps
[Epoch 14 Batch 870/4268] avg loss 0.0120489, throughput 3.99674K wps
[Epoch 14 Batch 900/4268] avg loss 0.0114694, throughput 3.99872K wps
[Epoch 14 Batch 930/4268] avg loss 0.0118443, throughput 3.99762K wps
[Epoch 14 Batch 960/4268] avg loss 0.011233, throughput 3.99639K wps
[Epoch 14 Batch 990/4268] avg loss 0.0109762, throughput 3.99633K wps
[Epoch 14 Batch 1020/4268] avg loss 0.0111817, throughput 3.99679K wps
[Epoch 14 Batch 1050/4268] avg loss 0.0110934, throughput 3.99468K wps
[Epoch 14 Batch 1080/4268] avg loss 0.0117267, throughput 3.99508K wps
[Epoch 14 Batch 1110/4268] avg loss 0.0102884, throughput 3.99709K wps
[Epoch 14 Batch 1140/4268] avg loss 0.0116417, throughput 3.99313K wps
[Epoch 14 Batch 1170/4268] avg loss 0.0115239, throughput 3.99217K wps
[Epoch 14 Batch 1200/4268] avg loss 0.0109261, throughput 3.99941K wps
[Epoch 14 Batch 1230/4268] avg loss 0.0124957, throughput 4.00111K wps
[Epoch 14 Batch 1260/4268] avg loss 0.011516, throughput 3.99613K wps
[Epoch 14 Batch 1290/4268] avg loss 0.0110649, throughput 3.99888K wps
[Epoch 14 Batch 1320/4268] avg loss 0.0116507, throughput 3.99599K wps
[Epoch 14 Batch 1350/4268] avg loss 0.0121366, throughput 3.99483K wps
[Epoch 14 Batch 1380/4268] avg loss 0.0110033, throughput 3.99242K wps
[Epoch 14 Batch 1410/4268] avg loss 0.0116074, throughput 3.99311K wps
[Epoch 14 Batch 1440/4268] avg loss 0.0109044, throughput 3.99306K wps
[Epoch 14 Batch 1470/4268] avg loss 0.0114124, throughput 3.99413K wps
[Epoch 14 Batch 1500/4268] avg loss 0.0117376, throughput 3.99375K wps
[Epoch 14 Batch 1530/4268] avg loss 0.0118551, throughput 3.98977K wps
[Epoch 14 Batch 1560/4268] avg loss 0.01226, throughput 3.99359K wps
[Epoch 14 Batch 1590/4268] avg loss 0.0112486, throughput 3.99445K wps
[Epoch 14 Batch 1620/4268] avg loss 0.0105794, throughput 3.99229K wps
[Epoch 14 Batch 1650/4268] avg loss 0.0112276, throughput 3.99444K wps
[Epoch 14 Batch 1680/4268] avg loss 0.0111602, throughput 3.99666K wps
[Epoch 14 Batch 1710/4268] avg loss 0.0114014, throughput 3.99693K wps
[Epoch 14 Batch 1740/4268] avg loss 0.0110626, throughput 3.99667K wps
[Epoch 14 Batch 1770/4268] avg loss 0.0105676, throughput 3.99304K wps
[Epoch 14 Batch 1800/4268] avg loss 0.011376, throughput 3.99833K wps
[Epoch 14 Batch 1830/4268] avg loss 0.0120421, throughput 3.99768K wps
[Epoch 14 Batch 1860/4268] avg loss 0.0112419, throughput 3.99621K wps
[Epoch 14 Batch 1890/4268] avg loss 0.0125914, throughput 3.99377K wps
[Epoch 14 Batch 1920/4268] avg loss 0.0112308, throughput 3.99281K wps
[Epoch 14 Batch 1950/4268] avg loss 0.0118506, throughput 3.99666K wps
[Epoch 14 Batch 1980/4268] avg loss 0.0120603, throughput 3.99905K wps
[Epoch 14 Batch 2010/4268] avg loss 0.0121911, throughput 3.99589K wps
[Epoch 14 Batch 2040/4268] avg loss 0.0116937, throughput 3.99286K wps
[Epoch 14 Batch 2070/4268] avg loss 0.0118029, throughput 3.99562K wps
[Epoch 14 Batch 2100/4268] avg loss 0.0126093, throughput 4.00154K wps
[Epoch 14 Batch 2130/4268] avg loss 0.0117525, throughput 3.99646K wps
[Epoch 14 Batch 2160/4268] avg loss 0.012089, throughput 3.99573K wps
[Epoch 14 Batch 2190/4268] avg loss 0.0115876, throughput 3.99569K wps
[Epoch 14 Batch 2220/4268] avg loss 0.0121481, throughput 3.9969K wps
[Epoch 14 Batch 2250/4268] avg loss 0.0119565, throughput 4.00088K wps
[Epoch 14 Batch 2280/4268] avg loss 0.0117321, throughput 3.99548K wps
[Epoch 14 Batch 2310/4268] avg loss 0.0114211, throughput 3.99928K wps
[Epoch 14 Batch 2340/4268] avg loss 0.011464, throughput 3.99077K wps
[Epoch 14 Batch 2370/4268] avg loss 0.0124222, throughput 3.99142K wps
[Epoch 14 Batch 2400/4268] avg loss 0.0112417, throughput 3.9945K wps
[Epoch 14 Batch 2430/4268] avg loss 0.0117503, throughput 3.99611K wps
[Epoch 14 Batch 2460/4268] avg loss 0.0117434, throughput 3.99738K wps
[Epoch 14 Batch 2490/4268] avg loss 0.0123078, throughput 3.9956K wps
[Epoch 14 Batch 2520/4268] avg loss 0.0110952, throughput 3.99285K wps
[Epoch 14 Batch 2550/4268] avg loss 0.0112514, throughput 4.00128K wps
[Epoch 14 Batch 2580/4268] avg loss 0.0117525, throughput 3.99636K wps
[Epoch 14 Batch 2610/4268] avg loss 0.0119875, throughput 3.99625K wps
[Epoch 14 Batch 2640/4268] avg loss 0.0121064, throughput 3.99445K wps
[Epoch 14 Batch 2670/4268] avg loss 0.0119643, throughput 3.99973K wps
[Epoch 14 Batch 2700/4268] avg loss 0.0117743, throughput 3.99241K wps
[Epoch 14 Batch 2730/4268] avg loss 0.0112496, throughput 3.99519K wps
[Epoch 14 Batch 2760/4268] avg loss 0.0114829, throughput 3.99293K wps
[Epoch 14 Batch 2790/4268] avg loss 0.0114681, throughput 3.99446K wps
[Epoch 14 Batch 2820/4268] avg loss 0.0115672, throughput 3.98863K wps
[Epoch 14 Batch 2850/4268] avg loss 0.011908, throughput 3.98358K wps
[Epoch 14 Batch 2880/4268] avg loss 0.0118042, throughput 4.00383K wps
[Epoch 14 Batch 2910/4268] avg loss 0.0122385, throughput 3.995K wps
[Epoch 14 Batch 2940/4268] avg loss 0.0122769, throughput 3.9955K wps
[Epoch 14 Batch 2970/4268] avg loss 0.0118426, throughput 3.99709K wps
[Epoch 14 Batch 3000/4268] avg loss 0.0114449, throughput 3.99819K wps
[Epoch 14 Batch 3030/4268] avg loss 0.0115496, throughput 4.00807K wps
[Epoch 14 Batch 3060/4268] avg loss 0.0114594, throughput 4.00978K wps
[Epoch 14 Batch 3090/4268] avg loss 0.0124354, throughput 4.00193K wps
[Epoch 14 Batch 3120/4268] avg loss 0.0124966, throughput 4.00396K wps
[Epoch 14 Batch 3150/4268] avg loss 0.0110825, throughput 3.99825K wps
[Epoch 14 Batch 3180/4268] avg loss 0.0118701, throughput 3.99839K wps
[Epoch 14 Batch 3210/4268] avg loss 0.0121484, throughput 3.99946K wps
[Epoch 14 Batch 3240/4268] avg loss 0.0115648, throughput 3.99825K wps
[Epoch 14 Batch 3270/4268] avg loss 0.0115352, throughput 3.98957K wps
[Epoch 14 Batch 3300/4268] avg loss 0.011687, throughput 3.99134K wps
[Epoch 14 Batch 3330/4268] avg loss 0.0118922, throughput 3.99423K wps
[Epoch 14 Batch 3360/4268] avg loss 0.0121297, throughput 3.9935K wps
[Epoch 14 Batch 3390/4268] avg loss 0.011693, throughput 3.99336K wps
[Epoch 14 Batch 3420/4268] avg loss 0.0120169, throughput 3.99356K wps
[Epoch 14 Batch 3450/4268] avg loss 0.0119498, throughput 3.99213K wps
[Epoch 14 Batch 3480/4268] avg loss 0.0117913, throughput 3.99617K wps
[Epoch 14 Batch 3510/4268] avg loss 0.0124805, throughput 3.99845K wps
[Epoch 14 Batch 3540/4268] avg loss 0.0113827, throughput 3.99502K wps
[Epoch 14 Batch 3570/4268] avg loss 0.0121734, throughput 4.00193K wps
[Epoch 14 Batch 3600/4268] avg loss 0.0121021, throughput 3.9943K wps
[Epoch 14 Batch 3630/4268] avg loss 0.0119859, throughput 3.99481K wps
[Epoch 14 Batch 3660/4268] avg loss 0.0120354, throughput 3.99855K wps
[Epoch 14 Batch 3690/4268] avg loss 0.0119537, throughput 3.99325K wps
[Epoch 14 Batch 3720/4268] avg loss 0.0119851, throughput 3.99269K wps
[Epoch 14 Batch 3750/4268] avg loss 0.0111456, throughput 3.99449K wps
[Epoch 14 Batch 3780/4268] avg loss 0.0114665, throughput 3.99788K wps
[Epoch 14 Batch 3810/4268] avg loss 0.0118507, throughput 3.99557K wps
[Epoch 14 Batch 3840/4268] avg loss 0.0118332, throughput 3.99497K wps
[Epoch 14 Batch 3870/4268] avg loss 0.0122721, throughput 3.99258K wps
[Epoch 14 Batch 3900/4268] avg loss 0.0118634, throughput 3.99588K wps
[Epoch 14 Batch 3930/4268] avg loss 0.0122465, throughput 3.99054K wps
[Epoch 14 Batch 3960/4268] avg loss 0.0111988, throughput 3.99354K wps
[Epoch 14 Batch 3990/4268] avg loss 0.012205, throughput 3.99178K wps
[Epoch 14 Batch 4020/4268] avg loss 0.0118231, throughput 3.99432K wps
[Epoch 14 Batch 4050/4268] avg loss 0.0118583, throughput 3.99484K wps
[Epoch 14 Batch 4080/4268] avg loss 0.011718, throughput 3.99772K wps
[Epoch 14 Batch 4110/4268] avg loss 0.0121051, throughput 3.99425K wps
[Epoch 14 Batch 4140/4268] avg loss 0.0119184, throughput 3.99727K wps
[Epoch 14 Batch 4170/4268] avg loss 0.0122707, throughput 3.99345K wps
[Epoch 14 Batch 4200/4268] avg loss 0.0121237, throughput 3.99304K wps
[Epoch 14 Batch 4230/4268] avg loss 0.0123901, throughput 3.9954K wps
[Epoch 14 Batch 4260/4268] avg loss 0.012466, throughput 3.99589K wps
Begin Testing...
[Batch 30/475] elapsed 0.46 s
[Batch 60/475] elapsed 0.43 s
[Batch 90/475] elapsed 0.43 s
[Batch 120/475] elapsed 0.43 s
[Batch 150/475] elapsed 0.43 s
[Batch 180/475] elapsed 0.43 s
[Batch 210/475] elapsed 0.43 s
[Batch 240/475] elapsed 0.43 s
[Batch 270/475] elapsed 0.43 s
[Batch 300/475] elapsed 0.43 s
[Batch 330/475] elapsed 0.43 s
[Batch 360/475] elapsed 0.43 s
[Batch 390/475] elapsed 0.43 s
[Batch 420/475] elapsed 0.43 s
[Batch 450/475] elapsed 0.43 s
[Epoch 14] train avg loss 0.0116324, test acc 0.6789, test avg loss 0.84337, throughput 3.99653K wps
[Epoch 15 Batch 30/4268] avg loss 0.0107373, throughput 4.08054K wps
[Epoch 15 Batch 60/4268] avg loss 0.0118397, throughput 3.99647K wps
[Epoch 15 Batch 90/4268] avg loss 0.0108773, throughput 3.99713K wps
[Epoch 15 Batch 120/4268] avg loss 0.0116115, throughput 3.9947K wps
[Epoch 15 Batch 150/4268] avg loss 0.0108462, throughput 3.9964K wps
[Epoch 15 Batch 180/4268] avg loss 0.0106889, throughput 3.99668K wps
[Epoch 15 Batch 210/4268] avg loss 0.0111708, throughput 4.00094K wps
[Epoch 15 Batch 240/4268] avg loss 0.0110987, throughput 3.99317K wps
[Epoch 15 Batch 270/4268] avg loss 0.0111144, throughput 3.99061K wps
[Epoch 15 Batch 300/4268] avg loss 0.0106846, throughput 3.99498K wps
[Epoch 15 Batch 330/4268] avg loss 0.0109528, throughput 3.99649K wps
[Epoch 15 Batch 360/4268] avg loss 0.0106794, throughput 3.9922K wps
[Epoch 15 Batch 390/4268] avg loss 0.0113001, throughput 4.00126K wps
[Epoch 15 Batch 420/4268] avg loss 0.0109192, throughput 4.00373K wps
[Epoch 15 Batch 450/4268] avg loss 0.0112321, throughput 4.00987K wps
[Epoch 15 Batch 480/4268] avg loss 0.0109233, throughput 4.00738K wps
[Epoch 15 Batch 510/4268] avg loss 0.010884, throughput 3.99792K wps
[Epoch 15 Batch 540/4268] avg loss 0.0109076, throughput 3.9948K wps
[Epoch 15 Batch 570/4268] avg loss 0.0108292, throughput 3.991K wps
[Epoch 15 Batch 600/4268] avg loss 0.0112677, throughput 3.99441K wps
[Epoch 15 Batch 630/4268] avg loss 0.0108471, throughput 3.99116K wps
[Epoch 15 Batch 660/4268] avg loss 0.0109929, throughput 3.99337K wps
[Epoch 15 Batch 690/4268] avg loss 0.0106789, throughput 3.9927K wps
[Epoch 15 Batch 720/4268] avg loss 0.0112085, throughput 3.99656K wps
[Epoch 15 Batch 750/4268] avg loss 0.0111746, throughput 3.99633K wps
[Epoch 15 Batch 780/4268] avg loss 0.0111933, throughput 3.99627K wps
[Epoch 15 Batch 810/4268] avg loss 0.0110716, throughput 3.99123K wps
[Epoch 15 Batch 840/4268] avg loss 0.0117685, throughput 3.99485K wps
[Epoch 15 Batch 870/4268] avg loss 0.0112524, throughput 3.99697K wps
[Epoch 15 Batch 900/4268] avg loss 0.0113493, throughput 3.99297K wps
[Epoch 15 Batch 930/4268] avg loss 0.0113033, throughput 3.99688K wps
[Epoch 15 Batch 960/4268] avg loss 0.0113198, throughput 3.99346K wps
[Epoch 15 Batch 990/4268] avg loss 0.0112438, throughput 3.99663K wps
[Epoch 15 Batch 1020/4268] avg loss 0.0110263, throughput 3.99486K wps
[Epoch 15 Batch 1050/4268] avg loss 0.0113337, throughput 3.99781K wps
[Epoch 15 Batch 1080/4268] avg loss 0.0110254, throughput 3.99709K wps
[Epoch 15 Batch 1110/4268] avg loss 0.0118595, throughput 4.00112K wps
[Epoch 15 Batch 1140/4268] avg loss 0.0116145, throughput 3.99459K wps
[Epoch 15 Batch 1170/4268] avg loss 0.0102198, throughput 3.99555K wps
[Epoch 15 Batch 1200/4268] avg loss 0.010934, throughput 3.99348K wps
[Epoch 15 Batch 1230/4268] avg loss 0.010878, throughput 3.99872K wps
[Epoch 15 Batch 1260/4268] avg loss 0.0106154, throughput 3.99641K wps
[Epoch 15 Batch 1290/4268] avg loss 0.0114329, throughput 3.99214K wps
[Epoch 15 Batch 1320/4268] avg loss 0.011796, throughput 3.99014K wps
[Epoch 15 Batch 1350/4268] avg loss 0.0107828, throughput 3.99425K wps
[Epoch 15 Batch 1380/4268] avg loss 0.0114418, throughput 4.00529K wps
[Epoch 15 Batch 1410/4268] avg loss 0.0115413, throughput 4.00102K wps
[Epoch 15 Batch 1440/4268] avg loss 0.0115122, throughput 3.9959K wps
[Epoch 15 Batch 1470/4268] avg loss 0.0115028, throughput 3.99903K wps
[Epoch 15 Batch 1500/4268] avg loss 0.011029, throughput 3.99689K wps
[Epoch 15 Batch 1530/4268] avg loss 0.0118583, throughput 3.99464K wps
[Epoch 15 Batch 1560/4268] avg loss 0.010812, throughput 4.00067K wps
[Epoch 15 Batch 1590/4268] avg loss 0.0108701, throughput 3.99294K wps
[Epoch 15 Batch 1620/4268] avg loss 0.0112176, throughput 3.99021K wps
[Epoch 15 Batch 1650/4268] avg loss 0.0112995, throughput 3.9958K wps
[Epoch 15 Batch 1680/4268] avg loss 0.0112054, throughput 3.99736K wps
[Epoch 15 Batch 1710/4268] avg loss 0.0122179, throughput 3.99598K wps
[Epoch 15 Batch 1740/4268] avg loss 0.0105985, throughput 3.99372K wps
[Epoch 15 Batch 1770/4268] avg loss 0.0106824, throughput 3.99381K wps
[Epoch 15 Batch 1800/4268] avg loss 0.0113465, throughput 3.9983K wps
[Epoch 15 Batch 1830/4268] avg loss 0.0109917, throughput 4.00428K wps
[Epoch 15 Batch 1860/4268] avg loss 0.0108296, throughput 4.00505K wps
[Epoch 15 Batch 1890/4268] avg loss 0.0110687, throughput 4.00016K wps
[Epoch 15 Batch 1920/4268] avg loss 0.0112099, throughput 4.00173K wps
[Epoch 15 Batch 1950/4268] avg loss 0.0109055, throughput 4.00113K wps
[Epoch 15 Batch 1980/4268] avg loss 0.0114699, throughput 4.00238K wps
[Epoch 15 Batch 2010/4268] avg loss 0.0119085, throughput 3.99363K wps
[Epoch 15 Batch 2040/4268] avg loss 0.011266, throughput 3.97624K wps
[Epoch 15 Batch 2070/4268] avg loss 0.0119751, throughput 3.99694K wps
[Epoch 15 Batch 2100/4268] avg loss 0.0114144, throughput 3.99665K wps
[Epoch 15 Batch 2130/4268] avg loss 0.0120879, throughput 3.99122K wps
[Epoch 15 Batch 2160/4268] avg loss 0.011178, throughput 3.99507K wps
[Epoch 15 Batch 2190/4268] avg loss 0.0113595, throughput 3.99477K wps
[Epoch 15 Batch 2220/4268] avg loss 0.0117284, throughput 3.9955K wps
[Epoch 15 Batch 2250/4268] avg loss 0.0115087, throughput 3.9941K wps
[Epoch 15 Batch 2280/4268] avg loss 0.0114797, throughput 3.99548K wps
[Epoch 15 Batch 2310/4268] avg loss 0.0111806, throughput 3.99121K wps
[Epoch 15 Batch 2340/4268] avg loss 0.0114367, throughput 3.99575K wps
[Epoch 15 Batch 2370/4268] avg loss 0.0122602, throughput 3.99192K wps
[Epoch 15 Batch 2400/4268] avg loss 0.0118921, throughput 3.99789K wps
[Epoch 15 Batch 2430/4268] avg loss 0.0111576, throughput 3.99966K wps
[Epoch 15 Batch 2460/4268] avg loss 0.0118267, throughput 3.99752K wps
[Epoch 15 Batch 2490/4268] avg loss 0.0119769, throughput 3.99669K wps
[Epoch 15 Batch 2520/4268] avg loss 0.0115793, throughput 3.99809K wps
[Epoch 15 Batch 2550/4268] avg loss 0.0107745, throughput 3.99481K wps
[Epoch 15 Batch 2580/4268] avg loss 0.0112621, throughput 3.99075K wps
[Epoch 15 Batch 2610/4268] avg loss 0.0118244, throughput 3.99338K wps
[Epoch 15 Batch 2640/4268] avg loss 0.0114581, throughput 3.99951K wps
[Epoch 15 Batch 2670/4268] avg loss 0.0109085, throughput 3.99726K wps
[Epoch 15 Batch 2700/4268] avg loss 0.0111298, throughput 3.99621K wps
[Epoch 15 Batch 2730/4268] avg loss 0.0112757, throughput 3.99403K wps
[Epoch 15 Batch 2760/4268] avg loss 0.011043, throughput 3.99334K wps
[Epoch 15 Batch 2790/4268] avg loss 0.0117734, throughput 3.99237K wps
[Epoch 15 Batch 2820/4268] avg loss 0.0111679, throughput 3.99345K wps
[Epoch 15 Batch 2850/4268] avg loss 0.0116535, throughput 3.99609K wps
[Epoch 15 Batch 2880/4268] avg loss 0.0114086, throughput 3.99833K wps
[Epoch 15 Batch 2910/4268] avg loss 0.0120901, throughput 3.99401K wps
[Epoch 15 Batch 2940/4268] avg loss 0.0119852, throughput 3.99364K wps
[Epoch 15 Batch 2970/4268] avg loss 0.0115915, throughput 3.99444K wps
[Epoch 15 Batch 3000/4268] avg loss 0.0112712, throughput 3.99386K wps
[Epoch 15 Batch 3030/4268] avg loss 0.0114035, throughput 3.99831K wps
[Epoch 15 Batch 3060/4268] avg loss 0.0120543, throughput 3.99753K wps
[Epoch 15 Batch 3090/4268] avg loss 0.01167, throughput 3.99689K wps
[Epoch 15 Batch 3120/4268] avg loss 0.0123508, throughput 3.99868K wps
[Epoch 15 Batch 3150/4268] avg loss 0.0119411, throughput 3.99367K wps
[Epoch 15 Batch 3180/4268] avg loss 0.0117476, throughput 3.9951K wps
[Epoch 15 Batch 3210/4268] avg loss 0.0118003, throughput 4.00143K wps
[Epoch 15 Batch 3240/4268] avg loss 0.011671, throughput 3.99544K wps
[Epoch 15 Batch 3270/4268] avg loss 0.0124167, throughput 3.9958K wps
[Epoch 15 Batch 3300/4268] avg loss 0.0115269, throughput 3.99525K wps
[Epoch 15 Batch 3330/4268] avg loss 0.011221, throughput 3.99308K wps
[Epoch 15 Batch 3360/4268] avg loss 0.0114351, throughput 3.99215K wps
[Epoch 15 Batch 3390/4268] avg loss 0.0120281, throughput 3.99508K wps
[Epoch 15 Batch 3420/4268] avg loss 0.0119356, throughput 3.99746K wps
[Epoch 15 Batch 3450/4268] avg loss 0.0112121, throughput 3.99476K wps
[Epoch 15 Batch 3480/4268] avg loss 0.0115636, throughput 3.99645K wps
[Epoch 15 Batch 3510/4268] avg loss 0.0113223, throughput 3.99905K wps
[Epoch 15 Batch 3540/4268] avg loss 0.0124388, throughput 3.99786K wps
[Epoch 15 Batch 3570/4268] avg loss 0.0118203, throughput 3.99508K wps
[Epoch 15 Batch 3600/4268] avg loss 0.0114303, throughput 3.99507K wps
[Epoch 15 Batch 3630/4268] avg loss 0.0113168, throughput 3.99392K wps
[Epoch 15 Batch 3660/4268] avg loss 0.011828, throughput 3.99392K wps
[Epoch 15 Batch 3690/4268] avg loss 0.011165, throughput 3.99449K wps
[Epoch 15 Batch 3720/4268] avg loss 0.0123953, throughput 3.99292K wps
[Epoch 15 Batch 3750/4268] avg loss 0.0122191, throughput 4.00322K wps
[Epoch 15 Batch 3780/4268] avg loss 0.0114785, throughput 4.00294K wps
[Epoch 15 Batch 3810/4268] avg loss 0.0116525, throughput 3.99981K wps
[Epoch 15 Batch 3840/4268] avg loss 0.0117638, throughput 3.99822K wps
[Epoch 15 Batch 3870/4268] avg loss 0.0117367, throughput 3.99645K wps
[Epoch 15 Batch 3900/4268] avg loss 0.0111792, throughput 3.99348K wps
[Epoch 15 Batch 3930/4268] avg loss 0.0121271, throughput 4.00001K wps
[Epoch 15 Batch 3960/4268] avg loss 0.0111281, throughput 3.99301K wps
[Epoch 15 Batch 3990/4268] avg loss 0.0113656, throughput 3.99385K wps
[Epoch 15 Batch 4020/4268] avg loss 0.0122177, throughput 4.00035K wps
[Epoch 15 Batch 4050/4268] avg loss 0.012214, throughput 3.99553K wps
[Epoch 15 Batch 4080/4268] avg loss 0.0117067, throughput 3.99701K wps
[Epoch 15 Batch 4110/4268] avg loss 0.0119393, throughput 3.99589K wps
[Epoch 15 Batch 4140/4268] avg loss 0.0113381, throughput 3.9983K wps
[Epoch 15 Batch 4170/4268] avg loss 0.0116413, throughput 3.99615K wps
[Epoch 15 Batch 4200/4268] avg loss 0.0110339, throughput 3.99498K wps
[Epoch 15 Batch 4230/4268] avg loss 0.0111716, throughput 3.99336K wps
[Epoch 15 Batch 4260/4268] avg loss 0.0116118, throughput 3.99599K wps
Begin Testing...
[Batch 30/475] elapsed 0.46 s
[Batch 60/475] elapsed 0.43 s
[Batch 90/475] elapsed 0.43 s
[Batch 120/475] elapsed 0.43 s
[Batch 150/475] elapsed 0.43 s
[Batch 180/475] elapsed 0.43 s
[Batch 210/475] elapsed 0.43 s
[Batch 240/475] elapsed 0.43 s
[Batch 270/475] elapsed 0.43 s
[Batch 300/475] elapsed 0.43 s
[Batch 330/475] elapsed 0.43 s
[Batch 360/475] elapsed 0.43 s
[Batch 390/475] elapsed 0.43 s
[Batch 420/475] elapsed 0.43 s
[Batch 450/475] elapsed 0.43 s
[Epoch 15] train avg loss 0.0113955, test acc 0.6797, test avg loss 0.851863, throughput 3.99666K wps
Observed Improvement.
Begin Testing...
[Batch 30/43] elapsed 0.43 s
[Epoch 16 Batch 30/4268] avg loss 0.0112815, throughput 4.07837K wps
[Epoch 16 Batch 60/4268] avg loss 0.0101513, throughput 3.99703K wps
[Epoch 16 Batch 90/4268] avg loss 0.0109123, throughput 3.99914K wps
[Epoch 16 Batch 120/4268] avg loss 0.0111275, throughput 3.99844K wps
[Epoch 16 Batch 150/4268] avg loss 0.010399, throughput 3.99464K wps
[Epoch 16 Batch 180/4268] avg loss 0.0104321, throughput 3.9928K wps
[Epoch 16 Batch 210/4268] avg loss 0.0106182, throughput 3.99399K wps
[Epoch 16 Batch 240/4268] avg loss 0.0105641, throughput 3.99848K wps
[Epoch 16 Batch 270/4268] avg loss 0.0110593, throughput 3.99209K wps
[Epoch 16 Batch 300/4268] avg loss 0.0105866, throughput 4.00192K wps
[Epoch 16 Batch 330/4268] avg loss 0.0112076, throughput 4.00153K wps
[Epoch 16 Batch 360/4268] avg loss 0.0102655, throughput 4.00795K wps
[Epoch 16 Batch 390/4268] avg loss 0.0111663, throughput 4.00604K wps
[Epoch 16 Batch 420/4268] avg loss 0.0102974, throughput 4.00296K wps
[Epoch 16 Batch 450/4268] avg loss 0.0105338, throughput 3.99142K wps
[Epoch 16 Batch 480/4268] avg loss 0.0113787, throughput 3.99536K wps
[Epoch 16 Batch 510/4268] avg loss 0.0114667, throughput 3.99224K wps
[Epoch 16 Batch 540/4268] avg loss 0.0106202, throughput 3.99595K wps
[Epoch 16 Batch 570/4268] avg loss 0.0106949, throughput 3.99733K wps
[Epoch 16 Batch 600/4268] avg loss 0.0104027, throughput 3.99793K wps
[Epoch 16 Batch 630/4268] avg loss 0.0114996, throughput 3.99326K wps
[Epoch 16 Batch 660/4268] avg loss 0.0107239, throughput 3.99594K wps
[Epoch 16 Batch 690/4268] avg loss 0.0113319, throughput 3.99706K wps
[Epoch 16 Batch 720/4268] avg loss 0.010579, throughput 3.99867K wps
[Epoch 16 Batch 750/4268] avg loss 0.01078, throughput 4.00141K wps
[Epoch 16 Batch 780/4268] avg loss 0.0111484, throughput 3.99602K wps
[Epoch 16 Batch 810/4268] avg loss 0.0113693, throughput 3.9956K wps
[Epoch 16 Batch 840/4268] avg loss 0.0106182, throughput 3.99478K wps
[Epoch 16 Batch 870/4268] avg loss 0.0106945, throughput 3.99868K wps
[Epoch 16 Batch 900/4268] avg loss 0.0113204, throughput 3.99998K wps
[Epoch 16 Batch 930/4268] avg loss 0.0103325, throughput 3.99232K wps
[Epoch 16 Batch 960/4268] avg loss 0.0109879, throughput 3.99219K wps
[Epoch 16 Batch 990/4268] avg loss 0.0108853, throughput 3.99428K wps
[Epoch 16 Batch 1020/4268] avg loss 0.0112362, throughput 3.997K wps
[Epoch 16 Batch 1050/4268] avg loss 0.0113206, throughput 3.99722K wps
[Epoch 16 Batch 1080/4268] avg loss 0.0117186, throughput 4.00008K wps
[Epoch 16 Batch 1110/4268] avg loss 0.0104448, throughput 3.99939K wps
[Epoch 16 Batch 1140/4268] avg loss 0.0109475, throughput 3.99705K wps
[Epoch 16 Batch 1170/4268] avg loss 0.0108828, throughput 3.99416K wps
[Epoch 16 Batch 1200/4268] avg loss 0.0108899, throughput 3.99004K wps
[Epoch 16 Batch 1230/4268] avg loss 0.011523, throughput 3.98608K wps
[Epoch 16 Batch 1260/4268] avg loss 0.0105074, throughput 3.99503K wps
[Epoch 16 Batch 1290/4268] avg loss 0.010573, throughput 3.99313K wps
[Epoch 16 Batch 1320/4268] avg loss 0.0106036, throughput 3.99579K wps
[Epoch 16 Batch 1350/4268] avg loss 0.0106151, throughput 3.99407K wps
[Epoch 16 Batch 1380/4268] avg loss 0.0114654, throughput 3.99489K wps
[Epoch 16 Batch 1410/4268] avg loss 0.0100384, throughput 3.99309K wps
[Epoch 16 Batch 1440/4268] avg loss 0.0112989, throughput 3.99408K wps
[Epoch 16 Batch 1470/4268] avg loss 0.0114134, throughput 4.00822K wps
[Epoch 16 Batch 1500/4268] avg loss 0.0109134, throughput 4.00607K wps
[Epoch 16 Batch 1530/4268] avg loss 0.011108, throughput 4.00413K wps
[Epoch 16 Batch 1560/4268] avg loss 0.0115093, throughput 4.00128K wps
[Epoch 16 Batch 1590/4268] avg loss 0.0112608, throughput 4.00017K wps
[Epoch 16 Batch 1620/4268] avg loss 0.0116107, throughput 4.00516K wps
[Epoch 16 Batch 1650/4268] avg loss 0.0112082, throughput 4.00178K wps
[Epoch 16 Batch 1680/4268] avg loss 0.0104869, throughput 4.00361K wps
[Epoch 16 Batch 1710/4268] avg loss 0.0108096, throughput 4.00418K wps
[Epoch 16 Batch 1740/4268] avg loss 0.0108984, throughput 4.00162K wps
[Epoch 16 Batch 1770/4268] avg loss 0.0113708, throughput 3.99519K wps
[Epoch 16 Batch 1800/4268] avg loss 0.0111531, throughput 3.99951K wps
[Epoch 16 Batch 1830/4268] avg loss 0.0108135, throughput 3.99589K wps
[Epoch 16 Batch 1860/4268] avg loss 0.010641, throughput 3.99997K wps
[Epoch 16 Batch 1890/4268] avg loss 0.0116278, throughput 3.99504K wps
[Epoch 16 Batch 1920/4268] avg loss 0.0114467, throughput 4.00052K wps
[Epoch 16 Batch 1950/4268] avg loss 0.0112882, throughput 3.99513K wps
[Epoch 16 Batch 1980/4268] avg loss 0.0109296, throughput 3.99942K wps
[Epoch 16 Batch 2010/4268] avg loss 0.0111704, throughput 3.99302K wps
[Epoch 16 Batch 2040/4268] avg loss 0.0114413, throughput 3.99598K wps
[Epoch 16 Batch 2070/4268] avg loss 0.010877, throughput 3.9932K wps
[Epoch 16 Batch 2100/4268] avg loss 0.0118932, throughput 3.99825K wps
[Epoch 16 Batch 2130/4268] avg loss 0.0117342, throughput 3.99473K wps
[Epoch 16 Batch 2160/4268] avg loss 0.0112784, throughput 3.99139K wps
[Epoch 16 Batch 2190/4268] avg loss 0.0116189, throughput 3.99666K wps
[Epoch 16 Batch 2220/4268] avg loss 0.01108, throughput 3.99392K wps
[Epoch 16 Batch 2250/4268] avg loss 0.0110208, throughput 3.99519K wps
[Epoch 16 Batch 2280/4268] avg loss 0.0113004, throughput 3.99471K wps
[Epoch 16 Batch 2310/4268] avg loss 0.0110855, throughput 3.99679K wps
[Epoch 16 Batch 2340/4268] avg loss 0.0112509, throughput 3.99727K wps
[Epoch 16 Batch 2370/4268] avg loss 0.0114123, throughput 3.99382K wps
[Epoch 16 Batch 2400/4268] avg loss 0.0108891, throughput 3.99287K wps
[Epoch 16 Batch 2430/4268] avg loss 0.0110636, throughput 3.99573K wps
[Epoch 16 Batch 2460/4268] avg loss 0.0119925, throughput 3.99743K wps
[Epoch 16 Batch 2490/4268] avg loss 0.0114982, throughput 3.99739K wps
[Epoch 16 Batch 2520/4268] avg loss 0.0117472, throughput 3.99226K wps
[Epoch 16 Batch 2550/4268] avg loss 0.0104092, throughput 3.99672K wps
[Epoch 16 Batch 2580/4268] avg loss 0.0117815, throughput 3.99651K wps
[Epoch 16 Batch 2610/4268] avg loss 0.0109169, throughput 3.99457K wps
[Epoch 16 Batch 2640/4268] avg loss 0.0111893, throughput 3.99873K wps
[Epoch 16 Batch 2670/4268] avg loss 0.0118056, throughput 3.99543K wps
[Epoch 16 Batch 2700/4268] avg loss 0.0116845, throughput 3.99247K wps
[Epoch 16 Batch 2730/4268] avg loss 0.0105966, throughput 3.99349K wps
[Epoch 16 Batch 2760/4268] avg loss 0.0115303, throughput 3.99615K wps
[Epoch 16 Batch 2790/4268] avg loss 0.011057, throughput 3.9985K wps
[Epoch 16 Batch 2820/4268] avg loss 0.0114167, throughput 3.99283K wps
[Epoch 16 Batch 2850/4268] avg loss 0.0112614, throughput 3.9897K wps
[Epoch 16 Batch 2880/4268] avg loss 0.0114312, throughput 3.9987K wps
[Epoch 16 Batch 2910/4268] avg loss 0.012276, throughput 3.9956K wps
[Epoch 16 Batch 2940/4268] avg loss 0.0117452, throughput 3.99852K wps
[Epoch 16 Batch 2970/4268] avg loss 0.0116881, throughput 3.99522K wps
[Epoch 16 Batch 3000/4268] avg loss 0.0105019, throughput 3.99516K wps
[Epoch 16 Batch 3030/4268] avg loss 0.0112529, throughput 3.99395K wps
[Epoch 16 Batch 3060/4268] avg loss 0.0111339, throughput 3.99648K wps
[Epoch 16 Batch 3090/4268] avg loss 0.0115595, throughput 3.99829K wps
[Epoch 16 Batch 3120/4268] avg loss 0.0121166, throughput 3.99709K wps
[Epoch 16 Batch 3150/4268] avg loss 0.0123183, throughput 3.99997K wps
[Epoch 16 Batch 3180/4268] avg loss 0.0114668, throughput 3.9946K wps
[Epoch 16 Batch 3210/4268] avg loss 0.0114651, throughput 3.99628K wps
[Epoch 16 Batch 3240/4268] avg loss 0.0114566, throughput 3.99505K wps
[Epoch 16 Batch 3270/4268] avg loss 0.0116071, throughput 3.9938K wps
[Epoch 16 Batch 3300/4268] avg loss 0.0116884, throughput 3.99661K wps
[Epoch 16 Batch 3330/4268] avg loss 0.0120632, throughput 3.99822K wps
[Epoch 16 Batch 3360/4268] avg loss 0.0113335, throughput 3.99389K wps
[Epoch 16 Batch 3390/4268] avg loss 0.0117688, throughput 3.9964K wps
[Epoch 16 Batch 3420/4268] avg loss 0.0114444, throughput 3.99611K wps
[Epoch 16 Batch 3450/4268] avg loss 0.0114213, throughput 3.99726K wps
[Epoch 16 Batch 3480/4268] avg loss 0.0114356, throughput 3.99774K wps
[Epoch 16 Batch 3510/4268] avg loss 0.0111412, throughput 3.99268K wps
[Epoch 16 Batch 3540/4268] avg loss 0.011418, throughput 3.99537K wps
[Epoch 16 Batch 3570/4268] avg loss 0.0110459, throughput 3.99853K wps
[Epoch 16 Batch 3600/4268] avg loss 0.0113402, throughput 3.99913K wps
[Epoch 16 Batch 3630/4268] avg loss 0.0112249, throughput 3.99552K wps
[Epoch 16 Batch 3660/4268] avg loss 0.0111562, throughput 3.99764K wps
[Epoch 16 Batch 3690/4268] avg loss 0.0115397, throughput 4.00124K wps
[Epoch 16 Batch 3720/4268] avg loss 0.0112182, throughput 3.99797K wps
[Epoch 16 Batch 3750/4268] avg loss 0.0112121, throughput 3.99309K wps
[Epoch 16 Batch 3780/4268] avg loss 0.0110336, throughput 3.99531K wps
[Epoch 16 Batch 3810/4268] avg loss 0.0117991, throughput 3.99671K wps
[Epoch 16 Batch 3840/4268] avg loss 0.0112127, throughput 3.99586K wps
[Epoch 16 Batch 3870/4268] avg loss 0.0120432, throughput 3.99612K wps
[Epoch 16 Batch 3900/4268] avg loss 0.0117466, throughput 4.00111K wps
[Epoch 16 Batch 3930/4268] avg loss 0.011079, throughput 3.99798K wps
[Epoch 16 Batch 3960/4268] avg loss 0.0118114, throughput 3.9978K wps
[Epoch 16 Batch 3990/4268] avg loss 0.012062, throughput 4.00115K wps
[Epoch 16 Batch 4020/4268] avg loss 0.0113233, throughput 3.99814K wps
[Epoch 16 Batch 4050/4268] avg loss 0.0116064, throughput 3.99761K wps
[Epoch 16 Batch 4080/4268] avg loss 0.0116966, throughput 3.99448K wps
[Epoch 16 Batch 4110/4268] avg loss 0.0116646, throughput 3.99658K wps
[Epoch 16 Batch 4140/4268] avg loss 0.0110781, throughput 3.99522K wps
[Epoch 16 Batch 4170/4268] avg loss 0.012232, throughput 3.99814K wps
[Epoch 16 Batch 4200/4268] avg loss 0.0117921, throughput 3.99465K wps
[Epoch 16 Batch 4230/4268] avg loss 0.011535, throughput 3.99451K wps
[Epoch 16 Batch 4260/4268] avg loss 0.0116913, throughput 3.99657K wps
Begin Testing...
[Batch 30/475] elapsed 0.46 s
[Batch 60/475] elapsed 0.43 s
[Batch 90/475] elapsed 0.43 s
[Batch 120/475] elapsed 0.43 s
[Batch 150/475] elapsed 0.43 s
[Batch 180/475] elapsed 0.43 s
[Batch 210/475] elapsed 0.43 s
[Batch 240/475] elapsed 0.43 s
[Batch 270/475] elapsed 0.43 s
[Batch 300/475] elapsed 0.43 s
[Batch 330/475] elapsed 0.43 s
[Batch 360/475] elapsed 0.43 s
[Batch 390/475] elapsed 0.43 s
[Batch 420/475] elapsed 0.43 s
[Batch 450/475] elapsed 0.43 s
[Epoch 16] train avg loss 0.0112118, test acc 0.6780, test avg loss 0.859405, throughput 3.99729K wps
[Epoch 17 Batch 30/4268] avg loss 0.0102587, throughput 4.08066K wps
[Epoch 17 Batch 60/4268] avg loss 0.0100649, throughput 3.99831K wps
[Epoch 17 Batch 90/4268] avg loss 0.0101826, throughput 3.99823K wps
[Epoch 17 Batch 120/4268] avg loss 0.0108026, throughput 3.99905K wps
[Epoch 17 Batch 150/4268] avg loss 0.0102, throughput 3.99647K wps
[Epoch 17 Batch 180/4268] avg loss 0.0114768, throughput 3.99843K wps
[Epoch 17 Batch 210/4268] avg loss 0.010638, throughput 3.99871K wps
[Epoch 17 Batch 240/4268] avg loss 0.0111231, throughput 4.00302K wps
[Epoch 17 Batch 270/4268] avg loss 0.0107506, throughput 3.99723K wps
[Epoch 17 Batch 300/4268] avg loss 0.0105042, throughput 3.99512K wps
[Epoch 17 Batch 330/4268] avg loss 0.0105321, throughput 3.99758K wps
[Epoch 17 Batch 360/4268] avg loss 0.0100359, throughput 3.99114K wps
[Epoch 17 Batch 390/4268] avg loss 0.0106673, throughput 3.98947K wps
[Epoch 17 Batch 420/4268] avg loss 0.00940806, throughput 3.98228K wps
[Epoch 17 Batch 450/4268] avg loss 0.0101468, throughput 4.00663K wps
[Epoch 17 Batch 480/4268] avg loss 0.0113885, throughput 3.99378K wps
[Epoch 17 Batch 510/4268] avg loss 0.010911, throughput 3.99259K wps
[Epoch 17 Batch 540/4268] avg loss 0.0107644, throughput 3.9898K wps
[Epoch 17 Batch 570/4268] avg loss 0.0109175, throughput 3.99452K wps
[Epoch 17 Batch 600/4268] avg loss 0.0105606, throughput 3.99611K wps
[Epoch 17 Batch 630/4268] avg loss 0.0109017, throughput 3.99525K wps
[Epoch 17 Batch 660/4268] avg loss 0.0105052, throughput 3.99526K wps
[Epoch 17 Batch 690/4268] avg loss 0.0110241, throughput 4.00097K wps
[Epoch 17 Batch 720/4268] avg loss 0.0104995, throughput 3.99965K wps
[Epoch 17 Batch 750/4268] avg loss 0.0108612, throughput 3.99138K wps
[Epoch 17 Batch 780/4268] avg loss 0.0105765, throughput 3.9921K wps
[Epoch 17 Batch 810/4268] avg loss 0.0106196, throughput 3.99183K wps
[Epoch 17 Batch 840/4268] avg loss 0.0103531, throughput 3.99119K wps
[Epoch 17 Batch 870/4268] avg loss 0.0110355, throughput 3.99253K wps
[Epoch 17 Batch 900/4268] avg loss 0.010014, throughput 3.99594K wps
[Epoch 17 Batch 930/4268] avg loss 0.010668, throughput 3.99584K wps
[Epoch 17 Batch 960/4268] avg loss 0.0105599, throughput 3.9949K wps
[Epoch 17 Batch 990/4268] avg loss 0.0106967, throughput 3.99536K wps
[Epoch 17 Batch 1020/4268] avg loss 0.0106643, throughput 3.99666K wps
[Epoch 17 Batch 1050/4268] avg loss 0.0103601, throughput 3.99573K wps
[Epoch 17 Batch 1080/4268] avg loss 0.0101606, throughput 3.99887K wps
[Epoch 17 Batch 1110/4268] avg loss 0.010889, throughput 3.99408K wps
[Epoch 17 Batch 1140/4268] avg loss 0.0105293, throughput 3.99321K wps
[Epoch 17 Batch 1170/4268] avg loss 0.0108256, throughput 4.00056K wps
[Epoch 17 Batch 1200/4268] avg loss 0.0113856, throughput 3.9928K wps
[Epoch 17 Batch 1230/4268] avg loss 0.0110739, throughput 3.99252K wps
[Epoch 17 Batch 1260/4268] avg loss 0.0111603, throughput 3.99519K wps
[Epoch 17 Batch 1290/4268] avg loss 0.0107131, throughput 3.99236K wps
[Epoch 17 Batch 1320/4268] avg loss 0.0111019, throughput 3.99696K wps
[Epoch 17 Batch 1350/4268] avg loss 0.0113615, throughput 3.99904K wps
[Epoch 17 Batch 1380/4268] avg loss 0.0107057, throughput 4.00358K wps
[Epoch 17 Batch 1410/4268] avg loss 0.01084, throughput 3.99957K wps
[Epoch 17 Batch 1440/4268] avg loss 0.0106701, throughput 4.0014K wps
[Epoch 17 Batch 1470/4268] avg loss 0.0101133, throughput 4.00676K wps
[Epoch 17 Batch 1500/4268] avg loss 0.0104686, throughput 3.99778K wps
[Epoch 17 Batch 1530/4268] avg loss 0.0115872, throughput 3.99554K wps
[Epoch 17 Batch 1560/4268] avg loss 0.0106862, throughput 3.9992K wps
[Epoch 17 Batch 1590/4268] avg loss 0.0107959, throughput 4.00269K wps
[Epoch 17 Batch 1620/4268] avg loss 0.0101157, throughput 3.99827K wps
[Epoch 17 Batch 1650/4268] avg loss 0.0112429, throughput 3.99968K wps
[Epoch 17 Batch 1680/4268] avg loss 0.0105262, throughput 3.99825K wps
[Epoch 17 Batch 1710/4268] avg loss 0.0113477, throughput 3.99875K wps
[Epoch 17 Batch 1740/4268] avg loss 0.0109405, throughput 3.99783K wps
[Epoch 17 Batch 1770/4268] avg loss 0.0113621, throughput 3.99953K wps
[Epoch 17 Batch 1800/4268] avg loss 0.0110295, throughput 4.00395K wps
[Epoch 17 Batch 1830/4268] avg loss 0.0104741, throughput 3.99527K wps
[Epoch 17 Batch 1860/4268] avg loss 0.0108734, throughput 4.00319K wps
[Epoch 17 Batch 1890/4268] avg loss 0.0110224, throughput 3.99791K wps
[Epoch 17 Batch 1920/4268] avg loss 0.0105091, throughput 3.99799K wps
[Epoch 17 Batch 1950/4268] avg loss 0.0107693, throughput 3.99792K wps
[Epoch 17 Batch 1980/4268] avg loss 0.0108219, throughput 3.99882K wps
[Epoch 17 Batch 2010/4268] avg loss 0.0111341, throughput 4.00068K wps
[Epoch 17 Batch 2040/4268] avg loss 0.0112214, throughput 3.99875K wps
[Epoch 17 Batch 2070/4268] avg loss 0.010699, throughput 4.00335K wps
[Epoch 17 Batch 2100/4268] avg loss 0.0115724, throughput 4.00246K wps
[Epoch 17 Batch 2130/4268] avg loss 0.0112223, throughput 4.0041K wps
[Epoch 17 Batch 2160/4268] avg loss 0.0110135, throughput 4.0042K wps
[Epoch 17 Batch 2190/4268] avg loss 0.0107098, throughput 4.00027K wps
[Epoch 17 Batch 2220/4268] avg loss 0.0108651, throughput 3.99668K wps
[Epoch 17 Batch 2250/4268] avg loss 0.0116135, throughput 3.99785K wps
[Epoch 17 Batch 2280/4268] avg loss 0.0108146, throughput 3.99658K wps
[Epoch 17 Batch 2310/4268] avg loss 0.010739, throughput 3.99831K wps
[Epoch 17 Batch 2340/4268] avg loss 0.0106115, throughput 3.99696K wps
[Epoch 17 Batch 2370/4268] avg loss 0.0108995, throughput 4.00252K wps
[Epoch 17 Batch 2400/4268] avg loss 0.0116735, throughput 4.00148K wps
[Epoch 17 Batch 2430/4268] avg loss 0.0108961, throughput 3.99964K wps
[Epoch 17 Batch 2460/4268] avg loss 0.0108751, throughput 3.99779K wps
[Epoch 17 Batch 2490/4268] avg loss 0.0112038, throughput 3.99819K wps
[Epoch 17 Batch 2520/4268] avg loss 0.0113361, throughput 3.99319K wps
[Epoch 17 Batch 2550/4268] avg loss 0.0112401, throughput 3.9946K wps
[Epoch 17 Batch 2580/4268] avg loss 0.0110276, throughput 3.99566K wps
[Epoch 17 Batch 2610/4268] avg loss 0.010905, throughput 3.99795K wps
[Epoch 17 Batch 2640/4268] avg loss 0.0112083, throughput 3.99532K wps
[Epoch 17 Batch 2670/4268] avg loss 0.0116678, throughput 3.99849K wps
[Epoch 17 Batch 2700/4268] avg loss 0.0114281, throughput 3.99317K wps
[Epoch 17 Batch 2730/4268] avg loss 0.0113997, throughput 3.99164K wps
[Epoch 17 Batch 2760/4268] avg loss 0.0114993, throughput 3.99642K wps
[Epoch 17 Batch 2790/4268] avg loss 0.0113359, throughput 3.99796K wps
[Epoch 17 Batch 2820/4268] avg loss 0.0110863, throughput 3.9953K wps
[Epoch 17 Batch 2850/4268] avg loss 0.0109932, throughput 4.00058K wps
[Epoch 17 Batch 2880/4268] avg loss 0.0118342, throughput 4.00122K wps
[Epoch 17 Batch 2910/4268] avg loss 0.0109391, throughput 4.00229K wps
[Epoch 17 Batch 2940/4268] avg loss 0.0111662, throughput 4.00462K wps
[Epoch 17 Batch 2970/4268] avg loss 0.0113783, throughput 4.00164K wps
[Epoch 17 Batch 3000/4268] avg loss 0.0112778, throughput 3.99382K wps
[Epoch 17 Batch 3030/4268] avg loss 0.0109801, throughput 3.99587K wps
[Epoch 17 Batch 3060/4268] avg loss 0.0115884, throughput 3.99989K wps
[Epoch 17 Batch 3090/4268] avg loss 0.0106444, throughput 3.99689K wps
[Epoch 17 Batch 3120/4268] avg loss 0.0112461, throughput 3.99668K wps
[Epoch 17 Batch 3150/4268] avg loss 0.0106782, throughput 3.99846K wps
[Epoch 17 Batch 3180/4268] avg loss 0.0115253, throughput 3.99837K wps
[Epoch 17 Batch 3210/4268] avg loss 0.0108379, throughput 3.99675K wps
[Epoch 17 Batch 3240/4268] avg loss 0.0112431, throughput 3.99699K wps
[Epoch 17 Batch 3270/4268] avg loss 0.011541, throughput 3.99232K wps
[Epoch 17 Batch 3300/4268] avg loss 0.0110093, throughput 3.99666K wps
[Epoch 17 Batch 3330/4268] avg loss 0.0109715, throughput 3.9956K wps
[Epoch 17 Batch 3360/4268] avg loss 0.0106461, throughput 4.00052K wps
[Epoch 17 Batch 3390/4268] avg loss 0.0112659, throughput 3.99423K wps
[Epoch 17 Batch 3420/4268] avg loss 0.011087, throughput 3.99851K wps
[Epoch 17 Batch 3450/4268] avg loss 0.0109032, throughput 3.99518K wps
[Epoch 17 Batch 3480/4268] avg loss 0.0112421, throughput 3.99662K wps
[Epoch 17 Batch 3510/4268] avg loss 0.0113182, throughput 3.99834K wps
[Epoch 17 Batch 3540/4268] avg loss 0.0114677, throughput 3.99706K wps
[Epoch 17 Batch 3570/4268] avg loss 0.0109913, throughput 3.99544K wps
[Epoch 17 Batch 3600/4268] avg loss 0.0116099, throughput 3.99027K wps
[Epoch 17 Batch 3630/4268] avg loss 0.0115838, throughput 3.99992K wps
[Epoch 17 Batch 3660/4268] avg loss 0.0120242, throughput 3.99662K wps
[Epoch 17 Batch 3690/4268] avg loss 0.0109972, throughput 3.99445K wps
[Epoch 17 Batch 3720/4268] avg loss 0.0121998, throughput 3.99683K wps
[Epoch 17 Batch 3750/4268] avg loss 0.0114717, throughput 3.99878K wps
[Epoch 17 Batch 3780/4268] avg loss 0.0108264, throughput 3.99762K wps
[Epoch 17 Batch 3810/4268] avg loss 0.0109453, throughput 3.99629K wps
[Epoch 17 Batch 3840/4268] avg loss 0.0111042, throughput 3.99694K wps
[Epoch 17 Batch 3870/4268] avg loss 0.0111097, throughput 3.99543K wps
[Epoch 17 Batch 3900/4268] avg loss 0.0107706, throughput 3.99448K wps
[Epoch 17 Batch 3930/4268] avg loss 0.0113668, throughput 3.99794K wps
[Epoch 17 Batch 3960/4268] avg loss 0.0113011, throughput 3.99709K wps
[Epoch 17 Batch 3990/4268] avg loss 0.0114936, throughput 3.98904K wps
[Epoch 17 Batch 4020/4268] avg loss 0.0120264, throughput 3.97839K wps
[Epoch 17 Batch 4050/4268] avg loss 0.0115812, throughput 3.99531K wps
[Epoch 17 Batch 4080/4268] avg loss 0.0110815, throughput 3.99481K wps
[Epoch 17 Batch 4110/4268] avg loss 0.0122953, throughput 3.99316K wps
[Epoch 17 Batch 4140/4268] avg loss 0.0112317, throughput 3.99676K wps
[Epoch 17 Batch 4170/4268] avg loss 0.0120893, throughput 3.99791K wps
[Epoch 17 Batch 4200/4268] avg loss 0.0109942, throughput 3.9927K wps
[Epoch 17 Batch 4230/4268] avg loss 0.0116945, throughput 3.99252K wps
[Epoch 17 Batch 4260/4268] avg loss 0.0114172, throughput 3.99373K wps
Begin Testing...
[Batch 30/475] elapsed 0.46 s
[Batch 60/475] elapsed 0.43 s
[Batch 90/475] elapsed 0.43 s
[Batch 120/475] elapsed 0.43 s
[Batch 150/475] elapsed 0.43 s
[Batch 180/475] elapsed 0.43 s
[Batch 210/475] elapsed 0.43 s
[Batch 240/475] elapsed 0.43 s
[Batch 270/475] elapsed 0.43 s
[Batch 300/475] elapsed 0.43 s
[Batch 330/475] elapsed 0.43 s
[Batch 360/475] elapsed 0.43 s
[Batch 390/475] elapsed 0.43 s
[Batch 420/475] elapsed 0.43 s
[Batch 450/475] elapsed 0.43 s
[Epoch 17] train avg loss 0.0109903, test acc 0.6754, test avg loss 0.869825, throughput 3.99744K wps
[Epoch 18 Batch 30/4268] avg loss 0.0102201, throughput 4.08633K wps
[Epoch 18 Batch 60/4268] avg loss 0.0104906, throughput 3.9934K wps
[Epoch 18 Batch 90/4268] avg loss 0.0101747, throughput 3.99536K wps
[Epoch 18 Batch 120/4268] avg loss 0.0102452, throughput 3.99515K wps
[Epoch 18 Batch 150/4268] avg loss 0.0106831, throughput 3.99842K wps
[Epoch 18 Batch 180/4268] avg loss 0.0105392, throughput 3.99585K wps
[Epoch 18 Batch 210/4268] avg loss 0.0101479, throughput 3.99877K wps
[Epoch 18 Batch 240/4268] avg loss 0.0103425, throughput 3.99611K wps
[Epoch 18 Batch 270/4268] avg loss 0.00999411, throughput 3.99722K wps
[Epoch 18 Batch 300/4268] avg loss 0.0103111, throughput 3.99557K wps
[Epoch 18 Batch 330/4268] avg loss 0.0106143, throughput 3.99863K wps
[Epoch 18 Batch 360/4268] avg loss 0.00956317, throughput 3.99339K wps
[Epoch 18 Batch 390/4268] avg loss 0.010537, throughput 3.99605K wps
[Epoch 18 Batch 420/4268] avg loss 0.00993182, throughput 3.99182K wps
[Epoch 18 Batch 450/4268] avg loss 0.00965108, throughput 4.00075K wps
[Epoch 18 Batch 480/4268] avg loss 0.0105218, throughput 3.99795K wps
[Epoch 18 Batch 510/4268] avg loss 0.0102889, throughput 3.99636K wps
[Epoch 18 Batch 540/4268] avg loss 0.0104456, throughput 3.9934K wps
[Epoch 18 Batch 570/4268] avg loss 0.0105161, throughput 3.99445K wps
[Epoch 18 Batch 600/4268] avg loss 0.0103909, throughput 3.99553K wps
[Epoch 18 Batch 630/4268] avg loss 0.0101341, throughput 3.99019K wps
[Epoch 18 Batch 660/4268] avg loss 0.0105527, throughput 3.99458K wps
[Epoch 18 Batch 690/4268] avg loss 0.0112439, throughput 3.9992K wps
[Epoch 18 Batch 720/4268] avg loss 0.0103584, throughput 3.99705K wps
[Epoch 18 Batch 750/4268] avg loss 0.0112873, throughput 3.9956K wps
[Epoch 18 Batch 780/4268] avg loss 0.0100864, throughput 3.99088K wps
[Epoch 18 Batch 810/4268] avg loss 0.0110968, throughput 3.99531K wps
[Epoch 18 Batch 840/4268] avg loss 0.0101082, throughput 3.99378K wps
[Epoch 18 Batch 870/4268] avg loss 0.0104859, throughput 3.99844K wps
[Epoch 18 Batch 900/4268] avg loss 0.0107705, throughput 3.99665K wps
[Epoch 18 Batch 930/4268] avg loss 0.0107365, throughput 3.9995K wps
[Epoch 18 Batch 960/4268] avg loss 0.0106741, throughput 3.99661K wps
[Epoch 18 Batch 990/4268] avg loss 0.01004, throughput 3.9927K wps
[Epoch 18 Batch 1020/4268] avg loss 0.0102992, throughput 3.99892K wps
[Epoch 18 Batch 1050/4268] avg loss 0.0101489, throughput 4.00108K wps
[Epoch 18 Batch 1080/4268] avg loss 0.0105764, throughput 3.99955K wps
[Epoch 18 Batch 1110/4268] avg loss 0.0114789, throughput 3.99895K wps
[Epoch 18 Batch 1140/4268] avg loss 0.010633, throughput 3.99616K wps
[Epoch 18 Batch 1170/4268] avg loss 0.0109589, throughput 3.99809K wps
[Epoch 18 Batch 1200/4268] avg loss 0.0103082, throughput 3.99627K wps
[Epoch 18 Batch 1230/4268] avg loss 0.0113563, throughput 3.9978K wps
[Epoch 18 Batch 1260/4268] avg loss 0.00986076, throughput 4.00425K wps
[Epoch 18 Batch 1290/4268] avg loss 0.010586, throughput 3.99892K wps
[Epoch 18 Batch 1320/4268] avg loss 0.0111368, throughput 3.99627K wps
[Epoch 18 Batch 1350/4268] avg loss 0.0111334, throughput 3.99844K wps
[Epoch 18 Batch 1380/4268] avg loss 0.0106502, throughput 3.99961K wps
[Epoch 18 Batch 1410/4268] avg loss 0.0110555, throughput 3.99301K wps
[Epoch 18 Batch 1440/4268] avg loss 0.010932, throughput 3.99322K wps
[Epoch 18 Batch 1470/4268] avg loss 0.0101027, throughput 3.9959K wps
[Epoch 18 Batch 1500/4268] avg loss 0.0104009, throughput 3.99578K wps
[Epoch 18 Batch 1530/4268] avg loss 0.0106061, throughput 3.99461K wps
[Epoch 18 Batch 1560/4268] avg loss 0.0107085, throughput 3.99701K wps
[Epoch 18 Batch 1590/4268] avg loss 0.0109194, throughput 3.9967K wps
[Epoch 18 Batch 1620/4268] avg loss 0.0108149, throughput 3.99974K wps
[Epoch 18 Batch 1650/4268] avg loss 0.0109811, throughput 3.99606K wps
[Epoch 18 Batch 1680/4268] avg loss 0.0105039, throughput 3.99749K wps
[Epoch 18 Batch 1710/4268] avg loss 0.011387, throughput 3.99479K wps
[Epoch 18 Batch 1740/4268] avg loss 0.0110918, throughput 3.99458K wps
[Epoch 18 Batch 1770/4268] avg loss 0.0100543, throughput 3.99505K wps
[Epoch 18 Batch 1800/4268] avg loss 0.010942, throughput 3.99555K wps
[Epoch 18 Batch 1830/4268] avg loss 0.0106236, throughput 3.99216K wps
[Epoch 18 Batch 1860/4268] avg loss 0.0100221, throughput 3.99542K wps
[Epoch 18 Batch 1890/4268] avg loss 0.0111597, throughput 3.98927K wps
[Epoch 18 Batch 1920/4268] avg loss 0.0114201, throughput 3.99666K wps
[Epoch 18 Batch 1950/4268] avg loss 0.0110601, throughput 3.99023K wps
[Epoch 18 Batch 1980/4268] avg loss 0.0112366, throughput 3.99402K wps
[Epoch 18 Batch 2010/4268] avg loss 0.0105954, throughput 3.99791K wps
[Epoch 18 Batch 2040/4268] avg loss 0.0105686, throughput 3.99868K wps
[Epoch 18 Batch 2070/4268] avg loss 0.0110203, throughput 3.99239K wps
[Epoch 18 Batch 2100/4268] avg loss 0.0109499, throughput 3.99836K wps
[Epoch 18 Batch 2130/4268] avg loss 0.0108025, throughput 3.9949K wps
[Epoch 18 Batch 2160/4268] avg loss 0.0109361, throughput 3.99682K wps
[Epoch 18 Batch 2190/4268] avg loss 0.0107048, throughput 3.99578K wps
[Epoch 18 Batch 2220/4268] avg loss 0.0106713, throughput 3.99244K wps
[Epoch 18 Batch 2250/4268] avg loss 0.0103415, throughput 3.99908K wps
[Epoch 18 Batch 2280/4268] avg loss 0.0109205, throughput 3.99643K wps
[Epoch 18 Batch 2310/4268] avg loss 0.0111624, throughput 3.99888K wps
[Epoch 18 Batch 2340/4268] avg loss 0.0104774, throughput 3.99801K wps
[Epoch 18 Batch 2370/4268] avg loss 0.010681, throughput 3.9957K wps
[Epoch 18 Batch 2400/4268] avg loss 0.0105635, throughput 3.99482K wps
[Epoch 18 Batch 2430/4268] avg loss 0.0106546, throughput 3.99489K wps
[Epoch 18 Batch 2460/4268] avg loss 0.0114595, throughput 3.9937K wps
[Epoch 18 Batch 2490/4268] avg loss 0.0111205, throughput 3.99827K wps
[Epoch 18 Batch 2520/4268] avg loss 0.0111366, throughput 3.9989K wps
[Epoch 18 Batch 2550/4268] avg loss 0.0104781, throughput 3.99646K wps
[Epoch 18 Batch 2580/4268] avg loss 0.0107547, throughput 3.99501K wps
[Epoch 18 Batch 2610/4268] avg loss 0.0108893, throughput 3.99589K wps
[Epoch 18 Batch 2640/4268] avg loss 0.0113239, throughput 3.99635K wps
[Epoch 18 Batch 2670/4268] avg loss 0.0110244, throughput 3.9939K wps
[Epoch 18 Batch 2700/4268] avg loss 0.0105224, throughput 3.99907K wps
[Epoch 18 Batch 2730/4268] avg loss 0.0106011, throughput 3.9977K wps
[Epoch 18 Batch 2760/4268] avg loss 0.011514, throughput 3.99762K wps
[Epoch 18 Batch 2790/4268] avg loss 0.0106641, throughput 3.99588K wps
[Epoch 18 Batch 2820/4268] avg loss 0.01129, throughput 3.99352K wps
[Epoch 18 Batch 2850/4268] avg loss 0.0109097, throughput 3.99964K wps
[Epoch 18 Batch 2880/4268] avg loss 0.0105204, throughput 4.00254K wps
[Epoch 18 Batch 2910/4268] avg loss 0.0108219, throughput 4.00115K wps
[Epoch 18 Batch 2940/4268] avg loss 0.0105863, throughput 4.00312K wps
[Epoch 18 Batch 2970/4268] avg loss 0.0107528, throughput 4.00286K wps
[Epoch 18 Batch 3000/4268] avg loss 0.0115329, throughput 3.99981K wps
[Epoch 18 Batch 3030/4268] avg loss 0.0113147, throughput 4.00676K wps
[Epoch 18 Batch 3060/4268] avg loss 0.010584, throughput 3.99887K wps
[Epoch 18 Batch 3090/4268] avg loss 0.0108954, throughput 4.0016K wps
[Epoch 18 Batch 3120/4268] avg loss 0.011483, throughput 3.99703K wps
[Epoch 18 Batch 3150/4268] avg loss 0.0105106, throughput 3.99581K wps
[Epoch 18 Batch 3180/4268] avg loss 0.0116707, throughput 3.99267K wps
[Epoch 18 Batch 3210/4268] avg loss 0.0109888, throughput 3.98316K wps
[Epoch 18 Batch 3240/4268] avg loss 0.0107049, throughput 3.99463K wps
[Epoch 18 Batch 3270/4268] avg loss 0.0113274, throughput 3.99514K wps
[Epoch 18 Batch 3300/4268] avg loss 0.0111148, throughput 3.9978K wps
[Epoch 18 Batch 3330/4268] avg loss 0.0107214, throughput 4.00081K wps
[Epoch 18 Batch 3360/4268] avg loss 0.010818, throughput 3.99894K wps
[Epoch 18 Batch 3390/4268] avg loss 0.0105729, throughput 3.99707K wps
[Epoch 18 Batch 3420/4268] avg loss 0.0110455, throughput 4.00173K wps
[Epoch 18 Batch 3450/4268] avg loss 0.0108847, throughput 3.99882K wps
[Epoch 18 Batch 3480/4268] avg loss 0.0106225, throughput 3.99895K wps
[Epoch 18 Batch 3510/4268] avg loss 0.0107565, throughput 3.9973K wps
[Epoch 18 Batch 3540/4268] avg loss 0.011138, throughput 3.99945K wps
[Epoch 18 Batch 3570/4268] avg loss 0.0106794, throughput 3.99796K wps
[Epoch 18 Batch 3600/4268] avg loss 0.0114511, throughput 3.99741K wps
[Epoch 18 Batch 3630/4268] avg loss 0.0117479, throughput 4.0024K wps
[Epoch 18 Batch 3660/4268] avg loss 0.0119588, throughput 3.99864K wps
[Epoch 18 Batch 3690/4268] avg loss 0.0110449, throughput 3.9987K wps
[Epoch 18 Batch 3720/4268] avg loss 0.0108334, throughput 3.99494K wps
[Epoch 18 Batch 3750/4268] avg loss 0.0113478, throughput 3.99694K wps
[Epoch 18 Batch 3780/4268] avg loss 0.0110212, throughput 3.99398K wps
[Epoch 18 Batch 3810/4268] avg loss 0.0111797, throughput 3.99556K wps
[Epoch 18 Batch 3840/4268] avg loss 0.0117213, throughput 3.99863K wps
[Epoch 18 Batch 3870/4268] avg loss 0.0107219, throughput 3.99892K wps
[Epoch 18 Batch 3900/4268] avg loss 0.0111966, throughput 3.99485K wps
[Epoch 18 Batch 3930/4268] avg loss 0.0110358, throughput 3.998K wps
[Epoch 18 Batch 3960/4268] avg loss 0.0107251, throughput 3.99263K wps
[Epoch 18 Batch 3990/4268] avg loss 0.0108188, throughput 3.99331K wps
[Epoch 18 Batch 4020/4268] avg loss 0.011579, throughput 3.996K wps
[Epoch 18 Batch 4050/4268] avg loss 0.0104868, throughput 3.99686K wps
[Epoch 18 Batch 4080/4268] avg loss 0.0112618, throughput 3.99812K wps
[Epoch 18 Batch 4110/4268] avg loss 0.0105889, throughput 3.99542K wps
[Epoch 18 Batch 4140/4268] avg loss 0.0112628, throughput 3.99738K wps
[Epoch 18 Batch 4170/4268] avg loss 0.0114447, throughput 3.99729K wps
[Epoch 18 Batch 4200/4268] avg loss 0.0109792, throughput 3.99738K wps
[Epoch 18 Batch 4230/4268] avg loss 0.0110026, throughput 3.99527K wps
[Epoch 18 Batch 4260/4268] avg loss 0.011568, throughput 3.99964K wps
Begin Testing...
[Batch 30/475] elapsed 0.46 s
[Batch 60/475] elapsed 0.43 s
[Batch 90/475] elapsed 0.43 s
[Batch 120/475] elapsed 0.43 s
[Batch 150/475] elapsed 0.43 s
[Batch 180/475] elapsed 0.43 s
[Batch 210/475] elapsed 0.43 s
[Batch 240/475] elapsed 0.43 s
[Batch 270/475] elapsed 0.43 s
[Batch 300/475] elapsed 0.43 s
[Batch 330/475] elapsed 0.43 s
[Batch 360/475] elapsed 0.43 s
[Batch 390/475] elapsed 0.43 s
[Batch 420/475] elapsed 0.43 s
[Batch 450/475] elapsed 0.43 s
[Epoch 18] train avg loss 0.0107912, test acc 0.6763, test avg loss 0.880881, throughput 3.99729K wps
[Epoch 19 Batch 30/4268] avg loss 0.0101979, throughput 4.08258K wps
[Epoch 19 Batch 60/4268] avg loss 0.00964499, throughput 3.99384K wps
[Epoch 19 Batch 90/4268] avg loss 0.0102169, throughput 3.99785K wps
[Epoch 19 Batch 120/4268] avg loss 0.0104177, throughput 3.99771K wps
[Epoch 19 Batch 150/4268] avg loss 0.0102024, throughput 4.00012K wps
[Epoch 19 Batch 180/4268] avg loss 0.00985025, throughput 3.9945K wps
[Epoch 19 Batch 210/4268] avg loss 0.0102111, throughput 3.99393K wps
[Epoch 19 Batch 240/4268] avg loss 0.0100603, throughput 3.99703K wps
[Epoch 19 Batch 270/4268] avg loss 0.00997214, throughput 3.99782K wps
[Epoch 19 Batch 300/4268] avg loss 0.0101952, throughput 3.999K wps
[Epoch 19 Batch 330/4268] avg loss 0.00994822, throughput 3.99849K wps
[Epoch 19 Batch 360/4268] avg loss 0.010145, throughput 4.00498K wps
[Epoch 19 Batch 390/4268] avg loss 0.0104153, throughput 3.99925K wps
[Epoch 19 Batch 420/4268] avg loss 0.0109742, throughput 3.99757K wps
[Epoch 19 Batch 450/4268] avg loss 0.0104746, throughput 3.99493K wps
[Epoch 19 Batch 480/4268] avg loss 0.010236, throughput 3.99718K wps
[Epoch 19 Batch 510/4268] avg loss 0.00993207, throughput 4.00302K wps
[Epoch 19 Batch 540/4268] avg loss 0.00995712, throughput 3.99279K wps
[Epoch 19 Batch 570/4268] avg loss 0.0102943, throughput 3.99572K wps
[Epoch 19 Batch 600/4268] avg loss 0.0102984, throughput 4.00357K wps
[Epoch 19 Batch 630/4268] avg loss 0.0100453, throughput 3.99239K wps
[Epoch 19 Batch 660/4268] avg loss 0.0100714, throughput 3.99791K wps
[Epoch 19 Batch 690/4268] avg loss 0.00952917, throughput 3.99749K wps
[Epoch 19 Batch 720/4268] avg loss 0.0100372, throughput 3.99876K wps
[Epoch 19 Batch 750/4268] avg loss 0.009963, throughput 3.99711K wps
[Epoch 19 Batch 780/4268] avg loss 0.0107961, throughput 3.99575K wps
[Epoch 19 Batch 810/4268] avg loss 0.0105433, throughput 3.99599K wps
[Epoch 19 Batch 840/4268] avg loss 0.00999397, throughput 3.99662K wps
[Epoch 19 Batch 870/4268] avg loss 0.0099675, throughput 3.99366K wps
[Epoch 19 Batch 900/4268] avg loss 0.0103341, throughput 3.99706K wps
[Epoch 19 Batch 930/4268] avg loss 0.0104205, throughput 3.99116K wps
[Epoch 19 Batch 960/4268] avg loss 0.0106325, throughput 3.99493K wps
[Epoch 19 Batch 990/4268] avg loss 0.0107363, throughput 3.99652K wps
[Epoch 19 Batch 1020/4268] avg loss 0.0106581, throughput 3.99948K wps
[Epoch 19 Batch 1050/4268] avg loss 0.0109135, throughput 4.00682K wps
[Epoch 19 Batch 1080/4268] avg loss 0.0107711, throughput 3.99606K wps
[Epoch 19 Batch 1110/4268] avg loss 0.0101365, throughput 3.99622K wps
[Epoch 19 Batch 1140/4268] avg loss 0.0103729, throughput 3.99463K wps
[Epoch 19 Batch 1170/4268] avg loss 0.010264, throughput 3.9956K wps
[Epoch 19 Batch 1200/4268] avg loss 0.0103322, throughput 3.99434K wps
[Epoch 19 Batch 1230/4268] avg loss 0.010317, throughput 3.99734K wps
[Epoch 19 Batch 1260/4268] avg loss 0.0111653, throughput 3.99391K wps
[Epoch 19 Batch 1290/4268] avg loss 0.0102957, throughput 3.99716K wps
[Epoch 19 Batch 1320/4268] avg loss 0.0104676, throughput 3.99857K wps
[Epoch 19 Batch 1350/4268] avg loss 0.010249, throughput 3.99388K wps
[Epoch 19 Batch 1380/4268] avg loss 0.0108502, throughput 3.99594K wps
[Epoch 19 Batch 1410/4268] avg loss 0.0101551, throughput 3.99417K wps
[Epoch 19 Batch 1440/4268] avg loss 0.0107495, throughput 3.9977K wps
[Epoch 19 Batch 1470/4268] avg loss 0.010435, throughput 3.99522K wps
[Epoch 19 Batch 1500/4268] avg loss 0.0100765, throughput 3.99277K wps
[Epoch 19 Batch 1530/4268] avg loss 0.0108904, throughput 3.99889K wps
[Epoch 19 Batch 1560/4268] avg loss 0.0110525, throughput 3.99706K wps
[Epoch 19 Batch 1590/4268] avg loss 0.00994495, throughput 3.99725K wps
[Epoch 19 Batch 1620/4268] avg loss 0.00987827, throughput 3.99334K wps
[Epoch 19 Batch 1650/4268] avg loss 0.0108247, throughput 4.00201K wps
[Epoch 19 Batch 1680/4268] avg loss 0.0103128, throughput 3.99698K wps
[Epoch 19 Batch 1710/4268] avg loss 0.0110762, throughput 3.99492K wps
[Epoch 19 Batch 1740/4268] avg loss 0.0106843, throughput 3.99659K wps
[Epoch 19 Batch 1770/4268] avg loss 0.00980121, throughput 3.99397K wps
[Epoch 19 Batch 1800/4268] avg loss 0.0108091, throughput 3.99345K wps
[Epoch 19 Batch 1830/4268] avg loss 0.010779, throughput 3.99417K wps
[Epoch 19 Batch 1860/4268] avg loss 0.0100909, throughput 3.99506K wps
[Epoch 19 Batch 1890/4268] avg loss 0.0103053, throughput 3.99503K wps
[Epoch 19 Batch 1920/4268] avg loss 0.0109075, throughput 3.99809K wps
[Epoch 19 Batch 1950/4268] avg loss 0.0103144, throughput 4.00137K wps
[Epoch 19 Batch 1980/4268] avg loss 0.0111065, throughput 3.99111K wps
[Epoch 19 Batch 2010/4268] avg loss 0.0101683, throughput 3.99894K wps
[Epoch 19 Batch 2040/4268] avg loss 0.0106477, throughput 4.00157K wps
[Epoch 19 Batch 2070/4268] avg loss 0.0111005, throughput 4.00101K wps
[Epoch 19 Batch 2100/4268] avg loss 0.0103923, throughput 3.99821K wps
[Epoch 19 Batch 2130/4268] avg loss 0.0106784, throughput 3.99255K wps
[Epoch 19 Batch 2160/4268] avg loss 0.0102506, throughput 3.99278K wps
[Epoch 19 Batch 2190/4268] avg loss 0.0116161, throughput 3.99182K wps
[Epoch 19 Batch 2220/4268] avg loss 0.0102006, throughput 3.99246K wps
[Epoch 19 Batch 2250/4268] avg loss 0.0112517, throughput 3.99536K wps
[Epoch 19 Batch 2280/4268] avg loss 0.010687, throughput 3.99534K wps
[Epoch 19 Batch 2310/4268] avg loss 0.0109208, throughput 3.99575K wps
[Epoch 19 Batch 2340/4268] avg loss 0.0104536, throughput 3.99538K wps
[Epoch 19 Batch 2370/4268] avg loss 0.0105497, throughput 3.99236K wps
[Epoch 19 Batch 2400/4268] avg loss 0.0107579, throughput 3.98147K wps
[Epoch 19 Batch 2430/4268] avg loss 0.0108184, throughput 3.99452K wps
[Epoch 19 Batch 2460/4268] avg loss 0.0110273, throughput 3.99488K wps
[Epoch 19 Batch 2490/4268] avg loss 0.0103499, throughput 3.99087K wps
[Epoch 19 Batch 2520/4268] avg loss 0.0103826, throughput 3.99047K wps
[Epoch 19 Batch 2550/4268] avg loss 0.0105711, throughput 3.9968K wps
[Epoch 19 Batch 2580/4268] avg loss 0.0106387, throughput 3.99647K wps
[Epoch 19 Batch 2610/4268] avg loss 0.0109301, throughput 3.99582K wps
[Epoch 19 Batch 2640/4268] avg loss 0.011158, throughput 3.99066K wps
[Epoch 19 Batch 2670/4268] avg loss 0.0109235, throughput 3.99706K wps
[Epoch 19 Batch 2700/4268] avg loss 0.0103613, throughput 3.99264K wps
[Epoch 19 Batch 2730/4268] avg loss 0.0105992, throughput 3.99909K wps
[Epoch 19 Batch 2760/4268] avg loss 0.0102733, throughput 3.98996K wps
[Epoch 19 Batch 2790/4268] avg loss 0.0103426, throughput 3.99649K wps
[Epoch 19 Batch 2820/4268] avg loss 0.0106652, throughput 3.99045K wps
[Epoch 19 Batch 2850/4268] avg loss 0.0110399, throughput 3.99332K wps
[Epoch 19 Batch 2880/4268] avg loss 0.0104511, throughput 3.99364K wps
[Epoch 19 Batch 2910/4268] avg loss 0.0109556, throughput 3.99332K wps
[Epoch 19 Batch 2940/4268] avg loss 0.0104237, throughput 3.9931K wps
[Epoch 19 Batch 2970/4268] avg loss 0.011209, throughput 3.99786K wps
[Epoch 19 Batch 3000/4268] avg loss 0.0107572, throughput 4.0021K wps
[Epoch 19 Batch 3030/4268] avg loss 0.0113128, throughput 4.00583K wps
[Epoch 19 Batch 3060/4268] avg loss 0.0104102, throughput 4.00779K wps
[Epoch 19 Batch 3090/4268] avg loss 0.0107389, throughput 4.01382K wps
[Epoch 19 Batch 3120/4268] avg loss 0.0112, throughput 4.01247K wps
[Epoch 19 Batch 3150/4268] avg loss 0.0108127, throughput 4.00611K wps
[Epoch 19 Batch 3180/4268] avg loss 0.0108339, throughput 3.99901K wps
[Epoch 19 Batch 3210/4268] avg loss 0.0110542, throughput 4.00031K wps
[Epoch 19 Batch 3240/4268] avg loss 0.0117658, throughput 3.99583K wps
[Epoch 19 Batch 3270/4268] avg loss 0.010494, throughput 4.00156K wps
[Epoch 19 Batch 3300/4268] avg loss 0.0107219, throughput 3.99772K wps
[Epoch 19 Batch 3330/4268] avg loss 0.0112616, throughput 4.00125K wps
[Epoch 19 Batch 3360/4268] avg loss 0.010991, throughput 4.00543K wps
[Epoch 19 Batch 3390/4268] avg loss 0.0109574, throughput 3.99702K wps
[Epoch 19 Batch 3420/4268] avg loss 0.010788, throughput 3.99948K wps
[Epoch 19 Batch 3450/4268] avg loss 0.0114347, throughput 3.99541K wps
[Epoch 19 Batch 3480/4268] avg loss 0.0107245, throughput 3.99522K wps
[Epoch 19 Batch 3510/4268] avg loss 0.0112267, throughput 3.99433K wps
[Epoch 19 Batch 3540/4268] avg loss 0.0103433, throughput 3.99449K wps
[Epoch 19 Batch 3570/4268] avg loss 0.0113943, throughput 4.00264K wps
[Epoch 19 Batch 3600/4268] avg loss 0.0119062, throughput 3.9947K wps
[Epoch 19 Batch 3630/4268] avg loss 0.0103789, throughput 3.99313K wps
[Epoch 19 Batch 3660/4268] avg loss 0.0107903, throughput 3.99351K wps
[Epoch 19 Batch 3690/4268] avg loss 0.0110601, throughput 3.98958K wps
[Epoch 19 Batch 3720/4268] avg loss 0.010381, throughput 3.99483K wps
[Epoch 19 Batch 3750/4268] avg loss 0.0120337, throughput 3.99351K wps
[Epoch 19 Batch 3780/4268] avg loss 0.0106697, throughput 4.00039K wps
[Epoch 19 Batch 3810/4268] avg loss 0.0109063, throughput 3.99654K wps
[Epoch 19 Batch 3840/4268] avg loss 0.0111287, throughput 3.99645K wps
[Epoch 19 Batch 3870/4268] avg loss 0.0111859, throughput 3.9989K wps
[Epoch 19 Batch 3900/4268] avg loss 0.0106861, throughput 3.99157K wps
[Epoch 19 Batch 3930/4268] avg loss 0.0109689, throughput 3.99618K wps
[Epoch 19 Batch 3960/4268] avg loss 0.0112309, throughput 3.99483K wps
[Epoch 19 Batch 3990/4268] avg loss 0.0112113, throughput 3.99914K wps
[Epoch 19 Batch 4020/4268] avg loss 0.0105891, throughput 3.99368K wps
[Epoch 19 Batch 4050/4268] avg loss 0.0103172, throughput 3.99634K wps
[Epoch 19 Batch 4080/4268] avg loss 0.0109741, throughput 3.99465K wps
[Epoch 19 Batch 4110/4268] avg loss 0.0109256, throughput 3.99468K wps
[Epoch 19 Batch 4140/4268] avg loss 0.0109982, throughput 3.99686K wps
[Epoch 19 Batch 4170/4268] avg loss 0.0115168, throughput 3.99351K wps
[Epoch 19 Batch 4200/4268] avg loss 0.0116247, throughput 3.99688K wps
[Epoch 19 Batch 4230/4268] avg loss 0.0103953, throughput 3.99561K wps
[Epoch 19 Batch 4260/4268] avg loss 0.0107663, throughput 3.9977K wps
Begin Testing...
[Batch 30/475] elapsed 0.46 s
[Batch 60/475] elapsed 0.43 s
[Batch 90/475] elapsed 0.43 s
[Batch 120/475] elapsed 0.43 s
[Batch 150/475] elapsed 0.43 s
[Batch 180/475] elapsed 0.43 s
[Batch 210/475] elapsed 0.43 s
[Batch 240/475] elapsed 0.43 s
[Batch 270/475] elapsed 0.43 s
[Batch 300/475] elapsed 0.43 s
[Batch 330/475] elapsed 0.43 s
[Batch 360/475] elapsed 0.43 s
[Batch 390/475] elapsed 0.43 s
[Batch 420/475] elapsed 0.43 s
[Batch 450/475] elapsed 0.43 s
[Epoch 19] train avg loss 0.0106238, test acc 0.6747, test avg loss 0.894783, throughput 3.99714K wps
[Epoch 20 Batch 30/4268] avg loss 0.00984714, throughput 4.08319K wps
[Epoch 20 Batch 60/4268] avg loss 0.0104099, throughput 3.99812K wps
[Epoch 20 Batch 90/4268] avg loss 0.0100199, throughput 3.99947K wps
[Epoch 20 Batch 120/4268] avg loss 0.0103157, throughput 4.00036K wps
[Epoch 20 Batch 150/4268] avg loss 0.00978964, throughput 3.99764K wps
[Epoch 20 Batch 180/4268] avg loss 0.0098397, throughput 3.99957K wps
[Epoch 20 Batch 210/4268] avg loss 0.0109088, throughput 4.00041K wps
[Epoch 20 Batch 240/4268] avg loss 0.0101165, throughput 4.00076K wps
[Epoch 20 Batch 270/4268] avg loss 0.00980931, throughput 3.99751K wps
[Epoch 20 Batch 300/4268] avg loss 0.00995867, throughput 3.99643K wps
[Epoch 20 Batch 330/4268] avg loss 0.0100483, throughput 4.00234K wps
[Epoch 20 Batch 360/4268] avg loss 0.0101412, throughput 3.99968K wps
[Epoch 20 Batch 390/4268] avg loss 0.0100282, throughput 3.99501K wps
[Epoch 20 Batch 420/4268] avg loss 0.0102852, throughput 3.99971K wps
[Epoch 20 Batch 450/4268] avg loss 0.0104033, throughput 4.00582K wps
[Epoch 20 Batch 480/4268] avg loss 0.00996505, throughput 3.99474K wps
[Epoch 20 Batch 510/4268] avg loss 0.0101168, throughput 3.9975K wps
[Epoch 20 Batch 540/4268] avg loss 0.00989795, throughput 3.99667K wps
[Epoch 20 Batch 570/4268] avg loss 0.0100859, throughput 3.99344K wps
[Epoch 20 Batch 600/4268] avg loss 0.00998105, throughput 3.99859K wps
[Epoch 20 Batch 630/4268] avg loss 0.0101736, throughput 3.99981K wps
[Epoch 20 Batch 660/4268] avg loss 0.00998968, throughput 3.99697K wps
[Epoch 20 Batch 690/4268] avg loss 0.0101457, throughput 3.99862K wps
[Epoch 20 Batch 720/4268] avg loss 0.0101028, throughput 3.99601K wps
[Epoch 20 Batch 750/4268] avg loss 0.010035, throughput 3.99757K wps
[Epoch 20 Batch 780/4268] avg loss 0.00934197, throughput 3.9965K wps
[Epoch 20 Batch 810/4268] avg loss 0.0100513, throughput 3.99425K wps
[Epoch 20 Batch 840/4268] avg loss 0.0102609, throughput 3.9975K wps
[Epoch 20 Batch 870/4268] avg loss 0.0100254, throughput 3.99516K wps
[Epoch 20 Batch 900/4268] avg loss 0.00958218, throughput 3.99609K wps
[Epoch 20 Batch 930/4268] avg loss 0.0100729, throughput 3.99808K wps
[Epoch 20 Batch 960/4268] avg loss 0.0100395, throughput 3.99866K wps
[Epoch 20 Batch 990/4268] avg loss 0.0101118, throughput 3.99806K wps
[Epoch 20 Batch 1020/4268] avg loss 0.0100561, throughput 3.99751K wps
[Epoch 20 Batch 1050/4268] avg loss 0.0102863, throughput 3.9999K wps
[Epoch 20 Batch 1080/4268] avg loss 0.0109256, throughput 4.00022K wps
[Epoch 20 Batch 1110/4268] avg loss 0.0104975, throughput 3.99803K wps
[Epoch 20 Batch 1140/4268] avg loss 0.010235, throughput 3.99625K wps
[Epoch 20 Batch 1170/4268] avg loss 0.0103192, throughput 3.99488K wps
[Epoch 20 Batch 1200/4268] avg loss 0.0100006, throughput 3.99702K wps
[Epoch 20 Batch 1230/4268] avg loss 0.0104096, throughput 3.99698K wps
[Epoch 20 Batch 1260/4268] avg loss 0.00970537, throughput 3.99702K wps
[Epoch 20 Batch 1290/4268] avg loss 0.0101718, throughput 3.99305K wps
[Epoch 20 Batch 1320/4268] avg loss 0.0102069, throughput 3.99552K wps
[Epoch 20 Batch 1350/4268] avg loss 0.00978947, throughput 3.99882K wps
[Epoch 20 Batch 1380/4268] avg loss 0.0102077, throughput 3.99635K wps
[Epoch 20 Batch 1410/4268] avg loss 0.0106232, throughput 3.99845K wps
[Epoch 20 Batch 1440/4268] avg loss 0.0105494, throughput 3.99539K wps
[Epoch 20 Batch 1470/4268] avg loss 0.00998977, throughput 3.99868K wps
[Epoch 20 Batch 1500/4268] avg loss 0.0108943, throughput 4.0005K wps
[Epoch 20 Batch 1530/4268] avg loss 0.0106278, throughput 3.99919K wps
[Epoch 20 Batch 1560/4268] avg loss 0.010391, throughput 3.99773K wps
[Epoch 20 Batch 1590/4268] avg loss 0.0110078, throughput 3.98021K wps
[Epoch 20 Batch 1620/4268] avg loss 0.0102698, throughput 3.98912K wps
[Epoch 20 Batch 1650/4268] avg loss 0.010603, throughput 3.99585K wps
[Epoch 20 Batch 1680/4268] avg loss 0.010392, throughput 3.99309K wps
[Epoch 20 Batch 1710/4268] avg loss 0.0110567, throughput 3.99653K wps
[Epoch 20 Batch 1740/4268] avg loss 0.0109492, throughput 3.99687K wps
[Epoch 20 Batch 1770/4268] avg loss 0.010306, throughput 3.99532K wps
[Epoch 20 Batch 1800/4268] avg loss 0.0107166, throughput 3.99798K wps
[Epoch 20 Batch 1830/4268] avg loss 0.0104542, throughput 3.9984K wps
[Epoch 20 Batch 1860/4268] avg loss 0.00982839, throughput 3.99445K wps
[Epoch 20 Batch 1890/4268] avg loss 0.0104115, throughput 3.99483K wps
[Epoch 20 Batch 1920/4268] avg loss 0.00986094, throughput 3.99341K wps
[Epoch 20 Batch 1950/4268] avg loss 0.0100745, throughput 3.99643K wps
[Epoch 20 Batch 1980/4268] avg loss 0.0106774, throughput 3.99331K wps
[Epoch 20 Batch 2010/4268] avg loss 0.00956869, throughput 3.99681K wps
[Epoch 20 Batch 2040/4268] avg loss 0.010239, throughput 3.9947K wps
[Epoch 20 Batch 2070/4268] avg loss 0.0097816, throughput 3.99487K wps
[Epoch 20 Batch 2100/4268] avg loss 0.0109919, throughput 3.99767K wps
[Epoch 20 Batch 2130/4268] avg loss 0.0101445, throughput 3.99838K wps
[Epoch 20 Batch 2160/4268] avg loss 0.010003, throughput 3.99917K wps
[Epoch 20 Batch 2190/4268] avg loss 0.0101862, throughput 3.9966K wps
[Epoch 20 Batch 2220/4268] avg loss 0.010471, throughput 3.99863K wps
[Epoch 20 Batch 2250/4268] avg loss 0.0101289, throughput 3.99275K wps
[Epoch 20 Batch 2280/4268] avg loss 0.00984671, throughput 3.99645K wps
[Epoch 20 Batch 2310/4268] avg loss 0.0102226, throughput 3.99908K wps
[Epoch 20 Batch 2340/4268] avg loss 0.0110677, throughput 3.99395K wps
[Epoch 20 Batch 2370/4268] avg loss 0.010757, throughput 3.99298K wps
[Epoch 20 Batch 2400/4268] avg loss 0.0111624, throughput 3.99494K wps
[Epoch 20 Batch 2430/4268] avg loss 0.0108506, throughput 3.99625K wps
[Epoch 20 Batch 2460/4268] avg loss 0.010767, throughput 3.99651K wps
[Epoch 20 Batch 2490/4268] avg loss 0.0111664, throughput 3.99981K wps
[Epoch 20 Batch 2520/4268] avg loss 0.0106149, throughput 3.99322K wps
[Epoch 20 Batch 2550/4268] avg loss 0.0103534, throughput 4.00014K wps
[Epoch 20 Batch 2580/4268] avg loss 0.0103882, throughput 3.9966K wps
[Epoch 20 Batch 2610/4268] avg loss 0.010284, throughput 3.99771K wps
[Epoch 20 Batch 2640/4268] avg loss 0.00981264, throughput 3.99799K wps
[Epoch 20 Batch 2670/4268] avg loss 0.0107962, throughput 3.99481K wps
[Epoch 20 Batch 2700/4268] avg loss 0.010448, throughput 3.99738K wps
[Epoch 20 Batch 2730/4268] avg loss 0.0117467, throughput 3.99522K wps
[Epoch 20 Batch 2760/4268] avg loss 0.0104072, throughput 3.99594K wps
[Epoch 20 Batch 2790/4268] avg loss 0.0105037, throughput 3.99755K wps
[Epoch 20 Batch 2820/4268] avg loss 0.0109682, throughput 3.99134K wps
[Epoch 20 Batch 2850/4268] avg loss 0.0102644, throughput 3.99989K wps
[Epoch 20 Batch 2880/4268] avg loss 0.0102844, throughput 3.99764K wps
[Epoch 20 Batch 2910/4268] avg loss 0.0106403, throughput 3.99873K wps
[Epoch 20 Batch 2940/4268] avg loss 0.0105367, throughput 3.99885K wps
[Epoch 20 Batch 2970/4268] avg loss 0.00999974, throughput 3.99988K wps
[Epoch 20 Batch 3000/4268] avg loss 0.0102167, throughput 3.99494K wps
[Epoch 20 Batch 3030/4268] avg loss 0.0101668, throughput 4.00063K wps
[Epoch 20 Batch 3060/4268] avg loss 0.0106518, throughput 3.99399K wps
[Epoch 20 Batch 3090/4268] avg loss 0.010604, throughput 4.00046K wps
[Epoch 20 Batch 3120/4268] avg loss 0.0110695, throughput 4.00101K wps
[Epoch 20 Batch 3150/4268] avg loss 0.0103578, throughput 3.99827K wps
[Epoch 20 Batch 3180/4268] avg loss 0.0103486, throughput 3.99778K wps
[Epoch 20 Batch 3210/4268] avg loss 0.0108537, throughput 3.99949K wps
[Epoch 20 Batch 3240/4268] avg loss 0.0107367, throughput 3.99528K wps
[Epoch 20 Batch 3270/4268] avg loss 0.0105203, throughput 4.00521K wps
[Epoch 20 Batch 3300/4268] avg loss 0.0103782, throughput 4.0027K wps
[Epoch 20 Batch 3330/4268] avg loss 0.0111374, throughput 3.99862K wps
[Epoch 20 Batch 3360/4268] avg loss 0.010856, throughput 3.99395K wps
[Epoch 20 Batch 3390/4268] avg loss 0.0107155, throughput 3.99505K wps
[Epoch 20 Batch 3420/4268] avg loss 0.0109831, throughput 3.99868K wps
[Epoch 20 Batch 3450/4268] avg loss 0.0106386, throughput 3.99995K wps
[Epoch 20 Batch 3480/4268] avg loss 0.0106086, throughput 3.99562K wps
[Epoch 20 Batch 3510/4268] avg loss 0.0111394, throughput 3.99898K wps
[Epoch 20 Batch 3540/4268] avg loss 0.0109358, throughput 3.99614K wps
[Epoch 20 Batch 3570/4268] avg loss 0.0107697, throughput 3.99645K wps
[Epoch 20 Batch 3600/4268] avg loss 0.0109508, throughput 4.00123K wps
[Epoch 20 Batch 3630/4268] avg loss 0.0113574, throughput 3.99432K wps
[Epoch 20 Batch 3660/4268] avg loss 0.0108972, throughput 4.0002K wps
[Epoch 20 Batch 3690/4268] avg loss 0.0116713, throughput 3.9949K wps
[Epoch 20 Batch 3720/4268] avg loss 0.0113598, throughput 4.00008K wps
[Epoch 20 Batch 3750/4268] avg loss 0.0105433, throughput 3.99812K wps
[Epoch 20 Batch 3780/4268] avg loss 0.0108947, throughput 3.99844K wps
[Epoch 20 Batch 3810/4268] avg loss 0.0106308, throughput 3.99892K wps
[Epoch 20 Batch 3840/4268] avg loss 0.0110487, throughput 4.00237K wps
[Epoch 20 Batch 3870/4268] avg loss 0.0113103, throughput 3.99812K wps
[Epoch 20 Batch 3900/4268] avg loss 0.0102086, throughput 3.99342K wps
[Epoch 20 Batch 3930/4268] avg loss 0.0111561, throughput 3.99865K wps
[Epoch 20 Batch 3960/4268] avg loss 0.00969971, throughput 3.99265K wps
[Epoch 20 Batch 3990/4268] avg loss 0.0106233, throughput 3.99853K wps
[Epoch 20 Batch 4020/4268] avg loss 0.0111208, throughput 3.99932K wps
[Epoch 20 Batch 4050/4268] avg loss 0.0100983, throughput 3.98998K wps
[Epoch 20 Batch 4080/4268] avg loss 0.0112159, throughput 3.99708K wps
[Epoch 20 Batch 4110/4268] avg loss 0.0109772, throughput 3.99924K wps
[Epoch 20 Batch 4140/4268] avg loss 0.0114236, throughput 3.99785K wps
[Epoch 20 Batch 4170/4268] avg loss 0.0105289, throughput 3.99572K wps
[Epoch 20 Batch 4200/4268] avg loss 0.0101531, throughput 3.99908K wps
[Epoch 20 Batch 4230/4268] avg loss 0.0105889, throughput 3.9951K wps
[Epoch 20 Batch 4260/4268] avg loss 0.0103906, throughput 3.98862K wps
Begin Testing...
[Batch 30/475] elapsed 0.46 s
[Batch 60/475] elapsed 0.43 s
[Batch 90/475] elapsed 0.43 s
[Batch 120/475] elapsed 0.43 s
[Batch 150/475] elapsed 0.43 s
[Batch 180/475] elapsed 0.43 s
[Batch 210/475] elapsed 0.43 s
[Batch 240/475] elapsed 0.43 s
[Batch 270/475] elapsed 0.43 s
[Batch 300/475] elapsed 0.43 s
[Batch 330/475] elapsed 0.43 s
[Batch 360/475] elapsed 0.43 s
[Batch 390/475] elapsed 0.43 s
[Batch 420/475] elapsed 0.43 s
[Batch 450/475] elapsed 0.43 s
[Epoch 20] train avg loss 0.0104398, test acc 0.6729, test avg loss 0.904368, throughput 3.99766K wps
[Epoch 21 Batch 30/4268] avg loss 0.00959176, throughput 4.08423K wps
[Epoch 21 Batch 60/4268] avg loss 0.00980777, throughput 3.99686K wps
[Epoch 21 Batch 90/4268] avg loss 0.00978489, throughput 3.99654K wps
[Epoch 21 Batch 120/4268] avg loss 0.0102398, throughput 3.99405K wps
[Epoch 21 Batch 150/4268] avg loss 0.00904764, throughput 3.99588K wps
[Epoch 21 Batch 180/4268] avg loss 0.00959307, throughput 3.99233K wps
[Epoch 21 Batch 210/4268] avg loss 0.010165, throughput 3.99026K wps
[Epoch 21 Batch 240/4268] avg loss 0.0100412, throughput 3.99246K wps
[Epoch 21 Batch 270/4268] avg loss 0.00969765, throughput 3.99095K wps
[Epoch 21 Batch 300/4268] avg loss 0.00985291, throughput 3.99923K wps
[Epoch 21 Batch 330/4268] avg loss 0.010337, throughput 3.99401K wps
[Epoch 21 Batch 360/4268] avg loss 0.00899487, throughput 3.99735K wps
[Epoch 21 Batch 390/4268] avg loss 0.00948815, throughput 3.99654K wps
[Epoch 21 Batch 420/4268] avg loss 0.00992563, throughput 3.99556K wps
[Epoch 21 Batch 450/4268] avg loss 0.00952017, throughput 3.99989K wps
[Epoch 21 Batch 480/4268] avg loss 0.0102984, throughput 3.99501K wps
[Epoch 21 Batch 510/4268] avg loss 0.00959682, throughput 3.99834K wps
[Epoch 21 Batch 540/4268] avg loss 0.00951367, throughput 3.99553K wps
[Epoch 21 Batch 570/4268] avg loss 0.00977556, throughput 3.99372K wps
[Epoch 21 Batch 600/4268] avg loss 0.0102566, throughput 3.99436K wps
[Epoch 21 Batch 630/4268] avg loss 0.00900459, throughput 3.99405K wps
[Epoch 21 Batch 660/4268] avg loss 0.00999325, throughput 3.99776K wps
[Epoch 21 Batch 690/4268] avg loss 0.0101148, throughput 3.99198K wps
[Epoch 21 Batch 720/4268] avg loss 0.00975306, throughput 3.99787K wps
[Epoch 21 Batch 750/4268] avg loss 0.00982142, throughput 3.99524K wps
[Epoch 21 Batch 780/4268] avg loss 0.010591, throughput 3.98647K wps
[Epoch 21 Batch 810/4268] avg loss 0.00983006, throughput 3.98926K wps
[Epoch 21 Batch 840/4268] avg loss 0.0096038, throughput 3.9963K wps
[Epoch 21 Batch 870/4268] avg loss 0.00981054, throughput 4.0095K wps
[Epoch 21 Batch 900/4268] avg loss 0.0105082, throughput 3.99298K wps
[Epoch 21 Batch 930/4268] avg loss 0.0100149, throughput 3.99726K wps
[Epoch 21 Batch 960/4268] avg loss 0.00995622, throughput 3.99609K wps
[Epoch 21 Batch 990/4268] avg loss 0.0107382, throughput 3.99144K wps
[Epoch 21 Batch 1020/4268] avg loss 0.0101404, throughput 3.99641K wps
[Epoch 21 Batch 1050/4268] avg loss 0.0101117, throughput 3.99617K wps
[Epoch 21 Batch 1080/4268] avg loss 0.00963715, throughput 3.99565K wps
[Epoch 21 Batch 1110/4268] avg loss 0.0103794, throughput 3.99518K wps
[Epoch 21 Batch 1140/4268] avg loss 0.00967965, throughput 3.99467K wps
[Epoch 21 Batch 1170/4268] avg loss 0.00972982, throughput 3.99308K wps
[Epoch 21 Batch 1200/4268] avg loss 0.00975244, throughput 3.99376K wps
[Epoch 21 Batch 1230/4268] avg loss 0.010479, throughput 3.99906K wps
[Epoch 21 Batch 1260/4268] avg loss 0.0102525, throughput 4.0039K wps
[Epoch 21 Batch 1290/4268] avg loss 0.010798, throughput 3.99983K wps
[Epoch 21 Batch 1320/4268] avg loss 0.00976743, throughput 4.00175K wps
[Epoch 21 Batch 1350/4268] avg loss 0.00947593, throughput 3.99354K wps
[Epoch 21 Batch 1380/4268] avg loss 0.00937563, throughput 4.00066K wps
[Epoch 21 Batch 1410/4268] avg loss 0.00956416, throughput 3.99998K wps
[Epoch 21 Batch 1440/4268] avg loss 0.00998416, throughput 3.99923K wps
[Epoch 21 Batch 1470/4268] avg loss 0.0105394, throughput 3.99854K wps
[Epoch 21 Batch 1500/4268] avg loss 0.0102818, throughput 3.99621K wps
[Epoch 21 Batch 1530/4268] avg loss 0.0103445, throughput 3.99781K wps
[Epoch 21 Batch 1560/4268] avg loss 0.00996857, throughput 3.9984K wps
[Epoch 21 Batch 1590/4268] avg loss 0.00975902, throughput 4.00421K wps
[Epoch 21 Batch 1620/4268] avg loss 0.00955775, throughput 4.00343K wps
[Epoch 21 Batch 1650/4268] avg loss 0.0105559, throughput 4.00983K wps
[Epoch 21 Batch 1680/4268] avg loss 0.00997159, throughput 4.00936K wps
[Epoch 21 Batch 1710/4268] avg loss 0.0105734, throughput 4.00549K wps
[Epoch 21 Batch 1740/4268] avg loss 0.0106891, throughput 4.00228K wps
[Epoch 21 Batch 1770/4268] avg loss 0.0107348, throughput 3.9969K wps
[Epoch 21 Batch 1800/4268] avg loss 0.00995508, throughput 3.99821K wps
[Epoch 21 Batch 1830/4268] avg loss 0.0102742, throughput 3.99406K wps
[Epoch 21 Batch 1860/4268] avg loss 0.0105283, throughput 3.99556K wps
[Epoch 21 Batch 1890/4268] avg loss 0.00975524, throughput 3.99167K wps
[Epoch 21 Batch 1920/4268] avg loss 0.0102173, throughput 3.99327K wps
[Epoch 21 Batch 1950/4268] avg loss 0.0102819, throughput 3.99416K wps
[Epoch 21 Batch 1980/4268] avg loss 0.0102642, throughput 4.00021K wps
[Epoch 21 Batch 2010/4268] avg loss 0.0103943, throughput 3.99552K wps
[Epoch 21 Batch 2040/4268] avg loss 0.0105156, throughput 4.00711K wps
[Epoch 21 Batch 2070/4268] avg loss 0.010468, throughput 4.0049K wps
[Epoch 21 Batch 2100/4268] avg loss 0.010507, throughput 3.99789K wps
[Epoch 21 Batch 2130/4268] avg loss 0.00962254, throughput 3.99682K wps
[Epoch 21 Batch 2160/4268] avg loss 0.0104361, throughput 3.99253K wps
[Epoch 21 Batch 2190/4268] avg loss 0.0107072, throughput 3.99885K wps
[Epoch 21 Batch 2220/4268] avg loss 0.010082, throughput 3.99081K wps
[Epoch 21 Batch 2250/4268] avg loss 0.0106441, throughput 3.99217K wps
[Epoch 21 Batch 2280/4268] avg loss 0.0106084, throughput 3.99664K wps
[Epoch 21 Batch 2310/4268] avg loss 0.00970051, throughput 3.99585K wps
[Epoch 21 Batch 2340/4268] avg loss 0.0105601, throughput 3.99771K wps
[Epoch 21 Batch 2370/4268] avg loss 0.0101707, throughput 4.00402K wps
[Epoch 21 Batch 2400/4268] avg loss 0.00992155, throughput 3.99747K wps
[Epoch 21 Batch 2430/4268] avg loss 0.0108425, throughput 3.99609K wps
[Epoch 21 Batch 2460/4268] avg loss 0.00989986, throughput 3.99698K wps
[Epoch 21 Batch 2490/4268] avg loss 0.0103356, throughput 4.00287K wps
[Epoch 21 Batch 2520/4268] avg loss 0.0104385, throughput 3.99293K wps
[Epoch 21 Batch 2550/4268] avg loss 0.0106565, throughput 3.99556K wps
[Epoch 21 Batch 2580/4268] avg loss 0.0105721, throughput 3.99201K wps
[Epoch 21 Batch 2610/4268] avg loss 0.0106922, throughput 3.99859K wps
[Epoch 21 Batch 2640/4268] avg loss 0.0104167, throughput 3.99421K wps
[Epoch 21 Batch 2670/4268] avg loss 0.0103363, throughput 3.99932K wps
[Epoch 21 Batch 2700/4268] avg loss 0.0100158, throughput 3.99575K wps
[Epoch 21 Batch 2730/4268] avg loss 0.0109176, throughput 4.00034K wps
[Epoch 21 Batch 2760/4268] avg loss 0.0105537, throughput 4.00314K wps
[Epoch 21 Batch 2790/4268] avg loss 0.010551, throughput 3.99853K wps
[Epoch 21 Batch 2820/4268] avg loss 0.0103683, throughput 3.99413K wps
[Epoch 21 Batch 2850/4268] avg loss 0.00988116, throughput 3.99795K wps
[Epoch 21 Batch 2880/4268] avg loss 0.0102019, throughput 3.9967K wps
[Epoch 21 Batch 2910/4268] avg loss 0.0102906, throughput 4.00034K wps
[Epoch 21 Batch 2940/4268] avg loss 0.0103466, throughput 3.99482K wps
[Epoch 21 Batch 2970/4268] avg loss 0.0107633, throughput 3.99638K wps
[Epoch 21 Batch 3000/4268] avg loss 0.0106074, throughput 3.99517K wps
[Epoch 21 Batch 3030/4268] avg loss 0.0109559, throughput 3.99423K wps
[Epoch 21 Batch 3060/4268] avg loss 0.0104456, throughput 3.99832K wps
[Epoch 21 Batch 3090/4268] avg loss 0.010417, throughput 3.99828K wps
[Epoch 21 Batch 3120/4268] avg loss 0.010245, throughput 3.99858K wps
[Epoch 21 Batch 3150/4268] avg loss 0.0105402, throughput 4.00238K wps
[Epoch 21 Batch 3180/4268] avg loss 0.010062, throughput 4.00154K wps
[Epoch 21 Batch 3210/4268] avg loss 0.0108335, throughput 4.00141K wps
[Epoch 21 Batch 3240/4268] avg loss 0.0107849, throughput 3.99463K wps
[Epoch 21 Batch 3270/4268] avg loss 0.0107443, throughput 3.99515K wps
[Epoch 21 Batch 3300/4268] avg loss 0.010679, throughput 3.99976K wps
[Epoch 21 Batch 3330/4268] avg loss 0.0109892, throughput 3.99563K wps
[Epoch 21 Batch 3360/4268] avg loss 0.0108874, throughput 4.00014K wps
[Epoch 21 Batch 3390/4268] avg loss 0.010914, throughput 4.00278K wps
[Epoch 21 Batch 3420/4268] avg loss 0.00963431, throughput 4.00619K wps
[Epoch 21 Batch 3450/4268] avg loss 0.0106267, throughput 3.99729K wps
[Epoch 21 Batch 3480/4268] avg loss 0.0106419, throughput 3.99704K wps
[Epoch 21 Batch 3510/4268] avg loss 0.0100905, throughput 3.99189K wps
[Epoch 21 Batch 3540/4268] avg loss 0.0108484, throughput 3.99784K wps
[Epoch 21 Batch 3570/4268] avg loss 0.0108063, throughput 4.00236K wps
[Epoch 21 Batch 3600/4268] avg loss 0.00977089, throughput 3.99696K wps
[Epoch 21 Batch 3630/4268] avg loss 0.0109315, throughput 3.99303K wps
[Epoch 21 Batch 3660/4268] avg loss 0.0109252, throughput 3.99387K wps
[Epoch 21 Batch 3690/4268] avg loss 0.0101539, throughput 4.00053K wps
[Epoch 21 Batch 3720/4268] avg loss 0.0104446, throughput 3.99446K wps
[Epoch 21 Batch 3750/4268] avg loss 0.0103243, throughput 3.99586K wps
[Epoch 21 Batch 3780/4268] avg loss 0.0105917, throughput 3.99398K wps
[Epoch 21 Batch 3810/4268] avg loss 0.0111552, throughput 3.99122K wps
[Epoch 21 Batch 3840/4268] avg loss 0.010293, throughput 3.99677K wps
[Epoch 21 Batch 3870/4268] avg loss 0.0106089, throughput 3.99561K wps
[Epoch 21 Batch 3900/4268] avg loss 0.0104742, throughput 3.99551K wps
[Epoch 21 Batch 3930/4268] avg loss 0.0103396, throughput 4.00076K wps
[Epoch 21 Batch 3960/4268] avg loss 0.0108378, throughput 3.99825K wps
[Epoch 21 Batch 3990/4268] avg loss 0.0107, throughput 3.99623K wps
[Epoch 21 Batch 4020/4268] avg loss 0.0110251, throughput 4.00008K wps
[Epoch 21 Batch 4050/4268] avg loss 0.0108152, throughput 3.98723K wps
[Epoch 21 Batch 4080/4268] avg loss 0.0109219, throughput 3.99912K wps
[Epoch 21 Batch 4110/4268] avg loss 0.0102457, throughput 3.9968K wps
[Epoch 21 Batch 4140/4268] avg loss 0.0109575, throughput 4.00124K wps
[Epoch 21 Batch 4170/4268] avg loss 0.010358, throughput 3.99892K wps
[Epoch 21 Batch 4200/4268] avg loss 0.0106706, throughput 3.99879K wps
[Epoch 21 Batch 4230/4268] avg loss 0.0112384, throughput 3.9982K wps
[Epoch 21 Batch 4260/4268] avg loss 0.0113304, throughput 3.99408K wps
Begin Testing...
[Batch 30/475] elapsed 0.46 s
[Batch 60/475] elapsed 0.43 s
[Batch 90/475] elapsed 0.43 s
[Batch 120/475] elapsed 0.43 s
[Batch 150/475] elapsed 0.43 s
[Batch 180/475] elapsed 0.43 s
[Batch 210/475] elapsed 0.43 s
[Batch 240/475] elapsed 0.43 s
[Batch 270/475] elapsed 0.44 s
[Batch 300/475] elapsed 0.44 s
[Batch 330/475] elapsed 0.44 s
[Batch 360/475] elapsed 0.44 s
[Batch 390/475] elapsed 0.44 s
[Batch 420/475] elapsed 0.43 s
[Batch 450/475] elapsed 0.43 s
[Epoch 21] train avg loss 0.0102704, test acc 0.6719, test avg loss 0.912217, throughput 3.99773K wps
[Epoch 22 Batch 30/4268] avg loss 0.00952405, throughput 4.08546K wps
[Epoch 22 Batch 60/4268] avg loss 0.00906249, throughput 3.99876K wps
[Epoch 22 Batch 90/4268] avg loss 0.010072, throughput 3.99646K wps
[Epoch 22 Batch 120/4268] avg loss 0.0097057, throughput 3.99345K wps
[Epoch 22 Batch 150/4268] avg loss 0.00995154, throughput 3.99639K wps
[Epoch 22 Batch 180/4268] avg loss 0.00959733, throughput 3.99819K wps
[Epoch 22 Batch 210/4268] avg loss 0.00927846, throughput 3.99917K wps
[Epoch 22 Batch 240/4268] avg loss 0.00944525, throughput 4.00027K wps
[Epoch 22 Batch 270/4268] avg loss 0.0102306, throughput 3.99989K wps
[Epoch 22 Batch 300/4268] avg loss 0.00951141, throughput 3.99849K wps
[Epoch 22 Batch 330/4268] avg loss 0.00925704, throughput 3.99776K wps
[Epoch 22 Batch 360/4268] avg loss 0.00974872, throughput 3.99231K wps
[Epoch 22 Batch 390/4268] avg loss 0.0097238, throughput 3.99664K wps
[Epoch 22 Batch 420/4268] avg loss 0.0102021, throughput 3.9914K wps
[Epoch 22 Batch 450/4268] avg loss 0.00986664, throughput 3.99715K wps
[Epoch 22 Batch 480/4268] avg loss 0.0100899, throughput 3.99647K wps
[Epoch 22 Batch 510/4268] avg loss 0.0102416, throughput 3.99949K wps
[Epoch 22 Batch 540/4268] avg loss 0.0100271, throughput 3.99895K wps
[Epoch 22 Batch 570/4268] avg loss 0.0093408, throughput 3.99945K wps
[Epoch 22 Batch 600/4268] avg loss 0.00942039, throughput 3.99609K wps
[Epoch 22 Batch 630/4268] avg loss 0.00963603, throughput 3.99671K wps
[Epoch 22 Batch 660/4268] avg loss 0.0100003, throughput 3.99778K wps
[Epoch 22 Batch 690/4268] avg loss 0.00988728, throughput 3.99645K wps
[Epoch 22 Batch 720/4268] avg loss 0.00947654, throughput 4.00046K wps
[Epoch 22 Batch 750/4268] avg loss 0.00954017, throughput 3.9978K wps
[Epoch 22 Batch 780/4268] avg loss 0.0095639, throughput 4.0012K wps
[Epoch 22 Batch 810/4268] avg loss 0.0101597, throughput 4.00076K wps
[Epoch 22 Batch 840/4268] avg loss 0.0091881, throughput 4.00028K wps
[Epoch 22 Batch 870/4268] avg loss 0.00949144, throughput 3.99722K wps
[Epoch 22 Batch 900/4268] avg loss 0.00968243, throughput 3.9958K wps
[Epoch 22 Batch 930/4268] avg loss 0.00975812, throughput 3.99808K wps
[Epoch 22 Batch 960/4268] avg loss 0.0104027, throughput 3.99606K wps
[Epoch 22 Batch 990/4268] avg loss 0.00939657, throughput 3.99523K wps
[Epoch 22 Batch 1020/4268] avg loss 0.0101061, throughput 3.99277K wps
[Epoch 22 Batch 1050/4268] avg loss 0.00973865, throughput 3.99213K wps
[Epoch 22 Batch 1080/4268] avg loss 0.0101063, throughput 3.99217K wps
[Epoch 22 Batch 1110/4268] avg loss 0.00983868, throughput 3.99643K wps
[Epoch 22 Batch 1140/4268] avg loss 0.00993623, throughput 3.99582K wps
[Epoch 22 Batch 1170/4268] avg loss 0.00955736, throughput 3.99851K wps
[Epoch 22 Batch 1200/4268] avg loss 0.0101857, throughput 4.00186K wps
[Epoch 22 Batch 1230/4268] avg loss 0.0099777, throughput 4.00594K wps
[Epoch 22 Batch 1260/4268] avg loss 0.0101651, throughput 4.00431K wps
[Epoch 22 Batch 1290/4268] avg loss 0.00914561, throughput 3.99776K wps
[Epoch 22 Batch 1320/4268] avg loss 0.0103615, throughput 3.99297K wps
[Epoch 22 Batch 1350/4268] avg loss 0.00972947, throughput 3.99495K wps
[Epoch 22 Batch 1380/4268] avg loss 0.0101754, throughput 4.00135K wps
[Epoch 22 Batch 1410/4268] avg loss 0.010264, throughput 3.99851K wps
[Epoch 22 Batch 1440/4268] avg loss 0.00925595, throughput 3.99744K wps
[Epoch 22 Batch 1470/4268] avg loss 0.00955636, throughput 3.99978K wps
[Epoch 22 Batch 1500/4268] avg loss 0.009684, throughput 3.99998K wps
[Epoch 22 Batch 1530/4268] avg loss 0.0100491, throughput 4.00178K wps
[Epoch 22 Batch 1560/4268] avg loss 0.00985183, throughput 3.99858K wps
[Epoch 22 Batch 1590/4268] avg loss 0.00962441, throughput 4.00137K wps
[Epoch 22 Batch 1620/4268] avg loss 0.0106516, throughput 3.99752K wps
[Epoch 22 Batch 1650/4268] avg loss 0.00988733, throughput 3.99204K wps
[Epoch 22 Batch 1680/4268] avg loss 0.0101559, throughput 4.00249K wps
[Epoch 22 Batch 1710/4268] avg loss 0.0100107, throughput 3.99567K wps
[Epoch 22 Batch 1740/4268] avg loss 0.00988278, throughput 3.99979K wps
[Epoch 22 Batch 1770/4268] avg loss 0.00972871, throughput 3.99599K wps
[Epoch 22 Batch 1800/4268] avg loss 0.0103217, throughput 3.99797K wps
[Epoch 22 Batch 1830/4268] avg loss 0.0099384, throughput 4.00155K wps
[Epoch 22 Batch 1860/4268] avg loss 0.0102997, throughput 3.99575K wps
[Epoch 22 Batch 1890/4268] avg loss 0.00988681, throughput 3.99726K wps
[Epoch 22 Batch 1920/4268] avg loss 0.0102818, throughput 3.9955K wps
[Epoch 22 Batch 1950/4268] avg loss 0.00971997, throughput 3.99587K wps
[Epoch 22 Batch 1980/4268] avg loss 0.0103242, throughput 3.99741K wps
[Epoch 22 Batch 2010/4268] avg loss 0.0105897, throughput 3.9988K wps
[Epoch 22 Batch 2040/4268] avg loss 0.0103731, throughput 3.99601K wps
[Epoch 22 Batch 2070/4268] avg loss 0.0100243, throughput 4.00181K wps
[Epoch 22 Batch 2100/4268] avg loss 0.0097257, throughput 4.00779K wps
[Epoch 22 Batch 2130/4268] avg loss 0.0100095, throughput 4.01409K wps
[Epoch 22 Batch 2160/4268] avg loss 0.00990985, throughput 4.00468K wps
[Epoch 22 Batch 2190/4268] avg loss 0.0100761, throughput 3.99809K wps
[Epoch 22 Batch 2220/4268] avg loss 0.00988702, throughput 3.99319K wps
[Epoch 22 Batch 2250/4268] avg loss 0.0101693, throughput 3.99701K wps
[Epoch 22 Batch 2280/4268] avg loss 0.0106198, throughput 3.99445K wps
[Epoch 22 Batch 2310/4268] avg loss 0.0101926, throughput 3.99171K wps
[Epoch 22 Batch 2340/4268] avg loss 0.00947609, throughput 3.99699K wps
[Epoch 22 Batch 2370/4268] avg loss 0.0100481, throughput 3.99609K wps
[Epoch 22 Batch 2400/4268] avg loss 0.010117, throughput 3.99506K wps
[Epoch 22 Batch 2430/4268] avg loss 0.0101358, throughput 4.00024K wps
[Epoch 22 Batch 2460/4268] avg loss 0.0101109, throughput 3.99804K wps
[Epoch 22 Batch 2490/4268] avg loss 0.00986431, throughput 3.99623K wps
[Epoch 22 Batch 2520/4268] avg loss 0.0109689, throughput 3.99753K wps
[Epoch 22 Batch 2550/4268] avg loss 0.0104027, throughput 3.99451K wps
[Epoch 22 Batch 2580/4268] avg loss 0.0104861, throughput 3.99724K wps
[Epoch 22 Batch 2610/4268] avg loss 0.0103674, throughput 3.99503K wps
[Epoch 22 Batch 2640/4268] avg loss 0.0100832, throughput 4.00032K wps
[Epoch 22 Batch 2670/4268] avg loss 0.0100261, throughput 3.99828K wps
[Epoch 22 Batch 2700/4268] avg loss 0.0106146, throughput 3.99751K wps
[Epoch 22 Batch 2730/4268] avg loss 0.0108821, throughput 3.99316K wps
[Epoch 22 Batch 2760/4268] avg loss 0.0107561, throughput 3.99387K wps
[Epoch 22 Batch 2790/4268] avg loss 0.0102415, throughput 3.99752K wps
[Epoch 22 Batch 2820/4268] avg loss 0.0104677, throughput 3.99713K wps
[Epoch 22 Batch 2850/4268] avg loss 0.010135, throughput 3.99604K wps
[Epoch 22 Batch 2880/4268] avg loss 0.00991712, throughput 3.99564K wps
[Epoch 22 Batch 2910/4268] avg loss 0.0106042, throughput 3.99725K wps
[Epoch 22 Batch 2940/4268] avg loss 0.0106999, throughput 3.99687K wps
[Epoch 22 Batch 2970/4268] avg loss 0.010139, throughput 3.99812K wps
[Epoch 22 Batch 3000/4268] avg loss 0.0102267, throughput 3.99893K wps
[Epoch 22 Batch 3030/4268] avg loss 0.0100284, throughput 3.99735K wps
[Epoch 22 Batch 3060/4268] avg loss 0.010378, throughput 3.99518K wps
[Epoch 22 Batch 3090/4268] avg loss 0.0101081, throughput 3.99403K wps
[Epoch 22 Batch 3120/4268] avg loss 0.0105129, throughput 3.99663K wps
[Epoch 22 Batch 3150/4268] avg loss 0.0100744, throughput 3.99928K wps
[Epoch 22 Batch 3180/4268] avg loss 0.010336, throughput 3.99916K wps
[Epoch 22 Batch 3210/4268] avg loss 0.0107089, throughput 3.9956K wps
[Epoch 22 Batch 3240/4268] avg loss 0.0106923, throughput 3.99553K wps
[Epoch 22 Batch 3270/4268] avg loss 0.0106992, throughput 3.99723K wps
[Epoch 22 Batch 3300/4268] avg loss 0.0103566, throughput 3.99398K wps
[Epoch 22 Batch 3330/4268] avg loss 0.0102848, throughput 3.99771K wps
[Epoch 22 Batch 3360/4268] avg loss 0.0108857, throughput 3.9977K wps
[Epoch 22 Batch 3390/4268] avg loss 0.0106611, throughput 3.99645K wps
[Epoch 22 Batch 3420/4268] avg loss 0.011046, throughput 3.99874K wps
[Epoch 22 Batch 3450/4268] avg loss 0.0102035, throughput 3.99725K wps
[Epoch 22 Batch 3480/4268] avg loss 0.0108949, throughput 3.99458K wps
[Epoch 22 Batch 3510/4268] avg loss 0.0100698, throughput 3.99724K wps
[Epoch 22 Batch 3540/4268] avg loss 0.0103532, throughput 4.00125K wps
[Epoch 22 Batch 3570/4268] avg loss 0.0103826, throughput 3.98507K wps
[Epoch 22 Batch 3600/4268] avg loss 0.0104767, throughput 3.99874K wps
[Epoch 22 Batch 3630/4268] avg loss 0.0100377, throughput 4.00468K wps
[Epoch 22 Batch 3660/4268] avg loss 0.010756, throughput 4.00231K wps
[Epoch 22 Batch 3690/4268] avg loss 0.0101616, throughput 4.00173K wps
[Epoch 22 Batch 3720/4268] avg loss 0.010086, throughput 3.9993K wps
[Epoch 22 Batch 3750/4268] avg loss 0.0109462, throughput 3.99173K wps
[Epoch 22 Batch 3780/4268] avg loss 0.00992453, throughput 4K wps
[Epoch 22 Batch 3810/4268] avg loss 0.0100898, throughput 4.00021K wps
[Epoch 22 Batch 3840/4268] avg loss 0.0100467, throughput 3.99637K wps
[Epoch 22 Batch 3870/4268] avg loss 0.0106177, throughput 3.99089K wps
[Epoch 22 Batch 3900/4268] avg loss 0.0103884, throughput 3.99373K wps
[Epoch 22 Batch 3930/4268] avg loss 0.0107607, throughput 3.99183K wps
[Epoch 22 Batch 3960/4268] avg loss 0.0111489, throughput 3.99315K wps
[Epoch 22 Batch 3990/4268] avg loss 0.010237, throughput 3.9951K wps
[Epoch 22 Batch 4020/4268] avg loss 0.00989262, throughput 3.99119K wps
[Epoch 22 Batch 4050/4268] avg loss 0.0103938, throughput 3.99147K wps
[Epoch 22 Batch 4080/4268] avg loss 0.0104139, throughput 3.99265K wps
[Epoch 22 Batch 4110/4268] avg loss 0.0104825, throughput 3.99244K wps
[Epoch 22 Batch 4140/4268] avg loss 0.0106035, throughput 3.99131K wps
[Epoch 22 Batch 4170/4268] avg loss 0.0107671, throughput 3.9936K wps
[Epoch 22 Batch 4200/4268] avg loss 0.0102432, throughput 3.99089K wps
[Epoch 22 Batch 4230/4268] avg loss 0.0106055, throughput 3.99464K wps
[Epoch 22 Batch 4260/4268] avg loss 0.0109274, throughput 4.00097K wps
Begin Testing...
[Batch 30/475] elapsed 0.47 s
[Batch 60/475] elapsed 0.43 s
[Batch 90/475] elapsed 0.43 s
[Batch 120/475] elapsed 0.43 s
[Batch 150/475] elapsed 0.43 s
[Batch 180/475] elapsed 0.43 s
[Batch 210/475] elapsed 0.43 s
[Batch 240/475] elapsed 0.43 s
[Batch 270/475] elapsed 0.43 s
[Batch 300/475] elapsed 0.43 s
[Batch 330/475] elapsed 0.43 s
[Batch 360/475] elapsed 0.43 s
[Batch 390/475] elapsed 0.43 s
[Batch 420/475] elapsed 0.43 s
[Batch 450/475] elapsed 0.43 s
[Epoch 22] train avg loss 0.0101147, test acc 0.6705, test avg loss 0.932569, throughput 3.99778K wps
[Epoch 23 Batch 30/4268] avg loss 0.009178, throughput 4.08876K wps
[Epoch 23 Batch 60/4268] avg loss 0.00905554, throughput 3.99992K wps
[Epoch 23 Batch 90/4268] avg loss 0.00934918, throughput 3.99777K wps
[Epoch 23 Batch 120/4268] avg loss 0.00963313, throughput 3.99007K wps
[Epoch 23 Batch 150/4268] avg loss 0.00933898, throughput 4.00002K wps
[Epoch 23 Batch 180/4268] avg loss 0.00958459, throughput 3.99651K wps
[Epoch 23 Batch 210/4268] avg loss 0.00985646, throughput 4.00005K wps
[Epoch 23 Batch 240/4268] avg loss 0.00983201, throughput 3.99595K wps
[Epoch 23 Batch 270/4268] avg loss 0.00976016, throughput 3.99426K wps
[Epoch 23 Batch 300/4268] avg loss 0.00993249, throughput 3.98678K wps
[Epoch 23 Batch 330/4268] avg loss 0.00954464, throughput 3.98879K wps
[Epoch 23 Batch 360/4268] avg loss 0.00953103, throughput 3.99232K wps
[Epoch 23 Batch 390/4268] avg loss 0.00970427, throughput 3.99697K wps
[Epoch 23 Batch 420/4268] avg loss 0.00927328, throughput 3.99659K wps
[Epoch 23 Batch 450/4268] avg loss 0.00955349, throughput 3.99982K wps
[Epoch 23 Batch 480/4268] avg loss 0.00942085, throughput 3.99547K wps
[Epoch 23 Batch 510/4268] avg loss 0.00944101, throughput 3.99491K wps
[Epoch 23 Batch 540/4268] avg loss 0.00991914, throughput 3.99832K wps
[Epoch 23 Batch 570/4268] avg loss 0.00966397, throughput 3.99738K wps
[Epoch 23 Batch 600/4268] avg loss 0.00988489, throughput 3.99857K wps
[Epoch 23 Batch 630/4268] avg loss 0.00953621, throughput 3.99439K wps
[Epoch 23 Batch 660/4268] avg loss 0.00969066, throughput 3.99116K wps
[Epoch 23 Batch 690/4268] avg loss 0.00960312, throughput 3.99907K wps
[Epoch 23 Batch 720/4268] avg loss 0.00942575, throughput 3.99843K wps
[Epoch 23 Batch 750/4268] avg loss 0.00959069, throughput 3.999K wps
[Epoch 23 Batch 780/4268] avg loss 0.00999792, throughput 3.99885K wps
[Epoch 23 Batch 810/4268] avg loss 0.00951569, throughput 3.9981K wps
[Epoch 23 Batch 840/4268] avg loss 0.0100304, throughput 3.99511K wps
[Epoch 23 Batch 870/4268] avg loss 0.00916041, throughput 3.99779K wps
[Epoch 23 Batch 900/4268] avg loss 0.00949513, throughput 3.99871K wps
[Epoch 23 Batch 930/4268] avg loss 0.0103965, throughput 3.99203K wps
[Epoch 23 Batch 960/4268] avg loss 0.00962303, throughput 3.99051K wps
[Epoch 23 Batch 990/4268] avg loss 0.00938161, throughput 3.99524K wps
[Epoch 23 Batch 1020/4268] avg loss 0.00962885, throughput 3.99463K wps
[Epoch 23 Batch 1050/4268] avg loss 0.00949927, throughput 3.98914K wps
[Epoch 23 Batch 1080/4268] avg loss 0.00943692, throughput 3.99479K wps
[Epoch 23 Batch 1110/4268] avg loss 0.00973224, throughput 3.99697K wps
[Epoch 23 Batch 1140/4268] avg loss 0.00939114, throughput 3.99646K wps
[Epoch 23 Batch 1170/4268] avg loss 0.0101832, throughput 3.99627K wps
[Epoch 23 Batch 1200/4268] avg loss 0.00997444, throughput 3.99589K wps
[Epoch 23 Batch 1230/4268] avg loss 0.00993506, throughput 3.99797K wps
[Epoch 23 Batch 1260/4268] avg loss 0.00921116, throughput 3.99472K wps
[Epoch 23 Batch 1290/4268] avg loss 0.00967235, throughput 3.99915K wps
[Epoch 23 Batch 1320/4268] avg loss 0.0101424, throughput 3.99884K wps
[Epoch 23 Batch 1350/4268] avg loss 0.00906686, throughput 3.99634K wps
[Epoch 23 Batch 1380/4268] avg loss 0.009683, throughput 3.99666K wps
[Epoch 23 Batch 1410/4268] avg loss 0.00954434, throughput 3.99646K wps
[Epoch 23 Batch 1440/4268] avg loss 0.00975632, throughput 3.99838K wps
[Epoch 23 Batch 1470/4268] avg loss 0.0102213, throughput 3.9974K wps
[Epoch 23 Batch 1500/4268] avg loss 0.0096985, throughput 3.9934K wps
[Epoch 23 Batch 1530/4268] avg loss 0.00962234, throughput 3.99973K wps
[Epoch 23 Batch 1560/4268] avg loss 0.0090966, throughput 3.99687K wps
[Epoch 23 Batch 1590/4268] avg loss 0.00949121, throughput 3.99587K wps
[Epoch 23 Batch 1620/4268] avg loss 0.00910471, throughput 3.99696K wps
[Epoch 23 Batch 1650/4268] avg loss 0.00972514, throughput 3.9948K wps
[Epoch 23 Batch 1680/4268] avg loss 0.010235, throughput 3.99246K wps
[Epoch 23 Batch 1710/4268] avg loss 0.00958263, throughput 3.9964K wps
[Epoch 23 Batch 1740/4268] avg loss 0.00951511, throughput 3.99651K wps
[Epoch 23 Batch 1770/4268] avg loss 0.00989743, throughput 3.99589K wps
[Epoch 23 Batch 1800/4268] avg loss 0.00907408, throughput 3.99487K wps
[Epoch 23 Batch 1830/4268] avg loss 0.0100373, throughput 4.0029K wps
[Epoch 23 Batch 1860/4268] avg loss 0.00971231, throughput 3.99973K wps
[Epoch 23 Batch 1890/4268] avg loss 0.0094943, throughput 4.00249K wps
[Epoch 23 Batch 1920/4268] avg loss 0.0102056, throughput 3.99961K wps
[Epoch 23 Batch 1950/4268] avg loss 0.0101981, throughput 4.00625K wps
[Epoch 23 Batch 1980/4268] avg loss 0.00988006, throughput 4.00428K wps
[Epoch 23 Batch 2010/4268] avg loss 0.00952873, throughput 4.0027K wps
[Epoch 23 Batch 2040/4268] avg loss 0.00953691, throughput 3.99835K wps
[Epoch 23 Batch 2070/4268] avg loss 0.00958655, throughput 3.99659K wps
[Epoch 23 Batch 2100/4268] avg loss 0.0102782, throughput 3.99836K wps
[Epoch 23 Batch 2130/4268] avg loss 0.010197, throughput 3.9996K wps
[Epoch 23 Batch 2160/4268] avg loss 0.0104487, throughput 3.99544K wps
[Epoch 23 Batch 2190/4268] avg loss 0.00964658, throughput 3.99702K wps
[Epoch 23 Batch 2220/4268] avg loss 0.0100499, throughput 3.99594K wps
[Epoch 23 Batch 2250/4268] avg loss 0.00928699, throughput 4.00017K wps
[Epoch 23 Batch 2280/4268] avg loss 0.010037, throughput 3.99568K wps
[Epoch 23 Batch 2310/4268] avg loss 0.00965196, throughput 3.99535K wps
[Epoch 23 Batch 2340/4268] avg loss 0.0104221, throughput 3.99236K wps
[Epoch 23 Batch 2370/4268] avg loss 0.0098923, throughput 3.99672K wps
[Epoch 23 Batch 2400/4268] avg loss 0.0107184, throughput 3.99693K wps
[Epoch 23 Batch 2430/4268] avg loss 0.00954677, throughput 3.99345K wps
[Epoch 23 Batch 2460/4268] avg loss 0.00924457, throughput 4.00005K wps
[Epoch 23 Batch 2490/4268] avg loss 0.0101753, throughput 3.99515K wps
[Epoch 23 Batch 2520/4268] avg loss 0.0101606, throughput 3.989K wps
[Epoch 23 Batch 2550/4268] avg loss 0.00973666, throughput 3.99509K wps
[Epoch 23 Batch 2580/4268] avg loss 0.00944357, throughput 3.99565K wps
[Epoch 23 Batch 2610/4268] avg loss 0.00946987, throughput 3.99895K wps
[Epoch 23 Batch 2640/4268] avg loss 0.0106636, throughput 3.99611K wps
[Epoch 23 Batch 2670/4268] avg loss 0.0100202, throughput 3.99856K wps
[Epoch 23 Batch 2700/4268] avg loss 0.00994813, throughput 3.99703K wps
[Epoch 23 Batch 2730/4268] avg loss 0.00921728, throughput 3.99702K wps
[Epoch 23 Batch 2760/4268] avg loss 0.00989351, throughput 3.99431K wps
[Epoch 23 Batch 2790/4268] avg loss 0.0108966, throughput 3.9993K wps
[Epoch 23 Batch 2820/4268] avg loss 0.0101755, throughput 3.99445K wps
[Epoch 23 Batch 2850/4268] avg loss 0.0104266, throughput 3.99986K wps
[Epoch 23 Batch 2880/4268] avg loss 0.0108362, throughput 3.99249K wps
[Epoch 23 Batch 2910/4268] avg loss 0.0105091, throughput 3.99489K wps
[Epoch 23 Batch 2940/4268] avg loss 0.0107771, throughput 3.99378K wps
[Epoch 23 Batch 2970/4268] avg loss 0.0105285, throughput 3.99602K wps
[Epoch 23 Batch 3000/4268] avg loss 0.0101228, throughput 4.00204K wps
[Epoch 23 Batch 3030/4268] avg loss 0.0100733, throughput 3.99841K wps
[Epoch 23 Batch 3060/4268] avg loss 0.0105505, throughput 3.99622K wps
[Epoch 23 Batch 3090/4268] avg loss 0.00998079, throughput 3.99519K wps
[Epoch 23 Batch 3120/4268] avg loss 0.00930244, throughput 3.99831K wps
[Epoch 23 Batch 3150/4268] avg loss 0.00993765, throughput 3.99219K wps
[Epoch 23 Batch 3180/4268] avg loss 0.0101744, throughput 3.99948K wps
[Epoch 23 Batch 3210/4268] avg loss 0.0106872, throughput 3.99834K wps
[Epoch 23 Batch 3240/4268] avg loss 0.0103646, throughput 3.99311K wps
[Epoch 23 Batch 3270/4268] avg loss 0.0105054, throughput 3.99474K wps
[Epoch 23 Batch 3300/4268] avg loss 0.0105269, throughput 4.00014K wps
[Epoch 23 Batch 3330/4268] avg loss 0.00985222, throughput 3.99881K wps
[Epoch 23 Batch 3360/4268] avg loss 0.0103658, throughput 3.99766K wps
[Epoch 23 Batch 3390/4268] avg loss 0.0102274, throughput 3.9952K wps
[Epoch 23 Batch 3420/4268] avg loss 0.0113106, throughput 3.99578K wps
[Epoch 23 Batch 3450/4268] avg loss 0.0100566, throughput 3.99664K wps
[Epoch 23 Batch 3480/4268] avg loss 0.0101745, throughput 3.99612K wps
[Epoch 23 Batch 3510/4268] avg loss 0.0103926, throughput 3.99502K wps
[Epoch 23 Batch 3540/4268] avg loss 0.0103742, throughput 3.99706K wps
[Epoch 23 Batch 3570/4268] avg loss 0.0102638, throughput 3.99981K wps
[Epoch 23 Batch 3600/4268] avg loss 0.0107896, throughput 3.99854K wps
[Epoch 23 Batch 3630/4268] avg loss 0.00950143, throughput 3.99889K wps
[Epoch 23 Batch 3660/4268] avg loss 0.0101983, throughput 3.99487K wps
[Epoch 23 Batch 3690/4268] avg loss 0.0104309, throughput 3.999K wps
[Epoch 23 Batch 3720/4268] avg loss 0.010673, throughput 3.99491K wps
[Epoch 23 Batch 3750/4268] avg loss 0.0107022, throughput 3.99413K wps
[Epoch 23 Batch 3780/4268] avg loss 0.00977305, throughput 3.99687K wps
[Epoch 23 Batch 3810/4268] avg loss 0.0101799, throughput 4.00112K wps
[Epoch 23 Batch 3840/4268] avg loss 0.0104483, throughput 3.99864K wps
[Epoch 23 Batch 3870/4268] avg loss 0.010345, throughput 4.00006K wps
[Epoch 23 Batch 3900/4268] avg loss 0.00994127, throughput 3.99269K wps
[Epoch 23 Batch 3930/4268] avg loss 0.00981218, throughput 3.99038K wps
[Epoch 23 Batch 3960/4268] avg loss 0.0108551, throughput 3.99361K wps
[Epoch 23 Batch 3990/4268] avg loss 0.0102707, throughput 3.99675K wps
[Epoch 23 Batch 4020/4268] avg loss 0.00993664, throughput 3.99602K wps
[Epoch 23 Batch 4050/4268] avg loss 0.0107014, throughput 3.99535K wps
[Epoch 23 Batch 4080/4268] avg loss 0.0103924, throughput 3.99261K wps
[Epoch 23 Batch 4110/4268] avg loss 0.0100195, throughput 3.99765K wps
[Epoch 23 Batch 4140/4268] avg loss 0.0101109, throughput 3.99356K wps
[Epoch 23 Batch 4170/4268] avg loss 0.010811, throughput 3.99401K wps
[Epoch 23 Batch 4200/4268] avg loss 0.0102502, throughput 3.99499K wps
[Epoch 23 Batch 4230/4268] avg loss 0.0109803, throughput 3.99937K wps
[Epoch 23 Batch 4260/4268] avg loss 0.0104086, throughput 3.99665K wps
Begin Testing...
[Batch 30/475] elapsed 0.46 s
[Batch 60/475] elapsed 0.43 s
[Batch 90/475] elapsed 0.43 s
[Batch 120/475] elapsed 0.43 s
[Batch 150/475] elapsed 0.43 s
[Batch 180/475] elapsed 0.43 s
[Batch 210/475] elapsed 0.43 s
[Batch 240/475] elapsed 0.43 s
[Batch 270/475] elapsed 0.43 s
[Batch 300/475] elapsed 0.43 s
[Batch 330/475] elapsed 0.43 s
[Batch 360/475] elapsed 0.43 s
[Batch 390/475] elapsed 0.43 s
[Batch 420/475] elapsed 0.43 s
[Batch 450/475] elapsed 0.43 s
[Epoch 23] train avg loss 0.00992956, test acc 0.6707, test avg loss 0.939868, throughput 3.99714K wps
[Epoch 24 Batch 30/4268] avg loss 0.00898108, throughput 4.0826K wps
[Epoch 24 Batch 60/4268] avg loss 0.00934895, throughput 4.00039K wps
[Epoch 24 Batch 90/4268] avg loss 0.00871656, throughput 3.99768K wps
[Epoch 24 Batch 120/4268] avg loss 0.00893639, throughput 3.99919K wps
[Epoch 24 Batch 150/4268] avg loss 0.00880395, throughput 3.99629K wps
[Epoch 24 Batch 180/4268] avg loss 0.00933066, throughput 3.9941K wps
[Epoch 24 Batch 210/4268] avg loss 0.0091019, throughput 3.99609K wps
[Epoch 24 Batch 240/4268] avg loss 0.00897489, throughput 3.99416K wps
[Epoch 24 Batch 270/4268] avg loss 0.00893876, throughput 3.99424K wps
[Epoch 24 Batch 300/4268] avg loss 0.00856896, throughput 3.99365K wps
[Epoch 24 Batch 330/4268] avg loss 0.00952288, throughput 3.99149K wps
[Epoch 24 Batch 360/4268] avg loss 0.00940716, throughput 3.99489K wps
[Epoch 24 Batch 390/4268] avg loss 0.00954928, throughput 3.99222K wps
[Epoch 24 Batch 420/4268] avg loss 0.00934724, throughput 3.98991K wps
[Epoch 24 Batch 450/4268] avg loss 0.00938663, throughput 3.99033K wps
[Epoch 24 Batch 480/4268] avg loss 0.00887264, throughput 3.98874K wps
[Epoch 24 Batch 510/4268] avg loss 0.00986798, throughput 3.99866K wps
[Epoch 24 Batch 540/4268] avg loss 0.00964375, throughput 3.99039K wps
[Epoch 24 Batch 570/4268] avg loss 0.00966606, throughput 4.00163K wps
[Epoch 24 Batch 600/4268] avg loss 0.00987743, throughput 3.99809K wps
[Epoch 24 Batch 630/4268] avg loss 0.00924161, throughput 3.99747K wps
[Epoch 24 Batch 660/4268] avg loss 0.00931644, throughput 3.99721K wps
[Epoch 24 Batch 690/4268] avg loss 0.0101814, throughput 4.00029K wps
[Epoch 24 Batch 720/4268] avg loss 0.00926819, throughput 3.99639K wps
[Epoch 24 Batch 750/4268] avg loss 0.00981907, throughput 3.99719K wps
[Epoch 24 Batch 780/4268] avg loss 0.00956563, throughput 3.99961K wps
[Epoch 24 Batch 810/4268] avg loss 0.00998901, throughput 3.99669K wps
[Epoch 24 Batch 840/4268] avg loss 0.00978017, throughput 3.99657K wps
[Epoch 24 Batch 870/4268] avg loss 0.00965875, throughput 3.99914K wps
[Epoch 24 Batch 900/4268] avg loss 0.00936696, throughput 3.99949K wps
[Epoch 24 Batch 930/4268] avg loss 0.0096002, throughput 3.99912K wps
[Epoch 24 Batch 960/4268] avg loss 0.00955121, throughput 4.00194K wps
[Epoch 24 Batch 990/4268] avg loss 0.010059, throughput 3.99245K wps
[Epoch 24 Batch 1020/4268] avg loss 0.00943199, throughput 3.99601K wps
[Epoch 24 Batch 1050/4268] avg loss 0.00954434, throughput 3.99246K wps
[Epoch 24 Batch 1080/4268] avg loss 0.00968465, throughput 3.99702K wps
[Epoch 24 Batch 1110/4268] avg loss 0.0102759, throughput 3.99856K wps
[Epoch 24 Batch 1140/4268] avg loss 0.0103125, throughput 3.99682K wps
[Epoch 24 Batch 1170/4268] avg loss 0.00999651, throughput 3.99635K wps
[Epoch 24 Batch 1200/4268] avg loss 0.00994143, throughput 3.99639K wps
[Epoch 24 Batch 1230/4268] avg loss 0.00972154, throughput 3.99792K wps
[Epoch 24 Batch 1260/4268] avg loss 0.00932785, throughput 3.99613K wps
[Epoch 24 Batch 1290/4268] avg loss 0.0090486, throughput 3.99334K wps
[Epoch 24 Batch 1320/4268] avg loss 0.00949717, throughput 3.99927K wps
[Epoch 24 Batch 1350/4268] avg loss 0.0095374, throughput 3.99821K wps
[Epoch 24 Batch 1380/4268] avg loss 0.00970922, throughput 3.99823K wps
[Epoch 24 Batch 1410/4268] avg loss 0.00983477, throughput 3.99691K wps
[Epoch 24 Batch 1440/4268] avg loss 0.00960662, throughput 3.99857K wps
[Epoch 24 Batch 1470/4268] avg loss 0.00946418, throughput 3.99547K wps
[Epoch 24 Batch 1500/4268] avg loss 0.00960457, throughput 3.99897K wps
[Epoch 24 Batch 1530/4268] avg loss 0.0097997, throughput 3.99777K wps
[Epoch 24 Batch 1560/4268] avg loss 0.00984069, throughput 3.99479K wps
[Epoch 24 Batch 1590/4268] avg loss 0.00986998, throughput 3.99852K wps
[Epoch 24 Batch 1620/4268] avg loss 0.00980752, throughput 4.002K wps
[Epoch 24 Batch 1650/4268] avg loss 0.0097148, throughput 3.99504K wps
[Epoch 24 Batch 1680/4268] avg loss 0.010066, throughput 3.9961K wps
[Epoch 24 Batch 1710/4268] avg loss 0.00953961, throughput 3.99307K wps
[Epoch 24 Batch 1740/4268] avg loss 0.00986183, throughput 3.99371K wps
[Epoch 24 Batch 1770/4268] avg loss 0.00966301, throughput 3.99836K wps
[Epoch 24 Batch 1800/4268] avg loss 0.00996241, throughput 3.99522K wps
[Epoch 24 Batch 1830/4268] avg loss 0.0097618, throughput 3.99936K wps
[Epoch 24 Batch 1860/4268] avg loss 0.00962477, throughput 4.00217K wps
[Epoch 24 Batch 1890/4268] avg loss 0.00954143, throughput 3.99357K wps
[Epoch 24 Batch 1920/4268] avg loss 0.00977996, throughput 3.99858K wps
[Epoch 24 Batch 1950/4268] avg loss 0.00961235, throughput 3.97889K wps
[Epoch 24 Batch 1980/4268] avg loss 0.00938402, throughput 3.99319K wps
[Epoch 24 Batch 2010/4268] avg loss 0.00912192, throughput 3.99959K wps
[Epoch 24 Batch 2040/4268] avg loss 0.00951214, throughput 3.99903K wps
[Epoch 24 Batch 2070/4268] avg loss 0.00931806, throughput 4.00083K wps
[Epoch 24 Batch 2100/4268] avg loss 0.00905704, throughput 3.9929K wps
[Epoch 24 Batch 2130/4268] avg loss 0.00956996, throughput 3.99295K wps
[Epoch 24 Batch 2160/4268] avg loss 0.0101001, throughput 3.99818K wps
[Epoch 24 Batch 2190/4268] avg loss 0.010071, throughput 3.99244K wps
[Epoch 24 Batch 2220/4268] avg loss 0.00985313, throughput 3.998K wps
[Epoch 24 Batch 2250/4268] avg loss 0.0104244, throughput 3.9949K wps
[Epoch 24 Batch 2280/4268] avg loss 0.00945034, throughput 3.99596K wps
[Epoch 24 Batch 2310/4268] avg loss 0.00960294, throughput 3.99648K wps
[Epoch 24 Batch 2340/4268] avg loss 0.00987446, throughput 3.99783K wps
[Epoch 24 Batch 2370/4268] avg loss 0.0100984, throughput 3.99776K wps
[Epoch 24 Batch 2400/4268] avg loss 0.0100123, throughput 3.99768K wps
[Epoch 24 Batch 2430/4268] avg loss 0.00955305, throughput 3.99536K wps
[Epoch 24 Batch 2460/4268] avg loss 0.00985886, throughput 3.99619K wps
[Epoch 24 Batch 2490/4268] avg loss 0.00929481, throughput 3.99183K wps
[Epoch 24 Batch 2520/4268] avg loss 0.0101161, throughput 3.99236K wps
[Epoch 24 Batch 2550/4268] avg loss 0.00978286, throughput 3.99034K wps
[Epoch 24 Batch 2580/4268] avg loss 0.00989044, throughput 3.99294K wps
[Epoch 24 Batch 2610/4268] avg loss 0.0101816, throughput 3.9931K wps
[Epoch 24 Batch 2640/4268] avg loss 0.0105192, throughput 3.99067K wps
[Epoch 24 Batch 2670/4268] avg loss 0.00969913, throughput 3.99647K wps
[Epoch 24 Batch 2700/4268] avg loss 0.00927276, throughput 3.99838K wps
[Epoch 24 Batch 2730/4268] avg loss 0.00999395, throughput 3.99553K wps
[Epoch 24 Batch 2760/4268] avg loss 0.0100537, throughput 3.99696K wps
[Epoch 24 Batch 2790/4268] avg loss 0.0101313, throughput 3.99656K wps
[Epoch 24 Batch 2820/4268] avg loss 0.00972501, throughput 3.99619K wps
[Epoch 24 Batch 2850/4268] avg loss 0.0101896, throughput 3.99115K wps
[Epoch 24 Batch 2880/4268] avg loss 0.0099668, throughput 4.00051K wps
[Epoch 24 Batch 2910/4268] avg loss 0.0104242, throughput 3.99322K wps
[Epoch 24 Batch 2940/4268] avg loss 0.0100148, throughput 3.99035K wps
[Epoch 24 Batch 2970/4268] avg loss 0.0103957, throughput 4.00077K wps
[Epoch 24 Batch 3000/4268] avg loss 0.0104656, throughput 3.9959K wps
[Epoch 24 Batch 3030/4268] avg loss 0.0094573, throughput 3.99595K wps
[Epoch 24 Batch 3060/4268] avg loss 0.00985795, throughput 3.99554K wps
[Epoch 24 Batch 3090/4268] avg loss 0.0101492, throughput 4.00091K wps
[Epoch 24 Batch 3120/4268] avg loss 0.0101145, throughput 4.00578K wps
[Epoch 24 Batch 3150/4268] avg loss 0.00998837, throughput 4.00327K wps
[Epoch 24 Batch 3180/4268] avg loss 0.00974467, throughput 4.00116K wps
[Epoch 24 Batch 3210/4268] avg loss 0.00998634, throughput 4.00329K wps
[Epoch 24 Batch 3240/4268] avg loss 0.00925223, throughput 4.00061K wps
[Epoch 24 Batch 3270/4268] avg loss 0.0099286, throughput 3.99427K wps
[Epoch 24 Batch 3300/4268] avg loss 0.00998217, throughput 3.99527K wps
[Epoch 24 Batch 3330/4268] avg loss 0.010636, throughput 3.99241K wps
[Epoch 24 Batch 3360/4268] avg loss 0.0106615, throughput 3.99754K wps
[Epoch 24 Batch 3390/4268] avg loss 0.0110765, throughput 3.99992K wps
[Epoch 24 Batch 3420/4268] avg loss 0.0101257, throughput 4.00539K wps
[Epoch 24 Batch 3450/4268] avg loss 0.010238, throughput 3.99864K wps
[Epoch 24 Batch 3480/4268] avg loss 0.010393, throughput 3.99781K wps
[Epoch 24 Batch 3510/4268] avg loss 0.0100667, throughput 4.00113K wps
[Epoch 24 Batch 3540/4268] avg loss 0.0101352, throughput 3.99957K wps
[Epoch 24 Batch 3570/4268] avg loss 0.00986807, throughput 3.99691K wps
[Epoch 24 Batch 3600/4268] avg loss 0.00983077, throughput 3.99436K wps
[Epoch 24 Batch 3630/4268] avg loss 0.00985016, throughput 3.99796K wps
[Epoch 24 Batch 3660/4268] avg loss 0.0103613, throughput 3.99663K wps
[Epoch 24 Batch 3690/4268] avg loss 0.00993984, throughput 3.99409K wps
[Epoch 24 Batch 3720/4268] avg loss 0.0101484, throughput 3.99689K wps
[Epoch 24 Batch 3750/4268] avg loss 0.0100126, throughput 3.99429K wps
[Epoch 24 Batch 3780/4268] avg loss 0.0103561, throughput 3.99533K wps
[Epoch 24 Batch 3810/4268] avg loss 0.00977868, throughput 3.99691K wps
[Epoch 24 Batch 3840/4268] avg loss 0.0105818, throughput 3.994K wps
[Epoch 24 Batch 3870/4268] avg loss 0.0100711, throughput 3.99479K wps
[Epoch 24 Batch 3900/4268] avg loss 0.00989055, throughput 3.99619K wps
[Epoch 24 Batch 3930/4268] avg loss 0.0106759, throughput 3.99353K wps
[Epoch 24 Batch 3960/4268] avg loss 0.0101815, throughput 3.99469K wps
[Epoch 24 Batch 3990/4268] avg loss 0.0103792, throughput 4.00083K wps
[Epoch 24 Batch 4020/4268] avg loss 0.010262, throughput 3.99334K wps
[Epoch 24 Batch 4050/4268] avg loss 0.00966701, throughput 3.99459K wps
[Epoch 24 Batch 4080/4268] avg loss 0.0102485, throughput 3.99538K wps
[Epoch 24 Batch 4110/4268] avg loss 0.0104194, throughput 3.99299K wps
[Epoch 24 Batch 4140/4268] avg loss 0.0101038, throughput 3.99337K wps
[Epoch 24 Batch 4170/4268] avg loss 0.0107285, throughput 4.00035K wps
[Epoch 24 Batch 4200/4268] avg loss 0.00960755, throughput 3.99362K wps
[Epoch 24 Batch 4230/4268] avg loss 0.00968933, throughput 3.99544K wps
[Epoch 24 Batch 4260/4268] avg loss 0.00994924, throughput 3.99598K wps
Begin Testing...
[Batch 30/475] elapsed 0.46 s
[Batch 60/475] elapsed 0.43 s
[Batch 90/475] elapsed 0.43 s
[Batch 120/475] elapsed 0.43 s
[Batch 150/475] elapsed 0.43 s
[Batch 180/475] elapsed 0.43 s
[Batch 210/475] elapsed 0.43 s
[Batch 240/475] elapsed 0.43 s
[Batch 270/475] elapsed 0.43 s
[Batch 300/475] elapsed 0.43 s
[Batch 330/475] elapsed 0.43 s
[Batch 360/475] elapsed 0.43 s
[Batch 390/475] elapsed 0.43 s
[Batch 420/475] elapsed 0.43 s
[Batch 450/475] elapsed 0.43 s
[Epoch 24] train avg loss 0.00979331, test acc 0.6705, test avg loss 0.953805, throughput 3.99688K wps