2018-08-07 02:39:45,606 param size: 4810000 2018-08-07 02:39:45,606 initial genotype: 2018-08-07 02:39:45,607 Genotype(recurrent=[('tanh', 0), ('relu', 0), ('relu', 0), ('tanh', 2), ('tanh', 1), ('relu', 0), ('relu', 2), ('relu', 6)], concat=range(1, 9)) 2018-08-07 02:39:45,614 Args: Namespace(alpha=0, arch_lr=0.003, arch_wdecay=0.001, batch_size=256, beta=0.001, bptt=35, clip=0.25, continue_train=False, cuda=True, data='../data/penn/', dropout=0.75, dropoute=0, dropouth=0.25, dropouti=0.2, dropoutx=0.75, emsize=300, epochs=50, gpu=0, log_interval=50, lr=20, max_seq_len_delta=20, nhid=300, nhidlast=300, nonmono=5, save='search-EXP-20180807-023935', seed=77331, single_gpu=True, small_batch_size=256, unrolled=False, wdecay=5e-07) 2018-08-07 02:39:45,614 Model total parameters: 4810000 2018-08-07 02:42:36,326 Genotype(recurrent=[('identity', 0), ('relu', 0), ('relu', 0), ('relu', 1), ('tanh', 2), ('relu', 2), ('relu', 0), ('relu', 7)], concat=range(1, 9)) 2018-08-07 02:42:36,333 | epoch 1 | 50/ 103 batches | lr 20.00 | ms/batch 3414.36 | loss 9.65 | ppl 15592.98 2018-08-07 02:45:25,171 Genotype(recurrent=[('identity', 0), ('tanh', 0), ('relu', 0), ('relu', 1), ('tanh', 3), ('relu', 2), ('relu', 0), ('relu', 0)], concat=range(1, 9)) 2018-08-07 02:45:25,178 | epoch 1 | 100/ 103 batches | lr 20.00 | ms/batch 3376.90 | loss 9.24 | ppl 10297.44 2018-08-07 02:46:30,088 ----------------------------------------------------------------------------------------- 2018-08-07 02:46:30,095 | end of epoch 1 | time: 404.48s | valid loss 8.76 | valid ppl 6363.88 2018-08-07 02:46:30,095 ----------------------------------------------------------------------------------------- 2018-08-07 02:46:30,138 Saving Normal! 2018-08-07 02:49:23,855 Genotype(recurrent=[('tanh', 0), ('tanh', 0), ('relu', 1), ('relu', 1), ('tanh', 3), ('identity', 0), ('relu', 6), ('relu', 0)], concat=range(1, 9)) 2018-08-07 02:49:23,861 | epoch 2 | 50/ 103 batches | lr 20.00 | ms/batch 3474.46 | loss 9.14 | ppl 9344.75 2018-08-07 02:52:10,088 Genotype(recurrent=[('tanh', 0), ('identity', 0), ('relu', 0), ('relu', 1), ('tanh', 3), ('relu', 2), ('relu', 6), ('relu', 0)], concat=range(1, 9)) 2018-08-07 02:52:10,096 | epoch 2 | 100/ 103 batches | lr 20.00 | ms/batch 3324.68 | loss 8.59 | ppl 5361.31 2018-08-07 02:53:11,359 ----------------------------------------------------------------------------------------- 2018-08-07 02:53:11,364 | end of epoch 2 | time: 401.23s | valid loss 7.88 | valid ppl 2653.08 2018-08-07 02:53:11,364 ----------------------------------------------------------------------------------------- 2018-08-07 02:53:11,604 Saving Normal! 2018-08-07 02:56:02,965 Genotype(recurrent=[('identity', 0), ('identity', 0), ('relu', 1), ('relu', 0), ('tanh', 0), ('identity', 0), ('relu', 6), ('relu', 7)], concat=range(1, 9)) 2018-08-07 02:56:02,972 | epoch 3 | 50/ 103 batches | lr 20.00 | ms/batch 3427.34 | loss 8.47 | ppl 4752.29 2018-08-07 02:58:51,499 Genotype(recurrent=[('identity', 0), ('relu', 1), ('relu', 1), ('identity', 0), ('tanh', 3), ('relu', 4), ('relu', 0), ('relu', 6)], concat=range(1, 9)) 2018-08-07 02:58:51,506 | epoch 3 | 100/ 103 batches | lr 20.00 | ms/batch 3370.69 | loss 8.27 | ppl 3891.71 2018-08-07 02:59:52,973 ----------------------------------------------------------------------------------------- 2018-08-07 02:59:52,980 | end of epoch 3 | time: 401.38s | valid loss 7.53 | valid ppl 1869.34 2018-08-07 02:59:52,981 ----------------------------------------------------------------------------------------- 2018-08-07 02:59:53,220 Saving Normal! 2018-08-07 03:02:42,723 Genotype(recurrent=[('relu', 0), ('relu', 1), ('relu', 1), ('relu', 1), ('tanh', 3), ('relu', 4), ('relu', 6), ('relu', 6)], concat=range(1, 9)) 2018-08-07 03:02:42,730 | epoch 4 | 50/ 103 batches | lr 20.00 | ms/batch 3390.19 | loss 8.35 | ppl 4228.56 2018-08-07 03:05:32,365 Genotype(recurrent=[('relu', 0), ('relu', 1), ('relu', 1), ('relu', 0), ('relu', 1), ('relu', 2), ('relu', 0), ('relu', 2)], concat=range(1, 9)) 2018-08-07 03:05:32,373 | epoch 4 | 100/ 103 batches | lr 20.00 | ms/batch 3392.86 | loss 7.84 | ppl 2545.07 2018-08-07 03:06:36,644 ----------------------------------------------------------------------------------------- 2018-08-07 03:06:36,649 | end of epoch 4 | time: 403.43s | valid loss 7.41 | valid ppl 1655.30 2018-08-07 03:06:36,649 ----------------------------------------------------------------------------------------- 2018-08-07 03:06:36,732 Saving Normal! 2018-08-07 03:09:28,256 Genotype(recurrent=[('relu', 0), ('relu', 0), ('relu', 0), ('relu', 0), ('relu', 0), ('relu', 2), ('tanh', 2), ('relu', 2)], concat=range(1, 9)) 2018-08-07 03:09:28,265 | epoch 5 | 50/ 103 batches | lr 20.00 | ms/batch 3430.66 | loss 9.28 | ppl 10757.90 2018-08-07 03:12:13,179 Genotype(recurrent=[('relu', 0), ('identity', 0), ('relu', 1), ('relu', 0), ('relu', 0), ('relu', 2), ('tanh', 2), ('relu', 2)], concat=range(1, 9)) 2018-08-07 03:12:13,186 | epoch 5 | 100/ 103 batches | lr 20.00 | ms/batch 3298.41 | loss 8.06 | ppl 3158.87 2018-08-07 03:13:22,487 ----------------------------------------------------------------------------------------- 2018-08-07 03:13:22,494 | end of epoch 5 | time: 405.76s | valid loss 7.39 | valid ppl 1613.05 2018-08-07 03:13:22,494 ----------------------------------------------------------------------------------------- 2018-08-07 03:13:22,836 Saving Normal! 2018-08-07 03:16:11,669 Genotype(recurrent=[('relu', 0), ('identity', 0), ('relu', 0), ('relu', 0), ('relu', 0), ('relu', 2), ('sigmoid', 6), ('relu', 2)], concat=range(1, 9)) 2018-08-07 03:16:11,676 | epoch 6 | 50/ 103 batches | lr 20.00 | ms/batch 3376.81 | loss 9.59 | ppl 14602.98 2018-08-07 03:19:00,667 Genotype(recurrent=[('relu', 0), ('identity', 0), ('relu', 0), ('relu', 0), ('relu', 0), ('relu', 2), ('sigmoid', 6), ('relu', 2)], concat=range(1, 9)) 2018-08-07 03:19:00,674 | epoch 6 | 100/ 103 batches | lr 20.00 | ms/batch 3379.96 | loss 8.95 | ppl 7737.42 2018-08-07 03:20:08,057 ----------------------------------------------------------------------------------------- 2018-08-07 03:20:08,064 | end of epoch 6 | time: 405.23s | valid loss 7.33 | valid ppl 1530.63 2018-08-07 03:20:08,064 ----------------------------------------------------------------------------------------- 2018-08-07 03:20:08,300 Saving Normal! 2018-08-07 03:23:00,097 Genotype(recurrent=[('relu', 0), ('identity', 0), ('relu', 0), ('relu', 0), ('relu', 0), ('relu', 2), ('relu', 0), ('relu', 0)], concat=range(1, 9)) 2018-08-07 03:23:00,104 | epoch 7 | 50/ 103 batches | lr 20.00 | ms/batch 3436.07 | loss 8.61 | ppl 5488.02 2018-08-07 03:25:46,002 Genotype(recurrent=[('relu', 0), ('identity', 0), ('relu', 0), ('relu', 0), ('relu', 0), ('relu', 2), ('relu', 0), ('relu', 0)], concat=range(1, 9)) 2018-08-07 03:25:46,009 | epoch 7 | 100/ 103 batches | lr 20.00 | ms/batch 3318.10 | loss 8.27 | ppl 3894.56 2018-08-07 03:26:53,667 ----------------------------------------------------------------------------------------- 2018-08-07 03:26:53,670 | end of epoch 7 | time: 405.37s | valid loss 7.29 | valid ppl 1466.27 2018-08-07 03:26:53,670 ----------------------------------------------------------------------------------------- 2018-08-07 03:26:53,908 Saving Normal! 2018-08-07 03:29:46,509 Genotype(recurrent=[('relu', 0), ('identity', 0), ('relu', 0), ('relu', 0), ('relu', 0), ('relu', 2), ('relu', 0), ('relu', 0)], concat=range(1, 9)) 2018-08-07 03:29:46,516 | epoch 8 | 50/ 103 batches | lr 20.00 | ms/batch 3452.16 | loss 8.74 | ppl 6230.81 2018-08-07 03:32:31,813 Genotype(recurrent=[('relu', 0), ('identity', 0), ('relu', 0), ('relu', 0), ('relu', 0), ('relu', 2), ('relu', 0), ('relu', 0)], concat=range(1, 9)) 2018-08-07 03:32:31,820 | epoch 8 | 100/ 103 batches | lr 20.00 | ms/batch 3306.08 | loss 8.52 | ppl 5011.25 2018-08-07 03:33:35,930 ----------------------------------------------------------------------------------------- 2018-08-07 03:33:35,938 | end of epoch 8 | time: 402.03s | valid loss 7.27 | valid ppl 1434.72 2018-08-07 03:33:35,938 ----------------------------------------------------------------------------------------- 2018-08-07 03:33:36,176 Saving Normal! 2018-08-07 03:36:24,986 Genotype(recurrent=[('relu', 0), ('identity', 0), ('relu', 0), ('relu', 0), ('relu', 0), ('relu', 2), ('relu', 0), ('relu', 0)], concat=range(1, 9)) 2018-08-07 03:36:24,993 | epoch 9 | 50/ 103 batches | lr 20.00 | ms/batch 3376.33 | loss 8.48 | ppl 4807.23 2018-08-07 03:39:13,818 Genotype(recurrent=[('relu', 0), ('identity', 0), ('relu', 0), ('relu', 0), ('relu', 0), ('relu', 2), ('relu', 0), ('relu', 0)], concat=range(1, 9)) 2018-08-07 03:39:13,824 | epoch 9 | 100/ 103 batches | lr 20.00 | ms/batch 3376.62 | loss 8.64 | ppl 5643.68 2018-08-07 03:40:10,887 ----------------------------------------------------------------------------------------- 2018-08-07 03:40:10,892 | end of epoch 9 | time: 394.72s | valid loss 7.12 | valid ppl 1239.47 2018-08-07 03:40:10,892 ----------------------------------------------------------------------------------------- 2018-08-07 03:40:11,132 Saving Normal! 2018-08-07 03:43:03,455 Genotype(recurrent=[('relu', 0), ('identity', 0), ('identity', 0), ('relu', 0), ('relu', 0), ('relu', 2), ('relu', 0), ('relu', 3)], concat=range(1, 9)) 2018-08-07 03:43:03,462 | epoch 10 | 50/ 103 batches | lr 20.00 | ms/batch 3446.59 | loss 9.01 | ppl 8222.52 2018-08-07 03:45:51,431 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 0), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 03:45:51,439 | epoch 10 | 100/ 103 batches | lr 20.00 | ms/batch 3359.55 | loss 9.74 | ppl 17032.23 2018-08-07 03:46:57,446 ----------------------------------------------------------------------------------------- 2018-08-07 03:46:57,452 | end of epoch 10 | time: 406.32s | valid loss 7.18 | valid ppl 1310.42 2018-08-07 03:46:57,452 ----------------------------------------------------------------------------------------- 2018-08-07 03:49:46,320 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 0), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 03:49:46,327 | epoch 11 | 50/ 103 batches | lr 20.00 | ms/batch 3377.51 | loss 10.19 | ppl 26558.01 2018-08-07 03:52:35,141 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 0), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 03:52:35,149 | epoch 11 | 100/ 103 batches | lr 20.00 | ms/batch 3376.44 | loss 9.14 | ppl 9360.75 2018-08-07 03:53:42,868 ----------------------------------------------------------------------------------------- 2018-08-07 03:53:42,875 | end of epoch 11 | time: 405.42s | valid loss 7.26 | valid ppl 1424.75 2018-08-07 03:53:42,875 ----------------------------------------------------------------------------------------- 2018-08-07 03:56:32,202 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 0), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 03:56:32,209 | epoch 12 | 50/ 103 batches | lr 20.00 | ms/batch 3386.68 | loss 9.06 | ppl 8587.42 2018-08-07 03:59:22,022 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 03:59:22,029 | epoch 12 | 100/ 103 batches | lr 20.00 | ms/batch 3396.40 | loss 9.68 | ppl 15989.83 2018-08-07 04:00:26,869 ----------------------------------------------------------------------------------------- 2018-08-07 04:00:26,874 | end of epoch 12 | time: 404.00s | valid loss 7.16 | valid ppl 1289.85 2018-08-07 04:00:26,875 ----------------------------------------------------------------------------------------- 2018-08-07 04:03:14,935 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 04:03:14,941 | epoch 13 | 50/ 103 batches | lr 20.00 | ms/batch 3361.33 | loss 10.44 | ppl 34212.39 2018-08-07 04:06:04,210 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 04:06:04,217 | epoch 13 | 100/ 103 batches | lr 20.00 | ms/batch 3385.51 | loss 10.76 | ppl 47012.79 2018-08-07 04:07:11,702 ----------------------------------------------------------------------------------------- 2018-08-07 04:07:11,709 | end of epoch 13 | time: 404.83s | valid loss 7.27 | valid ppl 1430.87 2018-08-07 04:07:11,709 ----------------------------------------------------------------------------------------- 2018-08-07 04:09:59,654 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 04:09:59,663 | epoch 14 | 50/ 103 batches | lr 20.00 | ms/batch 3359.07 | loss 11.19 | ppl 72203.05 2018-08-07 04:12:45,175 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 04:12:45,181 | epoch 14 | 100/ 103 batches | lr 20.00 | ms/batch 3310.37 | loss 9.99 | ppl 21838.67 2018-08-07 04:13:48,942 ----------------------------------------------------------------------------------------- 2018-08-07 04:13:48,948 | end of epoch 14 | time: 397.24s | valid loss 7.41 | valid ppl 1649.65 2018-08-07 04:13:48,948 ----------------------------------------------------------------------------------------- 2018-08-07 04:16:40,043 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 04:16:40,050 | epoch 15 | 50/ 103 batches | lr 20.00 | ms/batch 3422.03 | loss 9.17 | ppl 9602.33 2018-08-07 04:19:25,288 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 04:19:25,294 | epoch 15 | 100/ 103 batches | lr 20.00 | ms/batch 3304.89 | loss 8.63 | ppl 5596.26 2018-08-07 04:20:34,693 ----------------------------------------------------------------------------------------- 2018-08-07 04:20:34,699 | end of epoch 15 | time: 405.75s | valid loss 7.09 | valid ppl 1196.28 2018-08-07 04:20:34,699 ----------------------------------------------------------------------------------------- 2018-08-07 04:20:34,922 Saving Normal! 2018-08-07 04:23:27,867 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 04:23:27,873 | epoch 16 | 50/ 103 batches | lr 20.00 | ms/batch 3459.02 | loss 8.69 | ppl 5928.21 2018-08-07 04:26:15,239 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 04:26:15,246 | epoch 16 | 100/ 103 batches | lr 20.00 | ms/batch 3347.45 | loss 8.44 | ppl 4642.73 2018-08-07 04:27:16,540 ----------------------------------------------------------------------------------------- 2018-08-07 04:27:16,547 | end of epoch 16 | time: 401.63s | valid loss 7.06 | valid ppl 1165.47 2018-08-07 04:27:16,548 ----------------------------------------------------------------------------------------- 2018-08-07 04:27:16,753 Saving Normal! 2018-08-07 04:30:06,356 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 04:30:06,362 | epoch 17 | 50/ 103 batches | lr 20.00 | ms/batch 3392.19 | loss 8.51 | ppl 4952.49 2018-08-07 04:32:55,930 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 04:32:55,938 | epoch 17 | 100/ 103 batches | lr 20.00 | ms/batch 3391.51 | loss 8.30 | ppl 4043.48 2018-08-07 04:33:56,204 ----------------------------------------------------------------------------------------- 2018-08-07 04:33:56,207 | end of epoch 17 | time: 399.45s | valid loss 6.99 | valid ppl 1087.31 2018-08-07 04:33:56,207 ----------------------------------------------------------------------------------------- 2018-08-07 04:33:56,346 Saving Normal! 2018-08-07 04:36:46,557 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 04:36:46,564 | epoch 18 | 50/ 103 batches | lr 20.00 | ms/batch 3404.36 | loss 8.40 | ppl 4425.17 2018-08-07 04:39:31,639 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 04:39:31,646 | epoch 18 | 100/ 103 batches | lr 20.00 | ms/batch 3301.64 | loss 8.12 | ppl 3364.20 2018-08-07 04:40:41,155 ----------------------------------------------------------------------------------------- 2018-08-07 04:40:41,161 | end of epoch 18 | time: 404.82s | valid loss 6.87 | valid ppl 961.29 2018-08-07 04:40:41,161 ----------------------------------------------------------------------------------------- 2018-08-07 04:40:41,245 Saving Normal! 2018-08-07 04:43:31,857 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 04:43:31,865 | epoch 19 | 50/ 103 batches | lr 20.00 | ms/batch 3412.40 | loss 8.18 | ppl 3571.76 2018-08-07 04:46:19,691 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 04:46:19,698 | epoch 19 | 100/ 103 batches | lr 20.00 | ms/batch 3356.65 | loss 7.95 | ppl 2826.63 2018-08-07 04:47:17,358 ----------------------------------------------------------------------------------------- 2018-08-07 04:47:17,365 | end of epoch 19 | time: 396.12s | valid loss 6.88 | valid ppl 972.69 2018-08-07 04:47:17,365 ----------------------------------------------------------------------------------------- 2018-08-07 04:50:08,270 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 04:50:08,278 | epoch 20 | 50/ 103 batches | lr 20.00 | ms/batch 3418.26 | loss 7.95 | ppl 2848.71 2018-08-07 04:52:55,115 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 04:52:55,122 | epoch 20 | 100/ 103 batches | lr 20.00 | ms/batch 3336.88 | loss 7.73 | ppl 2278.41 2018-08-07 04:54:02,437 ----------------------------------------------------------------------------------------- 2018-08-07 04:54:02,442 | end of epoch 20 | time: 405.08s | valid loss 6.85 | valid ppl 948.60 2018-08-07 04:54:02,442 ----------------------------------------------------------------------------------------- 2018-08-07 04:54:02,541 Saving Normal! 2018-08-07 04:56:52,347 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 04:56:52,354 | epoch 21 | 50/ 103 batches | lr 20.00 | ms/batch 3396.25 | loss 7.85 | ppl 2568.12 2018-08-07 04:59:40,508 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 04:59:40,515 | epoch 21 | 100/ 103 batches | lr 20.00 | ms/batch 3363.22 | loss 7.67 | ppl 2150.25 2018-08-07 05:00:46,961 ----------------------------------------------------------------------------------------- 2018-08-07 05:00:46,967 | end of epoch 21 | time: 404.43s | valid loss 6.88 | valid ppl 968.19 2018-08-07 05:00:46,967 ----------------------------------------------------------------------------------------- 2018-08-07 05:03:37,873 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 05:03:37,880 | epoch 22 | 50/ 103 batches | lr 20.00 | ms/batch 3418.26 | loss 7.79 | ppl 2423.26 2018-08-07 05:06:27,244 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 05:06:27,251 | epoch 22 | 100/ 103 batches | lr 20.00 | ms/batch 3387.41 | loss 7.51 | ppl 1833.41 2018-08-07 05:07:31,135 ----------------------------------------------------------------------------------------- 2018-08-07 05:07:31,142 | end of epoch 22 | time: 404.18s | valid loss 6.79 | valid ppl 885.87 2018-08-07 05:07:31,142 ----------------------------------------------------------------------------------------- 2018-08-07 05:07:31,213 Saving Normal! 2018-08-07 05:10:20,816 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 05:10:20,822 | epoch 23 | 50/ 103 batches | lr 20.00 | ms/batch 3392.18 | loss 7.71 | ppl 2223.44 2018-08-07 05:13:08,080 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 05:13:08,087 | epoch 23 | 100/ 103 batches | lr 20.00 | ms/batch 3345.29 | loss 7.38 | ppl 1609.51 2018-08-07 05:14:09,362 ----------------------------------------------------------------------------------------- 2018-08-07 05:14:09,370 | end of epoch 23 | time: 398.16s | valid loss 6.76 | valid ppl 866.82 2018-08-07 05:14:09,370 ----------------------------------------------------------------------------------------- 2018-08-07 05:14:09,460 Saving Normal! 2018-08-07 05:16:58,512 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 05:16:58,519 | epoch 24 | 50/ 103 batches | lr 20.00 | ms/batch 3381.19 | loss 7.50 | ppl 1816.23 2018-08-07 05:19:44,152 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 05:19:44,159 | epoch 24 | 100/ 103 batches | lr 20.00 | ms/batch 3312.80 | loss 7.24 | ppl 1394.14 2018-08-07 05:20:51,683 ----------------------------------------------------------------------------------------- 2018-08-07 05:20:51,689 | end of epoch 24 | time: 402.23s | valid loss 6.77 | valid ppl 872.77 2018-08-07 05:20:51,689 ----------------------------------------------------------------------------------------- 2018-08-07 05:23:41,515 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 05:23:41,522 | epoch 25 | 50/ 103 batches | lr 20.00 | ms/batch 3396.66 | loss 7.40 | ppl 1633.08 2018-08-07 05:26:30,186 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 05:26:30,193 | epoch 25 | 100/ 103 batches | lr 20.00 | ms/batch 3373.42 | loss 7.15 | ppl 1270.55 2018-08-07 05:27:34,733 ----------------------------------------------------------------------------------------- 2018-08-07 05:27:34,739 | end of epoch 25 | time: 403.05s | valid loss 6.77 | valid ppl 870.81 2018-08-07 05:27:34,739 ----------------------------------------------------------------------------------------- 2018-08-07 05:30:28,200 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 05:30:28,208 | epoch 26 | 50/ 103 batches | lr 20.00 | ms/batch 3469.37 | loss 7.25 | ppl 1409.13 2018-08-07 05:33:16,065 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 05:33:16,072 | epoch 26 | 100/ 103 batches | lr 20.00 | ms/batch 3357.27 | loss 7.03 | ppl 1135.21 2018-08-07 05:34:13,627 ----------------------------------------------------------------------------------------- 2018-08-07 05:34:13,631 | end of epoch 26 | time: 398.89s | valid loss 6.82 | valid ppl 915.11 2018-08-07 05:34:13,631 ----------------------------------------------------------------------------------------- 2018-08-07 05:37:03,686 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 05:37:03,694 | epoch 27 | 50/ 103 batches | lr 20.00 | ms/batch 3401.25 | loss 7.18 | ppl 1310.19 2018-08-07 05:39:52,823 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 05:39:52,830 | epoch 27 | 100/ 103 batches | lr 20.00 | ms/batch 3382.73 | loss 6.93 | ppl 1021.58 2018-08-07 05:41:00,411 ----------------------------------------------------------------------------------------- 2018-08-07 05:41:00,416 | end of epoch 27 | time: 406.79s | valid loss 6.74 | valid ppl 849.19 2018-08-07 05:41:00,417 ----------------------------------------------------------------------------------------- 2018-08-07 05:41:00,505 Saving Normal! 2018-08-07 05:43:50,365 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 05:43:50,371 | epoch 28 | 50/ 103 batches | lr 20.00 | ms/batch 3397.32 | loss 7.00 | ppl 1096.44 2018-08-07 05:46:39,373 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 05:46:39,380 | epoch 28 | 100/ 103 batches | lr 20.00 | ms/batch 3380.16 | loss 6.83 | ppl 923.18 2018-08-07 05:47:40,536 ----------------------------------------------------------------------------------------- 2018-08-07 05:47:40,541 | end of epoch 28 | time: 400.04s | valid loss 6.72 | valid ppl 826.04 2018-08-07 05:47:40,541 ----------------------------------------------------------------------------------------- 2018-08-07 05:47:40,616 Saving Normal! 2018-08-07 05:50:32,212 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 05:50:32,221 | epoch 29 | 50/ 103 batches | lr 20.00 | ms/batch 3432.08 | loss 6.87 | ppl 965.53 2018-08-07 05:53:17,936 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 05:53:17,944 | epoch 29 | 100/ 103 batches | lr 20.00 | ms/batch 3314.45 | loss 6.66 | ppl 776.89 2018-08-07 05:54:19,249 ----------------------------------------------------------------------------------------- 2018-08-07 05:54:19,254 | end of epoch 29 | time: 398.64s | valid loss 6.68 | valid ppl 795.74 2018-08-07 05:54:19,255 ----------------------------------------------------------------------------------------- 2018-08-07 05:54:19,337 Saving Normal! 2018-08-07 05:57:11,456 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 05:57:11,463 | epoch 30 | 50/ 103 batches | lr 20.00 | ms/batch 3442.51 | loss 6.78 | ppl 881.93 2018-08-07 05:59:59,595 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 05:59:59,603 | epoch 30 | 100/ 103 batches | lr 20.00 | ms/batch 3362.80 | loss 6.59 | ppl 728.99 2018-08-07 06:01:04,171 ----------------------------------------------------------------------------------------- 2018-08-07 06:01:04,178 | end of epoch 30 | time: 404.84s | valid loss 6.65 | valid ppl 775.43 2018-08-07 06:01:04,178 ----------------------------------------------------------------------------------------- 2018-08-07 06:01:04,245 Saving Normal! 2018-08-07 06:03:55,871 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 06:03:55,877 | epoch 31 | 50/ 103 batches | lr 20.00 | ms/batch 3432.64 | loss 6.65 | ppl 769.64 2018-08-07 06:06:42,584 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 06:06:42,591 | epoch 31 | 100/ 103 batches | lr 20.00 | ms/batch 3334.26 | loss 6.47 | ppl 646.77 2018-08-07 06:07:43,840 ----------------------------------------------------------------------------------------- 2018-08-07 06:07:43,845 | end of epoch 31 | time: 399.60s | valid loss 6.66 | valid ppl 783.58 2018-08-07 06:07:43,845 ----------------------------------------------------------------------------------------- 2018-08-07 06:10:32,349 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 06:10:32,356 | epoch 32 | 50/ 103 batches | lr 20.00 | ms/batch 3370.23 | loss 6.56 | ppl 705.90 2018-08-07 06:13:18,013 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 06:13:18,020 | epoch 32 | 100/ 103 batches | lr 20.00 | ms/batch 3313.27 | loss 6.39 | ppl 593.78 2018-08-07 06:14:21,328 ----------------------------------------------------------------------------------------- 2018-08-07 06:14:21,333 | end of epoch 32 | time: 397.49s | valid loss 6.60 | valid ppl 733.01 2018-08-07 06:14:21,333 ----------------------------------------------------------------------------------------- 2018-08-07 06:14:21,401 Saving Normal! 2018-08-07 06:17:14,690 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 06:17:14,697 | epoch 33 | 50/ 103 batches | lr 20.00 | ms/batch 3465.91 | loss 6.48 | ppl 650.94 2018-08-07 06:20:04,428 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 06:20:04,436 | epoch 33 | 100/ 103 batches | lr 20.00 | ms/batch 3394.79 | loss 6.32 | ppl 556.93 2018-08-07 06:21:04,852 ----------------------------------------------------------------------------------------- 2018-08-07 06:21:04,860 | end of epoch 33 | time: 403.46s | valid loss 6.61 | valid ppl 738.88 2018-08-07 06:21:04,860 ----------------------------------------------------------------------------------------- 2018-08-07 06:23:58,399 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 06:23:58,407 | epoch 34 | 50/ 103 batches | lr 20.00 | ms/batch 3470.93 | loss 6.40 | ppl 601.39 2018-08-07 06:26:48,875 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 06:26:48,883 | epoch 34 | 100/ 103 batches | lr 20.00 | ms/batch 3409.51 | loss 6.26 | ppl 524.21 2018-08-07 06:27:42,881 ----------------------------------------------------------------------------------------- 2018-08-07 06:27:42,882 | end of epoch 34 | time: 398.02s | valid loss 6.76 | valid ppl 861.88 2018-08-07 06:27:42,882 ----------------------------------------------------------------------------------------- 2018-08-07 06:30:31,896 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 06:30:31,903 | epoch 35 | 50/ 103 batches | lr 20.00 | ms/batch 3380.42 | loss 6.38 | ppl 588.26 2018-08-07 06:33:18,663 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 06:33:18,673 | epoch 35 | 100/ 103 batches | lr 20.00 | ms/batch 3335.40 | loss 6.20 | ppl 492.14 2018-08-07 06:34:29,966 ----------------------------------------------------------------------------------------- 2018-08-07 06:34:29,971 | end of epoch 35 | time: 407.09s | valid loss 6.67 | valid ppl 791.21 2018-08-07 06:34:29,971 ----------------------------------------------------------------------------------------- 2018-08-07 06:37:16,793 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 06:37:16,801 | epoch 36 | 50/ 103 batches | lr 20.00 | ms/batch 3336.60 | loss 6.31 | ppl 551.49 2018-08-07 06:40:03,954 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 06:40:03,961 | epoch 36 | 100/ 103 batches | lr 20.00 | ms/batch 3343.21 | loss 6.17 | ppl 480.56 2018-08-07 06:41:13,296 ----------------------------------------------------------------------------------------- 2018-08-07 06:41:13,302 | end of epoch 36 | time: 403.33s | valid loss 6.64 | valid ppl 764.95 2018-08-07 06:41:13,302 ----------------------------------------------------------------------------------------- 2018-08-07 06:44:03,237 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 06:44:03,244 | epoch 37 | 50/ 103 batches | lr 20.00 | ms/batch 3398.84 | loss 6.29 | ppl 538.07 2018-08-07 06:46:51,298 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 06:46:51,306 | epoch 37 | 100/ 103 batches | lr 20.00 | ms/batch 3361.24 | loss 6.19 | ppl 486.69 2018-08-07 06:47:55,178 ----------------------------------------------------------------------------------------- 2018-08-07 06:47:55,183 | end of epoch 37 | time: 401.88s | valid loss 6.64 | valid ppl 763.01 2018-08-07 06:47:55,183 ----------------------------------------------------------------------------------------- 2018-08-07 06:50:43,073 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 06:50:43,081 | epoch 38 | 50/ 103 batches | lr 20.00 | ms/batch 3357.96 | loss 6.29 | ppl 536.74 2018-08-07 06:53:28,568 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 06:53:28,575 | epoch 38 | 100/ 103 batches | lr 20.00 | ms/batch 3309.87 | loss 6.15 | ppl 470.30 2018-08-07 06:54:38,104 ----------------------------------------------------------------------------------------- 2018-08-07 06:54:38,110 | end of epoch 38 | time: 402.93s | valid loss 6.62 | valid ppl 753.15 2018-08-07 06:54:38,110 ----------------------------------------------------------------------------------------- 2018-08-07 06:57:27,161 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 06:57:27,168 | epoch 39 | 50/ 103 batches | lr 20.00 | ms/batch 3381.16 | loss 6.24 | ppl 512.42 2018-08-07 07:00:13,254 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 07:00:13,262 | epoch 39 | 100/ 103 batches | lr 20.00 | ms/batch 3321.88 | loss 6.09 | ppl 443.63 2018-08-07 07:01:21,271 ----------------------------------------------------------------------------------------- 2018-08-07 07:01:21,277 | end of epoch 39 | time: 403.17s | valid loss 6.63 | valid ppl 760.01 2018-08-07 07:01:21,277 ----------------------------------------------------------------------------------------- 2018-08-07 07:04:14,674 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 07:04:14,682 | epoch 40 | 50/ 103 batches | lr 20.00 | ms/batch 3468.10 | loss 6.24 | ppl 513.17 2018-08-07 07:07:02,391 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 07:07:02,399 | epoch 40 | 100/ 103 batches | lr 20.00 | ms/batch 3354.33 | loss 6.09 | ppl 440.71 2018-08-07 07:08:00,055 ----------------------------------------------------------------------------------------- 2018-08-07 07:08:00,067 | end of epoch 40 | time: 398.79s | valid loss 6.61 | valid ppl 742.93 2018-08-07 07:08:00,067 ----------------------------------------------------------------------------------------- 2018-08-07 07:10:53,532 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 07:10:53,539 | epoch 41 | 50/ 103 batches | lr 20.00 | ms/batch 3469.43 | loss 6.21 | ppl 498.20 2018-08-07 07:13:40,105 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 07:13:40,112 | epoch 41 | 100/ 103 batches | lr 20.00 | ms/batch 3331.46 | loss 6.07 | ppl 433.94 2018-08-07 07:14:37,550 ----------------------------------------------------------------------------------------- 2018-08-07 07:14:37,554 | end of epoch 41 | time: 397.49s | valid loss 6.62 | valid ppl 749.67 2018-08-07 07:14:37,554 ----------------------------------------------------------------------------------------- 2018-08-07 07:17:26,902 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 07:17:26,909 | epoch 42 | 50/ 103 batches | lr 20.00 | ms/batch 3387.11 | loss 6.18 | ppl 482.99 2018-08-07 07:20:13,401 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 07:20:13,410 | epoch 42 | 100/ 103 batches | lr 20.00 | ms/batch 3330.01 | loss 6.03 | ppl 416.41 2018-08-07 07:21:19,123 ----------------------------------------------------------------------------------------- 2018-08-07 07:21:19,129 | end of epoch 42 | time: 401.57s | valid loss 6.61 | valid ppl 741.67 2018-08-07 07:21:19,129 ----------------------------------------------------------------------------------------- 2018-08-07 07:24:11,442 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 07:24:11,448 | epoch 43 | 50/ 103 batches | lr 20.00 | ms/batch 3446.38 | loss 6.17 | ppl 477.20 2018-08-07 07:26:59,287 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 07:26:59,293 | epoch 43 | 100/ 103 batches | lr 20.00 | ms/batch 3356.90 | loss 6.06 | ppl 427.55 2018-08-07 07:28:03,313 ----------------------------------------------------------------------------------------- 2018-08-07 07:28:03,319 | end of epoch 43 | time: 404.19s | valid loss 6.62 | valid ppl 750.55 2018-08-07 07:28:03,319 ----------------------------------------------------------------------------------------- 2018-08-07 07:30:54,169 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 07:30:54,176 | epoch 44 | 50/ 103 batches | lr 20.00 | ms/batch 3417.14 | loss 6.17 | ppl 477.69 2018-08-07 07:33:40,583 Genotype(recurrent=[('relu', 0), ('relu', 0), ('identity', 0), ('relu', 0), ('relu', 2), ('relu', 2), ('relu', 0), ('identity', 1)], concat=range(1, 9)) 2018-08-07 07:33:40,590 | epoch 44 | 100/ 103 batches | lr 20.00 | ms/batch 3328.27 | loss 6.03 | ppl 416.47 2018-08-07 07:34:44,553 ----------------------------------------------------------------------------------------- 2018-08-07 07:34:44,558 | end of epoch 44 | time: 401.24s | valid loss 6.64 | valid ppl 763.77 2018-08-07 07:34:44,558 -----------------------------------------------------------------------------------------