Skip to content
Permalink
master
Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?

web-data/gluonnlp/logs/nmt/transformer_en_de_u512.log

Go to file
 
 
Cannot retrieve contributors at this time
2108 lines (2108 sloc) 226 KB
Raw Blame
Open in GitHub Desktop
2018-06-25 19:17:22,519 - root - Namespace(average_checkpoint=False, average_start=5, batch_size=4096, beam_size=4, bleu='13a', bucket_ratio=0.0, bucket_scheme='exp', dataset='WMT2014BPE', dropout=0.1, epochs=40, epsilon=0.1, gpus='0,1,2,3', hidden_size=2048, log_interval=100, lp_alpha=0.6, lp_k=5, lr=1.0, magnitude=3.0, num_accumulated=8, num_averages=5, num_buckets=20, num_heads=8, num_layers=6, num_units=512, optimizer='adam', save_dir='transformer_en_de_u512', scaled=True, src_lang='en', src_max_len=-1, test_batch_size=256, tgt_lang='de', tgt_max_len=-1, warmup_steps=8000.0)
2018-06-25 19:19:12,102 - root - NMTModel(
(encoder): TransformerEncoder(
(dropout_layer): Dropout(p = 0.1, axes=())
(transformer_cells): HybridSequential(
(0): TransformerEncoderCell(
(dropout_layer): Dropout(p = 0.1, axes=())
(proj): Dense(None -> 512, linear)
(attention_cell): MultiHeadAttentionCell(
(proj_query): Dense(None -> 512, linear)
(proj_key): Dense(None -> 512, linear)
(_base_cell): DotProductAttentionCell(
(_dropout_layer): Dropout(p = 0.1, axes=())
)
(proj_value): Dense(None -> 512, linear)
)
(ffn): PositionwiseFFN(
(dropout_layer): Dropout(p = 0.1, axes=())
(ffn_1): Dense(None -> 2048, Activation(relu))
(ffn_2): Dense(None -> 512, linear)
(layer_norm): LayerNorm(axis=-1, center=True, eps=1e-05, scale=True, in_channels=0)
)
(layer_norm): LayerNorm(axis=-1, center=True, eps=1e-05, scale=True, in_channels=0)
)
(1): TransformerEncoderCell(
(dropout_layer): Dropout(p = 0.1, axes=())
(proj): Dense(None -> 512, linear)
(attention_cell): MultiHeadAttentionCell(
(proj_query): Dense(None -> 512, linear)
(proj_key): Dense(None -> 512, linear)
(_base_cell): DotProductAttentionCell(
(_dropout_layer): Dropout(p = 0.1, axes=())
)
(proj_value): Dense(None -> 512, linear)
)
(ffn): PositionwiseFFN(
(dropout_layer): Dropout(p = 0.1, axes=())
(ffn_1): Dense(None -> 2048, Activation(relu))
(ffn_2): Dense(None -> 512, linear)
(layer_norm): LayerNorm(axis=-1, center=True, eps=1e-05, scale=True, in_channels=0)
)
(layer_norm): LayerNorm(axis=-1, center=True, eps=1e-05, scale=True, in_channels=0)
)
(2): TransformerEncoderCell(
(dropout_layer): Dropout(p = 0.1, axes=())
(proj): Dense(None -> 512, linear)
(attention_cell): MultiHeadAttentionCell(
(proj_query): Dense(None -> 512, linear)
(proj_key): Dense(None -> 512, linear)
(_base_cell): DotProductAttentionCell(
(_dropout_layer): Dropout(p = 0.1, axes=())
)
(proj_value): Dense(None -> 512, linear)
)
(ffn): PositionwiseFFN(
(dropout_layer): Dropout(p = 0.1, axes=())
(ffn_1): Dense(None -> 2048, Activation(relu))
(ffn_2): Dense(None -> 512, linear)
(layer_norm): LayerNorm(axis=-1, center=True, eps=1e-05, scale=True, in_channels=0)
)
(layer_norm): LayerNorm(axis=-1, center=True, eps=1e-05, scale=True, in_channels=0)
)
(3): TransformerEncoderCell(
(dropout_layer): Dropout(p = 0.1, axes=())
(proj): Dense(None -> 512, linear)
(attention_cell): MultiHeadAttentionCell(
(proj_query): Dense(None -> 512, linear)
(proj_key): Dense(None -> 512, linear)
(_base_cell): DotProductAttentionCell(
(_dropout_layer): Dropout(p = 0.1, axes=())
)
(proj_value): Dense(None -> 512, linear)
)
(ffn): PositionwiseFFN(
(dropout_layer): Dropout(p = 0.1, axes=())
(ffn_1): Dense(None -> 2048, Activation(relu))
(ffn_2): Dense(None -> 512, linear)
(layer_norm): LayerNorm(axis=-1, center=True, eps=1e-05, scale=True, in_channels=0)
)
(layer_norm): LayerNorm(axis=-1, center=True, eps=1e-05, scale=True, in_channels=0)
)
(4): TransformerEncoderCell(
(dropout_layer): Dropout(p = 0.1, axes=())
(proj): Dense(None -> 512, linear)
(attention_cell): MultiHeadAttentionCell(
(proj_query): Dense(None -> 512, linear)
(proj_key): Dense(None -> 512, linear)
(_base_cell): DotProductAttentionCell(
(_dropout_layer): Dropout(p = 0.1, axes=())
)
(proj_value): Dense(None -> 512, linear)
)
(ffn): PositionwiseFFN(
(dropout_layer): Dropout(p = 0.1, axes=())
(ffn_1): Dense(None -> 2048, Activation(relu))
(ffn_2): Dense(None -> 512, linear)
(layer_norm): LayerNorm(axis=-1, center=True, eps=1e-05, scale=True, in_channels=0)
)
(layer_norm): LayerNorm(axis=-1, center=True, eps=1e-05, scale=True, in_channels=0)
)
(5): TransformerEncoderCell(
(dropout_layer): Dropout(p = 0.1, axes=())
(proj): Dense(None -> 512, linear)
(attention_cell): MultiHeadAttentionCell(
(proj_query): Dense(None -> 512, linear)
(proj_key): Dense(None -> 512, linear)
(_base_cell): DotProductAttentionCell(
(_dropout_layer): Dropout(p = 0.1, axes=())
)
(proj_value): Dense(None -> 512, linear)
)
(ffn): PositionwiseFFN(
(dropout_layer): Dropout(p = 0.1, axes=())
(ffn_1): Dense(None -> 2048, Activation(relu))
(ffn_2): Dense(None -> 512, linear)
(layer_norm): LayerNorm(axis=-1, center=True, eps=1e-05, scale=True, in_channels=0)
)
(layer_norm): LayerNorm(axis=-1, center=True, eps=1e-05, scale=True, in_channels=0)
)
)
(layer_norm): LayerNorm(axis=-1, center=True, eps=1e-05, scale=True, in_channels=0)
)
(src_embed): HybridSequential(
(0): Embedding(36794 -> 512, float32)
(1): Dropout(p = 0.0, axes=())
)
(tgt_proj): Dense(None -> 36794, linear)
(decoder): TransformerDecoder(
(dropout_layer): Dropout(p = 0.1, axes=())
(transformer_cells): HybridSequential(
(0): TransformerDecoderCell(
(layer_norm_in): LayerNorm(axis=-1, center=True, eps=1e-05, scale=True, in_channels=0)
(dropout_layer): Dropout(p = 0.1, axes=())
(attention_cell_in): MultiHeadAttentionCell(
(proj_query): Dense(None -> 512, linear)
(proj_key): Dense(None -> 512, linear)
(_base_cell): DotProductAttentionCell(
(_dropout_layer): Dropout(p = 0.1, axes=())
)
(proj_value): Dense(None -> 512, linear)
)
(layer_norm_inter): LayerNorm(axis=-1, center=True, eps=1e-05, scale=True, in_channels=0)
(proj_inter): Dense(None -> 512, linear)
(proj_in): Dense(None -> 512, linear)
(ffn): PositionwiseFFN(
(dropout_layer): Dropout(p = 0.1, axes=())
(ffn_1): Dense(None -> 2048, Activation(relu))
(ffn_2): Dense(None -> 512, linear)
(layer_norm): LayerNorm(axis=-1, center=True, eps=1e-05, scale=True, in_channels=0)
)
(attention_cell_inter): MultiHeadAttentionCell(
(proj_query): Dense(None -> 512, linear)
(proj_key): Dense(None -> 512, linear)
(_base_cell): DotProductAttentionCell(
(_dropout_layer): Dropout(p = 0.1, axes=())
)
(proj_value): Dense(None -> 512, linear)
)
)
(1): TransformerDecoderCell(
(layer_norm_in): LayerNorm(axis=-1, center=True, eps=1e-05, scale=True, in_channels=0)
(dropout_layer): Dropout(p = 0.1, axes=())
(attention_cell_in): MultiHeadAttentionCell(
(proj_query): Dense(None -> 512, linear)
(proj_key): Dense(None -> 512, linear)
(_base_cell): DotProductAttentionCell(
(_dropout_layer): Dropout(p = 0.1, axes=())
)
(proj_value): Dense(None -> 512, linear)
)
(layer_norm_inter): LayerNorm(axis=-1, center=True, eps=1e-05, scale=True, in_channels=0)
(proj_inter): Dense(None -> 512, linear)
(proj_in): Dense(None -> 512, linear)
(ffn): PositionwiseFFN(
(dropout_layer): Dropout(p = 0.1, axes=())
(ffn_1): Dense(None -> 2048, Activation(relu))
(ffn_2): Dense(None -> 512, linear)
(layer_norm): LayerNorm(axis=-1, center=True, eps=1e-05, scale=True, in_channels=0)
)
(attention_cell_inter): MultiHeadAttentionCell(
(proj_query): Dense(None -> 512, linear)
(proj_key): Dense(None -> 512, linear)
(_base_cell): DotProductAttentionCell(
(_dropout_layer): Dropout(p = 0.1, axes=())
)
(proj_value): Dense(None -> 512, linear)
)
)
(2): TransformerDecoderCell(
(layer_norm_in): LayerNorm(axis=-1, center=True, eps=1e-05, scale=True, in_channels=0)
(dropout_layer): Dropout(p = 0.1, axes=())
(attention_cell_in): MultiHeadAttentionCell(
(proj_query): Dense(None -> 512, linear)
(proj_key): Dense(None -> 512, linear)
(_base_cell): DotProductAttentionCell(
(_dropout_layer): Dropout(p = 0.1, axes=())
)
(proj_value): Dense(None -> 512, linear)
)
(layer_norm_inter): LayerNorm(axis=-1, center=True, eps=1e-05, scale=True, in_channels=0)
(proj_inter): Dense(None -> 512, linear)
(proj_in): Dense(None -> 512, linear)
(ffn): PositionwiseFFN(
(dropout_layer): Dropout(p = 0.1, axes=())
(ffn_1): Dense(None -> 2048, Activation(relu))
(ffn_2): Dense(None -> 512, linear)
(layer_norm): LayerNorm(axis=-1, center=True, eps=1e-05, scale=True, in_channels=0)
)
(attention_cell_inter): MultiHeadAttentionCell(
(proj_query): Dense(None -> 512, linear)
(proj_key): Dense(None -> 512, linear)
(_base_cell): DotProductAttentionCell(
(_dropout_layer): Dropout(p = 0.1, axes=())
)
(proj_value): Dense(None -> 512, linear)
)
)
(3): TransformerDecoderCell(
(layer_norm_in): LayerNorm(axis=-1, center=True, eps=1e-05, scale=True, in_channels=0)
(dropout_layer): Dropout(p = 0.1, axes=())
(attention_cell_in): MultiHeadAttentionCell(
(proj_query): Dense(None -> 512, linear)
(proj_key): Dense(None -> 512, linear)
(_base_cell): DotProductAttentionCell(
(_dropout_layer): Dropout(p = 0.1, axes=())
)
(proj_value): Dense(None -> 512, linear)
)
(layer_norm_inter): LayerNorm(axis=-1, center=True, eps=1e-05, scale=True, in_channels=0)
(proj_inter): Dense(None -> 512, linear)
(proj_in): Dense(None -> 512, linear)
(ffn): PositionwiseFFN(
(dropout_layer): Dropout(p = 0.1, axes=())
(ffn_1): Dense(None -> 2048, Activation(relu))
(ffn_2): Dense(None -> 512, linear)
(layer_norm): LayerNorm(axis=-1, center=True, eps=1e-05, scale=True, in_channels=0)
)
(attention_cell_inter): MultiHeadAttentionCell(
(proj_query): Dense(None -> 512, linear)
(proj_key): Dense(None -> 512, linear)
(_base_cell): DotProductAttentionCell(
(_dropout_layer): Dropout(p = 0.1, axes=())
)
(proj_value): Dense(None -> 512, linear)
)
)
(4): TransformerDecoderCell(
(layer_norm_in): LayerNorm(axis=-1, center=True, eps=1e-05, scale=True, in_channels=0)
(dropout_layer): Dropout(p = 0.1, axes=())
(attention_cell_in): MultiHeadAttentionCell(
(proj_query): Dense(None -> 512, linear)
(proj_key): Dense(None -> 512, linear)
(_base_cell): DotProductAttentionCell(
(_dropout_layer): Dropout(p = 0.1, axes=())
)
(proj_value): Dense(None -> 512, linear)
)
(layer_norm_inter): LayerNorm(axis=-1, center=True, eps=1e-05, scale=True, in_channels=0)
(proj_inter): Dense(None -> 512, linear)
(proj_in): Dense(None -> 512, linear)
(ffn): PositionwiseFFN(
(dropout_layer): Dropout(p = 0.1, axes=())
(ffn_1): Dense(None -> 2048, Activation(relu))
(ffn_2): Dense(None -> 512, linear)
(layer_norm): LayerNorm(axis=-1, center=True, eps=1e-05, scale=True, in_channels=0)
)
(attention_cell_inter): MultiHeadAttentionCell(
(proj_query): Dense(None -> 512, linear)
(proj_key): Dense(None -> 512, linear)
(_base_cell): DotProductAttentionCell(
(_dropout_layer): Dropout(p = 0.1, axes=())
)
(proj_value): Dense(None -> 512, linear)
)
)
(5): TransformerDecoderCell(
(layer_norm_in): LayerNorm(axis=-1, center=True, eps=1e-05, scale=True, in_channels=0)
(dropout_layer): Dropout(p = 0.1, axes=())
(attention_cell_in): MultiHeadAttentionCell(
(proj_query): Dense(None -> 512, linear)
(proj_key): Dense(None -> 512, linear)
(_base_cell): DotProductAttentionCell(
(_dropout_layer): Dropout(p = 0.1, axes=())
)
(proj_value): Dense(None -> 512, linear)
)
(layer_norm_inter): LayerNorm(axis=-1, center=True, eps=1e-05, scale=True, in_channels=0)
(proj_inter): Dense(None -> 512, linear)
(proj_in): Dense(None -> 512, linear)
(ffn): PositionwiseFFN(
(dropout_layer): Dropout(p = 0.1, axes=())
(ffn_1): Dense(None -> 2048, Activation(relu))
(ffn_2): Dense(None -> 512, linear)
(layer_norm): LayerNorm(axis=-1, center=True, eps=1e-05, scale=True, in_channels=0)
)
(attention_cell_inter): MultiHeadAttentionCell(
(proj_query): Dense(None -> 512, linear)
(proj_key): Dense(None -> 512, linear)
(_base_cell): DotProductAttentionCell(
(_dropout_layer): Dropout(p = 0.1, axes=())
)
(proj_value): Dense(None -> 512, linear)
)
)
)
(layer_norm): LayerNorm(axis=-1, center=True, eps=1e-05, scale=True, in_channels=0)
)
(tgt_embed): HybridSequential(
(0): Embedding(36794 -> 512, float32)
(1): Dropout(p = 0.0, axes=())
)
)
2018-06-25 19:19:12,106 - root - Use beam_size=4, alpha=0.6, K=5
2018-06-25 19:19:20,026 - root - Train Batch Sampler:
FixedBucketSampler:
sample_num=4493328, batch_num=34064
key=[(5, 6), (8, 9), (12, 14), (17, 19), (23, 25), (30, 32), (39, 41), (49, 51), (61, 64), (75, 79), (93, 97), (114, 119), (139, 145), (169, 176), (206, 214), (249, 259), (302, 313), (364, 378), (440, 456), (530, 549)]
cnt=[15338, 57067, 248232, 511919, 747010, 820181, 816207, 571262, 380721, 193794, 90070, 32452, 6865, 1445, 517, 171, 57, 11, 7, 2]
batch_size=[1013, 562, 375, 271, 204, 158, 124, 99, 81, 67, 55, 46, 40, 33, 29, 23, 18, 16, 16, 10]
2018-06-25 19:19:20,029 - root - Valid Batch Sampler:
FixedBucketSampler:
sample_num=3000, batch_num=370
key=[4, 5, 6, 7, 8, 10, 12, 15, 18, 21, 26, 31, 37, 45, 54, 65, 78, 94, 112, 135]
cnt=[13, 12, 14, 24, 53, 110, 141, 254, 253, 275, 411, 377, 321, 328, 207, 109, 61, 24, 7, 6]
batch_size=[67, 51, 42, 36, 32, 27, 22, 18, 15, 12, 10, 8, 7, 6, 5, 4, 3, 3, 2, 2]
2018-06-25 19:19:20,031 - root - Test Batch Sampler:
FixedBucketSampler:
sample_num=2737, batch_num=367
key=[7, 8, 9, 10, 11, 13, 14, 16, 19, 22, 26, 30, 36, 42, 50, 59, 71, 84, 100]
cnt=[13, 18, 29, 41, 39, 123, 81, 147, 239, 233, 324, 285, 392, 271, 206, 181, 87, 23, 5]
batch_size=[38, 32, 28, 25, 23, 20, 18, 16, 14, 12, 10, 8, 7, 6, 5, 4, 3, 3, 2]
2018-06-25 19:21:20,799 - root - [Epoch 0 Batch 800/34064] loss=10.3670, ppl=31793.0480, throughput=54.01K wps, wc=6521.39K
2018-06-25 19:23:08,412 - root - [Epoch 0 Batch 1600/34064] loss=9.9823, ppl=21639.1107, throughput=60.65K wps, wc=6526.33K
2018-06-25 19:24:54,481 - root - [Epoch 0 Batch 2400/34064] loss=9.4362, ppl=12534.1929, throughput=61.48K wps, wc=6520.68K
2018-06-25 19:26:41,120 - root - [Epoch 0 Batch 3200/34064] loss=8.8150, ppl=6734.5995, throughput=61.19K wps, wc=6524.98K
2018-06-25 19:28:27,000 - root - [Epoch 0 Batch 4000/34064] loss=8.3174, ppl=4094.5768, throughput=61.54K wps, wc=6515.48K
2018-06-25 19:30:14,140 - root - [Epoch 0 Batch 4800/34064] loss=8.0589, ppl=3161.7813, throughput=60.92K wps, wc=6526.64K
2018-06-25 19:32:01,518 - root - [Epoch 0 Batch 5600/34064] loss=7.8119, ppl=2469.9338, throughput=60.65K wps, wc=6511.90K
2018-06-25 19:33:47,478 - root - [Epoch 0 Batch 6400/34064] loss=7.5964, ppl=1991.0614, throughput=61.62K wps, wc=6529.01K
2018-06-25 19:35:33,410 - root - [Epoch 0 Batch 7200/34064] loss=7.4043, ppl=1642.9836, throughput=61.53K wps, wc=6517.51K
2018-06-25 19:37:19,545 - root - [Epoch 0 Batch 8000/34064] loss=7.2448, ppl=1400.8604, throughput=61.43K wps, wc=6519.60K
2018-06-25 19:39:05,883 - root - [Epoch 0 Batch 8800/34064] loss=7.1157, ppl=1231.1434, throughput=61.34K wps, wc=6522.45K
2018-06-25 19:40:51,860 - root - [Epoch 0 Batch 9600/34064] loss=7.0056, ppl=1102.7859, throughput=61.51K wps, wc=6518.89K
2018-06-25 19:42:38,850 - root - [Epoch 0 Batch 10400/34064] loss=6.8863, ppl=978.7619, throughput=60.97K wps, wc=6523.04K
2018-06-25 19:44:25,572 - root - [Epoch 0 Batch 11200/34064] loss=6.7758, ppl=876.3566, throughput=61.11K wps, wc=6521.74K
2018-06-25 19:46:13,841 - root - [Epoch 0 Batch 12000/34064] loss=6.6845, ppl=799.9049, throughput=60.26K wps, wc=6524.14K
2018-06-25 19:48:01,925 - root - [Epoch 0 Batch 12800/34064] loss=6.5846, ppl=723.8826, throughput=60.33K wps, wc=6520.16K
2018-06-25 19:49:50,553 - root - [Epoch 0 Batch 13600/34064] loss=6.5016, ppl=666.2175, throughput=59.96K wps, wc=6513.54K
2018-06-25 19:51:38,037 - root - [Epoch 0 Batch 14400/34064] loss=6.4134, ppl=609.9611, throughput=60.72K wps, wc=6526.57K
2018-06-25 19:53:26,008 - root - [Epoch 0 Batch 15200/34064] loss=6.3283, ppl=560.2173, throughput=60.40K wps, wc=6520.91K
2018-06-25 19:55:16,226 - root - [Epoch 0 Batch 16000/34064] loss=6.2392, ppl=512.4496, throughput=59.23K wps, wc=6527.94K
2018-06-25 19:57:05,095 - root - [Epoch 0 Batch 16800/34064] loss=6.1702, ppl=478.2667, throughput=59.88K wps, wc=6518.74K
2018-06-25 19:58:54,372 - root - [Epoch 0 Batch 17600/34064] loss=6.0995, ppl=445.6467, throughput=59.58K wps, wc=6510.67K
2018-06-25 20:00:41,816 - root - [Epoch 0 Batch 18400/34064] loss=6.0151, ppl=409.5769, throughput=60.72K wps, wc=6523.73K
2018-06-25 20:02:29,378 - root - [Epoch 0 Batch 19200/34064] loss=5.9392, ppl=379.6438, throughput=60.67K wps, wc=6525.99K
2018-06-25 20:04:17,016 - root - [Epoch 0 Batch 20000/34064] loss=5.8800, ppl=357.8255, throughput=60.61K wps, wc=6524.25K
2018-06-25 20:06:05,236 - root - [Epoch 0 Batch 20800/34064] loss=5.8281, ppl=339.7166, throughput=60.28K wps, wc=6523.63K
2018-06-25 20:07:51,849 - root - [Epoch 0 Batch 21600/34064] loss=5.7462, ppl=312.9855, throughput=61.11K wps, wc=6514.78K
2018-06-25 20:09:41,107 - root - [Epoch 0 Batch 22400/34064] loss=5.7037, ppl=299.9726, throughput=59.68K wps, wc=6520.85K
2018-06-25 20:11:29,604 - root - [Epoch 0 Batch 23200/34064] loss=5.6315, ppl=279.0762, throughput=60.07K wps, wc=6516.90K
2018-06-25 20:13:18,455 - root - [Epoch 0 Batch 24000/34064] loss=5.5815, ppl=265.4788, throughput=59.87K wps, wc=6516.35K
2018-06-25 20:15:05,955 - root - [Epoch 0 Batch 24800/34064] loss=5.5272, ppl=251.4360, throughput=60.69K wps, wc=6524.52K
2018-06-25 20:16:55,579 - root - [Epoch 0 Batch 25600/34064] loss=5.4724, ppl=238.0278, throughput=59.48K wps, wc=6520.32K
2018-06-25 20:18:45,271 - root - [Epoch 0 Batch 26400/34064] loss=5.3922, ppl=219.6777, throughput=59.44K wps, wc=6519.95K
2018-06-25 20:20:33,451 - root - [Epoch 0 Batch 27200/34064] loss=5.3460, ppl=209.7704, throughput=60.31K wps, wc=6524.05K
2018-06-25 20:22:20,574 - root - [Epoch 0 Batch 28000/34064] loss=5.2697, ppl=194.3564, throughput=60.89K wps, wc=6522.19K
2018-06-25 20:24:09,095 - root - [Epoch 0 Batch 28800/34064] loss=5.1937, ppl=180.1392, throughput=60.05K wps, wc=6516.06K
2018-06-25 20:25:56,445 - root - [Epoch 0 Batch 29600/34064] loss=5.1122, ppl=166.0363, throughput=60.79K wps, wc=6525.55K
2018-06-25 20:27:46,069 - root - [Epoch 0 Batch 30400/34064] loss=5.0683, ppl=158.9044, throughput=59.48K wps, wc=6520.53K
2018-06-25 20:29:33,863 - root - [Epoch 0 Batch 31200/34064] loss=4.9545, ppl=141.8073, throughput=60.51K wps, wc=6522.14K
2018-06-25 20:31:23,143 - root - [Epoch 0 Batch 32000/34064] loss=4.8761, ppl=131.1182, throughput=59.66K wps, wc=6519.83K
2018-06-25 20:33:10,235 - root - [Epoch 0 Batch 32800/34064] loss=4.7995, ppl=121.4494, throughput=60.87K wps, wc=6518.36K
2018-06-25 20:34:59,141 - root - [Epoch 0 Batch 33600/34064] loss=4.7318, ppl=113.5005, throughput=59.90K wps, wc=6523.04K
2018-06-25 20:42:47,764 - root - [Epoch 0] valid Loss=3.4307, valid ppl=30.8977, valid bleu=9.21
2018-06-25 20:49:29,136 - root - [Epoch 0] test Loss=3.5092, test ppl=33.4209, test bleu=7.18
2018-06-25 20:49:29,142 - root - Save best parameters to transformer_en_de_u512/valid_best.params
2018-06-25 20:51:23,281 - root - [Epoch 1 Batch 800/34064] loss=4.5984, ppl=99.3247, throughput=57.62K wps, wc=6517.40K
2018-06-25 20:53:10,987 - root - [Epoch 1 Batch 1600/34064] loss=4.5285, ppl=92.6176, throughput=60.58K wps, wc=6524.55K
2018-06-25 20:54:59,257 - root - [Epoch 1 Batch 2400/34064] loss=4.4722, ppl=87.5461, throughput=60.19K wps, wc=6516.80K
2018-06-25 20:56:47,306 - root - [Epoch 1 Batch 3200/34064] loss=4.3890, ppl=80.5578, throughput=60.37K wps, wc=6522.46K
2018-06-25 20:58:34,713 - root - [Epoch 1 Batch 4000/34064] loss=4.3505, ppl=77.5137, throughput=60.67K wps, wc=6516.63K
2018-06-25 21:00:23,724 - root - [Epoch 1 Batch 4800/34064] loss=4.3119, ppl=74.5804, throughput=59.79K wps, wc=6517.72K
2018-06-25 21:02:10,603 - root - [Epoch 1 Batch 5600/34064] loss=4.2435, ppl=69.6535, throughput=61.01K wps, wc=6520.25K
2018-06-25 21:03:58,870 - root - [Epoch 1 Batch 6400/34064] loss=4.2039, ppl=66.9493, throughput=60.20K wps, wc=6517.18K
2018-06-25 21:05:46,829 - root - [Epoch 1 Batch 7200/34064] loss=4.1532, ppl=63.6356, throughput=60.39K wps, wc=6520.04K
2018-06-25 21:07:35,451 - root - [Epoch 1 Batch 8000/34064] loss=4.1457, ppl=63.1611, throughput=60.01K wps, wc=6518.66K
2018-06-25 21:09:23,000 - root - [Epoch 1 Batch 8800/34064] loss=4.0981, ppl=60.2234, throughput=60.59K wps, wc=6515.87K
2018-06-25 21:11:10,058 - root - [Epoch 1 Batch 9600/34064] loss=4.0737, ppl=58.7752, throughput=60.93K wps, wc=6523.38K
2018-06-25 21:12:57,319 - root - [Epoch 1 Batch 10400/34064] loss=4.0513, ppl=57.4733, throughput=60.79K wps, wc=6520.75K
2018-06-25 21:14:44,501 - root - [Epoch 1 Batch 11200/34064] loss=4.0183, ppl=55.6087, throughput=60.85K wps, wc=6521.82K
2018-06-25 21:16:31,202 - root - [Epoch 1 Batch 12000/34064] loss=3.9935, ppl=54.2446, throughput=61.17K wps, wc=6526.77K
2018-06-25 21:18:20,323 - root - [Epoch 1 Batch 12800/34064] loss=3.9932, ppl=54.2261, throughput=59.78K wps, wc=6522.99K
2018-06-25 21:20:07,635 - root - [Epoch 1 Batch 13600/34064] loss=3.9356, ppl=51.1924, throughput=60.79K wps, wc=6523.38K
2018-06-25 21:21:56,312 - root - [Epoch 1 Batch 14400/34064] loss=3.9588, ppl=52.3948, throughput=60.03K wps, wc=6523.83K
2018-06-25 21:23:44,469 - root - [Epoch 1 Batch 15200/34064] loss=3.9206, ppl=50.4317, throughput=60.25K wps, wc=6516.86K
2018-06-25 21:25:33,437 - root - [Epoch 1 Batch 16000/34064] loss=3.9099, ppl=49.8960, throughput=59.81K wps, wc=6517.28K
2018-06-25 21:27:21,448 - root - [Epoch 1 Batch 16800/34064] loss=3.8941, ppl=49.1136, throughput=60.39K wps, wc=6522.50K
2018-06-25 21:29:10,007 - root - [Epoch 1 Batch 17600/34064] loss=3.8597, ppl=47.4495, throughput=60.08K wps, wc=6522.69K
2018-06-25 21:30:57,447 - root - [Epoch 1 Batch 18400/34064] loss=3.8444, ppl=46.7313, throughput=60.68K wps, wc=6519.57K
2018-06-25 21:32:45,005 - root - [Epoch 1 Batch 19200/34064] loss=3.8010, ppl=44.7453, throughput=60.71K wps, wc=6530.25K
2018-06-25 21:34:33,217 - root - [Epoch 1 Batch 20000/34064] loss=3.8197, ppl=45.5921, throughput=60.23K wps, wc=6517.08K
2018-06-25 21:36:20,967 - root - [Epoch 1 Batch 20800/34064] loss=3.8014, ppl=44.7630, throughput=60.53K wps, wc=6521.58K
2018-06-25 21:38:09,232 - root - [Epoch 1 Batch 21600/34064] loss=3.7890, ppl=44.2104, throughput=60.18K wps, wc=6514.99K
2018-06-25 21:39:57,234 - root - [Epoch 1 Batch 22400/34064] loss=3.7885, ppl=44.1902, throughput=60.42K wps, wc=6525.71K
2018-06-25 21:41:45,570 - root - [Epoch 1 Batch 23200/34064] loss=3.7784, ppl=43.7479, throughput=60.19K wps, wc=6521.05K
2018-06-25 21:43:34,050 - root - [Epoch 1 Batch 24000/34064] loss=3.7611, ppl=42.9943, throughput=60.04K wps, wc=6513.54K
2018-06-25 21:45:23,761 - root - [Epoch 1 Batch 24800/34064] loss=3.7476, ppl=42.4205, throughput=59.36K wps, wc=6512.78K
2018-06-25 21:47:12,398 - root - [Epoch 1 Batch 25600/34064] loss=3.7423, ppl=42.1948, throughput=60.01K wps, wc=6519.07K
2018-06-25 21:49:00,325 - root - [Epoch 1 Batch 26400/34064] loss=3.7181, ppl=41.1845, throughput=60.46K wps, wc=6525.01K
2018-06-25 21:50:49,551 - root - [Epoch 1 Batch 27200/34064] loss=3.7533, ppl=42.6609, throughput=59.71K wps, wc=6521.88K
2018-06-25 21:52:36,543 - root - [Epoch 1 Batch 28000/34064] loss=3.6884, ppl=39.9798, throughput=61.00K wps, wc=6526.58K
2018-06-25 21:54:23,603 - root - [Epoch 1 Batch 28800/34064] loss=3.7046, ppl=40.6354, throughput=60.97K wps, wc=6527.25K
2018-06-25 21:56:10,818 - root - [Epoch 1 Batch 29600/34064] loss=3.6827, ppl=39.7541, throughput=60.85K wps, wc=6523.91K
2018-06-25 21:57:58,695 - root - [Epoch 1 Batch 30400/34064] loss=3.6701, ppl=39.2566, throughput=60.50K wps, wc=6526.25K
2018-06-25 21:59:45,718 - root - [Epoch 1 Batch 31200/34064] loss=3.6671, ppl=39.1378, throughput=60.95K wps, wc=6522.89K
2018-06-25 22:01:34,201 - root - [Epoch 1 Batch 32000/34064] loss=3.6485, ppl=38.4167, throughput=60.16K wps, wc=6526.23K
2018-06-25 22:03:23,027 - root - [Epoch 1 Batch 32800/34064] loss=3.6510, ppl=38.5150, throughput=59.93K wps, wc=6521.98K
2018-06-25 22:05:11,157 - root - [Epoch 1 Batch 33600/34064] loss=3.6426, ppl=38.1897, throughput=60.28K wps, wc=6517.54K
2018-06-25 22:10:49,646 - root - [Epoch 1] valid Loss=2.1273, valid ppl=8.3925, valid bleu=20.74
2018-06-25 22:15:23,946 - root - [Epoch 1] test Loss=2.0115, test ppl=7.4748, test bleu=19.39
2018-06-25 22:15:23,952 - root - Save best parameters to transformer_en_de_u512/valid_best.params
2018-06-25 22:17:19,727 - root - [Epoch 2 Batch 800/34064] loss=3.6006, ppl=36.6194, throughput=57.50K wps, wc=6521.93K
2018-06-25 22:19:10,318 - root - [Epoch 2 Batch 1600/34064] loss=3.6253, ppl=37.5365, throughput=58.96K wps, wc=6520.45K
2018-06-25 22:20:58,564 - root - [Epoch 2 Batch 2400/34064] loss=3.5783, ppl=35.8131, throughput=60.25K wps, wc=6522.27K
2018-06-25 22:22:47,001 - root - [Epoch 2 Batch 3200/34064] loss=3.5777, ppl=35.7907, throughput=60.13K wps, wc=6520.69K
2018-06-25 22:24:33,821 - root - [Epoch 2 Batch 4000/34064] loss=3.5531, ppl=34.9201, throughput=61.06K wps, wc=6521.89K
2018-06-25 22:26:20,149 - root - [Epoch 2 Batch 4800/34064] loss=3.5567, ppl=35.0456, throughput=61.32K wps, wc=6519.64K
2018-06-25 22:28:07,521 - root - [Epoch 2 Batch 5600/34064] loss=3.5570, ppl=35.0563, throughput=60.75K wps, wc=6522.92K
2018-06-25 22:29:56,113 - root - [Epoch 2 Batch 6400/34064] loss=3.5605, ppl=35.1815, throughput=60.03K wps, wc=6519.26K
2018-06-25 22:31:44,930 - root - [Epoch 2 Batch 7200/34064] loss=3.5671, ppl=35.4122, throughput=59.82K wps, wc=6509.45K
2018-06-25 22:33:32,423 - root - [Epoch 2 Batch 8000/34064] loss=3.5270, ppl=34.0232, throughput=60.65K wps, wc=6519.56K
2018-06-25 22:35:20,346 - root - [Epoch 2 Batch 8800/34064] loss=3.5450, ppl=34.6393, throughput=60.44K wps, wc=6522.36K
2018-06-25 22:37:06,863 - root - [Epoch 2 Batch 9600/34064] loss=3.5254, ppl=33.9658, throughput=61.30K wps, wc=6529.02K
2018-06-25 22:38:53,690 - root - [Epoch 2 Batch 10400/34064] loss=3.5301, ppl=34.1262, throughput=61.02K wps, wc=6518.21K
2018-06-25 22:40:42,483 - root - [Epoch 2 Batch 11200/34064] loss=3.5200, ppl=33.7844, throughput=59.96K wps, wc=6523.47K
2018-06-25 22:42:31,217 - root - [Epoch 2 Batch 12000/34064] loss=3.5168, ppl=33.6766, throughput=59.98K wps, wc=6522.17K
2018-06-25 22:44:17,852 - root - [Epoch 2 Batch 12800/34064] loss=3.5214, ppl=33.8323, throughput=61.15K wps, wc=6520.97K
2018-06-25 22:46:05,399 - root - [Epoch 2 Batch 13600/34064] loss=3.5037, ppl=33.2398, throughput=60.62K wps, wc=6519.23K
2018-06-25 22:47:54,324 - root - [Epoch 2 Batch 14400/34064] loss=3.4886, ppl=32.7393, throughput=59.84K wps, wc=6517.54K
2018-06-25 22:49:42,855 - root - [Epoch 2 Batch 15200/34064] loss=3.4997, ppl=33.1069, throughput=60.01K wps, wc=6512.35K
2018-06-25 22:51:31,631 - root - [Epoch 2 Batch 16000/34064] loss=3.4820, ppl=32.5256, throughput=59.97K wps, wc=6522.82K
2018-06-25 22:53:19,357 - root - [Epoch 2 Batch 16800/34064] loss=3.4707, ppl=32.1585, throughput=60.57K wps, wc=6525.40K
2018-06-25 22:55:06,277 - root - [Epoch 2 Batch 17600/34064] loss=3.4545, ppl=31.6418, throughput=61.00K wps, wc=6521.61K
2018-06-25 22:56:54,285 - root - [Epoch 2 Batch 18400/34064] loss=3.4812, ppl=32.5002, throughput=60.42K wps, wc=6525.48K
2018-06-25 22:58:42,005 - root - [Epoch 2 Batch 19200/34064] loss=3.4610, ppl=31.8481, throughput=60.56K wps, wc=6523.30K
2018-06-25 23:00:31,442 - root - [Epoch 2 Batch 20000/34064] loss=3.4721, ppl=32.2044, throughput=59.64K wps, wc=6526.67K
2018-06-25 23:02:18,910 - root - [Epoch 2 Batch 20800/34064] loss=3.4467, ppl=31.3957, throughput=60.70K wps, wc=6523.69K
2018-06-25 23:04:07,140 - root - [Epoch 2 Batch 21600/34064] loss=3.4490, ppl=31.4699, throughput=60.28K wps, wc=6523.82K
2018-06-25 23:05:54,560 - root - [Epoch 2 Batch 22400/34064] loss=3.4487, ppl=31.4585, throughput=60.67K wps, wc=6517.02K
2018-06-25 23:07:41,808 - root - [Epoch 2 Batch 23200/34064] loss=3.4401, ppl=31.1891, throughput=60.85K wps, wc=6526.26K
2018-06-25 23:09:29,603 - root - [Epoch 2 Batch 24000/34064] loss=3.4364, ppl=31.0756, throughput=60.46K wps, wc=6517.19K
2018-06-25 23:11:16,711 - root - [Epoch 2 Batch 24800/34064] loss=3.4315, ppl=30.9216, throughput=60.82K wps, wc=6514.21K
2018-06-25 23:13:02,997 - root - [Epoch 2 Batch 25600/34064] loss=3.4353, ppl=31.0416, throughput=61.34K wps, wc=6519.26K
2018-06-25 23:14:51,756 - root - [Epoch 2 Batch 26400/34064] loss=3.4508, ppl=31.5258, throughput=59.97K wps, wc=6522.71K
2018-06-25 23:16:38,782 - root - [Epoch 2 Batch 27200/34064] loss=3.4188, ppl=30.5315, throughput=60.98K wps, wc=6526.88K
2018-06-25 23:18:26,627 - root - [Epoch 2 Batch 28000/34064] loss=3.4185, ppl=30.5228, throughput=60.45K wps, wc=6519.26K
2018-06-25 23:20:13,873 - root - [Epoch 2 Batch 28800/34064] loss=3.4168, ppl=30.4705, throughput=60.83K wps, wc=6524.19K
2018-06-25 23:22:01,374 - root - [Epoch 2 Batch 29600/34064] loss=3.4100, ppl=30.2664, throughput=60.66K wps, wc=6521.02K
2018-06-25 23:23:48,509 - root - [Epoch 2 Batch 30400/34064] loss=3.4155, ppl=30.4330, throughput=60.83K wps, wc=6516.85K
2018-06-25 23:25:34,894 - root - [Epoch 2 Batch 31200/34064] loss=3.4130, ppl=30.3570, throughput=61.29K wps, wc=6520.12K
2018-06-25 23:27:22,302 - root - [Epoch 2 Batch 32000/34064] loss=3.4017, ppl=30.0154, throughput=60.74K wps, wc=6523.60K
2018-06-25 23:29:08,812 - root - [Epoch 2 Batch 32800/34064] loss=3.3953, ppl=29.8248, throughput=61.25K wps, wc=6523.61K
2018-06-25 23:30:56,561 - root - [Epoch 2 Batch 33600/34064] loss=3.4098, ppl=30.2591, throughput=60.50K wps, wc=6518.29K
2018-06-25 23:36:45,172 - root - [Epoch 2] valid Loss=1.8743, valid ppl=6.5162, valid bleu=22.39
2018-06-25 23:41:17,226 - root - [Epoch 2] test Loss=1.7349, test ppl=5.6682, test bleu=21.76
2018-06-25 23:41:17,232 - root - Save best parameters to transformer_en_de_u512/valid_best.params
2018-06-25 23:43:11,573 - root - [Epoch 3 Batch 800/34064] loss=3.3510, ppl=28.5321, throughput=58.23K wps, wc=6523.88K
2018-06-25 23:44:59,404 - root - [Epoch 3 Batch 1600/34064] loss=3.3567, ppl=28.6938, throughput=60.52K wps, wc=6525.81K
2018-06-25 23:46:46,526 - root - [Epoch 3 Batch 2400/34064] loss=3.3494, ppl=28.4860, throughput=60.82K wps, wc=6514.74K
2018-06-25 23:48:34,283 - root - [Epoch 3 Batch 3200/34064] loss=3.3452, ppl=28.3670, throughput=60.48K wps, wc=6517.07K
2018-06-25 23:50:22,579 - root - [Epoch 3 Batch 4000/34064] loss=3.3619, ppl=28.8428, throughput=60.24K wps, wc=6523.22K
2018-06-25 23:52:10,541 - root - [Epoch 3 Batch 4800/34064] loss=3.3669, ppl=28.9889, throughput=60.38K wps, wc=6519.15K
2018-06-25 23:53:59,063 - root - [Epoch 3 Batch 5600/34064] loss=3.3653, ppl=28.9429, throughput=60.06K wps, wc=6518.28K
2018-06-25 23:55:46,811 - root - [Epoch 3 Batch 6400/34064] loss=3.3553, ppl=28.6546, throughput=60.53K wps, wc=6522.01K
2018-06-25 23:57:33,463 - root - [Epoch 3 Batch 7200/34064] loss=3.3442, ppl=28.3366, throughput=61.15K wps, wc=6521.29K
2018-06-25 23:59:20,045 - root - [Epoch 3 Batch 8000/34064] loss=3.3349, ppl=28.0747, throughput=61.21K wps, wc=6524.06K
2018-06-26 00:01:07,556 - root - [Epoch 3 Batch 8800/34064] loss=3.3554, ppl=28.6579, throughput=60.60K wps, wc=6515.19K
2018-06-26 00:02:54,382 - root - [Epoch 3 Batch 9600/34064] loss=3.3490, ppl=28.4738, throughput=60.99K wps, wc=6514.88K
2018-06-26 00:04:41,978 - root - [Epoch 3 Batch 10400/34064] loss=3.3434, ppl=28.3141, throughput=60.58K wps, wc=6518.23K
2018-06-26 00:06:28,397 - root - [Epoch 3 Batch 11200/34064] loss=3.3301, ppl=27.9412, throughput=61.31K wps, wc=6524.21K
2018-06-26 00:08:14,909 - root - [Epoch 3 Batch 12000/34064] loss=3.3460, ppl=28.3882, throughput=61.22K wps, wc=6520.15K
2018-06-26 00:10:01,838 - root - [Epoch 3 Batch 12800/34064] loss=3.3374, ppl=28.1457, throughput=61.00K wps, wc=6522.80K
2018-06-26 00:11:48,336 - root - [Epoch 3 Batch 13600/34064] loss=3.3249, ppl=27.7960, throughput=61.17K wps, wc=6514.66K
2018-06-26 00:13:35,518 - root - [Epoch 3 Batch 14400/34064] loss=3.3203, ppl=27.6699, throughput=60.87K wps, wc=6524.29K
2018-06-26 00:15:22,873 - root - [Epoch 3 Batch 15200/34064] loss=3.3286, ppl=27.9002, throughput=60.76K wps, wc=6523.33K
2018-06-26 00:17:10,483 - root - [Epoch 3 Batch 16000/34064] loss=3.3259, ppl=27.8229, throughput=60.60K wps, wc=6521.44K
2018-06-26 00:18:59,664 - root - [Epoch 3 Batch 16800/34064] loss=3.3292, ppl=27.9171, throughput=59.75K wps, wc=6523.26K
2018-06-26 00:20:46,643 - root - [Epoch 3 Batch 17600/34064] loss=3.3202, ppl=27.6661, throughput=61.00K wps, wc=6525.51K
2018-06-26 00:22:34,635 - root - [Epoch 3 Batch 18400/34064] loss=3.3176, ppl=27.5952, throughput=60.38K wps, wc=6520.48K
2018-06-26 00:24:22,664 - root - [Epoch 3 Batch 19200/34064] loss=3.3210, ppl=27.6881, throughput=60.37K wps, wc=6521.42K
2018-06-26 00:26:09,522 - root - [Epoch 3 Batch 20000/34064] loss=3.3120, ppl=27.4397, throughput=61.06K wps, wc=6525.09K
2018-06-26 00:27:57,348 - root - [Epoch 3 Batch 20800/34064] loss=3.3088, ppl=27.3513, throughput=60.46K wps, wc=6518.56K
2018-06-26 00:29:44,636 - root - [Epoch 3 Batch 21600/34064] loss=3.2864, ppl=26.7467, throughput=60.85K wps, wc=6528.66K
2018-06-26 00:31:33,093 - root - [Epoch 3 Batch 22400/34064] loss=3.3145, ppl=27.5076, throughput=60.15K wps, wc=6523.60K
2018-06-26 00:33:22,555 - root - [Epoch 3 Batch 23200/34064] loss=3.2904, ppl=26.8541, throughput=59.57K wps, wc=6520.83K
2018-06-26 00:35:10,536 - root - [Epoch 3 Batch 24000/34064] loss=3.3164, ppl=27.5621, throughput=60.35K wps, wc=6516.52K
2018-06-26 00:36:59,746 - root - [Epoch 3 Batch 24800/34064] loss=3.3168, ppl=27.5729, throughput=59.60K wps, wc=6509.35K
2018-06-26 00:38:47,592 - root - [Epoch 3 Batch 25600/34064] loss=3.2885, ppl=26.8023, throughput=60.50K wps, wc=6524.51K
2018-06-26 00:40:35,868 - root - [Epoch 3 Batch 26400/34064] loss=3.3137, ppl=27.4874, throughput=60.18K wps, wc=6516.21K
2018-06-26 00:42:24,293 - root - [Epoch 3 Batch 27200/34064] loss=3.3172, ppl=27.5832, throughput=60.16K wps, wc=6522.44K
2018-06-26 00:44:12,436 - root - [Epoch 3 Batch 28000/34064] loss=3.3218, ppl=27.7090, throughput=60.31K wps, wc=6521.56K
2018-06-26 00:45:59,329 - root - [Epoch 3 Batch 28800/34064] loss=3.2926, ppl=26.9125, throughput=61.02K wps, wc=6522.57K
2018-06-26 00:47:47,871 - root - [Epoch 3 Batch 29600/34064] loss=3.3125, ppl=27.4545, throughput=60.12K wps, wc=6524.96K
2018-06-26 00:49:35,993 - root - [Epoch 3 Batch 30400/34064] loss=3.2917, ppl=26.8876, throughput=60.32K wps, wc=6521.41K
2018-06-26 00:51:24,126 - root - [Epoch 3 Batch 31200/34064] loss=3.3016, ppl=27.1572, throughput=60.28K wps, wc=6518.32K
2018-06-26 00:53:11,147 - root - [Epoch 3 Batch 32000/34064] loss=3.2873, ppl=26.7717, throughput=60.96K wps, wc=6523.69K
2018-06-26 00:54:57,549 - root - [Epoch 3 Batch 32800/34064] loss=3.2848, ppl=26.7048, throughput=61.32K wps, wc=6524.36K
2018-06-26 00:56:44,787 - root - [Epoch 3 Batch 33600/34064] loss=3.2867, ppl=26.7546, throughput=60.84K wps, wc=6524.69K
2018-06-26 01:02:22,714 - root - [Epoch 3] valid Loss=1.7584, valid ppl=5.8029, valid bleu=23.61
2018-06-26 01:06:58,445 - root - [Epoch 3] test Loss=1.6207, test ppl=5.0568, test bleu=22.66
2018-06-26 01:06:58,451 - root - Save best parameters to transformer_en_de_u512/valid_best.params
2018-06-26 01:08:51,459 - root - [Epoch 4 Batch 800/34064] loss=3.2387, ppl=25.4994, throughput=58.94K wps, wc=6528.01K
2018-06-26 01:10:38,503 - root - [Epoch 4 Batch 1600/34064] loss=3.2378, ppl=25.4779, throughput=60.87K wps, wc=6516.22K
2018-06-26 01:12:26,864 - root - [Epoch 4 Batch 2400/34064] loss=3.2640, ppl=26.1546, throughput=60.18K wps, wc=6521.61K
2018-06-26 01:14:14,956 - root - [Epoch 4 Batch 3200/34064] loss=3.2594, ppl=26.0344, throughput=60.33K wps, wc=6520.66K
2018-06-26 01:16:01,877 - root - [Epoch 4 Batch 4000/34064] loss=3.2450, ppl=25.6624, throughput=61.00K wps, wc=6521.98K
2018-06-26 01:17:50,208 - root - [Epoch 4 Batch 4800/34064] loss=3.2400, ppl=25.5339, throughput=60.20K wps, wc=6522.03K
2018-06-26 01:19:35,877 - root - [Epoch 4 Batch 5600/34064] loss=3.2446, ppl=25.6505, throughput=61.66K wps, wc=6515.02K
2018-06-26 01:21:23,296 - root - [Epoch 4 Batch 6400/34064] loss=3.2666, ppl=26.2215, throughput=60.62K wps, wc=6511.47K
2018-06-26 01:23:11,503 - root - [Epoch 4 Batch 7200/34064] loss=3.2351, ppl=25.4099, throughput=60.31K wps, wc=6525.41K
2018-06-26 01:25:00,383 - root - [Epoch 4 Batch 8000/34064] loss=3.2595, ppl=26.0367, throughput=59.92K wps, wc=6523.94K
2018-06-26 01:26:47,096 - root - [Epoch 4 Batch 8800/34064] loss=3.2406, ppl=25.5490, throughput=61.14K wps, wc=6524.24K
2018-06-26 01:28:34,321 - root - [Epoch 4 Batch 9600/34064] loss=3.2481, ppl=25.7425, throughput=60.82K wps, wc=6521.05K
2018-06-26 01:30:21,325 - root - [Epoch 4 Batch 10400/34064] loss=3.2373, ppl=25.4640, throughput=60.95K wps, wc=6522.23K
2018-06-26 01:32:08,577 - root - [Epoch 4 Batch 11200/34064] loss=3.2544, ppl=25.9048, throughput=60.80K wps, wc=6520.95K
2018-06-26 01:33:55,171 - root - [Epoch 4 Batch 12000/34064] loss=3.2594, ppl=26.0348, throughput=61.20K wps, wc=6523.34K
2018-06-26 01:35:42,880 - root - [Epoch 4 Batch 12800/34064] loss=3.2525, ppl=25.8545, throughput=60.46K wps, wc=6511.94K
2018-06-26 01:37:28,937 - root - [Epoch 4 Batch 13600/34064] loss=3.2300, ppl=25.2784, throughput=61.48K wps, wc=6520.53K
2018-06-26 01:39:16,376 - root - [Epoch 4 Batch 14400/34064] loss=3.2498, ppl=25.7842, throughput=60.73K wps, wc=6524.48K
2018-06-26 01:41:04,299 - root - [Epoch 4 Batch 15200/34064] loss=3.2501, ppl=25.7917, throughput=60.40K wps, wc=6518.81K
2018-06-26 01:42:52,107 - root - [Epoch 4 Batch 16000/34064] loss=3.2306, ppl=25.2958, throughput=60.53K wps, wc=6525.55K
2018-06-26 01:44:40,142 - root - [Epoch 4 Batch 16800/34064] loss=3.2421, ppl=25.5877, throughput=60.37K wps, wc=6522.11K
2018-06-26 01:46:28,648 - root - [Epoch 4 Batch 17600/34064] loss=3.2366, ppl=25.4471, throughput=60.08K wps, wc=6519.22K
2018-06-26 01:48:15,680 - root - [Epoch 4 Batch 18400/34064] loss=3.2320, ppl=25.3298, throughput=60.93K wps, wc=6521.51K
2018-06-26 01:50:03,286 - root - [Epoch 4 Batch 19200/34064] loss=3.2341, ppl=25.3830, throughput=60.62K wps, wc=6522.60K
2018-06-26 01:51:51,216 - root - [Epoch 4 Batch 20000/34064] loss=3.2448, ppl=25.6574, throughput=60.43K wps, wc=6522.44K
2018-06-26 01:53:38,421 - root - [Epoch 4 Batch 20800/34064] loss=3.2530, ppl=25.8678, throughput=60.82K wps, wc=6519.69K
2018-06-26 01:55:25,687 - root - [Epoch 4 Batch 21600/34064] loss=3.2338, ppl=25.3770, throughput=60.75K wps, wc=6515.88K
2018-06-26 01:57:12,773 - root - [Epoch 4 Batch 22400/34064] loss=3.2376, ppl=25.4715, throughput=60.90K wps, wc=6521.54K
2018-06-26 01:59:01,425 - root - [Epoch 4 Batch 23200/34064] loss=3.2370, ppl=25.4570, throughput=60.03K wps, wc=6522.80K
2018-06-26 02:00:49,580 - root - [Epoch 4 Batch 24000/34064] loss=3.2356, ppl=25.4210, throughput=60.32K wps, wc=6523.39K
2018-06-26 02:02:36,737 - root - [Epoch 4 Batch 24800/34064] loss=3.2302, ppl=25.2851, throughput=60.87K wps, wc=6523.11K
2018-06-26 02:04:24,864 - root - [Epoch 4 Batch 25600/34064] loss=3.2349, ppl=25.4043, throughput=60.36K wps, wc=6526.57K
2018-06-26 02:06:12,232 - root - [Epoch 4 Batch 26400/34064] loss=3.2432, ppl=25.6166, throughput=60.71K wps, wc=6518.66K
2018-06-26 02:07:59,109 - root - [Epoch 4 Batch 27200/34064] loss=3.2314, ppl=25.3145, throughput=60.96K wps, wc=6515.38K
2018-06-26 02:09:46,501 - root - [Epoch 4 Batch 28000/34064] loss=3.2229, ppl=25.1017, throughput=60.74K wps, wc=6523.44K
2018-06-26 02:11:33,607 - root - [Epoch 4 Batch 28800/34064] loss=3.2263, ppl=25.1873, throughput=60.94K wps, wc=6527.00K
2018-06-26 02:13:21,281 - root - [Epoch 4 Batch 29600/34064] loss=3.2172, ppl=24.9570, throughput=60.53K wps, wc=6517.25K
2018-06-26 02:15:09,885 - root - [Epoch 4 Batch 30400/34064] loss=3.2362, ppl=25.4378, throughput=60.08K wps, wc=6524.54K
2018-06-26 02:16:57,988 - root - [Epoch 4 Batch 31200/34064] loss=3.2406, ppl=25.5480, throughput=60.28K wps, wc=6516.17K
2018-06-26 02:18:45,785 - root - [Epoch 4 Batch 32000/34064] loss=3.2257, ppl=25.1715, throughput=60.45K wps, wc=6516.02K
2018-06-26 02:20:34,046 - root - [Epoch 4 Batch 32800/34064] loss=3.2321, ppl=25.3332, throughput=60.19K wps, wc=6515.95K
2018-06-26 02:22:20,854 - root - [Epoch 4 Batch 33600/34064] loss=3.2200, ppl=25.0284, throughput=61.13K wps, wc=6528.62K
2018-06-26 02:27:57,924 - root - [Epoch 4] valid Loss=1.7029, valid ppl=5.4896, valid bleu=24.03
2018-06-26 02:32:34,801 - root - [Epoch 4] test Loss=1.5553, test ppl=4.7363, test bleu=23.74
2018-06-26 02:32:34,807 - root - Save best parameters to transformer_en_de_u512/valid_best.params
2018-06-26 02:34:27,720 - root - [Epoch 5 Batch 800/34064] loss=3.1798, ppl=24.0408, throughput=58.92K wps, wc=6519.59K
2018-06-26 02:36:14,875 - root - [Epoch 5 Batch 1600/34064] loss=3.1744, ppl=23.9119, throughput=60.88K wps, wc=6522.97K
2018-06-26 02:38:01,179 - root - [Epoch 5 Batch 2400/34064] loss=3.1838, ppl=24.1390, throughput=61.38K wps, wc=6525.09K
2018-06-26 02:39:48,572 - root - [Epoch 5 Batch 3200/34064] loss=3.1949, ppl=24.4067, throughput=60.71K wps, wc=6519.47K
2018-06-26 02:41:37,090 - root - [Epoch 5 Batch 4000/34064] loss=3.2005, ppl=24.5460, throughput=60.08K wps, wc=6519.41K
2018-06-26 02:43:24,879 - root - [Epoch 5 Batch 4800/34064] loss=3.1810, ppl=24.0709, throughput=60.48K wps, wc=6519.36K
2018-06-26 02:45:13,072 - root - [Epoch 5 Batch 5600/34064] loss=3.1984, ppl=24.4945, throughput=60.25K wps, wc=6518.36K
2018-06-26 02:47:01,984 - root - [Epoch 5 Batch 6400/34064] loss=3.1924, ppl=24.3475, throughput=59.86K wps, wc=6519.26K
2018-06-26 02:48:48,745 - root - [Epoch 5 Batch 7200/34064] loss=3.1914, ppl=24.3226, throughput=61.11K wps, wc=6523.86K
2018-06-26 02:50:36,227 - root - [Epoch 5 Batch 8000/34064] loss=3.1766, ppl=23.9654, throughput=60.73K wps, wc=6526.93K
2018-06-26 02:52:23,996 - root - [Epoch 5 Batch 8800/34064] loss=3.2034, ppl=24.6156, throughput=60.49K wps, wc=6518.35K
2018-06-26 02:54:09,982 - root - [Epoch 5 Batch 9600/34064] loss=3.1657, ppl=23.7058, throughput=61.56K wps, wc=6524.15K
2018-06-26 02:55:56,638 - root - [Epoch 5 Batch 10400/34064] loss=3.1952, ppl=24.4143, throughput=61.17K wps, wc=6524.07K
2018-06-26 02:57:44,634 - root - [Epoch 5 Batch 11200/34064] loss=3.1728, ppl=23.8749, throughput=60.35K wps, wc=6517.52K
2018-06-26 02:59:31,040 - root - [Epoch 5 Batch 12000/34064] loss=3.1910, ppl=24.3134, throughput=61.30K wps, wc=6522.68K
2018-06-26 03:01:17,916 - root - [Epoch 5 Batch 12800/34064] loss=3.2006, ppl=24.5482, throughput=60.99K wps, wc=6518.20K
2018-06-26 03:03:05,999 - root - [Epoch 5 Batch 13600/34064] loss=3.1916, ppl=24.3284, throughput=60.36K wps, wc=6523.32K
2018-06-26 03:04:52,699 - root - [Epoch 5 Batch 14400/34064] loss=3.1794, ppl=24.0314, throughput=61.17K wps, wc=6526.83K
2018-06-26 03:06:40,228 - root - [Epoch 5 Batch 15200/34064] loss=3.1899, ppl=24.2871, throughput=60.68K wps, wc=6524.30K
2018-06-26 03:08:27,847 - root - [Epoch 5 Batch 16000/34064] loss=3.1837, ppl=24.1359, throughput=60.55K wps, wc=6516.48K
2018-06-26 03:10:16,343 - root - [Epoch 5 Batch 16800/34064] loss=3.1817, ppl=24.0880, throughput=60.12K wps, wc=6522.28K
2018-06-26 03:12:03,333 - root - [Epoch 5 Batch 17600/34064] loss=3.1823, ppl=24.1015, throughput=60.99K wps, wc=6525.53K
2018-06-26 03:13:52,354 - root - [Epoch 5 Batch 18400/34064] loss=3.1850, ppl=24.1672, throughput=59.81K wps, wc=6520.20K
2018-06-26 03:15:41,855 - root - [Epoch 5 Batch 19200/34064] loss=3.1940, ppl=24.3846, throughput=59.59K wps, wc=6525.02K
2018-06-26 03:17:28,617 - root - [Epoch 5 Batch 20000/34064] loss=3.1876, ppl=24.2293, throughput=61.09K wps, wc=6522.02K
2018-06-26 03:19:15,871 - root - [Epoch 5 Batch 20800/34064] loss=3.1862, ppl=24.1975, throughput=60.83K wps, wc=6523.76K
2018-06-26 03:21:03,179 - root - [Epoch 5 Batch 21600/34064] loss=3.1833, ppl=24.1265, throughput=60.79K wps, wc=6522.73K
2018-06-26 03:22:51,552 - root - [Epoch 5 Batch 22400/34064] loss=3.1897, ppl=24.2804, throughput=60.19K wps, wc=6522.70K
2018-06-26 03:24:40,339 - root - [Epoch 5 Batch 23200/34064] loss=3.1835, ppl=24.1304, throughput=60.00K wps, wc=6527.66K
2018-06-26 03:26:27,768 - root - [Epoch 5 Batch 24000/34064] loss=3.1941, ppl=24.3880, throughput=60.72K wps, wc=6522.95K
2018-06-26 03:28:14,359 - root - [Epoch 5 Batch 24800/34064] loss=3.1773, ppl=23.9824, throughput=61.17K wps, wc=6520.59K
2018-06-26 03:30:02,371 - root - [Epoch 5 Batch 25600/34064] loss=3.1902, ppl=24.2930, throughput=60.24K wps, wc=6506.24K
2018-06-26 03:31:50,423 - root - [Epoch 5 Batch 26400/34064] loss=3.1789, ppl=24.0194, throughput=60.26K wps, wc=6510.99K
2018-06-26 03:33:37,271 - root - [Epoch 5 Batch 27200/34064] loss=3.1855, ppl=24.1793, throughput=61.02K wps, wc=6520.09K
2018-06-26 03:35:23,934 - root - [Epoch 5 Batch 28000/34064] loss=3.1867, ppl=24.2084, throughput=61.15K wps, wc=6522.88K
2018-06-26 03:37:12,230 - root - [Epoch 5 Batch 28800/34064] loss=3.1814, ppl=24.0798, throughput=60.23K wps, wc=6522.75K
2018-06-26 03:38:59,985 - root - [Epoch 5 Batch 29600/34064] loss=3.1901, ppl=24.2914, throughput=60.50K wps, wc=6519.28K
2018-06-26 03:40:49,803 - root - [Epoch 5 Batch 30400/34064] loss=3.1728, ppl=23.8731, throughput=59.36K wps, wc=6518.83K
2018-06-26 03:42:38,402 - root - [Epoch 5 Batch 31200/34064] loss=3.1723, ppl=23.8612, throughput=60.06K wps, wc=6522.40K
2018-06-26 03:44:27,255 - root - [Epoch 5 Batch 32000/34064] loss=3.1855, ppl=24.1806, throughput=59.90K wps, wc=6520.73K
2018-06-26 03:46:15,458 - root - [Epoch 5 Batch 32800/34064] loss=3.1825, ppl=24.1062, throughput=60.22K wps, wc=6515.52K
2018-06-26 03:48:03,911 - root - [Epoch 5 Batch 33600/34064] loss=3.1783, ppl=24.0069, throughput=60.15K wps, wc=6523.23K
2018-06-26 03:53:42,871 - root - [Epoch 5] valid Loss=1.6575, valid ppl=5.2460, valid bleu=24.37
2018-06-26 03:58:18,346 - root - [Epoch 5] test Loss=1.5064, test ppl=4.5103, test bleu=23.99
2018-06-26 03:58:18,352 - root - Save best parameters to transformer_en_de_u512/valid_best.params
2018-06-26 04:00:13,648 - root - [Epoch 6 Batch 800/34064] loss=3.1482, ppl=23.2944, throughput=57.68K wps, wc=6521.11K
2018-06-26 04:02:01,914 - root - [Epoch 6 Batch 1600/34064] loss=3.1434, ppl=23.1815, throughput=60.21K wps, wc=6518.49K
2018-06-26 04:03:49,157 - root - [Epoch 6 Batch 2400/34064] loss=3.1384, ppl=23.0659, throughput=60.78K wps, wc=6518.44K
2018-06-26 04:05:37,860 - root - [Epoch 6 Batch 3200/34064] loss=3.1542, ppl=23.4348, throughput=59.99K wps, wc=6521.23K
2018-06-26 04:07:25,583 - root - [Epoch 6 Batch 4000/34064] loss=3.1411, ppl=23.1289, throughput=60.52K wps, wc=6519.82K
2018-06-26 04:09:14,403 - root - [Epoch 6 Batch 4800/34064] loss=3.1492, ppl=23.3167, throughput=59.90K wps, wc=6518.81K
2018-06-26 04:11:02,900 - root - [Epoch 6 Batch 5600/34064] loss=3.1572, ppl=23.5053, throughput=60.10K wps, wc=6520.15K
2018-06-26 04:12:49,914 - root - [Epoch 6 Batch 6400/34064] loss=3.1411, ppl=23.1304, throughput=60.99K wps, wc=6526.64K
2018-06-26 04:14:36,019 - root - [Epoch 6 Batch 7200/34064] loss=3.1297, ppl=22.8676, throughput=61.47K wps, wc=6522.03K
2018-06-26 04:16:24,175 - root - [Epoch 6 Batch 8000/34064] loss=3.1511, ppl=23.3607, throughput=60.31K wps, wc=6522.95K
2018-06-26 04:18:11,561 - root - [Epoch 6 Batch 8800/34064] loss=3.1363, ppl=23.0188, throughput=60.76K wps, wc=6524.92K
2018-06-26 04:19:58,310 - root - [Epoch 6 Batch 9600/34064] loss=3.1455, ppl=23.2317, throughput=61.13K wps, wc=6525.12K
2018-06-26 04:21:47,642 - root - [Epoch 6 Batch 10400/34064] loss=3.1634, ppl=23.6503, throughput=59.62K wps, wc=6518.81K
2018-06-26 04:23:36,467 - root - [Epoch 6 Batch 11200/34064] loss=3.1583, ppl=23.5304, throughput=59.93K wps, wc=6521.90K
2018-06-26 04:25:23,669 - root - [Epoch 6 Batch 12000/34064] loss=3.1520, ppl=23.3818, throughput=60.83K wps, wc=6521.53K
2018-06-26 04:27:11,270 - root - [Epoch 6 Batch 12800/34064] loss=3.1408, ppl=23.1228, throughput=60.61K wps, wc=6521.30K
2018-06-26 04:28:58,054 - root - [Epoch 6 Batch 13600/34064] loss=3.1475, ppl=23.2786, throughput=61.12K wps, wc=6526.40K
2018-06-26 04:30:46,484 - root - [Epoch 6 Batch 14400/34064] loss=3.1482, ppl=23.2935, throughput=60.20K wps, wc=6527.94K
2018-06-26 04:32:34,057 - root - [Epoch 6 Batch 15200/34064] loss=3.1716, ppl=23.8450, throughput=60.67K wps, wc=6526.63K
2018-06-26 04:34:22,465 - root - [Epoch 6 Batch 16000/34064] loss=3.1559, ppl=23.4737, throughput=60.15K wps, wc=6521.04K
2018-06-26 04:36:10,796 - root - [Epoch 6 Batch 16800/34064] loss=3.1708, ppl=23.8265, throughput=60.15K wps, wc=6516.20K
2018-06-26 04:37:57,239 - root - [Epoch 6 Batch 17600/34064] loss=3.1269, ppl=22.8037, throughput=61.31K wps, wc=6525.84K
2018-06-26 04:39:45,341 - root - [Epoch 6 Batch 18400/34064] loss=3.1594, ppl=23.5554, throughput=60.37K wps, wc=6526.45K
2018-06-26 04:41:31,970 - root - [Epoch 6 Batch 19200/34064] loss=3.1344, ppl=22.9741, throughput=61.13K wps, wc=6518.51K
2018-06-26 04:43:20,314 - root - [Epoch 6 Batch 20000/34064] loss=3.1505, ppl=23.3469, throughput=60.18K wps, wc=6520.25K
2018-06-26 04:45:07,402 - root - [Epoch 6 Batch 20800/34064] loss=3.1374, ppl=23.0431, throughput=60.92K wps, wc=6523.72K
2018-06-26 04:46:54,240 - root - [Epoch 6 Batch 21600/34064] loss=3.1457, ppl=23.2352, throughput=61.06K wps, wc=6523.37K
2018-06-26 04:48:42,555 - root - [Epoch 6 Batch 22400/34064] loss=3.1685, ppl=23.7729, throughput=60.13K wps, wc=6513.20K
2018-06-26 04:50:31,366 - root - [Epoch 6 Batch 23200/34064] loss=3.1519, ppl=23.3802, throughput=59.92K wps, wc=6519.96K
2018-06-26 04:52:19,940 - root - [Epoch 6 Batch 24000/34064] loss=3.1532, ppl=23.4101, throughput=60.07K wps, wc=6521.59K
2018-06-26 04:54:07,289 - root - [Epoch 6 Batch 24800/34064] loss=3.1436, ppl=23.1879, throughput=60.76K wps, wc=6522.78K
2018-06-26 04:55:55,833 - root - [Epoch 6 Batch 25600/34064] loss=3.1558, ppl=23.4718, throughput=59.98K wps, wc=6510.85K
2018-06-26 04:57:43,444 - root - [Epoch 6 Batch 26400/34064] loss=3.1523, ppl=23.3904, throughput=60.54K wps, wc=6514.61K
2018-06-26 04:59:31,461 - root - [Epoch 6 Batch 27200/34064] loss=3.1453, ppl=23.2260, throughput=60.32K wps, wc=6515.40K
2018-06-26 05:01:18,732 - root - [Epoch 6 Batch 28000/34064] loss=3.1514, ppl=23.3691, throughput=60.74K wps, wc=6515.92K
2018-06-26 05:03:05,658 - root - [Epoch 6 Batch 28800/34064] loss=3.1445, ppl=23.2079, throughput=60.97K wps, wc=6519.04K
2018-06-26 05:04:51,655 - root - [Epoch 6 Batch 29600/34064] loss=3.1317, ppl=22.9137, throughput=61.53K wps, wc=6522.35K
2018-06-26 05:06:37,738 - root - [Epoch 6 Batch 30400/34064] loss=3.1229, ppl=22.7127, throughput=61.51K wps, wc=6524.81K
2018-06-26 05:08:24,791 - root - [Epoch 6 Batch 31200/34064] loss=3.1302, ppl=22.8776, throughput=60.95K wps, wc=6524.54K
2018-06-26 05:10:11,261 - root - [Epoch 6 Batch 32000/34064] loss=3.1434, ppl=23.1818, throughput=61.27K wps, wc=6523.79K
2018-06-26 05:11:59,312 - root - [Epoch 6 Batch 32800/34064] loss=3.1417, ppl=23.1423, throughput=60.34K wps, wc=6520.16K
2018-06-26 05:13:45,863 - root - [Epoch 6 Batch 33600/34064] loss=3.1277, ppl=22.8203, throughput=61.20K wps, wc=6520.54K
2018-06-26 05:19:19,303 - root - [Epoch 6] valid Loss=1.6335, valid ppl=5.1218, valid bleu=24.64
2018-06-26 05:23:54,166 - root - [Epoch 6] test Loss=1.4819, test ppl=4.4012, test bleu=24.40
2018-06-26 05:23:54,172 - root - Save best parameters to transformer_en_de_u512/valid_best.params
2018-06-26 05:25:49,917 - root - [Epoch 7 Batch 800/34064] loss=3.1070, ppl=22.3532, throughput=57.40K wps, wc=6516.39K
2018-06-26 05:27:37,807 - root - [Epoch 7 Batch 1600/34064] loss=3.1150, ppl=22.5338, throughput=60.46K wps, wc=6523.24K
2018-06-26 05:29:24,536 - root - [Epoch 7 Batch 2400/34064] loss=3.0973, ppl=22.1371, throughput=61.12K wps, wc=6523.06K
2018-06-26 05:31:11,999 - root - [Epoch 7 Batch 3200/34064] loss=3.1070, ppl=22.3538, throughput=60.67K wps, wc=6519.53K
2018-06-26 05:33:01,816 - root - [Epoch 7 Batch 4000/34064] loss=3.1234, ppl=22.7229, throughput=59.29K wps, wc=6510.80K
2018-06-26 05:34:49,224 - root - [Epoch 7 Batch 4800/34064] loss=3.1237, ppl=22.7297, throughput=60.74K wps, wc=6523.47K
2018-06-26 05:36:36,574 - root - [Epoch 7 Batch 5600/34064] loss=3.1126, ppl=22.4789, throughput=60.71K wps, wc=6517.45K
2018-06-26 05:38:24,596 - root - [Epoch 7 Batch 6400/34064] loss=3.1074, ppl=22.3636, throughput=60.38K wps, wc=6521.91K
2018-06-26 05:40:12,209 - root - [Epoch 7 Batch 7200/34064] loss=3.1130, ppl=22.4880, throughput=60.64K wps, wc=6525.49K
2018-06-26 05:41:59,460 - root - [Epoch 7 Batch 8000/34064] loss=3.1081, ppl=22.3789, throughput=60.84K wps, wc=6525.37K
2018-06-26 05:43:45,616 - root - [Epoch 7 Batch 8800/34064] loss=3.1197, ppl=22.6388, throughput=61.45K wps, wc=6523.46K
2018-06-26 05:45:34,010 - root - [Epoch 7 Batch 9600/34064] loss=3.1228, ppl=22.7101, throughput=60.19K wps, wc=6524.39K
2018-06-26 05:47:20,765 - root - [Epoch 7 Batch 10400/34064] loss=3.1140, ppl=22.5117, throughput=61.11K wps, wc=6523.65K
2018-06-26 05:49:06,332 - root - [Epoch 7 Batch 11200/34064] loss=3.1070, ppl=22.3535, throughput=61.83K wps, wc=6526.96K
2018-06-26 05:50:54,602 - root - [Epoch 7 Batch 12000/34064] loss=3.1185, ppl=22.6125, throughput=60.23K wps, wc=6521.52K
2018-06-26 05:52:42,115 - root - [Epoch 7 Batch 12800/34064] loss=3.1137, ppl=22.5037, throughput=60.70K wps, wc=6526.32K
2018-06-26 05:54:29,416 - root - [Epoch 7 Batch 13600/34064] loss=3.1316, ppl=22.9109, throughput=60.77K wps, wc=6520.67K
2018-06-26 05:56:16,311 - root - [Epoch 7 Batch 14400/34064] loss=3.1175, ppl=22.5894, throughput=61.00K wps, wc=6520.21K
2018-06-26 05:58:03,903 - root - [Epoch 7 Batch 15200/34064] loss=3.1396, ppl=23.0958, throughput=60.53K wps, wc=6512.18K
2018-06-26 05:59:52,440 - root - [Epoch 7 Batch 16000/34064] loss=3.1330, ppl=22.9422, throughput=60.04K wps, wc=6516.84K
2018-06-26 06:01:40,914 - root - [Epoch 7 Batch 16800/34064] loss=3.1174, ppl=22.5872, throughput=60.10K wps, wc=6519.51K
2018-06-26 06:03:27,671 - root - [Epoch 7 Batch 17600/34064] loss=3.1103, ppl=22.4271, throughput=61.14K wps, wc=6526.59K
2018-06-26 06:05:13,853 - root - [Epoch 7 Batch 18400/34064] loss=3.0981, ppl=22.1549, throughput=61.43K wps, wc=6522.91K
2018-06-26 06:07:01,756 - root - [Epoch 7 Batch 19200/34064] loss=3.1098, ppl=22.4173, throughput=60.46K wps, wc=6523.50K
2018-06-26 06:08:50,091 - root - [Epoch 7 Batch 20000/34064] loss=3.1230, ppl=22.7144, throughput=60.20K wps, wc=6521.76K
2018-06-26 06:10:37,049 - root - [Epoch 7 Batch 20800/34064] loss=3.1089, ppl=22.3973, throughput=60.97K wps, wc=6520.88K
2018-06-26 06:12:25,449 - root - [Epoch 7 Batch 21600/34064] loss=3.1276, ppl=22.8191, throughput=60.15K wps, wc=6520.13K
2018-06-26 06:14:13,254 - root - [Epoch 7 Batch 22400/34064] loss=3.1143, ppl=22.5186, throughput=60.48K wps, wc=6520.08K
2018-06-26 06:16:01,653 - root - [Epoch 7 Batch 23200/34064] loss=3.1025, ppl=22.2543, throughput=60.17K wps, wc=6522.66K
2018-06-26 06:17:48,990 - root - [Epoch 7 Batch 24000/34064] loss=3.1103, ppl=22.4286, throughput=60.74K wps, wc=6519.31K
2018-06-26 06:19:36,582 - root - [Epoch 7 Batch 24800/34064] loss=3.1195, ppl=22.6345, throughput=60.62K wps, wc=6522.26K
2018-06-26 06:21:25,308 - root - [Epoch 7 Batch 25600/34064] loss=3.1381, ppl=23.0602, throughput=59.82K wps, wc=6503.81K
2018-06-26 06:23:13,612 - root - [Epoch 7 Batch 26400/34064] loss=3.1288, ppl=22.8465, throughput=60.27K wps, wc=6527.27K
2018-06-26 06:25:01,652 - root - [Epoch 7 Batch 27200/34064] loss=3.1191, ppl=22.6266, throughput=60.40K wps, wc=6525.84K
2018-06-26 06:26:50,365 - root - [Epoch 7 Batch 28000/34064] loss=3.1240, ppl=22.7375, throughput=59.99K wps, wc=6522.06K
2018-06-26 06:28:36,363 - root - [Epoch 7 Batch 28800/34064] loss=3.0996, ppl=22.1892, throughput=61.57K wps, wc=6526.02K
2018-06-26 06:30:25,254 - root - [Epoch 7 Batch 29600/34064] loss=3.1208, ppl=22.6647, throughput=59.90K wps, wc=6522.81K
2018-06-26 06:32:13,002 - root - [Epoch 7 Batch 30400/34064] loss=3.1114, ppl=22.4532, throughput=60.50K wps, wc=6518.96K
2018-06-26 06:34:01,248 - root - [Epoch 7 Batch 31200/34064] loss=3.1347, ppl=22.9809, throughput=60.15K wps, wc=6510.57K
2018-06-26 06:35:48,761 - root - [Epoch 7 Batch 32000/34064] loss=3.1146, ppl=22.5249, throughput=60.69K wps, wc=6524.35K
2018-06-26 06:37:36,812 - root - [Epoch 7 Batch 32800/34064] loss=3.1281, ppl=22.8309, throughput=60.33K wps, wc=6519.06K
2018-06-26 06:39:23,675 - root - [Epoch 7 Batch 33600/34064] loss=3.1016, ppl=22.2341, throughput=61.07K wps, wc=6526.24K
2018-06-26 06:45:02,178 - root - [Epoch 7] valid Loss=1.6135, valid ppl=5.0202, valid bleu=24.95
2018-06-26 06:49:38,835 - root - [Epoch 7] test Loss=1.4517, test ppl=4.2703, test bleu=24.63
2018-06-26 06:49:38,841 - root - Save best parameters to transformer_en_de_u512/valid_best.params
2018-06-26 06:51:33,371 - root - [Epoch 8 Batch 800/34064] loss=3.0674, ppl=21.4855, throughput=57.97K wps, wc=6511.01K
2018-06-26 06:53:21,032 - root - [Epoch 8 Batch 1600/34064] loss=3.0741, ppl=21.6313, throughput=60.62K wps, wc=6526.74K
2018-06-26 06:55:09,607 - root - [Epoch 8 Batch 2400/34064] loss=3.0741, ppl=21.6312, throughput=60.07K wps, wc=6522.59K
2018-06-26 06:56:56,490 - root - [Epoch 8 Batch 3200/34064] loss=3.0787, ppl=21.7311, throughput=61.01K wps, wc=6521.25K
2018-06-26 06:58:43,561 - root - [Epoch 8 Batch 4000/34064] loss=3.0867, ppl=21.9050, throughput=60.92K wps, wc=6522.63K
2018-06-26 07:00:31,482 - root - [Epoch 8 Batch 4800/34064] loss=3.0898, ppl=21.9721, throughput=60.41K wps, wc=6519.94K
2018-06-26 07:02:21,366 - root - [Epoch 8 Batch 5600/34064] loss=3.1098, ppl=22.4156, throughput=59.36K wps, wc=6523.13K
2018-06-26 07:04:09,899 - root - [Epoch 8 Batch 6400/34064] loss=3.0984, ppl=22.1617, throughput=60.10K wps, wc=6522.22K
2018-06-26 07:05:56,834 - root - [Epoch 8 Batch 7200/34064] loss=3.0918, ppl=22.0162, throughput=61.00K wps, wc=6522.80K
2018-06-26 07:07:43,676 - root - [Epoch 8 Batch 8000/34064] loss=3.0937, ppl=22.0579, throughput=61.05K wps, wc=6522.21K
2018-06-26 07:09:31,180 - root - [Epoch 8 Batch 8800/34064] loss=3.0933, ppl=22.0492, throughput=60.62K wps, wc=6516.49K
2018-06-26 07:11:18,296 - root - [Epoch 8 Batch 9600/34064] loss=3.1040, ppl=22.2875, throughput=60.86K wps, wc=6518.81K
2018-06-26 07:13:05,752 - root - [Epoch 8 Batch 10400/34064] loss=3.0869, ppl=21.9098, throughput=60.72K wps, wc=6524.20K
2018-06-26 07:14:51,361 - root - [Epoch 8 Batch 11200/34064] loss=3.0745, ppl=21.6401, throughput=61.78K wps, wc=6524.39K
2018-06-26 07:16:39,374 - root - [Epoch 8 Batch 12000/34064] loss=3.0840, ppl=21.8458, throughput=60.33K wps, wc=6516.82K
2018-06-26 07:18:26,883 - root - [Epoch 8 Batch 12800/34064] loss=3.1032, ppl=22.2701, throughput=60.66K wps, wc=6521.45K
2018-06-26 07:20:14,440 - root - [Epoch 8 Batch 13600/34064] loss=3.0895, ppl=21.9667, throughput=60.63K wps, wc=6520.90K
2018-06-26 07:22:01,638 - root - [Epoch 8 Batch 14400/34064] loss=3.0966, ppl=22.1233, throughput=60.78K wps, wc=6515.88K
2018-06-26 07:23:50,861 - root - [Epoch 8 Batch 15200/34064] loss=3.1150, ppl=22.5340, throughput=59.69K wps, wc=6519.60K
2018-06-26 07:25:38,552 - root - [Epoch 8 Batch 16000/34064] loss=3.0960, ppl=22.1082, throughput=60.59K wps, wc=6524.65K
2018-06-26 07:27:25,275 - root - [Epoch 8 Batch 16800/34064] loss=3.0942, ppl=22.0706, throughput=61.14K wps, wc=6524.44K
2018-06-26 07:29:12,634 - root - [Epoch 8 Batch 17600/34064] loss=3.1006, ppl=22.2111, throughput=60.62K wps, wc=6508.39K
2018-06-26 07:31:00,671 - root - [Epoch 8 Batch 18400/34064] loss=3.0903, ppl=21.9839, throughput=60.40K wps, wc=6525.62K
2018-06-26 07:32:49,130 - root - [Epoch 8 Batch 19200/34064] loss=3.0974, ppl=22.1397, throughput=60.17K wps, wc=6525.80K
2018-06-26 07:34:37,203 - root - [Epoch 8 Batch 20000/34064] loss=3.0989, ppl=22.1735, throughput=60.38K wps, wc=6525.32K
2018-06-26 07:36:25,437 - root - [Epoch 8 Batch 20800/34064] loss=3.0986, ppl=22.1664, throughput=60.19K wps, wc=6515.07K
2018-06-26 07:38:13,359 - root - [Epoch 8 Batch 21600/34064] loss=3.0930, ppl=22.0432, throughput=60.47K wps, wc=6525.82K
2018-06-26 07:40:00,643 - root - [Epoch 8 Batch 22400/34064] loss=3.0806, ppl=21.7722, throughput=60.78K wps, wc=6520.87K
2018-06-26 07:41:47,953 - root - [Epoch 8 Batch 23200/34064] loss=3.1018, ppl=22.2377, throughput=60.72K wps, wc=6515.74K
2018-06-26 07:43:35,259 - root - [Epoch 8 Batch 24000/34064] loss=3.1040, ppl=22.2877, throughput=60.78K wps, wc=6522.03K
2018-06-26 07:45:23,556 - root - [Epoch 8 Batch 24800/34064] loss=3.0942, ppl=22.0691, throughput=60.19K wps, wc=6517.97K
2018-06-26 07:47:10,266 - root - [Epoch 8 Batch 25600/34064] loss=3.1074, ppl=22.3618, throughput=61.13K wps, wc=6523.27K
2018-06-26 07:48:57,530 - root - [Epoch 8 Batch 26400/34064] loss=3.0791, ppl=21.7390, throughput=60.76K wps, wc=6517.82K
2018-06-26 07:50:45,786 - root - [Epoch 8 Batch 27200/34064] loss=3.1109, ppl=22.4408, throughput=60.24K wps, wc=6521.37K
2018-06-26 07:52:33,468 - root - [Epoch 8 Batch 28000/34064] loss=3.0816, ppl=21.7931, throughput=60.57K wps, wc=6522.31K
2018-06-26 07:54:20,280 - root - [Epoch 8 Batch 28800/34064] loss=3.0853, ppl=21.8746, throughput=61.09K wps, wc=6524.99K
2018-06-26 07:56:07,598 - root - [Epoch 8 Batch 29600/34064] loss=3.0800, ppl=21.7590, throughput=60.68K wps, wc=6511.67K
2018-06-26 07:57:53,842 - root - [Epoch 8 Batch 30400/34064] loss=3.0990, ppl=22.1753, throughput=61.37K wps, wc=6519.67K
2018-06-26 07:59:41,641 - root - [Epoch 8 Batch 31200/34064] loss=3.0908, ppl=21.9955, throughput=60.54K wps, wc=6526.32K
2018-06-26 08:01:28,785 - root - [Epoch 8 Batch 32000/34064] loss=3.0985, ppl=22.1639, throughput=60.89K wps, wc=6523.59K
2018-06-26 08:03:15,565 - root - [Epoch 8 Batch 32800/34064] loss=3.0940, ppl=22.0649, throughput=61.11K wps, wc=6524.76K
2018-06-26 08:05:02,036 - root - [Epoch 8 Batch 33600/34064] loss=3.0892, ppl=21.9600, throughput=61.31K wps, wc=6528.07K
2018-06-26 08:10:40,544 - root - [Epoch 8] valid Loss=1.5918, valid ppl=4.9124, valid bleu=24.90
2018-06-26 08:15:18,612 - root - [Epoch 8] test Loss=1.4299, test ppl=4.1784, test bleu=24.95
2018-06-26 08:17:10,994 - root - [Epoch 9 Batch 800/34064] loss=3.0525, ppl=21.1686, throughput=58.26K wps, wc=6520.26K
2018-06-26 08:18:58,675 - root - [Epoch 9 Batch 1600/34064] loss=3.0574, ppl=21.2716, throughput=60.61K wps, wc=6526.02K
2018-06-26 08:20:46,682 - root - [Epoch 9 Batch 2400/34064] loss=3.0709, ppl=21.5615, throughput=60.38K wps, wc=6521.74K
2018-06-26 08:22:34,212 - root - [Epoch 9 Batch 3200/34064] loss=3.0731, ppl=21.6080, throughput=60.54K wps, wc=6510.29K
2018-06-26 08:24:23,295 - root - [Epoch 9 Batch 4000/34064] loss=3.0847, ppl=21.8619, throughput=59.78K wps, wc=6521.37K
2018-06-26 08:26:10,574 - root - [Epoch 9 Batch 4800/34064] loss=3.0699, ppl=21.5397, throughput=60.85K wps, wc=6527.36K
2018-06-26 08:27:56,563 - root - [Epoch 9 Batch 5600/34064] loss=3.0376, ppl=20.8546, throughput=61.48K wps, wc=6515.85K
2018-06-26 08:29:44,636 - root - [Epoch 9 Batch 6400/34064] loss=3.0700, ppl=21.5411, throughput=60.34K wps, wc=6521.59K
2018-06-26 08:31:32,791 - root - [Epoch 9 Batch 7200/34064] loss=3.0806, ppl=21.7714, throughput=60.34K wps, wc=6525.82K
2018-06-26 08:33:20,964 - root - [Epoch 9 Batch 8000/34064] loss=3.0730, ppl=21.6066, throughput=60.29K wps, wc=6521.68K
2018-06-26 08:35:07,567 - root - [Epoch 9 Batch 8800/34064] loss=3.0610, ppl=21.3479, throughput=61.09K wps, wc=6512.56K
2018-06-26 08:36:54,547 - root - [Epoch 9 Batch 9600/34064] loss=3.0558, ppl=21.2374, throughput=60.95K wps, wc=6520.23K
2018-06-26 08:38:41,286 - root - [Epoch 9 Batch 10400/34064] loss=3.0709, ppl=21.5622, throughput=61.10K wps, wc=6521.69K
2018-06-26 08:40:28,818 - root - [Epoch 9 Batch 11200/34064] loss=3.0763, ppl=21.6778, throughput=60.64K wps, wc=6520.62K
2018-06-26 08:42:15,757 - root - [Epoch 9 Batch 12000/34064] loss=3.0799, ppl=21.7571, throughput=60.94K wps, wc=6516.39K
2018-06-26 08:44:02,862 - root - [Epoch 9 Batch 12800/34064] loss=3.0627, ppl=21.3846, throughput=60.91K wps, wc=6523.68K
2018-06-26 08:45:51,794 - root - [Epoch 9 Batch 13600/34064] loss=3.0699, ppl=21.5404, throughput=59.89K wps, wc=6523.68K
2018-06-26 08:47:39,182 - root - [Epoch 9 Batch 14400/34064] loss=3.0843, ppl=21.8531, throughput=60.73K wps, wc=6521.18K
2018-06-26 08:49:27,115 - root - [Epoch 9 Batch 15200/34064] loss=3.0738, ppl=21.6233, throughput=60.37K wps, wc=6516.23K
2018-06-26 08:51:13,040 - root - [Epoch 9 Batch 16000/34064] loss=3.0709, ppl=21.5617, throughput=61.57K wps, wc=6522.04K
2018-06-26 08:53:01,168 - root - [Epoch 9 Batch 16800/34064] loss=3.0687, ppl=21.5146, throughput=60.34K wps, wc=6524.45K
2018-06-26 08:54:49,904 - root - [Epoch 9 Batch 17600/34064] loss=3.0799, ppl=21.7565, throughput=59.98K wps, wc=6522.43K
2018-06-26 08:56:37,588 - root - [Epoch 9 Batch 18400/34064] loss=3.0697, ppl=21.5359, throughput=60.60K wps, wc=6525.33K
2018-06-26 08:58:26,254 - root - [Epoch 9 Batch 19200/34064] loss=3.0697, ppl=21.5350, throughput=60.02K wps, wc=6522.61K
2018-06-26 09:00:13,943 - root - [Epoch 9 Batch 20000/34064] loss=3.0667, ppl=21.4718, throughput=60.57K wps, wc=6522.72K
2018-06-26 09:02:02,457 - root - [Epoch 9 Batch 20800/34064] loss=3.0816, ppl=21.7923, throughput=60.06K wps, wc=6517.72K
2018-06-26 09:03:50,232 - root - [Epoch 9 Batch 21600/34064] loss=3.0742, ppl=21.6324, throughput=60.55K wps, wc=6525.25K
2018-06-26 09:05:38,770 - root - [Epoch 9 Batch 22400/34064] loss=3.0766, ppl=21.6844, throughput=60.10K wps, wc=6522.62K
2018-06-26 09:07:26,471 - root - [Epoch 9 Batch 23200/34064] loss=3.0687, ppl=21.5133, throughput=60.58K wps, wc=6525.03K
2018-06-26 09:09:14,754 - root - [Epoch 9 Batch 24000/34064] loss=3.0800, ppl=21.7586, throughput=60.26K wps, wc=6525.14K
2018-06-26 09:11:03,774 - root - [Epoch 9 Batch 24800/34064] loss=3.0890, ppl=21.9548, throughput=59.78K wps, wc=6516.94K
2018-06-26 09:12:51,685 - root - [Epoch 9 Batch 25600/34064] loss=3.0738, ppl=21.6241, throughput=60.41K wps, wc=6518.78K
2018-06-26 09:14:40,610 - root - [Epoch 9 Batch 26400/34064] loss=3.0966, ppl=22.1216, throughput=59.87K wps, wc=6520.97K
2018-06-26 09:16:28,942 - root - [Epoch 9 Batch 27200/34064] loss=3.0716, ppl=21.5767, throughput=60.19K wps, wc=6520.65K
2018-06-26 09:18:17,088 - root - [Epoch 9 Batch 28000/34064] loss=3.0843, ppl=21.8522, throughput=60.32K wps, wc=6523.43K
2018-06-26 09:20:06,012 - root - [Epoch 9 Batch 28800/34064] loss=3.0872, ppl=21.9151, throughput=59.88K wps, wc=6522.63K
2018-06-26 09:21:53,668 - root - [Epoch 9 Batch 29600/34064] loss=3.0730, ppl=21.6062, throughput=60.62K wps, wc=6526.18K
2018-06-26 09:23:41,463 - root - [Epoch 9 Batch 30400/34064] loss=3.0715, ppl=21.5738, throughput=60.53K wps, wc=6524.44K
2018-06-26 09:25:28,427 - root - [Epoch 9 Batch 31200/34064] loss=3.0709, ppl=21.5603, throughput=60.84K wps, wc=6507.54K
2018-06-26 09:27:15,693 - root - [Epoch 9 Batch 32000/34064] loss=3.0731, ppl=21.6093, throughput=60.79K wps, wc=6520.52K
2018-06-26 09:29:03,472 - root - [Epoch 9 Batch 32800/34064] loss=3.0677, ppl=21.4922, throughput=60.50K wps, wc=6520.68K
2018-06-26 09:30:50,536 - root - [Epoch 9 Batch 33600/34064] loss=3.0752, ppl=21.6542, throughput=60.91K wps, wc=6521.06K
2018-06-26 09:36:25,273 - root - [Epoch 9] valid Loss=1.5766, valid ppl=4.8387, valid bleu=24.93
2018-06-26 09:41:00,731 - root - [Epoch 9] test Loss=1.4128, test ppl=4.1076, test bleu=25.12
2018-06-26 09:42:52,409 - root - [Epoch 10 Batch 800/34064] loss=3.0374, ppl=20.8518, throughput=58.71K wps, wc=6530.16K
2018-06-26 09:44:39,002 - root - [Epoch 10 Batch 1600/34064] loss=3.0368, ppl=20.8380, throughput=61.21K wps, wc=6524.36K
2018-06-26 09:46:27,147 - root - [Epoch 10 Batch 2400/34064] loss=3.0333, ppl=20.7662, throughput=60.31K wps, wc=6522.62K
2018-06-26 09:48:14,337 - root - [Epoch 10 Batch 3200/34064] loss=3.0550, ppl=21.2222, throughput=60.82K wps, wc=6519.09K
2018-06-26 09:50:02,333 - root - [Epoch 10 Batch 4000/34064] loss=3.0517, ppl=21.1520, throughput=60.29K wps, wc=6511.22K
2018-06-26 09:51:50,112 - root - [Epoch 10 Batch 4800/34064] loss=3.0318, ppl=20.7337, throughput=60.57K wps, wc=6528.01K
2018-06-26 09:53:37,854 - root - [Epoch 10 Batch 5600/34064] loss=3.0456, ppl=21.0224, throughput=60.56K wps, wc=6524.97K
2018-06-26 09:55:25,936 - root - [Epoch 10 Batch 6400/34064] loss=3.0694, ppl=21.5281, throughput=60.30K wps, wc=6517.71K
2018-06-26 09:57:12,414 - root - [Epoch 10 Batch 7200/34064] loss=3.0347, ppl=20.7941, throughput=61.33K wps, wc=6530.34K
2018-06-26 09:58:59,780 - root - [Epoch 10 Batch 8000/34064] loss=3.0593, ppl=21.3118, throughput=60.74K wps, wc=6521.70K
2018-06-26 10:00:48,677 - root - [Epoch 10 Batch 8800/34064] loss=3.0582, ppl=21.2890, throughput=59.92K wps, wc=6524.83K
2018-06-26 10:02:36,023 - root - [Epoch 10 Batch 9600/34064] loss=3.0471, ppl=21.0552, throughput=60.79K wps, wc=6525.48K
2018-06-26 10:04:23,754 - root - [Epoch 10 Batch 10400/34064] loss=3.0722, ppl=21.5896, throughput=60.45K wps, wc=6512.49K
2018-06-26 10:06:11,096 - root - [Epoch 10 Batch 11200/34064] loss=3.0532, ppl=21.1833, throughput=60.77K wps, wc=6523.47K
2018-06-26 10:07:58,828 - root - [Epoch 10 Batch 12000/34064] loss=3.0701, ppl=21.5437, throughput=60.51K wps, wc=6518.46K
2018-06-26 10:09:46,208 - root - [Epoch 10 Batch 12800/34064] loss=3.0607, ppl=21.3417, throughput=60.70K wps, wc=6517.98K
2018-06-26 10:11:33,400 - root - [Epoch 10 Batch 13600/34064] loss=3.0614, ppl=21.3568, throughput=60.82K wps, wc=6519.20K
2018-06-26 10:13:20,809 - root - [Epoch 10 Batch 14400/34064] loss=3.0539, ppl=21.1969, throughput=60.72K wps, wc=6522.02K
2018-06-26 10:15:09,475 - root - [Epoch 10 Batch 15200/34064] loss=3.0654, ppl=21.4432, throughput=60.03K wps, wc=6523.59K
2018-06-26 10:16:56,547 - root - [Epoch 10 Batch 16000/34064] loss=3.0521, ppl=21.1599, throughput=60.91K wps, wc=6522.09K
2018-06-26 10:18:44,999 - root - [Epoch 10 Batch 16800/34064] loss=3.0793, ppl=21.7433, throughput=60.01K wps, wc=6508.12K
2018-06-26 10:20:31,291 - root - [Epoch 10 Batch 17600/34064] loss=3.0397, ppl=20.8980, throughput=61.32K wps, wc=6517.91K
2018-06-26 10:22:19,451 - root - [Epoch 10 Batch 18400/34064] loss=3.0517, ppl=21.1507, throughput=60.30K wps, wc=6521.74K
2018-06-26 10:24:06,474 - root - [Epoch 10 Batch 19200/34064] loss=3.0528, ppl=21.1748, throughput=60.91K wps, wc=6518.53K
2018-06-26 10:25:54,648 - root - [Epoch 10 Batch 20000/34064] loss=3.0544, ppl=21.2079, throughput=60.33K wps, wc=6525.90K
2018-06-26 10:27:42,250 - root - [Epoch 10 Batch 20800/34064] loss=3.0536, ppl=21.1907, throughput=60.63K wps, wc=6524.13K
2018-06-26 10:29:32,217 - root - [Epoch 10 Batch 21600/34064] loss=3.0631, ppl=21.3947, throughput=59.27K wps, wc=6517.56K
2018-06-26 10:31:20,416 - root - [Epoch 10 Batch 22400/34064] loss=3.0590, ppl=21.3058, throughput=60.29K wps, wc=6523.47K
2018-06-26 10:33:08,455 - root - [Epoch 10 Batch 23200/34064] loss=3.0577, ppl=21.2789, throughput=60.32K wps, wc=6517.37K
2018-06-26 10:34:57,102 - root - [Epoch 10 Batch 24000/34064] loss=3.0545, ppl=21.2106, throughput=60.05K wps, wc=6524.64K
2018-06-26 10:36:43,753 - root - [Epoch 10 Batch 24800/34064] loss=3.0615, ppl=21.3588, throughput=61.16K wps, wc=6522.31K
2018-06-26 10:38:29,866 - root - [Epoch 10 Batch 25600/34064] loss=3.0497, ppl=21.1089, throughput=61.42K wps, wc=6517.67K
2018-06-26 10:40:17,695 - root - [Epoch 10 Batch 26400/34064] loss=3.0688, ppl=21.5159, throughput=60.47K wps, wc=6520.13K
2018-06-26 10:42:04,556 - root - [Epoch 10 Batch 27200/34064] loss=3.0601, ppl=21.3291, throughput=61.06K wps, wc=6524.91K
2018-06-26 10:43:52,964 - root - [Epoch 10 Batch 28000/34064] loss=3.0593, ppl=21.3116, throughput=60.08K wps, wc=6513.44K
2018-06-26 10:45:40,850 - root - [Epoch 10 Batch 28800/34064] loss=3.0542, ppl=21.2052, throughput=60.44K wps, wc=6520.56K
2018-06-26 10:47:28,413 - root - [Epoch 10 Batch 29600/34064] loss=3.0611, ppl=21.3521, throughput=60.63K wps, wc=6521.99K
2018-06-26 10:49:15,215 - root - [Epoch 10 Batch 30400/34064] loss=3.0510, ppl=21.1359, throughput=61.08K wps, wc=6523.20K
2018-06-26 10:51:02,949 - root - [Epoch 10 Batch 31200/34064] loss=3.0613, ppl=21.3561, throughput=60.56K wps, wc=6524.65K
2018-06-26 10:52:49,944 - root - [Epoch 10 Batch 32000/34064] loss=3.0519, ppl=21.1562, throughput=60.99K wps, wc=6525.91K
2018-06-26 10:54:36,267 - root - [Epoch 10 Batch 32800/34064] loss=3.0673, ppl=21.4841, throughput=61.34K wps, wc=6522.14K
2018-06-26 10:56:24,102 - root - [Epoch 10 Batch 33600/34064] loss=3.0570, ppl=21.2631, throughput=60.39K wps, wc=6512.03K
2018-06-26 11:02:01,724 - root - [Epoch 10] valid Loss=1.5740, valid ppl=4.8258, valid bleu=25.14
2018-06-26 11:06:35,575 - root - [Epoch 10] test Loss=1.4071, test ppl=4.0839, test bleu=25.32
2018-06-26 11:06:35,581 - root - Save best parameters to transformer_en_de_u512/valid_best.params
2018-06-26 11:08:29,964 - root - [Epoch 11 Batch 800/34064] loss=3.0417, ppl=20.9403, throughput=58.12K wps, wc=6518.86K
2018-06-26 11:10:17,191 - root - [Epoch 11 Batch 1600/34064] loss=3.0182, ppl=20.4542, throughput=60.75K wps, wc=6514.06K
2018-06-26 11:12:04,398 - root - [Epoch 11 Batch 2400/34064] loss=3.0278, ppl=20.6518, throughput=60.85K wps, wc=6523.30K
2018-06-26 11:13:52,263 - root - [Epoch 11 Batch 3200/34064] loss=3.0313, ppl=20.7242, throughput=60.36K wps, wc=6510.24K
2018-06-26 11:15:40,402 - root - [Epoch 11 Batch 4000/34064] loss=3.0303, ppl=20.7032, throughput=60.28K wps, wc=6519.07K
2018-06-26 11:17:30,147 - root - [Epoch 11 Batch 4800/34064] loss=3.0305, ppl=20.7083, throughput=59.34K wps, wc=6511.86K
2018-06-26 11:19:16,940 - root - [Epoch 11 Batch 5600/34064] loss=3.0373, ppl=20.8498, throughput=61.09K wps, wc=6523.57K
2018-06-26 11:21:03,981 - root - [Epoch 11 Batch 6400/34064] loss=3.0295, ppl=20.6861, throughput=60.93K wps, wc=6521.81K
2018-06-26 11:22:51,033 - root - [Epoch 11 Batch 7200/34064] loss=3.0430, ppl=20.9680, throughput=60.97K wps, wc=6526.54K
2018-06-26 11:24:38,355 - root - [Epoch 11 Batch 8000/34064] loss=3.0521, ppl=21.1588, throughput=60.76K wps, wc=6521.10K
2018-06-26 11:26:25,455 - root - [Epoch 11 Batch 8800/34064] loss=3.0482, ppl=21.0767, throughput=60.83K wps, wc=6514.72K
2018-06-26 11:28:12,865 - root - [Epoch 11 Batch 9600/34064] loss=3.0464, ppl=21.0386, throughput=60.73K wps, wc=6522.83K
2018-06-26 11:29:59,429 - root - [Epoch 11 Batch 10400/34064] loss=3.0385, ppl=20.8748, throughput=61.13K wps, wc=6514.53K
2018-06-26 11:31:47,795 - root - [Epoch 11 Batch 11200/34064] loss=3.0566, ppl=21.2560, throughput=60.19K wps, wc=6522.40K
2018-06-26 11:33:33,968 - root - [Epoch 11 Batch 12000/34064] loss=3.0475, ppl=21.0624, throughput=61.46K wps, wc=6525.45K
2018-06-26 11:35:21,288 - root - [Epoch 11 Batch 12800/34064] loss=3.0390, ppl=20.8849, throughput=60.75K wps, wc=6519.29K
2018-06-26 11:37:08,536 - root - [Epoch 11 Batch 13600/34064] loss=3.0389, ppl=20.8829, throughput=60.80K wps, wc=6520.85K
2018-06-26 11:38:55,450 - root - [Epoch 11 Batch 14400/34064] loss=3.0337, ppl=20.7732, throughput=61.00K wps, wc=6521.28K
2018-06-26 11:40:41,939 - root - [Epoch 11 Batch 15200/34064] loss=3.0515, ppl=21.1463, throughput=61.28K wps, wc=6525.83K
2018-06-26 11:42:28,712 - root - [Epoch 11 Batch 16000/34064] loss=3.0326, ppl=20.7505, throughput=61.09K wps, wc=6522.27K
2018-06-26 11:44:15,925 - root - [Epoch 11 Batch 16800/34064] loss=3.0419, ppl=20.9448, throughput=60.76K wps, wc=6513.94K
2018-06-26 11:46:03,200 - root - [Epoch 11 Batch 17600/34064] loss=3.0401, ppl=20.9067, throughput=60.82K wps, wc=6524.42K
2018-06-26 11:47:50,326 - root - [Epoch 11 Batch 18400/34064] loss=3.0423, ppl=20.9525, throughput=60.88K wps, wc=6521.52K
2018-06-26 11:49:37,545 - root - [Epoch 11 Batch 19200/34064] loss=3.0406, ppl=20.9177, throughput=60.84K wps, wc=6523.34K
2018-06-26 11:51:23,524 - root - [Epoch 11 Batch 20000/34064] loss=3.0427, ppl=20.9622, throughput=61.59K wps, wc=6526.85K
2018-06-26 11:53:12,313 - root - [Epoch 11 Batch 20800/34064] loss=3.0564, ppl=21.2503, throughput=59.84K wps, wc=6509.68K
2018-06-26 11:55:00,116 - root - [Epoch 11 Batch 21600/34064] loss=3.0376, ppl=20.8543, throughput=60.51K wps, wc=6522.66K
2018-06-26 11:56:47,842 - root - [Epoch 11 Batch 22400/34064] loss=3.0443, ppl=20.9943, throughput=60.52K wps, wc=6519.53K
2018-06-26 11:58:36,286 - root - [Epoch 11 Batch 23200/34064] loss=3.0489, ppl=21.0914, throughput=60.17K wps, wc=6524.70K
2018-06-26 12:00:24,407 - root - [Epoch 11 Batch 24000/34064] loss=3.0529, ppl=21.1756, throughput=60.36K wps, wc=6525.65K
2018-06-26 12:02:11,532 - root - [Epoch 11 Batch 24800/34064] loss=3.0387, ppl=20.8780, throughput=60.91K wps, wc=6524.46K
2018-06-26 12:03:58,878 - root - [Epoch 11 Batch 25600/34064] loss=3.0473, ppl=21.0577, throughput=60.69K wps, wc=6514.94K
2018-06-26 12:05:46,322 - root - [Epoch 11 Batch 26400/34064] loss=3.0246, ppl=20.5853, throughput=60.69K wps, wc=6521.11K
2018-06-26 12:07:34,769 - root - [Epoch 11 Batch 27200/34064] loss=3.0504, ppl=21.1242, throughput=60.19K wps, wc=6527.55K
2018-06-26 12:09:21,437 - root - [Epoch 11 Batch 28000/34064] loss=3.0397, ppl=20.8993, throughput=61.16K wps, wc=6524.27K
2018-06-26 12:11:07,680 - root - [Epoch 11 Batch 28800/34064] loss=3.0414, ppl=20.9354, throughput=61.38K wps, wc=6521.41K
2018-06-26 12:12:55,363 - root - [Epoch 11 Batch 29600/34064] loss=3.0453, ppl=21.0171, throughput=60.53K wps, wc=6518.05K
2018-06-26 12:14:43,552 - root - [Epoch 11 Batch 30400/34064] loss=3.0426, ppl=20.9600, throughput=60.33K wps, wc=6527.02K
2018-06-26 12:16:30,213 - root - [Epoch 11 Batch 31200/34064] loss=3.0524, ppl=21.1665, throughput=61.20K wps, wc=6527.94K
2018-06-26 12:18:17,965 - root - [Epoch 11 Batch 32000/34064] loss=3.0446, ppl=21.0012, throughput=60.57K wps, wc=6526.48K
2018-06-26 12:20:04,749 - root - [Epoch 11 Batch 32800/34064] loss=3.0404, ppl=20.9131, throughput=61.07K wps, wc=6521.36K
2018-06-26 12:21:51,071 - root - [Epoch 11 Batch 33600/34064] loss=3.0378, ppl=20.8594, throughput=61.33K wps, wc=6520.68K
2018-06-26 12:27:29,654 - root - [Epoch 11] valid Loss=1.5590, valid ppl=4.7538, valid bleu=25.41
2018-06-26 12:32:05,812 - root - [Epoch 11] test Loss=1.3907, test ppl=4.0175, test bleu=25.42
2018-06-26 12:32:05,818 - root - Save best parameters to transformer_en_de_u512/valid_best.params
2018-06-26 12:33:59,286 - root - [Epoch 12 Batch 800/34064] loss=3.0009, ppl=20.1037, throughput=58.66K wps, wc=6525.31K
2018-06-26 12:35:46,486 - root - [Epoch 12 Batch 1600/34064] loss=3.0077, ppl=20.2402, throughput=60.85K wps, wc=6523.48K
2018-06-26 12:37:34,533 - root - [Epoch 12 Batch 2400/34064] loss=3.0120, ppl=20.3277, throughput=60.39K wps, wc=6525.15K
2018-06-26 12:39:23,337 - root - [Epoch 12 Batch 3200/34064] loss=3.0165, ppl=20.4202, throughput=59.94K wps, wc=6522.14K
2018-06-26 12:41:11,255 - root - [Epoch 12 Batch 4000/34064] loss=3.0222, ppl=20.5361, throughput=60.43K wps, wc=6521.38K
2018-06-26 12:42:58,437 - root - [Epoch 12 Batch 4800/34064] loss=3.0227, ppl=20.5473, throughput=60.82K wps, wc=6519.11K
2018-06-26 12:44:45,852 - root - [Epoch 12 Batch 5600/34064] loss=3.0276, ppl=20.6484, throughput=60.75K wps, wc=6525.98K
2018-06-26 12:46:35,586 - root - [Epoch 12 Batch 6400/34064] loss=3.0182, ppl=20.4548, throughput=59.39K wps, wc=6517.36K
2018-06-26 12:48:23,027 - root - [Epoch 12 Batch 7200/34064] loss=3.0154, ppl=20.3963, throughput=60.73K wps, wc=6525.04K
2018-06-26 12:50:09,768 - root - [Epoch 12 Batch 8000/34064] loss=3.0189, ppl=20.4698, throughput=61.15K wps, wc=6526.68K
2018-06-26 12:51:57,283 - root - [Epoch 12 Batch 8800/34064] loss=3.0319, ppl=20.7361, throughput=60.61K wps, wc=6516.47K
2018-06-26 12:53:44,633 - root - [Epoch 12 Batch 9600/34064] loss=3.0326, ppl=20.7505, throughput=60.69K wps, wc=6514.72K
2018-06-26 12:55:31,069 - root - [Epoch 12 Batch 10400/34064] loss=3.0211, ppl=20.5145, throughput=61.28K wps, wc=6522.02K
2018-06-26 12:57:17,076 - root - [Epoch 12 Batch 11200/34064] loss=3.0240, ppl=20.5732, throughput=61.43K wps, wc=6511.48K
2018-06-26 12:59:04,471 - root - [Epoch 12 Batch 12000/34064] loss=3.0242, ppl=20.5766, throughput=60.69K wps, wc=6517.89K
2018-06-26 13:00:51,248 - root - [Epoch 12 Batch 12800/34064] loss=3.0222, ppl=20.5357, throughput=61.01K wps, wc=6514.13K
2018-06-26 13:02:39,844 - root - [Epoch 12 Batch 13600/34064] loss=3.0395, ppl=20.8943, throughput=60.09K wps, wc=6525.29K
2018-06-26 13:04:28,369 - root - [Epoch 12 Batch 14400/34064] loss=3.0389, ppl=20.8818, throughput=60.09K wps, wc=6521.07K
2018-06-26 13:06:15,197 - root - [Epoch 12 Batch 15200/34064] loss=3.0244, ppl=20.5817, throughput=61.10K wps, wc=6526.94K
2018-06-26 13:08:02,394 - root - [Epoch 12 Batch 16000/34064] loss=3.0356, ppl=20.8134, throughput=60.85K wps, wc=6522.67K
2018-06-26 13:09:49,904 - root - [Epoch 12 Batch 16800/34064] loss=3.0366, ppl=20.8344, throughput=60.65K wps, wc=6520.99K
2018-06-26 13:11:37,056 - root - [Epoch 12 Batch 17600/34064] loss=3.0272, ppl=20.6388, throughput=60.88K wps, wc=6523.73K
2018-06-26 13:13:24,420 - root - [Epoch 12 Batch 18400/34064] loss=3.0178, ppl=20.4455, throughput=60.81K wps, wc=6528.72K
2018-06-26 13:15:13,545 - root - [Epoch 12 Batch 19200/34064] loss=3.0365, ppl=20.8327, throughput=59.76K wps, wc=6520.88K
2018-06-26 13:17:00,705 - root - [Epoch 12 Batch 20000/34064] loss=3.0271, ppl=20.6381, throughput=60.83K wps, wc=6518.58K
2018-06-26 13:18:46,412 - root - [Epoch 12 Batch 20800/34064] loss=3.0258, ppl=20.6100, throughput=61.70K wps, wc=6521.65K
2018-06-26 13:20:33,032 - root - [Epoch 12 Batch 21600/34064] loss=3.0118, ppl=20.3237, throughput=61.21K wps, wc=6526.13K
2018-06-26 13:22:20,208 - root - [Epoch 12 Batch 22400/34064] loss=3.0260, ppl=20.6148, throughput=60.86K wps, wc=6522.92K
2018-06-26 13:24:07,546 - root - [Epoch 12 Batch 23200/34064] loss=3.0279, ppl=20.6531, throughput=60.74K wps, wc=6519.21K
2018-06-26 13:25:55,802 - root - [Epoch 12 Batch 24000/34064] loss=3.0306, ppl=20.7097, throughput=60.25K wps, wc=6522.74K
2018-06-26 13:27:44,913 - root - [Epoch 12 Batch 24800/34064] loss=3.0375, ppl=20.8523, throughput=59.80K wps, wc=6524.80K
2018-06-26 13:29:33,988 - root - [Epoch 12 Batch 25600/34064] loss=3.0333, ppl=20.7652, throughput=59.79K wps, wc=6522.04K
2018-06-26 13:31:21,791 - root - [Epoch 12 Batch 26400/34064] loss=3.0453, ppl=21.0155, throughput=60.45K wps, wc=6517.15K
2018-06-26 13:33:09,815 - root - [Epoch 12 Batch 27200/34064] loss=3.0335, ppl=20.7698, throughput=60.35K wps, wc=6519.72K
2018-06-26 13:34:56,631 - root - [Epoch 12 Batch 28000/34064] loss=3.0296, ppl=20.6887, throughput=61.04K wps, wc=6520.31K
2018-06-26 13:36:43,646 - root - [Epoch 12 Batch 28800/34064] loss=3.0365, ppl=20.8331, throughput=60.85K wps, wc=6511.74K
2018-06-26 13:38:31,093 - root - [Epoch 12 Batch 29600/34064] loss=3.0452, ppl=21.0146, throughput=60.71K wps, wc=6522.80K
2018-06-26 13:40:19,679 - root - [Epoch 12 Batch 30400/34064] loss=3.0484, ppl=21.0808, throughput=59.99K wps, wc=6514.34K
2018-06-26 13:42:08,049 - root - [Epoch 12 Batch 31200/34064] loss=3.0368, ppl=20.8391, throughput=60.18K wps, wc=6522.18K
2018-06-26 13:43:56,335 - root - [Epoch 12 Batch 32000/34064] loss=3.0532, ppl=21.1821, throughput=60.18K wps, wc=6516.95K
2018-06-26 13:45:43,395 - root - [Epoch 12 Batch 32800/34064] loss=3.0385, ppl=20.8736, throughput=60.92K wps, wc=6521.99K
2018-06-26 13:47:30,273 - root - [Epoch 12 Batch 33600/34064] loss=3.0268, ppl=20.6316, throughput=61.03K wps, wc=6522.55K
2018-06-26 13:53:06,982 - root - [Epoch 12] valid Loss=1.5482, valid ppl=4.7030, valid bleu=25.30
2018-06-26 13:57:55,418 - root - [Epoch 12] test Loss=1.3835, test ppl=3.9887, test bleu=25.72
2018-06-26 13:59:46,547 - root - [Epoch 13 Batch 800/34064] loss=3.0109, ppl=20.3067, throughput=58.91K wps, wc=6520.33K
2018-06-26 14:01:34,024 - root - [Epoch 13 Batch 1600/34064] loss=2.9985, ppl=20.0552, throughput=60.69K wps, wc=6522.96K
2018-06-26 14:03:22,443 - root - [Epoch 13 Batch 2400/34064] loss=2.9969, ppl=20.0233, throughput=60.12K wps, wc=6518.52K
2018-06-26 14:05:09,601 - root - [Epoch 13 Batch 3200/34064] loss=3.0045, ppl=20.1760, throughput=60.92K wps, wc=6528.04K
2018-06-26 14:06:57,531 - root - [Epoch 13 Batch 4000/34064] loss=3.0081, ppl=20.2493, throughput=60.47K wps, wc=6526.34K
2018-06-26 14:08:45,534 - root - [Epoch 13 Batch 4800/34064] loss=3.0065, ppl=20.2169, throughput=60.40K wps, wc=6523.68K
2018-06-26 14:10:32,315 - root - [Epoch 13 Batch 5600/34064] loss=2.9942, ppl=19.9693, throughput=61.05K wps, wc=6518.80K
2018-06-26 14:12:20,622 - root - [Epoch 13 Batch 6400/34064] loss=3.0143, ppl=20.3740, throughput=60.13K wps, wc=6512.26K
2018-06-26 14:14:08,649 - root - [Epoch 13 Batch 7200/34064] loss=3.0069, ppl=20.2245, throughput=60.38K wps, wc=6522.53K
2018-06-26 14:15:55,357 - root - [Epoch 13 Batch 8000/34064] loss=3.0119, ppl=20.3263, throughput=61.10K wps, wc=6520.15K
2018-06-26 14:17:44,078 - root - [Epoch 13 Batch 8800/34064] loss=2.9988, ppl=20.0608, throughput=59.98K wps, wc=6521.33K
2018-06-26 14:19:32,211 - root - [Epoch 13 Batch 9600/34064] loss=3.0284, ppl=20.6639, throughput=60.29K wps, wc=6519.09K
2018-06-26 14:21:20,783 - root - [Epoch 13 Batch 10400/34064] loss=3.0111, ppl=20.3094, throughput=60.07K wps, wc=6521.85K
2018-06-26 14:23:06,757 - root - [Epoch 13 Batch 11200/34064] loss=3.0256, ppl=20.6066, throughput=61.52K wps, wc=6519.57K
2018-06-26 14:24:52,880 - root - [Epoch 13 Batch 12000/34064] loss=3.0046, ppl=20.1785, throughput=61.51K wps, wc=6527.04K
2018-06-26 14:26:40,119 - root - [Epoch 13 Batch 12800/34064] loss=3.0254, ppl=20.6031, throughput=60.77K wps, wc=6516.84K
2018-06-26 14:28:27,682 - root - [Epoch 13 Batch 13600/34064] loss=3.0204, ppl=20.4999, throughput=60.65K wps, wc=6524.00K
2018-06-26 14:30:14,851 - root - [Epoch 13 Batch 14400/34064] loss=2.9962, ppl=20.0095, throughput=60.89K wps, wc=6525.17K
2018-06-26 14:32:01,149 - root - [Epoch 13 Batch 15200/34064] loss=3.0056, ppl=20.1984, throughput=61.39K wps, wc=6525.40K
2018-06-26 14:33:48,263 - root - [Epoch 13 Batch 16000/34064] loss=3.0206, ppl=20.5046, throughput=60.91K wps, wc=6524.02K
2018-06-26 14:35:36,715 - root - [Epoch 13 Batch 16800/34064] loss=3.0372, ppl=20.8469, throughput=60.08K wps, wc=6515.66K
2018-06-26 14:37:24,268 - root - [Epoch 13 Batch 17600/34064] loss=3.0128, ppl=20.3451, throughput=60.59K wps, wc=6516.12K
2018-06-26 14:39:11,155 - root - [Epoch 13 Batch 18400/34064] loss=3.0148, ppl=20.3858, throughput=61.00K wps, wc=6519.88K
2018-06-26 14:40:57,965 - root - [Epoch 13 Batch 19200/34064] loss=3.0143, ppl=20.3751, throughput=61.05K wps, wc=6520.20K
2018-06-26 14:42:45,748 - root - [Epoch 13 Batch 20000/34064] loss=3.0282, ppl=20.6598, throughput=60.57K wps, wc=6527.85K
2018-06-26 14:44:34,974 - root - [Epoch 13 Batch 20800/34064] loss=3.0235, ppl=20.5628, throughput=59.72K wps, wc=6522.58K
2018-06-26 14:46:22,295 - root - [Epoch 13 Batch 21600/34064] loss=3.0274, ppl=20.6427, throughput=60.81K wps, wc=6525.75K
2018-06-26 14:48:09,658 - root - [Epoch 13 Batch 22400/34064] loss=3.0247, ppl=20.5881, throughput=60.73K wps, wc=6519.68K
2018-06-26 14:49:56,509 - root - [Epoch 13 Batch 23200/34064] loss=3.0228, ppl=20.5483, throughput=61.07K wps, wc=6525.70K
2018-06-26 14:51:44,826 - root - [Epoch 13 Batch 24000/34064] loss=3.0268, ppl=20.6307, throughput=60.15K wps, wc=6515.11K
2018-06-26 14:53:32,592 - root - [Epoch 13 Batch 24800/34064] loss=3.0268, ppl=20.6308, throughput=60.53K wps, wc=6522.52K
2018-06-26 14:55:20,740 - root - [Epoch 13 Batch 25600/34064] loss=3.0344, ppl=20.7886, throughput=60.31K wps, wc=6521.96K
2018-06-26 14:57:07,269 - root - [Epoch 13 Batch 26400/34064] loss=3.0078, ppl=20.2425, throughput=61.19K wps, wc=6518.29K
2018-06-26 14:58:55,376 - root - [Epoch 13 Batch 27200/34064] loss=3.0290, ppl=20.6761, throughput=60.35K wps, wc=6524.52K
2018-06-26 15:00:42,932 - root - [Epoch 13 Batch 28000/34064] loss=3.0120, ppl=20.3276, throughput=60.65K wps, wc=6523.41K
2018-06-26 15:02:30,168 - root - [Epoch 13 Batch 28800/34064] loss=3.0140, ppl=20.3686, throughput=60.83K wps, wc=6522.79K
2018-06-26 15:04:17,519 - root - [Epoch 13 Batch 29600/34064] loss=3.0270, ppl=20.6362, throughput=60.74K wps, wc=6520.08K
2018-06-26 15:06:05,852 - root - [Epoch 13 Batch 30400/34064] loss=3.0300, ppl=20.6982, throughput=60.17K wps, wc=6518.70K
2018-06-26 15:07:53,110 - root - [Epoch 13 Batch 31200/34064] loss=3.0246, ppl=20.5862, throughput=60.79K wps, wc=6519.86K
2018-06-26 15:09:39,856 - root - [Epoch 13 Batch 32000/34064] loss=3.0138, ppl=20.3643, throughput=61.13K wps, wc=6525.08K
2018-06-26 15:11:27,577 - root - [Epoch 13 Batch 32800/34064] loss=3.0309, ppl=20.7161, throughput=60.54K wps, wc=6521.16K
2018-06-26 15:13:16,991 - root - [Epoch 13 Batch 33600/34064] loss=3.0350, ppl=20.8002, throughput=59.51K wps, wc=6510.68K
2018-06-26 15:19:14,477 - root - [Epoch 13] valid Loss=1.5468, valid ppl=4.6963, valid bleu=25.35
2018-06-26 15:23:50,324 - root - [Epoch 13] test Loss=1.3781, test ppl=3.9674, test bleu=25.57
2018-06-26 15:25:43,166 - root - [Epoch 14 Batch 800/34064] loss=2.9904, ppl=19.8939, throughput=57.99K wps, wc=6517.70K
2018-06-26 15:27:31,358 - root - [Epoch 14 Batch 1600/34064] loss=2.9898, ppl=19.8824, throughput=60.27K wps, wc=6520.82K
2018-06-26 15:29:18,936 - root - [Epoch 14 Batch 2400/34064] loss=2.9986, ppl=20.0574, throughput=60.66K wps, wc=6525.94K
2018-06-26 15:31:05,329 - root - [Epoch 14 Batch 3200/34064] loss=2.9798, ppl=19.6834, throughput=61.35K wps, wc=6527.13K
2018-06-26 15:32:52,694 - root - [Epoch 14 Batch 4000/34064] loss=2.9976, ppl=20.0374, throughput=60.71K wps, wc=6518.08K
2018-06-26 15:34:41,790 - root - [Epoch 14 Batch 4800/34064] loss=2.9996, ppl=20.0773, throughput=59.77K wps, wc=6521.08K
2018-06-26 15:36:28,066 - root - [Epoch 14 Batch 5600/34064] loss=2.9877, ppl=19.8395, throughput=61.39K wps, wc=6524.65K
2018-06-26 15:38:16,697 - root - [Epoch 14 Batch 6400/34064] loss=3.0060, ppl=20.2057, throughput=60.05K wps, wc=6523.76K
2018-06-26 15:40:04,562 - root - [Epoch 14 Batch 7200/34064] loss=3.0125, ppl=20.3380, throughput=60.47K wps, wc=6522.62K
2018-06-26 15:41:51,814 - root - [Epoch 14 Batch 8000/34064] loss=2.9923, ppl=19.9324, throughput=60.79K wps, wc=6519.69K
2018-06-26 15:43:38,491 - root - [Epoch 14 Batch 8800/34064] loss=2.9929, ppl=19.9426, throughput=61.13K wps, wc=6521.14K
2018-06-26 15:45:26,718 - root - [Epoch 14 Batch 9600/34064] loss=3.0056, ppl=20.1974, throughput=60.27K wps, wc=6522.34K
2018-06-26 15:47:14,817 - root - [Epoch 14 Batch 10400/34064] loss=3.0045, ppl=20.1766, throughput=60.29K wps, wc=6517.58K
2018-06-26 15:49:02,091 - root - [Epoch 14 Batch 11200/34064] loss=3.0073, ppl=20.2336, throughput=60.79K wps, wc=6521.55K
2018-06-26 15:50:48,593 - root - [Epoch 14 Batch 12000/34064] loss=3.0009, ppl=20.1043, throughput=61.27K wps, wc=6525.74K
2018-06-26 15:52:35,620 - root - [Epoch 14 Batch 12800/34064] loss=2.9948, ppl=19.9823, throughput=60.96K wps, wc=6523.95K
2018-06-26 15:54:22,639 - root - [Epoch 14 Batch 13600/34064] loss=3.0200, ppl=20.4905, throughput=60.95K wps, wc=6523.01K
2018-06-26 15:56:11,317 - root - [Epoch 14 Batch 14400/34064] loss=3.0269, ppl=20.6335, throughput=59.94K wps, wc=6513.60K
2018-06-26 15:57:59,250 - root - [Epoch 14 Batch 15200/34064] loss=2.9974, ppl=20.0337, throughput=60.38K wps, wc=6517.25K
2018-06-26 15:59:46,562 - root - [Epoch 14 Batch 16000/34064] loss=2.9923, ppl=19.9310, throughput=60.80K wps, wc=6524.08K
2018-06-26 16:01:32,410 - root - [Epoch 14 Batch 16800/34064] loss=3.0032, ppl=20.1497, throughput=61.59K wps, wc=6518.92K
2018-06-26 16:03:19,769 - root - [Epoch 14 Batch 17600/34064] loss=3.0138, ppl=20.3655, throughput=60.74K wps, wc=6520.79K
2018-06-26 16:05:06,093 - root - [Epoch 14 Batch 18400/34064] loss=3.0065, ppl=20.2175, throughput=61.31K wps, wc=6518.41K
2018-06-26 16:06:54,519 - root - [Epoch 14 Batch 19200/34064] loss=3.0243, ppl=20.5799, throughput=60.07K wps, wc=6512.76K
2018-06-26 16:08:39,823 - root - [Epoch 14 Batch 20000/34064] loss=3.0098, ppl=20.2828, throughput=61.82K wps, wc=6510.12K
2018-06-26 16:10:26,842 - root - [Epoch 14 Batch 20800/34064] loss=3.0027, ppl=20.1404, throughput=60.98K wps, wc=6525.80K
2018-06-26 16:12:13,439 - root - [Epoch 14 Batch 21600/34064] loss=3.0102, ppl=20.2905, throughput=61.24K wps, wc=6528.22K
2018-06-26 16:14:01,835 - root - [Epoch 14 Batch 22400/34064] loss=3.0206, ppl=20.5042, throughput=60.14K wps, wc=6519.15K
2018-06-26 16:15:49,226 - root - [Epoch 14 Batch 23200/34064] loss=3.0198, ppl=20.4864, throughput=60.72K wps, wc=6521.23K
2018-06-26 16:17:37,227 - root - [Epoch 14 Batch 24000/34064] loss=3.0044, ppl=20.1741, throughput=60.44K wps, wc=6527.30K
2018-06-26 16:19:23,980 - root - [Epoch 14 Batch 24800/34064] loss=3.0048, ppl=20.1827, throughput=61.10K wps, wc=6523.07K
2018-06-26 16:21:12,212 - root - [Epoch 14 Batch 25600/34064] loss=3.0096, ppl=20.2785, throughput=60.26K wps, wc=6521.80K
2018-06-26 16:22:59,351 - root - [Epoch 14 Batch 26400/34064] loss=3.0164, ppl=20.4183, throughput=60.87K wps, wc=6521.34K
2018-06-26 16:24:45,868 - root - [Epoch 14 Batch 27200/34064] loss=3.0153, ppl=20.3958, throughput=61.27K wps, wc=6525.87K
2018-06-26 16:26:34,516 - root - [Epoch 14 Batch 28000/34064] loss=3.0296, ppl=20.6899, throughput=59.97K wps, wc=6515.64K
2018-06-26 16:28:21,592 - root - [Epoch 14 Batch 28800/34064] loss=3.0053, ppl=20.1921, throughput=60.91K wps, wc=6521.53K
2018-06-26 16:30:09,294 - root - [Epoch 14 Batch 29600/34064] loss=3.0341, ppl=20.7825, throughput=60.47K wps, wc=6512.51K
2018-06-26 16:31:55,509 - root - [Epoch 14 Batch 30400/34064] loss=3.0133, ppl=20.3546, throughput=61.45K wps, wc=6527.10K
2018-06-26 16:33:44,233 - root - [Epoch 14 Batch 31200/34064] loss=3.0154, ppl=20.3973, throughput=59.94K wps, wc=6516.77K
2018-06-26 16:35:31,249 - root - [Epoch 14 Batch 32000/34064] loss=3.0122, ppl=20.3330, throughput=60.94K wps, wc=6521.48K
2018-06-26 16:37:18,570 - root - [Epoch 14 Batch 32800/34064] loss=3.0198, ppl=20.4880, throughput=60.77K wps, wc=6522.23K
2018-06-26 16:39:06,053 - root - [Epoch 14 Batch 33600/34064] loss=3.0175, ppl=20.4406, throughput=60.68K wps, wc=6522.34K
2018-06-26 16:44:41,349 - root - [Epoch 14] valid Loss=1.5444, valid ppl=4.6854, valid bleu=25.30
2018-06-26 16:49:15,799 - root - [Epoch 14] test Loss=1.3726, test ppl=3.9455, test bleu=25.59
2018-06-26 16:51:08,561 - root - [Epoch 15 Batch 800/34064] loss=2.9847, ppl=19.7804, throughput=58.05K wps, wc=6519.62K
2018-06-26 16:52:55,201 - root - [Epoch 15 Batch 1600/34064] loss=2.9713, ppl=19.5181, throughput=61.10K wps, wc=6516.10K
2018-06-26 16:54:43,264 - root - [Epoch 15 Batch 2400/34064] loss=2.9843, ppl=19.7733, throughput=60.35K wps, wc=6521.93K
2018-06-26 16:56:30,073 - root - [Epoch 15 Batch 3200/34064] loss=2.9751, ppl=19.5917, throughput=61.07K wps, wc=6522.60K
2018-06-26 16:58:18,260 - root - [Epoch 15 Batch 4000/34064] loss=2.9932, ppl=19.9492, throughput=60.28K wps, wc=6521.17K
2018-06-26 17:00:07,413 - root - [Epoch 15 Batch 4800/34064] loss=2.9927, ppl=19.9396, throughput=59.73K wps, wc=6519.82K
2018-06-26 17:01:54,473 - root - [Epoch 15 Batch 5600/34064] loss=3.0004, ppl=20.0937, throughput=60.92K wps, wc=6522.32K
2018-06-26 17:03:42,035 - root - [Epoch 15 Batch 6400/34064] loss=2.9938, ppl=19.9605, throughput=60.63K wps, wc=6521.22K
2018-06-26 17:05:29,973 - root - [Epoch 15 Batch 7200/34064] loss=3.0033, ppl=20.1514, throughput=60.41K wps, wc=6520.05K
2018-06-26 17:07:16,599 - root - [Epoch 15 Batch 8000/34064] loss=2.9894, ppl=19.8735, throughput=61.20K wps, wc=6525.79K
2018-06-26 17:09:04,149 - root - [Epoch 15 Batch 8800/34064] loss=3.0009, ppl=20.1039, throughput=60.66K wps, wc=6523.74K
2018-06-26 17:10:50,587 - root - [Epoch 15 Batch 9600/34064] loss=2.9897, ppl=19.8788, throughput=61.31K wps, wc=6525.46K
2018-06-26 17:12:37,733 - root - [Epoch 15 Batch 10400/34064] loss=2.9995, ppl=20.0761, throughput=60.84K wps, wc=6518.94K
2018-06-26 17:14:25,689 - root - [Epoch 15 Batch 11200/34064] loss=2.9948, ppl=19.9804, throughput=60.42K wps, wc=6522.33K
2018-06-26 17:16:12,176 - root - [Epoch 15 Batch 12000/34064] loss=2.9940, ppl=19.9654, throughput=61.24K wps, wc=6521.36K
2018-06-26 17:18:01,126 - root - [Epoch 15 Batch 12800/34064] loss=3.0186, ppl=20.4628, throughput=59.85K wps, wc=6520.14K
2018-06-26 17:19:48,651 - root - [Epoch 15 Batch 13600/34064] loss=3.0034, ppl=20.1547, throughput=60.60K wps, wc=6515.67K
2018-06-26 17:21:35,762 - root - [Epoch 15 Batch 14400/34064] loss=2.9978, ppl=20.0407, throughput=60.91K wps, wc=6524.15K
2018-06-26 17:23:22,343 - root - [Epoch 15 Batch 15200/34064] loss=3.0054, ppl=20.1936, throughput=61.20K wps, wc=6522.60K
2018-06-26 17:25:11,328 - root - [Epoch 15 Batch 16000/34064] loss=3.0023, ppl=20.1309, throughput=59.74K wps, wc=6510.54K
2018-06-26 17:26:59,351 - root - [Epoch 15 Batch 16800/34064] loss=2.9970, ppl=20.0257, throughput=60.29K wps, wc=6513.12K
2018-06-26 17:28:46,718 - root - [Epoch 15 Batch 17600/34064] loss=3.0024, ppl=20.1335, throughput=60.78K wps, wc=6525.27K
2018-06-26 17:30:34,051 - root - [Epoch 15 Batch 18400/34064] loss=2.9921, ppl=19.9266, throughput=60.76K wps, wc=6521.44K
2018-06-26 17:32:20,862 - root - [Epoch 15 Batch 19200/34064] loss=2.9946, ppl=19.9778, throughput=60.99K wps, wc=6513.98K
2018-06-26 17:34:08,237 - root - [Epoch 15 Batch 20000/34064] loss=2.9996, ppl=20.0767, throughput=60.75K wps, wc=6522.57K
2018-06-26 17:35:56,898 - root - [Epoch 15 Batch 20800/34064] loss=3.0025, ppl=20.1366, throughput=60.01K wps, wc=6520.24K
2018-06-26 17:37:44,812 - root - [Epoch 15 Batch 21600/34064] loss=3.0050, ppl=20.1852, throughput=60.47K wps, wc=6525.97K
2018-06-26 17:39:32,417 - root - [Epoch 15 Batch 22400/34064] loss=3.0008, ppl=20.1009, throughput=60.57K wps, wc=6517.59K
2018-06-26 17:41:19,963 - root - [Epoch 15 Batch 23200/34064] loss=3.0061, ppl=20.2085, throughput=60.67K wps, wc=6525.08K
2018-06-26 17:43:07,796 - root - [Epoch 15 Batch 24000/34064] loss=3.0031, ppl=20.1483, throughput=60.52K wps, wc=6526.00K
2018-06-26 17:44:56,484 - root - [Epoch 15 Batch 24800/34064] loss=3.0122, ppl=20.3316, throughput=60.05K wps, wc=6526.90K
2018-06-26 17:46:44,064 - root - [Epoch 15 Batch 25600/34064] loss=2.9846, ppl=19.7783, throughput=60.66K wps, wc=6526.27K
2018-06-26 17:48:31,060 - root - [Epoch 15 Batch 26400/34064] loss=2.9890, ppl=19.8655, throughput=60.92K wps, wc=6518.63K
2018-06-26 17:50:18,253 - root - [Epoch 15 Batch 27200/34064] loss=3.0080, ppl=20.2466, throughput=60.81K wps, wc=6518.17K
2018-06-26 17:52:05,462 - root - [Epoch 15 Batch 28000/34064] loss=2.9992, ppl=20.0703, throughput=60.85K wps, wc=6523.61K
2018-06-26 17:53:51,921 - root - [Epoch 15 Batch 28800/34064] loss=2.9951, ppl=19.9869, throughput=61.29K wps, wc=6525.34K
2018-06-26 17:55:39,309 - root - [Epoch 15 Batch 29600/34064] loss=3.0003, ppl=20.0911, throughput=60.68K wps, wc=6516.49K
2018-06-26 17:57:28,143 - root - [Epoch 15 Batch 30400/34064] loss=3.0009, ppl=20.1044, throughput=59.92K wps, wc=6521.68K
2018-06-26 17:59:15,501 - root - [Epoch 15 Batch 31200/34064] loss=3.0057, ppl=20.2003, throughput=60.69K wps, wc=6515.71K
2018-06-26 18:01:02,765 - root - [Epoch 15 Batch 32000/34064] loss=3.0074, ppl=20.2352, throughput=60.75K wps, wc=6516.21K
2018-06-26 18:02:50,377 - root - [Epoch 15 Batch 32800/34064] loss=3.0064, ppl=20.2150, throughput=60.68K wps, wc=6529.85K
2018-06-26 18:04:36,926 - root - [Epoch 15 Batch 33600/34064] loss=3.0143, ppl=20.3758, throughput=61.22K wps, wc=6522.48K
2018-06-26 18:10:13,846 - root - [Epoch 15] valid Loss=1.5356, valid ppl=4.6440, valid bleu=25.39
2018-06-26 18:14:49,483 - root - [Epoch 15] test Loss=1.3620, test ppl=3.9038, test bleu=26.05
2018-06-26 18:16:42,489 - root - [Epoch 16 Batch 800/34064] loss=2.9675, ppl=19.4441, throughput=57.95K wps, wc=6521.43K
2018-06-26 18:18:30,229 - root - [Epoch 16 Batch 1600/34064] loss=2.9758, ppl=19.6060, throughput=60.54K wps, wc=6522.15K
2018-06-26 18:20:17,038 - root - [Epoch 16 Batch 2400/34064] loss=2.9651, ppl=19.3976, throughput=60.99K wps, wc=6514.18K
2018-06-26 18:22:05,316 - root - [Epoch 16 Batch 3200/34064] loss=2.9783, ppl=19.6542, throughput=60.23K wps, wc=6522.03K
2018-06-26 18:23:52,693 - root - [Epoch 16 Batch 4000/34064] loss=2.9804, ppl=19.6962, throughput=60.71K wps, wc=6519.09K
2018-06-26 18:25:39,587 - root - [Epoch 16 Batch 4800/34064] loss=2.9779, ppl=19.6470, throughput=60.99K wps, wc=6519.15K
2018-06-26 18:27:26,395 - root - [Epoch 16 Batch 5600/34064] loss=2.9577, ppl=19.2539, throughput=61.06K wps, wc=6522.10K
2018-06-26 18:29:14,001 - root - [Epoch 16 Batch 6400/34064] loss=2.9823, ppl=19.7339, throughput=60.66K wps, wc=6527.32K
2018-06-26 18:31:01,189 - root - [Epoch 16 Batch 7200/34064] loss=2.9794, ppl=19.6761, throughput=60.81K wps, wc=6518.45K
2018-06-26 18:32:47,747 - root - [Epoch 16 Batch 8000/34064] loss=2.9759, ppl=19.6071, throughput=61.20K wps, wc=6520.84K
2018-06-26 18:34:35,364 - root - [Epoch 16 Batch 8800/34064] loss=2.9857, ppl=19.8004, throughput=60.58K wps, wc=6519.59K
2018-06-26 18:36:22,602 - root - [Epoch 16 Batch 9600/34064] loss=2.9834, ppl=19.7554, throughput=60.83K wps, wc=6523.16K
2018-06-26 18:38:10,134 - root - [Epoch 16 Batch 10400/34064] loss=2.9835, ppl=19.7577, throughput=60.69K wps, wc=6526.31K
2018-06-26 18:39:57,072 - root - [Epoch 16 Batch 11200/34064] loss=2.9859, ppl=19.8045, throughput=60.99K wps, wc=6522.58K
2018-06-26 18:41:43,670 - root - [Epoch 16 Batch 12000/34064] loss=2.9770, ppl=19.6291, throughput=61.20K wps, wc=6523.47K
2018-06-26 18:43:33,145 - root - [Epoch 16 Batch 12800/34064] loss=2.9949, ppl=19.9834, throughput=59.61K wps, wc=6525.60K
2018-06-26 18:45:21,176 - root - [Epoch 16 Batch 13600/34064] loss=3.0137, ppl=20.3627, throughput=60.35K wps, wc=6519.79K
2018-06-26 18:47:09,743 - root - [Epoch 16 Batch 14400/34064] loss=2.9934, ppl=19.9544, throughput=60.06K wps, wc=6520.74K
2018-06-26 18:48:56,123 - root - [Epoch 16 Batch 15200/34064] loss=2.9864, ppl=19.8147, throughput=61.20K wps, wc=6510.65K
2018-06-26 18:50:43,218 - root - [Epoch 16 Batch 16000/34064] loss=2.9900, ppl=19.8854, throughput=60.97K wps, wc=6530.02K
2018-06-26 18:52:31,534 - root - [Epoch 16 Batch 16800/34064] loss=2.9854, ppl=19.7951, throughput=60.19K wps, wc=6519.95K
2018-06-26 18:54:19,981 - root - [Epoch 16 Batch 17600/34064] loss=2.9969, ppl=20.0239, throughput=60.12K wps, wc=6520.04K
2018-06-26 18:56:08,117 - root - [Epoch 16 Batch 18400/34064] loss=2.9990, ppl=20.0655, throughput=60.23K wps, wc=6513.02K
2018-06-26 18:57:57,880 - root - [Epoch 16 Batch 19200/34064] loss=2.9974, ppl=20.0324, throughput=59.32K wps, wc=6511.59K
2018-06-26 18:59:45,253 - root - [Epoch 16 Batch 20000/34064] loss=3.0039, ppl=20.1643, throughput=60.80K wps, wc=6527.95K
2018-06-26 19:01:32,300 - root - [Epoch 16 Batch 20800/34064] loss=3.0049, ppl=20.1833, throughput=60.93K wps, wc=6522.22K
2018-06-26 19:03:19,785 - root - [Epoch 16 Batch 21600/34064] loss=2.9664, ppl=19.4221, throughput=60.72K wps, wc=6526.37K
2018-06-26 19:05:07,261 - root - [Epoch 16 Batch 22400/34064] loss=2.9939, ppl=19.9633, throughput=60.62K wps, wc=6515.34K
2018-06-26 19:06:54,506 - root - [Epoch 16 Batch 23200/34064] loss=2.9936, ppl=19.9574, throughput=60.80K wps, wc=6520.48K
2018-06-26 19:08:43,235 - root - [Epoch 16 Batch 24000/34064] loss=3.0077, ppl=20.2410, throughput=59.97K wps, wc=6520.50K
2018-06-26 19:10:30,852 - root - [Epoch 16 Batch 24800/34064] loss=2.9997, ppl=20.0791, throughput=60.58K wps, wc=6519.56K
2018-06-26 19:12:19,256 - root - [Epoch 16 Batch 25600/34064] loss=3.0090, ppl=20.2674, throughput=60.15K wps, wc=6520.37K
2018-06-26 19:14:05,819 - root - [Epoch 16 Batch 26400/34064] loss=3.0001, ppl=20.0872, throughput=61.23K wps, wc=6524.68K
2018-06-26 19:15:53,412 - root - [Epoch 16 Batch 27200/34064] loss=3.0055, ppl=20.1968, throughput=60.64K wps, wc=6524.46K
2018-06-26 19:17:40,778 - root - [Epoch 16 Batch 28000/34064] loss=3.0045, ppl=20.1754, throughput=60.74K wps, wc=6521.39K
2018-06-26 19:19:27,676 - root - [Epoch 16 Batch 28800/34064] loss=2.9933, ppl=19.9520, throughput=61.00K wps, wc=6520.23K
2018-06-26 19:21:15,282 - root - [Epoch 16 Batch 29600/34064] loss=3.0023, ppl=20.1319, throughput=60.51K wps, wc=6511.25K
2018-06-26 19:23:02,099 - root - [Epoch 16 Batch 30400/34064] loss=2.9999, ppl=20.0831, throughput=61.08K wps, wc=6524.65K
2018-06-26 19:24:49,456 - root - [Epoch 16 Batch 31200/34064] loss=2.9949, ppl=19.9827, throughput=60.77K wps, wc=6523.97K
2018-06-26 19:26:37,954 - root - [Epoch 16 Batch 32000/34064] loss=3.0009, ppl=20.1027, throughput=60.06K wps, wc=6516.83K
2018-06-26 19:28:26,081 - root - [Epoch 16 Batch 32800/34064] loss=2.9889, ppl=19.8642, throughput=60.37K wps, wc=6527.08K
2018-06-26 19:30:12,218 - root - [Epoch 16 Batch 33600/34064] loss=2.9875, ppl=19.8370, throughput=61.49K wps, wc=6526.33K
2018-06-26 19:35:50,937 - root - [Epoch 16] valid Loss=1.5300, valid ppl=4.6181, valid bleu=25.61
2018-06-26 19:40:27,288 - root - [Epoch 16] test Loss=1.3604, test ppl=3.8977, test bleu=25.71
2018-06-26 19:40:27,294 - root - Save best parameters to transformer_en_de_u512/valid_best.params
2018-06-26 19:42:22,746 - root - [Epoch 17 Batch 800/34064] loss=2.9607, ppl=19.3119, throughput=57.59K wps, wc=6520.80K
2018-06-26 19:44:10,327 - root - [Epoch 17 Batch 1600/34064] loss=2.9705, ppl=19.5017, throughput=60.62K wps, wc=6521.82K
2018-06-26 19:45:59,260 - root - [Epoch 17 Batch 2400/34064] loss=2.9616, ppl=19.3297, throughput=59.85K wps, wc=6519.13K
2018-06-26 19:47:46,395 - root - [Epoch 17 Batch 3200/34064] loss=2.9644, ppl=19.3826, throughput=60.89K wps, wc=6522.89K
2018-06-26 19:49:33,409 - root - [Epoch 17 Batch 4000/34064] loss=2.9583, ppl=19.2660, throughput=60.93K wps, wc=6520.35K
2018-06-26 19:51:19,178 - root - [Epoch 17 Batch 4800/34064] loss=2.9649, ppl=19.3921, throughput=61.66K wps, wc=6522.00K
2018-06-26 19:53:06,828 - root - [Epoch 17 Batch 5600/34064] loss=2.9784, ppl=19.6569, throughput=60.58K wps, wc=6521.11K
2018-06-26 19:54:54,169 - root - [Epoch 17 Batch 6400/34064] loss=2.9745, ppl=19.5806, throughput=60.75K wps, wc=6521.00K
2018-06-26 19:56:41,194 - root - [Epoch 17 Batch 7200/34064] loss=2.9771, ppl=19.6302, throughput=60.94K wps, wc=6522.45K
2018-06-26 19:58:26,461 - root - [Epoch 17 Batch 8000/34064] loss=2.9811, ppl=19.7086, throughput=61.98K wps, wc=6523.87K
2018-06-26 20:00:15,242 - root - [Epoch 17 Batch 8800/34064] loss=2.9733, ppl=19.5560, throughput=59.92K wps, wc=6517.96K
2018-06-26 20:02:02,154 - root - [Epoch 17 Batch 9600/34064] loss=2.9743, ppl=19.5761, throughput=60.98K wps, wc=6519.28K
2018-06-26 20:03:49,266 - root - [Epoch 17 Batch 10400/34064] loss=2.9877, ppl=19.8404, throughput=60.93K wps, wc=6526.07K
2018-06-26 20:05:36,002 - root - [Epoch 17 Batch 11200/34064] loss=2.9687, ppl=19.4664, throughput=61.01K wps, wc=6512.45K
2018-06-26 20:07:23,234 - root - [Epoch 17 Batch 12000/34064] loss=2.9729, ppl=19.5482, throughput=60.82K wps, wc=6521.32K
2018-06-26 20:09:09,256 - root - [Epoch 17 Batch 12800/34064] loss=2.9701, ppl=19.4943, throughput=61.55K wps, wc=6526.01K
2018-06-26 20:10:57,041 - root - [Epoch 17 Batch 13600/34064] loss=2.9831, ppl=19.7487, throughput=60.49K wps, wc=6519.91K
2018-06-26 20:12:45,276 - root - [Epoch 17 Batch 14400/34064] loss=2.9821, ppl=19.7286, throughput=60.25K wps, wc=6521.45K
2018-06-26 20:14:34,678 - root - [Epoch 17 Batch 15200/34064] loss=2.9895, ppl=19.8755, throughput=59.61K wps, wc=6521.09K
2018-06-26 20:16:22,115 - root - [Epoch 17 Batch 16000/34064] loss=2.9859, ppl=19.8037, throughput=60.63K wps, wc=6513.60K
2018-06-26 20:18:10,193 - root - [Epoch 17 Batch 16800/34064] loss=2.9897, ppl=19.8797, throughput=60.35K wps, wc=6522.57K
2018-06-26 20:19:57,706 - root - [Epoch 17 Batch 17600/34064] loss=2.9910, ppl=19.9047, throughput=60.65K wps, wc=6520.26K
2018-06-26 20:21:46,778 - root - [Epoch 17 Batch 18400/34064] loss=2.9902, ppl=19.8904, throughput=59.75K wps, wc=6517.26K
2018-06-26 20:23:34,722 - root - [Epoch 17 Batch 19200/34064] loss=2.9874, ppl=19.8346, throughput=60.41K wps, wc=6520.56K
2018-06-26 20:25:22,995 - root - [Epoch 17 Batch 20000/34064] loss=2.9966, ppl=20.0181, throughput=60.17K wps, wc=6514.28K
2018-06-26 20:27:09,989 - root - [Epoch 17 Batch 20800/34064] loss=2.9768, ppl=19.6240, throughput=60.98K wps, wc=6524.39K
2018-06-26 20:28:58,305 - root - [Epoch 17 Batch 21600/34064] loss=2.9872, ppl=19.8301, throughput=60.18K wps, wc=6518.83K
2018-06-26 20:30:45,808 - root - [Epoch 17 Batch 22400/34064] loss=2.9840, ppl=19.7672, throughput=60.62K wps, wc=6517.35K
2018-06-26 20:32:31,421 - root - [Epoch 17 Batch 23200/34064] loss=2.9783, ppl=19.6534, throughput=61.84K wps, wc=6531.09K
2018-06-26 20:34:19,408 - root - [Epoch 17 Batch 24000/34064] loss=2.9927, ppl=19.9389, throughput=60.38K wps, wc=6520.15K
2018-06-26 20:36:07,040 - root - [Epoch 17 Batch 24800/34064] loss=3.0025, ppl=20.1354, throughput=60.59K wps, wc=6521.22K
2018-06-26 20:37:54,726 - root - [Epoch 17 Batch 25600/34064] loss=2.9849, ppl=19.7852, throughput=60.56K wps, wc=6521.00K
2018-06-26 20:39:43,851 - root - [Epoch 17 Batch 26400/34064] loss=3.0110, ppl=20.3072, throughput=59.76K wps, wc=6521.37K
2018-06-26 20:41:31,525 - root - [Epoch 17 Batch 27200/34064] loss=2.9887, ppl=19.8606, throughput=60.54K wps, wc=6518.94K
2018-06-26 20:43:18,714 - root - [Epoch 17 Batch 28000/34064] loss=2.9881, ppl=19.8485, throughput=60.84K wps, wc=6521.55K
2018-06-26 20:45:07,261 - root - [Epoch 17 Batch 28800/34064] loss=2.9894, ppl=19.8742, throughput=60.08K wps, wc=6521.74K
2018-06-26 20:46:54,554 - root - [Epoch 17 Batch 29600/34064] loss=2.9838, ppl=19.7635, throughput=60.80K wps, wc=6523.08K
2018-06-26 20:48:42,340 - root - [Epoch 17 Batch 30400/34064] loss=2.9950, ppl=19.9849, throughput=60.46K wps, wc=6516.74K
2018-06-26 20:50:30,877 - root - [Epoch 17 Batch 31200/34064] loss=2.9971, ppl=20.0271, throughput=60.08K wps, wc=6521.33K
2018-06-26 20:52:17,381 - root - [Epoch 17 Batch 32000/34064] loss=2.9847, ppl=19.7796, throughput=61.32K wps, wc=6530.37K
2018-06-26 20:54:03,954 - root - [Epoch 17 Batch 32800/34064] loss=2.9925, ppl=19.9358, throughput=61.21K wps, wc=6523.80K
2018-06-26 20:55:50,878 - root - [Epoch 17 Batch 33600/34064] loss=2.9828, ppl=19.7435, throughput=61.06K wps, wc=6529.14K
2018-06-26 21:01:27,228 - root - [Epoch 17] valid Loss=1.5267, valid ppl=4.6028, valid bleu=25.27
2018-06-26 21:06:00,487 - root - [Epoch 17] test Loss=1.3577, test ppl=3.8871, test bleu=26.17
2018-06-26 21:07:51,215 - root - [Epoch 18 Batch 800/34064] loss=2.9635, ppl=19.3656, throughput=59.14K wps, wc=6521.15K
2018-06-26 21:09:38,352 - root - [Epoch 18 Batch 1600/34064] loss=2.9584, ppl=19.2663, throughput=60.82K wps, wc=6516.23K
2018-06-26 21:11:26,952 - root - [Epoch 18 Batch 2400/34064] loss=2.9693, ppl=19.4781, throughput=60.06K wps, wc=6522.73K
2018-06-26 21:13:14,183 - root - [Epoch 18 Batch 3200/34064] loss=2.9518, ppl=19.1399, throughput=60.80K wps, wc=6519.96K
2018-06-26 21:15:02,496 - root - [Epoch 18 Batch 4000/34064] loss=2.9645, ppl=19.3849, throughput=60.19K wps, wc=6519.60K
2018-06-26 21:16:49,645 - root - [Epoch 18 Batch 4800/34064] loss=2.9674, ppl=19.4413, throughput=60.88K wps, wc=6523.24K
2018-06-26 21:18:36,714 - root - [Epoch 18 Batch 5600/34064] loss=2.9710, ppl=19.5114, throughput=60.92K wps, wc=6522.16K
2018-06-26 21:20:23,633 - root - [Epoch 18 Batch 6400/34064] loss=2.9741, ppl=19.5711, throughput=60.99K wps, wc=6520.77K
2018-06-26 21:22:09,878 - root - [Epoch 18 Batch 7200/34064] loss=2.9689, ppl=19.4702, throughput=61.30K wps, wc=6512.61K
2018-06-26 21:23:56,071 - root - [Epoch 18 Batch 8000/34064] loss=2.9572, ppl=19.2431, throughput=61.47K wps, wc=6527.94K
2018-06-26 21:25:41,439 - root - [Epoch 18 Batch 8800/34064] loss=2.9695, ppl=19.4830, throughput=61.95K wps, wc=6527.70K
2018-06-26 21:27:29,147 - root - [Epoch 18 Batch 9600/34064] loss=2.9745, ppl=19.5806, throughput=60.51K wps, wc=6516.98K
2018-06-26 21:29:15,956 - root - [Epoch 18 Batch 10400/34064] loss=2.9682, ppl=19.4560, throughput=61.08K wps, wc=6523.74K
2018-06-26 21:31:02,259 - root - [Epoch 18 Batch 11200/34064] loss=2.9685, ppl=19.4618, throughput=61.35K wps, wc=6521.79K
2018-06-26 21:32:50,087 - root - [Epoch 18 Batch 12000/34064] loss=2.9924, ppl=19.9339, throughput=60.39K wps, wc=6512.10K
2018-06-26 21:34:36,481 - root - [Epoch 18 Batch 12800/34064] loss=2.9622, ppl=19.3409, throughput=61.32K wps, wc=6523.69K
2018-06-26 21:36:22,495 - root - [Epoch 18 Batch 13600/34064] loss=2.9633, ppl=19.3611, throughput=61.59K wps, wc=6529.53K
2018-06-26 21:38:09,340 - root - [Epoch 18 Batch 14400/34064] loss=2.9685, ppl=19.4622, throughput=61.04K wps, wc=6521.40K
2018-06-26 21:39:54,786 - root - [Epoch 18 Batch 15200/34064] loss=2.9818, ppl=19.7240, throughput=61.86K wps, wc=6522.79K
2018-06-26 21:41:41,608 - root - [Epoch 18 Batch 16000/34064] loss=2.9696, ppl=19.4835, throughput=61.04K wps, wc=6519.93K
2018-06-26 21:43:29,397 - root - [Epoch 18 Batch 16800/34064] loss=2.9744, ppl=19.5776, throughput=60.54K wps, wc=6525.64K
2018-06-26 21:45:16,058 - root - [Epoch 18 Batch 17600/34064] loss=2.9770, ppl=19.6282, throughput=61.16K wps, wc=6523.01K
2018-06-26 21:47:03,626 - root - [Epoch 18 Batch 18400/34064] loss=2.9829, ppl=19.7460, throughput=60.61K wps, wc=6519.78K
2018-06-26 21:48:50,885 - root - [Epoch 18 Batch 19200/34064] loss=2.9755, ppl=19.6001, throughput=60.84K wps, wc=6525.23K
2018-06-26 21:50:39,329 - root - [Epoch 18 Batch 20000/34064] loss=2.9914, ppl=19.9137, throughput=60.10K wps, wc=6517.77K
2018-06-26 21:52:26,792 - root - [Epoch 18 Batch 20800/34064] loss=2.9796, ppl=19.6794, throughput=60.70K wps, wc=6522.95K
2018-06-26 21:54:13,712 - root - [Epoch 18 Batch 21600/34064] loss=2.9852, ppl=19.7897, throughput=61.03K wps, wc=6524.85K
2018-06-26 21:56:01,213 - root - [Epoch 18 Batch 22400/34064] loss=2.9802, ppl=19.6917, throughput=60.68K wps, wc=6522.85K
2018-06-26 21:57:49,449 - root - [Epoch 18 Batch 23200/34064] loss=2.9829, ppl=19.7452, throughput=60.27K wps, wc=6523.49K
2018-06-26 21:59:36,648 - root - [Epoch 18 Batch 24000/34064] loss=2.9824, ppl=19.7356, throughput=60.86K wps, wc=6523.81K
2018-06-26 22:01:24,811 - root - [Epoch 18 Batch 24800/34064] loss=2.9773, ppl=19.6342, throughput=60.29K wps, wc=6520.95K
2018-06-26 22:03:13,185 - root - [Epoch 18 Batch 25600/34064] loss=2.9775, ppl=19.6383, throughput=60.13K wps, wc=6516.11K
2018-06-26 22:05:00,579 - root - [Epoch 18 Batch 26400/34064] loss=2.9885, ppl=19.8568, throughput=60.71K wps, wc=6519.31K
2018-06-26 22:06:47,611 - root - [Epoch 18 Batch 27200/34064] loss=2.9908, ppl=19.9024, throughput=60.94K wps, wc=6522.81K
2018-06-26 22:08:33,741 - root - [Epoch 18 Batch 28000/34064] loss=2.9811, ppl=19.7099, throughput=61.38K wps, wc=6513.81K
2018-06-26 22:10:20,733 - root - [Epoch 18 Batch 28800/34064] loss=2.9752, ppl=19.5932, throughput=60.94K wps, wc=6520.25K
2018-06-26 22:12:10,304 - root - [Epoch 18 Batch 29600/34064] loss=2.9951, ppl=19.9883, throughput=59.34K wps, wc=6501.69K
2018-06-26 22:13:57,167 - root - [Epoch 18 Batch 30400/34064] loss=2.9721, ppl=19.5338, throughput=61.05K wps, wc=6523.73K
2018-06-26 22:15:43,683 - root - [Epoch 18 Batch 31200/34064] loss=2.9691, ppl=19.4742, throughput=61.26K wps, wc=6525.21K
2018-06-26 22:17:31,749 - root - [Epoch 18 Batch 32000/34064] loss=2.9901, ppl=19.8885, throughput=60.34K wps, wc=6520.94K
2018-06-26 22:19:18,483 - root - [Epoch 18 Batch 32800/34064] loss=2.9871, ppl=19.8287, throughput=61.11K wps, wc=6522.27K
2018-06-26 22:21:05,606 - root - [Epoch 18 Batch 33600/34064] loss=2.9924, ppl=19.9331, throughput=60.91K wps, wc=6525.21K
2018-06-26 22:26:42,510 - root - [Epoch 18] valid Loss=1.5245, valid ppl=4.5929, valid bleu=25.43
2018-06-26 22:31:17,474 - root - [Epoch 18] test Loss=1.3489, test ppl=3.8532, test bleu=26.11
2018-06-26 22:33:09,337 - root - [Epoch 19 Batch 800/34064] loss=2.9609, ppl=19.3156, throughput=58.42K wps, wc=6508.81K
2018-06-26 22:34:56,735 - root - [Epoch 19 Batch 1600/34064] loss=2.9486, ppl=19.0793, throughput=60.70K wps, wc=6518.55K
2018-06-26 22:36:44,956 - root - [Epoch 19 Batch 2400/34064] loss=2.9608, ppl=19.3133, throughput=60.24K wps, wc=6518.93K
2018-06-26 22:38:32,275 - root - [Epoch 19 Batch 3200/34064] loss=2.9559, ppl=19.2196, throughput=60.78K wps, wc=6523.12K
2018-06-26 22:40:19,101 - root - [Epoch 19 Batch 4000/34064] loss=2.9513, ppl=19.1309, throughput=61.02K wps, wc=6518.93K
2018-06-26 22:42:06,841 - root - [Epoch 19 Batch 4800/34064] loss=2.9635, ppl=19.3650, throughput=60.57K wps, wc=6525.81K
2018-06-26 22:43:53,793 - root - [Epoch 19 Batch 5600/34064] loss=2.9475, ppl=19.0583, throughput=61.05K wps, wc=6528.98K
2018-06-26 22:45:41,585 - root - [Epoch 19 Batch 6400/34064] loss=2.9503, ppl=19.1119, throughput=60.57K wps, wc=6528.95K
2018-06-26 22:47:29,293 - root - [Epoch 19 Batch 7200/34064] loss=2.9535, ppl=19.1739, throughput=60.56K wps, wc=6523.05K
2018-06-26 22:49:15,693 - root - [Epoch 19 Batch 8000/34064] loss=2.9582, ppl=19.2633, throughput=61.27K wps, wc=6519.11K
2018-06-26 22:51:02,698 - root - [Epoch 19 Batch 8800/34064] loss=2.9707, ppl=19.5053, throughput=60.96K wps, wc=6523.29K
2018-06-26 22:52:50,001 - root - [Epoch 19 Batch 9600/34064] loss=2.9717, ppl=19.5244, throughput=60.79K wps, wc=6522.43K
2018-06-26 22:54:37,783 - root - [Epoch 19 Batch 10400/34064] loss=2.9781, ppl=19.6512, throughput=60.47K wps, wc=6517.04K
2018-06-26 22:56:24,803 - root - [Epoch 19 Batch 11200/34064] loss=2.9634, ppl=19.3640, throughput=60.94K wps, wc=6522.17K
2018-06-26 22:58:10,695 - root - [Epoch 19 Batch 12000/34064] loss=2.9698, ppl=19.4883, throughput=61.62K wps, wc=6525.03K
2018-06-26 22:59:59,082 - root - [Epoch 19 Batch 12800/34064] loss=2.9606, ppl=19.3095, throughput=60.17K wps, wc=6521.54K
2018-06-26 23:01:46,349 - root - [Epoch 19 Batch 13600/34064] loss=2.9730, ppl=19.5512, throughput=60.77K wps, wc=6518.98K
2018-06-26 23:03:33,413 - root - [Epoch 19 Batch 14400/34064] loss=2.9704, ppl=19.5004, throughput=60.97K wps, wc=6527.57K
2018-06-26 23:05:19,817 - root - [Epoch 19 Batch 15200/34064] loss=2.9643, ppl=19.3816, throughput=61.28K wps, wc=6520.32K
2018-06-26 23:07:07,884 - root - [Epoch 19 Batch 16000/34064] loss=2.9690, ppl=19.4727, throughput=60.35K wps, wc=6521.66K
2018-06-26 23:08:55,635 - root - [Epoch 19 Batch 16800/34064] loss=2.9572, ppl=19.2438, throughput=60.50K wps, wc=6518.67K
2018-06-26 23:10:42,675 - root - [Epoch 19 Batch 17600/34064] loss=2.9794, ppl=19.6764, throughput=60.92K wps, wc=6521.02K
2018-06-26 23:12:29,912 - root - [Epoch 19 Batch 18400/34064] loss=2.9650, ppl=19.3938, throughput=60.82K wps, wc=6522.04K
2018-06-26 23:14:18,260 - root - [Epoch 19 Batch 19200/34064] loss=2.9763, ppl=19.6157, throughput=60.22K wps, wc=6524.22K
2018-06-26 23:16:05,932 - root - [Epoch 19 Batch 20000/34064] loss=2.9880, ppl=19.8456, throughput=60.57K wps, wc=6521.41K
2018-06-26 23:17:53,827 - root - [Epoch 19 Batch 20800/34064] loss=2.9709, ppl=19.5099, throughput=60.42K wps, wc=6519.20K
2018-06-26 23:19:42,079 - root - [Epoch 19 Batch 21600/34064] loss=2.9616, ppl=19.3298, throughput=60.25K wps, wc=6522.14K
2018-06-26 23:21:29,512 - root - [Epoch 19 Batch 22400/34064] loss=2.9678, ppl=19.4486, throughput=60.76K wps, wc=6527.65K
2018-06-26 23:23:17,128 - root - [Epoch 19 Batch 23200/34064] loss=2.9662, ppl=19.4175, throughput=60.60K wps, wc=6521.06K
2018-06-26 23:25:04,214 - root - [Epoch 19 Batch 24000/34064] loss=2.9795, ppl=19.6788, throughput=60.85K wps, wc=6516.31K
2018-06-26 23:26:52,641 - root - [Epoch 19 Batch 24800/34064] loss=2.9869, ppl=19.8246, throughput=60.11K wps, wc=6517.92K
2018-06-26 23:28:40,534 - root - [Epoch 19 Batch 25600/34064] loss=2.9863, ppl=19.8121, throughput=60.37K wps, wc=6513.02K
2018-06-26 23:30:28,709 - root - [Epoch 19 Batch 26400/34064] loss=2.9789, ppl=19.6655, throughput=60.32K wps, wc=6525.13K
2018-06-26 23:32:15,687 - root - [Epoch 19 Batch 27200/34064] loss=2.9627, ppl=19.3504, throughput=61.00K wps, wc=6525.85K
2018-06-26 23:34:03,662 - root - [Epoch 19 Batch 28000/34064] loss=2.9791, ppl=19.6696, throughput=60.41K wps, wc=6522.78K
2018-06-26 23:35:50,663 - root - [Epoch 19 Batch 28800/34064] loss=2.9771, ppl=19.6312, throughput=60.97K wps, wc=6524.27K
2018-06-26 23:37:37,248 - root - [Epoch 19 Batch 29600/34064] loss=2.9792, ppl=19.6725, throughput=61.12K wps, wc=6514.28K
2018-06-26 23:39:25,701 - root - [Epoch 19 Batch 30400/34064] loss=2.9901, ppl=19.8875, throughput=60.06K wps, wc=6513.88K
2018-06-26 23:41:13,544 - root - [Epoch 19 Batch 31200/34064] loss=2.9639, ppl=19.3733, throughput=60.50K wps, wc=6524.11K
2018-06-26 23:43:01,107 - root - [Epoch 19 Batch 32000/34064] loss=2.9908, ppl=19.9018, throughput=60.53K wps, wc=6511.09K
2018-06-26 23:44:48,047 - root - [Epoch 19 Batch 32800/34064] loss=2.9826, ppl=19.7387, throughput=60.99K wps, wc=6522.73K
2018-06-26 23:46:34,914 - root - [Epoch 19 Batch 33600/34064] loss=2.9677, ppl=19.4467, throughput=61.03K wps, wc=6522.37K
2018-06-26 23:52:12,394 - root - [Epoch 19] valid Loss=1.5178, valid ppl=4.5623, valid bleu=25.50
2018-06-26 23:56:47,677 - root - [Epoch 19] test Loss=1.3458, test ppl=3.8413, test bleu=26.21
2018-06-26 23:58:42,234 - root - [Epoch 20 Batch 800/34064] loss=2.9394, ppl=18.9047, throughput=57.06K wps, wc=6510.82K
2018-06-27 00:00:29,143 - root - [Epoch 20 Batch 1600/34064] loss=2.9448, ppl=19.0063, throughput=60.95K wps, wc=6516.38K
2018-06-27 00:02:16,183 - root - [Epoch 20 Batch 2400/34064] loss=2.9454, ppl=19.0186, throughput=60.89K wps, wc=6517.74K
2018-06-27 00:04:03,561 - root - [Epoch 20 Batch 3200/34064] loss=2.9497, ppl=19.1008, throughput=60.73K wps, wc=6521.07K
2018-06-27 00:05:51,072 - root - [Epoch 20 Batch 4000/34064] loss=2.9590, ppl=19.2786, throughput=60.61K wps, wc=6516.15K
2018-06-27 00:07:38,196 - root - [Epoch 20 Batch 4800/34064] loss=2.9590, ppl=19.2780, throughput=60.80K wps, wc=6513.13K
2018-06-27 00:09:24,916 - root - [Epoch 20 Batch 5600/34064] loss=2.9353, ppl=18.8281, throughput=61.13K wps, wc=6523.60K
2018-06-27 00:11:12,752 - root - [Epoch 20 Batch 6400/34064] loss=2.9709, ppl=19.5097, throughput=60.42K wps, wc=6514.94K
2018-06-27 00:13:02,041 - root - [Epoch 20 Batch 7200/34064] loss=2.9593, ppl=19.2849, throughput=59.63K wps, wc=6516.85K
2018-06-27 00:14:47,764 - root - [Epoch 20 Batch 8000/34064] loss=2.9385, ppl=18.8874, throughput=61.68K wps, wc=6521.06K
2018-06-27 00:16:33,442 - root - [Epoch 20 Batch 8800/34064] loss=2.9592, ppl=19.2830, throughput=61.73K wps, wc=6523.24K
2018-06-27 00:18:19,252 - root - [Epoch 20 Batch 9600/34064] loss=2.9615, ppl=19.3262, throughput=61.64K wps, wc=6522.56K
2018-06-27 00:20:06,352 - root - [Epoch 20 Batch 10400/34064] loss=2.9571, ppl=19.2417, throughput=60.88K wps, wc=6519.78K
2018-06-27 00:21:53,638 - root - [Epoch 20 Batch 11200/34064] loss=2.9628, ppl=19.3526, throughput=60.80K wps, wc=6522.90K
2018-06-27 00:23:41,477 - root - [Epoch 20 Batch 12000/34064] loss=2.9679, ppl=19.4511, throughput=60.47K wps, wc=6521.25K
2018-06-27 00:25:28,681 - root - [Epoch 20 Batch 12800/34064] loss=2.9727, ppl=19.5442, throughput=60.78K wps, wc=6516.07K
2018-06-27 00:27:16,084 - root - [Epoch 20 Batch 13600/34064] loss=2.9689, ppl=19.4709, throughput=60.68K wps, wc=6517.66K
2018-06-27 00:29:03,987 - root - [Epoch 20 Batch 14400/34064] loss=2.9737, ppl=19.5651, throughput=60.35K wps, wc=6512.02K
2018-06-27 00:30:50,313 - root - [Epoch 20 Batch 15200/34064] loss=2.9556, ppl=19.2135, throughput=61.35K wps, wc=6522.98K
2018-06-27 00:32:37,881 - root - [Epoch 20 Batch 16000/34064] loss=2.9618, ppl=19.3334, throughput=60.68K wps, wc=6527.22K
2018-06-27 00:34:25,058 - root - [Epoch 20 Batch 16800/34064] loss=2.9653, ppl=19.4001, throughput=60.89K wps, wc=6526.29K
2018-06-27 00:36:12,515 - root - [Epoch 20 Batch 17600/34064] loss=2.9729, ppl=19.5477, throughput=60.65K wps, wc=6517.71K
2018-06-27 00:37:59,712 - root - [Epoch 20 Batch 18400/34064] loss=2.9667, ppl=19.4270, throughput=60.91K wps, wc=6528.84K
2018-06-27 00:39:48,773 - root - [Epoch 20 Batch 19200/34064] loss=2.9754, ppl=19.5971, throughput=59.76K wps, wc=6517.75K
2018-06-27 00:41:36,695 - root - [Epoch 20 Batch 20000/34064] loss=2.9641, ppl=19.3766, throughput=60.50K wps, wc=6529.21K
2018-06-27 00:43:24,425 - root - [Epoch 20 Batch 20800/34064] loss=2.9677, ppl=19.4469, throughput=60.55K wps, wc=6522.96K
2018-06-27 00:45:11,545 - root - [Epoch 20 Batch 21600/34064] loss=2.9642, ppl=19.3794, throughput=60.89K wps, wc=6522.61K
2018-06-27 00:46:59,828 - root - [Epoch 20 Batch 22400/34064] loss=2.9697, ppl=19.4866, throughput=60.24K wps, wc=6523.13K
2018-06-27 00:48:46,555 - root - [Epoch 20 Batch 23200/34064] loss=2.9847, ppl=19.7802, throughput=61.10K wps, wc=6520.44K
2018-06-27 00:50:33,570 - root - [Epoch 20 Batch 24000/34064] loss=2.9649, ppl=19.3928, throughput=60.96K wps, wc=6523.05K
2018-06-27 00:52:20,557 - root - [Epoch 20 Batch 24800/34064] loss=2.9797, ppl=19.6820, throughput=60.92K wps, wc=6518.13K
2018-06-27 00:54:07,986 - root - [Epoch 20 Batch 25600/34064] loss=2.9752, ppl=19.5926, throughput=60.74K wps, wc=6525.52K
2018-06-27 00:55:57,647 - root - [Epoch 20 Batch 26400/34064] loss=2.9742, ppl=19.5738, throughput=59.45K wps, wc=6519.13K
2018-06-27 00:57:44,384 - root - [Epoch 20 Batch 27200/34064] loss=2.9612, ppl=19.3219, throughput=61.14K wps, wc=6525.84K
2018-06-27 00:59:32,429 - root - [Epoch 20 Batch 28000/34064] loss=2.9596, ppl=19.2903, throughput=60.37K wps, wc=6522.22K
2018-06-27 01:01:20,209 - root - [Epoch 20 Batch 28800/34064] loss=2.9699, ppl=19.4897, throughput=60.52K wps, wc=6522.74K
2018-06-27 01:03:08,352 - root - [Epoch 20 Batch 29600/34064] loss=2.9692, ppl=19.4754, throughput=60.35K wps, wc=6526.62K
2018-06-27 01:04:55,282 - root - [Epoch 20 Batch 30400/34064] loss=2.9702, ppl=19.4959, throughput=61.03K wps, wc=6525.51K
2018-06-27 01:06:42,467 - root - [Epoch 20 Batch 31200/34064] loss=2.9715, ppl=19.5207, throughput=60.84K wps, wc=6520.78K
2018-06-27 01:08:30,240 - root - [Epoch 20 Batch 32000/34064] loss=2.9688, ppl=19.4693, throughput=60.50K wps, wc=6520.50K
2018-06-27 01:10:17,298 - root - [Epoch 20 Batch 32800/34064] loss=2.9701, ppl=19.4936, throughput=60.97K wps, wc=6527.14K
2018-06-27 01:12:03,064 - root - [Epoch 20 Batch 33600/34064] loss=2.9644, ppl=19.3834, throughput=61.65K wps, wc=6520.80K
2018-06-27 01:17:40,863 - root - [Epoch 20] valid Loss=1.5215, valid ppl=4.5793, valid bleu=25.51
2018-06-27 01:22:14,763 - root - [Epoch 20] test Loss=1.3464, test ppl=3.8435, test bleu=25.89
2018-06-27 01:24:07,740 - root - [Epoch 21 Batch 800/34064] loss=2.9439, ppl=18.9906, throughput=57.96K wps, wc=6521.36K
2018-06-27 01:25:55,592 - root - [Epoch 21 Batch 1600/34064] loss=2.9294, ppl=18.7170, throughput=60.54K wps, wc=6529.56K
2018-06-27 01:27:43,783 - root - [Epoch 21 Batch 2400/34064] loss=2.9382, ppl=18.8809, throughput=60.30K wps, wc=6523.95K
2018-06-27 01:29:30,722 - root - [Epoch 21 Batch 3200/34064] loss=2.9298, ppl=18.7233, throughput=60.99K wps, wc=6521.76K
2018-06-27 01:31:20,480 - root - [Epoch 21 Batch 4000/34064] loss=2.9508, ppl=19.1203, throughput=59.43K wps, wc=6522.48K
2018-06-27 01:33:08,585 - root - [Epoch 21 Batch 4800/34064] loss=2.9614, ppl=19.3247, throughput=60.33K wps, wc=6521.83K
2018-06-27 01:34:58,651 - root - [Epoch 21 Batch 5600/34064] loss=2.9609, ppl=19.3161, throughput=59.21K wps, wc=6517.30K
2018-06-27 01:36:46,286 - root - [Epoch 21 Batch 6400/34064] loss=2.9566, ppl=19.2323, throughput=60.57K wps, wc=6519.46K
2018-06-27 01:38:33,686 - root - [Epoch 21 Batch 7200/34064] loss=2.9517, ppl=19.1381, throughput=60.63K wps, wc=6511.63K
2018-06-27 01:40:19,936 - root - [Epoch 21 Batch 8000/34064] loss=2.9508, ppl=19.1206, throughput=61.41K wps, wc=6525.00K
2018-06-27 01:42:08,893 - root - [Epoch 21 Batch 8800/34064] loss=2.9556, ppl=19.2128, throughput=59.86K wps, wc=6521.84K
2018-06-27 01:43:55,816 - root - [Epoch 21 Batch 9600/34064] loss=2.9572, ppl=19.2435, throughput=61.06K wps, wc=6528.48K
2018-06-27 01:45:43,100 - root - [Epoch 21 Batch 10400/34064] loss=2.9617, ppl=19.3308, throughput=60.75K wps, wc=6516.98K
2018-06-27 01:47:29,184 - root - [Epoch 21 Batch 11200/34064] loss=2.9506, ppl=19.1172, throughput=61.47K wps, wc=6521.09K
2018-06-27 01:49:16,014 - root - [Epoch 21 Batch 12000/34064] loss=2.9443, ppl=18.9971, throughput=61.07K wps, wc=6524.40K
2018-06-27 01:51:01,832 - root - [Epoch 21 Batch 12800/34064] loss=2.9483, ppl=19.0735, throughput=61.67K wps, wc=6525.81K
2018-06-27 01:52:49,491 - root - [Epoch 21 Batch 13600/34064] loss=2.9607, ppl=19.3114, throughput=60.57K wps, wc=6520.38K
2018-06-27 01:54:38,321 - root - [Epoch 21 Batch 14400/34064] loss=2.9697, ppl=19.4855, throughput=59.93K wps, wc=6522.02K
2018-06-27 01:56:27,661 - root - [Epoch 21 Batch 15200/34064] loss=2.9794, ppl=19.6768, throughput=59.63K wps, wc=6520.31K
2018-06-27 01:58:16,246 - root - [Epoch 21 Batch 16000/34064] loss=2.9464, ppl=19.0381, throughput=60.10K wps, wc=6525.64K
2018-06-27 02:00:03,641 - root - [Epoch 21 Batch 16800/34064] loss=2.9465, ppl=19.0397, throughput=60.73K wps, wc=6522.45K
2018-06-27 02:01:53,198 - root - [Epoch 21 Batch 17600/34064] loss=2.9687, ppl=19.4666, throughput=59.50K wps, wc=6519.05K
2018-06-27 02:03:40,945 - root - [Epoch 21 Batch 18400/34064] loss=2.9522, ppl=19.1487, throughput=60.37K wps, wc=6504.30K
2018-06-27 02:05:28,253 - root - [Epoch 21 Batch 19200/34064] loss=2.9560, ppl=19.2213, throughput=60.74K wps, wc=6518.02K
2018-06-27 02:07:15,228 - root - [Epoch 21 Batch 20000/34064] loss=2.9588, ppl=19.2747, throughput=60.97K wps, wc=6521.96K
2018-06-27 02:09:02,850 - root - [Epoch 21 Batch 20800/34064] loss=2.9559, ppl=19.2193, throughput=60.59K wps, wc=6520.55K
2018-06-27 02:10:51,550 - root - [Epoch 21 Batch 21600/34064] loss=2.9534, ppl=19.1709, throughput=60.01K wps, wc=6522.63K
2018-06-27 02:12:39,616 - root - [Epoch 21 Batch 22400/34064] loss=2.9668, ppl=19.4295, throughput=60.38K wps, wc=6525.19K
2018-06-27 02:14:26,356 - root - [Epoch 21 Batch 23200/34064] loss=2.9505, ppl=19.1150, throughput=61.10K wps, wc=6521.85K
2018-06-27 02:16:13,811 - root - [Epoch 21 Batch 24000/34064] loss=2.9609, ppl=19.3146, throughput=60.71K wps, wc=6523.28K
2018-06-27 02:18:01,290 - root - [Epoch 21 Batch 24800/34064] loss=2.9577, ppl=19.2528, throughput=60.70K wps, wc=6523.52K
2018-06-27 02:19:49,145 - root - [Epoch 21 Batch 25600/34064] loss=2.9635, ppl=19.3652, throughput=60.44K wps, wc=6518.18K
2018-06-27 02:21:36,365 - root - [Epoch 21 Batch 26400/34064] loss=2.9745, ppl=19.5804, throughput=60.82K wps, wc=6520.81K
2018-06-27 02:23:24,454 - root - [Epoch 21 Batch 27200/34064] loss=2.9729, ppl=19.5480, throughput=60.35K wps, wc=6522.69K
2018-06-27 02:25:13,695 - root - [Epoch 21 Batch 28000/34064] loss=2.9745, ppl=19.5804, throughput=59.68K wps, wc=6519.96K
2018-06-27 02:27:00,581 - root - [Epoch 21 Batch 28800/34064] loss=2.9499, ppl=19.1044, throughput=61.08K wps, wc=6528.09K
2018-06-27 02:28:48,548 - root - [Epoch 21 Batch 29600/34064] loss=2.9679, ppl=19.4508, throughput=60.35K wps, wc=6515.74K
2018-06-27 02:30:36,587 - root - [Epoch 21 Batch 30400/34064] loss=2.9769, ppl=19.6276, throughput=60.26K wps, wc=6510.76K
2018-06-27 02:32:23,210 - root - [Epoch 21 Batch 31200/34064] loss=2.9582, ppl=19.2636, throughput=61.23K wps, wc=6528.33K
2018-06-27 02:34:10,880 - root - [Epoch 21 Batch 32000/34064] loss=2.9781, ppl=19.6504, throughput=60.49K wps, wc=6512.75K
2018-06-27 02:35:59,344 - root - [Epoch 21 Batch 32800/34064] loss=2.9709, ppl=19.5086, throughput=60.13K wps, wc=6521.92K
2018-06-27 02:37:47,571 - root - [Epoch 21 Batch 33600/34064] loss=2.9772, ppl=19.6332, throughput=60.28K wps, wc=6523.50K
2018-06-27 02:43:25,939 - root - [Epoch 21] valid Loss=1.5148, valid ppl=4.5485, valid bleu=25.54
2018-06-27 02:47:59,948 - root - [Epoch 21] test Loss=1.3454, test ppl=3.8395, test bleu=25.91
2018-06-27 02:49:51,394 - root - [Epoch 22 Batch 800/34064] loss=2.9402, ppl=18.9201, throughput=58.77K wps, wc=6522.85K
2018-06-27 02:51:38,916 - root - [Epoch 22 Batch 1600/34064] loss=2.9299, ppl=18.7254, throughput=60.68K wps, wc=6524.08K
2018-06-27 02:53:26,896 - root - [Epoch 22 Batch 2400/34064] loss=2.9379, ppl=18.8769, throughput=60.46K wps, wc=6528.14K
2018-06-27 02:55:14,810 - root - [Epoch 22 Batch 3200/34064] loss=2.9402, ppl=18.9197, throughput=60.45K wps, wc=6522.83K
2018-06-27 02:57:01,810 - root - [Epoch 22 Batch 4000/34064] loss=2.9375, ppl=18.8686, throughput=60.98K wps, wc=6524.33K
2018-06-27 02:58:50,330 - root - [Epoch 22 Batch 4800/34064] loss=2.9363, ppl=18.8452, throughput=60.04K wps, wc=6515.61K
2018-06-27 03:00:38,593 - root - [Epoch 22 Batch 5600/34064] loss=2.9509, ppl=19.1240, throughput=60.21K wps, wc=6518.49K
2018-06-27 03:02:26,345 - root - [Epoch 22 Batch 6400/34064] loss=2.9490, ppl=19.0865, throughput=60.49K wps, wc=6518.13K
2018-06-27 03:04:12,592 - root - [Epoch 22 Batch 7200/34064] loss=2.9385, ppl=18.8866, throughput=61.35K wps, wc=6517.90K
2018-06-27 03:06:00,114 - root - [Epoch 22 Batch 8000/34064] loss=2.9609, ppl=19.3154, throughput=60.71K wps, wc=6527.37K
2018-06-27 03:07:48,067 - root - [Epoch 22 Batch 8800/34064] loss=2.9483, ppl=19.0728, throughput=60.34K wps, wc=6513.80K
2018-06-27 03:09:34,507 - root - [Epoch 22 Batch 9600/34064] loss=2.9540, ppl=19.1833, throughput=61.27K wps, wc=6521.52K
2018-06-27 03:11:21,059 - root - [Epoch 22 Batch 10400/34064] loss=2.9465, ppl=19.0391, throughput=61.21K wps, wc=6522.47K
2018-06-27 03:13:08,661 - root - [Epoch 22 Batch 11200/34064] loss=2.9467, ppl=19.0434, throughput=60.54K wps, wc=6514.66K
2018-06-27 03:14:55,312 - root - [Epoch 22 Batch 12000/34064] loss=2.9449, ppl=19.0096, throughput=61.06K wps, wc=6512.08K
2018-06-27 03:16:41,553 - root - [Epoch 22 Batch 12800/34064] loss=2.9440, ppl=18.9926, throughput=61.43K wps, wc=6526.35K
2018-06-27 03:18:27,702 - root - [Epoch 22 Batch 13600/34064] loss=2.9375, ppl=18.8678, throughput=61.44K wps, wc=6521.74K
2018-06-27 03:20:15,455 - root - [Epoch 22 Batch 14400/34064] loss=2.9543, ppl=19.1889, throughput=60.50K wps, wc=6518.93K
2018-06-27 03:22:02,131 - root - [Epoch 22 Batch 15200/34064] loss=2.9470, ppl=19.0482, throughput=61.02K wps, wc=6509.65K
2018-06-27 03:23:49,704 - root - [Epoch 22 Batch 16000/34064] loss=2.9441, ppl=18.9943, throughput=60.63K wps, wc=6521.64K
2018-06-27 03:25:37,592 - root - [Epoch 22 Batch 16800/34064] loss=2.9536, ppl=19.1753, throughput=60.47K wps, wc=6523.58K
2018-06-27 03:27:25,254 - root - [Epoch 22 Batch 17600/34064] loss=2.9647, ppl=19.3890, throughput=60.59K wps, wc=6523.49K
2018-06-27 03:29:13,123 - root - [Epoch 22 Batch 18400/34064] loss=2.9573, ppl=19.2467, throughput=60.46K wps, wc=6521.62K
2018-06-27 03:31:01,022 - root - [Epoch 22 Batch 19200/34064] loss=2.9626, ppl=19.3474, throughput=60.39K wps, wc=6515.89K
2018-06-27 03:32:49,649 - root - [Epoch 22 Batch 20000/34064] loss=2.9631, ppl=19.3588, throughput=60.01K wps, wc=6518.74K
2018-06-27 03:34:36,969 - root - [Epoch 22 Batch 20800/34064] loss=2.9452, ppl=19.0154, throughput=60.78K wps, wc=6523.32K
2018-06-27 03:36:25,030 - root - [Epoch 22 Batch 21600/34064] loss=2.9662, ppl=19.4187, throughput=60.37K wps, wc=6523.76K
2018-06-27 03:38:12,141 - root - [Epoch 22 Batch 22400/34064] loss=2.9629, ppl=19.3547, throughput=60.87K wps, wc=6519.97K
2018-06-27 03:39:59,056 - root - [Epoch 22 Batch 23200/34064] loss=2.9511, ppl=19.1278, throughput=60.96K wps, wc=6518.00K
2018-06-27 03:41:45,443 - root - [Epoch 22 Batch 24000/34064] loss=2.9461, ppl=19.0310, throughput=61.30K wps, wc=6521.78K
2018-06-27 03:43:31,222 - root - [Epoch 22 Batch 24800/34064] loss=2.9468, ppl=19.0442, throughput=61.70K wps, wc=6526.92K
2018-06-27 03:45:19,179 - root - [Epoch 22 Batch 25600/34064] loss=2.9705, ppl=19.5025, throughput=60.38K wps, wc=6518.06K
2018-06-27 03:47:07,276 - root - [Epoch 22 Batch 26400/34064] loss=2.9699, ppl=19.4906, throughput=60.29K wps, wc=6517.58K
2018-06-27 03:48:53,602 - root - [Epoch 22 Batch 27200/34064] loss=2.9670, ppl=19.4338, throughput=61.34K wps, wc=6521.79K
2018-06-27 03:50:41,093 - root - [Epoch 22 Batch 28000/34064] loss=2.9599, ppl=19.2957, throughput=60.74K wps, wc=6528.86K
2018-06-27 03:52:28,872 - root - [Epoch 22 Batch 28800/34064] loss=2.9668, ppl=19.4292, throughput=60.48K wps, wc=6518.85K
2018-06-27 03:54:15,738 - root - [Epoch 22 Batch 29600/34064] loss=2.9560, ppl=19.2200, throughput=61.04K wps, wc=6523.42K
2018-06-27 03:56:02,785 - root - [Epoch 22 Batch 30400/34064] loss=2.9597, ppl=19.2918, throughput=60.96K wps, wc=6525.28K
2018-06-27 03:57:50,044 - root - [Epoch 22 Batch 31200/34064] loss=2.9596, ppl=19.2901, throughput=60.83K wps, wc=6524.47K
2018-06-27 03:59:37,134 - root - [Epoch 22 Batch 32000/34064] loss=2.9582, ppl=19.2642, throughput=60.91K wps, wc=6523.00K
2018-06-27 04:01:24,414 - root - [Epoch 22 Batch 32800/34064] loss=2.9753, ppl=19.5956, throughput=60.79K wps, wc=6521.43K
2018-06-27 04:03:10,841 - root - [Epoch 22 Batch 33600/34064] loss=2.9635, ppl=19.3663, throughput=61.31K wps, wc=6524.58K
2018-06-27 04:08:52,001 - root - [Epoch 22] valid Loss=1.5080, valid ppl=4.5175, valid bleu=25.73
2018-06-27 04:13:30,607 - root - [Epoch 22] test Loss=1.3400, test ppl=3.8190, test bleu=26.07
2018-06-27 04:13:30,614 - root - Save best parameters to transformer_en_de_u512/valid_best.params
2018-06-27 04:15:25,251 - root - [Epoch 23 Batch 800/34064] loss=2.9300, ppl=18.7280, throughput=58.17K wps, wc=6521.44K
2018-06-27 04:17:12,558 - root - [Epoch 23 Batch 1600/34064] loss=2.9166, ppl=18.4782, throughput=60.72K wps, wc=6515.38K
2018-06-27 04:19:00,959 - root - [Epoch 23 Batch 2400/34064] loss=2.9426, ppl=18.9650, throughput=60.16K wps, wc=6521.24K
2018-06-27 04:20:48,997 - root - [Epoch 23 Batch 3200/34064] loss=2.9332, ppl=18.7873, throughput=60.42K wps, wc=6527.10K
2018-06-27 04:22:37,388 - root - [Epoch 23 Batch 4000/34064] loss=2.9413, ppl=18.9402, throughput=60.12K wps, wc=6516.31K
2018-06-27 04:24:23,717 - root - [Epoch 23 Batch 4800/34064] loss=2.9320, ppl=18.7657, throughput=61.37K wps, wc=6524.91K
2018-06-27 04:26:12,031 - root - [Epoch 23 Batch 5600/34064] loss=2.9454, ppl=19.0190, throughput=60.24K wps, wc=6524.71K
2018-06-27 04:27:58,657 - root - [Epoch 23 Batch 6400/34064] loss=2.9275, ppl=18.6815, throughput=61.21K wps, wc=6526.66K
2018-06-27 04:29:45,645 - root - [Epoch 23 Batch 7200/34064] loss=2.9242, ppl=18.6189, throughput=60.98K wps, wc=6523.80K
2018-06-27 04:31:32,321 - root - [Epoch 23 Batch 8000/34064] loss=2.9394, ppl=18.9040, throughput=61.21K wps, wc=6529.57K
2018-06-27 04:33:17,441 - root - [Epoch 23 Batch 8800/34064] loss=2.9446, ppl=19.0027, throughput=62.01K wps, wc=6518.54K
2018-06-27 04:35:03,396 - root - [Epoch 23 Batch 9600/34064] loss=2.9485, ppl=19.0780, throughput=61.54K wps, wc=6520.71K
2018-06-27 04:36:50,537 - root - [Epoch 23 Batch 10400/34064] loss=2.9317, ppl=18.7593, throughput=60.83K wps, wc=6517.22K
2018-06-27 04:38:37,731 - root - [Epoch 23 Batch 11200/34064] loss=2.9536, ppl=19.1751, throughput=60.80K wps, wc=6517.23K
2018-06-27 04:40:24,512 - root - [Epoch 23 Batch 12000/34064] loss=2.9401, ppl=18.9173, throughput=61.08K wps, wc=6522.59K
2018-06-27 04:42:10,954 - root - [Epoch 23 Batch 12800/34064] loss=2.9340, ppl=18.8029, throughput=61.29K wps, wc=6524.23K
2018-06-27 04:43:59,199 - root - [Epoch 23 Batch 13600/34064] loss=2.9641, ppl=19.3770, throughput=60.21K wps, wc=6517.77K
2018-06-27 04:45:45,989 - root - [Epoch 23 Batch 14400/34064] loss=2.9349, ppl=18.8190, throughput=61.04K wps, wc=6517.94K
2018-06-27 04:47:33,339 - root - [Epoch 23 Batch 15200/34064] loss=2.9578, ppl=19.2563, throughput=60.75K wps, wc=6521.77K
2018-06-27 04:49:20,018 - root - [Epoch 23 Batch 16000/34064] loss=2.9552, ppl=19.2051, throughput=61.11K wps, wc=6519.54K
2018-06-27 04:51:07,914 - root - [Epoch 23 Batch 16800/34064] loss=2.9491, ppl=19.0884, throughput=60.45K wps, wc=6522.28K
2018-06-27 04:52:54,940 - root - [Epoch 23 Batch 17600/34064] loss=2.9491, ppl=19.0886, throughput=60.98K wps, wc=6526.11K
2018-06-27 04:54:42,857 - root - [Epoch 23 Batch 18400/34064] loss=2.9530, ppl=19.1628, throughput=60.44K wps, wc=6522.17K
2018-06-27 04:56:29,980 - root - [Epoch 23 Batch 19200/34064] loss=2.9512, ppl=19.1295, throughput=60.88K wps, wc=6521.27K
2018-06-27 04:58:17,601 - root - [Epoch 23 Batch 20000/34064] loss=2.9414, ppl=18.9422, throughput=60.61K wps, wc=6523.19K
2018-06-27 05:00:05,124 - root - [Epoch 23 Batch 20800/34064] loss=2.9662, ppl=19.4179, throughput=60.65K wps, wc=6520.96K
2018-06-27 05:01:53,283 - root - [Epoch 23 Batch 21600/34064] loss=2.9471, ppl=19.0508, throughput=60.32K wps, wc=6523.67K
2018-06-27 05:03:40,978 - root - [Epoch 23 Batch 22400/34064] loss=2.9591, ppl=19.2813, throughput=60.53K wps, wc=6519.15K
2018-06-27 05:05:29,272 - root - [Epoch 23 Batch 23200/34064] loss=2.9544, ppl=19.1911, throughput=60.15K wps, wc=6513.94K
2018-06-27 05:07:16,230 - root - [Epoch 23 Batch 24000/34064] loss=2.9540, ppl=19.1821, throughput=60.95K wps, wc=6519.11K
2018-06-27 05:09:03,448 - root - [Epoch 23 Batch 24800/34064] loss=2.9745, ppl=19.5804, throughput=60.79K wps, wc=6517.89K
2018-06-27 05:10:51,695 - root - [Epoch 23 Batch 25600/34064] loss=2.9630, ppl=19.3564, throughput=60.26K wps, wc=6523.34K
2018-06-27 05:12:38,552 - root - [Epoch 23 Batch 26400/34064] loss=2.9577, ppl=19.2540, throughput=61.00K wps, wc=6518.59K
2018-06-27 05:14:26,082 - root - [Epoch 23 Batch 27200/34064] loss=2.9609, ppl=19.3156, throughput=60.61K wps, wc=6517.67K
2018-06-27 05:16:12,137 - root - [Epoch 23 Batch 28000/34064] loss=2.9470, ppl=19.0497, throughput=61.51K wps, wc=6523.85K
2018-06-27 05:17:58,777 - root - [Epoch 23 Batch 28800/34064] loss=2.9556, ppl=19.2140, throughput=61.15K wps, wc=6520.59K
2018-06-27 05:19:47,051 - root - [Epoch 23 Batch 29600/34064] loss=2.9778, ppl=19.6438, throughput=60.16K wps, wc=6513.66K
2018-06-27 05:21:34,937 - root - [Epoch 23 Batch 30400/34064] loss=2.9620, ppl=19.3369, throughput=60.42K wps, wc=6518.34K
2018-06-27 05:23:21,679 - root - [Epoch 23 Batch 31200/34064] loss=2.9641, ppl=19.3776, throughput=61.16K wps, wc=6527.77K
2018-06-27 05:25:08,889 - root - [Epoch 23 Batch 32000/34064] loss=2.9399, ppl=18.9145, throughput=60.86K wps, wc=6524.90K
2018-06-27 05:26:54,771 - root - [Epoch 23 Batch 32800/34064] loss=2.9457, ppl=19.0237, throughput=61.60K wps, wc=6522.28K
2018-06-27 05:28:42,348 - root - [Epoch 23 Batch 33600/34064] loss=2.9589, ppl=19.2760, throughput=60.64K wps, wc=6523.84K
2018-06-27 05:34:21,850 - root - [Epoch 23] valid Loss=1.5082, valid ppl=4.5186, valid bleu=25.80
2018-06-27 05:38:56,958 - root - [Epoch 23] test Loss=1.3362, test ppl=3.8046, test bleu=26.13
2018-06-27 05:38:56,964 - root - Save best parameters to transformer_en_de_u512/valid_best.params
2018-06-27 05:40:51,687 - root - [Epoch 24 Batch 800/34064] loss=2.9299, ppl=18.7256, throughput=57.99K wps, wc=6523.41K
2018-06-27 05:42:38,885 - root - [Epoch 24 Batch 1600/34064] loss=2.9314, ppl=18.7539, throughput=60.89K wps, wc=6526.92K
2018-06-27 05:44:27,255 - root - [Epoch 24 Batch 2400/34064] loss=2.9313, ppl=18.7515, throughput=60.08K wps, wc=6511.05K
2018-06-27 05:46:15,516 - root - [Epoch 24 Batch 3200/34064] loss=2.9408, ppl=18.9315, throughput=60.19K wps, wc=6516.12K
2018-06-27 05:48:02,374 - root - [Epoch 24 Batch 4000/34064] loss=2.9228, ppl=18.5932, throughput=60.99K wps, wc=6517.44K
2018-06-27 05:49:49,208 - root - [Epoch 24 Batch 4800/34064] loss=2.9343, ppl=18.8087, throughput=61.10K wps, wc=6527.99K
2018-06-27 05:51:34,914 - root - [Epoch 24 Batch 5600/34064] loss=2.9375, ppl=18.8678, throughput=61.65K wps, wc=6516.87K
2018-06-27 05:53:22,674 - root - [Epoch 24 Batch 6400/34064] loss=2.9367, ppl=18.8544, throughput=60.44K wps, wc=6512.59K
2018-06-27 05:55:09,312 - root - [Epoch 24 Batch 7200/34064] loss=2.9277, ppl=18.6845, throughput=61.20K wps, wc=6526.13K
2018-06-27 05:56:58,794 - root - [Epoch 24 Batch 8000/34064] loss=2.9607, ppl=19.3112, throughput=59.52K wps, wc=6516.67K
2018-06-27 05:58:46,845 - root - [Epoch 24 Batch 8800/34064] loss=2.9470, ppl=19.0488, throughput=60.26K wps, wc=6511.19K
2018-06-27 06:00:34,165 - root - [Epoch 24 Batch 9600/34064] loss=2.9311, ppl=18.7482, throughput=60.73K wps, wc=6517.37K
2018-06-27 06:02:20,723 - root - [Epoch 24 Batch 10400/34064] loss=2.9444, ppl=19.0002, throughput=61.19K wps, wc=6519.79K
2018-06-27 06:04:08,006 - root - [Epoch 24 Batch 11200/34064] loss=2.9540, ppl=19.1821, throughput=60.81K wps, wc=6523.41K
2018-06-27 06:05:55,398 - root - [Epoch 24 Batch 12000/34064] loss=2.9449, ppl=19.0078, throughput=60.73K wps, wc=6522.01K
2018-06-27 06:07:41,367 - root - [Epoch 24 Batch 12800/34064] loss=2.9400, ppl=18.9158, throughput=61.54K wps, wc=6521.50K
2018-06-27 06:09:28,514 - root - [Epoch 24 Batch 13600/34064] loss=2.9480, ppl=19.0686, throughput=60.89K wps, wc=6524.63K
2018-06-27 06:11:17,574 - root - [Epoch 24 Batch 14400/34064] loss=2.9417, ppl=18.9479, throughput=59.82K wps, wc=6524.33K
2018-06-27 06:13:04,658 - root - [Epoch 24 Batch 15200/34064] loss=2.9358, ppl=18.8371, throughput=60.90K wps, wc=6521.55K
2018-06-27 06:14:50,842 - root - [Epoch 24 Batch 16000/34064] loss=2.9380, ppl=18.8780, throughput=61.44K wps, wc=6523.92K
2018-06-27 06:16:37,970 - root - [Epoch 24 Batch 16800/34064] loss=2.9337, ppl=18.7965, throughput=60.92K wps, wc=6526.64K
2018-06-27 06:18:24,719 - root - [Epoch 24 Batch 17600/34064] loss=2.9479, ppl=19.0652, throughput=61.08K wps, wc=6520.13K
2018-06-27 06:20:11,074 - root - [Epoch 24 Batch 18400/34064] loss=2.9371, ppl=18.8609, throughput=61.33K wps, wc=6522.82K
2018-06-27 06:21:57,898 - root - [Epoch 24 Batch 19200/34064] loss=2.9415, ppl=18.9446, throughput=61.00K wps, wc=6515.77K
2018-06-27 06:23:45,034 - root - [Epoch 24 Batch 20000/34064] loss=2.9344, ppl=18.8104, throughput=60.80K wps, wc=6514.20K
2018-06-27 06:25:32,448 - root - [Epoch 24 Batch 20800/34064] loss=2.9466, ppl=19.0408, throughput=60.77K wps, wc=6527.39K
2018-06-27 06:27:20,642 - root - [Epoch 24 Batch 21600/34064] loss=2.9640, ppl=19.3761, throughput=60.31K wps, wc=6524.77K
2018-06-27 06:29:08,219 - root - [Epoch 24 Batch 22400/34064] loss=2.9530, ppl=19.1625, throughput=60.56K wps, wc=6514.57K
2018-06-27 06:30:57,023 - root - [Epoch 24 Batch 23200/34064] loss=2.9524, ppl=19.1514, throughput=59.93K wps, wc=6520.12K
2018-06-27 06:32:44,161 - root - [Epoch 24 Batch 24000/34064] loss=2.9425, ppl=18.9641, throughput=60.88K wps, wc=6522.62K
2018-06-27 06:34:31,067 - root - [Epoch 24 Batch 24800/34064] loss=2.9452, ppl=19.0153, throughput=61.01K wps, wc=6522.67K
2018-06-27 06:36:18,850 - root - [Epoch 24 Batch 25600/34064] loss=2.9530, ppl=19.1641, throughput=60.44K wps, wc=6514.43K
2018-06-27 06:38:06,401 - root - [Epoch 24 Batch 26400/34064] loss=2.9487, ppl=19.0819, throughput=60.63K wps, wc=6520.54K
2018-06-27 06:39:54,390 - root - [Epoch 24 Batch 27200/34064] loss=2.9455, ppl=19.0207, throughput=60.44K wps, wc=6526.64K
2018-06-27 06:41:42,411 - root - [Epoch 24 Batch 28000/34064] loss=2.9549, ppl=19.1994, throughput=60.38K wps, wc=6522.57K
2018-06-27 06:43:31,092 - root - [Epoch 24 Batch 28800/34064] loss=2.9613, ppl=19.3225, throughput=60.04K wps, wc=6524.65K
2018-06-27 06:45:18,770 - root - [Epoch 24 Batch 29600/34064] loss=2.9513, ppl=19.1314, throughput=60.62K wps, wc=6527.01K
2018-06-27 06:47:06,799 - root - [Epoch 24 Batch 30400/34064] loss=2.9503, ppl=19.1117, throughput=60.36K wps, wc=6520.74K
2018-06-27 06:48:54,060 - root - [Epoch 24 Batch 31200/34064] loss=2.9529, ppl=19.1617, throughput=60.82K wps, wc=6523.97K
2018-06-27 06:50:41,294 - root - [Epoch 24 Batch 32000/34064] loss=2.9477, ppl=19.0613, throughput=60.83K wps, wc=6522.72K
2018-06-27 06:52:28,500 - root - [Epoch 24 Batch 32800/34064] loss=2.9388, ppl=18.8926, throughput=60.87K wps, wc=6525.22K
2018-06-27 06:54:16,897 - root - [Epoch 24 Batch 33600/34064] loss=2.9522, ppl=19.1482, throughput=60.17K wps, wc=6521.72K
2018-06-27 06:59:56,001 - root - [Epoch 24] valid Loss=1.5034, valid ppl=4.4968, valid bleu=25.60
2018-06-27 07:04:36,040 - root - [Epoch 24] test Loss=1.3279, test ppl=3.7730, test bleu=26.18
2018-06-27 07:06:26,824 - root - [Epoch 25 Batch 800/34064] loss=2.9183, ppl=18.5097, throughput=59.10K wps, wc=6521.20K
2018-06-27 07:08:14,391 - root - [Epoch 25 Batch 1600/34064] loss=2.9234, ppl=18.6043, throughput=60.65K wps, wc=6524.19K
2018-06-27 07:10:01,753 - root - [Epoch 25 Batch 2400/34064] loss=2.9289, ppl=18.7072, throughput=60.75K wps, wc=6522.15K
2018-06-27 07:11:50,948 - root - [Epoch 25 Batch 3200/34064] loss=2.9266, ppl=18.6641, throughput=59.65K wps, wc=6513.12K
2018-06-27 07:13:38,829 - root - [Epoch 25 Batch 4000/34064] loss=2.9344, ppl=18.8108, throughput=60.40K wps, wc=6516.47K
2018-06-27 07:15:25,963 - root - [Epoch 25 Batch 4800/34064] loss=2.9346, ppl=18.8140, throughput=60.83K wps, wc=6517.04K
2018-06-27 07:17:13,862 - root - [Epoch 25 Batch 5600/34064] loss=2.9506, ppl=19.1168, throughput=60.45K wps, wc=6522.28K
2018-06-27 07:19:02,225 - root - [Epoch 25 Batch 6400/34064] loss=2.9265, ppl=18.6617, throughput=60.21K wps, wc=6524.43K
2018-06-27 07:20:49,958 - root - [Epoch 25 Batch 7200/34064] loss=2.9418, ppl=18.9493, throughput=60.54K wps, wc=6521.82K
2018-06-27 07:22:37,667 - root - [Epoch 25 Batch 8000/34064] loss=2.9389, ppl=18.8944, throughput=60.50K wps, wc=6516.59K
2018-06-27 07:24:25,009 - root - [Epoch 25 Batch 8800/34064] loss=2.9373, ppl=18.8642, throughput=60.72K wps, wc=6517.66K
2018-06-27 07:26:14,310 - root - [Epoch 25 Batch 9600/34064] loss=2.9423, ppl=18.9588, throughput=59.62K wps, wc=6517.01K
2018-06-27 07:28:00,714 - root - [Epoch 25 Batch 10400/34064] loss=2.9316, ppl=18.7571, throughput=61.30K wps, wc=6522.74K
2018-06-27 07:29:47,678 - root - [Epoch 25 Batch 11200/34064] loss=2.9295, ppl=18.7178, throughput=61.02K wps, wc=6526.87K
2018-06-27 07:31:34,459 - root - [Epoch 25 Batch 12000/34064] loss=2.9271, ppl=18.6735, throughput=61.10K wps, wc=6524.35K
2018-06-27 07:33:21,559 - root - [Epoch 25 Batch 12800/34064] loss=2.9356, ppl=18.8335, throughput=60.96K wps, wc=6528.64K
2018-06-27 07:35:08,896 - root - [Epoch 25 Batch 13600/34064] loss=2.9362, ppl=18.8432, throughput=60.75K wps, wc=6520.95K
2018-06-27 07:36:55,566 - root - [Epoch 25 Batch 14400/34064] loss=2.9355, ppl=18.8313, throughput=61.16K wps, wc=6523.45K
2018-06-27 07:38:43,459 - root - [Epoch 25 Batch 15200/34064] loss=2.9345, ppl=18.8113, throughput=60.43K wps, wc=6519.95K
2018-06-27 07:40:31,843 - root - [Epoch 25 Batch 16000/34064] loss=2.9435, ppl=18.9818, throughput=60.10K wps, wc=6514.28K
2018-06-27 07:42:18,637 - root - [Epoch 25 Batch 16800/34064] loss=2.9419, ppl=18.9516, throughput=61.13K wps, wc=6528.01K
2018-06-27 07:44:05,714 - root - [Epoch 25 Batch 17600/34064] loss=2.9336, ppl=18.7946, throughput=60.93K wps, wc=6523.77K
2018-06-27 07:45:54,030 - root - [Epoch 25 Batch 18400/34064] loss=2.9332, ppl=18.7882, throughput=60.25K wps, wc=6526.26K
2018-06-27 07:47:42,390 - root - [Epoch 25 Batch 19200/34064] loss=2.9424, ppl=18.9622, throughput=60.16K wps, wc=6518.86K
2018-06-27 07:49:30,448 - root - [Epoch 25 Batch 20000/34064] loss=2.9380, ppl=18.8773, throughput=60.34K wps, wc=6520.33K
2018-06-27 07:51:17,652 - root - [Epoch 25 Batch 20800/34064] loss=2.9272, ppl=18.6744, throughput=60.86K wps, wc=6523.92K
2018-06-27 07:53:04,892 - root - [Epoch 25 Batch 21600/34064] loss=2.9208, ppl=18.5570, throughput=60.83K wps, wc=6523.12K
2018-06-27 07:54:52,887 - root - [Epoch 25 Batch 22400/34064] loss=2.9394, ppl=18.9054, throughput=60.42K wps, wc=6524.96K
2018-06-27 07:56:41,823 - root - [Epoch 25 Batch 23200/34064] loss=2.9465, ppl=19.0395, throughput=59.88K wps, wc=6522.76K
2018-06-27 07:58:31,118 - root - [Epoch 25 Batch 24000/34064] loss=2.9570, ppl=19.2394, throughput=59.65K wps, wc=6519.90K
2018-06-27 08:00:20,259 - root - [Epoch 25 Batch 24800/34064] loss=2.9452, ppl=19.0135, throughput=59.69K wps, wc=6515.07K
2018-06-27 08:02:08,300 - root - [Epoch 25 Batch 25600/34064] loss=2.9455, ppl=19.0209, throughput=60.36K wps, wc=6520.92K
2018-06-27 08:03:56,630 - root - [Epoch 25 Batch 26400/34064] loss=2.9544, ppl=19.1896, throughput=60.22K wps, wc=6523.33K
2018-06-27 08:05:43,162 - root - [Epoch 25 Batch 27200/34064] loss=2.9417, ppl=18.9475, throughput=61.22K wps, wc=6521.98K
2018-06-27 08:07:30,055 - root - [Epoch 25 Batch 28000/34064] loss=2.9476, ppl=19.0609, throughput=60.96K wps, wc=6516.31K
2018-06-27 08:09:19,164 - root - [Epoch 25 Batch 28800/34064] loss=2.9482, ppl=19.0722, throughput=59.75K wps, wc=6518.71K
2018-06-27 08:11:07,591 - root - [Epoch 25 Batch 29600/34064] loss=2.9592, ppl=19.2827, throughput=60.16K wps, wc=6523.36K
2018-06-27 08:12:54,298 - root - [Epoch 25 Batch 30400/34064] loss=2.9326, ppl=18.7770, throughput=61.11K wps, wc=6520.27K
2018-06-27 08:14:42,825 - root - [Epoch 25 Batch 31200/34064] loss=2.9490, ppl=19.0875, throughput=60.11K wps, wc=6523.47K
2018-06-27 08:16:29,762 - root - [Epoch 25 Batch 32000/34064] loss=2.9536, ppl=19.1744, throughput=60.95K wps, wc=6517.90K
2018-06-27 08:18:18,293 - root - [Epoch 25 Batch 32800/34064] loss=2.9651, ppl=19.3964, throughput=60.09K wps, wc=6521.35K
2018-06-27 08:20:06,567 - root - [Epoch 25 Batch 33600/34064] loss=2.9569, ppl=19.2385, throughput=60.19K wps, wc=6517.20K
2018-06-27 08:25:42,451 - root - [Epoch 25] valid Loss=1.5090, valid ppl=4.5221, valid bleu=25.62
2018-06-27 08:30:19,419 - root - [Epoch 25] test Loss=1.3326, test ppl=3.7910, test bleu=26.33
2018-06-27 08:32:10,851 - root - [Epoch 26 Batch 800/34064] loss=2.9125, ppl=18.4036, throughput=58.74K wps, wc=6519.84K
2018-06-27 08:34:00,441 - root - [Epoch 26 Batch 1600/34064] loss=2.9210, ppl=18.5608, throughput=59.52K wps, wc=6522.54K
2018-06-27 08:35:49,290 - root - [Epoch 26 Batch 2400/34064] loss=2.9238, ppl=18.6124, throughput=59.95K wps, wc=6525.75K
2018-06-27 08:37:38,187 - root - [Epoch 26 Batch 3200/34064] loss=2.9354, ppl=18.8293, throughput=59.87K wps, wc=6519.87K
2018-06-27 08:39:27,098 - root - [Epoch 26 Batch 4000/34064] loss=2.9218, ppl=18.5743, throughput=59.88K wps, wc=6521.96K
2018-06-27 08:41:16,764 - root - [Epoch 26 Batch 4800/34064] loss=2.9196, ppl=18.5345, throughput=59.48K wps, wc=6523.18K
2018-06-27 08:43:03,458 - root - [Epoch 26 Batch 5600/34064] loss=2.9279, ppl=18.6883, throughput=61.17K wps, wc=6526.45K
2018-06-27 08:44:51,398 - root - [Epoch 26 Batch 6400/34064] loss=2.9214, ppl=18.5678, throughput=60.46K wps, wc=6526.14K
2018-06-27 08:46:39,306 - root - [Epoch 26 Batch 7200/34064] loss=2.9242, ppl=18.6200, throughput=60.45K wps, wc=6523.11K
2018-06-27 08:48:27,334 - root - [Epoch 26 Batch 8000/34064] loss=2.9295, ppl=18.7186, throughput=60.41K wps, wc=6525.40K
2018-06-27 08:50:13,890 - root - [Epoch 26 Batch 8800/34064] loss=2.9243, ppl=18.6208, throughput=61.14K wps, wc=6515.09K
2018-06-27 08:52:00,323 - root - [Epoch 26 Batch 9600/34064] loss=2.9364, ppl=18.8483, throughput=61.24K wps, wc=6517.48K
2018-06-27 08:53:48,287 - root - [Epoch 26 Batch 10400/34064] loss=2.9408, ppl=18.9311, throughput=60.35K wps, wc=6515.03K
2018-06-27 08:55:37,611 - root - [Epoch 26 Batch 11200/34064] loss=2.9308, ppl=18.7417, throughput=59.65K wps, wc=6520.61K
2018-06-27 08:57:24,943 - root - [Epoch 26 Batch 12000/34064] loss=2.9226, ppl=18.5888, throughput=60.78K wps, wc=6523.55K
2018-06-27 08:59:13,705 - root - [Epoch 26 Batch 12800/34064] loss=2.9294, ppl=18.7173, throughput=59.99K wps, wc=6524.76K
2018-06-27 09:01:02,781 - root - [Epoch 26 Batch 13600/34064] loss=2.9543, ppl=19.1879, throughput=59.81K wps, wc=6524.03K
2018-06-27 09:02:49,547 - root - [Epoch 26 Batch 14400/34064] loss=2.9250, ppl=18.6336, throughput=61.10K wps, wc=6523.19K
2018-06-27 09:04:38,951 - root - [Epoch 26 Batch 15200/34064] loss=2.9385, ppl=18.8870, throughput=59.55K wps, wc=6514.69K
2018-06-27 09:06:28,780 - root - [Epoch 26 Batch 16000/34064] loss=2.9392, ppl=18.9011, throughput=59.42K wps, wc=6525.49K
2018-06-27 09:08:15,929 - root - [Epoch 26 Batch 16800/34064] loss=2.9265, ppl=18.6626, throughput=60.83K wps, wc=6517.69K
2018-06-27 09:10:04,340 - root - [Epoch 26 Batch 17600/34064] loss=2.9410, ppl=18.9350, throughput=60.18K wps, wc=6524.05K
2018-06-27 09:11:52,214 - root - [Epoch 26 Batch 18400/34064] loss=2.9319, ppl=18.7636, throughput=60.47K wps, wc=6523.05K
2018-06-27 09:13:39,430 - root - [Epoch 26 Batch 19200/34064] loss=2.9461, ppl=19.0314, throughput=60.86K wps, wc=6525.45K
2018-06-27 09:15:29,103 - root - [Epoch 26 Batch 20000/34064] loss=2.9394, ppl=18.9042, throughput=59.48K wps, wc=6522.88K
2018-06-27 09:17:16,870 - root - [Epoch 26 Batch 20800/34064] loss=2.9382, ppl=18.8826, throughput=60.48K wps, wc=6517.18K
2018-06-27 09:19:04,554 - root - [Epoch 26 Batch 21600/34064] loss=2.9385, ppl=18.8877, throughput=60.58K wps, wc=6523.03K
2018-06-27 09:20:52,138 - root - [Epoch 26 Batch 22400/34064] loss=2.9418, ppl=18.9490, throughput=60.66K wps, wc=6526.29K
2018-06-27 09:22:40,845 - root - [Epoch 26 Batch 23200/34064] loss=2.9477, ppl=19.0617, throughput=60.00K wps, wc=6522.31K
2018-06-27 09:24:30,554 - root - [Epoch 26 Batch 24000/34064] loss=2.9461, ppl=19.0322, throughput=59.43K wps, wc=6519.59K
2018-06-27 09:26:18,353 - root - [Epoch 26 Batch 24800/34064] loss=2.9350, ppl=18.8223, throughput=60.51K wps, wc=6523.12K
2018-06-27 09:28:06,804 - root - [Epoch 26 Batch 25600/34064] loss=2.9373, ppl=18.8647, throughput=60.14K wps, wc=6522.76K
2018-06-27 09:29:54,662 - root - [Epoch 26 Batch 26400/34064] loss=2.9451, ppl=19.0130, throughput=60.47K wps, wc=6521.91K
2018-06-27 09:31:43,104 - root - [Epoch 26 Batch 27200/34064] loss=2.9388, ppl=18.8927, throughput=60.10K wps, wc=6516.96K
2018-06-27 09:33:31,014 - root - [Epoch 26 Batch 28000/34064] loss=2.9416, ppl=18.9453, throughput=60.36K wps, wc=6513.62K
2018-06-27 09:35:18,397 - root - [Epoch 26 Batch 28800/34064] loss=2.9322, ppl=18.7687, throughput=60.70K wps, wc=6518.56K
2018-06-27 09:37:07,153 - root - [Epoch 26 Batch 29600/34064] loss=2.9423, ppl=18.9588, throughput=59.92K wps, wc=6516.16K
2018-06-27 09:38:57,099 - root - [Epoch 26 Batch 30400/34064] loss=2.9378, ppl=18.8737, throughput=59.29K wps, wc=6518.53K
2018-06-27 09:40:45,380 - root - [Epoch 26 Batch 31200/34064] loss=2.9482, ppl=19.0707, throughput=60.23K wps, wc=6522.06K
2018-06-27 09:42:33,852 - root - [Epoch 26 Batch 32000/34064] loss=2.9496, ppl=19.0991, throughput=60.10K wps, wc=6519.51K
2018-06-27 09:44:23,808 - root - [Epoch 26 Batch 32800/34064] loss=2.9613, ppl=19.3223, throughput=59.24K wps, wc=6513.93K
2018-06-27 09:46:11,885 - root - [Epoch 26 Batch 33600/34064] loss=2.9478, ppl=19.0638, throughput=60.36K wps, wc=6523.99K
2018-06-27 09:51:53,880 - root - [Epoch 26] valid Loss=1.5083, valid ppl=4.5193, valid bleu=25.65
2018-06-27 09:56:36,739 - root - [Epoch 26] test Loss=1.3307, test ppl=3.7836, test bleu=26.30
2018-06-27 09:58:29,174 - root - [Epoch 27 Batch 800/34064] loss=2.9133, ppl=18.4173, throughput=58.23K wps, wc=6520.78K
2018-06-27 10:00:16,481 - root - [Epoch 27 Batch 1600/34064] loss=2.9150, ppl=18.4484, throughput=60.74K wps, wc=6518.09K
2018-06-27 10:02:02,752 - root - [Epoch 27 Batch 2400/34064] loss=2.9033, ppl=18.2335, throughput=61.43K wps, wc=6527.83K
2018-06-27 10:03:49,449 - root - [Epoch 27 Batch 3200/34064] loss=2.9276, ppl=18.6826, throughput=61.03K wps, wc=6512.12K
2018-06-27 10:05:37,954 - root - [Epoch 27 Batch 4000/34064] loss=2.9103, ppl=18.3630, throughput=60.13K wps, wc=6523.91K
2018-06-27 10:07:26,124 - root - [Epoch 27 Batch 4800/34064] loss=2.9161, ppl=18.4691, throughput=60.35K wps, wc=6527.54K
2018-06-27 10:09:14,636 - root - [Epoch 27 Batch 5600/34064] loss=2.9251, ppl=18.6369, throughput=60.05K wps, wc=6516.07K
2018-06-27 10:11:04,406 - root - [Epoch 27 Batch 6400/34064] loss=2.9195, ppl=18.5326, throughput=59.44K wps, wc=6525.12K
2018-06-27 10:12:51,990 - root - [Epoch 27 Batch 7200/34064] loss=2.9243, ppl=18.6203, throughput=60.62K wps, wc=6522.05K
2018-06-27 10:14:40,473 - root - [Epoch 27 Batch 8000/34064] loss=2.9251, ppl=18.6363, throughput=60.14K wps, wc=6523.73K
2018-06-27 10:16:28,784 - root - [Epoch 27 Batch 8800/34064] loss=2.9308, ppl=18.7419, throughput=60.24K wps, wc=6524.37K
2018-06-27 10:18:16,158 - root - [Epoch 27 Batch 9600/34064] loss=2.9223, ppl=18.5848, throughput=60.75K wps, wc=6522.98K
2018-06-27 10:20:04,018 - root - [Epoch 27 Batch 10400/34064] loss=2.9205, ppl=18.5507, throughput=60.41K wps, wc=6516.04K
2018-06-27 10:21:51,972 - root - [Epoch 27 Batch 11200/34064] loss=2.9279, ppl=18.6877, throughput=60.38K wps, wc=6518.67K
2018-06-27 10:23:41,856 - root - [Epoch 27 Batch 12000/34064] loss=2.9381, ppl=18.8807, throughput=59.32K wps, wc=6518.46K
2018-06-27 10:25:30,537 - root - [Epoch 27 Batch 12800/34064] loss=2.9281, ppl=18.6924, throughput=59.97K wps, wc=6517.81K
2018-06-27 10:27:18,959 - root - [Epoch 27 Batch 13600/34064] loss=2.9026, ppl=18.2223, throughput=60.19K wps, wc=6525.42K
2018-06-27 10:29:07,780 - root - [Epoch 27 Batch 14400/34064] loss=2.9321, ppl=18.7677, throughput=59.94K wps, wc=6522.18K
2018-06-27 10:30:56,535 - root - [Epoch 27 Batch 15200/34064] loss=2.9393, ppl=18.9019, throughput=60.00K wps, wc=6524.97K
2018-06-27 10:32:45,354 - root - [Epoch 27 Batch 16000/34064] loss=2.9355, ppl=18.8307, throughput=59.81K wps, wc=6508.86K
2018-06-27 10:34:32,813 - root - [Epoch 27 Batch 16800/34064] loss=2.9333, ppl=18.7901, throughput=60.72K wps, wc=6524.89K
2018-06-27 10:36:20,044 - root - [Epoch 27 Batch 17600/34064] loss=2.9337, ppl=18.7972, throughput=60.80K wps, wc=6519.30K
2018-06-27 10:38:07,531 - root - [Epoch 27 Batch 18400/34064] loss=2.9298, ppl=18.7233, throughput=60.73K wps, wc=6527.58K
2018-06-27 10:39:55,173 - root - [Epoch 27 Batch 19200/34064] loss=2.9348, ppl=18.8185, throughput=60.54K wps, wc=6516.90K
2018-06-27 10:41:44,191 - root - [Epoch 27 Batch 20000/34064] loss=2.9386, ppl=18.8902, throughput=59.81K wps, wc=6520.30K
2018-06-27 10:43:31,795 - root - [Epoch 27 Batch 20800/34064] loss=2.9388, ppl=18.8930, throughput=60.53K wps, wc=6513.68K
2018-06-27 10:45:19,573 - root - [Epoch 27 Batch 21600/34064] loss=2.9304, ppl=18.7347, throughput=60.53K wps, wc=6524.31K
2018-06-27 10:47:09,198 - root - [Epoch 27 Batch 22400/34064] loss=2.9374, ppl=18.8660, throughput=59.50K wps, wc=6522.75K
2018-06-27 10:48:56,966 - root - [Epoch 27 Batch 23200/34064] loss=2.9341, ppl=18.8054, throughput=60.53K wps, wc=6523.23K
2018-06-27 10:50:46,560 - root - [Epoch 27 Batch 24000/34064] loss=2.9296, ppl=18.7197, throughput=59.58K wps, wc=6529.31K
2018-06-27 10:52:36,080 - root - [Epoch 27 Batch 24800/34064] loss=2.9418, ppl=18.9507, throughput=59.52K wps, wc=6519.07K
2018-06-27 10:54:25,766 - root - [Epoch 27 Batch 25600/34064] loss=2.9483, ppl=19.0741, throughput=59.45K wps, wc=6520.53K
2018-06-27 10:56:13,300 - root - [Epoch 27 Batch 26400/34064] loss=2.9331, ppl=18.7867, throughput=60.68K wps, wc=6525.44K
2018-06-27 10:58:01,231 - root - [Epoch 27 Batch 27200/34064] loss=2.9444, ppl=19.0001, throughput=60.40K wps, wc=6519.36K
2018-06-27 10:59:48,719 - root - [Epoch 27 Batch 28000/34064] loss=2.9456, ppl=19.0212, throughput=60.67K wps, wc=6520.84K
2018-06-27 11:01:37,650 - root - [Epoch 27 Batch 28800/34064] loss=2.9458, ppl=19.0265, throughput=59.75K wps, wc=6508.81K
2018-06-27 11:03:25,782 - root - [Epoch 27 Batch 29600/34064] loss=2.9549, ppl=19.2004, throughput=60.23K wps, wc=6512.95K
2018-06-27 11:05:13,515 - root - [Epoch 27 Batch 30400/34064] loss=2.9324, ppl=18.7725, throughput=60.59K wps, wc=6527.48K
2018-06-27 11:07:00,860 - root - [Epoch 27 Batch 31200/34064] loss=2.9588, ppl=19.2758, throughput=60.79K wps, wc=6525.12K
2018-06-27 11:08:47,793 - root - [Epoch 27 Batch 32000/34064] loss=2.9408, ppl=18.9316, throughput=61.02K wps, wc=6524.79K
2018-06-27 11:10:35,377 - root - [Epoch 27 Batch 32800/34064] loss=2.9329, ppl=18.7829, throughput=60.61K wps, wc=6521.01K
2018-06-27 11:12:24,038 - root - [Epoch 27 Batch 33600/34064] loss=2.9513, ppl=19.1315, throughput=60.01K wps, wc=6520.46K
2018-06-27 11:18:09,803 - root - [Epoch 27] valid Loss=1.5094, valid ppl=4.5242, valid bleu=25.71
2018-06-27 11:22:50,969 - root - [Epoch 27] test Loss=1.3293, test ppl=3.7785, test bleu=26.08
2018-06-27 11:24:44,150 - root - [Epoch 28 Batch 800/34064] loss=2.9172, ppl=18.4889, throughput=57.83K wps, wc=6518.98K
2018-06-27 11:26:31,142 - root - [Epoch 28 Batch 1600/34064] loss=2.9063, ppl=18.2893, throughput=61.01K wps, wc=6527.03K
2018-06-27 11:28:18,354 - root - [Epoch 28 Batch 2400/34064] loss=2.9114, ppl=18.3823, throughput=60.85K wps, wc=6523.75K
2018-06-27 11:30:06,324 - root - [Epoch 28 Batch 3200/34064] loss=2.9158, ppl=18.4634, throughput=60.42K wps, wc=6523.31K
2018-06-27 11:31:53,823 - root - [Epoch 28 Batch 4000/34064] loss=2.9064, ppl=18.2900, throughput=60.65K wps, wc=6520.13K
2018-06-27 11:33:42,298 - root - [Epoch 28 Batch 4800/34064] loss=2.9214, ppl=18.5679, throughput=60.11K wps, wc=6520.22K
2018-06-27 11:35:29,662 - root - [Epoch 28 Batch 5600/34064] loss=2.9088, ppl=18.3342, throughput=60.78K wps, wc=6525.14K
2018-06-27 11:37:18,368 - root - [Epoch 28 Batch 6400/34064] loss=2.9218, ppl=18.5752, throughput=60.05K wps, wc=6528.23K
2018-06-27 11:39:06,980 - root - [Epoch 28 Batch 7200/34064] loss=2.9141, ppl=18.4324, throughput=60.07K wps, wc=6524.14K
2018-06-27 11:40:55,368 - root - [Epoch 28 Batch 8000/34064] loss=2.9347, ppl=18.8158, throughput=60.19K wps, wc=6523.28K
2018-06-27 11:42:42,975 - root - [Epoch 28 Batch 8800/34064] loss=2.9347, ppl=18.8154, throughput=60.61K wps, wc=6522.38K
2018-06-27 11:44:30,852 - root - [Epoch 28 Batch 9600/34064] loss=2.9127, ppl=18.4068, throughput=60.52K wps, wc=6528.67K
2018-06-27 11:46:18,825 - root - [Epoch 28 Batch 10400/34064] loss=2.9235, ppl=18.6064, throughput=60.39K wps, wc=6520.01K
2018-06-27 11:48:05,027 - root - [Epoch 28 Batch 11200/34064] loss=2.9122, ppl=18.3968, throughput=61.36K wps, wc=6516.75K
2018-06-27 11:49:52,298 - root - [Epoch 28 Batch 12000/34064] loss=2.9268, ppl=18.6672, throughput=60.79K wps, wc=6520.95K
2018-06-27 11:51:40,864 - root - [Epoch 28 Batch 12800/34064] loss=2.9315, ppl=18.7550, throughput=60.04K wps, wc=6518.73K
2018-06-27 11:53:28,778 - root - [Epoch 28 Batch 13600/34064] loss=2.9420, ppl=18.9545, throughput=60.42K wps, wc=6519.97K
2018-06-27 11:55:16,928 - root - [Epoch 28 Batch 14400/34064] loss=2.9354, ppl=18.8284, throughput=60.25K wps, wc=6515.83K
2018-06-27 11:57:04,474 - root - [Epoch 28 Batch 15200/34064] loss=2.9181, ppl=18.5069, throughput=60.65K wps, wc=6522.79K
2018-06-27 11:58:51,306 - root - [Epoch 28 Batch 16000/34064] loss=2.9271, ppl=18.6735, throughput=61.04K wps, wc=6521.31K
2018-06-27 12:00:39,558 - root - [Epoch 28 Batch 16800/34064] loss=2.9324, ppl=18.7735, throughput=60.18K wps, wc=6514.48K
2018-06-27 12:02:25,984 - root - [Epoch 28 Batch 17600/34064] loss=2.9244, ppl=18.6231, throughput=61.33K wps, wc=6526.57K
2018-06-27 12:04:13,925 - root - [Epoch 28 Batch 18400/34064] loss=2.9328, ppl=18.7806, throughput=60.48K wps, wc=6528.05K
2018-06-27 12:06:01,191 - root - [Epoch 28 Batch 19200/34064] loss=2.9146, ppl=18.4411, throughput=60.83K wps, wc=6524.84K
2018-06-27 12:07:49,148 - root - [Epoch 28 Batch 20000/34064] loss=2.9267, ppl=18.6657, throughput=60.40K wps, wc=6520.32K
2018-06-27 12:09:36,913 - root - [Epoch 28 Batch 20800/34064] loss=2.9262, ppl=18.6562, throughput=60.57K wps, wc=6526.98K
2018-06-27 12:11:25,305 - root - [Epoch 28 Batch 21600/34064] loss=2.9340, ppl=18.8021, throughput=60.16K wps, wc=6521.35K
2018-06-27 12:13:13,156 - root - [Epoch 28 Batch 22400/34064] loss=2.9332, ppl=18.7874, throughput=60.46K wps, wc=6520.74K
2018-06-27 12:15:01,188 - root - [Epoch 28 Batch 23200/34064] loss=2.9510, ppl=19.1256, throughput=60.24K wps, wc=6507.63K
2018-06-27 12:16:49,145 - root - [Epoch 28 Batch 24000/34064] loss=2.9349, ppl=18.8200, throughput=60.46K wps, wc=6526.82K
2018-06-27 12:18:37,765 - root - [Epoch 28 Batch 24800/34064] loss=2.9364, ppl=18.8473, throughput=60.02K wps, wc=6518.83K
2018-06-27 12:20:26,171 - root - [Epoch 28 Batch 25600/34064] loss=2.9455, ppl=19.0207, throughput=60.10K wps, wc=6515.30K
2018-06-27 12:22:14,143 - root - [Epoch 28 Batch 26400/34064] loss=2.9268, ppl=18.6677, throughput=60.40K wps, wc=6521.88K
2018-06-27 12:24:01,084 - root - [Epoch 28 Batch 27200/34064] loss=2.9294, ppl=18.7165, throughput=61.03K wps, wc=6526.65K
2018-06-27 12:25:50,431 - root - [Epoch 28 Batch 28000/34064] loss=2.9495, ppl=19.0968, throughput=59.56K wps, wc=6512.48K
2018-06-27 12:27:38,208 - root - [Epoch 28 Batch 28800/34064] loss=2.9346, ppl=18.8148, throughput=60.52K wps, wc=6522.77K
2018-06-27 12:29:24,822 - root - [Epoch 28 Batch 29600/34064] loss=2.9230, ppl=18.5969, throughput=61.20K wps, wc=6524.55K
2018-06-27 12:31:12,708 - root - [Epoch 28 Batch 30400/34064] loss=2.9506, ppl=19.1182, throughput=60.38K wps, wc=6514.18K
2018-06-27 12:33:01,835 - root - [Epoch 28 Batch 31200/34064] loss=2.9479, ppl=19.0650, throughput=59.77K wps, wc=6522.52K
2018-06-27 12:34:50,059 - root - [Epoch 28 Batch 32000/34064] loss=2.9387, ppl=18.8914, throughput=60.21K wps, wc=6516.62K
2018-06-27 12:36:38,912 - root - [Epoch 28 Batch 32800/34064] loss=2.9428, ppl=18.9685, throughput=59.79K wps, wc=6508.14K
2018-06-27 12:38:26,737 - root - [Epoch 28 Batch 33600/34064] loss=2.9336, ppl=18.7953, throughput=60.52K wps, wc=6525.16K
2018-06-27 12:44:07,219 - root - [Epoch 28] valid Loss=1.5041, valid ppl=4.5002, valid bleu=25.53
2018-06-27 12:48:43,555 - root - [Epoch 28] test Loss=1.3235, test ppl=3.7566, test bleu=25.98
2018-06-27 12:50:37,213 - root - [Epoch 29 Batch 800/34064] loss=2.9136, ppl=18.4223, throughput=57.66K wps, wc=6526.99K
2018-06-27 12:52:25,427 - root - [Epoch 29 Batch 1600/34064] loss=2.9133, ppl=18.4170, throughput=60.29K wps, wc=6524.21K
2018-06-27 12:54:14,786 - root - [Epoch 29 Batch 2400/34064] loss=2.9242, ppl=18.6202, throughput=59.62K wps, wc=6519.68K
2018-06-27 12:56:01,998 - root - [Epoch 29 Batch 3200/34064] loss=2.9181, ppl=18.5061, throughput=60.80K wps, wc=6518.96K
2018-06-27 12:57:50,092 - root - [Epoch 29 Batch 4000/34064] loss=2.9054, ppl=18.2727, throughput=60.39K wps, wc=6528.11K
2018-06-27 12:59:38,482 - root - [Epoch 29 Batch 4800/34064] loss=2.9205, ppl=18.5497, throughput=60.17K wps, wc=6522.15K
2018-06-27 13:01:26,367 - root - [Epoch 29 Batch 5600/34064] loss=2.9344, ppl=18.8094, throughput=60.46K wps, wc=6522.31K
2018-06-27 13:03:14,339 - root - [Epoch 29 Batch 6400/34064] loss=2.9134, ppl=18.4189, throughput=60.43K wps, wc=6524.50K
2018-06-27 13:05:02,336 - root - [Epoch 29 Batch 7200/34064] loss=2.9181, ppl=18.5068, throughput=60.35K wps, wc=6518.06K
2018-06-27 13:06:49,901 - root - [Epoch 29 Batch 8000/34064] loss=2.9210, ppl=18.5594, throughput=60.64K wps, wc=6522.70K
2018-06-27 13:08:38,181 - root - [Epoch 29 Batch 8800/34064] loss=2.9257, ppl=18.6466, throughput=60.17K wps, wc=6514.92K
2018-06-27 13:10:25,857 - root - [Epoch 29 Batch 9600/34064] loss=2.9038, ppl=18.2435, throughput=60.61K wps, wc=6526.52K
2018-06-27 13:12:12,956 - root - [Epoch 29 Batch 10400/34064] loss=2.9114, ppl=18.3829, throughput=60.89K wps, wc=6521.46K
2018-06-27 13:14:00,773 - root - [Epoch 29 Batch 11200/34064] loss=2.9223, ppl=18.5832, throughput=60.37K wps, wc=6508.35K
2018-06-27 13:15:47,595 - root - [Epoch 29 Batch 12000/34064] loss=2.9117, ppl=18.3889, throughput=61.09K wps, wc=6525.66K
2018-06-27 13:17:35,982 - root - [Epoch 29 Batch 12800/34064] loss=2.9243, ppl=18.6204, throughput=60.17K wps, wc=6521.69K
2018-06-27 13:19:22,749 - root - [Epoch 29 Batch 13600/34064] loss=2.9233, ppl=18.6018, throughput=61.05K wps, wc=6518.15K
2018-06-27 13:21:10,542 - root - [Epoch 29 Batch 14400/34064] loss=2.9036, ppl=18.2403, throughput=60.52K wps, wc=6523.99K
2018-06-27 13:22:58,542 - root - [Epoch 29 Batch 15200/34064] loss=2.9298, ppl=18.7244, throughput=60.41K wps, wc=6524.03K
2018-06-27 13:24:45,323 - root - [Epoch 29 Batch 16000/34064] loss=2.9280, ppl=18.6910, throughput=61.07K wps, wc=6520.73K
2018-06-27 13:26:34,913 - root - [Epoch 29 Batch 16800/34064] loss=2.9378, ppl=18.8745, throughput=59.52K wps, wc=6522.19K
2018-06-27 13:28:21,979 - root - [Epoch 29 Batch 17600/34064] loss=2.9240, ppl=18.6152, throughput=60.88K wps, wc=6518.07K
2018-06-27 13:30:10,107 - root - [Epoch 29 Batch 18400/34064] loss=2.9308, ppl=18.7434, throughput=60.35K wps, wc=6525.19K
2018-06-27 13:31:58,353 - root - [Epoch 29 Batch 19200/34064] loss=2.9383, ppl=18.8840, throughput=60.26K wps, wc=6522.66K
2018-06-27 13:33:45,697 - root - [Epoch 29 Batch 20000/34064] loss=2.9259, ppl=18.6509, throughput=60.71K wps, wc=6516.64K
2018-06-27 13:35:32,509 - root - [Epoch 29 Batch 20800/34064] loss=2.9228, ppl=18.5934, throughput=61.09K wps, wc=6524.92K
2018-06-27 13:37:20,001 - root - [Epoch 29 Batch 21600/34064] loss=2.9289, ppl=18.7074, throughput=60.69K wps, wc=6523.31K
2018-06-27 13:39:07,412 - root - [Epoch 29 Batch 22400/34064] loss=2.9373, ppl=18.8658, throughput=60.72K wps, wc=6522.03K
2018-06-27 13:40:58,190 - root - [Epoch 29 Batch 23200/34064] loss=2.9298, ppl=18.7235, throughput=58.85K wps, wc=6518.71K
2018-06-27 13:42:46,706 - root - [Epoch 29 Batch 24000/34064] loss=2.9441, ppl=18.9930, throughput=60.14K wps, wc=6526.36K
2018-06-27 13:44:35,049 - root - [Epoch 29 Batch 24800/34064] loss=2.9169, ppl=18.4840, throughput=60.28K wps, wc=6530.44K
2018-06-27 13:46:22,671 - root - [Epoch 29 Batch 25600/34064] loss=2.9188, ppl=18.5198, throughput=60.61K wps, wc=6523.45K
2018-06-27 13:48:10,848 - root - [Epoch 29 Batch 26400/34064] loss=2.9256, ppl=18.6449, throughput=60.24K wps, wc=6516.80K
2018-06-27 13:50:01,014 - root - [Epoch 29 Batch 27200/34064] loss=2.9517, ppl=19.1380, throughput=59.19K wps, wc=6520.73K
2018-06-27 13:51:48,899 - root - [Epoch 29 Batch 28000/34064] loss=2.9458, ppl=19.0257, throughput=60.41K wps, wc=6516.97K
2018-06-27 13:53:37,024 - root - [Epoch 29 Batch 28800/34064] loss=2.9230, ppl=18.5966, throughput=60.26K wps, wc=6515.51K
2018-06-27 13:55:25,677 - root - [Epoch 29 Batch 29600/34064] loss=2.9191, ppl=18.5246, throughput=60.01K wps, wc=6519.96K
2018-06-27 13:57:13,044 - root - [Epoch 29 Batch 30400/34064] loss=2.9452, ppl=19.0147, throughput=60.74K wps, wc=6521.17K
2018-06-27 13:59:00,200 - root - [Epoch 29 Batch 31200/34064] loss=2.9313, ppl=18.7514, throughput=60.87K wps, wc=6522.60K
2018-06-27 14:00:48,883 - root - [Epoch 29 Batch 32000/34064] loss=2.9304, ppl=18.7342, throughput=59.90K wps, wc=6510.19K
2018-06-27 14:02:37,122 - root - [Epoch 29 Batch 32800/34064] loss=2.9369, ppl=18.8581, throughput=60.18K wps, wc=6513.46K
2018-06-27 14:04:23,531 - root - [Epoch 29 Batch 33600/34064] loss=2.9176, ppl=18.4970, throughput=61.28K wps, wc=6520.55K
2018-06-27 14:10:01,382 - root - [Epoch 29] valid Loss=1.4998, valid ppl=4.4807, valid bleu=25.61
2018-06-27 14:14:33,835 - root - [Epoch 29] test Loss=1.3284, test ppl=3.7752, test bleu=26.28
2018-06-27 14:16:25,395 - root - [Epoch 30 Batch 800/34064] loss=2.9019, ppl=18.2089, throughput=58.67K wps, wc=6519.56K
2018-06-27 14:18:12,232 - root - [Epoch 30 Batch 1600/34064] loss=2.9134, ppl=18.4198, throughput=61.10K wps, wc=6527.44K
2018-06-27 14:20:00,505 - root - [Epoch 30 Batch 2400/34064] loss=2.9054, ppl=18.2728, throughput=60.15K wps, wc=6512.70K
2018-06-27 14:21:48,027 - root - [Epoch 30 Batch 3200/34064] loss=2.9065, ppl=18.2925, throughput=60.65K wps, wc=6521.50K
2018-06-27 14:23:36,355 - root - [Epoch 30 Batch 4000/34064] loss=2.9156, ppl=18.4591, throughput=60.21K wps, wc=6522.55K
2018-06-27 14:25:24,051 - root - [Epoch 30 Batch 4800/34064] loss=2.9155, ppl=18.4579, throughput=60.53K wps, wc=6518.94K
2018-06-27 14:27:11,382 - root - [Epoch 30 Batch 5600/34064] loss=2.9158, ppl=18.4635, throughput=60.80K wps, wc=6526.19K
2018-06-27 14:28:58,911 - root - [Epoch 30 Batch 6400/34064] loss=2.9135, ppl=18.4205, throughput=60.65K wps, wc=6521.11K
2018-06-27 14:30:45,692 - root - [Epoch 30 Batch 7200/34064] loss=2.9147, ppl=18.4430, throughput=61.05K wps, wc=6519.36K
2018-06-27 14:32:32,703 - root - [Epoch 30 Batch 8000/34064] loss=2.9033, ppl=18.2340, throughput=60.91K wps, wc=6517.92K
2018-06-27 14:34:18,868 - root - [Epoch 30 Batch 8800/34064] loss=2.9134, ppl=18.4187, throughput=61.44K wps, wc=6522.89K
2018-06-27 14:36:05,717 - root - [Epoch 30 Batch 9600/34064] loss=2.9190, ppl=18.5233, throughput=61.02K wps, wc=6519.80K
2018-06-27 14:37:51,719 - root - [Epoch 30 Batch 10400/34064] loss=2.8963, ppl=18.1074, throughput=61.56K wps, wc=6525.40K
2018-06-27 14:39:38,308 - root - [Epoch 30 Batch 11200/34064] loss=2.9075, ppl=18.3108, throughput=61.21K wps, wc=6524.33K
2018-06-27 14:41:24,573 - root - [Epoch 30 Batch 12000/34064] loss=2.8994, ppl=18.1624, throughput=61.44K wps, wc=6529.34K
2018-06-27 14:43:11,506 - root - [Epoch 30 Batch 12800/34064] loss=2.9134, ppl=18.4201, throughput=61.01K wps, wc=6523.81K
2018-06-27 14:44:58,935 - root - [Epoch 30 Batch 13600/34064] loss=2.9168, ppl=18.4822, throughput=60.66K wps, wc=6517.08K
2018-06-27 14:46:45,834 - root - [Epoch 30 Batch 14400/34064] loss=2.9177, ppl=18.4981, throughput=60.96K wps, wc=6516.69K
2018-06-27 14:48:33,746 - root - [Epoch 30 Batch 15200/34064] loss=2.9301, ppl=18.7298, throughput=60.45K wps, wc=6522.74K
2018-06-27 14:50:20,739 - root - [Epoch 30 Batch 16000/34064] loss=2.9261, ppl=18.6547, throughput=60.94K wps, wc=6520.34K
2018-06-27 14:52:08,376 - root - [Epoch 30 Batch 16800/34064] loss=2.9274, ppl=18.6788, throughput=60.57K wps, wc=6519.60K
2018-06-27 14:53:56,592 - root - [Epoch 30 Batch 17600/34064] loss=2.9305, ppl=18.7369, throughput=60.28K wps, wc=6523.36K
2018-06-27 14:55:43,999 - root - [Epoch 30 Batch 18400/34064] loss=2.9121, ppl=18.3952, throughput=60.71K wps, wc=6520.11K
2018-06-27 14:57:32,624 - root - [Epoch 30 Batch 19200/34064] loss=2.9302, ppl=18.7311, throughput=60.07K wps, wc=6525.52K
2018-06-27 14:59:20,425 - root - [Epoch 30 Batch 20000/34064] loss=2.9183, ppl=18.5094, throughput=60.55K wps, wc=6526.91K
2018-06-27 15:01:08,279 - root - [Epoch 30 Batch 20800/34064] loss=2.9301, ppl=18.7303, throughput=60.51K wps, wc=6525.94K
2018-06-27 15:02:57,958 - root - [Epoch 30 Batch 21600/34064] loss=2.9315, ppl=18.7566, throughput=59.50K wps, wc=6525.54K
2018-06-27 15:04:46,699 - root - [Epoch 30 Batch 22400/34064] loss=2.9332, ppl=18.7878, throughput=59.94K wps, wc=6518.19K
2018-06-27 15:06:37,133 - root - [Epoch 30 Batch 23200/34064] loss=2.9264, ppl=18.6603, throughput=59.09K wps, wc=6525.65K
2018-06-27 15:08:25,184 - root - [Epoch 30 Batch 24000/34064] loss=2.9288, ppl=18.7052, throughput=60.20K wps, wc=6504.71K
2018-06-27 15:10:13,727 - root - [Epoch 30 Batch 24800/34064] loss=2.9330, ppl=18.7838, throughput=60.07K wps, wc=6520.51K
2018-06-27 15:12:03,911 - root - [Epoch 30 Batch 25600/34064] loss=2.9380, ppl=18.8779, throughput=59.22K wps, wc=6525.48K
2018-06-27 15:13:51,987 - root - [Epoch 30 Batch 26400/34064] loss=2.9227, ppl=18.5908, throughput=60.28K wps, wc=6515.29K
2018-06-27 15:15:40,159 - root - [Epoch 30 Batch 27200/34064] loss=2.9286, ppl=18.7016, throughput=60.26K wps, wc=6518.90K
2018-06-27 15:17:28,118 - root - [Epoch 30 Batch 28000/34064] loss=2.9339, ppl=18.8007, throughput=60.40K wps, wc=6520.42K
2018-06-27 15:19:15,829 - root - [Epoch 30 Batch 28800/34064] loss=2.9405, ppl=18.9259, throughput=60.57K wps, wc=6523.63K
2018-06-27 15:21:04,264 - root - [Epoch 30 Batch 29600/34064] loss=2.9266, ppl=18.6641, throughput=60.13K wps, wc=6520.45K
2018-06-27 15:22:52,878 - root - [Epoch 30 Batch 30400/34064] loss=2.9333, ppl=18.7890, throughput=60.03K wps, wc=6520.19K
2018-06-27 15:24:40,716 - root - [Epoch 30 Batch 31200/34064] loss=2.9254, ppl=18.6412, throughput=60.50K wps, wc=6524.15K
2018-06-27 15:26:29,828 - root - [Epoch 30 Batch 32000/34064] loss=2.9522, ppl=19.1478, throughput=59.67K wps, wc=6510.82K
2018-06-27 15:28:16,985 - root - [Epoch 30 Batch 32800/34064] loss=2.9373, ppl=18.8640, throughput=60.83K wps, wc=6518.56K
2018-06-27 15:30:05,445 - root - [Epoch 30 Batch 33600/34064] loss=2.9316, ppl=18.7580, throughput=60.13K wps, wc=6522.16K
2018-06-27 15:35:41,795 - root - [Epoch 30] valid Loss=1.5038, valid ppl=4.4987, valid bleu=25.66
2018-06-27 15:40:22,291 - root - [Epoch 30] test Loss=1.3242, test ppl=3.7593, test bleu=26.03
2018-06-27 15:42:13,887 - root - [Epoch 31 Batch 800/34064] loss=2.9009, ppl=18.1897, throughput=58.67K wps, wc=6520.93K
2018-06-27 15:44:01,581 - root - [Epoch 31 Batch 1600/34064] loss=2.8973, ppl=18.1250, throughput=60.60K wps, wc=6526.07K
2018-06-27 15:45:47,769 - root - [Epoch 31 Batch 2400/34064] loss=2.9038, ppl=18.2440, throughput=61.43K wps, wc=6523.27K
2018-06-27 15:47:35,100 - root - [Epoch 31 Batch 3200/34064] loss=2.8956, ppl=18.0935, throughput=60.72K wps, wc=6517.38K
2018-06-27 15:49:22,471 - root - [Epoch 31 Batch 4000/34064] loss=2.9148, ppl=18.4453, throughput=60.70K wps, wc=6517.33K
2018-06-27 15:51:09,775 - root - [Epoch 31 Batch 4800/34064] loss=2.9012, ppl=18.1951, throughput=60.82K wps, wc=6526.65K
2018-06-27 15:52:56,874 - root - [Epoch 31 Batch 5600/34064] loss=2.9054, ppl=18.2724, throughput=60.89K wps, wc=6521.08K
2018-06-27 15:54:45,860 - root - [Epoch 31 Batch 6400/34064] loss=2.9160, ppl=18.4674, throughput=59.79K wps, wc=6516.74K
2018-06-27 15:56:34,123 - root - [Epoch 31 Batch 7200/34064] loss=2.9149, ppl=18.4466, throughput=60.28K wps, wc=6525.61K
2018-06-27 15:58:23,288 - root - [Epoch 31 Batch 8000/34064] loss=2.9297, ppl=18.7213, throughput=59.72K wps, wc=6519.51K
2018-06-27 16:00:12,267 - root - [Epoch 31 Batch 8800/34064] loss=2.9152, ppl=18.4527, throughput=59.77K wps, wc=6513.44K
2018-06-27 16:02:00,315 - root - [Epoch 31 Batch 9600/34064] loss=2.9222, ppl=18.5821, throughput=60.26K wps, wc=6511.39K
2018-06-27 16:03:48,716 - root - [Epoch 31 Batch 10400/34064] loss=2.9114, ppl=18.3833, throughput=60.09K wps, wc=6513.46K
2018-06-27 16:05:35,866 - root - [Epoch 31 Batch 11200/34064] loss=2.9069, ppl=18.2992, throughput=60.91K wps, wc=6526.50K
2018-06-27 16:07:23,611 - root - [Epoch 31 Batch 12000/34064] loss=2.9078, ppl=18.3166, throughput=60.55K wps, wc=6523.56K
2018-06-27 16:09:12,027 - root - [Epoch 31 Batch 12800/34064] loss=2.9204, ppl=18.5492, throughput=60.11K wps, wc=6517.03K
2018-06-27 16:11:00,052 - root - [Epoch 31 Batch 13600/34064] loss=2.9036, ppl=18.2403, throughput=60.42K wps, wc=6527.16K
2018-06-27 16:12:48,400 - root - [Epoch 31 Batch 14400/34064] loss=2.9256, ppl=18.6455, throughput=60.18K wps, wc=6520.04K
2018-06-27 16:14:35,634 - root - [Epoch 31 Batch 15200/34064] loss=2.9277, ppl=18.6843, throughput=60.84K wps, wc=6523.75K
2018-06-27 16:16:23,425 - root - [Epoch 31 Batch 16000/34064] loss=2.9146, ppl=18.4417, throughput=60.51K wps, wc=6522.85K
2018-06-27 16:18:11,678 - root - [Epoch 31 Batch 16800/34064] loss=2.9167, ppl=18.4804, throughput=60.23K wps, wc=6519.64K
2018-06-27 16:19:59,161 - root - [Epoch 31 Batch 17600/34064] loss=2.9206, ppl=18.5523, throughput=60.65K wps, wc=6518.88K
2018-06-27 16:21:47,841 - root - [Epoch 31 Batch 18400/34064] loss=2.9322, ppl=18.7694, throughput=59.99K wps, wc=6519.86K
2018-06-27 16:23:34,205 - root - [Epoch 31 Batch 19200/34064] loss=2.9149, ppl=18.4478, throughput=61.33K wps, wc=6523.38K
2018-06-27 16:25:22,097 - root - [Epoch 31 Batch 20000/34064] loss=2.9168, ppl=18.4822, throughput=60.41K wps, wc=6517.30K
2018-06-27 16:27:09,640 - root - [Epoch 31 Batch 20800/34064] loss=2.9154, ppl=18.4567, throughput=60.64K wps, wc=6520.85K
2018-06-27 16:28:58,575 - root - [Epoch 31 Batch 21600/34064] loss=2.9239, ppl=18.6134, throughput=59.86K wps, wc=6520.97K
2018-06-27 16:30:46,344 - root - [Epoch 31 Batch 22400/34064] loss=2.9255, ppl=18.6440, throughput=60.47K wps, wc=6516.57K
2018-06-27 16:32:33,571 - root - [Epoch 31 Batch 23200/34064] loss=2.9178, ppl=18.5014, throughput=60.89K wps, wc=6528.68K
2018-06-27 16:34:20,832 - root - [Epoch 31 Batch 24000/34064] loss=2.9191, ppl=18.5243, throughput=60.78K wps, wc=6519.19K
2018-06-27 16:36:08,854 - root - [Epoch 31 Batch 24800/34064] loss=2.9349, ppl=18.8201, throughput=60.37K wps, wc=6520.80K
2018-06-27 16:37:56,759 - root - [Epoch 31 Batch 25600/34064] loss=2.9275, ppl=18.6810, throughput=60.46K wps, wc=6523.96K
2018-06-27 16:39:44,314 - root - [Epoch 31 Batch 26400/34064] loss=2.9237, ppl=18.6092, throughput=60.59K wps, wc=6516.83K
2018-06-27 16:41:33,537 - root - [Epoch 31 Batch 27200/34064] loss=2.9196, ppl=18.5337, throughput=59.76K wps, wc=6526.64K
2018-06-27 16:43:23,492 - root - [Epoch 31 Batch 28000/34064] loss=2.9329, ppl=18.7820, throughput=59.34K wps, wc=6525.01K
2018-06-27 16:45:12,386 - root - [Epoch 31 Batch 28800/34064] loss=2.9359, ppl=18.8392, throughput=59.92K wps, wc=6525.00K
2018-06-27 16:47:00,812 - root - [Epoch 31 Batch 29600/34064] loss=2.9304, ppl=18.7342, throughput=60.14K wps, wc=6520.24K
2018-06-27 16:48:49,693 - root - [Epoch 31 Batch 30400/34064] loss=2.9320, ppl=18.7645, throughput=59.92K wps, wc=6524.46K
2018-06-27 16:50:37,193 - root - [Epoch 31 Batch 31200/34064] loss=2.9219, ppl=18.5765, throughput=60.67K wps, wc=6521.51K
2018-06-27 16:52:25,752 - root - [Epoch 31 Batch 32000/34064] loss=2.9367, ppl=18.8540, throughput=60.02K wps, wc=6515.40K
2018-06-27 16:54:13,447 - root - [Epoch 31 Batch 32800/34064] loss=2.9321, ppl=18.7662, throughput=60.58K wps, wc=6523.89K
2018-06-27 16:56:01,924 - root - [Epoch 31 Batch 33600/34064] loss=2.9336, ppl=18.7953, throughput=60.13K wps, wc=6522.77K
2018-06-27 17:01:39,056 - root - [Epoch 31] valid Loss=1.5011, valid ppl=4.4867, valid bleu=25.72
2018-06-27 17:06:18,968 - root - [Epoch 31] test Loss=1.3213, test ppl=3.7481, test bleu=26.28
2018-06-27 17:08:11,927 - root - [Epoch 32 Batch 800/34064] loss=2.9034, ppl=18.2356, throughput=57.96K wps, wc=6519.31K
2018-06-27 17:09:59,933 - root - [Epoch 32 Batch 1600/34064] loss=2.8896, ppl=17.9853, throughput=60.41K wps, wc=6524.54K
2018-06-27 17:11:47,453 - root - [Epoch 32 Batch 2400/34064] loss=2.8979, ppl=18.1353, throughput=60.68K wps, wc=6524.08K
2018-06-27 17:13:35,903 - root - [Epoch 32 Batch 3200/34064] loss=2.9090, ppl=18.3384, throughput=60.14K wps, wc=6521.90K
2018-06-27 17:15:27,568 - root - [Epoch 32 Batch 4000/34064] loss=2.9164, ppl=18.4752, throughput=58.45K wps, wc=6526.53K
2018-06-27 17:17:15,058 - root - [Epoch 32 Batch 4800/34064] loss=2.9045, ppl=18.2564, throughput=60.63K wps, wc=6517.23K
2018-06-27 17:19:02,261 - root - [Epoch 32 Batch 5600/34064] loss=2.9008, ppl=18.1894, throughput=60.80K wps, wc=6518.16K
2018-06-27 17:20:49,486 - root - [Epoch 32 Batch 6400/34064] loss=2.9080, ppl=18.3207, throughput=60.84K wps, wc=6523.28K
2018-06-27 17:22:38,662 - root - [Epoch 32 Batch 7200/34064] loss=2.9055, ppl=18.2749, throughput=59.73K wps, wc=6520.77K
2018-06-27 17:24:25,691 - root - [Epoch 32 Batch 8000/34064] loss=2.9036, ppl=18.2391, throughput=60.92K wps, wc=6520.29K
2018-06-27 17:26:12,529 - root - [Epoch 32 Batch 8800/34064] loss=2.9011, ppl=18.1943, throughput=61.11K wps, wc=6528.94K
2018-06-27 17:28:01,425 - root - [Epoch 32 Batch 9600/34064] loss=2.9195, ppl=18.5325, throughput=59.85K wps, wc=6517.55K
2018-06-27 17:29:50,015 - root - [Epoch 32 Batch 10400/34064] loss=2.9020, ppl=18.2108, throughput=60.03K wps, wc=6518.13K
2018-06-27 17:31:38,250 - root - [Epoch 32 Batch 11200/34064] loss=2.9201, ppl=18.5437, throughput=60.29K wps, wc=6525.52K
2018-06-27 17:33:26,196 - root - [Epoch 32 Batch 12000/34064] loss=2.9185, ppl=18.5140, throughput=60.42K wps, wc=6521.85K
2018-06-27 17:35:14,037 - root - [Epoch 32 Batch 12800/34064] loss=2.9098, ppl=18.3536, throughput=60.39K wps, wc=6512.84K
2018-06-27 17:37:01,732 - root - [Epoch 32 Batch 13600/34064] loss=2.9193, ppl=18.5280, throughput=60.61K wps, wc=6526.87K
2018-06-27 17:38:49,461 - root - [Epoch 32 Batch 14400/34064] loss=2.9174, ppl=18.4929, throughput=60.52K wps, wc=6519.49K
2018-06-27 17:40:36,051 - root - [Epoch 32 Batch 15200/34064] loss=2.9193, ppl=18.5276, throughput=61.13K wps, wc=6516.18K
2018-06-27 17:42:24,017 - root - [Epoch 32 Batch 16000/34064] loss=2.9133, ppl=18.4176, throughput=60.38K wps, wc=6518.92K
2018-06-27 17:44:13,267 - root - [Epoch 32 Batch 16800/34064] loss=2.9259, ppl=18.6508, throughput=59.68K wps, wc=6520.52K
2018-06-27 17:46:00,503 - root - [Epoch 32 Batch 17600/34064] loss=2.9045, ppl=18.2561, throughput=60.85K wps, wc=6525.54K
2018-06-27 17:47:49,146 - root - [Epoch 32 Batch 18400/34064] loss=2.9286, ppl=18.7021, throughput=60.00K wps, wc=6518.20K
2018-06-27 17:49:37,304 - root - [Epoch 32 Batch 19200/34064] loss=2.9014, ppl=18.1989, throughput=60.29K wps, wc=6521.30K
2018-06-27 17:51:25,904 - root - [Epoch 32 Batch 20000/34064] loss=2.9315, ppl=18.7563, throughput=60.04K wps, wc=6520.70K
2018-06-27 17:53:12,594 - root - [Epoch 32 Batch 20800/34064] loss=2.9218, ppl=18.5744, throughput=61.16K wps, wc=6524.76K
2018-06-27 17:55:00,460 - root - [Epoch 32 Batch 21600/34064] loss=2.9155, ppl=18.4581, throughput=60.45K wps, wc=6520.98K
2018-06-27 17:56:50,089 - root - [Epoch 32 Batch 22400/34064] loss=2.9338, ppl=18.7989, throughput=59.45K wps, wc=6517.15K
2018-06-27 17:58:37,894 - root - [Epoch 32 Batch 23200/34064] loss=2.9134, ppl=18.4196, throughput=60.49K wps, wc=6521.52K
2018-06-27 18:00:25,871 - root - [Epoch 32 Batch 24000/34064] loss=2.9247, ppl=18.6283, throughput=60.41K wps, wc=6523.00K
2018-06-27 18:02:15,418 - root - [Epoch 32 Batch 24800/34064] loss=2.9342, ppl=18.8059, throughput=59.49K wps, wc=6516.97K
2018-06-27 18:04:03,602 - root - [Epoch 32 Batch 25600/34064] loss=2.9326, ppl=18.7758, throughput=60.25K wps, wc=6518.05K
2018-06-27 18:05:51,781 - root - [Epoch 32 Batch 26400/34064] loss=2.9284, ppl=18.6974, throughput=60.24K wps, wc=6516.38K
2018-06-27 18:07:39,099 - root - [Epoch 32 Batch 27200/34064] loss=2.9146, ppl=18.4421, throughput=60.76K wps, wc=6520.66K
2018-06-27 18:09:26,939 - root - [Epoch 32 Batch 28000/34064] loss=2.9312, ppl=18.7506, throughput=60.50K wps, wc=6524.78K
2018-06-27 18:11:15,597 - root - [Epoch 32 Batch 28800/34064] loss=2.9214, ppl=18.5666, throughput=60.03K wps, wc=6523.22K
2018-06-27 18:13:04,129 - root - [Epoch 32 Batch 29600/34064] loss=2.9245, ppl=18.6250, throughput=60.10K wps, wc=6522.45K
2018-06-27 18:14:51,772 - root - [Epoch 32 Batch 30400/34064] loss=2.9288, ppl=18.7052, throughput=60.60K wps, wc=6522.89K
2018-06-27 18:16:40,290 - root - [Epoch 32 Batch 31200/34064] loss=2.9318, ppl=18.7612, throughput=60.00K wps, wc=6511.13K
2018-06-27 18:18:27,351 - root - [Epoch 32 Batch 32000/34064] loss=2.9216, ppl=18.5717, throughput=60.91K wps, wc=6521.31K
2018-06-27 18:20:15,614 - root - [Epoch 32 Batch 32800/34064] loss=2.9211, ppl=18.5616, throughput=60.27K wps, wc=6524.42K
2018-06-27 18:22:02,739 - root - [Epoch 32 Batch 33600/34064] loss=2.9054, ppl=18.2718, throughput=60.94K wps, wc=6527.70K
2018-06-27 18:27:41,118 - root - [Epoch 32] valid Loss=1.4982, valid ppl=4.4738, valid bleu=25.80
2018-06-27 18:32:14,559 - root - [Epoch 32] test Loss=1.3159, test ppl=3.7280, test bleu=26.43
2018-06-27 18:32:14,565 - root - Save best parameters to transformer_en_de_u512/valid_best.params
2018-06-27 18:34:10,019 - root - [Epoch 33 Batch 800/34064] loss=2.9084, ppl=18.3268, throughput=57.59K wps, wc=6521.17K
2018-06-27 18:35:57,076 - root - [Epoch 33 Batch 1600/34064] loss=2.8976, ppl=18.1307, throughput=60.92K wps, wc=6522.35K
2018-06-27 18:37:43,633 - root - [Epoch 33 Batch 2400/34064] loss=2.8930, ppl=18.0465, throughput=61.20K wps, wc=6521.50K
2018-06-27 18:39:31,455 - root - [Epoch 33 Batch 3200/34064] loss=2.8905, ppl=18.0029, throughput=60.50K wps, wc=6523.27K
2018-06-27 18:41:20,865 - root - [Epoch 33 Batch 4000/34064] loss=2.9105, ppl=18.3664, throughput=59.57K wps, wc=6518.05K
2018-06-27 18:43:08,112 - root - [Epoch 33 Batch 4800/34064] loss=2.8943, ppl=18.0711, throughput=60.84K wps, wc=6524.58K
2018-06-27 18:44:56,324 - root - [Epoch 33 Batch 5600/34064] loss=2.9023, ppl=18.2151, throughput=60.26K wps, wc=6521.13K
2018-06-27 18:46:45,919 - root - [Epoch 33 Batch 6400/34064] loss=2.9043, ppl=18.2517, throughput=59.50K wps, wc=6521.06K
2018-06-27 18:48:33,716 - root - [Epoch 33 Batch 7200/34064] loss=2.9165, ppl=18.4759, throughput=60.52K wps, wc=6523.46K
2018-06-27 18:50:21,166 - root - [Epoch 33 Batch 8000/34064] loss=2.9139, ppl=18.4294, throughput=60.71K wps, wc=6523.30K
2018-06-27 18:52:09,331 - root - [Epoch 33 Batch 8800/34064] loss=2.9151, ppl=18.4515, throughput=60.21K wps, wc=6512.53K
2018-06-27 18:53:57,978 - root - [Epoch 33 Batch 9600/34064] loss=2.9221, ppl=18.5799, throughput=60.01K wps, wc=6519.96K
2018-06-27 18:55:46,463 - root - [Epoch 33 Batch 10400/34064] loss=2.8991, ppl=18.1570, throughput=60.07K wps, wc=6516.20K
2018-06-27 18:57:33,107 - root - [Epoch 33 Batch 11200/34064] loss=2.9010, ppl=18.1920, throughput=61.15K wps, wc=6520.81K
2018-06-27 18:59:18,988 - root - [Epoch 33 Batch 12000/34064] loss=2.9223, ppl=18.5837, throughput=61.61K wps, wc=6523.26K
2018-06-27 19:01:05,144 - root - [Epoch 33 Batch 12800/34064] loss=2.9021, ppl=18.2125, throughput=61.47K wps, wc=6525.51K
2018-06-27 19:02:53,376 - root - [Epoch 33 Batch 13600/34064] loss=2.9005, ppl=18.1839, throughput=60.23K wps, wc=6519.03K
2018-06-27 19:04:42,245 - root - [Epoch 33 Batch 14400/34064] loss=2.9204, ppl=18.5479, throughput=59.94K wps, wc=6525.51K
2018-06-27 19:06:30,568 - root - [Epoch 33 Batch 15200/34064] loss=2.9209, ppl=18.5587, throughput=60.11K wps, wc=6511.76K
2018-06-27 19:08:16,703 - root - [Epoch 33 Batch 16000/34064] loss=2.9096, ppl=18.3501, throughput=61.47K wps, wc=6523.62K
2018-06-27 19:10:05,185 - root - [Epoch 33 Batch 16800/34064] loss=2.9044, ppl=18.2539, throughput=60.09K wps, wc=6518.26K
2018-06-27 19:11:52,256 - root - [Epoch 33 Batch 17600/34064] loss=2.9071, ppl=18.3040, throughput=60.96K wps, wc=6526.48K
2018-06-27 19:13:40,860 - root - [Epoch 33 Batch 18400/34064] loss=2.9277, ppl=18.6843, throughput=60.03K wps, wc=6519.89K
2018-06-27 19:15:29,557 - root - [Epoch 33 Batch 19200/34064] loss=2.9120, ppl=18.3937, throughput=59.97K wps, wc=6518.71K
2018-06-27 19:17:18,198 - root - [Epoch 33 Batch 20000/34064] loss=2.9253, ppl=18.6403, throughput=59.95K wps, wc=6512.84K
2018-06-27 19:19:07,063 - root - [Epoch 33 Batch 20800/34064] loss=2.9224, ppl=18.5856, throughput=59.91K wps, wc=6522.26K
2018-06-27 19:20:57,128 - root - [Epoch 33 Batch 21600/34064] loss=2.9286, ppl=18.7010, throughput=59.25K wps, wc=6520.89K
2018-06-27 19:22:44,695 - root - [Epoch 33 Batch 22400/34064] loss=2.9082, ppl=18.3241, throughput=60.65K wps, wc=6523.45K
2018-06-27 19:24:32,400 - root - [Epoch 33 Batch 23200/34064] loss=2.8936, ppl=18.0575, throughput=60.62K wps, wc=6529.06K
2018-06-27 19:26:19,814 - root - [Epoch 33 Batch 24000/34064] loss=2.9251, ppl=18.6354, throughput=60.74K wps, wc=6524.81K
2018-06-27 19:28:07,653 - root - [Epoch 33 Batch 24800/34064] loss=2.9183, ppl=18.5089, throughput=60.51K wps, wc=6524.84K
2018-06-27 19:29:57,262 - root - [Epoch 33 Batch 25600/34064] loss=2.9226, ppl=18.5899, throughput=59.49K wps, wc=6520.81K
2018-06-27 19:31:45,207 - root - [Epoch 33 Batch 26400/34064] loss=2.9335, ppl=18.7937, throughput=60.41K wps, wc=6520.97K
2018-06-27 19:33:34,814 - root - [Epoch 33 Batch 27200/34064] loss=2.9244, ppl=18.6228, throughput=59.51K wps, wc=6523.01K
2018-06-27 19:35:23,892 - root - [Epoch 33 Batch 28000/34064] loss=2.9234, ppl=18.6053, throughput=59.83K wps, wc=6525.61K
2018-06-27 19:37:11,629 - root - [Epoch 33 Batch 28800/34064] loss=2.9289, ppl=18.7069, throughput=60.54K wps, wc=6521.94K
2018-06-27 19:38:58,195 - root - [Epoch 33 Batch 29600/34064] loss=2.9196, ppl=18.5343, throughput=61.16K wps, wc=6517.15K
2018-06-27 19:40:45,727 - root - [Epoch 33 Batch 30400/34064] loss=2.9155, ppl=18.4578, throughput=60.69K wps, wc=6526.53K
2018-06-27 19:42:33,241 - root - [Epoch 33 Batch 31200/34064] loss=2.9103, ppl=18.3622, throughput=60.60K wps, wc=6515.80K
2018-06-27 19:44:22,216 - root - [Epoch 33 Batch 32000/34064] loss=2.9144, ppl=18.4378, throughput=59.79K wps, wc=6515.45K
2018-06-27 19:46:10,087 - root - [Epoch 33 Batch 32800/34064] loss=2.9319, ppl=18.7628, throughput=60.47K wps, wc=6523.13K
2018-06-27 19:47:56,580 - root - [Epoch 33 Batch 33600/34064] loss=2.9135, ppl=18.4214, throughput=61.23K wps, wc=6520.97K
2018-06-27 19:53:41,493 - root - [Epoch 33] valid Loss=1.4946, valid ppl=4.4576, valid bleu=26.00
2018-06-27 19:58:16,566 - root - [Epoch 33] test Loss=1.3149, test ppl=3.7243, test bleu=26.38
2018-06-27 19:58:16,572 - root - Save best parameters to transformer_en_de_u512/valid_best.params
2018-06-27 20:00:10,669 - root - [Epoch 34 Batch 800/34064] loss=2.8942, ppl=18.0699, throughput=58.29K wps, wc=6521.98K
2018-06-27 20:01:58,756 - root - [Epoch 34 Batch 1600/34064] loss=2.8879, ppl=17.9561, throughput=60.38K wps, wc=6526.48K
2018-06-27 20:03:47,878 - root - [Epoch 34 Batch 2400/34064] loss=2.9092, ppl=18.3426, throughput=59.70K wps, wc=6514.47K
2018-06-27 20:05:36,101 - root - [Epoch 34 Batch 3200/34064] loss=2.8904, ppl=18.0013, throughput=60.31K wps, wc=6527.02K
2018-06-27 20:07:23,983 - root - [Epoch 34 Batch 4000/34064] loss=2.9036, ppl=18.2389, throughput=60.41K wps, wc=6516.89K
2018-06-27 20:09:12,165 - root - [Epoch 34 Batch 4800/34064] loss=2.9095, ppl=18.3472, throughput=60.30K wps, wc=6523.92K
2018-06-27 20:10:58,794 - root - [Epoch 34 Batch 5600/34064] loss=2.8931, ppl=18.0489, throughput=61.18K wps, wc=6523.18K
2018-06-27 20:12:46,076 - root - [Epoch 34 Batch 6400/34064] loss=2.8943, ppl=18.0709, throughput=60.81K wps, wc=6523.98K
2018-06-27 20:14:32,611 - root - [Epoch 34 Batch 7200/34064] loss=2.8962, ppl=18.1058, throughput=61.20K wps, wc=6519.75K
2018-06-27 20:16:20,050 - root - [Epoch 34 Batch 8000/34064] loss=2.9107, ppl=18.3694, throughput=60.72K wps, wc=6524.15K
2018-06-27 20:18:06,134 - root - [Epoch 34 Batch 8800/34064] loss=2.9124, ppl=18.4017, throughput=61.51K wps, wc=6525.41K
2018-06-27 20:19:52,981 - root - [Epoch 34 Batch 9600/34064] loss=2.9133, ppl=18.4180, throughput=61.06K wps, wc=6523.65K
2018-06-27 20:21:38,866 - root - [Epoch 34 Batch 10400/34064] loss=2.9121, ppl=18.3954, throughput=61.58K wps, wc=6519.83K
2018-06-27 20:23:26,350 - root - [Epoch 34 Batch 11200/34064] loss=2.9008, ppl=18.1894, throughput=60.67K wps, wc=6521.02K
2018-06-27 20:25:12,950 - root - [Epoch 34 Batch 12000/34064] loss=2.9066, ppl=18.2937, throughput=61.20K wps, wc=6523.89K
2018-06-27 20:27:01,132 - root - [Epoch 34 Batch 12800/34064] loss=2.8967, ppl=18.1147, throughput=60.28K wps, wc=6521.34K
2018-06-27 20:28:48,537 - root - [Epoch 34 Batch 13600/34064] loss=2.9044, ppl=18.2551, throughput=60.73K wps, wc=6522.23K
2018-06-27 20:30:38,000 - root - [Epoch 34 Batch 14400/34064] loss=2.9138, ppl=18.4274, throughput=59.63K wps, wc=6527.09K
2018-06-27 20:32:24,538 - root - [Epoch 34 Batch 15200/34064] loss=2.9074, ppl=18.3085, throughput=61.18K wps, wc=6517.68K
2018-06-27 20:34:13,326 - root - [Epoch 34 Batch 16000/34064] loss=2.9085, ppl=18.3293, throughput=59.92K wps, wc=6518.61K
2018-06-27 20:36:00,086 - root - [Epoch 34 Batch 16800/34064] loss=2.8986, ppl=18.1488, throughput=61.09K wps, wc=6522.37K
2018-06-27 20:37:47,043 - root - [Epoch 34 Batch 17600/34064] loss=2.9145, ppl=18.4396, throughput=60.96K wps, wc=6520.39K
2018-06-27 20:39:34,302 - root - [Epoch 34 Batch 18400/34064] loss=2.8987, ppl=18.1511, throughput=60.78K wps, wc=6519.11K
2018-06-27 20:41:22,915 - root - [Epoch 34 Batch 19200/34064] loss=2.9092, ppl=18.3427, throughput=60.02K wps, wc=6519.00K
2018-06-27 20:43:08,988 - root - [Epoch 34 Batch 20000/34064] loss=2.9092, ppl=18.3417, throughput=61.48K wps, wc=6521.44K
2018-06-27 20:44:56,582 - root - [Epoch 34 Batch 20800/34064] loss=2.9017, ppl=18.2051, throughput=60.63K wps, wc=6523.44K
2018-06-27 20:46:43,181 - root - [Epoch 34 Batch 21600/34064] loss=2.9219, ppl=18.5766, throughput=61.17K wps, wc=6520.60K
2018-06-27 20:48:30,415 - root - [Epoch 34 Batch 22400/34064] loss=2.9265, ppl=18.6622, throughput=60.78K wps, wc=6517.74K
2018-06-27 20:50:17,117 - root - [Epoch 34 Batch 23200/34064] loss=2.9200, ppl=18.5421, throughput=61.12K wps, wc=6521.83K
2018-06-27 20:52:04,422 - root - [Epoch 34 Batch 24000/34064] loss=2.9097, ppl=18.3518, throughput=60.76K wps, wc=6519.67K
2018-06-27 20:53:52,138 - root - [Epoch 34 Batch 24800/34064] loss=2.9341, ppl=18.8049, throughput=60.50K wps, wc=6516.87K
2018-06-27 20:55:39,488 - root - [Epoch 34 Batch 25600/34064] loss=2.9212, ppl=18.5642, throughput=60.67K wps, wc=6513.30K
2018-06-27 20:57:28,146 - root - [Epoch 34 Batch 26400/34064] loss=2.9242, ppl=18.6202, throughput=60.03K wps, wc=6522.89K
2018-06-27 20:59:16,674 - root - [Epoch 34 Batch 27200/34064] loss=2.9231, ppl=18.5988, throughput=60.13K wps, wc=6525.94K
2018-06-27 21:01:05,307 - root - [Epoch 34 Batch 28000/34064] loss=2.9292, ppl=18.7122, throughput=59.99K wps, wc=6517.37K
2018-06-27 21:02:54,015 - root - [Epoch 34 Batch 28800/34064] loss=2.9215, ppl=18.5691, throughput=59.97K wps, wc=6518.65K
2018-06-27 21:04:41,169 - root - [Epoch 34 Batch 29600/34064] loss=2.9120, ppl=18.3938, throughput=60.87K wps, wc=6522.86K
2018-06-27 21:06:30,169 - root - [Epoch 34 Batch 30400/34064] loss=2.9254, ppl=18.6410, throughput=59.75K wps, wc=6512.90K
2018-06-27 21:08:18,511 - root - [Epoch 34 Batch 31200/34064] loss=2.9232, ppl=18.6001, throughput=60.22K wps, wc=6524.20K
2018-06-27 21:10:05,597 - root - [Epoch 34 Batch 32000/34064] loss=2.9247, ppl=18.6283, throughput=60.90K wps, wc=6521.43K
2018-06-27 21:11:54,232 - root - [Epoch 34 Batch 32800/34064] loss=2.9096, ppl=18.3490, throughput=60.04K wps, wc=6522.16K
2018-06-27 21:13:43,067 - root - [Epoch 34 Batch 33600/34064] loss=2.9363, ppl=18.8461, throughput=59.88K wps, wc=6516.47K
2018-06-27 21:19:19,420 - root - [Epoch 34] valid Loss=1.4981, valid ppl=4.4731, valid bleu=25.76
2018-06-27 21:23:54,859 - root - [Epoch 34] test Loss=1.3165, test ppl=3.7304, test bleu=26.56
2018-06-27 21:25:51,011 - root - [Epoch 35 Batch 800/34064] loss=2.8958, ppl=18.0973, throughput=56.39K wps, wc=6524.08K
2018-06-27 21:27:41,403 - root - [Epoch 35 Batch 1600/34064] loss=2.8921, ppl=18.0312, throughput=59.06K wps, wc=6519.95K
2018-06-27 21:29:32,687 - root - [Epoch 35 Batch 2400/34064] loss=2.9067, ppl=18.2962, throughput=58.56K wps, wc=6517.21K
2018-06-27 21:31:24,010 - root - [Epoch 35 Batch 3200/34064] loss=2.8869, ppl=17.9371, throughput=58.60K wps, wc=6523.82K
2018-06-27 21:33:14,608 - root - [Epoch 35 Batch 4000/34064] loss=2.8955, ppl=18.0927, throughput=58.97K wps, wc=6521.64K
2018-06-27 21:35:03,983 - root - [Epoch 35 Batch 4800/34064] loss=2.9049, ppl=18.2643, throughput=59.64K wps, wc=6523.34K
2018-06-27 21:36:54,720 - root - [Epoch 35 Batch 5600/34064] loss=2.8995, ppl=18.1642, throughput=58.91K wps, wc=6522.92K
2018-06-27 21:38:45,803 - root - [Epoch 35 Batch 6400/34064] loss=2.8942, ppl=18.0694, throughput=58.74K wps, wc=6524.48K
2018-06-27 21:40:35,508 - root - [Epoch 35 Batch 7200/34064] loss=2.9009, ppl=18.1906, throughput=59.45K wps, wc=6522.43K
2018-06-27 21:42:25,002 - root - [Epoch 35 Batch 8000/34064] loss=2.8995, ppl=18.1644, throughput=59.59K wps, wc=6524.92K
2018-06-27 21:44:14,611 - root - [Epoch 35 Batch 8800/34064] loss=2.9035, ppl=18.2376, throughput=59.46K wps, wc=6517.21K
2018-06-27 21:46:04,838 - root - [Epoch 35 Batch 9600/34064] loss=2.9198, ppl=18.5382, throughput=59.17K wps, wc=6521.68K
2018-06-27 21:47:54,589 - root - [Epoch 35 Batch 10400/34064] loss=2.9135, ppl=18.4206, throughput=59.40K wps, wc=6519.58K
2018-06-27 21:49:44,725 - root - [Epoch 35 Batch 11200/34064] loss=2.9045, ppl=18.2564, throughput=59.17K wps, wc=6517.14K
2018-06-27 21:51:35,118 - root - [Epoch 35 Batch 12000/34064] loss=2.8944, ppl=18.0732, throughput=59.13K wps, wc=6527.38K
2018-06-27 21:53:25,178 - root - [Epoch 35 Batch 12800/34064] loss=2.9104, ppl=18.3633, throughput=59.26K wps, wc=6522.47K
2018-06-27 21:55:15,268 - root - [Epoch 35 Batch 13600/34064] loss=2.9065, ppl=18.2933, throughput=59.26K wps, wc=6523.69K
2018-06-27 21:57:06,526 - root - [Epoch 35 Batch 14400/34064] loss=2.9175, ppl=18.4946, throughput=58.55K wps, wc=6514.63K
2018-06-27 21:58:56,141 - root - [Epoch 35 Batch 15200/34064] loss=2.9178, ppl=18.4997, throughput=59.48K wps, wc=6520.21K
2018-06-27 22:00:45,988 - root - [Epoch 35 Batch 16000/34064] loss=2.8967, ppl=18.1141, throughput=59.38K wps, wc=6522.55K
2018-06-27 22:02:35,960 - root - [Epoch 35 Batch 16800/34064] loss=2.9096, ppl=18.3491, throughput=59.31K wps, wc=6522.26K
2018-06-27 22:04:25,602 - root - [Epoch 35 Batch 17600/34064] loss=2.9046, ppl=18.2583, throughput=59.56K wps, wc=6530.49K
2018-06-27 22:06:17,392 - root - [Epoch 35 Batch 18400/34064] loss=2.9190, ppl=18.5236, throughput=58.29K wps, wc=6516.26K
2018-06-27 22:08:08,176 - root - [Epoch 35 Batch 19200/34064] loss=2.9127, ppl=18.4073, throughput=58.82K wps, wc=6516.74K
2018-06-27 22:09:59,283 - root - [Epoch 35 Batch 20000/34064] loss=2.9234, ppl=18.6038, throughput=58.69K wps, wc=6520.59K
2018-06-27 22:11:49,389 - root - [Epoch 35 Batch 20800/34064] loss=2.9043, ppl=18.2521, throughput=59.20K wps, wc=6518.05K
2018-06-27 22:13:39,585 - root - [Epoch 35 Batch 21600/34064] loss=2.9034, ppl=18.2357, throughput=59.22K wps, wc=6526.24K
2018-06-27 22:15:29,135 - root - [Epoch 35 Batch 22400/34064] loss=2.9103, ppl=18.3626, throughput=59.55K wps, wc=6523.95K
2018-06-27 22:17:19,717 - root - [Epoch 35 Batch 23200/34064] loss=2.9161, ppl=18.4688, throughput=58.98K wps, wc=6522.35K
2018-06-27 22:19:09,313 - root - [Epoch 35 Batch 24000/34064] loss=2.9182, ppl=18.5078, throughput=59.48K wps, wc=6518.71K
2018-06-27 22:20:59,657 - root - [Epoch 35 Batch 24800/34064] loss=2.9109, ppl=18.3739, throughput=59.13K wps, wc=6524.28K
2018-06-27 22:22:50,218 - root - [Epoch 35 Batch 25600/34064] loss=2.9027, ppl=18.2233, throughput=58.99K wps, wc=6521.64K
2018-06-27 22:24:41,033 - root - [Epoch 35 Batch 26400/34064] loss=2.9245, ppl=18.6255, throughput=58.78K wps, wc=6513.65K
2018-06-27 22:26:32,317 - root - [Epoch 35 Batch 27200/34064] loss=2.9106, ppl=18.3687, throughput=58.59K wps, wc=6520.39K
2018-06-27 22:28:22,690 - root - [Epoch 35 Batch 28000/34064] loss=2.9074, ppl=18.3087, throughput=59.06K wps, wc=6519.09K
2018-06-27 22:30:14,458 - root - [Epoch 35 Batch 28800/34064] loss=2.9295, ppl=18.7191, throughput=58.27K wps, wc=6512.71K
2018-06-27 22:32:04,197 - root - [Epoch 35 Batch 29600/34064] loss=2.9047, ppl=18.2604, throughput=59.42K wps, wc=6520.15K
2018-06-27 22:33:54,577 - root - [Epoch 35 Batch 30400/34064] loss=2.9082, ppl=18.3232, throughput=59.10K wps, wc=6523.93K
2018-06-27 22:35:45,291 - root - [Epoch 35 Batch 31200/34064] loss=2.9268, ppl=18.6674, throughput=58.88K wps, wc=6518.36K
2018-06-27 22:37:36,200 - root - [Epoch 35 Batch 32000/34064] loss=2.8990, ppl=18.1568, throughput=58.82K wps, wc=6524.06K
2018-06-27 22:39:26,317 - root - [Epoch 35 Batch 32800/34064] loss=2.9131, ppl=18.4143, throughput=59.26K wps, wc=6525.59K
2018-06-27 22:41:16,980 - root - [Epoch 35 Batch 33600/34064] loss=2.9199, ppl=18.5391, throughput=58.92K wps, wc=6520.27K
2018-06-27 22:47:02,043 - root - [Epoch 35] valid Loss=1.4971, valid ppl=4.4688, valid bleu=25.99
2018-06-27 22:51:35,350 - root - [Epoch 35] test Loss=1.3154, test ppl=3.7262, test bleu=26.48
2018-06-27 22:53:31,348 - root - [Epoch 36 Batch 800/34064] loss=2.8892, ppl=17.9792, throughput=56.46K wps, wc=6523.03K
2018-06-27 22:55:22,229 - root - [Epoch 36 Batch 1600/34064] loss=2.8770, ppl=17.7603, throughput=58.79K wps, wc=6518.84K
2018-06-27 22:57:13,339 - root - [Epoch 36 Batch 2400/34064] loss=2.8825, ppl=17.8596, throughput=58.75K wps, wc=6527.94K
2018-06-27 22:59:03,904 - root - [Epoch 36 Batch 3200/34064] loss=2.8906, ppl=18.0045, throughput=58.98K wps, wc=6521.32K
2018-06-27 23:00:53,058 - root - [Epoch 36 Batch 4000/34064] loss=2.8902, ppl=17.9970, throughput=59.77K wps, wc=6523.62K
2018-06-27 23:02:45,180 - root - [Epoch 36 Batch 4800/34064] loss=2.9053, ppl=18.2715, throughput=58.12K wps, wc=6516.10K
2018-06-27 23:04:36,323 - root - [Epoch 36 Batch 5600/34064] loss=2.9003, ppl=18.1788, throughput=58.61K wps, wc=6514.43K
2018-06-27 23:06:26,175 - root - [Epoch 36 Batch 6400/34064] loss=2.8984, ppl=18.1453, throughput=59.35K wps, wc=6519.12K
2018-06-27 23:08:17,043 - root - [Epoch 36 Batch 7200/34064] loss=2.9016, ppl=18.2041, throughput=58.83K wps, wc=6522.42K
2018-06-27 23:10:09,175 - root - [Epoch 36 Batch 8000/34064] loss=2.9017, ppl=18.2044, throughput=58.12K wps, wc=6517.25K
2018-06-27 23:11:59,570 - root - [Epoch 36 Batch 8800/34064] loss=2.9073, ppl=18.3074, throughput=59.07K wps, wc=6521.17K
2018-06-27 23:13:48,341 - root - [Epoch 36 Batch 9600/34064] loss=2.8967, ppl=18.1148, throughput=59.94K wps, wc=6520.17K
2018-06-27 23:15:37,212 - root - [Epoch 36 Batch 10400/34064] loss=2.9007, ppl=18.1877, throughput=59.89K wps, wc=6520.21K
2018-06-27 23:17:26,712 - root - [Epoch 36 Batch 11200/34064] loss=2.9012, ppl=18.1957, throughput=59.47K wps, wc=6511.56K
2018-06-27 23:19:17,076 - root - [Epoch 36 Batch 12000/34064] loss=2.9084, ppl=18.3274, throughput=59.14K wps, wc=6526.72K
2018-06-27 23:21:06,725 - root - [Epoch 36 Batch 12800/34064] loss=2.8973, ppl=18.1260, throughput=59.52K wps, wc=6525.98K
2018-06-27 23:22:57,469 - root - [Epoch 36 Batch 13600/34064] loss=2.9127, ppl=18.4067, throughput=58.85K wps, wc=6517.37K
2018-06-27 23:24:49,135 - root - [Epoch 36 Batch 14400/34064] loss=2.9131, ppl=18.4140, throughput=58.43K wps, wc=6524.78K
2018-06-27 23:26:39,347 - root - [Epoch 36 Batch 15200/34064] loss=2.8949, ppl=18.0821, throughput=59.24K wps, wc=6529.12K
2018-06-27 23:28:28,786 - root - [Epoch 36 Batch 16000/34064] loss=2.8935, ppl=18.0560, throughput=59.62K wps, wc=6524.49K
2018-06-27 23:30:20,136 - root - [Epoch 36 Batch 16800/34064] loss=2.9067, ppl=18.2971, throughput=58.45K wps, wc=6508.00K
2018-06-27 23:32:10,441 - root - [Epoch 36 Batch 17600/34064] loss=2.9014, ppl=18.2001, throughput=59.13K wps, wc=6522.44K
2018-06-27 23:34:00,860 - root - [Epoch 36 Batch 18400/34064] loss=2.9076, ppl=18.3123, throughput=59.08K wps, wc=6523.72K
2018-06-27 23:35:51,200 - root - [Epoch 36 Batch 19200/34064] loss=2.9124, ppl=18.4004, throughput=59.15K wps, wc=6527.04K
2018-06-27 23:37:42,615 - root - [Epoch 36 Batch 20000/34064] loss=2.9207, ppl=18.5539, throughput=58.44K wps, wc=6511.03K
2018-06-27 23:39:33,705 - root - [Epoch 36 Batch 20800/34064] loss=2.9122, ppl=18.3969, throughput=58.69K wps, wc=6520.27K
2018-06-27 23:41:25,556 - root - [Epoch 36 Batch 21600/34064] loss=2.9156, ppl=18.4597, throughput=58.34K wps, wc=6524.89K
2018-06-27 23:43:16,300 - root - [Epoch 36 Batch 22400/34064] loss=2.9120, ppl=18.3937, throughput=58.85K wps, wc=6516.68K
2018-06-27 23:45:06,210 - root - [Epoch 36 Batch 23200/34064] loss=2.9126, ppl=18.4053, throughput=59.36K wps, wc=6524.70K
2018-06-27 23:46:55,717 - root - [Epoch 36 Batch 24000/34064] loss=2.9060, ppl=18.2844, throughput=59.54K wps, wc=6520.24K
2018-06-27 23:48:45,357 - root - [Epoch 36 Batch 24800/34064] loss=2.9041, ppl=18.2486, throughput=59.45K wps, wc=6517.62K
2018-06-27 23:50:36,518 - root - [Epoch 36 Batch 25600/34064] loss=2.9085, ppl=18.3300, throughput=58.71K wps, wc=6526.09K
2018-06-27 23:52:25,987 - root - [Epoch 36 Batch 26400/34064] loss=2.9111, ppl=18.3772, throughput=59.52K wps, wc=6515.85K
2018-06-27 23:54:17,643 - root - [Epoch 36 Batch 27200/34064] loss=2.9114, ppl=18.3832, throughput=58.38K wps, wc=6518.67K
2018-06-27 23:56:06,822 - root - [Epoch 36 Batch 28000/34064] loss=2.8947, ppl=18.0781, throughput=59.77K wps, wc=6525.49K
2018-06-27 23:57:58,433 - root - [Epoch 36 Batch 28800/34064] loss=2.9341, ppl=18.8052, throughput=58.41K wps, wc=6519.02K
2018-06-27 23:59:48,816 - root - [Epoch 36 Batch 29600/34064] loss=2.9149, ppl=18.4470, throughput=59.06K wps, wc=6519.59K
2018-06-28 00:01:39,984 - root - [Epoch 36 Batch 30400/34064] loss=2.9265, ppl=18.6623, throughput=58.74K wps, wc=6530.37K
2018-06-28 00:03:31,015 - root - [Epoch 36 Batch 31200/34064] loss=2.9086, ppl=18.3306, throughput=58.75K wps, wc=6522.99K
2018-06-28 00:05:20,354 - root - [Epoch 36 Batch 32000/34064] loss=2.9171, ppl=18.4875, throughput=59.64K wps, wc=6521.36K
2018-06-28 00:07:11,160 - root - [Epoch 36 Batch 32800/34064] loss=2.9314, ppl=18.7531, throughput=58.81K wps, wc=6516.68K
2018-06-28 00:09:01,193 - root - [Epoch 36 Batch 33600/34064] loss=2.9120, ppl=18.3941, throughput=59.32K wps, wc=6527.18K
2018-06-28 00:14:40,331 - root - [Epoch 36] valid Loss=1.4950, valid ppl=4.4593, valid bleu=25.91
2018-06-28 00:19:14,009 - root - [Epoch 36] test Loss=1.3121, test ppl=3.7138, test bleu=26.45
2018-06-28 00:21:07,608 - root - [Epoch 37 Batch 800/34064] loss=2.8789, ppl=17.7941, throughput=57.62K wps, wc=6517.42K
2018-06-28 00:22:58,886 - root - [Epoch 37 Batch 1600/34064] loss=2.8904, ppl=18.0008, throughput=58.62K wps, wc=6522.76K
2018-06-28 00:24:49,018 - root - [Epoch 37 Batch 2400/34064] loss=2.8791, ppl=17.7982, throughput=59.26K wps, wc=6525.97K
2018-06-28 00:26:39,629 - root - [Epoch 37 Batch 3200/34064] loss=2.8956, ppl=18.0948, throughput=58.94K wps, wc=6519.27K
2018-06-28 00:28:29,172 - root - [Epoch 37 Batch 4000/34064] loss=2.8787, ppl=17.7916, throughput=59.59K wps, wc=6527.20K
2018-06-28 00:30:20,237 - root - [Epoch 37 Batch 4800/34064] loss=2.8975, ppl=18.1279, throughput=58.75K wps, wc=6524.76K
2018-06-28 00:32:09,949 - root - [Epoch 37 Batch 5600/34064] loss=2.8847, ppl=17.8979, throughput=59.49K wps, wc=6526.54K
2018-06-28 00:34:00,281 - root - [Epoch 37 Batch 6400/34064] loss=2.9017, ppl=18.2059, throughput=59.16K wps, wc=6527.54K
2018-06-28 00:35:49,842 - root - [Epoch 37 Batch 7200/34064] loss=2.9086, ppl=18.3303, throughput=59.55K wps, wc=6524.04K
2018-06-28 00:37:39,176 - root - [Epoch 37 Batch 8000/34064] loss=2.8894, ppl=17.9833, throughput=59.64K wps, wc=6520.30K
2018-06-28 00:39:29,479 - root - [Epoch 37 Batch 8800/34064] loss=2.8921, ppl=18.0312, throughput=59.17K wps, wc=6526.19K
2018-06-28 00:41:19,562 - root - [Epoch 37 Batch 9600/34064] loss=2.8932, ppl=18.0509, throughput=59.21K wps, wc=6518.24K
2018-06-28 00:43:08,592 - root - [Epoch 37 Batch 10400/34064] loss=2.9031, ppl=18.2311, throughput=59.82K wps, wc=6522.20K
2018-06-28 00:44:58,056 - root - [Epoch 37 Batch 11200/34064] loss=2.8981, ppl=18.1405, throughput=59.54K wps, wc=6517.64K
2018-06-28 00:46:49,018 - root - [Epoch 37 Batch 12000/34064] loss=2.9113, ppl=18.3805, throughput=58.69K wps, wc=6512.50K
2018-06-28 00:48:40,166 - root - [Epoch 37 Batch 12800/34064] loss=2.9034, ppl=18.2367, throughput=58.69K wps, wc=6523.69K
2018-06-28 00:50:30,482 - root - [Epoch 37 Batch 13600/34064] loss=2.8921, ppl=18.0311, throughput=59.08K wps, wc=6517.01K
2018-06-28 00:52:20,091 - root - [Epoch 37 Batch 14400/34064] loss=2.8974, ppl=18.1265, throughput=59.55K wps, wc=6527.49K
2018-06-28 00:54:09,962 - root - [Epoch 37 Batch 15200/34064] loss=2.9023, ppl=18.2155, throughput=59.40K wps, wc=6526.50K
2018-06-28 00:56:01,959 - root - [Epoch 37 Batch 16000/34064] loss=2.9043, ppl=18.2517, throughput=58.19K wps, wc=6516.66K
2018-06-28 00:57:51,659 - root - [Epoch 37 Batch 16800/34064] loss=2.9015, ppl=18.2020, throughput=59.46K wps, wc=6523.00K
2018-06-28 00:59:42,233 - root - [Epoch 37 Batch 17600/34064] loss=2.9089, ppl=18.3372, throughput=58.96K wps, wc=6518.92K
2018-06-28 01:01:33,380 - root - [Epoch 37 Batch 18400/34064] loss=2.9132, ppl=18.4160, throughput=58.64K wps, wc=6517.37K
2018-06-28 01:03:25,387 - root - [Epoch 37 Batch 19200/34064] loss=2.9204, ppl=18.5480, throughput=58.20K wps, wc=6519.06K
2018-06-28 01:05:15,496 - root - [Epoch 37 Batch 20000/34064] loss=2.9153, ppl=18.4540, throughput=59.24K wps, wc=6522.83K
2018-06-28 01:07:05,789 - root - [Epoch 37 Batch 20800/34064] loss=2.8980, ppl=18.1380, throughput=59.09K wps, wc=6517.38K
2018-06-28 01:08:56,759 - root - [Epoch 37 Batch 21600/34064] loss=2.9141, ppl=18.4318, throughput=58.74K wps, wc=6518.56K
2018-06-28 01:10:46,623 - root - [Epoch 37 Batch 22400/34064] loss=2.9054, ppl=18.2719, throughput=59.35K wps, wc=6519.85K
2018-06-28 01:12:37,607 - root - [Epoch 37 Batch 23200/34064] loss=2.9138, ppl=18.4273, throughput=58.73K wps, wc=6518.36K
2018-06-28 01:14:28,220 - root - [Epoch 37 Batch 24000/34064] loss=2.9097, ppl=18.3517, throughput=59.00K wps, wc=6526.07K
2018-06-28 01:16:20,619 - root - [Epoch 37 Batch 24800/34064] loss=2.9078, ppl=18.3159, throughput=58.09K wps, wc=6529.31K
2018-06-28 01:18:11,492 - root - [Epoch 37 Batch 25600/34064] loss=2.9130, ppl=18.4113, throughput=58.75K wps, wc=6513.38K
2018-06-28 01:20:01,729 - root - [Epoch 37 Batch 26400/34064] loss=2.9124, ppl=18.4007, throughput=59.19K wps, wc=6524.79K
2018-06-28 01:21:50,027 - root - [Epoch 37 Batch 27200/34064] loss=2.9166, ppl=18.4784, throughput=60.18K wps, wc=6517.11K
2018-06-28 01:23:41,686 - root - [Epoch 37 Batch 28000/34064] loss=2.9088, ppl=18.3352, throughput=58.36K wps, wc=6516.35K
2018-06-28 01:25:31,521 - root - [Epoch 37 Batch 28800/34064] loss=2.9139, ppl=18.4292, throughput=59.33K wps, wc=6516.76K
2018-06-28 01:27:22,747 - root - [Epoch 37 Batch 29600/34064] loss=2.9164, ppl=18.4739, throughput=58.66K wps, wc=6524.94K
2018-06-28 01:29:13,356 - root - [Epoch 37 Batch 30400/34064] loss=2.9163, ppl=18.4728, throughput=58.95K wps, wc=6520.28K
2018-06-28 01:31:02,675 - root - [Epoch 37 Batch 31200/34064] loss=2.9042, ppl=18.2498, throughput=59.65K wps, wc=6520.62K
2018-06-28 01:32:53,449 - root - [Epoch 37 Batch 32000/34064] loss=2.9166, ppl=18.4791, throughput=58.83K wps, wc=6516.94K
2018-06-28 01:34:43,584 - root - [Epoch 37 Batch 32800/34064] loss=2.8957, ppl=18.0962, throughput=59.15K wps, wc=6513.96K
2018-06-28 01:36:33,804 - root - [Epoch 37 Batch 33600/34064] loss=2.9148, ppl=18.4457, throughput=59.20K wps, wc=6525.17K
2018-06-28 01:42:12,576 - root - [Epoch 37] valid Loss=1.4951, valid ppl=4.4597, valid bleu=25.73
2018-06-28 01:46:45,386 - root - [Epoch 37] test Loss=1.3136, test ppl=3.7195, test bleu=26.48
2018-06-28 01:48:39,211 - root - [Epoch 38 Batch 800/34064] loss=2.8856, ppl=17.9135, throughput=57.57K wps, wc=6526.18K
2018-06-28 01:50:30,118 - root - [Epoch 38 Batch 1600/34064] loss=2.8830, ppl=17.8669, throughput=58.68K wps, wc=6507.96K
2018-06-28 01:52:20,061 - root - [Epoch 38 Batch 2400/34064] loss=2.8960, ppl=18.1009, throughput=59.33K wps, wc=6522.65K
2018-06-28 01:54:11,819 - root - [Epoch 38 Batch 3200/34064] loss=2.8895, ppl=17.9849, throughput=58.30K wps, wc=6515.55K
2018-06-28 01:56:01,528 - root - [Epoch 38 Batch 4000/34064] loss=2.8773, ppl=17.7668, throughput=59.44K wps, wc=6520.93K
2018-06-28 01:57:50,401 - root - [Epoch 38 Batch 4800/34064] loss=2.8892, ppl=17.9787, throughput=59.88K wps, wc=6519.36K
2018-06-28 01:59:39,510 - root - [Epoch 38 Batch 5600/34064] loss=2.8860, ppl=17.9217, throughput=59.80K wps, wc=6524.19K
2018-06-28 02:01:28,806 - root - [Epoch 38 Batch 6400/34064] loss=2.8830, ppl=17.8682, throughput=59.68K wps, wc=6522.36K
2018-06-28 02:03:17,735 - root - [Epoch 38 Batch 7200/34064] loss=2.9031, ppl=18.2300, throughput=59.81K wps, wc=6515.23K
2018-06-28 02:05:07,119 - root - [Epoch 38 Batch 8000/34064] loss=2.8811, ppl=17.8341, throughput=59.61K wps, wc=6520.64K
2018-06-28 02:06:57,469 - root - [Epoch 38 Batch 8800/34064] loss=2.8968, ppl=18.1163, throughput=59.13K wps, wc=6524.72K
2018-06-28 02:08:47,527 - root - [Epoch 38 Batch 9600/34064] loss=2.8875, ppl=17.9475, throughput=59.30K wps, wc=6526.23K
2018-06-28 02:10:37,056 - root - [Epoch 38 Batch 10400/34064] loss=2.9069, ppl=18.2999, throughput=59.51K wps, wc=6518.16K
2018-06-28 02:12:27,982 - root - [Epoch 38 Batch 11200/34064] loss=2.9041, ppl=18.2493, throughput=58.70K wps, wc=6511.40K
2018-06-28 02:14:18,794 - root - [Epoch 38 Batch 12000/34064] loss=2.9021, ppl=18.2118, throughput=58.80K wps, wc=6515.91K
2018-06-28 02:16:08,795 - root - [Epoch 38 Batch 12800/34064] loss=2.8838, ppl=17.8827, throughput=59.34K wps, wc=6526.86K
2018-06-28 02:17:58,165 - root - [Epoch 38 Batch 13600/34064] loss=2.8874, ppl=17.9464, throughput=59.69K wps, wc=6527.67K
2018-06-28 02:19:48,700 - root - [Epoch 38 Batch 14400/34064] loss=2.8879, ppl=17.9559, throughput=59.06K wps, wc=6528.39K
2018-06-28 02:21:40,360 - root - [Epoch 38 Batch 15200/34064] loss=2.9000, ppl=18.1743, throughput=58.47K wps, wc=6529.11K
2018-06-28 02:23:33,772 - root - [Epoch 38 Batch 16000/34064] loss=2.9081, ppl=18.3214, throughput=57.49K wps, wc=6519.88K
2018-06-28 02:25:24,178 - root - [Epoch 38 Batch 16800/34064] loss=2.9006, ppl=18.1842, throughput=59.06K wps, wc=6520.08K
2018-06-28 02:27:15,723 - root - [Epoch 38 Batch 17600/34064] loss=2.9163, ppl=18.4726, throughput=58.50K wps, wc=6525.16K
2018-06-28 02:29:06,741 - root - [Epoch 38 Batch 18400/34064] loss=2.9047, ppl=18.2597, throughput=58.75K wps, wc=6522.44K
2018-06-28 02:30:57,484 - root - [Epoch 38 Batch 19200/34064] loss=2.9001, ppl=18.1765, throughput=58.92K wps, wc=6524.85K
2018-06-28 02:32:47,903 - root - [Epoch 38 Batch 20000/34064] loss=2.9145, ppl=18.4401, throughput=59.05K wps, wc=6520.35K
2018-06-28 02:34:39,672 - root - [Epoch 38 Batch 20800/34064] loss=2.9141, ppl=18.4329, throughput=58.35K wps, wc=6522.03K
2018-06-28 02:36:29,328 - root - [Epoch 38 Batch 21600/34064] loss=2.9038, ppl=18.2442, throughput=59.48K wps, wc=6522.19K
2018-06-28 02:38:22,147 - root - [Epoch 38 Batch 22400/34064] loss=2.8990, ppl=18.1554, throughput=57.81K wps, wc=6522.19K
2018-06-28 02:40:13,702 - root - [Epoch 38 Batch 23200/34064] loss=2.9158, ppl=18.4631, throughput=58.43K wps, wc=6518.30K
2018-06-28 02:42:03,729 - root - [Epoch 38 Batch 24000/34064] loss=2.9000, ppl=18.1736, throughput=59.30K wps, wc=6524.79K
2018-06-28 02:43:53,954 - root - [Epoch 38 Batch 24800/34064] loss=2.9113, ppl=18.3814, throughput=59.12K wps, wc=6516.63K
2018-06-28 02:45:45,400 - root - [Epoch 38 Batch 25600/34064] loss=2.9240, ppl=18.6162, throughput=58.49K wps, wc=6518.41K
2018-06-28 02:47:36,638 - root - [Epoch 38 Batch 26400/34064] loss=2.9129, ppl=18.4108, throughput=58.65K wps, wc=6523.85K
2018-06-28 02:49:27,195 - root - [Epoch 38 Batch 27200/34064] loss=2.9062, ppl=18.2869, throughput=58.94K wps, wc=6516.24K
2018-06-28 02:51:15,780 - root - [Epoch 38 Batch 28000/34064] loss=2.9013, ppl=18.1983, throughput=60.09K wps, wc=6525.20K
2018-06-28 02:53:06,413 - root - [Epoch 38 Batch 28800/34064] loss=2.9191, ppl=18.5245, throughput=58.95K wps, wc=6521.21K
2018-06-28 02:54:56,023 - root - [Epoch 38 Batch 29600/34064] loss=2.9071, ppl=18.3044, throughput=59.51K wps, wc=6522.42K
2018-06-28 02:56:46,266 - root - [Epoch 38 Batch 30400/34064] loss=2.9052, ppl=18.2695, throughput=59.08K wps, wc=6513.62K
2018-06-28 02:58:35,812 - root - [Epoch 38 Batch 31200/34064] loss=2.9022, ppl=18.2145, throughput=59.50K wps, wc=6517.42K
2018-06-28 03:00:24,744 - root - [Epoch 38 Batch 32000/34064] loss=2.9154, ppl=18.4553, throughput=59.90K wps, wc=6525.03K
2018-06-28 03:02:14,020 - root - [Epoch 38 Batch 32800/34064] loss=2.9165, ppl=18.4774, throughput=59.66K wps, wc=6519.54K
2018-06-28 03:04:03,360 - root - [Epoch 38 Batch 33600/34064] loss=2.9185, ppl=18.5144, throughput=59.64K wps, wc=6521.22K
2018-06-28 03:09:54,948 - root - [Epoch 38] valid Loss=1.4931, valid ppl=4.4507, valid bleu=25.82
2018-06-28 03:14:28,344 - root - [Epoch 38] test Loss=1.3106, test ppl=3.7086, test bleu=26.74
2018-06-28 03:16:24,585 - root - [Epoch 39 Batch 800/34064] loss=2.8906, ppl=18.0033, throughput=56.33K wps, wc=6523.00K
2018-06-28 03:18:15,220 - root - [Epoch 39 Batch 1600/34064] loss=2.8959, ppl=18.1001, throughput=58.92K wps, wc=6519.14K
2018-06-28 03:20:05,089 - root - [Epoch 39 Batch 2400/34064] loss=2.8803, ppl=17.8191, throughput=59.38K wps, wc=6524.10K
2018-06-28 03:21:53,699 - root - [Epoch 39 Batch 3200/34064] loss=2.8751, ppl=17.7269, throughput=60.04K wps, wc=6520.53K
2018-06-28 03:23:43,271 - root - [Epoch 39 Batch 4000/34064] loss=2.8881, ppl=17.9583, throughput=59.52K wps, wc=6521.31K
2018-06-28 03:25:33,918 - root - [Epoch 39 Batch 4800/34064] loss=2.8861, ppl=17.9241, throughput=58.94K wps, wc=6521.81K
2018-06-28 03:27:25,092 - root - [Epoch 39 Batch 5600/34064] loss=2.9034, ppl=18.2356, throughput=58.59K wps, wc=6514.07K
2018-06-28 03:29:15,939 - root - [Epoch 39 Batch 6400/34064] loss=2.8950, ppl=18.0829, throughput=58.87K wps, wc=6525.11K
2018-06-28 03:31:08,677 - root - [Epoch 39 Batch 7200/34064] loss=2.8876, ppl=17.9510, throughput=57.86K wps, wc=6522.68K
2018-06-28 03:32:59,525 - root - [Epoch 39 Batch 8000/34064] loss=2.8880, ppl=17.9565, throughput=58.82K wps, wc=6520.42K
2018-06-28 03:34:51,123 - root - [Epoch 39 Batch 8800/34064] loss=2.8980, ppl=18.1383, throughput=58.46K wps, wc=6523.49K
2018-06-28 03:36:41,712 - root - [Epoch 39 Batch 9600/34064] loss=2.8856, ppl=17.9137, throughput=58.86K wps, wc=6509.17K
2018-06-28 03:38:31,357 - root - [Epoch 39 Batch 10400/34064] loss=2.8806, ppl=17.8257, throughput=59.50K wps, wc=6523.69K
2018-06-28 03:40:22,979 - root - [Epoch 39 Batch 11200/34064] loss=2.8868, ppl=17.9357, throughput=58.46K wps, wc=6525.60K
2018-06-28 03:42:12,960 - root - [Epoch 39 Batch 12000/34064] loss=2.9018, ppl=18.2061, throughput=59.31K wps, wc=6522.56K
2018-06-28 03:44:04,052 - root - [Epoch 39 Batch 12800/34064] loss=2.8886, ppl=17.9683, throughput=58.72K wps, wc=6523.51K
2018-06-28 03:45:56,819 - root - [Epoch 39 Batch 13600/34064] loss=2.9006, ppl=18.1852, throughput=57.77K wps, wc=6514.61K
2018-06-28 03:47:47,638 - root - [Epoch 39 Batch 14400/34064] loss=2.8889, ppl=17.9730, throughput=58.87K wps, wc=6523.80K
2018-06-28 03:49:36,795 - root - [Epoch 39 Batch 15200/34064] loss=2.8847, ppl=17.8990, throughput=59.80K wps, wc=6527.25K
2018-06-28 03:51:30,261 - root - [Epoch 39 Batch 16000/34064] loss=2.9135, ppl=18.4214, throughput=57.47K wps, wc=6520.68K
2018-06-28 03:53:20,666 - root - [Epoch 39 Batch 16800/34064] loss=2.8983, ppl=18.1426, throughput=59.09K wps, wc=6523.36K
2018-06-28 03:55:11,103 - root - [Epoch 39 Batch 17600/34064] loss=2.9027, ppl=18.2231, throughput=59.05K wps, wc=6521.23K
2018-06-28 03:57:02,963 - root - [Epoch 39 Batch 18400/34064] loss=2.8951, ppl=18.0850, throughput=58.31K wps, wc=6522.30K
2018-06-28 03:58:54,635 - root - [Epoch 39 Batch 19200/34064] loss=2.9036, ppl=18.2397, throughput=58.38K wps, wc=6519.29K
2018-06-28 04:00:45,954 - root - [Epoch 39 Batch 20000/34064] loss=2.8943, ppl=18.0709, throughput=58.59K wps, wc=6522.46K
2018-06-28 04:02:37,175 - root - [Epoch 39 Batch 20800/34064] loss=2.9097, ppl=18.3511, throughput=58.60K wps, wc=6517.13K
2018-06-28 04:04:27,838 - root - [Epoch 39 Batch 21600/34064] loss=2.9003, ppl=18.1792, throughput=58.97K wps, wc=6525.95K
2018-06-28 04:06:18,295 - root - [Epoch 39 Batch 22400/34064] loss=2.8948, ppl=18.0802, throughput=59.07K wps, wc=6524.52K
2018-06-28 04:08:09,585 - root - [Epoch 39 Batch 23200/34064] loss=2.8976, ppl=18.1310, throughput=58.60K wps, wc=6521.15K
2018-06-28 04:09:59,675 - root - [Epoch 39 Batch 24000/34064] loss=2.8970, ppl=18.1193, throughput=59.22K wps, wc=6519.44K
2018-06-28 04:11:51,007 - root - [Epoch 39 Batch 24800/34064] loss=2.9032, ppl=18.2317, throughput=58.57K wps, wc=6520.78K
2018-06-28 04:13:42,316 - root - [Epoch 39 Batch 25600/34064] loss=2.9301, ppl=18.7296, throughput=58.55K wps, wc=6516.70K
2018-06-28 04:15:35,113 - root - [Epoch 39 Batch 26400/34064] loss=2.9166, ppl=18.4781, throughput=57.79K wps, wc=6518.90K
2018-06-28 04:17:25,474 - root - [Epoch 39 Batch 27200/34064] loss=2.9092, ppl=18.3417, throughput=59.06K wps, wc=6517.74K
2018-06-28 04:19:15,872 - root - [Epoch 39 Batch 28000/34064] loss=2.9062, ppl=18.2868, throughput=59.12K wps, wc=6526.64K
2018-06-28 04:21:05,757 - root - [Epoch 39 Batch 28800/34064] loss=2.9136, ppl=18.4229, throughput=59.30K wps, wc=6516.52K
2018-06-28 04:22:56,448 - root - [Epoch 39 Batch 29600/34064] loss=2.9226, ppl=18.5896, throughput=58.91K wps, wc=6521.24K
2018-06-28 04:24:45,070 - root - [Epoch 39 Batch 30400/34064] loss=2.9101, ppl=18.3581, throughput=60.02K wps, wc=6519.03K
2018-06-28 04:26:34,926 - root - [Epoch 39 Batch 31200/34064] loss=2.9062, ppl=18.2874, throughput=59.39K wps, wc=6524.19K
2018-06-28 04:28:23,924 - root - [Epoch 39 Batch 32000/34064] loss=2.9082, ppl=18.3244, throughput=59.86K wps, wc=6524.43K
2018-06-28 04:30:15,017 - root - [Epoch 39 Batch 32800/34064] loss=2.9103, ppl=18.3628, throughput=58.73K wps, wc=6524.73K
2018-06-28 04:32:05,725 - root - [Epoch 39 Batch 33600/34064] loss=2.9043, ppl=18.2530, throughput=58.91K wps, wc=6521.25K
2018-06-28 04:37:50,667 - root - [Epoch 39] valid Loss=1.4915, valid ppl=4.4436, valid bleu=25.68
2018-06-28 04:42:26,672 - root - [Epoch 39] test Loss=1.3113, test ppl=3.7108, test bleu=26.50
2018-06-28 04:47:02,468 - root - Best model valid Loss=1.4724, valid ppl=4.3598, valid bleu=26.06
2018-06-28 04:51:35,667 - root - Best model test Loss=1.2878, test ppl=3.6247, test bleu=26.95