sample_log

nohup: ignoring input
Using TensorFlow backend.
2018-09-25 16:22:09,942 : INFO : Apply doc gating, No dense layer on top, sum up scores
2018-09-25 16:22:09.965225: I tensorflow/core/platform/cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
2018-09-25 16:22:10.645841: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:898] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2018-09-25 16:22:10.646178: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1344] Found device 0 with properties: 
name: GeForce GTX 1060 6GB major: 6 minor: 1 memoryClockRate(GHz): 1.7085
pciBusID: 0000:0b:00.0
totalMemory: 5.93GiB freeMemory: 5.86GiB
2018-09-25 16:22:10.646340: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1423] Adding visible gpu devices: 0
2018-09-25 16:22:10.827968: I tensorflow/core/common_runtime/gpu/gpu_device.cc:911] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-09-25 16:22:10.828008: I tensorflow/core/common_runtime/gpu/gpu_device.cc:917]      0 
2018-09-25 16:22:10.828015: I tensorflow/core/common_runtime/gpu/gpu_device.cc:930] 0:   N 
2018-09-25 16:22:10.828179: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1041] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 5647 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1060 6GB, pci bus id: 0000:0b:00.0, compute capability: 6.1)
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
dd_d_input (InputLayer)         (None, 10, 20, 30)   0                                            
__________________________________________________________________________________________________
dd_q_input (InputLayer)         (None, 10, 20, 1)    0                                            
__________________________________________________________________________________________________
hidden (Dense)                  (None, 10, 20, 5)    155         dd_d_input[0][0]                 
__________________________________________________________________________________________________
dd_q_gate (Dense)               (None, 10, 20, 1)    1           dd_q_input[0][0]                 
__________________________________________________________________________________________________
dd_d_gate (Dense)               (None, 10, 20, 1)    6           hidden[0][0]                     
__________________________________________________________________________________________________
dd_q_softmax (Lambda)           (None, 10, 20)       0           dd_q_gate[0][0]                  
__________________________________________________________________________________________________
reshape_1 (Reshape)             (None, 10, 20)       0           dd_d_gate[0][0]                  
__________________________________________________________________________________________________
reshape_2 (Reshape)             (None, 10, 20)       0           dd_q_softmax[0][0]               
__________________________________________________________________________________________________
dd_pseudo_out (Lambda)          (None, 10, 10)       0           reshape_1[0][0]                  
                                                                 reshape_2[0][0]                  
__________________________________________________________________________________________________
baseline_doc_score (InputLayer) (None, 10, 1)        0                                            
__________________________________________________________________________________________________
dd_init_out (Lambda)            (None, 10)           0           dd_pseudo_out[0][0]              
__________________________________________________________________________________________________
dd_gate (Dense)                 (None, 10, 1)        1           baseline_doc_score[0][0]         
__________________________________________________________________________________________________
reshape_4 (Reshape)             (None, 10)           0           dd_init_out[0][0]                
__________________________________________________________________________________________________
reshape_3 (Reshape)             (None, 10)           0           dd_gate[0][0]                    
__________________________________________________________________________________________________
score (Dot)                     (None, 1)            0           reshape_4[0][0]                  
                                                                 reshape_3[0][0]                  
==================================================================================================
Total params: 163
Trainable params: 163
Non-trainable params: 0
__________________________________________________________________________________________________
None
([56, 57, 71, 73, 77, 88, 93, 100, 102, 103, 104, 109, 115, 121, 123, 124, 128, 134, 135, 138, 149, 152, 157, 160, 169, 177, 178, 183, 194, 198, 61, 63, 67, 68, 78, 79, 90, 101, 106, 110, 113, 119, 120, 127, 129, 136, 141, 154, 159, 161, 172, 173, 175, 176, 181, 186, 187, 193, 195, 197, 51, 58, 60, 70, 82, 84, 85, 86, 87, 89, 91, 92, 94, 96, 98, 105, 107, 111, 117, 133, 137, 146, 156, 158, 163, 166, 174, 179, 190, 196], [52, 54, 62, 64, 66, 72, 74, 76, 80, 832018-09-25 16:22:21,227 : INFO : Generate totally 133630 pairs
2018-09-25 16:22:21.369965: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1423] Adding visible gpu devices: 0
2018-09-25 16:22:21.370028: I tensorflow/core/common_runtime/gpu/gpu_device.cc:911] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-09-25 16:22:21.370043: I tensorflow/core/common_runtime/gpu/gpu_device.cc:917]      0 
2018-09-25 16:22:21.370056: I tensorflow/core/common_runtime/gpu/gpu_device.cc:930] 0:   N 
2018-09-25 16:22:21.370276: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1041] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 5647 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1060 6GB, pci bus id: 0000:0b:00.0, compute capability: 6.1)
, 99, 125, 126, 143, 144, 147, 148, 151, 155, 167, 168, 170, 171, 180, 184, 185, 188, 189, 191, 200], [53, 55, 59, 65, 69, 75, 81, 95, 97, 108, 112, 114, 116, 118, 122, 130, 131, 132, 139, 140, 142, 145, 150, 153, 162, 164, 165, 182, 192, 199])
([52, 54, 62, 64, 66, 72, 74, 76, 80, 83, 99, 125, 126, 143, 144, 147, 148, 151, 155, 167, 168, 170, 171, 180, 184, 185, 188, 189, 191, 200], [53, 55, 59, 65, 69, 75, 81, 95, 97, 108, 112, 114, 116, 118, 122, 130, 131, 132, 139, 140, 142, 145, 150, 153, 162, 164, 165, 182, 192, 199])
Epoch 0
[Iter 0]	Loss: [0.98447722673416138]
[Valid]		MAP	P20	NDCG20
		0.1902	0.3711	0.3508
[Test]		MAP	P20	NDCG20
		0.1829	0.4262	0.4260
[Iter 1]	Loss: [0.78223156213760381]
[Valid]		MAP	P20	NDCG20
		0.2034	0.3789	0.3824
[Test]		MAP	P20	NDCG20
		0.1940	0.4571	0.4526
[Iter 2]	Loss: [0.59759169131517409]
[Valid]		MAP	P20	NDCG20
		0.2107	0.3895	0.3990
[Test]		MAP	P20	NDCG20
		0.2008	0.4738	0.4680
[Iter 3]	Loss: [0.5766389189660549]
[Valid]		MAP	P20	NDCG20
		0.2142	0.4026	0.4132
[Test]		MAP	P20	NDCG20
		0.2050	0.4833	0.4808
[Iter 4]	Loss: [0.57218585148453716]
[Valid]		MAP	P20	NDCG20
		0.2170	0.4211	0.4262
[Test]		MAP	P20	NDCG20
		0.2064	0.4762	0.4778
[Iter 5]	Loss: [0.53590213015675547]
[Valid]		MAP	P20	NDCG20
		0.2193	0.4263	0.4302
[Test]		MAP	P20	NDCG20
		0.2091	0.4786	0.4804
[Iter 6]	Loss: [0.53116292268037801]
[Valid]		MAP	P20	NDCG20
		0.2210	0.4342	0.4359
[Test]		MAP	P20	NDCG20
		0.2102	0.4762	0.4791
[Iter 7]	Loss: [0.53632596276700495]
[Valid]		MAP	P20	NDCG20
		0.2237	0.4316	0.4387
[Test]		MAP	P20	NDCG20
		0.2128	0.4762	0.4806
[Iter 8]	Loss: [0.48075701013207434]
[Valid]		MAP	P20	NDCG20
		0.2245	0.4447	0.4455
[Test]		MAP	P20	NDCG20
		0.2123	0.4714	0.4739
[Iter 9]	Loss: [0.4814444409310818]
[Valid]		MAP	P20	NDCG20
		0.2280	0.4447	0.4483
[Test]		MAP	P20	NDCG20
		0.2163	0.4833	0.4819
[Iter 10]	Loss: [0.5042229290306568]
[Valid]		MAP	P20	NDCG20
		0.2299	0.4421	0.4469
[Test]		MAP	P20	NDCG20
		0.2188	0.4905	0.4874
[Iter 11]	Loss: [0.46541811510920522]
[Valid]		MAP	P20	NDCG20
		0.2315	0.4474	0.4517
[Test]		MAP	P20	NDCG20
		0.2200	0.4857	0.4837
[Iter 12]	Loss: [0.47471186801791193]
[Valid]		MAP	P20	NDCG20
		0.2327	0.4447	0.4496
[Test]		MAP	P20	NDCG20
		0.2204	0.4881	0.4848
[Iter 13]	Loss: [0.44979410007596016]
[Valid]		MAP	P20	NDCG20
		0.2328	0.4421	0.4473
[Test]		MAP	P20	NDCG20
		0.2201	0.4881	0.4838
[Iter 14]	Loss: [0.44492042161524298]
[Valid]		MAP	P20	NDCG20
		0.2341	0.4421	0.4476
[Test]		MAP	P20	NDCG20
		0.2224	0.4857	0.4848
[Iter 15]	Loss: [0.47987494796514513]
[Valid]		MAP	P20	NDCG20
		0.2375	0.4395	0.4471
[Test]		MAP	P20	NDCG20
		0.2273	0.4905	0.4920
[Iter 16]	Loss: [0.43539952956140043]
[Valid]		MAP	P20	NDCG20
		0.2387	0.4526	0.4556
[Test]		MAP	P20	NDCG20
		0.2286	0.5000	0.4993
[Iter 17]	Loss: [0.4391825757920742]
[Valid]		MAP	P20	NDCG20
		0.2420	0.4474	0.4579
[Test]		MAP	P20	NDCG20
		0.2348	0.4952	0.5036
[Iter 18]	Loss: [0.4197976917028427]
[Valid]		MAP	P20	NDCG20
		0.2441	0.4658	0.4727
[Test]		MAP	P20	NDCG20
		0.2366	0.4952	0.5041
[Iter 19]	Loss: [0.43038667365908623]
[Valid]		MAP	P20	NDCG20
		0.2454	0.4711	0.4773
[Test]		MAP	P20	NDCG20
		0.2375	0.5000	0.5064
[Iter 20]	Loss: [0.39294540338218215]
[Valid]		MAP	P20	NDCG20
		0.2433	0.4526	0.4630
[Test]		MAP	P20	NDCG20
		0.2328	0.4786	0.4891
[Iter 21]	Loss: [0.38711365863680841]
[Valid]		MAP	P20	NDCG20
		0.2455	0.4658	0.4736
[Test]		MAP	P20	NDCG20
		0.2373	0.4952	0.5037
[Iter 22]	Loss: [0.41446281671524049]
[Valid]		MAP	P20	NDCG20
		0.2475	0.4684	0.4818
[Test]		MAP	P20	NDCG20
		0.2390	0.5000	0.5145
[Iter 23]	Loss: [0.41357803739607335]
[Valid]		MAP	P20	NDCG20
		0.2474	0.4711	0.4836
[Test]		MAP	P20	NDCG20
		0.2400	0.5071	0.5213
[Iter 24]	Loss: [0.38969642728567122]
[Valid]		MAP	P20	NDCG20
		0.2476	0.4711	0.4843
[Test]		MAP	P20	NDCG20
		0.2409	0.5167	0.5275
[Iter 25]	Loss: [0.38718521066009998]
[Valid]		MAP	P20	NDCG20
		0.2470	0.4632	0.4830
[Test]		MAP	P20	NDCG20
		0.2401	0.5095	0.5207
[Iter 26]	Loss: [0.38582158438861369]
[Valid]		MAP	P20	NDCG20
		0.2470	0.4658	0.4831
[Test]		MAP	P20	NDCG20
		0.2402	0.5071	0.5177
[Iter 27]	Loss: [0.39854203417897227]
[Valid]		MAP	P20	NDCG20
		0.2472	0.4658	0.4846
[Test]		MAP	P20	NDCG20
		0.2403	0.5071	0.5179
[Iter 28]	Loss: [0.40086526021361352]
[Valid]		MAP	P20	NDCG20
		0.2491	0.4737	0.4960
[Test]		MAP	P20	NDCG20
		0.2412	0.5119	0.5229
[Iter 29]	Loss: [0.39314277216792104]
[Valid]		MAP	P20	NDCG20
		0.2491	0.4711	0.4940
[Test]		MAP	P20	NDCG20
		0.2417	0.5143	0.5250
[Iter 30]	Loss: [0.38039236679673194]
[Valid]		MAP	P20	NDCG20
		0.2476	0.4737	0.4967
[Test]		MAP	P20	NDCG20
		0.2400	0.5048	0.5179
[Iter 31]	Loss: [0.39552836820483206]
[Valid]		MAP	P20	NDCG20
		0.2478	0.4711	0.4955
[Test]		MAP	P20	NDCG20
		0.2411	0.5071	0.5206
[Iter 32]	Loss: [0.37727249957621095]
[Valid]		MAP	P20	NDCG20
		0.2490	0.4842	0.5044
[Test]		MAP	P20	NDCG20
		0.2413	0.5262	0.5318
[Iter 33]	Loss: [0.38844930991530419]
[Valid]		MAP	P20	NDCG20
		0.2495	0.4816	0.5035
[Test]		MAP	P20	NDCG20
		0.2423	0.5262	0.5328
[Iter 34]	Loss: [0.3650168614089489]
[Valid]		MAP	P20	NDCG20
		0.2495	0.4816	0.5032
[Test]		MAP	P20	NDCG20
		0.2419	0.5262	0.5324
[Iter 35]	Loss: [0.38589441195130347]
[Valid]		MAP	P20	NDCG20
		0.2492	0.4789	0.5016
[Test]		MAP	P20	NDCG20
		0.2411	0.5286	0.5331
[Iter 36]	Loss: [0.38793338298797608]
[Valid]		MAP	P20	NDCG20
		0.2492	0.4789	0.5026
[Test]		MAP	P20	NDCG20
		0.2413	0.5310	0.5338
[Iter 37]	Loss: [0.37571331135928632]
[Valid]		MAP	P20	NDCG20
		0.2489	0.4763	0.5007
[Test]		MAP	P20	NDCG20
		0.2408	0.5310	0.5339
[Iter 38]	Loss: [0.36206370912492275]
[Valid]		MAP	P20	NDCG20
		0.2487	0.4816	0.5040
[Test]		MAP	P20	NDCG20
		0.2402	0.5238	0.5295
[Iter 39]	Loss: [0.37531969763338568]
[Valid]		MAP	P20	NDCG20
		0.2487	0.4789	0.5020
[Test]		MAP	P20	NDCG20
		0.2404	0.5333	0.5344
[Iter 40]	Loss: [0.37986634373664857]
[Valid]		MAP	P20	NDCG20
		0.2494	0.4789	0.5012
[Test]		MAP	P20	NDCG20
		0.2418	0.5381	0.5367
[Iter 41]	Loss: [0.37793311266228558]
[Valid]		MAP	P20	NDCG20
		0.2485	0.4763	0.5005
[Test]		MAP	P20	NDCG20
		0.2402	0.5405	0.5368
[Iter 42]	Loss: [0.39694193318486215]
[Valid]		MAP	P20	NDCG20
		0.2471	0.4868	0.5086
[Test]		MAP	P20	NDCG20
		0.2388	0.5190	0.5261
[Iter 43]	Loss: [0.36439687713980673]
[Valid]		MAP	P20	NDCG20
		0.2491	0.4816	0.5025
[Test]		MAP	P20	NDCG20
		0.2419	0.5357	0.5376
[Iter 44]	Loss: [0.36377180136740206]
[Valid]		MAP	P20	NDCG20
		0.2492	0.4842	0.5053
[Test]		MAP	P20	NDCG20
		0.2420	0.5452	0.5399
[Iter 45]	Loss: [0.36644888088107108]
[Valid]		MAP	P20	NDCG20
		0.2491	0.4737	0.4982
[Test]		MAP	P20	NDCG20
		0.2426	0.5452	0.5445
[Iter 46]	Loss: [0.38525771144777538]
[Valid]		MAP	P20	NDCG20
		0.2496	0.4789	0.5020
[Test]		MAP	P20	NDCG20
		0.2419	0.5381	0.5367
[Iter 47]	Loss: [0.3765621363557875]
[Valid]		MAP	P20	NDCG20
		0.2495	0.4789	0.5020
[Test]		MAP	P20	NDCG20
		0.2418	0.5381	0.5365
[Iter 48]	Loss: [0.35139374218881131]
[Valid]		MAP	P20	NDCG20
		0.2493	0.4842	0.5060
[Test]		MAP	P20	NDCG20
		0.2427	0.5405	0.5387
[Iter 49]	Loss: [0.39703202307224272]
[Valid]		MAP	P20	NDCG20
		0.2485	0.4842	0.5069
[Test]		MAP	P20	NDCG20
		0.2409	0.5548	0.5486
[Iter 50]	Loss: [0.37038985230028632]
[Valid]		MAP	P20	NDCG20
		0.2488	0.4763	0.5004
[Test]		MAP	P20	NDCG20
		0.2418	0.5357	0.5373
[Iter 51]	Loss: [0.36577040385454895]
[Valid]		MAP	P20	NDCG20
		0.2492	0.4763	0.5002
[Test]		MAP	P20	NDCG20
		0.2419	0.5333	0.5358
[Iter 52]	Loss: [0.38234345756471155]
[Valid]		MAP	P20	NDCG20
		0.2495	0.4737	0.4960
[Test]		MAP	P20	NDCG20
		0.2427	0.5381	0.5383
[Iter 53]	Loss: [0.3503352914750576]
[Valid]		MAP	P20	NDCG20
		0.2494	0.4763	0.4952
[Test]		MAP	P20	NDCG20
		0.2431	0.5333	0.5331
[Iter 54]	Loss: [0.37125005468726158]
[Valid]		MAP	P20	NDCG20
		0.2493	0.4763	0.5004
[Test]		MAP	P20	NDCG20
		0.2421	0.5381	0.5388
[Iter 55]	Loss: [0.36760086655616758]
[Valid]		MAP	P20	NDCG20
		0.2488	0.4737	0.4982
[Test]		MAP	P20	NDCG20
		0.2415	0.5286	0.5320
[Iter 56]	Loss: [0.38662903610616922]
[Valid]		MAP	P20	NDCG20
		0.2486	0.4789	0.5020
[Test]		MAP	P20	NDCG20
		0.2421	0.5381	0.5400
[Iter 57]	Loss: [0.38991102419793605]
[Valid]		MAP	P20	NDCG20
		0.2484	0.4816	0.5045
[Test]		MAP	P20	NDCG20
		0.2418	0.5381	0.5403
[Iter 58]	Loss: [0.38202350502833726]
[Valid]		MAP	P20	NDCG20
		0.2495	0.42018-09-25 16:24:39,239 : INFO : Generate totally 133630 pairs
789	0.5031
[Test]		MAP	P20	NDCG20
		0.2426	0.5381	0.5378
[Iter 59]	Loss: [0.35779909394681453]
[Valid]		MAP	P20	NDCG20
		0.2496	0.4763	0.4976
[Test]		MAP	P20	NDCG20
		0.2433	0.5405	0.5383
[Iter 60]	Loss: [0.37846053026616572]
[Valid]		MAP	P20	NDCG20
		0.2497	0.4737	0.4979
[Test]		MAP	P20	NDCG20
		0.2415	0.5238	0.5264
[Iter 61]	Loss: [0.38958924382925031]
[Valid]		MAP	P20	NDCG20
		0.2473	0.4711	0.4976
[Test]		MAP	P20	NDCG20
		0.2392	0.5238	0.5283
[Iter 62]	Loss: [0.33926617581397295]
[Valid]		MAP	P20	NDCG20
		0.2489	0.4684	0.4956
[Test]		MAP	P20	NDCG20
		0.2410	0.5381	0.5373
[Iter 63]	Loss: [0.35827168948948385]
[Valid]		MAP	P20	NDCG20
		0.2492	0.4737	0.4990
[Test]		MAP	P20	NDCG20
		0.2422	0.5381	0.5383
[Iter 64]	Loss: [0.3399954340234399]
[Valid]		MAP	P20	NDCG20
		0.2490	0.4737	0.4987
[Test]		MAP	P20	NDCG20
		0.2419	0.5310	0.5347
[Iter 65]	Loss: [0.38278023302555086]
[Valid]		MAP	P20	NDCG20
		0.2489	0.4737	0.4992
[Test]		MAP	P20	NDCG20
		0.2428	0.5405	0.5399
[Attention]		Current best iteration 60

Epoch 1
[Iter 66]	Loss: [0.35772266495972871]
[Valid]		MAP	P20	NDCG20
		0.2491	0.4711	0.4977
[Test]		MAP	P20	NDCG20
		0.2421	0.5381	0.5390
[Iter 67]	Loss: [0.36714359752833842]
[Valid]		MAP	P20	NDCG20
		0.2495	0.4737	0.4957
[Test]		MAP	P20	NDCG20
		0.2424	0.5405	0.5404
[Iter 68]	Loss: [0.34829167433083058]
[Valid]		MAP	P20	NDCG20
		0.2495	0.4737	0.4989
[Test]		MAP	P20	NDCG20
		0.2420	0.5357	0.5371
[Iter 69]	Loss: [0.37515063017606737]
[Valid]		MAP	P20	NDCG20
		0.2496	0.4737	0.4987
[Test]		MAP	P20	NDCG20
		0.2422	0.5310	0.5337
[Iter 70]	Loss: [0.35553579945117236]
[Valid]		MAP	P20	NDCG20
		0.2488	0.4711	0.4974
[Test]		MAP	P20	NDCG20
		0.2410	0.5286	0.5320
[Iter 71]	Loss: [0.35356088861823082]
[Valid]		MAP	P20	NDCG20
		0.2489	0.4737	0.4955
[Test]		MAP	P20	NDCG20
		0.2416	0.5500	0.5440
[Iter 72]	Loss: [0.38509063161909579]
[Valid]		MAP	P20	NDCG20
		0.2481	0.4816	0.5043
[Test]		MAP	P20	NDCG20
		0.2399	0.5405	0.5400
[Iter 73]	Loss: [0.38955817911773921]
[Valid]		MAP	P20	NDCG20
		0.2496	0.4711	0.4940
[Test]		MAP	P20	NDCG20
		0.2430	0.5381	0.5396
[Iter 74]	Loss: [0.3662502697110176]
[Valid]		MAP	P20	NDCG20
		0.2493	0.4763	0.5008
[Test]		MAP	P20	NDCG20
		0.2417	0.5310	0.5340
[Iter 75]	Loss: [0.37347908534109592]
[Valid]		MAP	P20	NDCG20
		0.2496	0.4737	0.4969
[Test]		MAP	P20	NDCG20
		0.2426	0.5333	0.5365
[Iter 76]	Loss: [0.35255632054060698]
[Valid]		MAP	P20	NDCG20
		0.2496	0.4763	0.5011
[Test]		MAP	P20	NDCG20
		0.2420	0.5310	0.5372
[Iter 77]	Loss: [0.39103055335581305]
[Valid]		MAP	P20	NDCG20
		0.2505	0.4763	0.4986
[Test]		MAP	P20	NDCG20
		0.2436	0.5381	0.5389
[Iter 78]	Loss: [0.35681090444326402]
[Valid]		MAP	P20	NDCG20
		0.2485	0.4763	0.5018
[Test]		MAP	P20	NDCG20
		0.2402	0.5333	0.5362
[Iter 79]	Loss: [0.38434478081762791]
[Valid]		MAP	P20	NDCG20
		0.2500	0.4711	0.4949
[Test]		MAP	P20	NDCG20
		0.2434	0.5381	0.5394
[Iter 80]	Loss: [0.39672202564775944]
[Valid]		MAP	P20	NDCG20
		0.2499	0.4763	0.5023
[Test]		MAP	P20	NDCG20
		0.2428	0.5405	0.5416
[Iter 81]	Loss: [0.36139570225030182]
[Valid]		MAP	P20	NDCG20
		0.2496	0.4684	0.4923
[Test]		MAP	P20	NDCG20
		0.2433	0.5405	0.5408
[Iter 82]	Loss: [0.33171778723597528]
[Valid]		MAP	P20	NDCG20
		0.2497	0.4737	0.4966
[Test]		MAP	P20	NDCG20
		0.2442	0.5405	0.5444
[Iter 83]	Loss: [0.34551556468009947]
[Valid]		MAP	P20	NDCG20
		0.2499	0.4763	0.4985
[Test]		MAP	P20	NDCG20
		0.2448	0.5357	0.5412
[Iter 84]	Loss: [0.37382151339203118]
[Valid]		MAP	P20	NDCG20
		0.2503	0.4789	0.5032
[Test]		MAP	P20	NDCG20
		0.2429	0.5310	0.5373
[Iter 85]	Loss: [0.34230274528264998]
[Valid]		MAP	P20	NDCG20
		0.2493	0.4789	0.4992
[Test]		MAP	P20	NDCG20
		0.2430	0.5524	0.5507
[Iter 86]	Loss: [0.34910697270184754]
[Valid]		MAP	P20	NDCG20
		0.2499	0.4816	0.5049
[Test]		MAP	P20	NDCG20
		0.2432	0.5333	0.5396
[Iter 87]	Loss: [0.36351404763758183]
[Valid]		MAP	P20	NDCG20
		0.2499	0.4763	0.5020
[Test]		MAP	P20	NDCG20
		0.2423	0.5310	0.5355
[Iter 88]	Loss: [0.34622375227510926]
[Valid]		MAP	P20	NDCG20
		0.2504	0.4816	0.5054
[Test]		MAP	P20	NDCG20
		0.2425	0.5286	0.5336
[Iter 89]	Loss: [0.39032128028571605]
[Valid]		MAP	P20	NDCG20
		0.2500	0.4842	0.5013
[Test]		MAP	P20	NDCG20
		0.2432	0.5405	0.5389
[Iter 90]	Loss: [0.36300013825297356]
[Valid]		MAP	P20	NDCG20
		0.2488	0.4763	0.4958
[Test]		MAP	P20	NDCG20
		0.2423	0.5476	0.5482
[Iter 91]	Loss: [0.38615851633250714]
[Valid]		MAP	P20	NDCG20
		0.2486	0.4763	0.4957
[Test]		MAP	P20	NDCG20
		0.2423	0.5405	0.5445
[Iter 92]	Loss: [0.36121317923069002]
[Valid]		MAP	P20	NDCG20
		0.2489	0.4763	0.4979
[Test]		MAP	P20	NDCG20
		0.2421	0.5429	0.5451
[Iter 93]	Loss: [0.36131135098636152]
[Valid]		MAP	P20	NDCG20
		0.2505	0.4763	0.4989
[Test]		MAP	P20	NDCG20
		0.2430	0.5357	0.5371
[Iter 94]	Loss: [0.36195876352488993]
[Valid]		MAP	P20	NDCG20
		0.2506	0.4763	0.4992
[Test]		MAP	P20	NDCG20
		0.2430	0.5381	0.5397
[Iter 95]	Loss: [0.36300706934183835]
[Valid]		MAP	P20	NDCG20
		0.2505	0.4789	0.5004
[Test]		MAP	P20	NDCG20
		0.2419	0.5286	0.5315
[Iter 96]	Loss: [0.35257035419344901]
[Valid]		MAP	P20	NDCG20
		0.2500	0.4789	0.5037
[Test]		MAP	P20	NDCG20
		0.2421	0.5310	0.5370
[Iter 97]	Loss: [0.37273533836007117]
[Valid]		MAP	P20	NDCG20
		0.2507	0.4737	0.4974
[Test]		MAP	P20	NDCG20
		0.2434	0.5381	0.5422
[Iter 98]	Loss: [0.36837248146533969]
[Valid]		MAP	P20	NDCG20
		0.2508	0.4763	0.5021
[Test]		MAP	P20	NDCG20
		0.2414	0.5381	0.5389
[Iter 99]	Loss: [0.35809492878615856]
[Valid]		MAP	P20	NDCG20
		0.2502	0.4816	0.5061
[Test]		MAP	P20	NDCG20
		0.2422	0.5381	0.5426
[Iter 100]	Loss: [0.37335975799709559]
[Valid]		MAP	P20	NDCG20
		0.2510	0.4789	0.5041
[Test]		MAP	P20	NDCG20
		0.2427	0.5262	0.5336
[Iter 101]	Loss: [0.35018167436122893]
[Valid]		MAP	P20	NDCG20
		0.2517	0.4816	0.5013
[Test]		MAP	P20	NDCG20
		0.2442	0.5262	0.5332
[Iter 102]	Loss: [0.36144441403448579]
[Valid]		MAP	P20	NDCG20
		0.2507	0.4763	0.5019
[Test]		MAP	P20	NDCG20
		0.2417	0.5238	0.5303
[Iter 103]	Loss: [0.3713213508948684]
[Valid]		MAP	P20	NDCG20
		0.2501	0.4737	0.5005
[Test]		MAP	P20	NDCG20
		0.2406	0.5262	0.5316
[Iter 104]	Loss: [0.36143073953688143]
[Valid]		MAP	P20	NDCG20
		0.2516	0.4763	0.4992
[Test]		MAP	P20	NDCG20
		0.2433	0.5286	0.5355
[Iter 105]	Loss: [0.36630870327353476]
[Valid]		MAP	P20	NDCG20
		0.2503	0.4816	0.5032
[Test]		MAP	P20	NDCG20
		0.2430	0.5381	0.5454
[Iter 106]	Loss: [0.35327644724398849]
[Valid]		MAP	P20	NDCG20
		0.2516	0.4763	0.4997
[Test]		MAP	P20	NDCG20
		0.2434	0.5357	0.5408
[Iter 107]	Loss: [0.35394058937206863]
[Valid]		MAP	P20	NDCG20
		0.2509	0.4816	0.5060
[Test]		MAP	P20	NDCG20
		0.2413	0.5286	0.5353
[Iter 108]	Loss: [0.35478294298052787]
[Valid]		MAP	P20	NDCG20
		0.2513	0.4763	0.4992
[Test]		MAP	P20	NDCG20
		0.2439	0.5381	0.5429
[Iter 109]	Loss: [0.36536650434136392]
[Valid]		MAP	P20	NDCG20
		0.2518	0.4763	0.4993
[Test]		MAP	P20	NDCG20
		0.2434	0.5262	0.5353
[Iter 110]	Loss: [0.35344005674123763]
[Valid]		MAP	P20	NDCG20
		0.2503	0.4789	0.5042
[Test]		MAP	P20	NDCG20
		0.2419	0.5238	0.5346
[Iter 111]	Loss: [0.36905296642333268]
[Valid]		MAP	P20	NDCG20
		0.2505	0.4842	0.5075
[Test]		MAP	P20	NDCG20
		0.2418	0.5286	0.5356
[Iter 112]	Loss: [0.3951778610050678]
[Valid]		MAP	P20	NDCG20
		0.2508	0.4763	0.4997
[Test]		MAP	P20	NDCG20
		0.2433	0.5333	0.5409
[Iter 113]	Loss: [0.31913099907338621]
[Valid]		MAP	P20	NDCG20
		0.2515	0.4789	0.5037
[Test]		MAP	P20	NDCG20
		0.2427	0.5310	0.5363
[Iter 114]	Loss: [0.35815389290452004]
[Valid]		MAP	P20	NDCG20
		0.2516	0.4789	0.5041
[Test]		MAP	P20	NDCG20
		0.2420	0.5286	0.5339
[Iter 115]	Loss: [0.37229392819106577]
[Valid]		MAP	P20	NDCG20
		0.2510	0.4763	0.4992
[Test]		MAP	P20	NDCG20
		0.2438	0.5405	0.5418
[Iter 116]	Loss: [0.36584829151630399]
[Valid]		MAP	P20	NDCG20
		0.2506	0.4789	0.5016
[Test]		MAP	P20	NDCG20
		0.2432	0.5381	0.5452
[Iter 117]	Loss: [0.3719016535580158]
[Valid]		MAP	P20	NDCG20
		0.2495	0.4789	0.5042
[Test]		MAP	P20	NDCG20
		0.2410	0.5310	0.5388
[Iter 118]	Loss: [0.36056521587073803]
[Valid]		MAP	P20	NDCG20
		0.2512	0.4816	0.5056
[Test]		MAP	P20	NDCG20
		0.2427	0.5310	0.5361
[Iter 119]	Loss: [0.36621245108544825]
[Valid]		MAP	P20	NDCG20
		0.2508	0.4842	0.5075
[Test]		MAP	P20	NDCG20
		0.2423	0.5381	0.5433
[Iter 120]	Loss: [0.34211590483784676]
[Valid]		MAP	P20	NDCG20
		0.2512	0.4789	0.2018-09-25 16:26:56,666 : INFO : Generate totally 133630 pairs
5039
[Test]		MAP	P20	NDCG20
		0.2430	0.5238	0.5320
[Iter 121]	Loss: [0.37793557167053221]
[Valid]		MAP	P20	NDCG20
		0.2502	0.4763	0.5027
[Test]		MAP	P20	NDCG20
		0.2418	0.5238	0.5338
[Iter 122]	Loss: [0.35473058924078943]
[Valid]		MAP	P20	NDCG20
		0.2501	0.4842	0.5071
[Test]		MAP	P20	NDCG20
		0.2419	0.5333	0.5353
[Iter 123]	Loss: [0.35706973090767863]
[Valid]		MAP	P20	NDCG20
		0.2510	0.4737	0.4974
[Test]		MAP	P20	NDCG20
		0.2430	0.5310	0.5346
[Iter 124]	Loss: [0.36123026594519614]
[Valid]		MAP	P20	NDCG20
		0.2500	0.4816	0.5032
[Test]		MAP	P20	NDCG20
		0.2427	0.5357	0.5426
[Iter 125]	Loss: [0.35800644829869271]
[Valid]		MAP	P20	NDCG20
		0.2510	0.4816	0.5032
[Test]		MAP	P20	NDCG20
		0.2431	0.5310	0.5363
[Iter 126]	Loss: [0.35089013068005442]
[Valid]		MAP	P20	NDCG20
		0.2513	0.4763	0.5024
[Test]		MAP	P20	NDCG20
		0.2427	0.5310	0.5362
[Iter 127]	Loss: [0.35454844065010549]
[Valid]		MAP	P20	NDCG20
		0.2505	0.4842	0.5052
[Test]		MAP	P20	NDCG20
		0.2424	0.5262	0.5339
[Iter 128]	Loss: [0.34645816467702389]
[Valid]		MAP	P20	NDCG20
		0.2514	0.4763	0.5002
[Test]		MAP	P20	NDCG20
		0.2424	0.5310	0.5326
[Iter 129]	Loss: [0.36956984549760818]
[Valid]		MAP	P20	NDCG20
		0.2500	0.4763	0.5028
[Test]		MAP	P20	NDCG20
		0.2414	0.5262	0.5352
[Iter 130]	Loss: [0.35549493998289106]
[Valid]		MAP	P20	NDCG20
		0.2517	0.4763	0.5022
[Test]		MAP	P20	NDCG20
		0.2418	0.5262	0.5296
[Iter 131]	Loss: [0.3588053996115923]
[Valid]		MAP	P20	NDCG20
		0.2503	0.4789	0.5045
[Test]		MAP	P20	NDCG20
		0.2409	0.5310	0.5346
[Attention]		Current best iteration 109

Epoch 2
[Iter 132]	Loss: [0.35978879176080225]
[Valid]		MAP	P20	NDCG20
		0.2515	0.4789	0.5047
[Test]		MAP	P20	NDCG20
		0.2418	0.5286	0.5317
[Iter 133]	Loss: [0.33485786026343706]
[Valid]		MAP	P20	NDCG20
		0.2524	0.4763	0.5000
[Test]		MAP	P20	NDCG20
		0.2431	0.5286	0.5313
[Iter 134]	Loss: [0.34446905039250852]
[Valid]		MAP	P20	NDCG20
		0.2510	0.4763	0.5029
[Test]		MAP	P20	NDCG20
		0.2424	0.5190	0.5324
[Iter 135]	Loss: [0.3540548464655876]
[Valid]		MAP	P20	NDCG20
		0.2508	0.4789	0.5048
[Test]		MAP	P20	NDCG20
		0.2428	0.5286	0.5401
[Iter 136]	Loss: [0.38574750773608685]
[Valid]		MAP	P20	NDCG20
		0.2505	0.4816	0.5068
[Test]		MAP	P20	NDCG20
		0.2422	0.5262	0.5390
[Iter 137]	Loss: [0.36099629521369936]
[Valid]		MAP	P20	NDCG20
		0.2511	0.4763	0.5024
[Test]		MAP	P20	NDCG20
		0.2414	0.5190	0.5268
[Iter 138]	Loss: [0.38433399721980094]
[Valid]		MAP	P20	NDCG20
		0.2511	0.4763	0.5030
[Test]		MAP	P20	NDCG20
		0.2415	0.5262	0.5344
[Iter 139]	Loss: [0.35867050938308237]
[Valid]		MAP	P20	NDCG20
		0.2511	0.4789	0.5001
[Test]		MAP	P20	NDCG20
		0.2433	0.5333	0.5375
[Iter 140]	Loss: [0.34360571339726448]
[Valid]		MAP	P20	NDCG20
		0.2517	0.4763	0.4995
[Test]		MAP	P20	NDCG20
		0.2436	0.5286	0.5349
[Iter 141]	Loss: [0.35050936341285704]
[Valid]		MAP	P20	NDCG20
		0.2504	0.4789	0.5041
[Test]		MAP	P20	NDCG20
		0.2419	0.5238	0.5331
[Iter 142]	Loss: [0.36580208152532578]
[Valid]		MAP	P20	NDCG20
		0.2517	0.4737	0.5014
[Test]		MAP	P20	NDCG20
		0.2424	0.5238	0.5332
[Iter 143]	Loss: [0.35702087309211494]
[Valid]		MAP	P20	NDCG20
		0.2520	0.4763	0.4993
[Test]		MAP	P20	NDCG20
		0.2429	0.5310	0.5338
[Iter 144]	Loss: [0.36543836388736961]
[Valid]		MAP	P20	NDCG20
		0.2484	0.4842	0.5077
[Test]		MAP	P20	NDCG20
		0.2400	0.5286	0.5378
[Iter 145]	Loss: [0.3563807824999094]
[Valid]		MAP	P20	NDCG20
		0.2509	0.4737	0.5014
[Test]		MAP	P20	NDCG20
		0.2417	0.5214	0.5299
[Iter 146]	Loss: [0.36303536009043458]
[Valid]		MAP	P20	NDCG20
		0.2517	0.4763	0.5002
[Test]		MAP	P20	NDCG20
		0.2431	0.5333	0.5371
[Iter 147]	Loss: [0.35494564116001132]
[Valid]		MAP	P20	NDCG20
		0.2514	0.4789	0.5011
[Test]		MAP	P20	NDCG20
		0.2430	0.5286	0.5344
[Iter 148]	Loss: [0.34347371125593784]
[Valid]		MAP	P20	NDCG20
		0.2522	0.4737	0.4982
[Test]		MAP	P20	NDCG20
		0.2439	0.5310	0.5357
[Iter 149]	Loss: [0.35385761316865683]
[Valid]		MAP	P20	NDCG20
		0.2516	0.4816	0.5030
[Test]		MAP	P20	NDCG20
		0.2439	0.5310	0.5369
[Iter 150]	Loss: [0.37922018416225911]
[Valid]		MAP	P20	NDCG20
		0.2505	0.4816	0.5061
[Test]		MAP	P20	NDCG20
		0.2416	0.5262	0.5332
[Iter 151]	Loss: [0.34550945468246935]
[Valid]		MAP	P20	NDCG20
		0.2508	0.4737	0.5013
[Test]		MAP	P20	NDCG20
		0.2412	0.5190	0.5262
[Iter 152]	Loss: [0.37426963940262792]
[Valid]		MAP	P20	NDCG20
		0.2517	0.4711	0.4992
[Test]		MAP	P20	NDCG20
		0.2423	0.5238	0.5289
[Iter 153]	Loss: [0.34985111847519873]
[Valid]		MAP	P20	NDCG20
		0.2504	0.4868	0.5098
[Test]		MAP	P20	NDCG20
		0.2411	0.5333	0.5355
[Iter 154]	Loss: [0.36469493076205256]
[Valid]		MAP	P20	NDCG20
		0.2519	0.4763	0.4994
[Test]		MAP	P20	NDCG20
		0.2440	0.5357	0.5384
[Iter 155]	Loss: [0.35879890333861114]
[Valid]		MAP	P20	NDCG20
		0.2511	0.4763	0.5031
[Test]		MAP	P20	NDCG20
		0.2429	0.5214	0.5306
[Iter 156]	Loss: [0.36807915087789295]
[Valid]		MAP	P20	NDCG20
		0.2511	0.4789	0.5017
[Test]		MAP	P20	NDCG20
		0.2432	0.5286	0.5320
[Iter 157]	Loss: [0.33399821525439621]
[Valid]		MAP	P20	NDCG20
		0.2521	0.4816	0.5020
[Test]		MAP	P20	NDCG20
		0.2440	0.5405	0.5388
[Iter 158]	Loss: [0.35736838601529597]
[Valid]		MAP	P20	NDCG20
		0.2521	0.4711	0.4965
[Test]		MAP	P20	NDCG20
		0.2436	0.5333	0.5345
[Iter 159]	Loss: [0.35316068381071092]
[Valid]		MAP	P20	NDCG20
		0.2493	0.4842	0.5079
[Test]		MAP	P20	NDCG20
		0.2418	0.5286	0.5392
[Iter 160]	Loss: [0.35641022127121685]
[Valid]		MAP	P20	NDCG20
		0.2514	0.4789	0.5048
[Test]		MAP	P20	NDCG20
		0.2428	0.5333	0.5370
[Iter 161]	Loss: [0.35666180662810804]
[Valid]		MAP	P20	NDCG20
		0.2507	0.4816	0.5028
[Test]		MAP	P20	NDCG20
		0.2434	0.5310	0.5365
[Iter 162]	Loss: [0.36095232322812082]
[Valid]		MAP	P20	NDCG20
		0.2497	0.4816	0.5064
[Test]		MAP	P20	NDCG20
		0.2416	0.5357	0.5392
[Iter 163]	Loss: [0.36107544124126434]
[Valid]		MAP	P20	NDCG20
		0.2506	0.4816	0.5064
[Test]		MAP	P20	NDCG20
		0.2426	0.5405	0.5420
[Iter 164]	Loss: [0.35585510969161988]
[Valid]		MAP	P20	NDCG20
		0.2504	0.4842	0.5083
[Test]		MAP	P20	NDCG20
		0.2421	0.5357	0.5394
[Iter 165]	Loss: [0.36929513726383445]
[Valid]		MAP	P20	NDCG20
		0.2504	0.4868	0.5063
[Test]		MAP	P20	NDCG20
		0.2430	0.5381	0.5410
[Iter 166]	Loss: [0.35842060472816228]
[Valid]		MAP	P20	NDCG20
		0.2516	0.4737	0.5001
[Test]		MAP	P20	NDCG20
		0.2421	0.5214	0.5258
[Iter 167]	Loss: [0.38008167617022992]
[Valid]		MAP	P20	NDCG20
		0.2520	0.4737	0.5007
[Test]		MAP	P20	NDCG20
		0.2425	0.5214	0.5261
[Iter 168]	Loss: [0.36456909410655497]
[Valid]		MAP	P20	NDCG20
		0.2513	0.4763	0.5033
[Test]		MAP	P20	NDCG20
		0.2419	0.5238	0.5292
[Iter 169]	Loss: [0.34710526958107946]
[Valid]		MAP	P20	NDCG20
		0.2521	0.4737	0.4960
[Test]		MAP	P20	NDCG20
		0.2442	0.5333	0.5361
[Iter 170]	Loss: [0.34239069381728771]
[Valid]		MAP	P20	NDCG20
		0.2520	0.4763	0.5007
[Test]		MAP	P20	NDCG20
		0.2434	0.5214	0.5297
[Iter 171]	Loss: [0.34015699885785577]
[Valid]		MAP	P20	NDCG20
		0.2506	0.4789	0.5004
[Test]		MAP	P20	NDCG20
		0.2438	0.5357	0.5400
[Iter 172]	Loss: [0.3380613783746958]
[Valid]		MAP	P20	NDCG20
		0.2516	0.4763	0.4990
[Test]		MAP	P20	NDCG20
		0.2443	0.5333	0.5379
[Iter 173]	Loss: [0.35752316262573003]
[Valid]		MAP	P20	NDCG20
		0.2517	0.4763	0.5001
[Test]		MAP	P20	NDCG20
		0.2437	0.5286	0.5341
[Iter 174]	Loss: [0.36401632811874152]
[Valid]		MAP	P20	NDCG20
		0.2511	0.4868	0.5066
[Test]		MAP	P20	NDCG20
		0.2436	0.5381	0.5417
[Iter 175]	Loss: [0.34609276734292505]
[Valid]		MAP	P20	NDCG20
		0.2520	0.4763	0.4998
[Test]		MAP	P20	NDCG20
		0.2438	0.5310	0.5351
[Iter 176]	Loss: [0.33481670528650281]
[Valid]		MAP	P20	NDCG20
		0.2514	0.4737	0.4984
[Test]		MAP	P20	NDCG20
		0.2426	0.5262	0.5330
[Iter 177]	Loss: [0.37121778033673763]
[Valid]		MAP	P20	NDCG20
		0.2509	0.4895	0.5086
[Test]		MAP	P20	NDCG20
		0.2421	0.5286	0.5327
[Iter 178]	Loss: [0.36087185896933077]
[Valid]		MAP	P20	NDCG20
		0.2513	0.4842	0.5088
[Test]		MAP	P20	NDCG20
		0.2411	0.5286	0.5322
[Iter 179]	Loss: [0.33662224054336548]
[Valid]		MAP	P20	NDCG20
		0.2518	0.4737	0.4982
[Test]		MAP	P20	NDCG20
		0.2425	0.5310	0.5325
[Iter 180]	Loss: [0.36329417388886215]
[Valid]		MAP	P20	NDCG20
		0.2520	0.4763	0.5005
[Test]		MAP	P20	NDCG20
		0.2430	0.5214	0.5296
[Iter 181]	Loss: [0.33492658354341986]
[Valid]		MAP	P20	NDCG20
		0.2517	0.4763	0.5031
[Test]		MAP	P20	NDCG20
		0.2421	0.5262	0.5306
[Iter 182]	Loss: [0.35982443753629922]
[Vali2018-09-25 16:29:13,892 : INFO : Generate totally 133630 pairs
d]		MAP	P20	NDCG20
		0.2502	0.4842	0.5056
[Test]		MAP	P20	NDCG20
		0.2418	0.5262	0.5357
[Iter 183]	Loss: [0.35035861369222404]
[Valid]		MAP	P20	NDCG20
		0.2511	0.4816	0.5038
[Test]		MAP	P20	NDCG20
		0.2426	0.5310	0.5354
[Iter 184]	Loss: [0.37552345700562001]
[Valid]		MAP	P20	NDCG20
		0.2505	0.4763	0.5036
[Test]		MAP	P20	NDCG20
		0.2398	0.5214	0.5269
[Iter 185]	Loss: [0.34296812057495119]
[Valid]		MAP	P20	NDCG20
		0.2514	0.4789	0.5015
[Test]		MAP	P20	NDCG20
		0.2425	0.5262	0.5322
[Iter 186]	Loss: [0.37661228604614733]
[Valid]		MAP	P20	NDCG20
		0.2500	0.4842	0.5048
[Test]		MAP	P20	NDCG20
		0.2407	0.5262	0.5310
[Iter 187]	Loss: [0.32842390775680541]
[Valid]		MAP	P20	NDCG20
		0.2514	0.4789	0.5005
[Test]		MAP	P20	NDCG20
		0.2432	0.5238	0.5309
[Iter 188]	Loss: [0.37934098884463308]
[Valid]		MAP	P20	NDCG20
		0.2503	0.4895	0.5119
[Test]		MAP	P20	NDCG20
		0.2409	0.5262	0.5350
[Iter 189]	Loss: [0.34202339361654593]
[Valid]		MAP	P20	NDCG20
		0.2512	0.4763	0.5039
[Test]		MAP	P20	NDCG20
		0.2411	0.5238	0.5303
[Iter 190]	Loss: [0.35128993891179561]
[Valid]		MAP	P20	NDCG20
		0.2518	0.4763	0.5010
[Test]		MAP	P20	NDCG20
		0.2429	0.5286	0.5352
[Iter 191]	Loss: [0.33192466542124749]
[Valid]		MAP	P20	NDCG20
		0.2496	0.4842	0.5051
[Test]		MAP	P20	NDCG20
		0.2420	0.5357	0.5409
[Iter 192]	Loss: [0.37712468750774858]
[Valid]		MAP	P20	NDCG20
		0.2517	0.4763	0.5039
[Test]		MAP	P20	NDCG20
		0.2419	0.5262	0.5317
[Iter 193]	Loss: [0.3345466563105583]
[Valid]		MAP	P20	NDCG20
		0.2517	0.4789	0.4999
[Test]		MAP	P20	NDCG20
		0.2440	0.5286	0.5331
[Iter 194]	Loss: [0.34803494773805144]
[Valid]		MAP	P20	NDCG20
		0.2518	0.4789	0.5022
[Test]		MAP	P20	NDCG20
		0.2435	0.5286	0.5334
[Iter 195]	Loss: [0.36993773192167284]
[Valid]		MAP	P20	NDCG20
		0.2521	0.4763	0.4990
[Test]		MAP	P20	NDCG20
		0.2435	0.5262	0.5314
[Iter 196]	Loss: [0.32615167684853075]
[Valid]		MAP	P20	NDCG20
		0.2516	0.4816	0.5070
[Test]		MAP	P20	NDCG20
		0.2427	0.5238	0.5305
[Iter 197]	Loss: [0.36445805259048941]
[Valid]		MAP	P20	NDCG20
		0.2512	0.4816	0.5078
[Test]		MAP	P20	NDCG20
		0.2414	0.5286	0.5321
[Attention]		Current best iteration 133

Epoch 3
[Iter 198]	Loss: [0.35551691029220822]
[Valid]		MAP	P20	NDCG20
		0.2512	0.4895	0.5088
[Test]		MAP	P20	NDCG20
		0.2440	0.5310	0.5373
[Iter 199]	Loss: [0.35798561166971921]
[Valid]		MAP	P20	NDCG20
		0.2516	0.4895	0.5096
[Test]		MAP	P20	NDCG20
		0.2428	0.5333	0.5369
[Iter 200]	Loss: [0.32982564255595209]
[Valid]		MAP	P20	NDCG20
		0.2517	0.4816	0.5045
[Test]		MAP	P20	NDCG20
		0.2431	0.5286	0.5351
[Iter 201]	Loss: [0.36252812147140501]
[Valid]		MAP	P20	NDCG20
		0.2518	0.4789	0.5058
[Test]		MAP	P20	NDCG20
		0.2420	0.5286	0.5321
[Iter 202]	Loss: [0.36359374374151232]
[Valid]		MAP	P20	NDCG20
		0.2517	0.4842	0.5055
[Test]		MAP	P20	NDCG20
		0.2432	0.5286	0.5344
[Iter 203]	Loss: [0.37445247910916807]
[Valid]		MAP	P20	NDCG20
		0.2504	0.4842	0.5058
[Test]		MAP	P20	NDCG20
		0.2421	0.5262	0.5319
[Iter 204]	Loss: [0.34860934529453513]
[Valid]		MAP	P20	NDCG20
		0.2527	0.4763	0.4981
[Test]		MAP	P20	NDCG20
		0.2433	0.5262	0.5316
[Iter 205]	Loss: [0.36083263784646991]
[Valid]		MAP	P20	NDCG20
		0.2516	0.4816	0.5035
[Test]		MAP	P20	NDCG20
		0.2423	0.5238	0.5289
[Iter 206]	Loss: [0.34849751241505145]
[Valid]		MAP	P20	NDCG20
		0.2505	0.4842	0.5084
[Test]		MAP	P20	NDCG20
		0.2416	0.5238	0.5311
[Iter 207]	Loss: [0.34510294690728188]
[Valid]		MAP	P20	NDCG20
		0.2515	0.4842	0.5057
[Test]		MAP	P20	NDCG20
		0.2428	0.5286	0.5323
[Iter 208]	Loss: [0.37826380837708712]
[Valid]		MAP	P20	NDCG20
		0.2500	0.4789	0.5057
[Test]		MAP	P20	NDCG20
		0.2399	0.5286	0.5327
[Iter 209]	Loss: [0.34974762827157974]
[Valid]		MAP	P20	NDCG20
		0.2521	0.4763	0.5004
[Test]		MAP	P20	NDCG20
		0.2425	0.5262	0.5300
[Iter 210]	Loss: [0.33317090781405567]
[Valid]		MAP	P20	NDCG20
		0.2515	0.4842	0.5087
[Test]		MAP	P20	NDCG20
		0.2409	0.5214	0.5257
[Iter 211]	Loss: [0.35864415593445303]
[Valid]		MAP	P20	NDCG20
		0.2505	0.4842	0.5088
[Test]		MAP	P20	NDCG20
		0.2412	0.5238	0.5293
[Iter 212]	Loss: [0.34400460973381997]
[Valid]		MAP	P20	NDCG20
		0.2520	0.4737	0.4966
[Test]		MAP	P20	NDCG20
		0.2431	0.5214	0.5253
[Iter 213]	Loss: [0.35140760220587253]
[Valid]		MAP	P20	NDCG20
		0.2518	0.4763	0.5006
[Test]		MAP	P20	NDCG20
		0.2419	0.5262	0.5294
[Iter 214]	Loss: [0.34549729011952879]
[Valid]		MAP	P20	NDCG20
		0.2522	0.4737	0.4986
[Test]		MAP	P20	NDCG20
		0.2431	0.5357	0.5366
[Iter 215]	Loss: [0.37469746656715869]
[Valid]		MAP	P20	NDCG20
		0.2513	0.4895	0.5081
[Test]		MAP	P20	NDCG20
		0.2432	0.5333	0.5368
[Iter 216]	Loss: [0.34750524960458279]
[Valid]		MAP	P20	NDCG20
		0.2510	0.4868	0.5104
[Test]		MAP	P20	NDCG20
		0.2408	0.5262	0.5288
[Iter 217]	Loss: [0.36163011349737645]
[Valid]		MAP	P20	NDCG20
		0.2500	0.4842	0.5052
[Test]		MAP	P20	NDCG20
		0.2408	0.5262	0.5324
[Iter 218]	Loss: [0.36227894794195892]
[Valid]		MAP	P20	NDCG20
		0.2516	0.4763	0.5011
[Test]		MAP	P20	NDCG20
		0.2421	0.5310	0.5352
[Iter 219]	Loss: [0.36687338542193176]
[Valid]		MAP	P20	NDCG20
		0.2521	0.4789	0.5003
[Test]		MAP	P20	NDCG20
		0.2435	0.5310	0.5351
[Iter 220]	Loss: [0.36037946939468385]
[Valid]		MAP	P20	NDCG20
		0.2514	0.4842	0.5088
[Test]		MAP	P20	NDCG20
		0.2414	0.5214	0.5283
[Iter 221]	Loss: [0.3589814680814743]
[Valid]		MAP	P20	NDCG20
		0.2514	0.4868	0.5077
[Test]		MAP	P20	NDCG20
		0.2424	0.5262	0.5326
[Iter 222]	Loss: [0.34217941835522653]
[Valid]		MAP	P20	NDCG20
		0.2493	0.4816	0.5070
[Test]		MAP	P20	NDCG20
		0.2399	0.5214	0.5290
[Iter 223]	Loss: [0.34081375420093535]
[Valid]		MAP	P20	NDCG20
		0.2510	0.4895	0.5085
[Test]		MAP	P20	NDCG20
		0.2427	0.5310	0.5356
[Iter 224]	Loss: [0.39247813336551191]
[Valid]		MAP	P20	NDCG20
		0.2511	0.4842	0.5080
[Test]		MAP	P20	NDCG20
		0.2415	0.5238	0.5297
[Iter 225]	Loss: [0.35924645803868771]
[Valid]		MAP	P20	NDCG20
		0.2513	0.4842	0.5054
[Test]		MAP	P20	NDCG20
		0.2434	0.5262	0.5331
[Iter 226]	Loss: [0.354033127091825]
[Valid]		MAP	P20	NDCG20
		0.2506	0.4868	0.5075
[Test]		MAP	P20	NDCG20
		0.2418	0.5238	0.5316
[Iter 227]	Loss: [0.35585822880268098]
[Valid]		MAP	P20	NDCG20
		0.2520	0.4789	0.5023
[Test]		MAP	P20	NDCG20
		0.2428	0.5262	0.5305
[Iter 228]	Loss: [0.36102690804749726]
[Valid]		MAP	P20	NDCG20
		0.2523	0.4789	0.5007
[Test]		MAP	P20	NDCG20
		0.2442	0.5286	0.5339
[Iter 229]	Loss: [0.35616759151220323]
[Valid]		MAP	P20	NDCG20
		0.2519	0.4842	0.5058
[Test]		MAP	P20	NDCG20
		0.2438	0.5310	0.5337
[Iter 230]	Loss: [0.38759992279112337]
[Valid]		MAP	P20	NDCG20
		0.2516	0.4763	0.5040
[Test]		MAP	P20	NDCG20
		0.2419	0.5333	0.5351
[Iter 231]	Loss: [0.36760417297482489]
[Valid]		MAP	P20	NDCG20
		0.2517	0.4816	0.5073
[Test]		MAP	P20	NDCG20
		0.2414	0.5286	0.5324
[Iter 232]	Loss: [0.33402761287987232]
[Valid]		MAP	P20	NDCG20
		0.2516	0.4842	0.5062
[Test]		MAP	P20	NDCG20
		0.2426	0.5238	0.5319
[Iter 233]	Loss: [0.35633832294493911]
[Valid]		MAP	P20	NDCG20
		0.2514	0.4816	0.5074
[Test]		MAP	P20	NDCG20
		0.2417	0.5238	0.5315
[Iter 234]	Loss: [0.34758847732096909]
[Valid]		MAP	P20	NDCG20
		0.2516	0.4842	0.5057
[Test]		MAP	P20	NDCG20
		0.2429	0.5262	0.5332
[Iter 235]	Loss: [0.34997772071510552]
[Valid]		MAP	P20	NDCG20
		0.2515	0.4816	0.5038
[Test]		MAP	P20	NDCG20
		0.2425	0.5286	0.5348
[Iter 236]	Loss: [0.33450149431824683]
[Valid]		MAP	P20	NDCG20
		0.2521	0.4816	0.5043
[Test]		MAP	P20	NDCG20
		0.2431	0.5310	0.5333
[Iter 237]	Loss: [0.33886917624622581]
[Valid]		MAP	P20	NDCG20
		0.2517	0.4763	0.5033
[Test]		MAP	P20	NDCG20
		0.2421	0.5214	0.5298
[Iter 238]	Loss: [0.35352902330458164]
[Valid]		MAP	P20	NDCG20
		0.2510	0.4895	0.5092
[Test]		MAP	P20	NDCG20
		0.2428	0.5286	0.5352
[Iter 239]	Loss: [0.35706042394042015]
[Valid]		MAP	P20	NDCG20
		0.2520	0.4868	0.5061
[Test]		MAP	P20	NDCG20
		0.2442	0.5310	0.5355
[Iter 240]	Loss: [0.33135456983000039]
[Valid]		MAP	P20	NDCG20
		0.2521	0.4763	0.5030
[Test]		MAP	P20	NDCG20
		0.2415	0.5238	0.5264
[Iter 241]	Loss: [0.36457323767244815]
[Valid]		MAP	P20	NDCG20
		0.2519	0.4789	0.5052
[Test]		MAP	P20	NDCG20
		0.2416	0.5190	0.5249
[Iter 242]	Loss: [0.35555020123720171]
[Valid]		MAP	P20	NDCG20
		0.2524	0.4789	0.5023
[Test]		MAP	P20	NDCG20
		0.2423	0.5214	0.5257
[Iter 243]	Loss: [0.34224397856742145]
[Valid]		MAP	P20	NDCG20
		0.2526	0.4842	0.5049
[Test]		MAP	P20	NDCG20
		0.2429	0.5214	0.5295
[Iter 244]	Loss: [0.36106890598312019]
[Valid]		MAP	P20	NDCG20
		0.2514	0.4816	0.5071
[Test]		MAP	P20	NDCG20
		0.2402	0.5167	0.5223
[Iter 245]	Loss: [0.35500122785568239]
[Valid]		MAP	P20	NDCG20
		0.2515	0.4763	0.5037
[Test]		MAP	P20	NDCG20
		0.2406	0.5238	0.5283
[Iter 246]	Loss: [0.36290637992322444]
[Valid]		MAP	P20	NDCG20
		0.2516	0.4763	0.5036
[Test]		MAP	P20	NDCG20
		0.2405	0.5238	0.5280
[Iter 247]	Loss: [0.33607817168347537]
[Valid]		MAP	P20	NDCG20
		0.2518	0.4868	0.5103
[Test]		MAP	P20	NDCG20
		0.2409	0.5238	0.5282
[Iter 248]	Loss: [0.36693416686728597]
[Valid]		MAP	P20	NDCG20
		0.2520	0.4789	0.5050
[Test]		MAP	P20	NDCG20
		0.2410	0.5190	0.5243
[Iter 249]	Loss: [0.36227683804929256]
[Valid]		MAP	P20	NDCG20
		0.2513	0.4763	0.5042
[Test]		MAP	P20	NDCG20
		0.2403	0.5214	0.5280
[Iter 250]	Loss: [0.3653551872819662]
[Valid]		MAP	P20	NDCG20
		0.2522	0.4763	0.5043
[Test]		MAP	P20	NDCG20
		0.2410	0.5190	0.5263
[Iter 251]	Loss: [0.34435521792620422]
[Valid]		MAP	P20	NDCG20
		0.2511	0.4842	0.5052
[Test]		MAP	P20	NDCG20
		0.2426	0.5262	0.5324
[Iter 252]	Loss: [0.36097738265991208]
[Valid]		MAP	P20	NDCG20
		0.2506	0.4816	0.5038
[Test]		MAP	P20	NDCG20
		0.2416	0.5238	0.5315
[Iter 253]	Loss: [0.35909282278269528]
[Valid]		MAP	P20	NDCG20
		0.2511	0.4868	0.5069
[Test]		MAP	P20	NDCG20
		0.2423	0.5238	0.5289
[Iter 254]	Loss: [0.34459122762084005]
[Valid]		MAP	P20	NDCG20
		0.2509	0.4816	0.5066
[Test]		MAP	P20	NDCG20
		0.2400	0.5238	0.5276
[Iter 255]	Loss: [0.35612312506884336]
[Valid]		MAP	P20	NDCG20
		0.2501	0.4895	0.5092
[Test]		MAP	P20	NDCG20
		0.2407	0.5214	0.5287
[Iter 256]	Loss: [0.34425794392824172]
[Valid]		MAP	P20	NDCG20
		0.2515	0.4763	0.5039
[Test]		MAP	P20	NDCG20
		0.2403	0.5214	0.5276
[Iter 257]	Loss: [0.38100231971591714]
[Valid]		MAP	P20	NDCG20
		0.2516	0.4868	0.5069
[Test]		MAP	P20	NDCG20
		0.2432	0.5310	0.5360
[Iter 258]	Loss: [0.35382970519363882]
[Valid]		MAP	P20	NDCG20
		0.2515	0.4816	0.5040
[Test]		MAP	P20	NDCG20
		0.2421	0.5143	0.5247
[Iter 259]	Loss: [0.3566913910396397]
[Valid]		MAP	P20	NDCG20
		0.2512	0.4921	0.5099
[Test]		MAP	P20	NDCG20
		0.2430	0.5310	0.5358
[Iter 260]	Loss: [0.35129106629639861]
[Valid]		MAP	P20	NDCG20
		0.2506	0.4868	0.5105
[Test]		MAP	P20	NDCG20
		0.2401	0.5262	0.5311
[Iter 261]	Loss: [0.33766227819025518]
[Valid]		MAP	P20	NDCG20
		0.2524	0.4842	0.5056
[Test]		MAP	P20	NDCG20
		0.2424	0.5214	0.5296
[Iter 262]	Loss: [0.36835505962371828]
[Valid]		MAP	P20	NDCG20
		0.2517	0.4816	0.5043
[Test]		MAP	P20	NDCG20
		0.2421	0.5286	0.5334
[Iter 263]	Loss: [0.36808344632387163]
[Valid]		MAP	P20	NDCG20
		0.2524	0.4921	0.5105
[Test]		MAP	P20	NDCG20
		0.2437	0.5286	0.5344
[Attention]		Current best iteration 204

2018-09-25 16:31:29,885 : INFO : [EVAL RESULT] Based on best MAP on validation set
2018-09-25 16:31:29,886 : INFO : Achieve best result on iteration 204
2018-09-25 16:31:29,886 : INFO : 		MAP	P@20	NDCG@20
2018-09-25 16:31:29,886 : INFO : [Valid]	0.2527	0.4763	0.4981
2018-09-25 16:31:29,886 : INFO : [Test]		0.2433	0.5262	0.5316