save path : ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604 {'arch': 'acc2', 'arch_learning_rate': 0.0003, 'arch_weight_decay': 0.001, 'batch_size': 128, 'cutout': 16, 'data_path': '/mnt/dome/dongxuanyi/.torch/cifar.python', 'dataset': 'cifar10', 'epochs': 250, 'grad_clip': 5.0, 'init_channels': 16, 'layers': 8, 'learning_rate_max': 0.05, 'learning_rate_min': 0.001, 'manualSeed': 604, 'model_config': './configs/nas-cifar-cos.config', 'momentum': 0.9, 'only_base': False, 'print_freq': 100, 'resume': None, 'save_path': './snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604', 'tau_max': 10.0, 'tau_min': 4.0, 'train_portion': 0.5, 'validate': True, 'weight_decay': 0.0003, 'workers': 8} Random Seed: 604 Python version : 3.6.5 |Anaconda, Inc.| (default, Apr 29 2018, 16:14:56) [GCC 7.2.0] Torch version : 0.4.1 CUDA version : 9.0.176 cuDNN version : 7102 Num of GPUs : 1 Parameter size = 1.931 MB Train-transformation : Compose( RandomHorizontalFlip(p=0.5) RandomCrop(size=(32, 32), padding=4) ToTensor() Normalize(mean=[0.4913725490196078, 0.4823529411764706, 0.4466666666666667], std=[0.24705882352941178, 0.24352941176470588, 0.2615686274509804]) Cutout(length=16) ) Test--transformation : Compose( ToTensor() Normalize(mean=[0.4913725490196078, 0.4823529411764706, 0.4466666666666667], std=[0.24705882352941178, 0.24352941176470588, 0.2615686274509804]) ) Train model-search from scratch. ==>>[2018-10-02-12:52:58] [Epoch=000/250] [Need: 00:00:00] [LR=0.0500 ~ 0.0500] [Batch=128], tau=10.0 genotype = Genotype(normal=[('avg_pool_3x3', 0, 0.12510184943675995), ('sep_conv_3x3', 1, 0.1250765323638916), ('avg_pool_3x3', 1, 0.12522822618484497), ('sep_conv_3x3', 0, 0.1252187192440033), ('dil_conv_3x3', 0, 0.12520387768745422), ('dil_conv_3x3', 2, 0.12520188093185425), ('dil_conv_3x3', 1, 0.12530556321144104), ('dil_conv_5x5', 0, 0.12529577314853668)], normal_concat=range(2, 6), reduce=[('skip_connect', 1, 0.12531782686710358), ('avg_pool_3x3', 0, 0.125143364071846), ('dil_conv_5x5', 2, 0.1252886801958084), ('dil_conv_3x3', 1, 0.12526069581508636), ('sep_conv_3x3', 1, 0.1251547485589981), ('skip_connect', 3, 0.12515150010585785), ('max_pool_3x3', 0, 0.12521125376224518), ('sep_conv_5x5', 4, 0.12519711256027222)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-12:53:02] Epoch: [000][000/196] Time 3.17 (3.17) Data 0.75 (0.75) Loss 2.423 (2.423) Prec@1 13.28 (13.28) Prec@5 52.34 (52.34) TRAIN-SEARCH [2018-10-02-12:53:56] Epoch: [000][100/196] Time 0.52 (0.56) Data 0.00 (0.01) Loss 3.184 (2.882) Prec@1 14.84 (11.32) Prec@5 55.47 (52.48) TRAIN-SEARCH [2018-10-02-12:54:50] Epoch: [000][195/196] Time 1.39 (0.57) Data 0.00 (0.01) Loss 4.235 (3.266) Prec@1 10.00 (11.05) Prec@5 52.50 (51.56) VALID-SEARCH [2018-10-02-12:54:51] Epoch: [000][000/196] Loss 19.031 (19.031) Prec@1 12.50 (12.50) Prec@5 57.03 (57.03) VALID-SEARCH [2018-10-02-12:54:57] Epoch: [000][100/196] Loss 760.931 (86.013) Prec@1 7.03 (10.18) Prec@5 38.28 (50.91) VALID-SEARCH [2018-10-02-12:55:02] Epoch: [000][195/196] Loss 32.255 (79.634) Prec@1 10.00 (10.56) Prec@5 50.00 (51.04) 000/250, Train-Accuracy = 11.05, Test-Accuracy = 10.56 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-12:55:03] [Epoch=001/250] [Need: 08:38:09] [LR=0.0500 ~ 0.0500] [Batch=128], tau=9.976 genotype = Genotype(normal=[('dil_conv_3x3', 0, 0.1258767694234848), ('sep_conv_3x3', 1, 0.12577508389949799), ('dil_conv_5x5', 1, 0.12648478150367737), ('dil_conv_5x5', 2, 0.12624989449977875), ('sep_conv_3x3', 3, 0.1259712278842926), ('sep_conv_3x3', 2, 0.12568676471710205), ('dil_conv_5x5', 4, 0.12592409551143646), ('dil_conv_3x3', 1, 0.12585531175136566)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1, 0.12622256577014923), ('avg_pool_3x3', 0, 0.12545040249824524), ('max_pool_3x3', 0, 0.12571291625499725), ('max_pool_3x3', 1, 0.1255507916212082), ('skip_connect', 0, 0.12665025889873505), ('avg_pool_3x3', 1, 0.12625785171985626), ('sep_conv_3x3', 1, 0.1259431689977646), ('dil_conv_5x5', 3, 0.12588220834732056)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-12:55:05] Epoch: [001][000/196] Time 1.76 (1.76) Data 0.85 (0.85) Loss 4.264 (4.264) Prec@1 10.16 (10.16) Prec@5 54.69 (54.69) TRAIN-SEARCH [2018-10-02-12:56:00] Epoch: [001][100/196] Time 0.50 (0.56) Data 0.00 (0.01) Loss 6.826 (3.910) Prec@1 4.69 (10.44) Prec@5 45.31 (51.35) TRAIN-SEARCH [2018-10-02-12:56:54] Epoch: [001][195/196] Time 0.53 (0.56) Data 0.00 (0.01) Loss 5.192 (3.898) Prec@1 10.00 (10.57) Prec@5 52.50 (51.06) VALID-SEARCH [2018-10-02-12:56:55] Epoch: [001][000/196] Loss 4.932 (4.932) Prec@1 8.59 (8.59) Prec@5 50.00 (50.00) VALID-SEARCH [2018-10-02-12:57:00] Epoch: [001][100/196] Loss 11.281 (52.175) Prec@1 7.81 (10.10) Prec@5 45.31 (50.57) VALID-SEARCH [2018-10-02-12:57:05] Epoch: [001][195/196] Loss 2.816 (61.494) Prec@1 7.50 (10.17) Prec@5 52.50 (50.30) 001/250, Train-Accuracy = 10.57, Test-Accuracy = 10.17 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-12:57:06] [Epoch=002/250] [Need: 08:29:32] [LR=0.0500 ~ 0.0500] [Batch=128], tau=9.952 genotype = Genotype(normal=[('sep_conv_3x3', 1, 0.1265447437763214), ('sep_conv_5x5', 0, 0.12639746069908142), ('dil_conv_5x5', 1, 0.1262359321117401), ('sep_conv_5x5', 2, 0.1262275129556656), ('sep_conv_3x3', 2, 0.12643761932849884), ('avg_pool_3x3', 0, 0.1255975365638733), ('sep_conv_3x3', 1, 0.12577295303344727), ('sep_conv_3x3', 2, 0.1256410777568817)], normal_concat=range(2, 6), reduce=[('dil_conv_5x5', 0, 0.12672054767608643), ('sep_conv_3x3', 1, 0.12595340609550476), ('sep_conv_3x3', 1, 0.126859650015831), ('max_pool_3x3', 0, 0.12632939219474792), ('skip_connect', 0, 0.1270403265953064), ('avg_pool_3x3', 1, 0.12631961703300476), ('sep_conv_3x3', 1, 0.1270965188741684), ('sep_conv_3x3', 4, 0.12687663733959198)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-12:57:09] Epoch: [002][000/196] Time 1.82 (1.82) Data 1.14 (1.14) Loss 3.639 (3.639) Prec@1 7.03 (7.03) Prec@5 46.09 (46.09) TRAIN-SEARCH [2018-10-02-12:58:01] Epoch: [002][100/196] Time 0.54 (0.53) Data 0.00 (0.01) Loss 2.992 (3.870) Prec@1 14.06 (10.28) Prec@5 56.25 (50.09) TRAIN-SEARCH [2018-10-02-12:58:58] Epoch: [002][195/196] Time 0.42 (0.57) Data 0.00 (0.01) Loss 4.514 (3.791) Prec@1 5.00 (10.32) Prec@5 47.50 (50.60) VALID-SEARCH [2018-10-02-12:58:59] Epoch: [002][000/196] Loss 1587.455 (1587.455) Prec@1 10.16 (10.16) Prec@5 53.12 (53.12) VALID-SEARCH [2018-10-02-12:59:05] Epoch: [002][100/196] Loss 14.231 (223.717) Prec@1 7.81 (10.57) Prec@5 48.44 (50.18) VALID-SEARCH [2018-10-02-12:59:10] Epoch: [002][195/196] Loss 3.872 (155.612) Prec@1 10.00 (10.43) Prec@5 52.50 (50.15) 002/250, Train-Accuracy = 10.32, Test-Accuracy = 10.43 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-12:59:11] [Epoch=003/250] [Need: 08:33:54] [LR=0.0500 ~ 0.0500] [Batch=128], tau=9.928 genotype = Genotype(normal=[('sep_conv_3x3', 1, 0.1259808987379074), ('sep_conv_5x5', 0, 0.1254369616508484), ('sep_conv_5x5', 2, 0.12668009102344513), ('dil_conv_5x5', 1, 0.1261778622865677), ('sep_conv_3x3', 2, 0.12643013894557953), ('sep_conv_3x3', 3, 0.12605899572372437), ('dil_conv_3x3', 0, 0.12647771835327148), ('sep_conv_3x3', 2, 0.1256338655948639)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1, 0.1264667809009552), ('avg_pool_3x3', 0, 0.12612171471118927), ('sep_conv_3x3', 1, 0.1277884691953659), ('sep_conv_5x5', 0, 0.12597587704658508), ('sep_conv_5x5', 3, 0.1276656538248062), ('skip_connect', 0, 0.12683604657649994), ('sep_conv_3x3', 4, 0.12814761698246002), ('sep_conv_3x3', 1, 0.12726645171642303)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-12:59:13] Epoch: [003][000/196] Time 1.00 (1.00) Data 0.53 (0.53) Loss 4.263 (4.263) Prec@1 9.38 (9.38) Prec@5 42.97 (42.97) TRAIN-SEARCH [2018-10-02-13:00:05] Epoch: [003][100/196] Time 0.50 (0.53) Data 0.00 (0.01) Loss 3.955 (3.631) Prec@1 14.06 (10.26) Prec@5 55.47 (49.95) TRAIN-SEARCH [2018-10-02-13:00:58] Epoch: [003][195/196] Time 0.52 (0.54) Data 0.00 (0.00) Loss 3.055 (3.517) Prec@1 12.50 (10.30) Prec@5 40.00 (50.21) VALID-SEARCH [2018-10-02-13:00:59] Epoch: [003][000/196] Loss 17.388 (17.388) Prec@1 10.16 (10.16) Prec@5 55.47 (55.47) VALID-SEARCH [2018-10-02-13:01:05] Epoch: [003][100/196] Loss 2.480 (12.883) Prec@1 10.94 (10.52) Prec@5 56.25 (50.73) VALID-SEARCH [2018-10-02-13:01:10] Epoch: [003][195/196] Loss 5.535 (15.418) Prec@1 20.00 (10.40) Prec@5 45.00 (50.40) 003/250, Train-Accuracy = 10.30, Test-Accuracy = 10.40 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-13:01:11] [Epoch=004/250] [Need: 08:09:57] [LR=0.0500 ~ 0.0500] [Batch=128], tau=9.904 genotype = Genotype(normal=[('sep_conv_3x3', 1, 0.12579506635665894), ('sep_conv_5x5', 0, 0.1250472217798233), ('dil_conv_5x5', 1, 0.12718777358531952), ('sep_conv_5x5', 2, 0.12714225053787231), ('sep_conv_3x3', 2, 0.12593823671340942), ('sep_conv_5x5', 0, 0.1256582885980606), ('dil_conv_3x3', 0, 0.12595796585083008), ('avg_pool_3x3', 1, 0.12538009881973267)], normal_concat=range(2, 6), reduce=[('sep_conv_5x5', 1, 0.12732727825641632), ('dil_conv_5x5', 0, 0.1263682246208191), ('sep_conv_3x3', 1, 0.12705875933170319), ('sep_conv_5x5', 0, 0.12633384764194489), ('sep_conv_5x5', 3, 0.12852294743061066), ('sep_conv_5x5', 2, 0.12654881179332733), ('sep_conv_3x3', 1, 0.12848907709121704), ('dil_conv_3x3', 4, 0.128175288438797)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-13:01:12] Epoch: [004][000/196] Time 1.04 (1.04) Data 0.57 (0.57) Loss 3.318 (3.318) Prec@1 10.94 (10.94) Prec@5 53.91 (53.91) TRAIN-SEARCH [2018-10-02-13:02:04] Epoch: [004][100/196] Time 0.68 (0.53) Data 0.00 (0.01) Loss 3.501 (3.300) Prec@1 10.94 (10.50) Prec@5 48.44 (51.00) TRAIN-SEARCH [2018-10-02-13:02:59] Epoch: [004][195/196] Time 0.48 (0.55) Data 0.00 (0.01) Loss 2.413 (3.176) Prec@1 12.50 (10.53) Prec@5 55.00 (51.06) VALID-SEARCH [2018-10-02-13:03:00] Epoch: [004][000/196] Loss 2.354 (2.354) Prec@1 17.19 (17.19) Prec@5 57.81 (57.81) VALID-SEARCH [2018-10-02-13:03:06] Epoch: [004][100/196] Loss 2.373 (5.225) Prec@1 9.38 (11.48) Prec@5 57.81 (52.58) VALID-SEARCH [2018-10-02-13:03:10] Epoch: [004][195/196] Loss 3.554 (5.355) Prec@1 10.00 (11.35) Prec@5 42.50 (52.38) 004/250, Train-Accuracy = 10.53, Test-Accuracy = 11.35 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-13:03:11] [Epoch=005/250] [Need: 08:11:53] [LR=0.0500 ~ 0.0500] [Batch=128], tau=9.88 genotype = Genotype(normal=[('sep_conv_3x3', 1, 0.12655358016490936), ('dil_conv_3x3', 0, 0.12440474331378937), ('sep_conv_5x5', 2, 0.12720274925231934), ('dil_conv_5x5', 1, 0.127177432179451), ('sep_conv_5x5', 0, 0.12631218135356903), ('sep_conv_3x3', 2, 0.12597131729125977), ('sep_conv_3x3', 3, 0.12546797096729279), ('dil_conv_3x3', 1, 0.12541672587394714)], normal_concat=range(2, 6), reduce=[('sep_conv_5x5', 1, 0.12718409299850464), ('dil_conv_5x5', 0, 0.12599895894527435), ('sep_conv_3x3', 1, 0.12699167430400848), ('sep_conv_5x5', 0, 0.12688395380973816), ('sep_conv_5x5', 3, 0.1289527416229248), ('sep_conv_5x5', 2, 0.12678755819797516), ('sep_conv_3x3', 4, 0.12818771600723267), ('sep_conv_3x3', 1, 0.1280944049358368)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-13:03:12] Epoch: [005][000/196] Time 1.01 (1.01) Data 0.61 (0.61) Loss 3.434 (3.434) Prec@1 11.72 (11.72) Prec@5 39.84 (39.84) TRAIN-SEARCH [2018-10-02-13:04:07] Epoch: [005][100/196] Time 0.55 (0.55) Data 0.00 (0.01) Loss 2.607 (2.960) Prec@1 14.84 (10.95) Prec@5 59.38 (53.08) TRAIN-SEARCH [2018-10-02-13:05:00] Epoch: [005][195/196] Time 0.50 (0.55) Data 0.00 (0.01) Loss 2.906 (2.894) Prec@1 12.50 (11.16) Prec@5 57.50 (53.22) VALID-SEARCH [2018-10-02-13:05:01] Epoch: [005][000/196] Loss 2.753 (2.753) Prec@1 15.62 (15.62) Prec@5 50.78 (50.78) VALID-SEARCH [2018-10-02-13:05:07] Epoch: [005][100/196] Loss 2.945 (5.713) Prec@1 6.25 (10.59) Prec@5 50.78 (51.38) VALID-SEARCH [2018-10-02-13:05:12] Epoch: [005][195/196] Loss 3.069 (5.557) Prec@1 2.50 (10.52) Prec@5 40.00 (51.90) 005/250, Train-Accuracy = 11.16, Test-Accuracy = 10.52 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-13:05:12] [Epoch=006/250] [Need: 08:13:07] [LR=0.0499 ~ 0.0499] [Batch=128], tau=9.856 genotype = Genotype(normal=[('sep_conv_3x3', 1, 0.1267668753862381), ('dil_conv_3x3', 0, 0.12468340992927551), ('sep_conv_5x5', 2, 0.12800438702106476), ('dil_conv_5x5', 1, 0.1272728145122528), ('sep_conv_5x5', 0, 0.12618429958820343), ('sep_conv_3x3', 2, 0.1259736269712448), ('dil_conv_3x3', 1, 0.1253475844860077), ('sep_conv_3x3', 3, 0.12529125809669495)], normal_concat=range(2, 6), reduce=[('sep_conv_5x5', 1, 0.12704086303710938), ('dil_conv_5x5', 0, 0.12601014971733093), ('sep_conv_5x5', 0, 0.12720543146133423), ('sep_conv_3x3', 1, 0.12696358561515808), ('sep_conv_5x5', 3, 0.12873831391334534), ('sep_conv_5x5', 1, 0.12697847187519073), ('sep_conv_3x3', 1, 0.12823618948459625), ('sep_conv_3x3', 4, 0.12812404334545135)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-13:05:14] Epoch: [006][000/196] Time 1.07 (1.07) Data 0.62 (0.62) Loss 2.924 (2.924) Prec@1 10.94 (10.94) Prec@5 51.56 (51.56) TRAIN-SEARCH [2018-10-02-13:06:09] Epoch: [006][100/196] Time 0.50 (0.55) Data 0.00 (0.01) Loss 2.326 (2.659) Prec@1 11.72 (12.81) Prec@5 56.25 (56.13) TRAIN-SEARCH [2018-10-02-13:07:00] Epoch: [006][195/196] Time 0.43 (0.55) Data 0.00 (0.01) Loss 2.501 (2.589) Prec@1 15.00 (13.40) Prec@5 60.00 (57.65) VALID-SEARCH [2018-10-02-13:07:02] Epoch: [006][000/196] Loss 3.004 (3.004) Prec@1 16.41 (16.41) Prec@5 54.69 (54.69) VALID-SEARCH [2018-10-02-13:07:07] Epoch: [006][100/196] Loss 2.564 (2.782) Prec@1 8.59 (11.75) Prec@5 66.41 (56.17) VALID-SEARCH [2018-10-02-13:07:11] Epoch: [006][195/196] Loss 2.242 (2.958) Prec@1 17.50 (11.86) Prec@5 75.00 (56.29) 006/250, Train-Accuracy = 13.40, Test-Accuracy = 11.86 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-13:07:12] [Epoch=007/250] [Need: 08:05:20] [LR=0.0499 ~ 0.0499] [Batch=128], tau=9.832 genotype = Genotype(normal=[('sep_conv_3x3', 1, 0.12693236768245697), ('dil_conv_3x3', 0, 0.1249426007270813), ('sep_conv_5x5', 2, 0.12799976766109467), ('dil_conv_5x5', 1, 0.1273149996995926), ('sep_conv_5x5', 0, 0.12626926600933075), ('sep_conv_3x3', 2, 0.12608635425567627), ('dil_conv_3x3', 1, 0.1253713071346283), ('sep_conv_3x3', 3, 0.12536747753620148)], normal_concat=range(2, 6), reduce=[('sep_conv_5x5', 1, 0.1269925832748413), ('dil_conv_5x5', 0, 0.12612329423427582), ('sep_conv_5x5', 0, 0.12738899886608124), ('sep_conv_5x5', 2, 0.1268862634897232), ('sep_conv_5x5', 3, 0.12844277918338776), ('sep_conv_5x5', 1, 0.12693871557712555), ('sep_conv_3x3', 1, 0.1279827207326889), ('sep_conv_3x3', 4, 0.12791207432746887)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-13:07:14] Epoch: [007][000/196] Time 1.20 (1.20) Data 0.75 (0.75) Loss 2.456 (2.456) Prec@1 7.81 (7.81) Prec@5 46.09 (46.09) TRAIN-SEARCH [2018-10-02-13:08:09] Epoch: [007][100/196] Time 0.60 (0.56) Data 0.00 (0.01) Loss 2.330 (2.347) Prec@1 14.84 (15.38) Prec@5 62.50 (63.95) TRAIN-SEARCH [2018-10-02-13:09:02] Epoch: [007][195/196] Time 0.60 (0.56) Data 0.00 (0.01) Loss 2.192 (2.304) Prec@1 25.00 (15.91) Prec@5 70.00 (66.22) VALID-SEARCH [2018-10-02-13:09:03] Epoch: [007][000/196] Loss 2.123 (2.123) Prec@1 22.66 (22.66) Prec@5 71.88 (71.88) VALID-SEARCH [2018-10-02-13:09:09] Epoch: [007][100/196] Loss 2.458 (2.476) Prec@1 10.94 (14.07) Prec@5 50.00 (61.24) VALID-SEARCH [2018-10-02-13:09:14] Epoch: [007][195/196] Loss 3.190 (2.544) Prec@1 5.00 (14.15) Prec@5 47.50 (61.46) 007/250, Train-Accuracy = 15.91, Test-Accuracy = 14.15 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-13:09:14] [Epoch=008/250] [Need: 08:13:00] [LR=0.0499 ~ 0.0499] [Batch=128], tau=9.808 genotype = Genotype(normal=[('sep_conv_3x3', 1, 0.1268656700849533), ('dil_conv_3x3', 0, 0.12503595650196075), ('sep_conv_5x5', 2, 0.1279107928276062), ('dil_conv_5x5', 1, 0.12734125554561615), ('sep_conv_5x5', 0, 0.12640374898910522), ('sep_conv_3x3', 2, 0.12615953385829926), ('sep_conv_3x3', 3, 0.125563845038414), ('dil_conv_3x3', 1, 0.12543104588985443)], normal_concat=range(2, 6), reduce=[('sep_conv_5x5', 1, 0.12676525115966797), ('dil_conv_5x5', 0, 0.1260918378829956), ('sep_conv_5x5', 0, 0.1271100789308548), ('sep_conv_5x5', 2, 0.1267634481191635), ('sep_conv_5x5', 3, 0.1282270848751068), ('sep_conv_5x5', 1, 0.12681318819522858), ('sep_conv_3x3', 1, 0.1277783364057541), ('sep_conv_3x3', 4, 0.12765540182590485)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-13:09:16] Epoch: [008][000/196] Time 0.98 (0.98) Data 0.54 (0.54) Loss 2.341 (2.341) Prec@1 12.50 (12.50) Prec@5 68.75 (68.75) TRAIN-SEARCH [2018-10-02-13:10:09] Epoch: [008][100/196] Time 0.57 (0.53) Data 0.01 (0.01) Loss 2.196 (2.175) Prec@1 11.72 (17.57) Prec@5 64.06 (70.51) TRAIN-SEARCH [2018-10-02-13:11:03] Epoch: [008][195/196] Time 0.40 (0.55) Data 0.00 (0.01) Loss 2.228 (2.169) Prec@1 17.50 (18.04) Prec@5 62.50 (71.28) VALID-SEARCH [2018-10-02-13:11:05] Epoch: [008][000/196] Loss 2.483 (2.483) Prec@1 14.84 (14.84) Prec@5 72.66 (72.66) VALID-SEARCH [2018-10-02-13:11:10] Epoch: [008][100/196] Loss 2.271 (2.358) Prec@1 13.28 (16.71) Prec@5 64.06 (67.60) VALID-SEARCH [2018-10-02-13:11:15] Epoch: [008][195/196] Loss 2.168 (2.348) Prec@1 20.00 (16.86) Prec@5 70.00 (67.13) 008/250, Train-Accuracy = 18.04, Test-Accuracy = 16.86 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-13:11:15] [Epoch=009/250] [Need: 08:05:30] [LR=0.0498 ~ 0.0498] [Batch=128], tau=9.784 genotype = Genotype(normal=[('sep_conv_3x3', 1, 0.1269143670797348), ('dil_conv_3x3', 0, 0.12518420815467834), ('sep_conv_5x5', 2, 0.1280393898487091), ('dil_conv_5x5', 1, 0.12735311686992645), ('sep_conv_5x5', 0, 0.12642602622509003), ('sep_conv_3x3', 2, 0.12622816860675812), ('sep_conv_3x3', 3, 0.12559477984905243), ('dil_conv_3x3', 0, 0.12556660175323486)], normal_concat=range(2, 6), reduce=[('sep_conv_5x5', 1, 0.1265411376953125), ('dil_conv_5x5', 0, 0.12598522007465363), ('sep_conv_5x5', 0, 0.12687155604362488), ('sep_conv_5x5', 2, 0.12650468945503235), ('sep_conv_5x5', 3, 0.12795943021774292), ('dil_conv_5x5', 2, 0.1265641748905182), ('sep_conv_3x3', 1, 0.12755821645259857), ('sep_conv_5x5', 4, 0.1274576038122177)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-13:11:17] Epoch: [009][000/196] Time 1.07 (1.07) Data 0.60 (0.60) Loss 2.092 (2.092) Prec@1 21.88 (21.88) Prec@5 77.34 (77.34) TRAIN-SEARCH [2018-10-02-13:12:10] Epoch: [009][100/196] Time 0.61 (0.54) Data 0.01 (0.01) Loss 2.066 (2.138) Prec@1 28.12 (19.21) Prec@5 78.12 (73.24) TRAIN-SEARCH [2018-10-02-13:13:06] Epoch: [009][195/196] Time 0.39 (0.56) Data 0.00 (0.01) Loss 2.267 (2.129) Prec@1 17.50 (19.60) Prec@5 82.50 (73.76) VALID-SEARCH [2018-10-02-13:13:07] Epoch: [009][000/196] Loss 1.990 (1.990) Prec@1 19.53 (19.53) Prec@5 82.81 (82.81) VALID-SEARCH [2018-10-02-13:13:12] Epoch: [009][100/196] Loss 2.151 (2.360) Prec@1 19.53 (16.96) Prec@5 72.66 (68.93) VALID-SEARCH [2018-10-02-13:13:16] Epoch: [009][195/196] Loss 2.436 (2.363) Prec@1 17.50 (16.86) Prec@5 67.50 (68.29) 009/250, Train-Accuracy = 19.60, Test-Accuracy = 16.86 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-13:13:16] [Epoch=010/250] [Need: 08:03:23] [LR=0.0498 ~ 0.0498] [Batch=128], tau=9.76 genotype = Genotype(normal=[('sep_conv_3x3', 1, 0.1268506646156311), ('dil_conv_3x3', 0, 0.12529367208480835), ('sep_conv_5x5', 2, 0.12809424102306366), ('dil_conv_5x5', 1, 0.12729774415493011), ('sep_conv_3x3', 2, 0.1262792944908142), ('dil_conv_5x5', 1, 0.12615594267845154), ('dil_conv_3x3', 0, 0.12578728795051575), ('sep_conv_5x5', 4, 0.12570618093013763)], normal_concat=range(2, 6), reduce=[('sep_conv_5x5', 1, 0.12640118598937988), ('dil_conv_5x5', 0, 0.1259354203939438), ('sep_conv_5x5', 0, 0.1265779435634613), ('sep_conv_5x5', 2, 0.126329243183136), ('sep_conv_5x5', 3, 0.12763559818267822), ('sep_conv_5x5', 2, 0.12648779153823853), ('sep_conv_3x3', 1, 0.12709854543209076), ('dil_conv_5x5', 2, 0.12700963020324707)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-13:13:18] Epoch: [010][000/196] Time 1.10 (1.10) Data 0.61 (0.61) Loss 2.132 (2.132) Prec@1 17.97 (17.97) Prec@5 77.34 (77.34) TRAIN-SEARCH [2018-10-02-13:14:12] Epoch: [010][100/196] Time 0.52 (0.55) Data 0.00 (0.01) Loss 2.049 (2.101) Prec@1 20.31 (20.71) Prec@5 82.81 (75.97) TRAIN-SEARCH [2018-10-02-13:15:08] Epoch: [010][195/196] Time 0.55 (0.57) Data 0.00 (0.01) Loss 2.093 (2.098) Prec@1 22.50 (20.82) Prec@5 65.00 (75.78) VALID-SEARCH [2018-10-02-13:15:09] Epoch: [010][000/196] Loss 2.737 (2.737) Prec@1 10.94 (10.94) Prec@5 64.06 (64.06) VALID-SEARCH [2018-10-02-13:15:12] Epoch: [010][100/196] Loss 2.340 (2.261) Prec@1 11.72 (17.88) Prec@5 60.94 (70.10) VALID-SEARCH [2018-10-02-13:15:16] Epoch: [010][195/196] Loss 2.140 (2.271) Prec@1 20.00 (17.78) Prec@5 80.00 (70.73) 010/250, Train-Accuracy = 20.82, Test-Accuracy = 17.78 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-13:15:20] [Epoch=011/250] [Need: 08:12:17] [LR=0.0498 ~ 0.0498] [Batch=128], tau=9.736 genotype = Genotype(normal=[('sep_conv_3x3', 1, 0.12679289281368256), ('sep_conv_5x5', 0, 0.1253472864627838), ('sep_conv_5x5', 2, 0.12815827131271362), ('dil_conv_5x5', 1, 0.12717652320861816), ('sep_conv_3x3', 2, 0.12639999389648438), ('dil_conv_5x5', 1, 0.12619969248771667), ('sep_conv_5x5', 4, 0.1259656399488449), ('dil_conv_3x3', 0, 0.12593840062618256)], normal_concat=range(2, 6), reduce=[('sep_conv_5x5', 1, 0.12621204555034637), ('dil_conv_5x5', 0, 0.12570738792419434), ('sep_conv_5x5', 0, 0.12644100189208984), ('sep_conv_5x5', 2, 0.12618017196655273), ('sep_conv_5x5', 3, 0.1272910237312317), ('sep_conv_5x5', 2, 0.12631557881832123), ('dil_conv_5x5', 2, 0.1269507259130478), ('dil_conv_5x5', 3, 0.12678006291389465)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-13:15:22] Epoch: [011][000/196] Time 1.47 (1.47) Data 0.89 (0.89) Loss 2.091 (2.091) Prec@1 18.75 (18.75) Prec@5 72.66 (72.66) TRAIN-SEARCH [2018-10-02-13:16:20] Epoch: [011][100/196] Time 0.65 (0.60) Data 0.00 (0.01) Loss 2.065 (2.090) Prec@1 25.00 (22.12) Prec@5 76.56 (76.42) TRAIN-SEARCH [2018-10-02-13:17:11] Epoch: [011][195/196] Time 0.31 (0.57) Data 0.00 (0.01) Loss 2.131 (2.075) Prec@1 22.50 (21.90) Prec@5 72.50 (77.37) VALID-SEARCH [2018-10-02-13:17:12] Epoch: [011][000/196] Loss 2.144 (2.144) Prec@1 21.09 (21.09) Prec@5 75.00 (75.00) VALID-SEARCH [2018-10-02-13:17:17] Epoch: [011][100/196] Loss 2.162 (2.199) Prec@1 18.75 (17.59) Prec@5 75.78 (71.02) VALID-SEARCH [2018-10-02-13:17:20] Epoch: [011][195/196] Loss 1.968 (2.196) Prec@1 25.00 (18.00) Prec@5 82.50 (71.26) 011/250, Train-Accuracy = 21.90, Test-Accuracy = 18.00 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-13:17:21] [Epoch=012/250] [Need: 08:01:47] [LR=0.0497 ~ 0.0497] [Batch=128], tau=9.712 genotype = Genotype(normal=[('sep_conv_3x3', 1, 0.12673936784267426), ('sep_conv_5x5', 0, 0.12549738585948944), ('sep_conv_5x5', 2, 0.12823620438575745), ('dil_conv_5x5', 1, 0.12711022794246674), ('sep_conv_3x3', 2, 0.12636342644691467), ('dil_conv_5x5', 1, 0.12621590495109558), ('dil_conv_3x3', 0, 0.12620563805103302), ('sep_conv_5x5', 4, 0.12617214024066925)], normal_concat=range(2, 6), reduce=[('sep_conv_5x5', 1, 0.12622331082820892), ('dil_conv_5x5', 0, 0.12547604739665985), ('sep_conv_5x5', 0, 0.126180961728096), ('dil_conv_3x3', 2, 0.12594486773014069), ('sep_conv_5x5', 3, 0.12705475091934204), ('sep_conv_5x5', 2, 0.12612992525100708), ('dil_conv_5x5', 2, 0.1266750544309616), ('sep_conv_5x5', 3, 0.12656407058238983)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-13:17:23] Epoch: [012][000/196] Time 1.33 (1.33) Data 0.79 (0.79) Loss 2.133 (2.133) Prec@1 23.44 (23.44) Prec@5 76.56 (76.56) TRAIN-SEARCH [2018-10-02-13:18:20] Epoch: [012][100/196] Time 0.66 (0.58) Data 0.00 (0.01) Loss 2.133 (2.048) Prec@1 17.97 (22.21) Prec@5 82.03 (78.26) TRAIN-SEARCH [2018-10-02-13:19:11] Epoch: [012][195/196] Time 0.34 (0.56) Data 0.00 (0.01) Loss 1.976 (2.037) Prec@1 17.50 (22.70) Prec@5 75.00 (78.60) VALID-SEARCH [2018-10-02-13:19:12] Epoch: [012][000/196] Loss 2.017 (2.017) Prec@1 23.44 (23.44) Prec@5 78.12 (78.12) VALID-SEARCH [2018-10-02-13:19:16] Epoch: [012][100/196] Loss 2.133 (2.216) Prec@1 22.66 (18.64) Prec@5 74.22 (73.18) VALID-SEARCH [2018-10-02-13:19:20] Epoch: [012][195/196] Loss 1.924 (2.216) Prec@1 27.50 (18.50) Prec@5 77.50 (72.48) 012/250, Train-Accuracy = 22.70, Test-Accuracy = 18.50 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-13:19:21] [Epoch=013/250] [Need: 07:53:16] [LR=0.0497 ~ 0.0497] [Batch=128], tau=9.688 genotype = Genotype(normal=[('sep_conv_3x3', 1, 0.12674720585346222), ('dil_conv_3x3', 0, 0.12569476664066315), ('sep_conv_5x5', 2, 0.12828068435192108), ('dil_conv_5x5', 1, 0.1270330548286438), ('sep_conv_3x3', 2, 0.12645657360553741), ('dil_conv_5x5', 1, 0.1262192726135254), ('dil_conv_3x3', 0, 0.12635594606399536), ('sep_conv_5x5', 4, 0.12633873522281647)], normal_concat=range(2, 6), reduce=[('sep_conv_5x5', 1, 0.12608344852924347), ('sep_conv_5x5', 0, 0.1255914270877838), ('sep_conv_5x5', 0, 0.12600752711296082), ('dil_conv_3x3', 2, 0.12591619789600372), ('sep_conv_5x5', 3, 0.1268879920244217), ('sep_conv_5x5', 2, 0.1260501593351364), ('sep_conv_5x5', 3, 0.1264227032661438), ('sep_conv_3x3', 1, 0.12640605866909027)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-13:19:23] Epoch: [013][000/196] Time 1.52 (1.52) Data 0.91 (0.91) Loss 1.993 (1.993) Prec@1 27.34 (27.34) Prec@5 76.56 (76.56) TRAIN-SEARCH [2018-10-02-13:20:20] Epoch: [013][100/196] Time 0.46 (0.58) Data 0.00 (0.01) Loss 2.034 (2.020) Prec@1 22.66 (24.05) Prec@5 75.00 (79.19) TRAIN-SEARCH [2018-10-02-13:21:11] Epoch: [013][195/196] Time 0.35 (0.56) Data 0.00 (0.01) Loss 1.877 (2.003) Prec@1 22.50 (24.32) Prec@5 82.50 (79.77) VALID-SEARCH [2018-10-02-13:21:12] Epoch: [013][000/196] Loss 2.182 (2.182) Prec@1 18.75 (18.75) Prec@5 68.75 (68.75) VALID-SEARCH [2018-10-02-13:21:16] Epoch: [013][100/196] Loss 2.233 (2.196) Prec@1 18.75 (20.20) Prec@5 73.44 (74.40) VALID-SEARCH [2018-10-02-13:21:21] Epoch: [013][195/196] Loss 2.501 (2.209) Prec@1 22.50 (20.04) Prec@5 75.00 (74.50) 013/250, Train-Accuracy = 24.32, Test-Accuracy = 20.04 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-13:21:22] [Epoch=014/250] [Need: 07:57:03] [LR=0.0496 ~ 0.0496] [Batch=128], tau=9.664 genotype = Genotype(normal=[('sep_conv_3x3', 1, 0.12672759592533112), ('dil_conv_3x3', 0, 0.12580588459968567), ('sep_conv_5x5', 2, 0.12827417254447937), ('dil_conv_5x5', 1, 0.12702739238739014), ('sep_conv_3x3', 2, 0.12652121484279633), ('dil_conv_5x5', 1, 0.12615488469600677), ('dil_conv_3x3', 0, 0.12649787962436676), ('sep_conv_5x5', 4, 0.1264214664697647)], normal_concat=range(2, 6), reduce=[('sep_conv_5x5', 1, 0.1258910596370697), ('sep_conv_5x5', 0, 0.12555907666683197), ('dil_conv_3x3', 2, 0.1258062720298767), ('sep_conv_5x5', 1, 0.12562136352062225), ('sep_conv_5x5', 3, 0.12682408094406128), ('sep_conv_5x5', 2, 0.12585964798927307), ('sep_conv_3x3', 1, 0.12616592645645142), ('dil_conv_5x5', 2, 0.12604761123657227)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-13:21:24] Epoch: [014][000/196] Time 1.39 (1.39) Data 0.77 (0.77) Loss 1.901 (1.901) Prec@1 15.62 (15.62) Prec@5 84.38 (84.38) TRAIN-SEARCH [2018-10-02-13:22:20] Epoch: [014][100/196] Time 0.49 (0.56) Data 0.00 (0.01) Loss 1.968 (1.989) Prec@1 26.56 (24.57) Prec@5 83.59 (80.48) TRAIN-SEARCH [2018-10-02-13:23:13] Epoch: [014][195/196] Time 0.38 (0.56) Data 0.00 (0.01) Loss 1.989 (1.983) Prec@1 17.50 (24.55) Prec@5 80.00 (80.53) VALID-SEARCH [2018-10-02-13:23:13] Epoch: [014][000/196] Loss 2.031 (2.031) Prec@1 16.41 (16.41) Prec@5 78.12 (78.12) VALID-SEARCH [2018-10-02-13:23:18] Epoch: [014][100/196] Loss 2.059 (2.099) Prec@1 18.75 (21.18) Prec@5 79.69 (75.61) VALID-SEARCH [2018-10-02-13:23:23] Epoch: [014][195/196] Loss 1.971 (2.121) Prec@1 25.00 (21.00) Prec@5 80.00 (75.29) 014/250, Train-Accuracy = 24.55, Test-Accuracy = 21.00 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-13:23:24] [Epoch=015/250] [Need: 07:57:11] [LR=0.0496 ~ 0.0496] [Batch=128], tau=9.64 genotype = Genotype(normal=[('sep_conv_3x3', 1, 0.12672388553619385), ('dil_conv_3x3', 0, 0.1260216236114502), ('sep_conv_5x5', 2, 0.12828710675239563), ('dil_conv_5x5', 1, 0.1270388513803482), ('sep_conv_3x3', 2, 0.12662160396575928), ('sep_conv_5x5', 0, 0.12607480585575104), ('dil_conv_3x3', 0, 0.12679117918014526), ('sep_conv_5x5', 4, 0.12649571895599365)], normal_concat=range(2, 6), reduce=[('sep_conv_5x5', 1, 0.12563802301883698), ('sep_conv_5x5', 0, 0.1254882663488388), ('dil_conv_3x3', 2, 0.125748872756958), ('sep_conv_5x5', 1, 0.12539957463741302), ('sep_conv_5x5', 3, 0.1264212429523468), ('sep_conv_5x5', 2, 0.1257309913635254), ('sep_conv_3x3', 4, 0.12605655193328857), ('sep_conv_3x3', 1, 0.12605524063110352)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-13:23:26] Epoch: [015][000/196] Time 1.56 (1.56) Data 0.94 (0.94) Loss 2.006 (2.006) Prec@1 18.75 (18.75) Prec@5 77.34 (77.34) TRAIN-SEARCH [2018-10-02-13:24:21] Epoch: [015][100/196] Time 0.51 (0.56) Data 0.00 (0.01) Loss 1.963 (1.957) Prec@1 20.31 (25.97) Prec@5 83.59 (81.55) TRAIN-SEARCH [2018-10-02-13:25:13] Epoch: [015][195/196] Time 0.33 (0.55) Data 0.00 (0.01) Loss 2.144 (1.953) Prec@1 22.50 (26.11) Prec@5 75.00 (81.46) VALID-SEARCH [2018-10-02-13:25:13] Epoch: [015][000/196] Loss 2.742 (2.742) Prec@1 18.75 (18.75) Prec@5 75.00 (75.00) VALID-SEARCH [2018-10-02-13:25:19] Epoch: [015][100/196] Loss 2.296 (2.150) Prec@1 14.84 (20.64) Prec@5 71.88 (75.64) VALID-SEARCH [2018-10-02-13:25:24] Epoch: [015][195/196] Loss 1.928 (2.126) Prec@1 20.00 (21.04) Prec@5 90.00 (76.35) 015/250, Train-Accuracy = 26.11, Test-Accuracy = 21.04 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-13:25:25] [Epoch=016/250] [Need: 07:52:14] [LR=0.0495 ~ 0.0495] [Batch=128], tau=9.616 genotype = Genotype(normal=[('sep_conv_3x3', 1, 0.12662555277347565), ('dil_conv_3x3', 0, 0.12611648440361023), ('sep_conv_5x5', 2, 0.12828250229358673), ('dil_conv_5x5', 1, 0.1269814670085907), ('sep_conv_3x3', 2, 0.12654541432857513), ('sep_conv_5x5', 0, 0.12629784643650055), ('sep_conv_5x5', 4, 0.1265823096036911), ('sep_conv_3x3', 3, 0.12650781869888306)], normal_concat=range(2, 6), reduce=[('dil_conv_5x5', 0, 0.12554851174354553), ('avg_pool_3x3', 1, 0.1254633367061615), ('dil_conv_3x3', 2, 0.12560099363327026), ('dil_conv_5x5', 0, 0.12532445788383484), ('sep_conv_5x5', 3, 0.12623611092567444), ('sep_conv_5x5', 2, 0.12562960386276245), ('sep_conv_3x3', 4, 0.12584839761257172), ('dil_conv_5x5', 2, 0.12579099833965302)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-13:25:27] Epoch: [016][000/196] Time 1.45 (1.45) Data 0.79 (0.79) Loss 1.778 (1.778) Prec@1 30.47 (30.47) Prec@5 86.72 (86.72) TRAIN-SEARCH [2018-10-02-13:26:23] Epoch: [016][100/196] Time 0.54 (0.57) Data 0.01 (0.01) Loss 1.941 (1.936) Prec@1 23.44 (26.68) Prec@5 80.47 (82.14) TRAIN-SEARCH [2018-10-02-13:27:14] Epoch: [016][195/196] Time 0.30 (0.55) Data 0.00 (0.01) Loss 1.892 (1.930) Prec@1 30.00 (26.73) Prec@5 82.50 (82.61) VALID-SEARCH [2018-10-02-13:27:15] Epoch: [016][000/196] Loss 2.077 (2.077) Prec@1 14.06 (14.06) Prec@5 76.56 (76.56) VALID-SEARCH [2018-10-02-13:27:19] Epoch: [016][100/196] Loss 2.057 (2.123) Prec@1 21.88 (20.54) Prec@5 79.69 (75.28) VALID-SEARCH [2018-10-02-13:27:23] Epoch: [016][195/196] Loss 2.062 (2.123) Prec@1 25.00 (21.32) Prec@5 72.50 (76.02) 016/250, Train-Accuracy = 26.73, Test-Accuracy = 21.32 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-13:27:24] [Epoch=017/250] [Need: 07:43:24] [LR=0.0494 ~ 0.0494] [Batch=128], tau=9.592 genotype = Genotype(normal=[('sep_conv_3x3', 1, 0.12655946612358093), ('dil_conv_3x3', 0, 0.12642595171928406), ('sep_conv_5x5', 2, 0.12822367250919342), ('dil_conv_5x5', 1, 0.12688986957073212), ('sep_conv_3x3', 2, 0.12650179862976074), ('sep_conv_5x5', 0, 0.1263800859451294), ('sep_conv_3x3', 3, 0.12659746408462524), ('sep_conv_5x5', 4, 0.1264950931072235)], normal_concat=range(2, 6), reduce=[('dil_conv_5x5', 0, 0.12555725872516632), ('sep_conv_5x5', 1, 0.1255386769771576), ('dil_conv_5x5', 0, 0.12554161250591278), ('dil_conv_3x3', 2, 0.12543514370918274), ('sep_conv_5x5', 3, 0.1260625422000885), ('sep_conv_5x5', 2, 0.12561039626598358), ('sep_conv_5x5', 4, 0.1259082704782486), ('dil_conv_5x5', 2, 0.1257418692111969)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-13:27:27] Epoch: [017][000/196] Time 1.52 (1.52) Data 0.88 (0.88) Loss 1.871 (1.871) Prec@1 31.25 (31.25) Prec@5 84.38 (84.38) TRAIN-SEARCH [2018-10-02-13:28:23] Epoch: [017][100/196] Time 0.59 (0.58) Data 0.01 (0.01) Loss 1.719 (1.902) Prec@1 33.59 (28.16) Prec@5 89.06 (83.86) TRAIN-SEARCH [2018-10-02-13:29:14] Epoch: [017][195/196] Time 0.44 (0.56) Data 0.00 (0.01) Loss 1.712 (1.907) Prec@1 32.50 (27.82) Prec@5 90.00 (83.49) VALID-SEARCH [2018-10-02-13:29:16] Epoch: [017][000/196] Loss 1.936 (1.936) Prec@1 25.00 (25.00) Prec@5 85.94 (85.94) VALID-SEARCH [2018-10-02-13:29:21] Epoch: [017][100/196] Loss 2.093 (2.081) Prec@1 22.66 (22.43) Prec@5 78.12 (77.52) VALID-SEARCH [2018-10-02-13:29:26] Epoch: [017][195/196] Loss 2.464 (2.097) Prec@1 15.00 (22.13) Prec@5 70.00 (77.37) 017/250, Train-Accuracy = 27.82, Test-Accuracy = 22.13 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-13:29:27] [Epoch=018/250] [Need: 07:52:43] [LR=0.0494 ~ 0.0494] [Batch=128], tau=9.568 genotype = Genotype(normal=[('dil_conv_3x3', 0, 0.1264934241771698), ('sep_conv_3x3', 1, 0.12640976905822754), ('sep_conv_5x5', 2, 0.1282561719417572), ('dil_conv_5x5', 1, 0.12678544223308563), ('sep_conv_5x5', 0, 0.1264525204896927), ('sep_conv_3x3', 2, 0.1263822466135025), ('sep_conv_3x3', 3, 0.1265619695186615), ('sep_conv_5x5', 4, 0.12652356922626495)], normal_concat=range(2, 6), reduce=[('sep_conv_5x5', 1, 0.12543414533138275), ('dil_conv_5x5', 0, 0.12538334727287292), ('skip_connect', 0, 0.1254250407218933), ('sep_conv_5x5', 2, 0.1253846138715744), ('sep_conv_5x5', 3, 0.1258721798658371), ('dil_conv_5x5', 2, 0.1257539540529251), ('sep_conv_5x5', 4, 0.12572886049747467), ('sep_conv_5x5', 3, 0.12553907930850983)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-13:29:29] Epoch: [018][000/196] Time 1.33 (1.33) Data 0.83 (0.83) Loss 1.883 (1.883) Prec@1 25.00 (25.00) Prec@5 82.03 (82.03) TRAIN-SEARCH [2018-10-02-13:30:27] Epoch: [018][100/196] Time 0.51 (0.59) Data 0.00 (0.01) Loss 1.874 (1.889) Prec@1 25.78 (28.29) Prec@5 85.16 (83.87) TRAIN-SEARCH [2018-10-02-13:31:16] Epoch: [018][195/196] Time 0.56 (0.56) Data 0.00 (0.01) Loss 1.830 (1.891) Prec@1 32.50 (28.15) Prec@5 90.00 (83.88) VALID-SEARCH [2018-10-02-13:31:17] Epoch: [018][000/196] Loss 1.844 (1.844) Prec@1 29.69 (29.69) Prec@5 82.03 (82.03) VALID-SEARCH [2018-10-02-13:31:23] Epoch: [018][100/196] Loss 2.029 (2.078) Prec@1 26.56 (22.74) Prec@5 76.56 (78.91) VALID-SEARCH [2018-10-02-13:31:28] Epoch: [018][195/196] Loss 1.808 (2.079) Prec@1 35.00 (23.11) Prec@5 85.00 (78.34) 018/250, Train-Accuracy = 28.15, Test-Accuracy = 23.11 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-13:31:29] [Epoch=019/250] [Need: 07:51:03] [LR=0.0493 ~ 0.0493] [Batch=128], tau=9.544 genotype = Genotype(normal=[('dil_conv_3x3', 0, 0.1267625242471695), ('dil_conv_5x5', 1, 0.12625660002231598), ('sep_conv_5x5', 2, 0.1282491832971573), ('dil_conv_5x5', 1, 0.12683627009391785), ('sep_conv_5x5', 0, 0.12642383575439453), ('sep_conv_3x3', 2, 0.12641482055187225), ('sep_conv_5x5', 4, 0.12645702064037323), ('sep_conv_3x3', 3, 0.12641705572605133)], normal_concat=range(2, 6), reduce=[('sep_conv_5x5', 0, 0.12533405423164368), ('dil_conv_5x5', 1, 0.1253301501274109), ('max_pool_3x3', 0, 0.1253964751958847), ('dil_conv_3x3', 2, 0.12533853948116302), ('sep_conv_5x5', 3, 0.12579277157783508), ('avg_pool_3x3', 0, 0.12560515105724335), ('sep_conv_5x5', 4, 0.12564708292484283), ('sep_conv_5x5', 3, 0.12553450465202332)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-13:31:31] Epoch: [019][000/196] Time 1.38 (1.38) Data 0.80 (0.80) Loss 1.883 (1.883) Prec@1 32.81 (32.81) Prec@5 86.72 (86.72) TRAIN-SEARCH [2018-10-02-13:32:28] Epoch: [019][100/196] Time 0.53 (0.58) Data 0.02 (0.01) Loss 1.891 (1.852) Prec@1 32.81 (29.73) Prec@5 88.28 (85.06) TRAIN-SEARCH [2018-10-02-13:33:14] Epoch: [019][195/196] Time 0.53 (0.53) Data 0.00 (0.01) Loss 1.803 (1.858) Prec@1 32.50 (29.49) Prec@5 87.50 (84.87) VALID-SEARCH [2018-10-02-13:33:15] Epoch: [019][000/196] Loss 1.966 (1.966) Prec@1 21.88 (21.88) Prec@5 82.03 (82.03) VALID-SEARCH [2018-10-02-13:33:21] Epoch: [019][100/196] Loss 1.898 (2.075) Prec@1 31.25 (23.87) Prec@5 78.91 (78.28) VALID-SEARCH [2018-10-02-13:33:27] Epoch: [019][195/196] Loss 1.824 (2.073) Prec@1 30.00 (23.85) Prec@5 85.00 (77.85) 019/250, Train-Accuracy = 29.49, Test-Accuracy = 23.85 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-13:33:27] [Epoch=020/250] [Need: 07:33:55] [LR=0.0492 ~ 0.0492] [Batch=128], tau=9.52 genotype = Genotype(normal=[('dil_conv_3x3', 0, 0.1268116980791092), ('dil_conv_5x5', 1, 0.1261557638645172), ('sep_conv_5x5', 2, 0.1281689554452896), ('dil_conv_5x5', 1, 0.12677183747291565), ('sep_conv_3x3', 2, 0.12636271119117737), ('sep_conv_3x3', 3, 0.12620103359222412), ('sep_conv_5x5', 4, 0.1266223043203354), ('sep_conv_3x3', 3, 0.12641505897045135)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.12568674981594086), ('sep_conv_5x5', 1, 0.12546682357788086), ('dil_conv_3x3', 1, 0.12558749318122864), ('max_pool_3x3', 0, 0.12550513446331024), ('skip_connect', 0, 0.12573181092739105), ('dil_conv_5x5', 3, 0.12550048530101776), ('dil_conv_3x3', 4, 0.12559886276721954), ('dil_conv_3x3', 2, 0.12542510032653809)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-13:33:30] Epoch: [020][000/196] Time 1.41 (1.41) Data 0.85 (0.85) Loss 1.873 (1.873) Prec@1 24.22 (24.22) Prec@5 84.38 (84.38) TRAIN-SEARCH [2018-10-02-13:34:25] Epoch: [020][100/196] Time 0.51 (0.56) Data 0.00 (0.01) Loss 1.885 (1.859) Prec@1 28.91 (29.92) Prec@5 85.94 (84.69) TRAIN-SEARCH [2018-10-02-13:35:15] Epoch: [020][195/196] Time 0.52 (0.54) Data 0.00 (0.01) Loss 2.125 (1.850) Prec@1 15.00 (29.99) Prec@5 77.50 (84.67) VALID-SEARCH [2018-10-02-13:35:16] Epoch: [020][000/196] Loss 2.023 (2.023) Prec@1 23.44 (23.44) Prec@5 79.69 (79.69) VALID-SEARCH [2018-10-02-13:35:21] Epoch: [020][100/196] Loss 1.874 (2.064) Prec@1 28.91 (24.02) Prec@5 84.38 (78.05) VALID-SEARCH [2018-10-02-13:35:27] Epoch: [020][195/196] Loss 2.508 (2.089) Prec@1 22.50 (23.69) Prec@5 70.00 (78.07) 020/250, Train-Accuracy = 29.99, Test-Accuracy = 23.69 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-13:35:28] [Epoch=021/250] [Need: 07:38:42] [LR=0.0492 ~ 0.0492] [Batch=128], tau=9.496 genotype = Genotype(normal=[('dil_conv_3x3', 0, 0.12685538828372955), ('dil_conv_5x5', 1, 0.1262091100215912), ('sep_conv_5x5', 2, 0.1281188279390335), ('dil_conv_5x5', 1, 0.1267940253019333), ('sep_conv_3x3', 2, 0.12633246183395386), ('sep_conv_3x3', 3, 0.12627056241035461), ('sep_conv_5x5', 4, 0.12680639326572418), ('dil_conv_3x3', 0, 0.1263229250907898)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.12568256258964539), ('sep_conv_5x5', 1, 0.12542347609996796), ('dil_conv_5x5', 0, 0.1253511905670166), ('dil_conv_3x3', 2, 0.12534895539283752), ('avg_pool_3x3', 0, 0.12578749656677246), ('dil_conv_5x5', 3, 0.1256386637687683), ('dil_conv_3x3', 4, 0.12585680186748505), ('dil_conv_3x3', 2, 0.12555907666683197)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-13:35:30] Epoch: [021][000/196] Time 1.45 (1.45) Data 0.87 (0.87) Loss 1.958 (1.958) Prec@1 29.69 (29.69) Prec@5 80.47 (80.47) TRAIN-SEARCH [2018-10-02-13:36:25] Epoch: [021][100/196] Time 0.51 (0.56) Data 0.00 (0.01) Loss 1.731 (1.833) Prec@1 36.72 (30.61) Prec@5 86.72 (85.64) TRAIN-SEARCH [2018-10-02-13:37:16] Epoch: [021][195/196] Time 0.41 (0.55) Data 0.00 (0.01) Loss 1.911 (1.834) Prec@1 25.00 (30.80) Prec@5 80.00 (85.56) VALID-SEARCH [2018-10-02-13:37:17] Epoch: [021][000/196] Loss 2.049 (2.049) Prec@1 28.12 (28.12) Prec@5 77.34 (77.34) VALID-SEARCH [2018-10-02-13:37:23] Epoch: [021][100/196] Loss 2.333 (2.071) Prec@1 24.22 (25.15) Prec@5 79.69 (80.04) VALID-SEARCH [2018-10-02-13:37:28] Epoch: [021][195/196] Loss 2.661 (2.063) Prec@1 25.00 (24.76) Prec@5 72.50 (80.15) 021/250, Train-Accuracy = 30.80, Test-Accuracy = 24.76 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-13:37:29] [Epoch=022/250] [Need: 07:39:25] [LR=0.0491 ~ 0.0491] [Batch=128], tau=9.472 genotype = Genotype(normal=[('dil_conv_3x3', 0, 0.12678705155849457), ('dil_conv_5x5', 1, 0.1259479820728302), ('sep_conv_5x5', 2, 0.1280677169561386), ('dil_conv_5x5', 1, 0.12671460211277008), ('sep_conv_3x3', 2, 0.1262919157743454), ('sep_conv_3x3', 3, 0.12624797224998474), ('sep_conv_5x5', 4, 0.1268623173236847), ('sep_conv_3x3', 3, 0.1263791024684906)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.12563583254814148), ('sep_conv_5x5', 1, 0.12518912553787231), ('max_pool_3x3', 0, 0.12538398802280426), ('sep_conv_3x3', 2, 0.1253192275762558), ('sep_conv_5x5', 3, 0.12570157647132874), ('sep_conv_5x5', 2, 0.12554754316806793), ('dil_conv_5x5', 0, 0.1259019523859024), ('sep_conv_5x5', 3, 0.12566427886486053)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-13:37:31] Epoch: [022][000/196] Time 1.73 (1.73) Data 0.99 (0.99) Loss 1.793 (1.793) Prec@1 30.47 (30.47) Prec@5 82.03 (82.03) TRAIN-SEARCH [2018-10-02-13:38:25] Epoch: [022][100/196] Time 0.63 (0.55) Data 0.01 (0.01) Loss 1.985 (1.836) Prec@1 32.03 (30.56) Prec@5 80.47 (85.13) TRAIN-SEARCH [2018-10-02-13:39:15] Epoch: [022][195/196] Time 0.42 (0.54) Data 0.00 (0.01) Loss 1.607 (1.828) Prec@1 37.50 (30.68) Prec@5 90.00 (85.64) VALID-SEARCH [2018-10-02-13:39:16] Epoch: [022][000/196] Loss 3.596 (3.596) Prec@1 19.53 (19.53) Prec@5 69.53 (69.53) VALID-SEARCH [2018-10-02-13:39:22] Epoch: [022][100/196] Loss 1.966 (2.042) Prec@1 25.00 (25.09) Prec@5 79.69 (79.46) VALID-SEARCH [2018-10-02-13:39:27] Epoch: [022][195/196] Loss 1.890 (2.034) Prec@1 22.50 (24.88) Prec@5 80.00 (79.84) 022/250, Train-Accuracy = 30.68, Test-Accuracy = 24.88 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-13:39:28] [Epoch=023/250] [Need: 07:31:41] [LR=0.0490 ~ 0.0490] [Batch=128], tau=9.448 genotype = Genotype(normal=[('dil_conv_3x3', 0, 0.12672603130340576), ('dil_conv_5x5', 1, 0.1257937103509903), ('sep_conv_5x5', 2, 0.12795723974704742), ('dil_conv_5x5', 1, 0.12676231563091278), ('sep_conv_3x3', 3, 0.12623900175094604), ('sep_conv_3x3', 2, 0.1260557770729065), ('sep_conv_5x5', 4, 0.12656670808792114), ('sep_conv_3x3', 3, 0.12631089985370636)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.12549614906311035), ('dil_conv_3x3', 1, 0.12508761882781982), ('skip_connect', 0, 0.125804141163826), ('dil_conv_3x3', 2, 0.12541741132736206), ('dil_conv_5x5', 3, 0.1255376785993576), ('dil_conv_3x3', 2, 0.12545880675315857), ('avg_pool_3x3', 0, 0.12585976719856262), ('sep_conv_5x5', 3, 0.12578344345092773)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-13:39:30] Epoch: [023][000/196] Time 1.25 (1.25) Data 0.72 (0.72) Loss 1.793 (1.793) Prec@1 27.34 (27.34) Prec@5 87.50 (87.50) TRAIN-SEARCH [2018-10-02-13:40:26] Epoch: [023][100/196] Time 0.57 (0.57) Data 0.00 (0.01) Loss 1.749 (1.828) Prec@1 31.25 (31.02) Prec@5 85.94 (86.17) TRAIN-SEARCH [2018-10-02-13:41:15] Epoch: [023][195/196] Time 0.45 (0.54) Data 0.00 (0.01) Loss 1.768 (1.824) Prec@1 42.50 (31.33) Prec@5 92.50 (86.14) VALID-SEARCH [2018-10-02-13:41:16] Epoch: [023][000/196] Loss 2.063 (2.063) Prec@1 27.34 (27.34) Prec@5 73.44 (73.44) VALID-SEARCH [2018-10-02-13:41:21] Epoch: [023][100/196] Loss 1.715 (2.013) Prec@1 30.47 (26.35) Prec@5 87.50 (79.47) VALID-SEARCH [2018-10-02-13:41:26] Epoch: [023][195/196] Loss 2.182 (2.009) Prec@1 7.50 (25.84) Prec@5 82.50 (79.58) 023/250, Train-Accuracy = 31.33, Test-Accuracy = 25.84 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-13:41:27] [Epoch=024/250] [Need: 07:27:23] [LR=0.0489 ~ 0.0489] [Batch=128], tau=9.424 genotype = Genotype(normal=[('dil_conv_3x3', 0, 0.1265600174665451), ('dil_conv_5x5', 1, 0.12593089044094086), ('sep_conv_5x5', 2, 0.12784482538700104), ('dil_conv_5x5', 1, 0.12658169865608215), ('dil_conv_5x5', 0, 0.12648634612560272), ('sep_conv_3x3', 3, 0.12632529437541962), ('max_pool_3x3', 0, 0.1265164464712143), ('sep_conv_5x5', 4, 0.1264679878950119)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.1256103664636612), ('avg_pool_3x3', 1, 0.12527057528495789), ('skip_connect', 0, 0.12570668756961823), ('dil_conv_3x3', 2, 0.12543132901191711), ('avg_pool_3x3', 0, 0.12563540041446686), ('dil_conv_3x3', 2, 0.12549136579036713), ('sep_conv_5x5', 3, 0.12561796605587006), ('avg_pool_3x3', 0, 0.12560079991817474)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-13:41:29] Epoch: [024][000/196] Time 1.49 (1.49) Data 0.85 (0.85) Loss 1.816 (1.816) Prec@1 29.69 (29.69) Prec@5 88.28 (88.28) TRAIN-SEARCH [2018-10-02-13:42:27] Epoch: [024][100/196] Time 0.60 (0.59) Data 0.00 (0.01) Loss 1.823 (1.818) Prec@1 30.47 (31.16) Prec@5 89.06 (86.42) TRAIN-SEARCH [2018-10-02-13:43:17] Epoch: [024][195/196] Time 0.45 (0.56) Data 0.00 (0.01) Loss 1.656 (1.807) Prec@1 37.50 (31.67) Prec@5 90.00 (86.51) VALID-SEARCH [2018-10-02-13:43:18] Epoch: [024][000/196] Loss 1.999 (1.999) Prec@1 21.09 (21.09) Prec@5 78.91 (78.91) VALID-SEARCH [2018-10-02-13:43:24] Epoch: [024][100/196] Loss 2.022 (2.048) Prec@1 17.19 (23.99) Prec@5 82.03 (78.26) VALID-SEARCH [2018-10-02-13:43:30] Epoch: [024][195/196] Loss 2.016 (2.057) Prec@1 30.00 (23.97) Prec@5 75.00 (77.91) 024/250, Train-Accuracy = 31.67, Test-Accuracy = 23.97 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-13:43:31] [Epoch=025/250] [Need: 07:44:09] [LR=0.0488 ~ 0.0488] [Batch=128], tau=9.4 genotype = Genotype(normal=[('dil_conv_3x3', 0, 0.12699946761131287), ('dil_conv_5x5', 1, 0.1260296106338501), ('sep_conv_5x5', 2, 0.12760290503501892), ('dil_conv_5x5', 1, 0.12651371955871582), ('max_pool_3x3', 0, 0.12643498182296753), ('sep_conv_3x3', 3, 0.12642158567905426), ('max_pool_3x3', 0, 0.1271980106830597), ('dil_conv_5x5', 2, 0.12625862658023834)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.12573596835136414), ('dil_conv_5x5', 1, 0.12529413402080536), ('dil_conv_3x3', 2, 0.12564125657081604), ('max_pool_3x3', 1, 0.1255940943956375), ('dil_conv_3x3', 2, 0.12585808336734772), ('sep_conv_5x5', 1, 0.12568514049053192), ('sep_conv_5x5', 3, 0.1257760226726532), ('skip_connect', 0, 0.12550809979438782)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-13:43:33] Epoch: [025][000/196] Time 1.60 (1.60) Data 0.99 (0.99) Loss 1.756 (1.756) Prec@1 39.84 (39.84) Prec@5 85.16 (85.16) TRAIN-SEARCH [2018-10-02-13:44:31] Epoch: [025][100/196] Time 0.53 (0.59) Data 0.00 (0.01) Loss 1.792 (1.809) Prec@1 22.66 (31.80) Prec@5 89.06 (86.26) TRAIN-SEARCH [2018-10-02-13:45:19] Epoch: [025][195/196] Time 0.48 (0.55) Data 0.00 (0.01) Loss 1.885 (1.802) Prec@1 20.00 (32.10) Prec@5 82.50 (86.28) VALID-SEARCH [2018-10-02-13:45:20] Epoch: [025][000/196] Loss 2.049 (2.049) Prec@1 21.09 (21.09) Prec@5 78.12 (78.12) VALID-SEARCH [2018-10-02-13:45:26] Epoch: [025][100/196] Loss 1.916 (2.011) Prec@1 30.47 (26.26) Prec@5 81.25 (80.30) VALID-SEARCH [2018-10-02-13:45:31] Epoch: [025][195/196] Loss 1.613 (2.013) Prec@1 37.50 (26.63) Prec@5 87.50 (80.69) 025/250, Train-Accuracy = 32.10, Test-Accuracy = 26.63 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-13:45:32] [Epoch=026/250] [Need: 07:31:51] [LR=0.0487 ~ 0.0487] [Batch=128], tau=9.376 genotype = Genotype(normal=[('dil_conv_3x3', 0, 0.1270655393600464), ('dil_conv_5x5', 1, 0.12593628466129303), ('sep_conv_5x5', 2, 0.1274968832731247), ('skip_connect', 0, 0.12683482468128204), ('max_pool_3x3', 0, 0.126571923494339), ('sep_conv_5x5', 3, 0.1263914853334427), ('max_pool_3x3', 0, 0.12735699117183685), ('dil_conv_5x5', 4, 0.12642662227153778)], normal_concat=range(2, 6), reduce=[('skip_connect', 1, 0.12596428394317627), ('max_pool_3x3', 0, 0.12576985359191895), ('max_pool_3x3', 0, 0.12559589743614197), ('dil_conv_3x3', 2, 0.12550373375415802), ('sep_conv_5x5', 2, 0.125545471906662), ('dil_conv_5x5', 0, 0.12544620037078857), ('sep_conv_5x5', 3, 0.1258901208639145), ('dil_conv_5x5', 0, 0.12565138936042786)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-13:45:34] Epoch: [026][000/196] Time 1.48 (1.48) Data 0.89 (0.89) Loss 1.864 (1.864) Prec@1 32.03 (32.03) Prec@5 85.16 (85.16) TRAIN-SEARCH [2018-10-02-13:46:31] Epoch: [026][100/196] Time 0.57 (0.58) Data 0.00 (0.01) Loss 1.933 (1.789) Prec@1 28.91 (32.71) Prec@5 84.38 (86.73) TRAIN-SEARCH [2018-10-02-13:47:19] Epoch: [026][195/196] Time 0.50 (0.55) Data 0.00 (0.01) Loss 1.571 (1.773) Prec@1 32.50 (33.35) Prec@5 100.00 (86.82) VALID-SEARCH [2018-10-02-13:47:20] Epoch: [026][000/196] Loss 2.114 (2.114) Prec@1 19.53 (19.53) Prec@5 71.88 (71.88) VALID-SEARCH [2018-10-02-13:47:26] Epoch: [026][100/196] Loss 1.951 (2.076) Prec@1 31.25 (26.02) Prec@5 80.47 (80.17) VALID-SEARCH [2018-10-02-13:47:31] Epoch: [026][195/196] Loss 1.864 (2.046) Prec@1 27.50 (25.85) Prec@5 92.50 (80.68) 026/250, Train-Accuracy = 33.35, Test-Accuracy = 25.85 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-13:47:32] [Epoch=027/250] [Need: 07:26:37] [LR=0.0486 ~ 0.0486] [Batch=128], tau=9.352 genotype = Genotype(normal=[('dil_conv_3x3', 0, 0.12660863995552063), ('dil_conv_5x5', 1, 0.12606558203697205), ('sep_conv_5x5', 2, 0.12707164883613586), ('skip_connect', 0, 0.1266499161720276), ('max_pool_3x3', 0, 0.1271265745162964), ('sep_conv_5x5', 3, 0.1265392154455185), ('max_pool_3x3', 0, 0.12698189914226532), ('dil_conv_5x5', 4, 0.12658989429473877)], normal_concat=range(2, 6), reduce=[('dil_conv_5x5', 0, 0.1257818043231964), ('skip_connect', 1, 0.12537240982055664), ('max_pool_3x3', 0, 0.12612219154834747), ('dil_conv_3x3', 2, 0.1258193850517273), ('sep_conv_5x5', 2, 0.12563396990299225), ('max_pool_3x3', 0, 0.12555889785289764), ('sep_conv_3x3', 4, 0.12560665607452393), ('max_pool_3x3', 0, 0.12555067241191864)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-13:47:34] Epoch: [027][000/196] Time 1.65 (1.65) Data 1.00 (1.00) Loss 1.907 (1.907) Prec@1 25.78 (25.78) Prec@5 82.81 (82.81) TRAIN-SEARCH [2018-10-02-13:48:29] Epoch: [027][100/196] Time 0.49 (0.56) Data 0.00 (0.01) Loss 1.695 (1.779) Prec@1 35.94 (33.07) Prec@5 92.19 (86.88) TRAIN-SEARCH [2018-10-02-13:49:20] Epoch: [027][195/196] Time 0.52 (0.55) Data 0.00 (0.01) Loss 1.986 (1.769) Prec@1 35.00 (33.53) Prec@5 82.50 (87.00) VALID-SEARCH [2018-10-02-13:49:21] Epoch: [027][000/196] Loss 2.236 (2.236) Prec@1 30.47 (30.47) Prec@5 73.44 (73.44) VALID-SEARCH [2018-10-02-13:49:27] Epoch: [027][100/196] Loss 1.935 (1.988) Prec@1 23.44 (25.98) Prec@5 78.91 (80.64) VALID-SEARCH [2018-10-02-13:49:32] Epoch: [027][195/196] Loss 1.947 (1.993) Prec@1 27.50 (25.95) Prec@5 85.00 (80.53) 027/250, Train-Accuracy = 33.53, Test-Accuracy = 25.95 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-13:49:32] [Epoch=028/250] [Need: 07:25:47] [LR=0.0485 ~ 0.0485] [Batch=128], tau=9.328 genotype = Genotype(normal=[('dil_conv_3x3', 0, 0.12722329795360565), ('dil_conv_3x3', 1, 0.12620341777801514), ('skip_connect', 0, 0.12696145474910736), ('sep_conv_3x3', 2, 0.12690573930740356), ('max_pool_3x3', 0, 0.12711696326732635), ('sep_conv_5x5', 3, 0.12657196819782257), ('max_pool_3x3', 0, 0.1273016631603241), ('dil_conv_5x5', 4, 0.12696734070777893)], normal_concat=range(2, 6), reduce=[('dil_conv_5x5', 0, 0.12573041021823883), ('skip_connect', 1, 0.12544973194599152), ('max_pool_3x3', 0, 0.1261214315891266), ('sep_conv_5x5', 1, 0.12565314769744873), ('dil_conv_5x5', 0, 0.12568548321723938), ('sep_conv_5x5', 2, 0.12558691203594208), ('dil_conv_5x5', 2, 0.1261277198791504), ('dil_conv_5x5', 0, 0.12569589912891388)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-13:49:35] Epoch: [028][000/196] Time 1.70 (1.70) Data 1.00 (1.00) Loss 1.868 (1.868) Prec@1 33.59 (33.59) Prec@5 82.81 (82.81) TRAIN-SEARCH [2018-10-02-13:50:32] Epoch: [028][100/196] Time 0.65 (0.58) Data 0.00 (0.01) Loss 1.682 (1.775) Prec@1 35.16 (33.64) Prec@5 90.62 (87.14) TRAIN-SEARCH [2018-10-02-13:51:23] Epoch: [028][195/196] Time 0.41 (0.56) Data 0.00 (0.01) Loss 1.858 (1.778) Prec@1 30.00 (33.54) Prec@5 87.50 (87.18) VALID-SEARCH [2018-10-02-13:51:25] Epoch: [028][000/196] Loss 2.055 (2.055) Prec@1 28.12 (28.12) Prec@5 76.56 (76.56) VALID-SEARCH [2018-10-02-13:51:30] Epoch: [028][100/196] Loss 2.041 (1.901) Prec@1 27.34 (28.26) Prec@5 82.03 (83.00) VALID-SEARCH [2018-10-02-13:51:36] Epoch: [028][195/196] Loss 1.768 (1.934) Prec@1 27.50 (27.46) Prec@5 95.00 (82.42) 028/250, Train-Accuracy = 33.54, Test-Accuracy = 27.46 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-13:51:36] [Epoch=029/250] [Need: 07:37:44] [LR=0.0484 ~ 0.0484] [Batch=128], tau=9.304 genotype = Genotype(normal=[('dil_conv_3x3', 0, 0.12718285620212555), ('sep_conv_3x3', 1, 0.1260513961315155), ('skip_connect', 0, 0.1271076500415802), ('sep_conv_3x3', 2, 0.12706395983695984), ('max_pool_3x3', 0, 0.12694737315177917), ('sep_conv_3x3', 3, 0.12648749351501465), ('max_pool_3x3', 0, 0.12736891210079193), ('dil_conv_5x5', 4, 0.12690749764442444)], normal_concat=range(2, 6), reduce=[('dil_conv_5x5', 0, 0.12565462291240692), ('skip_connect', 1, 0.1252572387456894), ('max_pool_3x3', 0, 0.12571372091770172), ('sep_conv_5x5', 1, 0.12549546360969543), ('max_pool_3x3', 1, 0.12571115791797638), ('dil_conv_5x5', 0, 0.12563055753707886), ('max_pool_3x3', 0, 0.12558744847774506), ('dil_conv_3x3', 3, 0.12546011805534363)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-13:51:39] Epoch: [029][000/196] Time 1.55 (1.55) Data 1.01 (1.01) Loss 1.743 (1.743) Prec@1 34.38 (34.38) Prec@5 88.28 (88.28) TRAIN-SEARCH [2018-10-02-13:52:35] Epoch: [029][100/196] Time 0.60 (0.57) Data 0.03 (0.01) Loss 1.690 (1.744) Prec@1 39.06 (34.80) Prec@5 86.72 (87.52) TRAIN-SEARCH [2018-10-02-13:53:26] Epoch: [029][195/196] Time 0.45 (0.55) Data 0.00 (0.01) Loss 1.742 (1.760) Prec@1 27.50 (33.98) Prec@5 87.50 (87.42) VALID-SEARCH [2018-10-02-13:53:26] Epoch: [029][000/196] Loss 1.982 (1.982) Prec@1 17.19 (17.19) Prec@5 84.38 (84.38) VALID-SEARCH [2018-10-02-13:53:31] Epoch: [029][100/196] Loss 2.009 (1.994) Prec@1 17.19 (25.67) Prec@5 79.69 (79.48) VALID-SEARCH [2018-10-02-13:53:34] Epoch: [029][195/196] Loss 1.870 (1.959) Prec@1 32.50 (26.75) Prec@5 85.00 (80.83) 029/250, Train-Accuracy = 33.98, Test-Accuracy = 26.75 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-13:53:35] [Epoch=030/250] [Need: 07:15:08] [LR=0.0483 ~ 0.0483] [Batch=128], tau=9.28 genotype = Genotype(normal=[('sep_conv_3x3', 0, 0.1268475204706192), ('dil_conv_3x3', 1, 0.1258646696805954), ('sep_conv_3x3', 2, 0.12712369859218597), ('skip_connect', 0, 0.1265937089920044), ('sep_conv_3x3', 2, 0.12633191049098969), ('sep_conv_5x5', 3, 0.12626877427101135), ('max_pool_3x3', 0, 0.12695761024951935), ('dil_conv_5x5', 4, 0.12663577497005463)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1, 0.12559780478477478), ('dil_conv_5x5', 0, 0.1255929172039032), ('dil_conv_3x3', 0, 0.12563173472881317), ('dil_conv_3x3', 2, 0.1255674660205841), ('dil_conv_5x5', 0, 0.12622374296188354), ('dil_conv_5x5', 3, 0.12563572824001312), ('dil_conv_5x5', 0, 0.1258075088262558), ('dil_conv_3x3', 2, 0.12567093968391418)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-13:53:37] Epoch: [030][000/196] Time 1.45 (1.45) Data 0.83 (0.83) Loss 1.723 (1.723) Prec@1 39.84 (39.84) Prec@5 93.75 (93.75) TRAIN-SEARCH [2018-10-02-13:54:35] Epoch: [030][100/196] Time 0.53 (0.59) Data 0.00 (0.01) Loss 1.634 (1.754) Prec@1 37.50 (34.61) Prec@5 91.41 (87.79) TRAIN-SEARCH [2018-10-02-13:55:24] Epoch: [030][195/196] Time 0.55 (0.55) Data 0.00 (0.01) Loss 1.947 (1.749) Prec@1 40.00 (34.57) Prec@5 70.00 (87.86) VALID-SEARCH [2018-10-02-13:55:26] Epoch: [030][000/196] Loss 1.885 (1.885) Prec@1 27.34 (27.34) Prec@5 89.06 (89.06) VALID-SEARCH [2018-10-02-13:55:31] Epoch: [030][100/196] Loss 1.827 (1.992) Prec@1 32.81 (28.19) Prec@5 87.50 (82.89) VALID-SEARCH [2018-10-02-13:55:36] Epoch: [030][195/196] Loss 1.831 (1.966) Prec@1 42.50 (28.34) Prec@5 85.00 (83.04) 030/250, Train-Accuracy = 34.57, Test-Accuracy = 28.34 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-13:55:37] [Epoch=031/250] [Need: 07:24:05] [LR=0.0482 ~ 0.0482] [Batch=128], tau=9.256 genotype = Genotype(normal=[('dil_conv_3x3', 0, 0.12729087471961975), ('dil_conv_3x3', 1, 0.12579180300235748), ('sep_conv_3x3', 2, 0.1271471381187439), ('dil_conv_3x3', 0, 0.12618878483772278), ('sep_conv_3x3', 3, 0.12633256614208221), ('max_pool_3x3', 0, 0.12624605000019073), ('max_pool_3x3', 0, 0.1269114762544632), ('sep_conv_3x3', 4, 0.12644502520561218)], normal_concat=range(2, 6), reduce=[('sep_conv_5x5', 0, 0.12564143538475037), ('skip_connect', 1, 0.12528876960277557), ('max_pool_3x3', 0, 0.12585176527500153), ('dil_conv_3x3', 2, 0.12559406459331512), ('dil_conv_5x5', 3, 0.12584900856018066), ('skip_connect', 0, 0.1257934868335724), ('sep_conv_5x5', 2, 0.12573127448558807), ('dil_conv_5x5', 4, 0.12568217515945435)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-13:55:39] Epoch: [031][000/196] Time 1.64 (1.64) Data 0.93 (0.93) Loss 1.722 (1.722) Prec@1 36.72 (36.72) Prec@5 87.50 (87.50) TRAIN-SEARCH [2018-10-02-13:56:37] Epoch: [031][100/196] Time 0.65 (0.59) Data 0.00 (0.01) Loss 1.615 (1.745) Prec@1 38.28 (34.60) Prec@5 93.75 (87.76) TRAIN-SEARCH [2018-10-02-13:57:26] Epoch: [031][195/196] Time 0.45 (0.55) Data 0.00 (0.01) Loss 1.841 (1.743) Prec@1 25.00 (34.77) Prec@5 82.50 (87.69) VALID-SEARCH [2018-10-02-13:57:27] Epoch: [031][000/196] Loss 1.973 (1.973) Prec@1 32.81 (32.81) Prec@5 80.47 (80.47) VALID-SEARCH [2018-10-02-13:57:32] Epoch: [031][100/196] Loss 2.238 (1.973) Prec@1 20.31 (27.58) Prec@5 84.38 (81.71) VALID-SEARCH [2018-10-02-13:57:38] Epoch: [031][195/196] Loss 1.886 (1.958) Prec@1 22.50 (28.05) Prec@5 85.00 (81.91) 031/250, Train-Accuracy = 34.77, Test-Accuracy = 28.05 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-13:57:38] [Epoch=032/250] [Need: 07:21:16] [LR=0.0480 ~ 0.0480] [Batch=128], tau=9.232 genotype = Genotype(normal=[('dil_conv_5x5', 0, 0.12709544599056244), ('dil_conv_3x3', 1, 0.12578147649765015), ('sep_conv_3x3', 2, 0.12713505327701569), ('skip_connect', 0, 0.12713055312633514), ('sep_conv_3x3', 3, 0.12656067311763763), ('max_pool_3x3', 0, 0.12625734508037567), ('max_pool_3x3', 0, 0.12700946629047394), ('sep_conv_5x5', 3, 0.12658776342868805)], normal_concat=range(2, 6), reduce=[('dil_conv_5x5', 1, 0.12540508806705475), ('avg_pool_3x3', 0, 0.12528690695762634), ('sep_conv_3x3', 2, 0.12579160928726196), ('dil_conv_3x3', 0, 0.12578873336315155), ('dil_conv_5x5', 0, 0.12594769895076752), ('dil_conv_5x5', 3, 0.12547710537910461), ('dil_conv_5x5', 4, 0.1257501095533371), ('max_pool_3x3', 0, 0.12567074596881866)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-13:57:40] Epoch: [032][000/196] Time 1.41 (1.41) Data 0.83 (0.83) Loss 1.703 (1.703) Prec@1 42.19 (42.19) Prec@5 86.72 (86.72) TRAIN-SEARCH [2018-10-02-13:58:38] Epoch: [032][100/196] Time 0.53 (0.58) Data 0.00 (0.01) Loss 1.689 (1.722) Prec@1 34.38 (35.74) Prec@5 90.62 (87.96) TRAIN-SEARCH [2018-10-02-13:59:26] Epoch: [032][195/196] Time 0.50 (0.55) Data 0.00 (0.01) Loss 1.554 (1.717) Prec@1 47.50 (35.90) Prec@5 92.50 (88.04) VALID-SEARCH [2018-10-02-13:59:28] Epoch: [032][000/196] Loss 2.005 (2.005) Prec@1 29.69 (29.69) Prec@5 80.47 (80.47) VALID-SEARCH [2018-10-02-13:59:33] Epoch: [032][100/196] Loss 2.040 (1.940) Prec@1 24.22 (29.42) Prec@5 79.69 (82.85) VALID-SEARCH [2018-10-02-13:59:39] Epoch: [032][195/196] Loss 1.852 (1.939) Prec@1 37.50 (29.21) Prec@5 80.00 (82.92) 032/250, Train-Accuracy = 35.90, Test-Accuracy = 29.21 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-13:59:40] [Epoch=033/250] [Need: 07:18:45] [LR=0.0479 ~ 0.0479] [Batch=128], tau=9.208 genotype = Genotype(normal=[('dil_conv_3x3', 0, 0.12688128650188446), ('sep_conv_5x5', 1, 0.1256878525018692), ('skip_connect', 0, 0.12765958905220032), ('sep_conv_3x3', 2, 0.12745308876037598), ('sep_conv_3x3', 3, 0.12660855054855347), ('sep_conv_3x3', 2, 0.1264696568250656), ('sep_conv_5x5', 3, 0.12680406868457794), ('max_pool_3x3', 0, 0.126714289188385)], normal_concat=range(2, 6), reduce=[('sep_conv_5x5', 1, 0.12563906610012054), ('dil_conv_5x5', 0, 0.1252124309539795), ('skip_connect', 0, 0.1258697211742401), ('sep_conv_3x3', 2, 0.1254482865333557), ('avg_pool_3x3', 1, 0.12567728757858276), ('dil_conv_3x3', 2, 0.12564273178577423), ('max_pool_3x3', 0, 0.12556244432926178), ('dil_conv_5x5', 3, 0.12548570334911346)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-13:59:42] Epoch: [033][000/196] Time 1.57 (1.57) Data 0.98 (0.98) Loss 1.730 (1.730) Prec@1 35.94 (35.94) Prec@5 83.59 (83.59) TRAIN-SEARCH [2018-10-02-14:00:38] Epoch: [033][100/196] Time 0.44 (0.57) Data 0.00 (0.01) Loss 1.487 (1.744) Prec@1 46.09 (35.12) Prec@5 90.62 (87.62) TRAIN-SEARCH [2018-10-02-14:01:25] Epoch: [033][195/196] Time 0.43 (0.53) Data 0.00 (0.01) Loss 1.759 (1.736) Prec@1 30.00 (35.66) Prec@5 90.00 (87.82) VALID-SEARCH [2018-10-02-14:01:26] Epoch: [033][000/196] Loss 2.308 (2.308) Prec@1 24.22 (24.22) Prec@5 78.91 (78.91) VALID-SEARCH [2018-10-02-14:01:32] Epoch: [033][100/196] Loss 2.052 (1.916) Prec@1 27.34 (29.87) Prec@5 79.69 (83.36) VALID-SEARCH [2018-10-02-14:01:38] Epoch: [033][195/196] Loss 2.630 (1.928) Prec@1 12.50 (29.52) Prec@5 52.50 (82.88) 033/250, Train-Accuracy = 35.66, Test-Accuracy = 29.52 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-14:01:38] [Epoch=034/250] [Need: 07:06:56] [LR=0.0478 ~ 0.0478] [Batch=128], tau=9.184 genotype = Genotype(normal=[('skip_connect', 0, 0.1273014098405838), ('sep_conv_5x5', 1, 0.12634731829166412), ('skip_connect', 0, 0.1277579516172409), ('sep_conv_3x3', 2, 0.1275038719177246), ('sep_conv_3x3', 2, 0.1268472969532013), ('dil_conv_3x3', 3, 0.1267804056406021), ('sep_conv_3x3', 4, 0.126933753490448), ('skip_connect', 0, 0.12683583796024323)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1, 0.12557131052017212), ('avg_pool_3x3', 0, 0.1254531592130661), ('avg_pool_3x3', 0, 0.12620335817337036), ('dil_conv_3x3', 1, 0.12568090856075287), ('avg_pool_3x3', 0, 0.12564274668693542), ('sep_conv_5x5', 2, 0.1254485547542572), ('dil_conv_5x5', 4, 0.1256483495235443), ('max_pool_3x3', 0, 0.12564072012901306)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-14:01:41] Epoch: [034][000/196] Time 1.54 (1.54) Data 0.93 (0.93) Loss 1.628 (1.628) Prec@1 45.31 (45.31) Prec@5 90.62 (90.62) TRAIN-SEARCH [2018-10-02-14:02:36] Epoch: [034][100/196] Time 0.60 (0.56) Data 0.00 (0.01) Loss 1.649 (1.694) Prec@1 36.72 (37.24) Prec@5 87.50 (89.00) TRAIN-SEARCH [2018-10-02-14:03:27] Epoch: [034][195/196] Time 0.46 (0.55) Data 0.00 (0.01) Loss 1.621 (1.702) Prec@1 45.00 (36.58) Prec@5 90.00 (88.70) VALID-SEARCH [2018-10-02-14:03:28] Epoch: [034][000/196] Loss 2.037 (2.037) Prec@1 26.56 (26.56) Prec@5 78.91 (78.91) VALID-SEARCH [2018-10-02-14:03:34] Epoch: [034][100/196] Loss 2.157 (1.914) Prec@1 21.88 (30.04) Prec@5 75.78 (83.62) VALID-SEARCH [2018-10-02-14:03:39] Epoch: [034][195/196] Loss 1.902 (1.916) Prec@1 35.00 (30.07) Prec@5 92.50 (83.64) 034/250, Train-Accuracy = 36.58, Test-Accuracy = 30.07 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-14:03:40] [Epoch=035/250] [Need: 07:15:56] [LR=0.0477 ~ 0.0477] [Batch=128], tau=9.16 genotype = Genotype(normal=[('skip_connect', 0, 0.127327561378479), ('sep_conv_5x5', 1, 0.12572813034057617), ('sep_conv_3x3', 2, 0.1275634914636612), ('skip_connect', 0, 0.1272815763950348), ('dil_conv_3x3', 3, 0.1267998367547989), ('sep_conv_3x3', 2, 0.12631258368492126), ('skip_connect', 0, 0.1270373910665512), ('sep_conv_5x5', 4, 0.12685374915599823)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.12570138275623322), ('skip_connect', 1, 0.125393345952034), ('avg_pool_3x3', 0, 0.12647241353988647), ('avg_pool_3x3', 1, 0.12557366490364075), ('dil_conv_5x5', 0, 0.12582145631313324), ('dil_conv_5x5', 3, 0.12547406554222107), ('dil_conv_5x5', 0, 0.12607216835021973), ('max_pool_3x3', 1, 0.1257276087999344)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-14:03:42] Epoch: [035][000/196] Time 1.61 (1.61) Data 0.85 (0.85) Loss 1.610 (1.610) Prec@1 37.50 (37.50) Prec@5 87.50 (87.50) TRAIN-SEARCH [2018-10-02-14:04:36] Epoch: [035][100/196] Time 0.39 (0.55) Data 0.00 (0.01) Loss 1.609 (1.690) Prec@1 44.53 (37.19) Prec@5 90.62 (88.88) TRAIN-SEARCH [2018-10-02-14:05:25] Epoch: [035][195/196] Time 0.42 (0.53) Data 0.00 (0.01) Loss 1.898 (1.691) Prec@1 37.50 (37.21) Prec@5 82.50 (88.80) VALID-SEARCH [2018-10-02-14:05:26] Epoch: [035][000/196] Loss 1.803 (1.803) Prec@1 33.59 (33.59) Prec@5 82.81 (82.81) VALID-SEARCH [2018-10-02-14:05:32] Epoch: [035][100/196] Loss 2.241 (1.920) Prec@1 25.00 (29.63) Prec@5 82.81 (83.80) VALID-SEARCH [2018-10-02-14:05:37] Epoch: [035][195/196] Loss 1.943 (1.951) Prec@1 30.00 (29.63) Prec@5 77.50 (84.11) 035/250, Train-Accuracy = 37.21, Test-Accuracy = 29.63 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-14:05:37] [Epoch=036/250] [Need: 06:59:13] [LR=0.0475 ~ 0.0475] [Batch=128], tau=9.136 genotype = Genotype(normal=[('skip_connect', 1, 0.12632226943969727), ('sep_conv_3x3', 0, 0.12628445029258728), ('sep_conv_3x3', 2, 0.12772762775421143), ('skip_connect', 0, 0.12729674577713013), ('dil_conv_3x3', 3, 0.12664955854415894), ('skip_connect', 0, 0.12631572782993317), ('max_pool_3x3', 0, 0.1269676834344864), ('sep_conv_3x3', 3, 0.12656556069850922)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 1, 0.1263955980539322), ('sep_conv_5x5', 0, 0.12555119395256042), ('avg_pool_3x3', 0, 0.1260642409324646), ('max_pool_3x3', 1, 0.12540100514888763), ('dil_conv_5x5', 0, 0.12579528987407684), ('skip_connect', 2, 0.1252758651971817), ('dil_conv_5x5', 4, 0.12560003995895386), ('max_pool_3x3', 1, 0.1254786103963852)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-14:05:40] Epoch: [036][000/196] Time 1.75 (1.75) Data 0.89 (0.89) Loss 1.655 (1.655) Prec@1 34.38 (34.38) Prec@5 90.62 (90.62) TRAIN-SEARCH [2018-10-02-14:06:36] Epoch: [036][100/196] Time 0.44 (0.57) Data 0.00 (0.01) Loss 1.693 (1.697) Prec@1 42.19 (36.90) Prec@5 89.06 (88.40) TRAIN-SEARCH [2018-10-02-14:07:25] Epoch: [036][195/196] Time 0.50 (0.55) Data 0.00 (0.01) Loss 1.635 (1.677) Prec@1 40.00 (37.64) Prec@5 87.50 (88.85) VALID-SEARCH [2018-10-02-14:07:27] Epoch: [036][000/196] Loss 1.777 (1.777) Prec@1 28.12 (28.12) Prec@5 84.38 (84.38) VALID-SEARCH [2018-10-02-14:07:32] Epoch: [036][100/196] Loss 1.871 (1.976) Prec@1 28.12 (27.50) Prec@5 83.59 (80.61) VALID-SEARCH [2018-10-02-14:07:36] Epoch: [036][195/196] Loss 1.997 (2.005) Prec@1 22.50 (26.94) Prec@5 87.50 (79.83) 036/250, Train-Accuracy = 37.64, Test-Accuracy = 26.94 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-14:07:37] [Epoch=037/250] [Need: 07:04:50] [LR=0.0474 ~ 0.0474] [Batch=128], tau=9.112 genotype = Genotype(normal=[('sep_conv_3x3', 0, 0.1262814849615097), ('sep_conv_5x5', 1, 0.12593983113765717), ('sep_conv_3x3', 2, 0.1275593638420105), ('dil_conv_3x3', 0, 0.12695948779582977), ('sep_conv_3x3', 3, 0.1270795613527298), ('skip_connect', 0, 0.12641064822673798), ('max_pool_3x3', 0, 0.12752673029899597), ('sep_conv_3x3', 3, 0.12656636536121368)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 1, 0.1260027438402176), ('avg_pool_3x3', 0, 0.12559275329113007), ('max_pool_3x3', 0, 0.12633903324604034), ('dil_conv_5x5', 2, 0.12556849420070648), ('dil_conv_5x5', 0, 0.12624166905879974), ('dil_conv_3x3', 1, 0.12596261501312256), ('dil_conv_3x3', 0, 0.1258060336112976), ('avg_pool_3x3', 1, 0.12564076483249664)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-14:07:39] Epoch: [037][000/196] Time 1.34 (1.34) Data 0.84 (0.84) Loss 1.710 (1.710) Prec@1 35.94 (35.94) Prec@5 89.06 (89.06) TRAIN-SEARCH [2018-10-02-14:08:36] Epoch: [037][100/196] Time 0.38 (0.58) Data 0.00 (0.01) Loss 1.664 (1.674) Prec@1 39.84 (37.27) Prec@5 87.50 (89.22) TRAIN-SEARCH [2018-10-02-14:09:25] Epoch: [037][195/196] Time 0.44 (0.55) Data 0.00 (0.01) Loss 1.856 (1.658) Prec@1 32.50 (38.28) Prec@5 82.50 (89.36) VALID-SEARCH [2018-10-02-14:09:26] Epoch: [037][000/196] Loss 1.902 (1.902) Prec@1 30.47 (30.47) Prec@5 82.03 (82.03) VALID-SEARCH [2018-10-02-14:09:31] Epoch: [037][100/196] Loss 2.310 (1.952) Prec@1 19.53 (29.59) Prec@5 64.84 (83.44) VALID-SEARCH [2018-10-02-14:09:36] Epoch: [037][195/196] Loss 2.021 (1.931) Prec@1 30.00 (30.07) Prec@5 82.50 (83.64) 037/250, Train-Accuracy = 38.28, Test-Accuracy = 30.07 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-14:09:37] [Epoch=038/250] [Need: 07:04:31] [LR=0.0473 ~ 0.0473] [Batch=128], tau=9.088000000000001 genotype = Genotype(normal=[('skip_connect', 0, 0.126661017537117), ('skip_connect', 1, 0.12648551166057587), ('sep_conv_3x3', 2, 0.12733344733715057), ('dil_conv_3x3', 0, 0.12667104601860046), ('sep_conv_3x3', 3, 0.12727759778499603), ('max_pool_3x3', 1, 0.12697753310203552), ('max_pool_3x3', 0, 0.12711893022060394), ('dil_conv_5x5', 4, 0.1264931857585907)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.1257966160774231), ('avg_pool_3x3', 1, 0.12535780668258667), ('max_pool_3x3', 0, 0.12589240074157715), ('sep_conv_3x3', 2, 0.12566040456295013), ('dil_conv_5x5', 0, 0.1258503943681717), ('sep_conv_5x5', 2, 0.12569651007652283), ('max_pool_3x3', 0, 0.12545575201511383), ('sep_conv_3x3', 1, 0.1253938525915146)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-14:09:39] Epoch: [038][000/196] Time 1.52 (1.52) Data 0.89 (0.89) Loss 1.647 (1.647) Prec@1 38.28 (38.28) Prec@5 88.28 (88.28) TRAIN-SEARCH [2018-10-02-14:10:35] Epoch: [038][100/196] Time 0.41 (0.56) Data 0.00 (0.01) Loss 1.817 (1.662) Prec@1 32.03 (38.30) Prec@5 83.59 (89.34) TRAIN-SEARCH [2018-10-02-14:11:23] Epoch: [038][195/196] Time 0.39 (0.54) Data 0.00 (0.01) Loss 1.484 (1.658) Prec@1 37.50 (38.37) Prec@5 90.00 (89.43) VALID-SEARCH [2018-10-02-14:11:25] Epoch: [038][000/196] Loss 1.932 (1.932) Prec@1 28.91 (28.91) Prec@5 75.78 (75.78) VALID-SEARCH [2018-10-02-14:11:30] Epoch: [038][100/196] Loss 1.724 (1.993) Prec@1 32.03 (29.25) Prec@5 89.84 (81.81) VALID-SEARCH [2018-10-02-14:11:35] Epoch: [038][195/196] Loss 2.684 (1.955) Prec@1 25.00 (29.54) Prec@5 62.50 (82.19) 038/250, Train-Accuracy = 38.37, Test-Accuracy = 29.54 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-14:11:36] [Epoch=039/250] [Need: 06:56:40] [LR=0.0471 ~ 0.0471] [Batch=128], tau=9.064 genotype = Genotype(normal=[('dil_conv_3x3', 0, 0.1277935802936554), ('skip_connect', 1, 0.12685798108577728), ('sep_conv_3x3', 2, 0.12717989087104797), ('skip_connect', 0, 0.12708616256713867), ('sep_conv_3x3', 3, 0.1274063140153885), ('skip_connect', 0, 0.12734918296337128), ('max_pool_3x3', 0, 0.12715114653110504), ('sep_conv_3x3', 3, 0.12635372579097748)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.1257990300655365), ('sep_conv_5x5', 1, 0.12563028931617737), ('max_pool_3x3', 0, 0.12666520476341248), ('sep_conv_5x5', 2, 0.1257050484418869), ('max_pool_3x3', 1, 0.12586262822151184), ('avg_pool_3x3', 0, 0.12567037343978882), ('dil_conv_3x3', 0, 0.12618915736675262), ('sep_conv_5x5', 4, 0.12567077577114105)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-14:11:38] Epoch: [039][000/196] Time 1.50 (1.50) Data 0.90 (0.90) Loss 1.499 (1.499) Prec@1 43.75 (43.75) Prec@5 91.41 (91.41) TRAIN-SEARCH [2018-10-02-14:12:33] Epoch: [039][100/196] Time 0.47 (0.56) Data 0.00 (0.01) Loss 1.613 (1.654) Prec@1 39.06 (38.88) Prec@5 93.75 (89.55) TRAIN-SEARCH [2018-10-02-14:13:25] Epoch: [039][195/196] Time 0.45 (0.55) Data 0.00 (0.01) Loss 1.872 (1.650) Prec@1 30.00 (39.15) Prec@5 90.00 (89.80) VALID-SEARCH [2018-10-02-14:13:26] Epoch: [039][000/196] Loss 1.866 (1.866) Prec@1 32.03 (32.03) Prec@5 89.06 (89.06) VALID-SEARCH [2018-10-02-14:13:31] Epoch: [039][100/196] Loss 2.463 (1.999) Prec@1 16.41 (29.71) Prec@5 69.53 (82.71) VALID-SEARCH [2018-10-02-14:13:36] Epoch: [039][195/196] Loss 1.619 (1.994) Prec@1 45.00 (29.12) Prec@5 90.00 (81.96) 039/250, Train-Accuracy = 39.15, Test-Accuracy = 29.12 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-14:13:36] [Epoch=040/250] [Need: 07:01:59] [LR=0.0470 ~ 0.0470] [Batch=128], tau=9.04 genotype = Genotype(normal=[('dil_conv_3x3', 0, 0.12761270999908447), ('skip_connect', 1, 0.12618416547775269), ('dil_conv_3x3', 0, 0.12739874422550201), ('sep_conv_3x3', 2, 0.12662585079669952), ('skip_connect', 0, 0.12728828191757202), ('sep_conv_3x3', 3, 0.12708966434001923), ('max_pool_3x3', 0, 0.12681207060813904), ('dil_conv_5x5', 4, 0.1268056482076645)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.12610763311386108), ('sep_conv_5x5', 1, 0.12558595836162567), ('skip_connect', 1, 0.12572291493415833), ('avg_pool_3x3', 0, 0.1257145255804062), ('dil_conv_3x3', 3, 0.12596720457077026), ('max_pool_3x3', 0, 0.12592294812202454), ('dil_conv_3x3', 0, 0.12585563957691193), ('sep_conv_5x5', 3, 0.1256008893251419)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-14:13:39] Epoch: [040][000/196] Time 1.69 (1.69) Data 1.09 (1.09) Loss 1.657 (1.657) Prec@1 36.72 (36.72) Prec@5 91.41 (91.41) TRAIN-SEARCH [2018-10-02-14:14:34] Epoch: [040][100/196] Time 0.40 (0.56) Data 0.00 (0.01) Loss 1.627 (1.638) Prec@1 36.72 (39.81) Prec@5 89.06 (90.08) TRAIN-SEARCH [2018-10-02-14:15:25] Epoch: [040][195/196] Time 0.41 (0.55) Data 0.00 (0.01) Loss 1.540 (1.627) Prec@1 47.50 (40.20) Prec@5 90.00 (90.20) VALID-SEARCH [2018-10-02-14:15:26] Epoch: [040][000/196] Loss 4.115 (4.115) Prec@1 13.28 (13.28) Prec@5 62.50 (62.50) VALID-SEARCH [2018-10-02-14:15:32] Epoch: [040][100/196] Loss 1.946 (2.137) Prec@1 28.91 (31.85) Prec@5 87.50 (82.77) VALID-SEARCH [2018-10-02-14:15:37] Epoch: [040][195/196] Loss 2.558 (2.121) Prec@1 7.50 (31.30) Prec@5 57.50 (82.89) 040/250, Train-Accuracy = 40.20, Test-Accuracy = 31.30 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-14:15:38] [Epoch=041/250] [Need: 07:04:17] [LR=0.0468 ~ 0.0468] [Batch=128], tau=9.016 genotype = Genotype(normal=[('dil_conv_3x3', 0, 0.1278831660747528), ('sep_conv_5x5', 1, 0.12682747840881348), ('sep_conv_3x3', 2, 0.1272207349538803), ('dil_conv_3x3', 0, 0.12693117558956146), ('skip_connect', 0, 0.1273069828748703), ('sep_conv_3x3', 3, 0.12685419619083405), ('max_pool_3x3', 0, 0.1269095540046692), ('dil_conv_5x5', 4, 0.12689493596553802)], normal_concat=range(2, 6), reduce=[('sep_conv_5x5', 1, 0.12601228058338165), ('sep_conv_5x5', 0, 0.12561583518981934), ('skip_connect', 0, 0.1260806918144226), ('max_pool_3x3', 1, 0.1257157027721405), ('dil_conv_5x5', 0, 0.1257844865322113), ('sep_conv_3x3', 3, 0.1255757063627243), ('max_pool_3x3', 1, 0.12608104944229126), ('sep_conv_5x5', 4, 0.1258067637681961)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-14:15:40] Epoch: [041][000/196] Time 1.62 (1.62) Data 0.87 (0.87) Loss 1.576 (1.576) Prec@1 37.50 (37.50) Prec@5 89.06 (89.06) TRAIN-SEARCH [2018-10-02-14:16:32] Epoch: [041][100/196] Time 0.44 (0.53) Data 0.00 (0.01) Loss 1.688 (1.625) Prec@1 38.28 (40.06) Prec@5 85.94 (90.28) TRAIN-SEARCH [2018-10-02-14:17:26] Epoch: [041][195/196] Time 0.47 (0.54) Data 0.00 (0.01) Loss 1.501 (1.620) Prec@1 40.00 (40.36) Prec@5 92.50 (90.33) VALID-SEARCH [2018-10-02-14:17:27] Epoch: [041][000/196] Loss 1.652 (1.652) Prec@1 38.28 (38.28) Prec@5 91.41 (91.41) VALID-SEARCH [2018-10-02-14:17:32] Epoch: [041][100/196] Loss 1.969 (1.949) Prec@1 25.78 (30.14) Prec@5 78.91 (82.55) VALID-SEARCH [2018-10-02-14:17:37] Epoch: [041][195/196] Loss 1.495 (1.959) Prec@1 50.00 (30.16) Prec@5 92.50 (82.69) 041/250, Train-Accuracy = 40.36, Test-Accuracy = 30.16 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-14:17:38] [Epoch=042/250] [Need: 06:55:52] [LR=0.0467 ~ 0.0467] [Batch=128], tau=8.992 genotype = Genotype(normal=[('skip_connect', 0, 0.12802445888519287), ('sep_conv_5x5', 1, 0.12701497972011566), ('sep_conv_3x3', 2, 0.12798339128494263), ('dil_conv_3x3', 0, 0.12736265361309052), ('skip_connect', 0, 0.12759745121002197), ('sep_conv_3x3', 3, 0.1267886757850647), ('max_pool_3x3', 0, 0.1278257668018341), ('dil_conv_5x5', 4, 0.12695103883743286)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.12602782249450684), ('sep_conv_5x5', 1, 0.12578536570072174), ('skip_connect', 0, 0.12582077085971832), ('max_pool_3x3', 1, 0.12563923001289368), ('dil_conv_5x5', 1, 0.12584619224071503), ('max_pool_3x3', 0, 0.12560288608074188), ('sep_conv_5x5', 3, 0.12613871693611145), ('dil_conv_5x5', 2, 0.12595859169960022)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-14:17:40] Epoch: [042][000/196] Time 1.44 (1.44) Data 0.82 (0.82) Loss 1.699 (1.699) Prec@1 40.62 (40.62) Prec@5 88.28 (88.28) TRAIN-SEARCH [2018-10-02-14:18:33] Epoch: [042][100/196] Time 0.39 (0.53) Data 0.00 (0.01) Loss 1.577 (1.597) Prec@1 42.97 (41.25) Prec@5 89.84 (90.63) TRAIN-SEARCH [2018-10-02-14:19:28] Epoch: [042][195/196] Time 0.57 (0.56) Data 0.00 (0.01) Loss 1.674 (1.607) Prec@1 45.00 (40.93) Prec@5 90.00 (90.43) VALID-SEARCH [2018-10-02-14:19:29] Epoch: [042][000/196] Loss 1.792 (1.792) Prec@1 32.81 (32.81) Prec@5 90.62 (90.62) VALID-SEARCH [2018-10-02-14:19:34] Epoch: [042][100/196] Loss 1.977 (1.964) Prec@1 33.59 (29.82) Prec@5 82.81 (82.58) VALID-SEARCH [2018-10-02-14:19:40] Epoch: [042][195/196] Loss 2.137 (1.947) Prec@1 30.00 (30.43) Prec@5 80.00 (82.63) 042/250, Train-Accuracy = 40.93, Test-Accuracy = 30.43 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-14:19:40] [Epoch=043/250] [Need: 07:02:14] [LR=0.0465 ~ 0.0465] [Batch=128], tau=8.968 genotype = Genotype(normal=[('skip_connect', 0, 0.12762729823589325), ('dil_conv_3x3', 1, 0.12620407342910767), ('sep_conv_3x3', 2, 0.1273355782032013), ('skip_connect', 0, 0.1266930252313614), ('skip_connect', 0, 0.12783282995224), ('dil_conv_3x3', 3, 0.12691481411457062), ('max_pool_3x3', 0, 0.12710967659950256), ('dil_conv_5x5', 4, 0.12647855281829834)], normal_concat=range(2, 6), reduce=[('skip_connect', 1, 0.12611140310764313), ('skip_connect', 0, 0.12566915154457092), ('avg_pool_3x3', 0, 0.1260625571012497), ('dil_conv_3x3', 2, 0.12571381032466888), ('dil_conv_5x5', 1, 0.12591102719306946), ('avg_pool_3x3', 0, 0.12582911550998688), ('sep_conv_3x3', 0, 0.12611109018325806), ('sep_conv_5x5', 4, 0.12586989998817444)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-14:19:43] Epoch: [043][000/196] Time 1.53 (1.53) Data 0.92 (0.92) Loss 1.529 (1.529) Prec@1 44.53 (44.53) Prec@5 89.06 (89.06) TRAIN-SEARCH [2018-10-02-14:20:33] Epoch: [043][100/196] Time 0.50 (0.52) Data 0.00 (0.01) Loss 1.593 (1.591) Prec@1 42.19 (41.58) Prec@5 89.84 (90.51) TRAIN-SEARCH [2018-10-02-14:21:28] Epoch: [043][195/196] Time 0.40 (0.54) Data 0.00 (0.01) Loss 1.653 (1.589) Prec@1 40.00 (41.69) Prec@5 87.50 (90.68) VALID-SEARCH [2018-10-02-14:21:29] Epoch: [043][000/196] Loss 2.177 (2.177) Prec@1 29.69 (29.69) Prec@5 85.16 (85.16) VALID-SEARCH [2018-10-02-14:21:34] Epoch: [043][100/196] Loss 2.179 (2.023) Prec@1 28.91 (32.19) Prec@5 90.62 (85.06) VALID-SEARCH [2018-10-02-14:21:39] Epoch: [043][195/196] Loss 1.760 (1.978) Prec@1 32.50 (32.75) Prec@5 80.00 (85.26) 043/250, Train-Accuracy = 41.69, Test-Accuracy = 32.75 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-14:21:40] [Epoch=044/250] [Need: 06:49:07] [LR=0.0463 ~ 0.0463] [Batch=128], tau=8.943999999999999 genotype = Genotype(normal=[('skip_connect', 0, 0.1281307339668274), ('skip_connect', 1, 0.126241073012352), ('sep_conv_3x3', 2, 0.12737950682640076), ('skip_connect', 1, 0.12695293128490448), ('skip_connect', 0, 0.12800517678260803), ('dil_conv_3x3', 3, 0.12712997198104858), ('skip_connect', 0, 0.12706005573272705), ('max_pool_3x3', 1, 0.12656384706497192)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.12587447464466095), ('avg_pool_3x3', 1, 0.12532703578472137), ('dil_conv_3x3', 2, 0.1263953149318695), ('max_pool_3x3', 1, 0.1258106827735901), ('sep_conv_5x5', 0, 0.12583670020103455), ('dil_conv_3x3', 2, 0.12565556168556213), ('sep_conv_5x5', 1, 0.12622375786304474), ('max_pool_3x3', 0, 0.12618635594844818)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-14:21:42] Epoch: [044][000/196] Time 1.51 (1.51) Data 0.86 (0.86) Loss 1.400 (1.400) Prec@1 47.66 (47.66) Prec@5 91.41 (91.41) TRAIN-SEARCH [2018-10-02-14:22:33] Epoch: [044][100/196] Time 0.65 (0.52) Data 0.00 (0.01) Loss 1.540 (1.588) Prec@1 43.75 (41.54) Prec@5 90.62 (90.48) TRAIN-SEARCH [2018-10-02-14:23:27] Epoch: [044][195/196] Time 0.48 (0.54) Data 0.00 (0.01) Loss 1.527 (1.588) Prec@1 45.00 (41.68) Prec@5 90.00 (90.46) VALID-SEARCH [2018-10-02-14:23:28] Epoch: [044][000/196] Loss 1.829 (1.829) Prec@1 28.12 (28.12) Prec@5 83.59 (83.59) VALID-SEARCH [2018-10-02-14:23:33] Epoch: [044][100/196] Loss 1.996 (1.926) Prec@1 29.69 (31.18) Prec@5 83.59 (82.98) VALID-SEARCH [2018-10-02-14:23:38] Epoch: [044][195/196] Loss 1.369 (1.905) Prec@1 57.50 (32.24) Prec@5 97.50 (83.66) 044/250, Train-Accuracy = 41.68, Test-Accuracy = 32.24 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-14:23:38] [Epoch=045/250] [Need: 06:45:51] [LR=0.0462 ~ 0.0462] [Batch=128], tau=8.92 genotype = Genotype(normal=[('dil_conv_3x3', 0, 0.12717869877815247), ('skip_connect', 1, 0.12644295394420624), ('sep_conv_3x3', 2, 0.12723663449287415), ('dil_conv_3x3', 0, 0.12639980018138885), ('skip_connect', 0, 0.1287894994020462), ('dil_conv_3x3', 3, 0.1285201758146286), ('sep_conv_3x3', 4, 0.12666986882686615), ('dil_conv_3x3', 2, 0.12628625333309174)], normal_concat=range(2, 6), reduce=[('sep_conv_5x5', 1, 0.12579026818275452), ('avg_pool_3x3', 0, 0.1254691183567047), ('avg_pool_3x3', 0, 0.12593050301074982), ('sep_conv_5x5', 2, 0.12559868395328522), ('dil_conv_5x5', 0, 0.1258004605770111), ('dil_conv_3x3', 2, 0.1257365345954895), ('max_pool_3x3', 0, 0.12602770328521729), ('max_pool_3x3', 1, 0.1260271817445755)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-14:23:40] Epoch: [045][000/196] Time 1.13 (1.13) Data 0.68 (0.68) Loss 1.690 (1.690) Prec@1 35.94 (35.94) Prec@5 88.28 (88.28) TRAIN-SEARCH [2018-10-02-14:24:33] Epoch: [045][100/196] Time 0.54 (0.54) Data 0.00 (0.01) Loss 1.484 (1.569) Prec@1 42.97 (42.83) Prec@5 93.75 (90.40) TRAIN-SEARCH [2018-10-02-14:25:26] Epoch: [045][195/196] Time 0.38 (0.54) Data 0.00 (0.01) Loss 1.549 (1.568) Prec@1 35.00 (42.66) Prec@5 90.00 (90.84) VALID-SEARCH [2018-10-02-14:25:27] Epoch: [045][000/196] Loss 2.105 (2.105) Prec@1 25.00 (25.00) Prec@5 73.44 (73.44) VALID-SEARCH [2018-10-02-14:25:32] Epoch: [045][100/196] Loss 1.960 (1.887) Prec@1 29.69 (33.18) Prec@5 74.22 (84.60) VALID-SEARCH [2018-10-02-14:25:37] Epoch: [045][195/196] Loss 1.851 (1.882) Prec@1 30.00 (33.80) Prec@5 82.50 (85.05) 045/250, Train-Accuracy = 42.66, Test-Accuracy = 33.80 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-14:25:38] [Epoch=046/250] [Need: 06:45:34] [LR=0.0460 ~ 0.0460] [Batch=128], tau=8.896 genotype = Genotype(normal=[('skip_connect', 0, 0.12768863141536713), ('dil_conv_5x5', 1, 0.1267116814851761), ('sep_conv_3x3', 2, 0.12734656035900116), ('skip_connect', 0, 0.12703466415405273), ('skip_connect', 0, 0.12857481837272644), ('dil_conv_3x3', 3, 0.12804961204528809), ('sep_conv_3x3', 4, 0.1269940286874771), ('max_pool_3x3', 1, 0.12672778964042664)], normal_concat=range(2, 6), reduce=[('sep_conv_5x5', 0, 0.12586495280265808), ('dil_conv_3x3', 1, 0.12544244527816772), ('avg_pool_3x3', 0, 0.1261807233095169), ('sep_conv_5x5', 2, 0.1261158436536789), ('skip_connect', 0, 0.1257586032152176), ('max_pool_3x3', 1, 0.125563845038414), ('max_pool_3x3', 1, 0.12592822313308716), ('sep_conv_3x3', 4, 0.1256886124610901)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-14:25:40] Epoch: [046][000/196] Time 1.68 (1.68) Data 0.92 (0.92) Loss 1.497 (1.497) Prec@1 38.28 (38.28) Prec@5 91.41 (91.41) TRAIN-SEARCH [2018-10-02-14:26:33] Epoch: [046][100/196] Time 0.66 (0.55) Data 0.01 (0.01) Loss 1.555 (1.541) Prec@1 46.09 (43.19) Prec@5 92.19 (91.48) TRAIN-SEARCH [2018-10-02-14:27:26] Epoch: [046][195/196] Time 0.52 (0.55) Data 0.00 (0.01) Loss 1.778 (1.549) Prec@1 42.50 (43.14) Prec@5 85.00 (91.28) VALID-SEARCH [2018-10-02-14:27:27] Epoch: [046][000/196] Loss 1.722 (1.722) Prec@1 37.50 (37.50) Prec@5 86.72 (86.72) VALID-SEARCH [2018-10-02-14:27:33] Epoch: [046][100/196] Loss 1.893 (1.899) Prec@1 32.03 (33.26) Prec@5 78.91 (84.53) VALID-SEARCH [2018-10-02-14:27:38] Epoch: [046][195/196] Loss 1.830 (1.871) Prec@1 37.50 (33.45) Prec@5 82.50 (84.58) 046/250, Train-Accuracy = 43.14, Test-Accuracy = 33.45 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-14:27:39] [Epoch=047/250] [Need: 06:50:56] [LR=0.0458 ~ 0.0458] [Batch=128], tau=8.872 genotype = Genotype(normal=[('skip_connect', 0, 0.12739211320877075), ('dil_conv_3x3', 1, 0.12714417278766632), ('sep_conv_3x3', 2, 0.1273619830608368), ('skip_connect', 0, 0.12698058784008026), ('skip_connect', 0, 0.12913332879543304), ('sep_conv_3x3', 3, 0.12696245312690735), ('sep_conv_3x3', 4, 0.1269596964120865), ('sep_conv_3x3', 3, 0.1269286572933197)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.12588255107402802), ('sep_conv_5x5', 1, 0.1251271665096283), ('avg_pool_3x3', 0, 0.12652526795864105), ('dil_conv_3x3', 2, 0.12590648233890533), ('skip_connect', 0, 0.12618958950042725), ('dil_conv_3x3', 2, 0.12583734095096588), ('max_pool_3x3', 1, 0.12616732716560364), ('max_pool_3x3', 3, 0.12611418962478638)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-14:27:41] Epoch: [047][000/196] Time 1.64 (1.64) Data 0.89 (0.89) Loss 1.723 (1.723) Prec@1 36.72 (36.72) Prec@5 90.62 (90.62) TRAIN-SEARCH [2018-10-02-14:28:34] Epoch: [047][100/196] Time 0.53 (0.53) Data 0.00 (0.01) Loss 1.552 (1.537) Prec@1 42.19 (43.75) Prec@5 89.06 (91.48) TRAIN-SEARCH [2018-10-02-14:29:25] Epoch: [047][195/196] Time 0.46 (0.54) Data 0.00 (0.01) Loss 1.278 (1.544) Prec@1 55.00 (43.08) Prec@5 97.50 (91.40) VALID-SEARCH [2018-10-02-14:29:26] Epoch: [047][000/196] Loss 2.338 (2.338) Prec@1 29.69 (29.69) Prec@5 85.94 (85.94) VALID-SEARCH [2018-10-02-14:29:32] Epoch: [047][100/196] Loss 1.795 (1.893) Prec@1 34.38 (34.44) Prec@5 84.38 (84.78) VALID-SEARCH [2018-10-02-14:29:37] Epoch: [047][195/196] Loss 1.439 (1.892) Prec@1 47.50 (34.15) Prec@5 95.00 (84.62) 047/250, Train-Accuracy = 43.08, Test-Accuracy = 34.15 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-14:29:38] [Epoch=048/250] [Need: 06:38:45] [LR=0.0457 ~ 0.0457] [Batch=128], tau=8.847999999999999 genotype = Genotype(normal=[('dil_conv_3x3', 1, 0.12783581018447876), ('dil_conv_5x5', 0, 0.12712734937667847), ('skip_connect', 0, 0.12721170485019684), ('sep_conv_3x3', 2, 0.12693342566490173), ('skip_connect', 0, 0.1285657286643982), ('dil_conv_3x3', 3, 0.12696883082389832), ('sep_conv_3x3', 4, 0.12739010155200958), ('sep_conv_3x3', 3, 0.1271432787179947)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 1, 0.1256658285856247), ('avg_pool_3x3', 0, 0.12559565901756287), ('avg_pool_3x3', 0, 0.1260775923728943), ('dil_conv_3x3', 1, 0.12578807771205902), ('max_pool_3x3', 1, 0.126287043094635), ('skip_connect', 0, 0.12578988075256348), ('max_pool_3x3', 0, 0.1260533481836319), ('max_pool_3x3', 3, 0.12579558789730072)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-14:29:40] Epoch: [048][000/196] Time 1.55 (1.55) Data 0.94 (0.94) Loss 1.549 (1.549) Prec@1 42.19 (42.19) Prec@5 89.84 (89.84) TRAIN-SEARCH [2018-10-02-14:30:33] Epoch: [048][100/196] Time 0.48 (0.54) Data 0.01 (0.01) Loss 1.608 (1.531) Prec@1 50.00 (43.80) Prec@5 91.41 (91.72) TRAIN-SEARCH [2018-10-02-14:31:26] Epoch: [048][195/196] Time 0.47 (0.55) Data 0.00 (0.01) Loss 1.447 (1.535) Prec@1 42.50 (43.80) Prec@5 97.50 (91.60) VALID-SEARCH [2018-10-02-14:31:28] Epoch: [048][000/196] Loss 2.070 (2.070) Prec@1 23.44 (23.44) Prec@5 82.03 (82.03) VALID-SEARCH [2018-10-02-14:31:33] Epoch: [048][100/196] Loss 2.007 (1.992) Prec@1 38.28 (33.37) Prec@5 78.91 (83.61) VALID-SEARCH [2018-10-02-14:31:38] Epoch: [048][195/196] Loss 1.842 (1.973) Prec@1 40.00 (33.62) Prec@5 90.00 (83.89) 048/250, Train-Accuracy = 43.80, Test-Accuracy = 33.62 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-14:31:39] [Epoch=049/250] [Need: 06:46:20] [LR=0.0455 ~ 0.0455] [Batch=128], tau=8.824 genotype = Genotype(normal=[('skip_connect', 0, 0.1274995356798172), ('dil_conv_3x3', 1, 0.12731052935123444), ('sep_conv_3x3', 2, 0.12700629234313965), ('max_pool_3x3', 0, 0.12680833041667938), ('max_pool_3x3', 0, 0.1279073804616928), ('sep_conv_3x3', 3, 0.12721353769302368), ('sep_conv_3x3', 4, 0.12747865915298462), ('sep_conv_5x5', 2, 0.12649603188037872)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.1260007619857788), ('max_pool_3x3', 1, 0.12552407383918762), ('avg_pool_3x3', 0, 0.1265660673379898), ('sep_conv_3x3', 2, 0.12539029121398926), ('max_pool_3x3', 1, 0.12631277740001678), ('dil_conv_3x3', 3, 0.12598901987075806), ('dil_conv_5x5', 3, 0.1263381540775299), ('max_pool_3x3', 0, 0.1262524425983429)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-14:31:41] Epoch: [049][000/196] Time 1.46 (1.46) Data 0.87 (0.87) Loss 1.590 (1.590) Prec@1 39.06 (39.06) Prec@5 90.62 (90.62) TRAIN-SEARCH [2018-10-02-14:32:32] Epoch: [049][100/196] Time 0.45 (0.52) Data 0.00 (0.01) Loss 1.375 (1.508) Prec@1 47.66 (44.08) Prec@5 92.97 (91.84) TRAIN-SEARCH [2018-10-02-14:33:26] Epoch: [049][195/196] Time 0.46 (0.54) Data 0.00 (0.01) Loss 1.371 (1.528) Prec@1 47.50 (43.56) Prec@5 87.50 (91.46) VALID-SEARCH [2018-10-02-14:33:27] Epoch: [049][000/196] Loss 1.738 (1.738) Prec@1 35.94 (35.94) Prec@5 85.16 (85.16) VALID-SEARCH [2018-10-02-14:33:33] Epoch: [049][100/196] Loss 1.633 (1.871) Prec@1 41.41 (33.70) Prec@5 87.50 (84.79) VALID-SEARCH [2018-10-02-14:33:39] Epoch: [049][195/196] Loss 1.764 (1.891) Prec@1 27.50 (33.76) Prec@5 95.00 (84.81) 049/250, Train-Accuracy = 43.56, Test-Accuracy = 33.76 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-14:33:40] [Epoch=050/250] [Need: 06:43:04] [LR=0.0453 ~ 0.0453] [Batch=128], tau=8.8 genotype = Genotype(normal=[('skip_connect', 0, 0.12771758437156677), ('dil_conv_3x3', 1, 0.12684951722621918), ('sep_conv_3x3', 2, 0.12735038995742798), ('skip_connect', 0, 0.12703320384025574), ('max_pool_3x3', 0, 0.1279393583536148), ('sep_conv_5x5', 3, 0.12747295200824738), ('sep_conv_3x3', 4, 0.1269269585609436), ('dil_conv_3x3', 3, 0.12686967849731445)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.1262548267841339), ('sep_conv_3x3', 1, 0.1256202757358551), ('dil_conv_5x5', 0, 0.1259613037109375), ('sep_conv_5x5', 1, 0.12553519010543823), ('dil_conv_5x5', 2, 0.12564896047115326), ('avg_pool_3x3', 0, 0.12559360265731812), ('dil_conv_5x5', 3, 0.12620700895786285), ('sep_conv_5x5', 4, 0.12593169510364532)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-14:33:42] Epoch: [050][000/196] Time 1.52 (1.52) Data 1.02 (1.02) Loss 1.559 (1.559) Prec@1 45.31 (45.31) Prec@5 88.28 (88.28) TRAIN-SEARCH [2018-10-02-14:34:34] Epoch: [050][100/196] Time 0.45 (0.53) Data 0.00 (0.01) Loss 1.399 (1.521) Prec@1 45.31 (43.89) Prec@5 90.62 (91.95) TRAIN-SEARCH [2018-10-02-14:35:28] Epoch: [050][195/196] Time 0.46 (0.55) Data 0.00 (0.01) Loss 1.699 (1.523) Prec@1 37.50 (43.89) Prec@5 85.00 (91.74) VALID-SEARCH [2018-10-02-14:35:29] Epoch: [050][000/196] Loss 1.890 (1.890) Prec@1 21.09 (21.09) Prec@5 89.06 (89.06) VALID-SEARCH [2018-10-02-14:35:34] Epoch: [050][100/196] Loss 1.792 (1.927) Prec@1 31.25 (33.65) Prec@5 86.72 (85.33) VALID-SEARCH [2018-10-02-14:35:40] Epoch: [050][195/196] Loss 9.242 (1.945) Prec@1 10.00 (33.43) Prec@5 77.50 (85.33) 050/250, Train-Accuracy = 43.89, Test-Accuracy = 33.43 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-14:35:41] [Epoch=051/250] [Need: 06:40:34] [LR=0.0451 ~ 0.0451] [Batch=128], tau=8.776 genotype = Genotype(normal=[('dil_conv_3x3', 1, 0.12753382325172424), ('dil_conv_5x5', 0, 0.12734824419021606), ('sep_conv_3x3', 2, 0.12771405279636383), ('skip_connect', 0, 0.12740062177181244), ('skip_connect', 0, 0.12759464979171753), ('dil_conv_3x3', 3, 0.1271541863679886), ('sep_conv_3x3', 4, 0.1273501217365265), ('sep_conv_3x3', 3, 0.12727585434913635)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.12583161890506744), ('dil_conv_5x5', 1, 0.12570667266845703), ('dil_conv_5x5', 0, 0.12619943916797638), ('sep_conv_3x3', 2, 0.1255534440279007), ('dil_conv_3x3', 3, 0.12617278099060059), ('dil_conv_5x5', 2, 0.12572455406188965), ('max_pool_3x3', 1, 0.12628145515918732), ('dil_conv_5x5', 3, 0.12583743035793304)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-14:35:43] Epoch: [051][000/196] Time 1.46 (1.46) Data 0.84 (0.84) Loss 1.556 (1.556) Prec@1 39.06 (39.06) Prec@5 87.50 (87.50) TRAIN-SEARCH [2018-10-02-14:36:34] Epoch: [051][100/196] Time 0.46 (0.52) Data 0.00 (0.01) Loss 1.599 (1.511) Prec@1 40.62 (44.35) Prec@5 89.06 (91.87) TRAIN-SEARCH [2018-10-02-14:37:27] Epoch: [051][195/196] Time 0.36 (0.54) Data 0.00 (0.01) Loss 1.627 (1.511) Prec@1 40.00 (44.25) Prec@5 90.00 (92.04) VALID-SEARCH [2018-10-02-14:37:28] Epoch: [051][000/196] Loss 1.738 (1.738) Prec@1 33.59 (33.59) Prec@5 83.59 (83.59) VALID-SEARCH [2018-10-02-14:37:33] Epoch: [051][100/196] Loss 1.997 (1.947) Prec@1 26.56 (33.35) Prec@5 75.78 (84.51) VALID-SEARCH [2018-10-02-14:37:39] Epoch: [051][195/196] Loss 1.948 (1.995) Prec@1 25.00 (33.02) Prec@5 82.50 (84.19) 051/250, Train-Accuracy = 44.25, Test-Accuracy = 33.02 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-14:37:40] [Epoch=052/250] [Need: 06:32:43] [LR=0.0450 ~ 0.0450] [Batch=128], tau=8.752 genotype = Genotype(normal=[('dil_conv_5x5', 0, 0.12760032713413239), ('dil_conv_3x3', 1, 0.12714783847332), ('sep_conv_3x3', 2, 0.1281694918870926), ('skip_connect', 0, 0.12805752456188202), ('dil_conv_5x5', 0, 0.12743215262889862), ('dil_conv_3x3', 3, 0.12739835679531097), ('sep_conv_3x3', 4, 0.12736180424690247), ('sep_conv_3x3', 3, 0.1270349621772766)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.12600524723529816), ('avg_pool_3x3', 1, 0.12582199275493622), ('max_pool_3x3', 0, 0.12578807771205902), ('dil_conv_3x3', 1, 0.12572342157363892), ('max_pool_3x3', 1, 0.12578842043876648), ('avg_pool_3x3', 0, 0.1257539689540863), ('max_pool_3x3', 1, 0.1262158304452896), ('max_pool_3x3', 0, 0.1259983479976654)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-14:37:42] Epoch: [052][000/196] Time 1.88 (1.88) Data 1.19 (1.19) Loss 1.521 (1.521) Prec@1 45.31 (45.31) Prec@5 90.62 (90.62) TRAIN-SEARCH [2018-10-02-14:38:32] Epoch: [052][100/196] Time 0.64 (0.51) Data 0.02 (0.01) Loss 1.476 (1.520) Prec@1 46.88 (44.02) Prec@5 92.97 (91.66) TRAIN-SEARCH [2018-10-02-14:39:26] Epoch: [052][195/196] Time 0.37 (0.54) Data 0.00 (0.01) Loss 1.440 (1.520) Prec@1 45.00 (44.27) Prec@5 90.00 (91.64) VALID-SEARCH [2018-10-02-14:39:27] Epoch: [052][000/196] Loss 2.036 (2.036) Prec@1 26.56 (26.56) Prec@5 78.12 (78.12) VALID-SEARCH [2018-10-02-14:39:32] Epoch: [052][100/196] Loss 1.961 (1.913) Prec@1 21.88 (33.90) Prec@5 78.91 (85.33) VALID-SEARCH [2018-10-02-14:39:37] Epoch: [052][195/196] Loss 1.583 (1.876) Prec@1 32.50 (34.50) Prec@5 97.50 (85.54) 052/250, Train-Accuracy = 44.27, Test-Accuracy = 34.50 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-14:39:38] [Epoch=053/250] [Need: 06:28:58] [LR=0.0448 ~ 0.0448] [Batch=128], tau=8.728 genotype = Genotype(normal=[('dil_conv_5x5', 0, 0.1280612051486969), ('dil_conv_5x5', 1, 0.1258951723575592), ('skip_connect', 0, 0.12788483500480652), ('sep_conv_3x3', 2, 0.12773095071315765), ('dil_conv_5x5', 0, 0.1275547444820404), ('sep_conv_3x3', 2, 0.12728486955165863), ('sep_conv_3x3', 4, 0.12705081701278687), ('sep_conv_3x3', 3, 0.12685316801071167)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.1258520483970642), ('skip_connect', 1, 0.12582482397556305), ('max_pool_3x3', 0, 0.12606672942638397), ('sep_conv_3x3', 2, 0.12573452293872833), ('avg_pool_3x3', 0, 0.12594804167747498), ('max_pool_3x3', 1, 0.12591885030269623), ('max_pool_3x3', 1, 0.12581343948841095), ('dil_conv_5x5', 3, 0.12578654289245605)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-14:39:40] Epoch: [053][000/196] Time 1.29 (1.29) Data 0.77 (0.77) Loss 1.335 (1.335) Prec@1 53.91 (53.91) Prec@5 95.31 (95.31) TRAIN-SEARCH [2018-10-02-14:40:34] Epoch: [053][100/196] Time 0.53 (0.54) Data 0.00 (0.01) Loss 1.630 (1.485) Prec@1 41.41 (46.23) Prec@5 89.84 (92.21) TRAIN-SEARCH [2018-10-02-14:41:25] Epoch: [053][195/196] Time 0.44 (0.54) Data 0.00 (0.01) Loss 1.536 (1.494) Prec@1 45.00 (45.53) Prec@5 80.00 (91.90) VALID-SEARCH [2018-10-02-14:41:27] Epoch: [053][000/196] Loss 1.803 (1.803) Prec@1 28.12 (28.12) Prec@5 87.50 (87.50) VALID-SEARCH [2018-10-02-14:41:32] Epoch: [053][100/196] Loss 1.656 (1.882) Prec@1 44.53 (33.62) Prec@5 86.72 (85.71) VALID-SEARCH [2018-10-02-14:41:37] Epoch: [053][195/196] Loss 1.666 (1.867) Prec@1 45.00 (33.82) Prec@5 85.00 (85.16) 053/250, Train-Accuracy = 45.53, Test-Accuracy = 33.82 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-14:41:39] [Epoch=054/250] [Need: 06:33:39] [LR=0.0446 ~ 0.0446] [Batch=128], tau=8.704 genotype = Genotype(normal=[('skip_connect', 0, 0.12804293632507324), ('skip_connect', 1, 0.1264784336090088), ('sep_conv_3x3', 2, 0.12739603221416473), ('skip_connect', 0, 0.12708835303783417), ('max_pool_3x3', 0, 0.12744496762752533), ('sep_conv_5x5', 3, 0.12734676897525787), ('dil_conv_5x5', 4, 0.12709908187389374), ('sep_conv_3x3', 3, 0.12676076591014862)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.12587660551071167), ('dil_conv_3x3', 1, 0.12550893425941467), ('max_pool_3x3', 0, 0.1262652575969696), ('sep_conv_3x3', 2, 0.12565958499908447), ('max_pool_3x3', 1, 0.12616871297359467), ('sep_conv_5x5', 0, 0.1260836273431778), ('max_pool_3x3', 1, 0.12588079273700714), ('dil_conv_3x3', 0, 0.12566548585891724)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-14:41:41] Epoch: [054][000/196] Time 1.57 (1.57) Data 0.79 (0.79) Loss 1.378 (1.378) Prec@1 52.34 (52.34) Prec@5 96.09 (96.09) TRAIN-SEARCH [2018-10-02-14:42:33] Epoch: [054][100/196] Time 0.66 (0.53) Data 0.00 (0.01) Loss 1.667 (1.495) Prec@1 38.28 (45.67) Prec@5 91.41 (92.15) TRAIN-SEARCH [2018-10-02-14:43:25] Epoch: [054][195/196] Time 0.40 (0.54) Data 0.00 (0.01) Loss 1.413 (1.498) Prec@1 50.00 (45.45) Prec@5 92.50 (92.16) VALID-SEARCH [2018-10-02-14:43:26] Epoch: [054][000/196] Loss 2.119 (2.119) Prec@1 28.12 (28.12) Prec@5 71.09 (71.09) VALID-SEARCH [2018-10-02-14:43:32] Epoch: [054][100/196] Loss 1.904 (1.977) Prec@1 32.81 (34.07) Prec@5 78.91 (84.48) VALID-SEARCH [2018-10-02-14:43:37] Epoch: [054][195/196] Loss 2.705 (1.986) Prec@1 27.50 (34.02) Prec@5 85.00 (84.42) 054/250, Train-Accuracy = 45.45, Test-Accuracy = 34.02 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-14:43:38] [Epoch=055/250] [Need: 06:27:47] [LR=0.0444 ~ 0.0444] [Batch=128], tau=8.68 genotype = Genotype(normal=[('skip_connect', 0, 0.1282709538936615), ('skip_connect', 1, 0.1265231817960739), ('dil_conv_3x3', 2, 0.12736468017101288), ('skip_connect', 0, 0.1272619217634201), ('max_pool_3x3', 0, 0.12747935950756073), ('sep_conv_3x3', 3, 0.126888245344162), ('dil_conv_5x5', 4, 0.12730935215950012), ('sep_conv_3x3', 3, 0.12703680992126465)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.1262637823820114), ('dil_conv_3x3', 1, 0.12560957670211792), ('max_pool_3x3', 0, 0.12687252461910248), ('dil_conv_3x3', 2, 0.12570573389530182), ('max_pool_3x3', 1, 0.12607742846012115), ('avg_pool_3x3', 0, 0.12600193917751312), ('sep_conv_5x5', 4, 0.126140296459198), ('dil_conv_5x5', 1, 0.1259414702653885)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-14:43:40] Epoch: [055][000/196] Time 1.60 (1.60) Data 1.01 (1.01) Loss 1.487 (1.487) Prec@1 41.41 (41.41) Prec@5 89.84 (89.84) TRAIN-SEARCH [2018-10-02-14:44:34] Epoch: [055][100/196] Time 0.63 (0.54) Data 0.00 (0.01) Loss 1.447 (1.505) Prec@1 44.53 (45.58) Prec@5 93.75 (91.61) TRAIN-SEARCH [2018-10-02-14:45:26] Epoch: [055][195/196] Time 0.55 (0.55) Data 0.00 (0.01) Loss 1.143 (1.502) Prec@1 55.00 (45.28) Prec@5 97.50 (91.84) VALID-SEARCH [2018-10-02-14:45:27] Epoch: [055][000/196] Loss 2.506 (2.506) Prec@1 25.78 (25.78) Prec@5 91.41 (91.41) VALID-SEARCH [2018-10-02-14:45:32] Epoch: [055][100/196] Loss 2.042 (1.886) Prec@1 21.09 (34.17) Prec@5 70.31 (85.81) VALID-SEARCH [2018-10-02-14:45:37] Epoch: [055][195/196] Loss 1.958 (1.883) Prec@1 20.00 (34.61) Prec@5 75.00 (85.60) 055/250, Train-Accuracy = 45.28, Test-Accuracy = 34.61 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-14:45:39] [Epoch=056/250] [Need: 06:30:32] [LR=0.0442 ~ 0.0442] [Batch=128], tau=8.656 genotype = Genotype(normal=[('skip_connect', 0, 0.12809543311595917), ('skip_connect', 1, 0.12617893517017365), ('dil_conv_3x3', 2, 0.12741978466510773), ('skip_connect', 0, 0.1271176040172577), ('sep_conv_3x3', 2, 0.12755703926086426), ('skip_connect', 0, 0.12733399868011475), ('max_pool_3x3', 0, 0.1274213343858719), ('dil_conv_3x3', 4, 0.12704390287399292)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.12617957592010498), ('dil_conv_3x3', 1, 0.1254347264766693), ('max_pool_3x3', 0, 0.12653982639312744), ('sep_conv_3x3', 1, 0.12586171925067902), ('max_pool_3x3', 0, 0.1263417750597), ('max_pool_3x3', 1, 0.125812828540802), ('sep_conv_5x5', 4, 0.12599654495716095), ('dil_conv_5x5', 1, 0.1259852647781372)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-14:45:41] Epoch: [056][000/196] Time 1.47 (1.47) Data 0.88 (0.88) Loss 1.439 (1.439) Prec@1 48.44 (48.44) Prec@5 94.53 (94.53) TRAIN-SEARCH [2018-10-02-14:46:32] Epoch: [056][100/196] Time 0.50 (0.52) Data 0.00 (0.01) Loss 1.564 (1.497) Prec@1 47.66 (45.61) Prec@5 89.84 (92.01) TRAIN-SEARCH [2018-10-02-14:47:26] Epoch: [056][195/196] Time 0.46 (0.54) Data 0.00 (0.01) Loss 1.601 (1.496) Prec@1 45.00 (45.80) Prec@5 92.50 (91.84) VALID-SEARCH [2018-10-02-14:47:27] Epoch: [056][000/196] Loss 2.125 (2.125) Prec@1 29.69 (29.69) Prec@5 78.12 (78.12) VALID-SEARCH [2018-10-02-14:47:33] Epoch: [056][100/196] Loss 1.909 (2.047) Prec@1 28.91 (32.56) Prec@5 82.03 (83.48) VALID-SEARCH [2018-10-02-14:47:38] Epoch: [056][195/196] Loss 1.674 (1.978) Prec@1 30.00 (32.75) Prec@5 87.50 (83.98) 056/250, Train-Accuracy = 45.80, Test-Accuracy = 32.75 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-14:47:39] [Epoch=057/250] [Need: 06:26:59] [LR=0.0440 ~ 0.0440] [Batch=128], tau=8.632 genotype = Genotype(normal=[('skip_connect', 0, 0.1282428801059723), ('skip_connect', 1, 0.12646298110485077), ('skip_connect', 0, 0.12748382985591888), ('dil_conv_3x3', 2, 0.12713177502155304), ('sep_conv_3x3', 2, 0.12752893567085266), ('max_pool_3x3', 0, 0.12740179896354675), ('max_pool_3x3', 0, 0.12725481390953064), ('dil_conv_5x5', 4, 0.12717340886592865)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.126699760556221), ('max_pool_3x3', 1, 0.12575021386146545), ('max_pool_3x3', 0, 0.12621062994003296), ('dil_conv_3x3', 2, 0.12569527328014374), ('dil_conv_5x5', 3, 0.1259448379278183), ('dil_conv_5x5', 2, 0.12589600682258606), ('dil_conv_5x5', 1, 0.12599459290504456), ('sep_conv_3x3', 3, 0.12597255408763885)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-14:47:41] Epoch: [057][000/196] Time 1.66 (1.66) Data 1.00 (1.00) Loss 1.650 (1.650) Prec@1 40.62 (40.62) Prec@5 88.28 (88.28) TRAIN-SEARCH [2018-10-02-14:48:32] Epoch: [057][100/196] Time 0.67 (0.52) Data 0.02 (0.01) Loss 1.264 (1.472) Prec@1 51.56 (46.41) Prec@5 96.88 (92.54) TRAIN-SEARCH [2018-10-02-14:49:26] Epoch: [057][195/196] Time 0.40 (0.54) Data 0.00 (0.01) Loss 1.468 (1.478) Prec@1 37.50 (46.59) Prec@5 90.00 (92.41) VALID-SEARCH [2018-10-02-14:49:27] Epoch: [057][000/196] Loss 1.545 (1.545) Prec@1 44.53 (44.53) Prec@5 91.41 (91.41) VALID-SEARCH [2018-10-02-14:49:33] Epoch: [057][100/196] Loss 1.508 (1.834) Prec@1 40.62 (34.64) Prec@5 90.62 (85.44) VALID-SEARCH [2018-10-02-14:49:38] Epoch: [057][195/196] Loss 1.971 (1.866) Prec@1 20.00 (34.78) Prec@5 82.50 (85.56) 057/250, Train-Accuracy = 46.59, Test-Accuracy = 34.78 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-14:49:39] [Epoch=058/250] [Need: 06:23:59] [LR=0.0438 ~ 0.0438] [Batch=128], tau=8.608 genotype = Genotype(normal=[('skip_connect', 0, 0.12877491116523743), ('skip_connect', 1, 0.12672221660614014), ('skip_connect', 0, 0.12783224880695343), ('sep_conv_3x3', 2, 0.12683334946632385), ('max_pool_3x3', 0, 0.12768574059009552), ('sep_conv_3x3', 3, 0.12728384137153625), ('sep_conv_3x3', 4, 0.12750127911567688), ('skip_connect', 0, 0.12715741991996765)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.12604078650474548), ('max_pool_3x3', 1, 0.1255474090576172), ('dil_conv_5x5', 0, 0.12607954442501068), ('avg_pool_3x3', 1, 0.1258925497531891), ('dil_conv_5x5', 3, 0.1262885481119156), ('skip_connect', 0, 0.12608778476715088), ('avg_pool_3x3', 0, 0.12586914002895355), ('sep_conv_5x5', 4, 0.12577787041664124)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-14:49:42] Epoch: [058][000/196] Time 1.88 (1.88) Data 1.19 (1.19) Loss 1.441 (1.441) Prec@1 53.91 (53.91) Prec@5 92.19 (92.19) TRAIN-SEARCH [2018-10-02-14:50:31] Epoch: [058][100/196] Time 0.53 (0.51) Data 0.02 (0.01) Loss 1.467 (1.464) Prec@1 43.75 (46.38) Prec@5 90.62 (92.36) TRAIN-SEARCH [2018-10-02-14:51:25] Epoch: [058][195/196] Time 0.39 (0.54) Data 0.00 (0.01) Loss 1.565 (1.477) Prec@1 40.00 (45.92) Prec@5 95.00 (92.45) VALID-SEARCH [2018-10-02-14:51:27] Epoch: [058][000/196] Loss 2.805 (2.805) Prec@1 17.97 (17.97) Prec@5 65.62 (65.62) VALID-SEARCH [2018-10-02-14:51:32] Epoch: [058][100/196] Loss 1.744 (1.864) Prec@1 39.84 (34.70) Prec@5 86.72 (84.85) VALID-SEARCH [2018-10-02-14:51:37] Epoch: [058][195/196] Loss 1.728 (1.859) Prec@1 25.00 (35.05) Prec@5 92.50 (84.94) 058/250, Train-Accuracy = 45.92, Test-Accuracy = 35.05 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-14:51:38] [Epoch=059/250] [Need: 06:18:52] [LR=0.0436 ~ 0.0436] [Batch=128], tau=8.584 genotype = Genotype(normal=[('skip_connect', 0, 0.12833271920681), ('sep_conv_3x3', 1, 0.12598417699337006), ('skip_connect', 1, 0.1281585991382599), ('skip_connect', 0, 0.12805533409118652), ('skip_connect', 0, 0.1279265582561493), ('sep_conv_3x3', 3, 0.1271359771490097), ('skip_connect', 0, 0.12702225148677826), ('sep_conv_3x3', 4, 0.12699265778064728)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.12628769874572754), ('avg_pool_3x3', 1, 0.12574441730976105), ('max_pool_3x3', 0, 0.12628121674060822), ('max_pool_3x3', 1, 0.1262083798646927), ('skip_connect', 0, 0.1263912171125412), ('avg_pool_3x3', 1, 0.1255469024181366), ('avg_pool_3x3', 0, 0.12616969645023346), ('sep_conv_5x5', 4, 0.12590672075748444)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-14:51:40] Epoch: [059][000/196] Time 1.12 (1.12) Data 0.63 (0.63) Loss 1.409 (1.409) Prec@1 53.12 (53.12) Prec@5 90.62 (90.62) TRAIN-SEARCH [2018-10-02-14:52:35] Epoch: [059][100/196] Time 0.47 (0.55) Data 0.00 (0.01) Loss 1.410 (1.484) Prec@1 50.78 (46.25) Prec@5 92.19 (92.57) TRAIN-SEARCH [2018-10-02-14:53:28] Epoch: [059][195/196] Time 0.42 (0.56) Data 0.00 (0.00) Loss 1.265 (1.467) Prec@1 57.50 (46.62) Prec@5 95.00 (92.55) VALID-SEARCH [2018-10-02-14:53:29] Epoch: [059][000/196] Loss 1.536 (1.536) Prec@1 41.41 (41.41) Prec@5 90.62 (90.62) VALID-SEARCH [2018-10-02-14:53:33] Epoch: [059][100/196] Loss 1.783 (2.057) Prec@1 38.28 (36.70) Prec@5 89.84 (87.04) VALID-SEARCH [2018-10-02-14:53:38] Epoch: [059][195/196] Loss 1.875 (1.996) Prec@1 30.00 (37.15) Prec@5 85.00 (87.40) 059/250, Train-Accuracy = 46.62, Test-Accuracy = 37.15 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-14:53:39] [Epoch=060/250] [Need: 06:22:26] [LR=0.0434 ~ 0.0434] [Batch=128], tau=8.56 genotype = Genotype(normal=[('skip_connect', 0, 0.1290249228477478), ('skip_connect', 1, 0.12640200555324554), ('skip_connect', 0, 0.12784257531166077), ('skip_connect', 1, 0.12764199078083038), ('max_pool_3x3', 0, 0.1280539184808731), ('sep_conv_3x3', 2, 0.127310648560524), ('dil_conv_5x5', 4, 0.12721358239650726), ('max_pool_3x3', 0, 0.12701429426670074)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.12603288888931274), ('dil_conv_3x3', 1, 0.125631645321846), ('dil_conv_5x5', 0, 0.12599821388721466), ('skip_connect', 2, 0.12566787004470825), ('max_pool_3x3', 0, 0.1260678768157959), ('sep_conv_3x3', 2, 0.125738725066185), ('sep_conv_5x5', 4, 0.12608569860458374), ('dil_conv_5x5', 3, 0.12602242827415466)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-14:53:40] Epoch: [060][000/196] Time 1.01 (1.01) Data 0.57 (0.57) Loss 1.412 (1.412) Prec@1 50.78 (50.78) Prec@5 90.62 (90.62) TRAIN-SEARCH [2018-10-02-14:54:32] Epoch: [060][100/196] Time 0.64 (0.52) Data 0.01 (0.01) Loss 1.501 (1.454) Prec@1 46.88 (47.11) Prec@5 93.75 (92.21) TRAIN-SEARCH [2018-10-02-14:55:25] Epoch: [060][195/196] Time 0.42 (0.54) Data 0.00 (0.01) Loss 1.743 (1.467) Prec@1 35.00 (46.75) Prec@5 90.00 (92.24) VALID-SEARCH [2018-10-02-14:55:27] Epoch: [060][000/196] Loss 2.584 (2.584) Prec@1 25.00 (25.00) Prec@5 69.53 (69.53) VALID-SEARCH [2018-10-02-14:55:32] Epoch: [060][100/196] Loss 1.583 (1.770) Prec@1 46.88 (37.16) Prec@5 95.31 (87.93) VALID-SEARCH [2018-10-02-14:55:36] Epoch: [060][195/196] Loss 1.832 (1.754) Prec@1 45.00 (37.22) Prec@5 90.00 (87.77) 060/250, Train-Accuracy = 46.75, Test-Accuracy = 37.22 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-14:55:38] [Epoch=061/250] [Need: 06:15:14] [LR=0.0431 ~ 0.0431] [Batch=128], tau=8.536 genotype = Genotype(normal=[('skip_connect', 0, 0.12875115871429443), ('sep_conv_3x3', 1, 0.12648706138134003), ('skip_connect', 0, 0.1289379596710205), ('skip_connect', 1, 0.1275118887424469), ('skip_connect', 0, 0.12813600897789001), ('max_pool_3x3', 1, 0.12715324759483337), ('skip_connect', 0, 0.1279541403055191), ('dil_conv_5x5', 4, 0.1267547458410263)], normal_concat=range(2, 6), reduce=[('skip_connect', 0, 0.12629838287830353), ('sep_conv_3x3', 1, 0.12583300471305847), ('max_pool_3x3', 1, 0.12626631557941437), ('max_pool_3x3', 0, 0.12576070427894592), ('dil_conv_5x5', 0, 0.12593792378902435), ('avg_pool_3x3', 1, 0.12568704783916473), ('avg_pool_3x3', 0, 0.12600740790367126), ('sep_conv_5x5', 4, 0.12595954537391663)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-14:55:39] Epoch: [061][000/196] Time 1.05 (1.05) Data 0.64 (0.64) Loss 1.525 (1.525) Prec@1 41.41 (41.41) Prec@5 92.19 (92.19) TRAIN-SEARCH [2018-10-02-14:56:33] Epoch: [061][100/196] Time 0.68 (0.54) Data 0.00 (0.01) Loss 1.425 (1.435) Prec@1 40.62 (48.14) Prec@5 91.41 (92.85) TRAIN-SEARCH [2018-10-02-14:57:24] Epoch: [061][195/196] Time 0.40 (0.54) Data 0.00 (0.01) Loss 1.646 (1.451) Prec@1 30.00 (47.27) Prec@5 85.00 (92.64) VALID-SEARCH [2018-10-02-14:57:26] Epoch: [061][000/196] Loss 1.556 (1.556) Prec@1 39.84 (39.84) Prec@5 94.53 (94.53) VALID-SEARCH [2018-10-02-14:57:30] Epoch: [061][100/196] Loss 1.531 (1.819) Prec@1 40.62 (36.61) Prec@5 87.50 (87.20) VALID-SEARCH [2018-10-02-14:57:33] Epoch: [061][195/196] Loss 1.757 (1.842) Prec@1 40.00 (36.30) Prec@5 87.50 (86.79) 061/250, Train-Accuracy = 47.27, Test-Accuracy = 36.30 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-14:57:34] [Epoch=062/250] [Need: 06:04:25] [LR=0.0429 ~ 0.0429] [Batch=128], tau=8.512 genotype = Genotype(normal=[('skip_connect', 0, 0.12932421267032623), ('sep_conv_3x3', 1, 0.126272052526474), ('skip_connect', 0, 0.12864840030670166), ('sep_conv_5x5', 2, 0.12760357558727264), ('skip_connect', 0, 0.12749221920967102), ('dil_conv_3x3', 3, 0.12721139192581177), ('skip_connect', 0, 0.1277056187391281), ('sep_conv_5x5', 4, 0.12689261138439178)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 1, 0.12608380615711212), ('avg_pool_3x3', 0, 0.12600059807300568), ('avg_pool_3x3', 1, 0.12624143064022064), ('dil_conv_3x3', 2, 0.1258319914340973), ('avg_pool_3x3', 0, 0.1263439804315567), ('dil_conv_5x5', 2, 0.12559868395328522), ('max_pool_3x3', 0, 0.12586183845996857), ('sep_conv_3x3', 4, 0.1258615255355835)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-14:57:36] Epoch: [062][000/196] Time 1.05 (1.05) Data 0.60 (0.60) Loss 1.675 (1.675) Prec@1 42.19 (42.19) Prec@5 88.28 (88.28) TRAIN-SEARCH [2018-10-02-14:58:30] Epoch: [062][100/196] Time 0.46 (0.55) Data 0.00 (0.01) Loss 1.381 (1.438) Prec@1 45.31 (47.62) Prec@5 95.31 (92.74) TRAIN-SEARCH [2018-10-02-14:59:23] Epoch: [062][195/196] Time 0.41 (0.55) Data 0.00 (0.01) Loss 1.500 (1.439) Prec@1 47.50 (47.74) Prec@5 95.00 (92.74) VALID-SEARCH [2018-10-02-14:59:24] Epoch: [062][000/196] Loss 3.006 (3.006) Prec@1 33.59 (33.59) Prec@5 85.16 (85.16) VALID-SEARCH [2018-10-02-14:59:28] Epoch: [062][100/196] Loss 1.564 (1.852) Prec@1 43.75 (36.79) Prec@5 90.62 (86.01) VALID-SEARCH [2018-10-02-14:59:31] Epoch: [062][195/196] Loss 1.512 (1.807) Prec@1 40.00 (37.45) Prec@5 90.00 (86.70) 062/250, Train-Accuracy = 47.74, Test-Accuracy = 37.45 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-14:59:32] [Epoch=063/250] [Need: 06:07:48] [LR=0.0427 ~ 0.0427] [Batch=128], tau=8.488 genotype = Genotype(normal=[('skip_connect', 0, 0.13036352396011353), ('sep_conv_5x5', 1, 0.12604820728302002), ('skip_connect', 0, 0.1287686675786972), ('sep_conv_5x5', 2, 0.1277298480272293), ('max_pool_3x3', 0, 0.12741638720035553), ('dil_conv_3x3', 3, 0.12738530337810516), ('skip_connect', 0, 0.12833577394485474), ('sep_conv_5x5', 4, 0.127010315656662)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.12603768706321716), ('avg_pool_3x3', 1, 0.12552694976329803), ('max_pool_3x3', 0, 0.1263214498758316), ('dil_conv_3x3', 1, 0.12574215233325958), ('max_pool_3x3', 0, 0.1262769252061844), ('sep_conv_3x3', 3, 0.1255662441253662), ('max_pool_3x3', 1, 0.1259361207485199), ('dil_conv_5x5', 4, 0.12574313580989838)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-14:59:34] Epoch: [063][000/196] Time 1.05 (1.05) Data 0.64 (0.64) Loss 1.246 (1.246) Prec@1 50.78 (50.78) Prec@5 92.97 (92.97) TRAIN-SEARCH [2018-10-02-15:00:29] Epoch: [063][100/196] Time 0.43 (0.56) Data 0.00 (0.01) Loss 1.588 (1.442) Prec@1 42.97 (47.95) Prec@5 91.41 (92.60) TRAIN-SEARCH [2018-10-02-15:01:23] Epoch: [063][195/196] Time 0.47 (0.56) Data 0.00 (0.01) Loss 1.591 (1.434) Prec@1 40.00 (47.95) Prec@5 92.50 (92.73) VALID-SEARCH [2018-10-02-15:01:24] Epoch: [063][000/196] Loss 1.689 (1.689) Prec@1 36.72 (36.72) Prec@5 91.41 (91.41) VALID-SEARCH [2018-10-02-15:01:28] Epoch: [063][100/196] Loss 1.515 (1.847) Prec@1 43.75 (37.63) Prec@5 92.19 (86.98) VALID-SEARCH [2018-10-02-15:01:32] Epoch: [063][195/196] Loss 2.081 (1.913) Prec@1 22.50 (36.53) Prec@5 82.50 (86.51) 063/250, Train-Accuracy = 47.95, Test-Accuracy = 36.53 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-15:01:33] [Epoch=064/250] [Need: 06:13:14] [LR=0.0425 ~ 0.0425] [Batch=128], tau=8.464 genotype = Genotype(normal=[('skip_connect', 0, 0.13055792450904846), ('dil_conv_3x3', 1, 0.12606866657733917), ('skip_connect', 0, 0.12772883474826813), ('sep_conv_5x5', 2, 0.12698252499103546), ('skip_connect', 0, 0.12772653996944427), ('sep_conv_3x3', 2, 0.12747347354888916), ('skip_connect', 0, 0.12780176103115082), ('sep_conv_5x5', 4, 0.12741932272911072)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 1, 0.125884547829628), ('avg_pool_3x3', 0, 0.12565909326076508), ('avg_pool_3x3', 0, 0.12611348927021027), ('avg_pool_3x3', 1, 0.12572412192821503), ('max_pool_3x3', 0, 0.12658686935901642), ('sep_conv_5x5', 1, 0.12582534551620483), ('skip_connect', 1, 0.12598896026611328), ('max_pool_3x3', 0, 0.12597139179706573)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-15:01:34] Epoch: [064][000/196] Time 0.95 (0.95) Data 0.54 (0.54) Loss 1.517 (1.517) Prec@1 43.75 (43.75) Prec@5 90.62 (90.62) TRAIN-SEARCH [2018-10-02-15:02:29] Epoch: [064][100/196] Time 0.54 (0.56) Data 0.01 (0.01) Loss 1.329 (1.432) Prec@1 45.31 (48.44) Prec@5 93.75 (92.93) TRAIN-SEARCH [2018-10-02-15:03:22] Epoch: [064][195/196] Time 0.32 (0.56) Data 0.00 (0.01) Loss 1.563 (1.437) Prec@1 47.50 (48.34) Prec@5 90.00 (92.62) VALID-SEARCH [2018-10-02-15:03:23] Epoch: [064][000/196] Loss 1.663 (1.663) Prec@1 35.94 (35.94) Prec@5 86.72 (86.72) VALID-SEARCH [2018-10-02-15:03:27] Epoch: [064][100/196] Loss 2.039 (1.875) Prec@1 23.44 (34.93) Prec@5 75.00 (84.86) VALID-SEARCH [2018-10-02-15:03:31] Epoch: [064][195/196] Loss 1.791 (1.852) Prec@1 37.50 (35.38) Prec@5 92.50 (85.42) 064/250, Train-Accuracy = 48.34, Test-Accuracy = 35.38 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-15:03:34] [Epoch=065/250] [Need: 06:14:03] [LR=0.0423 ~ 0.0423] [Batch=128], tau=8.44 genotype = Genotype(normal=[('skip_connect', 0, 0.12961439788341522), ('dil_conv_3x3', 1, 0.1266012042760849), ('max_pool_3x3', 0, 0.12734399735927582), ('sep_conv_3x3', 2, 0.1272551715373993), ('skip_connect', 0, 0.12799890339374542), ('sep_conv_3x3', 2, 0.12744058668613434), ('max_pool_3x3', 0, 0.12777039408683777), ('sep_conv_5x5', 4, 0.12766961753368378)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.12677280604839325), ('avg_pool_3x3', 1, 0.12562069296836853), ('avg_pool_3x3', 0, 0.12619313597679138), ('avg_pool_3x3', 1, 0.1261867731809616), ('avg_pool_3x3', 0, 0.12599757313728333), ('dil_conv_5x5', 3, 0.12556234002113342), ('max_pool_3x3', 0, 0.12616057693958282), ('dil_conv_5x5', 4, 0.12605051696300507)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-15:03:36] Epoch: [065][000/196] Time 1.44 (1.44) Data 0.79 (0.79) Loss 1.411 (1.411) Prec@1 47.66 (47.66) Prec@5 91.41 (91.41) TRAIN-SEARCH [2018-10-02-15:04:32] Epoch: [065][100/196] Time 0.62 (0.57) Data 0.00 (0.01) Loss 1.268 (1.447) Prec@1 50.00 (47.85) Prec@5 90.62 (92.64) TRAIN-SEARCH [2018-10-02-15:05:24] Epoch: [065][195/196] Time 0.30 (0.56) Data 0.00 (0.01) Loss 1.496 (1.433) Prec@1 42.50 (48.13) Prec@5 95.00 (92.80) VALID-SEARCH [2018-10-02-15:05:24] Epoch: [065][000/196] Loss 1.603 (1.603) Prec@1 39.84 (39.84) Prec@5 91.41 (91.41) VALID-SEARCH [2018-10-02-15:05:28] Epoch: [065][100/196] Loss 1.777 (1.848) Prec@1 41.41 (34.82) Prec@5 85.94 (84.81) VALID-SEARCH [2018-10-02-15:05:32] Epoch: [065][195/196] Loss 1.733 (1.855) Prec@1 35.00 (35.30) Prec@5 87.50 (84.67) 065/250, Train-Accuracy = 48.13, Test-Accuracy = 35.30 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-15:05:33] [Epoch=066/250] [Need: 06:04:49] [LR=0.0420 ~ 0.0420] [Batch=128], tau=8.416 genotype = Genotype(normal=[('skip_connect', 0, 0.12976297736167908), ('dil_conv_3x3', 1, 0.126829132437706), ('sep_conv_5x5', 2, 0.12722130119800568), ('skip_connect', 0, 0.1270565390586853), ('max_pool_3x3', 0, 0.1273762732744217), ('max_pool_3x3', 1, 0.12729209661483765), ('skip_connect', 0, 0.12803848087787628), ('sep_conv_5x5', 4, 0.1274755746126175)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.12657873332500458), ('sep_conv_3x3', 1, 0.1258772313594818), ('avg_pool_3x3', 0, 0.12598147988319397), ('max_pool_3x3', 1, 0.12591294944286346), ('max_pool_3x3', 0, 0.12641708552837372), ('dil_conv_3x3', 2, 0.12576274573802948), ('skip_connect', 0, 0.12575837969779968), ('sep_conv_3x3', 3, 0.1255890280008316)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-15:05:35] Epoch: [066][000/196] Time 1.27 (1.27) Data 0.78 (0.78) Loss 1.513 (1.513) Prec@1 42.97 (42.97) Prec@5 90.62 (90.62) TRAIN-SEARCH [2018-10-02-15:06:32] Epoch: [066][100/196] Time 0.51 (0.58) Data 0.00 (0.01) Loss 1.643 (1.422) Prec@1 40.62 (48.38) Prec@5 87.50 (93.29) TRAIN-SEARCH [2018-10-02-15:07:24] Epoch: [066][195/196] Time 0.31 (0.57) Data 0.00 (0.01) Loss 1.380 (1.429) Prec@1 60.00 (48.12) Prec@5 90.00 (92.98) VALID-SEARCH [2018-10-02-15:07:25] Epoch: [066][000/196] Loss 1.916 (1.916) Prec@1 29.69 (29.69) Prec@5 76.56 (76.56) VALID-SEARCH [2018-10-02-15:07:29] Epoch: [066][100/196] Loss 1.714 (1.827) Prec@1 37.50 (37.56) Prec@5 87.50 (86.53) VALID-SEARCH [2018-10-02-15:07:33] Epoch: [066][195/196] Loss 1.852 (1.844) Prec@1 30.00 (37.22) Prec@5 85.00 (86.21) 066/250, Train-Accuracy = 48.12, Test-Accuracy = 37.22 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-15:07:34] [Epoch=067/250] [Need: 06:08:26] [LR=0.0418 ~ 0.0418] [Batch=128], tau=8.392 genotype = Genotype(normal=[('skip_connect', 0, 0.12869831919670105), ('dil_conv_3x3', 1, 0.126878559589386), ('skip_connect', 0, 0.12783344089984894), ('sep_conv_5x5', 2, 0.12758859992027283), ('max_pool_3x3', 1, 0.12754790484905243), ('skip_connect', 0, 0.1271163523197174), ('skip_connect', 0, 0.1275746375322342), ('sep_conv_5x5', 4, 0.1271383911371231)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.12610124051570892), ('sep_conv_3x3', 1, 0.12575490772724152), ('max_pool_3x3', 0, 0.12592795491218567), ('sep_conv_5x5', 2, 0.1256735622882843), ('dil_conv_5x5', 0, 0.1260896921157837), ('dil_conv_5x5', 3, 0.12556986510753632), ('avg_pool_3x3', 0, 0.12638068199157715), ('max_pool_3x3', 2, 0.12582997977733612)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-15:07:36] Epoch: [067][000/196] Time 1.49 (1.49) Data 0.80 (0.80) Loss 1.491 (1.491) Prec@1 49.22 (49.22) Prec@5 93.75 (93.75) TRAIN-SEARCH [2018-10-02-15:08:35] Epoch: [067][100/196] Time 0.52 (0.60) Data 0.00 (0.01) Loss 1.356 (1.431) Prec@1 56.25 (48.51) Prec@5 91.41 (92.67) TRAIN-SEARCH [2018-10-02-15:09:23] Epoch: [067][195/196] Time 0.32 (0.55) Data 0.00 (0.01) Loss 1.823 (1.420) Prec@1 35.00 (49.01) Prec@5 90.00 (92.93) VALID-SEARCH [2018-10-02-15:09:24] Epoch: [067][000/196] Loss 1.569 (1.569) Prec@1 43.75 (43.75) Prec@5 93.75 (93.75) VALID-SEARCH [2018-10-02-15:09:28] Epoch: [067][100/196] Loss 2.141 (1.884) Prec@1 27.34 (37.13) Prec@5 70.31 (86.87) VALID-SEARCH [2018-10-02-15:09:33] Epoch: [067][195/196] Loss 2.596 (1.879) Prec@1 20.00 (36.61) Prec@5 57.50 (86.46) 067/250, Train-Accuracy = 49.01, Test-Accuracy = 36.61 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-15:09:34] [Epoch=068/250] [Need: 06:05:04] [LR=0.0416 ~ 0.0416] [Batch=128], tau=8.368 genotype = Genotype(normal=[('skip_connect', 0, 0.12969748675823212), ('skip_connect', 1, 0.12639935314655304), ('sep_conv_5x5', 2, 0.127691388130188), ('max_pool_3x3', 1, 0.12728042900562286), ('max_pool_3x3', 1, 0.12768542766571045), ('max_pool_3x3', 0, 0.12759646773338318), ('max_pool_3x3', 1, 0.12719488143920898), ('max_pool_3x3', 0, 0.12699082493782043)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.12655247747898102), ('sep_conv_5x5', 1, 0.12574289739131927), ('max_pool_3x3', 0, 0.12634283304214478), ('sep_conv_5x5', 2, 0.1261725127696991), ('dil_conv_5x5', 0, 0.12632761895656586), ('dil_conv_5x5', 3, 0.12566806375980377), ('dil_conv_3x3', 2, 0.12595604360103607), ('sep_conv_5x5', 4, 0.12581901252269745)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-15:09:36] Epoch: [068][000/196] Time 1.59 (1.59) Data 1.01 (1.01) Loss 1.351 (1.351) Prec@1 53.12 (53.12) Prec@5 92.97 (92.97) TRAIN-SEARCH [2018-10-02-15:10:34] Epoch: [068][100/196] Time 0.52 (0.59) Data 0.00 (0.01) Loss 1.302 (1.411) Prec@1 55.47 (48.48) Prec@5 92.19 (93.40) TRAIN-SEARCH [2018-10-02-15:11:23] Epoch: [068][195/196] Time 0.33 (0.55) Data 0.00 (0.01) Loss 1.168 (1.402) Prec@1 57.50 (49.08) Prec@5 97.50 (93.18) VALID-SEARCH [2018-10-02-15:11:23] Epoch: [068][000/196] Loss 1.895 (1.895) Prec@1 31.25 (31.25) Prec@5 83.59 (83.59) VALID-SEARCH [2018-10-02-15:11:28] Epoch: [068][100/196] Loss 1.378 (1.803) Prec@1 51.56 (37.94) Prec@5 94.53 (86.81) VALID-SEARCH [2018-10-02-15:11:33] Epoch: [068][195/196] Loss 2.532 (1.802) Prec@1 22.50 (37.07) Prec@5 77.50 (86.70) 068/250, Train-Accuracy = 49.08, Test-Accuracy = 37.07 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-15:11:34] [Epoch=069/250] [Need: 06:03:02] [LR=0.0414 ~ 0.0414] [Batch=128], tau=8.344 genotype = Genotype(normal=[('skip_connect', 0, 0.12942570447921753), ('sep_conv_5x5', 1, 0.12674352526664734), ('skip_connect', 0, 0.12753020226955414), ('sep_conv_5x5', 2, 0.1272004395723343), ('max_pool_3x3', 0, 0.12808910012245178), ('max_pool_3x3', 1, 0.12690384685993195), ('skip_connect', 0, 0.12749572098255157), ('dil_conv_3x3', 3, 0.12694275379180908)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.12597927451133728), ('sep_conv_5x5', 1, 0.12574908137321472), ('max_pool_3x3', 0, 0.12724067270755768), ('max_pool_3x3', 1, 0.12571027874946594), ('avg_pool_3x3', 1, 0.12607456743717194), ('max_pool_3x3', 0, 0.12576249241828918), ('sep_conv_5x5', 0, 0.12560506165027618), ('sep_conv_5x5', 4, 0.12553785741329193)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-15:11:37] Epoch: [069][000/196] Time 1.76 (1.76) Data 0.94 (0.94) Loss 1.391 (1.391) Prec@1 48.44 (48.44) Prec@5 92.97 (92.97) TRAIN-SEARCH [2018-10-02-15:12:34] Epoch: [069][100/196] Time 0.43 (0.59) Data 0.00 (0.01) Loss 1.582 (1.386) Prec@1 43.75 (50.17) Prec@5 93.75 (93.51) TRAIN-SEARCH [2018-10-02-15:13:22] Epoch: [069][195/196] Time 0.29 (0.54) Data 0.00 (0.01) Loss 1.462 (1.401) Prec@1 45.00 (49.74) Prec@5 97.50 (93.18) VALID-SEARCH [2018-10-02-15:13:23] Epoch: [069][000/196] Loss 1.484 (1.484) Prec@1 45.31 (45.31) Prec@5 92.19 (92.19) VALID-SEARCH [2018-10-02-15:13:28] Epoch: [069][100/196] Loss 1.482 (1.760) Prec@1 46.88 (37.55) Prec@5 89.84 (87.41) VALID-SEARCH [2018-10-02-15:13:33] Epoch: [069][195/196] Loss 1.412 (1.756) Prec@1 45.00 (37.18) Prec@5 97.50 (87.27) 069/250, Train-Accuracy = 49.74, Test-Accuracy = 37.18 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-15:13:34] [Epoch=070/250] [Need: 05:57:35] [LR=0.0411 ~ 0.0411] [Batch=128], tau=8.32 genotype = Genotype(normal=[('skip_connect', 0, 0.1290237456560135), ('skip_connect', 1, 0.1270129680633545), ('skip_connect', 0, 0.12764370441436768), ('sep_conv_3x3', 2, 0.127310112118721), ('max_pool_3x3', 0, 0.12879759073257446), ('max_pool_3x3', 1, 0.12701010704040527), ('skip_connect', 0, 0.12822407484054565), ('dil_conv_5x5', 4, 0.1269415318965912)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.12594181299209595), ('avg_pool_3x3', 1, 0.12572649121284485), ('avg_pool_3x3', 0, 0.12636704742908478), ('max_pool_3x3', 1, 0.12612144649028778), ('dil_conv_5x5', 1, 0.12595336139202118), ('avg_pool_3x3', 0, 0.12578171491622925), ('sep_conv_5x5', 0, 0.12614385783672333), ('dil_conv_5x5', 2, 0.1260664314031601)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-15:13:36] Epoch: [070][000/196] Time 1.48 (1.48) Data 0.85 (0.85) Loss 1.276 (1.276) Prec@1 57.03 (57.03) Prec@5 92.19 (92.19) TRAIN-SEARCH [2018-10-02-15:14:32] Epoch: [070][100/196] Time 0.65 (0.57) Data 0.00 (0.01) Loss 1.377 (1.381) Prec@1 57.03 (50.99) Prec@5 93.75 (93.46) TRAIN-SEARCH [2018-10-02-15:15:23] Epoch: [070][195/196] Time 0.55 (0.55) Data 0.00 (0.01) Loss 1.504 (1.380) Prec@1 40.00 (50.52) Prec@5 92.50 (93.41) VALID-SEARCH [2018-10-02-15:15:24] Epoch: [070][000/196] Loss 1.867 (1.867) Prec@1 34.38 (34.38) Prec@5 86.72 (86.72) VALID-SEARCH [2018-10-02-15:15:30] Epoch: [070][100/196] Loss 1.515 (1.875) Prec@1 45.31 (36.23) Prec@5 89.06 (85.99) VALID-SEARCH [2018-10-02-15:15:35] Epoch: [070][195/196] Loss 1.484 (1.875) Prec@1 47.50 (36.52) Prec@5 97.50 (85.76) 070/250, Train-Accuracy = 50.52, Test-Accuracy = 36.52 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-15:15:36] [Epoch=071/250] [Need: 06:05:55] [LR=0.0409 ~ 0.0409] [Batch=128], tau=8.296 genotype = Genotype(normal=[('skip_connect', 0, 0.12937591969966888), ('skip_connect', 1, 0.1266942322254181), ('skip_connect', 0, 0.1281498223543167), ('sep_conv_3x3', 2, 0.12755361199378967), ('max_pool_3x3', 0, 0.12853692471981049), ('sep_conv_3x3', 3, 0.12736070156097412), ('max_pool_3x3', 0, 0.12847945094108582), ('dil_conv_5x5', 4, 0.12686026096343994)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.1262778341770172), ('sep_conv_3x3', 1, 0.12616154551506042), ('max_pool_3x3', 0, 0.1266123652458191), ('max_pool_3x3', 1, 0.12597426772117615), ('avg_pool_3x3', 0, 0.12625004351139069), ('dil_conv_3x3', 3, 0.12578943371772766), ('max_pool_3x3', 0, 0.12601076066493988), ('dil_conv_5x5', 2, 0.12574982643127441)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-15:15:39] Epoch: [071][000/196] Time 1.64 (1.64) Data 0.82 (0.82) Loss 1.491 (1.491) Prec@1 42.19 (42.19) Prec@5 91.41 (91.41) TRAIN-SEARCH [2018-10-02-15:16:35] Epoch: [071][100/196] Time 0.59 (0.58) Data 0.00 (0.01) Loss 1.380 (1.391) Prec@1 53.91 (49.76) Prec@5 93.75 (93.39) TRAIN-SEARCH [2018-10-02-15:17:25] Epoch: [071][195/196] Time 0.55 (0.55) Data 0.00 (0.01) Loss 1.303 (1.383) Prec@1 57.50 (50.23) Prec@5 90.00 (93.42) VALID-SEARCH [2018-10-02-15:17:26] Epoch: [071][000/196] Loss 1.684 (1.684) Prec@1 35.16 (35.16) Prec@5 88.28 (88.28) VALID-SEARCH [2018-10-02-15:17:31] Epoch: [071][100/196] Loss 1.505 (1.853) Prec@1 49.22 (39.03) Prec@5 92.97 (87.89) VALID-SEARCH [2018-10-02-15:17:36] Epoch: [071][195/196] Loss 2.204 (1.860) Prec@1 30.00 (38.51) Prec@5 87.50 (87.19) 071/250, Train-Accuracy = 50.23, Test-Accuracy = 38.51 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-15:17:37] [Epoch=072/250] [Need: 05:58:39] [LR=0.0406 ~ 0.0406] [Batch=128], tau=8.272 genotype = Genotype(normal=[('skip_connect', 0, 0.1289944350719452), ('dil_conv_3x3', 1, 0.12685620784759521), ('skip_connect', 0, 0.1284736543893814), ('sep_conv_3x3', 2, 0.12759564816951752), ('max_pool_3x3', 0, 0.12883450090885162), ('sep_conv_3x3', 3, 0.12744925916194916), ('max_pool_3x3', 0, 0.12768599390983582), ('dil_conv_5x5', 4, 0.12702125310897827)], normal_concat=range(2, 6), reduce=[('skip_connect', 0, 0.12616169452667236), ('max_pool_3x3', 1, 0.12548932433128357), ('skip_connect', 0, 0.12628567218780518), ('dil_conv_5x5', 2, 0.12594442069530487), ('max_pool_3x3', 0, 0.1261368840932846), ('dil_conv_5x5', 1, 0.1255403459072113), ('dil_conv_5x5', 1, 0.12668141722679138), ('dil_conv_5x5', 3, 0.1258038431406021)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-15:17:39] Epoch: [072][000/196] Time 1.44 (1.44) Data 0.87 (0.87) Loss 1.357 (1.357) Prec@1 53.12 (53.12) Prec@5 91.41 (91.41) TRAIN-SEARCH [2018-10-02-15:18:34] Epoch: [072][100/196] Time 0.61 (0.55) Data 0.00 (0.01) Loss 1.401 (1.392) Prec@1 46.88 (50.26) Prec@5 94.53 (93.25) TRAIN-SEARCH [2018-10-02-15:19:23] Epoch: [072][195/196] Time 0.46 (0.54) Data 0.00 (0.01) Loss 1.283 (1.387) Prec@1 50.00 (50.51) Prec@5 97.50 (93.44) VALID-SEARCH [2018-10-02-15:19:24] Epoch: [072][000/196] Loss 1.588 (1.588) Prec@1 31.25 (31.25) Prec@5 92.19 (92.19) VALID-SEARCH [2018-10-02-15:19:30] Epoch: [072][100/196] Loss 1.561 (1.825) Prec@1 46.09 (39.44) Prec@5 93.75 (88.58) VALID-SEARCH [2018-10-02-15:19:34] Epoch: [072][195/196] Loss 2.691 (1.820) Prec@1 32.50 (39.75) Prec@5 82.50 (88.55) 072/250, Train-Accuracy = 50.51, Test-Accuracy = 39.75 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-15:19:35] [Epoch=073/250] [Need: 05:46:38] [LR=0.0404 ~ 0.0404] [Batch=128], tau=8.248000000000001 genotype = Genotype(normal=[('skip_connect', 0, 0.12889400124549866), ('dil_conv_3x3', 1, 0.12658794224262238), ('skip_connect', 0, 0.128345787525177), ('sep_conv_5x5', 2, 0.12768380343914032), ('max_pool_3x3', 0, 0.12924028933048248), ('sep_conv_3x3', 3, 0.12691010534763336), ('dil_conv_5x5', 4, 0.12733644247055054), ('max_pool_3x3', 0, 0.12695194780826569)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.12659460306167603), ('skip_connect', 1, 0.12578408420085907), ('avg_pool_3x3', 1, 0.1262049376964569), ('skip_connect', 0, 0.12606863677501678), ('sep_conv_3x3', 2, 0.12618465721607208), ('max_pool_3x3', 0, 0.12617024779319763), ('sep_conv_3x3', 3, 0.12587836384773254), ('sep_conv_3x3', 2, 0.1256505697965622)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-15:19:36] Epoch: [073][000/196] Time 1.17 (1.17) Data 0.68 (0.68) Loss 1.461 (1.461) Prec@1 49.22 (49.22) Prec@5 92.97 (92.97) TRAIN-SEARCH [2018-10-02-15:20:32] Epoch: [073][100/196] Time 0.58 (0.56) Data 0.00 (0.01) Loss 1.350 (1.373) Prec@1 53.12 (50.65) Prec@5 93.75 (93.02) TRAIN-SEARCH [2018-10-02-15:21:20] Epoch: [073][195/196] Time 0.33 (0.54) Data 0.00 (0.01) Loss 1.273 (1.371) Prec@1 45.00 (50.40) Prec@5 92.50 (93.10) VALID-SEARCH [2018-10-02-15:21:21] Epoch: [073][000/196] Loss 1.748 (1.748) Prec@1 29.69 (29.69) Prec@5 85.16 (85.16) VALID-SEARCH [2018-10-02-15:21:26] Epoch: [073][100/196] Loss 2.155 (1.826) Prec@1 35.16 (37.37) Prec@5 88.28 (87.89) VALID-SEARCH [2018-10-02-15:21:30] Epoch: [073][195/196] Loss 2.187 (1.830) Prec@1 37.50 (37.00) Prec@5 72.50 (87.50) 073/250, Train-Accuracy = 50.40, Test-Accuracy = 37.00 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-15:21:31] [Epoch=074/250] [Need: 05:42:03] [LR=0.0401 ~ 0.0401] [Batch=128], tau=8.224 genotype = Genotype(normal=[('skip_connect', 0, 0.12896868586540222), ('skip_connect', 1, 0.1266276091337204), ('skip_connect', 0, 0.12890656292438507), ('sep_conv_3x3', 2, 0.12795789539813995), ('max_pool_3x3', 0, 0.12813235819339752), ('sep_conv_3x3', 3, 0.12727715075016022), ('dil_conv_5x5', 4, 0.12709303200244904), ('skip_connect', 0, 0.12683835625648499)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.12642206251621246), ('avg_pool_3x3', 1, 0.12573623657226562), ('avg_pool_3x3', 0, 0.12655407190322876), ('max_pool_3x3', 1, 0.12651021778583527), ('max_pool_3x3', 0, 0.12619294226169586), ('dil_conv_5x5', 1, 0.12573480606079102), ('avg_pool_3x3', 1, 0.12600603699684143), ('max_pool_3x3', 0, 0.12572059035301208)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-15:21:34] Epoch: [074][000/196] Time 1.61 (1.61) Data 0.92 (0.92) Loss 1.413 (1.413) Prec@1 48.44 (48.44) Prec@5 92.97 (92.97) TRAIN-SEARCH [2018-10-02-15:22:32] Epoch: [074][100/196] Time 0.63 (0.59) Data 0.00 (0.01) Loss 1.342 (1.368) Prec@1 53.12 (50.36) Prec@5 95.31 (93.56) TRAIN-SEARCH [2018-10-02-15:23:21] Epoch: [074][195/196] Time 0.46 (0.55) Data 0.00 (0.01) Loss 1.567 (1.367) Prec@1 55.00 (50.60) Prec@5 85.00 (93.66) VALID-SEARCH [2018-10-02-15:23:22] Epoch: [074][000/196] Loss 1.706 (1.706) Prec@1 37.50 (37.50) Prec@5 87.50 (87.50) VALID-SEARCH [2018-10-02-15:23:27] Epoch: [074][100/196] Loss 1.729 (1.809) Prec@1 41.41 (37.33) Prec@5 85.94 (85.26) VALID-SEARCH [2018-10-02-15:23:32] Epoch: [074][195/196] Loss 1.808 (1.787) Prec@1 32.50 (37.70) Prec@5 87.50 (85.58) 074/250, Train-Accuracy = 50.60, Test-Accuracy = 37.70 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-15:23:33] [Epoch=075/250] [Need: 05:55:32] [LR=0.0399 ~ 0.0399] [Batch=128], tau=8.2 genotype = Genotype(normal=[('skip_connect', 0, 0.12845824658870697), ('dil_conv_3x3', 1, 0.12691470980644226), ('skip_connect', 0, 0.12854044139385223), ('sep_conv_3x3', 2, 0.12721790373325348), ('sep_conv_3x3', 3, 0.12765762209892273), ('max_pool_3x3', 0, 0.12739673256874084), ('max_pool_3x3', 0, 0.1273777037858963), ('sep_conv_5x5', 4, 0.12719564139842987)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.12649409472942352), ('avg_pool_3x3', 1, 0.12641704082489014), ('avg_pool_3x3', 0, 0.12640273571014404), ('max_pool_3x3', 1, 0.12633603811264038), ('max_pool_3x3', 0, 0.12677150964736938), ('max_pool_3x3', 1, 0.1259184032678604), ('dil_conv_5x5', 1, 0.12590105831623077), ('dil_conv_5x5', 4, 0.12565681338310242)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-15:23:36] Epoch: [075][000/196] Time 1.66 (1.66) Data 0.96 (0.96) Loss 1.387 (1.387) Prec@1 51.56 (51.56) Prec@5 97.66 (97.66) TRAIN-SEARCH [2018-10-02-15:24:36] Epoch: [075][100/196] Time 0.52 (0.61) Data 0.00 (0.01) Loss 1.523 (1.345) Prec@1 44.53 (51.51) Prec@5 92.19 (93.69) TRAIN-SEARCH [2018-10-02-15:25:24] Epoch: [075][195/196] Time 0.59 (0.56) Data 0.00 (0.01) Loss 1.519 (1.352) Prec@1 35.00 (51.33) Prec@5 97.50 (93.77) VALID-SEARCH [2018-10-02-15:25:25] Epoch: [075][000/196] Loss 1.406 (1.406) Prec@1 51.56 (51.56) Prec@5 92.97 (92.97) VALID-SEARCH [2018-10-02-15:25:30] Epoch: [075][100/196] Loss 1.361 (1.861) Prec@1 49.22 (37.33) Prec@5 90.62 (86.39) VALID-SEARCH [2018-10-02-15:25:35] Epoch: [075][195/196] Loss 1.575 (1.824) Prec@1 37.50 (37.66) Prec@5 97.50 (86.82) 075/250, Train-Accuracy = 51.33, Test-Accuracy = 37.66 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-15:25:36] [Epoch=076/250] [Need: 05:56:17] [LR=0.0397 ~ 0.0397] [Batch=128], tau=8.176 genotype = Genotype(normal=[('skip_connect', 0, 0.12810102105140686), ('dil_conv_5x5', 1, 0.12647593021392822), ('sep_conv_3x3', 2, 0.1282993108034134), ('skip_connect', 0, 0.1282672882080078), ('sep_conv_3x3', 3, 0.12763361632823944), ('max_pool_3x3', 0, 0.12741421163082123), ('sep_conv_5x5', 4, 0.1274556815624237), ('max_pool_3x3', 0, 0.12720337510108948)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.12633241713047028), ('avg_pool_3x3', 1, 0.12556017935276031), ('max_pool_3x3', 1, 0.1263311803340912), ('sep_conv_3x3', 2, 0.1258813887834549), ('max_pool_3x3', 0, 0.12696991860866547), ('avg_pool_3x3', 1, 0.1265881061553955), ('dil_conv_5x5', 4, 0.1264331042766571), ('avg_pool_3x3', 0, 0.12602363526821136)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-15:25:38] Epoch: [076][000/196] Time 1.60 (1.60) Data 0.82 (0.82) Loss 1.369 (1.369) Prec@1 45.31 (45.31) Prec@5 96.09 (96.09) TRAIN-SEARCH [2018-10-02-15:26:37] Epoch: [076][100/196] Time 0.47 (0.60) Data 0.00 (0.01) Loss 1.463 (1.347) Prec@1 47.66 (51.81) Prec@5 92.97 (93.50) TRAIN-SEARCH [2018-10-02-15:27:26] Epoch: [076][195/196] Time 0.46 (0.56) Data 0.00 (0.01) Loss 1.126 (1.368) Prec@1 52.50 (50.95) Prec@5 97.50 (93.36) VALID-SEARCH [2018-10-02-15:27:27] Epoch: [076][000/196] Loss 1.843 (1.843) Prec@1 37.50 (37.50) Prec@5 82.81 (82.81) VALID-SEARCH [2018-10-02-15:27:33] Epoch: [076][100/196] Loss 1.498 (1.819) Prec@1 45.31 (37.91) Prec@5 94.53 (85.94) VALID-SEARCH [2018-10-02-15:27:37] Epoch: [076][195/196] Loss 1.387 (1.791) Prec@1 52.50 (38.39) Prec@5 95.00 (86.42) 076/250, Train-Accuracy = 50.95, Test-Accuracy = 38.39 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-15:27:38] [Epoch=077/250] [Need: 05:52:33] [LR=0.0394 ~ 0.0394] [Batch=128], tau=8.152000000000001 genotype = Genotype(normal=[('skip_connect', 0, 0.12804895639419556), ('dil_conv_5x5', 1, 0.12656712532043457), ('sep_conv_3x3', 2, 0.1282166987657547), ('skip_connect', 0, 0.12807554006576538), ('sep_conv_3x3', 3, 0.12755580246448517), ('dil_conv_3x3', 2, 0.12729133665561676), ('sep_conv_5x5', 4, 0.12790313363075256), ('max_pool_3x3', 0, 0.1277998387813568)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.12613816559314728), ('avg_pool_3x3', 1, 0.12560439109802246), ('max_pool_3x3', 1, 0.12571243941783905), ('max_pool_3x3', 0, 0.12565787136554718), ('max_pool_3x3', 0, 0.12717877328395844), ('max_pool_3x3', 1, 0.12579132616519928), ('dil_conv_5x5', 4, 0.1259334534406662), ('avg_pool_3x3', 0, 0.1258780062198639)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-15:27:41] Epoch: [077][000/196] Time 1.53 (1.53) Data 0.86 (0.86) Loss 1.244 (1.244) Prec@1 54.69 (54.69) Prec@5 92.19 (92.19) TRAIN-SEARCH [2018-10-02-15:28:38] Epoch: [077][100/196] Time 0.59 (0.58) Data 0.00 (0.01) Loss 1.406 (1.351) Prec@1 54.69 (51.86) Prec@5 94.53 (93.73) TRAIN-SEARCH [2018-10-02-15:29:29] Epoch: [077][195/196] Time 0.45 (0.56) Data 0.00 (0.01) Loss 1.551 (1.359) Prec@1 55.00 (51.33) Prec@5 90.00 (93.67) VALID-SEARCH [2018-10-02-15:29:30] Epoch: [077][000/196] Loss 1.521 (1.521) Prec@1 45.31 (45.31) Prec@5 87.50 (87.50) VALID-SEARCH [2018-10-02-15:29:36] Epoch: [077][100/196] Loss 1.667 (1.760) Prec@1 34.38 (38.85) Prec@5 88.28 (87.61) VALID-SEARCH [2018-10-02-15:29:41] Epoch: [077][195/196] Loss 1.699 (1.776) Prec@1 42.50 (39.25) Prec@5 87.50 (87.65) 077/250, Train-Accuracy = 51.33, Test-Accuracy = 39.25 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-15:29:42] [Epoch=078/250] [Need: 05:54:09] [LR=0.0391 ~ 0.0391] [Batch=128], tau=8.128 genotype = Genotype(normal=[('skip_connect', 0, 0.12776853144168854), ('skip_connect', 1, 0.1264771968126297), ('skip_connect', 0, 0.12883985042572021), ('sep_conv_3x3', 2, 0.12778815627098083), ('sep_conv_3x3', 3, 0.1275528520345688), ('dil_conv_3x3', 2, 0.12709960341453552), ('max_pool_3x3', 0, 0.12829448282718658), ('dil_conv_5x5', 4, 0.12724822759628296)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.12673652172088623), ('skip_connect', 1, 0.1256270855665207), ('max_pool_3x3', 0, 0.12611955404281616), ('skip_connect', 1, 0.12568295001983643), ('max_pool_3x3', 0, 0.1264934241771698), ('max_pool_3x3', 1, 0.1257999837398529), ('dil_conv_5x5', 3, 0.12618544697761536), ('avg_pool_3x3', 1, 0.1260644644498825)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-15:29:44] Epoch: [078][000/196] Time 1.40 (1.40) Data 0.79 (0.79) Loss 1.477 (1.477) Prec@1 48.44 (48.44) Prec@5 95.31 (95.31) TRAIN-SEARCH [2018-10-02-15:30:40] Epoch: [078][100/196] Time 0.58 (0.56) Data 0.00 (0.01) Loss 1.253 (1.349) Prec@1 53.12 (51.82) Prec@5 94.53 (93.87) TRAIN-SEARCH [2018-10-02-15:31:32] Epoch: [078][195/196] Time 0.52 (0.56) Data 0.00 (0.01) Loss 1.500 (1.345) Prec@1 50.00 (51.77) Prec@5 92.50 (93.83) VALID-SEARCH [2018-10-02-15:31:33] Epoch: [078][000/196] Loss 1.900 (1.900) Prec@1 32.81 (32.81) Prec@5 78.91 (78.91) VALID-SEARCH [2018-10-02-15:31:39] Epoch: [078][100/196] Loss 1.325 (1.760) Prec@1 49.22 (39.87) Prec@5 93.75 (88.32) VALID-SEARCH [2018-10-02-15:31:44] Epoch: [078][195/196] Loss 1.870 (1.786) Prec@1 35.00 (40.22) Prec@5 87.50 (88.70) 078/250, Train-Accuracy = 51.77, Test-Accuracy = 40.22 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-15:31:45] [Epoch=079/250] [Need: 05:50:10] [LR=0.0389 ~ 0.0389] [Batch=128], tau=8.104 genotype = Genotype(normal=[('skip_connect', 0, 0.12847614288330078), ('skip_connect', 1, 0.1266787052154541), ('skip_connect', 0, 0.12837466597557068), ('sep_conv_3x3', 2, 0.127577543258667), ('skip_connect', 0, 0.12696968019008636), ('skip_connect', 1, 0.12680594623088837), ('max_pool_3x3', 0, 0.12882669270038605), ('sep_conv_5x5', 4, 0.12714628875255585)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.12568487226963043), ('dil_conv_3x3', 1, 0.1255687177181244), ('max_pool_3x3', 0, 0.12665244936943054), ('avg_pool_3x3', 1, 0.1258854866027832), ('max_pool_3x3', 0, 0.12645134329795837), ('dil_conv_3x3', 2, 0.12585021555423737), ('max_pool_3x3', 0, 0.1264800876379013), ('dil_conv_5x5', 2, 0.1258227825164795)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-15:31:47] Epoch: [079][000/196] Time 1.47 (1.47) Data 0.85 (0.85) Loss 1.321 (1.321) Prec@1 51.56 (51.56) Prec@5 93.75 (93.75) TRAIN-SEARCH [2018-10-02-15:32:41] Epoch: [079][100/196] Time 0.66 (0.55) Data 0.03 (0.01) Loss 1.216 (1.347) Prec@1 56.25 (51.71) Prec@5 93.75 (94.06) TRAIN-SEARCH [2018-10-02-15:33:30] Epoch: [079][195/196] Time 0.42 (0.53) Data 0.00 (0.01) Loss 1.112 (1.344) Prec@1 62.50 (51.75) Prec@5 97.50 (93.99) VALID-SEARCH [2018-10-02-15:33:32] Epoch: [079][000/196] Loss 1.827 (1.827) Prec@1 39.84 (39.84) Prec@5 91.41 (91.41) VALID-SEARCH [2018-10-02-15:33:37] Epoch: [079][100/196] Loss 1.582 (1.829) Prec@1 45.31 (40.07) Prec@5 89.06 (88.33) VALID-SEARCH [2018-10-02-15:33:41] Epoch: [079][195/196] Loss 1.535 (1.811) Prec@1 45.00 (39.66) Prec@5 95.00 (87.86) 079/250, Train-Accuracy = 51.75, Test-Accuracy = 39.66 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-15:33:41] [Epoch=080/250] [Need: 05:30:53] [LR=0.0386 ~ 0.0386] [Batch=128], tau=8.08 genotype = Genotype(normal=[('skip_connect', 0, 0.1283828467130661), ('dil_conv_5x5', 1, 0.12669737637043), ('skip_connect', 0, 0.1291746199131012), ('sep_conv_5x5', 2, 0.12729573249816895), ('skip_connect', 0, 0.12756820023059845), ('sep_conv_3x3', 3, 0.12729747593402863), ('max_pool_3x3', 0, 0.1286323219537735), ('sep_conv_3x3', 4, 0.1270609349012375)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.12609824538230896), ('dil_conv_3x3', 1, 0.12598916888237), ('max_pool_3x3', 1, 0.12626726925373077), ('dil_conv_3x3', 2, 0.1262245625257492), ('max_pool_3x3', 0, 0.12660473585128784), ('dil_conv_3x3', 2, 0.12615767121315002), ('dil_conv_5x5', 3, 0.12593775987625122), ('max_pool_3x3', 0, 0.1257636994123459)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-15:33:43] Epoch: [080][000/196] Time 1.13 (1.13) Data 0.67 (0.67) Loss 1.438 (1.438) Prec@1 47.66 (47.66) Prec@5 93.75 (93.75) TRAIN-SEARCH [2018-10-02-15:34:41] Epoch: [080][100/196] Time 0.57 (0.58) Data 0.00 (0.01) Loss 1.156 (1.329) Prec@1 56.25 (52.41) Prec@5 98.44 (93.98) TRAIN-SEARCH [2018-10-02-15:35:30] Epoch: [080][195/196] Time 0.34 (0.55) Data 0.00 (0.01) Loss 1.216 (1.331) Prec@1 60.00 (52.37) Prec@5 92.50 (93.96) VALID-SEARCH [2018-10-02-15:35:31] Epoch: [080][000/196] Loss 1.807 (1.807) Prec@1 38.28 (38.28) Prec@5 81.25 (81.25) VALID-SEARCH [2018-10-02-15:35:36] Epoch: [080][100/196] Loss 1.928 (1.784) Prec@1 39.84 (40.67) Prec@5 92.19 (87.82) VALID-SEARCH [2018-10-02-15:35:41] Epoch: [080][195/196] Loss 1.520 (1.792) Prec@1 57.50 (41.44) Prec@5 92.50 (88.38) 080/250, Train-Accuracy = 52.37, Test-Accuracy = 41.44 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-15:35:43] [Epoch=081/250] [Need: 05:41:00] [LR=0.0384 ~ 0.0384] [Batch=128], tau=8.056000000000001 genotype = Genotype(normal=[('dil_conv_3x3', 0, 0.12815211713314056), ('dil_conv_5x5', 1, 0.12677045166492462), ('skip_connect', 0, 0.12964065372943878), ('sep_conv_3x3', 2, 0.1274903565645218), ('skip_connect', 0, 0.12777873873710632), ('sep_conv_5x5', 3, 0.12699401378631592), ('max_pool_3x3', 0, 0.12845231592655182), ('sep_conv_5x5', 4, 0.12725646793842316)], normal_concat=range(2, 6), reduce=[('skip_connect', 0, 0.12621575593948364), ('dil_conv_3x3', 1, 0.12548872828483582), ('max_pool_3x3', 0, 0.12659014761447906), ('sep_conv_3x3', 2, 0.12592267990112305), ('sep_conv_5x5', 1, 0.1259380429983139), ('max_pool_3x3', 0, 0.12577131390571594), ('max_pool_3x3', 0, 0.12585781514644623), ('dil_conv_3x3', 4, 0.12582062184810638)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-15:35:45] Epoch: [081][000/196] Time 1.55 (1.55) Data 0.91 (0.91) Loss 1.158 (1.158) Prec@1 56.25 (56.25) Prec@5 98.44 (98.44) TRAIN-SEARCH [2018-10-02-15:36:44] Epoch: [081][100/196] Time 0.73 (0.60) Data 0.01 (0.01) Loss 1.241 (1.362) Prec@1 56.25 (51.83) Prec@5 92.97 (93.39) TRAIN-SEARCH [2018-10-02-15:37:32] Epoch: [081][195/196] Time 0.44 (0.55) Data 0.00 (0.01) Loss 1.562 (1.342) Prec@1 47.50 (52.72) Prec@5 87.50 (93.62) VALID-SEARCH [2018-10-02-15:37:33] Epoch: [081][000/196] Loss 1.450 (1.450) Prec@1 44.53 (44.53) Prec@5 92.19 (92.19) VALID-SEARCH [2018-10-02-15:37:38] Epoch: [081][100/196] Loss 1.621 (1.757) Prec@1 42.19 (40.71) Prec@5 88.28 (88.68) VALID-SEARCH [2018-10-02-15:37:43] Epoch: [081][195/196] Loss 1.610 (1.756) Prec@1 42.50 (40.33) Prec@5 87.50 (88.42) 081/250, Train-Accuracy = 52.72, Test-Accuracy = 40.33 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-15:37:44] [Epoch=082/250] [Need: 05:40:56] [LR=0.0381 ~ 0.0381] [Batch=128], tau=8.032 genotype = Genotype(normal=[('dil_conv_3x3', 0, 0.12904506921768188), ('dil_conv_5x5', 1, 0.1270214021205902), ('skip_connect', 0, 0.12958502769470215), ('sep_conv_3x3', 2, 0.12824681401252747), ('skip_connect', 0, 0.12787406146526337), ('dil_conv_5x5', 3, 0.12757918238639832), ('max_pool_3x3', 0, 0.12783044576644897), ('sep_conv_3x3', 4, 0.12768253684043884)], normal_concat=range(2, 6), reduce=[('dil_conv_5x5', 0, 0.1261911243200302), ('max_pool_3x3', 1, 0.12535938620567322), ('max_pool_3x3', 0, 0.1264524757862091), ('max_pool_3x3', 1, 0.12585602700710297), ('sep_conv_5x5', 1, 0.1261732280254364), ('max_pool_3x3', 0, 0.12600961327552795), ('dil_conv_5x5', 1, 0.126339390873909), ('skip_connect', 0, 0.12623387575149536)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-15:37:47] Epoch: [082][000/196] Time 1.69 (1.69) Data 0.89 (0.89) Loss 1.217 (1.217) Prec@1 53.91 (53.91) Prec@5 93.75 (93.75) TRAIN-SEARCH [2018-10-02-15:38:47] Epoch: [082][100/196] Time 0.63 (0.61) Data 0.00 (0.01) Loss 1.282 (1.328) Prec@1 56.25 (52.72) Prec@5 94.53 (94.21) TRAIN-SEARCH [2018-10-02-15:39:35] Epoch: [082][195/196] Time 0.47 (0.56) Data 0.00 (0.01) Loss 1.275 (1.320) Prec@1 52.50 (53.24) Prec@5 95.00 (94.33) VALID-SEARCH [2018-10-02-15:39:36] Epoch: [082][000/196] Loss 1.442 (1.442) Prec@1 50.00 (50.00) Prec@5 92.97 (92.97) VALID-SEARCH [2018-10-02-15:39:42] Epoch: [082][100/196] Loss 1.992 (1.752) Prec@1 29.69 (40.73) Prec@5 85.94 (88.92) VALID-SEARCH [2018-10-02-15:39:47] Epoch: [082][195/196] Loss 1.668 (1.756) Prec@1 32.50 (40.68) Prec@5 87.50 (88.77) 082/250, Train-Accuracy = 53.24, Test-Accuracy = 40.68 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-15:39:48] [Epoch=083/250] [Need: 05:43:53] [LR=0.0378 ~ 0.0378] [Batch=128], tau=8.008 genotype = Genotype(normal=[('dil_conv_3x3', 0, 0.12841853499412537), ('dil_conv_5x5', 1, 0.12714172899723053), ('skip_connect', 0, 0.12920497357845306), ('sep_conv_5x5', 2, 0.12786218523979187), ('skip_connect', 0, 0.1280999332666397), ('dil_conv_5x5', 2, 0.12743385136127472), ('sep_conv_5x5', 4, 0.12774789333343506), ('max_pool_3x3', 0, 0.1277250349521637)], normal_concat=range(2, 6), reduce=[('dil_conv_5x5', 0, 0.1262410581111908), ('max_pool_3x3', 1, 0.1257425844669342), ('max_pool_3x3', 0, 0.12626193463802338), ('skip_connect', 1, 0.12566910684108734), ('avg_pool_3x3', 1, 0.12584072351455688), ('avg_pool_3x3', 0, 0.12580136954784393), ('skip_connect', 0, 0.1264050304889679), ('max_pool_3x3', 1, 0.12604466080665588)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-15:39:50] Epoch: [083][000/196] Time 1.69 (1.69) Data 0.96 (0.96) Loss 1.261 (1.261) Prec@1 53.91 (53.91) Prec@5 94.53 (94.53) TRAIN-SEARCH [2018-10-02-15:40:46] Epoch: [083][100/196] Time 0.52 (0.57) Data 0.00 (0.01) Loss 1.458 (1.320) Prec@1 48.44 (53.21) Prec@5 92.19 (93.86) TRAIN-SEARCH [2018-10-02-15:41:36] Epoch: [083][195/196] Time 0.42 (0.55) Data 0.00 (0.01) Loss 1.196 (1.319) Prec@1 52.50 (52.86) Prec@5 95.00 (93.95) VALID-SEARCH [2018-10-02-15:41:37] Epoch: [083][000/196] Loss 1.563 (1.563) Prec@1 36.72 (36.72) Prec@5 87.50 (87.50) VALID-SEARCH [2018-10-02-15:41:42] Epoch: [083][100/196] Loss 1.674 (1.639) Prec@1 42.19 (41.85) Prec@5 85.94 (89.05) VALID-SEARCH [2018-10-02-15:41:47] Epoch: [083][195/196] Loss 1.415 (1.678) Prec@1 45.00 (41.08) Prec@5 92.50 (88.80) 083/250, Train-Accuracy = 52.86, Test-Accuracy = 41.08 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-15:41:48] [Epoch=084/250] [Need: 05:33:24] [LR=0.0376 ~ 0.0376] [Batch=128], tau=7.984 genotype = Genotype(normal=[('dil_conv_3x3', 0, 0.1283925622701645), ('skip_connect', 1, 0.1267564445734024), ('skip_connect', 0, 0.1294906586408615), ('sep_conv_5x5', 2, 0.127482071518898), ('skip_connect', 0, 0.12835027277469635), ('sep_conv_3x3', 3, 0.12706726789474487), ('max_pool_3x3', 0, 0.1284628063440323), ('sep_conv_3x3', 4, 0.1271355301141739)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 1, 0.1258746087551117), ('max_pool_3x3', 0, 0.12575459480285645), ('avg_pool_3x3', 0, 0.12661024928092957), ('max_pool_3x3', 1, 0.12585510313510895), ('max_pool_3x3', 0, 0.12604065239429474), ('max_pool_3x3', 2, 0.1256876438856125), ('skip_connect', 0, 0.12621285021305084), ('dil_conv_5x5', 1, 0.12572528421878815)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-15:41:51] Epoch: [084][000/196] Time 1.82 (1.82) Data 1.13 (1.13) Loss 1.213 (1.213) Prec@1 51.56 (51.56) Prec@5 95.31 (95.31) TRAIN-SEARCH [2018-10-02-15:42:46] Epoch: [084][100/196] Time 0.65 (0.56) Data 0.00 (0.01) Loss 1.494 (1.310) Prec@1 51.56 (52.89) Prec@5 92.97 (94.11) TRAIN-SEARCH [2018-10-02-15:43:36] Epoch: [084][195/196] Time 0.50 (0.54) Data 0.00 (0.01) Loss 1.116 (1.303) Prec@1 62.50 (53.24) Prec@5 100.00 (94.24) VALID-SEARCH [2018-10-02-15:43:37] Epoch: [084][000/196] Loss 1.517 (1.517) Prec@1 42.97 (42.97) Prec@5 93.75 (93.75) VALID-SEARCH [2018-10-02-15:43:42] Epoch: [084][100/196] Loss 1.842 (1.731) Prec@1 36.72 (41.12) Prec@5 82.03 (88.41) VALID-SEARCH [2018-10-02-15:43:47] Epoch: [084][195/196] Loss 1.729 (1.732) Prec@1 35.00 (41.27) Prec@5 90.00 (88.18) 084/250, Train-Accuracy = 53.24, Test-Accuracy = 41.27 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-15:43:48] [Epoch=085/250] [Need: 05:28:55] [LR=0.0373 ~ 0.0373] [Batch=128], tau=7.96 genotype = Genotype(normal=[('dil_conv_3x3', 0, 0.12820222973823547), ('skip_connect', 1, 0.12666979432106018), ('skip_connect', 0, 0.12914729118347168), ('sep_conv_5x5', 2, 0.12735332548618317), ('skip_connect', 0, 0.12822575867176056), ('dil_conv_5x5', 2, 0.12670448422431946), ('max_pool_3x3', 0, 0.12765927612781525), ('dil_conv_3x3', 4, 0.12717388570308685)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.12591907382011414), ('avg_pool_3x3', 1, 0.12575192749500275), ('avg_pool_3x3', 0, 0.12688815593719482), ('max_pool_3x3', 1, 0.12626077234745026), ('max_pool_3x3', 0, 0.12610116600990295), ('sep_conv_3x3', 2, 0.12581917643547058), ('max_pool_3x3', 0, 0.12603983283042908), ('dil_conv_3x3', 3, 0.12602080404758453)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-15:43:50] Epoch: [085][000/196] Time 1.57 (1.57) Data 0.95 (0.95) Loss 1.250 (1.250) Prec@1 50.78 (50.78) Prec@5 93.75 (93.75) TRAIN-SEARCH [2018-10-02-15:44:46] Epoch: [085][100/196] Time 0.55 (0.57) Data 0.00 (0.01) Loss 1.178 (1.291) Prec@1 54.69 (53.91) Prec@5 95.31 (94.17) TRAIN-SEARCH [2018-10-02-15:45:37] Epoch: [085][195/196] Time 0.43 (0.55) Data 0.00 (0.01) Loss 1.372 (1.297) Prec@1 45.00 (53.86) Prec@5 97.50 (94.20) VALID-SEARCH [2018-10-02-15:45:39] Epoch: [085][000/196] Loss 1.484 (1.484) Prec@1 50.78 (50.78) Prec@5 93.75 (93.75) VALID-SEARCH [2018-10-02-15:45:44] Epoch: [085][100/196] Loss 1.341 (1.750) Prec@1 46.88 (42.57) Prec@5 96.88 (90.08) VALID-SEARCH [2018-10-02-15:45:49] Epoch: [085][195/196] Loss 2.352 (1.782) Prec@1 42.50 (41.56) Prec@5 95.00 (89.22) 085/250, Train-Accuracy = 53.86, Test-Accuracy = 41.56 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-15:45:50] [Epoch=086/250] [Need: 05:32:59] [LR=0.0370 ~ 0.0370] [Batch=128], tau=7.936 genotype = Genotype(normal=[('skip_connect', 0, 0.12752509117126465), ('dil_conv_5x5', 1, 0.1270996630191803), ('skip_connect', 0, 0.1294616460800171), ('sep_conv_3x3', 2, 0.12770585715770721), ('skip_connect', 0, 0.12859347462654114), ('sep_conv_3x3', 2, 0.12681534886360168), ('max_pool_3x3', 0, 0.12758365273475647), ('dil_conv_3x3', 3, 0.12733741104602814)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.1261206716299057), ('sep_conv_5x5', 1, 0.1253841668367386), ('avg_pool_3x3', 0, 0.12672872841358185), ('max_pool_3x3', 1, 0.12611865997314453), ('sep_conv_5x5', 2, 0.1261388212442398), ('max_pool_3x3', 1, 0.1260673701763153), ('dil_conv_5x5', 2, 0.1261342316865921), ('sep_conv_5x5', 3, 0.12583020329475403)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-15:45:51] Epoch: [086][000/196] Time 1.13 (1.13) Data 0.67 (0.67) Loss 1.367 (1.367) Prec@1 52.34 (52.34) Prec@5 92.19 (92.19) TRAIN-SEARCH [2018-10-02-15:46:46] Epoch: [086][100/196] Time 0.39 (0.55) Data 0.00 (0.01) Loss 1.216 (1.314) Prec@1 53.91 (53.18) Prec@5 93.75 (93.91) TRAIN-SEARCH [2018-10-02-15:47:38] Epoch: [086][195/196] Time 0.37 (0.55) Data 0.00 (0.01) Loss 1.511 (1.319) Prec@1 45.00 (52.83) Prec@5 92.50 (93.82) VALID-SEARCH [2018-10-02-15:47:39] Epoch: [086][000/196] Loss 2.018 (2.018) Prec@1 25.00 (25.00) Prec@5 81.25 (81.25) VALID-SEARCH [2018-10-02-15:47:43] Epoch: [086][100/196] Loss 1.528 (1.679) Prec@1 37.50 (42.45) Prec@5 90.62 (88.88) VALID-SEARCH [2018-10-02-15:47:47] Epoch: [086][195/196] Loss 1.532 (1.667) Prec@1 35.00 (42.22) Prec@5 92.50 (88.82) 086/250, Train-Accuracy = 52.83, Test-Accuracy = 42.22 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-15:47:48] [Epoch=087/250] [Need: 05:21:15] [LR=0.0368 ~ 0.0368] [Batch=128], tau=7.912 genotype = Genotype(normal=[('skip_connect', 0, 0.12766441702842712), ('dil_conv_5x5', 1, 0.12710584700107574), ('skip_connect', 0, 0.13022293150424957), ('sep_conv_3x3', 2, 0.12773528695106506), ('skip_connect', 0, 0.12833984196186066), ('sep_conv_5x5', 3, 0.12708625197410583), ('max_pool_3x3', 0, 0.12758412957191467), ('dil_conv_3x3', 3, 0.12728756666183472)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.12652377784252167), ('dil_conv_5x5', 1, 0.1254502832889557), ('avg_pool_3x3', 0, 0.12677361071109772), ('max_pool_3x3', 1, 0.12579844892024994), ('avg_pool_3x3', 0, 0.1262470930814743), ('max_pool_3x3', 1, 0.12581609189510345), ('max_pool_3x3', 1, 0.12624357640743256), ('sep_conv_5x5', 0, 0.12601245939731598)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-15:47:50] Epoch: [087][000/196] Time 1.41 (1.41) Data 0.96 (0.96) Loss 1.138 (1.138) Prec@1 57.03 (57.03) Prec@5 96.09 (96.09) TRAIN-SEARCH [2018-10-02-15:48:47] Epoch: [087][100/196] Time 0.40 (0.57) Data 0.00 (0.01) Loss 1.932 (1.298) Prec@1 43.75 (53.81) Prec@5 84.38 (94.11) TRAIN-SEARCH [2018-10-02-15:49:36] Epoch: [087][195/196] Time 0.47 (0.55) Data 0.00 (0.01) Loss 1.145 (1.307) Prec@1 52.50 (53.20) Prec@5 100.00 (94.14) VALID-SEARCH [2018-10-02-15:49:37] Epoch: [087][000/196] Loss 1.609 (1.609) Prec@1 46.09 (46.09) Prec@5 88.28 (88.28) VALID-SEARCH [2018-10-02-15:49:43] Epoch: [087][100/196] Loss 1.565 (1.705) Prec@1 43.75 (41.38) Prec@5 90.62 (88.97) VALID-SEARCH [2018-10-02-15:49:48] Epoch: [087][195/196] Loss 1.300 (1.712) Prec@1 55.00 (40.96) Prec@5 95.00 (88.76) 087/250, Train-Accuracy = 53.20, Test-Accuracy = 40.96 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-15:49:49] [Epoch=088/250] [Need: 05:25:59] [LR=0.0365 ~ 0.0365] [Batch=128], tau=7.888 genotype = Genotype(normal=[('skip_connect', 0, 0.12832006812095642), ('skip_connect', 1, 0.12617701292037964), ('skip_connect', 0, 0.12992560863494873), ('sep_conv_3x3', 2, 0.1277996301651001), ('skip_connect', 0, 0.12872231006622314), ('sep_conv_5x5', 3, 0.12750858068466187), ('sep_conv_5x5', 4, 0.12800994515419006), ('max_pool_3x3', 0, 0.12762869894504547)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.12643717229366302), ('dil_conv_5x5', 1, 0.12553246319293976), ('skip_connect', 0, 0.12605048716068268), ('max_pool_3x3', 1, 0.1259005069732666), ('avg_pool_3x3', 0, 0.1265667974948883), ('max_pool_3x3', 1, 0.1256677359342575), ('max_pool_3x3', 1, 0.1263386607170105), ('dil_conv_5x5', 4, 0.1260206401348114)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-15:49:51] Epoch: [088][000/196] Time 1.55 (1.55) Data 0.82 (0.82) Loss 1.236 (1.236) Prec@1 52.34 (52.34) Prec@5 92.97 (92.97) TRAIN-SEARCH [2018-10-02-15:50:46] Epoch: [088][100/196] Time 0.42 (0.56) Data 0.00 (0.01) Loss 1.225 (1.303) Prec@1 58.59 (53.89) Prec@5 92.19 (94.28) TRAIN-SEARCH [2018-10-02-15:51:35] Epoch: [088][195/196] Time 0.54 (0.54) Data 0.00 (0.01) Loss 1.362 (1.307) Prec@1 47.50 (53.82) Prec@5 100.00 (94.25) VALID-SEARCH [2018-10-02-15:51:36] Epoch: [088][000/196] Loss 1.687 (1.687) Prec@1 45.31 (45.31) Prec@5 87.50 (87.50) VALID-SEARCH [2018-10-02-15:51:41] Epoch: [088][100/196] Loss 2.354 (1.763) Prec@1 25.00 (41.76) Prec@5 69.53 (88.51) VALID-SEARCH [2018-10-02-15:51:46] Epoch: [088][195/196] Loss 1.586 (1.738) Prec@1 40.00 (41.21) Prec@5 92.50 (88.30) 088/250, Train-Accuracy = 53.82, Test-Accuracy = 41.21 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-15:51:47] [Epoch=089/250] [Need: 05:17:15] [LR=0.0362 ~ 0.0362] [Batch=128], tau=7.864 genotype = Genotype(normal=[('skip_connect', 0, 0.1288815289735794), ('skip_connect', 1, 0.1268056333065033), ('skip_connect', 0, 0.12951363623142242), ('sep_conv_3x3', 2, 0.12780220806598663), ('skip_connect', 0, 0.12941110134124756), ('dil_conv_3x3', 3, 0.127215176820755), ('max_pool_3x3', 0, 0.1275685727596283), ('sep_conv_5x5', 4, 0.1273176074028015)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.12631861865520477), ('sep_conv_3x3', 1, 0.12534263730049133), ('max_pool_3x3', 0, 0.1267886459827423), ('max_pool_3x3', 1, 0.12623077630996704), ('avg_pool_3x3', 0, 0.12702907621860504), ('max_pool_3x3', 1, 0.125852569937706), ('max_pool_3x3', 0, 0.12655143439769745), ('max_pool_3x3', 1, 0.12618112564086914)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-15:51:49] Epoch: [089][000/196] Time 1.58 (1.58) Data 0.97 (0.97) Loss 1.055 (1.055) Prec@1 58.59 (58.59) Prec@5 97.66 (97.66) TRAIN-SEARCH [2018-10-02-15:52:44] Epoch: [089][100/196] Time 0.42 (0.56) Data 0.00 (0.01) Loss 1.210 (1.296) Prec@1 55.47 (53.53) Prec@5 95.31 (94.11) TRAIN-SEARCH [2018-10-02-15:53:33] Epoch: [089][195/196] Time 0.45 (0.54) Data 0.00 (0.01) Loss 1.264 (1.301) Prec@1 60.00 (53.74) Prec@5 90.00 (94.08) VALID-SEARCH [2018-10-02-15:53:34] Epoch: [089][000/196] Loss 1.482 (1.482) Prec@1 39.84 (39.84) Prec@5 96.09 (96.09) VALID-SEARCH [2018-10-02-15:53:40] Epoch: [089][100/196] Loss 1.790 (1.720) Prec@1 36.72 (39.59) Prec@5 86.72 (87.43) VALID-SEARCH [2018-10-02-15:53:45] Epoch: [089][195/196] Loss 1.627 (1.745) Prec@1 37.50 (39.00) Prec@5 87.50 (86.96) 089/250, Train-Accuracy = 53.74, Test-Accuracy = 39.00 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-15:53:46] [Epoch=090/250] [Need: 05:18:30] [LR=0.0359 ~ 0.0359] [Batch=128], tau=7.84 genotype = Genotype(normal=[('skip_connect', 0, 0.128932386636734), ('dil_conv_3x3', 1, 0.12645921111106873), ('skip_connect', 0, 0.13021577894687653), ('sep_conv_3x3', 2, 0.1273408979177475), ('skip_connect', 0, 0.12877905368804932), ('sep_conv_5x5', 3, 0.12730278074741364), ('max_pool_3x3', 0, 0.12795618176460266), ('sep_conv_5x5', 4, 0.12729576230049133)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.12586110830307007), ('sep_conv_3x3', 1, 0.12544740736484528), ('avg_pool_3x3', 0, 0.12649518251419067), ('max_pool_3x3', 1, 0.12634871900081635), ('max_pool_3x3', 0, 0.12673886120319366), ('sep_conv_5x5', 1, 0.1257377713918686), ('max_pool_3x3', 0, 0.12588170170783997), ('max_pool_3x3', 1, 0.1255672574043274)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-15:53:49] Epoch: [090][000/196] Time 1.47 (1.47) Data 0.98 (0.98) Loss 1.237 (1.237) Prec@1 56.25 (56.25) Prec@5 92.97 (92.97) TRAIN-SEARCH [2018-10-02-15:54:44] Epoch: [090][100/196] Time 0.39 (0.56) Data 0.00 (0.01) Loss 1.169 (1.293) Prec@1 53.91 (53.93) Prec@5 93.75 (94.20) TRAIN-SEARCH [2018-10-02-15:55:35] Epoch: [090][195/196] Time 0.40 (0.55) Data 0.00 (0.01) Loss 1.316 (1.282) Prec@1 47.50 (54.28) Prec@5 92.50 (94.26) VALID-SEARCH [2018-10-02-15:55:36] Epoch: [090][000/196] Loss 1.344 (1.344) Prec@1 52.34 (52.34) Prec@5 92.19 (92.19) VALID-SEARCH [2018-10-02-15:55:42] Epoch: [090][100/196] Loss 1.686 (1.638) Prec@1 42.97 (42.90) Prec@5 89.06 (89.30) VALID-SEARCH [2018-10-02-15:55:47] Epoch: [090][195/196] Loss 1.722 (1.692) Prec@1 32.50 (41.60) Prec@5 92.50 (88.28) 090/250, Train-Accuracy = 54.28, Test-Accuracy = 41.60 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-15:55:48] [Epoch=091/250] [Need: 05:21:27] [LR=0.0357 ~ 0.0357] [Batch=128], tau=7.816 genotype = Genotype(normal=[('skip_connect', 0, 0.12835736572742462), ('dil_conv_5x5', 1, 0.12706606090068817), ('skip_connect', 0, 0.13018490374088287), ('sep_conv_3x3', 2, 0.12740673124790192), ('skip_connect', 0, 0.1280573457479477), ('skip_connect', 1, 0.127559632062912), ('max_pool_3x3', 1, 0.12770310044288635), ('dil_conv_5x5', 4, 0.12759292125701904)], normal_concat=range(2, 6), reduce=[('skip_connect', 0, 0.1258147805929184), ('max_pool_3x3', 1, 0.12553703784942627), ('dil_conv_5x5', 0, 0.1263744980096817), ('max_pool_3x3', 1, 0.1260269731283188), ('max_pool_3x3', 0, 0.12628982961177826), ('dil_conv_5x5', 3, 0.12566038966178894), ('max_pool_3x3', 0, 0.12599073350429535), ('avg_pool_3x3', 1, 0.12590479850769043)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-15:55:50] Epoch: [091][000/196] Time 1.62 (1.62) Data 1.03 (1.03) Loss 1.313 (1.313) Prec@1 55.47 (55.47) Prec@5 94.53 (94.53) TRAIN-SEARCH [2018-10-02-15:56:42] Epoch: [091][100/196] Time 0.44 (0.53) Data 0.00 (0.01) Loss 1.232 (1.267) Prec@1 62.50 (54.97) Prec@5 95.31 (94.03) TRAIN-SEARCH [2018-10-02-15:57:34] Epoch: [091][195/196] Time 0.41 (0.54) Data 0.00 (0.01) Loss 1.064 (1.277) Prec@1 67.50 (54.69) Prec@5 92.50 (94.16) VALID-SEARCH [2018-10-02-15:57:35] Epoch: [091][000/196] Loss 1.828 (1.828) Prec@1 35.94 (35.94) Prec@5 82.03 (82.03) VALID-SEARCH [2018-10-02-15:57:41] Epoch: [091][100/196] Loss 1.509 (1.699) Prec@1 46.88 (43.26) Prec@5 92.97 (88.74) VALID-SEARCH [2018-10-02-15:57:46] Epoch: [091][195/196] Loss 2.006 (1.678) Prec@1 40.00 (43.33) Prec@5 90.00 (89.23) 091/250, Train-Accuracy = 54.69, Test-Accuracy = 43.33 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-15:57:47] [Epoch=092/250] [Need: 05:13:14] [LR=0.0354 ~ 0.0354] [Batch=128], tau=7.792 genotype = Genotype(normal=[('skip_connect', 0, 0.12770822644233704), ('sep_conv_5x5', 1, 0.12732458114624023), ('skip_connect', 0, 0.13020046055316925), ('sep_conv_3x3', 2, 0.12754593789577484), ('skip_connect', 0, 0.1283998191356659), ('sep_conv_3x3', 2, 0.12711510062217712), ('dil_conv_5x5', 4, 0.12761472165584564), ('max_pool_3x3', 1, 0.1272658407688141)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 1, 0.1262604147195816), ('avg_pool_3x3', 0, 0.12606273591518402), ('dil_conv_5x5', 1, 0.12639515101909637), ('dil_conv_5x5', 0, 0.1262998878955841), ('max_pool_3x3', 0, 0.1263911873102188), ('max_pool_3x3', 1, 0.12577255070209503), ('avg_pool_3x3', 1, 0.12628008425235748), ('max_pool_3x3', 0, 0.12596121430397034)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-15:57:49] Epoch: [092][000/196] Time 1.55 (1.55) Data 0.94 (0.94) Loss 1.125 (1.125) Prec@1 64.06 (64.06) Prec@5 96.09 (96.09) TRAIN-SEARCH [2018-10-02-15:58:42] Epoch: [092][100/196] Time 0.39 (0.54) Data 0.00 (0.01) Loss 1.235 (1.287) Prec@1 54.69 (54.66) Prec@5 95.31 (94.35) TRAIN-SEARCH [2018-10-02-15:59:36] Epoch: [092][195/196] Time 0.56 (0.55) Data 0.00 (0.01) Loss 1.283 (1.285) Prec@1 67.50 (54.52) Prec@5 92.50 (94.36) VALID-SEARCH [2018-10-02-15:59:37] Epoch: [092][000/196] Loss 1.566 (1.566) Prec@1 46.09 (46.09) Prec@5 92.19 (92.19) VALID-SEARCH [2018-10-02-15:59:42] Epoch: [092][100/196] Loss 1.883 (1.761) Prec@1 32.81 (39.29) Prec@5 80.47 (87.71) VALID-SEARCH [2018-10-02-15:59:47] Epoch: [092][195/196] Loss 1.363 (1.750) Prec@1 52.50 (39.64) Prec@5 97.50 (87.64) 092/250, Train-Accuracy = 54.52, Test-Accuracy = 39.64 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-15:59:48] [Epoch=093/250] [Need: 05:16:58] [LR=0.0351 ~ 0.0351] [Batch=128], tau=7.768 genotype = Genotype(normal=[('sep_conv_5x5', 1, 0.12737098336219788), ('dil_conv_3x3', 0, 0.1273464560508728), ('skip_connect', 0, 0.129585400223732), ('sep_conv_5x5', 2, 0.12792696058750153), ('skip_connect', 0, 0.12844663858413696), ('sep_conv_3x3', 3, 0.12708811461925507), ('max_pool_3x3', 1, 0.12733983993530273), ('dil_conv_3x3', 4, 0.12703697383403778)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.12620916962623596), ('max_pool_3x3', 1, 0.12557236850261688), ('avg_pool_3x3', 0, 0.12672601640224457), ('avg_pool_3x3', 1, 0.12625829875469208), ('max_pool_3x3', 1, 0.12662139534950256), ('avg_pool_3x3', 0, 0.12600916624069214), ('avg_pool_3x3', 1, 0.12618155777454376), ('skip_connect', 0, 0.12595658004283905)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-15:59:50] Epoch: [093][000/196] Time 1.14 (1.14) Data 0.65 (0.65) Loss 0.985 (0.985) Prec@1 59.38 (59.38) Prec@5 94.53 (94.53) TRAIN-SEARCH [2018-10-02-16:00:42] Epoch: [093][100/196] Time 0.38 (0.53) Data 0.00 (0.01) Loss 1.325 (1.277) Prec@1 50.78 (54.78) Prec@5 94.53 (94.28) TRAIN-SEARCH [2018-10-02-16:01:35] Epoch: [093][195/196] Time 0.44 (0.54) Data 0.00 (0.01) Loss 1.161 (1.265) Prec@1 60.00 (55.19) Prec@5 95.00 (94.53) VALID-SEARCH [2018-10-02-16:01:36] Epoch: [093][000/196] Loss 2.101 (2.101) Prec@1 39.06 (39.06) Prec@5 89.84 (89.84) VALID-SEARCH [2018-10-02-16:01:40] Epoch: [093][100/196] Loss 1.483 (1.733) Prec@1 47.66 (42.58) Prec@5 94.53 (89.88) VALID-SEARCH [2018-10-02-16:01:45] Epoch: [093][195/196] Loss 1.198 (1.735) Prec@1 60.00 (42.43) Prec@5 87.50 (89.57) 093/250, Train-Accuracy = 55.19, Test-Accuracy = 42.43 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-16:01:46] [Epoch=094/250] [Need: 05:06:11] [LR=0.0348 ~ 0.0348] [Batch=128], tau=7.744 genotype = Genotype(normal=[('skip_connect', 0, 0.127452552318573), ('sep_conv_5x5', 1, 0.12713666260242462), ('skip_connect', 0, 0.13048608601093292), ('skip_connect', 1, 0.12727798521518707), ('skip_connect', 0, 0.12797588109970093), ('sep_conv_5x5', 3, 0.1272154301404953), ('max_pool_3x3', 0, 0.12777243554592133), ('max_pool_3x3', 1, 0.12717388570308685)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.1259036809206009), ('max_pool_3x3', 1, 0.12579116225242615), ('max_pool_3x3', 1, 0.12630976736545563), ('max_pool_3x3', 0, 0.12624572217464447), ('max_pool_3x3', 1, 0.1263013631105423), ('max_pool_3x3', 0, 0.12626376748085022), ('dil_conv_3x3', 3, 0.12608473002910614), ('avg_pool_3x3', 1, 0.12585225701332092)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-16:01:48] Epoch: [094][000/196] Time 1.69 (1.69) Data 1.04 (1.04) Loss 1.197 (1.197) Prec@1 56.25 (56.25) Prec@5 96.09 (96.09) TRAIN-SEARCH [2018-10-02-16:02:42] Epoch: [094][100/196] Time 0.59 (0.55) Data 0.00 (0.01) Loss 1.533 (1.261) Prec@1 50.78 (55.50) Prec@5 91.41 (94.36) TRAIN-SEARCH [2018-10-02-16:03:35] Epoch: [094][195/196] Time 0.47 (0.55) Data 0.00 (0.01) Loss 1.392 (1.266) Prec@1 42.50 (54.79) Prec@5 90.00 (94.37) VALID-SEARCH [2018-10-02-16:03:36] Epoch: [094][000/196] Loss 1.892 (1.892) Prec@1 25.00 (25.00) Prec@5 82.03 (82.03) VALID-SEARCH [2018-10-02-16:03:41] Epoch: [094][100/196] Loss 1.620 (1.793) Prec@1 41.41 (39.50) Prec@5 88.28 (86.84) VALID-SEARCH [2018-10-02-16:03:46] Epoch: [094][195/196] Loss 1.611 (1.776) Prec@1 47.50 (39.79) Prec@5 77.50 (86.90) 094/250, Train-Accuracy = 54.79, Test-Accuracy = 39.79 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-16:03:47] [Epoch=095/250] [Need: 05:14:38] [LR=0.0345 ~ 0.0345] [Batch=128], tau=7.72 genotype = Genotype(normal=[('dil_conv_5x5', 0, 0.12724211812019348), ('dil_conv_3x3', 1, 0.12718461453914642), ('skip_connect', 0, 0.13049718737602234), ('sep_conv_3x3', 2, 0.12758080661296844), ('max_pool_3x3', 0, 0.12857875227928162), ('skip_connect', 1, 0.1275092512369156), ('sep_conv_3x3', 4, 0.12745057046413422), ('max_pool_3x3', 0, 0.12737300992012024)], normal_concat=range(2, 6), reduce=[('skip_connect', 0, 0.12585251033306122), ('max_pool_3x3', 1, 0.12558545172214508), ('max_pool_3x3', 0, 0.12648072838783264), ('avg_pool_3x3', 1, 0.12636861205101013), ('max_pool_3x3', 0, 0.12704965472221375), ('max_pool_3x3', 1, 0.12605495750904083), ('dil_conv_5x5', 0, 0.12577436864376068), ('avg_pool_3x3', 4, 0.12568821012973785)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-16:03:49] Epoch: [095][000/196] Time 1.27 (1.27) Data 0.75 (0.75) Loss 1.322 (1.322) Prec@1 51.56 (51.56) Prec@5 95.31 (95.31) TRAIN-SEARCH [2018-10-02-16:04:43] Epoch: [095][100/196] Time 0.60 (0.55) Data 0.02 (0.01) Loss 1.050 (1.257) Prec@1 67.19 (55.12) Prec@5 97.66 (94.41) TRAIN-SEARCH [2018-10-02-16:05:35] Epoch: [095][195/196] Time 0.60 (0.55) Data 0.00 (0.01) Loss 1.209 (1.260) Prec@1 50.00 (55.36) Prec@5 92.50 (94.50) VALID-SEARCH [2018-10-02-16:05:36] Epoch: [095][000/196] Loss 1.456 (1.456) Prec@1 46.88 (46.88) Prec@5 94.53 (94.53) VALID-SEARCH [2018-10-02-16:05:41] Epoch: [095][100/196] Loss 1.578 (1.656) Prec@1 44.53 (42.23) Prec@5 89.06 (88.58) VALID-SEARCH [2018-10-02-16:05:46] Epoch: [095][195/196] Loss 1.498 (1.671) Prec@1 45.00 (41.48) Prec@5 92.50 (88.25) 095/250, Train-Accuracy = 55.36, Test-Accuracy = 41.48 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-16:05:47] [Epoch=096/250] [Need: 05:07:55] [LR=0.0342 ~ 0.0342] [Batch=128], tau=7.696 genotype = Genotype(normal=[('dil_conv_5x5', 0, 0.12807005643844604), ('dil_conv_3x3', 1, 0.12755662202835083), ('skip_connect', 0, 0.13029271364212036), ('sep_conv_3x3', 2, 0.127410888671875), ('max_pool_3x3', 0, 0.12827521562576294), ('dil_conv_3x3', 3, 0.12767519056797028), ('dil_conv_3x3', 3, 0.12754175066947937), ('max_pool_3x3', 1, 0.12751199305057526)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.1261209100484848), ('max_pool_3x3', 1, 0.12580953538417816), ('avg_pool_3x3', 1, 0.1263030469417572), ('max_pool_3x3', 0, 0.12613996863365173), ('max_pool_3x3', 0, 0.12655290961265564), ('dil_conv_5x5', 3, 0.12627921998500824), ('avg_pool_3x3', 0, 0.1258271187543869), ('avg_pool_3x3', 4, 0.12579260766506195)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-16:05:50] Epoch: [096][000/196] Time 1.61 (1.61) Data 0.96 (0.96) Loss 1.113 (1.113) Prec@1 63.28 (63.28) Prec@5 93.75 (93.75) TRAIN-SEARCH [2018-10-02-16:06:45] Epoch: [096][100/196] Time 0.61 (0.56) Data 0.00 (0.01) Loss 1.223 (1.270) Prec@1 53.91 (54.75) Prec@5 96.09 (94.51) TRAIN-SEARCH [2018-10-02-16:07:35] Epoch: [096][195/196] Time 0.39 (0.55) Data 0.00 (0.01) Loss 1.163 (1.259) Prec@1 52.50 (55.14) Prec@5 92.50 (94.67) VALID-SEARCH [2018-10-02-16:07:36] Epoch: [096][000/196] Loss 1.273 (1.273) Prec@1 55.47 (55.47) Prec@5 93.75 (93.75) VALID-SEARCH [2018-10-02-16:07:42] Epoch: [096][100/196] Loss 1.515 (1.651) Prec@1 46.88 (44.11) Prec@5 88.28 (89.40) VALID-SEARCH [2018-10-02-16:07:47] Epoch: [096][195/196] Loss 1.576 (1.705) Prec@1 42.50 (43.78) Prec@5 90.00 (89.19) 096/250, Train-Accuracy = 55.14, Test-Accuracy = 43.78 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-16:07:48] [Epoch=097/250] [Need: 05:07:29] [LR=0.0339 ~ 0.0339] [Batch=128], tau=7.672 genotype = Genotype(normal=[('skip_connect', 0, 0.12852540612220764), ('dil_conv_3x3', 1, 0.12767751514911652), ('skip_connect', 0, 0.13036227226257324), ('sep_conv_5x5', 2, 0.12707304954528809), ('max_pool_3x3', 0, 0.1280508190393448), ('sep_conv_5x5', 3, 0.12738469243049622), ('max_pool_3x3', 0, 0.1280679553747177), ('dil_conv_3x3', 3, 0.1271388977766037)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.12690198421478271), ('max_pool_3x3', 1, 0.12549585103988647), ('max_pool_3x3', 1, 0.12644048035144806), ('avg_pool_3x3', 0, 0.12611687183380127), ('max_pool_3x3', 1, 0.1261754333972931), ('max_pool_3x3', 0, 0.12597604095935822), ('max_pool_3x3', 0, 0.12600398063659668), ('max_pool_3x3', 1, 0.1257597655057907)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-16:07:50] Epoch: [097][000/196] Time 1.52 (1.52) Data 0.81 (0.81) Loss 1.459 (1.459) Prec@1 50.78 (50.78) Prec@5 89.06 (89.06) TRAIN-SEARCH [2018-10-02-16:08:41] Epoch: [097][100/196] Time 0.42 (0.51) Data 0.00 (0.01) Loss 1.303 (1.261) Prec@1 55.47 (55.79) Prec@5 92.97 (94.31) TRAIN-SEARCH [2018-10-02-16:09:33] Epoch: [097][195/196] Time 0.40 (0.53) Data 0.00 (0.01) Loss 1.286 (1.263) Prec@1 45.00 (55.24) Prec@5 95.00 (94.24) VALID-SEARCH [2018-10-02-16:09:34] Epoch: [097][000/196] Loss 1.766 (1.766) Prec@1 38.28 (38.28) Prec@5 85.94 (85.94) VALID-SEARCH [2018-10-02-16:09:39] Epoch: [097][100/196] Loss 2.121 (1.705) Prec@1 39.06 (43.97) Prec@5 92.19 (90.13) VALID-SEARCH [2018-10-02-16:09:44] Epoch: [097][195/196] Loss 2.062 (1.710) Prec@1 35.00 (43.57) Prec@5 82.50 (89.88) 097/250, Train-Accuracy = 55.24, Test-Accuracy = 43.57 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-16:09:45] [Epoch=098/250] [Need: 04:57:23] [LR=0.0337 ~ 0.0337] [Batch=128], tau=7.648 genotype = Genotype(normal=[('dil_conv_3x3', 1, 0.1280728429555893), ('skip_connect', 0, 0.12760962545871735), ('skip_connect', 0, 0.13014452159404755), ('sep_conv_3x3', 2, 0.12755735218524933), ('max_pool_3x3', 0, 0.12802888453006744), ('sep_conv_5x5', 3, 0.1271018534898758), ('max_pool_3x3', 0, 0.1286168247461319), ('skip_connect', 1, 0.1272617131471634)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.1266053020954132), ('avg_pool_3x3', 1, 0.1256345808506012), ('max_pool_3x3', 0, 0.12680427730083466), ('avg_pool_3x3', 1, 0.1260979175567627), ('max_pool_3x3', 1, 0.12612922489643097), ('max_pool_3x3', 0, 0.12595979869365692), ('max_pool_3x3', 0, 0.1261681318283081), ('dil_conv_5x5', 4, 0.1257357895374298)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-16:09:48] Epoch: [098][000/196] Time 1.61 (1.61) Data 0.98 (0.98) Loss 1.339 (1.339) Prec@1 57.03 (57.03) Prec@5 90.62 (90.62) TRAIN-SEARCH [2018-10-02-16:10:39] Epoch: [098][100/196] Time 0.63 (0.53) Data 0.00 (0.01) Loss 1.364 (1.239) Prec@1 51.56 (56.23) Prec@5 92.97 (94.72) TRAIN-SEARCH [2018-10-02-16:11:35] Epoch: [098][195/196] Time 0.43 (0.55) Data 0.00 (0.01) Loss 1.129 (1.239) Prec@1 55.00 (56.02) Prec@5 97.50 (94.82) VALID-SEARCH [2018-10-02-16:11:36] Epoch: [098][000/196] Loss 1.826 (1.826) Prec@1 27.34 (27.34) Prec@5 87.50 (87.50) VALID-SEARCH [2018-10-02-16:11:41] Epoch: [098][100/196] Loss 1.780 (1.754) Prec@1 35.94 (39.79) Prec@5 87.50 (87.37) VALID-SEARCH [2018-10-02-16:11:46] Epoch: [098][195/196] Loss 1.454 (1.704) Prec@1 47.50 (41.44) Prec@5 100.00 (88.04) 098/250, Train-Accuracy = 56.02, Test-Accuracy = 41.44 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-16:11:47] [Epoch=099/250] [Need: 05:06:59] [LR=0.0334 ~ 0.0334] [Batch=128], tau=7.624 genotype = Genotype(normal=[('skip_connect', 0, 0.12741893529891968), ('dil_conv_3x3', 1, 0.12729604542255402), ('skip_connect', 0, 0.13009363412857056), ('sep_conv_5x5', 2, 0.12724672257900238), ('max_pool_3x3', 0, 0.12736645340919495), ('sep_conv_3x3', 3, 0.12723588943481445), ('max_pool_3x3', 1, 0.1283407211303711), ('max_pool_3x3', 0, 0.1279781609773636)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.12607471644878387), ('dil_conv_3x3', 1, 0.12581981718540192), ('max_pool_3x3', 0, 0.12686540186405182), ('avg_pool_3x3', 1, 0.1262647807598114), ('max_pool_3x3', 0, 0.12617723643779755), ('sep_conv_3x3', 1, 0.12616699934005737), ('max_pool_3x3', 0, 0.12620317935943604), ('max_pool_3x3', 1, 0.1260872632265091)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-16:11:50] Epoch: [099][000/196] Time 1.59 (1.59) Data 1.00 (1.00) Loss 1.268 (1.268) Prec@1 54.69 (54.69) Prec@5 96.09 (96.09) TRAIN-SEARCH [2018-10-02-16:12:41] Epoch: [099][100/196] Time 0.46 (0.52) Data 0.00 (0.01) Loss 1.181 (1.265) Prec@1 61.72 (55.73) Prec@5 94.53 (94.59) TRAIN-SEARCH [2018-10-02-16:13:34] Epoch: [099][195/196] Time 0.44 (0.54) Data 0.00 (0.01) Loss 1.192 (1.257) Prec@1 50.00 (55.55) Prec@5 97.50 (94.70) VALID-SEARCH [2018-10-02-16:13:35] Epoch: [099][000/196] Loss 3.386 (3.386) Prec@1 36.72 (36.72) Prec@5 82.03 (82.03) VALID-SEARCH [2018-10-02-16:13:41] Epoch: [099][100/196] Loss 1.862 (1.717) Prec@1 41.41 (44.25) Prec@5 90.62 (90.15) VALID-SEARCH [2018-10-02-16:13:46] Epoch: [099][195/196] Loss 1.476 (1.679) Prec@1 57.50 (44.86) Prec@5 92.50 (90.31) 099/250, Train-Accuracy = 55.55, Test-Accuracy = 44.86 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-16:13:48] [Epoch=100/250] [Need: 05:00:53] [LR=0.0331 ~ 0.0331] [Batch=128], tau=7.6 genotype = Genotype(normal=[('skip_connect', 0, 0.12803947925567627), ('dil_conv_3x3', 1, 0.12777169048786163), ('skip_connect', 0, 0.13057415187358856), ('sep_conv_5x5', 2, 0.12734027206897736), ('sep_conv_3x3', 3, 0.1277061104774475), ('max_pool_3x3', 0, 0.12723606824874878), ('max_pool_3x3', 1, 0.1276513636112213), ('max_pool_3x3', 0, 0.12758101522922516)], normal_concat=range(2, 6), reduce=[('dil_conv_5x5', 1, 0.12609651684761047), ('max_pool_3x3', 0, 0.12608781456947327), ('avg_pool_3x3', 0, 0.12648814916610718), ('avg_pool_3x3', 1, 0.12602877616882324), ('max_pool_3x3', 0, 0.1263645440340042), ('skip_connect', 1, 0.12585870921611786), ('max_pool_3x3', 0, 0.12643420696258545), ('dil_conv_3x3', 1, 0.12572312355041504)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-16:13:49] Epoch: [100][000/196] Time 1.18 (1.18) Data 0.66 (0.66) Loss 1.360 (1.360) Prec@1 53.91 (53.91) Prec@5 94.53 (94.53) TRAIN-SEARCH [2018-10-02-16:14:41] Epoch: [100][100/196] Time 0.57 (0.52) Data 0.01 (0.01) Loss 1.133 (1.234) Prec@1 60.94 (55.41) Prec@5 96.88 (94.77) TRAIN-SEARCH [2018-10-02-16:15:34] Epoch: [100][195/196] Time 0.44 (0.54) Data 0.00 (0.01) Loss 1.475 (1.235) Prec@1 42.50 (55.76) Prec@5 97.50 (94.73) VALID-SEARCH [2018-10-02-16:15:35] Epoch: [100][000/196] Loss 1.706 (1.706) Prec@1 45.31 (45.31) Prec@5 90.62 (90.62) VALID-SEARCH [2018-10-02-16:15:40] Epoch: [100][100/196] Loss 1.538 (1.745) Prec@1 45.31 (42.62) Prec@5 89.06 (87.55) VALID-SEARCH [2018-10-02-16:15:44] Epoch: [100][195/196] Loss 1.477 (1.737) Prec@1 47.50 (42.10) Prec@5 95.00 (87.61) 100/250, Train-Accuracy = 55.76, Test-Accuracy = 42.10 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-16:15:45] [Epoch=101/250] [Need: 04:52:03] [LR=0.0328 ~ 0.0328] [Batch=128], tau=7.576 genotype = Genotype(normal=[('dil_conv_5x5', 0, 0.12797074019908905), ('dil_conv_3x3', 1, 0.1276426762342453), ('skip_connect', 0, 0.13045582175254822), ('skip_connect', 1, 0.12819866836071014), ('max_pool_3x3', 0, 0.12734775245189667), ('sep_conv_3x3', 3, 0.12733061611652374), ('max_pool_3x3', 0, 0.12794162333011627), ('dil_conv_5x5', 4, 0.12738309800624847)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.12629535794258118), ('dil_conv_5x5', 1, 0.12575224041938782), ('avg_pool_3x3', 0, 0.12725181877613068), ('avg_pool_3x3', 1, 0.1261308193206787), ('max_pool_3x3', 0, 0.12668795883655548), ('skip_connect', 1, 0.12578842043876648), ('avg_pool_3x3', 0, 0.12660710513591766), ('dil_conv_5x5', 3, 0.1263510137796402)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-16:15:48] Epoch: [101][000/196] Time 1.59 (1.59) Data 0.94 (0.94) Loss 1.157 (1.157) Prec@1 57.81 (57.81) Prec@5 94.53 (94.53) TRAIN-SEARCH [2018-10-02-16:16:41] Epoch: [101][100/196] Time 0.69 (0.55) Data 0.00 (0.01) Loss 1.255 (1.221) Prec@1 53.91 (56.23) Prec@5 95.31 (95.06) TRAIN-SEARCH [2018-10-02-16:17:32] Epoch: [101][195/196] Time 0.41 (0.54) Data 0.00 (0.01) Loss 1.081 (1.232) Prec@1 57.50 (56.09) Prec@5 100.00 (94.84) VALID-SEARCH [2018-10-02-16:17:33] Epoch: [101][000/196] Loss 1.795 (1.795) Prec@1 30.47 (30.47) Prec@5 85.94 (85.94) VALID-SEARCH [2018-10-02-16:17:39] Epoch: [101][100/196] Loss 1.378 (1.649) Prec@1 50.78 (44.47) Prec@5 95.31 (89.94) VALID-SEARCH [2018-10-02-16:17:44] Epoch: [101][195/196] Loss 1.835 (1.648) Prec@1 42.50 (44.77) Prec@5 85.00 (89.86) 101/250, Train-Accuracy = 56.09, Test-Accuracy = 44.77 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-16:17:45] [Epoch=102/250] [Need: 04:54:09] [LR=0.0325 ~ 0.0325] [Batch=128], tau=7.552 genotype = Genotype(normal=[('sep_conv_5x5', 0, 0.12758345901966095), ('dil_conv_3x3', 1, 0.12730376422405243), ('skip_connect', 0, 0.12963466346263885), ('skip_connect', 1, 0.12793004512786865), ('sep_conv_3x3', 3, 0.1270785629749298), ('dil_conv_5x5', 0, 0.12705522775650024), ('sep_conv_3x3', 4, 0.12744168937206268), ('max_pool_3x3', 0, 0.12713827192783356)], normal_concat=range(2, 6), reduce=[('dil_conv_5x5', 1, 0.12576763331890106), ('avg_pool_3x3', 0, 0.12575975060462952), ('avg_pool_3x3', 0, 0.12684866786003113), ('dil_conv_3x3', 2, 0.1257975697517395), ('dil_conv_5x5', 0, 0.12643766403198242), ('skip_connect', 1, 0.12588715553283691), ('max_pool_3x3', 0, 0.12645098567008972), ('sep_conv_3x3', 3, 0.12591113150119781)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-16:17:47] Epoch: [102][000/196] Time 1.58 (1.58) Data 0.97 (0.97) Loss 1.160 (1.160) Prec@1 60.16 (60.16) Prec@5 93.75 (93.75) TRAIN-SEARCH [2018-10-02-16:18:41] Epoch: [102][100/196] Time 0.54 (0.55) Data 0.00 (0.01) Loss 1.135 (1.222) Prec@1 58.59 (56.49) Prec@5 96.09 (95.00) TRAIN-SEARCH [2018-10-02-16:19:33] Epoch: [102][195/196] Time 0.56 (0.55) Data 0.00 (0.01) Loss 1.502 (1.238) Prec@1 47.50 (56.02) Prec@5 97.50 (94.89) VALID-SEARCH [2018-10-02-16:19:34] Epoch: [102][000/196] Loss 1.282 (1.282) Prec@1 49.22 (49.22) Prec@5 95.31 (95.31) VALID-SEARCH [2018-10-02-16:19:40] Epoch: [102][100/196] Loss 1.643 (1.767) Prec@1 41.41 (41.24) Prec@5 87.50 (88.46) VALID-SEARCH [2018-10-02-16:19:45] Epoch: [102][195/196] Loss 1.784 (1.751) Prec@1 27.50 (41.62) Prec@5 90.00 (88.66) 102/250, Train-Accuracy = 56.02, Test-Accuracy = 41.62 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-16:19:45] [Epoch=103/250] [Need: 04:56:13] [LR=0.0322 ~ 0.0322] [Batch=128], tau=7.5280000000000005 genotype = Genotype(normal=[('dil_conv_5x5', 0, 0.1278875172138214), ('skip_connect', 1, 0.12737151980400085), ('skip_connect', 0, 0.1291349083185196), ('dil_conv_3x3', 2, 0.12755946815013885), ('skip_connect', 1, 0.12743736803531647), ('sep_conv_3x3', 3, 0.12716403603553772), ('max_pool_3x3', 0, 0.12710455060005188), ('sep_conv_3x3', 4, 0.12705931067466736)], normal_concat=range(2, 6), reduce=[('skip_connect', 0, 0.12584298849105835), ('dil_conv_5x5', 1, 0.12533876299858093), ('avg_pool_3x3', 0, 0.1267872005701065), ('avg_pool_3x3', 1, 0.12608428299427032), ('max_pool_3x3', 0, 0.12677708268165588), ('skip_connect', 1, 0.12555374205112457), ('max_pool_3x3', 0, 0.12644626200199127), ('dil_conv_5x5', 3, 0.12595635652542114)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-16:19:48] Epoch: [103][000/196] Time 1.41 (1.41) Data 0.82 (0.82) Loss 1.147 (1.147) Prec@1 54.69 (54.69) Prec@5 98.44 (98.44) TRAIN-SEARCH [2018-10-02-16:20:41] Epoch: [103][100/196] Time 0.47 (0.54) Data 0.00 (0.01) Loss 1.206 (1.230) Prec@1 59.38 (56.66) Prec@5 95.31 (94.75) TRAIN-SEARCH [2018-10-02-16:21:33] Epoch: [103][195/196] Time 0.39 (0.54) Data 0.00 (0.01) Loss 1.165 (1.224) Prec@1 47.50 (56.75) Prec@5 100.00 (94.86) VALID-SEARCH [2018-10-02-16:21:34] Epoch: [103][000/196] Loss 1.382 (1.382) Prec@1 52.34 (52.34) Prec@5 93.75 (93.75) VALID-SEARCH [2018-10-02-16:21:39] Epoch: [103][100/196] Loss 1.578 (1.743) Prec@1 40.62 (43.36) Prec@5 87.50 (89.46) VALID-SEARCH [2018-10-02-16:21:44] Epoch: [103][195/196] Loss 1.447 (1.734) Prec@1 37.50 (43.90) Prec@5 92.50 (89.46) 103/250, Train-Accuracy = 56.75, Test-Accuracy = 43.90 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-16:21:45] [Epoch=104/250] [Need: 04:49:41] [LR=0.0319 ~ 0.0319] [Batch=128], tau=7.504 genotype = Genotype(normal=[('skip_connect', 0, 0.12790381908416748), ('skip_connect', 1, 0.12731921672821045), ('skip_connect', 0, 0.12878593802452087), ('sep_conv_5x5', 2, 0.12716571986675262), ('sep_conv_3x3', 3, 0.12719427049160004), ('max_pool_3x3', 0, 0.12682832777500153), ('max_pool_3x3', 0, 0.12714552879333496), ('dil_conv_5x5', 3, 0.12713702023029327)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.12639562785625458), ('dil_conv_3x3', 1, 0.12538829445838928), ('avg_pool_3x3', 0, 0.126666858792305), ('max_pool_3x3', 1, 0.12663650512695312), ('max_pool_3x3', 0, 0.12624964118003845), ('avg_pool_3x3', 1, 0.12569011747837067), ('max_pool_3x3', 0, 0.12613533437252045), ('dil_conv_5x5', 3, 0.12606626749038696)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-16:21:47] Epoch: [104][000/196] Time 1.48 (1.48) Data 0.90 (0.90) Loss 1.125 (1.125) Prec@1 57.81 (57.81) Prec@5 96.09 (96.09) TRAIN-SEARCH [2018-10-02-16:22:39] Epoch: [104][100/196] Time 0.42 (0.53) Data 0.00 (0.01) Loss 1.415 (1.228) Prec@1 46.88 (56.88) Prec@5 94.53 (94.62) TRAIN-SEARCH [2018-10-02-16:23:32] Epoch: [104][195/196] Time 0.48 (0.55) Data 0.00 (0.01) Loss 1.179 (1.226) Prec@1 65.00 (56.91) Prec@5 92.50 (94.86) VALID-SEARCH [2018-10-02-16:23:33] Epoch: [104][000/196] Loss 1.597 (1.597) Prec@1 44.53 (44.53) Prec@5 92.97 (92.97) VALID-SEARCH [2018-10-02-16:23:39] Epoch: [104][100/196] Loss 1.680 (1.667) Prec@1 46.88 (44.22) Prec@5 89.06 (88.54) VALID-SEARCH [2018-10-02-16:23:44] Epoch: [104][195/196] Loss 1.218 (1.673) Prec@1 55.00 (44.09) Prec@5 90.00 (89.06) 104/250, Train-Accuracy = 56.91, Test-Accuracy = 44.09 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-16:23:45] [Epoch=105/250] [Need: 04:50:24] [LR=0.0316 ~ 0.0316] [Batch=128], tau=7.48 genotype = Genotype(normal=[('skip_connect', 0, 0.1274646371603012), ('skip_connect', 1, 0.12662747502326965), ('skip_connect', 0, 0.12818767130374908), ('sep_conv_5x5', 2, 0.12716089189052582), ('skip_connect', 1, 0.12740325927734375), ('max_pool_3x3', 0, 0.1272590011358261), ('sep_conv_3x3', 4, 0.12764586508274078), ('skip_connect', 0, 0.12713655829429626)], normal_concat=range(2, 6), reduce=[('skip_connect', 0, 0.12597399950027466), ('dil_conv_3x3', 1, 0.1255863457918167), ('avg_pool_3x3', 0, 0.12638944387435913), ('max_pool_3x3', 1, 0.12624314427375793), ('skip_connect', 0, 0.12603694200515747), ('max_pool_3x3', 1, 0.12565945088863373), ('max_pool_3x3', 0, 0.1263130158185959), ('sep_conv_5x5', 1, 0.12585248053073883)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-16:23:47] Epoch: [105][000/196] Time 1.61 (1.61) Data 1.02 (1.02) Loss 1.111 (1.111) Prec@1 54.69 (54.69) Prec@5 98.44 (98.44) TRAIN-SEARCH [2018-10-02-16:24:38] Epoch: [105][100/196] Time 0.57 (0.52) Data 0.00 (0.01) Loss 1.219 (1.216) Prec@1 60.16 (56.28) Prec@5 98.44 (95.05) TRAIN-SEARCH [2018-10-02-16:25:34] Epoch: [105][195/196] Time 0.52 (0.55) Data 0.00 (0.01) Loss 1.181 (1.219) Prec@1 55.00 (56.49) Prec@5 92.50 (94.93) VALID-SEARCH [2018-10-02-16:25:35] Epoch: [105][000/196] Loss 1.350 (1.350) Prec@1 50.78 (50.78) Prec@5 93.75 (93.75) VALID-SEARCH [2018-10-02-16:25:41] Epoch: [105][100/196] Loss 1.617 (1.655) Prec@1 43.75 (44.10) Prec@5 88.28 (89.05) VALID-SEARCH [2018-10-02-16:25:46] Epoch: [105][195/196] Loss 1.159 (1.710) Prec@1 57.50 (44.01) Prec@5 92.50 (88.80) 105/250, Train-Accuracy = 56.49, Test-Accuracy = 44.01 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-16:25:47] [Epoch=106/250] [Need: 04:53:38] [LR=0.0313 ~ 0.0313] [Batch=128], tau=7.4559999999999995 genotype = Genotype(normal=[('skip_connect', 1, 0.1274213194847107), ('skip_connect', 0, 0.12737271189689636), ('skip_connect', 0, 0.12792928516864777), ('dil_conv_5x5', 2, 0.12711645662784576), ('max_pool_3x3', 0, 0.12700749933719635), ('skip_connect', 1, 0.12695905566215515), ('sep_conv_5x5', 4, 0.1276421993970871), ('skip_connect', 0, 0.12708443403244019)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.1260916143655777), ('dil_conv_3x3', 1, 0.125621497631073), ('max_pool_3x3', 0, 0.12620986998081207), ('avg_pool_3x3', 1, 0.1257578581571579), ('avg_pool_3x3', 0, 0.12647129595279694), ('max_pool_3x3', 1, 0.12588319182395935), ('skip_connect', 0, 0.12592239677906036), ('sep_conv_5x5', 1, 0.12586839497089386)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-16:25:49] Epoch: [106][000/196] Time 1.56 (1.56) Data 0.86 (0.86) Loss 1.306 (1.306) Prec@1 54.69 (54.69) Prec@5 93.75 (93.75) TRAIN-SEARCH [2018-10-02-16:26:40] Epoch: [106][100/196] Time 0.67 (0.51) Data 0.00 (0.01) Loss 1.163 (1.209) Prec@1 58.59 (57.59) Prec@5 94.53 (94.72) TRAIN-SEARCH [2018-10-02-16:27:34] Epoch: [106][195/196] Time 0.40 (0.54) Data 0.00 (0.01) Loss 1.147 (1.206) Prec@1 67.50 (57.32) Prec@5 97.50 (94.89) VALID-SEARCH [2018-10-02-16:27:35] Epoch: [106][000/196] Loss 1.432 (1.432) Prec@1 49.22 (49.22) Prec@5 87.50 (87.50) VALID-SEARCH [2018-10-02-16:27:40] Epoch: [106][100/196] Loss 2.408 (1.704) Prec@1 46.88 (45.71) Prec@5 85.16 (90.01) VALID-SEARCH [2018-10-02-16:27:46] Epoch: [106][195/196] Loss 1.262 (1.693) Prec@1 55.00 (45.85) Prec@5 95.00 (90.49) 106/250, Train-Accuracy = 57.32, Test-Accuracy = 45.85 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-16:27:47] [Epoch=107/250] [Need: 04:44:47] [LR=0.0310 ~ 0.0310] [Batch=128], tau=7.432 genotype = Genotype(normal=[('skip_connect', 1, 0.1272357553243637), ('dil_conv_3x3', 0, 0.12708427011966705), ('skip_connect', 0, 0.12750938534736633), ('skip_connect', 1, 0.1274234801530838), ('dil_conv_5x5', 3, 0.1271343231201172), ('max_pool_3x3', 1, 0.12696269154548645), ('sep_conv_5x5', 4, 0.1277094930410385), ('skip_connect', 0, 0.1272002011537552)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.12658901512622833), ('skip_connect', 1, 0.12549807131290436), ('max_pool_3x3', 1, 0.12646563351154327), ('max_pool_3x3', 0, 0.1264376938343048), ('avg_pool_3x3', 0, 0.12602271139621735), ('sep_conv_5x5', 2, 0.12595345079898834), ('sep_conv_5x5', 1, 0.1263509839773178), ('avg_pool_3x3', 0, 0.1262282431125641)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-16:27:49] Epoch: [107][000/196] Time 1.54 (1.54) Data 0.94 (0.94) Loss 1.235 (1.235) Prec@1 52.34 (52.34) Prec@5 95.31 (95.31) TRAIN-SEARCH [2018-10-02-16:28:40] Epoch: [107][100/196] Time 0.59 (0.52) Data 0.00 (0.01) Loss 1.704 (1.222) Prec@1 48.44 (57.58) Prec@5 92.19 (94.86) TRAIN-SEARCH [2018-10-02-16:29:33] Epoch: [107][195/196] Time 0.46 (0.54) Data 0.00 (0.01) Loss 1.448 (1.221) Prec@1 62.50 (57.40) Prec@5 87.50 (94.70) VALID-SEARCH [2018-10-02-16:29:34] Epoch: [107][000/196] Loss 1.671 (1.671) Prec@1 41.41 (41.41) Prec@5 96.09 (96.09) VALID-SEARCH [2018-10-02-16:29:39] Epoch: [107][100/196] Loss 1.895 (1.700) Prec@1 25.78 (43.04) Prec@5 81.25 (88.91) VALID-SEARCH [2018-10-02-16:29:43] Epoch: [107][195/196] Loss 1.125 (1.693) Prec@1 52.50 (43.00) Prec@5 97.50 (88.98) 107/250, Train-Accuracy = 57.40, Test-Accuracy = 43.00 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-16:29:45] [Epoch=108/250] [Need: 04:40:05] [LR=0.0307 ~ 0.0307] [Batch=128], tau=7.4079999999999995 genotype = Genotype(normal=[('skip_connect', 0, 0.1274094581604004), ('skip_connect', 1, 0.12722571194171906), ('skip_connect', 0, 0.12793825566768646), ('sep_conv_3x3', 2, 0.1275923103094101), ('max_pool_3x3', 1, 0.12765279412269592), ('sep_conv_3x3', 3, 0.1273326724767685), ('max_pool_3x3', 0, 0.12805601954460144), ('skip_connect', 1, 0.12760788202285767)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.12606090307235718), ('dil_conv_5x5', 1, 0.12583552300930023), ('max_pool_3x3', 0, 0.12648287415504456), ('max_pool_3x3', 1, 0.12639036774635315), ('dil_conv_5x5', 0, 0.12619523704051971), ('sep_conv_5x5', 2, 0.12597352266311646), ('sep_conv_5x5', 1, 0.12616661190986633), ('max_pool_3x3', 0, 0.12588618695735931)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-16:29:47] Epoch: [108][000/196] Time 1.54 (1.54) Data 0.81 (0.81) Loss 1.199 (1.199) Prec@1 57.81 (57.81) Prec@5 96.09 (96.09) TRAIN-SEARCH [2018-10-02-16:30:40] Epoch: [108][100/196] Time 0.56 (0.54) Data 0.00 (0.01) Loss 1.143 (1.221) Prec@1 64.84 (56.68) Prec@5 95.31 (94.88) TRAIN-SEARCH [2018-10-02-16:31:31] Epoch: [108][195/196] Time 0.32 (0.54) Data 0.00 (0.01) Loss 0.995 (1.218) Prec@1 55.00 (56.95) Prec@5 100.00 (94.93) VALID-SEARCH [2018-10-02-16:31:32] Epoch: [108][000/196] Loss 1.612 (1.612) Prec@1 35.16 (35.16) Prec@5 92.97 (92.97) VALID-SEARCH [2018-10-02-16:31:38] Epoch: [108][100/196] Loss 1.445 (1.776) Prec@1 54.69 (41.16) Prec@5 94.53 (88.34) VALID-SEARCH [2018-10-02-16:31:43] Epoch: [108][195/196] Loss 1.192 (1.746) Prec@1 62.50 (41.46) Prec@5 95.00 (88.29) 108/250, Train-Accuracy = 56.95, Test-Accuracy = 41.46 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-16:31:44] [Epoch=109/250] [Need: 04:39:29] [LR=0.0304 ~ 0.0304] [Batch=128], tau=7.384 genotype = Genotype(normal=[('dil_conv_5x5', 0, 0.12756767868995667), ('skip_connect', 1, 0.12750478088855743), ('skip_connect', 0, 0.12787120044231415), ('dil_conv_3x3', 2, 0.12751083076000214), ('max_pool_3x3', 1, 0.12746065855026245), ('dil_conv_5x5', 3, 0.12708017230033875), ('max_pool_3x3', 0, 0.1285213828086853), ('sep_conv_5x5', 4, 0.1276976764202118)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.126266747713089), ('dil_conv_5x5', 1, 0.12617124617099762), ('max_pool_3x3', 1, 0.1264706403017044), ('max_pool_3x3', 0, 0.12630897760391235), ('dil_conv_5x5', 0, 0.1263720691204071), ('sep_conv_5x5', 3, 0.1258922666311264), ('sep_conv_5x5', 1, 0.12639936804771423), ('max_pool_3x3', 0, 0.12625889480113983)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-16:31:46] Epoch: [109][000/196] Time 1.50 (1.50) Data 0.87 (0.87) Loss 1.359 (1.359) Prec@1 46.88 (46.88) Prec@5 92.97 (92.97) TRAIN-SEARCH [2018-10-02-16:32:41] Epoch: [109][100/196] Time 0.63 (0.56) Data 0.00 (0.01) Loss 1.214 (1.222) Prec@1 57.81 (56.47) Prec@5 94.53 (95.27) TRAIN-SEARCH [2018-10-02-16:33:34] Epoch: [109][195/196] Time 0.69 (0.56) Data 0.00 (0.01) Loss 1.218 (1.206) Prec@1 57.50 (57.49) Prec@5 90.00 (94.96) VALID-SEARCH [2018-10-02-16:33:35] Epoch: [109][000/196] Loss 2.532 (2.532) Prec@1 18.75 (18.75) Prec@5 79.69 (79.69) VALID-SEARCH [2018-10-02-16:33:41] Epoch: [109][100/196] Loss 2.347 (1.630) Prec@1 32.81 (45.68) Prec@5 86.72 (89.83) VALID-SEARCH [2018-10-02-16:33:46] Epoch: [109][195/196] Loss 1.614 (1.662) Prec@1 57.50 (45.14) Prec@5 95.00 (89.62) 109/250, Train-Accuracy = 57.49, Test-Accuracy = 45.14 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-16:33:47] [Epoch=110/250] [Need: 04:48:15] [LR=0.0301 ~ 0.0301] [Batch=128], tau=7.359999999999999 genotype = Genotype(normal=[('dil_conv_5x5', 0, 0.12711749970912933), ('skip_connect', 1, 0.12704849243164062), ('skip_connect', 0, 0.128559410572052), ('sep_conv_3x3', 2, 0.12755821645259857), ('max_pool_3x3', 1, 0.12737299501895905), ('dil_conv_5x5', 0, 0.12727566063404083), ('max_pool_3x3', 0, 0.12869548797607422), ('sep_conv_5x5', 4, 0.12792380154132843)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.12610462307929993), ('dil_conv_3x3', 1, 0.12595927715301514), ('max_pool_3x3', 0, 0.12676112353801727), ('max_pool_3x3', 1, 0.12660780549049377), ('dil_conv_5x5', 3, 0.12584105134010315), ('dil_conv_5x5', 0, 0.12574268877506256), ('max_pool_3x3', 1, 0.1262771189212799), ('dil_conv_5x5', 2, 0.12617774307727814)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-16:33:50] Epoch: [110][000/196] Time 1.46 (1.46) Data 0.80 (0.80) Loss 1.132 (1.132) Prec@1 59.38 (59.38) Prec@5 96.88 (96.88) TRAIN-SEARCH [2018-10-02-16:34:43] Epoch: [110][100/196] Time 0.53 (0.54) Data 0.00 (0.01) Loss 0.966 (1.197) Prec@1 67.97 (57.48) Prec@5 96.09 (95.44) TRAIN-SEARCH [2018-10-02-16:35:36] Epoch: [110][195/196] Time 0.50 (0.55) Data 0.00 (0.01) Loss 1.424 (1.201) Prec@1 50.00 (57.40) Prec@5 92.50 (95.33) VALID-SEARCH [2018-10-02-16:35:37] Epoch: [110][000/196] Loss 2.052 (2.052) Prec@1 30.47 (30.47) Prec@5 77.34 (77.34) VALID-SEARCH [2018-10-02-16:35:42] Epoch: [110][100/196] Loss 2.167 (1.648) Prec@1 50.00 (44.69) Prec@5 86.72 (89.71) VALID-SEARCH [2018-10-02-16:35:47] Epoch: [110][195/196] Loss 1.457 (1.634) Prec@1 47.50 (44.61) Prec@5 92.50 (89.98) 110/250, Train-Accuracy = 57.40, Test-Accuracy = 44.61 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-16:35:48] [Epoch=111/250] [Need: 04:40:37] [LR=0.0298 ~ 0.0298] [Batch=128], tau=7.336 genotype = Genotype(normal=[('skip_connect', 1, 0.12770715355873108), ('skip_connect', 0, 0.12722711265087128), ('skip_connect', 0, 0.12859681248664856), ('sep_conv_3x3', 2, 0.12784698605537415), ('max_pool_3x3', 0, 0.127345010638237), ('dil_conv_3x3', 3, 0.1270211935043335), ('max_pool_3x3', 0, 0.1283615231513977), ('skip_connect', 1, 0.12814335525035858)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1, 0.12601681053638458), ('max_pool_3x3', 0, 0.12593819200992584), ('max_pool_3x3', 0, 0.12754987180233002), ('dil_conv_5x5', 1, 0.12601549923419952), ('max_pool_3x3', 0, 0.12648804485797882), ('sep_conv_5x5', 2, 0.12635953724384308), ('max_pool_3x3', 0, 0.12575934827327728), ('dil_conv_5x5', 1, 0.1257067620754242)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-16:35:51] Epoch: [111][000/196] Time 1.53 (1.53) Data 0.96 (0.96) Loss 1.289 (1.289) Prec@1 55.47 (55.47) Prec@5 92.19 (92.19) TRAIN-SEARCH [2018-10-02-16:36:42] Epoch: [111][100/196] Time 0.53 (0.52) Data 0.00 (0.01) Loss 1.144 (1.191) Prec@1 53.91 (57.67) Prec@5 95.31 (95.37) TRAIN-SEARCH [2018-10-02-16:37:35] Epoch: [111][195/196] Time 0.38 (0.54) Data 0.00 (0.01) Loss 1.152 (1.190) Prec@1 55.00 (57.66) Prec@5 92.50 (95.36) VALID-SEARCH [2018-10-02-16:37:36] Epoch: [111][000/196] Loss 1.406 (1.406) Prec@1 50.78 (50.78) Prec@5 93.75 (93.75) VALID-SEARCH [2018-10-02-16:37:42] Epoch: [111][100/196] Loss 1.270 (1.686) Prec@1 50.00 (45.25) Prec@5 98.44 (89.90) VALID-SEARCH [2018-10-02-16:37:47] Epoch: [111][195/196] Loss 1.305 (1.656) Prec@1 57.50 (46.06) Prec@5 87.50 (90.69) 111/250, Train-Accuracy = 57.66, Test-Accuracy = 46.06 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-16:37:48] [Epoch=112/250] [Need: 04:34:38] [LR=0.0295 ~ 0.0295] [Batch=128], tau=7.311999999999999 genotype = Genotype(normal=[('skip_connect', 0, 0.12753775715827942), ('skip_connect', 1, 0.12747111916542053), ('skip_connect', 0, 0.1282702386379242), ('sep_conv_3x3', 2, 0.12764661014080048), ('skip_connect', 0, 0.12874673306941986), ('sep_conv_5x5', 3, 0.12648120522499084), ('max_pool_3x3', 0, 0.12791743874549866), ('max_pool_3x3', 1, 0.12724122405052185)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.12604743242263794), ('dil_conv_5x5', 1, 0.12542995810508728), ('max_pool_3x3', 0, 0.12750142812728882), ('avg_pool_3x3', 1, 0.12578405439853668), ('max_pool_3x3', 0, 0.1267552673816681), ('max_pool_3x3', 1, 0.125819593667984), ('sep_conv_5x5', 0, 0.12611380219459534), ('max_pool_3x3', 1, 0.12610940635204315)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-16:37:50] Epoch: [112][000/196] Time 1.44 (1.44) Data 0.78 (0.78) Loss 1.293 (1.293) Prec@1 57.03 (57.03) Prec@5 94.53 (94.53) TRAIN-SEARCH [2018-10-02-16:38:42] Epoch: [112][100/196] Time 0.50 (0.53) Data 0.00 (0.01) Loss 1.210 (1.190) Prec@1 63.28 (58.06) Prec@5 94.53 (95.06) TRAIN-SEARCH [2018-10-02-16:39:35] Epoch: [112][195/196] Time 0.58 (0.54) Data 0.00 (0.01) Loss 1.465 (1.185) Prec@1 60.00 (58.15) Prec@5 90.00 (95.06) VALID-SEARCH [2018-10-02-16:39:36] Epoch: [112][000/196] Loss 2.113 (2.113) Prec@1 39.84 (39.84) Prec@5 82.81 (82.81) VALID-SEARCH [2018-10-02-16:39:41] Epoch: [112][100/196] Loss 1.453 (1.662) Prec@1 49.22 (46.28) Prec@5 96.09 (89.72) VALID-SEARCH [2018-10-02-16:39:47] Epoch: [112][195/196] Loss 1.134 (1.637) Prec@1 55.00 (46.24) Prec@5 95.00 (90.02) 112/250, Train-Accuracy = 58.15, Test-Accuracy = 46.24 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-16:39:48] [Epoch=113/250] [Need: 04:33:22] [LR=0.0292 ~ 0.0292] [Batch=128], tau=7.288 genotype = Genotype(normal=[('dil_conv_5x5', 1, 0.1280810385942459), ('dil_conv_3x3', 0, 0.12736503779888153), ('skip_connect', 0, 0.12815944850444794), ('sep_conv_3x3', 2, 0.12724339962005615), ('skip_connect', 0, 0.12894415855407715), ('sep_conv_5x5', 3, 0.1269722878932953), ('max_pool_3x3', 0, 0.12741614878177643), ('max_pool_3x3', 1, 0.12701453268527985)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.1258317232131958), ('dil_conv_3x3', 1, 0.12569622695446014), ('max_pool_3x3', 0, 0.1275736391544342), ('avg_pool_3x3', 1, 0.12587332725524902), ('max_pool_3x3', 0, 0.12611006200313568), ('max_pool_3x3', 1, 0.12601010501384735), ('skip_connect', 0, 0.1260250061750412), ('max_pool_3x3', 1, 0.12564265727996826)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-16:39:49] Epoch: [113][000/196] Time 1.07 (1.07) Data 0.63 (0.63) Loss 1.150 (1.150) Prec@1 57.03 (57.03) Prec@5 94.53 (94.53) TRAIN-SEARCH [2018-10-02-16:40:40] Epoch: [113][100/196] Time 0.68 (0.51) Data 0.03 (0.01) Loss 1.129 (1.189) Prec@1 64.84 (57.64) Prec@5 94.53 (95.44) TRAIN-SEARCH [2018-10-02-16:41:35] Epoch: [113][195/196] Time 0.41 (0.55) Data 0.00 (0.01) Loss 1.418 (1.190) Prec@1 50.00 (57.94) Prec@5 95.00 (95.24) VALID-SEARCH [2018-10-02-16:41:36] Epoch: [113][000/196] Loss 2.231 (2.231) Prec@1 35.94 (35.94) Prec@5 81.25 (81.25) VALID-SEARCH [2018-10-02-16:41:42] Epoch: [113][100/196] Loss 1.992 (1.673) Prec@1 42.19 (45.37) Prec@5 84.38 (89.36) VALID-SEARCH [2018-10-02-16:41:47] Epoch: [113][195/196] Loss 1.777 (1.711) Prec@1 27.50 (44.55) Prec@5 90.00 (88.86) 113/250, Train-Accuracy = 57.94, Test-Accuracy = 44.55 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-16:41:48] [Epoch=114/250] [Need: 04:31:48] [LR=0.0289 ~ 0.0289] [Batch=128], tau=7.263999999999999 genotype = Genotype(normal=[('dil_conv_5x5', 1, 0.12737257778644562), ('dil_conv_3x3', 0, 0.12720558047294617), ('skip_connect', 0, 0.12847980856895447), ('sep_conv_3x3', 2, 0.12814836204051971), ('skip_connect', 0, 0.12906551361083984), ('sep_conv_5x5', 3, 0.12727747857570648), ('sep_conv_3x3', 4, 0.12734489142894745), ('max_pool_3x3', 0, 0.12719455361366272)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.12613724172115326), ('dil_conv_5x5', 1, 0.12576569616794586), ('avg_pool_3x3', 0, 0.12651696801185608), ('sep_conv_5x5', 2, 0.12607133388519287), ('max_pool_3x3', 0, 0.12681254744529724), ('dil_conv_3x3', 1, 0.12549975514411926), ('skip_connect', 0, 0.12601739168167114), ('dil_conv_5x5', 1, 0.12592147290706635)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-16:41:49] Epoch: [114][000/196] Time 1.04 (1.04) Data 0.65 (0.65) Loss 1.183 (1.183) Prec@1 58.59 (58.59) Prec@5 95.31 (95.31) TRAIN-SEARCH [2018-10-02-16:42:42] Epoch: [114][100/196] Time 0.59 (0.53) Data 0.00 (0.01) Loss 1.179 (1.187) Prec@1 60.94 (57.90) Prec@5 98.44 (95.14) TRAIN-SEARCH [2018-10-02-16:43:35] Epoch: [114][195/196] Time 0.41 (0.55) Data 0.00 (0.01) Loss 1.309 (1.189) Prec@1 52.50 (57.82) Prec@5 95.00 (95.10) VALID-SEARCH [2018-10-02-16:43:36] Epoch: [114][000/196] Loss 1.632 (1.632) Prec@1 54.69 (54.69) Prec@5 96.09 (96.09) VALID-SEARCH [2018-10-02-16:43:41] Epoch: [114][100/196] Loss 1.477 (1.699) Prec@1 43.75 (44.42) Prec@5 93.75 (89.77) VALID-SEARCH [2018-10-02-16:43:45] Epoch: [114][195/196] Loss 1.502 (1.649) Prec@1 37.50 (45.25) Prec@5 87.50 (90.00) 114/250, Train-Accuracy = 57.82, Test-Accuracy = 45.25 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-16:43:47] [Epoch=115/250] [Need: 04:27:42] [LR=0.0286 ~ 0.0286] [Batch=128], tau=7.24 genotype = Genotype(normal=[('dil_conv_3x3', 0, 0.12768617272377014), ('dil_conv_5x5', 1, 0.12672345340251923), ('skip_connect', 0, 0.1284589022397995), ('sep_conv_3x3', 2, 0.12802375853061676), ('skip_connect', 0, 0.12870046496391296), ('dil_conv_3x3', 3, 0.1268199235200882), ('max_pool_3x3', 0, 0.12741351127624512), ('max_pool_3x3', 1, 0.12686391174793243)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.12654708325862885), ('dil_conv_3x3', 1, 0.1258067637681961), ('avg_pool_3x3', 0, 0.12656845152378082), ('max_pool_3x3', 1, 0.12632706761360168), ('max_pool_3x3', 0, 0.12705031037330627), ('dil_conv_3x3', 2, 0.12563161551952362), ('skip_connect', 0, 0.12598828971385956), ('dil_conv_3x3', 4, 0.125731959939003)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-16:43:48] Epoch: [115][000/196] Time 1.04 (1.04) Data 0.61 (0.61) Loss 1.233 (1.233) Prec@1 53.12 (53.12) Prec@5 96.88 (96.88) TRAIN-SEARCH [2018-10-02-16:44:42] Epoch: [115][100/196] Time 0.47 (0.55) Data 0.00 (0.01) Loss 1.102 (1.199) Prec@1 57.81 (58.15) Prec@5 93.75 (95.00) TRAIN-SEARCH [2018-10-02-16:45:34] Epoch: [115][195/196] Time 0.40 (0.54) Data 0.00 (0.01) Loss 1.434 (1.184) Prec@1 45.00 (58.15) Prec@5 95.00 (95.25) VALID-SEARCH [2018-10-02-16:45:35] Epoch: [115][000/196] Loss 2.732 (2.732) Prec@1 42.19 (42.19) Prec@5 89.84 (89.84) VALID-SEARCH [2018-10-02-16:45:40] Epoch: [115][100/196] Loss 1.384 (1.638) Prec@1 47.66 (47.76) Prec@5 94.53 (91.25) VALID-SEARCH [2018-10-02-16:45:45] Epoch: [115][195/196] Loss 1.415 (1.653) Prec@1 50.00 (47.58) Prec@5 92.50 (90.90) 115/250, Train-Accuracy = 58.15, Test-Accuracy = 47.58 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-16:45:45] [Epoch=116/250] [Need: 04:25:39] [LR=0.0283 ~ 0.0283] [Batch=128], tau=7.215999999999999 genotype = Genotype(normal=[('dil_conv_5x5', 0, 0.12781628966331482), ('skip_connect', 1, 0.12710188329219818), ('sep_conv_3x3', 2, 0.12847217917442322), ('dil_conv_3x3', 0, 0.12761415541172028), ('skip_connect', 0, 0.1287636160850525), ('sep_conv_3x3', 3, 0.12664878368377686), ('max_pool_3x3', 0, 0.12798047065734863), ('max_pool_3x3', 1, 0.12730802595615387)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.1265198439359665), ('dil_conv_5x5', 1, 0.12573778629302979), ('max_pool_3x3', 0, 0.12644729018211365), ('sep_conv_5x5', 2, 0.1258641928434372), ('max_pool_3x3', 0, 0.12620499730110168), ('max_pool_3x3', 1, 0.12558995187282562), ('max_pool_3x3', 0, 0.1259000450372696), ('max_pool_3x3', 1, 0.12576225399971008)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-16:45:47] Epoch: [116][000/196] Time 0.98 (0.98) Data 0.54 (0.54) Loss 1.130 (1.130) Prec@1 61.72 (61.72) Prec@5 96.88 (96.88) TRAIN-SEARCH [2018-10-02-16:46:40] Epoch: [116][100/196] Time 0.69 (0.54) Data 0.00 (0.01) Loss 1.018 (1.168) Prec@1 64.06 (58.78) Prec@5 99.22 (95.43) TRAIN-SEARCH [2018-10-02-16:47:32] Epoch: [116][195/196] Time 0.42 (0.54) Data 0.00 (0.01) Loss 1.212 (1.172) Prec@1 55.00 (58.88) Prec@5 97.50 (95.32) VALID-SEARCH [2018-10-02-16:47:33] Epoch: [116][000/196] Loss 1.979 (1.979) Prec@1 41.41 (41.41) Prec@5 88.28 (88.28) VALID-SEARCH [2018-10-02-16:47:39] Epoch: [116][100/196] Loss 1.374 (1.701) Prec@1 49.22 (47.35) Prec@5 93.75 (90.83) VALID-SEARCH [2018-10-02-16:47:42] Epoch: [116][195/196] Loss 1.583 (1.661) Prec@1 50.00 (46.78) Prec@5 92.50 (90.46) 116/250, Train-Accuracy = 58.88, Test-Accuracy = 46.78 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-16:47:43] [Epoch=117/250] [Need: 04:20:48] [LR=0.0280 ~ 0.0280] [Batch=128], tau=7.192 genotype = Genotype(normal=[('dil_conv_5x5', 0, 0.1284959465265274), ('skip_connect', 1, 0.1266385018825531), ('sep_conv_3x3', 2, 0.12830235064029694), ('skip_connect', 0, 0.12764760851860046), ('skip_connect', 0, 0.12931112945079803), ('sep_conv_3x3', 3, 0.12688568234443665), ('max_pool_3x3', 0, 0.1281774938106537), ('max_pool_3x3', 1, 0.12744413316249847)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.12583529949188232), ('sep_conv_3x3', 1, 0.1257474422454834), ('max_pool_3x3', 0, 0.12661559879779816), ('max_pool_3x3', 1, 0.1258653700351715), ('avg_pool_3x3', 0, 0.1261933445930481), ('max_pool_3x3', 1, 0.12581007182598114), ('avg_pool_3x3', 0, 0.1261722594499588), ('sep_conv_5x5', 1, 0.12592683732509613)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-16:47:45] Epoch: [117][000/196] Time 1.08 (1.08) Data 0.65 (0.65) Loss 1.154 (1.154) Prec@1 57.03 (57.03) Prec@5 95.31 (95.31) TRAIN-SEARCH [2018-10-02-16:48:40] Epoch: [117][100/196] Time 0.61 (0.56) Data 0.00 (0.01) Loss 1.201 (1.179) Prec@1 58.59 (57.96) Prec@5 93.75 (95.20) TRAIN-SEARCH [2018-10-02-16:49:31] Epoch: [117][195/196] Time 0.41 (0.55) Data 0.00 (0.01) Loss 1.387 (1.175) Prec@1 55.00 (58.46) Prec@5 95.00 (95.27) VALID-SEARCH [2018-10-02-16:49:32] Epoch: [117][000/196] Loss 1.244 (1.244) Prec@1 53.12 (53.12) Prec@5 92.97 (92.97) VALID-SEARCH [2018-10-02-16:49:37] Epoch: [117][100/196] Loss 1.673 (1.620) Prec@1 48.44 (46.47) Prec@5 94.53 (90.84) VALID-SEARCH [2018-10-02-16:49:40] Epoch: [117][195/196] Loss 2.652 (1.609) Prec@1 7.50 (46.39) Prec@5 57.50 (90.55) 117/250, Train-Accuracy = 58.46, Test-Accuracy = 46.39 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-16:49:41] [Epoch=118/250] [Need: 04:19:54] [LR=0.0277 ~ 0.0277] [Batch=128], tau=7.168 genotype = Genotype(normal=[('dil_conv_5x5', 0, 0.1283608376979828), ('skip_connect', 1, 0.12755951285362244), ('sep_conv_3x3', 2, 0.12816649675369263), ('skip_connect', 1, 0.1275225430727005), ('skip_connect', 0, 0.12910066545009613), ('sep_conv_3x3', 3, 0.12739796936511993), ('max_pool_3x3', 0, 0.12937776744365692), ('max_pool_3x3', 1, 0.12720845639705658)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.12652193009853363), ('max_pool_3x3', 1, 0.12553343176841736), ('max_pool_3x3', 0, 0.12672127783298492), ('max_pool_3x3', 1, 0.12597794830799103), ('dil_conv_5x5', 0, 0.1263488233089447), ('dil_conv_5x5', 1, 0.12557733058929443), ('sep_conv_5x5', 4, 0.12648022174835205), ('sep_conv_5x5', 1, 0.12614436447620392)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-16:49:43] Epoch: [118][000/196] Time 1.21 (1.21) Data 0.75 (0.75) Loss 1.041 (1.041) Prec@1 64.84 (64.84) Prec@5 100.00 (100.00) TRAIN-SEARCH [2018-10-02-16:50:36] Epoch: [118][100/196] Time 0.47 (0.54) Data 0.00 (0.01) Loss 1.361 (1.159) Prec@1 53.12 (59.07) Prec@5 94.53 (95.29) TRAIN-SEARCH [2018-10-02-16:51:29] Epoch: [118][195/196] Time 0.42 (0.55) Data 0.00 (0.01) Loss 1.351 (1.161) Prec@1 40.00 (59.17) Prec@5 97.50 (95.30) VALID-SEARCH [2018-10-02-16:51:30] Epoch: [118][000/196] Loss 1.557 (1.557) Prec@1 42.97 (42.97) Prec@5 89.84 (89.84) VALID-SEARCH [2018-10-02-16:51:35] Epoch: [118][100/196] Loss 1.335 (1.595) Prec@1 53.91 (45.64) Prec@5 90.62 (89.48) VALID-SEARCH [2018-10-02-16:51:38] Epoch: [118][195/196] Loss 1.235 (1.575) Prec@1 55.00 (45.76) Prec@5 87.50 (89.99) 118/250, Train-Accuracy = 59.17, Test-Accuracy = 45.76 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-16:51:39] [Epoch=119/250] [Need: 04:17:17] [LR=0.0273 ~ 0.0273] [Batch=128], tau=7.144 genotype = Genotype(normal=[('dil_conv_5x5', 0, 0.12800469994544983), ('skip_connect', 1, 0.12764091789722443), ('dil_conv_5x5', 0, 0.12893928587436676), ('sep_conv_5x5', 2, 0.1279231160879135), ('skip_connect', 0, 0.12905216217041016), ('sep_conv_3x3', 3, 0.12758955359458923), ('max_pool_3x3', 0, 0.129447802901268), ('max_pool_3x3', 1, 0.1270325928926468)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.1264500468969345), ('skip_connect', 1, 0.12565883994102478), ('max_pool_3x3', 0, 0.12644436955451965), ('max_pool_3x3', 1, 0.12587659060955048), ('max_pool_3x3', 0, 0.12630046904087067), ('avg_pool_3x3', 1, 0.1256944090127945), ('sep_conv_3x3', 0, 0.1259572058916092), ('dil_conv_5x5', 2, 0.12580496072769165)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-16:51:41] Epoch: [119][000/196] Time 1.07 (1.07) Data 0.61 (0.61) Loss 1.291 (1.291) Prec@1 55.47 (55.47) Prec@5 96.09 (96.09) TRAIN-SEARCH [2018-10-02-16:52:35] Epoch: [119][100/196] Time 0.50 (0.55) Data 0.01 (0.01) Loss 1.253 (1.165) Prec@1 58.59 (59.07) Prec@5 97.66 (95.29) TRAIN-SEARCH [2018-10-02-16:53:29] Epoch: [119][195/196] Time 0.50 (0.56) Data 0.00 (0.01) Loss 1.008 (1.157) Prec@1 65.00 (59.25) Prec@5 95.00 (95.40) VALID-SEARCH [2018-10-02-16:53:30] Epoch: [119][000/196] Loss 1.587 (1.587) Prec@1 52.34 (52.34) Prec@5 92.19 (92.19) VALID-SEARCH [2018-10-02-16:53:35] Epoch: [119][100/196] Loss 1.549 (1.582) Prec@1 47.66 (46.16) Prec@5 92.97 (90.18) VALID-SEARCH [2018-10-02-16:53:39] Epoch: [119][195/196] Loss 1.508 (1.623) Prec@1 42.50 (45.18) Prec@5 100.00 (90.05) 119/250, Train-Accuracy = 59.25, Test-Accuracy = 45.18 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-16:53:39] [Epoch=120/250] [Need: 04:20:17] [LR=0.0270 ~ 0.0270] [Batch=128], tau=7.12 genotype = Genotype(normal=[('dil_conv_5x5', 0, 0.12815341353416443), ('skip_connect', 1, 0.12787780165672302), ('dil_conv_5x5', 0, 0.12969718873500824), ('sep_conv_5x5', 2, 0.12765496969223022), ('max_pool_3x3', 0, 0.12883052229881287), ('sep_conv_3x3', 2, 0.12673689424991608), ('max_pool_3x3', 0, 0.12990722060203552), ('max_pool_3x3', 1, 0.12682655453681946)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.12646172940731049), ('dil_conv_3x3', 1, 0.12555968761444092), ('max_pool_3x3', 0, 0.1271422952413559), ('avg_pool_3x3', 1, 0.12555526196956635), ('avg_pool_3x3', 1, 0.12638069689273834), ('dil_conv_5x5', 0, 0.1263652890920639), ('dil_conv_3x3', 4, 0.125780388712883), ('max_pool_3x3', 0, 0.12575167417526245)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-16:53:41] Epoch: [120][000/196] Time 1.11 (1.11) Data 0.66 (0.66) Loss 1.102 (1.102) Prec@1 60.16 (60.16) Prec@5 93.75 (93.75) TRAIN-SEARCH [2018-10-02-16:54:36] Epoch: [120][100/196] Time 0.56 (0.55) Data 0.00 (0.01) Loss 0.920 (1.152) Prec@1 62.50 (58.87) Prec@5 97.66 (95.36) TRAIN-SEARCH [2018-10-02-16:55:29] Epoch: [120][195/196] Time 0.50 (0.56) Data 0.00 (0.01) Loss 1.311 (1.149) Prec@1 52.50 (59.24) Prec@5 92.50 (95.48) VALID-SEARCH [2018-10-02-16:55:31] Epoch: [120][000/196] Loss 1.534 (1.534) Prec@1 54.69 (54.69) Prec@5 88.28 (88.28) VALID-SEARCH [2018-10-02-16:55:34] Epoch: [120][100/196] Loss 1.398 (1.791) Prec@1 50.00 (41.89) Prec@5 89.84 (85.87) VALID-SEARCH [2018-10-02-16:55:38] Epoch: [120][195/196] Loss 1.172 (1.775) Prec@1 67.50 (42.02) Prec@5 90.00 (85.94) 120/250, Train-Accuracy = 59.24, Test-Accuracy = 42.02 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-16:55:38] [Epoch=121/250] [Need: 04:16:19] [LR=0.0267 ~ 0.0267] [Batch=128], tau=7.096 genotype = Genotype(normal=[('dil_conv_5x5', 0, 0.12842363119125366), ('skip_connect', 1, 0.12764109671115875), ('dil_conv_5x5', 0, 0.1289132833480835), ('sep_conv_5x5', 2, 0.12834778428077698), ('max_pool_3x3', 0, 0.1287347972393036), ('sep_conv_3x3', 2, 0.1270676851272583), ('max_pool_3x3', 0, 0.12936155498027802), ('max_pool_3x3', 1, 0.12696322798728943)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.1263188272714615), ('dil_conv_3x3', 1, 0.12571215629577637), ('skip_connect', 0, 0.12668265402317047), ('dil_conv_3x3', 2, 0.1259133219718933), ('avg_pool_3x3', 1, 0.1264835000038147), ('sep_conv_5x5', 2, 0.12595802545547485), ('max_pool_3x3', 0, 0.12625156342983246), ('avg_pool_3x3', 1, 0.12595660984516144)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-16:55:40] Epoch: [121][000/196] Time 1.10 (1.10) Data 0.70 (0.70) Loss 1.282 (1.282) Prec@1 51.56 (51.56) Prec@5 97.66 (97.66) TRAIN-SEARCH [2018-10-02-16:56:33] Epoch: [121][100/196] Time 0.50 (0.54) Data 0.00 (0.01) Loss 1.002 (1.139) Prec@1 64.84 (59.68) Prec@5 95.31 (95.66) TRAIN-SEARCH [2018-10-02-16:57:27] Epoch: [121][195/196] Time 0.43 (0.55) Data 0.00 (0.01) Loss 1.170 (1.145) Prec@1 62.50 (59.70) Prec@5 95.00 (95.53) VALID-SEARCH [2018-10-02-16:57:28] Epoch: [121][000/196] Loss 1.140 (1.140) Prec@1 56.25 (56.25) Prec@5 94.53 (94.53) VALID-SEARCH [2018-10-02-16:57:32] Epoch: [121][100/196] Loss 1.507 (1.579) Prec@1 50.78 (48.93) Prec@5 89.06 (91.75) VALID-SEARCH [2018-10-02-16:57:36] Epoch: [121][195/196] Loss 1.747 (1.602) Prec@1 37.50 (48.16) Prec@5 90.00 (91.08) 121/250, Train-Accuracy = 59.70, Test-Accuracy = 48.16 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-16:57:37] [Epoch=122/250] [Need: 04:12:02] [LR=0.0264 ~ 0.0264] [Batch=128], tau=7.072 genotype = Genotype(normal=[('dil_conv_5x5', 0, 0.12841391563415527), ('skip_connect', 1, 0.1269199550151825), ('dil_conv_5x5', 0, 0.12865152955055237), ('sep_conv_5x5', 2, 0.12830385565757751), ('max_pool_3x3', 0, 0.1292201727628708), ('sep_conv_3x3', 3, 0.12737245857715607), ('max_pool_3x3', 0, 0.12938764691352844), ('max_pool_3x3', 1, 0.12696582078933716)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.12590548396110535), ('max_pool_3x3', 1, 0.1255607157945633), ('skip_connect', 0, 0.12718914449214935), ('dil_conv_3x3', 2, 0.1257523149251938), ('sep_conv_5x5', 2, 0.1264847218990326), ('max_pool_3x3', 0, 0.12642355263233185), ('max_pool_3x3', 0, 0.12627282738685608), ('dil_conv_5x5', 4, 0.12601116299629211)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-16:57:38] Epoch: [122][000/196] Time 1.01 (1.01) Data 0.58 (0.58) Loss 1.100 (1.100) Prec@1 61.72 (61.72) Prec@5 95.31 (95.31) TRAIN-SEARCH [2018-10-02-16:58:34] Epoch: [122][100/196] Time 0.47 (0.56) Data 0.00 (0.01) Loss 1.326 (1.158) Prec@1 51.56 (59.32) Prec@5 95.31 (95.51) TRAIN-SEARCH [2018-10-02-16:59:26] Epoch: [122][195/196] Time 0.48 (0.56) Data 0.00 (0.01) Loss 1.148 (1.149) Prec@1 62.50 (59.46) Prec@5 95.00 (95.52) VALID-SEARCH [2018-10-02-16:59:27] Epoch: [122][000/196] Loss 1.984 (1.984) Prec@1 38.28 (38.28) Prec@5 72.66 (72.66) VALID-SEARCH [2018-10-02-16:59:31] Epoch: [122][100/196] Loss 1.700 (1.679) Prec@1 45.31 (45.84) Prec@5 91.41 (89.00) VALID-SEARCH [2018-10-02-16:59:35] Epoch: [122][195/196] Loss 1.657 (1.667) Prec@1 37.50 (45.82) Prec@5 92.50 (88.99) 122/250, Train-Accuracy = 59.46, Test-Accuracy = 45.82 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-16:59:36] [Epoch=123/250] [Need: 04:12:06] [LR=0.0261 ~ 0.0261] [Batch=128], tau=7.048 genotype = Genotype(normal=[('dil_conv_5x5', 0, 0.1275767982006073), ('sep_conv_5x5', 1, 0.12620607018470764), ('sep_conv_5x5', 2, 0.12843847274780273), ('dil_conv_5x5', 0, 0.12775415182113647), ('max_pool_3x3', 0, 0.1281890869140625), ('sep_conv_3x3', 3, 0.1277657449245453), ('max_pool_3x3', 0, 0.12841802835464478), ('max_pool_3x3', 1, 0.12720124423503876)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 1, 0.12576809525489807), ('max_pool_3x3', 0, 0.12559115886688232), ('skip_connect', 0, 0.1268332451581955), ('avg_pool_3x3', 1, 0.1255294382572174), ('max_pool_3x3', 0, 0.1270780861377716), ('dil_conv_5x5', 1, 0.1258019655942917), ('avg_pool_3x3', 0, 0.1264643520116806), ('dil_conv_5x5', 3, 0.1264360547065735)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-16:59:37] Epoch: [123][000/196] Time 1.00 (1.00) Data 0.59 (0.59) Loss 1.023 (1.023) Prec@1 71.09 (71.09) Prec@5 94.53 (94.53) TRAIN-SEARCH [2018-10-02-17:00:31] Epoch: [123][100/196] Time 0.74 (0.55) Data 0.01 (0.01) Loss 1.104 (1.132) Prec@1 60.16 (59.48) Prec@5 94.53 (95.50) TRAIN-SEARCH [2018-10-02-17:01:23] Epoch: [123][195/196] Time 0.44 (0.54) Data 0.00 (0.01) Loss 1.036 (1.147) Prec@1 62.50 (59.01) Prec@5 97.50 (95.45) VALID-SEARCH [2018-10-02-17:01:24] Epoch: [123][000/196] Loss 1.532 (1.532) Prec@1 46.88 (46.88) Prec@5 88.28 (88.28) VALID-SEARCH [2018-10-02-17:01:30] Epoch: [123][100/196] Loss 1.222 (1.614) Prec@1 55.47 (48.31) Prec@5 94.53 (90.76) VALID-SEARCH [2018-10-02-17:01:33] Epoch: [123][195/196] Loss 1.507 (1.559) Prec@1 55.00 (48.96) Prec@5 95.00 (91.64) 123/250, Train-Accuracy = 59.01, Test-Accuracy = 48.96 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-17:01:34] [Epoch=124/250] [Need: 04:08:45] [LR=0.0258 ~ 0.0258] [Batch=128], tau=7.024 genotype = Genotype(normal=[('skip_connect', 0, 0.1268397867679596), ('skip_connect', 1, 0.1266372948884964), ('dil_conv_5x5', 0, 0.12835489213466644), ('sep_conv_5x5', 2, 0.12760144472122192), ('max_pool_3x3', 0, 0.12846311926841736), ('sep_conv_3x3', 3, 0.12753991782665253), ('max_pool_3x3', 0, 0.12787331640720367), ('sep_conv_3x3', 4, 0.12753698229789734)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.12630212306976318), ('dil_conv_3x3', 1, 0.12547940015792847), ('max_pool_3x3', 0, 0.12648138403892517), ('avg_pool_3x3', 1, 0.12598800659179688), ('max_pool_3x3', 0, 0.12693087756633759), ('max_pool_3x3', 1, 0.12582002580165863), ('avg_pool_3x3', 0, 0.12656669318675995), ('dil_conv_5x5', 2, 0.12635371088981628)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-17:01:36] Epoch: [124][000/196] Time 1.17 (1.17) Data 0.74 (0.74) Loss 1.012 (1.012) Prec@1 61.72 (61.72) Prec@5 97.66 (97.66) TRAIN-SEARCH [2018-10-02-17:02:33] Epoch: [124][100/196] Time 0.69 (0.58) Data 0.00 (0.01) Loss 1.150 (1.126) Prec@1 58.59 (60.81) Prec@5 96.09 (95.70) TRAIN-SEARCH [2018-10-02-17:03:25] Epoch: [124][195/196] Time 0.52 (0.57) Data 0.00 (0.01) Loss 0.981 (1.139) Prec@1 67.50 (59.90) Prec@5 92.50 (95.68) VALID-SEARCH [2018-10-02-17:03:27] Epoch: [124][000/196] Loss 1.204 (1.204) Prec@1 59.38 (59.38) Prec@5 94.53 (94.53) VALID-SEARCH [2018-10-02-17:03:31] Epoch: [124][100/196] Loss 1.371 (1.599) Prec@1 53.91 (45.13) Prec@5 92.19 (90.29) VALID-SEARCH [2018-10-02-17:03:35] Epoch: [124][195/196] Loss 1.070 (1.641) Prec@1 62.50 (44.59) Prec@5 97.50 (89.86) 124/250, Train-Accuracy = 59.90, Test-Accuracy = 44.59 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-17:03:35] [Epoch=125/250] [Need: 04:12:27] [LR=0.0255 ~ 0.0255] [Batch=128], tau=7.0 genotype = Genotype(normal=[('skip_connect', 0, 0.12714631855487823), ('skip_connect', 1, 0.12666940689086914), ('skip_connect', 0, 0.1281261444091797), ('sep_conv_5x5', 2, 0.12764817476272583), ('max_pool_3x3', 0, 0.12790487706661224), ('sep_conv_3x3', 3, 0.12764014303684235), ('max_pool_3x3', 0, 0.12748438119888306), ('sep_conv_3x3', 4, 0.1273270845413208)], normal_concat=range(2, 6), reduce=[('skip_connect', 0, 0.12685886025428772), ('dil_conv_3x3', 1, 0.12575000524520874), ('max_pool_3x3', 0, 0.12698107957839966), ('skip_connect', 1, 0.1258859783411026), ('max_pool_3x3', 0, 0.1265755444765091), ('max_pool_3x3', 1, 0.12654641270637512), ('avg_pool_3x3', 0, 0.1264617145061493), ('sep_conv_5x5', 4, 0.12585996091365814)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-17:03:37] Epoch: [125][000/196] Time 0.98 (0.98) Data 0.57 (0.57) Loss 1.177 (1.177) Prec@1 56.25 (56.25) Prec@5 96.88 (96.88) TRAIN-SEARCH [2018-10-02-17:04:33] Epoch: [125][100/196] Time 0.43 (0.56) Data 0.00 (0.01) Loss 1.006 (1.138) Prec@1 64.84 (60.33) Prec@5 96.88 (95.94) TRAIN-SEARCH [2018-10-02-17:05:25] Epoch: [125][195/196] Time 0.46 (0.56) Data 0.00 (0.01) Loss 1.368 (1.144) Prec@1 42.50 (59.82) Prec@5 92.50 (95.79) VALID-SEARCH [2018-10-02-17:05:26] Epoch: [125][000/196] Loss 1.571 (1.571) Prec@1 48.44 (48.44) Prec@5 92.19 (92.19) VALID-SEARCH [2018-10-02-17:05:30] Epoch: [125][100/196] Loss 1.491 (1.560) Prec@1 44.53 (47.17) Prec@5 91.41 (89.96) VALID-SEARCH [2018-10-02-17:05:33] Epoch: [125][195/196] Loss 1.520 (1.559) Prec@1 47.50 (46.67) Prec@5 90.00 (89.92) 125/250, Train-Accuracy = 59.82, Test-Accuracy = 46.67 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-17:05:34] [Epoch=126/250] [Need: 04:05:22] [LR=0.0252 ~ 0.0252] [Batch=128], tau=6.976 genotype = Genotype(normal=[('skip_connect', 0, 0.12715105712413788), ('skip_connect', 1, 0.12660923600196838), ('skip_connect', 0, 0.1284136325120926), ('skip_connect', 1, 0.12779974937438965), ('max_pool_3x3', 0, 0.12941141426563263), ('sep_conv_3x3', 3, 0.1273418813943863), ('skip_connect', 0, 0.12750062346458435), ('max_pool_3x3', 1, 0.12668746709823608)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.12666651606559753), ('dil_conv_5x5', 1, 0.12531664967536926), ('max_pool_3x3', 0, 0.12635491788387299), ('max_pool_3x3', 1, 0.12551364302635193), ('max_pool_3x3', 0, 0.1262119710445404), ('max_pool_3x3', 1, 0.1259680688381195), ('max_pool_3x3', 0, 0.12675705552101135), ('dil_conv_5x5', 1, 0.12617990374565125)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-17:05:36] Epoch: [126][000/196] Time 1.01 (1.01) Data 0.58 (0.58) Loss 1.223 (1.223) Prec@1 60.16 (60.16) Prec@5 95.31 (95.31) TRAIN-SEARCH [2018-10-02-17:06:30] Epoch: [126][100/196] Time 0.64 (0.55) Data 0.02 (0.01) Loss 1.078 (1.136) Prec@1 60.16 (60.27) Prec@5 99.22 (95.65) TRAIN-SEARCH [2018-10-02-17:07:24] Epoch: [126][195/196] Time 0.47 (0.56) Data 0.00 (0.01) Loss 1.433 (1.134) Prec@1 50.00 (60.25) Prec@5 97.50 (95.62) VALID-SEARCH [2018-10-02-17:07:25] Epoch: [126][000/196] Loss 1.262 (1.262) Prec@1 53.12 (53.12) Prec@5 95.31 (95.31) VALID-SEARCH [2018-10-02-17:07:29] Epoch: [126][100/196] Loss 1.159 (1.658) Prec@1 57.81 (45.95) Prec@5 95.31 (89.52) VALID-SEARCH [2018-10-02-17:07:32] Epoch: [126][195/196] Loss 0.950 (1.612) Prec@1 65.00 (46.88) Prec@5 100.00 (90.01) 126/250, Train-Accuracy = 60.25, Test-Accuracy = 46.88 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-17:07:35] [Epoch=127/250] [Need: 04:07:55] [LR=0.0249 ~ 0.0249] [Batch=128], tau=6.952 genotype = Genotype(normal=[('skip_connect', 0, 0.12789463996887207), ('dil_conv_5x5', 1, 0.12617026269435883), ('skip_connect', 0, 0.1293737143278122), ('skip_connect', 1, 0.12834559381008148), ('max_pool_3x3', 0, 0.12848560512065887), ('sep_conv_3x3', 3, 0.12734770774841309), ('sep_conv_3x3', 4, 0.12713642418384552), ('skip_connect', 0, 0.12699724733829498)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.1265982985496521), ('dil_conv_3x3', 1, 0.12577404081821442), ('max_pool_3x3', 0, 0.12668700516223907), ('max_pool_3x3', 1, 0.126275897026062), ('max_pool_3x3', 0, 0.12642547488212585), ('sep_conv_3x3', 3, 0.12625043094158173), ('max_pool_3x3', 0, 0.12648381292819977), ('dil_conv_3x3', 4, 0.12599746882915497)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-17:07:37] Epoch: [127][000/196] Time 1.22 (1.22) Data 0.70 (0.70) Loss 0.986 (0.986) Prec@1 67.97 (67.97) Prec@5 96.88 (96.88) TRAIN-SEARCH [2018-10-02-17:08:33] Epoch: [127][100/196] Time 0.56 (0.57) Data 0.00 (0.01) Loss 1.185 (1.125) Prec@1 61.72 (60.23) Prec@5 93.75 (95.70) TRAIN-SEARCH [2018-10-02-17:09:27] Epoch: [127][195/196] Time 0.39 (0.57) Data 0.00 (0.01) Loss 1.041 (1.140) Prec@1 67.50 (59.90) Prec@5 95.00 (95.74) VALID-SEARCH [2018-10-02-17:09:27] Epoch: [127][000/196] Loss 1.489 (1.489) Prec@1 50.00 (50.00) Prec@5 90.62 (90.62) VALID-SEARCH [2018-10-02-17:09:31] Epoch: [127][100/196] Loss 1.249 (1.550) Prec@1 55.47 (47.03) Prec@5 94.53 (90.72) VALID-SEARCH [2018-10-02-17:09:35] Epoch: [127][195/196] Loss 1.145 (1.532) Prec@1 55.00 (47.60) Prec@5 95.00 (90.81) 127/250, Train-Accuracy = 59.90, Test-Accuracy = 47.60 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-17:09:36] [Epoch=128/250] [Need: 04:06:41] [LR=0.0246 ~ 0.0246] [Batch=128], tau=6.928 genotype = Genotype(normal=[('skip_connect', 0, 0.1277453601360321), ('dil_conv_5x5', 1, 0.1269223392009735), ('skip_connect', 0, 0.12885093688964844), ('skip_connect', 1, 0.1287807673215866), ('max_pool_3x3', 0, 0.12753719091415405), ('sep_conv_3x3', 3, 0.12719324231147766), ('skip_connect', 0, 0.12795096635818481), ('sep_conv_3x3', 4, 0.12736278772354126)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.12686482071876526), ('avg_pool_3x3', 1, 0.1255832314491272), ('max_pool_3x3', 0, 0.126477912068367), ('max_pool_3x3', 1, 0.12623943388462067), ('max_pool_3x3', 0, 0.1262960582971573), ('sep_conv_3x3', 3, 0.1260802000761032), ('dil_conv_5x5', 2, 0.12594377994537354), ('max_pool_3x3', 0, 0.12580746412277222)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-17:09:39] Epoch: [128][000/196] Time 1.59 (1.59) Data 0.91 (0.91) Loss 1.059 (1.059) Prec@1 59.38 (59.38) Prec@5 95.31 (95.31) TRAIN-SEARCH [2018-10-02-17:10:35] Epoch: [128][100/196] Time 0.55 (0.57) Data 0.00 (0.01) Loss 1.055 (1.130) Prec@1 61.72 (60.06) Prec@5 95.31 (95.67) TRAIN-SEARCH [2018-10-02-17:11:26] Epoch: [128][195/196] Time 0.35 (0.55) Data 0.00 (0.01) Loss 1.356 (1.124) Prec@1 57.50 (60.36) Prec@5 97.50 (95.80) VALID-SEARCH [2018-10-02-17:11:26] Epoch: [128][000/196] Loss 1.598 (1.598) Prec@1 48.44 (48.44) Prec@5 92.19 (92.19) VALID-SEARCH [2018-10-02-17:11:30] Epoch: [128][100/196] Loss 1.352 (1.546) Prec@1 52.34 (47.38) Prec@5 92.19 (91.27) VALID-SEARCH [2018-10-02-17:11:34] Epoch: [128][195/196] Loss 1.053 (1.537) Prec@1 57.50 (48.14) Prec@5 95.00 (91.27) 128/250, Train-Accuracy = 60.36, Test-Accuracy = 48.14 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-17:11:35] [Epoch=129/250] [Need: 03:59:08] [LR=0.0243 ~ 0.0243] [Batch=128], tau=6.904 genotype = Genotype(normal=[('skip_connect', 0, 0.12786675989627838), ('dil_conv_5x5', 1, 0.1266293078660965), ('skip_connect', 1, 0.12923449277877808), ('skip_connect', 0, 0.12882806360721588), ('sep_conv_3x3', 3, 0.12753552198410034), ('max_pool_3x3', 0, 0.12722943723201752), ('max_pool_3x3', 0, 0.12817998230457306), ('max_pool_3x3', 1, 0.1269746571779251)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.12650026381015778), ('avg_pool_3x3', 1, 0.12539266049861908), ('max_pool_3x3', 0, 0.1267574578523636), ('dil_conv_3x3', 1, 0.12656061351299286), ('max_pool_3x3', 0, 0.12587375938892365), ('sep_conv_3x3', 3, 0.1256318837404251), ('sep_conv_5x5', 4, 0.12622001767158508), ('max_pool_3x3', 1, 0.125810906291008)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-17:11:37] Epoch: [129][000/196] Time 1.13 (1.13) Data 0.66 (0.66) Loss 0.974 (0.974) Prec@1 62.50 (62.50) Prec@5 100.00 (100.00) TRAIN-SEARCH [2018-10-02-17:12:33] Epoch: [129][100/196] Time 0.72 (0.57) Data 0.02 (0.01) Loss 1.204 (1.129) Prec@1 56.25 (59.96) Prec@5 95.31 (95.66) TRAIN-SEARCH [2018-10-02-17:13:25] Epoch: [129][195/196] Time 0.28 (0.56) Data 0.00 (0.01) Loss 1.461 (1.130) Prec@1 55.00 (60.05) Prec@5 92.50 (95.62) VALID-SEARCH [2018-10-02-17:13:25] Epoch: [129][000/196] Loss 2.357 (2.357) Prec@1 21.88 (21.88) Prec@5 64.84 (64.84) VALID-SEARCH [2018-10-02-17:13:29] Epoch: [129][100/196] Loss 1.111 (1.536) Prec@1 64.84 (48.30) Prec@5 94.53 (90.52) VALID-SEARCH [2018-10-02-17:13:33] Epoch: [129][195/196] Loss 0.879 (1.522) Prec@1 62.50 (48.64) Prec@5 100.00 (91.00) 129/250, Train-Accuracy = 60.05, Test-Accuracy = 48.64 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-17:13:34] [Epoch=130/250] [Need: 03:58:39] [LR=0.0240 ~ 0.0240] [Batch=128], tau=6.88 genotype = Genotype(normal=[('skip_connect', 0, 0.12808214128017426), ('dil_conv_5x5', 1, 0.1270071566104889), ('skip_connect', 1, 0.12880615890026093), ('skip_connect', 0, 0.12877881526947021), ('dil_conv_3x3', 3, 0.12758870422840118), ('skip_connect', 0, 0.1269729733467102), ('max_pool_3x3', 0, 0.1292257308959961), ('sep_conv_3x3', 4, 0.1270313411951065)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.12612777948379517), ('dil_conv_3x3', 1, 0.12558989226818085), ('max_pool_3x3', 1, 0.12674254179000854), ('dil_conv_3x3', 2, 0.12627409398555756), ('max_pool_3x3', 0, 0.1261582374572754), ('sep_conv_3x3', 3, 0.12595371901988983), ('dil_conv_5x5', 1, 0.12634992599487305), ('sep_conv_5x5', 4, 0.1261257529258728)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-17:13:37] Epoch: [130][000/196] Time 1.65 (1.65) Data 1.00 (1.00) Loss 1.073 (1.073) Prec@1 68.75 (68.75) Prec@5 96.88 (96.88) TRAIN-SEARCH [2018-10-02-17:14:35] Epoch: [130][100/196] Time 0.61 (0.59) Data 0.00 (0.01) Loss 1.058 (1.131) Prec@1 61.72 (59.84) Prec@5 96.09 (95.84) TRAIN-SEARCH [2018-10-02-17:15:25] Epoch: [130][195/196] Time 0.35 (0.56) Data 0.00 (0.01) Loss 1.155 (1.126) Prec@1 47.50 (59.98) Prec@5 100.00 (95.75) VALID-SEARCH [2018-10-02-17:15:26] Epoch: [130][000/196] Loss 1.338 (1.338) Prec@1 50.78 (50.78) Prec@5 94.53 (94.53) VALID-SEARCH [2018-10-02-17:15:30] Epoch: [130][100/196] Loss 1.689 (1.728) Prec@1 39.84 (46.99) Prec@5 89.06 (89.87) VALID-SEARCH [2018-10-02-17:15:35] Epoch: [130][195/196] Loss 1.402 (1.663) Prec@1 52.50 (47.77) Prec@5 87.50 (90.49) 130/250, Train-Accuracy = 59.98, Test-Accuracy = 47.77 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-17:15:36] [Epoch=131/250] [Need: 04:00:59] [LR=0.0237 ~ 0.0237] [Batch=128], tau=6.856 genotype = Genotype(normal=[('skip_connect', 0, 0.12875308096408844), ('dil_conv_5x5', 1, 0.1269179731607437), ('skip_connect', 0, 0.12977232038974762), ('skip_connect', 1, 0.12875424325466156), ('sep_conv_3x3', 3, 0.12757061421871185), ('skip_connect', 0, 0.12727923691272736), ('max_pool_3x3', 0, 0.12939848005771637), ('skip_connect', 1, 0.12706468999385834)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 0, 0.12632767856121063), ('dil_conv_3x3', 1, 0.1256055235862732), ('max_pool_3x3', 0, 0.12667106091976166), ('max_pool_3x3', 1, 0.12578338384628296), ('max_pool_3x3', 0, 0.12657001614570618), ('dil_conv_5x5', 1, 0.1259591281414032), ('dil_conv_5x5', 3, 0.1265491098165512), ('max_pool_3x3', 1, 0.12620526552200317)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-17:15:38] Epoch: [131][000/196] Time 1.79 (1.79) Data 0.97 (0.97) Loss 1.243 (1.243) Prec@1 53.12 (53.12) Prec@5 95.31 (95.31) TRAIN-SEARCH [2018-10-02-17:16:36] Epoch: [131][100/196] Time 0.46 (0.59) Data 0.00 (0.01) Loss 1.039 (1.116) Prec@1 64.06 (60.88) Prec@5 96.09 (95.62) TRAIN-SEARCH [2018-10-02-17:17:26] Epoch: [131][195/196] Time 0.32 (0.56) Data 0.00 (0.01) Loss 1.155 (1.118) Prec@1 57.50 (60.82) Prec@5 97.50 (95.65) VALID-SEARCH [2018-10-02-17:17:26] Epoch: [131][000/196] Loss 1.599 (1.599) Prec@1 49.22 (49.22) Prec@5 92.19 (92.19) VALID-SEARCH [2018-10-02-17:17:30] Epoch: [131][100/196] Loss 1.656 (1.640) Prec@1 47.66 (44.45) Prec@5 81.25 (88.58) VALID-SEARCH [2018-10-02-17:17:36] Epoch: [131][195/196] Loss 2.289 (1.614) Prec@1 25.00 (45.56) Prec@5 72.50 (88.91) 131/250, Train-Accuracy = 60.82, Test-Accuracy = 45.56 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-17:17:37] [Epoch=132/250] [Need: 03:57:50] [LR=0.0233 ~ 0.0233] [Batch=128], tau=6.832 genotype = Genotype(normal=[('skip_connect', 0, 0.1297396868467331), ('dil_conv_5x5', 1, 0.12646374106407166), ('skip_connect', 0, 0.12939384579658508), ('skip_connect', 1, 0.1284509301185608), ('max_pool_3x3', 0, 0.1272505819797516), ('sep_conv_3x3', 3, 0.12710876762866974), ('max_pool_3x3', 0, 0.12893345952033997), ('skip_connect', 1, 0.12750452756881714)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.12603072822093964), ('max_pool_3x3', 1, 0.125406414270401), ('dil_conv_5x5', 0, 0.12595787644386292), ('max_pool_3x3', 1, 0.1257357895374298), ('max_pool_3x3', 0, 0.12598581612110138), ('dil_conv_5x5', 1, 0.12595991790294647), ('max_pool_3x3', 0, 0.126225546002388), ('dil_conv_5x5', 4, 0.12585803866386414)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-17:17:39] Epoch: [132][000/196] Time 1.69 (1.69) Data 0.89 (0.89) Loss 1.191 (1.191) Prec@1 53.12 (53.12) Prec@5 95.31 (95.31) TRAIN-SEARCH [2018-10-02-17:18:37] Epoch: [132][100/196] Time 0.45 (0.59) Data 0.00 (0.01) Loss 0.969 (1.123) Prec@1 67.97 (60.44) Prec@5 96.88 (95.67) TRAIN-SEARCH [2018-10-02-17:19:26] Epoch: [132][195/196] Time 0.46 (0.56) Data 0.00 (0.01) Loss 1.077 (1.106) Prec@1 57.50 (60.86) Prec@5 100.00 (95.96) VALID-SEARCH [2018-10-02-17:19:27] Epoch: [132][000/196] Loss 1.188 (1.188) Prec@1 62.50 (62.50) Prec@5 94.53 (94.53) VALID-SEARCH [2018-10-02-17:19:33] Epoch: [132][100/196] Loss 1.258 (1.577) Prec@1 54.69 (50.11) Prec@5 92.97 (92.44) VALID-SEARCH [2018-10-02-17:19:38] Epoch: [132][195/196] Loss 1.434 (1.562) Prec@1 65.00 (50.02) Prec@5 95.00 (92.37) 132/250, Train-Accuracy = 60.86, Test-Accuracy = 50.02 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-17:19:39] [Epoch=133/250] [Need: 03:58:40] [LR=0.0230 ~ 0.0230] [Batch=128], tau=6.808 genotype = Genotype(normal=[('skip_connect', 0, 0.12953202426433563), ('dil_conv_5x5', 1, 0.12655037641525269), ('skip_connect', 0, 0.12932412326335907), ('skip_connect', 1, 0.12911854684352875), ('skip_connect', 0, 0.12788863480091095), ('sep_conv_3x3', 3, 0.12736673653125763), ('skip_connect', 0, 0.12840555608272552), ('sep_conv_3x3', 4, 0.12747330963611603)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.1261998414993286), ('dil_conv_5x5', 1, 0.12578800320625305), ('avg_pool_3x3', 0, 0.12647928297519684), ('avg_pool_3x3', 1, 0.12577812373638153), ('max_pool_3x3', 1, 0.12639246881008148), ('dil_conv_5x5', 3, 0.1260797530412674), ('dil_conv_5x5', 3, 0.1258624643087387), ('max_pool_3x3', 0, 0.1258019655942917)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-17:19:41] Epoch: [133][000/196] Time 1.46 (1.46) Data 0.78 (0.78) Loss 1.256 (1.256) Prec@1 53.91 (53.91) Prec@5 91.41 (91.41) TRAIN-SEARCH [2018-10-02-17:20:36] Epoch: [133][100/196] Time 0.60 (0.56) Data 0.00 (0.01) Loss 1.076 (1.115) Prec@1 61.72 (60.60) Prec@5 95.31 (95.75) TRAIN-SEARCH [2018-10-02-17:21:26] Epoch: [133][195/196] Time 0.42 (0.54) Data 0.00 (0.01) Loss 1.191 (1.111) Prec@1 50.00 (60.57) Prec@5 95.00 (95.98) VALID-SEARCH [2018-10-02-17:21:28] Epoch: [133][000/196] Loss 1.331 (1.331) Prec@1 51.56 (51.56) Prec@5 94.53 (94.53) VALID-SEARCH [2018-10-02-17:21:33] Epoch: [133][100/196] Loss 1.161 (1.500) Prec@1 57.03 (49.27) Prec@5 96.09 (91.51) VALID-SEARCH [2018-10-02-17:21:38] Epoch: [133][195/196] Loss 1.372 (1.540) Prec@1 45.00 (48.78) Prec@5 92.50 (90.40) 133/250, Train-Accuracy = 60.57, Test-Accuracy = 48.78 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-17:21:39] [Epoch=134/250] [Need: 03:51:48] [LR=0.0227 ~ 0.0227] [Batch=128], tau=6.784 genotype = Genotype(normal=[('skip_connect', 0, 0.12905260920524597), ('dil_conv_5x5', 1, 0.12678030133247375), ('skip_connect', 0, 0.12966306507587433), ('skip_connect', 1, 0.12893667817115784), ('sep_conv_3x3', 3, 0.1280677169561386), ('skip_connect', 0, 0.12803253531455994), ('skip_connect', 0, 0.12825094163417816), ('sep_conv_5x5', 4, 0.12742888927459717)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.12646549940109253), ('dil_conv_5x5', 1, 0.12625110149383545), ('max_pool_3x3', 0, 0.1262609362602234), ('sep_conv_5x5', 2, 0.12573957443237305), ('avg_pool_3x3', 0, 0.1261640191078186), ('max_pool_3x3', 1, 0.12585467100143433), ('max_pool_3x3', 1, 0.12604568898677826), ('dil_conv_3x3', 2, 0.12576502561569214)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-17:21:42] Epoch: [134][000/196] Time 1.86 (1.86) Data 1.07 (1.07) Loss 1.201 (1.201) Prec@1 53.91 (53.91) Prec@5 96.88 (96.88) TRAIN-SEARCH [2018-10-02-17:22:37] Epoch: [134][100/196] Time 0.48 (0.57) Data 0.00 (0.01) Loss 0.954 (1.087) Prec@1 67.19 (61.83) Prec@5 98.44 (96.08) TRAIN-SEARCH [2018-10-02-17:23:27] Epoch: [134][195/196] Time 0.46 (0.55) Data 0.00 (0.01) Loss 1.050 (1.097) Prec@1 60.00 (61.60) Prec@5 95.00 (95.90) VALID-SEARCH [2018-10-02-17:23:28] Epoch: [134][000/196] Loss 1.696 (1.696) Prec@1 36.72 (36.72) Prec@5 94.53 (94.53) VALID-SEARCH [2018-10-02-17:23:33] Epoch: [134][100/196] Loss 1.673 (1.529) Prec@1 36.72 (48.88) Prec@5 89.06 (91.19) VALID-SEARCH [2018-10-02-17:23:38] Epoch: [134][195/196] Loss 1.433 (1.559) Prec@1 62.50 (48.05) Prec@5 97.50 (90.89) 134/250, Train-Accuracy = 61.60, Test-Accuracy = 48.05 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-17:23:39] [Epoch=135/250] [Need: 03:50:33] [LR=0.0224 ~ 0.0224] [Batch=128], tau=6.76 genotype = Genotype(normal=[('skip_connect', 0, 0.12961627542972565), ('dil_conv_5x5', 1, 0.1267959475517273), ('skip_connect', 0, 0.1290781944990158), ('skip_connect', 1, 0.1289687156677246), ('sep_conv_3x3', 3, 0.12778528034687042), ('skip_connect', 0, 0.12762057781219482), ('max_pool_3x3', 0, 0.12764181196689606), ('dil_conv_3x3', 4, 0.12757761776447296)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.12704217433929443), ('dil_conv_5x5', 1, 0.1262567788362503), ('max_pool_3x3', 0, 0.12682677805423737), ('sep_conv_5x5', 2, 0.12611761689186096), ('skip_connect', 0, 0.12638896703720093), ('sep_conv_3x3', 2, 0.12581832706928253), ('dil_conv_5x5', 2, 0.1260455846786499), ('max_pool_3x3', 1, 0.1257994920015335)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-17:23:42] Epoch: [135][000/196] Time 1.71 (1.71) Data 0.97 (0.97) Loss 1.161 (1.161) Prec@1 62.50 (62.50) Prec@5 95.31 (95.31) TRAIN-SEARCH [2018-10-02-17:24:37] Epoch: [135][100/196] Time 0.62 (0.57) Data 0.00 (0.01) Loss 1.033 (1.105) Prec@1 66.41 (60.61) Prec@5 92.19 (96.09) TRAIN-SEARCH [2018-10-02-17:25:26] Epoch: [135][195/196] Time 0.43 (0.54) Data 0.00 (0.01) Loss 1.169 (1.106) Prec@1 47.50 (60.74) Prec@5 100.00 (96.02) VALID-SEARCH [2018-10-02-17:25:27] Epoch: [135][000/196] Loss 1.099 (1.099) Prec@1 60.16 (60.16) Prec@5 91.41 (91.41) VALID-SEARCH [2018-10-02-17:25:32] Epoch: [135][100/196] Loss 2.548 (1.650) Prec@1 47.66 (48.50) Prec@5 87.50 (90.68) VALID-SEARCH [2018-10-02-17:25:36] Epoch: [135][195/196] Loss 1.375 (1.693) Prec@1 47.50 (48.26) Prec@5 97.50 (90.95) 135/250, Train-Accuracy = 60.74, Test-Accuracy = 48.26 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-17:25:38] [Epoch=136/250] [Need: 03:45:24] [LR=0.0221 ~ 0.0221] [Batch=128], tau=6.736 genotype = Genotype(normal=[('skip_connect', 0, 0.1295134276151657), ('dil_conv_5x5', 1, 0.12650132179260254), ('skip_connect', 1, 0.1285281479358673), ('sep_conv_3x3', 2, 0.12845197319984436), ('skip_connect', 0, 0.12766322493553162), ('sep_conv_3x3', 3, 0.127579465508461), ('max_pool_3x3', 0, 0.12822335958480835), ('max_pool_3x3', 1, 0.12787148356437683)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.12660732865333557), ('avg_pool_3x3', 1, 0.1261567920446396), ('avg_pool_3x3', 0, 0.12637197971343994), ('max_pool_3x3', 1, 0.12626336514949799), ('dil_conv_3x3', 0, 0.1264721006155014), ('dil_conv_5x5', 2, 0.12602782249450684), ('dil_conv_5x5', 0, 0.12592671811580658), ('sep_conv_3x3', 4, 0.1256820112466812)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-17:25:40] Epoch: [136][000/196] Time 1.79 (1.79) Data 1.13 (1.13) Loss 1.106 (1.106) Prec@1 60.16 (60.16) Prec@5 97.66 (97.66) TRAIN-SEARCH [2018-10-02-17:26:39] Epoch: [136][100/196] Time 0.63 (0.60) Data 0.00 (0.02) Loss 1.076 (1.100) Prec@1 64.06 (61.25) Prec@5 97.66 (95.89) TRAIN-SEARCH [2018-10-02-17:27:28] Epoch: [136][195/196] Time 0.40 (0.56) Data 0.00 (0.01) Loss 1.133 (1.094) Prec@1 60.00 (61.33) Prec@5 97.50 (96.01) VALID-SEARCH [2018-10-02-17:27:30] Epoch: [136][000/196] Loss 1.380 (1.380) Prec@1 49.22 (49.22) Prec@5 91.41 (91.41) VALID-SEARCH [2018-10-02-17:27:35] Epoch: [136][100/196] Loss 1.437 (1.458) Prec@1 43.75 (50.89) Prec@5 92.97 (92.64) VALID-SEARCH [2018-10-02-17:27:40] Epoch: [136][195/196] Loss 1.852 (1.474) Prec@1 32.50 (51.04) Prec@5 82.50 (92.40) 136/250, Train-Accuracy = 61.33, Test-Accuracy = 51.04 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-17:27:41] [Epoch=137/250] [Need: 03:52:04] [LR=0.0218 ~ 0.0218] [Batch=128], tau=6.712 genotype = Genotype(normal=[('sep_conv_5x5', 0, 0.12800727784633636), ('dil_conv_3x3', 1, 0.12659448385238647), ('skip_connect', 1, 0.12828217446804047), ('skip_connect', 0, 0.12806010246276855), ('skip_connect', 1, 0.12816622853279114), ('skip_connect', 0, 0.12801110744476318), ('max_pool_3x3', 0, 0.12871024012565613), ('max_pool_3x3', 1, 0.12736046314239502)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.12700146436691284), ('avg_pool_3x3', 1, 0.12589360773563385), ('max_pool_3x3', 1, 0.12649431824684143), ('avg_pool_3x3', 0, 0.1264660656452179), ('avg_pool_3x3', 0, 0.12620332837104797), ('skip_connect', 1, 0.1257634162902832), ('max_pool_3x3', 0, 0.12603288888931274), ('sep_conv_5x5', 3, 0.1259683221578598)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-17:27:43] Epoch: [137][000/196] Time 1.59 (1.59) Data 1.00 (1.00) Loss 1.052 (1.052) Prec@1 64.84 (64.84) Prec@5 94.53 (94.53) TRAIN-SEARCH [2018-10-02-17:28:41] Epoch: [137][100/196] Time 0.59 (0.59) Data 0.00 (0.01) Loss 1.243 (1.093) Prec@1 57.81 (61.11) Prec@5 94.53 (96.16) TRAIN-SEARCH [2018-10-02-17:29:28] Epoch: [137][195/196] Time 0.63 (0.54) Data 0.00 (0.01) Loss 0.965 (1.098) Prec@1 62.50 (61.19) Prec@5 100.00 (96.07) VALID-SEARCH [2018-10-02-17:29:29] Epoch: [137][000/196] Loss 1.345 (1.345) Prec@1 53.12 (53.12) Prec@5 92.97 (92.97) VALID-SEARCH [2018-10-02-17:29:34] Epoch: [137][100/196] Loss 1.280 (1.556) Prec@1 50.00 (50.26) Prec@5 94.53 (91.48) VALID-SEARCH [2018-10-02-17:29:40] Epoch: [137][195/196] Loss 2.024 (1.533) Prec@1 30.00 (50.10) Prec@5 80.00 (91.73) 137/250, Train-Accuracy = 61.19, Test-Accuracy = 50.10 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-17:29:40] [Epoch=138/250] [Need: 03:42:18] [LR=0.0215 ~ 0.0215] [Batch=128], tau=6.688 genotype = Genotype(normal=[('sep_conv_5x5', 0, 0.12796254456043243), ('dil_conv_3x3', 1, 0.12665538489818573), ('sep_conv_3x3', 2, 0.1284228265285492), ('skip_connect', 1, 0.12776082754135132), ('skip_connect', 0, 0.1278102695941925), ('skip_connect', 1, 0.12756948173046112), ('max_pool_3x3', 0, 0.12882667779922485), ('max_pool_3x3', 1, 0.1278138905763626)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.12603625655174255), ('max_pool_3x3', 1, 0.12525400519371033), ('max_pool_3x3', 0, 0.12666049599647522), ('dil_conv_5x5', 2, 0.12593354284763336), ('max_pool_3x3', 0, 0.1266719400882721), ('max_pool_3x3', 1, 0.12606340646743774), ('dil_conv_5x5', 2, 0.1261148303747177), ('skip_connect', 0, 0.1258709728717804)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-17:29:42] Epoch: [138][000/196] Time 1.46 (1.46) Data 0.94 (0.94) Loss 0.955 (0.955) Prec@1 71.88 (71.88) Prec@5 96.09 (96.09) TRAIN-SEARCH [2018-10-02-17:30:40] Epoch: [138][100/196] Time 0.53 (0.58) Data 0.00 (0.01) Loss 1.133 (1.092) Prec@1 64.06 (61.07) Prec@5 96.88 (96.26) TRAIN-SEARCH [2018-10-02-17:31:27] Epoch: [138][195/196] Time 0.54 (0.54) Data 0.00 (0.01) Loss 1.021 (1.090) Prec@1 65.00 (61.30) Prec@5 95.00 (96.13) VALID-SEARCH [2018-10-02-17:31:28] Epoch: [138][000/196] Loss 1.557 (1.557) Prec@1 52.34 (52.34) Prec@5 84.38 (84.38) VALID-SEARCH [2018-10-02-17:31:33] Epoch: [138][100/196] Loss 1.390 (1.592) Prec@1 53.12 (46.23) Prec@5 92.19 (89.77) VALID-SEARCH [2018-10-02-17:31:38] Epoch: [138][195/196] Loss 2.046 (1.589) Prec@1 20.00 (46.20) Prec@5 75.00 (89.86) 138/250, Train-Accuracy = 61.30, Test-Accuracy = 46.20 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-17:31:39] [Epoch=139/250] [Need: 03:39:25] [LR=0.0212 ~ 0.0212] [Batch=128], tau=6.664 genotype = Genotype(normal=[('sep_conv_5x5', 0, 0.1279129981994629), ('dil_conv_3x3', 1, 0.12674838304519653), ('sep_conv_3x3', 2, 0.1281965970993042), ('skip_connect', 1, 0.12757247686386108), ('sep_conv_3x3', 3, 0.12757275998592377), ('skip_connect', 0, 0.12748783826828003), ('max_pool_3x3', 0, 0.12948711216449738), ('max_pool_3x3', 1, 0.12782704830169678)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.1259641945362091), ('avg_pool_3x3', 1, 0.12566666305065155), ('avg_pool_3x3', 0, 0.12686237692832947), ('max_pool_3x3', 1, 0.12626703083515167), ('max_pool_3x3', 0, 0.12662151455879211), ('dil_conv_3x3', 3, 0.12578727304935455), ('max_pool_3x3', 1, 0.12630397081375122), ('dil_conv_5x5', 2, 0.12613658607006073)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-17:31:41] Epoch: [139][000/196] Time 1.40 (1.40) Data 0.76 (0.76) Loss 1.115 (1.115) Prec@1 68.75 (68.75) Prec@5 97.66 (97.66) TRAIN-SEARCH [2018-10-02-17:32:37] Epoch: [139][100/196] Time 0.50 (0.57) Data 0.00 (0.01) Loss 1.150 (1.083) Prec@1 59.38 (62.19) Prec@5 94.53 (96.08) TRAIN-SEARCH [2018-10-02-17:33:26] Epoch: [139][195/196] Time 0.36 (0.54) Data 0.00 (0.01) Loss 1.062 (1.089) Prec@1 62.50 (61.75) Prec@5 97.50 (96.08) VALID-SEARCH [2018-10-02-17:33:27] Epoch: [139][000/196] Loss 1.176 (1.176) Prec@1 59.38 (59.38) Prec@5 96.09 (96.09) VALID-SEARCH [2018-10-02-17:33:33] Epoch: [139][100/196] Loss 1.414 (1.498) Prec@1 45.31 (49.70) Prec@5 92.19 (90.76) VALID-SEARCH [2018-10-02-17:33:38] Epoch: [139][195/196] Loss 1.370 (1.524) Prec@1 52.50 (48.70) Prec@5 90.00 (90.80) 139/250, Train-Accuracy = 61.75, Test-Accuracy = 48.70 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-17:33:39] [Epoch=140/250] [Need: 03:39:56] [LR=0.0209 ~ 0.0209] [Batch=128], tau=6.64 genotype = Genotype(normal=[('skip_connect', 0, 0.1280001699924469), ('dil_conv_3x3', 1, 0.12675608694553375), ('sep_conv_3x3', 2, 0.12875574827194214), ('skip_connect', 0, 0.12817253172397614), ('skip_connect', 0, 0.12844978272914886), ('sep_conv_3x3', 3, 0.1275440901517868), ('max_pool_3x3', 0, 0.12957054376602173), ('dil_conv_3x3', 4, 0.1274842619895935)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 1, 0.12587276101112366), ('dil_conv_5x5', 0, 0.12576651573181152), ('max_pool_3x3', 0, 0.1268615871667862), ('max_pool_3x3', 1, 0.1264820396900177), ('max_pool_3x3', 0, 0.12639717757701874), ('max_pool_3x3', 1, 0.12625104188919067), ('avg_pool_3x3', 0, 0.12636049091815948), ('sep_conv_3x3', 3, 0.12605538964271545)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-17:33:41] Epoch: [140][000/196] Time 1.44 (1.44) Data 0.84 (0.84) Loss 1.079 (1.079) Prec@1 64.06 (64.06) Prec@5 96.88 (96.88) TRAIN-SEARCH [2018-10-02-17:34:37] Epoch: [140][100/196] Time 0.70 (0.56) Data 0.00 (0.01) Loss 1.038 (1.074) Prec@1 62.50 (62.48) Prec@5 93.75 (96.33) TRAIN-SEARCH [2018-10-02-17:35:27] Epoch: [140][195/196] Time 0.42 (0.55) Data 0.00 (0.01) Loss 1.156 (1.077) Prec@1 62.50 (61.97) Prec@5 92.50 (96.18) VALID-SEARCH [2018-10-02-17:35:29] Epoch: [140][000/196] Loss 1.479 (1.479) Prec@1 48.44 (48.44) Prec@5 95.31 (95.31) VALID-SEARCH [2018-10-02-17:35:34] Epoch: [140][100/196] Loss 1.595 (1.545) Prec@1 43.75 (47.73) Prec@5 91.41 (89.84) VALID-SEARCH [2018-10-02-17:35:40] Epoch: [140][195/196] Loss 1.535 (1.545) Prec@1 37.50 (46.92) Prec@5 90.00 (89.96) 140/250, Train-Accuracy = 61.97, Test-Accuracy = 46.92 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-17:35:40] [Epoch=141/250] [Need: 03:40:48] [LR=0.0206 ~ 0.0206] [Batch=128], tau=6.6160000000000005 genotype = Genotype(normal=[('skip_connect', 0, 0.12784600257873535), ('dil_conv_3x3', 1, 0.12744088470935822), ('skip_connect', 0, 0.128601536154747), ('sep_conv_3x3', 2, 0.12827789783477783), ('skip_connect', 0, 0.12805584073066711), ('sep_conv_3x3', 3, 0.12746980786323547), ('max_pool_3x3', 0, 0.12895134091377258), ('dil_conv_3x3', 4, 0.1276548057794571)], normal_concat=range(2, 6), reduce=[('dil_conv_5x5', 0, 0.12625452876091003), ('dil_conv_5x5', 1, 0.1256961077451706), ('max_pool_3x3', 1, 0.12681706249713898), ('max_pool_3x3', 0, 0.12665563821792603), ('max_pool_3x3', 0, 0.1261376142501831), ('max_pool_3x3', 1, 0.12608887255191803), ('dil_conv_5x5', 2, 0.12624458968639374), ('max_pool_3x3', 1, 0.1261151134967804)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-17:35:43] Epoch: [141][000/196] Time 1.61 (1.61) Data 0.97 (0.97) Loss 1.275 (1.275) Prec@1 56.25 (56.25) Prec@5 96.09 (96.09) TRAIN-SEARCH [2018-10-02-17:36:37] Epoch: [141][100/196] Time 0.45 (0.55) Data 0.00 (0.01) Loss 1.139 (1.061) Prec@1 60.94 (62.56) Prec@5 94.53 (96.38) TRAIN-SEARCH [2018-10-02-17:37:28] Epoch: [141][195/196] Time 0.50 (0.55) Data 0.00 (0.01) Loss 1.259 (1.072) Prec@1 55.00 (61.96) Prec@5 92.50 (96.18) VALID-SEARCH [2018-10-02-17:37:29] Epoch: [141][000/196] Loss 1.499 (1.499) Prec@1 44.53 (44.53) Prec@5 90.62 (90.62) VALID-SEARCH [2018-10-02-17:37:35] Epoch: [141][100/196] Loss 1.384 (1.510) Prec@1 45.31 (48.13) Prec@5 92.19 (90.90) VALID-SEARCH [2018-10-02-17:37:40] Epoch: [141][195/196] Loss 1.217 (1.513) Prec@1 57.50 (48.72) Prec@5 92.50 (91.04) 141/250, Train-Accuracy = 61.96, Test-Accuracy = 48.72 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-17:37:41] [Epoch=142/250] [Need: 03:36:33] [LR=0.0203 ~ 0.0203] [Batch=128], tau=6.5920000000000005 genotype = Genotype(normal=[('dil_conv_3x3', 1, 0.12765048444271088), ('skip_connect', 0, 0.12733915448188782), ('skip_connect', 0, 0.12850184738636017), ('sep_conv_3x3', 2, 0.1282847821712494), ('skip_connect', 0, 0.12767016887664795), ('sep_conv_3x3', 3, 0.127442866563797), ('max_pool_3x3', 0, 0.1278846263885498), ('dil_conv_3x3', 4, 0.1275481879711151)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.12607459723949432), ('dil_conv_5x5', 1, 0.1259174346923828), ('max_pool_3x3', 0, 0.12646764516830444), ('max_pool_3x3', 1, 0.1262192279100418), ('max_pool_3x3', 1, 0.1267886459827423), ('sep_conv_5x5', 2, 0.12640942633152008), ('dil_conv_5x5', 2, 0.126586452126503), ('dil_conv_3x3', 3, 0.1261570304632187)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-17:37:43] Epoch: [142][000/196] Time 1.45 (1.45) Data 0.85 (0.85) Loss 0.901 (0.901) Prec@1 67.97 (67.97) Prec@5 97.66 (97.66) TRAIN-SEARCH [2018-10-02-17:38:38] Epoch: [142][100/196] Time 0.68 (0.56) Data 0.00 (0.01) Loss 0.904 (1.078) Prec@1 70.31 (61.80) Prec@5 97.66 (96.28) TRAIN-SEARCH [2018-10-02-17:39:28] Epoch: [142][195/196] Time 0.47 (0.55) Data 0.00 (0.01) Loss 1.168 (1.080) Prec@1 50.00 (61.73) Prec@5 95.00 (96.25) VALID-SEARCH [2018-10-02-17:39:29] Epoch: [142][000/196] Loss 1.221 (1.221) Prec@1 57.03 (57.03) Prec@5 95.31 (95.31) VALID-SEARCH [2018-10-02-17:39:34] Epoch: [142][100/196] Loss 1.397 (1.493) Prec@1 49.22 (49.52) Prec@5 95.31 (91.58) VALID-SEARCH [2018-10-02-17:39:38] Epoch: [142][195/196] Loss 1.735 (1.492) Prec@1 37.50 (50.26) Prec@5 87.50 (91.59) 142/250, Train-Accuracy = 61.73, Test-Accuracy = 50.26 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-17:39:40] [Epoch=143/250] [Need: 03:32:59] [LR=0.0200 ~ 0.0200] [Batch=128], tau=6.5680000000000005 genotype = Genotype(normal=[('skip_connect', 1, 0.1270761489868164), ('skip_connect', 0, 0.1270366758108139), ('skip_connect', 0, 0.12840448319911957), ('sep_conv_3x3', 2, 0.1283935010433197), ('skip_connect', 0, 0.12794607877731323), ('dil_conv_3x3', 3, 0.12724024057388306), ('skip_connect', 0, 0.12763071060180664), ('dil_conv_3x3', 4, 0.1274973750114441)], normal_concat=range(2, 6), reduce=[('dil_conv_5x5', 0, 0.12618307769298553), ('dil_conv_5x5', 1, 0.12615655362606049), ('max_pool_3x3', 0, 0.12629038095474243), ('max_pool_3x3', 1, 0.1260456144809723), ('max_pool_3x3', 0, 0.12661147117614746), ('max_pool_3x3', 1, 0.1264410763978958), ('max_pool_3x3', 0, 0.1261167973279953), ('avg_pool_3x3', 1, 0.12597617506980896)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-17:39:42] Epoch: [143][000/196] Time 1.42 (1.42) Data 0.89 (0.89) Loss 1.297 (1.297) Prec@1 54.69 (54.69) Prec@5 94.53 (94.53) TRAIN-SEARCH [2018-10-02-17:40:39] Epoch: [143][100/196] Time 0.62 (0.58) Data 0.01 (0.01) Loss 1.108 (1.083) Prec@1 62.50 (61.97) Prec@5 95.31 (96.12) TRAIN-SEARCH [2018-10-02-17:41:26] Epoch: [143][195/196] Time 0.44 (0.54) Data 0.00 (0.01) Loss 0.972 (1.072) Prec@1 62.50 (62.53) Prec@5 95.00 (96.16) VALID-SEARCH [2018-10-02-17:41:27] Epoch: [143][000/196] Loss 1.260 (1.260) Prec@1 60.16 (60.16) Prec@5 93.75 (93.75) VALID-SEARCH [2018-10-02-17:41:32] Epoch: [143][100/196] Loss 1.151 (1.583) Prec@1 57.03 (48.17) Prec@5 94.53 (90.69) VALID-SEARCH [2018-10-02-17:41:37] Epoch: [143][195/196] Loss 0.893 (1.587) Prec@1 65.00 (48.06) Prec@5 97.50 (90.51) 143/250, Train-Accuracy = 62.53, Test-Accuracy = 48.06 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-17:41:39] [Epoch=144/250] [Need: 03:29:20] [LR=0.0197 ~ 0.0197] [Batch=128], tau=6.5440000000000005 genotype = Genotype(normal=[('skip_connect', 0, 0.12730762362480164), ('skip_connect', 1, 0.12720973789691925), ('sep_conv_3x3', 2, 0.1286270022392273), ('skip_connect', 0, 0.12834487855434418), ('skip_connect', 0, 0.12918491661548615), ('sep_conv_3x3', 2, 0.1271805316209793), ('skip_connect', 0, 0.1274656504392624), ('sep_conv_3x3', 4, 0.12745794653892517)], normal_concat=range(2, 6), reduce=[('dil_conv_5x5', 0, 0.12584209442138672), ('dil_conv_5x5', 1, 0.12569470703601837), ('max_pool_3x3', 0, 0.12696678936481476), ('sep_conv_5x5', 2, 0.12611013650894165), ('skip_connect', 0, 0.12693895399570465), ('max_pool_3x3', 1, 0.1267070770263672), ('max_pool_3x3', 0, 0.1266244649887085), ('dil_conv_3x3', 1, 0.1262189894914627)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-17:41:41] Epoch: [144][000/196] Time 1.80 (1.80) Data 1.11 (1.11) Loss 1.073 (1.073) Prec@1 64.84 (64.84) Prec@5 98.44 (98.44) TRAIN-SEARCH [2018-10-02-17:42:41] Epoch: [144][100/196] Time 0.50 (0.61) Data 0.00 (0.02) Loss 0.815 (1.099) Prec@1 74.22 (61.83) Prec@5 98.44 (96.13) TRAIN-SEARCH [2018-10-02-17:43:28] Epoch: [144][195/196] Time 0.49 (0.56) Data 0.00 (0.01) Loss 0.946 (1.070) Prec@1 67.50 (62.63) Prec@5 92.50 (96.28) VALID-SEARCH [2018-10-02-17:43:30] Epoch: [144][000/196] Loss 1.518 (1.518) Prec@1 51.56 (51.56) Prec@5 95.31 (95.31) VALID-SEARCH [2018-10-02-17:43:35] Epoch: [144][100/196] Loss 1.629 (1.551) Prec@1 50.00 (48.08) Prec@5 94.53 (90.52) VALID-SEARCH [2018-10-02-17:43:41] Epoch: [144][195/196] Loss 1.254 (1.533) Prec@1 52.50 (49.28) Prec@5 97.50 (91.12) 144/250, Train-Accuracy = 62.63, Test-Accuracy = 49.28 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-17:43:41] [Epoch=145/250] [Need: 03:34:42] [LR=0.0194 ~ 0.0194] [Batch=128], tau=6.5200000000000005 genotype = Genotype(normal=[('dil_conv_3x3', 0, 0.1278693825006485), ('skip_connect', 1, 0.12769140303134918), ('skip_connect', 0, 0.12942981719970703), ('sep_conv_3x3', 2, 0.12802521884441376), ('skip_connect', 0, 0.13060030341148376), ('sep_conv_3x3', 2, 0.12696246802806854), ('skip_connect', 0, 0.1282937079668045), ('sep_conv_3x3', 4, 0.12724769115447998)], normal_concat=range(2, 6), reduce=[('dil_conv_5x5', 1, 0.12607541680335999), ('dil_conv_5x5', 0, 0.12590646743774414), ('avg_pool_3x3', 0, 0.126362144947052), ('max_pool_3x3', 1, 0.12606851756572723), ('max_pool_3x3', 1, 0.1261812150478363), ('skip_connect', 0, 0.1261032521724701), ('sep_conv_3x3', 4, 0.12606598436832428), ('max_pool_3x3', 0, 0.12602293491363525)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-17:43:44] Epoch: [145][000/196] Time 1.54 (1.54) Data 0.89 (0.89) Loss 1.044 (1.044) Prec@1 55.47 (55.47) Prec@5 98.44 (98.44) TRAIN-SEARCH [2018-10-02-17:44:40] Epoch: [145][100/196] Time 0.64 (0.57) Data 0.00 (0.01) Loss 0.982 (1.058) Prec@1 65.62 (62.90) Prec@5 96.88 (96.40) TRAIN-SEARCH [2018-10-02-17:45:29] Epoch: [145][195/196] Time 0.45 (0.55) Data 0.00 (0.01) Loss 1.057 (1.056) Prec@1 57.50 (62.97) Prec@5 95.00 (96.33) VALID-SEARCH [2018-10-02-17:45:31] Epoch: [145][000/196] Loss 1.445 (1.445) Prec@1 53.12 (53.12) Prec@5 89.84 (89.84) VALID-SEARCH [2018-10-02-17:45:37] Epoch: [145][100/196] Loss 1.303 (1.475) Prec@1 51.56 (50.13) Prec@5 92.97 (91.96) VALID-SEARCH [2018-10-02-17:45:42] Epoch: [145][195/196] Loss 2.060 (1.508) Prec@1 37.50 (50.49) Prec@5 85.00 (91.82) 145/250, Train-Accuracy = 62.97, Test-Accuracy = 50.49 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-17:45:43] [Epoch=146/250] [Need: 03:31:01] [LR=0.0191 ~ 0.0191] [Batch=128], tau=6.496 genotype = Genotype(normal=[('skip_connect', 0, 0.12820686399936676), ('skip_connect', 1, 0.12735050916671753), ('skip_connect', 0, 0.12964533269405365), ('sep_conv_3x3', 2, 0.12820754945278168), ('skip_connect', 0, 0.13018865883350372), ('sep_conv_3x3', 2, 0.12747490406036377), ('skip_connect', 0, 0.1282072514295578), ('sep_conv_3x3', 4, 0.12755471467971802)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.1269310712814331), ('sep_conv_5x5', 1, 0.12557080388069153), ('max_pool_3x3', 0, 0.12649041414260864), ('max_pool_3x3', 1, 0.12583990395069122), ('skip_connect', 0, 0.12703421711921692), ('max_pool_3x3', 1, 0.12590289115905762), ('dil_conv_5x5', 2, 0.12613502144813538), ('skip_connect', 0, 0.12591995298862457)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-17:45:45] Epoch: [146][000/196] Time 1.42 (1.42) Data 0.87 (0.87) Loss 1.128 (1.128) Prec@1 62.50 (62.50) Prec@5 94.53 (94.53) TRAIN-SEARCH [2018-10-02-17:46:39] Epoch: [146][100/196] Time 0.50 (0.55) Data 0.00 (0.01) Loss 1.157 (1.075) Prec@1 58.59 (61.94) Prec@5 98.44 (96.18) TRAIN-SEARCH [2018-10-02-17:47:29] Epoch: [146][195/196] Time 0.43 (0.53) Data 0.00 (0.01) Loss 0.852 (1.063) Prec@1 67.50 (62.48) Prec@5 95.00 (96.33) VALID-SEARCH [2018-10-02-17:47:30] Epoch: [146][000/196] Loss 1.938 (1.938) Prec@1 42.19 (42.19) Prec@5 85.94 (85.94) VALID-SEARCH [2018-10-02-17:47:36] Epoch: [146][100/196] Loss 1.187 (1.500) Prec@1 57.03 (49.06) Prec@5 94.53 (91.36) VALID-SEARCH [2018-10-02-17:47:41] Epoch: [146][195/196] Loss 1.222 (1.508) Prec@1 62.50 (48.42) Prec@5 95.00 (90.78) 146/250, Train-Accuracy = 62.48, Test-Accuracy = 48.42 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-17:47:42] [Epoch=147/250] [Need: 03:23:30] [LR=0.0188 ~ 0.0188] [Batch=128], tau=6.472 genotype = Genotype(normal=[('skip_connect', 0, 0.12813521921634674), ('skip_connect', 1, 0.127492755651474), ('skip_connect', 0, 0.1295943707227707), ('sep_conv_3x3', 2, 0.12877114117145538), ('skip_connect', 0, 0.1297801285982132), ('sep_conv_3x3', 2, 0.12762466073036194), ('sep_conv_3x3', 4, 0.12764210999011993), ('skip_connect', 1, 0.12747621536254883)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.12608718872070312), ('dil_conv_5x5', 1, 0.1253601461648941), ('avg_pool_3x3', 0, 0.12642139196395874), ('sep_conv_5x5', 2, 0.12634822726249695), ('max_pool_3x3', 0, 0.12659573554992676), ('avg_pool_3x3', 1, 0.12584000825881958), ('dil_conv_5x5', 2, 0.12672396004199982), ('avg_pool_3x3', 0, 0.1262844204902649)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-17:47:44] Epoch: [147][000/196] Time 1.49 (1.49) Data 0.84 (0.84) Loss 1.149 (1.149) Prec@1 57.81 (57.81) Prec@5 96.09 (96.09) TRAIN-SEARCH [2018-10-02-17:48:41] Epoch: [147][100/196] Time 0.57 (0.58) Data 0.00 (0.01) Loss 0.882 (1.053) Prec@1 67.19 (62.96) Prec@5 96.88 (96.41) TRAIN-SEARCH [2018-10-02-17:49:30] Epoch: [147][195/196] Time 0.40 (0.55) Data 0.00 (0.01) Loss 1.093 (1.051) Prec@1 62.50 (63.04) Prec@5 97.50 (96.52) VALID-SEARCH [2018-10-02-17:49:31] Epoch: [147][000/196] Loss 1.245 (1.245) Prec@1 51.56 (51.56) Prec@5 97.66 (97.66) VALID-SEARCH [2018-10-02-17:49:37] Epoch: [147][100/196] Loss 1.934 (1.517) Prec@1 35.16 (49.58) Prec@5 76.56 (91.26) VALID-SEARCH [2018-10-02-17:49:42] Epoch: [147][195/196] Loss 1.283 (1.594) Prec@1 57.50 (48.34) Prec@5 97.50 (90.36) 147/250, Train-Accuracy = 63.04, Test-Accuracy = 48.34 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-17:49:43] [Epoch=148/250] [Need: 03:26:11] [LR=0.0185 ~ 0.0185] [Batch=128], tau=6.448 genotype = Genotype(normal=[('skip_connect', 0, 0.12872780859470367), ('skip_connect', 1, 0.12725277245044708), ('skip_connect', 0, 0.12893837690353394), ('sep_conv_3x3', 2, 0.12851998209953308), ('skip_connect', 0, 0.12896201014518738), ('sep_conv_3x3', 2, 0.12744127213954926), ('sep_conv_3x3', 4, 0.12779350578784943), ('max_pool_3x3', 0, 0.12762385606765747)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.12574902176856995), ('max_pool_3x3', 1, 0.12548349797725677), ('max_pool_3x3', 0, 0.12691368162631989), ('sep_conv_5x5', 2, 0.1261855661869049), ('avg_pool_3x3', 0, 0.1265074610710144), ('max_pool_3x3', 1, 0.1259372979402542), ('dil_conv_5x5', 2, 0.1269460767507553), ('sep_conv_5x5', 4, 0.1258792132139206)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-17:49:45] Epoch: [148][000/196] Time 1.47 (1.47) Data 0.88 (0.88) Loss 1.066 (1.066) Prec@1 58.59 (58.59) Prec@5 95.31 (95.31) TRAIN-SEARCH [2018-10-02-17:50:41] Epoch: [148][100/196] Time 0.58 (0.56) Data 0.00 (0.01) Loss 1.054 (1.043) Prec@1 61.72 (63.10) Prec@5 96.88 (96.59) TRAIN-SEARCH [2018-10-02-17:51:31] Epoch: [148][195/196] Time 0.40 (0.55) Data 0.00 (0.01) Loss 1.033 (1.047) Prec@1 62.50 (63.15) Prec@5 100.00 (96.51) VALID-SEARCH [2018-10-02-17:51:32] Epoch: [148][000/196] Loss 1.614 (1.614) Prec@1 55.47 (55.47) Prec@5 92.97 (92.97) VALID-SEARCH [2018-10-02-17:51:37] Epoch: [148][100/196] Loss 3.845 (1.486) Prec@1 38.28 (50.46) Prec@5 89.06 (91.95) VALID-SEARCH [2018-10-02-17:51:42] Epoch: [148][195/196] Loss 1.168 (1.496) Prec@1 52.50 (50.03) Prec@5 92.50 (91.70) 148/250, Train-Accuracy = 63.15, Test-Accuracy = 50.03 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-17:51:43] [Epoch=149/250] [Need: 03:21:50] [LR=0.0182 ~ 0.0182] [Batch=128], tau=6.424 genotype = Genotype(normal=[('dil_conv_3x3', 0, 0.12827230989933014), ('skip_connect', 1, 0.12764261662960052), ('skip_connect', 0, 0.12942218780517578), ('sep_conv_3x3', 2, 0.1282133311033249), ('skip_connect', 0, 0.12897974252700806), ('sep_conv_3x3', 2, 0.1281069666147232), ('sep_conv_3x3', 4, 0.12836889922618866), ('skip_connect', 0, 0.12760598957538605)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 1, 0.12584108114242554), ('max_pool_3x3', 0, 0.1256590634584427), ('avg_pool_3x3', 0, 0.12695766985416412), ('max_pool_3x3', 1, 0.12630558013916016), ('avg_pool_3x3', 0, 0.12637941539287567), ('max_pool_3x3', 1, 0.12572376430034637), ('avg_pool_3x3', 0, 0.1264818012714386), ('dil_conv_5x5', 2, 0.12627975642681122)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-17:51:45] Epoch: [149][000/196] Time 1.27 (1.27) Data 0.67 (0.67) Loss 1.230 (1.230) Prec@1 59.38 (59.38) Prec@5 94.53 (94.53) TRAIN-SEARCH [2018-10-02-17:52:41] Epoch: [149][100/196] Time 0.40 (0.57) Data 0.00 (0.01) Loss 0.987 (1.049) Prec@1 68.75 (63.16) Prec@5 94.53 (96.42) TRAIN-SEARCH [2018-10-02-17:53:31] Epoch: [149][195/196] Time 0.41 (0.55) Data 0.00 (0.01) Loss 1.044 (1.050) Prec@1 70.00 (62.98) Prec@5 92.50 (96.46) VALID-SEARCH [2018-10-02-17:53:32] Epoch: [149][000/196] Loss 2.111 (2.111) Prec@1 30.47 (30.47) Prec@5 74.22 (74.22) VALID-SEARCH [2018-10-02-17:53:37] Epoch: [149][100/196] Loss 1.694 (1.531) Prec@1 42.19 (49.48) Prec@5 83.59 (91.64) VALID-SEARCH [2018-10-02-17:53:42] Epoch: [149][195/196] Loss 2.029 (1.542) Prec@1 40.00 (48.96) Prec@5 85.00 (91.54) 149/250, Train-Accuracy = 62.98, Test-Accuracy = 48.96 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-17:53:43] [Epoch=150/250] [Need: 03:19:39] [LR=0.0179 ~ 0.0179] [Batch=128], tau=6.4 genotype = Genotype(normal=[('dil_conv_3x3', 0, 0.12821008265018463), ('skip_connect', 1, 0.12685149908065796), ('skip_connect', 0, 0.12923891842365265), ('sep_conv_3x3', 2, 0.12899336218833923), ('skip_connect', 0, 0.12941192090511322), ('sep_conv_3x3', 2, 0.12793579697608948), ('sep_conv_3x3', 4, 0.12818105518817902), ('skip_connect', 0, 0.12754575908184052)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.12606100738048553), ('sep_conv_3x3', 1, 0.12542769312858582), ('max_pool_3x3', 1, 0.12641552090644836), ('avg_pool_3x3', 0, 0.12638655304908752), ('avg_pool_3x3', 0, 0.1271136999130249), ('avg_pool_3x3', 1, 0.1265874207019806), ('dil_conv_5x5', 4, 0.12630046904087067), ('avg_pool_3x3', 1, 0.12586216628551483)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-17:53:45] Epoch: [150][000/196] Time 1.61 (1.61) Data 1.01 (1.01) Loss 1.193 (1.193) Prec@1 57.03 (57.03) Prec@5 95.31 (95.31) TRAIN-SEARCH [2018-10-02-17:54:41] Epoch: [150][100/196] Time 0.42 (0.57) Data 0.00 (0.01) Loss 1.059 (1.031) Prec@1 63.28 (63.00) Prec@5 95.31 (96.40) TRAIN-SEARCH [2018-10-02-17:55:31] Epoch: [150][195/196] Time 0.42 (0.55) Data 0.00 (0.01) Loss 1.054 (1.047) Prec@1 65.00 (62.92) Prec@5 97.50 (96.35) VALID-SEARCH [2018-10-02-17:55:32] Epoch: [150][000/196] Loss 2.163 (2.163) Prec@1 32.03 (32.03) Prec@5 70.31 (70.31) VALID-SEARCH [2018-10-02-17:55:37] Epoch: [150][100/196] Loss 1.449 (1.493) Prec@1 52.34 (50.93) Prec@5 93.75 (91.24) VALID-SEARCH [2018-10-02-17:55:43] Epoch: [150][195/196] Loss 1.166 (1.513) Prec@1 57.50 (50.61) Prec@5 95.00 (91.40) 150/250, Train-Accuracy = 62.92, Test-Accuracy = 50.61 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-17:55:44] [Epoch=151/250] [Need: 03:20:03] [LR=0.0176 ~ 0.0176] [Batch=128], tau=6.376 genotype = Genotype(normal=[('dil_conv_5x5', 0, 0.12854209542274475), ('dil_conv_3x3', 1, 0.12743006646633148), ('sep_conv_3x3', 2, 0.12886188924312592), ('skip_connect', 0, 0.12885086238384247), ('skip_connect', 0, 0.12897929549217224), ('sep_conv_3x3', 3, 0.1275804340839386), ('sep_conv_3x3', 4, 0.12767963111400604), ('skip_connect', 0, 0.12756334245204926)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.1261158287525177), ('sep_conv_5x5', 1, 0.12535804510116577), ('max_pool_3x3', 1, 0.12677669525146484), ('max_pool_3x3', 0, 0.12641023099422455), ('max_pool_3x3', 0, 0.12662005424499512), ('avg_pool_3x3', 1, 0.12661167979240417), ('sep_conv_5x5', 4, 0.12613265216350555), ('max_pool_3x3', 0, 0.12603257596492767)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-17:55:46] Epoch: [151][000/196] Time 1.93 (1.93) Data 1.30 (1.30) Loss 1.177 (1.177) Prec@1 63.28 (63.28) Prec@5 96.88 (96.88) TRAIN-SEARCH [2018-10-02-17:56:41] Epoch: [151][100/196] Time 0.43 (0.56) Data 0.00 (0.02) Loss 1.147 (1.040) Prec@1 62.50 (63.85) Prec@5 95.31 (96.44) TRAIN-SEARCH [2018-10-02-17:57:31] Epoch: [151][195/196] Time 0.39 (0.55) Data 0.00 (0.01) Loss 1.046 (1.039) Prec@1 60.00 (63.72) Prec@5 100.00 (96.45) VALID-SEARCH [2018-10-02-17:57:33] Epoch: [151][000/196] Loss 1.107 (1.107) Prec@1 65.62 (65.62) Prec@5 94.53 (94.53) VALID-SEARCH [2018-10-02-17:57:38] Epoch: [151][100/196] Loss 1.911 (1.431) Prec@1 46.88 (52.42) Prec@5 94.53 (92.58) VALID-SEARCH [2018-10-02-17:57:44] Epoch: [151][195/196] Loss 1.664 (1.443) Prec@1 45.00 (51.47) Prec@5 92.50 (92.11) 151/250, Train-Accuracy = 63.72, Test-Accuracy = 51.47 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-17:57:45] [Epoch=152/250] [Need: 03:17:11] [LR=0.0173 ~ 0.0173] [Batch=128], tau=6.352 genotype = Genotype(normal=[('dil_conv_5x5', 0, 0.128171905875206), ('skip_connect', 1, 0.12771032750606537), ('sep_conv_3x3', 2, 0.12921027839183807), ('skip_connect', 0, 0.12904882431030273), ('skip_connect', 0, 0.12862831354141235), ('sep_conv_5x5', 3, 0.12738481163978577), ('skip_connect', 1, 0.12821155786514282), ('sep_conv_3x3', 3, 0.12741883099079132)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.1260284036397934), ('skip_connect', 1, 0.1253213882446289), ('max_pool_3x3', 1, 0.1261206567287445), ('max_pool_3x3', 0, 0.1260826289653778), ('max_pool_3x3', 0, 0.12643994390964508), ('dil_conv_5x5', 3, 0.12581749260425568), ('sep_conv_5x5', 4, 0.12608219683170319), ('max_pool_3x3', 0, 0.12585605680942535)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-17:57:47] Epoch: [152][000/196] Time 1.41 (1.41) Data 0.82 (0.82) Loss 0.841 (0.841) Prec@1 67.97 (67.97) Prec@5 98.44 (98.44) TRAIN-SEARCH [2018-10-02-17:58:39] Epoch: [152][100/196] Time 0.39 (0.53) Data 0.00 (0.01) Loss 1.129 (1.037) Prec@1 60.94 (63.58) Prec@5 94.53 (96.43) TRAIN-SEARCH [2018-10-02-17:59:32] Epoch: [152][195/196] Time 0.51 (0.54) Data 0.00 (0.01) Loss 0.976 (1.039) Prec@1 62.50 (63.41) Prec@5 100.00 (96.38) VALID-SEARCH [2018-10-02-17:59:33] Epoch: [152][000/196] Loss 1.443 (1.443) Prec@1 56.25 (56.25) Prec@5 95.31 (95.31) VALID-SEARCH [2018-10-02-17:59:39] Epoch: [152][100/196] Loss 2.038 (1.508) Prec@1 32.03 (51.14) Prec@5 86.72 (92.05) VALID-SEARCH [2018-10-02-17:59:44] Epoch: [152][195/196] Loss 1.066 (1.493) Prec@1 67.50 (50.39) Prec@5 97.50 (91.74) 152/250, Train-Accuracy = 63.41, Test-Accuracy = 50.39 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-17:59:45] [Epoch=153/250] [Need: 03:15:12] [LR=0.0171 ~ 0.0171] [Batch=128], tau=6.328 genotype = Genotype(normal=[('dil_conv_3x3', 0, 0.12835322320461273), ('skip_connect', 1, 0.12805432081222534), ('skip_connect', 0, 0.1291314661502838), ('sep_conv_3x3', 2, 0.1290988326072693), ('skip_connect', 0, 0.12818583846092224), ('skip_connect', 1, 0.1273137778043747), ('max_pool_3x3', 1, 0.1273154318332672), ('sep_conv_5x5', 4, 0.12718094885349274)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.1260094940662384), ('sep_conv_5x5', 1, 0.12564751505851746), ('sep_conv_5x5', 2, 0.12683048844337463), ('max_pool_3x3', 1, 0.12625238299369812), ('sep_conv_5x5', 2, 0.1263635754585266), ('avg_pool_3x3', 0, 0.12618470191955566), ('avg_pool_3x3', 0, 0.1264405995607376), ('sep_conv_5x5', 2, 0.1257757842540741)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-17:59:48] Epoch: [153][000/196] Time 1.62 (1.62) Data 0.96 (0.96) Loss 0.912 (0.912) Prec@1 71.09 (71.09) Prec@5 97.66 (97.66) TRAIN-SEARCH [2018-10-02-18:00:40] Epoch: [153][100/196] Time 0.58 (0.53) Data 0.00 (0.01) Loss 0.937 (1.021) Prec@1 70.31 (64.34) Prec@5 99.22 (96.58) TRAIN-SEARCH [2018-10-02-18:01:34] Epoch: [153][195/196] Time 0.60 (0.55) Data 0.00 (0.01) Loss 1.513 (1.031) Prec@1 57.50 (63.94) Prec@5 92.50 (96.58) VALID-SEARCH [2018-10-02-18:01:35] Epoch: [153][000/196] Loss 1.817 (1.817) Prec@1 53.91 (53.91) Prec@5 96.09 (96.09) VALID-SEARCH [2018-10-02-18:01:40] Epoch: [153][100/196] Loss 1.693 (1.432) Prec@1 42.97 (51.79) Prec@5 90.62 (92.64) VALID-SEARCH [2018-10-02-18:01:46] Epoch: [153][195/196] Loss 1.509 (1.448) Prec@1 50.00 (51.59) Prec@5 92.50 (92.34) 153/250, Train-Accuracy = 63.94, Test-Accuracy = 51.59 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-18:01:47] [Epoch=154/250] [Need: 03:14:06] [LR=0.0168 ~ 0.0168] [Batch=128], tau=6.304 genotype = Genotype(normal=[('dil_conv_3x3', 0, 0.12866339087486267), ('skip_connect', 1, 0.12766686081886292), ('skip_connect', 0, 0.12869471311569214), ('sep_conv_3x3', 2, 0.12865103781223297), ('sep_conv_3x3', 3, 0.12776349484920502), ('skip_connect', 0, 0.12746022641658783), ('max_pool_3x3', 1, 0.12785016000270844), ('max_pool_3x3', 0, 0.12742334604263306)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.12612450122833252), ('sep_conv_5x5', 1, 0.12601879239082336), ('sep_conv_5x5', 2, 0.12655286490917206), ('avg_pool_3x3', 0, 0.12587878108024597), ('max_pool_3x3', 1, 0.12647868692874908), ('skip_connect', 0, 0.12586240470409393), ('sep_conv_5x5', 4, 0.12612825632095337), ('sep_conv_5x5', 1, 0.125984326004982)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-18:01:49] Epoch: [154][000/196] Time 1.51 (1.51) Data 0.80 (0.80) Loss 1.208 (1.208) Prec@1 57.81 (57.81) Prec@5 97.66 (97.66) TRAIN-SEARCH [2018-10-02-18:02:40] Epoch: [154][100/196] Time 0.56 (0.52) Data 0.01 (0.01) Loss 1.052 (1.009) Prec@1 64.06 (64.48) Prec@5 98.44 (96.62) TRAIN-SEARCH [2018-10-02-18:03:36] Epoch: [154][195/196] Time 0.63 (0.55) Data 0.00 (0.01) Loss 1.107 (1.021) Prec@1 60.00 (63.97) Prec@5 95.00 (96.64) VALID-SEARCH [2018-10-02-18:03:37] Epoch: [154][000/196] Loss 1.675 (1.675) Prec@1 35.94 (35.94) Prec@5 86.72 (86.72) VALID-SEARCH [2018-10-02-18:03:43] Epoch: [154][100/196] Loss 5.097 (1.571) Prec@1 26.56 (49.42) Prec@5 75.00 (90.92) VALID-SEARCH [2018-10-02-18:03:48] Epoch: [154][195/196] Loss 1.757 (1.522) Prec@1 50.00 (50.06) Prec@5 95.00 (91.84) 154/250, Train-Accuracy = 63.97, Test-Accuracy = 50.06 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-18:03:49] [Epoch=155/250] [Need: 03:14:16] [LR=0.0165 ~ 0.0165] [Batch=128], tau=6.28 genotype = Genotype(normal=[('dil_conv_3x3', 0, 0.12901201844215393), ('skip_connect', 1, 0.12745852768421173), ('sep_conv_5x5', 2, 0.1284341812133789), ('skip_connect', 0, 0.12832550704479218), ('sep_conv_3x3', 3, 0.12802767753601074), ('skip_connect', 0, 0.12754705548286438), ('max_pool_3x3', 1, 0.12794427573680878), ('max_pool_3x3', 0, 0.12767182290554047)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.1261356621980667), ('sep_conv_5x5', 1, 0.1254168599843979), ('sep_conv_5x5', 2, 0.12603497505187988), ('max_pool_3x3', 0, 0.12602609395980835), ('sep_conv_5x5', 1, 0.12627984583377838), ('skip_connect', 0, 0.12607263028621674), ('max_pool_3x3', 0, 0.126021608710289), ('sep_conv_3x3', 4, 0.1260063201189041)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-18:03:52] Epoch: [155][000/196] Time 1.70 (1.70) Data 0.98 (0.98) Loss 1.128 (1.128) Prec@1 60.94 (60.94) Prec@5 95.31 (95.31) TRAIN-SEARCH [2018-10-02-18:04:43] Epoch: [155][100/196] Time 0.63 (0.53) Data 0.00 (0.01) Loss 1.040 (1.038) Prec@1 60.94 (63.82) Prec@5 98.44 (96.75) TRAIN-SEARCH [2018-10-02-18:05:38] Epoch: [155][195/196] Time 0.33 (0.55) Data 0.00 (0.01) Loss 0.963 (1.030) Prec@1 67.50 (63.80) Prec@5 100.00 (96.65) VALID-SEARCH [2018-10-02-18:05:39] Epoch: [155][000/196] Loss 1.457 (1.457) Prec@1 56.25 (56.25) Prec@5 91.41 (91.41) VALID-SEARCH [2018-10-02-18:05:44] Epoch: [155][100/196] Loss 1.186 (1.491) Prec@1 52.34 (51.00) Prec@5 96.09 (91.70) VALID-SEARCH [2018-10-02-18:05:49] Epoch: [155][195/196] Loss 1.115 (1.457) Prec@1 52.50 (51.40) Prec@5 100.00 (92.01) 155/250, Train-Accuracy = 63.80, Test-Accuracy = 51.40 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-18:05:50] [Epoch=156/250] [Need: 03:08:42] [LR=0.0162 ~ 0.0162] [Batch=128], tau=6.256 genotype = Genotype(normal=[('dil_conv_3x3', 0, 0.12853780388832092), ('skip_connect', 1, 0.12783144414424896), ('sep_conv_3x3', 2, 0.1288309395313263), ('skip_connect', 0, 0.12843629717826843), ('sep_conv_3x3', 3, 0.12893518805503845), ('skip_connect', 0, 0.12804876267910004), ('max_pool_3x3', 1, 0.12811270356178284), ('skip_connect', 0, 0.12742772698402405)], normal_concat=range(2, 6), reduce=[('skip_connect', 0, 0.12623198330402374), ('sep_conv_5x5', 1, 0.12571105360984802), ('dil_conv_5x5', 1, 0.12599673867225647), ('dil_conv_5x5', 2, 0.12597613036632538), ('max_pool_3x3', 1, 0.12670738995075226), ('dil_conv_3x3', 3, 0.1263301968574524), ('dil_conv_5x5', 2, 0.1261850893497467), ('dil_conv_5x5', 4, 0.12613029778003693)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-18:05:52] Epoch: [156][000/196] Time 1.37 (1.37) Data 0.81 (0.81) Loss 0.896 (0.896) Prec@1 64.84 (64.84) Prec@5 100.00 (100.00) TRAIN-SEARCH [2018-10-02-18:06:45] Epoch: [156][100/196] Time 0.50 (0.54) Data 0.00 (0.01) Loss 0.918 (1.019) Prec@1 67.97 (64.06) Prec@5 97.66 (96.79) TRAIN-SEARCH [2018-10-02-18:07:36] Epoch: [156][195/196] Time 0.54 (0.54) Data 0.00 (0.01) Loss 0.954 (1.017) Prec@1 60.00 (64.20) Prec@5 97.50 (96.67) VALID-SEARCH [2018-10-02-18:07:38] Epoch: [156][000/196] Loss 1.366 (1.366) Prec@1 63.28 (63.28) Prec@5 94.53 (94.53) VALID-SEARCH [2018-10-02-18:07:43] Epoch: [156][100/196] Loss 1.663 (1.426) Prec@1 50.00 (52.31) Prec@5 89.84 (92.24) VALID-SEARCH [2018-10-02-18:07:48] Epoch: [156][195/196] Loss 2.482 (1.412) Prec@1 42.50 (52.74) Prec@5 90.00 (92.44) 156/250, Train-Accuracy = 64.20, Test-Accuracy = 52.74 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-18:07:49] [Epoch=157/250] [Need: 03:04:25] [LR=0.0159 ~ 0.0159] [Batch=128], tau=6.232 genotype = Genotype(normal=[('dil_conv_3x3', 0, 0.12874369323253632), ('skip_connect', 1, 0.12752848863601685), ('sep_conv_3x3', 2, 0.12940087914466858), ('skip_connect', 0, 0.12809740006923676), ('sep_conv_3x3', 3, 0.12854714691638947), ('skip_connect', 0, 0.1277274489402771), ('skip_connect', 0, 0.12743604183197021), ('sep_conv_5x5', 4, 0.1273348033428192)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.1266254037618637), ('sep_conv_5x5', 1, 0.12574134767055511), ('sep_conv_5x5', 2, 0.12647351622581482), ('max_pool_3x3', 0, 0.12585434317588806), ('skip_connect', 0, 0.12623369693756104), ('sep_conv_5x5', 1, 0.12605242431163788), ('dil_conv_5x5', 4, 0.12604084610939026), ('avg_pool_3x3', 0, 0.12602628767490387)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-18:07:51] Epoch: [157][000/196] Time 1.48 (1.48) Data 0.87 (0.87) Loss 1.174 (1.174) Prec@1 60.16 (60.16) Prec@5 96.09 (96.09) TRAIN-SEARCH [2018-10-02-18:08:44] Epoch: [157][100/196] Time 0.50 (0.54) Data 0.00 (0.01) Loss 1.109 (1.015) Prec@1 62.50 (64.46) Prec@5 96.88 (96.48) TRAIN-SEARCH [2018-10-02-18:09:37] Epoch: [157][195/196] Time 0.38 (0.55) Data 0.00 (0.01) Loss 1.311 (1.020) Prec@1 55.00 (64.22) Prec@5 92.50 (96.54) VALID-SEARCH [2018-10-02-18:09:38] Epoch: [157][000/196] Loss 1.250 (1.250) Prec@1 59.38 (59.38) Prec@5 94.53 (94.53) VALID-SEARCH [2018-10-02-18:09:43] Epoch: [157][100/196] Loss 0.973 (1.475) Prec@1 61.72 (50.64) Prec@5 98.44 (91.61) VALID-SEARCH [2018-10-02-18:09:48] Epoch: [157][195/196] Loss 3.416 (1.477) Prec@1 40.00 (49.92) Prec@5 90.00 (91.71) 157/250, Train-Accuracy = 64.22, Test-Accuracy = 49.92 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-18:09:49] [Epoch=158/250] [Need: 03:04:23] [LR=0.0156 ~ 0.0156] [Batch=128], tau=6.208 genotype = Genotype(normal=[('dil_conv_3x3', 0, 0.12913529574871063), ('skip_connect', 1, 0.12724736332893372), ('sep_conv_3x3', 2, 0.12913550436496735), ('skip_connect', 0, 0.12790536880493164), ('skip_connect', 0, 0.12773986160755157), ('sep_conv_3x3', 3, 0.12773124873638153), ('sep_conv_5x5', 4, 0.12743493914604187), ('max_pool_3x3', 1, 0.12732668220996857)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.12629245221614838), ('dil_conv_5x5', 1, 0.1258939504623413), ('max_pool_3x3', 0, 0.12743417918682098), ('dil_conv_5x5', 2, 0.12632165849208832), ('max_pool_3x3', 0, 0.12688620388507843), ('dil_conv_3x3', 3, 0.12592118978500366), ('max_pool_3x3', 0, 0.1265571117401123), ('dil_conv_5x5', 3, 0.1259153038263321)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-18:09:51] Epoch: [158][000/196] Time 1.48 (1.48) Data 0.83 (0.83) Loss 1.028 (1.028) Prec@1 64.84 (64.84) Prec@5 98.44 (98.44) TRAIN-SEARCH [2018-10-02-18:10:44] Epoch: [158][100/196] Time 0.44 (0.54) Data 0.00 (0.01) Loss 1.147 (1.001) Prec@1 64.06 (64.63) Prec@5 96.88 (96.84) TRAIN-SEARCH [2018-10-02-18:11:37] Epoch: [158][195/196] Time 0.44 (0.55) Data 0.00 (0.01) Loss 1.179 (1.011) Prec@1 55.00 (64.39) Prec@5 95.00 (96.76) VALID-SEARCH [2018-10-02-18:11:38] Epoch: [158][000/196] Loss 1.106 (1.106) Prec@1 60.94 (60.94) Prec@5 94.53 (94.53) VALID-SEARCH [2018-10-02-18:11:44] Epoch: [158][100/196] Loss 1.498 (1.523) Prec@1 44.53 (50.58) Prec@5 92.97 (91.38) VALID-SEARCH [2018-10-02-18:11:49] Epoch: [158][195/196] Loss 2.042 (1.526) Prec@1 37.50 (50.85) Prec@5 77.50 (91.58) 158/250, Train-Accuracy = 64.39, Test-Accuracy = 50.85 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-18:11:50] [Epoch=159/250] [Need: 03:03:01] [LR=0.0153 ~ 0.0153] [Batch=128], tau=6.184 genotype = Genotype(normal=[('skip_connect', 0, 0.12870937585830688), ('skip_connect', 1, 0.12718462944030762), ('sep_conv_3x3', 2, 0.1291343718767166), ('skip_connect', 0, 0.12724493443965912), ('sep_conv_3x3', 3, 0.1278739720582962), ('skip_connect', 0, 0.12748688459396362), ('sep_conv_5x5', 4, 0.12807443737983704), ('max_pool_3x3', 1, 0.1272316426038742)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.1260346919298172), ('skip_connect', 1, 0.12545928359031677), ('skip_connect', 1, 0.12629766762256622), ('max_pool_3x3', 0, 0.1262248009443283), ('max_pool_3x3', 0, 0.1268974095582962), ('dil_conv_5x5', 3, 0.12650851905345917), ('avg_pool_3x3', 0, 0.12696222960948944), ('dil_conv_5x5', 3, 0.12615352869033813)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-18:11:52] Epoch: [159][000/196] Time 1.79 (1.79) Data 1.16 (1.16) Loss 0.833 (0.833) Prec@1 67.97 (67.97) Prec@5 98.44 (98.44) TRAIN-SEARCH [2018-10-02-18:12:45] Epoch: [159][100/196] Time 0.55 (0.54) Data 0.00 (0.01) Loss 1.014 (1.026) Prec@1 63.28 (64.29) Prec@5 95.31 (96.46) TRAIN-SEARCH [2018-10-02-18:13:39] Epoch: [159][195/196] Time 0.44 (0.55) Data 0.00 (0.01) Loss 0.978 (1.020) Prec@1 60.00 (64.56) Prec@5 95.00 (96.58) VALID-SEARCH [2018-10-02-18:13:40] Epoch: [159][000/196] Loss 1.261 (1.261) Prec@1 55.47 (55.47) Prec@5 96.09 (96.09) VALID-SEARCH [2018-10-02-18:13:46] Epoch: [159][100/196] Loss 1.598 (1.502) Prec@1 45.31 (51.56) Prec@5 88.28 (90.40) VALID-SEARCH [2018-10-02-18:13:51] Epoch: [159][195/196] Loss 1.399 (1.526) Prec@1 45.00 (51.20) Prec@5 90.00 (90.27) 159/250, Train-Accuracy = 64.56, Test-Accuracy = 51.20 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-18:13:52] [Epoch=160/250] [Need: 03:03:09] [LR=0.0151 ~ 0.0151] [Batch=128], tau=6.16 genotype = Genotype(normal=[('skip_connect', 0, 0.12905412912368774), ('skip_connect', 1, 0.12774503231048584), ('sep_conv_5x5', 2, 0.12961484491825104), ('dil_conv_5x5', 0, 0.12748783826828003), ('skip_connect', 1, 0.1279623955488205), ('sep_conv_3x3', 3, 0.12743809819221497), ('sep_conv_3x3', 4, 0.1273006796836853), ('max_pool_3x3', 0, 0.12729783356189728)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.12609511613845825), ('sep_conv_3x3', 1, 0.12562470138072968), ('max_pool_3x3', 0, 0.12726062536239624), ('skip_connect', 1, 0.12682516872882843), ('max_pool_3x3', 0, 0.12667979300022125), ('dil_conv_3x3', 3, 0.12615333497524261), ('dil_conv_5x5', 2, 0.12653127312660217), ('dil_conv_5x5', 0, 0.12619246542453766)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-18:13:54] Epoch: [160][000/196] Time 1.46 (1.46) Data 0.93 (0.93) Loss 0.877 (0.877) Prec@1 73.44 (73.44) Prec@5 98.44 (98.44) TRAIN-SEARCH [2018-10-02-18:14:46] Epoch: [160][100/196] Time 0.57 (0.53) Data 0.00 (0.01) Loss 0.906 (1.005) Prec@1 67.97 (64.75) Prec@5 96.88 (96.90) TRAIN-SEARCH [2018-10-02-18:15:40] Epoch: [160][195/196] Time 0.37 (0.55) Data 0.00 (0.01) Loss 0.971 (1.009) Prec@1 65.00 (64.59) Prec@5 97.50 (96.80) VALID-SEARCH [2018-10-02-18:15:41] Epoch: [160][000/196] Loss 1.108 (1.108) Prec@1 65.62 (65.62) Prec@5 92.19 (92.19) VALID-SEARCH [2018-10-02-18:15:46] Epoch: [160][100/196] Loss 1.247 (1.436) Prec@1 56.25 (54.05) Prec@5 92.97 (93.66) VALID-SEARCH [2018-10-02-18:15:51] Epoch: [160][195/196] Loss 1.466 (1.403) Prec@1 47.50 (54.30) Prec@5 95.00 (93.63) 160/250, Train-Accuracy = 64.59, Test-Accuracy = 54.30 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-18:15:53] [Epoch=161/250] [Need: 02:59:10] [LR=0.0148 ~ 0.0148] [Batch=128], tau=6.136 genotype = Genotype(normal=[('skip_connect', 0, 0.1288478672504425), ('skip_connect', 1, 0.12749110162258148), ('sep_conv_5x5', 2, 0.1292586475610733), ('skip_connect', 0, 0.1278606504201889), ('sep_conv_3x3', 3, 0.12766915559768677), ('skip_connect', 0, 0.1271456778049469), ('max_pool_3x3', 0, 0.12763315439224243), ('dil_conv_5x5', 4, 0.12716588377952576)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.12619321048259735), ('dil_conv_5x5', 1, 0.12554942071437836), ('max_pool_3x3', 0, 0.12720876932144165), ('max_pool_3x3', 1, 0.12619297206401825), ('max_pool_3x3', 0, 0.126900777220726), ('dil_conv_5x5', 3, 0.125724658370018), ('avg_pool_3x3', 0, 0.12632064521312714), ('dil_conv_5x5', 2, 0.12611229717731476)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-18:15:55] Epoch: [161][000/196] Time 1.45 (1.45) Data 0.80 (0.80) Loss 1.109 (1.109) Prec@1 58.59 (58.59) Prec@5 96.88 (96.88) TRAIN-SEARCH [2018-10-02-18:16:47] Epoch: [161][100/196] Time 0.52 (0.53) Data 0.00 (0.01) Loss 1.111 (0.997) Prec@1 58.59 (64.81) Prec@5 95.31 (96.94) TRAIN-SEARCH [2018-10-02-18:17:42] Epoch: [161][195/196] Time 0.43 (0.55) Data 0.00 (0.01) Loss 1.264 (0.997) Prec@1 60.00 (64.88) Prec@5 100.00 (96.92) VALID-SEARCH [2018-10-02-18:17:43] Epoch: [161][000/196] Loss 1.369 (1.369) Prec@1 55.47 (55.47) Prec@5 95.31 (95.31) VALID-SEARCH [2018-10-02-18:17:48] Epoch: [161][100/196] Loss 1.331 (1.390) Prec@1 51.56 (53.00) Prec@5 93.75 (92.54) VALID-SEARCH [2018-10-02-18:17:52] Epoch: [161][195/196] Loss 2.373 (1.413) Prec@1 27.50 (52.96) Prec@5 65.00 (92.29) 161/250, Train-Accuracy = 64.88, Test-Accuracy = 52.96 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-18:17:55] [Epoch=162/250] [Need: 02:59:13] [LR=0.0145 ~ 0.0145] [Batch=128], tau=6.112 genotype = Genotype(normal=[('sep_conv_3x3', 0, 0.1291254609823227), ('skip_connect', 1, 0.12695753574371338), ('sep_conv_5x5', 2, 0.12871192395687103), ('dil_conv_3x3', 1, 0.12812910974025726), ('skip_connect', 0, 0.12790459394454956), ('sep_conv_3x3', 2, 0.12723484635353088), ('max_pool_3x3', 0, 0.12785568833351135), ('dil_conv_5x5', 4, 0.12781091034412384)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.12652656435966492), ('max_pool_3x3', 1, 0.12555992603302002), ('max_pool_3x3', 0, 0.12681865692138672), ('max_pool_3x3', 1, 0.12664777040481567), ('max_pool_3x3', 0, 0.12672144174575806), ('max_pool_3x3', 1, 0.12606890499591827), ('dil_conv_5x5', 0, 0.12643010914325714), ('dil_conv_3x3', 3, 0.1261056661605835)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-18:17:57] Epoch: [162][000/196] Time 1.33 (1.33) Data 0.78 (0.78) Loss 1.130 (1.130) Prec@1 60.16 (60.16) Prec@5 97.66 (97.66) TRAIN-SEARCH [2018-10-02-18:18:50] Epoch: [162][100/196] Time 0.61 (0.54) Data 0.00 (0.01) Loss 0.974 (0.998) Prec@1 64.06 (64.83) Prec@5 98.44 (96.84) TRAIN-SEARCH [2018-10-02-18:19:42] Epoch: [162][195/196] Time 0.40 (0.54) Data 0.00 (0.01) Loss 1.121 (0.996) Prec@1 55.00 (65.03) Prec@5 92.50 (96.82) VALID-SEARCH [2018-10-02-18:19:44] Epoch: [162][000/196] Loss 1.574 (1.574) Prec@1 40.62 (40.62) Prec@5 88.28 (88.28) VALID-SEARCH [2018-10-02-18:19:49] Epoch: [162][100/196] Loss 1.047 (1.388) Prec@1 57.81 (52.20) Prec@5 97.66 (92.45) VALID-SEARCH [2018-10-02-18:19:54] Epoch: [162][195/196] Loss 1.011 (1.420) Prec@1 62.50 (50.94) Prec@5 95.00 (92.02) 162/250, Train-Accuracy = 65.03, Test-Accuracy = 50.94 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-18:19:55] [Epoch=163/250] [Need: 02:54:31] [LR=0.0142 ~ 0.0142] [Batch=128], tau=6.088 genotype = Genotype(normal=[('sep_conv_3x3', 0, 0.12872272729873657), ('skip_connect', 1, 0.12728215754032135), ('sep_conv_5x5', 2, 0.12869778275489807), ('dil_conv_3x3', 1, 0.12827108800411224), ('skip_connect', 0, 0.12761357426643372), ('sep_conv_3x3', 3, 0.12751396000385284), ('max_pool_3x3', 0, 0.12767015397548676), ('sep_conv_5x5', 2, 0.12751831114292145)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.12656213343143463), ('dil_conv_5x5', 1, 0.12639617919921875), ('max_pool_3x3', 0, 0.1268729865550995), ('max_pool_3x3', 1, 0.12599767744541168), ('max_pool_3x3', 0, 0.12646618485450745), ('max_pool_3x3', 1, 0.125935897231102), ('max_pool_3x3', 1, 0.12630736827850342), ('dil_conv_5x5', 0, 0.12603257596492767)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-18:19:57] Epoch: [163][000/196] Time 1.45 (1.45) Data 0.93 (0.93) Loss 1.017 (1.017) Prec@1 60.94 (60.94) Prec@5 96.09 (96.09) TRAIN-SEARCH [2018-10-02-18:20:51] Epoch: [163][100/196] Time 0.62 (0.55) Data 0.00 (0.01) Loss 1.002 (0.969) Prec@1 63.28 (66.00) Prec@5 96.88 (97.02) TRAIN-SEARCH [2018-10-02-18:21:44] Epoch: [163][195/196] Time 0.50 (0.55) Data 0.00 (0.01) Loss 1.224 (0.991) Prec@1 62.50 (65.23) Prec@5 95.00 (96.84) VALID-SEARCH [2018-10-02-18:21:45] Epoch: [163][000/196] Loss 1.052 (1.052) Prec@1 66.41 (66.41) Prec@5 96.88 (96.88) VALID-SEARCH [2018-10-02-18:21:50] Epoch: [163][100/196] Loss 1.247 (1.449) Prec@1 60.94 (53.03) Prec@5 94.53 (93.28) VALID-SEARCH [2018-10-02-18:21:55] Epoch: [163][195/196] Loss 1.280 (1.428) Prec@1 57.50 (53.73) Prec@5 95.00 (93.61) 163/250, Train-Accuracy = 65.23, Test-Accuracy = 53.73 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-18:21:56] [Epoch=164/250] [Need: 02:52:43] [LR=0.0140 ~ 0.0140] [Batch=128], tau=6.064 genotype = Genotype(normal=[('skip_connect', 0, 0.12919002771377563), ('sep_conv_3x3', 1, 0.12750846147537231), ('sep_conv_5x5', 2, 0.12873774766921997), ('dil_conv_3x3', 1, 0.12838423252105713), ('sep_conv_3x3', 3, 0.1278393417596817), ('skip_connect', 0, 0.12771737575531006), ('max_pool_3x3', 0, 0.12816183269023895), ('sep_conv_5x5', 2, 0.1275365650653839)], normal_concat=range(2, 6), reduce=[('dil_conv_5x5', 1, 0.12624825537204742), ('avg_pool_3x3', 0, 0.12621209025382996), ('max_pool_3x3', 0, 0.1267215609550476), ('avg_pool_3x3', 1, 0.12612605094909668), ('dil_conv_5x5', 3, 0.1262376308441162), ('skip_connect', 0, 0.12578463554382324), ('skip_connect', 2, 0.1259434074163437), ('sep_conv_3x3', 4, 0.12593773007392883)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-18:21:58] Epoch: [164][000/196] Time 1.50 (1.50) Data 0.98 (0.98) Loss 1.083 (1.083) Prec@1 57.03 (57.03) Prec@5 94.53 (94.53) TRAIN-SEARCH [2018-10-02-18:22:53] Epoch: [164][100/196] Time 0.57 (0.56) Data 0.01 (0.01) Loss 1.133 (0.996) Prec@1 62.50 (65.39) Prec@5 95.31 (96.67) TRAIN-SEARCH [2018-10-02-18:23:45] Epoch: [164][195/196] Time 0.51 (0.55) Data 0.00 (0.01) Loss 0.888 (0.996) Prec@1 65.00 (65.32) Prec@5 100.00 (96.76) VALID-SEARCH [2018-10-02-18:23:46] Epoch: [164][000/196] Loss 1.602 (1.602) Prec@1 42.97 (42.97) Prec@5 89.06 (89.06) VALID-SEARCH [2018-10-02-18:23:52] Epoch: [164][100/196] Loss 1.528 (1.461) Prec@1 58.59 (51.84) Prec@5 92.97 (91.65) VALID-SEARCH [2018-10-02-18:23:57] Epoch: [164][195/196] Loss 1.260 (1.500) Prec@1 50.00 (50.92) Prec@5 95.00 (91.17) 164/250, Train-Accuracy = 65.32, Test-Accuracy = 50.92 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-18:23:58] [Epoch=165/250] [Need: 02:52:51] [LR=0.0137 ~ 0.0137] [Batch=128], tau=6.04 genotype = Genotype(normal=[('skip_connect', 0, 0.12953859567642212), ('skip_connect', 1, 0.12744246423244476), ('dil_conv_3x3', 2, 0.1290607899427414), ('skip_connect', 0, 0.12794342637062073), ('sep_conv_3x3', 3, 0.1279377043247223), ('max_pool_3x3', 0, 0.12755414843559265), ('max_pool_3x3', 0, 0.12758034467697144), ('sep_conv_5x5', 4, 0.12716126441955566)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.12584958970546722), ('skip_connect', 1, 0.12562932074069977), ('max_pool_3x3', 0, 0.12648847699165344), ('avg_pool_3x3', 1, 0.12618288397789001), ('dil_conv_3x3', 3, 0.126260906457901), ('dil_conv_3x3', 1, 0.12601302564144135), ('dil_conv_5x5', 0, 0.12665143609046936), ('max_pool_3x3', 1, 0.12624812126159668)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-18:23:59] Epoch: [165][000/196] Time 1.04 (1.04) Data 0.62 (0.62) Loss 1.078 (1.078) Prec@1 59.38 (59.38) Prec@5 97.66 (97.66) TRAIN-SEARCH [2018-10-02-18:24:50] Epoch: [165][100/196] Time 0.54 (0.52) Data 0.00 (0.01) Loss 1.021 (1.001) Prec@1 62.50 (64.28) Prec@5 95.31 (96.99) TRAIN-SEARCH [2018-10-02-18:25:45] Epoch: [165][195/196] Time 0.53 (0.54) Data 0.00 (0.01) Loss 0.993 (0.989) Prec@1 57.50 (64.99) Prec@5 97.50 (97.07) VALID-SEARCH [2018-10-02-18:25:46] Epoch: [165][000/196] Loss 1.394 (1.394) Prec@1 51.56 (51.56) Prec@5 92.19 (92.19) VALID-SEARCH [2018-10-02-18:25:51] Epoch: [165][100/196] Loss 1.401 (1.424) Prec@1 49.22 (52.85) Prec@5 96.09 (92.87) VALID-SEARCH [2018-10-02-18:25:56] Epoch: [165][195/196] Loss 0.913 (1.448) Prec@1 70.00 (52.74) Prec@5 97.50 (92.47) 165/250, Train-Accuracy = 64.99, Test-Accuracy = 52.74 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-18:25:57] [Epoch=166/250] [Need: 02:47:04] [LR=0.0134 ~ 0.0134] [Batch=128], tau=6.016 genotype = Genotype(normal=[('skip_connect', 0, 0.12961672246456146), ('skip_connect', 1, 0.1281532347202301), ('dil_conv_3x3', 2, 0.1286630928516388), ('skip_connect', 1, 0.1276739090681076), ('sep_conv_3x3', 3, 0.1278359293937683), ('skip_connect', 0, 0.12780162692070007), ('max_pool_3x3', 0, 0.12736031413078308), ('sep_conv_5x5', 4, 0.12735490500926971)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.12581001222133636), ('sep_conv_3x3', 1, 0.12557770311832428), ('max_pool_3x3', 0, 0.12654756009578705), ('avg_pool_3x3', 1, 0.12599898874759674), ('max_pool_3x3', 0, 0.12625478208065033), ('dil_conv_3x3', 3, 0.1259545236825943), ('max_pool_3x3', 1, 0.12596029043197632), ('dil_conv_5x5', 0, 0.12577314674854279)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-18:25:59] Epoch: [166][000/196] Time 0.96 (0.96) Data 0.54 (0.54) Loss 1.007 (1.007) Prec@1 64.06 (64.06) Prec@5 98.44 (98.44) TRAIN-SEARCH [2018-10-02-18:26:50] Epoch: [166][100/196] Time 0.55 (0.51) Data 0.00 (0.01) Loss 1.073 (1.004) Prec@1 63.28 (65.04) Prec@5 92.19 (96.67) TRAIN-SEARCH [2018-10-02-18:27:44] Epoch: [166][195/196] Time 0.44 (0.54) Data 0.00 (0.01) Loss 1.001 (0.996) Prec@1 60.00 (65.20) Prec@5 95.00 (96.80) VALID-SEARCH [2018-10-02-18:27:45] Epoch: [166][000/196] Loss 1.146 (1.146) Prec@1 60.16 (60.16) Prec@5 98.44 (98.44) VALID-SEARCH [2018-10-02-18:27:50] Epoch: [166][100/196] Loss 1.005 (1.405) Prec@1 62.50 (55.86) Prec@5 96.09 (93.90) VALID-SEARCH [2018-10-02-18:27:55] Epoch: [166][195/196] Loss 1.499 (1.397) Prec@1 47.50 (55.84) Prec@5 95.00 (93.88) 166/250, Train-Accuracy = 65.20, Test-Accuracy = 55.84 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-18:27:56] [Epoch=167/250] [Need: 02:45:02] [LR=0.0132 ~ 0.0132] [Batch=128], tau=5.992 genotype = Genotype(normal=[('skip_connect', 0, 0.12940268218517303), ('skip_connect', 1, 0.12820126116275787), ('dil_conv_3x3', 2, 0.128824383020401), ('skip_connect', 0, 0.12817466259002686), ('skip_connect', 0, 0.1284995824098587), ('sep_conv_3x3', 3, 0.127923384308815), ('max_pool_3x3', 0, 0.12776802480220795), ('sep_conv_5x5', 4, 0.12750090658664703)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.12609179317951202), ('dil_conv_3x3', 1, 0.1255435347557068), ('max_pool_3x3', 0, 0.12641838192939758), ('sep_conv_5x5', 2, 0.1260775774717331), ('sep_conv_5x5', 2, 0.12628822028636932), ('max_pool_3x3', 0, 0.1261320859193802), ('avg_pool_3x3', 1, 0.12626390159130096), ('sep_conv_5x5', 0, 0.12611036002635956)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-18:27:58] Epoch: [167][000/196] Time 0.95 (0.95) Data 0.52 (0.52) Loss 0.959 (0.959) Prec@1 64.84 (64.84) Prec@5 97.66 (97.66) TRAIN-SEARCH [2018-10-02-18:28:51] Epoch: [167][100/196] Time 0.54 (0.53) Data 0.00 (0.01) Loss 0.907 (0.976) Prec@1 67.19 (66.01) Prec@5 100.00 (96.90) TRAIN-SEARCH [2018-10-02-18:29:45] Epoch: [167][195/196] Time 0.45 (0.55) Data 0.00 (0.01) Loss 1.280 (0.984) Prec@1 52.50 (65.70) Prec@5 95.00 (96.89) VALID-SEARCH [2018-10-02-18:29:46] Epoch: [167][000/196] Loss 1.473 (1.473) Prec@1 50.78 (50.78) Prec@5 91.41 (91.41) VALID-SEARCH [2018-10-02-18:29:51] Epoch: [167][100/196] Loss 1.368 (1.428) Prec@1 46.88 (53.36) Prec@5 93.75 (93.14) VALID-SEARCH [2018-10-02-18:29:55] Epoch: [167][195/196] Loss 1.678 (1.414) Prec@1 55.00 (54.04) Prec@5 82.50 (93.02) 167/250, Train-Accuracy = 65.70, Test-Accuracy = 54.04 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-18:29:56] [Epoch=168/250] [Need: 02:43:01] [LR=0.0129 ~ 0.0129] [Batch=128], tau=5.968 genotype = Genotype(normal=[('skip_connect', 0, 0.1289444863796234), ('skip_connect', 1, 0.12811119854450226), ('sep_conv_5x5', 2, 0.1292167752981186), ('skip_connect', 0, 0.1287708431482315), ('skip_connect', 0, 0.12808692455291748), ('sep_conv_3x3', 3, 0.12787532806396484), ('max_pool_3x3', 1, 0.12825152277946472), ('dil_conv_3x3', 4, 0.1276525855064392)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.12588568031787872), ('sep_conv_3x3', 1, 0.12546667456626892), ('avg_pool_3x3', 0, 0.12698780000209808), ('sep_conv_5x5', 2, 0.12569038569927216), ('max_pool_3x3', 0, 0.12654690444469452), ('sep_conv_5x5', 3, 0.12605324387550354), ('sep_conv_5x5', 1, 0.12644968926906586), ('dil_conv_5x5', 2, 0.12609119713306427)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-18:29:57] Epoch: [168][000/196] Time 1.07 (1.07) Data 0.66 (0.66) Loss 0.977 (0.977) Prec@1 64.84 (64.84) Prec@5 96.09 (96.09) TRAIN-SEARCH [2018-10-02-18:30:51] Epoch: [168][100/196] Time 0.47 (0.55) Data 0.00 (0.01) Loss 1.009 (0.969) Prec@1 67.19 (65.86) Prec@5 95.31 (96.97) TRAIN-SEARCH [2018-10-02-18:31:44] Epoch: [168][195/196] Time 0.42 (0.55) Data 0.00 (0.01) Loss 1.006 (0.966) Prec@1 62.50 (65.73) Prec@5 97.50 (97.10) VALID-SEARCH [2018-10-02-18:31:45] Epoch: [168][000/196] Loss 1.562 (1.562) Prec@1 46.09 (46.09) Prec@5 93.75 (93.75) VALID-SEARCH [2018-10-02-18:31:50] Epoch: [168][100/196] Loss 1.753 (1.451) Prec@1 46.88 (54.76) Prec@5 90.62 (93.53) VALID-SEARCH [2018-10-02-18:31:54] Epoch: [168][195/196] Loss 0.916 (1.484) Prec@1 70.00 (53.93) Prec@5 92.50 (92.96) 168/250, Train-Accuracy = 65.73, Test-Accuracy = 53.93 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-18:31:55] [Epoch=169/250] [Need: 02:41:36] [LR=0.0126 ~ 0.0126] [Batch=128], tau=5.944 genotype = Genotype(normal=[('skip_connect', 0, 0.12913452088832855), ('skip_connect', 1, 0.12792213261127472), ('skip_connect', 0, 0.12955762445926666), ('sep_conv_5x5', 2, 0.12933364510536194), ('skip_connect', 0, 0.12885764241218567), ('sep_conv_3x3', 3, 0.1281316578388214), ('sep_conv_5x5', 4, 0.127855122089386), ('skip_connect', 0, 0.12781408429145813)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.12590844929218292), ('dil_conv_3x3', 1, 0.12523101270198822), ('avg_pool_3x3', 0, 0.12653794884681702), ('sep_conv_5x5', 2, 0.12619319558143616), ('max_pool_3x3', 0, 0.12626346945762634), ('dil_conv_5x5', 2, 0.12622755765914917), ('sep_conv_5x5', 0, 0.12627553939819336), ('dil_conv_5x5', 2, 0.12619712948799133)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-18:31:57] Epoch: [169][000/196] Time 1.02 (1.02) Data 0.60 (0.60) Loss 1.044 (1.044) Prec@1 63.28 (63.28) Prec@5 96.09 (96.09) TRAIN-SEARCH [2018-10-02-18:32:52] Epoch: [169][100/196] Time 0.58 (0.55) Data 0.01 (0.01) Loss 0.989 (0.980) Prec@1 66.41 (65.42) Prec@5 96.09 (97.11) TRAIN-SEARCH [2018-10-02-18:33:42] Epoch: [169][195/196] Time 0.40 (0.54) Data 0.00 (0.01) Loss 0.840 (0.971) Prec@1 55.00 (65.86) Prec@5 97.50 (97.00) VALID-SEARCH [2018-10-02-18:33:43] Epoch: [169][000/196] Loss 1.411 (1.411) Prec@1 53.12 (53.12) Prec@5 95.31 (95.31) VALID-SEARCH [2018-10-02-18:33:48] Epoch: [169][100/196] Loss 1.147 (1.460) Prec@1 58.59 (52.66) Prec@5 95.31 (92.50) VALID-SEARCH [2018-10-02-18:33:52] Epoch: [169][195/196] Loss 2.133 (1.499) Prec@1 50.00 (51.70) Prec@5 92.50 (91.98) 169/250, Train-Accuracy = 65.86, Test-Accuracy = 51.70 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-18:33:52] [Epoch=170/250] [Need: 02:36:09] [LR=0.0124 ~ 0.0124] [Batch=128], tau=5.92 genotype = Genotype(normal=[('dil_conv_5x5', 0, 0.1291835755109787), ('skip_connect', 1, 0.12731358408927917), ('skip_connect', 0, 0.12952454388141632), ('sep_conv_5x5', 2, 0.12871669232845306), ('sep_conv_3x3', 3, 0.12805432081222534), ('skip_connect', 0, 0.12765324115753174), ('dil_conv_3x3', 4, 0.12790541350841522), ('max_pool_3x3', 0, 0.12741436064243317)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.1262964904308319), ('sep_conv_3x3', 1, 0.12533611059188843), ('max_pool_3x3', 0, 0.1271127611398697), ('dil_conv_5x5', 2, 0.12587732076644897), ('dil_conv_5x5', 1, 0.12628285586833954), ('dil_conv_5x5', 0, 0.1258532702922821), ('sep_conv_5x5', 0, 0.12619557976722717), ('dil_conv_5x5', 4, 0.12582825124263763)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-18:33:54] Epoch: [170][000/196] Time 1.00 (1.00) Data 0.59 (0.59) Loss 0.890 (0.890) Prec@1 65.62 (65.62) Prec@5 97.66 (97.66) TRAIN-SEARCH [2018-10-02-18:34:48] Epoch: [170][100/196] Time 0.54 (0.54) Data 0.00 (0.01) Loss 0.999 (0.977) Prec@1 67.19 (65.86) Prec@5 99.22 (97.04) TRAIN-SEARCH [2018-10-02-18:35:39] Epoch: [170][195/196] Time 0.52 (0.54) Data 0.00 (0.01) Loss 1.002 (0.975) Prec@1 60.00 (65.77) Prec@5 100.00 (97.10) VALID-SEARCH [2018-10-02-18:35:40] Epoch: [170][000/196] Loss 1.375 (1.375) Prec@1 53.12 (53.12) Prec@5 92.19 (92.19) VALID-SEARCH [2018-10-02-18:35:45] Epoch: [170][100/196] Loss 1.888 (1.410) Prec@1 39.06 (53.01) Prec@5 73.44 (92.86) VALID-SEARCH [2018-10-02-18:35:48] Epoch: [170][195/196] Loss 1.636 (1.405) Prec@1 32.50 (53.61) Prec@5 95.00 (92.82) 170/250, Train-Accuracy = 65.77, Test-Accuracy = 53.61 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-18:35:49] [Epoch=171/250] [Need: 02:33:53] [LR=0.0121 ~ 0.0121] [Batch=128], tau=5.896 genotype = Genotype(normal=[('dil_conv_5x5', 0, 0.129515141248703), ('skip_connect', 1, 0.12740986049175262), ('sep_conv_5x5', 2, 0.12923380732536316), ('skip_connect', 0, 0.12912696599960327), ('sep_conv_3x3', 3, 0.12818457186222076), ('skip_connect', 0, 0.12776584923267365), ('max_pool_3x3', 0, 0.12821656465530396), ('sep_conv_3x3', 4, 0.12790712714195251)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.1262030005455017), ('dil_conv_5x5', 1, 0.125589981675148), ('avg_pool_3x3', 0, 0.12649789452552795), ('sep_conv_5x5', 2, 0.12633612751960754), ('max_pool_3x3', 0, 0.12657926976680756), ('dil_conv_5x5', 1, 0.12597106397151947), ('sep_conv_5x5', 0, 0.12774880230426788), ('max_pool_3x3', 1, 0.12599316239356995)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-18:35:51] Epoch: [171][000/196] Time 1.02 (1.02) Data 0.61 (0.61) Loss 0.972 (0.972) Prec@1 68.75 (68.75) Prec@5 96.88 (96.88) TRAIN-SEARCH [2018-10-02-18:36:47] Epoch: [171][100/196] Time 0.61 (0.57) Data 0.00 (0.01) Loss 0.945 (0.952) Prec@1 61.72 (67.02) Prec@5 99.22 (97.12) TRAIN-SEARCH [2018-10-02-18:37:40] Epoch: [171][195/196] Time 0.33 (0.56) Data 0.00 (0.01) Loss 1.038 (0.958) Prec@1 60.00 (66.66) Prec@5 95.00 (97.02) VALID-SEARCH [2018-10-02-18:37:40] Epoch: [171][000/196] Loss 1.650 (1.650) Prec@1 38.28 (38.28) Prec@5 88.28 (88.28) VALID-SEARCH [2018-10-02-18:37:44] Epoch: [171][100/196] Loss 1.864 (1.434) Prec@1 53.12 (52.70) Prec@5 92.19 (92.20) VALID-SEARCH [2018-10-02-18:37:48] Epoch: [171][195/196] Loss 1.318 (1.420) Prec@1 55.00 (53.04) Prec@5 95.00 (92.23) 171/250, Train-Accuracy = 66.66, Test-Accuracy = 53.04 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-18:37:50] [Epoch=172/250] [Need: 02:36:31] [LR=0.0119 ~ 0.0119] [Batch=128], tau=5.872 genotype = Genotype(normal=[('dil_conv_5x5', 0, 0.12979139387607574), ('skip_connect', 1, 0.12839604914188385), ('skip_connect', 0, 0.12916851043701172), ('sep_conv_5x5', 2, 0.1290559023618698), ('skip_connect', 0, 0.1281379759311676), ('skip_connect', 1, 0.12775877118110657), ('max_pool_3x3', 0, 0.12834927439689636), ('dil_conv_3x3', 4, 0.12791961431503296)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.12670563161373138), ('dil_conv_5x5', 1, 0.12644848227500916), ('avg_pool_3x3', 0, 0.1260862499475479), ('avg_pool_3x3', 1, 0.12607423961162567), ('max_pool_3x3', 0, 0.12647166848182678), ('dil_conv_5x5', 1, 0.12586155533790588), ('sep_conv_5x5', 0, 0.12650106847286224), ('dil_conv_3x3', 4, 0.1261965036392212)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-18:37:52] Epoch: [172][000/196] Time 1.61 (1.61) Data 1.10 (1.10) Loss 0.905 (0.905) Prec@1 62.50 (62.50) Prec@5 98.44 (98.44) TRAIN-SEARCH [2018-10-02-18:38:47] Epoch: [172][100/196] Time 0.58 (0.56) Data 0.00 (0.01) Loss 0.919 (0.970) Prec@1 70.31 (66.15) Prec@5 96.09 (96.88) TRAIN-SEARCH [2018-10-02-18:39:40] Epoch: [172][195/196] Time 0.30 (0.56) Data 0.00 (0.01) Loss 0.812 (0.964) Prec@1 75.00 (66.22) Prec@5 97.50 (96.88) VALID-SEARCH [2018-10-02-18:39:41] Epoch: [172][000/196] Loss 1.298 (1.298) Prec@1 57.03 (57.03) Prec@5 93.75 (93.75) VALID-SEARCH [2018-10-02-18:39:45] Epoch: [172][100/196] Loss 1.238 (1.416) Prec@1 64.06 (52.81) Prec@5 92.19 (91.73) VALID-SEARCH [2018-10-02-18:39:49] Epoch: [172][195/196] Loss 1.326 (1.472) Prec@1 45.00 (51.34) Prec@5 95.00 (91.14) 172/250, Train-Accuracy = 66.22, Test-Accuracy = 51.34 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-18:39:50] [Epoch=173/250] [Need: 02:34:29] [LR=0.0116 ~ 0.0116] [Batch=128], tau=5.848000000000001 genotype = Genotype(normal=[('dil_conv_5x5', 0, 0.12956656515598297), ('sep_conv_3x3', 1, 0.12830278277397156), ('skip_connect', 0, 0.12923036515712738), ('sep_conv_5x5', 2, 0.12893123924732208), ('skip_connect', 0, 0.12796489894390106), ('sep_conv_5x5', 3, 0.12785382568836212), ('max_pool_3x3', 0, 0.12789350748062134), ('dil_conv_3x3', 4, 0.12753254175186157)], normal_concat=range(2, 6), reduce=[('skip_connect', 0, 0.12601172924041748), ('dil_conv_5x5', 1, 0.12587963044643402), ('max_pool_3x3', 0, 0.12653611600399017), ('dil_conv_3x3', 2, 0.12609383463859558), ('max_pool_3x3', 0, 0.12647846341133118), ('sep_conv_3x3', 3, 0.12614484131336212), ('max_pool_3x3', 0, 0.1268768161535263), ('dil_conv_3x3', 2, 0.12568944692611694)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-18:39:53] Epoch: [173][000/196] Time 1.64 (1.64) Data 1.03 (1.03) Loss 1.079 (1.079) Prec@1 61.72 (61.72) Prec@5 96.09 (96.09) TRAIN-SEARCH [2018-10-02-18:40:49] Epoch: [173][100/196] Time 0.65 (0.57) Data 0.01 (0.01) Loss 1.011 (0.939) Prec@1 64.84 (66.89) Prec@5 96.88 (97.05) TRAIN-SEARCH [2018-10-02-18:41:38] Epoch: [173][195/196] Time 0.32 (0.55) Data 0.00 (0.01) Loss 1.096 (0.952) Prec@1 62.50 (66.35) Prec@5 97.50 (97.00) VALID-SEARCH [2018-10-02-18:41:39] Epoch: [173][000/196] Loss 1.454 (1.454) Prec@1 46.88 (46.88) Prec@5 89.84 (89.84) VALID-SEARCH [2018-10-02-18:41:43] Epoch: [173][100/196] Loss 1.188 (1.413) Prec@1 54.69 (53.41) Prec@5 94.53 (92.61) VALID-SEARCH [2018-10-02-18:41:47] Epoch: [173][195/196] Loss 2.733 (1.394) Prec@1 20.00 (53.86) Prec@5 77.50 (92.63) 173/250, Train-Accuracy = 66.35, Test-Accuracy = 53.86 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-18:41:48] [Epoch=174/250] [Need: 02:28:59] [LR=0.0113 ~ 0.0113] [Batch=128], tau=5.824 genotype = Genotype(normal=[('dil_conv_5x5', 0, 0.13000993430614471), ('sep_conv_3x3', 1, 0.12836885452270508), ('sep_conv_5x5', 2, 0.12901966273784637), ('skip_connect', 0, 0.12897458672523499), ('sep_conv_5x5', 0, 0.128145232796669), ('sep_conv_3x3', 3, 0.12760710716247559), ('max_pool_3x3', 0, 0.12797197699546814), ('sep_conv_5x5', 2, 0.12750829756259918)], normal_concat=range(2, 6), reduce=[('skip_connect', 0, 0.1260640025138855), ('avg_pool_3x3', 1, 0.12560440599918365), ('max_pool_3x3', 0, 0.12659505009651184), ('dil_conv_3x3', 2, 0.12653040885925293), ('dil_conv_5x5', 0, 0.12617436051368713), ('sep_conv_3x3', 3, 0.12613803148269653), ('sep_conv_5x5', 0, 0.12634161114692688), ('dil_conv_3x3', 2, 0.12577596306800842)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-18:41:50] Epoch: [174][000/196] Time 1.51 (1.51) Data 0.92 (0.92) Loss 0.972 (0.972) Prec@1 64.06 (64.06) Prec@5 97.66 (97.66) TRAIN-SEARCH [2018-10-02-18:42:44] Epoch: [174][100/196] Time 0.58 (0.55) Data 0.02 (0.01) Loss 0.953 (0.955) Prec@1 67.19 (66.89) Prec@5 96.09 (97.08) TRAIN-SEARCH [2018-10-02-18:43:36] Epoch: [174][195/196] Time 0.33 (0.55) Data 0.00 (0.01) Loss 0.933 (0.958) Prec@1 62.50 (66.80) Prec@5 100.00 (97.16) VALID-SEARCH [2018-10-02-18:43:37] Epoch: [174][000/196] Loss 1.302 (1.302) Prec@1 50.78 (50.78) Prec@5 95.31 (95.31) VALID-SEARCH [2018-10-02-18:43:42] Epoch: [174][100/196] Loss 1.734 (1.380) Prec@1 36.72 (54.46) Prec@5 89.84 (93.47) VALID-SEARCH [2018-10-02-18:43:47] Epoch: [174][195/196] Loss 1.086 (1.361) Prec@1 70.00 (54.95) Prec@5 97.50 (93.46) 174/250, Train-Accuracy = 66.80, Test-Accuracy = 54.95 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-18:43:48] [Epoch=175/250] [Need: 02:30:45] [LR=0.0111 ~ 0.0111] [Batch=128], tau=5.800000000000001 genotype = Genotype(normal=[('dil_conv_5x5', 0, 0.12997592985630035), ('sep_conv_3x3', 1, 0.12905241549015045), ('sep_conv_5x5', 2, 0.12936033308506012), ('skip_connect', 0, 0.12777481973171234), ('sep_conv_5x5', 0, 0.12858812510967255), ('dil_conv_3x3', 3, 0.12763433158397675), ('sep_conv_5x5', 2, 0.12784351408481598), ('max_pool_3x3', 0, 0.12783607840538025)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.12589319050312042), ('max_pool_3x3', 1, 0.12564708292484283), ('max_pool_3x3', 0, 0.12730392813682556), ('max_pool_3x3', 1, 0.12600892782211304), ('dil_conv_5x5', 0, 0.12630057334899902), ('sep_conv_5x5', 3, 0.1258714497089386), ('avg_pool_3x3', 0, 0.12604807317256927), ('dil_conv_5x5', 1, 0.12590475380420685)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-18:43:51] Epoch: [175][000/196] Time 1.47 (1.47) Data 0.87 (0.87) Loss 1.005 (1.005) Prec@1 63.28 (63.28) Prec@5 96.88 (96.88) TRAIN-SEARCH [2018-10-02-18:44:44] Epoch: [175][100/196] Time 0.46 (0.55) Data 0.00 (0.01) Loss 1.133 (0.958) Prec@1 58.59 (66.29) Prec@5 96.09 (97.00) TRAIN-SEARCH [2018-10-02-18:45:33] Epoch: [175][195/196] Time 0.47 (0.53) Data 0.00 (0.01) Loss 0.585 (0.958) Prec@1 80.00 (66.21) Prec@5 100.00 (97.06) VALID-SEARCH [2018-10-02-18:45:34] Epoch: [175][000/196] Loss 1.835 (1.835) Prec@1 58.59 (58.59) Prec@5 98.44 (98.44) VALID-SEARCH [2018-10-02-18:45:38] Epoch: [175][100/196] Loss 1.294 (1.481) Prec@1 56.25 (52.91) Prec@5 92.97 (92.47) VALID-SEARCH [2018-10-02-18:45:43] Epoch: [175][195/196] Loss 1.376 (1.436) Prec@1 52.50 (53.55) Prec@5 95.00 (92.84) 175/250, Train-Accuracy = 66.21, Test-Accuracy = 53.55 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-18:45:44] [Epoch=176/250] [Need: 02:22:56] [LR=0.0109 ~ 0.0109] [Batch=128], tau=5.776 genotype = Genotype(normal=[('dil_conv_5x5', 0, 0.12929189205169678), ('sep_conv_3x3', 1, 0.12911829352378845), ('sep_conv_5x5', 2, 0.1286126971244812), ('dil_conv_3x3', 1, 0.12771227955818176), ('dil_conv_3x3', 3, 0.1278376281261444), ('sep_conv_5x5', 0, 0.12780553102493286), ('dil_conv_5x5', 0, 0.12772540748119354), ('sep_conv_5x5', 2, 0.1274237185716629)], normal_concat=range(2, 6), reduce=[('skip_connect', 0, 0.12574075162410736), ('sep_conv_5x5', 1, 0.12566548585891724), ('max_pool_3x3', 0, 0.1264360547065735), ('max_pool_3x3', 1, 0.1261652261018753), ('sep_conv_5x5', 3, 0.1262167990207672), ('max_pool_3x3', 0, 0.12616072595119476), ('max_pool_3x3', 1, 0.1263100802898407), ('dil_conv_5x5', 2, 0.12626804411411285)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-18:45:46] Epoch: [176][000/196] Time 1.40 (1.40) Data 0.85 (0.85) Loss 0.979 (0.979) Prec@1 61.72 (61.72) Prec@5 98.44 (98.44) TRAIN-SEARCH [2018-10-02-18:46:45] Epoch: [176][100/196] Time 0.58 (0.59) Data 0.00 (0.01) Loss 0.956 (0.960) Prec@1 64.84 (66.24) Prec@5 95.31 (97.04) TRAIN-SEARCH [2018-10-02-18:47:33] Epoch: [176][195/196] Time 0.44 (0.55) Data 0.00 (0.01) Loss 0.899 (0.953) Prec@1 65.00 (66.53) Prec@5 100.00 (97.10) VALID-SEARCH [2018-10-02-18:47:35] Epoch: [176][000/196] Loss 1.067 (1.067) Prec@1 64.06 (64.06) Prec@5 96.88 (96.88) VALID-SEARCH [2018-10-02-18:47:40] Epoch: [176][100/196] Loss 2.067 (1.448) Prec@1 28.12 (52.04) Prec@5 81.25 (91.48) VALID-SEARCH [2018-10-02-18:47:45] Epoch: [176][195/196] Loss 0.828 (1.433) Prec@1 67.50 (52.34) Prec@5 97.50 (91.49) 176/250, Train-Accuracy = 66.53, Test-Accuracy = 52.34 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-18:47:46] [Epoch=177/250] [Need: 02:27:56] [LR=0.0106 ~ 0.0106] [Batch=128], tau=5.752000000000001 genotype = Genotype(normal=[('sep_conv_3x3', 1, 0.12990789115428925), ('dil_conv_5x5', 0, 0.12837782502174377), ('sep_conv_5x5', 2, 0.12931911647319794), ('skip_connect', 0, 0.1271456927061081), ('sep_conv_5x5', 2, 0.1278391033411026), ('sep_conv_5x5', 0, 0.12776003777980804), ('sep_conv_5x5', 4, 0.12754832208156586), ('dil_conv_5x5', 0, 0.12714776396751404)], normal_concat=range(2, 6), reduce=[('sep_conv_5x5', 0, 0.12579506635665894), ('max_pool_3x3', 1, 0.12556681036949158), ('max_pool_3x3', 0, 0.12680505216121674), ('dil_conv_5x5', 2, 0.12609048187732697), ('dil_conv_5x5', 2, 0.12655939161777496), ('avg_pool_3x3', 0, 0.12636226415634155), ('dil_conv_3x3', 4, 0.1261710673570633), ('sep_conv_5x5', 3, 0.12615536153316498)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-18:47:48] Epoch: [177][000/196] Time 1.50 (1.50) Data 0.94 (0.94) Loss 1.102 (1.102) Prec@1 59.38 (59.38) Prec@5 97.66 (97.66) TRAIN-SEARCH [2018-10-02-18:48:46] Epoch: [177][100/196] Time 0.62 (0.59) Data 0.02 (0.01) Loss 0.851 (0.957) Prec@1 72.66 (66.49) Prec@5 97.66 (97.15) TRAIN-SEARCH [2018-10-02-18:49:33] Epoch: [177][195/196] Time 0.43 (0.55) Data 0.00 (0.01) Loss 1.252 (0.950) Prec@1 55.00 (66.63) Prec@5 92.50 (97.16) VALID-SEARCH [2018-10-02-18:49:35] Epoch: [177][000/196] Loss 1.880 (1.880) Prec@1 67.19 (67.19) Prec@5 96.09 (96.09) VALID-SEARCH [2018-10-02-18:49:40] Epoch: [177][100/196] Loss 1.190 (1.430) Prec@1 60.16 (52.74) Prec@5 96.88 (92.03) VALID-SEARCH [2018-10-02-18:49:45] Epoch: [177][195/196] Loss 1.244 (1.427) Prec@1 45.00 (52.93) Prec@5 100.00 (92.15) 177/250, Train-Accuracy = 66.63, Test-Accuracy = 52.93 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-18:49:46] [Epoch=178/250] [Need: 02:23:47] [LR=0.0104 ~ 0.0104] [Batch=128], tau=5.728 genotype = Genotype(normal=[('sep_conv_3x3', 1, 0.1299487203359604), ('skip_connect', 0, 0.12872029840946198), ('sep_conv_5x5', 2, 0.12921515107154846), ('skip_connect', 0, 0.12751249969005585), ('dil_conv_3x3', 3, 0.12797881662845612), ('skip_connect', 0, 0.12774907052516937), ('sep_conv_5x5', 4, 0.12758396565914154), ('sep_conv_3x3', 2, 0.1270504593849182)], normal_concat=range(2, 6), reduce=[('sep_conv_5x5', 0, 0.1258097141981125), ('max_pool_3x3', 1, 0.1255335509777069), ('dil_conv_5x5', 2, 0.12624825537204742), ('max_pool_3x3', 1, 0.1262068897485733), ('avg_pool_3x3', 0, 0.12663203477859497), ('dil_conv_5x5', 1, 0.12604883313179016), ('sep_conv_5x5', 3, 0.12613536417484283), ('dil_conv_3x3', 4, 0.12597469985485077)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-18:49:48] Epoch: [178][000/196] Time 1.48 (1.48) Data 0.95 (0.95) Loss 1.013 (1.013) Prec@1 64.06 (64.06) Prec@5 93.75 (93.75) TRAIN-SEARCH [2018-10-02-18:50:45] Epoch: [178][100/196] Time 0.53 (0.58) Data 0.01 (0.01) Loss 0.987 (0.933) Prec@1 68.75 (67.17) Prec@5 93.75 (97.20) TRAIN-SEARCH [2018-10-02-18:51:35] Epoch: [178][195/196] Time 0.52 (0.55) Data 0.00 (0.01) Loss 0.753 (0.941) Prec@1 75.00 (66.72) Prec@5 100.00 (97.15) VALID-SEARCH [2018-10-02-18:51:36] Epoch: [178][000/196] Loss 1.021 (1.021) Prec@1 64.06 (64.06) Prec@5 97.66 (97.66) VALID-SEARCH [2018-10-02-18:51:41] Epoch: [178][100/196] Loss 1.185 (1.287) Prec@1 55.47 (57.22) Prec@5 96.88 (93.65) VALID-SEARCH [2018-10-02-18:51:46] Epoch: [178][195/196] Loss 0.839 (1.343) Prec@1 70.00 (55.80) Prec@5 97.50 (93.08) 178/250, Train-Accuracy = 66.72, Test-Accuracy = 55.80 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-18:51:47] [Epoch=179/250] [Need: 02:23:32] [LR=0.0101 ~ 0.0101] [Batch=128], tau=5.704000000000001 genotype = Genotype(normal=[('sep_conv_3x3', 1, 0.1294163018465042), ('skip_connect', 0, 0.1287020593881607), ('sep_conv_3x3', 2, 0.12896031141281128), ('dil_conv_5x5', 0, 0.12759515643119812), ('skip_connect', 0, 0.1282431185245514), ('sep_conv_3x3', 3, 0.1282043606042862), ('sep_conv_5x5', 4, 0.1275826096534729), ('dil_conv_5x5', 3, 0.12741070985794067)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.1264040172100067), ('dil_conv_3x3', 1, 0.1258097141981125), ('max_pool_3x3', 1, 0.1270565241575241), ('dil_conv_5x5', 2, 0.12640468776226044), ('avg_pool_3x3', 0, 0.12690913677215576), ('max_pool_3x3', 1, 0.12597331404685974), ('dil_conv_3x3', 4, 0.1263723373413086), ('dil_conv_5x5', 1, 0.12584595382213593)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-18:51:49] Epoch: [179][000/196] Time 1.55 (1.55) Data 0.84 (0.84) Loss 0.893 (0.893) Prec@1 72.66 (72.66) Prec@5 95.31 (95.31) TRAIN-SEARCH [2018-10-02-18:52:46] Epoch: [179][100/196] Time 0.52 (0.58) Data 0.00 (0.01) Loss 0.916 (0.924) Prec@1 70.31 (67.76) Prec@5 97.66 (97.05) TRAIN-SEARCH [2018-10-02-18:53:34] Epoch: [179][195/196] Time 0.47 (0.54) Data 0.00 (0.01) Loss 1.100 (0.927) Prec@1 55.00 (67.39) Prec@5 97.50 (97.11) VALID-SEARCH [2018-10-02-18:53:35] Epoch: [179][000/196] Loss 1.745 (1.745) Prec@1 42.97 (42.97) Prec@5 94.53 (94.53) VALID-SEARCH [2018-10-02-18:53:41] Epoch: [179][100/196] Loss 2.106 (1.417) Prec@1 35.16 (53.33) Prec@5 72.66 (92.12) VALID-SEARCH [2018-10-02-18:53:46] Epoch: [179][195/196] Loss 1.728 (1.412) Prec@1 35.00 (52.95) Prec@5 90.00 (91.76) 179/250, Train-Accuracy = 67.39, Test-Accuracy = 52.95 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-18:53:47] [Epoch=180/250] [Need: 02:19:42] [LR=0.0099 ~ 0.0099] [Batch=128], tau=5.68 genotype = Genotype(normal=[('sep_conv_3x3', 1, 0.1293511688709259), ('skip_connect', 0, 0.12923239171504974), ('sep_conv_3x3', 2, 0.1291058212518692), ('dil_conv_5x5', 0, 0.12702614068984985), ('sep_conv_3x3', 3, 0.12776094675064087), ('skip_connect', 0, 0.12755750119686127), ('dil_conv_3x3', 4, 0.12822572886943817), ('skip_connect', 1, 0.12725451588630676)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.12597452104091644), ('dil_conv_3x3', 1, 0.12556244432926178), ('avg_pool_3x3', 0, 0.12608294188976288), ('max_pool_3x3', 1, 0.12598349153995514), ('avg_pool_3x3', 0, 0.12650971114635468), ('max_pool_3x3', 1, 0.1260104477405548), ('dil_conv_3x3', 4, 0.1264023780822754), ('max_pool_3x3', 0, 0.12619680166244507)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-18:53:49] Epoch: [180][000/196] Time 1.34 (1.34) Data 0.80 (0.80) Loss 1.088 (1.088) Prec@1 65.62 (65.62) Prec@5 92.97 (92.97) TRAIN-SEARCH [2018-10-02-18:54:44] Epoch: [180][100/196] Time 0.45 (0.56) Data 0.00 (0.01) Loss 0.823 (0.932) Prec@1 69.53 (66.99) Prec@5 97.66 (97.18) TRAIN-SEARCH [2018-10-02-18:55:35] Epoch: [180][195/196] Time 0.43 (0.55) Data 0.00 (0.01) Loss 0.831 (0.940) Prec@1 77.50 (67.22) Prec@5 97.50 (97.26) VALID-SEARCH [2018-10-02-18:55:36] Epoch: [180][000/196] Loss 1.051 (1.051) Prec@1 64.06 (64.06) Prec@5 97.66 (97.66) VALID-SEARCH [2018-10-02-18:55:41] Epoch: [180][100/196] Loss 1.922 (1.458) Prec@1 42.19 (52.74) Prec@5 80.47 (92.10) VALID-SEARCH [2018-10-02-18:55:46] Epoch: [180][195/196] Loss 1.316 (1.447) Prec@1 50.00 (52.80) Prec@5 100.00 (91.97) 180/250, Train-Accuracy = 67.22, Test-Accuracy = 52.80 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-18:55:47] [Epoch=181/250] [Need: 02:18:29] [LR=0.0096 ~ 0.0096] [Batch=128], tau=5.656000000000001 genotype = Genotype(normal=[('skip_connect', 0, 0.12912717461585999), ('sep_conv_3x3', 1, 0.12897202372550964), ('sep_conv_3x3', 2, 0.12885482609272003), ('skip_connect', 0, 0.12787280976772308), ('skip_connect', 0, 0.12790271639823914), ('skip_connect', 1, 0.12778519093990326), ('dil_conv_3x3', 4, 0.12814711034297943), ('dil_conv_3x3', 2, 0.12718529999256134)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.1261923909187317), ('skip_connect', 1, 0.1255357712507248), ('max_pool_3x3', 0, 0.12608177959918976), ('sep_conv_3x3', 2, 0.1259750872850418), ('max_pool_3x3', 0, 0.12620757520198822), ('dil_conv_5x5', 2, 0.1259799748659134), ('sep_conv_5x5', 0, 0.12621846795082092), ('dil_conv_3x3', 2, 0.12599895894527435)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-18:55:49] Epoch: [181][000/196] Time 1.43 (1.43) Data 0.91 (0.91) Loss 0.861 (0.861) Prec@1 66.41 (66.41) Prec@5 96.88 (96.88) TRAIN-SEARCH [2018-10-02-18:56:44] Epoch: [181][100/196] Time 0.58 (0.56) Data 0.00 (0.01) Loss 1.013 (0.935) Prec@1 67.19 (66.87) Prec@5 97.66 (97.15) TRAIN-SEARCH [2018-10-02-18:57:34] Epoch: [181][195/196] Time 0.41 (0.54) Data 0.00 (0.01) Loss 0.906 (0.929) Prec@1 70.00 (67.12) Prec@5 100.00 (97.22) VALID-SEARCH [2018-10-02-18:57:35] Epoch: [181][000/196] Loss 1.677 (1.677) Prec@1 42.97 (42.97) Prec@5 84.38 (84.38) VALID-SEARCH [2018-10-02-18:57:41] Epoch: [181][100/196] Loss 0.981 (1.318) Prec@1 62.50 (56.11) Prec@5 96.09 (93.43) VALID-SEARCH [2018-10-02-18:57:46] Epoch: [181][195/196] Loss 1.051 (1.375) Prec@1 65.00 (54.78) Prec@5 95.00 (93.11) 181/250, Train-Accuracy = 67.12, Test-Accuracy = 54.78 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-18:57:47] [Epoch=182/250] [Need: 02:15:30] [LR=0.0094 ~ 0.0094] [Batch=128], tau=5.632 genotype = Genotype(normal=[('skip_connect', 0, 0.12955893576145172), ('sep_conv_3x3', 1, 0.12867623567581177), ('sep_conv_3x3', 2, 0.12891511619091034), ('skip_connect', 0, 0.1283564567565918), ('skip_connect', 1, 0.12749814987182617), ('sep_conv_5x5', 2, 0.12743797898292542), ('dil_conv_3x3', 4, 0.12798112630844116), ('max_pool_3x3', 1, 0.12745483219623566)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.12637756764888763), ('skip_connect', 1, 0.12576530873775482), ('skip_connect', 0, 0.12664027512073517), ('sep_conv_5x5', 2, 0.12589840590953827), ('max_pool_3x3', 0, 0.12633603811264038), ('dil_conv_5x5', 1, 0.12578998506069183), ('sep_conv_3x3', 4, 0.12595604360103607), ('sep_conv_5x5', 0, 0.12579239904880524)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-18:57:49] Epoch: [182][000/196] Time 1.42 (1.42) Data 0.89 (0.89) Loss 0.902 (0.902) Prec@1 69.53 (69.53) Prec@5 98.44 (98.44) TRAIN-SEARCH [2018-10-02-18:58:44] Epoch: [182][100/196] Time 0.57 (0.56) Data 0.00 (0.01) Loss 1.108 (0.921) Prec@1 58.59 (67.44) Prec@5 99.22 (97.31) TRAIN-SEARCH [2018-10-02-18:59:36] Epoch: [182][195/196] Time 0.51 (0.56) Data 0.00 (0.01) Loss 0.993 (0.933) Prec@1 70.00 (67.08) Prec@5 92.50 (97.18) VALID-SEARCH [2018-10-02-18:59:38] Epoch: [182][000/196] Loss 1.120 (1.120) Prec@1 60.16 (60.16) Prec@5 96.09 (96.09) VALID-SEARCH [2018-10-02-18:59:43] Epoch: [182][100/196] Loss 1.855 (1.453) Prec@1 52.34 (52.77) Prec@5 91.41 (92.29) VALID-SEARCH [2018-10-02-18:59:49] Epoch: [182][195/196] Loss 1.545 (1.465) Prec@1 60.00 (53.45) Prec@5 90.00 (92.62) 182/250, Train-Accuracy = 67.08, Test-Accuracy = 53.45 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-18:59:50] [Epoch=183/250] [Need: 02:17:38] [LR=0.0092 ~ 0.0092] [Batch=128], tau=5.6080000000000005 genotype = Genotype(normal=[('skip_connect', 0, 0.1296616941690445), ('sep_conv_3x3', 1, 0.1276068538427353), ('sep_conv_3x3', 2, 0.1297985315322876), ('skip_connect', 0, 0.12881404161453247), ('dil_conv_3x3', 3, 0.1275215446949005), ('sep_conv_5x5', 2, 0.12744542956352234), ('dil_conv_3x3', 4, 0.12809652090072632), ('dil_conv_3x3', 2, 0.12711697816848755)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.12673546373844147), ('dil_conv_3x3', 1, 0.12558753788471222), ('avg_pool_3x3', 0, 0.12637315690517426), ('sep_conv_3x3', 2, 0.12600462138652802), ('sep_conv_5x5', 0, 0.1268669217824936), ('skip_connect', 1, 0.1262940615415573), ('max_pool_3x3', 0, 0.1261383444070816), ('dil_conv_3x3', 4, 0.12610508501529694)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-18:59:52] Epoch: [183][000/196] Time 1.58 (1.58) Data 0.95 (0.95) Loss 0.883 (0.883) Prec@1 65.62 (65.62) Prec@5 97.66 (97.66) TRAIN-SEARCH [2018-10-02-19:00:48] Epoch: [183][100/196] Time 0.40 (0.56) Data 0.00 (0.01) Loss 0.748 (0.911) Prec@1 68.75 (67.65) Prec@5 98.44 (97.25) TRAIN-SEARCH [2018-10-02-19:01:38] Epoch: [183][195/196] Time 0.38 (0.55) Data 0.00 (0.01) Loss 1.049 (0.918) Prec@1 62.50 (67.60) Prec@5 92.50 (97.36) VALID-SEARCH [2018-10-02-19:01:40] Epoch: [183][000/196] Loss 1.179 (1.179) Prec@1 53.12 (53.12) Prec@5 92.97 (92.97) VALID-SEARCH [2018-10-02-19:01:45] Epoch: [183][100/196] Loss 1.091 (1.404) Prec@1 60.16 (54.77) Prec@5 97.66 (92.75) VALID-SEARCH [2018-10-02-19:01:49] Epoch: [183][195/196] Loss 0.914 (1.372) Prec@1 70.00 (55.74) Prec@5 95.00 (93.30) 183/250, Train-Accuracy = 67.60, Test-Accuracy = 55.74 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-19:01:50] [Epoch=184/250] [Need: 02:12:09] [LR=0.0090 ~ 0.0090] [Batch=128], tau=5.584 genotype = Genotype(normal=[('skip_connect', 0, 0.12913954257965088), ('dil_conv_3x3', 1, 0.12757688760757446), ('skip_connect', 0, 0.12946613132953644), ('sep_conv_3x3', 2, 0.12942978739738464), ('sep_conv_5x5', 2, 0.1277078539133072), ('sep_conv_3x3', 3, 0.1276927888393402), ('dil_conv_3x3', 4, 0.12780508399009705), ('sep_conv_5x5', 2, 0.12725943326950073)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.12687455117702484), ('dil_conv_3x3', 1, 0.12589290738105774), ('max_pool_3x3', 0, 0.12628760933876038), ('dil_conv_5x5', 2, 0.1258174031972885), ('sep_conv_5x5', 0, 0.1269131600856781), ('dil_conv_5x5', 1, 0.126686230301857), ('avg_pool_3x3', 1, 0.12665820121765137), ('skip_connect', 0, 0.12595345079898834)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-19:01:52] Epoch: [184][000/196] Time 1.29 (1.29) Data 0.77 (0.77) Loss 0.994 (0.994) Prec@1 62.50 (62.50) Prec@5 98.44 (98.44) TRAIN-SEARCH [2018-10-02-19:02:46] Epoch: [184][100/196] Time 0.43 (0.54) Data 0.00 (0.01) Loss 0.815 (0.914) Prec@1 67.97 (68.09) Prec@5 98.44 (97.55) TRAIN-SEARCH [2018-10-02-19:03:36] Epoch: [184][195/196] Time 0.37 (0.54) Data 0.00 (0.01) Loss 1.051 (0.909) Prec@1 67.50 (68.11) Prec@5 92.50 (97.55) VALID-SEARCH [2018-10-02-19:03:37] Epoch: [184][000/196] Loss 1.376 (1.376) Prec@1 46.88 (46.88) Prec@5 92.97 (92.97) VALID-SEARCH [2018-10-02-19:03:42] Epoch: [184][100/196] Loss 1.093 (1.322) Prec@1 64.06 (57.90) Prec@5 95.31 (94.20) VALID-SEARCH [2018-10-02-19:03:46] Epoch: [184][195/196] Loss 1.035 (1.340) Prec@1 67.50 (57.52) Prec@5 97.50 (94.28) 184/250, Train-Accuracy = 68.11, Test-Accuracy = 57.52 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-19:03:47] [Epoch=185/250] [Need: 02:06:35] [LR=0.0087 ~ 0.0087] [Batch=128], tau=5.5600000000000005 genotype = Genotype(normal=[('skip_connect', 0, 0.13019539415836334), ('sep_conv_3x3', 1, 0.12724348902702332), ('sep_conv_3x3', 2, 0.13010068237781525), ('skip_connect', 0, 0.1294027864933014), ('sep_conv_3x3', 3, 0.12800244987010956), ('sep_conv_5x5', 2, 0.1273389756679535), ('dil_conv_3x3', 4, 0.1277533620595932), ('skip_connect', 1, 0.12736955285072327)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0, 0.12641985714435577), ('dil_conv_3x3', 1, 0.12574324011802673), ('max_pool_3x3', 0, 0.12683439254760742), ('dil_conv_5x5', 2, 0.1259590983390808), ('sep_conv_5x5', 0, 0.126662939786911), ('sep_conv_3x3', 3, 0.12597814202308655), ('sep_conv_3x3', 2, 0.12634798884391785), ('avg_pool_3x3', 1, 0.12631188333034515)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-19:03:49] Epoch: [185][000/196] Time 1.45 (1.45) Data 0.79 (0.79) Loss 0.828 (0.828) Prec@1 66.41 (66.41) Prec@5 100.00 (100.00) TRAIN-SEARCH [2018-10-02-19:04:44] Epoch: [185][100/196] Time 0.39 (0.55) Data 0.00 (0.01) Loss 0.753 (0.902) Prec@1 74.22 (67.91) Prec@5 96.88 (97.52) TRAIN-SEARCH [2018-10-02-19:05:36] Epoch: [185][195/196] Time 0.54 (0.55) Data 0.00 (0.01) Loss 0.708 (0.903) Prec@1 75.00 (68.26) Prec@5 97.50 (97.48) VALID-SEARCH [2018-10-02-19:05:37] Epoch: [185][000/196] Loss 1.162 (1.162) Prec@1 60.94 (60.94) Prec@5 98.44 (98.44) VALID-SEARCH [2018-10-02-19:05:43] Epoch: [185][100/196] Loss 1.384 (1.374) Prec@1 49.22 (54.61) Prec@5 97.66 (92.13) VALID-SEARCH [2018-10-02-19:05:48] Epoch: [185][195/196] Loss 1.426 (1.361) Prec@1 55.00 (54.92) Prec@5 97.50 (92.55) 185/250, Train-Accuracy = 68.26, Test-Accuracy = 54.92 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-19:05:49] [Epoch=186/250] [Need: 02:10:05] [LR=0.0085 ~ 0.0085] [Batch=128], tau=5.536 genotype = Genotype(normal=[('skip_connect', 0, 0.12945689260959625), ('sep_conv_3x3', 1, 0.12743590772151947), ('sep_conv_3x3', 2, 0.1298479437828064), ('skip_connect', 0, 0.12901753187179565), ('sep_conv_5x5', 2, 0.12865306437015533), ('sep_conv_3x3', 3, 0.12768438458442688), ('sep_conv_5x5', 4, 0.12774822115898132), ('skip_connect', 1, 0.1276588886976242)], normal_concat=range(2, 6), reduce=[('skip_connect', 0, 0.12609559297561646), ('dil_conv_5x5', 1, 0.1260407567024231), ('max_pool_3x3', 0, 0.12661625444889069), ('dil_conv_5x5', 2, 0.12621520459651947), ('sep_conv_3x3', 0, 0.12638431787490845), ('sep_conv_3x3', 3, 0.1262398064136505), ('dil_conv_3x3', 0, 0.12634436786174774), ('dil_conv_5x5', 1, 0.12606701254844666)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-19:05:51] Epoch: [186][000/196] Time 1.42 (1.42) Data 0.82 (0.82) Loss 0.985 (0.985) Prec@1 67.97 (67.97) Prec@5 92.97 (92.97) TRAIN-SEARCH [2018-10-02-19:06:44] Epoch: [186][100/196] Time 0.54 (0.54) Data 0.01 (0.01) Loss 0.859 (0.918) Prec@1 65.62 (67.38) Prec@5 99.22 (97.34) TRAIN-SEARCH [2018-10-02-19:07:37] Epoch: [186][195/196] Time 0.40 (0.55) Data 0.00 (0.01) Loss 1.095 (0.904) Prec@1 55.00 (68.09) Prec@5 100.00 (97.49) VALID-SEARCH [2018-10-02-19:07:38] Epoch: [186][000/196] Loss 1.244 (1.244) Prec@1 54.69 (54.69) Prec@5 89.84 (89.84) VALID-SEARCH [2018-10-02-19:07:44] Epoch: [186][100/196] Loss 1.856 (1.400) Prec@1 35.16 (52.64) Prec@5 89.06 (92.00) VALID-SEARCH [2018-10-02-19:07:49] Epoch: [186][195/196] Loss 2.941 (1.399) Prec@1 20.00 (53.52) Prec@5 62.50 (92.02) 186/250, Train-Accuracy = 68.09, Test-Accuracy = 53.52 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-19:07:49] [Epoch=187/250] [Need: 02:06:29] [LR=0.0083 ~ 0.0083] [Batch=128], tau=5.5120000000000005 genotype = Genotype(normal=[('sep_conv_5x5', 0, 0.1301417499780655), ('sep_conv_3x3', 1, 0.12821388244628906), ('sep_conv_3x3', 2, 0.13010720908641815), ('skip_connect', 0, 0.1291438788175583), ('sep_conv_5x5', 2, 0.12831231951713562), ('skip_connect', 0, 0.12747697532176971), ('skip_connect', 1, 0.12785030901432037), ('sep_conv_5x5', 2, 0.1274746060371399)], normal_concat=range(2, 6), reduce=[('skip_connect', 0, 0.12643718719482422), ('dil_conv_5x5', 1, 0.12598693370819092), ('max_pool_3x3', 0, 0.127162903547287), ('dil_conv_5x5', 1, 0.12595099210739136), ('sep_conv_5x5', 0, 0.1262182891368866), ('sep_conv_3x3', 3, 0.12606973946094513), ('max_pool_3x3', 0, 0.12666304409503937), ('dil_conv_3x3', 3, 0.12630826234817505)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-19:07:51] Epoch: [187][000/196] Time 1.41 (1.41) Data 0.77 (0.77) Loss 0.785 (0.785) Prec@1 71.09 (71.09) Prec@5 97.66 (97.66) TRAIN-SEARCH [2018-10-02-19:08:44] Epoch: [187][100/196] Time 0.53 (0.54) Data 0.00 (0.01) Loss 0.874 (0.897) Prec@1 70.31 (68.21) Prec@5 96.88 (97.51) TRAIN-SEARCH [2018-10-02-19:09:37] Epoch: [187][195/196] Time 0.59 (0.54) Data 0.00 (0.01) Loss 0.901 (0.900) Prec@1 72.50 (68.27) Prec@5 97.50 (97.53) VALID-SEARCH [2018-10-02-19:09:38] Epoch: [187][000/196] Loss 1.166 (1.166) Prec@1 56.25 (56.25) Prec@5 96.88 (96.88) VALID-SEARCH [2018-10-02-19:09:44] Epoch: [187][100/196] Loss 0.969 (1.330) Prec@1 64.84 (56.32) Prec@5 98.44 (94.04) VALID-SEARCH [2018-10-02-19:09:49] Epoch: [187][195/196] Loss 1.026 (1.349) Prec@1 60.00 (55.73) Prec@5 100.00 (93.16) 187/250, Train-Accuracy = 68.27, Test-Accuracy = 55.73 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-19:09:50] [Epoch=188/250] [Need: 02:04:11] [LR=0.0081 ~ 0.0081] [Batch=128], tau=5.4879999999999995 genotype = Genotype(normal=[('sep_conv_5x5', 0, 0.13061997294425964), ('sep_conv_3x3', 1, 0.1279505491256714), ('sep_conv_3x3', 2, 0.12970921397209167), ('skip_connect', 0, 0.12893792986869812), ('sep_conv_5x5', 2, 0.12844787538051605), ('sep_conv_5x5', 1, 0.1274389773607254), ('skip_connect', 1, 0.12778596580028534), ('sep_conv_3x3', 4, 0.12745720148086548)], normal_concat=range(2, 6), reduce=[('skip_connect', 0, 0.12597514688968658), ('sep_conv_3x3', 1, 0.12576492130756378), ('max_pool_3x3', 0, 0.12717273831367493), ('max_pool_3x3', 1, 0.1261974275112152), ('sep_conv_5x5', 2, 0.12614959478378296), ('avg_pool_3x3', 1, 0.12588773667812347), ('dil_conv_5x5', 1, 0.1269252747297287), ('max_pool_3x3', 0, 0.1267906129360199)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-19:09:52] Epoch: [188][000/196] Time 1.46 (1.46) Data 0.75 (0.75) Loss 0.822 (0.822) Prec@1 74.22 (74.22) Prec@5 96.09 (96.09) TRAIN-SEARCH [2018-10-02-19:10:44] Epoch: [188][100/196] Time 0.43 (0.53) Data 0.00 (0.01) Loss 1.142 (0.894) Prec@1 60.16 (68.72) Prec@5 98.44 (97.56) TRAIN-SEARCH [2018-10-02-19:11:38] Epoch: [188][195/196] Time 0.45 (0.55) Data 0.00 (0.01) Loss 0.959 (0.904) Prec@1 62.50 (68.41) Prec@5 100.00 (97.54) VALID-SEARCH [2018-10-02-19:11:39] Epoch: [188][000/196] Loss 1.299 (1.299) Prec@1 53.12 (53.12) Prec@5 96.88 (96.88) VALID-SEARCH [2018-10-02-19:11:44] Epoch: [188][100/196] Loss 0.947 (1.331) Prec@1 67.19 (54.76) Prec@5 97.66 (93.18) VALID-SEARCH [2018-10-02-19:11:49] Epoch: [188][195/196] Loss 1.709 (1.327) Prec@1 67.50 (55.16) Prec@5 92.50 (93.08) 188/250, Train-Accuracy = 68.41, Test-Accuracy = 55.16 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-19:11:50] [Epoch=189/250] [Need: 02:02:18] [LR=0.0079 ~ 0.0079] [Batch=128], tau=5.464 genotype = Genotype(normal=[('sep_conv_5x5', 0, 0.13064788281917572), ('sep_conv_3x3', 1, 0.12812279164791107), ('sep_conv_3x3', 2, 0.1291927993297577), ('skip_connect', 0, 0.12839634716510773), ('sep_conv_5x5', 2, 0.1276484578847885), ('dil_conv_3x3', 3, 0.1276203840970993), ('skip_connect', 1, 0.12753619253635406), ('sep_conv_5x5', 2, 0.12738677859306335)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.12651264667510986), ('dil_conv_5x5', 1, 0.12582722306251526), ('max_pool_3x3', 0, 0.12680472433567047), ('max_pool_3x3', 1, 0.1260991096496582), ('max_pool_3x3', 1, 0.12645819783210754), ('max_pool_3x3', 0, 0.12617015838623047), ('dil_conv_5x5', 1, 0.12647321820259094), ('dil_conv_3x3', 4, 0.12642502784729004)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-19:11:52] Epoch: [189][000/196] Time 1.52 (1.52) Data 0.91 (0.91) Loss 0.975 (0.975) Prec@1 66.41 (66.41) Prec@5 96.09 (96.09) TRAIN-SEARCH [2018-10-02-19:12:44] Epoch: [189][100/196] Time 0.52 (0.53) Data 0.00 (0.01) Loss 0.915 (0.881) Prec@1 67.19 (68.93) Prec@5 97.66 (97.52) TRAIN-SEARCH [2018-10-02-19:13:40] Epoch: [189][195/196] Time 0.67 (0.56) Data 0.00 (0.01) Loss 0.895 (0.894) Prec@1 70.00 (68.56) Prec@5 97.50 (97.47) VALID-SEARCH [2018-10-02-19:13:41] Epoch: [189][000/196] Loss 1.673 (1.673) Prec@1 60.16 (60.16) Prec@5 94.53 (94.53) VALID-SEARCH [2018-10-02-19:13:47] Epoch: [189][100/196] Loss 1.276 (1.398) Prec@1 58.59 (54.59) Prec@5 95.31 (92.11) VALID-SEARCH [2018-10-02-19:13:52] Epoch: [189][195/196] Loss 1.046 (1.340) Prec@1 60.00 (55.66) Prec@5 97.50 (93.00) 189/250, Train-Accuracy = 68.56, Test-Accuracy = 55.66 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-19:13:53] [Epoch=190/250] [Need: 02:02:50] [LR=0.0076 ~ 0.0076] [Batch=128], tau=5.4399999999999995 genotype = Genotype(normal=[('sep_conv_5x5', 0, 0.12985321879386902), ('sep_conv_5x5', 1, 0.12711133062839508), ('sep_conv_3x3', 2, 0.1296461820602417), ('skip_connect', 0, 0.12851528823375702), ('sep_conv_5x5', 1, 0.12790454924106598), ('sep_conv_3x3', 3, 0.12720495462417603), ('sep_conv_5x5', 4, 0.12820762395858765), ('skip_connect', 1, 0.1273542195558548)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.1261208951473236), ('skip_connect', 1, 0.1259801834821701), ('max_pool_3x3', 0, 0.1270332932472229), ('dil_conv_5x5', 2, 0.126225084066391), ('max_pool_3x3', 1, 0.12633079290390015), ('avg_pool_3x3', 0, 0.12616531550884247), ('max_pool_3x3', 0, 0.12651650607585907), ('dil_conv_3x3', 4, 0.12635457515716553)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-19:13:55] Epoch: [190][000/196] Time 1.65 (1.65) Data 1.00 (1.00) Loss 0.984 (0.984) Prec@1 71.09 (71.09) Prec@5 96.88 (96.88) TRAIN-SEARCH [2018-10-02-19:14:47] Epoch: [190][100/196] Time 0.56 (0.53) Data 0.00 (0.01) Loss 0.937 (0.891) Prec@1 71.09 (68.72) Prec@5 97.66 (97.52) TRAIN-SEARCH [2018-10-02-19:15:41] Epoch: [190][195/196] Time 0.40 (0.55) Data 0.00 (0.01) Loss 0.655 (0.893) Prec@1 77.50 (68.87) Prec@5 100.00 (97.47) VALID-SEARCH [2018-10-02-19:15:42] Epoch: [190][000/196] Loss 1.800 (1.800) Prec@1 41.41 (41.41) Prec@5 82.03 (82.03) VALID-SEARCH [2018-10-02-19:15:48] Epoch: [190][100/196] Loss 1.142 (1.270) Prec@1 66.41 (57.87) Prec@5 95.31 (94.06) VALID-SEARCH [2018-10-02-19:15:53] Epoch: [190][195/196] Loss 1.392 (1.282) Prec@1 42.50 (57.48) Prec@5 92.50 (94.10) 190/250, Train-Accuracy = 68.87, Test-Accuracy = 57.48 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-19:15:54] [Epoch=191/250] [Need: 01:59:10] [LR=0.0074 ~ 0.0074] [Batch=128], tau=5.416 genotype = Genotype(normal=[('sep_conv_5x5', 0, 0.12989157438278198), ('sep_conv_5x5', 1, 0.12654514610767365), ('sep_conv_3x3', 2, 0.1303364783525467), ('skip_connect', 0, 0.12744976580142975), ('sep_conv_3x3', 3, 0.12799017131328583), ('sep_conv_5x5', 1, 0.12781205773353577), ('sep_conv_3x3', 3, 0.12776777148246765), ('sep_conv_5x5', 4, 0.12761038541793823)], normal_concat=range(2, 6), reduce=[('skip_connect', 0, 0.12611986696720123), ('skip_connect', 1, 0.12547646462917328), ('dil_conv_5x5', 2, 0.12688729166984558), ('avg_pool_3x3', 0, 0.12665095925331116), ('avg_pool_3x3', 0, 0.12708301842212677), ('dil_conv_5x5', 3, 0.1264868974685669), ('avg_pool_3x3', 0, 0.12640774250030518), ('sep_conv_5x5', 4, 0.12633615732192993)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-19:15:56] Epoch: [191][000/196] Time 1.54 (1.54) Data 0.89 (0.89) Loss 0.781 (0.781) Prec@1 68.75 (68.75) Prec@5 97.66 (97.66) TRAIN-SEARCH [2018-10-02-19:16:48] Epoch: [191][100/196] Time 0.52 (0.52) Data 0.00 (0.01) Loss 0.778 (0.895) Prec@1 71.88 (68.60) Prec@5 98.44 (97.46) TRAIN-SEARCH [2018-10-02-19:17:40] Epoch: [191][195/196] Time 0.44 (0.54) Data 0.00 (0.01) Loss 1.244 (0.892) Prec@1 50.00 (68.58) Prec@5 100.00 (97.44) VALID-SEARCH [2018-10-02-19:17:41] Epoch: [191][000/196] Loss 1.200 (1.200) Prec@1 57.81 (57.81) Prec@5 94.53 (94.53) VALID-SEARCH [2018-10-02-19:17:46] Epoch: [191][100/196] Loss 1.258 (1.306) Prec@1 55.47 (56.40) Prec@5 95.31 (92.88) VALID-SEARCH [2018-10-02-19:17:52] Epoch: [191][195/196] Loss 1.614 (1.333) Prec@1 52.50 (56.02) Prec@5 90.00 (92.98) 191/250, Train-Accuracy = 68.58, Test-Accuracy = 56.02 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-19:17:53] [Epoch=192/250] [Need: 01:54:39] [LR=0.0072 ~ 0.0072] [Batch=128], tau=5.3919999999999995 genotype = Genotype(normal=[('sep_conv_5x5', 0, 0.13052484393119812), ('dil_conv_5x5', 1, 0.12702403962612152), ('sep_conv_3x3', 2, 0.13053496181964874), ('skip_connect', 0, 0.12791457772254944), ('sep_conv_3x3', 3, 0.12839913368225098), ('dil_conv_3x3', 2, 0.1273525506258011), ('sep_conv_3x3', 4, 0.12833555042743683), ('sep_conv_3x3', 3, 0.12752962112426758)], normal_concat=range(2, 6), reduce=[('dil_conv_5x5', 0, 0.12624520063400269), ('dil_conv_5x5', 1, 0.12563656270503998), ('dil_conv_5x5', 2, 0.12672609090805054), ('avg_pool_3x3', 0, 0.12619544565677643), ('max_pool_3x3', 0, 0.12714014947414398), ('dil_conv_5x5', 3, 0.12647990882396698), ('avg_pool_3x3', 0, 0.12633688747882843), ('dil_conv_5x5', 1, 0.12609480321407318)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-19:17:55] Epoch: [192][000/196] Time 1.61 (1.61) Data 0.94 (0.94) Loss 0.899 (0.899) Prec@1 66.41 (66.41) Prec@5 98.44 (98.44) TRAIN-SEARCH [2018-10-02-19:18:45] Epoch: [192][100/196] Time 0.55 (0.51) Data 0.01 (0.01) Loss 0.914 (0.885) Prec@1 68.75 (68.95) Prec@5 96.09 (97.57) TRAIN-SEARCH [2018-10-02-19:19:40] Epoch: [192][195/196] Time 0.52 (0.54) Data 0.00 (0.01) Loss 1.112 (0.882) Prec@1 57.50 (68.99) Prec@5 97.50 (97.55) VALID-SEARCH [2018-10-02-19:19:41] Epoch: [192][000/196] Loss 1.448 (1.448) Prec@1 58.59 (58.59) Prec@5 94.53 (94.53) VALID-SEARCH [2018-10-02-19:19:46] Epoch: [192][100/196] Loss 0.929 (1.425) Prec@1 65.62 (55.16) Prec@5 97.66 (92.78) VALID-SEARCH [2018-10-02-19:19:50] Epoch: [192][195/196] Loss 1.331 (1.375) Prec@1 55.00 (56.21) Prec@5 95.00 (93.41) 192/250, Train-Accuracy = 68.99, Test-Accuracy = 56.21 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-19:19:54] [Epoch=193/250] [Need: 01:54:57] [LR=0.0070 ~ 0.0070] [Batch=128], tau=5.368 genotype = Genotype(normal=[('sep_conv_5x5', 0, 0.1300065517425537), ('dil_conv_5x5', 1, 0.12640610337257385), ('sep_conv_3x3', 2, 0.12995736300945282), ('skip_connect', 1, 0.1281752735376358), ('skip_connect', 0, 0.12770524621009827), ('sep_conv_3x3', 3, 0.12750095129013062), ('sep_conv_3x3', 4, 0.12783701717853546), ('max_pool_3x3', 0, 0.12771812081336975)], normal_concat=range(2, 6), reduce=[('dil_conv_5x5', 0, 0.12616123259067535), ('dil_conv_5x5', 1, 0.1258782148361206), ('avg_pool_3x3', 0, 0.12715879082679749), ('dil_conv_5x5', 2, 0.12675872445106506), ('max_pool_3x3', 0, 0.1266881823539734), ('dil_conv_3x3', 3, 0.12595820426940918), ('skip_connect', 0, 0.1262340545654297), ('sep_conv_3x3', 3, 0.12601503729820251)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-19:19:56] Epoch: [193][000/196] Time 1.54 (1.54) Data 0.93 (0.93) Loss 0.980 (0.980) Prec@1 71.88 (71.88) Prec@5 97.66 (97.66) TRAIN-SEARCH [2018-10-02-19:20:49] Epoch: [193][100/196] Time 0.60 (0.54) Data 0.02 (0.01) Loss 0.943 (0.902) Prec@1 67.97 (68.71) Prec@5 99.22 (97.60) TRAIN-SEARCH [2018-10-02-19:21:43] Epoch: [193][195/196] Time 0.39 (0.55) Data 0.00 (0.01) Loss 0.786 (0.893) Prec@1 75.00 (68.82) Prec@5 100.00 (97.70) VALID-SEARCH [2018-10-02-19:21:44] Epoch: [193][000/196] Loss 1.014 (1.014) Prec@1 60.94 (60.94) Prec@5 96.88 (96.88) VALID-SEARCH [2018-10-02-19:21:49] Epoch: [193][100/196] Loss 1.113 (1.345) Prec@1 58.59 (55.41) Prec@5 96.09 (92.83) VALID-SEARCH [2018-10-02-19:21:54] Epoch: [193][195/196] Loss 1.346 (1.331) Prec@1 60.00 (55.37) Prec@5 90.00 (92.92) 193/250, Train-Accuracy = 68.82, Test-Accuracy = 55.37 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-19:21:55] [Epoch=194/250] [Need: 01:53:08] [LR=0.0068 ~ 0.0068] [Batch=128], tau=5.343999999999999 genotype = Genotype(normal=[('sep_conv_5x5', 0, 0.12927527725696564), ('sep_conv_3x3', 1, 0.12692727148532867), ('sep_conv_3x3', 2, 0.12948526442050934), ('skip_connect', 1, 0.12812183797359467), ('skip_connect', 0, 0.12755700945854187), ('sep_conv_5x5', 2, 0.1275440901517868), ('sep_conv_3x3', 4, 0.1282290816307068), ('max_pool_3x3', 0, 0.12755419313907623)], normal_concat=range(2, 6), reduce=[('dil_conv_5x5', 0, 0.1265525370836258), ('sep_conv_5x5', 1, 0.12571598589420319), ('max_pool_3x3', 0, 0.1265171766281128), ('dil_conv_5x5', 2, 0.1260504126548767), ('avg_pool_3x3', 0, 0.12636077404022217), ('dil_conv_5x5', 3, 0.1258678436279297), ('dil_conv_5x5', 4, 0.1264840066432953), ('dil_conv_5x5', 1, 0.1260613650083542)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-19:21:57] Epoch: [194][000/196] Time 1.68 (1.68) Data 1.11 (1.11) Loss 0.788 (0.788) Prec@1 74.22 (74.22) Prec@5 99.22 (99.22) TRAIN-SEARCH [2018-10-02-19:22:51] Epoch: [194][100/196] Time 0.72 (0.55) Data 0.00 (0.01) Loss 0.849 (0.884) Prec@1 71.88 (68.63) Prec@5 97.66 (97.54) TRAIN-SEARCH [2018-10-02-19:23:42] Epoch: [194][195/196] Time 0.54 (0.54) Data 0.00 (0.01) Loss 0.786 (0.889) Prec@1 67.50 (68.62) Prec@5 100.00 (97.47) VALID-SEARCH [2018-10-02-19:23:43] Epoch: [194][000/196] Loss 1.008 (1.008) Prec@1 71.09 (71.09) Prec@5 95.31 (95.31) VALID-SEARCH [2018-10-02-19:23:48] Epoch: [194][100/196] Loss 1.079 (1.266) Prec@1 66.41 (58.71) Prec@5 95.31 (94.46) VALID-SEARCH [2018-10-02-19:23:53] Epoch: [194][195/196] Loss 1.156 (1.295) Prec@1 65.00 (57.49) Prec@5 95.00 (93.93) 194/250, Train-Accuracy = 68.62, Test-Accuracy = 57.49 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-19:23:54] [Epoch=195/250] [Need: 01:49:18] [LR=0.0066 ~ 0.0066] [Batch=128], tau=5.32 genotype = Genotype(normal=[('sep_conv_5x5', 0, 0.12983116507530212), ('sep_conv_3x3', 1, 0.12704546749591827), ('sep_conv_3x3', 2, 0.1288098692893982), ('skip_connect', 0, 0.12837114930152893), ('sep_conv_5x5', 2, 0.12776200473308563), ('dil_conv_3x3', 3, 0.12766984105110168), ('sep_conv_3x3', 4, 0.1288624256849289), ('skip_connect', 0, 0.12723438441753387)], normal_concat=range(2, 6), reduce=[('dil_conv_5x5', 0, 0.12610219419002533), ('sep_conv_5x5', 1, 0.12566882371902466), ('max_pool_3x3', 0, 0.12699736654758453), ('dil_conv_5x5', 2, 0.12618689239025116), ('avg_pool_3x3', 0, 0.1265157163143158), ('dil_conv_5x5', 3, 0.12600716948509216), ('dil_conv_5x5', 4, 0.1265878975391388), ('sep_conv_5x5', 2, 0.12619850039482117)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-19:23:56] Epoch: [195][000/196] Time 1.58 (1.58) Data 0.99 (0.99) Loss 0.832 (0.832) Prec@1 70.31 (70.31) Prec@5 98.44 (98.44) TRAIN-SEARCH [2018-10-02-19:24:50] Epoch: [195][100/196] Time 0.62 (0.55) Data 0.00 (0.01) Loss 0.885 (0.877) Prec@1 67.97 (68.77) Prec@5 96.09 (97.76) TRAIN-SEARCH [2018-10-02-19:25:41] Epoch: [195][195/196] Time 0.48 (0.54) Data 0.00 (0.01) Loss 0.933 (0.882) Prec@1 70.00 (69.14) Prec@5 100.00 (97.62) VALID-SEARCH [2018-10-02-19:25:42] Epoch: [195][000/196] Loss 1.069 (1.069) Prec@1 65.62 (65.62) Prec@5 94.53 (94.53) VALID-SEARCH [2018-10-02-19:25:48] Epoch: [195][100/196] Loss 1.641 (1.188) Prec@1 39.06 (59.97) Prec@5 91.41 (95.17) VALID-SEARCH [2018-10-02-19:25:53] Epoch: [195][195/196] Loss 1.476 (1.231) Prec@1 55.00 (58.48) Prec@5 87.50 (94.66) 195/250, Train-Accuracy = 69.14, Test-Accuracy = 58.48 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-19:25:53] [Epoch=196/250] [Need: 01:47:26] [LR=0.0064 ~ 0.0064] [Batch=128], tau=5.295999999999999 genotype = Genotype(normal=[('sep_conv_5x5', 0, 0.1293078362941742), ('dil_conv_3x3', 1, 0.1278681457042694), ('sep_conv_3x3', 2, 0.12932977080345154), ('skip_connect', 1, 0.12900710105895996), ('dil_conv_3x3', 3, 0.1280449628829956), ('sep_conv_5x5', 2, 0.1277298778295517), ('sep_conv_3x3', 4, 0.1281208097934723), ('skip_connect', 0, 0.12737691402435303)], normal_concat=range(2, 6), reduce=[('sep_conv_5x5', 1, 0.12648573517799377), ('avg_pool_3x3', 0, 0.1260755956172943), ('sep_conv_3x3', 2, 0.12708410620689392), ('avg_pool_3x3', 0, 0.12698109447956085), ('avg_pool_3x3', 0, 0.12610289454460144), ('sep_conv_3x3', 2, 0.12593309581279755), ('dil_conv_5x5', 4, 0.1270073503255844), ('dil_conv_5x5', 1, 0.1262098252773285)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-19:25:56] Epoch: [196][000/196] Time 1.57 (1.57) Data 0.88 (0.88) Loss 0.695 (0.695) Prec@1 73.44 (73.44) Prec@5 99.22 (99.22) TRAIN-SEARCH [2018-10-02-19:26:48] Epoch: [196][100/196] Time 0.49 (0.53) Data 0.00 (0.01) Loss 0.828 (0.872) Prec@1 67.97 (69.84) Prec@5 99.22 (97.73) TRAIN-SEARCH [2018-10-02-19:27:38] Epoch: [196][195/196] Time 0.53 (0.53) Data 0.00 (0.01) Loss 1.048 (0.875) Prec@1 60.00 (69.72) Prec@5 100.00 (97.68) VALID-SEARCH [2018-10-02-19:27:39] Epoch: [196][000/196] Loss 1.284 (1.284) Prec@1 66.41 (66.41) Prec@5 91.41 (91.41) VALID-SEARCH [2018-10-02-19:27:44] Epoch: [196][100/196] Loss 1.096 (1.319) Prec@1 65.62 (58.16) Prec@5 96.09 (93.97) VALID-SEARCH [2018-10-02-19:27:50] Epoch: [196][195/196] Loss 1.469 (1.324) Prec@1 55.00 (57.90) Prec@5 90.00 (93.96) 196/250, Train-Accuracy = 69.72, Test-Accuracy = 57.90 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-19:27:51] [Epoch=197/250] [Need: 01:43:29] [LR=0.0062 ~ 0.0062] [Batch=128], tau=5.272 genotype = Genotype(normal=[('skip_connect', 0, 0.12933193147182465), ('dil_conv_3x3', 1, 0.12860500812530518), ('skip_connect', 1, 0.129953995347023), ('sep_conv_3x3', 2, 0.12937141954898834), ('dil_conv_3x3', 3, 0.1283886730670929), ('sep_conv_5x5', 0, 0.12745442986488342), ('sep_conv_3x3', 4, 0.12797006964683533), ('skip_connect', 0, 0.12788474559783936)], normal_concat=range(2, 6), reduce=[('dil_conv_5x5', 0, 0.12594592571258545), ('dil_conv_5x5', 1, 0.12560877203941345), ('avg_pool_3x3', 0, 0.12754853069782257), ('dil_conv_5x5', 2, 0.126261368393898), ('avg_pool_3x3', 0, 0.12612244486808777), ('dil_conv_3x3', 1, 0.12605567276477814), ('max_pool_3x3', 1, 0.12616650760173798), ('dil_conv_5x5', 2, 0.12615765631198883)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-19:27:53] Epoch: [197][000/196] Time 1.73 (1.73) Data 1.13 (1.13) Loss 0.911 (0.911) Prec@1 67.19 (67.19) Prec@5 96.09 (96.09) TRAIN-SEARCH [2018-10-02-19:28:47] Epoch: [197][100/196] Time 0.46 (0.55) Data 0.00 (0.01) Loss 0.801 (0.872) Prec@1 68.75 (69.36) Prec@5 98.44 (97.67) TRAIN-SEARCH [2018-10-02-19:29:39] Epoch: [197][195/196] Time 0.50 (0.55) Data 0.00 (0.01) Loss 1.153 (0.872) Prec@1 72.50 (69.34) Prec@5 95.00 (97.61) VALID-SEARCH [2018-10-02-19:29:40] Epoch: [197][000/196] Loss 0.846 (0.846) Prec@1 75.00 (75.00) Prec@5 96.88 (96.88) VALID-SEARCH [2018-10-02-19:29:46] Epoch: [197][100/196] Loss 1.130 (1.434) Prec@1 62.50 (53.88) Prec@5 96.09 (91.45) VALID-SEARCH [2018-10-02-19:29:50] Epoch: [197][195/196] Loss 1.333 (1.416) Prec@1 50.00 (53.81) Prec@5 100.00 (91.48) 197/250, Train-Accuracy = 69.34, Test-Accuracy = 53.81 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-19:29:51] [Epoch=198/250] [Need: 01:44:22] [LR=0.0060 ~ 0.0060] [Batch=128], tau=5.247999999999999 genotype = Genotype(normal=[('skip_connect', 0, 0.12886321544647217), ('dil_conv_3x3', 1, 0.12841905653476715), ('sep_conv_3x3', 2, 0.12993872165679932), ('skip_connect', 1, 0.12959517538547516), ('dil_conv_3x3', 3, 0.1284373253583908), ('sep_conv_5x5', 0, 0.12798278033733368), ('sep_conv_3x3', 4, 0.1283247470855713), ('skip_connect', 0, 0.1276879906654358)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.12629063427448273), ('dil_conv_5x5', 1, 0.12591154873371124), ('dil_conv_5x5', 2, 0.1269570291042328), ('avg_pool_3x3', 0, 0.12670952081680298), ('dil_conv_3x3', 1, 0.12619026005268097), ('sep_conv_5x5', 2, 0.12577317655086517), ('dil_conv_5x5', 4, 0.12608230113983154), ('dil_conv_3x3', 3, 0.12596715986728668)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-19:29:53] Epoch: [198][000/196] Time 1.03 (1.03) Data 0.61 (0.61) Loss 0.944 (0.944) Prec@1 67.97 (67.97) Prec@5 97.66 (97.66) TRAIN-SEARCH [2018-10-02-19:30:44] Epoch: [198][100/196] Time 0.75 (0.52) Data 0.00 (0.01) Loss 0.673 (0.870) Prec@1 75.78 (69.77) Prec@5 98.44 (97.50) TRAIN-SEARCH [2018-10-02-19:31:39] Epoch: [198][195/196] Time 0.55 (0.55) Data 0.00 (0.01) Loss 1.002 (0.870) Prec@1 65.00 (69.51) Prec@5 97.50 (97.54) VALID-SEARCH [2018-10-02-19:31:40] Epoch: [198][000/196] Loss 1.968 (1.968) Prec@1 40.62 (40.62) Prec@5 76.56 (76.56) VALID-SEARCH [2018-10-02-19:31:45] Epoch: [198][100/196] Loss 0.857 (1.316) Prec@1 68.75 (56.80) Prec@5 97.66 (93.78) VALID-SEARCH [2018-10-02-19:31:50] Epoch: [198][195/196] Loss 1.139 (1.335) Prec@1 60.00 (56.34) Prec@5 97.50 (93.41) 198/250, Train-Accuracy = 69.51, Test-Accuracy = 56.34 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-19:31:50] [Epoch=199/250] [Need: 01:41:16] [LR=0.0059 ~ 0.0059] [Batch=128], tau=5.224 genotype = Genotype(normal=[('dil_conv_3x3', 0, 0.12882332503795624), ('dil_conv_3x3', 1, 0.12854456901550293), ('skip_connect', 1, 0.13016276061534882), ('sep_conv_3x3', 2, 0.12980987131595612), ('sep_conv_3x3', 3, 0.12836910784244537), ('dil_conv_3x3', 2, 0.12758485972881317), ('skip_connect', 1, 0.12819607555866241), ('sep_conv_3x3', 4, 0.12806187570095062)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.12634682655334473), ('sep_conv_3x3', 1, 0.1258292943239212), ('max_pool_3x3', 0, 0.12670589983463287), ('dil_conv_5x5', 2, 0.12666559219360352), ('sep_conv_3x3', 2, 0.12614692747592926), ('dil_conv_3x3', 3, 0.1260225772857666), ('sep_conv_3x3', 0, 0.12636731564998627), ('dil_conv_5x5', 2, 0.12628240883350372)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-19:31:52] Epoch: [199][000/196] Time 1.01 (1.01) Data 0.55 (0.55) Loss 0.839 (0.839) Prec@1 71.09 (71.09) Prec@5 99.22 (99.22) TRAIN-SEARCH [2018-10-02-19:32:45] Epoch: [199][100/196] Time 0.59 (0.54) Data 0.00 (0.01) Loss 0.770 (0.874) Prec@1 72.66 (69.54) Prec@5 98.44 (97.56) TRAIN-SEARCH [2018-10-02-19:33:39] Epoch: [199][195/196] Time 0.48 (0.55) Data 0.00 (0.01) Loss 1.137 (0.872) Prec@1 75.00 (69.74) Prec@5 92.50 (97.61) VALID-SEARCH [2018-10-02-19:33:40] Epoch: [199][000/196] Loss 1.128 (1.128) Prec@1 63.28 (63.28) Prec@5 95.31 (95.31) VALID-SEARCH [2018-10-02-19:33:44] Epoch: [199][100/196] Loss 1.965 (1.283) Prec@1 53.91 (58.38) Prec@5 96.09 (94.23) VALID-SEARCH [2018-10-02-19:33:48] Epoch: [199][195/196] Loss 1.049 (1.274) Prec@1 55.00 (58.50) Prec@5 100.00 (94.06) 199/250, Train-Accuracy = 69.74, Test-Accuracy = 58.50 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-19:33:49] [Epoch=200/250] [Need: 01:38:53] [LR=0.0057 ~ 0.0057] [Batch=128], tau=5.199999999999999 genotype = Genotype(normal=[('dil_conv_3x3', 0, 0.12884919345378876), ('dil_conv_3x3', 1, 0.12883247435092926), ('sep_conv_3x3', 2, 0.12965688109397888), ('skip_connect', 1, 0.1292121708393097), ('sep_conv_3x3', 3, 0.12882578372955322), ('sep_conv_5x5', 0, 0.12834297120571136), ('skip_connect', 1, 0.1276668906211853), ('dil_conv_3x3', 4, 0.12732663750648499)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.12608639895915985), ('dil_conv_5x5', 1, 0.1257193386554718), ('dil_conv_5x5', 2, 0.12689457833766937), ('avg_pool_3x3', 0, 0.1265067458152771), ('avg_pool_3x3', 0, 0.12648016214370728), ('sep_conv_5x5', 3, 0.12611514329910278), ('sep_conv_3x3', 0, 0.1265270709991455), ('sep_conv_3x3', 3, 0.12612798810005188)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-19:33:50] Epoch: [200][000/196] Time 1.08 (1.08) Data 0.66 (0.66) Loss 0.587 (0.587) Prec@1 78.91 (78.91) Prec@5 99.22 (99.22) TRAIN-SEARCH [2018-10-02-19:34:43] Epoch: [200][100/196] Time 0.59 (0.53) Data 0.00 (0.01) Loss 0.645 (0.846) Prec@1 76.56 (70.13) Prec@5 96.09 (97.92) TRAIN-SEARCH [2018-10-02-19:35:37] Epoch: [200][195/196] Time 0.41 (0.55) Data 0.00 (0.01) Loss 0.812 (0.853) Prec@1 70.00 (69.89) Prec@5 100.00 (97.76) VALID-SEARCH [2018-10-02-19:35:38] Epoch: [200][000/196] Loss 1.000 (1.000) Prec@1 66.41 (66.41) Prec@5 97.66 (97.66) VALID-SEARCH [2018-10-02-19:35:43] Epoch: [200][100/196] Loss 0.934 (1.204) Prec@1 61.72 (59.62) Prec@5 96.88 (94.42) VALID-SEARCH [2018-10-02-19:35:46] Epoch: [200][195/196] Loss 0.906 (1.266) Prec@1 72.50 (57.95) Prec@5 100.00 (93.93) 200/250, Train-Accuracy = 69.89, Test-Accuracy = 57.95 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-19:35:49] [Epoch=201/250] [Need: 01:37:57] [LR=0.0055 ~ 0.0055] [Batch=128], tau=5.176 genotype = Genotype(normal=[('dil_conv_3x3', 1, 0.12830772995948792), ('sep_conv_3x3', 0, 0.12825410068035126), ('sep_conv_3x3', 2, 0.12976092100143433), ('skip_connect', 1, 0.12946030497550964), ('sep_conv_3x3', 3, 0.1286504864692688), ('sep_conv_5x5', 0, 0.12805870175361633), ('skip_connect', 1, 0.12725788354873657), ('sep_conv_3x3', 4, 0.1271500289440155)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.12609824538230896), ('sep_conv_5x5', 1, 0.1258070468902588), ('dil_conv_5x5', 2, 0.12759847939014435), ('avg_pool_3x3', 0, 0.12649600207805634), ('avg_pool_3x3', 0, 0.12617304921150208), ('sep_conv_5x5', 1, 0.12590950727462769), ('sep_conv_3x3', 0, 0.12655502557754517), ('sep_conv_5x5', 4, 0.1264737844467163)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-19:35:51] Epoch: [201][000/196] Time 1.25 (1.25) Data 0.56 (0.56) Loss 0.754 (0.754) Prec@1 70.31 (70.31) Prec@5 98.44 (98.44) TRAIN-SEARCH [2018-10-02-19:36:47] Epoch: [201][100/196] Time 0.64 (0.57) Data 0.00 (0.01) Loss 0.839 (0.864) Prec@1 73.44 (69.91) Prec@5 92.97 (97.59) TRAIN-SEARCH [2018-10-02-19:37:40] Epoch: [201][195/196] Time 0.36 (0.56) Data 0.00 (0.01) Loss 0.770 (0.860) Prec@1 80.00 (69.86) Prec@5 95.00 (97.68) VALID-SEARCH [2018-10-02-19:37:40] Epoch: [201][000/196] Loss 1.324 (1.324) Prec@1 57.03 (57.03) Prec@5 95.31 (95.31) VALID-SEARCH [2018-10-02-19:37:44] Epoch: [201][100/196] Loss 1.726 (1.357) Prec@1 48.44 (54.63) Prec@5 89.84 (93.05) VALID-SEARCH [2018-10-02-19:37:48] Epoch: [201][195/196] Loss 2.409 (1.347) Prec@1 37.50 (55.34) Prec@5 60.00 (92.96) 201/250, Train-Accuracy = 69.86, Test-Accuracy = 55.34 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-19:37:51] [Epoch=202/250] [Need: 01:38:06] [LR=0.0053 ~ 0.0053] [Batch=128], tau=5.151999999999999 genotype = Genotype(normal=[('sep_conv_3x3', 0, 0.12839587032794952), ('dil_conv_3x3', 1, 0.1283474713563919), ('sep_conv_3x3', 2, 0.12980148196220398), ('skip_connect', 1, 0.12885591387748718), ('sep_conv_3x3', 3, 0.1286768764257431), ('sep_conv_5x5', 0, 0.1282753348350525), ('sep_conv_3x3', 4, 0.12763410806655884), ('dil_conv_3x3', 3, 0.12736497819423676)], normal_concat=range(2, 6), reduce=[('sep_conv_5x5', 1, 0.12647943198680878), ('avg_pool_3x3', 0, 0.12639141082763672), ('dil_conv_5x5', 2, 0.12712393701076508), ('sep_conv_5x5', 0, 0.12627579271793365), ('dil_conv_5x5', 3, 0.12630093097686768), ('dil_conv_3x3', 0, 0.12621402740478516), ('sep_conv_5x5', 1, 0.12634707987308502), ('sep_conv_3x3', 0, 0.1261734664440155)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-19:37:54] Epoch: [202][000/196] Time 1.49 (1.49) Data 0.89 (0.89) Loss 0.773 (0.773) Prec@1 69.53 (69.53) Prec@5 97.66 (97.66) TRAIN-SEARCH [2018-10-02-19:38:50] Epoch: [202][100/196] Time 0.57 (0.57) Data 0.00 (0.01) Loss 0.740 (0.859) Prec@1 76.56 (69.86) Prec@5 99.22 (97.70) TRAIN-SEARCH [2018-10-02-19:39:40] Epoch: [202][195/196] Time 0.33 (0.55) Data 0.00 (0.01) Loss 1.191 (0.859) Prec@1 50.00 (69.84) Prec@5 95.00 (97.67) VALID-SEARCH [2018-10-02-19:39:41] Epoch: [202][000/196] Loss 1.084 (1.084) Prec@1 62.50 (62.50) Prec@5 94.53 (94.53) VALID-SEARCH [2018-10-02-19:39:46] Epoch: [202][100/196] Loss 1.737 (1.223) Prec@1 38.28 (60.30) Prec@5 89.84 (94.38) VALID-SEARCH [2018-10-02-19:39:51] Epoch: [202][195/196] Loss 1.271 (1.263) Prec@1 55.00 (59.07) Prec@5 90.00 (94.02) 202/250, Train-Accuracy = 69.84, Test-Accuracy = 59.07 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-19:39:52] [Epoch=203/250] [Need: 01:34:11] [LR=0.0052 ~ 0.0052] [Batch=128], tau=5.128 genotype = Genotype(normal=[('skip_connect', 0, 0.1286320686340332), ('dil_conv_3x3', 1, 0.12822900712490082), ('sep_conv_3x3', 2, 0.12887349724769592), ('skip_connect', 1, 0.12871316075325012), ('sep_conv_3x3', 3, 0.1284479796886444), ('sep_conv_5x5', 0, 0.1282210499048233), ('sep_conv_3x3', 4, 0.12805436551570892), ('dil_conv_3x3', 3, 0.12729203701019287)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.12677842378616333), ('sep_conv_5x5', 1, 0.12550820410251617), ('dil_conv_5x5', 2, 0.12697815895080566), ('sep_conv_5x5', 0, 0.12648361921310425), ('dil_conv_5x5', 3, 0.12655732035636902), ('sep_conv_5x5', 0, 0.1262301206588745), ('sep_conv_3x3', 0, 0.1267717331647873), ('dil_conv_5x5', 2, 0.126378133893013)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-19:39:54] Epoch: [203][000/196] Time 1.67 (1.67) Data 1.00 (1.00) Loss 0.656 (0.656) Prec@1 74.22 (74.22) Prec@5 100.00 (100.00) TRAIN-SEARCH [2018-10-02-19:40:51] Epoch: [203][100/196] Time 0.54 (0.58) Data 0.00 (0.01) Loss 0.883 (0.840) Prec@1 67.97 (70.99) Prec@5 100.00 (97.63) TRAIN-SEARCH [2018-10-02-19:41:38] Epoch: [203][195/196] Time 0.49 (0.54) Data 0.00 (0.01) Loss 0.768 (0.848) Prec@1 77.50 (70.46) Prec@5 97.50 (97.63) VALID-SEARCH [2018-10-02-19:41:40] Epoch: [203][000/196] Loss 1.230 (1.230) Prec@1 57.81 (57.81) Prec@5 91.41 (91.41) VALID-SEARCH [2018-10-02-19:41:45] Epoch: [203][100/196] Loss 1.251 (1.343) Prec@1 63.28 (55.82) Prec@5 94.53 (92.76) VALID-SEARCH [2018-10-02-19:41:50] Epoch: [203][195/196] Loss 1.464 (1.324) Prec@1 55.00 (56.47) Prec@5 92.50 (93.10) 203/250, Train-Accuracy = 70.46, Test-Accuracy = 56.47 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-19:41:51] [Epoch=204/250] [Need: 01:31:49] [LR=0.0050 ~ 0.0050] [Batch=128], tau=5.104 genotype = Genotype(normal=[('skip_connect', 0, 0.12918147444725037), ('dil_conv_3x3', 1, 0.12833288311958313), ('sep_conv_3x3', 2, 0.12932628393173218), ('skip_connect', 1, 0.1286502629518509), ('sep_conv_5x5', 0, 0.1283949464559555), ('sep_conv_3x3', 3, 0.12793385982513428), ('dil_conv_3x3', 3, 0.12737958133220673), ('sep_conv_3x3', 0, 0.12736667692661285)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.1261506974697113), ('max_pool_3x3', 1, 0.12524747848510742), ('max_pool_3x3', 0, 0.12679423391819), ('dil_conv_5x5', 2, 0.12623491883277893), ('dil_conv_5x5', 3, 0.12673822045326233), ('sep_conv_5x5', 1, 0.12562629580497742), ('sep_conv_5x5', 1, 0.12653791904449463), ('max_pool_3x3', 0, 0.12633934617042542)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-19:41:54] Epoch: [204][000/196] Time 1.40 (1.40) Data 0.84 (0.84) Loss 0.777 (0.777) Prec@1 73.44 (73.44) Prec@5 100.00 (100.00) TRAIN-SEARCH [2018-10-02-19:42:51] Epoch: [204][100/196] Time 0.51 (0.58) Data 0.00 (0.01) Loss 1.113 (0.854) Prec@1 61.72 (70.69) Prec@5 98.44 (97.83) TRAIN-SEARCH [2018-10-02-19:43:39] Epoch: [204][195/196] Time 0.51 (0.54) Data 0.00 (0.01) Loss 1.035 (0.852) Prec@1 62.50 (70.56) Prec@5 97.50 (97.89) VALID-SEARCH [2018-10-02-19:43:40] Epoch: [204][000/196] Loss 0.883 (0.883) Prec@1 71.88 (71.88) Prec@5 98.44 (98.44) VALID-SEARCH [2018-10-02-19:43:46] Epoch: [204][100/196] Loss 1.096 (1.318) Prec@1 62.50 (58.04) Prec@5 96.88 (94.16) VALID-SEARCH [2018-10-02-19:43:51] Epoch: [204][195/196] Loss 1.069 (1.315) Prec@1 60.00 (57.48) Prec@5 97.50 (93.90) 204/250, Train-Accuracy = 70.56, Test-Accuracy = 57.48 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-19:43:52] [Epoch=205/250] [Need: 01:30:16] [LR=0.0048 ~ 0.0048] [Batch=128], tau=5.08 genotype = Genotype(normal=[('skip_connect', 0, 0.12948428094387054), ('skip_connect', 1, 0.12816517055034637), ('skip_connect', 0, 0.12969541549682617), ('sep_conv_3x3', 2, 0.12917117774486542), ('sep_conv_5x5', 0, 0.12814554572105408), ('sep_conv_5x5', 3, 0.12812332808971405), ('max_pool_3x3', 1, 0.12760977447032928), ('sep_conv_3x3', 0, 0.12736758589744568)], normal_concat=range(2, 6), reduce=[('dil_conv_5x5', 0, 0.1260647475719452), ('sep_conv_3x3', 1, 0.12576380372047424), ('max_pool_3x3', 0, 0.1266385018825531), ('dil_conv_5x5', 1, 0.12598155438899994), ('dil_conv_5x5', 3, 0.12618683278560638), ('max_pool_3x3', 0, 0.12599600851535797), ('avg_pool_3x3', 0, 0.1262609213590622), ('dil_conv_5x5', 1, 0.1261802613735199)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-19:43:54] Epoch: [205][000/196] Time 1.65 (1.65) Data 1.00 (1.00) Loss 0.576 (0.576) Prec@1 78.91 (78.91) Prec@5 99.22 (99.22) TRAIN-SEARCH [2018-10-02-19:44:51] Epoch: [205][100/196] Time 0.45 (0.57) Data 0.00 (0.01) Loss 0.787 (0.838) Prec@1 75.00 (70.59) Prec@5 98.44 (97.87) TRAIN-SEARCH [2018-10-02-19:45:39] Epoch: [205][195/196] Time 0.49 (0.54) Data 0.00 (0.01) Loss 0.937 (0.846) Prec@1 60.00 (70.35) Prec@5 100.00 (97.82) VALID-SEARCH [2018-10-02-19:45:40] Epoch: [205][000/196] Loss 1.060 (1.060) Prec@1 61.72 (61.72) Prec@5 96.09 (96.09) VALID-SEARCH [2018-10-02-19:45:45] Epoch: [205][100/196] Loss 2.100 (1.251) Prec@1 31.25 (60.25) Prec@5 82.03 (94.78) VALID-SEARCH [2018-10-02-19:45:50] Epoch: [205][195/196] Loss 0.984 (1.249) Prec@1 60.00 (59.89) Prec@5 100.00 (94.88) 205/250, Train-Accuracy = 70.35, Test-Accuracy = 59.89 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-19:45:51] [Epoch=206/250] [Need: 01:27:37] [LR=0.0047 ~ 0.0047] [Batch=128], tau=5.056 genotype = Genotype(normal=[('skip_connect', 0, 0.12878666818141937), ('skip_connect', 1, 0.1287807822227478), ('sep_conv_3x3', 2, 0.12932869791984558), ('dil_conv_3x3', 0, 0.12878921627998352), ('sep_conv_5x5', 0, 0.12841448187828064), ('sep_conv_3x3', 3, 0.12807714939117432), ('max_pool_3x3', 1, 0.12851180136203766), ('sep_conv_3x3', 0, 0.12751418352127075)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 0, 0.12544646859169006), ('dil_conv_3x3', 1, 0.12534500658512115), ('dil_conv_5x5', 1, 0.12650738656520844), ('dil_conv_5x5', 2, 0.12606026232242584), ('max_pool_3x3', 0, 0.12618616223335266), ('sep_conv_3x3', 3, 0.12604622542858124), ('dil_conv_5x5', 2, 0.12629689276218414), ('dil_conv_5x5', 1, 0.12607696652412415)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-19:45:54] Epoch: [206][000/196] Time 1.57 (1.57) Data 0.95 (0.95) Loss 0.683 (0.683) Prec@1 78.12 (78.12) Prec@5 99.22 (99.22) TRAIN-SEARCH [2018-10-02-19:46:49] Epoch: [206][100/196] Time 0.40 (0.56) Data 0.00 (0.01) Loss 0.985 (0.837) Prec@1 66.41 (70.74) Prec@5 95.31 (97.73) TRAIN-SEARCH [2018-10-02-19:47:38] Epoch: [206][195/196] Time 0.57 (0.54) Data 0.00 (0.01) Loss 0.779 (0.843) Prec@1 75.00 (70.56) Prec@5 100.00 (97.82) VALID-SEARCH [2018-10-02-19:47:39] Epoch: [206][000/196] Loss 1.385 (1.385) Prec@1 52.34 (52.34) Prec@5 92.97 (92.97) VALID-SEARCH [2018-10-02-19:47:45] Epoch: [206][100/196] Loss 0.979 (1.196) Prec@1 63.28 (59.78) Prec@5 98.44 (94.85) VALID-SEARCH [2018-10-02-19:47:50] Epoch: [206][195/196] Loss 1.385 (1.259) Prec@1 47.50 (58.70) Prec@5 95.00 (93.79) 206/250, Train-Accuracy = 70.56, Test-Accuracy = 58.70 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-19:47:51] [Epoch=207/250] [Need: 01:25:36] [LR=0.0045 ~ 0.0045] [Batch=128], tau=5.032 genotype = Genotype(normal=[('skip_connect', 0, 0.1289718896150589), ('skip_connect', 1, 0.12790945172309875), ('sep_conv_3x3', 2, 0.12991100549697876), ('skip_connect', 0, 0.1293594390153885), ('sep_conv_5x5', 0, 0.1287710964679718), ('sep_conv_3x3', 3, 0.12770572304725647), ('max_pool_3x3', 1, 0.12807431817054749), ('sep_conv_5x5', 2, 0.12796534597873688)], normal_concat=range(2, 6), reduce=[('avg_pool_3x3', 1, 0.12560850381851196), ('max_pool_3x3', 0, 0.12560151517391205), ('dil_conv_5x5', 1, 0.1269674450159073), ('avg_pool_3x3', 0, 0.1264246255159378), ('max_pool_3x3', 1, 0.12627236545085907), ('sep_conv_3x3', 2, 0.12622985243797302), ('sep_conv_5x5', 1, 0.12671007215976715), ('sep_conv_5x5', 0, 0.12593518197536469)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-19:47:53] Epoch: [207][000/196] Time 1.42 (1.42) Data 0.84 (0.84) Loss 0.839 (0.839) Prec@1 69.53 (69.53) Prec@5 99.22 (99.22) TRAIN-SEARCH [2018-10-02-19:48:48] Epoch: [207][100/196] Time 0.63 (0.56) Data 0.00 (0.01) Loss 0.687 (0.833) Prec@1 75.00 (71.19) Prec@5 99.22 (97.94) TRAIN-SEARCH [2018-10-02-19:49:37] Epoch: [207][195/196] Time 0.55 (0.54) Data 0.00 (0.01) Loss 1.175 (0.840) Prec@1 60.00 (70.63) Prec@5 95.00 (97.91) VALID-SEARCH [2018-10-02-19:49:39] Epoch: [207][000/196] Loss 1.694 (1.694) Prec@1 52.34 (52.34) Prec@5 95.31 (95.31) VALID-SEARCH [2018-10-02-19:49:44] Epoch: [207][100/196] Loss 0.869 (1.326) Prec@1 71.09 (56.24) Prec@5 99.22 (93.29) VALID-SEARCH [2018-10-02-19:49:49] Epoch: [207][195/196] Loss 0.785 (1.304) Prec@1 80.00 (56.59) Prec@5 97.50 (93.68) 207/250, Train-Accuracy = 70.63, Test-Accuracy = 56.59 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-19:49:50] [Epoch=208/250] [Need: 01:23:10] [LR=0.0043 ~ 0.0043] [Batch=128], tau=5.008 genotype = Genotype(normal=[('dil_conv_3x3', 0, 0.1287277191877365), ('skip_connect', 1, 0.12813086807727814), ('sep_conv_5x5', 2, 0.1295812726020813), ('skip_connect', 0, 0.12955152988433838), ('sep_conv_5x5', 0, 0.12928295135498047), ('dil_conv_3x3', 2, 0.1276017129421234), ('sep_conv_3x3', 4, 0.1278541535139084), ('sep_conv_5x5', 2, 0.1277894824743271)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1, 0.12556889653205872), ('sep_conv_5x5', 0, 0.12532643973827362), ('skip_connect', 0, 0.12689585983753204), ('dil_conv_5x5', 1, 0.12639813125133514), ('sep_conv_5x5', 2, 0.12600642442703247), ('sep_conv_3x3', 3, 0.1260024607181549), ('dil_conv_5x5', 3, 0.1263890117406845), ('sep_conv_5x5', 1, 0.1263720542192459)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-19:49:52] Epoch: [208][000/196] Time 1.64 (1.64) Data 1.10 (1.10) Loss 1.013 (1.013) Prec@1 61.72 (61.72) Prec@5 97.66 (97.66) TRAIN-SEARCH [2018-10-02-19:50:47] Epoch: [208][100/196] Time 0.55 (0.56) Data 0.00 (0.01) Loss 0.829 (0.831) Prec@1 73.44 (71.00) Prec@5 97.66 (97.66) TRAIN-SEARCH [2018-10-02-19:51:36] Epoch: [208][195/196] Time 0.41 (0.54) Data 0.00 (0.01) Loss 0.853 (0.828) Prec@1 70.00 (71.28) Prec@5 97.50 (97.76) VALID-SEARCH [2018-10-02-19:51:37] Epoch: [208][000/196] Loss 1.713 (1.713) Prec@1 47.66 (47.66) Prec@5 93.75 (93.75) VALID-SEARCH [2018-10-02-19:51:43] Epoch: [208][100/196] Loss 1.079 (1.264) Prec@1 57.81 (58.35) Prec@5 97.66 (94.31) VALID-SEARCH [2018-10-02-19:51:48] Epoch: [208][195/196] Loss 0.710 (1.290) Prec@1 80.00 (57.36) Prec@5 95.00 (93.83) 208/250, Train-Accuracy = 71.28, Test-Accuracy = 57.36 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-19:51:49] [Epoch=209/250] [Need: 01:21:28] [LR=0.0042 ~ 0.0042] [Batch=128], tau=4.984 genotype = Genotype(normal=[('dil_conv_3x3', 0, 0.12979905307292938), ('skip_connect', 1, 0.1283504068851471), ('skip_connect', 0, 0.129797101020813), ('sep_conv_3x3', 2, 0.12969441711902618), ('sep_conv_5x5', 0, 0.12886999547481537), ('sep_conv_3x3', 3, 0.12803485989570618), ('max_pool_3x3', 0, 0.1281088888645172), ('sep_conv_3x3', 4, 0.12760856747627258)], normal_concat=range(2, 6), reduce=[('sep_conv_5x5', 1, 0.1258508414030075), ('sep_conv_3x3', 0, 0.12568803131580353), ('dil_conv_5x5', 1, 0.1268889456987381), ('skip_connect', 0, 0.12634976208209991), ('sep_conv_5x5', 2, 0.12651178240776062), ('dil_conv_3x3', 3, 0.1263687163591385), ('dil_conv_5x5', 3, 0.12658965587615967), ('sep_conv_5x5', 1, 0.12619107961654663)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-19:51:51] Epoch: [209][000/196] Time 1.56 (1.56) Data 0.93 (0.93) Loss 0.948 (0.948) Prec@1 64.84 (64.84) Prec@5 96.88 (96.88) TRAIN-SEARCH [2018-10-02-19:52:47] Epoch: [209][100/196] Time 0.58 (0.56) Data 0.02 (0.01) Loss 0.777 (0.829) Prec@1 74.22 (70.95) Prec@5 99.22 (97.90) TRAIN-SEARCH [2018-10-02-19:53:36] Epoch: [209][195/196] Time 0.47 (0.54) Data 0.00 (0.01) Loss 0.823 (0.833) Prec@1 72.50 (70.86) Prec@5 95.00 (97.89) VALID-SEARCH [2018-10-02-19:53:37] Epoch: [209][000/196] Loss 1.031 (1.031) Prec@1 62.50 (62.50) Prec@5 96.88 (96.88) VALID-SEARCH [2018-10-02-19:53:43] Epoch: [209][100/196] Loss 1.086 (1.279) Prec@1 59.38 (57.50) Prec@5 95.31 (93.35) VALID-SEARCH [2018-10-02-19:53:48] Epoch: [209][195/196] Loss 1.553 (1.273) Prec@1 50.00 (57.83) Prec@5 80.00 (93.60) 209/250, Train-Accuracy = 70.86, Test-Accuracy = 57.83 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-19:53:48] [Epoch=210/250] [Need: 01:19:42] [LR=0.0040 ~ 0.0040] [Batch=128], tau=4.96 genotype = Genotype(normal=[('dil_conv_3x3', 0, 0.129292830824852), ('dil_conv_3x3', 1, 0.12799368798732758), ('sep_conv_3x3', 2, 0.13050246238708496), ('skip_connect', 0, 0.13029640913009644), ('sep_conv_5x5', 0, 0.12794741988182068), ('dil_conv_5x5', 3, 0.1279333084821701), ('max_pool_3x3', 0, 0.1285739690065384), ('sep_conv_5x5', 2, 0.1276312619447708)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1, 0.1259050965309143), ('max_pool_3x3', 0, 0.125361829996109), ('sep_conv_3x3', 2, 0.1268623322248459), ('avg_pool_3x3', 0, 0.1262747198343277), ('sep_conv_5x5', 2, 0.1269383728504181), ('sep_conv_3x3', 3, 0.12606605887413025), ('sep_conv_5x5', 1, 0.12618066370487213), ('dil_conv_5x5', 3, 0.12602004408836365)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-19:53:50] Epoch: [210][000/196] Time 1.12 (1.12) Data 0.59 (0.59) Loss 0.893 (0.893) Prec@1 70.31 (70.31) Prec@5 99.22 (99.22) TRAIN-SEARCH [2018-10-02-19:54:46] Epoch: [210][100/196] Time 0.51 (0.57) Data 0.00 (0.01) Loss 0.809 (0.821) Prec@1 73.44 (71.10) Prec@5 97.66 (97.86) TRAIN-SEARCH [2018-10-02-19:55:36] Epoch: [210][195/196] Time 0.43 (0.54) Data 0.00 (0.01) Loss 0.575 (0.831) Prec@1 77.50 (70.82) Prec@5 100.00 (97.86) VALID-SEARCH [2018-10-02-19:55:37] Epoch: [210][000/196] Loss 1.093 (1.093) Prec@1 61.72 (61.72) Prec@5 93.75 (93.75) VALID-SEARCH [2018-10-02-19:55:41] Epoch: [210][100/196] Loss 0.975 (1.225) Prec@1 65.62 (59.96) Prec@5 95.31 (94.79) VALID-SEARCH [2018-10-02-19:55:46] Epoch: [210][195/196] Loss 0.931 (1.229) Prec@1 70.00 (59.98) Prec@5 97.50 (94.50) 210/250, Train-Accuracy = 70.82, Test-Accuracy = 59.98 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-19:55:48] [Epoch=211/250] [Need: 01:17:28] [LR=0.0039 ~ 0.0039] [Batch=128], tau=4.936 genotype = Genotype(normal=[('dil_conv_3x3', 0, 0.12913472950458527), ('dil_conv_3x3', 1, 0.12780903279781342), ('sep_conv_3x3', 2, 0.13053607940673828), ('skip_connect', 0, 0.13042061030864716), ('dil_conv_3x3', 3, 0.12789374589920044), ('sep_conv_5x5', 2, 0.1277712881565094), ('sep_conv_3x3', 4, 0.12782958149909973), ('max_pool_3x3', 0, 0.1275060623884201)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 0, 0.1262657195329666), ('sep_conv_3x3', 1, 0.12618865072727203), ('sep_conv_5x5', 2, 0.1265183538198471), ('avg_pool_3x3', 0, 0.1264672726392746), ('avg_pool_3x3', 0, 0.12617723643779755), ('max_pool_3x3', 1, 0.12610110640525818), ('dil_conv_5x5', 3, 0.1262870877981186), ('dil_conv_5x5', 4, 0.1262451559305191)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-19:55:50] Epoch: [211][000/196] Time 1.51 (1.51) Data 0.84 (0.84) Loss 0.864 (0.864) Prec@1 73.44 (73.44) Prec@5 98.44 (98.44) TRAIN-SEARCH [2018-10-02-19:56:49] Epoch: [211][100/196] Time 0.75 (0.60) Data 0.00 (0.01) Loss 0.668 (0.812) Prec@1 74.22 (71.36) Prec@5 99.22 (98.17) TRAIN-SEARCH [2018-10-02-19:57:35] Epoch: [211][195/196] Time 0.44 (0.55) Data 0.00 (0.01) Loss 0.856 (0.822) Prec@1 75.00 (71.10) Prec@5 100.00 (98.14) VALID-SEARCH [2018-10-02-19:57:37] Epoch: [211][000/196] Loss 1.251 (1.251) Prec@1 58.59 (58.59) Prec@5 96.88 (96.88) VALID-SEARCH [2018-10-02-19:57:42] Epoch: [211][100/196] Loss 1.736 (1.298) Prec@1 34.38 (57.44) Prec@5 92.19 (92.88) VALID-SEARCH [2018-10-02-19:57:47] Epoch: [211][195/196] Loss 1.099 (1.265) Prec@1 70.00 (58.50) Prec@5 92.50 (93.87) 211/250, Train-Accuracy = 71.10, Test-Accuracy = 58.50 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-19:57:48] [Epoch=212/250] [Need: 01:16:29] [LR=0.0037 ~ 0.0037] [Batch=128], tau=4.912 genotype = Genotype(normal=[('dil_conv_3x3', 0, 0.12890145182609558), ('dil_conv_3x3', 1, 0.12848542630672455), ('sep_conv_3x3', 2, 0.13035233318805695), ('skip_connect', 0, 0.13020473718643188), ('dil_conv_3x3', 2, 0.12749668955802917), ('sep_conv_5x5', 3, 0.12743490934371948), ('max_pool_3x3', 0, 0.1279703974723816), ('sep_conv_3x3', 2, 0.12763142585754395)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1, 0.12598930299282074), ('avg_pool_3x3', 0, 0.12578991055488586), ('sep_conv_5x5', 2, 0.1266600638628006), ('avg_pool_3x3', 0, 0.12663856148719788), ('max_pool_3x3', 1, 0.12679491937160492), ('sep_conv_5x5', 2, 0.12622308731079102), ('max_pool_3x3', 0, 0.12660323083400726), ('dil_conv_5x5', 4, 0.12601357698440552)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-19:57:51] Epoch: [212][000/196] Time 1.61 (1.61) Data 0.97 (0.97) Loss 0.970 (0.970) Prec@1 60.16 (60.16) Prec@5 98.44 (98.44) TRAIN-SEARCH [2018-10-02-19:58:48] Epoch: [212][100/196] Time 0.53 (0.59) Data 0.00 (0.01) Loss 0.720 (0.815) Prec@1 75.78 (71.19) Prec@5 100.00 (98.06) TRAIN-SEARCH [2018-10-02-19:59:36] Epoch: [212][195/196] Time 0.50 (0.55) Data 0.00 (0.01) Loss 0.937 (0.820) Prec@1 70.00 (70.96) Prec@5 97.50 (98.02) VALID-SEARCH [2018-10-02-19:59:37] Epoch: [212][000/196] Loss 1.098 (1.098) Prec@1 68.75 (68.75) Prec@5 96.88 (96.88) VALID-SEARCH [2018-10-02-19:59:42] Epoch: [212][100/196] Loss 1.101 (1.221) Prec@1 69.53 (61.07) Prec@5 96.88 (95.34) VALID-SEARCH [2018-10-02-19:59:47] Epoch: [212][195/196] Loss 1.456 (1.243) Prec@1 50.00 (60.71) Prec@5 92.50 (95.44) 212/250, Train-Accuracy = 70.96, Test-Accuracy = 60.71 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-19:59:49] [Epoch=213/250] [Need: 01:14:07] [LR=0.0036 ~ 0.0036] [Batch=128], tau=4.888 genotype = Genotype(normal=[('sep_conv_5x5', 0, 0.1292668730020523), ('dil_conv_3x3', 1, 0.12897200882434845), ('skip_connect', 0, 0.13033318519592285), ('sep_conv_3x3', 2, 0.13033133745193481), ('dil_conv_3x3', 2, 0.12770481407642365), ('sep_conv_3x3', 3, 0.12761518359184265), ('max_pool_3x3', 0, 0.12834516167640686), ('sep_conv_3x3', 4, 0.1282183825969696)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1, 0.12628737092018127), ('avg_pool_3x3', 0, 0.1260230392217636), ('sep_conv_5x5', 0, 0.12716637551784515), ('dil_conv_5x5', 2, 0.12609079480171204), ('sep_conv_5x5', 2, 0.12649735808372498), ('avg_pool_3x3', 0, 0.12638996541500092), ('sep_conv_5x5', 1, 0.12691755592823029), ('sep_conv_5x5', 4, 0.1262306123971939)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-19:59:51] Epoch: [213][000/196] Time 1.54 (1.54) Data 0.90 (0.90) Loss 0.727 (0.727) Prec@1 72.66 (72.66) Prec@5 98.44 (98.44) TRAIN-SEARCH [2018-10-02-20:00:46] Epoch: [213][100/196] Time 0.55 (0.57) Data 0.00 (0.01) Loss 0.755 (0.808) Prec@1 76.56 (71.64) Prec@5 97.66 (98.14) TRAIN-SEARCH [2018-10-02-20:01:34] Epoch: [213][195/196] Time 0.44 (0.53) Data 0.00 (0.01) Loss 0.898 (0.817) Prec@1 67.50 (71.18) Prec@5 97.50 (97.94) VALID-SEARCH [2018-10-02-20:01:35] Epoch: [213][000/196] Loss 1.028 (1.028) Prec@1 64.06 (64.06) Prec@5 94.53 (94.53) VALID-SEARCH [2018-10-02-20:01:40] Epoch: [213][100/196] Loss 1.729 (1.184) Prec@1 43.75 (60.88) Prec@5 85.94 (95.16) VALID-SEARCH [2018-10-02-20:01:45] Epoch: [213][195/196] Loss 0.800 (1.205) Prec@1 67.50 (60.55) Prec@5 97.50 (95.13) 213/250, Train-Accuracy = 71.18, Test-Accuracy = 60.55 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-20:01:46] [Epoch=214/250] [Need: 01:10:44] [LR=0.0035 ~ 0.0035] [Batch=128], tau=4.864 genotype = Genotype(normal=[('sep_conv_5x5', 0, 0.12957361340522766), ('dil_conv_3x3', 1, 0.1293697953224182), ('skip_connect', 0, 0.1305471956729889), ('sep_conv_3x3', 2, 0.12967757880687714), ('sep_conv_3x3', 3, 0.1282871514558792), ('dil_conv_3x3', 2, 0.12799569964408875), ('max_pool_3x3', 0, 0.12783044576644897), ('sep_conv_3x3', 4, 0.12773331999778748)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1, 0.12649644911289215), ('avg_pool_3x3', 0, 0.1259181648492813), ('avg_pool_3x3', 0, 0.1271429806947708), ('dil_conv_5x5', 1, 0.1261097639799118), ('sep_conv_5x5', 0, 0.12691152095794678), ('sep_conv_3x3', 1, 0.1260027289390564), ('sep_conv_5x5', 4, 0.12662753462791443), ('sep_conv_5x5', 0, 0.1262339949607849)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-20:01:49] Epoch: [214][000/196] Time 1.64 (1.64) Data 0.84 (0.84) Loss 0.801 (0.801) Prec@1 72.66 (72.66) Prec@5 97.66 (97.66) TRAIN-SEARCH [2018-10-02-20:02:46] Epoch: [214][100/196] Time 0.40 (0.59) Data 0.00 (0.01) Loss 0.853 (0.820) Prec@1 65.62 (71.53) Prec@5 99.22 (97.73) TRAIN-SEARCH [2018-10-02-20:03:38] Epoch: [214][195/196] Time 0.58 (0.56) Data 0.00 (0.01) Loss 0.874 (0.823) Prec@1 62.50 (71.41) Prec@5 100.00 (97.84) VALID-SEARCH [2018-10-02-20:03:39] Epoch: [214][000/196] Loss 1.154 (1.154) Prec@1 60.94 (60.94) Prec@5 94.53 (94.53) VALID-SEARCH [2018-10-02-20:03:44] Epoch: [214][100/196] Loss 1.739 (1.190) Prec@1 42.97 (60.50) Prec@5 88.28 (95.06) VALID-SEARCH [2018-10-02-20:03:49] Epoch: [214][195/196] Loss 0.857 (1.240) Prec@1 67.50 (59.66) Prec@5 97.50 (94.52) 214/250, Train-Accuracy = 71.41, Test-Accuracy = 59.66 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-20:03:50] [Epoch=215/250] [Need: 01:12:11] [LR=0.0033 ~ 0.0033] [Batch=128], tau=4.84 genotype = Genotype(normal=[('sep_conv_5x5', 0, 0.13042639195919037), ('dil_conv_3x3', 1, 0.1292448490858078), ('skip_connect', 0, 0.13035976886749268), ('sep_conv_5x5', 2, 0.12923406064510345), ('sep_conv_3x3', 3, 0.12809909880161285), ('sep_conv_3x3', 2, 0.12794215977191925), ('skip_connect', 0, 0.12816181778907776), ('sep_conv_3x3', 4, 0.12798212468624115)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1, 0.12643969058990479), ('sep_conv_5x5', 0, 0.1256922036409378), ('sep_conv_5x5', 0, 0.12698788940906525), ('dil_conv_5x5', 1, 0.12618863582611084), ('sep_conv_5x5', 0, 0.12743619084358215), ('sep_conv_5x5', 2, 0.12605339288711548), ('sep_conv_5x5', 4, 0.1264934092760086), ('max_pool_3x3', 0, 0.12622109055519104)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-20:03:53] Epoch: [215][000/196] Time 1.56 (1.56) Data 0.84 (0.84) Loss 0.639 (0.639) Prec@1 74.22 (74.22) Prec@5 100.00 (100.00) TRAIN-SEARCH [2018-10-02-20:04:45] Epoch: [215][100/196] Time 0.40 (0.53) Data 0.00 (0.01) Loss 0.940 (0.818) Prec@1 64.84 (71.41) Prec@5 98.44 (98.03) TRAIN-SEARCH [2018-10-02-20:05:36] Epoch: [215][195/196] Time 0.52 (0.54) Data 0.00 (0.01) Loss 0.887 (0.816) Prec@1 65.00 (71.50) Prec@5 100.00 (97.96) VALID-SEARCH [2018-10-02-20:05:37] Epoch: [215][000/196] Loss 2.153 (2.153) Prec@1 25.78 (25.78) Prec@5 68.75 (68.75) VALID-SEARCH [2018-10-02-20:05:43] Epoch: [215][100/196] Loss 1.507 (1.370) Prec@1 57.03 (55.37) Prec@5 95.31 (91.99) VALID-SEARCH [2018-10-02-20:05:48] Epoch: [215][195/196] Loss 1.057 (1.359) Prec@1 60.00 (55.60) Prec@5 95.00 (92.01) 215/250, Train-Accuracy = 71.50, Test-Accuracy = 55.60 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-20:05:49] [Epoch=216/250] [Need: 01:07:20] [LR=0.0032 ~ 0.0032] [Batch=128], tau=4.816 genotype = Genotype(normal=[('sep_conv_5x5', 0, 0.13069352507591248), ('dil_conv_3x3', 1, 0.12878581881523132), ('skip_connect', 0, 0.1299157440662384), ('sep_conv_5x5', 2, 0.1296212375164032), ('sep_conv_3x3', 2, 0.12781865894794464), ('sep_conv_3x3', 3, 0.12771300971508026), ('skip_connect', 1, 0.12868542969226837), ('sep_conv_3x3', 4, 0.12814544141292572)], normal_concat=range(2, 6), reduce=[('sep_conv_5x5', 0, 0.12624362111091614), ('dil_conv_5x5', 1, 0.1261412352323532), ('sep_conv_5x5', 0, 0.12746712565422058), ('sep_conv_5x5', 2, 0.12671531736850739), ('sep_conv_5x5', 0, 0.12720705568790436), ('sep_conv_5x5', 3, 0.1260370910167694), ('max_pool_3x3', 0, 0.12680977582931519), ('dil_conv_5x5', 3, 0.1261153668165207)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-20:05:51] Epoch: [216][000/196] Time 1.43 (1.43) Data 0.88 (0.88) Loss 0.807 (0.807) Prec@1 70.31 (70.31) Prec@5 98.44 (98.44) TRAIN-SEARCH [2018-10-02-20:06:43] Epoch: [216][100/196] Time 0.40 (0.52) Data 0.00 (0.01) Loss 0.789 (0.813) Prec@1 73.44 (71.87) Prec@5 99.22 (98.02) TRAIN-SEARCH [2018-10-02-20:07:35] Epoch: [216][195/196] Time 0.49 (0.54) Data 0.00 (0.01) Loss 1.047 (0.815) Prec@1 65.00 (71.69) Prec@5 95.00 (97.99) VALID-SEARCH [2018-10-02-20:07:36] Epoch: [216][000/196] Loss 0.844 (0.844) Prec@1 65.62 (65.62) Prec@5 99.22 (99.22) VALID-SEARCH [2018-10-02-20:07:42] Epoch: [216][100/196] Loss 1.016 (1.225) Prec@1 62.50 (58.69) Prec@5 96.88 (94.38) VALID-SEARCH [2018-10-02-20:07:47] Epoch: [216][195/196] Loss 1.493 (1.246) Prec@1 42.50 (57.99) Prec@5 90.00 (93.94) 216/250, Train-Accuracy = 71.69, Test-Accuracy = 57.99 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-20:07:48] [Epoch=217/250] [Need: 01:05:18] [LR=0.0031 ~ 0.0031] [Batch=128], tau=4.792 genotype = Genotype(normal=[('sep_conv_5x5', 0, 0.13000331819057465), ('dil_conv_3x3', 1, 0.12885639071464539), ('sep_conv_5x5', 2, 0.13020052015781403), ('skip_connect', 0, 0.12991569936275482), ('sep_conv_3x3', 2, 0.12791231274604797), ('sep_conv_3x3', 3, 0.12762774527072906), ('skip_connect', 1, 0.12876588106155396), ('sep_conv_3x3', 4, 0.12830057740211487)], normal_concat=range(2, 6), reduce=[('sep_conv_5x5', 0, 0.12641243636608124), ('sep_conv_3x3', 1, 0.1262694001197815), ('sep_conv_5x5', 0, 0.12768538296222687), ('sep_conv_5x5', 2, 0.12718306481838226), ('sep_conv_5x5', 0, 0.12730953097343445), ('sep_conv_5x5', 2, 0.12636014819145203), ('max_pool_3x3', 0, 0.1267566829919815), ('dil_conv_5x5', 3, 0.1261744499206543)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-20:07:50] Epoch: [217][000/196] Time 1.52 (1.52) Data 0.95 (0.95) Loss 0.767 (0.767) Prec@1 71.88 (71.88) Prec@5 99.22 (99.22) TRAIN-SEARCH [2018-10-02-20:08:42] Epoch: [217][100/196] Time 0.40 (0.53) Data 0.00 (0.01) Loss 0.761 (0.802) Prec@1 73.44 (72.11) Prec@5 98.44 (98.00) TRAIN-SEARCH [2018-10-02-20:09:34] Epoch: [217][195/196] Time 0.54 (0.54) Data 0.00 (0.01) Loss 0.752 (0.806) Prec@1 77.50 (71.83) Prec@5 97.50 (97.94) VALID-SEARCH [2018-10-02-20:09:35] Epoch: [217][000/196] Loss 1.738 (1.738) Prec@1 37.50 (37.50) Prec@5 85.94 (85.94) VALID-SEARCH [2018-10-02-20:09:40] Epoch: [217][100/196] Loss 1.097 (1.303) Prec@1 63.28 (58.77) Prec@5 96.88 (94.30) VALID-SEARCH [2018-10-02-20:09:45] Epoch: [217][195/196] Loss 0.862 (1.300) Prec@1 62.50 (58.43) Prec@5 97.50 (94.05) 217/250, Train-Accuracy = 71.83, Test-Accuracy = 58.43 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-20:09:46] [Epoch=218/250] [Need: 01:03:18] [LR=0.0030 ~ 0.0030] [Batch=128], tau=4.768 genotype = Genotype(normal=[('sep_conv_5x5', 0, 0.1296394020318985), ('dil_conv_3x3', 1, 0.12851564586162567), ('sep_conv_5x5', 2, 0.12992547452449799), ('skip_connect', 0, 0.12931224703788757), ('sep_conv_5x5', 3, 0.12789447605609894), ('sep_conv_3x3', 2, 0.12782832980155945), ('sep_conv_3x3', 4, 0.12915872037410736), ('skip_connect', 1, 0.1281287968158722)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1, 0.12604261934757233), ('sep_conv_5x5', 0, 0.12553146481513977), ('sep_conv_5x5', 0, 0.12730194628238678), ('sep_conv_5x5', 2, 0.127031147480011), ('sep_conv_5x5', 0, 0.12749674916267395), ('sep_conv_5x5', 2, 0.12661007046699524), ('max_pool_3x3', 0, 0.12735424935817719), ('sep_conv_5x5', 1, 0.12653520703315735)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-20:09:49] Epoch: [218][000/196] Time 1.46 (1.46) Data 0.87 (0.87) Loss 1.044 (1.044) Prec@1 60.16 (60.16) Prec@5 94.53 (94.53) TRAIN-SEARCH [2018-10-02-20:10:39] Epoch: [218][100/196] Time 0.59 (0.51) Data 0.00 (0.01) Loss 0.922 (0.818) Prec@1 64.84 (71.80) Prec@5 100.00 (97.69) TRAIN-SEARCH [2018-10-02-20:11:31] Epoch: [218][195/196] Time 0.48 (0.53) Data 0.00 (0.01) Loss 0.562 (0.817) Prec@1 80.00 (71.58) Prec@5 97.50 (97.84) VALID-SEARCH [2018-10-02-20:11:33] Epoch: [218][000/196] Loss 1.392 (1.392) Prec@1 66.41 (66.41) Prec@5 97.66 (97.66) VALID-SEARCH [2018-10-02-20:11:38] Epoch: [218][100/196] Loss 1.157 (1.229) Prec@1 57.81 (59.44) Prec@5 96.09 (94.38) VALID-SEARCH [2018-10-02-20:11:42] Epoch: [218][195/196] Loss 0.695 (1.203) Prec@1 77.50 (59.66) Prec@5 100.00 (94.40) 218/250, Train-Accuracy = 71.58, Test-Accuracy = 59.66 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-20:11:43] [Epoch=219/250] [Need: 01:00:15] [LR=0.0028 ~ 0.0028] [Batch=128], tau=4.744 genotype = Genotype(normal=[('sep_conv_5x5', 0, 0.12995445728302002), ('dil_conv_3x3', 1, 0.1285555511713028), ('skip_connect', 0, 0.1299256831407547), ('sep_conv_3x3', 2, 0.12992237508296967), ('sep_conv_5x5', 3, 0.12845738232135773), ('sep_conv_3x3', 2, 0.12801016867160797), ('skip_connect', 1, 0.12822192907333374), ('sep_conv_3x3', 4, 0.12816190719604492)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1, 0.12615221738815308), ('sep_conv_5x5', 0, 0.12566660344600677), ('sep_conv_5x5', 0, 0.12733474373817444), ('sep_conv_5x5', 2, 0.12696057558059692), ('sep_conv_5x5', 0, 0.12738201022148132), ('dil_conv_5x5', 2, 0.12652450799942017), ('sep_conv_5x5', 1, 0.1268828958272934), ('max_pool_3x3', 0, 0.12658284604549408)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-20:11:45] Epoch: [219][000/196] Time 1.30 (1.30) Data 0.81 (0.81) Loss 0.831 (0.831) Prec@1 71.88 (71.88) Prec@5 96.88 (96.88) TRAIN-SEARCH [2018-10-02-20:12:39] Epoch: [219][100/196] Time 0.55 (0.55) Data 0.00 (0.01) Loss 0.720 (0.804) Prec@1 71.88 (71.42) Prec@5 100.00 (98.07) TRAIN-SEARCH [2018-10-02-20:13:31] Epoch: [219][195/196] Time 0.48 (0.55) Data 0.00 (0.01) Loss 0.586 (0.802) Prec@1 82.50 (71.61) Prec@5 97.50 (98.10) VALID-SEARCH [2018-10-02-20:13:32] Epoch: [219][000/196] Loss 1.001 (1.001) Prec@1 65.62 (65.62) Prec@5 96.09 (96.09) VALID-SEARCH [2018-10-02-20:13:36] Epoch: [219][100/196] Loss 0.954 (1.288) Prec@1 73.44 (58.35) Prec@5 96.09 (94.54) VALID-SEARCH [2018-10-02-20:13:41] Epoch: [219][195/196] Loss 0.837 (1.281) Prec@1 72.50 (58.28) Prec@5 95.00 (94.57) 219/250, Train-Accuracy = 71.61, Test-Accuracy = 58.28 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-20:13:42] [Epoch=220/250] [Need: 00:59:32] [LR=0.0027 ~ 0.0027] [Batch=128], tau=4.72 genotype = Genotype(normal=[('sep_conv_5x5', 0, 0.1304772049188614), ('dil_conv_3x3', 1, 0.1282629370689392), ('skip_connect', 0, 0.130629763007164), ('sep_conv_3x3', 2, 0.1301836222410202), ('sep_conv_5x5', 3, 0.12949495017528534), ('sep_conv_5x5', 2, 0.12811006605625153), ('sep_conv_3x3', 4, 0.12828373908996582), ('skip_connect', 1, 0.12787699699401855)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1, 0.12682650983333588), ('sep_conv_3x3', 0, 0.12527737021446228), ('sep_conv_5x5', 2, 0.1270214319229126), ('sep_conv_5x5', 0, 0.12659238278865814), ('dil_conv_5x5', 2, 0.12676109373569489), ('sep_conv_5x5', 0, 0.12652061879634857), ('sep_conv_5x5', 1, 0.12720350921154022), ('sep_conv_5x5', 4, 0.12653066217899323)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-20:13:45] Epoch: [220][000/196] Time 1.50 (1.50) Data 0.96 (0.96) Loss 0.714 (0.714) Prec@1 74.22 (74.22) Prec@5 97.66 (97.66) TRAIN-SEARCH [2018-10-02-20:14:37] Epoch: [220][100/196] Time 0.66 (0.54) Data 0.01 (0.01) Loss 0.759 (0.802) Prec@1 76.56 (71.84) Prec@5 96.88 (97.87) TRAIN-SEARCH [2018-10-02-20:15:29] Epoch: [220][195/196] Time 0.40 (0.54) Data 0.00 (0.01) Loss 0.941 (0.799) Prec@1 57.50 (71.91) Prec@5 100.00 (98.02) VALID-SEARCH [2018-10-02-20:15:30] Epoch: [220][000/196] Loss 1.039 (1.039) Prec@1 68.75 (68.75) Prec@5 96.09 (96.09) VALID-SEARCH [2018-10-02-20:15:35] Epoch: [220][100/196] Loss 1.619 (1.292) Prec@1 52.34 (58.02) Prec@5 93.75 (93.42) VALID-SEARCH [2018-10-02-20:15:40] Epoch: [220][195/196] Loss 2.602 (1.289) Prec@1 37.50 (57.93) Prec@5 65.00 (93.72) 220/250, Train-Accuracy = 71.91, Test-Accuracy = 57.93 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-20:15:41] [Epoch=221/250] [Need: 00:57:32] [LR=0.0026 ~ 0.0026] [Batch=128], tau=4.696 genotype = Genotype(normal=[('sep_conv_5x5', 0, 0.13028886914253235), ('dil_conv_3x3', 1, 0.1288178712129593), ('sep_conv_3x3', 2, 0.1308351457118988), ('skip_connect', 0, 0.1307944804430008), ('sep_conv_5x5', 3, 0.12830030918121338), ('sep_conv_3x3', 2, 0.12825161218643188), ('sep_conv_3x3', 4, 0.1281294971704483), ('skip_connect', 1, 0.12808074057102203)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1, 0.12622205913066864), ('sep_conv_5x5', 0, 0.12572790682315826), ('max_pool_3x3', 0, 0.12680518627166748), ('dil_conv_5x5', 2, 0.12661783397197723), ('sep_conv_5x5', 0, 0.1270311325788498), ('dil_conv_5x5', 2, 0.1266404688358307), ('sep_conv_5x5', 1, 0.12642508745193481), ('dil_conv_5x5', 4, 0.12617145478725433)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-20:15:44] Epoch: [221][000/196] Time 1.72 (1.72) Data 1.10 (1.10) Loss 0.965 (0.965) Prec@1 69.53 (69.53) Prec@5 96.88 (96.88) TRAIN-SEARCH [2018-10-02-20:16:37] Epoch: [221][100/196] Time 0.55 (0.54) Data 0.00 (0.01) Loss 0.711 (0.808) Prec@1 72.66 (71.53) Prec@5 98.44 (98.04) TRAIN-SEARCH [2018-10-02-20:17:27] Epoch: [221][195/196] Time 0.46 (0.54) Data 0.00 (0.01) Loss 0.686 (0.808) Prec@1 67.50 (71.81) Prec@5 100.00 (98.00) VALID-SEARCH [2018-10-02-20:17:28] Epoch: [221][000/196] Loss 1.444 (1.444) Prec@1 57.03 (57.03) Prec@5 96.09 (96.09) VALID-SEARCH [2018-10-02-20:17:34] Epoch: [221][100/196] Loss 1.026 (1.310) Prec@1 60.16 (58.18) Prec@5 97.66 (93.85) VALID-SEARCH [2018-10-02-20:17:39] Epoch: [221][195/196] Loss 0.790 (1.295) Prec@1 75.00 (57.86) Prec@5 100.00 (93.63) 221/250, Train-Accuracy = 71.81, Test-Accuracy = 57.86 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-20:17:39] [Epoch=222/250] [Need: 00:55:11] [LR=0.0025 ~ 0.0025] [Batch=128], tau=4.672 genotype = Genotype(normal=[('sep_conv_5x5', 0, 0.13050539791584015), ('dil_conv_3x3', 1, 0.12885603308677673), ('skip_connect', 0, 0.13090252876281738), ('sep_conv_3x3', 2, 0.13023947179317474), ('sep_conv_5x5', 3, 0.1285361498594284), ('dil_conv_5x5', 2, 0.1284814327955246), ('sep_conv_3x3', 4, 0.12803387641906738), ('sep_conv_5x5', 2, 0.12787653505802155)], normal_concat=range(2, 6), reduce=[('sep_conv_3x3', 1, 0.1262102872133255), ('sep_conv_5x5', 0, 0.12560555338859558), ('sep_conv_5x5', 0, 0.12713229656219482), ('dil_conv_5x5', 2, 0.12703076004981995), ('dil_conv_5x5', 2, 0.12738938629627228), ('max_pool_3x3', 0, 0.12646137177944183), ('sep_conv_5x5', 1, 0.12633702158927917), ('dil_conv_3x3', 4, 0.12599581480026245)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-20:17:42] Epoch: [222][000/196] Time 1.42 (1.42) Data 0.83 (0.83) Loss 0.831 (0.831) Prec@1 70.31 (70.31) Prec@5 98.44 (98.44) TRAIN-SEARCH [2018-10-02-20:18:35] Epoch: [222][100/196] Time 0.58 (0.54) Data 0.00 (0.01) Loss 0.918 (0.795) Prec@1 70.31 (72.38) Prec@5 98.44 (98.12) TRAIN-SEARCH [2018-10-02-20:19:27] Epoch: [222][195/196] Time 0.41 (0.55) Data 0.00 (0.01) Loss 1.082 (0.797) Prec@1 52.50 (72.23) Prec@5 97.50 (98.11) VALID-SEARCH [2018-10-02-20:19:29] Epoch: [222][000/196] Loss 1.448 (1.448) Prec@1 47.66 (47.66) Prec@5 87.50 (87.50) VALID-SEARCH [2018-10-02-20:19:34] Epoch: [222][100/196] Loss 1.032 (1.207) Prec@1 69.53 (61.21) Prec@5 98.44 (95.37) VALID-SEARCH [2018-10-02-20:19:39] Epoch: [222][195/196] Loss 1.156 (1.209) Prec@1 65.00 (61.25) Prec@5 95.00 (95.19) 222/250, Train-Accuracy = 72.23, Test-Accuracy = 61.25 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-20:19:39] [Epoch=223/250] [Need: 00:53:55] [LR=0.0024 ~ 0.0024] [Batch=128], tau=4.648 genotype = Genotype(normal=[('sep_conv_5x5', 0, 0.13011202216148376), ('dil_conv_3x3', 1, 0.12831199169158936), ('skip_connect', 0, 0.13050474226474762), ('sep_conv_3x3', 2, 0.13029666244983673), ('skip_connect', 0, 0.12880350649356842), ('dil_conv_5x5', 2, 0.12816983461380005), ('sep_conv_3x3', 4, 0.12826913595199585), ('skip_connect', 1, 0.12824608385562897)], normal_concat=range(2, 6), reduce=[('dil_conv_5x5', 1, 0.12696895003318787), ('sep_conv_5x5', 0, 0.12654659152030945), ('sep_conv_5x5', 0, 0.12704767286777496), ('dil_conv_5x5', 2, 0.12678168714046478), ('dil_conv_5x5', 2, 0.12663160264492035), ('max_pool_3x3', 0, 0.1261935979127884), ('sep_conv_5x5', 1, 0.12718260288238525), ('dil_conv_3x3', 4, 0.12652970850467682)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-20:19:42] Epoch: [223][000/196] Time 1.58 (1.58) Data 0.91 (0.91) Loss 0.914 (0.914) Prec@1 68.75 (68.75) Prec@5 96.09 (96.09) TRAIN-SEARCH [2018-10-02-20:20:34] Epoch: [223][100/196] Time 0.45 (0.53) Data 0.00 (0.01) Loss 0.718 (0.791) Prec@1 75.78 (72.35) Prec@5 97.66 (98.04) TRAIN-SEARCH [2018-10-02-20:21:26] Epoch: [223][195/196] Time 0.45 (0.54) Data 0.00 (0.01) Loss 0.801 (0.790) Prec@1 70.00 (72.23) Prec@5 95.00 (98.12) VALID-SEARCH [2018-10-02-20:21:27] Epoch: [223][000/196] Loss 1.077 (1.077) Prec@1 64.06 (64.06) Prec@5 96.88 (96.88) VALID-SEARCH [2018-10-02-20:21:33] Epoch: [223][100/196] Loss 1.124 (1.303) Prec@1 60.94 (57.60) Prec@5 96.09 (93.54) VALID-SEARCH [2018-10-02-20:21:38] Epoch: [223][195/196] Loss 1.846 (1.259) Prec@1 47.50 (58.88) Prec@5 90.00 (94.02) 223/250, Train-Accuracy = 72.23, Test-Accuracy = 58.88 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-20:21:39] [Epoch=224/250] [Need: 00:51:51] [LR=0.0023 ~ 0.0023] [Batch=128], tau=4.624 genotype = Genotype(normal=[('sep_conv_5x5', 0, 0.1294565051794052), ('dil_conv_3x3', 1, 0.12826985120773315), ('skip_connect', 0, 0.13099294900894165), ('sep_conv_3x3', 2, 0.13071131706237793), ('sep_conv_5x5', 3, 0.12871964275836945), ('skip_connect', 0, 0.12821586430072784), ('skip_connect', 1, 0.12842124700546265), ('sep_conv_3x3', 4, 0.12816506624221802)], normal_concat=range(2, 6), reduce=[('dil_conv_5x5', 1, 0.1269267499446869), ('sep_conv_5x5', 0, 0.12643712759017944), ('sep_conv_5x5', 2, 0.12660060822963715), ('sep_conv_5x5', 0, 0.12654560804367065), ('dil_conv_5x5', 2, 0.12653689086437225), ('sep_conv_5x5', 0, 0.12637406587600708), ('sep_conv_5x5', 1, 0.12683580815792084), ('sep_conv_5x5', 2, 0.12641409039497375)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-20:21:41] Epoch: [224][000/196] Time 1.60 (1.60) Data 0.86 (0.86) Loss 0.759 (0.759) Prec@1 75.00 (75.00) Prec@5 99.22 (99.22) TRAIN-SEARCH [2018-10-02-20:22:32] Epoch: [224][100/196] Time 0.53 (0.52) Data 0.00 (0.01) Loss 1.002 (0.779) Prec@1 68.75 (72.85) Prec@5 97.66 (98.42) TRAIN-SEARCH [2018-10-02-20:23:26] Epoch: [224][195/196] Time 0.40 (0.54) Data 0.00 (0.01) Loss 1.141 (0.785) Prec@1 62.50 (72.67) Prec@5 95.00 (98.26) VALID-SEARCH [2018-10-02-20:23:28] Epoch: [224][000/196] Loss 1.244 (1.244) Prec@1 54.69 (54.69) Prec@5 92.97 (92.97) VALID-SEARCH [2018-10-02-20:23:33] Epoch: [224][100/196] Loss 1.686 (1.153) Prec@1 46.09 (60.57) Prec@5 81.25 (95.04) VALID-SEARCH [2018-10-02-20:23:37] Epoch: [224][195/196] Loss 1.308 (1.209) Prec@1 57.50 (59.25) Prec@5 90.00 (94.48) 224/250, Train-Accuracy = 72.67, Test-Accuracy = 59.25 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-20:23:38] [Epoch=225/250] [Need: 00:49:42] [LR=0.0022 ~ 0.0022] [Batch=128], tau=4.6 genotype = Genotype(normal=[('sep_conv_5x5', 0, 0.1299455612897873), ('dil_conv_3x3', 1, 0.12773357331752777), ('skip_connect', 0, 0.13042370975017548), ('sep_conv_3x3', 2, 0.13009700179100037), ('skip_connect', 0, 0.12832415103912354), ('sep_conv_5x5', 3, 0.12804445624351501), ('skip_connect', 1, 0.12936553359031677), ('sep_conv_3x3', 4, 0.12860047817230225)], normal_concat=range(2, 6), reduce=[('dil_conv_5x5', 1, 0.1271599382162094), ('sep_conv_3x3', 0, 0.1262023150920868), ('max_pool_3x3', 0, 0.12642259895801544), ('dil_conv_5x5', 2, 0.1260651797056198), ('dil_conv_5x5', 1, 0.12648896872997284), ('sep_conv_5x5', 0, 0.12641561031341553), ('sep_conv_5x5', 4, 0.12699927389621735), ('sep_conv_3x3', 0, 0.12670587003231049)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-20:23:41] Epoch: [225][000/196] Time 1.76 (1.76) Data 1.08 (1.08) Loss 0.722 (0.722) Prec@1 72.66 (72.66) Prec@5 99.22 (99.22) TRAIN-SEARCH [2018-10-02-20:24:34] Epoch: [225][100/196] Time 0.71 (0.55) Data 0.02 (0.01) Loss 0.841 (0.778) Prec@1 71.09 (72.54) Prec@5 99.22 (98.45) TRAIN-SEARCH [2018-10-02-20:25:29] Epoch: [225][195/196] Time 0.58 (0.56) Data 0.00 (0.01) Loss 0.757 (0.797) Prec@1 72.50 (71.97) Prec@5 92.50 (98.22) VALID-SEARCH [2018-10-02-20:25:31] Epoch: [225][000/196] Loss 1.050 (1.050) Prec@1 71.09 (71.09) Prec@5 95.31 (95.31) VALID-SEARCH [2018-10-02-20:25:36] Epoch: [225][100/196] Loss 0.871 (1.212) Prec@1 75.78 (60.21) Prec@5 96.09 (94.10) VALID-SEARCH [2018-10-02-20:25:41] Epoch: [225][195/196] Loss 1.109 (1.226) Prec@1 55.00 (59.64) Prec@5 100.00 (94.08) 225/250, Train-Accuracy = 71.97, Test-Accuracy = 59.64 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-20:25:41] [Epoch=226/250] [Need: 00:49:15] [LR=0.0021 ~ 0.0021] [Batch=128], tau=4.576 genotype = Genotype(normal=[('sep_conv_5x5', 0, 0.13078807294368744), ('dil_conv_3x3', 1, 0.12878838181495667), ('sep_conv_3x3', 2, 0.13022653758525848), ('skip_connect', 0, 0.1301712840795517), ('sep_conv_5x5', 3, 0.12803754210472107), ('skip_connect', 0, 0.12799109518527985), ('skip_connect', 1, 0.12884487211704254), ('sep_conv_5x5', 3, 0.12821988761425018)], normal_concat=range(2, 6), reduce=[('sep_conv_5x5', 0, 0.12664170563220978), ('sep_conv_3x3', 1, 0.12634873390197754), ('max_pool_3x3', 0, 0.12654677033424377), ('dil_conv_3x3', 2, 0.12653596699237823), ('sep_conv_5x5', 0, 0.12689292430877686), ('sep_conv_5x5', 2, 0.12649381160736084), ('sep_conv_3x3', 0, 0.127121239900589), ('dil_conv_5x5', 2, 0.1263527274131775)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-20:25:44] Epoch: [226][000/196] Time 1.58 (1.58) Data 0.98 (0.98) Loss 0.596 (0.596) Prec@1 82.81 (82.81) Prec@5 98.44 (98.44) TRAIN-SEARCH [2018-10-02-20:26:34] Epoch: [226][100/196] Time 0.58 (0.51) Data 0.01 (0.01) Loss 0.812 (0.787) Prec@1 68.75 (71.94) Prec@5 97.66 (98.15) TRAIN-SEARCH [2018-10-02-20:27:28] Epoch: [226][195/196] Time 0.63 (0.54) Data 0.00 (0.01) Loss 0.868 (0.793) Prec@1 65.00 (72.17) Prec@5 97.50 (98.12) VALID-SEARCH [2018-10-02-20:27:29] Epoch: [226][000/196] Loss 1.187 (1.187) Prec@1 56.25 (56.25) Prec@5 92.97 (92.97) VALID-SEARCH [2018-10-02-20:27:34] Epoch: [226][100/196] Loss 1.177 (1.224) Prec@1 56.25 (59.86) Prec@5 93.75 (94.44) VALID-SEARCH [2018-10-02-20:27:39] Epoch: [226][195/196] Loss 0.719 (1.212) Prec@1 72.50 (60.36) Prec@5 100.00 (94.70) 226/250, Train-Accuracy = 72.17, Test-Accuracy = 60.36 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-20:27:40] [Epoch=227/250] [Need: 00:45:28] [LR=0.0020 ~ 0.0020] [Batch=128], tau=4.552 genotype = Genotype(normal=[('sep_conv_5x5', 0, 0.13066060841083527), ('dil_conv_3x3', 1, 0.12782372534275055), ('sep_conv_3x3', 2, 0.13085676729679108), ('skip_connect', 0, 0.1299223154783249), ('sep_conv_3x3', 3, 0.12825429439544678), ('sep_conv_5x5', 0, 0.12788264453411102), ('skip_connect', 1, 0.12859535217285156), ('sep_conv_5x5', 2, 0.12827108800411224)], normal_concat=range(2, 6), reduce=[('sep_conv_5x5', 0, 0.1271331012248993), ('sep_conv_3x3', 1, 0.12687869369983673), ('dil_conv_5x5', 1, 0.1265638917684555), ('dil_conv_3x3', 2, 0.12630702555179596), ('sep_conv_5x5', 0, 0.12626878917217255), ('sep_conv_3x3', 3, 0.12624309957027435), ('sep_conv_3x3', 0, 0.12711697816848755), ('sep_conv_5x5', 1, 0.12631818652153015)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-20:27:42] Epoch: [227][000/196] Time 1.25 (1.25) Data 0.78 (0.78) Loss 0.654 (0.654) Prec@1 79.69 (79.69) Prec@5 99.22 (99.22) TRAIN-SEARCH [2018-10-02-20:28:34] Epoch: [227][100/196] Time 0.66 (0.53) Data 0.00 (0.01) Loss 0.701 (0.778) Prec@1 76.56 (72.52) Prec@5 97.66 (98.12) TRAIN-SEARCH [2018-10-02-20:29:28] Epoch: [227][195/196] Time 0.50 (0.55) Data 0.00 (0.01) Loss 0.934 (0.786) Prec@1 62.50 (72.52) Prec@5 100.00 (98.13) VALID-SEARCH [2018-10-02-20:29:29] Epoch: [227][000/196] Loss 1.198 (1.198) Prec@1 64.84 (64.84) Prec@5 95.31 (95.31) VALID-SEARCH [2018-10-02-20:29:33] Epoch: [227][100/196] Loss 1.042 (1.294) Prec@1 64.06 (60.72) Prec@5 96.09 (94.69) VALID-SEARCH [2018-10-02-20:29:37] Epoch: [227][195/196] Loss 0.902 (1.256) Prec@1 62.50 (61.27) Prec@5 97.50 (94.97) 227/250, Train-Accuracy = 72.52, Test-Accuracy = 61.27 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-20:29:39] [Epoch=228/250] [Need: 00:43:37] [LR=0.0019 ~ 0.0019] [Batch=128], tau=4.528 genotype = Genotype(normal=[('sep_conv_5x5', 0, 0.13043957948684692), ('sep_conv_5x5', 1, 0.12805546820163727), ('sep_conv_3x3', 2, 0.13079725205898285), ('skip_connect', 0, 0.13026973605155945), ('sep_conv_3x3', 3, 0.12869836390018463), ('sep_conv_3x3', 2, 0.12772460281848907), ('skip_connect', 1, 0.12821738421916962), ('sep_conv_5x5', 2, 0.12797017395496368)], normal_concat=range(2, 6), reduce=[('sep_conv_5x5', 0, 0.1274169236421585), ('sep_conv_3x3', 1, 0.12706348299980164), ('sep_conv_5x5', 2, 0.12654882669448853), ('dil_conv_5x5', 1, 0.12599323689937592), ('sep_conv_5x5', 0, 0.12666195631027222), ('sep_conv_5x5', 2, 0.12655051052570343), ('sep_conv_3x3', 0, 0.1272488236427307), ('sep_conv_3x3', 3, 0.1263679713010788)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-20:29:41] Epoch: [228][000/196] Time 1.57 (1.57) Data 0.89 (0.89) Loss 0.776 (0.776) Prec@1 75.00 (75.00) Prec@5 99.22 (99.22) TRAIN-SEARCH [2018-10-02-20:30:32] Epoch: [228][100/196] Time 0.50 (0.52) Data 0.00 (0.01) Loss 0.681 (0.785) Prec@1 77.34 (72.58) Prec@5 98.44 (98.02) TRAIN-SEARCH [2018-10-02-20:31:23] Epoch: [228][195/196] Time 0.38 (0.53) Data 0.00 (0.01) Loss 0.839 (0.780) Prec@1 65.00 (72.64) Prec@5 100.00 (98.10) VALID-SEARCH [2018-10-02-20:31:24] Epoch: [228][000/196] Loss 1.306 (1.306) Prec@1 63.28 (63.28) Prec@5 95.31 (95.31) VALID-SEARCH [2018-10-02-20:31:29] Epoch: [228][100/196] Loss 1.025 (1.239) Prec@1 64.06 (59.65) Prec@5 96.88 (94.21) VALID-SEARCH [2018-10-02-20:31:34] Epoch: [228][195/196] Loss 0.952 (1.267) Prec@1 62.50 (58.74) Prec@5 97.50 (93.82) 228/250, Train-Accuracy = 72.64, Test-Accuracy = 58.74 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-20:31:35] [Epoch=229/250] [Need: 00:40:38] [LR=0.0018 ~ 0.0018] [Batch=128], tau=4.504 genotype = Genotype(normal=[('dil_conv_3x3', 0, 0.12992256879806519), ('sep_conv_5x5', 1, 0.1275760680437088), ('skip_connect', 0, 0.1306218057870865), ('sep_conv_3x3', 2, 0.13058936595916748), ('sep_conv_5x5', 3, 0.1288730651140213), ('skip_connect', 1, 0.12770122289657593), ('sep_conv_5x5', 2, 0.12819741666316986), ('sep_conv_3x3', 4, 0.12765997648239136)], normal_concat=range(2, 6), reduce=[('sep_conv_5x5', 0, 0.1275259256362915), ('sep_conv_3x3', 1, 0.1258917599916458), ('max_pool_3x3', 0, 0.1271669566631317), ('sep_conv_5x5', 2, 0.1270059496164322), ('sep_conv_5x5', 0, 0.1276814043521881), ('sep_conv_5x5', 2, 0.12632092833518982), ('sep_conv_3x3', 0, 0.12680268287658691), ('sep_conv_5x5', 1, 0.12639759480953217)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-20:31:37] Epoch: [229][000/196] Time 1.46 (1.46) Data 0.87 (0.87) Loss 0.668 (0.668) Prec@1 78.91 (78.91) Prec@5 97.66 (97.66) TRAIN-SEARCH [2018-10-02-20:32:31] Epoch: [229][100/196] Time 0.59 (0.55) Data 0.00 (0.01) Loss 0.721 (0.780) Prec@1 71.88 (72.51) Prec@5 100.00 (98.07) TRAIN-SEARCH [2018-10-02-20:33:23] Epoch: [229][195/196] Time 0.53 (0.54) Data 0.00 (0.01) Loss 0.896 (0.779) Prec@1 70.00 (72.51) Prec@5 97.50 (98.10) VALID-SEARCH [2018-10-02-20:33:24] Epoch: [229][000/196] Loss 0.952 (0.952) Prec@1 60.94 (60.94) Prec@5 97.66 (97.66) VALID-SEARCH [2018-10-02-20:33:29] Epoch: [229][100/196] Loss 0.979 (1.164) Prec@1 66.41 (60.70) Prec@5 97.66 (94.90) VALID-SEARCH [2018-10-02-20:33:34] Epoch: [229][195/196] Loss 1.581 (1.201) Prec@1 52.50 (60.82) Prec@5 100.00 (95.05) 229/250, Train-Accuracy = 72.51, Test-Accuracy = 60.82 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-20:33:35] [Epoch=230/250] [Need: 00:39:55] [LR=0.0018 ~ 0.0018] [Batch=128], tau=4.4799999999999995 genotype = Genotype(normal=[('dil_conv_3x3', 0, 0.13016614317893982), ('sep_conv_5x5', 1, 0.12784618139266968), ('skip_connect', 0, 0.13048100471496582), ('sep_conv_3x3', 2, 0.13010385632514954), ('sep_conv_5x5', 3, 0.12827780842781067), ('skip_connect', 0, 0.12809687852859497), ('sep_conv_5x5', 2, 0.1282237023115158), ('sep_conv_3x3', 3, 0.1273658275604248)], normal_concat=range(2, 6), reduce=[('sep_conv_5x5', 0, 0.12741340696811676), ('dil_conv_5x5', 1, 0.12604451179504395), ('max_pool_3x3', 0, 0.1270783245563507), ('sep_conv_5x5', 2, 0.12652075290679932), ('sep_conv_5x5', 0, 0.12719790637493134), ('sep_conv_5x5', 2, 0.12692071497440338), ('sep_conv_5x5', 1, 0.12668293714523315), ('sep_conv_5x5', 4, 0.1266665905714035)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-20:33:37] Epoch: [230][000/196] Time 0.95 (0.95) Data 0.55 (0.55) Loss 0.773 (0.773) Prec@1 73.44 (73.44) Prec@5 99.22 (99.22) TRAIN-SEARCH [2018-10-02-20:34:32] Epoch: [230][100/196] Time 0.49 (0.55) Data 0.00 (0.01) Loss 0.784 (0.778) Prec@1 72.66 (72.88) Prec@5 100.00 (98.31) TRAIN-SEARCH [2018-10-02-20:35:23] Epoch: [230][195/196] Time 0.56 (0.55) Data 0.00 (0.01) Loss 0.535 (0.777) Prec@1 72.50 (72.80) Prec@5 100.00 (98.24) VALID-SEARCH [2018-10-02-20:35:24] Epoch: [230][000/196] Loss 0.946 (0.946) Prec@1 73.44 (73.44) Prec@5 94.53 (94.53) VALID-SEARCH [2018-10-02-20:35:29] Epoch: [230][100/196] Loss 0.820 (1.172) Prec@1 67.97 (60.35) Prec@5 100.00 (94.75) VALID-SEARCH [2018-10-02-20:35:33] Epoch: [230][195/196] Loss 0.950 (1.176) Prec@1 72.50 (60.10) Prec@5 92.50 (94.58) 230/250, Train-Accuracy = 72.80, Test-Accuracy = 60.10 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-20:35:34] [Epoch=231/250] [Need: 00:37:35] [LR=0.0017 ~ 0.0017] [Batch=128], tau=4.4559999999999995 genotype = Genotype(normal=[('dil_conv_5x5', 0, 0.12948866188526154), ('sep_conv_5x5', 1, 0.1281318962574005), ('sep_conv_3x3', 2, 0.12999196350574493), ('skip_connect', 0, 0.12960608303546906), ('sep_conv_3x3', 3, 0.12858346104621887), ('skip_connect', 0, 0.12814456224441528), ('skip_connect', 0, 0.12859205901622772), ('sep_conv_3x3', 3, 0.12847234308719635)], normal_concat=range(2, 6), reduce=[('sep_conv_5x5', 0, 0.1266694962978363), ('dil_conv_5x5', 1, 0.1258409023284912), ('sep_conv_5x5', 1, 0.1268012821674347), ('dil_conv_5x5', 2, 0.1266760528087616), ('dil_conv_5x5', 2, 0.1269780695438385), ('sep_conv_5x5', 0, 0.12633895874023438), ('sep_conv_5x5', 1, 0.12717430293560028), ('sep_conv_5x5', 4, 0.12661877274513245)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-20:35:35] Epoch: [231][000/196] Time 1.00 (1.00) Data 0.59 (0.59) Loss 0.673 (0.673) Prec@1 74.22 (74.22) Prec@5 99.22 (99.22) TRAIN-SEARCH [2018-10-02-20:36:31] Epoch: [231][100/196] Time 0.48 (0.57) Data 0.01 (0.01) Loss 0.775 (0.793) Prec@1 71.88 (72.35) Prec@5 97.66 (97.96) TRAIN-SEARCH [2018-10-02-20:37:21] Epoch: [231][195/196] Time 0.41 (0.55) Data 0.00 (0.01) Loss 0.853 (0.779) Prec@1 77.50 (72.77) Prec@5 97.50 (98.16) VALID-SEARCH [2018-10-02-20:37:23] Epoch: [231][000/196] Loss 0.928 (0.928) Prec@1 64.84 (64.84) Prec@5 99.22 (99.22) VALID-SEARCH [2018-10-02-20:37:27] Epoch: [231][100/196] Loss 1.172 (1.119) Prec@1 64.06 (62.87) Prec@5 96.09 (95.34) VALID-SEARCH [2018-10-02-20:37:31] Epoch: [231][195/196] Loss 0.888 (1.140) Prec@1 62.50 (62.40) Prec@5 97.50 (95.33) 231/250, Train-Accuracy = 72.77, Test-Accuracy = 62.40 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-20:37:32] [Epoch=232/250] [Need: 00:35:28] [LR=0.0016 ~ 0.0016] [Batch=128], tau=4.4319999999999995 genotype = Genotype(normal=[('dil_conv_5x5', 0, 0.130092054605484), ('sep_conv_5x5', 1, 0.12751315534114838), ('sep_conv_3x3', 2, 0.1297224760055542), ('skip_connect', 0, 0.12964364886283875), ('sep_conv_3x3', 3, 0.1288406252861023), ('skip_connect', 0, 0.1280793994665146), ('sep_conv_3x3', 3, 0.12848863005638123), ('skip_connect', 1, 0.12767094373703003)], normal_concat=range(2, 6), reduce=[('sep_conv_5x5', 0, 0.12640954554080963), ('dil_conv_5x5', 1, 0.1258571445941925), ('sep_conv_5x5', 1, 0.12698900699615479), ('dil_conv_3x3', 2, 0.12685371935367584), ('sep_conv_5x5', 2, 0.1269201785326004), ('sep_conv_5x5', 0, 0.12640203535556793), ('sep_conv_5x5', 1, 0.1269993782043457), ('sep_conv_5x5', 4, 0.12679512798786163)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-20:37:33] Epoch: [232][000/196] Time 1.04 (1.04) Data 0.61 (0.61) Loss 0.822 (0.822) Prec@1 70.31 (70.31) Prec@5 96.88 (96.88) TRAIN-SEARCH [2018-10-02-20:38:29] Epoch: [232][100/196] Time 0.39 (0.56) Data 0.00 (0.01) Loss 0.762 (0.758) Prec@1 75.00 (73.62) Prec@5 96.88 (98.31) TRAIN-SEARCH [2018-10-02-20:39:22] Epoch: [232][195/196] Time 0.31 (0.56) Data 0.00 (0.01) Loss 0.731 (0.766) Prec@1 77.50 (73.40) Prec@5 97.50 (98.23) VALID-SEARCH [2018-10-02-20:39:23] Epoch: [232][000/196] Loss 0.995 (0.995) Prec@1 66.41 (66.41) Prec@5 97.66 (97.66) VALID-SEARCH [2018-10-02-20:39:27] Epoch: [232][100/196] Loss 1.179 (1.269) Prec@1 55.47 (57.94) Prec@5 97.66 (93.01) VALID-SEARCH [2018-10-02-20:39:30] Epoch: [232][195/196] Loss 0.947 (1.231) Prec@1 67.50 (58.66) Prec@5 92.50 (93.67) 232/250, Train-Accuracy = 73.40, Test-Accuracy = 58.66 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-20:39:31] [Epoch=233/250] [Need: 00:33:49] [LR=0.0016 ~ 0.0016] [Batch=128], tau=4.4079999999999995 genotype = Genotype(normal=[('dil_conv_5x5', 0, 0.130025252699852), ('sep_conv_5x5', 1, 0.12652631103992462), ('sep_conv_3x3', 2, 0.13008424639701843), ('skip_connect', 0, 0.12924139201641083), ('sep_conv_3x3', 3, 0.1292622834444046), ('skip_connect', 0, 0.12794628739356995), ('sep_conv_5x5', 2, 0.128486767411232), ('sep_conv_3x3', 3, 0.12821412086486816)], normal_concat=range(2, 6), reduce=[('sep_conv_5x5', 0, 0.12682229280471802), ('sep_conv_3x3', 1, 0.1256691813468933), ('sep_conv_5x5', 2, 0.12653669714927673), ('sep_conv_5x5', 1, 0.12641246616840363), ('sep_conv_5x5', 2, 0.12639562785625458), ('dil_conv_3x3', 3, 0.12638896703720093), ('sep_conv_3x3', 1, 0.12658613920211792), ('sep_conv_5x5', 4, 0.1264052838087082)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-20:39:34] Epoch: [233][000/196] Time 1.72 (1.72) Data 1.02 (1.02) Loss 0.557 (0.557) Prec@1 83.59 (83.59) Prec@5 99.22 (99.22) TRAIN-SEARCH [2018-10-02-20:40:28] Epoch: [233][100/196] Time 0.48 (0.55) Data 0.00 (0.01) Loss 0.834 (0.758) Prec@1 70.31 (73.47) Prec@5 98.44 (98.47) TRAIN-SEARCH [2018-10-02-20:41:18] Epoch: [233][195/196] Time 0.34 (0.54) Data 0.00 (0.01) Loss 0.841 (0.769) Prec@1 67.50 (73.20) Prec@5 100.00 (98.30) VALID-SEARCH [2018-10-02-20:41:19] Epoch: [233][000/196] Loss 0.902 (0.902) Prec@1 67.19 (67.19) Prec@5 95.31 (95.31) VALID-SEARCH [2018-10-02-20:41:24] Epoch: [233][100/196] Loss 1.115 (1.213) Prec@1 61.72 (59.72) Prec@5 97.66 (94.17) VALID-SEARCH [2018-10-02-20:41:29] Epoch: [233][195/196] Loss 0.828 (1.248) Prec@1 70.00 (59.33) Prec@5 92.50 (93.89) 233/250, Train-Accuracy = 73.20, Test-Accuracy = 59.33 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-20:41:30] [Epoch=234/250] [Need: 00:31:40] [LR=0.0015 ~ 0.0015] [Batch=128], tau=4.3839999999999995 genotype = Genotype(normal=[('sep_conv_5x5', 0, 0.12931451201438904), ('sep_conv_3x3', 1, 0.12670905888080597), ('sep_conv_3x3', 2, 0.13017013669013977), ('skip_connect', 0, 0.12887245416641235), ('sep_conv_3x3', 3, 0.12883856892585754), ('skip_connect', 0, 0.1277233213186264), ('sep_conv_3x3', 3, 0.1282474547624588), ('sep_conv_5x5', 2, 0.12785875797271729)], normal_concat=range(2, 6), reduce=[('sep_conv_5x5', 0, 0.12675261497497559), ('dil_conv_5x5', 1, 0.12572279572486877), ('sep_conv_5x5', 2, 0.12652578949928284), ('max_pool_3x3', 0, 0.12624937295913696), ('sep_conv_5x5', 1, 0.1272280216217041), ('sep_conv_5x5', 0, 0.12653394043445587), ('sep_conv_5x5', 4, 0.12698215246200562), ('skip_connect', 0, 0.12662069499492645)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-20:41:33] Epoch: [234][000/196] Time 1.69 (1.69) Data 1.12 (1.12) Loss 0.813 (0.813) Prec@1 64.84 (64.84) Prec@5 100.00 (100.00) TRAIN-SEARCH [2018-10-02-20:42:27] Epoch: [234][100/196] Time 0.62 (0.55) Data 0.00 (0.01) Loss 0.774 (0.766) Prec@1 72.66 (73.20) Prec@5 98.44 (98.44) TRAIN-SEARCH [2018-10-02-20:43:16] Epoch: [234][195/196] Time 0.31 (0.54) Data 0.00 (0.01) Loss 0.852 (0.773) Prec@1 70.00 (72.88) Prec@5 100.00 (98.32) VALID-SEARCH [2018-10-02-20:43:17] Epoch: [234][000/196] Loss 0.903 (0.903) Prec@1 67.97 (67.97) Prec@5 96.09 (96.09) VALID-SEARCH [2018-10-02-20:43:21] Epoch: [234][100/196] Loss 1.300 (1.252) Prec@1 58.59 (59.01) Prec@5 93.75 (93.82) VALID-SEARCH [2018-10-02-20:43:26] Epoch: [234][195/196] Loss 0.934 (1.279) Prec@1 65.00 (58.19) Prec@5 97.50 (93.61) 234/250, Train-Accuracy = 72.88, Test-Accuracy = 58.19 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-20:43:27] [Epoch=235/250] [Need: 00:29:16] [LR=0.0014 ~ 0.0014] [Batch=128], tau=4.36 genotype = Genotype(normal=[('sep_conv_5x5', 0, 0.1303028166294098), ('dil_conv_3x3', 1, 0.12757430970668793), ('sep_conv_5x5', 2, 0.13025134801864624), ('skip_connect', 0, 0.12904156744480133), ('sep_conv_5x5', 2, 0.1282132863998413), ('sep_conv_3x3', 3, 0.12820112705230713), ('sep_conv_5x5', 2, 0.12823618948459625), ('sep_conv_3x3', 3, 0.12780408561229706)], normal_concat=range(2, 6), reduce=[('sep_conv_5x5', 0, 0.1267901062965393), ('dil_conv_5x5', 1, 0.12613558769226074), ('sep_conv_5x5', 2, 0.12705691158771515), ('avg_pool_3x3', 0, 0.1259865015745163), ('sep_conv_5x5', 1, 0.1267051249742508), ('sep_conv_5x5', 2, 0.12668254971504211), ('sep_conv_5x5', 4, 0.12657609581947327), ('sep_conv_3x3', 3, 0.1262504905462265)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-20:43:29] Epoch: [235][000/196] Time 1.61 (1.61) Data 0.93 (0.93) Loss 0.663 (0.663) Prec@1 76.56 (76.56) Prec@5 100.00 (100.00) TRAIN-SEARCH [2018-10-02-20:44:25] Epoch: [235][100/196] Time 0.49 (0.56) Data 0.00 (0.01) Loss 1.018 (0.773) Prec@1 70.31 (72.84) Prec@5 96.09 (98.17) TRAIN-SEARCH [2018-10-02-20:45:14] Epoch: [235][195/196] Time 0.37 (0.54) Data 0.00 (0.01) Loss 0.659 (0.779) Prec@1 72.50 (72.59) Prec@5 97.50 (98.21) VALID-SEARCH [2018-10-02-20:45:15] Epoch: [235][000/196] Loss 1.699 (1.699) Prec@1 42.97 (42.97) Prec@5 85.94 (85.94) VALID-SEARCH [2018-10-02-20:45:20] Epoch: [235][100/196] Loss 1.168 (1.301) Prec@1 62.50 (56.85) Prec@5 95.31 (92.84) VALID-SEARCH [2018-10-02-20:45:25] Epoch: [235][195/196] Loss 0.875 (1.312) Prec@1 80.00 (57.02) Prec@5 97.50 (92.94) 235/250, Train-Accuracy = 72.59, Test-Accuracy = 57.02 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-20:45:26] [Epoch=236/250] [Need: 00:27:40] [LR=0.0014 ~ 0.0014] [Batch=128], tau=4.336 genotype = Genotype(normal=[('sep_conv_5x5', 0, 0.12984666228294373), ('dil_conv_3x3', 1, 0.1277417242527008), ('sep_conv_5x5', 2, 0.1307383030653), ('skip_connect', 0, 0.12865999341011047), ('sep_conv_5x5', 2, 0.12879768013954163), ('sep_conv_3x3', 3, 0.12813255190849304), ('sep_conv_5x5', 2, 0.1280125379562378), ('sep_conv_5x5', 3, 0.12773334980010986)], normal_concat=range(2, 6), reduce=[('sep_conv_5x5', 0, 0.12713657319545746), ('sep_conv_3x3', 1, 0.12570735812187195), ('sep_conv_5x5', 2, 0.12701895833015442), ('sep_conv_3x3', 0, 0.12660560011863708), ('dil_conv_5x5', 3, 0.12672364711761475), ('sep_conv_5x5', 2, 0.1265512853860855), ('dil_conv_5x5', 3, 0.12677618861198425), ('sep_conv_5x5', 4, 0.12616939842700958)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-20:45:28] Epoch: [236][000/196] Time 1.57 (1.57) Data 1.00 (1.00) Loss 0.884 (0.884) Prec@1 68.75 (68.75) Prec@5 96.88 (96.88) TRAIN-SEARCH [2018-10-02-20:46:24] Epoch: [236][100/196] Time 0.48 (0.57) Data 0.00 (0.01) Loss 0.870 (0.774) Prec@1 71.09 (72.97) Prec@5 99.22 (98.29) TRAIN-SEARCH [2018-10-02-20:47:12] Epoch: [236][195/196] Time 0.29 (0.54) Data 0.00 (0.01) Loss 0.560 (0.773) Prec@1 85.00 (73.11) Prec@5 100.00 (98.15) VALID-SEARCH [2018-10-02-20:47:13] Epoch: [236][000/196] Loss 0.959 (0.959) Prec@1 66.41 (66.41) Prec@5 96.88 (96.88) VALID-SEARCH [2018-10-02-20:47:17] Epoch: [236][100/196] Loss 1.194 (1.314) Prec@1 60.94 (57.77) Prec@5 96.09 (92.88) VALID-SEARCH [2018-10-02-20:47:21] Epoch: [236][195/196] Loss 2.523 (1.261) Prec@1 47.50 (59.47) Prec@5 97.50 (93.78) 236/250, Train-Accuracy = 73.11, Test-Accuracy = 59.47 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-20:47:22] [Epoch=237/250] [Need: 00:25:13] [LR=0.0013 ~ 0.0013] [Batch=128], tau=4.312 genotype = Genotype(normal=[('sep_conv_5x5', 0, 0.13073508441448212), ('dil_conv_3x3', 1, 0.12813815474510193), ('sep_conv_5x5', 2, 0.13093528151512146), ('skip_connect', 0, 0.12855952978134155), ('sep_conv_3x3', 3, 0.12856385111808777), ('skip_connect', 0, 0.1279645413160324), ('sep_conv_5x5', 2, 0.12841272354125977), ('max_pool_3x3', 0, 0.1278686225414276)], normal_concat=range(2, 6), reduce=[('sep_conv_5x5', 0, 0.12743474543094635), ('dil_conv_3x3', 1, 0.1256919801235199), ('sep_conv_3x3', 2, 0.12702617049217224), ('sep_conv_3x3', 0, 0.12661562860012054), ('sep_conv_5x5', 0, 0.12675757706165314), ('sep_conv_5x5', 1, 0.12641330063343048), ('dil_conv_5x5', 3, 0.1272110790014267), ('dil_conv_5x5', 1, 0.1262637823820114)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-20:47:25] Epoch: [237][000/196] Time 1.65 (1.65) Data 0.97 (0.97) Loss 0.786 (0.786) Prec@1 72.66 (72.66) Prec@5 98.44 (98.44) TRAIN-SEARCH [2018-10-02-20:48:23] Epoch: [237][100/196] Time 0.58 (0.59) Data 0.00 (0.01) Loss 0.750 (0.777) Prec@1 74.22 (72.74) Prec@5 99.22 (98.19) TRAIN-SEARCH [2018-10-02-20:49:10] Epoch: [237][195/196] Time 0.29 (0.55) Data 0.00 (0.01) Loss 0.544 (0.767) Prec@1 77.50 (73.36) Prec@5 100.00 (98.18) VALID-SEARCH [2018-10-02-20:49:12] Epoch: [237][000/196] Loss 1.047 (1.047) Prec@1 65.62 (65.62) Prec@5 97.66 (97.66) VALID-SEARCH [2018-10-02-20:49:17] Epoch: [237][100/196] Loss 0.796 (1.185) Prec@1 71.09 (60.03) Prec@5 97.66 (94.69) VALID-SEARCH [2018-10-02-20:49:22] Epoch: [237][195/196] Loss 1.057 (1.204) Prec@1 60.00 (59.42) Prec@5 95.00 (94.36) 237/250, Train-Accuracy = 73.36, Test-Accuracy = 59.42 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-20:49:23] [Epoch=238/250] [Need: 00:24:11] [LR=0.0013 ~ 0.0013] [Batch=128], tau=4.288 genotype = Genotype(normal=[('sep_conv_5x5', 0, 0.1308635175228119), ('dil_conv_3x3', 1, 0.12859703600406647), ('sep_conv_5x5', 2, 0.13027796149253845), ('skip_connect', 1, 0.1287413388490677), ('skip_connect', 0, 0.12817800045013428), ('sep_conv_3x3', 3, 0.1280713677406311), ('skip_connect', 1, 0.12744881212711334), ('sep_conv_3x3', 3, 0.12743216753005981)], normal_concat=range(2, 6), reduce=[('sep_conv_5x5', 0, 0.12706241011619568), ('sep_conv_5x5', 1, 0.12589719891548157), ('sep_conv_3x3', 2, 0.12672396004199982), ('avg_pool_3x3', 0, 0.12616313993930817), ('sep_conv_5x5', 0, 0.12721045315265656), ('dil_conv_5x5', 3, 0.12657508254051208), ('dil_conv_5x5', 3, 0.1270100176334381), ('dil_conv_5x5', 4, 0.12642866373062134)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-20:49:25] Epoch: [238][000/196] Time 1.63 (1.63) Data 1.01 (1.01) Loss 0.788 (0.788) Prec@1 71.09 (71.09) Prec@5 96.09 (96.09) TRAIN-SEARCH [2018-10-02-20:50:23] Epoch: [238][100/196] Time 0.69 (0.58) Data 0.00 (0.01) Loss 0.875 (0.769) Prec@1 67.19 (73.04) Prec@5 97.66 (97.98) TRAIN-SEARCH [2018-10-02-20:51:10] Epoch: [238][195/196] Time 0.55 (0.54) Data 0.00 (0.01) Loss 0.914 (0.772) Prec@1 75.00 (72.81) Prec@5 97.50 (98.10) VALID-SEARCH [2018-10-02-20:51:11] Epoch: [238][000/196] Loss 1.181 (1.181) Prec@1 58.59 (58.59) Prec@5 93.75 (93.75) VALID-SEARCH [2018-10-02-20:51:17] Epoch: [238][100/196] Loss 0.914 (1.181) Prec@1 73.44 (61.05) Prec@5 99.22 (95.40) VALID-SEARCH [2018-10-02-20:51:22] Epoch: [238][195/196] Loss 0.736 (1.168) Prec@1 77.50 (61.24) Prec@5 100.00 (95.21) 238/250, Train-Accuracy = 72.81, Test-Accuracy = 61.24 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-20:51:23] [Epoch=239/250] [Need: 00:21:53] [LR=0.0012 ~ 0.0012] [Batch=128], tau=4.264 genotype = Genotype(normal=[('sep_conv_5x5', 0, 0.13019128143787384), ('dil_conv_3x3', 1, 0.1285093128681183), ('sep_conv_5x5', 2, 0.1307787150144577), ('skip_connect', 1, 0.12962201237678528), ('skip_connect', 1, 0.12924903631210327), ('skip_connect', 0, 0.12908804416656494), ('sep_conv_3x3', 3, 0.1283455789089203), ('skip_connect', 1, 0.12771987915039062)], normal_concat=range(2, 6), reduce=[('sep_conv_5x5', 0, 0.1266246736049652), ('sep_conv_5x5', 1, 0.12593922019004822), ('avg_pool_3x3', 0, 0.12663547694683075), ('sep_conv_3x3', 2, 0.12655514478683472), ('sep_conv_5x5', 0, 0.126629039645195), ('dil_conv_5x5', 2, 0.12635600566864014), ('dil_conv_5x5', 3, 0.1266939491033554), ('dil_conv_5x5', 4, 0.12624382972717285)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-20:51:25] Epoch: [239][000/196] Time 1.94 (1.94) Data 1.30 (1.30) Loss 0.999 (0.999) Prec@1 67.19 (67.19) Prec@5 95.31 (95.31) TRAIN-SEARCH [2018-10-02-20:52:24] Epoch: [239][100/196] Time 0.56 (0.60) Data 0.00 (0.02) Loss 0.629 (0.773) Prec@1 78.91 (72.53) Prec@5 97.66 (98.31) TRAIN-SEARCH [2018-10-02-20:53:10] Epoch: [239][195/196] Time 0.38 (0.54) Data 0.00 (0.01) Loss 0.572 (0.775) Prec@1 90.00 (72.60) Prec@5 100.00 (98.24) VALID-SEARCH [2018-10-02-20:53:12] Epoch: [239][000/196] Loss 1.193 (1.193) Prec@1 57.03 (57.03) Prec@5 97.66 (97.66) VALID-SEARCH [2018-10-02-20:53:17] Epoch: [239][100/196] Loss 1.067 (1.202) Prec@1 59.38 (60.59) Prec@5 95.31 (94.46) VALID-SEARCH [2018-10-02-20:53:23] Epoch: [239][195/196] Loss 1.541 (1.262) Prec@1 50.00 (59.59) Prec@5 82.50 (94.15) 239/250, Train-Accuracy = 72.60, Test-Accuracy = 59.59 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-20:53:23] [Epoch=240/250] [Need: 00:20:07] [LR=0.0012 ~ 0.0012] [Batch=128], tau=4.24 genotype = Genotype(normal=[('sep_conv_5x5', 0, 0.13007670640945435), ('dil_conv_3x3', 1, 0.12885339558124542), ('sep_conv_5x5', 2, 0.13039901852607727), ('skip_connect', 1, 0.1288817822933197), ('skip_connect', 1, 0.1293783038854599), ('skip_connect', 0, 0.12932325899600983), ('skip_connect', 1, 0.12788312137126923), ('max_pool_3x3', 0, 0.12786155939102173)], normal_concat=range(2, 6), reduce=[('sep_conv_5x5', 1, 0.12654605507850647), ('sep_conv_5x5', 0, 0.12644316256046295), ('dil_conv_5x5', 2, 0.12793852388858795), ('avg_pool_3x3', 0, 0.12598511576652527), ('sep_conv_5x5', 0, 0.12658877670764923), ('dil_conv_5x5', 2, 0.12634873390197754), ('dil_conv_3x3', 4, 0.12689854204654694), ('sep_conv_3x3', 0, 0.12667669355869293)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-20:53:26] Epoch: [240][000/196] Time 1.55 (1.55) Data 0.91 (0.91) Loss 0.717 (0.717) Prec@1 73.44 (73.44) Prec@5 97.66 (97.66) TRAIN-SEARCH [2018-10-02-20:54:22] Epoch: [240][100/196] Time 0.43 (0.57) Data 0.00 (0.01) Loss 0.589 (0.782) Prec@1 78.12 (72.86) Prec@5 100.00 (98.21) TRAIN-SEARCH [2018-10-02-20:55:11] Epoch: [240][195/196] Time 0.58 (0.55) Data 0.00 (0.01) Loss 1.055 (0.771) Prec@1 65.00 (73.08) Prec@5 97.50 (98.34) VALID-SEARCH [2018-10-02-20:55:12] Epoch: [240][000/196] Loss 1.166 (1.166) Prec@1 58.59 (58.59) Prec@5 96.09 (96.09) VALID-SEARCH [2018-10-02-20:55:18] Epoch: [240][100/196] Loss 1.062 (1.161) Prec@1 69.53 (62.12) Prec@5 97.66 (95.17) VALID-SEARCH [2018-10-02-20:55:22] Epoch: [240][195/196] Loss 1.857 (1.206) Prec@1 42.50 (61.82) Prec@5 77.50 (94.76) 240/250, Train-Accuracy = 73.08, Test-Accuracy = 61.82 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-20:55:23] [Epoch=241/250] [Need: 00:18:00] [LR=0.0012 ~ 0.0012] [Batch=128], tau=4.216 genotype = Genotype(normal=[('sep_conv_5x5', 0, 0.13021205365657806), ('dil_conv_3x3', 1, 0.1289520263671875), ('sep_conv_3x3', 2, 0.13043688237667084), ('skip_connect', 0, 0.12915778160095215), ('sep_conv_3x3', 3, 0.12924537062644958), ('skip_connect', 0, 0.12891489267349243), ('skip_connect', 1, 0.1287916898727417), ('sep_conv_3x3', 3, 0.12831789255142212)], normal_concat=range(2, 6), reduce=[('sep_conv_5x5', 0, 0.12634961307048798), ('sep_conv_5x5', 1, 0.12608830630779266), ('sep_conv_5x5', 0, 0.1268281191587448), ('sep_conv_3x3', 2, 0.126442089676857), ('sep_conv_5x5', 0, 0.1269282102584839), ('sep_conv_3x3', 1, 0.12616007030010223), ('sep_conv_5x5', 1, 0.126699760556221), ('sep_conv_3x3', 0, 0.12645767629146576)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-20:55:25] Epoch: [241][000/196] Time 1.37 (1.37) Data 0.86 (0.86) Loss 0.807 (0.807) Prec@1 69.53 (69.53) Prec@5 96.88 (96.88) TRAIN-SEARCH [2018-10-02-20:56:19] Epoch: [241][100/196] Time 0.60 (0.55) Data 0.00 (0.01) Loss 0.697 (0.779) Prec@1 77.34 (72.87) Prec@5 97.66 (98.04) TRAIN-SEARCH [2018-10-02-20:57:11] Epoch: [241][195/196] Time 0.44 (0.55) Data 0.00 (0.01) Loss 0.800 (0.765) Prec@1 67.50 (73.22) Prec@5 100.00 (98.20) VALID-SEARCH [2018-10-02-20:57:12] Epoch: [241][000/196] Loss 1.650 (1.650) Prec@1 54.69 (54.69) Prec@5 88.28 (88.28) VALID-SEARCH [2018-10-02-20:57:18] Epoch: [241][100/196] Loss 2.000 (1.223) Prec@1 30.47 (60.60) Prec@5 80.47 (94.27) VALID-SEARCH [2018-10-02-20:57:23] Epoch: [241][195/196] Loss 1.006 (1.221) Prec@1 67.50 (60.78) Prec@5 97.50 (94.41) 241/250, Train-Accuracy = 73.22, Test-Accuracy = 60.78 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-20:57:24] [Epoch=242/250] [Need: 00:16:05] [LR=0.0011 ~ 0.0011] [Batch=128], tau=4.192 genotype = Genotype(normal=[('sep_conv_5x5', 0, 0.13060326874256134), ('dil_conv_3x3', 1, 0.12876631319522858), ('sep_conv_3x3', 2, 0.13038048148155212), ('skip_connect', 0, 0.12892556190490723), ('sep_conv_3x3', 3, 0.12900598347187042), ('skip_connect', 0, 0.12887002527713776), ('skip_connect', 1, 0.12846502661705017), ('sep_conv_3x3', 4, 0.12780453264713287)], normal_concat=range(2, 6), reduce=[('dil_conv_5x5', 1, 0.12669609487056732), ('sep_conv_5x5', 0, 0.12643226981163025), ('sep_conv_5x5', 2, 0.12705911695957184), ('sep_conv_3x3', 0, 0.12664496898651123), ('sep_conv_5x5', 0, 0.12738454341888428), ('dil_conv_5x5', 2, 0.12722723186016083), ('sep_conv_5x5', 1, 0.1272675096988678), ('dil_conv_3x3', 4, 0.12645211815834045)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-20:57:26] Epoch: [242][000/196] Time 1.39 (1.39) Data 0.81 (0.81) Loss 0.780 (0.780) Prec@1 77.34 (77.34) Prec@5 97.66 (97.66) TRAIN-SEARCH [2018-10-02-20:58:22] Epoch: [242][100/196] Time 0.51 (0.56) Data 0.00 (0.01) Loss 0.758 (0.768) Prec@1 74.22 (73.13) Prec@5 97.66 (98.10) TRAIN-SEARCH [2018-10-02-20:59:13] Epoch: [242][195/196] Time 0.46 (0.55) Data 0.00 (0.01) Loss 0.996 (0.762) Prec@1 67.50 (73.28) Prec@5 95.00 (98.22) VALID-SEARCH [2018-10-02-20:59:14] Epoch: [242][000/196] Loss 1.116 (1.116) Prec@1 56.25 (56.25) Prec@5 96.09 (96.09) VALID-SEARCH [2018-10-02-20:59:20] Epoch: [242][100/196] Loss 1.540 (1.119) Prec@1 58.59 (62.88) Prec@5 89.06 (95.85) VALID-SEARCH [2018-10-02-20:59:25] Epoch: [242][195/196] Loss 0.603 (1.143) Prec@1 80.00 (62.57) Prec@5 97.50 (95.68) 242/250, Train-Accuracy = 73.28, Test-Accuracy = 62.57 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-20:59:26] [Epoch=243/250] [Need: 00:14:11] [LR=0.0011 ~ 0.0011] [Batch=128], tau=4.168 genotype = Genotype(normal=[('skip_connect', 0, 0.1298067420721054), ('sep_conv_3x3', 1, 0.12837564945220947), ('sep_conv_3x3', 2, 0.13117532432079315), ('skip_connect', 0, 0.12840497493743896), ('sep_conv_3x3', 3, 0.12943574786186218), ('skip_connect', 0, 0.12900272011756897), ('sep_conv_3x3', 4, 0.12795577943325043), ('sep_conv_3x3', 0, 0.1276368349790573)], normal_concat=range(2, 6), reduce=[('dil_conv_5x5', 1, 0.1274881660938263), ('sep_conv_5x5', 0, 0.12695738673210144), ('sep_conv_3x3', 0, 0.1266985386610031), ('dil_conv_3x3', 2, 0.1263657808303833), ('sep_conv_5x5', 0, 0.12764450907707214), ('dil_conv_5x5', 2, 0.1265280693769455), ('sep_conv_5x5', 1, 0.12678159773349762), ('dil_conv_5x5', 3, 0.12660124897956848)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-20:59:28] Epoch: [243][000/196] Time 1.46 (1.46) Data 0.87 (0.87) Loss 0.695 (0.695) Prec@1 71.09 (71.09) Prec@5 99.22 (99.22) TRAIN-SEARCH [2018-10-02-21:00:24] Epoch: [243][100/196] Time 0.58 (0.57) Data 0.01 (0.01) Loss 0.781 (0.769) Prec@1 67.97 (73.07) Prec@5 100.00 (98.40) TRAIN-SEARCH [2018-10-02-21:01:15] Epoch: [243][195/196] Time 0.48 (0.55) Data 0.00 (0.01) Loss 0.729 (0.769) Prec@1 80.00 (73.19) Prec@5 97.50 (98.25) VALID-SEARCH [2018-10-02-21:01:16] Epoch: [243][000/196] Loss 3.050 (3.050) Prec@1 31.25 (31.25) Prec@5 71.09 (71.09) VALID-SEARCH [2018-10-02-21:01:21] Epoch: [243][100/196] Loss 1.289 (1.186) Prec@1 50.00 (60.98) Prec@5 93.75 (94.68) VALID-SEARCH [2018-10-02-21:01:26] Epoch: [243][195/196] Loss 1.120 (1.206) Prec@1 57.50 (60.58) Prec@5 97.50 (94.49) 243/250, Train-Accuracy = 73.19, Test-Accuracy = 60.58 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-21:01:27] [Epoch=244/250] [Need: 00:12:07] [LR=0.0011 ~ 0.0011] [Batch=128], tau=4.144 genotype = Genotype(normal=[('skip_connect', 0, 0.1307835429906845), ('sep_conv_3x3', 1, 0.12860755622386932), ('sep_conv_3x3', 2, 0.1310858279466629), ('skip_connect', 0, 0.12915803492069244), ('sep_conv_3x3', 3, 0.12921595573425293), ('skip_connect', 0, 0.12899453938007355), ('sep_conv_3x3', 4, 0.12834931910037994), ('sep_conv_3x3', 3, 0.12781356275081635)], normal_concat=range(2, 6), reduce=[('dil_conv_5x5', 1, 0.12716150283813477), ('sep_conv_5x5', 0, 0.12676748633384705), ('sep_conv_5x5', 2, 0.12703506648540497), ('sep_conv_3x3', 0, 0.12689973413944244), ('sep_conv_5x5', 0, 0.128023162484169), ('dil_conv_3x3', 1, 0.12618687748908997), ('sep_conv_5x5', 0, 0.1268797516822815), ('dil_conv_5x5', 1, 0.12663312256336212)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-21:01:29] Epoch: [244][000/196] Time 1.26 (1.26) Data 0.67 (0.67) Loss 0.727 (0.727) Prec@1 74.22 (74.22) Prec@5 96.09 (96.09) TRAIN-SEARCH [2018-10-02-21:02:24] Epoch: [244][100/196] Time 0.57 (0.56) Data 0.00 (0.01) Loss 0.742 (0.772) Prec@1 78.91 (72.59) Prec@5 100.00 (98.09) TRAIN-SEARCH [2018-10-02-21:03:16] Epoch: [244][195/196] Time 0.45 (0.55) Data 0.00 (0.01) Loss 0.725 (0.771) Prec@1 70.00 (72.85) Prec@5 97.50 (98.15) VALID-SEARCH [2018-10-02-21:03:17] Epoch: [244][000/196] Loss 1.246 (1.246) Prec@1 63.28 (63.28) Prec@5 92.19 (92.19) VALID-SEARCH [2018-10-02-21:03:21] Epoch: [244][100/196] Loss 1.176 (1.125) Prec@1 60.94 (62.46) Prec@5 96.09 (95.28) VALID-SEARCH [2018-10-02-21:03:26] Epoch: [244][195/196] Loss 0.985 (1.239) Prec@1 62.50 (59.60) Prec@5 97.50 (93.92) 244/250, Train-Accuracy = 72.85, Test-Accuracy = 59.60 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-21:03:26] [Epoch=245/250] [Need: 00:09:57] [LR=0.0010 ~ 0.0010] [Batch=128], tau=4.12 genotype = Genotype(normal=[('skip_connect', 0, 0.13099978864192963), ('sep_conv_5x5', 1, 0.12819749116897583), ('sep_conv_3x3', 2, 0.13017961382865906), ('skip_connect', 0, 0.12889260053634644), ('sep_conv_3x3', 3, 0.13028278946876526), ('skip_connect', 0, 0.1288621574640274), ('sep_conv_3x3', 3, 0.12824712693691254), ('sep_conv_3x3', 4, 0.12815429270267487)], normal_concat=range(2, 6), reduce=[('dil_conv_5x5', 1, 0.12667562067508698), ('skip_connect', 0, 0.12601475417613983), ('sep_conv_3x3', 1, 0.126300647854805), ('sep_conv_3x3', 0, 0.12627173960208893), ('sep_conv_5x5', 0, 0.127572700381279), ('sep_conv_5x5', 1, 0.1264319270849228), ('sep_conv_5x5', 0, 0.12660422921180725), ('dil_conv_5x5', 3, 0.1264280378818512)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-21:03:29] Epoch: [245][000/196] Time 1.43 (1.43) Data 0.88 (0.88) Loss 0.689 (0.689) Prec@1 78.91 (78.91) Prec@5 97.66 (97.66) TRAIN-SEARCH [2018-10-02-21:04:28] Epoch: [245][100/196] Time 0.51 (0.60) Data 0.00 (0.01) Loss 0.776 (0.759) Prec@1 74.22 (73.48) Prec@5 99.22 (98.35) TRAIN-SEARCH [2018-10-02-21:05:15] Epoch: [245][195/196] Time 0.48 (0.55) Data 0.00 (0.01) Loss 0.746 (0.764) Prec@1 72.50 (73.39) Prec@5 100.00 (98.26) VALID-SEARCH [2018-10-02-21:05:16] Epoch: [245][000/196] Loss 0.983 (0.983) Prec@1 67.97 (67.97) Prec@5 96.88 (96.88) VALID-SEARCH [2018-10-02-21:05:22] Epoch: [245][100/196] Loss 0.915 (1.208) Prec@1 64.06 (59.92) Prec@5 99.22 (94.48) VALID-SEARCH [2018-10-02-21:05:27] Epoch: [245][195/196] Loss 0.943 (1.196) Prec@1 60.00 (60.26) Prec@5 100.00 (94.62) 245/250, Train-Accuracy = 73.39, Test-Accuracy = 60.26 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-21:05:28] [Epoch=246/250] [Need: 00:08:05] [LR=0.0010 ~ 0.0010] [Batch=128], tau=4.096 genotype = Genotype(normal=[('skip_connect', 0, 0.13058611750602722), ('sep_conv_5x5', 1, 0.12840701639652252), ('sep_conv_3x3', 2, 0.1297324001789093), ('skip_connect', 0, 0.12854331731796265), ('sep_conv_3x3', 3, 0.12978047132492065), ('skip_connect', 0, 0.1291378289461136), ('sep_conv_3x3', 4, 0.128929004073143), ('sep_conv_3x3', 3, 0.12857502698898315)], normal_concat=range(2, 6), reduce=[('sep_conv_5x5', 1, 0.12624961137771606), ('avg_pool_3x3', 0, 0.12608671188354492), ('sep_conv_5x5', 2, 0.12630096077919006), ('sep_conv_5x5', 1, 0.1262357383966446), ('sep_conv_5x5', 0, 0.12769798934459686), ('dil_conv_5x5', 2, 0.12617738544940948), ('dil_conv_5x5', 1, 0.1267356127500534), ('sep_conv_5x5', 0, 0.12670890986919403)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-21:05:30] Epoch: [246][000/196] Time 1.43 (1.43) Data 0.79 (0.79) Loss 0.665 (0.665) Prec@1 74.22 (74.22) Prec@5 99.22 (99.22) TRAIN-SEARCH [2018-10-02-21:06:27] Epoch: [246][100/196] Time 0.41 (0.58) Data 0.00 (0.01) Loss 0.727 (0.759) Prec@1 71.88 (73.66) Prec@5 97.66 (98.28) TRAIN-SEARCH [2018-10-02-21:07:16] Epoch: [246][195/196] Time 0.51 (0.55) Data 0.00 (0.01) Loss 0.868 (0.766) Prec@1 72.50 (73.29) Prec@5 95.00 (98.22) VALID-SEARCH [2018-10-02-21:07:18] Epoch: [246][000/196] Loss 0.858 (0.858) Prec@1 69.53 (69.53) Prec@5 98.44 (98.44) VALID-SEARCH [2018-10-02-21:07:23] Epoch: [246][100/196] Loss 1.402 (1.208) Prec@1 54.69 (60.88) Prec@5 87.50 (94.72) VALID-SEARCH [2018-10-02-21:07:28] Epoch: [246][195/196] Loss 0.871 (1.187) Prec@1 72.50 (61.14) Prec@5 97.50 (95.27) 246/250, Train-Accuracy = 73.29, Test-Accuracy = 61.14 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-21:07:29] [Epoch=247/250] [Need: 00:06:02] [LR=0.0010 ~ 0.0010] [Batch=128], tau=4.072 genotype = Genotype(normal=[('skip_connect', 0, 0.13081970810890198), ('sep_conv_5x5', 1, 0.1284486949443817), ('skip_connect', 0, 0.13029499351978302), ('sep_conv_3x3', 2, 0.1301770657300949), ('skip_connect', 0, 0.12919360399246216), ('sep_conv_3x3', 2, 0.12899497151374817), ('sep_conv_3x3', 4, 0.12843047082424164), ('max_pool_3x3', 0, 0.12789985537528992)], normal_concat=range(2, 6), reduce=[('sep_conv_5x5', 1, 0.12671759724617004), ('dil_conv_3x3', 0, 0.12596534192562103), ('sep_conv_5x5', 1, 0.1268557757139206), ('sep_conv_5x5', 2, 0.12669555842876434), ('sep_conv_5x5', 0, 0.1273602396249771), ('sep_conv_3x3', 2, 0.12621232867240906), ('sep_conv_5x5', 1, 0.12679386138916016), ('dil_conv_5x5', 3, 0.1265793740749359)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-21:07:31] Epoch: [247][000/196] Time 1.51 (1.51) Data 0.97 (0.97) Loss 0.927 (0.927) Prec@1 65.62 (65.62) Prec@5 97.66 (97.66) TRAIN-SEARCH [2018-10-02-21:08:26] Epoch: [247][100/196] Time 0.42 (0.56) Data 0.00 (0.01) Loss 0.914 (0.764) Prec@1 70.31 (73.41) Prec@5 97.66 (98.26) TRAIN-SEARCH [2018-10-02-21:09:17] Epoch: [247][195/196] Time 0.66 (0.55) Data 0.00 (0.01) Loss 0.693 (0.771) Prec@1 72.50 (73.10) Prec@5 100.00 (98.24) VALID-SEARCH [2018-10-02-21:09:18] Epoch: [247][000/196] Loss 2.936 (2.936) Prec@1 17.97 (17.97) Prec@5 70.31 (70.31) VALID-SEARCH [2018-10-02-21:09:23] Epoch: [247][100/196] Loss 1.221 (1.157) Prec@1 56.25 (61.42) Prec@5 96.88 (95.67) VALID-SEARCH [2018-10-02-21:09:28] Epoch: [247][195/196] Loss 1.135 (1.172) Prec@1 62.50 (61.04) Prec@5 97.50 (95.12) 247/250, Train-Accuracy = 73.10, Test-Accuracy = 61.04 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-21:09:29] [Epoch=248/250] [Need: 00:04:01] [LR=0.0010 ~ 0.0010] [Batch=128], tau=4.048 genotype = Genotype(normal=[('skip_connect', 0, 0.13030949234962463), ('dil_conv_3x3', 1, 0.12753826379776), ('sep_conv_3x3', 2, 0.13114969432353973), ('skip_connect', 0, 0.129338800907135), ('skip_connect', 0, 0.12932857871055603), ('sep_conv_3x3', 3, 0.12883427739143372), ('sep_conv_3x3', 4, 0.12870828807353973), ('max_pool_3x3', 0, 0.12795422971248627)], normal_concat=range(2, 6), reduce=[('sep_conv_5x5', 1, 0.1271253526210785), ('dil_conv_3x3', 0, 0.12664899230003357), ('sep_conv_5x5', 1, 0.1263100951910019), ('sep_conv_5x5', 2, 0.12629608809947968), ('sep_conv_5x5', 0, 0.12764056026935577), ('sep_conv_3x3', 2, 0.12627483904361725), ('sep_conv_5x5', 0, 0.12736846506595612), ('sep_conv_5x5', 1, 0.12729769945144653)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-21:09:32] Epoch: [248][000/196] Time 1.43 (1.43) Data 0.81 (0.81) Loss 0.795 (0.795) Prec@1 67.19 (67.19) Prec@5 99.22 (99.22) TRAIN-SEARCH [2018-10-02-21:10:25] Epoch: [248][100/196] Time 0.41 (0.55) Data 0.00 (0.01) Loss 0.765 (0.752) Prec@1 72.66 (73.62) Prec@5 96.09 (98.23) TRAIN-SEARCH [2018-10-02-21:11:16] Epoch: [248][195/196] Time 0.40 (0.54) Data 0.00 (0.01) Loss 0.957 (0.757) Prec@1 60.00 (73.62) Prec@5 100.00 (98.22) VALID-SEARCH [2018-10-02-21:11:18] Epoch: [248][000/196] Loss 1.255 (1.255) Prec@1 53.91 (53.91) Prec@5 92.97 (92.97) VALID-SEARCH [2018-10-02-21:11:23] Epoch: [248][100/196] Loss 0.769 (1.278) Prec@1 71.88 (59.58) Prec@5 98.44 (94.54) VALID-SEARCH [2018-10-02-21:11:28] Epoch: [248][195/196] Loss 1.532 (1.197) Prec@1 52.50 (61.08) Prec@5 92.50 (95.20) 248/250, Train-Accuracy = 73.62, Test-Accuracy = 61.08 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth ==>>[2018-10-02-21:11:29] [Epoch=249/250] [Need: 00:02:00] [LR=0.0010 ~ 0.0010] [Batch=128], tau=4.024 genotype = Genotype(normal=[('skip_connect', 0, 0.13080622255802155), ('dil_conv_3x3', 1, 0.12786200642585754), ('sep_conv_3x3', 2, 0.13117146492004395), ('skip_connect', 0, 0.12972070276737213), ('skip_connect', 0, 0.12949064373970032), ('sep_conv_3x3', 3, 0.129020094871521), ('sep_conv_3x3', 4, 0.1285228729248047), ('max_pool_3x3', 0, 0.12833330035209656)], normal_concat=range(2, 6), reduce=[('sep_conv_5x5', 1, 0.1268126666545868), ('dil_conv_3x3', 0, 0.12622934579849243), ('sep_conv_3x3', 2, 0.12680386006832123), ('sep_conv_5x5', 1, 0.126529723405838), ('sep_conv_3x3', 2, 0.12661665678024292), ('dil_conv_5x5', 3, 0.1266031563282013), ('sep_conv_5x5', 0, 0.12760508060455322), ('sep_conv_5x5', 1, 0.12698079645633698)], reduce_concat=range(2, 6)) TRAIN-SEARCH [2018-10-02-21:11:32] Epoch: [249][000/196] Time 1.63 (1.63) Data 0.84 (0.84) Loss 0.746 (0.746) Prec@1 75.78 (75.78) Prec@5 99.22 (99.22) TRAIN-SEARCH [2018-10-02-21:12:23] Epoch: [249][100/196] Time 0.46 (0.52) Data 0.00 (0.01) Loss 0.913 (0.750) Prec@1 69.53 (73.67) Prec@5 99.22 (98.04) TRAIN-SEARCH [2018-10-02-21:13:17] Epoch: [249][195/196] Time 0.53 (0.55) Data 0.00 (0.01) Loss 0.818 (0.757) Prec@1 70.00 (73.44) Prec@5 97.50 (98.18) VALID-SEARCH [2018-10-02-21:13:18] Epoch: [249][000/196] Loss 1.164 (1.164) Prec@1 60.94 (60.94) Prec@5 93.75 (93.75) VALID-SEARCH [2018-10-02-21:13:23] Epoch: [249][100/196] Loss 2.468 (1.248) Prec@1 30.47 (59.15) Prec@5 63.28 (93.95) VALID-SEARCH [2018-10-02-21:13:29] Epoch: [249][195/196] Loss 1.122 (1.228) Prec@1 65.00 (59.51) Prec@5 97.50 (94.00) 249/250, Train-Accuracy = 73.44, Test-Accuracy = 59.51 ----> Save into ./snapshots/NAS/ACC-V2-Search-acc2-cifar10-cut16-250-E600/seed-604/checkpoint-search.pth