[09/26 09:01:05] libai INFO: Rank of current process: 0. World size: 8 [09/26 09:01:05] libai INFO: Command line arguments: Namespace(config_file='configs/swin_imagenet.py', eval_only=False, fast_dev_run=False, opts=[], resume=False) [09/26 09:01:05] libai INFO: Contents of args.config_file=configs/swin_imagenet.py: from libai.config import LazyCall from .common.models.swin.swin_tiny_patch4_window7_224 import model from .common.models.graph import graph from .common.train import train from .common.optim import optim from .common.data.imagenet import dataloader from flowvision.data import Mixup from flowvision.loss.cross_entropy import SoftTargetCrossEntropy # Refine data path to imagenet dataloader.train.dataset[0].root = "/data/ImageNet/extract/" dataloader.test[0].dataset.root = "/data/ImageNet/extract/" # Add Mixup Func dataloader.train.mixup_func = LazyCall(Mixup)(  mixup_alpha=0.8,  cutmix_alpha=1.0,  prob=1.0,  switch_prob=0.5,  mode="batch",  num_classes=1000, ) # Refine model cfg for vit training on imagenet model.cfg.num_classes = 1000 model.cfg.loss_func = SoftTargetCrossEntropy() # Refine optimizer cfg for vit model optim.lr = 1e-3 optim.eps = 1e-8 optim.weight_decay = 0.05 optim.params.clip_grad_max_norm = 5.0 optim.params.clip_grad_norm_type = 2.0 # Refine train cfg for vit model train.train_micro_batch_size = 128 train.test_micro_batch_size = 128 train.train_epoch = 300 train.warmup_ratio = 20 / 300 train.eval_period = 1000 train.log_period = 100 train.output_dir = "./commit_swin" # Scheduler train.scheduler.warmup_factor = 0.001 train.scheduler.alpha = 0.01 train.scheduler.warmup_method = "linear" # Set fp16 ON train.amp.enabled = True [09/26 09:01:05] libai INFO: Full config saved to ./commit_swin/config.yaml [09/26 09:01:05] lb.engine.default INFO: > compiling dataset index builder ... [09/26 09:01:05] lb.engine.default INFO: >>> done with dataset index builder. Compilation time: 0.078 seconds [09/26 09:01:05] lb.engine.default INFO: >>> done with compiling. Compilation time: 0.079 seconds [09/26 09:01:09] lb.engine.default INFO: Prepare training, validating, testing set [09/26 09:01:12] lb.engine.default INFO: Prepare testing set [09/26 09:01:12] lb.engine.default INFO: Auto-scaling the config to train.train_iter=375342, train.warmup_iter=25023 [09/26 09:01:19] lb.engine.default INFO: Model: SwinTransformer( (patch_embed): PatchEmbed( (proj): Conv2d(3, 96, kernel_size=(4, 4), stride=(4, 4)) (norm): LayerNorm((96,), eps=1e-05, elementwise_affine=True) ) (pos_drop): Dropout(p=0.0, inplace=False) (layers): ModuleList( (0): BasicLayer( (blocks): ModuleList( (0): SwinTransformerBlock( (norm1): LayerNorm((96,), eps=1e-05, elementwise_affine=True) (attn): WindowAttention( (qkv): Linear1D(in_features=96, out_features=288, bias=True, parallel=data) (attn_drop): Dropout(p=0.0, inplace=False) (proj): Linear1D(in_features=96, out_features=96, bias=True, parallel=data) (proj_drop): Dropout(p=0.0, inplace=False) (softmax): Softmax(dim=-1) ) (drop_path): Identity() (norm2): LayerNorm((96,), eps=1e-05, elementwise_affine=True) (mlp): MLP( bias_gelu_fusion=True, bias_dropout_fusion=True, dropout=0.0 (dense_h_to_4h): Linear1D(in_features=96, out_features=384, bias=True, parallel=col) (dense_4h_to_h): Linear1D(in_features=384, out_features=96, bias=True, parallel=row) ) ) (1): SwinTransformerBlock( (norm1): LayerNorm((96,), eps=1e-05, elementwise_affine=True) (attn): WindowAttention( (qkv): Linear1D(in_features=96, out_features=288, bias=True, parallel=data) (attn_drop): Dropout(p=0.0, inplace=False) (proj): Linear1D(in_features=96, out_features=96, bias=True, parallel=data) (proj_drop): Dropout(p=0.0, inplace=False) (softmax): Softmax(dim=-1) ) (drop_path): DropPath() (norm2): LayerNorm((96,), eps=1e-05, elementwise_affine=True) (mlp): MLP( bias_gelu_fusion=True, bias_dropout_fusion=True, dropout=0.0 (dense_h_to_4h): Linear1D(in_features=96, out_features=384, bias=True, parallel=col) (dense_4h_to_h): Linear1D(in_features=384, out_features=96, bias=True, parallel=row) ) ) ) (downsample): PatchMerging( (reduction): Linear1D(in_features=384, out_features=192, bias=False, parallel=data) (norm): LayerNorm((384,), eps=1e-05, elementwise_affine=True) ) ) (1): BasicLayer( (blocks): ModuleList( (0): SwinTransformerBlock( (norm1): LayerNorm((192,), eps=1e-05, elementwise_affine=True) (attn): WindowAttention( (qkv): Linear1D(in_features=192, out_features=576, bias=True, parallel=data) (attn_drop): Dropout(p=0.0, inplace=False) (proj): Linear1D(in_features=192, out_features=192, bias=True, parallel=data) (proj_drop): Dropout(p=0.0, inplace=False) (softmax): Softmax(dim=-1) ) (drop_path): DropPath() (norm2): LayerNorm((192,), eps=1e-05, elementwise_affine=True) (mlp): MLP( bias_gelu_fusion=True, bias_dropout_fusion=True, dropout=0.0 (dense_h_to_4h): Linear1D(in_features=192, out_features=768, bias=True, parallel=col) (dense_4h_to_h): Linear1D(in_features=768, out_features=192, bias=True, parallel=row) ) ) (1): SwinTransformerBlock( (norm1): LayerNorm((192,), eps=1e-05, elementwise_affine=True) (attn): WindowAttention( (qkv): Linear1D(in_features=192, out_features=576, bias=True, parallel=data) (attn_drop): Dropout(p=0.0, inplace=False) (proj): Linear1D(in_features=192, out_features=192, bias=True, parallel=data) (proj_drop): Dropout(p=0.0, inplace=False) (softmax): Softmax(dim=-1) ) (drop_path): DropPath() (norm2): LayerNorm((192,), eps=1e-05, elementwise_affine=True) (mlp): MLP( bias_gelu_fusion=True, bias_dropout_fusion=True, dropout=0.0 (dense_h_to_4h): Linear1D(in_features=192, out_features=768, bias=True, parallel=col) (dense_4h_to_h): Linear1D(in_features=768, out_features=192, bias=True, parallel=row) ) ) ) (downsample): PatchMerging( (reduction): Linear1D(in_features=768, out_features=384, bias=False, parallel=data) (norm): LayerNorm((768,), eps=1e-05, elementwise_affine=True) ) ) (2): BasicLayer( (blocks): ModuleList( (0): SwinTransformerBlock( (norm1): LayerNorm((384,), eps=1e-05, elementwise_affine=True) (attn): WindowAttention( (qkv): Linear1D(in_features=384, out_features=1152, bias=True, parallel=data) (attn_drop): Dropout(p=0.0, inplace=False) (proj): Linear1D(in_features=384, out_features=384, bias=True, parallel=data) (proj_drop): Dropout(p=0.0, inplace=False) (softmax): Softmax(dim=-1) ) (drop_path): DropPath() (norm2): LayerNorm((384,), eps=1e-05, elementwise_affine=True) (mlp): MLP( bias_gelu_fusion=True, bias_dropout_fusion=True, dropout=0.0 (dense_h_to_4h): Linear1D(in_features=384, out_features=1536, bias=True, parallel=col) (dense_4h_to_h): Linear1D(in_features=1536, out_features=384, bias=True, parallel=row) ) ) (1): SwinTransformerBlock( (norm1): LayerNorm((384,), eps=1e-05, elementwise_affine=True) (attn): WindowAttention( (qkv): Linear1D(in_features=384, out_features=1152, bias=True, parallel=data) (attn_drop): Dropout(p=0.0, inplace=False) (proj): Linear1D(in_features=384, out_features=384, bias=True, parallel=data) (proj_drop): Dropout(p=0.0, inplace=False) (softmax): Softmax(dim=-1) ) (drop_path): DropPath() (norm2): LayerNorm((384,), eps=1e-05, elementwise_affine=True) (mlp): MLP( bias_gelu_fusion=True, bias_dropout_fusion=True, dropout=0.0 (dense_h_to_4h): Linear1D(in_features=384, out_features=1536, bias=True, parallel=col) (dense_4h_to_h): Linear1D(in_features=1536, out_features=384, bias=True, parallel=row) ) ) (2): SwinTransformerBlock( (norm1): LayerNorm((384,), eps=1e-05, elementwise_affine=True) (attn): WindowAttention( (qkv): Linear1D(in_features=384, out_features=1152, bias=True, parallel=data) (attn_drop): Dropout(p=0.0, inplace=False) (proj): Linear1D(in_features=384, out_features=384, bias=True, parallel=data) (proj_drop): Dropout(p=0.0, inplace=False) (softmax): Softmax(dim=-1) ) (drop_path): DropPath() (norm2): LayerNorm((384,), eps=1e-05, elementwise_affine=True) (mlp): MLP( bias_gelu_fusion=True, bias_dropout_fusion=True, dropout=0.0 (dense_h_to_4h): Linear1D(in_features=384, out_features=1536, bias=True, parallel=col) (dense_4h_to_h): Linear1D(in_features=1536, out_features=384, bias=True, parallel=row) ) ) (3): SwinTransformerBlock( (norm1): LayerNorm((384,), eps=1e-05, elementwise_affine=True) (attn): WindowAttention( (qkv): Linear1D(in_features=384, out_features=1152, bias=True, parallel=data) (attn_drop): Dropout(p=0.0, inplace=False) (proj): Linear1D(in_features=384, out_features=384, bias=True, parallel=data) (proj_drop): Dropout(p=0.0, inplace=False) (softmax): Softmax(dim=-1) ) (drop_path): DropPath() (norm2): LayerNorm((384,), eps=1e-05, elementwise_affine=True) (mlp): MLP( bias_gelu_fusion=True, bias_dropout_fusion=True, dropout=0.0 (dense_h_to_4h): Linear1D(in_features=384, out_features=1536, bias=True, parallel=col) (dense_4h_to_h): Linear1D(in_features=1536, out_features=384, bias=True, parallel=row) ) ) (4): SwinTransformerBlock( (norm1): LayerNorm((384,), eps=1e-05, elementwise_affine=True) (attn): WindowAttention( (qkv): Linear1D(in_features=384, out_features=1152, bias=True, parallel=data) (attn_drop): Dropout(p=0.0, inplace=False) (proj): Linear1D(in_features=384, out_features=384, bias=True, parallel=data) (proj_drop): Dropout(p=0.0, inplace=False) (softmax): Softmax(dim=-1) ) (drop_path): DropPath() (norm2): LayerNorm((384,), eps=1e-05, elementwise_affine=True) (mlp): MLP( bias_gelu_fusion=True, bias_dropout_fusion=True, dropout=0.0 (dense_h_to_4h): Linear1D(in_features=384, out_features=1536, bias=True, parallel=col) (dense_4h_to_h): Linear1D(in_features=1536, out_features=384, bias=True, parallel=row) ) ) (5): SwinTransformerBlock( (norm1): LayerNorm((384,), eps=1e-05, elementwise_affine=True) (attn): WindowAttention( (qkv): Linear1D(in_features=384, out_features=1152, bias=True, parallel=data) (attn_drop): Dropout(p=0.0, inplace=False) (proj): Linear1D(in_features=384, out_features=384, bias=True, parallel=data) (proj_drop): Dropout(p=0.0, inplace=False) (softmax): Softmax(dim=-1) ) (drop_path): DropPath() (norm2): LayerNorm((384,), eps=1e-05, elementwise_affine=True) (mlp): MLP( bias_gelu_fusion=True, bias_dropout_fusion=True, dropout=0.0 (dense_h_to_4h): Linear1D(in_features=384, out_features=1536, bias=True, parallel=col) (dense_4h_to_h): Linear1D(in_features=1536, out_features=384, bias=True, parallel=row) ) ) ) (downsample): PatchMerging( (reduction): Linear1D(in_features=1536, out_features=768, bias=False, parallel=data) (norm): LayerNorm((1536,), eps=1e-05, elementwise_affine=True) ) ) (3): BasicLayer( (blocks): ModuleList( (0): SwinTransformerBlock( (norm1): LayerNorm((768,), eps=1e-05, elementwise_affine=True) (attn): WindowAttention( (qkv): Linear1D(in_features=768, out_features=2304, bias=True, parallel=data) (attn_drop): Dropout(p=0.0, inplace=False) (proj): Linear1D(in_features=768, out_features=768, bias=True, parallel=data) (proj_drop): Dropout(p=0.0, inplace=False) (softmax): Softmax(dim=-1) ) (drop_path): DropPath() (norm2): LayerNorm((768,), eps=1e-05, elementwise_affine=True) (mlp): MLP( bias_gelu_fusion=True, bias_dropout_fusion=True, dropout=0.0 (dense_h_to_4h): Linear1D(in_features=768, out_features=3072, bias=True, parallel=col) (dense_4h_to_h): Linear1D(in_features=3072, out_features=768, bias=True, parallel=row) ) ) (1): SwinTransformerBlock( (norm1): LayerNorm((768,), eps=1e-05, elementwise_affine=True) (attn): WindowAttention( (qkv): Linear1D(in_features=768, out_features=2304, bias=True, parallel=data) (attn_drop): Dropout(p=0.0, inplace=False) (proj): Linear1D(in_features=768, out_features=768, bias=True, parallel=data) (proj_drop): Dropout(p=0.0, inplace=False) (softmax): Softmax(dim=-1) ) (drop_path): DropPath() (norm2): LayerNorm((768,), eps=1e-05, elementwise_affine=True) (mlp): MLP( bias_gelu_fusion=True, bias_dropout_fusion=True, dropout=0.0 (dense_h_to_4h): Linear1D(in_features=768, out_features=3072, bias=True, parallel=col) (dense_4h_to_h): Linear1D(in_features=3072, out_features=768, bias=True, parallel=row) ) ) ) ) ) (norm): LayerNorm((768,), eps=1e-05, elementwise_affine=True) (avgpool): AdaptiveAvgPool1d() (head): Linear1D(in_features=768, out_features=1000, bias=True, parallel=data) (loss_func): SoftTargetCrossEntropy() ) [09/26 09:01:20] lb.engine.trainer INFO: Starting training from iteration 0 [09/26 09:01:21] lb.models.utils.graph_base INFO: Start compling the train graph which may take some time. Please wait for a moment ... [09/26 09:02:18] lb.utils.events INFO: eta: 1 day, 2:18:37 iteration: 99/375342 consumed_samples: 102400 total_loss: 6.945 time: 0.3066 s/iter data_time: 0.2346 s/iter total_throughput: 3340.35 samples/s lr: 4.91e-06 [09/26 09:02:50] lb.utils.events INFO: eta: 23:58:49 iteration: 199/375342 consumed_samples: 204800 total_loss: 6.926 time: 0.3130 s/iter data_time: 0.2104 s/iter total_throughput: 3271.60 samples/s lr: 8.86e-06 [09/26 09:03:22] lb.utils.events INFO: eta: 22:31:51 iteration: 299/375342 consumed_samples: 307200 total_loss: 6.902 time: 0.3148 s/iter data_time: 0.2432 s/iter total_throughput: 3252.76 samples/s lr: 1.28e-05 [09/26 09:03:54] lb.utils.events INFO: eta: 23:38:51 iteration: 399/375342 consumed_samples: 409600 total_loss: 6.883 time: 0.3162 s/iter data_time: 0.2334 s/iter total_throughput: 3238.47 samples/s lr: 1.68e-05 [09/26 09:04:26] lb.utils.events INFO: eta: 1 day, 1:34:31 iteration: 499/375342 consumed_samples: 512000 total_loss: 6.863 time: 0.3174 s/iter data_time: 0.2159 s/iter total_throughput: 3226.51 samples/s lr: 2.07e-05 [09/26 09:04:58] lb.utils.events INFO: eta: 1 day, 1:39:35 iteration: 599/375342 consumed_samples: 614400 total_loss: 6.848 time: 0.3177 s/iter data_time: 0.2207 s/iter total_throughput: 3222.97 samples/s lr: 2.47e-05 [09/26 09:05:29] lb.utils.events INFO: eta: 1 day, 2:27:41 iteration: 699/375342 consumed_samples: 716800 total_loss: 6.832 time: 0.3174 s/iter data_time: 0.2342 s/iter total_throughput: 3225.89 samples/s lr: 2.86e-05 [09/26 09:06:01] lb.utils.events INFO: eta: 1 day, 4:01:59 iteration: 799/375342 consumed_samples: 819200 total_loss: 6.812 time: 0.3178 s/iter data_time: 0.2536 s/iter total_throughput: 3222.53 samples/s lr: 3.26e-05 [09/26 09:06:34] lb.utils.events INFO: eta: 1 day, 3:21:52 iteration: 899/375342 consumed_samples: 921600 total_loss: 6.789 time: 0.3181 s/iter data_time: 0.2057 s/iter total_throughput: 3218.61 samples/s lr: 3.65e-05 [09/26 09:07:05] lb.utils.events INFO: eta: 1 day, 1:42:40 iteration: 999/375342 consumed_samples: 1024000 total_loss: 6.762 time: 0.3180 s/iter data_time: 0.2265 s/iter total_throughput: 3219.97 samples/s lr: 4.05e-05 [09/26 09:07:37] lb.utils.events INFO: eta: 1 day, 1:12:59 iteration: 1099/375342 consumed_samples: 1126400 total_loss: 6.734 time: 0.3184 s/iter data_time: 0.2175 s/iter total_throughput: 3216.35 samples/s lr: 4.44e-05 [09/26 09:08:09] lb.utils.events INFO: eta: 1 day, 0:57:39 iteration: 1199/375342 consumed_samples: 1228800 total_loss: 6.691 time: 0.3185 s/iter data_time: 0.2085 s/iter total_throughput: 3214.86 samples/s lr: 4.83e-05 [09/26 09:08:41] lb.utils.events INFO: eta: 1 day, 1:09:48 iteration: 1299/375342 consumed_samples: 1331200 total_loss: 6.648 time: 0.3186 s/iter data_time: 0.2181 s/iter total_throughput: 3213.64 samples/s lr: 5.23e-05 [09/26 09:09:13] lb.utils.events INFO: eta: 1 day, 0:34:17 iteration: 1399/375342 consumed_samples: 1433600 total_loss: 6.613 time: 0.3187 s/iter data_time: 0.1964 s/iter total_throughput: 3213.02 samples/s lr: 5.62e-05 [09/26 09:09:45] lb.utils.events INFO: eta: 21:45:33 iteration: 1499/375342 consumed_samples: 1536000 total_loss: 6.582 time: 0.3188 s/iter data_time: 0.2152 s/iter total_throughput: 3212.13 samples/s lr: 6.02e-05 [09/26 09:10:18] lb.utils.events INFO: eta: 20:28:40 iteration: 1599/375342 consumed_samples: 1638400 total_loss: 6.551 time: 0.3190 s/iter data_time: 0.2230 s/iter total_throughput: 3210.40 samples/s lr: 6.41e-05 [09/26 09:10:50] lb.utils.events INFO: eta: 18:30:05 iteration: 1699/375342 consumed_samples: 1740800 total_loss: 6.523 time: 0.3193 s/iter data_time: 0.2183 s/iter total_throughput: 3206.89 samples/s lr: 6.81e-05 [09/26 09:11:23] lb.utils.events INFO: eta: 17:04:19 iteration: 1799/375342 consumed_samples: 1843200 total_loss: 6.504 time: 0.3196 s/iter data_time: 0.2315 s/iter total_throughput: 3204.39 samples/s lr: 7.20e-05 [09/26 09:11:55] lb.utils.events INFO: eta: 16:36:10 iteration: 1899/375342 consumed_samples: 1945600 total_loss: 6.48 time: 0.3198 s/iter data_time: 0.2057 s/iter total_throughput: 3202.13 samples/s lr: 7.60e-05 [09/26 09:12:26] lb.utils.events INFO: eta: 16:31:39 iteration: 1999/375342 consumed_samples: 2048000 total_loss: 6.445 time: 0.3195 s/iter data_time: 0.1877 s/iter total_throughput: 3204.64 samples/s lr: 7.99e-05 [09/26 09:12:58] lb.utils.events INFO: eta: 16:26:04 iteration: 2099/375342 consumed_samples: 2150400 total_loss: 6.43 time: 0.3195 s/iter data_time: 0.2099 s/iter total_throughput: 3204.95 samples/s lr: 8.39e-05 [09/26 09:13:30] lb.utils.events INFO: eta: 16:33:39 iteration: 2199/375342 consumed_samples: 2252800 total_loss: 6.412 time: 0.3194 s/iter data_time: 0.2342 s/iter total_throughput: 3206.13 samples/s lr: 8.78e-05 [09/26 09:14:02] lb.utils.events INFO: eta: 16:46:17 iteration: 2299/375342 consumed_samples: 2355200 total_loss: 6.383 time: 0.3195 s/iter data_time: 0.2383 s/iter total_throughput: 3205.08 samples/s lr: 9.18e-05 [09/26 09:14:34] lb.utils.events INFO: eta: 16:50:29 iteration: 2399/375342 consumed_samples: 2457600 total_loss: 6.363 time: 0.3196 s/iter data_time: 0.2113 s/iter total_throughput: 3203.62 samples/s lr: 9.57e-05 [09/26 09:15:06] lb.utils.events INFO: eta: 17:03:45 iteration: 2499/375342 consumed_samples: 2560000 total_loss: 6.354 time: 0.3197 s/iter data_time: 0.2188 s/iter total_throughput: 3203.47 samples/s lr: 9.97e-05 [09/26 09:15:39] lb.utils.events INFO: eta: 17:38:51 iteration: 2599/375342 consumed_samples: 2662400 total_loss: 6.33 time: 0.3197 s/iter data_time: 0.2368 s/iter total_throughput: 3203.29 samples/s lr: 1.04e-04 [09/26 09:16:11] lb.utils.events INFO: eta: 18:17:36 iteration: 2699/375342 consumed_samples: 2764800 total_loss: 6.301 time: 0.3199 s/iter data_time: 0.2298 s/iter total_throughput: 3201.28 samples/s lr: 1.08e-04 [09/26 09:16:43] lb.utils.events INFO: eta: 18:46:07 iteration: 2799/375342 consumed_samples: 2867200 total_loss: 6.293 time: 0.3199 s/iter data_time: 0.2315 s/iter total_throughput: 3201.18 samples/s lr: 1.12e-04 [09/26 09:17:15] lb.utils.events INFO: eta: 19:38:30 iteration: 2899/375342 consumed_samples: 2969600 total_loss: 6.277 time: 0.3199 s/iter data_time: 0.2349 s/iter total_throughput: 3201.19 samples/s lr: 1.15e-04 [09/26 09:17:47] lb.utils.events INFO: eta: 22:22:18 iteration: 2999/375342 consumed_samples: 3072000 total_loss: 6.25 time: 0.3200 s/iter data_time: 0.2170 s/iter total_throughput: 3199.77 samples/s lr: 1.19e-04 [09/26 09:18:20] lb.utils.events INFO: eta: 23:09:59 iteration: 3099/375342 consumed_samples: 3174400 total_loss: 6.238 time: 0.3201 s/iter data_time: 0.2083 s/iter total_throughput: 3199.30 samples/s lr: 1.23e-04 [09/26 09:18:52] lb.utils.events INFO: eta: 22:03:39 iteration: 3199/375342 consumed_samples: 3276800 total_loss: 6.211 time: 0.3202 s/iter data_time: 0.2035 s/iter total_throughput: 3197.87 samples/s lr: 1.27e-04 [09/26 09:19:24] lb.utils.events INFO: eta: 18:29:09 iteration: 3299/375342 consumed_samples: 3379200 total_loss: 6.193 time: 0.3202 s/iter data_time: 0.2042 s/iter total_throughput: 3197.94 samples/s lr: 1.31e-04 [09/26 09:19:56] lb.utils.events INFO: eta: 17:45:21 iteration: 3399/375342 consumed_samples: 3481600 total_loss: 6.188 time: 0.3202 s/iter data_time: 0.1999 s/iter total_throughput: 3197.55 samples/s lr: 1.35e-04 [09/26 09:20:29] lb.utils.events INFO: eta: 17:00:21 iteration: 3499/375342 consumed_samples: 3584000 total_loss: 6.164 time: 0.3204 s/iter data_time: 0.1993 s/iter total_throughput: 3196.45 samples/s lr: 1.39e-04 [09/26 09:21:01] lb.utils.events INFO: eta: 16:37:26 iteration: 3599/375342 consumed_samples: 3686400 total_loss: 6.148 time: 0.3203 s/iter data_time: 0.2106 s/iter total_throughput: 3196.99 samples/s lr: 1.43e-04 [09/26 09:21:33] lb.utils.events INFO: eta: 16:17:24 iteration: 3699/375342 consumed_samples: 3788800 total_loss: 6.141 time: 0.3204 s/iter data_time: 0.2109 s/iter total_throughput: 3196.18 samples/s lr: 1.47e-04 [09/26 09:22:05] lb.utils.events INFO: eta: 16:11:13 iteration: 3799/375342 consumed_samples: 3891200 total_loss: 6.105 time: 0.3203 s/iter data_time: 0.2005 s/iter total_throughput: 3196.73 samples/s lr: 1.51e-04 [09/26 09:22:37] lb.utils.events INFO: eta: 16:13:04 iteration: 3899/375342 consumed_samples: 3993600 total_loss: 6.082 time: 0.3205 s/iter data_time: 0.2083 s/iter total_throughput: 3195.45 samples/s lr: 1.55e-04 [09/26 09:23:10] lb.utils.events INFO: eta: 16:14:53 iteration: 3999/375342 consumed_samples: 4096000 total_loss: 6.074 time: 0.3206 s/iter data_time: 0.2441 s/iter total_throughput: 3194.19 samples/s lr: 1.59e-04 [09/26 09:23:42] lb.utils.events INFO: eta: 16:14:47 iteration: 4099/375342 consumed_samples: 4198400 total_loss: 6.061 time: 0.3207 s/iter data_time: 0.2223 s/iter total_throughput: 3193.08 samples/s lr: 1.63e-04 [09/26 09:24:15] lb.utils.events INFO: eta: 16:08:19 iteration: 4199/375342 consumed_samples: 4300800 total_loss: 6.027 time: 0.3208 s/iter data_time: 0.1965 s/iter total_throughput: 3191.69 samples/s lr: 1.67e-04 [09/26 09:24:47] lb.utils.events INFO: eta: 16:04:09 iteration: 4299/375342 consumed_samples: 4403200 total_loss: 6.023 time: 0.3209 s/iter data_time: 0.1992 s/iter total_throughput: 3190.83 samples/s lr: 1.71e-04 [09/26 09:25:20] lb.utils.events INFO: eta: 16:02:53 iteration: 4399/375342 consumed_samples: 4505600 total_loss: 6.008 time: 0.3210 s/iter data_time: 0.1973 s/iter total_throughput: 3190.40 samples/s lr: 1.75e-04 [09/26 09:25:52] lb.utils.events INFO: eta: 16:03:41 iteration: 4499/375342 consumed_samples: 4608000 total_loss: 5.984 time: 0.3210 s/iter data_time: 0.2102 s/iter total_throughput: 3190.03 samples/s lr: 1.79e-04 [09/26 09:26:24] lb.utils.events INFO: eta: 16:00:33 iteration: 4599/375342 consumed_samples: 4710400 total_loss: 5.973 time: 0.3210 s/iter data_time: 0.2107 s/iter total_throughput: 3189.81 samples/s lr: 1.83e-04 [09/26 09:26:57] lb.utils.events INFO: eta: 16:01:33 iteration: 4699/375342 consumed_samples: 4812800 total_loss: 5.961 time: 0.3211 s/iter data_time: 0.2103 s/iter total_throughput: 3189.26 samples/s lr: 1.87e-04 [09/26 09:27:29] lb.utils.events INFO: eta: 16:01:44 iteration: 4799/375342 consumed_samples: 4915200 total_loss: 5.939 time: 0.3211 s/iter data_time: 0.2227 s/iter total_throughput: 3188.56 samples/s lr: 1.91e-04 [09/26 09:28:02] lb.utils.events INFO: eta: 16:01:28 iteration: 4899/375342 consumed_samples: 5017600 total_loss: 5.93 time: 0.3212 s/iter data_time: 0.2483 s/iter total_throughput: 3187.78 samples/s lr: 1.94e-04 [09/26 09:28:34] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0004999 [09/26 09:28:35] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/26 09:28:35] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/26 09:28:36] lb.models.utils.graph_base INFO: Start compling the eval graph which may take some time. Please wait for a moment ... [09/26 09:28:41] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0003 s/iter. Inference: 0.1579 s/iter. Eval: 0.0020 s/iter. Total: 0.1603 s/iter. ETA=0:00:05 [09/26 09:28:46] lb.evaluation.evaluator INFO: Inference done 28672/50000. Dataloading: 0.0672 s/iter. Inference: 0.1909 s/iter. Eval: 0.0020 s/iter. Total: 0.2602 s/iter. ETA=0:00:05 [09/26 09:28:51] lb.evaluation.evaluator INFO: Inference done 47104/50000. Dataloading: 0.0802 s/iter. Inference: 0.2002 s/iter. Eval: 0.0020 s/iter. Total: 0.2825 s/iter. ETA=0:00:00 [09/26 09:28:52] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/26 09:28:52] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.072605 (0.000241 s / iter per device, on 8 devices) [09/26 09:28:52] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:08 (0.000174 s / iter per device, on 8 devices) [09/26 09:28:52] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/26 09:28:52] lb.evaluation.utils INFO: copypaste: Acc@1=18.093999999999998 [09/26 09:28:52] lb.evaluation.utils INFO: copypaste: Acc@5=38.9 [09/26 09:28:52] lb.engine.hooks INFO: Saved first model at 18.09400 @ 4999 steps [09/26 09:28:52] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/26 09:28:52] lb.utils.events INFO: eta: 15:55:45 iteration: 4999/375342 consumed_samples: 5120000 total_loss: 5.914 time: 0.3213 s/iter data_time: 0.2040 s/iter total_throughput: 3187.32 samples/s lr: 1.98e-04 [09/26 09:29:23] lb.utils.events INFO: eta: 16:00:15 iteration: 5099/375342 consumed_samples: 5222400 total_loss: 5.891 time: 0.3209 s/iter data_time: 0.2379 s/iter total_throughput: 3191.14 samples/s lr: 2.02e-04 [09/26 09:29:55] lb.utils.events INFO: eta: 16:07:23 iteration: 5199/375342 consumed_samples: 5324800 total_loss: 5.867 time: 0.3210 s/iter data_time: 0.2251 s/iter total_throughput: 3190.12 samples/s lr: 2.06e-04 [09/26 09:30:27] lb.utils.events INFO: eta: 16:13:15 iteration: 5299/375342 consumed_samples: 5427200 total_loss: 5.852 time: 0.3210 s/iter data_time: 0.2200 s/iter total_throughput: 3189.83 samples/s lr: 2.10e-04 [09/26 09:30:59] lb.utils.events INFO: eta: 16:24:16 iteration: 5399/375342 consumed_samples: 5529600 total_loss: 5.84 time: 0.3210 s/iter data_time: 0.2147 s/iter total_throughput: 3189.97 samples/s lr: 2.14e-04 [09/26 09:31:32] lb.utils.events INFO: eta: 16:34:19 iteration: 5499/375342 consumed_samples: 5632000 total_loss: 5.83 time: 0.3211 s/iter data_time: 0.1949 s/iter total_throughput: 3189.43 samples/s lr: 2.18e-04 [09/26 09:32:04] lb.utils.events INFO: eta: 16:38:34 iteration: 5599/375342 consumed_samples: 5734400 total_loss: 5.801 time: 0.3211 s/iter data_time: 0.2194 s/iter total_throughput: 3189.17 samples/s lr: 2.22e-04 [09/26 09:32:36] lb.utils.events INFO: eta: 16:56:28 iteration: 5699/375342 consumed_samples: 5836800 total_loss: 5.793 time: 0.3211 s/iter data_time: 0.2061 s/iter total_throughput: 3189.37 samples/s lr: 2.26e-04 [09/26 09:33:08] lb.utils.events INFO: eta: 17:22:07 iteration: 5799/375342 consumed_samples: 5939200 total_loss: 5.781 time: 0.3211 s/iter data_time: 0.2261 s/iter total_throughput: 3189.46 samples/s lr: 2.30e-04 [09/26 09:33:40] lb.utils.events INFO: eta: 17:12:25 iteration: 5899/375342 consumed_samples: 6041600 total_loss: 5.77 time: 0.3210 s/iter data_time: 0.2019 s/iter total_throughput: 3190.23 samples/s lr: 2.34e-04 [09/26 09:34:12] lb.utils.events INFO: eta: 18:38:23 iteration: 5999/375342 consumed_samples: 6144000 total_loss: 5.752 time: 0.3210 s/iter data_time: 0.2106 s/iter total_throughput: 3190.10 samples/s lr: 2.38e-04 [09/26 09:34:45] lb.utils.events INFO: eta: 17:18:58 iteration: 6099/375342 consumed_samples: 6246400 total_loss: 5.742 time: 0.3211 s/iter data_time: 0.2377 s/iter total_throughput: 3189.08 samples/s lr: 2.42e-04 [09/26 09:35:17] lb.utils.events INFO: eta: 17:04:04 iteration: 6199/375342 consumed_samples: 6348800 total_loss: 5.74 time: 0.3211 s/iter data_time: 0.2210 s/iter total_throughput: 3188.84 samples/s lr: 2.46e-04 [09/26 09:35:49] lb.utils.events INFO: eta: 17:05:24 iteration: 6299/375342 consumed_samples: 6451200 total_loss: 5.727 time: 0.3211 s/iter data_time: 0.2252 s/iter total_throughput: 3188.58 samples/s lr: 2.50e-04 [09/26 09:36:22] lb.utils.events INFO: eta: 16:51:06 iteration: 6399/375342 consumed_samples: 6553600 total_loss: 5.703 time: 0.3212 s/iter data_time: 0.1956 s/iter total_throughput: 3188.40 samples/s lr: 2.54e-04 [09/26 09:36:53] lb.utils.events INFO: eta: 16:43:23 iteration: 6499/375342 consumed_samples: 6656000 total_loss: 5.697 time: 0.3211 s/iter data_time: 0.2170 s/iter total_throughput: 3189.14 samples/s lr: 2.58e-04 [09/26 09:37:25] lb.utils.events INFO: eta: 16:45:29 iteration: 6599/375342 consumed_samples: 6758400 total_loss: 5.666 time: 0.3211 s/iter data_time: 0.2128 s/iter total_throughput: 3189.11 samples/s lr: 2.62e-04 [09/26 09:37:58] lb.utils.events INFO: eta: 16:36:47 iteration: 6699/375342 consumed_samples: 6860800 total_loss: 5.664 time: 0.3211 s/iter data_time: 0.2028 s/iter total_throughput: 3189.03 samples/s lr: 2.66e-04 [09/26 09:38:29] lb.utils.events INFO: eta: 16:24:20 iteration: 6799/375342 consumed_samples: 6963200 total_loss: 5.662 time: 0.3211 s/iter data_time: 0.1979 s/iter total_throughput: 3189.27 samples/s lr: 2.69e-04 [09/26 09:39:02] lb.utils.events INFO: eta: 16:23:50 iteration: 6899/375342 consumed_samples: 7065600 total_loss: 5.637 time: 0.3211 s/iter data_time: 0.2358 s/iter total_throughput: 3189.20 samples/s lr: 2.73e-04 [09/26 09:39:34] lb.utils.events INFO: eta: 16:20:15 iteration: 6999/375342 consumed_samples: 7168000 total_loss: 5.625 time: 0.3211 s/iter data_time: 0.2168 s/iter total_throughput: 3189.41 samples/s lr: 2.77e-04 [09/26 09:40:06] lb.utils.events INFO: eta: 16:20:15 iteration: 7099/375342 consumed_samples: 7270400 total_loss: 5.613 time: 0.3211 s/iter data_time: 0.2287 s/iter total_throughput: 3188.81 samples/s lr: 2.81e-04 [09/26 09:40:38] lb.utils.events INFO: eta: 16:20:14 iteration: 7199/375342 consumed_samples: 7372800 total_loss: 5.615 time: 0.3211 s/iter data_time: 0.2274 s/iter total_throughput: 3188.77 samples/s lr: 2.85e-04 [09/26 09:41:10] lb.utils.events INFO: eta: 16:21:09 iteration: 7299/375342 consumed_samples: 7475200 total_loss: 5.586 time: 0.3211 s/iter data_time: 0.2247 s/iter total_throughput: 3188.89 samples/s lr: 2.89e-04 [09/26 09:41:43] lb.utils.events INFO: eta: 16:22:23 iteration: 7399/375342 consumed_samples: 7577600 total_loss: 5.559 time: 0.3211 s/iter data_time: 0.2026 s/iter total_throughput: 3188.56 samples/s lr: 2.93e-04 [09/26 09:42:15] lb.utils.events INFO: eta: 16:31:54 iteration: 7499/375342 consumed_samples: 7680000 total_loss: 5.574 time: 0.3211 s/iter data_time: 0.2346 s/iter total_throughput: 3188.73 samples/s lr: 2.97e-04 [09/26 09:42:47] lb.utils.events INFO: eta: 16:32:37 iteration: 7599/375342 consumed_samples: 7782400 total_loss: 5.566 time: 0.3212 s/iter data_time: 0.2152 s/iter total_throughput: 3188.43 samples/s lr: 3.01e-04 [09/26 09:43:19] lb.utils.events INFO: eta: 16:37:22 iteration: 7699/375342 consumed_samples: 7884800 total_loss: 5.559 time: 0.3212 s/iter data_time: 0.2170 s/iter total_throughput: 3188.36 samples/s lr: 3.05e-04 [09/26 09:43:53] lb.utils.events INFO: eta: 16:53:23 iteration: 7799/375342 consumed_samples: 7987200 total_loss: 5.555 time: 0.3213 s/iter data_time: 0.2067 s/iter total_throughput: 3186.64 samples/s lr: 3.09e-04 [09/26 09:44:25] lb.utils.events INFO: eta: 16:34:29 iteration: 7899/375342 consumed_samples: 8089600 total_loss: 5.539 time: 0.3213 s/iter data_time: 0.1940 s/iter total_throughput: 3186.57 samples/s lr: 3.13e-04 [09/26 09:44:57] lb.utils.events INFO: eta: 16:26:38 iteration: 7999/375342 consumed_samples: 8192000 total_loss: 5.52 time: 0.3213 s/iter data_time: 0.1961 s/iter total_throughput: 3186.89 samples/s lr: 3.17e-04 [09/26 09:45:29] lb.utils.events INFO: eta: 16:26:43 iteration: 8099/375342 consumed_samples: 8294400 total_loss: 5.512 time: 0.3213 s/iter data_time: 0.2205 s/iter total_throughput: 3187.07 samples/s lr: 3.21e-04 [09/26 09:46:01] lb.utils.events INFO: eta: 16:12:36 iteration: 8199/375342 consumed_samples: 8396800 total_loss: 5.502 time: 0.3213 s/iter data_time: 0.1983 s/iter total_throughput: 3186.65 samples/s lr: 3.25e-04 [09/26 09:46:34] lb.utils.events INFO: eta: 16:05:18 iteration: 8299/375342 consumed_samples: 8499200 total_loss: 5.5 time: 0.3214 s/iter data_time: 0.2236 s/iter total_throughput: 3186.19 samples/s lr: 3.29e-04 [09/26 09:47:06] lb.utils.events INFO: eta: 16:06:01 iteration: 8399/375342 consumed_samples: 8601600 total_loss: 5.504 time: 0.3214 s/iter data_time: 0.2127 s/iter total_throughput: 3185.91 samples/s lr: 3.33e-04 [09/26 09:47:39] lb.utils.events INFO: eta: 16:03:31 iteration: 8499/375342 consumed_samples: 8704000 total_loss: 5.5 time: 0.3215 s/iter data_time: 0.2276 s/iter total_throughput: 3185.09 samples/s lr: 3.37e-04 [09/26 09:48:11] lb.utils.events INFO: eta: 15:59:42 iteration: 8599/375342 consumed_samples: 8806400 total_loss: 5.477 time: 0.3215 s/iter data_time: 0.2133 s/iter total_throughput: 3185.47 samples/s lr: 3.41e-04 [09/26 09:48:43] lb.utils.events INFO: eta: 16:00:48 iteration: 8699/375342 consumed_samples: 8908800 total_loss: 5.467 time: 0.3215 s/iter data_time: 0.2075 s/iter total_throughput: 3185.11 samples/s lr: 3.45e-04 [09/26 09:49:16] lb.utils.events INFO: eta: 16:01:17 iteration: 8799/375342 consumed_samples: 9011200 total_loss: 5.453 time: 0.3215 s/iter data_time: 0.2152 s/iter total_throughput: 3184.87 samples/s lr: 3.48e-04 [09/26 09:49:48] lb.utils.events INFO: eta: 16:08:34 iteration: 8899/375342 consumed_samples: 9113600 total_loss: 5.439 time: 0.3215 s/iter data_time: 0.2083 s/iter total_throughput: 3185.09 samples/s lr: 3.52e-04 [09/26 09:50:20] lb.utils.events INFO: eta: 16:12:41 iteration: 8999/375342 consumed_samples: 9216000 total_loss: 5.422 time: 0.3215 s/iter data_time: 0.2008 s/iter total_throughput: 3184.99 samples/s lr: 3.56e-04 [09/26 09:50:52] lb.utils.events INFO: eta: 16:05:48 iteration: 9099/375342 consumed_samples: 9318400 total_loss: 5.396 time: 0.3215 s/iter data_time: 0.1951 s/iter total_throughput: 3185.32 samples/s lr: 3.60e-04 [09/26 09:51:25] lb.utils.events INFO: eta: 16:13:39 iteration: 9199/375342 consumed_samples: 9420800 total_loss: 5.389 time: 0.3215 s/iter data_time: 0.2121 s/iter total_throughput: 3184.73 samples/s lr: 3.64e-04 [09/26 09:51:57] lb.utils.events INFO: eta: 16:08:38 iteration: 9299/375342 consumed_samples: 9523200 total_loss: 5.395 time: 0.3216 s/iter data_time: 0.2084 s/iter total_throughput: 3184.39 samples/s lr: 3.68e-04 [09/26 09:52:29] lb.utils.events INFO: eta: 15:59:29 iteration: 9399/375342 consumed_samples: 9625600 total_loss: 5.391 time: 0.3216 s/iter data_time: 0.2092 s/iter total_throughput: 3184.45 samples/s lr: 3.72e-04 [09/26 09:53:01] lb.utils.events INFO: eta: 15:55:25 iteration: 9499/375342 consumed_samples: 9728000 total_loss: 5.369 time: 0.3215 s/iter data_time: 0.1906 s/iter total_throughput: 3184.83 samples/s lr: 3.76e-04 [09/26 09:53:33] lb.utils.events INFO: eta: 15:53:59 iteration: 9599/375342 consumed_samples: 9830400 total_loss: 5.359 time: 0.3215 s/iter data_time: 0.2025 s/iter total_throughput: 3184.83 samples/s lr: 3.80e-04 [09/26 09:54:05] lb.utils.events INFO: eta: 15:49:15 iteration: 9699/375342 consumed_samples: 9932800 total_loss: 5.352 time: 0.3216 s/iter data_time: 0.2016 s/iter total_throughput: 3184.55 samples/s lr: 3.84e-04 [09/26 09:54:38] lb.utils.events INFO: eta: 15:44:41 iteration: 9799/375342 consumed_samples: 10035200 total_loss: 5.342 time: 0.3216 s/iter data_time: 0.2043 s/iter total_throughput: 3184.49 samples/s lr: 3.88e-04 [09/26 09:55:10] lb.utils.events INFO: eta: 15:41:08 iteration: 9899/375342 consumed_samples: 10137600 total_loss: 5.328 time: 0.3216 s/iter data_time: 0.1940 s/iter total_throughput: 3184.38 samples/s lr: 3.92e-04 [09/26 09:55:42] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0009999 [09/26 09:55:43] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/26 09:55:43] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/26 09:55:47] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0856 s/iter. Inference: 0.1571 s/iter. Eval: 0.0020 s/iter. Total: 0.2447 s/iter. ETA=0:00:09 [09/26 09:55:53] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1467 s/iter. Inference: 0.1531 s/iter. Eval: 0.0021 s/iter. Total: 0.3020 s/iter. ETA=0:00:05 [09/26 09:55:58] lb.evaluation.evaluator INFO: Inference done 49152/50000. Dataloading: 0.1319 s/iter. Inference: 0.1516 s/iter. Eval: 0.0020 s/iter. Total: 0.2856 s/iter. ETA=0:00:00 [09/26 09:55:58] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/26 09:55:58] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.766354 (0.000255 s / iter per device, on 8 devices) [09/26 09:55:58] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000134 s / iter per device, on 8 devices) [09/26 09:55:58] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/26 09:55:58] lb.evaluation.utils INFO: copypaste: Acc@1=36.656 [09/26 09:55:58] lb.evaluation.utils INFO: copypaste: Acc@5=62.088 [09/26 09:55:58] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 36.65600, better than last best score 18.09400 @ iteration 4999. [09/26 09:55:58] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/26 09:55:59] lb.utils.events INFO: eta: 15:38:52 iteration: 9999/375342 consumed_samples: 10240000 total_loss: 5.32 time: 0.3216 s/iter data_time: 0.2180 s/iter total_throughput: 3184.50 samples/s lr: 3.96e-04 [09/26 09:56:30] lb.utils.events INFO: eta: 15:40:23 iteration: 10099/375342 consumed_samples: 10342400 total_loss: 5.318 time: 0.3214 s/iter data_time: 0.2183 s/iter total_throughput: 3185.87 samples/s lr: 4.00e-04 [09/26 09:57:02] lb.utils.events INFO: eta: 15:36:12 iteration: 10199/375342 consumed_samples: 10444800 total_loss: 5.309 time: 0.3215 s/iter data_time: 0.2068 s/iter total_throughput: 3185.35 samples/s lr: 4.04e-04 [09/26 09:57:35] lb.utils.events INFO: eta: 15:37:00 iteration: 10299/375342 consumed_samples: 10547200 total_loss: 5.301 time: 0.3215 s/iter data_time: 0.2168 s/iter total_throughput: 3185.35 samples/s lr: 4.08e-04 [09/26 09:58:07] lb.utils.events INFO: eta: 15:43:08 iteration: 10399/375342 consumed_samples: 10649600 total_loss: 5.299 time: 0.3215 s/iter data_time: 0.2303 s/iter total_throughput: 3185.10 samples/s lr: 4.12e-04 [09/26 09:58:39] lb.utils.events INFO: eta: 15:52:34 iteration: 10499/375342 consumed_samples: 10752000 total_loss: 5.305 time: 0.3215 s/iter data_time: 0.2273 s/iter total_throughput: 3185.09 samples/s lr: 4.16e-04 [09/26 09:59:12] lb.utils.events INFO: eta: 15:59:25 iteration: 10599/375342 consumed_samples: 10854400 total_loss: 5.283 time: 0.3215 s/iter data_time: 0.2137 s/iter total_throughput: 3184.82 samples/s lr: 4.20e-04 [09/26 09:59:44] lb.utils.events INFO: eta: 16:09:57 iteration: 10699/375342 consumed_samples: 10956800 total_loss: 5.246 time: 0.3215 s/iter data_time: 0.2133 s/iter total_throughput: 3184.90 samples/s lr: 4.24e-04 [09/26 10:00:16] lb.utils.events INFO: eta: 16:22:49 iteration: 10799/375342 consumed_samples: 11059200 total_loss: 5.234 time: 0.3216 s/iter data_time: 0.2198 s/iter total_throughput: 3184.52 samples/s lr: 4.27e-04 [09/26 10:00:48] lb.utils.events INFO: eta: 16:40:17 iteration: 10899/375342 consumed_samples: 11161600 total_loss: 5.254 time: 0.3215 s/iter data_time: 0.2388 s/iter total_throughput: 3184.83 samples/s lr: 4.31e-04 [09/26 10:01:21] lb.utils.events INFO: eta: 17:30:07 iteration: 10999/375342 consumed_samples: 11264000 total_loss: 5.258 time: 0.3216 s/iter data_time: 0.2386 s/iter total_throughput: 3184.51 samples/s lr: 4.35e-04 [09/26 10:01:53] lb.utils.events INFO: eta: 17:14:12 iteration: 11099/375342 consumed_samples: 11366400 total_loss: 5.236 time: 0.3216 s/iter data_time: 0.2112 s/iter total_throughput: 3184.56 samples/s lr: 4.39e-04 [09/26 10:02:25] lb.utils.events INFO: eta: 17:18:37 iteration: 11199/375342 consumed_samples: 11468800 total_loss: 5.225 time: 0.3215 s/iter data_time: 0.2182 s/iter total_throughput: 3184.79 samples/s lr: 4.43e-04 [09/26 10:02:57] lb.utils.events INFO: eta: 18:20:20 iteration: 11299/375342 consumed_samples: 11571200 total_loss: 5.229 time: 0.3215 s/iter data_time: 0.2101 s/iter total_throughput: 3184.89 samples/s lr: 4.47e-04 [09/26 10:03:29] lb.utils.events INFO: eta: 16:57:19 iteration: 11399/375342 consumed_samples: 11673600 total_loss: 5.217 time: 0.3215 s/iter data_time: 0.2122 s/iter total_throughput: 3184.60 samples/s lr: 4.51e-04 [09/26 10:04:01] lb.utils.events INFO: eta: 16:26:16 iteration: 11499/375342 consumed_samples: 11776000 total_loss: 5.211 time: 0.3215 s/iter data_time: 0.2002 s/iter total_throughput: 3184.67 samples/s lr: 4.55e-04 [09/26 10:04:34] lb.utils.events INFO: eta: 16:20:36 iteration: 11599/375342 consumed_samples: 11878400 total_loss: 5.211 time: 0.3216 s/iter data_time: 0.2296 s/iter total_throughput: 3184.31 samples/s lr: 4.59e-04 [09/26 10:05:06] lb.utils.events INFO: eta: 16:22:57 iteration: 11699/375342 consumed_samples: 11980800 total_loss: 5.195 time: 0.3216 s/iter data_time: 0.2252 s/iter total_throughput: 3184.42 samples/s lr: 4.63e-04 [09/26 10:05:38] lb.utils.events INFO: eta: 16:22:41 iteration: 11799/375342 consumed_samples: 12083200 total_loss: 5.168 time: 0.3215 s/iter data_time: 0.1992 s/iter total_throughput: 3184.66 samples/s lr: 4.67e-04 [09/26 10:06:11] lb.utils.events INFO: eta: 16:08:07 iteration: 11899/375342 consumed_samples: 12185600 total_loss: 5.17 time: 0.3216 s/iter data_time: 0.2179 s/iter total_throughput: 3183.96 samples/s lr: 4.71e-04 [09/26 10:06:44] lb.utils.events INFO: eta: 15:57:04 iteration: 11999/375342 consumed_samples: 12288000 total_loss: 5.188 time: 0.3217 s/iter data_time: 0.2216 s/iter total_throughput: 3183.41 samples/s lr: 4.75e-04 [09/26 10:07:16] lb.utils.events INFO: eta: 15:54:52 iteration: 12099/375342 consumed_samples: 12390400 total_loss: 5.176 time: 0.3217 s/iter data_time: 0.1937 s/iter total_throughput: 3183.40 samples/s lr: 4.79e-04 [09/26 10:07:48] lb.utils.events INFO: eta: 15:54:53 iteration: 12199/375342 consumed_samples: 12492800 total_loss: 5.168 time: 0.3217 s/iter data_time: 0.2228 s/iter total_throughput: 3183.44 samples/s lr: 4.83e-04 [09/26 10:08:21] lb.utils.events INFO: eta: 15:49:26 iteration: 12299/375342 consumed_samples: 12595200 total_loss: 5.17 time: 0.3217 s/iter data_time: 0.2298 s/iter total_throughput: 3182.95 samples/s lr: 4.87e-04 [09/26 10:08:53] lb.utils.events INFO: eta: 15:54:54 iteration: 12399/375342 consumed_samples: 12697600 total_loss: 5.141 time: 0.3217 s/iter data_time: 0.2086 s/iter total_throughput: 3182.98 samples/s lr: 4.91e-04 [09/26 10:09:26] lb.utils.events INFO: eta: 16:00:06 iteration: 12499/375342 consumed_samples: 12800000 total_loss: 5.105 time: 0.3218 s/iter data_time: 0.2266 s/iter total_throughput: 3182.56 samples/s lr: 4.95e-04 [09/26 10:09:58] lb.utils.events INFO: eta: 16:00:32 iteration: 12599/375342 consumed_samples: 12902400 total_loss: 5.107 time: 0.3218 s/iter data_time: 0.2228 s/iter total_throughput: 3182.55 samples/s lr: 4.99e-04 [09/26 10:10:30] lb.utils.events INFO: eta: 15:50:05 iteration: 12699/375342 consumed_samples: 13004800 total_loss: 5.125 time: 0.3218 s/iter data_time: 0.1969 s/iter total_throughput: 3182.54 samples/s lr: 5.02e-04 [09/26 10:11:02] lb.utils.events INFO: eta: 15:47:27 iteration: 12799/375342 consumed_samples: 13107200 total_loss: 5.123 time: 0.3218 s/iter data_time: 0.2196 s/iter total_throughput: 3182.51 samples/s lr: 5.06e-04 [09/26 10:11:35] lb.utils.events INFO: eta: 15:46:50 iteration: 12899/375342 consumed_samples: 13209600 total_loss: 5.104 time: 0.3218 s/iter data_time: 0.1936 s/iter total_throughput: 3182.38 samples/s lr: 5.10e-04 [09/26 10:12:07] lb.utils.events INFO: eta: 15:45:50 iteration: 12999/375342 consumed_samples: 13312000 total_loss: 5.09 time: 0.3218 s/iter data_time: 0.2007 s/iter total_throughput: 3182.26 samples/s lr: 5.14e-04 [09/26 10:12:39] lb.utils.events INFO: eta: 15:42:24 iteration: 13099/375342 consumed_samples: 13414400 total_loss: 5.086 time: 0.3218 s/iter data_time: 0.2089 s/iter total_throughput: 3182.32 samples/s lr: 5.18e-04 [09/26 10:13:12] lb.utils.events INFO: eta: 15:39:06 iteration: 13199/375342 consumed_samples: 13516800 total_loss: 5.092 time: 0.3218 s/iter data_time: 0.2087 s/iter total_throughput: 3182.09 samples/s lr: 5.22e-04 [09/26 10:13:44] lb.utils.events INFO: eta: 15:35:58 iteration: 13299/375342 consumed_samples: 13619200 total_loss: 5.094 time: 0.3218 s/iter data_time: 0.2034 s/iter total_throughput: 3181.62 samples/s lr: 5.26e-04 [09/26 10:14:16] lb.utils.events INFO: eta: 15:33:51 iteration: 13399/375342 consumed_samples: 13721600 total_loss: 5.08 time: 0.3218 s/iter data_time: 0.2180 s/iter total_throughput: 3181.67 samples/s lr: 5.30e-04 [09/26 10:14:49] lb.utils.events INFO: eta: 15:33:47 iteration: 13499/375342 consumed_samples: 13824000 total_loss: 5.043 time: 0.3218 s/iter data_time: 0.2206 s/iter total_throughput: 3181.72 samples/s lr: 5.34e-04 [09/26 10:15:21] lb.utils.events INFO: eta: 15:33:31 iteration: 13599/375342 consumed_samples: 13926400 total_loss: 5.049 time: 0.3219 s/iter data_time: 0.2236 s/iter total_throughput: 3181.45 samples/s lr: 5.38e-04 [09/26 10:15:54] lb.utils.events INFO: eta: 15:37:40 iteration: 13699/375342 consumed_samples: 14028800 total_loss: 5.068 time: 0.3219 s/iter data_time: 0.2197 s/iter total_throughput: 3181.27 samples/s lr: 5.42e-04 [09/26 10:16:27] lb.utils.events INFO: eta: 15:32:07 iteration: 13799/375342 consumed_samples: 14131200 total_loss: 5.043 time: 0.3219 s/iter data_time: 0.2114 s/iter total_throughput: 3180.67 samples/s lr: 5.46e-04 [09/26 10:16:59] lb.utils.events INFO: eta: 15:32:59 iteration: 13899/375342 consumed_samples: 14233600 total_loss: 5.039 time: 0.3219 s/iter data_time: 0.1972 s/iter total_throughput: 3180.87 samples/s lr: 5.50e-04 [09/26 10:17:31] lb.utils.events INFO: eta: 15:32:44 iteration: 13999/375342 consumed_samples: 14336000 total_loss: 5.031 time: 0.3219 s/iter data_time: 0.2019 s/iter total_throughput: 3180.63 samples/s lr: 5.54e-04 [09/26 10:18:04] lb.utils.events INFO: eta: 15:34:33 iteration: 14099/375342 consumed_samples: 14438400 total_loss: 5.031 time: 0.3220 s/iter data_time: 0.2271 s/iter total_throughput: 3180.34 samples/s lr: 5.58e-04 [09/26 10:18:37] lb.utils.events INFO: eta: 15:38:21 iteration: 14199/375342 consumed_samples: 14540800 total_loss: 5.006 time: 0.3220 s/iter data_time: 0.2281 s/iter total_throughput: 3179.88 samples/s lr: 5.62e-04 [09/26 10:19:09] lb.utils.events INFO: eta: 15:46:04 iteration: 14299/375342 consumed_samples: 14643200 total_loss: 4.998 time: 0.3220 s/iter data_time: 0.2165 s/iter total_throughput: 3179.66 samples/s lr: 5.66e-04 [09/26 10:19:42] lb.utils.events INFO: eta: 15:57:25 iteration: 14399/375342 consumed_samples: 14745600 total_loss: 5 time: 0.3221 s/iter data_time: 0.2392 s/iter total_throughput: 3179.16 samples/s lr: 5.70e-04 [09/26 10:20:16] lb.utils.events INFO: eta: 16:05:03 iteration: 14499/375342 consumed_samples: 14848000 total_loss: 5.016 time: 0.3222 s/iter data_time: 0.2620 s/iter total_throughput: 3178.14 samples/s lr: 5.74e-04 [09/26 10:20:49] lb.utils.events INFO: eta: 16:05:20 iteration: 14599/375342 consumed_samples: 14950400 total_loss: 4.982 time: 0.3223 s/iter data_time: 0.2407 s/iter total_throughput: 3177.40 samples/s lr: 5.78e-04 [09/26 10:21:23] lb.utils.events INFO: eta: 16:33:15 iteration: 14699/375342 consumed_samples: 15052800 total_loss: 4.957 time: 0.3224 s/iter data_time: 0.2491 s/iter total_throughput: 3176.55 samples/s lr: 5.81e-04 [09/26 10:21:56] lb.utils.events INFO: eta: 16:59:56 iteration: 14799/375342 consumed_samples: 15155200 total_loss: 4.961 time: 0.3224 s/iter data_time: 0.2160 s/iter total_throughput: 3175.95 samples/s lr: 5.85e-04 [09/26 10:22:29] lb.utils.events INFO: eta: 17:21:48 iteration: 14899/375342 consumed_samples: 15257600 total_loss: 4.967 time: 0.3225 s/iter data_time: 0.2182 s/iter total_throughput: 3175.19 samples/s lr: 5.89e-04 [09/26 10:23:03] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0014999 [09/26 10:23:03] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/26 10:23:03] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/26 10:23:07] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0861 s/iter. Inference: 0.1470 s/iter. Eval: 0.0021 s/iter. Total: 0.2353 s/iter. ETA=0:00:08 [09/26 10:23:13] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1447 s/iter. Inference: 0.1499 s/iter. Eval: 0.0020 s/iter. Total: 0.2966 s/iter. ETA=0:00:05 [09/26 10:23:18] lb.evaluation.evaluator INFO: Inference done 50000/50000. Dataloading: 0.1323 s/iter. Inference: 0.1505 s/iter. Eval: 0.0020 s/iter. Total: 0.2849 s/iter. ETA=0:00:00 [09/26 10:23:18] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/26 10:23:18] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.539075 (0.000251 s / iter per device, on 8 devices) [09/26 10:23:18] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000132 s / iter per device, on 8 devices) [09/26 10:23:18] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/26 10:23:18] lb.evaluation.utils INFO: copypaste: Acc@1=46.652 [09/26 10:23:18] lb.evaluation.utils INFO: copypaste: Acc@5=71.97200000000001 [09/26 10:23:18] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 46.65200, better than last best score 36.65600 @ iteration 9999. [09/26 10:23:18] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/26 10:23:19] lb.utils.events INFO: eta: 18:19:31 iteration: 14999/375342 consumed_samples: 15360000 total_loss: 4.965 time: 0.3226 s/iter data_time: 0.2307 s/iter total_throughput: 3174.50 samples/s lr: 5.93e-04 [09/26 10:23:50] lb.utils.events INFO: eta: 20:02:15 iteration: 15099/375342 consumed_samples: 15462400 total_loss: 4.934 time: 0.3225 s/iter data_time: 0.2445 s/iter total_throughput: 3175.29 samples/s lr: 5.97e-04 [09/26 10:24:23] lb.utils.events INFO: eta: 21:20:46 iteration: 15199/375342 consumed_samples: 15564800 total_loss: 4.941 time: 0.3225 s/iter data_time: 0.2380 s/iter total_throughput: 3174.80 samples/s lr: 6.01e-04 [09/26 10:24:56] lb.utils.events INFO: eta: 21:58:42 iteration: 15299/375342 consumed_samples: 15667200 total_loss: 4.936 time: 0.3226 s/iter data_time: 0.2371 s/iter total_throughput: 3174.28 samples/s lr: 6.05e-04 [09/26 10:25:29] lb.utils.events INFO: eta: 22:48:28 iteration: 15399/375342 consumed_samples: 15769600 total_loss: 4.918 time: 0.3227 s/iter data_time: 0.2242 s/iter total_throughput: 3173.63 samples/s lr: 6.09e-04 [09/26 10:26:03] lb.utils.events INFO: eta: 22:40:36 iteration: 15499/375342 consumed_samples: 15872000 total_loss: 4.93 time: 0.3227 s/iter data_time: 0.2491 s/iter total_throughput: 3172.99 samples/s lr: 6.13e-04 [09/26 10:26:35] lb.utils.events INFO: eta: 20:20:19 iteration: 15599/375342 consumed_samples: 15974400 total_loss: 4.939 time: 0.3228 s/iter data_time: 0.2000 s/iter total_throughput: 3172.70 samples/s lr: 6.17e-04 [09/26 10:27:08] lb.utils.events INFO: eta: 17:47:41 iteration: 15699/375342 consumed_samples: 16076800 total_loss: 4.938 time: 0.3228 s/iter data_time: 0.2226 s/iter total_throughput: 3172.42 samples/s lr: 6.21e-04 [09/26 10:27:41] lb.utils.events INFO: eta: 18:02:20 iteration: 15799/375342 consumed_samples: 16179200 total_loss: 4.938 time: 0.3228 s/iter data_time: 0.2307 s/iter total_throughput: 3172.07 samples/s lr: 6.25e-04 [09/26 10:28:14] lb.utils.events INFO: eta: 20:07:05 iteration: 15899/375342 consumed_samples: 16281600 total_loss: 4.924 time: 0.3229 s/iter data_time: 0.2375 s/iter total_throughput: 3171.73 samples/s lr: 6.29e-04 [09/26 10:28:46] lb.utils.events INFO: eta: 23:50:58 iteration: 15999/375342 consumed_samples: 16384000 total_loss: 4.896 time: 0.3229 s/iter data_time: 0.2427 s/iter total_throughput: 3171.57 samples/s lr: 6.33e-04 [09/26 10:29:19] lb.utils.events INFO: eta: 1 day, 0:19:15 iteration: 16099/375342 consumed_samples: 16486400 total_loss: 4.896 time: 0.3229 s/iter data_time: 0.2308 s/iter total_throughput: 3171.48 samples/s lr: 6.37e-04 [09/26 10:29:52] lb.utils.events INFO: eta: 18:38:42 iteration: 16199/375342 consumed_samples: 16588800 total_loss: 4.906 time: 0.3230 s/iter data_time: 0.2019 s/iter total_throughput: 3170.68 samples/s lr: 6.41e-04 [09/26 10:30:25] lb.utils.events INFO: eta: 16:34:05 iteration: 16299/375342 consumed_samples: 16691200 total_loss: 4.895 time: 0.3230 s/iter data_time: 0.2178 s/iter total_throughput: 3170.41 samples/s lr: 6.45e-04 [09/26 10:30:58] lb.utils.events INFO: eta: 16:15:47 iteration: 16399/375342 consumed_samples: 16793600 total_loss: 4.879 time: 0.3230 s/iter data_time: 0.2310 s/iter total_throughput: 3170.26 samples/s lr: 6.49e-04 [09/26 10:31:31] lb.utils.events INFO: eta: 16:24:34 iteration: 16499/375342 consumed_samples: 16896000 total_loss: 4.867 time: 0.3230 s/iter data_time: 0.2419 s/iter total_throughput: 3169.92 samples/s lr: 6.53e-04 [09/26 10:32:03] lb.utils.events INFO: eta: 19:14:43 iteration: 16599/375342 consumed_samples: 16998400 total_loss: 4.85 time: 0.3231 s/iter data_time: 0.2501 s/iter total_throughput: 3169.67 samples/s lr: 6.57e-04 [09/26 10:32:36] lb.utils.events INFO: eta: 21:10:35 iteration: 16699/375342 consumed_samples: 17100800 total_loss: 4.848 time: 0.3231 s/iter data_time: 0.2177 s/iter total_throughput: 3169.40 samples/s lr: 6.60e-04 [09/26 10:33:09] lb.utils.events INFO: eta: 18:59:32 iteration: 16799/375342 consumed_samples: 17203200 total_loss: 4.867 time: 0.3231 s/iter data_time: 0.2108 s/iter total_throughput: 3169.13 samples/s lr: 6.64e-04 [09/26 10:33:42] lb.utils.events INFO: eta: 18:30:04 iteration: 16899/375342 consumed_samples: 17305600 total_loss: 4.875 time: 0.3232 s/iter data_time: 0.2155 s/iter total_throughput: 3168.58 samples/s lr: 6.68e-04 [09/26 10:34:15] lb.utils.events INFO: eta: 16:48:07 iteration: 16999/375342 consumed_samples: 17408000 total_loss: 4.861 time: 0.3232 s/iter data_time: 0.2318 s/iter total_throughput: 3168.39 samples/s lr: 6.72e-04 [09/26 10:34:48] lb.utils.events INFO: eta: 16:57:05 iteration: 17099/375342 consumed_samples: 17510400 total_loss: 4.85 time: 0.3232 s/iter data_time: 0.2435 s/iter total_throughput: 3167.97 samples/s lr: 6.76e-04 [09/26 10:35:20] lb.utils.events INFO: eta: 18:51:56 iteration: 17199/375342 consumed_samples: 17612800 total_loss: 4.852 time: 0.3232 s/iter data_time: 0.2346 s/iter total_throughput: 3167.85 samples/s lr: 6.80e-04 [09/26 10:35:53] lb.utils.events INFO: eta: 23:43:43 iteration: 17299/375342 consumed_samples: 17715200 total_loss: 4.852 time: 0.3233 s/iter data_time: 0.2424 s/iter total_throughput: 3167.72 samples/s lr: 6.84e-04 [09/26 10:36:26] lb.utils.events INFO: eta: 23:30:15 iteration: 17399/375342 consumed_samples: 17817600 total_loss: 4.834 time: 0.3233 s/iter data_time: 0.1995 s/iter total_throughput: 3167.36 samples/s lr: 6.88e-04 [09/26 10:36:59] lb.utils.events INFO: eta: 23:13:23 iteration: 17499/375342 consumed_samples: 17920000 total_loss: 4.818 time: 0.3233 s/iter data_time: 0.2276 s/iter total_throughput: 3167.11 samples/s lr: 6.92e-04 [09/26 10:37:32] lb.utils.events INFO: eta: 22:47:27 iteration: 17599/375342 consumed_samples: 18022400 total_loss: 4.809 time: 0.3233 s/iter data_time: 0.2382 s/iter total_throughput: 3166.87 samples/s lr: 6.96e-04 [09/26 10:38:05] lb.utils.events INFO: eta: 20:01:40 iteration: 17699/375342 consumed_samples: 18124800 total_loss: 4.807 time: 0.3234 s/iter data_time: 0.2181 s/iter total_throughput: 3166.41 samples/s lr: 7.00e-04 [09/26 10:38:37] lb.utils.events INFO: eta: 18:25:03 iteration: 17799/375342 consumed_samples: 18227200 total_loss: 4.809 time: 0.3234 s/iter data_time: 0.2206 s/iter total_throughput: 3166.22 samples/s lr: 7.04e-04 [09/26 10:39:10] lb.utils.events INFO: eta: 17:25:26 iteration: 17899/375342 consumed_samples: 18329600 total_loss: 4.814 time: 0.3235 s/iter data_time: 0.2339 s/iter total_throughput: 3165.80 samples/s lr: 7.08e-04 [09/26 10:39:44] lb.utils.events INFO: eta: 17:32:04 iteration: 17999/375342 consumed_samples: 18432000 total_loss: 4.797 time: 0.3235 s/iter data_time: 0.2251 s/iter total_throughput: 3165.38 samples/s lr: 7.12e-04 [09/26 10:40:17] lb.utils.events INFO: eta: 16:33:04 iteration: 18099/375342 consumed_samples: 18534400 total_loss: 4.777 time: 0.3235 s/iter data_time: 0.2341 s/iter total_throughput: 3165.00 samples/s lr: 7.16e-04 [09/26 10:40:50] lb.utils.events INFO: eta: 16:00:25 iteration: 18199/375342 consumed_samples: 18636800 total_loss: 4.797 time: 0.3236 s/iter data_time: 0.2030 s/iter total_throughput: 3164.55 samples/s lr: 7.20e-04 [09/26 10:41:22] lb.utils.events INFO: eta: 15:41:58 iteration: 18299/375342 consumed_samples: 18739200 total_loss: 4.811 time: 0.3236 s/iter data_time: 0.2033 s/iter total_throughput: 3164.47 samples/s lr: 7.24e-04 [09/26 10:41:55] lb.utils.events INFO: eta: 15:38:31 iteration: 18399/375342 consumed_samples: 18841600 total_loss: 4.809 time: 0.3236 s/iter data_time: 0.2034 s/iter total_throughput: 3164.30 samples/s lr: 7.28e-04 [09/26 10:42:28] lb.utils.events INFO: eta: 15:29:30 iteration: 18499/375342 consumed_samples: 18944000 total_loss: 4.779 time: 0.3236 s/iter data_time: 0.2001 s/iter total_throughput: 3164.14 samples/s lr: 7.32e-04 [09/26 10:43:00] lb.utils.events INFO: eta: 15:23:45 iteration: 18599/375342 consumed_samples: 19046400 total_loss: 4.744 time: 0.3236 s/iter data_time: 0.1999 s/iter total_throughput: 3164.02 samples/s lr: 7.35e-04 [09/26 10:43:33] lb.utils.events INFO: eta: 15:21:25 iteration: 18699/375342 consumed_samples: 19148800 total_loss: 4.76 time: 0.3237 s/iter data_time: 0.2028 s/iter total_throughput: 3163.82 samples/s lr: 7.39e-04 [09/26 10:44:06] lb.utils.events INFO: eta: 15:21:55 iteration: 18799/375342 consumed_samples: 19251200 total_loss: 4.756 time: 0.3237 s/iter data_time: 0.2240 s/iter total_throughput: 3163.58 samples/s lr: 7.43e-04 [09/26 10:44:39] lb.utils.events INFO: eta: 15:25:31 iteration: 18899/375342 consumed_samples: 19353600 total_loss: 4.762 time: 0.3237 s/iter data_time: 0.2268 s/iter total_throughput: 3163.37 samples/s lr: 7.47e-04 [09/26 10:45:11] lb.utils.events INFO: eta: 15:28:03 iteration: 18999/375342 consumed_samples: 19456000 total_loss: 4.756 time: 0.3237 s/iter data_time: 0.2191 s/iter total_throughput: 3163.34 samples/s lr: 7.51e-04 [09/26 10:45:44] lb.utils.events INFO: eta: 15:27:26 iteration: 19099/375342 consumed_samples: 19558400 total_loss: 4.74 time: 0.3238 s/iter data_time: 0.2287 s/iter total_throughput: 3162.91 samples/s lr: 7.55e-04 [09/26 10:46:17] lb.utils.events INFO: eta: 15:30:16 iteration: 19199/375342 consumed_samples: 19660800 total_loss: 4.742 time: 0.3238 s/iter data_time: 0.2163 s/iter total_throughput: 3162.56 samples/s lr: 7.59e-04 [09/26 10:46:50] lb.utils.events INFO: eta: 15:35:14 iteration: 19299/375342 consumed_samples: 19763200 total_loss: 4.711 time: 0.3238 s/iter data_time: 0.2161 s/iter total_throughput: 3162.34 samples/s lr: 7.63e-04 [09/26 10:47:24] lb.utils.events INFO: eta: 15:38:23 iteration: 19399/375342 consumed_samples: 19865600 total_loss: 4.734 time: 0.3239 s/iter data_time: 0.2199 s/iter total_throughput: 3161.90 samples/s lr: 7.67e-04 [09/26 10:47:57] lb.utils.events INFO: eta: 15:45:28 iteration: 19499/375342 consumed_samples: 19968000 total_loss: 4.76 time: 0.3239 s/iter data_time: 0.2145 s/iter total_throughput: 3161.51 samples/s lr: 7.71e-04 [09/26 10:48:30] lb.utils.events INFO: eta: 15:51:54 iteration: 19599/375342 consumed_samples: 20070400 total_loss: 4.75 time: 0.3239 s/iter data_time: 0.2050 s/iter total_throughput: 3161.24 samples/s lr: 7.75e-04 [09/26 10:49:02] lb.utils.events INFO: eta: 15:52:23 iteration: 19699/375342 consumed_samples: 20172800 total_loss: 4.727 time: 0.3239 s/iter data_time: 0.2205 s/iter total_throughput: 3161.04 samples/s lr: 7.79e-04 [09/26 10:49:35] lb.utils.events INFO: eta: 15:51:23 iteration: 19799/375342 consumed_samples: 20275200 total_loss: 4.703 time: 0.3240 s/iter data_time: 0.2229 s/iter total_throughput: 3160.92 samples/s lr: 7.83e-04 [09/26 10:50:08] lb.utils.events INFO: eta: 15:44:25 iteration: 19899/375342 consumed_samples: 20377600 total_loss: 4.695 time: 0.3240 s/iter data_time: 0.2029 s/iter total_throughput: 3160.58 samples/s lr: 7.87e-04 [09/26 10:50:42] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0019999 [09/26 10:50:42] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/26 10:50:42] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/26 10:50:46] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0865 s/iter. Inference: 0.1518 s/iter. Eval: 0.0020 s/iter. Total: 0.2404 s/iter. ETA=0:00:08 [09/26 10:50:52] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1484 s/iter. Inference: 0.1504 s/iter. Eval: 0.0020 s/iter. Total: 0.3009 s/iter. ETA=0:00:05 [09/26 10:50:57] lb.evaluation.evaluator INFO: Inference done 49152/50000. Dataloading: 0.1316 s/iter. Inference: 0.1508 s/iter. Eval: 0.0020 s/iter. Total: 0.2845 s/iter. ETA=0:00:00 [09/26 10:50:57] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/26 10:50:57] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.692486 (0.000254 s / iter per device, on 8 devices) [09/26 10:50:57] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000134 s / iter per device, on 8 devices) [09/26 10:50:57] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/26 10:50:57] lb.evaluation.utils INFO: copypaste: Acc@1=53.26 [09/26 10:50:57] lb.evaluation.utils INFO: copypaste: Acc@5=77.812 [09/26 10:50:57] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 53.26000, better than last best score 46.65200 @ iteration 14999. [09/26 10:50:58] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/26 10:50:58] lb.utils.events INFO: eta: 15:36:37 iteration: 19999/375342 consumed_samples: 20480000 total_loss: 4.734 time: 0.3240 s/iter data_time: 0.2311 s/iter total_throughput: 3160.19 samples/s lr: 7.91e-04 [09/26 10:51:29] lb.utils.events INFO: eta: 15:34:55 iteration: 20099/375342 consumed_samples: 20582400 total_loss: 4.727 time: 0.3240 s/iter data_time: 0.2497 s/iter total_throughput: 3160.85 samples/s lr: 7.95e-04 [09/26 10:52:03] lb.utils.events INFO: eta: 15:39:21 iteration: 20199/375342 consumed_samples: 20684800 total_loss: 4.688 time: 0.3240 s/iter data_time: 0.2225 s/iter total_throughput: 3160.47 samples/s lr: 7.99e-04 [09/26 10:52:36] lb.utils.events INFO: eta: 15:36:36 iteration: 20299/375342 consumed_samples: 20787200 total_loss: 4.688 time: 0.3241 s/iter data_time: 0.2336 s/iter total_throughput: 3159.91 samples/s lr: 8.03e-04 [09/26 10:53:09] lb.utils.events INFO: eta: 15:33:30 iteration: 20399/375342 consumed_samples: 20889600 total_loss: 4.699 time: 0.3241 s/iter data_time: 0.2080 s/iter total_throughput: 3159.65 samples/s lr: 8.07e-04 [09/26 10:53:42] lb.utils.events INFO: eta: 15:30:42 iteration: 20499/375342 consumed_samples: 20992000 total_loss: 4.703 time: 0.3241 s/iter data_time: 0.2221 s/iter total_throughput: 3159.53 samples/s lr: 8.11e-04 [09/26 10:54:15] lb.utils.events INFO: eta: 15:27:20 iteration: 20599/375342 consumed_samples: 21094400 total_loss: 4.684 time: 0.3241 s/iter data_time: 0.2164 s/iter total_throughput: 3159.16 samples/s lr: 8.14e-04 [09/26 10:54:48] lb.utils.events INFO: eta: 15:29:20 iteration: 20699/375342 consumed_samples: 21196800 total_loss: 4.684 time: 0.3241 s/iter data_time: 0.2188 s/iter total_throughput: 3159.08 samples/s lr: 8.18e-04 [09/26 10:55:20] lb.utils.events INFO: eta: 15:32:27 iteration: 20799/375342 consumed_samples: 21299200 total_loss: 4.703 time: 0.3242 s/iter data_time: 0.2181 s/iter total_throughput: 3158.95 samples/s lr: 8.22e-04 [09/26 10:55:53] lb.utils.events INFO: eta: 15:36:30 iteration: 20899/375342 consumed_samples: 21401600 total_loss: 4.713 time: 0.3242 s/iter data_time: 0.2168 s/iter total_throughput: 3158.81 samples/s lr: 8.26e-04 [09/26 10:56:25] lb.utils.events INFO: eta: 15:40:11 iteration: 20999/375342 consumed_samples: 21504000 total_loss: 4.703 time: 0.3242 s/iter data_time: 0.1977 s/iter total_throughput: 3158.84 samples/s lr: 8.30e-04 [09/26 10:56:58] lb.utils.events INFO: eta: 15:41:39 iteration: 21099/375342 consumed_samples: 21606400 total_loss: 4.699 time: 0.3242 s/iter data_time: 0.2182 s/iter total_throughput: 3158.65 samples/s lr: 8.34e-04 [09/26 10:57:31] lb.utils.events INFO: eta: 15:41:23 iteration: 21199/375342 consumed_samples: 21708800 total_loss: 4.672 time: 0.3242 s/iter data_time: 0.2034 s/iter total_throughput: 3158.34 samples/s lr: 8.38e-04 [09/26 10:58:04] lb.utils.events INFO: eta: 15:48:20 iteration: 21299/375342 consumed_samples: 21811200 total_loss: 4.664 time: 0.3242 s/iter data_time: 0.2389 s/iter total_throughput: 3158.27 samples/s lr: 8.42e-04 [09/26 10:58:37] lb.utils.events INFO: eta: 15:51:41 iteration: 21399/375342 consumed_samples: 21913600 total_loss: 4.664 time: 0.3243 s/iter data_time: 0.2248 s/iter total_throughput: 3157.97 samples/s lr: 8.46e-04 [09/26 10:59:10] lb.utils.events INFO: eta: 15:54:17 iteration: 21499/375342 consumed_samples: 22016000 total_loss: 4.666 time: 0.3243 s/iter data_time: 0.2305 s/iter total_throughput: 3157.80 samples/s lr: 8.50e-04 [09/26 10:59:43] lb.utils.events INFO: eta: 16:05:22 iteration: 21599/375342 consumed_samples: 22118400 total_loss: 4.668 time: 0.3243 s/iter data_time: 0.2225 s/iter total_throughput: 3157.62 samples/s lr: 8.54e-04 [09/26 11:00:15] lb.utils.events INFO: eta: 16:07:46 iteration: 21699/375342 consumed_samples: 22220800 total_loss: 4.643 time: 0.3243 s/iter data_time: 0.2157 s/iter total_throughput: 3157.45 samples/s lr: 8.58e-04 [09/26 11:00:48] lb.utils.events INFO: eta: 16:02:38 iteration: 21799/375342 consumed_samples: 22323200 total_loss: 4.668 time: 0.3243 s/iter data_time: 0.2311 s/iter total_throughput: 3157.21 samples/s lr: 8.62e-04 [09/26 11:01:21] lb.utils.events INFO: eta: 16:00:36 iteration: 21899/375342 consumed_samples: 22425600 total_loss: 4.664 time: 0.3243 s/iter data_time: 0.2423 s/iter total_throughput: 3157.11 samples/s lr: 8.66e-04 [09/26 11:01:54] lb.utils.events INFO: eta: 16:08:30 iteration: 21999/375342 consumed_samples: 22528000 total_loss: 4.66 time: 0.3244 s/iter data_time: 0.2603 s/iter total_throughput: 3156.91 samples/s lr: 8.70e-04 [09/26 11:02:27] lb.utils.events INFO: eta: 16:35:02 iteration: 22099/375342 consumed_samples: 22630400 total_loss: 4.664 time: 0.3244 s/iter data_time: 0.2291 s/iter total_throughput: 3156.69 samples/s lr: 8.74e-04 [09/26 11:03:00] lb.utils.events INFO: eta: 17:41:27 iteration: 22199/375342 consumed_samples: 22732800 total_loss: 4.656 time: 0.3244 s/iter data_time: 0.2263 s/iter total_throughput: 3156.48 samples/s lr: 8.78e-04 [09/26 11:03:33] lb.utils.events INFO: eta: 16:44:56 iteration: 22299/375342 consumed_samples: 22835200 total_loss: 4.625 time: 0.3244 s/iter data_time: 0.2168 s/iter total_throughput: 3156.35 samples/s lr: 8.82e-04 [09/26 11:04:06] lb.utils.events INFO: eta: 16:11:47 iteration: 22399/375342 consumed_samples: 22937600 total_loss: 4.621 time: 0.3245 s/iter data_time: 0.2079 s/iter total_throughput: 3156.07 samples/s lr: 8.86e-04 [09/26 11:04:39] lb.utils.events INFO: eta: 16:21:48 iteration: 22499/375342 consumed_samples: 23040000 total_loss: 4.623 time: 0.3245 s/iter data_time: 0.2162 s/iter total_throughput: 3155.78 samples/s lr: 8.90e-04 [09/26 11:05:12] lb.utils.events INFO: eta: 16:14:44 iteration: 22599/375342 consumed_samples: 23142400 total_loss: 4.615 time: 0.3245 s/iter data_time: 0.2248 s/iter total_throughput: 3155.63 samples/s lr: 8.93e-04 [09/26 11:05:44] lb.utils.events INFO: eta: 17:08:16 iteration: 22699/375342 consumed_samples: 23244800 total_loss: 4.609 time: 0.3245 s/iter data_time: 0.2357 s/iter total_throughput: 3155.63 samples/s lr: 8.97e-04 [09/26 11:06:16] lb.utils.events INFO: eta: 19:37:43 iteration: 22799/375342 consumed_samples: 23347200 total_loss: 4.596 time: 0.3245 s/iter data_time: 0.2289 s/iter total_throughput: 3155.71 samples/s lr: 9.01e-04 [09/26 11:06:49] lb.utils.events INFO: eta: 21:31:59 iteration: 22899/375342 consumed_samples: 23449600 total_loss: 4.607 time: 0.3245 s/iter data_time: 0.2406 s/iter total_throughput: 3155.54 samples/s lr: 9.05e-04 [09/26 11:07:22] lb.utils.events INFO: eta: 23:18:29 iteration: 22999/375342 consumed_samples: 23552000 total_loss: 4.604 time: 0.3245 s/iter data_time: 0.2441 s/iter total_throughput: 3155.34 samples/s lr: 9.09e-04 [09/26 11:07:55] lb.utils.events INFO: eta: 21:48:42 iteration: 23099/375342 consumed_samples: 23654400 total_loss: 4.615 time: 0.3245 s/iter data_time: 0.2061 s/iter total_throughput: 3155.18 samples/s lr: 9.13e-04 [09/26 11:08:28] lb.utils.events INFO: eta: 17:40:59 iteration: 23199/375342 consumed_samples: 23756800 total_loss: 4.604 time: 0.3246 s/iter data_time: 0.1964 s/iter total_throughput: 3155.02 samples/s lr: 9.17e-04 [09/26 11:09:01] lb.utils.events INFO: eta: 16:49:21 iteration: 23299/375342 consumed_samples: 23859200 total_loss: 4.6 time: 0.3246 s/iter data_time: 0.1983 s/iter total_throughput: 3154.86 samples/s lr: 9.21e-04 [09/26 11:09:34] lb.utils.events INFO: eta: 16:51:55 iteration: 23399/375342 consumed_samples: 23961600 total_loss: 4.629 time: 0.3246 s/iter data_time: 0.2161 s/iter total_throughput: 3154.61 samples/s lr: 9.25e-04 [09/26 11:10:07] lb.utils.events INFO: eta: 16:25:39 iteration: 23499/375342 consumed_samples: 24064000 total_loss: 4.609 time: 0.3246 s/iter data_time: 0.2052 s/iter total_throughput: 3154.47 samples/s lr: 9.29e-04 [09/26 11:10:39] lb.utils.events INFO: eta: 16:07:35 iteration: 23599/375342 consumed_samples: 24166400 total_loss: 4.598 time: 0.3246 s/iter data_time: 0.2023 s/iter total_throughput: 3154.42 samples/s lr: 9.33e-04 [09/26 11:11:11] lb.utils.events INFO: eta: 15:32:07 iteration: 23699/375342 consumed_samples: 24268800 total_loss: 4.594 time: 0.3246 s/iter data_time: 0.1958 s/iter total_throughput: 3154.46 samples/s lr: 9.37e-04 [09/26 11:11:44] lb.utils.events INFO: eta: 15:23:46 iteration: 23799/375342 consumed_samples: 24371200 total_loss: 4.586 time: 0.3246 s/iter data_time: 0.2448 s/iter total_throughput: 3154.33 samples/s lr: 9.41e-04 [09/26 11:12:17] lb.utils.events INFO: eta: 15:18:55 iteration: 23899/375342 consumed_samples: 24473600 total_loss: 4.59 time: 0.3247 s/iter data_time: 0.2115 s/iter total_throughput: 3154.12 samples/s lr: 9.45e-04 [09/26 11:12:50] lb.utils.events INFO: eta: 15:15:35 iteration: 23999/375342 consumed_samples: 24576000 total_loss: 4.572 time: 0.3247 s/iter data_time: 0.2268 s/iter total_throughput: 3154.01 samples/s lr: 9.49e-04 [09/26 11:13:23] lb.utils.events INFO: eta: 15:16:41 iteration: 24099/375342 consumed_samples: 24678400 total_loss: 4.557 time: 0.3247 s/iter data_time: 0.2196 s/iter total_throughput: 3153.96 samples/s lr: 9.53e-04 [09/26 11:13:55] lb.utils.events INFO: eta: 15:20:34 iteration: 24199/375342 consumed_samples: 24780800 total_loss: 4.574 time: 0.3247 s/iter data_time: 0.1966 s/iter total_throughput: 3153.83 samples/s lr: 9.57e-04 [09/26 11:14:28] lb.utils.events INFO: eta: 15:27:58 iteration: 24299/375342 consumed_samples: 24883200 total_loss: 4.594 time: 0.3247 s/iter data_time: 0.2108 s/iter total_throughput: 3153.79 samples/s lr: 9.61e-04 [09/26 11:15:01] lb.utils.events INFO: eta: 15:32:19 iteration: 24399/375342 consumed_samples: 24985600 total_loss: 4.598 time: 0.3247 s/iter data_time: 0.2164 s/iter total_throughput: 3153.67 samples/s lr: 9.65e-04 [09/26 11:15:33] lb.utils.events INFO: eta: 15:36:19 iteration: 24499/375342 consumed_samples: 25088000 total_loss: 4.559 time: 0.3247 s/iter data_time: 0.2169 s/iter total_throughput: 3153.58 samples/s lr: 9.68e-04 [09/26 11:16:06] lb.utils.events INFO: eta: 15:44:23 iteration: 24599/375342 consumed_samples: 25190400 total_loss: 4.553 time: 0.3247 s/iter data_time: 0.2193 s/iter total_throughput: 3153.39 samples/s lr: 9.72e-04 [09/26 11:16:39] lb.utils.events INFO: eta: 15:49:20 iteration: 24699/375342 consumed_samples: 25292800 total_loss: 4.586 time: 0.3247 s/iter data_time: 0.2175 s/iter total_throughput: 3153.20 samples/s lr: 9.76e-04 [09/26 11:17:12] lb.utils.events INFO: eta: 15:37:24 iteration: 24799/375342 consumed_samples: 25395200 total_loss: 4.59 time: 0.3248 s/iter data_time: 0.2088 s/iter total_throughput: 3153.14 samples/s lr: 9.80e-04 [09/26 11:17:44] lb.utils.events INFO: eta: 15:31:09 iteration: 24899/375342 consumed_samples: 25497600 total_loss: 4.578 time: 0.3248 s/iter data_time: 0.2024 s/iter total_throughput: 3153.15 samples/s lr: 9.84e-04 [09/26 11:18:17] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0024999 [09/26 11:18:18] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/26 11:18:18] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/26 11:18:22] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0893 s/iter. Inference: 0.1525 s/iter. Eval: 0.0018 s/iter. Total: 0.2435 s/iter. ETA=0:00:09 [09/26 11:18:27] lb.evaluation.evaluator INFO: Inference done 28672/50000. Dataloading: 0.1276 s/iter. Inference: 0.1514 s/iter. Eval: 0.0020 s/iter. Total: 0.2811 s/iter. ETA=0:00:05 [09/26 11:18:32] lb.evaluation.evaluator INFO: Inference done 46080/50000. Dataloading: 0.1452 s/iter. Inference: 0.1505 s/iter. Eval: 0.0020 s/iter. Total: 0.2978 s/iter. ETA=0:00:00 [09/26 11:18:33] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/26 11:18:33] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.790710 (0.000256 s / iter per device, on 8 devices) [09/26 11:18:33] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000133 s / iter per device, on 8 devices) [09/26 11:18:33] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/26 11:18:33] lb.evaluation.utils INFO: copypaste: Acc@1=56.814 [09/26 11:18:33] lb.evaluation.utils INFO: copypaste: Acc@5=80.804 [09/26 11:18:33] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 56.81400, better than last best score 53.26000 @ iteration 19999. [09/26 11:18:33] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/26 11:18:34] lb.utils.events INFO: eta: 15:19:50 iteration: 24999/375342 consumed_samples: 25600000 total_loss: 4.578 time: 0.3248 s/iter data_time: 0.1879 s/iter total_throughput: 3153.13 samples/s lr: 9.88e-04 [09/26 11:19:05] lb.utils.events INFO: eta: 15:13:18 iteration: 25099/375342 consumed_samples: 25702400 total_loss: 4.566 time: 0.3247 s/iter data_time: 0.2036 s/iter total_throughput: 3153.69 samples/s lr: 9.89e-04 [09/26 11:19:38] lb.utils.events INFO: eta: 15:11:18 iteration: 25199/375342 consumed_samples: 25804800 total_loss: 4.541 time: 0.3247 s/iter data_time: 0.2172 s/iter total_throughput: 3153.57 samples/s lr: 9.89e-04 [09/26 11:20:10] lb.utils.events INFO: eta: 15:10:48 iteration: 25299/375342 consumed_samples: 25907200 total_loss: 4.545 time: 0.3247 s/iter data_time: 0.2138 s/iter total_throughput: 3153.54 samples/s lr: 9.89e-04 [09/26 11:20:43] lb.utils.events INFO: eta: 15:11:53 iteration: 25399/375342 consumed_samples: 26009600 total_loss: 4.555 time: 0.3247 s/iter data_time: 0.2276 s/iter total_throughput: 3153.27 samples/s lr: 9.89e-04 [09/26 11:21:16] lb.utils.events INFO: eta: 15:17:14 iteration: 25499/375342 consumed_samples: 26112000 total_loss: 4.543 time: 0.3247 s/iter data_time: 0.2269 s/iter total_throughput: 3153.28 samples/s lr: 9.89e-04 [09/26 11:21:49] lb.utils.events INFO: eta: 15:10:01 iteration: 25599/375342 consumed_samples: 26214400 total_loss: 4.547 time: 0.3247 s/iter data_time: 0.2174 s/iter total_throughput: 3153.20 samples/s lr: 9.89e-04 [09/26 11:22:22] lb.utils.events INFO: eta: 15:08:22 iteration: 25699/375342 consumed_samples: 26316800 total_loss: 4.551 time: 0.3248 s/iter data_time: 0.2100 s/iter total_throughput: 3152.89 samples/s lr: 9.89e-04 [09/26 11:22:55] lb.utils.events INFO: eta: 15:06:02 iteration: 25799/375342 consumed_samples: 26419200 total_loss: 4.541 time: 0.3248 s/iter data_time: 0.2060 s/iter total_throughput: 3152.72 samples/s lr: 9.89e-04 [09/26 11:23:28] lb.utils.events INFO: eta: 15:09:09 iteration: 25899/375342 consumed_samples: 26521600 total_loss: 4.527 time: 0.3248 s/iter data_time: 0.2360 s/iter total_throughput: 3152.44 samples/s lr: 9.88e-04 [09/26 11:24:01] lb.utils.events INFO: eta: 15:14:33 iteration: 25999/375342 consumed_samples: 26624000 total_loss: 4.518 time: 0.3248 s/iter data_time: 0.1971 s/iter total_throughput: 3152.36 samples/s lr: 9.88e-04 [09/26 11:24:33] lb.utils.events INFO: eta: 15:10:09 iteration: 26099/375342 consumed_samples: 26726400 total_loss: 4.51 time: 0.3248 s/iter data_time: 0.2044 s/iter total_throughput: 3152.46 samples/s lr: 9.88e-04 [09/26 11:25:05] lb.utils.events INFO: eta: 15:10:40 iteration: 26199/375342 consumed_samples: 26828800 total_loss: 4.502 time: 0.3248 s/iter data_time: 0.2226 s/iter total_throughput: 3152.53 samples/s lr: 9.88e-04 [09/26 11:25:37] lb.utils.events INFO: eta: 15:09:22 iteration: 26299/375342 consumed_samples: 26931200 total_loss: 4.504 time: 0.3248 s/iter data_time: 0.2259 s/iter total_throughput: 3152.70 samples/s lr: 9.88e-04 [09/26 11:26:10] lb.utils.events INFO: eta: 15:03:46 iteration: 26399/375342 consumed_samples: 27033600 total_loss: 4.504 time: 0.3248 s/iter data_time: 0.2244 s/iter total_throughput: 3152.56 samples/s lr: 9.88e-04 [09/26 11:26:43] lb.utils.events INFO: eta: 15:02:17 iteration: 26499/375342 consumed_samples: 27136000 total_loss: 4.498 time: 0.3248 s/iter data_time: 0.2207 s/iter total_throughput: 3152.49 samples/s lr: 9.88e-04 [09/26 11:27:15] lb.utils.events INFO: eta: 15:08:37 iteration: 26599/375342 consumed_samples: 27238400 total_loss: 4.521 time: 0.3248 s/iter data_time: 0.2255 s/iter total_throughput: 3152.61 samples/s lr: 9.88e-04 [09/26 11:27:47] lb.utils.events INFO: eta: 15:15:06 iteration: 26699/375342 consumed_samples: 27340800 total_loss: 4.494 time: 0.3248 s/iter data_time: 0.2197 s/iter total_throughput: 3152.72 samples/s lr: 9.88e-04 [09/26 11:28:20] lb.utils.events INFO: eta: 15:36:56 iteration: 26799/375342 consumed_samples: 27443200 total_loss: 4.461 time: 0.3248 s/iter data_time: 0.2414 s/iter total_throughput: 3152.51 samples/s lr: 9.88e-04 [09/26 11:28:53] lb.utils.events INFO: eta: 15:54:01 iteration: 26899/375342 consumed_samples: 27545600 total_loss: 4.48 time: 0.3248 s/iter data_time: 0.2363 s/iter total_throughput: 3152.55 samples/s lr: 9.88e-04 [09/26 11:29:25] lb.utils.events INFO: eta: 16:29:09 iteration: 26999/375342 consumed_samples: 27648000 total_loss: 4.496 time: 0.3248 s/iter data_time: 0.2202 s/iter total_throughput: 3152.48 samples/s lr: 9.87e-04 [09/26 11:29:58] lb.utils.events INFO: eta: 16:39:22 iteration: 27099/375342 consumed_samples: 27750400 total_loss: 4.473 time: 0.3248 s/iter data_time: 0.2121 s/iter total_throughput: 3152.44 samples/s lr: 9.87e-04 [09/26 11:30:30] lb.utils.events INFO: eta: 16:13:43 iteration: 27199/375342 consumed_samples: 27852800 total_loss: 4.484 time: 0.3248 s/iter data_time: 0.2184 s/iter total_throughput: 3152.42 samples/s lr: 9.87e-04 [09/26 11:31:04] lb.utils.events INFO: eta: 16:20:02 iteration: 27299/375342 consumed_samples: 27955200 total_loss: 4.484 time: 0.3249 s/iter data_time: 0.2084 s/iter total_throughput: 3152.18 samples/s lr: 9.87e-04 [09/26 11:31:36] lb.utils.events INFO: eta: 16:19:32 iteration: 27399/375342 consumed_samples: 28057600 total_loss: 4.469 time: 0.3248 s/iter data_time: 0.2264 s/iter total_throughput: 3152.40 samples/s lr: 9.87e-04 [09/26 11:32:08] lb.utils.events INFO: eta: 15:57:38 iteration: 27499/375342 consumed_samples: 28160000 total_loss: 4.482 time: 0.3248 s/iter data_time: 0.2133 s/iter total_throughput: 3152.29 samples/s lr: 9.87e-04 [09/26 11:32:41] lb.utils.events INFO: eta: 15:29:36 iteration: 27599/375342 consumed_samples: 28262400 total_loss: 4.457 time: 0.3249 s/iter data_time: 0.2176 s/iter total_throughput: 3152.22 samples/s lr: 9.87e-04 [09/26 11:33:13] lb.utils.events INFO: eta: 15:23:21 iteration: 27699/375342 consumed_samples: 28364800 total_loss: 4.436 time: 0.3248 s/iter data_time: 0.1909 s/iter total_throughput: 3152.37 samples/s lr: 9.87e-04 [09/26 11:33:45] lb.utils.events INFO: eta: 15:11:30 iteration: 27799/375342 consumed_samples: 28467200 total_loss: 4.445 time: 0.3248 s/iter data_time: 0.2082 s/iter total_throughput: 3152.57 samples/s lr: 9.87e-04 [09/26 11:34:17] lb.utils.events INFO: eta: 15:06:42 iteration: 27899/375342 consumed_samples: 28569600 total_loss: 4.451 time: 0.3248 s/iter data_time: 0.2200 s/iter total_throughput: 3152.72 samples/s lr: 9.87e-04 [09/26 11:34:50] lb.utils.events INFO: eta: 15:00:44 iteration: 27999/375342 consumed_samples: 28672000 total_loss: 4.453 time: 0.3248 s/iter data_time: 0.2246 s/iter total_throughput: 3152.69 samples/s lr: 9.86e-04 [09/26 11:35:23] lb.utils.events INFO: eta: 15:04:31 iteration: 28099/375342 consumed_samples: 28774400 total_loss: 4.451 time: 0.3248 s/iter data_time: 0.2240 s/iter total_throughput: 3152.40 samples/s lr: 9.86e-04 [09/26 11:35:56] lb.utils.events INFO: eta: 15:08:23 iteration: 28199/375342 consumed_samples: 28876800 total_loss: 4.438 time: 0.3248 s/iter data_time: 0.2260 s/iter total_throughput: 3152.27 samples/s lr: 9.86e-04 [09/26 11:36:29] lb.utils.events INFO: eta: 15:05:58 iteration: 28299/375342 consumed_samples: 28979200 total_loss: 4.432 time: 0.3249 s/iter data_time: 0.2073 s/iter total_throughput: 3152.00 samples/s lr: 9.86e-04 [09/26 11:37:02] lb.utils.events INFO: eta: 15:03:45 iteration: 28399/375342 consumed_samples: 29081600 total_loss: 4.439 time: 0.3249 s/iter data_time: 0.2090 s/iter total_throughput: 3151.78 samples/s lr: 9.86e-04 [09/26 11:37:35] lb.utils.events INFO: eta: 15:03:19 iteration: 28499/375342 consumed_samples: 29184000 total_loss: 4.438 time: 0.3249 s/iter data_time: 0.2050 s/iter total_throughput: 3151.62 samples/s lr: 9.86e-04 [09/26 11:38:08] lb.utils.events INFO: eta: 15:10:39 iteration: 28599/375342 consumed_samples: 29286400 total_loss: 4.416 time: 0.3249 s/iter data_time: 0.2407 s/iter total_throughput: 3151.53 samples/s lr: 9.86e-04 [09/26 11:38:42] lb.utils.events INFO: eta: 15:25:32 iteration: 28699/375342 consumed_samples: 29388800 total_loss: 4.416 time: 0.3250 s/iter data_time: 0.2444 s/iter total_throughput: 3151.13 samples/s lr: 9.86e-04 [09/26 11:39:15] lb.utils.events INFO: eta: 15:38:43 iteration: 28799/375342 consumed_samples: 29491200 total_loss: 4.414 time: 0.3250 s/iter data_time: 0.2342 s/iter total_throughput: 3151.01 samples/s lr: 9.86e-04 [09/26 11:39:48] lb.utils.events INFO: eta: 15:39:26 iteration: 28899/375342 consumed_samples: 29593600 total_loss: 4.414 time: 0.3250 s/iter data_time: 0.2172 s/iter total_throughput: 3150.82 samples/s lr: 9.86e-04 [09/26 11:40:20] lb.utils.events INFO: eta: 15:40:24 iteration: 28999/375342 consumed_samples: 29696000 total_loss: 4.422 time: 0.3250 s/iter data_time: 0.2409 s/iter total_throughput: 3150.72 samples/s lr: 9.85e-04 [09/26 11:40:54] lb.utils.events INFO: eta: 15:42:47 iteration: 29099/375342 consumed_samples: 29798400 total_loss: 4.428 time: 0.3250 s/iter data_time: 0.2381 s/iter total_throughput: 3150.32 samples/s lr: 9.85e-04 [09/26 11:41:27] lb.utils.events INFO: eta: 15:39:51 iteration: 29199/375342 consumed_samples: 29900800 total_loss: 4.418 time: 0.3251 s/iter data_time: 0.2185 s/iter total_throughput: 3150.09 samples/s lr: 9.85e-04 [09/26 11:42:01] lb.utils.events INFO: eta: 15:57:49 iteration: 29299/375342 consumed_samples: 30003200 total_loss: 4.418 time: 0.3251 s/iter data_time: 0.2298 s/iter total_throughput: 3149.88 samples/s lr: 9.85e-04 [09/26 11:42:34] lb.utils.events INFO: eta: 17:07:50 iteration: 29399/375342 consumed_samples: 30105600 total_loss: 4.412 time: 0.3251 s/iter data_time: 0.2293 s/iter total_throughput: 3149.71 samples/s lr: 9.85e-04 [09/26 11:43:06] lb.utils.events INFO: eta: 20:52:23 iteration: 29499/375342 consumed_samples: 30208000 total_loss: 4.408 time: 0.3251 s/iter data_time: 0.2153 s/iter total_throughput: 3149.60 samples/s lr: 9.85e-04 [09/26 11:43:40] lb.utils.events INFO: eta: 17:11:41 iteration: 29599/375342 consumed_samples: 30310400 total_loss: 4.42 time: 0.3252 s/iter data_time: 0.2197 s/iter total_throughput: 3149.21 samples/s lr: 9.85e-04 [09/26 11:44:14] lb.utils.events INFO: eta: 15:42:38 iteration: 29699/375342 consumed_samples: 30412800 total_loss: 4.414 time: 0.3252 s/iter data_time: 0.2175 s/iter total_throughput: 3148.86 samples/s lr: 9.85e-04 [09/26 11:44:46] lb.utils.events INFO: eta: 15:24:23 iteration: 29799/375342 consumed_samples: 30515200 total_loss: 4.398 time: 0.3252 s/iter data_time: 0.2211 s/iter total_throughput: 3148.79 samples/s lr: 9.85e-04 [09/26 11:45:19] lb.utils.events INFO: eta: 15:24:49 iteration: 29899/375342 consumed_samples: 30617600 total_loss: 4.371 time: 0.3252 s/iter data_time: 0.2349 s/iter total_throughput: 3148.63 samples/s lr: 9.85e-04 [09/26 11:45:53] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0029999 [09/26 11:45:54] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/26 11:45:54] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/26 11:45:58] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0808 s/iter. Inference: 0.1493 s/iter. Eval: 0.0020 s/iter. Total: 0.2322 s/iter. ETA=0:00:08 [09/26 11:46:04] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1426 s/iter. Inference: 0.1504 s/iter. Eval: 0.0020 s/iter. Total: 0.2952 s/iter. ETA=0:00:05 [09/26 11:46:09] lb.evaluation.evaluator INFO: Inference done 50000/50000. Dataloading: 0.1314 s/iter. Inference: 0.1503 s/iter. Eval: 0.0020 s/iter. Total: 0.2838 s/iter. ETA=0:00:00 [09/26 11:46:09] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/26 11:46:09] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.490744 (0.000250 s / iter per device, on 8 devices) [09/26 11:46:09] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000132 s / iter per device, on 8 devices) [09/26 11:46:09] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/26 11:46:09] lb.evaluation.utils INFO: copypaste: Acc@1=60.526 [09/26 11:46:09] lb.evaluation.utils INFO: copypaste: Acc@5=83.748 [09/26 11:46:09] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 60.52600, better than last best score 56.81400 @ iteration 24999. [09/26 11:46:09] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/26 11:46:10] lb.utils.events INFO: eta: 15:20:22 iteration: 29999/375342 consumed_samples: 30720000 total_loss: 4.375 time: 0.3253 s/iter data_time: 0.2238 s/iter total_throughput: 3148.19 samples/s lr: 9.84e-04 [09/26 11:46:41] lb.utils.events INFO: eta: 15:22:24 iteration: 30099/375342 consumed_samples: 30822400 total_loss: 4.398 time: 0.3252 s/iter data_time: 0.2588 s/iter total_throughput: 3148.51 samples/s lr: 9.84e-04 [09/26 11:47:15] lb.utils.events INFO: eta: 15:34:06 iteration: 30199/375342 consumed_samples: 30924800 total_loss: 4.395 time: 0.3253 s/iter data_time: 0.2325 s/iter total_throughput: 3148.30 samples/s lr: 9.84e-04 [09/26 11:47:47] lb.utils.events INFO: eta: 15:33:25 iteration: 30299/375342 consumed_samples: 31027200 total_loss: 4.383 time: 0.3253 s/iter data_time: 0.2240 s/iter total_throughput: 3148.24 samples/s lr: 9.84e-04 [09/26 11:48:20] lb.utils.events INFO: eta: 15:31:50 iteration: 30399/375342 consumed_samples: 31129600 total_loss: 4.367 time: 0.3253 s/iter data_time: 0.2287 s/iter total_throughput: 3148.10 samples/s lr: 9.84e-04 [09/26 11:48:54] lb.utils.events INFO: eta: 15:32:12 iteration: 30499/375342 consumed_samples: 31232000 total_loss: 4.373 time: 0.3253 s/iter data_time: 0.2105 s/iter total_throughput: 3147.78 samples/s lr: 9.84e-04 [09/26 11:49:27] lb.utils.events INFO: eta: 15:37:50 iteration: 30599/375342 consumed_samples: 31334400 total_loss: 4.363 time: 0.3253 s/iter data_time: 0.2131 s/iter total_throughput: 3147.71 samples/s lr: 9.84e-04 [09/26 11:49:59] lb.utils.events INFO: eta: 15:42:43 iteration: 30699/375342 consumed_samples: 31436800 total_loss: 4.354 time: 0.3253 s/iter data_time: 0.2035 s/iter total_throughput: 3147.60 samples/s lr: 9.84e-04 [09/26 11:50:33] lb.utils.events INFO: eta: 15:45:47 iteration: 30799/375342 consumed_samples: 31539200 total_loss: 4.377 time: 0.3254 s/iter data_time: 0.2186 s/iter total_throughput: 3147.35 samples/s lr: 9.84e-04 [09/26 11:51:06] lb.utils.events INFO: eta: 15:31:07 iteration: 30899/375342 consumed_samples: 31641600 total_loss: 4.395 time: 0.3254 s/iter data_time: 0.2207 s/iter total_throughput: 3147.17 samples/s lr: 9.84e-04 [09/26 11:51:39] lb.utils.events INFO: eta: 15:37:00 iteration: 30999/375342 consumed_samples: 31744000 total_loss: 4.385 time: 0.3254 s/iter data_time: 0.2184 s/iter total_throughput: 3147.11 samples/s lr: 9.83e-04 [09/26 11:52:12] lb.utils.events INFO: eta: 15:36:44 iteration: 31099/375342 consumed_samples: 31846400 total_loss: 4.398 time: 0.3254 s/iter data_time: 0.2110 s/iter total_throughput: 3146.99 samples/s lr: 9.83e-04 [09/26 11:52:45] lb.utils.events INFO: eta: 15:22:56 iteration: 31199/375342 consumed_samples: 31948800 total_loss: 4.373 time: 0.3254 s/iter data_time: 0.2300 s/iter total_throughput: 3146.83 samples/s lr: 9.83e-04 [09/26 11:53:18] lb.utils.events INFO: eta: 15:21:22 iteration: 31299/375342 consumed_samples: 32051200 total_loss: 4.35 time: 0.3254 s/iter data_time: 0.2456 s/iter total_throughput: 3146.59 samples/s lr: 9.83e-04 [09/26 11:53:51] lb.utils.events INFO: eta: 15:18:23 iteration: 31399/375342 consumed_samples: 32153600 total_loss: 4.348 time: 0.3255 s/iter data_time: 0.2355 s/iter total_throughput: 3146.39 samples/s lr: 9.83e-04 [09/26 11:54:24] lb.utils.events INFO: eta: 15:18:07 iteration: 31499/375342 consumed_samples: 32256000 total_loss: 4.34 time: 0.3255 s/iter data_time: 0.2294 s/iter total_throughput: 3146.21 samples/s lr: 9.83e-04 [09/26 11:54:57] lb.utils.events INFO: eta: 15:20:24 iteration: 31599/375342 consumed_samples: 32358400 total_loss: 4.348 time: 0.3255 s/iter data_time: 0.2193 s/iter total_throughput: 3146.10 samples/s lr: 9.83e-04 [09/26 11:55:30] lb.utils.events INFO: eta: 15:34:04 iteration: 31699/375342 consumed_samples: 32460800 total_loss: 4.361 time: 0.3255 s/iter data_time: 0.2398 s/iter total_throughput: 3145.89 samples/s lr: 9.83e-04 [09/26 11:56:04] lb.utils.events INFO: eta: 15:43:32 iteration: 31799/375342 consumed_samples: 32563200 total_loss: 4.359 time: 0.3255 s/iter data_time: 0.2505 s/iter total_throughput: 3145.71 samples/s lr: 9.83e-04 [09/26 11:56:37] lb.utils.events INFO: eta: 15:45:50 iteration: 31899/375342 consumed_samples: 32665600 total_loss: 4.35 time: 0.3256 s/iter data_time: 0.2094 s/iter total_throughput: 3145.41 samples/s lr: 9.82e-04 [09/26 11:57:10] lb.utils.events INFO: eta: 15:25:21 iteration: 31999/375342 consumed_samples: 32768000 total_loss: 4.363 time: 0.3256 s/iter data_time: 0.2031 s/iter total_throughput: 3145.26 samples/s lr: 9.82e-04 [09/26 11:57:43] lb.utils.events INFO: eta: 15:14:18 iteration: 32099/375342 consumed_samples: 32870400 total_loss: 4.352 time: 0.3256 s/iter data_time: 0.2028 s/iter total_throughput: 3145.15 samples/s lr: 9.82e-04 [09/26 11:58:16] lb.utils.events INFO: eta: 15:12:02 iteration: 32199/375342 consumed_samples: 32972800 total_loss: 4.344 time: 0.3256 s/iter data_time: 0.2186 s/iter total_throughput: 3145.14 samples/s lr: 9.82e-04 [09/26 11:58:48] lb.utils.events INFO: eta: 15:15:43 iteration: 32299/375342 consumed_samples: 33075200 total_loss: 4.336 time: 0.3256 s/iter data_time: 0.2128 s/iter total_throughput: 3145.22 samples/s lr: 9.82e-04 [09/26 11:59:21] lb.utils.events INFO: eta: 15:02:58 iteration: 32399/375342 consumed_samples: 33177600 total_loss: 4.33 time: 0.3256 s/iter data_time: 0.2133 s/iter total_throughput: 3145.04 samples/s lr: 9.82e-04 [09/26 11:59:55] lb.utils.events INFO: eta: 15:00:14 iteration: 32499/375342 consumed_samples: 33280000 total_loss: 4.332 time: 0.3256 s/iter data_time: 0.2040 s/iter total_throughput: 3144.81 samples/s lr: 9.82e-04 [09/26 12:00:27] lb.utils.events INFO: eta: 14:57:50 iteration: 32599/375342 consumed_samples: 33382400 total_loss: 4.312 time: 0.3256 s/iter data_time: 0.2002 s/iter total_throughput: 3144.92 samples/s lr: 9.82e-04 [09/26 12:01:00] lb.utils.events INFO: eta: 14:59:54 iteration: 32699/375342 consumed_samples: 33484800 total_loss: 4.295 time: 0.3256 s/iter data_time: 0.2596 s/iter total_throughput: 3144.66 samples/s lr: 9.82e-04 [09/26 12:01:33] lb.utils.events INFO: eta: 15:03:14 iteration: 32799/375342 consumed_samples: 33587200 total_loss: 4.332 time: 0.3256 s/iter data_time: 0.2519 s/iter total_throughput: 3144.55 samples/s lr: 9.81e-04 [09/26 12:02:06] lb.utils.events INFO: eta: 15:16:14 iteration: 32899/375342 consumed_samples: 33689600 total_loss: 4.34 time: 0.3257 s/iter data_time: 0.2414 s/iter total_throughput: 3144.44 samples/s lr: 9.81e-04 [09/26 12:02:39] lb.utils.events INFO: eta: 15:30:29 iteration: 32999/375342 consumed_samples: 33792000 total_loss: 4.309 time: 0.3257 s/iter data_time: 0.2254 s/iter total_throughput: 3144.28 samples/s lr: 9.81e-04 [09/26 12:03:12] lb.utils.events INFO: eta: 15:32:31 iteration: 33099/375342 consumed_samples: 33894400 total_loss: 4.32 time: 0.3257 s/iter data_time: 0.2258 s/iter total_throughput: 3144.18 samples/s lr: 9.81e-04 [09/26 12:03:45] lb.utils.events INFO: eta: 15:37:28 iteration: 33199/375342 consumed_samples: 33996800 total_loss: 4.322 time: 0.3257 s/iter data_time: 0.2489 s/iter total_throughput: 3144.03 samples/s lr: 9.81e-04 [09/26 12:04:19] lb.utils.events INFO: eta: 15:31:19 iteration: 33299/375342 consumed_samples: 34099200 total_loss: 4.314 time: 0.3257 s/iter data_time: 0.2155 s/iter total_throughput: 3143.71 samples/s lr: 9.81e-04 [09/26 12:04:52] lb.utils.events INFO: eta: 15:38:25 iteration: 33399/375342 consumed_samples: 34201600 total_loss: 4.312 time: 0.3257 s/iter data_time: 0.2099 s/iter total_throughput: 3143.61 samples/s lr: 9.81e-04 [09/26 12:05:25] lb.utils.events INFO: eta: 15:47:19 iteration: 33499/375342 consumed_samples: 34304000 total_loss: 4.324 time: 0.3258 s/iter data_time: 0.2251 s/iter total_throughput: 3143.46 samples/s lr: 9.81e-04 [09/26 12:05:58] lb.utils.events INFO: eta: 15:58:05 iteration: 33599/375342 consumed_samples: 34406400 total_loss: 4.309 time: 0.3258 s/iter data_time: 0.2248 s/iter total_throughput: 3143.35 samples/s lr: 9.81e-04 [09/26 12:06:31] lb.utils.events INFO: eta: 16:01:50 iteration: 33699/375342 consumed_samples: 34508800 total_loss: 4.309 time: 0.3258 s/iter data_time: 0.2494 s/iter total_throughput: 3143.26 samples/s lr: 9.80e-04 [09/26 12:07:04] lb.utils.events INFO: eta: 15:57:31 iteration: 33799/375342 consumed_samples: 34611200 total_loss: 4.291 time: 0.3258 s/iter data_time: 0.2355 s/iter total_throughput: 3143.20 samples/s lr: 9.80e-04 [09/26 12:07:36] lb.utils.events INFO: eta: 15:46:10 iteration: 33899/375342 consumed_samples: 34713600 total_loss: 4.273 time: 0.3258 s/iter data_time: 0.1977 s/iter total_throughput: 3143.18 samples/s lr: 9.80e-04 [09/26 12:08:10] lb.utils.events INFO: eta: 15:47:11 iteration: 33999/375342 consumed_samples: 34816000 total_loss: 4.289 time: 0.3258 s/iter data_time: 0.2466 s/iter total_throughput: 3142.93 samples/s lr: 9.80e-04 [09/26 12:08:43] lb.utils.events INFO: eta: 15:49:59 iteration: 34099/375342 consumed_samples: 34918400 total_loss: 4.297 time: 0.3258 s/iter data_time: 0.2088 s/iter total_throughput: 3142.78 samples/s lr: 9.80e-04 [09/26 12:09:16] lb.utils.events INFO: eta: 15:36:17 iteration: 34199/375342 consumed_samples: 35020800 total_loss: 4.307 time: 0.3258 s/iter data_time: 0.1967 s/iter total_throughput: 3142.75 samples/s lr: 9.80e-04 [09/26 12:09:49] lb.utils.events INFO: eta: 15:27:21 iteration: 34299/375342 consumed_samples: 35123200 total_loss: 4.271 time: 0.3258 s/iter data_time: 0.2162 s/iter total_throughput: 3142.57 samples/s lr: 9.80e-04 [09/26 12:10:22] lb.utils.events INFO: eta: 15:16:54 iteration: 34399/375342 consumed_samples: 35225600 total_loss: 4.262 time: 0.3259 s/iter data_time: 0.2078 s/iter total_throughput: 3142.38 samples/s lr: 9.80e-04 [09/26 12:10:55] lb.utils.events INFO: eta: 15:14:28 iteration: 34499/375342 consumed_samples: 35328000 total_loss: 4.273 time: 0.3259 s/iter data_time: 0.2333 s/iter total_throughput: 3142.22 samples/s lr: 9.80e-04 [09/26 12:11:28] lb.utils.events INFO: eta: 15:13:26 iteration: 34599/375342 consumed_samples: 35430400 total_loss: 4.273 time: 0.3259 s/iter data_time: 0.2260 s/iter total_throughput: 3142.09 samples/s lr: 9.79e-04 [09/26 12:12:01] lb.utils.events INFO: eta: 15:09:56 iteration: 34699/375342 consumed_samples: 35532800 total_loss: 4.266 time: 0.3259 s/iter data_time: 0.2175 s/iter total_throughput: 3142.07 samples/s lr: 9.79e-04 [09/26 12:12:34] lb.utils.events INFO: eta: 15:05:49 iteration: 34799/375342 consumed_samples: 35635200 total_loss: 4.268 time: 0.3259 s/iter data_time: 0.2155 s/iter total_throughput: 3142.06 samples/s lr: 9.79e-04 [09/26 12:13:07] lb.utils.events INFO: eta: 15:06:54 iteration: 34899/375342 consumed_samples: 35737600 total_loss: 4.273 time: 0.3259 s/iter data_time: 0.2364 s/iter total_throughput: 3141.96 samples/s lr: 9.79e-04 [09/26 12:13:40] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0034999 [09/26 12:13:40] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/26 12:13:40] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/26 12:13:44] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0817 s/iter. Inference: 0.1492 s/iter. Eval: 0.0022 s/iter. Total: 0.2331 s/iter. ETA=0:00:08 [09/26 12:13:50] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1426 s/iter. Inference: 0.1488 s/iter. Eval: 0.0021 s/iter. Total: 0.2935 s/iter. ETA=0:00:05 [09/26 12:13:55] lb.evaluation.evaluator INFO: Inference done 50000/50000. Dataloading: 0.1309 s/iter. Inference: 0.1501 s/iter. Eval: 0.0020 s/iter. Total: 0.2832 s/iter. ETA=0:00:00 [09/26 12:13:55] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/26 12:13:55] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.465005 (0.000249 s / iter per device, on 8 devices) [09/26 12:13:55] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000132 s / iter per device, on 8 devices) [09/26 12:13:55] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/26 12:13:55] lb.evaluation.utils INFO: copypaste: Acc@1=63.205999999999996 [09/26 12:13:55] lb.evaluation.utils INFO: copypaste: Acc@5=85.384 [09/26 12:13:55] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 63.20600, better than last best score 60.52600 @ iteration 29999. [09/26 12:13:55] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/26 12:13:56] lb.utils.events INFO: eta: 14:58:33 iteration: 34999/375342 consumed_samples: 35840000 total_loss: 4.281 time: 0.3259 s/iter data_time: 0.2100 s/iter total_throughput: 3141.88 samples/s lr: 9.79e-04 [09/26 12:14:27] lb.utils.events INFO: eta: 14:54:09 iteration: 35099/375342 consumed_samples: 35942400 total_loss: 4.295 time: 0.3259 s/iter data_time: 0.2226 s/iter total_throughput: 3142.27 samples/s lr: 9.79e-04 [09/26 12:15:01] lb.utils.events INFO: eta: 14:56:19 iteration: 35199/375342 consumed_samples: 36044800 total_loss: 4.266 time: 0.3259 s/iter data_time: 0.2099 s/iter total_throughput: 3141.98 samples/s lr: 9.79e-04 [09/26 12:15:34] lb.utils.events INFO: eta: 14:57:01 iteration: 35299/375342 consumed_samples: 36147200 total_loss: 4.242 time: 0.3259 s/iter data_time: 0.2105 s/iter total_throughput: 3141.97 samples/s lr: 9.79e-04 [09/26 12:16:07] lb.utils.events INFO: eta: 14:55:47 iteration: 35399/375342 consumed_samples: 36249600 total_loss: 4.244 time: 0.3259 s/iter data_time: 0.2088 s/iter total_throughput: 3141.78 samples/s lr: 9.78e-04 [09/26 12:16:40] lb.utils.events INFO: eta: 14:55:05 iteration: 35499/375342 consumed_samples: 36352000 total_loss: 4.252 time: 0.3259 s/iter data_time: 0.2218 s/iter total_throughput: 3141.64 samples/s lr: 9.78e-04 [09/26 12:17:13] lb.utils.events INFO: eta: 14:54:50 iteration: 35599/375342 consumed_samples: 36454400 total_loss: 4.273 time: 0.3260 s/iter data_time: 0.2356 s/iter total_throughput: 3141.52 samples/s lr: 9.78e-04 [09/26 12:17:46] lb.utils.events INFO: eta: 14:54:02 iteration: 35699/375342 consumed_samples: 36556800 total_loss: 4.277 time: 0.3260 s/iter data_time: 0.1955 s/iter total_throughput: 3141.34 samples/s lr: 9.78e-04 [09/26 12:18:19] lb.utils.events INFO: eta: 14:49:51 iteration: 35799/375342 consumed_samples: 36659200 total_loss: 4.258 time: 0.3260 s/iter data_time: 0.2110 s/iter total_throughput: 3141.26 samples/s lr: 9.78e-04 [09/26 12:18:52] lb.utils.events INFO: eta: 14:49:35 iteration: 35899/375342 consumed_samples: 36761600 total_loss: 4.244 time: 0.3260 s/iter data_time: 0.2240 s/iter total_throughput: 3141.20 samples/s lr: 9.78e-04 [09/26 12:19:25] lb.utils.events INFO: eta: 14:51:41 iteration: 35999/375342 consumed_samples: 36864000 total_loss: 4.268 time: 0.3260 s/iter data_time: 0.2275 s/iter total_throughput: 3141.22 samples/s lr: 9.78e-04 [09/26 12:19:58] lb.utils.events INFO: eta: 14:53:28 iteration: 36099/375342 consumed_samples: 36966400 total_loss: 4.271 time: 0.3260 s/iter data_time: 0.2373 s/iter total_throughput: 3141.10 samples/s lr: 9.78e-04 [09/26 12:20:31] lb.utils.events INFO: eta: 14:52:02 iteration: 36199/375342 consumed_samples: 37068800 total_loss: 4.26 time: 0.3260 s/iter data_time: 0.2119 s/iter total_throughput: 3140.88 samples/s lr: 9.77e-04 [09/26 12:21:04] lb.utils.events INFO: eta: 14:51:41 iteration: 36299/375342 consumed_samples: 37171200 total_loss: 4.258 time: 0.3260 s/iter data_time: 0.2041 s/iter total_throughput: 3140.82 samples/s lr: 9.77e-04 [09/26 12:21:37] lb.utils.events INFO: eta: 14:48:08 iteration: 36399/375342 consumed_samples: 37273600 total_loss: 4.266 time: 0.3260 s/iter data_time: 0.2061 s/iter total_throughput: 3140.78 samples/s lr: 9.77e-04 [09/26 12:22:09] lb.utils.events INFO: eta: 14:44:11 iteration: 36499/375342 consumed_samples: 37376000 total_loss: 4.262 time: 0.3260 s/iter data_time: 0.2040 s/iter total_throughput: 3140.75 samples/s lr: 9.77e-04 [09/26 12:22:42] lb.utils.events INFO: eta: 14:41:45 iteration: 36599/375342 consumed_samples: 37478400 total_loss: 4.232 time: 0.3260 s/iter data_time: 0.2017 s/iter total_throughput: 3140.70 samples/s lr: 9.77e-04 [09/26 12:23:15] lb.utils.events INFO: eta: 14:36:33 iteration: 36699/375342 consumed_samples: 37580800 total_loss: 4.238 time: 0.3260 s/iter data_time: 0.1975 s/iter total_throughput: 3140.72 samples/s lr: 9.77e-04 [09/26 12:23:48] lb.utils.events INFO: eta: 14:35:28 iteration: 36799/375342 consumed_samples: 37683200 total_loss: 4.234 time: 0.3261 s/iter data_time: 0.2083 s/iter total_throughput: 3140.52 samples/s lr: 9.77e-04 [09/26 12:24:21] lb.utils.events INFO: eta: 14:29:53 iteration: 36899/375342 consumed_samples: 37785600 total_loss: 4.23 time: 0.3261 s/iter data_time: 0.2029 s/iter total_throughput: 3140.37 samples/s lr: 9.77e-04 [09/26 12:24:54] lb.utils.events INFO: eta: 14:29:46 iteration: 36999/375342 consumed_samples: 37888000 total_loss: 4.229 time: 0.3261 s/iter data_time: 0.2276 s/iter total_throughput: 3140.28 samples/s lr: 9.76e-04 [09/26 12:25:27] lb.utils.events INFO: eta: 14:27:39 iteration: 37099/375342 consumed_samples: 37990400 total_loss: 4.219 time: 0.3261 s/iter data_time: 0.2240 s/iter total_throughput: 3140.25 samples/s lr: 9.76e-04 [09/26 12:26:00] lb.utils.events INFO: eta: 14:33:34 iteration: 37199/375342 consumed_samples: 38092800 total_loss: 4.211 time: 0.3261 s/iter data_time: 0.2386 s/iter total_throughput: 3140.15 samples/s lr: 9.76e-04 [09/26 12:26:33] lb.utils.events INFO: eta: 14:37:22 iteration: 37299/375342 consumed_samples: 38195200 total_loss: 4.229 time: 0.3261 s/iter data_time: 0.2281 s/iter total_throughput: 3139.93 samples/s lr: 9.76e-04 [09/26 12:27:06] lb.utils.events INFO: eta: 14:44:47 iteration: 37399/375342 consumed_samples: 38297600 total_loss: 4.254 time: 0.3261 s/iter data_time: 0.2162 s/iter total_throughput: 3139.85 samples/s lr: 9.76e-04 [09/26 12:27:39] lb.utils.events INFO: eta: 14:51:45 iteration: 37499/375342 consumed_samples: 38400000 total_loss: 4.25 time: 0.3261 s/iter data_time: 0.2189 s/iter total_throughput: 3139.87 samples/s lr: 9.76e-04 [09/26 12:28:12] lb.utils.events INFO: eta: 14:59:28 iteration: 37599/375342 consumed_samples: 38502400 total_loss: 4.25 time: 0.3261 s/iter data_time: 0.2173 s/iter total_throughput: 3139.83 samples/s lr: 9.76e-04 [09/26 12:28:44] lb.utils.events INFO: eta: 15:14:04 iteration: 37699/375342 consumed_samples: 38604800 total_loss: 4.227 time: 0.3261 s/iter data_time: 0.2116 s/iter total_throughput: 3139.84 samples/s lr: 9.76e-04 [09/26 12:29:17] lb.utils.events INFO: eta: 15:17:03 iteration: 37799/375342 consumed_samples: 38707200 total_loss: 4.207 time: 0.3261 s/iter data_time: 0.2142 s/iter total_throughput: 3139.72 samples/s lr: 9.75e-04 [09/26 12:29:50] lb.utils.events INFO: eta: 15:16:53 iteration: 37899/375342 consumed_samples: 38809600 total_loss: 4.211 time: 0.3262 s/iter data_time: 0.2051 s/iter total_throughput: 3139.59 samples/s lr: 9.75e-04 [09/26 12:30:23] lb.utils.events INFO: eta: 15:10:17 iteration: 37999/375342 consumed_samples: 38912000 total_loss: 4.211 time: 0.3262 s/iter data_time: 0.2085 s/iter total_throughput: 3139.50 samples/s lr: 9.75e-04 [09/26 12:30:56] lb.utils.events INFO: eta: 15:04:32 iteration: 38099/375342 consumed_samples: 39014400 total_loss: 4.217 time: 0.3262 s/iter data_time: 0.2217 s/iter total_throughput: 3139.40 samples/s lr: 9.75e-04 [09/26 12:31:29] lb.utils.events INFO: eta: 14:57:50 iteration: 38199/375342 consumed_samples: 39116800 total_loss: 4.23 time: 0.3262 s/iter data_time: 0.2238 s/iter total_throughput: 3139.41 samples/s lr: 9.75e-04 [09/26 12:32:02] lb.utils.events INFO: eta: 14:54:08 iteration: 38299/375342 consumed_samples: 39219200 total_loss: 4.232 time: 0.3262 s/iter data_time: 0.2038 s/iter total_throughput: 3139.39 samples/s lr: 9.75e-04 [09/26 12:32:34] lb.utils.events INFO: eta: 14:53:08 iteration: 38399/375342 consumed_samples: 39321600 total_loss: 4.242 time: 0.3262 s/iter data_time: 0.2250 s/iter total_throughput: 3139.42 samples/s lr: 9.75e-04 [09/26 12:33:07] lb.utils.events INFO: eta: 14:50:49 iteration: 38499/375342 consumed_samples: 39424000 total_loss: 4.236 time: 0.3262 s/iter data_time: 0.2177 s/iter total_throughput: 3139.43 samples/s lr: 9.75e-04 [09/26 12:33:39] lb.utils.events INFO: eta: 14:50:12 iteration: 38599/375342 consumed_samples: 39526400 total_loss: 4.227 time: 0.3262 s/iter data_time: 0.2254 s/iter total_throughput: 3139.46 samples/s lr: 9.74e-04 [09/26 12:34:12] lb.utils.events INFO: eta: 14:50:59 iteration: 38699/375342 consumed_samples: 39628800 total_loss: 4.229 time: 0.3262 s/iter data_time: 0.2216 s/iter total_throughput: 3139.45 samples/s lr: 9.74e-04 [09/26 12:34:45] lb.utils.events INFO: eta: 14:56:06 iteration: 38799/375342 consumed_samples: 39731200 total_loss: 4.207 time: 0.3262 s/iter data_time: 0.2161 s/iter total_throughput: 3139.43 samples/s lr: 9.74e-04 [09/26 12:35:18] lb.utils.events INFO: eta: 15:03:45 iteration: 38899/375342 consumed_samples: 39833600 total_loss: 4.193 time: 0.3262 s/iter data_time: 0.2239 s/iter total_throughput: 3139.38 samples/s lr: 9.74e-04 [09/26 12:35:51] lb.utils.events INFO: eta: 15:11:14 iteration: 38999/375342 consumed_samples: 39936000 total_loss: 4.199 time: 0.3262 s/iter data_time: 0.2111 s/iter total_throughput: 3139.29 samples/s lr: 9.74e-04 [09/26 12:36:23] lb.utils.events INFO: eta: 15:12:17 iteration: 39099/375342 consumed_samples: 40038400 total_loss: 4.199 time: 0.3262 s/iter data_time: 0.2128 s/iter total_throughput: 3139.36 samples/s lr: 9.74e-04 [09/26 12:36:56] lb.utils.events INFO: eta: 15:15:27 iteration: 39199/375342 consumed_samples: 40140800 total_loss: 4.188 time: 0.3262 s/iter data_time: 0.2270 s/iter total_throughput: 3139.34 samples/s lr: 9.74e-04 [09/26 12:37:28] lb.utils.events INFO: eta: 16:10:21 iteration: 39299/375342 consumed_samples: 40243200 total_loss: 4.195 time: 0.3262 s/iter data_time: 0.2352 s/iter total_throughput: 3139.39 samples/s lr: 9.73e-04 [09/26 12:38:01] lb.utils.events INFO: eta: 18:39:54 iteration: 39399/375342 consumed_samples: 40345600 total_loss: 4.219 time: 0.3262 s/iter data_time: 0.2422 s/iter total_throughput: 3139.41 samples/s lr: 9.73e-04 [09/26 12:38:33] lb.utils.events INFO: eta: 20:58:46 iteration: 39499/375342 consumed_samples: 40448000 total_loss: 4.229 time: 0.3262 s/iter data_time: 0.2287 s/iter total_throughput: 3139.38 samples/s lr: 9.73e-04 [09/26 12:39:06] lb.utils.events INFO: eta: 21:06:58 iteration: 39599/375342 consumed_samples: 40550400 total_loss: 4.197 time: 0.3262 s/iter data_time: 0.2055 s/iter total_throughput: 3139.39 samples/s lr: 9.73e-04 [09/26 12:39:39] lb.utils.events INFO: eta: 18:58:51 iteration: 39699/375342 consumed_samples: 40652800 total_loss: 4.186 time: 0.3262 s/iter data_time: 0.2141 s/iter total_throughput: 3139.37 samples/s lr: 9.73e-04 [09/26 12:40:12] lb.utils.events INFO: eta: 17:18:51 iteration: 39799/375342 consumed_samples: 40755200 total_loss: 4.199 time: 0.3262 s/iter data_time: 0.2102 s/iter total_throughput: 3139.15 samples/s lr: 9.73e-04 [09/26 12:40:46] lb.utils.events INFO: eta: 15:42:28 iteration: 39899/375342 consumed_samples: 40857600 total_loss: 4.205 time: 0.3262 s/iter data_time: 0.2153 s/iter total_throughput: 3138.96 samples/s lr: 9.73e-04 [09/26 12:41:18] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0039999 [09/26 12:41:19] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/26 12:41:19] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/26 12:41:23] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0816 s/iter. Inference: 0.1497 s/iter. Eval: 0.0021 s/iter. Total: 0.2333 s/iter. ETA=0:00:08 [09/26 12:41:29] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1450 s/iter. Inference: 0.1483 s/iter. Eval: 0.0020 s/iter. Total: 0.2953 s/iter. ETA=0:00:05 [09/26 12:41:34] lb.evaluation.evaluator INFO: Inference done 49152/50000. Dataloading: 0.1310 s/iter. Inference: 0.1482 s/iter. Eval: 0.0020 s/iter. Total: 0.2812 s/iter. ETA=0:00:00 [09/26 12:41:34] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/26 12:41:34] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.528752 (0.000251 s / iter per device, on 8 devices) [09/26 12:41:34] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000131 s / iter per device, on 8 devices) [09/26 12:41:34] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/26 12:41:34] lb.evaluation.utils INFO: copypaste: Acc@1=64.86 [09/26 12:41:34] lb.evaluation.utils INFO: copypaste: Acc@5=86.5 [09/26 12:41:34] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 64.86000, better than last best score 63.20600 @ iteration 34999. [09/26 12:41:34] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/26 12:41:35] lb.utils.events INFO: eta: 15:23:56 iteration: 39999/375342 consumed_samples: 40960000 total_loss: 4.203 time: 0.3262 s/iter data_time: 0.2117 s/iter total_throughput: 3138.93 samples/s lr: 9.73e-04 [09/26 12:42:06] lb.utils.events INFO: eta: 15:06:37 iteration: 40099/375342 consumed_samples: 41062400 total_loss: 4.176 time: 0.3262 s/iter data_time: 0.2125 s/iter total_throughput: 3139.27 samples/s lr: 9.72e-04 [09/26 12:42:39] lb.utils.events INFO: eta: 15:02:51 iteration: 40199/375342 consumed_samples: 41164800 total_loss: 4.16 time: 0.3262 s/iter data_time: 0.2355 s/iter total_throughput: 3139.19 samples/s lr: 9.72e-04 [09/26 12:43:12] lb.utils.events INFO: eta: 14:56:08 iteration: 40299/375342 consumed_samples: 41267200 total_loss: 4.188 time: 0.3262 s/iter data_time: 0.2411 s/iter total_throughput: 3139.22 samples/s lr: 9.72e-04 [09/26 12:43:44] lb.utils.events INFO: eta: 14:55:15 iteration: 40399/375342 consumed_samples: 41369600 total_loss: 4.193 time: 0.3262 s/iter data_time: 0.2426 s/iter total_throughput: 3139.18 samples/s lr: 9.72e-04 [09/26 12:44:17] lb.utils.events INFO: eta: 14:51:51 iteration: 40499/375342 consumed_samples: 41472000 total_loss: 4.158 time: 0.3262 s/iter data_time: 0.2410 s/iter total_throughput: 3139.16 samples/s lr: 9.72e-04 [09/26 12:44:49] lb.utils.events INFO: eta: 14:58:07 iteration: 40599/375342 consumed_samples: 41574400 total_loss: 4.182 time: 0.3262 s/iter data_time: 0.2327 s/iter total_throughput: 3139.22 samples/s lr: 9.72e-04 [09/26 12:45:22] lb.utils.events INFO: eta: 15:08:58 iteration: 40699/375342 consumed_samples: 41676800 total_loss: 4.178 time: 0.3262 s/iter data_time: 0.2421 s/iter total_throughput: 3139.30 samples/s lr: 9.72e-04 [09/26 12:45:55] lb.utils.events INFO: eta: 15:07:12 iteration: 40799/375342 consumed_samples: 41779200 total_loss: 4.188 time: 0.3262 s/iter data_time: 0.2239 s/iter total_throughput: 3139.23 samples/s lr: 9.71e-04 [09/26 12:46:27] lb.utils.events INFO: eta: 15:12:13 iteration: 40899/375342 consumed_samples: 41881600 total_loss: 4.182 time: 0.3262 s/iter data_time: 0.1963 s/iter total_throughput: 3139.36 samples/s lr: 9.71e-04 [09/26 12:46:59] lb.utils.events INFO: eta: 15:16:56 iteration: 40999/375342 consumed_samples: 41984000 total_loss: 4.176 time: 0.3262 s/iter data_time: 0.2094 s/iter total_throughput: 3139.41 samples/s lr: 9.71e-04 [09/26 12:47:32] lb.utils.events INFO: eta: 15:44:51 iteration: 41099/375342 consumed_samples: 42086400 total_loss: 4.199 time: 0.3262 s/iter data_time: 0.2196 s/iter total_throughput: 3139.45 samples/s lr: 9.71e-04 [09/26 12:48:04] lb.utils.events INFO: eta: 16:00:28 iteration: 41199/375342 consumed_samples: 42188800 total_loss: 4.211 time: 0.3262 s/iter data_time: 0.2425 s/iter total_throughput: 3139.47 samples/s lr: 9.71e-04 [09/26 12:48:37] lb.utils.events INFO: eta: 16:15:27 iteration: 41299/375342 consumed_samples: 42291200 total_loss: 4.18 time: 0.3262 s/iter data_time: 0.2234 s/iter total_throughput: 3139.43 samples/s lr: 9.71e-04 [09/26 12:49:09] lb.utils.events INFO: eta: 15:35:16 iteration: 41399/375342 consumed_samples: 42393600 total_loss: 4.15 time: 0.3262 s/iter data_time: 0.2211 s/iter total_throughput: 3139.46 samples/s lr: 9.71e-04 [09/26 12:49:42] lb.utils.events INFO: eta: 15:06:31 iteration: 41499/375342 consumed_samples: 42496000 total_loss: 4.148 time: 0.3262 s/iter data_time: 0.2012 s/iter total_throughput: 3139.45 samples/s lr: 9.70e-04 [09/26 12:50:15] lb.utils.events INFO: eta: 14:49:59 iteration: 41599/375342 consumed_samples: 42598400 total_loss: 4.178 time: 0.3262 s/iter data_time: 0.2025 s/iter total_throughput: 3139.45 samples/s lr: 9.70e-04 [09/26 12:50:47] lb.utils.events INFO: eta: 14:36:40 iteration: 41699/375342 consumed_samples: 42700800 total_loss: 4.18 time: 0.3262 s/iter data_time: 0.1984 s/iter total_throughput: 3139.54 samples/s lr: 9.70e-04 [09/26 12:51:20] lb.utils.events INFO: eta: 14:32:54 iteration: 41799/375342 consumed_samples: 42803200 total_loss: 4.158 time: 0.3262 s/iter data_time: 0.2079 s/iter total_throughput: 3139.54 samples/s lr: 9.70e-04 [09/26 12:51:52] lb.utils.events INFO: eta: 14:29:39 iteration: 41899/375342 consumed_samples: 42905600 total_loss: 4.162 time: 0.3261 s/iter data_time: 0.2062 s/iter total_throughput: 3139.68 samples/s lr: 9.70e-04 [09/26 12:52:24] lb.utils.events INFO: eta: 14:33:21 iteration: 41999/375342 consumed_samples: 43008000 total_loss: 4.166 time: 0.3261 s/iter data_time: 0.2312 s/iter total_throughput: 3139.77 samples/s lr: 9.70e-04 [09/26 12:52:57] lb.utils.events INFO: eta: 14:38:16 iteration: 42099/375342 consumed_samples: 43110400 total_loss: 4.164 time: 0.3261 s/iter data_time: 0.2377 s/iter total_throughput: 3139.66 samples/s lr: 9.70e-04 [09/26 12:53:31] lb.utils.events INFO: eta: 14:31:58 iteration: 42199/375342 consumed_samples: 43212800 total_loss: 4.184 time: 0.3262 s/iter data_time: 0.2160 s/iter total_throughput: 3139.35 samples/s lr: 9.69e-04 [09/26 12:54:04] lb.utils.events INFO: eta: 14:22:51 iteration: 42299/375342 consumed_samples: 43315200 total_loss: 4.17 time: 0.3262 s/iter data_time: 0.2092 s/iter total_throughput: 3139.26 samples/s lr: 9.69e-04 [09/26 12:54:37] lb.utils.events INFO: eta: 14:20:03 iteration: 42399/375342 consumed_samples: 43417600 total_loss: 4.166 time: 0.3262 s/iter data_time: 0.2076 s/iter total_throughput: 3139.16 samples/s lr: 9.69e-04 [09/26 12:55:10] lb.utils.events INFO: eta: 14:21:23 iteration: 42499/375342 consumed_samples: 43520000 total_loss: 4.162 time: 0.3262 s/iter data_time: 0.2372 s/iter total_throughput: 3139.03 samples/s lr: 9.69e-04 [09/26 12:55:43] lb.utils.events INFO: eta: 14:25:05 iteration: 42599/375342 consumed_samples: 43622400 total_loss: 4.146 time: 0.3262 s/iter data_time: 0.2134 s/iter total_throughput: 3138.91 samples/s lr: 9.69e-04 [09/26 12:56:17] lb.utils.events INFO: eta: 14:30:07 iteration: 42699/375342 consumed_samples: 43724800 total_loss: 4.135 time: 0.3263 s/iter data_time: 0.2276 s/iter total_throughput: 3138.66 samples/s lr: 9.69e-04 [09/26 12:56:50] lb.utils.events INFO: eta: 14:37:42 iteration: 42799/375342 consumed_samples: 43827200 total_loss: 4.129 time: 0.3263 s/iter data_time: 0.2371 s/iter total_throughput: 3138.55 samples/s lr: 9.69e-04 [09/26 12:57:23] lb.utils.events INFO: eta: 14:52:32 iteration: 42899/375342 consumed_samples: 43929600 total_loss: 4.156 time: 0.3263 s/iter data_time: 0.2139 s/iter total_throughput: 3138.53 samples/s lr: 9.68e-04 [09/26 12:57:57] lb.utils.events INFO: eta: 14:44:33 iteration: 42999/375342 consumed_samples: 44032000 total_loss: 4.16 time: 0.3263 s/iter data_time: 0.2054 s/iter total_throughput: 3138.27 samples/s lr: 9.68e-04 [09/26 12:58:30] lb.utils.events INFO: eta: 14:35:16 iteration: 43099/375342 consumed_samples: 44134400 total_loss: 4.143 time: 0.3263 s/iter data_time: 0.2120 s/iter total_throughput: 3138.21 samples/s lr: 9.68e-04 [09/26 12:59:03] lb.utils.events INFO: eta: 14:31:47 iteration: 43199/375342 consumed_samples: 44236800 total_loss: 4.145 time: 0.3263 s/iter data_time: 0.2145 s/iter total_throughput: 3138.03 samples/s lr: 9.68e-04 [09/26 12:59:37] lb.utils.events INFO: eta: 14:35:41 iteration: 43299/375342 consumed_samples: 44339200 total_loss: 4.146 time: 0.3263 s/iter data_time: 0.2342 s/iter total_throughput: 3137.77 samples/s lr: 9.68e-04 [09/26 13:00:10] lb.utils.events INFO: eta: 14:36:08 iteration: 43399/375342 consumed_samples: 44441600 total_loss: 4.141 time: 0.3263 s/iter data_time: 0.2179 s/iter total_throughput: 3137.78 samples/s lr: 9.68e-04 [09/26 13:00:43] lb.utils.events INFO: eta: 14:32:01 iteration: 43499/375342 consumed_samples: 44544000 total_loss: 4.145 time: 0.3264 s/iter data_time: 0.2186 s/iter total_throughput: 3137.61 samples/s lr: 9.68e-04 [09/26 13:01:16] lb.utils.events INFO: eta: 14:28:05 iteration: 43599/375342 consumed_samples: 44646400 total_loss: 4.143 time: 0.3264 s/iter data_time: 0.2119 s/iter total_throughput: 3137.43 samples/s lr: 9.67e-04 [09/26 13:01:50] lb.utils.events INFO: eta: 14:23:18 iteration: 43699/375342 consumed_samples: 44748800 total_loss: 4.121 time: 0.3264 s/iter data_time: 0.2083 s/iter total_throughput: 3137.24 samples/s lr: 9.67e-04 [09/26 13:02:23] lb.utils.events INFO: eta: 14:16:49 iteration: 43799/375342 consumed_samples: 44851200 total_loss: 4.129 time: 0.3264 s/iter data_time: 0.2109 s/iter total_throughput: 3137.18 samples/s lr: 9.67e-04 [09/26 13:02:56] lb.utils.events INFO: eta: 14:10:55 iteration: 43899/375342 consumed_samples: 44953600 total_loss: 4.125 time: 0.3264 s/iter data_time: 0.2016 s/iter total_throughput: 3137.10 samples/s lr: 9.67e-04 [09/26 13:03:29] lb.utils.events INFO: eta: 14:10:55 iteration: 43999/375342 consumed_samples: 45056000 total_loss: 4.131 time: 0.3264 s/iter data_time: 0.1982 s/iter total_throughput: 3137.01 samples/s lr: 9.67e-04 [09/26 13:04:02] lb.utils.events INFO: eta: 14:09:54 iteration: 44099/375342 consumed_samples: 45158400 total_loss: 4.121 time: 0.3264 s/iter data_time: 0.2104 s/iter total_throughput: 3136.91 samples/s lr: 9.67e-04 [09/26 13:04:35] lb.utils.events INFO: eta: 14:09:29 iteration: 44199/375342 consumed_samples: 45260800 total_loss: 4.117 time: 0.3264 s/iter data_time: 0.2055 s/iter total_throughput: 3136.84 samples/s lr: 9.67e-04 [09/26 13:05:08] lb.utils.events INFO: eta: 14:08:48 iteration: 44299/375342 consumed_samples: 45363200 total_loss: 4.139 time: 0.3264 s/iter data_time: 0.2099 s/iter total_throughput: 3136.81 samples/s lr: 9.66e-04 [09/26 13:05:41] lb.utils.events INFO: eta: 14:09:00 iteration: 44399/375342 consumed_samples: 45465600 total_loss: 4.146 time: 0.3265 s/iter data_time: 0.2277 s/iter total_throughput: 3136.74 samples/s lr: 9.66e-04 [09/26 13:06:14] lb.utils.events INFO: eta: 14:11:53 iteration: 44499/375342 consumed_samples: 45568000 total_loss: 4.098 time: 0.3265 s/iter data_time: 0.2194 s/iter total_throughput: 3136.69 samples/s lr: 9.66e-04 [09/26 13:06:46] lb.utils.events INFO: eta: 14:18:19 iteration: 44599/375342 consumed_samples: 45670400 total_loss: 4.113 time: 0.3265 s/iter data_time: 0.2479 s/iter total_throughput: 3136.66 samples/s lr: 9.66e-04 [09/26 13:07:20] lb.utils.events INFO: eta: 14:25:46 iteration: 44699/375342 consumed_samples: 45772800 total_loss: 4.117 time: 0.3265 s/iter data_time: 0.2252 s/iter total_throughput: 3136.55 samples/s lr: 9.66e-04 [09/26 13:07:52] lb.utils.events INFO: eta: 14:27:30 iteration: 44799/375342 consumed_samples: 45875200 total_loss: 4.121 time: 0.3265 s/iter data_time: 0.2068 s/iter total_throughput: 3136.51 samples/s lr: 9.66e-04 [09/26 13:08:26] lb.utils.events INFO: eta: 14:35:26 iteration: 44899/375342 consumed_samples: 45977600 total_loss: 4.141 time: 0.3265 s/iter data_time: 0.2434 s/iter total_throughput: 3136.40 samples/s lr: 9.65e-04 [09/26 13:08:59] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0044999 [09/26 13:09:00] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/26 13:09:00] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/26 13:09:04] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0897 s/iter. Inference: 0.1491 s/iter. Eval: 0.0019 s/iter. Total: 0.2406 s/iter. ETA=0:00:08 [09/26 13:09:10] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1520 s/iter. Inference: 0.1501 s/iter. Eval: 0.0019 s/iter. Total: 0.3041 s/iter. ETA=0:00:05 [09/26 13:09:15] lb.evaluation.evaluator INFO: Inference done 49152/50000. Dataloading: 0.1359 s/iter. Inference: 0.1483 s/iter. Eval: 0.0020 s/iter. Total: 0.2863 s/iter. ETA=0:00:00 [09/26 13:09:15] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/26 13:09:15] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.742106 (0.000255 s / iter per device, on 8 devices) [09/26 13:09:15] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000132 s / iter per device, on 8 devices) [09/26 13:09:15] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/26 13:09:15] lb.evaluation.utils INFO: copypaste: Acc@1=65.86999999999999 [09/26 13:09:15] lb.evaluation.utils INFO: copypaste: Acc@5=87.22999999999999 [09/26 13:09:15] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 65.87000, better than last best score 64.86000 @ iteration 39999. [09/26 13:09:15] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/26 13:09:16] lb.utils.events INFO: eta: 14:38:31 iteration: 44999/375342 consumed_samples: 46080000 total_loss: 4.141 time: 0.3265 s/iter data_time: 0.2246 s/iter total_throughput: 3136.27 samples/s lr: 9.65e-04 [09/26 13:09:47] lb.utils.events INFO: eta: 14:44:11 iteration: 45099/375342 consumed_samples: 46182400 total_loss: 4.107 time: 0.3265 s/iter data_time: 0.2607 s/iter total_throughput: 3136.51 samples/s lr: 9.65e-04 [09/26 13:10:21] lb.utils.events INFO: eta: 15:11:50 iteration: 45199/375342 consumed_samples: 46284800 total_loss: 4.102 time: 0.3265 s/iter data_time: 0.2310 s/iter total_throughput: 3136.38 samples/s lr: 9.65e-04 [09/26 13:10:55] lb.utils.events INFO: eta: 15:39:02 iteration: 45299/375342 consumed_samples: 46387200 total_loss: 4.107 time: 0.3265 s/iter data_time: 0.2379 s/iter total_throughput: 3136.09 samples/s lr: 9.65e-04 [09/26 13:11:28] lb.utils.events INFO: eta: 15:26:07 iteration: 45399/375342 consumed_samples: 46489600 total_loss: 4.113 time: 0.3265 s/iter data_time: 0.2089 s/iter total_throughput: 3135.90 samples/s lr: 9.65e-04 [09/26 13:12:02] lb.utils.events INFO: eta: 15:05:50 iteration: 45499/375342 consumed_samples: 46592000 total_loss: 4.119 time: 0.3266 s/iter data_time: 0.2082 s/iter total_throughput: 3135.71 samples/s lr: 9.65e-04 [09/26 13:12:35] lb.utils.events INFO: eta: 14:39:13 iteration: 45599/375342 consumed_samples: 46694400 total_loss: 4.113 time: 0.3266 s/iter data_time: 0.2108 s/iter total_throughput: 3135.61 samples/s lr: 9.64e-04 [09/26 13:13:08] lb.utils.events INFO: eta: 14:29:53 iteration: 45699/375342 consumed_samples: 46796800 total_loss: 4.102 time: 0.3266 s/iter data_time: 0.1996 s/iter total_throughput: 3135.52 samples/s lr: 9.64e-04 [09/26 13:13:41] lb.utils.events INFO: eta: 14:26:38 iteration: 45799/375342 consumed_samples: 46899200 total_loss: 4.105 time: 0.3266 s/iter data_time: 0.2045 s/iter total_throughput: 3135.53 samples/s lr: 9.64e-04 [09/26 13:14:14] lb.utils.events INFO: eta: 14:18:57 iteration: 45899/375342 consumed_samples: 47001600 total_loss: 4.125 time: 0.3266 s/iter data_time: 0.2041 s/iter total_throughput: 3135.44 samples/s lr: 9.64e-04 [09/26 13:14:47] lb.utils.events INFO: eta: 14:12:05 iteration: 45999/375342 consumed_samples: 47104000 total_loss: 4.133 time: 0.3266 s/iter data_time: 0.2152 s/iter total_throughput: 3135.32 samples/s lr: 9.64e-04 [09/26 13:15:20] lb.utils.events INFO: eta: 14:09:31 iteration: 46099/375342 consumed_samples: 47206400 total_loss: 4.121 time: 0.3266 s/iter data_time: 0.2188 s/iter total_throughput: 3135.14 samples/s lr: 9.64e-04 [09/26 13:15:54] lb.utils.events INFO: eta: 14:03:11 iteration: 46199/375342 consumed_samples: 47308800 total_loss: 4.113 time: 0.3266 s/iter data_time: 0.2071 s/iter total_throughput: 3134.99 samples/s lr: 9.63e-04 [09/26 13:16:27] lb.utils.events INFO: eta: 14:00:32 iteration: 46299/375342 consumed_samples: 47411200 total_loss: 4.107 time: 0.3266 s/iter data_time: 0.2017 s/iter total_throughput: 3134.86 samples/s lr: 9.63e-04 [09/26 13:17:01] lb.utils.events INFO: eta: 14:01:44 iteration: 46399/375342 consumed_samples: 47513600 total_loss: 4.09 time: 0.3267 s/iter data_time: 0.2028 s/iter total_throughput: 3134.72 samples/s lr: 9.63e-04 [09/26 13:17:34] lb.utils.events INFO: eta: 14:01:40 iteration: 46499/375342 consumed_samples: 47616000 total_loss: 4.09 time: 0.3267 s/iter data_time: 0.2065 s/iter total_throughput: 3134.59 samples/s lr: 9.63e-04 [09/26 13:18:07] lb.utils.events INFO: eta: 13:59:38 iteration: 46599/375342 consumed_samples: 47718400 total_loss: 4.102 time: 0.3267 s/iter data_time: 0.2175 s/iter total_throughput: 3134.42 samples/s lr: 9.63e-04 [09/26 13:18:41] lb.utils.events INFO: eta: 13:58:28 iteration: 46699/375342 consumed_samples: 47820800 total_loss: 4.096 time: 0.3267 s/iter data_time: 0.2174 s/iter total_throughput: 3134.27 samples/s lr: 9.63e-04 [09/26 13:19:14] lb.utils.events INFO: eta: 13:58:31 iteration: 46799/375342 consumed_samples: 47923200 total_loss: 4.111 time: 0.3267 s/iter data_time: 0.2052 s/iter total_throughput: 3134.24 samples/s lr: 9.63e-04 [09/26 13:19:46] lb.utils.events INFO: eta: 13:57:47 iteration: 46899/375342 consumed_samples: 48025600 total_loss: 4.105 time: 0.3267 s/iter data_time: 0.2150 s/iter total_throughput: 3134.19 samples/s lr: 9.62e-04 [09/26 13:20:20] lb.utils.events INFO: eta: 13:57:31 iteration: 46999/375342 consumed_samples: 48128000 total_loss: 4.105 time: 0.3267 s/iter data_time: 0.2139 s/iter total_throughput: 3133.98 samples/s lr: 9.62e-04 [09/26 13:20:53] lb.utils.events INFO: eta: 13:57:27 iteration: 47099/375342 consumed_samples: 48230400 total_loss: 4.098 time: 0.3268 s/iter data_time: 0.2148 s/iter total_throughput: 3133.87 samples/s lr: 9.62e-04 [09/26 13:21:26] lb.utils.events INFO: eta: 13:58:11 iteration: 47199/375342 consumed_samples: 48332800 total_loss: 4.08 time: 0.3267 s/iter data_time: 0.2044 s/iter total_throughput: 3133.90 samples/s lr: 9.62e-04 [09/26 13:21:59] lb.utils.events INFO: eta: 13:59:15 iteration: 47299/375342 consumed_samples: 48435200 total_loss: 4.086 time: 0.3268 s/iter data_time: 0.2097 s/iter total_throughput: 3133.76 samples/s lr: 9.62e-04 [09/26 13:22:32] lb.utils.events INFO: eta: 13:59:24 iteration: 47399/375342 consumed_samples: 48537600 total_loss: 4.111 time: 0.3268 s/iter data_time: 0.2046 s/iter total_throughput: 3133.66 samples/s lr: 9.62e-04 [09/26 13:23:06] lb.utils.events INFO: eta: 13:59:14 iteration: 47499/375342 consumed_samples: 48640000 total_loss: 4.121 time: 0.3268 s/iter data_time: 0.2100 s/iter total_throughput: 3133.57 samples/s lr: 9.61e-04 [09/26 13:23:39] lb.utils.events INFO: eta: 13:59:42 iteration: 47599/375342 consumed_samples: 48742400 total_loss: 4.078 time: 0.3268 s/iter data_time: 0.2095 s/iter total_throughput: 3133.51 samples/s lr: 9.61e-04 [09/26 13:24:11] lb.utils.events INFO: eta: 14:00:14 iteration: 47699/375342 consumed_samples: 48844800 total_loss: 4.084 time: 0.3268 s/iter data_time: 0.1991 s/iter total_throughput: 3133.55 samples/s lr: 9.61e-04 [09/26 13:24:44] lb.utils.events INFO: eta: 14:04:31 iteration: 47799/375342 consumed_samples: 48947200 total_loss: 4.098 time: 0.3268 s/iter data_time: 0.2540 s/iter total_throughput: 3133.51 samples/s lr: 9.61e-04 [09/26 13:25:17] lb.utils.events INFO: eta: 14:05:43 iteration: 47899/375342 consumed_samples: 49049600 total_loss: 4.09 time: 0.3268 s/iter data_time: 0.2211 s/iter total_throughput: 3133.47 samples/s lr: 9.61e-04 [09/26 13:25:50] lb.utils.events INFO: eta: 14:13:16 iteration: 47999/375342 consumed_samples: 49152000 total_loss: 4.076 time: 0.3268 s/iter data_time: 0.2583 s/iter total_throughput: 3133.42 samples/s lr: 9.61e-04 [09/26 13:26:23] lb.utils.events INFO: eta: 14:23:45 iteration: 48099/375342 consumed_samples: 49254400 total_loss: 4.055 time: 0.3268 s/iter data_time: 0.2194 s/iter total_throughput: 3133.39 samples/s lr: 9.60e-04 [09/26 13:26:56] lb.utils.events INFO: eta: 14:35:56 iteration: 48199/375342 consumed_samples: 49356800 total_loss: 4.07 time: 0.3268 s/iter data_time: 0.2464 s/iter total_throughput: 3133.23 samples/s lr: 9.60e-04 [09/26 13:27:29] lb.utils.events INFO: eta: 14:50:47 iteration: 48299/375342 consumed_samples: 49459200 total_loss: 4.1 time: 0.3268 s/iter data_time: 0.2319 s/iter total_throughput: 3133.12 samples/s lr: 9.60e-04 [09/26 13:28:02] lb.utils.events INFO: eta: 14:58:11 iteration: 48399/375342 consumed_samples: 49561600 total_loss: 4.088 time: 0.3268 s/iter data_time: 0.2210 s/iter total_throughput: 3133.08 samples/s lr: 9.60e-04 [09/26 13:28:35] lb.utils.events INFO: eta: 15:05:09 iteration: 48499/375342 consumed_samples: 49664000 total_loss: 4.086 time: 0.3268 s/iter data_time: 0.2129 s/iter total_throughput: 3133.04 samples/s lr: 9.60e-04 [09/26 13:29:08] lb.utils.events INFO: eta: 15:00:47 iteration: 48599/375342 consumed_samples: 49766400 total_loss: 4.102 time: 0.3268 s/iter data_time: 0.2100 s/iter total_throughput: 3132.99 samples/s lr: 9.60e-04 [09/26 13:29:42] lb.utils.events INFO: eta: 15:03:40 iteration: 48699/375342 consumed_samples: 49868800 total_loss: 4.096 time: 0.3269 s/iter data_time: 0.2325 s/iter total_throughput: 3132.84 samples/s lr: 9.59e-04 [09/26 13:30:14] lb.utils.events INFO: eta: 14:58:52 iteration: 48799/375342 consumed_samples: 49971200 total_loss: 4.102 time: 0.3269 s/iter data_time: 0.2180 s/iter total_throughput: 3132.83 samples/s lr: 9.59e-04 [09/26 13:30:47] lb.utils.events INFO: eta: 14:53:28 iteration: 48899/375342 consumed_samples: 50073600 total_loss: 4.092 time: 0.3269 s/iter data_time: 0.2306 s/iter total_throughput: 3132.77 samples/s lr: 9.59e-04 [09/26 13:31:21] lb.utils.events INFO: eta: 14:37:40 iteration: 48999/375342 consumed_samples: 50176000 total_loss: 4.082 time: 0.3269 s/iter data_time: 0.2084 s/iter total_throughput: 3132.66 samples/s lr: 9.59e-04 [09/26 13:31:54] lb.utils.events INFO: eta: 14:20:57 iteration: 49099/375342 consumed_samples: 50278400 total_loss: 4.086 time: 0.3269 s/iter data_time: 0.2155 s/iter total_throughput: 3132.60 samples/s lr: 9.59e-04 [09/26 13:32:27] lb.utils.events INFO: eta: 14:11:09 iteration: 49199/375342 consumed_samples: 50380800 total_loss: 4.102 time: 0.3269 s/iter data_time: 0.2066 s/iter total_throughput: 3132.48 samples/s lr: 9.59e-04 [09/26 13:33:00] lb.utils.events INFO: eta: 14:02:05 iteration: 49299/375342 consumed_samples: 50483200 total_loss: 4.113 time: 0.3269 s/iter data_time: 0.2045 s/iter total_throughput: 3132.46 samples/s lr: 9.58e-04 [09/26 13:33:33] lb.utils.events INFO: eta: 13:58:15 iteration: 49399/375342 consumed_samples: 50585600 total_loss: 4.08 time: 0.3269 s/iter data_time: 0.2091 s/iter total_throughput: 3132.32 samples/s lr: 9.58e-04 [09/26 13:34:06] lb.utils.events INFO: eta: 13:58:17 iteration: 49499/375342 consumed_samples: 50688000 total_loss: 4.07 time: 0.3269 s/iter data_time: 0.2573 s/iter total_throughput: 3132.33 samples/s lr: 9.58e-04 [09/26 13:34:39] lb.utils.events INFO: eta: 14:01:34 iteration: 49599/375342 consumed_samples: 50790400 total_loss: 4.098 time: 0.3269 s/iter data_time: 0.2210 s/iter total_throughput: 3132.22 samples/s lr: 9.58e-04 [09/26 13:35:12] lb.utils.events INFO: eta: 14:03:09 iteration: 49699/375342 consumed_samples: 50892800 total_loss: 4.098 time: 0.3269 s/iter data_time: 0.2302 s/iter total_throughput: 3132.13 samples/s lr: 9.58e-04 [09/26 13:35:45] lb.utils.events INFO: eta: 14:02:18 iteration: 49799/375342 consumed_samples: 50995200 total_loss: 4.098 time: 0.3269 s/iter data_time: 0.2246 s/iter total_throughput: 3132.12 samples/s lr: 9.58e-04 [09/26 13:36:18] lb.utils.events INFO: eta: 14:03:01 iteration: 49899/375342 consumed_samples: 51097600 total_loss: 4.102 time: 0.3269 s/iter data_time: 0.2247 s/iter total_throughput: 3132.04 samples/s lr: 9.57e-04 [09/26 13:36:51] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0049999 [09/26 13:36:52] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/26 13:36:52] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/26 13:36:56] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0820 s/iter. Inference: 0.1500 s/iter. Eval: 0.0019 s/iter. Total: 0.2339 s/iter. ETA=0:00:08 [09/26 13:37:02] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1461 s/iter. Inference: 0.1493 s/iter. Eval: 0.0020 s/iter. Total: 0.2975 s/iter. ETA=0:00:05 [09/26 13:37:07] lb.evaluation.evaluator INFO: Inference done 49152/50000. Dataloading: 0.1303 s/iter. Inference: 0.1500 s/iter. Eval: 0.0020 s/iter. Total: 0.2824 s/iter. ETA=0:00:00 [09/26 13:37:07] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/26 13:37:07] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.596142 (0.000252 s / iter per device, on 8 devices) [09/26 13:37:07] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000133 s / iter per device, on 8 devices) [09/26 13:37:07] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/26 13:37:07] lb.evaluation.utils INFO: copypaste: Acc@1=66.702 [09/26 13:37:07] lb.evaluation.utils INFO: copypaste: Acc@5=87.776 [09/26 13:37:07] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 66.70200, better than last best score 65.87000 @ iteration 44999. [09/26 13:37:07] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/26 13:37:08] lb.utils.events INFO: eta: 14:05:53 iteration: 49999/375342 consumed_samples: 51200000 total_loss: 4.082 time: 0.3270 s/iter data_time: 0.2193 s/iter total_throughput: 3131.97 samples/s lr: 9.57e-04 [09/26 13:37:39] lb.utils.events INFO: eta: 14:14:55 iteration: 50099/375342 consumed_samples: 51302400 total_loss: 4.041 time: 0.3269 s/iter data_time: 0.2237 s/iter total_throughput: 3132.16 samples/s lr: 9.57e-04 [09/26 13:38:13] lb.utils.events INFO: eta: 14:20:43 iteration: 50199/375342 consumed_samples: 51404800 total_loss: 4.055 time: 0.3269 s/iter data_time: 0.2089 s/iter total_throughput: 3132.04 samples/s lr: 9.57e-04 [09/26 13:38:47] lb.utils.events INFO: eta: 14:17:32 iteration: 50299/375342 consumed_samples: 51507200 total_loss: 4.08 time: 0.3270 s/iter data_time: 0.2072 s/iter total_throughput: 3131.82 samples/s lr: 9.57e-04 [09/26 13:39:20] lb.utils.events INFO: eta: 14:17:08 iteration: 50399/375342 consumed_samples: 51609600 total_loss: 4.084 time: 0.3270 s/iter data_time: 0.2129 s/iter total_throughput: 3131.71 samples/s lr: 9.57e-04 [09/26 13:39:53] lb.utils.events INFO: eta: 14:13:43 iteration: 50499/375342 consumed_samples: 51712000 total_loss: 4.066 time: 0.3270 s/iter data_time: 0.2055 s/iter total_throughput: 3131.67 samples/s lr: 9.56e-04 [09/26 13:40:26] lb.utils.events INFO: eta: 14:09:59 iteration: 50599/375342 consumed_samples: 51814400 total_loss: 4.074 time: 0.3270 s/iter data_time: 0.2121 s/iter total_throughput: 3131.61 samples/s lr: 9.56e-04 [09/26 13:40:58] lb.utils.events INFO: eta: 14:02:56 iteration: 50699/375342 consumed_samples: 51916800 total_loss: 4.074 time: 0.3270 s/iter data_time: 0.1996 s/iter total_throughput: 3131.62 samples/s lr: 9.56e-04 [09/26 13:41:32] lb.utils.events INFO: eta: 13:59:15 iteration: 50799/375342 consumed_samples: 52019200 total_loss: 4.074 time: 0.3270 s/iter data_time: 0.2088 s/iter total_throughput: 3131.52 samples/s lr: 9.56e-04 [09/26 13:42:04] lb.utils.events INFO: eta: 13:56:59 iteration: 50899/375342 consumed_samples: 52121600 total_loss: 4.062 time: 0.3270 s/iter data_time: 0.2086 s/iter total_throughput: 3131.52 samples/s lr: 9.56e-04 [09/26 13:42:38] lb.utils.events INFO: eta: 13:55:13 iteration: 50999/375342 consumed_samples: 52224000 total_loss: 4.047 time: 0.3270 s/iter data_time: 0.2126 s/iter total_throughput: 3131.44 samples/s lr: 9.56e-04 [09/26 13:43:10] lb.utils.events INFO: eta: 13:51:18 iteration: 51099/375342 consumed_samples: 52326400 total_loss: 4.059 time: 0.3270 s/iter data_time: 0.2060 s/iter total_throughput: 3131.52 samples/s lr: 9.55e-04 [09/26 13:43:43] lb.utils.events INFO: eta: 13:47:38 iteration: 51199/375342 consumed_samples: 52428800 total_loss: 4.076 time: 0.3270 s/iter data_time: 0.1939 s/iter total_throughput: 3131.48 samples/s lr: 9.55e-04 [09/26 13:44:15] lb.utils.events INFO: eta: 13:48:57 iteration: 51299/375342 consumed_samples: 52531200 total_loss: 4.062 time: 0.3270 s/iter data_time: 0.2040 s/iter total_throughput: 3131.47 samples/s lr: 9.55e-04 [09/26 13:44:48] lb.utils.events INFO: eta: 13:50:05 iteration: 51399/375342 consumed_samples: 52633600 total_loss: 4.055 time: 0.3270 s/iter data_time: 0.2166 s/iter total_throughput: 3131.49 samples/s lr: 9.55e-04 [09/26 13:45:21] lb.utils.events INFO: eta: 13:51:11 iteration: 51499/375342 consumed_samples: 52736000 total_loss: 4.023 time: 0.3270 s/iter data_time: 0.2070 s/iter total_throughput: 3131.49 samples/s lr: 9.55e-04 [09/26 13:45:54] lb.utils.events INFO: eta: 13:50:42 iteration: 51599/375342 consumed_samples: 52838400 total_loss: 4.031 time: 0.3270 s/iter data_time: 0.2031 s/iter total_throughput: 3131.45 samples/s lr: 9.55e-04 [09/26 13:46:26] lb.utils.events INFO: eta: 13:51:55 iteration: 51699/375342 consumed_samples: 52940800 total_loss: 4.059 time: 0.3270 s/iter data_time: 0.2170 s/iter total_throughput: 3131.45 samples/s lr: 9.54e-04 [09/26 13:46:59] lb.utils.events INFO: eta: 13:53:46 iteration: 51799/375342 consumed_samples: 53043200 total_loss: 4.066 time: 0.3270 s/iter data_time: 0.2409 s/iter total_throughput: 3131.39 samples/s lr: 9.54e-04 [09/26 13:47:32] lb.utils.events INFO: eta: 13:53:58 iteration: 51899/375342 consumed_samples: 53145600 total_loss: 4.053 time: 0.3270 s/iter data_time: 0.1958 s/iter total_throughput: 3131.36 samples/s lr: 9.54e-04 [09/26 13:48:06] lb.utils.events INFO: eta: 13:54:20 iteration: 51999/375342 consumed_samples: 53248000 total_loss: 4.064 time: 0.3270 s/iter data_time: 0.2139 s/iter total_throughput: 3131.28 samples/s lr: 9.54e-04 [09/26 13:48:38] lb.utils.events INFO: eta: 13:54:02 iteration: 52099/375342 consumed_samples: 53350400 total_loss: 4.062 time: 0.3270 s/iter data_time: 0.2026 s/iter total_throughput: 3131.29 samples/s lr: 9.54e-04 [09/26 13:49:11] lb.utils.events INFO: eta: 13:54:39 iteration: 52199/375342 consumed_samples: 53452800 total_loss: 4.064 time: 0.3270 s/iter data_time: 0.2003 s/iter total_throughput: 3131.33 samples/s lr: 9.54e-04 [09/26 13:49:43] lb.utils.events INFO: eta: 13:54:33 iteration: 52299/375342 consumed_samples: 53555200 total_loss: 4.094 time: 0.3270 s/iter data_time: 0.2277 s/iter total_throughput: 3131.38 samples/s lr: 9.53e-04 [09/26 13:50:16] lb.utils.events INFO: eta: 14:03:06 iteration: 52399/375342 consumed_samples: 53657600 total_loss: 4.094 time: 0.3270 s/iter data_time: 0.2234 s/iter total_throughput: 3131.42 samples/s lr: 9.53e-04 [09/26 13:50:48] lb.utils.events INFO: eta: 14:06:15 iteration: 52499/375342 consumed_samples: 53760000 total_loss: 4.072 time: 0.3270 s/iter data_time: 0.2177 s/iter total_throughput: 3131.46 samples/s lr: 9.53e-04 [09/26 13:51:21] lb.utils.events INFO: eta: 14:17:15 iteration: 52599/375342 consumed_samples: 53862400 total_loss: 4.049 time: 0.3270 s/iter data_time: 0.2445 s/iter total_throughput: 3131.35 samples/s lr: 9.53e-04 [09/26 13:51:55] lb.utils.events INFO: eta: 14:21:28 iteration: 52699/375342 consumed_samples: 53964800 total_loss: 4.057 time: 0.3270 s/iter data_time: 0.2121 s/iter total_throughput: 3131.27 samples/s lr: 9.53e-04 [09/26 13:52:27] lb.utils.events INFO: eta: 14:14:45 iteration: 52799/375342 consumed_samples: 54067200 total_loss: 4.053 time: 0.3270 s/iter data_time: 0.2254 s/iter total_throughput: 3131.28 samples/s lr: 9.52e-04 [09/26 13:53:00] lb.utils.events INFO: eta: 14:18:22 iteration: 52899/375342 consumed_samples: 54169600 total_loss: 4.047 time: 0.3270 s/iter data_time: 0.2232 s/iter total_throughput: 3131.32 samples/s lr: 9.52e-04 [09/26 13:53:32] lb.utils.events INFO: eta: 14:30:20 iteration: 52999/375342 consumed_samples: 54272000 total_loss: 4.053 time: 0.3270 s/iter data_time: 0.2201 s/iter total_throughput: 3131.34 samples/s lr: 9.52e-04 [09/26 13:54:05] lb.utils.events INFO: eta: 14:32:24 iteration: 53099/375342 consumed_samples: 54374400 total_loss: 4.059 time: 0.3270 s/iter data_time: 0.2121 s/iter total_throughput: 3131.29 samples/s lr: 9.52e-04 [09/26 13:54:38] lb.utils.events INFO: eta: 14:38:06 iteration: 53199/375342 consumed_samples: 54476800 total_loss: 4.055 time: 0.3270 s/iter data_time: 0.2169 s/iter total_throughput: 3131.26 samples/s lr: 9.52e-04 [09/26 13:55:11] lb.utils.events INFO: eta: 14:51:06 iteration: 53299/375342 consumed_samples: 54579200 total_loss: 4.043 time: 0.3270 s/iter data_time: 0.2221 s/iter total_throughput: 3131.30 samples/s lr: 9.52e-04 [09/26 13:55:44] lb.utils.events INFO: eta: 14:40:39 iteration: 53399/375342 consumed_samples: 54681600 total_loss: 4.051 time: 0.3270 s/iter data_time: 0.2168 s/iter total_throughput: 3131.14 samples/s lr: 9.51e-04 [09/26 13:56:16] lb.utils.events INFO: eta: 14:49:06 iteration: 53499/375342 consumed_samples: 54784000 total_loss: 4.051 time: 0.3270 s/iter data_time: 0.2243 s/iter total_throughput: 3131.27 samples/s lr: 9.51e-04 [09/26 13:56:49] lb.utils.events INFO: eta: 14:46:19 iteration: 53599/375342 consumed_samples: 54886400 total_loss: 4.051 time: 0.3270 s/iter data_time: 0.2380 s/iter total_throughput: 3131.27 samples/s lr: 9.51e-04 [09/26 13:57:22] lb.utils.events INFO: eta: 14:53:02 iteration: 53699/375342 consumed_samples: 54988800 total_loss: 4.066 time: 0.3270 s/iter data_time: 0.2558 s/iter total_throughput: 3131.20 samples/s lr: 9.51e-04 [09/26 13:57:55] lb.utils.events INFO: eta: 15:00:32 iteration: 53799/375342 consumed_samples: 55091200 total_loss: 4.064 time: 0.3270 s/iter data_time: 0.2069 s/iter total_throughput: 3131.17 samples/s lr: 9.51e-04 [09/26 13:58:28] lb.utils.events INFO: eta: 14:51:52 iteration: 53899/375342 consumed_samples: 55193600 total_loss: 4.055 time: 0.3270 s/iter data_time: 0.2190 s/iter total_throughput: 3131.17 samples/s lr: 9.50e-04 [09/26 13:59:00] lb.utils.events INFO: eta: 14:37:42 iteration: 53999/375342 consumed_samples: 55296000 total_loss: 4.045 time: 0.3270 s/iter data_time: 0.2020 s/iter total_throughput: 3131.18 samples/s lr: 9.50e-04 [09/26 13:59:33] lb.utils.events INFO: eta: 14:51:34 iteration: 54099/375342 consumed_samples: 55398400 total_loss: 4.027 time: 0.3270 s/iter data_time: 0.2198 s/iter total_throughput: 3131.17 samples/s lr: 9.50e-04 [09/26 14:00:06] lb.utils.events INFO: eta: 14:59:29 iteration: 54199/375342 consumed_samples: 55500800 total_loss: 4.045 time: 0.3270 s/iter data_time: 0.2179 s/iter total_throughput: 3131.21 samples/s lr: 9.50e-04 [09/26 14:00:38] lb.utils.events INFO: eta: 15:43:22 iteration: 54299/375342 consumed_samples: 55603200 total_loss: 4.043 time: 0.3270 s/iter data_time: 0.2604 s/iter total_throughput: 3131.20 samples/s lr: 9.50e-04 [09/26 14:01:11] lb.utils.events INFO: eta: 15:21:05 iteration: 54399/375342 consumed_samples: 55705600 total_loss: 4.029 time: 0.3270 s/iter data_time: 0.2199 s/iter total_throughput: 3131.18 samples/s lr: 9.50e-04 [09/26 14:01:44] lb.utils.events INFO: eta: 14:54:55 iteration: 54499/375342 consumed_samples: 55808000 total_loss: 4.039 time: 0.3270 s/iter data_time: 0.2071 s/iter total_throughput: 3131.19 samples/s lr: 9.49e-04 [09/26 14:02:16] lb.utils.events INFO: eta: 14:37:38 iteration: 54599/375342 consumed_samples: 55910400 total_loss: 4.043 time: 0.3270 s/iter data_time: 0.2247 s/iter total_throughput: 3131.22 samples/s lr: 9.49e-04 [09/26 14:02:49] lb.utils.events INFO: eta: 14:39:44 iteration: 54699/375342 consumed_samples: 56012800 total_loss: 4.043 time: 0.3270 s/iter data_time: 0.2168 s/iter total_throughput: 3131.22 samples/s lr: 9.49e-04 [09/26 14:03:22] lb.utils.events INFO: eta: 14:47:51 iteration: 54799/375342 consumed_samples: 56115200 total_loss: 4.029 time: 0.3270 s/iter data_time: 0.2284 s/iter total_throughput: 3131.25 samples/s lr: 9.49e-04 [09/26 14:03:54] lb.utils.events INFO: eta: 14:48:34 iteration: 54899/375342 consumed_samples: 56217600 total_loss: 4.035 time: 0.3270 s/iter data_time: 0.2146 s/iter total_throughput: 3131.28 samples/s lr: 9.49e-04 [09/26 14:04:26] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0054999 [09/26 14:04:27] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/26 14:04:27] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/26 14:04:31] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0831 s/iter. Inference: 0.1482 s/iter. Eval: 0.0018 s/iter. Total: 0.2331 s/iter. ETA=0:00:08 [09/26 14:04:37] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1456 s/iter. Inference: 0.1490 s/iter. Eval: 0.0020 s/iter. Total: 0.2968 s/iter. ETA=0:00:05 [09/26 14:04:42] lb.evaluation.evaluator INFO: Inference done 49152/50000. Dataloading: 0.1296 s/iter. Inference: 0.1508 s/iter. Eval: 0.0020 s/iter. Total: 0.2824 s/iter. ETA=0:00:00 [09/26 14:04:42] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/26 14:04:42] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.594518 (0.000252 s / iter per device, on 8 devices) [09/26 14:04:42] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000134 s / iter per device, on 8 devices) [09/26 14:04:42] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/26 14:04:42] lb.evaluation.utils INFO: copypaste: Acc@1=67.51599999999999 [09/26 14:04:42] lb.evaluation.utils INFO: copypaste: Acc@5=88.354 [09/26 14:04:42] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 67.51600, better than last best score 66.70200 @ iteration 49999. [09/26 14:04:42] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/26 14:04:43] lb.utils.events INFO: eta: 14:44:59 iteration: 54999/375342 consumed_samples: 56320000 total_loss: 4.062 time: 0.3270 s/iter data_time: 0.2120 s/iter total_throughput: 3131.38 samples/s lr: 9.48e-04 [09/26 14:05:14] lb.utils.events INFO: eta: 14:45:32 iteration: 55099/375342 consumed_samples: 56422400 total_loss: 4.025 time: 0.3270 s/iter data_time: 0.2352 s/iter total_throughput: 3131.66 samples/s lr: 9.48e-04 [09/26 14:05:46] lb.utils.events INFO: eta: 15:02:54 iteration: 55199/375342 consumed_samples: 56524800 total_loss: 4.004 time: 0.3270 s/iter data_time: 0.2316 s/iter total_throughput: 3131.72 samples/s lr: 9.48e-04 [09/26 14:06:19] lb.utils.events INFO: eta: 14:38:26 iteration: 55299/375342 consumed_samples: 56627200 total_loss: 4.016 time: 0.3270 s/iter data_time: 0.2067 s/iter total_throughput: 3131.69 samples/s lr: 9.48e-04 [09/26 14:06:51] lb.utils.events INFO: eta: 14:27:08 iteration: 55399/375342 consumed_samples: 56729600 total_loss: 4.021 time: 0.3270 s/iter data_time: 0.2151 s/iter total_throughput: 3131.78 samples/s lr: 9.48e-04 [09/26 14:07:24] lb.utils.events INFO: eta: 14:42:16 iteration: 55499/375342 consumed_samples: 56832000 total_loss: 4.02 time: 0.3270 s/iter data_time: 0.2223 s/iter total_throughput: 3131.81 samples/s lr: 9.48e-04 [09/26 14:07:56] lb.utils.events INFO: eta: 14:42:21 iteration: 55599/375342 consumed_samples: 56934400 total_loss: 4.023 time: 0.3270 s/iter data_time: 0.2160 s/iter total_throughput: 3131.86 samples/s lr: 9.47e-04 [09/26 14:08:29] lb.utils.events INFO: eta: 14:27:45 iteration: 55699/375342 consumed_samples: 57036800 total_loss: 4.039 time: 0.3270 s/iter data_time: 0.2139 s/iter total_throughput: 3131.92 samples/s lr: 9.47e-04 [09/26 14:09:01] lb.utils.events INFO: eta: 14:30:14 iteration: 55799/375342 consumed_samples: 57139200 total_loss: 4.037 time: 0.3269 s/iter data_time: 0.2130 s/iter total_throughput: 3132.03 samples/s lr: 9.47e-04 [09/26 14:09:34] lb.utils.events INFO: eta: 14:35:20 iteration: 55899/375342 consumed_samples: 57241600 total_loss: 4.031 time: 0.3269 s/iter data_time: 0.2207 s/iter total_throughput: 3132.02 samples/s lr: 9.47e-04 [09/26 14:10:06] lb.utils.events INFO: eta: 14:34:38 iteration: 55999/375342 consumed_samples: 57344000 total_loss: 4.039 time: 0.3269 s/iter data_time: 0.2220 s/iter total_throughput: 3132.09 samples/s lr: 9.47e-04 [09/26 14:10:39] lb.utils.events INFO: eta: 14:41:47 iteration: 56099/375342 consumed_samples: 57446400 total_loss: 4.033 time: 0.3269 s/iter data_time: 0.2425 s/iter total_throughput: 3132.06 samples/s lr: 9.46e-04 [09/26 14:11:12] lb.utils.events INFO: eta: 14:50:33 iteration: 56199/375342 consumed_samples: 57548800 total_loss: 4.037 time: 0.3269 s/iter data_time: 0.2626 s/iter total_throughput: 3131.98 samples/s lr: 9.46e-04 [09/26 14:11:45] lb.utils.events INFO: eta: 15:08:17 iteration: 56299/375342 consumed_samples: 57651200 total_loss: 4.027 time: 0.3270 s/iter data_time: 0.2353 s/iter total_throughput: 3131.91 samples/s lr: 9.46e-04 [09/26 14:12:18] lb.utils.events INFO: eta: 15:30:57 iteration: 56399/375342 consumed_samples: 57753600 total_loss: 4.016 time: 0.3270 s/iter data_time: 0.2258 s/iter total_throughput: 3131.79 samples/s lr: 9.46e-04 [09/26 14:12:52] lb.utils.events INFO: eta: 15:35:44 iteration: 56499/375342 consumed_samples: 57856000 total_loss: 4.016 time: 0.3270 s/iter data_time: 0.2238 s/iter total_throughput: 3131.64 samples/s lr: 9.46e-04 [09/26 14:13:25] lb.utils.events INFO: eta: 16:19:44 iteration: 56599/375342 consumed_samples: 57958400 total_loss: 4.016 time: 0.3270 s/iter data_time: 0.2284 s/iter total_throughput: 3131.52 samples/s lr: 9.45e-04 [09/26 14:13:59] lb.utils.events INFO: eta: 17:21:23 iteration: 56699/375342 consumed_samples: 58060800 total_loss: 4.006 time: 0.3270 s/iter data_time: 0.2448 s/iter total_throughput: 3131.40 samples/s lr: 9.45e-04 [09/26 14:14:32] lb.utils.events INFO: eta: 19:11:50 iteration: 56799/375342 consumed_samples: 58163200 total_loss: 4.014 time: 0.3270 s/iter data_time: 0.2303 s/iter total_throughput: 3131.34 samples/s lr: 9.45e-04 [09/26 14:15:05] lb.utils.events INFO: eta: 20:29:19 iteration: 56899/375342 consumed_samples: 58265600 total_loss: 4.029 time: 0.3270 s/iter data_time: 0.2499 s/iter total_throughput: 3131.28 samples/s lr: 9.45e-04 [09/26 14:15:39] lb.utils.events INFO: eta: 22:40:03 iteration: 56999/375342 consumed_samples: 58368000 total_loss: 4.047 time: 0.3270 s/iter data_time: 0.2321 s/iter total_throughput: 3131.12 samples/s lr: 9.45e-04 [09/26 14:16:12] lb.utils.events INFO: eta: 22:17:39 iteration: 57099/375342 consumed_samples: 58470400 total_loss: 4.037 time: 0.3270 s/iter data_time: 0.2171 s/iter total_throughput: 3131.08 samples/s lr: 9.45e-04 [09/26 14:16:45] lb.utils.events INFO: eta: 17:07:34 iteration: 57199/375342 consumed_samples: 58572800 total_loss: 4.012 time: 0.3271 s/iter data_time: 0.2168 s/iter total_throughput: 3130.99 samples/s lr: 9.44e-04 [09/26 14:17:18] lb.utils.events INFO: eta: 15:02:13 iteration: 57299/375342 consumed_samples: 58675200 total_loss: 4.014 time: 0.3271 s/iter data_time: 0.2011 s/iter total_throughput: 3130.93 samples/s lr: 9.44e-04 [09/26 14:17:51] lb.utils.events INFO: eta: 14:41:23 iteration: 57399/375342 consumed_samples: 58777600 total_loss: 4.02 time: 0.3271 s/iter data_time: 0.2133 s/iter total_throughput: 3130.82 samples/s lr: 9.44e-04 [09/26 14:18:24] lb.utils.events INFO: eta: 14:22:05 iteration: 57499/375342 consumed_samples: 58880000 total_loss: 4.02 time: 0.3271 s/iter data_time: 0.2074 s/iter total_throughput: 3130.75 samples/s lr: 9.44e-04 [09/26 14:18:57] lb.utils.events INFO: eta: 14:11:25 iteration: 57599/375342 consumed_samples: 58982400 total_loss: 4.008 time: 0.3271 s/iter data_time: 0.2330 s/iter total_throughput: 3130.76 samples/s lr: 9.44e-04 [09/26 14:19:30] lb.utils.events INFO: eta: 14:16:28 iteration: 57699/375342 consumed_samples: 59084800 total_loss: 4.008 time: 0.3271 s/iter data_time: 0.2247 s/iter total_throughput: 3130.73 samples/s lr: 9.43e-04 [09/26 14:20:03] lb.utils.events INFO: eta: 14:07:04 iteration: 57799/375342 consumed_samples: 59187200 total_loss: 3.984 time: 0.3271 s/iter data_time: 0.2241 s/iter total_throughput: 3130.62 samples/s lr: 9.43e-04 [09/26 14:20:37] lb.utils.events INFO: eta: 13:54:53 iteration: 57899/375342 consumed_samples: 59289600 total_loss: 3.979 time: 0.3271 s/iter data_time: 0.2287 s/iter total_throughput: 3130.45 samples/s lr: 9.43e-04 [09/26 14:21:10] lb.utils.events INFO: eta: 13:52:53 iteration: 57999/375342 consumed_samples: 59392000 total_loss: 3.981 time: 0.3271 s/iter data_time: 0.2301 s/iter total_throughput: 3130.40 samples/s lr: 9.43e-04 [09/26 14:21:43] lb.utils.events INFO: eta: 13:54:22 iteration: 58099/375342 consumed_samples: 59494400 total_loss: 3.995 time: 0.3271 s/iter data_time: 0.2214 s/iter total_throughput: 3130.35 samples/s lr: 9.43e-04 [09/26 14:22:16] lb.utils.events INFO: eta: 14:05:15 iteration: 58199/375342 consumed_samples: 59596800 total_loss: 3.999 time: 0.3271 s/iter data_time: 0.2217 s/iter total_throughput: 3130.31 samples/s lr: 9.42e-04 [09/26 14:22:49] lb.utils.events INFO: eta: 14:08:34 iteration: 58299/375342 consumed_samples: 59699200 total_loss: 4.004 time: 0.3271 s/iter data_time: 0.2080 s/iter total_throughput: 3130.27 samples/s lr: 9.42e-04 [09/26 14:23:22] lb.utils.events INFO: eta: 14:15:09 iteration: 58399/375342 consumed_samples: 59801600 total_loss: 4.016 time: 0.3271 s/iter data_time: 0.2250 s/iter total_throughput: 3130.17 samples/s lr: 9.42e-04 [09/26 14:23:56] lb.utils.events INFO: eta: 14:17:45 iteration: 58499/375342 consumed_samples: 59904000 total_loss: 4.016 time: 0.3271 s/iter data_time: 0.2045 s/iter total_throughput: 3130.09 samples/s lr: 9.42e-04 [09/26 14:24:28] lb.utils.events INFO: eta: 14:19:07 iteration: 58599/375342 consumed_samples: 60006400 total_loss: 4.031 time: 0.3271 s/iter data_time: 0.2232 s/iter total_throughput: 3130.12 samples/s lr: 9.42e-04 [09/26 14:25:01] lb.utils.events INFO: eta: 14:30:45 iteration: 58699/375342 consumed_samples: 60108800 total_loss: 4.049 time: 0.3271 s/iter data_time: 0.2348 s/iter total_throughput: 3130.06 samples/s lr: 9.41e-04 [09/26 14:25:34] lb.utils.events INFO: eta: 14:33:05 iteration: 58799/375342 consumed_samples: 60211200 total_loss: 4.027 time: 0.3272 s/iter data_time: 0.2441 s/iter total_throughput: 3130.02 samples/s lr: 9.41e-04 [09/26 14:26:07] lb.utils.events INFO: eta: 14:45:29 iteration: 58899/375342 consumed_samples: 60313600 total_loss: 4.016 time: 0.3272 s/iter data_time: 0.2324 s/iter total_throughput: 3129.99 samples/s lr: 9.41e-04 [09/26 14:26:40] lb.utils.events INFO: eta: 14:41:29 iteration: 58999/375342 consumed_samples: 60416000 total_loss: 3.993 time: 0.3272 s/iter data_time: 0.2222 s/iter total_throughput: 3129.92 samples/s lr: 9.41e-04 [09/26 14:27:13] lb.utils.events INFO: eta: 14:18:44 iteration: 59099/375342 consumed_samples: 60518400 total_loss: 3.981 time: 0.3272 s/iter data_time: 0.2045 s/iter total_throughput: 3129.88 samples/s lr: 9.41e-04 [09/26 14:27:47] lb.utils.events INFO: eta: 14:21:24 iteration: 59199/375342 consumed_samples: 60620800 total_loss: 3.991 time: 0.3272 s/iter data_time: 0.2276 s/iter total_throughput: 3129.75 samples/s lr: 9.40e-04 [09/26 14:28:20] lb.utils.events INFO: eta: 14:44:21 iteration: 59299/375342 consumed_samples: 60723200 total_loss: 4.012 time: 0.3272 s/iter data_time: 0.2398 s/iter total_throughput: 3129.71 samples/s lr: 9.40e-04 [09/26 14:28:53] lb.utils.events INFO: eta: 15:04:16 iteration: 59399/375342 consumed_samples: 60825600 total_loss: 3.999 time: 0.3272 s/iter data_time: 0.2481 s/iter total_throughput: 3129.64 samples/s lr: 9.40e-04 [09/26 14:29:26] lb.utils.events INFO: eta: 15:46:18 iteration: 59499/375342 consumed_samples: 60928000 total_loss: 4 time: 0.3272 s/iter data_time: 0.2185 s/iter total_throughput: 3129.57 samples/s lr: 9.40e-04 [09/26 14:29:59] lb.utils.events INFO: eta: 18:30:47 iteration: 59599/375342 consumed_samples: 61030400 total_loss: 4.014 time: 0.3272 s/iter data_time: 0.2394 s/iter total_throughput: 3129.57 samples/s lr: 9.40e-04 [09/26 14:30:32] lb.utils.events INFO: eta: 16:22:20 iteration: 59699/375342 consumed_samples: 61132800 total_loss: 4.008 time: 0.3272 s/iter data_time: 0.2391 s/iter total_throughput: 3129.57 samples/s lr: 9.39e-04 [09/26 14:31:05] lb.utils.events INFO: eta: 15:40:17 iteration: 59799/375342 consumed_samples: 61235200 total_loss: 3.996 time: 0.3272 s/iter data_time: 0.2269 s/iter total_throughput: 3129.51 samples/s lr: 9.39e-04 [09/26 14:31:38] lb.utils.events INFO: eta: 15:53:56 iteration: 59899/375342 consumed_samples: 61337600 total_loss: 3.987 time: 0.3272 s/iter data_time: 0.2436 s/iter total_throughput: 3129.41 samples/s lr: 9.39e-04 [09/26 14:32:11] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0059999 [09/26 14:32:12] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/26 14:32:12] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/26 14:32:16] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0819 s/iter. Inference: 0.1496 s/iter. Eval: 0.0021 s/iter. Total: 0.2336 s/iter. ETA=0:00:08 [09/26 14:32:21] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1421 s/iter. Inference: 0.1493 s/iter. Eval: 0.0020 s/iter. Total: 0.2935 s/iter. ETA=0:00:05 [09/26 14:32:26] lb.evaluation.evaluator INFO: Inference done 49152/50000. Dataloading: 0.1312 s/iter. Inference: 0.1481 s/iter. Eval: 0.0020 s/iter. Total: 0.2813 s/iter. ETA=0:00:00 [09/26 14:32:27] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/26 14:32:27] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.470355 (0.000249 s / iter per device, on 8 devices) [09/26 14:32:27] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000131 s / iter per device, on 8 devices) [09/26 14:32:27] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/26 14:32:27] lb.evaluation.utils INFO: copypaste: Acc@1=68.022 [09/26 14:32:27] lb.evaluation.utils INFO: copypaste: Acc@5=88.574 [09/26 14:32:27] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 68.02200, better than last best score 67.51600 @ iteration 54999. [09/26 14:32:27] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/26 14:32:27] lb.utils.events INFO: eta: 17:45:37 iteration: 59999/375342 consumed_samples: 61440000 total_loss: 4.002 time: 0.3272 s/iter data_time: 0.2394 s/iter total_throughput: 3129.40 samples/s lr: 9.39e-04 [09/26 14:32:59] lb.utils.events INFO: eta: 18:39:26 iteration: 60099/375342 consumed_samples: 61542400 total_loss: 3.979 time: 0.3272 s/iter data_time: 0.2458 s/iter total_throughput: 3129.55 samples/s lr: 9.39e-04 [09/26 14:33:33] lb.utils.events INFO: eta: 18:41:14 iteration: 60199/375342 consumed_samples: 61644800 total_loss: 3.981 time: 0.3272 s/iter data_time: 0.2263 s/iter total_throughput: 3129.46 samples/s lr: 9.38e-04 [09/26 14:34:06] lb.utils.events INFO: eta: 15:56:15 iteration: 60299/375342 consumed_samples: 61747200 total_loss: 3.994 time: 0.3272 s/iter data_time: 0.2089 s/iter total_throughput: 3129.35 samples/s lr: 9.38e-04 [09/26 14:34:39] lb.utils.events INFO: eta: 14:37:07 iteration: 60399/375342 consumed_samples: 61849600 total_loss: 3.992 time: 0.3272 s/iter data_time: 0.2235 s/iter total_throughput: 3129.33 samples/s lr: 9.38e-04 [09/26 14:35:12] lb.utils.events INFO: eta: 14:36:04 iteration: 60499/375342 consumed_samples: 61952000 total_loss: 3.992 time: 0.3272 s/iter data_time: 0.2197 s/iter total_throughput: 3129.32 samples/s lr: 9.38e-04 [09/26 14:35:45] lb.utils.events INFO: eta: 14:21:03 iteration: 60599/375342 consumed_samples: 62054400 total_loss: 3.984 time: 0.3272 s/iter data_time: 0.2206 s/iter total_throughput: 3129.27 samples/s lr: 9.38e-04 [09/26 14:36:17] lb.utils.events INFO: eta: 14:18:34 iteration: 60699/375342 consumed_samples: 62156800 total_loss: 3.97 time: 0.3272 s/iter data_time: 0.2098 s/iter total_throughput: 3129.25 samples/s lr: 9.37e-04 [09/26 14:36:50] lb.utils.events INFO: eta: 14:06:15 iteration: 60799/375342 consumed_samples: 62259200 total_loss: 4.002 time: 0.3272 s/iter data_time: 0.2140 s/iter total_throughput: 3129.21 samples/s lr: 9.37e-04 [09/26 14:37:23] lb.utils.events INFO: eta: 14:13:45 iteration: 60899/375342 consumed_samples: 62361600 total_loss: 4.018 time: 0.3272 s/iter data_time: 0.2628 s/iter total_throughput: 3129.19 samples/s lr: 9.37e-04 [09/26 14:37:56] lb.utils.events INFO: eta: 14:27:05 iteration: 60999/375342 consumed_samples: 62464000 total_loss: 4.008 time: 0.3272 s/iter data_time: 0.2491 s/iter total_throughput: 3129.15 samples/s lr: 9.37e-04 [09/26 14:38:30] lb.utils.events INFO: eta: 15:37:03 iteration: 61099/375342 consumed_samples: 62566400 total_loss: 4.004 time: 0.3273 s/iter data_time: 0.2191 s/iter total_throughput: 3129.04 samples/s lr: 9.37e-04 [09/26 14:39:03] lb.utils.events INFO: eta: 18:37:51 iteration: 61199/375342 consumed_samples: 62668800 total_loss: 4.002 time: 0.3273 s/iter data_time: 0.2491 s/iter total_throughput: 3129.02 samples/s lr: 9.36e-04 [09/26 14:39:36] lb.utils.events INFO: eta: 20:37:29 iteration: 61299/375342 consumed_samples: 62771200 total_loss: 3.98 time: 0.3273 s/iter data_time: 0.2423 s/iter total_throughput: 3128.97 samples/s lr: 9.36e-04 [09/26 14:40:09] lb.utils.events INFO: eta: 21:22:44 iteration: 61399/375342 consumed_samples: 62873600 total_loss: 3.977 time: 0.3273 s/iter data_time: 0.2314 s/iter total_throughput: 3128.88 samples/s lr: 9.36e-04 [09/26 14:40:42] lb.utils.events INFO: eta: 20:36:41 iteration: 61499/375342 consumed_samples: 62976000 total_loss: 3.97 time: 0.3273 s/iter data_time: 0.2460 s/iter total_throughput: 3128.82 samples/s lr: 9.36e-04 [09/26 14:41:15] lb.utils.events INFO: eta: 20:32:06 iteration: 61599/375342 consumed_samples: 63078400 total_loss: 3.97 time: 0.3273 s/iter data_time: 0.2417 s/iter total_throughput: 3128.78 samples/s lr: 9.36e-04 [09/26 14:41:48] lb.utils.events INFO: eta: 22:05:44 iteration: 61699/375342 consumed_samples: 63180800 total_loss: 3.988 time: 0.3273 s/iter data_time: 0.2364 s/iter total_throughput: 3128.76 samples/s lr: 9.35e-04 [09/26 14:42:21] lb.utils.events INFO: eta: 23:11:41 iteration: 61799/375342 consumed_samples: 63283200 total_loss: 3.954 time: 0.3273 s/iter data_time: 0.2233 s/iter total_throughput: 3128.72 samples/s lr: 9.35e-04 [09/26 14:42:54] lb.utils.events INFO: eta: 22:29:07 iteration: 61899/375342 consumed_samples: 63385600 total_loss: 3.959 time: 0.3273 s/iter data_time: 0.2321 s/iter total_throughput: 3128.68 samples/s lr: 9.35e-04 [09/26 14:43:27] lb.utils.events INFO: eta: 20:20:27 iteration: 61999/375342 consumed_samples: 63488000 total_loss: 3.97 time: 0.3273 s/iter data_time: 0.2505 s/iter total_throughput: 3128.64 samples/s lr: 9.35e-04 [09/26 14:44:01] lb.utils.events INFO: eta: 18:52:30 iteration: 62099/375342 consumed_samples: 63590400 total_loss: 3.982 time: 0.3273 s/iter data_time: 0.2329 s/iter total_throughput: 3128.51 samples/s lr: 9.35e-04 [09/26 14:44:33] lb.utils.events INFO: eta: 16:12:02 iteration: 62199/375342 consumed_samples: 63692800 total_loss: 3.993 time: 0.3273 s/iter data_time: 0.2228 s/iter total_throughput: 3128.48 samples/s lr: 9.34e-04 [09/26 14:45:07] lb.utils.events INFO: eta: 15:03:46 iteration: 62299/375342 consumed_samples: 63795200 total_loss: 3.994 time: 0.3273 s/iter data_time: 0.2349 s/iter total_throughput: 3128.42 samples/s lr: 9.34e-04 [09/26 14:45:40] lb.utils.events INFO: eta: 15:01:04 iteration: 62399/375342 consumed_samples: 63897600 total_loss: 4.012 time: 0.3273 s/iter data_time: 0.2242 s/iter total_throughput: 3128.33 samples/s lr: 9.34e-04 [09/26 14:46:13] lb.utils.events INFO: eta: 15:04:49 iteration: 62499/375342 consumed_samples: 64000000 total_loss: 3.991 time: 0.3273 s/iter data_time: 0.2174 s/iter total_throughput: 3128.32 samples/s lr: 9.34e-04 [09/26 14:46:46] lb.utils.events INFO: eta: 14:31:51 iteration: 62599/375342 consumed_samples: 64102400 total_loss: 3.967 time: 0.3273 s/iter data_time: 0.1961 s/iter total_throughput: 3128.31 samples/s lr: 9.34e-04 [09/26 14:47:19] lb.utils.events INFO: eta: 14:19:24 iteration: 62699/375342 consumed_samples: 64204800 total_loss: 3.966 time: 0.3273 s/iter data_time: 0.2468 s/iter total_throughput: 3128.24 samples/s lr: 9.33e-04 [09/26 14:47:52] lb.utils.events INFO: eta: 14:35:12 iteration: 62799/375342 consumed_samples: 64307200 total_loss: 3.984 time: 0.3273 s/iter data_time: 0.2316 s/iter total_throughput: 3128.16 samples/s lr: 9.33e-04 [09/26 14:48:25] lb.utils.events INFO: eta: 14:08:53 iteration: 62899/375342 consumed_samples: 64409600 total_loss: 3.986 time: 0.3274 s/iter data_time: 0.2047 s/iter total_throughput: 3128.08 samples/s lr: 9.33e-04 [09/26 14:48:58] lb.utils.events INFO: eta: 14:03:03 iteration: 62999/375342 consumed_samples: 64512000 total_loss: 3.97 time: 0.3274 s/iter data_time: 0.2255 s/iter total_throughput: 3128.08 samples/s lr: 9.33e-04 [09/26 14:49:31] lb.utils.events INFO: eta: 14:00:42 iteration: 63099/375342 consumed_samples: 64614400 total_loss: 3.969 time: 0.3274 s/iter data_time: 0.2241 s/iter total_throughput: 3128.08 samples/s lr: 9.33e-04 [09/26 14:50:03] lb.utils.events INFO: eta: 14:05:57 iteration: 63199/375342 consumed_samples: 64716800 total_loss: 3.971 time: 0.3274 s/iter data_time: 0.2376 s/iter total_throughput: 3128.09 samples/s lr: 9.32e-04 [09/26 14:50:36] lb.utils.events INFO: eta: 14:32:15 iteration: 63299/375342 consumed_samples: 64819200 total_loss: 3.973 time: 0.3274 s/iter data_time: 0.2345 s/iter total_throughput: 3128.08 samples/s lr: 9.32e-04 [09/26 14:51:09] lb.utils.events INFO: eta: 14:29:53 iteration: 63399/375342 consumed_samples: 64921600 total_loss: 3.973 time: 0.3274 s/iter data_time: 0.2181 s/iter total_throughput: 3128.02 samples/s lr: 9.32e-04 [09/26 14:51:42] lb.utils.events INFO: eta: 14:08:10 iteration: 63499/375342 consumed_samples: 65024000 total_loss: 3.956 time: 0.3274 s/iter data_time: 0.2030 s/iter total_throughput: 3127.97 samples/s lr: 9.32e-04 [09/26 14:52:16] lb.utils.events INFO: eta: 14:17:20 iteration: 63599/375342 consumed_samples: 65126400 total_loss: 3.963 time: 0.3274 s/iter data_time: 0.2355 s/iter total_throughput: 3127.82 samples/s lr: 9.32e-04 [09/26 14:52:49] lb.utils.events INFO: eta: 13:59:22 iteration: 63699/375342 consumed_samples: 65228800 total_loss: 4.001 time: 0.3274 s/iter data_time: 0.2013 s/iter total_throughput: 3127.78 samples/s lr: 9.31e-04 [09/26 14:53:22] lb.utils.events INFO: eta: 13:53:25 iteration: 63799/375342 consumed_samples: 65331200 total_loss: 4.016 time: 0.3274 s/iter data_time: 0.2159 s/iter total_throughput: 3127.77 samples/s lr: 9.31e-04 [09/26 14:53:55] lb.utils.events INFO: eta: 13:57:23 iteration: 63899/375342 consumed_samples: 65433600 total_loss: 3.982 time: 0.3274 s/iter data_time: 0.2157 s/iter total_throughput: 3127.80 samples/s lr: 9.31e-04 [09/26 14:54:28] lb.utils.events INFO: eta: 13:59:26 iteration: 63999/375342 consumed_samples: 65536000 total_loss: 3.969 time: 0.3274 s/iter data_time: 0.2290 s/iter total_throughput: 3127.73 samples/s lr: 9.31e-04 [09/26 14:55:01] lb.utils.events INFO: eta: 13:58:28 iteration: 64099/375342 consumed_samples: 65638400 total_loss: 3.971 time: 0.3274 s/iter data_time: 0.2058 s/iter total_throughput: 3127.71 samples/s lr: 9.30e-04 [09/26 14:55:34] lb.utils.events INFO: eta: 13:48:32 iteration: 64199/375342 consumed_samples: 65740800 total_loss: 3.973 time: 0.3274 s/iter data_time: 0.2214 s/iter total_throughput: 3127.70 samples/s lr: 9.30e-04 [09/26 14:56:07] lb.utils.events INFO: eta: 13:39:45 iteration: 64299/375342 consumed_samples: 65843200 total_loss: 4.004 time: 0.3274 s/iter data_time: 0.2284 s/iter total_throughput: 3127.62 samples/s lr: 9.30e-04 [09/26 14:56:39] lb.utils.events INFO: eta: 13:37:16 iteration: 64399/375342 consumed_samples: 65945600 total_loss: 3.976 time: 0.3274 s/iter data_time: 0.2369 s/iter total_throughput: 3127.67 samples/s lr: 9.30e-04 [09/26 14:57:12] lb.utils.events INFO: eta: 13:57:41 iteration: 64499/375342 consumed_samples: 66048000 total_loss: 3.959 time: 0.3274 s/iter data_time: 0.2488 s/iter total_throughput: 3127.66 samples/s lr: 9.30e-04 [09/26 14:57:45] lb.utils.events INFO: eta: 14:17:07 iteration: 64599/375342 consumed_samples: 66150400 total_loss: 3.963 time: 0.3274 s/iter data_time: 0.2177 s/iter total_throughput: 3127.61 samples/s lr: 9.29e-04 [09/26 14:58:18] lb.utils.events INFO: eta: 14:40:48 iteration: 64699/375342 consumed_samples: 66252800 total_loss: 3.99 time: 0.3274 s/iter data_time: 0.2393 s/iter total_throughput: 3127.60 samples/s lr: 9.29e-04 [09/26 14:58:51] lb.utils.events INFO: eta: 18:39:37 iteration: 64799/375342 consumed_samples: 66355200 total_loss: 4.006 time: 0.3274 s/iter data_time: 0.2452 s/iter total_throughput: 3127.59 samples/s lr: 9.29e-04 [09/26 14:59:24] lb.utils.events INFO: eta: 19:06:39 iteration: 64899/375342 consumed_samples: 66457600 total_loss: 4 time: 0.3274 s/iter data_time: 0.2354 s/iter total_throughput: 3127.50 samples/s lr: 9.29e-04 [09/26 14:59:57] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0064999 [09/26 14:59:57] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/26 14:59:57] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/26 15:00:02] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0920 s/iter. Inference: 0.1506 s/iter. Eval: 0.0019 s/iter. Total: 0.2445 s/iter. ETA=0:00:09 [09/26 15:00:07] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1502 s/iter. Inference: 0.1508 s/iter. Eval: 0.0019 s/iter. Total: 0.3030 s/iter. ETA=0:00:05 [09/26 15:00:13] lb.evaluation.evaluator INFO: Inference done 50000/50000. Dataloading: 0.1351 s/iter. Inference: 0.1512 s/iter. Eval: 0.0020 s/iter. Total: 0.2884 s/iter. ETA=0:00:00 [09/26 15:00:13] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/26 15:00:13] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.689732 (0.000254 s / iter per device, on 8 devices) [09/26 15:00:13] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000133 s / iter per device, on 8 devices) [09/26 15:00:13] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/26 15:00:13] lb.evaluation.utils INFO: copypaste: Acc@1=68.756 [09/26 15:00:13] lb.evaluation.utils INFO: copypaste: Acc@5=89.022 [09/26 15:00:13] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 68.75600, better than last best score 68.02200 @ iteration 59999. [09/26 15:00:13] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/26 15:00:14] lb.utils.events INFO: eta: 19:49:25 iteration: 64999/375342 consumed_samples: 66560000 total_loss: 3.989 time: 0.3274 s/iter data_time: 0.2597 s/iter total_throughput: 3127.52 samples/s lr: 9.29e-04 [09/26 15:00:45] lb.utils.events INFO: eta: 20:42:33 iteration: 65099/375342 consumed_samples: 66662400 total_loss: 3.987 time: 0.3274 s/iter data_time: 0.2540 s/iter total_throughput: 3127.72 samples/s lr: 9.28e-04 [09/26 15:01:18] lb.utils.events INFO: eta: 22:12:35 iteration: 65199/375342 consumed_samples: 66764800 total_loss: 3.974 time: 0.3274 s/iter data_time: 0.2366 s/iter total_throughput: 3127.68 samples/s lr: 9.28e-04 [09/26 15:01:51] lb.utils.events INFO: eta: 23:32:02 iteration: 65299/375342 consumed_samples: 66867200 total_loss: 3.956 time: 0.3274 s/iter data_time: 0.2428 s/iter total_throughput: 3127.65 samples/s lr: 9.28e-04 [09/26 15:02:24] lb.utils.events INFO: eta: 22:39:10 iteration: 65399/375342 consumed_samples: 66969600 total_loss: 3.953 time: 0.3274 s/iter data_time: 0.2241 s/iter total_throughput: 3127.60 samples/s lr: 9.28e-04 [09/26 15:02:57] lb.utils.events INFO: eta: 21:56:38 iteration: 65499/375342 consumed_samples: 67072000 total_loss: 3.944 time: 0.3274 s/iter data_time: 0.2281 s/iter total_throughput: 3127.53 samples/s lr: 9.27e-04 [09/26 15:03:31] lb.utils.events INFO: eta: 20:43:39 iteration: 65599/375342 consumed_samples: 67174400 total_loss: 3.941 time: 0.3274 s/iter data_time: 0.2132 s/iter total_throughput: 3127.44 samples/s lr: 9.27e-04 [09/26 15:04:04] lb.utils.events INFO: eta: 19:28:46 iteration: 65699/375342 consumed_samples: 67276800 total_loss: 3.954 time: 0.3274 s/iter data_time: 0.2282 s/iter total_throughput: 3127.38 samples/s lr: 9.27e-04 [09/26 15:04:36] lb.utils.events INFO: eta: 17:28:31 iteration: 65799/375342 consumed_samples: 67379200 total_loss: 3.977 time: 0.3274 s/iter data_time: 0.2224 s/iter total_throughput: 3127.39 samples/s lr: 9.27e-04 [09/26 15:05:09] lb.utils.events INFO: eta: 16:43:14 iteration: 65899/375342 consumed_samples: 67481600 total_loss: 3.967 time: 0.3274 s/iter data_time: 0.2176 s/iter total_throughput: 3127.36 samples/s lr: 9.27e-04 [09/26 15:05:42] lb.utils.events INFO: eta: 14:38:42 iteration: 65999/375342 consumed_samples: 67584000 total_loss: 3.944 time: 0.3274 s/iter data_time: 0.2041 s/iter total_throughput: 3127.37 samples/s lr: 9.26e-04 [09/26 15:06:15] lb.utils.events INFO: eta: 14:25:37 iteration: 66099/375342 consumed_samples: 67686400 total_loss: 3.959 time: 0.3274 s/iter data_time: 0.2166 s/iter total_throughput: 3127.37 samples/s lr: 9.26e-04 [09/26 15:06:48] lb.utils.events INFO: eta: 14:12:19 iteration: 66199/375342 consumed_samples: 67788800 total_loss: 3.974 time: 0.3274 s/iter data_time: 0.2322 s/iter total_throughput: 3127.37 samples/s lr: 9.26e-04 [09/26 15:07:21] lb.utils.events INFO: eta: 14:03:07 iteration: 66299/375342 consumed_samples: 67891200 total_loss: 3.973 time: 0.3274 s/iter data_time: 0.2181 s/iter total_throughput: 3127.36 samples/s lr: 9.26e-04 [09/26 15:07:53] lb.utils.events INFO: eta: 13:57:10 iteration: 66399/375342 consumed_samples: 67993600 total_loss: 3.948 time: 0.3274 s/iter data_time: 0.2196 s/iter total_throughput: 3127.34 samples/s lr: 9.26e-04 [09/26 15:08:26] lb.utils.events INFO: eta: 13:54:25 iteration: 66499/375342 consumed_samples: 68096000 total_loss: 3.936 time: 0.3274 s/iter data_time: 0.2418 s/iter total_throughput: 3127.31 samples/s lr: 9.25e-04 [09/26 15:08:59] lb.utils.events INFO: eta: 14:01:20 iteration: 66599/375342 consumed_samples: 68198400 total_loss: 3.96 time: 0.3274 s/iter data_time: 0.2416 s/iter total_throughput: 3127.32 samples/s lr: 9.25e-04 [09/26 15:09:32] lb.utils.events INFO: eta: 14:15:36 iteration: 66699/375342 consumed_samples: 68300800 total_loss: 3.952 time: 0.3274 s/iter data_time: 0.2404 s/iter total_throughput: 3127.32 samples/s lr: 9.25e-04 [09/26 15:10:04] lb.utils.events INFO: eta: 14:50:09 iteration: 66799/375342 consumed_samples: 68403200 total_loss: 3.972 time: 0.3274 s/iter data_time: 0.2296 s/iter total_throughput: 3127.38 samples/s lr: 9.25e-04 [09/26 15:10:37] lb.utils.events INFO: eta: 15:16:22 iteration: 66899/375342 consumed_samples: 68505600 total_loss: 3.951 time: 0.3274 s/iter data_time: 0.2416 s/iter total_throughput: 3127.32 samples/s lr: 9.24e-04 [09/26 15:11:10] lb.utils.events INFO: eta: 17:41:26 iteration: 66999/375342 consumed_samples: 68608000 total_loss: 3.938 time: 0.3274 s/iter data_time: 0.2371 s/iter total_throughput: 3127.36 samples/s lr: 9.24e-04 [09/26 15:11:43] lb.utils.events INFO: eta: 18:12:21 iteration: 67099/375342 consumed_samples: 68710400 total_loss: 3.944 time: 0.3274 s/iter data_time: 0.2224 s/iter total_throughput: 3127.29 samples/s lr: 9.24e-04 [09/26 15:12:16] lb.utils.events INFO: eta: 17:57:47 iteration: 67199/375342 consumed_samples: 68812800 total_loss: 3.951 time: 0.3274 s/iter data_time: 0.2060 s/iter total_throughput: 3127.29 samples/s lr: 9.24e-04 [09/26 15:12:48] lb.utils.events INFO: eta: 15:14:19 iteration: 67299/375342 consumed_samples: 68915200 total_loss: 3.987 time: 0.3274 s/iter data_time: 0.2091 s/iter total_throughput: 3127.33 samples/s lr: 9.24e-04 [09/26 15:13:21] lb.utils.events INFO: eta: 14:47:07 iteration: 67399/375342 consumed_samples: 69017600 total_loss: 3.972 time: 0.3274 s/iter data_time: 0.2287 s/iter total_throughput: 3127.34 samples/s lr: 9.23e-04 [09/26 15:13:54] lb.utils.events INFO: eta: 14:46:50 iteration: 67499/375342 consumed_samples: 69120000 total_loss: 3.968 time: 0.3274 s/iter data_time: 0.2174 s/iter total_throughput: 3127.29 samples/s lr: 9.23e-04 [09/26 15:14:26] lb.utils.events INFO: eta: 14:29:57 iteration: 67599/375342 consumed_samples: 69222400 total_loss: 3.938 time: 0.3274 s/iter data_time: 0.2215 s/iter total_throughput: 3127.34 samples/s lr: 9.23e-04 [09/26 15:14:59] lb.utils.events INFO: eta: 14:16:36 iteration: 67699/375342 consumed_samples: 69324800 total_loss: 3.925 time: 0.3274 s/iter data_time: 0.2227 s/iter total_throughput: 3127.32 samples/s lr: 9.23e-04 [09/26 15:15:32] lb.utils.events INFO: eta: 14:08:19 iteration: 67799/375342 consumed_samples: 69427200 total_loss: 3.933 time: 0.3274 s/iter data_time: 0.2502 s/iter total_throughput: 3127.29 samples/s lr: 9.22e-04 [09/26 15:16:05] lb.utils.events INFO: eta: 13:50:43 iteration: 67899/375342 consumed_samples: 69529600 total_loss: 3.943 time: 0.3274 s/iter data_time: 0.2061 s/iter total_throughput: 3127.23 samples/s lr: 9.22e-04 [09/26 15:16:38] lb.utils.events INFO: eta: 13:35:08 iteration: 67999/375342 consumed_samples: 69632000 total_loss: 3.947 time: 0.3274 s/iter data_time: 0.2017 s/iter total_throughput: 3127.23 samples/s lr: 9.22e-04 [09/26 15:17:11] lb.utils.events INFO: eta: 13:24:06 iteration: 68099/375342 consumed_samples: 69734400 total_loss: 3.928 time: 0.3274 s/iter data_time: 0.2007 s/iter total_throughput: 3127.22 samples/s lr: 9.22e-04 [09/26 15:17:44] lb.utils.events INFO: eta: 13:20:54 iteration: 68199/375342 consumed_samples: 69836800 total_loss: 3.947 time: 0.3274 s/iter data_time: 0.1956 s/iter total_throughput: 3127.25 samples/s lr: 9.22e-04 [09/26 15:18:16] lb.utils.events INFO: eta: 13:20:05 iteration: 68299/375342 consumed_samples: 69939200 total_loss: 3.96 time: 0.3274 s/iter data_time: 0.2091 s/iter total_throughput: 3127.27 samples/s lr: 9.21e-04 [09/26 15:18:48] lb.utils.events INFO: eta: 13:20:38 iteration: 68399/375342 consumed_samples: 70041600 total_loss: 3.948 time: 0.3274 s/iter data_time: 0.2302 s/iter total_throughput: 3127.36 samples/s lr: 9.21e-04 [09/26 15:19:21] lb.utils.events INFO: eta: 13:20:18 iteration: 68499/375342 consumed_samples: 70144000 total_loss: 3.948 time: 0.3274 s/iter data_time: 0.2160 s/iter total_throughput: 3127.43 samples/s lr: 9.21e-04 [09/26 15:19:53] lb.utils.events INFO: eta: 13:20:26 iteration: 68599/375342 consumed_samples: 70246400 total_loss: 3.961 time: 0.3274 s/iter data_time: 0.2131 s/iter total_throughput: 3127.46 samples/s lr: 9.21e-04 [09/26 15:20:26] lb.utils.events INFO: eta: 13:20:27 iteration: 68699/375342 consumed_samples: 70348800 total_loss: 3.963 time: 0.3274 s/iter data_time: 0.2303 s/iter total_throughput: 3127.41 samples/s lr: 9.20e-04 [09/26 15:20:58] lb.utils.events INFO: eta: 13:25:38 iteration: 68799/375342 consumed_samples: 70451200 total_loss: 3.955 time: 0.3274 s/iter data_time: 0.2346 s/iter total_throughput: 3127.49 samples/s lr: 9.20e-04 [09/26 15:21:31] lb.utils.events INFO: eta: 13:31:11 iteration: 68899/375342 consumed_samples: 70553600 total_loss: 3.941 time: 0.3274 s/iter data_time: 0.2257 s/iter total_throughput: 3127.51 samples/s lr: 9.20e-04 [09/26 15:22:04] lb.utils.events INFO: eta: 13:40:44 iteration: 68999/375342 consumed_samples: 70656000 total_loss: 3.958 time: 0.3274 s/iter data_time: 0.2212 s/iter total_throughput: 3127.45 samples/s lr: 9.20e-04 [09/26 15:22:37] lb.utils.events INFO: eta: 13:46:18 iteration: 69099/375342 consumed_samples: 70758400 total_loss: 3.949 time: 0.3274 s/iter data_time: 0.2028 s/iter total_throughput: 3127.45 samples/s lr: 9.19e-04 [09/26 15:23:10] lb.utils.events INFO: eta: 13:44:43 iteration: 69199/375342 consumed_samples: 70860800 total_loss: 3.943 time: 0.3274 s/iter data_time: 0.2090 s/iter total_throughput: 3127.45 samples/s lr: 9.19e-04 [09/26 15:23:42] lb.utils.events INFO: eta: 13:45:46 iteration: 69299/375342 consumed_samples: 70963200 total_loss: 3.966 time: 0.3274 s/iter data_time: 0.1919 s/iter total_throughput: 3127.44 samples/s lr: 9.19e-04 [09/26 15:24:15] lb.utils.events INFO: eta: 13:42:50 iteration: 69399/375342 consumed_samples: 71065600 total_loss: 3.951 time: 0.3274 s/iter data_time: 0.1937 s/iter total_throughput: 3127.52 samples/s lr: 9.19e-04 [09/26 15:24:47] lb.utils.events INFO: eta: 13:34:04 iteration: 69499/375342 consumed_samples: 71168000 total_loss: 3.945 time: 0.3274 s/iter data_time: 0.1935 s/iter total_throughput: 3127.61 samples/s lr: 9.19e-04 [09/26 15:25:19] lb.utils.events INFO: eta: 13:23:31 iteration: 69599/375342 consumed_samples: 71270400 total_loss: 3.94 time: 0.3274 s/iter data_time: 0.2027 s/iter total_throughput: 3127.65 samples/s lr: 9.18e-04 [09/26 15:25:51] lb.utils.events INFO: eta: 13:22:02 iteration: 69699/375342 consumed_samples: 71372800 total_loss: 3.945 time: 0.3274 s/iter data_time: 0.2342 s/iter total_throughput: 3127.78 samples/s lr: 9.18e-04 [09/26 15:26:23] lb.utils.events INFO: eta: 13:23:14 iteration: 69799/375342 consumed_samples: 71475200 total_loss: 3.953 time: 0.3274 s/iter data_time: 0.2351 s/iter total_throughput: 3127.88 samples/s lr: 9.18e-04 [09/26 15:26:55] lb.utils.events INFO: eta: 13:22:19 iteration: 69899/375342 consumed_samples: 71577600 total_loss: 3.947 time: 0.3274 s/iter data_time: 0.2015 s/iter total_throughput: 3127.91 samples/s lr: 9.18e-04 [09/26 15:27:28] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0069999 [09/26 15:27:29] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/26 15:27:29] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/26 15:27:33] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0825 s/iter. Inference: 0.1473 s/iter. Eval: 0.0020 s/iter. Total: 0.2318 s/iter. ETA=0:00:08 [09/26 15:27:39] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1418 s/iter. Inference: 0.1498 s/iter. Eval: 0.0020 s/iter. Total: 0.2936 s/iter. ETA=0:00:05 [09/26 15:27:44] lb.evaluation.evaluator INFO: Inference done 50000/50000. Dataloading: 0.1317 s/iter. Inference: 0.1497 s/iter. Eval: 0.0020 s/iter. Total: 0.2834 s/iter. ETA=0:00:00 [09/26 15:27:44] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/26 15:27:44] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.475877 (0.000250 s / iter per device, on 8 devices) [09/26 15:27:44] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000132 s / iter per device, on 8 devices) [09/26 15:27:44] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/26 15:27:44] lb.evaluation.utils INFO: copypaste: Acc@1=69.27600000000001 [09/26 15:27:44] lb.evaluation.utils INFO: copypaste: Acc@5=89.42999999999999 [09/26 15:27:44] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 69.27600, better than last best score 68.75600 @ iteration 64999. [09/26 15:27:44] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/26 15:27:45] lb.utils.events INFO: eta: 13:16:36 iteration: 69999/375342 consumed_samples: 71680000 total_loss: 3.945 time: 0.3274 s/iter data_time: 0.2065 s/iter total_throughput: 3127.92 samples/s lr: 9.17e-04 [09/26 15:28:16] lb.utils.events INFO: eta: 13:18:55 iteration: 70099/375342 consumed_samples: 71782400 total_loss: 3.958 time: 0.3274 s/iter data_time: 0.2579 s/iter total_throughput: 3128.12 samples/s lr: 9.17e-04 [09/26 15:28:49] lb.utils.events INFO: eta: 13:43:51 iteration: 70199/375342 consumed_samples: 71884800 total_loss: 3.956 time: 0.3274 s/iter data_time: 0.2502 s/iter total_throughput: 3128.07 samples/s lr: 9.17e-04 [09/26 15:29:23] lb.utils.events INFO: eta: 14:05:24 iteration: 70299/375342 consumed_samples: 71987200 total_loss: 3.942 time: 0.3274 s/iter data_time: 0.2178 s/iter total_throughput: 3127.95 samples/s lr: 9.17e-04 [09/26 15:29:56] lb.utils.events INFO: eta: 14:05:51 iteration: 70399/375342 consumed_samples: 72089600 total_loss: 3.914 time: 0.3274 s/iter data_time: 0.2103 s/iter total_throughput: 3127.83 samples/s lr: 9.17e-04 [09/26 15:30:29] lb.utils.events INFO: eta: 14:14:36 iteration: 70499/375342 consumed_samples: 72192000 total_loss: 3.912 time: 0.3274 s/iter data_time: 0.2249 s/iter total_throughput: 3127.81 samples/s lr: 9.16e-04 [09/26 15:31:03] lb.utils.events INFO: eta: 14:49:36 iteration: 70599/375342 consumed_samples: 72294400 total_loss: 3.942 time: 0.3274 s/iter data_time: 0.2369 s/iter total_throughput: 3127.72 samples/s lr: 9.16e-04 [09/26 15:31:36] lb.utils.events INFO: eta: 16:24:55 iteration: 70699/375342 consumed_samples: 72396800 total_loss: 3.955 time: 0.3274 s/iter data_time: 0.2442 s/iter total_throughput: 3127.66 samples/s lr: 9.16e-04 [09/26 15:32:09] lb.utils.events INFO: eta: 14:21:33 iteration: 70799/375342 consumed_samples: 72499200 total_loss: 3.957 time: 0.3274 s/iter data_time: 0.2389 s/iter total_throughput: 3127.56 samples/s lr: 9.16e-04 [09/26 15:32:43] lb.utils.events INFO: eta: 14:12:15 iteration: 70899/375342 consumed_samples: 72601600 total_loss: 3.942 time: 0.3274 s/iter data_time: 0.2189 s/iter total_throughput: 3127.46 samples/s lr: 9.15e-04 [09/26 15:33:16] lb.utils.events INFO: eta: 14:14:02 iteration: 70999/375342 consumed_samples: 72704000 total_loss: 3.952 time: 0.3274 s/iter data_time: 0.1956 s/iter total_throughput: 3127.47 samples/s lr: 9.15e-04 [09/26 15:33:49] lb.utils.events INFO: eta: 13:49:28 iteration: 71099/375342 consumed_samples: 72806400 total_loss: 3.947 time: 0.3274 s/iter data_time: 0.2308 s/iter total_throughput: 3127.43 samples/s lr: 9.15e-04 [09/26 15:34:22] lb.utils.events INFO: eta: 13:36:13 iteration: 71199/375342 consumed_samples: 72908800 total_loss: 3.928 time: 0.3274 s/iter data_time: 0.2233 s/iter total_throughput: 3127.32 samples/s lr: 9.15e-04 [09/26 15:34:56] lb.utils.events INFO: eta: 13:25:04 iteration: 71299/375342 consumed_samples: 73011200 total_loss: 3.943 time: 0.3275 s/iter data_time: 0.2276 s/iter total_throughput: 3127.19 samples/s lr: 9.14e-04 [09/26 15:35:29] lb.utils.events INFO: eta: 13:22:16 iteration: 71399/375342 consumed_samples: 73113600 total_loss: 3.935 time: 0.3275 s/iter data_time: 0.2130 s/iter total_throughput: 3127.13 samples/s lr: 9.14e-04 [09/26 15:36:03] lb.utils.events INFO: eta: 13:14:57 iteration: 71499/375342 consumed_samples: 73216000 total_loss: 3.927 time: 0.3275 s/iter data_time: 0.2174 s/iter total_throughput: 3126.96 samples/s lr: 9.14e-04 [09/26 15:36:37] lb.utils.events INFO: eta: 13:09:15 iteration: 71599/375342 consumed_samples: 73318400 total_loss: 3.928 time: 0.3275 s/iter data_time: 0.2207 s/iter total_throughput: 3126.86 samples/s lr: 9.14e-04 [09/26 15:37:10] lb.utils.events INFO: eta: 13:03:55 iteration: 71699/375342 consumed_samples: 73420800 total_loss: 3.928 time: 0.3275 s/iter data_time: 0.2220 s/iter total_throughput: 3126.77 samples/s lr: 9.14e-04 [09/26 15:37:43] lb.utils.events INFO: eta: 13:02:32 iteration: 71799/375342 consumed_samples: 73523200 total_loss: 3.945 time: 0.3275 s/iter data_time: 0.2040 s/iter total_throughput: 3126.77 samples/s lr: 9.13e-04 [09/26 15:38:16] lb.utils.events INFO: eta: 12:59:18 iteration: 71899/375342 consumed_samples: 73625600 total_loss: 3.945 time: 0.3275 s/iter data_time: 0.1989 s/iter total_throughput: 3126.74 samples/s lr: 9.13e-04 [09/26 15:38:49] lb.utils.events INFO: eta: 12:56:47 iteration: 71999/375342 consumed_samples: 73728000 total_loss: 3.927 time: 0.3275 s/iter data_time: 0.2087 s/iter total_throughput: 3126.72 samples/s lr: 9.13e-04 [09/26 15:39:21] lb.utils.events INFO: eta: 12:59:40 iteration: 72099/375342 consumed_samples: 73830400 total_loss: 3.952 time: 0.3275 s/iter data_time: 0.2427 s/iter total_throughput: 3126.73 samples/s lr: 9.13e-04 [09/26 15:39:55] lb.utils.events INFO: eta: 13:01:52 iteration: 72199/375342 consumed_samples: 73932800 total_loss: 3.951 time: 0.3275 s/iter data_time: 0.2471 s/iter total_throughput: 3126.63 samples/s lr: 9.12e-04 [09/26 15:40:28] lb.utils.events INFO: eta: 13:07:26 iteration: 72299/375342 consumed_samples: 74035200 total_loss: 3.928 time: 0.3275 s/iter data_time: 0.2150 s/iter total_throughput: 3126.55 samples/s lr: 9.12e-04 [09/26 15:41:02] lb.utils.events INFO: eta: 13:09:03 iteration: 72399/375342 consumed_samples: 74137600 total_loss: 3.926 time: 0.3275 s/iter data_time: 0.2086 s/iter total_throughput: 3126.49 samples/s lr: 9.12e-04 [09/26 15:41:35] lb.utils.events INFO: eta: 13:17:03 iteration: 72499/375342 consumed_samples: 74240000 total_loss: 3.94 time: 0.3275 s/iter data_time: 0.2308 s/iter total_throughput: 3126.43 samples/s lr: 9.12e-04 [09/26 15:42:08] lb.utils.events INFO: eta: 13:19:58 iteration: 72599/375342 consumed_samples: 74342400 total_loss: 3.951 time: 0.3275 s/iter data_time: 0.2055 s/iter total_throughput: 3126.34 samples/s lr: 9.11e-04 [09/26 15:42:41] lb.utils.events INFO: eta: 13:19:55 iteration: 72699/375342 consumed_samples: 74444800 total_loss: 3.933 time: 0.3275 s/iter data_time: 0.2096 s/iter total_throughput: 3126.32 samples/s lr: 9.11e-04 [09/26 15:43:14] lb.utils.events INFO: eta: 13:17:20 iteration: 72799/375342 consumed_samples: 74547200 total_loss: 3.897 time: 0.3275 s/iter data_time: 0.2125 s/iter total_throughput: 3126.26 samples/s lr: 9.11e-04 [09/26 15:43:48] lb.utils.events INFO: eta: 13:14:26 iteration: 72899/375342 consumed_samples: 74649600 total_loss: 3.896 time: 0.3276 s/iter data_time: 0.2149 s/iter total_throughput: 3126.14 samples/s lr: 9.11e-04 [09/26 15:44:21] lb.utils.events INFO: eta: 13:15:01 iteration: 72999/375342 consumed_samples: 74752000 total_loss: 3.891 time: 0.3276 s/iter data_time: 0.2140 s/iter total_throughput: 3126.08 samples/s lr: 9.10e-04 [09/26 15:44:54] lb.utils.events INFO: eta: 13:07:54 iteration: 73099/375342 consumed_samples: 74854400 total_loss: 3.915 time: 0.3276 s/iter data_time: 0.2119 s/iter total_throughput: 3126.04 samples/s lr: 9.10e-04 [09/26 15:45:27] lb.utils.events INFO: eta: 13:02:28 iteration: 73199/375342 consumed_samples: 74956800 total_loss: 3.93 time: 0.3276 s/iter data_time: 0.2065 s/iter total_throughput: 3126.00 samples/s lr: 9.10e-04 [09/26 15:46:01] lb.utils.events INFO: eta: 12:58:55 iteration: 73299/375342 consumed_samples: 75059200 total_loss: 3.935 time: 0.3276 s/iter data_time: 0.2134 s/iter total_throughput: 3125.89 samples/s lr: 9.10e-04 [09/26 15:46:34] lb.utils.events INFO: eta: 12:56:44 iteration: 73399/375342 consumed_samples: 75161600 total_loss: 3.934 time: 0.3276 s/iter data_time: 0.2129 s/iter total_throughput: 3125.80 samples/s lr: 9.09e-04 [09/26 15:47:07] lb.utils.events INFO: eta: 12:53:40 iteration: 73499/375342 consumed_samples: 75264000 total_loss: 3.928 time: 0.3276 s/iter data_time: 0.2054 s/iter total_throughput: 3125.78 samples/s lr: 9.09e-04 [09/26 15:47:40] lb.utils.events INFO: eta: 12:50:58 iteration: 73599/375342 consumed_samples: 75366400 total_loss: 3.932 time: 0.3276 s/iter data_time: 0.2115 s/iter total_throughput: 3125.74 samples/s lr: 9.09e-04 [09/26 15:48:14] lb.utils.events INFO: eta: 12:52:47 iteration: 73699/375342 consumed_samples: 75468800 total_loss: 3.933 time: 0.3276 s/iter data_time: 0.2301 s/iter total_throughput: 3125.69 samples/s lr: 9.09e-04 [09/26 15:48:47] lb.utils.events INFO: eta: 12:52:53 iteration: 73799/375342 consumed_samples: 75571200 total_loss: 3.934 time: 0.3276 s/iter data_time: 0.2125 s/iter total_throughput: 3125.62 samples/s lr: 9.09e-04 [09/26 15:49:20] lb.utils.events INFO: eta: 12:55:05 iteration: 73899/375342 consumed_samples: 75673600 total_loss: 3.899 time: 0.3276 s/iter data_time: 0.2116 s/iter total_throughput: 3125.56 samples/s lr: 9.08e-04 [09/26 15:49:53] lb.utils.events INFO: eta: 12:57:42 iteration: 73999/375342 consumed_samples: 75776000 total_loss: 3.903 time: 0.3276 s/iter data_time: 0.2261 s/iter total_throughput: 3125.52 samples/s lr: 9.08e-04 [09/26 15:50:26] lb.utils.events INFO: eta: 12:59:48 iteration: 74099/375342 consumed_samples: 75878400 total_loss: 3.929 time: 0.3276 s/iter data_time: 0.2238 s/iter total_throughput: 3125.49 samples/s lr: 9.08e-04 [09/26 15:51:00] lb.utils.events INFO: eta: 13:02:21 iteration: 74199/375342 consumed_samples: 75980800 total_loss: 3.926 time: 0.3276 s/iter data_time: 0.2229 s/iter total_throughput: 3125.41 samples/s lr: 9.08e-04 [09/26 15:51:33] lb.utils.events INFO: eta: 13:08:18 iteration: 74299/375342 consumed_samples: 76083200 total_loss: 3.932 time: 0.3276 s/iter data_time: 0.2406 s/iter total_throughput: 3125.39 samples/s lr: 9.07e-04 [09/26 15:52:06] lb.utils.events INFO: eta: 13:14:48 iteration: 74399/375342 consumed_samples: 76185600 total_loss: 3.941 time: 0.3276 s/iter data_time: 0.2085 s/iter total_throughput: 3125.34 samples/s lr: 9.07e-04 [09/26 15:52:39] lb.utils.events INFO: eta: 13:14:10 iteration: 74499/375342 consumed_samples: 76288000 total_loss: 3.939 time: 0.3276 s/iter data_time: 0.2166 s/iter total_throughput: 3125.29 samples/s lr: 9.07e-04 [09/26 15:53:12] lb.utils.events INFO: eta: 13:14:21 iteration: 74599/375342 consumed_samples: 76390400 total_loss: 3.936 time: 0.3277 s/iter data_time: 0.2176 s/iter total_throughput: 3125.25 samples/s lr: 9.07e-04 [09/26 15:53:45] lb.utils.events INFO: eta: 13:14:50 iteration: 74699/375342 consumed_samples: 76492800 total_loss: 3.91 time: 0.3277 s/iter data_time: 0.2243 s/iter total_throughput: 3125.25 samples/s lr: 9.06e-04 [09/26 15:54:18] lb.utils.events INFO: eta: 13:09:56 iteration: 74799/375342 consumed_samples: 76595200 total_loss: 3.906 time: 0.3277 s/iter data_time: 0.2113 s/iter total_throughput: 3125.21 samples/s lr: 9.06e-04 [09/26 15:54:51] lb.utils.events INFO: eta: 13:10:46 iteration: 74899/375342 consumed_samples: 76697600 total_loss: 3.931 time: 0.3277 s/iter data_time: 0.2577 s/iter total_throughput: 3125.17 samples/s lr: 9.06e-04 [09/26 15:55:25] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0074999 [09/26 15:55:25] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/26 15:55:25] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/26 15:55:29] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0821 s/iter. Inference: 0.1530 s/iter. Eval: 0.0020 s/iter. Total: 0.2371 s/iter. ETA=0:00:08 [09/26 15:55:35] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1399 s/iter. Inference: 0.1520 s/iter. Eval: 0.0019 s/iter. Total: 0.2939 s/iter. ETA=0:00:05 [09/26 15:55:40] lb.evaluation.evaluator INFO: Inference done 49152/50000. Dataloading: 0.1290 s/iter. Inference: 0.1503 s/iter. Eval: 0.0020 s/iter. Total: 0.2813 s/iter. ETA=0:00:00 [09/26 15:55:40] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/26 15:55:40] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.463651 (0.000249 s / iter per device, on 8 devices) [09/26 15:55:40] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000133 s / iter per device, on 8 devices) [09/26 15:55:40] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/26 15:55:40] lb.evaluation.utils INFO: copypaste: Acc@1=69.66 [09/26 15:55:40] lb.evaluation.utils INFO: copypaste: Acc@5=89.474 [09/26 15:55:40] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 69.66000, better than last best score 69.27600 @ iteration 69999. [09/26 15:55:40] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/26 15:55:41] lb.utils.events INFO: eta: 13:04:16 iteration: 74999/375342 consumed_samples: 76800000 total_loss: 3.933 time: 0.3277 s/iter data_time: 0.2121 s/iter total_throughput: 3125.04 samples/s lr: 9.06e-04 [09/26 15:56:13] lb.utils.events INFO: eta: 13:02:34 iteration: 75099/375342 consumed_samples: 76902400 total_loss: 3.918 time: 0.3277 s/iter data_time: 0.2421 s/iter total_throughput: 3125.21 samples/s lr: 9.05e-04 [09/26 15:56:46] lb.utils.events INFO: eta: 13:07:53 iteration: 75199/375342 consumed_samples: 77004800 total_loss: 3.914 time: 0.3277 s/iter data_time: 0.2537 s/iter total_throughput: 3125.15 samples/s lr: 9.05e-04 [09/26 15:57:19] lb.utils.events INFO: eta: 13:04:18 iteration: 75299/375342 consumed_samples: 77107200 total_loss: 3.913 time: 0.3277 s/iter data_time: 0.2115 s/iter total_throughput: 3125.09 samples/s lr: 9.05e-04 [09/26 15:57:52] lb.utils.events INFO: eta: 13:02:52 iteration: 75399/375342 consumed_samples: 77209600 total_loss: 3.922 time: 0.3277 s/iter data_time: 0.2218 s/iter total_throughput: 3125.06 samples/s lr: 9.05e-04 [09/26 15:58:25] lb.utils.events INFO: eta: 13:08:48 iteration: 75499/375342 consumed_samples: 77312000 total_loss: 3.918 time: 0.3277 s/iter data_time: 0.2395 s/iter total_throughput: 3125.06 samples/s lr: 9.04e-04 [09/26 15:58:58] lb.utils.events INFO: eta: 13:23:03 iteration: 75599/375342 consumed_samples: 77414400 total_loss: 3.908 time: 0.3277 s/iter data_time: 0.2494 s/iter total_throughput: 3125.03 samples/s lr: 9.04e-04 [09/26 15:59:31] lb.utils.events INFO: eta: 13:27:58 iteration: 75699/375342 consumed_samples: 77516800 total_loss: 3.913 time: 0.3277 s/iter data_time: 0.2291 s/iter total_throughput: 3124.94 samples/s lr: 9.04e-04 [09/26 16:00:04] lb.utils.events INFO: eta: 14:02:10 iteration: 75799/375342 consumed_samples: 77619200 total_loss: 3.887 time: 0.3277 s/iter data_time: 0.2537 s/iter total_throughput: 3124.96 samples/s lr: 9.04e-04 [09/26 16:00:37] lb.utils.events INFO: eta: 16:06:15 iteration: 75899/375342 consumed_samples: 77721600 total_loss: 3.922 time: 0.3277 s/iter data_time: 0.2426 s/iter total_throughput: 3124.94 samples/s lr: 9.03e-04 [09/26 16:01:11] lb.utils.events INFO: eta: 15:19:56 iteration: 75999/375342 consumed_samples: 77824000 total_loss: 3.946 time: 0.3277 s/iter data_time: 0.2223 s/iter total_throughput: 3124.79 samples/s lr: 9.03e-04 [09/26 16:01:44] lb.utils.events INFO: eta: 15:05:43 iteration: 76099/375342 consumed_samples: 77926400 total_loss: 3.927 time: 0.3277 s/iter data_time: 0.2128 s/iter total_throughput: 3124.73 samples/s lr: 9.03e-04 [09/26 16:02:18] lb.utils.events INFO: eta: 13:49:05 iteration: 76199/375342 consumed_samples: 78028800 total_loss: 3.906 time: 0.3277 s/iter data_time: 0.2026 s/iter total_throughput: 3124.63 samples/s lr: 9.03e-04 [09/26 16:02:51] lb.utils.events INFO: eta: 13:36:53 iteration: 76299/375342 consumed_samples: 78131200 total_loss: 3.91 time: 0.3277 s/iter data_time: 0.2101 s/iter total_throughput: 3124.58 samples/s lr: 9.02e-04 [09/26 16:03:24] lb.utils.events INFO: eta: 13:31:14 iteration: 76399/375342 consumed_samples: 78233600 total_loss: 3.883 time: 0.3277 s/iter data_time: 0.2030 s/iter total_throughput: 3124.58 samples/s lr: 9.02e-04 [09/26 16:03:57] lb.utils.events INFO: eta: 13:18:39 iteration: 76499/375342 consumed_samples: 78336000 total_loss: 3.885 time: 0.3277 s/iter data_time: 0.2190 s/iter total_throughput: 3124.56 samples/s lr: 9.02e-04 [09/26 16:04:30] lb.utils.events INFO: eta: 13:05:35 iteration: 76599/375342 consumed_samples: 78438400 total_loss: 3.902 time: 0.3277 s/iter data_time: 0.2051 s/iter total_throughput: 3124.53 samples/s lr: 9.02e-04 [09/26 16:05:03] lb.utils.events INFO: eta: 13:03:03 iteration: 76699/375342 consumed_samples: 78540800 total_loss: 3.896 time: 0.3277 s/iter data_time: 0.2290 s/iter total_throughput: 3124.52 samples/s lr: 9.01e-04 [09/26 16:05:35] lb.utils.events INFO: eta: 13:00:09 iteration: 76799/375342 consumed_samples: 78643200 total_loss: 3.906 time: 0.3277 s/iter data_time: 0.2339 s/iter total_throughput: 3124.52 samples/s lr: 9.01e-04 [09/26 16:06:08] lb.utils.events INFO: eta: 12:58:03 iteration: 76899/375342 consumed_samples: 78745600 total_loss: 3.904 time: 0.3277 s/iter data_time: 0.2371 s/iter total_throughput: 3124.50 samples/s lr: 9.01e-04 [09/26 16:06:41] lb.utils.events INFO: eta: 13:06:42 iteration: 76999/375342 consumed_samples: 78848000 total_loss: 3.913 time: 0.3277 s/iter data_time: 0.2410 s/iter total_throughput: 3124.46 samples/s lr: 9.01e-04 [09/26 16:07:14] lb.utils.events INFO: eta: 13:33:28 iteration: 77099/375342 consumed_samples: 78950400 total_loss: 3.914 time: 0.3277 s/iter data_time: 0.2429 s/iter total_throughput: 3124.44 samples/s lr: 9.00e-04 [09/26 16:07:48] lb.utils.events INFO: eta: 14:01:51 iteration: 77199/375342 consumed_samples: 79052800 total_loss: 3.911 time: 0.3277 s/iter data_time: 0.2265 s/iter total_throughput: 3124.38 samples/s lr: 9.00e-04 [09/26 16:08:21] lb.utils.events INFO: eta: 14:39:42 iteration: 77299/375342 consumed_samples: 79155200 total_loss: 3.918 time: 0.3278 s/iter data_time: 0.2118 s/iter total_throughput: 3124.27 samples/s lr: 9.00e-04 [09/26 16:08:54] lb.utils.events INFO: eta: 14:43:15 iteration: 77399/375342 consumed_samples: 79257600 total_loss: 3.923 time: 0.3278 s/iter data_time: 0.2080 s/iter total_throughput: 3124.24 samples/s lr: 9.00e-04 [09/26 16:09:27] lb.utils.events INFO: eta: 14:28:02 iteration: 77499/375342 consumed_samples: 79360000 total_loss: 3.921 time: 0.3278 s/iter data_time: 0.2016 s/iter total_throughput: 3124.27 samples/s lr: 8.99e-04 [09/26 16:10:00] lb.utils.events INFO: eta: 14:22:15 iteration: 77599/375342 consumed_samples: 79462400 total_loss: 3.906 time: 0.3278 s/iter data_time: 0.2018 s/iter total_throughput: 3124.21 samples/s lr: 8.99e-04 [09/26 16:10:33] lb.utils.events INFO: eta: 13:52:33 iteration: 77699/375342 consumed_samples: 79564800 total_loss: 3.917 time: 0.3278 s/iter data_time: 0.2130 s/iter total_throughput: 3124.17 samples/s lr: 8.99e-04 [09/26 16:11:06] lb.utils.events INFO: eta: 13:31:13 iteration: 77799/375342 consumed_samples: 79667200 total_loss: 3.932 time: 0.3278 s/iter data_time: 0.2280 s/iter total_throughput: 3124.15 samples/s lr: 8.99e-04 [09/26 16:11:39] lb.utils.events INFO: eta: 13:11:31 iteration: 77899/375342 consumed_samples: 79769600 total_loss: 3.915 time: 0.3278 s/iter data_time: 0.2039 s/iter total_throughput: 3124.15 samples/s lr: 8.98e-04 [09/26 16:12:12] lb.utils.events INFO: eta: 13:02:16 iteration: 77999/375342 consumed_samples: 79872000 total_loss: 3.913 time: 0.3278 s/iter data_time: 0.1988 s/iter total_throughput: 3124.13 samples/s lr: 8.98e-04 [09/26 16:12:45] lb.utils.events INFO: eta: 12:51:41 iteration: 78099/375342 consumed_samples: 79974400 total_loss: 3.895 time: 0.3278 s/iter data_time: 0.2206 s/iter total_throughput: 3124.12 samples/s lr: 8.98e-04 [09/26 16:13:18] lb.utils.events INFO: eta: 12:48:21 iteration: 78199/375342 consumed_samples: 80076800 total_loss: 3.867 time: 0.3278 s/iter data_time: 0.2351 s/iter total_throughput: 3124.06 samples/s lr: 8.98e-04 [09/26 16:13:51] lb.utils.events INFO: eta: 12:45:17 iteration: 78299/375342 consumed_samples: 80179200 total_loss: 3.879 time: 0.3278 s/iter data_time: 0.2118 s/iter total_throughput: 3124.03 samples/s lr: 8.97e-04 [09/26 16:14:24] lb.utils.events INFO: eta: 12:44:20 iteration: 78399/375342 consumed_samples: 80281600 total_loss: 3.91 time: 0.3278 s/iter data_time: 0.2031 s/iter total_throughput: 3124.04 samples/s lr: 8.97e-04 [09/26 16:14:57] lb.utils.events INFO: eta: 12:43:11 iteration: 78499/375342 consumed_samples: 80384000 total_loss: 3.913 time: 0.3278 s/iter data_time: 0.2015 s/iter total_throughput: 3123.99 samples/s lr: 8.97e-04 [09/26 16:15:30] lb.utils.events INFO: eta: 12:41:41 iteration: 78599/375342 consumed_samples: 80486400 total_loss: 3.929 time: 0.3278 s/iter data_time: 0.2136 s/iter total_throughput: 3124.02 samples/s lr: 8.97e-04 [09/26 16:16:02] lb.utils.events INFO: eta: 12:48:52 iteration: 78699/375342 consumed_samples: 80588800 total_loss: 3.911 time: 0.3278 s/iter data_time: 0.2642 s/iter total_throughput: 3124.05 samples/s lr: 8.96e-04 [09/26 16:16:35] lb.utils.events INFO: eta: 12:58:44 iteration: 78799/375342 consumed_samples: 80691200 total_loss: 3.902 time: 0.3278 s/iter data_time: 0.2315 s/iter total_throughput: 3124.00 samples/s lr: 8.96e-04 [09/26 16:17:09] lb.utils.events INFO: eta: 13:05:34 iteration: 78899/375342 consumed_samples: 80793600 total_loss: 3.898 time: 0.3278 s/iter data_time: 0.2405 s/iter total_throughput: 3123.94 samples/s lr: 8.96e-04 [09/26 16:17:42] lb.utils.events INFO: eta: 13:10:23 iteration: 78999/375342 consumed_samples: 80896000 total_loss: 3.91 time: 0.3278 s/iter data_time: 0.2160 s/iter total_throughput: 3123.87 samples/s lr: 8.96e-04 [09/26 16:18:15] lb.utils.events INFO: eta: 13:08:57 iteration: 79099/375342 consumed_samples: 80998400 total_loss: 3.902 time: 0.3278 s/iter data_time: 0.2328 s/iter total_throughput: 3123.88 samples/s lr: 8.95e-04 [09/26 16:18:48] lb.utils.events INFO: eta: 13:09:09 iteration: 79199/375342 consumed_samples: 81100800 total_loss: 3.889 time: 0.3278 s/iter data_time: 0.2253 s/iter total_throughput: 3123.85 samples/s lr: 8.95e-04 [09/26 16:19:20] lb.utils.events INFO: eta: 13:18:10 iteration: 79299/375342 consumed_samples: 81203200 total_loss: 3.886 time: 0.3278 s/iter data_time: 0.2201 s/iter total_throughput: 3123.87 samples/s lr: 8.95e-04 [09/26 16:19:53] lb.utils.events INFO: eta: 13:29:29 iteration: 79399/375342 consumed_samples: 81305600 total_loss: 3.894 time: 0.3278 s/iter data_time: 0.2220 s/iter total_throughput: 3123.88 samples/s lr: 8.95e-04 [09/26 16:20:27] lb.utils.events INFO: eta: 13:41:36 iteration: 79499/375342 consumed_samples: 81408000 total_loss: 3.92 time: 0.3278 s/iter data_time: 0.2149 s/iter total_throughput: 3123.75 samples/s lr: 8.94e-04 [09/26 16:21:00] lb.utils.events INFO: eta: 13:37:41 iteration: 79599/375342 consumed_samples: 81510400 total_loss: 3.908 time: 0.3278 s/iter data_time: 0.2098 s/iter total_throughput: 3123.72 samples/s lr: 8.94e-04 [09/26 16:21:33] lb.utils.events INFO: eta: 13:11:51 iteration: 79699/375342 consumed_samples: 81612800 total_loss: 3.875 time: 0.3278 s/iter data_time: 0.2040 s/iter total_throughput: 3123.70 samples/s lr: 8.94e-04 [09/26 16:22:05] lb.utils.events INFO: eta: 12:59:13 iteration: 79799/375342 consumed_samples: 81715200 total_loss: 3.872 time: 0.3278 s/iter data_time: 0.1988 s/iter total_throughput: 3123.73 samples/s lr: 8.94e-04 [09/26 16:22:38] lb.utils.events INFO: eta: 12:52:48 iteration: 79899/375342 consumed_samples: 81817600 total_loss: 3.892 time: 0.3278 s/iter data_time: 0.2086 s/iter total_throughput: 3123.71 samples/s lr: 8.93e-04 [09/26 16:23:11] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0079999 [09/26 16:23:12] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/26 16:23:12] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/26 16:23:16] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0827 s/iter. Inference: 0.1511 s/iter. Eval: 0.0021 s/iter. Total: 0.2359 s/iter. ETA=0:00:08 [09/26 16:23:22] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1411 s/iter. Inference: 0.1507 s/iter. Eval: 0.0021 s/iter. Total: 0.2939 s/iter. ETA=0:00:05 [09/26 16:23:27] lb.evaluation.evaluator INFO: Inference done 50000/50000. Dataloading: 0.1292 s/iter. Inference: 0.1508 s/iter. Eval: 0.0020 s/iter. Total: 0.2821 s/iter. ETA=0:00:00 [09/26 16:23:27] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/26 16:23:27] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.414569 (0.000248 s / iter per device, on 8 devices) [09/26 16:23:27] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000133 s / iter per device, on 8 devices) [09/26 16:23:27] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/26 16:23:27] lb.evaluation.utils INFO: copypaste: Acc@1=69.916 [09/26 16:23:27] lb.evaluation.utils INFO: copypaste: Acc@5=89.782 [09/26 16:23:27] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 69.91600, better than last best score 69.66000 @ iteration 74999. [09/26 16:23:27] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/26 16:23:28] lb.utils.events INFO: eta: 12:47:40 iteration: 79999/375342 consumed_samples: 81920000 total_loss: 3.875 time: 0.3278 s/iter data_time: 0.2139 s/iter total_throughput: 3123.70 samples/s lr: 8.93e-04 [09/26 16:23:59] lb.utils.events INFO: eta: 12:50:15 iteration: 80099/375342 consumed_samples: 82022400 total_loss: 3.882 time: 0.3278 s/iter data_time: 0.2436 s/iter total_throughput: 3123.86 samples/s lr: 8.93e-04 [09/26 16:24:32] lb.utils.events INFO: eta: 12:49:05 iteration: 80199/375342 consumed_samples: 82124800 total_loss: 3.891 time: 0.3278 s/iter data_time: 0.2150 s/iter total_throughput: 3123.82 samples/s lr: 8.93e-04 [09/26 16:25:05] lb.utils.events INFO: eta: 12:44:05 iteration: 80299/375342 consumed_samples: 82227200 total_loss: 3.886 time: 0.3278 s/iter data_time: 0.2122 s/iter total_throughput: 3123.83 samples/s lr: 8.92e-04 [09/26 16:25:38] lb.utils.events INFO: eta: 12:43:50 iteration: 80399/375342 consumed_samples: 82329600 total_loss: 3.898 time: 0.3278 s/iter data_time: 0.2174 s/iter total_throughput: 3123.81 samples/s lr: 8.92e-04 [09/26 16:26:10] lb.utils.events INFO: eta: 12:45:26 iteration: 80499/375342 consumed_samples: 82432000 total_loss: 3.898 time: 0.3278 s/iter data_time: 0.2213 s/iter total_throughput: 3123.86 samples/s lr: 8.92e-04 [09/26 16:26:43] lb.utils.events INFO: eta: 12:47:49 iteration: 80599/375342 consumed_samples: 82534400 total_loss: 3.897 time: 0.3278 s/iter data_time: 0.2103 s/iter total_throughput: 3123.86 samples/s lr: 8.92e-04 [09/26 16:27:16] lb.utils.events INFO: eta: 12:46:46 iteration: 80699/375342 consumed_samples: 82636800 total_loss: 3.896 time: 0.3278 s/iter data_time: 0.2061 s/iter total_throughput: 3123.88 samples/s lr: 8.91e-04 [09/26 16:27:48] lb.utils.events INFO: eta: 12:44:37 iteration: 80799/375342 consumed_samples: 82739200 total_loss: 3.904 time: 0.3278 s/iter data_time: 0.2033 s/iter total_throughput: 3123.90 samples/s lr: 8.91e-04 [09/26 16:28:21] lb.utils.events INFO: eta: 12:46:30 iteration: 80899/375342 consumed_samples: 82841600 total_loss: 3.898 time: 0.3278 s/iter data_time: 0.2168 s/iter total_throughput: 3123.90 samples/s lr: 8.91e-04 [09/26 16:28:54] lb.utils.events INFO: eta: 12:47:40 iteration: 80999/375342 consumed_samples: 82944000 total_loss: 3.882 time: 0.3278 s/iter data_time: 0.2269 s/iter total_throughput: 3123.93 samples/s lr: 8.91e-04 [09/26 16:29:27] lb.utils.events INFO: eta: 12:52:31 iteration: 81099/375342 consumed_samples: 83046400 total_loss: 3.885 time: 0.3278 s/iter data_time: 0.2605 s/iter total_throughput: 3123.91 samples/s lr: 8.90e-04 [09/26 16:30:00] lb.utils.events INFO: eta: 13:00:25 iteration: 81199/375342 consumed_samples: 83148800 total_loss: 3.888 time: 0.3278 s/iter data_time: 0.2414 s/iter total_throughput: 3123.88 samples/s lr: 8.90e-04 [09/26 16:30:33] lb.utils.events INFO: eta: 13:06:50 iteration: 81299/375342 consumed_samples: 83251200 total_loss: 3.888 time: 0.3278 s/iter data_time: 0.2233 s/iter total_throughput: 3123.84 samples/s lr: 8.90e-04 [09/26 16:31:05] lb.utils.events INFO: eta: 13:16:32 iteration: 81399/375342 consumed_samples: 83353600 total_loss: 3.888 time: 0.3278 s/iter data_time: 0.2259 s/iter total_throughput: 3123.88 samples/s lr: 8.89e-04 [09/26 16:31:38] lb.utils.events INFO: eta: 13:17:44 iteration: 81499/375342 consumed_samples: 83456000 total_loss: 3.879 time: 0.3278 s/iter data_time: 0.2359 s/iter total_throughput: 3123.92 samples/s lr: 8.89e-04 [09/26 16:32:10] lb.utils.events INFO: eta: 13:22:02 iteration: 81599/375342 consumed_samples: 83558400 total_loss: 3.868 time: 0.3278 s/iter data_time: 0.2035 s/iter total_throughput: 3123.98 samples/s lr: 8.89e-04 [09/26 16:32:44] lb.utils.events INFO: eta: 13:33:33 iteration: 81699/375342 consumed_samples: 83660800 total_loss: 3.871 time: 0.3278 s/iter data_time: 0.2364 s/iter total_throughput: 3123.88 samples/s lr: 8.89e-04 [09/26 16:33:17] lb.utils.events INFO: eta: 13:41:21 iteration: 81799/375342 consumed_samples: 83763200 total_loss: 3.872 time: 0.3278 s/iter data_time: 0.2289 s/iter total_throughput: 3123.86 samples/s lr: 8.88e-04 [09/26 16:33:49] lb.utils.events INFO: eta: 13:40:52 iteration: 81899/375342 consumed_samples: 83865600 total_loss: 3.889 time: 0.3278 s/iter data_time: 0.2128 s/iter total_throughput: 3123.90 samples/s lr: 8.88e-04 [09/26 16:34:22] lb.utils.events INFO: eta: 13:45:52 iteration: 81999/375342 consumed_samples: 83968000 total_loss: 3.902 time: 0.3278 s/iter data_time: 0.2446 s/iter total_throughput: 3123.83 samples/s lr: 8.88e-04 [09/26 16:34:56] lb.utils.events INFO: eta: 13:40:52 iteration: 82099/375342 consumed_samples: 84070400 total_loss: 3.891 time: 0.3278 s/iter data_time: 0.2388 s/iter total_throughput: 3123.78 samples/s lr: 8.88e-04 [09/26 16:35:28] lb.utils.events INFO: eta: 13:35:59 iteration: 82199/375342 consumed_samples: 84172800 total_loss: 3.875 time: 0.3278 s/iter data_time: 0.2162 s/iter total_throughput: 3123.80 samples/s lr: 8.87e-04 [09/26 16:36:01] lb.utils.events INFO: eta: 13:25:01 iteration: 82299/375342 consumed_samples: 84275200 total_loss: 3.883 time: 0.3278 s/iter data_time: 0.2039 s/iter total_throughput: 3123.80 samples/s lr: 8.87e-04 [09/26 16:36:34] lb.utils.events INFO: eta: 13:15:35 iteration: 82399/375342 consumed_samples: 84377600 total_loss: 3.884 time: 0.3278 s/iter data_time: 0.2173 s/iter total_throughput: 3123.82 samples/s lr: 8.87e-04 [09/26 16:37:06] lb.utils.events INFO: eta: 13:14:43 iteration: 82499/375342 consumed_samples: 84480000 total_loss: 3.883 time: 0.3278 s/iter data_time: 0.2250 s/iter total_throughput: 3123.85 samples/s lr: 8.87e-04 [09/26 16:37:39] lb.utils.events INFO: eta: 13:15:51 iteration: 82599/375342 consumed_samples: 84582400 total_loss: 3.883 time: 0.3278 s/iter data_time: 0.2251 s/iter total_throughput: 3123.82 samples/s lr: 8.86e-04 [09/26 16:38:12] lb.utils.events INFO: eta: 13:18:06 iteration: 82699/375342 consumed_samples: 84684800 total_loss: 3.87 time: 0.3278 s/iter data_time: 0.2203 s/iter total_throughput: 3123.80 samples/s lr: 8.86e-04 [09/26 16:38:45] lb.utils.events INFO: eta: 13:19:27 iteration: 82799/375342 consumed_samples: 84787200 total_loss: 3.864 time: 0.3278 s/iter data_time: 0.2174 s/iter total_throughput: 3123.83 samples/s lr: 8.86e-04 [09/26 16:39:17] lb.utils.events INFO: eta: 13:25:22 iteration: 82899/375342 consumed_samples: 84889600 total_loss: 3.882 time: 0.3278 s/iter data_time: 0.2251 s/iter total_throughput: 3123.85 samples/s lr: 8.86e-04 [09/26 16:39:50] lb.utils.events INFO: eta: 13:35:11 iteration: 82999/375342 consumed_samples: 84992000 total_loss: 3.886 time: 0.3278 s/iter data_time: 0.2324 s/iter total_throughput: 3123.87 samples/s lr: 8.85e-04 [09/26 16:40:23] lb.utils.events INFO: eta: 13:20:54 iteration: 83099/375342 consumed_samples: 85094400 total_loss: 3.871 time: 0.3278 s/iter data_time: 0.2057 s/iter total_throughput: 3123.86 samples/s lr: 8.85e-04 [09/26 16:40:55] lb.utils.events INFO: eta: 13:24:16 iteration: 83199/375342 consumed_samples: 85196800 total_loss: 3.872 time: 0.3278 s/iter data_time: 0.2410 s/iter total_throughput: 3123.91 samples/s lr: 8.85e-04 [09/26 16:41:28] lb.utils.events INFO: eta: 13:48:04 iteration: 83299/375342 consumed_samples: 85299200 total_loss: 3.901 time: 0.3278 s/iter data_time: 0.2242 s/iter total_throughput: 3123.97 samples/s lr: 8.84e-04 [09/26 16:42:00] lb.utils.events INFO: eta: 15:02:25 iteration: 83399/375342 consumed_samples: 85401600 total_loss: 3.899 time: 0.3278 s/iter data_time: 0.2367 s/iter total_throughput: 3124.04 samples/s lr: 8.84e-04 [09/26 16:42:33] lb.utils.events INFO: eta: 14:13:25 iteration: 83499/375342 consumed_samples: 85504000 total_loss: 3.882 time: 0.3278 s/iter data_time: 0.2142 s/iter total_throughput: 3123.95 samples/s lr: 8.84e-04 [09/26 16:43:06] lb.utils.events INFO: eta: 13:37:41 iteration: 83599/375342 consumed_samples: 85606400 total_loss: 3.873 time: 0.3278 s/iter data_time: 0.2170 s/iter total_throughput: 3124.00 samples/s lr: 8.84e-04 [09/26 16:43:38] lb.utils.events INFO: eta: 13:46:39 iteration: 83699/375342 consumed_samples: 85708800 total_loss: 3.884 time: 0.3278 s/iter data_time: 0.2111 s/iter total_throughput: 3124.05 samples/s lr: 8.83e-04 [09/26 16:44:10] lb.utils.events INFO: eta: 14:08:37 iteration: 83799/375342 consumed_samples: 85811200 total_loss: 3.892 time: 0.3278 s/iter data_time: 0.2183 s/iter total_throughput: 3124.11 samples/s lr: 8.83e-04 [09/26 16:44:43] lb.utils.events INFO: eta: 13:59:19 iteration: 83899/375342 consumed_samples: 85913600 total_loss: 3.862 time: 0.3278 s/iter data_time: 0.2391 s/iter total_throughput: 3124.15 samples/s lr: 8.83e-04 [09/26 16:45:15] lb.utils.events INFO: eta: 14:51:03 iteration: 83999/375342 consumed_samples: 86016000 total_loss: 3.876 time: 0.3278 s/iter data_time: 0.2315 s/iter total_throughput: 3124.20 samples/s lr: 8.83e-04 [09/26 16:45:48] lb.utils.events INFO: eta: 14:46:00 iteration: 84099/375342 consumed_samples: 86118400 total_loss: 3.865 time: 0.3278 s/iter data_time: 0.2129 s/iter total_throughput: 3124.15 samples/s lr: 8.82e-04 [09/26 16:46:21] lb.utils.events INFO: eta: 14:49:21 iteration: 84199/375342 consumed_samples: 86220800 total_loss: 3.854 time: 0.3278 s/iter data_time: 0.2537 s/iter total_throughput: 3124.14 samples/s lr: 8.82e-04 [09/26 16:46:54] lb.utils.events INFO: eta: 15:51:54 iteration: 84299/375342 consumed_samples: 86323200 total_loss: 3.873 time: 0.3278 s/iter data_time: 0.2560 s/iter total_throughput: 3124.09 samples/s lr: 8.82e-04 [09/26 16:47:28] lb.utils.events INFO: eta: 15:51:34 iteration: 84399/375342 consumed_samples: 86425600 total_loss: 3.873 time: 0.3278 s/iter data_time: 0.2409 s/iter total_throughput: 3124.05 samples/s lr: 8.82e-04 [09/26 16:48:01] lb.utils.events INFO: eta: 16:42:11 iteration: 84499/375342 consumed_samples: 86528000 total_loss: 3.866 time: 0.3278 s/iter data_time: 0.2253 s/iter total_throughput: 3123.99 samples/s lr: 8.81e-04 [09/26 16:48:34] lb.utils.events INFO: eta: 17:32:15 iteration: 84599/375342 consumed_samples: 86630400 total_loss: 3.882 time: 0.3278 s/iter data_time: 0.2240 s/iter total_throughput: 3123.89 samples/s lr: 8.81e-04 [09/26 16:49:08] lb.utils.events INFO: eta: 16:03:47 iteration: 84699/375342 consumed_samples: 86732800 total_loss: 3.893 time: 0.3278 s/iter data_time: 0.2138 s/iter total_throughput: 3123.78 samples/s lr: 8.81e-04 [09/26 16:49:41] lb.utils.events INFO: eta: 14:33:25 iteration: 84799/375342 consumed_samples: 86835200 total_loss: 3.89 time: 0.3278 s/iter data_time: 0.2103 s/iter total_throughput: 3123.74 samples/s lr: 8.80e-04 [09/26 16:50:14] lb.utils.events INFO: eta: 13:42:45 iteration: 84899/375342 consumed_samples: 86937600 total_loss: 3.899 time: 0.3278 s/iter data_time: 0.2128 s/iter total_throughput: 3123.73 samples/s lr: 8.80e-04 [09/26 16:50:48] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0084999 [09/26 16:50:48] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/26 16:50:48] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/26 16:50:53] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0905 s/iter. Inference: 0.1471 s/iter. Eval: 0.0020 s/iter. Total: 0.2396 s/iter. ETA=0:00:08 [09/26 16:50:58] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1489 s/iter. Inference: 0.1507 s/iter. Eval: 0.0021 s/iter. Total: 0.3017 s/iter. ETA=0:00:05 [09/26 16:51:04] lb.evaluation.evaluator INFO: Inference done 50000/50000. Dataloading: 0.1342 s/iter. Inference: 0.1515 s/iter. Eval: 0.0021 s/iter. Total: 0.2879 s/iter. ETA=0:00:00 [09/26 16:51:04] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/26 16:51:04] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.671172 (0.000253 s / iter per device, on 8 devices) [09/26 16:51:04] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000133 s / iter per device, on 8 devices) [09/26 16:51:04] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/26 16:51:04] lb.evaluation.utils INFO: copypaste: Acc@1=70.07600000000001 [09/26 16:51:04] lb.evaluation.utils INFO: copypaste: Acc@5=89.86 [09/26 16:51:04] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 70.07600, better than last best score 69.91600 @ iteration 79999. [09/26 16:51:04] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/26 16:51:05] lb.utils.events INFO: eta: 13:07:35 iteration: 84999/375342 consumed_samples: 87040000 total_loss: 3.901 time: 0.3278 s/iter data_time: 0.2077 s/iter total_throughput: 3123.66 samples/s lr: 8.80e-04 [09/26 16:51:36] lb.utils.events INFO: eta: 13:15:51 iteration: 85099/375342 consumed_samples: 87142400 total_loss: 3.877 time: 0.3278 s/iter data_time: 0.2552 s/iter total_throughput: 3123.76 samples/s lr: 8.80e-04 [09/26 16:52:09] lb.utils.events INFO: eta: 13:19:36 iteration: 85199/375342 consumed_samples: 87244800 total_loss: 3.85 time: 0.3278 s/iter data_time: 0.2310 s/iter total_throughput: 3123.77 samples/s lr: 8.79e-04 [09/26 16:52:43] lb.utils.events INFO: eta: 13:09:54 iteration: 85299/375342 consumed_samples: 87347200 total_loss: 3.868 time: 0.3278 s/iter data_time: 0.2465 s/iter total_throughput: 3123.69 samples/s lr: 8.79e-04 [09/26 16:53:16] lb.utils.events INFO: eta: 12:56:05 iteration: 85399/375342 consumed_samples: 87449600 total_loss: 3.873 time: 0.3278 s/iter data_time: 0.2268 s/iter total_throughput: 3123.65 samples/s lr: 8.79e-04 [09/26 16:53:49] lb.utils.events INFO: eta: 12:58:22 iteration: 85499/375342 consumed_samples: 87552000 total_loss: 3.889 time: 0.3278 s/iter data_time: 0.2482 s/iter total_throughput: 3123.62 samples/s lr: 8.79e-04 [09/26 16:54:22] lb.utils.events INFO: eta: 12:55:33 iteration: 85599/375342 consumed_samples: 87654400 total_loss: 3.886 time: 0.3278 s/iter data_time: 0.2060 s/iter total_throughput: 3123.54 samples/s lr: 8.78e-04 [09/26 16:54:56] lb.utils.events INFO: eta: 12:50:12 iteration: 85699/375342 consumed_samples: 87756800 total_loss: 3.879 time: 0.3278 s/iter data_time: 0.2083 s/iter total_throughput: 3123.47 samples/s lr: 8.78e-04 [09/26 16:55:29] lb.utils.events INFO: eta: 12:52:22 iteration: 85799/375342 consumed_samples: 87859200 total_loss: 3.895 time: 0.3278 s/iter data_time: 0.2092 s/iter total_throughput: 3123.45 samples/s lr: 8.78e-04 [09/26 16:56:02] lb.utils.events INFO: eta: 12:52:06 iteration: 85899/375342 consumed_samples: 87961600 total_loss: 3.896 time: 0.3278 s/iter data_time: 0.2064 s/iter total_throughput: 3123.43 samples/s lr: 8.77e-04 [09/26 16:56:35] lb.utils.events INFO: eta: 12:54:47 iteration: 85999/375342 consumed_samples: 88064000 total_loss: 3.881 time: 0.3279 s/iter data_time: 0.2141 s/iter total_throughput: 3123.36 samples/s lr: 8.77e-04 [09/26 16:57:09] lb.utils.events INFO: eta: 12:44:43 iteration: 86099/375342 consumed_samples: 88166400 total_loss: 3.894 time: 0.3279 s/iter data_time: 0.2146 s/iter total_throughput: 3123.28 samples/s lr: 8.77e-04 [09/26 16:57:42] lb.utils.events INFO: eta: 12:36:00 iteration: 86199/375342 consumed_samples: 88268800 total_loss: 3.865 time: 0.3279 s/iter data_time: 0.2069 s/iter total_throughput: 3123.24 samples/s lr: 8.77e-04 [09/26 16:58:15] lb.utils.events INFO: eta: 12:29:16 iteration: 86299/375342 consumed_samples: 88371200 total_loss: 3.863 time: 0.3279 s/iter data_time: 0.2075 s/iter total_throughput: 3123.19 samples/s lr: 8.76e-04 [09/26 16:58:49] lb.utils.events INFO: eta: 12:28:18 iteration: 86399/375342 consumed_samples: 88473600 total_loss: 3.879 time: 0.3279 s/iter data_time: 0.2137 s/iter total_throughput: 3123.11 samples/s lr: 8.76e-04 [09/26 16:59:22] lb.utils.events INFO: eta: 12:22:59 iteration: 86499/375342 consumed_samples: 88576000 total_loss: 3.872 time: 0.3279 s/iter data_time: 0.2128 s/iter total_throughput: 3123.08 samples/s lr: 8.76e-04 [09/26 16:59:55] lb.utils.events INFO: eta: 12:21:18 iteration: 86599/375342 consumed_samples: 88678400 total_loss: 3.861 time: 0.3279 s/iter data_time: 0.2100 s/iter total_throughput: 3122.98 samples/s lr: 8.76e-04 [09/26 17:00:29] lb.utils.events INFO: eta: 12:22:30 iteration: 86699/375342 consumed_samples: 88780800 total_loss: 3.867 time: 0.3279 s/iter data_time: 0.2061 s/iter total_throughput: 3122.94 samples/s lr: 8.75e-04 [09/26 17:01:01] lb.utils.events INFO: eta: 12:21:46 iteration: 86799/375342 consumed_samples: 88883200 total_loss: 3.895 time: 0.3279 s/iter data_time: 0.1984 s/iter total_throughput: 3122.93 samples/s lr: 8.75e-04 [09/26 17:01:35] lb.utils.events INFO: eta: 12:20:08 iteration: 86899/375342 consumed_samples: 88985600 total_loss: 3.885 time: 0.3279 s/iter data_time: 0.2033 s/iter total_throughput: 3122.85 samples/s lr: 8.75e-04 [09/26 17:02:09] lb.utils.events INFO: eta: 12:18:36 iteration: 86999/375342 consumed_samples: 89088000 total_loss: 3.866 time: 0.3279 s/iter data_time: 0.2092 s/iter total_throughput: 3122.74 samples/s lr: 8.74e-04 [09/26 17:02:42] lb.utils.events INFO: eta: 12:18:08 iteration: 87099/375342 consumed_samples: 89190400 total_loss: 3.841 time: 0.3279 s/iter data_time: 0.2135 s/iter total_throughput: 3122.69 samples/s lr: 8.74e-04 [09/26 17:03:15] lb.utils.events INFO: eta: 12:17:58 iteration: 87199/375342 consumed_samples: 89292800 total_loss: 3.858 time: 0.3279 s/iter data_time: 0.2138 s/iter total_throughput: 3122.71 samples/s lr: 8.74e-04 [09/26 17:03:48] lb.utils.events INFO: eta: 12:19:49 iteration: 87299/375342 consumed_samples: 89395200 total_loss: 3.863 time: 0.3279 s/iter data_time: 0.2120 s/iter total_throughput: 3122.67 samples/s lr: 8.74e-04 [09/26 17:04:21] lb.utils.events INFO: eta: 12:21:18 iteration: 87399/375342 consumed_samples: 89497600 total_loss: 3.863 time: 0.3279 s/iter data_time: 0.2312 s/iter total_throughput: 3122.65 samples/s lr: 8.73e-04 [09/26 17:04:54] lb.utils.events INFO: eta: 12:22:57 iteration: 87499/375342 consumed_samples: 89600000 total_loss: 3.871 time: 0.3279 s/iter data_time: 0.2082 s/iter total_throughput: 3122.63 samples/s lr: 8.73e-04 [09/26 17:05:27] lb.utils.events INFO: eta: 12:28:59 iteration: 87599/375342 consumed_samples: 89702400 total_loss: 3.861 time: 0.3279 s/iter data_time: 0.2190 s/iter total_throughput: 3122.60 samples/s lr: 8.73e-04 [09/26 17:06:00] lb.utils.events INFO: eta: 12:32:04 iteration: 87699/375342 consumed_samples: 89804800 total_loss: 3.841 time: 0.3279 s/iter data_time: 0.2236 s/iter total_throughput: 3122.55 samples/s lr: 8.73e-04 [09/26 17:06:33] lb.utils.events INFO: eta: 12:33:11 iteration: 87799/375342 consumed_samples: 89907200 total_loss: 3.852 time: 0.3279 s/iter data_time: 0.2096 s/iter total_throughput: 3122.51 samples/s lr: 8.72e-04 [09/26 17:07:07] lb.utils.events INFO: eta: 12:33:54 iteration: 87899/375342 consumed_samples: 90009600 total_loss: 3.867 time: 0.3279 s/iter data_time: 0.2006 s/iter total_throughput: 3122.46 samples/s lr: 8.72e-04 [09/26 17:07:39] lb.utils.events INFO: eta: 12:39:17 iteration: 87999/375342 consumed_samples: 90112000 total_loss: 3.865 time: 0.3279 s/iter data_time: 0.2205 s/iter total_throughput: 3122.47 samples/s lr: 8.72e-04 [09/26 17:08:13] lb.utils.events INFO: eta: 12:47:19 iteration: 88099/375342 consumed_samples: 90214400 total_loss: 3.843 time: 0.3280 s/iter data_time: 0.2310 s/iter total_throughput: 3122.40 samples/s lr: 8.71e-04 [09/26 17:08:46] lb.utils.events INFO: eta: 12:47:01 iteration: 88199/375342 consumed_samples: 90316800 total_loss: 3.866 time: 0.3280 s/iter data_time: 0.2019 s/iter total_throughput: 3122.34 samples/s lr: 8.71e-04 [09/26 17:09:19] lb.utils.events INFO: eta: 12:46:22 iteration: 88299/375342 consumed_samples: 90419200 total_loss: 3.87 time: 0.3280 s/iter data_time: 0.2101 s/iter total_throughput: 3122.33 samples/s lr: 8.71e-04 [09/26 17:09:52] lb.utils.events INFO: eta: 12:40:26 iteration: 88399/375342 consumed_samples: 90521600 total_loss: 3.856 time: 0.3280 s/iter data_time: 0.2007 s/iter total_throughput: 3122.31 samples/s lr: 8.71e-04 [09/26 17:10:25] lb.utils.events INFO: eta: 12:38:00 iteration: 88499/375342 consumed_samples: 90624000 total_loss: 3.857 time: 0.3280 s/iter data_time: 0.2103 s/iter total_throughput: 3122.33 samples/s lr: 8.70e-04 [09/26 17:10:58] lb.utils.events INFO: eta: 12:31:21 iteration: 88599/375342 consumed_samples: 90726400 total_loss: 3.866 time: 0.3280 s/iter data_time: 0.2023 s/iter total_throughput: 3122.28 samples/s lr: 8.70e-04 [09/26 17:11:31] lb.utils.events INFO: eta: 12:27:36 iteration: 88699/375342 consumed_samples: 90828800 total_loss: 3.879 time: 0.3280 s/iter data_time: 0.2279 s/iter total_throughput: 3122.25 samples/s lr: 8.70e-04 [09/26 17:12:04] lb.utils.events INFO: eta: 12:25:17 iteration: 88799/375342 consumed_samples: 90931200 total_loss: 3.872 time: 0.3280 s/iter data_time: 0.2169 s/iter total_throughput: 3122.21 samples/s lr: 8.69e-04 [09/26 17:12:37] lb.utils.events INFO: eta: 12:27:00 iteration: 88899/375342 consumed_samples: 91033600 total_loss: 3.845 time: 0.3280 s/iter data_time: 0.2034 s/iter total_throughput: 3122.18 samples/s lr: 8.69e-04 [09/26 17:13:10] lb.utils.events INFO: eta: 12:26:54 iteration: 88999/375342 consumed_samples: 91136000 total_loss: 3.86 time: 0.3280 s/iter data_time: 0.2063 s/iter total_throughput: 3122.15 samples/s lr: 8.69e-04 [09/26 17:13:43] lb.utils.events INFO: eta: 12:22:23 iteration: 89099/375342 consumed_samples: 91238400 total_loss: 3.871 time: 0.3280 s/iter data_time: 0.2079 s/iter total_throughput: 3122.12 samples/s lr: 8.69e-04 [09/26 17:14:16] lb.utils.events INFO: eta: 12:20:44 iteration: 89199/375342 consumed_samples: 91340800 total_loss: 3.862 time: 0.3280 s/iter data_time: 0.2012 s/iter total_throughput: 3122.11 samples/s lr: 8.68e-04 [09/26 17:14:50] lb.utils.events INFO: eta: 12:21:57 iteration: 89299/375342 consumed_samples: 91443200 total_loss: 3.852 time: 0.3280 s/iter data_time: 0.2392 s/iter total_throughput: 3122.06 samples/s lr: 8.68e-04 [09/26 17:15:23] lb.utils.events INFO: eta: 12:20:13 iteration: 89399/375342 consumed_samples: 91545600 total_loss: 3.841 time: 0.3280 s/iter data_time: 0.2202 s/iter total_throughput: 3122.01 samples/s lr: 8.68e-04 [09/26 17:15:55] lb.utils.events INFO: eta: 12:23:26 iteration: 89499/375342 consumed_samples: 91648000 total_loss: 3.844 time: 0.3280 s/iter data_time: 0.2176 s/iter total_throughput: 3122.04 samples/s lr: 8.67e-04 [09/26 17:16:29] lb.utils.events INFO: eta: 12:25:14 iteration: 89599/375342 consumed_samples: 91750400 total_loss: 3.865 time: 0.3280 s/iter data_time: 0.2253 s/iter total_throughput: 3121.98 samples/s lr: 8.67e-04 [09/26 17:17:02] lb.utils.events INFO: eta: 12:22:14 iteration: 89699/375342 consumed_samples: 91852800 total_loss: 3.863 time: 0.3280 s/iter data_time: 0.2113 s/iter total_throughput: 3121.91 samples/s lr: 8.67e-04 [09/26 17:17:35] lb.utils.events INFO: eta: 12:19:11 iteration: 89799/375342 consumed_samples: 91955200 total_loss: 3.868 time: 0.3280 s/iter data_time: 0.2083 s/iter total_throughput: 3121.87 samples/s lr: 8.67e-04 [09/26 17:18:09] lb.utils.events INFO: eta: 12:17:47 iteration: 89899/375342 consumed_samples: 92057600 total_loss: 3.839 time: 0.3280 s/iter data_time: 0.2080 s/iter total_throughput: 3121.83 samples/s lr: 8.66e-04 [09/26 17:18:41] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0089999 [09/26 17:18:42] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/26 17:18:42] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/26 17:18:46] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0928 s/iter. Inference: 0.1498 s/iter. Eval: 0.0021 s/iter. Total: 0.2447 s/iter. ETA=0:00:09 [09/26 17:18:51] lb.evaluation.evaluator INFO: Inference done 28672/50000. Dataloading: 0.1283 s/iter. Inference: 0.1509 s/iter. Eval: 0.0021 s/iter. Total: 0.2813 s/iter. ETA=0:00:05 [09/26 17:18:57] lb.evaluation.evaluator INFO: Inference done 46080/50000. Dataloading: 0.1467 s/iter. Inference: 0.1497 s/iter. Eval: 0.0020 s/iter. Total: 0.2985 s/iter. ETA=0:00:00 [09/26 17:18:58] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/26 17:18:58] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.847996 (0.000257 s / iter per device, on 8 devices) [09/26 17:18:58] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000132 s / iter per device, on 8 devices) [09/26 17:18:58] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/26 17:18:58] lb.evaluation.utils INFO: copypaste: Acc@1=70.76400000000001 [09/26 17:18:58] lb.evaluation.utils INFO: copypaste: Acc@5=90.324 [09/26 17:18:58] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 70.76400, better than last best score 70.07600 @ iteration 84999. [09/26 17:18:58] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/26 17:18:58] lb.utils.events INFO: eta: 12:14:21 iteration: 89999/375342 consumed_samples: 92160000 total_loss: 3.832 time: 0.3280 s/iter data_time: 0.2111 s/iter total_throughput: 3121.84 samples/s lr: 8.66e-04 [09/26 17:19:30] lb.utils.events INFO: eta: 12:16:33 iteration: 90099/375342 consumed_samples: 92262400 total_loss: 3.864 time: 0.3280 s/iter data_time: 0.2308 s/iter total_throughput: 3122.00 samples/s lr: 8.66e-04 [09/26 17:20:03] lb.utils.events INFO: eta: 12:21:02 iteration: 90199/375342 consumed_samples: 92364800 total_loss: 3.856 time: 0.3280 s/iter data_time: 0.2376 s/iter total_throughput: 3121.99 samples/s lr: 8.66e-04 [09/26 17:20:36] lb.utils.events INFO: eta: 12:30:07 iteration: 90299/375342 consumed_samples: 92467200 total_loss: 3.857 time: 0.3280 s/iter data_time: 0.2471 s/iter total_throughput: 3121.96 samples/s lr: 8.65e-04 [09/26 17:21:09] lb.utils.events INFO: eta: 12:39:39 iteration: 90399/375342 consumed_samples: 92569600 total_loss: 3.883 time: 0.3280 s/iter data_time: 0.2291 s/iter total_throughput: 3121.93 samples/s lr: 8.65e-04 [09/26 17:21:42] lb.utils.events INFO: eta: 12:34:22 iteration: 90499/375342 consumed_samples: 92672000 total_loss: 3.882 time: 0.3280 s/iter data_time: 0.2111 s/iter total_throughput: 3121.90 samples/s lr: 8.65e-04 [09/26 17:22:15] lb.utils.events INFO: eta: 12:32:58 iteration: 90599/375342 consumed_samples: 92774400 total_loss: 3.846 time: 0.3280 s/iter data_time: 0.2074 s/iter total_throughput: 3121.85 samples/s lr: 8.64e-04 [09/26 17:22:48] lb.utils.events INFO: eta: 12:38:27 iteration: 90699/375342 consumed_samples: 92876800 total_loss: 3.853 time: 0.3280 s/iter data_time: 0.2256 s/iter total_throughput: 3121.83 samples/s lr: 8.64e-04 [09/26 17:23:21] lb.utils.events INFO: eta: 12:48:12 iteration: 90799/375342 consumed_samples: 92979200 total_loss: 3.854 time: 0.3280 s/iter data_time: 0.2166 s/iter total_throughput: 3121.80 samples/s lr: 8.64e-04 [09/26 17:23:54] lb.utils.events INFO: eta: 12:57:50 iteration: 90899/375342 consumed_samples: 93081600 total_loss: 3.85 time: 0.3280 s/iter data_time: 0.2216 s/iter total_throughput: 3121.76 samples/s lr: 8.64e-04 [09/26 17:24:27] lb.utils.events INFO: eta: 13:00:50 iteration: 90999/375342 consumed_samples: 93184000 total_loss: 3.841 time: 0.3280 s/iter data_time: 0.2150 s/iter total_throughput: 3121.74 samples/s lr: 8.63e-04 [09/26 17:25:00] lb.utils.events INFO: eta: 13:00:33 iteration: 91099/375342 consumed_samples: 93286400 total_loss: 3.864 time: 0.3280 s/iter data_time: 0.2152 s/iter total_throughput: 3121.74 samples/s lr: 8.63e-04 [09/26 17:25:33] lb.utils.events INFO: eta: 12:42:03 iteration: 91199/375342 consumed_samples: 93388800 total_loss: 3.856 time: 0.3280 s/iter data_time: 0.2096 s/iter total_throughput: 3121.70 samples/s lr: 8.63e-04 [09/26 17:26:06] lb.utils.events INFO: eta: 12:32:34 iteration: 91299/375342 consumed_samples: 93491200 total_loss: 3.854 time: 0.3280 s/iter data_time: 0.2044 s/iter total_throughput: 3121.72 samples/s lr: 8.62e-04 [09/26 17:26:39] lb.utils.events INFO: eta: 12:21:50 iteration: 91399/375342 consumed_samples: 93593600 total_loss: 3.84 time: 0.3280 s/iter data_time: 0.2127 s/iter total_throughput: 3121.67 samples/s lr: 8.62e-04 [09/26 17:27:12] lb.utils.events INFO: eta: 12:25:41 iteration: 91499/375342 consumed_samples: 93696000 total_loss: 3.832 time: 0.3280 s/iter data_time: 0.2385 s/iter total_throughput: 3121.68 samples/s lr: 8.62e-04 [09/26 17:27:45] lb.utils.events INFO: eta: 12:31:57 iteration: 91599/375342 consumed_samples: 93798400 total_loss: 3.833 time: 0.3280 s/iter data_time: 0.2157 s/iter total_throughput: 3121.66 samples/s lr: 8.62e-04 [09/26 17:28:18] lb.utils.events INFO: eta: 12:32:09 iteration: 91699/375342 consumed_samples: 93900800 total_loss: 3.829 time: 0.3280 s/iter data_time: 0.2145 s/iter total_throughput: 3121.62 samples/s lr: 8.61e-04 [09/26 17:28:51] lb.utils.events INFO: eta: 12:31:14 iteration: 91799/375342 consumed_samples: 94003200 total_loss: 3.839 time: 0.3280 s/iter data_time: 0.2307 s/iter total_throughput: 3121.60 samples/s lr: 8.61e-04 [09/26 17:29:25] lb.utils.events INFO: eta: 12:23:57 iteration: 91899/375342 consumed_samples: 94105600 total_loss: 3.845 time: 0.3280 s/iter data_time: 0.2125 s/iter total_throughput: 3121.51 samples/s lr: 8.61e-04 [09/26 17:29:58] lb.utils.events INFO: eta: 12:22:22 iteration: 91999/375342 consumed_samples: 94208000 total_loss: 3.859 time: 0.3281 s/iter data_time: 0.2060 s/iter total_throughput: 3121.45 samples/s lr: 8.60e-04 [09/26 17:30:31] lb.utils.events INFO: eta: 12:18:11 iteration: 92099/375342 consumed_samples: 94310400 total_loss: 3.846 time: 0.3281 s/iter data_time: 0.2144 s/iter total_throughput: 3121.46 samples/s lr: 8.60e-04 [09/26 17:31:04] lb.utils.events INFO: eta: 12:20:13 iteration: 92199/375342 consumed_samples: 94412800 total_loss: 3.823 time: 0.3280 s/iter data_time: 0.2187 s/iter total_throughput: 3121.49 samples/s lr: 8.60e-04 [09/26 17:31:37] lb.utils.events INFO: eta: 12:24:57 iteration: 92299/375342 consumed_samples: 94515200 total_loss: 3.83 time: 0.3280 s/iter data_time: 0.2319 s/iter total_throughput: 3121.48 samples/s lr: 8.59e-04 [09/26 17:32:09] lb.utils.events INFO: eta: 12:32:58 iteration: 92399/375342 consumed_samples: 94617600 total_loss: 3.84 time: 0.3281 s/iter data_time: 0.2422 s/iter total_throughput: 3121.46 samples/s lr: 8.59e-04 [09/26 17:32:42] lb.utils.events INFO: eta: 12:29:21 iteration: 92499/375342 consumed_samples: 94720000 total_loss: 3.855 time: 0.3281 s/iter data_time: 0.2334 s/iter total_throughput: 3121.47 samples/s lr: 8.59e-04 [09/26 17:33:15] lb.utils.events INFO: eta: 12:31:48 iteration: 92599/375342 consumed_samples: 94822400 total_loss: 3.855 time: 0.3281 s/iter data_time: 0.2336 s/iter total_throughput: 3121.44 samples/s lr: 8.59e-04 [09/26 17:33:48] lb.utils.events INFO: eta: 12:28:35 iteration: 92699/375342 consumed_samples: 94924800 total_loss: 3.839 time: 0.3281 s/iter data_time: 0.2296 s/iter total_throughput: 3121.43 samples/s lr: 8.58e-04 [09/26 17:34:21] lb.utils.events INFO: eta: 12:36:40 iteration: 92799/375342 consumed_samples: 95027200 total_loss: 3.828 time: 0.3281 s/iter data_time: 0.2214 s/iter total_throughput: 3121.43 samples/s lr: 8.58e-04 [09/26 17:34:54] lb.utils.events INFO: eta: 12:43:42 iteration: 92899/375342 consumed_samples: 95129600 total_loss: 3.824 time: 0.3281 s/iter data_time: 0.2389 s/iter total_throughput: 3121.44 samples/s lr: 8.58e-04 [09/26 17:35:27] lb.utils.events INFO: eta: 13:07:26 iteration: 92999/375342 consumed_samples: 95232000 total_loss: 3.848 time: 0.3281 s/iter data_time: 0.2351 s/iter total_throughput: 3121.41 samples/s lr: 8.57e-04 [09/26 17:35:59] lb.utils.events INFO: eta: 15:34:43 iteration: 93099/375342 consumed_samples: 95334400 total_loss: 3.857 time: 0.3281 s/iter data_time: 0.2415 s/iter total_throughput: 3121.45 samples/s lr: 8.57e-04 [09/26 17:36:32] lb.utils.events INFO: eta: 16:50:18 iteration: 93199/375342 consumed_samples: 95436800 total_loss: 3.842 time: 0.3281 s/iter data_time: 0.2461 s/iter total_throughput: 3121.44 samples/s lr: 8.57e-04 [09/26 17:37:06] lb.utils.events INFO: eta: 15:51:29 iteration: 93299/375342 consumed_samples: 95539200 total_loss: 3.841 time: 0.3281 s/iter data_time: 0.2271 s/iter total_throughput: 3121.39 samples/s lr: 8.57e-04 [09/26 17:37:39] lb.utils.events INFO: eta: 16:21:28 iteration: 93399/375342 consumed_samples: 95641600 total_loss: 3.846 time: 0.3281 s/iter data_time: 0.2282 s/iter total_throughput: 3121.34 samples/s lr: 8.56e-04 [09/26 17:38:12] lb.utils.events INFO: eta: 16:16:29 iteration: 93499/375342 consumed_samples: 95744000 total_loss: 3.848 time: 0.3281 s/iter data_time: 0.2130 s/iter total_throughput: 3121.35 samples/s lr: 8.56e-04 [09/26 17:38:44] lb.utils.events INFO: eta: 15:41:29 iteration: 93599/375342 consumed_samples: 95846400 total_loss: 3.853 time: 0.3281 s/iter data_time: 0.2132 s/iter total_throughput: 3121.34 samples/s lr: 8.56e-04 [09/26 17:39:17] lb.utils.events INFO: eta: 15:26:05 iteration: 93699/375342 consumed_samples: 95948800 total_loss: 3.864 time: 0.3281 s/iter data_time: 0.2385 s/iter total_throughput: 3121.36 samples/s lr: 8.55e-04 [09/26 17:39:50] lb.utils.events INFO: eta: 14:27:46 iteration: 93799/375342 consumed_samples: 96051200 total_loss: 3.861 time: 0.3281 s/iter data_time: 0.2134 s/iter total_throughput: 3121.39 samples/s lr: 8.55e-04 [09/26 17:40:23] lb.utils.events INFO: eta: 13:43:09 iteration: 93899/375342 consumed_samples: 96153600 total_loss: 3.856 time: 0.3281 s/iter data_time: 0.2195 s/iter total_throughput: 3121.37 samples/s lr: 8.55e-04 [09/26 17:40:56] lb.utils.events INFO: eta: 13:13:33 iteration: 93999/375342 consumed_samples: 96256000 total_loss: 3.84 time: 0.3281 s/iter data_time: 0.2528 s/iter total_throughput: 3121.33 samples/s lr: 8.55e-04 [09/26 17:41:29] lb.utils.events INFO: eta: 12:53:15 iteration: 94099/375342 consumed_samples: 96358400 total_loss: 3.825 time: 0.3281 s/iter data_time: 0.2047 s/iter total_throughput: 3121.26 samples/s lr: 8.54e-04 [09/26 17:42:02] lb.utils.events INFO: eta: 12:37:18 iteration: 94199/375342 consumed_samples: 96460800 total_loss: 3.826 time: 0.3281 s/iter data_time: 0.2184 s/iter total_throughput: 3121.26 samples/s lr: 8.54e-04 [09/26 17:42:35] lb.utils.events INFO: eta: 12:35:05 iteration: 94299/375342 consumed_samples: 96563200 total_loss: 3.841 time: 0.3281 s/iter data_time: 0.2158 s/iter total_throughput: 3121.29 samples/s lr: 8.54e-04 [09/26 17:43:07] lb.utils.events INFO: eta: 12:34:20 iteration: 94399/375342 consumed_samples: 96665600 total_loss: 3.847 time: 0.3281 s/iter data_time: 0.2130 s/iter total_throughput: 3121.29 samples/s lr: 8.53e-04 [09/26 17:43:40] lb.utils.events INFO: eta: 12:25:48 iteration: 94499/375342 consumed_samples: 96768000 total_loss: 3.847 time: 0.3281 s/iter data_time: 0.2087 s/iter total_throughput: 3121.29 samples/s lr: 8.53e-04 [09/26 17:44:13] lb.utils.events INFO: eta: 12:19:46 iteration: 94599/375342 consumed_samples: 96870400 total_loss: 3.858 time: 0.3281 s/iter data_time: 0.2095 s/iter total_throughput: 3121.29 samples/s lr: 8.53e-04 [09/26 17:44:46] lb.utils.events INFO: eta: 12:12:04 iteration: 94699/375342 consumed_samples: 96972800 total_loss: 3.857 time: 0.3281 s/iter data_time: 0.2002 s/iter total_throughput: 3121.27 samples/s lr: 8.52e-04 [09/26 17:45:19] lb.utils.events INFO: eta: 12:07:57 iteration: 94799/375342 consumed_samples: 97075200 total_loss: 3.844 time: 0.3281 s/iter data_time: 0.2046 s/iter total_throughput: 3121.32 samples/s lr: 8.52e-04 [09/26 17:45:52] lb.utils.events INFO: eta: 12:06:23 iteration: 94899/375342 consumed_samples: 97177600 total_loss: 3.841 time: 0.3281 s/iter data_time: 0.2146 s/iter total_throughput: 3121.28 samples/s lr: 8.52e-04 [09/26 17:46:24] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0094999 [09/26 17:46:25] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/26 17:46:25] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/26 17:46:29] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0852 s/iter. Inference: 0.1464 s/iter. Eval: 0.0022 s/iter. Total: 0.2337 s/iter. ETA=0:00:08 [09/26 17:46:35] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1450 s/iter. Inference: 0.1495 s/iter. Eval: 0.0021 s/iter. Total: 0.2967 s/iter. ETA=0:00:05 [09/26 17:46:40] lb.evaluation.evaluator INFO: Inference done 49152/50000. Dataloading: 0.1295 s/iter. Inference: 0.1502 s/iter. Eval: 0.0021 s/iter. Total: 0.2819 s/iter. ETA=0:00:00 [09/26 17:46:40] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/26 17:46:40] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.567817 (0.000251 s / iter per device, on 8 devices) [09/26 17:46:40] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000133 s / iter per device, on 8 devices) [09/26 17:46:40] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/26 17:46:40] lb.evaluation.utils INFO: copypaste: Acc@1=70.53399999999999 [09/26 17:46:40] lb.evaluation.utils INFO: copypaste: Acc@5=90.254 [09/26 17:46:40] lb.engine.hooks INFO: Not saving as latest eval score for Acc@1 is 70.53400, not better than best score 70.76400 @ iteration 89999. [09/26 17:46:40] lb.utils.events INFO: eta: 12:04:11 iteration: 94999/375342 consumed_samples: 97280000 total_loss: 3.839 time: 0.3281 s/iter data_time: 0.2142 s/iter total_throughput: 3121.31 samples/s lr: 8.52e-04 [09/26 17:47:11] lb.utils.events INFO: eta: 12:03:10 iteration: 95099/375342 consumed_samples: 97382400 total_loss: 3.836 time: 0.3281 s/iter data_time: 0.2227 s/iter total_throughput: 3121.47 samples/s lr: 8.51e-04 [09/26 17:47:44] lb.utils.events INFO: eta: 12:05:54 iteration: 95199/375342 consumed_samples: 97484800 total_loss: 3.843 time: 0.3281 s/iter data_time: 0.2423 s/iter total_throughput: 3121.45 samples/s lr: 8.51e-04 [09/26 17:48:17] lb.utils.events INFO: eta: 12:12:18 iteration: 95299/375342 consumed_samples: 97587200 total_loss: 3.84 time: 0.3280 s/iter data_time: 0.2470 s/iter total_throughput: 3121.48 samples/s lr: 8.51e-04 [09/26 17:48:50] lb.utils.events INFO: eta: 12:16:22 iteration: 95399/375342 consumed_samples: 97689600 total_loss: 3.845 time: 0.3281 s/iter data_time: 0.2213 s/iter total_throughput: 3121.46 samples/s lr: 8.50e-04 [09/26 17:49:22] lb.utils.events INFO: eta: 12:22:18 iteration: 95499/375342 consumed_samples: 97792000 total_loss: 3.836 time: 0.3280 s/iter data_time: 0.2330 s/iter total_throughput: 3121.48 samples/s lr: 8.50e-04 [09/26 17:49:55] lb.utils.events INFO: eta: 12:36:39 iteration: 95599/375342 consumed_samples: 97894400 total_loss: 3.829 time: 0.3280 s/iter data_time: 0.2214 s/iter total_throughput: 3121.50 samples/s lr: 8.50e-04 [09/26 17:50:28] lb.utils.events INFO: eta: 12:52:56 iteration: 95699/375342 consumed_samples: 97996800 total_loss: 3.838 time: 0.3281 s/iter data_time: 0.2513 s/iter total_throughput: 3121.46 samples/s lr: 8.50e-04 [09/26 17:51:01] lb.utils.events INFO: eta: 13:19:37 iteration: 95799/375342 consumed_samples: 98099200 total_loss: 3.85 time: 0.3281 s/iter data_time: 0.2315 s/iter total_throughput: 3121.44 samples/s lr: 8.49e-04 [09/26 17:51:34] lb.utils.events INFO: eta: 14:46:35 iteration: 95899/375342 consumed_samples: 98201600 total_loss: 3.858 time: 0.3281 s/iter data_time: 0.2273 s/iter total_throughput: 3121.44 samples/s lr: 8.49e-04 [09/26 17:52:07] lb.utils.events INFO: eta: 16:33:32 iteration: 95999/375342 consumed_samples: 98304000 total_loss: 3.846 time: 0.3280 s/iter data_time: 0.2180 s/iter total_throughput: 3121.48 samples/s lr: 8.49e-04 [09/26 17:52:39] lb.utils.events INFO: eta: 17:33:43 iteration: 96099/375342 consumed_samples: 98406400 total_loss: 3.848 time: 0.3280 s/iter data_time: 0.2245 s/iter total_throughput: 3121.52 samples/s lr: 8.48e-04 [09/26 17:53:12] lb.utils.events INFO: eta: 18:14:12 iteration: 96199/375342 consumed_samples: 98508800 total_loss: 3.84 time: 0.3280 s/iter data_time: 0.2214 s/iter total_throughput: 3121.52 samples/s lr: 8.48e-04 [09/26 17:53:44] lb.utils.events INFO: eta: 13:16:04 iteration: 96299/375342 consumed_samples: 98611200 total_loss: 3.835 time: 0.3280 s/iter data_time: 0.1934 s/iter total_throughput: 3121.53 samples/s lr: 8.48e-04 [09/26 17:54:17] lb.utils.events INFO: eta: 12:56:15 iteration: 96399/375342 consumed_samples: 98713600 total_loss: 3.844 time: 0.3280 s/iter data_time: 0.2206 s/iter total_throughput: 3121.58 samples/s lr: 8.47e-04 [09/26 17:54:50] lb.utils.events INFO: eta: 13:11:34 iteration: 96499/375342 consumed_samples: 98816000 total_loss: 3.833 time: 0.3280 s/iter data_time: 0.2385 s/iter total_throughput: 3121.58 samples/s lr: 8.47e-04 [09/26 17:55:22] lb.utils.events INFO: eta: 13:24:18 iteration: 96599/375342 consumed_samples: 98918400 total_loss: 3.843 time: 0.3280 s/iter data_time: 0.2338 s/iter total_throughput: 3121.57 samples/s lr: 8.47e-04 [09/26 17:55:56] lb.utils.events INFO: eta: 12:57:49 iteration: 96699/375342 consumed_samples: 99020800 total_loss: 3.862 time: 0.3280 s/iter data_time: 0.2065 s/iter total_throughput: 3121.53 samples/s lr: 8.47e-04 [09/26 17:56:28] lb.utils.events INFO: eta: 12:44:46 iteration: 96799/375342 consumed_samples: 99123200 total_loss: 3.843 time: 0.3280 s/iter data_time: 0.2156 s/iter total_throughput: 3121.54 samples/s lr: 8.46e-04 [09/26 17:57:01] lb.utils.events INFO: eta: 12:54:07 iteration: 96899/375342 consumed_samples: 99225600 total_loss: 3.812 time: 0.3280 s/iter data_time: 0.2482 s/iter total_throughput: 3121.58 samples/s lr: 8.46e-04 [09/26 17:57:34] lb.utils.events INFO: eta: 12:56:29 iteration: 96999/375342 consumed_samples: 99328000 total_loss: 3.82 time: 0.3280 s/iter data_time: 0.2147 s/iter total_throughput: 3121.57 samples/s lr: 8.46e-04 [09/26 17:58:06] lb.utils.events INFO: eta: 12:43:41 iteration: 97099/375342 consumed_samples: 99430400 total_loss: 3.837 time: 0.3280 s/iter data_time: 0.1958 s/iter total_throughput: 3121.57 samples/s lr: 8.45e-04 [09/26 17:58:39] lb.utils.events INFO: eta: 12:34:47 iteration: 97199/375342 consumed_samples: 99532800 total_loss: 3.827 time: 0.3280 s/iter data_time: 0.2283 s/iter total_throughput: 3121.64 samples/s lr: 8.45e-04 [09/26 17:59:11] lb.utils.events INFO: eta: 12:53:51 iteration: 97299/375342 consumed_samples: 99635200 total_loss: 3.826 time: 0.3280 s/iter data_time: 0.2317 s/iter total_throughput: 3121.71 samples/s lr: 8.45e-04 [09/26 17:59:43] lb.utils.events INFO: eta: 12:57:52 iteration: 97399/375342 consumed_samples: 99737600 total_loss: 3.839 time: 0.3280 s/iter data_time: 0.2215 s/iter total_throughput: 3121.72 samples/s lr: 8.44e-04 [09/26 18:00:16] lb.utils.events INFO: eta: 12:44:57 iteration: 97499/375342 consumed_samples: 99840000 total_loss: 3.814 time: 0.3280 s/iter data_time: 0.2285 s/iter total_throughput: 3121.76 samples/s lr: 8.44e-04 [09/26 18:00:48] lb.utils.events INFO: eta: 12:39:41 iteration: 97599/375342 consumed_samples: 99942400 total_loss: 3.812 time: 0.3280 s/iter data_time: 0.2186 s/iter total_throughput: 3121.82 samples/s lr: 8.44e-04 [09/26 18:01:21] lb.utils.events INFO: eta: 12:42:02 iteration: 97699/375342 consumed_samples: 100044800 total_loss: 3.852 time: 0.3280 s/iter data_time: 0.2170 s/iter total_throughput: 3121.84 samples/s lr: 8.44e-04 [09/26 18:01:53] lb.utils.events INFO: eta: 12:45:51 iteration: 97799/375342 consumed_samples: 100147200 total_loss: 3.827 time: 0.3280 s/iter data_time: 0.2371 s/iter total_throughput: 3121.86 samples/s lr: 8.43e-04 [09/26 18:02:26] lb.utils.events INFO: eta: 12:40:17 iteration: 97899/375342 consumed_samples: 100249600 total_loss: 3.797 time: 0.3280 s/iter data_time: 0.2278 s/iter total_throughput: 3121.87 samples/s lr: 8.43e-04 [09/26 18:02:58] lb.utils.events INFO: eta: 12:42:01 iteration: 97999/375342 consumed_samples: 100352000 total_loss: 3.817 time: 0.3280 s/iter data_time: 0.2459 s/iter total_throughput: 3121.96 samples/s lr: 8.43e-04 [09/26 18:03:31] lb.utils.events INFO: eta: 13:32:46 iteration: 98099/375342 consumed_samples: 100454400 total_loss: 3.852 time: 0.3280 s/iter data_time: 0.2325 s/iter total_throughput: 3121.93 samples/s lr: 8.42e-04 [09/26 18:04:04] lb.utils.events INFO: eta: 15:15:53 iteration: 98199/375342 consumed_samples: 100556800 total_loss: 3.852 time: 0.3280 s/iter data_time: 0.2429 s/iter total_throughput: 3121.86 samples/s lr: 8.42e-04 [09/26 18:04:38] lb.utils.events INFO: eta: 13:33:14 iteration: 98299/375342 consumed_samples: 100659200 total_loss: 3.834 time: 0.3280 s/iter data_time: 0.2232 s/iter total_throughput: 3121.83 samples/s lr: 8.42e-04 [09/26 18:05:11] lb.utils.events INFO: eta: 13:02:34 iteration: 98399/375342 consumed_samples: 100761600 total_loss: 3.828 time: 0.3280 s/iter data_time: 0.2122 s/iter total_throughput: 3121.80 samples/s lr: 8.41e-04 [09/26 18:05:44] lb.utils.events INFO: eta: 13:15:57 iteration: 98499/375342 consumed_samples: 100864000 total_loss: 3.826 time: 0.3280 s/iter data_time: 0.2435 s/iter total_throughput: 3121.76 samples/s lr: 8.41e-04 [09/26 18:06:17] lb.utils.events INFO: eta: 14:24:31 iteration: 98599/375342 consumed_samples: 100966400 total_loss: 3.828 time: 0.3280 s/iter data_time: 0.2374 s/iter total_throughput: 3121.74 samples/s lr: 8.41e-04 [09/26 18:06:51] lb.utils.events INFO: eta: 15:08:29 iteration: 98699/375342 consumed_samples: 101068800 total_loss: 3.852 time: 0.3280 s/iter data_time: 0.2552 s/iter total_throughput: 3121.64 samples/s lr: 8.40e-04 [09/26 18:07:25] lb.utils.events INFO: eta: 15:28:22 iteration: 98799/375342 consumed_samples: 101171200 total_loss: 3.859 time: 0.3280 s/iter data_time: 0.2544 s/iter total_throughput: 3121.55 samples/s lr: 8.40e-04 [09/26 18:07:58] lb.utils.events INFO: eta: 16:54:19 iteration: 98899/375342 consumed_samples: 101273600 total_loss: 3.836 time: 0.3280 s/iter data_time: 0.2411 s/iter total_throughput: 3121.48 samples/s lr: 8.40e-04 [09/26 18:08:31] lb.utils.events INFO: eta: 15:08:02 iteration: 98999/375342 consumed_samples: 101376000 total_loss: 3.83 time: 0.3281 s/iter data_time: 0.2029 s/iter total_throughput: 3121.43 samples/s lr: 8.40e-04 [09/26 18:09:04] lb.utils.events INFO: eta: 13:06:24 iteration: 99099/375342 consumed_samples: 101478400 total_loss: 3.832 time: 0.3281 s/iter data_time: 0.2167 s/iter total_throughput: 3121.41 samples/s lr: 8.39e-04 [09/26 18:09:37] lb.utils.events INFO: eta: 12:27:44 iteration: 99199/375342 consumed_samples: 101580800 total_loss: 3.817 time: 0.3281 s/iter data_time: 0.1971 s/iter total_throughput: 3121.41 samples/s lr: 8.39e-04 [09/26 18:10:10] lb.utils.events INFO: eta: 12:20:46 iteration: 99299/375342 consumed_samples: 101683200 total_loss: 3.806 time: 0.3281 s/iter data_time: 0.2296 s/iter total_throughput: 3121.38 samples/s lr: 8.39e-04 [09/26 18:10:43] lb.utils.events INFO: eta: 12:21:12 iteration: 99399/375342 consumed_samples: 101785600 total_loss: 3.813 time: 0.3281 s/iter data_time: 0.2204 s/iter total_throughput: 3121.36 samples/s lr: 8.38e-04 [09/26 18:11:17] lb.utils.events INFO: eta: 12:12:11 iteration: 99499/375342 consumed_samples: 101888000 total_loss: 3.839 time: 0.3281 s/iter data_time: 0.2347 s/iter total_throughput: 3121.28 samples/s lr: 8.38e-04 [09/26 18:11:50] lb.utils.events INFO: eta: 12:04:36 iteration: 99599/375342 consumed_samples: 101990400 total_loss: 3.84 time: 0.3281 s/iter data_time: 0.2071 s/iter total_throughput: 3121.24 samples/s lr: 8.38e-04 [09/26 18:12:24] lb.utils.events INFO: eta: 11:59:03 iteration: 99699/375342 consumed_samples: 102092800 total_loss: 3.828 time: 0.3281 s/iter data_time: 0.2210 s/iter total_throughput: 3121.16 samples/s lr: 8.37e-04 [09/26 18:12:57] lb.utils.events INFO: eta: 11:55:03 iteration: 99799/375342 consumed_samples: 102195200 total_loss: 3.816 time: 0.3281 s/iter data_time: 0.2094 s/iter total_throughput: 3121.12 samples/s lr: 8.37e-04 [09/26 18:13:30] lb.utils.events INFO: eta: 11:49:33 iteration: 99899/375342 consumed_samples: 102297600 total_loss: 3.812 time: 0.3281 s/iter data_time: 0.2254 s/iter total_throughput: 3121.12 samples/s lr: 8.37e-04 [09/26 18:14:03] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0099999 [09/26 18:14:04] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/26 18:14:04] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/26 18:14:08] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0899 s/iter. Inference: 0.1516 s/iter. Eval: 0.0019 s/iter. Total: 0.2434 s/iter. ETA=0:00:09 [09/26 18:14:14] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1465 s/iter. Inference: 0.1517 s/iter. Eval: 0.0020 s/iter. Total: 0.3002 s/iter. ETA=0:00:05 [09/26 18:14:19] lb.evaluation.evaluator INFO: Inference done 50000/50000. Dataloading: 0.1339 s/iter. Inference: 0.1509 s/iter. Eval: 0.0020 s/iter. Total: 0.2868 s/iter. ETA=0:00:00 [09/26 18:14:19] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/26 18:14:19] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.625424 (0.000253 s / iter per device, on 8 devices) [09/26 18:14:19] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000133 s / iter per device, on 8 devices) [09/26 18:14:19] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/26 18:14:19] lb.evaluation.utils INFO: copypaste: Acc@1=70.94200000000001 [09/26 18:14:19] lb.evaluation.utils INFO: copypaste: Acc@5=90.518 [09/26 18:14:19] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 70.94200, better than last best score 70.76400 @ iteration 89999. [09/26 18:14:19] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/26 18:14:20] lb.utils.events INFO: eta: 11:51:53 iteration: 99999/375342 consumed_samples: 102400000 total_loss: 3.827 time: 0.3281 s/iter data_time: 0.2267 s/iter total_throughput: 3121.11 samples/s lr: 8.37e-04 [09/26 18:14:51] lb.utils.events INFO: eta: 11:51:10 iteration: 100099/375342 consumed_samples: 102502400 total_loss: 3.817 time: 0.3281 s/iter data_time: 0.2317 s/iter total_throughput: 3121.23 samples/s lr: 8.36e-04 [09/26 18:15:25] lb.utils.events INFO: eta: 11:54:03 iteration: 100199/375342 consumed_samples: 102604800 total_loss: 3.797 time: 0.3281 s/iter data_time: 0.2221 s/iter total_throughput: 3121.19 samples/s lr: 8.36e-04 [09/26 18:15:58] lb.utils.events INFO: eta: 11:59:00 iteration: 100299/375342 consumed_samples: 102707200 total_loss: 3.804 time: 0.3281 s/iter data_time: 0.2241 s/iter total_throughput: 3121.16 samples/s lr: 8.36e-04 [09/26 18:16:31] lb.utils.events INFO: eta: 12:01:54 iteration: 100399/375342 consumed_samples: 102809600 total_loss: 3.814 time: 0.3281 s/iter data_time: 0.2195 s/iter total_throughput: 3121.16 samples/s lr: 8.35e-04 [09/26 18:17:03] lb.utils.events INFO: eta: 12:05:44 iteration: 100499/375342 consumed_samples: 102912000 total_loss: 3.841 time: 0.3281 s/iter data_time: 0.2361 s/iter total_throughput: 3121.15 samples/s lr: 8.35e-04 [09/26 18:17:37] lb.utils.events INFO: eta: 12:14:51 iteration: 100599/375342 consumed_samples: 103014400 total_loss: 3.832 time: 0.3281 s/iter data_time: 0.2280 s/iter total_throughput: 3121.12 samples/s lr: 8.35e-04 [09/26 18:18:10] lb.utils.events INFO: eta: 12:19:18 iteration: 100699/375342 consumed_samples: 103116800 total_loss: 3.828 time: 0.3281 s/iter data_time: 0.2267 s/iter total_throughput: 3121.07 samples/s lr: 8.34e-04 [09/26 18:18:43] lb.utils.events INFO: eta: 12:39:01 iteration: 100799/375342 consumed_samples: 103219200 total_loss: 3.819 time: 0.3281 s/iter data_time: 0.2277 s/iter total_throughput: 3121.01 samples/s lr: 8.34e-04 [09/26 18:19:17] lb.utils.events INFO: eta: 12:46:10 iteration: 100899/375342 consumed_samples: 103321600 total_loss: 3.822 time: 0.3281 s/iter data_time: 0.2261 s/iter total_throughput: 3120.94 samples/s lr: 8.34e-04 [09/26 18:19:50] lb.utils.events INFO: eta: 12:28:59 iteration: 100999/375342 consumed_samples: 103424000 total_loss: 3.836 time: 0.3281 s/iter data_time: 0.2145 s/iter total_throughput: 3120.89 samples/s lr: 8.33e-04 [09/26 18:20:24] lb.utils.events INFO: eta: 12:23:30 iteration: 101099/375342 consumed_samples: 103526400 total_loss: 3.837 time: 0.3281 s/iter data_time: 0.2155 s/iter total_throughput: 3120.85 samples/s lr: 8.33e-04 [09/26 18:20:57] lb.utils.events INFO: eta: 12:19:31 iteration: 101199/375342 consumed_samples: 103628800 total_loss: 3.812 time: 0.3281 s/iter data_time: 0.2219 s/iter total_throughput: 3120.83 samples/s lr: 8.33e-04 [09/26 18:21:30] lb.utils.events INFO: eta: 12:18:22 iteration: 101299/375342 consumed_samples: 103731200 total_loss: 3.807 time: 0.3281 s/iter data_time: 0.2213 s/iter total_throughput: 3120.81 samples/s lr: 8.32e-04 [09/26 18:22:03] lb.utils.events INFO: eta: 12:16:01 iteration: 101399/375342 consumed_samples: 103833600 total_loss: 3.806 time: 0.3281 s/iter data_time: 0.2038 s/iter total_throughput: 3120.75 samples/s lr: 8.32e-04 [09/26 18:22:36] lb.utils.events INFO: eta: 12:08:44 iteration: 101499/375342 consumed_samples: 103936000 total_loss: 3.809 time: 0.3281 s/iter data_time: 0.2170 s/iter total_throughput: 3120.72 samples/s lr: 8.32e-04 [09/26 18:23:09] lb.utils.events INFO: eta: 12:09:48 iteration: 101599/375342 consumed_samples: 104038400 total_loss: 3.819 time: 0.3281 s/iter data_time: 0.2229 s/iter total_throughput: 3120.70 samples/s lr: 8.32e-04 [09/26 18:23:42] lb.utils.events INFO: eta: 12:12:54 iteration: 101699/375342 consumed_samples: 104140800 total_loss: 3.795 time: 0.3281 s/iter data_time: 0.2191 s/iter total_throughput: 3120.68 samples/s lr: 8.31e-04 [09/26 18:24:15] lb.utils.events INFO: eta: 12:08:54 iteration: 101799/375342 consumed_samples: 104243200 total_loss: 3.779 time: 0.3281 s/iter data_time: 0.2249 s/iter total_throughput: 3120.66 samples/s lr: 8.31e-04 [09/26 18:24:48] lb.utils.events INFO: eta: 12:09:30 iteration: 101899/375342 consumed_samples: 104345600 total_loss: 3.812 time: 0.3281 s/iter data_time: 0.2321 s/iter total_throughput: 3120.63 samples/s lr: 8.31e-04 [09/26 18:25:21] lb.utils.events INFO: eta: 12:16:20 iteration: 101999/375342 consumed_samples: 104448000 total_loss: 3.801 time: 0.3281 s/iter data_time: 0.2500 s/iter total_throughput: 3120.63 samples/s lr: 8.30e-04 [09/26 18:25:55] lb.utils.events INFO: eta: 12:25:23 iteration: 102099/375342 consumed_samples: 104550400 total_loss: 3.803 time: 0.3281 s/iter data_time: 0.2314 s/iter total_throughput: 3120.57 samples/s lr: 8.30e-04 [09/26 18:26:28] lb.utils.events INFO: eta: 12:37:44 iteration: 102199/375342 consumed_samples: 104652800 total_loss: 3.832 time: 0.3281 s/iter data_time: 0.2263 s/iter total_throughput: 3120.57 samples/s lr: 8.30e-04 [09/26 18:27:01] lb.utils.events INFO: eta: 12:34:09 iteration: 102299/375342 consumed_samples: 104755200 total_loss: 3.832 time: 0.3281 s/iter data_time: 0.2026 s/iter total_throughput: 3120.54 samples/s lr: 8.29e-04 [09/26 18:27:34] lb.utils.events INFO: eta: 12:24:33 iteration: 102399/375342 consumed_samples: 104857600 total_loss: 3.829 time: 0.3282 s/iter data_time: 0.2526 s/iter total_throughput: 3120.48 samples/s lr: 8.29e-04 [09/26 18:28:07] lb.utils.events INFO: eta: 12:26:13 iteration: 102499/375342 consumed_samples: 104960000 total_loss: 3.824 time: 0.3282 s/iter data_time: 0.2328 s/iter total_throughput: 3120.45 samples/s lr: 8.29e-04 [09/26 18:28:40] lb.utils.events INFO: eta: 12:27:06 iteration: 102599/375342 consumed_samples: 105062400 total_loss: 3.816 time: 0.3282 s/iter data_time: 0.2430 s/iter total_throughput: 3120.44 samples/s lr: 8.28e-04 [09/26 18:29:13] lb.utils.events INFO: eta: 12:37:44 iteration: 102699/375342 consumed_samples: 105164800 total_loss: 3.805 time: 0.3282 s/iter data_time: 0.2401 s/iter total_throughput: 3120.44 samples/s lr: 8.28e-04 [09/26 18:29:46] lb.utils.events INFO: eta: 12:57:51 iteration: 102799/375342 consumed_samples: 105267200 total_loss: 3.791 time: 0.3282 s/iter data_time: 0.2432 s/iter total_throughput: 3120.41 samples/s lr: 8.28e-04 [09/26 18:30:20] lb.utils.events INFO: eta: 13:24:22 iteration: 102899/375342 consumed_samples: 105369600 total_loss: 3.793 time: 0.3282 s/iter data_time: 0.2163 s/iter total_throughput: 3120.35 samples/s lr: 8.27e-04 [09/26 18:30:53] lb.utils.events INFO: eta: 12:33:52 iteration: 102999/375342 consumed_samples: 105472000 total_loss: 3.823 time: 0.3282 s/iter data_time: 0.2088 s/iter total_throughput: 3120.32 samples/s lr: 8.27e-04 [09/26 18:31:26] lb.utils.events INFO: eta: 12:14:48 iteration: 103099/375342 consumed_samples: 105574400 total_loss: 3.83 time: 0.3282 s/iter data_time: 0.2099 s/iter total_throughput: 3120.26 samples/s lr: 8.27e-04 [09/26 18:32:00] lb.utils.events INFO: eta: 12:06:00 iteration: 103199/375342 consumed_samples: 105676800 total_loss: 3.826 time: 0.3282 s/iter data_time: 0.2027 s/iter total_throughput: 3120.23 samples/s lr: 8.27e-04 [09/26 18:32:33] lb.utils.events INFO: eta: 12:01:28 iteration: 103299/375342 consumed_samples: 105779200 total_loss: 3.821 time: 0.3282 s/iter data_time: 0.2094 s/iter total_throughput: 3120.19 samples/s lr: 8.26e-04 [09/26 18:33:06] lb.utils.events INFO: eta: 11:57:46 iteration: 103399/375342 consumed_samples: 105881600 total_loss: 3.82 time: 0.3282 s/iter data_time: 0.2005 s/iter total_throughput: 3120.19 samples/s lr: 8.26e-04 [09/26 18:33:39] lb.utils.events INFO: eta: 11:55:08 iteration: 103499/375342 consumed_samples: 105984000 total_loss: 3.816 time: 0.3282 s/iter data_time: 0.2274 s/iter total_throughput: 3120.17 samples/s lr: 8.26e-04 [09/26 18:34:11] lb.utils.events INFO: eta: 11:50:24 iteration: 103599/375342 consumed_samples: 106086400 total_loss: 3.797 time: 0.3282 s/iter data_time: 0.2160 s/iter total_throughput: 3120.18 samples/s lr: 8.25e-04 [09/26 18:34:45] lb.utils.events INFO: eta: 11:44:42 iteration: 103699/375342 consumed_samples: 106188800 total_loss: 3.805 time: 0.3282 s/iter data_time: 0.2097 s/iter total_throughput: 3120.13 samples/s lr: 8.25e-04 [09/26 18:35:18] lb.utils.events INFO: eta: 11:40:54 iteration: 103799/375342 consumed_samples: 106291200 total_loss: 3.8 time: 0.3282 s/iter data_time: 0.2163 s/iter total_throughput: 3120.06 samples/s lr: 8.25e-04 [09/26 18:35:52] lb.utils.events INFO: eta: 11:34:55 iteration: 103899/375342 consumed_samples: 106393600 total_loss: 3.797 time: 0.3282 s/iter data_time: 0.2083 s/iter total_throughput: 3120.02 samples/s lr: 8.24e-04 [09/26 18:36:25] lb.utils.events INFO: eta: 11:34:47 iteration: 103999/375342 consumed_samples: 106496000 total_loss: 3.808 time: 0.3282 s/iter data_time: 0.2269 s/iter total_throughput: 3119.97 samples/s lr: 8.24e-04 [09/26 18:36:59] lb.utils.events INFO: eta: 11:37:19 iteration: 104099/375342 consumed_samples: 106598400 total_loss: 3.801 time: 0.3282 s/iter data_time: 0.2133 s/iter total_throughput: 3119.86 samples/s lr: 8.24e-04 [09/26 18:37:32] lb.utils.events INFO: eta: 11:40:45 iteration: 104199/375342 consumed_samples: 106700800 total_loss: 3.823 time: 0.3282 s/iter data_time: 0.2347 s/iter total_throughput: 3119.82 samples/s lr: 8.23e-04 [09/26 18:38:06] lb.utils.events INFO: eta: 11:40:27 iteration: 104299/375342 consumed_samples: 106803200 total_loss: 3.813 time: 0.3282 s/iter data_time: 0.2146 s/iter total_throughput: 3119.77 samples/s lr: 8.23e-04 [09/26 18:38:38] lb.utils.events INFO: eta: 11:40:49 iteration: 104399/375342 consumed_samples: 106905600 total_loss: 3.779 time: 0.3282 s/iter data_time: 0.2217 s/iter total_throughput: 3119.78 samples/s lr: 8.23e-04 [09/26 18:39:11] lb.utils.events INFO: eta: 11:41:33 iteration: 104499/375342 consumed_samples: 107008000 total_loss: 3.818 time: 0.3282 s/iter data_time: 0.2263 s/iter total_throughput: 3119.76 samples/s lr: 8.22e-04 [09/26 18:39:44] lb.utils.events INFO: eta: 11:41:56 iteration: 104599/375342 consumed_samples: 107110400 total_loss: 3.832 time: 0.3282 s/iter data_time: 0.2299 s/iter total_throughput: 3119.78 samples/s lr: 8.22e-04 [09/26 18:40:17] lb.utils.events INFO: eta: 11:43:28 iteration: 104699/375342 consumed_samples: 107212800 total_loss: 3.807 time: 0.3282 s/iter data_time: 0.2306 s/iter total_throughput: 3119.76 samples/s lr: 8.22e-04 [09/26 18:40:51] lb.utils.events INFO: eta: 11:45:05 iteration: 104799/375342 consumed_samples: 107315200 total_loss: 3.795 time: 0.3282 s/iter data_time: 0.2113 s/iter total_throughput: 3119.69 samples/s lr: 8.21e-04 [09/26 18:41:24] lb.utils.events INFO: eta: 11:44:50 iteration: 104899/375342 consumed_samples: 107417600 total_loss: 3.783 time: 0.3282 s/iter data_time: 0.2092 s/iter total_throughput: 3119.63 samples/s lr: 8.21e-04 [09/26 18:41:57] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0104999 [09/26 18:41:58] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/26 18:41:58] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/26 18:42:02] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0902 s/iter. Inference: 0.1468 s/iter. Eval: 0.0023 s/iter. Total: 0.2393 s/iter. ETA=0:00:08 [09/26 18:42:07] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1484 s/iter. Inference: 0.1489 s/iter. Eval: 0.0021 s/iter. Total: 0.2994 s/iter. ETA=0:00:05 [09/26 18:42:13] lb.evaluation.evaluator INFO: Inference done 50000/50000. Dataloading: 0.1341 s/iter. Inference: 0.1499 s/iter. Eval: 0.0021 s/iter. Total: 0.2862 s/iter. ETA=0:00:00 [09/26 18:42:13] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/26 18:42:13] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.596164 (0.000252 s / iter per device, on 8 devices) [09/26 18:42:13] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000132 s / iter per device, on 8 devices) [09/26 18:42:13] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/26 18:42:13] lb.evaluation.utils INFO: copypaste: Acc@1=71.092 [09/26 18:42:13] lb.evaluation.utils INFO: copypaste: Acc@5=90.544 [09/26 18:42:13] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 71.09200, better than last best score 70.94200 @ iteration 99999. [09/26 18:42:13] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/26 18:42:14] lb.utils.events INFO: eta: 11:44:19 iteration: 104999/375342 consumed_samples: 107520000 total_loss: 3.774 time: 0.3282 s/iter data_time: 0.2064 s/iter total_throughput: 3119.64 samples/s lr: 8.21e-04 [09/26 18:42:45] lb.utils.events INFO: eta: 11:46:02 iteration: 105099/375342 consumed_samples: 107622400 total_loss: 3.779 time: 0.3282 s/iter data_time: 0.2470 s/iter total_throughput: 3119.79 samples/s lr: 8.21e-04 [09/26 18:43:18] lb.utils.events INFO: eta: 11:53:53 iteration: 105199/375342 consumed_samples: 107724800 total_loss: 3.776 time: 0.3282 s/iter data_time: 0.2428 s/iter total_throughput: 3119.77 samples/s lr: 8.20e-04 [09/26 18:43:51] lb.utils.events INFO: eta: 12:14:51 iteration: 105299/375342 consumed_samples: 107827200 total_loss: 3.788 time: 0.3282 s/iter data_time: 0.2388 s/iter total_throughput: 3119.76 samples/s lr: 8.20e-04 [09/26 18:44:24] lb.utils.events INFO: eta: 13:05:10 iteration: 105399/375342 consumed_samples: 107929600 total_loss: 3.798 time: 0.3282 s/iter data_time: 0.2346 s/iter total_throughput: 3119.75 samples/s lr: 8.20e-04 [09/26 18:44:57] lb.utils.events INFO: eta: 13:51:26 iteration: 105499/375342 consumed_samples: 108032000 total_loss: 3.802 time: 0.3282 s/iter data_time: 0.2275 s/iter total_throughput: 3119.74 samples/s lr: 8.19e-04 [09/26 18:45:30] lb.utils.events INFO: eta: 15:44:09 iteration: 105599/375342 consumed_samples: 108134400 total_loss: 3.82 time: 0.3282 s/iter data_time: 0.2471 s/iter total_throughput: 3119.69 samples/s lr: 8.19e-04 [09/26 18:46:03] lb.utils.events INFO: eta: 15:17:50 iteration: 105699/375342 consumed_samples: 108236800 total_loss: 3.826 time: 0.3282 s/iter data_time: 0.2095 s/iter total_throughput: 3119.68 samples/s lr: 8.19e-04 [09/26 18:46:36] lb.utils.events INFO: eta: 14:23:06 iteration: 105799/375342 consumed_samples: 108339200 total_loss: 3.812 time: 0.3282 s/iter data_time: 0.2297 s/iter total_throughput: 3119.68 samples/s lr: 8.18e-04 [09/26 18:47:09] lb.utils.events INFO: eta: 14:54:04 iteration: 105899/375342 consumed_samples: 108441600 total_loss: 3.799 time: 0.3282 s/iter data_time: 0.1965 s/iter total_throughput: 3119.67 samples/s lr: 8.18e-04 [09/26 18:47:42] lb.utils.events INFO: eta: 15:34:09 iteration: 105999/375342 consumed_samples: 108544000 total_loss: 3.81 time: 0.3282 s/iter data_time: 0.2045 s/iter total_throughput: 3119.64 samples/s lr: 8.18e-04 [09/26 18:48:14] lb.utils.events INFO: eta: 14:15:37 iteration: 106099/375342 consumed_samples: 108646400 total_loss: 3.806 time: 0.3282 s/iter data_time: 0.2175 s/iter total_throughput: 3119.67 samples/s lr: 8.17e-04 [09/26 18:48:48] lb.utils.events INFO: eta: 12:27:19 iteration: 106199/375342 consumed_samples: 108748800 total_loss: 3.789 time: 0.3282 s/iter data_time: 0.2156 s/iter total_throughput: 3119.63 samples/s lr: 8.17e-04 [09/26 18:49:21] lb.utils.events INFO: eta: 12:04:21 iteration: 106299/375342 consumed_samples: 108851200 total_loss: 3.819 time: 0.3282 s/iter data_time: 0.2078 s/iter total_throughput: 3119.59 samples/s lr: 8.17e-04 [09/26 18:49:54] lb.utils.events INFO: eta: 11:49:10 iteration: 106399/375342 consumed_samples: 108953600 total_loss: 3.82 time: 0.3283 s/iter data_time: 0.2131 s/iter total_throughput: 3119.55 samples/s lr: 8.16e-04 [09/26 18:50:27] lb.utils.events INFO: eta: 11:41:10 iteration: 106499/375342 consumed_samples: 109056000 total_loss: 3.807 time: 0.3283 s/iter data_time: 0.2086 s/iter total_throughput: 3119.54 samples/s lr: 8.16e-04 [09/26 18:51:00] lb.utils.events INFO: eta: 11:34:16 iteration: 106599/375342 consumed_samples: 109158400 total_loss: 3.812 time: 0.3283 s/iter data_time: 0.2259 s/iter total_throughput: 3119.56 samples/s lr: 8.16e-04 [09/26 18:51:33] lb.utils.events INFO: eta: 11:36:26 iteration: 106699/375342 consumed_samples: 109260800 total_loss: 3.807 time: 0.3283 s/iter data_time: 0.2434 s/iter total_throughput: 3119.55 samples/s lr: 8.15e-04 [09/26 18:52:06] lb.utils.events INFO: eta: 11:38:40 iteration: 106799/375342 consumed_samples: 109363200 total_loss: 3.809 time: 0.3283 s/iter data_time: 0.2205 s/iter total_throughput: 3119.52 samples/s lr: 8.15e-04 [09/26 18:52:39] lb.utils.events INFO: eta: 11:38:47 iteration: 106899/375342 consumed_samples: 109465600 total_loss: 3.797 time: 0.3283 s/iter data_time: 0.2243 s/iter total_throughput: 3119.51 samples/s lr: 8.15e-04 [09/26 18:53:12] lb.utils.events INFO: eta: 11:37:58 iteration: 106999/375342 consumed_samples: 109568000 total_loss: 3.8 time: 0.3283 s/iter data_time: 0.2210 s/iter total_throughput: 3119.53 samples/s lr: 8.14e-04 [09/26 18:53:45] lb.utils.events INFO: eta: 11:38:16 iteration: 107099/375342 consumed_samples: 109670400 total_loss: 3.814 time: 0.3283 s/iter data_time: 0.2225 s/iter total_throughput: 3119.51 samples/s lr: 8.14e-04 [09/26 18:54:17] lb.utils.events INFO: eta: 11:45:13 iteration: 107199/375342 consumed_samples: 109772800 total_loss: 3.791 time: 0.3283 s/iter data_time: 0.2179 s/iter total_throughput: 3119.51 samples/s lr: 8.14e-04 [09/26 18:54:51] lb.utils.events INFO: eta: 11:48:42 iteration: 107299/375342 consumed_samples: 109875200 total_loss: 3.789 time: 0.3283 s/iter data_time: 0.2343 s/iter total_throughput: 3119.45 samples/s lr: 8.13e-04 [09/26 18:55:24] lb.utils.events INFO: eta: 11:52:44 iteration: 107399/375342 consumed_samples: 109977600 total_loss: 3.812 time: 0.3283 s/iter data_time: 0.2281 s/iter total_throughput: 3119.45 samples/s lr: 8.13e-04 [09/26 18:55:56] lb.utils.events INFO: eta: 12:04:51 iteration: 107499/375342 consumed_samples: 110080000 total_loss: 3.816 time: 0.3283 s/iter data_time: 0.2204 s/iter total_throughput: 3119.49 samples/s lr: 8.13e-04 [09/26 18:56:30] lb.utils.events INFO: eta: 12:07:03 iteration: 107599/375342 consumed_samples: 110182400 total_loss: 3.821 time: 0.3283 s/iter data_time: 0.2178 s/iter total_throughput: 3119.44 samples/s lr: 8.12e-04 [09/26 18:57:03] lb.utils.events INFO: eta: 11:51:57 iteration: 107699/375342 consumed_samples: 110284800 total_loss: 3.792 time: 0.3283 s/iter data_time: 0.2023 s/iter total_throughput: 3119.37 samples/s lr: 8.12e-04 [09/26 18:57:36] lb.utils.events INFO: eta: 11:46:01 iteration: 107799/375342 consumed_samples: 110387200 total_loss: 3.786 time: 0.3283 s/iter data_time: 0.2086 s/iter total_throughput: 3119.37 samples/s lr: 8.12e-04 [09/26 18:58:09] lb.utils.events INFO: eta: 11:42:56 iteration: 107899/375342 consumed_samples: 110489600 total_loss: 3.806 time: 0.3283 s/iter data_time: 0.2032 s/iter total_throughput: 3119.37 samples/s lr: 8.11e-04 [09/26 18:58:42] lb.utils.events INFO: eta: 11:40:47 iteration: 107999/375342 consumed_samples: 110592000 total_loss: 3.794 time: 0.3283 s/iter data_time: 0.2104 s/iter total_throughput: 3119.38 samples/s lr: 8.11e-04 [09/26 18:59:15] lb.utils.events INFO: eta: 11:36:23 iteration: 108099/375342 consumed_samples: 110694400 total_loss: 3.791 time: 0.3283 s/iter data_time: 0.2153 s/iter total_throughput: 3119.37 samples/s lr: 8.11e-04 [09/26 18:59:47] lb.utils.events INFO: eta: 11:33:08 iteration: 108199/375342 consumed_samples: 110796800 total_loss: 3.793 time: 0.3283 s/iter data_time: 0.2071 s/iter total_throughput: 3119.38 samples/s lr: 8.11e-04 [09/26 19:00:20] lb.utils.events INFO: eta: 11:32:21 iteration: 108299/375342 consumed_samples: 110899200 total_loss: 3.795 time: 0.3283 s/iter data_time: 0.2174 s/iter total_throughput: 3119.41 samples/s lr: 8.10e-04 [09/26 19:00:52] lb.utils.events INFO: eta: 11:34:47 iteration: 108399/375342 consumed_samples: 111001600 total_loss: 3.796 time: 0.3283 s/iter data_time: 0.2325 s/iter total_throughput: 3119.43 samples/s lr: 8.10e-04 [09/26 19:01:26] lb.utils.events INFO: eta: 11:36:38 iteration: 108499/375342 consumed_samples: 111104000 total_loss: 3.791 time: 0.3283 s/iter data_time: 0.2755 s/iter total_throughput: 3119.40 samples/s lr: 8.10e-04 [09/26 19:01:58] lb.utils.events INFO: eta: 11:38:16 iteration: 108599/375342 consumed_samples: 111206400 total_loss: 3.775 time: 0.3283 s/iter data_time: 0.2259 s/iter total_throughput: 3119.41 samples/s lr: 8.09e-04 [09/26 19:02:31] lb.utils.events INFO: eta: 11:38:48 iteration: 108699/375342 consumed_samples: 111308800 total_loss: 3.785 time: 0.3283 s/iter data_time: 0.2102 s/iter total_throughput: 3119.37 samples/s lr: 8.09e-04 [09/26 19:03:04] lb.utils.events INFO: eta: 11:38:16 iteration: 108799/375342 consumed_samples: 111411200 total_loss: 3.803 time: 0.3283 s/iter data_time: 0.2036 s/iter total_throughput: 3119.37 samples/s lr: 8.09e-04 [09/26 19:03:38] lb.utils.events INFO: eta: 11:36:32 iteration: 108899/375342 consumed_samples: 111513600 total_loss: 3.791 time: 0.3283 s/iter data_time: 0.2117 s/iter total_throughput: 3119.34 samples/s lr: 8.08e-04 [09/26 19:04:11] lb.utils.events INFO: eta: 11:36:16 iteration: 108999/375342 consumed_samples: 111616000 total_loss: 3.772 time: 0.3283 s/iter data_time: 0.2055 s/iter total_throughput: 3119.33 samples/s lr: 8.08e-04 [09/26 19:04:43] lb.utils.events INFO: eta: 11:37:12 iteration: 109099/375342 consumed_samples: 111718400 total_loss: 3.779 time: 0.3283 s/iter data_time: 0.2074 s/iter total_throughput: 3119.32 samples/s lr: 8.08e-04 [09/26 19:05:16] lb.utils.events INFO: eta: 11:35:53 iteration: 109199/375342 consumed_samples: 111820800 total_loss: 3.797 time: 0.3283 s/iter data_time: 0.2116 s/iter total_throughput: 3119.32 samples/s lr: 8.07e-04 [09/26 19:05:49] lb.utils.events INFO: eta: 11:35:29 iteration: 109299/375342 consumed_samples: 111923200 total_loss: 3.797 time: 0.3283 s/iter data_time: 0.1914 s/iter total_throughput: 3119.35 samples/s lr: 8.07e-04 [09/26 19:06:21] lb.utils.events INFO: eta: 11:30:06 iteration: 109399/375342 consumed_samples: 112025600 total_loss: 3.786 time: 0.3283 s/iter data_time: 0.2347 s/iter total_throughput: 3119.36 samples/s lr: 8.07e-04 [09/26 19:06:54] lb.utils.events INFO: eta: 11:26:37 iteration: 109499/375342 consumed_samples: 112128000 total_loss: 3.797 time: 0.3283 s/iter data_time: 0.2252 s/iter total_throughput: 3119.39 samples/s lr: 8.06e-04 [09/26 19:07:27] lb.utils.events INFO: eta: 11:24:13 iteration: 109599/375342 consumed_samples: 112230400 total_loss: 3.786 time: 0.3283 s/iter data_time: 0.2068 s/iter total_throughput: 3119.37 samples/s lr: 8.06e-04 [09/26 19:08:00] lb.utils.events INFO: eta: 11:22:55 iteration: 109699/375342 consumed_samples: 112332800 total_loss: 3.791 time: 0.3283 s/iter data_time: 0.2055 s/iter total_throughput: 3119.38 samples/s lr: 8.06e-04 [09/26 19:08:33] lb.utils.events INFO: eta: 11:24:26 iteration: 109799/375342 consumed_samples: 112435200 total_loss: 3.801 time: 0.3283 s/iter data_time: 0.2043 s/iter total_throughput: 3119.38 samples/s lr: 8.05e-04 [09/26 19:09:05] lb.utils.events INFO: eta: 11:24:13 iteration: 109899/375342 consumed_samples: 112537600 total_loss: 3.796 time: 0.3283 s/iter data_time: 0.2027 s/iter total_throughput: 3119.40 samples/s lr: 8.05e-04 [09/26 19:09:38] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0109999 [09/26 19:09:39] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/26 19:09:39] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/26 19:09:43] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0844 s/iter. Inference: 0.1499 s/iter. Eval: 0.0021 s/iter. Total: 0.2365 s/iter. ETA=0:00:08 [09/26 19:09:48] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1448 s/iter. Inference: 0.1486 s/iter. Eval: 0.0021 s/iter. Total: 0.2955 s/iter. ETA=0:00:05 [09/26 19:09:54] lb.evaluation.evaluator INFO: Inference done 50000/50000. Dataloading: 0.1316 s/iter. Inference: 0.1500 s/iter. Eval: 0.0020 s/iter. Total: 0.2837 s/iter. ETA=0:00:00 [09/26 19:09:54] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/26 19:09:54] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.484843 (0.000250 s / iter per device, on 8 devices) [09/26 19:09:54] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000132 s / iter per device, on 8 devices) [09/26 19:09:54] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/26 19:09:54] lb.evaluation.utils INFO: copypaste: Acc@1=71.60600000000001 [09/26 19:09:54] lb.evaluation.utils INFO: copypaste: Acc@5=90.738 [09/26 19:09:54] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 71.60600, better than last best score 71.09200 @ iteration 104999. [09/26 19:09:54] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/26 19:09:55] lb.utils.events INFO: eta: 11:24:19 iteration: 109999/375342 consumed_samples: 112640000 total_loss: 3.798 time: 0.3283 s/iter data_time: 0.2039 s/iter total_throughput: 3119.41 samples/s lr: 8.05e-04 [09/26 19:10:26] lb.utils.events INFO: eta: 11:25:53 iteration: 110099/375342 consumed_samples: 112742400 total_loss: 3.791 time: 0.3283 s/iter data_time: 0.2505 s/iter total_throughput: 3119.55 samples/s lr: 8.04e-04 [09/26 19:10:58] lb.utils.events INFO: eta: 11:32:48 iteration: 110199/375342 consumed_samples: 112844800 total_loss: 3.798 time: 0.3283 s/iter data_time: 0.2465 s/iter total_throughput: 3119.56 samples/s lr: 8.04e-04 [09/26 19:11:31] lb.utils.events INFO: eta: 11:35:15 iteration: 110299/375342 consumed_samples: 112947200 total_loss: 3.785 time: 0.3283 s/iter data_time: 0.2223 s/iter total_throughput: 3119.55 samples/s lr: 8.04e-04 [09/26 19:12:04] lb.utils.events INFO: eta: 11:35:51 iteration: 110399/375342 consumed_samples: 113049600 total_loss: 3.778 time: 0.3282 s/iter data_time: 0.2163 s/iter total_throughput: 3119.58 samples/s lr: 8.03e-04 [09/26 19:12:37] lb.utils.events INFO: eta: 11:36:14 iteration: 110499/375342 consumed_samples: 113152000 total_loss: 3.794 time: 0.3282 s/iter data_time: 0.2165 s/iter total_throughput: 3119.59 samples/s lr: 8.03e-04 [09/26 19:13:09] lb.utils.events INFO: eta: 11:43:25 iteration: 110599/375342 consumed_samples: 113254400 total_loss: 3.788 time: 0.3282 s/iter data_time: 0.2356 s/iter total_throughput: 3119.63 samples/s lr: 8.03e-04 [09/26 19:13:42] lb.utils.events INFO: eta: 11:52:47 iteration: 110699/375342 consumed_samples: 113356800 total_loss: 3.794 time: 0.3282 s/iter data_time: 0.2215 s/iter total_throughput: 3119.62 samples/s lr: 8.02e-04 [09/26 19:14:15] lb.utils.events INFO: eta: 12:08:12 iteration: 110799/375342 consumed_samples: 113459200 total_loss: 3.802 time: 0.3282 s/iter data_time: 0.2213 s/iter total_throughput: 3119.64 samples/s lr: 8.02e-04 [09/26 19:14:47] lb.utils.events INFO: eta: 12:35:03 iteration: 110899/375342 consumed_samples: 113561600 total_loss: 3.793 time: 0.3282 s/iter data_time: 0.2257 s/iter total_throughput: 3119.66 samples/s lr: 8.02e-04 [09/26 19:15:19] lb.utils.events INFO: eta: 15:25:48 iteration: 110999/375342 consumed_samples: 113664000 total_loss: 3.794 time: 0.3282 s/iter data_time: 0.2091 s/iter total_throughput: 3119.70 samples/s lr: 8.01e-04 [09/26 19:15:52] lb.utils.events INFO: eta: 16:04:35 iteration: 111099/375342 consumed_samples: 113766400 total_loss: 3.814 time: 0.3282 s/iter data_time: 0.2200 s/iter total_throughput: 3119.74 samples/s lr: 8.01e-04 [09/26 19:16:24] lb.utils.events INFO: eta: 12:45:50 iteration: 111199/375342 consumed_samples: 113868800 total_loss: 3.784 time: 0.3282 s/iter data_time: 0.2188 s/iter total_throughput: 3119.78 samples/s lr: 8.01e-04 [09/26 19:16:57] lb.utils.events INFO: eta: 12:19:07 iteration: 111299/375342 consumed_samples: 113971200 total_loss: 3.787 time: 0.3282 s/iter data_time: 0.2088 s/iter total_throughput: 3119.77 samples/s lr: 8.00e-04 [09/26 19:17:29] lb.utils.events INFO: eta: 12:16:42 iteration: 111399/375342 consumed_samples: 114073600 total_loss: 3.799 time: 0.3282 s/iter data_time: 0.2062 s/iter total_throughput: 3119.82 samples/s lr: 8.00e-04 [09/26 19:18:02] lb.utils.events INFO: eta: 12:14:37 iteration: 111499/375342 consumed_samples: 114176000 total_loss: 3.772 time: 0.3282 s/iter data_time: 0.2341 s/iter total_throughput: 3119.86 samples/s lr: 8.00e-04 [09/26 19:18:34] lb.utils.events INFO: eta: 12:11:11 iteration: 111599/375342 consumed_samples: 114278400 total_loss: 3.781 time: 0.3282 s/iter data_time: 0.2266 s/iter total_throughput: 3119.89 samples/s lr: 7.99e-04 [09/26 19:19:07] lb.utils.events INFO: eta: 12:10:54 iteration: 111699/375342 consumed_samples: 114380800 total_loss: 3.766 time: 0.3282 s/iter data_time: 0.2385 s/iter total_throughput: 3119.91 samples/s lr: 7.99e-04 [09/26 19:19:39] lb.utils.events INFO: eta: 12:18:20 iteration: 111799/375342 consumed_samples: 114483200 total_loss: 3.769 time: 0.3282 s/iter data_time: 0.2282 s/iter total_throughput: 3119.95 samples/s lr: 7.99e-04 [09/26 19:20:11] lb.utils.events INFO: eta: 12:22:08 iteration: 111899/375342 consumed_samples: 114585600 total_loss: 3.789 time: 0.3282 s/iter data_time: 0.2310 s/iter total_throughput: 3120.00 samples/s lr: 7.98e-04 [09/26 19:20:44] lb.utils.events INFO: eta: 12:24:16 iteration: 111999/375342 consumed_samples: 114688000 total_loss: 3.784 time: 0.3282 s/iter data_time: 0.2310 s/iter total_throughput: 3120.02 samples/s lr: 7.98e-04 [09/26 19:21:17] lb.utils.events INFO: eta: 12:29:26 iteration: 112099/375342 consumed_samples: 114790400 total_loss: 3.779 time: 0.3282 s/iter data_time: 0.2268 s/iter total_throughput: 3120.01 samples/s lr: 7.98e-04 [09/26 19:21:50] lb.utils.events INFO: eta: 12:23:42 iteration: 112199/375342 consumed_samples: 114892800 total_loss: 3.764 time: 0.3282 s/iter data_time: 0.2170 s/iter total_throughput: 3120.00 samples/s lr: 7.97e-04 [09/26 19:22:23] lb.utils.events INFO: eta: 12:20:32 iteration: 112299/375342 consumed_samples: 114995200 total_loss: 3.784 time: 0.3282 s/iter data_time: 0.2166 s/iter total_throughput: 3119.96 samples/s lr: 7.97e-04 [09/26 19:22:56] lb.utils.events INFO: eta: 12:00:10 iteration: 112399/375342 consumed_samples: 115097600 total_loss: 3.783 time: 0.3282 s/iter data_time: 0.2019 s/iter total_throughput: 3119.93 samples/s lr: 7.97e-04 [09/26 19:23:30] lb.utils.events INFO: eta: 11:48:44 iteration: 112499/375342 consumed_samples: 115200000 total_loss: 3.752 time: 0.3282 s/iter data_time: 0.2290 s/iter total_throughput: 3119.90 samples/s lr: 7.96e-04 [09/26 19:24:03] lb.utils.events INFO: eta: 11:38:19 iteration: 112599/375342 consumed_samples: 115302400 total_loss: 3.781 time: 0.3282 s/iter data_time: 0.2183 s/iter total_throughput: 3119.85 samples/s lr: 7.96e-04 [09/26 19:24:36] lb.utils.events INFO: eta: 11:31:25 iteration: 112699/375342 consumed_samples: 115404800 total_loss: 3.782 time: 0.3282 s/iter data_time: 0.2141 s/iter total_throughput: 3119.83 samples/s lr: 7.96e-04 [09/26 19:25:09] lb.utils.events INFO: eta: 11:26:35 iteration: 112799/375342 consumed_samples: 115507200 total_loss: 3.781 time: 0.3282 s/iter data_time: 0.2290 s/iter total_throughput: 3119.79 samples/s lr: 7.95e-04 [09/26 19:25:43] lb.utils.events INFO: eta: 11:23:34 iteration: 112899/375342 consumed_samples: 115609600 total_loss: 3.798 time: 0.3282 s/iter data_time: 0.2413 s/iter total_throughput: 3119.73 samples/s lr: 7.95e-04 [09/26 19:26:16] lb.utils.events INFO: eta: 11:24:11 iteration: 112999/375342 consumed_samples: 115712000 total_loss: 3.776 time: 0.3282 s/iter data_time: 0.2408 s/iter total_throughput: 3119.71 samples/s lr: 7.95e-04 [09/26 19:26:49] lb.utils.events INFO: eta: 11:29:48 iteration: 113099/375342 consumed_samples: 115814400 total_loss: 3.765 time: 0.3282 s/iter data_time: 0.2321 s/iter total_throughput: 3119.72 samples/s lr: 7.94e-04 [09/26 19:27:22] lb.utils.events INFO: eta: 11:36:24 iteration: 113199/375342 consumed_samples: 115916800 total_loss: 3.786 time: 0.3282 s/iter data_time: 0.2376 s/iter total_throughput: 3119.69 samples/s lr: 7.94e-04 [09/26 19:27:55] lb.utils.events INFO: eta: 11:46:19 iteration: 113299/375342 consumed_samples: 116019200 total_loss: 3.775 time: 0.3282 s/iter data_time: 0.2292 s/iter total_throughput: 3119.64 samples/s lr: 7.94e-04 [09/26 19:28:28] lb.utils.events INFO: eta: 12:02:32 iteration: 113399/375342 consumed_samples: 116121600 total_loss: 3.768 time: 0.3282 s/iter data_time: 0.2199 s/iter total_throughput: 3119.62 samples/s lr: 7.93e-04 [09/26 19:29:02] lb.utils.events INFO: eta: 12:21:22 iteration: 113499/375342 consumed_samples: 116224000 total_loss: 3.78 time: 0.3283 s/iter data_time: 0.2333 s/iter total_throughput: 3119.55 samples/s lr: 7.93e-04 [09/26 19:29:35] lb.utils.events INFO: eta: 13:24:56 iteration: 113599/375342 consumed_samples: 116326400 total_loss: 3.785 time: 0.3283 s/iter data_time: 0.2261 s/iter total_throughput: 3119.54 samples/s lr: 7.93e-04 [09/26 19:30:08] lb.utils.events INFO: eta: 15:49:54 iteration: 113699/375342 consumed_samples: 116428800 total_loss: 3.794 time: 0.3283 s/iter data_time: 0.2313 s/iter total_throughput: 3119.50 samples/s lr: 7.92e-04 [09/26 19:30:42] lb.utils.events INFO: eta: 12:34:04 iteration: 113799/375342 consumed_samples: 116531200 total_loss: 3.785 time: 0.3283 s/iter data_time: 0.2116 s/iter total_throughput: 3119.46 samples/s lr: 7.92e-04 [09/26 19:31:15] lb.utils.events INFO: eta: 12:01:22 iteration: 113899/375342 consumed_samples: 116633600 total_loss: 3.771 time: 0.3283 s/iter data_time: 0.2063 s/iter total_throughput: 3119.42 samples/s lr: 7.92e-04 [09/26 19:31:48] lb.utils.events INFO: eta: 11:46:03 iteration: 113999/375342 consumed_samples: 116736000 total_loss: 3.76 time: 0.3283 s/iter data_time: 0.2014 s/iter total_throughput: 3119.41 samples/s lr: 7.91e-04 [09/26 19:32:21] lb.utils.events INFO: eta: 11:30:14 iteration: 114099/375342 consumed_samples: 116838400 total_loss: 3.762 time: 0.3283 s/iter data_time: 0.2142 s/iter total_throughput: 3119.42 samples/s lr: 7.91e-04 [09/26 19:32:54] lb.utils.events INFO: eta: 11:25:36 iteration: 114199/375342 consumed_samples: 116940800 total_loss: 3.763 time: 0.3283 s/iter data_time: 0.2094 s/iter total_throughput: 3119.37 samples/s lr: 7.91e-04 [09/26 19:33:27] lb.utils.events INFO: eta: 11:23:53 iteration: 114299/375342 consumed_samples: 117043200 total_loss: 3.76 time: 0.3283 s/iter data_time: 0.2378 s/iter total_throughput: 3119.37 samples/s lr: 7.90e-04 [09/26 19:34:00] lb.utils.events INFO: eta: 11:21:53 iteration: 114399/375342 consumed_samples: 117145600 total_loss: 3.762 time: 0.3283 s/iter data_time: 0.2141 s/iter total_throughput: 3119.33 samples/s lr: 7.90e-04 [09/26 19:34:33] lb.utils.events INFO: eta: 11:19:48 iteration: 114499/375342 consumed_samples: 117248000 total_loss: 3.763 time: 0.3283 s/iter data_time: 0.2268 s/iter total_throughput: 3119.31 samples/s lr: 7.90e-04 [09/26 19:35:06] lb.utils.events INFO: eta: 11:19:02 iteration: 114599/375342 consumed_samples: 117350400 total_loss: 3.762 time: 0.3283 s/iter data_time: 0.2249 s/iter total_throughput: 3119.29 samples/s lr: 7.89e-04 [09/26 19:35:40] lb.utils.events INFO: eta: 11:22:50 iteration: 114699/375342 consumed_samples: 117452800 total_loss: 3.76 time: 0.3283 s/iter data_time: 0.2385 s/iter total_throughput: 3119.26 samples/s lr: 7.89e-04 [09/26 19:36:13] lb.utils.events INFO: eta: 11:22:29 iteration: 114799/375342 consumed_samples: 117555200 total_loss: 3.771 time: 0.3283 s/iter data_time: 0.2113 s/iter total_throughput: 3119.21 samples/s lr: 7.89e-04 [09/26 19:36:46] lb.utils.events INFO: eta: 11:21:24 iteration: 114899/375342 consumed_samples: 117657600 total_loss: 3.766 time: 0.3283 s/iter data_time: 0.2061 s/iter total_throughput: 3119.20 samples/s lr: 7.88e-04 [09/26 19:37:19] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0114999 [09/26 19:37:20] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/26 19:37:20] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/26 19:37:24] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0842 s/iter. Inference: 0.1464 s/iter. Eval: 0.0020 s/iter. Total: 0.2327 s/iter. ETA=0:00:08 [09/26 19:37:29] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1468 s/iter. Inference: 0.1496 s/iter. Eval: 0.0021 s/iter. Total: 0.2985 s/iter. ETA=0:00:05 [09/26 19:37:34] lb.evaluation.evaluator INFO: Inference done 49152/50000. Dataloading: 0.1326 s/iter. Inference: 0.1488 s/iter. Eval: 0.0020 s/iter. Total: 0.2835 s/iter. ETA=0:00:00 [09/26 19:37:35] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/26 19:37:35] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.623211 (0.000252 s / iter per device, on 8 devices) [09/26 19:37:35] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000132 s / iter per device, on 8 devices) [09/26 19:37:35] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/26 19:37:35] lb.evaluation.utils INFO: copypaste: Acc@1=71.896 [09/26 19:37:35] lb.evaluation.utils INFO: copypaste: Acc@5=90.826 [09/26 19:37:35] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 71.89600, better than last best score 71.60600 @ iteration 109999. [09/26 19:37:35] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/26 19:37:36] lb.utils.events INFO: eta: 11:20:32 iteration: 114999/375342 consumed_samples: 117760000 total_loss: 3.766 time: 0.3283 s/iter data_time: 0.2240 s/iter total_throughput: 3119.19 samples/s lr: 7.88e-04 [09/26 19:38:07] lb.utils.events INFO: eta: 11:32:05 iteration: 115099/375342 consumed_samples: 117862400 total_loss: 3.772 time: 0.3283 s/iter data_time: 0.2462 s/iter total_throughput: 3119.33 samples/s lr: 7.88e-04 [09/26 19:38:40] lb.utils.events INFO: eta: 11:37:37 iteration: 115199/375342 consumed_samples: 117964800 total_loss: 3.772 time: 0.3283 s/iter data_time: 0.2336 s/iter total_throughput: 3119.29 samples/s lr: 7.87e-04 [09/26 19:39:13] lb.utils.events INFO: eta: 11:39:17 iteration: 115299/375342 consumed_samples: 118067200 total_loss: 3.771 time: 0.3283 s/iter data_time: 0.2170 s/iter total_throughput: 3119.28 samples/s lr: 7.87e-04 [09/26 19:39:46] lb.utils.events INFO: eta: 11:39:01 iteration: 115399/375342 consumed_samples: 118169600 total_loss: 3.783 time: 0.3283 s/iter data_time: 0.2193 s/iter total_throughput: 3119.25 samples/s lr: 7.87e-04 [09/26 19:40:19] lb.utils.events INFO: eta: 11:47:18 iteration: 115499/375342 consumed_samples: 118272000 total_loss: 3.787 time: 0.3283 s/iter data_time: 0.2538 s/iter total_throughput: 3119.28 samples/s lr: 7.86e-04 [09/26 19:40:52] lb.utils.events INFO: eta: 12:07:04 iteration: 115599/375342 consumed_samples: 118374400 total_loss: 3.769 time: 0.3283 s/iter data_time: 0.2386 s/iter total_throughput: 3119.28 samples/s lr: 7.86e-04 [09/26 19:41:25] lb.utils.events INFO: eta: 11:50:29 iteration: 115699/375342 consumed_samples: 118476800 total_loss: 3.761 time: 0.3283 s/iter data_time: 0.2054 s/iter total_throughput: 3119.22 samples/s lr: 7.85e-04 [09/26 19:41:58] lb.utils.events INFO: eta: 12:04:28 iteration: 115799/375342 consumed_samples: 118579200 total_loss: 3.766 time: 0.3283 s/iter data_time: 0.2276 s/iter total_throughput: 3119.20 samples/s lr: 7.85e-04 [09/26 19:42:31] lb.utils.events INFO: eta: 12:06:46 iteration: 115899/375342 consumed_samples: 118681600 total_loss: 3.786 time: 0.3283 s/iter data_time: 0.2156 s/iter total_throughput: 3119.19 samples/s lr: 7.85e-04 [09/26 19:43:04] lb.utils.events INFO: eta: 12:04:32 iteration: 115999/375342 consumed_samples: 118784000 total_loss: 3.805 time: 0.3283 s/iter data_time: 0.2048 s/iter total_throughput: 3119.15 samples/s lr: 7.84e-04 [09/26 19:43:38] lb.utils.events INFO: eta: 11:45:29 iteration: 116099/375342 consumed_samples: 118886400 total_loss: 3.803 time: 0.3283 s/iter data_time: 0.2278 s/iter total_throughput: 3119.13 samples/s lr: 7.84e-04 [09/26 19:44:11] lb.utils.events INFO: eta: 11:47:09 iteration: 116199/375342 consumed_samples: 118988800 total_loss: 3.803 time: 0.3283 s/iter data_time: 0.2182 s/iter total_throughput: 3119.11 samples/s lr: 7.84e-04 [09/26 19:44:44] lb.utils.events INFO: eta: 11:47:26 iteration: 116299/375342 consumed_samples: 119091200 total_loss: 3.805 time: 0.3283 s/iter data_time: 0.2140 s/iter total_throughput: 3119.10 samples/s lr: 7.83e-04 [09/26 19:45:16] lb.utils.events INFO: eta: 11:41:37 iteration: 116399/375342 consumed_samples: 119193600 total_loss: 3.801 time: 0.3283 s/iter data_time: 0.2078 s/iter total_throughput: 3119.10 samples/s lr: 7.83e-04 [09/26 19:45:49] lb.utils.events INFO: eta: 11:31:22 iteration: 116499/375342 consumed_samples: 119296000 total_loss: 3.767 time: 0.3283 s/iter data_time: 0.2393 s/iter total_throughput: 3119.09 samples/s lr: 7.83e-04 [09/26 19:46:23] lb.utils.events INFO: eta: 11:21:41 iteration: 116599/375342 consumed_samples: 119398400 total_loss: 3.764 time: 0.3283 s/iter data_time: 0.2351 s/iter total_throughput: 3119.06 samples/s lr: 7.82e-04 [09/26 19:46:56] lb.utils.events INFO: eta: 11:24:45 iteration: 116699/375342 consumed_samples: 119500800 total_loss: 3.765 time: 0.3283 s/iter data_time: 0.2186 s/iter total_throughput: 3119.05 samples/s lr: 7.82e-04 [09/26 19:47:29] lb.utils.events INFO: eta: 11:23:33 iteration: 116799/375342 consumed_samples: 119603200 total_loss: 3.771 time: 0.3283 s/iter data_time: 0.2162 s/iter total_throughput: 3119.04 samples/s lr: 7.82e-04 [09/26 19:48:01] lb.utils.events INFO: eta: 11:27:11 iteration: 116899/375342 consumed_samples: 119705600 total_loss: 3.771 time: 0.3283 s/iter data_time: 0.2135 s/iter total_throughput: 3119.06 samples/s lr: 7.81e-04 [09/26 19:48:34] lb.utils.events INFO: eta: 11:39:43 iteration: 116999/375342 consumed_samples: 119808000 total_loss: 3.761 time: 0.3283 s/iter data_time: 0.2242 s/iter total_throughput: 3119.04 samples/s lr: 7.81e-04 [09/26 19:49:07] lb.utils.events INFO: eta: 11:45:21 iteration: 117099/375342 consumed_samples: 119910400 total_loss: 3.75 time: 0.3283 s/iter data_time: 0.2301 s/iter total_throughput: 3119.02 samples/s lr: 7.81e-04 [09/26 19:49:41] lb.utils.events INFO: eta: 11:37:44 iteration: 117199/375342 consumed_samples: 120012800 total_loss: 3.755 time: 0.3283 s/iter data_time: 0.2159 s/iter total_throughput: 3118.99 samples/s lr: 7.80e-04 [09/26 19:50:14] lb.utils.events INFO: eta: 11:27:24 iteration: 117299/375342 consumed_samples: 120115200 total_loss: 3.756 time: 0.3283 s/iter data_time: 0.2061 s/iter total_throughput: 3118.98 samples/s lr: 7.80e-04 [09/26 19:50:47] lb.utils.events INFO: eta: 11:41:11 iteration: 117399/375342 consumed_samples: 120217600 total_loss: 3.765 time: 0.3283 s/iter data_time: 0.2422 s/iter total_throughput: 3118.94 samples/s lr: 7.80e-04 [09/26 19:51:20] lb.utils.events INFO: eta: 11:39:35 iteration: 117499/375342 consumed_samples: 120320000 total_loss: 3.779 time: 0.3283 s/iter data_time: 0.2257 s/iter total_throughput: 3118.95 samples/s lr: 7.79e-04 [09/26 19:51:53] lb.utils.events INFO: eta: 11:46:08 iteration: 117599/375342 consumed_samples: 120422400 total_loss: 3.771 time: 0.3283 s/iter data_time: 0.2283 s/iter total_throughput: 3118.93 samples/s lr: 7.79e-04 [09/26 19:52:26] lb.utils.events INFO: eta: 11:49:37 iteration: 117699/375342 consumed_samples: 120524800 total_loss: 3.75 time: 0.3283 s/iter data_time: 0.2395 s/iter total_throughput: 3118.92 samples/s lr: 7.79e-04 [09/26 19:52:59] lb.utils.events INFO: eta: 11:55:58 iteration: 117799/375342 consumed_samples: 120627200 total_loss: 3.734 time: 0.3283 s/iter data_time: 0.2330 s/iter total_throughput: 3118.89 samples/s lr: 7.78e-04 [09/26 19:53:32] lb.utils.events INFO: eta: 11:54:26 iteration: 117899/375342 consumed_samples: 120729600 total_loss: 3.756 time: 0.3283 s/iter data_time: 0.2164 s/iter total_throughput: 3118.90 samples/s lr: 7.78e-04 [09/26 19:54:04] lb.utils.events INFO: eta: 11:48:12 iteration: 117999/375342 consumed_samples: 120832000 total_loss: 3.775 time: 0.3283 s/iter data_time: 0.2272 s/iter total_throughput: 3118.90 samples/s lr: 7.78e-04 [09/26 19:54:38] lb.utils.events INFO: eta: 11:39:03 iteration: 118099/375342 consumed_samples: 120934400 total_loss: 3.768 time: 0.3283 s/iter data_time: 0.2093 s/iter total_throughput: 3118.86 samples/s lr: 7.77e-04 [09/26 19:55:10] lb.utils.events INFO: eta: 11:43:13 iteration: 118199/375342 consumed_samples: 121036800 total_loss: 3.762 time: 0.3283 s/iter data_time: 0.2223 s/iter total_throughput: 3118.88 samples/s lr: 7.77e-04 [09/26 19:55:44] lb.utils.events INFO: eta: 11:54:11 iteration: 118299/375342 consumed_samples: 121139200 total_loss: 3.759 time: 0.3283 s/iter data_time: 0.2292 s/iter total_throughput: 3118.83 samples/s lr: 7.77e-04 [09/26 19:56:17] lb.utils.events INFO: eta: 11:54:45 iteration: 118399/375342 consumed_samples: 121241600 total_loss: 3.749 time: 0.3283 s/iter data_time: 0.2386 s/iter total_throughput: 3118.82 samples/s lr: 7.76e-04 [09/26 19:56:49] lb.utils.events INFO: eta: 12:06:33 iteration: 118499/375342 consumed_samples: 121344000 total_loss: 3.75 time: 0.3283 s/iter data_time: 0.2340 s/iter total_throughput: 3118.83 samples/s lr: 7.76e-04 [09/26 19:57:22] lb.utils.events INFO: eta: 12:23:53 iteration: 118599/375342 consumed_samples: 121446400 total_loss: 3.766 time: 0.3283 s/iter data_time: 0.2275 s/iter total_throughput: 3118.83 samples/s lr: 7.75e-04 [09/26 19:57:55] lb.utils.events INFO: eta: 12:09:21 iteration: 118699/375342 consumed_samples: 121548800 total_loss: 3.766 time: 0.3283 s/iter data_time: 0.2282 s/iter total_throughput: 3118.80 samples/s lr: 7.75e-04 [09/26 19:58:28] lb.utils.events INFO: eta: 12:15:30 iteration: 118799/375342 consumed_samples: 121651200 total_loss: 3.741 time: 0.3283 s/iter data_time: 0.2215 s/iter total_throughput: 3118.81 samples/s lr: 7.75e-04 [09/26 19:59:02] lb.utils.events INFO: eta: 11:46:35 iteration: 118899/375342 consumed_samples: 121753600 total_loss: 3.756 time: 0.3283 s/iter data_time: 0.1966 s/iter total_throughput: 3118.77 samples/s lr: 7.74e-04 [09/26 19:59:34] lb.utils.events INFO: eta: 11:42:59 iteration: 118999/375342 consumed_samples: 121856000 total_loss: 3.762 time: 0.3283 s/iter data_time: 0.2258 s/iter total_throughput: 3118.77 samples/s lr: 7.74e-04 [09/26 20:00:07] lb.utils.events INFO: eta: 11:53:58 iteration: 119099/375342 consumed_samples: 121958400 total_loss: 3.742 time: 0.3283 s/iter data_time: 0.2245 s/iter total_throughput: 3118.77 samples/s lr: 7.74e-04 [09/26 20:00:40] lb.utils.events INFO: eta: 12:07:39 iteration: 119199/375342 consumed_samples: 122060800 total_loss: 3.746 time: 0.3283 s/iter data_time: 0.2142 s/iter total_throughput: 3118.75 samples/s lr: 7.73e-04 [09/26 20:01:13] lb.utils.events INFO: eta: 12:05:01 iteration: 119299/375342 consumed_samples: 122163200 total_loss: 3.781 time: 0.3283 s/iter data_time: 0.2358 s/iter total_throughput: 3118.76 samples/s lr: 7.73e-04 [09/26 20:01:46] lb.utils.events INFO: eta: 12:03:19 iteration: 119399/375342 consumed_samples: 122265600 total_loss: 3.783 time: 0.3283 s/iter data_time: 0.2267 s/iter total_throughput: 3118.75 samples/s lr: 7.73e-04 [09/26 20:02:19] lb.utils.events INFO: eta: 11:44:01 iteration: 119499/375342 consumed_samples: 122368000 total_loss: 3.753 time: 0.3283 s/iter data_time: 0.2155 s/iter total_throughput: 3118.76 samples/s lr: 7.72e-04 [09/26 20:02:52] lb.utils.events INFO: eta: 11:28:38 iteration: 119599/375342 consumed_samples: 122470400 total_loss: 3.756 time: 0.3283 s/iter data_time: 0.2148 s/iter total_throughput: 3118.75 samples/s lr: 7.72e-04 [09/26 20:03:25] lb.utils.events INFO: eta: 11:28:15 iteration: 119699/375342 consumed_samples: 122572800 total_loss: 3.777 time: 0.3283 s/iter data_time: 0.2137 s/iter total_throughput: 3118.72 samples/s lr: 7.72e-04 [09/26 20:03:58] lb.utils.events INFO: eta: 11:36:42 iteration: 119799/375342 consumed_samples: 122675200 total_loss: 3.76 time: 0.3283 s/iter data_time: 0.2445 s/iter total_throughput: 3118.71 samples/s lr: 7.71e-04 [09/26 20:04:31] lb.utils.events INFO: eta: 11:51:37 iteration: 119899/375342 consumed_samples: 122777600 total_loss: 3.741 time: 0.3283 s/iter data_time: 0.2139 s/iter total_throughput: 3118.67 samples/s lr: 7.71e-04 [09/26 20:05:04] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0119999 [09/26 20:05:05] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/26 20:05:05] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/26 20:05:09] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0847 s/iter. Inference: 0.1477 s/iter. Eval: 0.0019 s/iter. Total: 0.2344 s/iter. ETA=0:00:08 [09/26 20:05:14] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1424 s/iter. Inference: 0.1502 s/iter. Eval: 0.0020 s/iter. Total: 0.2947 s/iter. ETA=0:00:05 [09/26 20:05:20] lb.evaluation.evaluator INFO: Inference done 50000/50000. Dataloading: 0.1308 s/iter. Inference: 0.1510 s/iter. Eval: 0.0020 s/iter. Total: 0.2839 s/iter. ETA=0:00:00 [09/26 20:05:20] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/26 20:05:20] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.497184 (0.000250 s / iter per device, on 8 devices) [09/26 20:05:20] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000133 s / iter per device, on 8 devices) [09/26 20:05:20] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/26 20:05:20] lb.evaluation.utils INFO: copypaste: Acc@1=72.162 [09/26 20:05:20] lb.evaluation.utils INFO: copypaste: Acc@5=91.042 [09/26 20:05:20] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 72.16200, better than last best score 71.89600 @ iteration 114999. [09/26 20:05:20] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/26 20:05:21] lb.utils.events INFO: eta: 11:41:55 iteration: 119999/375342 consumed_samples: 122880000 total_loss: 3.742 time: 0.3283 s/iter data_time: 0.2240 s/iter total_throughput: 3118.67 samples/s lr: 7.71e-04 [09/26 20:05:52] lb.utils.events INFO: eta: 11:38:17 iteration: 120099/375342 consumed_samples: 122982400 total_loss: 3.747 time: 0.3283 s/iter data_time: 0.2396 s/iter total_throughput: 3118.79 samples/s lr: 7.70e-04 [09/26 20:06:25] lb.utils.events INFO: eta: 11:29:26 iteration: 120199/375342 consumed_samples: 123084800 total_loss: 3.743 time: 0.3283 s/iter data_time: 0.2037 s/iter total_throughput: 3118.75 samples/s lr: 7.70e-04 [09/26 20:06:58] lb.utils.events INFO: eta: 11:22:53 iteration: 120299/375342 consumed_samples: 123187200 total_loss: 3.729 time: 0.3283 s/iter data_time: 0.2213 s/iter total_throughput: 3118.75 samples/s lr: 7.70e-04 [09/26 20:07:31] lb.utils.events INFO: eta: 11:15:34 iteration: 120399/375342 consumed_samples: 123289600 total_loss: 3.741 time: 0.3283 s/iter data_time: 0.2035 s/iter total_throughput: 3118.75 samples/s lr: 7.69e-04 [09/26 20:08:04] lb.utils.events INFO: eta: 11:17:28 iteration: 120499/375342 consumed_samples: 123392000 total_loss: 3.763 time: 0.3283 s/iter data_time: 0.2200 s/iter total_throughput: 3118.77 samples/s lr: 7.69e-04 [09/26 20:08:37] lb.utils.events INFO: eta: 11:19:06 iteration: 120599/375342 consumed_samples: 123494400 total_loss: 3.77 time: 0.3283 s/iter data_time: 0.2360 s/iter total_throughput: 3118.72 samples/s lr: 7.69e-04 [09/26 20:09:10] lb.utils.events INFO: eta: 11:19:34 iteration: 120699/375342 consumed_samples: 123596800 total_loss: 3.766 time: 0.3283 s/iter data_time: 0.2244 s/iter total_throughput: 3118.74 samples/s lr: 7.68e-04 [09/26 20:09:43] lb.utils.events INFO: eta: 11:13:25 iteration: 120799/375342 consumed_samples: 123699200 total_loss: 3.742 time: 0.3283 s/iter data_time: 0.1961 s/iter total_throughput: 3118.73 samples/s lr: 7.68e-04 [09/26 20:10:16] lb.utils.events INFO: eta: 11:11:42 iteration: 120899/375342 consumed_samples: 123801600 total_loss: 3.76 time: 0.3283 s/iter data_time: 0.2039 s/iter total_throughput: 3118.71 samples/s lr: 7.67e-04 [09/26 20:10:49] lb.utils.events INFO: eta: 11:08:13 iteration: 120999/375342 consumed_samples: 123904000 total_loss: 3.761 time: 0.3283 s/iter data_time: 0.2112 s/iter total_throughput: 3118.69 samples/s lr: 7.67e-04 [09/26 20:11:22] lb.utils.events INFO: eta: 11:00:20 iteration: 121099/375342 consumed_samples: 124006400 total_loss: 3.753 time: 0.3283 s/iter data_time: 0.2007 s/iter total_throughput: 3118.69 samples/s lr: 7.67e-04 [09/26 20:11:54] lb.utils.events INFO: eta: 10:58:54 iteration: 121199/375342 consumed_samples: 124108800 total_loss: 3.762 time: 0.3283 s/iter data_time: 0.2101 s/iter total_throughput: 3118.70 samples/s lr: 7.66e-04 [09/26 20:12:27] lb.utils.events INFO: eta: 10:58:36 iteration: 121299/375342 consumed_samples: 124211200 total_loss: 3.771 time: 0.3283 s/iter data_time: 0.2029 s/iter total_throughput: 3118.69 samples/s lr: 7.66e-04 [09/26 20:13:00] lb.utils.events INFO: eta: 11:00:33 iteration: 121399/375342 consumed_samples: 124313600 total_loss: 3.776 time: 0.3283 s/iter data_time: 0.2266 s/iter total_throughput: 3118.69 samples/s lr: 7.66e-04 [09/26 20:13:33] lb.utils.events INFO: eta: 10:59:43 iteration: 121499/375342 consumed_samples: 124416000 total_loss: 3.753 time: 0.3283 s/iter data_time: 0.2129 s/iter total_throughput: 3118.69 samples/s lr: 7.65e-04 [09/26 20:14:06] lb.utils.events INFO: eta: 10:58:18 iteration: 121599/375342 consumed_samples: 124518400 total_loss: 3.741 time: 0.3283 s/iter data_time: 0.2267 s/iter total_throughput: 3118.72 samples/s lr: 7.65e-04 [09/26 20:14:39] lb.utils.events INFO: eta: 10:54:42 iteration: 121699/375342 consumed_samples: 124620800 total_loss: 3.738 time: 0.3283 s/iter data_time: 0.2183 s/iter total_throughput: 3118.66 samples/s lr: 7.65e-04 [09/26 20:15:12] lb.utils.events INFO: eta: 10:52:47 iteration: 121799/375342 consumed_samples: 124723200 total_loss: 3.76 time: 0.3283 s/iter data_time: 0.2035 s/iter total_throughput: 3118.66 samples/s lr: 7.64e-04 [09/26 20:15:45] lb.utils.events INFO: eta: 10:51:32 iteration: 121899/375342 consumed_samples: 124825600 total_loss: 3.771 time: 0.3283 s/iter data_time: 0.2057 s/iter total_throughput: 3118.64 samples/s lr: 7.64e-04 [09/26 20:16:18] lb.utils.events INFO: eta: 10:53:56 iteration: 121999/375342 consumed_samples: 124928000 total_loss: 3.751 time: 0.3283 s/iter data_time: 0.2079 s/iter total_throughput: 3118.66 samples/s lr: 7.64e-04 [09/26 20:16:51] lb.utils.events INFO: eta: 10:57:06 iteration: 122099/375342 consumed_samples: 125030400 total_loss: 3.737 time: 0.3283 s/iter data_time: 0.2144 s/iter total_throughput: 3118.65 samples/s lr: 7.63e-04 [09/26 20:17:23] lb.utils.events INFO: eta: 10:58:43 iteration: 122199/375342 consumed_samples: 125132800 total_loss: 3.736 time: 0.3283 s/iter data_time: 0.2200 s/iter total_throughput: 3118.67 samples/s lr: 7.63e-04 [09/26 20:17:56] lb.utils.events INFO: eta: 11:01:21 iteration: 122299/375342 consumed_samples: 125235200 total_loss: 3.743 time: 0.3283 s/iter data_time: 0.2181 s/iter total_throughput: 3118.64 samples/s lr: 7.63e-04 [09/26 20:18:29] lb.utils.events INFO: eta: 11:04:27 iteration: 122399/375342 consumed_samples: 125337600 total_loss: 3.757 time: 0.3283 s/iter data_time: 0.2165 s/iter total_throughput: 3118.66 samples/s lr: 7.62e-04 [09/26 20:19:02] lb.utils.events INFO: eta: 11:08:01 iteration: 122499/375342 consumed_samples: 125440000 total_loss: 3.774 time: 0.3283 s/iter data_time: 0.2186 s/iter total_throughput: 3118.67 samples/s lr: 7.62e-04 [09/26 20:19:34] lb.utils.events INFO: eta: 11:12:51 iteration: 122599/375342 consumed_samples: 125542400 total_loss: 3.773 time: 0.3283 s/iter data_time: 0.2218 s/iter total_throughput: 3118.69 samples/s lr: 7.61e-04 [09/26 20:20:07] lb.utils.events INFO: eta: 11:23:21 iteration: 122699/375342 consumed_samples: 125644800 total_loss: 3.73 time: 0.3283 s/iter data_time: 0.2230 s/iter total_throughput: 3118.67 samples/s lr: 7.61e-04 [09/26 20:20:40] lb.utils.events INFO: eta: 11:34:56 iteration: 122799/375342 consumed_samples: 125747200 total_loss: 3.712 time: 0.3283 s/iter data_time: 0.2333 s/iter total_throughput: 3118.70 samples/s lr: 7.61e-04 [09/26 20:21:12] lb.utils.events INFO: eta: 11:57:27 iteration: 122899/375342 consumed_samples: 125849600 total_loss: 3.73 time: 0.3283 s/iter data_time: 0.2362 s/iter total_throughput: 3118.73 samples/s lr: 7.60e-04 [09/26 20:21:45] lb.utils.events INFO: eta: 12:26:32 iteration: 122999/375342 consumed_samples: 125952000 total_loss: 3.734 time: 0.3283 s/iter data_time: 0.2022 s/iter total_throughput: 3118.71 samples/s lr: 7.60e-04 [09/26 20:22:18] lb.utils.events INFO: eta: 12:23:48 iteration: 123099/375342 consumed_samples: 126054400 total_loss: 3.746 time: 0.3283 s/iter data_time: 0.2159 s/iter total_throughput: 3118.73 samples/s lr: 7.60e-04 [09/26 20:22:51] lb.utils.events INFO: eta: 12:48:55 iteration: 123199/375342 consumed_samples: 126156800 total_loss: 3.761 time: 0.3283 s/iter data_time: 0.2449 s/iter total_throughput: 3118.73 samples/s lr: 7.59e-04 [09/26 20:23:24] lb.utils.events INFO: eta: 15:11:31 iteration: 123299/375342 consumed_samples: 126259200 total_loss: 3.732 time: 0.3283 s/iter data_time: 0.2232 s/iter total_throughput: 3118.73 samples/s lr: 7.59e-04 [09/26 20:23:57] lb.utils.events INFO: eta: 15:37:16 iteration: 123399/375342 consumed_samples: 126361600 total_loss: 3.722 time: 0.3283 s/iter data_time: 0.2394 s/iter total_throughput: 3118.73 samples/s lr: 7.59e-04 [09/26 20:24:29] lb.utils.events INFO: eta: 17:16:29 iteration: 123499/375342 consumed_samples: 126464000 total_loss: 3.751 time: 0.3283 s/iter data_time: 0.2338 s/iter total_throughput: 3118.74 samples/s lr: 7.58e-04 [09/26 20:25:02] lb.utils.events INFO: eta: 18:40:34 iteration: 123599/375342 consumed_samples: 126566400 total_loss: 3.772 time: 0.3283 s/iter data_time: 0.2422 s/iter total_throughput: 3118.73 samples/s lr: 7.58e-04 [09/26 20:25:35] lb.utils.events INFO: eta: 18:05:59 iteration: 123699/375342 consumed_samples: 126668800 total_loss: 3.759 time: 0.3283 s/iter data_time: 0.2269 s/iter total_throughput: 3118.71 samples/s lr: 7.58e-04 [09/26 20:26:08] lb.utils.events INFO: eta: 19:24:20 iteration: 123799/375342 consumed_samples: 126771200 total_loss: 3.744 time: 0.3283 s/iter data_time: 0.2354 s/iter total_throughput: 3118.74 samples/s lr: 7.57e-04 [09/26 20:26:41] lb.utils.events INFO: eta: 17:59:29 iteration: 123899/375342 consumed_samples: 126873600 total_loss: 3.758 time: 0.3283 s/iter data_time: 0.2078 s/iter total_throughput: 3118.74 samples/s lr: 7.57e-04 [09/26 20:27:14] lb.utils.events INFO: eta: 16:07:24 iteration: 123999/375342 consumed_samples: 126976000 total_loss: 3.751 time: 0.3283 s/iter data_time: 0.2041 s/iter total_throughput: 3118.73 samples/s lr: 7.56e-04 [09/26 20:27:46] lb.utils.events INFO: eta: 16:29:14 iteration: 124099/375342 consumed_samples: 127078400 total_loss: 3.736 time: 0.3283 s/iter data_time: 0.2170 s/iter total_throughput: 3118.75 samples/s lr: 7.56e-04 [09/26 20:28:19] lb.utils.events INFO: eta: 12:28:28 iteration: 124199/375342 consumed_samples: 127180800 total_loss: 3.754 time: 0.3283 s/iter data_time: 0.2239 s/iter total_throughput: 3118.75 samples/s lr: 7.56e-04 [09/26 20:28:52] lb.utils.events INFO: eta: 11:30:43 iteration: 124299/375342 consumed_samples: 127283200 total_loss: 3.749 time: 0.3283 s/iter data_time: 0.2085 s/iter total_throughput: 3118.73 samples/s lr: 7.55e-04 [09/26 20:29:24] lb.utils.events INFO: eta: 11:15:29 iteration: 124399/375342 consumed_samples: 127385600 total_loss: 3.749 time: 0.3283 s/iter data_time: 0.2083 s/iter total_throughput: 3118.80 samples/s lr: 7.55e-04 [09/26 20:29:57] lb.utils.events INFO: eta: 11:06:56 iteration: 124499/375342 consumed_samples: 127488000 total_loss: 3.752 time: 0.3283 s/iter data_time: 0.2232 s/iter total_throughput: 3118.79 samples/s lr: 7.55e-04 [09/26 20:30:29] lb.utils.events INFO: eta: 10:59:29 iteration: 124599/375342 consumed_samples: 127590400 total_loss: 3.754 time: 0.3283 s/iter data_time: 0.2380 s/iter total_throughput: 3118.83 samples/s lr: 7.54e-04 [09/26 20:31:02] lb.utils.events INFO: eta: 10:59:28 iteration: 124699/375342 consumed_samples: 127692800 total_loss: 3.752 time: 0.3283 s/iter data_time: 0.2229 s/iter total_throughput: 3118.84 samples/s lr: 7.54e-04 [09/26 20:31:35] lb.utils.events INFO: eta: 10:59:39 iteration: 124799/375342 consumed_samples: 127795200 total_loss: 3.739 time: 0.3283 s/iter data_time: 0.2243 s/iter total_throughput: 3118.85 samples/s lr: 7.54e-04 [09/26 20:32:08] lb.utils.events INFO: eta: 11:01:20 iteration: 124899/375342 consumed_samples: 127897600 total_loss: 3.752 time: 0.3283 s/iter data_time: 0.2380 s/iter total_throughput: 3118.85 samples/s lr: 7.53e-04 [09/26 20:32:40] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0124999 [09/26 20:32:41] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/26 20:32:41] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/26 20:32:45] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0943 s/iter. Inference: 0.1492 s/iter. Eval: 0.0020 s/iter. Total: 0.2454 s/iter. ETA=0:00:09 [09/26 20:32:51] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1520 s/iter. Inference: 0.1490 s/iter. Eval: 0.0020 s/iter. Total: 0.3031 s/iter. ETA=0:00:05 [09/26 20:32:56] lb.evaluation.evaluator INFO: Inference done 50000/50000. Dataloading: 0.1366 s/iter. Inference: 0.1501 s/iter. Eval: 0.0020 s/iter. Total: 0.2889 s/iter. ETA=0:00:00 [09/26 20:32:56] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/26 20:32:56] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.713756 (0.000254 s / iter per device, on 8 devices) [09/26 20:32:56] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000132 s / iter per device, on 8 devices) [09/26 20:32:56] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/26 20:32:56] lb.evaluation.utils INFO: copypaste: Acc@1=72.466 [09/26 20:32:56] lb.evaluation.utils INFO: copypaste: Acc@5=91.006 [09/26 20:32:56] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 72.46600, better than last best score 72.16200 @ iteration 119999. [09/26 20:32:56] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/26 20:32:57] lb.utils.events INFO: eta: 11:07:01 iteration: 124999/375342 consumed_samples: 128000000 total_loss: 3.752 time: 0.3283 s/iter data_time: 0.2203 s/iter total_throughput: 3118.87 samples/s lr: 7.53e-04 [09/26 20:33:28] lb.utils.events INFO: eta: 11:10:06 iteration: 125099/375342 consumed_samples: 128102400 total_loss: 3.739 time: 0.3283 s/iter data_time: 0.2592 s/iter total_throughput: 3119.01 samples/s lr: 7.53e-04 [09/26 20:34:01] lb.utils.events INFO: eta: 11:22:16 iteration: 125199/375342 consumed_samples: 128204800 total_loss: 3.723 time: 0.3283 s/iter data_time: 0.2091 s/iter total_throughput: 3118.99 samples/s lr: 7.52e-04 [09/26 20:34:33] lb.utils.events INFO: eta: 11:27:22 iteration: 125299/375342 consumed_samples: 128307200 total_loss: 3.727 time: 0.3283 s/iter data_time: 0.2152 s/iter total_throughput: 3119.04 samples/s lr: 7.52e-04 [09/26 20:35:06] lb.utils.events INFO: eta: 11:39:16 iteration: 125399/375342 consumed_samples: 128409600 total_loss: 3.723 time: 0.3283 s/iter data_time: 0.2221 s/iter total_throughput: 3119.05 samples/s lr: 7.51e-04 [09/26 20:35:38] lb.utils.events INFO: eta: 12:21:58 iteration: 125499/375342 consumed_samples: 128512000 total_loss: 3.707 time: 0.3283 s/iter data_time: 0.2125 s/iter total_throughput: 3119.12 samples/s lr: 7.51e-04 [09/26 20:36:10] lb.utils.events INFO: eta: 14:12:14 iteration: 125599/375342 consumed_samples: 128614400 total_loss: 3.742 time: 0.3283 s/iter data_time: 0.2391 s/iter total_throughput: 3119.15 samples/s lr: 7.51e-04 [09/26 20:36:43] lb.utils.events INFO: eta: 13:10:46 iteration: 125699/375342 consumed_samples: 128716800 total_loss: 3.76 time: 0.3283 s/iter data_time: 0.2069 s/iter total_throughput: 3119.17 samples/s lr: 7.50e-04 [09/26 20:37:15] lb.utils.events INFO: eta: 13:01:15 iteration: 125799/375342 consumed_samples: 128819200 total_loss: 3.732 time: 0.3283 s/iter data_time: 0.2141 s/iter total_throughput: 3119.20 samples/s lr: 7.50e-04 [09/26 20:37:48] lb.utils.events INFO: eta: 13:00:56 iteration: 125899/375342 consumed_samples: 128921600 total_loss: 3.729 time: 0.3283 s/iter data_time: 0.2222 s/iter total_throughput: 3119.24 samples/s lr: 7.50e-04 [09/26 20:38:20] lb.utils.events INFO: eta: 12:10:56 iteration: 125999/375342 consumed_samples: 129024000 total_loss: 3.752 time: 0.3283 s/iter data_time: 0.2217 s/iter total_throughput: 3119.25 samples/s lr: 7.49e-04 [09/26 20:38:53] lb.utils.events INFO: eta: 11:34:37 iteration: 126099/375342 consumed_samples: 129126400 total_loss: 3.752 time: 0.3283 s/iter data_time: 0.2063 s/iter total_throughput: 3119.26 samples/s lr: 7.49e-04 [09/26 20:39:27] lb.utils.events INFO: eta: 11:15:19 iteration: 126199/375342 consumed_samples: 129228800 total_loss: 3.737 time: 0.3283 s/iter data_time: 0.2144 s/iter total_throughput: 3119.18 samples/s lr: 7.49e-04 [09/26 20:39:59] lb.utils.events INFO: eta: 11:15:03 iteration: 126299/375342 consumed_samples: 129331200 total_loss: 3.742 time: 0.3283 s/iter data_time: 0.2129 s/iter total_throughput: 3119.21 samples/s lr: 7.48e-04 [09/26 20:40:33] lb.utils.events INFO: eta: 11:20:20 iteration: 126399/375342 consumed_samples: 129433600 total_loss: 3.728 time: 0.3283 s/iter data_time: 0.2444 s/iter total_throughput: 3119.17 samples/s lr: 7.48e-04 [09/26 20:41:06] lb.utils.events INFO: eta: 11:09:54 iteration: 126499/375342 consumed_samples: 129536000 total_loss: 3.738 time: 0.3283 s/iter data_time: 0.2056 s/iter total_throughput: 3119.10 samples/s lr: 7.48e-04 [09/26 20:41:39] lb.utils.events INFO: eta: 11:03:20 iteration: 126599/375342 consumed_samples: 129638400 total_loss: 3.761 time: 0.3283 s/iter data_time: 0.2499 s/iter total_throughput: 3119.11 samples/s lr: 7.47e-04 [09/26 20:42:13] lb.utils.events INFO: eta: 11:10:08 iteration: 126699/375342 consumed_samples: 129740800 total_loss: 3.741 time: 0.3283 s/iter data_time: 0.2423 s/iter total_throughput: 3119.05 samples/s lr: 7.47e-04 [09/26 20:42:47] lb.utils.events INFO: eta: 11:10:11 iteration: 126799/375342 consumed_samples: 129843200 total_loss: 3.734 time: 0.3283 s/iter data_time: 0.2304 s/iter total_throughput: 3118.97 samples/s lr: 7.46e-04 [09/26 20:43:20] lb.utils.events INFO: eta: 11:04:32 iteration: 126899/375342 consumed_samples: 129945600 total_loss: 3.735 time: 0.3283 s/iter data_time: 0.2309 s/iter total_throughput: 3118.93 samples/s lr: 7.46e-04 [09/26 20:43:54] lb.utils.events INFO: eta: 11:03:18 iteration: 126999/375342 consumed_samples: 130048000 total_loss: 3.726 time: 0.3283 s/iter data_time: 0.2079 s/iter total_throughput: 3118.88 samples/s lr: 7.46e-04 [09/26 20:44:27] lb.utils.events INFO: eta: 11:00:59 iteration: 127099/375342 consumed_samples: 130150400 total_loss: 3.742 time: 0.3283 s/iter data_time: 0.2063 s/iter total_throughput: 3118.85 samples/s lr: 7.45e-04 [09/26 20:45:00] lb.utils.events INFO: eta: 11:03:02 iteration: 127199/375342 consumed_samples: 130252800 total_loss: 3.752 time: 0.3283 s/iter data_time: 0.2191 s/iter total_throughput: 3118.81 samples/s lr: 7.45e-04 [09/26 20:45:33] lb.utils.events INFO: eta: 10:59:54 iteration: 127299/375342 consumed_samples: 130355200 total_loss: 3.737 time: 0.3283 s/iter data_time: 0.2229 s/iter total_throughput: 3118.82 samples/s lr: 7.45e-04 [09/26 20:46:07] lb.utils.events INFO: eta: 10:57:57 iteration: 127399/375342 consumed_samples: 130457600 total_loss: 3.733 time: 0.3283 s/iter data_time: 0.2338 s/iter total_throughput: 3118.76 samples/s lr: 7.44e-04 [09/26 20:46:39] lb.utils.events INFO: eta: 11:02:17 iteration: 127499/375342 consumed_samples: 130560000 total_loss: 3.746 time: 0.3283 s/iter data_time: 0.2243 s/iter total_throughput: 3118.76 samples/s lr: 7.44e-04 [09/26 20:47:13] lb.utils.events INFO: eta: 10:57:26 iteration: 127599/375342 consumed_samples: 130662400 total_loss: 3.743 time: 0.3283 s/iter data_time: 0.2166 s/iter total_throughput: 3118.72 samples/s lr: 7.44e-04 [09/26 20:47:46] lb.utils.events INFO: eta: 10:49:23 iteration: 127699/375342 consumed_samples: 130764800 total_loss: 3.732 time: 0.3283 s/iter data_time: 0.2078 s/iter total_throughput: 3118.69 samples/s lr: 7.43e-04 [09/26 20:48:19] lb.utils.events INFO: eta: 10:44:40 iteration: 127799/375342 consumed_samples: 130867200 total_loss: 3.738 time: 0.3283 s/iter data_time: 0.2250 s/iter total_throughput: 3118.69 samples/s lr: 7.43e-04 [09/26 20:48:52] lb.utils.events INFO: eta: 10:47:28 iteration: 127899/375342 consumed_samples: 130969600 total_loss: 3.756 time: 0.3283 s/iter data_time: 0.2178 s/iter total_throughput: 3118.69 samples/s lr: 7.42e-04 [09/26 20:49:25] lb.utils.events INFO: eta: 10:48:36 iteration: 127999/375342 consumed_samples: 131072000 total_loss: 3.751 time: 0.3283 s/iter data_time: 0.2228 s/iter total_throughput: 3118.66 samples/s lr: 7.42e-04 [09/26 20:49:58] lb.utils.events INFO: eta: 10:54:18 iteration: 128099/375342 consumed_samples: 131174400 total_loss: 3.73 time: 0.3283 s/iter data_time: 0.2227 s/iter total_throughput: 3118.64 samples/s lr: 7.42e-04 [09/26 20:50:31] lb.utils.events INFO: eta: 10:53:41 iteration: 128199/375342 consumed_samples: 131276800 total_loss: 3.723 time: 0.3283 s/iter data_time: 0.2327 s/iter total_throughput: 3118.63 samples/s lr: 7.41e-04 [09/26 20:51:05] lb.utils.events INFO: eta: 10:52:02 iteration: 128299/375342 consumed_samples: 131379200 total_loss: 3.721 time: 0.3284 s/iter data_time: 0.2164 s/iter total_throughput: 3118.55 samples/s lr: 7.41e-04 [09/26 20:51:38] lb.utils.events INFO: eta: 10:47:36 iteration: 128399/375342 consumed_samples: 131481600 total_loss: 3.741 time: 0.3284 s/iter data_time: 0.2083 s/iter total_throughput: 3118.53 samples/s lr: 7.41e-04 [09/26 20:52:11] lb.utils.events INFO: eta: 10:44:30 iteration: 128499/375342 consumed_samples: 131584000 total_loss: 3.721 time: 0.3284 s/iter data_time: 0.2101 s/iter total_throughput: 3118.55 samples/s lr: 7.40e-04 [09/26 20:52:44] lb.utils.events INFO: eta: 10:46:08 iteration: 128599/375342 consumed_samples: 131686400 total_loss: 3.73 time: 0.3284 s/iter data_time: 0.2184 s/iter total_throughput: 3118.53 samples/s lr: 7.40e-04 [09/26 20:53:17] lb.utils.events INFO: eta: 10:48:19 iteration: 128699/375342 consumed_samples: 131788800 total_loss: 3.766 time: 0.3284 s/iter data_time: 0.2090 s/iter total_throughput: 3118.52 samples/s lr: 7.40e-04 [09/26 20:53:50] lb.utils.events INFO: eta: 10:50:13 iteration: 128799/375342 consumed_samples: 131891200 total_loss: 3.744 time: 0.3284 s/iter data_time: 0.2237 s/iter total_throughput: 3118.50 samples/s lr: 7.39e-04 [09/26 20:54:23] lb.utils.events INFO: eta: 10:54:17 iteration: 128899/375342 consumed_samples: 131993600 total_loss: 3.746 time: 0.3284 s/iter data_time: 0.2377 s/iter total_throughput: 3118.49 samples/s lr: 7.39e-04 [09/26 20:54:56] lb.utils.events INFO: eta: 10:52:47 iteration: 128999/375342 consumed_samples: 132096000 total_loss: 3.745 time: 0.3284 s/iter data_time: 0.2167 s/iter total_throughput: 3118.48 samples/s lr: 7.38e-04 [09/26 20:55:29] lb.utils.events INFO: eta: 10:48:00 iteration: 129099/375342 consumed_samples: 132198400 total_loss: 3.728 time: 0.3284 s/iter data_time: 0.2181 s/iter total_throughput: 3118.47 samples/s lr: 7.38e-04 [09/26 20:56:02] lb.utils.events INFO: eta: 10:52:24 iteration: 129199/375342 consumed_samples: 132300800 total_loss: 3.712 time: 0.3284 s/iter data_time: 0.2255 s/iter total_throughput: 3118.45 samples/s lr: 7.38e-04 [09/26 20:56:35] lb.utils.events INFO: eta: 10:55:47 iteration: 129299/375342 consumed_samples: 132403200 total_loss: 3.732 time: 0.3284 s/iter data_time: 0.2116 s/iter total_throughput: 3118.43 samples/s lr: 7.37e-04 [09/26 20:57:08] lb.utils.events INFO: eta: 11:05:25 iteration: 129399/375342 consumed_samples: 132505600 total_loss: 3.749 time: 0.3284 s/iter data_time: 0.2389 s/iter total_throughput: 3118.43 samples/s lr: 7.37e-04 [09/26 20:57:41] lb.utils.events INFO: eta: 11:26:15 iteration: 129499/375342 consumed_samples: 132608000 total_loss: 3.747 time: 0.3284 s/iter data_time: 0.2318 s/iter total_throughput: 3118.38 samples/s lr: 7.37e-04 [09/26 20:58:15] lb.utils.events INFO: eta: 11:14:04 iteration: 129599/375342 consumed_samples: 132710400 total_loss: 3.757 time: 0.3284 s/iter data_time: 0.2053 s/iter total_throughput: 3118.33 samples/s lr: 7.36e-04 [09/26 20:58:48] lb.utils.events INFO: eta: 11:11:08 iteration: 129699/375342 consumed_samples: 132812800 total_loss: 3.736 time: 0.3284 s/iter data_time: 0.2167 s/iter total_throughput: 3118.30 samples/s lr: 7.36e-04 [09/26 20:59:21] lb.utils.events INFO: eta: 11:05:10 iteration: 129799/375342 consumed_samples: 132915200 total_loss: 3.719 time: 0.3284 s/iter data_time: 0.2046 s/iter total_throughput: 3118.28 samples/s lr: 7.36e-04 [09/26 20:59:54] lb.utils.events INFO: eta: 10:55:45 iteration: 129899/375342 consumed_samples: 133017600 total_loss: 3.731 time: 0.3284 s/iter data_time: 0.2094 s/iter total_throughput: 3118.27 samples/s lr: 7.35e-04 [09/26 21:00:27] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0129999 [09/26 21:00:28] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/26 21:00:28] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/26 21:00:32] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0872 s/iter. Inference: 0.1478 s/iter. Eval: 0.0020 s/iter. Total: 0.2371 s/iter. ETA=0:00:08 [09/26 21:00:38] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1463 s/iter. Inference: 0.1484 s/iter. Eval: 0.0019 s/iter. Total: 0.2968 s/iter. ETA=0:00:05 [09/26 21:00:43] lb.evaluation.evaluator INFO: Inference done 50000/50000. Dataloading: 0.1324 s/iter. Inference: 0.1502 s/iter. Eval: 0.0020 s/iter. Total: 0.2846 s/iter. ETA=0:00:00 [09/26 21:00:43] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/26 21:00:43] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.526400 (0.000251 s / iter per device, on 8 devices) [09/26 21:00:43] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000132 s / iter per device, on 8 devices) [09/26 21:00:43] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/26 21:00:43] lb.evaluation.utils INFO: copypaste: Acc@1=72.36399999999999 [09/26 21:00:43] lb.evaluation.utils INFO: copypaste: Acc@5=91.082 [09/26 21:00:43] lb.engine.hooks INFO: Not saving as latest eval score for Acc@1 is 72.36400, not better than best score 72.46600 @ iteration 124999. [09/26 21:00:43] lb.utils.events INFO: eta: 10:48:51 iteration: 129999/375342 consumed_samples: 133120000 total_loss: 3.748 time: 0.3284 s/iter data_time: 0.2082 s/iter total_throughput: 3118.27 samples/s lr: 7.35e-04 [09/26 21:01:15] lb.utils.events INFO: eta: 10:53:14 iteration: 130099/375342 consumed_samples: 133222400 total_loss: 3.75 time: 0.3284 s/iter data_time: 0.2382 s/iter total_throughput: 3118.36 samples/s lr: 7.34e-04 [09/26 21:01:48] lb.utils.events INFO: eta: 10:52:58 iteration: 130199/375342 consumed_samples: 133324800 total_loss: 3.738 time: 0.3284 s/iter data_time: 0.2245 s/iter total_throughput: 3118.34 samples/s lr: 7.34e-04 [09/26 21:02:21] lb.utils.events INFO: eta: 10:54:30 iteration: 130299/375342 consumed_samples: 133427200 total_loss: 3.74 time: 0.3284 s/iter data_time: 0.2294 s/iter total_throughput: 3118.34 samples/s lr: 7.34e-04 [09/26 21:02:54] lb.utils.events INFO: eta: 10:56:11 iteration: 130399/375342 consumed_samples: 133529600 total_loss: 3.74 time: 0.3284 s/iter data_time: 0.2501 s/iter total_throughput: 3118.31 samples/s lr: 7.33e-04 [09/26 21:03:27] lb.utils.events INFO: eta: 10:51:01 iteration: 130499/375342 consumed_samples: 133632000 total_loss: 3.741 time: 0.3284 s/iter data_time: 0.2357 s/iter total_throughput: 3118.29 samples/s lr: 7.33e-04 [09/26 21:04:00] lb.utils.events INFO: eta: 10:53:44 iteration: 130599/375342 consumed_samples: 133734400 total_loss: 3.757 time: 0.3284 s/iter data_time: 0.2155 s/iter total_throughput: 3118.26 samples/s lr: 7.33e-04 [09/26 21:04:34] lb.utils.events INFO: eta: 10:54:36 iteration: 130699/375342 consumed_samples: 133836800 total_loss: 3.764 time: 0.3284 s/iter data_time: 0.2121 s/iter total_throughput: 3118.19 samples/s lr: 7.32e-04 [09/26 21:05:07] lb.utils.events INFO: eta: 10:58:15 iteration: 130799/375342 consumed_samples: 133939200 total_loss: 3.736 time: 0.3284 s/iter data_time: 0.2381 s/iter total_throughput: 3118.17 samples/s lr: 7.32e-04 [09/26 21:05:40] lb.utils.events INFO: eta: 11:03:55 iteration: 130899/375342 consumed_samples: 134041600 total_loss: 3.707 time: 0.3284 s/iter data_time: 0.2275 s/iter total_throughput: 3118.16 samples/s lr: 7.31e-04 [09/26 21:06:13] lb.utils.events INFO: eta: 11:31:58 iteration: 130999/375342 consumed_samples: 134144000 total_loss: 3.734 time: 0.3284 s/iter data_time: 0.2266 s/iter total_throughput: 3118.14 samples/s lr: 7.31e-04 [09/26 21:06:46] lb.utils.events INFO: eta: 11:54:35 iteration: 131099/375342 consumed_samples: 134246400 total_loss: 3.747 time: 0.3284 s/iter data_time: 0.2385 s/iter total_throughput: 3118.13 samples/s lr: 7.31e-04 [09/26 21:07:19] lb.utils.events INFO: eta: 12:57:41 iteration: 131199/375342 consumed_samples: 134348800 total_loss: 3.756 time: 0.3284 s/iter data_time: 0.2493 s/iter total_throughput: 3118.12 samples/s lr: 7.30e-04 [09/26 21:07:52] lb.utils.events INFO: eta: 13:47:26 iteration: 131299/375342 consumed_samples: 134451200 total_loss: 3.747 time: 0.3284 s/iter data_time: 0.2356 s/iter total_throughput: 3118.11 samples/s lr: 7.30e-04 [09/26 21:08:25] lb.utils.events INFO: eta: 11:51:27 iteration: 131399/375342 consumed_samples: 134553600 total_loss: 3.723 time: 0.3284 s/iter data_time: 0.2252 s/iter total_throughput: 3118.09 samples/s lr: 7.30e-04 [09/26 21:08:58] lb.utils.events INFO: eta: 12:24:49 iteration: 131499/375342 consumed_samples: 134656000 total_loss: 3.726 time: 0.3284 s/iter data_time: 0.2307 s/iter total_throughput: 3118.07 samples/s lr: 7.29e-04 [09/26 21:09:32] lb.utils.events INFO: eta: 12:27:24 iteration: 131599/375342 consumed_samples: 134758400 total_loss: 3.745 time: 0.3284 s/iter data_time: 0.2220 s/iter total_throughput: 3118.05 samples/s lr: 7.29e-04 [09/26 21:10:05] lb.utils.events INFO: eta: 12:44:51 iteration: 131699/375342 consumed_samples: 134860800 total_loss: 3.722 time: 0.3284 s/iter data_time: 0.2311 s/iter total_throughput: 3118.01 samples/s lr: 7.29e-04 [09/26 21:10:38] lb.utils.events INFO: eta: 14:02:24 iteration: 131799/375342 consumed_samples: 134963200 total_loss: 3.741 time: 0.3284 s/iter data_time: 0.2445 s/iter total_throughput: 3117.99 samples/s lr: 7.28e-04 [09/26 21:11:11] lb.utils.events INFO: eta: 15:04:42 iteration: 131899/375342 consumed_samples: 135065600 total_loss: 3.747 time: 0.3284 s/iter data_time: 0.2331 s/iter total_throughput: 3117.98 samples/s lr: 7.28e-04 [09/26 21:11:44] lb.utils.events INFO: eta: 14:07:25 iteration: 131999/375342 consumed_samples: 135168000 total_loss: 3.733 time: 0.3284 s/iter data_time: 0.2015 s/iter total_throughput: 3117.96 samples/s lr: 7.27e-04 [09/26 21:12:17] lb.utils.events INFO: eta: 12:22:29 iteration: 132099/375342 consumed_samples: 135270400 total_loss: 3.712 time: 0.3284 s/iter data_time: 0.2351 s/iter total_throughput: 3117.94 samples/s lr: 7.27e-04 [09/26 21:12:50] lb.utils.events INFO: eta: 11:34:24 iteration: 132199/375342 consumed_samples: 135372800 total_loss: 3.726 time: 0.3284 s/iter data_time: 0.2208 s/iter total_throughput: 3117.93 samples/s lr: 7.27e-04 [09/26 21:13:23] lb.utils.events INFO: eta: 11:23:59 iteration: 132299/375342 consumed_samples: 135475200 total_loss: 3.737 time: 0.3284 s/iter data_time: 0.2426 s/iter total_throughput: 3117.92 samples/s lr: 7.26e-04 [09/26 21:13:56] lb.utils.events INFO: eta: 12:23:58 iteration: 132399/375342 consumed_samples: 135577600 total_loss: 3.721 time: 0.3284 s/iter data_time: 0.2443 s/iter total_throughput: 3117.92 samples/s lr: 7.26e-04 [09/26 21:14:29] lb.utils.events INFO: eta: 14:15:31 iteration: 132499/375342 consumed_samples: 135680000 total_loss: 3.706 time: 0.3284 s/iter data_time: 0.2629 s/iter total_throughput: 3117.93 samples/s lr: 7.26e-04 [09/26 21:15:02] lb.utils.events INFO: eta: 16:22:09 iteration: 132599/375342 consumed_samples: 135782400 total_loss: 3.711 time: 0.3284 s/iter data_time: 0.2427 s/iter total_throughput: 3117.93 samples/s lr: 7.25e-04 [09/26 21:15:35] lb.utils.events INFO: eta: 16:40:30 iteration: 132699/375342 consumed_samples: 135884800 total_loss: 3.71 time: 0.3284 s/iter data_time: 0.2214 s/iter total_throughput: 3117.90 samples/s lr: 7.25e-04 [09/26 21:16:08] lb.utils.events INFO: eta: 16:48:56 iteration: 132799/375342 consumed_samples: 135987200 total_loss: 3.716 time: 0.3284 s/iter data_time: 0.2307 s/iter total_throughput: 3117.88 samples/s lr: 7.24e-04 [09/26 21:16:41] lb.utils.events INFO: eta: 16:12:43 iteration: 132899/375342 consumed_samples: 136089600 total_loss: 3.72 time: 0.3284 s/iter data_time: 0.2251 s/iter total_throughput: 3117.87 samples/s lr: 7.24e-04 [09/26 21:17:15] lb.utils.events INFO: eta: 15:39:52 iteration: 132999/375342 consumed_samples: 136192000 total_loss: 3.709 time: 0.3284 s/iter data_time: 0.2113 s/iter total_throughput: 3117.82 samples/s lr: 7.24e-04 [09/26 21:17:48] lb.utils.events INFO: eta: 15:16:35 iteration: 133099/375342 consumed_samples: 136294400 total_loss: 3.709 time: 0.3284 s/iter data_time: 0.2229 s/iter total_throughput: 3117.80 samples/s lr: 7.23e-04 [09/26 21:18:21] lb.utils.events INFO: eta: 15:14:49 iteration: 133199/375342 consumed_samples: 136396800 total_loss: 3.712 time: 0.3284 s/iter data_time: 0.2252 s/iter total_throughput: 3117.81 samples/s lr: 7.23e-04 [09/26 21:18:53] lb.utils.events INFO: eta: 14:05:49 iteration: 133299/375342 consumed_samples: 136499200 total_loss: 3.709 time: 0.3284 s/iter data_time: 0.2375 s/iter total_throughput: 3117.81 samples/s lr: 7.23e-04 [09/26 21:19:27] lb.utils.events INFO: eta: 13:20:22 iteration: 133399/375342 consumed_samples: 136601600 total_loss: 3.713 time: 0.3284 s/iter data_time: 0.2451 s/iter total_throughput: 3117.77 samples/s lr: 7.22e-04 [09/26 21:20:00] lb.utils.events INFO: eta: 12:04:04 iteration: 133499/375342 consumed_samples: 136704000 total_loss: 3.704 time: 0.3284 s/iter data_time: 0.2414 s/iter total_throughput: 3117.73 samples/s lr: 7.22e-04 [09/26 21:20:33] lb.utils.events INFO: eta: 11:10:09 iteration: 133599/375342 consumed_samples: 136806400 total_loss: 3.713 time: 0.3284 s/iter data_time: 0.2312 s/iter total_throughput: 3117.72 samples/s lr: 7.21e-04 [09/26 21:21:07] lb.utils.events INFO: eta: 11:01:11 iteration: 133699/375342 consumed_samples: 136908800 total_loss: 3.729 time: 0.3285 s/iter data_time: 0.2111 s/iter total_throughput: 3117.67 samples/s lr: 7.21e-04 [09/26 21:21:40] lb.utils.events INFO: eta: 10:45:35 iteration: 133799/375342 consumed_samples: 137011200 total_loss: 3.714 time: 0.3285 s/iter data_time: 0.1992 s/iter total_throughput: 3117.66 samples/s lr: 7.21e-04 [09/26 21:22:13] lb.utils.events INFO: eta: 10:36:34 iteration: 133899/375342 consumed_samples: 137113600 total_loss: 3.711 time: 0.3285 s/iter data_time: 0.2078 s/iter total_throughput: 3117.66 samples/s lr: 7.20e-04 [09/26 21:22:45] lb.utils.events INFO: eta: 10:38:28 iteration: 133999/375342 consumed_samples: 137216000 total_loss: 3.69 time: 0.3284 s/iter data_time: 0.2133 s/iter total_throughput: 3117.68 samples/s lr: 7.20e-04 [09/26 21:23:18] lb.utils.events INFO: eta: 10:53:35 iteration: 134099/375342 consumed_samples: 137318400 total_loss: 3.703 time: 0.3284 s/iter data_time: 0.2725 s/iter total_throughput: 3117.69 samples/s lr: 7.20e-04 [09/26 21:23:51] lb.utils.events INFO: eta: 11:12:28 iteration: 134199/375342 consumed_samples: 137420800 total_loss: 3.738 time: 0.3285 s/iter data_time: 0.2433 s/iter total_throughput: 3117.66 samples/s lr: 7.19e-04 [09/26 21:24:24] lb.utils.events INFO: eta: 11:34:54 iteration: 134299/375342 consumed_samples: 137523200 total_loss: 3.723 time: 0.3285 s/iter data_time: 0.2557 s/iter total_throughput: 3117.64 samples/s lr: 7.19e-04 [09/26 21:24:57] lb.utils.events INFO: eta: 11:42:47 iteration: 134399/375342 consumed_samples: 137625600 total_loss: 3.716 time: 0.3285 s/iter data_time: 0.2368 s/iter total_throughput: 3117.65 samples/s lr: 7.18e-04 [09/26 21:25:30] lb.utils.events INFO: eta: 11:46:14 iteration: 134499/375342 consumed_samples: 137728000 total_loss: 3.715 time: 0.3285 s/iter data_time: 0.2361 s/iter total_throughput: 3117.66 samples/s lr: 7.18e-04 [09/26 21:26:03] lb.utils.events INFO: eta: 11:45:00 iteration: 134599/375342 consumed_samples: 137830400 total_loss: 3.716 time: 0.3285 s/iter data_time: 0.2065 s/iter total_throughput: 3117.66 samples/s lr: 7.18e-04 [09/26 21:26:36] lb.utils.events INFO: eta: 11:31:26 iteration: 134699/375342 consumed_samples: 137932800 total_loss: 3.723 time: 0.3285 s/iter data_time: 0.2120 s/iter total_throughput: 3117.63 samples/s lr: 7.17e-04 [09/26 21:27:09] lb.utils.events INFO: eta: 11:46:53 iteration: 134799/375342 consumed_samples: 138035200 total_loss: 3.724 time: 0.3285 s/iter data_time: 0.2183 s/iter total_throughput: 3117.64 samples/s lr: 7.17e-04 [09/26 21:27:42] lb.utils.events INFO: eta: 11:46:35 iteration: 134899/375342 consumed_samples: 138137600 total_loss: 3.734 time: 0.3285 s/iter data_time: 0.1951 s/iter total_throughput: 3117.63 samples/s lr: 7.17e-04 [09/26 21:28:15] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0134999 [09/26 21:28:15] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/26 21:28:15] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/26 21:28:19] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0840 s/iter. Inference: 0.1490 s/iter. Eval: 0.0019 s/iter. Total: 0.2349 s/iter. ETA=0:00:08 [09/26 21:28:25] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1431 s/iter. Inference: 0.1494 s/iter. Eval: 0.0020 s/iter. Total: 0.2946 s/iter. ETA=0:00:05 [09/26 21:28:30] lb.evaluation.evaluator INFO: Inference done 49152/50000. Dataloading: 0.1308 s/iter. Inference: 0.1484 s/iter. Eval: 0.0021 s/iter. Total: 0.2814 s/iter. ETA=0:00:00 [09/26 21:28:30] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/26 21:28:30] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.477687 (0.000250 s / iter per device, on 8 devices) [09/26 21:28:30] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000132 s / iter per device, on 8 devices) [09/26 21:28:30] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/26 21:28:30] lb.evaluation.utils INFO: copypaste: Acc@1=72.71600000000001 [09/26 21:28:30] lb.evaluation.utils INFO: copypaste: Acc@5=91.146 [09/26 21:28:30] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 72.71600, better than last best score 72.46600 @ iteration 124999. [09/26 21:28:30] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/26 21:28:31] lb.utils.events INFO: eta: 11:58:33 iteration: 134999/375342 consumed_samples: 138240000 total_loss: 3.736 time: 0.3285 s/iter data_time: 0.2245 s/iter total_throughput: 3117.64 samples/s lr: 7.16e-04 [09/26 21:29:02] lb.utils.events INFO: eta: 11:25:11 iteration: 135099/375342 consumed_samples: 138342400 total_loss: 3.744 time: 0.3284 s/iter data_time: 0.2385 s/iter total_throughput: 3117.74 samples/s lr: 7.16e-04 [09/26 21:29:35] lb.utils.events INFO: eta: 10:48:43 iteration: 135199/375342 consumed_samples: 138444800 total_loss: 3.735 time: 0.3284 s/iter data_time: 0.2078 s/iter total_throughput: 3117.73 samples/s lr: 7.15e-04 [09/26 21:30:08] lb.utils.events INFO: eta: 10:40:20 iteration: 135299/375342 consumed_samples: 138547200 total_loss: 3.71 time: 0.3284 s/iter data_time: 0.2222 s/iter total_throughput: 3117.73 samples/s lr: 7.15e-04 [09/26 21:30:41] lb.utils.events INFO: eta: 10:30:02 iteration: 135399/375342 consumed_samples: 138649600 total_loss: 3.704 time: 0.3284 s/iter data_time: 0.2212 s/iter total_throughput: 3117.74 samples/s lr: 7.15e-04 [09/26 21:31:14] lb.utils.events INFO: eta: 10:25:09 iteration: 135499/375342 consumed_samples: 138752000 total_loss: 3.729 time: 0.3284 s/iter data_time: 0.2393 s/iter total_throughput: 3117.72 samples/s lr: 7.14e-04 [09/26 21:31:47] lb.utils.events INFO: eta: 10:23:39 iteration: 135599/375342 consumed_samples: 138854400 total_loss: 3.723 time: 0.3284 s/iter data_time: 0.2313 s/iter total_throughput: 3117.73 samples/s lr: 7.14e-04 [09/26 21:32:19] lb.utils.events INFO: eta: 10:32:24 iteration: 135699/375342 consumed_samples: 138956800 total_loss: 3.728 time: 0.3284 s/iter data_time: 0.2367 s/iter total_throughput: 3117.74 samples/s lr: 7.14e-04 [09/26 21:32:52] lb.utils.events INFO: eta: 10:36:05 iteration: 135799/375342 consumed_samples: 139059200 total_loss: 3.732 time: 0.3284 s/iter data_time: 0.2162 s/iter total_throughput: 3117.73 samples/s lr: 7.13e-04 [09/26 21:33:25] lb.utils.events INFO: eta: 10:39:48 iteration: 135899/375342 consumed_samples: 139161600 total_loss: 3.722 time: 0.3284 s/iter data_time: 0.2225 s/iter total_throughput: 3117.73 samples/s lr: 7.13e-04 [09/26 21:33:58] lb.utils.events INFO: eta: 10:37:21 iteration: 135999/375342 consumed_samples: 139264000 total_loss: 3.719 time: 0.3284 s/iter data_time: 0.2331 s/iter total_throughput: 3117.70 samples/s lr: 7.12e-04 [09/26 21:34:31] lb.utils.events INFO: eta: 10:36:34 iteration: 136099/375342 consumed_samples: 139366400 total_loss: 3.718 time: 0.3284 s/iter data_time: 0.2221 s/iter total_throughput: 3117.72 samples/s lr: 7.12e-04 [09/26 21:35:04] lb.utils.events INFO: eta: 10:52:47 iteration: 136199/375342 consumed_samples: 139468800 total_loss: 3.731 time: 0.3284 s/iter data_time: 0.2216 s/iter total_throughput: 3117.74 samples/s lr: 7.12e-04 [09/26 21:35:36] lb.utils.events INFO: eta: 10:58:01 iteration: 136299/375342 consumed_samples: 139571200 total_loss: 3.731 time: 0.3284 s/iter data_time: 0.2373 s/iter total_throughput: 3117.75 samples/s lr: 7.11e-04 [09/26 21:36:09] lb.utils.events INFO: eta: 11:51:46 iteration: 136399/375342 consumed_samples: 139673600 total_loss: 3.723 time: 0.3284 s/iter data_time: 0.2376 s/iter total_throughput: 3117.76 samples/s lr: 7.11e-04 [09/26 21:36:42] lb.utils.events INFO: eta: 12:15:03 iteration: 136499/375342 consumed_samples: 139776000 total_loss: 3.72 time: 0.3284 s/iter data_time: 0.2296 s/iter total_throughput: 3117.76 samples/s lr: 7.11e-04 [09/26 21:37:15] lb.utils.events INFO: eta: 11:47:53 iteration: 136599/375342 consumed_samples: 139878400 total_loss: 3.73 time: 0.3284 s/iter data_time: 0.2205 s/iter total_throughput: 3117.75 samples/s lr: 7.10e-04 [09/26 21:37:48] lb.utils.events INFO: eta: 11:29:10 iteration: 136699/375342 consumed_samples: 139980800 total_loss: 3.721 time: 0.3284 s/iter data_time: 0.2594 s/iter total_throughput: 3117.75 samples/s lr: 7.10e-04 [09/26 21:38:20] lb.utils.events INFO: eta: 11:55:13 iteration: 136799/375342 consumed_samples: 140083200 total_loss: 3.713 time: 0.3284 s/iter data_time: 0.2211 s/iter total_throughput: 3117.78 samples/s lr: 7.09e-04 [09/26 21:38:53] lb.utils.events INFO: eta: 11:10:49 iteration: 136899/375342 consumed_samples: 140185600 total_loss: 3.703 time: 0.3284 s/iter data_time: 0.2138 s/iter total_throughput: 3117.80 samples/s lr: 7.09e-04 [09/26 21:39:26] lb.utils.events INFO: eta: 11:31:58 iteration: 136999/375342 consumed_samples: 140288000 total_loss: 3.706 time: 0.3284 s/iter data_time: 0.2303 s/iter total_throughput: 3117.80 samples/s lr: 7.09e-04 [09/26 21:39:58] lb.utils.events INFO: eta: 11:07:43 iteration: 137099/375342 consumed_samples: 140390400 total_loss: 3.721 time: 0.3284 s/iter data_time: 0.2257 s/iter total_throughput: 3117.80 samples/s lr: 7.08e-04 [09/26 21:40:31] lb.utils.events INFO: eta: 10:47:21 iteration: 137199/375342 consumed_samples: 140492800 total_loss: 3.729 time: 0.3284 s/iter data_time: 0.2076 s/iter total_throughput: 3117.79 samples/s lr: 7.08e-04 [09/26 21:41:05] lb.utils.events INFO: eta: 10:37:53 iteration: 137299/375342 consumed_samples: 140595200 total_loss: 3.711 time: 0.3284 s/iter data_time: 0.2153 s/iter total_throughput: 3117.77 samples/s lr: 7.08e-04 [09/26 21:41:37] lb.utils.events INFO: eta: 10:26:32 iteration: 137399/375342 consumed_samples: 140697600 total_loss: 3.691 time: 0.3284 s/iter data_time: 0.1990 s/iter total_throughput: 3117.78 samples/s lr: 7.07e-04 [09/26 21:42:10] lb.utils.events INFO: eta: 10:21:57 iteration: 137499/375342 consumed_samples: 140800000 total_loss: 3.7 time: 0.3284 s/iter data_time: 0.2034 s/iter total_throughput: 3117.78 samples/s lr: 7.07e-04 [09/26 21:42:43] lb.utils.events INFO: eta: 10:18:14 iteration: 137599/375342 consumed_samples: 140902400 total_loss: 3.711 time: 0.3284 s/iter data_time: 0.2138 s/iter total_throughput: 3117.79 samples/s lr: 7.06e-04 [09/26 21:43:15] lb.utils.events INFO: eta: 10:13:26 iteration: 137699/375342 consumed_samples: 141004800 total_loss: 3.707 time: 0.3284 s/iter data_time: 0.1917 s/iter total_throughput: 3117.81 samples/s lr: 7.06e-04 [09/26 21:43:48] lb.utils.events INFO: eta: 10:11:31 iteration: 137799/375342 consumed_samples: 141107200 total_loss: 3.686 time: 0.3284 s/iter data_time: 0.2055 s/iter total_throughput: 3117.80 samples/s lr: 7.06e-04 [09/26 21:44:21] lb.utils.events INFO: eta: 10:10:14 iteration: 137899/375342 consumed_samples: 141209600 total_loss: 3.665 time: 0.3284 s/iter data_time: 0.1897 s/iter total_throughput: 3117.82 samples/s lr: 7.05e-04 [09/26 21:44:53] lb.utils.events INFO: eta: 10:06:36 iteration: 137999/375342 consumed_samples: 141312000 total_loss: 3.695 time: 0.3284 s/iter data_time: 0.2290 s/iter total_throughput: 3117.85 samples/s lr: 7.05e-04 [09/26 21:45:26] lb.utils.events INFO: eta: 10:06:26 iteration: 138099/375342 consumed_samples: 141414400 total_loss: 3.706 time: 0.3284 s/iter data_time: 0.2260 s/iter total_throughput: 3117.88 samples/s lr: 7.05e-04 [09/26 21:45:58] lb.utils.events INFO: eta: 10:08:30 iteration: 138199/375342 consumed_samples: 141516800 total_loss: 3.721 time: 0.3284 s/iter data_time: 0.2234 s/iter total_throughput: 3117.89 samples/s lr: 7.04e-04 [09/26 21:46:31] lb.utils.events INFO: eta: 10:11:35 iteration: 138299/375342 consumed_samples: 141619200 total_loss: 3.712 time: 0.3284 s/iter data_time: 0.2175 s/iter total_throughput: 3117.91 samples/s lr: 7.04e-04 [09/26 21:47:04] lb.utils.events INFO: eta: 10:13:20 iteration: 138399/375342 consumed_samples: 141721600 total_loss: 3.701 time: 0.3284 s/iter data_time: 0.2103 s/iter total_throughput: 3117.92 samples/s lr: 7.03e-04 [09/26 21:47:36] lb.utils.events INFO: eta: 10:17:12 iteration: 138499/375342 consumed_samples: 141824000 total_loss: 3.7 time: 0.3284 s/iter data_time: 0.2372 s/iter total_throughput: 3117.93 samples/s lr: 7.03e-04 [09/26 21:48:09] lb.utils.events INFO: eta: 10:23:57 iteration: 138599/375342 consumed_samples: 141926400 total_loss: 3.699 time: 0.3284 s/iter data_time: 0.2289 s/iter total_throughput: 3117.92 samples/s lr: 7.03e-04 [09/26 21:48:42] lb.utils.events INFO: eta: 10:25:04 iteration: 138699/375342 consumed_samples: 142028800 total_loss: 3.717 time: 0.3284 s/iter data_time: 0.2096 s/iter total_throughput: 3117.92 samples/s lr: 7.02e-04 [09/26 21:49:14] lb.utils.events INFO: eta: 10:24:47 iteration: 138799/375342 consumed_samples: 142131200 total_loss: 3.719 time: 0.3284 s/iter data_time: 0.2004 s/iter total_throughput: 3117.98 samples/s lr: 7.02e-04 [09/26 21:49:47] lb.utils.events INFO: eta: 10:25:09 iteration: 138899/375342 consumed_samples: 142233600 total_loss: 3.701 time: 0.3284 s/iter data_time: 0.1996 s/iter total_throughput: 3118.02 samples/s lr: 7.02e-04 [09/26 21:50:20] lb.utils.events INFO: eta: 10:28:04 iteration: 138999/375342 consumed_samples: 142336000 total_loss: 3.698 time: 0.3284 s/iter data_time: 0.2247 s/iter total_throughput: 3118.01 samples/s lr: 7.01e-04 [09/26 21:50:52] lb.utils.events INFO: eta: 10:22:51 iteration: 139099/375342 consumed_samples: 142438400 total_loss: 3.707 time: 0.3284 s/iter data_time: 0.2047 s/iter total_throughput: 3118.02 samples/s lr: 7.01e-04 [09/26 21:51:24] lb.utils.events INFO: eta: 10:22:01 iteration: 139199/375342 consumed_samples: 142540800 total_loss: 3.706 time: 0.3284 s/iter data_time: 0.2248 s/iter total_throughput: 3118.07 samples/s lr: 7.00e-04 [09/26 21:51:57] lb.utils.events INFO: eta: 10:22:50 iteration: 139299/375342 consumed_samples: 142643200 total_loss: 3.709 time: 0.3284 s/iter data_time: 0.1996 s/iter total_throughput: 3118.10 samples/s lr: 7.00e-04 [09/26 21:52:30] lb.utils.events INFO: eta: 10:23:08 iteration: 139399/375342 consumed_samples: 142745600 total_loss: 3.709 time: 0.3284 s/iter data_time: 0.2377 s/iter total_throughput: 3118.11 samples/s lr: 7.00e-04 [09/26 21:53:03] lb.utils.events INFO: eta: 10:21:30 iteration: 139499/375342 consumed_samples: 142848000 total_loss: 3.699 time: 0.3284 s/iter data_time: 0.2341 s/iter total_throughput: 3118.08 samples/s lr: 6.99e-04 [09/26 21:53:36] lb.utils.events INFO: eta: 10:13:41 iteration: 139599/375342 consumed_samples: 142950400 total_loss: 3.723 time: 0.3284 s/iter data_time: 0.2057 s/iter total_throughput: 3118.08 samples/s lr: 6.99e-04 [09/26 21:54:08] lb.utils.events INFO: eta: 10:12:08 iteration: 139699/375342 consumed_samples: 143052800 total_loss: 3.709 time: 0.3284 s/iter data_time: 0.1919 s/iter total_throughput: 3118.11 samples/s lr: 6.98e-04 [09/26 21:54:40] lb.utils.events INFO: eta: 10:11:14 iteration: 139799/375342 consumed_samples: 143155200 total_loss: 3.709 time: 0.3284 s/iter data_time: 0.1933 s/iter total_throughput: 3118.15 samples/s lr: 6.98e-04 [09/26 21:55:12] lb.utils.events INFO: eta: 10:13:11 iteration: 139899/375342 consumed_samples: 143257600 total_loss: 3.715 time: 0.3284 s/iter data_time: 0.2466 s/iter total_throughput: 3118.21 samples/s lr: 6.98e-04 [09/26 21:55:45] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0139999 [09/26 21:55:46] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/26 21:55:46] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/26 21:55:50] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0863 s/iter. Inference: 0.1505 s/iter. Eval: 0.0022 s/iter. Total: 0.2390 s/iter. ETA=0:00:08 [09/26 21:55:55] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1401 s/iter. Inference: 0.1521 s/iter. Eval: 0.0020 s/iter. Total: 0.2943 s/iter. ETA=0:00:05 [09/26 21:56:01] lb.evaluation.evaluator INFO: Inference done 50000/50000. Dataloading: 0.1301 s/iter. Inference: 0.1508 s/iter. Eval: 0.0020 s/iter. Total: 0.2830 s/iter. ETA=0:00:00 [09/26 21:56:01] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/26 21:56:01] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.457069 (0.000249 s / iter per device, on 8 devices) [09/26 21:56:01] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000133 s / iter per device, on 8 devices) [09/26 21:56:01] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/26 21:56:01] lb.evaluation.utils INFO: copypaste: Acc@1=72.994 [09/26 21:56:01] lb.evaluation.utils INFO: copypaste: Acc@5=91.36999999999999 [09/26 21:56:01] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 72.99400, better than last best score 72.71600 @ iteration 134999. [09/26 21:56:01] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/26 21:56:01] lb.utils.events INFO: eta: 10:14:34 iteration: 139999/375342 consumed_samples: 143360000 total_loss: 3.688 time: 0.3284 s/iter data_time: 0.2032 s/iter total_throughput: 3118.24 samples/s lr: 6.97e-04 [09/26 21:56:33] lb.utils.events INFO: eta: 10:19:44 iteration: 140099/375342 consumed_samples: 143462400 total_loss: 3.708 time: 0.3284 s/iter data_time: 0.2340 s/iter total_throughput: 3118.33 samples/s lr: 6.97e-04 [09/26 21:57:06] lb.utils.events INFO: eta: 10:19:34 iteration: 140199/375342 consumed_samples: 143564800 total_loss: 3.725 time: 0.3284 s/iter data_time: 0.2283 s/iter total_throughput: 3118.29 samples/s lr: 6.97e-04 [09/26 21:57:40] lb.utils.events INFO: eta: 10:12:10 iteration: 140299/375342 consumed_samples: 143667200 total_loss: 3.726 time: 0.3284 s/iter data_time: 0.1996 s/iter total_throughput: 3118.27 samples/s lr: 6.96e-04 [09/26 21:58:13] lb.utils.events INFO: eta: 10:13:08 iteration: 140399/375342 consumed_samples: 143769600 total_loss: 3.691 time: 0.3284 s/iter data_time: 0.2166 s/iter total_throughput: 3118.24 samples/s lr: 6.96e-04 [09/26 21:58:46] lb.utils.events INFO: eta: 10:10:20 iteration: 140499/375342 consumed_samples: 143872000 total_loss: 3.688 time: 0.3284 s/iter data_time: 0.2098 s/iter total_throughput: 3118.20 samples/s lr: 6.95e-04 [09/26 21:59:20] lb.utils.events INFO: eta: 10:12:30 iteration: 140599/375342 consumed_samples: 143974400 total_loss: 3.718 time: 0.3284 s/iter data_time: 0.2110 s/iter total_throughput: 3118.17 samples/s lr: 6.95e-04 [09/26 21:59:54] lb.utils.events INFO: eta: 10:12:05 iteration: 140699/375342 consumed_samples: 144076800 total_loss: 3.692 time: 0.3284 s/iter data_time: 0.2218 s/iter total_throughput: 3118.09 samples/s lr: 6.95e-04 [09/26 22:00:27] lb.utils.events INFO: eta: 10:11:09 iteration: 140799/375342 consumed_samples: 144179200 total_loss: 3.691 time: 0.3284 s/iter data_time: 0.2057 s/iter total_throughput: 3118.08 samples/s lr: 6.94e-04 [09/26 22:00:59] lb.utils.events INFO: eta: 10:09:54 iteration: 140899/375342 consumed_samples: 144281600 total_loss: 3.7 time: 0.3284 s/iter data_time: 0.2082 s/iter total_throughput: 3118.08 samples/s lr: 6.94e-04 [09/26 22:01:33] lb.utils.events INFO: eta: 10:07:43 iteration: 140999/375342 consumed_samples: 144384000 total_loss: 3.705 time: 0.3284 s/iter data_time: 0.2117 s/iter total_throughput: 3118.06 samples/s lr: 6.93e-04 [09/26 22:02:06] lb.utils.events INFO: eta: 10:04:15 iteration: 141099/375342 consumed_samples: 144486400 total_loss: 3.703 time: 0.3284 s/iter data_time: 0.2292 s/iter total_throughput: 3118.06 samples/s lr: 6.93e-04 [09/26 22:02:39] lb.utils.events INFO: eta: 10:03:30 iteration: 141199/375342 consumed_samples: 144588800 total_loss: 3.699 time: 0.3284 s/iter data_time: 0.2255 s/iter total_throughput: 3118.02 samples/s lr: 6.93e-04 [09/26 22:03:12] lb.utils.events INFO: eta: 10:06:50 iteration: 141299/375342 consumed_samples: 144691200 total_loss: 3.702 time: 0.3284 s/iter data_time: 0.2327 s/iter total_throughput: 3117.99 samples/s lr: 6.92e-04 [09/26 22:03:45] lb.utils.events INFO: eta: 10:06:28 iteration: 141399/375342 consumed_samples: 144793600 total_loss: 3.693 time: 0.3284 s/iter data_time: 0.2055 s/iter total_throughput: 3117.96 samples/s lr: 6.92e-04 [09/26 22:04:18] lb.utils.events INFO: eta: 10:07:08 iteration: 141499/375342 consumed_samples: 144896000 total_loss: 3.691 time: 0.3284 s/iter data_time: 0.2212 s/iter total_throughput: 3117.95 samples/s lr: 6.92e-04 [09/26 22:04:52] lb.utils.events INFO: eta: 10:10:36 iteration: 141599/375342 consumed_samples: 144998400 total_loss: 3.692 time: 0.3284 s/iter data_time: 0.2462 s/iter total_throughput: 3117.93 samples/s lr: 6.91e-04 [09/26 22:05:25] lb.utils.events INFO: eta: 10:15:27 iteration: 141699/375342 consumed_samples: 145100800 total_loss: 3.688 time: 0.3284 s/iter data_time: 0.2250 s/iter total_throughput: 3117.89 samples/s lr: 6.91e-04 [09/26 22:05:58] lb.utils.events INFO: eta: 10:22:24 iteration: 141799/375342 consumed_samples: 145203200 total_loss: 3.688 time: 0.3284 s/iter data_time: 0.2401 s/iter total_throughput: 3117.89 samples/s lr: 6.90e-04 [09/26 22:06:31] lb.utils.events INFO: eta: 10:22:08 iteration: 141899/375342 consumed_samples: 145305600 total_loss: 3.721 time: 0.3284 s/iter data_time: 0.2010 s/iter total_throughput: 3117.90 samples/s lr: 6.90e-04 [09/26 22:07:05] lb.utils.events INFO: eta: 10:26:06 iteration: 141999/375342 consumed_samples: 145408000 total_loss: 3.729 time: 0.3284 s/iter data_time: 0.2393 s/iter total_throughput: 3117.83 samples/s lr: 6.90e-04 [09/26 22:07:38] lb.utils.events INFO: eta: 10:30:50 iteration: 142099/375342 consumed_samples: 145510400 total_loss: 3.678 time: 0.3284 s/iter data_time: 0.2312 s/iter total_throughput: 3117.81 samples/s lr: 6.89e-04 [09/26 22:08:11] lb.utils.events INFO: eta: 10:40:08 iteration: 142199/375342 consumed_samples: 145612800 total_loss: 3.671 time: 0.3284 s/iter data_time: 0.2494 s/iter total_throughput: 3117.78 samples/s lr: 6.89e-04 [09/26 22:08:44] lb.utils.events INFO: eta: 10:45:48 iteration: 142299/375342 consumed_samples: 145715200 total_loss: 3.707 time: 0.3284 s/iter data_time: 0.2207 s/iter total_throughput: 3117.79 samples/s lr: 6.88e-04 [09/26 22:09:17] lb.utils.events INFO: eta: 12:14:22 iteration: 142399/375342 consumed_samples: 145817600 total_loss: 3.7 time: 0.3284 s/iter data_time: 0.2631 s/iter total_throughput: 3117.79 samples/s lr: 6.88e-04 [09/26 22:09:49] lb.utils.events INFO: eta: 14:28:36 iteration: 142499/375342 consumed_samples: 145920000 total_loss: 3.692 time: 0.3284 s/iter data_time: 0.2659 s/iter total_throughput: 3117.79 samples/s lr: 6.88e-04 [09/26 22:10:23] lb.utils.events INFO: eta: 13:37:47 iteration: 142599/375342 consumed_samples: 146022400 total_loss: 3.709 time: 0.3284 s/iter data_time: 0.2236 s/iter total_throughput: 3117.73 samples/s lr: 6.87e-04 [09/26 22:10:57] lb.utils.events INFO: eta: 12:30:32 iteration: 142699/375342 consumed_samples: 146124800 total_loss: 3.696 time: 0.3284 s/iter data_time: 0.2087 s/iter total_throughput: 3117.70 samples/s lr: 6.87e-04 [09/26 22:11:29] lb.utils.events INFO: eta: 10:59:37 iteration: 142799/375342 consumed_samples: 146227200 total_loss: 3.674 time: 0.3284 s/iter data_time: 0.2152 s/iter total_throughput: 3117.73 samples/s lr: 6.87e-04 [09/26 22:12:02] lb.utils.events INFO: eta: 11:39:19 iteration: 142899/375342 consumed_samples: 146329600 total_loss: 3.673 time: 0.3284 s/iter data_time: 0.2538 s/iter total_throughput: 3117.73 samples/s lr: 6.86e-04 [09/26 22:12:35] lb.utils.events INFO: eta: 12:28:01 iteration: 142999/375342 consumed_samples: 146432000 total_loss: 3.699 time: 0.3284 s/iter data_time: 0.2623 s/iter total_throughput: 3117.72 samples/s lr: 6.86e-04 [09/26 22:13:08] lb.utils.events INFO: eta: 13:07:44 iteration: 143099/375342 consumed_samples: 146534400 total_loss: 3.702 time: 0.3284 s/iter data_time: 0.2215 s/iter total_throughput: 3117.68 samples/s lr: 6.85e-04 [09/26 22:13:41] lb.utils.events INFO: eta: 11:53:59 iteration: 143199/375342 consumed_samples: 146636800 total_loss: 3.673 time: 0.3284 s/iter data_time: 0.2201 s/iter total_throughput: 3117.69 samples/s lr: 6.85e-04 [09/26 22:14:14] lb.utils.events INFO: eta: 11:30:07 iteration: 143299/375342 consumed_samples: 146739200 total_loss: 3.69 time: 0.3285 s/iter data_time: 0.2208 s/iter total_throughput: 3117.67 samples/s lr: 6.85e-04 [09/26 22:14:47] lb.utils.events INFO: eta: 10:38:33 iteration: 143399/375342 consumed_samples: 146841600 total_loss: 3.708 time: 0.3285 s/iter data_time: 0.2089 s/iter total_throughput: 3117.64 samples/s lr: 6.84e-04 [09/26 22:15:21] lb.utils.events INFO: eta: 10:24:03 iteration: 143499/375342 consumed_samples: 146944000 total_loss: 3.67 time: 0.3285 s/iter data_time: 0.2303 s/iter total_throughput: 3117.62 samples/s lr: 6.84e-04 [09/26 22:15:53] lb.utils.events INFO: eta: 10:26:00 iteration: 143599/375342 consumed_samples: 147046400 total_loss: 3.687 time: 0.3285 s/iter data_time: 0.2497 s/iter total_throughput: 3117.63 samples/s lr: 6.83e-04 [09/26 22:16:27] lb.utils.events INFO: eta: 10:34:39 iteration: 143699/375342 consumed_samples: 147148800 total_loss: 3.712 time: 0.3285 s/iter data_time: 0.2251 s/iter total_throughput: 3117.59 samples/s lr: 6.83e-04 [09/26 22:17:00] lb.utils.events INFO: eta: 10:40:29 iteration: 143799/375342 consumed_samples: 147251200 total_loss: 3.714 time: 0.3285 s/iter data_time: 0.2453 s/iter total_throughput: 3117.55 samples/s lr: 6.83e-04 [09/26 22:17:33] lb.utils.events INFO: eta: 10:37:10 iteration: 143899/375342 consumed_samples: 147353600 total_loss: 3.699 time: 0.3285 s/iter data_time: 0.2277 s/iter total_throughput: 3117.52 samples/s lr: 6.82e-04 [09/26 22:18:07] lb.utils.events INFO: eta: 10:26:36 iteration: 143999/375342 consumed_samples: 147456000 total_loss: 3.683 time: 0.3285 s/iter data_time: 0.2237 s/iter total_throughput: 3117.50 samples/s lr: 6.82e-04 [09/26 22:18:40] lb.utils.events INFO: eta: 10:17:46 iteration: 144099/375342 consumed_samples: 147558400 total_loss: 3.656 time: 0.3285 s/iter data_time: 0.2095 s/iter total_throughput: 3117.46 samples/s lr: 6.82e-04 [09/26 22:19:13] lb.utils.events INFO: eta: 10:15:13 iteration: 144199/375342 consumed_samples: 147660800 total_loss: 3.673 time: 0.3285 s/iter data_time: 0.2095 s/iter total_throughput: 3117.45 samples/s lr: 6.81e-04 [09/26 22:19:46] lb.utils.events INFO: eta: 10:17:50 iteration: 144299/375342 consumed_samples: 147763200 total_loss: 3.694 time: 0.3285 s/iter data_time: 0.2280 s/iter total_throughput: 3117.47 samples/s lr: 6.81e-04 [09/26 22:20:19] lb.utils.events INFO: eta: 10:29:16 iteration: 144399/375342 consumed_samples: 147865600 total_loss: 3.702 time: 0.3285 s/iter data_time: 0.2360 s/iter total_throughput: 3117.47 samples/s lr: 6.80e-04 [09/26 22:20:51] lb.utils.events INFO: eta: 10:45:45 iteration: 144499/375342 consumed_samples: 147968000 total_loss: 3.692 time: 0.3285 s/iter data_time: 0.2358 s/iter total_throughput: 3117.47 samples/s lr: 6.80e-04 [09/26 22:21:25] lb.utils.events INFO: eta: 10:43:27 iteration: 144599/375342 consumed_samples: 148070400 total_loss: 3.687 time: 0.3285 s/iter data_time: 0.2334 s/iter total_throughput: 3117.45 samples/s lr: 6.80e-04 [09/26 22:21:58] lb.utils.events INFO: eta: 10:32:57 iteration: 144699/375342 consumed_samples: 148172800 total_loss: 3.679 time: 0.3285 s/iter data_time: 0.2124 s/iter total_throughput: 3117.43 samples/s lr: 6.79e-04 [09/26 22:22:31] lb.utils.events INFO: eta: 10:18:43 iteration: 144799/375342 consumed_samples: 148275200 total_loss: 3.688 time: 0.3285 s/iter data_time: 0.2148 s/iter total_throughput: 3117.37 samples/s lr: 6.79e-04 [09/26 22:23:04] lb.utils.events INFO: eta: 10:16:13 iteration: 144899/375342 consumed_samples: 148377600 total_loss: 3.68 time: 0.3285 s/iter data_time: 0.2154 s/iter total_throughput: 3117.37 samples/s lr: 6.78e-04 [09/26 22:23:37] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0144999 [09/26 22:23:38] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/26 22:23:38] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/26 22:23:42] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0949 s/iter. Inference: 0.1462 s/iter. Eval: 0.0021 s/iter. Total: 0.2432 s/iter. ETA=0:00:08 [09/26 22:23:48] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1526 s/iter. Inference: 0.1484 s/iter. Eval: 0.0021 s/iter. Total: 0.3032 s/iter. ETA=0:00:05 [09/26 22:23:53] lb.evaluation.evaluator INFO: Inference done 49152/50000. Dataloading: 0.1372 s/iter. Inference: 0.1482 s/iter. Eval: 0.0020 s/iter. Total: 0.2875 s/iter. ETA=0:00:00 [09/26 22:23:53] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/26 22:23:53] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.800867 (0.000256 s / iter per device, on 8 devices) [09/26 22:23:53] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000131 s / iter per device, on 8 devices) [09/26 22:23:53] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/26 22:23:53] lb.evaluation.utils INFO: copypaste: Acc@1=73.378 [09/26 22:23:53] lb.evaluation.utils INFO: copypaste: Acc@5=91.60000000000001 [09/26 22:23:53] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 73.37800, better than last best score 72.99400 @ iteration 139999. [09/26 22:23:53] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/26 22:23:54] lb.utils.events INFO: eta: 10:16:11 iteration: 144999/375342 consumed_samples: 148480000 total_loss: 3.68 time: 0.3285 s/iter data_time: 0.2269 s/iter total_throughput: 3117.39 samples/s lr: 6.78e-04 [09/26 22:24:25] lb.utils.events INFO: eta: 10:14:19 iteration: 145099/375342 consumed_samples: 148582400 total_loss: 3.681 time: 0.3285 s/iter data_time: 0.2243 s/iter total_throughput: 3117.50 samples/s lr: 6.78e-04 [09/26 22:24:59] lb.utils.events INFO: eta: 10:19:56 iteration: 145199/375342 consumed_samples: 148684800 total_loss: 3.665 time: 0.3285 s/iter data_time: 0.2436 s/iter total_throughput: 3117.46 samples/s lr: 6.77e-04 [09/26 22:25:31] lb.utils.events INFO: eta: 10:18:26 iteration: 145299/375342 consumed_samples: 148787200 total_loss: 3.68 time: 0.3285 s/iter data_time: 0.2266 s/iter total_throughput: 3117.46 samples/s lr: 6.77e-04 [09/26 22:26:04] lb.utils.events INFO: eta: 10:14:38 iteration: 145399/375342 consumed_samples: 148889600 total_loss: 3.695 time: 0.3285 s/iter data_time: 0.2117 s/iter total_throughput: 3117.45 samples/s lr: 6.77e-04 [09/26 22:26:38] lb.utils.events INFO: eta: 10:07:29 iteration: 145499/375342 consumed_samples: 148992000 total_loss: 3.686 time: 0.3285 s/iter data_time: 0.2174 s/iter total_throughput: 3117.40 samples/s lr: 6.76e-04 [09/26 22:27:11] lb.utils.events INFO: eta: 10:03:53 iteration: 145599/375342 consumed_samples: 149094400 total_loss: 3.688 time: 0.3285 s/iter data_time: 0.2070 s/iter total_throughput: 3117.39 samples/s lr: 6.76e-04 [09/26 22:27:44] lb.utils.events INFO: eta: 10:05:58 iteration: 145699/375342 consumed_samples: 149196800 total_loss: 3.696 time: 0.3285 s/iter data_time: 0.2314 s/iter total_throughput: 3117.39 samples/s lr: 6.75e-04 [09/26 22:28:17] lb.utils.events INFO: eta: 10:10:37 iteration: 145799/375342 consumed_samples: 149299200 total_loss: 3.681 time: 0.3285 s/iter data_time: 0.2341 s/iter total_throughput: 3117.38 samples/s lr: 6.75e-04 [09/26 22:28:50] lb.utils.events INFO: eta: 10:13:45 iteration: 145899/375342 consumed_samples: 149401600 total_loss: 3.673 time: 0.3285 s/iter data_time: 0.2270 s/iter total_throughput: 3117.39 samples/s lr: 6.75e-04 [09/26 22:29:23] lb.utils.events INFO: eta: 10:21:46 iteration: 145999/375342 consumed_samples: 149504000 total_loss: 3.686 time: 0.3285 s/iter data_time: 0.2497 s/iter total_throughput: 3117.38 samples/s lr: 6.74e-04 [09/26 22:29:56] lb.utils.events INFO: eta: 10:51:15 iteration: 146099/375342 consumed_samples: 149606400 total_loss: 3.68 time: 0.3285 s/iter data_time: 0.2428 s/iter total_throughput: 3117.37 samples/s lr: 6.74e-04 [09/26 22:30:29] lb.utils.events INFO: eta: 10:52:13 iteration: 146199/375342 consumed_samples: 149708800 total_loss: 3.678 time: 0.3285 s/iter data_time: 0.2184 s/iter total_throughput: 3117.36 samples/s lr: 6.73e-04 [09/26 22:31:02] lb.utils.events INFO: eta: 10:51:03 iteration: 146299/375342 consumed_samples: 149811200 total_loss: 3.678 time: 0.3285 s/iter data_time: 0.2358 s/iter total_throughput: 3117.35 samples/s lr: 6.73e-04 [09/26 22:31:34] lb.utils.events INFO: eta: 13:10:28 iteration: 146399/375342 consumed_samples: 149913600 total_loss: 3.68 time: 0.3285 s/iter data_time: 0.2408 s/iter total_throughput: 3117.35 samples/s lr: 6.73e-04 [09/26 22:32:07] lb.utils.events INFO: eta: 14:22:35 iteration: 146499/375342 consumed_samples: 150016000 total_loss: 3.69 time: 0.3285 s/iter data_time: 0.2192 s/iter total_throughput: 3117.34 samples/s lr: 6.72e-04 [09/26 22:32:41] lb.utils.events INFO: eta: 16:44:12 iteration: 146599/375342 consumed_samples: 150118400 total_loss: 3.677 time: 0.3285 s/iter data_time: 0.2529 s/iter total_throughput: 3117.31 samples/s lr: 6.72e-04 [09/26 22:33:14] lb.utils.events INFO: eta: 17:28:15 iteration: 146699/375342 consumed_samples: 150220800 total_loss: 3.669 time: 0.3285 s/iter data_time: 0.2415 s/iter total_throughput: 3117.29 samples/s lr: 6.71e-04 [09/26 22:33:47] lb.utils.events INFO: eta: 17:25:05 iteration: 146799/375342 consumed_samples: 150323200 total_loss: 3.67 time: 0.3285 s/iter data_time: 0.2263 s/iter total_throughput: 3117.27 samples/s lr: 6.71e-04 [09/26 22:34:20] lb.utils.events INFO: eta: 17:41:58 iteration: 146899/375342 consumed_samples: 150425600 total_loss: 3.676 time: 0.3285 s/iter data_time: 0.2251 s/iter total_throughput: 3117.26 samples/s lr: 6.71e-04 [09/26 22:34:53] lb.utils.events INFO: eta: 16:26:29 iteration: 146999/375342 consumed_samples: 150528000 total_loss: 3.69 time: 0.3285 s/iter data_time: 0.2211 s/iter total_throughput: 3117.25 samples/s lr: 6.70e-04 [09/26 22:35:26] lb.utils.events INFO: eta: 13:27:03 iteration: 147099/375342 consumed_samples: 150630400 total_loss: 3.7 time: 0.3285 s/iter data_time: 0.1966 s/iter total_throughput: 3117.27 samples/s lr: 6.70e-04 [09/26 22:35:59] lb.utils.events INFO: eta: 13:12:46 iteration: 147199/375342 consumed_samples: 150732800 total_loss: 3.702 time: 0.3285 s/iter data_time: 0.2671 s/iter total_throughput: 3117.25 samples/s lr: 6.69e-04 [09/26 22:36:32] lb.utils.events INFO: eta: 13:26:32 iteration: 147299/375342 consumed_samples: 150835200 total_loss: 3.7 time: 0.3285 s/iter data_time: 0.2490 s/iter total_throughput: 3117.22 samples/s lr: 6.69e-04 [09/26 22:37:05] lb.utils.events INFO: eta: 12:42:18 iteration: 147399/375342 consumed_samples: 150937600 total_loss: 3.683 time: 0.3285 s/iter data_time: 0.2346 s/iter total_throughput: 3117.23 samples/s lr: 6.69e-04 [09/26 22:37:38] lb.utils.events INFO: eta: 11:22:23 iteration: 147499/375342 consumed_samples: 151040000 total_loss: 3.682 time: 0.3285 s/iter data_time: 0.2316 s/iter total_throughput: 3117.24 samples/s lr: 6.68e-04 [09/26 22:38:10] lb.utils.events INFO: eta: 11:05:45 iteration: 147599/375342 consumed_samples: 151142400 total_loss: 3.694 time: 0.3285 s/iter data_time: 0.2446 s/iter total_throughput: 3117.24 samples/s lr: 6.68e-04 [09/26 22:38:44] lb.utils.events INFO: eta: 11:22:58 iteration: 147699/375342 consumed_samples: 151244800 total_loss: 3.685 time: 0.3285 s/iter data_time: 0.2363 s/iter total_throughput: 3117.23 samples/s lr: 6.68e-04 [09/26 22:39:17] lb.utils.events INFO: eta: 12:35:21 iteration: 147799/375342 consumed_samples: 151347200 total_loss: 3.666 time: 0.3285 s/iter data_time: 0.2599 s/iter total_throughput: 3117.19 samples/s lr: 6.67e-04 [09/26 22:39:49] lb.utils.events INFO: eta: 12:26:17 iteration: 147899/375342 consumed_samples: 151449600 total_loss: 3.676 time: 0.3285 s/iter data_time: 0.2151 s/iter total_throughput: 3117.21 samples/s lr: 6.67e-04 [09/26 22:40:23] lb.utils.events INFO: eta: 13:08:26 iteration: 147999/375342 consumed_samples: 151552000 total_loss: 3.688 time: 0.3285 s/iter data_time: 0.2333 s/iter total_throughput: 3117.20 samples/s lr: 6.66e-04 [09/26 22:40:55] lb.utils.events INFO: eta: 14:12:11 iteration: 148099/375342 consumed_samples: 151654400 total_loss: 3.678 time: 0.3285 s/iter data_time: 0.2154 s/iter total_throughput: 3117.20 samples/s lr: 6.66e-04 [09/26 22:41:28] lb.utils.events INFO: eta: 14:24:37 iteration: 148199/375342 consumed_samples: 151756800 total_loss: 3.676 time: 0.3285 s/iter data_time: 0.2195 s/iter total_throughput: 3117.19 samples/s lr: 6.66e-04 [09/26 22:42:01] lb.utils.events INFO: eta: 14:29:40 iteration: 148299/375342 consumed_samples: 151859200 total_loss: 3.688 time: 0.3285 s/iter data_time: 0.2376 s/iter total_throughput: 3117.18 samples/s lr: 6.65e-04 [09/26 22:42:34] lb.utils.events INFO: eta: 14:03:10 iteration: 148399/375342 consumed_samples: 151961600 total_loss: 3.684 time: 0.3285 s/iter data_time: 0.2294 s/iter total_throughput: 3117.19 samples/s lr: 6.65e-04 [09/26 22:43:07] lb.utils.events INFO: eta: 14:57:34 iteration: 148499/375342 consumed_samples: 152064000 total_loss: 3.668 time: 0.3285 s/iter data_time: 0.2438 s/iter total_throughput: 3117.17 samples/s lr: 6.64e-04 [09/26 22:43:40] lb.utils.events INFO: eta: 14:44:20 iteration: 148599/375342 consumed_samples: 152166400 total_loss: 3.679 time: 0.3285 s/iter data_time: 0.2196 s/iter total_throughput: 3117.18 samples/s lr: 6.64e-04 [09/26 22:44:13] lb.utils.events INFO: eta: 14:39:50 iteration: 148699/375342 consumed_samples: 152268800 total_loss: 3.688 time: 0.3285 s/iter data_time: 0.2307 s/iter total_throughput: 3117.18 samples/s lr: 6.64e-04 [09/26 22:44:46] lb.utils.events INFO: eta: 14:04:38 iteration: 148799/375342 consumed_samples: 152371200 total_loss: 3.669 time: 0.3285 s/iter data_time: 0.2545 s/iter total_throughput: 3117.17 samples/s lr: 6.63e-04 [09/26 22:45:19] lb.utils.events INFO: eta: 12:52:42 iteration: 148899/375342 consumed_samples: 152473600 total_loss: 3.654 time: 0.3285 s/iter data_time: 0.2268 s/iter total_throughput: 3117.17 samples/s lr: 6.63e-04 [09/26 22:45:52] lb.utils.events INFO: eta: 12:52:21 iteration: 148999/375342 consumed_samples: 152576000 total_loss: 3.655 time: 0.3285 s/iter data_time: 0.2380 s/iter total_throughput: 3117.12 samples/s lr: 6.62e-04 [09/26 22:46:25] lb.utils.events INFO: eta: 12:28:51 iteration: 149099/375342 consumed_samples: 152678400 total_loss: 3.659 time: 0.3285 s/iter data_time: 0.2139 s/iter total_throughput: 3117.12 samples/s lr: 6.62e-04 [09/26 22:46:58] lb.utils.events INFO: eta: 11:31:02 iteration: 149199/375342 consumed_samples: 152780800 total_loss: 3.683 time: 0.3285 s/iter data_time: 0.2005 s/iter total_throughput: 3117.14 samples/s lr: 6.62e-04 [09/26 22:47:31] lb.utils.events INFO: eta: 10:39:56 iteration: 149299/375342 consumed_samples: 152883200 total_loss: 3.692 time: 0.3285 s/iter data_time: 0.2309 s/iter total_throughput: 3117.14 samples/s lr: 6.61e-04 [09/26 22:48:03] lb.utils.events INFO: eta: 10:34:23 iteration: 149399/375342 consumed_samples: 152985600 total_loss: 3.679 time: 0.3285 s/iter data_time: 0.2181 s/iter total_throughput: 3117.15 samples/s lr: 6.61e-04 [09/26 22:48:36] lb.utils.events INFO: eta: 10:13:14 iteration: 149499/375342 consumed_samples: 153088000 total_loss: 3.676 time: 0.3285 s/iter data_time: 0.2304 s/iter total_throughput: 3117.15 samples/s lr: 6.60e-04 [09/26 22:49:09] lb.utils.events INFO: eta: 10:08:17 iteration: 149599/375342 consumed_samples: 153190400 total_loss: 3.671 time: 0.3285 s/iter data_time: 0.2323 s/iter total_throughput: 3117.13 samples/s lr: 6.60e-04 [09/26 22:49:42] lb.utils.events INFO: eta: 10:06:34 iteration: 149699/375342 consumed_samples: 153292800 total_loss: 3.686 time: 0.3285 s/iter data_time: 0.2337 s/iter total_throughput: 3117.16 samples/s lr: 6.60e-04 [09/26 22:50:15] lb.utils.events INFO: eta: 10:00:05 iteration: 149799/375342 consumed_samples: 153395200 total_loss: 3.668 time: 0.3285 s/iter data_time: 0.2267 s/iter total_throughput: 3117.14 samples/s lr: 6.59e-04 [09/26 22:50:48] lb.utils.events INFO: eta: 10:02:32 iteration: 149899/375342 consumed_samples: 153497600 total_loss: 3.672 time: 0.3285 s/iter data_time: 0.2246 s/iter total_throughput: 3117.13 samples/s lr: 6.59e-04 [09/26 22:51:21] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0149999 [09/26 22:51:21] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/26 22:51:21] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/26 22:51:25] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0907 s/iter. Inference: 0.1500 s/iter. Eval: 0.0020 s/iter. Total: 0.2428 s/iter. ETA=0:00:08 [09/26 22:51:31] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1511 s/iter. Inference: 0.1520 s/iter. Eval: 0.0021 s/iter. Total: 0.3053 s/iter. ETA=0:00:05 [09/26 22:51:36] lb.evaluation.evaluator INFO: Inference done 49152/50000. Dataloading: 0.1357 s/iter. Inference: 0.1502 s/iter. Eval: 0.0021 s/iter. Total: 0.2880 s/iter. ETA=0:00:00 [09/26 22:51:37] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/26 22:51:37] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.864989 (0.000257 s / iter per device, on 8 devices) [09/26 22:51:37] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000133 s / iter per device, on 8 devices) [09/26 22:51:37] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/26 22:51:37] lb.evaluation.utils INFO: copypaste: Acc@1=73.402 [09/26 22:51:37] lb.evaluation.utils INFO: copypaste: Acc@5=91.66 [09/26 22:51:37] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 73.40200, better than last best score 73.37800 @ iteration 144999. [09/26 22:51:37] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/26 22:51:38] lb.utils.events INFO: eta: 10:00:17 iteration: 149999/375342 consumed_samples: 153600000 total_loss: 3.671 time: 0.3285 s/iter data_time: 0.2234 s/iter total_throughput: 3117.15 samples/s lr: 6.59e-04 [09/26 22:52:08] lb.utils.events INFO: eta: 10:00:26 iteration: 150099/375342 consumed_samples: 153702400 total_loss: 3.683 time: 0.3285 s/iter data_time: 0.2273 s/iter total_throughput: 3117.27 samples/s lr: 6.58e-04 [09/26 22:52:42] lb.utils.events INFO: eta: 10:03:28 iteration: 150199/375342 consumed_samples: 153804800 total_loss: 3.669 time: 0.3285 s/iter data_time: 0.1970 s/iter total_throughput: 3117.25 samples/s lr: 6.58e-04 [09/26 22:53:15] lb.utils.events INFO: eta: 10:00:37 iteration: 150299/375342 consumed_samples: 153907200 total_loss: 3.651 time: 0.3285 s/iter data_time: 0.2276 s/iter total_throughput: 3117.25 samples/s lr: 6.57e-04 [09/26 22:53:47] lb.utils.events INFO: eta: 10:06:25 iteration: 150399/375342 consumed_samples: 154009600 total_loss: 3.663 time: 0.3285 s/iter data_time: 0.2384 s/iter total_throughput: 3117.25 samples/s lr: 6.57e-04 [09/26 22:54:20] lb.utils.events INFO: eta: 10:10:09 iteration: 150499/375342 consumed_samples: 154112000 total_loss: 3.666 time: 0.3285 s/iter data_time: 0.2305 s/iter total_throughput: 3117.26 samples/s lr: 6.57e-04 [09/26 22:54:53] lb.utils.events INFO: eta: 10:17:11 iteration: 150599/375342 consumed_samples: 154214400 total_loss: 3.668 time: 0.3285 s/iter data_time: 0.2426 s/iter total_throughput: 3117.24 samples/s lr: 6.56e-04 [09/26 22:55:26] lb.utils.events INFO: eta: 10:07:50 iteration: 150699/375342 consumed_samples: 154316800 total_loss: 3.652 time: 0.3285 s/iter data_time: 0.2190 s/iter total_throughput: 3117.23 samples/s lr: 6.56e-04 [09/26 22:55:59] lb.utils.events INFO: eta: 10:02:31 iteration: 150799/375342 consumed_samples: 154419200 total_loss: 3.666 time: 0.3285 s/iter data_time: 0.2100 s/iter total_throughput: 3117.25 samples/s lr: 6.55e-04 [09/26 22:56:32] lb.utils.events INFO: eta: 10:02:51 iteration: 150899/375342 consumed_samples: 154521600 total_loss: 3.657 time: 0.3285 s/iter data_time: 0.2166 s/iter total_throughput: 3117.24 samples/s lr: 6.55e-04 [09/26 22:57:05] lb.utils.events INFO: eta: 10:08:28 iteration: 150999/375342 consumed_samples: 154624000 total_loss: 3.646 time: 0.3285 s/iter data_time: 0.2470 s/iter total_throughput: 3117.25 samples/s lr: 6.55e-04 [09/26 22:57:37] lb.utils.events INFO: eta: 10:20:47 iteration: 151099/375342 consumed_samples: 154726400 total_loss: 3.672 time: 0.3285 s/iter data_time: 0.2372 s/iter total_throughput: 3117.27 samples/s lr: 6.54e-04 [09/26 22:58:10] lb.utils.events INFO: eta: 10:20:30 iteration: 151199/375342 consumed_samples: 154828800 total_loss: 3.679 time: 0.3285 s/iter data_time: 0.2029 s/iter total_throughput: 3117.26 samples/s lr: 6.54e-04 [09/26 22:58:43] lb.utils.events INFO: eta: 10:26:34 iteration: 151299/375342 consumed_samples: 154931200 total_loss: 3.676 time: 0.3285 s/iter data_time: 0.2207 s/iter total_throughput: 3117.26 samples/s lr: 6.53e-04 [09/26 22:59:16] lb.utils.events INFO: eta: 10:17:55 iteration: 151399/375342 consumed_samples: 155033600 total_loss: 3.676 time: 0.3285 s/iter data_time: 0.2413 s/iter total_throughput: 3117.27 samples/s lr: 6.53e-04 [09/26 22:59:48] lb.utils.events INFO: eta: 10:17:38 iteration: 151499/375342 consumed_samples: 155136000 total_loss: 3.654 time: 0.3285 s/iter data_time: 0.2218 s/iter total_throughput: 3117.30 samples/s lr: 6.53e-04 [09/26 23:00:20] lb.utils.events INFO: eta: 10:16:44 iteration: 151599/375342 consumed_samples: 155238400 total_loss: 3.66 time: 0.3285 s/iter data_time: 0.2357 s/iter total_throughput: 3117.33 samples/s lr: 6.52e-04 [09/26 23:00:53] lb.utils.events INFO: eta: 11:09:25 iteration: 151699/375342 consumed_samples: 155340800 total_loss: 3.658 time: 0.3285 s/iter data_time: 0.2553 s/iter total_throughput: 3117.35 samples/s lr: 6.52e-04 [09/26 23:01:26] lb.utils.events INFO: eta: 14:46:19 iteration: 151799/375342 consumed_samples: 155443200 total_loss: 3.648 time: 0.3285 s/iter data_time: 0.2457 s/iter total_throughput: 3117.35 samples/s lr: 6.51e-04 [09/26 23:01:58] lb.utils.events INFO: eta: 15:10:11 iteration: 151899/375342 consumed_samples: 155545600 total_loss: 3.675 time: 0.3285 s/iter data_time: 0.2163 s/iter total_throughput: 3117.37 samples/s lr: 6.51e-04 [09/26 23:02:31] lb.utils.events INFO: eta: 13:48:10 iteration: 151999/375342 consumed_samples: 155648000 total_loss: 3.685 time: 0.3285 s/iter data_time: 0.2340 s/iter total_throughput: 3117.38 samples/s lr: 6.51e-04 [09/26 23:03:04] lb.utils.events INFO: eta: 13:14:00 iteration: 152099/375342 consumed_samples: 155750400 total_loss: 3.684 time: 0.3285 s/iter data_time: 0.2224 s/iter total_throughput: 3117.40 samples/s lr: 6.50e-04 [09/26 23:03:37] lb.utils.events INFO: eta: 14:05:07 iteration: 152199/375342 consumed_samples: 155852800 total_loss: 3.673 time: 0.3285 s/iter data_time: 0.2224 s/iter total_throughput: 3117.39 samples/s lr: 6.50e-04 [09/26 23:04:10] lb.utils.events INFO: eta: 14:39:04 iteration: 152299/375342 consumed_samples: 155955200 total_loss: 3.676 time: 0.3285 s/iter data_time: 0.2178 s/iter total_throughput: 3117.39 samples/s lr: 6.49e-04 [09/26 23:04:43] lb.utils.events INFO: eta: 13:54:47 iteration: 152399/375342 consumed_samples: 156057600 total_loss: 3.669 time: 0.3285 s/iter data_time: 0.2074 s/iter total_throughput: 3117.37 samples/s lr: 6.49e-04 [09/26 23:05:15] lb.utils.events INFO: eta: 12:34:24 iteration: 152499/375342 consumed_samples: 156160000 total_loss: 3.657 time: 0.3285 s/iter data_time: 0.2132 s/iter total_throughput: 3117.38 samples/s lr: 6.49e-04 [09/26 23:05:48] lb.utils.events INFO: eta: 12:00:46 iteration: 152599/375342 consumed_samples: 156262400 total_loss: 3.65 time: 0.3285 s/iter data_time: 0.2145 s/iter total_throughput: 3117.37 samples/s lr: 6.48e-04 [09/26 23:06:21] lb.utils.events INFO: eta: 10:16:59 iteration: 152699/375342 consumed_samples: 156364800 total_loss: 3.645 time: 0.3285 s/iter data_time: 0.2234 s/iter total_throughput: 3117.40 samples/s lr: 6.48e-04 [09/26 23:06:54] lb.utils.events INFO: eta: 10:01:41 iteration: 152799/375342 consumed_samples: 156467200 total_loss: 3.649 time: 0.3285 s/iter data_time: 0.2290 s/iter total_throughput: 3117.40 samples/s lr: 6.47e-04 [09/26 23:07:26] lb.utils.events INFO: eta: 9:50:54 iteration: 152899/375342 consumed_samples: 156569600 total_loss: 3.646 time: 0.3285 s/iter data_time: 0.2129 s/iter total_throughput: 3117.42 samples/s lr: 6.47e-04 [09/26 23:07:59] lb.utils.events INFO: eta: 9:49:58 iteration: 152999/375342 consumed_samples: 156672000 total_loss: 3.654 time: 0.3285 s/iter data_time: 0.1937 s/iter total_throughput: 3117.42 samples/s lr: 6.47e-04 [09/26 23:08:31] lb.utils.events INFO: eta: 9:43:37 iteration: 153099/375342 consumed_samples: 156774400 total_loss: 3.667 time: 0.3285 s/iter data_time: 0.1994 s/iter total_throughput: 3117.46 samples/s lr: 6.46e-04 [09/26 23:09:04] lb.utils.events INFO: eta: 9:40:23 iteration: 153199/375342 consumed_samples: 156876800 total_loss: 3.662 time: 0.3285 s/iter data_time: 0.2197 s/iter total_throughput: 3117.49 samples/s lr: 6.46e-04 [09/26 23:09:36] lb.utils.events INFO: eta: 9:36:27 iteration: 153299/375342 consumed_samples: 156979200 total_loss: 3.656 time: 0.3285 s/iter data_time: 0.2011 s/iter total_throughput: 3117.51 samples/s lr: 6.45e-04 [09/26 23:10:09] lb.utils.events INFO: eta: 9:37:49 iteration: 153399/375342 consumed_samples: 157081600 total_loss: 3.656 time: 0.3285 s/iter data_time: 0.2159 s/iter total_throughput: 3117.52 samples/s lr: 6.45e-04 [09/26 23:10:41] lb.utils.events INFO: eta: 9:38:14 iteration: 153499/375342 consumed_samples: 157184000 total_loss: 3.657 time: 0.3285 s/iter data_time: 0.2190 s/iter total_throughput: 3117.55 samples/s lr: 6.45e-04 [09/26 23:11:13] lb.utils.events INFO: eta: 9:38:05 iteration: 153599/375342 consumed_samples: 157286400 total_loss: 3.677 time: 0.3285 s/iter data_time: 0.2250 s/iter total_throughput: 3117.58 samples/s lr: 6.44e-04 [09/26 23:11:46] lb.utils.events INFO: eta: 9:37:38 iteration: 153699/375342 consumed_samples: 157388800 total_loss: 3.678 time: 0.3285 s/iter data_time: 0.2002 s/iter total_throughput: 3117.62 samples/s lr: 6.44e-04 [09/26 23:12:18] lb.utils.events INFO: eta: 9:36:28 iteration: 153799/375342 consumed_samples: 157491200 total_loss: 3.659 time: 0.3285 s/iter data_time: 0.2012 s/iter total_throughput: 3117.65 samples/s lr: 6.43e-04 [09/26 23:12:50] lb.utils.events INFO: eta: 9:39:46 iteration: 153899/375342 consumed_samples: 157593600 total_loss: 3.669 time: 0.3284 s/iter data_time: 0.2119 s/iter total_throughput: 3117.69 samples/s lr: 6.43e-04 [09/26 23:13:23] lb.utils.events INFO: eta: 9:40:34 iteration: 153999/375342 consumed_samples: 157696000 total_loss: 3.693 time: 0.3284 s/iter data_time: 0.2099 s/iter total_throughput: 3117.70 samples/s lr: 6.43e-04 [09/26 23:13:56] lb.utils.events INFO: eta: 9:38:40 iteration: 154099/375342 consumed_samples: 157798400 total_loss: 3.692 time: 0.3285 s/iter data_time: 0.2128 s/iter total_throughput: 3117.67 samples/s lr: 6.42e-04 [09/26 23:14:29] lb.utils.events INFO: eta: 9:37:24 iteration: 154199/375342 consumed_samples: 157900800 total_loss: 3.66 time: 0.3284 s/iter data_time: 0.2052 s/iter total_throughput: 3117.68 samples/s lr: 6.42e-04 [09/26 23:15:02] lb.utils.events INFO: eta: 9:41:50 iteration: 154299/375342 consumed_samples: 158003200 total_loss: 3.647 time: 0.3285 s/iter data_time: 0.2494 s/iter total_throughput: 3117.65 samples/s lr: 6.41e-04 [09/26 23:15:35] lb.utils.events INFO: eta: 9:43:36 iteration: 154399/375342 consumed_samples: 158105600 total_loss: 3.649 time: 0.3285 s/iter data_time: 0.2336 s/iter total_throughput: 3117.65 samples/s lr: 6.41e-04 [09/26 23:16:08] lb.utils.events INFO: eta: 9:46:14 iteration: 154499/375342 consumed_samples: 158208000 total_loss: 3.66 time: 0.3285 s/iter data_time: 0.2267 s/iter total_throughput: 3117.63 samples/s lr: 6.41e-04 [09/26 23:16:42] lb.utils.events INFO: eta: 9:41:24 iteration: 154599/375342 consumed_samples: 158310400 total_loss: 3.641 time: 0.3285 s/iter data_time: 0.2156 s/iter total_throughput: 3117.59 samples/s lr: 6.40e-04 [09/26 23:17:15] lb.utils.events INFO: eta: 9:40:22 iteration: 154699/375342 consumed_samples: 158412800 total_loss: 3.663 time: 0.3285 s/iter data_time: 0.2069 s/iter total_throughput: 3117.56 samples/s lr: 6.40e-04 [09/26 23:17:48] lb.utils.events INFO: eta: 9:36:20 iteration: 154799/375342 consumed_samples: 158515200 total_loss: 3.678 time: 0.3285 s/iter data_time: 0.2080 s/iter total_throughput: 3117.55 samples/s lr: 6.39e-04 [09/26 23:18:21] lb.utils.events INFO: eta: 9:33:26 iteration: 154899/375342 consumed_samples: 158617600 total_loss: 3.693 time: 0.3285 s/iter data_time: 0.2281 s/iter total_throughput: 3117.53 samples/s lr: 6.39e-04 [09/26 23:18:55] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0154999 [09/26 23:18:56] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/26 23:18:56] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/26 23:19:00] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0889 s/iter. Inference: 0.1488 s/iter. Eval: 0.0020 s/iter. Total: 0.2398 s/iter. ETA=0:00:08 [09/26 23:19:05] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1439 s/iter. Inference: 0.1530 s/iter. Eval: 0.0019 s/iter. Total: 0.2989 s/iter. ETA=0:00:05 [09/26 23:19:10] lb.evaluation.evaluator INFO: Inference done 49152/50000. Dataloading: 0.1318 s/iter. Inference: 0.1496 s/iter. Eval: 0.0020 s/iter. Total: 0.2835 s/iter. ETA=0:00:00 [09/26 23:19:11] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/26 23:19:11] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.636900 (0.000253 s / iter per device, on 8 devices) [09/26 23:19:11] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000133 s / iter per device, on 8 devices) [09/26 23:19:11] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/26 23:19:11] lb.evaluation.utils INFO: copypaste: Acc@1=73.358 [09/26 23:19:11] lb.evaluation.utils INFO: copypaste: Acc@5=91.724 [09/26 23:19:11] lb.engine.hooks INFO: Not saving as latest eval score for Acc@1 is 73.35800, not better than best score 73.40200 @ iteration 149999. [09/26 23:19:11] lb.utils.events INFO: eta: 9:34:05 iteration: 154999/375342 consumed_samples: 158720000 total_loss: 3.699 time: 0.3285 s/iter data_time: 0.2048 s/iter total_throughput: 3117.49 samples/s lr: 6.39e-04 [09/26 23:19:42] lb.utils.events INFO: eta: 9:37:28 iteration: 155099/375342 consumed_samples: 158822400 total_loss: 3.682 time: 0.3285 s/iter data_time: 0.2388 s/iter total_throughput: 3117.60 samples/s lr: 6.38e-04 [09/26 23:20:15] lb.utils.events INFO: eta: 9:42:03 iteration: 155199/375342 consumed_samples: 158924800 total_loss: 3.658 time: 0.3285 s/iter data_time: 0.2252 s/iter total_throughput: 3117.57 samples/s lr: 6.38e-04 [09/26 23:20:49] lb.utils.events INFO: eta: 9:35:39 iteration: 155299/375342 consumed_samples: 159027200 total_loss: 3.64 time: 0.3285 s/iter data_time: 0.2101 s/iter total_throughput: 3117.52 samples/s lr: 6.37e-04 [09/26 23:21:22] lb.utils.events INFO: eta: 9:31:32 iteration: 155399/375342 consumed_samples: 159129600 total_loss: 3.648 time: 0.3285 s/iter data_time: 0.2061 s/iter total_throughput: 3117.51 samples/s lr: 6.37e-04 [09/26 23:21:55] lb.utils.events INFO: eta: 9:30:34 iteration: 155499/375342 consumed_samples: 159232000 total_loss: 3.646 time: 0.3285 s/iter data_time: 0.2123 s/iter total_throughput: 3117.50 samples/s lr: 6.37e-04 [09/26 23:22:29] lb.utils.events INFO: eta: 9:29:48 iteration: 155599/375342 consumed_samples: 159334400 total_loss: 3.647 time: 0.3285 s/iter data_time: 0.2132 s/iter total_throughput: 3117.46 samples/s lr: 6.36e-04 [09/26 23:23:01] lb.utils.events INFO: eta: 9:28:58 iteration: 155699/375342 consumed_samples: 159436800 total_loss: 3.654 time: 0.3285 s/iter data_time: 0.2110 s/iter total_throughput: 3117.46 samples/s lr: 6.36e-04 [09/26 23:23:35] lb.utils.events INFO: eta: 9:31:48 iteration: 155799/375342 consumed_samples: 159539200 total_loss: 3.657 time: 0.3285 s/iter data_time: 0.2296 s/iter total_throughput: 3117.44 samples/s lr: 6.35e-04 [09/26 23:24:07] lb.utils.events INFO: eta: 9:36:18 iteration: 155899/375342 consumed_samples: 159641600 total_loss: 3.659 time: 0.3285 s/iter data_time: 0.2420 s/iter total_throughput: 3117.45 samples/s lr: 6.35e-04 [09/26 23:24:41] lb.utils.events INFO: eta: 9:37:46 iteration: 155999/375342 consumed_samples: 159744000 total_loss: 3.65 time: 0.3285 s/iter data_time: 0.2214 s/iter total_throughput: 3117.42 samples/s lr: 6.35e-04 [09/26 23:25:14] lb.utils.events INFO: eta: 9:47:49 iteration: 156099/375342 consumed_samples: 159846400 total_loss: 3.65 time: 0.3285 s/iter data_time: 0.2670 s/iter total_throughput: 3117.39 samples/s lr: 6.34e-04 [09/26 23:25:47] lb.utils.events INFO: eta: 9:50:47 iteration: 156199/375342 consumed_samples: 159948800 total_loss: 3.653 time: 0.3285 s/iter data_time: 0.2432 s/iter total_throughput: 3117.38 samples/s lr: 6.34e-04 [09/26 23:26:20] lb.utils.events INFO: eta: 10:12:37 iteration: 156299/375342 consumed_samples: 160051200 total_loss: 3.655 time: 0.3285 s/iter data_time: 0.2440 s/iter total_throughput: 3117.36 samples/s lr: 6.33e-04 [09/26 23:26:53] lb.utils.events INFO: eta: 10:53:17 iteration: 156399/375342 consumed_samples: 160153600 total_loss: 3.663 time: 0.3285 s/iter data_time: 0.2375 s/iter total_throughput: 3117.36 samples/s lr: 6.33e-04 [09/26 23:27:26] lb.utils.events INFO: eta: 13:29:13 iteration: 156499/375342 consumed_samples: 160256000 total_loss: 3.656 time: 0.3285 s/iter data_time: 0.2287 s/iter total_throughput: 3117.35 samples/s lr: 6.33e-04 [09/26 23:28:00] lb.utils.events INFO: eta: 15:11:14 iteration: 156599/375342 consumed_samples: 160358400 total_loss: 3.646 time: 0.3285 s/iter data_time: 0.2152 s/iter total_throughput: 3117.29 samples/s lr: 6.32e-04 [09/26 23:28:33] lb.utils.events INFO: eta: 15:59:02 iteration: 156699/375342 consumed_samples: 160460800 total_loss: 3.663 time: 0.3285 s/iter data_time: 0.2317 s/iter total_throughput: 3117.27 samples/s lr: 6.32e-04 [09/26 23:29:06] lb.utils.events INFO: eta: 15:17:29 iteration: 156799/375342 consumed_samples: 160563200 total_loss: 3.651 time: 0.3285 s/iter data_time: 0.2278 s/iter total_throughput: 3117.25 samples/s lr: 6.31e-04 [09/26 23:29:40] lb.utils.events INFO: eta: 14:28:17 iteration: 156899/375342 consumed_samples: 160665600 total_loss: 3.671 time: 0.3285 s/iter data_time: 0.2300 s/iter total_throughput: 3117.22 samples/s lr: 6.31e-04 [09/26 23:30:13] lb.utils.events INFO: eta: 14:27:53 iteration: 156999/375342 consumed_samples: 160768000 total_loss: 3.671 time: 0.3285 s/iter data_time: 0.2340 s/iter total_throughput: 3117.22 samples/s lr: 6.31e-04 [09/26 23:30:46] lb.utils.events INFO: eta: 12:46:33 iteration: 157099/375342 consumed_samples: 160870400 total_loss: 3.642 time: 0.3285 s/iter data_time: 0.2263 s/iter total_throughput: 3117.21 samples/s lr: 6.30e-04 [09/26 23:31:19] lb.utils.events INFO: eta: 10:58:26 iteration: 157199/375342 consumed_samples: 160972800 total_loss: 3.631 time: 0.3285 s/iter data_time: 0.2227 s/iter total_throughput: 3117.19 samples/s lr: 6.30e-04 [09/26 23:31:52] lb.utils.events INFO: eta: 10:11:20 iteration: 157299/375342 consumed_samples: 161075200 total_loss: 3.651 time: 0.3285 s/iter data_time: 0.2343 s/iter total_throughput: 3117.16 samples/s lr: 6.29e-04 [09/26 23:32:25] lb.utils.events INFO: eta: 9:56:25 iteration: 157399/375342 consumed_samples: 161177600 total_loss: 3.66 time: 0.3285 s/iter data_time: 0.2399 s/iter total_throughput: 3117.15 samples/s lr: 6.29e-04 [09/26 23:32:58] lb.utils.events INFO: eta: 9:43:53 iteration: 157499/375342 consumed_samples: 161280000 total_loss: 3.655 time: 0.3285 s/iter data_time: 0.2081 s/iter total_throughput: 3117.13 samples/s lr: 6.29e-04 [09/26 23:33:32] lb.utils.events INFO: eta: 9:41:42 iteration: 157599/375342 consumed_samples: 161382400 total_loss: 3.659 time: 0.3285 s/iter data_time: 0.2317 s/iter total_throughput: 3117.10 samples/s lr: 6.28e-04 [09/26 23:34:05] lb.utils.events INFO: eta: 9:40:42 iteration: 157699/375342 consumed_samples: 161484800 total_loss: 3.66 time: 0.3285 s/iter data_time: 0.1987 s/iter total_throughput: 3117.09 samples/s lr: 6.28e-04 [09/26 23:34:38] lb.utils.events INFO: eta: 9:41:24 iteration: 157799/375342 consumed_samples: 161587200 total_loss: 3.67 time: 0.3285 s/iter data_time: 0.2182 s/iter total_throughput: 3117.07 samples/s lr: 6.27e-04 [09/26 23:35:11] lb.utils.events INFO: eta: 9:47:36 iteration: 157899/375342 consumed_samples: 161689600 total_loss: 3.65 time: 0.3285 s/iter data_time: 0.2315 s/iter total_throughput: 3117.05 samples/s lr: 6.27e-04 [09/26 23:35:44] lb.utils.events INFO: eta: 9:40:52 iteration: 157999/375342 consumed_samples: 161792000 total_loss: 3.654 time: 0.3285 s/iter data_time: 0.2068 s/iter total_throughput: 3117.02 samples/s lr: 6.27e-04 [09/26 23:36:17] lb.utils.events INFO: eta: 9:39:47 iteration: 158099/375342 consumed_samples: 161894400 total_loss: 3.657 time: 0.3285 s/iter data_time: 0.2275 s/iter total_throughput: 3117.02 samples/s lr: 6.26e-04 [09/26 23:36:51] lb.utils.events INFO: eta: 9:40:14 iteration: 158199/375342 consumed_samples: 161996800 total_loss: 3.659 time: 0.3285 s/iter data_time: 0.2251 s/iter total_throughput: 3116.98 samples/s lr: 6.26e-04 [09/26 23:37:24] lb.utils.events INFO: eta: 9:39:06 iteration: 158299/375342 consumed_samples: 162099200 total_loss: 3.66 time: 0.3285 s/iter data_time: 0.2250 s/iter total_throughput: 3116.98 samples/s lr: 6.25e-04 [09/26 23:37:56] lb.utils.events INFO: eta: 9:48:14 iteration: 158399/375342 consumed_samples: 162201600 total_loss: 3.64 time: 0.3285 s/iter data_time: 0.2226 s/iter total_throughput: 3116.99 samples/s lr: 6.25e-04 [09/26 23:38:29] lb.utils.events INFO: eta: 9:47:11 iteration: 158499/375342 consumed_samples: 162304000 total_loss: 3.655 time: 0.3285 s/iter data_time: 0.2213 s/iter total_throughput: 3117.00 samples/s lr: 6.25e-04 [09/26 23:39:02] lb.utils.events INFO: eta: 9:46:22 iteration: 158599/375342 consumed_samples: 162406400 total_loss: 3.664 time: 0.3285 s/iter data_time: 0.2111 s/iter total_throughput: 3116.99 samples/s lr: 6.24e-04 [09/26 23:39:35] lb.utils.events INFO: eta: 9:44:45 iteration: 158699/375342 consumed_samples: 162508800 total_loss: 3.661 time: 0.3285 s/iter data_time: 0.2316 s/iter total_throughput: 3116.99 samples/s lr: 6.24e-04 [09/26 23:40:09] lb.utils.events INFO: eta: 9:38:42 iteration: 158799/375342 consumed_samples: 162611200 total_loss: 3.664 time: 0.3285 s/iter data_time: 0.2101 s/iter total_throughput: 3116.95 samples/s lr: 6.23e-04 [09/26 23:40:41] lb.utils.events INFO: eta: 9:34:17 iteration: 158899/375342 consumed_samples: 162713600 total_loss: 3.649 time: 0.3285 s/iter data_time: 0.2078 s/iter total_throughput: 3116.97 samples/s lr: 6.23e-04 [09/26 23:41:14] lb.utils.events INFO: eta: 9:42:57 iteration: 158999/375342 consumed_samples: 162816000 total_loss: 3.631 time: 0.3285 s/iter data_time: 0.2512 s/iter total_throughput: 3116.96 samples/s lr: 6.23e-04 [09/26 23:41:47] lb.utils.events INFO: eta: 9:52:37 iteration: 159099/375342 consumed_samples: 162918400 total_loss: 3.615 time: 0.3285 s/iter data_time: 0.2407 s/iter total_throughput: 3116.96 samples/s lr: 6.22e-04 [09/26 23:42:20] lb.utils.events INFO: eta: 9:56:48 iteration: 159199/375342 consumed_samples: 163020800 total_loss: 3.624 time: 0.3285 s/iter data_time: 0.2284 s/iter total_throughput: 3116.92 samples/s lr: 6.22e-04 [09/26 23:42:53] lb.utils.events INFO: eta: 10:04:35 iteration: 159299/375342 consumed_samples: 163123200 total_loss: 3.63 time: 0.3285 s/iter data_time: 0.2309 s/iter total_throughput: 3116.91 samples/s lr: 6.21e-04 [09/26 23:43:27] lb.utils.events INFO: eta: 9:48:50 iteration: 159399/375342 consumed_samples: 163225600 total_loss: 3.631 time: 0.3285 s/iter data_time: 0.2056 s/iter total_throughput: 3116.85 samples/s lr: 6.21e-04 [09/26 23:44:00] lb.utils.events INFO: eta: 9:47:27 iteration: 159499/375342 consumed_samples: 163328000 total_loss: 3.631 time: 0.3285 s/iter data_time: 0.2213 s/iter total_throughput: 3116.84 samples/s lr: 6.21e-04 [09/26 23:44:33] lb.utils.events INFO: eta: 9:44:40 iteration: 159599/375342 consumed_samples: 163430400 total_loss: 3.63 time: 0.3285 s/iter data_time: 0.2312 s/iter total_throughput: 3116.83 samples/s lr: 6.20e-04 [09/26 23:45:06] lb.utils.events INFO: eta: 9:59:07 iteration: 159699/375342 consumed_samples: 163532800 total_loss: 3.66 time: 0.3285 s/iter data_time: 0.2371 s/iter total_throughput: 3116.84 samples/s lr: 6.20e-04 [09/26 23:45:39] lb.utils.events INFO: eta: 11:35:16 iteration: 159799/375342 consumed_samples: 163635200 total_loss: 3.654 time: 0.3285 s/iter data_time: 0.2394 s/iter total_throughput: 3116.83 samples/s lr: 6.19e-04 [09/26 23:46:13] lb.utils.events INFO: eta: 13:49:35 iteration: 159899/375342 consumed_samples: 163737600 total_loss: 3.64 time: 0.3285 s/iter data_time: 0.2423 s/iter total_throughput: 3116.80 samples/s lr: 6.19e-04 [09/26 23:46:45] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0159999 [09/26 23:46:46] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/26 23:46:46] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/26 23:46:50] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0829 s/iter. Inference: 0.1503 s/iter. Eval: 0.0022 s/iter. Total: 0.2354 s/iter. ETA=0:00:08 [09/26 23:46:56] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1479 s/iter. Inference: 0.1498 s/iter. Eval: 0.0021 s/iter. Total: 0.2998 s/iter. ETA=0:00:05 [09/26 23:47:01] lb.evaluation.evaluator INFO: Inference done 50000/50000. Dataloading: 0.1346 s/iter. Inference: 0.1503 s/iter. Eval: 0.0020 s/iter. Total: 0.2869 s/iter. ETA=0:00:00 [09/26 23:47:01] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/26 23:47:01] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.625001 (0.000253 s / iter per device, on 8 devices) [09/26 23:47:01] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000132 s / iter per device, on 8 devices) [09/26 23:47:01] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/26 23:47:01] lb.evaluation.utils INFO: copypaste: Acc@1=73.944 [09/26 23:47:01] lb.evaluation.utils INFO: copypaste: Acc@5=91.986 [09/26 23:47:01] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 73.94400, better than last best score 73.40200 @ iteration 149999. [09/26 23:47:01] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/26 23:47:02] lb.utils.events INFO: eta: 11:43:55 iteration: 159999/375342 consumed_samples: 163840000 total_loss: 3.637 time: 0.3285 s/iter data_time: 0.2299 s/iter total_throughput: 3116.80 samples/s lr: 6.19e-04 [09/26 23:47:33] lb.utils.events INFO: eta: 10:27:16 iteration: 160099/375342 consumed_samples: 163942400 total_loss: 3.658 time: 0.3285 s/iter data_time: 0.2420 s/iter total_throughput: 3116.89 samples/s lr: 6.18e-04 [09/26 23:48:07] lb.utils.events INFO: eta: 10:52:01 iteration: 160199/375342 consumed_samples: 164044800 total_loss: 3.658 time: 0.3285 s/iter data_time: 0.2272 s/iter total_throughput: 3116.86 samples/s lr: 6.18e-04 [09/26 23:48:39] lb.utils.events INFO: eta: 10:11:19 iteration: 160299/375342 consumed_samples: 164147200 total_loss: 3.645 time: 0.3285 s/iter data_time: 0.2176 s/iter total_throughput: 3116.87 samples/s lr: 6.17e-04 [09/26 23:49:12] lb.utils.events INFO: eta: 10:05:17 iteration: 160399/375342 consumed_samples: 164249600 total_loss: 3.626 time: 0.3285 s/iter data_time: 0.2067 s/iter total_throughput: 3116.86 samples/s lr: 6.17e-04 [09/26 23:49:45] lb.utils.events INFO: eta: 9:59:39 iteration: 160499/375342 consumed_samples: 164352000 total_loss: 3.637 time: 0.3285 s/iter data_time: 0.2092 s/iter total_throughput: 3116.86 samples/s lr: 6.17e-04 [09/26 23:50:19] lb.utils.events INFO: eta: 10:06:37 iteration: 160599/375342 consumed_samples: 164454400 total_loss: 3.646 time: 0.3285 s/iter data_time: 0.2332 s/iter total_throughput: 3116.83 samples/s lr: 6.16e-04 [09/26 23:50:51] lb.utils.events INFO: eta: 9:51:46 iteration: 160699/375342 consumed_samples: 164556800 total_loss: 3.641 time: 0.3285 s/iter data_time: 0.2222 s/iter total_throughput: 3116.85 samples/s lr: 6.16e-04 [09/26 23:51:24] lb.utils.events INFO: eta: 9:37:51 iteration: 160799/375342 consumed_samples: 164659200 total_loss: 3.633 time: 0.3285 s/iter data_time: 0.2259 s/iter total_throughput: 3116.83 samples/s lr: 6.15e-04 [09/26 23:51:58] lb.utils.events INFO: eta: 9:33:33 iteration: 160899/375342 consumed_samples: 164761600 total_loss: 3.639 time: 0.3285 s/iter data_time: 0.2291 s/iter total_throughput: 3116.79 samples/s lr: 6.15e-04 [09/26 23:52:31] lb.utils.events INFO: eta: 9:32:28 iteration: 160999/375342 consumed_samples: 164864000 total_loss: 3.658 time: 0.3285 s/iter data_time: 0.2247 s/iter total_throughput: 3116.77 samples/s lr: 6.15e-04 [09/26 23:53:04] lb.utils.events INFO: eta: 9:31:20 iteration: 161099/375342 consumed_samples: 164966400 total_loss: 3.651 time: 0.3285 s/iter data_time: 0.2331 s/iter total_throughput: 3116.78 samples/s lr: 6.14e-04 [09/26 23:53:37] lb.utils.events INFO: eta: 9:22:56 iteration: 161199/375342 consumed_samples: 165068800 total_loss: 3.649 time: 0.3285 s/iter data_time: 0.2165 s/iter total_throughput: 3116.78 samples/s lr: 6.14e-04 [09/26 23:54:10] lb.utils.events INFO: eta: 9:26:35 iteration: 161299/375342 consumed_samples: 165171200 total_loss: 3.652 time: 0.3285 s/iter data_time: 0.2503 s/iter total_throughput: 3116.78 samples/s lr: 6.13e-04 [09/26 23:54:43] lb.utils.events INFO: eta: 9:44:56 iteration: 161399/375342 consumed_samples: 165273600 total_loss: 3.622 time: 0.3285 s/iter data_time: 0.2277 s/iter total_throughput: 3116.77 samples/s lr: 6.13e-04 [09/26 23:55:16] lb.utils.events INFO: eta: 9:55:33 iteration: 161499/375342 consumed_samples: 165376000 total_loss: 3.602 time: 0.3285 s/iter data_time: 0.2215 s/iter total_throughput: 3116.75 samples/s lr: 6.13e-04 [09/26 23:55:49] lb.utils.events INFO: eta: 9:48:41 iteration: 161599/375342 consumed_samples: 165478400 total_loss: 3.603 time: 0.3285 s/iter data_time: 0.2041 s/iter total_throughput: 3116.75 samples/s lr: 6.12e-04 [09/26 23:56:22] lb.utils.events INFO: eta: 9:47:43 iteration: 161699/375342 consumed_samples: 165580800 total_loss: 3.618 time: 0.3285 s/iter data_time: 0.2200 s/iter total_throughput: 3116.74 samples/s lr: 6.12e-04 [09/26 23:56:55] lb.utils.events INFO: eta: 9:43:29 iteration: 161799/375342 consumed_samples: 165683200 total_loss: 3.625 time: 0.3285 s/iter data_time: 0.2411 s/iter total_throughput: 3116.74 samples/s lr: 6.11e-04 [09/26 23:57:27] lb.utils.events INFO: eta: 9:50:44 iteration: 161899/375342 consumed_samples: 165785600 total_loss: 3.642 time: 0.3285 s/iter data_time: 0.2334 s/iter total_throughput: 3116.75 samples/s lr: 6.11e-04 [09/26 23:58:00] lb.utils.events INFO: eta: 9:51:26 iteration: 161999/375342 consumed_samples: 165888000 total_loss: 3.65 time: 0.3285 s/iter data_time: 0.2244 s/iter total_throughput: 3116.74 samples/s lr: 6.11e-04 [09/26 23:58:34] lb.utils.events INFO: eta: 9:50:28 iteration: 162099/375342 consumed_samples: 165990400 total_loss: 3.638 time: 0.3286 s/iter data_time: 0.2280 s/iter total_throughput: 3116.71 samples/s lr: 6.10e-04 [09/26 23:59:07] lb.utils.events INFO: eta: 9:59:05 iteration: 162199/375342 consumed_samples: 166092800 total_loss: 3.628 time: 0.3286 s/iter data_time: 0.2431 s/iter total_throughput: 3116.70 samples/s lr: 6.10e-04 [09/26 23:59:40] lb.utils.events INFO: eta: 9:49:04 iteration: 162299/375342 consumed_samples: 166195200 total_loss: 3.633 time: 0.3286 s/iter data_time: 0.2380 s/iter total_throughput: 3116.70 samples/s lr: 6.09e-04 [09/27 00:00:12] lb.utils.events INFO: eta: 9:39:49 iteration: 162399/375342 consumed_samples: 166297600 total_loss: 3.632 time: 0.3286 s/iter data_time: 0.2242 s/iter total_throughput: 3116.70 samples/s lr: 6.09e-04 [09/27 00:00:45] lb.utils.events INFO: eta: 9:49:18 iteration: 162499/375342 consumed_samples: 166400000 total_loss: 3.633 time: 0.3286 s/iter data_time: 0.2482 s/iter total_throughput: 3116.69 samples/s lr: 6.09e-04 [09/27 00:01:18] lb.utils.events INFO: eta: 10:10:31 iteration: 162599/375342 consumed_samples: 166502400 total_loss: 3.634 time: 0.3286 s/iter data_time: 0.2409 s/iter total_throughput: 3116.70 samples/s lr: 6.08e-04 [09/27 00:01:51] lb.utils.events INFO: eta: 12:35:31 iteration: 162699/375342 consumed_samples: 166604800 total_loss: 3.631 time: 0.3286 s/iter data_time: 0.2525 s/iter total_throughput: 3116.69 samples/s lr: 6.08e-04 [09/27 00:02:24] lb.utils.events INFO: eta: 12:49:30 iteration: 162799/375342 consumed_samples: 166707200 total_loss: 3.622 time: 0.3286 s/iter data_time: 0.2214 s/iter total_throughput: 3116.69 samples/s lr: 6.07e-04 [09/27 00:02:57] lb.utils.events INFO: eta: 12:13:05 iteration: 162899/375342 consumed_samples: 166809600 total_loss: 3.617 time: 0.3286 s/iter data_time: 0.2246 s/iter total_throughput: 3116.67 samples/s lr: 6.07e-04 [09/27 00:03:30] lb.utils.events INFO: eta: 12:40:04 iteration: 162999/375342 consumed_samples: 166912000 total_loss: 3.63 time: 0.3286 s/iter data_time: 0.2314 s/iter total_throughput: 3116.68 samples/s lr: 6.06e-04 [09/27 00:04:03] lb.utils.events INFO: eta: 13:13:24 iteration: 163099/375342 consumed_samples: 167014400 total_loss: 3.633 time: 0.3286 s/iter data_time: 0.2348 s/iter total_throughput: 3116.67 samples/s lr: 6.06e-04 [09/27 00:04:36] lb.utils.events INFO: eta: 15:01:23 iteration: 163199/375342 consumed_samples: 167116800 total_loss: 3.617 time: 0.3286 s/iter data_time: 0.2304 s/iter total_throughput: 3116.67 samples/s lr: 6.06e-04 [09/27 00:05:09] lb.utils.events INFO: eta: 14:59:12 iteration: 163299/375342 consumed_samples: 167219200 total_loss: 3.615 time: 0.3286 s/iter data_time: 0.2499 s/iter total_throughput: 3116.66 samples/s lr: 6.05e-04 [09/27 00:05:42] lb.utils.events INFO: eta: 15:42:54 iteration: 163399/375342 consumed_samples: 167321600 total_loss: 3.624 time: 0.3286 s/iter data_time: 0.2409 s/iter total_throughput: 3116.66 samples/s lr: 6.05e-04 [09/27 00:06:14] lb.utils.events INFO: eta: 15:28:13 iteration: 163499/375342 consumed_samples: 167424000 total_loss: 3.62 time: 0.3286 s/iter data_time: 0.2335 s/iter total_throughput: 3116.67 samples/s lr: 6.04e-04 [09/27 00:06:47] lb.utils.events INFO: eta: 15:04:47 iteration: 163599/375342 consumed_samples: 167526400 total_loss: 3.609 time: 0.3286 s/iter data_time: 0.2273 s/iter total_throughput: 3116.67 samples/s lr: 6.04e-04 [09/27 00:07:20] lb.utils.events INFO: eta: 13:54:09 iteration: 163699/375342 consumed_samples: 167628800 total_loss: 3.634 time: 0.3286 s/iter data_time: 0.2203 s/iter total_throughput: 3116.67 samples/s lr: 6.04e-04 [09/27 00:07:53] lb.utils.events INFO: eta: 14:05:41 iteration: 163799/375342 consumed_samples: 167731200 total_loss: 3.646 time: 0.3286 s/iter data_time: 0.2225 s/iter total_throughput: 3116.67 samples/s lr: 6.03e-04 [09/27 00:08:26] lb.utils.events INFO: eta: 14:05:17 iteration: 163899/375342 consumed_samples: 167833600 total_loss: 3.635 time: 0.3286 s/iter data_time: 0.2177 s/iter total_throughput: 3116.65 samples/s lr: 6.03e-04 [09/27 00:08:59] lb.utils.events INFO: eta: 12:46:57 iteration: 163999/375342 consumed_samples: 167936000 total_loss: 3.616 time: 0.3286 s/iter data_time: 0.2081 s/iter total_throughput: 3116.66 samples/s lr: 6.02e-04 [09/27 00:09:32] lb.utils.events INFO: eta: 10:26:36 iteration: 164099/375342 consumed_samples: 168038400 total_loss: 3.599 time: 0.3286 s/iter data_time: 0.2153 s/iter total_throughput: 3116.64 samples/s lr: 6.02e-04 [09/27 00:10:05] lb.utils.events INFO: eta: 9:53:25 iteration: 164199/375342 consumed_samples: 168140800 total_loss: 3.631 time: 0.3286 s/iter data_time: 0.2014 s/iter total_throughput: 3116.66 samples/s lr: 6.02e-04 [09/27 00:10:37] lb.utils.events INFO: eta: 9:25:56 iteration: 164299/375342 consumed_samples: 168243200 total_loss: 3.637 time: 0.3286 s/iter data_time: 0.1985 s/iter total_throughput: 3116.67 samples/s lr: 6.01e-04 [09/27 00:11:11] lb.utils.events INFO: eta: 9:21:10 iteration: 164399/375342 consumed_samples: 168345600 total_loss: 3.621 time: 0.3286 s/iter data_time: 0.2340 s/iter total_throughput: 3116.65 samples/s lr: 6.01e-04 [09/27 00:11:43] lb.utils.events INFO: eta: 9:19:41 iteration: 164499/375342 consumed_samples: 168448000 total_loss: 3.618 time: 0.3286 s/iter data_time: 0.2322 s/iter total_throughput: 3116.65 samples/s lr: 6.00e-04 [09/27 00:12:16] lb.utils.events INFO: eta: 9:21:21 iteration: 164599/375342 consumed_samples: 168550400 total_loss: 3.616 time: 0.3286 s/iter data_time: 0.2295 s/iter total_throughput: 3116.66 samples/s lr: 6.00e-04 [09/27 00:12:49] lb.utils.events INFO: eta: 9:24:08 iteration: 164699/375342 consumed_samples: 168652800 total_loss: 3.614 time: 0.3286 s/iter data_time: 0.2171 s/iter total_throughput: 3116.68 samples/s lr: 6.00e-04 [09/27 00:13:21] lb.utils.events INFO: eta: 9:24:40 iteration: 164799/375342 consumed_samples: 168755200 total_loss: 3.604 time: 0.3286 s/iter data_time: 0.2287 s/iter total_throughput: 3116.68 samples/s lr: 5.99e-04 [09/27 00:13:54] lb.utils.events INFO: eta: 9:21:08 iteration: 164899/375342 consumed_samples: 168857600 total_loss: 3.605 time: 0.3286 s/iter data_time: 0.1985 s/iter total_throughput: 3116.68 samples/s lr: 5.99e-04 [09/27 00:14:27] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0164999 [09/27 00:14:28] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/27 00:14:28] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/27 00:14:32] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0873 s/iter. Inference: 0.1492 s/iter. Eval: 0.0021 s/iter. Total: 0.2385 s/iter. ETA=0:00:08 [09/27 00:14:38] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1490 s/iter. Inference: 0.1502 s/iter. Eval: 0.0021 s/iter. Total: 0.3014 s/iter. ETA=0:00:05 [09/27 00:14:43] lb.evaluation.evaluator INFO: Inference done 50000/50000. Dataloading: 0.1347 s/iter. Inference: 0.1506 s/iter. Eval: 0.0021 s/iter. Total: 0.2875 s/iter. ETA=0:00:00 [09/27 00:14:43] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/27 00:14:43] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.653690 (0.000253 s / iter per device, on 8 devices) [09/27 00:14:43] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000133 s / iter per device, on 8 devices) [09/27 00:14:43] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/27 00:14:43] lb.evaluation.utils INFO: copypaste: Acc@1=74.32 [09/27 00:14:43] lb.evaluation.utils INFO: copypaste: Acc@5=92.12599999999999 [09/27 00:14:43] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 74.32000, better than last best score 73.94400 @ iteration 159999. [09/27 00:14:43] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/27 00:14:44] lb.utils.events INFO: eta: 9:19:47 iteration: 164999/375342 consumed_samples: 168960000 total_loss: 3.62 time: 0.3286 s/iter data_time: 0.2102 s/iter total_throughput: 3116.67 samples/s lr: 5.98e-04 [09/27 00:15:15] lb.utils.events INFO: eta: 9:25:27 iteration: 165099/375342 consumed_samples: 169062400 total_loss: 3.613 time: 0.3285 s/iter data_time: 0.2566 s/iter total_throughput: 3116.79 samples/s lr: 5.98e-04 [09/27 00:15:48] lb.utils.events INFO: eta: 9:41:31 iteration: 165199/375342 consumed_samples: 169164800 total_loss: 3.606 time: 0.3285 s/iter data_time: 0.2452 s/iter total_throughput: 3116.80 samples/s lr: 5.98e-04 [09/27 00:16:20] lb.utils.events INFO: eta: 10:23:07 iteration: 165299/375342 consumed_samples: 169267200 total_loss: 3.597 time: 0.3285 s/iter data_time: 0.2416 s/iter total_throughput: 3116.80 samples/s lr: 5.97e-04 [09/27 00:16:54] lb.utils.events INFO: eta: 10:40:46 iteration: 165399/375342 consumed_samples: 169369600 total_loss: 3.592 time: 0.3285 s/iter data_time: 0.2270 s/iter total_throughput: 3116.78 samples/s lr: 5.97e-04 [09/27 00:17:26] lb.utils.events INFO: eta: 10:11:04 iteration: 165499/375342 consumed_samples: 169472000 total_loss: 3.614 time: 0.3285 s/iter data_time: 0.2193 s/iter total_throughput: 3116.79 samples/s lr: 5.96e-04 [09/27 00:17:59] lb.utils.events INFO: eta: 9:57:10 iteration: 165599/375342 consumed_samples: 169574400 total_loss: 3.619 time: 0.3285 s/iter data_time: 0.2226 s/iter total_throughput: 3116.79 samples/s lr: 5.96e-04 [09/27 00:18:32] lb.utils.events INFO: eta: 9:56:59 iteration: 165699/375342 consumed_samples: 169676800 total_loss: 3.63 time: 0.3285 s/iter data_time: 0.2316 s/iter total_throughput: 3116.79 samples/s lr: 5.96e-04 [09/27 00:19:05] lb.utils.events INFO: eta: 10:03:56 iteration: 165799/375342 consumed_samples: 169779200 total_loss: 3.638 time: 0.3285 s/iter data_time: 0.2296 s/iter total_throughput: 3116.81 samples/s lr: 5.95e-04 [09/27 00:19:37] lb.utils.events INFO: eta: 11:02:57 iteration: 165899/375342 consumed_samples: 169881600 total_loss: 3.623 time: 0.3285 s/iter data_time: 0.2125 s/iter total_throughput: 3116.83 samples/s lr: 5.95e-04 [09/27 00:20:10] lb.utils.events INFO: eta: 11:57:01 iteration: 165999/375342 consumed_samples: 169984000 total_loss: 3.625 time: 0.3285 s/iter data_time: 0.1957 s/iter total_throughput: 3116.84 samples/s lr: 5.94e-04 [09/27 00:20:43] lb.utils.events INFO: eta: 10:01:42 iteration: 166099/375342 consumed_samples: 170086400 total_loss: 3.631 time: 0.3285 s/iter data_time: 0.2125 s/iter total_throughput: 3116.84 samples/s lr: 5.94e-04 [09/27 00:21:15] lb.utils.events INFO: eta: 9:33:56 iteration: 166199/375342 consumed_samples: 170188800 total_loss: 3.624 time: 0.3285 s/iter data_time: 0.2151 s/iter total_throughput: 3116.86 samples/s lr: 5.93e-04 [09/27 00:21:48] lb.utils.events INFO: eta: 9:22:21 iteration: 166299/375342 consumed_samples: 170291200 total_loss: 3.629 time: 0.3285 s/iter data_time: 0.2089 s/iter total_throughput: 3116.85 samples/s lr: 5.93e-04 [09/27 00:22:21] lb.utils.events INFO: eta: 9:14:38 iteration: 166399/375342 consumed_samples: 170393600 total_loss: 3.623 time: 0.3285 s/iter data_time: 0.2306 s/iter total_throughput: 3116.87 samples/s lr: 5.93e-04 [09/27 00:22:54] lb.utils.events INFO: eta: 9:14:08 iteration: 166499/375342 consumed_samples: 170496000 total_loss: 3.587 time: 0.3285 s/iter data_time: 0.2278 s/iter total_throughput: 3116.85 samples/s lr: 5.92e-04 [09/27 00:23:27] lb.utils.events INFO: eta: 9:09:45 iteration: 166599/375342 consumed_samples: 170598400 total_loss: 3.586 time: 0.3285 s/iter data_time: 0.1993 s/iter total_throughput: 3116.86 samples/s lr: 5.92e-04 [09/27 00:23:59] lb.utils.events INFO: eta: 9:06:43 iteration: 166699/375342 consumed_samples: 170700800 total_loss: 3.597 time: 0.3285 s/iter data_time: 0.2308 s/iter total_throughput: 3116.88 samples/s lr: 5.91e-04 [09/27 00:24:32] lb.utils.events INFO: eta: 9:06:21 iteration: 166799/375342 consumed_samples: 170803200 total_loss: 3.601 time: 0.3285 s/iter data_time: 0.2438 s/iter total_throughput: 3116.91 samples/s lr: 5.91e-04 [09/27 00:25:04] lb.utils.events INFO: eta: 9:03:27 iteration: 166899/375342 consumed_samples: 170905600 total_loss: 3.604 time: 0.3285 s/iter data_time: 0.2134 s/iter total_throughput: 3116.93 samples/s lr: 5.91e-04 [09/27 00:25:37] lb.utils.events INFO: eta: 9:04:41 iteration: 166999/375342 consumed_samples: 171008000 total_loss: 3.619 time: 0.3285 s/iter data_time: 0.2165 s/iter total_throughput: 3116.93 samples/s lr: 5.90e-04 [09/27 00:26:09] lb.utils.events INFO: eta: 9:10:22 iteration: 167099/375342 consumed_samples: 171110400 total_loss: 3.623 time: 0.3285 s/iter data_time: 0.2459 s/iter total_throughput: 3116.98 samples/s lr: 5.90e-04 [09/27 00:26:42] lb.utils.events INFO: eta: 9:19:39 iteration: 167199/375342 consumed_samples: 171212800 total_loss: 3.634 time: 0.3285 s/iter data_time: 0.2366 s/iter total_throughput: 3116.99 samples/s lr: 5.89e-04 [09/27 00:27:14] lb.utils.events INFO: eta: 9:42:56 iteration: 167299/375342 consumed_samples: 171315200 total_loss: 3.617 time: 0.3285 s/iter data_time: 0.2529 s/iter total_throughput: 3117.03 samples/s lr: 5.89e-04 [09/27 00:27:47] lb.utils.events INFO: eta: 11:40:53 iteration: 167399/375342 consumed_samples: 171417600 total_loss: 3.619 time: 0.3285 s/iter data_time: 0.2422 s/iter total_throughput: 3117.03 samples/s lr: 5.89e-04 [09/27 00:28:19] lb.utils.events INFO: eta: 12:26:43 iteration: 167499/375342 consumed_samples: 171520000 total_loss: 3.633 time: 0.3285 s/iter data_time: 0.2271 s/iter total_throughput: 3117.06 samples/s lr: 5.88e-04 [09/27 00:28:51] lb.utils.events INFO: eta: 13:08:40 iteration: 167599/375342 consumed_samples: 171622400 total_loss: 3.636 time: 0.3285 s/iter data_time: 0.2160 s/iter total_throughput: 3117.08 samples/s lr: 5.88e-04 [09/27 00:29:24] lb.utils.events INFO: eta: 13:48:14 iteration: 167699/375342 consumed_samples: 171724800 total_loss: 3.614 time: 0.3285 s/iter data_time: 0.2180 s/iter total_throughput: 3117.10 samples/s lr: 5.87e-04 [09/27 00:29:56] lb.utils.events INFO: eta: 13:28:06 iteration: 167799/375342 consumed_samples: 171827200 total_loss: 3.58 time: 0.3285 s/iter data_time: 0.2215 s/iter total_throughput: 3117.13 samples/s lr: 5.87e-04 [09/27 00:30:28] lb.utils.events INFO: eta: 13:50:19 iteration: 167899/375342 consumed_samples: 171929600 total_loss: 3.601 time: 0.3285 s/iter data_time: 0.2136 s/iter total_throughput: 3117.17 samples/s lr: 5.87e-04 [09/27 00:31:01] lb.utils.events INFO: eta: 12:48:16 iteration: 167999/375342 consumed_samples: 172032000 total_loss: 3.611 time: 0.3285 s/iter data_time: 0.2106 s/iter total_throughput: 3117.20 samples/s lr: 5.86e-04 [09/27 00:31:34] lb.utils.events INFO: eta: 13:08:54 iteration: 168099/375342 consumed_samples: 172134400 total_loss: 3.621 time: 0.3285 s/iter data_time: 0.2469 s/iter total_throughput: 3117.20 samples/s lr: 5.86e-04 [09/27 00:32:07] lb.utils.events INFO: eta: 11:57:13 iteration: 168199/375342 consumed_samples: 172236800 total_loss: 3.625 time: 0.3285 s/iter data_time: 0.2163 s/iter total_throughput: 3117.19 samples/s lr: 5.85e-04 [09/27 00:32:40] lb.utils.events INFO: eta: 10:31:59 iteration: 168299/375342 consumed_samples: 172339200 total_loss: 3.619 time: 0.3285 s/iter data_time: 0.2221 s/iter total_throughput: 3117.17 samples/s lr: 5.85e-04 [09/27 00:33:13] lb.utils.events INFO: eta: 9:38:23 iteration: 168399/375342 consumed_samples: 172441600 total_loss: 3.605 time: 0.3285 s/iter data_time: 0.2145 s/iter total_throughput: 3117.13 samples/s lr: 5.85e-04 [09/27 00:33:47] lb.utils.events INFO: eta: 9:19:07 iteration: 168499/375342 consumed_samples: 172544000 total_loss: 3.615 time: 0.3285 s/iter data_time: 0.2087 s/iter total_throughput: 3117.11 samples/s lr: 5.84e-04 [09/27 00:34:20] lb.utils.events INFO: eta: 9:08:53 iteration: 168599/375342 consumed_samples: 172646400 total_loss: 3.619 time: 0.3285 s/iter data_time: 0.2117 s/iter total_throughput: 3117.08 samples/s lr: 5.84e-04 [09/27 00:34:54] lb.utils.events INFO: eta: 9:04:02 iteration: 168699/375342 consumed_samples: 172748800 total_loss: 3.611 time: 0.3285 s/iter data_time: 0.2084 s/iter total_throughput: 3117.04 samples/s lr: 5.83e-04 [09/27 00:35:27] lb.utils.events INFO: eta: 9:00:06 iteration: 168799/375342 consumed_samples: 172851200 total_loss: 3.613 time: 0.3285 s/iter data_time: 0.2045 s/iter total_throughput: 3116.99 samples/s lr: 5.83e-04 [09/27 00:36:00] lb.utils.events INFO: eta: 8:59:08 iteration: 168899/375342 consumed_samples: 172953600 total_loss: 3.621 time: 0.3285 s/iter data_time: 0.2200 s/iter total_throughput: 3116.98 samples/s lr: 5.82e-04 [09/27 00:36:33] lb.utils.events INFO: eta: 9:02:10 iteration: 168999/375342 consumed_samples: 173056000 total_loss: 3.617 time: 0.3285 s/iter data_time: 0.2223 s/iter total_throughput: 3116.99 samples/s lr: 5.82e-04 [09/27 00:37:07] lb.utils.events INFO: eta: 8:58:12 iteration: 169099/375342 consumed_samples: 173158400 total_loss: 3.604 time: 0.3285 s/iter data_time: 0.2084 s/iter total_throughput: 3116.95 samples/s lr: 5.82e-04 [09/27 00:37:40] lb.utils.events INFO: eta: 8:56:15 iteration: 169199/375342 consumed_samples: 173260800 total_loss: 3.585 time: 0.3285 s/iter data_time: 0.2380 s/iter total_throughput: 3116.94 samples/s lr: 5.81e-04 [09/27 00:38:13] lb.utils.events INFO: eta: 8:54:39 iteration: 169299/375342 consumed_samples: 173363200 total_loss: 3.613 time: 0.3285 s/iter data_time: 0.2134 s/iter total_throughput: 3116.91 samples/s lr: 5.81e-04 [09/27 00:38:46] lb.utils.events INFO: eta: 8:52:14 iteration: 169399/375342 consumed_samples: 173465600 total_loss: 3.63 time: 0.3285 s/iter data_time: 0.2151 s/iter total_throughput: 3116.90 samples/s lr: 5.80e-04 [09/27 00:39:19] lb.utils.events INFO: eta: 8:55:10 iteration: 169499/375342 consumed_samples: 173568000 total_loss: 3.634 time: 0.3285 s/iter data_time: 0.2407 s/iter total_throughput: 3116.90 samples/s lr: 5.80e-04 [09/27 00:39:52] lb.utils.events INFO: eta: 8:58:39 iteration: 169599/375342 consumed_samples: 173670400 total_loss: 3.619 time: 0.3285 s/iter data_time: 0.2526 s/iter total_throughput: 3116.88 samples/s lr: 5.80e-04 [09/27 00:40:25] lb.utils.events INFO: eta: 9:03:32 iteration: 169699/375342 consumed_samples: 173772800 total_loss: 3.585 time: 0.3285 s/iter data_time: 0.2273 s/iter total_throughput: 3116.86 samples/s lr: 5.79e-04 [09/27 00:40:58] lb.utils.events INFO: eta: 9:08:18 iteration: 169799/375342 consumed_samples: 173875200 total_loss: 3.614 time: 0.3285 s/iter data_time: 0.2234 s/iter total_throughput: 3116.86 samples/s lr: 5.79e-04 [09/27 00:41:31] lb.utils.events INFO: eta: 9:19:02 iteration: 169899/375342 consumed_samples: 173977600 total_loss: 3.646 time: 0.3285 s/iter data_time: 0.2465 s/iter total_throughput: 3116.85 samples/s lr: 5.78e-04 [09/27 00:42:05] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0169999 [09/27 00:42:05] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/27 00:42:05] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/27 00:42:09] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0848 s/iter. Inference: 0.1488 s/iter. Eval: 0.0021 s/iter. Total: 0.2357 s/iter. ETA=0:00:08 [09/27 00:42:15] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1466 s/iter. Inference: 0.1483 s/iter. Eval: 0.0021 s/iter. Total: 0.2970 s/iter. ETA=0:00:05 [09/27 00:42:20] lb.evaluation.evaluator INFO: Inference done 49152/50000. Dataloading: 0.1301 s/iter. Inference: 0.1501 s/iter. Eval: 0.0021 s/iter. Total: 0.2823 s/iter. ETA=0:00:00 [09/27 00:42:20] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/27 00:42:20] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.563832 (0.000251 s / iter per device, on 8 devices) [09/27 00:42:20] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000133 s / iter per device, on 8 devices) [09/27 00:42:20] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/27 00:42:20] lb.evaluation.utils INFO: copypaste: Acc@1=74.404 [09/27 00:42:20] lb.evaluation.utils INFO: copypaste: Acc@5=92.31 [09/27 00:42:20] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 74.40400, better than last best score 74.32000 @ iteration 164999. [09/27 00:42:20] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/27 00:42:21] lb.utils.events INFO: eta: 9:33:25 iteration: 169999/375342 consumed_samples: 174080000 total_loss: 3.608 time: 0.3285 s/iter data_time: 0.2266 s/iter total_throughput: 3116.83 samples/s lr: 5.78e-04 [09/27 00:42:53] lb.utils.events INFO: eta: 9:33:09 iteration: 170099/375342 consumed_samples: 174182400 total_loss: 3.625 time: 0.3285 s/iter data_time: 0.2287 s/iter total_throughput: 3116.91 samples/s lr: 5.78e-04 [09/27 00:43:26] lb.utils.events INFO: eta: 9:33:55 iteration: 170199/375342 consumed_samples: 174284800 total_loss: 3.603 time: 0.3285 s/iter data_time: 0.2326 s/iter total_throughput: 3116.88 samples/s lr: 5.77e-04 [09/27 00:43:59] lb.utils.events INFO: eta: 9:37:34 iteration: 170299/375342 consumed_samples: 174387200 total_loss: 3.602 time: 0.3285 s/iter data_time: 0.2228 s/iter total_throughput: 3116.86 samples/s lr: 5.77e-04 [09/27 00:44:32] lb.utils.events INFO: eta: 9:57:02 iteration: 170399/375342 consumed_samples: 174489600 total_loss: 3.613 time: 0.3285 s/iter data_time: 0.2260 s/iter total_throughput: 3116.86 samples/s lr: 5.76e-04 [09/27 00:45:05] lb.utils.events INFO: eta: 9:59:49 iteration: 170499/375342 consumed_samples: 174592000 total_loss: 3.613 time: 0.3285 s/iter data_time: 0.2228 s/iter total_throughput: 3116.84 samples/s lr: 5.76e-04 [09/27 00:45:38] lb.utils.events INFO: eta: 9:56:27 iteration: 170599/375342 consumed_samples: 174694400 total_loss: 3.63 time: 0.3285 s/iter data_time: 0.2355 s/iter total_throughput: 3116.83 samples/s lr: 5.75e-04 [09/27 00:46:12] lb.utils.events INFO: eta: 10:25:04 iteration: 170699/375342 consumed_samples: 174796800 total_loss: 3.638 time: 0.3285 s/iter data_time: 0.2508 s/iter total_throughput: 3116.81 samples/s lr: 5.75e-04 [09/27 00:46:45] lb.utils.events INFO: eta: 11:02:06 iteration: 170799/375342 consumed_samples: 174899200 total_loss: 3.62 time: 0.3285 s/iter data_time: 0.2249 s/iter total_throughput: 3116.81 samples/s lr: 5.75e-04 [09/27 00:47:18] lb.utils.events INFO: eta: 9:41:36 iteration: 170899/375342 consumed_samples: 175001600 total_loss: 3.604 time: 0.3285 s/iter data_time: 0.2364 s/iter total_throughput: 3116.77 samples/s lr: 5.74e-04 [09/27 00:47:51] lb.utils.events INFO: eta: 9:27:57 iteration: 170999/375342 consumed_samples: 175104000 total_loss: 3.609 time: 0.3285 s/iter data_time: 0.2500 s/iter total_throughput: 3116.75 samples/s lr: 5.74e-04 [09/27 00:48:25] lb.utils.events INFO: eta: 9:22:31 iteration: 171099/375342 consumed_samples: 175206400 total_loss: 3.611 time: 0.3286 s/iter data_time: 0.2171 s/iter total_throughput: 3116.71 samples/s lr: 5.73e-04 [09/27 00:48:58] lb.utils.events INFO: eta: 9:14:52 iteration: 171199/375342 consumed_samples: 175308800 total_loss: 3.608 time: 0.3286 s/iter data_time: 0.2048 s/iter total_throughput: 3116.71 samples/s lr: 5.73e-04 [09/27 00:49:31] lb.utils.events INFO: eta: 9:06:19 iteration: 171299/375342 consumed_samples: 175411200 total_loss: 3.604 time: 0.3286 s/iter data_time: 0.2052 s/iter total_throughput: 3116.71 samples/s lr: 5.73e-04 [09/27 00:50:04] lb.utils.events INFO: eta: 9:12:12 iteration: 171399/375342 consumed_samples: 175513600 total_loss: 3.59 time: 0.3286 s/iter data_time: 0.2431 s/iter total_throughput: 3116.70 samples/s lr: 5.72e-04 [09/27 00:50:37] lb.utils.events INFO: eta: 9:21:35 iteration: 171499/375342 consumed_samples: 175616000 total_loss: 3.574 time: 0.3286 s/iter data_time: 0.2329 s/iter total_throughput: 3116.70 samples/s lr: 5.72e-04 [09/27 00:51:10] lb.utils.events INFO: eta: 9:28:34 iteration: 171599/375342 consumed_samples: 175718400 total_loss: 3.584 time: 0.3286 s/iter data_time: 0.2323 s/iter total_throughput: 3116.68 samples/s lr: 5.71e-04 [09/27 00:51:43] lb.utils.events INFO: eta: 9:29:12 iteration: 171699/375342 consumed_samples: 175820800 total_loss: 3.584 time: 0.3286 s/iter data_time: 0.2335 s/iter total_throughput: 3116.65 samples/s lr: 5.71e-04 [09/27 00:52:16] lb.utils.events INFO: eta: 9:20:17 iteration: 171799/375342 consumed_samples: 175923200 total_loss: 3.569 time: 0.3286 s/iter data_time: 0.2221 s/iter total_throughput: 3116.65 samples/s lr: 5.71e-04 [09/27 00:52:49] lb.utils.events INFO: eta: 9:31:14 iteration: 171899/375342 consumed_samples: 176025600 total_loss: 3.588 time: 0.3286 s/iter data_time: 0.2256 s/iter total_throughput: 3116.64 samples/s lr: 5.70e-04 [09/27 00:53:22] lb.utils.events INFO: eta: 9:33:53 iteration: 171999/375342 consumed_samples: 176128000 total_loss: 3.609 time: 0.3286 s/iter data_time: 0.2381 s/iter total_throughput: 3116.63 samples/s lr: 5.70e-04 [09/27 00:53:55] lb.utils.events INFO: eta: 11:36:06 iteration: 172099/375342 consumed_samples: 176230400 total_loss: 3.589 time: 0.3286 s/iter data_time: 0.2625 s/iter total_throughput: 3116.63 samples/s lr: 5.69e-04 [09/27 00:54:28] lb.utils.events INFO: eta: 14:03:37 iteration: 172199/375342 consumed_samples: 176332800 total_loss: 3.582 time: 0.3286 s/iter data_time: 0.2472 s/iter total_throughput: 3116.61 samples/s lr: 5.69e-04 [09/27 00:55:02] lb.utils.events INFO: eta: 14:58:02 iteration: 172299/375342 consumed_samples: 176435200 total_loss: 3.58 time: 0.3286 s/iter data_time: 0.2080 s/iter total_throughput: 3116.57 samples/s lr: 5.69e-04 [09/27 00:55:35] lb.utils.events INFO: eta: 14:30:58 iteration: 172399/375342 consumed_samples: 176537600 total_loss: 3.584 time: 0.3286 s/iter data_time: 0.2308 s/iter total_throughput: 3116.56 samples/s lr: 5.68e-04 [09/27 00:56:07] lb.utils.events INFO: eta: 12:33:48 iteration: 172499/375342 consumed_samples: 176640000 total_loss: 3.604 time: 0.3286 s/iter data_time: 0.2379 s/iter total_throughput: 3116.58 samples/s lr: 5.68e-04 [09/27 00:56:41] lb.utils.events INFO: eta: 11:21:10 iteration: 172599/375342 consumed_samples: 176742400 total_loss: 3.599 time: 0.3286 s/iter data_time: 0.2105 s/iter total_throughput: 3116.55 samples/s lr: 5.67e-04 [09/27 00:57:14] lb.utils.events INFO: eta: 9:30:55 iteration: 172699/375342 consumed_samples: 176844800 total_loss: 3.591 time: 0.3286 s/iter data_time: 0.2047 s/iter total_throughput: 3116.55 samples/s lr: 5.67e-04 [09/27 00:57:47] lb.utils.events INFO: eta: 9:41:10 iteration: 172799/375342 consumed_samples: 176947200 total_loss: 3.593 time: 0.3286 s/iter data_time: 0.2232 s/iter total_throughput: 3116.54 samples/s lr: 5.66e-04 [09/27 00:58:20] lb.utils.events INFO: eta: 9:43:03 iteration: 172899/375342 consumed_samples: 177049600 total_loss: 3.594 time: 0.3286 s/iter data_time: 0.2204 s/iter total_throughput: 3116.51 samples/s lr: 5.66e-04 [09/27 00:58:53] lb.utils.events INFO: eta: 9:33:37 iteration: 172999/375342 consumed_samples: 177152000 total_loss: 3.586 time: 0.3286 s/iter data_time: 0.2215 s/iter total_throughput: 3116.48 samples/s lr: 5.66e-04 [09/27 00:59:27] lb.utils.events INFO: eta: 9:14:42 iteration: 173099/375342 consumed_samples: 177254400 total_loss: 3.586 time: 0.3286 s/iter data_time: 0.2315 s/iter total_throughput: 3116.47 samples/s lr: 5.65e-04 [09/27 00:59:59] lb.utils.events INFO: eta: 9:04:20 iteration: 173199/375342 consumed_samples: 177356800 total_loss: 3.591 time: 0.3286 s/iter data_time: 0.2506 s/iter total_throughput: 3116.48 samples/s lr: 5.65e-04 [09/27 01:00:32] lb.utils.events INFO: eta: 9:14:29 iteration: 173299/375342 consumed_samples: 177459200 total_loss: 3.582 time: 0.3286 s/iter data_time: 0.2444 s/iter total_throughput: 3116.48 samples/s lr: 5.64e-04 [09/27 01:01:05] lb.utils.events INFO: eta: 9:18:01 iteration: 173399/375342 consumed_samples: 177561600 total_loss: 3.588 time: 0.3286 s/iter data_time: 0.2423 s/iter total_throughput: 3116.46 samples/s lr: 5.64e-04 [09/27 01:01:39] lb.utils.events INFO: eta: 9:26:15 iteration: 173499/375342 consumed_samples: 177664000 total_loss: 3.593 time: 0.3286 s/iter data_time: 0.2168 s/iter total_throughput: 3116.43 samples/s lr: 5.64e-04 [09/27 01:02:12] lb.utils.events INFO: eta: 9:45:00 iteration: 173599/375342 consumed_samples: 177766400 total_loss: 3.584 time: 0.3286 s/iter data_time: 0.2320 s/iter total_throughput: 3116.41 samples/s lr: 5.63e-04 [09/27 01:02:45] lb.utils.events INFO: eta: 11:45:57 iteration: 173699/375342 consumed_samples: 177868800 total_loss: 3.597 time: 0.3286 s/iter data_time: 0.2405 s/iter total_throughput: 3116.41 samples/s lr: 5.63e-04 [09/27 01:03:18] lb.utils.events INFO: eta: 12:36:35 iteration: 173799/375342 consumed_samples: 177971200 total_loss: 3.604 time: 0.3286 s/iter data_time: 0.2405 s/iter total_throughput: 3116.41 samples/s lr: 5.62e-04 [09/27 01:03:51] lb.utils.events INFO: eta: 12:47:01 iteration: 173899/375342 consumed_samples: 178073600 total_loss: 3.604 time: 0.3286 s/iter data_time: 0.2431 s/iter total_throughput: 3116.41 samples/s lr: 5.62e-04 [09/27 01:04:24] lb.utils.events INFO: eta: 12:39:40 iteration: 173999/375342 consumed_samples: 178176000 total_loss: 3.577 time: 0.3286 s/iter data_time: 0.2253 s/iter total_throughput: 3116.39 samples/s lr: 5.62e-04 [09/27 01:04:57] lb.utils.events INFO: eta: 12:56:15 iteration: 174099/375342 consumed_samples: 178278400 total_loss: 3.57 time: 0.3286 s/iter data_time: 0.2247 s/iter total_throughput: 3116.37 samples/s lr: 5.61e-04 [09/27 01:05:30] lb.utils.events INFO: eta: 13:28:55 iteration: 174199/375342 consumed_samples: 178380800 total_loss: 3.584 time: 0.3286 s/iter data_time: 0.2231 s/iter total_throughput: 3116.35 samples/s lr: 5.61e-04 [09/27 01:06:04] lb.utils.events INFO: eta: 11:31:20 iteration: 174299/375342 consumed_samples: 178483200 total_loss: 3.595 time: 0.3286 s/iter data_time: 0.2127 s/iter total_throughput: 3116.33 samples/s lr: 5.60e-04 [09/27 01:06:37] lb.utils.events INFO: eta: 11:08:43 iteration: 174399/375342 consumed_samples: 178585600 total_loss: 3.597 time: 0.3286 s/iter data_time: 0.2290 s/iter total_throughput: 3116.31 samples/s lr: 5.60e-04 [09/27 01:07:10] lb.utils.events INFO: eta: 10:20:22 iteration: 174499/375342 consumed_samples: 178688000 total_loss: 3.593 time: 0.3286 s/iter data_time: 0.2185 s/iter total_throughput: 3116.30 samples/s lr: 5.59e-04 [09/27 01:07:43] lb.utils.events INFO: eta: 9:57:13 iteration: 174599/375342 consumed_samples: 178790400 total_loss: 3.591 time: 0.3286 s/iter data_time: 0.2280 s/iter total_throughput: 3116.28 samples/s lr: 5.59e-04 [09/27 01:08:16] lb.utils.events INFO: eta: 9:44:35 iteration: 174699/375342 consumed_samples: 178892800 total_loss: 3.592 time: 0.3286 s/iter data_time: 0.2223 s/iter total_throughput: 3116.26 samples/s lr: 5.59e-04 [09/27 01:08:50] lb.utils.events INFO: eta: 9:08:49 iteration: 174799/375342 consumed_samples: 178995200 total_loss: 3.596 time: 0.3286 s/iter data_time: 0.2103 s/iter total_throughput: 3116.23 samples/s lr: 5.58e-04 [09/27 01:09:23] lb.utils.events INFO: eta: 8:58:41 iteration: 174899/375342 consumed_samples: 179097600 total_loss: 3.596 time: 0.3286 s/iter data_time: 0.2205 s/iter total_throughput: 3116.23 samples/s lr: 5.58e-04 [09/27 01:09:55] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0174999 [09/27 01:09:56] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/27 01:09:56] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/27 01:10:00] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0875 s/iter. Inference: 0.1508 s/iter. Eval: 0.0022 s/iter. Total: 0.2405 s/iter. ETA=0:00:08 [09/27 01:10:06] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1511 s/iter. Inference: 0.1496 s/iter. Eval: 0.0021 s/iter. Total: 0.3029 s/iter. ETA=0:00:05 [09/27 01:10:11] lb.evaluation.evaluator INFO: Inference done 49152/50000. Dataloading: 0.1351 s/iter. Inference: 0.1488 s/iter. Eval: 0.0021 s/iter. Total: 0.2861 s/iter. ETA=0:00:00 [09/27 01:10:12] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/27 01:10:12] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.752403 (0.000255 s / iter per device, on 8 devices) [09/27 01:10:12] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000132 s / iter per device, on 8 devices) [09/27 01:10:12] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/27 01:10:12] lb.evaluation.utils INFO: copypaste: Acc@1=74.298 [09/27 01:10:12] lb.evaluation.utils INFO: copypaste: Acc@5=92.074 [09/27 01:10:12] lb.engine.hooks INFO: Not saving as latest eval score for Acc@1 is 74.29800, not better than best score 74.40400 @ iteration 169999. [09/27 01:10:12] lb.utils.events INFO: eta: 8:53:39 iteration: 174999/375342 consumed_samples: 179200000 total_loss: 3.605 time: 0.3286 s/iter data_time: 0.2189 s/iter total_throughput: 3116.24 samples/s lr: 5.57e-04 [09/27 01:10:43] lb.utils.events INFO: eta: 8:51:16 iteration: 175099/375342 consumed_samples: 179302400 total_loss: 3.6 time: 0.3286 s/iter data_time: 0.2205 s/iter total_throughput: 3116.32 samples/s lr: 5.57e-04 [09/27 01:11:15] lb.utils.events INFO: eta: 8:53:04 iteration: 175199/375342 consumed_samples: 179404800 total_loss: 3.588 time: 0.3286 s/iter data_time: 0.2522 s/iter total_throughput: 3116.34 samples/s lr: 5.57e-04 [09/27 01:11:49] lb.utils.events INFO: eta: 8:57:13 iteration: 175299/375342 consumed_samples: 179507200 total_loss: 3.588 time: 0.3286 s/iter data_time: 0.2607 s/iter total_throughput: 3116.30 samples/s lr: 5.56e-04 [09/27 01:12:22] lb.utils.events INFO: eta: 9:10:42 iteration: 175399/375342 consumed_samples: 179609600 total_loss: 3.593 time: 0.3286 s/iter data_time: 0.2609 s/iter total_throughput: 3116.31 samples/s lr: 5.56e-04 [09/27 01:12:55] lb.utils.events INFO: eta: 9:23:26 iteration: 175499/375342 consumed_samples: 179712000 total_loss: 3.585 time: 0.3286 s/iter data_time: 0.2369 s/iter total_throughput: 3116.27 samples/s lr: 5.55e-04 [09/27 01:13:28] lb.utils.events INFO: eta: 10:10:20 iteration: 175599/375342 consumed_samples: 179814400 total_loss: 3.566 time: 0.3286 s/iter data_time: 0.2471 s/iter total_throughput: 3116.28 samples/s lr: 5.55e-04 [09/27 01:14:01] lb.utils.events INFO: eta: 11:13:11 iteration: 175699/375342 consumed_samples: 179916800 total_loss: 3.577 time: 0.3286 s/iter data_time: 0.2421 s/iter total_throughput: 3116.25 samples/s lr: 5.55e-04 [09/27 01:14:34] lb.utils.events INFO: eta: 11:52:58 iteration: 175799/375342 consumed_samples: 180019200 total_loss: 3.577 time: 0.3286 s/iter data_time: 0.2283 s/iter total_throughput: 3116.25 samples/s lr: 5.54e-04 [09/27 01:15:07] lb.utils.events INFO: eta: 13:29:57 iteration: 175899/375342 consumed_samples: 180121600 total_loss: 3.582 time: 0.3286 s/iter data_time: 0.2355 s/iter total_throughput: 3116.24 samples/s lr: 5.54e-04 [09/27 01:15:40] lb.utils.events INFO: eta: 14:29:31 iteration: 175999/375342 consumed_samples: 180224000 total_loss: 3.591 time: 0.3286 s/iter data_time: 0.2266 s/iter total_throughput: 3116.25 samples/s lr: 5.53e-04 [09/27 01:16:13] lb.utils.events INFO: eta: 13:59:59 iteration: 176099/375342 consumed_samples: 180326400 total_loss: 3.569 time: 0.3286 s/iter data_time: 0.2315 s/iter total_throughput: 3116.27 samples/s lr: 5.53e-04 [09/27 01:16:46] lb.utils.events INFO: eta: 13:59:17 iteration: 176199/375342 consumed_samples: 180428800 total_loss: 3.573 time: 0.3286 s/iter data_time: 0.2242 s/iter total_throughput: 3116.26 samples/s lr: 5.52e-04 [09/27 01:17:18] lb.utils.events INFO: eta: 14:04:25 iteration: 176299/375342 consumed_samples: 180531200 total_loss: 3.574 time: 0.3286 s/iter data_time: 0.2500 s/iter total_throughput: 3116.26 samples/s lr: 5.52e-04 [09/27 01:17:51] lb.utils.events INFO: eta: 12:50:38 iteration: 176399/375342 consumed_samples: 180633600 total_loss: 3.568 time: 0.3286 s/iter data_time: 0.2081 s/iter total_throughput: 3116.25 samples/s lr: 5.52e-04 [09/27 01:18:24] lb.utils.events INFO: eta: 11:07:40 iteration: 176499/375342 consumed_samples: 180736000 total_loss: 3.567 time: 0.3286 s/iter data_time: 0.2247 s/iter total_throughput: 3116.27 samples/s lr: 5.51e-04 [09/27 01:18:57] lb.utils.events INFO: eta: 10:35:31 iteration: 176599/375342 consumed_samples: 180838400 total_loss: 3.566 time: 0.3286 s/iter data_time: 0.2144 s/iter total_throughput: 3116.27 samples/s lr: 5.51e-04 [09/27 01:19:30] lb.utils.events INFO: eta: 9:19:27 iteration: 176699/375342 consumed_samples: 180940800 total_loss: 3.57 time: 0.3286 s/iter data_time: 0.2347 s/iter total_throughput: 3116.26 samples/s lr: 5.50e-04 [09/27 01:20:03] lb.utils.events INFO: eta: 9:08:43 iteration: 176799/375342 consumed_samples: 181043200 total_loss: 3.57 time: 0.3286 s/iter data_time: 0.2272 s/iter total_throughput: 3116.24 samples/s lr: 5.50e-04 [09/27 01:20:35] lb.utils.events INFO: eta: 8:59:10 iteration: 176899/375342 consumed_samples: 181145600 total_loss: 3.574 time: 0.3286 s/iter data_time: 0.2117 s/iter total_throughput: 3116.28 samples/s lr: 5.50e-04 [09/27 01:21:08] lb.utils.events INFO: eta: 9:08:14 iteration: 176999/375342 consumed_samples: 181248000 total_loss: 3.565 time: 0.3286 s/iter data_time: 0.2594 s/iter total_throughput: 3116.28 samples/s lr: 5.49e-04 [09/27 01:21:41] lb.utils.events INFO: eta: 10:55:59 iteration: 177099/375342 consumed_samples: 181350400 total_loss: 3.568 time: 0.3286 s/iter data_time: 0.2608 s/iter total_throughput: 3116.29 samples/s lr: 5.49e-04 [09/27 01:22:14] lb.utils.events INFO: eta: 9:44:20 iteration: 177199/375342 consumed_samples: 181452800 total_loss: 3.582 time: 0.3286 s/iter data_time: 0.2256 s/iter total_throughput: 3116.29 samples/s lr: 5.48e-04 [09/27 01:22:46] lb.utils.events INFO: eta: 9:22:33 iteration: 177299/375342 consumed_samples: 181555200 total_loss: 3.594 time: 0.3286 s/iter data_time: 0.2174 s/iter total_throughput: 3116.30 samples/s lr: 5.48e-04 [09/27 01:23:19] lb.utils.events INFO: eta: 9:33:56 iteration: 177399/375342 consumed_samples: 181657600 total_loss: 3.594 time: 0.3286 s/iter data_time: 0.2480 s/iter total_throughput: 3116.29 samples/s lr: 5.48e-04 [09/27 01:23:52] lb.utils.events INFO: eta: 10:00:16 iteration: 177499/375342 consumed_samples: 181760000 total_loss: 3.579 time: 0.3286 s/iter data_time: 0.2108 s/iter total_throughput: 3116.29 samples/s lr: 5.47e-04 [09/27 01:24:25] lb.utils.events INFO: eta: 9:43:09 iteration: 177599/375342 consumed_samples: 181862400 total_loss: 3.576 time: 0.3286 s/iter data_time: 0.2148 s/iter total_throughput: 3116.27 samples/s lr: 5.47e-04 [09/27 01:24:58] lb.utils.events INFO: eta: 9:45:39 iteration: 177699/375342 consumed_samples: 181964800 total_loss: 3.587 time: 0.3286 s/iter data_time: 0.2193 s/iter total_throughput: 3116.28 samples/s lr: 5.46e-04 [09/27 01:25:31] lb.utils.events INFO: eta: 9:50:29 iteration: 177799/375342 consumed_samples: 182067200 total_loss: 3.616 time: 0.3286 s/iter data_time: 0.2243 s/iter total_throughput: 3116.28 samples/s lr: 5.46e-04 [09/27 01:26:04] lb.utils.events INFO: eta: 10:38:19 iteration: 177899/375342 consumed_samples: 182169600 total_loss: 3.594 time: 0.3286 s/iter data_time: 0.2165 s/iter total_throughput: 3116.28 samples/s lr: 5.45e-04 [09/27 01:26:36] lb.utils.events INFO: eta: 9:36:01 iteration: 177999/375342 consumed_samples: 182272000 total_loss: 3.569 time: 0.3286 s/iter data_time: 0.2395 s/iter total_throughput: 3116.30 samples/s lr: 5.45e-04 [09/27 01:27:09] lb.utils.events INFO: eta: 9:03:07 iteration: 178099/375342 consumed_samples: 182374400 total_loss: 3.574 time: 0.3286 s/iter data_time: 0.2288 s/iter total_throughput: 3116.30 samples/s lr: 5.45e-04 [09/27 01:27:42] lb.utils.events INFO: eta: 8:58:06 iteration: 178199/375342 consumed_samples: 182476800 total_loss: 3.598 time: 0.3286 s/iter data_time: 0.2266 s/iter total_throughput: 3116.30 samples/s lr: 5.44e-04 [09/27 01:28:15] lb.utils.events INFO: eta: 9:00:47 iteration: 178299/375342 consumed_samples: 182579200 total_loss: 3.596 time: 0.3286 s/iter data_time: 0.2270 s/iter total_throughput: 3116.31 samples/s lr: 5.44e-04 [09/27 01:28:48] lb.utils.events INFO: eta: 8:58:38 iteration: 178399/375342 consumed_samples: 182681600 total_loss: 3.583 time: 0.3286 s/iter data_time: 0.2157 s/iter total_throughput: 3116.31 samples/s lr: 5.43e-04 [09/27 01:29:21] lb.utils.events INFO: eta: 8:59:12 iteration: 178499/375342 consumed_samples: 182784000 total_loss: 3.584 time: 0.3286 s/iter data_time: 0.2214 s/iter total_throughput: 3116.32 samples/s lr: 5.43e-04 [09/27 01:29:53] lb.utils.events INFO: eta: 8:56:32 iteration: 178599/375342 consumed_samples: 182886400 total_loss: 3.587 time: 0.3286 s/iter data_time: 0.2253 s/iter total_throughput: 3116.33 samples/s lr: 5.43e-04 [09/27 01:30:26] lb.utils.events INFO: eta: 8:57:57 iteration: 178699/375342 consumed_samples: 182988800 total_loss: 3.574 time: 0.3286 s/iter data_time: 0.2228 s/iter total_throughput: 3116.32 samples/s lr: 5.42e-04 [09/27 01:30:58] lb.utils.events INFO: eta: 9:03:44 iteration: 178799/375342 consumed_samples: 183091200 total_loss: 3.57 time: 0.3286 s/iter data_time: 0.2368 s/iter total_throughput: 3116.35 samples/s lr: 5.42e-04 [09/27 01:31:31] lb.utils.events INFO: eta: 9:09:57 iteration: 178899/375342 consumed_samples: 183193600 total_loss: 3.578 time: 0.3286 s/iter data_time: 0.2271 s/iter total_throughput: 3116.35 samples/s lr: 5.41e-04 [09/27 01:32:04] lb.utils.events INFO: eta: 9:12:02 iteration: 178999/375342 consumed_samples: 183296000 total_loss: 3.578 time: 0.3286 s/iter data_time: 0.2451 s/iter total_throughput: 3116.36 samples/s lr: 5.41e-04 [09/27 01:32:36] lb.utils.events INFO: eta: 9:14:46 iteration: 179099/375342 consumed_samples: 183398400 total_loss: 3.573 time: 0.3286 s/iter data_time: 0.2308 s/iter total_throughput: 3116.39 samples/s lr: 5.40e-04 [09/27 01:33:09] lb.utils.events INFO: eta: 9:48:36 iteration: 179199/375342 consumed_samples: 183500800 total_loss: 3.567 time: 0.3286 s/iter data_time: 0.2178 s/iter total_throughput: 3116.40 samples/s lr: 5.40e-04 [09/27 01:33:42] lb.utils.events INFO: eta: 9:45:47 iteration: 179299/375342 consumed_samples: 183603200 total_loss: 3.57 time: 0.3286 s/iter data_time: 0.2206 s/iter total_throughput: 3116.38 samples/s lr: 5.40e-04 [09/27 01:34:15] lb.utils.events INFO: eta: 10:10:59 iteration: 179399/375342 consumed_samples: 183705600 total_loss: 3.589 time: 0.3286 s/iter data_time: 0.2434 s/iter total_throughput: 3116.40 samples/s lr: 5.39e-04 [09/27 01:34:47] lb.utils.events INFO: eta: 10:38:14 iteration: 179499/375342 consumed_samples: 183808000 total_loss: 3.578 time: 0.3286 s/iter data_time: 0.2310 s/iter total_throughput: 3116.42 samples/s lr: 5.39e-04 [09/27 01:35:20] lb.utils.events INFO: eta: 12:12:54 iteration: 179599/375342 consumed_samples: 183910400 total_loss: 3.59 time: 0.3286 s/iter data_time: 0.2411 s/iter total_throughput: 3116.43 samples/s lr: 5.38e-04 [09/27 01:35:53] lb.utils.events INFO: eta: 12:41:04 iteration: 179699/375342 consumed_samples: 184012800 total_loss: 3.594 time: 0.3286 s/iter data_time: 0.2359 s/iter total_throughput: 3116.42 samples/s lr: 5.38e-04 [09/27 01:36:26] lb.utils.events INFO: eta: 12:12:59 iteration: 179799/375342 consumed_samples: 184115200 total_loss: 3.59 time: 0.3286 s/iter data_time: 0.2318 s/iter total_throughput: 3116.41 samples/s lr: 5.38e-04 [09/27 01:36:59] lb.utils.events INFO: eta: 11:51:11 iteration: 179899/375342 consumed_samples: 184217600 total_loss: 3.592 time: 0.3286 s/iter data_time: 0.2215 s/iter total_throughput: 3116.41 samples/s lr: 5.37e-04 [09/27 01:37:31] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0179999 [09/27 01:37:32] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/27 01:37:32] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/27 01:37:36] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0828 s/iter. Inference: 0.1508 s/iter. Eval: 0.0020 s/iter. Total: 0.2356 s/iter. ETA=0:00:08 [09/27 01:37:42] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1434 s/iter. Inference: 0.1488 s/iter. Eval: 0.0020 s/iter. Total: 0.2944 s/iter. ETA=0:00:05 [09/27 01:37:47] lb.evaluation.evaluator INFO: Inference done 49152/50000. Dataloading: 0.1293 s/iter. Inference: 0.1496 s/iter. Eval: 0.0020 s/iter. Total: 0.2810 s/iter. ETA=0:00:00 [09/27 01:37:47] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/27 01:37:47] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.566632 (0.000251 s / iter per device, on 8 devices) [09/27 01:37:47] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000133 s / iter per device, on 8 devices) [09/27 01:37:47] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/27 01:37:47] lb.evaluation.utils INFO: copypaste: Acc@1=74.738 [09/27 01:37:47] lb.evaluation.utils INFO: copypaste: Acc@5=92.364 [09/27 01:37:47] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 74.73800, better than last best score 74.40400 @ iteration 169999. [09/27 01:37:47] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/27 01:37:48] lb.utils.events INFO: eta: 11:59:47 iteration: 179999/375342 consumed_samples: 184320000 total_loss: 3.591 time: 0.3286 s/iter data_time: 0.2380 s/iter total_throughput: 3116.43 samples/s lr: 5.37e-04 [09/27 01:38:19] lb.utils.events INFO: eta: 11:18:32 iteration: 180099/375342 consumed_samples: 184422400 total_loss: 3.584 time: 0.3286 s/iter data_time: 0.2315 s/iter total_throughput: 3116.52 samples/s lr: 5.36e-04 [09/27 01:38:52] lb.utils.events INFO: eta: 10:56:55 iteration: 180199/375342 consumed_samples: 184524800 total_loss: 3.563 time: 0.3286 s/iter data_time: 0.2296 s/iter total_throughput: 3116.50 samples/s lr: 5.36e-04 [09/27 01:39:25] lb.utils.events INFO: eta: 11:23:31 iteration: 180299/375342 consumed_samples: 184627200 total_loss: 3.563 time: 0.3286 s/iter data_time: 0.2200 s/iter total_throughput: 3116.51 samples/s lr: 5.36e-04 [09/27 01:39:58] lb.utils.events INFO: eta: 10:10:19 iteration: 180399/375342 consumed_samples: 184729600 total_loss: 3.549 time: 0.3286 s/iter data_time: 0.2074 s/iter total_throughput: 3116.52 samples/s lr: 5.35e-04 [09/27 01:40:30] lb.utils.events INFO: eta: 9:17:13 iteration: 180499/375342 consumed_samples: 184832000 total_loss: 3.563 time: 0.3286 s/iter data_time: 0.2039 s/iter total_throughput: 3116.53 samples/s lr: 5.35e-04 [09/27 01:41:03] lb.utils.events INFO: eta: 8:52:40 iteration: 180599/375342 consumed_samples: 184934400 total_loss: 3.576 time: 0.3286 s/iter data_time: 0.2190 s/iter total_throughput: 3116.56 samples/s lr: 5.34e-04 [09/27 01:41:35] lb.utils.events INFO: eta: 8:46:38 iteration: 180699/375342 consumed_samples: 185036800 total_loss: 3.574 time: 0.3286 s/iter data_time: 0.2203 s/iter total_throughput: 3116.58 samples/s lr: 5.34e-04 [09/27 01:42:08] lb.utils.events INFO: eta: 8:44:54 iteration: 180799/375342 consumed_samples: 185139200 total_loss: 3.582 time: 0.3286 s/iter data_time: 0.2263 s/iter total_throughput: 3116.58 samples/s lr: 5.33e-04 [09/27 01:42:41] lb.utils.events INFO: eta: 8:41:14 iteration: 180899/375342 consumed_samples: 185241600 total_loss: 3.567 time: 0.3286 s/iter data_time: 0.2326 s/iter total_throughput: 3116.58 samples/s lr: 5.33e-04 [09/27 01:43:14] lb.utils.events INFO: eta: 8:42:41 iteration: 180999/375342 consumed_samples: 185344000 total_loss: 3.567 time: 0.3286 s/iter data_time: 0.2297 s/iter total_throughput: 3116.59 samples/s lr: 5.33e-04 [09/27 01:43:46] lb.utils.events INFO: eta: 8:47:35 iteration: 181099/375342 consumed_samples: 185446400 total_loss: 3.575 time: 0.3286 s/iter data_time: 0.2295 s/iter total_throughput: 3116.60 samples/s lr: 5.32e-04 [09/27 01:44:19] lb.utils.events INFO: eta: 8:40:59 iteration: 181199/375342 consumed_samples: 185548800 total_loss: 3.59 time: 0.3286 s/iter data_time: 0.2219 s/iter total_throughput: 3116.62 samples/s lr: 5.32e-04 [09/27 01:44:51] lb.utils.events INFO: eta: 8:36:59 iteration: 181299/375342 consumed_samples: 185651200 total_loss: 3.583 time: 0.3286 s/iter data_time: 0.2064 s/iter total_throughput: 3116.63 samples/s lr: 5.31e-04 [09/27 01:45:24] lb.utils.events INFO: eta: 8:42:21 iteration: 181399/375342 consumed_samples: 185753600 total_loss: 3.58 time: 0.3286 s/iter data_time: 0.2174 s/iter total_throughput: 3116.64 samples/s lr: 5.31e-04 [09/27 01:45:56] lb.utils.events INFO: eta: 8:42:34 iteration: 181499/375342 consumed_samples: 185856000 total_loss: 3.566 time: 0.3286 s/iter data_time: 0.2090 s/iter total_throughput: 3116.68 samples/s lr: 5.31e-04 [09/27 01:46:29] lb.utils.events INFO: eta: 8:47:00 iteration: 181599/375342 consumed_samples: 185958400 total_loss: 3.564 time: 0.3286 s/iter data_time: 0.1985 s/iter total_throughput: 3116.69 samples/s lr: 5.30e-04 [09/27 01:47:01] lb.utils.events INFO: eta: 8:44:57 iteration: 181699/375342 consumed_samples: 186060800 total_loss: 3.584 time: 0.3286 s/iter data_time: 0.2237 s/iter total_throughput: 3116.72 samples/s lr: 5.30e-04 [09/27 01:47:34] lb.utils.events INFO: eta: 8:42:37 iteration: 181799/375342 consumed_samples: 186163200 total_loss: 3.56 time: 0.3285 s/iter data_time: 0.2307 s/iter total_throughput: 3116.74 samples/s lr: 5.29e-04 [09/27 01:48:06] lb.utils.events INFO: eta: 8:45:22 iteration: 181899/375342 consumed_samples: 186265600 total_loss: 3.565 time: 0.3285 s/iter data_time: 0.2179 s/iter total_throughput: 3116.74 samples/s lr: 5.29e-04 [09/27 01:48:38] lb.utils.events INFO: eta: 8:37:46 iteration: 181999/375342 consumed_samples: 186368000 total_loss: 3.595 time: 0.3285 s/iter data_time: 0.2120 s/iter total_throughput: 3116.79 samples/s lr: 5.28e-04 [09/27 01:49:12] lb.utils.events INFO: eta: 8:31:16 iteration: 182099/375342 consumed_samples: 186470400 total_loss: 3.585 time: 0.3285 s/iter data_time: 0.1979 s/iter total_throughput: 3116.77 samples/s lr: 5.28e-04 [09/27 01:49:44] lb.utils.events INFO: eta: 8:26:37 iteration: 182199/375342 consumed_samples: 186572800 total_loss: 3.572 time: 0.3285 s/iter data_time: 0.2069 s/iter total_throughput: 3116.78 samples/s lr: 5.28e-04 [09/27 01:50:17] lb.utils.events INFO: eta: 8:25:46 iteration: 182299/375342 consumed_samples: 186675200 total_loss: 3.563 time: 0.3285 s/iter data_time: 0.2326 s/iter total_throughput: 3116.78 samples/s lr: 5.27e-04 [09/27 01:50:50] lb.utils.events INFO: eta: 8:24:03 iteration: 182399/375342 consumed_samples: 186777600 total_loss: 3.564 time: 0.3285 s/iter data_time: 0.2207 s/iter total_throughput: 3116.78 samples/s lr: 5.27e-04 [09/27 01:51:24] lb.utils.events INFO: eta: 8:26:07 iteration: 182499/375342 consumed_samples: 186880000 total_loss: 3.577 time: 0.3285 s/iter data_time: 0.2064 s/iter total_throughput: 3116.74 samples/s lr: 5.26e-04 [09/27 01:51:57] lb.utils.events INFO: eta: 8:25:52 iteration: 182599/375342 consumed_samples: 186982400 total_loss: 3.58 time: 0.3286 s/iter data_time: 0.2059 s/iter total_throughput: 3116.71 samples/s lr: 5.26e-04 [09/27 01:52:30] lb.utils.events INFO: eta: 8:27:35 iteration: 182699/375342 consumed_samples: 187084800 total_loss: 3.572 time: 0.3286 s/iter data_time: 0.2517 s/iter total_throughput: 3116.70 samples/s lr: 5.26e-04 [09/27 01:53:04] lb.utils.events INFO: eta: 8:27:38 iteration: 182799/375342 consumed_samples: 187187200 total_loss: 3.562 time: 0.3286 s/iter data_time: 0.2347 s/iter total_throughput: 3116.67 samples/s lr: 5.25e-04 [09/27 01:53:37] lb.utils.events INFO: eta: 8:31:46 iteration: 182899/375342 consumed_samples: 187289600 total_loss: 3.563 time: 0.3286 s/iter data_time: 0.2374 s/iter total_throughput: 3116.64 samples/s lr: 5.25e-04 [09/27 01:54:10] lb.utils.events INFO: eta: 8:35:08 iteration: 182999/375342 consumed_samples: 187392000 total_loss: 3.573 time: 0.3286 s/iter data_time: 0.2298 s/iter total_throughput: 3116.63 samples/s lr: 5.24e-04 [09/27 01:54:43] lb.utils.events INFO: eta: 8:39:34 iteration: 183099/375342 consumed_samples: 187494400 total_loss: 3.571 time: 0.3286 s/iter data_time: 0.2109 s/iter total_throughput: 3116.60 samples/s lr: 5.24e-04 [09/27 01:55:17] lb.utils.events INFO: eta: 8:40:07 iteration: 183199/375342 consumed_samples: 187596800 total_loss: 3.573 time: 0.3286 s/iter data_time: 0.2034 s/iter total_throughput: 3116.59 samples/s lr: 5.24e-04 [09/27 01:55:50] lb.utils.events INFO: eta: 8:37:10 iteration: 183299/375342 consumed_samples: 187699200 total_loss: 3.571 time: 0.3286 s/iter data_time: 0.2122 s/iter total_throughput: 3116.56 samples/s lr: 5.23e-04 [09/27 01:56:23] lb.utils.events INFO: eta: 8:33:14 iteration: 183399/375342 consumed_samples: 187801600 total_loss: 3.564 time: 0.3286 s/iter data_time: 0.2131 s/iter total_throughput: 3116.54 samples/s lr: 5.23e-04 [09/27 01:56:57] lb.utils.events INFO: eta: 8:28:28 iteration: 183499/375342 consumed_samples: 187904000 total_loss: 3.564 time: 0.3286 s/iter data_time: 0.2191 s/iter total_throughput: 3116.51 samples/s lr: 5.22e-04 [09/27 01:57:30] lb.utils.events INFO: eta: 8:25:03 iteration: 183599/375342 consumed_samples: 188006400 total_loss: 3.549 time: 0.3286 s/iter data_time: 0.2140 s/iter total_throughput: 3116.49 samples/s lr: 5.22e-04 [09/27 01:58:03] lb.utils.events INFO: eta: 8:20:25 iteration: 183699/375342 consumed_samples: 188108800 total_loss: 3.555 time: 0.3286 s/iter data_time: 0.1974 s/iter total_throughput: 3116.47 samples/s lr: 5.21e-04 [09/27 01:58:36] lb.utils.events INFO: eta: 8:18:43 iteration: 183799/375342 consumed_samples: 188211200 total_loss: 3.573 time: 0.3286 s/iter data_time: 0.2046 s/iter total_throughput: 3116.48 samples/s lr: 5.21e-04 [09/27 01:59:09] lb.utils.events INFO: eta: 8:14:15 iteration: 183899/375342 consumed_samples: 188313600 total_loss: 3.577 time: 0.3286 s/iter data_time: 0.2067 s/iter total_throughput: 3116.46 samples/s lr: 5.21e-04 [09/27 01:59:42] lb.utils.events INFO: eta: 8:11:31 iteration: 183999/375342 consumed_samples: 188416000 total_loss: 3.567 time: 0.3286 s/iter data_time: 0.2098 s/iter total_throughput: 3116.45 samples/s lr: 5.20e-04 [09/27 02:00:16] lb.utils.events INFO: eta: 8:10:20 iteration: 184099/375342 consumed_samples: 188518400 total_loss: 3.539 time: 0.3286 s/iter data_time: 0.2153 s/iter total_throughput: 3116.42 samples/s lr: 5.20e-04 [09/27 02:00:49] lb.utils.events INFO: eta: 8:09:47 iteration: 184199/375342 consumed_samples: 188620800 total_loss: 3.547 time: 0.3286 s/iter data_time: 0.2146 s/iter total_throughput: 3116.39 samples/s lr: 5.19e-04 [09/27 02:01:22] lb.utils.events INFO: eta: 8:10:34 iteration: 184299/375342 consumed_samples: 188723200 total_loss: 3.571 time: 0.3286 s/iter data_time: 0.2072 s/iter total_throughput: 3116.38 samples/s lr: 5.19e-04 [09/27 02:01:55] lb.utils.events INFO: eta: 8:09:35 iteration: 184399/375342 consumed_samples: 188825600 total_loss: 3.572 time: 0.3286 s/iter data_time: 0.2059 s/iter total_throughput: 3116.37 samples/s lr: 5.19e-04 [09/27 02:02:28] lb.utils.events INFO: eta: 8:09:02 iteration: 184499/375342 consumed_samples: 188928000 total_loss: 3.561 time: 0.3286 s/iter data_time: 0.2195 s/iter total_throughput: 3116.36 samples/s lr: 5.18e-04 [09/27 02:03:02] lb.utils.events INFO: eta: 8:07:24 iteration: 184599/375342 consumed_samples: 189030400 total_loss: 3.568 time: 0.3286 s/iter data_time: 0.2042 s/iter total_throughput: 3116.34 samples/s lr: 5.18e-04 [09/27 02:03:35] lb.utils.events INFO: eta: 8:06:53 iteration: 184699/375342 consumed_samples: 189132800 total_loss: 3.571 time: 0.3286 s/iter data_time: 0.2123 s/iter total_throughput: 3116.32 samples/s lr: 5.17e-04 [09/27 02:04:08] lb.utils.events INFO: eta: 8:05:43 iteration: 184799/375342 consumed_samples: 189235200 total_loss: 3.567 time: 0.3286 s/iter data_time: 0.2120 s/iter total_throughput: 3116.32 samples/s lr: 5.17e-04 [09/27 02:04:41] lb.utils.events INFO: eta: 8:06:43 iteration: 184899/375342 consumed_samples: 189337600 total_loss: 3.545 time: 0.3286 s/iter data_time: 0.2023 s/iter total_throughput: 3116.32 samples/s lr: 5.16e-04 [09/27 02:05:14] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0184999 [09/27 02:05:14] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/27 02:05:14] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/27 02:05:18] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0897 s/iter. Inference: 0.1532 s/iter. Eval: 0.0019 s/iter. Total: 0.2447 s/iter. ETA=0:00:09 [09/27 02:05:24] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1505 s/iter. Inference: 0.1516 s/iter. Eval: 0.0020 s/iter. Total: 0.3042 s/iter. ETA=0:00:05 [09/27 02:05:30] lb.evaluation.evaluator INFO: Inference done 50000/50000. Dataloading: 0.1365 s/iter. Inference: 0.1508 s/iter. Eval: 0.0020 s/iter. Total: 0.2894 s/iter. ETA=0:00:00 [09/27 02:05:30] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/27 02:05:30] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.740310 (0.000255 s / iter per device, on 8 devices) [09/27 02:05:30] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000133 s / iter per device, on 8 devices) [09/27 02:05:30] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/27 02:05:30] lb.evaluation.utils INFO: copypaste: Acc@1=75.002 [09/27 02:05:30] lb.evaluation.utils INFO: copypaste: Acc@5=92.47 [09/27 02:05:30] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 75.00200, better than last best score 74.73800 @ iteration 179999. [09/27 02:05:30] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/27 02:05:31] lb.utils.events INFO: eta: 8:06:28 iteration: 184999/375342 consumed_samples: 189440000 total_loss: 3.546 time: 0.3286 s/iter data_time: 0.2085 s/iter total_throughput: 3116.31 samples/s lr: 5.16e-04 [09/27 02:06:02] lb.utils.events INFO: eta: 8:06:01 iteration: 185099/375342 consumed_samples: 189542400 total_loss: 3.565 time: 0.3286 s/iter data_time: 0.2255 s/iter total_throughput: 3116.39 samples/s lr: 5.16e-04 [09/27 02:06:35] lb.utils.events INFO: eta: 8:08:09 iteration: 185199/375342 consumed_samples: 189644800 total_loss: 3.548 time: 0.3286 s/iter data_time: 0.2185 s/iter total_throughput: 3116.38 samples/s lr: 5.15e-04 [09/27 02:07:08] lb.utils.events INFO: eta: 8:09:43 iteration: 185299/375342 consumed_samples: 189747200 total_loss: 3.548 time: 0.3286 s/iter data_time: 0.2279 s/iter total_throughput: 3116.36 samples/s lr: 5.15e-04 [09/27 02:07:42] lb.utils.events INFO: eta: 8:10:29 iteration: 185399/375342 consumed_samples: 189849600 total_loss: 3.558 time: 0.3286 s/iter data_time: 0.2136 s/iter total_throughput: 3116.34 samples/s lr: 5.14e-04 [09/27 02:08:14] lb.utils.events INFO: eta: 8:12:52 iteration: 185499/375342 consumed_samples: 189952000 total_loss: 3.547 time: 0.3286 s/iter data_time: 0.2068 s/iter total_throughput: 3116.33 samples/s lr: 5.14e-04 [09/27 02:08:47] lb.utils.events INFO: eta: 8:15:08 iteration: 185599/375342 consumed_samples: 190054400 total_loss: 3.549 time: 0.3286 s/iter data_time: 0.2273 s/iter total_throughput: 3116.33 samples/s lr: 5.14e-04 [09/27 02:09:21] lb.utils.events INFO: eta: 8:17:11 iteration: 185699/375342 consumed_samples: 190156800 total_loss: 3.562 time: 0.3286 s/iter data_time: 0.2056 s/iter total_throughput: 3116.30 samples/s lr: 5.13e-04 [09/27 02:09:54] lb.utils.events INFO: eta: 8:17:18 iteration: 185799/375342 consumed_samples: 190259200 total_loss: 3.558 time: 0.3286 s/iter data_time: 0.2040 s/iter total_throughput: 3116.28 samples/s lr: 5.13e-04 [09/27 02:10:27] lb.utils.events INFO: eta: 8:17:17 iteration: 185899/375342 consumed_samples: 190361600 total_loss: 3.546 time: 0.3286 s/iter data_time: 0.2090 s/iter total_throughput: 3116.25 samples/s lr: 5.12e-04 [09/27 02:11:01] lb.utils.events INFO: eta: 8:16:32 iteration: 185999/375342 consumed_samples: 190464000 total_loss: 3.563 time: 0.3286 s/iter data_time: 0.2093 s/iter total_throughput: 3116.24 samples/s lr: 5.12e-04 [09/27 02:11:33] lb.utils.events INFO: eta: 8:17:00 iteration: 186099/375342 consumed_samples: 190566400 total_loss: 3.562 time: 0.3286 s/iter data_time: 0.2066 s/iter total_throughput: 3116.24 samples/s lr: 5.12e-04 [09/27 02:12:06] lb.utils.events INFO: eta: 8:14:06 iteration: 186199/375342 consumed_samples: 190668800 total_loss: 3.558 time: 0.3286 s/iter data_time: 0.2023 s/iter total_throughput: 3116.24 samples/s lr: 5.11e-04 [09/27 02:12:39] lb.utils.events INFO: eta: 8:10:21 iteration: 186299/375342 consumed_samples: 190771200 total_loss: 3.554 time: 0.3286 s/iter data_time: 0.2116 s/iter total_throughput: 3116.25 samples/s lr: 5.11e-04 [09/27 02:13:12] lb.utils.events INFO: eta: 8:11:03 iteration: 186399/375342 consumed_samples: 190873600 total_loss: 3.551 time: 0.3286 s/iter data_time: 0.2202 s/iter total_throughput: 3116.25 samples/s lr: 5.10e-04 [09/27 02:13:45] lb.utils.events INFO: eta: 8:08:31 iteration: 186499/375342 consumed_samples: 190976000 total_loss: 3.553 time: 0.3286 s/iter data_time: 0.2023 s/iter total_throughput: 3116.24 samples/s lr: 5.10e-04 [09/27 02:14:18] lb.utils.events INFO: eta: 8:07:35 iteration: 186599/375342 consumed_samples: 191078400 total_loss: 3.553 time: 0.3286 s/iter data_time: 0.2271 s/iter total_throughput: 3116.22 samples/s lr: 5.09e-04 [09/27 02:14:51] lb.utils.events INFO: eta: 8:06:25 iteration: 186699/375342 consumed_samples: 191180800 total_loss: 3.55 time: 0.3286 s/iter data_time: 0.2271 s/iter total_throughput: 3116.21 samples/s lr: 5.09e-04 [09/27 02:15:24] lb.utils.events INFO: eta: 8:08:55 iteration: 186799/375342 consumed_samples: 191283200 total_loss: 3.547 time: 0.3286 s/iter data_time: 0.2064 s/iter total_throughput: 3116.20 samples/s lr: 5.09e-04 [09/27 02:15:57] lb.utils.events INFO: eta: 8:08:39 iteration: 186899/375342 consumed_samples: 191385600 total_loss: 3.551 time: 0.3286 s/iter data_time: 0.2205 s/iter total_throughput: 3116.19 samples/s lr: 5.08e-04 [09/27 02:16:30] lb.utils.events INFO: eta: 8:08:15 iteration: 186999/375342 consumed_samples: 191488000 total_loss: 3.552 time: 0.3286 s/iter data_time: 0.2038 s/iter total_throughput: 3116.19 samples/s lr: 5.08e-04 [09/27 02:17:03] lb.utils.events INFO: eta: 8:07:28 iteration: 187099/375342 consumed_samples: 191590400 total_loss: 3.559 time: 0.3286 s/iter data_time: 0.2129 s/iter total_throughput: 3116.19 samples/s lr: 5.07e-04 [09/27 02:17:37] lb.utils.events INFO: eta: 8:06:51 iteration: 187199/375342 consumed_samples: 191692800 total_loss: 3.559 time: 0.3286 s/iter data_time: 0.2068 s/iter total_throughput: 3116.16 samples/s lr: 5.07e-04 [09/27 02:18:10] lb.utils.events INFO: eta: 8:06:57 iteration: 187299/375342 consumed_samples: 191795200 total_loss: 3.534 time: 0.3286 s/iter data_time: 0.2039 s/iter total_throughput: 3116.14 samples/s lr: 5.07e-04 [09/27 02:18:43] lb.utils.events INFO: eta: 8:04:31 iteration: 187399/375342 consumed_samples: 191897600 total_loss: 3.543 time: 0.3286 s/iter data_time: 0.2064 s/iter total_throughput: 3116.14 samples/s lr: 5.06e-04 [09/27 02:19:16] lb.utils.events INFO: eta: 8:03:36 iteration: 187499/375342 consumed_samples: 192000000 total_loss: 3.553 time: 0.3286 s/iter data_time: 0.2162 s/iter total_throughput: 3116.12 samples/s lr: 5.06e-04 [09/27 02:19:49] lb.utils.events INFO: eta: 8:04:25 iteration: 187599/375342 consumed_samples: 192102400 total_loss: 3.541 time: 0.3286 s/iter data_time: 0.2102 s/iter total_throughput: 3116.11 samples/s lr: 5.05e-04 [09/27 02:20:22] lb.utils.events INFO: eta: 8:02:34 iteration: 187699/375342 consumed_samples: 192204800 total_loss: 3.545 time: 0.3286 s/iter data_time: 0.2047 s/iter total_throughput: 3116.11 samples/s lr: 5.05e-04 [09/27 02:20:55] lb.utils.events INFO: eta: 8:01:01 iteration: 187799/375342 consumed_samples: 192307200 total_loss: 3.545 time: 0.3286 s/iter data_time: 0.2086 s/iter total_throughput: 3116.09 samples/s lr: 5.04e-04 [09/27 02:21:29] lb.utils.events INFO: eta: 7:59:49 iteration: 187899/375342 consumed_samples: 192409600 total_loss: 3.54 time: 0.3286 s/iter data_time: 0.2094 s/iter total_throughput: 3116.06 samples/s lr: 5.04e-04 [09/27 02:22:02] lb.utils.events INFO: eta: 7:59:44 iteration: 187999/375342 consumed_samples: 192512000 total_loss: 3.54 time: 0.3286 s/iter data_time: 0.2316 s/iter total_throughput: 3116.03 samples/s lr: 5.04e-04 [09/27 02:22:35] lb.utils.events INFO: eta: 7:58:40 iteration: 188099/375342 consumed_samples: 192614400 total_loss: 3.546 time: 0.3286 s/iter data_time: 0.2085 s/iter total_throughput: 3116.04 samples/s lr: 5.03e-04 [09/27 02:23:08] lb.utils.events INFO: eta: 7:59:19 iteration: 188199/375342 consumed_samples: 192716800 total_loss: 3.546 time: 0.3286 s/iter data_time: 0.2080 s/iter total_throughput: 3116.02 samples/s lr: 5.03e-04 [09/27 02:23:41] lb.utils.events INFO: eta: 7:59:21 iteration: 188299/375342 consumed_samples: 192819200 total_loss: 3.549 time: 0.3286 s/iter data_time: 0.2087 s/iter total_throughput: 3116.00 samples/s lr: 5.02e-04 [09/27 02:24:15] lb.utils.events INFO: eta: 8:00:07 iteration: 188399/375342 consumed_samples: 192921600 total_loss: 3.557 time: 0.3286 s/iter data_time: 0.2107 s/iter total_throughput: 3115.97 samples/s lr: 5.02e-04 [09/27 02:24:47] lb.utils.events INFO: eta: 8:01:31 iteration: 188499/375342 consumed_samples: 193024000 total_loss: 3.555 time: 0.3286 s/iter data_time: 0.2073 s/iter total_throughput: 3115.98 samples/s lr: 5.02e-04 [09/27 02:25:20] lb.utils.events INFO: eta: 7:59:20 iteration: 188599/375342 consumed_samples: 193126400 total_loss: 3.557 time: 0.3286 s/iter data_time: 0.2110 s/iter total_throughput: 3115.98 samples/s lr: 5.01e-04 [09/27 02:25:53] lb.utils.events INFO: eta: 8:02:26 iteration: 188699/375342 consumed_samples: 193228800 total_loss: 3.549 time: 0.3286 s/iter data_time: 0.2295 s/iter total_throughput: 3115.97 samples/s lr: 5.01e-04 [09/27 02:26:26] lb.utils.events INFO: eta: 8:05:52 iteration: 188799/375342 consumed_samples: 193331200 total_loss: 3.55 time: 0.3286 s/iter data_time: 0.2191 s/iter total_throughput: 3115.97 samples/s lr: 5.00e-04 [09/27 02:26:59] lb.utils.events INFO: eta: 8:07:56 iteration: 188899/375342 consumed_samples: 193433600 total_loss: 3.554 time: 0.3286 s/iter data_time: 0.2256 s/iter total_throughput: 3115.96 samples/s lr: 5.00e-04 [09/27 02:27:32] lb.utils.events INFO: eta: 8:07:40 iteration: 188999/375342 consumed_samples: 193536000 total_loss: 3.529 time: 0.3286 s/iter data_time: 0.1986 s/iter total_throughput: 3115.96 samples/s lr: 4.99e-04 [09/27 02:28:05] lb.utils.events INFO: eta: 8:07:20 iteration: 189099/375342 consumed_samples: 193638400 total_loss: 3.508 time: 0.3286 s/iter data_time: 0.2088 s/iter total_throughput: 3115.96 samples/s lr: 4.99e-04 [09/27 02:28:38] lb.utils.events INFO: eta: 8:06:24 iteration: 189199/375342 consumed_samples: 193740800 total_loss: 3.515 time: 0.3286 s/iter data_time: 0.2141 s/iter total_throughput: 3115.95 samples/s lr: 4.99e-04 [09/27 02:29:11] lb.utils.events INFO: eta: 8:07:01 iteration: 189299/375342 consumed_samples: 193843200 total_loss: 3.522 time: 0.3286 s/iter data_time: 0.2174 s/iter total_throughput: 3115.97 samples/s lr: 4.98e-04 [09/27 02:29:44] lb.utils.events INFO: eta: 8:09:27 iteration: 189399/375342 consumed_samples: 193945600 total_loss: 3.536 time: 0.3286 s/iter data_time: 0.2464 s/iter total_throughput: 3115.95 samples/s lr: 4.98e-04 [09/27 02:30:17] lb.utils.events INFO: eta: 8:10:37 iteration: 189499/375342 consumed_samples: 194048000 total_loss: 3.535 time: 0.3286 s/iter data_time: 0.2353 s/iter total_throughput: 3115.94 samples/s lr: 4.97e-04 [09/27 02:30:50] lb.utils.events INFO: eta: 8:12:52 iteration: 189599/375342 consumed_samples: 194150400 total_loss: 3.528 time: 0.3286 s/iter data_time: 0.2271 s/iter total_throughput: 3115.93 samples/s lr: 4.97e-04 [09/27 02:31:23] lb.utils.events INFO: eta: 8:12:45 iteration: 189699/375342 consumed_samples: 194252800 total_loss: 3.529 time: 0.3286 s/iter data_time: 0.2338 s/iter total_throughput: 3115.94 samples/s lr: 4.97e-04 [09/27 02:31:56] lb.utils.events INFO: eta: 8:13:52 iteration: 189799/375342 consumed_samples: 194355200 total_loss: 3.532 time: 0.3286 s/iter data_time: 0.2449 s/iter total_throughput: 3115.94 samples/s lr: 4.96e-04 [09/27 02:32:28] lb.utils.events INFO: eta: 8:18:49 iteration: 189899/375342 consumed_samples: 194457600 total_loss: 3.532 time: 0.3286 s/iter data_time: 0.2253 s/iter total_throughput: 3115.95 samples/s lr: 4.96e-04 [09/27 02:33:02] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0189999 [09/27 02:33:02] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/27 02:33:02] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/27 02:33:06] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0821 s/iter. Inference: 0.1518 s/iter. Eval: 0.0020 s/iter. Total: 0.2359 s/iter. ETA=0:00:08 [09/27 02:33:12] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1373 s/iter. Inference: 0.1553 s/iter. Eval: 0.0020 s/iter. Total: 0.2947 s/iter. ETA=0:00:05 [09/27 02:33:17] lb.evaluation.evaluator INFO: Inference done 50000/50000. Dataloading: 0.1286 s/iter. Inference: 0.1531 s/iter. Eval: 0.0020 s/iter. Total: 0.2838 s/iter. ETA=0:00:00 [09/27 02:33:17] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/27 02:33:17] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.490782 (0.000250 s / iter per device, on 8 devices) [09/27 02:33:17] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000135 s / iter per device, on 8 devices) [09/27 02:33:17] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/27 02:33:17] lb.evaluation.utils INFO: copypaste: Acc@1=75.136 [09/27 02:33:17] lb.evaluation.utils INFO: copypaste: Acc@5=92.50399999999999 [09/27 02:33:17] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 75.13600, better than last best score 75.00200 @ iteration 184999. [09/27 02:33:17] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/27 02:33:18] lb.utils.events INFO: eta: 8:19:39 iteration: 189999/375342 consumed_samples: 194560000 total_loss: 3.537 time: 0.3286 s/iter data_time: 0.2126 s/iter total_throughput: 3115.93 samples/s lr: 4.95e-04 [09/27 02:33:49] lb.utils.events INFO: eta: 8:19:13 iteration: 190099/375342 consumed_samples: 194662400 total_loss: 3.553 time: 0.3286 s/iter data_time: 0.2053 s/iter total_throughput: 3116.02 samples/s lr: 4.95e-04 [09/27 02:34:22] lb.utils.events INFO: eta: 8:24:52 iteration: 190199/375342 consumed_samples: 194764800 total_loss: 3.548 time: 0.3286 s/iter data_time: 0.2282 s/iter total_throughput: 3116.01 samples/s lr: 4.95e-04 [09/27 02:34:55] lb.utils.events INFO: eta: 8:42:31 iteration: 190299/375342 consumed_samples: 194867200 total_loss: 3.549 time: 0.3286 s/iter data_time: 0.2219 s/iter total_throughput: 3116.00 samples/s lr: 4.94e-04 [09/27 02:35:28] lb.utils.events INFO: eta: 8:35:46 iteration: 190399/375342 consumed_samples: 194969600 total_loss: 3.546 time: 0.3286 s/iter data_time: 0.2227 s/iter total_throughput: 3116.00 samples/s lr: 4.94e-04 [09/27 02:36:01] lb.utils.events INFO: eta: 8:39:42 iteration: 190499/375342 consumed_samples: 195072000 total_loss: 3.536 time: 0.3286 s/iter data_time: 0.2165 s/iter total_throughput: 3116.01 samples/s lr: 4.93e-04 [09/27 02:36:34] lb.utils.events INFO: eta: 8:28:02 iteration: 190599/375342 consumed_samples: 195174400 total_loss: 3.547 time: 0.3286 s/iter data_time: 0.1983 s/iter total_throughput: 3116.00 samples/s lr: 4.93e-04 [09/27 02:37:07] lb.utils.events INFO: eta: 8:18:44 iteration: 190699/375342 consumed_samples: 195276800 total_loss: 3.534 time: 0.3286 s/iter data_time: 0.2070 s/iter total_throughput: 3116.00 samples/s lr: 4.92e-04 [09/27 02:37:40] lb.utils.events INFO: eta: 8:15:40 iteration: 190799/375342 consumed_samples: 195379200 total_loss: 3.521 time: 0.3286 s/iter data_time: 0.2268 s/iter total_throughput: 3116.00 samples/s lr: 4.92e-04 [09/27 02:38:12] lb.utils.events INFO: eta: 8:12:20 iteration: 190899/375342 consumed_samples: 195481600 total_loss: 3.517 time: 0.3286 s/iter data_time: 0.2434 s/iter total_throughput: 3116.01 samples/s lr: 4.92e-04 [09/27 02:38:46] lb.utils.events INFO: eta: 8:14:11 iteration: 190999/375342 consumed_samples: 195584000 total_loss: 3.54 time: 0.3286 s/iter data_time: 0.2105 s/iter total_throughput: 3115.99 samples/s lr: 4.91e-04 [09/27 02:39:18] lb.utils.events INFO: eta: 8:18:47 iteration: 191099/375342 consumed_samples: 195686400 total_loss: 3.548 time: 0.3286 s/iter data_time: 0.2225 s/iter total_throughput: 3116.01 samples/s lr: 4.91e-04 [09/27 02:39:51] lb.utils.events INFO: eta: 8:21:15 iteration: 191199/375342 consumed_samples: 195788800 total_loss: 3.548 time: 0.3286 s/iter data_time: 0.2289 s/iter total_throughput: 3116.00 samples/s lr: 4.90e-04 [09/27 02:40:24] lb.utils.events INFO: eta: 8:19:44 iteration: 191299/375342 consumed_samples: 195891200 total_loss: 3.552 time: 0.3286 s/iter data_time: 0.2295 s/iter total_throughput: 3116.00 samples/s lr: 4.90e-04 [09/27 02:40:57] lb.utils.events INFO: eta: 8:12:29 iteration: 191399/375342 consumed_samples: 195993600 total_loss: 3.557 time: 0.3286 s/iter data_time: 0.2138 s/iter total_throughput: 3115.99 samples/s lr: 4.90e-04 [09/27 02:41:30] lb.utils.events INFO: eta: 8:06:29 iteration: 191499/375342 consumed_samples: 196096000 total_loss: 3.562 time: 0.3286 s/iter data_time: 0.1985 s/iter total_throughput: 3116.00 samples/s lr: 4.89e-04 [09/27 02:42:03] lb.utils.events INFO: eta: 8:04:42 iteration: 191599/375342 consumed_samples: 196198400 total_loss: 3.553 time: 0.3286 s/iter data_time: 0.2099 s/iter total_throughput: 3116.00 samples/s lr: 4.89e-04 [09/27 02:42:35] lb.utils.events INFO: eta: 8:04:15 iteration: 191699/375342 consumed_samples: 196300800 total_loss: 3.524 time: 0.3286 s/iter data_time: 0.2122 s/iter total_throughput: 3116.00 samples/s lr: 4.88e-04 [09/27 02:43:08] lb.utils.events INFO: eta: 8:01:21 iteration: 191799/375342 consumed_samples: 196403200 total_loss: 3.522 time: 0.3286 s/iter data_time: 0.2247 s/iter total_throughput: 3116.02 samples/s lr: 4.88e-04 [09/27 02:43:41] lb.utils.events INFO: eta: 8:02:26 iteration: 191899/375342 consumed_samples: 196505600 total_loss: 3.529 time: 0.3286 s/iter data_time: 0.2241 s/iter total_throughput: 3116.02 samples/s lr: 4.87e-04 [09/27 02:44:14] lb.utils.events INFO: eta: 8:04:21 iteration: 191999/375342 consumed_samples: 196608000 total_loss: 3.528 time: 0.3286 s/iter data_time: 0.2493 s/iter total_throughput: 3116.03 samples/s lr: 4.87e-04 [09/27 02:44:46] lb.utils.events INFO: eta: 8:05:18 iteration: 192099/375342 consumed_samples: 196710400 total_loss: 3.543 time: 0.3286 s/iter data_time: 0.2256 s/iter total_throughput: 3116.03 samples/s lr: 4.87e-04 [09/27 02:45:19] lb.utils.events INFO: eta: 8:07:51 iteration: 192199/375342 consumed_samples: 196812800 total_loss: 3.552 time: 0.3286 s/iter data_time: 0.2321 s/iter total_throughput: 3116.02 samples/s lr: 4.86e-04 [09/27 02:45:53] lb.utils.events INFO: eta: 8:06:26 iteration: 192299/375342 consumed_samples: 196915200 total_loss: 3.53 time: 0.3286 s/iter data_time: 0.2271 s/iter total_throughput: 3116.00 samples/s lr: 4.86e-04 [09/27 02:46:25] lb.utils.events INFO: eta: 8:07:20 iteration: 192399/375342 consumed_samples: 197017600 total_loss: 3.523 time: 0.3286 s/iter data_time: 0.2147 s/iter total_throughput: 3116.01 samples/s lr: 4.85e-04 [09/27 02:46:58] lb.utils.events INFO: eta: 8:13:00 iteration: 192499/375342 consumed_samples: 197120000 total_loss: 3.536 time: 0.3286 s/iter data_time: 0.2228 s/iter total_throughput: 3116.01 samples/s lr: 4.85e-04 [09/27 02:47:31] lb.utils.events INFO: eta: 8:17:00 iteration: 192599/375342 consumed_samples: 197222400 total_loss: 3.543 time: 0.3286 s/iter data_time: 0.2257 s/iter total_throughput: 3116.03 samples/s lr: 4.85e-04 [09/27 02:48:04] lb.utils.events INFO: eta: 8:29:28 iteration: 192699/375342 consumed_samples: 197324800 total_loss: 3.524 time: 0.3286 s/iter data_time: 0.2333 s/iter total_throughput: 3116.02 samples/s lr: 4.84e-04 [09/27 02:48:37] lb.utils.events INFO: eta: 8:52:30 iteration: 192799/375342 consumed_samples: 197427200 total_loss: 3.508 time: 0.3286 s/iter data_time: 0.2248 s/iter total_throughput: 3116.02 samples/s lr: 4.84e-04 [09/27 02:49:09] lb.utils.events INFO: eta: 9:13:34 iteration: 192899/375342 consumed_samples: 197529600 total_loss: 3.498 time: 0.3286 s/iter data_time: 0.2202 s/iter total_throughput: 3116.03 samples/s lr: 4.83e-04 [09/27 02:49:42] lb.utils.events INFO: eta: 8:34:19 iteration: 192999/375342 consumed_samples: 197632000 total_loss: 3.524 time: 0.3286 s/iter data_time: 0.2246 s/iter total_throughput: 3116.04 samples/s lr: 4.83e-04 [09/27 02:50:15] lb.utils.events INFO: eta: 8:34:48 iteration: 193099/375342 consumed_samples: 197734400 total_loss: 3.541 time: 0.3286 s/iter data_time: 0.2261 s/iter total_throughput: 3116.04 samples/s lr: 4.83e-04 [09/27 02:50:48] lb.utils.events INFO: eta: 8:26:44 iteration: 193199/375342 consumed_samples: 197836800 total_loss: 3.531 time: 0.3286 s/iter data_time: 0.2205 s/iter total_throughput: 3116.05 samples/s lr: 4.82e-04 [09/27 02:51:21] lb.utils.events INFO: eta: 8:28:25 iteration: 193299/375342 consumed_samples: 197939200 total_loss: 3.544 time: 0.3286 s/iter data_time: 0.2344 s/iter total_throughput: 3116.05 samples/s lr: 4.82e-04 [09/27 02:51:53] lb.utils.events INFO: eta: 9:14:23 iteration: 193399/375342 consumed_samples: 198041600 total_loss: 3.558 time: 0.3286 s/iter data_time: 0.2300 s/iter total_throughput: 3116.07 samples/s lr: 4.81e-04 [09/27 02:52:26] lb.utils.events INFO: eta: 10:07:56 iteration: 193499/375342 consumed_samples: 198144000 total_loss: 3.546 time: 0.3286 s/iter data_time: 0.2356 s/iter total_throughput: 3116.08 samples/s lr: 4.81e-04 [09/27 02:52:59] lb.utils.events INFO: eta: 10:07:46 iteration: 193599/375342 consumed_samples: 198246400 total_loss: 3.53 time: 0.3286 s/iter data_time: 0.2351 s/iter total_throughput: 3116.06 samples/s lr: 4.80e-04 [09/27 02:53:32] lb.utils.events INFO: eta: 8:45:10 iteration: 193699/375342 consumed_samples: 198348800 total_loss: 3.521 time: 0.3286 s/iter data_time: 0.2291 s/iter total_throughput: 3116.07 samples/s lr: 4.80e-04 [09/27 02:54:04] lb.utils.events INFO: eta: 9:10:52 iteration: 193799/375342 consumed_samples: 198451200 total_loss: 3.541 time: 0.3286 s/iter data_time: 0.2260 s/iter total_throughput: 3116.08 samples/s lr: 4.80e-04 [09/27 02:54:37] lb.utils.events INFO: eta: 8:29:51 iteration: 193899/375342 consumed_samples: 198553600 total_loss: 3.527 time: 0.3286 s/iter data_time: 0.2060 s/iter total_throughput: 3116.09 samples/s lr: 4.79e-04 [09/27 02:55:10] lb.utils.events INFO: eta: 8:15:50 iteration: 193999/375342 consumed_samples: 198656000 total_loss: 3.514 time: 0.3286 s/iter data_time: 0.2089 s/iter total_throughput: 3116.09 samples/s lr: 4.79e-04 [09/27 02:55:43] lb.utils.events INFO: eta: 8:07:42 iteration: 194099/375342 consumed_samples: 198758400 total_loss: 3.531 time: 0.3286 s/iter data_time: 0.2120 s/iter total_throughput: 3116.08 samples/s lr: 4.78e-04 [09/27 02:56:16] lb.utils.events INFO: eta: 8:01:10 iteration: 194199/375342 consumed_samples: 198860800 total_loss: 3.527 time: 0.3286 s/iter data_time: 0.2088 s/iter total_throughput: 3116.07 samples/s lr: 4.78e-04 [09/27 02:56:49] lb.utils.events INFO: eta: 7:59:17 iteration: 194299/375342 consumed_samples: 198963200 total_loss: 3.494 time: 0.3286 s/iter data_time: 0.2270 s/iter total_throughput: 3116.08 samples/s lr: 4.78e-04 [09/27 02:57:21] lb.utils.events INFO: eta: 7:53:25 iteration: 194399/375342 consumed_samples: 199065600 total_loss: 3.512 time: 0.3286 s/iter data_time: 0.2043 s/iter total_throughput: 3116.09 samples/s lr: 4.77e-04 [09/27 02:57:53] lb.utils.events INFO: eta: 7:49:59 iteration: 194499/375342 consumed_samples: 199168000 total_loss: 3.517 time: 0.3286 s/iter data_time: 0.2037 s/iter total_throughput: 3116.12 samples/s lr: 4.77e-04 [09/27 02:58:26] lb.utils.events INFO: eta: 7:49:13 iteration: 194599/375342 consumed_samples: 199270400 total_loss: 3.515 time: 0.3286 s/iter data_time: 0.2299 s/iter total_throughput: 3116.11 samples/s lr: 4.76e-04 [09/27 02:58:59] lb.utils.events INFO: eta: 7:46:36 iteration: 194699/375342 consumed_samples: 199372800 total_loss: 3.526 time: 0.3286 s/iter data_time: 0.2184 s/iter total_throughput: 3116.12 samples/s lr: 4.76e-04 [09/27 02:59:32] lb.utils.events INFO: eta: 7:45:18 iteration: 194799/375342 consumed_samples: 199475200 total_loss: 3.539 time: 0.3286 s/iter data_time: 0.2057 s/iter total_throughput: 3116.13 samples/s lr: 4.75e-04 [09/27 03:00:04] lb.utils.events INFO: eta: 7:43:07 iteration: 194899/375342 consumed_samples: 199577600 total_loss: 3.549 time: 0.3286 s/iter data_time: 0.2214 s/iter total_throughput: 3116.16 samples/s lr: 4.75e-04 [09/27 03:00:37] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0194999 [09/27 03:00:37] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/27 03:00:37] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/27 03:00:41] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0811 s/iter. Inference: 0.1502 s/iter. Eval: 0.0020 s/iter. Total: 0.2333 s/iter. ETA=0:00:08 [09/27 03:00:47] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1411 s/iter. Inference: 0.1509 s/iter. Eval: 0.0020 s/iter. Total: 0.2942 s/iter. ETA=0:00:05 [09/27 03:00:52] lb.evaluation.evaluator INFO: Inference done 49152/50000. Dataloading: 0.1297 s/iter. Inference: 0.1491 s/iter. Eval: 0.0020 s/iter. Total: 0.2809 s/iter. ETA=0:00:00 [09/27 03:00:52] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/27 03:00:52] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.516977 (0.000250 s / iter per device, on 8 devices) [09/27 03:00:52] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000132 s / iter per device, on 8 devices) [09/27 03:00:52] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/27 03:00:52] lb.evaluation.utils INFO: copypaste: Acc@1=75.21600000000001 [09/27 03:00:52] lb.evaluation.utils INFO: copypaste: Acc@5=92.66799999999999 [09/27 03:00:52] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 75.21600, better than last best score 75.13600 @ iteration 189999. [09/27 03:00:52] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/27 03:00:53] lb.utils.events INFO: eta: 7:45:23 iteration: 194999/375342 consumed_samples: 199680000 total_loss: 3.525 time: 0.3286 s/iter data_time: 0.2090 s/iter total_throughput: 3116.18 samples/s lr: 4.75e-04 [09/27 03:01:24] lb.utils.events INFO: eta: 7:48:55 iteration: 195099/375342 consumed_samples: 199782400 total_loss: 3.522 time: 0.3286 s/iter data_time: 0.2354 s/iter total_throughput: 3116.28 samples/s lr: 4.74e-04 [09/27 03:01:57] lb.utils.events INFO: eta: 7:51:31 iteration: 195199/375342 consumed_samples: 199884800 total_loss: 3.521 time: 0.3286 s/iter data_time: 0.2243 s/iter total_throughput: 3116.29 samples/s lr: 4.74e-04 [09/27 03:02:29] lb.utils.events INFO: eta: 7:52:36 iteration: 195299/375342 consumed_samples: 199987200 total_loss: 3.514 time: 0.3286 s/iter data_time: 0.2172 s/iter total_throughput: 3116.33 samples/s lr: 4.73e-04 [09/27 03:03:01] lb.utils.events INFO: eta: 7:53:57 iteration: 195399/375342 consumed_samples: 200089600 total_loss: 3.518 time: 0.3286 s/iter data_time: 0.1970 s/iter total_throughput: 3116.33 samples/s lr: 4.73e-04 [09/27 03:03:34] lb.utils.events INFO: eta: 7:57:00 iteration: 195499/375342 consumed_samples: 200192000 total_loss: 3.54 time: 0.3286 s/iter data_time: 0.2182 s/iter total_throughput: 3116.36 samples/s lr: 4.73e-04 [09/27 03:04:06] lb.utils.events INFO: eta: 8:00:40 iteration: 195599/375342 consumed_samples: 200294400 total_loss: 3.545 time: 0.3286 s/iter data_time: 0.2206 s/iter total_throughput: 3116.38 samples/s lr: 4.72e-04 [09/27 03:04:39] lb.utils.events INFO: eta: 8:03:29 iteration: 195699/375342 consumed_samples: 200396800 total_loss: 3.52 time: 0.3286 s/iter data_time: 0.2085 s/iter total_throughput: 3116.40 samples/s lr: 4.72e-04 [09/27 03:05:11] lb.utils.events INFO: eta: 8:03:32 iteration: 195799/375342 consumed_samples: 200499200 total_loss: 3.506 time: 0.3286 s/iter data_time: 0.2538 s/iter total_throughput: 3116.43 samples/s lr: 4.71e-04 [09/27 03:05:43] lb.utils.events INFO: eta: 8:12:28 iteration: 195899/375342 consumed_samples: 200601600 total_loss: 3.479 time: 0.3286 s/iter data_time: 0.2044 s/iter total_throughput: 3116.46 samples/s lr: 4.71e-04 [09/27 03:06:16] lb.utils.events INFO: eta: 8:12:12 iteration: 195999/375342 consumed_samples: 200704000 total_loss: 3.484 time: 0.3286 s/iter data_time: 0.2118 s/iter total_throughput: 3116.45 samples/s lr: 4.71e-04 [09/27 03:06:49] lb.utils.events INFO: eta: 8:06:21 iteration: 196099/375342 consumed_samples: 200806400 total_loss: 3.509 time: 0.3286 s/iter data_time: 0.2239 s/iter total_throughput: 3116.45 samples/s lr: 4.70e-04 [09/27 03:07:22] lb.utils.events INFO: eta: 8:04:09 iteration: 196199/375342 consumed_samples: 200908800 total_loss: 3.538 time: 0.3286 s/iter data_time: 0.2047 s/iter total_throughput: 3116.44 samples/s lr: 4.70e-04 [09/27 03:07:55] lb.utils.events INFO: eta: 7:56:36 iteration: 196299/375342 consumed_samples: 201011200 total_loss: 3.544 time: 0.3286 s/iter data_time: 0.1994 s/iter total_throughput: 3116.45 samples/s lr: 4.69e-04 [09/27 03:08:28] lb.utils.events INFO: eta: 7:54:59 iteration: 196399/375342 consumed_samples: 201113600 total_loss: 3.529 time: 0.3286 s/iter data_time: 0.2115 s/iter total_throughput: 3116.44 samples/s lr: 4.69e-04 [09/27 03:09:01] lb.utils.events INFO: eta: 7:51:19 iteration: 196499/375342 consumed_samples: 201216000 total_loss: 3.512 time: 0.3286 s/iter data_time: 0.2148 s/iter total_throughput: 3116.42 samples/s lr: 4.68e-04 [09/27 03:09:34] lb.utils.events INFO: eta: 7:49:16 iteration: 196599/375342 consumed_samples: 201318400 total_loss: 3.505 time: 0.3286 s/iter data_time: 0.2136 s/iter total_throughput: 3116.40 samples/s lr: 4.68e-04 [09/27 03:10:08] lb.utils.events INFO: eta: 7:45:59 iteration: 196699/375342 consumed_samples: 201420800 total_loss: 3.521 time: 0.3286 s/iter data_time: 0.2097 s/iter total_throughput: 3116.39 samples/s lr: 4.68e-04 [09/27 03:10:41] lb.utils.events INFO: eta: 7:43:56 iteration: 196799/375342 consumed_samples: 201523200 total_loss: 3.521 time: 0.3286 s/iter data_time: 0.2151 s/iter total_throughput: 3116.37 samples/s lr: 4.67e-04 [09/27 03:11:14] lb.utils.events INFO: eta: 7:40:24 iteration: 196899/375342 consumed_samples: 201625600 total_loss: 3.503 time: 0.3286 s/iter data_time: 0.2135 s/iter total_throughput: 3116.35 samples/s lr: 4.67e-04 [09/27 03:11:46] lb.utils.events INFO: eta: 7:40:24 iteration: 196999/375342 consumed_samples: 201728000 total_loss: 3.514 time: 0.3286 s/iter data_time: 0.2181 s/iter total_throughput: 3116.38 samples/s lr: 4.66e-04 [09/27 03:12:20] lb.utils.events INFO: eta: 7:40:54 iteration: 197099/375342 consumed_samples: 201830400 total_loss: 3.535 time: 0.3286 s/iter data_time: 0.2381 s/iter total_throughput: 3116.35 samples/s lr: 4.66e-04 [09/27 03:12:53] lb.utils.events INFO: eta: 7:40:39 iteration: 197199/375342 consumed_samples: 201932800 total_loss: 3.537 time: 0.3286 s/iter data_time: 0.2232 s/iter total_throughput: 3116.34 samples/s lr: 4.66e-04 [09/27 03:13:26] lb.utils.events INFO: eta: 7:42:52 iteration: 197299/375342 consumed_samples: 202035200 total_loss: 3.51 time: 0.3286 s/iter data_time: 0.2212 s/iter total_throughput: 3116.35 samples/s lr: 4.65e-04 [09/27 03:13:59] lb.utils.events INFO: eta: 7:44:39 iteration: 197399/375342 consumed_samples: 202137600 total_loss: 3.519 time: 0.3286 s/iter data_time: 0.2183 s/iter total_throughput: 3116.35 samples/s lr: 4.65e-04 [09/27 03:14:32] lb.utils.events INFO: eta: 7:49:19 iteration: 197499/375342 consumed_samples: 202240000 total_loss: 3.536 time: 0.3286 s/iter data_time: 0.2234 s/iter total_throughput: 3116.34 samples/s lr: 4.64e-04 [09/27 03:15:05] lb.utils.events INFO: eta: 7:54:21 iteration: 197599/375342 consumed_samples: 202342400 total_loss: 3.522 time: 0.3286 s/iter data_time: 0.2336 s/iter total_throughput: 3116.32 samples/s lr: 4.64e-04 [09/27 03:15:38] lb.utils.events INFO: eta: 7:55:20 iteration: 197699/375342 consumed_samples: 202444800 total_loss: 3.497 time: 0.3286 s/iter data_time: 0.1987 s/iter total_throughput: 3116.31 samples/s lr: 4.64e-04 [09/27 03:16:11] lb.utils.events INFO: eta: 7:56:49 iteration: 197799/375342 consumed_samples: 202547200 total_loss: 3.487 time: 0.3286 s/iter data_time: 0.2078 s/iter total_throughput: 3116.30 samples/s lr: 4.63e-04 [09/27 03:16:44] lb.utils.events INFO: eta: 7:57:15 iteration: 197899/375342 consumed_samples: 202649600 total_loss: 3.498 time: 0.3286 s/iter data_time: 0.2103 s/iter total_throughput: 3116.28 samples/s lr: 4.63e-04 [09/27 03:17:17] lb.utils.events INFO: eta: 7:54:41 iteration: 197999/375342 consumed_samples: 202752000 total_loss: 3.497 time: 0.3286 s/iter data_time: 0.2098 s/iter total_throughput: 3116.28 samples/s lr: 4.62e-04 [09/27 03:17:50] lb.utils.events INFO: eta: 7:50:13 iteration: 198099/375342 consumed_samples: 202854400 total_loss: 3.503 time: 0.3286 s/iter data_time: 0.2124 s/iter total_throughput: 3116.26 samples/s lr: 4.62e-04 [09/27 03:18:23] lb.utils.events INFO: eta: 7:46:35 iteration: 198199/375342 consumed_samples: 202956800 total_loss: 3.518 time: 0.3286 s/iter data_time: 0.2013 s/iter total_throughput: 3116.26 samples/s lr: 4.61e-04 [09/27 03:18:56] lb.utils.events INFO: eta: 7:45:39 iteration: 198299/375342 consumed_samples: 203059200 total_loss: 3.505 time: 0.3286 s/iter data_time: 0.2241 s/iter total_throughput: 3116.27 samples/s lr: 4.61e-04 [09/27 03:19:29] lb.utils.events INFO: eta: 7:45:28 iteration: 198399/375342 consumed_samples: 203161600 total_loss: 3.504 time: 0.3286 s/iter data_time: 0.2219 s/iter total_throughput: 3116.24 samples/s lr: 4.61e-04 [09/27 03:20:02] lb.utils.events INFO: eta: 7:44:02 iteration: 198499/375342 consumed_samples: 203264000 total_loss: 3.516 time: 0.3286 s/iter data_time: 0.2283 s/iter total_throughput: 3116.24 samples/s lr: 4.60e-04 [09/27 03:20:35] lb.utils.events INFO: eta: 7:43:05 iteration: 198599/375342 consumed_samples: 203366400 total_loss: 3.519 time: 0.3286 s/iter data_time: 0.2236 s/iter total_throughput: 3116.25 samples/s lr: 4.60e-04 [09/27 03:21:08] lb.utils.events INFO: eta: 7:42:47 iteration: 198699/375342 consumed_samples: 203468800 total_loss: 3.495 time: 0.3286 s/iter data_time: 0.2081 s/iter total_throughput: 3116.23 samples/s lr: 4.59e-04 [09/27 03:21:41] lb.utils.events INFO: eta: 7:43:19 iteration: 198799/375342 consumed_samples: 203571200 total_loss: 3.531 time: 0.3286 s/iter data_time: 0.2493 s/iter total_throughput: 3116.23 samples/s lr: 4.59e-04 [09/27 03:22:14] lb.utils.events INFO: eta: 7:47:30 iteration: 198899/375342 consumed_samples: 203673600 total_loss: 3.543 time: 0.3286 s/iter data_time: 0.2178 s/iter total_throughput: 3116.22 samples/s lr: 4.59e-04 [09/27 03:22:47] lb.utils.events INFO: eta: 7:46:30 iteration: 198999/375342 consumed_samples: 203776000 total_loss: 3.517 time: 0.3286 s/iter data_time: 0.1995 s/iter total_throughput: 3116.21 samples/s lr: 4.58e-04 [09/27 03:23:20] lb.utils.events INFO: eta: 7:48:13 iteration: 199099/375342 consumed_samples: 203878400 total_loss: 3.49 time: 0.3286 s/iter data_time: 0.2275 s/iter total_throughput: 3116.21 samples/s lr: 4.58e-04 [09/27 03:23:53] lb.utils.events INFO: eta: 7:51:54 iteration: 199199/375342 consumed_samples: 203980800 total_loss: 3.485 time: 0.3286 s/iter data_time: 0.2204 s/iter total_throughput: 3116.20 samples/s lr: 4.57e-04 [09/27 03:24:26] lb.utils.events INFO: eta: 7:53:39 iteration: 199299/375342 consumed_samples: 204083200 total_loss: 3.497 time: 0.3286 s/iter data_time: 0.2350 s/iter total_throughput: 3116.20 samples/s lr: 4.57e-04 [09/27 03:24:59] lb.utils.events INFO: eta: 7:52:39 iteration: 199399/375342 consumed_samples: 204185600 total_loss: 3.508 time: 0.3286 s/iter data_time: 0.2264 s/iter total_throughput: 3116.20 samples/s lr: 4.56e-04 [09/27 03:25:32] lb.utils.events INFO: eta: 7:52:36 iteration: 199499/375342 consumed_samples: 204288000 total_loss: 3.519 time: 0.3286 s/iter data_time: 0.2293 s/iter total_throughput: 3116.18 samples/s lr: 4.56e-04 [09/27 03:26:05] lb.utils.events INFO: eta: 7:49:27 iteration: 199599/375342 consumed_samples: 204390400 total_loss: 3.509 time: 0.3286 s/iter data_time: 0.2171 s/iter total_throughput: 3116.19 samples/s lr: 4.56e-04 [09/27 03:26:38] lb.utils.events INFO: eta: 7:52:02 iteration: 199699/375342 consumed_samples: 204492800 total_loss: 3.482 time: 0.3286 s/iter data_time: 0.2264 s/iter total_throughput: 3116.18 samples/s lr: 4.55e-04 [09/27 03:27:11] lb.utils.events INFO: eta: 7:47:53 iteration: 199799/375342 consumed_samples: 204595200 total_loss: 3.5 time: 0.3286 s/iter data_time: 0.2148 s/iter total_throughput: 3116.18 samples/s lr: 4.55e-04 [09/27 03:27:43] lb.utils.events INFO: eta: 7:45:21 iteration: 199899/375342 consumed_samples: 204697600 total_loss: 3.502 time: 0.3286 s/iter data_time: 0.2161 s/iter total_throughput: 3116.19 samples/s lr: 4.54e-04 [09/27 03:28:17] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0199999 [09/27 03:28:17] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/27 03:28:17] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/27 03:28:21] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0841 s/iter. Inference: 0.1492 s/iter. Eval: 0.0021 s/iter. Total: 0.2353 s/iter. ETA=0:00:08 [09/27 03:28:27] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1427 s/iter. Inference: 0.1494 s/iter. Eval: 0.0020 s/iter. Total: 0.2942 s/iter. ETA=0:00:05 [09/27 03:28:32] lb.evaluation.evaluator INFO: Inference done 50000/50000. Dataloading: 0.1307 s/iter. Inference: 0.1504 s/iter. Eval: 0.0021 s/iter. Total: 0.2832 s/iter. ETA=0:00:00 [09/27 03:28:32] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/27 03:28:32] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.467096 (0.000249 s / iter per device, on 8 devices) [09/27 03:28:32] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000132 s / iter per device, on 8 devices) [09/27 03:28:32] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/27 03:28:32] lb.evaluation.utils INFO: copypaste: Acc@1=75.40599999999999 [09/27 03:28:32] lb.evaluation.utils INFO: copypaste: Acc@5=92.714 [09/27 03:28:32] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 75.40600, better than last best score 75.21600 @ iteration 194999. [09/27 03:28:32] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/27 03:28:33] lb.utils.events INFO: eta: 7:47:42 iteration: 199999/375342 consumed_samples: 204800000 total_loss: 3.497 time: 0.3286 s/iter data_time: 0.2100 s/iter total_throughput: 3116.18 samples/s lr: 4.54e-04 [09/27 03:29:04] lb.utils.events INFO: eta: 7:47:26 iteration: 200099/375342 consumed_samples: 204902400 total_loss: 3.516 time: 0.3286 s/iter data_time: 0.2312 s/iter total_throughput: 3116.26 samples/s lr: 4.54e-04 [09/27 03:29:37] lb.utils.events INFO: eta: 7:52:46 iteration: 200199/375342 consumed_samples: 205004800 total_loss: 3.522 time: 0.3286 s/iter data_time: 0.2468 s/iter total_throughput: 3116.25 samples/s lr: 4.53e-04 [09/27 03:30:10] lb.utils.events INFO: eta: 8:03:58 iteration: 200299/375342 consumed_samples: 205107200 total_loss: 3.5 time: 0.3286 s/iter data_time: 0.2411 s/iter total_throughput: 3116.26 samples/s lr: 4.53e-04 [09/27 03:30:43] lb.utils.events INFO: eta: 8:22:22 iteration: 200399/375342 consumed_samples: 205209600 total_loss: 3.477 time: 0.3286 s/iter data_time: 0.2537 s/iter total_throughput: 3116.25 samples/s lr: 4.52e-04 [09/27 03:31:16] lb.utils.events INFO: eta: 8:29:57 iteration: 200499/375342 consumed_samples: 205312000 total_loss: 3.479 time: 0.3286 s/iter data_time: 0.2538 s/iter total_throughput: 3116.25 samples/s lr: 4.52e-04 [09/27 03:31:49] lb.utils.events INFO: eta: 9:21:40 iteration: 200599/375342 consumed_samples: 205414400 total_loss: 3.49 time: 0.3286 s/iter data_time: 0.2349 s/iter total_throughput: 3116.24 samples/s lr: 4.52e-04 [09/27 03:32:22] lb.utils.events INFO: eta: 9:21:21 iteration: 200699/375342 consumed_samples: 205516800 total_loss: 3.502 time: 0.3286 s/iter data_time: 0.2203 s/iter total_throughput: 3116.23 samples/s lr: 4.51e-04 [09/27 03:32:55] lb.utils.events INFO: eta: 9:53:03 iteration: 200799/375342 consumed_samples: 205619200 total_loss: 3.519 time: 0.3286 s/iter data_time: 0.2189 s/iter total_throughput: 3116.23 samples/s lr: 4.51e-04 [09/27 03:33:28] lb.utils.events INFO: eta: 9:21:28 iteration: 200899/375342 consumed_samples: 205721600 total_loss: 3.512 time: 0.3286 s/iter data_time: 0.2031 s/iter total_throughput: 3116.22 samples/s lr: 4.50e-04 [09/27 03:34:01] lb.utils.events INFO: eta: 9:12:58 iteration: 200999/375342 consumed_samples: 205824000 total_loss: 3.496 time: 0.3286 s/iter data_time: 0.2069 s/iter total_throughput: 3116.21 samples/s lr: 4.50e-04 [09/27 03:34:34] lb.utils.events INFO: eta: 8:42:39 iteration: 201099/375342 consumed_samples: 205926400 total_loss: 3.521 time: 0.3286 s/iter data_time: 0.2049 s/iter total_throughput: 3116.21 samples/s lr: 4.49e-04 [09/27 03:35:07] lb.utils.events INFO: eta: 7:56:49 iteration: 201199/375342 consumed_samples: 206028800 total_loss: 3.522 time: 0.3286 s/iter data_time: 0.2121 s/iter total_throughput: 3116.22 samples/s lr: 4.49e-04 [09/27 03:35:40] lb.utils.events INFO: eta: 7:43:36 iteration: 201299/375342 consumed_samples: 206131200 total_loss: 3.529 time: 0.3286 s/iter data_time: 0.2205 s/iter total_throughput: 3116.21 samples/s lr: 4.49e-04 [09/27 03:36:13] lb.utils.events INFO: eta: 7:35:41 iteration: 201399/375342 consumed_samples: 206233600 total_loss: 3.526 time: 0.3286 s/iter data_time: 0.2130 s/iter total_throughput: 3116.19 samples/s lr: 4.48e-04 [09/27 03:36:46] lb.utils.events INFO: eta: 7:31:31 iteration: 201499/375342 consumed_samples: 206336000 total_loss: 3.518 time: 0.3286 s/iter data_time: 0.2031 s/iter total_throughput: 3116.19 samples/s lr: 4.48e-04 [09/27 03:37:19] lb.utils.events INFO: eta: 7:28:57 iteration: 201599/375342 consumed_samples: 206438400 total_loss: 3.509 time: 0.3286 s/iter data_time: 0.2027 s/iter total_throughput: 3116.18 samples/s lr: 4.47e-04 [09/27 03:37:52] lb.utils.events INFO: eta: 7:27:04 iteration: 201699/375342 consumed_samples: 206540800 total_loss: 3.485 time: 0.3286 s/iter data_time: 0.2147 s/iter total_throughput: 3116.17 samples/s lr: 4.47e-04 [09/27 03:38:25] lb.utils.events INFO: eta: 7:25:12 iteration: 201799/375342 consumed_samples: 206643200 total_loss: 3.51 time: 0.3286 s/iter data_time: 0.1997 s/iter total_throughput: 3116.16 samples/s lr: 4.47e-04 [09/27 03:38:59] lb.utils.events INFO: eta: 7:24:09 iteration: 201899/375342 consumed_samples: 206745600 total_loss: 3.525 time: 0.3286 s/iter data_time: 0.2106 s/iter total_throughput: 3116.13 samples/s lr: 4.46e-04 [09/27 03:39:31] lb.utils.events INFO: eta: 7:23:26 iteration: 201999/375342 consumed_samples: 206848000 total_loss: 3.498 time: 0.3286 s/iter data_time: 0.2044 s/iter total_throughput: 3116.15 samples/s lr: 4.46e-04 [09/27 03:40:04] lb.utils.events INFO: eta: 7:24:07 iteration: 202099/375342 consumed_samples: 206950400 total_loss: 3.481 time: 0.3286 s/iter data_time: 0.2120 s/iter total_throughput: 3116.13 samples/s lr: 4.45e-04 [09/27 03:40:37] lb.utils.events INFO: eta: 7:24:12 iteration: 202199/375342 consumed_samples: 207052800 total_loss: 3.494 time: 0.3286 s/iter data_time: 0.2181 s/iter total_throughput: 3116.13 samples/s lr: 4.45e-04 [09/27 03:41:11] lb.utils.events INFO: eta: 7:24:30 iteration: 202299/375342 consumed_samples: 207155200 total_loss: 3.506 time: 0.3286 s/iter data_time: 0.2092 s/iter total_throughput: 3116.11 samples/s lr: 4.45e-04 [09/27 03:41:43] lb.utils.events INFO: eta: 7:24:28 iteration: 202399/375342 consumed_samples: 207257600 total_loss: 3.507 time: 0.3286 s/iter data_time: 0.1989 s/iter total_throughput: 3116.11 samples/s lr: 4.44e-04 [09/27 03:42:17] lb.utils.events INFO: eta: 7:24:49 iteration: 202499/375342 consumed_samples: 207360000 total_loss: 3.504 time: 0.3286 s/iter data_time: 0.2074 s/iter total_throughput: 3116.09 samples/s lr: 4.44e-04 [09/27 03:42:50] lb.utils.events INFO: eta: 7:24:36 iteration: 202599/375342 consumed_samples: 207462400 total_loss: 3.506 time: 0.3286 s/iter data_time: 0.2297 s/iter total_throughput: 3116.08 samples/s lr: 4.43e-04 [09/27 03:43:23] lb.utils.events INFO: eta: 7:25:24 iteration: 202699/375342 consumed_samples: 207564800 total_loss: 3.501 time: 0.3286 s/iter data_time: 0.2042 s/iter total_throughput: 3116.06 samples/s lr: 4.43e-04 [09/27 03:43:56] lb.utils.events INFO: eta: 7:25:10 iteration: 202799/375342 consumed_samples: 207667200 total_loss: 3.479 time: 0.3286 s/iter data_time: 0.1999 s/iter total_throughput: 3116.06 samples/s lr: 4.42e-04 [09/27 03:44:29] lb.utils.events INFO: eta: 7:25:13 iteration: 202899/375342 consumed_samples: 207769600 total_loss: 3.485 time: 0.3286 s/iter data_time: 0.2060 s/iter total_throughput: 3116.07 samples/s lr: 4.42e-04 [09/27 03:45:02] lb.utils.events INFO: eta: 7:24:57 iteration: 202999/375342 consumed_samples: 207872000 total_loss: 3.501 time: 0.3286 s/iter data_time: 0.2178 s/iter total_throughput: 3116.03 samples/s lr: 4.42e-04 [09/27 03:45:36] lb.utils.events INFO: eta: 7:23:41 iteration: 203099/375342 consumed_samples: 207974400 total_loss: 3.489 time: 0.3286 s/iter data_time: 0.2069 s/iter total_throughput: 3115.99 samples/s lr: 4.41e-04 [09/27 03:46:09] lb.utils.events INFO: eta: 7:22:55 iteration: 203199/375342 consumed_samples: 208076800 total_loss: 3.487 time: 0.3286 s/iter data_time: 0.2007 s/iter total_throughput: 3115.99 samples/s lr: 4.41e-04 [09/27 03:46:42] lb.utils.events INFO: eta: 7:22:05 iteration: 203299/375342 consumed_samples: 208179200 total_loss: 3.491 time: 0.3286 s/iter data_time: 0.2079 s/iter total_throughput: 3115.97 samples/s lr: 4.40e-04 [09/27 03:47:15] lb.utils.events INFO: eta: 7:21:22 iteration: 203399/375342 consumed_samples: 208281600 total_loss: 3.496 time: 0.3286 s/iter data_time: 0.2143 s/iter total_throughput: 3115.97 samples/s lr: 4.40e-04 [09/27 03:47:48] lb.utils.events INFO: eta: 7:21:25 iteration: 203499/375342 consumed_samples: 208384000 total_loss: 3.499 time: 0.3286 s/iter data_time: 0.2114 s/iter total_throughput: 3115.97 samples/s lr: 4.40e-04 [09/27 03:48:21] lb.utils.events INFO: eta: 7:20:27 iteration: 203599/375342 consumed_samples: 208486400 total_loss: 3.498 time: 0.3286 s/iter data_time: 0.2186 s/iter total_throughput: 3115.95 samples/s lr: 4.39e-04 [09/27 03:48:54] lb.utils.events INFO: eta: 7:19:41 iteration: 203699/375342 consumed_samples: 208588800 total_loss: 3.497 time: 0.3286 s/iter data_time: 0.1953 s/iter total_throughput: 3115.97 samples/s lr: 4.39e-04 [09/27 03:49:27] lb.utils.events INFO: eta: 7:19:25 iteration: 203799/375342 consumed_samples: 208691200 total_loss: 3.499 time: 0.3286 s/iter data_time: 0.2039 s/iter total_throughput: 3115.96 samples/s lr: 4.38e-04 [09/27 03:50:00] lb.utils.events INFO: eta: 7:18:06 iteration: 203899/375342 consumed_samples: 208793600 total_loss: 3.513 time: 0.3286 s/iter data_time: 0.2149 s/iter total_throughput: 3115.95 samples/s lr: 4.38e-04 [09/27 03:50:32] lb.utils.events INFO: eta: 7:17:53 iteration: 203999/375342 consumed_samples: 208896000 total_loss: 3.503 time: 0.3286 s/iter data_time: 0.1993 s/iter total_throughput: 3115.96 samples/s lr: 4.38e-04 [09/27 03:51:05] lb.utils.events INFO: eta: 7:18:37 iteration: 204099/375342 consumed_samples: 208998400 total_loss: 3.478 time: 0.3286 s/iter data_time: 0.2089 s/iter total_throughput: 3115.96 samples/s lr: 4.37e-04 [09/27 03:51:39] lb.utils.events INFO: eta: 7:19:08 iteration: 204199/375342 consumed_samples: 209100800 total_loss: 3.462 time: 0.3286 s/iter data_time: 0.2204 s/iter total_throughput: 3115.94 samples/s lr: 4.37e-04 [09/27 03:52:12] lb.utils.events INFO: eta: 7:18:38 iteration: 204299/375342 consumed_samples: 209203200 total_loss: 3.475 time: 0.3286 s/iter data_time: 0.2036 s/iter total_throughput: 3115.94 samples/s lr: 4.36e-04 [09/27 03:52:45] lb.utils.events INFO: eta: 7:17:55 iteration: 204399/375342 consumed_samples: 209305600 total_loss: 3.485 time: 0.3286 s/iter data_time: 0.2053 s/iter total_throughput: 3115.93 samples/s lr: 4.36e-04 [09/27 03:53:18] lb.utils.events INFO: eta: 7:17:39 iteration: 204499/375342 consumed_samples: 209408000 total_loss: 3.494 time: 0.3286 s/iter data_time: 0.2056 s/iter total_throughput: 3115.93 samples/s lr: 4.36e-04 [09/27 03:53:50] lb.utils.events INFO: eta: 7:19:35 iteration: 204599/375342 consumed_samples: 209510400 total_loss: 3.497 time: 0.3286 s/iter data_time: 0.2179 s/iter total_throughput: 3115.95 samples/s lr: 4.35e-04 [09/27 03:54:23] lb.utils.events INFO: eta: 7:22:07 iteration: 204699/375342 consumed_samples: 209612800 total_loss: 3.487 time: 0.3286 s/iter data_time: 0.2410 s/iter total_throughput: 3115.95 samples/s lr: 4.35e-04 [09/27 03:54:56] lb.utils.events INFO: eta: 7:28:16 iteration: 204799/375342 consumed_samples: 209715200 total_loss: 3.5 time: 0.3286 s/iter data_time: 0.2302 s/iter total_throughput: 3115.96 samples/s lr: 4.34e-04 [09/27 03:55:29] lb.utils.events INFO: eta: 7:36:36 iteration: 204899/375342 consumed_samples: 209817600 total_loss: 3.506 time: 0.3286 s/iter data_time: 0.2424 s/iter total_throughput: 3115.95 samples/s lr: 4.34e-04 [09/27 03:56:01] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0204999 [09/27 03:56:02] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/27 03:56:02] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/27 03:56:06] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0882 s/iter. Inference: 0.1522 s/iter. Eval: 0.0021 s/iter. Total: 0.2425 s/iter. ETA=0:00:08 [09/27 03:56:12] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1501 s/iter. Inference: 0.1517 s/iter. Eval: 0.0019 s/iter. Total: 0.3039 s/iter. ETA=0:00:05 [09/27 03:56:17] lb.evaluation.evaluator INFO: Inference done 49152/50000. Dataloading: 0.1338 s/iter. Inference: 0.1505 s/iter. Eval: 0.0019 s/iter. Total: 0.2864 s/iter. ETA=0:00:00 [09/27 03:56:17] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/27 03:56:17] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.770744 (0.000255 s / iter per device, on 8 devices) [09/27 03:56:17] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000134 s / iter per device, on 8 devices) [09/27 03:56:17] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/27 03:56:17] lb.evaluation.utils INFO: copypaste: Acc@1=75.614 [09/27 03:56:17] lb.evaluation.utils INFO: copypaste: Acc@5=92.92200000000001 [09/27 03:56:17] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 75.61400, better than last best score 75.40600 @ iteration 199999. [09/27 03:56:17] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/27 03:56:18] lb.utils.events INFO: eta: 7:40:54 iteration: 204999/375342 consumed_samples: 209920000 total_loss: 3.501 time: 0.3286 s/iter data_time: 0.2238 s/iter total_throughput: 3115.96 samples/s lr: 4.33e-04 [09/27 03:56:49] lb.utils.events INFO: eta: 7:49:35 iteration: 205099/375342 consumed_samples: 210022400 total_loss: 3.503 time: 0.3286 s/iter data_time: 0.2468 s/iter total_throughput: 3116.04 samples/s lr: 4.33e-04 [09/27 03:57:22] lb.utils.events INFO: eta: 8:08:13 iteration: 205199/375342 consumed_samples: 210124800 total_loss: 3.508 time: 0.3286 s/iter data_time: 0.2197 s/iter total_throughput: 3116.04 samples/s lr: 4.33e-04 [09/27 03:57:55] lb.utils.events INFO: eta: 9:25:37 iteration: 205299/375342 consumed_samples: 210227200 total_loss: 3.497 time: 0.3286 s/iter data_time: 0.2141 s/iter total_throughput: 3116.04 samples/s lr: 4.32e-04 [09/27 03:58:28] lb.utils.events INFO: eta: 10:50:01 iteration: 205399/375342 consumed_samples: 210329600 total_loss: 3.494 time: 0.3286 s/iter data_time: 0.2416 s/iter total_throughput: 3116.05 samples/s lr: 4.32e-04 [09/27 03:59:00] lb.utils.events INFO: eta: 11:57:39 iteration: 205499/375342 consumed_samples: 210432000 total_loss: 3.492 time: 0.3286 s/iter data_time: 0.2365 s/iter total_throughput: 3116.06 samples/s lr: 4.31e-04 [09/27 03:59:33] lb.utils.events INFO: eta: 11:06:03 iteration: 205599/375342 consumed_samples: 210534400 total_loss: 3.51 time: 0.3286 s/iter data_time: 0.2122 s/iter total_throughput: 3116.05 samples/s lr: 4.31e-04 [09/27 04:00:06] lb.utils.events INFO: eta: 10:14:58 iteration: 205699/375342 consumed_samples: 210636800 total_loss: 3.5 time: 0.3286 s/iter data_time: 0.2161 s/iter total_throughput: 3116.05 samples/s lr: 4.31e-04 [09/27 04:00:39] lb.utils.events INFO: eta: 8:58:32 iteration: 205799/375342 consumed_samples: 210739200 total_loss: 3.487 time: 0.3286 s/iter data_time: 0.2291 s/iter total_throughput: 3116.07 samples/s lr: 4.30e-04 [09/27 04:01:12] lb.utils.events INFO: eta: 8:39:21 iteration: 205899/375342 consumed_samples: 210841600 total_loss: 3.46 time: 0.3286 s/iter data_time: 0.2489 s/iter total_throughput: 3116.07 samples/s lr: 4.30e-04 [09/27 04:01:44] lb.utils.events INFO: eta: 9:23:17 iteration: 205999/375342 consumed_samples: 210944000 total_loss: 3.462 time: 0.3286 s/iter data_time: 0.2313 s/iter total_throughput: 3116.08 samples/s lr: 4.29e-04 [09/27 04:02:18] lb.utils.events INFO: eta: 9:16:00 iteration: 206099/375342 consumed_samples: 211046400 total_loss: 3.481 time: 0.3286 s/iter data_time: 0.2400 s/iter total_throughput: 3116.06 samples/s lr: 4.29e-04 [09/27 04:02:50] lb.utils.events INFO: eta: 9:13:38 iteration: 206199/375342 consumed_samples: 211148800 total_loss: 3.495 time: 0.3286 s/iter data_time: 0.2197 s/iter total_throughput: 3116.06 samples/s lr: 4.29e-04 [09/27 04:03:23] lb.utils.events INFO: eta: 8:39:29 iteration: 206299/375342 consumed_samples: 211251200 total_loss: 3.503 time: 0.3286 s/iter data_time: 0.2229 s/iter total_throughput: 3116.08 samples/s lr: 4.28e-04 [09/27 04:03:56] lb.utils.events INFO: eta: 8:13:29 iteration: 206399/375342 consumed_samples: 211353600 total_loss: 3.492 time: 0.3286 s/iter data_time: 0.2217 s/iter total_throughput: 3116.08 samples/s lr: 4.28e-04 [09/27 04:04:29] lb.utils.events INFO: eta: 7:44:00 iteration: 206499/375342 consumed_samples: 211456000 total_loss: 3.486 time: 0.3286 s/iter data_time: 0.2006 s/iter total_throughput: 3116.08 samples/s lr: 4.27e-04 [09/27 04:05:01] lb.utils.events INFO: eta: 7:39:09 iteration: 206599/375342 consumed_samples: 211558400 total_loss: 3.502 time: 0.3286 s/iter data_time: 0.2082 s/iter total_throughput: 3116.09 samples/s lr: 4.27e-04 [09/27 04:05:34] lb.utils.events INFO: eta: 7:35:03 iteration: 206699/375342 consumed_samples: 211660800 total_loss: 3.499 time: 0.3286 s/iter data_time: 0.2029 s/iter total_throughput: 3116.09 samples/s lr: 4.26e-04 [09/27 04:06:07] lb.utils.events INFO: eta: 7:30:14 iteration: 206799/375342 consumed_samples: 211763200 total_loss: 3.471 time: 0.3286 s/iter data_time: 0.2003 s/iter total_throughput: 3116.10 samples/s lr: 4.26e-04 [09/27 04:06:39] lb.utils.events INFO: eta: 7:24:54 iteration: 206899/375342 consumed_samples: 211865600 total_loss: 3.494 time: 0.3286 s/iter data_time: 0.1986 s/iter total_throughput: 3116.13 samples/s lr: 4.26e-04 [09/27 04:07:12] lb.utils.events INFO: eta: 7:20:18 iteration: 206999/375342 consumed_samples: 211968000 total_loss: 3.508 time: 0.3286 s/iter data_time: 0.2406 s/iter total_throughput: 3116.14 samples/s lr: 4.25e-04 [09/27 04:07:45] lb.utils.events INFO: eta: 7:19:33 iteration: 207099/375342 consumed_samples: 212070400 total_loss: 3.51 time: 0.3286 s/iter data_time: 0.2280 s/iter total_throughput: 3116.14 samples/s lr: 4.25e-04 [09/27 04:08:17] lb.utils.events INFO: eta: 7:20:17 iteration: 207199/375342 consumed_samples: 212172800 total_loss: 3.516 time: 0.3286 s/iter data_time: 0.2502 s/iter total_throughput: 3116.14 samples/s lr: 4.24e-04 [09/27 04:08:50] lb.utils.events INFO: eta: 7:22:02 iteration: 207299/375342 consumed_samples: 212275200 total_loss: 3.515 time: 0.3286 s/iter data_time: 0.2226 s/iter total_throughput: 3116.13 samples/s lr: 4.24e-04 [09/27 04:09:23] lb.utils.events INFO: eta: 7:20:45 iteration: 207399/375342 consumed_samples: 212377600 total_loss: 3.498 time: 0.3286 s/iter data_time: 0.2031 s/iter total_throughput: 3116.13 samples/s lr: 4.24e-04 [09/27 04:09:56] lb.utils.events INFO: eta: 7:21:20 iteration: 207499/375342 consumed_samples: 212480000 total_loss: 3.494 time: 0.3286 s/iter data_time: 0.2143 s/iter total_throughput: 3116.15 samples/s lr: 4.23e-04 [09/27 04:10:28] lb.utils.events INFO: eta: 7:23:53 iteration: 207599/375342 consumed_samples: 212582400 total_loss: 3.479 time: 0.3286 s/iter data_time: 0.2154 s/iter total_throughput: 3116.16 samples/s lr: 4.23e-04 [09/27 04:11:01] lb.utils.events INFO: eta: 7:24:42 iteration: 207699/375342 consumed_samples: 212684800 total_loss: 3.473 time: 0.3286 s/iter data_time: 0.2086 s/iter total_throughput: 3116.15 samples/s lr: 4.22e-04 [09/27 04:11:34] lb.utils.events INFO: eta: 7:23:58 iteration: 207799/375342 consumed_samples: 212787200 total_loss: 3.46 time: 0.3286 s/iter data_time: 0.2052 s/iter total_throughput: 3116.17 samples/s lr: 4.22e-04 [09/27 04:12:07] lb.utils.events INFO: eta: 7:22:43 iteration: 207899/375342 consumed_samples: 212889600 total_loss: 3.447 time: 0.3286 s/iter data_time: 0.2067 s/iter total_throughput: 3116.18 samples/s lr: 4.22e-04 [09/27 04:12:39] lb.utils.events INFO: eta: 7:19:40 iteration: 207999/375342 consumed_samples: 212992000 total_loss: 3.461 time: 0.3286 s/iter data_time: 0.2063 s/iter total_throughput: 3116.21 samples/s lr: 4.21e-04 [09/27 04:13:12] lb.utils.events INFO: eta: 7:18:55 iteration: 208099/375342 consumed_samples: 213094400 total_loss: 3.484 time: 0.3286 s/iter data_time: 0.2008 s/iter total_throughput: 3116.22 samples/s lr: 4.21e-04 [09/27 04:13:44] lb.utils.events INFO: eta: 7:15:31 iteration: 208199/375342 consumed_samples: 213196800 total_loss: 3.482 time: 0.3286 s/iter data_time: 0.2078 s/iter total_throughput: 3116.23 samples/s lr: 4.20e-04 [09/27 04:14:17] lb.utils.events INFO: eta: 7:12:40 iteration: 208299/375342 consumed_samples: 213299200 total_loss: 3.469 time: 0.3286 s/iter data_time: 0.2127 s/iter total_throughput: 3116.25 samples/s lr: 4.20e-04 [09/27 04:14:49] lb.utils.events INFO: eta: 7:13:57 iteration: 208399/375342 consumed_samples: 213401600 total_loss: 3.461 time: 0.3286 s/iter data_time: 0.2175 s/iter total_throughput: 3116.27 samples/s lr: 4.20e-04 [09/27 04:15:22] lb.utils.events INFO: eta: 7:14:25 iteration: 208499/375342 consumed_samples: 213504000 total_loss: 3.469 time: 0.3286 s/iter data_time: 0.2274 s/iter total_throughput: 3116.28 samples/s lr: 4.19e-04 [09/27 04:15:55] lb.utils.events INFO: eta: 7:11:44 iteration: 208599/375342 consumed_samples: 213606400 total_loss: 3.485 time: 0.3286 s/iter data_time: 0.1991 s/iter total_throughput: 3116.28 samples/s lr: 4.19e-04 [09/27 04:16:27] lb.utils.events INFO: eta: 7:11:46 iteration: 208699/375342 consumed_samples: 213708800 total_loss: 3.506 time: 0.3286 s/iter data_time: 0.2293 s/iter total_throughput: 3116.29 samples/s lr: 4.18e-04 [09/27 04:17:00] lb.utils.events INFO: eta: 7:13:45 iteration: 208799/375342 consumed_samples: 213811200 total_loss: 3.482 time: 0.3286 s/iter data_time: 0.2254 s/iter total_throughput: 3116.31 samples/s lr: 4.18e-04 [09/27 04:17:32] lb.utils.events INFO: eta: 7:14:17 iteration: 208899/375342 consumed_samples: 213913600 total_loss: 3.472 time: 0.3286 s/iter data_time: 0.2303 s/iter total_throughput: 3116.32 samples/s lr: 4.18e-04 [09/27 04:18:05] lb.utils.events INFO: eta: 7:18:32 iteration: 208999/375342 consumed_samples: 214016000 total_loss: 3.458 time: 0.3286 s/iter data_time: 0.2224 s/iter total_throughput: 3116.32 samples/s lr: 4.17e-04 [09/27 04:18:38] lb.utils.events INFO: eta: 7:23:15 iteration: 209099/375342 consumed_samples: 214118400 total_loss: 3.454 time: 0.3286 s/iter data_time: 0.2190 s/iter total_throughput: 3116.34 samples/s lr: 4.17e-04 [09/27 04:19:10] lb.utils.events INFO: eta: 7:22:11 iteration: 209199/375342 consumed_samples: 214220800 total_loss: 3.467 time: 0.3286 s/iter data_time: 0.1946 s/iter total_throughput: 3116.36 samples/s lr: 4.16e-04 [09/27 04:19:42] lb.utils.events INFO: eta: 7:22:53 iteration: 209299/375342 consumed_samples: 214323200 total_loss: 3.475 time: 0.3286 s/iter data_time: 0.1957 s/iter total_throughput: 3116.39 samples/s lr: 4.16e-04 [09/27 04:20:15] lb.utils.events INFO: eta: 7:22:25 iteration: 209399/375342 consumed_samples: 214425600 total_loss: 3.474 time: 0.3286 s/iter data_time: 0.2592 s/iter total_throughput: 3116.41 samples/s lr: 4.15e-04 [09/27 04:20:47] lb.utils.events INFO: eta: 7:25:40 iteration: 209499/375342 consumed_samples: 214528000 total_loss: 3.459 time: 0.3286 s/iter data_time: 0.2244 s/iter total_throughput: 3116.42 samples/s lr: 4.15e-04 [09/27 04:21:20] lb.utils.events INFO: eta: 7:30:18 iteration: 209599/375342 consumed_samples: 214630400 total_loss: 3.469 time: 0.3286 s/iter data_time: 0.2112 s/iter total_throughput: 3116.43 samples/s lr: 4.15e-04 [09/27 04:21:52] lb.utils.events INFO: eta: 7:33:35 iteration: 209699/375342 consumed_samples: 214732800 total_loss: 3.475 time: 0.3286 s/iter data_time: 0.2187 s/iter total_throughput: 3116.46 samples/s lr: 4.14e-04 [09/27 04:22:24] lb.utils.events INFO: eta: 7:36:45 iteration: 209799/375342 consumed_samples: 214835200 total_loss: 3.495 time: 0.3286 s/iter data_time: 0.2367 s/iter total_throughput: 3116.49 samples/s lr: 4.14e-04 [09/27 04:22:57] lb.utils.events INFO: eta: 7:53:48 iteration: 209899/375342 consumed_samples: 214937600 total_loss: 3.498 time: 0.3286 s/iter data_time: 0.2398 s/iter total_throughput: 3116.51 samples/s lr: 4.13e-04 [09/27 04:23:29] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0209999 [09/27 04:23:30] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/27 04:23:30] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/27 04:23:34] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0822 s/iter. Inference: 0.1485 s/iter. Eval: 0.0020 s/iter. Total: 0.2327 s/iter. ETA=0:00:08 [09/27 04:23:39] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1423 s/iter. Inference: 0.1494 s/iter. Eval: 0.0020 s/iter. Total: 0.2938 s/iter. ETA=0:00:05 [09/27 04:23:44] lb.evaluation.evaluator INFO: Inference done 49152/50000. Dataloading: 0.1298 s/iter. Inference: 0.1493 s/iter. Eval: 0.0020 s/iter. Total: 0.2811 s/iter. ETA=0:00:00 [09/27 04:23:45] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/27 04:23:45] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.531132 (0.000251 s / iter per device, on 8 devices) [09/27 04:23:45] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000132 s / iter per device, on 8 devices) [09/27 04:23:45] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/27 04:23:45] lb.evaluation.utils INFO: copypaste: Acc@1=75.964 [09/27 04:23:45] lb.evaluation.utils INFO: copypaste: Acc@5=92.96799999999999 [09/27 04:23:45] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 75.96400, better than last best score 75.61400 @ iteration 204999. [09/27 04:23:45] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/27 04:23:46] lb.utils.events INFO: eta: 7:59:29 iteration: 209999/375342 consumed_samples: 215040000 total_loss: 3.484 time: 0.3286 s/iter data_time: 0.2409 s/iter total_throughput: 3116.54 samples/s lr: 4.13e-04 [09/27 04:24:17] lb.utils.events INFO: eta: 7:33:41 iteration: 210099/375342 consumed_samples: 215142400 total_loss: 3.483 time: 0.3286 s/iter data_time: 0.2168 s/iter total_throughput: 3116.61 samples/s lr: 4.13e-04 [09/27 04:24:50] lb.utils.events INFO: eta: 7:32:23 iteration: 210199/375342 consumed_samples: 215244800 total_loss: 3.485 time: 0.3286 s/iter data_time: 0.2227 s/iter total_throughput: 3116.60 samples/s lr: 4.12e-04 [09/27 04:25:23] lb.utils.events INFO: eta: 7:43:17 iteration: 210299/375342 consumed_samples: 215347200 total_loss: 3.467 time: 0.3286 s/iter data_time: 0.2258 s/iter total_throughput: 3116.59 samples/s lr: 4.12e-04 [09/27 04:25:56] lb.utils.events INFO: eta: 7:44:19 iteration: 210399/375342 consumed_samples: 215449600 total_loss: 3.438 time: 0.3286 s/iter data_time: 0.2353 s/iter total_throughput: 3116.58 samples/s lr: 4.11e-04 [09/27 04:26:30] lb.utils.events INFO: eta: 7:34:00 iteration: 210499/375342 consumed_samples: 215552000 total_loss: 3.453 time: 0.3286 s/iter data_time: 0.2263 s/iter total_throughput: 3116.55 samples/s lr: 4.11e-04 [09/27 04:27:04] lb.utils.events INFO: eta: 7:30:24 iteration: 210599/375342 consumed_samples: 215654400 total_loss: 3.467 time: 0.3286 s/iter data_time: 0.2107 s/iter total_throughput: 3116.50 samples/s lr: 4.11e-04 [09/27 04:27:37] lb.utils.events INFO: eta: 7:26:59 iteration: 210699/375342 consumed_samples: 215756800 total_loss: 3.465 time: 0.3286 s/iter data_time: 0.2327 s/iter total_throughput: 3116.48 samples/s lr: 4.10e-04 [09/27 04:28:10] lb.utils.events INFO: eta: 7:24:02 iteration: 210799/375342 consumed_samples: 215859200 total_loss: 3.454 time: 0.3286 s/iter data_time: 0.2171 s/iter total_throughput: 3116.46 samples/s lr: 4.10e-04 [09/27 04:28:43] lb.utils.events INFO: eta: 7:18:40 iteration: 210899/375342 consumed_samples: 215961600 total_loss: 3.473 time: 0.3286 s/iter data_time: 0.2264 s/iter total_throughput: 3116.45 samples/s lr: 4.09e-04 [09/27 04:29:16] lb.utils.events INFO: eta: 7:12:59 iteration: 210999/375342 consumed_samples: 216064000 total_loss: 3.485 time: 0.3286 s/iter data_time: 0.2435 s/iter total_throughput: 3116.44 samples/s lr: 4.09e-04 [09/27 04:29:49] lb.utils.events INFO: eta: 7:16:23 iteration: 211099/375342 consumed_samples: 216166400 total_loss: 3.482 time: 0.3286 s/iter data_time: 0.2486 s/iter total_throughput: 3116.45 samples/s lr: 4.09e-04 [09/27 04:30:22] lb.utils.events INFO: eta: 7:32:47 iteration: 211199/375342 consumed_samples: 216268800 total_loss: 3.47 time: 0.3286 s/iter data_time: 0.2425 s/iter total_throughput: 3116.44 samples/s lr: 4.08e-04 [09/27 04:30:55] lb.utils.events INFO: eta: 7:39:56 iteration: 211299/375342 consumed_samples: 216371200 total_loss: 3.47 time: 0.3286 s/iter data_time: 0.2448 s/iter total_throughput: 3116.42 samples/s lr: 4.08e-04 [09/27 04:31:29] lb.utils.events INFO: eta: 7:39:10 iteration: 211399/375342 consumed_samples: 216473600 total_loss: 3.456 time: 0.3286 s/iter data_time: 0.2287 s/iter total_throughput: 3116.39 samples/s lr: 4.07e-04 [09/27 04:32:02] lb.utils.events INFO: eta: 7:44:28 iteration: 211499/375342 consumed_samples: 216576000 total_loss: 3.461 time: 0.3286 s/iter data_time: 0.2310 s/iter total_throughput: 3116.38 samples/s lr: 4.07e-04 [09/27 04:32:35] lb.utils.events INFO: eta: 8:20:50 iteration: 211599/375342 consumed_samples: 216678400 total_loss: 3.464 time: 0.3286 s/iter data_time: 0.2258 s/iter total_throughput: 3116.37 samples/s lr: 4.07e-04 [09/27 04:33:08] lb.utils.events INFO: eta: 9:42:13 iteration: 211699/375342 consumed_samples: 216780800 total_loss: 3.44 time: 0.3286 s/iter data_time: 0.2245 s/iter total_throughput: 3116.36 samples/s lr: 4.06e-04 [09/27 04:33:41] lb.utils.events INFO: eta: 10:25:44 iteration: 211799/375342 consumed_samples: 216883200 total_loss: 3.451 time: 0.3286 s/iter data_time: 0.2329 s/iter total_throughput: 3116.35 samples/s lr: 4.06e-04 [09/27 04:34:14] lb.utils.events INFO: eta: 10:32:32 iteration: 211899/375342 consumed_samples: 216985600 total_loss: 3.465 time: 0.3286 s/iter data_time: 0.2238 s/iter total_throughput: 3116.34 samples/s lr: 4.05e-04 [09/27 04:34:47] lb.utils.events INFO: eta: 10:53:21 iteration: 211999/375342 consumed_samples: 217088000 total_loss: 3.455 time: 0.3286 s/iter data_time: 0.2338 s/iter total_throughput: 3116.34 samples/s lr: 4.05e-04 [09/27 04:35:20] lb.utils.events INFO: eta: 7:59:31 iteration: 212099/375342 consumed_samples: 217190400 total_loss: 3.462 time: 0.3286 s/iter data_time: 0.2087 s/iter total_throughput: 3116.33 samples/s lr: 4.04e-04 [09/27 04:35:54] lb.utils.events INFO: eta: 7:23:05 iteration: 212199/375342 consumed_samples: 217292800 total_loss: 3.466 time: 0.3286 s/iter data_time: 0.2083 s/iter total_throughput: 3116.31 samples/s lr: 4.04e-04 [09/27 04:36:27] lb.utils.events INFO: eta: 7:15:51 iteration: 212299/375342 consumed_samples: 217395200 total_loss: 3.461 time: 0.3286 s/iter data_time: 0.2022 s/iter total_throughput: 3116.30 samples/s lr: 4.04e-04 [09/27 04:37:00] lb.utils.events INFO: eta: 7:11:33 iteration: 212399/375342 consumed_samples: 217497600 total_loss: 3.46 time: 0.3286 s/iter data_time: 0.2120 s/iter total_throughput: 3116.29 samples/s lr: 4.03e-04 [09/27 04:37:33] lb.utils.events INFO: eta: 7:07:49 iteration: 212499/375342 consumed_samples: 217600000 total_loss: 3.459 time: 0.3286 s/iter data_time: 0.2098 s/iter total_throughput: 3116.27 samples/s lr: 4.03e-04 [09/27 04:38:06] lb.utils.events INFO: eta: 7:05:39 iteration: 212599/375342 consumed_samples: 217702400 total_loss: 3.463 time: 0.3286 s/iter data_time: 0.2172 s/iter total_throughput: 3116.26 samples/s lr: 4.02e-04 [09/27 04:38:39] lb.utils.events INFO: eta: 7:01:40 iteration: 212699/375342 consumed_samples: 217804800 total_loss: 3.452 time: 0.3286 s/iter data_time: 0.2036 s/iter total_throughput: 3116.25 samples/s lr: 4.02e-04 [09/27 04:39:12] lb.utils.events INFO: eta: 6:59:43 iteration: 212799/375342 consumed_samples: 217907200 total_loss: 3.439 time: 0.3286 s/iter data_time: 0.2097 s/iter total_throughput: 3116.25 samples/s lr: 4.02e-04 [09/27 04:39:46] lb.utils.events INFO: eta: 6:57:59 iteration: 212899/375342 consumed_samples: 218009600 total_loss: 3.453 time: 0.3286 s/iter data_time: 0.2067 s/iter total_throughput: 3116.23 samples/s lr: 4.01e-04 [09/27 04:40:19] lb.utils.events INFO: eta: 6:56:13 iteration: 212999/375342 consumed_samples: 218112000 total_loss: 3.445 time: 0.3286 s/iter data_time: 0.2046 s/iter total_throughput: 3116.22 samples/s lr: 4.01e-04 [09/27 04:40:51] lb.utils.events INFO: eta: 6:55:14 iteration: 213099/375342 consumed_samples: 218214400 total_loss: 3.437 time: 0.3286 s/iter data_time: 0.2034 s/iter total_throughput: 3116.23 samples/s lr: 4.00e-04 [09/27 04:41:25] lb.utils.events INFO: eta: 6:53:41 iteration: 213199/375342 consumed_samples: 218316800 total_loss: 3.467 time: 0.3286 s/iter data_time: 0.2143 s/iter total_throughput: 3116.20 samples/s lr: 4.00e-04 [09/27 04:41:58] lb.utils.events INFO: eta: 6:53:44 iteration: 213299/375342 consumed_samples: 218419200 total_loss: 3.478 time: 0.3286 s/iter data_time: 0.2075 s/iter total_throughput: 3116.17 samples/s lr: 4.00e-04 [09/27 04:42:32] lb.utils.events INFO: eta: 6:54:00 iteration: 213399/375342 consumed_samples: 218521600 total_loss: 3.471 time: 0.3286 s/iter data_time: 0.2181 s/iter total_throughput: 3116.15 samples/s lr: 3.99e-04 [09/27 04:43:05] lb.utils.events INFO: eta: 6:54:11 iteration: 213499/375342 consumed_samples: 218624000 total_loss: 3.466 time: 0.3286 s/iter data_time: 0.2040 s/iter total_throughput: 3116.13 samples/s lr: 3.99e-04 [09/27 04:43:38] lb.utils.events INFO: eta: 6:52:57 iteration: 213599/375342 consumed_samples: 218726400 total_loss: 3.463 time: 0.3286 s/iter data_time: 0.2133 s/iter total_throughput: 3116.12 samples/s lr: 3.98e-04 [09/27 04:44:11] lb.utils.events INFO: eta: 6:52:03 iteration: 213699/375342 consumed_samples: 218828800 total_loss: 3.466 time: 0.3286 s/iter data_time: 0.2071 s/iter total_throughput: 3116.13 samples/s lr: 3.98e-04 [09/27 04:44:44] lb.utils.events INFO: eta: 6:52:41 iteration: 213799/375342 consumed_samples: 218931200 total_loss: 3.472 time: 0.3286 s/iter data_time: 0.2300 s/iter total_throughput: 3116.11 samples/s lr: 3.98e-04 [09/27 04:45:17] lb.utils.events INFO: eta: 6:54:14 iteration: 213899/375342 consumed_samples: 219033600 total_loss: 3.449 time: 0.3286 s/iter data_time: 0.2321 s/iter total_throughput: 3116.10 samples/s lr: 3.97e-04 [09/27 04:45:50] lb.utils.events INFO: eta: 6:55:50 iteration: 213999/375342 consumed_samples: 219136000 total_loss: 3.432 time: 0.3286 s/iter data_time: 0.2377 s/iter total_throughput: 3116.08 samples/s lr: 3.97e-04 [09/27 04:46:23] lb.utils.events INFO: eta: 6:58:24 iteration: 214099/375342 consumed_samples: 219238400 total_loss: 3.448 time: 0.3286 s/iter data_time: 0.2136 s/iter total_throughput: 3116.10 samples/s lr: 3.96e-04 [09/27 04:46:56] lb.utils.events INFO: eta: 7:00:26 iteration: 214199/375342 consumed_samples: 219340800 total_loss: 3.455 time: 0.3286 s/iter data_time: 0.2167 s/iter total_throughput: 3116.09 samples/s lr: 3.96e-04 [09/27 04:47:29] lb.utils.events INFO: eta: 7:01:04 iteration: 214299/375342 consumed_samples: 219443200 total_loss: 3.453 time: 0.3286 s/iter data_time: 0.2292 s/iter total_throughput: 3116.07 samples/s lr: 3.96e-04 [09/27 04:48:03] lb.utils.events INFO: eta: 7:00:37 iteration: 214399/375342 consumed_samples: 219545600 total_loss: 3.443 time: 0.3286 s/iter data_time: 0.2112 s/iter total_throughput: 3116.03 samples/s lr: 3.95e-04 [09/27 04:48:36] lb.utils.events INFO: eta: 7:00:22 iteration: 214499/375342 consumed_samples: 219648000 total_loss: 3.446 time: 0.3286 s/iter data_time: 0.2084 s/iter total_throughput: 3116.02 samples/s lr: 3.95e-04 [09/27 04:49:09] lb.utils.events INFO: eta: 7:01:16 iteration: 214599/375342 consumed_samples: 219750400 total_loss: 3.451 time: 0.3286 s/iter data_time: 0.2091 s/iter total_throughput: 3116.04 samples/s lr: 3.94e-04 [09/27 04:49:42] lb.utils.events INFO: eta: 7:03:49 iteration: 214699/375342 consumed_samples: 219852800 total_loss: 3.439 time: 0.3286 s/iter data_time: 0.2181 s/iter total_throughput: 3116.03 samples/s lr: 3.94e-04 [09/27 04:50:15] lb.utils.events INFO: eta: 7:03:20 iteration: 214799/375342 consumed_samples: 219955200 total_loss: 3.458 time: 0.3286 s/iter data_time: 0.2365 s/iter total_throughput: 3116.00 samples/s lr: 3.94e-04 [09/27 04:50:48] lb.utils.events INFO: eta: 7:04:28 iteration: 214899/375342 consumed_samples: 220057600 total_loss: 3.46 time: 0.3286 s/iter data_time: 0.2310 s/iter total_throughput: 3116.00 samples/s lr: 3.93e-04 [09/27 04:51:21] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0214999 [09/27 04:51:22] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/27 04:51:22] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/27 04:51:26] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0869 s/iter. Inference: 0.1477 s/iter. Eval: 0.0020 s/iter. Total: 0.2365 s/iter. ETA=0:00:08 [09/27 04:51:32] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1455 s/iter. Inference: 0.1493 s/iter. Eval: 0.0020 s/iter. Total: 0.2969 s/iter. ETA=0:00:05 [09/27 04:51:37] lb.evaluation.evaluator INFO: Inference done 50000/50000. Dataloading: 0.1323 s/iter. Inference: 0.1509 s/iter. Eval: 0.0020 s/iter. Total: 0.2854 s/iter. ETA=0:00:00 [09/27 04:51:37] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/27 04:51:37] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.560636 (0.000251 s / iter per device, on 8 devices) [09/27 04:51:37] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000133 s / iter per device, on 8 devices) [09/27 04:51:37] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/27 04:51:37] lb.evaluation.utils INFO: copypaste: Acc@1=76.06 [09/27 04:51:37] lb.evaluation.utils INFO: copypaste: Acc@5=93.17999999999999 [09/27 04:51:37] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 76.06000, better than last best score 75.96400 @ iteration 209999. [09/27 04:51:37] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/27 04:51:38] lb.utils.events INFO: eta: 7:05:06 iteration: 214999/375342 consumed_samples: 220160000 total_loss: 3.449 time: 0.3286 s/iter data_time: 0.2190 s/iter total_throughput: 3115.99 samples/s lr: 3.93e-04 [09/27 04:52:10] lb.utils.events INFO: eta: 7:00:52 iteration: 215099/375342 consumed_samples: 220262400 total_loss: 3.448 time: 0.3286 s/iter data_time: 0.2211 s/iter total_throughput: 3116.02 samples/s lr: 3.92e-04 [09/27 04:52:43] lb.utils.events INFO: eta: 6:57:23 iteration: 215199/375342 consumed_samples: 220364800 total_loss: 3.46 time: 0.3286 s/iter data_time: 0.2062 s/iter total_throughput: 3116.02 samples/s lr: 3.92e-04 [09/27 04:53:16] lb.utils.events INFO: eta: 6:56:48 iteration: 215299/375342 consumed_samples: 220467200 total_loss: 3.459 time: 0.3286 s/iter data_time: 0.2085 s/iter total_throughput: 3116.02 samples/s lr: 3.92e-04 [09/27 04:53:49] lb.utils.events INFO: eta: 6:57:28 iteration: 215399/375342 consumed_samples: 220569600 total_loss: 3.448 time: 0.3286 s/iter data_time: 0.2355 s/iter total_throughput: 3116.00 samples/s lr: 3.91e-04 [09/27 04:54:22] lb.utils.events INFO: eta: 7:00:18 iteration: 215499/375342 consumed_samples: 220672000 total_loss: 3.438 time: 0.3286 s/iter data_time: 0.2236 s/iter total_throughput: 3115.97 samples/s lr: 3.91e-04 [09/27 04:54:55] lb.utils.events INFO: eta: 6:59:19 iteration: 215599/375342 consumed_samples: 220774400 total_loss: 3.451 time: 0.3286 s/iter data_time: 0.2212 s/iter total_throughput: 3115.98 samples/s lr: 3.90e-04 [09/27 04:55:29] lb.utils.events INFO: eta: 6:55:50 iteration: 215699/375342 consumed_samples: 220876800 total_loss: 3.449 time: 0.3286 s/iter data_time: 0.2163 s/iter total_throughput: 3115.95 samples/s lr: 3.90e-04 [09/27 04:56:02] lb.utils.events INFO: eta: 6:55:14 iteration: 215799/375342 consumed_samples: 220979200 total_loss: 3.447 time: 0.3286 s/iter data_time: 0.2063 s/iter total_throughput: 3115.93 samples/s lr: 3.90e-04 [09/27 04:56:35] lb.utils.events INFO: eta: 6:53:13 iteration: 215899/375342 consumed_samples: 221081600 total_loss: 3.454 time: 0.3286 s/iter data_time: 0.2100 s/iter total_throughput: 3115.93 samples/s lr: 3.89e-04 [09/27 04:57:08] lb.utils.events INFO: eta: 6:50:32 iteration: 215999/375342 consumed_samples: 221184000 total_loss: 3.45 time: 0.3286 s/iter data_time: 0.2039 s/iter total_throughput: 3115.90 samples/s lr: 3.89e-04 [09/27 04:57:42] lb.utils.events INFO: eta: 6:50:47 iteration: 216099/375342 consumed_samples: 221286400 total_loss: 3.438 time: 0.3286 s/iter data_time: 0.2105 s/iter total_throughput: 3115.89 samples/s lr: 3.88e-04 [09/27 04:58:15] lb.utils.events INFO: eta: 6:49:51 iteration: 216199/375342 consumed_samples: 221388800 total_loss: 3.449 time: 0.3286 s/iter data_time: 0.2161 s/iter total_throughput: 3115.88 samples/s lr: 3.88e-04 [09/27 04:58:48] lb.utils.events INFO: eta: 6:49:11 iteration: 216299/375342 consumed_samples: 221491200 total_loss: 3.448 time: 0.3286 s/iter data_time: 0.2069 s/iter total_throughput: 3115.88 samples/s lr: 3.88e-04 [09/27 04:59:21] lb.utils.events INFO: eta: 6:48:09 iteration: 216399/375342 consumed_samples: 221593600 total_loss: 3.459 time: 0.3286 s/iter data_time: 0.2173 s/iter total_throughput: 3115.88 samples/s lr: 3.87e-04 [09/27 04:59:54] lb.utils.events INFO: eta: 6:46:30 iteration: 216499/375342 consumed_samples: 221696000 total_loss: 3.45 time: 0.3286 s/iter data_time: 0.2174 s/iter total_throughput: 3115.86 samples/s lr: 3.87e-04 [09/27 05:00:26] lb.utils.events INFO: eta: 6:46:17 iteration: 216599/375342 consumed_samples: 221798400 total_loss: 3.412 time: 0.3286 s/iter data_time: 0.2137 s/iter total_throughput: 3115.87 samples/s lr: 3.86e-04 [09/27 05:00:59] lb.utils.events INFO: eta: 6:46:14 iteration: 216699/375342 consumed_samples: 221900800 total_loss: 3.426 time: 0.3286 s/iter data_time: 0.2105 s/iter total_throughput: 3115.88 samples/s lr: 3.86e-04 [09/27 05:01:32] lb.utils.events INFO: eta: 6:47:10 iteration: 216799/375342 consumed_samples: 222003200 total_loss: 3.438 time: 0.3286 s/iter data_time: 0.2287 s/iter total_throughput: 3115.90 samples/s lr: 3.86e-04 [09/27 05:02:04] lb.utils.events INFO: eta: 6:47:45 iteration: 216899/375342 consumed_samples: 222105600 total_loss: 3.426 time: 0.3286 s/iter data_time: 0.2250 s/iter total_throughput: 3115.91 samples/s lr: 3.85e-04 [09/27 05:02:37] lb.utils.events INFO: eta: 6:49:23 iteration: 216999/375342 consumed_samples: 222208000 total_loss: 3.446 time: 0.3286 s/iter data_time: 0.2138 s/iter total_throughput: 3115.90 samples/s lr: 3.85e-04 [09/27 05:03:10] lb.utils.events INFO: eta: 6:50:29 iteration: 217099/375342 consumed_samples: 222310400 total_loss: 3.46 time: 0.3286 s/iter data_time: 0.2441 s/iter total_throughput: 3115.92 samples/s lr: 3.84e-04 [09/27 05:03:42] lb.utils.events INFO: eta: 6:52:51 iteration: 217199/375342 consumed_samples: 222412800 total_loss: 3.448 time: 0.3286 s/iter data_time: 0.2206 s/iter total_throughput: 3115.93 samples/s lr: 3.84e-04 [09/27 05:04:15] lb.utils.events INFO: eta: 6:58:16 iteration: 217299/375342 consumed_samples: 222515200 total_loss: 3.434 time: 0.3286 s/iter data_time: 0.2511 s/iter total_throughput: 3115.93 samples/s lr: 3.84e-04 [09/27 05:04:49] lb.utils.events INFO: eta: 7:00:53 iteration: 217399/375342 consumed_samples: 222617600 total_loss: 3.443 time: 0.3286 s/iter data_time: 0.2244 s/iter total_throughput: 3115.91 samples/s lr: 3.83e-04 [09/27 05:05:22] lb.utils.events INFO: eta: 7:08:21 iteration: 217499/375342 consumed_samples: 222720000 total_loss: 3.46 time: 0.3286 s/iter data_time: 0.2320 s/iter total_throughput: 3115.90 samples/s lr: 3.83e-04 [09/27 05:05:54] lb.utils.events INFO: eta: 7:18:04 iteration: 217599/375342 consumed_samples: 222822400 total_loss: 3.446 time: 0.3286 s/iter data_time: 0.2190 s/iter total_throughput: 3115.90 samples/s lr: 3.82e-04 [09/27 05:06:27] lb.utils.events INFO: eta: 7:23:45 iteration: 217699/375342 consumed_samples: 222924800 total_loss: 3.459 time: 0.3286 s/iter data_time: 0.2194 s/iter total_throughput: 3115.89 samples/s lr: 3.82e-04 [09/27 05:07:00] lb.utils.events INFO: eta: 7:21:36 iteration: 217799/375342 consumed_samples: 223027200 total_loss: 3.458 time: 0.3286 s/iter data_time: 0.2294 s/iter total_throughput: 3115.89 samples/s lr: 3.81e-04 [09/27 05:07:33] lb.utils.events INFO: eta: 8:06:43 iteration: 217899/375342 consumed_samples: 223129600 total_loss: 3.446 time: 0.3286 s/iter data_time: 0.2365 s/iter total_throughput: 3115.89 samples/s lr: 3.81e-04 [09/27 05:08:06] lb.utils.events INFO: eta: 8:54:23 iteration: 217999/375342 consumed_samples: 223232000 total_loss: 3.454 time: 0.3286 s/iter data_time: 0.2424 s/iter total_throughput: 3115.88 samples/s lr: 3.81e-04 [09/27 05:08:39] lb.utils.events INFO: eta: 9:26:32 iteration: 218099/375342 consumed_samples: 223334400 total_loss: 3.443 time: 0.3286 s/iter data_time: 0.2249 s/iter total_throughput: 3115.90 samples/s lr: 3.80e-04 [09/27 05:09:12] lb.utils.events INFO: eta: 8:06:54 iteration: 218199/375342 consumed_samples: 223436800 total_loss: 3.43 time: 0.3286 s/iter data_time: 0.2287 s/iter total_throughput: 3115.90 samples/s lr: 3.80e-04 [09/27 05:09:44] lb.utils.events INFO: eta: 7:46:57 iteration: 218299/375342 consumed_samples: 223539200 total_loss: 3.422 time: 0.3286 s/iter data_time: 0.2247 s/iter total_throughput: 3115.91 samples/s lr: 3.79e-04 [09/27 05:10:18] lb.utils.events INFO: eta: 7:26:30 iteration: 218399/375342 consumed_samples: 223641600 total_loss: 3.438 time: 0.3286 s/iter data_time: 0.2181 s/iter total_throughput: 3115.88 samples/s lr: 3.79e-04 [09/27 05:10:51] lb.utils.events INFO: eta: 7:12:44 iteration: 218499/375342 consumed_samples: 223744000 total_loss: 3.442 time: 0.3286 s/iter data_time: 0.2086 s/iter total_throughput: 3115.89 samples/s lr: 3.79e-04 [09/27 05:11:24] lb.utils.events INFO: eta: 7:07:04 iteration: 218599/375342 consumed_samples: 223846400 total_loss: 3.438 time: 0.3286 s/iter data_time: 0.2203 s/iter total_throughput: 3115.87 samples/s lr: 3.78e-04 [09/27 05:11:57] lb.utils.events INFO: eta: 7:05:52 iteration: 218699/375342 consumed_samples: 223948800 total_loss: 3.458 time: 0.3286 s/iter data_time: 0.2106 s/iter total_throughput: 3115.88 samples/s lr: 3.78e-04 [09/27 05:12:29] lb.utils.events INFO: eta: 7:03:56 iteration: 218799/375342 consumed_samples: 224051200 total_loss: 3.463 time: 0.3286 s/iter data_time: 0.2169 s/iter total_throughput: 3115.90 samples/s lr: 3.77e-04 [09/27 05:13:01] lb.utils.events INFO: eta: 6:58:46 iteration: 218899/375342 consumed_samples: 224153600 total_loss: 3.43 time: 0.3286 s/iter data_time: 0.2242 s/iter total_throughput: 3115.92 samples/s lr: 3.77e-04 [09/27 05:13:34] lb.utils.events INFO: eta: 6:59:15 iteration: 218999/375342 consumed_samples: 224256000 total_loss: 3.408 time: 0.3286 s/iter data_time: 0.2207 s/iter total_throughput: 3115.93 samples/s lr: 3.77e-04 [09/27 05:14:07] lb.utils.events INFO: eta: 7:00:25 iteration: 219099/375342 consumed_samples: 224358400 total_loss: 3.411 time: 0.3286 s/iter data_time: 0.2376 s/iter total_throughput: 3115.93 samples/s lr: 3.76e-04 [09/27 05:14:40] lb.utils.events INFO: eta: 7:03:09 iteration: 219199/375342 consumed_samples: 224460800 total_loss: 3.447 time: 0.3286 s/iter data_time: 0.2149 s/iter total_throughput: 3115.94 samples/s lr: 3.76e-04 [09/27 05:15:12] lb.utils.events INFO: eta: 7:01:20 iteration: 219299/375342 consumed_samples: 224563200 total_loss: 3.447 time: 0.3286 s/iter data_time: 0.2290 s/iter total_throughput: 3115.94 samples/s lr: 3.75e-04 [09/27 05:15:45] lb.utils.events INFO: eta: 7:03:27 iteration: 219399/375342 consumed_samples: 224665600 total_loss: 3.432 time: 0.3286 s/iter data_time: 0.2220 s/iter total_throughput: 3115.93 samples/s lr: 3.75e-04 [09/27 05:16:18] lb.utils.events INFO: eta: 7:03:38 iteration: 219499/375342 consumed_samples: 224768000 total_loss: 3.434 time: 0.3286 s/iter data_time: 0.2098 s/iter total_throughput: 3115.93 samples/s lr: 3.75e-04 [09/27 05:16:52] lb.utils.events INFO: eta: 6:58:07 iteration: 219599/375342 consumed_samples: 224870400 total_loss: 3.432 time: 0.3286 s/iter data_time: 0.2114 s/iter total_throughput: 3115.92 samples/s lr: 3.74e-04 [09/27 05:17:24] lb.utils.events INFO: eta: 6:58:19 iteration: 219699/375342 consumed_samples: 224972800 total_loss: 3.433 time: 0.3286 s/iter data_time: 0.2053 s/iter total_throughput: 3115.93 samples/s lr: 3.74e-04 [09/27 05:17:57] lb.utils.events INFO: eta: 6:56:42 iteration: 219799/375342 consumed_samples: 225075200 total_loss: 3.455 time: 0.3286 s/iter data_time: 0.2232 s/iter total_throughput: 3115.94 samples/s lr: 3.73e-04 [09/27 05:18:29] lb.utils.events INFO: eta: 6:55:38 iteration: 219899/375342 consumed_samples: 225177600 total_loss: 3.432 time: 0.3286 s/iter data_time: 0.2173 s/iter total_throughput: 3115.95 samples/s lr: 3.73e-04 [09/27 05:19:02] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0219999 [09/27 05:19:03] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/27 05:19:03] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/27 05:19:07] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0889 s/iter. Inference: 0.1486 s/iter. Eval: 0.0021 s/iter. Total: 0.2396 s/iter. ETA=0:00:08 [09/27 05:19:12] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1421 s/iter. Inference: 0.1496 s/iter. Eval: 0.0020 s/iter. Total: 0.2937 s/iter. ETA=0:00:05 [09/27 05:19:17] lb.evaluation.evaluator INFO: Inference done 49152/50000. Dataloading: 0.1297 s/iter. Inference: 0.1487 s/iter. Eval: 0.0020 s/iter. Total: 0.2805 s/iter. ETA=0:00:00 [09/27 05:19:18] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/27 05:19:18] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.470601 (0.000249 s / iter per device, on 8 devices) [09/27 05:19:18] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000132 s / iter per device, on 8 devices) [09/27 05:19:18] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/27 05:19:18] lb.evaluation.utils INFO: copypaste: Acc@1=76.214 [09/27 05:19:18] lb.evaluation.utils INFO: copypaste: Acc@5=93.196 [09/27 05:19:18] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 76.21400, better than last best score 76.06000 @ iteration 214999. [09/27 05:19:18] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/27 05:19:18] lb.utils.events INFO: eta: 6:52:35 iteration: 219999/375342 consumed_samples: 225280000 total_loss: 3.425 time: 0.3286 s/iter data_time: 0.2286 s/iter total_throughput: 3115.96 samples/s lr: 3.73e-04 [09/27 05:19:49] lb.utils.events INFO: eta: 6:45:00 iteration: 220099/375342 consumed_samples: 225382400 total_loss: 3.426 time: 0.3286 s/iter data_time: 0.2134 s/iter total_throughput: 3116.04 samples/s lr: 3.72e-04 [09/27 05:20:22] lb.utils.events INFO: eta: 6:44:33 iteration: 220199/375342 consumed_samples: 225484800 total_loss: 3.428 time: 0.3286 s/iter data_time: 0.2291 s/iter total_throughput: 3116.04 samples/s lr: 3.72e-04 [09/27 05:20:55] lb.utils.events INFO: eta: 6:43:32 iteration: 220299/375342 consumed_samples: 225587200 total_loss: 3.414 time: 0.3286 s/iter data_time: 0.2183 s/iter total_throughput: 3116.03 samples/s lr: 3.71e-04 [09/27 05:21:28] lb.utils.events INFO: eta: 6:43:26 iteration: 220399/375342 consumed_samples: 225689600 total_loss: 3.413 time: 0.3286 s/iter data_time: 0.2105 s/iter total_throughput: 3116.04 samples/s lr: 3.71e-04 [09/27 05:22:01] lb.utils.events INFO: eta: 6:45:08 iteration: 220499/375342 consumed_samples: 225792000 total_loss: 3.429 time: 0.3286 s/iter data_time: 0.2140 s/iter total_throughput: 3116.06 samples/s lr: 3.71e-04 [09/27 05:22:33] lb.utils.events INFO: eta: 6:45:42 iteration: 220599/375342 consumed_samples: 225894400 total_loss: 3.445 time: 0.3286 s/iter data_time: 0.1984 s/iter total_throughput: 3116.07 samples/s lr: 3.70e-04 [09/27 05:23:06] lb.utils.events INFO: eta: 6:46:40 iteration: 220699/375342 consumed_samples: 225996800 total_loss: 3.424 time: 0.3286 s/iter data_time: 0.2212 s/iter total_throughput: 3116.06 samples/s lr: 3.70e-04 [09/27 05:23:39] lb.utils.events INFO: eta: 6:47:20 iteration: 220799/375342 consumed_samples: 226099200 total_loss: 3.398 time: 0.3286 s/iter data_time: 0.2253 s/iter total_throughput: 3116.08 samples/s lr: 3.69e-04 [09/27 05:24:11] lb.utils.events INFO: eta: 6:47:48 iteration: 220899/375342 consumed_samples: 226201600 total_loss: 3.404 time: 0.3286 s/iter data_time: 0.2223 s/iter total_throughput: 3116.08 samples/s lr: 3.69e-04 [09/27 05:24:44] lb.utils.events INFO: eta: 6:49:09 iteration: 220999/375342 consumed_samples: 226304000 total_loss: 3.407 time: 0.3286 s/iter data_time: 0.2174 s/iter total_throughput: 3116.11 samples/s lr: 3.69e-04 [09/27 05:25:17] lb.utils.events INFO: eta: 6:52:04 iteration: 221099/375342 consumed_samples: 226406400 total_loss: 3.412 time: 0.3286 s/iter data_time: 0.2335 s/iter total_throughput: 3116.11 samples/s lr: 3.68e-04 [09/27 05:25:49] lb.utils.events INFO: eta: 6:52:47 iteration: 221199/375342 consumed_samples: 226508800 total_loss: 3.408 time: 0.3286 s/iter data_time: 0.2215 s/iter total_throughput: 3116.12 samples/s lr: 3.68e-04 [09/27 05:26:22] lb.utils.events INFO: eta: 6:55:53 iteration: 221299/375342 consumed_samples: 226611200 total_loss: 3.406 time: 0.3286 s/iter data_time: 0.2334 s/iter total_throughput: 3116.13 samples/s lr: 3.68e-04 [09/27 05:26:55] lb.utils.events INFO: eta: 6:55:30 iteration: 221399/375342 consumed_samples: 226713600 total_loss: 3.433 time: 0.3286 s/iter data_time: 0.2461 s/iter total_throughput: 3116.12 samples/s lr: 3.67e-04 [09/27 05:27:28] lb.utils.events INFO: eta: 6:55:44 iteration: 221499/375342 consumed_samples: 226816000 total_loss: 3.435 time: 0.3286 s/iter data_time: 0.2271 s/iter total_throughput: 3116.12 samples/s lr: 3.67e-04 [09/27 05:28:00] lb.utils.events INFO: eta: 7:00:19 iteration: 221599/375342 consumed_samples: 226918400 total_loss: 3.423 time: 0.3286 s/iter data_time: 0.2165 s/iter total_throughput: 3116.13 samples/s lr: 3.66e-04 [09/27 05:28:33] lb.utils.events INFO: eta: 6:56:31 iteration: 221699/375342 consumed_samples: 227020800 total_loss: 3.413 time: 0.3286 s/iter data_time: 0.2017 s/iter total_throughput: 3116.14 samples/s lr: 3.66e-04 [09/27 05:29:06] lb.utils.events INFO: eta: 6:54:22 iteration: 221799/375342 consumed_samples: 227123200 total_loss: 3.411 time: 0.3286 s/iter data_time: 0.2188 s/iter total_throughput: 3116.15 samples/s lr: 3.66e-04 [09/27 05:29:38] lb.utils.events INFO: eta: 6:53:02 iteration: 221899/375342 consumed_samples: 227225600 total_loss: 3.419 time: 0.3286 s/iter data_time: 0.2177 s/iter total_throughput: 3116.16 samples/s lr: 3.65e-04 [09/27 05:30:11] lb.utils.events INFO: eta: 6:51:13 iteration: 221999/375342 consumed_samples: 227328000 total_loss: 3.408 time: 0.3286 s/iter data_time: 0.2308 s/iter total_throughput: 3116.16 samples/s lr: 3.65e-04 [09/27 05:30:44] lb.utils.events INFO: eta: 6:52:57 iteration: 222099/375342 consumed_samples: 227430400 total_loss: 3.419 time: 0.3286 s/iter data_time: 0.2187 s/iter total_throughput: 3116.18 samples/s lr: 3.64e-04 [09/27 05:31:16] lb.utils.events INFO: eta: 6:50:19 iteration: 222199/375342 consumed_samples: 227532800 total_loss: 3.42 time: 0.3286 s/iter data_time: 0.2183 s/iter total_throughput: 3116.18 samples/s lr: 3.64e-04 [09/27 05:31:49] lb.utils.events INFO: eta: 6:43:11 iteration: 222299/375342 consumed_samples: 227635200 total_loss: 3.434 time: 0.3286 s/iter data_time: 0.2148 s/iter total_throughput: 3116.18 samples/s lr: 3.64e-04 [09/27 05:32:22] lb.utils.events INFO: eta: 6:41:50 iteration: 222399/375342 consumed_samples: 227737600 total_loss: 3.45 time: 0.3286 s/iter data_time: 0.2039 s/iter total_throughput: 3116.20 samples/s lr: 3.63e-04 [09/27 05:32:54] lb.utils.events INFO: eta: 6:39:03 iteration: 222499/375342 consumed_samples: 227840000 total_loss: 3.426 time: 0.3286 s/iter data_time: 0.2044 s/iter total_throughput: 3116.22 samples/s lr: 3.63e-04 [09/27 05:33:27] lb.utils.events INFO: eta: 6:35:50 iteration: 222599/375342 consumed_samples: 227942400 total_loss: 3.4 time: 0.3286 s/iter data_time: 0.1986 s/iter total_throughput: 3116.23 samples/s lr: 3.62e-04 [09/27 05:33:59] lb.utils.events INFO: eta: 6:36:00 iteration: 222699/375342 consumed_samples: 228044800 total_loss: 3.421 time: 0.3286 s/iter data_time: 0.2028 s/iter total_throughput: 3116.25 samples/s lr: 3.62e-04 [09/27 05:34:31] lb.utils.events INFO: eta: 6:35:45 iteration: 222799/375342 consumed_samples: 228147200 total_loss: 3.435 time: 0.3286 s/iter data_time: 0.2072 s/iter total_throughput: 3116.27 samples/s lr: 3.62e-04 [09/27 05:35:04] lb.utils.events INFO: eta: 6:34:38 iteration: 222899/375342 consumed_samples: 228249600 total_loss: 3.428 time: 0.3286 s/iter data_time: 0.2209 s/iter total_throughput: 3116.29 samples/s lr: 3.61e-04 [09/27 05:35:37] lb.utils.events INFO: eta: 6:34:28 iteration: 222999/375342 consumed_samples: 228352000 total_loss: 3.421 time: 0.3286 s/iter data_time: 0.2274 s/iter total_throughput: 3116.30 samples/s lr: 3.61e-04 [09/27 05:36:09] lb.utils.events INFO: eta: 6:32:59 iteration: 223099/375342 consumed_samples: 228454400 total_loss: 3.41 time: 0.3286 s/iter data_time: 0.2220 s/iter total_throughput: 3116.32 samples/s lr: 3.60e-04 [09/27 05:36:41] lb.utils.events INFO: eta: 6:34:30 iteration: 223199/375342 consumed_samples: 228556800 total_loss: 3.44 time: 0.3286 s/iter data_time: 0.2216 s/iter total_throughput: 3116.34 samples/s lr: 3.60e-04 [09/27 05:37:14] lb.utils.events INFO: eta: 6:35:03 iteration: 223299/375342 consumed_samples: 228659200 total_loss: 3.431 time: 0.3286 s/iter data_time: 0.2073 s/iter total_throughput: 3116.34 samples/s lr: 3.60e-04 [09/27 05:37:48] lb.utils.events INFO: eta: 6:34:38 iteration: 223399/375342 consumed_samples: 228761600 total_loss: 3.428 time: 0.3286 s/iter data_time: 0.2068 s/iter total_throughput: 3116.32 samples/s lr: 3.59e-04 [09/27 05:38:20] lb.utils.events INFO: eta: 6:34:32 iteration: 223499/375342 consumed_samples: 228864000 total_loss: 3.437 time: 0.3286 s/iter data_time: 0.2134 s/iter total_throughput: 3116.34 samples/s lr: 3.59e-04 [09/27 05:38:52] lb.utils.events INFO: eta: 6:34:55 iteration: 223599/375342 consumed_samples: 228966400 total_loss: 3.44 time: 0.3286 s/iter data_time: 0.1966 s/iter total_throughput: 3116.36 samples/s lr: 3.58e-04 [09/27 05:39:25] lb.utils.events INFO: eta: 6:34:03 iteration: 223699/375342 consumed_samples: 229068800 total_loss: 3.437 time: 0.3286 s/iter data_time: 0.2000 s/iter total_throughput: 3116.38 samples/s lr: 3.58e-04 [09/27 05:39:57] lb.utils.events INFO: eta: 6:35:07 iteration: 223799/375342 consumed_samples: 229171200 total_loss: 3.434 time: 0.3286 s/iter data_time: 0.2141 s/iter total_throughput: 3116.40 samples/s lr: 3.58e-04 [09/27 05:40:30] lb.utils.events INFO: eta: 6:33:17 iteration: 223899/375342 consumed_samples: 229273600 total_loss: 3.453 time: 0.3286 s/iter data_time: 0.1902 s/iter total_throughput: 3116.41 samples/s lr: 3.57e-04 [09/27 05:41:02] lb.utils.events INFO: eta: 6:32:57 iteration: 223999/375342 consumed_samples: 229376000 total_loss: 3.423 time: 0.3286 s/iter data_time: 0.2237 s/iter total_throughput: 3116.44 samples/s lr: 3.57e-04 [09/27 05:41:35] lb.utils.events INFO: eta: 6:33:39 iteration: 224099/375342 consumed_samples: 229478400 total_loss: 3.414 time: 0.3286 s/iter data_time: 0.2149 s/iter total_throughput: 3116.45 samples/s lr: 3.56e-04 [09/27 05:42:08] lb.utils.events INFO: eta: 6:33:34 iteration: 224199/375342 consumed_samples: 229580800 total_loss: 3.406 time: 0.3286 s/iter data_time: 0.2410 s/iter total_throughput: 3116.45 samples/s lr: 3.56e-04 [09/27 05:42:41] lb.utils.events INFO: eta: 6:33:08 iteration: 224299/375342 consumed_samples: 229683200 total_loss: 3.401 time: 0.3286 s/iter data_time: 0.2122 s/iter total_throughput: 3116.43 samples/s lr: 3.56e-04 [09/27 05:43:14] lb.utils.events INFO: eta: 6:34:05 iteration: 224399/375342 consumed_samples: 229785600 total_loss: 3.414 time: 0.3286 s/iter data_time: 0.2303 s/iter total_throughput: 3116.42 samples/s lr: 3.55e-04 [09/27 05:43:47] lb.utils.events INFO: eta: 6:34:37 iteration: 224499/375342 consumed_samples: 229888000 total_loss: 3.429 time: 0.3286 s/iter data_time: 0.2107 s/iter total_throughput: 3116.42 samples/s lr: 3.55e-04 [09/27 05:44:20] lb.utils.events INFO: eta: 6:32:42 iteration: 224599/375342 consumed_samples: 229990400 total_loss: 3.428 time: 0.3286 s/iter data_time: 0.2058 s/iter total_throughput: 3116.39 samples/s lr: 3.54e-04 [09/27 05:44:53] lb.utils.events INFO: eta: 6:32:22 iteration: 224699/375342 consumed_samples: 230092800 total_loss: 3.404 time: 0.3286 s/iter data_time: 0.2136 s/iter total_throughput: 3116.39 samples/s lr: 3.54e-04 [09/27 05:45:26] lb.utils.events INFO: eta: 6:31:50 iteration: 224799/375342 consumed_samples: 230195200 total_loss: 3.4 time: 0.3286 s/iter data_time: 0.2367 s/iter total_throughput: 3116.40 samples/s lr: 3.54e-04 [09/27 05:45:59] lb.utils.events INFO: eta: 6:37:13 iteration: 224899/375342 consumed_samples: 230297600 total_loss: 3.418 time: 0.3286 s/iter data_time: 0.2336 s/iter total_throughput: 3116.38 samples/s lr: 3.53e-04 [09/27 05:46:33] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0224999 [09/27 05:46:33] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/27 05:46:33] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/27 05:46:37] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0883 s/iter. Inference: 0.1514 s/iter. Eval: 0.0020 s/iter. Total: 0.2418 s/iter. ETA=0:00:08 [09/27 05:46:43] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1464 s/iter. Inference: 0.1504 s/iter. Eval: 0.0020 s/iter. Total: 0.2989 s/iter. ETA=0:00:05 [09/27 05:46:48] lb.evaluation.evaluator INFO: Inference done 50000/50000. Dataloading: 0.1336 s/iter. Inference: 0.1506 s/iter. Eval: 0.0020 s/iter. Total: 0.2863 s/iter. ETA=0:00:00 [09/27 05:46:48] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/27 05:46:48] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.601850 (0.000252 s / iter per device, on 8 devices) [09/27 05:46:48] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000133 s / iter per device, on 8 devices) [09/27 05:46:48] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/27 05:46:48] lb.evaluation.utils INFO: copypaste: Acc@1=76.466 [09/27 05:46:48] lb.evaluation.utils INFO: copypaste: Acc@5=93.274 [09/27 05:46:48] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 76.46600, better than last best score 76.21400 @ iteration 219999. [09/27 05:46:48] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/27 05:46:49] lb.utils.events INFO: eta: 6:35:12 iteration: 224999/375342 consumed_samples: 230400000 total_loss: 3.416 time: 0.3286 s/iter data_time: 0.2116 s/iter total_throughput: 3116.36 samples/s lr: 3.53e-04 [09/27 05:47:20] lb.utils.events INFO: eta: 6:36:49 iteration: 225099/375342 consumed_samples: 230502400 total_loss: 3.404 time: 0.3286 s/iter data_time: 0.2456 s/iter total_throughput: 3116.43 samples/s lr: 3.52e-04 [09/27 05:47:53] lb.utils.events INFO: eta: 6:37:36 iteration: 225199/375342 consumed_samples: 230604800 total_loss: 3.391 time: 0.3286 s/iter data_time: 0.2514 s/iter total_throughput: 3116.42 samples/s lr: 3.52e-04 [09/27 05:48:26] lb.utils.events INFO: eta: 6:45:05 iteration: 225299/375342 consumed_samples: 230707200 total_loss: 3.426 time: 0.3286 s/iter data_time: 0.2270 s/iter total_throughput: 3116.42 samples/s lr: 3.52e-04 [09/27 05:48:59] lb.utils.events INFO: eta: 6:41:35 iteration: 225399/375342 consumed_samples: 230809600 total_loss: 3.412 time: 0.3286 s/iter data_time: 0.2037 s/iter total_throughput: 3116.41 samples/s lr: 3.51e-04 [09/27 05:49:33] lb.utils.events INFO: eta: 6:43:59 iteration: 225499/375342 consumed_samples: 230912000 total_loss: 3.402 time: 0.3286 s/iter data_time: 0.2187 s/iter total_throughput: 3116.39 samples/s lr: 3.51e-04 [09/27 05:50:06] lb.utils.events INFO: eta: 6:54:26 iteration: 225599/375342 consumed_samples: 231014400 total_loss: 3.414 time: 0.3286 s/iter data_time: 0.2379 s/iter total_throughput: 3116.38 samples/s lr: 3.50e-04 [09/27 05:50:39] lb.utils.events INFO: eta: 6:59:38 iteration: 225699/375342 consumed_samples: 231116800 total_loss: 3.389 time: 0.3286 s/iter data_time: 0.2238 s/iter total_throughput: 3116.38 samples/s lr: 3.50e-04 [09/27 05:51:12] lb.utils.events INFO: eta: 7:23:02 iteration: 225799/375342 consumed_samples: 231219200 total_loss: 3.407 time: 0.3286 s/iter data_time: 0.2435 s/iter total_throughput: 3116.37 samples/s lr: 3.50e-04 [09/27 05:51:45] lb.utils.events INFO: eta: 7:24:16 iteration: 225899/375342 consumed_samples: 231321600 total_loss: 3.426 time: 0.3286 s/iter data_time: 0.2278 s/iter total_throughput: 3116.37 samples/s lr: 3.49e-04 [09/27 05:52:18] lb.utils.events INFO: eta: 7:49:24 iteration: 225999/375342 consumed_samples: 231424000 total_loss: 3.425 time: 0.3286 s/iter data_time: 0.2211 s/iter total_throughput: 3116.36 samples/s lr: 3.49e-04 [09/27 05:52:51] lb.utils.events INFO: eta: 7:07:14 iteration: 226099/375342 consumed_samples: 231526400 total_loss: 3.423 time: 0.3286 s/iter data_time: 0.2428 s/iter total_throughput: 3116.35 samples/s lr: 3.49e-04 [09/27 05:53:24] lb.utils.events INFO: eta: 6:56:55 iteration: 226199/375342 consumed_samples: 231628800 total_loss: 3.407 time: 0.3286 s/iter data_time: 0.2278 s/iter total_throughput: 3116.35 samples/s lr: 3.48e-04 [09/27 05:53:57] lb.utils.events INFO: eta: 6:45:36 iteration: 226299/375342 consumed_samples: 231731200 total_loss: 3.406 time: 0.3286 s/iter data_time: 0.2107 s/iter total_throughput: 3116.34 samples/s lr: 3.48e-04 [09/27 05:54:30] lb.utils.events INFO: eta: 6:49:03 iteration: 226399/375342 consumed_samples: 231833600 total_loss: 3.418 time: 0.3286 s/iter data_time: 0.2301 s/iter total_throughput: 3116.33 samples/s lr: 3.47e-04 [09/27 05:55:03] lb.utils.events INFO: eta: 6:53:23 iteration: 226499/375342 consumed_samples: 231936000 total_loss: 3.43 time: 0.3286 s/iter data_time: 0.2308 s/iter total_throughput: 3116.32 samples/s lr: 3.47e-04 [09/27 05:55:36] lb.utils.events INFO: eta: 6:52:12 iteration: 226599/375342 consumed_samples: 232038400 total_loss: 3.412 time: 0.3286 s/iter data_time: 0.2217 s/iter total_throughput: 3116.30 samples/s lr: 3.47e-04 [09/27 05:56:09] lb.utils.events INFO: eta: 6:54:49 iteration: 226699/375342 consumed_samples: 232140800 total_loss: 3.382 time: 0.3286 s/iter data_time: 0.2164 s/iter total_throughput: 3116.31 samples/s lr: 3.46e-04 [09/27 05:56:43] lb.utils.events INFO: eta: 6:43:48 iteration: 226799/375342 consumed_samples: 232243200 total_loss: 3.392 time: 0.3286 s/iter data_time: 0.2179 s/iter total_throughput: 3116.28 samples/s lr: 3.46e-04 [09/27 05:57:16] lb.utils.events INFO: eta: 6:39:23 iteration: 226899/375342 consumed_samples: 232345600 total_loss: 3.393 time: 0.3286 s/iter data_time: 0.2273 s/iter total_throughput: 3116.25 samples/s lr: 3.45e-04 [09/27 05:57:49] lb.utils.events INFO: eta: 6:37:41 iteration: 226999/375342 consumed_samples: 232448000 total_loss: 3.393 time: 0.3286 s/iter data_time: 0.2149 s/iter total_throughput: 3116.24 samples/s lr: 3.45e-04 [09/27 05:58:22] lb.utils.events INFO: eta: 6:36:38 iteration: 227099/375342 consumed_samples: 232550400 total_loss: 3.391 time: 0.3286 s/iter data_time: 0.2281 s/iter total_throughput: 3116.24 samples/s lr: 3.45e-04 [09/27 05:58:55] lb.utils.events INFO: eta: 6:31:30 iteration: 227199/375342 consumed_samples: 232652800 total_loss: 3.412 time: 0.3286 s/iter data_time: 0.2072 s/iter total_throughput: 3116.23 samples/s lr: 3.44e-04 [09/27 05:59:28] lb.utils.events INFO: eta: 6:32:24 iteration: 227299/375342 consumed_samples: 232755200 total_loss: 3.411 time: 0.3286 s/iter data_time: 0.2108 s/iter total_throughput: 3116.22 samples/s lr: 3.44e-04 [09/27 06:00:01] lb.utils.events INFO: eta: 6:29:33 iteration: 227399/375342 consumed_samples: 232857600 total_loss: 3.406 time: 0.3286 s/iter data_time: 0.2076 s/iter total_throughput: 3116.21 samples/s lr: 3.43e-04 [09/27 06:00:35] lb.utils.events INFO: eta: 6:25:14 iteration: 227499/375342 consumed_samples: 232960000 total_loss: 3.402 time: 0.3286 s/iter data_time: 0.2087 s/iter total_throughput: 3116.19 samples/s lr: 3.43e-04 [09/27 06:01:07] lb.utils.events INFO: eta: 6:22:44 iteration: 227599/375342 consumed_samples: 233062400 total_loss: 3.396 time: 0.3286 s/iter data_time: 0.2071 s/iter total_throughput: 3116.21 samples/s lr: 3.43e-04 [09/27 06:01:40] lb.utils.events INFO: eta: 6:21:55 iteration: 227699/375342 consumed_samples: 233164800 total_loss: 3.396 time: 0.3286 s/iter data_time: 0.2208 s/iter total_throughput: 3116.21 samples/s lr: 3.42e-04 [09/27 06:02:13] lb.utils.events INFO: eta: 6:24:26 iteration: 227799/375342 consumed_samples: 233267200 total_loss: 3.364 time: 0.3286 s/iter data_time: 0.2086 s/iter total_throughput: 3116.19 samples/s lr: 3.42e-04 [09/27 06:02:46] lb.utils.events INFO: eta: 6:25:17 iteration: 227899/375342 consumed_samples: 233369600 total_loss: 3.382 time: 0.3286 s/iter data_time: 0.2264 s/iter total_throughput: 3116.19 samples/s lr: 3.41e-04 [09/27 06:03:19] lb.utils.events INFO: eta: 6:25:25 iteration: 227999/375342 consumed_samples: 233472000 total_loss: 3.411 time: 0.3286 s/iter data_time: 0.2257 s/iter total_throughput: 3116.19 samples/s lr: 3.41e-04 [09/27 06:03:53] lb.utils.events INFO: eta: 6:25:41 iteration: 228099/375342 consumed_samples: 233574400 total_loss: 3.393 time: 0.3286 s/iter data_time: 0.2286 s/iter total_throughput: 3116.17 samples/s lr: 3.41e-04 [09/27 06:04:26] lb.utils.events INFO: eta: 6:26:26 iteration: 228199/375342 consumed_samples: 233676800 total_loss: 3.375 time: 0.3286 s/iter data_time: 0.2092 s/iter total_throughput: 3116.16 samples/s lr: 3.40e-04 [09/27 06:04:59] lb.utils.events INFO: eta: 6:27:13 iteration: 228299/375342 consumed_samples: 233779200 total_loss: 3.379 time: 0.3286 s/iter data_time: 0.2202 s/iter total_throughput: 3116.16 samples/s lr: 3.40e-04 [09/27 06:05:32] lb.utils.events INFO: eta: 6:31:21 iteration: 228399/375342 consumed_samples: 233881600 total_loss: 3.399 time: 0.3286 s/iter data_time: 0.2418 s/iter total_throughput: 3116.14 samples/s lr: 3.40e-04 [09/27 06:06:05] lb.utils.events INFO: eta: 6:32:33 iteration: 228499/375342 consumed_samples: 233984000 total_loss: 3.411 time: 0.3286 s/iter data_time: 0.2205 s/iter total_throughput: 3116.13 samples/s lr: 3.39e-04 [09/27 06:06:38] lb.utils.events INFO: eta: 6:35:05 iteration: 228599/375342 consumed_samples: 234086400 total_loss: 3.422 time: 0.3286 s/iter data_time: 0.2249 s/iter total_throughput: 3116.12 samples/s lr: 3.39e-04 [09/27 06:07:11] lb.utils.events INFO: eta: 6:36:19 iteration: 228699/375342 consumed_samples: 234188800 total_loss: 3.401 time: 0.3286 s/iter data_time: 0.2218 s/iter total_throughput: 3116.13 samples/s lr: 3.38e-04 [09/27 06:07:44] lb.utils.events INFO: eta: 6:37:35 iteration: 228799/375342 consumed_samples: 234291200 total_loss: 3.392 time: 0.3286 s/iter data_time: 0.2296 s/iter total_throughput: 3116.11 samples/s lr: 3.38e-04 [09/27 06:08:17] lb.utils.events INFO: eta: 6:38:52 iteration: 228899/375342 consumed_samples: 234393600 total_loss: 3.391 time: 0.3286 s/iter data_time: 0.2277 s/iter total_throughput: 3116.11 samples/s lr: 3.38e-04 [09/27 06:08:50] lb.utils.events INFO: eta: 6:34:20 iteration: 228999/375342 consumed_samples: 234496000 total_loss: 3.394 time: 0.3286 s/iter data_time: 0.2101 s/iter total_throughput: 3116.10 samples/s lr: 3.37e-04 [09/27 06:09:23] lb.utils.events INFO: eta: 6:29:17 iteration: 229099/375342 consumed_samples: 234598400 total_loss: 3.394 time: 0.3286 s/iter data_time: 0.2047 s/iter total_throughput: 3116.09 samples/s lr: 3.37e-04 [09/27 06:09:56] lb.utils.events INFO: eta: 6:27:51 iteration: 229199/375342 consumed_samples: 234700800 total_loss: 3.391 time: 0.3286 s/iter data_time: 0.2158 s/iter total_throughput: 3116.08 samples/s lr: 3.36e-04 [09/27 06:10:29] lb.utils.events INFO: eta: 6:27:25 iteration: 229299/375342 consumed_samples: 234803200 total_loss: 3.388 time: 0.3286 s/iter data_time: 0.2282 s/iter total_throughput: 3116.07 samples/s lr: 3.36e-04 [09/27 06:11:03] lb.utils.events INFO: eta: 6:24:07 iteration: 229399/375342 consumed_samples: 234905600 total_loss: 3.379 time: 0.3286 s/iter data_time: 0.2142 s/iter total_throughput: 3116.02 samples/s lr: 3.36e-04 [09/27 06:11:36] lb.utils.events INFO: eta: 6:25:29 iteration: 229499/375342 consumed_samples: 235008000 total_loss: 3.379 time: 0.3286 s/iter data_time: 0.1963 s/iter total_throughput: 3116.02 samples/s lr: 3.35e-04 [09/27 06:12:09] lb.utils.events INFO: eta: 6:24:11 iteration: 229599/375342 consumed_samples: 235110400 total_loss: 3.387 time: 0.3286 s/iter data_time: 0.1994 s/iter total_throughput: 3116.02 samples/s lr: 3.35e-04 [09/27 06:12:42] lb.utils.events INFO: eta: 6:25:06 iteration: 229699/375342 consumed_samples: 235212800 total_loss: 3.399 time: 0.3286 s/iter data_time: 0.2442 s/iter total_throughput: 3116.03 samples/s lr: 3.34e-04 [09/27 06:13:15] lb.utils.events INFO: eta: 6:22:29 iteration: 229799/375342 consumed_samples: 235315200 total_loss: 3.397 time: 0.3286 s/iter data_time: 0.2277 s/iter total_throughput: 3116.01 samples/s lr: 3.34e-04 [09/27 06:13:48] lb.utils.events INFO: eta: 6:22:37 iteration: 229899/375342 consumed_samples: 235417600 total_loss: 3.402 time: 0.3286 s/iter data_time: 0.2319 s/iter total_throughput: 3116.01 samples/s lr: 3.34e-04 [09/27 06:14:21] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0229999 [09/27 06:14:22] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/27 06:14:22] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/27 06:14:26] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0888 s/iter. Inference: 0.1510 s/iter. Eval: 0.0020 s/iter. Total: 0.2417 s/iter. ETA=0:00:08 [09/27 06:14:32] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1517 s/iter. Inference: 0.1500 s/iter. Eval: 0.0019 s/iter. Total: 0.3036 s/iter. ETA=0:00:05 [09/27 06:14:37] lb.evaluation.evaluator INFO: Inference done 49152/50000. Dataloading: 0.1353 s/iter. Inference: 0.1491 s/iter. Eval: 0.0019 s/iter. Total: 0.2864 s/iter. ETA=0:00:00 [09/27 06:14:37] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/27 06:14:37] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.762933 (0.000255 s / iter per device, on 8 devices) [09/27 06:14:37] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000132 s / iter per device, on 8 devices) [09/27 06:14:37] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/27 06:14:37] lb.evaluation.utils INFO: copypaste: Acc@1=76.524 [09/27 06:14:37] lb.evaluation.utils INFO: copypaste: Acc@5=93.28800000000001 [09/27 06:14:37] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 76.52400, better than last best score 76.46600 @ iteration 224999. [09/27 06:14:37] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/27 06:14:38] lb.utils.events INFO: eta: 6:24:43 iteration: 229999/375342 consumed_samples: 235520000 total_loss: 3.414 time: 0.3286 s/iter data_time: 0.2388 s/iter total_throughput: 3116.01 samples/s lr: 3.33e-04 [09/27 06:15:09] lb.utils.events INFO: eta: 6:28:57 iteration: 230099/375342 consumed_samples: 235622400 total_loss: 3.409 time: 0.3286 s/iter data_time: 0.2525 s/iter total_throughput: 3116.07 samples/s lr: 3.33e-04 [09/27 06:15:43] lb.utils.events INFO: eta: 6:29:54 iteration: 230199/375342 consumed_samples: 235724800 total_loss: 3.396 time: 0.3286 s/iter data_time: 0.2185 s/iter total_throughput: 3116.04 samples/s lr: 3.32e-04 [09/27 06:16:16] lb.utils.events INFO: eta: 6:24:24 iteration: 230299/375342 consumed_samples: 235827200 total_loss: 3.391 time: 0.3286 s/iter data_time: 0.1930 s/iter total_throughput: 3116.05 samples/s lr: 3.32e-04 [09/27 06:16:49] lb.utils.events INFO: eta: 6:23:10 iteration: 230399/375342 consumed_samples: 235929600 total_loss: 3.39 time: 0.3286 s/iter data_time: 0.2087 s/iter total_throughput: 3116.03 samples/s lr: 3.32e-04 [09/27 06:17:22] lb.utils.events INFO: eta: 6:21:09 iteration: 230499/375342 consumed_samples: 236032000 total_loss: 3.394 time: 0.3286 s/iter data_time: 0.2121 s/iter total_throughput: 3116.02 samples/s lr: 3.31e-04 [09/27 06:17:55] lb.utils.events INFO: eta: 6:20:53 iteration: 230599/375342 consumed_samples: 236134400 total_loss: 3.393 time: 0.3286 s/iter data_time: 0.2159 s/iter total_throughput: 3116.03 samples/s lr: 3.31e-04 [09/27 06:18:27] lb.utils.events INFO: eta: 6:18:31 iteration: 230699/375342 consumed_samples: 236236800 total_loss: 3.387 time: 0.3286 s/iter data_time: 0.2021 s/iter total_throughput: 3116.03 samples/s lr: 3.31e-04 [09/27 06:19:00] lb.utils.events INFO: eta: 6:16:19 iteration: 230799/375342 consumed_samples: 236339200 total_loss: 3.39 time: 0.3286 s/iter data_time: 0.2110 s/iter total_throughput: 3116.03 samples/s lr: 3.30e-04 [09/27 06:19:33] lb.utils.events INFO: eta: 6:13:46 iteration: 230899/375342 consumed_samples: 236441600 total_loss: 3.406 time: 0.3286 s/iter data_time: 0.2092 s/iter total_throughput: 3116.02 samples/s lr: 3.30e-04 [09/27 06:20:06] lb.utils.events INFO: eta: 6:11:41 iteration: 230999/375342 consumed_samples: 236544000 total_loss: 3.411 time: 0.3286 s/iter data_time: 0.2275 s/iter total_throughput: 3116.02 samples/s lr: 3.29e-04 [09/27 06:20:40] lb.utils.events INFO: eta: 6:09:37 iteration: 231099/375342 consumed_samples: 236646400 total_loss: 3.402 time: 0.3286 s/iter data_time: 0.2137 s/iter total_throughput: 3116.00 samples/s lr: 3.29e-04 [09/27 06:21:13] lb.utils.events INFO: eta: 6:08:37 iteration: 231199/375342 consumed_samples: 236748800 total_loss: 3.395 time: 0.3286 s/iter data_time: 0.2132 s/iter total_throughput: 3116.00 samples/s lr: 3.29e-04 [09/27 06:21:46] lb.utils.events INFO: eta: 6:09:06 iteration: 231299/375342 consumed_samples: 236851200 total_loss: 3.406 time: 0.3286 s/iter data_time: 0.2163 s/iter total_throughput: 3116.00 samples/s lr: 3.28e-04 [09/27 06:22:19] lb.utils.events INFO: eta: 6:08:17 iteration: 231399/375342 consumed_samples: 236953600 total_loss: 3.4 time: 0.3286 s/iter data_time: 0.2024 s/iter total_throughput: 3115.98 samples/s lr: 3.28e-04 [09/27 06:22:52] lb.utils.events INFO: eta: 6:08:53 iteration: 231499/375342 consumed_samples: 237056000 total_loss: 3.389 time: 0.3286 s/iter data_time: 0.2256 s/iter total_throughput: 3115.98 samples/s lr: 3.27e-04 [09/27 06:23:25] lb.utils.events INFO: eta: 6:09:39 iteration: 231599/375342 consumed_samples: 237158400 total_loss: 3.395 time: 0.3286 s/iter data_time: 0.2269 s/iter total_throughput: 3115.98 samples/s lr: 3.27e-04 [09/27 06:23:58] lb.utils.events INFO: eta: 6:11:43 iteration: 231699/375342 consumed_samples: 237260800 total_loss: 3.404 time: 0.3286 s/iter data_time: 0.2307 s/iter total_throughput: 3115.97 samples/s lr: 3.27e-04 [09/27 06:24:30] lb.utils.events INFO: eta: 6:13:57 iteration: 231799/375342 consumed_samples: 237363200 total_loss: 3.375 time: 0.3286 s/iter data_time: 0.2269 s/iter total_throughput: 3115.98 samples/s lr: 3.26e-04 [09/27 06:25:03] lb.utils.events INFO: eta: 6:17:53 iteration: 231899/375342 consumed_samples: 237465600 total_loss: 3.395 time: 0.3286 s/iter data_time: 0.2156 s/iter total_throughput: 3115.98 samples/s lr: 3.26e-04 [09/27 06:25:36] lb.utils.events INFO: eta: 6:21:09 iteration: 231999/375342 consumed_samples: 237568000 total_loss: 3.404 time: 0.3286 s/iter data_time: 0.2563 s/iter total_throughput: 3116.00 samples/s lr: 3.26e-04 [09/27 06:26:09] lb.utils.events INFO: eta: 6:31:01 iteration: 232099/375342 consumed_samples: 237670400 total_loss: 3.389 time: 0.3286 s/iter data_time: 0.2343 s/iter total_throughput: 3115.99 samples/s lr: 3.25e-04 [09/27 06:26:42] lb.utils.events INFO: eta: 6:38:35 iteration: 232199/375342 consumed_samples: 237772800 total_loss: 3.383 time: 0.3286 s/iter data_time: 0.2387 s/iter total_throughput: 3115.99 samples/s lr: 3.25e-04 [09/27 06:27:14] lb.utils.events INFO: eta: 6:44:57 iteration: 232299/375342 consumed_samples: 237875200 total_loss: 3.398 time: 0.3286 s/iter data_time: 0.2232 s/iter total_throughput: 3115.99 samples/s lr: 3.24e-04 [09/27 06:27:47] lb.utils.events INFO: eta: 7:06:10 iteration: 232399/375342 consumed_samples: 237977600 total_loss: 3.392 time: 0.3286 s/iter data_time: 0.2258 s/iter total_throughput: 3115.99 samples/s lr: 3.24e-04 [09/27 06:28:20] lb.utils.events INFO: eta: 8:01:44 iteration: 232499/375342 consumed_samples: 238080000 total_loss: 3.378 time: 0.3286 s/iter data_time: 0.2289 s/iter total_throughput: 3115.99 samples/s lr: 3.24e-04 [09/27 06:28:53] lb.utils.events INFO: eta: 9:29:40 iteration: 232599/375342 consumed_samples: 238182400 total_loss: 3.402 time: 0.3286 s/iter data_time: 0.2375 s/iter total_throughput: 3116.00 samples/s lr: 3.23e-04 [09/27 06:29:26] lb.utils.events INFO: eta: 9:29:16 iteration: 232699/375342 consumed_samples: 238284800 total_loss: 3.385 time: 0.3286 s/iter data_time: 0.2267 s/iter total_throughput: 3115.99 samples/s lr: 3.23e-04 [09/27 06:29:59] lb.utils.events INFO: eta: 9:17:02 iteration: 232799/375342 consumed_samples: 238387200 total_loss: 3.355 time: 0.3286 s/iter data_time: 0.2134 s/iter total_throughput: 3115.98 samples/s lr: 3.22e-04 [09/27 06:30:32] lb.utils.events INFO: eta: 8:50:03 iteration: 232899/375342 consumed_samples: 238489600 total_loss: 3.359 time: 0.3286 s/iter data_time: 0.2225 s/iter total_throughput: 3115.99 samples/s lr: 3.22e-04 [09/27 06:31:05] lb.utils.events INFO: eta: 8:41:12 iteration: 232999/375342 consumed_samples: 238592000 total_loss: 3.39 time: 0.3286 s/iter data_time: 0.2325 s/iter total_throughput: 3115.99 samples/s lr: 3.22e-04 [09/27 06:31:38] lb.utils.events INFO: eta: 6:55:06 iteration: 233099/375342 consumed_samples: 238694400 total_loss: 3.389 time: 0.3286 s/iter data_time: 0.2108 s/iter total_throughput: 3115.98 samples/s lr: 3.21e-04 [09/27 06:32:10] lb.utils.events INFO: eta: 6:37:57 iteration: 233199/375342 consumed_samples: 238796800 total_loss: 3.369 time: 0.3286 s/iter data_time: 0.2101 s/iter total_throughput: 3115.99 samples/s lr: 3.21e-04 [09/27 06:32:43] lb.utils.events INFO: eta: 6:28:51 iteration: 233299/375342 consumed_samples: 238899200 total_loss: 3.395 time: 0.3286 s/iter data_time: 0.1993 s/iter total_throughput: 3115.98 samples/s lr: 3.21e-04 [09/27 06:33:16] lb.utils.events INFO: eta: 6:23:53 iteration: 233399/375342 consumed_samples: 239001600 total_loss: 3.406 time: 0.3286 s/iter data_time: 0.2112 s/iter total_throughput: 3115.99 samples/s lr: 3.20e-04 [09/27 06:33:49] lb.utils.events INFO: eta: 6:19:55 iteration: 233499/375342 consumed_samples: 239104000 total_loss: 3.396 time: 0.3286 s/iter data_time: 0.2300 s/iter total_throughput: 3115.99 samples/s lr: 3.20e-04 [09/27 06:34:22] lb.utils.events INFO: eta: 6:17:13 iteration: 233599/375342 consumed_samples: 239206400 total_loss: 3.381 time: 0.3286 s/iter data_time: 0.2295 s/iter total_throughput: 3115.98 samples/s lr: 3.19e-04 [09/27 06:34:55] lb.utils.events INFO: eta: 6:17:31 iteration: 233699/375342 consumed_samples: 239308800 total_loss: 3.375 time: 0.3286 s/iter data_time: 0.2232 s/iter total_throughput: 3115.98 samples/s lr: 3.19e-04 [09/27 06:35:27] lb.utils.events INFO: eta: 6:16:47 iteration: 233799/375342 consumed_samples: 239411200 total_loss: 3.382 time: 0.3286 s/iter data_time: 0.2238 s/iter total_throughput: 3115.99 samples/s lr: 3.19e-04 [09/27 06:36:00] lb.utils.events INFO: eta: 6:18:36 iteration: 233899/375342 consumed_samples: 239513600 total_loss: 3.393 time: 0.3286 s/iter data_time: 0.2245 s/iter total_throughput: 3115.99 samples/s lr: 3.18e-04 [09/27 06:36:33] lb.utils.events INFO: eta: 6:18:02 iteration: 233999/375342 consumed_samples: 239616000 total_loss: 3.369 time: 0.3286 s/iter data_time: 0.2469 s/iter total_throughput: 3116.00 samples/s lr: 3.18e-04 [09/27 06:37:06] lb.utils.events INFO: eta: 6:19:46 iteration: 234099/375342 consumed_samples: 239718400 total_loss: 3.349 time: 0.3286 s/iter data_time: 0.2361 s/iter total_throughput: 3115.98 samples/s lr: 3.17e-04 [09/27 06:37:39] lb.utils.events INFO: eta: 6:22:14 iteration: 234199/375342 consumed_samples: 239820800 total_loss: 3.351 time: 0.3286 s/iter data_time: 0.2230 s/iter total_throughput: 3115.97 samples/s lr: 3.17e-04 [09/27 06:38:12] lb.utils.events INFO: eta: 6:26:06 iteration: 234299/375342 consumed_samples: 239923200 total_loss: 3.368 time: 0.3286 s/iter data_time: 0.2197 s/iter total_throughput: 3115.98 samples/s lr: 3.17e-04 [09/27 06:38:45] lb.utils.events INFO: eta: 6:33:25 iteration: 234399/375342 consumed_samples: 240025600 total_loss: 3.361 time: 0.3286 s/iter data_time: 0.2158 s/iter total_throughput: 3116.00 samples/s lr: 3.16e-04 [09/27 06:39:17] lb.utils.events INFO: eta: 6:33:38 iteration: 234499/375342 consumed_samples: 240128000 total_loss: 3.37 time: 0.3286 s/iter data_time: 0.2055 s/iter total_throughput: 3116.00 samples/s lr: 3.16e-04 [09/27 06:39:50] lb.utils.events INFO: eta: 6:39:06 iteration: 234599/375342 consumed_samples: 240230400 total_loss: 3.383 time: 0.3286 s/iter data_time: 0.2418 s/iter total_throughput: 3116.01 samples/s lr: 3.16e-04 [09/27 06:40:23] lb.utils.events INFO: eta: 6:38:49 iteration: 234699/375342 consumed_samples: 240332800 total_loss: 3.385 time: 0.3286 s/iter data_time: 0.2381 s/iter total_throughput: 3116.00 samples/s lr: 3.15e-04 [09/27 06:40:56] lb.utils.events INFO: eta: 6:34:30 iteration: 234799/375342 consumed_samples: 240435200 total_loss: 3.397 time: 0.3286 s/iter data_time: 0.2058 s/iter total_throughput: 3116.00 samples/s lr: 3.15e-04 [09/27 06:41:29] lb.utils.events INFO: eta: 6:23:24 iteration: 234899/375342 consumed_samples: 240537600 total_loss: 3.386 time: 0.3286 s/iter data_time: 0.2056 s/iter total_throughput: 3116.01 samples/s lr: 3.14e-04 [09/27 06:42:02] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0234999 [09/27 06:42:02] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/27 06:42:02] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/27 06:42:06] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0838 s/iter. Inference: 0.1501 s/iter. Eval: 0.0020 s/iter. Total: 0.2358 s/iter. ETA=0:00:08 [09/27 06:42:12] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1382 s/iter. Inference: 0.1542 s/iter. Eval: 0.0020 s/iter. Total: 0.2945 s/iter. ETA=0:00:05 [09/27 06:42:17] lb.evaluation.evaluator INFO: Inference done 50000/50000. Dataloading: 0.1296 s/iter. Inference: 0.1520 s/iter. Eval: 0.0020 s/iter. Total: 0.2837 s/iter. ETA=0:00:00 [09/27 06:42:17] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/27 06:42:17] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.486114 (0.000250 s / iter per device, on 8 devices) [09/27 06:42:17] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000134 s / iter per device, on 8 devices) [09/27 06:42:17] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/27 06:42:17] lb.evaluation.utils INFO: copypaste: Acc@1=76.588 [09/27 06:42:17] lb.evaluation.utils INFO: copypaste: Acc@5=93.34599999999999 [09/27 06:42:17] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 76.58800, better than last best score 76.52400 @ iteration 229999. [09/27 06:42:17] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/27 06:42:18] lb.utils.events INFO: eta: 6:16:22 iteration: 234999/375342 consumed_samples: 240640000 total_loss: 3.376 time: 0.3286 s/iter data_time: 0.2130 s/iter total_throughput: 3116.00 samples/s lr: 3.14e-04 [09/27 06:42:49] lb.utils.events INFO: eta: 6:16:11 iteration: 235099/375342 consumed_samples: 240742400 total_loss: 3.383 time: 0.3286 s/iter data_time: 0.2286 s/iter total_throughput: 3116.08 samples/s lr: 3.14e-04 [09/27 06:43:22] lb.utils.events INFO: eta: 6:20:02 iteration: 235199/375342 consumed_samples: 240844800 total_loss: 3.401 time: 0.3286 s/iter data_time: 0.2295 s/iter total_throughput: 3116.09 samples/s lr: 3.13e-04 [09/27 06:43:55] lb.utils.events INFO: eta: 6:17:42 iteration: 235299/375342 consumed_samples: 240947200 total_loss: 3.396 time: 0.3286 s/iter data_time: 0.2119 s/iter total_throughput: 3116.08 samples/s lr: 3.13e-04 [09/27 06:44:28] lb.utils.events INFO: eta: 6:13:01 iteration: 235399/375342 consumed_samples: 241049600 total_loss: 3.373 time: 0.3286 s/iter data_time: 0.2119 s/iter total_throughput: 3116.08 samples/s lr: 3.12e-04 [09/27 06:45:00] lb.utils.events INFO: eta: 6:10:50 iteration: 235499/375342 consumed_samples: 241152000 total_loss: 3.365 time: 0.3286 s/iter data_time: 0.1966 s/iter total_throughput: 3116.09 samples/s lr: 3.12e-04 [09/27 06:45:33] lb.utils.events INFO: eta: 6:06:44 iteration: 235599/375342 consumed_samples: 241254400 total_loss: 3.368 time: 0.3286 s/iter data_time: 0.2284 s/iter total_throughput: 3116.10 samples/s lr: 3.12e-04 [09/27 06:46:05] lb.utils.events INFO: eta: 6:06:26 iteration: 235699/375342 consumed_samples: 241356800 total_loss: 3.379 time: 0.3286 s/iter data_time: 0.2467 s/iter total_throughput: 3116.12 samples/s lr: 3.11e-04 [09/27 06:46:38] lb.utils.events INFO: eta: 6:09:34 iteration: 235799/375342 consumed_samples: 241459200 total_loss: 3.36 time: 0.3286 s/iter data_time: 0.2358 s/iter total_throughput: 3116.11 samples/s lr: 3.11e-04 [09/27 06:47:11] lb.utils.events INFO: eta: 6:14:06 iteration: 235899/375342 consumed_samples: 241561600 total_loss: 3.352 time: 0.3286 s/iter data_time: 0.2174 s/iter total_throughput: 3116.12 samples/s lr: 3.11e-04 [09/27 06:47:44] lb.utils.events INFO: eta: 6:16:23 iteration: 235999/375342 consumed_samples: 241664000 total_loss: 3.37 time: 0.3286 s/iter data_time: 0.2217 s/iter total_throughput: 3116.12 samples/s lr: 3.10e-04 [09/27 06:48:17] lb.utils.events INFO: eta: 6:12:29 iteration: 236099/375342 consumed_samples: 241766400 total_loss: 3.39 time: 0.3286 s/iter data_time: 0.2312 s/iter total_throughput: 3116.12 samples/s lr: 3.10e-04 [09/27 06:48:49] lb.utils.events INFO: eta: 6:08:25 iteration: 236199/375342 consumed_samples: 241868800 total_loss: 3.389 time: 0.3286 s/iter data_time: 0.2126 s/iter total_throughput: 3116.13 samples/s lr: 3.09e-04 [09/27 06:49:22] lb.utils.events INFO: eta: 6:12:10 iteration: 236299/375342 consumed_samples: 241971200 total_loss: 3.387 time: 0.3286 s/iter data_time: 0.2505 s/iter total_throughput: 3116.15 samples/s lr: 3.09e-04 [09/27 06:49:54] lb.utils.events INFO: eta: 6:17:57 iteration: 236399/375342 consumed_samples: 242073600 total_loss: 3.388 time: 0.3286 s/iter data_time: 0.2280 s/iter total_throughput: 3116.17 samples/s lr: 3.09e-04 [09/27 06:50:27] lb.utils.events INFO: eta: 6:32:56 iteration: 236499/375342 consumed_samples: 242176000 total_loss: 3.385 time: 0.3286 s/iter data_time: 0.2213 s/iter total_throughput: 3116.18 samples/s lr: 3.08e-04 [09/27 06:51:00] lb.utils.events INFO: eta: 6:26:55 iteration: 236599/375342 consumed_samples: 242278400 total_loss: 3.353 time: 0.3286 s/iter data_time: 0.1969 s/iter total_throughput: 3116.18 samples/s lr: 3.08e-04 [09/27 06:51:33] lb.utils.events INFO: eta: 6:19:26 iteration: 236699/375342 consumed_samples: 242380800 total_loss: 3.383 time: 0.3286 s/iter data_time: 0.2053 s/iter total_throughput: 3116.18 samples/s lr: 3.08e-04 [09/27 06:52:05] lb.utils.events INFO: eta: 6:11:05 iteration: 236799/375342 consumed_samples: 242483200 total_loss: 3.389 time: 0.3286 s/iter data_time: 0.2111 s/iter total_throughput: 3116.20 samples/s lr: 3.07e-04 [09/27 06:52:38] lb.utils.events INFO: eta: 6:07:13 iteration: 236899/375342 consumed_samples: 242585600 total_loss: 3.374 time: 0.3286 s/iter data_time: 0.2279 s/iter total_throughput: 3116.20 samples/s lr: 3.07e-04 [09/27 06:53:10] lb.utils.events INFO: eta: 6:08:54 iteration: 236999/375342 consumed_samples: 242688000 total_loss: 3.376 time: 0.3286 s/iter data_time: 0.2210 s/iter total_throughput: 3116.21 samples/s lr: 3.06e-04 [09/27 06:53:43] lb.utils.events INFO: eta: 6:13:07 iteration: 237099/375342 consumed_samples: 242790400 total_loss: 3.371 time: 0.3286 s/iter data_time: 0.2211 s/iter total_throughput: 3116.23 samples/s lr: 3.06e-04 [09/27 06:54:15] lb.utils.events INFO: eta: 6:12:27 iteration: 237199/375342 consumed_samples: 242892800 total_loss: 3.369 time: 0.3286 s/iter data_time: 0.2168 s/iter total_throughput: 3116.24 samples/s lr: 3.06e-04 [09/27 06:54:48] lb.utils.events INFO: eta: 6:06:21 iteration: 237299/375342 consumed_samples: 242995200 total_loss: 3.387 time: 0.3286 s/iter data_time: 0.2086 s/iter total_throughput: 3116.24 samples/s lr: 3.05e-04 [09/27 06:55:20] lb.utils.events INFO: eta: 6:02:24 iteration: 237399/375342 consumed_samples: 243097600 total_loss: 3.4 time: 0.3286 s/iter data_time: 0.2004 s/iter total_throughput: 3116.28 samples/s lr: 3.05e-04 [09/27 06:55:53] lb.utils.events INFO: eta: 5:59:13 iteration: 237499/375342 consumed_samples: 243200000 total_loss: 3.393 time: 0.3286 s/iter data_time: 0.2026 s/iter total_throughput: 3116.28 samples/s lr: 3.04e-04 [09/27 06:56:25] lb.utils.events INFO: eta: 5:58:16 iteration: 237599/375342 consumed_samples: 243302400 total_loss: 3.385 time: 0.3286 s/iter data_time: 0.2053 s/iter total_throughput: 3116.29 samples/s lr: 3.04e-04 [09/27 06:56:58] lb.utils.events INFO: eta: 5:57:33 iteration: 237699/375342 consumed_samples: 243404800 total_loss: 3.375 time: 0.3286 s/iter data_time: 0.2070 s/iter total_throughput: 3116.31 samples/s lr: 3.04e-04 [09/27 06:57:30] lb.utils.events INFO: eta: 5:58:57 iteration: 237799/375342 consumed_samples: 243507200 total_loss: 3.373 time: 0.3286 s/iter data_time: 0.1962 s/iter total_throughput: 3116.33 samples/s lr: 3.03e-04 [09/27 06:58:03] lb.utils.events INFO: eta: 5:58:03 iteration: 237899/375342 consumed_samples: 243609600 total_loss: 3.356 time: 0.3286 s/iter data_time: 0.2252 s/iter total_throughput: 3116.35 samples/s lr: 3.03e-04 [09/27 06:58:35] lb.utils.events INFO: eta: 5:56:56 iteration: 237999/375342 consumed_samples: 243712000 total_loss: 3.347 time: 0.3286 s/iter data_time: 0.2196 s/iter total_throughput: 3116.37 samples/s lr: 3.03e-04 [09/27 06:59:08] lb.utils.events INFO: eta: 5:55:25 iteration: 238099/375342 consumed_samples: 243814400 total_loss: 3.352 time: 0.3286 s/iter data_time: 0.1946 s/iter total_throughput: 3116.37 samples/s lr: 3.02e-04 [09/27 06:59:41] lb.utils.events INFO: eta: 5:53:16 iteration: 238199/375342 consumed_samples: 243916800 total_loss: 3.356 time: 0.3286 s/iter data_time: 0.2098 s/iter total_throughput: 3116.37 samples/s lr: 3.02e-04 [09/27 07:00:14] lb.utils.events INFO: eta: 5:53:01 iteration: 238299/375342 consumed_samples: 244019200 total_loss: 3.361 time: 0.3286 s/iter data_time: 0.2280 s/iter total_throughput: 3116.35 samples/s lr: 3.01e-04 [09/27 07:00:48] lb.utils.events INFO: eta: 5:52:48 iteration: 238399/375342 consumed_samples: 244121600 total_loss: 3.365 time: 0.3286 s/iter data_time: 0.2157 s/iter total_throughput: 3116.33 samples/s lr: 3.01e-04 [09/27 07:01:21] lb.utils.events INFO: eta: 5:52:11 iteration: 238499/375342 consumed_samples: 244224000 total_loss: 3.379 time: 0.3286 s/iter data_time: 0.2151 s/iter total_throughput: 3116.32 samples/s lr: 3.01e-04 [09/27 07:01:54] lb.utils.events INFO: eta: 5:52:01 iteration: 238599/375342 consumed_samples: 244326400 total_loss: 3.385 time: 0.3286 s/iter data_time: 0.2155 s/iter total_throughput: 3116.29 samples/s lr: 3.00e-04 [09/27 07:02:28] lb.utils.events INFO: eta: 5:52:33 iteration: 238699/375342 consumed_samples: 244428800 total_loss: 3.376 time: 0.3286 s/iter data_time: 0.2037 s/iter total_throughput: 3116.27 samples/s lr: 3.00e-04 [09/27 07:03:01] lb.utils.events INFO: eta: 5:51:38 iteration: 238799/375342 consumed_samples: 244531200 total_loss: 3.357 time: 0.3286 s/iter data_time: 0.2035 s/iter total_throughput: 3116.26 samples/s lr: 3.00e-04 [09/27 07:03:34] lb.utils.events INFO: eta: 5:51:28 iteration: 238899/375342 consumed_samples: 244633600 total_loss: 3.366 time: 0.3286 s/iter data_time: 0.2230 s/iter total_throughput: 3116.26 samples/s lr: 2.99e-04 [09/27 07:04:07] lb.utils.events INFO: eta: 5:51:17 iteration: 238999/375342 consumed_samples: 244736000 total_loss: 3.366 time: 0.3286 s/iter data_time: 0.2046 s/iter total_throughput: 3116.26 samples/s lr: 2.99e-04 [09/27 07:04:40] lb.utils.events INFO: eta: 5:51:36 iteration: 239099/375342 consumed_samples: 244838400 total_loss: 3.331 time: 0.3286 s/iter data_time: 0.2302 s/iter total_throughput: 3116.22 samples/s lr: 2.98e-04 [09/27 07:05:14] lb.utils.events INFO: eta: 5:51:51 iteration: 239199/375342 consumed_samples: 244940800 total_loss: 3.354 time: 0.3286 s/iter data_time: 0.2142 s/iter total_throughput: 3116.20 samples/s lr: 2.98e-04 [09/27 07:05:47] lb.utils.events INFO: eta: 5:51:23 iteration: 239299/375342 consumed_samples: 245043200 total_loss: 3.373 time: 0.3286 s/iter data_time: 0.2083 s/iter total_throughput: 3116.19 samples/s lr: 2.98e-04 [09/27 07:06:20] lb.utils.events INFO: eta: 5:51:13 iteration: 239399/375342 consumed_samples: 245145600 total_loss: 3.355 time: 0.3286 s/iter data_time: 0.2196 s/iter total_throughput: 3116.16 samples/s lr: 2.97e-04 [09/27 07:06:54] lb.utils.events INFO: eta: 5:51:14 iteration: 239499/375342 consumed_samples: 245248000 total_loss: 3.33 time: 0.3286 s/iter data_time: 0.2082 s/iter total_throughput: 3116.14 samples/s lr: 2.97e-04 [09/27 07:07:27] lb.utils.events INFO: eta: 5:51:18 iteration: 239599/375342 consumed_samples: 245350400 total_loss: 3.336 time: 0.3286 s/iter data_time: 0.2082 s/iter total_throughput: 3116.14 samples/s lr: 2.97e-04 [09/27 07:07:59] lb.utils.events INFO: eta: 5:51:41 iteration: 239699/375342 consumed_samples: 245452800 total_loss: 3.343 time: 0.3286 s/iter data_time: 0.2353 s/iter total_throughput: 3116.15 samples/s lr: 2.96e-04 [09/27 07:08:33] lb.utils.events INFO: eta: 5:53:41 iteration: 239799/375342 consumed_samples: 245555200 total_loss: 3.35 time: 0.3286 s/iter data_time: 0.2242 s/iter total_throughput: 3116.13 samples/s lr: 2.96e-04 [09/27 07:09:06] lb.utils.events INFO: eta: 5:54:39 iteration: 239899/375342 consumed_samples: 245657600 total_loss: 3.375 time: 0.3286 s/iter data_time: 0.2253 s/iter total_throughput: 3116.13 samples/s lr: 2.95e-04 [09/27 07:09:39] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0239999 [09/27 07:09:40] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/27 07:09:40] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/27 07:09:44] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0891 s/iter. Inference: 0.1514 s/iter. Eval: 0.0020 s/iter. Total: 0.2425 s/iter. ETA=0:00:08 [09/27 07:09:50] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1509 s/iter. Inference: 0.1493 s/iter. Eval: 0.0020 s/iter. Total: 0.3023 s/iter. ETA=0:00:05 [09/27 07:09:55] lb.evaluation.evaluator INFO: Inference done 49152/50000. Dataloading: 0.1342 s/iter. Inference: 0.1490 s/iter. Eval: 0.0020 s/iter. Total: 0.2853 s/iter. ETA=0:00:00 [09/27 07:09:55] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/27 07:09:55] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.707367 (0.000254 s / iter per device, on 8 devices) [09/27 07:09:55] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000132 s / iter per device, on 8 devices) [09/27 07:09:55] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/27 07:09:55] lb.evaluation.utils INFO: copypaste: Acc@1=77.036 [09/27 07:09:55] lb.evaluation.utils INFO: copypaste: Acc@5=93.378 [09/27 07:09:55] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 77.03600, better than last best score 76.58800 @ iteration 234999. [09/27 07:09:55] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/27 07:09:56] lb.utils.events INFO: eta: 5:57:56 iteration: 239999/375342 consumed_samples: 245760000 total_loss: 3.345 time: 0.3286 s/iter data_time: 0.2465 s/iter total_throughput: 3116.11 samples/s lr: 2.95e-04 [09/27 07:10:27] lb.utils.events INFO: eta: 5:57:19 iteration: 240099/375342 consumed_samples: 245862400 total_loss: 3.352 time: 0.3286 s/iter data_time: 0.2305 s/iter total_throughput: 3116.17 samples/s lr: 2.95e-04 [09/27 07:11:00] lb.utils.events INFO: eta: 5:59:35 iteration: 240199/375342 consumed_samples: 245964800 total_loss: 3.36 time: 0.3286 s/iter data_time: 0.2164 s/iter total_throughput: 3116.17 samples/s lr: 2.94e-04 [09/27 07:11:33] lb.utils.events INFO: eta: 6:02:26 iteration: 240299/375342 consumed_samples: 246067200 total_loss: 3.349 time: 0.3286 s/iter data_time: 0.2400 s/iter total_throughput: 3116.16 samples/s lr: 2.94e-04 [09/27 07:12:06] lb.utils.events INFO: eta: 6:09:20 iteration: 240399/375342 consumed_samples: 246169600 total_loss: 3.332 time: 0.3286 s/iter data_time: 0.2363 s/iter total_throughput: 3116.15 samples/s lr: 2.94e-04 [09/27 07:12:39] lb.utils.events INFO: eta: 6:51:17 iteration: 240499/375342 consumed_samples: 246272000 total_loss: 3.332 time: 0.3286 s/iter data_time: 0.2426 s/iter total_throughput: 3116.15 samples/s lr: 2.93e-04 [09/27 07:13:12] lb.utils.events INFO: eta: 8:35:15 iteration: 240599/375342 consumed_samples: 246374400 total_loss: 3.339 time: 0.3286 s/iter data_time: 0.2405 s/iter total_throughput: 3116.15 samples/s lr: 2.93e-04 [09/27 07:13:45] lb.utils.events INFO: eta: 9:12:39 iteration: 240699/375342 consumed_samples: 246476800 total_loss: 3.346 time: 0.3286 s/iter data_time: 0.2473 s/iter total_throughput: 3116.14 samples/s lr: 2.92e-04 [09/27 07:14:19] lb.utils.events INFO: eta: 8:56:53 iteration: 240799/375342 consumed_samples: 246579200 total_loss: 3.321 time: 0.3286 s/iter data_time: 0.2308 s/iter total_throughput: 3116.12 samples/s lr: 2.92e-04 [09/27 07:14:52] lb.utils.events INFO: eta: 8:38:48 iteration: 240899/375342 consumed_samples: 246681600 total_loss: 3.325 time: 0.3286 s/iter data_time: 0.2194 s/iter total_throughput: 3116.10 samples/s lr: 2.92e-04 [09/27 07:15:25] lb.utils.events INFO: eta: 7:27:15 iteration: 240999/375342 consumed_samples: 246784000 total_loss: 3.342 time: 0.3286 s/iter data_time: 0.2022 s/iter total_throughput: 3116.09 samples/s lr: 2.91e-04 [09/27 07:15:58] lb.utils.events INFO: eta: 6:48:12 iteration: 241099/375342 consumed_samples: 246886400 total_loss: 3.344 time: 0.3286 s/iter data_time: 0.2028 s/iter total_throughput: 3116.08 samples/s lr: 2.91e-04 [09/27 07:16:31] lb.utils.events INFO: eta: 6:22:21 iteration: 241199/375342 consumed_samples: 246988800 total_loss: 3.355 time: 0.3286 s/iter data_time: 0.2174 s/iter total_throughput: 3116.08 samples/s lr: 2.91e-04 [09/27 07:17:04] lb.utils.events INFO: eta: 6:20:54 iteration: 241299/375342 consumed_samples: 247091200 total_loss: 3.354 time: 0.3286 s/iter data_time: 0.2213 s/iter total_throughput: 3116.07 samples/s lr: 2.90e-04 [09/27 07:17:38] lb.utils.events INFO: eta: 6:02:50 iteration: 241399/375342 consumed_samples: 247193600 total_loss: 3.355 time: 0.3286 s/iter data_time: 0.2078 s/iter total_throughput: 3116.05 samples/s lr: 2.90e-04 [09/27 07:18:10] lb.utils.events INFO: eta: 5:55:32 iteration: 241499/375342 consumed_samples: 247296000 total_loss: 3.348 time: 0.3286 s/iter data_time: 0.2037 s/iter total_throughput: 3116.05 samples/s lr: 2.89e-04 [09/27 07:18:43] lb.utils.events INFO: eta: 5:50:32 iteration: 241599/375342 consumed_samples: 247398400 total_loss: 3.357 time: 0.3286 s/iter data_time: 0.2021 s/iter total_throughput: 3116.05 samples/s lr: 2.89e-04 [09/27 07:19:17] lb.utils.events INFO: eta: 5:45:30 iteration: 241699/375342 consumed_samples: 247500800 total_loss: 3.36 time: 0.3286 s/iter data_time: 0.2187 s/iter total_throughput: 3116.03 samples/s lr: 2.89e-04 [09/27 07:19:49] lb.utils.events INFO: eta: 5:44:26 iteration: 241799/375342 consumed_samples: 247603200 total_loss: 3.342 time: 0.3286 s/iter data_time: 0.2068 s/iter total_throughput: 3116.03 samples/s lr: 2.88e-04 [09/27 07:20:22] lb.utils.events INFO: eta: 5:44:06 iteration: 241899/375342 consumed_samples: 247705600 total_loss: 3.338 time: 0.3286 s/iter data_time: 0.2220 s/iter total_throughput: 3116.03 samples/s lr: 2.88e-04 [09/27 07:20:55] lb.utils.events INFO: eta: 5:44:57 iteration: 241999/375342 consumed_samples: 247808000 total_loss: 3.333 time: 0.3286 s/iter data_time: 0.2154 s/iter total_throughput: 3116.03 samples/s lr: 2.88e-04 [09/27 07:21:29] lb.utils.events INFO: eta: 5:45:01 iteration: 242099/375342 consumed_samples: 247910400 total_loss: 3.35 time: 0.3286 s/iter data_time: 0.2079 s/iter total_throughput: 3116.01 samples/s lr: 2.87e-04 [09/27 07:22:01] lb.utils.events INFO: eta: 5:44:25 iteration: 242199/375342 consumed_samples: 248012800 total_loss: 3.349 time: 0.3286 s/iter data_time: 0.2140 s/iter total_throughput: 3116.02 samples/s lr: 2.87e-04 [09/27 07:22:34] lb.utils.events INFO: eta: 5:43:24 iteration: 242299/375342 consumed_samples: 248115200 total_loss: 3.362 time: 0.3286 s/iter data_time: 0.2241 s/iter total_throughput: 3116.02 samples/s lr: 2.86e-04 [09/27 07:23:07] lb.utils.events INFO: eta: 5:45:00 iteration: 242399/375342 consumed_samples: 248217600 total_loss: 3.35 time: 0.3286 s/iter data_time: 0.2309 s/iter total_throughput: 3116.02 samples/s lr: 2.86e-04 [09/27 07:23:40] lb.utils.events INFO: eta: 5:45:54 iteration: 242499/375342 consumed_samples: 248320000 total_loss: 3.35 time: 0.3286 s/iter data_time: 0.2052 s/iter total_throughput: 3116.02 samples/s lr: 2.86e-04 [09/27 07:24:13] lb.utils.events INFO: eta: 5:45:38 iteration: 242599/375342 consumed_samples: 248422400 total_loss: 3.363 time: 0.3286 s/iter data_time: 0.2120 s/iter total_throughput: 3116.00 samples/s lr: 2.85e-04 [09/27 07:24:46] lb.utils.events INFO: eta: 5:47:37 iteration: 242699/375342 consumed_samples: 248524800 total_loss: 3.361 time: 0.3286 s/iter data_time: 0.2189 s/iter total_throughput: 3116.01 samples/s lr: 2.85e-04 [09/27 07:25:19] lb.utils.events INFO: eta: 5:49:44 iteration: 242799/375342 consumed_samples: 248627200 total_loss: 3.354 time: 0.3286 s/iter data_time: 0.2279 s/iter total_throughput: 3115.99 samples/s lr: 2.85e-04 [09/27 07:25:53] lb.utils.events INFO: eta: 5:51:08 iteration: 242899/375342 consumed_samples: 248729600 total_loss: 3.352 time: 0.3286 s/iter data_time: 0.2134 s/iter total_throughput: 3115.98 samples/s lr: 2.84e-04 [09/27 07:26:25] lb.utils.events INFO: eta: 5:51:18 iteration: 242999/375342 consumed_samples: 248832000 total_loss: 3.351 time: 0.3286 s/iter data_time: 0.2394 s/iter total_throughput: 3115.99 samples/s lr: 2.84e-04 [09/27 07:26:58] lb.utils.events INFO: eta: 5:54:29 iteration: 243099/375342 consumed_samples: 248934400 total_loss: 3.349 time: 0.3286 s/iter data_time: 0.2400 s/iter total_throughput: 3115.99 samples/s lr: 2.84e-04 [09/27 07:27:31] lb.utils.events INFO: eta: 5:58:05 iteration: 243199/375342 consumed_samples: 249036800 total_loss: 3.353 time: 0.3286 s/iter data_time: 0.2491 s/iter total_throughput: 3115.97 samples/s lr: 2.83e-04 [09/27 07:28:04] lb.utils.events INFO: eta: 6:01:52 iteration: 243299/375342 consumed_samples: 249139200 total_loss: 3.337 time: 0.3286 s/iter data_time: 0.2362 s/iter total_throughput: 3115.97 samples/s lr: 2.83e-04 [09/27 07:28:37] lb.utils.events INFO: eta: 6:08:49 iteration: 243399/375342 consumed_samples: 249241600 total_loss: 3.339 time: 0.3286 s/iter data_time: 0.2448 s/iter total_throughput: 3115.96 samples/s lr: 2.82e-04 [09/27 07:29:11] lb.utils.events INFO: eta: 6:12:30 iteration: 243499/375342 consumed_samples: 249344000 total_loss: 3.362 time: 0.3286 s/iter data_time: 0.2197 s/iter total_throughput: 3115.93 samples/s lr: 2.82e-04 [09/27 07:29:44] lb.utils.events INFO: eta: 6:51:40 iteration: 243599/375342 consumed_samples: 249446400 total_loss: 3.364 time: 0.3286 s/iter data_time: 0.2274 s/iter total_throughput: 3115.93 samples/s lr: 2.82e-04 [09/27 07:30:17] lb.utils.events INFO: eta: 6:49:52 iteration: 243699/375342 consumed_samples: 249548800 total_loss: 3.354 time: 0.3286 s/iter data_time: 0.2112 s/iter total_throughput: 3115.91 samples/s lr: 2.81e-04 [09/27 07:30:51] lb.utils.events INFO: eta: 6:14:56 iteration: 243799/375342 consumed_samples: 249651200 total_loss: 3.331 time: 0.3286 s/iter data_time: 0.2154 s/iter total_throughput: 3115.89 samples/s lr: 2.81e-04 [09/27 07:31:23] lb.utils.events INFO: eta: 6:09:03 iteration: 243899/375342 consumed_samples: 249753600 total_loss: 3.339 time: 0.3286 s/iter data_time: 0.2064 s/iter total_throughput: 3115.89 samples/s lr: 2.81e-04 [09/27 07:31:57] lb.utils.events INFO: eta: 6:05:47 iteration: 243999/375342 consumed_samples: 249856000 total_loss: 3.349 time: 0.3286 s/iter data_time: 0.2296 s/iter total_throughput: 3115.87 samples/s lr: 2.80e-04 [09/27 07:32:30] lb.utils.events INFO: eta: 5:57:33 iteration: 244099/375342 consumed_samples: 249958400 total_loss: 3.34 time: 0.3286 s/iter data_time: 0.2082 s/iter total_throughput: 3115.86 samples/s lr: 2.80e-04 [09/27 07:33:03] lb.utils.events INFO: eta: 5:52:18 iteration: 244199/375342 consumed_samples: 250060800 total_loss: 3.327 time: 0.3286 s/iter data_time: 0.2051 s/iter total_throughput: 3115.84 samples/s lr: 2.79e-04 [09/27 07:33:37] lb.utils.events INFO: eta: 5:48:16 iteration: 244299/375342 consumed_samples: 250163200 total_loss: 3.328 time: 0.3286 s/iter data_time: 0.2096 s/iter total_throughput: 3115.83 samples/s lr: 2.79e-04 [09/27 07:34:09] lb.utils.events INFO: eta: 5:42:23 iteration: 244399/375342 consumed_samples: 250265600 total_loss: 3.345 time: 0.3286 s/iter data_time: 0.2032 s/iter total_throughput: 3115.83 samples/s lr: 2.79e-04 [09/27 07:34:42] lb.utils.events INFO: eta: 5:40:04 iteration: 244499/375342 consumed_samples: 250368000 total_loss: 3.35 time: 0.3286 s/iter data_time: 0.2275 s/iter total_throughput: 3115.83 samples/s lr: 2.78e-04 [09/27 07:35:15] lb.utils.events INFO: eta: 5:39:49 iteration: 244599/375342 consumed_samples: 250470400 total_loss: 3.347 time: 0.3286 s/iter data_time: 0.2229 s/iter total_throughput: 3115.83 samples/s lr: 2.78e-04 [09/27 07:35:48] lb.utils.events INFO: eta: 5:39:22 iteration: 244699/375342 consumed_samples: 250572800 total_loss: 3.344 time: 0.3286 s/iter data_time: 0.2395 s/iter total_throughput: 3115.82 samples/s lr: 2.78e-04 [09/27 07:36:21] lb.utils.events INFO: eta: 5:41:39 iteration: 244799/375342 consumed_samples: 250675200 total_loss: 3.359 time: 0.3286 s/iter data_time: 0.2363 s/iter total_throughput: 3115.81 samples/s lr: 2.77e-04 [09/27 07:36:55] lb.utils.events INFO: eta: 5:42:09 iteration: 244899/375342 consumed_samples: 250777600 total_loss: 3.361 time: 0.3286 s/iter data_time: 0.2068 s/iter total_throughput: 3115.79 samples/s lr: 2.77e-04 [09/27 07:37:28] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0244999 [09/27 07:37:28] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/27 07:37:28] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/27 07:37:33] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0888 s/iter. Inference: 0.1508 s/iter. Eval: 0.0021 s/iter. Total: 0.2417 s/iter. ETA=0:00:08 [09/27 07:37:38] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1479 s/iter. Inference: 0.1517 s/iter. Eval: 0.0020 s/iter. Total: 0.3018 s/iter. ETA=0:00:05 [09/27 07:37:44] lb.evaluation.evaluator INFO: Inference done 50000/50000. Dataloading: 0.1345 s/iter. Inference: 0.1516 s/iter. Eval: 0.0020 s/iter. Total: 0.2882 s/iter. ETA=0:00:00 [09/27 07:37:44] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/27 07:37:44] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.684815 (0.000254 s / iter per device, on 8 devices) [09/27 07:37:44] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000133 s / iter per device, on 8 devices) [09/27 07:37:44] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/27 07:37:44] lb.evaluation.utils INFO: copypaste: Acc@1=77.172 [09/27 07:37:44] lb.evaluation.utils INFO: copypaste: Acc@5=93.512 [09/27 07:37:44] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 77.17200, better than last best score 77.03600 @ iteration 239999. [09/27 07:37:44] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/27 07:37:45] lb.utils.events INFO: eta: 5:40:52 iteration: 244999/375342 consumed_samples: 250880000 total_loss: 3.347 time: 0.3286 s/iter data_time: 0.2025 s/iter total_throughput: 3115.79 samples/s lr: 2.76e-04 [09/27 07:38:16] lb.utils.events INFO: eta: 5:41:29 iteration: 245099/375342 consumed_samples: 250982400 total_loss: 3.342 time: 0.3286 s/iter data_time: 0.2274 s/iter total_throughput: 3115.85 samples/s lr: 2.76e-04 [09/27 07:38:49] lb.utils.events INFO: eta: 5:45:32 iteration: 245199/375342 consumed_samples: 251084800 total_loss: 3.349 time: 0.3286 s/iter data_time: 0.2623 s/iter total_throughput: 3115.84 samples/s lr: 2.76e-04 [09/27 07:39:22] lb.utils.events INFO: eta: 5:50:38 iteration: 245299/375342 consumed_samples: 251187200 total_loss: 3.343 time: 0.3286 s/iter data_time: 0.2371 s/iter total_throughput: 3115.84 samples/s lr: 2.75e-04 [09/27 07:39:55] lb.utils.events INFO: eta: 5:51:41 iteration: 245399/375342 consumed_samples: 251289600 total_loss: 3.333 time: 0.3286 s/iter data_time: 0.2207 s/iter total_throughput: 3115.83 samples/s lr: 2.75e-04 [09/27 07:40:28] lb.utils.events INFO: eta: 5:51:42 iteration: 245499/375342 consumed_samples: 251392000 total_loss: 3.331 time: 0.3286 s/iter data_time: 0.2208 s/iter total_throughput: 3115.83 samples/s lr: 2.75e-04 [09/27 07:41:01] lb.utils.events INFO: eta: 5:56:21 iteration: 245599/375342 consumed_samples: 251494400 total_loss: 3.327 time: 0.3286 s/iter data_time: 0.2593 s/iter total_throughput: 3115.83 samples/s lr: 2.74e-04 [09/27 07:41:34] lb.utils.events INFO: eta: 5:59:29 iteration: 245699/375342 consumed_samples: 251596800 total_loss: 3.321 time: 0.3286 s/iter data_time: 0.2347 s/iter total_throughput: 3115.83 samples/s lr: 2.74e-04 [09/27 07:42:07] lb.utils.events INFO: eta: 5:55:48 iteration: 245799/375342 consumed_samples: 251699200 total_loss: 3.336 time: 0.3286 s/iter data_time: 0.2163 s/iter total_throughput: 3115.82 samples/s lr: 2.74e-04 [09/27 07:42:40] lb.utils.events INFO: eta: 5:56:51 iteration: 245899/375342 consumed_samples: 251801600 total_loss: 3.338 time: 0.3286 s/iter data_time: 0.2256 s/iter total_throughput: 3115.81 samples/s lr: 2.73e-04 [09/27 07:43:13] lb.utils.events INFO: eta: 5:59:16 iteration: 245999/375342 consumed_samples: 251904000 total_loss: 3.308 time: 0.3286 s/iter data_time: 0.2239 s/iter total_throughput: 3115.82 samples/s lr: 2.73e-04 [09/27 07:43:46] lb.utils.events INFO: eta: 5:58:48 iteration: 246099/375342 consumed_samples: 252006400 total_loss: 3.336 time: 0.3286 s/iter data_time: 0.2099 s/iter total_throughput: 3115.81 samples/s lr: 2.72e-04 [09/27 07:44:18] lb.utils.events INFO: eta: 5:50:13 iteration: 246199/375342 consumed_samples: 252108800 total_loss: 3.348 time: 0.3286 s/iter data_time: 0.2038 s/iter total_throughput: 3115.82 samples/s lr: 2.72e-04 [09/27 07:44:51] lb.utils.events INFO: eta: 5:42:30 iteration: 246299/375342 consumed_samples: 252211200 total_loss: 3.34 time: 0.3286 s/iter data_time: 0.2075 s/iter total_throughput: 3115.82 samples/s lr: 2.72e-04 [09/27 07:45:24] lb.utils.events INFO: eta: 5:42:14 iteration: 246399/375342 consumed_samples: 252313600 total_loss: 3.364 time: 0.3286 s/iter data_time: 0.2161 s/iter total_throughput: 3115.81 samples/s lr: 2.71e-04 [09/27 07:45:57] lb.utils.events INFO: eta: 5:40:17 iteration: 246499/375342 consumed_samples: 252416000 total_loss: 3.355 time: 0.3286 s/iter data_time: 0.2022 s/iter total_throughput: 3115.81 samples/s lr: 2.71e-04 [09/27 07:46:30] lb.utils.events INFO: eta: 5:35:16 iteration: 246599/375342 consumed_samples: 252518400 total_loss: 3.31 time: 0.3286 s/iter data_time: 0.2000 s/iter total_throughput: 3115.81 samples/s lr: 2.71e-04 [09/27 07:47:03] lb.utils.events INFO: eta: 5:33:12 iteration: 246699/375342 consumed_samples: 252620800 total_loss: 3.317 time: 0.3286 s/iter data_time: 0.2240 s/iter total_throughput: 3115.81 samples/s lr: 2.70e-04 [09/27 07:47:36] lb.utils.events INFO: eta: 5:32:10 iteration: 246799/375342 consumed_samples: 252723200 total_loss: 3.324 time: 0.3286 s/iter data_time: 0.2321 s/iter total_throughput: 3115.80 samples/s lr: 2.70e-04 [09/27 07:48:09] lb.utils.events INFO: eta: 5:31:36 iteration: 246899/375342 consumed_samples: 252825600 total_loss: 3.317 time: 0.3286 s/iter data_time: 0.2050 s/iter total_throughput: 3115.80 samples/s lr: 2.70e-04 [09/27 07:48:42] lb.utils.events INFO: eta: 5:31:13 iteration: 246999/375342 consumed_samples: 252928000 total_loss: 3.317 time: 0.3286 s/iter data_time: 0.2162 s/iter total_throughput: 3115.80 samples/s lr: 2.69e-04 [09/27 07:49:14] lb.utils.events INFO: eta: 5:31:05 iteration: 247099/375342 consumed_samples: 253030400 total_loss: 3.344 time: 0.3286 s/iter data_time: 0.2253 s/iter total_throughput: 3115.82 samples/s lr: 2.69e-04 [09/27 07:49:47] lb.utils.events INFO: eta: 5:30:45 iteration: 247199/375342 consumed_samples: 253132800 total_loss: 3.323 time: 0.3286 s/iter data_time: 0.2032 s/iter total_throughput: 3115.82 samples/s lr: 2.68e-04 [09/27 07:50:20] lb.utils.events INFO: eta: 5:30:30 iteration: 247299/375342 consumed_samples: 253235200 total_loss: 3.316 time: 0.3286 s/iter data_time: 0.2126 s/iter total_throughput: 3115.81 samples/s lr: 2.68e-04 [09/27 07:50:53] lb.utils.events INFO: eta: 5:30:57 iteration: 247399/375342 consumed_samples: 253337600 total_loss: 3.345 time: 0.3286 s/iter data_time: 0.2189 s/iter total_throughput: 3115.80 samples/s lr: 2.68e-04 [09/27 07:51:26] lb.utils.events INFO: eta: 5:32:17 iteration: 247499/375342 consumed_samples: 253440000 total_loss: 3.343 time: 0.3286 s/iter data_time: 0.2303 s/iter total_throughput: 3115.79 samples/s lr: 2.67e-04 [09/27 07:51:59] lb.utils.events INFO: eta: 5:34:23 iteration: 247599/375342 consumed_samples: 253542400 total_loss: 3.335 time: 0.3286 s/iter data_time: 0.2187 s/iter total_throughput: 3115.80 samples/s lr: 2.67e-04 [09/27 07:52:32] lb.utils.events INFO: eta: 5:34:22 iteration: 247699/375342 consumed_samples: 253644800 total_loss: 3.326 time: 0.3286 s/iter data_time: 0.2194 s/iter total_throughput: 3115.81 samples/s lr: 2.67e-04 [09/27 07:53:04] lb.utils.events INFO: eta: 5:33:42 iteration: 247799/375342 consumed_samples: 253747200 total_loss: 3.297 time: 0.3286 s/iter data_time: 0.2108 s/iter total_throughput: 3115.81 samples/s lr: 2.66e-04 [09/27 07:53:37] lb.utils.events INFO: eta: 5:34:45 iteration: 247899/375342 consumed_samples: 253849600 total_loss: 3.288 time: 0.3286 s/iter data_time: 0.2205 s/iter total_throughput: 3115.81 samples/s lr: 2.66e-04 [09/27 07:54:10] lb.utils.events INFO: eta: 5:37:06 iteration: 247999/375342 consumed_samples: 253952000 total_loss: 3.294 time: 0.3286 s/iter data_time: 0.2255 s/iter total_throughput: 3115.82 samples/s lr: 2.66e-04 [09/27 07:54:43] lb.utils.events INFO: eta: 5:42:01 iteration: 248099/375342 consumed_samples: 254054400 total_loss: 3.321 time: 0.3286 s/iter data_time: 0.2569 s/iter total_throughput: 3115.83 samples/s lr: 2.65e-04 [09/27 07:55:16] lb.utils.events INFO: eta: 5:54:06 iteration: 248199/375342 consumed_samples: 254156800 total_loss: 3.336 time: 0.3286 s/iter data_time: 0.2615 s/iter total_throughput: 3115.81 samples/s lr: 2.65e-04 [09/27 07:55:49] lb.utils.events INFO: eta: 7:23:53 iteration: 248299/375342 consumed_samples: 254259200 total_loss: 3.353 time: 0.3286 s/iter data_time: 0.2560 s/iter total_throughput: 3115.82 samples/s lr: 2.64e-04 [09/27 07:56:21] lb.utils.events INFO: eta: 8:07:10 iteration: 248399/375342 consumed_samples: 254361600 total_loss: 3.332 time: 0.3286 s/iter data_time: 0.2473 s/iter total_throughput: 3115.82 samples/s lr: 2.64e-04 [09/27 07:56:54] lb.utils.events INFO: eta: 8:32:50 iteration: 248499/375342 consumed_samples: 254464000 total_loss: 3.329 time: 0.3286 s/iter data_time: 0.2369 s/iter total_throughput: 3115.82 samples/s lr: 2.64e-04 [09/27 07:57:27] lb.utils.events INFO: eta: 8:35:05 iteration: 248599/375342 consumed_samples: 254566400 total_loss: 3.334 time: 0.3286 s/iter data_time: 0.2388 s/iter total_throughput: 3115.83 samples/s lr: 2.63e-04 [09/27 07:58:00] lb.utils.events INFO: eta: 9:14:39 iteration: 248699/375342 consumed_samples: 254668800 total_loss: 3.327 time: 0.3286 s/iter data_time: 0.2303 s/iter total_throughput: 3115.84 samples/s lr: 2.63e-04 [09/27 07:58:32] lb.utils.events INFO: eta: 9:53:45 iteration: 248799/375342 consumed_samples: 254771200 total_loss: 3.322 time: 0.3286 s/iter data_time: 0.2342 s/iter total_throughput: 3115.85 samples/s lr: 2.63e-04 [09/27 07:59:05] lb.utils.events INFO: eta: 10:06:14 iteration: 248899/375342 consumed_samples: 254873600 total_loss: 3.314 time: 0.3286 s/iter data_time: 0.2556 s/iter total_throughput: 3115.86 samples/s lr: 2.62e-04 [09/27 07:59:38] lb.utils.events INFO: eta: 10:04:17 iteration: 248999/375342 consumed_samples: 254976000 total_loss: 3.328 time: 0.3286 s/iter data_time: 0.2349 s/iter total_throughput: 3115.86 samples/s lr: 2.62e-04 [09/27 08:00:10] lb.utils.events INFO: eta: 10:09:59 iteration: 249099/375342 consumed_samples: 255078400 total_loss: 3.321 time: 0.3286 s/iter data_time: 0.2456 s/iter total_throughput: 3115.87 samples/s lr: 2.62e-04 [09/27 08:00:44] lb.utils.events INFO: eta: 10:18:40 iteration: 249199/375342 consumed_samples: 255180800 total_loss: 3.298 time: 0.3286 s/iter data_time: 0.2370 s/iter total_throughput: 3115.85 samples/s lr: 2.61e-04 [09/27 08:01:16] lb.utils.events INFO: eta: 10:03:06 iteration: 249299/375342 consumed_samples: 255283200 total_loss: 3.307 time: 0.3286 s/iter data_time: 0.2266 s/iter total_throughput: 3115.86 samples/s lr: 2.61e-04 [09/27 08:01:49] lb.utils.events INFO: eta: 10:02:37 iteration: 249399/375342 consumed_samples: 255385600 total_loss: 3.319 time: 0.3286 s/iter data_time: 0.2202 s/iter total_throughput: 3115.86 samples/s lr: 2.60e-04 [09/27 08:02:22] lb.utils.events INFO: eta: 9:18:18 iteration: 249499/375342 consumed_samples: 255488000 total_loss: 3.309 time: 0.3286 s/iter data_time: 0.2050 s/iter total_throughput: 3115.86 samples/s lr: 2.60e-04 [09/27 08:02:55] lb.utils.events INFO: eta: 8:59:01 iteration: 249599/375342 consumed_samples: 255590400 total_loss: 3.312 time: 0.3286 s/iter data_time: 0.2367 s/iter total_throughput: 3115.87 samples/s lr: 2.60e-04 [09/27 08:03:27] lb.utils.events INFO: eta: 8:29:10 iteration: 249699/375342 consumed_samples: 255692800 total_loss: 3.314 time: 0.3286 s/iter data_time: 0.2438 s/iter total_throughput: 3115.88 samples/s lr: 2.59e-04 [09/27 08:04:00] lb.utils.events INFO: eta: 7:57:34 iteration: 249799/375342 consumed_samples: 255795200 total_loss: 3.33 time: 0.3286 s/iter data_time: 0.2312 s/iter total_throughput: 3115.87 samples/s lr: 2.59e-04 [09/27 08:04:33] lb.utils.events INFO: eta: 7:43:26 iteration: 249899/375342 consumed_samples: 255897600 total_loss: 3.339 time: 0.3286 s/iter data_time: 0.2382 s/iter total_throughput: 3115.88 samples/s lr: 2.59e-04 [09/27 08:05:06] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0249999 [09/27 08:05:07] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/27 08:05:07] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/27 08:05:11] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0880 s/iter. Inference: 0.1544 s/iter. Eval: 0.0020 s/iter. Total: 0.2444 s/iter. ETA=0:00:09 [09/27 08:05:17] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1486 s/iter. Inference: 0.1509 s/iter. Eval: 0.0020 s/iter. Total: 0.3015 s/iter. ETA=0:00:05 [09/27 08:05:22] lb.evaluation.evaluator INFO: Inference done 49152/50000. Dataloading: 0.1346 s/iter. Inference: 0.1496 s/iter. Eval: 0.0021 s/iter. Total: 0.2864 s/iter. ETA=0:00:00 [09/27 08:05:22] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/27 08:05:22] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.755783 (0.000255 s / iter per device, on 8 devices) [09/27 08:05:22] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000133 s / iter per device, on 8 devices) [09/27 08:05:22] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/27 08:05:22] lb.evaluation.utils INFO: copypaste: Acc@1=77.35600000000001 [09/27 08:05:22] lb.evaluation.utils INFO: copypaste: Acc@5=93.626 [09/27 08:05:22] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 77.35600, better than last best score 77.17200 @ iteration 244999. [09/27 08:05:22] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/27 08:05:23] lb.utils.events INFO: eta: 6:56:01 iteration: 249999/375342 consumed_samples: 256000000 total_loss: 3.336 time: 0.3286 s/iter data_time: 0.2275 s/iter total_throughput: 3115.87 samples/s lr: 2.58e-04 [09/27 08:05:54] lb.utils.events INFO: eta: 6:10:40 iteration: 250099/375342 consumed_samples: 256102400 total_loss: 3.341 time: 0.3286 s/iter data_time: 0.2303 s/iter total_throughput: 3115.94 samples/s lr: 2.58e-04 [09/27 08:06:27] lb.utils.events INFO: eta: 5:54:24 iteration: 250199/375342 consumed_samples: 256204800 total_loss: 3.321 time: 0.3286 s/iter data_time: 0.2272 s/iter total_throughput: 3115.94 samples/s lr: 2.58e-04 [09/27 08:06:59] lb.utils.events INFO: eta: 5:47:58 iteration: 250299/375342 consumed_samples: 256307200 total_loss: 3.308 time: 0.3286 s/iter data_time: 0.2232 s/iter total_throughput: 3115.95 samples/s lr: 2.57e-04 [09/27 08:07:32] lb.utils.events INFO: eta: 5:46:41 iteration: 250399/375342 consumed_samples: 256409600 total_loss: 3.316 time: 0.3286 s/iter data_time: 0.2360 s/iter total_throughput: 3115.95 samples/s lr: 2.57e-04 [09/27 08:08:05] lb.utils.events INFO: eta: 5:54:38 iteration: 250499/375342 consumed_samples: 256512000 total_loss: 3.315 time: 0.3286 s/iter data_time: 0.2225 s/iter total_throughput: 3115.95 samples/s lr: 2.57e-04 [09/27 08:08:38] lb.utils.events INFO: eta: 5:44:03 iteration: 250599/375342 consumed_samples: 256614400 total_loss: 3.308 time: 0.3286 s/iter data_time: 0.2064 s/iter total_throughput: 3115.94 samples/s lr: 2.56e-04 [09/27 08:09:11] lb.utils.events INFO: eta: 5:39:05 iteration: 250699/375342 consumed_samples: 256716800 total_loss: 3.32 time: 0.3286 s/iter data_time: 0.1979 s/iter total_throughput: 3115.95 samples/s lr: 2.56e-04 [09/27 08:09:43] lb.utils.events INFO: eta: 5:34:04 iteration: 250799/375342 consumed_samples: 256819200 total_loss: 3.317 time: 0.3286 s/iter data_time: 0.2213 s/iter total_throughput: 3115.98 samples/s lr: 2.55e-04 [09/27 08:10:16] lb.utils.events INFO: eta: 5:31:45 iteration: 250899/375342 consumed_samples: 256921600 total_loss: 3.314 time: 0.3286 s/iter data_time: 0.2202 s/iter total_throughput: 3115.99 samples/s lr: 2.55e-04 [09/27 08:10:49] lb.utils.events INFO: eta: 5:30:09 iteration: 250999/375342 consumed_samples: 257024000 total_loss: 3.329 time: 0.3286 s/iter data_time: 0.2058 s/iter total_throughput: 3115.99 samples/s lr: 2.55e-04 [09/27 08:11:21] lb.utils.events INFO: eta: 5:28:09 iteration: 251099/375342 consumed_samples: 257126400 total_loss: 3.325 time: 0.3286 s/iter data_time: 0.1951 s/iter total_throughput: 3116.01 samples/s lr: 2.54e-04 [09/27 08:11:54] lb.utils.events INFO: eta: 5:26:41 iteration: 251199/375342 consumed_samples: 257228800 total_loss: 3.303 time: 0.3286 s/iter data_time: 0.2128 s/iter total_throughput: 3116.01 samples/s lr: 2.54e-04 [09/27 08:12:26] lb.utils.events INFO: eta: 5:23:50 iteration: 251299/375342 consumed_samples: 257331200 total_loss: 3.312 time: 0.3286 s/iter data_time: 0.2080 s/iter total_throughput: 3116.01 samples/s lr: 2.54e-04 [09/27 08:12:59] lb.utils.events INFO: eta: 5:22:20 iteration: 251399/375342 consumed_samples: 257433600 total_loss: 3.328 time: 0.3286 s/iter data_time: 0.2114 s/iter total_throughput: 3116.02 samples/s lr: 2.53e-04 [09/27 08:13:31] lb.utils.events INFO: eta: 5:22:19 iteration: 251499/375342 consumed_samples: 257536000 total_loss: 3.321 time: 0.3286 s/iter data_time: 0.2298 s/iter total_throughput: 3116.04 samples/s lr: 2.53e-04 [09/27 08:14:03] lb.utils.events INFO: eta: 5:25:24 iteration: 251599/375342 consumed_samples: 257638400 total_loss: 3.315 time: 0.3286 s/iter data_time: 0.2221 s/iter total_throughput: 3116.08 samples/s lr: 2.53e-04 [09/27 08:14:36] lb.utils.events INFO: eta: 5:27:10 iteration: 251699/375342 consumed_samples: 257740800 total_loss: 3.313 time: 0.3286 s/iter data_time: 0.2221 s/iter total_throughput: 3116.09 samples/s lr: 2.52e-04 [09/27 08:15:08] lb.utils.events INFO: eta: 5:30:16 iteration: 251799/375342 consumed_samples: 257843200 total_loss: 3.332 time: 0.3286 s/iter data_time: 0.2141 s/iter total_throughput: 3116.11 samples/s lr: 2.52e-04 [09/27 08:15:40] lb.utils.events INFO: eta: 5:30:31 iteration: 251899/375342 consumed_samples: 257945600 total_loss: 3.323 time: 0.3286 s/iter data_time: 0.2112 s/iter total_throughput: 3116.13 samples/s lr: 2.52e-04 [09/27 08:16:13] lb.utils.events INFO: eta: 5:31:29 iteration: 251999/375342 consumed_samples: 258048000 total_loss: 3.284 time: 0.3286 s/iter data_time: 0.2140 s/iter total_throughput: 3116.15 samples/s lr: 2.51e-04 [09/27 08:16:46] lb.utils.events INFO: eta: 5:34:55 iteration: 252099/375342 consumed_samples: 258150400 total_loss: 3.301 time: 0.3286 s/iter data_time: 0.2280 s/iter total_throughput: 3116.16 samples/s lr: 2.51e-04 [09/27 08:17:19] lb.utils.events INFO: eta: 5:40:45 iteration: 252199/375342 consumed_samples: 258252800 total_loss: 3.327 time: 0.3286 s/iter data_time: 0.2464 s/iter total_throughput: 3116.15 samples/s lr: 2.50e-04 [09/27 08:17:51] lb.utils.events INFO: eta: 5:48:36 iteration: 252299/375342 consumed_samples: 258355200 total_loss: 3.345 time: 0.3286 s/iter data_time: 0.2182 s/iter total_throughput: 3116.16 samples/s lr: 2.50e-04 [09/27 08:18:24] lb.utils.events INFO: eta: 5:56:21 iteration: 252399/375342 consumed_samples: 258457600 total_loss: 3.334 time: 0.3286 s/iter data_time: 0.2334 s/iter total_throughput: 3116.15 samples/s lr: 2.50e-04 [09/27 08:18:58] lb.utils.events INFO: eta: 5:49:39 iteration: 252499/375342 consumed_samples: 258560000 total_loss: 3.31 time: 0.3286 s/iter data_time: 0.2190 s/iter total_throughput: 3116.12 samples/s lr: 2.49e-04 [09/27 08:19:31] lb.utils.events INFO: eta: 5:47:45 iteration: 252599/375342 consumed_samples: 258662400 total_loss: 3.304 time: 0.3286 s/iter data_time: 0.2329 s/iter total_throughput: 3116.10 samples/s lr: 2.49e-04 [09/27 08:20:05] lb.utils.events INFO: eta: 5:39:14 iteration: 252699/375342 consumed_samples: 258764800 total_loss: 3.301 time: 0.3286 s/iter data_time: 0.2104 s/iter total_throughput: 3116.07 samples/s lr: 2.49e-04 [09/27 08:20:39] lb.utils.events INFO: eta: 5:32:43 iteration: 252799/375342 consumed_samples: 258867200 total_loss: 3.293 time: 0.3286 s/iter data_time: 0.2101 s/iter total_throughput: 3116.05 samples/s lr: 2.48e-04 [09/27 08:21:11] lb.utils.events INFO: eta: 5:32:39 iteration: 252899/375342 consumed_samples: 258969600 total_loss: 3.269 time: 0.3286 s/iter data_time: 0.2228 s/iter total_throughput: 3116.05 samples/s lr: 2.48e-04 [09/27 08:21:44] lb.utils.events INFO: eta: 5:33:13 iteration: 252999/375342 consumed_samples: 259072000 total_loss: 3.279 time: 0.3286 s/iter data_time: 0.2225 s/iter total_throughput: 3116.04 samples/s lr: 2.48e-04 [09/27 08:22:18] lb.utils.events INFO: eta: 5:32:32 iteration: 253099/375342 consumed_samples: 259174400 total_loss: 3.294 time: 0.3286 s/iter data_time: 0.2390 s/iter total_throughput: 3116.02 samples/s lr: 2.47e-04 [09/27 08:22:51] lb.utils.events INFO: eta: 5:30:01 iteration: 253199/375342 consumed_samples: 259276800 total_loss: 3.317 time: 0.3286 s/iter data_time: 0.2213 s/iter total_throughput: 3116.02 samples/s lr: 2.47e-04 [09/27 08:23:24] lb.utils.events INFO: eta: 5:27:39 iteration: 253299/375342 consumed_samples: 259379200 total_loss: 3.321 time: 0.3286 s/iter data_time: 0.2232 s/iter total_throughput: 3116.01 samples/s lr: 2.47e-04 [09/27 08:23:57] lb.utils.events INFO: eta: 5:26:22 iteration: 253399/375342 consumed_samples: 259481600 total_loss: 3.315 time: 0.3286 s/iter data_time: 0.2282 s/iter total_throughput: 3116.01 samples/s lr: 2.46e-04 [09/27 08:24:30] lb.utils.events INFO: eta: 5:27:39 iteration: 253499/375342 consumed_samples: 259584000 total_loss: 3.312 time: 0.3286 s/iter data_time: 0.2273 s/iter total_throughput: 3116.00 samples/s lr: 2.46e-04 [09/27 08:25:03] lb.utils.events INFO: eta: 5:25:50 iteration: 253599/375342 consumed_samples: 259686400 total_loss: 3.312 time: 0.3286 s/iter data_time: 0.2228 s/iter total_throughput: 3115.98 samples/s lr: 2.46e-04 [09/27 08:25:36] lb.utils.events INFO: eta: 5:23:43 iteration: 253699/375342 consumed_samples: 259788800 total_loss: 3.316 time: 0.3286 s/iter data_time: 0.2050 s/iter total_throughput: 3115.97 samples/s lr: 2.45e-04 [09/27 08:26:09] lb.utils.events INFO: eta: 5:22:51 iteration: 253799/375342 consumed_samples: 259891200 total_loss: 3.31 time: 0.3286 s/iter data_time: 0.2001 s/iter total_throughput: 3115.97 samples/s lr: 2.45e-04 [09/27 08:26:43] lb.utils.events INFO: eta: 5:20:51 iteration: 253899/375342 consumed_samples: 259993600 total_loss: 3.305 time: 0.3286 s/iter data_time: 0.2043 s/iter total_throughput: 3115.95 samples/s lr: 2.44e-04 [09/27 08:27:16] lb.utils.events INFO: eta: 5:20:24 iteration: 253999/375342 consumed_samples: 260096000 total_loss: 3.294 time: 0.3286 s/iter data_time: 0.2513 s/iter total_throughput: 3115.94 samples/s lr: 2.44e-04 [09/27 08:27:50] lb.utils.events INFO: eta: 5:17:31 iteration: 254099/375342 consumed_samples: 260198400 total_loss: 3.284 time: 0.3286 s/iter data_time: 0.2129 s/iter total_throughput: 3115.91 samples/s lr: 2.44e-04 [09/27 08:28:23] lb.utils.events INFO: eta: 5:16:55 iteration: 254199/375342 consumed_samples: 260300800 total_loss: 3.266 time: 0.3286 s/iter data_time: 0.2249 s/iter total_throughput: 3115.90 samples/s lr: 2.43e-04 [09/27 08:28:56] lb.utils.events INFO: eta: 5:17:52 iteration: 254299/375342 consumed_samples: 260403200 total_loss: 3.289 time: 0.3286 s/iter data_time: 0.2292 s/iter total_throughput: 3115.89 samples/s lr: 2.43e-04 [09/27 08:29:29] lb.utils.events INFO: eta: 5:17:44 iteration: 254399/375342 consumed_samples: 260505600 total_loss: 3.305 time: 0.3286 s/iter data_time: 0.2234 s/iter total_throughput: 3115.89 samples/s lr: 2.43e-04 [09/27 08:30:02] lb.utils.events INFO: eta: 5:16:09 iteration: 254499/375342 consumed_samples: 260608000 total_loss: 3.305 time: 0.3286 s/iter data_time: 0.2098 s/iter total_throughput: 3115.88 samples/s lr: 2.42e-04 [09/27 08:30:35] lb.utils.events INFO: eta: 5:15:46 iteration: 254599/375342 consumed_samples: 260710400 total_loss: 3.326 time: 0.3286 s/iter data_time: 0.2246 s/iter total_throughput: 3115.86 samples/s lr: 2.42e-04 [09/27 08:31:09] lb.utils.events INFO: eta: 5:15:55 iteration: 254699/375342 consumed_samples: 260812800 total_loss: 3.33 time: 0.3286 s/iter data_time: 0.2098 s/iter total_throughput: 3115.84 samples/s lr: 2.42e-04 [09/27 08:31:42] lb.utils.events INFO: eta: 5:16:10 iteration: 254799/375342 consumed_samples: 260915200 total_loss: 3.315 time: 0.3286 s/iter data_time: 0.1989 s/iter total_throughput: 3115.84 samples/s lr: 2.41e-04 [09/27 08:32:14] lb.utils.events INFO: eta: 5:15:26 iteration: 254899/375342 consumed_samples: 261017600 total_loss: 3.294 time: 0.3286 s/iter data_time: 0.2128 s/iter total_throughput: 3115.84 samples/s lr: 2.41e-04 [09/27 08:32:48] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0254999 [09/27 08:32:49] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/27 08:32:49] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/27 08:32:53] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0875 s/iter. Inference: 0.1532 s/iter. Eval: 0.0021 s/iter. Total: 0.2428 s/iter. ETA=0:00:08 [09/27 08:32:59] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1514 s/iter. Inference: 0.1502 s/iter. Eval: 0.0020 s/iter. Total: 0.3036 s/iter. ETA=0:00:05 [09/27 08:33:04] lb.evaluation.evaluator INFO: Inference done 49152/50000. Dataloading: 0.1318 s/iter. Inference: 0.1519 s/iter. Eval: 0.0021 s/iter. Total: 0.2859 s/iter. ETA=0:00:00 [09/27 08:33:04] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/27 08:33:04] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.739950 (0.000255 s / iter per device, on 8 devices) [09/27 08:33:04] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000135 s / iter per device, on 8 devices) [09/27 08:33:04] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/27 08:33:04] lb.evaluation.utils INFO: copypaste: Acc@1=77.73400000000001 [09/27 08:33:04] lb.evaluation.utils INFO: copypaste: Acc@5=93.83200000000001 [09/27 08:33:04] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 77.73400, better than last best score 77.35600 @ iteration 249999. [09/27 08:33:04] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/27 08:33:05] lb.utils.events INFO: eta: 5:13:06 iteration: 254999/375342 consumed_samples: 261120000 total_loss: 3.338 time: 0.3286 s/iter data_time: 0.2030 s/iter total_throughput: 3115.82 samples/s lr: 2.41e-04 [09/27 08:33:36] lb.utils.events INFO: eta: 5:12:46 iteration: 255099/375342 consumed_samples: 261222400 total_loss: 3.332 time: 0.3286 s/iter data_time: 0.2231 s/iter total_throughput: 3115.88 samples/s lr: 2.40e-04 [09/27 08:34:09] lb.utils.events INFO: eta: 5:13:17 iteration: 255199/375342 consumed_samples: 261324800 total_loss: 3.317 time: 0.3286 s/iter data_time: 0.2380 s/iter total_throughput: 3115.87 samples/s lr: 2.40e-04 [09/27 08:34:43] lb.utils.events INFO: eta: 5:13:10 iteration: 255299/375342 consumed_samples: 261427200 total_loss: 3.297 time: 0.3286 s/iter data_time: 0.2201 s/iter total_throughput: 3115.85 samples/s lr: 2.40e-04 [09/27 08:35:16] lb.utils.events INFO: eta: 5:12:15 iteration: 255399/375342 consumed_samples: 261529600 total_loss: 3.281 time: 0.3286 s/iter data_time: 0.1959 s/iter total_throughput: 3115.84 samples/s lr: 2.39e-04 [09/27 08:35:49] lb.utils.events INFO: eta: 5:11:59 iteration: 255499/375342 consumed_samples: 261632000 total_loss: 3.27 time: 0.3286 s/iter data_time: 0.2238 s/iter total_throughput: 3115.84 samples/s lr: 2.39e-04 [09/27 08:36:22] lb.utils.events INFO: eta: 5:13:08 iteration: 255599/375342 consumed_samples: 261734400 total_loss: 3.268 time: 0.3286 s/iter data_time: 0.2263 s/iter total_throughput: 3115.84 samples/s lr: 2.38e-04 [09/27 08:36:55] lb.utils.events INFO: eta: 5:13:22 iteration: 255699/375342 consumed_samples: 261836800 total_loss: 3.298 time: 0.3286 s/iter data_time: 0.2295 s/iter total_throughput: 3115.82 samples/s lr: 2.38e-04 [09/27 08:37:28] lb.utils.events INFO: eta: 5:14:51 iteration: 255799/375342 consumed_samples: 261939200 total_loss: 3.309 time: 0.3286 s/iter data_time: 0.2287 s/iter total_throughput: 3115.82 samples/s lr: 2.38e-04 [09/27 08:38:01] lb.utils.events INFO: eta: 5:16:36 iteration: 255899/375342 consumed_samples: 262041600 total_loss: 3.303 time: 0.3286 s/iter data_time: 0.2105 s/iter total_throughput: 3115.81 samples/s lr: 2.37e-04 [09/27 08:38:34] lb.utils.events INFO: eta: 5:21:13 iteration: 255999/375342 consumed_samples: 262144000 total_loss: 3.301 time: 0.3286 s/iter data_time: 0.2291 s/iter total_throughput: 3115.81 samples/s lr: 2.37e-04 [09/27 08:39:07] lb.utils.events INFO: eta: 5:27:28 iteration: 256099/375342 consumed_samples: 262246400 total_loss: 3.3 time: 0.3286 s/iter data_time: 0.2089 s/iter total_throughput: 3115.80 samples/s lr: 2.37e-04 [09/27 08:39:40] lb.utils.events INFO: eta: 5:24:18 iteration: 256199/375342 consumed_samples: 262348800 total_loss: 3.306 time: 0.3286 s/iter data_time: 0.2332 s/iter total_throughput: 3115.79 samples/s lr: 2.36e-04 [09/27 08:40:13] lb.utils.events INFO: eta: 5:20:35 iteration: 256299/375342 consumed_samples: 262451200 total_loss: 3.3 time: 0.3286 s/iter data_time: 0.2067 s/iter total_throughput: 3115.78 samples/s lr: 2.36e-04 [09/27 08:40:46] lb.utils.events INFO: eta: 5:22:30 iteration: 256399/375342 consumed_samples: 262553600 total_loss: 3.29 time: 0.3287 s/iter data_time: 0.2075 s/iter total_throughput: 3115.77 samples/s lr: 2.36e-04 [09/27 08:41:20] lb.utils.events INFO: eta: 5:18:08 iteration: 256499/375342 consumed_samples: 262656000 total_loss: 3.289 time: 0.3287 s/iter data_time: 0.1999 s/iter total_throughput: 3115.75 samples/s lr: 2.35e-04 [09/27 08:41:53] lb.utils.events INFO: eta: 5:14:37 iteration: 256599/375342 consumed_samples: 262758400 total_loss: 3.273 time: 0.3287 s/iter data_time: 0.1993 s/iter total_throughput: 3115.75 samples/s lr: 2.35e-04 [09/27 08:42:26] lb.utils.events INFO: eta: 5:12:52 iteration: 256699/375342 consumed_samples: 262860800 total_loss: 3.309 time: 0.3287 s/iter data_time: 0.2000 s/iter total_throughput: 3115.74 samples/s lr: 2.35e-04 [09/27 08:42:59] lb.utils.events INFO: eta: 5:10:43 iteration: 256799/375342 consumed_samples: 262963200 total_loss: 3.309 time: 0.3287 s/iter data_time: 0.2158 s/iter total_throughput: 3115.72 samples/s lr: 2.34e-04 [09/27 08:43:33] lb.utils.events INFO: eta: 5:08:59 iteration: 256899/375342 consumed_samples: 263065600 total_loss: 3.271 time: 0.3287 s/iter data_time: 0.2113 s/iter total_throughput: 3115.71 samples/s lr: 2.34e-04 [09/27 08:44:06] lb.utils.events INFO: eta: 5:07:48 iteration: 256999/375342 consumed_samples: 263168000 total_loss: 3.277 time: 0.3287 s/iter data_time: 0.2145 s/iter total_throughput: 3115.69 samples/s lr: 2.34e-04 [09/27 08:44:39] lb.utils.events INFO: eta: 5:06:19 iteration: 257099/375342 consumed_samples: 263270400 total_loss: 3.293 time: 0.3287 s/iter data_time: 0.2067 s/iter total_throughput: 3115.67 samples/s lr: 2.33e-04 [09/27 08:45:12] lb.utils.events INFO: eta: 5:03:43 iteration: 257199/375342 consumed_samples: 263372800 total_loss: 3.294 time: 0.3287 s/iter data_time: 0.2044 s/iter total_throughput: 3115.67 samples/s lr: 2.33e-04 [09/27 08:45:45] lb.utils.events INFO: eta: 5:02:57 iteration: 257299/375342 consumed_samples: 263475200 total_loss: 3.28 time: 0.3287 s/iter data_time: 0.2101 s/iter total_throughput: 3115.66 samples/s lr: 2.33e-04 [09/27 08:46:19] lb.utils.events INFO: eta: 5:02:32 iteration: 257399/375342 consumed_samples: 263577600 total_loss: 3.274 time: 0.3287 s/iter data_time: 0.2064 s/iter total_throughput: 3115.65 samples/s lr: 2.32e-04 [09/27 08:46:52] lb.utils.events INFO: eta: 5:02:26 iteration: 257499/375342 consumed_samples: 263680000 total_loss: 3.294 time: 0.3287 s/iter data_time: 0.2132 s/iter total_throughput: 3115.62 samples/s lr: 2.32e-04 [09/27 08:47:25] lb.utils.events INFO: eta: 5:02:06 iteration: 257599/375342 consumed_samples: 263782400 total_loss: 3.303 time: 0.3287 s/iter data_time: 0.2085 s/iter total_throughput: 3115.61 samples/s lr: 2.32e-04 [09/27 08:47:59] lb.utils.events INFO: eta: 5:01:52 iteration: 257699/375342 consumed_samples: 263884800 total_loss: 3.301 time: 0.3287 s/iter data_time: 0.2177 s/iter total_throughput: 3115.60 samples/s lr: 2.31e-04 [09/27 08:48:32] lb.utils.events INFO: eta: 5:01:56 iteration: 257799/375342 consumed_samples: 263987200 total_loss: 3.284 time: 0.3287 s/iter data_time: 0.2025 s/iter total_throughput: 3115.59 samples/s lr: 2.31e-04 [09/27 08:49:04] lb.utils.events INFO: eta: 5:01:27 iteration: 257899/375342 consumed_samples: 264089600 total_loss: 3.274 time: 0.3287 s/iter data_time: 0.2080 s/iter total_throughput: 3115.60 samples/s lr: 2.31e-04 [09/27 08:49:37] lb.utils.events INFO: eta: 5:01:51 iteration: 257999/375342 consumed_samples: 264192000 total_loss: 3.278 time: 0.3287 s/iter data_time: 0.2263 s/iter total_throughput: 3115.59 samples/s lr: 2.30e-04 [09/27 08:50:11] lb.utils.events INFO: eta: 5:01:06 iteration: 258099/375342 consumed_samples: 264294400 total_loss: 3.294 time: 0.3287 s/iter data_time: 0.2108 s/iter total_throughput: 3115.56 samples/s lr: 2.30e-04 [09/27 08:50:44] lb.utils.events INFO: eta: 5:00:55 iteration: 258199/375342 consumed_samples: 264396800 total_loss: 3.293 time: 0.3287 s/iter data_time: 0.2062 s/iter total_throughput: 3115.54 samples/s lr: 2.29e-04 [09/27 08:51:18] lb.utils.events INFO: eta: 5:00:12 iteration: 258299/375342 consumed_samples: 264499200 total_loss: 3.283 time: 0.3287 s/iter data_time: 0.2207 s/iter total_throughput: 3115.53 samples/s lr: 2.29e-04 [09/27 08:51:50] lb.utils.events INFO: eta: 5:00:14 iteration: 258399/375342 consumed_samples: 264601600 total_loss: 3.284 time: 0.3287 s/iter data_time: 0.2055 s/iter total_throughput: 3115.53 samples/s lr: 2.29e-04 [09/27 08:52:24] lb.utils.events INFO: eta: 5:00:22 iteration: 258499/375342 consumed_samples: 264704000 total_loss: 3.278 time: 0.3287 s/iter data_time: 0.2234 s/iter total_throughput: 3115.52 samples/s lr: 2.28e-04 [09/27 08:52:57] lb.utils.events INFO: eta: 5:00:30 iteration: 258599/375342 consumed_samples: 264806400 total_loss: 3.288 time: 0.3287 s/iter data_time: 0.2249 s/iter total_throughput: 3115.50 samples/s lr: 2.28e-04 [09/27 08:53:30] lb.utils.events INFO: eta: 5:01:23 iteration: 258699/375342 consumed_samples: 264908800 total_loss: 3.298 time: 0.3287 s/iter data_time: 0.2392 s/iter total_throughput: 3115.50 samples/s lr: 2.28e-04 [09/27 08:54:03] lb.utils.events INFO: eta: 5:02:17 iteration: 258799/375342 consumed_samples: 265011200 total_loss: 3.298 time: 0.3287 s/iter data_time: 0.2137 s/iter total_throughput: 3115.49 samples/s lr: 2.27e-04 [09/27 08:54:36] lb.utils.events INFO: eta: 5:00:52 iteration: 258899/375342 consumed_samples: 265113600 total_loss: 3.299 time: 0.3287 s/iter data_time: 0.2061 s/iter total_throughput: 3115.48 samples/s lr: 2.27e-04 [09/27 08:55:10] lb.utils.events INFO: eta: 5:00:56 iteration: 258999/375342 consumed_samples: 265216000 total_loss: 3.282 time: 0.3287 s/iter data_time: 0.2390 s/iter total_throughput: 3115.47 samples/s lr: 2.27e-04 [09/27 08:55:43] lb.utils.events INFO: eta: 5:01:33 iteration: 259099/375342 consumed_samples: 265318400 total_loss: 3.27 time: 0.3287 s/iter data_time: 0.2052 s/iter total_throughput: 3115.46 samples/s lr: 2.26e-04 [09/27 08:56:16] lb.utils.events INFO: eta: 5:01:56 iteration: 259199/375342 consumed_samples: 265420800 total_loss: 3.275 time: 0.3287 s/iter data_time: 0.2279 s/iter total_throughput: 3115.45 samples/s lr: 2.26e-04 [09/27 08:56:49] lb.utils.events INFO: eta: 5:04:51 iteration: 259299/375342 consumed_samples: 265523200 total_loss: 3.273 time: 0.3287 s/iter data_time: 0.2315 s/iter total_throughput: 3115.45 samples/s lr: 2.26e-04 [09/27 08:57:22] lb.utils.events INFO: eta: 5:05:44 iteration: 259399/375342 consumed_samples: 265625600 total_loss: 3.278 time: 0.3287 s/iter data_time: 0.2115 s/iter total_throughput: 3115.43 samples/s lr: 2.25e-04 [09/27 08:57:55] lb.utils.events INFO: eta: 5:05:28 iteration: 259499/375342 consumed_samples: 265728000 total_loss: 3.275 time: 0.3287 s/iter data_time: 0.2136 s/iter total_throughput: 3115.42 samples/s lr: 2.25e-04 [09/27 08:58:28] lb.utils.events INFO: eta: 5:03:55 iteration: 259599/375342 consumed_samples: 265830400 total_loss: 3.278 time: 0.3287 s/iter data_time: 0.2062 s/iter total_throughput: 3115.41 samples/s lr: 2.25e-04 [09/27 08:59:01] lb.utils.events INFO: eta: 5:01:48 iteration: 259699/375342 consumed_samples: 265932800 total_loss: 3.288 time: 0.3287 s/iter data_time: 0.2060 s/iter total_throughput: 3115.40 samples/s lr: 2.24e-04 [09/27 08:59:34] lb.utils.events INFO: eta: 5:00:16 iteration: 259799/375342 consumed_samples: 266035200 total_loss: 3.271 time: 0.3287 s/iter data_time: 0.2190 s/iter total_throughput: 3115.41 samples/s lr: 2.24e-04 [09/27 09:00:07] lb.utils.events INFO: eta: 5:02:50 iteration: 259899/375342 consumed_samples: 266137600 total_loss: 3.272 time: 0.3287 s/iter data_time: 0.2323 s/iter total_throughput: 3115.39 samples/s lr: 2.24e-04 [09/27 09:00:40] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0259999 [09/27 09:00:41] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/27 09:00:41] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/27 09:00:45] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0830 s/iter. Inference: 0.1493 s/iter. Eval: 0.0021 s/iter. Total: 0.2344 s/iter. ETA=0:00:08 [09/27 09:00:51] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1439 s/iter. Inference: 0.1507 s/iter. Eval: 0.0020 s/iter. Total: 0.2966 s/iter. ETA=0:00:05 [09/27 09:00:56] lb.evaluation.evaluator INFO: Inference done 49152/50000. Dataloading: 0.1316 s/iter. Inference: 0.1492 s/iter. Eval: 0.0020 s/iter. Total: 0.2828 s/iter. ETA=0:00:00 [09/27 09:00:56] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/27 09:00:56] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.582516 (0.000252 s / iter per device, on 8 devices) [09/27 09:00:56] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000132 s / iter per device, on 8 devices) [09/27 09:00:56] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/27 09:00:56] lb.evaluation.utils INFO: copypaste: Acc@1=78.058 [09/27 09:00:56] lb.evaluation.utils INFO: copypaste: Acc@5=93.798 [09/27 09:00:56] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 78.05800, better than last best score 77.73400 @ iteration 254999. [09/27 09:00:56] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/27 09:00:57] lb.utils.events INFO: eta: 5:01:59 iteration: 259999/375342 consumed_samples: 266240000 total_loss: 3.297 time: 0.3287 s/iter data_time: 0.2136 s/iter total_throughput: 3115.39 samples/s lr: 2.23e-04 [09/27 09:01:28] lb.utils.events INFO: eta: 5:02:14 iteration: 260099/375342 consumed_samples: 266342400 total_loss: 3.311 time: 0.3287 s/iter data_time: 0.2194 s/iter total_throughput: 3115.45 samples/s lr: 2.23e-04 [09/27 09:02:01] lb.utils.events INFO: eta: 5:02:35 iteration: 260199/375342 consumed_samples: 266444800 total_loss: 3.299 time: 0.3287 s/iter data_time: 0.2185 s/iter total_throughput: 3115.45 samples/s lr: 2.23e-04 [09/27 09:02:34] lb.utils.events INFO: eta: 5:00:27 iteration: 260299/375342 consumed_samples: 266547200 total_loss: 3.272 time: 0.3287 s/iter data_time: 0.1994 s/iter total_throughput: 3115.44 samples/s lr: 2.22e-04 [09/27 09:03:07] lb.utils.events INFO: eta: 5:00:42 iteration: 260399/375342 consumed_samples: 266649600 total_loss: 3.271 time: 0.3287 s/iter data_time: 0.2204 s/iter total_throughput: 3115.43 samples/s lr: 2.22e-04 [09/27 09:03:41] lb.utils.events INFO: eta: 5:03:43 iteration: 260499/375342 consumed_samples: 266752000 total_loss: 3.286 time: 0.3287 s/iter data_time: 0.2568 s/iter total_throughput: 3115.42 samples/s lr: 2.22e-04 [09/27 09:04:13] lb.utils.events INFO: eta: 5:07:26 iteration: 260599/375342 consumed_samples: 266854400 total_loss: 3.268 time: 0.3287 s/iter data_time: 0.2268 s/iter total_throughput: 3115.43 samples/s lr: 2.21e-04 [09/27 09:04:46] lb.utils.events INFO: eta: 5:10:01 iteration: 260699/375342 consumed_samples: 266956800 total_loss: 3.254 time: 0.3287 s/iter data_time: 0.2379 s/iter total_throughput: 3115.44 samples/s lr: 2.21e-04 [09/27 09:05:19] lb.utils.events INFO: eta: 5:09:44 iteration: 260799/375342 consumed_samples: 267059200 total_loss: 3.275 time: 0.3287 s/iter data_time: 0.2099 s/iter total_throughput: 3115.43 samples/s lr: 2.21e-04 [09/27 09:05:52] lb.utils.events INFO: eta: 5:07:30 iteration: 260899/375342 consumed_samples: 267161600 total_loss: 3.292 time: 0.3287 s/iter data_time: 0.2183 s/iter total_throughput: 3115.43 samples/s lr: 2.20e-04 [09/27 09:06:25] lb.utils.events INFO: eta: 5:10:10 iteration: 260999/375342 consumed_samples: 267264000 total_loss: 3.276 time: 0.3287 s/iter data_time: 0.2354 s/iter total_throughput: 3115.43 samples/s lr: 2.20e-04 [09/27 09:06:58] lb.utils.events INFO: eta: 5:11:34 iteration: 261099/375342 consumed_samples: 267366400 total_loss: 3.262 time: 0.3287 s/iter data_time: 0.2246 s/iter total_throughput: 3115.42 samples/s lr: 2.20e-04 [09/27 09:07:31] lb.utils.events INFO: eta: 5:11:11 iteration: 261199/375342 consumed_samples: 267468800 total_loss: 3.276 time: 0.3287 s/iter data_time: 0.2091 s/iter total_throughput: 3115.42 samples/s lr: 2.19e-04 [09/27 09:08:04] lb.utils.events INFO: eta: 5:16:57 iteration: 261299/375342 consumed_samples: 267571200 total_loss: 3.271 time: 0.3287 s/iter data_time: 0.2224 s/iter total_throughput: 3115.41 samples/s lr: 2.19e-04 [09/27 09:08:37] lb.utils.events INFO: eta: 5:16:14 iteration: 261399/375342 consumed_samples: 267673600 total_loss: 3.286 time: 0.3287 s/iter data_time: 0.2147 s/iter total_throughput: 3115.42 samples/s lr: 2.19e-04 [09/27 09:09:10] lb.utils.events INFO: eta: 5:06:56 iteration: 261499/375342 consumed_samples: 267776000 total_loss: 3.288 time: 0.3287 s/iter data_time: 0.2139 s/iter total_throughput: 3115.41 samples/s lr: 2.18e-04 [09/27 09:09:42] lb.utils.events INFO: eta: 5:01:18 iteration: 261599/375342 consumed_samples: 267878400 total_loss: 3.279 time: 0.3287 s/iter data_time: 0.2029 s/iter total_throughput: 3115.41 samples/s lr: 2.18e-04 [09/27 09:10:15] lb.utils.events INFO: eta: 4:56:49 iteration: 261699/375342 consumed_samples: 267980800 total_loss: 3.275 time: 0.3287 s/iter data_time: 0.2091 s/iter total_throughput: 3115.42 samples/s lr: 2.18e-04 [09/27 09:10:48] lb.utils.events INFO: eta: 5:00:45 iteration: 261799/375342 consumed_samples: 268083200 total_loss: 3.254 time: 0.3287 s/iter data_time: 0.2594 s/iter total_throughput: 3115.43 samples/s lr: 2.17e-04 [09/27 09:11:20] lb.utils.events INFO: eta: 5:08:24 iteration: 261899/375342 consumed_samples: 268185600 total_loss: 3.254 time: 0.3287 s/iter data_time: 0.2634 s/iter total_throughput: 3115.44 samples/s lr: 2.17e-04 [09/27 09:11:54] lb.utils.events INFO: eta: 5:07:09 iteration: 261999/375342 consumed_samples: 268288000 total_loss: 3.285 time: 0.3287 s/iter data_time: 0.2278 s/iter total_throughput: 3115.43 samples/s lr: 2.17e-04 [09/27 09:12:26] lb.utils.events INFO: eta: 5:08:32 iteration: 262099/375342 consumed_samples: 268390400 total_loss: 3.257 time: 0.3287 s/iter data_time: 0.2395 s/iter total_throughput: 3115.43 samples/s lr: 2.16e-04 [09/27 09:12:59] lb.utils.events INFO: eta: 5:09:43 iteration: 262199/375342 consumed_samples: 268492800 total_loss: 3.242 time: 0.3287 s/iter data_time: 0.2246 s/iter total_throughput: 3115.43 samples/s lr: 2.16e-04 [09/27 09:13:32] lb.utils.events INFO: eta: 5:14:39 iteration: 262299/375342 consumed_samples: 268595200 total_loss: 3.259 time: 0.3287 s/iter data_time: 0.2441 s/iter total_throughput: 3115.44 samples/s lr: 2.16e-04 [09/27 09:14:05] lb.utils.events INFO: eta: 5:29:08 iteration: 262399/375342 consumed_samples: 268697600 total_loss: 3.275 time: 0.3287 s/iter data_time: 0.2483 s/iter total_throughput: 3115.44 samples/s lr: 2.15e-04 [09/27 09:14:37] lb.utils.events INFO: eta: 5:44:36 iteration: 262499/375342 consumed_samples: 268800000 total_loss: 3.262 time: 0.3287 s/iter data_time: 0.2325 s/iter total_throughput: 3115.44 samples/s lr: 2.15e-04 [09/27 09:15:10] lb.utils.events INFO: eta: 6:07:34 iteration: 262599/375342 consumed_samples: 268902400 total_loss: 3.281 time: 0.3287 s/iter data_time: 0.2157 s/iter total_throughput: 3115.44 samples/s lr: 2.15e-04 [09/27 09:15:43] lb.utils.events INFO: eta: 6:12:34 iteration: 262699/375342 consumed_samples: 269004800 total_loss: 3.295 time: 0.3287 s/iter data_time: 0.2110 s/iter total_throughput: 3115.43 samples/s lr: 2.14e-04 [09/27 09:16:16] lb.utils.events INFO: eta: 5:27:25 iteration: 262799/375342 consumed_samples: 269107200 total_loss: 3.269 time: 0.3287 s/iter data_time: 0.2165 s/iter total_throughput: 3115.44 samples/s lr: 2.14e-04 [09/27 09:16:49] lb.utils.events INFO: eta: 5:08:46 iteration: 262899/375342 consumed_samples: 269209600 total_loss: 3.258 time: 0.3287 s/iter data_time: 0.2232 s/iter total_throughput: 3115.45 samples/s lr: 2.14e-04 [09/27 09:17:22] lb.utils.events INFO: eta: 5:07:51 iteration: 262999/375342 consumed_samples: 269312000 total_loss: 3.277 time: 0.3287 s/iter data_time: 0.2340 s/iter total_throughput: 3115.45 samples/s lr: 2.13e-04 [09/27 09:17:55] lb.utils.events INFO: eta: 5:06:55 iteration: 263099/375342 consumed_samples: 269414400 total_loss: 3.282 time: 0.3287 s/iter data_time: 0.2266 s/iter total_throughput: 3115.45 samples/s lr: 2.13e-04 [09/27 09:18:28] lb.utils.events INFO: eta: 5:06:09 iteration: 263199/375342 consumed_samples: 269516800 total_loss: 3.272 time: 0.3287 s/iter data_time: 0.2162 s/iter total_throughput: 3115.45 samples/s lr: 2.13e-04 [09/27 09:19:00] lb.utils.events INFO: eta: 5:02:05 iteration: 263299/375342 consumed_samples: 269619200 total_loss: 3.265 time: 0.3287 s/iter data_time: 0.2286 s/iter total_throughput: 3115.46 samples/s lr: 2.12e-04 [09/27 09:19:33] lb.utils.events INFO: eta: 5:00:24 iteration: 263399/375342 consumed_samples: 269721600 total_loss: 3.256 time: 0.3287 s/iter data_time: 0.2108 s/iter total_throughput: 3115.47 samples/s lr: 2.12e-04 [09/27 09:20:05] lb.utils.events INFO: eta: 5:00:48 iteration: 263499/375342 consumed_samples: 269824000 total_loss: 3.261 time: 0.3287 s/iter data_time: 0.2198 s/iter total_throughput: 3115.47 samples/s lr: 2.12e-04 [09/27 09:20:38] lb.utils.events INFO: eta: 5:02:46 iteration: 263599/375342 consumed_samples: 269926400 total_loss: 3.272 time: 0.3287 s/iter data_time: 0.2233 s/iter total_throughput: 3115.47 samples/s lr: 2.11e-04 [09/27 09:21:11] lb.utils.events INFO: eta: 5:09:01 iteration: 263699/375342 consumed_samples: 270028800 total_loss: 3.272 time: 0.3287 s/iter data_time: 0.2242 s/iter total_throughput: 3115.47 samples/s lr: 2.11e-04 [09/27 09:21:44] lb.utils.events INFO: eta: 5:11:27 iteration: 263799/375342 consumed_samples: 270131200 total_loss: 3.271 time: 0.3287 s/iter data_time: 0.2172 s/iter total_throughput: 3115.47 samples/s lr: 2.11e-04 [09/27 09:22:17] lb.utils.events INFO: eta: 5:10:08 iteration: 263899/375342 consumed_samples: 270233600 total_loss: 3.274 time: 0.3287 s/iter data_time: 0.2146 s/iter total_throughput: 3115.47 samples/s lr: 2.10e-04 [09/27 09:22:50] lb.utils.events INFO: eta: 5:10:19 iteration: 263999/375342 consumed_samples: 270336000 total_loss: 3.267 time: 0.3287 s/iter data_time: 0.2279 s/iter total_throughput: 3115.47 samples/s lr: 2.10e-04 [09/27 09:23:23] lb.utils.events INFO: eta: 5:16:18 iteration: 264099/375342 consumed_samples: 270438400 total_loss: 3.273 time: 0.3287 s/iter data_time: 0.2522 s/iter total_throughput: 3115.46 samples/s lr: 2.10e-04 [09/27 09:23:56] lb.utils.events INFO: eta: 5:12:46 iteration: 264199/375342 consumed_samples: 270540800 total_loss: 3.277 time: 0.3287 s/iter data_time: 0.2087 s/iter total_throughput: 3115.47 samples/s lr: 2.09e-04 [09/27 09:24:28] lb.utils.events INFO: eta: 5:13:21 iteration: 264299/375342 consumed_samples: 270643200 total_loss: 3.273 time: 0.3287 s/iter data_time: 0.2208 s/iter total_throughput: 3115.48 samples/s lr: 2.09e-04 [09/27 09:25:01] lb.utils.events INFO: eta: 5:11:11 iteration: 264399/375342 consumed_samples: 270745600 total_loss: 3.274 time: 0.3287 s/iter data_time: 0.2201 s/iter total_throughput: 3115.49 samples/s lr: 2.09e-04 [09/27 09:25:34] lb.utils.events INFO: eta: 5:07:57 iteration: 264499/375342 consumed_samples: 270848000 total_loss: 3.26 time: 0.3287 s/iter data_time: 0.2191 s/iter total_throughput: 3115.50 samples/s lr: 2.08e-04 [09/27 09:26:06] lb.utils.events INFO: eta: 5:13:08 iteration: 264599/375342 consumed_samples: 270950400 total_loss: 3.267 time: 0.3287 s/iter data_time: 0.2302 s/iter total_throughput: 3115.50 samples/s lr: 2.08e-04 [09/27 09:26:39] lb.utils.events INFO: eta: 5:07:06 iteration: 264699/375342 consumed_samples: 271052800 total_loss: 3.262 time: 0.3287 s/iter data_time: 0.2231 s/iter total_throughput: 3115.51 samples/s lr: 2.08e-04 [09/27 09:27:12] lb.utils.events INFO: eta: 5:07:43 iteration: 264799/375342 consumed_samples: 271155200 total_loss: 3.255 time: 0.3287 s/iter data_time: 0.2340 s/iter total_throughput: 3115.52 samples/s lr: 2.07e-04 [09/27 09:27:45] lb.utils.events INFO: eta: 5:05:55 iteration: 264899/375342 consumed_samples: 271257600 total_loss: 3.258 time: 0.3287 s/iter data_time: 0.2111 s/iter total_throughput: 3115.51 samples/s lr: 2.07e-04 [09/27 09:28:18] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0264999 [09/27 09:28:18] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/27 09:28:18] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/27 09:28:22] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0893 s/iter. Inference: 0.1504 s/iter. Eval: 0.0024 s/iter. Total: 0.2421 s/iter. ETA=0:00:08 [09/27 09:28:28] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1488 s/iter. Inference: 0.1514 s/iter. Eval: 0.0021 s/iter. Total: 0.3024 s/iter. ETA=0:00:05 [09/27 09:28:33] lb.evaluation.evaluator INFO: Inference done 49152/50000. Dataloading: 0.1327 s/iter. Inference: 0.1506 s/iter. Eval: 0.0021 s/iter. Total: 0.2855 s/iter. ETA=0:00:00 [09/27 09:28:34] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/27 09:28:34] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.707798 (0.000254 s / iter per device, on 8 devices) [09/27 09:28:34] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000134 s / iter per device, on 8 devices) [09/27 09:28:34] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/27 09:28:34] lb.evaluation.utils INFO: copypaste: Acc@1=77.99000000000001 [09/27 09:28:34] lb.evaluation.utils INFO: copypaste: Acc@5=93.884 [09/27 09:28:34] lb.engine.hooks INFO: Not saving as latest eval score for Acc@1 is 77.99000, not better than best score 78.05800 @ iteration 259999. [09/27 09:28:34] lb.utils.events INFO: eta: 5:00:40 iteration: 264999/375342 consumed_samples: 271360000 total_loss: 3.257 time: 0.3287 s/iter data_time: 0.2061 s/iter total_throughput: 3115.51 samples/s lr: 2.07e-04 [09/27 09:29:04] lb.utils.events INFO: eta: 4:54:51 iteration: 265099/375342 consumed_samples: 271462400 total_loss: 3.26 time: 0.3287 s/iter data_time: 0.2092 s/iter total_throughput: 3115.59 samples/s lr: 2.06e-04 [09/27 09:29:37] lb.utils.events INFO: eta: 4:54:38 iteration: 265199/375342 consumed_samples: 271564800 total_loss: 3.26 time: 0.3287 s/iter data_time: 0.2114 s/iter total_throughput: 3115.60 samples/s lr: 2.06e-04 [09/27 09:30:09] lb.utils.events INFO: eta: 4:52:07 iteration: 265299/375342 consumed_samples: 271667200 total_loss: 3.262 time: 0.3287 s/iter data_time: 0.1972 s/iter total_throughput: 3115.62 samples/s lr: 2.06e-04 [09/27 09:30:42] lb.utils.events INFO: eta: 4:47:59 iteration: 265399/375342 consumed_samples: 271769600 total_loss: 3.254 time: 0.3287 s/iter data_time: 0.2013 s/iter total_throughput: 3115.63 samples/s lr: 2.05e-04 [09/27 09:31:14] lb.utils.events INFO: eta: 4:45:52 iteration: 265499/375342 consumed_samples: 271872000 total_loss: 3.246 time: 0.3287 s/iter data_time: 0.1974 s/iter total_throughput: 3115.65 samples/s lr: 2.05e-04 [09/27 09:31:47] lb.utils.events INFO: eta: 4:43:42 iteration: 265599/375342 consumed_samples: 271974400 total_loss: 3.25 time: 0.3287 s/iter data_time: 0.2073 s/iter total_throughput: 3115.65 samples/s lr: 2.05e-04 [09/27 09:32:19] lb.utils.events INFO: eta: 4:42:37 iteration: 265699/375342 consumed_samples: 272076800 total_loss: 3.249 time: 0.3287 s/iter data_time: 0.1994 s/iter total_throughput: 3115.67 samples/s lr: 2.04e-04 [09/27 09:32:52] lb.utils.events INFO: eta: 4:41:35 iteration: 265799/375342 consumed_samples: 272179200 total_loss: 3.238 time: 0.3287 s/iter data_time: 0.2010 s/iter total_throughput: 3115.69 samples/s lr: 2.04e-04 [09/27 09:33:24] lb.utils.events INFO: eta: 4:40:49 iteration: 265899/375342 consumed_samples: 272281600 total_loss: 3.249 time: 0.3287 s/iter data_time: 0.2400 s/iter total_throughput: 3115.72 samples/s lr: 2.04e-04 [09/27 09:33:57] lb.utils.events INFO: eta: 4:42:50 iteration: 265999/375342 consumed_samples: 272384000 total_loss: 3.25 time: 0.3287 s/iter data_time: 0.2204 s/iter total_throughput: 3115.72 samples/s lr: 2.03e-04 [09/27 09:34:29] lb.utils.events INFO: eta: 4:43:19 iteration: 266099/375342 consumed_samples: 272486400 total_loss: 3.249 time: 0.3287 s/iter data_time: 0.2199 s/iter total_throughput: 3115.73 samples/s lr: 2.03e-04 [09/27 09:35:02] lb.utils.events INFO: eta: 4:43:02 iteration: 266199/375342 consumed_samples: 272588800 total_loss: 3.23 time: 0.3287 s/iter data_time: 0.2101 s/iter total_throughput: 3115.72 samples/s lr: 2.03e-04 [09/27 09:35:35] lb.utils.events INFO: eta: 4:43:09 iteration: 266299/375342 consumed_samples: 272691200 total_loss: 3.23 time: 0.3287 s/iter data_time: 0.2179 s/iter total_throughput: 3115.71 samples/s lr: 2.02e-04 [09/27 09:36:09] lb.utils.events INFO: eta: 4:42:58 iteration: 266399/375342 consumed_samples: 272793600 total_loss: 3.249 time: 0.3287 s/iter data_time: 0.2146 s/iter total_throughput: 3115.70 samples/s lr: 2.02e-04 [09/27 09:36:42] lb.utils.events INFO: eta: 4:42:44 iteration: 266499/375342 consumed_samples: 272896000 total_loss: 3.25 time: 0.3287 s/iter data_time: 0.2115 s/iter total_throughput: 3115.70 samples/s lr: 2.02e-04 [09/27 09:37:15] lb.utils.events INFO: eta: 4:43:11 iteration: 266599/375342 consumed_samples: 272998400 total_loss: 3.235 time: 0.3287 s/iter data_time: 0.2285 s/iter total_throughput: 3115.68 samples/s lr: 2.01e-04 [09/27 09:37:49] lb.utils.events INFO: eta: 4:46:03 iteration: 266699/375342 consumed_samples: 273100800 total_loss: 3.238 time: 0.3287 s/iter data_time: 0.2285 s/iter total_throughput: 3115.65 samples/s lr: 2.01e-04 [09/27 09:38:22] lb.utils.events INFO: eta: 4:47:40 iteration: 266799/375342 consumed_samples: 273203200 total_loss: 3.243 time: 0.3287 s/iter data_time: 0.2459 s/iter total_throughput: 3115.64 samples/s lr: 2.01e-04 [09/27 09:38:56] lb.utils.events INFO: eta: 4:47:45 iteration: 266899/375342 consumed_samples: 273305600 total_loss: 3.235 time: 0.3287 s/iter data_time: 0.2118 s/iter total_throughput: 3115.60 samples/s lr: 2.00e-04 [09/27 09:39:29] lb.utils.events INFO: eta: 4:43:18 iteration: 266999/375342 consumed_samples: 273408000 total_loss: 3.235 time: 0.3287 s/iter data_time: 0.2106 s/iter total_throughput: 3115.59 samples/s lr: 2.00e-04 [09/27 09:40:02] lb.utils.events INFO: eta: 4:41:51 iteration: 267099/375342 consumed_samples: 273510400 total_loss: 3.234 time: 0.3287 s/iter data_time: 0.2266 s/iter total_throughput: 3115.59 samples/s lr: 2.00e-04 [09/27 09:40:35] lb.utils.events INFO: eta: 4:43:04 iteration: 267199/375342 consumed_samples: 273612800 total_loss: 3.252 time: 0.3287 s/iter data_time: 0.2038 s/iter total_throughput: 3115.57 samples/s lr: 1.99e-04 [09/27 09:41:08] lb.utils.events INFO: eta: 4:42:31 iteration: 267299/375342 consumed_samples: 273715200 total_loss: 3.259 time: 0.3287 s/iter data_time: 0.2055 s/iter total_throughput: 3115.56 samples/s lr: 1.99e-04 [09/27 09:41:41] lb.utils.events INFO: eta: 4:42:15 iteration: 267399/375342 consumed_samples: 273817600 total_loss: 3.261 time: 0.3287 s/iter data_time: 0.2085 s/iter total_throughput: 3115.56 samples/s lr: 1.99e-04 [09/27 09:42:15] lb.utils.events INFO: eta: 4:42:24 iteration: 267499/375342 consumed_samples: 273920000 total_loss: 3.27 time: 0.3287 s/iter data_time: 0.2266 s/iter total_throughput: 3115.54 samples/s lr: 1.98e-04 [09/27 09:42:48] lb.utils.events INFO: eta: 4:42:38 iteration: 267599/375342 consumed_samples: 274022400 total_loss: 3.255 time: 0.3287 s/iter data_time: 0.2282 s/iter total_throughput: 3115.54 samples/s lr: 1.98e-04 [09/27 09:43:21] lb.utils.events INFO: eta: 4:42:04 iteration: 267699/375342 consumed_samples: 274124800 total_loss: 3.251 time: 0.3287 s/iter data_time: 0.2308 s/iter total_throughput: 3115.53 samples/s lr: 1.98e-04 [09/27 09:43:54] lb.utils.events INFO: eta: 4:40:14 iteration: 267799/375342 consumed_samples: 274227200 total_loss: 3.235 time: 0.3287 s/iter data_time: 0.2243 s/iter total_throughput: 3115.52 samples/s lr: 1.97e-04 [09/27 09:44:27] lb.utils.events INFO: eta: 4:39:53 iteration: 267899/375342 consumed_samples: 274329600 total_loss: 3.22 time: 0.3287 s/iter data_time: 0.2057 s/iter total_throughput: 3115.51 samples/s lr: 1.97e-04 [09/27 09:45:00] lb.utils.events INFO: eta: 4:39:59 iteration: 267999/375342 consumed_samples: 274432000 total_loss: 3.227 time: 0.3287 s/iter data_time: 0.2082 s/iter total_throughput: 3115.50 samples/s lr: 1.97e-04 [09/27 09:45:33] lb.utils.events INFO: eta: 4:38:47 iteration: 268099/375342 consumed_samples: 274534400 total_loss: 3.239 time: 0.3287 s/iter data_time: 0.2042 s/iter total_throughput: 3115.50 samples/s lr: 1.96e-04 [09/27 09:46:06] lb.utils.events INFO: eta: 4:37:25 iteration: 268199/375342 consumed_samples: 274636800 total_loss: 3.235 time: 0.3287 s/iter data_time: 0.2103 s/iter total_throughput: 3115.49 samples/s lr: 1.96e-04 [09/27 09:46:39] lb.utils.events INFO: eta: 4:37:01 iteration: 268299/375342 consumed_samples: 274739200 total_loss: 3.238 time: 0.3287 s/iter data_time: 0.2036 s/iter total_throughput: 3115.48 samples/s lr: 1.96e-04 [09/27 09:47:13] lb.utils.events INFO: eta: 4:36:43 iteration: 268399/375342 consumed_samples: 274841600 total_loss: 3.234 time: 0.3287 s/iter data_time: 0.2349 s/iter total_throughput: 3115.46 samples/s lr: 1.95e-04 [09/27 09:47:46] lb.utils.events INFO: eta: 4:35:10 iteration: 268499/375342 consumed_samples: 274944000 total_loss: 3.236 time: 0.3287 s/iter data_time: 0.1995 s/iter total_throughput: 3115.46 samples/s lr: 1.95e-04 [09/27 09:48:19] lb.utils.events INFO: eta: 4:34:24 iteration: 268599/375342 consumed_samples: 275046400 total_loss: 3.251 time: 0.3287 s/iter data_time: 0.2067 s/iter total_throughput: 3115.46 samples/s lr: 1.95e-04 [09/27 09:48:52] lb.utils.events INFO: eta: 4:32:53 iteration: 268699/375342 consumed_samples: 275148800 total_loss: 3.267 time: 0.3287 s/iter data_time: 0.2003 s/iter total_throughput: 3115.45 samples/s lr: 1.94e-04 [09/27 09:49:25] lb.utils.events INFO: eta: 4:32:41 iteration: 268799/375342 consumed_samples: 275251200 total_loss: 3.273 time: 0.3287 s/iter data_time: 0.2072 s/iter total_throughput: 3115.45 samples/s lr: 1.94e-04 [09/27 09:49:58] lb.utils.events INFO: eta: 4:31:34 iteration: 268899/375342 consumed_samples: 275353600 total_loss: 3.252 time: 0.3287 s/iter data_time: 0.2169 s/iter total_throughput: 3115.43 samples/s lr: 1.94e-04 [09/27 09:50:31] lb.utils.events INFO: eta: 4:31:35 iteration: 268999/375342 consumed_samples: 275456000 total_loss: 3.252 time: 0.3287 s/iter data_time: 0.1972 s/iter total_throughput: 3115.44 samples/s lr: 1.93e-04 [09/27 09:51:04] lb.utils.events INFO: eta: 4:32:20 iteration: 269099/375342 consumed_samples: 275558400 total_loss: 3.23 time: 0.3287 s/iter data_time: 0.2038 s/iter total_throughput: 3115.43 samples/s lr: 1.93e-04 [09/27 09:51:36] lb.utils.events INFO: eta: 4:31:39 iteration: 269199/375342 consumed_samples: 275660800 total_loss: 3.217 time: 0.3287 s/iter data_time: 0.1952 s/iter total_throughput: 3115.44 samples/s lr: 1.93e-04 [09/27 09:52:09] lb.utils.events INFO: eta: 4:32:48 iteration: 269299/375342 consumed_samples: 275763200 total_loss: 3.24 time: 0.3287 s/iter data_time: 0.2289 s/iter total_throughput: 3115.44 samples/s lr: 1.93e-04 [09/27 09:52:43] lb.utils.events INFO: eta: 4:32:44 iteration: 269399/375342 consumed_samples: 275865600 total_loss: 3.252 time: 0.3287 s/iter data_time: 0.2123 s/iter total_throughput: 3115.42 samples/s lr: 1.92e-04 [09/27 09:53:16] lb.utils.events INFO: eta: 4:34:36 iteration: 269499/375342 consumed_samples: 275968000 total_loss: 3.25 time: 0.3287 s/iter data_time: 0.2053 s/iter total_throughput: 3115.41 samples/s lr: 1.92e-04 [09/27 09:53:49] lb.utils.events INFO: eta: 4:34:30 iteration: 269599/375342 consumed_samples: 276070400 total_loss: 3.251 time: 0.3287 s/iter data_time: 0.1992 s/iter total_throughput: 3115.41 samples/s lr: 1.92e-04 [09/27 09:54:22] lb.utils.events INFO: eta: 4:34:05 iteration: 269699/375342 consumed_samples: 276172800 total_loss: 3.232 time: 0.3287 s/iter data_time: 0.2041 s/iter total_throughput: 3115.41 samples/s lr: 1.91e-04 [09/27 09:54:55] lb.utils.events INFO: eta: 4:34:25 iteration: 269799/375342 consumed_samples: 276275200 total_loss: 3.234 time: 0.3287 s/iter data_time: 0.2052 s/iter total_throughput: 3115.39 samples/s lr: 1.91e-04 [09/27 09:55:28] lb.utils.events INFO: eta: 4:34:18 iteration: 269899/375342 consumed_samples: 276377600 total_loss: 3.234 time: 0.3287 s/iter data_time: 0.2110 s/iter total_throughput: 3115.38 samples/s lr: 1.91e-04 [09/27 09:56:02] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0269999 [09/27 09:56:03] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/27 09:56:03] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/27 09:56:06] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0884 s/iter. Inference: 0.1487 s/iter. Eval: 0.0020 s/iter. Total: 0.2392 s/iter. ETA=0:00:08 [09/27 09:56:12] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1460 s/iter. Inference: 0.1485 s/iter. Eval: 0.0020 s/iter. Total: 0.2966 s/iter. ETA=0:00:05 [09/27 09:56:18] lb.evaluation.evaluator INFO: Inference done 50000/50000. Dataloading: 0.1336 s/iter. Inference: 0.1494 s/iter. Eval: 0.0020 s/iter. Total: 0.2851 s/iter. ETA=0:00:00 [09/27 09:56:18] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/27 09:56:18] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.549557 (0.000251 s / iter per device, on 8 devices) [09/27 09:56:18] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000131 s / iter per device, on 8 devices) [09/27 09:56:18] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/27 09:56:18] lb.evaluation.utils INFO: copypaste: Acc@1=78.282 [09/27 09:56:18] lb.evaluation.utils INFO: copypaste: Acc@5=93.97999999999999 [09/27 09:56:18] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 78.28200, better than last best score 78.05800 @ iteration 259999. [09/27 09:56:18] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/27 09:56:18] lb.utils.events INFO: eta: 4:33:20 iteration: 269999/375342 consumed_samples: 276480000 total_loss: 3.227 time: 0.3287 s/iter data_time: 0.2169 s/iter total_throughput: 3115.36 samples/s lr: 1.90e-04 [09/27 09:56:50] lb.utils.events INFO: eta: 4:32:50 iteration: 270099/375342 consumed_samples: 276582400 total_loss: 3.243 time: 0.3287 s/iter data_time: 0.2387 s/iter total_throughput: 3115.41 samples/s lr: 1.90e-04 [09/27 09:57:23] lb.utils.events INFO: eta: 4:33:35 iteration: 270199/375342 consumed_samples: 276684800 total_loss: 3.251 time: 0.3287 s/iter data_time: 0.2400 s/iter total_throughput: 3115.38 samples/s lr: 1.90e-04 [09/27 09:57:56] lb.utils.events INFO: eta: 4:33:03 iteration: 270299/375342 consumed_samples: 276787200 total_loss: 3.217 time: 0.3287 s/iter data_time: 0.2321 s/iter total_throughput: 3115.39 samples/s lr: 1.89e-04 [09/27 09:58:30] lb.utils.events INFO: eta: 4:34:06 iteration: 270399/375342 consumed_samples: 276889600 total_loss: 3.21 time: 0.3287 s/iter data_time: 0.2298 s/iter total_throughput: 3115.37 samples/s lr: 1.89e-04 [09/27 09:59:03] lb.utils.events INFO: eta: 4:31:48 iteration: 270499/375342 consumed_samples: 276992000 total_loss: 3.215 time: 0.3287 s/iter data_time: 0.2078 s/iter total_throughput: 3115.36 samples/s lr: 1.89e-04 [09/27 09:59:36] lb.utils.events INFO: eta: 4:31:09 iteration: 270599/375342 consumed_samples: 277094400 total_loss: 3.215 time: 0.3287 s/iter data_time: 0.2036 s/iter total_throughput: 3115.36 samples/s lr: 1.88e-04 [09/27 10:00:09] lb.utils.events INFO: eta: 4:30:38 iteration: 270699/375342 consumed_samples: 277196800 total_loss: 3.225 time: 0.3287 s/iter data_time: 0.2139 s/iter total_throughput: 3115.34 samples/s lr: 1.88e-04 [09/27 10:00:42] lb.utils.events INFO: eta: 4:30:22 iteration: 270799/375342 consumed_samples: 277299200 total_loss: 3.264 time: 0.3287 s/iter data_time: 0.2085 s/iter total_throughput: 3115.35 samples/s lr: 1.88e-04 [09/27 10:01:15] lb.utils.events INFO: eta: 4:29:54 iteration: 270899/375342 consumed_samples: 277401600 total_loss: 3.248 time: 0.3287 s/iter data_time: 0.2122 s/iter total_throughput: 3115.34 samples/s lr: 1.87e-04 [09/27 10:01:48] lb.utils.events INFO: eta: 4:30:20 iteration: 270999/375342 consumed_samples: 277504000 total_loss: 3.23 time: 0.3287 s/iter data_time: 0.2246 s/iter total_throughput: 3115.34 samples/s lr: 1.87e-04 [09/27 10:02:21] lb.utils.events INFO: eta: 4:31:42 iteration: 271099/375342 consumed_samples: 277606400 total_loss: 3.23 time: 0.3287 s/iter data_time: 0.2240 s/iter total_throughput: 3115.34 samples/s lr: 1.87e-04 [09/27 10:02:54] lb.utils.events INFO: eta: 4:30:24 iteration: 271199/375342 consumed_samples: 277708800 total_loss: 3.225 time: 0.3287 s/iter data_time: 0.2288 s/iter total_throughput: 3115.33 samples/s lr: 1.86e-04 [09/27 10:03:26] lb.utils.events INFO: eta: 4:30:11 iteration: 271299/375342 consumed_samples: 277811200 total_loss: 3.193 time: 0.3287 s/iter data_time: 0.2159 s/iter total_throughput: 3115.34 samples/s lr: 1.86e-04 [09/27 10:04:00] lb.utils.events INFO: eta: 4:29:33 iteration: 271399/375342 consumed_samples: 277913600 total_loss: 3.224 time: 0.3287 s/iter data_time: 0.2096 s/iter total_throughput: 3115.32 samples/s lr: 1.86e-04 [09/27 10:04:33] lb.utils.events INFO: eta: 4:29:40 iteration: 271499/375342 consumed_samples: 278016000 total_loss: 3.242 time: 0.3287 s/iter data_time: 0.2290 s/iter total_throughput: 3115.31 samples/s lr: 1.85e-04 [09/27 10:05:06] lb.utils.events INFO: eta: 4:30:26 iteration: 271599/375342 consumed_samples: 278118400 total_loss: 3.225 time: 0.3287 s/iter data_time: 0.2306 s/iter total_throughput: 3115.30 samples/s lr: 1.85e-04 [09/27 10:05:39] lb.utils.events INFO: eta: 4:33:45 iteration: 271699/375342 consumed_samples: 278220800 total_loss: 3.23 time: 0.3287 s/iter data_time: 0.2661 s/iter total_throughput: 3115.30 samples/s lr: 1.85e-04 [09/27 10:06:13] lb.utils.events INFO: eta: 4:37:26 iteration: 271799/375342 consumed_samples: 278323200 total_loss: 3.232 time: 0.3287 s/iter data_time: 0.2127 s/iter total_throughput: 3115.27 samples/s lr: 1.85e-04 [09/27 10:06:46] lb.utils.events INFO: eta: 4:43:02 iteration: 271899/375342 consumed_samples: 278425600 total_loss: 3.205 time: 0.3287 s/iter data_time: 0.2422 s/iter total_throughput: 3115.27 samples/s lr: 1.84e-04 [09/27 10:07:19] lb.utils.events INFO: eta: 4:48:14 iteration: 271999/375342 consumed_samples: 278528000 total_loss: 3.208 time: 0.3287 s/iter data_time: 0.2535 s/iter total_throughput: 3115.27 samples/s lr: 1.84e-04 [09/27 10:07:52] lb.utils.events INFO: eta: 4:49:25 iteration: 272099/375342 consumed_samples: 278630400 total_loss: 3.223 time: 0.3287 s/iter data_time: 0.2386 s/iter total_throughput: 3115.26 samples/s lr: 1.84e-04 [09/27 10:08:25] lb.utils.events INFO: eta: 5:05:20 iteration: 272199/375342 consumed_samples: 278732800 total_loss: 3.225 time: 0.3287 s/iter data_time: 0.2292 s/iter total_throughput: 3115.25 samples/s lr: 1.83e-04 [09/27 10:08:58] lb.utils.events INFO: eta: 5:20:25 iteration: 272299/375342 consumed_samples: 278835200 total_loss: 3.226 time: 0.3287 s/iter data_time: 0.2158 s/iter total_throughput: 3115.25 samples/s lr: 1.83e-04 [09/27 10:09:31] lb.utils.events INFO: eta: 5:07:58 iteration: 272399/375342 consumed_samples: 278937600 total_loss: 3.234 time: 0.3287 s/iter data_time: 0.2006 s/iter total_throughput: 3115.25 samples/s lr: 1.83e-04 [09/27 10:10:04] lb.utils.events INFO: eta: 5:05:22 iteration: 272499/375342 consumed_samples: 279040000 total_loss: 3.234 time: 0.3287 s/iter data_time: 0.2119 s/iter total_throughput: 3115.24 samples/s lr: 1.82e-04 [09/27 10:10:37] lb.utils.events INFO: eta: 4:52:08 iteration: 272599/375342 consumed_samples: 279142400 total_loss: 3.228 time: 0.3287 s/iter data_time: 0.2091 s/iter total_throughput: 3115.22 samples/s lr: 1.82e-04 [09/27 10:11:10] lb.utils.events INFO: eta: 4:38:57 iteration: 272699/375342 consumed_samples: 279244800 total_loss: 3.217 time: 0.3287 s/iter data_time: 0.2074 s/iter total_throughput: 3115.22 samples/s lr: 1.82e-04 [09/27 10:11:43] lb.utils.events INFO: eta: 4:31:26 iteration: 272799/375342 consumed_samples: 279347200 total_loss: 3.216 time: 0.3287 s/iter data_time: 0.2011 s/iter total_throughput: 3115.21 samples/s lr: 1.81e-04 [09/27 10:12:16] lb.utils.events INFO: eta: 4:28:39 iteration: 272899/375342 consumed_samples: 279449600 total_loss: 3.2 time: 0.3287 s/iter data_time: 0.2077 s/iter total_throughput: 3115.20 samples/s lr: 1.81e-04 [09/27 10:12:49] lb.utils.events INFO: eta: 4:26:18 iteration: 272999/375342 consumed_samples: 279552000 total_loss: 3.199 time: 0.3287 s/iter data_time: 0.2113 s/iter total_throughput: 3115.21 samples/s lr: 1.81e-04 [09/27 10:13:22] lb.utils.events INFO: eta: 4:24:52 iteration: 273099/375342 consumed_samples: 279654400 total_loss: 3.221 time: 0.3287 s/iter data_time: 0.2122 s/iter total_throughput: 3115.20 samples/s lr: 1.80e-04 [09/27 10:13:55] lb.utils.events INFO: eta: 4:23:03 iteration: 273199/375342 consumed_samples: 279756800 total_loss: 3.228 time: 0.3287 s/iter data_time: 0.1937 s/iter total_throughput: 3115.21 samples/s lr: 1.80e-04 [09/27 10:14:28] lb.utils.events INFO: eta: 4:22:16 iteration: 273299/375342 consumed_samples: 279859200 total_loss: 3.229 time: 0.3287 s/iter data_time: 0.2298 s/iter total_throughput: 3115.20 samples/s lr: 1.80e-04 [09/27 10:15:01] lb.utils.events INFO: eta: 4:22:28 iteration: 273399/375342 consumed_samples: 279961600 total_loss: 3.227 time: 0.3287 s/iter data_time: 0.2211 s/iter total_throughput: 3115.19 samples/s lr: 1.80e-04 [09/27 10:15:34] lb.utils.events INFO: eta: 4:22:34 iteration: 273499/375342 consumed_samples: 280064000 total_loss: 3.197 time: 0.3287 s/iter data_time: 0.2259 s/iter total_throughput: 3115.18 samples/s lr: 1.79e-04 [09/27 10:16:07] lb.utils.events INFO: eta: 4:23:20 iteration: 273599/375342 consumed_samples: 280166400 total_loss: 3.203 time: 0.3287 s/iter data_time: 0.2208 s/iter total_throughput: 3115.19 samples/s lr: 1.79e-04 [09/27 10:16:40] lb.utils.events INFO: eta: 4:25:19 iteration: 273699/375342 consumed_samples: 280268800 total_loss: 3.22 time: 0.3287 s/iter data_time: 0.2491 s/iter total_throughput: 3115.18 samples/s lr: 1.79e-04 [09/27 10:17:13] lb.utils.events INFO: eta: 4:25:01 iteration: 273799/375342 consumed_samples: 280371200 total_loss: 3.21 time: 0.3287 s/iter data_time: 0.2192 s/iter total_throughput: 3115.17 samples/s lr: 1.78e-04 [09/27 10:17:46] lb.utils.events INFO: eta: 4:26:59 iteration: 273899/375342 consumed_samples: 280473600 total_loss: 3.212 time: 0.3287 s/iter data_time: 0.2454 s/iter total_throughput: 3115.18 samples/s lr: 1.78e-04 [09/27 10:18:19] lb.utils.events INFO: eta: 4:30:11 iteration: 273999/375342 consumed_samples: 280576000 total_loss: 3.211 time: 0.3287 s/iter data_time: 0.2286 s/iter total_throughput: 3115.16 samples/s lr: 1.78e-04 [09/27 10:18:52] lb.utils.events INFO: eta: 4:33:30 iteration: 274099/375342 consumed_samples: 280678400 total_loss: 3.204 time: 0.3287 s/iter data_time: 0.2180 s/iter total_throughput: 3115.15 samples/s lr: 1.77e-04 [09/27 10:19:25] lb.utils.events INFO: eta: 4:36:52 iteration: 274199/375342 consumed_samples: 280780800 total_loss: 3.184 time: 0.3287 s/iter data_time: 0.2237 s/iter total_throughput: 3115.14 samples/s lr: 1.77e-04 [09/27 10:19:58] lb.utils.events INFO: eta: 4:36:42 iteration: 274299/375342 consumed_samples: 280883200 total_loss: 3.208 time: 0.3287 s/iter data_time: 0.2316 s/iter total_throughput: 3115.16 samples/s lr: 1.77e-04 [09/27 10:20:31] lb.utils.events INFO: eta: 4:52:13 iteration: 274399/375342 consumed_samples: 280985600 total_loss: 3.234 time: 0.3287 s/iter data_time: 0.2487 s/iter total_throughput: 3115.16 samples/s lr: 1.76e-04 [09/27 10:21:04] lb.utils.events INFO: eta: 5:45:39 iteration: 274499/375342 consumed_samples: 281088000 total_loss: 3.21 time: 0.3287 s/iter data_time: 0.2387 s/iter total_throughput: 3115.16 samples/s lr: 1.76e-04 [09/27 10:21:37] lb.utils.events INFO: eta: 6:19:59 iteration: 274599/375342 consumed_samples: 281190400 total_loss: 3.203 time: 0.3287 s/iter data_time: 0.2403 s/iter total_throughput: 3115.16 samples/s lr: 1.76e-04 [09/27 10:22:09] lb.utils.events INFO: eta: 6:48:48 iteration: 274699/375342 consumed_samples: 281292800 total_loss: 3.235 time: 0.3287 s/iter data_time: 0.2422 s/iter total_throughput: 3115.18 samples/s lr: 1.75e-04 [09/27 10:22:42] lb.utils.events INFO: eta: 7:36:20 iteration: 274799/375342 consumed_samples: 281395200 total_loss: 3.235 time: 0.3287 s/iter data_time: 0.2610 s/iter total_throughput: 3115.17 samples/s lr: 1.75e-04 [09/27 10:23:15] lb.utils.events INFO: eta: 7:40:20 iteration: 274899/375342 consumed_samples: 281497600 total_loss: 3.232 time: 0.3287 s/iter data_time: 0.2125 s/iter total_throughput: 3115.17 samples/s lr: 1.75e-04 [09/27 10:23:48] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0274999 [09/27 10:23:49] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/27 10:23:49] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/27 10:23:53] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0818 s/iter. Inference: 0.1540 s/iter. Eval: 0.0022 s/iter. Total: 0.2380 s/iter. ETA=0:00:08 [09/27 10:23:58] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1411 s/iter. Inference: 0.1504 s/iter. Eval: 0.0020 s/iter. Total: 0.2936 s/iter. ETA=0:00:05 [09/27 10:24:04] lb.evaluation.evaluator INFO: Inference done 50000/50000. Dataloading: 0.1306 s/iter. Inference: 0.1507 s/iter. Eval: 0.0021 s/iter. Total: 0.2835 s/iter. ETA=0:00:00 [09/27 10:24:04] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/27 10:24:04] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.478043 (0.000250 s / iter per device, on 8 devices) [09/27 10:24:04] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000133 s / iter per device, on 8 devices) [09/27 10:24:04] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/27 10:24:04] lb.evaluation.utils INFO: copypaste: Acc@1=78.34 [09/27 10:24:04] lb.evaluation.utils INFO: copypaste: Acc@5=93.96 [09/27 10:24:04] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 78.34000, better than last best score 78.28200 @ iteration 269999. [09/27 10:24:04] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/27 10:24:05] lb.utils.events INFO: eta: 7:43:16 iteration: 274999/375342 consumed_samples: 281600000 total_loss: 3.236 time: 0.3287 s/iter data_time: 0.2280 s/iter total_throughput: 3115.17 samples/s lr: 1.75e-04 [09/27 10:24:36] lb.utils.events INFO: eta: 7:07:15 iteration: 275099/375342 consumed_samples: 281702400 total_loss: 3.223 time: 0.3287 s/iter data_time: 0.2174 s/iter total_throughput: 3115.22 samples/s lr: 1.74e-04 [09/27 10:25:10] lb.utils.events INFO: eta: 6:42:36 iteration: 275199/375342 consumed_samples: 281804800 total_loss: 3.231 time: 0.3287 s/iter data_time: 0.2099 s/iter total_throughput: 3115.19 samples/s lr: 1.74e-04 [09/27 10:25:42] lb.utils.events INFO: eta: 6:10:01 iteration: 275299/375342 consumed_samples: 281907200 total_loss: 3.214 time: 0.3287 s/iter data_time: 0.1989 s/iter total_throughput: 3115.20 samples/s lr: 1.74e-04 [09/27 10:26:15] lb.utils.events INFO: eta: 4:49:31 iteration: 275399/375342 consumed_samples: 282009600 total_loss: 3.202 time: 0.3287 s/iter data_time: 0.2128 s/iter total_throughput: 3115.20 samples/s lr: 1.73e-04 [09/27 10:26:48] lb.utils.events INFO: eta: 4:30:36 iteration: 275499/375342 consumed_samples: 282112000 total_loss: 3.208 time: 0.3287 s/iter data_time: 0.2045 s/iter total_throughput: 3115.21 samples/s lr: 1.73e-04 [09/27 10:27:21] lb.utils.events INFO: eta: 4:23:13 iteration: 275599/375342 consumed_samples: 282214400 total_loss: 3.205 time: 0.3287 s/iter data_time: 0.2010 s/iter total_throughput: 3115.21 samples/s lr: 1.73e-04 [09/27 10:27:53] lb.utils.events INFO: eta: 4:19:20 iteration: 275699/375342 consumed_samples: 282316800 total_loss: 3.229 time: 0.3287 s/iter data_time: 0.2032 s/iter total_throughput: 3115.22 samples/s lr: 1.72e-04 [09/27 10:28:26] lb.utils.events INFO: eta: 4:17:15 iteration: 275799/375342 consumed_samples: 282419200 total_loss: 3.21 time: 0.3287 s/iter data_time: 0.2179 s/iter total_throughput: 3115.22 samples/s lr: 1.72e-04 [09/27 10:28:59] lb.utils.events INFO: eta: 4:16:18 iteration: 275899/375342 consumed_samples: 282521600 total_loss: 3.196 time: 0.3287 s/iter data_time: 0.2313 s/iter total_throughput: 3115.22 samples/s lr: 1.72e-04 [09/27 10:29:32] lb.utils.events INFO: eta: 4:16:11 iteration: 275999/375342 consumed_samples: 282624000 total_loss: 3.217 time: 0.3287 s/iter data_time: 0.2329 s/iter total_throughput: 3115.21 samples/s lr: 1.71e-04 [09/27 10:30:05] lb.utils.events INFO: eta: 4:16:41 iteration: 276099/375342 consumed_samples: 282726400 total_loss: 3.225 time: 0.3287 s/iter data_time: 0.2200 s/iter total_throughput: 3115.21 samples/s lr: 1.71e-04 [09/27 10:30:38] lb.utils.events INFO: eta: 4:18:14 iteration: 276199/375342 consumed_samples: 282828800 total_loss: 3.222 time: 0.3287 s/iter data_time: 0.2479 s/iter total_throughput: 3115.20 samples/s lr: 1.71e-04 [09/27 10:31:11] lb.utils.events INFO: eta: 4:19:54 iteration: 276299/375342 consumed_samples: 282931200 total_loss: 3.221 time: 0.3287 s/iter data_time: 0.2231 s/iter total_throughput: 3115.21 samples/s lr: 1.71e-04 [09/27 10:31:43] lb.utils.events INFO: eta: 4:24:09 iteration: 276399/375342 consumed_samples: 283033600 total_loss: 3.219 time: 0.3287 s/iter data_time: 0.2349 s/iter total_throughput: 3115.22 samples/s lr: 1.70e-04 [09/27 10:32:16] lb.utils.events INFO: eta: 4:30:03 iteration: 276499/375342 consumed_samples: 283136000 total_loss: 3.203 time: 0.3287 s/iter data_time: 0.2545 s/iter total_throughput: 3115.22 samples/s lr: 1.70e-04 [09/27 10:32:49] lb.utils.events INFO: eta: 4:54:44 iteration: 276599/375342 consumed_samples: 283238400 total_loss: 3.202 time: 0.3287 s/iter data_time: 0.2459 s/iter total_throughput: 3115.23 samples/s lr: 1.70e-04 [09/27 10:33:22] lb.utils.events INFO: eta: 5:54:05 iteration: 276699/375342 consumed_samples: 283340800 total_loss: 3.211 time: 0.3287 s/iter data_time: 0.2269 s/iter total_throughput: 3115.22 samples/s lr: 1.69e-04 [09/27 10:33:55] lb.utils.events INFO: eta: 6:21:03 iteration: 276799/375342 consumed_samples: 283443200 total_loss: 3.197 time: 0.3287 s/iter data_time: 0.2264 s/iter total_throughput: 3115.21 samples/s lr: 1.69e-04 [09/27 10:34:28] lb.utils.events INFO: eta: 6:19:53 iteration: 276899/375342 consumed_samples: 283545600 total_loss: 3.189 time: 0.3287 s/iter data_time: 0.2077 s/iter total_throughput: 3115.21 samples/s lr: 1.69e-04 [09/27 10:35:01] lb.utils.events INFO: eta: 5:46:09 iteration: 276999/375342 consumed_samples: 283648000 total_loss: 3.198 time: 0.3287 s/iter data_time: 0.2021 s/iter total_throughput: 3115.21 samples/s lr: 1.68e-04 [09/27 10:35:33] lb.utils.events INFO: eta: 5:14:14 iteration: 277099/375342 consumed_samples: 283750400 total_loss: 3.207 time: 0.3287 s/iter data_time: 0.2149 s/iter total_throughput: 3115.23 samples/s lr: 1.68e-04 [09/27 10:36:06] lb.utils.events INFO: eta: 5:09:59 iteration: 277199/375342 consumed_samples: 283852800 total_loss: 3.199 time: 0.3287 s/iter data_time: 0.2112 s/iter total_throughput: 3115.25 samples/s lr: 1.68e-04 [09/27 10:36:38] lb.utils.events INFO: eta: 4:38:54 iteration: 277299/375342 consumed_samples: 283955200 total_loss: 3.197 time: 0.3287 s/iter data_time: 0.2143 s/iter total_throughput: 3115.25 samples/s lr: 1.68e-04 [09/27 10:37:11] lb.utils.events INFO: eta: 4:26:56 iteration: 277399/375342 consumed_samples: 284057600 total_loss: 3.205 time: 0.3287 s/iter data_time: 0.2131 s/iter total_throughput: 3115.25 samples/s lr: 1.67e-04 [09/27 10:37:44] lb.utils.events INFO: eta: 4:21:52 iteration: 277499/375342 consumed_samples: 284160000 total_loss: 3.209 time: 0.3287 s/iter data_time: 0.2249 s/iter total_throughput: 3115.25 samples/s lr: 1.67e-04 [09/27 10:38:17] lb.utils.events INFO: eta: 4:19:31 iteration: 277599/375342 consumed_samples: 284262400 total_loss: 3.215 time: 0.3287 s/iter data_time: 0.2038 s/iter total_throughput: 3115.25 samples/s lr: 1.67e-04 [09/27 10:38:50] lb.utils.events INFO: eta: 4:17:25 iteration: 277699/375342 consumed_samples: 284364800 total_loss: 3.228 time: 0.3287 s/iter data_time: 0.2334 s/iter total_throughput: 3115.25 samples/s lr: 1.66e-04 [09/27 10:39:23] lb.utils.events INFO: eta: 4:15:21 iteration: 277799/375342 consumed_samples: 284467200 total_loss: 3.214 time: 0.3287 s/iter data_time: 0.2036 s/iter total_throughput: 3115.24 samples/s lr: 1.66e-04 [09/27 10:39:55] lb.utils.events INFO: eta: 4:15:13 iteration: 277899/375342 consumed_samples: 284569600 total_loss: 3.169 time: 0.3287 s/iter data_time: 0.2262 s/iter total_throughput: 3115.26 samples/s lr: 1.66e-04 [09/27 10:40:29] lb.utils.events INFO: eta: 4:15:10 iteration: 277999/375342 consumed_samples: 284672000 total_loss: 3.177 time: 0.3287 s/iter data_time: 0.2121 s/iter total_throughput: 3115.25 samples/s lr: 1.65e-04 [09/27 10:41:02] lb.utils.events INFO: eta: 4:13:27 iteration: 278099/375342 consumed_samples: 284774400 total_loss: 3.198 time: 0.3287 s/iter data_time: 0.1977 s/iter total_throughput: 3115.24 samples/s lr: 1.65e-04 [09/27 10:41:34] lb.utils.events INFO: eta: 4:12:54 iteration: 278199/375342 consumed_samples: 284876800 total_loss: 3.217 time: 0.3287 s/iter data_time: 0.2228 s/iter total_throughput: 3115.26 samples/s lr: 1.65e-04 [09/27 10:42:07] lb.utils.events INFO: eta: 4:12:56 iteration: 278299/375342 consumed_samples: 284979200 total_loss: 3.213 time: 0.3287 s/iter data_time: 0.2279 s/iter total_throughput: 3115.26 samples/s lr: 1.65e-04 [09/27 10:42:40] lb.utils.events INFO: eta: 4:12:41 iteration: 278399/375342 consumed_samples: 285081600 total_loss: 3.195 time: 0.3287 s/iter data_time: 0.2135 s/iter total_throughput: 3115.26 samples/s lr: 1.64e-04 [09/27 10:43:12] lb.utils.events INFO: eta: 4:11:47 iteration: 278499/375342 consumed_samples: 285184000 total_loss: 3.202 time: 0.3287 s/iter data_time: 0.2055 s/iter total_throughput: 3115.27 samples/s lr: 1.64e-04 [09/27 10:43:45] lb.utils.events INFO: eta: 4:11:20 iteration: 278599/375342 consumed_samples: 285286400 total_loss: 3.224 time: 0.3287 s/iter data_time: 0.2230 s/iter total_throughput: 3115.27 samples/s lr: 1.64e-04 [09/27 10:44:18] lb.utils.events INFO: eta: 4:11:11 iteration: 278699/375342 consumed_samples: 285388800 total_loss: 3.223 time: 0.3287 s/iter data_time: 0.2229 s/iter total_throughput: 3115.28 samples/s lr: 1.63e-04 [09/27 10:44:51] lb.utils.events INFO: eta: 4:11:53 iteration: 278799/375342 consumed_samples: 285491200 total_loss: 3.194 time: 0.3287 s/iter data_time: 0.2274 s/iter total_throughput: 3115.28 samples/s lr: 1.63e-04 [09/27 10:45:23] lb.utils.events INFO: eta: 4:11:45 iteration: 278899/375342 consumed_samples: 285593600 total_loss: 3.188 time: 0.3287 s/iter data_time: 0.2120 s/iter total_throughput: 3115.29 samples/s lr: 1.63e-04 [09/27 10:45:56] lb.utils.events INFO: eta: 4:11:54 iteration: 278999/375342 consumed_samples: 285696000 total_loss: 3.194 time: 0.3287 s/iter data_time: 0.2174 s/iter total_throughput: 3115.30 samples/s lr: 1.62e-04 [09/27 10:46:28] lb.utils.events INFO: eta: 4:13:41 iteration: 279099/375342 consumed_samples: 285798400 total_loss: 3.191 time: 0.3287 s/iter data_time: 0.2171 s/iter total_throughput: 3115.32 samples/s lr: 1.62e-04 [09/27 10:47:01] lb.utils.events INFO: eta: 4:15:40 iteration: 279199/375342 consumed_samples: 285900800 total_loss: 3.195 time: 0.3287 s/iter data_time: 0.2231 s/iter total_throughput: 3115.31 samples/s lr: 1.62e-04 [09/27 10:47:34] lb.utils.events INFO: eta: 4:14:21 iteration: 279299/375342 consumed_samples: 286003200 total_loss: 3.211 time: 0.3287 s/iter data_time: 0.2246 s/iter total_throughput: 3115.32 samples/s lr: 1.62e-04 [09/27 10:48:07] lb.utils.events INFO: eta: 4:13:31 iteration: 279399/375342 consumed_samples: 286105600 total_loss: 3.191 time: 0.3287 s/iter data_time: 0.2107 s/iter total_throughput: 3115.32 samples/s lr: 1.61e-04 [09/27 10:48:39] lb.utils.events INFO: eta: 4:13:01 iteration: 279499/375342 consumed_samples: 286208000 total_loss: 3.188 time: 0.3287 s/iter data_time: 0.2006 s/iter total_throughput: 3115.34 samples/s lr: 1.61e-04 [09/27 10:49:12] lb.utils.events INFO: eta: 4:12:33 iteration: 279599/375342 consumed_samples: 286310400 total_loss: 3.206 time: 0.3287 s/iter data_time: 0.2202 s/iter total_throughput: 3115.36 samples/s lr: 1.61e-04 [09/27 10:49:44] lb.utils.events INFO: eta: 4:11:49 iteration: 279699/375342 consumed_samples: 286412800 total_loss: 3.204 time: 0.3287 s/iter data_time: 0.2289 s/iter total_throughput: 3115.37 samples/s lr: 1.60e-04 [09/27 10:50:17] lb.utils.events INFO: eta: 4:11:53 iteration: 279799/375342 consumed_samples: 286515200 total_loss: 3.182 time: 0.3287 s/iter data_time: 0.2240 s/iter total_throughput: 3115.37 samples/s lr: 1.60e-04 [09/27 10:50:50] lb.utils.events INFO: eta: 4:12:05 iteration: 279899/375342 consumed_samples: 286617600 total_loss: 3.176 time: 0.3287 s/iter data_time: 0.2214 s/iter total_throughput: 3115.37 samples/s lr: 1.60e-04 [09/27 10:51:22] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0279999 [09/27 10:51:23] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/27 10:51:23] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/27 10:51:27] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0900 s/iter. Inference: 0.1499 s/iter. Eval: 0.0021 s/iter. Total: 0.2421 s/iter. ETA=0:00:08 [09/27 10:51:33] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1470 s/iter. Inference: 0.1524 s/iter. Eval: 0.0020 s/iter. Total: 0.3014 s/iter. ETA=0:00:05 [09/27 10:51:38] lb.evaluation.evaluator INFO: Inference done 49152/50000. Dataloading: 0.1317 s/iter. Inference: 0.1513 s/iter. Eval: 0.0020 s/iter. Total: 0.2851 s/iter. ETA=0:00:00 [09/27 10:51:38] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/27 10:51:38] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.701630 (0.000254 s / iter per device, on 8 devices) [09/27 10:51:38] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000134 s / iter per device, on 8 devices) [09/27 10:51:38] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/27 10:51:38] lb.evaluation.utils INFO: copypaste: Acc@1=78.45400000000001 [09/27 10:51:38] lb.evaluation.utils INFO: copypaste: Acc@5=94.048 [09/27 10:51:38] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 78.45400, better than last best score 78.34000 @ iteration 274999. [09/27 10:51:38] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/27 10:51:39] lb.utils.events INFO: eta: 4:13:48 iteration: 279999/375342 consumed_samples: 286720000 total_loss: 3.197 time: 0.3287 s/iter data_time: 0.2182 s/iter total_throughput: 3115.39 samples/s lr: 1.59e-04 [09/27 10:52:10] lb.utils.events INFO: eta: 4:14:52 iteration: 280099/375342 consumed_samples: 286822400 total_loss: 3.19 time: 0.3287 s/iter data_time: 0.2419 s/iter total_throughput: 3115.46 samples/s lr: 1.59e-04 [09/27 10:52:43] lb.utils.events INFO: eta: 4:13:53 iteration: 280199/375342 consumed_samples: 286924800 total_loss: 3.187 time: 0.3287 s/iter data_time: 0.2231 s/iter total_throughput: 3115.45 samples/s lr: 1.59e-04 [09/27 10:53:16] lb.utils.events INFO: eta: 4:14:19 iteration: 280299/375342 consumed_samples: 287027200 total_loss: 3.198 time: 0.3287 s/iter data_time: 0.2228 s/iter total_throughput: 3115.45 samples/s lr: 1.59e-04 [09/27 10:53:49] lb.utils.events INFO: eta: 4:18:44 iteration: 280399/375342 consumed_samples: 287129600 total_loss: 3.184 time: 0.3287 s/iter data_time: 0.2370 s/iter total_throughput: 3115.44 samples/s lr: 1.58e-04 [09/27 10:54:23] lb.utils.events INFO: eta: 4:19:50 iteration: 280499/375342 consumed_samples: 287232000 total_loss: 3.184 time: 0.3287 s/iter data_time: 0.2199 s/iter total_throughput: 3115.41 samples/s lr: 1.58e-04 [09/27 10:54:56] lb.utils.events INFO: eta: 4:16:53 iteration: 280599/375342 consumed_samples: 287334400 total_loss: 3.188 time: 0.3287 s/iter data_time: 0.2185 s/iter total_throughput: 3115.40 samples/s lr: 1.58e-04 [09/27 10:55:30] lb.utils.events INFO: eta: 4:13:22 iteration: 280699/375342 consumed_samples: 287436800 total_loss: 3.192 time: 0.3287 s/iter data_time: 0.2082 s/iter total_throughput: 3115.37 samples/s lr: 1.57e-04 [09/27 10:56:02] lb.utils.events INFO: eta: 4:10:19 iteration: 280799/375342 consumed_samples: 287539200 total_loss: 3.199 time: 0.3287 s/iter data_time: 0.2238 s/iter total_throughput: 3115.37 samples/s lr: 1.57e-04 [09/27 10:56:35] lb.utils.events INFO: eta: 4:10:07 iteration: 280899/375342 consumed_samples: 287641600 total_loss: 3.197 time: 0.3287 s/iter data_time: 0.2194 s/iter total_throughput: 3115.37 samples/s lr: 1.57e-04 [09/27 10:57:08] lb.utils.events INFO: eta: 4:09:07 iteration: 280999/375342 consumed_samples: 287744000 total_loss: 3.186 time: 0.3287 s/iter data_time: 0.2230 s/iter total_throughput: 3115.37 samples/s lr: 1.56e-04 [09/27 10:57:42] lb.utils.events INFO: eta: 4:08:00 iteration: 281099/375342 consumed_samples: 287846400 total_loss: 3.2 time: 0.3287 s/iter data_time: 0.2301 s/iter total_throughput: 3115.36 samples/s lr: 1.56e-04 [09/27 10:58:15] lb.utils.events INFO: eta: 4:08:09 iteration: 281199/375342 consumed_samples: 287948800 total_loss: 3.208 time: 0.3287 s/iter data_time: 0.2275 s/iter total_throughput: 3115.34 samples/s lr: 1.56e-04 [09/27 10:58:48] lb.utils.events INFO: eta: 4:06:30 iteration: 281299/375342 consumed_samples: 288051200 total_loss: 3.184 time: 0.3287 s/iter data_time: 0.2173 s/iter total_throughput: 3115.32 samples/s lr: 1.56e-04 [09/27 10:59:21] lb.utils.events INFO: eta: 4:06:10 iteration: 281399/375342 consumed_samples: 288153600 total_loss: 3.195 time: 0.3287 s/iter data_time: 0.2413 s/iter total_throughput: 3115.33 samples/s lr: 1.55e-04 [09/27 10:59:55] lb.utils.events INFO: eta: 4:08:28 iteration: 281499/375342 consumed_samples: 288256000 total_loss: 3.194 time: 0.3287 s/iter data_time: 0.2366 s/iter total_throughput: 3115.31 samples/s lr: 1.55e-04 [09/27 11:00:28] lb.utils.events INFO: eta: 4:13:14 iteration: 281599/375342 consumed_samples: 288358400 total_loss: 3.201 time: 0.3287 s/iter data_time: 0.2602 s/iter total_throughput: 3115.29 samples/s lr: 1.55e-04 [09/27 11:01:02] lb.utils.events INFO: eta: 4:18:12 iteration: 281699/375342 consumed_samples: 288460800 total_loss: 3.202 time: 0.3287 s/iter data_time: 0.2182 s/iter total_throughput: 3115.27 samples/s lr: 1.54e-04 [09/27 11:01:35] lb.utils.events INFO: eta: 4:19:44 iteration: 281799/375342 consumed_samples: 288563200 total_loss: 3.182 time: 0.3287 s/iter data_time: 0.2289 s/iter total_throughput: 3115.26 samples/s lr: 1.54e-04 [09/27 11:02:08] lb.utils.events INFO: eta: 4:13:02 iteration: 281899/375342 consumed_samples: 288665600 total_loss: 3.173 time: 0.3287 s/iter data_time: 0.2099 s/iter total_throughput: 3115.25 samples/s lr: 1.54e-04 [09/27 11:02:41] lb.utils.events INFO: eta: 4:10:33 iteration: 281999/375342 consumed_samples: 288768000 total_loss: 3.174 time: 0.3287 s/iter data_time: 0.2096 s/iter total_throughput: 3115.23 samples/s lr: 1.54e-04 [09/27 11:03:14] lb.utils.events INFO: eta: 4:07:44 iteration: 282099/375342 consumed_samples: 288870400 total_loss: 3.178 time: 0.3287 s/iter data_time: 0.2069 s/iter total_throughput: 3115.23 samples/s lr: 1.53e-04 [09/27 11:03:47] lb.utils.events INFO: eta: 4:07:02 iteration: 282199/375342 consumed_samples: 288972800 total_loss: 3.164 time: 0.3287 s/iter data_time: 0.2150 s/iter total_throughput: 3115.23 samples/s lr: 1.53e-04 [09/27 11:04:20] lb.utils.events INFO: eta: 4:08:10 iteration: 282299/375342 consumed_samples: 289075200 total_loss: 3.175 time: 0.3287 s/iter data_time: 0.2175 s/iter total_throughput: 3115.22 samples/s lr: 1.53e-04 [09/27 11:04:53] lb.utils.events INFO: eta: 4:05:40 iteration: 282399/375342 consumed_samples: 289177600 total_loss: 3.197 time: 0.3287 s/iter data_time: 0.2065 s/iter total_throughput: 3115.21 samples/s lr: 1.52e-04 [09/27 11:05:27] lb.utils.events INFO: eta: 4:03:02 iteration: 282499/375342 consumed_samples: 289280000 total_loss: 3.193 time: 0.3287 s/iter data_time: 0.2108 s/iter total_throughput: 3115.20 samples/s lr: 1.52e-04 [09/27 11:06:00] lb.utils.events INFO: eta: 4:00:57 iteration: 282599/375342 consumed_samples: 289382400 total_loss: 3.206 time: 0.3287 s/iter data_time: 0.2000 s/iter total_throughput: 3115.19 samples/s lr: 1.52e-04 [09/27 11:06:32] lb.utils.events INFO: eta: 4:00:02 iteration: 282699/375342 consumed_samples: 289484800 total_loss: 3.21 time: 0.3287 s/iter data_time: 0.2129 s/iter total_throughput: 3115.19 samples/s lr: 1.52e-04 [09/27 11:07:06] lb.utils.events INFO: eta: 4:00:18 iteration: 282799/375342 consumed_samples: 289587200 total_loss: 3.198 time: 0.3287 s/iter data_time: 0.2346 s/iter total_throughput: 3115.18 samples/s lr: 1.51e-04 [09/27 11:07:38] lb.utils.events INFO: eta: 4:01:15 iteration: 282899/375342 consumed_samples: 289689600 total_loss: 3.188 time: 0.3287 s/iter data_time: 0.2200 s/iter total_throughput: 3115.18 samples/s lr: 1.51e-04 [09/27 11:08:12] lb.utils.events INFO: eta: 4:00:59 iteration: 282999/375342 consumed_samples: 289792000 total_loss: 3.172 time: 0.3287 s/iter data_time: 0.2042 s/iter total_throughput: 3115.17 samples/s lr: 1.51e-04 [09/27 11:08:45] lb.utils.events INFO: eta: 4:00:29 iteration: 283099/375342 consumed_samples: 289894400 total_loss: 3.165 time: 0.3287 s/iter data_time: 0.2106 s/iter total_throughput: 3115.16 samples/s lr: 1.50e-04 [09/27 11:09:18] lb.utils.events INFO: eta: 3:59:29 iteration: 283199/375342 consumed_samples: 289996800 total_loss: 3.166 time: 0.3287 s/iter data_time: 0.2124 s/iter total_throughput: 3115.15 samples/s lr: 1.50e-04 [09/27 11:09:51] lb.utils.events INFO: eta: 3:58:08 iteration: 283299/375342 consumed_samples: 290099200 total_loss: 3.182 time: 0.3287 s/iter data_time: 0.2264 s/iter total_throughput: 3115.15 samples/s lr: 1.50e-04 [09/27 11:10:24] lb.utils.events INFO: eta: 3:58:57 iteration: 283399/375342 consumed_samples: 290201600 total_loss: 3.177 time: 0.3287 s/iter data_time: 0.2297 s/iter total_throughput: 3115.13 samples/s lr: 1.49e-04 [09/27 11:10:58] lb.utils.events INFO: eta: 3:59:46 iteration: 283499/375342 consumed_samples: 290304000 total_loss: 3.181 time: 0.3287 s/iter data_time: 0.2296 s/iter total_throughput: 3115.12 samples/s lr: 1.49e-04 [09/27 11:11:31] lb.utils.events INFO: eta: 3:59:57 iteration: 283599/375342 consumed_samples: 290406400 total_loss: 3.171 time: 0.3287 s/iter data_time: 0.2162 s/iter total_throughput: 3115.11 samples/s lr: 1.49e-04 [09/27 11:12:04] lb.utils.events INFO: eta: 4:00:12 iteration: 283699/375342 consumed_samples: 290508800 total_loss: 3.188 time: 0.3287 s/iter data_time: 0.2286 s/iter total_throughput: 3115.10 samples/s lr: 1.49e-04 [09/27 11:12:37] lb.utils.events INFO: eta: 4:00:03 iteration: 283799/375342 consumed_samples: 290611200 total_loss: 3.186 time: 0.3287 s/iter data_time: 0.2283 s/iter total_throughput: 3115.09 samples/s lr: 1.48e-04 [09/27 11:13:10] lb.utils.events INFO: eta: 3:59:21 iteration: 283899/375342 consumed_samples: 290713600 total_loss: 3.179 time: 0.3287 s/iter data_time: 0.2167 s/iter total_throughput: 3115.08 samples/s lr: 1.48e-04 [09/27 11:13:43] lb.utils.events INFO: eta: 4:00:17 iteration: 283999/375342 consumed_samples: 290816000 total_loss: 3.187 time: 0.3287 s/iter data_time: 0.2274 s/iter total_throughput: 3115.07 samples/s lr: 1.48e-04 [09/27 11:14:17] lb.utils.events INFO: eta: 4:02:27 iteration: 284099/375342 consumed_samples: 290918400 total_loss: 3.173 time: 0.3287 s/iter data_time: 0.2317 s/iter total_throughput: 3115.06 samples/s lr: 1.47e-04 [09/27 11:14:50] lb.utils.events INFO: eta: 4:04:51 iteration: 284199/375342 consumed_samples: 291020800 total_loss: 3.176 time: 0.3287 s/iter data_time: 0.2384 s/iter total_throughput: 3115.04 samples/s lr: 1.47e-04 [09/27 11:15:23] lb.utils.events INFO: eta: 4:06:59 iteration: 284299/375342 consumed_samples: 291123200 total_loss: 3.187 time: 0.3287 s/iter data_time: 0.2211 s/iter total_throughput: 3115.04 samples/s lr: 1.47e-04 [09/27 11:15:57] lb.utils.events INFO: eta: 4:06:43 iteration: 284399/375342 consumed_samples: 291225600 total_loss: 3.196 time: 0.3287 s/iter data_time: 0.2353 s/iter total_throughput: 3115.02 samples/s lr: 1.47e-04 [09/27 11:16:30] lb.utils.events INFO: eta: 4:06:04 iteration: 284499/375342 consumed_samples: 291328000 total_loss: 3.194 time: 0.3287 s/iter data_time: 0.2253 s/iter total_throughput: 3115.01 samples/s lr: 1.46e-04 [09/27 11:17:03] lb.utils.events INFO: eta: 4:08:21 iteration: 284599/375342 consumed_samples: 291430400 total_loss: 3.187 time: 0.3287 s/iter data_time: 0.2235 s/iter total_throughput: 3115.01 samples/s lr: 1.46e-04 [09/27 11:17:36] lb.utils.events INFO: eta: 4:10:04 iteration: 284699/375342 consumed_samples: 291532800 total_loss: 3.181 time: 0.3287 s/iter data_time: 0.2232 s/iter total_throughput: 3115.00 samples/s lr: 1.46e-04 [09/27 11:18:09] lb.utils.events INFO: eta: 4:08:47 iteration: 284799/375342 consumed_samples: 291635200 total_loss: 3.188 time: 0.3287 s/iter data_time: 0.2292 s/iter total_throughput: 3115.01 samples/s lr: 1.45e-04 [09/27 11:18:42] lb.utils.events INFO: eta: 4:08:05 iteration: 284899/375342 consumed_samples: 291737600 total_loss: 3.193 time: 0.3287 s/iter data_time: 0.2203 s/iter total_throughput: 3115.00 samples/s lr: 1.45e-04 [09/27 11:19:15] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0284999 [09/27 11:19:15] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/27 11:19:15] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/27 11:19:19] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0866 s/iter. Inference: 0.1504 s/iter. Eval: 0.0023 s/iter. Total: 0.2393 s/iter. ETA=0:00:08 [09/27 11:19:25] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1484 s/iter. Inference: 0.1502 s/iter. Eval: 0.0021 s/iter. Total: 0.3008 s/iter. ETA=0:00:05 [09/27 11:19:31] lb.evaluation.evaluator INFO: Inference done 50000/50000. Dataloading: 0.1340 s/iter. Inference: 0.1509 s/iter. Eval: 0.0021 s/iter. Total: 0.2871 s/iter. ETA=0:00:00 [09/27 11:19:31] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/27 11:19:31] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.635457 (0.000253 s / iter per device, on 8 devices) [09/27 11:19:31] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000133 s / iter per device, on 8 devices) [09/27 11:19:31] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/27 11:19:31] lb.evaluation.utils INFO: copypaste: Acc@1=78.642 [09/27 11:19:31] lb.evaluation.utils INFO: copypaste: Acc@5=94.16600000000001 [09/27 11:19:31] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 78.64200, better than last best score 78.45400 @ iteration 279999. [09/27 11:19:31] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/27 11:19:32] lb.utils.events INFO: eta: 4:06:56 iteration: 284999/375342 consumed_samples: 291840000 total_loss: 3.184 time: 0.3287 s/iter data_time: 0.2189 s/iter total_throughput: 3115.00 samples/s lr: 1.45e-04 [09/27 11:20:03] lb.utils.events INFO: eta: 4:03:30 iteration: 285099/375342 consumed_samples: 291942400 total_loss: 3.179 time: 0.3287 s/iter data_time: 0.2369 s/iter total_throughput: 3115.04 samples/s lr: 1.45e-04 [09/27 11:20:36] lb.utils.events INFO: eta: 4:06:17 iteration: 285199/375342 consumed_samples: 292044800 total_loss: 3.166 time: 0.3287 s/iter data_time: 0.2481 s/iter total_throughput: 3115.03 samples/s lr: 1.44e-04 [09/27 11:21:09] lb.utils.events INFO: eta: 4:16:18 iteration: 285299/375342 consumed_samples: 292147200 total_loss: 3.158 time: 0.3287 s/iter data_time: 0.2434 s/iter total_throughput: 3115.02 samples/s lr: 1.44e-04 [09/27 11:21:43] lb.utils.events INFO: eta: 4:41:00 iteration: 285399/375342 consumed_samples: 292249600 total_loss: 3.186 time: 0.3287 s/iter data_time: 0.2328 s/iter total_throughput: 3115.01 samples/s lr: 1.44e-04 [09/27 11:22:16] lb.utils.events INFO: eta: 4:40:41 iteration: 285499/375342 consumed_samples: 292352000 total_loss: 3.195 time: 0.3287 s/iter data_time: 0.2397 s/iter total_throughput: 3114.99 samples/s lr: 1.43e-04 [09/27 11:22:49] lb.utils.events INFO: eta: 4:35:02 iteration: 285599/375342 consumed_samples: 292454400 total_loss: 3.18 time: 0.3287 s/iter data_time: 0.2249 s/iter total_throughput: 3114.97 samples/s lr: 1.43e-04 [09/27 11:23:22] lb.utils.events INFO: eta: 4:29:23 iteration: 285699/375342 consumed_samples: 292556800 total_loss: 3.182 time: 0.3287 s/iter data_time: 0.2248 s/iter total_throughput: 3114.98 samples/s lr: 1.43e-04 [09/27 11:23:55] lb.utils.events INFO: eta: 4:32:10 iteration: 285799/375342 consumed_samples: 292659200 total_loss: 3.19 time: 0.3287 s/iter data_time: 0.2240 s/iter total_throughput: 3114.99 samples/s lr: 1.43e-04 [09/27 11:24:28] lb.utils.events INFO: eta: 4:28:47 iteration: 285899/375342 consumed_samples: 292761600 total_loss: 3.191 time: 0.3287 s/iter data_time: 0.2346 s/iter total_throughput: 3114.97 samples/s lr: 1.42e-04 [09/27 11:25:02] lb.utils.events INFO: eta: 4:24:21 iteration: 285999/375342 consumed_samples: 292864000 total_loss: 3.151 time: 0.3287 s/iter data_time: 0.2272 s/iter total_throughput: 3114.95 samples/s lr: 1.42e-04 [09/27 11:25:35] lb.utils.events INFO: eta: 4:32:20 iteration: 286099/375342 consumed_samples: 292966400 total_loss: 3.151 time: 0.3287 s/iter data_time: 0.2248 s/iter total_throughput: 3114.94 samples/s lr: 1.42e-04 [09/27 11:26:08] lb.utils.events INFO: eta: 4:24:24 iteration: 286199/375342 consumed_samples: 293068800 total_loss: 3.168 time: 0.3287 s/iter data_time: 0.2331 s/iter total_throughput: 3114.94 samples/s lr: 1.42e-04 [09/27 11:26:41] lb.utils.events INFO: eta: 4:06:29 iteration: 286299/375342 consumed_samples: 293171200 total_loss: 3.163 time: 0.3287 s/iter data_time: 0.2184 s/iter total_throughput: 3114.93 samples/s lr: 1.41e-04 [09/27 11:27:14] lb.utils.events INFO: eta: 4:04:28 iteration: 286399/375342 consumed_samples: 293273600 total_loss: 3.172 time: 0.3287 s/iter data_time: 0.2163 s/iter total_throughput: 3114.94 samples/s lr: 1.41e-04 [09/27 11:27:47] lb.utils.events INFO: eta: 4:01:27 iteration: 286499/375342 consumed_samples: 293376000 total_loss: 3.172 time: 0.3287 s/iter data_time: 0.2339 s/iter total_throughput: 3114.93 samples/s lr: 1.41e-04 [09/27 11:28:20] lb.utils.events INFO: eta: 4:04:25 iteration: 286599/375342 consumed_samples: 293478400 total_loss: 3.145 time: 0.3287 s/iter data_time: 0.2273 s/iter total_throughput: 3114.92 samples/s lr: 1.40e-04 [09/27 11:28:53] lb.utils.events INFO: eta: 4:03:57 iteration: 286699/375342 consumed_samples: 293580800 total_loss: 3.13 time: 0.3287 s/iter data_time: 0.2469 s/iter total_throughput: 3114.91 samples/s lr: 1.40e-04 [09/27 11:29:26] lb.utils.events INFO: eta: 4:02:22 iteration: 286799/375342 consumed_samples: 293683200 total_loss: 3.166 time: 0.3287 s/iter data_time: 0.2300 s/iter total_throughput: 3114.90 samples/s lr: 1.40e-04 [09/27 11:29:59] lb.utils.events INFO: eta: 4:03:15 iteration: 286899/375342 consumed_samples: 293785600 total_loss: 3.181 time: 0.3287 s/iter data_time: 0.2315 s/iter total_throughput: 3114.90 samples/s lr: 1.40e-04 [09/27 11:30:32] lb.utils.events INFO: eta: 4:01:09 iteration: 286999/375342 consumed_samples: 293888000 total_loss: 3.184 time: 0.3287 s/iter data_time: 0.2092 s/iter total_throughput: 3114.89 samples/s lr: 1.39e-04 [09/27 11:31:05] lb.utils.events INFO: eta: 4:01:44 iteration: 287099/375342 consumed_samples: 293990400 total_loss: 3.164 time: 0.3287 s/iter data_time: 0.2134 s/iter total_throughput: 3114.89 samples/s lr: 1.39e-04 [09/27 11:31:38] lb.utils.events INFO: eta: 3:56:48 iteration: 287199/375342 consumed_samples: 294092800 total_loss: 3.146 time: 0.3287 s/iter data_time: 0.2238 s/iter total_throughput: 3114.88 samples/s lr: 1.39e-04 [09/27 11:32:12] lb.utils.events INFO: eta: 3:57:31 iteration: 287299/375342 consumed_samples: 294195200 total_loss: 3.161 time: 0.3287 s/iter data_time: 0.2267 s/iter total_throughput: 3114.87 samples/s lr: 1.38e-04 [09/27 11:32:45] lb.utils.events INFO: eta: 3:58:15 iteration: 287399/375342 consumed_samples: 294297600 total_loss: 3.176 time: 0.3287 s/iter data_time: 0.2243 s/iter total_throughput: 3114.87 samples/s lr: 1.38e-04 [09/27 11:33:18] lb.utils.events INFO: eta: 3:59:38 iteration: 287499/375342 consumed_samples: 294400000 total_loss: 3.169 time: 0.3287 s/iter data_time: 0.2438 s/iter total_throughput: 3114.86 samples/s lr: 1.38e-04 [09/27 11:33:51] lb.utils.events INFO: eta: 3:55:09 iteration: 287599/375342 consumed_samples: 294502400 total_loss: 3.163 time: 0.3287 s/iter data_time: 0.2062 s/iter total_throughput: 3114.85 samples/s lr: 1.38e-04 [09/27 11:34:24] lb.utils.events INFO: eta: 3:53:28 iteration: 287699/375342 consumed_samples: 294604800 total_loss: 3.16 time: 0.3287 s/iter data_time: 0.2194 s/iter total_throughput: 3114.85 samples/s lr: 1.37e-04 [09/27 11:34:57] lb.utils.events INFO: eta: 3:51:54 iteration: 287799/375342 consumed_samples: 294707200 total_loss: 3.147 time: 0.3287 s/iter data_time: 0.2019 s/iter total_throughput: 3114.85 samples/s lr: 1.37e-04 [09/27 11:35:29] lb.utils.events INFO: eta: 3:51:06 iteration: 287899/375342 consumed_samples: 294809600 total_loss: 3.159 time: 0.3287 s/iter data_time: 0.2060 s/iter total_throughput: 3114.85 samples/s lr: 1.37e-04 [09/27 11:36:03] lb.utils.events INFO: eta: 3:50:03 iteration: 287999/375342 consumed_samples: 294912000 total_loss: 3.172 time: 0.3287 s/iter data_time: 0.2105 s/iter total_throughput: 3114.84 samples/s lr: 1.36e-04 [09/27 11:36:36] lb.utils.events INFO: eta: 3:48:18 iteration: 288099/375342 consumed_samples: 295014400 total_loss: 3.172 time: 0.3288 s/iter data_time: 0.1998 s/iter total_throughput: 3114.82 samples/s lr: 1.36e-04 [09/27 11:37:09] lb.utils.events INFO: eta: 3:47:59 iteration: 288199/375342 consumed_samples: 295116800 total_loss: 3.156 time: 0.3288 s/iter data_time: 0.2094 s/iter total_throughput: 3114.82 samples/s lr: 1.36e-04 [09/27 11:37:42] lb.utils.events INFO: eta: 3:45:26 iteration: 288299/375342 consumed_samples: 295219200 total_loss: 3.135 time: 0.3288 s/iter data_time: 0.2123 s/iter total_throughput: 3114.82 samples/s lr: 1.36e-04 [09/27 11:38:15] lb.utils.events INFO: eta: 3:44:35 iteration: 288399/375342 consumed_samples: 295321600 total_loss: 3.142 time: 0.3288 s/iter data_time: 0.1987 s/iter total_throughput: 3114.82 samples/s lr: 1.35e-04 [09/27 11:38:48] lb.utils.events INFO: eta: 3:43:15 iteration: 288499/375342 consumed_samples: 295424000 total_loss: 3.156 time: 0.3288 s/iter data_time: 0.2076 s/iter total_throughput: 3114.81 samples/s lr: 1.35e-04 [09/27 11:39:21] lb.utils.events INFO: eta: 3:42:57 iteration: 288599/375342 consumed_samples: 295526400 total_loss: 3.162 time: 0.3288 s/iter data_time: 0.2005 s/iter total_throughput: 3114.81 samples/s lr: 1.35e-04 [09/27 11:39:54] lb.utils.events INFO: eta: 3:42:07 iteration: 288699/375342 consumed_samples: 295628800 total_loss: 3.16 time: 0.3288 s/iter data_time: 0.2093 s/iter total_throughput: 3114.81 samples/s lr: 1.35e-04 [09/27 11:40:27] lb.utils.events INFO: eta: 3:41:20 iteration: 288799/375342 consumed_samples: 295731200 total_loss: 3.176 time: 0.3288 s/iter data_time: 0.2032 s/iter total_throughput: 3114.80 samples/s lr: 1.34e-04 [09/27 11:41:00] lb.utils.events INFO: eta: 3:40:35 iteration: 288899/375342 consumed_samples: 295833600 total_loss: 3.177 time: 0.3288 s/iter data_time: 0.2141 s/iter total_throughput: 3114.80 samples/s lr: 1.34e-04 [09/27 11:41:32] lb.utils.events INFO: eta: 3:40:30 iteration: 288999/375342 consumed_samples: 295936000 total_loss: 3.16 time: 0.3288 s/iter data_time: 0.2091 s/iter total_throughput: 3114.81 samples/s lr: 1.34e-04 [09/27 11:42:05] lb.utils.events INFO: eta: 3:40:50 iteration: 289099/375342 consumed_samples: 296038400 total_loss: 3.163 time: 0.3287 s/iter data_time: 0.2070 s/iter total_throughput: 3114.83 samples/s lr: 1.33e-04 [09/27 11:42:38] lb.utils.events INFO: eta: 3:41:25 iteration: 289199/375342 consumed_samples: 296140800 total_loss: 3.182 time: 0.3287 s/iter data_time: 0.2153 s/iter total_throughput: 3114.83 samples/s lr: 1.33e-04 [09/27 11:43:11] lb.utils.events INFO: eta: 3:41:23 iteration: 289299/375342 consumed_samples: 296243200 total_loss: 3.166 time: 0.3288 s/iter data_time: 0.2209 s/iter total_throughput: 3114.82 samples/s lr: 1.33e-04 [09/27 11:43:44] lb.utils.events INFO: eta: 3:40:55 iteration: 289399/375342 consumed_samples: 296345600 total_loss: 3.151 time: 0.3288 s/iter data_time: 0.2150 s/iter total_throughput: 3114.82 samples/s lr: 1.33e-04 [09/27 11:44:17] lb.utils.events INFO: eta: 3:41:11 iteration: 289499/375342 consumed_samples: 296448000 total_loss: 3.154 time: 0.3288 s/iter data_time: 0.2192 s/iter total_throughput: 3114.82 samples/s lr: 1.32e-04 [09/27 11:44:50] lb.utils.events INFO: eta: 3:40:42 iteration: 289599/375342 consumed_samples: 296550400 total_loss: 3.153 time: 0.3288 s/iter data_time: 0.2060 s/iter total_throughput: 3114.81 samples/s lr: 1.32e-04 [09/27 11:45:23] lb.utils.events INFO: eta: 3:41:19 iteration: 289699/375342 consumed_samples: 296652800 total_loss: 3.156 time: 0.3288 s/iter data_time: 0.2365 s/iter total_throughput: 3114.80 samples/s lr: 1.32e-04 [09/27 11:45:56] lb.utils.events INFO: eta: 3:42:25 iteration: 289799/375342 consumed_samples: 296755200 total_loss: 3.167 time: 0.3288 s/iter data_time: 0.1985 s/iter total_throughput: 3114.81 samples/s lr: 1.32e-04 [09/27 11:46:29] lb.utils.events INFO: eta: 3:42:14 iteration: 289899/375342 consumed_samples: 296857600 total_loss: 3.156 time: 0.3288 s/iter data_time: 0.2034 s/iter total_throughput: 3114.80 samples/s lr: 1.31e-04 [09/27 11:47:01] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0289999 [09/27 11:47:02] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/27 11:47:02] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/27 11:47:06] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0833 s/iter. Inference: 0.1517 s/iter. Eval: 0.0021 s/iter. Total: 0.2371 s/iter. ETA=0:00:08 [09/27 11:47:11] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1429 s/iter. Inference: 0.1510 s/iter. Eval: 0.0022 s/iter. Total: 0.2961 s/iter. ETA=0:00:05 [09/27 11:47:16] lb.evaluation.evaluator INFO: Inference done 49152/50000. Dataloading: 0.1300 s/iter. Inference: 0.1496 s/iter. Eval: 0.0022 s/iter. Total: 0.2819 s/iter. ETA=0:00:00 [09/27 11:47:17] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/27 11:47:17] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.569377 (0.000251 s / iter per device, on 8 devices) [09/27 11:47:17] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000133 s / iter per device, on 8 devices) [09/27 11:47:17] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/27 11:47:17] lb.evaluation.utils INFO: copypaste: Acc@1=78.898 [09/27 11:47:17] lb.evaluation.utils INFO: copypaste: Acc@5=94.262 [09/27 11:47:17] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 78.89800, better than last best score 78.64200 @ iteration 284999. [09/27 11:47:17] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/27 11:47:18] lb.utils.events INFO: eta: 3:43:04 iteration: 289999/375342 consumed_samples: 296960000 total_loss: 3.145 time: 0.3288 s/iter data_time: 0.2255 s/iter total_throughput: 3114.82 samples/s lr: 1.31e-04 [09/27 11:47:49] lb.utils.events INFO: eta: 3:45:13 iteration: 290099/375342 consumed_samples: 297062400 total_loss: 3.145 time: 0.3287 s/iter data_time: 0.2491 s/iter total_throughput: 3114.87 samples/s lr: 1.31e-04 [09/27 11:48:22] lb.utils.events INFO: eta: 3:45:59 iteration: 290199/375342 consumed_samples: 297164800 total_loss: 3.174 time: 0.3287 s/iter data_time: 0.2543 s/iter total_throughput: 3114.88 samples/s lr: 1.30e-04 [09/27 11:48:55] lb.utils.events INFO: eta: 3:47:12 iteration: 290299/375342 consumed_samples: 297267200 total_loss: 3.164 time: 0.3287 s/iter data_time: 0.2375 s/iter total_throughput: 3114.88 samples/s lr: 1.30e-04 [09/27 11:49:27] lb.utils.events INFO: eta: 3:50:23 iteration: 290399/375342 consumed_samples: 297369600 total_loss: 3.129 time: 0.3287 s/iter data_time: 0.2198 s/iter total_throughput: 3114.88 samples/s lr: 1.30e-04 [09/27 11:50:00] lb.utils.events INFO: eta: 3:50:20 iteration: 290499/375342 consumed_samples: 297472000 total_loss: 3.138 time: 0.3287 s/iter data_time: 0.2151 s/iter total_throughput: 3114.88 samples/s lr: 1.30e-04 [09/27 11:50:33] lb.utils.events INFO: eta: 3:53:17 iteration: 290599/375342 consumed_samples: 297574400 total_loss: 3.14 time: 0.3287 s/iter data_time: 0.2156 s/iter total_throughput: 3114.88 samples/s lr: 1.29e-04 [09/27 11:51:06] lb.utils.events INFO: eta: 3:54:49 iteration: 290699/375342 consumed_samples: 297676800 total_loss: 3.135 time: 0.3287 s/iter data_time: 0.2126 s/iter total_throughput: 3114.88 samples/s lr: 1.29e-04 [09/27 11:51:39] lb.utils.events INFO: eta: 3:58:45 iteration: 290799/375342 consumed_samples: 297779200 total_loss: 3.127 time: 0.3287 s/iter data_time: 0.2214 s/iter total_throughput: 3114.88 samples/s lr: 1.29e-04 [09/27 11:52:11] lb.utils.events INFO: eta: 4:05:04 iteration: 290899/375342 consumed_samples: 297881600 total_loss: 3.15 time: 0.3287 s/iter data_time: 0.2262 s/iter total_throughput: 3114.90 samples/s lr: 1.29e-04 [09/27 11:52:44] lb.utils.events INFO: eta: 4:33:55 iteration: 290999/375342 consumed_samples: 297984000 total_loss: 3.156 time: 0.3287 s/iter data_time: 0.2358 s/iter total_throughput: 3114.90 samples/s lr: 1.28e-04 [09/27 11:53:17] lb.utils.events INFO: eta: 4:10:51 iteration: 291099/375342 consumed_samples: 298086400 total_loss: 3.155 time: 0.3287 s/iter data_time: 0.2202 s/iter total_throughput: 3114.89 samples/s lr: 1.28e-04 [09/27 11:53:50] lb.utils.events INFO: eta: 3:50:47 iteration: 291199/375342 consumed_samples: 298188800 total_loss: 3.153 time: 0.3287 s/iter data_time: 0.2103 s/iter total_throughput: 3114.88 samples/s lr: 1.28e-04 [09/27 11:54:23] lb.utils.events INFO: eta: 3:47:01 iteration: 291299/375342 consumed_samples: 298291200 total_loss: 3.149 time: 0.3287 s/iter data_time: 0.2049 s/iter total_throughput: 3114.89 samples/s lr: 1.28e-04 [09/27 11:54:56] lb.utils.events INFO: eta: 3:43:29 iteration: 291399/375342 consumed_samples: 298393600 total_loss: 3.138 time: 0.3287 s/iter data_time: 0.2058 s/iter total_throughput: 3114.88 samples/s lr: 1.27e-04 [09/27 11:55:29] lb.utils.events INFO: eta: 3:41:57 iteration: 291499/375342 consumed_samples: 298496000 total_loss: 3.142 time: 0.3287 s/iter data_time: 0.2042 s/iter total_throughput: 3114.89 samples/s lr: 1.27e-04 [09/27 11:56:01] lb.utils.events INFO: eta: 3:41:02 iteration: 291599/375342 consumed_samples: 298598400 total_loss: 3.141 time: 0.3287 s/iter data_time: 0.2136 s/iter total_throughput: 3114.90 samples/s lr: 1.27e-04 [09/27 11:56:34] lb.utils.events INFO: eta: 3:39:27 iteration: 291699/375342 consumed_samples: 298700800 total_loss: 3.133 time: 0.3287 s/iter data_time: 0.2046 s/iter total_throughput: 3114.91 samples/s lr: 1.26e-04 [09/27 11:57:07] lb.utils.events INFO: eta: 3:38:38 iteration: 291799/375342 consumed_samples: 298803200 total_loss: 3.157 time: 0.3287 s/iter data_time: 0.2260 s/iter total_throughput: 3114.92 samples/s lr: 1.26e-04 [09/27 11:57:39] lb.utils.events INFO: eta: 3:37:03 iteration: 291899/375342 consumed_samples: 298905600 total_loss: 3.161 time: 0.3287 s/iter data_time: 0.2026 s/iter total_throughput: 3114.93 samples/s lr: 1.26e-04 [09/27 11:58:12] lb.utils.events INFO: eta: 3:35:00 iteration: 291999/375342 consumed_samples: 299008000 total_loss: 3.151 time: 0.3287 s/iter data_time: 0.2110 s/iter total_throughput: 3114.93 samples/s lr: 1.26e-04 [09/27 11:58:45] lb.utils.events INFO: eta: 3:35:03 iteration: 292099/375342 consumed_samples: 299110400 total_loss: 3.138 time: 0.3287 s/iter data_time: 0.2183 s/iter total_throughput: 3114.94 samples/s lr: 1.25e-04 [09/27 11:59:18] lb.utils.events INFO: eta: 3:36:17 iteration: 292199/375342 consumed_samples: 299212800 total_loss: 3.142 time: 0.3287 s/iter data_time: 0.2172 s/iter total_throughput: 3114.93 samples/s lr: 1.25e-04 [09/27 11:59:50] lb.utils.events INFO: eta: 3:36:47 iteration: 292299/375342 consumed_samples: 299315200 total_loss: 3.164 time: 0.3287 s/iter data_time: 0.2186 s/iter total_throughput: 3114.96 samples/s lr: 1.25e-04 [09/27 12:00:23] lb.utils.events INFO: eta: 3:36:42 iteration: 292399/375342 consumed_samples: 299417600 total_loss: 3.14 time: 0.3287 s/iter data_time: 0.2157 s/iter total_throughput: 3114.96 samples/s lr: 1.25e-04 [09/27 12:00:56] lb.utils.events INFO: eta: 3:37:22 iteration: 292499/375342 consumed_samples: 299520000 total_loss: 3.123 time: 0.3287 s/iter data_time: 0.2295 s/iter total_throughput: 3114.96 samples/s lr: 1.24e-04 [09/27 12:01:28] lb.utils.events INFO: eta: 3:38:04 iteration: 292599/375342 consumed_samples: 299622400 total_loss: 3.12 time: 0.3287 s/iter data_time: 0.2321 s/iter total_throughput: 3114.96 samples/s lr: 1.24e-04 [09/27 12:02:01] lb.utils.events INFO: eta: 3:38:51 iteration: 292699/375342 consumed_samples: 299724800 total_loss: 3.151 time: 0.3287 s/iter data_time: 0.2191 s/iter total_throughput: 3114.96 samples/s lr: 1.24e-04 [09/27 12:02:33] lb.utils.events INFO: eta: 3:39:13 iteration: 292799/375342 consumed_samples: 299827200 total_loss: 3.156 time: 0.3287 s/iter data_time: 0.2134 s/iter total_throughput: 3114.98 samples/s lr: 1.24e-04 [09/27 12:03:06] lb.utils.events INFO: eta: 3:38:47 iteration: 292899/375342 consumed_samples: 299929600 total_loss: 3.154 time: 0.3287 s/iter data_time: 0.2075 s/iter total_throughput: 3114.98 samples/s lr: 1.23e-04 [09/27 12:03:39] lb.utils.events INFO: eta: 3:38:11 iteration: 292999/375342 consumed_samples: 300032000 total_loss: 3.149 time: 0.3287 s/iter data_time: 0.2047 s/iter total_throughput: 3114.99 samples/s lr: 1.23e-04 [09/27 12:04:11] lb.utils.events INFO: eta: 3:36:29 iteration: 293099/375342 consumed_samples: 300134400 total_loss: 3.139 time: 0.3287 s/iter data_time: 0.2148 s/iter total_throughput: 3115.00 samples/s lr: 1.23e-04 [09/27 12:04:44] lb.utils.events INFO: eta: 3:35:41 iteration: 293199/375342 consumed_samples: 300236800 total_loss: 3.136 time: 0.3287 s/iter data_time: 0.2104 s/iter total_throughput: 3115.00 samples/s lr: 1.22e-04 [09/27 12:05:17] lb.utils.events INFO: eta: 3:33:45 iteration: 293299/375342 consumed_samples: 300339200 total_loss: 3.13 time: 0.3287 s/iter data_time: 0.2053 s/iter total_throughput: 3115.02 samples/s lr: 1.22e-04 [09/27 12:05:49] lb.utils.events INFO: eta: 3:34:43 iteration: 293399/375342 consumed_samples: 300441600 total_loss: 3.13 time: 0.3287 s/iter data_time: 0.2213 s/iter total_throughput: 3115.04 samples/s lr: 1.22e-04 [09/27 12:06:22] lb.utils.events INFO: eta: 3:34:15 iteration: 293499/375342 consumed_samples: 300544000 total_loss: 3.128 time: 0.3287 s/iter data_time: 0.2306 s/iter total_throughput: 3115.05 samples/s lr: 1.22e-04 [09/27 12:06:54] lb.utils.events INFO: eta: 3:33:39 iteration: 293599/375342 consumed_samples: 300646400 total_loss: 3.139 time: 0.3287 s/iter data_time: 0.1961 s/iter total_throughput: 3115.06 samples/s lr: 1.21e-04 [09/27 12:07:26] lb.utils.events INFO: eta: 3:32:40 iteration: 293699/375342 consumed_samples: 300748800 total_loss: 3.146 time: 0.3287 s/iter data_time: 0.2111 s/iter total_throughput: 3115.08 samples/s lr: 1.21e-04 [09/27 12:07:59] lb.utils.events INFO: eta: 3:32:19 iteration: 293799/375342 consumed_samples: 300851200 total_loss: 3.125 time: 0.3287 s/iter data_time: 0.2146 s/iter total_throughput: 3115.10 samples/s lr: 1.21e-04 [09/27 12:08:31] lb.utils.events INFO: eta: 3:33:22 iteration: 293899/375342 consumed_samples: 300953600 total_loss: 3.129 time: 0.3287 s/iter data_time: 0.2117 s/iter total_throughput: 3115.11 samples/s lr: 1.21e-04 [09/27 12:09:04] lb.utils.events INFO: eta: 3:34:36 iteration: 293999/375342 consumed_samples: 301056000 total_loss: 3.152 time: 0.3287 s/iter data_time: 0.2122 s/iter total_throughput: 3115.12 samples/s lr: 1.20e-04 [09/27 12:09:36] lb.utils.events INFO: eta: 3:36:12 iteration: 294099/375342 consumed_samples: 301158400 total_loss: 3.144 time: 0.3287 s/iter data_time: 0.2297 s/iter total_throughput: 3115.13 samples/s lr: 1.20e-04 [09/27 12:10:10] lb.utils.events INFO: eta: 3:36:10 iteration: 294199/375342 consumed_samples: 301260800 total_loss: 3.131 time: 0.3287 s/iter data_time: 0.2310 s/iter total_throughput: 3115.11 samples/s lr: 1.20e-04 [09/27 12:10:43] lb.utils.events INFO: eta: 3:35:46 iteration: 294299/375342 consumed_samples: 301363200 total_loss: 3.111 time: 0.3287 s/iter data_time: 0.2164 s/iter total_throughput: 3115.12 samples/s lr: 1.20e-04 [09/27 12:11:16] lb.utils.events INFO: eta: 3:33:56 iteration: 294399/375342 consumed_samples: 301465600 total_loss: 3.113 time: 0.3287 s/iter data_time: 0.2107 s/iter total_throughput: 3115.10 samples/s lr: 1.19e-04 [09/27 12:11:49] lb.utils.events INFO: eta: 3:32:29 iteration: 294499/375342 consumed_samples: 301568000 total_loss: 3.115 time: 0.3287 s/iter data_time: 0.2052 s/iter total_throughput: 3115.10 samples/s lr: 1.19e-04 [09/27 12:12:22] lb.utils.events INFO: eta: 3:31:02 iteration: 294599/375342 consumed_samples: 301670400 total_loss: 3.142 time: 0.3287 s/iter data_time: 0.2053 s/iter total_throughput: 3115.09 samples/s lr: 1.19e-04 [09/27 12:12:56] lb.utils.events INFO: eta: 3:30:47 iteration: 294699/375342 consumed_samples: 301772800 total_loss: 3.14 time: 0.3287 s/iter data_time: 0.2061 s/iter total_throughput: 3115.07 samples/s lr: 1.19e-04 [09/27 12:13:29] lb.utils.events INFO: eta: 3:29:32 iteration: 294799/375342 consumed_samples: 301875200 total_loss: 3.136 time: 0.3287 s/iter data_time: 0.2036 s/iter total_throughput: 3115.06 samples/s lr: 1.18e-04 [09/27 12:14:02] lb.utils.events INFO: eta: 3:28:41 iteration: 294899/375342 consumed_samples: 301977600 total_loss: 3.138 time: 0.3287 s/iter data_time: 0.2421 s/iter total_throughput: 3115.06 samples/s lr: 1.18e-04 [09/27 12:14:35] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0294999 [09/27 12:14:36] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/27 12:14:36] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/27 12:14:40] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0820 s/iter. Inference: 0.1500 s/iter. Eval: 0.0021 s/iter. Total: 0.2342 s/iter. ETA=0:00:08 [09/27 12:14:45] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1456 s/iter. Inference: 0.1497 s/iter. Eval: 0.0021 s/iter. Total: 0.2974 s/iter. ETA=0:00:05 [09/27 12:14:51] lb.evaluation.evaluator INFO: Inference done 50000/50000. Dataloading: 0.1327 s/iter. Inference: 0.1505 s/iter. Eval: 0.0021 s/iter. Total: 0.2854 s/iter. ETA=0:00:00 [09/27 12:14:51] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/27 12:14:51] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.559751 (0.000251 s / iter per device, on 8 devices) [09/27 12:14:51] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000132 s / iter per device, on 8 devices) [09/27 12:14:51] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/27 12:14:51] lb.evaluation.utils INFO: copypaste: Acc@1=78.97800000000001 [09/27 12:14:51] lb.evaluation.utils INFO: copypaste: Acc@5=94.152 [09/27 12:14:51] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 78.97800, better than last best score 78.89800 @ iteration 289999. [09/27 12:14:51] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/27 12:14:52] lb.utils.events INFO: eta: 3:28:21 iteration: 294999/375342 consumed_samples: 302080000 total_loss: 3.142 time: 0.3287 s/iter data_time: 0.2171 s/iter total_throughput: 3115.05 samples/s lr: 1.18e-04 [09/27 12:15:23] lb.utils.events INFO: eta: 3:28:06 iteration: 295099/375342 consumed_samples: 302182400 total_loss: 3.147 time: 0.3287 s/iter data_time: 0.2624 s/iter total_throughput: 3115.08 samples/s lr: 1.18e-04 [09/27 12:15:57] lb.utils.events INFO: eta: 3:27:22 iteration: 295199/375342 consumed_samples: 302284800 total_loss: 3.148 time: 0.3287 s/iter data_time: 0.2041 s/iter total_throughput: 3115.06 samples/s lr: 1.17e-04 [09/27 12:16:30] lb.utils.events INFO: eta: 3:27:20 iteration: 295299/375342 consumed_samples: 302387200 total_loss: 3.148 time: 0.3287 s/iter data_time: 0.2202 s/iter total_throughput: 3115.06 samples/s lr: 1.17e-04 [09/27 12:17:03] lb.utils.events INFO: eta: 3:28:04 iteration: 295399/375342 consumed_samples: 302489600 total_loss: 3.135 time: 0.3287 s/iter data_time: 0.2219 s/iter total_throughput: 3115.05 samples/s lr: 1.17e-04 [09/27 12:17:36] lb.utils.events INFO: eta: 3:28:13 iteration: 295499/375342 consumed_samples: 302592000 total_loss: 3.104 time: 0.3287 s/iter data_time: 0.2247 s/iter total_throughput: 3115.04 samples/s lr: 1.16e-04 [09/27 12:18:09] lb.utils.events INFO: eta: 3:27:42 iteration: 295599/375342 consumed_samples: 302694400 total_loss: 3.133 time: 0.3287 s/iter data_time: 0.2074 s/iter total_throughput: 3115.03 samples/s lr: 1.16e-04 [09/27 12:18:43] lb.utils.events INFO: eta: 3:26:58 iteration: 295699/375342 consumed_samples: 302796800 total_loss: 3.14 time: 0.3287 s/iter data_time: 0.2065 s/iter total_throughput: 3115.02 samples/s lr: 1.16e-04 [09/27 12:19:15] lb.utils.events INFO: eta: 3:27:22 iteration: 295799/375342 consumed_samples: 302899200 total_loss: 3.128 time: 0.3287 s/iter data_time: 0.2069 s/iter total_throughput: 3115.03 samples/s lr: 1.16e-04 [09/27 12:19:48] lb.utils.events INFO: eta: 3:26:25 iteration: 295899/375342 consumed_samples: 303001600 total_loss: 3.143 time: 0.3287 s/iter data_time: 0.2340 s/iter total_throughput: 3115.03 samples/s lr: 1.15e-04 [09/27 12:20:21] lb.utils.events INFO: eta: 3:27:39 iteration: 295999/375342 consumed_samples: 303104000 total_loss: 3.145 time: 0.3287 s/iter data_time: 0.2439 s/iter total_throughput: 3115.04 samples/s lr: 1.15e-04 [09/27 12:20:54] lb.utils.events INFO: eta: 3:29:17 iteration: 296099/375342 consumed_samples: 303206400 total_loss: 3.141 time: 0.3287 s/iter data_time: 0.2440 s/iter total_throughput: 3115.03 samples/s lr: 1.15e-04 [09/27 12:21:27] lb.utils.events INFO: eta: 3:32:14 iteration: 296199/375342 consumed_samples: 303308800 total_loss: 3.141 time: 0.3287 s/iter data_time: 0.2507 s/iter total_throughput: 3115.02 samples/s lr: 1.15e-04 [09/27 12:22:00] lb.utils.events INFO: eta: 3:31:25 iteration: 296299/375342 consumed_samples: 303411200 total_loss: 3.125 time: 0.3287 s/iter data_time: 0.2023 s/iter total_throughput: 3115.01 samples/s lr: 1.14e-04 [09/27 12:22:33] lb.utils.events INFO: eta: 3:28:56 iteration: 296399/375342 consumed_samples: 303513600 total_loss: 3.139 time: 0.3287 s/iter data_time: 0.2179 s/iter total_throughput: 3115.00 samples/s lr: 1.14e-04 [09/27 12:23:06] lb.utils.events INFO: eta: 3:28:02 iteration: 296499/375342 consumed_samples: 303616000 total_loss: 3.135 time: 0.3287 s/iter data_time: 0.2030 s/iter total_throughput: 3114.99 samples/s lr: 1.14e-04 [09/27 12:23:40] lb.utils.events INFO: eta: 3:27:39 iteration: 296599/375342 consumed_samples: 303718400 total_loss: 3.115 time: 0.3287 s/iter data_time: 0.2007 s/iter total_throughput: 3114.98 samples/s lr: 1.14e-04 [09/27 12:24:12] lb.utils.events INFO: eta: 3:27:49 iteration: 296699/375342 consumed_samples: 303820800 total_loss: 3.109 time: 0.3287 s/iter data_time: 0.2002 s/iter total_throughput: 3114.99 samples/s lr: 1.13e-04 [09/27 12:24:46] lb.utils.events INFO: eta: 3:27:18 iteration: 296799/375342 consumed_samples: 303923200 total_loss: 3.114 time: 0.3287 s/iter data_time: 0.2108 s/iter total_throughput: 3114.97 samples/s lr: 1.13e-04 [09/27 12:25:19] lb.utils.events INFO: eta: 3:26:42 iteration: 296899/375342 consumed_samples: 304025600 total_loss: 3.125 time: 0.3287 s/iter data_time: 0.2029 s/iter total_throughput: 3114.98 samples/s lr: 1.13e-04 [09/27 12:25:51] lb.utils.events INFO: eta: 3:24:33 iteration: 296999/375342 consumed_samples: 304128000 total_loss: 3.121 time: 0.3287 s/iter data_time: 0.2263 s/iter total_throughput: 3114.99 samples/s lr: 1.13e-04 [09/27 12:26:24] lb.utils.events INFO: eta: 3:23:50 iteration: 297099/375342 consumed_samples: 304230400 total_loss: 3.126 time: 0.3287 s/iter data_time: 0.2236 s/iter total_throughput: 3114.98 samples/s lr: 1.12e-04 [09/27 12:26:58] lb.utils.events INFO: eta: 3:22:32 iteration: 297199/375342 consumed_samples: 304332800 total_loss: 3.12 time: 0.3287 s/iter data_time: 0.2065 s/iter total_throughput: 3114.96 samples/s lr: 1.12e-04 [09/27 12:27:31] lb.utils.events INFO: eta: 3:22:00 iteration: 297299/375342 consumed_samples: 304435200 total_loss: 3.109 time: 0.3287 s/iter data_time: 0.2285 s/iter total_throughput: 3114.95 samples/s lr: 1.12e-04 [09/27 12:28:04] lb.utils.events INFO: eta: 3:22:17 iteration: 297399/375342 consumed_samples: 304537600 total_loss: 3.111 time: 0.3287 s/iter data_time: 0.2161 s/iter total_throughput: 3114.94 samples/s lr: 1.12e-04 [09/27 12:28:37] lb.utils.events INFO: eta: 3:22:26 iteration: 297499/375342 consumed_samples: 304640000 total_loss: 3.127 time: 0.3287 s/iter data_time: 0.2246 s/iter total_throughput: 3114.94 samples/s lr: 1.11e-04 [09/27 12:29:10] lb.utils.events INFO: eta: 3:22:19 iteration: 297599/375342 consumed_samples: 304742400 total_loss: 3.136 time: 0.3287 s/iter data_time: 0.2272 s/iter total_throughput: 3114.95 samples/s lr: 1.11e-04 [09/27 12:29:43] lb.utils.events INFO: eta: 3:22:41 iteration: 297699/375342 consumed_samples: 304844800 total_loss: 3.121 time: 0.3287 s/iter data_time: 0.2400 s/iter total_throughput: 3114.94 samples/s lr: 1.11e-04 [09/27 12:30:16] lb.utils.events INFO: eta: 3:24:26 iteration: 297799/375342 consumed_samples: 304947200 total_loss: 3.121 time: 0.3287 s/iter data_time: 0.2475 s/iter total_throughput: 3114.93 samples/s lr: 1.11e-04 [09/27 12:30:49] lb.utils.events INFO: eta: 3:27:48 iteration: 297899/375342 consumed_samples: 305049600 total_loss: 3.095 time: 0.3287 s/iter data_time: 0.2209 s/iter total_throughput: 3114.93 samples/s lr: 1.10e-04 [09/27 12:31:22] lb.utils.events INFO: eta: 3:31:19 iteration: 297999/375342 consumed_samples: 305152000 total_loss: 3.099 time: 0.3287 s/iter data_time: 0.2459 s/iter total_throughput: 3114.93 samples/s lr: 1.10e-04 [09/27 12:31:55] lb.utils.events INFO: eta: 3:32:29 iteration: 298099/375342 consumed_samples: 305254400 total_loss: 3.118 time: 0.3287 s/iter data_time: 0.2348 s/iter total_throughput: 3114.92 samples/s lr: 1.10e-04 [09/27 12:32:28] lb.utils.events INFO: eta: 3:35:07 iteration: 298199/375342 consumed_samples: 305356800 total_loss: 3.124 time: 0.3287 s/iter data_time: 0.2204 s/iter total_throughput: 3114.93 samples/s lr: 1.10e-04 [09/27 12:33:01] lb.utils.events INFO: eta: 3:45:07 iteration: 298299/375342 consumed_samples: 305459200 total_loss: 3.126 time: 0.3287 s/iter data_time: 0.2061 s/iter total_throughput: 3114.91 samples/s lr: 1.09e-04 [09/27 12:33:34] lb.utils.events INFO: eta: 3:40:00 iteration: 298399/375342 consumed_samples: 305561600 total_loss: 3.107 time: 0.3287 s/iter data_time: 0.2077 s/iter total_throughput: 3114.89 samples/s lr: 1.09e-04 [09/27 12:34:07] lb.utils.events INFO: eta: 3:38:17 iteration: 298499/375342 consumed_samples: 305664000 total_loss: 3.112 time: 0.3287 s/iter data_time: 0.2107 s/iter total_throughput: 3114.89 samples/s lr: 1.09e-04 [09/27 12:34:40] lb.utils.events INFO: eta: 3:55:43 iteration: 298599/375342 consumed_samples: 305766400 total_loss: 3.118 time: 0.3287 s/iter data_time: 0.2424 s/iter total_throughput: 3114.89 samples/s lr: 1.09e-04 [09/27 12:35:13] lb.utils.events INFO: eta: 4:22:02 iteration: 298699/375342 consumed_samples: 305868800 total_loss: 3.111 time: 0.3287 s/iter data_time: 0.2249 s/iter total_throughput: 3114.90 samples/s lr: 1.08e-04 [09/27 12:35:46] lb.utils.events INFO: eta: 4:06:51 iteration: 298799/375342 consumed_samples: 305971200 total_loss: 3.116 time: 0.3287 s/iter data_time: 0.2195 s/iter total_throughput: 3114.89 samples/s lr: 1.08e-04 [09/27 12:36:19] lb.utils.events INFO: eta: 4:04:57 iteration: 298899/375342 consumed_samples: 306073600 total_loss: 3.139 time: 0.3287 s/iter data_time: 0.2405 s/iter total_throughput: 3114.90 samples/s lr: 1.08e-04 [09/27 12:36:52] lb.utils.events INFO: eta: 3:44:41 iteration: 298999/375342 consumed_samples: 306176000 total_loss: 3.147 time: 0.3287 s/iter data_time: 0.2322 s/iter total_throughput: 3114.89 samples/s lr: 1.08e-04 [09/27 12:37:25] lb.utils.events INFO: eta: 3:32:47 iteration: 299099/375342 consumed_samples: 306278400 total_loss: 3.118 time: 0.3287 s/iter data_time: 0.2206 s/iter total_throughput: 3114.88 samples/s lr: 1.07e-04 [09/27 12:37:58] lb.utils.events INFO: eta: 3:33:12 iteration: 299199/375342 consumed_samples: 306380800 total_loss: 3.104 time: 0.3287 s/iter data_time: 0.2255 s/iter total_throughput: 3114.88 samples/s lr: 1.07e-04 [09/27 12:38:31] lb.utils.events INFO: eta: 3:30:14 iteration: 299299/375342 consumed_samples: 306483200 total_loss: 3.104 time: 0.3287 s/iter data_time: 0.2163 s/iter total_throughput: 3114.89 samples/s lr: 1.07e-04 [09/27 12:39:04] lb.utils.events INFO: eta: 3:30:38 iteration: 299399/375342 consumed_samples: 306585600 total_loss: 3.114 time: 0.3287 s/iter data_time: 0.2040 s/iter total_throughput: 3114.88 samples/s lr: 1.07e-04 [09/27 12:39:37] lb.utils.events INFO: eta: 3:29:25 iteration: 299499/375342 consumed_samples: 306688000 total_loss: 3.134 time: 0.3287 s/iter data_time: 0.2238 s/iter total_throughput: 3114.87 samples/s lr: 1.06e-04 [09/27 12:40:10] lb.utils.events INFO: eta: 3:28:45 iteration: 299599/375342 consumed_samples: 306790400 total_loss: 3.113 time: 0.3287 s/iter data_time: 0.2311 s/iter total_throughput: 3114.87 samples/s lr: 1.06e-04 [09/27 12:40:43] lb.utils.events INFO: eta: 3:22:16 iteration: 299699/375342 consumed_samples: 306892800 total_loss: 3.107 time: 0.3287 s/iter data_time: 0.2159 s/iter total_throughput: 3114.86 samples/s lr: 1.06e-04 [09/27 12:41:16] lb.utils.events INFO: eta: 3:19:52 iteration: 299799/375342 consumed_samples: 306995200 total_loss: 3.112 time: 0.3287 s/iter data_time: 0.2033 s/iter total_throughput: 3114.85 samples/s lr: 1.06e-04 [09/27 12:41:49] lb.utils.events INFO: eta: 3:17:38 iteration: 299899/375342 consumed_samples: 307097600 total_loss: 3.11 time: 0.3287 s/iter data_time: 0.2056 s/iter total_throughput: 3114.85 samples/s lr: 1.05e-04 [09/27 12:42:22] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0299999 [09/27 12:42:23] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/27 12:42:23] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/27 12:42:27] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0909 s/iter. Inference: 0.1511 s/iter. Eval: 0.0021 s/iter. Total: 0.2440 s/iter. ETA=0:00:09 [09/27 12:42:32] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1514 s/iter. Inference: 0.1505 s/iter. Eval: 0.0020 s/iter. Total: 0.3040 s/iter. ETA=0:00:05 [09/27 12:42:38] lb.evaluation.evaluator INFO: Inference done 50000/50000. Dataloading: 0.1365 s/iter. Inference: 0.1507 s/iter. Eval: 0.0020 s/iter. Total: 0.2894 s/iter. ETA=0:00:00 [09/27 12:42:38] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/27 12:42:38] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.737439 (0.000255 s / iter per device, on 8 devices) [09/27 12:42:38] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000133 s / iter per device, on 8 devices) [09/27 12:42:38] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/27 12:42:38] lb.evaluation.utils INFO: copypaste: Acc@1=79.23599999999999 [09/27 12:42:38] lb.evaluation.utils INFO: copypaste: Acc@5=94.28999999999999 [09/27 12:42:38] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 79.23600, better than last best score 78.97800 @ iteration 294999. [09/27 12:42:38] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/27 12:42:39] lb.utils.events INFO: eta: 3:16:23 iteration: 299999/375342 consumed_samples: 307200000 total_loss: 3.115 time: 0.3287 s/iter data_time: 0.2131 s/iter total_throughput: 3114.85 samples/s lr: 1.05e-04 [09/27 12:43:10] lb.utils.events INFO: eta: 3:14:56 iteration: 300099/375342 consumed_samples: 307302400 total_loss: 3.118 time: 0.3287 s/iter data_time: 0.2111 s/iter total_throughput: 3114.91 samples/s lr: 1.05e-04 [09/27 12:43:43] lb.utils.events INFO: eta: 3:14:51 iteration: 300199/375342 consumed_samples: 307404800 total_loss: 3.12 time: 0.3287 s/iter data_time: 0.2279 s/iter total_throughput: 3114.89 samples/s lr: 1.05e-04 [09/27 12:44:16] lb.utils.events INFO: eta: 3:15:20 iteration: 300299/375342 consumed_samples: 307507200 total_loss: 3.096 time: 0.3287 s/iter data_time: 0.2235 s/iter total_throughput: 3114.89 samples/s lr: 1.04e-04 [09/27 12:44:49] lb.utils.events INFO: eta: 3:15:38 iteration: 300399/375342 consumed_samples: 307609600 total_loss: 3.102 time: 0.3287 s/iter data_time: 0.2285 s/iter total_throughput: 3114.89 samples/s lr: 1.04e-04 [09/27 12:45:22] lb.utils.events INFO: eta: 3:16:41 iteration: 300499/375342 consumed_samples: 307712000 total_loss: 3.109 time: 0.3287 s/iter data_time: 0.2254 s/iter total_throughput: 3114.89 samples/s lr: 1.04e-04 [09/27 12:45:55] lb.utils.events INFO: eta: 3:15:36 iteration: 300599/375342 consumed_samples: 307814400 total_loss: 3.093 time: 0.3287 s/iter data_time: 0.2236 s/iter total_throughput: 3114.89 samples/s lr: 1.04e-04 [09/27 12:46:28] lb.utils.events INFO: eta: 3:17:57 iteration: 300699/375342 consumed_samples: 307916800 total_loss: 3.1 time: 0.3287 s/iter data_time: 0.2226 s/iter total_throughput: 3114.89 samples/s lr: 1.04e-04 [09/27 12:47:01] lb.utils.events INFO: eta: 3:20:49 iteration: 300799/375342 consumed_samples: 308019200 total_loss: 3.108 time: 0.3287 s/iter data_time: 0.2438 s/iter total_throughput: 3114.89 samples/s lr: 1.03e-04 [09/27 12:47:34] lb.utils.events INFO: eta: 3:22:28 iteration: 300899/375342 consumed_samples: 308121600 total_loss: 3.103 time: 0.3287 s/iter data_time: 0.2277 s/iter total_throughput: 3114.88 samples/s lr: 1.03e-04 [09/27 12:48:07] lb.utils.events INFO: eta: 3:24:27 iteration: 300999/375342 consumed_samples: 308224000 total_loss: 3.111 time: 0.3287 s/iter data_time: 0.2267 s/iter total_throughput: 3114.87 samples/s lr: 1.03e-04 [09/27 12:48:40] lb.utils.events INFO: eta: 3:27:35 iteration: 301099/375342 consumed_samples: 308326400 total_loss: 3.113 time: 0.3287 s/iter data_time: 0.2171 s/iter total_throughput: 3114.87 samples/s lr: 1.03e-04 [09/27 12:49:13] lb.utils.events INFO: eta: 3:26:42 iteration: 301199/375342 consumed_samples: 308428800 total_loss: 3.12 time: 0.3287 s/iter data_time: 0.2219 s/iter total_throughput: 3114.87 samples/s lr: 1.02e-04 [09/27 12:49:46] lb.utils.events INFO: eta: 3:23:37 iteration: 301299/375342 consumed_samples: 308531200 total_loss: 3.134 time: 0.3287 s/iter data_time: 0.2346 s/iter total_throughput: 3114.86 samples/s lr: 1.02e-04 [09/27 12:50:19] lb.utils.events INFO: eta: 3:22:25 iteration: 301399/375342 consumed_samples: 308633600 total_loss: 3.107 time: 0.3287 s/iter data_time: 0.2247 s/iter total_throughput: 3114.86 samples/s lr: 1.02e-04 [09/27 12:50:51] lb.utils.events INFO: eta: 3:22:03 iteration: 301499/375342 consumed_samples: 308736000 total_loss: 3.09 time: 0.3287 s/iter data_time: 0.2363 s/iter total_throughput: 3114.87 samples/s lr: 1.02e-04 [09/27 12:51:25] lb.utils.events INFO: eta: 3:21:47 iteration: 301599/375342 consumed_samples: 308838400 total_loss: 3.089 time: 0.3287 s/iter data_time: 0.2333 s/iter total_throughput: 3114.86 samples/s lr: 1.01e-04 [09/27 12:51:58] lb.utils.events INFO: eta: 3:18:34 iteration: 301699/375342 consumed_samples: 308940800 total_loss: 3.089 time: 0.3287 s/iter data_time: 0.2053 s/iter total_throughput: 3114.84 samples/s lr: 1.01e-04 [09/27 12:52:32] lb.utils.events INFO: eta: 3:14:13 iteration: 301799/375342 consumed_samples: 309043200 total_loss: 3.083 time: 0.3288 s/iter data_time: 0.2068 s/iter total_throughput: 3114.82 samples/s lr: 1.01e-04 [09/27 12:53:04] lb.utils.events INFO: eta: 3:11:47 iteration: 301899/375342 consumed_samples: 309145600 total_loss: 3.087 time: 0.3288 s/iter data_time: 0.2028 s/iter total_throughput: 3114.82 samples/s lr: 1.01e-04 [09/27 12:53:37] lb.utils.events INFO: eta: 3:10:20 iteration: 301999/375342 consumed_samples: 309248000 total_loss: 3.093 time: 0.3288 s/iter data_time: 0.2041 s/iter total_throughput: 3114.83 samples/s lr: 1.00e-04 [09/27 12:54:10] lb.utils.events INFO: eta: 3:09:51 iteration: 302099/375342 consumed_samples: 309350400 total_loss: 3.104 time: 0.3287 s/iter data_time: 0.2048 s/iter total_throughput: 3114.83 samples/s lr: 1.00e-04 [09/27 12:54:43] lb.utils.events INFO: eta: 3:09:01 iteration: 302199/375342 consumed_samples: 309452800 total_loss: 3.114 time: 0.3288 s/iter data_time: 0.2045 s/iter total_throughput: 3114.82 samples/s lr: 9.99e-05 [09/27 12:55:16] lb.utils.events INFO: eta: 3:08:39 iteration: 302299/375342 consumed_samples: 309555200 total_loss: 3.122 time: 0.3288 s/iter data_time: 0.2181 s/iter total_throughput: 3114.82 samples/s lr: 9.97e-05 [09/27 12:55:49] lb.utils.events INFO: eta: 3:08:16 iteration: 302399/375342 consumed_samples: 309657600 total_loss: 3.128 time: 0.3288 s/iter data_time: 0.2487 s/iter total_throughput: 3114.83 samples/s lr: 9.94e-05 [09/27 12:56:22] lb.utils.events INFO: eta: 3:08:11 iteration: 302499/375342 consumed_samples: 309760000 total_loss: 3.132 time: 0.3288 s/iter data_time: 0.2242 s/iter total_throughput: 3114.82 samples/s lr: 9.92e-05 [09/27 12:56:55] lb.utils.events INFO: eta: 3:08:08 iteration: 302599/375342 consumed_samples: 309862400 total_loss: 3.135 time: 0.3288 s/iter data_time: 0.2303 s/iter total_throughput: 3114.82 samples/s lr: 9.89e-05 [09/27 12:57:28] lb.utils.events INFO: eta: 3:08:45 iteration: 302699/375342 consumed_samples: 309964800 total_loss: 3.124 time: 0.3288 s/iter data_time: 0.2198 s/iter total_throughput: 3114.82 samples/s lr: 9.87e-05 [09/27 12:58:00] lb.utils.events INFO: eta: 3:10:15 iteration: 302799/375342 consumed_samples: 310067200 total_loss: 3.102 time: 0.3288 s/iter data_time: 0.2224 s/iter total_throughput: 3114.82 samples/s lr: 9.85e-05 [09/27 12:58:33] lb.utils.events INFO: eta: 3:12:22 iteration: 302899/375342 consumed_samples: 310169600 total_loss: 3.087 time: 0.3288 s/iter data_time: 0.2291 s/iter total_throughput: 3114.82 samples/s lr: 9.82e-05 [09/27 12:59:06] lb.utils.events INFO: eta: 3:14:47 iteration: 302999/375342 consumed_samples: 310272000 total_loss: 3.102 time: 0.3288 s/iter data_time: 0.2299 s/iter total_throughput: 3114.82 samples/s lr: 9.80e-05 [09/27 12:59:39] lb.utils.events INFO: eta: 3:14:57 iteration: 303099/375342 consumed_samples: 310374400 total_loss: 3.104 time: 0.3288 s/iter data_time: 0.2127 s/iter total_throughput: 3114.82 samples/s lr: 9.78e-05 [09/27 13:00:12] lb.utils.events INFO: eta: 3:14:13 iteration: 303199/375342 consumed_samples: 310476800 total_loss: 3.097 time: 0.3288 s/iter data_time: 0.2010 s/iter total_throughput: 3114.83 samples/s lr: 9.75e-05 [09/27 13:00:45] lb.utils.events INFO: eta: 3:14:08 iteration: 303299/375342 consumed_samples: 310579200 total_loss: 3.092 time: 0.3288 s/iter data_time: 0.2225 s/iter total_throughput: 3114.83 samples/s lr: 9.73e-05 [09/27 13:01:18] lb.utils.events INFO: eta: 3:16:27 iteration: 303399/375342 consumed_samples: 310681600 total_loss: 3.089 time: 0.3288 s/iter data_time: 0.2208 s/iter total_throughput: 3114.83 samples/s lr: 9.71e-05 [09/27 13:01:50] lb.utils.events INFO: eta: 3:15:04 iteration: 303499/375342 consumed_samples: 310784000 total_loss: 3.106 time: 0.3287 s/iter data_time: 0.2221 s/iter total_throughput: 3114.83 samples/s lr: 9.68e-05 [09/27 13:02:23] lb.utils.events INFO: eta: 3:15:01 iteration: 303599/375342 consumed_samples: 310886400 total_loss: 3.108 time: 0.3287 s/iter data_time: 0.2209 s/iter total_throughput: 3114.83 samples/s lr: 9.66e-05 [09/27 13:02:56] lb.utils.events INFO: eta: 3:12:15 iteration: 303699/375342 consumed_samples: 310988800 total_loss: 3.1 time: 0.3288 s/iter data_time: 0.2066 s/iter total_throughput: 3114.83 samples/s lr: 9.64e-05 [09/27 13:03:29] lb.utils.events INFO: eta: 3:09:18 iteration: 303799/375342 consumed_samples: 311091200 total_loss: 3.106 time: 0.3288 s/iter data_time: 0.2037 s/iter total_throughput: 3114.82 samples/s lr: 9.61e-05 [09/27 13:04:03] lb.utils.events INFO: eta: 3:07:06 iteration: 303899/375342 consumed_samples: 311193600 total_loss: 3.083 time: 0.3288 s/iter data_time: 0.2081 s/iter total_throughput: 3114.81 samples/s lr: 9.59e-05 [09/27 13:04:36] lb.utils.events INFO: eta: 3:05:59 iteration: 303999/375342 consumed_samples: 311296000 total_loss: 3.091 time: 0.3288 s/iter data_time: 0.2002 s/iter total_throughput: 3114.81 samples/s lr: 9.57e-05 [09/27 13:05:08] lb.utils.events INFO: eta: 3:04:59 iteration: 304099/375342 consumed_samples: 311398400 total_loss: 3.099 time: 0.3288 s/iter data_time: 0.2098 s/iter total_throughput: 3114.81 samples/s lr: 9.54e-05 [09/27 13:05:41] lb.utils.events INFO: eta: 3:04:52 iteration: 304199/375342 consumed_samples: 311500800 total_loss: 3.091 time: 0.3288 s/iter data_time: 0.2032 s/iter total_throughput: 3114.81 samples/s lr: 9.52e-05 [09/27 13:06:14] lb.utils.events INFO: eta: 3:04:34 iteration: 304299/375342 consumed_samples: 311603200 total_loss: 3.074 time: 0.3288 s/iter data_time: 0.2063 s/iter total_throughput: 3114.82 samples/s lr: 9.50e-05 [09/27 13:06:47] lb.utils.events INFO: eta: 3:02:57 iteration: 304399/375342 consumed_samples: 311705600 total_loss: 3.085 time: 0.3288 s/iter data_time: 0.2180 s/iter total_throughput: 3114.82 samples/s lr: 9.47e-05 [09/27 13:07:20] lb.utils.events INFO: eta: 3:02:44 iteration: 304499/375342 consumed_samples: 311808000 total_loss: 3.104 time: 0.3288 s/iter data_time: 0.2060 s/iter total_throughput: 3114.82 samples/s lr: 9.45e-05 [09/27 13:07:53] lb.utils.events INFO: eta: 3:01:37 iteration: 304599/375342 consumed_samples: 311910400 total_loss: 3.107 time: 0.3288 s/iter data_time: 0.2101 s/iter total_throughput: 3114.82 samples/s lr: 9.43e-05 [09/27 13:08:25] lb.utils.events INFO: eta: 3:01:22 iteration: 304699/375342 consumed_samples: 312012800 total_loss: 3.104 time: 0.3288 s/iter data_time: 0.2021 s/iter total_throughput: 3114.82 samples/s lr: 9.40e-05 [09/27 13:08:58] lb.utils.events INFO: eta: 3:01:34 iteration: 304799/375342 consumed_samples: 312115200 total_loss: 3.104 time: 0.3288 s/iter data_time: 0.2279 s/iter total_throughput: 3114.82 samples/s lr: 9.38e-05 [09/27 13:09:31] lb.utils.events INFO: eta: 3:01:23 iteration: 304899/375342 consumed_samples: 312217600 total_loss: 3.089 time: 0.3288 s/iter data_time: 0.2066 s/iter total_throughput: 3114.82 samples/s lr: 9.36e-05 [09/27 13:10:04] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0304999 [09/27 13:10:04] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/27 13:10:04] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/27 13:10:08] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0856 s/iter. Inference: 0.1517 s/iter. Eval: 0.0021 s/iter. Total: 0.2394 s/iter. ETA=0:00:08 [09/27 13:10:14] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1478 s/iter. Inference: 0.1502 s/iter. Eval: 0.0024 s/iter. Total: 0.3005 s/iter. ETA=0:00:05 [09/27 13:10:20] lb.evaluation.evaluator INFO: Inference done 50000/50000. Dataloading: 0.1314 s/iter. Inference: 0.1536 s/iter. Eval: 0.0022 s/iter. Total: 0.2873 s/iter. ETA=0:00:00 [09/27 13:10:20] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/27 13:10:20] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.641315 (0.000253 s / iter per device, on 8 devices) [09/27 13:10:20] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000135 s / iter per device, on 8 devices) [09/27 13:10:20] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/27 13:10:20] lb.evaluation.utils INFO: copypaste: Acc@1=79.288 [09/27 13:10:20] lb.evaluation.utils INFO: copypaste: Acc@5=94.41199999999999 [09/27 13:10:20] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 79.28800, better than last best score 79.23600 @ iteration 299999. [09/27 13:10:20] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/27 13:10:20] lb.utils.events INFO: eta: 3:01:04 iteration: 304999/375342 consumed_samples: 312320000 total_loss: 3.086 time: 0.3287 s/iter data_time: 0.2019 s/iter total_throughput: 3114.83 samples/s lr: 9.33e-05 [09/27 13:10:51] lb.utils.events INFO: eta: 3:02:14 iteration: 305099/375342 consumed_samples: 312422400 total_loss: 3.087 time: 0.3287 s/iter data_time: 0.2248 s/iter total_throughput: 3114.89 samples/s lr: 9.31e-05 [09/27 13:11:24] lb.utils.events INFO: eta: 3:02:49 iteration: 305199/375342 consumed_samples: 312524800 total_loss: 3.069 time: 0.3287 s/iter data_time: 0.2284 s/iter total_throughput: 3114.89 samples/s lr: 9.29e-05 [09/27 13:11:57] lb.utils.events INFO: eta: 3:04:01 iteration: 305299/375342 consumed_samples: 312627200 total_loss: 3.099 time: 0.3287 s/iter data_time: 0.2236 s/iter total_throughput: 3114.89 samples/s lr: 9.27e-05 [09/27 13:12:30] lb.utils.events INFO: eta: 3:06:27 iteration: 305399/375342 consumed_samples: 312729600 total_loss: 3.094 time: 0.3287 s/iter data_time: 0.2417 s/iter total_throughput: 3114.91 samples/s lr: 9.24e-05 [09/27 13:13:02] lb.utils.events INFO: eta: 3:08:34 iteration: 305499/375342 consumed_samples: 312832000 total_loss: 3.087 time: 0.3287 s/iter data_time: 0.2340 s/iter total_throughput: 3114.92 samples/s lr: 9.22e-05 [09/27 13:13:36] lb.utils.events INFO: eta: 3:11:21 iteration: 305599/375342 consumed_samples: 312934400 total_loss: 3.085 time: 0.3287 s/iter data_time: 0.2312 s/iter total_throughput: 3114.90 samples/s lr: 9.20e-05 [09/27 13:14:08] lb.utils.events INFO: eta: 3:14:33 iteration: 305699/375342 consumed_samples: 313036800 total_loss: 3.074 time: 0.3287 s/iter data_time: 0.2271 s/iter total_throughput: 3114.90 samples/s lr: 9.17e-05 [09/27 13:14:41] lb.utils.events INFO: eta: 3:15:15 iteration: 305799/375342 consumed_samples: 313139200 total_loss: 3.089 time: 0.3287 s/iter data_time: 0.2201 s/iter total_throughput: 3114.91 samples/s lr: 9.15e-05 [09/27 13:15:14] lb.utils.events INFO: eta: 3:41:19 iteration: 305899/375342 consumed_samples: 313241600 total_loss: 3.114 time: 0.3287 s/iter data_time: 0.2472 s/iter total_throughput: 3114.92 samples/s lr: 9.13e-05 [09/27 13:15:46] lb.utils.events INFO: eta: 4:47:11 iteration: 305999/375342 consumed_samples: 313344000 total_loss: 3.112 time: 0.3287 s/iter data_time: 0.2285 s/iter total_throughput: 3114.93 samples/s lr: 9.11e-05 [09/27 13:16:19] lb.utils.events INFO: eta: 4:48:29 iteration: 306099/375342 consumed_samples: 313446400 total_loss: 3.108 time: 0.3287 s/iter data_time: 0.2223 s/iter total_throughput: 3114.92 samples/s lr: 9.08e-05 [09/27 13:16:52] lb.utils.events INFO: eta: 4:49:01 iteration: 306199/375342 consumed_samples: 313548800 total_loss: 3.112 time: 0.3287 s/iter data_time: 0.2175 s/iter total_throughput: 3114.93 samples/s lr: 9.06e-05 [09/27 13:17:24] lb.utils.events INFO: eta: 4:41:38 iteration: 306299/375342 consumed_samples: 313651200 total_loss: 3.107 time: 0.3287 s/iter data_time: 0.2490 s/iter total_throughput: 3114.94 samples/s lr: 9.04e-05 [09/27 13:17:57] lb.utils.events INFO: eta: 4:47:14 iteration: 306399/375342 consumed_samples: 313753600 total_loss: 3.083 time: 0.3287 s/iter data_time: 0.2266 s/iter total_throughput: 3114.95 samples/s lr: 9.02e-05 [09/27 13:18:30] lb.utils.events INFO: eta: 5:01:40 iteration: 306499/375342 consumed_samples: 313856000 total_loss: 3.084 time: 0.3287 s/iter data_time: 0.2633 s/iter total_throughput: 3114.96 samples/s lr: 8.99e-05 [09/27 13:19:02] lb.utils.events INFO: eta: 5:20:46 iteration: 306599/375342 consumed_samples: 313958400 total_loss: 3.092 time: 0.3287 s/iter data_time: 0.2433 s/iter total_throughput: 3114.97 samples/s lr: 8.97e-05 [09/27 13:19:35] lb.utils.events INFO: eta: 5:12:54 iteration: 306699/375342 consumed_samples: 314060800 total_loss: 3.071 time: 0.3287 s/iter data_time: 0.2372 s/iter total_throughput: 3114.97 samples/s lr: 8.95e-05 [09/27 13:20:08] lb.utils.events INFO: eta: 5:24:01 iteration: 306799/375342 consumed_samples: 314163200 total_loss: 3.083 time: 0.3287 s/iter data_time: 0.2286 s/iter total_throughput: 3114.97 samples/s lr: 8.93e-05 [09/27 13:20:41] lb.utils.events INFO: eta: 5:16:31 iteration: 306899/375342 consumed_samples: 314265600 total_loss: 3.062 time: 0.3287 s/iter data_time: 0.2365 s/iter total_throughput: 3114.97 samples/s lr: 8.90e-05 [09/27 13:21:13] lb.utils.events INFO: eta: 5:12:28 iteration: 306999/375342 consumed_samples: 314368000 total_loss: 3.062 time: 0.3287 s/iter data_time: 0.2408 s/iter total_throughput: 3114.99 samples/s lr: 8.88e-05 [09/27 13:21:46] lb.utils.events INFO: eta: 5:18:26 iteration: 307099/375342 consumed_samples: 314470400 total_loss: 3.071 time: 0.3287 s/iter data_time: 0.2240 s/iter total_throughput: 3114.99 samples/s lr: 8.86e-05 [09/27 13:22:18] lb.utils.events INFO: eta: 4:44:49 iteration: 307199/375342 consumed_samples: 314572800 total_loss: 3.085 time: 0.3287 s/iter data_time: 0.1995 s/iter total_throughput: 3115.00 samples/s lr: 8.84e-05 [09/27 13:22:51] lb.utils.events INFO: eta: 4:56:31 iteration: 307299/375342 consumed_samples: 314675200 total_loss: 3.093 time: 0.3287 s/iter data_time: 0.2219 s/iter total_throughput: 3115.02 samples/s lr: 8.81e-05 [09/27 13:23:23] lb.utils.events INFO: eta: 4:39:04 iteration: 307399/375342 consumed_samples: 314777600 total_loss: 3.083 time: 0.3287 s/iter data_time: 0.2194 s/iter total_throughput: 3115.04 samples/s lr: 8.79e-05 [09/27 13:23:56] lb.utils.events INFO: eta: 4:26:56 iteration: 307499/375342 consumed_samples: 314880000 total_loss: 3.087 time: 0.3287 s/iter data_time: 0.2179 s/iter total_throughput: 3115.05 samples/s lr: 8.77e-05 [09/27 13:24:28] lb.utils.events INFO: eta: 3:39:30 iteration: 307599/375342 consumed_samples: 314982400 total_loss: 3.087 time: 0.3287 s/iter data_time: 0.2215 s/iter total_throughput: 3115.05 samples/s lr: 8.75e-05 [09/27 13:25:01] lb.utils.events INFO: eta: 3:15:55 iteration: 307699/375342 consumed_samples: 315084800 total_loss: 3.073 time: 0.3287 s/iter data_time: 0.2093 s/iter total_throughput: 3115.06 samples/s lr: 8.72e-05 [09/27 13:25:33] lb.utils.events INFO: eta: 3:10:29 iteration: 307799/375342 consumed_samples: 315187200 total_loss: 3.085 time: 0.3287 s/iter data_time: 0.2365 s/iter total_throughput: 3115.08 samples/s lr: 8.70e-05 [09/27 13:26:06] lb.utils.events INFO: eta: 3:07:06 iteration: 307899/375342 consumed_samples: 315289600 total_loss: 3.094 time: 0.3287 s/iter data_time: 0.2116 s/iter total_throughput: 3115.09 samples/s lr: 8.68e-05 [09/27 13:26:38] lb.utils.events INFO: eta: 3:06:31 iteration: 307999/375342 consumed_samples: 315392000 total_loss: 3.085 time: 0.3287 s/iter data_time: 0.2347 s/iter total_throughput: 3115.10 samples/s lr: 8.66e-05 [09/27 13:27:11] lb.utils.events INFO: eta: 3:04:46 iteration: 308099/375342 consumed_samples: 315494400 total_loss: 3.085 time: 0.3287 s/iter data_time: 0.2103 s/iter total_throughput: 3115.09 samples/s lr: 8.64e-05 [09/27 13:27:44] lb.utils.events INFO: eta: 3:04:19 iteration: 308199/375342 consumed_samples: 315596800 total_loss: 3.098 time: 0.3287 s/iter data_time: 0.2029 s/iter total_throughput: 3115.09 samples/s lr: 8.61e-05 [09/27 13:28:17] lb.utils.events INFO: eta: 3:01:03 iteration: 308299/375342 consumed_samples: 315699200 total_loss: 3.104 time: 0.3287 s/iter data_time: 0.2057 s/iter total_throughput: 3115.10 samples/s lr: 8.59e-05 [09/27 13:28:50] lb.utils.events INFO: eta: 2:59:19 iteration: 308399/375342 consumed_samples: 315801600 total_loss: 3.1 time: 0.3287 s/iter data_time: 0.2248 s/iter total_throughput: 3115.10 samples/s lr: 8.57e-05 [09/27 13:29:24] lb.utils.events INFO: eta: 2:58:14 iteration: 308499/375342 consumed_samples: 315904000 total_loss: 3.065 time: 0.3287 s/iter data_time: 0.2127 s/iter total_throughput: 3115.07 samples/s lr: 8.55e-05 [09/27 13:29:57] lb.utils.events INFO: eta: 2:55:50 iteration: 308599/375342 consumed_samples: 316006400 total_loss: 3.077 time: 0.3287 s/iter data_time: 0.2390 s/iter total_throughput: 3115.05 samples/s lr: 8.53e-05 [09/27 13:30:31] lb.utils.events INFO: eta: 2:54:33 iteration: 308699/375342 consumed_samples: 316108800 total_loss: 3.094 time: 0.3287 s/iter data_time: 0.2067 s/iter total_throughput: 3115.03 samples/s lr: 8.50e-05 [09/27 13:31:03] lb.utils.events INFO: eta: 2:53:23 iteration: 308799/375342 consumed_samples: 316211200 total_loss: 3.076 time: 0.3287 s/iter data_time: 0.2289 s/iter total_throughput: 3115.03 samples/s lr: 8.48e-05 [09/27 13:31:37] lb.utils.events INFO: eta: 2:53:27 iteration: 308899/375342 consumed_samples: 316313600 total_loss: 3.082 time: 0.3287 s/iter data_time: 0.2432 s/iter total_throughput: 3115.03 samples/s lr: 8.46e-05 [09/27 13:32:10] lb.utils.events INFO: eta: 2:53:15 iteration: 308999/375342 consumed_samples: 316416000 total_loss: 3.104 time: 0.3287 s/iter data_time: 0.2418 s/iter total_throughput: 3115.02 samples/s lr: 8.44e-05 [09/27 13:32:43] lb.utils.events INFO: eta: 2:53:54 iteration: 309099/375342 consumed_samples: 316518400 total_loss: 3.089 time: 0.3287 s/iter data_time: 0.2374 s/iter total_throughput: 3115.02 samples/s lr: 8.42e-05 [09/27 13:33:16] lb.utils.events INFO: eta: 2:54:43 iteration: 309199/375342 consumed_samples: 316620800 total_loss: 3.081 time: 0.3287 s/iter data_time: 0.2329 s/iter total_throughput: 3115.01 samples/s lr: 8.39e-05 [09/27 13:33:49] lb.utils.events INFO: eta: 2:56:35 iteration: 309299/375342 consumed_samples: 316723200 total_loss: 3.083 time: 0.3287 s/iter data_time: 0.2205 s/iter total_throughput: 3115.00 samples/s lr: 8.37e-05 [09/27 13:34:22] lb.utils.events INFO: eta: 2:58:02 iteration: 309399/375342 consumed_samples: 316825600 total_loss: 3.094 time: 0.3287 s/iter data_time: 0.2274 s/iter total_throughput: 3114.98 samples/s lr: 8.35e-05 [09/27 13:34:56] lb.utils.events INFO: eta: 2:59:04 iteration: 309499/375342 consumed_samples: 316928000 total_loss: 3.069 time: 0.3287 s/iter data_time: 0.2220 s/iter total_throughput: 3114.97 samples/s lr: 8.33e-05 [09/27 13:35:29] lb.utils.events INFO: eta: 3:00:01 iteration: 309599/375342 consumed_samples: 317030400 total_loss: 3.095 time: 0.3287 s/iter data_time: 0.2212 s/iter total_throughput: 3114.96 samples/s lr: 8.31e-05 [09/27 13:36:02] lb.utils.events INFO: eta: 3:05:23 iteration: 309699/375342 consumed_samples: 317132800 total_loss: 3.113 time: 0.3287 s/iter data_time: 0.2254 s/iter total_throughput: 3114.96 samples/s lr: 8.29e-05 [09/27 13:36:35] lb.utils.events INFO: eta: 3:08:00 iteration: 309799/375342 consumed_samples: 317235200 total_loss: 3.103 time: 0.3287 s/iter data_time: 0.2246 s/iter total_throughput: 3114.96 samples/s lr: 8.26e-05 [09/27 13:37:08] lb.utils.events INFO: eta: 3:01:30 iteration: 309899/375342 consumed_samples: 317337600 total_loss: 3.087 time: 0.3287 s/iter data_time: 0.2136 s/iter total_throughput: 3114.94 samples/s lr: 8.24e-05 [09/27 13:37:42] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0309999 [09/27 13:37:42] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/27 13:37:42] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/27 13:37:46] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0827 s/iter. Inference: 0.1503 s/iter. Eval: 0.0022 s/iter. Total: 0.2352 s/iter. ETA=0:00:08 [09/27 13:37:52] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1455 s/iter. Inference: 0.1499 s/iter. Eval: 0.0020 s/iter. Total: 0.2974 s/iter. ETA=0:00:05 [09/27 13:37:57] lb.evaluation.evaluator INFO: Inference done 49152/50000. Dataloading: 0.1325 s/iter. Inference: 0.1485 s/iter. Eval: 0.0021 s/iter. Total: 0.2832 s/iter. ETA=0:00:00 [09/27 13:37:58] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/27 13:37:58] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.621545 (0.000252 s / iter per device, on 8 devices) [09/27 13:37:58] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000132 s / iter per device, on 8 devices) [09/27 13:37:58] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/27 13:37:58] lb.evaluation.utils INFO: copypaste: Acc@1=79.324 [09/27 13:37:58] lb.evaluation.utils INFO: copypaste: Acc@5=94.374 [09/27 13:37:58] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 79.32400, better than last best score 79.28800 @ iteration 304999. [09/27 13:37:58] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/27 13:37:58] lb.utils.events INFO: eta: 2:58:52 iteration: 309999/375342 consumed_samples: 317440000 total_loss: 3.082 time: 0.3287 s/iter data_time: 0.2269 s/iter total_throughput: 3114.92 samples/s lr: 8.22e-05 [09/27 13:38:30] lb.utils.events INFO: eta: 2:58:08 iteration: 310099/375342 consumed_samples: 317542400 total_loss: 3.081 time: 0.3287 s/iter data_time: 0.2490 s/iter total_throughput: 3114.97 samples/s lr: 8.20e-05 [09/27 13:39:04] lb.utils.events INFO: eta: 2:58:23 iteration: 310199/375342 consumed_samples: 317644800 total_loss: 3.075 time: 0.3287 s/iter data_time: 0.2342 s/iter total_throughput: 3114.94 samples/s lr: 8.18e-05 [09/27 13:39:37] lb.utils.events INFO: eta: 2:58:03 iteration: 310299/375342 consumed_samples: 317747200 total_loss: 3.069 time: 0.3287 s/iter data_time: 0.2197 s/iter total_throughput: 3114.94 samples/s lr: 8.16e-05 [09/27 13:40:10] lb.utils.events INFO: eta: 2:54:57 iteration: 310399/375342 consumed_samples: 317849600 total_loss: 3.071 time: 0.3287 s/iter data_time: 0.2089 s/iter total_throughput: 3114.92 samples/s lr: 8.13e-05 [09/27 13:40:43] lb.utils.events INFO: eta: 2:53:36 iteration: 310499/375342 consumed_samples: 317952000 total_loss: 3.08 time: 0.3287 s/iter data_time: 0.2002 s/iter total_throughput: 3114.93 samples/s lr: 8.11e-05 [09/27 13:41:16] lb.utils.events INFO: eta: 2:51:30 iteration: 310599/375342 consumed_samples: 318054400 total_loss: 3.091 time: 0.3287 s/iter data_time: 0.2037 s/iter total_throughput: 3114.92 samples/s lr: 8.09e-05 [09/27 13:41:49] lb.utils.events INFO: eta: 2:49:33 iteration: 310699/375342 consumed_samples: 318156800 total_loss: 3.09 time: 0.3287 s/iter data_time: 0.2083 s/iter total_throughput: 3114.92 samples/s lr: 8.07e-05 [09/27 13:42:21] lb.utils.events INFO: eta: 2:48:06 iteration: 310799/375342 consumed_samples: 318259200 total_loss: 3.067 time: 0.3287 s/iter data_time: 0.2249 s/iter total_throughput: 3114.93 samples/s lr: 8.05e-05 [09/27 13:42:55] lb.utils.events INFO: eta: 2:47:21 iteration: 310899/375342 consumed_samples: 318361600 total_loss: 3.067 time: 0.3287 s/iter data_time: 0.2117 s/iter total_throughput: 3114.91 samples/s lr: 8.03e-05 [09/27 13:43:28] lb.utils.events INFO: eta: 2:46:26 iteration: 310999/375342 consumed_samples: 318464000 total_loss: 3.055 time: 0.3287 s/iter data_time: 0.2181 s/iter total_throughput: 3114.90 samples/s lr: 8.01e-05 [09/27 13:44:00] lb.utils.events INFO: eta: 2:44:51 iteration: 311099/375342 consumed_samples: 318566400 total_loss: 3.065 time: 0.3287 s/iter data_time: 0.2066 s/iter total_throughput: 3114.91 samples/s lr: 7.99e-05 [09/27 13:44:33] lb.utils.events INFO: eta: 2:44:44 iteration: 311199/375342 consumed_samples: 318668800 total_loss: 3.087 time: 0.3287 s/iter data_time: 0.2188 s/iter total_throughput: 3114.92 samples/s lr: 7.96e-05 [09/27 13:45:06] lb.utils.events INFO: eta: 2:44:25 iteration: 311299/375342 consumed_samples: 318771200 total_loss: 3.083 time: 0.3287 s/iter data_time: 0.2169 s/iter total_throughput: 3114.91 samples/s lr: 7.94e-05 [09/27 13:45:39] lb.utils.events INFO: eta: 2:45:16 iteration: 311399/375342 consumed_samples: 318873600 total_loss: 3.082 time: 0.3287 s/iter data_time: 0.2420 s/iter total_throughput: 3114.91 samples/s lr: 7.92e-05 [09/27 13:46:12] lb.utils.events INFO: eta: 2:45:48 iteration: 311499/375342 consumed_samples: 318976000 total_loss: 3.095 time: 0.3287 s/iter data_time: 0.2258 s/iter total_throughput: 3114.92 samples/s lr: 7.90e-05 [09/27 13:46:45] lb.utils.events INFO: eta: 2:46:20 iteration: 311599/375342 consumed_samples: 319078400 total_loss: 3.094 time: 0.3287 s/iter data_time: 0.2349 s/iter total_throughput: 3114.91 samples/s lr: 7.88e-05 [09/27 13:47:19] lb.utils.events INFO: eta: 2:47:52 iteration: 311699/375342 consumed_samples: 319180800 total_loss: 3.052 time: 0.3287 s/iter data_time: 0.2547 s/iter total_throughput: 3114.89 samples/s lr: 7.86e-05 [09/27 13:47:52] lb.utils.events INFO: eta: 2:49:08 iteration: 311799/375342 consumed_samples: 319283200 total_loss: 3.049 time: 0.3287 s/iter data_time: 0.2259 s/iter total_throughput: 3114.88 samples/s lr: 7.84e-05 [09/27 13:48:25] lb.utils.events INFO: eta: 2:51:27 iteration: 311899/375342 consumed_samples: 319385600 total_loss: 3.062 time: 0.3287 s/iter data_time: 0.2267 s/iter total_throughput: 3114.86 samples/s lr: 7.82e-05 [09/27 13:48:58] lb.utils.events INFO: eta: 2:54:43 iteration: 311999/375342 consumed_samples: 319488000 total_loss: 3.058 time: 0.3287 s/iter data_time: 0.2202 s/iter total_throughput: 3114.86 samples/s lr: 7.80e-05 [09/27 13:49:31] lb.utils.events INFO: eta: 3:01:24 iteration: 312099/375342 consumed_samples: 319590400 total_loss: 3.056 time: 0.3287 s/iter data_time: 0.2351 s/iter total_throughput: 3114.87 samples/s lr: 7.77e-05 [09/27 13:50:04] lb.utils.events INFO: eta: 3:04:42 iteration: 312199/375342 consumed_samples: 319692800 total_loss: 3.058 time: 0.3287 s/iter data_time: 0.2152 s/iter total_throughput: 3114.86 samples/s lr: 7.75e-05 [09/27 13:50:37] lb.utils.events INFO: eta: 3:13:39 iteration: 312299/375342 consumed_samples: 319795200 total_loss: 3.064 time: 0.3287 s/iter data_time: 0.2328 s/iter total_throughput: 3114.85 samples/s lr: 7.73e-05 [09/27 13:51:10] lb.utils.events INFO: eta: 2:59:47 iteration: 312399/375342 consumed_samples: 319897600 total_loss: 3.071 time: 0.3287 s/iter data_time: 0.2202 s/iter total_throughput: 3114.84 samples/s lr: 7.71e-05 [09/27 13:51:43] lb.utils.events INFO: eta: 2:53:55 iteration: 312499/375342 consumed_samples: 320000000 total_loss: 3.068 time: 0.3287 s/iter data_time: 0.1980 s/iter total_throughput: 3114.84 samples/s lr: 7.69e-05 [09/27 13:52:16] lb.utils.events INFO: eta: 2:51:10 iteration: 312599/375342 consumed_samples: 320102400 total_loss: 3.06 time: 0.3287 s/iter data_time: 0.2014 s/iter total_throughput: 3114.84 samples/s lr: 7.67e-05 [09/27 13:52:49] lb.utils.events INFO: eta: 2:46:41 iteration: 312699/375342 consumed_samples: 320204800 total_loss: 3.048 time: 0.3287 s/iter data_time: 0.2092 s/iter total_throughput: 3114.84 samples/s lr: 7.65e-05 [09/27 13:53:22] lb.utils.events INFO: eta: 2:45:08 iteration: 312799/375342 consumed_samples: 320307200 total_loss: 3.055 time: 0.3287 s/iter data_time: 0.2235 s/iter total_throughput: 3114.85 samples/s lr: 7.63e-05 [09/27 13:53:55] lb.utils.events INFO: eta: 2:44:39 iteration: 312899/375342 consumed_samples: 320409600 total_loss: 3.05 time: 0.3287 s/iter data_time: 0.2272 s/iter total_throughput: 3114.84 samples/s lr: 7.61e-05 [09/27 13:54:27] lb.utils.events INFO: eta: 2:43:20 iteration: 312999/375342 consumed_samples: 320512000 total_loss: 3.042 time: 0.3287 s/iter data_time: 0.2294 s/iter total_throughput: 3114.85 samples/s lr: 7.59e-05 [09/27 13:55:00] lb.utils.events INFO: eta: 2:42:59 iteration: 313099/375342 consumed_samples: 320614400 total_loss: 3.071 time: 0.3287 s/iter data_time: 0.2052 s/iter total_throughput: 3114.86 samples/s lr: 7.57e-05 [09/27 13:55:33] lb.utils.events INFO: eta: 2:42:44 iteration: 313199/375342 consumed_samples: 320716800 total_loss: 3.078 time: 0.3287 s/iter data_time: 0.2256 s/iter total_throughput: 3114.85 samples/s lr: 7.55e-05 [09/27 13:56:06] lb.utils.events INFO: eta: 2:41:58 iteration: 313299/375342 consumed_samples: 320819200 total_loss: 3.058 time: 0.3287 s/iter data_time: 0.2170 s/iter total_throughput: 3114.84 samples/s lr: 7.53e-05 [09/27 13:56:39] lb.utils.events INFO: eta: 2:41:30 iteration: 313399/375342 consumed_samples: 320921600 total_loss: 3.056 time: 0.3287 s/iter data_time: 0.2309 s/iter total_throughput: 3114.83 samples/s lr: 7.51e-05 [09/27 13:57:13] lb.utils.events INFO: eta: 2:42:49 iteration: 313499/375342 consumed_samples: 321024000 total_loss: 3.078 time: 0.3288 s/iter data_time: 0.2310 s/iter total_throughput: 3114.83 samples/s lr: 7.48e-05 [09/27 13:57:46] lb.utils.events INFO: eta: 2:43:13 iteration: 313599/375342 consumed_samples: 321126400 total_loss: 3.077 time: 0.3288 s/iter data_time: 0.2320 s/iter total_throughput: 3114.82 samples/s lr: 7.46e-05 [09/27 13:58:19] lb.utils.events INFO: eta: 2:44:43 iteration: 313699/375342 consumed_samples: 321228800 total_loss: 3.064 time: 0.3288 s/iter data_time: 0.2276 s/iter total_throughput: 3114.81 samples/s lr: 7.44e-05 [09/27 13:58:52] lb.utils.events INFO: eta: 2:44:37 iteration: 313799/375342 consumed_samples: 321331200 total_loss: 3.046 time: 0.3288 s/iter data_time: 0.2235 s/iter total_throughput: 3114.80 samples/s lr: 7.42e-05 [09/27 13:59:25] lb.utils.events INFO: eta: 2:44:28 iteration: 313899/375342 consumed_samples: 321433600 total_loss: 3.062 time: 0.3288 s/iter data_time: 0.2258 s/iter total_throughput: 3114.79 samples/s lr: 7.40e-05 [09/27 13:59:58] lb.utils.events INFO: eta: 2:44:12 iteration: 313999/375342 consumed_samples: 321536000 total_loss: 3.066 time: 0.3288 s/iter data_time: 0.2216 s/iter total_throughput: 3114.79 samples/s lr: 7.38e-05 [09/27 14:00:31] lb.utils.events INFO: eta: 2:43:51 iteration: 314099/375342 consumed_samples: 321638400 total_loss: 3.051 time: 0.3288 s/iter data_time: 0.2232 s/iter total_throughput: 3114.78 samples/s lr: 7.36e-05 [09/27 14:01:04] lb.utils.events INFO: eta: 2:46:01 iteration: 314199/375342 consumed_samples: 321740800 total_loss: 3.053 time: 0.3288 s/iter data_time: 0.2433 s/iter total_throughput: 3114.78 samples/s lr: 7.34e-05 [09/27 14:01:38] lb.utils.events INFO: eta: 2:49:26 iteration: 314299/375342 consumed_samples: 321843200 total_loss: 3.071 time: 0.3288 s/iter data_time: 0.2387 s/iter total_throughput: 3114.77 samples/s lr: 7.32e-05 [09/27 14:02:11] lb.utils.events INFO: eta: 2:49:41 iteration: 314399/375342 consumed_samples: 321945600 total_loss: 3.05 time: 0.3288 s/iter data_time: 0.2012 s/iter total_throughput: 3114.76 samples/s lr: 7.30e-05 [09/27 14:02:44] lb.utils.events INFO: eta: 2:47:06 iteration: 314499/375342 consumed_samples: 322048000 total_loss: 3.052 time: 0.3288 s/iter data_time: 0.2085 s/iter total_throughput: 3114.76 samples/s lr: 7.28e-05 [09/27 14:03:17] lb.utils.events INFO: eta: 2:46:49 iteration: 314599/375342 consumed_samples: 322150400 total_loss: 3.069 time: 0.3288 s/iter data_time: 0.2234 s/iter total_throughput: 3114.76 samples/s lr: 7.26e-05 [09/27 14:03:50] lb.utils.events INFO: eta: 2:45:56 iteration: 314699/375342 consumed_samples: 322252800 total_loss: 3.057 time: 0.3288 s/iter data_time: 0.2165 s/iter total_throughput: 3114.75 samples/s lr: 7.24e-05 [09/27 14:04:23] lb.utils.events INFO: eta: 2:43:33 iteration: 314799/375342 consumed_samples: 322355200 total_loss: 3.048 time: 0.3288 s/iter data_time: 0.2106 s/iter total_throughput: 3114.73 samples/s lr: 7.22e-05 [09/27 14:04:56] lb.utils.events INFO: eta: 2:41:08 iteration: 314899/375342 consumed_samples: 322457600 total_loss: 3.04 time: 0.3288 s/iter data_time: 0.2199 s/iter total_throughput: 3114.72 samples/s lr: 7.20e-05 [09/27 14:05:29] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0314999 [09/27 14:05:30] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/27 14:05:30] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/27 14:05:34] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0805 s/iter. Inference: 0.1505 s/iter. Eval: 0.0020 s/iter. Total: 0.2329 s/iter. ETA=0:00:08 [09/27 14:05:40] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1422 s/iter. Inference: 0.1495 s/iter. Eval: 0.0020 s/iter. Total: 0.2938 s/iter. ETA=0:00:05 [09/27 14:05:45] lb.evaluation.evaluator INFO: Inference done 50000/50000. Dataloading: 0.1296 s/iter. Inference: 0.1503 s/iter. Eval: 0.0020 s/iter. Total: 0.2820 s/iter. ETA=0:00:00 [09/27 14:05:45] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/27 14:05:45] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.413048 (0.000248 s / iter per device, on 8 devices) [09/27 14:05:45] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000132 s / iter per device, on 8 devices) [09/27 14:05:45] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/27 14:05:45] lb.evaluation.utils INFO: copypaste: Acc@1=79.44 [09/27 14:05:45] lb.evaluation.utils INFO: copypaste: Acc@5=94.308 [09/27 14:05:45] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 79.44000, better than last best score 79.32400 @ iteration 309999. [09/27 14:05:45] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/27 14:05:46] lb.utils.events INFO: eta: 2:40:11 iteration: 314999/375342 consumed_samples: 322560000 total_loss: 3.043 time: 0.3288 s/iter data_time: 0.2183 s/iter total_throughput: 3114.72 samples/s lr: 7.18e-05 [09/27 14:06:17] lb.utils.events INFO: eta: 2:38:48 iteration: 315099/375342 consumed_samples: 322662400 total_loss: 3.047 time: 0.3288 s/iter data_time: 0.2071 s/iter total_throughput: 3114.77 samples/s lr: 7.16e-05 [09/27 14:06:50] lb.utils.events INFO: eta: 2:38:24 iteration: 315199/375342 consumed_samples: 322764800 total_loss: 3.066 time: 0.3288 s/iter data_time: 0.2472 s/iter total_throughput: 3114.77 samples/s lr: 7.14e-05 [09/27 14:07:23] lb.utils.events INFO: eta: 2:38:13 iteration: 315299/375342 consumed_samples: 322867200 total_loss: 3.072 time: 0.3288 s/iter data_time: 0.2406 s/iter total_throughput: 3114.76 samples/s lr: 7.12e-05 [09/27 14:07:56] lb.utils.events INFO: eta: 2:38:15 iteration: 315399/375342 consumed_samples: 322969600 total_loss: 3.038 time: 0.3288 s/iter data_time: 0.2383 s/iter total_throughput: 3114.76 samples/s lr: 7.10e-05 [09/27 14:08:29] lb.utils.events INFO: eta: 2:39:16 iteration: 315499/375342 consumed_samples: 323072000 total_loss: 3.039 time: 0.3288 s/iter data_time: 0.2173 s/iter total_throughput: 3114.77 samples/s lr: 7.08e-05 [09/27 14:09:02] lb.utils.events INFO: eta: 2:39:20 iteration: 315599/375342 consumed_samples: 323174400 total_loss: 3.027 time: 0.3288 s/iter data_time: 0.2378 s/iter total_throughput: 3114.76 samples/s lr: 7.06e-05 [09/27 14:09:35] lb.utils.events INFO: eta: 2:39:04 iteration: 315699/375342 consumed_samples: 323276800 total_loss: 3.01 time: 0.3288 s/iter data_time: 0.2216 s/iter total_throughput: 3114.76 samples/s lr: 7.04e-05 [09/27 14:10:08] lb.utils.events INFO: eta: 2:38:52 iteration: 315799/375342 consumed_samples: 323379200 total_loss: 3.054 time: 0.3288 s/iter data_time: 0.2032 s/iter total_throughput: 3114.76 samples/s lr: 7.02e-05 [09/27 14:10:41] lb.utils.events INFO: eta: 2:39:05 iteration: 315899/375342 consumed_samples: 323481600 total_loss: 3.065 time: 0.3288 s/iter data_time: 0.2034 s/iter total_throughput: 3114.75 samples/s lr: 7.00e-05 [09/27 14:11:13] lb.utils.events INFO: eta: 2:39:15 iteration: 315999/375342 consumed_samples: 323584000 total_loss: 3.064 time: 0.3288 s/iter data_time: 0.2180 s/iter total_throughput: 3114.76 samples/s lr: 6.98e-05 [09/27 14:11:47] lb.utils.events INFO: eta: 2:40:13 iteration: 316099/375342 consumed_samples: 323686400 total_loss: 3.048 time: 0.3288 s/iter data_time: 0.2292 s/iter total_throughput: 3114.75 samples/s lr: 6.96e-05 [09/27 14:12:20] lb.utils.events INFO: eta: 2:38:50 iteration: 316199/375342 consumed_samples: 323788800 total_loss: 3.044 time: 0.3288 s/iter data_time: 0.2059 s/iter total_throughput: 3114.75 samples/s lr: 6.94e-05 [09/27 14:12:52] lb.utils.events INFO: eta: 2:37:37 iteration: 316299/375342 consumed_samples: 323891200 total_loss: 3.036 time: 0.3288 s/iter data_time: 0.2167 s/iter total_throughput: 3114.76 samples/s lr: 6.92e-05 [09/27 14:13:25] lb.utils.events INFO: eta: 2:37:21 iteration: 316399/375342 consumed_samples: 323993600 total_loss: 3.029 time: 0.3288 s/iter data_time: 0.2288 s/iter total_throughput: 3114.76 samples/s lr: 6.90e-05 [09/27 14:13:58] lb.utils.events INFO: eta: 2:37:42 iteration: 316499/375342 consumed_samples: 324096000 total_loss: 3.042 time: 0.3288 s/iter data_time: 0.2429 s/iter total_throughput: 3114.77 samples/s lr: 6.88e-05 [09/27 14:14:31] lb.utils.events INFO: eta: 2:37:30 iteration: 316599/375342 consumed_samples: 324198400 total_loss: 3.05 time: 0.3288 s/iter data_time: 0.2373 s/iter total_throughput: 3114.76 samples/s lr: 6.86e-05 [09/27 14:15:04] lb.utils.events INFO: eta: 2:36:19 iteration: 316699/375342 consumed_samples: 324300800 total_loss: 3.036 time: 0.3288 s/iter data_time: 0.2136 s/iter total_throughput: 3114.76 samples/s lr: 6.84e-05 [09/27 14:15:37] lb.utils.events INFO: eta: 2:35:57 iteration: 316799/375342 consumed_samples: 324403200 total_loss: 3.035 time: 0.3288 s/iter data_time: 0.2177 s/iter total_throughput: 3114.76 samples/s lr: 6.82e-05 [09/27 14:16:10] lb.utils.events INFO: eta: 2:35:53 iteration: 316899/375342 consumed_samples: 324505600 total_loss: 3.051 time: 0.3288 s/iter data_time: 0.2016 s/iter total_throughput: 3114.76 samples/s lr: 6.81e-05 [09/27 14:16:43] lb.utils.events INFO: eta: 2:36:11 iteration: 316999/375342 consumed_samples: 324608000 total_loss: 3.052 time: 0.3288 s/iter data_time: 0.2051 s/iter total_throughput: 3114.75 samples/s lr: 6.79e-05 [09/27 14:17:15] lb.utils.events INFO: eta: 2:37:05 iteration: 317099/375342 consumed_samples: 324710400 total_loss: 3.048 time: 0.3288 s/iter data_time: 0.2436 s/iter total_throughput: 3114.76 samples/s lr: 6.77e-05 [09/27 14:17:48] lb.utils.events INFO: eta: 2:38:33 iteration: 317199/375342 consumed_samples: 324812800 total_loss: 3.054 time: 0.3288 s/iter data_time: 0.2340 s/iter total_throughput: 3114.76 samples/s lr: 6.75e-05 [09/27 14:18:21] lb.utils.events INFO: eta: 2:41:48 iteration: 317299/375342 consumed_samples: 324915200 total_loss: 3.064 time: 0.3288 s/iter data_time: 0.2284 s/iter total_throughput: 3114.75 samples/s lr: 6.73e-05 [09/27 14:18:54] lb.utils.events INFO: eta: 2:41:51 iteration: 317399/375342 consumed_samples: 325017600 total_loss: 3.058 time: 0.3288 s/iter data_time: 0.2307 s/iter total_throughput: 3114.76 samples/s lr: 6.71e-05 [09/27 14:19:27] lb.utils.events INFO: eta: 2:40:02 iteration: 317499/375342 consumed_samples: 325120000 total_loss: 3.046 time: 0.3288 s/iter data_time: 0.2258 s/iter total_throughput: 3114.76 samples/s lr: 6.69e-05 [09/27 14:19:59] lb.utils.events INFO: eta: 2:38:41 iteration: 317599/375342 consumed_samples: 325222400 total_loss: 3.064 time: 0.3288 s/iter data_time: 0.2208 s/iter total_throughput: 3114.77 samples/s lr: 6.67e-05 [09/27 14:20:32] lb.utils.events INFO: eta: 2:40:35 iteration: 317699/375342 consumed_samples: 325324800 total_loss: 3.064 time: 0.3288 s/iter data_time: 0.2291 s/iter total_throughput: 3114.78 samples/s lr: 6.65e-05 [09/27 14:21:04] lb.utils.events INFO: eta: 2:46:52 iteration: 317799/375342 consumed_samples: 325427200 total_loss: 3.058 time: 0.3288 s/iter data_time: 0.2432 s/iter total_throughput: 3114.80 samples/s lr: 6.63e-05 [09/27 14:21:37] lb.utils.events INFO: eta: 3:29:03 iteration: 317899/375342 consumed_samples: 325529600 total_loss: 3.043 time: 0.3288 s/iter data_time: 0.2463 s/iter total_throughput: 3114.80 samples/s lr: 6.61e-05 [09/27 14:22:10] lb.utils.events INFO: eta: 3:54:25 iteration: 317999/375342 consumed_samples: 325632000 total_loss: 3.032 time: 0.3288 s/iter data_time: 0.2181 s/iter total_throughput: 3114.80 samples/s lr: 6.59e-05 [09/27 14:22:43] lb.utils.events INFO: eta: 4:13:45 iteration: 318099/375342 consumed_samples: 325734400 total_loss: 3.034 time: 0.3288 s/iter data_time: 0.2260 s/iter total_throughput: 3114.79 samples/s lr: 6.57e-05 [09/27 14:23:16] lb.utils.events INFO: eta: 3:36:32 iteration: 318199/375342 consumed_samples: 325836800 total_loss: 3.058 time: 0.3288 s/iter data_time: 0.2039 s/iter total_throughput: 3114.78 samples/s lr: 6.55e-05 [09/27 14:23:49] lb.utils.events INFO: eta: 2:50:25 iteration: 318299/375342 consumed_samples: 325939200 total_loss: 3.062 time: 0.3288 s/iter data_time: 0.2144 s/iter total_throughput: 3114.78 samples/s lr: 6.54e-05 [09/27 14:24:22] lb.utils.events INFO: eta: 2:36:06 iteration: 318399/375342 consumed_samples: 326041600 total_loss: 3.045 time: 0.3288 s/iter data_time: 0.2176 s/iter total_throughput: 3114.78 samples/s lr: 6.52e-05 [09/27 14:24:55] lb.utils.events INFO: eta: 2:33:10 iteration: 318499/375342 consumed_samples: 326144000 total_loss: 3.035 time: 0.3288 s/iter data_time: 0.2084 s/iter total_throughput: 3114.78 samples/s lr: 6.50e-05 [09/27 14:25:28] lb.utils.events INFO: eta: 2:31:41 iteration: 318599/375342 consumed_samples: 326246400 total_loss: 3.039 time: 0.3288 s/iter data_time: 0.2024 s/iter total_throughput: 3114.78 samples/s lr: 6.48e-05 [09/27 14:26:00] lb.utils.events INFO: eta: 2:30:44 iteration: 318699/375342 consumed_samples: 326348800 total_loss: 3.049 time: 0.3288 s/iter data_time: 0.2134 s/iter total_throughput: 3114.79 samples/s lr: 6.46e-05 [09/27 14:26:33] lb.utils.events INFO: eta: 2:29:48 iteration: 318799/375342 consumed_samples: 326451200 total_loss: 3.042 time: 0.3288 s/iter data_time: 0.2237 s/iter total_throughput: 3114.79 samples/s lr: 6.44e-05 [09/27 14:27:06] lb.utils.events INFO: eta: 2:27:48 iteration: 318899/375342 consumed_samples: 326553600 total_loss: 3.062 time: 0.3288 s/iter data_time: 0.2067 s/iter total_throughput: 3114.80 samples/s lr: 6.42e-05 [09/27 14:27:39] lb.utils.events INFO: eta: 2:27:32 iteration: 318999/375342 consumed_samples: 326656000 total_loss: 3.061 time: 0.3288 s/iter data_time: 0.2325 s/iter total_throughput: 3114.80 samples/s lr: 6.40e-05 [09/27 14:28:12] lb.utils.events INFO: eta: 2:27:37 iteration: 319099/375342 consumed_samples: 326758400 total_loss: 3.05 time: 0.3288 s/iter data_time: 0.2234 s/iter total_throughput: 3114.79 samples/s lr: 6.38e-05 [09/27 14:28:45] lb.utils.events INFO: eta: 2:28:14 iteration: 319199/375342 consumed_samples: 326860800 total_loss: 3.032 time: 0.3288 s/iter data_time: 0.2193 s/iter total_throughput: 3114.79 samples/s lr: 6.37e-05 [09/27 14:29:17] lb.utils.events INFO: eta: 2:28:55 iteration: 319299/375342 consumed_samples: 326963200 total_loss: 3.027 time: 0.3288 s/iter data_time: 0.2183 s/iter total_throughput: 3114.80 samples/s lr: 6.35e-05 [09/27 14:29:50] lb.utils.events INFO: eta: 2:31:59 iteration: 319399/375342 consumed_samples: 327065600 total_loss: 3.039 time: 0.3288 s/iter data_time: 0.2437 s/iter total_throughput: 3114.82 samples/s lr: 6.33e-05 [09/27 14:30:22] lb.utils.events INFO: eta: 2:34:52 iteration: 319499/375342 consumed_samples: 327168000 total_loss: 3.042 time: 0.3288 s/iter data_time: 0.2324 s/iter total_throughput: 3114.82 samples/s lr: 6.31e-05 [09/27 14:30:56] lb.utils.events INFO: eta: 2:55:25 iteration: 319599/375342 consumed_samples: 327270400 total_loss: 3.053 time: 0.3288 s/iter data_time: 0.2648 s/iter total_throughput: 3114.81 samples/s lr: 6.29e-05 [09/27 14:31:28] lb.utils.events INFO: eta: 2:59:42 iteration: 319699/375342 consumed_samples: 327372800 total_loss: 3.056 time: 0.3288 s/iter data_time: 0.2138 s/iter total_throughput: 3114.82 samples/s lr: 6.27e-05 [09/27 14:32:01] lb.utils.events INFO: eta: 3:16:06 iteration: 319799/375342 consumed_samples: 327475200 total_loss: 3.047 time: 0.3287 s/iter data_time: 0.2451 s/iter total_throughput: 3114.83 samples/s lr: 6.25e-05 [09/27 14:32:33] lb.utils.events INFO: eta: 3:45:53 iteration: 319899/375342 consumed_samples: 327577600 total_loss: 3.038 time: 0.3287 s/iter data_time: 0.2234 s/iter total_throughput: 3114.84 samples/s lr: 6.23e-05 [09/27 14:33:07] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0319999 [09/27 14:33:07] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/27 14:33:07] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/27 14:33:11] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0811 s/iter. Inference: 0.1485 s/iter. Eval: 0.0025 s/iter. Total: 0.2321 s/iter. ETA=0:00:08 [09/27 14:33:17] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1430 s/iter. Inference: 0.1496 s/iter. Eval: 0.0022 s/iter. Total: 0.2949 s/iter. ETA=0:00:05 [09/27 14:33:22] lb.evaluation.evaluator INFO: Inference done 50000/50000. Dataloading: 0.1301 s/iter. Inference: 0.1509 s/iter. Eval: 0.0021 s/iter. Total: 0.2832 s/iter. ETA=0:00:00 [09/27 14:33:22] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/27 14:33:22] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.465619 (0.000249 s / iter per device, on 8 devices) [09/27 14:33:22] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000133 s / iter per device, on 8 devices) [09/27 14:33:22] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/27 14:33:22] lb.evaluation.utils INFO: copypaste: Acc@1=79.59 [09/27 14:33:22] lb.evaluation.utils INFO: copypaste: Acc@5=94.312 [09/27 14:33:22] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 79.59000, better than last best score 79.44000 @ iteration 314999. [09/27 14:33:22] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/27 14:33:23] lb.utils.events INFO: eta: 3:38:16 iteration: 319999/375342 consumed_samples: 327680000 total_loss: 3.025 time: 0.3287 s/iter data_time: 0.2151 s/iter total_throughput: 3114.83 samples/s lr: 6.22e-05 [09/27 14:33:54] lb.utils.events INFO: eta: 3:32:32 iteration: 320099/375342 consumed_samples: 327782400 total_loss: 3.037 time: 0.3287 s/iter data_time: 0.2298 s/iter total_throughput: 3114.88 samples/s lr: 6.20e-05 [09/27 14:34:27] lb.utils.events INFO: eta: 3:25:42 iteration: 320199/375342 consumed_samples: 327884800 total_loss: 3.056 time: 0.3287 s/iter data_time: 0.2200 s/iter total_throughput: 3114.88 samples/s lr: 6.18e-05 [09/27 14:35:00] lb.utils.events INFO: eta: 3:25:51 iteration: 320299/375342 consumed_samples: 327987200 total_loss: 3.058 time: 0.3287 s/iter data_time: 0.2380 s/iter total_throughput: 3114.89 samples/s lr: 6.16e-05 [09/27 14:35:32] lb.utils.events INFO: eta: 3:03:12 iteration: 320399/375342 consumed_samples: 328089600 total_loss: 3.048 time: 0.3287 s/iter data_time: 0.2135 s/iter total_throughput: 3114.90 samples/s lr: 6.14e-05 [09/27 14:36:05] lb.utils.events INFO: eta: 2:48:01 iteration: 320499/375342 consumed_samples: 328192000 total_loss: 3.023 time: 0.3287 s/iter data_time: 0.2108 s/iter total_throughput: 3114.91 samples/s lr: 6.12e-05 [09/27 14:36:38] lb.utils.events INFO: eta: 2:34:51 iteration: 320599/375342 consumed_samples: 328294400 total_loss: 3.016 time: 0.3287 s/iter data_time: 0.2245 s/iter total_throughput: 3114.91 samples/s lr: 6.11e-05 [09/27 14:37:11] lb.utils.events INFO: eta: 2:29:42 iteration: 320699/375342 consumed_samples: 328396800 total_loss: 3.026 time: 0.3287 s/iter data_time: 0.2090 s/iter total_throughput: 3114.91 samples/s lr: 6.09e-05 [09/27 14:37:43] lb.utils.events INFO: eta: 2:25:26 iteration: 320799/375342 consumed_samples: 328499200 total_loss: 3.021 time: 0.3287 s/iter data_time: 0.2076 s/iter total_throughput: 3114.91 samples/s lr: 6.07e-05 [09/27 14:38:16] lb.utils.events INFO: eta: 2:23:20 iteration: 320899/375342 consumed_samples: 328601600 total_loss: 3.011 time: 0.3287 s/iter data_time: 0.2065 s/iter total_throughput: 3114.93 samples/s lr: 6.05e-05 [09/27 14:38:48] lb.utils.events INFO: eta: 2:23:34 iteration: 320999/375342 consumed_samples: 328704000 total_loss: 3.029 time: 0.3287 s/iter data_time: 0.2396 s/iter total_throughput: 3114.95 samples/s lr: 6.03e-05 [09/27 14:39:21] lb.utils.events INFO: eta: 2:24:41 iteration: 321099/375342 consumed_samples: 328806400 total_loss: 3.036 time: 0.3287 s/iter data_time: 0.2595 s/iter total_throughput: 3114.96 samples/s lr: 6.01e-05 [09/27 14:39:53] lb.utils.events INFO: eta: 2:26:18 iteration: 321199/375342 consumed_samples: 328908800 total_loss: 3.035 time: 0.3287 s/iter data_time: 0.2542 s/iter total_throughput: 3114.97 samples/s lr: 6.00e-05 [09/27 14:40:26] lb.utils.events INFO: eta: 2:27:57 iteration: 321299/375342 consumed_samples: 329011200 total_loss: 3.055 time: 0.3287 s/iter data_time: 0.2286 s/iter total_throughput: 3114.98 samples/s lr: 5.98e-05 [09/27 14:40:58] lb.utils.events INFO: eta: 2:33:37 iteration: 321399/375342 consumed_samples: 329113600 total_loss: 3.052 time: 0.3287 s/iter data_time: 0.2360 s/iter total_throughput: 3114.98 samples/s lr: 5.96e-05 [09/27 14:41:31] lb.utils.events INFO: eta: 2:33:51 iteration: 321499/375342 consumed_samples: 329216000 total_loss: 3.051 time: 0.3287 s/iter data_time: 0.2208 s/iter total_throughput: 3115.00 samples/s lr: 5.94e-05 [09/27 14:42:03] lb.utils.events INFO: eta: 2:41:54 iteration: 321599/375342 consumed_samples: 329318400 total_loss: 3.037 time: 0.3287 s/iter data_time: 0.2426 s/iter total_throughput: 3115.01 samples/s lr: 5.92e-05 [09/27 14:42:36] lb.utils.events INFO: eta: 3:06:45 iteration: 321699/375342 consumed_samples: 329420800 total_loss: 3.034 time: 0.3287 s/iter data_time: 0.2075 s/iter total_throughput: 3115.00 samples/s lr: 5.91e-05 [09/27 14:43:09] lb.utils.events INFO: eta: 3:06:24 iteration: 321799/375342 consumed_samples: 329523200 total_loss: 3.032 time: 0.3287 s/iter data_time: 0.2035 s/iter total_throughput: 3115.02 samples/s lr: 5.89e-05 [09/27 14:43:41] lb.utils.events INFO: eta: 3:06:03 iteration: 321899/375342 consumed_samples: 329625600 total_loss: 3.017 time: 0.3287 s/iter data_time: 0.1934 s/iter total_throughput: 3115.03 samples/s lr: 5.87e-05 [09/27 14:44:14] lb.utils.events INFO: eta: 2:38:22 iteration: 321999/375342 consumed_samples: 329728000 total_loss: 3.026 time: 0.3287 s/iter data_time: 0.2027 s/iter total_throughput: 3115.03 samples/s lr: 5.85e-05 [09/27 14:44:46] lb.utils.events INFO: eta: 2:26:12 iteration: 322099/375342 consumed_samples: 329830400 total_loss: 3.041 time: 0.3287 s/iter data_time: 0.1930 s/iter total_throughput: 3115.05 samples/s lr: 5.83e-05 [09/27 14:45:19] lb.utils.events INFO: eta: 2:21:53 iteration: 322199/375342 consumed_samples: 329932800 total_loss: 3.052 time: 0.3287 s/iter data_time: 0.2328 s/iter total_throughput: 3115.06 samples/s lr: 5.82e-05 [09/27 14:45:52] lb.utils.events INFO: eta: 2:19:48 iteration: 322299/375342 consumed_samples: 330035200 total_loss: 3.053 time: 0.3287 s/iter data_time: 0.1958 s/iter total_throughput: 3115.06 samples/s lr: 5.80e-05 [09/27 14:46:24] lb.utils.events INFO: eta: 2:18:04 iteration: 322399/375342 consumed_samples: 330137600 total_loss: 3.041 time: 0.3287 s/iter data_time: 0.2027 s/iter total_throughput: 3115.07 samples/s lr: 5.78e-05 [09/27 14:46:57] lb.utils.events INFO: eta: 2:17:20 iteration: 322499/375342 consumed_samples: 330240000 total_loss: 3.035 time: 0.3287 s/iter data_time: 0.2401 s/iter total_throughput: 3115.07 samples/s lr: 5.76e-05 [09/27 14:47:30] lb.utils.events INFO: eta: 2:17:38 iteration: 322599/375342 consumed_samples: 330342400 total_loss: 3.021 time: 0.3287 s/iter data_time: 0.2350 s/iter total_throughput: 3115.06 samples/s lr: 5.75e-05 [09/27 14:48:04] lb.utils.events INFO: eta: 2:17:46 iteration: 322699/375342 consumed_samples: 330444800 total_loss: 3.031 time: 0.3287 s/iter data_time: 0.2319 s/iter total_throughput: 3115.04 samples/s lr: 5.73e-05 [09/27 14:48:37] lb.utils.events INFO: eta: 2:17:32 iteration: 322799/375342 consumed_samples: 330547200 total_loss: 3.027 time: 0.3287 s/iter data_time: 0.2204 s/iter total_throughput: 3115.02 samples/s lr: 5.71e-05 [09/27 14:49:10] lb.utils.events INFO: eta: 2:17:01 iteration: 322899/375342 consumed_samples: 330649600 total_loss: 3.023 time: 0.3287 s/iter data_time: 0.2036 s/iter total_throughput: 3115.02 samples/s lr: 5.69e-05 [09/27 14:49:44] lb.utils.events INFO: eta: 2:16:42 iteration: 322999/375342 consumed_samples: 330752000 total_loss: 3.028 time: 0.3287 s/iter data_time: 0.2156 s/iter total_throughput: 3115.00 samples/s lr: 5.67e-05 [09/27 14:50:17] lb.utils.events INFO: eta: 2:16:36 iteration: 323099/375342 consumed_samples: 330854400 total_loss: 3.029 time: 0.3287 s/iter data_time: 0.2083 s/iter total_throughput: 3114.99 samples/s lr: 5.66e-05 [09/27 14:50:50] lb.utils.events INFO: eta: 2:16:26 iteration: 323199/375342 consumed_samples: 330956800 total_loss: 3.021 time: 0.3287 s/iter data_time: 0.2313 s/iter total_throughput: 3114.99 samples/s lr: 5.64e-05 [09/27 14:51:24] lb.utils.events INFO: eta: 2:16:17 iteration: 323299/375342 consumed_samples: 331059200 total_loss: 3.034 time: 0.3287 s/iter data_time: 0.2312 s/iter total_throughput: 3114.97 samples/s lr: 5.62e-05 [09/27 14:51:57] lb.utils.events INFO: eta: 2:17:11 iteration: 323399/375342 consumed_samples: 331161600 total_loss: 3.041 time: 0.3287 s/iter data_time: 0.2235 s/iter total_throughput: 3114.97 samples/s lr: 5.60e-05 [09/27 14:52:30] lb.utils.events INFO: eta: 2:17:47 iteration: 323499/375342 consumed_samples: 331264000 total_loss: 3.031 time: 0.3287 s/iter data_time: 0.2409 s/iter total_throughput: 3114.96 samples/s lr: 5.59e-05 [09/27 14:53:02] lb.utils.events INFO: eta: 2:15:43 iteration: 323599/375342 consumed_samples: 331366400 total_loss: 3.028 time: 0.3287 s/iter data_time: 0.2279 s/iter total_throughput: 3114.96 samples/s lr: 5.57e-05 [09/27 14:53:36] lb.utils.events INFO: eta: 2:14:15 iteration: 323699/375342 consumed_samples: 331468800 total_loss: 3.019 time: 0.3287 s/iter data_time: 0.2061 s/iter total_throughput: 3114.96 samples/s lr: 5.55e-05 [09/27 14:54:09] lb.utils.events INFO: eta: 2:13:52 iteration: 323799/375342 consumed_samples: 331571200 total_loss: 3.021 time: 0.3287 s/iter data_time: 0.2064 s/iter total_throughput: 3114.95 samples/s lr: 5.54e-05 [09/27 14:54:42] lb.utils.events INFO: eta: 2:13:56 iteration: 323899/375342 consumed_samples: 331673600 total_loss: 3.013 time: 0.3287 s/iter data_time: 0.2322 s/iter total_throughput: 3114.94 samples/s lr: 5.52e-05 [09/27 14:55:15] lb.utils.events INFO: eta: 2:14:44 iteration: 323999/375342 consumed_samples: 331776000 total_loss: 3.021 time: 0.3287 s/iter data_time: 0.2239 s/iter total_throughput: 3114.94 samples/s lr: 5.50e-05 [09/27 14:55:48] lb.utils.events INFO: eta: 2:14:54 iteration: 324099/375342 consumed_samples: 331878400 total_loss: 3.029 time: 0.3287 s/iter data_time: 0.2187 s/iter total_throughput: 3114.93 samples/s lr: 5.48e-05 [09/27 14:56:22] lb.utils.events INFO: eta: 2:15:42 iteration: 324199/375342 consumed_samples: 331980800 total_loss: 3.007 time: 0.3287 s/iter data_time: 0.2253 s/iter total_throughput: 3114.91 samples/s lr: 5.47e-05 [09/27 14:56:55] lb.utils.events INFO: eta: 2:14:51 iteration: 324299/375342 consumed_samples: 332083200 total_loss: 3.01 time: 0.3287 s/iter data_time: 0.2103 s/iter total_throughput: 3114.90 samples/s lr: 5.45e-05 [09/27 14:57:27] lb.utils.events INFO: eta: 2:13:49 iteration: 324399/375342 consumed_samples: 332185600 total_loss: 3.032 time: 0.3287 s/iter data_time: 0.2039 s/iter total_throughput: 3114.91 samples/s lr: 5.43e-05 [09/27 14:58:01] lb.utils.events INFO: eta: 2:13:44 iteration: 324499/375342 consumed_samples: 332288000 total_loss: 3.019 time: 0.3287 s/iter data_time: 0.2249 s/iter total_throughput: 3114.89 samples/s lr: 5.41e-05 [09/27 14:58:34] lb.utils.events INFO: eta: 2:13:20 iteration: 324599/375342 consumed_samples: 332390400 total_loss: 3.004 time: 0.3287 s/iter data_time: 0.2304 s/iter total_throughput: 3114.89 samples/s lr: 5.40e-05 [09/27 14:59:07] lb.utils.events INFO: eta: 2:14:02 iteration: 324699/375342 consumed_samples: 332492800 total_loss: 3.016 time: 0.3287 s/iter data_time: 0.2345 s/iter total_throughput: 3114.88 samples/s lr: 5.38e-05 [09/27 14:59:40] lb.utils.events INFO: eta: 2:14:36 iteration: 324799/375342 consumed_samples: 332595200 total_loss: 3.018 time: 0.3287 s/iter data_time: 0.2147 s/iter total_throughput: 3114.87 samples/s lr: 5.36e-05 [09/27 15:00:14] lb.utils.events INFO: eta: 2:14:06 iteration: 324899/375342 consumed_samples: 332697600 total_loss: 3.022 time: 0.3287 s/iter data_time: 0.2268 s/iter total_throughput: 3114.86 samples/s lr: 5.35e-05 [09/27 15:00:46] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0324999 [09/27 15:00:47] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/27 15:00:47] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/27 15:00:51] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0894 s/iter. Inference: 0.1524 s/iter. Eval: 0.0021 s/iter. Total: 0.2439 s/iter. ETA=0:00:09 [09/27 15:00:57] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1515 s/iter. Inference: 0.1505 s/iter. Eval: 0.0021 s/iter. Total: 0.3042 s/iter. ETA=0:00:05 [09/27 15:01:02] lb.evaluation.evaluator INFO: Inference done 50000/50000. Dataloading: 0.1359 s/iter. Inference: 0.1508 s/iter. Eval: 0.0021 s/iter. Total: 0.2889 s/iter. ETA=0:00:00 [09/27 15:01:02] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/27 15:01:02] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.715486 (0.000254 s / iter per device, on 8 devices) [09/27 15:01:02] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000133 s / iter per device, on 8 devices) [09/27 15:01:02] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/27 15:01:02] lb.evaluation.utils INFO: copypaste: Acc@1=79.632 [09/27 15:01:02] lb.evaluation.utils INFO: copypaste: Acc@5=94.384 [09/27 15:01:02] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 79.63200, better than last best score 79.59000 @ iteration 319999. [09/27 15:01:02] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/27 15:01:03] lb.utils.events INFO: eta: 2:12:51 iteration: 324999/375342 consumed_samples: 332800000 total_loss: 3.032 time: 0.3287 s/iter data_time: 0.2233 s/iter total_throughput: 3114.86 samples/s lr: 5.33e-05 [09/27 15:01:34] lb.utils.events INFO: eta: 2:12:18 iteration: 325099/375342 consumed_samples: 332902400 total_loss: 3.02 time: 0.3287 s/iter data_time: 0.2457 s/iter total_throughput: 3114.91 samples/s lr: 5.31e-05 [09/27 15:02:08] lb.utils.events INFO: eta: 2:12:02 iteration: 325199/375342 consumed_samples: 333004800 total_loss: 3.021 time: 0.3287 s/iter data_time: 0.2365 s/iter total_throughput: 3114.90 samples/s lr: 5.30e-05 [09/27 15:02:41] lb.utils.events INFO: eta: 2:12:03 iteration: 325299/375342 consumed_samples: 333107200 total_loss: 3.021 time: 0.3287 s/iter data_time: 0.2290 s/iter total_throughput: 3114.90 samples/s lr: 5.28e-05 [09/27 15:03:13] lb.utils.events INFO: eta: 2:14:24 iteration: 325399/375342 consumed_samples: 333209600 total_loss: 3.035 time: 0.3287 s/iter data_time: 0.2543 s/iter total_throughput: 3114.90 samples/s lr: 5.26e-05 [09/27 15:03:47] lb.utils.events INFO: eta: 2:16:57 iteration: 325499/375342 consumed_samples: 333312000 total_loss: 3.022 time: 0.3287 s/iter data_time: 0.2533 s/iter total_throughput: 3114.89 samples/s lr: 5.25e-05 [09/27 15:04:20] lb.utils.events INFO: eta: 2:15:30 iteration: 325599/375342 consumed_samples: 333414400 total_loss: 3.007 time: 0.3287 s/iter data_time: 0.2328 s/iter total_throughput: 3114.88 samples/s lr: 5.23e-05 [09/27 15:04:53] lb.utils.events INFO: eta: 2:15:27 iteration: 325699/375342 consumed_samples: 333516800 total_loss: 3.006 time: 0.3287 s/iter data_time: 0.2291 s/iter total_throughput: 3114.88 samples/s lr: 5.21e-05 [09/27 15:05:26] lb.utils.events INFO: eta: 2:16:32 iteration: 325799/375342 consumed_samples: 333619200 total_loss: 3.01 time: 0.3287 s/iter data_time: 0.2357 s/iter total_throughput: 3114.87 samples/s lr: 5.20e-05 [09/27 15:05:59] lb.utils.events INFO: eta: 2:15:20 iteration: 325899/375342 consumed_samples: 333721600 total_loss: 3.023 time: 0.3287 s/iter data_time: 0.2296 s/iter total_throughput: 3114.86 samples/s lr: 5.18e-05 [09/27 15:06:33] lb.utils.events INFO: eta: 2:14:21 iteration: 325999/375342 consumed_samples: 333824000 total_loss: 3.023 time: 0.3287 s/iter data_time: 0.2092 s/iter total_throughput: 3114.85 samples/s lr: 5.16e-05 [09/27 15:07:06] lb.utils.events INFO: eta: 2:13:49 iteration: 326099/375342 consumed_samples: 333926400 total_loss: 3.007 time: 0.3287 s/iter data_time: 0.2047 s/iter total_throughput: 3114.84 samples/s lr: 5.15e-05 [09/27 15:07:39] lb.utils.events INFO: eta: 2:11:37 iteration: 326199/375342 consumed_samples: 334028800 total_loss: 3.014 time: 0.3288 s/iter data_time: 0.2140 s/iter total_throughput: 3114.83 samples/s lr: 5.13e-05 [09/27 15:08:12] lb.utils.events INFO: eta: 2:11:14 iteration: 326299/375342 consumed_samples: 334131200 total_loss: 3.033 time: 0.3287 s/iter data_time: 0.2090 s/iter total_throughput: 3114.83 samples/s lr: 5.11e-05 [09/27 15:08:44] lb.utils.events INFO: eta: 2:09:05 iteration: 326399/375342 consumed_samples: 334233600 total_loss: 3.016 time: 0.3287 s/iter data_time: 0.2177 s/iter total_throughput: 3114.84 samples/s lr: 5.10e-05 [09/27 15:09:18] lb.utils.events INFO: eta: 2:06:45 iteration: 326499/375342 consumed_samples: 334336000 total_loss: 3.007 time: 0.3288 s/iter data_time: 0.2094 s/iter total_throughput: 3114.83 samples/s lr: 5.08e-05 [09/27 15:09:51] lb.utils.events INFO: eta: 2:06:04 iteration: 326599/375342 consumed_samples: 334438400 total_loss: 3.007 time: 0.3288 s/iter data_time: 0.2057 s/iter total_throughput: 3114.82 samples/s lr: 5.06e-05 [09/27 15:10:24] lb.utils.events INFO: eta: 2:05:25 iteration: 326699/375342 consumed_samples: 334540800 total_loss: 3.019 time: 0.3288 s/iter data_time: 0.2045 s/iter total_throughput: 3114.82 samples/s lr: 5.05e-05 [09/27 15:10:57] lb.utils.events INFO: eta: 2:04:22 iteration: 326799/375342 consumed_samples: 334643200 total_loss: 3.033 time: 0.3288 s/iter data_time: 0.2156 s/iter total_throughput: 3114.82 samples/s lr: 5.03e-05 [09/27 15:11:29] lb.utils.events INFO: eta: 2:04:40 iteration: 326899/375342 consumed_samples: 334745600 total_loss: 3.008 time: 0.3288 s/iter data_time: 0.2282 s/iter total_throughput: 3114.82 samples/s lr: 5.01e-05 [09/27 15:12:02] lb.utils.events INFO: eta: 2:05:09 iteration: 326999/375342 consumed_samples: 334848000 total_loss: 3.004 time: 0.3287 s/iter data_time: 0.2203 s/iter total_throughput: 3114.83 samples/s lr: 5.00e-05 [09/27 15:12:36] lb.utils.events INFO: eta: 2:04:59 iteration: 327099/375342 consumed_samples: 334950400 total_loss: 3.022 time: 0.3288 s/iter data_time: 0.2051 s/iter total_throughput: 3114.81 samples/s lr: 4.98e-05 [09/27 15:13:08] lb.utils.events INFO: eta: 2:05:19 iteration: 327199/375342 consumed_samples: 335052800 total_loss: 3.036 time: 0.3288 s/iter data_time: 0.2336 s/iter total_throughput: 3114.81 samples/s lr: 4.96e-05 [09/27 15:13:42] lb.utils.events INFO: eta: 2:05:28 iteration: 327299/375342 consumed_samples: 335155200 total_loss: 3.023 time: 0.3288 s/iter data_time: 0.2333 s/iter total_throughput: 3114.80 samples/s lr: 4.95e-05 [09/27 15:14:15] lb.utils.events INFO: eta: 2:05:31 iteration: 327399/375342 consumed_samples: 335257600 total_loss: 3.021 time: 0.3288 s/iter data_time: 0.2216 s/iter total_throughput: 3114.79 samples/s lr: 4.93e-05 [09/27 15:14:47] lb.utils.events INFO: eta: 2:06:23 iteration: 327499/375342 consumed_samples: 335360000 total_loss: 3.004 time: 0.3288 s/iter data_time: 0.2236 s/iter total_throughput: 3114.80 samples/s lr: 4.92e-05 [09/27 15:15:20] lb.utils.events INFO: eta: 2:07:48 iteration: 327599/375342 consumed_samples: 335462400 total_loss: 3.004 time: 0.3288 s/iter data_time: 0.2396 s/iter total_throughput: 3114.81 samples/s lr: 4.90e-05 [09/27 15:15:53] lb.utils.events INFO: eta: 2:12:39 iteration: 327699/375342 consumed_samples: 335564800 total_loss: 3.021 time: 0.3288 s/iter data_time: 0.2515 s/iter total_throughput: 3114.80 samples/s lr: 4.88e-05 [09/27 15:16:26] lb.utils.events INFO: eta: 2:18:38 iteration: 327799/375342 consumed_samples: 335667200 total_loss: 3.023 time: 0.3288 s/iter data_time: 0.2360 s/iter total_throughput: 3114.80 samples/s lr: 4.87e-05 [09/27 15:16:59] lb.utils.events INFO: eta: 2:38:46 iteration: 327899/375342 consumed_samples: 335769600 total_loss: 3.027 time: 0.3288 s/iter data_time: 0.2470 s/iter total_throughput: 3114.79 samples/s lr: 4.85e-05 [09/27 15:17:32] lb.utils.events INFO: eta: 2:54:17 iteration: 327999/375342 consumed_samples: 335872000 total_loss: 3.031 time: 0.3288 s/iter data_time: 0.2441 s/iter total_throughput: 3114.79 samples/s lr: 4.84e-05 [09/27 15:18:06] lb.utils.events INFO: eta: 2:48:50 iteration: 328099/375342 consumed_samples: 335974400 total_loss: 3.014 time: 0.3288 s/iter data_time: 0.2230 s/iter total_throughput: 3114.77 samples/s lr: 4.82e-05 [09/27 15:18:39] lb.utils.events INFO: eta: 2:37:14 iteration: 328199/375342 consumed_samples: 336076800 total_loss: 2.996 time: 0.3288 s/iter data_time: 0.2141 s/iter total_throughput: 3114.77 samples/s lr: 4.80e-05 [09/27 15:19:12] lb.utils.events INFO: eta: 2:36:54 iteration: 328299/375342 consumed_samples: 336179200 total_loss: 2.998 time: 0.3288 s/iter data_time: 0.2328 s/iter total_throughput: 3114.77 samples/s lr: 4.79e-05 [09/27 15:19:45] lb.utils.events INFO: eta: 2:49:35 iteration: 328399/375342 consumed_samples: 336281600 total_loss: 3.005 time: 0.3288 s/iter data_time: 0.2161 s/iter total_throughput: 3114.76 samples/s lr: 4.77e-05 [09/27 15:20:17] lb.utils.events INFO: eta: 2:46:04 iteration: 328499/375342 consumed_samples: 336384000 total_loss: 3.008 time: 0.3288 s/iter data_time: 0.2236 s/iter total_throughput: 3114.77 samples/s lr: 4.76e-05 [09/27 15:20:51] lb.utils.events INFO: eta: 2:14:56 iteration: 328599/375342 consumed_samples: 336486400 total_loss: 3.018 time: 0.3288 s/iter data_time: 0.2070 s/iter total_throughput: 3114.76 samples/s lr: 4.74e-05 [09/27 15:21:24] lb.utils.events INFO: eta: 2:05:34 iteration: 328699/375342 consumed_samples: 336588800 total_loss: 3.004 time: 0.3288 s/iter data_time: 0.2147 s/iter total_throughput: 3114.76 samples/s lr: 4.72e-05 [09/27 15:21:56] lb.utils.events INFO: eta: 2:04:03 iteration: 328799/375342 consumed_samples: 336691200 total_loss: 3.001 time: 0.3288 s/iter data_time: 0.2216 s/iter total_throughput: 3114.76 samples/s lr: 4.71e-05 [09/27 15:22:29] lb.utils.events INFO: eta: 2:02:45 iteration: 328899/375342 consumed_samples: 336793600 total_loss: 3.005 time: 0.3288 s/iter data_time: 0.2088 s/iter total_throughput: 3114.76 samples/s lr: 4.69e-05 [09/27 15:23:02] lb.utils.events INFO: eta: 2:01:29 iteration: 328999/375342 consumed_samples: 336896000 total_loss: 3.014 time: 0.3288 s/iter data_time: 0.2176 s/iter total_throughput: 3114.75 samples/s lr: 4.68e-05 [09/27 15:23:36] lb.utils.events INFO: eta: 2:02:00 iteration: 329099/375342 consumed_samples: 336998400 total_loss: 3.015 time: 0.3288 s/iter data_time: 0.2315 s/iter total_throughput: 3114.74 samples/s lr: 4.66e-05 [09/27 15:24:09] lb.utils.events INFO: eta: 2:02:18 iteration: 329199/375342 consumed_samples: 337100800 total_loss: 2.984 time: 0.3288 s/iter data_time: 0.2245 s/iter total_throughput: 3114.73 samples/s lr: 4.65e-05 [09/27 15:24:42] lb.utils.events INFO: eta: 2:01:42 iteration: 329299/375342 consumed_samples: 337203200 total_loss: 3.003 time: 0.3288 s/iter data_time: 0.2256 s/iter total_throughput: 3114.73 samples/s lr: 4.63e-05 [09/27 15:25:15] lb.utils.events INFO: eta: 2:01:11 iteration: 329399/375342 consumed_samples: 337305600 total_loss: 3.021 time: 0.3288 s/iter data_time: 0.2257 s/iter total_throughput: 3114.73 samples/s lr: 4.61e-05 [09/27 15:25:48] lb.utils.events INFO: eta: 2:00:08 iteration: 329499/375342 consumed_samples: 337408000 total_loss: 3.02 time: 0.3288 s/iter data_time: 0.1967 s/iter total_throughput: 3114.73 samples/s lr: 4.60e-05 [09/27 15:26:21] lb.utils.events INFO: eta: 1:59:55 iteration: 329599/375342 consumed_samples: 337510400 total_loss: 3.025 time: 0.3288 s/iter data_time: 0.2031 s/iter total_throughput: 3114.72 samples/s lr: 4.58e-05 [09/27 15:26:53] lb.utils.events INFO: eta: 1:59:38 iteration: 329699/375342 consumed_samples: 337612800 total_loss: 3.026 time: 0.3288 s/iter data_time: 0.2035 s/iter total_throughput: 3114.73 samples/s lr: 4.57e-05 [09/27 15:27:26] lb.utils.events INFO: eta: 1:59:00 iteration: 329799/375342 consumed_samples: 337715200 total_loss: 3.019 time: 0.3288 s/iter data_time: 0.2024 s/iter total_throughput: 3114.74 samples/s lr: 4.55e-05 [09/27 15:27:59] lb.utils.events INFO: eta: 1:59:11 iteration: 329899/375342 consumed_samples: 337817600 total_loss: 2.999 time: 0.3288 s/iter data_time: 0.2405 s/iter total_throughput: 3114.73 samples/s lr: 4.54e-05 [09/27 15:28:32] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0329999 [09/27 15:28:33] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/27 15:28:33] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/27 15:28:37] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0830 s/iter. Inference: 0.1505 s/iter. Eval: 0.0022 s/iter. Total: 0.2357 s/iter. ETA=0:00:08 [09/27 15:28:42] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1432 s/iter. Inference: 0.1505 s/iter. Eval: 0.0022 s/iter. Total: 0.2959 s/iter. ETA=0:00:05 [09/27 15:28:47] lb.evaluation.evaluator INFO: Inference done 49152/50000. Dataloading: 0.1301 s/iter. Inference: 0.1492 s/iter. Eval: 0.0021 s/iter. Total: 0.2816 s/iter. ETA=0:00:00 [09/27 15:28:48] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/27 15:28:48] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.548965 (0.000251 s / iter per device, on 8 devices) [09/27 15:28:48] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000132 s / iter per device, on 8 devices) [09/27 15:28:48] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/27 15:28:48] lb.evaluation.utils INFO: copypaste: Acc@1=79.72200000000001 [09/27 15:28:48] lb.evaluation.utils INFO: copypaste: Acc@5=94.446 [09/27 15:28:48] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 79.72200, better than last best score 79.63200 @ iteration 324999. [09/27 15:28:48] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/27 15:28:49] lb.utils.events INFO: eta: 2:00:06 iteration: 329999/375342 consumed_samples: 337920000 total_loss: 3.003 time: 0.3288 s/iter data_time: 0.2494 s/iter total_throughput: 3114.73 samples/s lr: 4.52e-05 [09/27 15:29:20] lb.utils.events INFO: eta: 1:59:18 iteration: 330099/375342 consumed_samples: 338022400 total_loss: 3.014 time: 0.3288 s/iter data_time: 0.2368 s/iter total_throughput: 3114.77 samples/s lr: 4.51e-05 [09/27 15:29:53] lb.utils.events INFO: eta: 1:58:42 iteration: 330199/375342 consumed_samples: 338124800 total_loss: 3.002 time: 0.3288 s/iter data_time: 0.2088 s/iter total_throughput: 3114.77 samples/s lr: 4.49e-05 [09/27 15:30:26] lb.utils.events INFO: eta: 1:58:19 iteration: 330299/375342 consumed_samples: 338227200 total_loss: 3.008 time: 0.3288 s/iter data_time: 0.2339 s/iter total_throughput: 3114.76 samples/s lr: 4.48e-05 [09/27 15:30:59] lb.utils.events INFO: eta: 1:58:38 iteration: 330399/375342 consumed_samples: 338329600 total_loss: 3.004 time: 0.3288 s/iter data_time: 0.2419 s/iter total_throughput: 3114.76 samples/s lr: 4.46e-05 [09/27 15:31:32] lb.utils.events INFO: eta: 2:01:42 iteration: 330499/375342 consumed_samples: 338432000 total_loss: 3.016 time: 0.3288 s/iter data_time: 0.2661 s/iter total_throughput: 3114.76 samples/s lr: 4.45e-05 [09/27 15:32:05] lb.utils.events INFO: eta: 2:05:11 iteration: 330599/375342 consumed_samples: 338534400 total_loss: 3.016 time: 0.3288 s/iter data_time: 0.2166 s/iter total_throughput: 3114.75 samples/s lr: 4.43e-05 [09/27 15:32:38] lb.utils.events INFO: eta: 2:05:40 iteration: 330699/375342 consumed_samples: 338636800 total_loss: 3.008 time: 0.3288 s/iter data_time: 0.2053 s/iter total_throughput: 3114.75 samples/s lr: 4.42e-05 [09/27 15:33:11] lb.utils.events INFO: eta: 2:09:26 iteration: 330799/375342 consumed_samples: 338739200 total_loss: 2.996 time: 0.3288 s/iter data_time: 0.2157 s/iter total_throughput: 3114.74 samples/s lr: 4.40e-05 [09/27 15:33:44] lb.utils.events INFO: eta: 2:04:12 iteration: 330899/375342 consumed_samples: 338841600 total_loss: 3.004 time: 0.3288 s/iter data_time: 0.2061 s/iter total_throughput: 3114.74 samples/s lr: 4.39e-05 [09/27 15:34:18] lb.utils.events INFO: eta: 2:00:41 iteration: 330999/375342 consumed_samples: 338944000 total_loss: 3.003 time: 0.3288 s/iter data_time: 0.2258 s/iter total_throughput: 3114.73 samples/s lr: 4.37e-05 [09/27 15:34:50] lb.utils.events INFO: eta: 2:01:09 iteration: 331099/375342 consumed_samples: 339046400 total_loss: 2.997 time: 0.3288 s/iter data_time: 0.2152 s/iter total_throughput: 3114.74 samples/s lr: 4.36e-05 [09/27 15:35:23] lb.utils.events INFO: eta: 2:03:36 iteration: 331199/375342 consumed_samples: 339148800 total_loss: 3 time: 0.3288 s/iter data_time: 0.2392 s/iter total_throughput: 3114.73 samples/s lr: 4.34e-05 [09/27 15:35:56] lb.utils.events INFO: eta: 2:07:28 iteration: 331299/375342 consumed_samples: 339251200 total_loss: 3.012 time: 0.3288 s/iter data_time: 0.2359 s/iter total_throughput: 3114.74 samples/s lr: 4.33e-05 [09/27 15:36:29] lb.utils.events INFO: eta: 2:03:36 iteration: 331399/375342 consumed_samples: 339353600 total_loss: 3.016 time: 0.3288 s/iter data_time: 0.2358 s/iter total_throughput: 3114.73 samples/s lr: 4.31e-05 [09/27 15:37:02] lb.utils.events INFO: eta: 1:59:33 iteration: 331499/375342 consumed_samples: 339456000 total_loss: 3.009 time: 0.3288 s/iter data_time: 0.2301 s/iter total_throughput: 3114.72 samples/s lr: 4.30e-05 [09/27 15:37:35] lb.utils.events INFO: eta: 1:57:48 iteration: 331599/375342 consumed_samples: 339558400 total_loss: 3.002 time: 0.3288 s/iter data_time: 0.2082 s/iter total_throughput: 3114.72 samples/s lr: 4.28e-05 [09/27 15:38:08] lb.utils.events INFO: eta: 1:57:03 iteration: 331699/375342 consumed_samples: 339660800 total_loss: 3.002 time: 0.3288 s/iter data_time: 0.2094 s/iter total_throughput: 3114.72 samples/s lr: 4.27e-05 [09/27 15:38:41] lb.utils.events INFO: eta: 1:55:58 iteration: 331799/375342 consumed_samples: 339763200 total_loss: 3.019 time: 0.3288 s/iter data_time: 0.2089 s/iter total_throughput: 3114.72 samples/s lr: 4.25e-05 [09/27 15:39:14] lb.utils.events INFO: eta: 1:55:44 iteration: 331899/375342 consumed_samples: 339865600 total_loss: 2.999 time: 0.3288 s/iter data_time: 0.2294 s/iter total_throughput: 3114.72 samples/s lr: 4.24e-05 [09/27 15:39:47] lb.utils.events INFO: eta: 1:55:35 iteration: 331999/375342 consumed_samples: 339968000 total_loss: 2.991 time: 0.3288 s/iter data_time: 0.2263 s/iter total_throughput: 3114.71 samples/s lr: 4.22e-05 [09/27 15:40:20] lb.utils.events INFO: eta: 1:56:11 iteration: 332099/375342 consumed_samples: 340070400 total_loss: 3.002 time: 0.3288 s/iter data_time: 0.2277 s/iter total_throughput: 3114.71 samples/s lr: 4.21e-05 [09/27 15:40:53] lb.utils.events INFO: eta: 1:54:58 iteration: 332199/375342 consumed_samples: 340172800 total_loss: 2.999 time: 0.3288 s/iter data_time: 0.2265 s/iter total_throughput: 3114.71 samples/s lr: 4.19e-05 [09/27 15:41:26] lb.utils.events INFO: eta: 1:54:40 iteration: 332299/375342 consumed_samples: 340275200 total_loss: 2.996 time: 0.3288 s/iter data_time: 0.2190 s/iter total_throughput: 3114.71 samples/s lr: 4.18e-05 [09/27 15:41:59] lb.utils.events INFO: eta: 1:54:29 iteration: 332399/375342 consumed_samples: 340377600 total_loss: 2.999 time: 0.3288 s/iter data_time: 0.2177 s/iter total_throughput: 3114.71 samples/s lr: 4.16e-05 [09/27 15:42:31] lb.utils.events INFO: eta: 1:53:58 iteration: 332499/375342 consumed_samples: 340480000 total_loss: 3 time: 0.3288 s/iter data_time: 0.2160 s/iter total_throughput: 3114.72 samples/s lr: 4.15e-05 [09/27 15:43:04] lb.utils.events INFO: eta: 1:53:14 iteration: 332599/375342 consumed_samples: 340582400 total_loss: 3.007 time: 0.3288 s/iter data_time: 0.2125 s/iter total_throughput: 3114.72 samples/s lr: 4.13e-05 [09/27 15:43:37] lb.utils.events INFO: eta: 1:53:23 iteration: 332699/375342 consumed_samples: 340684800 total_loss: 3.007 time: 0.3288 s/iter data_time: 0.2296 s/iter total_throughput: 3114.73 samples/s lr: 4.12e-05 [09/27 15:44:09] lb.utils.events INFO: eta: 1:54:54 iteration: 332799/375342 consumed_samples: 340787200 total_loss: 2.987 time: 0.3288 s/iter data_time: 0.2431 s/iter total_throughput: 3114.73 samples/s lr: 4.11e-05 [09/27 15:44:42] lb.utils.events INFO: eta: 1:55:19 iteration: 332899/375342 consumed_samples: 340889600 total_loss: 2.991 time: 0.3288 s/iter data_time: 0.2171 s/iter total_throughput: 3114.74 samples/s lr: 4.09e-05 [09/27 15:45:15] lb.utils.events INFO: eta: 1:53:52 iteration: 332999/375342 consumed_samples: 340992000 total_loss: 3.003 time: 0.3288 s/iter data_time: 0.2029 s/iter total_throughput: 3114.74 samples/s lr: 4.08e-05 [09/27 15:45:48] lb.utils.events INFO: eta: 1:53:25 iteration: 333099/375342 consumed_samples: 341094400 total_loss: 3 time: 0.3288 s/iter data_time: 0.2250 s/iter total_throughput: 3114.74 samples/s lr: 4.06e-05 [09/27 15:46:21] lb.utils.events INFO: eta: 1:53:12 iteration: 333199/375342 consumed_samples: 341196800 total_loss: 2.991 time: 0.3288 s/iter data_time: 0.2269 s/iter total_throughput: 3114.74 samples/s lr: 4.05e-05 [09/27 15:46:53] lb.utils.events INFO: eta: 1:51:54 iteration: 333299/375342 consumed_samples: 341299200 total_loss: 2.993 time: 0.3288 s/iter data_time: 0.2057 s/iter total_throughput: 3114.74 samples/s lr: 4.03e-05 [09/27 15:47:26] lb.utils.events INFO: eta: 1:51:03 iteration: 333399/375342 consumed_samples: 341401600 total_loss: 2.994 time: 0.3288 s/iter data_time: 0.2141 s/iter total_throughput: 3114.75 samples/s lr: 4.02e-05 [09/27 15:47:59] lb.utils.events INFO: eta: 1:50:22 iteration: 333499/375342 consumed_samples: 341504000 total_loss: 2.995 time: 0.3288 s/iter data_time: 0.1938 s/iter total_throughput: 3114.76 samples/s lr: 4.00e-05 [09/27 15:48:32] lb.utils.events INFO: eta: 1:50:31 iteration: 333599/375342 consumed_samples: 341606400 total_loss: 3.007 time: 0.3288 s/iter data_time: 0.2275 s/iter total_throughput: 3114.75 samples/s lr: 3.99e-05 [09/27 15:49:04] lb.utils.events INFO: eta: 1:50:46 iteration: 333699/375342 consumed_samples: 341708800 total_loss: 3.006 time: 0.3288 s/iter data_time: 0.2166 s/iter total_throughput: 3114.76 samples/s lr: 3.98e-05 [09/27 15:49:37] lb.utils.events INFO: eta: 1:49:46 iteration: 333799/375342 consumed_samples: 341811200 total_loss: 3.002 time: 0.3288 s/iter data_time: 0.2216 s/iter total_throughput: 3114.76 samples/s lr: 3.96e-05 [09/27 15:50:10] lb.utils.events INFO: eta: 1:49:06 iteration: 333899/375342 consumed_samples: 341913600 total_loss: 3.002 time: 0.3288 s/iter data_time: 0.2103 s/iter total_throughput: 3114.76 samples/s lr: 3.95e-05 [09/27 15:50:43] lb.utils.events INFO: eta: 1:48:50 iteration: 333999/375342 consumed_samples: 342016000 total_loss: 3.012 time: 0.3288 s/iter data_time: 0.2065 s/iter total_throughput: 3114.76 samples/s lr: 3.93e-05 [09/27 15:51:15] lb.utils.events INFO: eta: 1:48:14 iteration: 334099/375342 consumed_samples: 342118400 total_loss: 2.996 time: 0.3288 s/iter data_time: 0.2019 s/iter total_throughput: 3114.77 samples/s lr: 3.92e-05 [09/27 15:51:48] lb.utils.events INFO: eta: 1:47:30 iteration: 334199/375342 consumed_samples: 342220800 total_loss: 2.987 time: 0.3288 s/iter data_time: 0.2056 s/iter total_throughput: 3114.78 samples/s lr: 3.91e-05 [09/27 15:52:21] lb.utils.events INFO: eta: 1:47:31 iteration: 334299/375342 consumed_samples: 342323200 total_loss: 2.996 time: 0.3288 s/iter data_time: 0.2137 s/iter total_throughput: 3114.78 samples/s lr: 3.89e-05 [09/27 15:52:54] lb.utils.events INFO: eta: 1:47:13 iteration: 334399/375342 consumed_samples: 342425600 total_loss: 3.007 time: 0.3288 s/iter data_time: 0.2180 s/iter total_throughput: 3114.79 samples/s lr: 3.88e-05 [09/27 15:53:26] lb.utils.events INFO: eta: 1:47:21 iteration: 334499/375342 consumed_samples: 342528000 total_loss: 2.992 time: 0.3288 s/iter data_time: 0.2173 s/iter total_throughput: 3114.80 samples/s lr: 3.86e-05 [09/27 15:53:58] lb.utils.events INFO: eta: 1:47:19 iteration: 334599/375342 consumed_samples: 342630400 total_loss: 2.992 time: 0.3288 s/iter data_time: 0.2279 s/iter total_throughput: 3114.82 samples/s lr: 3.85e-05 [09/27 15:54:31] lb.utils.events INFO: eta: 1:47:51 iteration: 334699/375342 consumed_samples: 342732800 total_loss: 2.997 time: 0.3288 s/iter data_time: 0.2154 s/iter total_throughput: 3114.82 samples/s lr: 3.84e-05 [09/27 15:55:03] lb.utils.events INFO: eta: 1:47:42 iteration: 334799/375342 consumed_samples: 342835200 total_loss: 2.99 time: 0.3287 s/iter data_time: 0.2514 s/iter total_throughput: 3114.84 samples/s lr: 3.82e-05 [09/27 15:55:36] lb.utils.events INFO: eta: 1:47:46 iteration: 334899/375342 consumed_samples: 342937600 total_loss: 2.984 time: 0.3287 s/iter data_time: 0.2153 s/iter total_throughput: 3114.83 samples/s lr: 3.81e-05 [09/27 15:56:09] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0334999 [09/27 15:56:10] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/27 15:56:10] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/27 15:56:14] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0876 s/iter. Inference: 0.1484 s/iter. Eval: 0.0019 s/iter. Total: 0.2379 s/iter. ETA=0:00:08 [09/27 15:56:20] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1452 s/iter. Inference: 0.1498 s/iter. Eval: 0.0020 s/iter. Total: 0.2970 s/iter. ETA=0:00:05 [09/27 15:56:25] lb.evaluation.evaluator INFO: Inference done 50000/50000. Dataloading: 0.1321 s/iter. Inference: 0.1504 s/iter. Eval: 0.0020 s/iter. Total: 0.2846 s/iter. ETA=0:00:00 [09/27 15:56:25] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/27 15:56:25] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.527324 (0.000251 s / iter per device, on 8 devices) [09/27 15:56:25] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000132 s / iter per device, on 8 devices) [09/27 15:56:25] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/27 15:56:25] lb.evaluation.utils INFO: copypaste: Acc@1=79.74799999999999 [09/27 15:56:25] lb.evaluation.utils INFO: copypaste: Acc@5=94.498 [09/27 15:56:25] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 79.74800, better than last best score 79.72200 @ iteration 329999. [09/27 15:56:25] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/27 15:56:26] lb.utils.events INFO: eta: 1:47:59 iteration: 334999/375342 consumed_samples: 343040000 total_loss: 2.988 time: 0.3287 s/iter data_time: 0.2319 s/iter total_throughput: 3114.83 samples/s lr: 3.80e-05 [09/27 15:56:57] lb.utils.events INFO: eta: 1:47:57 iteration: 335099/375342 consumed_samples: 343142400 total_loss: 2.988 time: 0.3287 s/iter data_time: 0.2013 s/iter total_throughput: 3114.89 samples/s lr: 3.78e-05 [09/27 15:57:29] lb.utils.events INFO: eta: 1:47:32 iteration: 335199/375342 consumed_samples: 343244800 total_loss: 2.994 time: 0.3287 s/iter data_time: 0.1989 s/iter total_throughput: 3114.90 samples/s lr: 3.77e-05 [09/27 15:58:02] lb.utils.events INFO: eta: 1:45:57 iteration: 335299/375342 consumed_samples: 343347200 total_loss: 2.995 time: 0.3287 s/iter data_time: 0.2062 s/iter total_throughput: 3114.92 samples/s lr: 3.75e-05 [09/27 15:58:35] lb.utils.events INFO: eta: 1:45:22 iteration: 335399/375342 consumed_samples: 343449600 total_loss: 2.993 time: 0.3287 s/iter data_time: 0.2101 s/iter total_throughput: 3114.90 samples/s lr: 3.74e-05 [09/27 15:59:08] lb.utils.events INFO: eta: 1:44:40 iteration: 335499/375342 consumed_samples: 343552000 total_loss: 2.988 time: 0.3287 s/iter data_time: 0.2007 s/iter total_throughput: 3114.91 samples/s lr: 3.73e-05 [09/27 15:59:40] lb.utils.events INFO: eta: 1:43:29 iteration: 335599/375342 consumed_samples: 343654400 total_loss: 2.984 time: 0.3287 s/iter data_time: 0.2000 s/iter total_throughput: 3114.91 samples/s lr: 3.71e-05 [09/27 16:00:12] lb.utils.events INFO: eta: 1:42:42 iteration: 335699/375342 consumed_samples: 343756800 total_loss: 2.982 time: 0.3287 s/iter data_time: 0.2164 s/iter total_throughput: 3114.94 samples/s lr: 3.70e-05 [09/27 16:00:45] lb.utils.events INFO: eta: 1:42:15 iteration: 335799/375342 consumed_samples: 343859200 total_loss: 2.982 time: 0.3287 s/iter data_time: 0.1974 s/iter total_throughput: 3114.95 samples/s lr: 3.69e-05 [09/27 16:01:17] lb.utils.events INFO: eta: 1:41:23 iteration: 335899/375342 consumed_samples: 343961600 total_loss: 2.974 time: 0.3287 s/iter data_time: 0.1931 s/iter total_throughput: 3114.97 samples/s lr: 3.67e-05 [09/27 16:01:49] lb.utils.events INFO: eta: 1:41:01 iteration: 335999/375342 consumed_samples: 344064000 total_loss: 2.985 time: 0.3287 s/iter data_time: 0.2042 s/iter total_throughput: 3114.99 samples/s lr: 3.66e-05 [09/27 16:02:22] lb.utils.events INFO: eta: 1:40:46 iteration: 336099/375342 consumed_samples: 344166400 total_loss: 2.989 time: 0.3287 s/iter data_time: 0.2025 s/iter total_throughput: 3114.99 samples/s lr: 3.65e-05 [09/27 16:02:55] lb.utils.events INFO: eta: 1:40:36 iteration: 336199/375342 consumed_samples: 344268800 total_loss: 2.986 time: 0.3287 s/iter data_time: 0.2248 s/iter total_throughput: 3114.99 samples/s lr: 3.63e-05 [09/27 16:03:29] lb.utils.events INFO: eta: 1:40:42 iteration: 336299/375342 consumed_samples: 344371200 total_loss: 2.992 time: 0.3287 s/iter data_time: 0.2347 s/iter total_throughput: 3114.97 samples/s lr: 3.62e-05 [09/27 16:04:02] lb.utils.events INFO: eta: 1:40:46 iteration: 336399/375342 consumed_samples: 344473600 total_loss: 3.014 time: 0.3287 s/iter data_time: 0.2313 s/iter total_throughput: 3114.97 samples/s lr: 3.61e-05 [09/27 16:04:35] lb.utils.events INFO: eta: 1:41:01 iteration: 336499/375342 consumed_samples: 344576000 total_loss: 3.005 time: 0.3287 s/iter data_time: 0.2189 s/iter total_throughput: 3114.96 samples/s lr: 3.59e-05 [09/27 16:05:08] lb.utils.events INFO: eta: 1:40:56 iteration: 336599/375342 consumed_samples: 344678400 total_loss: 2.978 time: 0.3287 s/iter data_time: 0.2267 s/iter total_throughput: 3114.94 samples/s lr: 3.58e-05 [09/27 16:05:41] lb.utils.events INFO: eta: 1:40:53 iteration: 336699/375342 consumed_samples: 344780800 total_loss: 2.959 time: 0.3287 s/iter data_time: 0.2309 s/iter total_throughput: 3114.94 samples/s lr: 3.57e-05 [09/27 16:06:15] lb.utils.events INFO: eta: 1:40:41 iteration: 336799/375342 consumed_samples: 344883200 total_loss: 2.95 time: 0.3287 s/iter data_time: 0.2241 s/iter total_throughput: 3114.92 samples/s lr: 3.55e-05 [09/27 16:06:48] lb.utils.events INFO: eta: 1:41:26 iteration: 336899/375342 consumed_samples: 344985600 total_loss: 2.966 time: 0.3287 s/iter data_time: 0.2288 s/iter total_throughput: 3114.91 samples/s lr: 3.54e-05 [09/27 16:07:21] lb.utils.events INFO: eta: 1:42:09 iteration: 336999/375342 consumed_samples: 345088000 total_loss: 2.983 time: 0.3287 s/iter data_time: 0.2084 s/iter total_throughput: 3114.91 samples/s lr: 3.53e-05 [09/27 16:07:54] lb.utils.events INFO: eta: 1:42:47 iteration: 337099/375342 consumed_samples: 345190400 total_loss: 2.97 time: 0.3287 s/iter data_time: 0.2339 s/iter total_throughput: 3114.89 samples/s lr: 3.51e-05 [09/27 16:08:28] lb.utils.events INFO: eta: 1:42:27 iteration: 337199/375342 consumed_samples: 345292800 total_loss: 2.984 time: 0.3287 s/iter data_time: 0.2111 s/iter total_throughput: 3114.87 samples/s lr: 3.50e-05 [09/27 16:09:01] lb.utils.events INFO: eta: 1:42:19 iteration: 337299/375342 consumed_samples: 345395200 total_loss: 3.004 time: 0.3287 s/iter data_time: 0.2296 s/iter total_throughput: 3114.87 samples/s lr: 3.49e-05 [09/27 16:09:34] lb.utils.events INFO: eta: 1:41:40 iteration: 337399/375342 consumed_samples: 345497600 total_loss: 2.997 time: 0.3287 s/iter data_time: 0.2243 s/iter total_throughput: 3114.87 samples/s lr: 3.48e-05 [09/27 16:10:07] lb.utils.events INFO: eta: 1:41:59 iteration: 337499/375342 consumed_samples: 345600000 total_loss: 2.993 time: 0.3287 s/iter data_time: 0.2342 s/iter total_throughput: 3114.86 samples/s lr: 3.46e-05 [09/27 16:10:41] lb.utils.events INFO: eta: 1:42:44 iteration: 337599/375342 consumed_samples: 345702400 total_loss: 2.983 time: 0.3287 s/iter data_time: 0.2175 s/iter total_throughput: 3114.84 samples/s lr: 3.45e-05 [09/27 16:11:14] lb.utils.events INFO: eta: 1:41:23 iteration: 337699/375342 consumed_samples: 345804800 total_loss: 2.982 time: 0.3288 s/iter data_time: 0.2065 s/iter total_throughput: 3114.82 samples/s lr: 3.44e-05 [09/27 16:11:47] lb.utils.events INFO: eta: 1:40:15 iteration: 337799/375342 consumed_samples: 345907200 total_loss: 2.977 time: 0.3288 s/iter data_time: 0.2143 s/iter total_throughput: 3114.82 samples/s lr: 3.42e-05 [09/27 16:12:21] lb.utils.events INFO: eta: 1:38:44 iteration: 337899/375342 consumed_samples: 346009600 total_loss: 2.977 time: 0.3288 s/iter data_time: 0.2108 s/iter total_throughput: 3114.80 samples/s lr: 3.41e-05 [09/27 16:12:54] lb.utils.events INFO: eta: 1:37:46 iteration: 337999/375342 consumed_samples: 346112000 total_loss: 2.994 time: 0.3288 s/iter data_time: 0.2134 s/iter total_throughput: 3114.80 samples/s lr: 3.40e-05 [09/27 16:13:27] lb.utils.events INFO: eta: 1:36:56 iteration: 338099/375342 consumed_samples: 346214400 total_loss: 3 time: 0.3288 s/iter data_time: 0.2152 s/iter total_throughput: 3114.79 samples/s lr: 3.39e-05 [09/27 16:14:00] lb.utils.events INFO: eta: 1:36:33 iteration: 338199/375342 consumed_samples: 346316800 total_loss: 2.985 time: 0.3288 s/iter data_time: 0.2244 s/iter total_throughput: 3114.78 samples/s lr: 3.37e-05 [09/27 16:14:33] lb.utils.events INFO: eta: 1:36:26 iteration: 338299/375342 consumed_samples: 346419200 total_loss: 2.991 time: 0.3288 s/iter data_time: 0.2081 s/iter total_throughput: 3114.78 samples/s lr: 3.36e-05 [09/27 16:15:06] lb.utils.events INFO: eta: 1:36:16 iteration: 338399/375342 consumed_samples: 346521600 total_loss: 3.017 time: 0.3288 s/iter data_time: 0.2268 s/iter total_throughput: 3114.78 samples/s lr: 3.35e-05 [09/27 16:15:39] lb.utils.events INFO: eta: 1:37:11 iteration: 338499/375342 consumed_samples: 346624000 total_loss: 3.004 time: 0.3288 s/iter data_time: 0.2463 s/iter total_throughput: 3114.78 samples/s lr: 3.33e-05 [09/27 16:16:12] lb.utils.events INFO: eta: 1:37:41 iteration: 338599/375342 consumed_samples: 346726400 total_loss: 2.975 time: 0.3288 s/iter data_time: 0.2375 s/iter total_throughput: 3114.78 samples/s lr: 3.32e-05 [09/27 16:16:45] lb.utils.events INFO: eta: 1:38:15 iteration: 338699/375342 consumed_samples: 346828800 total_loss: 2.976 time: 0.3288 s/iter data_time: 0.2358 s/iter total_throughput: 3114.77 samples/s lr: 3.31e-05 [09/27 16:17:18] lb.utils.events INFO: eta: 1:39:29 iteration: 338799/375342 consumed_samples: 346931200 total_loss: 2.978 time: 0.3288 s/iter data_time: 0.2419 s/iter total_throughput: 3114.77 samples/s lr: 3.30e-05 [09/27 16:17:51] lb.utils.events INFO: eta: 1:42:30 iteration: 338899/375342 consumed_samples: 347033600 total_loss: 2.971 time: 0.3288 s/iter data_time: 0.2375 s/iter total_throughput: 3114.76 samples/s lr: 3.28e-05 [09/27 16:18:24] lb.utils.events INFO: eta: 2:03:09 iteration: 338999/375342 consumed_samples: 347136000 total_loss: 2.965 time: 0.3288 s/iter data_time: 0.2299 s/iter total_throughput: 3114.75 samples/s lr: 3.27e-05 [09/27 16:18:58] lb.utils.events INFO: eta: 2:17:50 iteration: 339099/375342 consumed_samples: 347238400 total_loss: 2.982 time: 0.3288 s/iter data_time: 0.2338 s/iter total_throughput: 3114.74 samples/s lr: 3.26e-05 [09/27 16:19:31] lb.utils.events INFO: eta: 2:16:55 iteration: 339199/375342 consumed_samples: 347340800 total_loss: 2.991 time: 0.3288 s/iter data_time: 0.2120 s/iter total_throughput: 3114.72 samples/s lr: 3.25e-05 [09/27 16:20:04] lb.utils.events INFO: eta: 2:08:39 iteration: 339299/375342 consumed_samples: 347443200 total_loss: 2.989 time: 0.3288 s/iter data_time: 0.2191 s/iter total_throughput: 3114.72 samples/s lr: 3.24e-05 [09/27 16:20:37] lb.utils.events INFO: eta: 2:09:03 iteration: 339399/375342 consumed_samples: 347545600 total_loss: 2.987 time: 0.3288 s/iter data_time: 0.2207 s/iter total_throughput: 3114.71 samples/s lr: 3.22e-05 [09/27 16:21:10] lb.utils.events INFO: eta: 1:46:32 iteration: 339499/375342 consumed_samples: 347648000 total_loss: 2.988 time: 0.3288 s/iter data_time: 0.2207 s/iter total_throughput: 3114.71 samples/s lr: 3.21e-05 [09/27 16:21:43] lb.utils.events INFO: eta: 1:39:49 iteration: 339599/375342 consumed_samples: 347750400 total_loss: 2.993 time: 0.3288 s/iter data_time: 0.2249 s/iter total_throughput: 3114.70 samples/s lr: 3.20e-05 [09/27 16:22:17] lb.utils.events INFO: eta: 1:40:01 iteration: 339699/375342 consumed_samples: 347852800 total_loss: 2.981 time: 0.3288 s/iter data_time: 0.2409 s/iter total_throughput: 3114.69 samples/s lr: 3.19e-05 [09/27 16:22:50] lb.utils.events INFO: eta: 1:38:18 iteration: 339799/375342 consumed_samples: 347955200 total_loss: 2.982 time: 0.3288 s/iter data_time: 0.2196 s/iter total_throughput: 3114.68 samples/s lr: 3.17e-05 [09/27 16:23:23] lb.utils.events INFO: eta: 1:36:59 iteration: 339899/375342 consumed_samples: 348057600 total_loss: 2.983 time: 0.3288 s/iter data_time: 0.2279 s/iter total_throughput: 3114.68 samples/s lr: 3.16e-05 [09/27 16:23:56] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0339999 [09/27 16:23:56] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/27 16:23:56] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/27 16:24:00] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0830 s/iter. Inference: 0.1549 s/iter. Eval: 0.0021 s/iter. Total: 0.2400 s/iter. ETA=0:00:08 [09/27 16:24:06] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1385 s/iter. Inference: 0.1543 s/iter. Eval: 0.0021 s/iter. Total: 0.2950 s/iter. ETA=0:00:05 [09/27 16:24:11] lb.evaluation.evaluator INFO: Inference done 50000/50000. Dataloading: 0.1292 s/iter. Inference: 0.1529 s/iter. Eval: 0.0021 s/iter. Total: 0.2843 s/iter. ETA=0:00:00 [09/27 16:24:11] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/27 16:24:11] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.513054 (0.000250 s / iter per device, on 8 devices) [09/27 16:24:11] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000135 s / iter per device, on 8 devices) [09/27 16:24:11] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/27 16:24:11] lb.evaluation.utils INFO: copypaste: Acc@1=79.84599999999999 [09/27 16:24:11] lb.evaluation.utils INFO: copypaste: Acc@5=94.464 [09/27 16:24:11] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 79.84600, better than last best score 79.74800 @ iteration 334999. [09/27 16:24:11] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/27 16:24:12] lb.utils.events INFO: eta: 1:35:23 iteration: 339999/375342 consumed_samples: 348160000 total_loss: 2.971 time: 0.3288 s/iter data_time: 0.2298 s/iter total_throughput: 3114.68 samples/s lr: 3.15e-05 [09/27 16:24:44] lb.utils.events INFO: eta: 1:35:41 iteration: 340099/375342 consumed_samples: 348262400 total_loss: 2.973 time: 0.3288 s/iter data_time: 0.2482 s/iter total_throughput: 3114.72 samples/s lr: 3.14e-05 [09/27 16:25:17] lb.utils.events INFO: eta: 1:38:12 iteration: 340199/375342 consumed_samples: 348364800 total_loss: 2.982 time: 0.3288 s/iter data_time: 0.2210 s/iter total_throughput: 3114.72 samples/s lr: 3.13e-05 [09/27 16:25:50] lb.utils.events INFO: eta: 1:41:09 iteration: 340299/375342 consumed_samples: 348467200 total_loss: 2.991 time: 0.3288 s/iter data_time: 0.2492 s/iter total_throughput: 3114.71 samples/s lr: 3.11e-05 [09/27 16:26:23] lb.utils.events INFO: eta: 1:41:03 iteration: 340399/375342 consumed_samples: 348569600 total_loss: 2.99 time: 0.3288 s/iter data_time: 0.2361 s/iter total_throughput: 3114.69 samples/s lr: 3.10e-05 [09/27 16:26:56] lb.utils.events INFO: eta: 1:48:33 iteration: 340499/375342 consumed_samples: 348672000 total_loss: 2.97 time: 0.3288 s/iter data_time: 0.2416 s/iter total_throughput: 3114.69 samples/s lr: 3.09e-05 [09/27 16:27:30] lb.utils.events INFO: eta: 1:53:19 iteration: 340599/375342 consumed_samples: 348774400 total_loss: 2.952 time: 0.3288 s/iter data_time: 0.2318 s/iter total_throughput: 3114.68 samples/s lr: 3.08e-05 [09/27 16:28:03] lb.utils.events INFO: eta: 1:42:33 iteration: 340699/375342 consumed_samples: 348876800 total_loss: 2.979 time: 0.3288 s/iter data_time: 0.2264 s/iter total_throughput: 3114.68 samples/s lr: 3.07e-05 [09/27 16:28:36] lb.utils.events INFO: eta: 1:40:16 iteration: 340799/375342 consumed_samples: 348979200 total_loss: 2.988 time: 0.3288 s/iter data_time: 0.2095 s/iter total_throughput: 3114.67 samples/s lr: 3.05e-05 [09/27 16:29:08] lb.utils.events INFO: eta: 1:38:00 iteration: 340899/375342 consumed_samples: 349081600 total_loss: 2.978 time: 0.3288 s/iter data_time: 0.2295 s/iter total_throughput: 3114.67 samples/s lr: 3.04e-05 [09/27 16:29:41] lb.utils.events INFO: eta: 1:39:25 iteration: 340999/375342 consumed_samples: 349184000 total_loss: 2.977 time: 0.3288 s/iter data_time: 0.2350 s/iter total_throughput: 3114.68 samples/s lr: 3.03e-05 [09/27 16:30:14] lb.utils.events INFO: eta: 1:36:32 iteration: 341099/375342 consumed_samples: 349286400 total_loss: 2.981 time: 0.3288 s/iter data_time: 0.2193 s/iter total_throughput: 3114.68 samples/s lr: 3.02e-05 [09/27 16:30:47] lb.utils.events INFO: eta: 1:33:32 iteration: 341199/375342 consumed_samples: 349388800 total_loss: 2.977 time: 0.3288 s/iter data_time: 0.2089 s/iter total_throughput: 3114.67 samples/s lr: 3.01e-05 [09/27 16:31:20] lb.utils.events INFO: eta: 1:31:50 iteration: 341299/375342 consumed_samples: 349491200 total_loss: 2.979 time: 0.3288 s/iter data_time: 0.1991 s/iter total_throughput: 3114.66 samples/s lr: 3.00e-05 [09/27 16:31:54] lb.utils.events INFO: eta: 1:30:24 iteration: 341399/375342 consumed_samples: 349593600 total_loss: 2.973 time: 0.3288 s/iter data_time: 0.2139 s/iter total_throughput: 3114.65 samples/s lr: 2.98e-05 [09/27 16:32:27] lb.utils.events INFO: eta: 1:28:32 iteration: 341499/375342 consumed_samples: 349696000 total_loss: 2.968 time: 0.3288 s/iter data_time: 0.2122 s/iter total_throughput: 3114.64 samples/s lr: 2.97e-05 [09/27 16:33:00] lb.utils.events INFO: eta: 1:27:39 iteration: 341599/375342 consumed_samples: 349798400 total_loss: 2.978 time: 0.3288 s/iter data_time: 0.2215 s/iter total_throughput: 3114.63 samples/s lr: 2.96e-05 [09/27 16:33:33] lb.utils.events INFO: eta: 1:27:09 iteration: 341699/375342 consumed_samples: 349900800 total_loss: 2.959 time: 0.3288 s/iter data_time: 0.2074 s/iter total_throughput: 3114.63 samples/s lr: 2.95e-05 [09/27 16:34:05] lb.utils.events INFO: eta: 1:26:51 iteration: 341799/375342 consumed_samples: 350003200 total_loss: 2.944 time: 0.3288 s/iter data_time: 0.1985 s/iter total_throughput: 3114.65 samples/s lr: 2.94e-05 [09/27 16:34:39] lb.utils.events INFO: eta: 1:26:42 iteration: 341899/375342 consumed_samples: 350105600 total_loss: 2.966 time: 0.3288 s/iter data_time: 0.2373 s/iter total_throughput: 3114.64 samples/s lr: 2.93e-05 [09/27 16:35:12] lb.utils.events INFO: eta: 1:26:28 iteration: 341999/375342 consumed_samples: 350208000 total_loss: 2.983 time: 0.3288 s/iter data_time: 0.2321 s/iter total_throughput: 3114.63 samples/s lr: 2.92e-05 [09/27 16:35:45] lb.utils.events INFO: eta: 1:26:12 iteration: 342099/375342 consumed_samples: 350310400 total_loss: 2.965 time: 0.3288 s/iter data_time: 0.2445 s/iter total_throughput: 3114.62 samples/s lr: 2.90e-05 [09/27 16:36:18] lb.utils.events INFO: eta: 1:26:28 iteration: 342199/375342 consumed_samples: 350412800 total_loss: 2.965 time: 0.3288 s/iter data_time: 0.2273 s/iter total_throughput: 3114.61 samples/s lr: 2.89e-05 [09/27 16:36:51] lb.utils.events INFO: eta: 1:26:57 iteration: 342299/375342 consumed_samples: 350515200 total_loss: 2.96 time: 0.3288 s/iter data_time: 0.2298 s/iter total_throughput: 3114.60 samples/s lr: 2.88e-05 [09/27 16:37:24] lb.utils.events INFO: eta: 1:27:31 iteration: 342399/375342 consumed_samples: 350617600 total_loss: 2.979 time: 0.3288 s/iter data_time: 0.2319 s/iter total_throughput: 3114.61 samples/s lr: 2.87e-05 [09/27 16:37:57] lb.utils.events INFO: eta: 1:29:05 iteration: 342499/375342 consumed_samples: 350720000 total_loss: 2.996 time: 0.3288 s/iter data_time: 0.2427 s/iter total_throughput: 3114.60 samples/s lr: 2.86e-05 [09/27 16:38:30] lb.utils.events INFO: eta: 1:29:24 iteration: 342599/375342 consumed_samples: 350822400 total_loss: 2.991 time: 0.3288 s/iter data_time: 0.2142 s/iter total_throughput: 3114.59 samples/s lr: 2.85e-05 [09/27 16:39:03] lb.utils.events INFO: eta: 1:30:48 iteration: 342699/375342 consumed_samples: 350924800 total_loss: 2.966 time: 0.3288 s/iter data_time: 0.2152 s/iter total_throughput: 3114.59 samples/s lr: 2.84e-05 [09/27 16:39:36] lb.utils.events INFO: eta: 1:38:01 iteration: 342799/375342 consumed_samples: 351027200 total_loss: 2.958 time: 0.3288 s/iter data_time: 0.2200 s/iter total_throughput: 3114.59 samples/s lr: 2.82e-05 [09/27 16:40:10] lb.utils.events INFO: eta: 1:33:44 iteration: 342899/375342 consumed_samples: 351129600 total_loss: 2.947 time: 0.3288 s/iter data_time: 0.2214 s/iter total_throughput: 3114.58 samples/s lr: 2.81e-05 [09/27 16:40:42] lb.utils.events INFO: eta: 1:28:32 iteration: 342999/375342 consumed_samples: 351232000 total_loss: 2.957 time: 0.3288 s/iter data_time: 0.2080 s/iter total_throughput: 3114.58 samples/s lr: 2.80e-05 [09/27 16:41:15] lb.utils.events INFO: eta: 1:26:54 iteration: 343099/375342 consumed_samples: 351334400 total_loss: 2.973 time: 0.3288 s/iter data_time: 0.2297 s/iter total_throughput: 3114.58 samples/s lr: 2.79e-05 [09/27 16:41:48] lb.utils.events INFO: eta: 1:26:38 iteration: 343199/375342 consumed_samples: 351436800 total_loss: 2.973 time: 0.3288 s/iter data_time: 0.2156 s/iter total_throughput: 3114.58 samples/s lr: 2.78e-05 [09/27 16:42:21] lb.utils.events INFO: eta: 1:26:10 iteration: 343299/375342 consumed_samples: 351539200 total_loss: 2.968 time: 0.3288 s/iter data_time: 0.2405 s/iter total_throughput: 3114.57 samples/s lr: 2.77e-05 [09/27 16:42:54] lb.utils.events INFO: eta: 1:28:55 iteration: 343399/375342 consumed_samples: 351641600 total_loss: 2.964 time: 0.3288 s/iter data_time: 0.2554 s/iter total_throughput: 3114.58 samples/s lr: 2.76e-05 [09/27 16:43:27] lb.utils.events INFO: eta: 1:32:49 iteration: 343499/375342 consumed_samples: 351744000 total_loss: 2.966 time: 0.3288 s/iter data_time: 0.2456 s/iter total_throughput: 3114.57 samples/s lr: 2.75e-05 [09/27 16:44:00] lb.utils.events INFO: eta: 1:36:15 iteration: 343599/375342 consumed_samples: 351846400 total_loss: 2.973 time: 0.3288 s/iter data_time: 0.2275 s/iter total_throughput: 3114.57 samples/s lr: 2.74e-05 [09/27 16:44:33] lb.utils.events INFO: eta: 1:43:12 iteration: 343699/375342 consumed_samples: 351948800 total_loss: 2.969 time: 0.3288 s/iter data_time: 0.2613 s/iter total_throughput: 3114.56 samples/s lr: 2.73e-05 [09/27 16:45:06] lb.utils.events INFO: eta: 1:41:36 iteration: 343799/375342 consumed_samples: 352051200 total_loss: 2.968 time: 0.3288 s/iter data_time: 0.2221 s/iter total_throughput: 3114.55 samples/s lr: 2.72e-05 [09/27 16:45:39] lb.utils.events INFO: eta: 1:46:04 iteration: 343899/375342 consumed_samples: 352153600 total_loss: 2.977 time: 0.3288 s/iter data_time: 0.2389 s/iter total_throughput: 3114.56 samples/s lr: 2.70e-05 [09/27 16:46:12] lb.utils.events INFO: eta: 1:49:23 iteration: 343999/375342 consumed_samples: 352256000 total_loss: 2.979 time: 0.3288 s/iter data_time: 0.2314 s/iter total_throughput: 3114.56 samples/s lr: 2.69e-05 [09/27 16:46:45] lb.utils.events INFO: eta: 2:03:17 iteration: 344099/375342 consumed_samples: 352358400 total_loss: 2.967 time: 0.3288 s/iter data_time: 0.2437 s/iter total_throughput: 3114.56 samples/s lr: 2.68e-05 [09/27 16:47:18] lb.utils.events INFO: eta: 2:01:49 iteration: 344199/375342 consumed_samples: 352460800 total_loss: 2.967 time: 0.3288 s/iter data_time: 0.2363 s/iter total_throughput: 3114.56 samples/s lr: 2.67e-05 [09/27 16:47:51] lb.utils.events INFO: eta: 2:03:41 iteration: 344299/375342 consumed_samples: 352563200 total_loss: 2.969 time: 0.3288 s/iter data_time: 0.2369 s/iter total_throughput: 3114.55 samples/s lr: 2.66e-05 [09/27 16:48:24] lb.utils.events INFO: eta: 2:00:00 iteration: 344399/375342 consumed_samples: 352665600 total_loss: 2.962 time: 0.3288 s/iter data_time: 0.2192 s/iter total_throughput: 3114.54 samples/s lr: 2.65e-05 [09/27 16:48:57] lb.utils.events INFO: eta: 1:35:15 iteration: 344499/375342 consumed_samples: 352768000 total_loss: 2.964 time: 0.3288 s/iter data_time: 0.2283 s/iter total_throughput: 3114.55 samples/s lr: 2.64e-05 [09/27 16:49:30] lb.utils.events INFO: eta: 1:37:19 iteration: 344599/375342 consumed_samples: 352870400 total_loss: 2.973 time: 0.3288 s/iter data_time: 0.2332 s/iter total_throughput: 3114.54 samples/s lr: 2.63e-05 [09/27 16:50:03] lb.utils.events INFO: eta: 1:35:24 iteration: 344699/375342 consumed_samples: 352972800 total_loss: 2.986 time: 0.3288 s/iter data_time: 0.2191 s/iter total_throughput: 3114.54 samples/s lr: 2.62e-05 [09/27 16:50:36] lb.utils.events INFO: eta: 1:27:33 iteration: 344799/375342 consumed_samples: 353075200 total_loss: 2.987 time: 0.3288 s/iter data_time: 0.2084 s/iter total_throughput: 3114.53 samples/s lr: 2.61e-05 [09/27 16:51:09] lb.utils.events INFO: eta: 1:25:13 iteration: 344899/375342 consumed_samples: 353177600 total_loss: 2.999 time: 0.3288 s/iter data_time: 0.2199 s/iter total_throughput: 3114.54 samples/s lr: 2.60e-05 [09/27 16:51:42] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0344999 [09/27 16:51:43] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/27 16:51:43] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/27 16:51:47] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0919 s/iter. Inference: 0.1515 s/iter. Eval: 0.0024 s/iter. Total: 0.2458 s/iter. ETA=0:00:09 [09/27 16:51:53] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1492 s/iter. Inference: 0.1528 s/iter. Eval: 0.0023 s/iter. Total: 0.3043 s/iter. ETA=0:00:05 [09/27 16:51:58] lb.evaluation.evaluator INFO: Inference done 50000/50000. Dataloading: 0.1361 s/iter. Inference: 0.1513 s/iter. Eval: 0.0022 s/iter. Total: 0.2896 s/iter. ETA=0:00:00 [09/27 16:51:58] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/27 16:51:58] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.747394 (0.000255 s / iter per device, on 8 devices) [09/27 16:51:58] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000133 s / iter per device, on 8 devices) [09/27 16:51:58] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/27 16:51:58] lb.evaluation.utils INFO: copypaste: Acc@1=79.784 [09/27 16:51:58] lb.evaluation.utils INFO: copypaste: Acc@5=94.49600000000001 [09/27 16:51:58] lb.engine.hooks INFO: Not saving as latest eval score for Acc@1 is 79.78400, not better than best score 79.84600 @ iteration 339999. [09/27 16:51:58] lb.utils.events INFO: eta: 1:25:09 iteration: 344999/375342 consumed_samples: 353280000 total_loss: 2.981 time: 0.3288 s/iter data_time: 0.2241 s/iter total_throughput: 3114.53 samples/s lr: 2.59e-05 [09/27 16:52:29] lb.utils.events INFO: eta: 1:21:58 iteration: 345099/375342 consumed_samples: 353382400 total_loss: 2.97 time: 0.3288 s/iter data_time: 0.2156 s/iter total_throughput: 3114.57 samples/s lr: 2.58e-05 [09/27 16:53:02] lb.utils.events INFO: eta: 1:21:02 iteration: 345199/375342 consumed_samples: 353484800 total_loss: 2.973 time: 0.3288 s/iter data_time: 0.2189 s/iter total_throughput: 3114.57 samples/s lr: 2.57e-05 [09/27 16:53:35] lb.utils.events INFO: eta: 1:20:01 iteration: 345299/375342 consumed_samples: 353587200 total_loss: 2.976 time: 0.3288 s/iter data_time: 0.2331 s/iter total_throughput: 3114.58 samples/s lr: 2.56e-05 [09/27 16:54:08] lb.utils.events INFO: eta: 1:20:53 iteration: 345399/375342 consumed_samples: 353689600 total_loss: 2.959 time: 0.3288 s/iter data_time: 0.2318 s/iter total_throughput: 3114.56 samples/s lr: 2.55e-05 [09/27 16:54:42] lb.utils.events INFO: eta: 1:20:53 iteration: 345499/375342 consumed_samples: 353792000 total_loss: 2.969 time: 0.3288 s/iter data_time: 0.2217 s/iter total_throughput: 3114.56 samples/s lr: 2.54e-05 [09/27 16:55:15] lb.utils.events INFO: eta: 1:19:12 iteration: 345599/375342 consumed_samples: 353894400 total_loss: 2.978 time: 0.3288 s/iter data_time: 0.2273 s/iter total_throughput: 3114.55 samples/s lr: 2.53e-05 [09/27 16:55:48] lb.utils.events INFO: eta: 1:18:20 iteration: 345699/375342 consumed_samples: 353996800 total_loss: 2.978 time: 0.3288 s/iter data_time: 0.2164 s/iter total_throughput: 3114.54 samples/s lr: 2.52e-05 [09/27 16:56:21] lb.utils.events INFO: eta: 1:18:18 iteration: 345799/375342 consumed_samples: 354099200 total_loss: 2.971 time: 0.3288 s/iter data_time: 0.2243 s/iter total_throughput: 3114.54 samples/s lr: 2.51e-05 [09/27 16:56:54] lb.utils.events INFO: eta: 1:17:49 iteration: 345899/375342 consumed_samples: 354201600 total_loss: 2.952 time: 0.3288 s/iter data_time: 0.2125 s/iter total_throughput: 3114.54 samples/s lr: 2.50e-05 [09/27 16:57:27] lb.utils.events INFO: eta: 1:17:00 iteration: 345999/375342 consumed_samples: 354304000 total_loss: 2.955 time: 0.3288 s/iter data_time: 0.2122 s/iter total_throughput: 3114.54 samples/s lr: 2.49e-05 [09/27 16:57:59] lb.utils.events INFO: eta: 1:16:42 iteration: 346099/375342 consumed_samples: 354406400 total_loss: 2.971 time: 0.3288 s/iter data_time: 0.2066 s/iter total_throughput: 3114.54 samples/s lr: 2.48e-05 [09/27 16:58:33] lb.utils.events INFO: eta: 1:16:24 iteration: 346199/375342 consumed_samples: 354508800 total_loss: 2.984 time: 0.3288 s/iter data_time: 0.2405 s/iter total_throughput: 3114.53 samples/s lr: 2.47e-05 [09/27 16:59:05] lb.utils.events INFO: eta: 1:16:08 iteration: 346299/375342 consumed_samples: 354611200 total_loss: 2.975 time: 0.3288 s/iter data_time: 0.2230 s/iter total_throughput: 3114.54 samples/s lr: 2.46e-05 [09/27 16:59:38] lb.utils.events INFO: eta: 1:15:46 iteration: 346399/375342 consumed_samples: 354713600 total_loss: 2.969 time: 0.3288 s/iter data_time: 0.2244 s/iter total_throughput: 3114.54 samples/s lr: 2.45e-05 [09/27 17:00:12] lb.utils.events INFO: eta: 1:15:57 iteration: 346499/375342 consumed_samples: 354816000 total_loss: 2.968 time: 0.3288 s/iter data_time: 0.2173 s/iter total_throughput: 3114.52 samples/s lr: 2.44e-05 [09/27 17:00:45] lb.utils.events INFO: eta: 1:16:01 iteration: 346599/375342 consumed_samples: 354918400 total_loss: 2.961 time: 0.3288 s/iter data_time: 0.2265 s/iter total_throughput: 3114.52 samples/s lr: 2.43e-05 [09/27 17:01:21] lb.utils.events INFO: eta: 1:18:03 iteration: 346699/375342 consumed_samples: 355020800 total_loss: 2.964 time: 0.3288 s/iter data_time: 0.2529 s/iter total_throughput: 3114.44 samples/s lr: 2.42e-05 [09/27 17:03:03] lb.utils.events INFO: eta: 1:25:21 iteration: 346799/375342 consumed_samples: 355123200 total_loss: 2.972 time: 0.3290 s/iter data_time: 0.0062 s/iter total_throughput: 3112.53 samples/s lr: 2.41e-05 [09/27 17:04:49] lb.utils.events INFO: eta: 1:50:47 iteration: 346899/375342 consumed_samples: 355225600 total_loss: 2.971 time: 0.3292 s/iter data_time: 0.0084 s/iter total_throughput: 3110.53 samples/s lr: 2.40e-05 [09/27 17:06:23] lb.utils.events INFO: eta: 2:19:17 iteration: 346999/375342 consumed_samples: 355328000 total_loss: 2.969 time: 0.3294 s/iter data_time: 0.0066 s/iter total_throughput: 3108.89 samples/s lr: 2.39e-05 [09/27 17:08:03] lb.utils.events INFO: eta: 3:00:26 iteration: 347099/375342 consumed_samples: 355430400 total_loss: 2.936 time: 0.3296 s/iter data_time: 0.0066 s/iter total_throughput: 3107.05 samples/s lr: 2.38e-05 [09/27 17:09:45] lb.utils.events INFO: eta: 3:48:16 iteration: 347199/375342 consumed_samples: 355532800 total_loss: 2.936 time: 0.3298 s/iter data_time: 0.0066 s/iter total_throughput: 3105.20 samples/s lr: 2.37e-05 [09/27 17:11:32] lb.utils.events INFO: eta: 4:40:43 iteration: 347299/375342 consumed_samples: 355635200 total_loss: 2.949 time: 0.3300 s/iter data_time: 0.0068 s/iter total_throughput: 3103.19 samples/s lr: 2.36e-05 [09/27 17:13:09] lb.utils.events INFO: eta: 5:11:46 iteration: 347399/375342 consumed_samples: 355737600 total_loss: 2.958 time: 0.3302 s/iter data_time: 0.0069 s/iter total_throughput: 3101.44 samples/s lr: 2.35e-05 [09/27 17:14:51] lb.utils.events INFO: eta: 5:37:23 iteration: 347499/375342 consumed_samples: 355840000 total_loss: 2.947 time: 0.3304 s/iter data_time: 0.0056 s/iter total_throughput: 3099.59 samples/s lr: 2.34e-05 [09/27 17:16:35] lb.utils.events INFO: eta: 6:03:46 iteration: 347599/375342 consumed_samples: 355942400 total_loss: 2.957 time: 0.3306 s/iter data_time: 0.0083 s/iter total_throughput: 3097.67 samples/s lr: 2.33e-05 [09/27 17:18:10] lb.utils.events INFO: eta: 6:43:57 iteration: 347699/375342 consumed_samples: 356044800 total_loss: 2.972 time: 0.3307 s/iter data_time: 0.0082 s/iter total_throughput: 3096.00 samples/s lr: 2.32e-05 [09/27 17:19:47] lb.utils.events INFO: eta: 6:37:38 iteration: 347799/375342 consumed_samples: 356147200 total_loss: 2.965 time: 0.3309 s/iter data_time: 0.0078 s/iter total_throughput: 3094.30 samples/s lr: 2.31e-05 [09/27 17:21:28] lb.utils.events INFO: eta: 6:30:53 iteration: 347899/375342 consumed_samples: 356249600 total_loss: 2.977 time: 0.3311 s/iter data_time: 0.0075 s/iter total_throughput: 3092.46 samples/s lr: 2.30e-05 [09/27 17:23:14] lb.utils.events INFO: eta: 6:43:44 iteration: 347999/375342 consumed_samples: 356352000 total_loss: 2.958 time: 0.3313 s/iter data_time: 0.0059 s/iter total_throughput: 3090.51 samples/s lr: 2.29e-05 [09/27 17:24:50] lb.utils.events INFO: eta: 6:37:40 iteration: 348099/375342 consumed_samples: 356454400 total_loss: 2.941 time: 0.3315 s/iter data_time: 0.0075 s/iter total_throughput: 3088.83 samples/s lr: 2.28e-05 [09/27 17:26:26] lb.utils.events INFO: eta: 6:29:10 iteration: 348199/375342 consumed_samples: 356556800 total_loss: 2.961 time: 0.3317 s/iter data_time: 0.0069 s/iter total_throughput: 3087.13 samples/s lr: 2.27e-05 [09/27 17:27:59] lb.utils.events INFO: eta: 6:18:12 iteration: 348299/375342 consumed_samples: 356659200 total_loss: 2.969 time: 0.3319 s/iter data_time: 0.0058 s/iter total_throughput: 3085.55 samples/s lr: 2.26e-05 [09/27 17:29:38] lb.utils.events INFO: eta: 6:15:12 iteration: 348399/375342 consumed_samples: 356761600 total_loss: 2.984 time: 0.3321 s/iter data_time: 0.0089 s/iter total_throughput: 3083.80 samples/s lr: 2.25e-05 [09/27 17:31:11] lb.utils.events INFO: eta: 6:09:54 iteration: 348499/375342 consumed_samples: 356864000 total_loss: 2.96 time: 0.3322 s/iter data_time: 0.0067 s/iter total_throughput: 3082.20 samples/s lr: 2.24e-05 [09/27 17:32:52] lb.utils.events INFO: eta: 6:07:11 iteration: 348599/375342 consumed_samples: 356966400 total_loss: 2.971 time: 0.3324 s/iter data_time: 0.0062 s/iter total_throughput: 3080.39 samples/s lr: 2.23e-05 [09/27 17:34:38] lb.utils.events INFO: eta: 6:07:37 iteration: 348699/375342 consumed_samples: 357068800 total_loss: 2.97 time: 0.3326 s/iter data_time: 0.0073 s/iter total_throughput: 3078.48 samples/s lr: 2.23e-05 [09/27 17:36:15] lb.utils.events INFO: eta: 6:05:21 iteration: 348799/375342 consumed_samples: 357171200 total_loss: 2.944 time: 0.3328 s/iter data_time: 0.0067 s/iter total_throughput: 3076.77 samples/s lr: 2.22e-05 [09/27 17:37:52] lb.utils.events INFO: eta: 6:03:27 iteration: 348899/375342 consumed_samples: 357273600 total_loss: 2.961 time: 0.3330 s/iter data_time: 0.0064 s/iter total_throughput: 3075.08 samples/s lr: 2.21e-05 [09/27 17:39:30] lb.utils.events INFO: eta: 5:55:04 iteration: 348999/375342 consumed_samples: 357376000 total_loss: 2.965 time: 0.3332 s/iter data_time: 0.0070 s/iter total_throughput: 3073.38 samples/s lr: 2.20e-05 [09/27 17:41:07] lb.utils.events INFO: eta: 5:55:16 iteration: 349099/375342 consumed_samples: 357478400 total_loss: 2.949 time: 0.3334 s/iter data_time: 0.0063 s/iter total_throughput: 3071.71 samples/s lr: 2.19e-05 [09/27 17:42:18] lb.utils.events INFO: eta: 5:42:44 iteration: 349199/375342 consumed_samples: 357580800 total_loss: 2.945 time: 0.3335 s/iter data_time: 0.1415 s/iter total_throughput: 3070.70 samples/s lr: 2.18e-05 [09/27 17:43:34] lb.utils.events INFO: eta: 5:36:58 iteration: 349299/375342 consumed_samples: 357683200 total_loss: 2.946 time: 0.3336 s/iter data_time: 0.0071 s/iter total_throughput: 3069.57 samples/s lr: 2.17e-05 [09/27 17:45:15] lb.utils.events INFO: eta: 5:36:41 iteration: 349399/375342 consumed_samples: 357785600 total_loss: 2.966 time: 0.3338 s/iter data_time: 0.0063 s/iter total_throughput: 3067.79 samples/s lr: 2.16e-05 [09/27 17:46:40] lb.utils.events INFO: eta: 5:33:04 iteration: 349499/375342 consumed_samples: 357888000 total_loss: 2.973 time: 0.3339 s/iter data_time: 0.0060 s/iter total_throughput: 3066.45 samples/s lr: 2.15e-05 [09/27 17:48:21] lb.utils.events INFO: eta: 5:30:26 iteration: 349599/375342 consumed_samples: 357990400 total_loss: 2.955 time: 0.3341 s/iter data_time: 0.0070 s/iter total_throughput: 3064.67 samples/s lr: 2.14e-05 [09/27 17:50:08] lb.utils.events INFO: eta: 5:34:00 iteration: 349699/375342 consumed_samples: 358092800 total_loss: 2.951 time: 0.3343 s/iter data_time: 0.0064 s/iter total_throughput: 3062.75 samples/s lr: 2.14e-05 [09/27 17:51:46] lb.utils.events INFO: eta: 5:28:48 iteration: 349799/375342 consumed_samples: 358195200 total_loss: 2.975 time: 0.3345 s/iter data_time: 0.0075 s/iter total_throughput: 3061.04 samples/s lr: 2.13e-05 [09/27 17:53:38] lb.utils.events INFO: eta: 5:34:20 iteration: 349899/375342 consumed_samples: 358297600 total_loss: 2.975 time: 0.3347 s/iter data_time: 0.0070 s/iter total_throughput: 3059.01 samples/s lr: 2.12e-05 [09/27 17:55:12] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0349999 [09/27 17:55:14] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/27 17:55:14] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/27 17:55:19] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0004 s/iter. Inference: 0.3719 s/iter. Eval: 0.0055 s/iter. Total: 0.3777 s/iter. ETA=0:00:13 [09/27 17:55:25] lb.evaluation.evaluator INFO: Inference done 25600/50000. Dataloading: 0.0006 s/iter. Inference: 0.3720 s/iter. Eval: 0.0066 s/iter. Total: 0.3792 s/iter. ETA=0:00:08 [09/27 17:55:30] lb.evaluation.evaluator INFO: Inference done 39936/50000. Dataloading: 0.0005 s/iter. Inference: 0.3697 s/iter. Eval: 0.0059 s/iter. Total: 0.3763 s/iter. ETA=0:00:03 [09/27 17:55:34] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/27 17:55:34] lb.evaluation.evaluator INFO: Total inference time: 0:00:16.772346 (0.000335 s / iter per device, on 8 devices) [09/27 17:55:34] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:16 (0.000330 s / iter per device, on 8 devices) [09/27 17:55:34] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/27 17:55:34] lb.evaluation.utils INFO: copypaste: Acc@1=79.946 [09/27 17:55:34] lb.evaluation.utils INFO: copypaste: Acc@5=94.46799999999999 [09/27 17:55:34] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 79.94600, better than last best score 79.84600 @ iteration 339999. [09/27 17:55:34] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/27 17:55:35] lb.utils.events INFO: eta: 5:31:21 iteration: 349999/375342 consumed_samples: 358400000 total_loss: 2.955 time: 0.3349 s/iter data_time: 0.0054 s/iter total_throughput: 3057.43 samples/s lr: 2.11e-05 [09/27 17:57:23] lb.utils.events INFO: eta: 5:31:43 iteration: 350099/375342 consumed_samples: 358502400 total_loss: 2.953 time: 0.3351 s/iter data_time: 0.0066 s/iter total_throughput: 3055.49 samples/s lr: 2.10e-05 [09/27 17:59:07] lb.utils.events INFO: eta: 5:56:12 iteration: 350199/375342 consumed_samples: 358604800 total_loss: 2.967 time: 0.3353 s/iter data_time: 0.0069 s/iter total_throughput: 3053.67 samples/s lr: 2.09e-05 [09/27 18:00:49] lb.utils.events INFO: eta: 6:08:19 iteration: 350299/375342 consumed_samples: 358707200 total_loss: 2.959 time: 0.3355 s/iter data_time: 0.0078 s/iter total_throughput: 3051.89 samples/s lr: 2.08e-05 [09/27 18:02:24] lb.utils.events INFO: eta: 6:00:10 iteration: 350399/375342 consumed_samples: 358809600 total_loss: 2.949 time: 0.3357 s/iter data_time: 0.0072 s/iter total_throughput: 3050.30 samples/s lr: 2.07e-05 [09/27 18:03:59] lb.utils.events INFO: eta: 6:02:59 iteration: 350499/375342 consumed_samples: 358912000 total_loss: 2.954 time: 0.3359 s/iter data_time: 0.0055 s/iter total_throughput: 3048.71 samples/s lr: 2.07e-05 [09/27 18:05:37] lb.utils.events INFO: eta: 5:55:43 iteration: 350599/375342 consumed_samples: 359014400 total_loss: 2.955 time: 0.3361 s/iter data_time: 0.0070 s/iter total_throughput: 3047.02 samples/s lr: 2.06e-05 [09/27 18:07:20] lb.utils.events INFO: eta: 5:50:00 iteration: 350699/375342 consumed_samples: 359116800 total_loss: 2.965 time: 0.3363 s/iter data_time: 0.0071 s/iter total_throughput: 3045.24 samples/s lr: 2.05e-05 [09/27 18:08:59] lb.utils.events INFO: eta: 5:49:24 iteration: 350799/375342 consumed_samples: 359219200 total_loss: 2.965 time: 0.3364 s/iter data_time: 0.0077 s/iter total_throughput: 3043.56 samples/s lr: 2.04e-05 [09/27 18:10:36] lb.utils.events INFO: eta: 5:38:02 iteration: 350899/375342 consumed_samples: 359321600 total_loss: 2.959 time: 0.3366 s/iter data_time: 0.0082 s/iter total_throughput: 3041.92 samples/s lr: 2.03e-05 [09/27 18:12:17] lb.utils.events INFO: eta: 5:37:45 iteration: 350999/375342 consumed_samples: 359424000 total_loss: 2.962 time: 0.3368 s/iter data_time: 0.0077 s/iter total_throughput: 3040.20 samples/s lr: 2.02e-05 [09/27 18:13:52] lb.utils.events INFO: eta: 5:32:49 iteration: 351099/375342 consumed_samples: 359526400 total_loss: 2.953 time: 0.3370 s/iter data_time: 0.0053 s/iter total_throughput: 3038.60 samples/s lr: 2.02e-05 [09/27 18:15:30] lb.utils.events INFO: eta: 5:30:19 iteration: 351199/375342 consumed_samples: 359628800 total_loss: 2.962 time: 0.3372 s/iter data_time: 0.0063 s/iter total_throughput: 3036.97 samples/s lr: 2.01e-05 [09/27 18:17:07] lb.utils.events INFO: eta: 5:24:51 iteration: 351299/375342 consumed_samples: 359731200 total_loss: 2.968 time: 0.3374 s/iter data_time: 0.0075 s/iter total_throughput: 3035.34 samples/s lr: 2.00e-05 [09/27 18:18:46] lb.utils.events INFO: eta: 5:26:13 iteration: 351399/375342 consumed_samples: 359833600 total_loss: 2.955 time: 0.3375 s/iter data_time: 0.0060 s/iter total_throughput: 3033.67 samples/s lr: 1.99e-05 [09/27 18:20:31] lb.utils.events INFO: eta: 5:27:39 iteration: 351499/375342 consumed_samples: 359936000 total_loss: 2.975 time: 0.3377 s/iter data_time: 0.0086 s/iter total_throughput: 3031.86 samples/s lr: 1.98e-05 [09/27 18:22:17] lb.utils.events INFO: eta: 5:31:46 iteration: 351599/375342 consumed_samples: 360038400 total_loss: 2.961 time: 0.3380 s/iter data_time: 0.0063 s/iter total_throughput: 3030.01 samples/s lr: 1.97e-05 [09/27 18:23:13] lb.utils.events INFO: eta: 5:12:17 iteration: 351699/375342 consumed_samples: 360140800 total_loss: 2.952 time: 0.3380 s/iter data_time: 0.2463 s/iter total_throughput: 3029.45 samples/s lr: 1.97e-05 [09/27 18:24:50] lb.utils.events INFO: eta: 5:08:15 iteration: 351799/375342 consumed_samples: 360243200 total_loss: 2.934 time: 0.3382 s/iter data_time: 0.0059 s/iter total_throughput: 3027.84 samples/s lr: 1.96e-05 [09/27 18:26:39] lb.utils.events INFO: eta: 5:16:07 iteration: 351899/375342 consumed_samples: 360345600 total_loss: 2.942 time: 0.3384 s/iter data_time: 0.0056 s/iter total_throughput: 3025.93 samples/s lr: 1.95e-05 [09/27 18:28:26] lb.utils.events INFO: eta: 5:19:00 iteration: 351999/375342 consumed_samples: 360448000 total_loss: 2.973 time: 0.3386 s/iter data_time: 0.0070 s/iter total_throughput: 3024.08 samples/s lr: 1.94e-05 [09/27 18:30:13] lb.utils.events INFO: eta: 5:25:49 iteration: 352099/375342 consumed_samples: 360550400 total_loss: 2.966 time: 0.3388 s/iter data_time: 0.0064 s/iter total_throughput: 3022.22 samples/s lr: 1.93e-05 [09/27 18:31:56] lb.utils.events INFO: eta: 5:22:58 iteration: 352199/375342 consumed_samples: 360652800 total_loss: 2.957 time: 0.3390 s/iter data_time: 0.0069 s/iter total_throughput: 3020.48 samples/s lr: 1.93e-05 [09/27 18:33:36] lb.utils.events INFO: eta: 5:31:00 iteration: 352299/375342 consumed_samples: 360755200 total_loss: 2.945 time: 0.3392 s/iter data_time: 0.0079 s/iter total_throughput: 3018.79 samples/s lr: 1.92e-05 [09/27 18:35:07] lb.utils.events INFO: eta: 5:23:02 iteration: 352399/375342 consumed_samples: 360857600 total_loss: 2.953 time: 0.3394 s/iter data_time: 0.0111 s/iter total_throughput: 3017.35 samples/s lr: 1.91e-05 [09/27 18:36:41] lb.utils.events INFO: eta: 5:18:43 iteration: 352499/375342 consumed_samples: 360960000 total_loss: 2.961 time: 0.3395 s/iter data_time: 0.0061 s/iter total_throughput: 3015.84 samples/s lr: 1.90e-05 [09/27 18:38:13] lb.utils.events INFO: eta: 5:11:04 iteration: 352599/375342 consumed_samples: 361062400 total_loss: 2.976 time: 0.3397 s/iter data_time: 0.0081 s/iter total_throughput: 3014.38 samples/s lr: 1.89e-05 [09/27 18:39:54] lb.utils.events INFO: eta: 5:35:21 iteration: 352699/375342 consumed_samples: 361164800 total_loss: 2.945 time: 0.3399 s/iter data_time: 0.0071 s/iter total_throughput: 3012.70 samples/s lr: 1.89e-05 [09/27 18:41:29] lb.utils.events INFO: eta: 5:35:44 iteration: 352799/375342 consumed_samples: 361267200 total_loss: 2.938 time: 0.3401 s/iter data_time: 0.0067 s/iter total_throughput: 3011.15 samples/s lr: 1.88e-05 [09/27 18:43:12] lb.utils.events INFO: eta: 5:31:23 iteration: 352899/375342 consumed_samples: 361369600 total_loss: 2.938 time: 0.3403 s/iter data_time: 0.0058 s/iter total_throughput: 3009.42 samples/s lr: 1.87e-05 [09/27 18:44:58] lb.utils.events INFO: eta: 5:26:05 iteration: 352999/375342 consumed_samples: 361472000 total_loss: 2.938 time: 0.3405 s/iter data_time: 0.0071 s/iter total_throughput: 3007.63 samples/s lr: 1.86e-05 [09/27 18:46:35] lb.utils.events INFO: eta: 5:17:58 iteration: 353099/375342 consumed_samples: 361574400 total_loss: 2.966 time: 0.3406 s/iter data_time: 0.0074 s/iter total_throughput: 3006.05 samples/s lr: 1.86e-05 [09/27 18:48:23] lb.utils.events INFO: eta: 5:21:34 iteration: 353199/375342 consumed_samples: 361676800 total_loss: 2.96 time: 0.3409 s/iter data_time: 0.0072 s/iter total_throughput: 3004.20 samples/s lr: 1.85e-05 [09/27 18:49:57] lb.utils.events INFO: eta: 5:09:36 iteration: 353299/375342 consumed_samples: 361779200 total_loss: 2.947 time: 0.3410 s/iter data_time: 0.0061 s/iter total_throughput: 3002.71 samples/s lr: 1.84e-05 [09/27 18:51:35] lb.utils.events INFO: eta: 5:13:09 iteration: 353399/375342 consumed_samples: 361881600 total_loss: 2.959 time: 0.3412 s/iter data_time: 0.0061 s/iter total_throughput: 3001.13 samples/s lr: 1.83e-05 [09/27 18:53:17] lb.utils.events INFO: eta: 5:11:32 iteration: 353499/375342 consumed_samples: 361984000 total_loss: 2.951 time: 0.3414 s/iter data_time: 0.0080 s/iter total_throughput: 2999.44 samples/s lr: 1.82e-05 [09/27 18:54:53] lb.utils.events INFO: eta: 5:09:53 iteration: 353599/375342 consumed_samples: 362086400 total_loss: 2.951 time: 0.3416 s/iter data_time: 0.0058 s/iter total_throughput: 2997.90 samples/s lr: 1.82e-05 [09/27 18:56:30] lb.utils.events INFO: eta: 5:08:28 iteration: 353699/375342 consumed_samples: 362188800 total_loss: 2.945 time: 0.3417 s/iter data_time: 0.0070 s/iter total_throughput: 2996.35 samples/s lr: 1.81e-05 [09/27 18:58:14] lb.utils.events INFO: eta: 5:09:42 iteration: 353799/375342 consumed_samples: 362291200 total_loss: 2.927 time: 0.3419 s/iter data_time: 0.0070 s/iter total_throughput: 2994.60 samples/s lr: 1.80e-05 [09/27 19:00:08] lb.utils.events INFO: eta: 5:09:17 iteration: 353899/375342 consumed_samples: 362393600 total_loss: 2.938 time: 0.3422 s/iter data_time: 0.0068 s/iter total_throughput: 2992.65 samples/s lr: 1.80e-05 [09/27 19:01:53] lb.utils.events INFO: eta: 5:11:00 iteration: 353999/375342 consumed_samples: 362496000 total_loss: 2.953 time: 0.3424 s/iter data_time: 0.0060 s/iter total_throughput: 2990.89 samples/s lr: 1.79e-05 [09/27 19:03:22] lb.utils.events INFO: eta: 5:03:31 iteration: 354099/375342 consumed_samples: 362598400 total_loss: 2.967 time: 0.3425 s/iter data_time: 0.0102 s/iter total_throughput: 2989.55 samples/s lr: 1.78e-05 [09/27 19:04:39] lb.utils.events INFO: eta: 4:45:00 iteration: 354199/375342 consumed_samples: 362700800 total_loss: 2.96 time: 0.3426 s/iter data_time: 0.1486 s/iter total_throughput: 2988.48 samples/s lr: 1.77e-05 [09/27 19:05:54] lb.utils.events INFO: eta: 4:37:29 iteration: 354299/375342 consumed_samples: 362803200 total_loss: 2.94 time: 0.3428 s/iter data_time: 0.0073 s/iter total_throughput: 2987.48 samples/s lr: 1.77e-05 [09/27 19:07:45] lb.utils.events INFO: eta: 4:41:54 iteration: 354399/375342 consumed_samples: 362905600 total_loss: 2.93 time: 0.3430 s/iter data_time: 0.0072 s/iter total_throughput: 2985.60 samples/s lr: 1.76e-05 [09/27 19:09:28] lb.utils.events INFO: eta: 4:41:53 iteration: 354499/375342 consumed_samples: 363008000 total_loss: 2.944 time: 0.3432 s/iter data_time: 0.0078 s/iter total_throughput: 2983.91 samples/s lr: 1.75e-05 [09/27 19:11:12] lb.utils.events INFO: eta: 4:45:56 iteration: 354599/375342 consumed_samples: 363110400 total_loss: 2.965 time: 0.3434 s/iter data_time: 0.0079 s/iter total_throughput: 2982.21 samples/s lr: 1.74e-05 [09/27 19:12:51] lb.utils.events INFO: eta: 4:46:32 iteration: 354699/375342 consumed_samples: 363212800 total_loss: 2.957 time: 0.3436 s/iter data_time: 0.0075 s/iter total_throughput: 2980.62 samples/s lr: 1.74e-05 [09/27 19:14:30] lb.utils.events INFO: eta: 4:37:41 iteration: 354799/375342 consumed_samples: 363315200 total_loss: 2.96 time: 0.3437 s/iter data_time: 0.0071 s/iter total_throughput: 2979.04 samples/s lr: 1.73e-05 [09/27 19:16:09] lb.utils.events INFO: eta: 4:32:24 iteration: 354899/375342 consumed_samples: 363417600 total_loss: 2.953 time: 0.3439 s/iter data_time: 0.0056 s/iter total_throughput: 2977.47 samples/s lr: 1.72e-05 [09/27 19:17:53] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0354999 [09/27 19:17:55] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/27 19:17:55] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/27 19:18:00] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0003 s/iter. Inference: 0.3842 s/iter. Eval: 0.0054 s/iter. Total: 0.3899 s/iter. ETA=0:00:14 [09/27 19:18:06] lb.evaluation.evaluator INFO: Inference done 25600/50000. Dataloading: 0.0004 s/iter. Inference: 0.3716 s/iter. Eval: 0.0046 s/iter. Total: 0.3768 s/iter. ETA=0:00:08 [09/27 19:18:11] lb.evaluation.evaluator INFO: Inference done 38912/50000. Dataloading: 0.0005 s/iter. Inference: 0.3758 s/iter. Eval: 0.0049 s/iter. Total: 0.3812 s/iter. ETA=0:00:03 [09/27 19:18:15] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/27 19:18:15] lb.evaluation.evaluator INFO: Total inference time: 0:00:17.042922 (0.000341 s / iter per device, on 8 devices) [09/27 19:18:15] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:16 (0.000336 s / iter per device, on 8 devices) [09/27 19:18:15] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/27 19:18:15] lb.evaluation.utils INFO: copypaste: Acc@1=80.014 [09/27 19:18:15] lb.evaluation.utils INFO: copypaste: Acc@5=94.574 [09/27 19:18:15] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 80.01400, better than last best score 79.94600 @ iteration 349999. [09/27 19:18:15] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/27 19:18:17] lb.utils.events INFO: eta: 4:29:54 iteration: 354999/375342 consumed_samples: 363520000 total_loss: 2.945 time: 0.3441 s/iter data_time: 0.0063 s/iter total_throughput: 2975.79 samples/s lr: 1.72e-05 [09/27 19:20:00] lb.utils.events INFO: eta: 4:33:57 iteration: 355099/375342 consumed_samples: 363622400 total_loss: 2.96 time: 0.3443 s/iter data_time: 0.0078 s/iter total_throughput: 2974.12 samples/s lr: 1.71e-05 [09/27 19:21:35] lb.utils.events INFO: eta: 4:42:40 iteration: 355199/375342 consumed_samples: 363724800 total_loss: 2.961 time: 0.3445 s/iter data_time: 0.0055 s/iter total_throughput: 2972.63 samples/s lr: 1.70e-05 [09/27 19:23:12] lb.utils.events INFO: eta: 4:54:31 iteration: 355299/375342 consumed_samples: 363827200 total_loss: 2.961 time: 0.3447 s/iter data_time: 0.0075 s/iter total_throughput: 2971.13 samples/s lr: 1.69e-05 [09/27 19:25:00] lb.utils.events INFO: eta: 4:49:23 iteration: 355399/375342 consumed_samples: 363929600 total_loss: 2.942 time: 0.3449 s/iter data_time: 0.0078 s/iter total_throughput: 2969.34 samples/s lr: 1.69e-05 [09/27 19:26:39] lb.utils.events INFO: eta: 4:45:03 iteration: 355499/375342 consumed_samples: 364032000 total_loss: 2.934 time: 0.3450 s/iter data_time: 0.0065 s/iter total_throughput: 2967.78 samples/s lr: 1.68e-05 [09/27 19:28:16] lb.utils.events INFO: eta: 4:37:03 iteration: 355599/375342 consumed_samples: 364134400 total_loss: 2.936 time: 0.3452 s/iter data_time: 0.0060 s/iter total_throughput: 2966.26 samples/s lr: 1.67e-05 [09/27 19:29:59] lb.utils.events INFO: eta: 4:41:03 iteration: 355699/375342 consumed_samples: 364236800 total_loss: 2.93 time: 0.3454 s/iter data_time: 0.0060 s/iter total_throughput: 2964.62 samples/s lr: 1.67e-05 [09/27 19:31:46] lb.utils.events INFO: eta: 4:47:21 iteration: 355799/375342 consumed_samples: 364339200 total_loss: 2.939 time: 0.3456 s/iter data_time: 0.0059 s/iter total_throughput: 2962.88 samples/s lr: 1.66e-05 [09/27 19:33:30] lb.utils.events INFO: eta: 4:47:19 iteration: 355899/375342 consumed_samples: 364441600 total_loss: 2.932 time: 0.3458 s/iter data_time: 0.0073 s/iter total_throughput: 2961.20 samples/s lr: 1.65e-05 [09/27 19:35:03] lb.utils.events INFO: eta: 4:41:15 iteration: 355999/375342 consumed_samples: 364544000 total_loss: 2.931 time: 0.3460 s/iter data_time: 0.0072 s/iter total_throughput: 2959.79 samples/s lr: 1.65e-05 [09/27 19:36:47] lb.utils.events INFO: eta: 4:40:31 iteration: 356099/375342 consumed_samples: 364646400 total_loss: 2.94 time: 0.3462 s/iter data_time: 0.0074 s/iter total_throughput: 2958.12 samples/s lr: 1.64e-05 [09/27 19:38:27] lb.utils.events INFO: eta: 4:42:30 iteration: 356199/375342 consumed_samples: 364748800 total_loss: 2.943 time: 0.3463 s/iter data_time: 0.0073 s/iter total_throughput: 2956.55 samples/s lr: 1.63e-05 [09/27 19:40:08] lb.utils.events INFO: eta: 4:43:34 iteration: 356299/375342 consumed_samples: 364851200 total_loss: 2.956 time: 0.3465 s/iter data_time: 0.0066 s/iter total_throughput: 2954.96 samples/s lr: 1.63e-05 [09/27 19:41:45] lb.utils.events INFO: eta: 4:41:50 iteration: 356399/375342 consumed_samples: 364953600 total_loss: 2.951 time: 0.3467 s/iter data_time: 0.0083 s/iter total_throughput: 2953.48 samples/s lr: 1.62e-05 [09/27 19:43:30] lb.utils.events INFO: eta: 4:41:03 iteration: 356499/375342 consumed_samples: 365056000 total_loss: 2.929 time: 0.3469 s/iter data_time: 0.0062 s/iter total_throughput: 2951.80 samples/s lr: 1.61e-05 [09/27 19:44:45] lb.utils.events INFO: eta: 4:33:09 iteration: 356599/375342 consumed_samples: 365158400 total_loss: 2.938 time: 0.3470 s/iter data_time: 0.0604 s/iter total_throughput: 2950.84 samples/s lr: 1.61e-05 [09/27 19:46:08] lb.utils.events INFO: eta: 4:18:05 iteration: 356699/375342 consumed_samples: 365260800 total_loss: 2.951 time: 0.3472 s/iter data_time: 0.0074 s/iter total_throughput: 2949.68 samples/s lr: 1.60e-05 [09/27 19:47:48] lb.utils.events INFO: eta: 4:11:24 iteration: 356799/375342 consumed_samples: 365363200 total_loss: 2.956 time: 0.3473 s/iter data_time: 0.0069 s/iter total_throughput: 2948.14 samples/s lr: 1.59e-05 [09/27 19:49:35] lb.utils.events INFO: eta: 4:10:03 iteration: 356899/375342 consumed_samples: 365465600 total_loss: 2.954 time: 0.3475 s/iter data_time: 0.0072 s/iter total_throughput: 2946.43 samples/s lr: 1.59e-05 [09/27 19:51:22] lb.utils.events INFO: eta: 4:15:29 iteration: 356999/375342 consumed_samples: 365568000 total_loss: 2.956 time: 0.3477 s/iter data_time: 0.0057 s/iter total_throughput: 2944.69 samples/s lr: 1.58e-05 [09/27 19:52:59] lb.utils.events INFO: eta: 4:11:26 iteration: 357099/375342 consumed_samples: 365670400 total_loss: 2.954 time: 0.3479 s/iter data_time: 0.0072 s/iter total_throughput: 2943.22 samples/s lr: 1.58e-05 [09/27 19:54:43] lb.utils.events INFO: eta: 4:02:46 iteration: 357199/375342 consumed_samples: 365772800 total_loss: 2.945 time: 0.3481 s/iter data_time: 0.0065 s/iter total_throughput: 2941.60 samples/s lr: 1.57e-05 [09/27 19:56:24] lb.utils.events INFO: eta: 3:57:13 iteration: 357299/375342 consumed_samples: 365875200 total_loss: 2.94 time: 0.3483 s/iter data_time: 0.0079 s/iter total_throughput: 2940.03 samples/s lr: 1.56e-05 [09/27 19:57:56] lb.utils.events INFO: eta: 3:52:00 iteration: 357399/375342 consumed_samples: 365977600 total_loss: 2.938 time: 0.3485 s/iter data_time: 0.0069 s/iter total_throughput: 2938.67 samples/s lr: 1.56e-05 [09/27 19:59:36] lb.utils.events INFO: eta: 3:49:32 iteration: 357499/375342 consumed_samples: 366080000 total_loss: 2.949 time: 0.3486 s/iter data_time: 0.0056 s/iter total_throughput: 2937.15 samples/s lr: 1.55e-05 [09/27 20:01:14] lb.utils.events INFO: eta: 3:51:25 iteration: 357599/375342 consumed_samples: 366182400 total_loss: 2.955 time: 0.3488 s/iter data_time: 0.0090 s/iter total_throughput: 2935.65 samples/s lr: 1.54e-05 [09/27 20:02:54] lb.utils.events INFO: eta: 4:01:33 iteration: 357699/375342 consumed_samples: 366284800 total_loss: 2.955 time: 0.3490 s/iter data_time: 0.0065 s/iter total_throughput: 2934.12 samples/s lr: 1.54e-05 [09/27 20:04:39] lb.utils.events INFO: eta: 4:00:50 iteration: 357799/375342 consumed_samples: 366387200 total_loss: 2.943 time: 0.3492 s/iter data_time: 0.0066 s/iter total_throughput: 2932.49 samples/s lr: 1.53e-05 [09/27 20:06:05] lb.utils.events INFO: eta: 3:57:00 iteration: 357899/375342 consumed_samples: 366489600 total_loss: 2.933 time: 0.3493 s/iter data_time: 0.0078 s/iter total_throughput: 2931.27 samples/s lr: 1.53e-05 [09/27 20:07:39] lb.utils.events INFO: eta: 3:50:33 iteration: 357999/375342 consumed_samples: 366592000 total_loss: 2.953 time: 0.3495 s/iter data_time: 0.0083 s/iter total_throughput: 2929.90 samples/s lr: 1.52e-05 [09/27 20:09:21] lb.utils.events INFO: eta: 3:51:10 iteration: 358099/375342 consumed_samples: 366694400 total_loss: 2.951 time: 0.3497 s/iter data_time: 0.0076 s/iter total_throughput: 2928.32 samples/s lr: 1.51e-05 [09/27 20:11:08] lb.utils.events INFO: eta: 3:52:52 iteration: 358199/375342 consumed_samples: 366796800 total_loss: 2.925 time: 0.3499 s/iter data_time: 0.0077 s/iter total_throughput: 2926.65 samples/s lr: 1.51e-05 [09/27 20:12:53] lb.utils.events INFO: eta: 3:56:35 iteration: 358299/375342 consumed_samples: 366899200 total_loss: 2.938 time: 0.3501 s/iter data_time: 0.0064 s/iter total_throughput: 2925.00 samples/s lr: 1.50e-05 [09/27 20:14:43] lb.utils.events INFO: eta: 4:00:47 iteration: 358399/375342 consumed_samples: 367001600 total_loss: 2.951 time: 0.3503 s/iter data_time: 0.0076 s/iter total_throughput: 2923.27 samples/s lr: 1.50e-05 [09/27 20:16:27] lb.utils.events INFO: eta: 3:59:59 iteration: 358499/375342 consumed_samples: 367104000 total_loss: 2.94 time: 0.3505 s/iter data_time: 0.0058 s/iter total_throughput: 2921.66 samples/s lr: 1.49e-05 [09/27 20:18:08] lb.utils.events INFO: eta: 4:02:07 iteration: 358599/375342 consumed_samples: 367206400 total_loss: 2.945 time: 0.3507 s/iter data_time: 0.0066 s/iter total_throughput: 2920.12 samples/s lr: 1.49e-05 [09/27 20:19:52] lb.utils.events INFO: eta: 3:58:19 iteration: 358699/375342 consumed_samples: 367308800 total_loss: 2.963 time: 0.3509 s/iter data_time: 0.0080 s/iter total_throughput: 2918.52 samples/s lr: 1.48e-05 [09/27 20:21:30] lb.utils.events INFO: eta: 3:52:19 iteration: 358799/375342 consumed_samples: 367411200 total_loss: 2.946 time: 0.3510 s/iter data_time: 0.0061 s/iter total_throughput: 2917.07 samples/s lr: 1.47e-05 [09/27 20:23:13] lb.utils.events INFO: eta: 3:55:33 iteration: 358899/375342 consumed_samples: 367513600 total_loss: 2.933 time: 0.3512 s/iter data_time: 0.0072 s/iter total_throughput: 2915.51 samples/s lr: 1.47e-05 [09/27 20:24:52] lb.utils.events INFO: eta: 3:57:12 iteration: 358999/375342 consumed_samples: 367616000 total_loss: 2.955 time: 0.3514 s/iter data_time: 0.0080 s/iter total_throughput: 2914.02 samples/s lr: 1.46e-05 [09/27 20:26:08] lb.utils.events INFO: eta: 3:45:10 iteration: 359099/375342 consumed_samples: 367718400 total_loss: 2.949 time: 0.3515 s/iter data_time: 0.2642 s/iter total_throughput: 2913.07 samples/s lr: 1.46e-05 [09/27 20:27:19] lb.utils.events INFO: eta: 3:35:52 iteration: 359199/375342 consumed_samples: 367820800 total_loss: 2.938 time: 0.3516 s/iter data_time: 0.0076 s/iter total_throughput: 2912.26 samples/s lr: 1.45e-05 [09/27 20:28:59] lb.utils.events INFO: eta: 3:28:21 iteration: 359299/375342 consumed_samples: 367923200 total_loss: 2.932 time: 0.3518 s/iter data_time: 0.0060 s/iter total_throughput: 2910.76 samples/s lr: 1.45e-05 [09/27 20:30:38] lb.utils.events INFO: eta: 3:24:35 iteration: 359399/375342 consumed_samples: 368025600 total_loss: 2.955 time: 0.3520 s/iter data_time: 0.0079 s/iter total_throughput: 2909.29 samples/s lr: 1.44e-05 [09/27 20:32:22] lb.utils.events INFO: eta: 3:25:12 iteration: 359499/375342 consumed_samples: 368128000 total_loss: 2.939 time: 0.3522 s/iter data_time: 0.0077 s/iter total_throughput: 2907.71 samples/s lr: 1.43e-05 [09/27 20:34:06] lb.utils.events INFO: eta: 3:28:36 iteration: 359599/375342 consumed_samples: 368230400 total_loss: 2.917 time: 0.3524 s/iter data_time: 0.0081 s/iter total_throughput: 2906.14 samples/s lr: 1.43e-05 [09/27 20:35:48] lb.utils.events INFO: eta: 3:29:03 iteration: 359699/375342 consumed_samples: 368332800 total_loss: 2.938 time: 0.3525 s/iter data_time: 0.0071 s/iter total_throughput: 2904.61 samples/s lr: 1.42e-05 [09/27 20:37:29] lb.utils.events INFO: eta: 3:28:28 iteration: 359799/375342 consumed_samples: 368435200 total_loss: 2.952 time: 0.3527 s/iter data_time: 0.0066 s/iter total_throughput: 2903.09 samples/s lr: 1.42e-05 [09/27 20:39:18] lb.utils.events INFO: eta: 3:28:09 iteration: 359899/375342 consumed_samples: 368537600 total_loss: 2.961 time: 0.3529 s/iter data_time: 0.0082 s/iter total_throughput: 2901.42 samples/s lr: 1.41e-05 [09/27 20:41:02] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0359999 [09/27 20:41:04] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/27 20:41:04] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/27 20:41:10] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0005 s/iter. Inference: 0.3684 s/iter. Eval: 0.0090 s/iter. Total: 0.3779 s/iter. ETA=0:00:13 [09/27 20:41:15] lb.evaluation.evaluator INFO: Inference done 24576/50000. Dataloading: 0.0005 s/iter. Inference: 0.3830 s/iter. Eval: 0.0060 s/iter. Total: 0.3897 s/iter. ETA=0:00:09 [09/27 20:41:20] lb.evaluation.evaluator INFO: Inference done 38912/50000. Dataloading: 0.0006 s/iter. Inference: 0.3768 s/iter. Eval: 0.0054 s/iter. Total: 0.3829 s/iter. ETA=0:00:03 [09/27 20:41:25] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/27 20:41:25] lb.evaluation.evaluator INFO: Total inference time: 0:00:17.183160 (0.000344 s / iter per device, on 8 devices) [09/27 20:41:25] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:16 (0.000339 s / iter per device, on 8 devices) [09/27 20:41:25] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/27 20:41:25] lb.evaluation.utils INFO: copypaste: Acc@1=80.04599999999999 [09/27 20:41:25] lb.evaluation.utils INFO: copypaste: Acc@5=94.484 [09/27 20:41:25] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 80.04600, better than last best score 80.01400 @ iteration 354999. [09/27 20:41:25] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/27 20:41:26] lb.utils.events INFO: eta: 3:32:19 iteration: 359999/375342 consumed_samples: 368640000 total_loss: 2.948 time: 0.3531 s/iter data_time: 0.0062 s/iter total_throughput: 2899.86 samples/s lr: 1.41e-05 [09/27 20:42:57] lb.utils.events INFO: eta: 3:37:49 iteration: 360099/375342 consumed_samples: 368742400 total_loss: 2.942 time: 0.3533 s/iter data_time: 0.0075 s/iter total_throughput: 2898.59 samples/s lr: 1.40e-05 [09/27 20:44:48] lb.utils.events INFO: eta: 3:46:05 iteration: 360199/375342 consumed_samples: 368844800 total_loss: 2.946 time: 0.3535 s/iter data_time: 0.0075 s/iter total_throughput: 2896.87 samples/s lr: 1.40e-05 [09/27 20:46:30] lb.utils.events INFO: eta: 3:48:58 iteration: 360299/375342 consumed_samples: 368947200 total_loss: 2.941 time: 0.3537 s/iter data_time: 0.0081 s/iter total_throughput: 2895.36 samples/s lr: 1.39e-05 [09/27 20:48:11] lb.utils.events INFO: eta: 3:48:12 iteration: 360399/375342 consumed_samples: 369049600 total_loss: 2.922 time: 0.3539 s/iter data_time: 0.0077 s/iter total_throughput: 2893.87 samples/s lr: 1.39e-05 [09/27 20:49:51] lb.utils.events INFO: eta: 3:42:30 iteration: 360499/375342 consumed_samples: 369152000 total_loss: 2.932 time: 0.3540 s/iter data_time: 0.0071 s/iter total_throughput: 2892.41 samples/s lr: 1.38e-05 [09/27 20:51:28] lb.utils.events INFO: eta: 3:38:40 iteration: 360599/375342 consumed_samples: 369254400 total_loss: 2.939 time: 0.3542 s/iter data_time: 0.0101 s/iter total_throughput: 2891.00 samples/s lr: 1.38e-05 [09/27 20:53:01] lb.utils.events INFO: eta: 3:33:40 iteration: 360699/375342 consumed_samples: 369356800 total_loss: 2.937 time: 0.3544 s/iter data_time: 0.0086 s/iter total_throughput: 2889.71 samples/s lr: 1.37e-05 [09/27 20:54:42] lb.utils.events INFO: eta: 3:28:41 iteration: 360799/375342 consumed_samples: 369459200 total_loss: 2.926 time: 0.3545 s/iter data_time: 0.0066 s/iter total_throughput: 2888.23 samples/s lr: 1.37e-05 [09/27 20:56:31] lb.utils.events INFO: eta: 3:28:18 iteration: 360899/375342 consumed_samples: 369561600 total_loss: 2.957 time: 0.3547 s/iter data_time: 0.0063 s/iter total_throughput: 2886.57 samples/s lr: 1.36e-05 [09/27 20:58:06] lb.utils.events INFO: eta: 3:21:40 iteration: 360999/375342 consumed_samples: 369664000 total_loss: 2.953 time: 0.3549 s/iter data_time: 0.0078 s/iter total_throughput: 2885.22 samples/s lr: 1.36e-05 [09/27 20:59:50] lb.utils.events INFO: eta: 3:20:38 iteration: 361099/375342 consumed_samples: 369766400 total_loss: 2.929 time: 0.3551 s/iter data_time: 0.0072 s/iter total_throughput: 2883.69 samples/s lr: 1.35e-05 [09/27 21:01:38] lb.utils.events INFO: eta: 3:20:38 iteration: 361199/375342 consumed_samples: 369868800 total_loss: 2.946 time: 0.3553 s/iter data_time: 0.0062 s/iter total_throughput: 2882.06 samples/s lr: 1.35e-05 [09/27 21:03:15] lb.utils.events INFO: eta: 3:17:40 iteration: 361299/375342 consumed_samples: 369971200 total_loss: 2.952 time: 0.3555 s/iter data_time: 0.0072 s/iter total_throughput: 2880.69 samples/s lr: 1.34e-05 [09/27 21:05:00] lb.utils.events INFO: eta: 3:16:25 iteration: 361399/375342 consumed_samples: 370073600 total_loss: 2.949 time: 0.3557 s/iter data_time: 0.0070 s/iter total_throughput: 2879.12 samples/s lr: 1.34e-05 [09/27 21:06:34] lb.utils.events INFO: eta: 3:14:32 iteration: 361499/375342 consumed_samples: 370176000 total_loss: 2.947 time: 0.3558 s/iter data_time: 0.1084 s/iter total_throughput: 2877.82 samples/s lr: 1.33e-05 [09/27 21:07:44] lb.utils.events INFO: eta: 3:02:05 iteration: 361599/375342 consumed_samples: 370278400 total_loss: 2.947 time: 0.3559 s/iter data_time: 0.0070 s/iter total_throughput: 2877.04 samples/s lr: 1.33e-05 [09/27 21:09:32] lb.utils.events INFO: eta: 3:00:46 iteration: 361699/375342 consumed_samples: 370380800 total_loss: 2.948 time: 0.3561 s/iter data_time: 0.0083 s/iter total_throughput: 2875.42 samples/s lr: 1.32e-05 [09/27 21:11:17] lb.utils.events INFO: eta: 3:03:00 iteration: 361799/375342 consumed_samples: 370483200 total_loss: 2.946 time: 0.3563 s/iter data_time: 0.0066 s/iter total_throughput: 2873.87 samples/s lr: 1.32e-05 [09/27 21:12:57] lb.utils.events INFO: eta: 2:57:29 iteration: 361899/375342 consumed_samples: 370585600 total_loss: 2.933 time: 0.3565 s/iter data_time: 0.0074 s/iter total_throughput: 2872.45 samples/s lr: 1.31e-05 [09/27 21:14:42] lb.utils.events INFO: eta: 3:02:39 iteration: 361999/375342 consumed_samples: 370688000 total_loss: 2.922 time: 0.3567 s/iter data_time: 0.0070 s/iter total_throughput: 2870.91 samples/s lr: 1.31e-05 [09/27 21:16:22] lb.utils.events INFO: eta: 3:01:17 iteration: 362099/375342 consumed_samples: 370790400 total_loss: 2.922 time: 0.3569 s/iter data_time: 0.0067 s/iter total_throughput: 2869.47 samples/s lr: 1.30e-05 [09/27 21:18:02] lb.utils.events INFO: eta: 2:54:19 iteration: 362199/375342 consumed_samples: 370892800 total_loss: 2.939 time: 0.3570 s/iter data_time: 0.0075 s/iter total_throughput: 2868.05 samples/s lr: 1.30e-05 [09/27 21:19:43] lb.utils.events INFO: eta: 2:55:26 iteration: 362299/375342 consumed_samples: 370995200 total_loss: 2.942 time: 0.3572 s/iter data_time: 0.0060 s/iter total_throughput: 2866.60 samples/s lr: 1.29e-05 [09/27 21:21:30] lb.utils.events INFO: eta: 2:54:37 iteration: 362399/375342 consumed_samples: 371097600 total_loss: 2.948 time: 0.3574 s/iter data_time: 0.0071 s/iter total_throughput: 2865.02 samples/s lr: 1.29e-05 [09/27 21:23:06] lb.utils.events INFO: eta: 2:56:37 iteration: 362499/375342 consumed_samples: 371200000 total_loss: 2.945 time: 0.3576 s/iter data_time: 0.0077 s/iter total_throughput: 2863.69 samples/s lr: 1.29e-05 [09/27 21:24:51] lb.utils.events INFO: eta: 3:05:06 iteration: 362599/375342 consumed_samples: 371302400 total_loss: 2.941 time: 0.3578 s/iter data_time: 0.0082 s/iter total_throughput: 2862.17 samples/s lr: 1.28e-05 [09/27 21:26:25] lb.utils.events INFO: eta: 3:01:09 iteration: 362699/375342 consumed_samples: 371404800 total_loss: 2.945 time: 0.3579 s/iter data_time: 0.0070 s/iter total_throughput: 2860.89 samples/s lr: 1.28e-05 [09/27 21:28:09] lb.utils.events INFO: eta: 3:01:56 iteration: 362799/375342 consumed_samples: 371507200 total_loss: 2.935 time: 0.3581 s/iter data_time: 0.0063 s/iter total_throughput: 2859.38 samples/s lr: 1.27e-05 [09/27 21:29:56] lb.utils.events INFO: eta: 3:01:17 iteration: 362899/375342 consumed_samples: 371609600 total_loss: 2.938 time: 0.3583 s/iter data_time: 0.0067 s/iter total_throughput: 2857.81 samples/s lr: 1.27e-05 [09/27 21:31:34] lb.utils.events INFO: eta: 2:55:53 iteration: 362999/375342 consumed_samples: 371712000 total_loss: 2.954 time: 0.3585 s/iter data_time: 0.0086 s/iter total_throughput: 2856.44 samples/s lr: 1.26e-05 [09/27 21:33:17] lb.utils.events INFO: eta: 2:55:43 iteration: 363099/375342 consumed_samples: 371814400 total_loss: 2.941 time: 0.3587 s/iter data_time: 0.0090 s/iter total_throughput: 2854.97 samples/s lr: 1.26e-05 [09/27 21:34:55] lb.utils.events INFO: eta: 2:53:00 iteration: 363199/375342 consumed_samples: 371916800 total_loss: 2.937 time: 0.3588 s/iter data_time: 0.0068 s/iter total_throughput: 2853.62 samples/s lr: 1.26e-05 [09/27 21:36:37] lb.utils.events INFO: eta: 2:51:43 iteration: 363299/375342 consumed_samples: 372019200 total_loss: 2.943 time: 0.3590 s/iter data_time: 0.0056 s/iter total_throughput: 2852.18 samples/s lr: 1.25e-05 [09/27 21:38:14] lb.utils.events INFO: eta: 2:48:50 iteration: 363399/375342 consumed_samples: 372121600 total_loss: 2.94 time: 0.3592 s/iter data_time: 0.0062 s/iter total_throughput: 2850.83 samples/s lr: 1.25e-05 [09/27 21:39:48] lb.utils.events INFO: eta: 2:47:16 iteration: 363499/375342 consumed_samples: 372224000 total_loss: 2.953 time: 0.3594 s/iter data_time: 0.0090 s/iter total_throughput: 2849.57 samples/s lr: 1.24e-05 [09/27 21:41:25] lb.utils.events INFO: eta: 2:43:28 iteration: 363599/375342 consumed_samples: 372326400 total_loss: 2.961 time: 0.3595 s/iter data_time: 0.0059 s/iter total_throughput: 2848.23 samples/s lr: 1.24e-05 [09/27 21:43:05] lb.utils.events INFO: eta: 2:45:22 iteration: 363699/375342 consumed_samples: 372428800 total_loss: 2.932 time: 0.3597 s/iter data_time: 0.0061 s/iter total_throughput: 2846.83 samples/s lr: 1.23e-05 [09/27 21:44:48] lb.utils.events INFO: eta: 2:43:57 iteration: 363799/375342 consumed_samples: 372531200 total_loss: 2.941 time: 0.3599 s/iter data_time: 0.0073 s/iter total_throughput: 2845.38 samples/s lr: 1.23e-05 [09/27 21:46:19] lb.utils.events INFO: eta: 2:39:56 iteration: 363899/375342 consumed_samples: 372633600 total_loss: 2.951 time: 0.3600 s/iter data_time: 0.0067 s/iter total_throughput: 2844.20 samples/s lr: 1.23e-05 [09/27 21:47:46] lb.utils.events INFO: eta: 2:37:09 iteration: 363999/375342 consumed_samples: 372736000 total_loss: 2.939 time: 0.3602 s/iter data_time: 0.1984 s/iter total_throughput: 2843.07 samples/s lr: 1.22e-05 [09/27 21:48:54] lb.utils.events INFO: eta: 2:28:04 iteration: 364099/375342 consumed_samples: 372838400 total_loss: 2.938 time: 0.3603 s/iter data_time: 0.0093 s/iter total_throughput: 2842.39 samples/s lr: 1.22e-05 [09/27 21:50:38] lb.utils.events INFO: eta: 2:27:56 iteration: 364199/375342 consumed_samples: 372940800 total_loss: 2.957 time: 0.3604 s/iter data_time: 0.0057 s/iter total_throughput: 2840.91 samples/s lr: 1.22e-05 [09/27 21:52:19] lb.utils.events INFO: eta: 2:25:14 iteration: 364299/375342 consumed_samples: 373043200 total_loss: 2.958 time: 0.3606 s/iter data_time: 0.0070 s/iter total_throughput: 2839.52 samples/s lr: 1.21e-05 [09/27 21:54:03] lb.utils.events INFO: eta: 2:24:35 iteration: 364399/375342 consumed_samples: 373145600 total_loss: 2.929 time: 0.3608 s/iter data_time: 0.0069 s/iter total_throughput: 2838.05 samples/s lr: 1.21e-05 [09/27 21:55:39] lb.utils.events INFO: eta: 2:24:21 iteration: 364499/375342 consumed_samples: 373248000 total_loss: 2.946 time: 0.3610 s/iter data_time: 0.0068 s/iter total_throughput: 2836.75 samples/s lr: 1.20e-05 [09/27 21:57:32] lb.utils.events INFO: eta: 2:27:29 iteration: 364599/375342 consumed_samples: 373350400 total_loss: 2.934 time: 0.3612 s/iter data_time: 0.0073 s/iter total_throughput: 2835.11 samples/s lr: 1.20e-05 [09/27 21:59:03] lb.utils.events INFO: eta: 2:21:41 iteration: 364699/375342 consumed_samples: 373452800 total_loss: 2.933 time: 0.3613 s/iter data_time: 0.0091 s/iter total_throughput: 2833.91 samples/s lr: 1.20e-05 [09/27 22:00:53] lb.utils.events INFO: eta: 2:20:27 iteration: 364799/375342 consumed_samples: 373555200 total_loss: 2.947 time: 0.3615 s/iter data_time: 0.0064 s/iter total_throughput: 2832.34 samples/s lr: 1.19e-05 [09/27 22:02:37] lb.utils.events INFO: eta: 2:24:15 iteration: 364899/375342 consumed_samples: 373657600 total_loss: 2.935 time: 0.3617 s/iter data_time: 0.0066 s/iter total_throughput: 2830.87 samples/s lr: 1.19e-05 [09/27 22:04:25] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0364999 [09/27 22:04:27] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/27 22:04:27] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/27 22:04:32] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0005 s/iter. Inference: 0.4155 s/iter. Eval: 0.0054 s/iter. Total: 0.4214 s/iter. ETA=0:00:15 [09/27 22:04:37] lb.evaluation.evaluator INFO: Inference done 25600/50000. Dataloading: 0.0006 s/iter. Inference: 0.3716 s/iter. Eval: 0.0049 s/iter. Total: 0.3772 s/iter. ETA=0:00:08 [09/27 22:04:42] lb.evaluation.evaluator INFO: Inference done 38912/50000. Dataloading: 0.0006 s/iter. Inference: 0.3768 s/iter. Eval: 0.0051 s/iter. Total: 0.3826 s/iter. ETA=0:00:03 [09/27 22:04:47] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/27 22:04:47] lb.evaluation.evaluator INFO: Total inference time: 0:00:16.753879 (0.000335 s / iter per device, on 8 devices) [09/27 22:04:47] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:16 (0.000330 s / iter per device, on 8 devices) [09/27 22:04:47] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/27 22:04:47] lb.evaluation.utils INFO: copypaste: Acc@1=80.086 [09/27 22:04:47] lb.evaluation.utils INFO: copypaste: Acc@5=94.516 [09/27 22:04:47] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 80.08600, better than last best score 80.04600 @ iteration 359999. [09/27 22:04:47] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/27 22:04:48] lb.utils.events INFO: eta: 2:29:33 iteration: 364999/375342 consumed_samples: 373760000 total_loss: 2.929 time: 0.3619 s/iter data_time: 0.0070 s/iter total_throughput: 2829.36 samples/s lr: 1.19e-05 [09/27 22:06:21] lb.utils.events INFO: eta: 2:35:15 iteration: 365099/375342 consumed_samples: 373862400 total_loss: 2.942 time: 0.3621 s/iter data_time: 0.0067 s/iter total_throughput: 2828.14 samples/s lr: 1.18e-05 [09/27 22:07:54] lb.utils.events INFO: eta: 2:32:51 iteration: 365199/375342 consumed_samples: 373964800 total_loss: 2.948 time: 0.3622 s/iter data_time: 0.0091 s/iter total_throughput: 2826.92 samples/s lr: 1.18e-05 [09/27 22:09:33] lb.utils.events INFO: eta: 2:31:20 iteration: 365299/375342 consumed_samples: 374067200 total_loss: 2.934 time: 0.3624 s/iter data_time: 0.0072 s/iter total_throughput: 2825.58 samples/s lr: 1.17e-05 [09/27 22:11:14] lb.utils.events INFO: eta: 2:28:12 iteration: 365399/375342 consumed_samples: 374169600 total_loss: 2.937 time: 0.3626 s/iter data_time: 0.0058 s/iter total_throughput: 2824.21 samples/s lr: 1.17e-05 [09/27 22:12:54] lb.utils.events INFO: eta: 2:26:06 iteration: 365499/375342 consumed_samples: 374272000 total_loss: 2.937 time: 0.3628 s/iter data_time: 0.0072 s/iter total_throughput: 2822.85 samples/s lr: 1.17e-05 [09/27 22:14:41] lb.utils.events INFO: eta: 2:20:54 iteration: 365599/375342 consumed_samples: 374374400 total_loss: 2.934 time: 0.3629 s/iter data_time: 0.0074 s/iter total_throughput: 2821.35 samples/s lr: 1.16e-05 [09/27 22:16:23] lb.utils.events INFO: eta: 2:20:42 iteration: 365699/375342 consumed_samples: 374476800 total_loss: 2.934 time: 0.3631 s/iter data_time: 0.0080 s/iter total_throughput: 2819.94 samples/s lr: 1.16e-05 [09/27 22:18:01] lb.utils.events INFO: eta: 2:14:52 iteration: 365799/375342 consumed_samples: 374579200 total_loss: 2.935 time: 0.3633 s/iter data_time: 0.0089 s/iter total_throughput: 2818.64 samples/s lr: 1.16e-05 [09/27 22:19:46] lb.utils.events INFO: eta: 2:11:50 iteration: 365899/375342 consumed_samples: 374681600 total_loss: 2.935 time: 0.3635 s/iter data_time: 0.0075 s/iter total_throughput: 2817.18 samples/s lr: 1.15e-05 [09/27 22:21:31] lb.utils.events INFO: eta: 2:09:52 iteration: 365999/375342 consumed_samples: 374784000 total_loss: 2.922 time: 0.3637 s/iter data_time: 0.0071 s/iter total_throughput: 2815.73 samples/s lr: 1.15e-05 [09/27 22:23:10] lb.utils.events INFO: eta: 2:09:02 iteration: 366099/375342 consumed_samples: 374886400 total_loss: 2.925 time: 0.3638 s/iter data_time: 0.0076 s/iter total_throughput: 2814.41 samples/s lr: 1.15e-05 [09/27 22:24:47] lb.utils.events INFO: eta: 2:05:29 iteration: 366199/375342 consumed_samples: 374988800 total_loss: 2.937 time: 0.3640 s/iter data_time: 0.0080 s/iter total_throughput: 2813.13 samples/s lr: 1.14e-05 [09/27 22:26:31] lb.utils.events INFO: eta: 2:06:41 iteration: 366299/375342 consumed_samples: 375091200 total_loss: 2.945 time: 0.3642 s/iter data_time: 0.0056 s/iter total_throughput: 2811.70 samples/s lr: 1.14e-05 [09/27 22:27:48] lb.utils.events INFO: eta: 2:00:14 iteration: 366399/375342 consumed_samples: 375193600 total_loss: 2.952 time: 0.3643 s/iter data_time: 0.0108 s/iter total_throughput: 2810.85 samples/s lr: 1.14e-05 [09/27 22:29:08] lb.utils.events INFO: eta: 1:57:16 iteration: 366499/375342 consumed_samples: 375296000 total_loss: 2.949 time: 0.3644 s/iter data_time: 0.0074 s/iter total_throughput: 2809.94 samples/s lr: 1.14e-05 [09/27 22:30:59] lb.utils.events INFO: eta: 1:56:22 iteration: 366599/375342 consumed_samples: 375398400 total_loss: 2.936 time: 0.3646 s/iter data_time: 0.0061 s/iter total_throughput: 2808.38 samples/s lr: 1.13e-05 [09/27 22:32:51] lb.utils.events INFO: eta: 1:56:24 iteration: 366699/375342 consumed_samples: 375500800 total_loss: 2.931 time: 0.3648 s/iter data_time: 0.0081 s/iter total_throughput: 2806.79 samples/s lr: 1.13e-05 [09/27 22:34:38] lb.utils.events INFO: eta: 1:59:12 iteration: 366799/375342 consumed_samples: 375603200 total_loss: 2.947 time: 0.3650 s/iter data_time: 0.0071 s/iter total_throughput: 2805.30 samples/s lr: 1.13e-05 [09/27 22:36:26] lb.utils.events INFO: eta: 1:56:59 iteration: 366899/375342 consumed_samples: 375705600 total_loss: 2.944 time: 0.3652 s/iter data_time: 0.0075 s/iter total_throughput: 2803.81 samples/s lr: 1.12e-05 [09/27 22:38:15] lb.utils.events INFO: eta: 1:58:45 iteration: 366999/375342 consumed_samples: 375808000 total_loss: 2.949 time: 0.3654 s/iter data_time: 0.0079 s/iter total_throughput: 2802.29 samples/s lr: 1.12e-05 [09/27 22:40:04] lb.utils.events INFO: eta: 1:59:31 iteration: 367099/375342 consumed_samples: 375910400 total_loss: 2.929 time: 0.3656 s/iter data_time: 0.0094 s/iter total_throughput: 2800.78 samples/s lr: 1.12e-05 [09/27 22:41:48] lb.utils.events INFO: eta: 2:02:30 iteration: 367199/375342 consumed_samples: 376012800 total_loss: 2.928 time: 0.3658 s/iter data_time: 0.0072 s/iter total_throughput: 2799.37 samples/s lr: 1.11e-05 [09/27 22:43:35] lb.utils.events INFO: eta: 2:00:35 iteration: 367299/375342 consumed_samples: 376115200 total_loss: 2.938 time: 0.3660 s/iter data_time: 0.0078 s/iter total_throughput: 2797.91 samples/s lr: 1.11e-05 [09/27 22:45:10] lb.utils.events INFO: eta: 2:03:42 iteration: 367399/375342 consumed_samples: 376217600 total_loss: 2.929 time: 0.3661 s/iter data_time: 0.0082 s/iter total_throughput: 2796.69 samples/s lr: 1.11e-05 [09/27 22:46:42] lb.utils.events INFO: eta: 2:04:17 iteration: 367499/375342 consumed_samples: 376320000 total_loss: 2.934 time: 0.3663 s/iter data_time: 0.0054 s/iter total_throughput: 2795.54 samples/s lr: 1.11e-05 [09/27 22:48:17] lb.utils.events INFO: eta: 1:58:07 iteration: 367599/375342 consumed_samples: 376422400 total_loss: 2.935 time: 0.3665 s/iter data_time: 0.0058 s/iter total_throughput: 2794.33 samples/s lr: 1.10e-05 [09/27 22:49:52] lb.utils.events INFO: eta: 1:52:09 iteration: 367699/375342 consumed_samples: 376524800 total_loss: 2.933 time: 0.3666 s/iter data_time: 0.0060 s/iter total_throughput: 2793.12 samples/s lr: 1.10e-05 [09/27 22:51:34] lb.utils.events INFO: eta: 1:47:49 iteration: 367799/375342 consumed_samples: 376627200 total_loss: 2.927 time: 0.3668 s/iter data_time: 0.0079 s/iter total_throughput: 2791.77 samples/s lr: 1.10e-05 [09/27 22:53:18] lb.utils.events INFO: eta: 1:48:44 iteration: 367899/375342 consumed_samples: 376729600 total_loss: 2.923 time: 0.3670 s/iter data_time: 0.0061 s/iter total_throughput: 2790.38 samples/s lr: 1.10e-05 [09/27 22:54:56] lb.utils.events INFO: eta: 1:44:11 iteration: 367999/375342 consumed_samples: 376832000 total_loss: 2.929 time: 0.3671 s/iter data_time: 0.0071 s/iter total_throughput: 2789.12 samples/s lr: 1.09e-05 [09/27 22:56:39] lb.utils.events INFO: eta: 1:41:36 iteration: 368099/375342 consumed_samples: 376934400 total_loss: 2.945 time: 0.3673 s/iter data_time: 0.0076 s/iter total_throughput: 2787.75 samples/s lr: 1.09e-05 [09/27 22:58:23] lb.utils.events INFO: eta: 1:37:53 iteration: 368199/375342 consumed_samples: 377036800 total_loss: 2.939 time: 0.3675 s/iter data_time: 0.0070 s/iter total_throughput: 2786.35 samples/s lr: 1.09e-05 [09/27 23:00:02] lb.utils.events INFO: eta: 1:35:02 iteration: 368299/375342 consumed_samples: 377139200 total_loss: 2.945 time: 0.3677 s/iter data_time: 0.0065 s/iter total_throughput: 2785.08 samples/s lr: 1.09e-05 [09/27 23:01:44] lb.utils.events INFO: eta: 1:35:40 iteration: 368399/375342 consumed_samples: 377241600 total_loss: 2.942 time: 0.3679 s/iter data_time: 0.0072 s/iter total_throughput: 2783.74 samples/s lr: 1.08e-05 [09/27 23:03:24] lb.utils.events INFO: eta: 1:36:03 iteration: 368499/375342 consumed_samples: 377344000 total_loss: 2.923 time: 0.3680 s/iter data_time: 0.0058 s/iter total_throughput: 2782.43 samples/s lr: 1.08e-05 [09/27 23:05:13] lb.utils.events INFO: eta: 1:39:34 iteration: 368599/375342 consumed_samples: 377446400 total_loss: 2.928 time: 0.3682 s/iter data_time: 0.0066 s/iter total_throughput: 2780.96 samples/s lr: 1.08e-05 [09/27 23:06:56] lb.utils.events INFO: eta: 1:39:27 iteration: 368699/375342 consumed_samples: 377548800 total_loss: 2.913 time: 0.3684 s/iter data_time: 0.0061 s/iter total_throughput: 2779.60 samples/s lr: 1.08e-05 [09/27 23:08:31] lb.utils.events INFO: eta: 1:37:43 iteration: 368799/375342 consumed_samples: 377651200 total_loss: 2.922 time: 0.3686 s/iter data_time: 0.0063 s/iter total_throughput: 2778.42 samples/s lr: 1.07e-05 [09/27 23:09:50] lb.utils.events INFO: eta: 1:31:09 iteration: 368899/375342 consumed_samples: 377753600 total_loss: 2.937 time: 0.3687 s/iter data_time: 0.2109 s/iter total_throughput: 2777.55 samples/s lr: 1.07e-05 [09/27 23:11:11] lb.utils.events INFO: eta: 1:25:24 iteration: 368999/375342 consumed_samples: 377856000 total_loss: 2.926 time: 0.3688 s/iter data_time: 0.0090 s/iter total_throughput: 2776.66 samples/s lr: 1.07e-05 [09/27 23:12:53] lb.utils.events INFO: eta: 1:24:03 iteration: 369099/375342 consumed_samples: 377958400 total_loss: 2.914 time: 0.3690 s/iter data_time: 0.0077 s/iter total_throughput: 2775.33 samples/s lr: 1.07e-05 [09/27 23:14:30] lb.utils.events INFO: eta: 1:22:55 iteration: 369199/375342 consumed_samples: 378060800 total_loss: 2.927 time: 0.3691 s/iter data_time: 0.0079 s/iter total_throughput: 2774.11 samples/s lr: 1.07e-05 [09/27 23:16:11] lb.utils.events INFO: eta: 1:21:44 iteration: 369299/375342 consumed_samples: 378163200 total_loss: 2.939 time: 0.3693 s/iter data_time: 0.0064 s/iter total_throughput: 2772.81 samples/s lr: 1.06e-05 [09/27 23:17:48] lb.utils.events INFO: eta: 1:19:35 iteration: 369399/375342 consumed_samples: 378265600 total_loss: 2.936 time: 0.3695 s/iter data_time: 0.0069 s/iter total_throughput: 2771.58 samples/s lr: 1.06e-05 [09/27 23:19:37] lb.utils.events INFO: eta: 1:20:27 iteration: 369499/375342 consumed_samples: 378368000 total_loss: 2.923 time: 0.3697 s/iter data_time: 0.0055 s/iter total_throughput: 2770.12 samples/s lr: 1.06e-05 [09/27 23:21:24] lb.utils.events INFO: eta: 1:17:18 iteration: 369599/375342 consumed_samples: 378470400 total_loss: 2.917 time: 0.3698 s/iter data_time: 0.0059 s/iter total_throughput: 2768.70 samples/s lr: 1.06e-05 [09/27 23:23:17] lb.utils.events INFO: eta: 1:18:28 iteration: 369699/375342 consumed_samples: 378572800 total_loss: 2.925 time: 0.3701 s/iter data_time: 0.0080 s/iter total_throughput: 2767.16 samples/s lr: 1.06e-05 [09/27 23:24:54] lb.utils.events INFO: eta: 1:16:03 iteration: 369799/375342 consumed_samples: 378675200 total_loss: 2.926 time: 0.3702 s/iter data_time: 0.0064 s/iter total_throughput: 2765.94 samples/s lr: 1.05e-05 [09/27 23:26:37] lb.utils.events INFO: eta: 1:18:43 iteration: 369899/375342 consumed_samples: 378777600 total_loss: 2.943 time: 0.3704 s/iter data_time: 0.0077 s/iter total_throughput: 2764.62 samples/s lr: 1.05e-05 [09/27 23:28:10] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0369999 [09/27 23:28:12] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/27 23:28:12] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/27 23:28:18] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0004 s/iter. Inference: 0.3418 s/iter. Eval: 0.0046 s/iter. Total: 0.3468 s/iter. ETA=0:00:12 [09/27 23:28:23] lb.evaluation.evaluator INFO: Inference done 24576/50000. Dataloading: 0.0006 s/iter. Inference: 0.3858 s/iter. Eval: 0.0051 s/iter. Total: 0.3915 s/iter. ETA=0:00:09 [09/27 23:28:28] lb.evaluation.evaluator INFO: Inference done 37888/50000. Dataloading: 0.0006 s/iter. Inference: 0.3900 s/iter. Eval: 0.0051 s/iter. Total: 0.3958 s/iter. ETA=0:00:04 [09/27 23:28:33] lb.evaluation.evaluator INFO: Inference done 50000/50000. Dataloading: 0.0006 s/iter. Inference: 0.3973 s/iter. Eval: 0.0052 s/iter. Total: 0.4032 s/iter. ETA=0:00:00 [09/27 23:28:33] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/27 23:28:33] lb.evaluation.evaluator INFO: Total inference time: 0:00:17.741367 (0.000355 s / iter per device, on 8 devices) [09/27 23:28:33] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:17 (0.000350 s / iter per device, on 8 devices) [09/27 23:28:33] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/27 23:28:33] lb.evaluation.utils INFO: copypaste: Acc@1=79.982 [09/27 23:28:33] lb.evaluation.utils INFO: copypaste: Acc@5=94.464 [09/27 23:28:33] lb.engine.hooks INFO: Not saving as latest eval score for Acc@1 is 79.98200, not better than best score 80.08600 @ iteration 364999. [09/27 23:28:33] lb.utils.events INFO: eta: 1:19:58 iteration: 369999/375342 consumed_samples: 378880000 total_loss: 2.932 time: 0.3705 s/iter data_time: 0.0067 s/iter total_throughput: 2763.49 samples/s lr: 1.05e-05 [09/27 23:30:21] lb.utils.events INFO: eta: 1:19:43 iteration: 370099/375342 consumed_samples: 378982400 total_loss: 2.926 time: 0.3707 s/iter data_time: 0.0075 s/iter total_throughput: 2762.06 samples/s lr: 1.05e-05 [09/27 23:31:58] lb.utils.events INFO: eta: 1:18:12 iteration: 370199/375342 consumed_samples: 379084800 total_loss: 2.93 time: 0.3709 s/iter data_time: 0.0066 s/iter total_throughput: 2760.86 samples/s lr: 1.05e-05 [09/27 23:33:41] lb.utils.events INFO: eta: 1:14:35 iteration: 370299/375342 consumed_samples: 379187200 total_loss: 2.915 time: 0.3711 s/iter data_time: 0.0297 s/iter total_throughput: 2759.54 samples/s lr: 1.04e-05 [09/27 23:35:16] lb.utils.events INFO: eta: 1:13:27 iteration: 370399/375342 consumed_samples: 379289600 total_loss: 2.914 time: 0.3712 s/iter data_time: 0.0078 s/iter total_throughput: 2758.37 samples/s lr: 1.04e-05 [09/27 23:37:02] lb.utils.events INFO: eta: 1:12:55 iteration: 370499/375342 consumed_samples: 379392000 total_loss: 2.905 time: 0.3714 s/iter data_time: 0.0058 s/iter total_throughput: 2756.99 samples/s lr: 1.04e-05 [09/27 23:38:53] lb.utils.events INFO: eta: 1:11:56 iteration: 370599/375342 consumed_samples: 379494400 total_loss: 2.913 time: 0.3716 s/iter data_time: 0.0057 s/iter total_throughput: 2755.52 samples/s lr: 1.04e-05 [09/27 23:40:33] lb.utils.events INFO: eta: 1:09:03 iteration: 370699/375342 consumed_samples: 379596800 total_loss: 2.954 time: 0.3718 s/iter data_time: 0.0077 s/iter total_throughput: 2754.26 samples/s lr: 1.04e-05 [09/27 23:42:13] lb.utils.events INFO: eta: 1:08:59 iteration: 370799/375342 consumed_samples: 379699200 total_loss: 2.954 time: 0.3720 s/iter data_time: 0.0076 s/iter total_throughput: 2753.00 samples/s lr: 1.04e-05 [09/27 23:43:44] lb.utils.events INFO: eta: 1:04:30 iteration: 370899/375342 consumed_samples: 379801600 total_loss: 2.933 time: 0.3721 s/iter data_time: 0.0100 s/iter total_throughput: 2751.93 samples/s lr: 1.03e-05 [09/27 23:45:30] lb.utils.events INFO: eta: 1:04:36 iteration: 370999/375342 consumed_samples: 379904000 total_loss: 2.913 time: 0.3723 s/iter data_time: 0.0085 s/iter total_throughput: 2750.56 samples/s lr: 1.03e-05 [09/27 23:47:03] lb.utils.events INFO: eta: 1:01:32 iteration: 371099/375342 consumed_samples: 380006400 total_loss: 2.921 time: 0.3724 s/iter data_time: 0.0058 s/iter total_throughput: 2749.46 samples/s lr: 1.03e-05 [09/27 23:48:36] lb.utils.events INFO: eta: 0:59:45 iteration: 371199/375342 consumed_samples: 380108800 total_loss: 2.94 time: 0.3726 s/iter data_time: 0.0062 s/iter total_throughput: 2748.35 samples/s lr: 1.03e-05 [09/27 23:49:54] lb.utils.events INFO: eta: 0:56:20 iteration: 371299/375342 consumed_samples: 380211200 total_loss: 2.947 time: 0.3727 s/iter data_time: 0.2210 s/iter total_throughput: 2747.55 samples/s lr: 1.03e-05 [09/27 23:51:02] lb.utils.events INFO: eta: 0:51:04 iteration: 371399/375342 consumed_samples: 380313600 total_loss: 2.927 time: 0.3728 s/iter data_time: 0.0078 s/iter total_throughput: 2746.93 samples/s lr: 1.03e-05 [09/27 23:52:39] lb.utils.events INFO: eta: 0:48:53 iteration: 371499/375342 consumed_samples: 380416000 total_loss: 2.935 time: 0.3729 s/iter data_time: 0.0054 s/iter total_throughput: 2745.73 samples/s lr: 1.03e-05 [09/27 23:54:25] lb.utils.events INFO: eta: 0:47:19 iteration: 371599/375342 consumed_samples: 380518400 total_loss: 2.936 time: 0.3731 s/iter data_time: 0.0060 s/iter total_throughput: 2744.38 samples/s lr: 1.02e-05 [09/27 23:56:03] lb.utils.events INFO: eta: 0:45:45 iteration: 371699/375342 consumed_samples: 380620800 total_loss: 2.919 time: 0.3733 s/iter data_time: 0.0062 s/iter total_throughput: 2743.18 samples/s lr: 1.02e-05 [09/27 23:57:44] lb.utils.events INFO: eta: 0:43:59 iteration: 371799/375342 consumed_samples: 380723200 total_loss: 2.935 time: 0.3735 s/iter data_time: 0.0082 s/iter total_throughput: 2741.92 samples/s lr: 1.02e-05 [09/27 23:59:25] lb.utils.events INFO: eta: 0:43:31 iteration: 371899/375342 consumed_samples: 380825600 total_loss: 2.941 time: 0.3736 s/iter data_time: 0.0054 s/iter total_throughput: 2740.68 samples/s lr: 1.02e-05 [09/28 00:00:58] lb.utils.events INFO: eta: 0:40:41 iteration: 371999/375342 consumed_samples: 380928000 total_loss: 2.925 time: 0.3738 s/iter data_time: 0.0073 s/iter total_throughput: 2739.56 samples/s lr: 1.02e-05 [09/28 00:02:43] lb.utils.events INFO: eta: 0:40:56 iteration: 372099/375342 consumed_samples: 381030400 total_loss: 2.915 time: 0.3740 s/iter data_time: 0.0059 s/iter total_throughput: 2738.25 samples/s lr: 1.02e-05 [09/28 00:04:25] lb.utils.events INFO: eta: 0:40:41 iteration: 372199/375342 consumed_samples: 381132800 total_loss: 2.93 time: 0.3741 s/iter data_time: 0.0070 s/iter total_throughput: 2736.98 samples/s lr: 1.02e-05 [09/28 00:06:00] lb.utils.events INFO: eta: 0:40:41 iteration: 372299/375342 consumed_samples: 381235200 total_loss: 2.933 time: 0.3743 s/iter data_time: 0.0062 s/iter total_throughput: 2735.84 samples/s lr: 1.02e-05 [09/28 00:07:32] lb.utils.events INFO: eta: 0:40:37 iteration: 372399/375342 consumed_samples: 381337600 total_loss: 2.93 time: 0.3744 s/iter data_time: 0.0074 s/iter total_throughput: 2734.77 samples/s lr: 1.02e-05 [09/28 00:09:13] lb.utils.events INFO: eta: 0:39:16 iteration: 372499/375342 consumed_samples: 381440000 total_loss: 2.931 time: 0.3746 s/iter data_time: 0.0066 s/iter total_throughput: 2733.52 samples/s lr: 1.01e-05 [09/28 00:11:01] lb.utils.events INFO: eta: 0:38:39 iteration: 372599/375342 consumed_samples: 381542400 total_loss: 2.936 time: 0.3748 s/iter data_time: 0.0067 s/iter total_throughput: 2732.15 samples/s lr: 1.01e-05 [09/28 00:12:50] lb.utils.events INFO: eta: 0:38:01 iteration: 372699/375342 consumed_samples: 381644800 total_loss: 2.934 time: 0.3750 s/iter data_time: 0.0055 s/iter total_throughput: 2730.75 samples/s lr: 1.01e-05 [09/28 00:14:27] lb.utils.events INFO: eta: 0:36:31 iteration: 372799/375342 consumed_samples: 381747200 total_loss: 2.926 time: 0.3751 s/iter data_time: 0.0961 s/iter total_throughput: 2729.58 samples/s lr: 1.01e-05 [09/28 00:16:07] lb.utils.events INFO: eta: 0:35:36 iteration: 372899/375342 consumed_samples: 381849600 total_loss: 2.919 time: 0.3753 s/iter data_time: 0.0059 s/iter total_throughput: 2728.37 samples/s lr: 1.01e-05 [09/28 00:18:01] lb.utils.events INFO: eta: 0:35:11 iteration: 372999/375342 consumed_samples: 381952000 total_loss: 2.925 time: 0.3755 s/iter data_time: 0.0069 s/iter total_throughput: 2726.87 samples/s lr: 1.01e-05 [09/28 00:19:33] lb.utils.events INFO: eta: 0:33:05 iteration: 373099/375342 consumed_samples: 382054400 total_loss: 2.936 time: 0.3757 s/iter data_time: 0.0080 s/iter total_throughput: 2725.81 samples/s lr: 1.01e-05 [09/28 00:21:18] lb.utils.events INFO: eta: 0:31:46 iteration: 373199/375342 consumed_samples: 382156800 total_loss: 2.934 time: 0.3758 s/iter data_time: 0.0054 s/iter total_throughput: 2724.51 samples/s lr: 1.01e-05 [09/28 00:23:04] lb.utils.events INFO: eta: 0:31:15 iteration: 373299/375342 consumed_samples: 382259200 total_loss: 2.943 time: 0.3760 s/iter data_time: 0.0054 s/iter total_throughput: 2723.18 samples/s lr: 1.01e-05 [09/28 00:24:42] lb.utils.events INFO: eta: 0:29:32 iteration: 373399/375342 consumed_samples: 382361600 total_loss: 2.935 time: 0.3762 s/iter data_time: 0.0069 s/iter total_throughput: 2722.01 samples/s lr: 1.01e-05 [09/28 00:26:17] lb.utils.events INFO: eta: 0:27:47 iteration: 373499/375342 consumed_samples: 382464000 total_loss: 2.922 time: 0.3763 s/iter data_time: 0.0060 s/iter total_throughput: 2720.89 samples/s lr: 1.01e-05 [09/28 00:27:45] lb.utils.events INFO: eta: 0:25:36 iteration: 373599/375342 consumed_samples: 382566400 total_loss: 2.9 time: 0.3765 s/iter data_time: 0.0083 s/iter total_throughput: 2719.92 samples/s lr: 1.01e-05 [09/28 00:29:29] lb.utils.events INFO: eta: 0:23:45 iteration: 373699/375342 consumed_samples: 382668800 total_loss: 2.903 time: 0.3767 s/iter data_time: 0.0069 s/iter total_throughput: 2718.65 samples/s lr: 1.00e-05 [09/28 00:31:02] lb.utils.events INFO: eta: 0:22:15 iteration: 373799/375342 consumed_samples: 382771200 total_loss: 2.923 time: 0.3768 s/iter data_time: 0.0054 s/iter total_throughput: 2717.57 samples/s lr: 1.00e-05 [09/28 00:31:54] lb.utils.events INFO: eta: 0:18:59 iteration: 373899/375342 consumed_samples: 382873600 total_loss: 2.92 time: 0.3768 s/iter data_time: 0.1383 s/iter total_throughput: 2717.31 samples/s lr: 1.00e-05 [09/28 00:33:39] lb.utils.events INFO: eta: 0:17:27 iteration: 373999/375342 consumed_samples: 382976000 total_loss: 2.94 time: 0.3770 s/iter data_time: 0.0063 s/iter total_throughput: 2716.00 samples/s lr: 1.00e-05 [09/28 00:35:21] lb.utils.events INFO: eta: 0:16:26 iteration: 374099/375342 consumed_samples: 383078400 total_loss: 2.952 time: 0.3772 s/iter data_time: 0.0067 s/iter total_throughput: 2714.76 samples/s lr: 1.00e-05 [09/28 00:37:05] lb.utils.events INFO: eta: 0:15:07 iteration: 374199/375342 consumed_samples: 383180800 total_loss: 2.924 time: 0.3774 s/iter data_time: 0.0071 s/iter total_throughput: 2713.48 samples/s lr: 1.00e-05 [09/28 00:38:56] lb.utils.events INFO: eta: 0:14:00 iteration: 374299/375342 consumed_samples: 383283200 total_loss: 2.915 time: 0.3776 s/iter data_time: 0.0055 s/iter total_throughput: 2712.08 samples/s lr: 1.00e-05 [09/28 00:40:43] lb.utils.events INFO: eta: 0:12:46 iteration: 374399/375342 consumed_samples: 383385600 total_loss: 2.918 time: 0.3778 s/iter data_time: 0.0059 s/iter total_throughput: 2710.76 samples/s lr: 1.00e-05 [09/28 00:42:30] lb.utils.events INFO: eta: 0:11:46 iteration: 374499/375342 consumed_samples: 383488000 total_loss: 2.916 time: 0.3779 s/iter data_time: 0.0056 s/iter total_throughput: 2709.44 samples/s lr: 1.00e-05 [09/28 00:44:10] lb.utils.events INFO: eta: 0:10:41 iteration: 374599/375342 consumed_samples: 383590400 total_loss: 2.916 time: 0.3781 s/iter data_time: 0.0072 s/iter total_throughput: 2708.24 samples/s lr: 1.00e-05 [09/28 00:45:43] lb.utils.events INFO: eta: 0:09:04 iteration: 374699/375342 consumed_samples: 383692800 total_loss: 2.92 time: 0.3783 s/iter data_time: 0.0072 s/iter total_throughput: 2707.19 samples/s lr: 1.00e-05 [09/28 00:47:17] lb.utils.events INFO: eta: 0:07:36 iteration: 374799/375342 consumed_samples: 383795200 total_loss: 2.916 time: 0.3784 s/iter data_time: 0.0068 s/iter total_throughput: 2706.11 samples/s lr: 1.00e-05 [09/28 00:48:55] lb.utils.events INFO: eta: 0:06:43 iteration: 374899/375342 consumed_samples: 383897600 total_loss: 2.934 time: 0.3786 s/iter data_time: 0.0072 s/iter total_throughput: 2704.96 samples/s lr: 1.00e-05 [09/28 00:50:30] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0374999 [09/28 00:50:31] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/28 00:50:31] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/28 00:50:37] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0003 s/iter. Inference: 0.4151 s/iter. Eval: 0.0068 s/iter. Total: 0.4222 s/iter. ETA=0:00:15 [09/28 00:50:42] lb.evaluation.evaluator INFO: Inference done 24576/50000. Dataloading: 0.0004 s/iter. Inference: 0.4012 s/iter. Eval: 0.0062 s/iter. Total: 0.4079 s/iter. ETA=0:00:09 [09/28 00:50:47] lb.evaluation.evaluator INFO: Inference done 37888/50000. Dataloading: 0.0005 s/iter. Inference: 0.4015 s/iter. Eval: 0.0056 s/iter. Total: 0.4077 s/iter. ETA=0:00:04 [09/28 00:50:52] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/28 00:50:52] lb.evaluation.evaluator INFO: Total inference time: 0:00:17.861301 (0.000357 s / iter per device, on 8 devices) [09/28 00:50:52] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:17 (0.000352 s / iter per device, on 8 devices) [09/28 00:50:52] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/28 00:50:52] lb.evaluation.utils INFO: copypaste: Acc@1=80.012 [09/28 00:50:52] lb.evaluation.utils INFO: copypaste: Acc@5=94.434 [09/28 00:50:52] lb.engine.hooks INFO: Not saving as latest eval score for Acc@1 is 80.01200, not better than best score 80.08600 @ iteration 364999. [09/28 00:50:52] lb.utils.events INFO: eta: 0:05:07 iteration: 374999/375342 consumed_samples: 384000000 total_loss: 2.952 time: 0.3787 s/iter data_time: 0.0093 s/iter total_throughput: 2703.89 samples/s lr: 1.00e-05 [09/28 00:52:33] lb.utils.events INFO: eta: 0:03:34 iteration: 375099/375342 consumed_samples: 384102400 total_loss: 2.948 time: 0.3789 s/iter data_time: 0.0066 s/iter total_throughput: 2702.68 samples/s lr: 1.00e-05 [09/28 00:54:16] lb.utils.events INFO: eta: 0:02:05 iteration: 375199/375342 consumed_samples: 384204800 total_loss: 2.932 time: 0.3791 s/iter data_time: 0.0079 s/iter total_throughput: 2701.46 samples/s lr: 1.00e-05 [09/28 00:55:48] lb.utils.events INFO: eta: 0:00:35 iteration: 375299/375342 consumed_samples: 384307200 total_loss: 2.914 time: 0.3792 s/iter data_time: 0.0190 s/iter total_throughput: 2700.43 samples/s lr: 1.00e-05 [09/28 00:56:23] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_final [09/28 00:56:25] lb.utils.events INFO: eta: 0:00:00 iteration: 375341/375342 consumed_samples: 384350208 total_loss: 2.931 time: 0.3792 s/iter data_time: 0.0089 s/iter total_throughput: 2700.08 samples/s lr: 1.00e-05 [09/28 00:56:25] lb.engine.hooks INFO: Overall training speed: 375340 iterations in 1 day, 15:32:31 (0.3793 s / it) [09/28 00:56:25] lb.engine.hooks INFO: Total training time: 1 day, 15:54:37 (0:22:05 on hooks) [09/28 00:56:25] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/28 00:56:25] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/28 00:56:30] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0002 s/iter. Inference: 0.3902 s/iter. Eval: 0.0044 s/iter. Total: 0.3948 s/iter. ETA=0:00:14 [09/28 00:56:36] lb.evaluation.evaluator INFO: Inference done 25600/50000. Dataloading: 0.0004 s/iter. Inference: 0.3832 s/iter. Eval: 0.0051 s/iter. Total: 0.3888 s/iter. ETA=0:00:08 [09/28 00:56:41] lb.evaluation.evaluator INFO: Inference done 38912/50000. Dataloading: 0.0005 s/iter. Inference: 0.3859 s/iter. Eval: 0.0050 s/iter. Total: 0.3915 s/iter. ETA=0:00:03 [09/28 00:56:45] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/28 00:56:45] lb.evaluation.evaluator INFO: Total inference time: 0:00:17.278965 (0.000346 s / iter per device, on 8 devices) [09/28 00:56:45] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:17 (0.000341 s / iter per device, on 8 devices) [09/28 00:56:45] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/28 00:56:45] lb.evaluation.utils INFO: copypaste: Acc@1=80.002 [09/28 00:56:45] lb.evaluation.utils INFO: copypaste: Acc@5=94.48 [09/28 00:56:45] lb.engine.hooks INFO: Not saving as latest eval score for Acc@1 is 80.00200, not better than best score 80.08600 @ iteration 364999.