[09/22 03:16:44] libai INFO: Rank of current process: 0. World size: 8 [09/22 03:16:44] libai INFO: Command line arguments: Namespace(config_file='configs/swin_imagenet.py', eval_only=False, fast_dev_run=False, opts=[], resume=False) [09/22 03:16:44] libai INFO: Contents of args.config_file=configs/swin_imagenet.py: from libai.config import LazyCall from .common.models.swin.swin_tiny_patch4_window7_224 import model from .common.models.graph import graph from .common.train import train from .common.optim import optim from .common.data.imagenet import dataloader from flowvision.data import Mixup from flowvision.loss.cross_entropy import SoftTargetCrossEntropy # Refine data path to imagenet dataloader.train.dataset[0].root = "/data/ImageNet/extract/" dataloader.test[0].dataset.root = "/data/ImageNet/extract/" # Add Mixup Func dataloader.train.mixup_func = LazyCall(Mixup)(  mixup_alpha=0.8,  cutmix_alpha=1.0,  prob=1.0,  switch_prob=0.5,  mode="batch",  num_classes=1000, ) # Refine model cfg for vit training on imagenet model.cfg.num_classes = 1000 model.cfg.loss_func = SoftTargetCrossEntropy() # Refine optimizer cfg for vit model optim.lr = 1e-3 optim.eps = 1e-8 optim.weight_decay = 0.05 # Refine train cfg for vit model train.train_micro_batch_size = 128 train.test_micro_batch_size = 128 train.train_epoch = 300 train.warmup_ratio = 20 / 300 train.eval_period = 1000 train.log_period = 100 train.output_dir = "./commit_swin" # Scheduler train.scheduler.warmup_factor = 0.001 train.scheduler.alpha = 0.01 train.scheduler.warmup_method = "linear" # Set fp16 ON train.amp.enabled = True [09/22 03:16:44] libai INFO: Full config saved to ./commit_swin/config.yaml [09/22 03:16:44] lb.engine.default INFO: > compiling dataset index builder ... [09/22 03:16:44] lb.engine.default INFO: >>> done with dataset index builder. Compilation time: 0.072 seconds [09/22 03:16:44] lb.engine.default INFO: >>> done with compiling. Compilation time: 0.073 seconds [09/22 03:16:44] lb.engine.default INFO: Prepare training, validating, testing set [09/22 03:16:47] lb.engine.default INFO: Prepare testing set [09/22 03:16:47] lb.engine.default INFO: Auto-scaling the config to train.train_iter=375342, train.warmup_iter=25023 [09/22 03:16:57] lb.engine.default INFO: Model: SwinTransformer( (patch_embed): PatchEmbed( (proj): Conv2d(3, 96, kernel_size=(4, 4), stride=(4, 4)) (norm): LayerNorm((96,), eps=1e-05, elementwise_affine=True) ) (pos_drop): Dropout(p=0.0, inplace=False) (layers): ModuleList( (0): BasicLayer( (blocks): ModuleList( (0): SwinTransformerBlock( (norm1): LayerNorm((96,), eps=1e-05, elementwise_affine=True) (attn): WindowAttention( (qkv): Linear1D(in_features=96, out_features=288, bias=True, parallel=data) (attn_drop): Dropout(p=0.0, inplace=False) (proj): Linear1D(in_features=96, out_features=96, bias=True, parallel=data) (proj_drop): Dropout(p=0.0, inplace=False) (softmax): Softmax(dim=-1) ) (drop_path): Identity() (norm2): LayerNorm((96,), eps=1e-05, elementwise_affine=True) (mlp): MLP( bias_gelu_fusion=True, bias_dropout_fusion=True, dropout=0.0 (dense_h_to_4h): Linear1D(in_features=96, out_features=384, bias=True, parallel=col) (dense_4h_to_h): Linear1D(in_features=384, out_features=96, bias=True, parallel=row) ) ) (1): SwinTransformerBlock( (norm1): LayerNorm((96,), eps=1e-05, elementwise_affine=True) (attn): WindowAttention( (qkv): Linear1D(in_features=96, out_features=288, bias=True, parallel=data) (attn_drop): Dropout(p=0.0, inplace=False) (proj): Linear1D(in_features=96, out_features=96, bias=True, parallel=data) (proj_drop): Dropout(p=0.0, inplace=False) (softmax): Softmax(dim=-1) ) (drop_path): DropPath() (norm2): LayerNorm((96,), eps=1e-05, elementwise_affine=True) (mlp): MLP( bias_gelu_fusion=True, bias_dropout_fusion=True, dropout=0.0 (dense_h_to_4h): Linear1D(in_features=96, out_features=384, bias=True, parallel=col) (dense_4h_to_h): Linear1D(in_features=384, out_features=96, bias=True, parallel=row) ) ) ) (downsample): PatchMerging( (reduction): Linear1D(in_features=384, out_features=192, bias=False, parallel=data) (norm): LayerNorm((384,), eps=1e-05, elementwise_affine=True) ) ) (1): BasicLayer( (blocks): ModuleList( (0): SwinTransformerBlock( (norm1): LayerNorm((192,), eps=1e-05, elementwise_affine=True) (attn): WindowAttention( (qkv): Linear1D(in_features=192, out_features=576, bias=True, parallel=data) (attn_drop): Dropout(p=0.0, inplace=False) (proj): Linear1D(in_features=192, out_features=192, bias=True, parallel=data) (proj_drop): Dropout(p=0.0, inplace=False) (softmax): Softmax(dim=-1) ) (drop_path): DropPath() (norm2): LayerNorm((192,), eps=1e-05, elementwise_affine=True) (mlp): MLP( bias_gelu_fusion=True, bias_dropout_fusion=True, dropout=0.0 (dense_h_to_4h): Linear1D(in_features=192, out_features=768, bias=True, parallel=col) (dense_4h_to_h): Linear1D(in_features=768, out_features=192, bias=True, parallel=row) ) ) (1): SwinTransformerBlock( (norm1): LayerNorm((192,), eps=1e-05, elementwise_affine=True) (attn): WindowAttention( (qkv): Linear1D(in_features=192, out_features=576, bias=True, parallel=data) (attn_drop): Dropout(p=0.0, inplace=False) (proj): Linear1D(in_features=192, out_features=192, bias=True, parallel=data) (proj_drop): Dropout(p=0.0, inplace=False) (softmax): Softmax(dim=-1) ) (drop_path): DropPath() (norm2): LayerNorm((192,), eps=1e-05, elementwise_affine=True) (mlp): MLP( bias_gelu_fusion=True, bias_dropout_fusion=True, dropout=0.0 (dense_h_to_4h): Linear1D(in_features=192, out_features=768, bias=True, parallel=col) (dense_4h_to_h): Linear1D(in_features=768, out_features=192, bias=True, parallel=row) ) ) ) (downsample): PatchMerging( (reduction): Linear1D(in_features=768, out_features=384, bias=False, parallel=data) (norm): LayerNorm((768,), eps=1e-05, elementwise_affine=True) ) ) (2): BasicLayer( (blocks): ModuleList( (0): SwinTransformerBlock( (norm1): LayerNorm((384,), eps=1e-05, elementwise_affine=True) (attn): WindowAttention( (qkv): Linear1D(in_features=384, out_features=1152, bias=True, parallel=data) (attn_drop): Dropout(p=0.0, inplace=False) (proj): Linear1D(in_features=384, out_features=384, bias=True, parallel=data) (proj_drop): Dropout(p=0.0, inplace=False) (softmax): Softmax(dim=-1) ) (drop_path): DropPath() (norm2): LayerNorm((384,), eps=1e-05, elementwise_affine=True) (mlp): MLP( bias_gelu_fusion=True, bias_dropout_fusion=True, dropout=0.0 (dense_h_to_4h): Linear1D(in_features=384, out_features=1536, bias=True, parallel=col) (dense_4h_to_h): Linear1D(in_features=1536, out_features=384, bias=True, parallel=row) ) ) (1): SwinTransformerBlock( (norm1): LayerNorm((384,), eps=1e-05, elementwise_affine=True) (attn): WindowAttention( (qkv): Linear1D(in_features=384, out_features=1152, bias=True, parallel=data) (attn_drop): Dropout(p=0.0, inplace=False) (proj): Linear1D(in_features=384, out_features=384, bias=True, parallel=data) (proj_drop): Dropout(p=0.0, inplace=False) (softmax): Softmax(dim=-1) ) (drop_path): DropPath() (norm2): LayerNorm((384,), eps=1e-05, elementwise_affine=True) (mlp): MLP( bias_gelu_fusion=True, bias_dropout_fusion=True, dropout=0.0 (dense_h_to_4h): Linear1D(in_features=384, out_features=1536, bias=True, parallel=col) (dense_4h_to_h): Linear1D(in_features=1536, out_features=384, bias=True, parallel=row) ) ) (2): SwinTransformerBlock( (norm1): LayerNorm((384,), eps=1e-05, elementwise_affine=True) (attn): WindowAttention( (qkv): Linear1D(in_features=384, out_features=1152, bias=True, parallel=data) (attn_drop): Dropout(p=0.0, inplace=False) (proj): Linear1D(in_features=384, out_features=384, bias=True, parallel=data) (proj_drop): Dropout(p=0.0, inplace=False) (softmax): Softmax(dim=-1) ) (drop_path): DropPath() (norm2): LayerNorm((384,), eps=1e-05, elementwise_affine=True) (mlp): MLP( bias_gelu_fusion=True, bias_dropout_fusion=True, dropout=0.0 (dense_h_to_4h): Linear1D(in_features=384, out_features=1536, bias=True, parallel=col) (dense_4h_to_h): Linear1D(in_features=1536, out_features=384, bias=True, parallel=row) ) ) (3): SwinTransformerBlock( (norm1): LayerNorm((384,), eps=1e-05, elementwise_affine=True) (attn): WindowAttention( (qkv): Linear1D(in_features=384, out_features=1152, bias=True, parallel=data) (attn_drop): Dropout(p=0.0, inplace=False) (proj): Linear1D(in_features=384, out_features=384, bias=True, parallel=data) (proj_drop): Dropout(p=0.0, inplace=False) (softmax): Softmax(dim=-1) ) (drop_path): DropPath() (norm2): LayerNorm((384,), eps=1e-05, elementwise_affine=True) (mlp): MLP( bias_gelu_fusion=True, bias_dropout_fusion=True, dropout=0.0 (dense_h_to_4h): Linear1D(in_features=384, out_features=1536, bias=True, parallel=col) (dense_4h_to_h): Linear1D(in_features=1536, out_features=384, bias=True, parallel=row) ) ) (4): SwinTransformerBlock( (norm1): LayerNorm((384,), eps=1e-05, elementwise_affine=True) (attn): WindowAttention( (qkv): Linear1D(in_features=384, out_features=1152, bias=True, parallel=data) (attn_drop): Dropout(p=0.0, inplace=False) (proj): Linear1D(in_features=384, out_features=384, bias=True, parallel=data) (proj_drop): Dropout(p=0.0, inplace=False) (softmax): Softmax(dim=-1) ) (drop_path): DropPath() (norm2): LayerNorm((384,), eps=1e-05, elementwise_affine=True) (mlp): MLP( bias_gelu_fusion=True, bias_dropout_fusion=True, dropout=0.0 (dense_h_to_4h): Linear1D(in_features=384, out_features=1536, bias=True, parallel=col) (dense_4h_to_h): Linear1D(in_features=1536, out_features=384, bias=True, parallel=row) ) ) (5): SwinTransformerBlock( (norm1): LayerNorm((384,), eps=1e-05, elementwise_affine=True) (attn): WindowAttention( (qkv): Linear1D(in_features=384, out_features=1152, bias=True, parallel=data) (attn_drop): Dropout(p=0.0, inplace=False) (proj): Linear1D(in_features=384, out_features=384, bias=True, parallel=data) (proj_drop): Dropout(p=0.0, inplace=False) (softmax): Softmax(dim=-1) ) (drop_path): DropPath() (norm2): LayerNorm((384,), eps=1e-05, elementwise_affine=True) (mlp): MLP( bias_gelu_fusion=True, bias_dropout_fusion=True, dropout=0.0 (dense_h_to_4h): Linear1D(in_features=384, out_features=1536, bias=True, parallel=col) (dense_4h_to_h): Linear1D(in_features=1536, out_features=384, bias=True, parallel=row) ) ) ) (downsample): PatchMerging( (reduction): Linear1D(in_features=1536, out_features=768, bias=False, parallel=data) (norm): LayerNorm((1536,), eps=1e-05, elementwise_affine=True) ) ) (3): BasicLayer( (blocks): ModuleList( (0): SwinTransformerBlock( (norm1): LayerNorm((768,), eps=1e-05, elementwise_affine=True) (attn): WindowAttention( (qkv): Linear1D(in_features=768, out_features=2304, bias=True, parallel=data) (attn_drop): Dropout(p=0.0, inplace=False) (proj): Linear1D(in_features=768, out_features=768, bias=True, parallel=data) (proj_drop): Dropout(p=0.0, inplace=False) (softmax): Softmax(dim=-1) ) (drop_path): DropPath() (norm2): LayerNorm((768,), eps=1e-05, elementwise_affine=True) (mlp): MLP( bias_gelu_fusion=True, bias_dropout_fusion=True, dropout=0.0 (dense_h_to_4h): Linear1D(in_features=768, out_features=3072, bias=True, parallel=col) (dense_4h_to_h): Linear1D(in_features=3072, out_features=768, bias=True, parallel=row) ) ) (1): SwinTransformerBlock( (norm1): LayerNorm((768,), eps=1e-05, elementwise_affine=True) (attn): WindowAttention( (qkv): Linear1D(in_features=768, out_features=2304, bias=True, parallel=data) (attn_drop): Dropout(p=0.0, inplace=False) (proj): Linear1D(in_features=768, out_features=768, bias=True, parallel=data) (proj_drop): Dropout(p=0.0, inplace=False) (softmax): Softmax(dim=-1) ) (drop_path): DropPath() (norm2): LayerNorm((768,), eps=1e-05, elementwise_affine=True) (mlp): MLP( bias_gelu_fusion=True, bias_dropout_fusion=True, dropout=0.0 (dense_h_to_4h): Linear1D(in_features=768, out_features=3072, bias=True, parallel=col) (dense_4h_to_h): Linear1D(in_features=3072, out_features=768, bias=True, parallel=row) ) ) ) ) ) (norm): LayerNorm((768,), eps=1e-05, elementwise_affine=True) (avgpool): AdaptiveAvgPool1d() (head): Linear1D(in_features=768, out_features=1000, bias=True, parallel=data) (loss_func): SoftTargetCrossEntropy() ) [09/22 03:16:57] lb.engine.trainer INFO: Starting training from iteration 0 [09/22 03:16:59] lb.models.utils.graph_base INFO: Start compling the train graph which may take some time. Please wait for a moment ... [09/22 03:17:58] lb.utils.events INFO: eta: 20:14:30 iteration: 99/375342 consumed_samples: 102400 total_loss: 6.957 time: 0.3083 s/iter data_time: 0.2007 s/iter total_throughput: 3321.65 samples/s lr: 4.91e-06 [09/22 03:18:30] lb.utils.events INFO: eta: 19:37:07 iteration: 199/375342 consumed_samples: 204800 total_loss: 6.934 time: 0.3150 s/iter data_time: 0.2308 s/iter total_throughput: 3250.30 samples/s lr: 8.86e-06 [09/22 03:19:03] lb.utils.events INFO: eta: 17:18:23 iteration: 299/375342 consumed_samples: 307200 total_loss: 6.906 time: 0.3194 s/iter data_time: 0.2047 s/iter total_throughput: 3205.89 samples/s lr: 1.28e-05 [09/22 03:19:36] lb.utils.events INFO: eta: 16:53:55 iteration: 399/375342 consumed_samples: 409600 total_loss: 6.883 time: 0.3227 s/iter data_time: 0.2024 s/iter total_throughput: 3173.23 samples/s lr: 1.68e-05 [09/22 03:20:08] lb.utils.events INFO: eta: 16:46:37 iteration: 499/375342 consumed_samples: 512000 total_loss: 6.863 time: 0.3233 s/iter data_time: 0.2065 s/iter total_throughput: 3167.36 samples/s lr: 2.07e-05 [09/22 03:20:41] lb.utils.events INFO: eta: 16:42:38 iteration: 599/375342 consumed_samples: 614400 total_loss: 6.844 time: 0.3244 s/iter data_time: 0.2081 s/iter total_throughput: 3157.03 samples/s lr: 2.47e-05 [09/22 03:21:14] lb.utils.events INFO: eta: 16:38:43 iteration: 699/375342 consumed_samples: 716800 total_loss: 6.828 time: 0.3251 s/iter data_time: 0.1993 s/iter total_throughput: 3149.40 samples/s lr: 2.86e-05 [09/22 03:21:48] lb.utils.events INFO: eta: 16:34:41 iteration: 799/375342 consumed_samples: 819200 total_loss: 6.809 time: 0.3261 s/iter data_time: 0.2054 s/iter total_throughput: 3139.70 samples/s lr: 3.26e-05 [09/22 03:22:21] lb.utils.events INFO: eta: 16:31:05 iteration: 899/375342 consumed_samples: 921600 total_loss: 6.785 time: 0.3269 s/iter data_time: 0.2209 s/iter total_throughput: 3132.39 samples/s lr: 3.65e-05 [09/22 03:22:54] lb.utils.events INFO: eta: 16:29:58 iteration: 999/375342 consumed_samples: 1024000 total_loss: 6.766 time: 0.3273 s/iter data_time: 0.1994 s/iter total_throughput: 3128.48 samples/s lr: 4.05e-05 [09/22 03:23:28] lb.utils.events INFO: eta: 16:23:12 iteration: 1099/375342 consumed_samples: 1126400 total_loss: 6.738 time: 0.3280 s/iter data_time: 0.2107 s/iter total_throughput: 3121.61 samples/s lr: 4.44e-05 [09/22 03:24:01] lb.utils.events INFO: eta: 16:17:43 iteration: 1199/375342 consumed_samples: 1228800 total_loss: 6.711 time: 0.3285 s/iter data_time: 0.2072 s/iter total_throughput: 3117.34 samples/s lr: 4.83e-05 [09/22 03:24:34] lb.utils.events INFO: eta: 16:17:05 iteration: 1299/375342 consumed_samples: 1331200 total_loss: 6.672 time: 0.3287 s/iter data_time: 0.2140 s/iter total_throughput: 3114.92 samples/s lr: 5.23e-05 [09/22 03:25:07] lb.utils.events INFO: eta: 16:12:45 iteration: 1399/375342 consumed_samples: 1433600 total_loss: 6.641 time: 0.3290 s/iter data_time: 0.2060 s/iter total_throughput: 3112.35 samples/s lr: 5.62e-05 [09/22 03:25:40] lb.utils.events INFO: eta: 16:10:41 iteration: 1499/375342 consumed_samples: 1536000 total_loss: 6.609 time: 0.3290 s/iter data_time: 0.2021 s/iter total_throughput: 3112.22 samples/s lr: 6.02e-05 [09/22 03:26:14] lb.utils.events INFO: eta: 16:10:54 iteration: 1599/375342 consumed_samples: 1638400 total_loss: 6.582 time: 0.3292 s/iter data_time: 0.2150 s/iter total_throughput: 3110.49 samples/s lr: 6.41e-05 [09/22 03:26:47] lb.utils.events INFO: eta: 16:10:38 iteration: 1699/375342 consumed_samples: 1740800 total_loss: 6.559 time: 0.3293 s/iter data_time: 0.2074 s/iter total_throughput: 3109.33 samples/s lr: 6.81e-05 [09/22 03:27:20] lb.utils.events INFO: eta: 16:12:51 iteration: 1799/375342 consumed_samples: 1843200 total_loss: 6.531 time: 0.3294 s/iter data_time: 0.1994 s/iter total_throughput: 3109.01 samples/s lr: 7.20e-05 [09/22 03:27:53] lb.utils.events INFO: eta: 16:16:52 iteration: 1899/375342 consumed_samples: 1945600 total_loss: 6.506 time: 0.3295 s/iter data_time: 0.2033 s/iter total_throughput: 3108.16 samples/s lr: 7.60e-05 [09/22 03:28:26] lb.utils.events INFO: eta: 16:13:50 iteration: 1999/375342 consumed_samples: 2048000 total_loss: 6.484 time: 0.3298 s/iter data_time: 0.2064 s/iter total_throughput: 3105.33 samples/s lr: 7.99e-05 [09/22 03:29:00] lb.utils.events INFO: eta: 16:13:35 iteration: 2099/375342 consumed_samples: 2150400 total_loss: 6.461 time: 0.3301 s/iter data_time: 0.2166 s/iter total_throughput: 3101.93 samples/s lr: 8.39e-05 [09/22 03:29:34] lb.utils.events INFO: eta: 16:13:01 iteration: 2199/375342 consumed_samples: 2252800 total_loss: 6.441 time: 0.3304 s/iter data_time: 0.2113 s/iter total_throughput: 3099.49 samples/s lr: 8.78e-05 [09/22 03:30:07] lb.utils.events INFO: eta: 16:12:27 iteration: 2299/375342 consumed_samples: 2355200 total_loss: 6.426 time: 0.3304 s/iter data_time: 0.2105 s/iter total_throughput: 3099.14 samples/s lr: 9.18e-05 [09/22 03:30:40] lb.utils.events INFO: eta: 16:17:46 iteration: 2399/375342 consumed_samples: 2457600 total_loss: 6.404 time: 0.3305 s/iter data_time: 0.2137 s/iter total_throughput: 3098.42 samples/s lr: 9.57e-05 [09/22 03:31:13] lb.utils.events INFO: eta: 16:18:22 iteration: 2499/375342 consumed_samples: 2560000 total_loss: 6.383 time: 0.3304 s/iter data_time: 0.2032 s/iter total_throughput: 3099.20 samples/s lr: 9.97e-05 [09/22 03:31:46] lb.utils.events INFO: eta: 16:17:44 iteration: 2599/375342 consumed_samples: 2662400 total_loss: 6.359 time: 0.3305 s/iter data_time: 0.2189 s/iter total_throughput: 3098.58 samples/s lr: 1.04e-04 [09/22 03:32:20] lb.utils.events INFO: eta: 16:17:28 iteration: 2699/375342 consumed_samples: 2764800 total_loss: 6.34 time: 0.3307 s/iter data_time: 0.2082 s/iter total_throughput: 3096.50 samples/s lr: 1.08e-04 [09/22 03:32:53] lb.utils.events INFO: eta: 16:20:57 iteration: 2799/375342 consumed_samples: 2867200 total_loss: 6.324 time: 0.3308 s/iter data_time: 0.2037 s/iter total_throughput: 3095.22 samples/s lr: 1.12e-04 [09/22 03:33:26] lb.utils.events INFO: eta: 16:17:11 iteration: 2899/375342 consumed_samples: 2969600 total_loss: 6.312 time: 0.3308 s/iter data_time: 0.2030 s/iter total_throughput: 3095.47 samples/s lr: 1.15e-04 [09/22 03:34:00] lb.utils.events INFO: eta: 16:19:07 iteration: 2999/375342 consumed_samples: 3072000 total_loss: 6.297 time: 0.3309 s/iter data_time: 0.2114 s/iter total_throughput: 3094.35 samples/s lr: 1.19e-04 [09/22 03:34:33] lb.utils.events INFO: eta: 16:21:54 iteration: 3099/375342 consumed_samples: 3174400 total_loss: 6.277 time: 0.3310 s/iter data_time: 0.1962 s/iter total_throughput: 3093.75 samples/s lr: 1.23e-04 [09/22 03:35:06] lb.utils.events INFO: eta: 16:23:46 iteration: 3199/375342 consumed_samples: 3276800 total_loss: 6.262 time: 0.3311 s/iter data_time: 0.1944 s/iter total_throughput: 3093.07 samples/s lr: 1.27e-04 [09/22 03:35:40] lb.utils.events INFO: eta: 16:24:11 iteration: 3299/375342 consumed_samples: 3379200 total_loss: 6.24 time: 0.3312 s/iter data_time: 0.2080 s/iter total_throughput: 3091.95 samples/s lr: 1.31e-04 [09/22 03:36:13] lb.utils.events INFO: eta: 16:23:14 iteration: 3399/375342 consumed_samples: 3481600 total_loss: 6.213 time: 0.3312 s/iter data_time: 0.2102 s/iter total_throughput: 3091.69 samples/s lr: 1.35e-04 [09/22 03:36:47] lb.utils.events INFO: eta: 16:21:10 iteration: 3499/375342 consumed_samples: 3584000 total_loss: 6.197 time: 0.3315 s/iter data_time: 0.2077 s/iter total_throughput: 3088.76 samples/s lr: 1.39e-04 [09/22 03:37:21] lb.utils.events INFO: eta: 16:20:58 iteration: 3599/375342 consumed_samples: 3686400 total_loss: 6.193 time: 0.3318 s/iter data_time: 0.2105 s/iter total_throughput: 3086.24 samples/s lr: 1.43e-04 [09/22 03:37:55] lb.utils.events INFO: eta: 16:20:17 iteration: 3699/375342 consumed_samples: 3788800 total_loss: 6.172 time: 0.3319 s/iter data_time: 0.2094 s/iter total_throughput: 3084.81 samples/s lr: 1.47e-04 [09/22 03:38:29] lb.utils.events INFO: eta: 16:18:53 iteration: 3799/375342 consumed_samples: 3891200 total_loss: 6.137 time: 0.3321 s/iter data_time: 0.2088 s/iter total_throughput: 3083.78 samples/s lr: 1.51e-04 [09/22 03:39:03] lb.utils.events INFO: eta: 16:19:54 iteration: 3899/375342 consumed_samples: 3993600 total_loss: 6.125 time: 0.3322 s/iter data_time: 0.2053 s/iter total_throughput: 3082.69 samples/s lr: 1.55e-04 [09/22 03:39:36] lb.utils.events INFO: eta: 16:19:38 iteration: 3999/375342 consumed_samples: 4096000 total_loss: 6.109 time: 0.3323 s/iter data_time: 0.2136 s/iter total_throughput: 3081.12 samples/s lr: 1.59e-04 [09/22 03:40:11] lb.utils.events INFO: eta: 16:16:41 iteration: 4099/375342 consumed_samples: 4198400 total_loss: 6.092 time: 0.3326 s/iter data_time: 0.2224 s/iter total_throughput: 3078.50 samples/s lr: 1.63e-04 [09/22 03:40:45] lb.utils.events INFO: eta: 16:14:11 iteration: 4199/375342 consumed_samples: 4300800 total_loss: 6.078 time: 0.3327 s/iter data_time: 0.2098 s/iter total_throughput: 3077.44 samples/s lr: 1.67e-04 [09/22 03:41:19] lb.utils.events INFO: eta: 16:13:55 iteration: 4299/375342 consumed_samples: 4403200 total_loss: 6.064 time: 0.3329 s/iter data_time: 0.2061 s/iter total_throughput: 3075.84 samples/s lr: 1.71e-04 [09/22 03:41:52] lb.utils.events INFO: eta: 16:13:21 iteration: 4399/375342 consumed_samples: 4505600 total_loss: 6.043 time: 0.3329 s/iter data_time: 0.2036 s/iter total_throughput: 3075.65 samples/s lr: 1.75e-04 [09/22 03:42:26] lb.utils.events INFO: eta: 16:14:16 iteration: 4499/375342 consumed_samples: 4608000 total_loss: 6.027 time: 0.3331 s/iter data_time: 0.2072 s/iter total_throughput: 3074.49 samples/s lr: 1.79e-04 [09/22 03:43:00] lb.utils.events INFO: eta: 16:13:30 iteration: 4599/375342 consumed_samples: 4710400 total_loss: 6.016 time: 0.3332 s/iter data_time: 0.2121 s/iter total_throughput: 3073.40 samples/s lr: 1.83e-04 [09/22 03:43:34] lb.utils.events INFO: eta: 16:14:02 iteration: 4699/375342 consumed_samples: 4812800 total_loss: 5.994 time: 0.3333 s/iter data_time: 0.2129 s/iter total_throughput: 3072.24 samples/s lr: 1.87e-04 [09/22 03:44:08] lb.utils.events INFO: eta: 16:10:45 iteration: 4799/375342 consumed_samples: 4915200 total_loss: 5.984 time: 0.3335 s/iter data_time: 0.2114 s/iter total_throughput: 3070.78 samples/s lr: 1.91e-04 [09/22 03:44:41] lb.utils.events INFO: eta: 16:09:24 iteration: 4899/375342 consumed_samples: 5017600 total_loss: 5.977 time: 0.3335 s/iter data_time: 0.2042 s/iter total_throughput: 3070.05 samples/s lr: 1.94e-04 [09/22 03:45:16] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0004999 [09/22 03:45:16] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/22 03:45:16] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/22 03:45:18] lb.models.utils.graph_base INFO: Start compling the eval graph which may take some time. Please wait for a moment ... [09/22 03:45:23] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0205 s/iter. Inference: 0.1834 s/iter. Eval: 0.0021 s/iter. Total: 0.2060 s/iter. ETA=0:00:07 [09/22 03:45:28] lb.evaluation.evaluator INFO: Inference done 28672/50000. Dataloading: 0.0498 s/iter. Inference: 0.2204 s/iter. Eval: 0.0020 s/iter. Total: 0.2723 s/iter. ETA=0:00:05 [09/22 03:45:33] lb.evaluation.evaluator INFO: Inference done 47104/50000. Dataloading: 0.0296 s/iter. Inference: 0.2447 s/iter. Eval: 0.0020 s/iter. Total: 0.2764 s/iter. ETA=0:00:00 [09/22 03:45:34] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/22 03:45:34] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.160068 (0.000243 s / iter per device, on 8 devices) [09/22 03:45:34] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:10 (0.000217 s / iter per device, on 8 devices) [09/22 03:45:34] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/22 03:45:34] lb.evaluation.utils INFO: copypaste: Acc@1=16.724 [09/22 03:45:34] lb.evaluation.utils INFO: copypaste: Acc@5=36.738 [09/22 03:45:34] lb.engine.hooks INFO: Saved first model at 16.72400 @ 4999 steps [09/22 03:45:34] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/22 03:45:34] lb.utils.events INFO: eta: 16:09:24 iteration: 4999/375342 consumed_samples: 5120000 total_loss: 5.955 time: 0.3337 s/iter data_time: 0.2181 s/iter total_throughput: 3068.52 samples/s lr: 1.98e-04 [09/22 03:46:05] lb.utils.events INFO: eta: 16:15:42 iteration: 5099/375342 consumed_samples: 5222400 total_loss: 5.934 time: 0.3332 s/iter data_time: 0.2531 s/iter total_throughput: 3073.00 samples/s lr: 2.02e-04 [09/22 03:46:38] lb.utils.events INFO: eta: 16:15:55 iteration: 5199/375342 consumed_samples: 5324800 total_loss: 5.91 time: 0.3332 s/iter data_time: 0.2051 s/iter total_throughput: 3073.44 samples/s lr: 2.06e-04 [09/22 03:47:11] lb.utils.events INFO: eta: 16:15:50 iteration: 5299/375342 consumed_samples: 5427200 total_loss: 5.883 time: 0.3331 s/iter data_time: 0.2123 s/iter total_throughput: 3074.15 samples/s lr: 2.10e-04 [09/22 03:47:45] lb.utils.events INFO: eta: 16:15:34 iteration: 5399/375342 consumed_samples: 5529600 total_loss: 5.883 time: 0.3331 s/iter data_time: 0.2290 s/iter total_throughput: 3073.95 samples/s lr: 2.14e-04 [09/22 03:48:18] lb.utils.events INFO: eta: 16:18:28 iteration: 5499/375342 consumed_samples: 5632000 total_loss: 5.879 time: 0.3330 s/iter data_time: 0.1937 s/iter total_throughput: 3074.84 samples/s lr: 2.18e-04 [09/22 03:48:50] lb.utils.events INFO: eta: 16:19:46 iteration: 5599/375342 consumed_samples: 5734400 total_loss: 5.857 time: 0.3329 s/iter data_time: 0.1989 s/iter total_throughput: 3075.76 samples/s lr: 2.22e-04 [09/22 03:49:23] lb.utils.events INFO: eta: 16:15:26 iteration: 5699/375342 consumed_samples: 5836800 total_loss: 5.84 time: 0.3328 s/iter data_time: 0.1980 s/iter total_throughput: 3076.57 samples/s lr: 2.26e-04 [09/22 03:49:56] lb.utils.events INFO: eta: 16:17:28 iteration: 5799/375342 consumed_samples: 5939200 total_loss: 5.812 time: 0.3328 s/iter data_time: 0.2024 s/iter total_throughput: 3077.22 samples/s lr: 2.30e-04 [09/22 03:50:29] lb.utils.events INFO: eta: 16:18:08 iteration: 5899/375342 consumed_samples: 6041600 total_loss: 5.801 time: 0.3327 s/iter data_time: 0.1965 s/iter total_throughput: 3078.12 samples/s lr: 2.34e-04 [09/22 03:51:02] lb.utils.events INFO: eta: 16:19:20 iteration: 5999/375342 consumed_samples: 6144000 total_loss: 5.801 time: 0.3327 s/iter data_time: 0.2068 s/iter total_throughput: 3078.09 samples/s lr: 2.38e-04 [09/22 03:51:35] lb.utils.events INFO: eta: 16:12:21 iteration: 6099/375342 consumed_samples: 6246400 total_loss: 5.785 time: 0.3326 s/iter data_time: 0.1911 s/iter total_throughput: 3078.73 samples/s lr: 2.42e-04 [09/22 03:52:08] lb.utils.events INFO: eta: 16:11:42 iteration: 6199/375342 consumed_samples: 6348800 total_loss: 5.773 time: 0.3326 s/iter data_time: 0.2038 s/iter total_throughput: 3079.17 samples/s lr: 2.46e-04 [09/22 03:52:42] lb.utils.events INFO: eta: 16:08:47 iteration: 6299/375342 consumed_samples: 6451200 total_loss: 5.762 time: 0.3327 s/iter data_time: 0.2072 s/iter total_throughput: 3078.22 samples/s lr: 2.50e-04 [09/22 03:53:15] lb.utils.events INFO: eta: 16:07:01 iteration: 6399/375342 consumed_samples: 6553600 total_loss: 5.748 time: 0.3326 s/iter data_time: 0.2120 s/iter total_throughput: 3078.62 samples/s lr: 2.54e-04 [09/22 03:53:48] lb.utils.events INFO: eta: 16:06:40 iteration: 6499/375342 consumed_samples: 6656000 total_loss: 5.746 time: 0.3326 s/iter data_time: 0.2113 s/iter total_throughput: 3078.75 samples/s lr: 2.58e-04 [09/22 03:54:21] lb.utils.events INFO: eta: 16:04:09 iteration: 6599/375342 consumed_samples: 6758400 total_loss: 5.734 time: 0.3326 s/iter data_time: 0.2223 s/iter total_throughput: 3078.82 samples/s lr: 2.62e-04 [09/22 03:54:55] lb.utils.events INFO: eta: 16:04:57 iteration: 6699/375342 consumed_samples: 6860800 total_loss: 5.715 time: 0.3327 s/iter data_time: 0.2128 s/iter total_throughput: 3078.24 samples/s lr: 2.66e-04 [09/22 03:55:28] lb.utils.events INFO: eta: 16:03:05 iteration: 6799/375342 consumed_samples: 6963200 total_loss: 5.699 time: 0.3326 s/iter data_time: 0.1996 s/iter total_throughput: 3078.35 samples/s lr: 2.69e-04 [09/22 03:56:02] lb.utils.events INFO: eta: 15:59:59 iteration: 6899/375342 consumed_samples: 7065600 total_loss: 5.678 time: 0.3328 s/iter data_time: 0.2194 s/iter total_throughput: 3077.37 samples/s lr: 2.73e-04 [09/22 03:56:36] lb.utils.events INFO: eta: 15:58:46 iteration: 6999/375342 consumed_samples: 7168000 total_loss: 5.67 time: 0.3328 s/iter data_time: 0.2150 s/iter total_throughput: 3076.61 samples/s lr: 2.77e-04 [09/22 03:57:09] lb.utils.events INFO: eta: 15:59:15 iteration: 7099/375342 consumed_samples: 7270400 total_loss: 5.658 time: 0.3328 s/iter data_time: 0.2116 s/iter total_throughput: 3076.56 samples/s lr: 2.81e-04 [09/22 03:57:43] lb.utils.events INFO: eta: 15:57:58 iteration: 7199/375342 consumed_samples: 7372800 total_loss: 5.656 time: 0.3328 s/iter data_time: 0.2034 s/iter total_throughput: 3076.48 samples/s lr: 2.85e-04 [09/22 03:58:16] lb.utils.events INFO: eta: 15:57:51 iteration: 7299/375342 consumed_samples: 7475200 total_loss: 5.637 time: 0.3329 s/iter data_time: 0.2202 s/iter total_throughput: 3075.94 samples/s lr: 2.89e-04 [09/22 03:58:50] lb.utils.events INFO: eta: 15:59:42 iteration: 7399/375342 consumed_samples: 7577600 total_loss: 5.609 time: 0.3329 s/iter data_time: 0.2164 s/iter total_throughput: 3075.84 samples/s lr: 2.93e-04 [09/22 03:59:23] lb.utils.events INFO: eta: 16:02:18 iteration: 7499/375342 consumed_samples: 7680000 total_loss: 5.611 time: 0.3329 s/iter data_time: 0.1971 s/iter total_throughput: 3076.17 samples/s lr: 2.97e-04 [09/22 03:59:57] lb.utils.events INFO: eta: 16:02:30 iteration: 7599/375342 consumed_samples: 7782400 total_loss: 5.602 time: 0.3330 s/iter data_time: 0.2218 s/iter total_throughput: 3074.90 samples/s lr: 3.01e-04 [09/22 04:00:31] lb.utils.events INFO: eta: 16:01:55 iteration: 7699/375342 consumed_samples: 7884800 total_loss: 5.582 time: 0.3331 s/iter data_time: 0.2127 s/iter total_throughput: 3074.49 samples/s lr: 3.05e-04 [09/22 04:01:04] lb.utils.events INFO: eta: 16:01:40 iteration: 7799/375342 consumed_samples: 7987200 total_loss: 5.578 time: 0.3331 s/iter data_time: 0.2175 s/iter total_throughput: 3074.09 samples/s lr: 3.09e-04 [09/22 04:01:38] lb.utils.events INFO: eta: 16:04:41 iteration: 7899/375342 consumed_samples: 8089600 total_loss: 5.578 time: 0.3331 s/iter data_time: 0.1976 s/iter total_throughput: 3073.94 samples/s lr: 3.13e-04 [09/22 04:02:12] lb.utils.events INFO: eta: 16:08:17 iteration: 7999/375342 consumed_samples: 8192000 total_loss: 5.551 time: 0.3332 s/iter data_time: 0.2043 s/iter total_throughput: 3073.39 samples/s lr: 3.17e-04 [09/22 04:02:45] lb.utils.events INFO: eta: 16:08:33 iteration: 8099/375342 consumed_samples: 8294400 total_loss: 5.543 time: 0.3332 s/iter data_time: 0.2051 s/iter total_throughput: 3073.02 samples/s lr: 3.21e-04 [09/22 04:03:19] lb.utils.events INFO: eta: 16:10:41 iteration: 8199/375342 consumed_samples: 8396800 total_loss: 5.549 time: 0.3333 s/iter data_time: 0.2161 s/iter total_throughput: 3072.23 samples/s lr: 3.25e-04 [09/22 04:03:53] lb.utils.events INFO: eta: 16:11:07 iteration: 8299/375342 consumed_samples: 8499200 total_loss: 5.529 time: 0.3334 s/iter data_time: 0.2021 s/iter total_throughput: 3071.69 samples/s lr: 3.29e-04 [09/22 04:04:27] lb.utils.events INFO: eta: 16:10:37 iteration: 8399/375342 consumed_samples: 8601600 total_loss: 5.52 time: 0.3335 s/iter data_time: 0.2122 s/iter total_throughput: 3070.92 samples/s lr: 3.33e-04 [09/22 04:05:01] lb.utils.events INFO: eta: 16:08:20 iteration: 8499/375342 consumed_samples: 8704000 total_loss: 5.494 time: 0.3335 s/iter data_time: 0.2109 s/iter total_throughput: 3070.69 samples/s lr: 3.37e-04 [09/22 04:05:34] lb.utils.events INFO: eta: 16:10:11 iteration: 8599/375342 consumed_samples: 8806400 total_loss: 5.494 time: 0.3335 s/iter data_time: 0.2014 s/iter total_throughput: 3070.82 samples/s lr: 3.41e-04 [09/22 04:06:08] lb.utils.events INFO: eta: 16:09:32 iteration: 8699/375342 consumed_samples: 8908800 total_loss: 5.498 time: 0.3335 s/iter data_time: 0.2114 s/iter total_throughput: 3070.16 samples/s lr: 3.45e-04 [09/22 04:06:42] lb.utils.events INFO: eta: 16:09:24 iteration: 8799/375342 consumed_samples: 9011200 total_loss: 5.473 time: 0.3336 s/iter data_time: 0.2158 s/iter total_throughput: 3069.57 samples/s lr: 3.48e-04 [09/22 04:07:16] lb.utils.events INFO: eta: 16:06:45 iteration: 8899/375342 consumed_samples: 9113600 total_loss: 5.451 time: 0.3337 s/iter data_time: 0.2272 s/iter total_throughput: 3068.79 samples/s lr: 3.52e-04 [09/22 04:07:49] lb.utils.events INFO: eta: 16:04:38 iteration: 8999/375342 consumed_samples: 9216000 total_loss: 5.438 time: 0.3337 s/iter data_time: 0.1901 s/iter total_throughput: 3068.73 samples/s lr: 3.56e-04 [09/22 04:08:23] lb.utils.events INFO: eta: 16:01:24 iteration: 9099/375342 consumed_samples: 9318400 total_loss: 5.426 time: 0.3337 s/iter data_time: 0.2020 s/iter total_throughput: 3068.44 samples/s lr: 3.60e-04 [09/22 04:08:58] lb.utils.events INFO: eta: 16:00:54 iteration: 9199/375342 consumed_samples: 9420800 total_loss: 5.428 time: 0.3338 s/iter data_time: 0.2159 s/iter total_throughput: 3067.25 samples/s lr: 3.64e-04 [09/22 04:09:32] lb.utils.events INFO: eta: 16:00:49 iteration: 9299/375342 consumed_samples: 9523200 total_loss: 5.441 time: 0.3339 s/iter data_time: 0.2051 s/iter total_throughput: 3066.38 samples/s lr: 3.68e-04 [09/22 04:10:06] lb.utils.events INFO: eta: 16:00:33 iteration: 9399/375342 consumed_samples: 9625600 total_loss: 5.438 time: 0.3340 s/iter data_time: 0.2104 s/iter total_throughput: 3065.85 samples/s lr: 3.72e-04 [09/22 04:10:40] lb.utils.events INFO: eta: 16:02:00 iteration: 9499/375342 consumed_samples: 9728000 total_loss: 5.414 time: 0.3341 s/iter data_time: 0.2047 s/iter total_throughput: 3065.30 samples/s lr: 3.76e-04 [09/22 04:11:14] lb.utils.events INFO: eta: 15:59:41 iteration: 9599/375342 consumed_samples: 9830400 total_loss: 5.393 time: 0.3341 s/iter data_time: 0.2297 s/iter total_throughput: 3064.54 samples/s lr: 3.80e-04 [09/22 04:11:48] lb.utils.events INFO: eta: 16:00:30 iteration: 9699/375342 consumed_samples: 9932800 total_loss: 5.379 time: 0.3342 s/iter data_time: 0.2180 s/iter total_throughput: 3064.06 samples/s lr: 3.84e-04 [09/22 04:12:22] lb.utils.events INFO: eta: 16:00:15 iteration: 9799/375342 consumed_samples: 10035200 total_loss: 5.379 time: 0.3343 s/iter data_time: 0.2138 s/iter total_throughput: 3063.56 samples/s lr: 3.88e-04 [09/22 04:12:56] lb.utils.events INFO: eta: 15:58:52 iteration: 9899/375342 consumed_samples: 10137600 total_loss: 5.363 time: 0.3343 s/iter data_time: 0.2098 s/iter total_throughput: 3062.87 samples/s lr: 3.92e-04 [09/22 04:13:30] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0009999 [09/22 04:13:31] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/22 04:13:31] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/22 04:13:35] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0998 s/iter. Inference: 0.1564 s/iter. Eval: 0.0021 s/iter. Total: 0.2583 s/iter. ETA=0:00:09 [09/22 04:13:40] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1091 s/iter. Inference: 0.1793 s/iter. Eval: 0.0020 s/iter. Total: 0.2905 s/iter. ETA=0:00:05 [09/22 04:13:45] lb.evaluation.evaluator INFO: Inference done 48128/50000. Dataloading: 0.0911 s/iter. Inference: 0.1920 s/iter. Eval: 0.0020 s/iter. Total: 0.2852 s/iter. ETA=0:00:00 [09/22 04:13:45] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/22 04:13:45] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.365212 (0.000247 s / iter per device, on 8 devices) [09/22 04:13:45] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:08 (0.000169 s / iter per device, on 8 devices) [09/22 04:13:45] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/22 04:13:45] lb.evaluation.utils INFO: copypaste: Acc@1=34.82 [09/22 04:13:45] lb.evaluation.utils INFO: copypaste: Acc@5=60.27400000000001 [09/22 04:13:45] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 34.82000, better than last best score 16.72400 @ iteration 4999. [09/22 04:13:45] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/22 04:13:46] lb.utils.events INFO: eta: 15:58:28 iteration: 9999/375342 consumed_samples: 10240000 total_loss: 5.348 time: 0.3344 s/iter data_time: 0.2264 s/iter total_throughput: 3062.48 samples/s lr: 3.96e-04 [09/22 04:14:19] lb.utils.events INFO: eta: 16:01:09 iteration: 10099/375342 consumed_samples: 10342400 total_loss: 5.346 time: 0.3343 s/iter data_time: 0.2213 s/iter total_throughput: 3063.38 samples/s lr: 4.00e-04 [09/22 04:14:53] lb.utils.events INFO: eta: 16:00:16 iteration: 10199/375342 consumed_samples: 10444800 total_loss: 5.332 time: 0.3343 s/iter data_time: 0.2062 s/iter total_throughput: 3062.86 samples/s lr: 4.04e-04 [09/22 04:15:27] lb.utils.events INFO: eta: 15:58:58 iteration: 10299/375342 consumed_samples: 10547200 total_loss: 5.309 time: 0.3344 s/iter data_time: 0.2049 s/iter total_throughput: 3062.30 samples/s lr: 4.08e-04 [09/22 04:16:01] lb.utils.events INFO: eta: 15:59:17 iteration: 10399/375342 consumed_samples: 10649600 total_loss: 5.305 time: 0.3344 s/iter data_time: 0.2146 s/iter total_throughput: 3061.79 samples/s lr: 4.12e-04 [09/22 04:16:35] lb.utils.events INFO: eta: 15:59:27 iteration: 10499/375342 consumed_samples: 10752000 total_loss: 5.305 time: 0.3345 s/iter data_time: 0.2125 s/iter total_throughput: 3061.29 samples/s lr: 4.16e-04 [09/22 04:17:09] lb.utils.events INFO: eta: 15:59:40 iteration: 10599/375342 consumed_samples: 10854400 total_loss: 5.303 time: 0.3346 s/iter data_time: 0.2175 s/iter total_throughput: 3060.74 samples/s lr: 4.20e-04 [09/22 04:17:43] lb.utils.events INFO: eta: 16:00:07 iteration: 10699/375342 consumed_samples: 10956800 total_loss: 5.307 time: 0.3346 s/iter data_time: 0.2026 s/iter total_throughput: 3060.66 samples/s lr: 4.24e-04 [09/22 04:18:16] lb.utils.events INFO: eta: 16:00:47 iteration: 10799/375342 consumed_samples: 11059200 total_loss: 5.281 time: 0.3346 s/iter data_time: 0.2080 s/iter total_throughput: 3060.31 samples/s lr: 4.27e-04 [09/22 04:18:50] lb.utils.events INFO: eta: 16:04:37 iteration: 10899/375342 consumed_samples: 11161600 total_loss: 5.271 time: 0.3346 s/iter data_time: 0.2107 s/iter total_throughput: 3059.95 samples/s lr: 4.31e-04 [09/22 04:19:24] lb.utils.events INFO: eta: 16:03:49 iteration: 10999/375342 consumed_samples: 11264000 total_loss: 5.275 time: 0.3347 s/iter data_time: 0.2147 s/iter total_throughput: 3059.58 samples/s lr: 4.35e-04 [09/22 04:19:58] lb.utils.events INFO: eta: 16:02:14 iteration: 11099/375342 consumed_samples: 11366400 total_loss: 5.264 time: 0.3347 s/iter data_time: 0.2206 s/iter total_throughput: 3059.42 samples/s lr: 4.39e-04 [09/22 04:20:32] lb.utils.events INFO: eta: 16:03:08 iteration: 11199/375342 consumed_samples: 11468800 total_loss: 5.246 time: 0.3347 s/iter data_time: 0.2093 s/iter total_throughput: 3059.04 samples/s lr: 4.43e-04 [09/22 04:21:06] lb.utils.events INFO: eta: 16:04:39 iteration: 11299/375342 consumed_samples: 11571200 total_loss: 5.232 time: 0.3348 s/iter data_time: 0.2094 s/iter total_throughput: 3058.58 samples/s lr: 4.47e-04 [09/22 04:21:40] lb.utils.events INFO: eta: 16:03:11 iteration: 11399/375342 consumed_samples: 11673600 total_loss: 5.225 time: 0.3348 s/iter data_time: 0.2020 s/iter total_throughput: 3058.12 samples/s lr: 4.51e-04 [09/22 04:22:14] lb.utils.events INFO: eta: 16:02:31 iteration: 11499/375342 consumed_samples: 11776000 total_loss: 5.238 time: 0.3349 s/iter data_time: 0.2057 s/iter total_throughput: 3057.75 samples/s lr: 4.55e-04 [09/22 04:22:48] lb.utils.events INFO: eta: 16:02:05 iteration: 11599/375342 consumed_samples: 11878400 total_loss: 5.23 time: 0.3349 s/iter data_time: 0.2162 s/iter total_throughput: 3057.18 samples/s lr: 4.59e-04 [09/22 04:23:22] lb.utils.events INFO: eta: 16:00:11 iteration: 11699/375342 consumed_samples: 11980800 total_loss: 5.203 time: 0.3350 s/iter data_time: 0.2126 s/iter total_throughput: 3056.59 samples/s lr: 4.63e-04 [09/22 04:23:56] lb.utils.events INFO: eta: 15:58:37 iteration: 11799/375342 consumed_samples: 12083200 total_loss: 5.188 time: 0.3350 s/iter data_time: 0.2163 s/iter total_throughput: 3056.67 samples/s lr: 4.67e-04 [09/22 04:24:30] lb.utils.events INFO: eta: 15:57:44 iteration: 11899/375342 consumed_samples: 12185600 total_loss: 5.201 time: 0.3350 s/iter data_time: 0.2219 s/iter total_throughput: 3056.27 samples/s lr: 4.71e-04 [09/22 04:25:04] lb.utils.events INFO: eta: 15:54:36 iteration: 11999/375342 consumed_samples: 12288000 total_loss: 5.197 time: 0.3351 s/iter data_time: 0.2073 s/iter total_throughput: 3055.73 samples/s lr: 4.75e-04 [09/22 04:25:38] lb.utils.events INFO: eta: 15:53:42 iteration: 12099/375342 consumed_samples: 12390400 total_loss: 5.18 time: 0.3351 s/iter data_time: 0.2171 s/iter total_throughput: 3055.38 samples/s lr: 4.79e-04 [09/22 04:26:12] lb.utils.events INFO: eta: 15:52:30 iteration: 12199/375342 consumed_samples: 12492800 total_loss: 5.182 time: 0.3352 s/iter data_time: 0.2338 s/iter total_throughput: 3054.72 samples/s lr: 4.83e-04 [09/22 04:26:47] lb.utils.events INFO: eta: 15:51:44 iteration: 12299/375342 consumed_samples: 12595200 total_loss: 5.168 time: 0.3353 s/iter data_time: 0.2181 s/iter total_throughput: 3054.25 samples/s lr: 4.87e-04 [09/22 04:27:20] lb.utils.events INFO: eta: 15:52:18 iteration: 12399/375342 consumed_samples: 12697600 total_loss: 5.162 time: 0.3353 s/iter data_time: 0.2132 s/iter total_throughput: 3054.17 samples/s lr: 4.91e-04 [09/22 04:27:54] lb.utils.events INFO: eta: 15:51:13 iteration: 12499/375342 consumed_samples: 12800000 total_loss: 5.152 time: 0.3353 s/iter data_time: 0.2154 s/iter total_throughput: 3053.94 samples/s lr: 4.95e-04 [09/22 04:28:28] lb.utils.events INFO: eta: 15:50:48 iteration: 12599/375342 consumed_samples: 12902400 total_loss: 5.148 time: 0.3354 s/iter data_time: 0.2270 s/iter total_throughput: 3053.50 samples/s lr: 4.99e-04 [09/22 04:29:03] lb.utils.events INFO: eta: 15:52:15 iteration: 12699/375342 consumed_samples: 13004800 total_loss: 5.137 time: 0.3354 s/iter data_time: 0.2193 s/iter total_throughput: 3052.99 samples/s lr: 5.02e-04 [09/22 04:29:37] lb.utils.events INFO: eta: 15:51:26 iteration: 12799/375342 consumed_samples: 13107200 total_loss: 5.121 time: 0.3355 s/iter data_time: 0.2211 s/iter total_throughput: 3052.50 samples/s lr: 5.06e-04 [09/22 04:30:11] lb.utils.events INFO: eta: 15:50:24 iteration: 12899/375342 consumed_samples: 13209600 total_loss: 5.109 time: 0.3355 s/iter data_time: 0.2050 s/iter total_throughput: 3052.14 samples/s lr: 5.10e-04 [09/22 04:30:44] lb.utils.events INFO: eta: 15:51:08 iteration: 12999/375342 consumed_samples: 13312000 total_loss: 5.1 time: 0.3355 s/iter data_time: 0.2040 s/iter total_throughput: 3052.26 samples/s lr: 5.14e-04 [09/22 04:31:19] lb.utils.events INFO: eta: 15:50:22 iteration: 13099/375342 consumed_samples: 13414400 total_loss: 5.117 time: 0.3355 s/iter data_time: 0.2224 s/iter total_throughput: 3051.74 samples/s lr: 5.18e-04 [09/22 04:31:53] lb.utils.events INFO: eta: 15:49:14 iteration: 13199/375342 consumed_samples: 13516800 total_loss: 5.117 time: 0.3356 s/iter data_time: 0.2292 s/iter total_throughput: 3051.21 samples/s lr: 5.22e-04 [09/22 04:32:27] lb.utils.events INFO: eta: 15:48:33 iteration: 13299/375342 consumed_samples: 13619200 total_loss: 5.117 time: 0.3356 s/iter data_time: 0.2070 s/iter total_throughput: 3051.06 samples/s lr: 5.26e-04 [09/22 04:33:01] lb.utils.events INFO: eta: 15:46:36 iteration: 13399/375342 consumed_samples: 13721600 total_loss: 5.109 time: 0.3357 s/iter data_time: 0.2153 s/iter total_throughput: 3050.20 samples/s lr: 5.30e-04 [09/22 04:33:36] lb.utils.events INFO: eta: 15:47:48 iteration: 13499/375342 consumed_samples: 13824000 total_loss: 5.082 time: 0.3358 s/iter data_time: 0.2041 s/iter total_throughput: 3049.87 samples/s lr: 5.34e-04 [09/22 04:34:10] lb.utils.events INFO: eta: 15:46:42 iteration: 13599/375342 consumed_samples: 13926400 total_loss: 5.072 time: 0.3358 s/iter data_time: 0.2075 s/iter total_throughput: 3049.33 samples/s lr: 5.38e-04 [09/22 04:34:44] lb.utils.events INFO: eta: 15:47:18 iteration: 13699/375342 consumed_samples: 14028800 total_loss: 5.061 time: 0.3358 s/iter data_time: 0.2128 s/iter total_throughput: 3049.16 samples/s lr: 5.42e-04 [09/22 04:35:18] lb.utils.events INFO: eta: 15:46:45 iteration: 13799/375342 consumed_samples: 14131200 total_loss: 5.053 time: 0.3358 s/iter data_time: 0.2146 s/iter total_throughput: 3049.03 samples/s lr: 5.46e-04 [09/22 04:35:52] lb.utils.events INFO: eta: 15:46:24 iteration: 13899/375342 consumed_samples: 14233600 total_loss: 5.062 time: 0.3359 s/iter data_time: 0.2201 s/iter total_throughput: 3048.65 samples/s lr: 5.50e-04 [09/22 04:36:26] lb.utils.events INFO: eta: 15:47:29 iteration: 13999/375342 consumed_samples: 14336000 total_loss: 5.051 time: 0.3359 s/iter data_time: 0.2130 s/iter total_throughput: 3048.42 samples/s lr: 5.54e-04 [09/22 04:37:00] lb.utils.events INFO: eta: 15:47:14 iteration: 14099/375342 consumed_samples: 14438400 total_loss: 5.037 time: 0.3360 s/iter data_time: 0.2127 s/iter total_throughput: 3047.84 samples/s lr: 5.58e-04 [09/22 04:37:35] lb.utils.events INFO: eta: 15:49:55 iteration: 14199/375342 consumed_samples: 14540800 total_loss: 5.039 time: 0.3360 s/iter data_time: 0.2262 s/iter total_throughput: 3047.28 samples/s lr: 5.62e-04 [09/22 04:38:10] lb.utils.events INFO: eta: 15:49:07 iteration: 14299/375342 consumed_samples: 14643200 total_loss: 5.039 time: 0.3361 s/iter data_time: 0.2204 s/iter total_throughput: 3046.47 samples/s lr: 5.66e-04 [09/22 04:38:44] lb.utils.events INFO: eta: 15:53:16 iteration: 14399/375342 consumed_samples: 14745600 total_loss: 5.055 time: 0.3362 s/iter data_time: 0.2133 s/iter total_throughput: 3045.91 samples/s lr: 5.70e-04 [09/22 04:39:19] lb.utils.events INFO: eta: 15:52:41 iteration: 14499/375342 consumed_samples: 14848000 total_loss: 5.055 time: 0.3363 s/iter data_time: 0.2217 s/iter total_throughput: 3045.31 samples/s lr: 5.74e-04 [09/22 04:39:53] lb.utils.events INFO: eta: 15:52:25 iteration: 14599/375342 consumed_samples: 14950400 total_loss: 5.008 time: 0.3363 s/iter data_time: 0.2061 s/iter total_throughput: 3044.71 samples/s lr: 5.78e-04 [09/22 04:40:28] lb.utils.events INFO: eta: 15:49:27 iteration: 14699/375342 consumed_samples: 15052800 total_loss: 5.002 time: 0.3364 s/iter data_time: 0.2223 s/iter total_throughput: 3043.88 samples/s lr: 5.81e-04 [09/22 04:41:03] lb.utils.events INFO: eta: 15:48:28 iteration: 14799/375342 consumed_samples: 15155200 total_loss: 5.004 time: 0.3365 s/iter data_time: 0.2186 s/iter total_throughput: 3042.93 samples/s lr: 5.85e-04 [09/22 04:41:38] lb.utils.events INFO: eta: 15:49:55 iteration: 14899/375342 consumed_samples: 15257600 total_loss: 5.008 time: 0.3366 s/iter data_time: 0.2394 s/iter total_throughput: 3042.28 samples/s lr: 5.89e-04 [09/22 04:42:13] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0014999 [09/22 04:42:14] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/22 04:42:14] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/22 04:42:18] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0943 s/iter. Inference: 0.1554 s/iter. Eval: 0.0020 s/iter. Total: 0.2517 s/iter. ETA=0:00:09 [09/22 04:42:23] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1193 s/iter. Inference: 0.1674 s/iter. Eval: 0.0020 s/iter. Total: 0.2888 s/iter. ETA=0:00:05 [09/22 04:42:28] lb.evaluation.evaluator INFO: Inference done 48128/50000. Dataloading: 0.0868 s/iter. Inference: 0.1972 s/iter. Eval: 0.0020 s/iter. Total: 0.2861 s/iter. ETA=0:00:00 [09/22 04:42:29] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/22 04:42:29] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.463154 (0.000249 s / iter per device, on 8 devices) [09/22 04:42:29] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:08 (0.000174 s / iter per device, on 8 devices) [09/22 04:42:29] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/22 04:42:29] lb.evaluation.utils INFO: copypaste: Acc@1=45.506 [09/22 04:42:29] lb.evaluation.utils INFO: copypaste: Acc@5=71.00800000000001 [09/22 04:42:29] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 45.50600, better than last best score 34.82000 @ iteration 9999. [09/22 04:42:29] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/22 04:42:29] lb.utils.events INFO: eta: 15:48:05 iteration: 14999/375342 consumed_samples: 15360000 total_loss: 5.02 time: 0.3367 s/iter data_time: 0.2100 s/iter total_throughput: 3041.62 samples/s lr: 5.93e-04 [09/22 04:43:02] lb.utils.events INFO: eta: 15:52:36 iteration: 15099/375342 consumed_samples: 15462400 total_loss: 4.973 time: 0.3366 s/iter data_time: 0.2416 s/iter total_throughput: 3042.27 samples/s lr: 5.97e-04 [09/22 04:43:37] lb.utils.events INFO: eta: 15:51:06 iteration: 15199/375342 consumed_samples: 15564800 total_loss: 4.961 time: 0.3367 s/iter data_time: 0.2286 s/iter total_throughput: 3041.48 samples/s lr: 6.01e-04 [09/22 04:44:12] lb.utils.events INFO: eta: 15:51:03 iteration: 15299/375342 consumed_samples: 15667200 total_loss: 4.961 time: 0.3367 s/iter data_time: 0.2200 s/iter total_throughput: 3040.88 samples/s lr: 6.05e-04 [09/22 04:44:46] lb.utils.events INFO: eta: 15:47:22 iteration: 15399/375342 consumed_samples: 15769600 total_loss: 4.977 time: 0.3368 s/iter data_time: 0.2258 s/iter total_throughput: 3040.43 samples/s lr: 6.09e-04 [09/22 04:45:21] lb.utils.events INFO: eta: 15:47:27 iteration: 15499/375342 consumed_samples: 15872000 total_loss: 4.98 time: 0.3369 s/iter data_time: 0.2311 s/iter total_throughput: 3039.89 samples/s lr: 6.13e-04 [09/22 04:45:55] lb.utils.events INFO: eta: 15:48:12 iteration: 15599/375342 consumed_samples: 15974400 total_loss: 4.961 time: 0.3369 s/iter data_time: 0.2105 s/iter total_throughput: 3039.26 samples/s lr: 6.17e-04 [09/22 04:46:30] lb.utils.events INFO: eta: 15:47:22 iteration: 15699/375342 consumed_samples: 16076800 total_loss: 4.951 time: 0.3370 s/iter data_time: 0.2304 s/iter total_throughput: 3038.66 samples/s lr: 6.21e-04 [09/22 04:47:05] lb.utils.events INFO: eta: 15:47:35 iteration: 15799/375342 consumed_samples: 16179200 total_loss: 4.941 time: 0.3371 s/iter data_time: 0.2195 s/iter total_throughput: 3037.99 samples/s lr: 6.25e-04 [09/22 04:47:40] lb.utils.events INFO: eta: 15:46:50 iteration: 15899/375342 consumed_samples: 16281600 total_loss: 4.939 time: 0.3371 s/iter data_time: 0.2346 s/iter total_throughput: 3037.23 samples/s lr: 6.29e-04 [09/22 04:48:15] lb.utils.events INFO: eta: 15:46:00 iteration: 15999/375342 consumed_samples: 16384000 total_loss: 4.953 time: 0.3372 s/iter data_time: 0.2219 s/iter total_throughput: 3036.70 samples/s lr: 6.33e-04 [09/22 04:48:50] lb.utils.events INFO: eta: 15:38:55 iteration: 16099/375342 consumed_samples: 16486400 total_loss: 4.949 time: 0.3373 s/iter data_time: 0.2097 s/iter total_throughput: 3035.91 samples/s lr: 6.37e-04 [09/22 04:49:25] lb.utils.events INFO: eta: 15:40:03 iteration: 16199/375342 consumed_samples: 16588800 total_loss: 4.945 time: 0.3374 s/iter data_time: 0.2165 s/iter total_throughput: 3035.38 samples/s lr: 6.41e-04 [09/22 04:49:59] lb.utils.events INFO: eta: 15:41:47 iteration: 16299/375342 consumed_samples: 16691200 total_loss: 4.914 time: 0.3374 s/iter data_time: 0.2145 s/iter total_throughput: 3034.82 samples/s lr: 6.45e-04 [09/22 04:50:34] lb.utils.events INFO: eta: 15:42:42 iteration: 16399/375342 consumed_samples: 16793600 total_loss: 4.869 time: 0.3375 s/iter data_time: 0.2262 s/iter total_throughput: 3034.39 samples/s lr: 6.49e-04 [09/22 04:51:09] lb.utils.events INFO: eta: 15:39:22 iteration: 16499/375342 consumed_samples: 16896000 total_loss: 4.85 time: 0.3375 s/iter data_time: 0.2298 s/iter total_throughput: 3033.76 samples/s lr: 6.53e-04 [09/22 04:51:44] lb.utils.events INFO: eta: 15:38:47 iteration: 16599/375342 consumed_samples: 16998400 total_loss: 4.883 time: 0.3376 s/iter data_time: 0.2202 s/iter total_throughput: 3033.25 samples/s lr: 6.57e-04 [09/22 04:52:18] lb.utils.events INFO: eta: 15:40:44 iteration: 16699/375342 consumed_samples: 17100800 total_loss: 4.881 time: 0.3376 s/iter data_time: 0.2084 s/iter total_throughput: 3032.89 samples/s lr: 6.60e-04 [09/22 04:52:53] lb.utils.events INFO: eta: 15:41:21 iteration: 16799/375342 consumed_samples: 17203200 total_loss: 4.883 time: 0.3377 s/iter data_time: 0.2158 s/iter total_throughput: 3032.14 samples/s lr: 6.64e-04 [09/22 04:53:28] lb.utils.events INFO: eta: 15:41:23 iteration: 16899/375342 consumed_samples: 17305600 total_loss: 4.875 time: 0.3378 s/iter data_time: 0.2296 s/iter total_throughput: 3031.66 samples/s lr: 6.68e-04 [09/22 04:54:03] lb.utils.events INFO: eta: 15:37:44 iteration: 16999/375342 consumed_samples: 17408000 total_loss: 4.859 time: 0.3378 s/iter data_time: 0.2222 s/iter total_throughput: 3031.21 samples/s lr: 6.72e-04 [09/22 04:54:37] lb.utils.events INFO: eta: 15:37:22 iteration: 17099/375342 consumed_samples: 17510400 total_loss: 4.867 time: 0.3379 s/iter data_time: 0.2177 s/iter total_throughput: 3030.69 samples/s lr: 6.76e-04 [09/22 04:55:12] lb.utils.events INFO: eta: 15:34:51 iteration: 17199/375342 consumed_samples: 17612800 total_loss: 4.861 time: 0.3380 s/iter data_time: 0.2417 s/iter total_throughput: 3030.01 samples/s lr: 6.80e-04 [09/22 04:55:47] lb.utils.events INFO: eta: 15:33:37 iteration: 17299/375342 consumed_samples: 17715200 total_loss: 4.863 time: 0.3380 s/iter data_time: 0.2074 s/iter total_throughput: 3029.59 samples/s lr: 6.84e-04 [09/22 04:56:22] lb.utils.events INFO: eta: 15:33:34 iteration: 17399/375342 consumed_samples: 17817600 total_loss: 4.848 time: 0.3381 s/iter data_time: 0.2220 s/iter total_throughput: 3029.03 samples/s lr: 6.88e-04 [09/22 04:56:57] lb.utils.events INFO: eta: 15:33:20 iteration: 17499/375342 consumed_samples: 17920000 total_loss: 4.842 time: 0.3381 s/iter data_time: 0.2255 s/iter total_throughput: 3028.49 samples/s lr: 6.92e-04 [09/22 04:57:32] lb.utils.events INFO: eta: 15:32:29 iteration: 17599/375342 consumed_samples: 18022400 total_loss: 4.828 time: 0.3382 s/iter data_time: 0.2286 s/iter total_throughput: 3027.97 samples/s lr: 6.96e-04 [09/22 04:58:07] lb.utils.events INFO: eta: 15:31:23 iteration: 17699/375342 consumed_samples: 18124800 total_loss: 4.82 time: 0.3382 s/iter data_time: 0.2046 s/iter total_throughput: 3027.36 samples/s lr: 7.00e-04 [09/22 04:58:42] lb.utils.events INFO: eta: 15:28:12 iteration: 17799/375342 consumed_samples: 18227200 total_loss: 4.822 time: 0.3383 s/iter data_time: 0.2323 s/iter total_throughput: 3026.82 samples/s lr: 7.04e-04 [09/22 04:59:17] lb.utils.events INFO: eta: 15:25:20 iteration: 17899/375342 consumed_samples: 18329600 total_loss: 4.801 time: 0.3384 s/iter data_time: 0.2405 s/iter total_throughput: 3026.19 samples/s lr: 7.08e-04 [09/22 04:59:51] lb.utils.events INFO: eta: 15:25:52 iteration: 17999/375342 consumed_samples: 18432000 total_loss: 4.807 time: 0.3384 s/iter data_time: 0.2223 s/iter total_throughput: 3025.84 samples/s lr: 7.12e-04 [09/22 05:00:26] lb.utils.events INFO: eta: 15:26:49 iteration: 18099/375342 consumed_samples: 18534400 total_loss: 4.82 time: 0.3385 s/iter data_time: 0.2125 s/iter total_throughput: 3025.54 samples/s lr: 7.16e-04 [09/22 05:01:01] lb.utils.events INFO: eta: 15:26:44 iteration: 18199/375342 consumed_samples: 18636800 total_loss: 4.811 time: 0.3385 s/iter data_time: 0.2375 s/iter total_throughput: 3024.83 samples/s lr: 7.20e-04 [09/22 05:01:35] lb.utils.events INFO: eta: 15:27:02 iteration: 18299/375342 consumed_samples: 18739200 total_loss: 4.805 time: 0.3386 s/iter data_time: 0.2107 s/iter total_throughput: 3024.62 samples/s lr: 7.24e-04 [09/22 05:02:10] lb.utils.events INFO: eta: 15:26:39 iteration: 18399/375342 consumed_samples: 18841600 total_loss: 4.787 time: 0.3386 s/iter data_time: 0.2052 s/iter total_throughput: 3024.36 samples/s lr: 7.28e-04 [09/22 05:02:44] lb.utils.events INFO: eta: 15:26:05 iteration: 18499/375342 consumed_samples: 18944000 total_loss: 4.793 time: 0.3386 s/iter data_time: 0.2178 s/iter total_throughput: 3024.01 samples/s lr: 7.32e-04 [09/22 05:03:19] lb.utils.events INFO: eta: 15:27:27 iteration: 18599/375342 consumed_samples: 19046400 total_loss: 4.801 time: 0.3387 s/iter data_time: 0.2151 s/iter total_throughput: 3023.53 samples/s lr: 7.35e-04 [09/22 05:03:54] lb.utils.events INFO: eta: 15:27:44 iteration: 18699/375342 consumed_samples: 19148800 total_loss: 4.791 time: 0.3387 s/iter data_time: 0.2169 s/iter total_throughput: 3023.27 samples/s lr: 7.39e-04 [09/22 05:04:28] lb.utils.events INFO: eta: 15:32:28 iteration: 18799/375342 consumed_samples: 19251200 total_loss: 4.781 time: 0.3387 s/iter data_time: 0.2021 s/iter total_throughput: 3023.23 samples/s lr: 7.43e-04 [09/22 05:05:03] lb.utils.events INFO: eta: 15:34:40 iteration: 18899/375342 consumed_samples: 19353600 total_loss: 4.758 time: 0.3388 s/iter data_time: 0.2269 s/iter total_throughput: 3022.63 samples/s lr: 7.47e-04 [09/22 05:05:37] lb.utils.events INFO: eta: 15:35:35 iteration: 18999/375342 consumed_samples: 19456000 total_loss: 4.746 time: 0.3388 s/iter data_time: 0.2104 s/iter total_throughput: 3022.53 samples/s lr: 7.51e-04 [09/22 05:06:11] lb.utils.events INFO: eta: 15:34:18 iteration: 19099/375342 consumed_samples: 19558400 total_loss: 4.768 time: 0.3388 s/iter data_time: 0.2264 s/iter total_throughput: 3022.24 samples/s lr: 7.55e-04 [09/22 05:06:46] lb.utils.events INFO: eta: 15:35:10 iteration: 19199/375342 consumed_samples: 19660800 total_loss: 4.766 time: 0.3388 s/iter data_time: 0.2213 s/iter total_throughput: 3022.00 samples/s lr: 7.59e-04 [09/22 05:07:20] lb.utils.events INFO: eta: 15:36:07 iteration: 19299/375342 consumed_samples: 19763200 total_loss: 4.773 time: 0.3389 s/iter data_time: 0.2182 s/iter total_throughput: 3021.64 samples/s lr: 7.63e-04 [09/22 05:07:55] lb.utils.events INFO: eta: 15:36:16 iteration: 19399/375342 consumed_samples: 19865600 total_loss: 4.783 time: 0.3389 s/iter data_time: 0.2109 s/iter total_throughput: 3021.46 samples/s lr: 7.67e-04 [09/22 05:08:29] lb.utils.events INFO: eta: 15:37:38 iteration: 19499/375342 consumed_samples: 19968000 total_loss: 4.76 time: 0.3389 s/iter data_time: 0.2177 s/iter total_throughput: 3021.42 samples/s lr: 7.71e-04 [09/22 05:09:03] lb.utils.events INFO: eta: 15:36:54 iteration: 19599/375342 consumed_samples: 20070400 total_loss: 4.742 time: 0.3389 s/iter data_time: 0.2177 s/iter total_throughput: 3021.19 samples/s lr: 7.75e-04 [09/22 05:09:37] lb.utils.events INFO: eta: 15:37:07 iteration: 19699/375342 consumed_samples: 20172800 total_loss: 4.738 time: 0.3390 s/iter data_time: 0.2194 s/iter total_throughput: 3021.01 samples/s lr: 7.79e-04 [09/22 05:10:11] lb.utils.events INFO: eta: 15:35:03 iteration: 19799/375342 consumed_samples: 20275200 total_loss: 4.744 time: 0.3390 s/iter data_time: 0.2189 s/iter total_throughput: 3020.96 samples/s lr: 7.83e-04 [09/22 05:10:46] lb.utils.events INFO: eta: 15:33:27 iteration: 19899/375342 consumed_samples: 20377600 total_loss: 4.754 time: 0.3390 s/iter data_time: 0.2270 s/iter total_throughput: 3020.82 samples/s lr: 7.87e-04 [09/22 05:11:20] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0019999 [09/22 05:11:21] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/22 05:11:21] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/22 05:11:25] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0520 s/iter. Inference: 0.2040 s/iter. Eval: 0.0020 s/iter. Total: 0.2580 s/iter. ETA=0:00:09 [09/22 05:11:30] lb.evaluation.evaluator INFO: Inference done 28672/50000. Dataloading: 0.0361 s/iter. Inference: 0.2497 s/iter. Eval: 0.0020 s/iter. Total: 0.2878 s/iter. ETA=0:00:05 [09/22 05:11:35] lb.evaluation.evaluator INFO: Inference done 46080/50000. Dataloading: 0.0209 s/iter. Inference: 0.2716 s/iter. Eval: 0.0020 s/iter. Total: 0.2946 s/iter. ETA=0:00:00 [09/22 05:11:36] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/22 05:11:36] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.736557 (0.000255 s / iter per device, on 8 devices) [09/22 05:11:36] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:11 (0.000236 s / iter per device, on 8 devices) [09/22 05:11:36] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/22 05:11:36] lb.evaluation.utils INFO: copypaste: Acc@1=52.44799999999999 [09/22 05:11:36] lb.evaluation.utils INFO: copypaste: Acc@5=77.538 [09/22 05:11:36] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 52.44800, better than last best score 45.50600 @ iteration 14999. [09/22 05:11:36] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/22 05:11:37] lb.utils.events INFO: eta: 15:32:52 iteration: 19999/375342 consumed_samples: 20480000 total_loss: 4.729 time: 0.3390 s/iter data_time: 0.2262 s/iter total_throughput: 3020.65 samples/s lr: 7.91e-04 [09/22 05:12:09] lb.utils.events INFO: eta: 15:38:00 iteration: 20099/375342 consumed_samples: 20582400 total_loss: 4.719 time: 0.3389 s/iter data_time: 0.2057 s/iter total_throughput: 3021.47 samples/s lr: 7.95e-04 [09/22 05:12:43] lb.utils.events INFO: eta: 15:36:02 iteration: 20199/375342 consumed_samples: 20684800 total_loss: 4.727 time: 0.3389 s/iter data_time: 0.2113 s/iter total_throughput: 3021.13 samples/s lr: 7.99e-04 [09/22 05:13:18] lb.utils.events INFO: eta: 15:37:15 iteration: 20299/375342 consumed_samples: 20787200 total_loss: 4.746 time: 0.3390 s/iter data_time: 0.2150 s/iter total_throughput: 3020.88 samples/s lr: 8.03e-04 [09/22 05:13:52] lb.utils.events INFO: eta: 15:35:11 iteration: 20399/375342 consumed_samples: 20889600 total_loss: 4.73 time: 0.3390 s/iter data_time: 0.2339 s/iter total_throughput: 3020.55 samples/s lr: 8.07e-04 [09/22 05:14:27] lb.utils.events INFO: eta: 15:32:20 iteration: 20499/375342 consumed_samples: 20992000 total_loss: 4.707 time: 0.3391 s/iter data_time: 0.2415 s/iter total_throughput: 3020.08 samples/s lr: 8.11e-04 [09/22 05:15:02] lb.utils.events INFO: eta: 15:31:02 iteration: 20599/375342 consumed_samples: 21094400 total_loss: 4.717 time: 0.3391 s/iter data_time: 0.2316 s/iter total_throughput: 3019.83 samples/s lr: 8.14e-04 [09/22 05:15:36] lb.utils.events INFO: eta: 15:30:25 iteration: 20699/375342 consumed_samples: 21196800 total_loss: 4.707 time: 0.3391 s/iter data_time: 0.2038 s/iter total_throughput: 3019.77 samples/s lr: 8.18e-04 [09/22 05:16:11] lb.utils.events INFO: eta: 15:29:49 iteration: 20799/375342 consumed_samples: 21299200 total_loss: 4.703 time: 0.3391 s/iter data_time: 0.2309 s/iter total_throughput: 3019.42 samples/s lr: 8.22e-04 [09/22 05:16:45] lb.utils.events INFO: eta: 15:29:33 iteration: 20899/375342 consumed_samples: 21401600 total_loss: 4.713 time: 0.3392 s/iter data_time: 0.2033 s/iter total_throughput: 3019.28 samples/s lr: 8.26e-04 [09/22 05:17:19] lb.utils.events INFO: eta: 15:29:04 iteration: 20999/375342 consumed_samples: 21504000 total_loss: 4.719 time: 0.3392 s/iter data_time: 0.2107 s/iter total_throughput: 3019.12 samples/s lr: 8.30e-04 [09/22 05:17:54] lb.utils.events INFO: eta: 15:25:20 iteration: 21099/375342 consumed_samples: 21606400 total_loss: 4.719 time: 0.3392 s/iter data_time: 0.2254 s/iter total_throughput: 3018.95 samples/s lr: 8.34e-04 [09/22 05:18:27] lb.utils.events INFO: eta: 15:27:32 iteration: 21199/375342 consumed_samples: 21708800 total_loss: 4.705 time: 0.3392 s/iter data_time: 0.2187 s/iter total_throughput: 3019.04 samples/s lr: 8.38e-04 [09/22 05:19:02] lb.utils.events INFO: eta: 15:26:08 iteration: 21299/375342 consumed_samples: 21811200 total_loss: 4.666 time: 0.3392 s/iter data_time: 0.2197 s/iter total_throughput: 3018.85 samples/s lr: 8.42e-04 [09/22 05:19:36] lb.utils.events INFO: eta: 15:26:45 iteration: 21399/375342 consumed_samples: 21913600 total_loss: 4.664 time: 0.3392 s/iter data_time: 0.2125 s/iter total_throughput: 3018.87 samples/s lr: 8.46e-04 [09/22 05:20:10] lb.utils.events INFO: eta: 15:27:45 iteration: 21499/375342 consumed_samples: 22016000 total_loss: 4.688 time: 0.3392 s/iter data_time: 0.2244 s/iter total_throughput: 3018.63 samples/s lr: 8.50e-04 [09/22 05:20:44] lb.utils.events INFO: eta: 15:28:24 iteration: 21599/375342 consumed_samples: 22118400 total_loss: 4.672 time: 0.3392 s/iter data_time: 0.2091 s/iter total_throughput: 3018.59 samples/s lr: 8.54e-04 [09/22 05:21:19] lb.utils.events INFO: eta: 15:27:14 iteration: 21699/375342 consumed_samples: 22220800 total_loss: 4.672 time: 0.3393 s/iter data_time: 0.2050 s/iter total_throughput: 3018.37 samples/s lr: 8.58e-04 [09/22 05:21:53] lb.utils.events INFO: eta: 15:27:56 iteration: 21799/375342 consumed_samples: 22323200 total_loss: 4.697 time: 0.3393 s/iter data_time: 0.2107 s/iter total_throughput: 3018.25 samples/s lr: 8.62e-04 [09/22 05:22:28] lb.utils.events INFO: eta: 15:28:45 iteration: 21899/375342 consumed_samples: 22425600 total_loss: 4.676 time: 0.3393 s/iter data_time: 0.2231 s/iter total_throughput: 3017.95 samples/s lr: 8.66e-04 [09/22 05:23:01] lb.utils.events INFO: eta: 15:29:00 iteration: 21999/375342 consumed_samples: 22528000 total_loss: 4.686 time: 0.3393 s/iter data_time: 0.2145 s/iter total_throughput: 3017.93 samples/s lr: 8.70e-04 [09/22 05:23:36] lb.utils.events INFO: eta: 15:28:52 iteration: 22099/375342 consumed_samples: 22630400 total_loss: 4.674 time: 0.3393 s/iter data_time: 0.2148 s/iter total_throughput: 3017.78 samples/s lr: 8.74e-04 [09/22 05:24:10] lb.utils.events INFO: eta: 15:26:29 iteration: 22199/375342 consumed_samples: 22732800 total_loss: 4.652 time: 0.3393 s/iter data_time: 0.2182 s/iter total_throughput: 3017.62 samples/s lr: 8.78e-04 [09/22 05:24:44] lb.utils.events INFO: eta: 15:28:30 iteration: 22299/375342 consumed_samples: 22835200 total_loss: 4.652 time: 0.3394 s/iter data_time: 0.2047 s/iter total_throughput: 3017.51 samples/s lr: 8.82e-04 [09/22 05:25:19] lb.utils.events INFO: eta: 15:26:20 iteration: 22399/375342 consumed_samples: 22937600 total_loss: 4.652 time: 0.3394 s/iter data_time: 0.2224 s/iter total_throughput: 3017.36 samples/s lr: 8.86e-04 [09/22 05:25:53] lb.utils.events INFO: eta: 15:25:24 iteration: 22499/375342 consumed_samples: 23040000 total_loss: 4.652 time: 0.3394 s/iter data_time: 0.2073 s/iter total_throughput: 3017.07 samples/s lr: 8.90e-04 [09/22 05:26:27] lb.utils.events INFO: eta: 15:23:56 iteration: 22599/375342 consumed_samples: 23142400 total_loss: 4.637 time: 0.3394 s/iter data_time: 0.2052 s/iter total_throughput: 3017.01 samples/s lr: 8.93e-04 [09/22 05:27:01] lb.utils.events INFO: eta: 15:27:20 iteration: 22699/375342 consumed_samples: 23244800 total_loss: 4.645 time: 0.3394 s/iter data_time: 0.2094 s/iter total_throughput: 3017.10 samples/s lr: 8.97e-04 [09/22 05:27:35] lb.utils.events INFO: eta: 15:25:49 iteration: 22799/375342 consumed_samples: 23347200 total_loss: 4.641 time: 0.3394 s/iter data_time: 0.2203 s/iter total_throughput: 3016.97 samples/s lr: 9.01e-04 [09/22 05:28:10] lb.utils.events INFO: eta: 15:25:26 iteration: 22899/375342 consumed_samples: 23449600 total_loss: 4.625 time: 0.3394 s/iter data_time: 0.2141 s/iter total_throughput: 3016.92 samples/s lr: 9.05e-04 [09/22 05:28:44] lb.utils.events INFO: eta: 15:23:42 iteration: 22999/375342 consumed_samples: 23552000 total_loss: 4.625 time: 0.3394 s/iter data_time: 0.2263 s/iter total_throughput: 3016.69 samples/s lr: 9.09e-04 [09/22 05:29:18] lb.utils.events INFO: eta: 15:23:02 iteration: 23099/375342 consumed_samples: 23654400 total_loss: 4.625 time: 0.3395 s/iter data_time: 0.2208 s/iter total_throughput: 3016.54 samples/s lr: 9.13e-04 [09/22 05:29:53] lb.utils.events INFO: eta: 15:20:41 iteration: 23199/375342 consumed_samples: 23756800 total_loss: 4.623 time: 0.3395 s/iter data_time: 0.2247 s/iter total_throughput: 3016.39 samples/s lr: 9.17e-04 [09/22 05:30:27] lb.utils.events INFO: eta: 15:20:43 iteration: 23299/375342 consumed_samples: 23859200 total_loss: 4.605 time: 0.3395 s/iter data_time: 0.2137 s/iter total_throughput: 3016.38 samples/s lr: 9.21e-04 [09/22 05:31:01] lb.utils.events INFO: eta: 15:22:47 iteration: 23399/375342 consumed_samples: 23961600 total_loss: 4.605 time: 0.3395 s/iter data_time: 0.2199 s/iter total_throughput: 3016.20 samples/s lr: 9.25e-04 [09/22 05:31:36] lb.utils.events INFO: eta: 15:23:46 iteration: 23499/375342 consumed_samples: 24064000 total_loss: 4.607 time: 0.3395 s/iter data_time: 0.2151 s/iter total_throughput: 3015.99 samples/s lr: 9.29e-04 [09/22 05:32:10] lb.utils.events INFO: eta: 15:23:45 iteration: 23599/375342 consumed_samples: 24166400 total_loss: 4.611 time: 0.3395 s/iter data_time: 0.2101 s/iter total_throughput: 3015.93 samples/s lr: 9.33e-04 [09/22 05:32:44] lb.utils.events INFO: eta: 15:23:35 iteration: 23699/375342 consumed_samples: 24268800 total_loss: 4.6 time: 0.3395 s/iter data_time: 0.2168 s/iter total_throughput: 3015.98 samples/s lr: 9.37e-04 [09/22 05:33:18] lb.utils.events INFO: eta: 15:23:32 iteration: 23799/375342 consumed_samples: 24371200 total_loss: 4.602 time: 0.3395 s/iter data_time: 0.2228 s/iter total_throughput: 3015.96 samples/s lr: 9.41e-04 [09/22 05:33:51] lb.utils.events INFO: eta: 15:24:28 iteration: 23899/375342 consumed_samples: 24473600 total_loss: 4.598 time: 0.3395 s/iter data_time: 0.2042 s/iter total_throughput: 3016.06 samples/s lr: 9.45e-04 [09/22 05:34:25] lb.utils.events INFO: eta: 15:28:10 iteration: 23999/375342 consumed_samples: 24576000 total_loss: 4.59 time: 0.3395 s/iter data_time: 0.2221 s/iter total_throughput: 3016.10 samples/s lr: 9.49e-04 [09/22 05:34:59] lb.utils.events INFO: eta: 15:26:16 iteration: 24099/375342 consumed_samples: 24678400 total_loss: 4.582 time: 0.3395 s/iter data_time: 0.2153 s/iter total_throughput: 3015.98 samples/s lr: 9.53e-04 [09/22 05:35:33] lb.utils.events INFO: eta: 15:28:48 iteration: 24199/375342 consumed_samples: 24780800 total_loss: 4.572 time: 0.3395 s/iter data_time: 0.2256 s/iter total_throughput: 3016.08 samples/s lr: 9.57e-04 [09/22 05:36:07] lb.utils.events INFO: eta: 15:28:11 iteration: 24299/375342 consumed_samples: 24883200 total_loss: 4.566 time: 0.3395 s/iter data_time: 0.2141 s/iter total_throughput: 3016.16 samples/s lr: 9.61e-04 [09/22 05:36:41] lb.utils.events INFO: eta: 15:27:55 iteration: 24399/375342 consumed_samples: 24985600 total_loss: 4.586 time: 0.3395 s/iter data_time: 0.2199 s/iter total_throughput: 3016.04 samples/s lr: 9.65e-04 [09/22 05:37:15] lb.utils.events INFO: eta: 15:26:34 iteration: 24499/375342 consumed_samples: 25088000 total_loss: 4.598 time: 0.3395 s/iter data_time: 0.2102 s/iter total_throughput: 3015.95 samples/s lr: 9.68e-04 [09/22 05:37:49] lb.utils.events INFO: eta: 15:26:44 iteration: 24599/375342 consumed_samples: 25190400 total_loss: 4.578 time: 0.3395 s/iter data_time: 0.2191 s/iter total_throughput: 3015.91 samples/s lr: 9.72e-04 [09/22 05:38:24] lb.utils.events INFO: eta: 15:25:20 iteration: 24699/375342 consumed_samples: 25292800 total_loss: 4.586 time: 0.3395 s/iter data_time: 0.2159 s/iter total_throughput: 3015.79 samples/s lr: 9.76e-04 [09/22 05:38:58] lb.utils.events INFO: eta: 15:25:04 iteration: 24799/375342 consumed_samples: 25395200 total_loss: 4.592 time: 0.3395 s/iter data_time: 0.2046 s/iter total_throughput: 3015.82 samples/s lr: 9.80e-04 [09/22 05:39:32] lb.utils.events INFO: eta: 15:23:32 iteration: 24899/375342 consumed_samples: 25497600 total_loss: 4.572 time: 0.3395 s/iter data_time: 0.2144 s/iter total_throughput: 3015.79 samples/s lr: 9.84e-04 [09/22 05:40:06] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0024999 [09/22 05:40:06] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/22 05:40:06] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/22 05:40:10] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0837 s/iter. Inference: 0.1679 s/iter. Eval: 0.0018 s/iter. Total: 0.2535 s/iter. ETA=0:00:09 [09/22 05:40:16] lb.evaluation.evaluator INFO: Inference done 28672/50000. Dataloading: 0.0677 s/iter. Inference: 0.2201 s/iter. Eval: 0.0019 s/iter. Total: 0.2899 s/iter. ETA=0:00:05 [09/22 05:40:21] lb.evaluation.evaluator INFO: Inference done 45056/50000. Dataloading: 0.0438 s/iter. Inference: 0.2568 s/iter. Eval: 0.0019 s/iter. Total: 0.3027 s/iter. ETA=0:00:01 [09/22 05:40:22] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/22 05:40:22] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.985298 (0.000260 s / iter per device, on 8 devices) [09/22 05:40:22] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:11 (0.000224 s / iter per device, on 8 devices) [09/22 05:40:22] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/22 05:40:22] lb.evaluation.utils INFO: copypaste: Acc@1=55.96 [09/22 05:40:22] lb.evaluation.utils INFO: copypaste: Acc@5=80.464 [09/22 05:40:22] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 55.96000, better than last best score 52.44800 @ iteration 19999. [09/22 05:40:22] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/22 05:40:23] lb.utils.events INFO: eta: 15:20:10 iteration: 24999/375342 consumed_samples: 25600000 total_loss: 4.566 time: 0.3395 s/iter data_time: 0.2182 s/iter total_throughput: 3015.77 samples/s lr: 9.88e-04 [09/22 05:40:55] lb.utils.events INFO: eta: 15:25:30 iteration: 25099/375342 consumed_samples: 25702400 total_loss: 4.559 time: 0.3395 s/iter data_time: 0.2196 s/iter total_throughput: 3016.43 samples/s lr: 9.89e-04 [09/22 05:41:29] lb.utils.events INFO: eta: 15:21:46 iteration: 25199/375342 consumed_samples: 25804800 total_loss: 4.543 time: 0.3395 s/iter data_time: 0.2106 s/iter total_throughput: 3016.42 samples/s lr: 9.89e-04 [09/22 05:42:03] lb.utils.events INFO: eta: 15:20:04 iteration: 25299/375342 consumed_samples: 25907200 total_loss: 4.557 time: 0.3395 s/iter data_time: 0.2170 s/iter total_throughput: 3016.40 samples/s lr: 9.89e-04 [09/22 05:42:37] lb.utils.events INFO: eta: 15:19:48 iteration: 25399/375342 consumed_samples: 26009600 total_loss: 4.539 time: 0.3395 s/iter data_time: 0.2183 s/iter total_throughput: 3016.35 samples/s lr: 9.89e-04 [09/22 05:43:11] lb.utils.events INFO: eta: 15:20:05 iteration: 25499/375342 consumed_samples: 26112000 total_loss: 4.531 time: 0.3395 s/iter data_time: 0.2227 s/iter total_throughput: 3016.31 samples/s lr: 9.89e-04 [09/22 05:43:45] lb.utils.events INFO: eta: 15:20:04 iteration: 25599/375342 consumed_samples: 26214400 total_loss: 4.531 time: 0.3395 s/iter data_time: 0.2236 s/iter total_throughput: 3016.29 samples/s lr: 9.89e-04 [09/22 05:44:19] lb.utils.events INFO: eta: 15:20:53 iteration: 25699/375342 consumed_samples: 26316800 total_loss: 4.541 time: 0.3395 s/iter data_time: 0.2148 s/iter total_throughput: 3016.25 samples/s lr: 9.89e-04 [09/22 05:44:53] lb.utils.events INFO: eta: 15:22:16 iteration: 25799/375342 consumed_samples: 26419200 total_loss: 4.539 time: 0.3395 s/iter data_time: 0.2090 s/iter total_throughput: 3016.34 samples/s lr: 9.89e-04 [09/22 05:45:27] lb.utils.events INFO: eta: 15:21:13 iteration: 25899/375342 consumed_samples: 26521600 total_loss: 4.531 time: 0.3395 s/iter data_time: 0.2189 s/iter total_throughput: 3016.32 samples/s lr: 9.88e-04 [09/22 05:46:01] lb.utils.events INFO: eta: 15:21:13 iteration: 25999/375342 consumed_samples: 26624000 total_loss: 4.523 time: 0.3395 s/iter data_time: 0.2164 s/iter total_throughput: 3016.17 samples/s lr: 9.88e-04 [09/22 05:46:35] lb.utils.events INFO: eta: 15:17:40 iteration: 26099/375342 consumed_samples: 26726400 total_loss: 4.514 time: 0.3395 s/iter data_time: 0.2102 s/iter total_throughput: 3016.22 samples/s lr: 9.88e-04 [09/22 05:47:09] lb.utils.events INFO: eta: 15:19:05 iteration: 26199/375342 consumed_samples: 26828800 total_loss: 4.514 time: 0.3395 s/iter data_time: 0.2001 s/iter total_throughput: 3016.19 samples/s lr: 9.88e-04 [09/22 05:47:43] lb.utils.events INFO: eta: 15:20:06 iteration: 26299/375342 consumed_samples: 26931200 total_loss: 4.525 time: 0.3395 s/iter data_time: 0.2174 s/iter total_throughput: 3016.04 samples/s lr: 9.88e-04 [09/22 05:48:18] lb.utils.events INFO: eta: 15:18:33 iteration: 26399/375342 consumed_samples: 27033600 total_loss: 4.516 time: 0.3395 s/iter data_time: 0.2142 s/iter total_throughput: 3015.90 samples/s lr: 9.88e-04 [09/22 05:48:52] lb.utils.events INFO: eta: 15:18:56 iteration: 26499/375342 consumed_samples: 27136000 total_loss: 4.5 time: 0.3395 s/iter data_time: 0.2227 s/iter total_throughput: 3015.91 samples/s lr: 9.88e-04 [09/22 05:49:26] lb.utils.events INFO: eta: 15:17:29 iteration: 26599/375342 consumed_samples: 27238400 total_loss: 4.504 time: 0.3395 s/iter data_time: 0.2173 s/iter total_throughput: 3015.79 samples/s lr: 9.88e-04 [09/22 05:50:00] lb.utils.events INFO: eta: 15:17:03 iteration: 26699/375342 consumed_samples: 27340800 total_loss: 4.516 time: 0.3396 s/iter data_time: 0.2142 s/iter total_throughput: 3015.70 samples/s lr: 9.88e-04 [09/22 05:50:34] lb.utils.events INFO: eta: 15:18:04 iteration: 26799/375342 consumed_samples: 27443200 total_loss: 4.518 time: 0.3395 s/iter data_time: 0.2075 s/iter total_throughput: 3015.79 samples/s lr: 9.88e-04 [09/22 05:51:08] lb.utils.events INFO: eta: 15:16:44 iteration: 26899/375342 consumed_samples: 27545600 total_loss: 4.502 time: 0.3396 s/iter data_time: 0.2210 s/iter total_throughput: 3015.68 samples/s lr: 9.88e-04 [09/22 05:51:42] lb.utils.events INFO: eta: 15:17:32 iteration: 26999/375342 consumed_samples: 27648000 total_loss: 4.5 time: 0.3395 s/iter data_time: 0.1974 s/iter total_throughput: 3015.77 samples/s lr: 9.87e-04 [09/22 05:52:16] lb.utils.events INFO: eta: 15:19:18 iteration: 27099/375342 consumed_samples: 27750400 total_loss: 4.502 time: 0.3395 s/iter data_time: 0.2094 s/iter total_throughput: 3015.91 samples/s lr: 9.87e-04 [09/22 05:52:49] lb.utils.events INFO: eta: 15:20:04 iteration: 27199/375342 consumed_samples: 27852800 total_loss: 4.496 time: 0.3395 s/iter data_time: 0.2043 s/iter total_throughput: 3016.10 samples/s lr: 9.87e-04 [09/22 05:53:22] lb.utils.events INFO: eta: 15:21:01 iteration: 27299/375342 consumed_samples: 27955200 total_loss: 4.477 time: 0.3395 s/iter data_time: 0.2189 s/iter total_throughput: 3016.23 samples/s lr: 9.87e-04 [09/22 05:53:57] lb.utils.events INFO: eta: 15:20:30 iteration: 27399/375342 consumed_samples: 28057600 total_loss: 4.469 time: 0.3395 s/iter data_time: 0.2118 s/iter total_throughput: 3016.17 samples/s lr: 9.87e-04 [09/22 05:54:30] lb.utils.events INFO: eta: 15:20:01 iteration: 27499/375342 consumed_samples: 28160000 total_loss: 4.488 time: 0.3395 s/iter data_time: 0.2101 s/iter total_throughput: 3016.22 samples/s lr: 9.87e-04 [09/22 05:55:04] lb.utils.events INFO: eta: 15:20:52 iteration: 27599/375342 consumed_samples: 28262400 total_loss: 4.453 time: 0.3395 s/iter data_time: 0.2055 s/iter total_throughput: 3016.29 samples/s lr: 9.87e-04 [09/22 05:55:38] lb.utils.events INFO: eta: 15:20:53 iteration: 27699/375342 consumed_samples: 28364800 total_loss: 4.457 time: 0.3395 s/iter data_time: 0.2019 s/iter total_throughput: 3016.41 samples/s lr: 9.87e-04 [09/22 05:56:11] lb.utils.events INFO: eta: 15:19:28 iteration: 27799/375342 consumed_samples: 28467200 total_loss: 4.471 time: 0.3395 s/iter data_time: 0.2086 s/iter total_throughput: 3016.54 samples/s lr: 9.87e-04 [09/22 05:56:45] lb.utils.events INFO: eta: 15:21:37 iteration: 27899/375342 consumed_samples: 28569600 total_loss: 4.469 time: 0.3395 s/iter data_time: 0.2030 s/iter total_throughput: 3016.58 samples/s lr: 9.87e-04 [09/22 05:57:19] lb.utils.events INFO: eta: 15:22:26 iteration: 27999/375342 consumed_samples: 28672000 total_loss: 4.449 time: 0.3395 s/iter data_time: 0.2170 s/iter total_throughput: 3016.55 samples/s lr: 9.86e-04 [09/22 05:57:54] lb.utils.events INFO: eta: 15:20:41 iteration: 28099/375342 consumed_samples: 28774400 total_loss: 4.447 time: 0.3395 s/iter data_time: 0.2097 s/iter total_throughput: 3016.19 samples/s lr: 9.86e-04 [09/22 05:58:29] lb.utils.events INFO: eta: 15:17:31 iteration: 28199/375342 consumed_samples: 28876800 total_loss: 4.463 time: 0.3395 s/iter data_time: 0.2149 s/iter total_throughput: 3016.08 samples/s lr: 9.86e-04 [09/22 05:59:03] lb.utils.events INFO: eta: 15:17:09 iteration: 28299/375342 consumed_samples: 28979200 total_loss: 4.469 time: 0.3395 s/iter data_time: 0.2034 s/iter total_throughput: 3016.08 samples/s lr: 9.86e-04 [09/22 05:59:36] lb.utils.events INFO: eta: 15:17:57 iteration: 28399/375342 consumed_samples: 29081600 total_loss: 4.455 time: 0.3395 s/iter data_time: 0.2210 s/iter total_throughput: 3016.13 samples/s lr: 9.86e-04 [09/22 06:00:10] lb.utils.events INFO: eta: 15:19:51 iteration: 28499/375342 consumed_samples: 29184000 total_loss: 4.459 time: 0.3395 s/iter data_time: 0.2089 s/iter total_throughput: 3016.20 samples/s lr: 9.86e-04 [09/22 06:00:44] lb.utils.events INFO: eta: 15:22:33 iteration: 28599/375342 consumed_samples: 29286400 total_loss: 4.439 time: 0.3395 s/iter data_time: 0.2415 s/iter total_throughput: 3016.16 samples/s lr: 9.86e-04 [09/22 06:01:19] lb.utils.events INFO: eta: 15:22:51 iteration: 28699/375342 consumed_samples: 29388800 total_loss: 4.418 time: 0.3395 s/iter data_time: 0.2286 s/iter total_throughput: 3016.01 samples/s lr: 9.86e-04 [09/22 06:01:53] lb.utils.events INFO: eta: 15:22:01 iteration: 28799/375342 consumed_samples: 29491200 total_loss: 4.424 time: 0.3395 s/iter data_time: 0.2345 s/iter total_throughput: 3015.78 samples/s lr: 9.86e-04 [09/22 06:02:27] lb.utils.events INFO: eta: 15:21:14 iteration: 28899/375342 consumed_samples: 29593600 total_loss: 4.41 time: 0.3396 s/iter data_time: 0.2161 s/iter total_throughput: 3015.72 samples/s lr: 9.86e-04 [09/22 06:03:02] lb.utils.events INFO: eta: 15:21:33 iteration: 28999/375342 consumed_samples: 29696000 total_loss: 4.414 time: 0.3396 s/iter data_time: 0.2100 s/iter total_throughput: 3015.64 samples/s lr: 9.85e-04 [09/22 06:03:36] lb.utils.events INFO: eta: 15:21:41 iteration: 29099/375342 consumed_samples: 29798400 total_loss: 4.404 time: 0.3396 s/iter data_time: 0.2102 s/iter total_throughput: 3015.66 samples/s lr: 9.85e-04 [09/22 06:04:09] lb.utils.events INFO: eta: 15:22:50 iteration: 29199/375342 consumed_samples: 29900800 total_loss: 4.402 time: 0.3396 s/iter data_time: 0.2205 s/iter total_throughput: 3015.70 samples/s lr: 9.85e-04 [09/22 06:04:43] lb.utils.events INFO: eta: 15:21:01 iteration: 29299/375342 consumed_samples: 30003200 total_loss: 4.4 time: 0.3396 s/iter data_time: 0.2076 s/iter total_throughput: 3015.73 samples/s lr: 9.85e-04 [09/22 06:05:17] lb.utils.events INFO: eta: 15:20:22 iteration: 29399/375342 consumed_samples: 30105600 total_loss: 4.424 time: 0.3396 s/iter data_time: 0.2051 s/iter total_throughput: 3015.69 samples/s lr: 9.85e-04 [09/22 06:05:51] lb.utils.events INFO: eta: 15:22:50 iteration: 29499/375342 consumed_samples: 30208000 total_loss: 4.43 time: 0.3395 s/iter data_time: 0.2116 s/iter total_throughput: 3015.76 samples/s lr: 9.85e-04 [09/22 06:06:25] lb.utils.events INFO: eta: 15:22:34 iteration: 29599/375342 consumed_samples: 30310400 total_loss: 4.424 time: 0.3395 s/iter data_time: 0.2244 s/iter total_throughput: 3015.76 samples/s lr: 9.85e-04 [09/22 06:06:59] lb.utils.events INFO: eta: 15:20:44 iteration: 29699/375342 consumed_samples: 30412800 total_loss: 4.42 time: 0.3395 s/iter data_time: 0.2070 s/iter total_throughput: 3015.80 samples/s lr: 9.85e-04 [09/22 06:07:33] lb.utils.events INFO: eta: 15:20:46 iteration: 29799/375342 consumed_samples: 30515200 total_loss: 4.42 time: 0.3396 s/iter data_time: 0.2060 s/iter total_throughput: 3015.73 samples/s lr: 9.85e-04 [09/22 06:08:07] lb.utils.events INFO: eta: 15:21:31 iteration: 29899/375342 consumed_samples: 30617600 total_loss: 4.408 time: 0.3396 s/iter data_time: 0.2174 s/iter total_throughput: 3015.75 samples/s lr: 9.85e-04 [09/22 06:08:41] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0029999 [09/22 06:08:42] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/22 06:08:42] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/22 06:08:46] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0908 s/iter. Inference: 0.1659 s/iter. Eval: 0.0020 s/iter. Total: 0.2587 s/iter. ETA=0:00:09 [09/22 06:08:51] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.0981 s/iter. Inference: 0.1902 s/iter. Eval: 0.0021 s/iter. Total: 0.2905 s/iter. ETA=0:00:05 [09/22 06:08:56] lb.evaluation.evaluator INFO: Inference done 47104/50000. Dataloading: 0.0609 s/iter. Inference: 0.2313 s/iter. Eval: 0.0021 s/iter. Total: 0.2943 s/iter. ETA=0:00:00 [09/22 06:08:57] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/22 06:08:57] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.566647 (0.000251 s / iter per device, on 8 devices) [09/22 06:08:57] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:09 (0.000199 s / iter per device, on 8 devices) [09/22 06:08:57] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/22 06:08:57] lb.evaluation.utils INFO: copypaste: Acc@1=60.004000000000005 [09/22 06:08:57] lb.evaluation.utils INFO: copypaste: Acc@5=83.05799999999999 [09/22 06:08:57] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 60.00400, better than last best score 55.96000 @ iteration 24999. [09/22 06:08:57] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/22 06:08:58] lb.utils.events INFO: eta: 15:20:14 iteration: 29999/375342 consumed_samples: 30720000 total_loss: 4.412 time: 0.3396 s/iter data_time: 0.2120 s/iter total_throughput: 3015.68 samples/s lr: 9.84e-04 [09/22 06:09:30] lb.utils.events INFO: eta: 15:20:59 iteration: 30099/375342 consumed_samples: 30822400 total_loss: 4.42 time: 0.3395 s/iter data_time: 0.2495 s/iter total_throughput: 3016.29 samples/s lr: 9.84e-04 [09/22 06:10:04] lb.utils.events INFO: eta: 15:26:59 iteration: 30199/375342 consumed_samples: 30924800 total_loss: 4.402 time: 0.3395 s/iter data_time: 0.2411 s/iter total_throughput: 3016.17 samples/s lr: 9.84e-04 [09/22 06:10:38] lb.utils.events INFO: eta: 15:26:13 iteration: 30299/375342 consumed_samples: 31027200 total_loss: 4.391 time: 0.3395 s/iter data_time: 0.2158 s/iter total_throughput: 3016.06 samples/s lr: 9.84e-04 [09/22 06:11:12] lb.utils.events INFO: eta: 15:27:04 iteration: 30399/375342 consumed_samples: 31129600 total_loss: 4.365 time: 0.3395 s/iter data_time: 0.1968 s/iter total_throughput: 3016.15 samples/s lr: 9.84e-04 [09/22 06:11:46] lb.utils.events INFO: eta: 15:25:03 iteration: 30499/375342 consumed_samples: 31232000 total_loss: 4.367 time: 0.3395 s/iter data_time: 0.2106 s/iter total_throughput: 3016.08 samples/s lr: 9.84e-04 [09/22 06:12:20] lb.utils.events INFO: eta: 15:21:24 iteration: 30599/375342 consumed_samples: 31334400 total_loss: 4.389 time: 0.3395 s/iter data_time: 0.2227 s/iter total_throughput: 3016.09 samples/s lr: 9.84e-04 [09/22 06:12:54] lb.utils.events INFO: eta: 15:19:44 iteration: 30699/375342 consumed_samples: 31436800 total_loss: 4.387 time: 0.3395 s/iter data_time: 0.2203 s/iter total_throughput: 3016.05 samples/s lr: 9.84e-04 [09/22 06:13:28] lb.utils.events INFO: eta: 15:19:13 iteration: 30799/375342 consumed_samples: 31539200 total_loss: 4.359 time: 0.3395 s/iter data_time: 0.2028 s/iter total_throughput: 3016.03 samples/s lr: 9.84e-04 [09/22 06:14:02] lb.utils.events INFO: eta: 15:20:14 iteration: 30899/375342 consumed_samples: 31641600 total_loss: 4.373 time: 0.3395 s/iter data_time: 0.2077 s/iter total_throughput: 3016.09 samples/s lr: 9.84e-04 [09/22 06:14:36] lb.utils.events INFO: eta: 15:19:52 iteration: 30999/375342 consumed_samples: 31744000 total_loss: 4.367 time: 0.3395 s/iter data_time: 0.2162 s/iter total_throughput: 3015.96 samples/s lr: 9.83e-04 [09/22 06:15:11] lb.utils.events INFO: eta: 15:16:25 iteration: 31099/375342 consumed_samples: 31846400 total_loss: 4.352 time: 0.3395 s/iter data_time: 0.2068 s/iter total_throughput: 3015.83 samples/s lr: 9.83e-04 [09/22 06:15:45] lb.utils.events INFO: eta: 15:07:49 iteration: 31199/375342 consumed_samples: 31948800 total_loss: 4.375 time: 0.3396 s/iter data_time: 0.2049 s/iter total_throughput: 3015.76 samples/s lr: 9.83e-04 [09/22 06:16:19] lb.utils.events INFO: eta: 15:08:03 iteration: 31299/375342 consumed_samples: 32051200 total_loss: 4.375 time: 0.3396 s/iter data_time: 0.2096 s/iter total_throughput: 3015.74 samples/s lr: 9.83e-04 [09/22 06:16:53] lb.utils.events INFO: eta: 15:07:33 iteration: 31399/375342 consumed_samples: 32153600 total_loss: 4.363 time: 0.3395 s/iter data_time: 0.2149 s/iter total_throughput: 3015.78 samples/s lr: 9.83e-04 [09/22 06:17:27] lb.utils.events INFO: eta: 15:07:31 iteration: 31499/375342 consumed_samples: 32256000 total_loss: 4.348 time: 0.3395 s/iter data_time: 0.2099 s/iter total_throughput: 3015.77 samples/s lr: 9.83e-04 [09/22 06:18:01] lb.utils.events INFO: eta: 15:07:10 iteration: 31599/375342 consumed_samples: 32358400 total_loss: 4.344 time: 0.3396 s/iter data_time: 0.2096 s/iter total_throughput: 3015.72 samples/s lr: 9.83e-04 [09/22 06:18:35] lb.utils.events INFO: eta: 15:07:22 iteration: 31699/375342 consumed_samples: 32460800 total_loss: 4.34 time: 0.3395 s/iter data_time: 0.1937 s/iter total_throughput: 3015.82 samples/s lr: 9.83e-04 [09/22 06:19:09] lb.utils.events INFO: eta: 15:09:25 iteration: 31799/375342 consumed_samples: 32563200 total_loss: 4.359 time: 0.3395 s/iter data_time: 0.2095 s/iter total_throughput: 3015.83 samples/s lr: 9.83e-04 [09/22 06:19:42] lb.utils.events INFO: eta: 15:07:42 iteration: 31899/375342 consumed_samples: 32665600 total_loss: 4.359 time: 0.3395 s/iter data_time: 0.2161 s/iter total_throughput: 3015.88 samples/s lr: 9.82e-04 [09/22 06:20:16] lb.utils.events INFO: eta: 15:08:07 iteration: 31999/375342 consumed_samples: 32768000 total_loss: 4.344 time: 0.3395 s/iter data_time: 0.2093 s/iter total_throughput: 3015.87 samples/s lr: 9.82e-04 [09/22 06:20:51] lb.utils.events INFO: eta: 15:06:35 iteration: 32099/375342 consumed_samples: 32870400 total_loss: 4.352 time: 0.3396 s/iter data_time: 0.2173 s/iter total_throughput: 3015.71 samples/s lr: 9.82e-04 [09/22 06:21:26] lb.utils.events INFO: eta: 15:06:55 iteration: 32199/375342 consumed_samples: 32972800 total_loss: 4.354 time: 0.3396 s/iter data_time: 0.2198 s/iter total_throughput: 3015.52 samples/s lr: 9.82e-04 [09/22 06:22:00] lb.utils.events INFO: eta: 15:05:55 iteration: 32299/375342 consumed_samples: 33075200 total_loss: 4.346 time: 0.3396 s/iter data_time: 0.2257 s/iter total_throughput: 3015.41 samples/s lr: 9.82e-04 [09/22 06:22:34] lb.utils.events INFO: eta: 15:05:39 iteration: 32399/375342 consumed_samples: 33177600 total_loss: 4.32 time: 0.3396 s/iter data_time: 0.2238 s/iter total_throughput: 3015.39 samples/s lr: 9.82e-04 [09/22 06:23:08] lb.utils.events INFO: eta: 15:04:02 iteration: 32499/375342 consumed_samples: 33280000 total_loss: 4.324 time: 0.3396 s/iter data_time: 0.2094 s/iter total_throughput: 3015.40 samples/s lr: 9.82e-04 [09/22 06:23:42] lb.utils.events INFO: eta: 15:03:19 iteration: 32599/375342 consumed_samples: 33382400 total_loss: 4.324 time: 0.3396 s/iter data_time: 0.2064 s/iter total_throughput: 3015.31 samples/s lr: 9.82e-04 [09/22 06:24:17] lb.utils.events INFO: eta: 15:01:52 iteration: 32699/375342 consumed_samples: 33484800 total_loss: 4.312 time: 0.3396 s/iter data_time: 0.2025 s/iter total_throughput: 3015.16 samples/s lr: 9.82e-04 [09/22 06:24:51] lb.utils.events INFO: eta: 15:01:49 iteration: 32799/375342 consumed_samples: 33587200 total_loss: 4.334 time: 0.3396 s/iter data_time: 0.2115 s/iter total_throughput: 3015.18 samples/s lr: 9.81e-04 [09/22 06:25:25] lb.utils.events INFO: eta: 15:00:59 iteration: 32899/375342 consumed_samples: 33689600 total_loss: 4.328 time: 0.3396 s/iter data_time: 0.2122 s/iter total_throughput: 3015.16 samples/s lr: 9.81e-04 [09/22 06:25:59] lb.utils.events INFO: eta: 15:01:31 iteration: 32999/375342 consumed_samples: 33792000 total_loss: 4.309 time: 0.3396 s/iter data_time: 0.2182 s/iter total_throughput: 3015.11 samples/s lr: 9.81e-04 [09/22 06:26:33] lb.utils.events INFO: eta: 15:01:23 iteration: 33099/375342 consumed_samples: 33894400 total_loss: 4.328 time: 0.3396 s/iter data_time: 0.2111 s/iter total_throughput: 3015.04 samples/s lr: 9.81e-04 [09/22 06:27:07] lb.utils.events INFO: eta: 15:01:41 iteration: 33199/375342 consumed_samples: 33996800 total_loss: 4.311 time: 0.3396 s/iter data_time: 0.2090 s/iter total_throughput: 3015.04 samples/s lr: 9.81e-04 [09/22 06:27:41] lb.utils.events INFO: eta: 15:01:59 iteration: 33299/375342 consumed_samples: 34099200 total_loss: 4.305 time: 0.3396 s/iter data_time: 0.2144 s/iter total_throughput: 3014.95 samples/s lr: 9.81e-04 [09/22 06:28:16] lb.utils.events INFO: eta: 15:00:42 iteration: 33399/375342 consumed_samples: 34201600 total_loss: 4.303 time: 0.3397 s/iter data_time: 0.2137 s/iter total_throughput: 3014.79 samples/s lr: 9.81e-04 [09/22 06:28:50] lb.utils.events INFO: eta: 15:00:20 iteration: 33499/375342 consumed_samples: 34304000 total_loss: 4.285 time: 0.3397 s/iter data_time: 0.2138 s/iter total_throughput: 3014.75 samples/s lr: 9.81e-04 [09/22 06:29:24] lb.utils.events INFO: eta: 15:02:15 iteration: 33599/375342 consumed_samples: 34406400 total_loss: 4.301 time: 0.3397 s/iter data_time: 0.2177 s/iter total_throughput: 3014.65 samples/s lr: 9.81e-04 [09/22 06:29:58] lb.utils.events INFO: eta: 15:01:19 iteration: 33699/375342 consumed_samples: 34508800 total_loss: 4.297 time: 0.3397 s/iter data_time: 0.2038 s/iter total_throughput: 3014.66 samples/s lr: 9.80e-04 [09/22 06:30:32] lb.utils.events INFO: eta: 14:59:21 iteration: 33799/375342 consumed_samples: 34611200 total_loss: 4.275 time: 0.3397 s/iter data_time: 0.2137 s/iter total_throughput: 3014.61 samples/s lr: 9.80e-04 [09/22 06:31:07] lb.utils.events INFO: eta: 14:59:02 iteration: 33899/375342 consumed_samples: 34713600 total_loss: 4.287 time: 0.3397 s/iter data_time: 0.2154 s/iter total_throughput: 3014.46 samples/s lr: 9.80e-04 [09/22 06:31:41] lb.utils.events INFO: eta: 14:58:32 iteration: 33999/375342 consumed_samples: 34816000 total_loss: 4.273 time: 0.3397 s/iter data_time: 0.2137 s/iter total_throughput: 3014.39 samples/s lr: 9.80e-04 [09/22 06:32:16] lb.utils.events INFO: eta: 15:00:53 iteration: 34099/375342 consumed_samples: 34918400 total_loss: 4.273 time: 0.3397 s/iter data_time: 0.2179 s/iter total_throughput: 3014.28 samples/s lr: 9.80e-04 [09/22 06:32:50] lb.utils.events INFO: eta: 15:05:50 iteration: 34199/375342 consumed_samples: 35020800 total_loss: 4.268 time: 0.3397 s/iter data_time: 0.2061 s/iter total_throughput: 3014.25 samples/s lr: 9.80e-04 [09/22 06:33:24] lb.utils.events INFO: eta: 15:04:42 iteration: 34299/375342 consumed_samples: 35123200 total_loss: 4.297 time: 0.3397 s/iter data_time: 0.2050 s/iter total_throughput: 3014.15 samples/s lr: 9.80e-04 [09/22 06:33:58] lb.utils.events INFO: eta: 15:05:55 iteration: 34399/375342 consumed_samples: 35225600 total_loss: 4.309 time: 0.3397 s/iter data_time: 0.2023 s/iter total_throughput: 3014.18 samples/s lr: 9.80e-04 [09/22 06:34:32] lb.utils.events INFO: eta: 15:07:14 iteration: 34499/375342 consumed_samples: 35328000 total_loss: 4.285 time: 0.3397 s/iter data_time: 0.2130 s/iter total_throughput: 3014.23 samples/s lr: 9.80e-04 [09/22 06:35:06] lb.utils.events INFO: eta: 15:06:54 iteration: 34599/375342 consumed_samples: 35430400 total_loss: 4.285 time: 0.3397 s/iter data_time: 0.2067 s/iter total_throughput: 3014.20 samples/s lr: 9.79e-04 [09/22 06:35:40] lb.utils.events INFO: eta: 15:07:57 iteration: 34699/375342 consumed_samples: 35532800 total_loss: 4.285 time: 0.3397 s/iter data_time: 0.2163 s/iter total_throughput: 3014.10 samples/s lr: 9.79e-04 [09/22 06:36:14] lb.utils.events INFO: eta: 15:08:57 iteration: 34799/375342 consumed_samples: 35635200 total_loss: 4.283 time: 0.3397 s/iter data_time: 0.2211 s/iter total_throughput: 3014.11 samples/s lr: 9.79e-04 [09/22 06:36:49] lb.utils.events INFO: eta: 15:08:41 iteration: 34899/375342 consumed_samples: 35737600 total_loss: 4.268 time: 0.3397 s/iter data_time: 0.2150 s/iter total_throughput: 3013.99 samples/s lr: 9.79e-04 [09/22 06:37:23] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0034999 [09/22 06:37:24] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/22 06:37:24] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/22 06:37:28] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0508 s/iter. Inference: 0.1902 s/iter. Eval: 0.0020 s/iter. Total: 0.2430 s/iter. ETA=0:00:08 [09/22 06:37:33] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.0981 s/iter. Inference: 0.1865 s/iter. Eval: 0.0020 s/iter. Total: 0.2866 s/iter. ETA=0:00:05 [09/22 06:37:38] lb.evaluation.evaluator INFO: Inference done 48128/50000. Dataloading: 0.1023 s/iter. Inference: 0.1812 s/iter. Eval: 0.0020 s/iter. Total: 0.2856 s/iter. ETA=0:00:00 [09/22 06:37:39] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/22 06:37:39] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.341234 (0.000247 s / iter per device, on 8 devices) [09/22 06:37:39] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000159 s / iter per device, on 8 devices) [09/22 06:37:39] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/22 06:37:39] lb.evaluation.utils INFO: copypaste: Acc@1=62.980000000000004 [09/22 06:37:39] lb.evaluation.utils INFO: copypaste: Acc@5=85.548 [09/22 06:37:39] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 62.98000, better than last best score 60.00400 @ iteration 29999. [09/22 06:37:39] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/22 06:37:40] lb.utils.events INFO: eta: 15:07:30 iteration: 34999/375342 consumed_samples: 35840000 total_loss: 4.266 time: 0.3398 s/iter data_time: 0.2169 s/iter total_throughput: 3013.84 samples/s lr: 9.79e-04 [09/22 06:38:11] lb.utils.events INFO: eta: 15:11:03 iteration: 35099/375342 consumed_samples: 35942400 total_loss: 4.264 time: 0.3397 s/iter data_time: 0.2248 s/iter total_throughput: 3014.44 samples/s lr: 9.79e-04 [09/22 06:38:46] lb.utils.events INFO: eta: 15:10:03 iteration: 35199/375342 consumed_samples: 36044800 total_loss: 4.252 time: 0.3397 s/iter data_time: 0.2499 s/iter total_throughput: 3014.33 samples/s lr: 9.79e-04 [09/22 06:39:20] lb.utils.events INFO: eta: 15:09:32 iteration: 35299/375342 consumed_samples: 36147200 total_loss: 4.25 time: 0.3397 s/iter data_time: 0.2074 s/iter total_throughput: 3014.25 samples/s lr: 9.79e-04 [09/22 06:39:54] lb.utils.events INFO: eta: 15:07:21 iteration: 35399/375342 consumed_samples: 36249600 total_loss: 4.25 time: 0.3397 s/iter data_time: 0.2073 s/iter total_throughput: 3014.32 samples/s lr: 9.78e-04 [09/22 06:40:27] lb.utils.events INFO: eta: 15:06:04 iteration: 35499/375342 consumed_samples: 36352000 total_loss: 4.24 time: 0.3397 s/iter data_time: 0.2090 s/iter total_throughput: 3014.35 samples/s lr: 9.78e-04 [09/22 06:41:01] lb.utils.events INFO: eta: 15:03:59 iteration: 35599/375342 consumed_samples: 36454400 total_loss: 4.246 time: 0.3397 s/iter data_time: 0.2042 s/iter total_throughput: 3014.41 samples/s lr: 9.78e-04 [09/22 06:41:35] lb.utils.events INFO: eta: 15:03:01 iteration: 35699/375342 consumed_samples: 36556800 total_loss: 4.266 time: 0.3397 s/iter data_time: 0.2074 s/iter total_throughput: 3014.47 samples/s lr: 9.78e-04 [09/22 06:42:09] lb.utils.events INFO: eta: 15:00:37 iteration: 35799/375342 consumed_samples: 36659200 total_loss: 4.24 time: 0.3397 s/iter data_time: 0.2284 s/iter total_throughput: 3014.45 samples/s lr: 9.78e-04 [09/22 06:42:43] lb.utils.events INFO: eta: 15:02:45 iteration: 35899/375342 consumed_samples: 36761600 total_loss: 4.236 time: 0.3397 s/iter data_time: 0.2103 s/iter total_throughput: 3014.53 samples/s lr: 9.78e-04 [09/22 06:43:17] lb.utils.events INFO: eta: 15:02:52 iteration: 35999/375342 consumed_samples: 36864000 total_loss: 4.254 time: 0.3397 s/iter data_time: 0.2084 s/iter total_throughput: 3014.47 samples/s lr: 9.78e-04 [09/22 06:43:51] lb.utils.events INFO: eta: 14:58:36 iteration: 36099/375342 consumed_samples: 36966400 total_loss: 4.254 time: 0.3397 s/iter data_time: 0.2213 s/iter total_throughput: 3014.47 samples/s lr: 9.78e-04 [09/22 06:44:25] lb.utils.events INFO: eta: 14:56:39 iteration: 36199/375342 consumed_samples: 37068800 total_loss: 4.223 time: 0.3397 s/iter data_time: 0.2115 s/iter total_throughput: 3014.45 samples/s lr: 9.77e-04 [09/22 06:44:59] lb.utils.events INFO: eta: 14:55:34 iteration: 36299/375342 consumed_samples: 37171200 total_loss: 4.23 time: 0.3397 s/iter data_time: 0.2198 s/iter total_throughput: 3014.46 samples/s lr: 9.77e-04 [09/22 06:45:33] lb.utils.events INFO: eta: 14:54:55 iteration: 36399/375342 consumed_samples: 37273600 total_loss: 4.242 time: 0.3397 s/iter data_time: 0.2175 s/iter total_throughput: 3014.42 samples/s lr: 9.77e-04 [09/22 06:46:07] lb.utils.events INFO: eta: 14:55:12 iteration: 36499/375342 consumed_samples: 37376000 total_loss: 4.238 time: 0.3397 s/iter data_time: 0.2132 s/iter total_throughput: 3014.42 samples/s lr: 9.77e-04 [09/22 06:46:41] lb.utils.events INFO: eta: 14:55:52 iteration: 36599/375342 consumed_samples: 37478400 total_loss: 4.221 time: 0.3397 s/iter data_time: 0.2048 s/iter total_throughput: 3014.34 samples/s lr: 9.77e-04 [09/22 06:47:15] lb.utils.events INFO: eta: 14:57:09 iteration: 36699/375342 consumed_samples: 37580800 total_loss: 4.209 time: 0.3397 s/iter data_time: 0.2185 s/iter total_throughput: 3014.52 samples/s lr: 9.77e-04 [09/22 06:47:49] lb.utils.events INFO: eta: 14:55:56 iteration: 36799/375342 consumed_samples: 37683200 total_loss: 4.232 time: 0.3397 s/iter data_time: 0.2122 s/iter total_throughput: 3014.45 samples/s lr: 9.77e-04 [09/22 06:48:23] lb.utils.events INFO: eta: 14:54:25 iteration: 36899/375342 consumed_samples: 37785600 total_loss: 4.25 time: 0.3397 s/iter data_time: 0.2204 s/iter total_throughput: 3014.37 samples/s lr: 9.77e-04 [09/22 06:48:57] lb.utils.events INFO: eta: 14:54:09 iteration: 36999/375342 consumed_samples: 37888000 total_loss: 4.236 time: 0.3397 s/iter data_time: 0.2132 s/iter total_throughput: 3014.38 samples/s lr: 9.76e-04 [09/22 06:49:30] lb.utils.events INFO: eta: 14:52:13 iteration: 37099/375342 consumed_samples: 37990400 total_loss: 4.221 time: 0.3397 s/iter data_time: 0.2104 s/iter total_throughput: 3014.51 samples/s lr: 9.76e-04 [09/22 06:50:04] lb.utils.events INFO: eta: 14:51:36 iteration: 37199/375342 consumed_samples: 38092800 total_loss: 4.232 time: 0.3397 s/iter data_time: 0.2115 s/iter total_throughput: 3014.52 samples/s lr: 9.76e-04 [09/22 06:50:38] lb.utils.events INFO: eta: 14:51:21 iteration: 37299/375342 consumed_samples: 38195200 total_loss: 4.234 time: 0.3397 s/iter data_time: 0.2113 s/iter total_throughput: 3014.50 samples/s lr: 9.76e-04 [09/22 06:51:12] lb.utils.events INFO: eta: 14:51:16 iteration: 37399/375342 consumed_samples: 38297600 total_loss: 4.234 time: 0.3397 s/iter data_time: 0.1979 s/iter total_throughput: 3014.64 samples/s lr: 9.76e-04 [09/22 06:51:46] lb.utils.events INFO: eta: 14:50:37 iteration: 37499/375342 consumed_samples: 38400000 total_loss: 4.219 time: 0.3397 s/iter data_time: 0.2056 s/iter total_throughput: 3014.72 samples/s lr: 9.76e-04 [09/22 06:52:20] lb.utils.events INFO: eta: 14:48:37 iteration: 37599/375342 consumed_samples: 38502400 total_loss: 4.217 time: 0.3397 s/iter data_time: 0.2218 s/iter total_throughput: 3014.68 samples/s lr: 9.76e-04 [09/22 06:52:54] lb.utils.events INFO: eta: 14:46:02 iteration: 37699/375342 consumed_samples: 38604800 total_loss: 4.203 time: 0.3397 s/iter data_time: 0.2188 s/iter total_throughput: 3014.65 samples/s lr: 9.76e-04 [09/22 06:53:28] lb.utils.events INFO: eta: 14:45:07 iteration: 37799/375342 consumed_samples: 38707200 total_loss: 4.195 time: 0.3397 s/iter data_time: 0.2221 s/iter total_throughput: 3014.54 samples/s lr: 9.75e-04 [09/22 06:54:02] lb.utils.events INFO: eta: 14:44:48 iteration: 37899/375342 consumed_samples: 38809600 total_loss: 4.217 time: 0.3397 s/iter data_time: 0.1978 s/iter total_throughput: 3014.49 samples/s lr: 9.75e-04 [09/22 06:54:36] lb.utils.events INFO: eta: 14:44:36 iteration: 37999/375342 consumed_samples: 38912000 total_loss: 4.221 time: 0.3397 s/iter data_time: 0.2295 s/iter total_throughput: 3014.49 samples/s lr: 9.75e-04 [09/22 06:55:10] lb.utils.events INFO: eta: 14:43:07 iteration: 38099/375342 consumed_samples: 39014400 total_loss: 4.207 time: 0.3397 s/iter data_time: 0.2118 s/iter total_throughput: 3014.59 samples/s lr: 9.75e-04 [09/22 06:55:44] lb.utils.events INFO: eta: 14:41:21 iteration: 38199/375342 consumed_samples: 39116800 total_loss: 4.193 time: 0.3397 s/iter data_time: 0.2079 s/iter total_throughput: 3014.61 samples/s lr: 9.75e-04 [09/22 06:56:17] lb.utils.events INFO: eta: 14:43:42 iteration: 38299/375342 consumed_samples: 39219200 total_loss: 4.201 time: 0.3397 s/iter data_time: 0.2064 s/iter total_throughput: 3014.77 samples/s lr: 9.75e-04 [09/22 06:56:51] lb.utils.events INFO: eta: 14:42:02 iteration: 38399/375342 consumed_samples: 39321600 total_loss: 4.215 time: 0.3397 s/iter data_time: 0.2067 s/iter total_throughput: 3014.77 samples/s lr: 9.75e-04 [09/22 06:57:25] lb.utils.events INFO: eta: 14:40:53 iteration: 38499/375342 consumed_samples: 39424000 total_loss: 4.205 time: 0.3397 s/iter data_time: 0.2018 s/iter total_throughput: 3014.69 samples/s lr: 9.75e-04 [09/22 06:57:59] lb.utils.events INFO: eta: 14:43:33 iteration: 38599/375342 consumed_samples: 39526400 total_loss: 4.197 time: 0.3397 s/iter data_time: 0.2177 s/iter total_throughput: 3014.76 samples/s lr: 9.74e-04 [09/22 06:58:33] lb.utils.events INFO: eta: 14:43:47 iteration: 38699/375342 consumed_samples: 39628800 total_loss: 4.201 time: 0.3397 s/iter data_time: 0.2127 s/iter total_throughput: 3014.84 samples/s lr: 9.74e-04 [09/22 06:59:06] lb.utils.events INFO: eta: 14:44:21 iteration: 38799/375342 consumed_samples: 39731200 total_loss: 4.219 time: 0.3396 s/iter data_time: 0.2087 s/iter total_throughput: 3014.91 samples/s lr: 9.74e-04 [09/22 06:59:41] lb.utils.events INFO: eta: 14:40:53 iteration: 38899/375342 consumed_samples: 39833600 total_loss: 4.207 time: 0.3397 s/iter data_time: 0.2211 s/iter total_throughput: 3014.79 samples/s lr: 9.74e-04 [09/22 07:00:15] lb.utils.events INFO: eta: 14:42:26 iteration: 38999/375342 consumed_samples: 39936000 total_loss: 4.195 time: 0.3397 s/iter data_time: 0.2106 s/iter total_throughput: 3014.86 samples/s lr: 9.74e-04 [09/22 07:00:48] lb.utils.events INFO: eta: 14:42:25 iteration: 39099/375342 consumed_samples: 40038400 total_loss: 4.182 time: 0.3396 s/iter data_time: 0.2086 s/iter total_throughput: 3014.89 samples/s lr: 9.74e-04 [09/22 07:01:22] lb.utils.events INFO: eta: 14:39:57 iteration: 39199/375342 consumed_samples: 40140800 total_loss: 4.188 time: 0.3397 s/iter data_time: 0.2230 s/iter total_throughput: 3014.86 samples/s lr: 9.74e-04 [09/22 07:01:56] lb.utils.events INFO: eta: 14:42:09 iteration: 39299/375342 consumed_samples: 40243200 total_loss: 4.211 time: 0.3396 s/iter data_time: 0.2135 s/iter total_throughput: 3014.96 samples/s lr: 9.73e-04 [09/22 07:02:30] lb.utils.events INFO: eta: 14:41:53 iteration: 39399/375342 consumed_samples: 40345600 total_loss: 4.211 time: 0.3396 s/iter data_time: 0.2055 s/iter total_throughput: 3015.01 samples/s lr: 9.73e-04 [09/22 07:03:04] lb.utils.events INFO: eta: 14:39:48 iteration: 39499/375342 consumed_samples: 40448000 total_loss: 4.211 time: 0.3396 s/iter data_time: 0.2258 s/iter total_throughput: 3015.04 samples/s lr: 9.73e-04 [09/22 07:03:38] lb.utils.events INFO: eta: 14:38:30 iteration: 39599/375342 consumed_samples: 40550400 total_loss: 4.182 time: 0.3396 s/iter data_time: 0.2116 s/iter total_throughput: 3014.90 samples/s lr: 9.73e-04 [09/22 07:04:12] lb.utils.events INFO: eta: 14:39:17 iteration: 39699/375342 consumed_samples: 40652800 total_loss: 4.189 time: 0.3396 s/iter data_time: 0.1986 s/iter total_throughput: 3014.94 samples/s lr: 9.73e-04 [09/22 07:04:46] lb.utils.events INFO: eta: 14:37:29 iteration: 39799/375342 consumed_samples: 40755200 total_loss: 4.191 time: 0.3396 s/iter data_time: 0.2041 s/iter total_throughput: 3014.95 samples/s lr: 9.73e-04 [09/22 07:05:20] lb.utils.events INFO: eta: 14:42:39 iteration: 39899/375342 consumed_samples: 40857600 total_loss: 4.172 time: 0.3396 s/iter data_time: 0.2145 s/iter total_throughput: 3014.93 samples/s lr: 9.73e-04 [09/22 07:05:55] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0039999 [09/22 07:05:55] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/22 07:05:55] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/22 07:06:00] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0971 s/iter. Inference: 0.1608 s/iter. Eval: 0.0020 s/iter. Total: 0.2599 s/iter. ETA=0:00:09 [09/22 07:06:05] lb.evaluation.evaluator INFO: Inference done 28672/50000. Dataloading: 0.0724 s/iter. Inference: 0.2114 s/iter. Eval: 0.0020 s/iter. Total: 0.2859 s/iter. ETA=0:00:05 [09/22 07:06:10] lb.evaluation.evaluator INFO: Inference done 46080/50000. Dataloading: 0.0654 s/iter. Inference: 0.2242 s/iter. Eval: 0.0020 s/iter. Total: 0.2916 s/iter. ETA=0:00:00 [09/22 07:06:11] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/22 07:06:11] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.607516 (0.000252 s / iter per device, on 8 devices) [09/22 07:06:11] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:09 (0.000197 s / iter per device, on 8 devices) [09/22 07:06:11] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/22 07:06:11] lb.evaluation.utils INFO: copypaste: Acc@1=64.67 [09/22 07:06:11] lb.evaluation.utils INFO: copypaste: Acc@5=86.424 [09/22 07:06:11] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 64.67000, better than last best score 62.98000 @ iteration 34999. [09/22 07:06:11] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/22 07:06:11] lb.utils.events INFO: eta: 14:38:34 iteration: 39999/375342 consumed_samples: 40960000 total_loss: 4.195 time: 0.3397 s/iter data_time: 0.2063 s/iter total_throughput: 3014.75 samples/s lr: 9.73e-04 [09/22 07:06:43] lb.utils.events INFO: eta: 14:41:58 iteration: 40099/375342 consumed_samples: 41062400 total_loss: 4.188 time: 0.3396 s/iter data_time: 0.2302 s/iter total_throughput: 3015.31 samples/s lr: 9.72e-04 [09/22 07:07:17] lb.utils.events INFO: eta: 14:42:48 iteration: 40199/375342 consumed_samples: 41164800 total_loss: 4.193 time: 0.3396 s/iter data_time: 0.2245 s/iter total_throughput: 3015.30 samples/s lr: 9.72e-04 [09/22 07:07:51] lb.utils.events INFO: eta: 14:42:26 iteration: 40299/375342 consumed_samples: 41267200 total_loss: 4.195 time: 0.3396 s/iter data_time: 0.2180 s/iter total_throughput: 3015.36 samples/s lr: 9.72e-04 [09/22 07:08:24] lb.utils.events INFO: eta: 14:41:58 iteration: 40399/375342 consumed_samples: 41369600 total_loss: 4.178 time: 0.3396 s/iter data_time: 0.1996 s/iter total_throughput: 3015.41 samples/s lr: 9.72e-04 [09/22 07:08:58] lb.utils.events INFO: eta: 14:42:52 iteration: 40499/375342 consumed_samples: 41472000 total_loss: 4.18 time: 0.3396 s/iter data_time: 0.2116 s/iter total_throughput: 3015.49 samples/s lr: 9.72e-04 [09/22 07:09:32] lb.utils.events INFO: eta: 14:43:48 iteration: 40599/375342 consumed_samples: 41574400 total_loss: 4.176 time: 0.3396 s/iter data_time: 0.2199 s/iter total_throughput: 3015.49 samples/s lr: 9.72e-04 [09/22 07:10:06] lb.utils.events INFO: eta: 14:40:21 iteration: 40699/375342 consumed_samples: 41676800 total_loss: 4.164 time: 0.3396 s/iter data_time: 0.2135 s/iter total_throughput: 3015.46 samples/s lr: 9.72e-04 [09/22 07:10:40] lb.utils.events INFO: eta: 14:40:37 iteration: 40799/375342 consumed_samples: 41779200 total_loss: 4.141 time: 0.3396 s/iter data_time: 0.2217 s/iter total_throughput: 3015.46 samples/s lr: 9.71e-04 [09/22 07:11:14] lb.utils.events INFO: eta: 14:39:14 iteration: 40899/375342 consumed_samples: 41881600 total_loss: 4.141 time: 0.3396 s/iter data_time: 0.2115 s/iter total_throughput: 3015.43 samples/s lr: 9.71e-04 [09/22 07:11:48] lb.utils.events INFO: eta: 14:39:48 iteration: 40999/375342 consumed_samples: 41984000 total_loss: 4.15 time: 0.3396 s/iter data_time: 0.1979 s/iter total_throughput: 3015.50 samples/s lr: 9.71e-04 [09/22 07:12:21] lb.utils.events INFO: eta: 14:38:17 iteration: 41099/375342 consumed_samples: 42086400 total_loss: 4.162 time: 0.3396 s/iter data_time: 0.2206 s/iter total_throughput: 3015.57 samples/s lr: 9.71e-04 [09/22 07:12:55] lb.utils.events INFO: eta: 14:37:03 iteration: 41199/375342 consumed_samples: 42188800 total_loss: 4.186 time: 0.3396 s/iter data_time: 0.2168 s/iter total_throughput: 3015.59 samples/s lr: 9.71e-04 [09/22 07:13:29] lb.utils.events INFO: eta: 14:34:41 iteration: 41299/375342 consumed_samples: 42291200 total_loss: 4.186 time: 0.3396 s/iter data_time: 0.2183 s/iter total_throughput: 3015.73 samples/s lr: 9.71e-04 [09/22 07:14:03] lb.utils.events INFO: eta: 14:32:38 iteration: 41399/375342 consumed_samples: 42393600 total_loss: 4.156 time: 0.3396 s/iter data_time: 0.2267 s/iter total_throughput: 3015.70 samples/s lr: 9.71e-04 [09/22 07:14:36] lb.utils.events INFO: eta: 14:30:45 iteration: 41499/375342 consumed_samples: 42496000 total_loss: 4.143 time: 0.3396 s/iter data_time: 0.2179 s/iter total_throughput: 3015.75 samples/s lr: 9.70e-04 [09/22 07:15:10] lb.utils.events INFO: eta: 14:32:38 iteration: 41599/375342 consumed_samples: 42598400 total_loss: 4.162 time: 0.3395 s/iter data_time: 0.1921 s/iter total_throughput: 3015.86 samples/s lr: 9.70e-04 [09/22 07:15:43] lb.utils.events INFO: eta: 14:32:59 iteration: 41699/375342 consumed_samples: 42700800 total_loss: 4.148 time: 0.3395 s/iter data_time: 0.2080 s/iter total_throughput: 3015.92 samples/s lr: 9.70e-04 [09/22 07:16:17] lb.utils.events INFO: eta: 14:32:22 iteration: 41799/375342 consumed_samples: 42803200 total_loss: 4.145 time: 0.3395 s/iter data_time: 0.1947 s/iter total_throughput: 3016.04 samples/s lr: 9.70e-04 [09/22 07:16:51] lb.utils.events INFO: eta: 14:30:35 iteration: 41899/375342 consumed_samples: 42905600 total_loss: 4.156 time: 0.3395 s/iter data_time: 0.2065 s/iter total_throughput: 3016.11 samples/s lr: 9.70e-04 [09/22 07:17:25] lb.utils.events INFO: eta: 14:30:31 iteration: 41999/375342 consumed_samples: 43008000 total_loss: 4.15 time: 0.3395 s/iter data_time: 0.2251 s/iter total_throughput: 3016.01 samples/s lr: 9.70e-04 [09/22 07:17:59] lb.utils.events INFO: eta: 14:30:18 iteration: 42099/375342 consumed_samples: 43110400 total_loss: 4.156 time: 0.3395 s/iter data_time: 0.2011 s/iter total_throughput: 3016.06 samples/s lr: 9.70e-04 [09/22 07:18:33] lb.utils.events INFO: eta: 14:30:02 iteration: 42199/375342 consumed_samples: 43212800 total_loss: 4.176 time: 0.3395 s/iter data_time: 0.2260 s/iter total_throughput: 3015.99 samples/s lr: 9.69e-04 [09/22 07:19:07] lb.utils.events INFO: eta: 14:30:11 iteration: 42299/375342 consumed_samples: 43315200 total_loss: 4.172 time: 0.3395 s/iter data_time: 0.2162 s/iter total_throughput: 3015.92 samples/s lr: 9.69e-04 [09/22 07:19:41] lb.utils.events INFO: eta: 14:31:02 iteration: 42399/375342 consumed_samples: 43417600 total_loss: 4.164 time: 0.3395 s/iter data_time: 0.2106 s/iter total_throughput: 3015.90 samples/s lr: 9.69e-04 [09/22 07:20:16] lb.utils.events INFO: eta: 14:30:46 iteration: 42499/375342 consumed_samples: 43520000 total_loss: 4.141 time: 0.3395 s/iter data_time: 0.2328 s/iter total_throughput: 3015.81 samples/s lr: 9.69e-04 [09/22 07:20:50] lb.utils.events INFO: eta: 14:30:25 iteration: 42599/375342 consumed_samples: 43622400 total_loss: 4.133 time: 0.3396 s/iter data_time: 0.2179 s/iter total_throughput: 3015.69 samples/s lr: 9.69e-04 [09/22 07:21:25] lb.utils.events INFO: eta: 14:30:40 iteration: 42699/375342 consumed_samples: 43724800 total_loss: 4.166 time: 0.3396 s/iter data_time: 0.2256 s/iter total_throughput: 3015.62 samples/s lr: 9.69e-04 [09/22 07:21:59] lb.utils.events INFO: eta: 14:31:55 iteration: 42799/375342 consumed_samples: 43827200 total_loss: 4.156 time: 0.3396 s/iter data_time: 0.2227 s/iter total_throughput: 3015.50 samples/s lr: 9.69e-04 [09/22 07:22:34] lb.utils.events INFO: eta: 14:32:12 iteration: 42899/375342 consumed_samples: 43929600 total_loss: 4.135 time: 0.3396 s/iter data_time: 0.2184 s/iter total_throughput: 3015.39 samples/s lr: 9.68e-04 [09/22 07:23:08] lb.utils.events INFO: eta: 14:32:21 iteration: 42999/375342 consumed_samples: 44032000 total_loss: 4.143 time: 0.3396 s/iter data_time: 0.2184 s/iter total_throughput: 3015.34 samples/s lr: 9.68e-04 [09/22 07:23:42] lb.utils.events INFO: eta: 14:32:25 iteration: 43099/375342 consumed_samples: 44134400 total_loss: 4.152 time: 0.3396 s/iter data_time: 0.2094 s/iter total_throughput: 3015.35 samples/s lr: 9.68e-04 [09/22 07:24:16] lb.utils.events INFO: eta: 14:32:01 iteration: 43199/375342 consumed_samples: 44236800 total_loss: 4.145 time: 0.3396 s/iter data_time: 0.2201 s/iter total_throughput: 3015.31 samples/s lr: 9.68e-04 [09/22 07:24:50] lb.utils.events INFO: eta: 14:30:36 iteration: 43299/375342 consumed_samples: 44339200 total_loss: 4.127 time: 0.3396 s/iter data_time: 0.2166 s/iter total_throughput: 3015.21 samples/s lr: 9.68e-04 [09/22 07:25:25] lb.utils.events INFO: eta: 14:29:57 iteration: 43399/375342 consumed_samples: 44441600 total_loss: 4.113 time: 0.3396 s/iter data_time: 0.2180 s/iter total_throughput: 3015.05 samples/s lr: 9.68e-04 [09/22 07:25:59] lb.utils.events INFO: eta: 14:30:51 iteration: 43499/375342 consumed_samples: 44544000 total_loss: 4.139 time: 0.3396 s/iter data_time: 0.2161 s/iter total_throughput: 3015.09 samples/s lr: 9.68e-04 [09/22 07:26:33] lb.utils.events INFO: eta: 14:30:40 iteration: 43599/375342 consumed_samples: 44646400 total_loss: 4.143 time: 0.3396 s/iter data_time: 0.2169 s/iter total_throughput: 3015.02 samples/s lr: 9.67e-04 [09/22 07:27:07] lb.utils.events INFO: eta: 14:30:32 iteration: 43699/375342 consumed_samples: 44748800 total_loss: 4.137 time: 0.3396 s/iter data_time: 0.2205 s/iter total_throughput: 3014.96 samples/s lr: 9.67e-04 [09/22 07:27:42] lb.utils.events INFO: eta: 14:30:08 iteration: 43799/375342 consumed_samples: 44851200 total_loss: 4.139 time: 0.3396 s/iter data_time: 0.1977 s/iter total_throughput: 3014.93 samples/s lr: 9.67e-04 [09/22 07:28:16] lb.utils.events INFO: eta: 14:30:00 iteration: 43899/375342 consumed_samples: 44953600 total_loss: 4.113 time: 0.3396 s/iter data_time: 0.2078 s/iter total_throughput: 3014.91 samples/s lr: 9.67e-04 [09/22 07:28:50] lb.utils.events INFO: eta: 14:26:46 iteration: 43999/375342 consumed_samples: 45056000 total_loss: 4.105 time: 0.3397 s/iter data_time: 0.2243 s/iter total_throughput: 3014.77 samples/s lr: 9.67e-04 [09/22 07:29:25] lb.utils.events INFO: eta: 14:25:46 iteration: 44099/375342 consumed_samples: 45158400 total_loss: 4.109 time: 0.3397 s/iter data_time: 0.2117 s/iter total_throughput: 3014.70 samples/s lr: 9.67e-04 [09/22 07:29:59] lb.utils.events INFO: eta: 14:23:39 iteration: 44199/375342 consumed_samples: 45260800 total_loss: 4.107 time: 0.3397 s/iter data_time: 0.2277 s/iter total_throughput: 3014.53 samples/s lr: 9.67e-04 [09/22 07:30:34] lb.utils.events INFO: eta: 14:25:43 iteration: 44299/375342 consumed_samples: 45363200 total_loss: 4.115 time: 0.3397 s/iter data_time: 0.2134 s/iter total_throughput: 3014.47 samples/s lr: 9.66e-04 [09/22 07:31:08] lb.utils.events INFO: eta: 14:26:45 iteration: 44399/375342 consumed_samples: 45465600 total_loss: 4.113 time: 0.3397 s/iter data_time: 0.2100 s/iter total_throughput: 3014.44 samples/s lr: 9.66e-04 [09/22 07:31:43] lb.utils.events INFO: eta: 14:26:30 iteration: 44499/375342 consumed_samples: 45568000 total_loss: 4.111 time: 0.3397 s/iter data_time: 0.2150 s/iter total_throughput: 3014.31 samples/s lr: 9.66e-04 [09/22 07:32:17] lb.utils.events INFO: eta: 14:24:47 iteration: 44599/375342 consumed_samples: 45670400 total_loss: 4.109 time: 0.3397 s/iter data_time: 0.2344 s/iter total_throughput: 3014.21 samples/s lr: 9.66e-04 [09/22 07:32:50] lb.utils.events INFO: eta: 14:23:55 iteration: 44699/375342 consumed_samples: 45772800 total_loss: 4.154 time: 0.3397 s/iter data_time: 0.2169 s/iter total_throughput: 3014.31 samples/s lr: 9.66e-04 [09/22 07:33:25] lb.utils.events INFO: eta: 14:23:45 iteration: 44799/375342 consumed_samples: 45875200 total_loss: 4.133 time: 0.3397 s/iter data_time: 0.2097 s/iter total_throughput: 3014.22 samples/s lr: 9.66e-04 [09/22 07:33:59] lb.utils.events INFO: eta: 14:23:11 iteration: 44899/375342 consumed_samples: 45977600 total_loss: 4.127 time: 0.3397 s/iter data_time: 0.2206 s/iter total_throughput: 3014.15 samples/s lr: 9.65e-04 [09/22 07:34:33] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0044999 [09/22 07:34:34] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/22 07:34:34] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/22 07:34:38] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0943 s/iter. Inference: 0.1630 s/iter. Eval: 0.0021 s/iter. Total: 0.2593 s/iter. ETA=0:00:09 [09/22 07:34:43] lb.evaluation.evaluator INFO: Inference done 28672/50000. Dataloading: 0.0949 s/iter. Inference: 0.1888 s/iter. Eval: 0.0021 s/iter. Total: 0.2858 s/iter. ETA=0:00:05 [09/22 07:34:48] lb.evaluation.evaluator INFO: Inference done 46080/50000. Dataloading: 0.0947 s/iter. Inference: 0.1958 s/iter. Eval: 0.0020 s/iter. Total: 0.2925 s/iter. ETA=0:00:00 [09/22 07:34:49] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/22 07:34:49] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.632699 (0.000253 s / iter per device, on 8 devices) [09/22 07:34:49] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:08 (0.000173 s / iter per device, on 8 devices) [09/22 07:34:49] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/22 07:34:49] lb.evaluation.utils INFO: copypaste: Acc@1=65.848 [09/22 07:34:49] lb.evaluation.utils INFO: copypaste: Acc@5=87.354 [09/22 07:34:49] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 65.84800, better than last best score 64.67000 @ iteration 39999. [09/22 07:34:49] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/22 07:34:50] lb.utils.events INFO: eta: 14:24:03 iteration: 44999/375342 consumed_samples: 46080000 total_loss: 4.139 time: 0.3397 s/iter data_time: 0.2185 s/iter total_throughput: 3014.13 samples/s lr: 9.65e-04 [09/22 07:35:22] lb.utils.events INFO: eta: 14:26:16 iteration: 45099/375342 consumed_samples: 46182400 total_loss: 4.113 time: 0.3397 s/iter data_time: 0.2475 s/iter total_throughput: 3014.47 samples/s lr: 9.65e-04 [09/22 07:35:56] lb.utils.events INFO: eta: 14:28:58 iteration: 45199/375342 consumed_samples: 46284800 total_loss: 4.117 time: 0.3397 s/iter data_time: 0.2032 s/iter total_throughput: 3014.51 samples/s lr: 9.65e-04 [09/22 07:36:30] lb.utils.events INFO: eta: 14:27:27 iteration: 45299/375342 consumed_samples: 46387200 total_loss: 4.117 time: 0.3397 s/iter data_time: 0.2034 s/iter total_throughput: 3014.53 samples/s lr: 9.65e-04 [09/22 07:37:04] lb.utils.events INFO: eta: 14:27:58 iteration: 45399/375342 consumed_samples: 46489600 total_loss: 4.092 time: 0.3397 s/iter data_time: 0.2070 s/iter total_throughput: 3014.46 samples/s lr: 9.65e-04 [09/22 07:37:38] lb.utils.events INFO: eta: 14:27:43 iteration: 45499/375342 consumed_samples: 46592000 total_loss: 4.098 time: 0.3397 s/iter data_time: 0.2078 s/iter total_throughput: 3014.42 samples/s lr: 9.65e-04 [09/22 07:38:13] lb.utils.events INFO: eta: 14:28:00 iteration: 45599/375342 consumed_samples: 46694400 total_loss: 4.102 time: 0.3397 s/iter data_time: 0.2224 s/iter total_throughput: 3014.35 samples/s lr: 9.64e-04 [09/22 07:38:47] lb.utils.events INFO: eta: 14:26:55 iteration: 45699/375342 consumed_samples: 46796800 total_loss: 4.109 time: 0.3397 s/iter data_time: 0.2061 s/iter total_throughput: 3014.37 samples/s lr: 9.64e-04 [09/22 07:39:21] lb.utils.events INFO: eta: 14:27:01 iteration: 45799/375342 consumed_samples: 46899200 total_loss: 4.115 time: 0.3397 s/iter data_time: 0.2083 s/iter total_throughput: 3014.38 samples/s lr: 9.64e-04 [09/22 07:39:54] lb.utils.events INFO: eta: 14:27:42 iteration: 45899/375342 consumed_samples: 47001600 total_loss: 4.121 time: 0.3397 s/iter data_time: 0.2099 s/iter total_throughput: 3014.41 samples/s lr: 9.64e-04 [09/22 07:40:29] lb.utils.events INFO: eta: 14:26:40 iteration: 45999/375342 consumed_samples: 47104000 total_loss: 4.123 time: 0.3397 s/iter data_time: 0.2257 s/iter total_throughput: 3014.24 samples/s lr: 9.64e-04 [09/22 07:41:03] lb.utils.events INFO: eta: 14:22:49 iteration: 46099/375342 consumed_samples: 47206400 total_loss: 4.125 time: 0.3397 s/iter data_time: 0.2102 s/iter total_throughput: 3014.27 samples/s lr: 9.64e-04 [09/22 07:41:37] lb.utils.events INFO: eta: 14:20:17 iteration: 46199/375342 consumed_samples: 47308800 total_loss: 4.109 time: 0.3397 s/iter data_time: 0.2034 s/iter total_throughput: 3014.23 samples/s lr: 9.63e-04 [09/22 07:42:12] lb.utils.events INFO: eta: 14:18:49 iteration: 46299/375342 consumed_samples: 47411200 total_loss: 4.09 time: 0.3397 s/iter data_time: 0.2267 s/iter total_throughput: 3014.05 samples/s lr: 9.63e-04 [09/22 07:42:46] lb.utils.events INFO: eta: 14:18:01 iteration: 46399/375342 consumed_samples: 47513600 total_loss: 4.084 time: 0.3397 s/iter data_time: 0.2058 s/iter total_throughput: 3014.06 samples/s lr: 9.63e-04 [09/22 07:43:20] lb.utils.events INFO: eta: 14:16:38 iteration: 46499/375342 consumed_samples: 47616000 total_loss: 4.098 time: 0.3397 s/iter data_time: 0.2032 s/iter total_throughput: 3014.12 samples/s lr: 9.63e-04 [09/22 07:43:54] lb.utils.events INFO: eta: 14:17:03 iteration: 46599/375342 consumed_samples: 47718400 total_loss: 4.098 time: 0.3397 s/iter data_time: 0.2071 s/iter total_throughput: 3014.13 samples/s lr: 9.63e-04 [09/22 07:44:28] lb.utils.events INFO: eta: 14:17:38 iteration: 46699/375342 consumed_samples: 47820800 total_loss: 4.094 time: 0.3397 s/iter data_time: 0.2172 s/iter total_throughput: 3014.01 samples/s lr: 9.63e-04 [09/22 07:45:03] lb.utils.events INFO: eta: 14:15:44 iteration: 46799/375342 consumed_samples: 47923200 total_loss: 4.086 time: 0.3398 s/iter data_time: 0.2218 s/iter total_throughput: 3013.80 samples/s lr: 9.63e-04 [09/22 07:45:38] lb.utils.events INFO: eta: 14:15:23 iteration: 46899/375342 consumed_samples: 48025600 total_loss: 4.086 time: 0.3398 s/iter data_time: 0.2101 s/iter total_throughput: 3013.77 samples/s lr: 9.62e-04 [09/22 07:46:11] lb.utils.events INFO: eta: 14:16:43 iteration: 46999/375342 consumed_samples: 48128000 total_loss: 4.102 time: 0.3398 s/iter data_time: 0.2137 s/iter total_throughput: 3013.79 samples/s lr: 9.62e-04 [09/22 07:46:46] lb.utils.events INFO: eta: 14:16:28 iteration: 47099/375342 consumed_samples: 48230400 total_loss: 4.117 time: 0.3398 s/iter data_time: 0.2264 s/iter total_throughput: 3013.67 samples/s lr: 9.62e-04 [09/22 07:47:21] lb.utils.events INFO: eta: 14:15:25 iteration: 47199/375342 consumed_samples: 48332800 total_loss: 4.117 time: 0.3398 s/iter data_time: 0.2483 s/iter total_throughput: 3013.48 samples/s lr: 9.62e-04 [09/22 07:47:55] lb.utils.events INFO: eta: 14:17:43 iteration: 47299/375342 consumed_samples: 48435200 total_loss: 4.113 time: 0.3398 s/iter data_time: 0.2207 s/iter total_throughput: 3013.53 samples/s lr: 9.62e-04 [09/22 07:48:29] lb.utils.events INFO: eta: 14:16:33 iteration: 47399/375342 consumed_samples: 48537600 total_loss: 4.094 time: 0.3398 s/iter data_time: 0.2087 s/iter total_throughput: 3013.45 samples/s lr: 9.62e-04 [09/22 07:49:03] lb.utils.events INFO: eta: 14:15:46 iteration: 47499/375342 consumed_samples: 48640000 total_loss: 4.098 time: 0.3398 s/iter data_time: 0.2156 s/iter total_throughput: 3013.39 samples/s lr: 9.61e-04 [09/22 07:49:37] lb.utils.events INFO: eta: 14:17:08 iteration: 47599/375342 consumed_samples: 48742400 total_loss: 4.109 time: 0.3398 s/iter data_time: 0.2176 s/iter total_throughput: 3013.42 samples/s lr: 9.61e-04 [09/22 07:50:11] lb.utils.events INFO: eta: 14:17:50 iteration: 47699/375342 consumed_samples: 48844800 total_loss: 4.094 time: 0.3398 s/iter data_time: 0.2002 s/iter total_throughput: 3013.52 samples/s lr: 9.61e-04 [09/22 07:50:44] lb.utils.events INFO: eta: 14:20:13 iteration: 47799/375342 consumed_samples: 48947200 total_loss: 4.082 time: 0.3398 s/iter data_time: 0.2089 s/iter total_throughput: 3013.60 samples/s lr: 9.61e-04 [09/22 07:51:19] lb.utils.events INFO: eta: 14:18:23 iteration: 47899/375342 consumed_samples: 49049600 total_loss: 4.09 time: 0.3398 s/iter data_time: 0.2222 s/iter total_throughput: 3013.53 samples/s lr: 9.61e-04 [09/22 07:51:53] lb.utils.events INFO: eta: 14:17:36 iteration: 47999/375342 consumed_samples: 49152000 total_loss: 4.07 time: 0.3398 s/iter data_time: 0.2137 s/iter total_throughput: 3013.48 samples/s lr: 9.61e-04 [09/22 07:52:27] lb.utils.events INFO: eta: 14:17:17 iteration: 48099/375342 consumed_samples: 49254400 total_loss: 4.068 time: 0.3398 s/iter data_time: 0.2171 s/iter total_throughput: 3013.40 samples/s lr: 9.60e-04 [09/22 07:53:01] lb.utils.events INFO: eta: 14:19:49 iteration: 48199/375342 consumed_samples: 49356800 total_loss: 4.09 time: 0.3398 s/iter data_time: 0.2135 s/iter total_throughput: 3013.39 samples/s lr: 9.60e-04 [09/22 07:53:35] lb.utils.events INFO: eta: 14:19:33 iteration: 48299/375342 consumed_samples: 49459200 total_loss: 4.08 time: 0.3398 s/iter data_time: 0.2092 s/iter total_throughput: 3013.41 samples/s lr: 9.60e-04 [09/22 07:54:09] lb.utils.events INFO: eta: 14:21:36 iteration: 48399/375342 consumed_samples: 49561600 total_loss: 4.09 time: 0.3398 s/iter data_time: 0.2025 s/iter total_throughput: 3013.40 samples/s lr: 9.60e-04 [09/22 07:54:43] lb.utils.events INFO: eta: 14:22:57 iteration: 48499/375342 consumed_samples: 49664000 total_loss: 4.102 time: 0.3398 s/iter data_time: 0.2135 s/iter total_throughput: 3013.45 samples/s lr: 9.60e-04 [09/22 07:55:17] lb.utils.events INFO: eta: 14:22:51 iteration: 48599/375342 consumed_samples: 49766400 total_loss: 4.092 time: 0.3398 s/iter data_time: 0.2232 s/iter total_throughput: 3013.43 samples/s lr: 9.60e-04 [09/22 07:55:51] lb.utils.events INFO: eta: 14:22:26 iteration: 48699/375342 consumed_samples: 49868800 total_loss: 4.082 time: 0.3398 s/iter data_time: 0.2015 s/iter total_throughput: 3013.52 samples/s lr: 9.59e-04 [09/22 07:56:25] lb.utils.events INFO: eta: 14:20:01 iteration: 48799/375342 consumed_samples: 49971200 total_loss: 4.074 time: 0.3398 s/iter data_time: 0.2260 s/iter total_throughput: 3013.44 samples/s lr: 9.59e-04 [09/22 07:57:00] lb.utils.events INFO: eta: 14:21:40 iteration: 48899/375342 consumed_samples: 50073600 total_loss: 4.041 time: 0.3398 s/iter data_time: 0.2268 s/iter total_throughput: 3013.29 samples/s lr: 9.59e-04 [09/22 07:57:34] lb.utils.events INFO: eta: 14:20:21 iteration: 48999/375342 consumed_samples: 50176000 total_loss: 4.039 time: 0.3398 s/iter data_time: 0.2246 s/iter total_throughput: 3013.20 samples/s lr: 9.59e-04 [09/22 07:58:09] lb.utils.events INFO: eta: 14:19:13 iteration: 49099/375342 consumed_samples: 50278400 total_loss: 4.064 time: 0.3398 s/iter data_time: 0.2087 s/iter total_throughput: 3013.12 samples/s lr: 9.59e-04 [09/22 07:58:43] lb.utils.events INFO: eta: 14:17:21 iteration: 49199/375342 consumed_samples: 50380800 total_loss: 4.072 time: 0.3398 s/iter data_time: 0.2206 s/iter total_throughput: 3013.14 samples/s lr: 9.59e-04 [09/22 07:59:16] lb.utils.events INFO: eta: 14:15:49 iteration: 49299/375342 consumed_samples: 50483200 total_loss: 4.055 time: 0.3398 s/iter data_time: 0.2013 s/iter total_throughput: 3013.21 samples/s lr: 9.58e-04 [09/22 07:59:50] lb.utils.events INFO: eta: 14:16:14 iteration: 49399/375342 consumed_samples: 50585600 total_loss: 4.055 time: 0.3398 s/iter data_time: 0.2009 s/iter total_throughput: 3013.24 samples/s lr: 9.58e-04 [09/22 08:00:24] lb.utils.events INFO: eta: 14:15:16 iteration: 49499/375342 consumed_samples: 50688000 total_loss: 4.07 time: 0.3398 s/iter data_time: 0.2136 s/iter total_throughput: 3013.17 samples/s lr: 9.58e-04 [09/22 08:00:59] lb.utils.events INFO: eta: 14:13:49 iteration: 49599/375342 consumed_samples: 50790400 total_loss: 4.082 time: 0.3398 s/iter data_time: 0.2212 s/iter total_throughput: 3013.17 samples/s lr: 9.58e-04 [09/22 08:01:33] lb.utils.events INFO: eta: 14:14:28 iteration: 49699/375342 consumed_samples: 50892800 total_loss: 4.072 time: 0.3398 s/iter data_time: 0.2249 s/iter total_throughput: 3013.16 samples/s lr: 9.58e-04 [09/22 08:02:06] lb.utils.events INFO: eta: 14:14:12 iteration: 49799/375342 consumed_samples: 50995200 total_loss: 4.072 time: 0.3398 s/iter data_time: 0.2056 s/iter total_throughput: 3013.21 samples/s lr: 9.58e-04 [09/22 08:02:40] lb.utils.events INFO: eta: 14:15:03 iteration: 49899/375342 consumed_samples: 51097600 total_loss: 4.072 time: 0.3398 s/iter data_time: 0.2094 s/iter total_throughput: 3013.23 samples/s lr: 9.57e-04 [09/22 08:03:14] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0049999 [09/22 08:03:15] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/22 08:03:15] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/22 08:03:19] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0960 s/iter. Inference: 0.1589 s/iter. Eval: 0.0020 s/iter. Total: 0.2569 s/iter. ETA=0:00:09 [09/22 08:03:24] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1316 s/iter. Inference: 0.1677 s/iter. Eval: 0.0020 s/iter. Total: 0.3014 s/iter. ETA=0:00:05 [09/22 08:03:30] lb.evaluation.evaluator INFO: Inference done 50000/50000. Dataloading: 0.1130 s/iter. Inference: 0.1726 s/iter. Eval: 0.0020 s/iter. Total: 0.2876 s/iter. ETA=0:00:00 [09/22 08:03:30] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/22 08:03:30] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.660346 (0.000253 s / iter per device, on 8 devices) [09/22 08:03:30] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000152 s / iter per device, on 8 devices) [09/22 08:03:30] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/22 08:03:30] lb.evaluation.utils INFO: copypaste: Acc@1=66.72200000000001 [09/22 08:03:30] lb.evaluation.utils INFO: copypaste: Acc@5=87.968 [09/22 08:03:30] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 66.72200, better than last best score 65.84800 @ iteration 44999. [09/22 08:03:30] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/22 08:03:31] lb.utils.events INFO: eta: 14:14:23 iteration: 49999/375342 consumed_samples: 51200000 total_loss: 4.078 time: 0.3398 s/iter data_time: 0.2118 s/iter total_throughput: 3013.28 samples/s lr: 9.57e-04 [09/22 08:04:02] lb.utils.events INFO: eta: 14:17:47 iteration: 50099/375342 consumed_samples: 51302400 total_loss: 4.09 time: 0.3398 s/iter data_time: 0.2289 s/iter total_throughput: 3013.66 samples/s lr: 9.57e-04 [09/22 08:04:37] lb.utils.events INFO: eta: 14:23:36 iteration: 50199/375342 consumed_samples: 51404800 total_loss: 4.086 time: 0.3398 s/iter data_time: 0.2156 s/iter total_throughput: 3013.62 samples/s lr: 9.57e-04 [09/22 08:05:11] lb.utils.events INFO: eta: 14:22:40 iteration: 50299/375342 consumed_samples: 51507200 total_loss: 4.07 time: 0.3398 s/iter data_time: 0.2166 s/iter total_throughput: 3013.50 samples/s lr: 9.57e-04 [09/22 08:05:45] lb.utils.events INFO: eta: 14:20:06 iteration: 50399/375342 consumed_samples: 51609600 total_loss: 4.031 time: 0.3398 s/iter data_time: 0.2105 s/iter total_throughput: 3013.54 samples/s lr: 9.57e-04 [09/22 08:06:19] lb.utils.events INFO: eta: 14:20:06 iteration: 50499/375342 consumed_samples: 51712000 total_loss: 4.064 time: 0.3398 s/iter data_time: 0.2159 s/iter total_throughput: 3013.52 samples/s lr: 9.56e-04 [09/22 08:06:54] lb.utils.events INFO: eta: 14:19:34 iteration: 50599/375342 consumed_samples: 51814400 total_loss: 4.076 time: 0.3398 s/iter data_time: 0.2167 s/iter total_throughput: 3013.42 samples/s lr: 9.56e-04 [09/22 08:07:28] lb.utils.events INFO: eta: 14:16:55 iteration: 50699/375342 consumed_samples: 51916800 total_loss: 4.043 time: 0.3398 s/iter data_time: 0.2147 s/iter total_throughput: 3013.43 samples/s lr: 9.56e-04 [09/22 08:08:01] lb.utils.events INFO: eta: 14:16:13 iteration: 50799/375342 consumed_samples: 52019200 total_loss: 4.037 time: 0.3398 s/iter data_time: 0.2051 s/iter total_throughput: 3013.46 samples/s lr: 9.56e-04 [09/22 08:08:36] lb.utils.events INFO: eta: 14:13:26 iteration: 50899/375342 consumed_samples: 52121600 total_loss: 4.055 time: 0.3398 s/iter data_time: 0.1998 s/iter total_throughput: 3013.42 samples/s lr: 9.56e-04 [09/22 08:09:10] lb.utils.events INFO: eta: 14:12:14 iteration: 50999/375342 consumed_samples: 52224000 total_loss: 4.059 time: 0.3398 s/iter data_time: 0.2371 s/iter total_throughput: 3013.34 samples/s lr: 9.56e-04 [09/22 08:09:45] lb.utils.events INFO: eta: 14:08:01 iteration: 51099/375342 consumed_samples: 52326400 total_loss: 4.07 time: 0.3398 s/iter data_time: 0.2122 s/iter total_throughput: 3013.26 samples/s lr: 9.55e-04 [09/22 08:10:19] lb.utils.events INFO: eta: 14:04:03 iteration: 51199/375342 consumed_samples: 52428800 total_loss: 4.074 time: 0.3398 s/iter data_time: 0.2091 s/iter total_throughput: 3013.24 samples/s lr: 9.55e-04 [09/22 08:10:52] lb.utils.events INFO: eta: 14:06:07 iteration: 51299/375342 consumed_samples: 52531200 total_loss: 4.068 time: 0.3398 s/iter data_time: 0.2053 s/iter total_throughput: 3013.34 samples/s lr: 9.55e-04 [09/22 08:11:26] lb.utils.events INFO: eta: 14:07:14 iteration: 51399/375342 consumed_samples: 52633600 total_loss: 4.062 time: 0.3398 s/iter data_time: 0.2094 s/iter total_throughput: 3013.44 samples/s lr: 9.55e-04 [09/22 08:12:00] lb.utils.events INFO: eta: 14:07:00 iteration: 51499/375342 consumed_samples: 52736000 total_loss: 4.057 time: 0.3398 s/iter data_time: 0.2155 s/iter total_throughput: 3013.40 samples/s lr: 9.55e-04 [09/22 08:12:34] lb.utils.events INFO: eta: 14:06:46 iteration: 51599/375342 consumed_samples: 52838400 total_loss: 4.039 time: 0.3398 s/iter data_time: 0.2124 s/iter total_throughput: 3013.40 samples/s lr: 9.55e-04 [09/22 08:13:07] lb.utils.events INFO: eta: 14:07:56 iteration: 51699/375342 consumed_samples: 52940800 total_loss: 4.047 time: 0.3398 s/iter data_time: 0.2066 s/iter total_throughput: 3013.52 samples/s lr: 9.54e-04 [09/22 08:13:41] lb.utils.events INFO: eta: 14:06:13 iteration: 51799/375342 consumed_samples: 53043200 total_loss: 4.066 time: 0.3398 s/iter data_time: 0.2091 s/iter total_throughput: 3013.49 samples/s lr: 9.54e-04 [09/22 08:14:15] lb.utils.events INFO: eta: 14:08:50 iteration: 51899/375342 consumed_samples: 53145600 total_loss: 4.049 time: 0.3398 s/iter data_time: 0.2063 s/iter total_throughput: 3013.51 samples/s lr: 9.54e-04 [09/22 08:14:49] lb.utils.events INFO: eta: 14:09:45 iteration: 51999/375342 consumed_samples: 53248000 total_loss: 4.039 time: 0.3398 s/iter data_time: 0.2290 s/iter total_throughput: 3013.51 samples/s lr: 9.54e-04 [09/22 08:15:23] lb.utils.events INFO: eta: 14:11:45 iteration: 52099/375342 consumed_samples: 53350400 total_loss: 4.051 time: 0.3398 s/iter data_time: 0.2240 s/iter total_throughput: 3013.54 samples/s lr: 9.54e-04 [09/22 08:15:57] lb.utils.events INFO: eta: 14:11:21 iteration: 52199/375342 consumed_samples: 53452800 total_loss: 4.066 time: 0.3398 s/iter data_time: 0.2100 s/iter total_throughput: 3013.48 samples/s lr: 9.54e-04 [09/22 08:16:31] lb.utils.events INFO: eta: 14:08:38 iteration: 52299/375342 consumed_samples: 53555200 total_loss: 4.084 time: 0.3398 s/iter data_time: 0.2085 s/iter total_throughput: 3013.49 samples/s lr: 9.53e-04 [09/22 08:17:05] lb.utils.events INFO: eta: 14:07:22 iteration: 52399/375342 consumed_samples: 53657600 total_loss: 4.074 time: 0.3398 s/iter data_time: 0.2086 s/iter total_throughput: 3013.53 samples/s lr: 9.53e-04 [09/22 08:17:39] lb.utils.events INFO: eta: 14:06:14 iteration: 52499/375342 consumed_samples: 53760000 total_loss: 4.068 time: 0.3398 s/iter data_time: 0.2046 s/iter total_throughput: 3013.49 samples/s lr: 9.53e-04 [09/22 08:18:13] lb.utils.events INFO: eta: 14:06:00 iteration: 52599/375342 consumed_samples: 53862400 total_loss: 4.049 time: 0.3398 s/iter data_time: 0.2183 s/iter total_throughput: 3013.48 samples/s lr: 9.53e-04 [09/22 08:18:47] lb.utils.events INFO: eta: 14:04:09 iteration: 52699/375342 consumed_samples: 53964800 total_loss: 4.035 time: 0.3398 s/iter data_time: 0.2213 s/iter total_throughput: 3013.48 samples/s lr: 9.53e-04 [09/22 08:19:21] lb.utils.events INFO: eta: 14:04:49 iteration: 52799/375342 consumed_samples: 54067200 total_loss: 4.047 time: 0.3398 s/iter data_time: 0.2168 s/iter total_throughput: 3013.54 samples/s lr: 9.52e-04 [09/22 08:19:55] lb.utils.events INFO: eta: 14:03:11 iteration: 52899/375342 consumed_samples: 54169600 total_loss: 4.047 time: 0.3398 s/iter data_time: 0.2244 s/iter total_throughput: 3013.56 samples/s lr: 9.52e-04 [09/22 08:20:29] lb.utils.events INFO: eta: 14:01:34 iteration: 52999/375342 consumed_samples: 54272000 total_loss: 4.062 time: 0.3398 s/iter data_time: 0.2103 s/iter total_throughput: 3013.56 samples/s lr: 9.52e-04 [09/22 08:21:03] lb.utils.events INFO: eta: 14:00:06 iteration: 53099/375342 consumed_samples: 54374400 total_loss: 4.066 time: 0.3398 s/iter data_time: 0.2119 s/iter total_throughput: 3013.50 samples/s lr: 9.52e-04 [09/22 08:21:37] lb.utils.events INFO: eta: 13:58:02 iteration: 53199/375342 consumed_samples: 54476800 total_loss: 4.035 time: 0.3398 s/iter data_time: 0.2236 s/iter total_throughput: 3013.51 samples/s lr: 9.52e-04 [09/22 08:22:11] lb.utils.events INFO: eta: 14:00:18 iteration: 53299/375342 consumed_samples: 54579200 total_loss: 4.023 time: 0.3398 s/iter data_time: 0.2043 s/iter total_throughput: 3013.52 samples/s lr: 9.52e-04 [09/22 08:22:45] lb.utils.events INFO: eta: 13:59:44 iteration: 53399/375342 consumed_samples: 54681600 total_loss: 4.037 time: 0.3398 s/iter data_time: 0.2033 s/iter total_throughput: 3013.55 samples/s lr: 9.51e-04 [09/22 08:23:19] lb.utils.events INFO: eta: 13:59:28 iteration: 53499/375342 consumed_samples: 54784000 total_loss: 4.057 time: 0.3398 s/iter data_time: 0.2194 s/iter total_throughput: 3013.46 samples/s lr: 9.51e-04 [09/22 08:23:53] lb.utils.events INFO: eta: 13:59:12 iteration: 53599/375342 consumed_samples: 54886400 total_loss: 4.047 time: 0.3398 s/iter data_time: 0.2229 s/iter total_throughput: 3013.46 samples/s lr: 9.51e-04 [09/22 08:24:27] lb.utils.events INFO: eta: 13:59:15 iteration: 53699/375342 consumed_samples: 54988800 total_loss: 4.037 time: 0.3398 s/iter data_time: 0.2024 s/iter total_throughput: 3013.56 samples/s lr: 9.51e-04 [09/22 08:25:00] lb.utils.events INFO: eta: 13:59:31 iteration: 53799/375342 consumed_samples: 55091200 total_loss: 4.045 time: 0.3398 s/iter data_time: 0.2012 s/iter total_throughput: 3013.62 samples/s lr: 9.51e-04 [09/22 08:25:35] lb.utils.events INFO: eta: 13:59:16 iteration: 53899/375342 consumed_samples: 55193600 total_loss: 4.039 time: 0.3398 s/iter data_time: 0.2301 s/iter total_throughput: 3013.57 samples/s lr: 9.50e-04 [09/22 08:26:08] lb.utils.events INFO: eta: 13:59:21 iteration: 53999/375342 consumed_samples: 55296000 total_loss: 4.025 time: 0.3398 s/iter data_time: 0.2092 s/iter total_throughput: 3013.60 samples/s lr: 9.50e-04 [09/22 08:26:43] lb.utils.events INFO: eta: 13:58:42 iteration: 54099/375342 consumed_samples: 55398400 total_loss: 4.012 time: 0.3398 s/iter data_time: 0.2377 s/iter total_throughput: 3013.51 samples/s lr: 9.50e-04 [09/22 08:27:17] lb.utils.events INFO: eta: 14:00:17 iteration: 54199/375342 consumed_samples: 55500800 total_loss: 4.02 time: 0.3398 s/iter data_time: 0.2181 s/iter total_throughput: 3013.53 samples/s lr: 9.50e-04 [09/22 08:27:51] lb.utils.events INFO: eta: 13:58:34 iteration: 54299/375342 consumed_samples: 55603200 total_loss: 4.039 time: 0.3398 s/iter data_time: 0.2221 s/iter total_throughput: 3013.55 samples/s lr: 9.50e-04 [09/22 08:28:24] lb.utils.events INFO: eta: 13:58:47 iteration: 54399/375342 consumed_samples: 55705600 total_loss: 4.027 time: 0.3398 s/iter data_time: 0.2199 s/iter total_throughput: 3013.61 samples/s lr: 9.50e-04 [09/22 08:28:58] lb.utils.events INFO: eta: 13:58:20 iteration: 54499/375342 consumed_samples: 55808000 total_loss: 4.006 time: 0.3398 s/iter data_time: 0.2070 s/iter total_throughput: 3013.61 samples/s lr: 9.49e-04 [09/22 08:29:32] lb.utils.events INFO: eta: 13:57:17 iteration: 54599/375342 consumed_samples: 55910400 total_loss: 4.018 time: 0.3398 s/iter data_time: 0.2105 s/iter total_throughput: 3013.57 samples/s lr: 9.49e-04 [09/22 08:30:06] lb.utils.events INFO: eta: 13:58:22 iteration: 54699/375342 consumed_samples: 56012800 total_loss: 4.02 time: 0.3398 s/iter data_time: 0.2046 s/iter total_throughput: 3013.67 samples/s lr: 9.49e-04 [09/22 08:30:40] lb.utils.events INFO: eta: 13:56:25 iteration: 54799/375342 consumed_samples: 56115200 total_loss: 4.033 time: 0.3398 s/iter data_time: 0.2145 s/iter total_throughput: 3013.64 samples/s lr: 9.49e-04 [09/22 08:31:14] lb.utils.events INFO: eta: 13:56:38 iteration: 54899/375342 consumed_samples: 56217600 total_loss: 4.055 time: 0.3398 s/iter data_time: 0.2318 s/iter total_throughput: 3013.67 samples/s lr: 9.49e-04 [09/22 08:31:48] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0054999 [09/22 08:31:48] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/22 08:31:48] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/22 08:31:53] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0949 s/iter. Inference: 0.1676 s/iter. Eval: 0.0022 s/iter. Total: 0.2646 s/iter. ETA=0:00:09 [09/22 08:31:58] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1190 s/iter. Inference: 0.1755 s/iter. Eval: 0.0021 s/iter. Total: 0.2967 s/iter. ETA=0:00:05 [09/22 08:32:03] lb.evaluation.evaluator INFO: Inference done 49152/50000. Dataloading: 0.0972 s/iter. Inference: 0.1828 s/iter. Eval: 0.0020 s/iter. Total: 0.2821 s/iter. ETA=0:00:00 [09/22 08:32:03] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/22 08:32:03] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.568164 (0.000251 s / iter per device, on 8 devices) [09/22 08:32:03] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:08 (0.000161 s / iter per device, on 8 devices) [09/22 08:32:03] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/22 08:32:03] lb.evaluation.utils INFO: copypaste: Acc@1=67.42399999999999 [09/22 08:32:03] lb.evaluation.utils INFO: copypaste: Acc@5=88.188 [09/22 08:32:03] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 67.42400, better than last best score 66.72200 @ iteration 49999. [09/22 08:32:04] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/22 08:32:04] lb.utils.events INFO: eta: 13:55:30 iteration: 54999/375342 consumed_samples: 56320000 total_loss: 4.053 time: 0.3398 s/iter data_time: 0.2081 s/iter total_throughput: 3013.67 samples/s lr: 9.48e-04 [09/22 08:32:36] lb.utils.events INFO: eta: 13:58:40 iteration: 55099/375342 consumed_samples: 56422400 total_loss: 4.023 time: 0.3397 s/iter data_time: 0.2138 s/iter total_throughput: 3014.03 samples/s lr: 9.48e-04 [09/22 08:33:09] lb.utils.events INFO: eta: 13:58:40 iteration: 55199/375342 consumed_samples: 56524800 total_loss: 4.02 time: 0.3397 s/iter data_time: 0.1977 s/iter total_throughput: 3014.13 samples/s lr: 9.48e-04 [09/22 08:33:43] lb.utils.events INFO: eta: 13:58:31 iteration: 55299/375342 consumed_samples: 56627200 total_loss: 4.016 time: 0.3397 s/iter data_time: 0.2131 s/iter total_throughput: 3014.14 samples/s lr: 9.48e-04 [09/22 08:34:17] lb.utils.events INFO: eta: 13:57:04 iteration: 55399/375342 consumed_samples: 56729600 total_loss: 4.016 time: 0.3397 s/iter data_time: 0.2090 s/iter total_throughput: 3014.14 samples/s lr: 9.48e-04 [09/22 08:34:51] lb.utils.events INFO: eta: 13:57:26 iteration: 55499/375342 consumed_samples: 56832000 total_loss: 4.031 time: 0.3397 s/iter data_time: 0.2000 s/iter total_throughput: 3014.25 samples/s lr: 9.48e-04 [09/22 08:35:24] lb.utils.events INFO: eta: 13:57:33 iteration: 55599/375342 consumed_samples: 56934400 total_loss: 4.049 time: 0.3397 s/iter data_time: 0.2147 s/iter total_throughput: 3014.34 samples/s lr: 9.47e-04 [09/22 08:35:57] lb.utils.events INFO: eta: 13:57:12 iteration: 55699/375342 consumed_samples: 57036800 total_loss: 4.02 time: 0.3397 s/iter data_time: 0.2026 s/iter total_throughput: 3014.43 samples/s lr: 9.47e-04 [09/22 08:36:31] lb.utils.events INFO: eta: 13:58:46 iteration: 55799/375342 consumed_samples: 57139200 total_loss: 4.001 time: 0.3397 s/iter data_time: 0.2020 s/iter total_throughput: 3014.54 samples/s lr: 9.47e-04 [09/22 08:37:04] lb.utils.events INFO: eta: 14:00:31 iteration: 55899/375342 consumed_samples: 57241600 total_loss: 4.016 time: 0.3397 s/iter data_time: 0.2048 s/iter total_throughput: 3014.63 samples/s lr: 9.47e-04 [09/22 08:37:38] lb.utils.events INFO: eta: 14:01:25 iteration: 55999/375342 consumed_samples: 57344000 total_loss: 4.02 time: 0.3397 s/iter data_time: 0.2176 s/iter total_throughput: 3014.68 samples/s lr: 9.47e-04 [09/22 08:38:12] lb.utils.events INFO: eta: 13:59:16 iteration: 56099/375342 consumed_samples: 57446400 total_loss: 4.035 time: 0.3397 s/iter data_time: 0.2114 s/iter total_throughput: 3014.66 samples/s lr: 9.46e-04 [09/22 08:38:46] lb.utils.events INFO: eta: 13:58:09 iteration: 56199/375342 consumed_samples: 57548800 total_loss: 4.059 time: 0.3397 s/iter data_time: 0.2141 s/iter total_throughput: 3014.60 samples/s lr: 9.46e-04 [09/22 08:39:20] lb.utils.events INFO: eta: 13:59:28 iteration: 56299/375342 consumed_samples: 57651200 total_loss: 4.041 time: 0.3397 s/iter data_time: 0.2071 s/iter total_throughput: 3014.63 samples/s lr: 9.46e-04 [09/22 08:39:54] lb.utils.events INFO: eta: 14:02:07 iteration: 56399/375342 consumed_samples: 57753600 total_loss: 4.004 time: 0.3397 s/iter data_time: 0.2122 s/iter total_throughput: 3014.61 samples/s lr: 9.46e-04 [09/22 08:40:28] lb.utils.events INFO: eta: 14:02:23 iteration: 56499/375342 consumed_samples: 57856000 total_loss: 3.999 time: 0.3397 s/iter data_time: 0.2242 s/iter total_throughput: 3014.56 samples/s lr: 9.46e-04 [09/22 08:41:03] lb.utils.events INFO: eta: 14:02:27 iteration: 56599/375342 consumed_samples: 57958400 total_loss: 4.012 time: 0.3397 s/iter data_time: 0.2213 s/iter total_throughput: 3014.46 samples/s lr: 9.45e-04 [09/22 08:41:38] lb.utils.events INFO: eta: 14:01:19 iteration: 56699/375342 consumed_samples: 58060800 total_loss: 4.027 time: 0.3397 s/iter data_time: 0.2324 s/iter total_throughput: 3014.36 samples/s lr: 9.45e-04 [09/22 08:42:11] lb.utils.events INFO: eta: 14:00:56 iteration: 56799/375342 consumed_samples: 58163200 total_loss: 4.01 time: 0.3397 s/iter data_time: 0.2082 s/iter total_throughput: 3014.39 samples/s lr: 9.45e-04 [09/22 08:42:45] lb.utils.events INFO: eta: 14:00:47 iteration: 56899/375342 consumed_samples: 58265600 total_loss: 4.008 time: 0.3397 s/iter data_time: 0.2171 s/iter total_throughput: 3014.41 samples/s lr: 9.45e-04 [09/22 08:43:19] lb.utils.events INFO: eta: 14:00:31 iteration: 56999/375342 consumed_samples: 58368000 total_loss: 4.023 time: 0.3397 s/iter data_time: 0.2123 s/iter total_throughput: 3014.44 samples/s lr: 9.45e-04 [09/22 08:43:53] lb.utils.events INFO: eta: 13:59:31 iteration: 57099/375342 consumed_samples: 58470400 total_loss: 4.029 time: 0.3397 s/iter data_time: 0.2111 s/iter total_throughput: 3014.47 samples/s lr: 9.45e-04 [09/22 08:44:27] lb.utils.events INFO: eta: 13:59:15 iteration: 57199/375342 consumed_samples: 58572800 total_loss: 4.025 time: 0.3397 s/iter data_time: 0.1960 s/iter total_throughput: 3014.48 samples/s lr: 9.44e-04 [09/22 08:45:01] lb.utils.events INFO: eta: 13:57:56 iteration: 57299/375342 consumed_samples: 58675200 total_loss: 4.012 time: 0.3397 s/iter data_time: 0.2239 s/iter total_throughput: 3014.48 samples/s lr: 9.44e-04 [09/22 08:45:36] lb.utils.events INFO: eta: 13:55:34 iteration: 57399/375342 consumed_samples: 58777600 total_loss: 4.01 time: 0.3397 s/iter data_time: 0.2190 s/iter total_throughput: 3014.35 samples/s lr: 9.44e-04 [09/22 08:46:10] lb.utils.events INFO: eta: 13:56:42 iteration: 57499/375342 consumed_samples: 58880000 total_loss: 4.02 time: 0.3397 s/iter data_time: 0.2094 s/iter total_throughput: 3014.31 samples/s lr: 9.44e-04 [09/22 08:46:44] lb.utils.events INFO: eta: 13:54:59 iteration: 57599/375342 consumed_samples: 58982400 total_loss: 4.025 time: 0.3397 s/iter data_time: 0.2254 s/iter total_throughput: 3014.24 samples/s lr: 9.44e-04 [09/22 08:47:18] lb.utils.events INFO: eta: 13:54:37 iteration: 57699/375342 consumed_samples: 59084800 total_loss: 4.016 time: 0.3397 s/iter data_time: 0.2258 s/iter total_throughput: 3014.25 samples/s lr: 9.43e-04 [09/22 08:47:52] lb.utils.events INFO: eta: 13:56:07 iteration: 57799/375342 consumed_samples: 59187200 total_loss: 4.014 time: 0.3397 s/iter data_time: 0.2186 s/iter total_throughput: 3014.30 samples/s lr: 9.43e-04 [09/22 08:48:25] lb.utils.events INFO: eta: 13:56:19 iteration: 57899/375342 consumed_samples: 59289600 total_loss: 4.037 time: 0.3397 s/iter data_time: 0.2070 s/iter total_throughput: 3014.40 samples/s lr: 9.43e-04 [09/22 08:48:59] lb.utils.events INFO: eta: 13:55:50 iteration: 57999/375342 consumed_samples: 59392000 total_loss: 4.027 time: 0.3397 s/iter data_time: 0.2033 s/iter total_throughput: 3014.44 samples/s lr: 9.43e-04 [09/22 08:49:33] lb.utils.events INFO: eta: 13:55:48 iteration: 58099/375342 consumed_samples: 59494400 total_loss: 4.004 time: 0.3397 s/iter data_time: 0.2267 s/iter total_throughput: 3014.44 samples/s lr: 9.43e-04 [09/22 08:50:07] lb.utils.events INFO: eta: 13:55:01 iteration: 58199/375342 consumed_samples: 59596800 total_loss: 3.999 time: 0.3397 s/iter data_time: 0.2147 s/iter total_throughput: 3014.46 samples/s lr: 9.42e-04 [09/22 08:50:40] lb.utils.events INFO: eta: 13:55:02 iteration: 58299/375342 consumed_samples: 59699200 total_loss: 4.004 time: 0.3397 s/iter data_time: 0.2063 s/iter total_throughput: 3014.53 samples/s lr: 9.42e-04 [09/22 08:51:14] lb.utils.events INFO: eta: 13:57:06 iteration: 58399/375342 consumed_samples: 59801600 total_loss: 4 time: 0.3397 s/iter data_time: 0.2129 s/iter total_throughput: 3014.54 samples/s lr: 9.42e-04 [09/22 08:51:48] lb.utils.events INFO: eta: 13:55:21 iteration: 58499/375342 consumed_samples: 59904000 total_loss: 3.986 time: 0.3397 s/iter data_time: 0.2086 s/iter total_throughput: 3014.51 samples/s lr: 9.42e-04 [09/22 08:52:23] lb.utils.events INFO: eta: 13:56:40 iteration: 58599/375342 consumed_samples: 60006400 total_loss: 3.981 time: 0.3397 s/iter data_time: 0.2101 s/iter total_throughput: 3014.48 samples/s lr: 9.42e-04 [09/22 08:52:57] lb.utils.events INFO: eta: 13:55:56 iteration: 58699/375342 consumed_samples: 60108800 total_loss: 3.993 time: 0.3397 s/iter data_time: 0.2199 s/iter total_throughput: 3014.45 samples/s lr: 9.41e-04 [09/22 08:53:31] lb.utils.events INFO: eta: 13:52:42 iteration: 58799/375342 consumed_samples: 60211200 total_loss: 4.016 time: 0.3397 s/iter data_time: 0.2142 s/iter total_throughput: 3014.45 samples/s lr: 9.41e-04 [09/22 08:54:04] lb.utils.events INFO: eta: 13:51:44 iteration: 58899/375342 consumed_samples: 60313600 total_loss: 3.997 time: 0.3397 s/iter data_time: 0.2036 s/iter total_throughput: 3014.49 samples/s lr: 9.41e-04 [09/22 08:54:38] lb.utils.events INFO: eta: 13:52:10 iteration: 58999/375342 consumed_samples: 60416000 total_loss: 3.982 time: 0.3397 s/iter data_time: 0.2120 s/iter total_throughput: 3014.53 samples/s lr: 9.41e-04 [09/22 08:55:12] lb.utils.events INFO: eta: 13:50:26 iteration: 59099/375342 consumed_samples: 60518400 total_loss: 3.983 time: 0.3397 s/iter data_time: 0.2246 s/iter total_throughput: 3014.51 samples/s lr: 9.41e-04 [09/22 08:55:47] lb.utils.events INFO: eta: 13:51:24 iteration: 59199/375342 consumed_samples: 60620800 total_loss: 3.991 time: 0.3397 s/iter data_time: 0.2331 s/iter total_throughput: 3014.43 samples/s lr: 9.40e-04 [09/22 08:56:21] lb.utils.events INFO: eta: 13:49:57 iteration: 59299/375342 consumed_samples: 60723200 total_loss: 3.985 time: 0.3397 s/iter data_time: 0.2084 s/iter total_throughput: 3014.44 samples/s lr: 9.40e-04 [09/22 08:56:55] lb.utils.events INFO: eta: 13:49:29 iteration: 59399/375342 consumed_samples: 60825600 total_loss: 3.984 time: 0.3397 s/iter data_time: 0.2206 s/iter total_throughput: 3014.44 samples/s lr: 9.40e-04 [09/22 08:57:29] lb.utils.events INFO: eta: 13:49:25 iteration: 59499/375342 consumed_samples: 60928000 total_loss: 3.99 time: 0.3397 s/iter data_time: 0.2140 s/iter total_throughput: 3014.44 samples/s lr: 9.40e-04 [09/22 08:58:03] lb.utils.events INFO: eta: 13:48:19 iteration: 59599/375342 consumed_samples: 61030400 total_loss: 4 time: 0.3397 s/iter data_time: 0.2061 s/iter total_throughput: 3014.44 samples/s lr: 9.40e-04 [09/22 08:58:37] lb.utils.events INFO: eta: 13:46:39 iteration: 59699/375342 consumed_samples: 61132800 total_loss: 3.993 time: 0.3397 s/iter data_time: 0.2151 s/iter total_throughput: 3014.39 samples/s lr: 9.39e-04 [09/22 08:59:11] lb.utils.events INFO: eta: 13:48:51 iteration: 59799/375342 consumed_samples: 61235200 total_loss: 3.984 time: 0.3397 s/iter data_time: 0.2085 s/iter total_throughput: 3014.41 samples/s lr: 9.39e-04 [09/22 08:59:45] lb.utils.events INFO: eta: 13:48:40 iteration: 59899/375342 consumed_samples: 61337600 total_loss: 4.004 time: 0.3397 s/iter data_time: 0.2152 s/iter total_throughput: 3014.41 samples/s lr: 9.39e-04 [09/22 09:00:19] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0059999 [09/22 09:00:19] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/22 09:00:19] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/22 09:00:23] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0824 s/iter. Inference: 0.1728 s/iter. Eval: 0.0021 s/iter. Total: 0.2573 s/iter. ETA=0:00:09 [09/22 09:00:28] lb.evaluation.evaluator INFO: Inference done 28672/50000. Dataloading: 0.0797 s/iter. Inference: 0.2049 s/iter. Eval: 0.0019 s/iter. Total: 0.2866 s/iter. ETA=0:00:05 [09/22 09:00:34] lb.evaluation.evaluator INFO: Inference done 46080/50000. Dataloading: 0.0752 s/iter. Inference: 0.2140 s/iter. Eval: 0.0020 s/iter. Total: 0.2912 s/iter. ETA=0:00:00 [09/22 09:00:34] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/22 09:00:34] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.597567 (0.000252 s / iter per device, on 8 devices) [09/22 09:00:34] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:09 (0.000190 s / iter per device, on 8 devices) [09/22 09:00:34] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/22 09:00:34] lb.evaluation.utils INFO: copypaste: Acc@1=68.438 [09/22 09:00:34] lb.evaluation.utils INFO: copypaste: Acc@5=88.804 [09/22 09:00:34] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 68.43800, better than last best score 67.42400 @ iteration 54999. [09/22 09:00:34] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/22 09:00:35] lb.utils.events INFO: eta: 13:48:41 iteration: 59999/375342 consumed_samples: 61440000 total_loss: 4.004 time: 0.3397 s/iter data_time: 0.2129 s/iter total_throughput: 3014.40 samples/s lr: 9.39e-04 [09/22 09:01:07] lb.utils.events INFO: eta: 13:49:24 iteration: 60099/375342 consumed_samples: 61542400 total_loss: 4.014 time: 0.3397 s/iter data_time: 0.2051 s/iter total_throughput: 3014.75 samples/s lr: 9.39e-04 [09/22 09:01:40] lb.utils.events INFO: eta: 13:49:31 iteration: 60199/375342 consumed_samples: 61644800 total_loss: 3.999 time: 0.3397 s/iter data_time: 0.2233 s/iter total_throughput: 3014.82 samples/s lr: 9.38e-04 [09/22 09:02:14] lb.utils.events INFO: eta: 13:52:02 iteration: 60299/375342 consumed_samples: 61747200 total_loss: 3.991 time: 0.3397 s/iter data_time: 0.2105 s/iter total_throughput: 3014.82 samples/s lr: 9.38e-04 [09/22 09:02:48] lb.utils.events INFO: eta: 13:54:02 iteration: 60399/375342 consumed_samples: 61849600 total_loss: 3.993 time: 0.3397 s/iter data_time: 0.2131 s/iter total_throughput: 3014.81 samples/s lr: 9.38e-04 [09/22 09:03:22] lb.utils.events INFO: eta: 13:55:24 iteration: 60499/375342 consumed_samples: 61952000 total_loss: 3.995 time: 0.3397 s/iter data_time: 0.1997 s/iter total_throughput: 3014.81 samples/s lr: 9.38e-04 [09/22 09:03:56] lb.utils.events INFO: eta: 13:55:49 iteration: 60599/375342 consumed_samples: 62054400 total_loss: 3.997 time: 0.3397 s/iter data_time: 0.2080 s/iter total_throughput: 3014.81 samples/s lr: 9.38e-04 [09/22 09:04:30] lb.utils.events INFO: eta: 13:58:21 iteration: 60699/375342 consumed_samples: 62156800 total_loss: 4.008 time: 0.3397 s/iter data_time: 0.2019 s/iter total_throughput: 3014.81 samples/s lr: 9.37e-04 [09/22 09:05:05] lb.utils.events INFO: eta: 13:56:51 iteration: 60799/375342 consumed_samples: 62259200 total_loss: 4.016 time: 0.3397 s/iter data_time: 0.2103 s/iter total_throughput: 3014.77 samples/s lr: 9.37e-04 [09/22 09:05:38] lb.utils.events INFO: eta: 13:55:56 iteration: 60899/375342 consumed_samples: 62361600 total_loss: 3.995 time: 0.3397 s/iter data_time: 0.2128 s/iter total_throughput: 3014.78 samples/s lr: 9.37e-04 [09/22 09:06:12] lb.utils.events INFO: eta: 13:54:51 iteration: 60999/375342 consumed_samples: 62464000 total_loss: 3.994 time: 0.3397 s/iter data_time: 0.2143 s/iter total_throughput: 3014.81 samples/s lr: 9.37e-04 [09/22 09:06:46] lb.utils.events INFO: eta: 13:52:41 iteration: 61099/375342 consumed_samples: 62566400 total_loss: 3.99 time: 0.3397 s/iter data_time: 0.2125 s/iter total_throughput: 3014.85 samples/s lr: 9.37e-04 [09/22 09:07:20] lb.utils.events INFO: eta: 13:49:15 iteration: 61199/375342 consumed_samples: 62668800 total_loss: 3.976 time: 0.3397 s/iter data_time: 0.2264 s/iter total_throughput: 3014.77 samples/s lr: 9.36e-04 [09/22 09:07:54] lb.utils.events INFO: eta: 13:47:37 iteration: 61299/375342 consumed_samples: 62771200 total_loss: 3.976 time: 0.3397 s/iter data_time: 0.2110 s/iter total_throughput: 3014.81 samples/s lr: 9.36e-04 [09/22 09:08:28] lb.utils.events INFO: eta: 13:44:59 iteration: 61399/375342 consumed_samples: 62873600 total_loss: 3.963 time: 0.3397 s/iter data_time: 0.2102 s/iter total_throughput: 3014.76 samples/s lr: 9.36e-04 [09/22 09:09:02] lb.utils.events INFO: eta: 13:44:39 iteration: 61499/375342 consumed_samples: 62976000 total_loss: 3.976 time: 0.3397 s/iter data_time: 0.2191 s/iter total_throughput: 3014.77 samples/s lr: 9.36e-04 [09/22 09:09:37] lb.utils.events INFO: eta: 13:44:14 iteration: 61599/375342 consumed_samples: 63078400 total_loss: 3.994 time: 0.3397 s/iter data_time: 0.2135 s/iter total_throughput: 3014.74 samples/s lr: 9.36e-04 [09/22 09:10:11] lb.utils.events INFO: eta: 13:43:44 iteration: 61699/375342 consumed_samples: 63180800 total_loss: 3.979 time: 0.3397 s/iter data_time: 0.2159 s/iter total_throughput: 3014.72 samples/s lr: 9.35e-04 [09/22 09:10:45] lb.utils.events INFO: eta: 13:44:32 iteration: 61799/375342 consumed_samples: 63283200 total_loss: 3.975 time: 0.3397 s/iter data_time: 0.2224 s/iter total_throughput: 3014.72 samples/s lr: 9.35e-04 [09/22 09:11:18] lb.utils.events INFO: eta: 13:44:02 iteration: 61899/375342 consumed_samples: 63385600 total_loss: 3.978 time: 0.3397 s/iter data_time: 0.2067 s/iter total_throughput: 3014.78 samples/s lr: 9.35e-04 [09/22 09:11:52] lb.utils.events INFO: eta: 13:46:24 iteration: 61999/375342 consumed_samples: 63488000 total_loss: 3.984 time: 0.3397 s/iter data_time: 0.2088 s/iter total_throughput: 3014.84 samples/s lr: 9.35e-04 [09/22 09:12:26] lb.utils.events INFO: eta: 13:45:53 iteration: 62099/375342 consumed_samples: 63590400 total_loss: 3.99 time: 0.3397 s/iter data_time: 0.2397 s/iter total_throughput: 3014.82 samples/s lr: 9.35e-04 [09/22 09:13:00] lb.utils.events INFO: eta: 13:45:23 iteration: 62199/375342 consumed_samples: 63692800 total_loss: 3.973 time: 0.3397 s/iter data_time: 0.2137 s/iter total_throughput: 3014.82 samples/s lr: 9.34e-04 [09/22 09:13:34] lb.utils.events INFO: eta: 13:44:48 iteration: 62299/375342 consumed_samples: 63795200 total_loss: 3.973 time: 0.3397 s/iter data_time: 0.2121 s/iter total_throughput: 3014.87 samples/s lr: 9.34e-04 [09/22 09:14:07] lb.utils.events INFO: eta: 13:44:42 iteration: 62399/375342 consumed_samples: 63897600 total_loss: 3.986 time: 0.3396 s/iter data_time: 0.2214 s/iter total_throughput: 3014.89 samples/s lr: 9.34e-04 [09/22 09:14:42] lb.utils.events INFO: eta: 13:43:10 iteration: 62499/375342 consumed_samples: 64000000 total_loss: 3.983 time: 0.3397 s/iter data_time: 0.2043 s/iter total_throughput: 3014.83 samples/s lr: 9.34e-04 [09/22 09:15:16] lb.utils.events INFO: eta: 13:42:54 iteration: 62599/375342 consumed_samples: 64102400 total_loss: 3.983 time: 0.3397 s/iter data_time: 0.2153 s/iter total_throughput: 3014.81 samples/s lr: 9.34e-04 [09/22 09:15:50] lb.utils.events INFO: eta: 13:42:09 iteration: 62699/375342 consumed_samples: 64204800 total_loss: 3.973 time: 0.3397 s/iter data_time: 0.2190 s/iter total_throughput: 3014.83 samples/s lr: 9.33e-04 [09/22 09:16:23] lb.utils.events INFO: eta: 13:41:40 iteration: 62799/375342 consumed_samples: 64307200 total_loss: 3.967 time: 0.3397 s/iter data_time: 0.2110 s/iter total_throughput: 3014.86 samples/s lr: 9.33e-04 [09/22 09:16:57] lb.utils.events INFO: eta: 13:42:48 iteration: 62899/375342 consumed_samples: 64409600 total_loss: 3.961 time: 0.3396 s/iter data_time: 0.2086 s/iter total_throughput: 3014.92 samples/s lr: 9.33e-04 [09/22 09:17:31] lb.utils.events INFO: eta: 13:40:58 iteration: 62999/375342 consumed_samples: 64512000 total_loss: 3.947 time: 0.3396 s/iter data_time: 0.2088 s/iter total_throughput: 3014.94 samples/s lr: 9.33e-04 [09/22 09:18:04] lb.utils.events INFO: eta: 13:41:17 iteration: 63099/375342 consumed_samples: 64614400 total_loss: 3.965 time: 0.3396 s/iter data_time: 0.2043 s/iter total_throughput: 3014.99 samples/s lr: 9.33e-04 [09/22 09:18:39] lb.utils.events INFO: eta: 13:41:50 iteration: 63199/375342 consumed_samples: 64716800 total_loss: 3.977 time: 0.3396 s/iter data_time: 0.2092 s/iter total_throughput: 3014.94 samples/s lr: 9.32e-04 [09/22 09:19:13] lb.utils.events INFO: eta: 13:40:35 iteration: 63299/375342 consumed_samples: 64819200 total_loss: 3.983 time: 0.3396 s/iter data_time: 0.2109 s/iter total_throughput: 3014.94 samples/s lr: 9.32e-04 [09/22 09:19:47] lb.utils.events INFO: eta: 13:40:29 iteration: 63399/375342 consumed_samples: 64921600 total_loss: 3.984 time: 0.3396 s/iter data_time: 0.2132 s/iter total_throughput: 3014.93 samples/s lr: 9.32e-04 [09/22 09:20:21] lb.utils.events INFO: eta: 13:40:14 iteration: 63499/375342 consumed_samples: 65024000 total_loss: 3.974 time: 0.3396 s/iter data_time: 0.2054 s/iter total_throughput: 3014.93 samples/s lr: 9.32e-04 [09/22 09:20:54] lb.utils.events INFO: eta: 13:41:29 iteration: 63599/375342 consumed_samples: 65126400 total_loss: 3.979 time: 0.3396 s/iter data_time: 0.2076 s/iter total_throughput: 3014.99 samples/s lr: 9.32e-04 [09/22 09:21:28] lb.utils.events INFO: eta: 13:41:13 iteration: 63699/375342 consumed_samples: 65228800 total_loss: 3.987 time: 0.3396 s/iter data_time: 0.2155 s/iter total_throughput: 3015.06 samples/s lr: 9.31e-04 [09/22 09:22:02] lb.utils.events INFO: eta: 13:40:21 iteration: 63799/375342 consumed_samples: 65331200 total_loss: 3.979 time: 0.3396 s/iter data_time: 0.2106 s/iter total_throughput: 3015.04 samples/s lr: 9.31e-04 [09/22 09:22:36] lb.utils.events INFO: eta: 13:40:04 iteration: 63899/375342 consumed_samples: 65433600 total_loss: 3.958 time: 0.3396 s/iter data_time: 0.2190 s/iter total_throughput: 3015.02 samples/s lr: 9.31e-04 [09/22 09:23:10] lb.utils.events INFO: eta: 13:40:45 iteration: 63999/375342 consumed_samples: 65536000 total_loss: 3.957 time: 0.3396 s/iter data_time: 0.2041 s/iter total_throughput: 3015.02 samples/s lr: 9.31e-04 [09/22 09:23:44] lb.utils.events INFO: eta: 13:39:14 iteration: 64099/375342 consumed_samples: 65638400 total_loss: 3.974 time: 0.3396 s/iter data_time: 0.2256 s/iter total_throughput: 3014.96 samples/s lr: 9.30e-04 [09/22 09:24:18] lb.utils.events INFO: eta: 13:41:04 iteration: 64199/375342 consumed_samples: 65740800 total_loss: 3.98 time: 0.3396 s/iter data_time: 0.2080 s/iter total_throughput: 3014.96 samples/s lr: 9.30e-04 [09/22 09:24:52] lb.utils.events INFO: eta: 13:41:26 iteration: 64299/375342 consumed_samples: 65843200 total_loss: 3.975 time: 0.3396 s/iter data_time: 0.2129 s/iter total_throughput: 3014.98 samples/s lr: 9.30e-04 [09/22 09:25:26] lb.utils.events INFO: eta: 13:40:42 iteration: 64399/375342 consumed_samples: 65945600 total_loss: 3.982 time: 0.3396 s/iter data_time: 0.2087 s/iter total_throughput: 3014.95 samples/s lr: 9.30e-04 [09/22 09:26:01] lb.utils.events INFO: eta: 13:41:07 iteration: 64499/375342 consumed_samples: 66048000 total_loss: 3.994 time: 0.3396 s/iter data_time: 0.2193 s/iter total_throughput: 3014.90 samples/s lr: 9.30e-04 [09/22 09:26:34] lb.utils.events INFO: eta: 13:40:36 iteration: 64599/375342 consumed_samples: 66150400 total_loss: 3.979 time: 0.3396 s/iter data_time: 0.2062 s/iter total_throughput: 3015.01 samples/s lr: 9.29e-04 [09/22 09:27:09] lb.utils.events INFO: eta: 13:39:01 iteration: 64699/375342 consumed_samples: 66252800 total_loss: 3.979 time: 0.3396 s/iter data_time: 0.2269 s/iter total_throughput: 3014.92 samples/s lr: 9.29e-04 [09/22 09:27:42] lb.utils.events INFO: eta: 13:39:35 iteration: 64799/375342 consumed_samples: 66355200 total_loss: 3.973 time: 0.3396 s/iter data_time: 0.2107 s/iter total_throughput: 3014.94 samples/s lr: 9.29e-04 [09/22 09:28:16] lb.utils.events INFO: eta: 13:39:19 iteration: 64899/375342 consumed_samples: 66457600 total_loss: 3.979 time: 0.3396 s/iter data_time: 0.2174 s/iter total_throughput: 3014.98 samples/s lr: 9.29e-04 [09/22 09:28:50] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0064999 [09/22 09:28:50] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/22 09:28:50] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/22 09:28:54] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0930 s/iter. Inference: 0.1624 s/iter. Eval: 0.0019 s/iter. Total: 0.2574 s/iter. ETA=0:00:09 [09/22 09:29:00] lb.evaluation.evaluator INFO: Inference done 28672/50000. Dataloading: 0.0829 s/iter. Inference: 0.2003 s/iter. Eval: 0.0019 s/iter. Total: 0.2853 s/iter. ETA=0:00:05 [09/22 09:29:05] lb.evaluation.evaluator INFO: Inference done 46080/50000. Dataloading: 0.0817 s/iter. Inference: 0.2074 s/iter. Eval: 0.0019 s/iter. Total: 0.2911 s/iter. ETA=0:00:00 [09/22 09:29:06] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/22 09:29:06] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.579963 (0.000252 s / iter per device, on 8 devices) [09/22 09:29:06] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:09 (0.000182 s / iter per device, on 8 devices) [09/22 09:29:06] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/22 09:29:06] lb.evaluation.utils INFO: copypaste: Acc@1=68.80199999999999 [09/22 09:29:06] lb.evaluation.utils INFO: copypaste: Acc@5=89.038 [09/22 09:29:06] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 68.80200, better than last best score 68.43800 @ iteration 59999. [09/22 09:29:06] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/22 09:29:06] lb.utils.events INFO: eta: 13:38:50 iteration: 64999/375342 consumed_samples: 66560000 total_loss: 3.986 time: 0.3396 s/iter data_time: 0.2114 s/iter total_throughput: 3015.01 samples/s lr: 9.29e-04 [09/22 09:29:38] lb.utils.events INFO: eta: 13:43:34 iteration: 65099/375342 consumed_samples: 66662400 total_loss: 3.961 time: 0.3396 s/iter data_time: 0.2089 s/iter total_throughput: 3015.33 samples/s lr: 9.28e-04 [09/22 09:30:11] lb.utils.events INFO: eta: 13:43:14 iteration: 65199/375342 consumed_samples: 66764800 total_loss: 3.956 time: 0.3396 s/iter data_time: 0.1992 s/iter total_throughput: 3015.41 samples/s lr: 9.28e-04 [09/22 09:30:45] lb.utils.events INFO: eta: 13:45:41 iteration: 65299/375342 consumed_samples: 66867200 total_loss: 3.967 time: 0.3396 s/iter data_time: 0.2318 s/iter total_throughput: 3015.40 samples/s lr: 9.28e-04 [09/22 09:31:19] lb.utils.events INFO: eta: 13:47:03 iteration: 65399/375342 consumed_samples: 66969600 total_loss: 3.944 time: 0.3396 s/iter data_time: 0.2044 s/iter total_throughput: 3015.47 samples/s lr: 9.28e-04 [09/22 09:31:53] lb.utils.events INFO: eta: 13:44:36 iteration: 65499/375342 consumed_samples: 67072000 total_loss: 3.946 time: 0.3396 s/iter data_time: 0.2242 s/iter total_throughput: 3015.45 samples/s lr: 9.27e-04 [09/22 09:32:27] lb.utils.events INFO: eta: 13:44:07 iteration: 65599/375342 consumed_samples: 67174400 total_loss: 3.958 time: 0.3396 s/iter data_time: 0.2105 s/iter total_throughput: 3015.45 samples/s lr: 9.27e-04 [09/22 09:33:00] lb.utils.events INFO: eta: 13:44:27 iteration: 65699/375342 consumed_samples: 67276800 total_loss: 3.949 time: 0.3396 s/iter data_time: 0.2027 s/iter total_throughput: 3015.51 samples/s lr: 9.27e-04 [09/22 09:33:34] lb.utils.events INFO: eta: 13:43:48 iteration: 65799/375342 consumed_samples: 67379200 total_loss: 3.974 time: 0.3396 s/iter data_time: 0.2011 s/iter total_throughput: 3015.52 samples/s lr: 9.27e-04 [09/22 09:34:08] lb.utils.events INFO: eta: 13:43:32 iteration: 65899/375342 consumed_samples: 67481600 total_loss: 3.981 time: 0.3396 s/iter data_time: 0.2057 s/iter total_throughput: 3015.58 samples/s lr: 9.27e-04 [09/22 09:34:41] lb.utils.events INFO: eta: 13:43:46 iteration: 65999/375342 consumed_samples: 67584000 total_loss: 3.943 time: 0.3396 s/iter data_time: 0.2025 s/iter total_throughput: 3015.66 samples/s lr: 9.26e-04 [09/22 09:35:16] lb.utils.events INFO: eta: 13:39:50 iteration: 66099/375342 consumed_samples: 67686400 total_loss: 3.944 time: 0.3396 s/iter data_time: 0.2016 s/iter total_throughput: 3015.62 samples/s lr: 9.26e-04 [09/22 09:35:50] lb.utils.events INFO: eta: 13:37:35 iteration: 66199/375342 consumed_samples: 67788800 total_loss: 3.98 time: 0.3396 s/iter data_time: 0.2150 s/iter total_throughput: 3015.53 samples/s lr: 9.26e-04 [09/22 09:36:24] lb.utils.events INFO: eta: 13:34:04 iteration: 66299/375342 consumed_samples: 67891200 total_loss: 3.979 time: 0.3396 s/iter data_time: 0.2081 s/iter total_throughput: 3015.53 samples/s lr: 9.26e-04 [09/22 09:36:58] lb.utils.events INFO: eta: 13:31:35 iteration: 66399/375342 consumed_samples: 67993600 total_loss: 3.93 time: 0.3396 s/iter data_time: 0.2265 s/iter total_throughput: 3015.54 samples/s lr: 9.26e-04 [09/22 09:37:32] lb.utils.events INFO: eta: 13:31:10 iteration: 66499/375342 consumed_samples: 68096000 total_loss: 3.944 time: 0.3396 s/iter data_time: 0.2170 s/iter total_throughput: 3015.53 samples/s lr: 9.25e-04 [09/22 09:38:06] lb.utils.events INFO: eta: 13:30:48 iteration: 66599/375342 consumed_samples: 68198400 total_loss: 3.955 time: 0.3396 s/iter data_time: 0.2093 s/iter total_throughput: 3015.59 samples/s lr: 9.25e-04 [09/22 09:38:39] lb.utils.events INFO: eta: 13:30:17 iteration: 66699/375342 consumed_samples: 68300800 total_loss: 3.946 time: 0.3396 s/iter data_time: 0.2037 s/iter total_throughput: 3015.60 samples/s lr: 9.25e-04 [09/22 09:39:14] lb.utils.events INFO: eta: 13:27:24 iteration: 66799/375342 consumed_samples: 68403200 total_loss: 3.946 time: 0.3396 s/iter data_time: 0.2103 s/iter total_throughput: 3015.54 samples/s lr: 9.25e-04 [09/22 09:39:48] lb.utils.events INFO: eta: 13:26:26 iteration: 66899/375342 consumed_samples: 68505600 total_loss: 3.954 time: 0.3396 s/iter data_time: 0.2111 s/iter total_throughput: 3015.55 samples/s lr: 9.24e-04 [09/22 09:40:21] lb.utils.events INFO: eta: 13:25:49 iteration: 66999/375342 consumed_samples: 68608000 total_loss: 3.956 time: 0.3396 s/iter data_time: 0.2160 s/iter total_throughput: 3015.62 samples/s lr: 9.24e-04 [09/22 09:40:55] lb.utils.events INFO: eta: 13:23:50 iteration: 67099/375342 consumed_samples: 68710400 total_loss: 3.939 time: 0.3396 s/iter data_time: 0.2199 s/iter total_throughput: 3015.62 samples/s lr: 9.24e-04 [09/22 09:41:29] lb.utils.events INFO: eta: 13:24:37 iteration: 67199/375342 consumed_samples: 68812800 total_loss: 3.959 time: 0.3396 s/iter data_time: 0.2115 s/iter total_throughput: 3015.58 samples/s lr: 9.24e-04 [09/22 09:42:03] lb.utils.events INFO: eta: 13:24:57 iteration: 67299/375342 consumed_samples: 68915200 total_loss: 3.948 time: 0.3396 s/iter data_time: 0.2000 s/iter total_throughput: 3015.63 samples/s lr: 9.24e-04 [09/22 09:42:37] lb.utils.events INFO: eta: 13:24:31 iteration: 67399/375342 consumed_samples: 69017600 total_loss: 3.938 time: 0.3396 s/iter data_time: 0.2151 s/iter total_throughput: 3015.68 samples/s lr: 9.23e-04 [09/22 09:43:10] lb.utils.events INFO: eta: 13:25:50 iteration: 67499/375342 consumed_samples: 69120000 total_loss: 3.952 time: 0.3395 s/iter data_time: 0.1987 s/iter total_throughput: 3015.81 samples/s lr: 9.23e-04 [09/22 09:43:43] lb.utils.events INFO: eta: 13:26:45 iteration: 67599/375342 consumed_samples: 69222400 total_loss: 3.952 time: 0.3395 s/iter data_time: 0.2051 s/iter total_throughput: 3015.82 samples/s lr: 9.23e-04 [09/22 09:44:17] lb.utils.events INFO: eta: 13:26:15 iteration: 67699/375342 consumed_samples: 69324800 total_loss: 3.939 time: 0.3395 s/iter data_time: 0.2058 s/iter total_throughput: 3015.90 samples/s lr: 9.23e-04 [09/22 09:44:51] lb.utils.events INFO: eta: 13:25:59 iteration: 67799/375342 consumed_samples: 69427200 total_loss: 3.944 time: 0.3395 s/iter data_time: 0.1990 s/iter total_throughput: 3015.86 samples/s lr: 9.22e-04 [09/22 09:45:24] lb.utils.events INFO: eta: 13:27:22 iteration: 67899/375342 consumed_samples: 69529600 total_loss: 3.963 time: 0.3395 s/iter data_time: 0.2055 s/iter total_throughput: 3015.95 samples/s lr: 9.22e-04 [09/22 09:45:58] lb.utils.events INFO: eta: 13:25:48 iteration: 67999/375342 consumed_samples: 69632000 total_loss: 3.977 time: 0.3395 s/iter data_time: 0.2110 s/iter total_throughput: 3015.94 samples/s lr: 9.22e-04 [09/22 09:46:32] lb.utils.events INFO: eta: 13:28:05 iteration: 68099/375342 consumed_samples: 69734400 total_loss: 3.958 time: 0.3395 s/iter data_time: 0.2204 s/iter total_throughput: 3015.96 samples/s lr: 9.22e-04 [09/22 09:47:06] lb.utils.events INFO: eta: 13:28:36 iteration: 68199/375342 consumed_samples: 69836800 total_loss: 3.952 time: 0.3395 s/iter data_time: 0.2177 s/iter total_throughput: 3015.97 samples/s lr: 9.22e-04 [09/22 09:47:40] lb.utils.events INFO: eta: 13:27:24 iteration: 68299/375342 consumed_samples: 69939200 total_loss: 3.958 time: 0.3395 s/iter data_time: 0.2017 s/iter total_throughput: 3016.01 samples/s lr: 9.21e-04 [09/22 09:48:14] lb.utils.events INFO: eta: 13:26:28 iteration: 68399/375342 consumed_samples: 70041600 total_loss: 3.971 time: 0.3395 s/iter data_time: 0.2196 s/iter total_throughput: 3016.03 samples/s lr: 9.21e-04 [09/22 09:48:48] lb.utils.events INFO: eta: 13:24:35 iteration: 68499/375342 consumed_samples: 70144000 total_loss: 3.968 time: 0.3395 s/iter data_time: 0.2118 s/iter total_throughput: 3016.01 samples/s lr: 9.21e-04 [09/22 09:49:21] lb.utils.events INFO: eta: 13:23:32 iteration: 68599/375342 consumed_samples: 70246400 total_loss: 3.953 time: 0.3395 s/iter data_time: 0.2005 s/iter total_throughput: 3016.08 samples/s lr: 9.21e-04 [09/22 09:49:54] lb.utils.events INFO: eta: 13:23:13 iteration: 68699/375342 consumed_samples: 70348800 total_loss: 3.938 time: 0.3395 s/iter data_time: 0.1928 s/iter total_throughput: 3016.18 samples/s lr: 9.20e-04 [09/22 09:50:27] lb.utils.events INFO: eta: 13:24:37 iteration: 68799/375342 consumed_samples: 70451200 total_loss: 3.946 time: 0.3395 s/iter data_time: 0.2025 s/iter total_throughput: 3016.30 samples/s lr: 9.20e-04 [09/22 09:51:01] lb.utils.events INFO: eta: 13:23:58 iteration: 68899/375342 consumed_samples: 70553600 total_loss: 3.952 time: 0.3395 s/iter data_time: 0.1969 s/iter total_throughput: 3016.35 samples/s lr: 9.20e-04 [09/22 09:51:35] lb.utils.events INFO: eta: 13:23:22 iteration: 68999/375342 consumed_samples: 70656000 total_loss: 3.96 time: 0.3395 s/iter data_time: 0.2087 s/iter total_throughput: 3016.35 samples/s lr: 9.20e-04 [09/22 09:52:08] lb.utils.events INFO: eta: 13:22:21 iteration: 69099/375342 consumed_samples: 70758400 total_loss: 3.937 time: 0.3395 s/iter data_time: 0.2084 s/iter total_throughput: 3016.49 samples/s lr: 9.19e-04 [09/22 09:52:41] lb.utils.events INFO: eta: 13:22:43 iteration: 69199/375342 consumed_samples: 70860800 total_loss: 3.925 time: 0.3395 s/iter data_time: 0.2144 s/iter total_throughput: 3016.51 samples/s lr: 9.19e-04 [09/22 09:53:15] lb.utils.events INFO: eta: 13:25:34 iteration: 69299/375342 consumed_samples: 70963200 total_loss: 3.937 time: 0.3395 s/iter data_time: 0.1999 s/iter total_throughput: 3016.63 samples/s lr: 9.19e-04 [09/22 09:53:48] lb.utils.events INFO: eta: 13:26:15 iteration: 69399/375342 consumed_samples: 71065600 total_loss: 3.918 time: 0.3394 s/iter data_time: 0.2178 s/iter total_throughput: 3016.72 samples/s lr: 9.19e-04 [09/22 09:54:21] lb.utils.events INFO: eta: 13:27:30 iteration: 69499/375342 consumed_samples: 71168000 total_loss: 3.933 time: 0.3394 s/iter data_time: 0.2077 s/iter total_throughput: 3016.80 samples/s lr: 9.19e-04 [09/22 09:54:54] lb.utils.events INFO: eta: 13:27:38 iteration: 69599/375342 consumed_samples: 71270400 total_loss: 3.951 time: 0.3394 s/iter data_time: 0.2107 s/iter total_throughput: 3016.87 samples/s lr: 9.18e-04 [09/22 09:55:28] lb.utils.events INFO: eta: 13:27:23 iteration: 69699/375342 consumed_samples: 71372800 total_loss: 3.948 time: 0.3394 s/iter data_time: 0.1997 s/iter total_throughput: 3016.95 samples/s lr: 9.18e-04 [09/22 09:56:02] lb.utils.events INFO: eta: 13:26:05 iteration: 69799/375342 consumed_samples: 71475200 total_loss: 3.942 time: 0.3394 s/iter data_time: 0.1957 s/iter total_throughput: 3016.97 samples/s lr: 9.18e-04 [09/22 09:56:35] lb.utils.events INFO: eta: 13:25:12 iteration: 69899/375342 consumed_samples: 71577600 total_loss: 3.95 time: 0.3394 s/iter data_time: 0.1949 s/iter total_throughput: 3017.07 samples/s lr: 9.18e-04 [09/22 09:57:08] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0069999 [09/22 09:57:08] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/22 09:57:08] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/22 09:57:12] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0975 s/iter. Inference: 0.1580 s/iter. Eval: 0.0023 s/iter. Total: 0.2578 s/iter. ETA=0:00:09 [09/22 09:57:18] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1142 s/iter. Inference: 0.1796 s/iter. Eval: 0.0020 s/iter. Total: 0.2958 s/iter. ETA=0:00:05 [09/22 09:57:23] lb.evaluation.evaluator INFO: Inference done 49152/50000. Dataloading: 0.0966 s/iter. Inference: 0.1852 s/iter. Eval: 0.0020 s/iter. Total: 0.2840 s/iter. ETA=0:00:00 [09/22 09:57:23] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/22 09:57:23] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.694773 (0.000254 s / iter per device, on 8 devices) [09/22 09:57:23] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:08 (0.000169 s / iter per device, on 8 devices) [09/22 09:57:23] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/22 09:57:23] lb.evaluation.utils INFO: copypaste: Acc@1=69.328 [09/22 09:57:23] lb.evaluation.utils INFO: copypaste: Acc@5=89.41799999999999 [09/22 09:57:23] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 69.32800, better than last best score 68.80200 @ iteration 64999. [09/22 09:57:23] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/22 09:57:24] lb.utils.events INFO: eta: 13:24:56 iteration: 69999/375342 consumed_samples: 71680000 total_loss: 3.926 time: 0.3394 s/iter data_time: 0.2034 s/iter total_throughput: 3017.22 samples/s lr: 9.17e-04 [09/22 09:57:56] lb.utils.events INFO: eta: 13:29:02 iteration: 70099/375342 consumed_samples: 71782400 total_loss: 3.911 time: 0.3393 s/iter data_time: 0.2456 s/iter total_throughput: 3017.54 samples/s lr: 9.17e-04 [09/22 09:58:30] lb.utils.events INFO: eta: 13:30:23 iteration: 70199/375342 consumed_samples: 71884800 total_loss: 3.892 time: 0.3393 s/iter data_time: 0.2099 s/iter total_throughput: 3017.54 samples/s lr: 9.17e-04 [09/22 09:59:04] lb.utils.events INFO: eta: 13:29:04 iteration: 70299/375342 consumed_samples: 71987200 total_loss: 3.88 time: 0.3394 s/iter data_time: 0.2020 s/iter total_throughput: 3017.51 samples/s lr: 9.17e-04 [09/22 09:59:38] lb.utils.events INFO: eta: 13:28:36 iteration: 70399/375342 consumed_samples: 72089600 total_loss: 3.889 time: 0.3394 s/iter data_time: 0.2233 s/iter total_throughput: 3017.48 samples/s lr: 9.17e-04 [09/22 10:00:12] lb.utils.events INFO: eta: 13:27:22 iteration: 70499/375342 consumed_samples: 72192000 total_loss: 3.913 time: 0.3394 s/iter data_time: 0.2136 s/iter total_throughput: 3017.42 samples/s lr: 9.16e-04 [09/22 10:00:47] lb.utils.events INFO: eta: 13:26:15 iteration: 70599/375342 consumed_samples: 72294400 total_loss: 3.934 time: 0.3394 s/iter data_time: 0.2100 s/iter total_throughput: 3017.37 samples/s lr: 9.16e-04 [09/22 10:01:21] lb.utils.events INFO: eta: 13:27:55 iteration: 70699/375342 consumed_samples: 72396800 total_loss: 3.927 time: 0.3394 s/iter data_time: 0.2156 s/iter total_throughput: 3017.30 samples/s lr: 9.16e-04 [09/22 10:01:55] lb.utils.events INFO: eta: 13:25:16 iteration: 70799/375342 consumed_samples: 72499200 total_loss: 3.945 time: 0.3394 s/iter data_time: 0.2134 s/iter total_throughput: 3017.26 samples/s lr: 9.16e-04 [09/22 10:02:30] lb.utils.events INFO: eta: 13:24:20 iteration: 70899/375342 consumed_samples: 72601600 total_loss: 3.94 time: 0.3394 s/iter data_time: 0.2126 s/iter total_throughput: 3017.23 samples/s lr: 9.15e-04 [09/22 10:03:04] lb.utils.events INFO: eta: 13:23:33 iteration: 70999/375342 consumed_samples: 72704000 total_loss: 3.911 time: 0.3394 s/iter data_time: 0.2087 s/iter total_throughput: 3017.16 samples/s lr: 9.15e-04 [09/22 10:03:38] lb.utils.events INFO: eta: 13:19:31 iteration: 71099/375342 consumed_samples: 72806400 total_loss: 3.896 time: 0.3394 s/iter data_time: 0.2113 s/iter total_throughput: 3017.16 samples/s lr: 9.15e-04 [09/22 10:04:12] lb.utils.events INFO: eta: 13:17:59 iteration: 71199/375342 consumed_samples: 72908800 total_loss: 3.925 time: 0.3394 s/iter data_time: 0.2104 s/iter total_throughput: 3017.13 samples/s lr: 9.15e-04 [09/22 10:04:46] lb.utils.events INFO: eta: 13:19:00 iteration: 71299/375342 consumed_samples: 73011200 total_loss: 3.948 time: 0.3394 s/iter data_time: 0.2156 s/iter total_throughput: 3017.16 samples/s lr: 9.14e-04 [09/22 10:05:20] lb.utils.events INFO: eta: 13:19:49 iteration: 71399/375342 consumed_samples: 73113600 total_loss: 3.968 time: 0.3394 s/iter data_time: 0.2202 s/iter total_throughput: 3017.16 samples/s lr: 9.14e-04 [09/22 10:05:54] lb.utils.events INFO: eta: 13:17:17 iteration: 71499/375342 consumed_samples: 73216000 total_loss: 3.951 time: 0.3394 s/iter data_time: 0.2116 s/iter total_throughput: 3017.10 samples/s lr: 9.14e-04 [09/22 10:06:29] lb.utils.events INFO: eta: 13:18:16 iteration: 71599/375342 consumed_samples: 73318400 total_loss: 3.926 time: 0.3394 s/iter data_time: 0.2159 s/iter total_throughput: 3017.01 samples/s lr: 9.14e-04 [09/22 10:07:03] lb.utils.events INFO: eta: 13:16:44 iteration: 71699/375342 consumed_samples: 73420800 total_loss: 3.918 time: 0.3394 s/iter data_time: 0.2310 s/iter total_throughput: 3016.96 samples/s lr: 9.14e-04 [09/22 10:07:37] lb.utils.events INFO: eta: 13:18:03 iteration: 71799/375342 consumed_samples: 73523200 total_loss: 3.918 time: 0.3394 s/iter data_time: 0.2161 s/iter total_throughput: 3017.00 samples/s lr: 9.13e-04 [09/22 10:08:11] lb.utils.events INFO: eta: 13:18:22 iteration: 71899/375342 consumed_samples: 73625600 total_loss: 3.923 time: 0.3394 s/iter data_time: 0.2081 s/iter total_throughput: 3016.99 samples/s lr: 9.13e-04 [09/22 10:08:45] lb.utils.events INFO: eta: 13:18:43 iteration: 71999/375342 consumed_samples: 73728000 total_loss: 3.916 time: 0.3394 s/iter data_time: 0.2161 s/iter total_throughput: 3016.98 samples/s lr: 9.13e-04 [09/22 10:09:20] lb.utils.events INFO: eta: 13:15:33 iteration: 72099/375342 consumed_samples: 73830400 total_loss: 3.913 time: 0.3394 s/iter data_time: 0.2105 s/iter total_throughput: 3016.92 samples/s lr: 9.13e-04 [09/22 10:09:54] lb.utils.events INFO: eta: 13:14:34 iteration: 72199/375342 consumed_samples: 73932800 total_loss: 3.924 time: 0.3394 s/iter data_time: 0.2076 s/iter total_throughput: 3016.90 samples/s lr: 9.12e-04 [09/22 10:10:28] lb.utils.events INFO: eta: 13:14:06 iteration: 72299/375342 consumed_samples: 74035200 total_loss: 3.941 time: 0.3394 s/iter data_time: 0.2169 s/iter total_throughput: 3016.88 samples/s lr: 9.12e-04 [09/22 10:11:02] lb.utils.events INFO: eta: 13:13:53 iteration: 72399/375342 consumed_samples: 74137600 total_loss: 3.949 time: 0.3394 s/iter data_time: 0.2312 s/iter total_throughput: 3016.87 samples/s lr: 9.12e-04 [09/22 10:11:36] lb.utils.events INFO: eta: 13:14:54 iteration: 72499/375342 consumed_samples: 74240000 total_loss: 3.93 time: 0.3394 s/iter data_time: 0.2182 s/iter total_throughput: 3016.80 samples/s lr: 9.12e-04 [09/22 10:12:10] lb.utils.events INFO: eta: 13:14:44 iteration: 72599/375342 consumed_samples: 74342400 total_loss: 3.926 time: 0.3394 s/iter data_time: 0.2243 s/iter total_throughput: 3016.77 samples/s lr: 9.11e-04 [09/22 10:12:45] lb.utils.events INFO: eta: 13:15:20 iteration: 72699/375342 consumed_samples: 74444800 total_loss: 3.898 time: 0.3394 s/iter data_time: 0.2179 s/iter total_throughput: 3016.76 samples/s lr: 9.11e-04 [09/22 10:13:18] lb.utils.events INFO: eta: 13:15:14 iteration: 72799/375342 consumed_samples: 74547200 total_loss: 3.892 time: 0.3394 s/iter data_time: 0.2172 s/iter total_throughput: 3016.77 samples/s lr: 9.11e-04 [09/22 10:13:52] lb.utils.events INFO: eta: 13:15:05 iteration: 72899/375342 consumed_samples: 74649600 total_loss: 3.902 time: 0.3394 s/iter data_time: 0.2119 s/iter total_throughput: 3016.79 samples/s lr: 9.11e-04 [09/22 10:14:26] lb.utils.events INFO: eta: 13:14:44 iteration: 72999/375342 consumed_samples: 74752000 total_loss: 3.911 time: 0.3394 s/iter data_time: 0.2103 s/iter total_throughput: 3016.79 samples/s lr: 9.10e-04 [09/22 10:15:00] lb.utils.events INFO: eta: 13:15:52 iteration: 73099/375342 consumed_samples: 74854400 total_loss: 3.904 time: 0.3394 s/iter data_time: 0.2184 s/iter total_throughput: 3016.75 samples/s lr: 9.10e-04 [09/22 10:15:35] lb.utils.events INFO: eta: 13:15:49 iteration: 73199/375342 consumed_samples: 74956800 total_loss: 3.917 time: 0.3394 s/iter data_time: 0.2100 s/iter total_throughput: 3016.73 samples/s lr: 9.10e-04 [09/22 10:16:08] lb.utils.events INFO: eta: 13:16:28 iteration: 73299/375342 consumed_samples: 75059200 total_loss: 3.945 time: 0.3394 s/iter data_time: 0.2202 s/iter total_throughput: 3016.77 samples/s lr: 9.10e-04 [09/22 10:16:42] lb.utils.events INFO: eta: 13:16:06 iteration: 73399/375342 consumed_samples: 75161600 total_loss: 3.933 time: 0.3394 s/iter data_time: 0.2094 s/iter total_throughput: 3016.77 samples/s lr: 9.09e-04 [09/22 10:17:16] lb.utils.events INFO: eta: 13:15:50 iteration: 73499/375342 consumed_samples: 75264000 total_loss: 3.913 time: 0.3394 s/iter data_time: 0.2225 s/iter total_throughput: 3016.77 samples/s lr: 9.09e-04 [09/22 10:17:50] lb.utils.events INFO: eta: 13:16:18 iteration: 73599/375342 consumed_samples: 75366400 total_loss: 3.911 time: 0.3394 s/iter data_time: 0.2123 s/iter total_throughput: 3016.80 samples/s lr: 9.09e-04 [09/22 10:18:24] lb.utils.events INFO: eta: 13:16:43 iteration: 73699/375342 consumed_samples: 75468800 total_loss: 3.93 time: 0.3394 s/iter data_time: 0.2088 s/iter total_throughput: 3016.82 samples/s lr: 9.09e-04 [09/22 10:18:58] lb.utils.events INFO: eta: 13:16:58 iteration: 73799/375342 consumed_samples: 75571200 total_loss: 3.937 time: 0.3394 s/iter data_time: 0.2235 s/iter total_throughput: 3016.76 samples/s lr: 9.09e-04 [09/22 10:19:32] lb.utils.events INFO: eta: 13:17:44 iteration: 73899/375342 consumed_samples: 75673600 total_loss: 3.893 time: 0.3394 s/iter data_time: 0.2246 s/iter total_throughput: 3016.79 samples/s lr: 9.08e-04 [09/22 10:20:06] lb.utils.events INFO: eta: 13:17:44 iteration: 73999/375342 consumed_samples: 75776000 total_loss: 3.879 time: 0.3394 s/iter data_time: 0.2174 s/iter total_throughput: 3016.75 samples/s lr: 9.08e-04 [09/22 10:20:40] lb.utils.events INFO: eta: 13:18:03 iteration: 74099/375342 consumed_samples: 75878400 total_loss: 3.901 time: 0.3394 s/iter data_time: 0.2026 s/iter total_throughput: 3016.77 samples/s lr: 9.08e-04 [09/22 10:21:14] lb.utils.events INFO: eta: 13:19:18 iteration: 74199/375342 consumed_samples: 75980800 total_loss: 3.925 time: 0.3394 s/iter data_time: 0.2260 s/iter total_throughput: 3016.73 samples/s lr: 9.08e-04 [09/22 10:21:48] lb.utils.events INFO: eta: 13:17:32 iteration: 74299/375342 consumed_samples: 76083200 total_loss: 3.915 time: 0.3394 s/iter data_time: 0.2164 s/iter total_throughput: 3016.70 samples/s lr: 9.07e-04 [09/22 10:22:23] lb.utils.events INFO: eta: 13:16:34 iteration: 74399/375342 consumed_samples: 76185600 total_loss: 3.903 time: 0.3395 s/iter data_time: 0.2234 s/iter total_throughput: 3016.62 samples/s lr: 9.07e-04 [09/22 10:22:57] lb.utils.events INFO: eta: 13:15:43 iteration: 74499/375342 consumed_samples: 76288000 total_loss: 3.888 time: 0.3395 s/iter data_time: 0.2072 s/iter total_throughput: 3016.57 samples/s lr: 9.07e-04 [09/22 10:23:31] lb.utils.events INFO: eta: 13:14:48 iteration: 74599/375342 consumed_samples: 76390400 total_loss: 3.902 time: 0.3395 s/iter data_time: 0.2084 s/iter total_throughput: 3016.61 samples/s lr: 9.07e-04 [09/22 10:24:04] lb.utils.events INFO: eta: 13:15:44 iteration: 74699/375342 consumed_samples: 76492800 total_loss: 3.925 time: 0.3394 s/iter data_time: 0.2166 s/iter total_throughput: 3016.66 samples/s lr: 9.06e-04 [09/22 10:24:38] lb.utils.events INFO: eta: 13:14:44 iteration: 74799/375342 consumed_samples: 76595200 total_loss: 3.92 time: 0.3394 s/iter data_time: 0.2112 s/iter total_throughput: 3016.69 samples/s lr: 9.06e-04 [09/22 10:25:12] lb.utils.events INFO: eta: 13:13:49 iteration: 74899/375342 consumed_samples: 76697600 total_loss: 3.919 time: 0.3394 s/iter data_time: 0.2092 s/iter total_throughput: 3016.68 samples/s lr: 9.06e-04 [09/22 10:25:46] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0074999 [09/22 10:25:47] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/22 10:25:47] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/22 10:25:51] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0995 s/iter. Inference: 0.1619 s/iter. Eval: 0.0021 s/iter. Total: 0.2636 s/iter. ETA=0:00:09 [09/22 10:25:56] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1186 s/iter. Inference: 0.1712 s/iter. Eval: 0.0020 s/iter. Total: 0.2920 s/iter. ETA=0:00:05 [09/22 10:26:02] lb.evaluation.evaluator INFO: Inference done 48128/50000. Dataloading: 0.1112 s/iter. Inference: 0.1745 s/iter. Eval: 0.0020 s/iter. Total: 0.2877 s/iter. ETA=0:00:00 [09/22 10:26:02] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/22 10:26:02] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.437035 (0.000249 s / iter per device, on 8 devices) [09/22 10:26:02] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000153 s / iter per device, on 8 devices) [09/22 10:26:02] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/22 10:26:02] lb.evaluation.utils INFO: copypaste: Acc@1=69.482 [09/22 10:26:02] lb.evaluation.utils INFO: copypaste: Acc@5=89.622 [09/22 10:26:02] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 69.48200, better than last best score 69.32800 @ iteration 69999. [09/22 10:26:02] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/22 10:26:03] lb.utils.events INFO: eta: 13:13:24 iteration: 74999/375342 consumed_samples: 76800000 total_loss: 3.94 time: 0.3394 s/iter data_time: 0.2227 s/iter total_throughput: 3016.67 samples/s lr: 9.06e-04 [09/22 10:26:35] lb.utils.events INFO: eta: 13:16:06 iteration: 75099/375342 consumed_samples: 76902400 total_loss: 3.938 time: 0.3394 s/iter data_time: 0.2418 s/iter total_throughput: 3016.89 samples/s lr: 9.05e-04 [09/22 10:27:09] lb.utils.events INFO: eta: 13:15:55 iteration: 75199/375342 consumed_samples: 77004800 total_loss: 3.893 time: 0.3394 s/iter data_time: 0.2066 s/iter total_throughput: 3016.87 samples/s lr: 9.05e-04 [09/22 10:27:43] lb.utils.events INFO: eta: 13:16:47 iteration: 75299/375342 consumed_samples: 77107200 total_loss: 3.898 time: 0.3394 s/iter data_time: 0.2268 s/iter total_throughput: 3016.90 samples/s lr: 9.05e-04 [09/22 10:28:17] lb.utils.events INFO: eta: 13:17:16 iteration: 75399/375342 consumed_samples: 77209600 total_loss: 3.912 time: 0.3394 s/iter data_time: 0.2168 s/iter total_throughput: 3016.89 samples/s lr: 9.05e-04 [09/22 10:28:51] lb.utils.events INFO: eta: 13:19:23 iteration: 75499/375342 consumed_samples: 77312000 total_loss: 3.902 time: 0.3394 s/iter data_time: 0.2297 s/iter total_throughput: 3016.89 samples/s lr: 9.04e-04 [09/22 10:29:25] lb.utils.events INFO: eta: 13:17:47 iteration: 75599/375342 consumed_samples: 77414400 total_loss: 3.906 time: 0.3394 s/iter data_time: 0.2181 s/iter total_throughput: 3016.83 samples/s lr: 9.04e-04 [09/22 10:29:59] lb.utils.events INFO: eta: 13:15:37 iteration: 75699/375342 consumed_samples: 77516800 total_loss: 3.914 time: 0.3394 s/iter data_time: 0.2290 s/iter total_throughput: 3016.78 samples/s lr: 9.04e-04 [09/22 10:30:33] lb.utils.events INFO: eta: 13:17:16 iteration: 75799/375342 consumed_samples: 77619200 total_loss: 3.894 time: 0.3394 s/iter data_time: 0.2032 s/iter total_throughput: 3016.79 samples/s lr: 9.04e-04 [09/22 10:31:07] lb.utils.events INFO: eta: 13:16:30 iteration: 75899/375342 consumed_samples: 77721600 total_loss: 3.896 time: 0.3394 s/iter data_time: 0.2112 s/iter total_throughput: 3016.78 samples/s lr: 9.03e-04 [09/22 10:31:41] lb.utils.events INFO: eta: 13:17:16 iteration: 75999/375342 consumed_samples: 77824000 total_loss: 3.912 time: 0.3394 s/iter data_time: 0.2007 s/iter total_throughput: 3016.78 samples/s lr: 9.03e-04 [09/22 10:32:16] lb.utils.events INFO: eta: 13:12:34 iteration: 76099/375342 consumed_samples: 77926400 total_loss: 3.919 time: 0.3394 s/iter data_time: 0.2171 s/iter total_throughput: 3016.74 samples/s lr: 9.03e-04 [09/22 10:32:50] lb.utils.events INFO: eta: 13:09:42 iteration: 76199/375342 consumed_samples: 78028800 total_loss: 3.931 time: 0.3394 s/iter data_time: 0.2106 s/iter total_throughput: 3016.75 samples/s lr: 9.03e-04 [09/22 10:33:23] lb.utils.events INFO: eta: 13:09:38 iteration: 76299/375342 consumed_samples: 78131200 total_loss: 3.93 time: 0.3394 s/iter data_time: 0.2267 s/iter total_throughput: 3016.75 samples/s lr: 9.02e-04 [09/22 10:33:57] lb.utils.events INFO: eta: 13:08:46 iteration: 76399/375342 consumed_samples: 78233600 total_loss: 3.925 time: 0.3394 s/iter data_time: 0.2095 s/iter total_throughput: 3016.77 samples/s lr: 9.02e-04 [09/22 10:34:32] lb.utils.events INFO: eta: 13:07:09 iteration: 76499/375342 consumed_samples: 78336000 total_loss: 3.927 time: 0.3394 s/iter data_time: 0.2088 s/iter total_throughput: 3016.73 samples/s lr: 9.02e-04 [09/22 10:35:06] lb.utils.events INFO: eta: 13:06:53 iteration: 76599/375342 consumed_samples: 78438400 total_loss: 3.93 time: 0.3394 s/iter data_time: 0.2197 s/iter total_throughput: 3016.68 samples/s lr: 9.02e-04 [09/22 10:35:40] lb.utils.events INFO: eta: 13:07:39 iteration: 76699/375342 consumed_samples: 78540800 total_loss: 3.917 time: 0.3395 s/iter data_time: 0.2313 s/iter total_throughput: 3016.62 samples/s lr: 9.01e-04 [09/22 10:36:15] lb.utils.events INFO: eta: 13:05:39 iteration: 76799/375342 consumed_samples: 78643200 total_loss: 3.893 time: 0.3395 s/iter data_time: 0.2266 s/iter total_throughput: 3016.56 samples/s lr: 9.01e-04 [09/22 10:36:49] lb.utils.events INFO: eta: 13:03:48 iteration: 76899/375342 consumed_samples: 78745600 total_loss: 3.898 time: 0.3395 s/iter data_time: 0.2159 s/iter total_throughput: 3016.48 samples/s lr: 9.01e-04 [09/22 10:37:24] lb.utils.events INFO: eta: 13:00:24 iteration: 76999/375342 consumed_samples: 78848000 total_loss: 3.907 time: 0.3395 s/iter data_time: 0.2175 s/iter total_throughput: 3016.46 samples/s lr: 9.01e-04 [09/22 10:37:58] lb.utils.events INFO: eta: 12:59:55 iteration: 77099/375342 consumed_samples: 78950400 total_loss: 3.882 time: 0.3395 s/iter data_time: 0.2216 s/iter total_throughput: 3016.45 samples/s lr: 9.00e-04 [09/22 10:38:32] lb.utils.events INFO: eta: 12:58:35 iteration: 77199/375342 consumed_samples: 79052800 total_loss: 3.899 time: 0.3395 s/iter data_time: 0.2185 s/iter total_throughput: 3016.35 samples/s lr: 9.00e-04 [09/22 10:39:06] lb.utils.events INFO: eta: 12:58:43 iteration: 77299/375342 consumed_samples: 79155200 total_loss: 3.907 time: 0.3395 s/iter data_time: 0.2030 s/iter total_throughput: 3016.35 samples/s lr: 9.00e-04 [09/22 10:39:41] lb.utils.events INFO: eta: 12:57:02 iteration: 77399/375342 consumed_samples: 79257600 total_loss: 3.907 time: 0.3395 s/iter data_time: 0.2199 s/iter total_throughput: 3016.32 samples/s lr: 9.00e-04 [09/22 10:40:16] lb.utils.events INFO: eta: 12:56:46 iteration: 77499/375342 consumed_samples: 79360000 total_loss: 3.908 time: 0.3395 s/iter data_time: 0.2371 s/iter total_throughput: 3016.21 samples/s lr: 8.99e-04 [09/22 10:40:50] lb.utils.events INFO: eta: 12:57:12 iteration: 77599/375342 consumed_samples: 79462400 total_loss: 3.898 time: 0.3395 s/iter data_time: 0.2222 s/iter total_throughput: 3016.19 samples/s lr: 8.99e-04 [09/22 10:41:24] lb.utils.events INFO: eta: 12:57:18 iteration: 77699/375342 consumed_samples: 79564800 total_loss: 3.902 time: 0.3395 s/iter data_time: 0.2125 s/iter total_throughput: 3016.20 samples/s lr: 8.99e-04 [09/22 10:41:58] lb.utils.events INFO: eta: 12:56:22 iteration: 77799/375342 consumed_samples: 79667200 total_loss: 3.912 time: 0.3395 s/iter data_time: 0.2095 s/iter total_throughput: 3016.16 samples/s lr: 8.99e-04 [09/22 10:42:31] lb.utils.events INFO: eta: 12:59:20 iteration: 77899/375342 consumed_samples: 79769600 total_loss: 3.891 time: 0.3395 s/iter data_time: 0.1979 s/iter total_throughput: 3016.20 samples/s lr: 8.98e-04 [09/22 10:43:06] lb.utils.events INFO: eta: 13:00:33 iteration: 77999/375342 consumed_samples: 79872000 total_loss: 3.887 time: 0.3395 s/iter data_time: 0.2082 s/iter total_throughput: 3016.18 samples/s lr: 8.98e-04 [09/22 10:43:40] lb.utils.events INFO: eta: 13:00:31 iteration: 78099/375342 consumed_samples: 79974400 total_loss: 3.913 time: 0.3395 s/iter data_time: 0.2171 s/iter total_throughput: 3016.16 samples/s lr: 8.98e-04 [09/22 10:44:14] lb.utils.events INFO: eta: 13:00:37 iteration: 78199/375342 consumed_samples: 80076800 total_loss: 3.915 time: 0.3395 s/iter data_time: 0.2220 s/iter total_throughput: 3016.12 samples/s lr: 8.98e-04 [09/22 10:44:48] lb.utils.events INFO: eta: 12:59:05 iteration: 78299/375342 consumed_samples: 80179200 total_loss: 3.915 time: 0.3395 s/iter data_time: 0.2134 s/iter total_throughput: 3016.09 samples/s lr: 8.97e-04 [09/22 10:45:22] lb.utils.events INFO: eta: 12:59:25 iteration: 78399/375342 consumed_samples: 80281600 total_loss: 3.9 time: 0.3395 s/iter data_time: 0.2063 s/iter total_throughput: 3016.08 samples/s lr: 8.97e-04 [09/22 10:45:56] lb.utils.events INFO: eta: 12:58:33 iteration: 78499/375342 consumed_samples: 80384000 total_loss: 3.886 time: 0.3395 s/iter data_time: 0.2168 s/iter total_throughput: 3016.07 samples/s lr: 8.97e-04 [09/22 10:46:31] lb.utils.events INFO: eta: 12:58:18 iteration: 78599/375342 consumed_samples: 80486400 total_loss: 3.881 time: 0.3395 s/iter data_time: 0.2194 s/iter total_throughput: 3016.03 samples/s lr: 8.97e-04 [09/22 10:47:04] lb.utils.events INFO: eta: 12:58:10 iteration: 78699/375342 consumed_samples: 80588800 total_loss: 3.878 time: 0.3395 s/iter data_time: 0.2043 s/iter total_throughput: 3016.05 samples/s lr: 8.96e-04 [09/22 10:47:39] lb.utils.events INFO: eta: 12:57:55 iteration: 78799/375342 consumed_samples: 80691200 total_loss: 3.883 time: 0.3395 s/iter data_time: 0.2118 s/iter total_throughput: 3016.04 samples/s lr: 8.96e-04 [09/22 10:48:12] lb.utils.events INFO: eta: 12:56:47 iteration: 78899/375342 consumed_samples: 80793600 total_loss: 3.875 time: 0.3395 s/iter data_time: 0.2046 s/iter total_throughput: 3016.06 samples/s lr: 8.96e-04 [09/22 10:48:46] lb.utils.events INFO: eta: 12:56:42 iteration: 78999/375342 consumed_samples: 80896000 total_loss: 3.866 time: 0.3395 s/iter data_time: 0.2127 s/iter total_throughput: 3016.07 samples/s lr: 8.96e-04 [09/22 10:49:20] lb.utils.events INFO: eta: 12:56:45 iteration: 79099/375342 consumed_samples: 80998400 total_loss: 3.862 time: 0.3395 s/iter data_time: 0.2147 s/iter total_throughput: 3016.07 samples/s lr: 8.95e-04 [09/22 10:49:54] lb.utils.events INFO: eta: 12:55:53 iteration: 79199/375342 consumed_samples: 81100800 total_loss: 3.879 time: 0.3395 s/iter data_time: 0.2215 s/iter total_throughput: 3016.08 samples/s lr: 8.95e-04 [09/22 10:50:28] lb.utils.events INFO: eta: 12:56:58 iteration: 79299/375342 consumed_samples: 81203200 total_loss: 3.897 time: 0.3395 s/iter data_time: 0.2148 s/iter total_throughput: 3016.12 samples/s lr: 8.95e-04 [09/22 10:51:02] lb.utils.events INFO: eta: 12:56:23 iteration: 79399/375342 consumed_samples: 81305600 total_loss: 3.888 time: 0.3395 s/iter data_time: 0.2200 s/iter total_throughput: 3016.06 samples/s lr: 8.95e-04 [09/22 10:51:36] lb.utils.events INFO: eta: 12:56:54 iteration: 79499/375342 consumed_samples: 81408000 total_loss: 3.881 time: 0.3395 s/iter data_time: 0.2037 s/iter total_throughput: 3016.09 samples/s lr: 8.94e-04 [09/22 10:52:10] lb.utils.events INFO: eta: 12:56:38 iteration: 79599/375342 consumed_samples: 81510400 total_loss: 3.889 time: 0.3395 s/iter data_time: 0.2165 s/iter total_throughput: 3016.04 samples/s lr: 8.94e-04 [09/22 10:52:44] lb.utils.events INFO: eta: 12:56:46 iteration: 79699/375342 consumed_samples: 81612800 total_loss: 3.89 time: 0.3395 s/iter data_time: 0.2234 s/iter total_throughput: 3016.07 samples/s lr: 8.94e-04 [09/22 10:53:18] lb.utils.events INFO: eta: 12:55:38 iteration: 79799/375342 consumed_samples: 81715200 total_loss: 3.886 time: 0.3395 s/iter data_time: 0.2167 s/iter total_throughput: 3016.03 samples/s lr: 8.94e-04 [09/22 10:53:52] lb.utils.events INFO: eta: 12:54:55 iteration: 79899/375342 consumed_samples: 81817600 total_loss: 3.87 time: 0.3395 s/iter data_time: 0.2225 s/iter total_throughput: 3016.01 samples/s lr: 8.93e-04 [09/22 10:54:27] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0079999 [09/22 10:54:27] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/22 10:54:27] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/22 10:54:32] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0988 s/iter. Inference: 0.1626 s/iter. Eval: 0.0020 s/iter. Total: 0.2634 s/iter. ETA=0:00:09 [09/22 10:54:37] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.0848 s/iter. Inference: 0.2122 s/iter. Eval: 0.0020 s/iter. Total: 0.2991 s/iter. ETA=0:00:05 [09/22 10:54:42] lb.evaluation.evaluator INFO: Inference done 49152/50000. Dataloading: 0.0745 s/iter. Inference: 0.2072 s/iter. Eval: 0.0020 s/iter. Total: 0.2839 s/iter. ETA=0:00:00 [09/22 10:54:43] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/22 10:54:43] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.570490 (0.000251 s / iter per device, on 8 devices) [09/22 10:54:43] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:09 (0.000185 s / iter per device, on 8 devices) [09/22 10:54:43] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/22 10:54:43] lb.evaluation.utils INFO: copypaste: Acc@1=70.336 [09/22 10:54:43] lb.evaluation.utils INFO: copypaste: Acc@5=90.104 [09/22 10:54:43] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 70.33600, better than last best score 69.48200 @ iteration 74999. [09/22 10:54:43] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/22 10:54:43] lb.utils.events INFO: eta: 12:54:53 iteration: 79999/375342 consumed_samples: 81920000 total_loss: 3.895 time: 0.3395 s/iter data_time: 0.2279 s/iter total_throughput: 3015.97 samples/s lr: 8.93e-04 [09/22 10:55:15] lb.utils.events INFO: eta: 12:57:21 iteration: 80099/375342 consumed_samples: 82022400 total_loss: 3.91 time: 0.3395 s/iter data_time: 0.2571 s/iter total_throughput: 3016.20 samples/s lr: 8.93e-04 [09/22 10:55:49] lb.utils.events INFO: eta: 12:57:37 iteration: 80199/375342 consumed_samples: 82124800 total_loss: 3.881 time: 0.3395 s/iter data_time: 0.2194 s/iter total_throughput: 3016.17 samples/s lr: 8.93e-04 [09/22 10:56:23] lb.utils.events INFO: eta: 12:55:55 iteration: 80299/375342 consumed_samples: 82227200 total_loss: 3.865 time: 0.3395 s/iter data_time: 0.2122 s/iter total_throughput: 3016.19 samples/s lr: 8.92e-04 [09/22 10:56:57] lb.utils.events INFO: eta: 12:56:13 iteration: 80399/375342 consumed_samples: 82329600 total_loss: 3.855 time: 0.3395 s/iter data_time: 0.2115 s/iter total_throughput: 3016.18 samples/s lr: 8.92e-04 [09/22 10:57:31] lb.utils.events INFO: eta: 12:56:32 iteration: 80499/375342 consumed_samples: 82432000 total_loss: 3.889 time: 0.3395 s/iter data_time: 0.2076 s/iter total_throughput: 3016.20 samples/s lr: 8.92e-04 [09/22 10:58:05] lb.utils.events INFO: eta: 12:56:45 iteration: 80599/375342 consumed_samples: 82534400 total_loss: 3.89 time: 0.3395 s/iter data_time: 0.2123 s/iter total_throughput: 3016.17 samples/s lr: 8.92e-04 [09/22 10:58:39] lb.utils.events INFO: eta: 12:56:54 iteration: 80699/375342 consumed_samples: 82636800 total_loss: 3.892 time: 0.3395 s/iter data_time: 0.2220 s/iter total_throughput: 3016.18 samples/s lr: 8.91e-04 [09/22 10:59:13] lb.utils.events INFO: eta: 12:57:14 iteration: 80799/375342 consumed_samples: 82739200 total_loss: 3.893 time: 0.3395 s/iter data_time: 0.2084 s/iter total_throughput: 3016.17 samples/s lr: 8.91e-04 [09/22 10:59:47] lb.utils.events INFO: eta: 12:57:55 iteration: 80899/375342 consumed_samples: 82841600 total_loss: 3.886 time: 0.3395 s/iter data_time: 0.2071 s/iter total_throughput: 3016.20 samples/s lr: 8.91e-04 [09/22 11:00:21] lb.utils.events INFO: eta: 12:56:42 iteration: 80999/375342 consumed_samples: 82944000 total_loss: 3.888 time: 0.3395 s/iter data_time: 0.2120 s/iter total_throughput: 3016.18 samples/s lr: 8.91e-04 [09/22 11:00:55] lb.utils.events INFO: eta: 12:52:32 iteration: 81099/375342 consumed_samples: 83046400 total_loss: 3.896 time: 0.3395 s/iter data_time: 0.2116 s/iter total_throughput: 3016.16 samples/s lr: 8.90e-04 [09/22 11:01:30] lb.utils.events INFO: eta: 12:52:47 iteration: 81199/375342 consumed_samples: 83148800 total_loss: 3.896 time: 0.3395 s/iter data_time: 0.2180 s/iter total_throughput: 3016.11 samples/s lr: 8.90e-04 [09/22 11:02:03] lb.utils.events INFO: eta: 12:53:03 iteration: 81299/375342 consumed_samples: 83251200 total_loss: 3.896 time: 0.3395 s/iter data_time: 0.2082 s/iter total_throughput: 3016.16 samples/s lr: 8.90e-04 [09/22 11:02:37] lb.utils.events INFO: eta: 12:53:43 iteration: 81399/375342 consumed_samples: 83353600 total_loss: 3.897 time: 0.3395 s/iter data_time: 0.2195 s/iter total_throughput: 3016.12 samples/s lr: 8.89e-04 [09/22 11:03:11] lb.utils.events INFO: eta: 12:51:46 iteration: 81499/375342 consumed_samples: 83456000 total_loss: 3.888 time: 0.3395 s/iter data_time: 0.2151 s/iter total_throughput: 3016.14 samples/s lr: 8.89e-04 [09/22 11:03:45] lb.utils.events INFO: eta: 12:51:42 iteration: 81599/375342 consumed_samples: 83558400 total_loss: 3.871 time: 0.3395 s/iter data_time: 0.2093 s/iter total_throughput: 3016.17 samples/s lr: 8.89e-04 [09/22 11:04:18] lb.utils.events INFO: eta: 12:50:58 iteration: 81699/375342 consumed_samples: 83660800 total_loss: 3.873 time: 0.3395 s/iter data_time: 0.2083 s/iter total_throughput: 3016.25 samples/s lr: 8.89e-04 [09/22 11:04:52] lb.utils.events INFO: eta: 12:49:12 iteration: 81799/375342 consumed_samples: 83763200 total_loss: 3.874 time: 0.3395 s/iter data_time: 0.2117 s/iter total_throughput: 3016.24 samples/s lr: 8.88e-04 [09/22 11:05:26] lb.utils.events INFO: eta: 12:48:34 iteration: 81899/375342 consumed_samples: 83865600 total_loss: 3.885 time: 0.3395 s/iter data_time: 0.2169 s/iter total_throughput: 3016.22 samples/s lr: 8.88e-04 [09/22 11:06:01] lb.utils.events INFO: eta: 12:47:35 iteration: 81999/375342 consumed_samples: 83968000 total_loss: 3.907 time: 0.3395 s/iter data_time: 0.2126 s/iter total_throughput: 3016.17 samples/s lr: 8.88e-04 [09/22 11:06:36] lb.utils.events INFO: eta: 12:47:32 iteration: 82099/375342 consumed_samples: 84070400 total_loss: 3.887 time: 0.3395 s/iter data_time: 0.2320 s/iter total_throughput: 3016.06 samples/s lr: 8.88e-04 [09/22 11:07:10] lb.utils.events INFO: eta: 12:47:56 iteration: 82199/375342 consumed_samples: 84172800 total_loss: 3.883 time: 0.3395 s/iter data_time: 0.2165 s/iter total_throughput: 3016.04 samples/s lr: 8.87e-04 [09/22 11:07:44] lb.utils.events INFO: eta: 12:46:31 iteration: 82299/375342 consumed_samples: 84275200 total_loss: 3.894 time: 0.3395 s/iter data_time: 0.2145 s/iter total_throughput: 3016.05 samples/s lr: 8.87e-04 [09/22 11:08:17] lb.utils.events INFO: eta: 12:46:39 iteration: 82399/375342 consumed_samples: 84377600 total_loss: 3.886 time: 0.3395 s/iter data_time: 0.2124 s/iter total_throughput: 3016.08 samples/s lr: 8.87e-04 [09/22 11:08:52] lb.utils.events INFO: eta: 12:46:29 iteration: 82499/375342 consumed_samples: 84480000 total_loss: 3.896 time: 0.3395 s/iter data_time: 0.2053 s/iter total_throughput: 3016.06 samples/s lr: 8.87e-04 [09/22 11:09:26] lb.utils.events INFO: eta: 12:45:47 iteration: 82599/375342 consumed_samples: 84582400 total_loss: 3.89 time: 0.3395 s/iter data_time: 0.2132 s/iter total_throughput: 3016.02 samples/s lr: 8.86e-04 [09/22 11:10:00] lb.utils.events INFO: eta: 12:43:05 iteration: 82699/375342 consumed_samples: 84684800 total_loss: 3.883 time: 0.3395 s/iter data_time: 0.2108 s/iter total_throughput: 3016.02 samples/s lr: 8.86e-04 [09/22 11:10:34] lb.utils.events INFO: eta: 12:43:42 iteration: 82799/375342 consumed_samples: 84787200 total_loss: 3.899 time: 0.3395 s/iter data_time: 0.2177 s/iter total_throughput: 3016.04 samples/s lr: 8.86e-04 [09/22 11:11:07] lb.utils.events INFO: eta: 12:44:39 iteration: 82899/375342 consumed_samples: 84889600 total_loss: 3.883 time: 0.3395 s/iter data_time: 0.2091 s/iter total_throughput: 3016.09 samples/s lr: 8.86e-04 [09/22 11:11:41] lb.utils.events INFO: eta: 12:45:41 iteration: 82999/375342 consumed_samples: 84992000 total_loss: 3.898 time: 0.3395 s/iter data_time: 0.2211 s/iter total_throughput: 3016.15 samples/s lr: 8.85e-04 [09/22 11:12:15] lb.utils.events INFO: eta: 12:45:30 iteration: 83099/375342 consumed_samples: 85094400 total_loss: 3.908 time: 0.3395 s/iter data_time: 0.2114 s/iter total_throughput: 3016.14 samples/s lr: 8.85e-04 [09/22 11:12:48] lb.utils.events INFO: eta: 12:45:14 iteration: 83199/375342 consumed_samples: 85196800 total_loss: 3.896 time: 0.3395 s/iter data_time: 0.2015 s/iter total_throughput: 3016.15 samples/s lr: 8.85e-04 [09/22 11:13:23] lb.utils.events INFO: eta: 12:44:58 iteration: 83299/375342 consumed_samples: 85299200 total_loss: 3.89 time: 0.3395 s/iter data_time: 0.2139 s/iter total_throughput: 3016.13 samples/s lr: 8.84e-04 [09/22 11:13:56] lb.utils.events INFO: eta: 12:45:00 iteration: 83399/375342 consumed_samples: 85401600 total_loss: 3.908 time: 0.3395 s/iter data_time: 0.2136 s/iter total_throughput: 3016.23 samples/s lr: 8.84e-04 [09/22 11:14:29] lb.utils.events INFO: eta: 12:45:03 iteration: 83499/375342 consumed_samples: 85504000 total_loss: 3.914 time: 0.3395 s/iter data_time: 0.2041 s/iter total_throughput: 3016.28 samples/s lr: 8.84e-04 [09/22 11:15:03] lb.utils.events INFO: eta: 12:44:11 iteration: 83599/375342 consumed_samples: 85606400 total_loss: 3.902 time: 0.3395 s/iter data_time: 0.2284 s/iter total_throughput: 3016.31 samples/s lr: 8.84e-04 [09/22 11:15:36] lb.utils.events INFO: eta: 12:45:33 iteration: 83699/375342 consumed_samples: 85708800 total_loss: 3.884 time: 0.3395 s/iter data_time: 0.1992 s/iter total_throughput: 3016.34 samples/s lr: 8.83e-04 [09/22 11:16:10] lb.utils.events INFO: eta: 12:46:27 iteration: 83799/375342 consumed_samples: 85811200 total_loss: 3.877 time: 0.3395 s/iter data_time: 0.2158 s/iter total_throughput: 3016.37 samples/s lr: 8.83e-04 [09/22 11:16:44] lb.utils.events INFO: eta: 12:46:11 iteration: 83899/375342 consumed_samples: 85913600 total_loss: 3.86 time: 0.3395 s/iter data_time: 0.2083 s/iter total_throughput: 3016.39 samples/s lr: 8.83e-04 [09/22 11:17:18] lb.utils.events INFO: eta: 12:45:00 iteration: 83999/375342 consumed_samples: 86016000 total_loss: 3.852 time: 0.3395 s/iter data_time: 0.2067 s/iter total_throughput: 3016.42 samples/s lr: 8.83e-04 [09/22 11:17:51] lb.utils.events INFO: eta: 12:45:44 iteration: 84099/375342 consumed_samples: 86118400 total_loss: 3.867 time: 0.3395 s/iter data_time: 0.2204 s/iter total_throughput: 3016.45 samples/s lr: 8.82e-04 [09/22 11:18:26] lb.utils.events INFO: eta: 12:44:18 iteration: 84199/375342 consumed_samples: 86220800 total_loss: 3.879 time: 0.3395 s/iter data_time: 0.2419 s/iter total_throughput: 3016.32 samples/s lr: 8.82e-04 [09/22 11:19:01] lb.utils.events INFO: eta: 12:44:00 iteration: 84299/375342 consumed_samples: 86323200 total_loss: 3.868 time: 0.3395 s/iter data_time: 0.2150 s/iter total_throughput: 3016.27 samples/s lr: 8.82e-04 [09/22 11:19:35] lb.utils.events INFO: eta: 12:44:16 iteration: 84399/375342 consumed_samples: 86425600 total_loss: 3.886 time: 0.3395 s/iter data_time: 0.2152 s/iter total_throughput: 3016.26 samples/s lr: 8.82e-04 [09/22 11:20:10] lb.utils.events INFO: eta: 12:43:32 iteration: 84499/375342 consumed_samples: 86528000 total_loss: 3.875 time: 0.3395 s/iter data_time: 0.2152 s/iter total_throughput: 3016.19 samples/s lr: 8.81e-04 [09/22 11:20:46] lb.utils.events INFO: eta: 12:42:22 iteration: 84599/375342 consumed_samples: 86630400 total_loss: 3.863 time: 0.3395 s/iter data_time: 0.2394 s/iter total_throughput: 3015.96 samples/s lr: 8.81e-04 [09/22 11:21:21] lb.utils.events INFO: eta: 12:41:30 iteration: 84699/375342 consumed_samples: 86732800 total_loss: 3.888 time: 0.3395 s/iter data_time: 0.2195 s/iter total_throughput: 3015.85 samples/s lr: 8.81e-04 [09/22 11:21:56] lb.utils.events INFO: eta: 12:37:22 iteration: 84799/375342 consumed_samples: 86835200 total_loss: 3.868 time: 0.3396 s/iter data_time: 0.2306 s/iter total_throughput: 3015.71 samples/s lr: 8.80e-04 [09/22 11:22:31] lb.utils.events INFO: eta: 12:34:44 iteration: 84899/375342 consumed_samples: 86937600 total_loss: 3.842 time: 0.3396 s/iter data_time: 0.2375 s/iter total_throughput: 3015.59 samples/s lr: 8.80e-04 [09/22 11:23:06] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0084999 [09/22 11:23:07] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/22 11:23:07] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/22 11:23:11] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0899 s/iter. Inference: 0.1656 s/iter. Eval: 0.0021 s/iter. Total: 0.2576 s/iter. ETA=0:00:09 [09/22 11:23:16] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1102 s/iter. Inference: 0.1836 s/iter. Eval: 0.0021 s/iter. Total: 0.2960 s/iter. ETA=0:00:05 [09/22 11:23:22] lb.evaluation.evaluator INFO: Inference done 50000/50000. Dataloading: 0.1051 s/iter. Inference: 0.1760 s/iter. Eval: 0.0021 s/iter. Total: 0.2833 s/iter. ETA=0:00:00 [09/22 11:23:22] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/22 11:23:22] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.467986 (0.000249 s / iter per device, on 8 devices) [09/22 11:23:22] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000155 s / iter per device, on 8 devices) [09/22 11:23:22] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/22 11:23:22] lb.evaluation.utils INFO: copypaste: Acc@1=69.91000000000001 [09/22 11:23:22] lb.evaluation.utils INFO: copypaste: Acc@5=89.786 [09/22 11:23:22] lb.engine.hooks INFO: Not saving as latest eval score for Acc@1 is 69.91000, not better than best score 70.33600 @ iteration 79999. [09/22 11:23:22] lb.utils.events INFO: eta: 12:33:58 iteration: 84999/375342 consumed_samples: 87040000 total_loss: 3.866 time: 0.3396 s/iter data_time: 0.2247 s/iter total_throughput: 3015.51 samples/s lr: 8.80e-04 [09/22 11:23:53] lb.utils.events INFO: eta: 12:35:37 iteration: 85099/375342 consumed_samples: 87142400 total_loss: 3.876 time: 0.3395 s/iter data_time: 0.2458 s/iter total_throughput: 3015.76 samples/s lr: 8.80e-04 [09/22 11:24:27] lb.utils.events INFO: eta: 12:46:10 iteration: 85199/375342 consumed_samples: 87244800 total_loss: 3.849 time: 0.3395 s/iter data_time: 0.2337 s/iter total_throughput: 3015.77 samples/s lr: 8.79e-04 [09/22 11:25:01] lb.utils.events INFO: eta: 12:46:54 iteration: 85299/375342 consumed_samples: 87347200 total_loss: 3.848 time: 0.3395 s/iter data_time: 0.2069 s/iter total_throughput: 3015.80 samples/s lr: 8.79e-04 [09/22 11:25:36] lb.utils.events INFO: eta: 12:45:24 iteration: 85399/375342 consumed_samples: 87449600 total_loss: 3.863 time: 0.3396 s/iter data_time: 0.2304 s/iter total_throughput: 3015.65 samples/s lr: 8.79e-04 [09/22 11:26:11] lb.utils.events INFO: eta: 12:44:13 iteration: 85499/375342 consumed_samples: 87552000 total_loss: 3.864 time: 0.3396 s/iter data_time: 0.2183 s/iter total_throughput: 3015.57 samples/s lr: 8.79e-04 [09/22 11:26:45] lb.utils.events INFO: eta: 12:44:53 iteration: 85599/375342 consumed_samples: 87654400 total_loss: 3.875 time: 0.3396 s/iter data_time: 0.2134 s/iter total_throughput: 3015.53 samples/s lr: 8.78e-04 [09/22 11:27:19] lb.utils.events INFO: eta: 12:44:53 iteration: 85699/375342 consumed_samples: 87756800 total_loss: 3.903 time: 0.3396 s/iter data_time: 0.2133 s/iter total_throughput: 3015.50 samples/s lr: 8.78e-04 [09/22 11:27:54] lb.utils.events INFO: eta: 12:44:41 iteration: 85799/375342 consumed_samples: 87859200 total_loss: 3.885 time: 0.3396 s/iter data_time: 0.2154 s/iter total_throughput: 3015.40 samples/s lr: 8.78e-04 [09/22 11:28:29] lb.utils.events INFO: eta: 12:44:41 iteration: 85899/375342 consumed_samples: 87961600 total_loss: 3.861 time: 0.3396 s/iter data_time: 0.2200 s/iter total_throughput: 3015.32 samples/s lr: 8.77e-04 [09/22 11:29:04] lb.utils.events INFO: eta: 12:43:54 iteration: 85999/375342 consumed_samples: 88064000 total_loss: 3.859 time: 0.3396 s/iter data_time: 0.2257 s/iter total_throughput: 3015.18 samples/s lr: 8.77e-04 [09/22 11:29:39] lb.utils.events INFO: eta: 12:41:42 iteration: 86099/375342 consumed_samples: 88166400 total_loss: 3.845 time: 0.3396 s/iter data_time: 0.2076 s/iter total_throughput: 3015.14 samples/s lr: 8.77e-04 [09/22 11:30:13] lb.utils.events INFO: eta: 12:34:13 iteration: 86199/375342 consumed_samples: 88268800 total_loss: 3.854 time: 0.3396 s/iter data_time: 0.2172 s/iter total_throughput: 3015.12 samples/s lr: 8.77e-04 [09/22 11:30:47] lb.utils.events INFO: eta: 12:32:33 iteration: 86299/375342 consumed_samples: 88371200 total_loss: 3.865 time: 0.3396 s/iter data_time: 0.2216 s/iter total_throughput: 3015.07 samples/s lr: 8.76e-04 [09/22 11:31:22] lb.utils.events INFO: eta: 12:32:48 iteration: 86399/375342 consumed_samples: 88473600 total_loss: 3.866 time: 0.3396 s/iter data_time: 0.2129 s/iter total_throughput: 3015.01 samples/s lr: 8.76e-04 [09/22 11:31:57] lb.utils.events INFO: eta: 12:32:32 iteration: 86499/375342 consumed_samples: 88576000 total_loss: 3.846 time: 0.3396 s/iter data_time: 0.2327 s/iter total_throughput: 3014.90 samples/s lr: 8.76e-04 [09/22 11:32:32] lb.utils.events INFO: eta: 12:32:31 iteration: 86599/375342 consumed_samples: 88678400 total_loss: 3.846 time: 0.3397 s/iter data_time: 0.2336 s/iter total_throughput: 3014.83 samples/s lr: 8.76e-04 [09/22 11:33:06] lb.utils.events INFO: eta: 12:31:21 iteration: 86699/375342 consumed_samples: 88780800 total_loss: 3.868 time: 0.3397 s/iter data_time: 0.2256 s/iter total_throughput: 3014.77 samples/s lr: 8.75e-04 [09/22 11:33:41] lb.utils.events INFO: eta: 12:31:47 iteration: 86799/375342 consumed_samples: 88883200 total_loss: 3.871 time: 0.3397 s/iter data_time: 0.2264 s/iter total_throughput: 3014.70 samples/s lr: 8.75e-04 [09/22 11:34:15] lb.utils.events INFO: eta: 12:31:36 iteration: 86899/375342 consumed_samples: 88985600 total_loss: 3.846 time: 0.3397 s/iter data_time: 0.2120 s/iter total_throughput: 3014.65 samples/s lr: 8.75e-04 [09/22 11:34:50] lb.utils.events INFO: eta: 12:32:09 iteration: 86999/375342 consumed_samples: 89088000 total_loss: 3.858 time: 0.3397 s/iter data_time: 0.2372 s/iter total_throughput: 3014.59 samples/s lr: 8.74e-04 [09/22 11:35:24] lb.utils.events INFO: eta: 12:33:14 iteration: 87099/375342 consumed_samples: 89190400 total_loss: 3.871 time: 0.3397 s/iter data_time: 0.2098 s/iter total_throughput: 3014.55 samples/s lr: 8.74e-04 [09/22 11:35:59] lb.utils.events INFO: eta: 12:32:28 iteration: 87199/375342 consumed_samples: 89292800 total_loss: 3.846 time: 0.3397 s/iter data_time: 0.2314 s/iter total_throughput: 3014.47 samples/s lr: 8.74e-04 [09/22 11:36:34] lb.utils.events INFO: eta: 12:31:18 iteration: 87299/375342 consumed_samples: 89395200 total_loss: 3.854 time: 0.3397 s/iter data_time: 0.2273 s/iter total_throughput: 3014.40 samples/s lr: 8.74e-04 [09/22 11:37:08] lb.utils.events INFO: eta: 12:30:33 iteration: 87399/375342 consumed_samples: 89497600 total_loss: 3.864 time: 0.3397 s/iter data_time: 0.2175 s/iter total_throughput: 3014.33 samples/s lr: 8.73e-04 [09/22 11:37:44] lb.utils.events INFO: eta: 12:30:25 iteration: 87499/375342 consumed_samples: 89600000 total_loss: 3.866 time: 0.3397 s/iter data_time: 0.2350 s/iter total_throughput: 3014.20 samples/s lr: 8.73e-04 [09/22 11:38:18] lb.utils.events INFO: eta: 12:30:07 iteration: 87599/375342 consumed_samples: 89702400 total_loss: 3.87 time: 0.3397 s/iter data_time: 0.2264 s/iter total_throughput: 3014.14 samples/s lr: 8.73e-04 [09/22 11:38:53] lb.utils.events INFO: eta: 12:29:22 iteration: 87699/375342 consumed_samples: 89804800 total_loss: 3.832 time: 0.3397 s/iter data_time: 0.2257 s/iter total_throughput: 3014.02 samples/s lr: 8.73e-04 [09/22 11:39:28] lb.utils.events INFO: eta: 12:29:26 iteration: 87799/375342 consumed_samples: 89907200 total_loss: 3.832 time: 0.3398 s/iter data_time: 0.2145 s/iter total_throughput: 3013.97 samples/s lr: 8.72e-04 [09/22 11:40:02] lb.utils.events INFO: eta: 12:29:07 iteration: 87899/375342 consumed_samples: 90009600 total_loss: 3.861 time: 0.3398 s/iter data_time: 0.2131 s/iter total_throughput: 3013.91 samples/s lr: 8.72e-04 [09/22 11:40:37] lb.utils.events INFO: eta: 12:28:04 iteration: 87999/375342 consumed_samples: 90112000 total_loss: 3.864 time: 0.3398 s/iter data_time: 0.2082 s/iter total_throughput: 3013.84 samples/s lr: 8.72e-04 [09/22 11:41:12] lb.utils.events INFO: eta: 12:24:03 iteration: 88099/375342 consumed_samples: 90214400 total_loss: 3.855 time: 0.3398 s/iter data_time: 0.2194 s/iter total_throughput: 3013.75 samples/s lr: 8.71e-04 [09/22 11:41:46] lb.utils.events INFO: eta: 12:25:07 iteration: 88199/375342 consumed_samples: 90316800 total_loss: 3.85 time: 0.3398 s/iter data_time: 0.1968 s/iter total_throughput: 3013.73 samples/s lr: 8.71e-04 [09/22 11:42:21] lb.utils.events INFO: eta: 12:26:11 iteration: 88299/375342 consumed_samples: 90419200 total_loss: 3.878 time: 0.3398 s/iter data_time: 0.2112 s/iter total_throughput: 3013.67 samples/s lr: 8.71e-04 [09/22 11:42:55] lb.utils.events INFO: eta: 12:26:32 iteration: 88399/375342 consumed_samples: 90521600 total_loss: 3.874 time: 0.3398 s/iter data_time: 0.2248 s/iter total_throughput: 3013.63 samples/s lr: 8.71e-04 [09/22 11:43:30] lb.utils.events INFO: eta: 12:25:52 iteration: 88499/375342 consumed_samples: 90624000 total_loss: 3.843 time: 0.3398 s/iter data_time: 0.2116 s/iter total_throughput: 3013.58 samples/s lr: 8.70e-04 [09/22 11:44:04] lb.utils.events INFO: eta: 12:25:27 iteration: 88599/375342 consumed_samples: 90726400 total_loss: 3.864 time: 0.3398 s/iter data_time: 0.2238 s/iter total_throughput: 3013.53 samples/s lr: 8.70e-04 [09/22 11:44:39] lb.utils.events INFO: eta: 12:26:24 iteration: 88699/375342 consumed_samples: 90828800 total_loss: 3.869 time: 0.3398 s/iter data_time: 0.2281 s/iter total_throughput: 3013.45 samples/s lr: 8.70e-04 [09/22 11:45:13] lb.utils.events INFO: eta: 12:25:17 iteration: 88799/375342 consumed_samples: 90931200 total_loss: 3.835 time: 0.3398 s/iter data_time: 0.2174 s/iter total_throughput: 3013.45 samples/s lr: 8.69e-04 [09/22 11:45:47] lb.utils.events INFO: eta: 12:26:45 iteration: 88899/375342 consumed_samples: 91033600 total_loss: 3.82 time: 0.3398 s/iter data_time: 0.2301 s/iter total_throughput: 3013.43 samples/s lr: 8.69e-04 [09/22 11:46:22] lb.utils.events INFO: eta: 12:26:00 iteration: 88999/375342 consumed_samples: 91136000 total_loss: 3.835 time: 0.3398 s/iter data_time: 0.2066 s/iter total_throughput: 3013.39 samples/s lr: 8.69e-04 [09/22 11:46:56] lb.utils.events INFO: eta: 12:32:18 iteration: 89099/375342 consumed_samples: 91238400 total_loss: 3.831 time: 0.3398 s/iter data_time: 0.2191 s/iter total_throughput: 3013.35 samples/s lr: 8.69e-04 [09/22 11:47:30] lb.utils.events INFO: eta: 12:26:38 iteration: 89199/375342 consumed_samples: 91340800 total_loss: 3.827 time: 0.3398 s/iter data_time: 0.2192 s/iter total_throughput: 3013.29 samples/s lr: 8.68e-04 [09/22 11:48:05] lb.utils.events INFO: eta: 12:29:03 iteration: 89299/375342 consumed_samples: 91443200 total_loss: 3.842 time: 0.3398 s/iter data_time: 0.2107 s/iter total_throughput: 3013.25 samples/s lr: 8.68e-04 [09/22 11:48:39] lb.utils.events INFO: eta: 12:27:20 iteration: 89399/375342 consumed_samples: 91545600 total_loss: 3.854 time: 0.3398 s/iter data_time: 0.2239 s/iter total_throughput: 3013.19 samples/s lr: 8.68e-04 [09/22 11:49:14] lb.utils.events INFO: eta: 12:25:46 iteration: 89499/375342 consumed_samples: 91648000 total_loss: 3.859 time: 0.3398 s/iter data_time: 0.2166 s/iter total_throughput: 3013.15 samples/s lr: 8.67e-04 [09/22 11:49:48] lb.utils.events INFO: eta: 12:25:50 iteration: 89599/375342 consumed_samples: 91750400 total_loss: 3.88 time: 0.3398 s/iter data_time: 0.2083 s/iter total_throughput: 3013.12 samples/s lr: 8.67e-04 [09/22 11:50:22] lb.utils.events INFO: eta: 12:25:36 iteration: 89699/375342 consumed_samples: 91852800 total_loss: 3.861 time: 0.3398 s/iter data_time: 0.2189 s/iter total_throughput: 3013.12 samples/s lr: 8.67e-04 [09/22 11:50:56] lb.utils.events INFO: eta: 12:26:30 iteration: 89799/375342 consumed_samples: 91955200 total_loss: 3.854 time: 0.3398 s/iter data_time: 0.2145 s/iter total_throughput: 3013.11 samples/s lr: 8.67e-04 [09/22 11:51:30] lb.utils.events INFO: eta: 12:25:35 iteration: 89899/375342 consumed_samples: 92057600 total_loss: 3.857 time: 0.3398 s/iter data_time: 0.2162 s/iter total_throughput: 3013.15 samples/s lr: 8.66e-04 [09/22 11:52:04] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0089999 [09/22 11:52:05] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/22 11:52:05] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/22 11:52:09] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0915 s/iter. Inference: 0.1582 s/iter. Eval: 0.0021 s/iter. Total: 0.2518 s/iter. ETA=0:00:09 [09/22 11:52:14] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1090 s/iter. Inference: 0.1841 s/iter. Eval: 0.0019 s/iter. Total: 0.2951 s/iter. ETA=0:00:05 [09/22 11:52:19] lb.evaluation.evaluator INFO: Inference done 49152/50000. Dataloading: 0.0912 s/iter. Inference: 0.1880 s/iter. Eval: 0.0020 s/iter. Total: 0.2813 s/iter. ETA=0:00:00 [09/22 11:52:20] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/22 11:52:20] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.557328 (0.000251 s / iter per device, on 8 devices) [09/22 11:52:20] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:08 (0.000169 s / iter per device, on 8 devices) [09/22 11:52:20] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/22 11:52:20] lb.evaluation.utils INFO: copypaste: Acc@1=70.77799999999999 [09/22 11:52:20] lb.evaluation.utils INFO: copypaste: Acc@5=90.18599999999999 [09/22 11:52:20] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 70.77800, better than last best score 70.33600 @ iteration 79999. [09/22 11:52:20] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/22 11:52:21] lb.utils.events INFO: eta: 12:27:02 iteration: 89999/375342 consumed_samples: 92160000 total_loss: 3.874 time: 0.3398 s/iter data_time: 0.2244 s/iter total_throughput: 3013.14 samples/s lr: 8.66e-04 [09/22 11:52:53] lb.utils.events INFO: eta: 12:28:36 iteration: 90099/375342 consumed_samples: 92262400 total_loss: 3.864 time: 0.3398 s/iter data_time: 0.2372 s/iter total_throughput: 3013.32 samples/s lr: 8.66e-04 [09/22 11:53:27] lb.utils.events INFO: eta: 12:29:10 iteration: 90199/375342 consumed_samples: 92364800 total_loss: 3.833 time: 0.3398 s/iter data_time: 0.2125 s/iter total_throughput: 3013.27 samples/s lr: 8.66e-04 [09/22 11:54:02] lb.utils.events INFO: eta: 12:26:25 iteration: 90299/375342 consumed_samples: 92467200 total_loss: 3.842 time: 0.3398 s/iter data_time: 0.2147 s/iter total_throughput: 3013.22 samples/s lr: 8.65e-04 [09/22 11:54:36] lb.utils.events INFO: eta: 12:26:40 iteration: 90399/375342 consumed_samples: 92569600 total_loss: 3.84 time: 0.3398 s/iter data_time: 0.2234 s/iter total_throughput: 3013.17 samples/s lr: 8.65e-04 [09/22 11:55:10] lb.utils.events INFO: eta: 12:30:38 iteration: 90499/375342 consumed_samples: 92672000 total_loss: 3.832 time: 0.3398 s/iter data_time: 0.2048 s/iter total_throughput: 3013.14 samples/s lr: 8.65e-04 [09/22 11:55:45] lb.utils.events INFO: eta: 12:30:11 iteration: 90599/375342 consumed_samples: 92774400 total_loss: 3.829 time: 0.3399 s/iter data_time: 0.2268 s/iter total_throughput: 3013.08 samples/s lr: 8.64e-04 [09/22 11:56:19] lb.utils.events INFO: eta: 12:29:35 iteration: 90699/375342 consumed_samples: 92876800 total_loss: 3.832 time: 0.3399 s/iter data_time: 0.2087 s/iter total_throughput: 3013.06 samples/s lr: 8.64e-04 [09/22 11:56:54] lb.utils.events INFO: eta: 12:29:31 iteration: 90799/375342 consumed_samples: 92979200 total_loss: 3.863 time: 0.3399 s/iter data_time: 0.2165 s/iter total_throughput: 3013.01 samples/s lr: 8.64e-04 [09/22 11:57:28] lb.utils.events INFO: eta: 12:30:03 iteration: 90899/375342 consumed_samples: 93081600 total_loss: 3.859 time: 0.3399 s/iter data_time: 0.2176 s/iter total_throughput: 3013.01 samples/s lr: 8.64e-04 [09/22 11:58:02] lb.utils.events INFO: eta: 12:31:03 iteration: 90999/375342 consumed_samples: 93184000 total_loss: 3.845 time: 0.3399 s/iter data_time: 0.2205 s/iter total_throughput: 3013.00 samples/s lr: 8.63e-04 [09/22 11:58:37] lb.utils.events INFO: eta: 12:27:56 iteration: 91099/375342 consumed_samples: 93286400 total_loss: 3.857 time: 0.3399 s/iter data_time: 0.2174 s/iter total_throughput: 3012.94 samples/s lr: 8.63e-04 [09/22 11:59:11] lb.utils.events INFO: eta: 12:25:31 iteration: 91199/375342 consumed_samples: 93388800 total_loss: 3.865 time: 0.3399 s/iter data_time: 0.2181 s/iter total_throughput: 3012.92 samples/s lr: 8.63e-04 [09/22 11:59:45] lb.utils.events INFO: eta: 12:26:22 iteration: 91299/375342 consumed_samples: 93491200 total_loss: 3.86 time: 0.3399 s/iter data_time: 0.2110 s/iter total_throughput: 3012.86 samples/s lr: 8.62e-04 [09/22 12:00:20] lb.utils.events INFO: eta: 12:25:25 iteration: 91399/375342 consumed_samples: 93593600 total_loss: 3.84 time: 0.3399 s/iter data_time: 0.2163 s/iter total_throughput: 3012.81 samples/s lr: 8.62e-04 [09/22 12:00:54] lb.utils.events INFO: eta: 12:22:39 iteration: 91499/375342 consumed_samples: 93696000 total_loss: 3.855 time: 0.3399 s/iter data_time: 0.2168 s/iter total_throughput: 3012.75 samples/s lr: 8.62e-04 [09/22 12:01:29] lb.utils.events INFO: eta: 12:20:58 iteration: 91599/375342 consumed_samples: 93798400 total_loss: 3.855 time: 0.3399 s/iter data_time: 0.2183 s/iter total_throughput: 3012.74 samples/s lr: 8.62e-04 [09/22 12:02:03] lb.utils.events INFO: eta: 12:21:42 iteration: 91699/375342 consumed_samples: 93900800 total_loss: 3.833 time: 0.3399 s/iter data_time: 0.2264 s/iter total_throughput: 3012.72 samples/s lr: 8.61e-04 [09/22 12:02:37] lb.utils.events INFO: eta: 12:20:11 iteration: 91799/375342 consumed_samples: 94003200 total_loss: 3.838 time: 0.3399 s/iter data_time: 0.2233 s/iter total_throughput: 3012.69 samples/s lr: 8.61e-04 [09/22 12:03:11] lb.utils.events INFO: eta: 12:19:55 iteration: 91899/375342 consumed_samples: 94105600 total_loss: 3.848 time: 0.3399 s/iter data_time: 0.2067 s/iter total_throughput: 3012.72 samples/s lr: 8.61e-04 [09/22 12:03:45] lb.utils.events INFO: eta: 12:18:50 iteration: 91999/375342 consumed_samples: 94208000 total_loss: 3.848 time: 0.3399 s/iter data_time: 0.2144 s/iter total_throughput: 3012.71 samples/s lr: 8.60e-04 [09/22 12:04:19] lb.utils.events INFO: eta: 12:20:19 iteration: 92099/375342 consumed_samples: 94310400 total_loss: 3.852 time: 0.3399 s/iter data_time: 0.2160 s/iter total_throughput: 3012.74 samples/s lr: 8.60e-04 [09/22 12:04:53] lb.utils.events INFO: eta: 12:19:22 iteration: 92199/375342 consumed_samples: 94412800 total_loss: 3.859 time: 0.3399 s/iter data_time: 0.2037 s/iter total_throughput: 3012.74 samples/s lr: 8.60e-04 [09/22 12:05:27] lb.utils.events INFO: eta: 12:19:57 iteration: 92299/375342 consumed_samples: 94515200 total_loss: 3.848 time: 0.3399 s/iter data_time: 0.2253 s/iter total_throughput: 3012.74 samples/s lr: 8.59e-04 [09/22 12:06:01] lb.utils.events INFO: eta: 12:20:32 iteration: 92399/375342 consumed_samples: 94617600 total_loss: 3.841 time: 0.3399 s/iter data_time: 0.2106 s/iter total_throughput: 3012.72 samples/s lr: 8.59e-04 [09/22 12:06:35] lb.utils.events INFO: eta: 12:20:53 iteration: 92499/375342 consumed_samples: 94720000 total_loss: 3.849 time: 0.3399 s/iter data_time: 0.2123 s/iter total_throughput: 3012.69 samples/s lr: 8.59e-04 [09/22 12:07:10] lb.utils.events INFO: eta: 12:21:07 iteration: 92599/375342 consumed_samples: 94822400 total_loss: 3.836 time: 0.3399 s/iter data_time: 0.2098 s/iter total_throughput: 3012.65 samples/s lr: 8.59e-04 [09/22 12:07:44] lb.utils.events INFO: eta: 12:21:02 iteration: 92699/375342 consumed_samples: 94924800 total_loss: 3.82 time: 0.3399 s/iter data_time: 0.2152 s/iter total_throughput: 3012.60 samples/s lr: 8.58e-04 [09/22 12:08:18] lb.utils.events INFO: eta: 12:22:47 iteration: 92799/375342 consumed_samples: 95027200 total_loss: 3.844 time: 0.3399 s/iter data_time: 0.2076 s/iter total_throughput: 3012.61 samples/s lr: 8.58e-04 [09/22 12:08:52] lb.utils.events INFO: eta: 12:21:38 iteration: 92899/375342 consumed_samples: 95129600 total_loss: 3.861 time: 0.3399 s/iter data_time: 0.2195 s/iter total_throughput: 3012.58 samples/s lr: 8.58e-04 [09/22 12:09:26] lb.utils.events INFO: eta: 12:20:23 iteration: 92999/375342 consumed_samples: 95232000 total_loss: 3.855 time: 0.3399 s/iter data_time: 0.2073 s/iter total_throughput: 3012.57 samples/s lr: 8.57e-04 [09/22 12:10:00] lb.utils.events INFO: eta: 12:18:07 iteration: 93099/375342 consumed_samples: 95334400 total_loss: 3.83 time: 0.3399 s/iter data_time: 0.2334 s/iter total_throughput: 3012.56 samples/s lr: 8.57e-04 [09/22 12:10:35] lb.utils.events INFO: eta: 12:18:29 iteration: 93199/375342 consumed_samples: 95436800 total_loss: 3.824 time: 0.3399 s/iter data_time: 0.2094 s/iter total_throughput: 3012.54 samples/s lr: 8.57e-04 [09/22 12:11:09] lb.utils.events INFO: eta: 12:18:14 iteration: 93299/375342 consumed_samples: 95539200 total_loss: 3.845 time: 0.3399 s/iter data_time: 0.2201 s/iter total_throughput: 3012.52 samples/s lr: 8.57e-04 [09/22 12:11:43] lb.utils.events INFO: eta: 12:17:42 iteration: 93399/375342 consumed_samples: 95641600 total_loss: 3.854 time: 0.3399 s/iter data_time: 0.2172 s/iter total_throughput: 3012.50 samples/s lr: 8.56e-04 [09/22 12:12:18] lb.utils.events INFO: eta: 12:16:58 iteration: 93499/375342 consumed_samples: 95744000 total_loss: 3.856 time: 0.3399 s/iter data_time: 0.2286 s/iter total_throughput: 3012.43 samples/s lr: 8.56e-04 [09/22 12:12:52] lb.utils.events INFO: eta: 12:14:38 iteration: 93599/375342 consumed_samples: 95846400 total_loss: 3.851 time: 0.3399 s/iter data_time: 0.2143 s/iter total_throughput: 3012.40 samples/s lr: 8.56e-04 [09/22 12:13:26] lb.utils.events INFO: eta: 12:14:30 iteration: 93699/375342 consumed_samples: 95948800 total_loss: 3.842 time: 0.3399 s/iter data_time: 0.2205 s/iter total_throughput: 3012.38 samples/s lr: 8.55e-04 [09/22 12:14:01] lb.utils.events INFO: eta: 12:11:36 iteration: 93799/375342 consumed_samples: 96051200 total_loss: 3.846 time: 0.3399 s/iter data_time: 0.2180 s/iter total_throughput: 3012.32 samples/s lr: 8.55e-04 [09/22 12:14:35] lb.utils.events INFO: eta: 12:13:53 iteration: 93899/375342 consumed_samples: 96153600 total_loss: 3.848 time: 0.3399 s/iter data_time: 0.2106 s/iter total_throughput: 3012.32 samples/s lr: 8.55e-04 [09/22 12:15:09] lb.utils.events INFO: eta: 12:15:51 iteration: 93999/375342 consumed_samples: 96256000 total_loss: 3.841 time: 0.3399 s/iter data_time: 0.2241 s/iter total_throughput: 3012.30 samples/s lr: 8.55e-04 [09/22 12:15:43] lb.utils.events INFO: eta: 12:15:49 iteration: 94099/375342 consumed_samples: 96358400 total_loss: 3.847 time: 0.3399 s/iter data_time: 0.2154 s/iter total_throughput: 3012.30 samples/s lr: 8.54e-04 [09/22 12:16:17] lb.utils.events INFO: eta: 12:15:45 iteration: 94199/375342 consumed_samples: 96460800 total_loss: 3.85 time: 0.3399 s/iter data_time: 0.2120 s/iter total_throughput: 3012.33 samples/s lr: 8.54e-04 [09/22 12:16:51] lb.utils.events INFO: eta: 12:15:04 iteration: 94299/375342 consumed_samples: 96563200 total_loss: 3.822 time: 0.3399 s/iter data_time: 0.2136 s/iter total_throughput: 3012.34 samples/s lr: 8.54e-04 [09/22 12:17:25] lb.utils.events INFO: eta: 12:15:01 iteration: 94399/375342 consumed_samples: 96665600 total_loss: 3.829 time: 0.3399 s/iter data_time: 0.2084 s/iter total_throughput: 3012.32 samples/s lr: 8.53e-04 [09/22 12:17:58] lb.utils.events INFO: eta: 12:17:11 iteration: 94499/375342 consumed_samples: 96768000 total_loss: 3.844 time: 0.3399 s/iter data_time: 0.1992 s/iter total_throughput: 3012.37 samples/s lr: 8.53e-04 [09/22 12:18:33] lb.utils.events INFO: eta: 12:16:46 iteration: 94599/375342 consumed_samples: 96870400 total_loss: 3.827 time: 0.3399 s/iter data_time: 0.2139 s/iter total_throughput: 3012.35 samples/s lr: 8.53e-04 [09/22 12:19:06] lb.utils.events INFO: eta: 12:16:01 iteration: 94699/375342 consumed_samples: 96972800 total_loss: 3.832 time: 0.3399 s/iter data_time: 0.2132 s/iter total_throughput: 3012.38 samples/s lr: 8.52e-04 [09/22 12:19:40] lb.utils.events INFO: eta: 12:16:47 iteration: 94799/375342 consumed_samples: 97075200 total_loss: 3.844 time: 0.3399 s/iter data_time: 0.2121 s/iter total_throughput: 3012.43 samples/s lr: 8.52e-04 [09/22 12:20:14] lb.utils.events INFO: eta: 12:15:29 iteration: 94899/375342 consumed_samples: 97177600 total_loss: 3.864 time: 0.3399 s/iter data_time: 0.2043 s/iter total_throughput: 3012.43 samples/s lr: 8.52e-04 [09/22 12:20:48] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0094999 [09/22 12:20:49] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/22 12:20:49] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/22 12:20:53] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0911 s/iter. Inference: 0.1641 s/iter. Eval: 0.0022 s/iter. Total: 0.2575 s/iter. ETA=0:00:09 [09/22 12:20:59] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1227 s/iter. Inference: 0.1702 s/iter. Eval: 0.0020 s/iter. Total: 0.2949 s/iter. ETA=0:00:05 [09/22 12:21:04] lb.evaluation.evaluator INFO: Inference done 50000/50000. Dataloading: 0.1153 s/iter. Inference: 0.1666 s/iter. Eval: 0.0019 s/iter. Total: 0.2838 s/iter. ETA=0:00:00 [09/22 12:21:04] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/22 12:21:04] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.494068 (0.000250 s / iter per device, on 8 devices) [09/22 12:21:04] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000147 s / iter per device, on 8 devices) [09/22 12:21:04] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/22 12:21:04] lb.evaluation.utils INFO: copypaste: Acc@1=70.532 [09/22 12:21:04] lb.evaluation.utils INFO: copypaste: Acc@5=89.962 [09/22 12:21:04] lb.engine.hooks INFO: Not saving as latest eval score for Acc@1 is 70.53200, not better than best score 70.77800 @ iteration 89999. [09/22 12:21:04] lb.utils.events INFO: eta: 12:14:42 iteration: 94999/375342 consumed_samples: 97280000 total_loss: 3.86 time: 0.3399 s/iter data_time: 0.2354 s/iter total_throughput: 3012.39 samples/s lr: 8.52e-04 [09/22 12:21:36] lb.utils.events INFO: eta: 12:17:02 iteration: 95099/375342 consumed_samples: 97382400 total_loss: 3.827 time: 0.3399 s/iter data_time: 0.2332 s/iter total_throughput: 3012.60 samples/s lr: 8.51e-04 [09/22 12:22:10] lb.utils.events INFO: eta: 12:18:08 iteration: 95199/375342 consumed_samples: 97484800 total_loss: 3.815 time: 0.3399 s/iter data_time: 0.1981 s/iter total_throughput: 3012.54 samples/s lr: 8.51e-04 [09/22 12:22:44] lb.utils.events INFO: eta: 12:19:52 iteration: 95299/375342 consumed_samples: 97587200 total_loss: 3.814 time: 0.3399 s/iter data_time: 0.2213 s/iter total_throughput: 3012.59 samples/s lr: 8.51e-04 [09/22 12:23:17] lb.utils.events INFO: eta: 12:19:51 iteration: 95399/375342 consumed_samples: 97689600 total_loss: 3.807 time: 0.3399 s/iter data_time: 0.2016 s/iter total_throughput: 3012.64 samples/s lr: 8.50e-04 [09/22 12:23:52] lb.utils.events INFO: eta: 12:18:14 iteration: 95499/375342 consumed_samples: 97792000 total_loss: 3.832 time: 0.3399 s/iter data_time: 0.2148 s/iter total_throughput: 3012.61 samples/s lr: 8.50e-04 [09/22 12:24:25] lb.utils.events INFO: eta: 12:18:51 iteration: 95599/375342 consumed_samples: 97894400 total_loss: 3.854 time: 0.3399 s/iter data_time: 0.2096 s/iter total_throughput: 3012.64 samples/s lr: 8.50e-04 [09/22 12:24:59] lb.utils.events INFO: eta: 12:19:05 iteration: 95699/375342 consumed_samples: 97996800 total_loss: 3.84 time: 0.3399 s/iter data_time: 0.2223 s/iter total_throughput: 3012.63 samples/s lr: 8.50e-04 [09/22 12:25:33] lb.utils.events INFO: eta: 12:17:35 iteration: 95799/375342 consumed_samples: 98099200 total_loss: 3.82 time: 0.3399 s/iter data_time: 0.2147 s/iter total_throughput: 3012.67 samples/s lr: 8.49e-04 [09/22 12:26:07] lb.utils.events INFO: eta: 12:15:21 iteration: 95899/375342 consumed_samples: 98201600 total_loss: 3.819 time: 0.3399 s/iter data_time: 0.2133 s/iter total_throughput: 3012.69 samples/s lr: 8.49e-04 [09/22 12:26:41] lb.utils.events INFO: eta: 12:14:29 iteration: 95999/375342 consumed_samples: 98304000 total_loss: 3.817 time: 0.3399 s/iter data_time: 0.2258 s/iter total_throughput: 3012.65 samples/s lr: 8.49e-04 [09/22 12:27:15] lb.utils.events INFO: eta: 12:11:58 iteration: 96099/375342 consumed_samples: 98406400 total_loss: 3.824 time: 0.3399 s/iter data_time: 0.2322 s/iter total_throughput: 3012.63 samples/s lr: 8.48e-04 [09/22 12:27:49] lb.utils.events INFO: eta: 12:11:33 iteration: 96199/375342 consumed_samples: 98508800 total_loss: 3.83 time: 0.3399 s/iter data_time: 0.2058 s/iter total_throughput: 3012.67 samples/s lr: 8.48e-04 [09/22 12:28:23] lb.utils.events INFO: eta: 12:09:00 iteration: 96299/375342 consumed_samples: 98611200 total_loss: 3.821 time: 0.3399 s/iter data_time: 0.2117 s/iter total_throughput: 3012.65 samples/s lr: 8.48e-04 [09/22 12:28:57] lb.utils.events INFO: eta: 12:07:36 iteration: 96399/375342 consumed_samples: 98713600 total_loss: 3.827 time: 0.3399 s/iter data_time: 0.2129 s/iter total_throughput: 3012.69 samples/s lr: 8.47e-04 [09/22 12:29:30] lb.utils.events INFO: eta: 12:08:07 iteration: 96499/375342 consumed_samples: 98816000 total_loss: 3.828 time: 0.3399 s/iter data_time: 0.2198 s/iter total_throughput: 3012.72 samples/s lr: 8.47e-04 [09/22 12:30:05] lb.utils.events INFO: eta: 12:07:09 iteration: 96599/375342 consumed_samples: 98918400 total_loss: 3.826 time: 0.3399 s/iter data_time: 0.2185 s/iter total_throughput: 3012.67 samples/s lr: 8.47e-04 [09/22 12:30:39] lb.utils.events INFO: eta: 12:06:13 iteration: 96699/375342 consumed_samples: 99020800 total_loss: 3.833 time: 0.3399 s/iter data_time: 0.2162 s/iter total_throughput: 3012.68 samples/s lr: 8.47e-04 [09/22 12:31:13] lb.utils.events INFO: eta: 12:05:43 iteration: 96799/375342 consumed_samples: 99123200 total_loss: 3.819 time: 0.3399 s/iter data_time: 0.2215 s/iter total_throughput: 3012.67 samples/s lr: 8.46e-04 [09/22 12:31:47] lb.utils.events INFO: eta: 12:04:23 iteration: 96899/375342 consumed_samples: 99225600 total_loss: 3.797 time: 0.3399 s/iter data_time: 0.2197 s/iter total_throughput: 3012.66 samples/s lr: 8.46e-04 [09/22 12:32:21] lb.utils.events INFO: eta: 12:05:37 iteration: 96999/375342 consumed_samples: 99328000 total_loss: 3.824 time: 0.3399 s/iter data_time: 0.2069 s/iter total_throughput: 3012.68 samples/s lr: 8.46e-04 [09/22 12:32:54] lb.utils.events INFO: eta: 12:06:14 iteration: 97099/375342 consumed_samples: 99430400 total_loss: 3.839 time: 0.3399 s/iter data_time: 0.2082 s/iter total_throughput: 3012.72 samples/s lr: 8.45e-04 [09/22 12:33:28] lb.utils.events INFO: eta: 12:05:48 iteration: 97199/375342 consumed_samples: 99532800 total_loss: 3.813 time: 0.3399 s/iter data_time: 0.2083 s/iter total_throughput: 3012.74 samples/s lr: 8.45e-04 [09/22 12:34:02] lb.utils.events INFO: eta: 12:05:42 iteration: 97299/375342 consumed_samples: 99635200 total_loss: 3.82 time: 0.3399 s/iter data_time: 0.2116 s/iter total_throughput: 3012.77 samples/s lr: 8.45e-04 [09/22 12:34:36] lb.utils.events INFO: eta: 12:04:51 iteration: 97399/375342 consumed_samples: 99737600 total_loss: 3.82 time: 0.3399 s/iter data_time: 0.2180 s/iter total_throughput: 3012.77 samples/s lr: 8.44e-04 [09/22 12:35:09] lb.utils.events INFO: eta: 12:04:23 iteration: 97499/375342 consumed_samples: 99840000 total_loss: 3.809 time: 0.3399 s/iter data_time: 0.2039 s/iter total_throughput: 3012.83 samples/s lr: 8.44e-04 [09/22 12:35:43] lb.utils.events INFO: eta: 12:01:55 iteration: 97599/375342 consumed_samples: 99942400 total_loss: 3.812 time: 0.3399 s/iter data_time: 0.2090 s/iter total_throughput: 3012.84 samples/s lr: 8.44e-04 [09/22 12:36:16] lb.utils.events INFO: eta: 12:04:30 iteration: 97699/375342 consumed_samples: 100044800 total_loss: 3.832 time: 0.3399 s/iter data_time: 0.2000 s/iter total_throughput: 3012.92 samples/s lr: 8.44e-04 [09/22 12:36:49] lb.utils.events INFO: eta: 12:04:21 iteration: 97799/375342 consumed_samples: 100147200 total_loss: 3.808 time: 0.3399 s/iter data_time: 0.2070 s/iter total_throughput: 3013.00 samples/s lr: 8.43e-04 [09/22 12:37:22] lb.utils.events INFO: eta: 12:04:56 iteration: 97899/375342 consumed_samples: 100249600 total_loss: 3.796 time: 0.3399 s/iter data_time: 0.2117 s/iter total_throughput: 3013.08 samples/s lr: 8.43e-04 [09/22 12:37:56] lb.utils.events INFO: eta: 12:05:55 iteration: 97999/375342 consumed_samples: 100352000 total_loss: 3.805 time: 0.3398 s/iter data_time: 0.2174 s/iter total_throughput: 3013.15 samples/s lr: 8.43e-04 [09/22 12:38:29] lb.utils.events INFO: eta: 12:05:58 iteration: 98099/375342 consumed_samples: 100454400 total_loss: 3.806 time: 0.3398 s/iter data_time: 0.2158 s/iter total_throughput: 3013.19 samples/s lr: 8.42e-04 [09/22 12:39:03] lb.utils.events INFO: eta: 12:05:42 iteration: 98199/375342 consumed_samples: 100556800 total_loss: 3.847 time: 0.3398 s/iter data_time: 0.2132 s/iter total_throughput: 3013.21 samples/s lr: 8.42e-04 [09/22 12:39:37] lb.utils.events INFO: eta: 12:07:36 iteration: 98299/375342 consumed_samples: 100659200 total_loss: 3.841 time: 0.3398 s/iter data_time: 0.2081 s/iter total_throughput: 3013.22 samples/s lr: 8.42e-04 [09/22 12:40:11] lb.utils.events INFO: eta: 12:08:27 iteration: 98399/375342 consumed_samples: 100761600 total_loss: 3.814 time: 0.3398 s/iter data_time: 0.2078 s/iter total_throughput: 3013.21 samples/s lr: 8.41e-04 [09/22 12:40:45] lb.utils.events INFO: eta: 12:07:44 iteration: 98499/375342 consumed_samples: 100864000 total_loss: 3.81 time: 0.3398 s/iter data_time: 0.2050 s/iter total_throughput: 3013.22 samples/s lr: 8.41e-04 [09/22 12:41:19] lb.utils.events INFO: eta: 12:09:12 iteration: 98599/375342 consumed_samples: 100966400 total_loss: 3.844 time: 0.3398 s/iter data_time: 0.2072 s/iter total_throughput: 3013.21 samples/s lr: 8.41e-04 [09/22 12:41:53] lb.utils.events INFO: eta: 12:09:07 iteration: 98699/375342 consumed_samples: 101068800 total_loss: 3.852 time: 0.3398 s/iter data_time: 0.2167 s/iter total_throughput: 3013.17 samples/s lr: 8.40e-04 [09/22 12:42:27] lb.utils.events INFO: eta: 12:08:12 iteration: 98799/375342 consumed_samples: 101171200 total_loss: 3.838 time: 0.3398 s/iter data_time: 0.2139 s/iter total_throughput: 3013.16 samples/s lr: 8.40e-04 [09/22 12:43:02] lb.utils.events INFO: eta: 12:07:33 iteration: 98899/375342 consumed_samples: 101273600 total_loss: 3.811 time: 0.3398 s/iter data_time: 0.2152 s/iter total_throughput: 3013.12 samples/s lr: 8.40e-04 [09/22 12:43:36] lb.utils.events INFO: eta: 12:06:26 iteration: 98999/375342 consumed_samples: 101376000 total_loss: 3.799 time: 0.3398 s/iter data_time: 0.2106 s/iter total_throughput: 3013.14 samples/s lr: 8.40e-04 [09/22 12:44:10] lb.utils.events INFO: eta: 12:07:01 iteration: 99099/375342 consumed_samples: 101478400 total_loss: 3.805 time: 0.3398 s/iter data_time: 0.2166 s/iter total_throughput: 3013.11 samples/s lr: 8.39e-04 [09/22 12:44:44] lb.utils.events INFO: eta: 12:05:54 iteration: 99199/375342 consumed_samples: 101580800 total_loss: 3.81 time: 0.3399 s/iter data_time: 0.2142 s/iter total_throughput: 3013.08 samples/s lr: 8.39e-04 [09/22 12:45:18] lb.utils.events INFO: eta: 12:05:22 iteration: 99299/375342 consumed_samples: 101683200 total_loss: 3.815 time: 0.3398 s/iter data_time: 0.2142 s/iter total_throughput: 3013.14 samples/s lr: 8.39e-04 [09/22 12:45:52] lb.utils.events INFO: eta: 12:05:23 iteration: 99399/375342 consumed_samples: 101785600 total_loss: 3.813 time: 0.3398 s/iter data_time: 0.2147 s/iter total_throughput: 3013.13 samples/s lr: 8.38e-04 [09/22 12:46:26] lb.utils.events INFO: eta: 12:05:23 iteration: 99499/375342 consumed_samples: 101888000 total_loss: 3.804 time: 0.3398 s/iter data_time: 0.2108 s/iter total_throughput: 3013.10 samples/s lr: 8.38e-04 [09/22 12:47:01] lb.utils.events INFO: eta: 12:03:04 iteration: 99599/375342 consumed_samples: 101990400 total_loss: 3.819 time: 0.3399 s/iter data_time: 0.2253 s/iter total_throughput: 3013.03 samples/s lr: 8.38e-04 [09/22 12:47:35] lb.utils.events INFO: eta: 12:01:56 iteration: 99699/375342 consumed_samples: 102092800 total_loss: 3.836 time: 0.3399 s/iter data_time: 0.2173 s/iter total_throughput: 3013.01 samples/s lr: 8.37e-04 [09/22 12:48:09] lb.utils.events INFO: eta: 12:02:50 iteration: 99799/375342 consumed_samples: 102195200 total_loss: 3.824 time: 0.3399 s/iter data_time: 0.2150 s/iter total_throughput: 3012.99 samples/s lr: 8.37e-04 [09/22 12:48:44] lb.utils.events INFO: eta: 12:03:06 iteration: 99899/375342 consumed_samples: 102297600 total_loss: 3.846 time: 0.3399 s/iter data_time: 0.2394 s/iter total_throughput: 3012.92 samples/s lr: 8.37e-04 [09/22 12:49:18] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0099999 [09/22 12:49:19] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/22 12:49:19] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/22 12:49:23] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0992 s/iter. Inference: 0.1644 s/iter. Eval: 0.0021 s/iter. Total: 0.2657 s/iter. ETA=0:00:09 [09/22 12:49:29] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.0948 s/iter. Inference: 0.2011 s/iter. Eval: 0.0021 s/iter. Total: 0.2981 s/iter. ETA=0:00:05 [09/22 12:49:34] lb.evaluation.evaluator INFO: Inference done 49152/50000. Dataloading: 0.0737 s/iter. Inference: 0.2071 s/iter. Eval: 0.0020 s/iter. Total: 0.2829 s/iter. ETA=0:00:00 [09/22 12:49:34] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/22 12:49:34] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.522067 (0.000250 s / iter per device, on 8 devices) [09/22 12:49:34] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:09 (0.000185 s / iter per device, on 8 devices) [09/22 12:49:34] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/22 12:49:34] lb.evaluation.utils INFO: copypaste: Acc@1=71.44 [09/22 12:49:34] lb.evaluation.utils INFO: copypaste: Acc@5=90.498 [09/22 12:49:34] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 71.44000, better than last best score 70.77800 @ iteration 89999. [09/22 12:49:34] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/22 12:49:35] lb.utils.events INFO: eta: 12:03:17 iteration: 99999/375342 consumed_samples: 102400000 total_loss: 3.843 time: 0.3399 s/iter data_time: 0.2060 s/iter total_throughput: 3012.91 samples/s lr: 8.37e-04 [09/22 12:50:06] lb.utils.events INFO: eta: 12:07:34 iteration: 100099/375342 consumed_samples: 102502400 total_loss: 3.817 time: 0.3398 s/iter data_time: 0.2450 s/iter total_throughput: 3013.10 samples/s lr: 8.36e-04 [09/22 12:50:41] lb.utils.events INFO: eta: 12:07:25 iteration: 100199/375342 consumed_samples: 102604800 total_loss: 3.815 time: 0.3399 s/iter data_time: 0.2107 s/iter total_throughput: 3013.03 samples/s lr: 8.36e-04 [09/22 12:51:15] lb.utils.events INFO: eta: 12:05:35 iteration: 100299/375342 consumed_samples: 102707200 total_loss: 3.804 time: 0.3399 s/iter data_time: 0.2197 s/iter total_throughput: 3013.03 samples/s lr: 8.36e-04 [09/22 12:51:49] lb.utils.events INFO: eta: 12:04:37 iteration: 100399/375342 consumed_samples: 102809600 total_loss: 3.801 time: 0.3399 s/iter data_time: 0.2208 s/iter total_throughput: 3013.01 samples/s lr: 8.35e-04 [09/22 12:52:24] lb.utils.events INFO: eta: 12:03:52 iteration: 100499/375342 consumed_samples: 102912000 total_loss: 3.814 time: 0.3399 s/iter data_time: 0.2165 s/iter total_throughput: 3013.00 samples/s lr: 8.35e-04 [09/22 12:52:58] lb.utils.events INFO: eta: 12:04:11 iteration: 100599/375342 consumed_samples: 103014400 total_loss: 3.805 time: 0.3399 s/iter data_time: 0.2103 s/iter total_throughput: 3012.98 samples/s lr: 8.35e-04 [09/22 12:53:32] lb.utils.events INFO: eta: 12:03:55 iteration: 100699/375342 consumed_samples: 103116800 total_loss: 3.796 time: 0.3399 s/iter data_time: 0.2150 s/iter total_throughput: 3012.97 samples/s lr: 8.34e-04 [09/22 12:54:06] lb.utils.events INFO: eta: 12:04:03 iteration: 100799/375342 consumed_samples: 103219200 total_loss: 3.789 time: 0.3399 s/iter data_time: 0.2107 s/iter total_throughput: 3012.94 samples/s lr: 8.34e-04 [09/22 12:54:40] lb.utils.events INFO: eta: 12:03:47 iteration: 100899/375342 consumed_samples: 103321600 total_loss: 3.809 time: 0.3399 s/iter data_time: 0.2167 s/iter total_throughput: 3012.95 samples/s lr: 8.34e-04 [09/22 12:55:14] lb.utils.events INFO: eta: 12:02:38 iteration: 100999/375342 consumed_samples: 103424000 total_loss: 3.814 time: 0.3399 s/iter data_time: 0.2114 s/iter total_throughput: 3012.97 samples/s lr: 8.33e-04 [09/22 12:55:48] lb.utils.events INFO: eta: 11:58:50 iteration: 101099/375342 consumed_samples: 103526400 total_loss: 3.818 time: 0.3399 s/iter data_time: 0.2113 s/iter total_throughput: 3012.99 samples/s lr: 8.33e-04 [09/22 12:56:22] lb.utils.events INFO: eta: 11:58:31 iteration: 101199/375342 consumed_samples: 103628800 total_loss: 3.795 time: 0.3399 s/iter data_time: 0.2251 s/iter total_throughput: 3012.96 samples/s lr: 8.33e-04 [09/22 12:56:56] lb.utils.events INFO: eta: 11:57:50 iteration: 101299/375342 consumed_samples: 103731200 total_loss: 3.788 time: 0.3399 s/iter data_time: 0.2229 s/iter total_throughput: 3012.95 samples/s lr: 8.32e-04 [09/22 12:57:30] lb.utils.events INFO: eta: 11:56:55 iteration: 101399/375342 consumed_samples: 103833600 total_loss: 3.785 time: 0.3399 s/iter data_time: 0.2218 s/iter total_throughput: 3012.94 samples/s lr: 8.32e-04 [09/22 12:58:05] lb.utils.events INFO: eta: 11:55:53 iteration: 101499/375342 consumed_samples: 103936000 total_loss: 3.78 time: 0.3399 s/iter data_time: 0.2193 s/iter total_throughput: 3012.89 samples/s lr: 8.32e-04 [09/22 12:58:39] lb.utils.events INFO: eta: 11:55:31 iteration: 101599/375342 consumed_samples: 104038400 total_loss: 3.809 time: 0.3399 s/iter data_time: 0.2086 s/iter total_throughput: 3012.86 samples/s lr: 8.32e-04 [09/22 12:59:13] lb.utils.events INFO: eta: 11:55:19 iteration: 101699/375342 consumed_samples: 104140800 total_loss: 3.813 time: 0.3399 s/iter data_time: 0.2125 s/iter total_throughput: 3012.84 samples/s lr: 8.31e-04 [09/22 12:59:47] lb.utils.events INFO: eta: 11:56:13 iteration: 101799/375342 consumed_samples: 104243200 total_loss: 3.798 time: 0.3399 s/iter data_time: 0.2185 s/iter total_throughput: 3012.83 samples/s lr: 8.31e-04 [09/22 13:00:22] lb.utils.events INFO: eta: 11:55:57 iteration: 101899/375342 consumed_samples: 104345600 total_loss: 3.809 time: 0.3399 s/iter data_time: 0.2237 s/iter total_throughput: 3012.81 samples/s lr: 8.31e-04 [09/22 13:00:56] lb.utils.events INFO: eta: 11:55:47 iteration: 101999/375342 consumed_samples: 104448000 total_loss: 3.816 time: 0.3399 s/iter data_time: 0.2157 s/iter total_throughput: 3012.82 samples/s lr: 8.30e-04 [09/22 13:01:29] lb.utils.events INFO: eta: 11:55:32 iteration: 102099/375342 consumed_samples: 104550400 total_loss: 3.814 time: 0.3399 s/iter data_time: 0.2028 s/iter total_throughput: 3012.83 samples/s lr: 8.30e-04 [09/22 13:02:03] lb.utils.events INFO: eta: 11:56:03 iteration: 102199/375342 consumed_samples: 104652800 total_loss: 3.809 time: 0.3399 s/iter data_time: 0.2117 s/iter total_throughput: 3012.84 samples/s lr: 8.30e-04 [09/22 13:02:37] lb.utils.events INFO: eta: 11:57:23 iteration: 102299/375342 consumed_samples: 104755200 total_loss: 3.818 time: 0.3399 s/iter data_time: 0.2187 s/iter total_throughput: 3012.85 samples/s lr: 8.29e-04 [09/22 13:03:11] lb.utils.events INFO: eta: 11:58:35 iteration: 102399/375342 consumed_samples: 104857600 total_loss: 3.837 time: 0.3399 s/iter data_time: 0.2196 s/iter total_throughput: 3012.83 samples/s lr: 8.29e-04 [09/22 13:03:45] lb.utils.events INFO: eta: 11:58:26 iteration: 102499/375342 consumed_samples: 104960000 total_loss: 3.832 time: 0.3399 s/iter data_time: 0.2010 s/iter total_throughput: 3012.86 samples/s lr: 8.29e-04 [09/22 13:04:19] lb.utils.events INFO: eta: 11:58:39 iteration: 102599/375342 consumed_samples: 105062400 total_loss: 3.821 time: 0.3399 s/iter data_time: 0.2080 s/iter total_throughput: 3012.83 samples/s lr: 8.28e-04 [09/22 13:04:54] lb.utils.events INFO: eta: 11:57:53 iteration: 102699/375342 consumed_samples: 105164800 total_loss: 3.809 time: 0.3399 s/iter data_time: 0.2042 s/iter total_throughput: 3012.81 samples/s lr: 8.28e-04 [09/22 13:05:28] lb.utils.events INFO: eta: 11:56:06 iteration: 102799/375342 consumed_samples: 105267200 total_loss: 3.808 time: 0.3399 s/iter data_time: 0.2131 s/iter total_throughput: 3012.78 samples/s lr: 8.28e-04 [09/22 13:06:02] lb.utils.events INFO: eta: 11:56:15 iteration: 102899/375342 consumed_samples: 105369600 total_loss: 3.801 time: 0.3399 s/iter data_time: 0.2118 s/iter total_throughput: 3012.77 samples/s lr: 8.27e-04 [09/22 13:06:36] lb.utils.events INFO: eta: 11:55:47 iteration: 102999/375342 consumed_samples: 105472000 total_loss: 3.802 time: 0.3399 s/iter data_time: 0.2128 s/iter total_throughput: 3012.78 samples/s lr: 8.27e-04 [09/22 13:07:10] lb.utils.events INFO: eta: 11:56:57 iteration: 103099/375342 consumed_samples: 105574400 total_loss: 3.797 time: 0.3399 s/iter data_time: 0.2136 s/iter total_throughput: 3012.79 samples/s lr: 8.27e-04 [09/22 13:07:43] lb.utils.events INFO: eta: 11:56:13 iteration: 103199/375342 consumed_samples: 105676800 total_loss: 3.797 time: 0.3399 s/iter data_time: 0.2193 s/iter total_throughput: 3012.82 samples/s lr: 8.27e-04 [09/22 13:08:17] lb.utils.events INFO: eta: 11:55:57 iteration: 103299/375342 consumed_samples: 105779200 total_loss: 3.819 time: 0.3399 s/iter data_time: 0.2087 s/iter total_throughput: 3012.85 samples/s lr: 8.26e-04 [09/22 13:08:52] lb.utils.events INFO: eta: 11:55:30 iteration: 103399/375342 consumed_samples: 105881600 total_loss: 3.819 time: 0.3399 s/iter data_time: 0.2146 s/iter total_throughput: 3012.81 samples/s lr: 8.26e-04 [09/22 13:09:26] lb.utils.events INFO: eta: 11:56:15 iteration: 103499/375342 consumed_samples: 105984000 total_loss: 3.814 time: 0.3399 s/iter data_time: 0.2062 s/iter total_throughput: 3012.81 samples/s lr: 8.26e-04 [09/22 13:10:00] lb.utils.events INFO: eta: 11:56:13 iteration: 103599/375342 consumed_samples: 106086400 total_loss: 3.82 time: 0.3399 s/iter data_time: 0.2241 s/iter total_throughput: 3012.77 samples/s lr: 8.25e-04 [09/22 13:10:34] lb.utils.events INFO: eta: 11:56:32 iteration: 103699/375342 consumed_samples: 106188800 total_loss: 3.82 time: 0.3399 s/iter data_time: 0.2119 s/iter total_throughput: 3012.78 samples/s lr: 8.25e-04 [09/22 13:11:08] lb.utils.events INFO: eta: 11:58:01 iteration: 103799/375342 consumed_samples: 106291200 total_loss: 3.8 time: 0.3399 s/iter data_time: 0.2222 s/iter total_throughput: 3012.79 samples/s lr: 8.25e-04 [09/22 13:11:42] lb.utils.events INFO: eta: 11:59:21 iteration: 103899/375342 consumed_samples: 106393600 total_loss: 3.787 time: 0.3399 s/iter data_time: 0.2108 s/iter total_throughput: 3012.79 samples/s lr: 8.24e-04 [09/22 13:12:16] lb.utils.events INFO: eta: 11:58:07 iteration: 103999/375342 consumed_samples: 106496000 total_loss: 3.793 time: 0.3399 s/iter data_time: 0.2190 s/iter total_throughput: 3012.78 samples/s lr: 8.24e-04 [09/22 13:12:50] lb.utils.events INFO: eta: 11:57:06 iteration: 104099/375342 consumed_samples: 106598400 total_loss: 3.801 time: 0.3399 s/iter data_time: 0.2050 s/iter total_throughput: 3012.76 samples/s lr: 8.24e-04 [09/22 13:13:25] lb.utils.events INFO: eta: 11:55:48 iteration: 104199/375342 consumed_samples: 106700800 total_loss: 3.798 time: 0.3399 s/iter data_time: 0.2222 s/iter total_throughput: 3012.70 samples/s lr: 8.23e-04 [09/22 13:13:59] lb.utils.events INFO: eta: 11:56:11 iteration: 104299/375342 consumed_samples: 106803200 total_loss: 3.797 time: 0.3399 s/iter data_time: 0.2099 s/iter total_throughput: 3012.72 samples/s lr: 8.23e-04 [09/22 13:14:33] lb.utils.events INFO: eta: 11:56:02 iteration: 104399/375342 consumed_samples: 106905600 total_loss: 3.797 time: 0.3399 s/iter data_time: 0.2206 s/iter total_throughput: 3012.69 samples/s lr: 8.23e-04 [09/22 13:15:07] lb.utils.events INFO: eta: 11:55:23 iteration: 104499/375342 consumed_samples: 107008000 total_loss: 3.801 time: 0.3399 s/iter data_time: 0.2109 s/iter total_throughput: 3012.68 samples/s lr: 8.22e-04 [09/22 13:15:41] lb.utils.events INFO: eta: 11:54:33 iteration: 104599/375342 consumed_samples: 107110400 total_loss: 3.801 time: 0.3399 s/iter data_time: 0.2184 s/iter total_throughput: 3012.67 samples/s lr: 8.22e-04 [09/22 13:16:15] lb.utils.events INFO: eta: 11:53:22 iteration: 104699/375342 consumed_samples: 107212800 total_loss: 3.793 time: 0.3399 s/iter data_time: 0.2182 s/iter total_throughput: 3012.65 samples/s lr: 8.22e-04 [09/22 13:16:49] lb.utils.events INFO: eta: 11:52:39 iteration: 104799/375342 consumed_samples: 107315200 total_loss: 3.789 time: 0.3399 s/iter data_time: 0.2112 s/iter total_throughput: 3012.66 samples/s lr: 8.21e-04 [09/22 13:17:23] lb.utils.events INFO: eta: 11:51:16 iteration: 104899/375342 consumed_samples: 107417600 total_loss: 3.804 time: 0.3399 s/iter data_time: 0.2200 s/iter total_throughput: 3012.66 samples/s lr: 8.21e-04 [09/22 13:17:57] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0104999 [09/22 13:17:58] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/22 13:17:58] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/22 13:18:02] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0883 s/iter. Inference: 0.1620 s/iter. Eval: 0.0021 s/iter. Total: 0.2525 s/iter. ETA=0:00:09 [09/22 13:18:08] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1122 s/iter. Inference: 0.1807 s/iter. Eval: 0.0020 s/iter. Total: 0.2950 s/iter. ETA=0:00:05 [09/22 13:18:13] lb.evaluation.evaluator INFO: Inference done 50000/50000. Dataloading: 0.1049 s/iter. Inference: 0.1761 s/iter. Eval: 0.0020 s/iter. Total: 0.2831 s/iter. ETA=0:00:00 [09/22 13:18:13] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/22 13:18:13] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.458616 (0.000249 s / iter per device, on 8 devices) [09/22 13:18:13] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000155 s / iter per device, on 8 devices) [09/22 13:18:13] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/22 13:18:13] lb.evaluation.utils INFO: copypaste: Acc@1=71.624 [09/22 13:18:13] lb.evaluation.utils INFO: copypaste: Acc@5=90.656 [09/22 13:18:13] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 71.62400, better than last best score 71.44000 @ iteration 99999. [09/22 13:18:13] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/22 13:18:14] lb.utils.events INFO: eta: 11:52:11 iteration: 104999/375342 consumed_samples: 107520000 total_loss: 3.824 time: 0.3399 s/iter data_time: 0.2162 s/iter total_throughput: 3012.65 samples/s lr: 8.21e-04 [09/22 13:18:45] lb.utils.events INFO: eta: 11:52:24 iteration: 105099/375342 consumed_samples: 107622400 total_loss: 3.807 time: 0.3399 s/iter data_time: 0.2318 s/iter total_throughput: 3012.88 samples/s lr: 8.21e-04 [09/22 13:19:19] lb.utils.events INFO: eta: 11:56:55 iteration: 105199/375342 consumed_samples: 107724800 total_loss: 3.79 time: 0.3399 s/iter data_time: 0.2155 s/iter total_throughput: 3012.86 samples/s lr: 8.20e-04 [09/22 13:19:53] lb.utils.events INFO: eta: 11:54:48 iteration: 105299/375342 consumed_samples: 107827200 total_loss: 3.789 time: 0.3399 s/iter data_time: 0.2115 s/iter total_throughput: 3012.88 samples/s lr: 8.20e-04 [09/22 13:20:28] lb.utils.events INFO: eta: 11:52:27 iteration: 105399/375342 consumed_samples: 107929600 total_loss: 3.797 time: 0.3399 s/iter data_time: 0.2183 s/iter total_throughput: 3012.84 samples/s lr: 8.20e-04 [09/22 13:21:01] lb.utils.events INFO: eta: 11:51:50 iteration: 105499/375342 consumed_samples: 108032000 total_loss: 3.802 time: 0.3399 s/iter data_time: 0.2038 s/iter total_throughput: 3012.87 samples/s lr: 8.19e-04 [09/22 13:21:35] lb.utils.events INFO: eta: 11:52:22 iteration: 105599/375342 consumed_samples: 108134400 total_loss: 3.799 time: 0.3399 s/iter data_time: 0.2111 s/iter total_throughput: 3012.89 samples/s lr: 8.19e-04 [09/22 13:22:09] lb.utils.events INFO: eta: 11:51:07 iteration: 105699/375342 consumed_samples: 108236800 total_loss: 3.784 time: 0.3399 s/iter data_time: 0.2122 s/iter total_throughput: 3012.89 samples/s lr: 8.19e-04 [09/22 13:22:43] lb.utils.events INFO: eta: 11:50:15 iteration: 105799/375342 consumed_samples: 108339200 total_loss: 3.771 time: 0.3399 s/iter data_time: 0.2140 s/iter total_throughput: 3012.88 samples/s lr: 8.18e-04 [09/22 13:23:18] lb.utils.events INFO: eta: 11:48:15 iteration: 105899/375342 consumed_samples: 108441600 total_loss: 3.788 time: 0.3399 s/iter data_time: 0.2155 s/iter total_throughput: 3012.83 samples/s lr: 8.18e-04 [09/22 13:23:52] lb.utils.events INFO: eta: 11:47:42 iteration: 105999/375342 consumed_samples: 108544000 total_loss: 3.802 time: 0.3399 s/iter data_time: 0.2068 s/iter total_throughput: 3012.84 samples/s lr: 8.18e-04 [09/22 13:24:26] lb.utils.events INFO: eta: 11:46:23 iteration: 106099/375342 consumed_samples: 108646400 total_loss: 3.789 time: 0.3399 s/iter data_time: 0.2251 s/iter total_throughput: 3012.80 samples/s lr: 8.17e-04 [09/22 13:25:00] lb.utils.events INFO: eta: 11:44:27 iteration: 106199/375342 consumed_samples: 108748800 total_loss: 3.79 time: 0.3399 s/iter data_time: 0.2040 s/iter total_throughput: 3012.80 samples/s lr: 8.17e-04 [09/22 13:25:34] lb.utils.events INFO: eta: 11:42:59 iteration: 106299/375342 consumed_samples: 108851200 total_loss: 3.798 time: 0.3399 s/iter data_time: 0.2178 s/iter total_throughput: 3012.78 samples/s lr: 8.17e-04 [09/22 13:26:08] lb.utils.events INFO: eta: 11:44:00 iteration: 106399/375342 consumed_samples: 108953600 total_loss: 3.793 time: 0.3399 s/iter data_time: 0.2061 s/iter total_throughput: 3012.79 samples/s lr: 8.16e-04 [09/22 13:26:43] lb.utils.events INFO: eta: 11:43:57 iteration: 106499/375342 consumed_samples: 109056000 total_loss: 3.787 time: 0.3399 s/iter data_time: 0.2102 s/iter total_throughput: 3012.75 samples/s lr: 8.16e-04 [09/22 13:27:17] lb.utils.events INFO: eta: 11:43:24 iteration: 106599/375342 consumed_samples: 109158400 total_loss: 3.793 time: 0.3399 s/iter data_time: 0.2140 s/iter total_throughput: 3012.74 samples/s lr: 8.16e-04 [09/22 13:27:50] lb.utils.events INFO: eta: 11:44:18 iteration: 106699/375342 consumed_samples: 109260800 total_loss: 3.793 time: 0.3399 s/iter data_time: 0.2167 s/iter total_throughput: 3012.77 samples/s lr: 8.15e-04 [09/22 13:28:24] lb.utils.events INFO: eta: 11:42:53 iteration: 106799/375342 consumed_samples: 109363200 total_loss: 3.81 time: 0.3399 s/iter data_time: 0.2077 s/iter total_throughput: 3012.80 samples/s lr: 8.15e-04 [09/22 13:28:58] lb.utils.events INFO: eta: 11:42:17 iteration: 106899/375342 consumed_samples: 109465600 total_loss: 3.812 time: 0.3399 s/iter data_time: 0.2113 s/iter total_throughput: 3012.79 samples/s lr: 8.15e-04 [09/22 13:29:33] lb.utils.events INFO: eta: 11:40:22 iteration: 106999/375342 consumed_samples: 109568000 total_loss: 3.805 time: 0.3399 s/iter data_time: 0.2216 s/iter total_throughput: 3012.75 samples/s lr: 8.14e-04 [09/22 13:30:07] lb.utils.events INFO: eta: 11:41:13 iteration: 107099/375342 consumed_samples: 109670400 total_loss: 3.803 time: 0.3399 s/iter data_time: 0.2239 s/iter total_throughput: 3012.76 samples/s lr: 8.14e-04 [09/22 13:30:40] lb.utils.events INFO: eta: 11:40:05 iteration: 107199/375342 consumed_samples: 109772800 total_loss: 3.801 time: 0.3399 s/iter data_time: 0.2200 s/iter total_throughput: 3012.77 samples/s lr: 8.14e-04 [09/22 13:31:15] lb.utils.events INFO: eta: 11:39:49 iteration: 107299/375342 consumed_samples: 109875200 total_loss: 3.802 time: 0.3399 s/iter data_time: 0.2144 s/iter total_throughput: 3012.76 samples/s lr: 8.13e-04 [09/22 13:31:49] lb.utils.events INFO: eta: 11:39:21 iteration: 107399/375342 consumed_samples: 109977600 total_loss: 3.803 time: 0.3399 s/iter data_time: 0.2111 s/iter total_throughput: 3012.76 samples/s lr: 8.13e-04 [09/22 13:32:23] lb.utils.events INFO: eta: 11:38:07 iteration: 107499/375342 consumed_samples: 110080000 total_loss: 3.777 time: 0.3399 s/iter data_time: 0.2145 s/iter total_throughput: 3012.75 samples/s lr: 8.13e-04 [09/22 13:32:57] lb.utils.events INFO: eta: 11:36:16 iteration: 107599/375342 consumed_samples: 110182400 total_loss: 3.787 time: 0.3399 s/iter data_time: 0.2158 s/iter total_throughput: 3012.73 samples/s lr: 8.12e-04 [09/22 13:33:31] lb.utils.events INFO: eta: 11:35:01 iteration: 107699/375342 consumed_samples: 110284800 total_loss: 3.799 time: 0.3399 s/iter data_time: 0.2153 s/iter total_throughput: 3012.72 samples/s lr: 8.12e-04 [09/22 13:34:05] lb.utils.events INFO: eta: 11:36:39 iteration: 107799/375342 consumed_samples: 110387200 total_loss: 3.795 time: 0.3399 s/iter data_time: 0.2128 s/iter total_throughput: 3012.75 samples/s lr: 8.12e-04 [09/22 13:34:39] lb.utils.events INFO: eta: 11:38:29 iteration: 107899/375342 consumed_samples: 110489600 total_loss: 3.796 time: 0.3399 s/iter data_time: 0.2134 s/iter total_throughput: 3012.72 samples/s lr: 8.11e-04 [09/22 13:35:13] lb.utils.events INFO: eta: 11:41:03 iteration: 107999/375342 consumed_samples: 110592000 total_loss: 3.785 time: 0.3399 s/iter data_time: 0.2319 s/iter total_throughput: 3012.74 samples/s lr: 8.11e-04 [09/22 13:35:47] lb.utils.events INFO: eta: 11:38:36 iteration: 108099/375342 consumed_samples: 110694400 total_loss: 3.784 time: 0.3399 s/iter data_time: 0.2092 s/iter total_throughput: 3012.75 samples/s lr: 8.11e-04 [09/22 13:36:20] lb.utils.events INFO: eta: 11:38:26 iteration: 108199/375342 consumed_samples: 110796800 total_loss: 3.776 time: 0.3399 s/iter data_time: 0.2209 s/iter total_throughput: 3012.79 samples/s lr: 8.11e-04 [09/22 13:36:54] lb.utils.events INFO: eta: 11:38:06 iteration: 108299/375342 consumed_samples: 110899200 total_loss: 3.779 time: 0.3399 s/iter data_time: 0.2205 s/iter total_throughput: 3012.76 samples/s lr: 8.10e-04 [09/22 13:37:29] lb.utils.events INFO: eta: 11:37:39 iteration: 108399/375342 consumed_samples: 111001600 total_loss: 3.804 time: 0.3399 s/iter data_time: 0.2205 s/iter total_throughput: 3012.75 samples/s lr: 8.10e-04 [09/22 13:38:02] lb.utils.events INFO: eta: 11:39:45 iteration: 108499/375342 consumed_samples: 111104000 total_loss: 3.808 time: 0.3399 s/iter data_time: 0.2033 s/iter total_throughput: 3012.80 samples/s lr: 8.10e-04 [09/22 13:38:35] lb.utils.events INFO: eta: 11:42:50 iteration: 108599/375342 consumed_samples: 111206400 total_loss: 3.824 time: 0.3399 s/iter data_time: 0.2119 s/iter total_throughput: 3012.85 samples/s lr: 8.09e-04 [09/22 13:39:09] lb.utils.events INFO: eta: 11:42:59 iteration: 108699/375342 consumed_samples: 111308800 total_loss: 3.812 time: 0.3399 s/iter data_time: 0.2076 s/iter total_throughput: 3012.87 samples/s lr: 8.09e-04 [09/22 13:39:43] lb.utils.events INFO: eta: 11:41:36 iteration: 108799/375342 consumed_samples: 111411200 total_loss: 3.796 time: 0.3399 s/iter data_time: 0.2058 s/iter total_throughput: 3012.90 samples/s lr: 8.09e-04 [09/22 13:40:17] lb.utils.events INFO: eta: 11:39:37 iteration: 108899/375342 consumed_samples: 111513600 total_loss: 3.792 time: 0.3399 s/iter data_time: 0.2146 s/iter total_throughput: 3012.91 samples/s lr: 8.08e-04 [09/22 13:40:51] lb.utils.events INFO: eta: 11:37:37 iteration: 108999/375342 consumed_samples: 111616000 total_loss: 3.797 time: 0.3399 s/iter data_time: 0.2068 s/iter total_throughput: 3012.90 samples/s lr: 8.08e-04 [09/22 13:41:25] lb.utils.events INFO: eta: 11:39:31 iteration: 109099/375342 consumed_samples: 111718400 total_loss: 3.792 time: 0.3399 s/iter data_time: 0.2127 s/iter total_throughput: 3012.92 samples/s lr: 8.08e-04 [09/22 13:41:59] lb.utils.events INFO: eta: 11:40:01 iteration: 109199/375342 consumed_samples: 111820800 total_loss: 3.781 time: 0.3399 s/iter data_time: 0.2219 s/iter total_throughput: 3012.92 samples/s lr: 8.07e-04 [09/22 13:42:33] lb.utils.events INFO: eta: 11:39:16 iteration: 109299/375342 consumed_samples: 111923200 total_loss: 3.79 time: 0.3399 s/iter data_time: 0.2077 s/iter total_throughput: 3012.92 samples/s lr: 8.07e-04 [09/22 13:43:07] lb.utils.events INFO: eta: 11:39:00 iteration: 109399/375342 consumed_samples: 112025600 total_loss: 3.784 time: 0.3399 s/iter data_time: 0.2024 s/iter total_throughput: 3012.91 samples/s lr: 8.07e-04 [09/22 13:43:40] lb.utils.events INFO: eta: 11:38:08 iteration: 109499/375342 consumed_samples: 112128000 total_loss: 3.797 time: 0.3399 s/iter data_time: 0.2189 s/iter total_throughput: 3012.96 samples/s lr: 8.06e-04 [09/22 13:44:13] lb.utils.events INFO: eta: 11:37:46 iteration: 109599/375342 consumed_samples: 112230400 total_loss: 3.796 time: 0.3399 s/iter data_time: 0.1948 s/iter total_throughput: 3013.01 samples/s lr: 8.06e-04 [09/22 13:44:47] lb.utils.events INFO: eta: 11:36:56 iteration: 109699/375342 consumed_samples: 112332800 total_loss: 3.792 time: 0.3399 s/iter data_time: 0.2186 s/iter total_throughput: 3013.02 samples/s lr: 8.06e-04 [09/22 13:45:21] lb.utils.events INFO: eta: 11:36:33 iteration: 109799/375342 consumed_samples: 112435200 total_loss: 3.795 time: 0.3399 s/iter data_time: 0.2159 s/iter total_throughput: 3013.03 samples/s lr: 8.05e-04 [09/22 13:45:55] lb.utils.events INFO: eta: 11:36:42 iteration: 109899/375342 consumed_samples: 112537600 total_loss: 3.776 time: 0.3399 s/iter data_time: 0.2148 s/iter total_throughput: 3013.04 samples/s lr: 8.05e-04 [09/22 13:46:29] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0109999 [09/22 13:46:29] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/22 13:46:29] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/22 13:46:33] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0943 s/iter. Inference: 0.1628 s/iter. Eval: 0.0019 s/iter. Total: 0.2589 s/iter. ETA=0:00:09 [09/22 13:46:38] lb.evaluation.evaluator INFO: Inference done 28672/50000. Dataloading: 0.0836 s/iter. Inference: 0.2004 s/iter. Eval: 0.0019 s/iter. Total: 0.2860 s/iter. ETA=0:00:05 [09/22 13:46:44] lb.evaluation.evaluator INFO: Inference done 46080/50000. Dataloading: 0.0653 s/iter. Inference: 0.2304 s/iter. Eval: 0.0019 s/iter. Total: 0.2977 s/iter. ETA=0:00:00 [09/22 13:46:44] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/22 13:46:44] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.791427 (0.000256 s / iter per device, on 8 devices) [09/22 13:46:44] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:10 (0.000202 s / iter per device, on 8 devices) [09/22 13:46:44] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/22 13:46:44] lb.evaluation.utils INFO: copypaste: Acc@1=71.468 [09/22 13:46:44] lb.evaluation.utils INFO: copypaste: Acc@5=90.598 [09/22 13:46:44] lb.engine.hooks INFO: Not saving as latest eval score for Acc@1 is 71.46800, not better than best score 71.62400 @ iteration 104999. [09/22 13:46:44] lb.utils.events INFO: eta: 11:38:14 iteration: 109999/375342 consumed_samples: 112640000 total_loss: 3.779 time: 0.3399 s/iter data_time: 0.2026 s/iter total_throughput: 3013.09 samples/s lr: 8.05e-04 [09/22 13:47:17] lb.utils.events INFO: eta: 11:39:03 iteration: 110099/375342 consumed_samples: 112742400 total_loss: 3.803 time: 0.3398 s/iter data_time: 0.2043 s/iter total_throughput: 3013.22 samples/s lr: 8.04e-04 [09/22 13:47:50] lb.utils.events INFO: eta: 11:38:42 iteration: 110199/375342 consumed_samples: 112844800 total_loss: 3.812 time: 0.3398 s/iter data_time: 0.1987 s/iter total_throughput: 3013.24 samples/s lr: 8.04e-04 [09/22 13:48:24] lb.utils.events INFO: eta: 11:39:39 iteration: 110299/375342 consumed_samples: 112947200 total_loss: 3.812 time: 0.3398 s/iter data_time: 0.2187 s/iter total_throughput: 3013.26 samples/s lr: 8.04e-04 [09/22 13:48:58] lb.utils.events INFO: eta: 11:39:07 iteration: 110399/375342 consumed_samples: 113049600 total_loss: 3.797 time: 0.3398 s/iter data_time: 0.1952 s/iter total_throughput: 3013.30 samples/s lr: 8.03e-04 [09/22 13:49:32] lb.utils.events INFO: eta: 11:37:31 iteration: 110499/375342 consumed_samples: 113152000 total_loss: 3.776 time: 0.3398 s/iter data_time: 0.2185 s/iter total_throughput: 3013.27 samples/s lr: 8.03e-04 [09/22 13:50:06] lb.utils.events INFO: eta: 11:35:46 iteration: 110599/375342 consumed_samples: 113254400 total_loss: 3.773 time: 0.3398 s/iter data_time: 0.2254 s/iter total_throughput: 3013.30 samples/s lr: 8.03e-04 [09/22 13:50:39] lb.utils.events INFO: eta: 11:35:19 iteration: 110699/375342 consumed_samples: 113356800 total_loss: 3.794 time: 0.3398 s/iter data_time: 0.2161 s/iter total_throughput: 3013.33 samples/s lr: 8.02e-04 [09/22 13:51:13] lb.utils.events INFO: eta: 11:34:41 iteration: 110799/375342 consumed_samples: 113459200 total_loss: 3.794 time: 0.3398 s/iter data_time: 0.2042 s/iter total_throughput: 3013.36 samples/s lr: 8.02e-04 [09/22 13:51:46] lb.utils.events INFO: eta: 11:34:25 iteration: 110899/375342 consumed_samples: 113561600 total_loss: 3.796 time: 0.3398 s/iter data_time: 0.2051 s/iter total_throughput: 3013.41 samples/s lr: 8.02e-04 [09/22 13:52:20] lb.utils.events INFO: eta: 11:32:08 iteration: 110999/375342 consumed_samples: 113664000 total_loss: 3.787 time: 0.3398 s/iter data_time: 0.2124 s/iter total_throughput: 3013.47 samples/s lr: 8.01e-04 [09/22 13:52:53] lb.utils.events INFO: eta: 11:28:53 iteration: 111099/375342 consumed_samples: 113766400 total_loss: 3.771 time: 0.3398 s/iter data_time: 0.2073 s/iter total_throughput: 3013.50 samples/s lr: 8.01e-04 [09/22 13:53:27] lb.utils.events INFO: eta: 11:28:37 iteration: 111199/375342 consumed_samples: 113868800 total_loss: 3.795 time: 0.3398 s/iter data_time: 0.2135 s/iter total_throughput: 3013.53 samples/s lr: 8.01e-04 [09/22 13:54:00] lb.utils.events INFO: eta: 11:27:55 iteration: 111299/375342 consumed_samples: 113971200 total_loss: 3.801 time: 0.3398 s/iter data_time: 0.1992 s/iter total_throughput: 3013.57 samples/s lr: 8.00e-04 [09/22 13:54:33] lb.utils.events INFO: eta: 11:28:12 iteration: 111399/375342 consumed_samples: 114073600 total_loss: 3.774 time: 0.3398 s/iter data_time: 0.1989 s/iter total_throughput: 3013.65 samples/s lr: 8.00e-04 [09/22 13:55:06] lb.utils.events INFO: eta: 11:29:49 iteration: 111499/375342 consumed_samples: 114176000 total_loss: 3.771 time: 0.3398 s/iter data_time: 0.2019 s/iter total_throughput: 3013.72 samples/s lr: 8.00e-04 [09/22 13:55:40] lb.utils.events INFO: eta: 11:30:11 iteration: 111599/375342 consumed_samples: 114278400 total_loss: 3.78 time: 0.3398 s/iter data_time: 0.2082 s/iter total_throughput: 3013.75 samples/s lr: 7.99e-04 [09/22 13:56:14] lb.utils.events INFO: eta: 11:29:18 iteration: 111699/375342 consumed_samples: 114380800 total_loss: 3.773 time: 0.3398 s/iter data_time: 0.2129 s/iter total_throughput: 3013.76 samples/s lr: 7.99e-04 [09/22 13:56:47] lb.utils.events INFO: eta: 11:29:13 iteration: 111799/375342 consumed_samples: 114483200 total_loss: 3.769 time: 0.3398 s/iter data_time: 0.2105 s/iter total_throughput: 3013.80 samples/s lr: 7.99e-04 [09/22 13:57:21] lb.utils.events INFO: eta: 11:28:57 iteration: 111899/375342 consumed_samples: 114585600 total_loss: 3.76 time: 0.3398 s/iter data_time: 0.2022 s/iter total_throughput: 3013.84 samples/s lr: 7.98e-04 [09/22 13:57:54] lb.utils.events INFO: eta: 11:28:49 iteration: 111999/375342 consumed_samples: 114688000 total_loss: 3.778 time: 0.3398 s/iter data_time: 0.2150 s/iter total_throughput: 3013.88 samples/s lr: 7.98e-04 [09/22 13:58:27] lb.utils.events INFO: eta: 11:28:36 iteration: 112099/375342 consumed_samples: 114790400 total_loss: 3.778 time: 0.3398 s/iter data_time: 0.2035 s/iter total_throughput: 3013.95 samples/s lr: 7.98e-04 [09/22 13:59:02] lb.utils.events INFO: eta: 11:27:19 iteration: 112199/375342 consumed_samples: 114892800 total_loss: 3.764 time: 0.3398 s/iter data_time: 0.2174 s/iter total_throughput: 3013.91 samples/s lr: 7.97e-04 [09/22 13:59:35] lb.utils.events INFO: eta: 11:27:44 iteration: 112299/375342 consumed_samples: 114995200 total_loss: 3.785 time: 0.3398 s/iter data_time: 0.2042 s/iter total_throughput: 3013.97 samples/s lr: 7.97e-04 [09/22 14:00:09] lb.utils.events INFO: eta: 11:27:25 iteration: 112399/375342 consumed_samples: 115097600 total_loss: 3.782 time: 0.3398 s/iter data_time: 0.2083 s/iter total_throughput: 3013.95 samples/s lr: 7.97e-04 [09/22 14:00:44] lb.utils.events INFO: eta: 11:26:24 iteration: 112499/375342 consumed_samples: 115200000 total_loss: 3.777 time: 0.3398 s/iter data_time: 0.2117 s/iter total_throughput: 3013.92 samples/s lr: 7.96e-04 [09/22 14:01:19] lb.utils.events INFO: eta: 11:25:22 iteration: 112599/375342 consumed_samples: 115302400 total_loss: 3.779 time: 0.3398 s/iter data_time: 0.2141 s/iter total_throughput: 3013.84 samples/s lr: 7.96e-04 [09/22 14:01:53] lb.utils.events INFO: eta: 11:25:09 iteration: 112699/375342 consumed_samples: 115404800 total_loss: 3.78 time: 0.3398 s/iter data_time: 0.2179 s/iter total_throughput: 3013.79 samples/s lr: 7.96e-04 [09/22 14:02:28] lb.utils.events INFO: eta: 11:24:51 iteration: 112799/375342 consumed_samples: 115507200 total_loss: 3.773 time: 0.3398 s/iter data_time: 0.2108 s/iter total_throughput: 3013.74 samples/s lr: 7.95e-04 [09/22 14:03:02] lb.utils.events INFO: eta: 11:24:24 iteration: 112899/375342 consumed_samples: 115609600 total_loss: 3.773 time: 0.3398 s/iter data_time: 0.2097 s/iter total_throughput: 3013.70 samples/s lr: 7.95e-04 [09/22 14:03:37] lb.utils.events INFO: eta: 11:24:15 iteration: 112999/375342 consumed_samples: 115712000 total_loss: 3.771 time: 0.3398 s/iter data_time: 0.2108 s/iter total_throughput: 3013.65 samples/s lr: 7.95e-04 [09/22 14:04:11] lb.utils.events INFO: eta: 11:23:47 iteration: 113099/375342 consumed_samples: 115814400 total_loss: 3.777 time: 0.3398 s/iter data_time: 0.2127 s/iter total_throughput: 3013.63 samples/s lr: 7.94e-04 [09/22 14:04:45] lb.utils.events INFO: eta: 11:24:53 iteration: 113199/375342 consumed_samples: 115916800 total_loss: 3.777 time: 0.3398 s/iter data_time: 0.2260 s/iter total_throughput: 3013.63 samples/s lr: 7.94e-04 [09/22 14:05:20] lb.utils.events INFO: eta: 11:21:16 iteration: 113299/375342 consumed_samples: 116019200 total_loss: 3.77 time: 0.3398 s/iter data_time: 0.2363 s/iter total_throughput: 3013.55 samples/s lr: 7.94e-04 [09/22 14:05:55] lb.utils.events INFO: eta: 11:18:07 iteration: 113399/375342 consumed_samples: 116121600 total_loss: 3.78 time: 0.3398 s/iter data_time: 0.2210 s/iter total_throughput: 3013.47 samples/s lr: 7.93e-04 [09/22 14:06:30] lb.utils.events INFO: eta: 11:19:09 iteration: 113499/375342 consumed_samples: 116224000 total_loss: 3.788 time: 0.3398 s/iter data_time: 0.2239 s/iter total_throughput: 3013.44 samples/s lr: 7.93e-04 [09/22 14:07:04] lb.utils.events INFO: eta: 11:21:41 iteration: 113599/375342 consumed_samples: 116326400 total_loss: 3.773 time: 0.3398 s/iter data_time: 0.2148 s/iter total_throughput: 3013.42 samples/s lr: 7.93e-04 [09/22 14:07:38] lb.utils.events INFO: eta: 11:21:26 iteration: 113699/375342 consumed_samples: 116428800 total_loss: 3.781 time: 0.3398 s/iter data_time: 0.2185 s/iter total_throughput: 3013.40 samples/s lr: 7.92e-04 [09/22 14:08:13] lb.utils.events INFO: eta: 11:19:43 iteration: 113799/375342 consumed_samples: 116531200 total_loss: 3.789 time: 0.3398 s/iter data_time: 0.2106 s/iter total_throughput: 3013.36 samples/s lr: 7.92e-04 [09/22 14:08:46] lb.utils.events INFO: eta: 11:21:34 iteration: 113899/375342 consumed_samples: 116633600 total_loss: 3.781 time: 0.3398 s/iter data_time: 0.2110 s/iter total_throughput: 3013.40 samples/s lr: 7.92e-04 [09/22 14:09:20] lb.utils.events INFO: eta: 11:22:01 iteration: 113999/375342 consumed_samples: 116736000 total_loss: 3.77 time: 0.3398 s/iter data_time: 0.2156 s/iter total_throughput: 3013.41 samples/s lr: 7.91e-04 [09/22 14:09:54] lb.utils.events INFO: eta: 11:23:12 iteration: 114099/375342 consumed_samples: 116838400 total_loss: 3.747 time: 0.3398 s/iter data_time: 0.2054 s/iter total_throughput: 3013.45 samples/s lr: 7.91e-04 [09/22 14:10:27] lb.utils.events INFO: eta: 11:23:08 iteration: 114199/375342 consumed_samples: 116940800 total_loss: 3.758 time: 0.3398 s/iter data_time: 0.2153 s/iter total_throughput: 3013.46 samples/s lr: 7.91e-04 [09/22 14:11:02] lb.utils.events INFO: eta: 11:23:24 iteration: 114299/375342 consumed_samples: 117043200 total_loss: 3.774 time: 0.3398 s/iter data_time: 0.2160 s/iter total_throughput: 3013.43 samples/s lr: 7.90e-04 [09/22 14:11:36] lb.utils.events INFO: eta: 11:26:12 iteration: 114399/375342 consumed_samples: 117145600 total_loss: 3.766 time: 0.3398 s/iter data_time: 0.2122 s/iter total_throughput: 3013.44 samples/s lr: 7.90e-04 [09/22 14:12:10] lb.utils.events INFO: eta: 11:25:32 iteration: 114499/375342 consumed_samples: 117248000 total_loss: 3.746 time: 0.3398 s/iter data_time: 0.2283 s/iter total_throughput: 3013.44 samples/s lr: 7.90e-04 [09/22 14:12:44] lb.utils.events INFO: eta: 11:23:01 iteration: 114599/375342 consumed_samples: 117350400 total_loss: 3.769 time: 0.3398 s/iter data_time: 0.2088 s/iter total_throughput: 3013.43 samples/s lr: 7.89e-04 [09/22 14:13:18] lb.utils.events INFO: eta: 11:25:37 iteration: 114699/375342 consumed_samples: 117452800 total_loss: 3.779 time: 0.3398 s/iter data_time: 0.2029 s/iter total_throughput: 3013.44 samples/s lr: 7.89e-04 [09/22 14:13:52] lb.utils.events INFO: eta: 11:25:31 iteration: 114799/375342 consumed_samples: 117555200 total_loss: 3.776 time: 0.3398 s/iter data_time: 0.2237 s/iter total_throughput: 3013.42 samples/s lr: 7.89e-04 [09/22 14:14:26] lb.utils.events INFO: eta: 11:23:57 iteration: 114899/375342 consumed_samples: 117657600 total_loss: 3.779 time: 0.3398 s/iter data_time: 0.2209 s/iter total_throughput: 3013.44 samples/s lr: 7.88e-04 [09/22 14:15:00] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0114999 [09/22 14:15:01] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/22 14:15:01] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/22 14:15:05] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.1009 s/iter. Inference: 0.1587 s/iter. Eval: 0.0021 s/iter. Total: 0.2617 s/iter. ETA=0:00:09 [09/22 14:15:10] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1187 s/iter. Inference: 0.1740 s/iter. Eval: 0.0020 s/iter. Total: 0.2948 s/iter. ETA=0:00:05 [09/22 14:15:15] lb.evaluation.evaluator INFO: Inference done 48128/50000. Dataloading: 0.1201 s/iter. Inference: 0.1677 s/iter. Eval: 0.0021 s/iter. Total: 0.2899 s/iter. ETA=0:00:00 [09/22 14:15:16] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/22 14:15:16] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.530676 (0.000251 s / iter per device, on 8 devices) [09/22 14:15:16] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000148 s / iter per device, on 8 devices) [09/22 14:15:16] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/22 14:15:16] lb.evaluation.utils INFO: copypaste: Acc@1=71.92399999999999 [09/22 14:15:16] lb.evaluation.utils INFO: copypaste: Acc@5=90.84400000000001 [09/22 14:15:16] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 71.92400, better than last best score 71.62400 @ iteration 104999. [09/22 14:15:16] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/22 14:15:16] lb.utils.events INFO: eta: 11:22:14 iteration: 114999/375342 consumed_samples: 117760000 total_loss: 3.765 time: 0.3398 s/iter data_time: 0.2138 s/iter total_throughput: 3013.41 samples/s lr: 7.88e-04 [09/22 14:15:48] lb.utils.events INFO: eta: 11:26:23 iteration: 115099/375342 consumed_samples: 117862400 total_loss: 3.766 time: 0.3398 s/iter data_time: 0.2448 s/iter total_throughput: 3013.58 samples/s lr: 7.88e-04 [09/22 14:16:22] lb.utils.events INFO: eta: 11:25:53 iteration: 115199/375342 consumed_samples: 117964800 total_loss: 3.767 time: 0.3398 s/iter data_time: 0.2161 s/iter total_throughput: 3013.55 samples/s lr: 7.87e-04 [09/22 14:16:57] lb.utils.events INFO: eta: 11:25:51 iteration: 115299/375342 consumed_samples: 118067200 total_loss: 3.773 time: 0.3398 s/iter data_time: 0.2173 s/iter total_throughput: 3013.52 samples/s lr: 7.87e-04 [09/22 14:17:31] lb.utils.events INFO: eta: 11:25:49 iteration: 115399/375342 consumed_samples: 118169600 total_loss: 3.778 time: 0.3398 s/iter data_time: 0.2059 s/iter total_throughput: 3013.54 samples/s lr: 7.87e-04 [09/22 14:18:05] lb.utils.events INFO: eta: 11:26:03 iteration: 115499/375342 consumed_samples: 118272000 total_loss: 3.783 time: 0.3398 s/iter data_time: 0.2108 s/iter total_throughput: 3013.55 samples/s lr: 7.86e-04 [09/22 14:18:39] lb.utils.events INFO: eta: 11:25:31 iteration: 115599/375342 consumed_samples: 118374400 total_loss: 3.781 time: 0.3398 s/iter data_time: 0.2158 s/iter total_throughput: 3013.51 samples/s lr: 7.86e-04 [09/22 14:19:13] lb.utils.events INFO: eta: 11:24:51 iteration: 115699/375342 consumed_samples: 118476800 total_loss: 3.778 time: 0.3398 s/iter data_time: 0.2111 s/iter total_throughput: 3013.53 samples/s lr: 7.85e-04 [09/22 14:19:47] lb.utils.events INFO: eta: 11:24:35 iteration: 115799/375342 consumed_samples: 118579200 total_loss: 3.775 time: 0.3398 s/iter data_time: 0.2135 s/iter total_throughput: 3013.51 samples/s lr: 7.85e-04 [09/22 14:20:22] lb.utils.events INFO: eta: 11:24:52 iteration: 115899/375342 consumed_samples: 118681600 total_loss: 3.759 time: 0.3398 s/iter data_time: 0.2209 s/iter total_throughput: 3013.46 samples/s lr: 7.85e-04 [09/22 14:20:56] lb.utils.events INFO: eta: 11:26:06 iteration: 115999/375342 consumed_samples: 118784000 total_loss: 3.753 time: 0.3398 s/iter data_time: 0.2134 s/iter total_throughput: 3013.45 samples/s lr: 7.84e-04 [09/22 14:21:30] lb.utils.events INFO: eta: 11:19:57 iteration: 116099/375342 consumed_samples: 118886400 total_loss: 3.753 time: 0.3398 s/iter data_time: 0.2068 s/iter total_throughput: 3013.44 samples/s lr: 7.84e-04 [09/22 14:22:04] lb.utils.events INFO: eta: 11:19:39 iteration: 116199/375342 consumed_samples: 118988800 total_loss: 3.754 time: 0.3398 s/iter data_time: 0.2026 s/iter total_throughput: 3013.46 samples/s lr: 7.84e-04 [09/22 14:22:37] lb.utils.events INFO: eta: 11:19:43 iteration: 116299/375342 consumed_samples: 119091200 total_loss: 3.783 time: 0.3398 s/iter data_time: 0.2184 s/iter total_throughput: 3013.48 samples/s lr: 7.83e-04 [09/22 14:23:12] lb.utils.events INFO: eta: 11:19:21 iteration: 116399/375342 consumed_samples: 119193600 total_loss: 3.77 time: 0.3398 s/iter data_time: 0.2147 s/iter total_throughput: 3013.44 samples/s lr: 7.83e-04 [09/22 14:23:47] lb.utils.events INFO: eta: 11:18:18 iteration: 116499/375342 consumed_samples: 119296000 total_loss: 3.752 time: 0.3398 s/iter data_time: 0.2334 s/iter total_throughput: 3013.36 samples/s lr: 7.83e-04 [09/22 14:24:21] lb.utils.events INFO: eta: 11:17:58 iteration: 116599/375342 consumed_samples: 119398400 total_loss: 3.745 time: 0.3398 s/iter data_time: 0.2100 s/iter total_throughput: 3013.36 samples/s lr: 7.82e-04 [09/22 14:24:55] lb.utils.events INFO: eta: 11:17:05 iteration: 116699/375342 consumed_samples: 119500800 total_loss: 3.731 time: 0.3398 s/iter data_time: 0.2048 s/iter total_throughput: 3013.36 samples/s lr: 7.82e-04 [09/22 14:25:29] lb.utils.events INFO: eta: 11:17:08 iteration: 116799/375342 consumed_samples: 119603200 total_loss: 3.735 time: 0.3398 s/iter data_time: 0.2123 s/iter total_throughput: 3013.36 samples/s lr: 7.82e-04 [09/22 14:26:03] lb.utils.events INFO: eta: 11:18:02 iteration: 116899/375342 consumed_samples: 119705600 total_loss: 3.777 time: 0.3398 s/iter data_time: 0.2047 s/iter total_throughput: 3013.37 samples/s lr: 7.81e-04 [09/22 14:26:37] lb.utils.events INFO: eta: 11:16:58 iteration: 116999/375342 consumed_samples: 119808000 total_loss: 3.775 time: 0.3398 s/iter data_time: 0.2037 s/iter total_throughput: 3013.38 samples/s lr: 7.81e-04 [09/22 14:27:10] lb.utils.events INFO: eta: 11:17:02 iteration: 117099/375342 consumed_samples: 119910400 total_loss: 3.765 time: 0.3398 s/iter data_time: 0.2088 s/iter total_throughput: 3013.41 samples/s lr: 7.81e-04 [09/22 14:27:44] lb.utils.events INFO: eta: 11:16:16 iteration: 117199/375342 consumed_samples: 120012800 total_loss: 3.77 time: 0.3398 s/iter data_time: 0.2097 s/iter total_throughput: 3013.42 samples/s lr: 7.80e-04 [09/22 14:28:17] lb.utils.events INFO: eta: 11:17:38 iteration: 117299/375342 consumed_samples: 120115200 total_loss: 3.764 time: 0.3398 s/iter data_time: 0.2017 s/iter total_throughput: 3013.47 samples/s lr: 7.80e-04 [09/22 14:28:52] lb.utils.events INFO: eta: 11:16:16 iteration: 117399/375342 consumed_samples: 120217600 total_loss: 3.754 time: 0.3398 s/iter data_time: 0.2219 s/iter total_throughput: 3013.45 samples/s lr: 7.80e-04 [09/22 14:29:26] lb.utils.events INFO: eta: 11:17:21 iteration: 117499/375342 consumed_samples: 120320000 total_loss: 3.762 time: 0.3398 s/iter data_time: 0.2022 s/iter total_throughput: 3013.45 samples/s lr: 7.79e-04 [09/22 14:30:00] lb.utils.events INFO: eta: 11:17:16 iteration: 117599/375342 consumed_samples: 120422400 total_loss: 3.762 time: 0.3398 s/iter data_time: 0.2231 s/iter total_throughput: 3013.44 samples/s lr: 7.79e-04 [09/22 14:30:34] lb.utils.events INFO: eta: 11:16:51 iteration: 117699/375342 consumed_samples: 120524800 total_loss: 3.747 time: 0.3398 s/iter data_time: 0.2089 s/iter total_throughput: 3013.45 samples/s lr: 7.79e-04 [09/22 14:31:08] lb.utils.events INFO: eta: 11:16:43 iteration: 117799/375342 consumed_samples: 120627200 total_loss: 3.75 time: 0.3398 s/iter data_time: 0.2226 s/iter total_throughput: 3013.41 samples/s lr: 7.78e-04 [09/22 14:31:42] lb.utils.events INFO: eta: 11:14:22 iteration: 117899/375342 consumed_samples: 120729600 total_loss: 3.759 time: 0.3398 s/iter data_time: 0.2167 s/iter total_throughput: 3013.42 samples/s lr: 7.78e-04 [09/22 14:32:16] lb.utils.events INFO: eta: 11:13:24 iteration: 117999/375342 consumed_samples: 120832000 total_loss: 3.751 time: 0.3398 s/iter data_time: 0.2190 s/iter total_throughput: 3013.41 samples/s lr: 7.78e-04 [09/22 14:32:50] lb.utils.events INFO: eta: 11:12:37 iteration: 118099/375342 consumed_samples: 120934400 total_loss: 3.743 time: 0.3398 s/iter data_time: 0.2005 s/iter total_throughput: 3013.42 samples/s lr: 7.77e-04 [09/22 14:33:25] lb.utils.events INFO: eta: 11:12:21 iteration: 118199/375342 consumed_samples: 121036800 total_loss: 3.756 time: 0.3398 s/iter data_time: 0.2348 s/iter total_throughput: 3013.37 samples/s lr: 7.77e-04 [09/22 14:33:58] lb.utils.events INFO: eta: 11:09:03 iteration: 118299/375342 consumed_samples: 121139200 total_loss: 3.758 time: 0.3398 s/iter data_time: 0.2118 s/iter total_throughput: 3013.38 samples/s lr: 7.77e-04 [09/22 14:34:33] lb.utils.events INFO: eta: 11:09:01 iteration: 118399/375342 consumed_samples: 121241600 total_loss: 3.764 time: 0.3398 s/iter data_time: 0.2116 s/iter total_throughput: 3013.36 samples/s lr: 7.76e-04 [09/22 14:35:07] lb.utils.events INFO: eta: 11:10:02 iteration: 118499/375342 consumed_samples: 121344000 total_loss: 3.775 time: 0.3398 s/iter data_time: 0.2216 s/iter total_throughput: 3013.37 samples/s lr: 7.76e-04 [09/22 14:35:41] lb.utils.events INFO: eta: 11:10:14 iteration: 118599/375342 consumed_samples: 121446400 total_loss: 3.771 time: 0.3398 s/iter data_time: 0.2052 s/iter total_throughput: 3013.36 samples/s lr: 7.75e-04 [09/22 14:36:14] lb.utils.events INFO: eta: 11:10:35 iteration: 118699/375342 consumed_samples: 121548800 total_loss: 3.751 time: 0.3398 s/iter data_time: 0.2141 s/iter total_throughput: 3013.39 samples/s lr: 7.75e-04 [09/22 14:36:48] lb.utils.events INFO: eta: 11:10:16 iteration: 118799/375342 consumed_samples: 121651200 total_loss: 3.769 time: 0.3398 s/iter data_time: 0.2150 s/iter total_throughput: 3013.42 samples/s lr: 7.75e-04 [09/22 14:37:22] lb.utils.events INFO: eta: 11:10:30 iteration: 118899/375342 consumed_samples: 121753600 total_loss: 3.76 time: 0.3398 s/iter data_time: 0.2068 s/iter total_throughput: 3013.40 samples/s lr: 7.74e-04 [09/22 14:37:56] lb.utils.events INFO: eta: 11:11:24 iteration: 118999/375342 consumed_samples: 121856000 total_loss: 3.74 time: 0.3398 s/iter data_time: 0.1968 s/iter total_throughput: 3013.44 samples/s lr: 7.74e-04 [09/22 14:38:30] lb.utils.events INFO: eta: 11:11:12 iteration: 119099/375342 consumed_samples: 121958400 total_loss: 3.771 time: 0.3398 s/iter data_time: 0.2219 s/iter total_throughput: 3013.40 samples/s lr: 7.74e-04 [09/22 14:39:04] lb.utils.events INFO: eta: 11:10:40 iteration: 119199/375342 consumed_samples: 122060800 total_loss: 3.77 time: 0.3398 s/iter data_time: 0.2290 s/iter total_throughput: 3013.38 samples/s lr: 7.73e-04 [09/22 14:39:38] lb.utils.events INFO: eta: 11:10:22 iteration: 119299/375342 consumed_samples: 122163200 total_loss: 3.744 time: 0.3398 s/iter data_time: 0.2189 s/iter total_throughput: 3013.40 samples/s lr: 7.73e-04 [09/22 14:40:12] lb.utils.events INFO: eta: 11:10:06 iteration: 119399/375342 consumed_samples: 122265600 total_loss: 3.735 time: 0.3398 s/iter data_time: 0.2139 s/iter total_throughput: 3013.41 samples/s lr: 7.73e-04 [09/22 14:40:46] lb.utils.events INFO: eta: 11:08:58 iteration: 119499/375342 consumed_samples: 122368000 total_loss: 3.749 time: 0.3398 s/iter data_time: 0.2169 s/iter total_throughput: 3013.41 samples/s lr: 7.72e-04 [09/22 14:41:20] lb.utils.events INFO: eta: 11:08:28 iteration: 119599/375342 consumed_samples: 122470400 total_loss: 3.762 time: 0.3398 s/iter data_time: 0.2177 s/iter total_throughput: 3013.43 samples/s lr: 7.72e-04 [09/22 14:41:54] lb.utils.events INFO: eta: 11:07:36 iteration: 119699/375342 consumed_samples: 122572800 total_loss: 3.774 time: 0.3398 s/iter data_time: 0.2128 s/iter total_throughput: 3013.41 samples/s lr: 7.72e-04 [09/22 14:42:28] lb.utils.events INFO: eta: 11:06:54 iteration: 119799/375342 consumed_samples: 122675200 total_loss: 3.775 time: 0.3398 s/iter data_time: 0.2145 s/iter total_throughput: 3013.41 samples/s lr: 7.71e-04 [09/22 14:43:02] lb.utils.events INFO: eta: 11:06:01 iteration: 119899/375342 consumed_samples: 122777600 total_loss: 3.764 time: 0.3398 s/iter data_time: 0.2295 s/iter total_throughput: 3013.39 samples/s lr: 7.71e-04 [09/22 14:43:36] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0119999 [09/22 14:43:37] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/22 14:43:37] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/22 14:43:41] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0966 s/iter. Inference: 0.1621 s/iter. Eval: 0.0020 s/iter. Total: 0.2607 s/iter. ETA=0:00:09 [09/22 14:43:46] lb.evaluation.evaluator INFO: Inference done 28672/50000. Dataloading: 0.1080 s/iter. Inference: 0.1759 s/iter. Eval: 0.0022 s/iter. Total: 0.2861 s/iter. ETA=0:00:05 [09/22 14:43:51] lb.evaluation.evaluator INFO: Inference done 47104/50000. Dataloading: 0.1211 s/iter. Inference: 0.1686 s/iter. Eval: 0.0020 s/iter. Total: 0.2919 s/iter. ETA=0:00:00 [09/22 14:43:52] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/22 14:43:52] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.596377 (0.000252 s / iter per device, on 8 devices) [09/22 14:43:52] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000148 s / iter per device, on 8 devices) [09/22 14:43:52] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/22 14:43:52] lb.evaluation.utils INFO: copypaste: Acc@1=72.166 [09/22 14:43:52] lb.evaluation.utils INFO: copypaste: Acc@5=91.122 [09/22 14:43:52] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 72.16600, better than last best score 71.92400 @ iteration 114999. [09/22 14:43:52] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/22 14:43:53] lb.utils.events INFO: eta: 11:04:39 iteration: 119999/375342 consumed_samples: 122880000 total_loss: 3.764 time: 0.3398 s/iter data_time: 0.2147 s/iter total_throughput: 3013.39 samples/s lr: 7.71e-04 [09/22 14:44:25] lb.utils.events INFO: eta: 11:08:04 iteration: 120099/375342 consumed_samples: 122982400 total_loss: 3.772 time: 0.3398 s/iter data_time: 0.2215 s/iter total_throughput: 3013.55 samples/s lr: 7.70e-04 [09/22 14:44:59] lb.utils.events INFO: eta: 11:08:44 iteration: 120199/375342 consumed_samples: 123084800 total_loss: 3.762 time: 0.3398 s/iter data_time: 0.2160 s/iter total_throughput: 3013.55 samples/s lr: 7.70e-04 [09/22 14:45:33] lb.utils.events INFO: eta: 11:08:32 iteration: 120299/375342 consumed_samples: 123187200 total_loss: 3.72 time: 0.3398 s/iter data_time: 0.2241 s/iter total_throughput: 3013.53 samples/s lr: 7.70e-04 [09/22 14:46:08] lb.utils.events INFO: eta: 11:08:16 iteration: 120399/375342 consumed_samples: 123289600 total_loss: 3.738 time: 0.3398 s/iter data_time: 0.2337 s/iter total_throughput: 3013.48 samples/s lr: 7.69e-04 [09/22 14:46:42] lb.utils.events INFO: eta: 11:06:42 iteration: 120499/375342 consumed_samples: 123392000 total_loss: 3.739 time: 0.3398 s/iter data_time: 0.2063 s/iter total_throughput: 3013.48 samples/s lr: 7.69e-04 [09/22 14:47:16] lb.utils.events INFO: eta: 11:07:29 iteration: 120599/375342 consumed_samples: 123494400 total_loss: 3.733 time: 0.3398 s/iter data_time: 0.2173 s/iter total_throughput: 3013.47 samples/s lr: 7.69e-04 [09/22 14:47:49] lb.utils.events INFO: eta: 11:08:20 iteration: 120699/375342 consumed_samples: 123596800 total_loss: 3.755 time: 0.3398 s/iter data_time: 0.2096 s/iter total_throughput: 3013.50 samples/s lr: 7.68e-04 [09/22 14:48:24] lb.utils.events INFO: eta: 11:07:58 iteration: 120799/375342 consumed_samples: 123699200 total_loss: 3.74 time: 0.3398 s/iter data_time: 0.2093 s/iter total_throughput: 3013.48 samples/s lr: 7.68e-04 [09/22 14:48:58] lb.utils.events INFO: eta: 11:07:12 iteration: 120899/375342 consumed_samples: 123801600 total_loss: 3.735 time: 0.3398 s/iter data_time: 0.2118 s/iter total_throughput: 3013.47 samples/s lr: 7.67e-04 [09/22 14:49:31] lb.utils.events INFO: eta: 11:06:20 iteration: 120999/375342 consumed_samples: 123904000 total_loss: 3.757 time: 0.3398 s/iter data_time: 0.2159 s/iter total_throughput: 3013.48 samples/s lr: 7.67e-04 [09/22 14:50:05] lb.utils.events INFO: eta: 11:02:56 iteration: 121099/375342 consumed_samples: 124006400 total_loss: 3.775 time: 0.3398 s/iter data_time: 0.2209 s/iter total_throughput: 3013.49 samples/s lr: 7.67e-04 [09/22 14:50:39] lb.utils.events INFO: eta: 11:02:54 iteration: 121199/375342 consumed_samples: 124108800 total_loss: 3.745 time: 0.3398 s/iter data_time: 0.2013 s/iter total_throughput: 3013.51 samples/s lr: 7.66e-04 [09/22 14:51:13] lb.utils.events INFO: eta: 11:04:32 iteration: 121299/375342 consumed_samples: 124211200 total_loss: 3.747 time: 0.3398 s/iter data_time: 0.2102 s/iter total_throughput: 3013.53 samples/s lr: 7.66e-04 [09/22 14:51:47] lb.utils.events INFO: eta: 11:04:31 iteration: 121399/375342 consumed_samples: 124313600 total_loss: 3.755 time: 0.3398 s/iter data_time: 0.2061 s/iter total_throughput: 3013.55 samples/s lr: 7.66e-04 [09/22 14:52:20] lb.utils.events INFO: eta: 11:05:01 iteration: 121499/375342 consumed_samples: 124416000 total_loss: 3.729 time: 0.3398 s/iter data_time: 0.2087 s/iter total_throughput: 3013.59 samples/s lr: 7.65e-04 [09/22 14:52:54] lb.utils.events INFO: eta: 11:04:38 iteration: 121599/375342 consumed_samples: 124518400 total_loss: 3.734 time: 0.3398 s/iter data_time: 0.2451 s/iter total_throughput: 3013.58 samples/s lr: 7.65e-04 [09/22 14:53:28] lb.utils.events INFO: eta: 11:04:17 iteration: 121699/375342 consumed_samples: 124620800 total_loss: 3.747 time: 0.3398 s/iter data_time: 0.2092 s/iter total_throughput: 3013.59 samples/s lr: 7.65e-04 [09/22 14:54:02] lb.utils.events INFO: eta: 11:05:31 iteration: 121799/375342 consumed_samples: 124723200 total_loss: 3.746 time: 0.3398 s/iter data_time: 0.2182 s/iter total_throughput: 3013.60 samples/s lr: 7.64e-04 [09/22 14:54:36] lb.utils.events INFO: eta: 11:05:35 iteration: 121899/375342 consumed_samples: 124825600 total_loss: 3.745 time: 0.3398 s/iter data_time: 0.2308 s/iter total_throughput: 3013.59 samples/s lr: 7.64e-04 [09/22 14:55:09] lb.utils.events INFO: eta: 11:07:04 iteration: 121999/375342 consumed_samples: 124928000 total_loss: 3.74 time: 0.3398 s/iter data_time: 0.2117 s/iter total_throughput: 3013.62 samples/s lr: 7.64e-04 [09/22 14:55:43] lb.utils.events INFO: eta: 11:07:17 iteration: 122099/375342 consumed_samples: 125030400 total_loss: 3.729 time: 0.3398 s/iter data_time: 0.1902 s/iter total_throughput: 3013.63 samples/s lr: 7.63e-04 [09/22 14:56:17] lb.utils.events INFO: eta: 11:06:31 iteration: 122199/375342 consumed_samples: 125132800 total_loss: 3.724 time: 0.3398 s/iter data_time: 0.2067 s/iter total_throughput: 3013.66 samples/s lr: 7.63e-04 [09/22 14:56:51] lb.utils.events INFO: eta: 11:06:04 iteration: 122299/375342 consumed_samples: 125235200 total_loss: 3.743 time: 0.3398 s/iter data_time: 0.2279 s/iter total_throughput: 3013.67 samples/s lr: 7.63e-04 [09/22 14:57:24] lb.utils.events INFO: eta: 11:05:15 iteration: 122399/375342 consumed_samples: 125337600 total_loss: 3.743 time: 0.3398 s/iter data_time: 0.2024 s/iter total_throughput: 3013.71 samples/s lr: 7.62e-04 [09/22 14:57:58] lb.utils.events INFO: eta: 11:04:00 iteration: 122499/375342 consumed_samples: 125440000 total_loss: 3.756 time: 0.3398 s/iter data_time: 0.2198 s/iter total_throughput: 3013.72 samples/s lr: 7.62e-04 [09/22 14:58:32] lb.utils.events INFO: eta: 11:03:39 iteration: 122599/375342 consumed_samples: 125542400 total_loss: 3.752 time: 0.3398 s/iter data_time: 0.1953 s/iter total_throughput: 3013.75 samples/s lr: 7.61e-04 [09/22 14:59:06] lb.utils.events INFO: eta: 11:02:30 iteration: 122699/375342 consumed_samples: 125644800 total_loss: 3.732 time: 0.3398 s/iter data_time: 0.2074 s/iter total_throughput: 3013.75 samples/s lr: 7.61e-04 [09/22 14:59:39] lb.utils.events INFO: eta: 11:02:53 iteration: 122799/375342 consumed_samples: 125747200 total_loss: 3.733 time: 0.3398 s/iter data_time: 0.2054 s/iter total_throughput: 3013.78 samples/s lr: 7.61e-04 [09/22 15:00:13] lb.utils.events INFO: eta: 11:03:26 iteration: 122899/375342 consumed_samples: 125849600 total_loss: 3.732 time: 0.3398 s/iter data_time: 0.2066 s/iter total_throughput: 3013.79 samples/s lr: 7.60e-04 [09/22 15:00:47] lb.utils.events INFO: eta: 11:02:22 iteration: 122999/375342 consumed_samples: 125952000 total_loss: 3.735 time: 0.3398 s/iter data_time: 0.2083 s/iter total_throughput: 3013.79 samples/s lr: 7.60e-04 [09/22 15:01:21] lb.utils.events INFO: eta: 10:59:08 iteration: 123099/375342 consumed_samples: 126054400 total_loss: 3.744 time: 0.3398 s/iter data_time: 0.2093 s/iter total_throughput: 3013.77 samples/s lr: 7.60e-04 [09/22 15:01:55] lb.utils.events INFO: eta: 11:00:14 iteration: 123199/375342 consumed_samples: 126156800 total_loss: 3.736 time: 0.3398 s/iter data_time: 0.2094 s/iter total_throughput: 3013.80 samples/s lr: 7.59e-04 [09/22 15:02:28] lb.utils.events INFO: eta: 11:00:45 iteration: 123299/375342 consumed_samples: 126259200 total_loss: 3.715 time: 0.3398 s/iter data_time: 0.2098 s/iter total_throughput: 3013.84 samples/s lr: 7.59e-04 [09/22 15:03:01] lb.utils.events INFO: eta: 11:02:49 iteration: 123399/375342 consumed_samples: 126361600 total_loss: 3.722 time: 0.3398 s/iter data_time: 0.2002 s/iter total_throughput: 3013.91 samples/s lr: 7.59e-04 [09/22 15:03:34] lb.utils.events INFO: eta: 11:04:10 iteration: 123499/375342 consumed_samples: 126464000 total_loss: 3.738 time: 0.3398 s/iter data_time: 0.2012 s/iter total_throughput: 3013.98 samples/s lr: 7.58e-04 [09/22 15:04:08] lb.utils.events INFO: eta: 11:03:32 iteration: 123599/375342 consumed_samples: 126566400 total_loss: 3.738 time: 0.3397 s/iter data_time: 0.2086 s/iter total_throughput: 3014.00 samples/s lr: 7.58e-04 [09/22 15:04:41] lb.utils.events INFO: eta: 11:04:20 iteration: 123699/375342 consumed_samples: 126668800 total_loss: 3.743 time: 0.3397 s/iter data_time: 0.2048 s/iter total_throughput: 3014.05 samples/s lr: 7.58e-04 [09/22 15:05:15] lb.utils.events INFO: eta: 11:02:35 iteration: 123799/375342 consumed_samples: 126771200 total_loss: 3.736 time: 0.3397 s/iter data_time: 0.2237 s/iter total_throughput: 3014.06 samples/s lr: 7.57e-04 [09/22 15:05:49] lb.utils.events INFO: eta: 11:01:32 iteration: 123899/375342 consumed_samples: 126873600 total_loss: 3.721 time: 0.3397 s/iter data_time: 0.2046 s/iter total_throughput: 3014.10 samples/s lr: 7.57e-04 [09/22 15:06:23] lb.utils.events INFO: eta: 10:59:06 iteration: 123999/375342 consumed_samples: 126976000 total_loss: 3.721 time: 0.3397 s/iter data_time: 0.2111 s/iter total_throughput: 3014.09 samples/s lr: 7.56e-04 [09/22 15:06:56] lb.utils.events INFO: eta: 10:59:09 iteration: 124099/375342 consumed_samples: 127078400 total_loss: 3.711 time: 0.3397 s/iter data_time: 0.2143 s/iter total_throughput: 3014.13 samples/s lr: 7.56e-04 [09/22 15:07:30] lb.utils.events INFO: eta: 10:57:51 iteration: 124199/375342 consumed_samples: 127180800 total_loss: 3.717 time: 0.3397 s/iter data_time: 0.2139 s/iter total_throughput: 3014.16 samples/s lr: 7.56e-04 [09/22 15:08:03] lb.utils.events INFO: eta: 10:57:18 iteration: 124299/375342 consumed_samples: 127283200 total_loss: 3.725 time: 0.3397 s/iter data_time: 0.2142 s/iter total_throughput: 3014.20 samples/s lr: 7.55e-04 [09/22 15:08:37] lb.utils.events INFO: eta: 10:54:36 iteration: 124399/375342 consumed_samples: 127385600 total_loss: 3.72 time: 0.3397 s/iter data_time: 0.2165 s/iter total_throughput: 3014.20 samples/s lr: 7.55e-04 [09/22 15:09:11] lb.utils.events INFO: eta: 10:53:11 iteration: 124499/375342 consumed_samples: 127488000 total_loss: 3.742 time: 0.3397 s/iter data_time: 0.2106 s/iter total_throughput: 3014.23 samples/s lr: 7.55e-04 [09/22 15:09:44] lb.utils.events INFO: eta: 10:54:17 iteration: 124599/375342 consumed_samples: 127590400 total_loss: 3.756 time: 0.3397 s/iter data_time: 0.2156 s/iter total_throughput: 3014.26 samples/s lr: 7.54e-04 [09/22 15:10:18] lb.utils.events INFO: eta: 10:53:19 iteration: 124699/375342 consumed_samples: 127692800 total_loss: 3.753 time: 0.3397 s/iter data_time: 0.2044 s/iter total_throughput: 3014.29 samples/s lr: 7.54e-04 [09/22 15:10:52] lb.utils.events INFO: eta: 10:54:21 iteration: 124799/375342 consumed_samples: 127795200 total_loss: 3.734 time: 0.3397 s/iter data_time: 0.2020 s/iter total_throughput: 3014.30 samples/s lr: 7.54e-04 [09/22 15:11:25] lb.utils.events INFO: eta: 10:55:03 iteration: 124899/375342 consumed_samples: 127897600 total_loss: 3.717 time: 0.3397 s/iter data_time: 0.2070 s/iter total_throughput: 3014.34 samples/s lr: 7.53e-04 [09/22 15:11:59] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0124999 [09/22 15:11:59] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/22 15:11:59] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/22 15:12:03] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0981 s/iter. Inference: 0.1652 s/iter. Eval: 0.0020 s/iter. Total: 0.2652 s/iter. ETA=0:00:09 [09/22 15:12:09] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1176 s/iter. Inference: 0.1764 s/iter. Eval: 0.0020 s/iter. Total: 0.2961 s/iter. ETA=0:00:05 [09/22 15:12:14] lb.evaluation.evaluator INFO: Inference done 49152/50000. Dataloading: 0.1118 s/iter. Inference: 0.1687 s/iter. Eval: 0.0020 s/iter. Total: 0.2826 s/iter. ETA=0:00:00 [09/22 15:12:14] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/22 15:12:14] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.513201 (0.000250 s / iter per device, on 8 devices) [09/22 15:12:14] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000149 s / iter per device, on 8 devices) [09/22 15:12:14] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/22 15:12:14] lb.evaluation.utils INFO: copypaste: Acc@1=72.49600000000001 [09/22 15:12:14] lb.evaluation.utils INFO: copypaste: Acc@5=91.196 [09/22 15:12:14] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 72.49600, better than last best score 72.16600 @ iteration 119999. [09/22 15:12:14] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/22 15:12:15] lb.utils.events INFO: eta: 10:55:45 iteration: 124999/375342 consumed_samples: 128000000 total_loss: 3.748 time: 0.3397 s/iter data_time: 0.2094 s/iter total_throughput: 3014.37 samples/s lr: 7.53e-04 [09/22 15:12:46] lb.utils.events INFO: eta: 10:58:49 iteration: 125099/375342 consumed_samples: 128102400 total_loss: 3.742 time: 0.3397 s/iter data_time: 0.2419 s/iter total_throughput: 3014.58 samples/s lr: 7.53e-04 [09/22 15:13:20] lb.utils.events INFO: eta: 11:02:24 iteration: 125199/375342 consumed_samples: 128204800 total_loss: 3.712 time: 0.3397 s/iter data_time: 0.2162 s/iter total_throughput: 3014.58 samples/s lr: 7.52e-04 [09/22 15:13:54] lb.utils.events INFO: eta: 11:00:41 iteration: 125299/375342 consumed_samples: 128307200 total_loss: 3.715 time: 0.3397 s/iter data_time: 0.2260 s/iter total_throughput: 3014.58 samples/s lr: 7.52e-04 [09/22 15:14:28] lb.utils.events INFO: eta: 10:59:30 iteration: 125399/375342 consumed_samples: 128409600 total_loss: 3.736 time: 0.3397 s/iter data_time: 0.2287 s/iter total_throughput: 3014.58 samples/s lr: 7.51e-04 [09/22 15:15:01] lb.utils.events INFO: eta: 10:59:59 iteration: 125499/375342 consumed_samples: 128512000 total_loss: 3.749 time: 0.3397 s/iter data_time: 0.2135 s/iter total_throughput: 3014.61 samples/s lr: 7.51e-04 [09/22 15:15:35] lb.utils.events INFO: eta: 10:58:11 iteration: 125599/375342 consumed_samples: 128614400 total_loss: 3.732 time: 0.3397 s/iter data_time: 0.2069 s/iter total_throughput: 3014.65 samples/s lr: 7.51e-04 [09/22 15:16:08] lb.utils.events INFO: eta: 10:58:30 iteration: 125699/375342 consumed_samples: 128716800 total_loss: 3.736 time: 0.3397 s/iter data_time: 0.2122 s/iter total_throughput: 3014.69 samples/s lr: 7.50e-04 [09/22 15:16:41] lb.utils.events INFO: eta: 10:59:11 iteration: 125799/375342 consumed_samples: 128819200 total_loss: 3.722 time: 0.3397 s/iter data_time: 0.2032 s/iter total_throughput: 3014.77 samples/s lr: 7.50e-04 [09/22 15:17:14] lb.utils.events INFO: eta: 10:57:43 iteration: 125899/375342 consumed_samples: 128921600 total_loss: 3.72 time: 0.3397 s/iter data_time: 0.2084 s/iter total_throughput: 3014.83 samples/s lr: 7.50e-04 [09/22 15:17:48] lb.utils.events INFO: eta: 10:58:41 iteration: 125999/375342 consumed_samples: 129024000 total_loss: 3.751 time: 0.3397 s/iter data_time: 0.2069 s/iter total_throughput: 3014.86 samples/s lr: 7.49e-04 [09/22 15:18:21] lb.utils.events INFO: eta: 10:56:07 iteration: 126099/375342 consumed_samples: 129126400 total_loss: 3.745 time: 0.3396 s/iter data_time: 0.2068 s/iter total_throughput: 3014.91 samples/s lr: 7.49e-04 [09/22 15:18:55] lb.utils.events INFO: eta: 10:52:43 iteration: 126199/375342 consumed_samples: 129228800 total_loss: 3.728 time: 0.3396 s/iter data_time: 0.2173 s/iter total_throughput: 3014.92 samples/s lr: 7.49e-04 [09/22 15:19:29] lb.utils.events INFO: eta: 10:51:38 iteration: 126299/375342 consumed_samples: 129331200 total_loss: 3.737 time: 0.3396 s/iter data_time: 0.2213 s/iter total_throughput: 3014.89 samples/s lr: 7.48e-04 [09/22 15:20:04] lb.utils.events INFO: eta: 10:51:50 iteration: 126399/375342 consumed_samples: 129433600 total_loss: 3.724 time: 0.3397 s/iter data_time: 0.2201 s/iter total_throughput: 3014.81 samples/s lr: 7.48e-04 [09/22 15:20:40] lb.utils.events INFO: eta: 10:48:52 iteration: 126499/375342 consumed_samples: 129536000 total_loss: 3.718 time: 0.3397 s/iter data_time: 0.2182 s/iter total_throughput: 3014.72 samples/s lr: 7.48e-04 [09/22 15:21:15] lb.utils.events INFO: eta: 10:48:51 iteration: 126599/375342 consumed_samples: 129638400 total_loss: 3.713 time: 0.3397 s/iter data_time: 0.2259 s/iter total_throughput: 3014.60 samples/s lr: 7.47e-04 [09/22 15:21:51] lb.utils.events INFO: eta: 10:47:13 iteration: 126699/375342 consumed_samples: 129740800 total_loss: 3.7 time: 0.3397 s/iter data_time: 0.2385 s/iter total_throughput: 3014.49 samples/s lr: 7.47e-04 [09/22 15:22:26] lb.utils.events INFO: eta: 10:47:26 iteration: 126799/375342 consumed_samples: 129843200 total_loss: 3.727 time: 0.3397 s/iter data_time: 0.2144 s/iter total_throughput: 3014.42 samples/s lr: 7.46e-04 [09/22 15:23:01] lb.utils.events INFO: eta: 10:45:24 iteration: 126899/375342 consumed_samples: 129945600 total_loss: 3.722 time: 0.3397 s/iter data_time: 0.2343 s/iter total_throughput: 3014.31 samples/s lr: 7.46e-04 [09/22 15:23:36] lb.utils.events INFO: eta: 10:43:45 iteration: 126999/375342 consumed_samples: 130048000 total_loss: 3.723 time: 0.3397 s/iter data_time: 0.2217 s/iter total_throughput: 3014.24 samples/s lr: 7.46e-04 [09/22 15:24:12] lb.utils.events INFO: eta: 10:43:00 iteration: 127099/375342 consumed_samples: 130150400 total_loss: 3.729 time: 0.3397 s/iter data_time: 0.2254 s/iter total_throughput: 3014.15 samples/s lr: 7.45e-04 [09/22 15:24:47] lb.utils.events INFO: eta: 10:42:06 iteration: 127199/375342 consumed_samples: 130252800 total_loss: 3.715 time: 0.3397 s/iter data_time: 0.2212 s/iter total_throughput: 3014.05 samples/s lr: 7.45e-04 [09/22 15:25:22] lb.utils.events INFO: eta: 10:40:07 iteration: 127299/375342 consumed_samples: 130355200 total_loss: 3.685 time: 0.3398 s/iter data_time: 0.2268 s/iter total_throughput: 3013.96 samples/s lr: 7.45e-04 [09/22 15:25:57] lb.utils.events INFO: eta: 10:36:44 iteration: 127399/375342 consumed_samples: 130457600 total_loss: 3.687 time: 0.3398 s/iter data_time: 0.2236 s/iter total_throughput: 3013.89 samples/s lr: 7.44e-04 [09/22 15:26:32] lb.utils.events INFO: eta: 10:37:48 iteration: 127499/375342 consumed_samples: 130560000 total_loss: 3.733 time: 0.3398 s/iter data_time: 0.2279 s/iter total_throughput: 3013.82 samples/s lr: 7.44e-04 [09/22 15:27:07] lb.utils.events INFO: eta: 10:37:29 iteration: 127599/375342 consumed_samples: 130662400 total_loss: 3.744 time: 0.3398 s/iter data_time: 0.2257 s/iter total_throughput: 3013.75 samples/s lr: 7.44e-04 [09/22 15:27:42] lb.utils.events INFO: eta: 10:38:33 iteration: 127699/375342 consumed_samples: 130764800 total_loss: 3.741 time: 0.3398 s/iter data_time: 0.2226 s/iter total_throughput: 3013.68 samples/s lr: 7.43e-04 [09/22 15:28:17] lb.utils.events INFO: eta: 10:35:01 iteration: 127799/375342 consumed_samples: 130867200 total_loss: 3.731 time: 0.3398 s/iter data_time: 0.2316 s/iter total_throughput: 3013.60 samples/s lr: 7.43e-04 [09/22 15:28:52] lb.utils.events INFO: eta: 10:39:49 iteration: 127899/375342 consumed_samples: 130969600 total_loss: 3.726 time: 0.3398 s/iter data_time: 0.2082 s/iter total_throughput: 3013.55 samples/s lr: 7.42e-04 [09/22 15:29:27] lb.utils.events INFO: eta: 10:40:43 iteration: 127999/375342 consumed_samples: 131072000 total_loss: 3.731 time: 0.3398 s/iter data_time: 0.2328 s/iter total_throughput: 3013.48 samples/s lr: 7.42e-04 [09/22 15:30:03] lb.utils.events INFO: eta: 10:39:18 iteration: 128099/375342 consumed_samples: 131174400 total_loss: 3.726 time: 0.3398 s/iter data_time: 0.2262 s/iter total_throughput: 3013.36 samples/s lr: 7.42e-04 [09/22 15:30:38] lb.utils.events INFO: eta: 10:39:29 iteration: 128199/375342 consumed_samples: 131276800 total_loss: 3.723 time: 0.3398 s/iter data_time: 0.2140 s/iter total_throughput: 3013.29 samples/s lr: 7.41e-04 [09/22 15:31:13] lb.utils.events INFO: eta: 10:39:32 iteration: 128299/375342 consumed_samples: 131379200 total_loss: 3.706 time: 0.3398 s/iter data_time: 0.2200 s/iter total_throughput: 3013.23 samples/s lr: 7.41e-04 [09/22 15:31:48] lb.utils.events INFO: eta: 10:39:24 iteration: 128399/375342 consumed_samples: 131481600 total_loss: 3.709 time: 0.3398 s/iter data_time: 0.2344 s/iter total_throughput: 3013.15 samples/s lr: 7.41e-04 [09/22 15:32:23] lb.utils.events INFO: eta: 10:38:32 iteration: 128499/375342 consumed_samples: 131584000 total_loss: 3.707 time: 0.3399 s/iter data_time: 0.2299 s/iter total_throughput: 3013.05 samples/s lr: 7.40e-04 [09/22 15:32:59] lb.utils.events INFO: eta: 10:37:50 iteration: 128599/375342 consumed_samples: 131686400 total_loss: 3.722 time: 0.3399 s/iter data_time: 0.2311 s/iter total_throughput: 3012.96 samples/s lr: 7.40e-04 [09/22 15:33:34] lb.utils.events INFO: eta: 10:35:12 iteration: 128699/375342 consumed_samples: 131788800 total_loss: 3.743 time: 0.3399 s/iter data_time: 0.2187 s/iter total_throughput: 3012.89 samples/s lr: 7.40e-04 [09/22 15:34:09] lb.utils.events INFO: eta: 10:35:35 iteration: 128799/375342 consumed_samples: 131891200 total_loss: 3.74 time: 0.3399 s/iter data_time: 0.2301 s/iter total_throughput: 3012.82 samples/s lr: 7.39e-04 [09/22 15:34:43] lb.utils.events INFO: eta: 10:35:20 iteration: 128899/375342 consumed_samples: 131993600 total_loss: 3.724 time: 0.3399 s/iter data_time: 0.2253 s/iter total_throughput: 3012.78 samples/s lr: 7.39e-04 [09/22 15:35:18] lb.utils.events INFO: eta: 10:34:49 iteration: 128999/375342 consumed_samples: 132096000 total_loss: 3.697 time: 0.3399 s/iter data_time: 0.2281 s/iter total_throughput: 3012.70 samples/s lr: 7.38e-04 [09/22 15:35:54] lb.utils.events INFO: eta: 10:34:55 iteration: 129099/375342 consumed_samples: 132198400 total_loss: 3.7 time: 0.3399 s/iter data_time: 0.2180 s/iter total_throughput: 3012.63 samples/s lr: 7.38e-04 [09/22 15:36:28] lb.utils.events INFO: eta: 10:34:32 iteration: 129199/375342 consumed_samples: 132300800 total_loss: 3.708 time: 0.3399 s/iter data_time: 0.2158 s/iter total_throughput: 3012.57 samples/s lr: 7.38e-04 [09/22 15:37:03] lb.utils.events INFO: eta: 10:34:16 iteration: 129299/375342 consumed_samples: 132403200 total_loss: 3.704 time: 0.3399 s/iter data_time: 0.2241 s/iter total_throughput: 3012.51 samples/s lr: 7.37e-04 [09/22 15:37:39] lb.utils.events INFO: eta: 10:33:48 iteration: 129399/375342 consumed_samples: 132505600 total_loss: 3.722 time: 0.3399 s/iter data_time: 0.2264 s/iter total_throughput: 3012.41 samples/s lr: 7.37e-04 [09/22 15:38:14] lb.utils.events INFO: eta: 10:34:01 iteration: 129499/375342 consumed_samples: 132608000 total_loss: 3.737 time: 0.3399 s/iter data_time: 0.2188 s/iter total_throughput: 3012.35 samples/s lr: 7.37e-04 [09/22 15:38:49] lb.utils.events INFO: eta: 10:33:45 iteration: 129599/375342 consumed_samples: 132710400 total_loss: 3.726 time: 0.3399 s/iter data_time: 0.2196 s/iter total_throughput: 3012.27 samples/s lr: 7.36e-04 [09/22 15:39:24] lb.utils.events INFO: eta: 10:34:19 iteration: 129699/375342 consumed_samples: 132812800 total_loss: 3.728 time: 0.3399 s/iter data_time: 0.2166 s/iter total_throughput: 3012.21 samples/s lr: 7.36e-04 [09/22 15:39:58] lb.utils.events INFO: eta: 10:33:41 iteration: 129799/375342 consumed_samples: 132915200 total_loss: 3.717 time: 0.3400 s/iter data_time: 0.2215 s/iter total_throughput: 3012.17 samples/s lr: 7.36e-04 [09/22 15:40:33] lb.utils.events INFO: eta: 10:32:33 iteration: 129899/375342 consumed_samples: 133017600 total_loss: 3.715 time: 0.3400 s/iter data_time: 0.2261 s/iter total_throughput: 3012.10 samples/s lr: 7.35e-04 [09/22 15:41:08] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0129999 [09/22 15:41:09] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/22 15:41:09] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/22 15:41:13] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0958 s/iter. Inference: 0.1614 s/iter. Eval: 0.0020 s/iter. Total: 0.2593 s/iter. ETA=0:00:09 [09/22 15:41:18] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1319 s/iter. Inference: 0.1661 s/iter. Eval: 0.0020 s/iter. Total: 0.3001 s/iter. ETA=0:00:05 [09/22 15:41:24] lb.evaluation.evaluator INFO: Inference done 50000/50000. Dataloading: 0.1191 s/iter. Inference: 0.1651 s/iter. Eval: 0.0020 s/iter. Total: 0.2862 s/iter. ETA=0:00:00 [09/22 15:41:24] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/22 15:41:24] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.597294 (0.000252 s / iter per device, on 8 devices) [09/22 15:41:24] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000145 s / iter per device, on 8 devices) [09/22 15:41:24] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/22 15:41:24] lb.evaluation.utils INFO: copypaste: Acc@1=72.898 [09/22 15:41:24] lb.evaluation.utils INFO: copypaste: Acc@5=91.418 [09/22 15:41:24] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 72.89800, better than last best score 72.49600 @ iteration 124999. [09/22 15:41:24] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/22 15:41:25] lb.utils.events INFO: eta: 10:31:47 iteration: 129999/375342 consumed_samples: 133120000 total_loss: 3.734 time: 0.3400 s/iter data_time: 0.2299 s/iter total_throughput: 3012.05 samples/s lr: 7.35e-04 [09/22 15:41:57] lb.utils.events INFO: eta: 10:33:17 iteration: 130099/375342 consumed_samples: 133222400 total_loss: 3.727 time: 0.3400 s/iter data_time: 0.2243 s/iter total_throughput: 3012.14 samples/s lr: 7.34e-04 [09/22 15:42:32] lb.utils.events INFO: eta: 10:33:48 iteration: 130199/375342 consumed_samples: 133324800 total_loss: 3.722 time: 0.3400 s/iter data_time: 0.2202 s/iter total_throughput: 3012.09 samples/s lr: 7.34e-04 [09/22 15:43:07] lb.utils.events INFO: eta: 10:34:07 iteration: 130299/375342 consumed_samples: 133427200 total_loss: 3.719 time: 0.3400 s/iter data_time: 0.2273 s/iter total_throughput: 3012.02 samples/s lr: 7.34e-04 [09/22 15:43:42] lb.utils.events INFO: eta: 10:35:21 iteration: 130399/375342 consumed_samples: 133529600 total_loss: 3.704 time: 0.3400 s/iter data_time: 0.2324 s/iter total_throughput: 3011.94 samples/s lr: 7.33e-04 [09/22 15:44:18] lb.utils.events INFO: eta: 10:34:37 iteration: 130499/375342 consumed_samples: 133632000 total_loss: 3.716 time: 0.3400 s/iter data_time: 0.2387 s/iter total_throughput: 3011.85 samples/s lr: 7.33e-04 [09/22 15:44:53] lb.utils.events INFO: eta: 10:35:33 iteration: 130599/375342 consumed_samples: 133734400 total_loss: 3.719 time: 0.3400 s/iter data_time: 0.2251 s/iter total_throughput: 3011.78 samples/s lr: 7.33e-04 [09/22 15:45:28] lb.utils.events INFO: eta: 10:35:18 iteration: 130699/375342 consumed_samples: 133836800 total_loss: 3.716 time: 0.3400 s/iter data_time: 0.2328 s/iter total_throughput: 3011.71 samples/s lr: 7.32e-04 [09/22 15:46:03] lb.utils.events INFO: eta: 10:35:35 iteration: 130799/375342 consumed_samples: 133939200 total_loss: 3.716 time: 0.3400 s/iter data_time: 0.2133 s/iter total_throughput: 3011.64 samples/s lr: 7.32e-04 [09/22 15:46:38] lb.utils.events INFO: eta: 10:35:16 iteration: 130899/375342 consumed_samples: 134041600 total_loss: 3.716 time: 0.3400 s/iter data_time: 0.2275 s/iter total_throughput: 3011.58 samples/s lr: 7.31e-04 [09/22 15:47:13] lb.utils.events INFO: eta: 10:34:47 iteration: 130999/375342 consumed_samples: 134144000 total_loss: 3.723 time: 0.3400 s/iter data_time: 0.2280 s/iter total_throughput: 3011.50 samples/s lr: 7.31e-04 [09/22 15:47:48] lb.utils.events INFO: eta: 10:30:39 iteration: 131099/375342 consumed_samples: 134246400 total_loss: 3.709 time: 0.3400 s/iter data_time: 0.2323 s/iter total_throughput: 3011.40 samples/s lr: 7.31e-04 [09/22 15:48:23] lb.utils.events INFO: eta: 10:30:34 iteration: 131199/375342 consumed_samples: 134348800 total_loss: 3.702 time: 0.3400 s/iter data_time: 0.2195 s/iter total_throughput: 3011.36 samples/s lr: 7.30e-04 [09/22 15:48:58] lb.utils.events INFO: eta: 10:29:37 iteration: 131299/375342 consumed_samples: 134451200 total_loss: 3.713 time: 0.3401 s/iter data_time: 0.2110 s/iter total_throughput: 3011.30 samples/s lr: 7.30e-04 [09/22 15:49:33] lb.utils.events INFO: eta: 10:27:43 iteration: 131399/375342 consumed_samples: 134553600 total_loss: 3.704 time: 0.3401 s/iter data_time: 0.2231 s/iter total_throughput: 3011.23 samples/s lr: 7.30e-04 [09/22 15:50:08] lb.utils.events INFO: eta: 10:27:18 iteration: 131499/375342 consumed_samples: 134656000 total_loss: 3.702 time: 0.3401 s/iter data_time: 0.2185 s/iter total_throughput: 3011.17 samples/s lr: 7.29e-04 [09/22 15:50:43] lb.utils.events INFO: eta: 10:26:35 iteration: 131599/375342 consumed_samples: 134758400 total_loss: 3.713 time: 0.3401 s/iter data_time: 0.2304 s/iter total_throughput: 3011.08 samples/s lr: 7.29e-04 [09/22 15:51:18] lb.utils.events INFO: eta: 10:26:41 iteration: 131699/375342 consumed_samples: 134860800 total_loss: 3.721 time: 0.3401 s/iter data_time: 0.2300 s/iter total_throughput: 3011.02 samples/s lr: 7.29e-04 [09/22 15:51:53] lb.utils.events INFO: eta: 10:25:51 iteration: 131799/375342 consumed_samples: 134963200 total_loss: 3.726 time: 0.3401 s/iter data_time: 0.2223 s/iter total_throughput: 3010.94 samples/s lr: 7.28e-04 [09/22 15:52:28] lb.utils.events INFO: eta: 10:25:33 iteration: 131899/375342 consumed_samples: 135065600 total_loss: 3.703 time: 0.3401 s/iter data_time: 0.2295 s/iter total_throughput: 3010.87 samples/s lr: 7.28e-04 [09/22 15:53:03] lb.utils.events INFO: eta: 10:26:15 iteration: 131999/375342 consumed_samples: 135168000 total_loss: 3.703 time: 0.3401 s/iter data_time: 0.2137 s/iter total_throughput: 3010.80 samples/s lr: 7.27e-04 [09/22 15:53:39] lb.utils.events INFO: eta: 10:27:07 iteration: 132099/375342 consumed_samples: 135270400 total_loss: 3.715 time: 0.3401 s/iter data_time: 0.2242 s/iter total_throughput: 3010.72 samples/s lr: 7.27e-04 [09/22 15:54:14] lb.utils.events INFO: eta: 10:24:57 iteration: 132199/375342 consumed_samples: 135372800 total_loss: 3.712 time: 0.3401 s/iter data_time: 0.2174 s/iter total_throughput: 3010.63 samples/s lr: 7.27e-04 [09/22 15:54:49] lb.utils.events INFO: eta: 10:25:38 iteration: 132299/375342 consumed_samples: 135475200 total_loss: 3.731 time: 0.3401 s/iter data_time: 0.2282 s/iter total_throughput: 3010.58 samples/s lr: 7.26e-04 [09/22 15:55:24] lb.utils.events INFO: eta: 10:25:52 iteration: 132399/375342 consumed_samples: 135577600 total_loss: 3.729 time: 0.3401 s/iter data_time: 0.2197 s/iter total_throughput: 3010.52 samples/s lr: 7.26e-04 [09/22 15:55:58] lb.utils.events INFO: eta: 10:26:55 iteration: 132499/375342 consumed_samples: 135680000 total_loss: 3.713 time: 0.3401 s/iter data_time: 0.2219 s/iter total_throughput: 3010.48 samples/s lr: 7.26e-04 [09/22 15:56:33] lb.utils.events INFO: eta: 10:26:42 iteration: 132599/375342 consumed_samples: 135782400 total_loss: 3.715 time: 0.3401 s/iter data_time: 0.2186 s/iter total_throughput: 3010.45 samples/s lr: 7.25e-04 [09/22 15:57:08] lb.utils.events INFO: eta: 10:26:37 iteration: 132699/375342 consumed_samples: 135884800 total_loss: 3.717 time: 0.3402 s/iter data_time: 0.2240 s/iter total_throughput: 3010.38 samples/s lr: 7.25e-04 [09/22 15:57:43] lb.utils.events INFO: eta: 10:26:01 iteration: 132799/375342 consumed_samples: 135987200 total_loss: 3.7 time: 0.3402 s/iter data_time: 0.2164 s/iter total_throughput: 3010.30 samples/s lr: 7.24e-04 [09/22 15:58:18] lb.utils.events INFO: eta: 10:25:29 iteration: 132899/375342 consumed_samples: 136089600 total_loss: 3.711 time: 0.3402 s/iter data_time: 0.2319 s/iter total_throughput: 3010.22 samples/s lr: 7.24e-04 [09/22 15:58:54] lb.utils.events INFO: eta: 10:24:59 iteration: 132999/375342 consumed_samples: 136192000 total_loss: 3.711 time: 0.3402 s/iter data_time: 0.2216 s/iter total_throughput: 3010.13 samples/s lr: 7.24e-04 [09/22 15:59:29] lb.utils.events INFO: eta: 10:25:36 iteration: 133099/375342 consumed_samples: 136294400 total_loss: 3.712 time: 0.3402 s/iter data_time: 0.2260 s/iter total_throughput: 3010.07 samples/s lr: 7.23e-04 [09/22 16:00:03] lb.utils.events INFO: eta: 10:27:22 iteration: 133199/375342 consumed_samples: 136396800 total_loss: 3.73 time: 0.3402 s/iter data_time: 0.2172 s/iter total_throughput: 3010.06 samples/s lr: 7.23e-04 [09/22 16:00:37] lb.utils.events INFO: eta: 10:26:54 iteration: 133299/375342 consumed_samples: 136499200 total_loss: 3.737 time: 0.3402 s/iter data_time: 0.2188 s/iter total_throughput: 3010.02 samples/s lr: 7.23e-04 [09/22 16:01:12] lb.utils.events INFO: eta: 10:24:58 iteration: 133399/375342 consumed_samples: 136601600 total_loss: 3.734 time: 0.3402 s/iter data_time: 0.2135 s/iter total_throughput: 3009.96 samples/s lr: 7.22e-04 [09/22 16:01:47] lb.utils.events INFO: eta: 10:23:25 iteration: 133499/375342 consumed_samples: 136704000 total_loss: 3.719 time: 0.3402 s/iter data_time: 0.2209 s/iter total_throughput: 3009.88 samples/s lr: 7.22e-04 [09/22 16:02:23] lb.utils.events INFO: eta: 10:23:10 iteration: 133599/375342 consumed_samples: 136806400 total_loss: 3.702 time: 0.3402 s/iter data_time: 0.2219 s/iter total_throughput: 3009.81 samples/s lr: 7.21e-04 [09/22 16:02:58] lb.utils.events INFO: eta: 10:22:54 iteration: 133699/375342 consumed_samples: 136908800 total_loss: 3.703 time: 0.3402 s/iter data_time: 0.2228 s/iter total_throughput: 3009.74 samples/s lr: 7.21e-04 [09/22 16:03:33] lb.utils.events INFO: eta: 10:23:33 iteration: 133799/375342 consumed_samples: 137011200 total_loss: 3.706 time: 0.3402 s/iter data_time: 0.2121 s/iter total_throughput: 3009.67 samples/s lr: 7.21e-04 [09/22 16:04:07] lb.utils.events INFO: eta: 10:23:47 iteration: 133899/375342 consumed_samples: 137113600 total_loss: 3.698 time: 0.3402 s/iter data_time: 0.2228 s/iter total_throughput: 3009.62 samples/s lr: 7.20e-04 [09/22 16:04:42] lb.utils.events INFO: eta: 10:23:13 iteration: 133999/375342 consumed_samples: 137216000 total_loss: 3.681 time: 0.3402 s/iter data_time: 0.2116 s/iter total_throughput: 3009.57 samples/s lr: 7.20e-04 [09/22 16:05:17] lb.utils.events INFO: eta: 10:25:02 iteration: 134099/375342 consumed_samples: 137318400 total_loss: 3.706 time: 0.3403 s/iter data_time: 0.2137 s/iter total_throughput: 3009.55 samples/s lr: 7.20e-04 [09/22 16:05:51] lb.utils.events INFO: eta: 10:25:30 iteration: 134199/375342 consumed_samples: 137420800 total_loss: 3.732 time: 0.3403 s/iter data_time: 0.2134 s/iter total_throughput: 3009.50 samples/s lr: 7.19e-04 [09/22 16:06:26] lb.utils.events INFO: eta: 10:26:57 iteration: 134299/375342 consumed_samples: 137523200 total_loss: 3.7 time: 0.3403 s/iter data_time: 0.2302 s/iter total_throughput: 3009.44 samples/s lr: 7.19e-04 [09/22 16:07:01] lb.utils.events INFO: eta: 10:26:20 iteration: 134399/375342 consumed_samples: 137625600 total_loss: 3.675 time: 0.3403 s/iter data_time: 0.2203 s/iter total_throughput: 3009.39 samples/s lr: 7.18e-04 [09/22 16:07:36] lb.utils.events INFO: eta: 10:25:26 iteration: 134499/375342 consumed_samples: 137728000 total_loss: 3.689 time: 0.3403 s/iter data_time: 0.2173 s/iter total_throughput: 3009.34 samples/s lr: 7.18e-04 [09/22 16:08:11] lb.utils.events INFO: eta: 10:25:25 iteration: 134599/375342 consumed_samples: 137830400 total_loss: 3.704 time: 0.3403 s/iter data_time: 0.2259 s/iter total_throughput: 3009.30 samples/s lr: 7.18e-04 [09/22 16:08:46] lb.utils.events INFO: eta: 10:24:19 iteration: 134699/375342 consumed_samples: 137932800 total_loss: 3.712 time: 0.3403 s/iter data_time: 0.2091 s/iter total_throughput: 3009.25 samples/s lr: 7.17e-04 [09/22 16:09:20] lb.utils.events INFO: eta: 10:23:50 iteration: 134799/375342 consumed_samples: 138035200 total_loss: 3.723 time: 0.3403 s/iter data_time: 0.2346 s/iter total_throughput: 3009.19 samples/s lr: 7.17e-04 [09/22 16:09:55] lb.utils.events INFO: eta: 10:22:43 iteration: 134899/375342 consumed_samples: 138137600 total_loss: 3.707 time: 0.3403 s/iter data_time: 0.2115 s/iter total_throughput: 3009.15 samples/s lr: 7.17e-04 [09/22 16:10:31] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0134999 [09/22 16:10:31] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/22 16:10:31] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/22 16:10:35] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0968 s/iter. Inference: 0.1593 s/iter. Eval: 0.0019 s/iter. Total: 0.2581 s/iter. ETA=0:00:09 [09/22 16:10:41] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1185 s/iter. Inference: 0.1766 s/iter. Eval: 0.0020 s/iter. Total: 0.2971 s/iter. ETA=0:00:05 [09/22 16:10:46] lb.evaluation.evaluator INFO: Inference done 49152/50000. Dataloading: 0.1125 s/iter. Inference: 0.1689 s/iter. Eval: 0.0020 s/iter. Total: 0.2835 s/iter. ETA=0:00:00 [09/22 16:10:46] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/22 16:10:46] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.557946 (0.000251 s / iter per device, on 8 devices) [09/22 16:10:46] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000149 s / iter per device, on 8 devices) [09/22 16:10:46] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/22 16:10:46] lb.evaluation.utils INFO: copypaste: Acc@1=73.11800000000001 [09/22 16:10:46] lb.evaluation.utils INFO: copypaste: Acc@5=91.53 [09/22 16:10:46] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 73.11800, better than last best score 72.89800 @ iteration 129999. [09/22 16:10:46] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/22 16:10:47] lb.utils.events INFO: eta: 10:20:53 iteration: 134999/375342 consumed_samples: 138240000 total_loss: 3.705 time: 0.3403 s/iter data_time: 0.2230 s/iter total_throughput: 3009.06 samples/s lr: 7.16e-04 [09/22 16:11:20] lb.utils.events INFO: eta: 10:22:29 iteration: 135099/375342 consumed_samples: 138342400 total_loss: 3.714 time: 0.3403 s/iter data_time: 0.2277 s/iter total_throughput: 3009.14 samples/s lr: 7.16e-04 [09/22 16:11:55] lb.utils.events INFO: eta: 10:19:20 iteration: 135199/375342 consumed_samples: 138444800 total_loss: 3.699 time: 0.3403 s/iter data_time: 0.2188 s/iter total_throughput: 3009.09 samples/s lr: 7.15e-04 [09/22 16:12:29] lb.utils.events INFO: eta: 10:19:40 iteration: 135299/375342 consumed_samples: 138547200 total_loss: 3.688 time: 0.3403 s/iter data_time: 0.2131 s/iter total_throughput: 3009.05 samples/s lr: 7.15e-04 [09/22 16:13:04] lb.utils.events INFO: eta: 10:19:43 iteration: 135399/375342 consumed_samples: 138649600 total_loss: 3.701 time: 0.3403 s/iter data_time: 0.2158 s/iter total_throughput: 3009.03 samples/s lr: 7.15e-04 [09/22 16:13:38] lb.utils.events INFO: eta: 10:19:21 iteration: 135499/375342 consumed_samples: 138752000 total_loss: 3.71 time: 0.3403 s/iter data_time: 0.2347 s/iter total_throughput: 3008.98 samples/s lr: 7.14e-04 [09/22 16:14:13] lb.utils.events INFO: eta: 10:18:21 iteration: 135599/375342 consumed_samples: 138854400 total_loss: 3.704 time: 0.3403 s/iter data_time: 0.2199 s/iter total_throughput: 3008.92 samples/s lr: 7.14e-04 [09/22 16:14:48] lb.utils.events INFO: eta: 10:17:44 iteration: 135699/375342 consumed_samples: 138956800 total_loss: 3.699 time: 0.3403 s/iter data_time: 0.2159 s/iter total_throughput: 3008.87 samples/s lr: 7.14e-04 [09/22 16:15:23] lb.utils.events INFO: eta: 10:18:41 iteration: 135799/375342 consumed_samples: 139059200 total_loss: 3.695 time: 0.3403 s/iter data_time: 0.2262 s/iter total_throughput: 3008.82 samples/s lr: 7.13e-04 [09/22 16:15:58] lb.utils.events INFO: eta: 10:18:24 iteration: 135899/375342 consumed_samples: 139161600 total_loss: 3.725 time: 0.3403 s/iter data_time: 0.2269 s/iter total_throughput: 3008.78 samples/s lr: 7.13e-04 [09/22 16:16:32] lb.utils.events INFO: eta: 10:20:37 iteration: 135999/375342 consumed_samples: 139264000 total_loss: 3.724 time: 0.3403 s/iter data_time: 0.2170 s/iter total_throughput: 3008.73 samples/s lr: 7.12e-04 [09/22 16:17:07] lb.utils.events INFO: eta: 10:17:12 iteration: 136099/375342 consumed_samples: 139366400 total_loss: 3.704 time: 0.3403 s/iter data_time: 0.2186 s/iter total_throughput: 3008.68 samples/s lr: 7.12e-04 [09/22 16:17:42] lb.utils.events INFO: eta: 10:16:44 iteration: 136199/375342 consumed_samples: 139468800 total_loss: 3.717 time: 0.3404 s/iter data_time: 0.2183 s/iter total_throughput: 3008.63 samples/s lr: 7.12e-04 [09/22 16:18:17] lb.utils.events INFO: eta: 10:16:01 iteration: 136299/375342 consumed_samples: 139571200 total_loss: 3.722 time: 0.3404 s/iter data_time: 0.2162 s/iter total_throughput: 3008.60 samples/s lr: 7.11e-04 [09/22 16:18:51] lb.utils.events INFO: eta: 10:15:45 iteration: 136399/375342 consumed_samples: 139673600 total_loss: 3.7 time: 0.3404 s/iter data_time: 0.2201 s/iter total_throughput: 3008.57 samples/s lr: 7.11e-04 [09/22 16:19:26] lb.utils.events INFO: eta: 10:16:50 iteration: 136499/375342 consumed_samples: 139776000 total_loss: 3.676 time: 0.3404 s/iter data_time: 0.2194 s/iter total_throughput: 3008.52 samples/s lr: 7.11e-04 [09/22 16:20:01] lb.utils.events INFO: eta: 10:16:44 iteration: 136599/375342 consumed_samples: 139878400 total_loss: 3.691 time: 0.3404 s/iter data_time: 0.2127 s/iter total_throughput: 3008.46 samples/s lr: 7.10e-04 [09/22 16:20:36] lb.utils.events INFO: eta: 10:16:25 iteration: 136699/375342 consumed_samples: 139980800 total_loss: 3.703 time: 0.3404 s/iter data_time: 0.2296 s/iter total_throughput: 3008.40 samples/s lr: 7.10e-04 [09/22 16:21:11] lb.utils.events INFO: eta: 10:16:02 iteration: 136799/375342 consumed_samples: 140083200 total_loss: 3.705 time: 0.3404 s/iter data_time: 0.2201 s/iter total_throughput: 3008.36 samples/s lr: 7.09e-04 [09/22 16:21:45] lb.utils.events INFO: eta: 10:15:46 iteration: 136899/375342 consumed_samples: 140185600 total_loss: 3.705 time: 0.3404 s/iter data_time: 0.2352 s/iter total_throughput: 3008.31 samples/s lr: 7.09e-04 [09/22 16:22:20] lb.utils.events INFO: eta: 10:15:23 iteration: 136999/375342 consumed_samples: 140288000 total_loss: 3.679 time: 0.3404 s/iter data_time: 0.2203 s/iter total_throughput: 3008.28 samples/s lr: 7.09e-04 [09/22 16:22:54] lb.utils.events INFO: eta: 10:15:11 iteration: 137099/375342 consumed_samples: 140390400 total_loss: 3.686 time: 0.3404 s/iter data_time: 0.2178 s/iter total_throughput: 3008.25 samples/s lr: 7.08e-04 [09/22 16:23:29] lb.utils.events INFO: eta: 10:15:13 iteration: 137199/375342 consumed_samples: 140492800 total_loss: 3.699 time: 0.3404 s/iter data_time: 0.2137 s/iter total_throughput: 3008.21 samples/s lr: 7.08e-04 [09/22 16:24:03] lb.utils.events INFO: eta: 10:15:01 iteration: 137299/375342 consumed_samples: 140595200 total_loss: 3.699 time: 0.3404 s/iter data_time: 0.2260 s/iter total_throughput: 3008.19 samples/s lr: 7.08e-04 [09/22 16:24:38] lb.utils.events INFO: eta: 10:15:51 iteration: 137399/375342 consumed_samples: 140697600 total_loss: 3.71 time: 0.3404 s/iter data_time: 0.2075 s/iter total_throughput: 3008.16 samples/s lr: 7.07e-04 [09/22 16:25:13] lb.utils.events INFO: eta: 10:15:20 iteration: 137499/375342 consumed_samples: 140800000 total_loss: 3.718 time: 0.3404 s/iter data_time: 0.2342 s/iter total_throughput: 3008.12 samples/s lr: 7.07e-04 [09/22 16:25:47] lb.utils.events INFO: eta: 10:15:21 iteration: 137599/375342 consumed_samples: 140902400 total_loss: 3.687 time: 0.3404 s/iter data_time: 0.2104 s/iter total_throughput: 3008.11 samples/s lr: 7.06e-04 [09/22 16:26:22] lb.utils.events INFO: eta: 10:16:08 iteration: 137699/375342 consumed_samples: 141004800 total_loss: 3.689 time: 0.3404 s/iter data_time: 0.2098 s/iter total_throughput: 3008.05 samples/s lr: 7.06e-04 [09/22 16:26:57] lb.utils.events INFO: eta: 10:15:53 iteration: 137799/375342 consumed_samples: 141107200 total_loss: 3.695 time: 0.3404 s/iter data_time: 0.2117 s/iter total_throughput: 3008.00 samples/s lr: 7.06e-04 [09/22 16:27:31] lb.utils.events INFO: eta: 10:16:49 iteration: 137899/375342 consumed_samples: 141209600 total_loss: 3.679 time: 0.3404 s/iter data_time: 0.2122 s/iter total_throughput: 3007.98 samples/s lr: 7.05e-04 [09/22 16:28:05] lb.utils.events INFO: eta: 10:16:45 iteration: 137999/375342 consumed_samples: 141312000 total_loss: 3.675 time: 0.3404 s/iter data_time: 0.2373 s/iter total_throughput: 3007.96 samples/s lr: 7.05e-04 [09/22 16:28:40] lb.utils.events INFO: eta: 10:17:34 iteration: 138099/375342 consumed_samples: 141414400 total_loss: 3.679 time: 0.3404 s/iter data_time: 0.2193 s/iter total_throughput: 3007.94 samples/s lr: 7.05e-04 [09/22 16:29:14] lb.utils.events INFO: eta: 10:17:55 iteration: 138199/375342 consumed_samples: 141516800 total_loss: 3.679 time: 0.3404 s/iter data_time: 0.2155 s/iter total_throughput: 3007.90 samples/s lr: 7.04e-04 [09/22 16:29:49] lb.utils.events INFO: eta: 10:16:40 iteration: 138299/375342 consumed_samples: 141619200 total_loss: 3.692 time: 0.3404 s/iter data_time: 0.2089 s/iter total_throughput: 3007.88 samples/s lr: 7.04e-04 [09/22 16:30:23] lb.utils.events INFO: eta: 10:16:28 iteration: 138399/375342 consumed_samples: 141721600 total_loss: 3.698 time: 0.3404 s/iter data_time: 0.2247 s/iter total_throughput: 3007.85 samples/s lr: 7.03e-04 [09/22 16:30:58] lb.utils.events INFO: eta: 10:14:53 iteration: 138499/375342 consumed_samples: 141824000 total_loss: 3.704 time: 0.3404 s/iter data_time: 0.2153 s/iter total_throughput: 3007.83 samples/s lr: 7.03e-04 [09/22 16:31:32] lb.utils.events INFO: eta: 10:15:51 iteration: 138599/375342 consumed_samples: 141926400 total_loss: 3.723 time: 0.3404 s/iter data_time: 0.2101 s/iter total_throughput: 3007.82 samples/s lr: 7.03e-04 [09/22 16:32:06] lb.utils.events INFO: eta: 10:14:44 iteration: 138699/375342 consumed_samples: 142028800 total_loss: 3.712 time: 0.3404 s/iter data_time: 0.2172 s/iter total_throughput: 3007.81 samples/s lr: 7.02e-04 [09/22 16:32:40] lb.utils.events INFO: eta: 10:15:12 iteration: 138799/375342 consumed_samples: 142131200 total_loss: 3.706 time: 0.3404 s/iter data_time: 0.2065 s/iter total_throughput: 3007.80 samples/s lr: 7.02e-04 [09/22 16:33:14] lb.utils.events INFO: eta: 10:15:20 iteration: 138899/375342 consumed_samples: 142233600 total_loss: 3.693 time: 0.3404 s/iter data_time: 0.2042 s/iter total_throughput: 3007.81 samples/s lr: 7.02e-04 [09/22 16:33:49] lb.utils.events INFO: eta: 10:14:49 iteration: 138999/375342 consumed_samples: 142336000 total_loss: 3.684 time: 0.3404 s/iter data_time: 0.2137 s/iter total_throughput: 3007.79 samples/s lr: 7.01e-04 [09/22 16:34:23] lb.utils.events INFO: eta: 10:12:37 iteration: 139099/375342 consumed_samples: 142438400 total_loss: 3.691 time: 0.3404 s/iter data_time: 0.2148 s/iter total_throughput: 3007.79 samples/s lr: 7.01e-04 [09/22 16:34:57] lb.utils.events INFO: eta: 10:12:19 iteration: 139199/375342 consumed_samples: 142540800 total_loss: 3.702 time: 0.3405 s/iter data_time: 0.2118 s/iter total_throughput: 3007.78 samples/s lr: 7.00e-04 [09/22 16:35:31] lb.utils.events INFO: eta: 10:13:07 iteration: 139299/375342 consumed_samples: 142643200 total_loss: 3.711 time: 0.3404 s/iter data_time: 0.2094 s/iter total_throughput: 3007.80 samples/s lr: 7.00e-04 [09/22 16:36:05] lb.utils.events INFO: eta: 10:14:02 iteration: 139399/375342 consumed_samples: 142745600 total_loss: 3.703 time: 0.3405 s/iter data_time: 0.2284 s/iter total_throughput: 3007.78 samples/s lr: 7.00e-04 [09/22 16:36:40] lb.utils.events INFO: eta: 10:14:17 iteration: 139499/375342 consumed_samples: 142848000 total_loss: 3.687 time: 0.3405 s/iter data_time: 0.2347 s/iter total_throughput: 3007.73 samples/s lr: 6.99e-04 [09/22 16:37:14] lb.utils.events INFO: eta: 10:12:15 iteration: 139599/375342 consumed_samples: 142950400 total_loss: 3.688 time: 0.3405 s/iter data_time: 0.2161 s/iter total_throughput: 3007.70 samples/s lr: 6.99e-04 [09/22 16:37:48] lb.utils.events INFO: eta: 10:12:51 iteration: 139699/375342 consumed_samples: 143052800 total_loss: 3.696 time: 0.3405 s/iter data_time: 0.2062 s/iter total_throughput: 3007.70 samples/s lr: 6.98e-04 [09/22 16:38:22] lb.utils.events INFO: eta: 10:10:47 iteration: 139799/375342 consumed_samples: 143155200 total_loss: 3.691 time: 0.3405 s/iter data_time: 0.2125 s/iter total_throughput: 3007.71 samples/s lr: 6.98e-04 [09/22 16:38:56] lb.utils.events INFO: eta: 10:09:56 iteration: 139899/375342 consumed_samples: 143257600 total_loss: 3.688 time: 0.3405 s/iter data_time: 0.2097 s/iter total_throughput: 3007.73 samples/s lr: 6.98e-04 [09/22 16:39:30] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0139999 [09/22 16:39:31] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/22 16:39:31] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/22 16:39:35] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0948 s/iter. Inference: 0.1641 s/iter. Eval: 0.0020 s/iter. Total: 0.2609 s/iter. ETA=0:00:09 [09/22 16:39:41] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1177 s/iter. Inference: 0.1767 s/iter. Eval: 0.0019 s/iter. Total: 0.2964 s/iter. ETA=0:00:05 [09/22 16:39:46] lb.evaluation.evaluator INFO: Inference done 49152/50000. Dataloading: 0.1113 s/iter. Inference: 0.1691 s/iter. Eval: 0.0020 s/iter. Total: 0.2824 s/iter. ETA=0:00:00 [09/22 16:39:46] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/22 16:39:46] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.511276 (0.000250 s / iter per device, on 8 devices) [09/22 16:39:46] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000149 s / iter per device, on 8 devices) [09/22 16:39:46] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/22 16:39:46] lb.evaluation.utils INFO: copypaste: Acc@1=73.072 [09/22 16:39:46] lb.evaluation.utils INFO: copypaste: Acc@5=91.592 [09/22 16:39:46] lb.engine.hooks INFO: Not saving as latest eval score for Acc@1 is 73.07200, not better than best score 73.11800 @ iteration 134999. [09/22 16:39:46] lb.utils.events INFO: eta: 10:09:40 iteration: 139999/375342 consumed_samples: 143360000 total_loss: 3.7 time: 0.3405 s/iter data_time: 0.2111 s/iter total_throughput: 3007.72 samples/s lr: 6.97e-04 [09/22 16:40:19] lb.utils.events INFO: eta: 10:12:34 iteration: 140099/375342 consumed_samples: 143462400 total_loss: 3.7 time: 0.3404 s/iter data_time: 0.2334 s/iter total_throughput: 3007.81 samples/s lr: 6.97e-04 [09/22 16:40:53] lb.utils.events INFO: eta: 10:12:57 iteration: 140199/375342 consumed_samples: 143564800 total_loss: 3.703 time: 0.3404 s/iter data_time: 0.2251 s/iter total_throughput: 3007.79 samples/s lr: 6.97e-04 [09/22 16:41:27] lb.utils.events INFO: eta: 10:11:35 iteration: 140299/375342 consumed_samples: 143667200 total_loss: 3.701 time: 0.3405 s/iter data_time: 0.2141 s/iter total_throughput: 3007.76 samples/s lr: 6.96e-04 [09/22 16:42:02] lb.utils.events INFO: eta: 10:11:56 iteration: 140399/375342 consumed_samples: 143769600 total_loss: 3.691 time: 0.3405 s/iter data_time: 0.2175 s/iter total_throughput: 3007.70 samples/s lr: 6.96e-04 [09/22 16:42:37] lb.utils.events INFO: eta: 10:11:04 iteration: 140499/375342 consumed_samples: 143872000 total_loss: 3.686 time: 0.3405 s/iter data_time: 0.2222 s/iter total_throughput: 3007.65 samples/s lr: 6.95e-04 [09/22 16:43:13] lb.utils.events INFO: eta: 10:11:23 iteration: 140599/375342 consumed_samples: 143974400 total_loss: 3.672 time: 0.3405 s/iter data_time: 0.2270 s/iter total_throughput: 3007.54 samples/s lr: 6.95e-04 [09/22 16:43:49] lb.utils.events INFO: eta: 10:09:49 iteration: 140699/375342 consumed_samples: 144076800 total_loss: 3.683 time: 0.3405 s/iter data_time: 0.2226 s/iter total_throughput: 3007.46 samples/s lr: 6.95e-04 [09/22 16:44:24] lb.utils.events INFO: eta: 10:09:16 iteration: 140799/375342 consumed_samples: 144179200 total_loss: 3.693 time: 0.3405 s/iter data_time: 0.2246 s/iter total_throughput: 3007.37 samples/s lr: 6.94e-04 [09/22 16:44:59] lb.utils.events INFO: eta: 10:09:24 iteration: 140899/375342 consumed_samples: 144281600 total_loss: 3.707 time: 0.3405 s/iter data_time: 0.2233 s/iter total_throughput: 3007.31 samples/s lr: 6.94e-04 [09/22 16:45:33] lb.utils.events INFO: eta: 10:10:58 iteration: 140999/375342 consumed_samples: 144384000 total_loss: 3.709 time: 0.3405 s/iter data_time: 0.2089 s/iter total_throughput: 3007.29 samples/s lr: 6.93e-04 [09/22 16:46:08] lb.utils.events INFO: eta: 10:07:34 iteration: 141099/375342 consumed_samples: 144486400 total_loss: 3.703 time: 0.3405 s/iter data_time: 0.2189 s/iter total_throughput: 3007.26 samples/s lr: 6.93e-04 [09/22 16:46:43] lb.utils.events INFO: eta: 10:06:27 iteration: 141199/375342 consumed_samples: 144588800 total_loss: 3.701 time: 0.3405 s/iter data_time: 0.2314 s/iter total_throughput: 3007.17 samples/s lr: 6.93e-04 [09/22 16:47:19] lb.utils.events INFO: eta: 10:05:12 iteration: 141299/375342 consumed_samples: 144691200 total_loss: 3.688 time: 0.3405 s/iter data_time: 0.2194 s/iter total_throughput: 3007.10 samples/s lr: 6.92e-04 [09/22 16:47:53] lb.utils.events INFO: eta: 10:04:51 iteration: 141399/375342 consumed_samples: 144793600 total_loss: 3.691 time: 0.3405 s/iter data_time: 0.2264 s/iter total_throughput: 3007.05 samples/s lr: 6.92e-04 [09/22 16:48:29] lb.utils.events INFO: eta: 10:04:16 iteration: 141499/375342 consumed_samples: 144896000 total_loss: 3.7 time: 0.3405 s/iter data_time: 0.2296 s/iter total_throughput: 3006.95 samples/s lr: 6.92e-04 [09/22 16:49:05] lb.utils.events INFO: eta: 10:03:32 iteration: 141599/375342 consumed_samples: 144998400 total_loss: 3.682 time: 0.3406 s/iter data_time: 0.2212 s/iter total_throughput: 3006.86 samples/s lr: 6.91e-04 [09/22 16:49:39] lb.utils.events INFO: eta: 10:04:27 iteration: 141699/375342 consumed_samples: 145100800 total_loss: 3.663 time: 0.3406 s/iter data_time: 0.2179 s/iter total_throughput: 3006.83 samples/s lr: 6.91e-04 [09/22 16:50:14] lb.utils.events INFO: eta: 10:06:23 iteration: 141799/375342 consumed_samples: 145203200 total_loss: 3.668 time: 0.3406 s/iter data_time: 0.2258 s/iter total_throughput: 3006.78 samples/s lr: 6.90e-04 [09/22 16:50:48] lb.utils.events INFO: eta: 10:04:21 iteration: 141899/375342 consumed_samples: 145305600 total_loss: 3.693 time: 0.3406 s/iter data_time: 0.2323 s/iter total_throughput: 3006.77 samples/s lr: 6.90e-04 [09/22 16:51:24] lb.utils.events INFO: eta: 10:00:43 iteration: 141999/375342 consumed_samples: 145408000 total_loss: 3.7 time: 0.3406 s/iter data_time: 0.2298 s/iter total_throughput: 3006.69 samples/s lr: 6.90e-04 [09/22 16:51:58] lb.utils.events INFO: eta: 10:00:53 iteration: 142099/375342 consumed_samples: 145510400 total_loss: 3.677 time: 0.3406 s/iter data_time: 0.2229 s/iter total_throughput: 3006.64 samples/s lr: 6.89e-04 [09/22 16:52:34] lb.utils.events INFO: eta: 10:00:19 iteration: 142199/375342 consumed_samples: 145612800 total_loss: 3.685 time: 0.3406 s/iter data_time: 0.2287 s/iter total_throughput: 3006.58 samples/s lr: 6.89e-04 [09/22 16:53:08] lb.utils.events INFO: eta: 10:01:18 iteration: 142299/375342 consumed_samples: 145715200 total_loss: 3.724 time: 0.3406 s/iter data_time: 0.2176 s/iter total_throughput: 3006.55 samples/s lr: 6.88e-04 [09/22 16:53:43] lb.utils.events INFO: eta: 10:00:13 iteration: 142399/375342 consumed_samples: 145817600 total_loss: 3.717 time: 0.3406 s/iter data_time: 0.2135 s/iter total_throughput: 3006.51 samples/s lr: 6.88e-04 [09/22 16:54:17] lb.utils.events INFO: eta: 10:01:54 iteration: 142499/375342 consumed_samples: 145920000 total_loss: 3.687 time: 0.3406 s/iter data_time: 0.2183 s/iter total_throughput: 3006.49 samples/s lr: 6.88e-04 [09/22 16:54:52] lb.utils.events INFO: eta: 10:02:05 iteration: 142599/375342 consumed_samples: 146022400 total_loss: 3.682 time: 0.3406 s/iter data_time: 0.2177 s/iter total_throughput: 3006.45 samples/s lr: 6.87e-04 [09/22 16:55:27] lb.utils.events INFO: eta: 10:02:09 iteration: 142699/375342 consumed_samples: 146124800 total_loss: 3.683 time: 0.3406 s/iter data_time: 0.2176 s/iter total_throughput: 3006.40 samples/s lr: 6.87e-04 [09/22 16:56:01] lb.utils.events INFO: eta: 10:01:54 iteration: 142799/375342 consumed_samples: 146227200 total_loss: 3.673 time: 0.3406 s/iter data_time: 0.2171 s/iter total_throughput: 3006.37 samples/s lr: 6.87e-04 [09/22 16:56:36] lb.utils.events INFO: eta: 10:02:08 iteration: 142899/375342 consumed_samples: 146329600 total_loss: 3.695 time: 0.3406 s/iter data_time: 0.2202 s/iter total_throughput: 3006.33 samples/s lr: 6.86e-04 [09/22 16:57:11] lb.utils.events INFO: eta: 10:03:32 iteration: 142999/375342 consumed_samples: 146432000 total_loss: 3.689 time: 0.3406 s/iter data_time: 0.2115 s/iter total_throughput: 3006.28 samples/s lr: 6.86e-04 [09/22 16:57:46] lb.utils.events INFO: eta: 10:03:08 iteration: 143099/375342 consumed_samples: 146534400 total_loss: 3.687 time: 0.3406 s/iter data_time: 0.2376 s/iter total_throughput: 3006.19 samples/s lr: 6.85e-04 [09/22 16:58:21] lb.utils.events INFO: eta: 10:03:38 iteration: 143199/375342 consumed_samples: 146636800 total_loss: 3.691 time: 0.3406 s/iter data_time: 0.2297 s/iter total_throughput: 3006.15 samples/s lr: 6.85e-04 [09/22 16:58:56] lb.utils.events INFO: eta: 10:03:25 iteration: 143299/375342 consumed_samples: 146739200 total_loss: 3.681 time: 0.3406 s/iter data_time: 0.2118 s/iter total_throughput: 3006.09 samples/s lr: 6.85e-04 [09/22 16:59:31] lb.utils.events INFO: eta: 10:02:27 iteration: 143399/375342 consumed_samples: 146841600 total_loss: 3.682 time: 0.3406 s/iter data_time: 0.2251 s/iter total_throughput: 3006.04 samples/s lr: 6.84e-04 [09/22 17:00:05] lb.utils.events INFO: eta: 10:01:33 iteration: 143499/375342 consumed_samples: 146944000 total_loss: 3.686 time: 0.3406 s/iter data_time: 0.2085 s/iter total_throughput: 3006.02 samples/s lr: 6.84e-04 [09/22 17:00:40] lb.utils.events INFO: eta: 10:01:01 iteration: 143599/375342 consumed_samples: 147046400 total_loss: 3.68 time: 0.3407 s/iter data_time: 0.2194 s/iter total_throughput: 3005.97 samples/s lr: 6.83e-04 [09/22 17:01:16] lb.utils.events INFO: eta: 9:59:14 iteration: 143699/375342 consumed_samples: 147148800 total_loss: 3.666 time: 0.3407 s/iter data_time: 0.2360 s/iter total_throughput: 3005.89 samples/s lr: 6.83e-04 [09/22 17:01:51] lb.utils.events INFO: eta: 9:58:39 iteration: 143799/375342 consumed_samples: 147251200 total_loss: 3.672 time: 0.3407 s/iter data_time: 0.2358 s/iter total_throughput: 3005.83 samples/s lr: 6.83e-04 [09/22 17:02:26] lb.utils.events INFO: eta: 9:57:50 iteration: 143899/375342 consumed_samples: 147353600 total_loss: 3.681 time: 0.3407 s/iter data_time: 0.2385 s/iter total_throughput: 3005.74 samples/s lr: 6.82e-04 [09/22 17:03:01] lb.utils.events INFO: eta: 9:57:01 iteration: 143999/375342 consumed_samples: 147456000 total_loss: 3.659 time: 0.3407 s/iter data_time: 0.2133 s/iter total_throughput: 3005.69 samples/s lr: 6.82e-04 [09/22 17:03:36] lb.utils.events INFO: eta: 9:57:34 iteration: 144099/375342 consumed_samples: 147558400 total_loss: 3.656 time: 0.3407 s/iter data_time: 0.2232 s/iter total_throughput: 3005.68 samples/s lr: 6.82e-04 [09/22 17:04:10] lb.utils.events INFO: eta: 9:59:49 iteration: 144199/375342 consumed_samples: 147660800 total_loss: 3.66 time: 0.3407 s/iter data_time: 0.2204 s/iter total_throughput: 3005.63 samples/s lr: 6.81e-04 [09/22 17:04:45] lb.utils.events INFO: eta: 9:59:36 iteration: 144299/375342 consumed_samples: 147763200 total_loss: 3.674 time: 0.3407 s/iter data_time: 0.2174 s/iter total_throughput: 3005.58 samples/s lr: 6.81e-04 [09/22 17:05:20] lb.utils.events INFO: eta: 9:58:11 iteration: 144399/375342 consumed_samples: 147865600 total_loss: 3.665 time: 0.3407 s/iter data_time: 0.2243 s/iter total_throughput: 3005.56 samples/s lr: 6.80e-04 [09/22 17:05:54] lb.utils.events INFO: eta: 9:58:35 iteration: 144499/375342 consumed_samples: 147968000 total_loss: 3.665 time: 0.3407 s/iter data_time: 0.2206 s/iter total_throughput: 3005.52 samples/s lr: 6.80e-04 [09/22 17:06:30] lb.utils.events INFO: eta: 9:57:53 iteration: 144599/375342 consumed_samples: 148070400 total_loss: 3.689 time: 0.3407 s/iter data_time: 0.2225 s/iter total_throughput: 3005.45 samples/s lr: 6.80e-04 [09/22 17:07:04] lb.utils.events INFO: eta: 9:58:21 iteration: 144699/375342 consumed_samples: 148172800 total_loss: 3.676 time: 0.3407 s/iter data_time: 0.2206 s/iter total_throughput: 3005.41 samples/s lr: 6.79e-04 [09/22 17:07:40] lb.utils.events INFO: eta: 9:58:25 iteration: 144799/375342 consumed_samples: 148275200 total_loss: 3.655 time: 0.3407 s/iter data_time: 0.2347 s/iter total_throughput: 3005.32 samples/s lr: 6.79e-04 [09/22 17:08:15] lb.utils.events INFO: eta: 9:58:25 iteration: 144899/375342 consumed_samples: 148377600 total_loss: 3.667 time: 0.3407 s/iter data_time: 0.2148 s/iter total_throughput: 3005.29 samples/s lr: 6.78e-04 [09/22 17:08:49] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0144999 [09/22 17:08:50] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/22 17:08:50] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/22 17:08:54] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0966 s/iter. Inference: 0.1604 s/iter. Eval: 0.0020 s/iter. Total: 0.2591 s/iter. ETA=0:00:09 [09/22 17:08:59] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1061 s/iter. Inference: 0.1885 s/iter. Eval: 0.0020 s/iter. Total: 0.2968 s/iter. ETA=0:00:05 [09/22 17:09:04] lb.evaluation.evaluator INFO: Inference done 49152/50000. Dataloading: 0.1034 s/iter. Inference: 0.1787 s/iter. Eval: 0.0021 s/iter. Total: 0.2842 s/iter. ETA=0:00:00 [09/22 17:09:05] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/22 17:09:05] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.520151 (0.000250 s / iter per device, on 8 devices) [09/22 17:09:05] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000158 s / iter per device, on 8 devices) [09/22 17:09:05] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/22 17:09:05] lb.evaluation.utils INFO: copypaste: Acc@1=73.31400000000001 [09/22 17:09:05] lb.evaluation.utils INFO: copypaste: Acc@5=91.494 [09/22 17:09:05] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 73.31400, better than last best score 73.11800 @ iteration 134999. [09/22 17:09:05] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/22 17:09:06] lb.utils.events INFO: eta: 9:57:54 iteration: 144999/375342 consumed_samples: 148480000 total_loss: 3.678 time: 0.3407 s/iter data_time: 0.2218 s/iter total_throughput: 3005.27 samples/s lr: 6.78e-04 [09/22 17:09:38] lb.utils.events INFO: eta: 9:59:44 iteration: 145099/375342 consumed_samples: 148582400 total_loss: 3.684 time: 0.3407 s/iter data_time: 0.2155 s/iter total_throughput: 3005.34 samples/s lr: 6.78e-04 [09/22 17:10:13] lb.utils.events INFO: eta: 9:58:47 iteration: 145199/375342 consumed_samples: 148684800 total_loss: 3.687 time: 0.3407 s/iter data_time: 0.2154 s/iter total_throughput: 3005.31 samples/s lr: 6.77e-04 [09/22 17:10:48] lb.utils.events INFO: eta: 9:58:31 iteration: 145299/375342 consumed_samples: 148787200 total_loss: 3.661 time: 0.3407 s/iter data_time: 0.2143 s/iter total_throughput: 3005.26 samples/s lr: 6.77e-04 [09/22 17:11:23] lb.utils.events INFO: eta: 9:58:25 iteration: 145399/375342 consumed_samples: 148889600 total_loss: 3.653 time: 0.3407 s/iter data_time: 0.2221 s/iter total_throughput: 3005.21 samples/s lr: 6.77e-04 [09/22 17:11:58] lb.utils.events INFO: eta: 9:57:56 iteration: 145499/375342 consumed_samples: 148992000 total_loss: 3.668 time: 0.3407 s/iter data_time: 0.2281 s/iter total_throughput: 3005.15 samples/s lr: 6.76e-04 [09/22 17:12:33] lb.utils.events INFO: eta: 9:57:54 iteration: 145599/375342 consumed_samples: 149094400 total_loss: 3.66 time: 0.3408 s/iter data_time: 0.2250 s/iter total_throughput: 3005.10 samples/s lr: 6.76e-04 [09/22 17:13:07] lb.utils.events INFO: eta: 9:57:12 iteration: 145699/375342 consumed_samples: 149196800 total_loss: 3.628 time: 0.3408 s/iter data_time: 0.2222 s/iter total_throughput: 3005.07 samples/s lr: 6.75e-04 [09/22 17:13:42] lb.utils.events INFO: eta: 9:57:06 iteration: 145799/375342 consumed_samples: 149299200 total_loss: 3.646 time: 0.3408 s/iter data_time: 0.2152 s/iter total_throughput: 3005.03 samples/s lr: 6.75e-04 [09/22 17:14:17] lb.utils.events INFO: eta: 9:56:41 iteration: 145899/375342 consumed_samples: 149401600 total_loss: 3.693 time: 0.3408 s/iter data_time: 0.2202 s/iter total_throughput: 3004.98 samples/s lr: 6.75e-04 [09/22 17:14:52] lb.utils.events INFO: eta: 9:57:14 iteration: 145999/375342 consumed_samples: 149504000 total_loss: 3.689 time: 0.3408 s/iter data_time: 0.2289 s/iter total_throughput: 3004.94 samples/s lr: 6.74e-04 [09/22 17:15:26] lb.utils.events INFO: eta: 9:53:55 iteration: 146099/375342 consumed_samples: 149606400 total_loss: 3.682 time: 0.3408 s/iter data_time: 0.2211 s/iter total_throughput: 3004.91 samples/s lr: 6.74e-04 [09/22 17:16:01] lb.utils.events INFO: eta: 9:53:12 iteration: 146199/375342 consumed_samples: 149708800 total_loss: 3.69 time: 0.3408 s/iter data_time: 0.2140 s/iter total_throughput: 3004.87 samples/s lr: 6.73e-04 [09/22 17:16:36] lb.utils.events INFO: eta: 9:53:32 iteration: 146299/375342 consumed_samples: 149811200 total_loss: 3.678 time: 0.3408 s/iter data_time: 0.2275 s/iter total_throughput: 3004.83 samples/s lr: 6.73e-04 [09/22 17:17:10] lb.utils.events INFO: eta: 9:53:38 iteration: 146399/375342 consumed_samples: 149913600 total_loss: 3.65 time: 0.3408 s/iter data_time: 0.2169 s/iter total_throughput: 3004.82 samples/s lr: 6.73e-04 [09/22 17:17:45] lb.utils.events INFO: eta: 9:53:22 iteration: 146499/375342 consumed_samples: 150016000 total_loss: 3.644 time: 0.3408 s/iter data_time: 0.2278 s/iter total_throughput: 3004.79 samples/s lr: 6.72e-04 [09/22 17:18:19] lb.utils.events INFO: eta: 9:55:08 iteration: 146599/375342 consumed_samples: 150118400 total_loss: 3.648 time: 0.3408 s/iter data_time: 0.2230 s/iter total_throughput: 3004.77 samples/s lr: 6.72e-04 [09/22 17:18:54] lb.utils.events INFO: eta: 9:55:45 iteration: 146699/375342 consumed_samples: 150220800 total_loss: 3.654 time: 0.3408 s/iter data_time: 0.2152 s/iter total_throughput: 3004.75 samples/s lr: 6.71e-04 [09/22 17:19:27] lb.utils.events INFO: eta: 9:56:25 iteration: 146799/375342 consumed_samples: 150323200 total_loss: 3.666 time: 0.3408 s/iter data_time: 0.2124 s/iter total_throughput: 3004.76 samples/s lr: 6.71e-04 [09/22 17:20:02] lb.utils.events INFO: eta: 9:56:36 iteration: 146899/375342 consumed_samples: 150425600 total_loss: 3.663 time: 0.3408 s/iter data_time: 0.2086 s/iter total_throughput: 3004.72 samples/s lr: 6.71e-04 [09/22 17:20:37] lb.utils.events INFO: eta: 9:56:23 iteration: 146999/375342 consumed_samples: 150528000 total_loss: 3.661 time: 0.3408 s/iter data_time: 0.2197 s/iter total_throughput: 3004.68 samples/s lr: 6.70e-04 [09/22 17:21:12] lb.utils.events INFO: eta: 9:56:08 iteration: 147099/375342 consumed_samples: 150630400 total_loss: 3.663 time: 0.3408 s/iter data_time: 0.2195 s/iter total_throughput: 3004.65 samples/s lr: 6.70e-04 [09/22 17:21:46] lb.utils.events INFO: eta: 9:56:18 iteration: 147199/375342 consumed_samples: 150732800 total_loss: 3.646 time: 0.3408 s/iter data_time: 0.2200 s/iter total_throughput: 3004.60 samples/s lr: 6.69e-04 [09/22 17:22:21] lb.utils.events INFO: eta: 9:56:11 iteration: 147299/375342 consumed_samples: 150835200 total_loss: 3.665 time: 0.3408 s/iter data_time: 0.2154 s/iter total_throughput: 3004.57 samples/s lr: 6.69e-04 [09/22 17:22:55] lb.utils.events INFO: eta: 9:54:23 iteration: 147399/375342 consumed_samples: 150937600 total_loss: 3.684 time: 0.3408 s/iter data_time: 0.2129 s/iter total_throughput: 3004.56 samples/s lr: 6.69e-04 [09/22 17:23:30] lb.utils.events INFO: eta: 9:54:09 iteration: 147499/375342 consumed_samples: 151040000 total_loss: 3.679 time: 0.3408 s/iter data_time: 0.2131 s/iter total_throughput: 3004.54 samples/s lr: 6.68e-04 [09/22 17:24:04] lb.utils.events INFO: eta: 9:53:56 iteration: 147599/375342 consumed_samples: 151142400 total_loss: 3.679 time: 0.3408 s/iter data_time: 0.2143 s/iter total_throughput: 3004.51 samples/s lr: 6.68e-04 [09/22 17:24:39] lb.utils.events INFO: eta: 9:52:51 iteration: 147699/375342 consumed_samples: 151244800 total_loss: 3.664 time: 0.3408 s/iter data_time: 0.2197 s/iter total_throughput: 3004.47 samples/s lr: 6.68e-04 [09/22 17:25:15] lb.utils.events INFO: eta: 9:50:54 iteration: 147799/375342 consumed_samples: 151347200 total_loss: 3.628 time: 0.3408 s/iter data_time: 0.2569 s/iter total_throughput: 3004.38 samples/s lr: 6.67e-04 [09/22 17:25:49] lb.utils.events INFO: eta: 9:49:14 iteration: 147899/375342 consumed_samples: 151449600 total_loss: 3.646 time: 0.3408 s/iter data_time: 0.2226 s/iter total_throughput: 3004.35 samples/s lr: 6.67e-04 [09/22 17:26:24] lb.utils.events INFO: eta: 9:48:10 iteration: 147999/375342 consumed_samples: 151552000 total_loss: 3.671 time: 0.3408 s/iter data_time: 0.2183 s/iter total_throughput: 3004.31 samples/s lr: 6.66e-04 [09/22 17:26:59] lb.utils.events INFO: eta: 9:48:37 iteration: 148099/375342 consumed_samples: 151654400 total_loss: 3.66 time: 0.3408 s/iter data_time: 0.2137 s/iter total_throughput: 3004.27 samples/s lr: 6.66e-04 [09/22 17:27:34] lb.utils.events INFO: eta: 9:47:45 iteration: 148199/375342 consumed_samples: 151756800 total_loss: 3.654 time: 0.3409 s/iter data_time: 0.2309 s/iter total_throughput: 3004.21 samples/s lr: 6.66e-04 [09/22 17:28:09] lb.utils.events INFO: eta: 9:47:24 iteration: 148299/375342 consumed_samples: 151859200 total_loss: 3.668 time: 0.3409 s/iter data_time: 0.2263 s/iter total_throughput: 3004.17 samples/s lr: 6.65e-04 [09/22 17:28:43] lb.utils.events INFO: eta: 9:48:11 iteration: 148399/375342 consumed_samples: 151961600 total_loss: 3.676 time: 0.3409 s/iter data_time: 0.2152 s/iter total_throughput: 3004.15 samples/s lr: 6.65e-04 [09/22 17:29:18] lb.utils.events INFO: eta: 9:49:29 iteration: 148499/375342 consumed_samples: 152064000 total_loss: 3.684 time: 0.3409 s/iter data_time: 0.2172 s/iter total_throughput: 3004.12 samples/s lr: 6.64e-04 [09/22 17:29:52] lb.utils.events INFO: eta: 9:49:33 iteration: 148599/375342 consumed_samples: 152166400 total_loss: 3.684 time: 0.3409 s/iter data_time: 0.2233 s/iter total_throughput: 3004.12 samples/s lr: 6.64e-04 [09/22 17:30:27] lb.utils.events INFO: eta: 9:49:51 iteration: 148699/375342 consumed_samples: 152268800 total_loss: 3.684 time: 0.3409 s/iter data_time: 0.2156 s/iter total_throughput: 3004.07 samples/s lr: 6.64e-04 [09/22 17:31:02] lb.utils.events INFO: eta: 9:49:41 iteration: 148799/375342 consumed_samples: 152371200 total_loss: 3.668 time: 0.3409 s/iter data_time: 0.2199 s/iter total_throughput: 3004.03 samples/s lr: 6.63e-04 [09/22 17:31:37] lb.utils.events INFO: eta: 9:49:31 iteration: 148899/375342 consumed_samples: 152473600 total_loss: 3.647 time: 0.3409 s/iter data_time: 0.2383 s/iter total_throughput: 3003.95 samples/s lr: 6.63e-04 [09/22 17:32:12] lb.utils.events INFO: eta: 9:48:32 iteration: 148999/375342 consumed_samples: 152576000 total_loss: 3.66 time: 0.3409 s/iter data_time: 0.2233 s/iter total_throughput: 3003.89 samples/s lr: 6.62e-04 [09/22 17:32:47] lb.utils.events INFO: eta: 9:47:23 iteration: 149099/375342 consumed_samples: 152678400 total_loss: 3.662 time: 0.3409 s/iter data_time: 0.2226 s/iter total_throughput: 3003.84 samples/s lr: 6.62e-04 [09/22 17:33:21] lb.utils.events INFO: eta: 9:48:30 iteration: 149199/375342 consumed_samples: 152780800 total_loss: 3.668 time: 0.3409 s/iter data_time: 0.2139 s/iter total_throughput: 3003.83 samples/s lr: 6.62e-04 [09/22 17:33:56] lb.utils.events INFO: eta: 9:48:11 iteration: 149299/375342 consumed_samples: 152883200 total_loss: 3.664 time: 0.3409 s/iter data_time: 0.2245 s/iter total_throughput: 3003.80 samples/s lr: 6.61e-04 [09/22 17:34:30] lb.utils.events INFO: eta: 9:47:57 iteration: 149399/375342 consumed_samples: 152985600 total_loss: 3.668 time: 0.3409 s/iter data_time: 0.2127 s/iter total_throughput: 3003.79 samples/s lr: 6.61e-04 [09/22 17:35:05] lb.utils.events INFO: eta: 9:47:32 iteration: 149499/375342 consumed_samples: 153088000 total_loss: 3.671 time: 0.3409 s/iter data_time: 0.2327 s/iter total_throughput: 3003.73 samples/s lr: 6.60e-04 [09/22 17:35:40] lb.utils.events INFO: eta: 9:46:20 iteration: 149599/375342 consumed_samples: 153190400 total_loss: 3.66 time: 0.3409 s/iter data_time: 0.2094 s/iter total_throughput: 3003.72 samples/s lr: 6.60e-04 [09/22 17:36:14] lb.utils.events INFO: eta: 9:45:06 iteration: 149699/375342 consumed_samples: 153292800 total_loss: 3.661 time: 0.3409 s/iter data_time: 0.2204 s/iter total_throughput: 3003.68 samples/s lr: 6.60e-04 [09/22 17:36:49] lb.utils.events INFO: eta: 9:45:17 iteration: 149799/375342 consumed_samples: 153395200 total_loss: 3.667 time: 0.3409 s/iter data_time: 0.2071 s/iter total_throughput: 3003.65 samples/s lr: 6.59e-04 [09/22 17:37:23] lb.utils.events INFO: eta: 9:45:15 iteration: 149899/375342 consumed_samples: 153497600 total_loss: 3.676 time: 0.3409 s/iter data_time: 0.2135 s/iter total_throughput: 3003.65 samples/s lr: 6.59e-04 [09/22 17:37:58] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0149999 [09/22 17:37:58] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/22 17:37:58] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/22 17:38:02] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0988 s/iter. Inference: 0.1584 s/iter. Eval: 0.0020 s/iter. Total: 0.2592 s/iter. ETA=0:00:09 [09/22 17:38:08] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1424 s/iter. Inference: 0.1605 s/iter. Eval: 0.0020 s/iter. Total: 0.3049 s/iter. ETA=0:00:05 [09/22 17:38:13] lb.evaluation.evaluator INFO: Inference done 49152/50000. Dataloading: 0.1256 s/iter. Inference: 0.1612 s/iter. Eval: 0.0020 s/iter. Total: 0.2889 s/iter. ETA=0:00:00 [09/22 17:38:14] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/22 17:38:14] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.777679 (0.000256 s / iter per device, on 8 devices) [09/22 17:38:14] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000143 s / iter per device, on 8 devices) [09/22 17:38:14] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/22 17:38:14] lb.evaluation.utils INFO: copypaste: Acc@1=73.576 [09/22 17:38:14] lb.evaluation.utils INFO: copypaste: Acc@5=91.738 [09/22 17:38:14] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 73.57600, better than last best score 73.31400 @ iteration 144999. [09/22 17:38:14] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/22 17:38:14] lb.utils.events INFO: eta: 9:46:45 iteration: 149999/375342 consumed_samples: 153600000 total_loss: 3.667 time: 0.3409 s/iter data_time: 0.2121 s/iter total_throughput: 3003.62 samples/s lr: 6.59e-04 [09/22 17:38:47] lb.utils.events INFO: eta: 9:49:47 iteration: 150099/375342 consumed_samples: 153702400 total_loss: 3.673 time: 0.3409 s/iter data_time: 0.2506 s/iter total_throughput: 3003.74 samples/s lr: 6.58e-04 [09/22 17:39:21] lb.utils.events INFO: eta: 9:50:02 iteration: 150199/375342 consumed_samples: 153804800 total_loss: 3.68 time: 0.3409 s/iter data_time: 0.2254 s/iter total_throughput: 3003.70 samples/s lr: 6.58e-04 [09/22 17:39:56] lb.utils.events INFO: eta: 9:48:10 iteration: 150299/375342 consumed_samples: 153907200 total_loss: 3.659 time: 0.3409 s/iter data_time: 0.2228 s/iter total_throughput: 3003.64 samples/s lr: 6.57e-04 [09/22 17:40:31] lb.utils.events INFO: eta: 9:46:23 iteration: 150399/375342 consumed_samples: 154009600 total_loss: 3.653 time: 0.3409 s/iter data_time: 0.2146 s/iter total_throughput: 3003.60 samples/s lr: 6.57e-04 [09/22 17:41:06] lb.utils.events INFO: eta: 9:45:06 iteration: 150499/375342 consumed_samples: 154112000 total_loss: 3.657 time: 0.3409 s/iter data_time: 0.2101 s/iter total_throughput: 3003.58 samples/s lr: 6.57e-04 [09/22 17:41:40] lb.utils.events INFO: eta: 9:45:52 iteration: 150599/375342 consumed_samples: 154214400 total_loss: 3.655 time: 0.3409 s/iter data_time: 0.2076 s/iter total_throughput: 3003.58 samples/s lr: 6.56e-04 [09/22 17:42:15] lb.utils.events INFO: eta: 9:45:52 iteration: 150699/375342 consumed_samples: 154316800 total_loss: 3.649 time: 0.3409 s/iter data_time: 0.2231 s/iter total_throughput: 3003.51 samples/s lr: 6.56e-04 [09/22 17:42:50] lb.utils.events INFO: eta: 9:45:27 iteration: 150799/375342 consumed_samples: 154419200 total_loss: 3.648 time: 0.3409 s/iter data_time: 0.2275 s/iter total_throughput: 3003.47 samples/s lr: 6.55e-04 [09/22 17:43:24] lb.utils.events INFO: eta: 9:43:54 iteration: 150899/375342 consumed_samples: 154521600 total_loss: 3.646 time: 0.3409 s/iter data_time: 0.2191 s/iter total_throughput: 3003.44 samples/s lr: 6.55e-04 [09/22 17:43:59] lb.utils.events INFO: eta: 9:40:44 iteration: 150999/375342 consumed_samples: 154624000 total_loss: 3.652 time: 0.3409 s/iter data_time: 0.2364 s/iter total_throughput: 3003.40 samples/s lr: 6.55e-04 [09/22 17:44:33] lb.utils.events INFO: eta: 9:37:18 iteration: 151099/375342 consumed_samples: 154726400 total_loss: 3.668 time: 0.3409 s/iter data_time: 0.2130 s/iter total_throughput: 3003.39 samples/s lr: 6.54e-04 [09/22 17:45:08] lb.utils.events INFO: eta: 9:33:58 iteration: 151199/375342 consumed_samples: 154828800 total_loss: 3.66 time: 0.3410 s/iter data_time: 0.2211 s/iter total_throughput: 3003.34 samples/s lr: 6.54e-04 [09/22 17:45:43] lb.utils.events INFO: eta: 9:35:07 iteration: 151299/375342 consumed_samples: 154931200 total_loss: 3.643 time: 0.3410 s/iter data_time: 0.2184 s/iter total_throughput: 3003.30 samples/s lr: 6.53e-04 [09/22 17:46:18] lb.utils.events INFO: eta: 9:37:02 iteration: 151399/375342 consumed_samples: 155033600 total_loss: 3.659 time: 0.3410 s/iter data_time: 0.2397 s/iter total_throughput: 3003.28 samples/s lr: 6.53e-04 [09/22 17:46:52] lb.utils.events INFO: eta: 9:37:00 iteration: 151499/375342 consumed_samples: 155136000 total_loss: 3.649 time: 0.3410 s/iter data_time: 0.2288 s/iter total_throughput: 3003.25 samples/s lr: 6.53e-04 [09/22 17:47:27] lb.utils.events INFO: eta: 9:36:45 iteration: 151599/375342 consumed_samples: 155238400 total_loss: 3.639 time: 0.3410 s/iter data_time: 0.2187 s/iter total_throughput: 3003.23 samples/s lr: 6.52e-04 [09/22 17:48:01] lb.utils.events INFO: eta: 9:38:10 iteration: 151699/375342 consumed_samples: 155340800 total_loss: 3.621 time: 0.3410 s/iter data_time: 0.2205 s/iter total_throughput: 3003.21 samples/s lr: 6.52e-04 [09/22 17:48:35] lb.utils.events INFO: eta: 9:38:10 iteration: 151799/375342 consumed_samples: 155443200 total_loss: 3.618 time: 0.3410 s/iter data_time: 0.2189 s/iter total_throughput: 3003.22 samples/s lr: 6.51e-04 [09/22 17:49:09] lb.utils.events INFO: eta: 9:39:44 iteration: 151899/375342 consumed_samples: 155545600 total_loss: 3.649 time: 0.3410 s/iter data_time: 0.2114 s/iter total_throughput: 3003.23 samples/s lr: 6.51e-04 [09/22 17:49:44] lb.utils.events INFO: eta: 9:40:36 iteration: 151999/375342 consumed_samples: 155648000 total_loss: 3.664 time: 0.3410 s/iter data_time: 0.2207 s/iter total_throughput: 3003.20 samples/s lr: 6.51e-04 [09/22 17:50:18] lb.utils.events INFO: eta: 9:41:29 iteration: 152099/375342 consumed_samples: 155750400 total_loss: 3.649 time: 0.3410 s/iter data_time: 0.2198 s/iter total_throughput: 3003.19 samples/s lr: 6.50e-04 [09/22 17:50:52] lb.utils.events INFO: eta: 9:42:24 iteration: 152199/375342 consumed_samples: 155852800 total_loss: 3.649 time: 0.3410 s/iter data_time: 0.2090 s/iter total_throughput: 3003.17 samples/s lr: 6.50e-04 [09/22 17:51:27] lb.utils.events INFO: eta: 9:41:57 iteration: 152299/375342 consumed_samples: 155955200 total_loss: 3.662 time: 0.3410 s/iter data_time: 0.2147 s/iter total_throughput: 3003.14 samples/s lr: 6.49e-04 [09/22 17:52:01] lb.utils.events INFO: eta: 9:41:41 iteration: 152399/375342 consumed_samples: 156057600 total_loss: 3.655 time: 0.3410 s/iter data_time: 0.2256 s/iter total_throughput: 3003.13 samples/s lr: 6.49e-04 [09/22 17:52:36] lb.utils.events INFO: eta: 9:42:34 iteration: 152499/375342 consumed_samples: 156160000 total_loss: 3.658 time: 0.3410 s/iter data_time: 0.2256 s/iter total_throughput: 3003.12 samples/s lr: 6.49e-04 [09/22 17:53:10] lb.utils.events INFO: eta: 9:41:46 iteration: 152599/375342 consumed_samples: 156262400 total_loss: 3.68 time: 0.3410 s/iter data_time: 0.2230 s/iter total_throughput: 3003.11 samples/s lr: 6.48e-04 [09/22 17:53:44] lb.utils.events INFO: eta: 9:41:22 iteration: 152699/375342 consumed_samples: 156364800 total_loss: 3.657 time: 0.3410 s/iter data_time: 0.2154 s/iter total_throughput: 3003.10 samples/s lr: 6.48e-04 [09/22 17:54:18] lb.utils.events INFO: eta: 9:40:44 iteration: 152799/375342 consumed_samples: 156467200 total_loss: 3.641 time: 0.3410 s/iter data_time: 0.2155 s/iter total_throughput: 3003.13 samples/s lr: 6.47e-04 [09/22 17:54:52] lb.utils.events INFO: eta: 9:41:06 iteration: 152899/375342 consumed_samples: 156569600 total_loss: 3.611 time: 0.3410 s/iter data_time: 0.2118 s/iter total_throughput: 3003.13 samples/s lr: 6.47e-04 [09/22 17:55:26] lb.utils.events INFO: eta: 9:40:10 iteration: 152999/375342 consumed_samples: 156672000 total_loss: 3.638 time: 0.3410 s/iter data_time: 0.2171 s/iter total_throughput: 3003.11 samples/s lr: 6.47e-04 [09/22 17:56:01] lb.utils.events INFO: eta: 9:39:19 iteration: 153099/375342 consumed_samples: 156774400 total_loss: 3.655 time: 0.3410 s/iter data_time: 0.2233 s/iter total_throughput: 3003.09 samples/s lr: 6.46e-04 [09/22 17:56:35] lb.utils.events INFO: eta: 9:38:29 iteration: 153199/375342 consumed_samples: 156876800 total_loss: 3.655 time: 0.3410 s/iter data_time: 0.2127 s/iter total_throughput: 3003.08 samples/s lr: 6.46e-04 [09/22 17:57:09] lb.utils.events INFO: eta: 9:38:08 iteration: 153299/375342 consumed_samples: 156979200 total_loss: 3.662 time: 0.3410 s/iter data_time: 0.2139 s/iter total_throughput: 3003.08 samples/s lr: 6.45e-04 [09/22 17:57:43] lb.utils.events INFO: eta: 9:37:06 iteration: 153399/375342 consumed_samples: 157081600 total_loss: 3.661 time: 0.3410 s/iter data_time: 0.2067 s/iter total_throughput: 3003.08 samples/s lr: 6.45e-04 [09/22 17:58:17] lb.utils.events INFO: eta: 9:36:31 iteration: 153499/375342 consumed_samples: 157184000 total_loss: 3.664 time: 0.3410 s/iter data_time: 0.2086 s/iter total_throughput: 3003.09 samples/s lr: 6.45e-04 [09/22 17:58:52] lb.utils.events INFO: eta: 9:36:16 iteration: 153599/375342 consumed_samples: 157286400 total_loss: 3.676 time: 0.3410 s/iter data_time: 0.2140 s/iter total_throughput: 3003.08 samples/s lr: 6.44e-04 [09/22 17:59:26] lb.utils.events INFO: eta: 9:34:30 iteration: 153699/375342 consumed_samples: 157388800 total_loss: 3.682 time: 0.3410 s/iter data_time: 0.2081 s/iter total_throughput: 3003.05 samples/s lr: 6.44e-04 [09/22 18:00:00] lb.utils.events INFO: eta: 9:34:23 iteration: 153799/375342 consumed_samples: 157491200 total_loss: 3.665 time: 0.3410 s/iter data_time: 0.2147 s/iter total_throughput: 3003.06 samples/s lr: 6.43e-04 [09/22 18:00:34] lb.utils.events INFO: eta: 9:34:01 iteration: 153899/375342 consumed_samples: 157593600 total_loss: 3.628 time: 0.3410 s/iter data_time: 0.2029 s/iter total_throughput: 3003.07 samples/s lr: 6.43e-04 [09/22 18:01:08] lb.utils.events INFO: eta: 9:32:53 iteration: 153999/375342 consumed_samples: 157696000 total_loss: 3.634 time: 0.3410 s/iter data_time: 0.2134 s/iter total_throughput: 3003.07 samples/s lr: 6.43e-04 [09/22 18:01:43] lb.utils.events INFO: eta: 9:32:47 iteration: 154099/375342 consumed_samples: 157798400 total_loss: 3.653 time: 0.3410 s/iter data_time: 0.2380 s/iter total_throughput: 3003.03 samples/s lr: 6.42e-04 [09/22 18:02:18] lb.utils.events INFO: eta: 9:33:11 iteration: 154199/375342 consumed_samples: 157900800 total_loss: 3.652 time: 0.3410 s/iter data_time: 0.2477 s/iter total_throughput: 3002.96 samples/s lr: 6.42e-04 [09/22 18:02:53] lb.utils.events INFO: eta: 9:31:25 iteration: 154299/375342 consumed_samples: 158003200 total_loss: 3.637 time: 0.3410 s/iter data_time: 0.2350 s/iter total_throughput: 3002.90 samples/s lr: 6.41e-04 [09/22 18:03:28] lb.utils.events INFO: eta: 9:31:22 iteration: 154399/375342 consumed_samples: 158105600 total_loss: 3.637 time: 0.3410 s/iter data_time: 0.2217 s/iter total_throughput: 3002.87 samples/s lr: 6.41e-04 [09/22 18:04:03] lb.utils.events INFO: eta: 9:29:25 iteration: 154499/375342 consumed_samples: 158208000 total_loss: 3.631 time: 0.3410 s/iter data_time: 0.2345 s/iter total_throughput: 3002.81 samples/s lr: 6.41e-04 [09/22 18:04:39] lb.utils.events INFO: eta: 9:28:18 iteration: 154599/375342 consumed_samples: 158310400 total_loss: 3.625 time: 0.3410 s/iter data_time: 0.2247 s/iter total_throughput: 3002.74 samples/s lr: 6.40e-04 [09/22 18:05:14] lb.utils.events INFO: eta: 9:28:54 iteration: 154699/375342 consumed_samples: 158412800 total_loss: 3.628 time: 0.3410 s/iter data_time: 0.2304 s/iter total_throughput: 3002.67 samples/s lr: 6.40e-04 [09/22 18:05:49] lb.utils.events INFO: eta: 9:26:57 iteration: 154799/375342 consumed_samples: 158515200 total_loss: 3.652 time: 0.3410 s/iter data_time: 0.2251 s/iter total_throughput: 3002.58 samples/s lr: 6.39e-04 [09/22 18:06:25] lb.utils.events INFO: eta: 9:26:41 iteration: 154899/375342 consumed_samples: 158617600 total_loss: 3.655 time: 0.3410 s/iter data_time: 0.2241 s/iter total_throughput: 3002.52 samples/s lr: 6.39e-04 [09/22 18:07:00] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0154999 [09/22 18:07:01] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/22 18:07:01] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/22 18:07:05] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.1028 s/iter. Inference: 0.1605 s/iter. Eval: 0.0020 s/iter. Total: 0.2653 s/iter. ETA=0:00:09 [09/22 18:07:10] lb.evaluation.evaluator INFO: Inference done 28672/50000. Dataloading: 0.0948 s/iter. Inference: 0.1927 s/iter. Eval: 0.0021 s/iter. Total: 0.2896 s/iter. ETA=0:00:05 [09/22 18:07:15] lb.evaluation.evaluator INFO: Inference done 46080/50000. Dataloading: 0.0932 s/iter. Inference: 0.1980 s/iter. Eval: 0.0021 s/iter. Total: 0.2933 s/iter. ETA=0:00:00 [09/22 18:07:16] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/22 18:07:16] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.684084 (0.000254 s / iter per device, on 8 devices) [09/22 18:07:16] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:08 (0.000174 s / iter per device, on 8 devices) [09/22 18:07:16] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/22 18:07:16] lb.evaluation.utils INFO: copypaste: Acc@1=73.75399999999999 [09/22 18:07:16] lb.evaluation.utils INFO: copypaste: Acc@5=91.874 [09/22 18:07:16] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 73.75400, better than last best score 73.57600 @ iteration 149999. [09/22 18:07:16] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/22 18:07:17] lb.utils.events INFO: eta: 9:26:53 iteration: 154999/375342 consumed_samples: 158720000 total_loss: 3.657 time: 0.3411 s/iter data_time: 0.2299 s/iter total_throughput: 3002.46 samples/s lr: 6.39e-04 [09/22 18:07:50] lb.utils.events INFO: eta: 9:27:30 iteration: 155099/375342 consumed_samples: 158822400 total_loss: 3.645 time: 0.3410 s/iter data_time: 0.2265 s/iter total_throughput: 3002.50 samples/s lr: 6.38e-04 [09/22 18:08:25] lb.utils.events INFO: eta: 9:27:28 iteration: 155199/375342 consumed_samples: 158924800 total_loss: 3.604 time: 0.3411 s/iter data_time: 0.2217 s/iter total_throughput: 3002.46 samples/s lr: 6.38e-04 [09/22 18:09:01] lb.utils.events INFO: eta: 9:27:03 iteration: 155299/375342 consumed_samples: 159027200 total_loss: 3.621 time: 0.3411 s/iter data_time: 0.2328 s/iter total_throughput: 3002.37 samples/s lr: 6.37e-04 [09/22 18:09:36] lb.utils.events INFO: eta: 9:26:53 iteration: 155399/375342 consumed_samples: 159129600 total_loss: 3.655 time: 0.3411 s/iter data_time: 0.2340 s/iter total_throughput: 3002.28 samples/s lr: 6.37e-04 [09/22 18:10:11] lb.utils.events INFO: eta: 9:27:16 iteration: 155499/375342 consumed_samples: 159232000 total_loss: 3.651 time: 0.3411 s/iter data_time: 0.2207 s/iter total_throughput: 3002.22 samples/s lr: 6.37e-04 [09/22 18:10:47] lb.utils.events INFO: eta: 9:27:02 iteration: 155599/375342 consumed_samples: 159334400 total_loss: 3.634 time: 0.3411 s/iter data_time: 0.2343 s/iter total_throughput: 3002.15 samples/s lr: 6.36e-04 [09/22 18:11:22] lb.utils.events INFO: eta: 9:25:59 iteration: 155699/375342 consumed_samples: 159436800 total_loss: 3.638 time: 0.3411 s/iter data_time: 0.2266 s/iter total_throughput: 3002.09 samples/s lr: 6.36e-04 [09/22 18:11:57] lb.utils.events INFO: eta: 9:26:20 iteration: 155799/375342 consumed_samples: 159539200 total_loss: 3.634 time: 0.3411 s/iter data_time: 0.2346 s/iter total_throughput: 3002.03 samples/s lr: 6.35e-04 [09/22 18:12:32] lb.utils.events INFO: eta: 9:24:04 iteration: 155899/375342 consumed_samples: 159641600 total_loss: 3.633 time: 0.3411 s/iter data_time: 0.2293 s/iter total_throughput: 3001.97 samples/s lr: 6.35e-04 [09/22 18:13:07] lb.utils.events INFO: eta: 9:23:49 iteration: 155999/375342 consumed_samples: 159744000 total_loss: 3.647 time: 0.3411 s/iter data_time: 0.2180 s/iter total_throughput: 3001.92 samples/s lr: 6.35e-04 [09/22 18:13:42] lb.utils.events INFO: eta: 9:21:50 iteration: 156099/375342 consumed_samples: 159846400 total_loss: 3.65 time: 0.3411 s/iter data_time: 0.2240 s/iter total_throughput: 3001.88 samples/s lr: 6.34e-04 [09/22 18:14:17] lb.utils.events INFO: eta: 9:21:49 iteration: 156199/375342 consumed_samples: 159948800 total_loss: 3.646 time: 0.3411 s/iter data_time: 0.2254 s/iter total_throughput: 3001.83 samples/s lr: 6.34e-04 [09/22 18:14:52] lb.utils.events INFO: eta: 9:24:36 iteration: 156299/375342 consumed_samples: 160051200 total_loss: 3.654 time: 0.3411 s/iter data_time: 0.2316 s/iter total_throughput: 3001.79 samples/s lr: 6.33e-04 [09/22 18:15:27] lb.utils.events INFO: eta: 9:24:54 iteration: 156399/375342 consumed_samples: 160153600 total_loss: 3.648 time: 0.3411 s/iter data_time: 0.2145 s/iter total_throughput: 3001.75 samples/s lr: 6.33e-04 [09/22 18:16:02] lb.utils.events INFO: eta: 9:25:06 iteration: 156499/375342 consumed_samples: 160256000 total_loss: 3.639 time: 0.3411 s/iter data_time: 0.2217 s/iter total_throughput: 3001.71 samples/s lr: 6.33e-04 [09/22 18:16:36] lb.utils.events INFO: eta: 9:24:48 iteration: 156599/375342 consumed_samples: 160358400 total_loss: 3.62 time: 0.3411 s/iter data_time: 0.2216 s/iter total_throughput: 3001.67 samples/s lr: 6.32e-04 [09/22 18:17:11] lb.utils.events INFO: eta: 9:25:16 iteration: 156699/375342 consumed_samples: 160460800 total_loss: 3.613 time: 0.3411 s/iter data_time: 0.2136 s/iter total_throughput: 3001.62 samples/s lr: 6.32e-04 [09/22 18:17:46] lb.utils.events INFO: eta: 9:25:15 iteration: 156799/375342 consumed_samples: 160563200 total_loss: 3.622 time: 0.3412 s/iter data_time: 0.2247 s/iter total_throughput: 3001.57 samples/s lr: 6.31e-04 [09/22 18:18:21] lb.utils.events INFO: eta: 9:26:00 iteration: 156899/375342 consumed_samples: 160665600 total_loss: 3.637 time: 0.3412 s/iter data_time: 0.2218 s/iter total_throughput: 3001.55 samples/s lr: 6.31e-04 [09/22 18:18:56] lb.utils.events INFO: eta: 9:25:11 iteration: 156999/375342 consumed_samples: 160768000 total_loss: 3.622 time: 0.3412 s/iter data_time: 0.2165 s/iter total_throughput: 3001.51 samples/s lr: 6.31e-04 [09/22 18:19:31] lb.utils.events INFO: eta: 9:23:57 iteration: 157099/375342 consumed_samples: 160870400 total_loss: 3.633 time: 0.3412 s/iter data_time: 0.2334 s/iter total_throughput: 3001.45 samples/s lr: 6.30e-04 [09/22 18:20:06] lb.utils.events INFO: eta: 9:21:58 iteration: 157199/375342 consumed_samples: 160972800 total_loss: 3.646 time: 0.3412 s/iter data_time: 0.2202 s/iter total_throughput: 3001.39 samples/s lr: 6.30e-04 [09/22 18:20:41] lb.utils.events INFO: eta: 9:21:41 iteration: 157299/375342 consumed_samples: 161075200 total_loss: 3.625 time: 0.3412 s/iter data_time: 0.2254 s/iter total_throughput: 3001.36 samples/s lr: 6.29e-04 [09/22 18:21:16] lb.utils.events INFO: eta: 9:20:46 iteration: 157399/375342 consumed_samples: 161177600 total_loss: 3.627 time: 0.3412 s/iter data_time: 0.2121 s/iter total_throughput: 3001.31 samples/s lr: 6.29e-04 [09/22 18:21:51] lb.utils.events INFO: eta: 9:20:18 iteration: 157499/375342 consumed_samples: 161280000 total_loss: 3.66 time: 0.3412 s/iter data_time: 0.2165 s/iter total_throughput: 3001.28 samples/s lr: 6.29e-04 [09/22 18:22:26] lb.utils.events INFO: eta: 9:20:08 iteration: 157599/375342 consumed_samples: 161382400 total_loss: 3.653 time: 0.3412 s/iter data_time: 0.2134 s/iter total_throughput: 3001.22 samples/s lr: 6.28e-04 [09/22 18:23:01] lb.utils.events INFO: eta: 9:18:27 iteration: 157699/375342 consumed_samples: 161484800 total_loss: 3.632 time: 0.3412 s/iter data_time: 0.2206 s/iter total_throughput: 3001.16 samples/s lr: 6.28e-04 [09/22 18:23:36] lb.utils.events INFO: eta: 9:17:30 iteration: 157799/375342 consumed_samples: 161587200 total_loss: 3.646 time: 0.3412 s/iter data_time: 0.2160 s/iter total_throughput: 3001.10 samples/s lr: 6.27e-04 [09/22 18:24:11] lb.utils.events INFO: eta: 9:17:58 iteration: 157899/375342 consumed_samples: 161689600 total_loss: 3.646 time: 0.3412 s/iter data_time: 0.2278 s/iter total_throughput: 3001.07 samples/s lr: 6.27e-04 [09/22 18:24:46] lb.utils.events INFO: eta: 9:18:27 iteration: 157999/375342 consumed_samples: 161792000 total_loss: 3.629 time: 0.3412 s/iter data_time: 0.2319 s/iter total_throughput: 3001.01 samples/s lr: 6.27e-04 [09/22 18:25:21] lb.utils.events INFO: eta: 9:21:00 iteration: 158099/375342 consumed_samples: 161894400 total_loss: 3.626 time: 0.3412 s/iter data_time: 0.2331 s/iter total_throughput: 3000.98 samples/s lr: 6.26e-04 [09/22 18:25:55] lb.utils.events INFO: eta: 9:21:32 iteration: 158199/375342 consumed_samples: 161996800 total_loss: 3.633 time: 0.3412 s/iter data_time: 0.2195 s/iter total_throughput: 3000.95 samples/s lr: 6.26e-04 [09/22 18:26:30] lb.utils.events INFO: eta: 9:20:18 iteration: 158299/375342 consumed_samples: 162099200 total_loss: 3.627 time: 0.3412 s/iter data_time: 0.2265 s/iter total_throughput: 3000.90 samples/s lr: 6.25e-04 [09/22 18:27:05] lb.utils.events INFO: eta: 9:20:51 iteration: 158399/375342 consumed_samples: 162201600 total_loss: 3.608 time: 0.3412 s/iter data_time: 0.2150 s/iter total_throughput: 3000.87 samples/s lr: 6.25e-04 [09/22 18:27:40] lb.utils.events INFO: eta: 9:20:57 iteration: 158499/375342 consumed_samples: 162304000 total_loss: 3.623 time: 0.3412 s/iter data_time: 0.2194 s/iter total_throughput: 3000.85 samples/s lr: 6.25e-04 [09/22 18:28:15] lb.utils.events INFO: eta: 9:20:43 iteration: 158599/375342 consumed_samples: 162406400 total_loss: 3.634 time: 0.3412 s/iter data_time: 0.2249 s/iter total_throughput: 3000.80 samples/s lr: 6.24e-04 [09/22 18:28:50] lb.utils.events INFO: eta: 9:21:02 iteration: 158699/375342 consumed_samples: 162508800 total_loss: 3.619 time: 0.3412 s/iter data_time: 0.2184 s/iter total_throughput: 3000.74 samples/s lr: 6.24e-04 [09/22 18:29:25] lb.utils.events INFO: eta: 9:20:38 iteration: 158799/375342 consumed_samples: 162611200 total_loss: 3.646 time: 0.3413 s/iter data_time: 0.2334 s/iter total_throughput: 3000.68 samples/s lr: 6.23e-04 [09/22 18:30:00] lb.utils.events INFO: eta: 9:19:58 iteration: 158899/375342 consumed_samples: 162713600 total_loss: 3.639 time: 0.3413 s/iter data_time: 0.2206 s/iter total_throughput: 3000.61 samples/s lr: 6.23e-04 [09/22 18:30:35] lb.utils.events INFO: eta: 9:19:47 iteration: 158999/375342 consumed_samples: 162816000 total_loss: 3.615 time: 0.3413 s/iter data_time: 0.2202 s/iter total_throughput: 3000.58 samples/s lr: 6.23e-04 [09/22 18:31:10] lb.utils.events INFO: eta: 9:17:34 iteration: 159099/375342 consumed_samples: 162918400 total_loss: 3.606 time: 0.3413 s/iter data_time: 0.2261 s/iter total_throughput: 3000.53 samples/s lr: 6.22e-04 [09/22 18:31:46] lb.utils.events INFO: eta: 9:17:09 iteration: 159199/375342 consumed_samples: 163020800 total_loss: 3.606 time: 0.3413 s/iter data_time: 0.2261 s/iter total_throughput: 3000.46 samples/s lr: 6.22e-04 [09/22 18:32:21] lb.utils.events INFO: eta: 9:18:13 iteration: 159299/375342 consumed_samples: 163123200 total_loss: 3.636 time: 0.3413 s/iter data_time: 0.2219 s/iter total_throughput: 3000.40 samples/s lr: 6.21e-04 [09/22 18:32:56] lb.utils.events INFO: eta: 9:17:23 iteration: 159399/375342 consumed_samples: 163225600 total_loss: 3.643 time: 0.3413 s/iter data_time: 0.2246 s/iter total_throughput: 3000.33 samples/s lr: 6.21e-04 [09/22 18:33:31] lb.utils.events INFO: eta: 9:16:30 iteration: 159499/375342 consumed_samples: 163328000 total_loss: 3.641 time: 0.3413 s/iter data_time: 0.2195 s/iter total_throughput: 3000.28 samples/s lr: 6.21e-04 [09/22 18:34:06] lb.utils.events INFO: eta: 9:17:31 iteration: 159599/375342 consumed_samples: 163430400 total_loss: 3.637 time: 0.3413 s/iter data_time: 0.2173 s/iter total_throughput: 3000.25 samples/s lr: 6.20e-04 [09/22 18:34:41] lb.utils.events INFO: eta: 9:17:11 iteration: 159699/375342 consumed_samples: 163532800 total_loss: 3.629 time: 0.3413 s/iter data_time: 0.2215 s/iter total_throughput: 3000.22 samples/s lr: 6.20e-04 [09/22 18:35:15] lb.utils.events INFO: eta: 9:17:50 iteration: 159799/375342 consumed_samples: 163635200 total_loss: 3.62 time: 0.3413 s/iter data_time: 0.2147 s/iter total_throughput: 3000.21 samples/s lr: 6.19e-04 [09/22 18:35:50] lb.utils.events INFO: eta: 9:18:00 iteration: 159899/375342 consumed_samples: 163737600 total_loss: 3.608 time: 0.3413 s/iter data_time: 0.2130 s/iter total_throughput: 3000.18 samples/s lr: 6.19e-04 [09/22 18:36:25] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0159999 [09/22 18:36:25] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/22 18:36:25] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/22 18:36:30] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.1088 s/iter. Inference: 0.1598 s/iter. Eval: 0.0020 s/iter. Total: 0.2707 s/iter. ETA=0:00:10 [09/22 18:36:35] lb.evaluation.evaluator INFO: Inference done 28672/50000. Dataloading: 0.1091 s/iter. Inference: 0.1788 s/iter. Eval: 0.0020 s/iter. Total: 0.2899 s/iter. ETA=0:00:05 [09/22 18:36:40] lb.evaluation.evaluator INFO: Inference done 47104/50000. Dataloading: 0.1193 s/iter. Inference: 0.1717 s/iter. Eval: 0.0020 s/iter. Total: 0.2930 s/iter. ETA=0:00:00 [09/22 18:36:41] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/22 18:36:41] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.589060 (0.000252 s / iter per device, on 8 devices) [09/22 18:36:41] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000151 s / iter per device, on 8 devices) [09/22 18:36:41] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/22 18:36:41] lb.evaluation.utils INFO: copypaste: Acc@1=73.536 [09/22 18:36:41] lb.evaluation.utils INFO: copypaste: Acc@5=91.808 [09/22 18:36:41] lb.engine.hooks INFO: Not saving as latest eval score for Acc@1 is 73.53600, not better than best score 73.75400 @ iteration 154999. [09/22 18:36:41] lb.utils.events INFO: eta: 9:17:45 iteration: 159999/375342 consumed_samples: 163840000 total_loss: 3.622 time: 0.3413 s/iter data_time: 0.2196 s/iter total_throughput: 3000.13 samples/s lr: 6.19e-04 [09/22 18:37:13] lb.utils.events INFO: eta: 9:19:28 iteration: 160099/375342 consumed_samples: 163942400 total_loss: 3.654 time: 0.3413 s/iter data_time: 0.2239 s/iter total_throughput: 3000.19 samples/s lr: 6.18e-04 [09/22 18:37:48] lb.utils.events INFO: eta: 9:21:17 iteration: 160199/375342 consumed_samples: 164044800 total_loss: 3.646 time: 0.3413 s/iter data_time: 0.2240 s/iter total_throughput: 3000.17 samples/s lr: 6.18e-04 [09/22 18:38:23] lb.utils.events INFO: eta: 9:20:56 iteration: 160299/375342 consumed_samples: 164147200 total_loss: 3.625 time: 0.3413 s/iter data_time: 0.2368 s/iter total_throughput: 3000.13 samples/s lr: 6.17e-04 [09/22 18:38:58] lb.utils.events INFO: eta: 9:19:58 iteration: 160399/375342 consumed_samples: 164249600 total_loss: 3.624 time: 0.3413 s/iter data_time: 0.2181 s/iter total_throughput: 3000.08 samples/s lr: 6.17e-04 [09/22 18:39:33] lb.utils.events INFO: eta: 9:19:33 iteration: 160499/375342 consumed_samples: 164352000 total_loss: 3.608 time: 0.3413 s/iter data_time: 0.2352 s/iter total_throughput: 3000.03 samples/s lr: 6.17e-04 [09/22 18:40:09] lb.utils.events INFO: eta: 9:17:46 iteration: 160599/375342 consumed_samples: 164454400 total_loss: 3.636 time: 0.3413 s/iter data_time: 0.2245 s/iter total_throughput: 2999.95 samples/s lr: 6.16e-04 [09/22 18:40:44] lb.utils.events INFO: eta: 9:17:39 iteration: 160699/375342 consumed_samples: 164556800 total_loss: 3.66 time: 0.3413 s/iter data_time: 0.2192 s/iter total_throughput: 2999.91 samples/s lr: 6.16e-04 [09/22 18:41:19] lb.utils.events INFO: eta: 9:15:56 iteration: 160799/375342 consumed_samples: 164659200 total_loss: 3.621 time: 0.3414 s/iter data_time: 0.2254 s/iter total_throughput: 2999.82 samples/s lr: 6.15e-04 [09/22 18:41:55] lb.utils.events INFO: eta: 9:15:04 iteration: 160899/375342 consumed_samples: 164761600 total_loss: 3.614 time: 0.3414 s/iter data_time: 0.2353 s/iter total_throughput: 2999.76 samples/s lr: 6.15e-04 [09/22 18:42:29] lb.utils.events INFO: eta: 9:15:19 iteration: 160999/375342 consumed_samples: 164864000 total_loss: 3.619 time: 0.3414 s/iter data_time: 0.2218 s/iter total_throughput: 2999.72 samples/s lr: 6.15e-04 [09/22 18:43:05] lb.utils.events INFO: eta: 9:13:48 iteration: 161099/375342 consumed_samples: 164966400 total_loss: 3.63 time: 0.3414 s/iter data_time: 0.2157 s/iter total_throughput: 2999.66 samples/s lr: 6.14e-04 [09/22 18:43:40] lb.utils.events INFO: eta: 9:13:14 iteration: 161199/375342 consumed_samples: 165068800 total_loss: 3.634 time: 0.3414 s/iter data_time: 0.2283 s/iter total_throughput: 2999.60 samples/s lr: 6.14e-04 [09/22 18:44:15] lb.utils.events INFO: eta: 9:13:17 iteration: 161299/375342 consumed_samples: 165171200 total_loss: 3.636 time: 0.3414 s/iter data_time: 0.2096 s/iter total_throughput: 2999.56 samples/s lr: 6.13e-04 [09/22 18:44:49] lb.utils.events INFO: eta: 9:12:57 iteration: 161399/375342 consumed_samples: 165273600 total_loss: 3.624 time: 0.3414 s/iter data_time: 0.2165 s/iter total_throughput: 2999.53 samples/s lr: 6.13e-04 [09/22 18:45:24] lb.utils.events INFO: eta: 9:13:38 iteration: 161499/375342 consumed_samples: 165376000 total_loss: 3.612 time: 0.3414 s/iter data_time: 0.2233 s/iter total_throughput: 2999.48 samples/s lr: 6.13e-04 [09/22 18:46:00] lb.utils.events INFO: eta: 9:13:45 iteration: 161599/375342 consumed_samples: 165478400 total_loss: 3.625 time: 0.3414 s/iter data_time: 0.2232 s/iter total_throughput: 2999.43 samples/s lr: 6.12e-04 [09/22 18:46:35] lb.utils.events INFO: eta: 9:14:02 iteration: 161699/375342 consumed_samples: 165580800 total_loss: 3.626 time: 0.3414 s/iter data_time: 0.2193 s/iter total_throughput: 2999.39 samples/s lr: 6.12e-04 [09/22 18:47:09] lb.utils.events INFO: eta: 9:15:45 iteration: 161799/375342 consumed_samples: 165683200 total_loss: 3.626 time: 0.3414 s/iter data_time: 0.2196 s/iter total_throughput: 2999.35 samples/s lr: 6.11e-04 [09/22 18:47:44] lb.utils.events INFO: eta: 9:16:05 iteration: 161899/375342 consumed_samples: 165785600 total_loss: 3.613 time: 0.3414 s/iter data_time: 0.2178 s/iter total_throughput: 2999.31 samples/s lr: 6.11e-04 [09/22 18:48:19] lb.utils.events INFO: eta: 9:16:16 iteration: 161999/375342 consumed_samples: 165888000 total_loss: 3.606 time: 0.3414 s/iter data_time: 0.2260 s/iter total_throughput: 2999.30 samples/s lr: 6.11e-04 [09/22 18:48:53] lb.utils.events INFO: eta: 9:16:13 iteration: 162099/375342 consumed_samples: 165990400 total_loss: 3.625 time: 0.3414 s/iter data_time: 0.2170 s/iter total_throughput: 2999.28 samples/s lr: 6.10e-04 [09/22 18:49:28] lb.utils.events INFO: eta: 9:15:11 iteration: 162199/375342 consumed_samples: 166092800 total_loss: 3.65 time: 0.3414 s/iter data_time: 0.2235 s/iter total_throughput: 2999.23 samples/s lr: 6.10e-04 [09/22 18:50:03] lb.utils.events INFO: eta: 9:12:23 iteration: 162299/375342 consumed_samples: 166195200 total_loss: 3.63 time: 0.3414 s/iter data_time: 0.2312 s/iter total_throughput: 2999.20 samples/s lr: 6.09e-04 [09/22 18:50:37] lb.utils.events INFO: eta: 9:14:03 iteration: 162399/375342 consumed_samples: 166297600 total_loss: 3.627 time: 0.3414 s/iter data_time: 0.2257 s/iter total_throughput: 2999.19 samples/s lr: 6.09e-04 [09/22 18:51:12] lb.utils.events INFO: eta: 9:13:47 iteration: 162499/375342 consumed_samples: 166400000 total_loss: 3.635 time: 0.3414 s/iter data_time: 0.2219 s/iter total_throughput: 2999.16 samples/s lr: 6.09e-04 [09/22 18:51:46] lb.utils.events INFO: eta: 9:14:09 iteration: 162599/375342 consumed_samples: 166502400 total_loss: 3.617 time: 0.3414 s/iter data_time: 0.2158 s/iter total_throughput: 2999.14 samples/s lr: 6.08e-04 [09/22 18:52:21] lb.utils.events INFO: eta: 9:13:19 iteration: 162699/375342 consumed_samples: 166604800 total_loss: 3.626 time: 0.3414 s/iter data_time: 0.2215 s/iter total_throughput: 2999.09 samples/s lr: 6.08e-04 [09/22 18:52:56] lb.utils.events INFO: eta: 9:13:31 iteration: 162799/375342 consumed_samples: 166707200 total_loss: 3.614 time: 0.3414 s/iter data_time: 0.2335 s/iter total_throughput: 2999.05 samples/s lr: 6.07e-04 [09/22 18:53:31] lb.utils.events INFO: eta: 9:13:46 iteration: 162899/375342 consumed_samples: 166809600 total_loss: 3.617 time: 0.3414 s/iter data_time: 0.2215 s/iter total_throughput: 2999.03 samples/s lr: 6.07e-04 [09/22 18:54:05] lb.utils.events INFO: eta: 9:12:54 iteration: 162999/375342 consumed_samples: 166912000 total_loss: 3.626 time: 0.3414 s/iter data_time: 0.2184 s/iter total_throughput: 2999.02 samples/s lr: 6.06e-04 [09/22 18:54:40] lb.utils.events INFO: eta: 9:12:03 iteration: 163099/375342 consumed_samples: 167014400 total_loss: 3.617 time: 0.3415 s/iter data_time: 0.2454 s/iter total_throughput: 2998.96 samples/s lr: 6.06e-04 [09/22 18:55:16] lb.utils.events INFO: eta: 9:11:22 iteration: 163199/375342 consumed_samples: 167116800 total_loss: 3.609 time: 0.3415 s/iter data_time: 0.2175 s/iter total_throughput: 2998.91 samples/s lr: 6.06e-04 [09/22 18:55:50] lb.utils.events INFO: eta: 9:11:14 iteration: 163299/375342 consumed_samples: 167219200 total_loss: 3.644 time: 0.3415 s/iter data_time: 0.2230 s/iter total_throughput: 2998.87 samples/s lr: 6.05e-04 [09/22 18:56:25] lb.utils.events INFO: eta: 9:10:47 iteration: 163399/375342 consumed_samples: 167321600 total_loss: 3.663 time: 0.3415 s/iter data_time: 0.2122 s/iter total_throughput: 2998.83 samples/s lr: 6.05e-04 [09/22 18:57:00] lb.utils.events INFO: eta: 9:10:46 iteration: 163499/375342 consumed_samples: 167424000 total_loss: 3.641 time: 0.3415 s/iter data_time: 0.2116 s/iter total_throughput: 2998.82 samples/s lr: 6.04e-04 [09/22 18:57:34] lb.utils.events INFO: eta: 9:11:50 iteration: 163599/375342 consumed_samples: 167526400 total_loss: 3.622 time: 0.3415 s/iter data_time: 0.2201 s/iter total_throughput: 2998.80 samples/s lr: 6.04e-04 [09/22 18:58:09] lb.utils.events INFO: eta: 9:10:58 iteration: 163699/375342 consumed_samples: 167628800 total_loss: 3.63 time: 0.3415 s/iter data_time: 0.2166 s/iter total_throughput: 2998.77 samples/s lr: 6.04e-04 [09/22 18:58:43] lb.utils.events INFO: eta: 9:09:36 iteration: 163799/375342 consumed_samples: 167731200 total_loss: 3.622 time: 0.3415 s/iter data_time: 0.2127 s/iter total_throughput: 2998.75 samples/s lr: 6.03e-04 [09/22 18:59:18] lb.utils.events INFO: eta: 9:09:36 iteration: 163899/375342 consumed_samples: 167833600 total_loss: 3.606 time: 0.3415 s/iter data_time: 0.2123 s/iter total_throughput: 2998.76 samples/s lr: 6.03e-04 [09/22 18:59:52] lb.utils.events INFO: eta: 9:10:07 iteration: 163999/375342 consumed_samples: 167936000 total_loss: 3.59 time: 0.3415 s/iter data_time: 0.2134 s/iter total_throughput: 2998.74 samples/s lr: 6.02e-04 [09/22 19:00:27] lb.utils.events INFO: eta: 9:10:44 iteration: 164099/375342 consumed_samples: 168038400 total_loss: 3.59 time: 0.3415 s/iter data_time: 0.2263 s/iter total_throughput: 2998.72 samples/s lr: 6.02e-04 [09/22 19:01:01] lb.utils.events INFO: eta: 9:11:05 iteration: 164199/375342 consumed_samples: 168140800 total_loss: 3.598 time: 0.3415 s/iter data_time: 0.2177 s/iter total_throughput: 2998.70 samples/s lr: 6.02e-04 [09/22 19:01:36] lb.utils.events INFO: eta: 9:10:30 iteration: 164299/375342 consumed_samples: 168243200 total_loss: 3.605 time: 0.3415 s/iter data_time: 0.2184 s/iter total_throughput: 2998.67 samples/s lr: 6.01e-04 [09/22 19:02:10] lb.utils.events INFO: eta: 9:10:07 iteration: 164399/375342 consumed_samples: 168345600 total_loss: 3.615 time: 0.3415 s/iter data_time: 0.2163 s/iter total_throughput: 2998.66 samples/s lr: 6.01e-04 [09/22 19:02:45] lb.utils.events INFO: eta: 9:08:39 iteration: 164499/375342 consumed_samples: 168448000 total_loss: 3.617 time: 0.3415 s/iter data_time: 0.2204 s/iter total_throughput: 2998.64 samples/s lr: 6.00e-04 [09/22 19:03:19] lb.utils.events INFO: eta: 9:07:13 iteration: 164599/375342 consumed_samples: 168550400 total_loss: 3.604 time: 0.3415 s/iter data_time: 0.2275 s/iter total_throughput: 2998.64 samples/s lr: 6.00e-04 [09/22 19:03:54] lb.utils.events INFO: eta: 9:08:01 iteration: 164699/375342 consumed_samples: 168652800 total_loss: 3.617 time: 0.3415 s/iter data_time: 0.2123 s/iter total_throughput: 2998.60 samples/s lr: 6.00e-04 [09/22 19:04:28] lb.utils.events INFO: eta: 9:08:26 iteration: 164799/375342 consumed_samples: 168755200 total_loss: 3.617 time: 0.3415 s/iter data_time: 0.2052 s/iter total_throughput: 2998.58 samples/s lr: 5.99e-04 [09/22 19:05:03] lb.utils.events INFO: eta: 9:06:37 iteration: 164899/375342 consumed_samples: 168857600 total_loss: 3.639 time: 0.3415 s/iter data_time: 0.2053 s/iter total_throughput: 2998.57 samples/s lr: 5.99e-04 [09/22 19:05:37] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0164999 [09/22 19:05:38] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/22 19:05:38] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/22 19:05:42] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0978 s/iter. Inference: 0.1608 s/iter. Eval: 0.0022 s/iter. Total: 0.2608 s/iter. ETA=0:00:09 [09/22 19:05:47] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1180 s/iter. Inference: 0.1770 s/iter. Eval: 0.0020 s/iter. Total: 0.2971 s/iter. ETA=0:00:05 [09/22 19:05:52] lb.evaluation.evaluator INFO: Inference done 48128/50000. Dataloading: 0.1195 s/iter. Inference: 0.1698 s/iter. Eval: 0.0020 s/iter. Total: 0.2915 s/iter. ETA=0:00:00 [09/22 19:05:53] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/22 19:05:53] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.596519 (0.000252 s / iter per device, on 8 devices) [09/22 19:05:53] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000149 s / iter per device, on 8 devices) [09/22 19:05:53] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/22 19:05:53] lb.evaluation.utils INFO: copypaste: Acc@1=74.264 [09/22 19:05:53] lb.evaluation.utils INFO: copypaste: Acc@5=92.074 [09/22 19:05:53] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 74.26400, better than last best score 73.75400 @ iteration 154999. [09/22 19:05:53] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/22 19:05:54] lb.utils.events INFO: eta: 9:06:11 iteration: 164999/375342 consumed_samples: 168960000 total_loss: 3.648 time: 0.3415 s/iter data_time: 0.2068 s/iter total_throughput: 2998.56 samples/s lr: 5.98e-04 [09/22 19:06:26] lb.utils.events INFO: eta: 9:08:27 iteration: 165099/375342 consumed_samples: 169062400 total_loss: 3.623 time: 0.3415 s/iter data_time: 0.2117 s/iter total_throughput: 2998.66 samples/s lr: 5.98e-04 [09/22 19:07:00] lb.utils.events INFO: eta: 9:09:00 iteration: 165199/375342 consumed_samples: 169164800 total_loss: 3.593 time: 0.3415 s/iter data_time: 0.2127 s/iter total_throughput: 2998.66 samples/s lr: 5.98e-04 [09/22 19:07:35] lb.utils.events INFO: eta: 9:08:42 iteration: 165299/375342 consumed_samples: 169267200 total_loss: 3.593 time: 0.3415 s/iter data_time: 0.2224 s/iter total_throughput: 2998.64 samples/s lr: 5.97e-04 [09/22 19:08:10] lb.utils.events INFO: eta: 9:07:25 iteration: 165399/375342 consumed_samples: 169369600 total_loss: 3.604 time: 0.3415 s/iter data_time: 0.2216 s/iter total_throughput: 2998.58 samples/s lr: 5.97e-04 [09/22 19:08:44] lb.utils.events INFO: eta: 9:07:01 iteration: 165499/375342 consumed_samples: 169472000 total_loss: 3.602 time: 0.3415 s/iter data_time: 0.2169 s/iter total_throughput: 2998.56 samples/s lr: 5.96e-04 [09/22 19:09:19] lb.utils.events INFO: eta: 9:05:30 iteration: 165599/375342 consumed_samples: 169574400 total_loss: 3.605 time: 0.3415 s/iter data_time: 0.2226 s/iter total_throughput: 2998.53 samples/s lr: 5.96e-04 [09/22 19:09:54] lb.utils.events INFO: eta: 9:04:01 iteration: 165699/375342 consumed_samples: 169676800 total_loss: 3.619 time: 0.3415 s/iter data_time: 0.2373 s/iter total_throughput: 2998.47 samples/s lr: 5.96e-04 [09/22 19:10:29] lb.utils.events INFO: eta: 9:03:58 iteration: 165799/375342 consumed_samples: 169779200 total_loss: 3.605 time: 0.3415 s/iter data_time: 0.2282 s/iter total_throughput: 2998.46 samples/s lr: 5.95e-04 [09/22 19:11:03] lb.utils.events INFO: eta: 9:04:37 iteration: 165899/375342 consumed_samples: 169881600 total_loss: 3.576 time: 0.3415 s/iter data_time: 0.2005 s/iter total_throughput: 2998.47 samples/s lr: 5.95e-04 [09/22 19:11:37] lb.utils.events INFO: eta: 9:04:40 iteration: 165999/375342 consumed_samples: 169984000 total_loss: 3.594 time: 0.3415 s/iter data_time: 0.2172 s/iter total_throughput: 2998.48 samples/s lr: 5.94e-04 [09/22 19:12:11] lb.utils.events INFO: eta: 9:00:36 iteration: 166099/375342 consumed_samples: 170086400 total_loss: 3.626 time: 0.3415 s/iter data_time: 0.2110 s/iter total_throughput: 2998.45 samples/s lr: 5.94e-04 [09/22 19:12:46] lb.utils.events INFO: eta: 9:00:56 iteration: 166199/375342 consumed_samples: 170188800 total_loss: 3.625 time: 0.3415 s/iter data_time: 0.2177 s/iter total_throughput: 2998.45 samples/s lr: 5.93e-04 [09/22 19:13:20] lb.utils.events INFO: eta: 9:01:52 iteration: 166299/375342 consumed_samples: 170291200 total_loss: 3.618 time: 0.3415 s/iter data_time: 0.2139 s/iter total_throughput: 2998.43 samples/s lr: 5.93e-04 [09/22 19:13:54] lb.utils.events INFO: eta: 9:03:09 iteration: 166399/375342 consumed_samples: 170393600 total_loss: 3.594 time: 0.3415 s/iter data_time: 0.2091 s/iter total_throughput: 2998.45 samples/s lr: 5.93e-04 [09/22 19:14:29] lb.utils.events INFO: eta: 9:04:21 iteration: 166499/375342 consumed_samples: 170496000 total_loss: 3.59 time: 0.3415 s/iter data_time: 0.2181 s/iter total_throughput: 2998.43 samples/s lr: 5.92e-04 [09/22 19:15:03] lb.utils.events INFO: eta: 9:04:14 iteration: 166599/375342 consumed_samples: 170598400 total_loss: 3.599 time: 0.3415 s/iter data_time: 0.2183 s/iter total_throughput: 2998.40 samples/s lr: 5.92e-04 [09/22 19:15:37] lb.utils.events INFO: eta: 9:04:41 iteration: 166699/375342 consumed_samples: 170700800 total_loss: 3.604 time: 0.3415 s/iter data_time: 0.2161 s/iter total_throughput: 2998.40 samples/s lr: 5.91e-04 [09/22 19:16:12] lb.utils.events INFO: eta: 9:04:09 iteration: 166799/375342 consumed_samples: 170803200 total_loss: 3.621 time: 0.3415 s/iter data_time: 0.2198 s/iter total_throughput: 2998.37 samples/s lr: 5.91e-04 [09/22 19:16:46] lb.utils.events INFO: eta: 9:02:59 iteration: 166899/375342 consumed_samples: 170905600 total_loss: 3.603 time: 0.3415 s/iter data_time: 0.2174 s/iter total_throughput: 2998.38 samples/s lr: 5.91e-04 [09/22 19:17:20] lb.utils.events INFO: eta: 9:01:43 iteration: 166999/375342 consumed_samples: 171008000 total_loss: 3.578 time: 0.3415 s/iter data_time: 0.2165 s/iter total_throughput: 2998.39 samples/s lr: 5.90e-04 [09/22 19:17:54] lb.utils.events INFO: eta: 9:02:13 iteration: 167099/375342 consumed_samples: 171110400 total_loss: 3.582 time: 0.3415 s/iter data_time: 0.2147 s/iter total_throughput: 2998.40 samples/s lr: 5.90e-04 [09/22 19:18:28] lb.utils.events INFO: eta: 9:00:55 iteration: 167199/375342 consumed_samples: 171212800 total_loss: 3.596 time: 0.3415 s/iter data_time: 0.2142 s/iter total_throughput: 2998.40 samples/s lr: 5.89e-04 [09/22 19:19:02] lb.utils.events INFO: eta: 9:01:21 iteration: 167299/375342 consumed_samples: 171315200 total_loss: 3.599 time: 0.3415 s/iter data_time: 0.2118 s/iter total_throughput: 2998.42 samples/s lr: 5.89e-04 [09/22 19:19:36] lb.utils.events INFO: eta: 9:00:50 iteration: 167399/375342 consumed_samples: 171417600 total_loss: 3.609 time: 0.3415 s/iter data_time: 0.2255 s/iter total_throughput: 2998.41 samples/s lr: 5.89e-04 [09/22 19:20:10] lb.utils.events INFO: eta: 9:00:32 iteration: 167499/375342 consumed_samples: 171520000 total_loss: 3.613 time: 0.3415 s/iter data_time: 0.2174 s/iter total_throughput: 2998.42 samples/s lr: 5.88e-04 [09/22 19:20:44] lb.utils.events INFO: eta: 9:00:16 iteration: 167599/375342 consumed_samples: 171622400 total_loss: 3.621 time: 0.3415 s/iter data_time: 0.2047 s/iter total_throughput: 2998.44 samples/s lr: 5.88e-04 [09/22 19:21:18] lb.utils.events INFO: eta: 9:00:01 iteration: 167699/375342 consumed_samples: 171724800 total_loss: 3.622 time: 0.3415 s/iter data_time: 0.2112 s/iter total_throughput: 2998.46 samples/s lr: 5.87e-04 [09/22 19:21:51] lb.utils.events INFO: eta: 9:00:42 iteration: 167799/375342 consumed_samples: 171827200 total_loss: 3.617 time: 0.3415 s/iter data_time: 0.2099 s/iter total_throughput: 2998.49 samples/s lr: 5.87e-04 [09/22 19:22:26] lb.utils.events INFO: eta: 9:00:37 iteration: 167899/375342 consumed_samples: 171929600 total_loss: 3.607 time: 0.3415 s/iter data_time: 0.2124 s/iter total_throughput: 2998.46 samples/s lr: 5.87e-04 [09/22 19:23:00] lb.utils.events INFO: eta: 9:00:35 iteration: 167999/375342 consumed_samples: 172032000 total_loss: 3.608 time: 0.3415 s/iter data_time: 0.2136 s/iter total_throughput: 2998.45 samples/s lr: 5.86e-04 [09/22 19:23:35] lb.utils.events INFO: eta: 9:01:47 iteration: 168099/375342 consumed_samples: 172134400 total_loss: 3.612 time: 0.3415 s/iter data_time: 0.2275 s/iter total_throughput: 2998.45 samples/s lr: 5.86e-04 [09/22 19:24:09] lb.utils.events INFO: eta: 9:00:03 iteration: 168199/375342 consumed_samples: 172236800 total_loss: 3.625 time: 0.3415 s/iter data_time: 0.2220 s/iter total_throughput: 2998.42 samples/s lr: 5.85e-04 [09/22 19:24:45] lb.utils.events INFO: eta: 8:58:32 iteration: 168299/375342 consumed_samples: 172339200 total_loss: 3.609 time: 0.3415 s/iter data_time: 0.2215 s/iter total_throughput: 2998.36 samples/s lr: 5.85e-04 [09/22 19:25:20] lb.utils.events INFO: eta: 8:59:08 iteration: 168399/375342 consumed_samples: 172441600 total_loss: 3.591 time: 0.3415 s/iter data_time: 0.2202 s/iter total_throughput: 2998.32 samples/s lr: 5.85e-04 [09/22 19:25:55] lb.utils.events INFO: eta: 8:57:53 iteration: 168499/375342 consumed_samples: 172544000 total_loss: 3.612 time: 0.3415 s/iter data_time: 0.2351 s/iter total_throughput: 2998.27 samples/s lr: 5.84e-04 [09/22 19:26:30] lb.utils.events INFO: eta: 8:57:48 iteration: 168599/375342 consumed_samples: 172646400 total_loss: 3.606 time: 0.3415 s/iter data_time: 0.2197 s/iter total_throughput: 2998.21 samples/s lr: 5.84e-04 [09/22 19:27:06] lb.utils.events INFO: eta: 8:56:24 iteration: 168699/375342 consumed_samples: 172748800 total_loss: 3.602 time: 0.3415 s/iter data_time: 0.2213 s/iter total_throughput: 2998.13 samples/s lr: 5.83e-04 [09/22 19:27:41] lb.utils.events INFO: eta: 8:54:45 iteration: 168799/375342 consumed_samples: 172851200 total_loss: 3.598 time: 0.3416 s/iter data_time: 0.2289 s/iter total_throughput: 2998.06 samples/s lr: 5.83e-04 [09/22 19:28:17] lb.utils.events INFO: eta: 8:53:24 iteration: 168899/375342 consumed_samples: 172953600 total_loss: 3.605 time: 0.3416 s/iter data_time: 0.2362 s/iter total_throughput: 2997.99 samples/s lr: 5.82e-04 [09/22 19:28:51] lb.utils.events INFO: eta: 8:54:12 iteration: 168999/375342 consumed_samples: 173056000 total_loss: 3.601 time: 0.3416 s/iter data_time: 0.2170 s/iter total_throughput: 2997.97 samples/s lr: 5.82e-04 [09/22 19:29:26] lb.utils.events INFO: eta: 8:52:55 iteration: 169099/375342 consumed_samples: 173158400 total_loss: 3.587 time: 0.3416 s/iter data_time: 0.2179 s/iter total_throughput: 2997.94 samples/s lr: 5.82e-04 [09/22 19:30:01] lb.utils.events INFO: eta: 8:53:44 iteration: 169199/375342 consumed_samples: 173260800 total_loss: 3.599 time: 0.3416 s/iter data_time: 0.2163 s/iter total_throughput: 2997.90 samples/s lr: 5.81e-04 [09/22 19:30:36] lb.utils.events INFO: eta: 8:52:19 iteration: 169299/375342 consumed_samples: 173363200 total_loss: 3.592 time: 0.3416 s/iter data_time: 0.2303 s/iter total_throughput: 2997.85 samples/s lr: 5.81e-04 [09/22 19:31:11] lb.utils.events INFO: eta: 8:50:25 iteration: 169399/375342 consumed_samples: 173465600 total_loss: 3.584 time: 0.3416 s/iter data_time: 0.2345 s/iter total_throughput: 2997.80 samples/s lr: 5.80e-04 [09/22 19:31:47] lb.utils.events INFO: eta: 8:50:41 iteration: 169499/375342 consumed_samples: 173568000 total_loss: 3.614 time: 0.3416 s/iter data_time: 0.2216 s/iter total_throughput: 2997.71 samples/s lr: 5.80e-04 [09/22 19:32:22] lb.utils.events INFO: eta: 8:50:34 iteration: 169599/375342 consumed_samples: 173670400 total_loss: 3.6 time: 0.3416 s/iter data_time: 0.2220 s/iter total_throughput: 2997.68 samples/s lr: 5.80e-04 [09/22 19:32:57] lb.utils.events INFO: eta: 8:50:29 iteration: 169699/375342 consumed_samples: 173772800 total_loss: 3.58 time: 0.3416 s/iter data_time: 0.2245 s/iter total_throughput: 2997.63 samples/s lr: 5.79e-04 [09/22 19:33:32] lb.utils.events INFO: eta: 8:51:22 iteration: 169799/375342 consumed_samples: 173875200 total_loss: 3.586 time: 0.3416 s/iter data_time: 0.2273 s/iter total_throughput: 2997.59 samples/s lr: 5.79e-04 [09/22 19:34:07] lb.utils.events INFO: eta: 8:51:56 iteration: 169899/375342 consumed_samples: 173977600 total_loss: 3.606 time: 0.3416 s/iter data_time: 0.2380 s/iter total_throughput: 2997.55 samples/s lr: 5.78e-04 [09/22 19:34:42] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0169999 [09/22 19:34:42] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/22 19:34:42] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/22 19:34:46] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0958 s/iter. Inference: 0.1614 s/iter. Eval: 0.0019 s/iter. Total: 0.2591 s/iter. ETA=0:00:09 [09/22 19:34:51] lb.evaluation.evaluator INFO: Inference done 28672/50000. Dataloading: 0.1055 s/iter. Inference: 0.1778 s/iter. Eval: 0.0020 s/iter. Total: 0.2853 s/iter. ETA=0:00:05 [09/22 19:34:56] lb.evaluation.evaluator INFO: Inference done 46080/50000. Dataloading: 0.1154 s/iter. Inference: 0.1719 s/iter. Eval: 0.0020 s/iter. Total: 0.2895 s/iter. ETA=0:00:00 [09/22 19:34:57] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/22 19:34:57] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.503556 (0.000250 s / iter per device, on 8 devices) [09/22 19:34:57] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000151 s / iter per device, on 8 devices) [09/22 19:34:57] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/22 19:34:57] lb.evaluation.utils INFO: copypaste: Acc@1=74.51 [09/22 19:34:57] lb.evaluation.utils INFO: copypaste: Acc@5=92.054 [09/22 19:34:57] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 74.51000, better than last best score 74.26400 @ iteration 164999. [09/22 19:34:57] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/22 19:34:58] lb.utils.events INFO: eta: 8:49:55 iteration: 169999/375342 consumed_samples: 174080000 total_loss: 3.615 time: 0.3416 s/iter data_time: 0.2238 s/iter total_throughput: 2997.52 samples/s lr: 5.78e-04 [09/22 19:35:31] lb.utils.events INFO: eta: 8:52:41 iteration: 170099/375342 consumed_samples: 174182400 total_loss: 3.599 time: 0.3416 s/iter data_time: 0.2354 s/iter total_throughput: 2997.60 samples/s lr: 5.78e-04 [09/22 19:36:06] lb.utils.events INFO: eta: 8:52:30 iteration: 170199/375342 consumed_samples: 174284800 total_loss: 3.596 time: 0.3416 s/iter data_time: 0.2337 s/iter total_throughput: 2997.55 samples/s lr: 5.77e-04 [09/22 19:36:41] lb.utils.events INFO: eta: 8:54:15 iteration: 170299/375342 consumed_samples: 174387200 total_loss: 3.616 time: 0.3416 s/iter data_time: 0.2222 s/iter total_throughput: 2997.50 samples/s lr: 5.77e-04 [09/22 19:37:16] lb.utils.events INFO: eta: 8:52:56 iteration: 170399/375342 consumed_samples: 174489600 total_loss: 3.624 time: 0.3416 s/iter data_time: 0.2309 s/iter total_throughput: 2997.47 samples/s lr: 5.76e-04 [09/22 19:37:51] lb.utils.events INFO: eta: 8:52:49 iteration: 170499/375342 consumed_samples: 174592000 total_loss: 3.599 time: 0.3416 s/iter data_time: 0.2155 s/iter total_throughput: 2997.43 samples/s lr: 5.76e-04 [09/22 19:38:25] lb.utils.events INFO: eta: 8:52:42 iteration: 170599/375342 consumed_samples: 174694400 total_loss: 3.598 time: 0.3416 s/iter data_time: 0.2093 s/iter total_throughput: 2997.43 samples/s lr: 5.75e-04 [09/22 19:39:00] lb.utils.events INFO: eta: 8:55:04 iteration: 170699/375342 consumed_samples: 174796800 total_loss: 3.604 time: 0.3416 s/iter data_time: 0.2224 s/iter total_throughput: 2997.39 samples/s lr: 5.75e-04 [09/22 19:39:35] lb.utils.events INFO: eta: 8:52:21 iteration: 170799/375342 consumed_samples: 174899200 total_loss: 3.623 time: 0.3416 s/iter data_time: 0.2229 s/iter total_throughput: 2997.36 samples/s lr: 5.75e-04 [09/22 19:40:10] lb.utils.events INFO: eta: 8:53:22 iteration: 170899/375342 consumed_samples: 175001600 total_loss: 3.617 time: 0.3416 s/iter data_time: 0.2366 s/iter total_throughput: 2997.31 samples/s lr: 5.74e-04 [09/22 19:40:45] lb.utils.events INFO: eta: 8:52:38 iteration: 170999/375342 consumed_samples: 175104000 total_loss: 3.591 time: 0.3416 s/iter data_time: 0.2270 s/iter total_throughput: 2997.27 samples/s lr: 5.74e-04 [09/22 19:41:21] lb.utils.events INFO: eta: 8:48:32 iteration: 171099/375342 consumed_samples: 175206400 total_loss: 3.601 time: 0.3417 s/iter data_time: 0.2499 s/iter total_throughput: 2997.18 samples/s lr: 5.73e-04 [09/22 19:41:56] lb.utils.events INFO: eta: 8:48:17 iteration: 171199/375342 consumed_samples: 175308800 total_loss: 3.598 time: 0.3417 s/iter data_time: 0.2143 s/iter total_throughput: 2997.14 samples/s lr: 5.73e-04 [09/22 19:42:30] lb.utils.events INFO: eta: 8:48:03 iteration: 171299/375342 consumed_samples: 175411200 total_loss: 3.594 time: 0.3417 s/iter data_time: 0.2125 s/iter total_throughput: 2997.11 samples/s lr: 5.73e-04 [09/22 19:43:06] lb.utils.events INFO: eta: 8:48:19 iteration: 171399/375342 consumed_samples: 175513600 total_loss: 3.603 time: 0.3417 s/iter data_time: 0.2306 s/iter total_throughput: 2997.05 samples/s lr: 5.72e-04 [09/22 19:43:41] lb.utils.events INFO: eta: 8:48:27 iteration: 171499/375342 consumed_samples: 175616000 total_loss: 3.595 time: 0.3417 s/iter data_time: 0.2208 s/iter total_throughput: 2997.02 samples/s lr: 5.72e-04 [09/22 19:44:16] lb.utils.events INFO: eta: 8:46:49 iteration: 171599/375342 consumed_samples: 175718400 total_loss: 3.583 time: 0.3417 s/iter data_time: 0.2250 s/iter total_throughput: 2996.98 samples/s lr: 5.71e-04 [09/22 19:44:50] lb.utils.events INFO: eta: 8:47:15 iteration: 171699/375342 consumed_samples: 175820800 total_loss: 3.587 time: 0.3417 s/iter data_time: 0.2270 s/iter total_throughput: 2996.94 samples/s lr: 5.71e-04 [09/22 19:45:26] lb.utils.events INFO: eta: 8:46:51 iteration: 171799/375342 consumed_samples: 175923200 total_loss: 3.6 time: 0.3417 s/iter data_time: 0.2204 s/iter total_throughput: 2996.90 samples/s lr: 5.71e-04 [09/22 19:46:00] lb.utils.events INFO: eta: 8:45:48 iteration: 171899/375342 consumed_samples: 176025600 total_loss: 3.594 time: 0.3417 s/iter data_time: 0.2295 s/iter total_throughput: 2996.88 samples/s lr: 5.70e-04 [09/22 19:46:35] lb.utils.events INFO: eta: 8:46:33 iteration: 171999/375342 consumed_samples: 176128000 total_loss: 3.597 time: 0.3417 s/iter data_time: 0.2395 s/iter total_throughput: 2996.83 samples/s lr: 5.70e-04 [09/22 19:47:10] lb.utils.events INFO: eta: 8:47:03 iteration: 172099/375342 consumed_samples: 176230400 total_loss: 3.61 time: 0.3417 s/iter data_time: 0.2169 s/iter total_throughput: 2996.79 samples/s lr: 5.69e-04 [09/22 19:47:45] lb.utils.events INFO: eta: 8:46:09 iteration: 172199/375342 consumed_samples: 176332800 total_loss: 3.604 time: 0.3417 s/iter data_time: 0.2183 s/iter total_throughput: 2996.74 samples/s lr: 5.69e-04 [09/22 19:48:21] lb.utils.events INFO: eta: 8:44:53 iteration: 172299/375342 consumed_samples: 176435200 total_loss: 3.594 time: 0.3417 s/iter data_time: 0.2132 s/iter total_throughput: 2996.69 samples/s lr: 5.69e-04 [09/22 19:48:56] lb.utils.events INFO: eta: 8:45:21 iteration: 172399/375342 consumed_samples: 176537600 total_loss: 3.587 time: 0.3417 s/iter data_time: 0.2276 s/iter total_throughput: 2996.65 samples/s lr: 5.68e-04 [09/22 19:49:30] lb.utils.events INFO: eta: 8:45:24 iteration: 172499/375342 consumed_samples: 176640000 total_loss: 3.584 time: 0.3417 s/iter data_time: 0.2249 s/iter total_throughput: 2996.64 samples/s lr: 5.68e-04 [09/22 19:50:05] lb.utils.events INFO: eta: 8:45:07 iteration: 172599/375342 consumed_samples: 176742400 total_loss: 3.581 time: 0.3417 s/iter data_time: 0.2319 s/iter total_throughput: 2996.60 samples/s lr: 5.67e-04 [09/22 19:50:39] lb.utils.events INFO: eta: 8:44:26 iteration: 172699/375342 consumed_samples: 176844800 total_loss: 3.581 time: 0.3417 s/iter data_time: 0.2175 s/iter total_throughput: 2996.60 samples/s lr: 5.67e-04 [09/22 19:51:14] lb.utils.events INFO: eta: 8:44:36 iteration: 172799/375342 consumed_samples: 176947200 total_loss: 3.571 time: 0.3417 s/iter data_time: 0.2110 s/iter total_throughput: 2996.56 samples/s lr: 5.66e-04 [09/22 19:51:49] lb.utils.events INFO: eta: 8:44:39 iteration: 172899/375342 consumed_samples: 177049600 total_loss: 3.573 time: 0.3417 s/iter data_time: 0.2279 s/iter total_throughput: 2996.53 samples/s lr: 5.66e-04 [09/22 19:52:24] lb.utils.events INFO: eta: 8:44:07 iteration: 172999/375342 consumed_samples: 177152000 total_loss: 3.592 time: 0.3417 s/iter data_time: 0.2252 s/iter total_throughput: 2996.46 samples/s lr: 5.66e-04 [09/22 19:52:59] lb.utils.events INFO: eta: 8:44:03 iteration: 173099/375342 consumed_samples: 177254400 total_loss: 3.59 time: 0.3417 s/iter data_time: 0.2293 s/iter total_throughput: 2996.42 samples/s lr: 5.65e-04 [09/22 19:53:34] lb.utils.events INFO: eta: 8:43:59 iteration: 173199/375342 consumed_samples: 177356800 total_loss: 3.582 time: 0.3417 s/iter data_time: 0.2207 s/iter total_throughput: 2996.39 samples/s lr: 5.65e-04 [09/22 19:54:09] lb.utils.events INFO: eta: 8:43:55 iteration: 173299/375342 consumed_samples: 177459200 total_loss: 3.578 time: 0.3417 s/iter data_time: 0.2294 s/iter total_throughput: 2996.35 samples/s lr: 5.64e-04 [09/22 19:54:44] lb.utils.events INFO: eta: 8:43:08 iteration: 173399/375342 consumed_samples: 177561600 total_loss: 3.578 time: 0.3418 s/iter data_time: 0.2278 s/iter total_throughput: 2996.32 samples/s lr: 5.64e-04 [09/22 19:55:19] lb.utils.events INFO: eta: 8:43:17 iteration: 173499/375342 consumed_samples: 177664000 total_loss: 3.587 time: 0.3418 s/iter data_time: 0.2147 s/iter total_throughput: 2996.29 samples/s lr: 5.64e-04 [09/22 19:55:53] lb.utils.events INFO: eta: 8:44:44 iteration: 173599/375342 consumed_samples: 177766400 total_loss: 3.605 time: 0.3418 s/iter data_time: 0.2172 s/iter total_throughput: 2996.27 samples/s lr: 5.63e-04 [09/22 19:56:28] lb.utils.events INFO: eta: 8:44:16 iteration: 173699/375342 consumed_samples: 177868800 total_loss: 3.615 time: 0.3418 s/iter data_time: 0.2275 s/iter total_throughput: 2996.24 samples/s lr: 5.63e-04 [09/22 19:57:03] lb.utils.events INFO: eta: 8:45:15 iteration: 173799/375342 consumed_samples: 177971200 total_loss: 3.577 time: 0.3418 s/iter data_time: 0.2104 s/iter total_throughput: 2996.21 samples/s lr: 5.62e-04 [09/22 19:57:37] lb.utils.events INFO: eta: 8:44:50 iteration: 173899/375342 consumed_samples: 178073600 total_loss: 3.566 time: 0.3418 s/iter data_time: 0.2204 s/iter total_throughput: 2996.19 samples/s lr: 5.62e-04 [09/22 19:58:12] lb.utils.events INFO: eta: 8:44:45 iteration: 173999/375342 consumed_samples: 178176000 total_loss: 3.573 time: 0.3418 s/iter data_time: 0.2362 s/iter total_throughput: 2996.15 samples/s lr: 5.62e-04 [09/22 19:58:48] lb.utils.events INFO: eta: 8:43:04 iteration: 174099/375342 consumed_samples: 178278400 total_loss: 3.568 time: 0.3418 s/iter data_time: 0.2211 s/iter total_throughput: 2996.09 samples/s lr: 5.61e-04 [09/22 19:59:23] lb.utils.events INFO: eta: 8:43:31 iteration: 174199/375342 consumed_samples: 178380800 total_loss: 3.583 time: 0.3418 s/iter data_time: 0.2111 s/iter total_throughput: 2996.06 samples/s lr: 5.61e-04 [09/22 19:59:57] lb.utils.events INFO: eta: 8:45:24 iteration: 174299/375342 consumed_samples: 178483200 total_loss: 3.604 time: 0.3418 s/iter data_time: 0.2088 s/iter total_throughput: 2996.05 samples/s lr: 5.60e-04 [09/22 20:00:32] lb.utils.events INFO: eta: 8:44:07 iteration: 174399/375342 consumed_samples: 178585600 total_loss: 3.591 time: 0.3418 s/iter data_time: 0.2275 s/iter total_throughput: 2995.99 samples/s lr: 5.60e-04 [09/22 20:01:07] lb.utils.events INFO: eta: 8:42:56 iteration: 174499/375342 consumed_samples: 178688000 total_loss: 3.577 time: 0.3418 s/iter data_time: 0.2112 s/iter total_throughput: 2995.95 samples/s lr: 5.59e-04 [09/22 20:01:42] lb.utils.events INFO: eta: 8:42:14 iteration: 174599/375342 consumed_samples: 178790400 total_loss: 3.592 time: 0.3418 s/iter data_time: 0.2181 s/iter total_throughput: 2995.92 samples/s lr: 5.59e-04 [09/22 20:02:17] lb.utils.events INFO: eta: 8:41:30 iteration: 174699/375342 consumed_samples: 178892800 total_loss: 3.594 time: 0.3418 s/iter data_time: 0.2254 s/iter total_throughput: 2995.88 samples/s lr: 5.59e-04 [09/22 20:02:52] lb.utils.events INFO: eta: 8:41:07 iteration: 174799/375342 consumed_samples: 178995200 total_loss: 3.587 time: 0.3418 s/iter data_time: 0.2179 s/iter total_throughput: 2995.83 samples/s lr: 5.58e-04 [09/22 20:03:27] lb.utils.events INFO: eta: 8:40:00 iteration: 174899/375342 consumed_samples: 179097600 total_loss: 3.585 time: 0.3418 s/iter data_time: 0.2118 s/iter total_throughput: 2995.81 samples/s lr: 5.58e-04 [09/22 20:04:01] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0174999 [09/22 20:04:02] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/22 20:04:02] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/22 20:04:06] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0982 s/iter. Inference: 0.1609 s/iter. Eval: 0.0022 s/iter. Total: 0.2614 s/iter. ETA=0:00:09 [09/22 20:04:12] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1261 s/iter. Inference: 0.1702 s/iter. Eval: 0.0021 s/iter. Total: 0.2985 s/iter. ETA=0:00:05 [09/22 20:04:17] lb.evaluation.evaluator INFO: Inference done 49152/50000. Dataloading: 0.1167 s/iter. Inference: 0.1645 s/iter. Eval: 0.0021 s/iter. Total: 0.2834 s/iter. ETA=0:00:00 [09/22 20:04:17] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/22 20:04:17] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.632253 (0.000253 s / iter per device, on 8 devices) [09/22 20:04:17] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000146 s / iter per device, on 8 devices) [09/22 20:04:17] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/22 20:04:17] lb.evaluation.utils INFO: copypaste: Acc@1=74.49 [09/22 20:04:17] lb.evaluation.utils INFO: copypaste: Acc@5=92.412 [09/22 20:04:17] lb.engine.hooks INFO: Not saving as latest eval score for Acc@1 is 74.49000, not better than best score 74.51000 @ iteration 169999. [09/22 20:04:17] lb.utils.events INFO: eta: 8:40:13 iteration: 174999/375342 consumed_samples: 179200000 total_loss: 3.592 time: 0.3418 s/iter data_time: 0.2265 s/iter total_throughput: 2995.80 samples/s lr: 5.57e-04 [09/22 20:04:50] lb.utils.events INFO: eta: 8:41:21 iteration: 175099/375342 consumed_samples: 179302400 total_loss: 3.582 time: 0.3418 s/iter data_time: 0.2076 s/iter total_throughput: 2995.85 samples/s lr: 5.57e-04 [09/22 20:05:25] lb.utils.events INFO: eta: 8:40:14 iteration: 175199/375342 consumed_samples: 179404800 total_loss: 3.573 time: 0.3418 s/iter data_time: 0.2186 s/iter total_throughput: 2995.83 samples/s lr: 5.57e-04 [09/22 20:05:59] lb.utils.events INFO: eta: 8:39:23 iteration: 175299/375342 consumed_samples: 179507200 total_loss: 3.593 time: 0.3418 s/iter data_time: 0.2191 s/iter total_throughput: 2995.81 samples/s lr: 5.56e-04 [09/22 20:06:34] lb.utils.events INFO: eta: 8:39:32 iteration: 175399/375342 consumed_samples: 179609600 total_loss: 3.59 time: 0.3418 s/iter data_time: 0.2269 s/iter total_throughput: 2995.79 samples/s lr: 5.56e-04 [09/22 20:07:09] lb.utils.events INFO: eta: 8:38:49 iteration: 175499/375342 consumed_samples: 179712000 total_loss: 3.567 time: 0.3418 s/iter data_time: 0.2109 s/iter total_throughput: 2995.76 samples/s lr: 5.55e-04 [09/22 20:07:44] lb.utils.events INFO: eta: 8:37:37 iteration: 175599/375342 consumed_samples: 179814400 total_loss: 3.557 time: 0.3418 s/iter data_time: 0.2346 s/iter total_throughput: 2995.72 samples/s lr: 5.55e-04 [09/22 20:08:19] lb.utils.events INFO: eta: 8:37:11 iteration: 175699/375342 consumed_samples: 179916800 total_loss: 3.578 time: 0.3418 s/iter data_time: 0.2210 s/iter total_throughput: 2995.68 samples/s lr: 5.55e-04 [09/22 20:08:53] lb.utils.events INFO: eta: 8:36:39 iteration: 175799/375342 consumed_samples: 180019200 total_loss: 3.601 time: 0.3418 s/iter data_time: 0.2207 s/iter total_throughput: 2995.66 samples/s lr: 5.54e-04 [09/22 20:09:28] lb.utils.events INFO: eta: 8:36:22 iteration: 175899/375342 consumed_samples: 180121600 total_loss: 3.594 time: 0.3418 s/iter data_time: 0.2273 s/iter total_throughput: 2995.62 samples/s lr: 5.54e-04 [09/22 20:10:03] lb.utils.events INFO: eta: 8:35:59 iteration: 175999/375342 consumed_samples: 180224000 total_loss: 3.571 time: 0.3418 s/iter data_time: 0.2356 s/iter total_throughput: 2995.59 samples/s lr: 5.53e-04 [09/22 20:10:38] lb.utils.events INFO: eta: 8:34:32 iteration: 176099/375342 consumed_samples: 180326400 total_loss: 3.57 time: 0.3418 s/iter data_time: 0.2280 s/iter total_throughput: 2995.57 samples/s lr: 5.53e-04 [09/22 20:11:12] lb.utils.events INFO: eta: 8:34:29 iteration: 176199/375342 consumed_samples: 180428800 total_loss: 3.583 time: 0.3418 s/iter data_time: 0.2095 s/iter total_throughput: 2995.55 samples/s lr: 5.52e-04 [09/22 20:11:47] lb.utils.events INFO: eta: 8:34:22 iteration: 176299/375342 consumed_samples: 180531200 total_loss: 3.585 time: 0.3418 s/iter data_time: 0.2154 s/iter total_throughput: 2995.53 samples/s lr: 5.52e-04 [09/22 20:12:21] lb.utils.events INFO: eta: 8:34:04 iteration: 176399/375342 consumed_samples: 180633600 total_loss: 3.593 time: 0.3418 s/iter data_time: 0.2111 s/iter total_throughput: 2995.52 samples/s lr: 5.52e-04 [09/22 20:12:56] lb.utils.events INFO: eta: 8:33:53 iteration: 176499/375342 consumed_samples: 180736000 total_loss: 3.584 time: 0.3418 s/iter data_time: 0.2123 s/iter total_throughput: 2995.51 samples/s lr: 5.51e-04 [09/22 20:13:30] lb.utils.events INFO: eta: 8:34:33 iteration: 176599/375342 consumed_samples: 180838400 total_loss: 3.606 time: 0.3418 s/iter data_time: 0.2210 s/iter total_throughput: 2995.51 samples/s lr: 5.51e-04 [09/22 20:14:05] lb.utils.events INFO: eta: 8:34:17 iteration: 176699/375342 consumed_samples: 180940800 total_loss: 3.612 time: 0.3418 s/iter data_time: 0.2175 s/iter total_throughput: 2995.47 samples/s lr: 5.50e-04 [09/22 20:14:40] lb.utils.events INFO: eta: 8:34:02 iteration: 176799/375342 consumed_samples: 181043200 total_loss: 3.604 time: 0.3419 s/iter data_time: 0.2099 s/iter total_throughput: 2995.45 samples/s lr: 5.50e-04 [09/22 20:15:14] lb.utils.events INFO: eta: 8:34:39 iteration: 176899/375342 consumed_samples: 181145600 total_loss: 3.594 time: 0.3419 s/iter data_time: 0.2172 s/iter total_throughput: 2995.44 samples/s lr: 5.50e-04 [09/22 20:15:48] lb.utils.events INFO: eta: 8:34:23 iteration: 176999/375342 consumed_samples: 181248000 total_loss: 3.571 time: 0.3419 s/iter data_time: 0.2182 s/iter total_throughput: 2995.42 samples/s lr: 5.49e-04 [09/22 20:16:23] lb.utils.events INFO: eta: 8:35:33 iteration: 177099/375342 consumed_samples: 181350400 total_loss: 3.585 time: 0.3419 s/iter data_time: 0.2259 s/iter total_throughput: 2995.39 samples/s lr: 5.49e-04 [09/22 20:16:58] lb.utils.events INFO: eta: 8:34:01 iteration: 177199/375342 consumed_samples: 181452800 total_loss: 3.603 time: 0.3419 s/iter data_time: 0.2106 s/iter total_throughput: 2995.37 samples/s lr: 5.48e-04 [09/22 20:17:33] lb.utils.events INFO: eta: 8:33:54 iteration: 177299/375342 consumed_samples: 181555200 total_loss: 3.594 time: 0.3419 s/iter data_time: 0.2195 s/iter total_throughput: 2995.35 samples/s lr: 5.48e-04 [09/22 20:18:07] lb.utils.events INFO: eta: 8:34:53 iteration: 177399/375342 consumed_samples: 181657600 total_loss: 3.564 time: 0.3419 s/iter data_time: 0.2144 s/iter total_throughput: 2995.35 samples/s lr: 5.48e-04 [09/22 20:18:41] lb.utils.events INFO: eta: 8:35:28 iteration: 177499/375342 consumed_samples: 181760000 total_loss: 3.57 time: 0.3419 s/iter data_time: 0.2212 s/iter total_throughput: 2995.34 samples/s lr: 5.47e-04 [09/22 20:19:16] lb.utils.events INFO: eta: 8:35:12 iteration: 177599/375342 consumed_samples: 181862400 total_loss: 3.579 time: 0.3419 s/iter data_time: 0.2076 s/iter total_throughput: 2995.33 samples/s lr: 5.47e-04 [09/22 20:19:51] lb.utils.events INFO: eta: 8:35:32 iteration: 177699/375342 consumed_samples: 181964800 total_loss: 3.565 time: 0.3419 s/iter data_time: 0.2368 s/iter total_throughput: 2995.28 samples/s lr: 5.46e-04 [09/22 20:20:26] lb.utils.events INFO: eta: 8:34:54 iteration: 177799/375342 consumed_samples: 182067200 total_loss: 3.554 time: 0.3419 s/iter data_time: 0.2284 s/iter total_throughput: 2995.24 samples/s lr: 5.46e-04 [09/22 20:21:00] lb.utils.events INFO: eta: 8:34:04 iteration: 177899/375342 consumed_samples: 182169600 total_loss: 3.562 time: 0.3419 s/iter data_time: 0.2147 s/iter total_throughput: 2995.22 samples/s lr: 5.45e-04 [09/22 20:21:35] lb.utils.events INFO: eta: 8:33:28 iteration: 177999/375342 consumed_samples: 182272000 total_loss: 3.571 time: 0.3419 s/iter data_time: 0.2181 s/iter total_throughput: 2995.19 samples/s lr: 5.45e-04 [09/22 20:22:09] lb.utils.events INFO: eta: 8:33:16 iteration: 178099/375342 consumed_samples: 182374400 total_loss: 3.589 time: 0.3419 s/iter data_time: 0.2198 s/iter total_throughput: 2995.18 samples/s lr: 5.45e-04 [09/22 20:22:44] lb.utils.events INFO: eta: 8:32:57 iteration: 178199/375342 consumed_samples: 182476800 total_loss: 3.596 time: 0.3419 s/iter data_time: 0.2329 s/iter total_throughput: 2995.15 samples/s lr: 5.44e-04 [09/22 20:23:19] lb.utils.events INFO: eta: 8:32:17 iteration: 178299/375342 consumed_samples: 182579200 total_loss: 3.571 time: 0.3419 s/iter data_time: 0.2244 s/iter total_throughput: 2995.12 samples/s lr: 5.44e-04 [09/22 20:23:54] lb.utils.events INFO: eta: 8:31:25 iteration: 178399/375342 consumed_samples: 182681600 total_loss: 3.57 time: 0.3419 s/iter data_time: 0.2168 s/iter total_throughput: 2995.10 samples/s lr: 5.43e-04 [09/22 20:24:29] lb.utils.events INFO: eta: 8:29:56 iteration: 178499/375342 consumed_samples: 182784000 total_loss: 3.562 time: 0.3419 s/iter data_time: 0.2383 s/iter total_throughput: 2995.05 samples/s lr: 5.43e-04 [09/22 20:25:03] lb.utils.events INFO: eta: 8:29:52 iteration: 178599/375342 consumed_samples: 182886400 total_loss: 3.57 time: 0.3419 s/iter data_time: 0.2207 s/iter total_throughput: 2995.04 samples/s lr: 5.43e-04 [09/22 20:25:38] lb.utils.events INFO: eta: 8:29:41 iteration: 178699/375342 consumed_samples: 182988800 total_loss: 3.573 time: 0.3419 s/iter data_time: 0.2137 s/iter total_throughput: 2995.02 samples/s lr: 5.42e-04 [09/22 20:26:13] lb.utils.events INFO: eta: 8:29:57 iteration: 178799/375342 consumed_samples: 183091200 total_loss: 3.572 time: 0.3419 s/iter data_time: 0.2052 s/iter total_throughput: 2995.00 samples/s lr: 5.42e-04 [09/22 20:26:47] lb.utils.events INFO: eta: 8:30:01 iteration: 178899/375342 consumed_samples: 183193600 total_loss: 3.573 time: 0.3419 s/iter data_time: 0.2178 s/iter total_throughput: 2995.00 samples/s lr: 5.41e-04 [09/22 20:27:21] lb.utils.events INFO: eta: 8:30:34 iteration: 178999/375342 consumed_samples: 183296000 total_loss: 3.579 time: 0.3419 s/iter data_time: 0.2164 s/iter total_throughput: 2995.00 samples/s lr: 5.41e-04 [09/22 20:27:55] lb.utils.events INFO: eta: 8:29:35 iteration: 179099/375342 consumed_samples: 183398400 total_loss: 3.567 time: 0.3419 s/iter data_time: 0.2137 s/iter total_throughput: 2995.01 samples/s lr: 5.40e-04 [09/22 20:28:29] lb.utils.events INFO: eta: 8:29:33 iteration: 179199/375342 consumed_samples: 183500800 total_loss: 3.571 time: 0.3419 s/iter data_time: 0.2171 s/iter total_throughput: 2995.00 samples/s lr: 5.40e-04 [09/22 20:29:04] lb.utils.events INFO: eta: 8:30:22 iteration: 179299/375342 consumed_samples: 183603200 total_loss: 3.571 time: 0.3419 s/iter data_time: 0.2236 s/iter total_throughput: 2994.99 samples/s lr: 5.40e-04 [09/22 20:29:38] lb.utils.events INFO: eta: 8:29:46 iteration: 179399/375342 consumed_samples: 183705600 total_loss: 3.559 time: 0.3419 s/iter data_time: 0.2238 s/iter total_throughput: 2994.97 samples/s lr: 5.39e-04 [09/22 20:30:13] lb.utils.events INFO: eta: 8:30:29 iteration: 179499/375342 consumed_samples: 183808000 total_loss: 3.578 time: 0.3419 s/iter data_time: 0.2157 s/iter total_throughput: 2994.93 samples/s lr: 5.39e-04 [09/22 20:30:48] lb.utils.events INFO: eta: 8:29:25 iteration: 179599/375342 consumed_samples: 183910400 total_loss: 3.561 time: 0.3419 s/iter data_time: 0.2168 s/iter total_throughput: 2994.93 samples/s lr: 5.38e-04 [09/22 20:31:22] lb.utils.events INFO: eta: 8:28:31 iteration: 179699/375342 consumed_samples: 184012800 total_loss: 3.559 time: 0.3419 s/iter data_time: 0.2206 s/iter total_throughput: 2994.90 samples/s lr: 5.38e-04 [09/22 20:31:57] lb.utils.events INFO: eta: 8:28:04 iteration: 179799/375342 consumed_samples: 184115200 total_loss: 3.584 time: 0.3419 s/iter data_time: 0.2132 s/iter total_throughput: 2994.90 samples/s lr: 5.38e-04 [09/22 20:32:31] lb.utils.events INFO: eta: 8:27:38 iteration: 179899/375342 consumed_samples: 184217600 total_loss: 3.565 time: 0.3419 s/iter data_time: 0.2141 s/iter total_throughput: 2994.88 samples/s lr: 5.37e-04 [09/22 20:33:06] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0179999 [09/22 20:33:07] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/22 20:33:07] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/22 20:33:11] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0915 s/iter. Inference: 0.1606 s/iter. Eval: 0.0021 s/iter. Total: 0.2542 s/iter. ETA=0:00:09 [09/22 20:33:17] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1418 s/iter. Inference: 0.1611 s/iter. Eval: 0.0021 s/iter. Total: 0.3050 s/iter. ETA=0:00:05 [09/22 20:33:22] lb.evaluation.evaluator INFO: Inference done 50000/50000. Dataloading: 0.1249 s/iter. Inference: 0.1627 s/iter. Eval: 0.0020 s/iter. Total: 0.2897 s/iter. ETA=0:00:00 [09/22 20:33:22] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/22 20:33:22] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.751464 (0.000255 s / iter per device, on 8 devices) [09/22 20:33:22] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000143 s / iter per device, on 8 devices) [09/22 20:33:22] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/22 20:33:22] lb.evaluation.utils INFO: copypaste: Acc@1=74.57000000000001 [09/22 20:33:22] lb.evaluation.utils INFO: copypaste: Acc@5=92.202 [09/22 20:33:22] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 74.57000, better than last best score 74.51000 @ iteration 169999. [09/22 20:33:22] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/22 20:33:23] lb.utils.events INFO: eta: 8:27:23 iteration: 179999/375342 consumed_samples: 184320000 total_loss: 3.537 time: 0.3419 s/iter data_time: 0.2265 s/iter total_throughput: 2994.86 samples/s lr: 5.37e-04 [09/22 20:33:55] lb.utils.events INFO: eta: 8:28:13 iteration: 180099/375342 consumed_samples: 184422400 total_loss: 3.536 time: 0.3419 s/iter data_time: 0.2107 s/iter total_throughput: 2994.96 samples/s lr: 5.36e-04 [09/22 20:34:30] lb.utils.events INFO: eta: 8:27:20 iteration: 180199/375342 consumed_samples: 184524800 total_loss: 3.545 time: 0.3419 s/iter data_time: 0.2247 s/iter total_throughput: 2994.93 samples/s lr: 5.36e-04 [09/22 20:35:04] lb.utils.events INFO: eta: 8:26:58 iteration: 180299/375342 consumed_samples: 184627200 total_loss: 3.542 time: 0.3419 s/iter data_time: 0.2132 s/iter total_throughput: 2994.93 samples/s lr: 5.36e-04 [09/22 20:35:38] lb.utils.events INFO: eta: 8:26:55 iteration: 180399/375342 consumed_samples: 184729600 total_loss: 3.545 time: 0.3419 s/iter data_time: 0.2101 s/iter total_throughput: 2994.93 samples/s lr: 5.35e-04 [09/22 20:36:13] lb.utils.events INFO: eta: 8:26:06 iteration: 180499/375342 consumed_samples: 184832000 total_loss: 3.554 time: 0.3419 s/iter data_time: 0.2189 s/iter total_throughput: 2994.92 samples/s lr: 5.35e-04 [09/22 20:36:47] lb.utils.events INFO: eta: 8:26:55 iteration: 180599/375342 consumed_samples: 184934400 total_loss: 3.556 time: 0.3419 s/iter data_time: 0.2221 s/iter total_throughput: 2994.92 samples/s lr: 5.34e-04 [09/22 20:37:21] lb.utils.events INFO: eta: 8:27:25 iteration: 180699/375342 consumed_samples: 185036800 total_loss: 3.551 time: 0.3419 s/iter data_time: 0.2066 s/iter total_throughput: 2994.92 samples/s lr: 5.34e-04 [09/22 20:37:55] lb.utils.events INFO: eta: 8:25:57 iteration: 180799/375342 consumed_samples: 185139200 total_loss: 3.541 time: 0.3419 s/iter data_time: 0.2263 s/iter total_throughput: 2994.92 samples/s lr: 5.33e-04 [09/22 20:38:30] lb.utils.events INFO: eta: 8:24:57 iteration: 180899/375342 consumed_samples: 185241600 total_loss: 3.565 time: 0.3419 s/iter data_time: 0.2178 s/iter total_throughput: 2994.89 samples/s lr: 5.33e-04 [09/22 20:39:04] lb.utils.events INFO: eta: 8:22:45 iteration: 180999/375342 consumed_samples: 185344000 total_loss: 3.586 time: 0.3419 s/iter data_time: 0.2148 s/iter total_throughput: 2994.89 samples/s lr: 5.33e-04 [09/22 20:39:39] lb.utils.events INFO: eta: 8:21:30 iteration: 181099/375342 consumed_samples: 185446400 total_loss: 3.568 time: 0.3419 s/iter data_time: 0.2223 s/iter total_throughput: 2994.88 samples/s lr: 5.32e-04 [09/22 20:40:13] lb.utils.events INFO: eta: 8:21:20 iteration: 181199/375342 consumed_samples: 185548800 total_loss: 3.568 time: 0.3419 s/iter data_time: 0.2186 s/iter total_throughput: 2994.87 samples/s lr: 5.32e-04 [09/22 20:40:48] lb.utils.events INFO: eta: 8:20:41 iteration: 181299/375342 consumed_samples: 185651200 total_loss: 3.587 time: 0.3419 s/iter data_time: 0.2117 s/iter total_throughput: 2994.85 samples/s lr: 5.31e-04 [09/22 20:41:22] lb.utils.events INFO: eta: 8:20:11 iteration: 181399/375342 consumed_samples: 185753600 total_loss: 3.572 time: 0.3419 s/iter data_time: 0.2158 s/iter total_throughput: 2994.82 samples/s lr: 5.31e-04 [09/22 20:41:56] lb.utils.events INFO: eta: 8:20:38 iteration: 181499/375342 consumed_samples: 185856000 total_loss: 3.54 time: 0.3419 s/iter data_time: 0.2096 s/iter total_throughput: 2994.83 samples/s lr: 5.31e-04 [09/22 20:42:30] lb.utils.events INFO: eta: 8:20:09 iteration: 181599/375342 consumed_samples: 185958400 total_loss: 3.526 time: 0.3419 s/iter data_time: 0.2121 s/iter total_throughput: 2994.84 samples/s lr: 5.30e-04 [09/22 20:43:05] lb.utils.events INFO: eta: 8:20:02 iteration: 181699/375342 consumed_samples: 186060800 total_loss: 3.54 time: 0.3419 s/iter data_time: 0.2054 s/iter total_throughput: 2994.82 samples/s lr: 5.30e-04 [09/22 20:43:39] lb.utils.events INFO: eta: 8:19:58 iteration: 181799/375342 consumed_samples: 186163200 total_loss: 3.552 time: 0.3419 s/iter data_time: 0.2119 s/iter total_throughput: 2994.83 samples/s lr: 5.29e-04 [09/22 20:44:13] lb.utils.events INFO: eta: 8:20:36 iteration: 181899/375342 consumed_samples: 186265600 total_loss: 3.573 time: 0.3419 s/iter data_time: 0.2126 s/iter total_throughput: 2994.83 samples/s lr: 5.29e-04 [09/22 20:44:47] lb.utils.events INFO: eta: 8:20:50 iteration: 181999/375342 consumed_samples: 186368000 total_loss: 3.549 time: 0.3419 s/iter data_time: 0.2209 s/iter total_throughput: 2994.84 samples/s lr: 5.28e-04 [09/22 20:45:21] lb.utils.events INFO: eta: 8:21:13 iteration: 182099/375342 consumed_samples: 186470400 total_loss: 3.548 time: 0.3419 s/iter data_time: 0.2178 s/iter total_throughput: 2994.86 samples/s lr: 5.28e-04 [09/22 20:45:56] lb.utils.events INFO: eta: 8:21:33 iteration: 182199/375342 consumed_samples: 186572800 total_loss: 3.566 time: 0.3419 s/iter data_time: 0.2140 s/iter total_throughput: 2994.80 samples/s lr: 5.28e-04 [09/22 20:46:32] lb.utils.events INFO: eta: 8:20:17 iteration: 182299/375342 consumed_samples: 186675200 total_loss: 3.562 time: 0.3419 s/iter data_time: 0.2266 s/iter total_throughput: 2994.74 samples/s lr: 5.27e-04 [09/22 20:47:07] lb.utils.events INFO: eta: 8:19:35 iteration: 182399/375342 consumed_samples: 186777600 total_loss: 3.57 time: 0.3419 s/iter data_time: 0.2289 s/iter total_throughput: 2994.70 samples/s lr: 5.27e-04 [09/22 20:47:42] lb.utils.events INFO: eta: 8:17:49 iteration: 182499/375342 consumed_samples: 186880000 total_loss: 3.581 time: 0.3419 s/iter data_time: 0.2268 s/iter total_throughput: 2994.65 samples/s lr: 5.26e-04 [09/22 20:48:17] lb.utils.events INFO: eta: 8:17:17 iteration: 182599/375342 consumed_samples: 186982400 total_loss: 3.572 time: 0.3419 s/iter data_time: 0.2248 s/iter total_throughput: 2994.60 samples/s lr: 5.26e-04 [09/22 20:48:53] lb.utils.events INFO: eta: 8:17:05 iteration: 182699/375342 consumed_samples: 187084800 total_loss: 3.574 time: 0.3420 s/iter data_time: 0.2394 s/iter total_throughput: 2994.54 samples/s lr: 5.26e-04 [09/22 20:49:28] lb.utils.events INFO: eta: 8:15:54 iteration: 182799/375342 consumed_samples: 187187200 total_loss: 3.572 time: 0.3420 s/iter data_time: 0.2256 s/iter total_throughput: 2994.48 samples/s lr: 5.25e-04 [09/22 20:50:03] lb.utils.events INFO: eta: 8:15:29 iteration: 182899/375342 consumed_samples: 187289600 total_loss: 3.572 time: 0.3420 s/iter data_time: 0.2215 s/iter total_throughput: 2994.44 samples/s lr: 5.25e-04 [09/22 20:50:38] lb.utils.events INFO: eta: 8:15:24 iteration: 182999/375342 consumed_samples: 187392000 total_loss: 3.569 time: 0.3420 s/iter data_time: 0.2065 s/iter total_throughput: 2994.40 samples/s lr: 5.24e-04 [09/22 20:51:13] lb.utils.events INFO: eta: 8:15:13 iteration: 183099/375342 consumed_samples: 187494400 total_loss: 3.537 time: 0.3420 s/iter data_time: 0.2444 s/iter total_throughput: 2994.36 samples/s lr: 5.24e-04 [09/22 20:51:49] lb.utils.events INFO: eta: 8:15:02 iteration: 183199/375342 consumed_samples: 187596800 total_loss: 3.522 time: 0.3420 s/iter data_time: 0.2233 s/iter total_throughput: 2994.30 samples/s lr: 5.24e-04 [09/22 20:52:24] lb.utils.events INFO: eta: 8:15:10 iteration: 183299/375342 consumed_samples: 187699200 total_loss: 3.539 time: 0.3420 s/iter data_time: 0.2237 s/iter total_throughput: 2994.27 samples/s lr: 5.23e-04 [09/22 20:52:59] lb.utils.events INFO: eta: 8:15:39 iteration: 183399/375342 consumed_samples: 187801600 total_loss: 3.551 time: 0.3420 s/iter data_time: 0.2166 s/iter total_throughput: 2994.22 samples/s lr: 5.23e-04 [09/22 20:53:35] lb.utils.events INFO: eta: 8:15:33 iteration: 183499/375342 consumed_samples: 187904000 total_loss: 3.568 time: 0.3420 s/iter data_time: 0.2295 s/iter total_throughput: 2994.15 samples/s lr: 5.22e-04 [09/22 20:54:09] lb.utils.events INFO: eta: 8:15:35 iteration: 183599/375342 consumed_samples: 188006400 total_loss: 3.562 time: 0.3420 s/iter data_time: 0.2225 s/iter total_throughput: 2994.12 samples/s lr: 5.22e-04 [09/22 20:54:45] lb.utils.events INFO: eta: 8:14:24 iteration: 183699/375342 consumed_samples: 188108800 total_loss: 3.546 time: 0.3420 s/iter data_time: 0.2307 s/iter total_throughput: 2994.08 samples/s lr: 5.21e-04 [09/22 20:55:19] lb.utils.events INFO: eta: 8:14:02 iteration: 183799/375342 consumed_samples: 188211200 total_loss: 3.544 time: 0.3420 s/iter data_time: 0.2299 s/iter total_throughput: 2994.05 samples/s lr: 5.21e-04 [09/22 20:55:55] lb.utils.events INFO: eta: 8:13:01 iteration: 183899/375342 consumed_samples: 188313600 total_loss: 3.537 time: 0.3420 s/iter data_time: 0.2342 s/iter total_throughput: 2994.00 samples/s lr: 5.21e-04 [09/22 20:56:29] lb.utils.events INFO: eta: 8:13:08 iteration: 183999/375342 consumed_samples: 188416000 total_loss: 3.535 time: 0.3420 s/iter data_time: 0.2111 s/iter total_throughput: 2993.98 samples/s lr: 5.20e-04 [09/22 20:57:04] lb.utils.events INFO: eta: 8:12:51 iteration: 184099/375342 consumed_samples: 188518400 total_loss: 3.544 time: 0.3420 s/iter data_time: 0.2238 s/iter total_throughput: 2993.93 samples/s lr: 5.20e-04 [09/22 20:57:40] lb.utils.events INFO: eta: 8:12:03 iteration: 184199/375342 consumed_samples: 188620800 total_loss: 3.543 time: 0.3420 s/iter data_time: 0.2289 s/iter total_throughput: 2993.87 samples/s lr: 5.19e-04 [09/22 20:58:14] lb.utils.events INFO: eta: 8:13:44 iteration: 184299/375342 consumed_samples: 188723200 total_loss: 3.547 time: 0.3420 s/iter data_time: 0.2118 s/iter total_throughput: 2993.86 samples/s lr: 5.19e-04 [09/22 20:58:49] lb.utils.events INFO: eta: 8:13:26 iteration: 184399/375342 consumed_samples: 188825600 total_loss: 3.561 time: 0.3420 s/iter data_time: 0.2192 s/iter total_throughput: 2993.83 samples/s lr: 5.19e-04 [09/22 20:59:24] lb.utils.events INFO: eta: 8:13:27 iteration: 184499/375342 consumed_samples: 188928000 total_loss: 3.553 time: 0.3420 s/iter data_time: 0.2197 s/iter total_throughput: 2993.78 samples/s lr: 5.18e-04 [09/22 21:00:00] lb.utils.events INFO: eta: 8:12:55 iteration: 184599/375342 consumed_samples: 189030400 total_loss: 3.538 time: 0.3420 s/iter data_time: 0.2339 s/iter total_throughput: 2993.72 samples/s lr: 5.18e-04 [09/22 21:00:35] lb.utils.events INFO: eta: 8:13:14 iteration: 184699/375342 consumed_samples: 189132800 total_loss: 3.529 time: 0.3421 s/iter data_time: 0.2249 s/iter total_throughput: 2993.68 samples/s lr: 5.17e-04 [09/22 21:01:10] lb.utils.events INFO: eta: 8:13:33 iteration: 184799/375342 consumed_samples: 189235200 total_loss: 3.542 time: 0.3421 s/iter data_time: 0.2124 s/iter total_throughput: 2993.66 samples/s lr: 5.17e-04 [09/22 21:01:45] lb.utils.events INFO: eta: 8:13:24 iteration: 184899/375342 consumed_samples: 189337600 total_loss: 3.555 time: 0.3421 s/iter data_time: 0.2279 s/iter total_throughput: 2993.60 samples/s lr: 5.16e-04 [09/22 21:02:21] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0184999 [09/22 21:02:21] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/22 21:02:21] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/22 21:02:25] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0913 s/iter. Inference: 0.1656 s/iter. Eval: 0.0022 s/iter. Total: 0.2590 s/iter. ETA=0:00:09 [09/22 21:02:31] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1323 s/iter. Inference: 0.1685 s/iter. Eval: 0.0021 s/iter. Total: 0.3029 s/iter. ETA=0:00:05 [09/22 21:02:36] lb.evaluation.evaluator INFO: Inference done 49152/50000. Dataloading: 0.1208 s/iter. Inference: 0.1647 s/iter. Eval: 0.0020 s/iter. Total: 0.2876 s/iter. ETA=0:00:00 [09/22 21:02:36] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/22 21:02:36] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.664004 (0.000253 s / iter per device, on 8 devices) [09/22 21:02:36] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000146 s / iter per device, on 8 devices) [09/22 21:02:36] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/22 21:02:36] lb.evaluation.utils INFO: copypaste: Acc@1=75.21 [09/22 21:02:36] lb.evaluation.utils INFO: copypaste: Acc@5=92.55799999999999 [09/22 21:02:36] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 75.21000, better than last best score 74.57000 @ iteration 179999. [09/22 21:02:36] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/22 21:02:37] lb.utils.events INFO: eta: 8:12:30 iteration: 184999/375342 consumed_samples: 189440000 total_loss: 3.554 time: 0.3421 s/iter data_time: 0.2268 s/iter total_throughput: 2993.55 samples/s lr: 5.16e-04 [09/22 21:03:10] lb.utils.events INFO: eta: 8:12:55 iteration: 185099/375342 consumed_samples: 189542400 total_loss: 3.548 time: 0.3421 s/iter data_time: 0.2398 s/iter total_throughput: 2993.60 samples/s lr: 5.16e-04 [09/22 21:03:46] lb.utils.events INFO: eta: 8:13:45 iteration: 185199/375342 consumed_samples: 189644800 total_loss: 3.562 time: 0.3421 s/iter data_time: 0.2294 s/iter total_throughput: 2993.55 samples/s lr: 5.15e-04 [09/22 21:04:21] lb.utils.events INFO: eta: 8:12:15 iteration: 185299/375342 consumed_samples: 189747200 total_loss: 3.54 time: 0.3421 s/iter data_time: 0.2361 s/iter total_throughput: 2993.51 samples/s lr: 5.15e-04 [09/22 21:04:56] lb.utils.events INFO: eta: 8:12:45 iteration: 185399/375342 consumed_samples: 189849600 total_loss: 3.521 time: 0.3421 s/iter data_time: 0.2099 s/iter total_throughput: 2993.47 samples/s lr: 5.14e-04 [09/22 21:05:31] lb.utils.events INFO: eta: 8:11:28 iteration: 185499/375342 consumed_samples: 189952000 total_loss: 3.525 time: 0.3421 s/iter data_time: 0.2285 s/iter total_throughput: 2993.42 samples/s lr: 5.14e-04 [09/22 21:06:06] lb.utils.events INFO: eta: 8:10:36 iteration: 185599/375342 consumed_samples: 190054400 total_loss: 3.542 time: 0.3421 s/iter data_time: 0.2287 s/iter total_throughput: 2993.37 samples/s lr: 5.14e-04 [09/22 21:06:41] lb.utils.events INFO: eta: 8:09:28 iteration: 185699/375342 consumed_samples: 190156800 total_loss: 3.558 time: 0.3421 s/iter data_time: 0.2303 s/iter total_throughput: 2993.33 samples/s lr: 5.13e-04 [09/22 21:07:17] lb.utils.events INFO: eta: 8:09:36 iteration: 185799/375342 consumed_samples: 190259200 total_loss: 3.558 time: 0.3421 s/iter data_time: 0.2212 s/iter total_throughput: 2993.29 samples/s lr: 5.13e-04 [09/22 21:07:51] lb.utils.events INFO: eta: 8:10:13 iteration: 185899/375342 consumed_samples: 190361600 total_loss: 3.542 time: 0.3421 s/iter data_time: 0.2115 s/iter total_throughput: 2993.25 samples/s lr: 5.12e-04 [09/22 21:08:26] lb.utils.events INFO: eta: 8:10:31 iteration: 185999/375342 consumed_samples: 190464000 total_loss: 3.536 time: 0.3421 s/iter data_time: 0.2166 s/iter total_throughput: 2993.22 samples/s lr: 5.12e-04 [09/22 21:09:01] lb.utils.events INFO: eta: 8:08:27 iteration: 186099/375342 consumed_samples: 190566400 total_loss: 3.543 time: 0.3421 s/iter data_time: 0.2336 s/iter total_throughput: 2993.19 samples/s lr: 5.12e-04 [09/22 21:09:36] lb.utils.events INFO: eta: 8:08:07 iteration: 186199/375342 consumed_samples: 190668800 total_loss: 3.54 time: 0.3421 s/iter data_time: 0.2089 s/iter total_throughput: 2993.17 samples/s lr: 5.11e-04 [09/22 21:10:11] lb.utils.events INFO: eta: 8:07:45 iteration: 186299/375342 consumed_samples: 190771200 total_loss: 3.536 time: 0.3421 s/iter data_time: 0.2233 s/iter total_throughput: 2993.12 samples/s lr: 5.11e-04 [09/22 21:10:46] lb.utils.events INFO: eta: 8:07:12 iteration: 186399/375342 consumed_samples: 190873600 total_loss: 3.55 time: 0.3421 s/iter data_time: 0.2375 s/iter total_throughput: 2993.09 samples/s lr: 5.10e-04 [09/22 21:11:21] lb.utils.events INFO: eta: 8:08:32 iteration: 186499/375342 consumed_samples: 190976000 total_loss: 3.544 time: 0.3421 s/iter data_time: 0.2301 s/iter total_throughput: 2993.05 samples/s lr: 5.10e-04 [09/22 21:11:56] lb.utils.events INFO: eta: 8:08:54 iteration: 186599/375342 consumed_samples: 191078400 total_loss: 3.543 time: 0.3421 s/iter data_time: 0.2330 s/iter total_throughput: 2993.00 samples/s lr: 5.09e-04 [09/22 21:12:31] lb.utils.events INFO: eta: 8:08:44 iteration: 186699/375342 consumed_samples: 191180800 total_loss: 3.546 time: 0.3421 s/iter data_time: 0.2473 s/iter total_throughput: 2992.96 samples/s lr: 5.09e-04 [09/22 21:13:07] lb.utils.events INFO: eta: 8:07:41 iteration: 186799/375342 consumed_samples: 191283200 total_loss: 3.543 time: 0.3421 s/iter data_time: 0.2175 s/iter total_throughput: 2992.91 samples/s lr: 5.09e-04 [09/22 21:13:42] lb.utils.events INFO: eta: 8:06:24 iteration: 186899/375342 consumed_samples: 191385600 total_loss: 3.536 time: 0.3421 s/iter data_time: 0.2307 s/iter total_throughput: 2992.85 samples/s lr: 5.08e-04 [09/22 21:14:17] lb.utils.events INFO: eta: 8:05:38 iteration: 186999/375342 consumed_samples: 191488000 total_loss: 3.525 time: 0.3422 s/iter data_time: 0.2271 s/iter total_throughput: 2992.81 samples/s lr: 5.08e-04 [09/22 21:14:53] lb.utils.events INFO: eta: 8:05:26 iteration: 187099/375342 consumed_samples: 191590400 total_loss: 3.529 time: 0.3422 s/iter data_time: 0.2468 s/iter total_throughput: 2992.76 samples/s lr: 5.07e-04 [09/22 21:15:27] lb.utils.events INFO: eta: 8:04:44 iteration: 187199/375342 consumed_samples: 191692800 total_loss: 3.547 time: 0.3422 s/iter data_time: 0.2181 s/iter total_throughput: 2992.73 samples/s lr: 5.07e-04 [09/22 21:16:03] lb.utils.events INFO: eta: 8:06:07 iteration: 187299/375342 consumed_samples: 191795200 total_loss: 3.554 time: 0.3422 s/iter data_time: 0.2478 s/iter total_throughput: 2992.69 samples/s lr: 5.07e-04 [09/22 21:16:38] lb.utils.events INFO: eta: 8:04:08 iteration: 187399/375342 consumed_samples: 191897600 total_loss: 3.551 time: 0.3422 s/iter data_time: 0.2267 s/iter total_throughput: 2992.63 samples/s lr: 5.06e-04 [09/22 21:17:13] lb.utils.events INFO: eta: 8:03:14 iteration: 187499/375342 consumed_samples: 192000000 total_loss: 3.544 time: 0.3422 s/iter data_time: 0.2255 s/iter total_throughput: 2992.59 samples/s lr: 5.06e-04 [09/22 21:17:49] lb.utils.events INFO: eta: 8:01:32 iteration: 187599/375342 consumed_samples: 192102400 total_loss: 3.54 time: 0.3422 s/iter data_time: 0.2204 s/iter total_throughput: 2992.53 samples/s lr: 5.05e-04 [09/22 21:18:23] lb.utils.events INFO: eta: 8:01:34 iteration: 187699/375342 consumed_samples: 192204800 total_loss: 3.552 time: 0.3422 s/iter data_time: 0.2141 s/iter total_throughput: 2992.51 samples/s lr: 5.05e-04 [09/22 21:18:58] lb.utils.events INFO: eta: 8:02:49 iteration: 187799/375342 consumed_samples: 192307200 total_loss: 3.546 time: 0.3422 s/iter data_time: 0.2174 s/iter total_throughput: 2992.48 samples/s lr: 5.04e-04 [09/22 21:19:33] lb.utils.events INFO: eta: 8:03:41 iteration: 187899/375342 consumed_samples: 192409600 total_loss: 3.515 time: 0.3422 s/iter data_time: 0.2255 s/iter total_throughput: 2992.46 samples/s lr: 5.04e-04 [09/22 21:20:08] lb.utils.events INFO: eta: 8:02:34 iteration: 187999/375342 consumed_samples: 192512000 total_loss: 3.519 time: 0.3422 s/iter data_time: 0.2402 s/iter total_throughput: 2992.40 samples/s lr: 5.04e-04 [09/22 21:20:43] lb.utils.events INFO: eta: 8:01:52 iteration: 188099/375342 consumed_samples: 192614400 total_loss: 3.527 time: 0.3422 s/iter data_time: 0.2204 s/iter total_throughput: 2992.36 samples/s lr: 5.03e-04 [09/22 21:21:18] lb.utils.events INFO: eta: 8:04:16 iteration: 188199/375342 consumed_samples: 192716800 total_loss: 3.538 time: 0.3422 s/iter data_time: 0.2136 s/iter total_throughput: 2992.35 samples/s lr: 5.03e-04 [09/22 21:21:53] lb.utils.events INFO: eta: 8:02:14 iteration: 188299/375342 consumed_samples: 192819200 total_loss: 3.551 time: 0.3422 s/iter data_time: 0.2221 s/iter total_throughput: 2992.32 samples/s lr: 5.02e-04 [09/22 21:22:27] lb.utils.events INFO: eta: 8:04:25 iteration: 188399/375342 consumed_samples: 192921600 total_loss: 3.541 time: 0.3422 s/iter data_time: 0.2207 s/iter total_throughput: 2992.30 samples/s lr: 5.02e-04 [09/22 21:23:03] lb.utils.events INFO: eta: 8:03:49 iteration: 188499/375342 consumed_samples: 193024000 total_loss: 3.541 time: 0.3422 s/iter data_time: 0.2213 s/iter total_throughput: 2992.25 samples/s lr: 5.02e-04 [09/22 21:23:38] lb.utils.events INFO: eta: 8:03:55 iteration: 188599/375342 consumed_samples: 193126400 total_loss: 3.544 time: 0.3422 s/iter data_time: 0.2238 s/iter total_throughput: 2992.22 samples/s lr: 5.01e-04 [09/22 21:24:12] lb.utils.events INFO: eta: 8:03:41 iteration: 188699/375342 consumed_samples: 193228800 total_loss: 3.541 time: 0.3422 s/iter data_time: 0.2308 s/iter total_throughput: 2992.19 samples/s lr: 5.01e-04 [09/22 21:24:47] lb.utils.events INFO: eta: 8:02:35 iteration: 188799/375342 consumed_samples: 193331200 total_loss: 3.548 time: 0.3422 s/iter data_time: 0.2241 s/iter total_throughput: 2992.17 samples/s lr: 5.00e-04 [09/22 21:25:22] lb.utils.events INFO: eta: 8:01:08 iteration: 188899/375342 consumed_samples: 193433600 total_loss: 3.556 time: 0.3422 s/iter data_time: 0.2186 s/iter total_throughput: 2992.15 samples/s lr: 5.00e-04 [09/22 21:25:57] lb.utils.events INFO: eta: 8:02:22 iteration: 188999/375342 consumed_samples: 193536000 total_loss: 3.529 time: 0.3422 s/iter data_time: 0.2228 s/iter total_throughput: 2992.11 samples/s lr: 4.99e-04 [09/22 21:26:32] lb.utils.events INFO: eta: 8:02:24 iteration: 189099/375342 consumed_samples: 193638400 total_loss: 3.51 time: 0.3422 s/iter data_time: 0.2176 s/iter total_throughput: 2992.09 samples/s lr: 4.99e-04 [09/22 21:27:07] lb.utils.events INFO: eta: 8:02:04 iteration: 189199/375342 consumed_samples: 193740800 total_loss: 3.541 time: 0.3422 s/iter data_time: 0.2232 s/iter total_throughput: 2992.04 samples/s lr: 4.99e-04 [09/22 21:27:42] lb.utils.events INFO: eta: 8:01:53 iteration: 189299/375342 consumed_samples: 193843200 total_loss: 3.535 time: 0.3422 s/iter data_time: 0.2292 s/iter total_throughput: 2992.01 samples/s lr: 4.98e-04 [09/22 21:28:17] lb.utils.events INFO: eta: 8:01:08 iteration: 189399/375342 consumed_samples: 193945600 total_loss: 3.546 time: 0.3422 s/iter data_time: 0.2214 s/iter total_throughput: 2991.97 samples/s lr: 4.98e-04 [09/22 21:28:52] lb.utils.events INFO: eta: 8:01:48 iteration: 189499/375342 consumed_samples: 194048000 total_loss: 3.517 time: 0.3423 s/iter data_time: 0.2254 s/iter total_throughput: 2991.92 samples/s lr: 4.97e-04 [09/22 21:29:27] lb.utils.events INFO: eta: 8:03:13 iteration: 189599/375342 consumed_samples: 194150400 total_loss: 3.501 time: 0.3423 s/iter data_time: 0.2254 s/iter total_throughput: 2991.90 samples/s lr: 4.97e-04 [09/22 21:30:01] lb.utils.events INFO: eta: 8:01:46 iteration: 189699/375342 consumed_samples: 194252800 total_loss: 3.524 time: 0.3423 s/iter data_time: 0.2065 s/iter total_throughput: 2991.89 samples/s lr: 4.97e-04 [09/22 21:30:36] lb.utils.events INFO: eta: 8:02:15 iteration: 189799/375342 consumed_samples: 194355200 total_loss: 3.519 time: 0.3423 s/iter data_time: 0.2219 s/iter total_throughput: 2991.87 samples/s lr: 4.96e-04 [09/22 21:31:11] lb.utils.events INFO: eta: 8:02:13 iteration: 189899/375342 consumed_samples: 194457600 total_loss: 3.531 time: 0.3423 s/iter data_time: 0.2232 s/iter total_throughput: 2991.85 samples/s lr: 4.96e-04 [09/22 21:31:46] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0189999 [09/22 21:31:46] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/22 21:31:46] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/22 21:31:50] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0983 s/iter. Inference: 0.1623 s/iter. Eval: 0.0020 s/iter. Total: 0.2626 s/iter. ETA=0:00:09 [09/22 21:31:55] lb.evaluation.evaluator INFO: Inference done 28672/50000. Dataloading: 0.0943 s/iter. Inference: 0.1927 s/iter. Eval: 0.0020 s/iter. Total: 0.2891 s/iter. ETA=0:00:05 [09/22 21:32:01] lb.evaluation.evaluator INFO: Inference done 46080/50000. Dataloading: 0.0909 s/iter. Inference: 0.2002 s/iter. Eval: 0.0020 s/iter. Total: 0.2932 s/iter. ETA=0:00:00 [09/22 21:32:01] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/22 21:32:01] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.663567 (0.000253 s / iter per device, on 8 devices) [09/22 21:32:01] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:08 (0.000177 s / iter per device, on 8 devices) [09/22 21:32:01] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/22 21:32:01] lb.evaluation.utils INFO: copypaste: Acc@1=75.166 [09/22 21:32:01] lb.evaluation.utils INFO: copypaste: Acc@5=92.586 [09/22 21:32:01] lb.engine.hooks INFO: Not saving as latest eval score for Acc@1 is 75.16600, not better than best score 75.21000 @ iteration 184999. [09/22 21:32:01] lb.utils.events INFO: eta: 8:00:59 iteration: 189999/375342 consumed_samples: 194560000 total_loss: 3.531 time: 0.3423 s/iter data_time: 0.2148 s/iter total_throughput: 2991.82 samples/s lr: 4.95e-04 [09/22 21:32:34] lb.utils.events INFO: eta: 8:03:12 iteration: 190099/375342 consumed_samples: 194662400 total_loss: 3.537 time: 0.3423 s/iter data_time: 0.2180 s/iter total_throughput: 2991.89 samples/s lr: 4.95e-04 [09/22 21:33:09] lb.utils.events INFO: eta: 8:02:10 iteration: 190199/375342 consumed_samples: 194764800 total_loss: 3.526 time: 0.3423 s/iter data_time: 0.2287 s/iter total_throughput: 2991.85 samples/s lr: 4.95e-04 [09/22 21:33:44] lb.utils.events INFO: eta: 8:00:53 iteration: 190299/375342 consumed_samples: 194867200 total_loss: 3.522 time: 0.3423 s/iter data_time: 0.2197 s/iter total_throughput: 2991.84 samples/s lr: 4.94e-04 [09/22 21:34:18] lb.utils.events INFO: eta: 8:00:54 iteration: 190399/375342 consumed_samples: 194969600 total_loss: 3.527 time: 0.3423 s/iter data_time: 0.2224 s/iter total_throughput: 2991.84 samples/s lr: 4.94e-04 [09/22 21:34:53] lb.utils.events INFO: eta: 8:00:55 iteration: 190499/375342 consumed_samples: 195072000 total_loss: 3.52 time: 0.3423 s/iter data_time: 0.2249 s/iter total_throughput: 2991.83 samples/s lr: 4.93e-04 [09/22 21:35:27] lb.utils.events INFO: eta: 8:00:40 iteration: 190599/375342 consumed_samples: 195174400 total_loss: 3.518 time: 0.3423 s/iter data_time: 0.2171 s/iter total_throughput: 2991.80 samples/s lr: 4.93e-04 [09/22 21:36:02] lb.utils.events INFO: eta: 8:00:57 iteration: 190699/375342 consumed_samples: 195276800 total_loss: 3.525 time: 0.3423 s/iter data_time: 0.2101 s/iter total_throughput: 2991.79 samples/s lr: 4.92e-04 [09/22 21:36:36] lb.utils.events INFO: eta: 7:59:55 iteration: 190799/375342 consumed_samples: 195379200 total_loss: 3.549 time: 0.3423 s/iter data_time: 0.2260 s/iter total_throughput: 2991.77 samples/s lr: 4.92e-04 [09/22 21:37:11] lb.utils.events INFO: eta: 7:59:27 iteration: 190899/375342 consumed_samples: 195481600 total_loss: 3.519 time: 0.3423 s/iter data_time: 0.2343 s/iter total_throughput: 2991.74 samples/s lr: 4.92e-04 [09/22 21:37:46] lb.utils.events INFO: eta: 7:59:15 iteration: 190999/375342 consumed_samples: 195584000 total_loss: 3.526 time: 0.3423 s/iter data_time: 0.2129 s/iter total_throughput: 2991.73 samples/s lr: 4.91e-04 [09/22 21:38:20] lb.utils.events INFO: eta: 7:58:20 iteration: 191099/375342 consumed_samples: 195686400 total_loss: 3.547 time: 0.3423 s/iter data_time: 0.2159 s/iter total_throughput: 2991.74 samples/s lr: 4.91e-04 [09/22 21:38:55] lb.utils.events INFO: eta: 7:58:09 iteration: 191199/375342 consumed_samples: 195788800 total_loss: 3.534 time: 0.3423 s/iter data_time: 0.2369 s/iter total_throughput: 2991.70 samples/s lr: 4.90e-04 [09/22 21:39:29] lb.utils.events INFO: eta: 7:57:49 iteration: 191299/375342 consumed_samples: 195891200 total_loss: 3.542 time: 0.3423 s/iter data_time: 0.2063 s/iter total_throughput: 2991.70 samples/s lr: 4.90e-04 [09/22 21:40:04] lb.utils.events INFO: eta: 7:55:50 iteration: 191399/375342 consumed_samples: 195993600 total_loss: 3.55 time: 0.3423 s/iter data_time: 0.2146 s/iter total_throughput: 2991.66 samples/s lr: 4.90e-04 [09/22 21:40:39] lb.utils.events INFO: eta: 7:55:27 iteration: 191499/375342 consumed_samples: 196096000 total_loss: 3.523 time: 0.3423 s/iter data_time: 0.2070 s/iter total_throughput: 2991.63 samples/s lr: 4.89e-04 [09/22 21:41:14] lb.utils.events INFO: eta: 7:53:58 iteration: 191599/375342 consumed_samples: 196198400 total_loss: 3.522 time: 0.3423 s/iter data_time: 0.2239 s/iter total_throughput: 2991.59 samples/s lr: 4.89e-04 [09/22 21:41:49] lb.utils.events INFO: eta: 7:53:39 iteration: 191699/375342 consumed_samples: 196300800 total_loss: 3.519 time: 0.3423 s/iter data_time: 0.2273 s/iter total_throughput: 2991.58 samples/s lr: 4.88e-04 [09/22 21:42:23] lb.utils.events INFO: eta: 7:55:27 iteration: 191799/375342 consumed_samples: 196403200 total_loss: 3.51 time: 0.3423 s/iter data_time: 0.2106 s/iter total_throughput: 2991.58 samples/s lr: 4.88e-04 [09/22 21:42:58] lb.utils.events INFO: eta: 7:56:24 iteration: 191899/375342 consumed_samples: 196505600 total_loss: 3.52 time: 0.3423 s/iter data_time: 0.2226 s/iter total_throughput: 2991.57 samples/s lr: 4.87e-04 [09/22 21:43:33] lb.utils.events INFO: eta: 7:55:48 iteration: 191999/375342 consumed_samples: 196608000 total_loss: 3.524 time: 0.3423 s/iter data_time: 0.2264 s/iter total_throughput: 2991.53 samples/s lr: 4.87e-04 [09/22 21:44:07] lb.utils.events INFO: eta: 7:54:34 iteration: 192099/375342 consumed_samples: 196710400 total_loss: 3.529 time: 0.3423 s/iter data_time: 0.2149 s/iter total_throughput: 2991.51 samples/s lr: 4.87e-04 [09/22 21:44:42] lb.utils.events INFO: eta: 7:54:08 iteration: 192199/375342 consumed_samples: 196812800 total_loss: 3.536 time: 0.3423 s/iter data_time: 0.2181 s/iter total_throughput: 2991.51 samples/s lr: 4.86e-04 [09/22 21:45:17] lb.utils.events INFO: eta: 7:53:59 iteration: 192299/375342 consumed_samples: 196915200 total_loss: 3.513 time: 0.3423 s/iter data_time: 0.2342 s/iter total_throughput: 2991.47 samples/s lr: 4.86e-04 [09/22 21:45:51] lb.utils.events INFO: eta: 7:54:50 iteration: 192399/375342 consumed_samples: 197017600 total_loss: 3.511 time: 0.3423 s/iter data_time: 0.2069 s/iter total_throughput: 2991.46 samples/s lr: 4.85e-04 [09/22 21:46:26] lb.utils.events INFO: eta: 7:54:22 iteration: 192499/375342 consumed_samples: 197120000 total_loss: 3.53 time: 0.3423 s/iter data_time: 0.2272 s/iter total_throughput: 2991.42 samples/s lr: 4.85e-04 [09/22 21:47:00] lb.utils.events INFO: eta: 7:55:25 iteration: 192599/375342 consumed_samples: 197222400 total_loss: 3.531 time: 0.3423 s/iter data_time: 0.2180 s/iter total_throughput: 2991.43 samples/s lr: 4.85e-04 [09/22 21:47:35] lb.utils.events INFO: eta: 7:54:15 iteration: 192699/375342 consumed_samples: 197324800 total_loss: 3.516 time: 0.3423 s/iter data_time: 0.2216 s/iter total_throughput: 2991.42 samples/s lr: 4.84e-04 [09/22 21:48:09] lb.utils.events INFO: eta: 7:53:38 iteration: 192799/375342 consumed_samples: 197427200 total_loss: 3.52 time: 0.3423 s/iter data_time: 0.2113 s/iter total_throughput: 2991.41 samples/s lr: 4.84e-04 [09/22 21:48:43] lb.utils.events INFO: eta: 7:53:26 iteration: 192899/375342 consumed_samples: 197529600 total_loss: 3.52 time: 0.3423 s/iter data_time: 0.2139 s/iter total_throughput: 2991.41 samples/s lr: 4.83e-04 [09/22 21:49:18] lb.utils.events INFO: eta: 7:53:34 iteration: 192999/375342 consumed_samples: 197632000 total_loss: 3.51 time: 0.3423 s/iter data_time: 0.2129 s/iter total_throughput: 2991.40 samples/s lr: 4.83e-04 [09/22 21:49:52] lb.utils.events INFO: eta: 7:53:34 iteration: 193099/375342 consumed_samples: 197734400 total_loss: 3.508 time: 0.3423 s/iter data_time: 0.2230 s/iter total_throughput: 2991.40 samples/s lr: 4.83e-04 [09/22 21:50:27] lb.utils.events INFO: eta: 7:53:32 iteration: 193199/375342 consumed_samples: 197836800 total_loss: 3.522 time: 0.3423 s/iter data_time: 0.2153 s/iter total_throughput: 2991.38 samples/s lr: 4.82e-04 [09/22 21:51:01] lb.utils.events INFO: eta: 7:52:37 iteration: 193299/375342 consumed_samples: 197939200 total_loss: 3.499 time: 0.3423 s/iter data_time: 0.2173 s/iter total_throughput: 2991.37 samples/s lr: 4.82e-04 [09/22 21:51:36] lb.utils.events INFO: eta: 7:52:08 iteration: 193399/375342 consumed_samples: 198041600 total_loss: 3.499 time: 0.3423 s/iter data_time: 0.2213 s/iter total_throughput: 2991.35 samples/s lr: 4.81e-04 [09/22 21:52:10] lb.utils.events INFO: eta: 7:52:12 iteration: 193499/375342 consumed_samples: 198144000 total_loss: 3.527 time: 0.3423 s/iter data_time: 0.2043 s/iter total_throughput: 2991.34 samples/s lr: 4.81e-04 [09/22 21:52:45] lb.utils.events INFO: eta: 7:51:30 iteration: 193599/375342 consumed_samples: 198246400 total_loss: 3.52 time: 0.3423 s/iter data_time: 0.2257 s/iter total_throughput: 2991.31 samples/s lr: 4.80e-04 [09/22 21:53:20] lb.utils.events INFO: eta: 7:51:33 iteration: 193699/375342 consumed_samples: 198348800 total_loss: 3.51 time: 0.3423 s/iter data_time: 0.2197 s/iter total_throughput: 2991.30 samples/s lr: 4.80e-04 [09/22 21:53:54] lb.utils.events INFO: eta: 7:50:35 iteration: 193799/375342 consumed_samples: 198451200 total_loss: 3.519 time: 0.3423 s/iter data_time: 0.2177 s/iter total_throughput: 2991.28 samples/s lr: 4.80e-04 [09/22 21:54:29] lb.utils.events INFO: eta: 7:48:57 iteration: 193899/375342 consumed_samples: 198553600 total_loss: 3.515 time: 0.3423 s/iter data_time: 0.2240 s/iter total_throughput: 2991.26 samples/s lr: 4.79e-04 [09/22 21:55:04] lb.utils.events INFO: eta: 7:48:33 iteration: 193999/375342 consumed_samples: 198656000 total_loss: 3.519 time: 0.3423 s/iter data_time: 0.2185 s/iter total_throughput: 2991.24 samples/s lr: 4.79e-04 [09/22 21:55:39] lb.utils.events INFO: eta: 7:47:14 iteration: 194099/375342 consumed_samples: 198758400 total_loss: 3.527 time: 0.3423 s/iter data_time: 0.2321 s/iter total_throughput: 2991.20 samples/s lr: 4.78e-04 [09/22 21:56:14] lb.utils.events INFO: eta: 7:46:28 iteration: 194199/375342 consumed_samples: 198860800 total_loss: 3.516 time: 0.3423 s/iter data_time: 0.2212 s/iter total_throughput: 2991.17 samples/s lr: 4.78e-04 [09/22 21:56:48] lb.utils.events INFO: eta: 7:47:17 iteration: 194299/375342 consumed_samples: 198963200 total_loss: 3.503 time: 0.3423 s/iter data_time: 0.2245 s/iter total_throughput: 2991.16 samples/s lr: 4.78e-04 [09/22 21:57:23] lb.utils.events INFO: eta: 7:45:24 iteration: 194399/375342 consumed_samples: 199065600 total_loss: 3.514 time: 0.3423 s/iter data_time: 0.2186 s/iter total_throughput: 2991.14 samples/s lr: 4.77e-04 [09/22 21:57:57] lb.utils.events INFO: eta: 7:46:53 iteration: 194499/375342 consumed_samples: 199168000 total_loss: 3.512 time: 0.3423 s/iter data_time: 0.2055 s/iter total_throughput: 2991.13 samples/s lr: 4.77e-04 [09/22 21:58:32] lb.utils.events INFO: eta: 7:47:28 iteration: 194599/375342 consumed_samples: 199270400 total_loss: 3.512 time: 0.3423 s/iter data_time: 0.2026 s/iter total_throughput: 2991.13 samples/s lr: 4.76e-04 [09/22 21:59:06] lb.utils.events INFO: eta: 7:47:16 iteration: 194699/375342 consumed_samples: 199372800 total_loss: 3.515 time: 0.3423 s/iter data_time: 0.2263 s/iter total_throughput: 2991.12 samples/s lr: 4.76e-04 [09/22 21:59:41] lb.utils.events INFO: eta: 7:47:35 iteration: 194799/375342 consumed_samples: 199475200 total_loss: 3.521 time: 0.3423 s/iter data_time: 0.2189 s/iter total_throughput: 2991.10 samples/s lr: 4.75e-04 [09/22 22:00:15] lb.utils.events INFO: eta: 7:49:12 iteration: 194899/375342 consumed_samples: 199577600 total_loss: 3.539 time: 0.3423 s/iter data_time: 0.2204 s/iter total_throughput: 2991.11 samples/s lr: 4.75e-04 [09/22 22:00:49] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0194999 [09/22 22:00:50] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/22 22:00:50] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/22 22:00:54] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0920 s/iter. Inference: 0.1639 s/iter. Eval: 0.0022 s/iter. Total: 0.2581 s/iter. ETA=0:00:09 [09/22 22:01:00] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1352 s/iter. Inference: 0.1648 s/iter. Eval: 0.0021 s/iter. Total: 0.3021 s/iter. ETA=0:00:05 [09/22 22:01:05] lb.evaluation.evaluator INFO: Inference done 49152/50000. Dataloading: 0.1201 s/iter. Inference: 0.1630 s/iter. Eval: 0.0021 s/iter. Total: 0.2853 s/iter. ETA=0:00:00 [09/22 22:01:05] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/22 22:01:05] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.727652 (0.000255 s / iter per device, on 8 devices) [09/22 22:01:05] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000144 s / iter per device, on 8 devices) [09/22 22:01:05] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/22 22:01:05] lb.evaluation.utils INFO: copypaste: Acc@1=75.36 [09/22 22:01:05] lb.evaluation.utils INFO: copypaste: Acc@5=92.80199999999999 [09/22 22:01:05] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 75.36000, better than last best score 75.21000 @ iteration 184999. [09/22 22:01:05] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/22 22:01:06] lb.utils.events INFO: eta: 7:48:51 iteration: 194999/375342 consumed_samples: 199680000 total_loss: 3.523 time: 0.3423 s/iter data_time: 0.2130 s/iter total_throughput: 2991.11 samples/s lr: 4.75e-04 [09/22 22:01:38] lb.utils.events INFO: eta: 7:49:51 iteration: 195099/375342 consumed_samples: 199782400 total_loss: 3.515 time: 0.3423 s/iter data_time: 0.2105 s/iter total_throughput: 2991.19 samples/s lr: 4.74e-04 [09/22 22:02:13] lb.utils.events INFO: eta: 7:49:46 iteration: 195199/375342 consumed_samples: 199884800 total_loss: 3.514 time: 0.3423 s/iter data_time: 0.2091 s/iter total_throughput: 2991.19 samples/s lr: 4.74e-04 [09/22 22:02:47] lb.utils.events INFO: eta: 7:49:43 iteration: 195299/375342 consumed_samples: 199987200 total_loss: 3.508 time: 0.3423 s/iter data_time: 0.2109 s/iter total_throughput: 2991.19 samples/s lr: 4.73e-04 [09/22 22:03:21] lb.utils.events INFO: eta: 7:50:05 iteration: 195399/375342 consumed_samples: 200089600 total_loss: 3.497 time: 0.3423 s/iter data_time: 0.2145 s/iter total_throughput: 2991.18 samples/s lr: 4.73e-04 [09/22 22:03:55] lb.utils.events INFO: eta: 7:49:04 iteration: 195499/375342 consumed_samples: 200192000 total_loss: 3.499 time: 0.3423 s/iter data_time: 0.2219 s/iter total_throughput: 2991.20 samples/s lr: 4.73e-04 [09/22 22:04:30] lb.utils.events INFO: eta: 7:48:26 iteration: 195599/375342 consumed_samples: 200294400 total_loss: 3.514 time: 0.3423 s/iter data_time: 0.2133 s/iter total_throughput: 2991.19 samples/s lr: 4.72e-04 [09/22 22:05:03] lb.utils.events INFO: eta: 7:48:13 iteration: 195699/375342 consumed_samples: 200396800 total_loss: 3.527 time: 0.3423 s/iter data_time: 0.2101 s/iter total_throughput: 2991.22 samples/s lr: 4.72e-04 [09/22 22:05:37] lb.utils.events INFO: eta: 7:48:20 iteration: 195799/375342 consumed_samples: 200499200 total_loss: 3.523 time: 0.3423 s/iter data_time: 0.2164 s/iter total_throughput: 2991.22 samples/s lr: 4.71e-04 [09/22 22:06:11] lb.utils.events INFO: eta: 7:47:13 iteration: 195899/375342 consumed_samples: 200601600 total_loss: 3.499 time: 0.3423 s/iter data_time: 0.2122 s/iter total_throughput: 2991.23 samples/s lr: 4.71e-04 [09/22 22:06:46] lb.utils.events INFO: eta: 7:47:48 iteration: 195999/375342 consumed_samples: 200704000 total_loss: 3.514 time: 0.3423 s/iter data_time: 0.2117 s/iter total_throughput: 2991.23 samples/s lr: 4.71e-04 [09/22 22:07:20] lb.utils.events INFO: eta: 7:46:16 iteration: 196099/375342 consumed_samples: 200806400 total_loss: 3.521 time: 0.3423 s/iter data_time: 0.2099 s/iter total_throughput: 2991.24 samples/s lr: 4.70e-04 [09/22 22:07:54] lb.utils.events INFO: eta: 7:44:46 iteration: 196199/375342 consumed_samples: 200908800 total_loss: 3.506 time: 0.3423 s/iter data_time: 0.2124 s/iter total_throughput: 2991.24 samples/s lr: 4.70e-04 [09/22 22:08:29] lb.utils.events INFO: eta: 7:44:16 iteration: 196299/375342 consumed_samples: 201011200 total_loss: 3.503 time: 0.3423 s/iter data_time: 0.2152 s/iter total_throughput: 2991.21 samples/s lr: 4.69e-04 [09/22 22:09:04] lb.utils.events INFO: eta: 7:43:52 iteration: 196399/375342 consumed_samples: 201113600 total_loss: 3.515 time: 0.3423 s/iter data_time: 0.2263 s/iter total_throughput: 2991.18 samples/s lr: 4.69e-04 [09/22 22:09:38] lb.utils.events INFO: eta: 7:44:02 iteration: 196499/375342 consumed_samples: 201216000 total_loss: 3.502 time: 0.3423 s/iter data_time: 0.2241 s/iter total_throughput: 2991.16 samples/s lr: 4.68e-04 [09/22 22:10:14] lb.utils.events INFO: eta: 7:44:40 iteration: 196599/375342 consumed_samples: 201318400 total_loss: 3.485 time: 0.3423 s/iter data_time: 0.2200 s/iter total_throughput: 2991.12 samples/s lr: 4.68e-04 [09/22 22:10:49] lb.utils.events INFO: eta: 7:43:25 iteration: 196699/375342 consumed_samples: 201420800 total_loss: 3.495 time: 0.3424 s/iter data_time: 0.2286 s/iter total_throughput: 2991.08 samples/s lr: 4.68e-04 [09/22 22:11:24] lb.utils.events INFO: eta: 7:42:35 iteration: 196799/375342 consumed_samples: 201523200 total_loss: 3.497 time: 0.3424 s/iter data_time: 0.2235 s/iter total_throughput: 2991.04 samples/s lr: 4.67e-04 [09/22 22:11:59] lb.utils.events INFO: eta: 7:42:20 iteration: 196899/375342 consumed_samples: 201625600 total_loss: 3.512 time: 0.3424 s/iter data_time: 0.2266 s/iter total_throughput: 2991.00 samples/s lr: 4.67e-04 [09/22 22:12:34] lb.utils.events INFO: eta: 7:41:45 iteration: 196999/375342 consumed_samples: 201728000 total_loss: 3.523 time: 0.3424 s/iter data_time: 0.2188 s/iter total_throughput: 2990.96 samples/s lr: 4.66e-04 [09/22 22:13:09] lb.utils.events INFO: eta: 7:41:01 iteration: 197099/375342 consumed_samples: 201830400 total_loss: 3.516 time: 0.3424 s/iter data_time: 0.2333 s/iter total_throughput: 2990.91 samples/s lr: 4.66e-04 [09/22 22:13:44] lb.utils.events INFO: eta: 7:40:30 iteration: 197199/375342 consumed_samples: 201932800 total_loss: 3.503 time: 0.3424 s/iter data_time: 0.2183 s/iter total_throughput: 2990.88 samples/s lr: 4.66e-04 [09/22 22:14:19] lb.utils.events INFO: eta: 7:40:32 iteration: 197299/375342 consumed_samples: 202035200 total_loss: 3.514 time: 0.3424 s/iter data_time: 0.2186 s/iter total_throughput: 2990.85 samples/s lr: 4.65e-04 [09/22 22:14:55] lb.utils.events INFO: eta: 7:40:49 iteration: 197399/375342 consumed_samples: 202137600 total_loss: 3.526 time: 0.3424 s/iter data_time: 0.2336 s/iter total_throughput: 2990.81 samples/s lr: 4.65e-04 [09/22 22:15:30] lb.utils.events INFO: eta: 7:38:13 iteration: 197499/375342 consumed_samples: 202240000 total_loss: 3.529 time: 0.3424 s/iter data_time: 0.2342 s/iter total_throughput: 2990.76 samples/s lr: 4.64e-04 [09/22 22:16:05] lb.utils.events INFO: eta: 7:37:26 iteration: 197599/375342 consumed_samples: 202342400 total_loss: 3.518 time: 0.3424 s/iter data_time: 0.2348 s/iter total_throughput: 2990.71 samples/s lr: 4.64e-04 [09/22 22:16:40] lb.utils.events INFO: eta: 7:36:55 iteration: 197699/375342 consumed_samples: 202444800 total_loss: 3.503 time: 0.3424 s/iter data_time: 0.2247 s/iter total_throughput: 2990.68 samples/s lr: 4.64e-04 [09/22 22:17:15] lb.utils.events INFO: eta: 7:36:00 iteration: 197799/375342 consumed_samples: 202547200 total_loss: 3.515 time: 0.3424 s/iter data_time: 0.2230 s/iter total_throughput: 2990.65 samples/s lr: 4.63e-04 [09/22 22:17:50] lb.utils.events INFO: eta: 7:35:02 iteration: 197899/375342 consumed_samples: 202649600 total_loss: 3.495 time: 0.3424 s/iter data_time: 0.2217 s/iter total_throughput: 2990.60 samples/s lr: 4.63e-04 [09/22 22:18:26] lb.utils.events INFO: eta: 7:34:27 iteration: 197999/375342 consumed_samples: 202752000 total_loss: 3.477 time: 0.3424 s/iter data_time: 0.2297 s/iter total_throughput: 2990.56 samples/s lr: 4.62e-04 [09/22 22:19:01] lb.utils.events INFO: eta: 7:33:16 iteration: 198099/375342 consumed_samples: 202854400 total_loss: 3.498 time: 0.3424 s/iter data_time: 0.2248 s/iter total_throughput: 2990.53 samples/s lr: 4.62e-04 [09/22 22:19:36] lb.utils.events INFO: eta: 7:33:25 iteration: 198199/375342 consumed_samples: 202956800 total_loss: 3.503 time: 0.3424 s/iter data_time: 0.2283 s/iter total_throughput: 2990.49 samples/s lr: 4.61e-04 [09/22 22:20:10] lb.utils.events INFO: eta: 7:32:54 iteration: 198299/375342 consumed_samples: 203059200 total_loss: 3.506 time: 0.3424 s/iter data_time: 0.2201 s/iter total_throughput: 2990.47 samples/s lr: 4.61e-04 [09/22 22:20:45] lb.utils.events INFO: eta: 7:32:43 iteration: 198399/375342 consumed_samples: 203161600 total_loss: 3.494 time: 0.3424 s/iter data_time: 0.2294 s/iter total_throughput: 2990.44 samples/s lr: 4.61e-04 [09/22 22:21:20] lb.utils.events INFO: eta: 7:32:59 iteration: 198499/375342 consumed_samples: 203264000 total_loss: 3.494 time: 0.3424 s/iter data_time: 0.2178 s/iter total_throughput: 2990.42 samples/s lr: 4.60e-04 [09/22 22:21:55] lb.utils.events INFO: eta: 7:33:30 iteration: 198599/375342 consumed_samples: 203366400 total_loss: 3.51 time: 0.3424 s/iter data_time: 0.2151 s/iter total_throughput: 2990.41 samples/s lr: 4.60e-04 [09/22 22:22:29] lb.utils.events INFO: eta: 7:34:18 iteration: 198699/375342 consumed_samples: 203468800 total_loss: 3.498 time: 0.3424 s/iter data_time: 0.2237 s/iter total_throughput: 2990.38 samples/s lr: 4.59e-04 [09/22 22:23:04] lb.utils.events INFO: eta: 7:34:36 iteration: 198799/375342 consumed_samples: 203571200 total_loss: 3.504 time: 0.3424 s/iter data_time: 0.2084 s/iter total_throughput: 2990.37 samples/s lr: 4.59e-04 [09/22 22:23:39] lb.utils.events INFO: eta: 7:35:20 iteration: 198899/375342 consumed_samples: 203673600 total_loss: 3.526 time: 0.3424 s/iter data_time: 0.2191 s/iter total_throughput: 2990.34 samples/s lr: 4.59e-04 [09/22 22:24:13] lb.utils.events INFO: eta: 7:36:12 iteration: 198999/375342 consumed_samples: 203776000 total_loss: 3.51 time: 0.3424 s/iter data_time: 0.2152 s/iter total_throughput: 2990.32 samples/s lr: 4.58e-04 [09/22 22:24:49] lb.utils.events INFO: eta: 7:36:38 iteration: 199099/375342 consumed_samples: 203878400 total_loss: 3.469 time: 0.3424 s/iter data_time: 0.2387 s/iter total_throughput: 2990.28 samples/s lr: 4.58e-04 [09/22 22:25:24] lb.utils.events INFO: eta: 7:36:26 iteration: 199199/375342 consumed_samples: 203980800 total_loss: 3.461 time: 0.3424 s/iter data_time: 0.2169 s/iter total_throughput: 2990.25 samples/s lr: 4.57e-04 [09/22 22:25:59] lb.utils.events INFO: eta: 7:34:41 iteration: 199299/375342 consumed_samples: 204083200 total_loss: 3.501 time: 0.3425 s/iter data_time: 0.2283 s/iter total_throughput: 2990.20 samples/s lr: 4.57e-04 [09/22 22:26:34] lb.utils.events INFO: eta: 7:33:54 iteration: 199399/375342 consumed_samples: 204185600 total_loss: 3.506 time: 0.3425 s/iter data_time: 0.2352 s/iter total_throughput: 2990.16 samples/s lr: 4.56e-04 [09/22 22:27:10] lb.utils.events INFO: eta: 7:33:21 iteration: 199499/375342 consumed_samples: 204288000 total_loss: 3.501 time: 0.3425 s/iter data_time: 0.2215 s/iter total_throughput: 2990.10 samples/s lr: 4.56e-04 [09/22 22:27:45] lb.utils.events INFO: eta: 7:32:54 iteration: 199599/375342 consumed_samples: 204390400 total_loss: 3.498 time: 0.3425 s/iter data_time: 0.2186 s/iter total_throughput: 2990.06 samples/s lr: 4.56e-04 [09/22 22:28:20] lb.utils.events INFO: eta: 7:32:41 iteration: 199699/375342 consumed_samples: 204492800 total_loss: 3.496 time: 0.3425 s/iter data_time: 0.2400 s/iter total_throughput: 2990.03 samples/s lr: 4.55e-04 [09/22 22:28:55] lb.utils.events INFO: eta: 7:32:20 iteration: 199799/375342 consumed_samples: 204595200 total_loss: 3.51 time: 0.3425 s/iter data_time: 0.2380 s/iter total_throughput: 2990.00 samples/s lr: 4.55e-04 [09/22 22:29:29] lb.utils.events INFO: eta: 7:32:12 iteration: 199899/375342 consumed_samples: 204697600 total_loss: 3.508 time: 0.3425 s/iter data_time: 0.2163 s/iter total_throughput: 2989.99 samples/s lr: 4.54e-04 [09/22 22:30:05] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0199999 [09/22 22:30:05] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/22 22:30:05] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/22 22:30:09] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0966 s/iter. Inference: 0.1658 s/iter. Eval: 0.0020 s/iter. Total: 0.2644 s/iter. ETA=0:00:09 [09/22 22:30:14] lb.evaluation.evaluator INFO: Inference done 28672/50000. Dataloading: 0.1115 s/iter. Inference: 0.1730 s/iter. Eval: 0.0020 s/iter. Total: 0.2865 s/iter. ETA=0:00:05 [09/22 22:30:19] lb.evaluation.evaluator INFO: Inference done 46080/50000. Dataloading: 0.1228 s/iter. Inference: 0.1677 s/iter. Eval: 0.0020 s/iter. Total: 0.2925 s/iter. ETA=0:00:00 [09/22 22:30:20] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/22 22:30:20] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.771609 (0.000255 s / iter per device, on 8 devices) [09/22 22:30:20] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000148 s / iter per device, on 8 devices) [09/22 22:30:20] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/22 22:30:20] lb.evaluation.utils INFO: copypaste: Acc@1=75.332 [09/22 22:30:20] lb.evaluation.utils INFO: copypaste: Acc@5=92.812 [09/22 22:30:20] lb.engine.hooks INFO: Not saving as latest eval score for Acc@1 is 75.33200, not better than best score 75.36000 @ iteration 194999. [09/22 22:30:20] lb.utils.events INFO: eta: 7:31:22 iteration: 199999/375342 consumed_samples: 204800000 total_loss: 3.496 time: 0.3425 s/iter data_time: 0.2304 s/iter total_throughput: 2989.95 samples/s lr: 4.54e-04 [09/22 22:30:54] lb.utils.events INFO: eta: 7:31:12 iteration: 200099/375342 consumed_samples: 204902400 total_loss: 3.504 time: 0.3425 s/iter data_time: 0.2341 s/iter total_throughput: 2990.00 samples/s lr: 4.54e-04 [09/22 22:31:28] lb.utils.events INFO: eta: 7:30:33 iteration: 200199/375342 consumed_samples: 205004800 total_loss: 3.502 time: 0.3425 s/iter data_time: 0.2175 s/iter total_throughput: 2989.98 samples/s lr: 4.53e-04 [09/22 22:32:03] lb.utils.events INFO: eta: 7:30:56 iteration: 200299/375342 consumed_samples: 205107200 total_loss: 3.5 time: 0.3425 s/iter data_time: 0.2241 s/iter total_throughput: 2989.95 samples/s lr: 4.53e-04 [09/22 22:32:39] lb.utils.events INFO: eta: 7:30:40 iteration: 200399/375342 consumed_samples: 205209600 total_loss: 3.491 time: 0.3425 s/iter data_time: 0.2286 s/iter total_throughput: 2989.91 samples/s lr: 4.52e-04 [09/22 22:33:13] lb.utils.events INFO: eta: 7:30:49 iteration: 200499/375342 consumed_samples: 205312000 total_loss: 3.501 time: 0.3425 s/iter data_time: 0.2262 s/iter total_throughput: 2989.89 samples/s lr: 4.52e-04 [09/22 22:33:48] lb.utils.events INFO: eta: 7:31:02 iteration: 200599/375342 consumed_samples: 205414400 total_loss: 3.519 time: 0.3425 s/iter data_time: 0.2173 s/iter total_throughput: 2989.88 samples/s lr: 4.52e-04 [09/22 22:34:22] lb.utils.events INFO: eta: 7:30:30 iteration: 200699/375342 consumed_samples: 205516800 total_loss: 3.517 time: 0.3425 s/iter data_time: 0.2163 s/iter total_throughput: 2989.87 samples/s lr: 4.51e-04 [09/22 22:34:57] lb.utils.events INFO: eta: 7:30:23 iteration: 200799/375342 consumed_samples: 205619200 total_loss: 3.495 time: 0.3425 s/iter data_time: 0.2133 s/iter total_throughput: 2989.85 samples/s lr: 4.51e-04 [09/22 22:35:32] lb.utils.events INFO: eta: 7:29:43 iteration: 200899/375342 consumed_samples: 205721600 total_loss: 3.488 time: 0.3425 s/iter data_time: 0.2207 s/iter total_throughput: 2989.82 samples/s lr: 4.50e-04 [09/22 22:36:07] lb.utils.events INFO: eta: 7:29:35 iteration: 200999/375342 consumed_samples: 205824000 total_loss: 3.488 time: 0.3425 s/iter data_time: 0.2221 s/iter total_throughput: 2989.80 samples/s lr: 4.50e-04 [09/22 22:36:42] lb.utils.events INFO: eta: 7:29:09 iteration: 201099/375342 consumed_samples: 205926400 total_loss: 3.5 time: 0.3425 s/iter data_time: 0.2283 s/iter total_throughput: 2989.76 samples/s lr: 4.49e-04 [09/22 22:37:16] lb.utils.events INFO: eta: 7:29:04 iteration: 201199/375342 consumed_samples: 206028800 total_loss: 3.508 time: 0.3425 s/iter data_time: 0.2143 s/iter total_throughput: 2989.75 samples/s lr: 4.49e-04 [09/22 22:37:51] lb.utils.events INFO: eta: 7:28:22 iteration: 201299/375342 consumed_samples: 206131200 total_loss: 3.506 time: 0.3425 s/iter data_time: 0.2284 s/iter total_throughput: 2989.72 samples/s lr: 4.49e-04 [09/22 22:38:26] lb.utils.events INFO: eta: 7:29:11 iteration: 201399/375342 consumed_samples: 206233600 total_loss: 3.516 time: 0.3425 s/iter data_time: 0.2234 s/iter total_throughput: 2989.68 samples/s lr: 4.48e-04 [09/22 22:39:01] lb.utils.events INFO: eta: 7:28:56 iteration: 201499/375342 consumed_samples: 206336000 total_loss: 3.498 time: 0.3425 s/iter data_time: 0.2272 s/iter total_throughput: 2989.65 samples/s lr: 4.48e-04 [09/22 22:39:36] lb.utils.events INFO: eta: 7:28:40 iteration: 201599/375342 consumed_samples: 206438400 total_loss: 3.489 time: 0.3425 s/iter data_time: 0.2203 s/iter total_throughput: 2989.63 samples/s lr: 4.47e-04 [09/22 22:40:11] lb.utils.events INFO: eta: 7:28:23 iteration: 201699/375342 consumed_samples: 206540800 total_loss: 3.492 time: 0.3425 s/iter data_time: 0.2238 s/iter total_throughput: 2989.61 samples/s lr: 4.47e-04 [09/22 22:40:46] lb.utils.events INFO: eta: 7:28:58 iteration: 201799/375342 consumed_samples: 206643200 total_loss: 3.482 time: 0.3425 s/iter data_time: 0.2219 s/iter total_throughput: 2989.59 samples/s lr: 4.47e-04 [09/22 22:41:20] lb.utils.events INFO: eta: 7:29:24 iteration: 201899/375342 consumed_samples: 206745600 total_loss: 3.48 time: 0.3425 s/iter data_time: 0.2092 s/iter total_throughput: 2989.58 samples/s lr: 4.46e-04 [09/22 22:41:55] lb.utils.events INFO: eta: 7:29:11 iteration: 201999/375342 consumed_samples: 206848000 total_loss: 3.485 time: 0.3425 s/iter data_time: 0.2193 s/iter total_throughput: 2989.55 samples/s lr: 4.46e-04 [09/22 22:42:30] lb.utils.events INFO: eta: 7:28:56 iteration: 202099/375342 consumed_samples: 206950400 total_loss: 3.492 time: 0.3425 s/iter data_time: 0.2202 s/iter total_throughput: 2989.53 samples/s lr: 4.45e-04 [09/22 22:43:04] lb.utils.events INFO: eta: 7:29:03 iteration: 202199/375342 consumed_samples: 207052800 total_loss: 3.49 time: 0.3425 s/iter data_time: 0.2179 s/iter total_throughput: 2989.52 samples/s lr: 4.45e-04 [09/22 22:43:39] lb.utils.events INFO: eta: 7:30:11 iteration: 202299/375342 consumed_samples: 207155200 total_loss: 3.493 time: 0.3425 s/iter data_time: 0.2197 s/iter total_throughput: 2989.51 samples/s lr: 4.45e-04 [09/22 22:44:14] lb.utils.events INFO: eta: 7:29:05 iteration: 202399/375342 consumed_samples: 207257600 total_loss: 3.512 time: 0.3425 s/iter data_time: 0.2185 s/iter total_throughput: 2989.48 samples/s lr: 4.44e-04 [09/22 22:44:49] lb.utils.events INFO: eta: 7:28:49 iteration: 202499/375342 consumed_samples: 207360000 total_loss: 3.502 time: 0.3425 s/iter data_time: 0.2307 s/iter total_throughput: 2989.45 samples/s lr: 4.44e-04 [09/22 22:45:24] lb.utils.events INFO: eta: 7:27:52 iteration: 202599/375342 consumed_samples: 207462400 total_loss: 3.482 time: 0.3425 s/iter data_time: 0.2277 s/iter total_throughput: 2989.40 samples/s lr: 4.43e-04 [09/22 22:45:58] lb.utils.events INFO: eta: 7:28:08 iteration: 202699/375342 consumed_samples: 207564800 total_loss: 3.483 time: 0.3425 s/iter data_time: 0.2146 s/iter total_throughput: 2989.39 samples/s lr: 4.43e-04 [09/22 22:46:33] lb.utils.events INFO: eta: 7:28:06 iteration: 202799/375342 consumed_samples: 207667200 total_loss: 3.483 time: 0.3425 s/iter data_time: 0.2200 s/iter total_throughput: 2989.38 samples/s lr: 4.42e-04 [09/22 22:47:08] lb.utils.events INFO: eta: 7:25:15 iteration: 202899/375342 consumed_samples: 207769600 total_loss: 3.491 time: 0.3426 s/iter data_time: 0.2217 s/iter total_throughput: 2989.34 samples/s lr: 4.42e-04 [09/22 22:47:43] lb.utils.events INFO: eta: 7:24:40 iteration: 202999/375342 consumed_samples: 207872000 total_loss: 3.491 time: 0.3426 s/iter data_time: 0.2254 s/iter total_throughput: 2989.30 samples/s lr: 4.42e-04 [09/22 22:48:18] lb.utils.events INFO: eta: 7:25:10 iteration: 203099/375342 consumed_samples: 207974400 total_loss: 3.473 time: 0.3426 s/iter data_time: 0.2195 s/iter total_throughput: 2989.27 samples/s lr: 4.41e-04 [09/22 22:48:53] lb.utils.events INFO: eta: 7:26:45 iteration: 203199/375342 consumed_samples: 208076800 total_loss: 3.465 time: 0.3426 s/iter data_time: 0.2126 s/iter total_throughput: 2989.27 samples/s lr: 4.41e-04 [09/22 22:49:28] lb.utils.events INFO: eta: 7:24:31 iteration: 203299/375342 consumed_samples: 208179200 total_loss: 3.465 time: 0.3426 s/iter data_time: 0.2296 s/iter total_throughput: 2989.22 samples/s lr: 4.40e-04 [09/22 22:50:03] lb.utils.events INFO: eta: 7:25:52 iteration: 203399/375342 consumed_samples: 208281600 total_loss: 3.485 time: 0.3426 s/iter data_time: 0.2122 s/iter total_throughput: 2989.20 samples/s lr: 4.40e-04 [09/22 22:50:38] lb.utils.events INFO: eta: 7:25:53 iteration: 203499/375342 consumed_samples: 208384000 total_loss: 3.483 time: 0.3426 s/iter data_time: 0.2226 s/iter total_throughput: 2989.17 samples/s lr: 4.40e-04 [09/22 22:51:12] lb.utils.events INFO: eta: 7:26:47 iteration: 203599/375342 consumed_samples: 208486400 total_loss: 3.472 time: 0.3426 s/iter data_time: 0.2132 s/iter total_throughput: 2989.16 samples/s lr: 4.39e-04 [09/22 22:51:47] lb.utils.events INFO: eta: 7:26:01 iteration: 203699/375342 consumed_samples: 208588800 total_loss: 3.486 time: 0.3426 s/iter data_time: 0.2253 s/iter total_throughput: 2989.13 samples/s lr: 4.39e-04 [09/22 22:52:22] lb.utils.events INFO: eta: 7:25:21 iteration: 203799/375342 consumed_samples: 208691200 total_loss: 3.491 time: 0.3426 s/iter data_time: 0.2261 s/iter total_throughput: 2989.12 samples/s lr: 4.38e-04 [09/22 22:52:57] lb.utils.events INFO: eta: 7:25:53 iteration: 203899/375342 consumed_samples: 208793600 total_loss: 3.476 time: 0.3426 s/iter data_time: 0.2313 s/iter total_throughput: 2989.10 samples/s lr: 4.38e-04 [09/22 22:53:31] lb.utils.events INFO: eta: 7:25:14 iteration: 203999/375342 consumed_samples: 208896000 total_loss: 3.482 time: 0.3426 s/iter data_time: 0.2099 s/iter total_throughput: 2989.08 samples/s lr: 4.38e-04 [09/22 22:54:06] lb.utils.events INFO: eta: 7:24:39 iteration: 204099/375342 consumed_samples: 208998400 total_loss: 3.482 time: 0.3426 s/iter data_time: 0.2210 s/iter total_throughput: 2989.06 samples/s lr: 4.37e-04 [09/22 22:54:41] lb.utils.events INFO: eta: 7:22:23 iteration: 204199/375342 consumed_samples: 209100800 total_loss: 3.474 time: 0.3426 s/iter data_time: 0.2374 s/iter total_throughput: 2989.02 samples/s lr: 4.37e-04 [09/22 22:55:16] lb.utils.events INFO: eta: 7:21:46 iteration: 204299/375342 consumed_samples: 209203200 total_loss: 3.47 time: 0.3426 s/iter data_time: 0.2222 s/iter total_throughput: 2988.99 samples/s lr: 4.36e-04 [09/22 22:55:51] lb.utils.events INFO: eta: 7:21:07 iteration: 204399/375342 consumed_samples: 209305600 total_loss: 3.474 time: 0.3426 s/iter data_time: 0.2299 s/iter total_throughput: 2988.95 samples/s lr: 4.36e-04 [09/22 22:56:26] lb.utils.events INFO: eta: 7:20:28 iteration: 204499/375342 consumed_samples: 209408000 total_loss: 3.468 time: 0.3426 s/iter data_time: 0.2116 s/iter total_throughput: 2988.94 samples/s lr: 4.36e-04 [09/22 22:57:01] lb.utils.events INFO: eta: 7:20:07 iteration: 204599/375342 consumed_samples: 209510400 total_loss: 3.47 time: 0.3426 s/iter data_time: 0.2114 s/iter total_throughput: 2988.92 samples/s lr: 4.35e-04 [09/22 22:57:36] lb.utils.events INFO: eta: 7:19:52 iteration: 204699/375342 consumed_samples: 209612800 total_loss: 3.499 time: 0.3426 s/iter data_time: 0.2321 s/iter total_throughput: 2988.89 samples/s lr: 4.35e-04 [09/22 22:58:10] lb.utils.events INFO: eta: 7:19:48 iteration: 204799/375342 consumed_samples: 209715200 total_loss: 3.497 time: 0.3426 s/iter data_time: 0.2093 s/iter total_throughput: 2988.89 samples/s lr: 4.34e-04 [09/22 22:58:45] lb.utils.events INFO: eta: 7:19:25 iteration: 204899/375342 consumed_samples: 209817600 total_loss: 3.494 time: 0.3426 s/iter data_time: 0.2143 s/iter total_throughput: 2988.86 samples/s lr: 4.34e-04 [09/22 22:59:20] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0204999 [09/22 22:59:20] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/22 22:59:20] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/22 22:59:25] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0960 s/iter. Inference: 0.1676 s/iter. Eval: 0.0019 s/iter. Total: 0.2655 s/iter. ETA=0:00:09 [09/22 22:59:30] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1248 s/iter. Inference: 0.1705 s/iter. Eval: 0.0020 s/iter. Total: 0.2974 s/iter. ETA=0:00:05 [09/22 22:59:35] lb.evaluation.evaluator INFO: Inference done 48128/50000. Dataloading: 0.1232 s/iter. Inference: 0.1669 s/iter. Eval: 0.0020 s/iter. Total: 0.2922 s/iter. ETA=0:00:00 [09/22 22:59:36] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/22 22:59:36] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.948284 (0.000259 s / iter per device, on 8 devices) [09/22 22:59:36] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000147 s / iter per device, on 8 devices) [09/22 22:59:36] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/22 22:59:36] lb.evaluation.utils INFO: copypaste: Acc@1=75.752 [09/22 22:59:36] lb.evaluation.utils INFO: copypaste: Acc@5=92.83 [09/22 22:59:36] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 75.75200, better than last best score 75.36000 @ iteration 194999. [09/22 22:59:36] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/22 22:59:37] lb.utils.events INFO: eta: 7:20:10 iteration: 204999/375342 consumed_samples: 209920000 total_loss: 3.51 time: 0.3426 s/iter data_time: 0.2216 s/iter total_throughput: 2988.83 samples/s lr: 4.33e-04 [09/22 23:00:10] lb.utils.events INFO: eta: 7:21:26 iteration: 205099/375342 consumed_samples: 210022400 total_loss: 3.524 time: 0.3426 s/iter data_time: 0.2238 s/iter total_throughput: 2988.88 samples/s lr: 4.33e-04 [09/22 23:00:44] lb.utils.events INFO: eta: 7:21:41 iteration: 205199/375342 consumed_samples: 210124800 total_loss: 3.515 time: 0.3426 s/iter data_time: 0.2241 s/iter total_throughput: 2988.86 samples/s lr: 4.33e-04 [09/22 23:01:19] lb.utils.events INFO: eta: 7:21:49 iteration: 205299/375342 consumed_samples: 210227200 total_loss: 3.499 time: 0.3426 s/iter data_time: 0.2095 s/iter total_throughput: 2988.85 samples/s lr: 4.32e-04 [09/22 23:01:54] lb.utils.events INFO: eta: 7:21:53 iteration: 205399/375342 consumed_samples: 210329600 total_loss: 3.478 time: 0.3426 s/iter data_time: 0.2150 s/iter total_throughput: 2988.83 samples/s lr: 4.32e-04 [09/22 23:02:29] lb.utils.events INFO: eta: 7:21:12 iteration: 205499/375342 consumed_samples: 210432000 total_loss: 3.473 time: 0.3426 s/iter data_time: 0.2235 s/iter total_throughput: 2988.80 samples/s lr: 4.31e-04 [09/22 23:03:04] lb.utils.events INFO: eta: 7:20:55 iteration: 205599/375342 consumed_samples: 210534400 total_loss: 3.485 time: 0.3426 s/iter data_time: 0.2237 s/iter total_throughput: 2988.77 samples/s lr: 4.31e-04 [09/22 23:03:39] lb.utils.events INFO: eta: 7:20:03 iteration: 205699/375342 consumed_samples: 210636800 total_loss: 3.488 time: 0.3426 s/iter data_time: 0.2096 s/iter total_throughput: 2988.73 samples/s lr: 4.31e-04 [09/22 23:04:13] lb.utils.events INFO: eta: 7:19:58 iteration: 205799/375342 consumed_samples: 210739200 total_loss: 3.497 time: 0.3426 s/iter data_time: 0.2312 s/iter total_throughput: 2988.72 samples/s lr: 4.30e-04 [09/22 23:04:48] lb.utils.events INFO: eta: 7:19:49 iteration: 205899/375342 consumed_samples: 210841600 total_loss: 3.489 time: 0.3426 s/iter data_time: 0.2175 s/iter total_throughput: 2988.71 samples/s lr: 4.30e-04 [09/22 23:05:22] lb.utils.events INFO: eta: 7:20:05 iteration: 205999/375342 consumed_samples: 210944000 total_loss: 3.484 time: 0.3426 s/iter data_time: 0.2162 s/iter total_throughput: 2988.72 samples/s lr: 4.29e-04 [09/22 23:05:57] lb.utils.events INFO: eta: 7:18:37 iteration: 206099/375342 consumed_samples: 211046400 total_loss: 3.501 time: 0.3426 s/iter data_time: 0.2270 s/iter total_throughput: 2988.71 samples/s lr: 4.29e-04 [09/22 23:06:31] lb.utils.events INFO: eta: 7:18:30 iteration: 206199/375342 consumed_samples: 211148800 total_loss: 3.5 time: 0.3426 s/iter data_time: 0.2107 s/iter total_throughput: 2988.71 samples/s lr: 4.29e-04 [09/22 23:07:05] lb.utils.events INFO: eta: 7:18:21 iteration: 206299/375342 consumed_samples: 211251200 total_loss: 3.478 time: 0.3426 s/iter data_time: 0.2201 s/iter total_throughput: 2988.71 samples/s lr: 4.28e-04 [09/22 23:07:40] lb.utils.events INFO: eta: 7:17:53 iteration: 206399/375342 consumed_samples: 211353600 total_loss: 3.45 time: 0.3426 s/iter data_time: 0.2239 s/iter total_throughput: 2988.68 samples/s lr: 4.28e-04 [09/22 23:08:14] lb.utils.events INFO: eta: 7:18:54 iteration: 206499/375342 consumed_samples: 211456000 total_loss: 3.445 time: 0.3426 s/iter data_time: 0.2183 s/iter total_throughput: 2988.68 samples/s lr: 4.27e-04 [09/22 23:08:49] lb.utils.events INFO: eta: 7:18:39 iteration: 206599/375342 consumed_samples: 211558400 total_loss: 3.465 time: 0.3426 s/iter data_time: 0.2264 s/iter total_throughput: 2988.67 samples/s lr: 4.27e-04 [09/22 23:09:24] lb.utils.events INFO: eta: 7:18:38 iteration: 206699/375342 consumed_samples: 211660800 total_loss: 3.478 time: 0.3426 s/iter data_time: 0.2176 s/iter total_throughput: 2988.65 samples/s lr: 4.26e-04 [09/22 23:09:58] lb.utils.events INFO: eta: 7:18:07 iteration: 206799/375342 consumed_samples: 211763200 total_loss: 3.491 time: 0.3426 s/iter data_time: 0.2160 s/iter total_throughput: 2988.63 samples/s lr: 4.26e-04 [09/22 23:10:33] lb.utils.events INFO: eta: 7:16:46 iteration: 206899/375342 consumed_samples: 211865600 total_loss: 3.487 time: 0.3426 s/iter data_time: 0.2293 s/iter total_throughput: 2988.61 samples/s lr: 4.26e-04 [09/22 23:11:07] lb.utils.events INFO: eta: 7:16:46 iteration: 206999/375342 consumed_samples: 211968000 total_loss: 3.496 time: 0.3426 s/iter data_time: 0.2109 s/iter total_throughput: 2988.61 samples/s lr: 4.25e-04 [09/22 23:11:42] lb.utils.events INFO: eta: 7:16:30 iteration: 207099/375342 consumed_samples: 212070400 total_loss: 3.488 time: 0.3426 s/iter data_time: 0.2080 s/iter total_throughput: 2988.60 samples/s lr: 4.25e-04 [09/22 23:12:17] lb.utils.events INFO: eta: 7:16:36 iteration: 207199/375342 consumed_samples: 212172800 total_loss: 3.485 time: 0.3426 s/iter data_time: 0.2216 s/iter total_throughput: 2988.57 samples/s lr: 4.24e-04 [09/22 23:12:52] lb.utils.events INFO: eta: 7:15:35 iteration: 207299/375342 consumed_samples: 212275200 total_loss: 3.488 time: 0.3426 s/iter data_time: 0.2200 s/iter total_throughput: 2988.54 samples/s lr: 4.24e-04 [09/22 23:13:26] lb.utils.events INFO: eta: 7:15:16 iteration: 207399/375342 consumed_samples: 212377600 total_loss: 3.485 time: 0.3426 s/iter data_time: 0.2125 s/iter total_throughput: 2988.53 samples/s lr: 4.24e-04 [09/22 23:14:01] lb.utils.events INFO: eta: 7:15:18 iteration: 207499/375342 consumed_samples: 212480000 total_loss: 3.479 time: 0.3426 s/iter data_time: 0.2069 s/iter total_throughput: 2988.51 samples/s lr: 4.23e-04 [09/22 23:14:36] lb.utils.events INFO: eta: 7:14:48 iteration: 207599/375342 consumed_samples: 212582400 total_loss: 3.481 time: 0.3426 s/iter data_time: 0.2081 s/iter total_throughput: 2988.50 samples/s lr: 4.23e-04 [09/22 23:15:10] lb.utils.events INFO: eta: 7:15:02 iteration: 207699/375342 consumed_samples: 212684800 total_loss: 3.477 time: 0.3426 s/iter data_time: 0.2193 s/iter total_throughput: 2988.48 samples/s lr: 4.22e-04 [09/22 23:15:45] lb.utils.events INFO: eta: 7:13:43 iteration: 207799/375342 consumed_samples: 212787200 total_loss: 3.468 time: 0.3427 s/iter data_time: 0.2138 s/iter total_throughput: 2988.47 samples/s lr: 4.22e-04 [09/22 23:16:19] lb.utils.events INFO: eta: 7:14:21 iteration: 207899/375342 consumed_samples: 212889600 total_loss: 3.48 time: 0.3427 s/iter data_time: 0.2246 s/iter total_throughput: 2988.46 samples/s lr: 4.22e-04 [09/22 23:16:54] lb.utils.events INFO: eta: 7:13:57 iteration: 207999/375342 consumed_samples: 212992000 total_loss: 3.475 time: 0.3427 s/iter data_time: 0.2285 s/iter total_throughput: 2988.45 samples/s lr: 4.21e-04 [09/22 23:17:29] lb.utils.events INFO: eta: 7:13:01 iteration: 208099/375342 consumed_samples: 213094400 total_loss: 3.472 time: 0.3427 s/iter data_time: 0.2129 s/iter total_throughput: 2988.42 samples/s lr: 4.21e-04 [09/22 23:18:03] lb.utils.events INFO: eta: 7:12:00 iteration: 208199/375342 consumed_samples: 213196800 total_loss: 3.477 time: 0.3427 s/iter data_time: 0.2163 s/iter total_throughput: 2988.42 samples/s lr: 4.20e-04 [09/22 23:18:38] lb.utils.events INFO: eta: 7:12:37 iteration: 208299/375342 consumed_samples: 213299200 total_loss: 3.439 time: 0.3427 s/iter data_time: 0.2168 s/iter total_throughput: 2988.41 samples/s lr: 4.20e-04 [09/22 23:19:12] lb.utils.events INFO: eta: 7:11:43 iteration: 208399/375342 consumed_samples: 213401600 total_loss: 3.436 time: 0.3427 s/iter data_time: 0.2229 s/iter total_throughput: 2988.40 samples/s lr: 4.20e-04 [09/22 23:19:48] lb.utils.events INFO: eta: 7:09:51 iteration: 208499/375342 consumed_samples: 213504000 total_loss: 3.461 time: 0.3427 s/iter data_time: 0.2340 s/iter total_throughput: 2988.36 samples/s lr: 4.19e-04 [09/22 23:20:22] lb.utils.events INFO: eta: 7:09:22 iteration: 208599/375342 consumed_samples: 213606400 total_loss: 3.493 time: 0.3427 s/iter data_time: 0.2218 s/iter total_throughput: 2988.33 samples/s lr: 4.19e-04 [09/22 23:20:57] lb.utils.events INFO: eta: 7:09:13 iteration: 208699/375342 consumed_samples: 213708800 total_loss: 3.507 time: 0.3427 s/iter data_time: 0.2250 s/iter total_throughput: 2988.33 samples/s lr: 4.18e-04 [09/22 23:21:31] lb.utils.events INFO: eta: 7:08:58 iteration: 208799/375342 consumed_samples: 213811200 total_loss: 3.479 time: 0.3427 s/iter data_time: 0.2156 s/iter total_throughput: 2988.32 samples/s lr: 4.18e-04 [09/22 23:22:06] lb.utils.events INFO: eta: 7:08:30 iteration: 208899/375342 consumed_samples: 213913600 total_loss: 3.447 time: 0.3427 s/iter data_time: 0.2183 s/iter total_throughput: 2988.31 samples/s lr: 4.18e-04 [09/22 23:22:40] lb.utils.events INFO: eta: 7:08:07 iteration: 208999/375342 consumed_samples: 214016000 total_loss: 3.442 time: 0.3427 s/iter data_time: 0.2061 s/iter total_throughput: 2988.30 samples/s lr: 4.17e-04 [09/22 23:23:15] lb.utils.events INFO: eta: 7:07:51 iteration: 209099/375342 consumed_samples: 214118400 total_loss: 3.448 time: 0.3427 s/iter data_time: 0.2134 s/iter total_throughput: 2988.30 samples/s lr: 4.17e-04 [09/22 23:23:49] lb.utils.events INFO: eta: 7:07:36 iteration: 209199/375342 consumed_samples: 214220800 total_loss: 3.455 time: 0.3427 s/iter data_time: 0.2184 s/iter total_throughput: 2988.29 samples/s lr: 4.16e-04 [09/22 23:24:23] lb.utils.events INFO: eta: 7:07:35 iteration: 209299/375342 consumed_samples: 214323200 total_loss: 3.475 time: 0.3427 s/iter data_time: 0.2122 s/iter total_throughput: 2988.30 samples/s lr: 4.16e-04 [09/22 23:24:58] lb.utils.events INFO: eta: 7:08:10 iteration: 209399/375342 consumed_samples: 214425600 total_loss: 3.458 time: 0.3427 s/iter data_time: 0.2277 s/iter total_throughput: 2988.28 samples/s lr: 4.15e-04 [09/22 23:25:32] lb.utils.events INFO: eta: 7:08:57 iteration: 209499/375342 consumed_samples: 214528000 total_loss: 3.451 time: 0.3427 s/iter data_time: 0.2193 s/iter total_throughput: 2988.27 samples/s lr: 4.15e-04 [09/22 23:26:07] lb.utils.events INFO: eta: 7:09:18 iteration: 209599/375342 consumed_samples: 214630400 total_loss: 3.478 time: 0.3427 s/iter data_time: 0.2144 s/iter total_throughput: 2988.25 samples/s lr: 4.15e-04 [09/22 23:26:41] lb.utils.events INFO: eta: 7:08:26 iteration: 209699/375342 consumed_samples: 214732800 total_loss: 3.481 time: 0.3427 s/iter data_time: 0.2178 s/iter total_throughput: 2988.25 samples/s lr: 4.14e-04 [09/22 23:27:16] lb.utils.events INFO: eta: 7:07:50 iteration: 209799/375342 consumed_samples: 214835200 total_loss: 3.47 time: 0.3427 s/iter data_time: 0.2364 s/iter total_throughput: 2988.24 samples/s lr: 4.14e-04 [09/22 23:27:50] lb.utils.events INFO: eta: 7:08:10 iteration: 209899/375342 consumed_samples: 214937600 total_loss: 3.477 time: 0.3427 s/iter data_time: 0.2040 s/iter total_throughput: 2988.26 samples/s lr: 4.13e-04 [09/22 23:28:24] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0209999 [09/22 23:28:25] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/22 23:28:25] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/22 23:28:29] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0979 s/iter. Inference: 0.1608 s/iter. Eval: 0.0021 s/iter. Total: 0.2608 s/iter. ETA=0:00:09 [09/22 23:28:34] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1144 s/iter. Inference: 0.1796 s/iter. Eval: 0.0022 s/iter. Total: 0.2963 s/iter. ETA=0:00:05 [09/22 23:28:39] lb.evaluation.evaluator INFO: Inference done 48128/50000. Dataloading: 0.1152 s/iter. Inference: 0.1719 s/iter. Eval: 0.0021 s/iter. Total: 0.2894 s/iter. ETA=0:00:00 [09/22 23:28:40] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/22 23:28:40] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.538280 (0.000251 s / iter per device, on 8 devices) [09/22 23:28:40] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000152 s / iter per device, on 8 devices) [09/22 23:28:40] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/22 23:28:40] lb.evaluation.utils INFO: copypaste: Acc@1=76.024 [09/22 23:28:40] lb.evaluation.utils INFO: copypaste: Acc@5=92.988 [09/22 23:28:40] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 76.02400, better than last best score 75.75200 @ iteration 204999. [09/22 23:28:40] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/22 23:28:41] lb.utils.events INFO: eta: 7:07:43 iteration: 209999/375342 consumed_samples: 215040000 total_loss: 3.483 time: 0.3427 s/iter data_time: 0.2171 s/iter total_throughput: 2988.25 samples/s lr: 4.13e-04 [09/22 23:29:13] lb.utils.events INFO: eta: 7:09:28 iteration: 210099/375342 consumed_samples: 215142400 total_loss: 3.504 time: 0.3427 s/iter data_time: 0.2492 s/iter total_throughput: 2988.33 samples/s lr: 4.13e-04 [09/22 23:29:48] lb.utils.events INFO: eta: 7:09:00 iteration: 210199/375342 consumed_samples: 215244800 total_loss: 3.484 time: 0.3427 s/iter data_time: 0.2242 s/iter total_throughput: 2988.31 samples/s lr: 4.12e-04 [09/22 23:30:23] lb.utils.events INFO: eta: 7:06:35 iteration: 210299/375342 consumed_samples: 215347200 total_loss: 3.469 time: 0.3427 s/iter data_time: 0.2289 s/iter total_throughput: 2988.29 samples/s lr: 4.12e-04 [09/22 23:30:58] lb.utils.events INFO: eta: 7:05:07 iteration: 210399/375342 consumed_samples: 215449600 total_loss: 3.456 time: 0.3427 s/iter data_time: 0.2239 s/iter total_throughput: 2988.25 samples/s lr: 4.11e-04 [09/22 23:31:33] lb.utils.events INFO: eta: 7:06:17 iteration: 210499/375342 consumed_samples: 215552000 total_loss: 3.434 time: 0.3427 s/iter data_time: 0.2390 s/iter total_throughput: 2988.20 samples/s lr: 4.11e-04 [09/22 23:32:08] lb.utils.events INFO: eta: 7:05:45 iteration: 210599/375342 consumed_samples: 215654400 total_loss: 3.446 time: 0.3427 s/iter data_time: 0.2273 s/iter total_throughput: 2988.16 samples/s lr: 4.11e-04 [09/22 23:32:44] lb.utils.events INFO: eta: 7:05:29 iteration: 210699/375342 consumed_samples: 215756800 total_loss: 3.464 time: 0.3427 s/iter data_time: 0.2226 s/iter total_throughput: 2988.12 samples/s lr: 4.10e-04 [09/22 23:33:19] lb.utils.events INFO: eta: 7:06:16 iteration: 210799/375342 consumed_samples: 215859200 total_loss: 3.454 time: 0.3427 s/iter data_time: 0.2165 s/iter total_throughput: 2988.09 samples/s lr: 4.10e-04 [09/22 23:33:54] lb.utils.events INFO: eta: 7:05:38 iteration: 210899/375342 consumed_samples: 215961600 total_loss: 3.463 time: 0.3427 s/iter data_time: 0.2185 s/iter total_throughput: 2988.06 samples/s lr: 4.09e-04 [09/22 23:34:29] lb.utils.events INFO: eta: 7:06:09 iteration: 210999/375342 consumed_samples: 216064000 total_loss: 3.473 time: 0.3427 s/iter data_time: 0.2228 s/iter total_throughput: 2988.02 samples/s lr: 4.09e-04 [09/22 23:35:04] lb.utils.events INFO: eta: 7:04:47 iteration: 211099/375342 consumed_samples: 216166400 total_loss: 3.474 time: 0.3427 s/iter data_time: 0.2127 s/iter total_throughput: 2988.00 samples/s lr: 4.09e-04 [09/22 23:35:39] lb.utils.events INFO: eta: 7:05:26 iteration: 211199/375342 consumed_samples: 216268800 total_loss: 3.473 time: 0.3427 s/iter data_time: 0.2182 s/iter total_throughput: 2987.97 samples/s lr: 4.08e-04 [09/22 23:36:14] lb.utils.events INFO: eta: 7:05:54 iteration: 211299/375342 consumed_samples: 216371200 total_loss: 3.465 time: 0.3427 s/iter data_time: 0.2180 s/iter total_throughput: 2987.92 samples/s lr: 4.08e-04 [09/22 23:36:50] lb.utils.events INFO: eta: 7:06:25 iteration: 211399/375342 consumed_samples: 216473600 total_loss: 3.459 time: 0.3427 s/iter data_time: 0.2203 s/iter total_throughput: 2987.86 samples/s lr: 4.07e-04 [09/22 23:37:25] lb.utils.events INFO: eta: 7:04:59 iteration: 211499/375342 consumed_samples: 216576000 total_loss: 3.448 time: 0.3427 s/iter data_time: 0.2195 s/iter total_throughput: 2987.82 samples/s lr: 4.07e-04 [09/22 23:38:00] lb.utils.events INFO: eta: 7:06:00 iteration: 211599/375342 consumed_samples: 216678400 total_loss: 3.449 time: 0.3427 s/iter data_time: 0.2160 s/iter total_throughput: 2987.79 samples/s lr: 4.07e-04 [09/22 23:38:36] lb.utils.events INFO: eta: 7:06:04 iteration: 211699/375342 consumed_samples: 216780800 total_loss: 3.453 time: 0.3427 s/iter data_time: 0.2321 s/iter total_throughput: 2987.74 samples/s lr: 4.06e-04 [09/22 23:39:11] lb.utils.events INFO: eta: 7:04:26 iteration: 211799/375342 consumed_samples: 216883200 total_loss: 3.464 time: 0.3427 s/iter data_time: 0.2215 s/iter total_throughput: 2987.72 samples/s lr: 4.06e-04 [09/22 23:39:46] lb.utils.events INFO: eta: 7:04:11 iteration: 211899/375342 consumed_samples: 216985600 total_loss: 3.468 time: 0.3427 s/iter data_time: 0.2335 s/iter total_throughput: 2987.67 samples/s lr: 4.05e-04 [09/22 23:40:21] lb.utils.events INFO: eta: 7:03:15 iteration: 211999/375342 consumed_samples: 217088000 total_loss: 3.442 time: 0.3427 s/iter data_time: 0.2264 s/iter total_throughput: 2987.64 samples/s lr: 4.05e-04 [09/22 23:40:57] lb.utils.events INFO: eta: 7:01:02 iteration: 212099/375342 consumed_samples: 217190400 total_loss: 3.446 time: 0.3428 s/iter data_time: 0.2294 s/iter total_throughput: 2987.59 samples/s lr: 4.04e-04 [09/22 23:41:32] lb.utils.events INFO: eta: 7:00:10 iteration: 212199/375342 consumed_samples: 217292800 total_loss: 3.444 time: 0.3428 s/iter data_time: 0.2210 s/iter total_throughput: 2987.55 samples/s lr: 4.04e-04 [09/22 23:42:07] lb.utils.events INFO: eta: 6:59:34 iteration: 212299/375342 consumed_samples: 217395200 total_loss: 3.459 time: 0.3428 s/iter data_time: 0.2154 s/iter total_throughput: 2987.50 samples/s lr: 4.04e-04 [09/22 23:42:42] lb.utils.events INFO: eta: 6:58:34 iteration: 212399/375342 consumed_samples: 217497600 total_loss: 3.475 time: 0.3428 s/iter data_time: 0.2329 s/iter total_throughput: 2987.47 samples/s lr: 4.03e-04 [09/22 23:43:18] lb.utils.events INFO: eta: 6:57:43 iteration: 212499/375342 consumed_samples: 217600000 total_loss: 3.469 time: 0.3428 s/iter data_time: 0.2219 s/iter total_throughput: 2987.43 samples/s lr: 4.03e-04 [09/22 23:43:53] lb.utils.events INFO: eta: 6:57:48 iteration: 212599/375342 consumed_samples: 217702400 total_loss: 3.452 time: 0.3428 s/iter data_time: 0.2290 s/iter total_throughput: 2987.40 samples/s lr: 4.02e-04 [09/22 23:44:28] lb.utils.events INFO: eta: 6:57:18 iteration: 212699/375342 consumed_samples: 217804800 total_loss: 3.457 time: 0.3428 s/iter data_time: 0.2159 s/iter total_throughput: 2987.38 samples/s lr: 4.02e-04 [09/22 23:45:02] lb.utils.events INFO: eta: 6:57:03 iteration: 212799/375342 consumed_samples: 217907200 total_loss: 3.448 time: 0.3428 s/iter data_time: 0.2336 s/iter total_throughput: 2987.36 samples/s lr: 4.02e-04 [09/22 23:45:37] lb.utils.events INFO: eta: 6:56:57 iteration: 212899/375342 consumed_samples: 218009600 total_loss: 3.437 time: 0.3428 s/iter data_time: 0.2285 s/iter total_throughput: 2987.32 samples/s lr: 4.01e-04 [09/22 23:46:12] lb.utils.events INFO: eta: 6:56:21 iteration: 212999/375342 consumed_samples: 218112000 total_loss: 3.434 time: 0.3428 s/iter data_time: 0.2159 s/iter total_throughput: 2987.29 samples/s lr: 4.01e-04 [09/22 23:46:47] lb.utils.events INFO: eta: 6:57:50 iteration: 213099/375342 consumed_samples: 218214400 total_loss: 3.437 time: 0.3428 s/iter data_time: 0.2260 s/iter total_throughput: 2987.27 samples/s lr: 4.00e-04 [09/22 23:47:23] lb.utils.events INFO: eta: 6:57:13 iteration: 213199/375342 consumed_samples: 218316800 total_loss: 3.45 time: 0.3428 s/iter data_time: 0.2285 s/iter total_throughput: 2987.22 samples/s lr: 4.00e-04 [09/22 23:47:58] lb.utils.events INFO: eta: 6:56:12 iteration: 213299/375342 consumed_samples: 218419200 total_loss: 3.458 time: 0.3428 s/iter data_time: 0.2160 s/iter total_throughput: 2987.20 samples/s lr: 4.00e-04 [09/22 23:48:33] lb.utils.events INFO: eta: 6:57:14 iteration: 213399/375342 consumed_samples: 218521600 total_loss: 3.459 time: 0.3428 s/iter data_time: 0.2320 s/iter total_throughput: 2987.16 samples/s lr: 3.99e-04 [09/22 23:49:08] lb.utils.events INFO: eta: 6:57:48 iteration: 213499/375342 consumed_samples: 218624000 total_loss: 3.456 time: 0.3428 s/iter data_time: 0.2231 s/iter total_throughput: 2987.13 samples/s lr: 3.99e-04 [09/22 23:49:43] lb.utils.events INFO: eta: 6:57:13 iteration: 213599/375342 consumed_samples: 218726400 total_loss: 3.443 time: 0.3428 s/iter data_time: 0.2344 s/iter total_throughput: 2987.11 samples/s lr: 3.98e-04 [09/22 23:50:17] lb.utils.events INFO: eta: 6:57:29 iteration: 213699/375342 consumed_samples: 218828800 total_loss: 3.456 time: 0.3428 s/iter data_time: 0.2264 s/iter total_throughput: 2987.09 samples/s lr: 3.98e-04 [09/22 23:50:52] lb.utils.events INFO: eta: 6:56:58 iteration: 213799/375342 consumed_samples: 218931200 total_loss: 3.464 time: 0.3428 s/iter data_time: 0.2263 s/iter total_throughput: 2987.06 samples/s lr: 3.98e-04 [09/22 23:51:28] lb.utils.events INFO: eta: 6:56:19 iteration: 213899/375342 consumed_samples: 219033600 total_loss: 3.452 time: 0.3428 s/iter data_time: 0.2250 s/iter total_throughput: 2987.02 samples/s lr: 3.97e-04 [09/22 23:52:03] lb.utils.events INFO: eta: 6:56:48 iteration: 213999/375342 consumed_samples: 219136000 total_loss: 3.447 time: 0.3428 s/iter data_time: 0.2338 s/iter total_throughput: 2986.99 samples/s lr: 3.97e-04 [09/22 23:52:37] lb.utils.events INFO: eta: 6:56:39 iteration: 214099/375342 consumed_samples: 219238400 total_loss: 3.452 time: 0.3428 s/iter data_time: 0.2145 s/iter total_throughput: 2986.98 samples/s lr: 3.96e-04 [09/22 23:53:12] lb.utils.events INFO: eta: 6:55:27 iteration: 214199/375342 consumed_samples: 219340800 total_loss: 3.46 time: 0.3428 s/iter data_time: 0.2206 s/iter total_throughput: 2986.95 samples/s lr: 3.96e-04 [09/22 23:53:47] lb.utils.events INFO: eta: 6:55:35 iteration: 214299/375342 consumed_samples: 219443200 total_loss: 3.461 time: 0.3428 s/iter data_time: 0.2253 s/iter total_throughput: 2986.92 samples/s lr: 3.96e-04 [09/22 23:54:22] lb.utils.events INFO: eta: 6:56:22 iteration: 214399/375342 consumed_samples: 219545600 total_loss: 3.457 time: 0.3428 s/iter data_time: 0.2140 s/iter total_throughput: 2986.90 samples/s lr: 3.95e-04 [09/22 23:54:57] lb.utils.events INFO: eta: 6:56:40 iteration: 214499/375342 consumed_samples: 219648000 total_loss: 3.461 time: 0.3428 s/iter data_time: 0.2222 s/iter total_throughput: 2986.89 samples/s lr: 3.95e-04 [09/22 23:55:31] lb.utils.events INFO: eta: 6:57:07 iteration: 214599/375342 consumed_samples: 219750400 total_loss: 3.456 time: 0.3428 s/iter data_time: 0.2257 s/iter total_throughput: 2986.87 samples/s lr: 3.94e-04 [09/22 23:56:06] lb.utils.events INFO: eta: 6:55:58 iteration: 214699/375342 consumed_samples: 219852800 total_loss: 3.45 time: 0.3428 s/iter data_time: 0.2160 s/iter total_throughput: 2986.85 samples/s lr: 3.94e-04 [09/22 23:56:41] lb.utils.events INFO: eta: 6:55:20 iteration: 214799/375342 consumed_samples: 219955200 total_loss: 3.473 time: 0.3428 s/iter data_time: 0.2301 s/iter total_throughput: 2986.82 samples/s lr: 3.94e-04 [09/22 23:57:16] lb.utils.events INFO: eta: 6:55:12 iteration: 214899/375342 consumed_samples: 220057600 total_loss: 3.473 time: 0.3428 s/iter data_time: 0.2181 s/iter total_throughput: 2986.81 samples/s lr: 3.93e-04 [09/22 23:57:51] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0214999 [09/22 23:57:52] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/22 23:57:52] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/22 23:57:56] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.1279 s/iter. Inference: 0.1602 s/iter. Eval: 0.0020 s/iter. Total: 0.2901 s/iter. ETA=0:00:10 [09/22 23:58:01] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1165 s/iter. Inference: 0.1778 s/iter. Eval: 0.0020 s/iter. Total: 0.2963 s/iter. ETA=0:00:05 [09/22 23:58:06] lb.evaluation.evaluator INFO: Inference done 48128/50000. Dataloading: 0.1226 s/iter. Inference: 0.1691 s/iter. Eval: 0.0020 s/iter. Total: 0.2938 s/iter. ETA=0:00:00 [09/22 23:58:07] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/22 23:58:07] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.686537 (0.000254 s / iter per device, on 8 devices) [09/22 23:58:07] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000149 s / iter per device, on 8 devices) [09/22 23:58:07] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/22 23:58:07] lb.evaluation.utils INFO: copypaste: Acc@1=76.13799999999999 [09/22 23:58:07] lb.evaluation.utils INFO: copypaste: Acc@5=93.2 [09/22 23:58:07] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 76.13800, better than last best score 76.02400 @ iteration 209999. [09/22 23:58:07] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/22 23:58:07] lb.utils.events INFO: eta: 6:54:38 iteration: 214999/375342 consumed_samples: 220160000 total_loss: 3.455 time: 0.3428 s/iter data_time: 0.2245 s/iter total_throughput: 2986.78 samples/s lr: 3.93e-04 [09/22 23:58:41] lb.utils.events INFO: eta: 6:55:22 iteration: 215099/375342 consumed_samples: 220262400 total_loss: 3.453 time: 0.3428 s/iter data_time: 0.2322 s/iter total_throughput: 2986.82 samples/s lr: 3.92e-04 [09/22 23:59:16] lb.utils.events INFO: eta: 6:56:36 iteration: 215199/375342 consumed_samples: 220364800 total_loss: 3.453 time: 0.3428 s/iter data_time: 0.2131 s/iter total_throughput: 2986.79 samples/s lr: 3.92e-04 [09/22 23:59:51] lb.utils.events INFO: eta: 6:55:52 iteration: 215299/375342 consumed_samples: 220467200 total_loss: 3.438 time: 0.3428 s/iter data_time: 0.2234 s/iter total_throughput: 2986.76 samples/s lr: 3.92e-04 [09/23 00:00:25] lb.utils.events INFO: eta: 6:54:45 iteration: 215399/375342 consumed_samples: 220569600 total_loss: 3.433 time: 0.3428 s/iter data_time: 0.2146 s/iter total_throughput: 2986.75 samples/s lr: 3.91e-04 [09/23 00:01:00] lb.utils.events INFO: eta: 6:53:48 iteration: 215499/375342 consumed_samples: 220672000 total_loss: 3.453 time: 0.3429 s/iter data_time: 0.2236 s/iter total_throughput: 2986.72 samples/s lr: 3.91e-04 [09/23 00:01:36] lb.utils.events INFO: eta: 6:52:06 iteration: 215599/375342 consumed_samples: 220774400 total_loss: 3.45 time: 0.3429 s/iter data_time: 0.2329 s/iter total_throughput: 2986.68 samples/s lr: 3.90e-04 [09/23 00:02:10] lb.utils.events INFO: eta: 6:51:16 iteration: 215699/375342 consumed_samples: 220876800 total_loss: 3.448 time: 0.3429 s/iter data_time: 0.2311 s/iter total_throughput: 2986.67 samples/s lr: 3.90e-04 [09/23 00:02:45] lb.utils.events INFO: eta: 6:52:15 iteration: 215799/375342 consumed_samples: 220979200 total_loss: 3.458 time: 0.3429 s/iter data_time: 0.2157 s/iter total_throughput: 2986.66 samples/s lr: 3.90e-04 [09/23 00:03:19] lb.utils.events INFO: eta: 6:52:37 iteration: 215899/375342 consumed_samples: 221081600 total_loss: 3.464 time: 0.3429 s/iter data_time: 0.2155 s/iter total_throughput: 2986.64 samples/s lr: 3.89e-04 [09/23 00:03:55] lb.utils.events INFO: eta: 6:52:27 iteration: 215999/375342 consumed_samples: 221184000 total_loss: 3.457 time: 0.3429 s/iter data_time: 0.2232 s/iter total_throughput: 2986.61 samples/s lr: 3.89e-04 [09/23 00:04:30] lb.utils.events INFO: eta: 6:51:00 iteration: 216099/375342 consumed_samples: 221286400 total_loss: 3.445 time: 0.3429 s/iter data_time: 0.2247 s/iter total_throughput: 2986.58 samples/s lr: 3.88e-04 [09/23 00:05:04] lb.utils.events INFO: eta: 6:50:09 iteration: 216199/375342 consumed_samples: 221388800 total_loss: 3.451 time: 0.3429 s/iter data_time: 0.2127 s/iter total_throughput: 2986.56 samples/s lr: 3.88e-04 [09/23 00:05:40] lb.utils.events INFO: eta: 6:50:48 iteration: 216299/375342 consumed_samples: 221491200 total_loss: 3.443 time: 0.3429 s/iter data_time: 0.2214 s/iter total_throughput: 2986.52 samples/s lr: 3.88e-04 [09/23 00:06:14] lb.utils.events INFO: eta: 6:50:42 iteration: 216399/375342 consumed_samples: 221593600 total_loss: 3.441 time: 0.3429 s/iter data_time: 0.2171 s/iter total_throughput: 2986.50 samples/s lr: 3.87e-04 [09/23 00:06:49] lb.utils.events INFO: eta: 6:50:54 iteration: 216499/375342 consumed_samples: 221696000 total_loss: 3.442 time: 0.3429 s/iter data_time: 0.2169 s/iter total_throughput: 2986.50 samples/s lr: 3.87e-04 [09/23 00:07:24] lb.utils.events INFO: eta: 6:51:46 iteration: 216599/375342 consumed_samples: 221798400 total_loss: 3.43 time: 0.3429 s/iter data_time: 0.2242 s/iter total_throughput: 2986.48 samples/s lr: 3.86e-04 [09/23 00:07:59] lb.utils.events INFO: eta: 6:52:07 iteration: 216699/375342 consumed_samples: 221900800 total_loss: 3.439 time: 0.3429 s/iter data_time: 0.2175 s/iter total_throughput: 2986.44 samples/s lr: 3.86e-04 [09/23 00:08:34] lb.utils.events INFO: eta: 6:51:41 iteration: 216799/375342 consumed_samples: 222003200 total_loss: 3.458 time: 0.3429 s/iter data_time: 0.2217 s/iter total_throughput: 2986.41 samples/s lr: 3.86e-04 [09/23 00:09:09] lb.utils.events INFO: eta: 6:51:11 iteration: 216899/375342 consumed_samples: 222105600 total_loss: 3.456 time: 0.3429 s/iter data_time: 0.2295 s/iter total_throughput: 2986.37 samples/s lr: 3.85e-04 [09/23 00:09:44] lb.utils.events INFO: eta: 6:50:17 iteration: 216999/375342 consumed_samples: 222208000 total_loss: 3.443 time: 0.3429 s/iter data_time: 0.2195 s/iter total_throughput: 2986.34 samples/s lr: 3.85e-04 [09/23 00:10:19] lb.utils.events INFO: eta: 6:50:34 iteration: 217099/375342 consumed_samples: 222310400 total_loss: 3.465 time: 0.3429 s/iter data_time: 0.2150 s/iter total_throughput: 2986.34 samples/s lr: 3.84e-04 [09/23 00:10:53] lb.utils.events INFO: eta: 6:50:15 iteration: 217199/375342 consumed_samples: 222412800 total_loss: 3.462 time: 0.3429 s/iter data_time: 0.2204 s/iter total_throughput: 2986.34 samples/s lr: 3.84e-04 [09/23 00:11:28] lb.utils.events INFO: eta: 6:49:34 iteration: 217299/375342 consumed_samples: 222515200 total_loss: 3.446 time: 0.3429 s/iter data_time: 0.2273 s/iter total_throughput: 2986.31 samples/s lr: 3.84e-04 [09/23 00:12:03] lb.utils.events INFO: eta: 6:49:03 iteration: 217399/375342 consumed_samples: 222617600 total_loss: 3.436 time: 0.3429 s/iter data_time: 0.2187 s/iter total_throughput: 2986.26 samples/s lr: 3.83e-04 [09/23 00:12:38] lb.utils.events INFO: eta: 6:48:55 iteration: 217499/375342 consumed_samples: 222720000 total_loss: 3.449 time: 0.3429 s/iter data_time: 0.2210 s/iter total_throughput: 2986.23 samples/s lr: 3.83e-04 [09/23 00:13:14] lb.utils.events INFO: eta: 6:48:00 iteration: 217599/375342 consumed_samples: 222822400 total_loss: 3.464 time: 0.3429 s/iter data_time: 0.2375 s/iter total_throughput: 2986.20 samples/s lr: 3.82e-04 [09/23 00:13:49] lb.utils.events INFO: eta: 6:47:21 iteration: 217699/375342 consumed_samples: 222924800 total_loss: 3.466 time: 0.3429 s/iter data_time: 0.2235 s/iter total_throughput: 2986.16 samples/s lr: 3.82e-04 [09/23 00:14:23] lb.utils.events INFO: eta: 6:47:15 iteration: 217799/375342 consumed_samples: 223027200 total_loss: 3.449 time: 0.3429 s/iter data_time: 0.2205 s/iter total_throughput: 2986.15 samples/s lr: 3.81e-04 [09/23 00:14:58] lb.utils.events INFO: eta: 6:47:16 iteration: 217899/375342 consumed_samples: 223129600 total_loss: 3.431 time: 0.3429 s/iter data_time: 0.2103 s/iter total_throughput: 2986.15 samples/s lr: 3.81e-04 [09/23 00:15:33] lb.utils.events INFO: eta: 6:46:56 iteration: 217999/375342 consumed_samples: 223232000 total_loss: 3.444 time: 0.3429 s/iter data_time: 0.2181 s/iter total_throughput: 2986.12 samples/s lr: 3.81e-04 [09/23 00:16:08] lb.utils.events INFO: eta: 6:46:20 iteration: 218099/375342 consumed_samples: 223334400 total_loss: 3.444 time: 0.3429 s/iter data_time: 0.2293 s/iter total_throughput: 2986.10 samples/s lr: 3.80e-04 [09/23 00:16:42] lb.utils.events INFO: eta: 6:45:45 iteration: 218199/375342 consumed_samples: 223436800 total_loss: 3.418 time: 0.3429 s/iter data_time: 0.2181 s/iter total_throughput: 2986.08 samples/s lr: 3.80e-04 [09/23 00:17:17] lb.utils.events INFO: eta: 6:46:14 iteration: 218299/375342 consumed_samples: 223539200 total_loss: 3.437 time: 0.3429 s/iter data_time: 0.2166 s/iter total_throughput: 2986.07 samples/s lr: 3.79e-04 [09/23 00:17:51] lb.utils.events INFO: eta: 6:46:19 iteration: 218399/375342 consumed_samples: 223641600 total_loss: 3.445 time: 0.3429 s/iter data_time: 0.2180 s/iter total_throughput: 2986.07 samples/s lr: 3.79e-04 [09/23 00:18:26] lb.utils.events INFO: eta: 6:45:54 iteration: 218499/375342 consumed_samples: 223744000 total_loss: 3.438 time: 0.3429 s/iter data_time: 0.2212 s/iter total_throughput: 2986.05 samples/s lr: 3.79e-04 [09/23 00:19:01] lb.utils.events INFO: eta: 6:45:51 iteration: 218599/375342 consumed_samples: 223846400 total_loss: 3.427 time: 0.3429 s/iter data_time: 0.2118 s/iter total_throughput: 2986.03 samples/s lr: 3.78e-04 [09/23 00:19:36] lb.utils.events INFO: eta: 6:46:46 iteration: 218699/375342 consumed_samples: 223948800 total_loss: 3.431 time: 0.3429 s/iter data_time: 0.2244 s/iter total_throughput: 2986.00 samples/s lr: 3.78e-04 [09/23 00:20:10] lb.utils.events INFO: eta: 6:45:38 iteration: 218799/375342 consumed_samples: 224051200 total_loss: 3.436 time: 0.3429 s/iter data_time: 0.2264 s/iter total_throughput: 2985.99 samples/s lr: 3.77e-04 [09/23 00:20:45] lb.utils.events INFO: eta: 6:44:51 iteration: 218899/375342 consumed_samples: 224153600 total_loss: 3.434 time: 0.3429 s/iter data_time: 0.2177 s/iter total_throughput: 2985.97 samples/s lr: 3.77e-04 [09/23 00:21:19] lb.utils.events INFO: eta: 6:46:40 iteration: 218999/375342 consumed_samples: 224256000 total_loss: 3.433 time: 0.3429 s/iter data_time: 0.2162 s/iter total_throughput: 2985.98 samples/s lr: 3.77e-04 [09/23 00:21:54] lb.utils.events INFO: eta: 6:46:24 iteration: 219099/375342 consumed_samples: 224358400 total_loss: 3.419 time: 0.3429 s/iter data_time: 0.2169 s/iter total_throughput: 2985.97 samples/s lr: 3.76e-04 [09/23 00:22:28] lb.utils.events INFO: eta: 6:46:20 iteration: 219199/375342 consumed_samples: 224460800 total_loss: 3.419 time: 0.3429 s/iter data_time: 0.2168 s/iter total_throughput: 2985.97 samples/s lr: 3.76e-04 [09/23 00:23:03] lb.utils.events INFO: eta: 6:44:46 iteration: 219299/375342 consumed_samples: 224563200 total_loss: 3.422 time: 0.3429 s/iter data_time: 0.2171 s/iter total_throughput: 2985.95 samples/s lr: 3.75e-04 [09/23 00:23:38] lb.utils.events INFO: eta: 6:44:41 iteration: 219399/375342 consumed_samples: 224665600 total_loss: 3.42 time: 0.3429 s/iter data_time: 0.2180 s/iter total_throughput: 2985.94 samples/s lr: 3.75e-04 [09/23 00:24:12] lb.utils.events INFO: eta: 6:44:25 iteration: 219499/375342 consumed_samples: 224768000 total_loss: 3.43 time: 0.3429 s/iter data_time: 0.2212 s/iter total_throughput: 2985.93 samples/s lr: 3.75e-04 [09/23 00:24:47] lb.utils.events INFO: eta: 6:44:56 iteration: 219599/375342 consumed_samples: 224870400 total_loss: 3.43 time: 0.3429 s/iter data_time: 0.2113 s/iter total_throughput: 2985.90 samples/s lr: 3.74e-04 [09/23 00:25:22] lb.utils.events INFO: eta: 6:44:47 iteration: 219699/375342 consumed_samples: 224972800 total_loss: 3.422 time: 0.3429 s/iter data_time: 0.2347 s/iter total_throughput: 2985.88 samples/s lr: 3.74e-04 [09/23 00:25:57] lb.utils.events INFO: eta: 6:43:20 iteration: 219799/375342 consumed_samples: 225075200 total_loss: 3.424 time: 0.3430 s/iter data_time: 0.2417 s/iter total_throughput: 2985.85 samples/s lr: 3.73e-04 [09/23 00:26:32] lb.utils.events INFO: eta: 6:43:24 iteration: 219899/375342 consumed_samples: 225177600 total_loss: 3.419 time: 0.3430 s/iter data_time: 0.2216 s/iter total_throughput: 2985.83 samples/s lr: 3.73e-04 [09/23 00:27:07] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0219999 [09/23 00:27:07] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/23 00:27:07] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/23 00:27:11] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.1029 s/iter. Inference: 0.1615 s/iter. Eval: 0.0020 s/iter. Total: 0.2664 s/iter. ETA=0:00:09 [09/23 00:27:17] lb.evaluation.evaluator INFO: Inference done 28672/50000. Dataloading: 0.1195 s/iter. Inference: 0.1689 s/iter. Eval: 0.0020 s/iter. Total: 0.2904 s/iter. ETA=0:00:05 [09/23 00:27:22] lb.evaluation.evaluator INFO: Inference done 47104/50000. Dataloading: 0.1250 s/iter. Inference: 0.1657 s/iter. Eval: 0.0020 s/iter. Total: 0.2927 s/iter. ETA=0:00:00 [09/23 00:27:22] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/23 00:27:22] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.579033 (0.000252 s / iter per device, on 8 devices) [09/23 00:27:22] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000146 s / iter per device, on 8 devices) [09/23 00:27:22] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/23 00:27:22] lb.evaluation.utils INFO: copypaste: Acc@1=75.994 [09/23 00:27:22] lb.evaluation.utils INFO: copypaste: Acc@5=93.108 [09/23 00:27:22] lb.engine.hooks INFO: Not saving as latest eval score for Acc@1 is 75.99400, not better than best score 76.13800 @ iteration 214999. [09/23 00:27:22] lb.utils.events INFO: eta: 6:42:40 iteration: 219999/375342 consumed_samples: 225280000 total_loss: 3.438 time: 0.3430 s/iter data_time: 0.2268 s/iter total_throughput: 2985.81 samples/s lr: 3.73e-04 [09/23 00:27:55] lb.utils.events INFO: eta: 6:44:28 iteration: 220099/375342 consumed_samples: 225382400 total_loss: 3.445 time: 0.3429 s/iter data_time: 0.2271 s/iter total_throughput: 2985.89 samples/s lr: 3.72e-04 [09/23 00:28:29] lb.utils.events INFO: eta: 6:44:56 iteration: 220199/375342 consumed_samples: 225484800 total_loss: 3.438 time: 0.3429 s/iter data_time: 0.2184 s/iter total_throughput: 2985.88 samples/s lr: 3.72e-04 [09/23 00:29:04] lb.utils.events INFO: eta: 6:45:00 iteration: 220299/375342 consumed_samples: 225587200 total_loss: 3.439 time: 0.3430 s/iter data_time: 0.2346 s/iter total_throughput: 2985.86 samples/s lr: 3.71e-04 [09/23 00:29:39] lb.utils.events INFO: eta: 6:44:27 iteration: 220399/375342 consumed_samples: 225689600 total_loss: 3.438 time: 0.3430 s/iter data_time: 0.2170 s/iter total_throughput: 2985.85 samples/s lr: 3.71e-04 [09/23 00:30:13] lb.utils.events INFO: eta: 6:44:29 iteration: 220499/375342 consumed_samples: 225792000 total_loss: 3.416 time: 0.3430 s/iter data_time: 0.2140 s/iter total_throughput: 2985.85 samples/s lr: 3.71e-04 [09/23 00:30:47] lb.utils.events INFO: eta: 6:44:22 iteration: 220599/375342 consumed_samples: 225894400 total_loss: 3.414 time: 0.3430 s/iter data_time: 0.2244 s/iter total_throughput: 2985.85 samples/s lr: 3.70e-04 [09/23 00:31:22] lb.utils.events INFO: eta: 6:44:06 iteration: 220699/375342 consumed_samples: 225996800 total_loss: 3.426 time: 0.3430 s/iter data_time: 0.2168 s/iter total_throughput: 2985.85 samples/s lr: 3.70e-04 [09/23 00:31:56] lb.utils.events INFO: eta: 6:44:32 iteration: 220799/375342 consumed_samples: 226099200 total_loss: 3.438 time: 0.3430 s/iter data_time: 0.2235 s/iter total_throughput: 2985.84 samples/s lr: 3.69e-04 [09/23 00:32:31] lb.utils.events INFO: eta: 6:44:07 iteration: 220899/375342 consumed_samples: 226201600 total_loss: 3.443 time: 0.3430 s/iter data_time: 0.2200 s/iter total_throughput: 2985.83 samples/s lr: 3.69e-04 [09/23 00:33:06] lb.utils.events INFO: eta: 6:43:42 iteration: 220999/375342 consumed_samples: 226304000 total_loss: 3.443 time: 0.3430 s/iter data_time: 0.2157 s/iter total_throughput: 2985.81 samples/s lr: 3.69e-04 [09/23 00:33:40] lb.utils.events INFO: eta: 6:41:32 iteration: 221099/375342 consumed_samples: 226406400 total_loss: 3.418 time: 0.3430 s/iter data_time: 0.2175 s/iter total_throughput: 2985.80 samples/s lr: 3.68e-04 [09/23 00:34:15] lb.utils.events INFO: eta: 6:41:01 iteration: 221199/375342 consumed_samples: 226508800 total_loss: 3.431 time: 0.3430 s/iter data_time: 0.2205 s/iter total_throughput: 2985.79 samples/s lr: 3.68e-04 [09/23 00:34:50] lb.utils.events INFO: eta: 6:40:42 iteration: 221299/375342 consumed_samples: 226611200 total_loss: 3.443 time: 0.3430 s/iter data_time: 0.2312 s/iter total_throughput: 2985.76 samples/s lr: 3.68e-04 [09/23 00:35:25] lb.utils.events INFO: eta: 6:40:21 iteration: 221399/375342 consumed_samples: 226713600 total_loss: 3.44 time: 0.3430 s/iter data_time: 0.2200 s/iter total_throughput: 2985.74 samples/s lr: 3.67e-04 [09/23 00:35:59] lb.utils.events INFO: eta: 6:39:17 iteration: 221499/375342 consumed_samples: 226816000 total_loss: 3.428 time: 0.3430 s/iter data_time: 0.2246 s/iter total_throughput: 2985.73 samples/s lr: 3.67e-04 [09/23 00:36:34] lb.utils.events INFO: eta: 6:38:53 iteration: 221599/375342 consumed_samples: 226918400 total_loss: 3.413 time: 0.3430 s/iter data_time: 0.2264 s/iter total_throughput: 2985.71 samples/s lr: 3.66e-04 [09/23 00:37:09] lb.utils.events INFO: eta: 6:38:10 iteration: 221699/375342 consumed_samples: 227020800 total_loss: 3.42 time: 0.3430 s/iter data_time: 0.2188 s/iter total_throughput: 2985.69 samples/s lr: 3.66e-04 [09/23 00:37:43] lb.utils.events INFO: eta: 6:38:10 iteration: 221799/375342 consumed_samples: 227123200 total_loss: 3.43 time: 0.3430 s/iter data_time: 0.2104 s/iter total_throughput: 2985.68 samples/s lr: 3.66e-04 [09/23 00:38:18] lb.utils.events INFO: eta: 6:37:55 iteration: 221899/375342 consumed_samples: 227225600 total_loss: 3.432 time: 0.3430 s/iter data_time: 0.2218 s/iter total_throughput: 2985.65 samples/s lr: 3.65e-04 [09/23 00:38:53] lb.utils.events INFO: eta: 6:36:35 iteration: 221999/375342 consumed_samples: 227328000 total_loss: 3.421 time: 0.3430 s/iter data_time: 0.2308 s/iter total_throughput: 2985.64 samples/s lr: 3.65e-04 [09/23 00:39:28] lb.utils.events INFO: eta: 6:36:20 iteration: 222099/375342 consumed_samples: 227430400 total_loss: 3.429 time: 0.3430 s/iter data_time: 0.2185 s/iter total_throughput: 2985.62 samples/s lr: 3.64e-04 [09/23 00:40:02] lb.utils.events INFO: eta: 6:36:32 iteration: 222199/375342 consumed_samples: 227532800 total_loss: 3.442 time: 0.3430 s/iter data_time: 0.2161 s/iter total_throughput: 2985.61 samples/s lr: 3.64e-04 [09/23 00:40:37] lb.utils.events INFO: eta: 6:36:10 iteration: 222299/375342 consumed_samples: 227635200 total_loss: 3.439 time: 0.3430 s/iter data_time: 0.2130 s/iter total_throughput: 2985.60 samples/s lr: 3.64e-04 [09/23 00:41:12] lb.utils.events INFO: eta: 6:36:02 iteration: 222399/375342 consumed_samples: 227737600 total_loss: 3.41 time: 0.3430 s/iter data_time: 0.2150 s/iter total_throughput: 2985.57 samples/s lr: 3.63e-04 [09/23 00:41:46] lb.utils.events INFO: eta: 6:36:26 iteration: 222499/375342 consumed_samples: 227840000 total_loss: 3.432 time: 0.3430 s/iter data_time: 0.2187 s/iter total_throughput: 2985.57 samples/s lr: 3.63e-04 [09/23 00:42:21] lb.utils.events INFO: eta: 6:35:53 iteration: 222599/375342 consumed_samples: 227942400 total_loss: 3.447 time: 0.3430 s/iter data_time: 0.2187 s/iter total_throughput: 2985.55 samples/s lr: 3.62e-04 [09/23 00:42:56] lb.utils.events INFO: eta: 6:35:38 iteration: 222699/375342 consumed_samples: 228044800 total_loss: 3.423 time: 0.3430 s/iter data_time: 0.2149 s/iter total_throughput: 2985.54 samples/s lr: 3.62e-04 [09/23 00:43:30] lb.utils.events INFO: eta: 6:34:58 iteration: 222799/375342 consumed_samples: 228147200 total_loss: 3.434 time: 0.3430 s/iter data_time: 0.2225 s/iter total_throughput: 2985.55 samples/s lr: 3.62e-04 [09/23 00:44:04] lb.utils.events INFO: eta: 6:33:41 iteration: 222899/375342 consumed_samples: 228249600 total_loss: 3.431 time: 0.3430 s/iter data_time: 0.2080 s/iter total_throughput: 2985.54 samples/s lr: 3.61e-04 [09/23 00:44:39] lb.utils.events INFO: eta: 6:34:31 iteration: 222999/375342 consumed_samples: 228352000 total_loss: 3.417 time: 0.3430 s/iter data_time: 0.2246 s/iter total_throughput: 2985.54 samples/s lr: 3.61e-04 [09/23 00:45:13] lb.utils.events INFO: eta: 6:34:23 iteration: 223099/375342 consumed_samples: 228454400 total_loss: 3.422 time: 0.3430 s/iter data_time: 0.2125 s/iter total_throughput: 2985.54 samples/s lr: 3.60e-04 [09/23 00:45:47] lb.utils.events INFO: eta: 6:34:07 iteration: 223199/375342 consumed_samples: 228556800 total_loss: 3.419 time: 0.3430 s/iter data_time: 0.2118 s/iter total_throughput: 2985.55 samples/s lr: 3.60e-04 [09/23 00:46:21] lb.utils.events INFO: eta: 6:33:48 iteration: 223299/375342 consumed_samples: 228659200 total_loss: 3.397 time: 0.3430 s/iter data_time: 0.2099 s/iter total_throughput: 2985.55 samples/s lr: 3.60e-04 [09/23 00:46:56] lb.utils.events INFO: eta: 6:33:58 iteration: 223399/375342 consumed_samples: 228761600 total_loss: 3.391 time: 0.3430 s/iter data_time: 0.2064 s/iter total_throughput: 2985.55 samples/s lr: 3.59e-04 [09/23 00:47:30] lb.utils.events INFO: eta: 6:35:18 iteration: 223499/375342 consumed_samples: 228864000 total_loss: 3.415 time: 0.3430 s/iter data_time: 0.2298 s/iter total_throughput: 2985.55 samples/s lr: 3.59e-04 [09/23 00:48:04] lb.utils.events INFO: eta: 6:36:01 iteration: 223599/375342 consumed_samples: 228966400 total_loss: 3.421 time: 0.3430 s/iter data_time: 0.2106 s/iter total_throughput: 2985.56 samples/s lr: 3.58e-04 [09/23 00:48:39] lb.utils.events INFO: eta: 6:35:09 iteration: 223699/375342 consumed_samples: 229068800 total_loss: 3.423 time: 0.3430 s/iter data_time: 0.2291 s/iter total_throughput: 2985.54 samples/s lr: 3.58e-04 [09/23 00:49:13] lb.utils.events INFO: eta: 6:34:47 iteration: 223799/375342 consumed_samples: 229171200 total_loss: 3.423 time: 0.3430 s/iter data_time: 0.2095 s/iter total_throughput: 2985.55 samples/s lr: 3.58e-04 [09/23 00:49:47] lb.utils.events INFO: eta: 6:35:06 iteration: 223899/375342 consumed_samples: 229273600 total_loss: 3.412 time: 0.3430 s/iter data_time: 0.2150 s/iter total_throughput: 2985.55 samples/s lr: 3.57e-04 [09/23 00:50:22] lb.utils.events INFO: eta: 6:35:30 iteration: 223999/375342 consumed_samples: 229376000 total_loss: 3.414 time: 0.3430 s/iter data_time: 0.2187 s/iter total_throughput: 2985.55 samples/s lr: 3.57e-04 [09/23 00:50:56] lb.utils.events INFO: eta: 6:35:08 iteration: 224099/375342 consumed_samples: 229478400 total_loss: 3.411 time: 0.3430 s/iter data_time: 0.2195 s/iter total_throughput: 2985.53 samples/s lr: 3.56e-04 [09/23 00:51:31] lb.utils.events INFO: eta: 6:33:08 iteration: 224199/375342 consumed_samples: 229580800 total_loss: 3.406 time: 0.3430 s/iter data_time: 0.2092 s/iter total_throughput: 2985.52 samples/s lr: 3.56e-04 [09/23 00:52:06] lb.utils.events INFO: eta: 6:33:11 iteration: 224299/375342 consumed_samples: 229683200 total_loss: 3.44 time: 0.3430 s/iter data_time: 0.2234 s/iter total_throughput: 2985.50 samples/s lr: 3.56e-04 [09/23 00:52:41] lb.utils.events INFO: eta: 6:32:26 iteration: 224399/375342 consumed_samples: 229785600 total_loss: 3.424 time: 0.3430 s/iter data_time: 0.2403 s/iter total_throughput: 2985.46 samples/s lr: 3.55e-04 [09/23 00:53:16] lb.utils.events INFO: eta: 6:30:42 iteration: 224499/375342 consumed_samples: 229888000 total_loss: 3.396 time: 0.3430 s/iter data_time: 0.2236 s/iter total_throughput: 2985.42 samples/s lr: 3.55e-04 [09/23 00:53:52] lb.utils.events INFO: eta: 6:29:47 iteration: 224599/375342 consumed_samples: 229990400 total_loss: 3.409 time: 0.3430 s/iter data_time: 0.2197 s/iter total_throughput: 2985.37 samples/s lr: 3.54e-04 [09/23 00:54:27] lb.utils.events INFO: eta: 6:30:08 iteration: 224699/375342 consumed_samples: 230092800 total_loss: 3.411 time: 0.3430 s/iter data_time: 0.2341 s/iter total_throughput: 2985.33 samples/s lr: 3.54e-04 [09/23 00:55:02] lb.utils.events INFO: eta: 6:30:00 iteration: 224799/375342 consumed_samples: 230195200 total_loss: 3.422 time: 0.3430 s/iter data_time: 0.2113 s/iter total_throughput: 2985.31 samples/s lr: 3.54e-04 [09/23 00:55:37] lb.utils.events INFO: eta: 6:29:56 iteration: 224899/375342 consumed_samples: 230297600 total_loss: 3.414 time: 0.3430 s/iter data_time: 0.2338 s/iter total_throughput: 2985.29 samples/s lr: 3.53e-04 [09/23 00:56:12] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0224999 [09/23 00:56:13] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/23 00:56:13] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/23 00:56:17] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0914 s/iter. Inference: 0.1610 s/iter. Eval: 0.0017 s/iter. Total: 0.2542 s/iter. ETA=0:00:09 [09/23 00:56:22] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1374 s/iter. Inference: 0.1625 s/iter. Eval: 0.0020 s/iter. Total: 0.3019 s/iter. ETA=0:00:05 [09/23 00:56:27] lb.evaluation.evaluator INFO: Inference done 49152/50000. Dataloading: 0.1206 s/iter. Inference: 0.1624 s/iter. Eval: 0.0020 s/iter. Total: 0.2852 s/iter. ETA=0:00:00 [09/23 00:56:28] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/23 00:56:28] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.645186 (0.000253 s / iter per device, on 8 devices) [09/23 00:56:28] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000144 s / iter per device, on 8 devices) [09/23 00:56:28] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/23 00:56:28] lb.evaluation.utils INFO: copypaste: Acc@1=76.41 [09/23 00:56:28] lb.evaluation.utils INFO: copypaste: Acc@5=93.176 [09/23 00:56:28] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 76.41000, better than last best score 76.13800 @ iteration 214999. [09/23 00:56:28] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/23 00:56:29] lb.utils.events INFO: eta: 6:29:41 iteration: 224999/375342 consumed_samples: 230400000 total_loss: 3.409 time: 0.3430 s/iter data_time: 0.2178 s/iter total_throughput: 2985.26 samples/s lr: 3.53e-04 [09/23 00:57:01] lb.utils.events INFO: eta: 6:30:00 iteration: 225099/375342 consumed_samples: 230502400 total_loss: 3.419 time: 0.3430 s/iter data_time: 0.2161 s/iter total_throughput: 2985.32 samples/s lr: 3.52e-04 [09/23 00:57:36] lb.utils.events INFO: eta: 6:30:20 iteration: 225199/375342 consumed_samples: 230604800 total_loss: 3.4 time: 0.3430 s/iter data_time: 0.2129 s/iter total_throughput: 2985.30 samples/s lr: 3.52e-04 [09/23 00:58:11] lb.utils.events INFO: eta: 6:29:03 iteration: 225299/375342 consumed_samples: 230707200 total_loss: 3.376 time: 0.3430 s/iter data_time: 0.2229 s/iter total_throughput: 2985.27 samples/s lr: 3.52e-04 [09/23 00:58:47] lb.utils.events INFO: eta: 6:28:58 iteration: 225399/375342 consumed_samples: 230809600 total_loss: 3.381 time: 0.3430 s/iter data_time: 0.2335 s/iter total_throughput: 2985.23 samples/s lr: 3.51e-04 [09/23 00:59:22] lb.utils.events INFO: eta: 6:28:29 iteration: 225499/375342 consumed_samples: 230912000 total_loss: 3.416 time: 0.3430 s/iter data_time: 0.2226 s/iter total_throughput: 2985.20 samples/s lr: 3.51e-04 [09/23 00:59:57] lb.utils.events INFO: eta: 6:28:10 iteration: 225599/375342 consumed_samples: 231014400 total_loss: 3.413 time: 0.3430 s/iter data_time: 0.2273 s/iter total_throughput: 2985.16 samples/s lr: 3.50e-04 [09/23 01:00:32] lb.utils.events INFO: eta: 6:28:11 iteration: 225699/375342 consumed_samples: 231116800 total_loss: 3.396 time: 0.3430 s/iter data_time: 0.2213 s/iter total_throughput: 2985.14 samples/s lr: 3.50e-04 [09/23 01:01:07] lb.utils.events INFO: eta: 6:29:04 iteration: 225799/375342 consumed_samples: 231219200 total_loss: 3.396 time: 0.3430 s/iter data_time: 0.2272 s/iter total_throughput: 2985.12 samples/s lr: 3.50e-04 [09/23 01:01:41] lb.utils.events INFO: eta: 6:28:28 iteration: 225899/375342 consumed_samples: 231321600 total_loss: 3.416 time: 0.3430 s/iter data_time: 0.2268 s/iter total_throughput: 2985.11 samples/s lr: 3.49e-04 [09/23 01:02:17] lb.utils.events INFO: eta: 6:27:30 iteration: 225999/375342 consumed_samples: 231424000 total_loss: 3.43 time: 0.3430 s/iter data_time: 0.2257 s/iter total_throughput: 2985.08 samples/s lr: 3.49e-04 [09/23 01:02:51] lb.utils.events INFO: eta: 6:26:18 iteration: 226099/375342 consumed_samples: 231526400 total_loss: 3.427 time: 0.3430 s/iter data_time: 0.2201 s/iter total_throughput: 2985.07 samples/s lr: 3.49e-04 [09/23 01:03:26] lb.utils.events INFO: eta: 6:26:25 iteration: 226199/375342 consumed_samples: 231628800 total_loss: 3.41 time: 0.3430 s/iter data_time: 0.2211 s/iter total_throughput: 2985.05 samples/s lr: 3.48e-04 [09/23 01:04:01] lb.utils.events INFO: eta: 6:26:49 iteration: 226299/375342 consumed_samples: 231731200 total_loss: 3.408 time: 0.3430 s/iter data_time: 0.2190 s/iter total_throughput: 2985.03 samples/s lr: 3.48e-04 [09/23 01:04:35] lb.utils.events INFO: eta: 6:26:51 iteration: 226399/375342 consumed_samples: 231833600 total_loss: 3.399 time: 0.3430 s/iter data_time: 0.2287 s/iter total_throughput: 2985.01 samples/s lr: 3.47e-04 [09/23 01:05:10] lb.utils.events INFO: eta: 6:26:05 iteration: 226499/375342 consumed_samples: 231936000 total_loss: 3.384 time: 0.3431 s/iter data_time: 0.2238 s/iter total_throughput: 2984.99 samples/s lr: 3.47e-04 [09/23 01:05:45] lb.utils.events INFO: eta: 6:25:49 iteration: 226599/375342 consumed_samples: 232038400 total_loss: 3.378 time: 0.3431 s/iter data_time: 0.2158 s/iter total_throughput: 2984.96 samples/s lr: 3.47e-04 [09/23 01:06:21] lb.utils.events INFO: eta: 6:24:22 iteration: 226699/375342 consumed_samples: 232140800 total_loss: 3.397 time: 0.3431 s/iter data_time: 0.2205 s/iter total_throughput: 2984.92 samples/s lr: 3.46e-04 [09/23 01:06:55] lb.utils.events INFO: eta: 6:23:33 iteration: 226799/375342 consumed_samples: 232243200 total_loss: 3.398 time: 0.3431 s/iter data_time: 0.2255 s/iter total_throughput: 2984.91 samples/s lr: 3.46e-04 [09/23 01:07:30] lb.utils.events INFO: eta: 6:24:05 iteration: 226899/375342 consumed_samples: 232345600 total_loss: 3.393 time: 0.3431 s/iter data_time: 0.2166 s/iter total_throughput: 2984.91 samples/s lr: 3.45e-04 [09/23 01:08:05] lb.utils.events INFO: eta: 6:23:35 iteration: 226999/375342 consumed_samples: 232448000 total_loss: 3.397 time: 0.3431 s/iter data_time: 0.2317 s/iter total_throughput: 2984.89 samples/s lr: 3.45e-04 [09/23 01:08:40] lb.utils.events INFO: eta: 6:23:12 iteration: 227099/375342 consumed_samples: 232550400 total_loss: 3.398 time: 0.3431 s/iter data_time: 0.2221 s/iter total_throughput: 2984.86 samples/s lr: 3.45e-04 [09/23 01:09:15] lb.utils.events INFO: eta: 6:22:42 iteration: 227199/375342 consumed_samples: 232652800 total_loss: 3.408 time: 0.3431 s/iter data_time: 0.2165 s/iter total_throughput: 2984.84 samples/s lr: 3.44e-04 [09/23 01:09:50] lb.utils.events INFO: eta: 6:22:45 iteration: 227299/375342 consumed_samples: 232755200 total_loss: 3.407 time: 0.3431 s/iter data_time: 0.2191 s/iter total_throughput: 2984.80 samples/s lr: 3.44e-04 [09/23 01:10:24] lb.utils.events INFO: eta: 6:22:01 iteration: 227399/375342 consumed_samples: 232857600 total_loss: 3.396 time: 0.3431 s/iter data_time: 0.2173 s/iter total_throughput: 2984.79 samples/s lr: 3.43e-04 [09/23 01:11:00] lb.utils.events INFO: eta: 6:22:13 iteration: 227499/375342 consumed_samples: 232960000 total_loss: 3.404 time: 0.3431 s/iter data_time: 0.2058 s/iter total_throughput: 2984.74 samples/s lr: 3.43e-04 [09/23 01:11:36] lb.utils.events INFO: eta: 6:21:36 iteration: 227599/375342 consumed_samples: 233062400 total_loss: 3.424 time: 0.3431 s/iter data_time: 0.2292 s/iter total_throughput: 2984.70 samples/s lr: 3.43e-04 [09/23 01:12:10] lb.utils.events INFO: eta: 6:22:06 iteration: 227699/375342 consumed_samples: 233164800 total_loss: 3.423 time: 0.3431 s/iter data_time: 0.2130 s/iter total_throughput: 2984.67 samples/s lr: 3.42e-04 [09/23 01:12:46] lb.utils.events INFO: eta: 6:21:05 iteration: 227799/375342 consumed_samples: 233267200 total_loss: 3.403 time: 0.3431 s/iter data_time: 0.2267 s/iter total_throughput: 2984.64 samples/s lr: 3.42e-04 [09/23 01:13:20] lb.utils.events INFO: eta: 6:20:32 iteration: 227899/375342 consumed_samples: 233369600 total_loss: 3.394 time: 0.3431 s/iter data_time: 0.2203 s/iter total_throughput: 2984.63 samples/s lr: 3.41e-04 [09/23 01:13:55] lb.utils.events INFO: eta: 6:20:16 iteration: 227999/375342 consumed_samples: 233472000 total_loss: 3.401 time: 0.3431 s/iter data_time: 0.2258 s/iter total_throughput: 2984.60 samples/s lr: 3.41e-04 [09/23 01:14:30] lb.utils.events INFO: eta: 6:19:53 iteration: 228099/375342 consumed_samples: 233574400 total_loss: 3.408 time: 0.3431 s/iter data_time: 0.2175 s/iter total_throughput: 2984.58 samples/s lr: 3.41e-04 [09/23 01:15:05] lb.utils.events INFO: eta: 6:19:41 iteration: 228199/375342 consumed_samples: 233676800 total_loss: 3.402 time: 0.3431 s/iter data_time: 0.2232 s/iter total_throughput: 2984.56 samples/s lr: 3.40e-04 [09/23 01:15:40] lb.utils.events INFO: eta: 6:19:15 iteration: 228299/375342 consumed_samples: 233779200 total_loss: 3.394 time: 0.3431 s/iter data_time: 0.2273 s/iter total_throughput: 2984.54 samples/s lr: 3.40e-04 [09/23 01:16:15] lb.utils.events INFO: eta: 6:18:58 iteration: 228399/375342 consumed_samples: 233881600 total_loss: 3.402 time: 0.3431 s/iter data_time: 0.2279 s/iter total_throughput: 2984.51 samples/s lr: 3.40e-04 [09/23 01:16:50] lb.utils.events INFO: eta: 6:19:14 iteration: 228499/375342 consumed_samples: 233984000 total_loss: 3.422 time: 0.3431 s/iter data_time: 0.2241 s/iter total_throughput: 2984.50 samples/s lr: 3.39e-04 [09/23 01:17:25] lb.utils.events INFO: eta: 6:19:10 iteration: 228599/375342 consumed_samples: 234086400 total_loss: 3.411 time: 0.3431 s/iter data_time: 0.2225 s/iter total_throughput: 2984.46 samples/s lr: 3.39e-04 [09/23 01:18:00] lb.utils.events INFO: eta: 6:20:11 iteration: 228699/375342 consumed_samples: 234188800 total_loss: 3.405 time: 0.3431 s/iter data_time: 0.2138 s/iter total_throughput: 2984.44 samples/s lr: 3.38e-04 [09/23 01:18:35] lb.utils.events INFO: eta: 6:19:55 iteration: 228799/375342 consumed_samples: 234291200 total_loss: 3.417 time: 0.3431 s/iter data_time: 0.2342 s/iter total_throughput: 2984.41 samples/s lr: 3.38e-04 [09/23 01:19:10] lb.utils.events INFO: eta: 6:19:04 iteration: 228899/375342 consumed_samples: 234393600 total_loss: 3.412 time: 0.3431 s/iter data_time: 0.2254 s/iter total_throughput: 2984.38 samples/s lr: 3.38e-04 [09/23 01:19:45] lb.utils.events INFO: eta: 6:19:33 iteration: 228999/375342 consumed_samples: 234496000 total_loss: 3.394 time: 0.3431 s/iter data_time: 0.2176 s/iter total_throughput: 2984.37 samples/s lr: 3.37e-04 [09/23 01:20:20] lb.utils.events INFO: eta: 6:19:25 iteration: 229099/375342 consumed_samples: 234598400 total_loss: 3.387 time: 0.3431 s/iter data_time: 0.2314 s/iter total_throughput: 2984.33 samples/s lr: 3.37e-04 [09/23 01:20:55] lb.utils.events INFO: eta: 6:17:18 iteration: 229199/375342 consumed_samples: 234700800 total_loss: 3.398 time: 0.3431 s/iter data_time: 0.2278 s/iter total_throughput: 2984.29 samples/s lr: 3.36e-04 [09/23 01:21:30] lb.utils.events INFO: eta: 6:18:11 iteration: 229299/375342 consumed_samples: 234803200 total_loss: 3.401 time: 0.3431 s/iter data_time: 0.2192 s/iter total_throughput: 2984.28 samples/s lr: 3.36e-04 [09/23 01:22:05] lb.utils.events INFO: eta: 6:18:16 iteration: 229399/375342 consumed_samples: 234905600 total_loss: 3.399 time: 0.3431 s/iter data_time: 0.2157 s/iter total_throughput: 2984.25 samples/s lr: 3.36e-04 [09/23 01:22:40] lb.utils.events INFO: eta: 6:16:56 iteration: 229499/375342 consumed_samples: 235008000 total_loss: 3.403 time: 0.3431 s/iter data_time: 0.2318 s/iter total_throughput: 2984.23 samples/s lr: 3.35e-04 [09/23 01:23:15] lb.utils.events INFO: eta: 6:16:15 iteration: 229599/375342 consumed_samples: 235110400 total_loss: 3.401 time: 0.3431 s/iter data_time: 0.2400 s/iter total_throughput: 2984.19 samples/s lr: 3.35e-04 [09/23 01:23:51] lb.utils.events INFO: eta: 6:15:22 iteration: 229699/375342 consumed_samples: 235212800 total_loss: 3.397 time: 0.3431 s/iter data_time: 0.2175 s/iter total_throughput: 2984.15 samples/s lr: 3.34e-04 [09/23 01:24:26] lb.utils.events INFO: eta: 6:16:24 iteration: 229799/375342 consumed_samples: 235315200 total_loss: 3.393 time: 0.3431 s/iter data_time: 0.2146 s/iter total_throughput: 2984.13 samples/s lr: 3.34e-04 [09/23 01:25:01] lb.utils.events INFO: eta: 6:15:34 iteration: 229899/375342 consumed_samples: 235417600 total_loss: 3.398 time: 0.3432 s/iter data_time: 0.2340 s/iter total_throughput: 2984.09 samples/s lr: 3.34e-04 [09/23 01:25:36] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0229999 [09/23 01:25:36] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/23 01:25:36] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/23 01:25:40] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0924 s/iter. Inference: 0.1702 s/iter. Eval: 0.0021 s/iter. Total: 0.2647 s/iter. ETA=0:00:09 [09/23 01:25:46] lb.evaluation.evaluator INFO: Inference done 28672/50000. Dataloading: 0.1037 s/iter. Inference: 0.1848 s/iter. Eval: 0.0020 s/iter. Total: 0.2906 s/iter. ETA=0:00:05 [09/23 01:25:51] lb.evaluation.evaluator INFO: Inference done 47104/50000. Dataloading: 0.1167 s/iter. Inference: 0.1749 s/iter. Eval: 0.0020 s/iter. Total: 0.2938 s/iter. ETA=0:00:00 [09/23 01:25:51] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/23 01:25:51] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.555834 (0.000251 s / iter per device, on 8 devices) [09/23 01:25:51] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000153 s / iter per device, on 8 devices) [09/23 01:25:51] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/23 01:25:51] lb.evaluation.utils INFO: copypaste: Acc@1=76.57000000000001 [09/23 01:25:51] lb.evaluation.utils INFO: copypaste: Acc@5=93.286 [09/23 01:25:51] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 76.57000, better than last best score 76.41000 @ iteration 224999. [09/23 01:25:51] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/23 01:25:52] lb.utils.events INFO: eta: 6:15:10 iteration: 229999/375342 consumed_samples: 235520000 total_loss: 3.401 time: 0.3432 s/iter data_time: 0.2218 s/iter total_throughput: 2984.07 samples/s lr: 3.33e-04 [09/23 01:26:25] lb.utils.events INFO: eta: 6:18:20 iteration: 230099/375342 consumed_samples: 235622400 total_loss: 3.404 time: 0.3431 s/iter data_time: 0.2159 s/iter total_throughput: 2984.14 samples/s lr: 3.33e-04 [09/23 01:26:59] lb.utils.events INFO: eta: 6:20:35 iteration: 230199/375342 consumed_samples: 235724800 total_loss: 3.402 time: 0.3431 s/iter data_time: 0.2109 s/iter total_throughput: 2984.13 samples/s lr: 3.32e-04 [09/23 01:27:34] lb.utils.events INFO: eta: 6:18:57 iteration: 230299/375342 consumed_samples: 235827200 total_loss: 3.391 time: 0.3432 s/iter data_time: 0.2199 s/iter total_throughput: 2984.11 samples/s lr: 3.32e-04 [09/23 01:28:09] lb.utils.events INFO: eta: 6:19:00 iteration: 230399/375342 consumed_samples: 235929600 total_loss: 3.388 time: 0.3432 s/iter data_time: 0.2174 s/iter total_throughput: 2984.09 samples/s lr: 3.32e-04 [09/23 01:28:44] lb.utils.events INFO: eta: 6:19:35 iteration: 230499/375342 consumed_samples: 236032000 total_loss: 3.38 time: 0.3432 s/iter data_time: 0.2341 s/iter total_throughput: 2984.07 samples/s lr: 3.31e-04 [09/23 01:29:19] lb.utils.events INFO: eta: 6:19:30 iteration: 230599/375342 consumed_samples: 236134400 total_loss: 3.388 time: 0.3432 s/iter data_time: 0.2261 s/iter total_throughput: 2984.05 samples/s lr: 3.31e-04 [09/23 01:29:54] lb.utils.events INFO: eta: 6:19:04 iteration: 230699/375342 consumed_samples: 236236800 total_loss: 3.391 time: 0.3432 s/iter data_time: 0.2297 s/iter total_throughput: 2984.01 samples/s lr: 3.31e-04 [09/23 01:30:29] lb.utils.events INFO: eta: 6:16:59 iteration: 230799/375342 consumed_samples: 236339200 total_loss: 3.399 time: 0.3432 s/iter data_time: 0.2157 s/iter total_throughput: 2983.99 samples/s lr: 3.30e-04 [09/23 01:31:04] lb.utils.events INFO: eta: 6:16:35 iteration: 230899/375342 consumed_samples: 236441600 total_loss: 3.387 time: 0.3432 s/iter data_time: 0.2382 s/iter total_throughput: 2983.96 samples/s lr: 3.30e-04 [09/23 01:31:39] lb.utils.events INFO: eta: 6:15:41 iteration: 230999/375342 consumed_samples: 236544000 total_loss: 3.381 time: 0.3432 s/iter data_time: 0.2188 s/iter total_throughput: 2983.92 samples/s lr: 3.29e-04 [09/23 01:32:14] lb.utils.events INFO: eta: 6:12:46 iteration: 231099/375342 consumed_samples: 236646400 total_loss: 3.397 time: 0.3432 s/iter data_time: 0.2158 s/iter total_throughput: 2983.91 samples/s lr: 3.29e-04 [09/23 01:32:49] lb.utils.events INFO: eta: 6:12:45 iteration: 231199/375342 consumed_samples: 236748800 total_loss: 3.389 time: 0.3432 s/iter data_time: 0.2094 s/iter total_throughput: 2983.90 samples/s lr: 3.29e-04 [09/23 01:33:24] lb.utils.events INFO: eta: 6:12:24 iteration: 231299/375342 consumed_samples: 236851200 total_loss: 3.398 time: 0.3432 s/iter data_time: 0.2272 s/iter total_throughput: 2983.87 samples/s lr: 3.28e-04 [09/23 01:33:59] lb.utils.events INFO: eta: 6:11:51 iteration: 231399/375342 consumed_samples: 236953600 total_loss: 3.412 time: 0.3432 s/iter data_time: 0.2128 s/iter total_throughput: 2983.85 samples/s lr: 3.28e-04 [09/23 01:34:33] lb.utils.events INFO: eta: 6:11:47 iteration: 231499/375342 consumed_samples: 237056000 total_loss: 3.383 time: 0.3432 s/iter data_time: 0.2225 s/iter total_throughput: 2983.83 samples/s lr: 3.27e-04 [09/23 01:35:09] lb.utils.events INFO: eta: 6:10:30 iteration: 231599/375342 consumed_samples: 237158400 total_loss: 3.389 time: 0.3432 s/iter data_time: 0.2150 s/iter total_throughput: 2983.80 samples/s lr: 3.27e-04 [09/23 01:35:43] lb.utils.events INFO: eta: 6:11:50 iteration: 231699/375342 consumed_samples: 237260800 total_loss: 3.404 time: 0.3432 s/iter data_time: 0.2256 s/iter total_throughput: 2983.79 samples/s lr: 3.27e-04 [09/23 01:36:18] lb.utils.events INFO: eta: 6:12:37 iteration: 231799/375342 consumed_samples: 237363200 total_loss: 3.394 time: 0.3432 s/iter data_time: 0.2249 s/iter total_throughput: 2983.78 samples/s lr: 3.26e-04 [09/23 01:36:53] lb.utils.events INFO: eta: 6:12:59 iteration: 231899/375342 consumed_samples: 237465600 total_loss: 3.384 time: 0.3432 s/iter data_time: 0.2084 s/iter total_throughput: 2983.77 samples/s lr: 3.26e-04 [09/23 01:37:27] lb.utils.events INFO: eta: 6:12:46 iteration: 231999/375342 consumed_samples: 237568000 total_loss: 3.371 time: 0.3432 s/iter data_time: 0.2159 s/iter total_throughput: 2983.75 samples/s lr: 3.26e-04 [09/23 01:38:02] lb.utils.events INFO: eta: 6:12:46 iteration: 232099/375342 consumed_samples: 237670400 total_loss: 3.371 time: 0.3432 s/iter data_time: 0.2272 s/iter total_throughput: 2983.72 samples/s lr: 3.25e-04 [09/23 01:38:37] lb.utils.events INFO: eta: 6:12:00 iteration: 232199/375342 consumed_samples: 237772800 total_loss: 3.371 time: 0.3432 s/iter data_time: 0.2131 s/iter total_throughput: 2983.72 samples/s lr: 3.25e-04 [09/23 01:39:11] lb.utils.events INFO: eta: 6:11:43 iteration: 232299/375342 consumed_samples: 237875200 total_loss: 3.381 time: 0.3432 s/iter data_time: 0.2285 s/iter total_throughput: 2983.70 samples/s lr: 3.24e-04 [09/23 01:39:47] lb.utils.events INFO: eta: 6:10:27 iteration: 232399/375342 consumed_samples: 237977600 total_loss: 3.381 time: 0.3432 s/iter data_time: 0.2298 s/iter total_throughput: 2983.66 samples/s lr: 3.24e-04 [09/23 01:40:22] lb.utils.events INFO: eta: 6:09:20 iteration: 232499/375342 consumed_samples: 238080000 total_loss: 3.38 time: 0.3432 s/iter data_time: 0.2171 s/iter total_throughput: 2983.64 samples/s lr: 3.24e-04 [09/23 01:40:57] lb.utils.events INFO: eta: 6:09:27 iteration: 232599/375342 consumed_samples: 238182400 total_loss: 3.404 time: 0.3432 s/iter data_time: 0.2180 s/iter total_throughput: 2983.61 samples/s lr: 3.23e-04 [09/23 01:41:32] lb.utils.events INFO: eta: 6:07:48 iteration: 232699/375342 consumed_samples: 238284800 total_loss: 3.406 time: 0.3432 s/iter data_time: 0.2221 s/iter total_throughput: 2983.58 samples/s lr: 3.23e-04 [09/23 01:42:07] lb.utils.events INFO: eta: 6:07:21 iteration: 232799/375342 consumed_samples: 238387200 total_loss: 3.378 time: 0.3432 s/iter data_time: 0.2125 s/iter total_throughput: 2983.58 samples/s lr: 3.22e-04 [09/23 01:42:42] lb.utils.events INFO: eta: 6:06:23 iteration: 232899/375342 consumed_samples: 238489600 total_loss: 3.376 time: 0.3432 s/iter data_time: 0.2285 s/iter total_throughput: 2983.55 samples/s lr: 3.22e-04 [09/23 01:43:17] lb.utils.events INFO: eta: 6:05:49 iteration: 232999/375342 consumed_samples: 238592000 total_loss: 3.386 time: 0.3432 s/iter data_time: 0.2231 s/iter total_throughput: 2983.53 samples/s lr: 3.22e-04 [09/23 01:43:52] lb.utils.events INFO: eta: 6:05:27 iteration: 233099/375342 consumed_samples: 238694400 total_loss: 3.367 time: 0.3432 s/iter data_time: 0.2282 s/iter total_throughput: 2983.49 samples/s lr: 3.21e-04 [09/23 01:44:27] lb.utils.events INFO: eta: 6:04:31 iteration: 233199/375342 consumed_samples: 238796800 total_loss: 3.367 time: 0.3432 s/iter data_time: 0.2339 s/iter total_throughput: 2983.47 samples/s lr: 3.21e-04 [09/23 01:45:02] lb.utils.events INFO: eta: 6:04:13 iteration: 233299/375342 consumed_samples: 238899200 total_loss: 3.373 time: 0.3432 s/iter data_time: 0.2153 s/iter total_throughput: 2983.43 samples/s lr: 3.21e-04 [09/23 01:45:37] lb.utils.events INFO: eta: 6:05:04 iteration: 233399/375342 consumed_samples: 239001600 total_loss: 3.392 time: 0.3432 s/iter data_time: 0.2250 s/iter total_throughput: 2983.41 samples/s lr: 3.20e-04 [09/23 01:46:12] lb.utils.events INFO: eta: 6:05:40 iteration: 233499/375342 consumed_samples: 239104000 total_loss: 3.396 time: 0.3432 s/iter data_time: 0.2205 s/iter total_throughput: 2983.40 samples/s lr: 3.20e-04 [09/23 01:46:47] lb.utils.events INFO: eta: 6:04:42 iteration: 233599/375342 consumed_samples: 239206400 total_loss: 3.383 time: 0.3432 s/iter data_time: 0.2331 s/iter total_throughput: 2983.37 samples/s lr: 3.19e-04 [09/23 01:47:22] lb.utils.events INFO: eta: 6:05:36 iteration: 233699/375342 consumed_samples: 239308800 total_loss: 3.388 time: 0.3432 s/iter data_time: 0.2178 s/iter total_throughput: 2983.35 samples/s lr: 3.19e-04 [09/23 01:47:56] lb.utils.events INFO: eta: 6:04:08 iteration: 233799/375342 consumed_samples: 239411200 total_loss: 3.387 time: 0.3432 s/iter data_time: 0.2211 s/iter total_throughput: 2983.35 samples/s lr: 3.19e-04 [09/23 01:48:30] lb.utils.events INFO: eta: 6:04:28 iteration: 233899/375342 consumed_samples: 239513600 total_loss: 3.393 time: 0.3432 s/iter data_time: 0.2202 s/iter total_throughput: 2983.34 samples/s lr: 3.18e-04 [09/23 01:49:05] lb.utils.events INFO: eta: 6:05:26 iteration: 233999/375342 consumed_samples: 239616000 total_loss: 3.393 time: 0.3432 s/iter data_time: 0.2378 s/iter total_throughput: 2983.33 samples/s lr: 3.18e-04 [09/23 01:49:40] lb.utils.events INFO: eta: 6:04:37 iteration: 234099/375342 consumed_samples: 239718400 total_loss: 3.355 time: 0.3432 s/iter data_time: 0.2132 s/iter total_throughput: 2983.32 samples/s lr: 3.17e-04 [09/23 01:50:14] lb.utils.events INFO: eta: 6:04:46 iteration: 234199/375342 consumed_samples: 239820800 total_loss: 3.345 time: 0.3432 s/iter data_time: 0.2206 s/iter total_throughput: 2983.31 samples/s lr: 3.17e-04 [09/23 01:50:49] lb.utils.events INFO: eta: 6:05:28 iteration: 234299/375342 consumed_samples: 239923200 total_loss: 3.364 time: 0.3432 s/iter data_time: 0.2225 s/iter total_throughput: 2983.31 samples/s lr: 3.17e-04 [09/23 01:51:23] lb.utils.events INFO: eta: 6:05:06 iteration: 234399/375342 consumed_samples: 240025600 total_loss: 3.373 time: 0.3432 s/iter data_time: 0.2226 s/iter total_throughput: 2983.29 samples/s lr: 3.16e-04 [09/23 01:51:58] lb.utils.events INFO: eta: 6:04:50 iteration: 234499/375342 consumed_samples: 240128000 total_loss: 3.393 time: 0.3432 s/iter data_time: 0.2048 s/iter total_throughput: 2983.30 samples/s lr: 3.16e-04 [09/23 01:52:33] lb.utils.events INFO: eta: 6:04:48 iteration: 234599/375342 consumed_samples: 240230400 total_loss: 3.409 time: 0.3432 s/iter data_time: 0.2253 s/iter total_throughput: 2983.28 samples/s lr: 3.16e-04 [09/23 01:53:07] lb.utils.events INFO: eta: 6:04:17 iteration: 234699/375342 consumed_samples: 240332800 total_loss: 3.376 time: 0.3432 s/iter data_time: 0.2273 s/iter total_throughput: 2983.26 samples/s lr: 3.15e-04 [09/23 01:53:42] lb.utils.events INFO: eta: 6:05:09 iteration: 234799/375342 consumed_samples: 240435200 total_loss: 3.376 time: 0.3433 s/iter data_time: 0.2128 s/iter total_throughput: 2983.25 samples/s lr: 3.15e-04 [09/23 01:54:16] lb.utils.events INFO: eta: 6:04:12 iteration: 234899/375342 consumed_samples: 240537600 total_loss: 3.397 time: 0.3432 s/iter data_time: 0.2181 s/iter total_throughput: 2983.25 samples/s lr: 3.14e-04 [09/23 01:54:51] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0234999 [09/23 01:54:52] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/23 01:54:52] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/23 01:54:56] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.1063 s/iter. Inference: 0.1608 s/iter. Eval: 0.0021 s/iter. Total: 0.2692 s/iter. ETA=0:00:09 [09/23 01:55:02] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1359 s/iter. Inference: 0.1635 s/iter. Eval: 0.0019 s/iter. Total: 0.3013 s/iter. ETA=0:00:05 [09/23 01:55:07] lb.evaluation.evaluator INFO: Inference done 49152/50000. Dataloading: 0.1243 s/iter. Inference: 0.1617 s/iter. Eval: 0.0020 s/iter. Total: 0.2880 s/iter. ETA=0:00:00 [09/23 01:55:07] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/23 01:55:07] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.672490 (0.000253 s / iter per device, on 8 devices) [09/23 01:55:07] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000143 s / iter per device, on 8 devices) [09/23 01:55:07] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/23 01:55:07] lb.evaluation.utils INFO: copypaste: Acc@1=76.82 [09/23 01:55:07] lb.evaluation.utils INFO: copypaste: Acc@5=93.374 [09/23 01:55:07] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 76.82000, better than last best score 76.57000 @ iteration 229999. [09/23 01:55:07] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/23 01:55:08] lb.utils.events INFO: eta: 6:03:44 iteration: 234999/375342 consumed_samples: 240640000 total_loss: 3.401 time: 0.3433 s/iter data_time: 0.2272 s/iter total_throughput: 2983.23 samples/s lr: 3.14e-04 [09/23 01:55:40] lb.utils.events INFO: eta: 6:06:21 iteration: 235099/375342 consumed_samples: 240742400 total_loss: 3.403 time: 0.3432 s/iter data_time: 0.2244 s/iter total_throughput: 2983.30 samples/s lr: 3.14e-04 [09/23 01:56:15] lb.utils.events INFO: eta: 6:06:11 iteration: 235199/375342 consumed_samples: 240844800 total_loss: 3.387 time: 0.3432 s/iter data_time: 0.2175 s/iter total_throughput: 2983.28 samples/s lr: 3.13e-04 [09/23 01:56:50] lb.utils.events INFO: eta: 6:05:40 iteration: 235299/375342 consumed_samples: 240947200 total_loss: 3.376 time: 0.3432 s/iter data_time: 0.2169 s/iter total_throughput: 2983.28 samples/s lr: 3.13e-04 [09/23 01:57:24] lb.utils.events INFO: eta: 6:05:52 iteration: 235399/375342 consumed_samples: 241049600 total_loss: 3.372 time: 0.3432 s/iter data_time: 0.2203 s/iter total_throughput: 2983.28 samples/s lr: 3.12e-04 [09/23 01:57:59] lb.utils.events INFO: eta: 6:05:28 iteration: 235499/375342 consumed_samples: 241152000 total_loss: 3.36 time: 0.3432 s/iter data_time: 0.2253 s/iter total_throughput: 2983.27 samples/s lr: 3.12e-04 [09/23 01:58:33] lb.utils.events INFO: eta: 6:04:13 iteration: 235599/375342 consumed_samples: 241254400 total_loss: 3.351 time: 0.3432 s/iter data_time: 0.2241 s/iter total_throughput: 2983.26 samples/s lr: 3.12e-04 [09/23 01:59:08] lb.utils.events INFO: eta: 6:04:16 iteration: 235699/375342 consumed_samples: 241356800 total_loss: 3.355 time: 0.3432 s/iter data_time: 0.2228 s/iter total_throughput: 2983.25 samples/s lr: 3.11e-04 [09/23 01:59:43] lb.utils.events INFO: eta: 6:02:51 iteration: 235799/375342 consumed_samples: 241459200 total_loss: 3.373 time: 0.3433 s/iter data_time: 0.2314 s/iter total_throughput: 2983.23 samples/s lr: 3.11e-04 [09/23 02:00:17] lb.utils.events INFO: eta: 6:03:02 iteration: 235899/375342 consumed_samples: 241561600 total_loss: 3.385 time: 0.3433 s/iter data_time: 0.2222 s/iter total_throughput: 2983.22 samples/s lr: 3.11e-04 [09/23 02:00:52] lb.utils.events INFO: eta: 6:02:29 iteration: 235999/375342 consumed_samples: 241664000 total_loss: 3.371 time: 0.3433 s/iter data_time: 0.2289 s/iter total_throughput: 2983.20 samples/s lr: 3.10e-04 [09/23 02:01:27] lb.utils.events INFO: eta: 6:00:13 iteration: 236099/375342 consumed_samples: 241766400 total_loss: 3.391 time: 0.3433 s/iter data_time: 0.2170 s/iter total_throughput: 2983.19 samples/s lr: 3.10e-04 [09/23 02:02:01] lb.utils.events INFO: eta: 6:00:10 iteration: 236199/375342 consumed_samples: 241868800 total_loss: 3.391 time: 0.3433 s/iter data_time: 0.2112 s/iter total_throughput: 2983.18 samples/s lr: 3.09e-04 [09/23 02:02:35] lb.utils.events INFO: eta: 5:59:53 iteration: 236299/375342 consumed_samples: 241971200 total_loss: 3.356 time: 0.3433 s/iter data_time: 0.2077 s/iter total_throughput: 2983.19 samples/s lr: 3.09e-04 [09/23 02:03:10] lb.utils.events INFO: eta: 5:59:44 iteration: 236399/375342 consumed_samples: 242073600 total_loss: 3.369 time: 0.3433 s/iter data_time: 0.2207 s/iter total_throughput: 2983.19 samples/s lr: 3.09e-04 [09/23 02:03:44] lb.utils.events INFO: eta: 5:59:45 iteration: 236499/375342 consumed_samples: 242176000 total_loss: 3.383 time: 0.3433 s/iter data_time: 0.2185 s/iter total_throughput: 2983.19 samples/s lr: 3.08e-04 [09/23 02:04:19] lb.utils.events INFO: eta: 5:59:38 iteration: 236599/375342 consumed_samples: 242278400 total_loss: 3.383 time: 0.3433 s/iter data_time: 0.2163 s/iter total_throughput: 2983.18 samples/s lr: 3.08e-04 [09/23 02:04:53] lb.utils.events INFO: eta: 6:00:02 iteration: 236699/375342 consumed_samples: 242380800 total_loss: 3.378 time: 0.3433 s/iter data_time: 0.2243 s/iter total_throughput: 2983.17 samples/s lr: 3.08e-04 [09/23 02:05:28] lb.utils.events INFO: eta: 6:00:31 iteration: 236799/375342 consumed_samples: 242483200 total_loss: 3.374 time: 0.3433 s/iter data_time: 0.2155 s/iter total_throughput: 2983.16 samples/s lr: 3.07e-04 [09/23 02:06:03] lb.utils.events INFO: eta: 5:59:22 iteration: 236899/375342 consumed_samples: 242585600 total_loss: 3.364 time: 0.3433 s/iter data_time: 0.2218 s/iter total_throughput: 2983.15 samples/s lr: 3.07e-04 [09/23 02:06:37] lb.utils.events INFO: eta: 5:58:27 iteration: 236999/375342 consumed_samples: 242688000 total_loss: 3.347 time: 0.3433 s/iter data_time: 0.2268 s/iter total_throughput: 2983.14 samples/s lr: 3.06e-04 [09/23 02:07:12] lb.utils.events INFO: eta: 5:58:12 iteration: 237099/375342 consumed_samples: 242790400 total_loss: 3.379 time: 0.3433 s/iter data_time: 0.2271 s/iter total_throughput: 2983.14 samples/s lr: 3.06e-04 [09/23 02:07:46] lb.utils.events INFO: eta: 5:57:41 iteration: 237199/375342 consumed_samples: 242892800 total_loss: 3.385 time: 0.3433 s/iter data_time: 0.2137 s/iter total_throughput: 2983.13 samples/s lr: 3.06e-04 [09/23 02:08:21] lb.utils.events INFO: eta: 5:56:58 iteration: 237299/375342 consumed_samples: 242995200 total_loss: 3.355 time: 0.3433 s/iter data_time: 0.2309 s/iter total_throughput: 2983.12 samples/s lr: 3.05e-04 [09/23 02:08:55] lb.utils.events INFO: eta: 5:56:32 iteration: 237399/375342 consumed_samples: 243097600 total_loss: 3.348 time: 0.3433 s/iter data_time: 0.2075 s/iter total_throughput: 2983.12 samples/s lr: 3.05e-04 [09/23 02:09:29] lb.utils.events INFO: eta: 5:56:31 iteration: 237499/375342 consumed_samples: 243200000 total_loss: 3.346 time: 0.3433 s/iter data_time: 0.2264 s/iter total_throughput: 2983.13 samples/s lr: 3.04e-04 [09/23 02:10:04] lb.utils.events INFO: eta: 5:56:02 iteration: 237599/375342 consumed_samples: 243302400 total_loss: 3.363 time: 0.3433 s/iter data_time: 0.2102 s/iter total_throughput: 2983.12 samples/s lr: 3.04e-04 [09/23 02:10:38] lb.utils.events INFO: eta: 5:56:00 iteration: 237699/375342 consumed_samples: 243404800 total_loss: 3.385 time: 0.3433 s/iter data_time: 0.2155 s/iter total_throughput: 2983.12 samples/s lr: 3.04e-04 [09/23 02:11:12] lb.utils.events INFO: eta: 5:55:28 iteration: 237799/375342 consumed_samples: 243507200 total_loss: 3.377 time: 0.3433 s/iter data_time: 0.2034 s/iter total_throughput: 2983.13 samples/s lr: 3.03e-04 [09/23 02:11:47] lb.utils.events INFO: eta: 5:55:15 iteration: 237899/375342 consumed_samples: 243609600 total_loss: 3.354 time: 0.3433 s/iter data_time: 0.2297 s/iter total_throughput: 2983.12 samples/s lr: 3.03e-04 [09/23 02:12:21] lb.utils.events INFO: eta: 5:55:13 iteration: 237999/375342 consumed_samples: 243712000 total_loss: 3.358 time: 0.3433 s/iter data_time: 0.2096 s/iter total_throughput: 2983.13 samples/s lr: 3.03e-04 [09/23 02:12:56] lb.utils.events INFO: eta: 5:54:56 iteration: 238099/375342 consumed_samples: 243814400 total_loss: 3.374 time: 0.3433 s/iter data_time: 0.2245 s/iter total_throughput: 2983.11 samples/s lr: 3.02e-04 [09/23 02:13:30] lb.utils.events INFO: eta: 5:54:34 iteration: 238199/375342 consumed_samples: 243916800 total_loss: 3.384 time: 0.3433 s/iter data_time: 0.2413 s/iter total_throughput: 2983.09 samples/s lr: 3.02e-04 [09/23 02:14:06] lb.utils.events INFO: eta: 5:53:40 iteration: 238299/375342 consumed_samples: 244019200 total_loss: 3.374 time: 0.3433 s/iter data_time: 0.2280 s/iter total_throughput: 2983.06 samples/s lr: 3.01e-04 [09/23 02:14:41] lb.utils.events INFO: eta: 5:52:14 iteration: 238399/375342 consumed_samples: 244121600 total_loss: 3.355 time: 0.3433 s/iter data_time: 0.2230 s/iter total_throughput: 2983.02 samples/s lr: 3.01e-04 [09/23 02:15:17] lb.utils.events INFO: eta: 5:51:34 iteration: 238499/375342 consumed_samples: 244224000 total_loss: 3.343 time: 0.3433 s/iter data_time: 0.2261 s/iter total_throughput: 2982.98 samples/s lr: 3.01e-04 [09/23 02:15:52] lb.utils.events INFO: eta: 5:51:16 iteration: 238599/375342 consumed_samples: 244326400 total_loss: 3.364 time: 0.3433 s/iter data_time: 0.2320 s/iter total_throughput: 2982.93 samples/s lr: 3.00e-04 [09/23 02:16:27] lb.utils.events INFO: eta: 5:50:40 iteration: 238699/375342 consumed_samples: 244428800 total_loss: 3.385 time: 0.3433 s/iter data_time: 0.2266 s/iter total_throughput: 2982.90 samples/s lr: 3.00e-04 [09/23 02:17:02] lb.utils.events INFO: eta: 5:50:06 iteration: 238799/375342 consumed_samples: 244531200 total_loss: 3.386 time: 0.3433 s/iter data_time: 0.2246 s/iter total_throughput: 2982.87 samples/s lr: 3.00e-04 [09/23 02:17:37] lb.utils.events INFO: eta: 5:50:27 iteration: 238899/375342 consumed_samples: 244633600 total_loss: 3.393 time: 0.3433 s/iter data_time: 0.2308 s/iter total_throughput: 2982.85 samples/s lr: 2.99e-04 [09/23 02:18:13] lb.utils.events INFO: eta: 5:49:41 iteration: 238999/375342 consumed_samples: 244736000 total_loss: 3.359 time: 0.3433 s/iter data_time: 0.2193 s/iter total_throughput: 2982.82 samples/s lr: 2.99e-04 [09/23 02:18:48] lb.utils.events INFO: eta: 5:49:20 iteration: 239099/375342 consumed_samples: 244838400 total_loss: 3.363 time: 0.3433 s/iter data_time: 0.2264 s/iter total_throughput: 2982.77 samples/s lr: 2.98e-04 [09/23 02:19:23] lb.utils.events INFO: eta: 5:50:18 iteration: 239199/375342 consumed_samples: 244940800 total_loss: 3.363 time: 0.3433 s/iter data_time: 0.2204 s/iter total_throughput: 2982.75 samples/s lr: 2.98e-04 [09/23 02:19:59] lb.utils.events INFO: eta: 5:51:12 iteration: 239299/375342 consumed_samples: 245043200 total_loss: 3.346 time: 0.3433 s/iter data_time: 0.2482 s/iter total_throughput: 2982.71 samples/s lr: 2.98e-04 [09/23 02:20:34] lb.utils.events INFO: eta: 5:51:07 iteration: 239399/375342 consumed_samples: 245145600 total_loss: 3.341 time: 0.3433 s/iter data_time: 0.2264 s/iter total_throughput: 2982.66 samples/s lr: 2.97e-04 [09/23 02:21:09] lb.utils.events INFO: eta: 5:50:43 iteration: 239499/375342 consumed_samples: 245248000 total_loss: 3.356 time: 0.3433 s/iter data_time: 0.2306 s/iter total_throughput: 2982.63 samples/s lr: 2.97e-04 [09/23 02:21:45] lb.utils.events INFO: eta: 5:51:02 iteration: 239599/375342 consumed_samples: 245350400 total_loss: 3.367 time: 0.3433 s/iter data_time: 0.2339 s/iter total_throughput: 2982.60 samples/s lr: 2.97e-04 [09/23 02:22:20] lb.utils.events INFO: eta: 5:50:12 iteration: 239699/375342 consumed_samples: 245452800 total_loss: 3.37 time: 0.3433 s/iter data_time: 0.2302 s/iter total_throughput: 2982.56 samples/s lr: 2.96e-04 [09/23 02:22:56] lb.utils.events INFO: eta: 5:50:05 iteration: 239799/375342 consumed_samples: 245555200 total_loss: 3.369 time: 0.3433 s/iter data_time: 0.2359 s/iter total_throughput: 2982.52 samples/s lr: 2.96e-04 [09/23 02:23:30] lb.utils.events INFO: eta: 5:50:03 iteration: 239899/375342 consumed_samples: 245657600 total_loss: 3.352 time: 0.3433 s/iter data_time: 0.2164 s/iter total_throughput: 2982.50 samples/s lr: 2.95e-04 [09/23 02:24:05] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0239999 [09/23 02:24:06] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/23 02:24:06] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/23 02:24:10] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.1048 s/iter. Inference: 0.1613 s/iter. Eval: 0.0020 s/iter. Total: 0.2682 s/iter. ETA=0:00:09 [09/23 02:24:15] lb.evaluation.evaluator INFO: Inference done 28672/50000. Dataloading: 0.1142 s/iter. Inference: 0.1738 s/iter. Eval: 0.0020 s/iter. Total: 0.2902 s/iter. ETA=0:00:05 [09/23 02:24:21] lb.evaluation.evaluator INFO: Inference done 47104/50000. Dataloading: 0.1214 s/iter. Inference: 0.1680 s/iter. Eval: 0.0021 s/iter. Total: 0.2915 s/iter. ETA=0:00:00 [09/23 02:24:21] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/23 02:24:21] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.603027 (0.000252 s / iter per device, on 8 devices) [09/23 02:24:21] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000148 s / iter per device, on 8 devices) [09/23 02:24:21] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/23 02:24:21] lb.evaluation.utils INFO: copypaste: Acc@1=77.13 [09/23 02:24:21] lb.evaluation.utils INFO: copypaste: Acc@5=93.53 [09/23 02:24:21] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 77.13000, better than last best score 76.82000 @ iteration 234999. [09/23 02:24:21] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/23 02:24:22] lb.utils.events INFO: eta: 5:49:43 iteration: 239999/375342 consumed_samples: 245760000 total_loss: 3.354 time: 0.3433 s/iter data_time: 0.2331 s/iter total_throughput: 2982.48 samples/s lr: 2.95e-04 [09/23 02:24:55] lb.utils.events INFO: eta: 5:51:29 iteration: 240099/375342 consumed_samples: 245862400 total_loss: 3.378 time: 0.3433 s/iter data_time: 0.2223 s/iter total_throughput: 2982.53 samples/s lr: 2.95e-04 [09/23 02:25:30] lb.utils.events INFO: eta: 5:51:25 iteration: 240199/375342 consumed_samples: 245964800 total_loss: 3.36 time: 0.3433 s/iter data_time: 0.2177 s/iter total_throughput: 2982.52 samples/s lr: 2.94e-04 [09/23 02:26:05] lb.utils.events INFO: eta: 5:50:46 iteration: 240299/375342 consumed_samples: 246067200 total_loss: 3.34 time: 0.3433 s/iter data_time: 0.2251 s/iter total_throughput: 2982.49 samples/s lr: 2.94e-04 [09/23 02:26:40] lb.utils.events INFO: eta: 5:51:49 iteration: 240399/375342 consumed_samples: 246169600 total_loss: 3.342 time: 0.3433 s/iter data_time: 0.2221 s/iter total_throughput: 2982.48 samples/s lr: 2.94e-04 [09/23 02:27:15] lb.utils.events INFO: eta: 5:52:13 iteration: 240499/375342 consumed_samples: 246272000 total_loss: 3.347 time: 0.3433 s/iter data_time: 0.2305 s/iter total_throughput: 2982.44 samples/s lr: 2.93e-04 [09/23 02:27:50] lb.utils.events INFO: eta: 5:52:21 iteration: 240599/375342 consumed_samples: 246374400 total_loss: 3.354 time: 0.3433 s/iter data_time: 0.2237 s/iter total_throughput: 2982.43 samples/s lr: 2.93e-04 [09/23 02:28:25] lb.utils.events INFO: eta: 5:52:24 iteration: 240699/375342 consumed_samples: 246476800 total_loss: 3.363 time: 0.3433 s/iter data_time: 0.2438 s/iter total_throughput: 2982.40 samples/s lr: 2.92e-04 [09/23 02:29:00] lb.utils.events INFO: eta: 5:52:24 iteration: 240799/375342 consumed_samples: 246579200 total_loss: 3.349 time: 0.3434 s/iter data_time: 0.2394 s/iter total_throughput: 2982.36 samples/s lr: 2.92e-04 [09/23 02:29:35] lb.utils.events INFO: eta: 5:51:56 iteration: 240899/375342 consumed_samples: 246681600 total_loss: 3.345 time: 0.3434 s/iter data_time: 0.2183 s/iter total_throughput: 2982.33 samples/s lr: 2.92e-04 [09/23 02:30:10] lb.utils.events INFO: eta: 5:50:56 iteration: 240999/375342 consumed_samples: 246784000 total_loss: 3.356 time: 0.3434 s/iter data_time: 0.2198 s/iter total_throughput: 2982.30 samples/s lr: 2.91e-04 [09/23 02:30:45] lb.utils.events INFO: eta: 5:49:53 iteration: 241099/375342 consumed_samples: 246886400 total_loss: 3.367 time: 0.3434 s/iter data_time: 0.2229 s/iter total_throughput: 2982.28 samples/s lr: 2.91e-04 [09/23 02:31:21] lb.utils.events INFO: eta: 5:48:56 iteration: 241199/375342 consumed_samples: 246988800 total_loss: 3.359 time: 0.3434 s/iter data_time: 0.2355 s/iter total_throughput: 2982.23 samples/s lr: 2.91e-04 [09/23 02:31:56] lb.utils.events INFO: eta: 5:48:57 iteration: 241299/375342 consumed_samples: 247091200 total_loss: 3.354 time: 0.3434 s/iter data_time: 0.2278 s/iter total_throughput: 2982.21 samples/s lr: 2.90e-04 [09/23 02:32:31] lb.utils.events INFO: eta: 5:48:44 iteration: 241399/375342 consumed_samples: 247193600 total_loss: 3.356 time: 0.3434 s/iter data_time: 0.2195 s/iter total_throughput: 2982.19 samples/s lr: 2.90e-04 [09/23 02:33:06] lb.utils.events INFO: eta: 5:48:26 iteration: 241499/375342 consumed_samples: 247296000 total_loss: 3.368 time: 0.3434 s/iter data_time: 0.2210 s/iter total_throughput: 2982.17 samples/s lr: 2.89e-04 [09/23 02:33:41] lb.utils.events INFO: eta: 5:47:22 iteration: 241599/375342 consumed_samples: 247398400 total_loss: 3.354 time: 0.3434 s/iter data_time: 0.2285 s/iter total_throughput: 2982.15 samples/s lr: 2.89e-04 [09/23 02:34:16] lb.utils.events INFO: eta: 5:47:23 iteration: 241699/375342 consumed_samples: 247500800 total_loss: 3.341 time: 0.3434 s/iter data_time: 0.2188 s/iter total_throughput: 2982.12 samples/s lr: 2.89e-04 [09/23 02:34:51] lb.utils.events INFO: eta: 5:46:35 iteration: 241799/375342 consumed_samples: 247603200 total_loss: 3.347 time: 0.3434 s/iter data_time: 0.2352 s/iter total_throughput: 2982.08 samples/s lr: 2.88e-04 [09/23 02:35:26] lb.utils.events INFO: eta: 5:45:56 iteration: 241899/375342 consumed_samples: 247705600 total_loss: 3.348 time: 0.3434 s/iter data_time: 0.2131 s/iter total_throughput: 2982.05 samples/s lr: 2.88e-04 [09/23 02:36:02] lb.utils.events INFO: eta: 5:45:49 iteration: 241999/375342 consumed_samples: 247808000 total_loss: 3.346 time: 0.3434 s/iter data_time: 0.2390 s/iter total_throughput: 2982.02 samples/s lr: 2.88e-04 [09/23 02:36:37] lb.utils.events INFO: eta: 5:45:17 iteration: 242099/375342 consumed_samples: 247910400 total_loss: 3.346 time: 0.3434 s/iter data_time: 0.2204 s/iter total_throughput: 2981.99 samples/s lr: 2.87e-04 [09/23 02:37:11] lb.utils.events INFO: eta: 5:45:37 iteration: 242199/375342 consumed_samples: 248012800 total_loss: 3.346 time: 0.3434 s/iter data_time: 0.2151 s/iter total_throughput: 2981.99 samples/s lr: 2.87e-04 [09/23 02:37:46] lb.utils.events INFO: eta: 5:45:02 iteration: 242299/375342 consumed_samples: 248115200 total_loss: 3.362 time: 0.3434 s/iter data_time: 0.2202 s/iter total_throughput: 2981.98 samples/s lr: 2.86e-04 [09/23 02:38:21] lb.utils.events INFO: eta: 5:44:03 iteration: 242399/375342 consumed_samples: 248217600 total_loss: 3.348 time: 0.3434 s/iter data_time: 0.2150 s/iter total_throughput: 2981.96 samples/s lr: 2.86e-04 [09/23 02:38:56] lb.utils.events INFO: eta: 5:43:28 iteration: 242499/375342 consumed_samples: 248320000 total_loss: 3.339 time: 0.3434 s/iter data_time: 0.2303 s/iter total_throughput: 2981.93 samples/s lr: 2.86e-04 [09/23 02:39:31] lb.utils.events INFO: eta: 5:43:27 iteration: 242599/375342 consumed_samples: 248422400 total_loss: 3.337 time: 0.3434 s/iter data_time: 0.2190 s/iter total_throughput: 2981.91 samples/s lr: 2.85e-04 [09/23 02:40:06] lb.utils.events INFO: eta: 5:43:23 iteration: 242699/375342 consumed_samples: 248524800 total_loss: 3.342 time: 0.3434 s/iter data_time: 0.2141 s/iter total_throughput: 2981.90 samples/s lr: 2.85e-04 [09/23 02:40:41] lb.utils.events INFO: eta: 5:43:23 iteration: 242799/375342 consumed_samples: 248627200 total_loss: 3.342 time: 0.3434 s/iter data_time: 0.2188 s/iter total_throughput: 2981.87 samples/s lr: 2.85e-04 [09/23 02:41:15] lb.utils.events INFO: eta: 5:43:32 iteration: 242899/375342 consumed_samples: 248729600 total_loss: 3.358 time: 0.3434 s/iter data_time: 0.2136 s/iter total_throughput: 2981.86 samples/s lr: 2.84e-04 [09/23 02:41:51] lb.utils.events INFO: eta: 5:43:18 iteration: 242999/375342 consumed_samples: 248832000 total_loss: 3.361 time: 0.3434 s/iter data_time: 0.2390 s/iter total_throughput: 2981.83 samples/s lr: 2.84e-04 [09/23 02:42:25] lb.utils.events INFO: eta: 5:42:42 iteration: 243099/375342 consumed_samples: 248934400 total_loss: 3.32 time: 0.3434 s/iter data_time: 0.2145 s/iter total_throughput: 2981.81 samples/s lr: 2.84e-04 [09/23 02:43:01] lb.utils.events INFO: eta: 5:42:12 iteration: 243199/375342 consumed_samples: 249036800 total_loss: 3.32 time: 0.3434 s/iter data_time: 0.2267 s/iter total_throughput: 2981.77 samples/s lr: 2.83e-04 [09/23 02:43:36] lb.utils.events INFO: eta: 5:41:57 iteration: 243299/375342 consumed_samples: 249139200 total_loss: 3.332 time: 0.3434 s/iter data_time: 0.2183 s/iter total_throughput: 2981.76 samples/s lr: 2.83e-04 [09/23 02:44:11] lb.utils.events INFO: eta: 5:41:21 iteration: 243399/375342 consumed_samples: 249241600 total_loss: 3.333 time: 0.3434 s/iter data_time: 0.2166 s/iter total_throughput: 2981.74 samples/s lr: 2.82e-04 [09/23 02:44:46] lb.utils.events INFO: eta: 5:40:51 iteration: 243499/375342 consumed_samples: 249344000 total_loss: 3.351 time: 0.3434 s/iter data_time: 0.2382 s/iter total_throughput: 2981.70 samples/s lr: 2.82e-04 [09/23 02:45:21] lb.utils.events INFO: eta: 5:40:50 iteration: 243599/375342 consumed_samples: 249446400 total_loss: 3.358 time: 0.3434 s/iter data_time: 0.2275 s/iter total_throughput: 2981.67 samples/s lr: 2.82e-04 [09/23 02:45:56] lb.utils.events INFO: eta: 5:40:14 iteration: 243699/375342 consumed_samples: 249548800 total_loss: 3.363 time: 0.3434 s/iter data_time: 0.2316 s/iter total_throughput: 2981.66 samples/s lr: 2.81e-04 [09/23 02:46:31] lb.utils.events INFO: eta: 5:39:45 iteration: 243799/375342 consumed_samples: 249651200 total_loss: 3.361 time: 0.3434 s/iter data_time: 0.2223 s/iter total_throughput: 2981.63 samples/s lr: 2.81e-04 [09/23 02:47:06] lb.utils.events INFO: eta: 5:39:32 iteration: 243899/375342 consumed_samples: 249753600 total_loss: 3.349 time: 0.3434 s/iter data_time: 0.2126 s/iter total_throughput: 2981.60 samples/s lr: 2.81e-04 [09/23 02:47:42] lb.utils.events INFO: eta: 5:39:28 iteration: 243999/375342 consumed_samples: 249856000 total_loss: 3.349 time: 0.3434 s/iter data_time: 0.2212 s/iter total_throughput: 2981.56 samples/s lr: 2.80e-04 [09/23 02:48:17] lb.utils.events INFO: eta: 5:39:34 iteration: 244099/375342 consumed_samples: 249958400 total_loss: 3.336 time: 0.3434 s/iter data_time: 0.2329 s/iter total_throughput: 2981.52 samples/s lr: 2.80e-04 [09/23 02:48:52] lb.utils.events INFO: eta: 5:39:25 iteration: 244199/375342 consumed_samples: 250060800 total_loss: 3.318 time: 0.3435 s/iter data_time: 0.2162 s/iter total_throughput: 2981.51 samples/s lr: 2.79e-04 [09/23 02:49:27] lb.utils.events INFO: eta: 5:38:42 iteration: 244299/375342 consumed_samples: 250163200 total_loss: 3.318 time: 0.3435 s/iter data_time: 0.2276 s/iter total_throughput: 2981.47 samples/s lr: 2.79e-04 [09/23 02:50:02] lb.utils.events INFO: eta: 5:39:08 iteration: 244399/375342 consumed_samples: 250265600 total_loss: 3.318 time: 0.3435 s/iter data_time: 0.2296 s/iter total_throughput: 2981.45 samples/s lr: 2.79e-04 [09/23 02:50:38] lb.utils.events INFO: eta: 5:38:52 iteration: 244499/375342 consumed_samples: 250368000 total_loss: 3.312 time: 0.3435 s/iter data_time: 0.2396 s/iter total_throughput: 2981.41 samples/s lr: 2.78e-04 [09/23 02:51:12] lb.utils.events INFO: eta: 5:38:39 iteration: 244599/375342 consumed_samples: 250470400 total_loss: 3.333 time: 0.3435 s/iter data_time: 0.2246 s/iter total_throughput: 2981.40 samples/s lr: 2.78e-04 [09/23 02:51:47] lb.utils.events INFO: eta: 5:38:16 iteration: 244699/375342 consumed_samples: 250572800 total_loss: 3.355 time: 0.3435 s/iter data_time: 0.2179 s/iter total_throughput: 2981.38 samples/s lr: 2.78e-04 [09/23 02:52:23] lb.utils.events INFO: eta: 5:37:58 iteration: 244799/375342 consumed_samples: 250675200 total_loss: 3.347 time: 0.3435 s/iter data_time: 0.2172 s/iter total_throughput: 2981.33 samples/s lr: 2.77e-04 [09/23 02:52:58] lb.utils.events INFO: eta: 5:37:38 iteration: 244899/375342 consumed_samples: 250777600 total_loss: 3.348 time: 0.3435 s/iter data_time: 0.2176 s/iter total_throughput: 2981.31 samples/s lr: 2.77e-04 [09/23 02:53:33] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0244999 [09/23 02:53:34] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/23 02:53:34] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/23 02:53:38] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0826 s/iter. Inference: 0.1671 s/iter. Eval: 0.0020 s/iter. Total: 0.2518 s/iter. ETA=0:00:09 [09/23 02:53:43] lb.evaluation.evaluator INFO: Inference done 28672/50000. Dataloading: 0.1183 s/iter. Inference: 0.1633 s/iter. Eval: 0.0020 s/iter. Total: 0.2836 s/iter. ETA=0:00:05 [09/23 02:53:48] lb.evaluation.evaluator INFO: Inference done 46080/50000. Dataloading: 0.1274 s/iter. Inference: 0.1628 s/iter. Eval: 0.0021 s/iter. Total: 0.2924 s/iter. ETA=0:00:00 [09/23 02:53:49] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/23 02:53:49] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.638263 (0.000253 s / iter per device, on 8 devices) [09/23 02:53:49] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000143 s / iter per device, on 8 devices) [09/23 02:53:49] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/23 02:53:49] lb.evaluation.utils INFO: copypaste: Acc@1=77.21000000000001 [09/23 02:53:49] lb.evaluation.utils INFO: copypaste: Acc@5=93.446 [09/23 02:53:49] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 77.21000, better than last best score 77.13000 @ iteration 239999. [09/23 02:53:49] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/23 02:53:50] lb.utils.events INFO: eta: 5:36:57 iteration: 244999/375342 consumed_samples: 250880000 total_loss: 3.334 time: 0.3435 s/iter data_time: 0.2204 s/iter total_throughput: 2981.28 samples/s lr: 2.76e-04 [09/23 02:54:23] lb.utils.events INFO: eta: 5:37:39 iteration: 245099/375342 consumed_samples: 250982400 total_loss: 3.348 time: 0.3435 s/iter data_time: 0.2248 s/iter total_throughput: 2981.32 samples/s lr: 2.76e-04 [09/23 02:54:58] lb.utils.events INFO: eta: 5:36:56 iteration: 245199/375342 consumed_samples: 251084800 total_loss: 3.355 time: 0.3435 s/iter data_time: 0.2533 s/iter total_throughput: 2981.30 samples/s lr: 2.76e-04 [09/23 02:55:33] lb.utils.events INFO: eta: 5:36:40 iteration: 245299/375342 consumed_samples: 251187200 total_loss: 3.351 time: 0.3435 s/iter data_time: 0.2271 s/iter total_throughput: 2981.27 samples/s lr: 2.75e-04 [09/23 02:56:09] lb.utils.events INFO: eta: 5:35:13 iteration: 245399/375342 consumed_samples: 251289600 total_loss: 3.323 time: 0.3435 s/iter data_time: 0.2351 s/iter total_throughput: 2981.24 samples/s lr: 2.75e-04 [09/23 02:56:44] lb.utils.events INFO: eta: 5:35:01 iteration: 245499/375342 consumed_samples: 251392000 total_loss: 3.323 time: 0.3435 s/iter data_time: 0.2230 s/iter total_throughput: 2981.21 samples/s lr: 2.75e-04 [09/23 02:57:19] lb.utils.events INFO: eta: 5:34:42 iteration: 245599/375342 consumed_samples: 251494400 total_loss: 3.317 time: 0.3435 s/iter data_time: 0.2196 s/iter total_throughput: 2981.19 samples/s lr: 2.74e-04 [09/23 02:57:54] lb.utils.events INFO: eta: 5:34:20 iteration: 245699/375342 consumed_samples: 251596800 total_loss: 3.318 time: 0.3435 s/iter data_time: 0.2232 s/iter total_throughput: 2981.17 samples/s lr: 2.74e-04 [09/23 02:58:28] lb.utils.events INFO: eta: 5:34:40 iteration: 245799/375342 consumed_samples: 251699200 total_loss: 3.323 time: 0.3435 s/iter data_time: 0.2237 s/iter total_throughput: 2981.16 samples/s lr: 2.74e-04 [09/23 02:59:03] lb.utils.events INFO: eta: 5:34:43 iteration: 245899/375342 consumed_samples: 251801600 total_loss: 3.311 time: 0.3435 s/iter data_time: 0.2259 s/iter total_throughput: 2981.14 samples/s lr: 2.73e-04 [09/23 02:59:38] lb.utils.events INFO: eta: 5:34:23 iteration: 245999/375342 consumed_samples: 251904000 total_loss: 3.343 time: 0.3435 s/iter data_time: 0.2229 s/iter total_throughput: 2981.11 samples/s lr: 2.73e-04 [09/23 03:00:14] lb.utils.events INFO: eta: 5:33:18 iteration: 246099/375342 consumed_samples: 252006400 total_loss: 3.356 time: 0.3435 s/iter data_time: 0.2180 s/iter total_throughput: 2981.08 samples/s lr: 2.72e-04 [09/23 03:00:48] lb.utils.events INFO: eta: 5:33:44 iteration: 246199/375342 consumed_samples: 252108800 total_loss: 3.322 time: 0.3435 s/iter data_time: 0.2090 s/iter total_throughput: 2981.08 samples/s lr: 2.72e-04 [09/23 03:01:23] lb.utils.events INFO: eta: 5:33:40 iteration: 246299/375342 consumed_samples: 252211200 total_loss: 3.321 time: 0.3435 s/iter data_time: 0.2173 s/iter total_throughput: 2981.07 samples/s lr: 2.72e-04 [09/23 03:01:57] lb.utils.events INFO: eta: 5:34:54 iteration: 246399/375342 consumed_samples: 252313600 total_loss: 3.336 time: 0.3435 s/iter data_time: 0.2277 s/iter total_throughput: 2981.06 samples/s lr: 2.71e-04 [09/23 03:02:32] lb.utils.events INFO: eta: 5:34:38 iteration: 246499/375342 consumed_samples: 252416000 total_loss: 3.332 time: 0.3435 s/iter data_time: 0.2163 s/iter total_throughput: 2981.05 samples/s lr: 2.71e-04 [09/23 03:03:07] lb.utils.events INFO: eta: 5:34:23 iteration: 246599/375342 consumed_samples: 252518400 total_loss: 3.318 time: 0.3435 s/iter data_time: 0.2300 s/iter total_throughput: 2981.04 samples/s lr: 2.71e-04 [09/23 03:03:41] lb.utils.events INFO: eta: 5:34:05 iteration: 246699/375342 consumed_samples: 252620800 total_loss: 3.312 time: 0.3435 s/iter data_time: 0.2201 s/iter total_throughput: 2981.02 samples/s lr: 2.70e-04 [09/23 03:04:16] lb.utils.events INFO: eta: 5:33:47 iteration: 246799/375342 consumed_samples: 252723200 total_loss: 3.315 time: 0.3435 s/iter data_time: 0.2156 s/iter total_throughput: 2981.01 samples/s lr: 2.70e-04 [09/23 03:04:51] lb.utils.events INFO: eta: 5:33:37 iteration: 246899/375342 consumed_samples: 252825600 total_loss: 3.325 time: 0.3435 s/iter data_time: 0.2316 s/iter total_throughput: 2980.99 samples/s lr: 2.70e-04 [09/23 03:05:26] lb.utils.events INFO: eta: 5:33:22 iteration: 246999/375342 consumed_samples: 252928000 total_loss: 3.328 time: 0.3435 s/iter data_time: 0.2278 s/iter total_throughput: 2980.98 samples/s lr: 2.69e-04 [09/23 03:06:01] lb.utils.events INFO: eta: 5:33:05 iteration: 247099/375342 consumed_samples: 253030400 total_loss: 3.343 time: 0.3435 s/iter data_time: 0.2287 s/iter total_throughput: 2980.96 samples/s lr: 2.69e-04 [09/23 03:06:35] lb.utils.events INFO: eta: 5:32:36 iteration: 247199/375342 consumed_samples: 253132800 total_loss: 3.35 time: 0.3435 s/iter data_time: 0.2122 s/iter total_throughput: 2980.96 samples/s lr: 2.68e-04 [09/23 03:07:10] lb.utils.events INFO: eta: 5:32:29 iteration: 247299/375342 consumed_samples: 253235200 total_loss: 3.349 time: 0.3435 s/iter data_time: 0.2304 s/iter total_throughput: 2980.95 samples/s lr: 2.68e-04 [09/23 03:07:44] lb.utils.events INFO: eta: 5:31:17 iteration: 247399/375342 consumed_samples: 253337600 total_loss: 3.336 time: 0.3435 s/iter data_time: 0.2176 s/iter total_throughput: 2980.94 samples/s lr: 2.68e-04 [09/23 03:08:19] lb.utils.events INFO: eta: 5:31:41 iteration: 247499/375342 consumed_samples: 253440000 total_loss: 3.337 time: 0.3435 s/iter data_time: 0.2276 s/iter total_throughput: 2980.93 samples/s lr: 2.67e-04 [09/23 03:08:54] lb.utils.events INFO: eta: 5:31:31 iteration: 247599/375342 consumed_samples: 253542400 total_loss: 3.338 time: 0.3435 s/iter data_time: 0.2257 s/iter total_throughput: 2980.91 samples/s lr: 2.67e-04 [09/23 03:09:28] lb.utils.events INFO: eta: 5:31:31 iteration: 247699/375342 consumed_samples: 253644800 total_loss: 3.319 time: 0.3435 s/iter data_time: 0.2152 s/iter total_throughput: 2980.91 samples/s lr: 2.67e-04 [09/23 03:10:02] lb.utils.events INFO: eta: 5:30:58 iteration: 247799/375342 consumed_samples: 253747200 total_loss: 3.291 time: 0.3435 s/iter data_time: 0.2117 s/iter total_throughput: 2980.91 samples/s lr: 2.66e-04 [09/23 03:10:37] lb.utils.events INFO: eta: 5:30:25 iteration: 247899/375342 consumed_samples: 253849600 total_loss: 3.328 time: 0.3435 s/iter data_time: 0.2134 s/iter total_throughput: 2980.91 samples/s lr: 2.66e-04 [09/23 03:11:12] lb.utils.events INFO: eta: 5:30:17 iteration: 247999/375342 consumed_samples: 253952000 total_loss: 3.332 time: 0.3435 s/iter data_time: 0.2227 s/iter total_throughput: 2980.90 samples/s lr: 2.66e-04 [09/23 03:11:47] lb.utils.events INFO: eta: 5:29:26 iteration: 248099/375342 consumed_samples: 254054400 total_loss: 3.294 time: 0.3435 s/iter data_time: 0.2352 s/iter total_throughput: 2980.88 samples/s lr: 2.65e-04 [09/23 03:12:21] lb.utils.events INFO: eta: 5:29:28 iteration: 248199/375342 consumed_samples: 254156800 total_loss: 3.318 time: 0.3435 s/iter data_time: 0.2290 s/iter total_throughput: 2980.87 samples/s lr: 2.65e-04 [09/23 03:12:56] lb.utils.events INFO: eta: 5:28:38 iteration: 248299/375342 consumed_samples: 254259200 total_loss: 3.344 time: 0.3435 s/iter data_time: 0.2318 s/iter total_throughput: 2980.84 samples/s lr: 2.64e-04 [09/23 03:13:31] lb.utils.events INFO: eta: 5:28:35 iteration: 248399/375342 consumed_samples: 254361600 total_loss: 3.34 time: 0.3435 s/iter data_time: 0.2114 s/iter total_throughput: 2980.84 samples/s lr: 2.64e-04 [09/23 03:14:05] lb.utils.events INFO: eta: 5:28:16 iteration: 248499/375342 consumed_samples: 254464000 total_loss: 3.312 time: 0.3435 s/iter data_time: 0.2259 s/iter total_throughput: 2980.83 samples/s lr: 2.64e-04 [09/23 03:14:40] lb.utils.events INFO: eta: 5:27:59 iteration: 248599/375342 consumed_samples: 254566400 total_loss: 3.309 time: 0.3435 s/iter data_time: 0.2104 s/iter total_throughput: 2980.83 samples/s lr: 2.63e-04 [09/23 03:15:15] lb.utils.events INFO: eta: 5:27:24 iteration: 248699/375342 consumed_samples: 254668800 total_loss: 3.328 time: 0.3435 s/iter data_time: 0.2238 s/iter total_throughput: 2980.81 samples/s lr: 2.63e-04 [09/23 03:15:49] lb.utils.events INFO: eta: 5:27:03 iteration: 248799/375342 consumed_samples: 254771200 total_loss: 3.328 time: 0.3435 s/iter data_time: 0.2251 s/iter total_throughput: 2980.80 samples/s lr: 2.63e-04 [09/23 03:16:24] lb.utils.events INFO: eta: 5:27:01 iteration: 248899/375342 consumed_samples: 254873600 total_loss: 3.32 time: 0.3435 s/iter data_time: 0.2208 s/iter total_throughput: 2980.79 samples/s lr: 2.62e-04 [09/23 03:16:59] lb.utils.events INFO: eta: 5:26:25 iteration: 248999/375342 consumed_samples: 254976000 total_loss: 3.331 time: 0.3435 s/iter data_time: 0.2215 s/iter total_throughput: 2980.78 samples/s lr: 2.62e-04 [09/23 03:17:33] lb.utils.events INFO: eta: 5:27:03 iteration: 249099/375342 consumed_samples: 255078400 total_loss: 3.34 time: 0.3435 s/iter data_time: 0.2202 s/iter total_throughput: 2980.77 samples/s lr: 2.62e-04 [09/23 03:18:08] lb.utils.events INFO: eta: 5:26:58 iteration: 249199/375342 consumed_samples: 255180800 total_loss: 3.33 time: 0.3435 s/iter data_time: 0.2092 s/iter total_throughput: 2980.76 samples/s lr: 2.61e-04 [09/23 03:18:43] lb.utils.events INFO: eta: 5:26:32 iteration: 249299/375342 consumed_samples: 255283200 total_loss: 3.321 time: 0.3435 s/iter data_time: 0.2130 s/iter total_throughput: 2980.75 samples/s lr: 2.61e-04 [09/23 03:19:17] lb.utils.events INFO: eta: 5:26:29 iteration: 249399/375342 consumed_samples: 255385600 total_loss: 3.324 time: 0.3435 s/iter data_time: 0.2120 s/iter total_throughput: 2980.74 samples/s lr: 2.60e-04 [09/23 03:19:52] lb.utils.events INFO: eta: 5:25:53 iteration: 249499/375342 consumed_samples: 255488000 total_loss: 3.32 time: 0.3435 s/iter data_time: 0.2147 s/iter total_throughput: 2980.74 samples/s lr: 2.60e-04 [09/23 03:20:26] lb.utils.events INFO: eta: 5:25:37 iteration: 249599/375342 consumed_samples: 255590400 total_loss: 3.312 time: 0.3435 s/iter data_time: 0.2163 s/iter total_throughput: 2980.73 samples/s lr: 2.60e-04 [09/23 03:21:00] lb.utils.events INFO: eta: 5:25:30 iteration: 249699/375342 consumed_samples: 255692800 total_loss: 3.32 time: 0.3435 s/iter data_time: 0.2093 s/iter total_throughput: 2980.74 samples/s lr: 2.59e-04 [09/23 03:21:35] lb.utils.events INFO: eta: 5:25:15 iteration: 249799/375342 consumed_samples: 255795200 total_loss: 3.325 time: 0.3435 s/iter data_time: 0.2252 s/iter total_throughput: 2980.73 samples/s lr: 2.59e-04 [09/23 03:22:10] lb.utils.events INFO: eta: 5:24:51 iteration: 249899/375342 consumed_samples: 255897600 total_loss: 3.333 time: 0.3435 s/iter data_time: 0.2225 s/iter total_throughput: 2980.72 samples/s lr: 2.59e-04 [09/23 03:22:45] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0249999 [09/23 03:22:45] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/23 03:22:45] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/23 03:22:50] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.1018 s/iter. Inference: 0.1634 s/iter. Eval: 0.0020 s/iter. Total: 0.2672 s/iter. ETA=0:00:09 [09/23 03:22:55] lb.evaluation.evaluator INFO: Inference done 28672/50000. Dataloading: 0.1162 s/iter. Inference: 0.1694 s/iter. Eval: 0.0021 s/iter. Total: 0.2877 s/iter. ETA=0:00:05 [09/23 03:23:00] lb.evaluation.evaluator INFO: Inference done 46080/50000. Dataloading: 0.1221 s/iter. Inference: 0.1672 s/iter. Eval: 0.0021 s/iter. Total: 0.2915 s/iter. ETA=0:00:00 [09/23 03:23:01] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/23 03:23:01] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.614043 (0.000252 s / iter per device, on 8 devices) [09/23 03:23:01] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000148 s / iter per device, on 8 devices) [09/23 03:23:01] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/23 03:23:01] lb.evaluation.utils INFO: copypaste: Acc@1=77.622 [09/23 03:23:01] lb.evaluation.utils INFO: copypaste: Acc@5=93.756 [09/23 03:23:01] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 77.62200, better than last best score 77.21000 @ iteration 244999. [09/23 03:23:01] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/23 03:23:01] lb.utils.events INFO: eta: 5:24:28 iteration: 249999/375342 consumed_samples: 256000000 total_loss: 3.316 time: 0.3435 s/iter data_time: 0.2350 s/iter total_throughput: 2980.69 samples/s lr: 2.58e-04 [09/23 03:23:34] lb.utils.events INFO: eta: 5:25:51 iteration: 250099/375342 consumed_samples: 256102400 total_loss: 3.312 time: 0.3435 s/iter data_time: 0.2340 s/iter total_throughput: 2980.75 samples/s lr: 2.58e-04 [09/23 03:24:09] lb.utils.events INFO: eta: 5:25:08 iteration: 250199/375342 consumed_samples: 256204800 total_loss: 3.32 time: 0.3435 s/iter data_time: 0.2156 s/iter total_throughput: 2980.74 samples/s lr: 2.58e-04 [09/23 03:24:43] lb.utils.events INFO: eta: 5:25:37 iteration: 250299/375342 consumed_samples: 256307200 total_loss: 3.316 time: 0.3435 s/iter data_time: 0.2084 s/iter total_throughput: 2980.75 samples/s lr: 2.57e-04 [09/23 03:25:17] lb.utils.events INFO: eta: 5:24:51 iteration: 250399/375342 consumed_samples: 256409600 total_loss: 3.303 time: 0.3435 s/iter data_time: 0.2149 s/iter total_throughput: 2980.74 samples/s lr: 2.57e-04 [09/23 03:25:52] lb.utils.events INFO: eta: 5:24:47 iteration: 250499/375342 consumed_samples: 256512000 total_loss: 3.331 time: 0.3435 s/iter data_time: 0.2127 s/iter total_throughput: 2980.74 samples/s lr: 2.57e-04 [09/23 03:26:27] lb.utils.events INFO: eta: 5:24:47 iteration: 250599/375342 consumed_samples: 256614400 total_loss: 3.313 time: 0.3435 s/iter data_time: 0.2268 s/iter total_throughput: 2980.71 samples/s lr: 2.56e-04 [09/23 03:27:02] lb.utils.events INFO: eta: 5:24:15 iteration: 250699/375342 consumed_samples: 256716800 total_loss: 3.299 time: 0.3435 s/iter data_time: 0.2382 s/iter total_throughput: 2980.70 samples/s lr: 2.56e-04 [09/23 03:27:36] lb.utils.events INFO: eta: 5:24:00 iteration: 250799/375342 consumed_samples: 256819200 total_loss: 3.305 time: 0.3435 s/iter data_time: 0.2222 s/iter total_throughput: 2980.70 samples/s lr: 2.55e-04 [09/23 03:28:11] lb.utils.events INFO: eta: 5:23:38 iteration: 250899/375342 consumed_samples: 256921600 total_loss: 3.301 time: 0.3435 s/iter data_time: 0.2211 s/iter total_throughput: 2980.69 samples/s lr: 2.55e-04 [09/23 03:28:46] lb.utils.events INFO: eta: 5:23:51 iteration: 250999/375342 consumed_samples: 257024000 total_loss: 3.309 time: 0.3435 s/iter data_time: 0.2265 s/iter total_throughput: 2980.68 samples/s lr: 2.55e-04 [09/23 03:29:20] lb.utils.events INFO: eta: 5:21:57 iteration: 251099/375342 consumed_samples: 257126400 total_loss: 3.324 time: 0.3435 s/iter data_time: 0.2175 s/iter total_throughput: 2980.67 samples/s lr: 2.54e-04 [09/23 03:29:55] lb.utils.events INFO: eta: 5:21:33 iteration: 251199/375342 consumed_samples: 257228800 total_loss: 3.319 time: 0.3435 s/iter data_time: 0.2212 s/iter total_throughput: 2980.67 samples/s lr: 2.54e-04 [09/23 03:30:29] lb.utils.events INFO: eta: 5:21:07 iteration: 251299/375342 consumed_samples: 257331200 total_loss: 3.308 time: 0.3435 s/iter data_time: 0.2248 s/iter total_throughput: 2980.65 samples/s lr: 2.54e-04 [09/23 03:31:04] lb.utils.events INFO: eta: 5:21:03 iteration: 251399/375342 consumed_samples: 257433600 total_loss: 3.322 time: 0.3435 s/iter data_time: 0.2199 s/iter total_throughput: 2980.65 samples/s lr: 2.53e-04 [09/23 03:31:38] lb.utils.events INFO: eta: 5:20:19 iteration: 251499/375342 consumed_samples: 257536000 total_loss: 3.326 time: 0.3435 s/iter data_time: 0.2160 s/iter total_throughput: 2980.65 samples/s lr: 2.53e-04 [09/23 03:32:13] lb.utils.events INFO: eta: 5:19:49 iteration: 251599/375342 consumed_samples: 257638400 total_loss: 3.307 time: 0.3435 s/iter data_time: 0.2137 s/iter total_throughput: 2980.65 samples/s lr: 2.53e-04 [09/23 03:32:47] lb.utils.events INFO: eta: 5:20:13 iteration: 251699/375342 consumed_samples: 257740800 total_loss: 3.315 time: 0.3436 s/iter data_time: 0.2132 s/iter total_throughput: 2980.64 samples/s lr: 2.52e-04 [09/23 03:33:21] lb.utils.events INFO: eta: 5:19:35 iteration: 251799/375342 consumed_samples: 257843200 total_loss: 3.318 time: 0.3435 s/iter data_time: 0.2125 s/iter total_throughput: 2980.65 samples/s lr: 2.52e-04 [09/23 03:33:55] lb.utils.events INFO: eta: 5:20:06 iteration: 251899/375342 consumed_samples: 257945600 total_loss: 3.317 time: 0.3435 s/iter data_time: 0.2070 s/iter total_throughput: 2980.66 samples/s lr: 2.52e-04 [09/23 03:34:29] lb.utils.events INFO: eta: 5:19:54 iteration: 251999/375342 consumed_samples: 258048000 total_loss: 3.311 time: 0.3435 s/iter data_time: 0.2116 s/iter total_throughput: 2980.67 samples/s lr: 2.51e-04 [09/23 03:35:04] lb.utils.events INFO: eta: 5:19:35 iteration: 252099/375342 consumed_samples: 258150400 total_loss: 3.297 time: 0.3435 s/iter data_time: 0.2364 s/iter total_throughput: 2980.65 samples/s lr: 2.51e-04 [09/23 03:35:39] lb.utils.events INFO: eta: 5:19:34 iteration: 252199/375342 consumed_samples: 258252800 total_loss: 3.286 time: 0.3436 s/iter data_time: 0.2171 s/iter total_throughput: 2980.63 samples/s lr: 2.50e-04 [09/23 03:36:14] lb.utils.events INFO: eta: 5:19:19 iteration: 252299/375342 consumed_samples: 258355200 total_loss: 3.308 time: 0.3436 s/iter data_time: 0.2375 s/iter total_throughput: 2980.61 samples/s lr: 2.50e-04 [09/23 03:36:49] lb.utils.events INFO: eta: 5:19:16 iteration: 252399/375342 consumed_samples: 258457600 total_loss: 3.317 time: 0.3436 s/iter data_time: 0.2384 s/iter total_throughput: 2980.58 samples/s lr: 2.50e-04 [09/23 03:37:24] lb.utils.events INFO: eta: 5:19:10 iteration: 252499/375342 consumed_samples: 258560000 total_loss: 3.301 time: 0.3436 s/iter data_time: 0.2193 s/iter total_throughput: 2980.56 samples/s lr: 2.49e-04 [09/23 03:38:00] lb.utils.events INFO: eta: 5:18:52 iteration: 252599/375342 consumed_samples: 258662400 total_loss: 3.282 time: 0.3436 s/iter data_time: 0.2220 s/iter total_throughput: 2980.53 samples/s lr: 2.49e-04 [09/23 03:38:35] lb.utils.events INFO: eta: 5:18:02 iteration: 252699/375342 consumed_samples: 258764800 total_loss: 3.309 time: 0.3436 s/iter data_time: 0.2435 s/iter total_throughput: 2980.49 samples/s lr: 2.49e-04 [09/23 03:39:11] lb.utils.events INFO: eta: 5:17:02 iteration: 252799/375342 consumed_samples: 258867200 total_loss: 3.312 time: 0.3436 s/iter data_time: 0.2386 s/iter total_throughput: 2980.44 samples/s lr: 2.48e-04 [09/23 03:39:46] lb.utils.events INFO: eta: 5:16:30 iteration: 252899/375342 consumed_samples: 258969600 total_loss: 3.302 time: 0.3436 s/iter data_time: 0.2306 s/iter total_throughput: 2980.42 samples/s lr: 2.48e-04 [09/23 03:40:21] lb.utils.events INFO: eta: 5:16:15 iteration: 252999/375342 consumed_samples: 259072000 total_loss: 3.297 time: 0.3436 s/iter data_time: 0.2140 s/iter total_throughput: 2980.40 samples/s lr: 2.48e-04 [09/23 03:40:56] lb.utils.events INFO: eta: 5:16:10 iteration: 253099/375342 consumed_samples: 259174400 total_loss: 3.305 time: 0.3436 s/iter data_time: 0.2266 s/iter total_throughput: 2980.38 samples/s lr: 2.47e-04 [09/23 03:41:31] lb.utils.events INFO: eta: 5:15:51 iteration: 253199/375342 consumed_samples: 259276800 total_loss: 3.316 time: 0.3436 s/iter data_time: 0.2199 s/iter total_throughput: 2980.36 samples/s lr: 2.47e-04 [09/23 03:42:05] lb.utils.events INFO: eta: 5:15:48 iteration: 253299/375342 consumed_samples: 259379200 total_loss: 3.302 time: 0.3436 s/iter data_time: 0.2154 s/iter total_throughput: 2980.35 samples/s lr: 2.47e-04 [09/23 03:42:41] lb.utils.events INFO: eta: 5:15:51 iteration: 253399/375342 consumed_samples: 259481600 total_loss: 3.293 time: 0.3436 s/iter data_time: 0.2196 s/iter total_throughput: 2980.33 samples/s lr: 2.46e-04 [09/23 03:43:16] lb.utils.events INFO: eta: 5:14:46 iteration: 253499/375342 consumed_samples: 259584000 total_loss: 3.303 time: 0.3436 s/iter data_time: 0.2230 s/iter total_throughput: 2980.29 samples/s lr: 2.46e-04 [09/23 03:43:51] lb.utils.events INFO: eta: 5:14:31 iteration: 253599/375342 consumed_samples: 259686400 total_loss: 3.297 time: 0.3436 s/iter data_time: 0.2305 s/iter total_throughput: 2980.27 samples/s lr: 2.46e-04 [09/23 03:44:26] lb.utils.events INFO: eta: 5:14:39 iteration: 253699/375342 consumed_samples: 259788800 total_loss: 3.287 time: 0.3436 s/iter data_time: 0.2247 s/iter total_throughput: 2980.25 samples/s lr: 2.45e-04 [09/23 03:45:01] lb.utils.events INFO: eta: 5:15:07 iteration: 253799/375342 consumed_samples: 259891200 total_loss: 3.292 time: 0.3436 s/iter data_time: 0.2354 s/iter total_throughput: 2980.22 samples/s lr: 2.45e-04 [09/23 03:45:37] lb.utils.events INFO: eta: 5:13:46 iteration: 253899/375342 consumed_samples: 259993600 total_loss: 3.291 time: 0.3436 s/iter data_time: 0.2243 s/iter total_throughput: 2980.18 samples/s lr: 2.44e-04 [09/23 03:46:12] lb.utils.events INFO: eta: 5:14:31 iteration: 253999/375342 consumed_samples: 260096000 total_loss: 3.292 time: 0.3436 s/iter data_time: 0.2338 s/iter total_throughput: 2980.15 samples/s lr: 2.44e-04 [09/23 03:46:48] lb.utils.events INFO: eta: 5:13:39 iteration: 254099/375342 consumed_samples: 260198400 total_loss: 3.285 time: 0.3436 s/iter data_time: 0.2342 s/iter total_throughput: 2980.10 samples/s lr: 2.44e-04 [09/23 03:47:23] lb.utils.events INFO: eta: 5:13:23 iteration: 254199/375342 consumed_samples: 260300800 total_loss: 3.286 time: 0.3436 s/iter data_time: 0.2204 s/iter total_throughput: 2980.07 samples/s lr: 2.43e-04 [09/23 03:47:58] lb.utils.events INFO: eta: 5:12:43 iteration: 254299/375342 consumed_samples: 260403200 total_loss: 3.281 time: 0.3436 s/iter data_time: 0.2165 s/iter total_throughput: 2980.06 samples/s lr: 2.43e-04 [09/23 03:48:33] lb.utils.events INFO: eta: 5:12:18 iteration: 254399/375342 consumed_samples: 260505600 total_loss: 3.273 time: 0.3436 s/iter data_time: 0.2238 s/iter total_throughput: 2980.03 samples/s lr: 2.43e-04 [09/23 03:49:08] lb.utils.events INFO: eta: 5:12:38 iteration: 254499/375342 consumed_samples: 260608000 total_loss: 3.297 time: 0.3436 s/iter data_time: 0.2148 s/iter total_throughput: 2980.02 samples/s lr: 2.42e-04 [09/23 03:49:42] lb.utils.events INFO: eta: 5:12:25 iteration: 254599/375342 consumed_samples: 260710400 total_loss: 3.274 time: 0.3436 s/iter data_time: 0.2259 s/iter total_throughput: 2980.00 samples/s lr: 2.42e-04 [09/23 03:50:17] lb.utils.events INFO: eta: 5:12:33 iteration: 254699/375342 consumed_samples: 260812800 total_loss: 3.287 time: 0.3436 s/iter data_time: 0.2216 s/iter total_throughput: 2979.99 samples/s lr: 2.42e-04 [09/23 03:50:53] lb.utils.events INFO: eta: 5:12:12 iteration: 254799/375342 consumed_samples: 260915200 total_loss: 3.308 time: 0.3436 s/iter data_time: 0.2319 s/iter total_throughput: 2979.95 samples/s lr: 2.41e-04 [09/23 03:51:27] lb.utils.events INFO: eta: 5:12:20 iteration: 254899/375342 consumed_samples: 261017600 total_loss: 3.29 time: 0.3436 s/iter data_time: 0.2206 s/iter total_throughput: 2979.95 samples/s lr: 2.41e-04 [09/23 03:52:03] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0254999 [09/23 03:52:03] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/23 03:52:03] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/23 03:52:08] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.1086 s/iter. Inference: 0.1607 s/iter. Eval: 0.0022 s/iter. Total: 0.2716 s/iter. ETA=0:00:10 [09/23 03:52:13] lb.evaluation.evaluator INFO: Inference done 28672/50000. Dataloading: 0.1195 s/iter. Inference: 0.1687 s/iter. Eval: 0.0021 s/iter. Total: 0.2905 s/iter. ETA=0:00:05 [09/23 03:52:18] lb.evaluation.evaluator INFO: Inference done 47104/50000. Dataloading: 0.1248 s/iter. Inference: 0.1677 s/iter. Eval: 0.0021 s/iter. Total: 0.2947 s/iter. ETA=0:00:00 [09/23 03:52:19] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/23 03:52:19] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.590637 (0.000252 s / iter per device, on 8 devices) [09/23 03:52:19] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000147 s / iter per device, on 8 devices) [09/23 03:52:19] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/23 03:52:19] lb.evaluation.utils INFO: copypaste: Acc@1=77.45 [09/23 03:52:19] lb.evaluation.utils INFO: copypaste: Acc@5=93.732 [09/23 03:52:19] lb.engine.hooks INFO: Not saving as latest eval score for Acc@1 is 77.45000, not better than best score 77.62200 @ iteration 249999. [09/23 03:52:19] lb.utils.events INFO: eta: 5:11:27 iteration: 254999/375342 consumed_samples: 261120000 total_loss: 3.293 time: 0.3436 s/iter data_time: 0.2375 s/iter total_throughput: 2979.91 samples/s lr: 2.41e-04 [09/23 03:52:52] lb.utils.events INFO: eta: 5:12:00 iteration: 255099/375342 consumed_samples: 261222400 total_loss: 3.299 time: 0.3436 s/iter data_time: 0.2275 s/iter total_throughput: 2979.95 samples/s lr: 2.40e-04 [09/23 03:53:27] lb.utils.events INFO: eta: 5:11:39 iteration: 255199/375342 consumed_samples: 261324800 total_loss: 3.309 time: 0.3436 s/iter data_time: 0.2324 s/iter total_throughput: 2979.91 samples/s lr: 2.40e-04 [09/23 03:54:03] lb.utils.events INFO: eta: 5:11:06 iteration: 255299/375342 consumed_samples: 261427200 total_loss: 3.303 time: 0.3436 s/iter data_time: 0.2184 s/iter total_throughput: 2979.87 samples/s lr: 2.40e-04 [09/23 03:54:38] lb.utils.events INFO: eta: 5:10:15 iteration: 255399/375342 consumed_samples: 261529600 total_loss: 3.312 time: 0.3436 s/iter data_time: 0.2363 s/iter total_throughput: 2979.84 samples/s lr: 2.39e-04 [09/23 03:55:13] lb.utils.events INFO: eta: 5:10:23 iteration: 255499/375342 consumed_samples: 261632000 total_loss: 3.301 time: 0.3436 s/iter data_time: 0.2148 s/iter total_throughput: 2979.82 samples/s lr: 2.39e-04 [09/23 03:55:48] lb.utils.events INFO: eta: 5:10:21 iteration: 255599/375342 consumed_samples: 261734400 total_loss: 3.285 time: 0.3436 s/iter data_time: 0.2242 s/iter total_throughput: 2979.81 samples/s lr: 2.38e-04 [09/23 03:56:23] lb.utils.events INFO: eta: 5:10:16 iteration: 255699/375342 consumed_samples: 261836800 total_loss: 3.296 time: 0.3436 s/iter data_time: 0.2265 s/iter total_throughput: 2979.79 samples/s lr: 2.38e-04 [09/23 03:56:58] lb.utils.events INFO: eta: 5:10:30 iteration: 255799/375342 consumed_samples: 261939200 total_loss: 3.292 time: 0.3437 s/iter data_time: 0.2277 s/iter total_throughput: 2979.77 samples/s lr: 2.38e-04 [09/23 03:57:32] lb.utils.events INFO: eta: 5:10:18 iteration: 255899/375342 consumed_samples: 262041600 total_loss: 3.291 time: 0.3437 s/iter data_time: 0.2274 s/iter total_throughput: 2979.76 samples/s lr: 2.37e-04 [09/23 03:58:07] lb.utils.events INFO: eta: 5:10:02 iteration: 255999/375342 consumed_samples: 262144000 total_loss: 3.293 time: 0.3437 s/iter data_time: 0.2326 s/iter total_throughput: 2979.74 samples/s lr: 2.37e-04 [09/23 03:58:42] lb.utils.events INFO: eta: 5:08:36 iteration: 256099/375342 consumed_samples: 262246400 total_loss: 3.301 time: 0.3437 s/iter data_time: 0.2175 s/iter total_throughput: 2979.72 samples/s lr: 2.37e-04 [09/23 03:59:17] lb.utils.events INFO: eta: 5:08:24 iteration: 256199/375342 consumed_samples: 262348800 total_loss: 3.297 time: 0.3437 s/iter data_time: 0.2229 s/iter total_throughput: 2979.70 samples/s lr: 2.36e-04 [09/23 03:59:52] lb.utils.events INFO: eta: 5:09:02 iteration: 256299/375342 consumed_samples: 262451200 total_loss: 3.305 time: 0.3437 s/iter data_time: 0.2151 s/iter total_throughput: 2979.69 samples/s lr: 2.36e-04 [09/23 04:00:27] lb.utils.events INFO: eta: 5:09:01 iteration: 256399/375342 consumed_samples: 262553600 total_loss: 3.305 time: 0.3437 s/iter data_time: 0.2117 s/iter total_throughput: 2979.67 samples/s lr: 2.36e-04 [09/23 04:01:02] lb.utils.events INFO: eta: 5:08:48 iteration: 256499/375342 consumed_samples: 262656000 total_loss: 3.28 time: 0.3437 s/iter data_time: 0.2289 s/iter total_throughput: 2979.65 samples/s lr: 2.35e-04 [09/23 04:01:37] lb.utils.events INFO: eta: 5:08:30 iteration: 256599/375342 consumed_samples: 262758400 total_loss: 3.27 time: 0.3437 s/iter data_time: 0.2228 s/iter total_throughput: 2979.62 samples/s lr: 2.35e-04 [09/23 04:02:12] lb.utils.events INFO: eta: 5:07:49 iteration: 256699/375342 consumed_samples: 262860800 total_loss: 3.277 time: 0.3437 s/iter data_time: 0.2272 s/iter total_throughput: 2979.60 samples/s lr: 2.35e-04 [09/23 04:02:47] lb.utils.events INFO: eta: 5:07:06 iteration: 256799/375342 consumed_samples: 262963200 total_loss: 3.288 time: 0.3437 s/iter data_time: 0.2212 s/iter total_throughput: 2979.59 samples/s lr: 2.34e-04 [09/23 04:03:22] lb.utils.events INFO: eta: 5:06:45 iteration: 256899/375342 consumed_samples: 263065600 total_loss: 3.279 time: 0.3437 s/iter data_time: 0.2271 s/iter total_throughput: 2979.57 samples/s lr: 2.34e-04 [09/23 04:03:57] lb.utils.events INFO: eta: 5:06:36 iteration: 256999/375342 consumed_samples: 263168000 total_loss: 3.301 time: 0.3437 s/iter data_time: 0.2196 s/iter total_throughput: 2979.54 samples/s lr: 2.34e-04 [09/23 04:04:32] lb.utils.events INFO: eta: 5:06:14 iteration: 257099/375342 consumed_samples: 263270400 total_loss: 3.302 time: 0.3437 s/iter data_time: 0.2226 s/iter total_throughput: 2979.52 samples/s lr: 2.33e-04 [09/23 04:05:06] lb.utils.events INFO: eta: 5:06:35 iteration: 257199/375342 consumed_samples: 263372800 total_loss: 3.293 time: 0.3437 s/iter data_time: 0.2199 s/iter total_throughput: 2979.51 samples/s lr: 2.33e-04 [09/23 04:05:42] lb.utils.events INFO: eta: 5:05:34 iteration: 257299/375342 consumed_samples: 263475200 total_loss: 3.296 time: 0.3437 s/iter data_time: 0.2207 s/iter total_throughput: 2979.48 samples/s lr: 2.33e-04 [09/23 04:06:17] lb.utils.events INFO: eta: 5:05:09 iteration: 257399/375342 consumed_samples: 263577600 total_loss: 3.287 time: 0.3437 s/iter data_time: 0.2225 s/iter total_throughput: 2979.45 samples/s lr: 2.32e-04 [09/23 04:06:53] lb.utils.events INFO: eta: 5:04:28 iteration: 257499/375342 consumed_samples: 263680000 total_loss: 3.284 time: 0.3437 s/iter data_time: 0.2258 s/iter total_throughput: 2979.41 samples/s lr: 2.32e-04 [09/23 04:07:27] lb.utils.events INFO: eta: 5:04:58 iteration: 257599/375342 consumed_samples: 263782400 total_loss: 3.287 time: 0.3437 s/iter data_time: 0.2237 s/iter total_throughput: 2979.40 samples/s lr: 2.32e-04 [09/23 04:08:02] lb.utils.events INFO: eta: 5:04:43 iteration: 257699/375342 consumed_samples: 263884800 total_loss: 3.286 time: 0.3437 s/iter data_time: 0.2247 s/iter total_throughput: 2979.38 samples/s lr: 2.31e-04 [09/23 04:08:37] lb.utils.events INFO: eta: 5:04:44 iteration: 257799/375342 consumed_samples: 263987200 total_loss: 3.283 time: 0.3437 s/iter data_time: 0.2190 s/iter total_throughput: 2979.38 samples/s lr: 2.31e-04 [09/23 04:09:12] lb.utils.events INFO: eta: 5:03:45 iteration: 257899/375342 consumed_samples: 264089600 total_loss: 3.27 time: 0.3437 s/iter data_time: 0.2254 s/iter total_throughput: 2979.36 samples/s lr: 2.31e-04 [09/23 04:09:47] lb.utils.events INFO: eta: 5:04:15 iteration: 257999/375342 consumed_samples: 264192000 total_loss: 3.278 time: 0.3437 s/iter data_time: 0.2178 s/iter total_throughput: 2979.34 samples/s lr: 2.30e-04 [09/23 04:10:21] lb.utils.events INFO: eta: 5:04:18 iteration: 258099/375342 consumed_samples: 264294400 total_loss: 3.284 time: 0.3437 s/iter data_time: 0.2323 s/iter total_throughput: 2979.33 samples/s lr: 2.30e-04 [09/23 04:10:56] lb.utils.events INFO: eta: 5:03:44 iteration: 258199/375342 consumed_samples: 264396800 total_loss: 3.271 time: 0.3437 s/iter data_time: 0.2194 s/iter total_throughput: 2979.31 samples/s lr: 2.29e-04 [09/23 04:11:31] lb.utils.events INFO: eta: 5:03:44 iteration: 258299/375342 consumed_samples: 264499200 total_loss: 3.27 time: 0.3437 s/iter data_time: 0.2202 s/iter total_throughput: 2979.29 samples/s lr: 2.29e-04 [09/23 04:12:06] lb.utils.events INFO: eta: 5:03:31 iteration: 258399/375342 consumed_samples: 264601600 total_loss: 3.295 time: 0.3437 s/iter data_time: 0.2351 s/iter total_throughput: 2979.27 samples/s lr: 2.29e-04 [09/23 04:12:41] lb.utils.events INFO: eta: 5:04:12 iteration: 258499/375342 consumed_samples: 264704000 total_loss: 3.301 time: 0.3437 s/iter data_time: 0.2180 s/iter total_throughput: 2979.25 samples/s lr: 2.28e-04 [09/23 04:13:17] lb.utils.events INFO: eta: 5:02:45 iteration: 258599/375342 consumed_samples: 264806400 total_loss: 3.296 time: 0.3437 s/iter data_time: 0.2282 s/iter total_throughput: 2979.22 samples/s lr: 2.28e-04 [09/23 04:13:51] lb.utils.events INFO: eta: 5:02:49 iteration: 258699/375342 consumed_samples: 264908800 total_loss: 3.273 time: 0.3437 s/iter data_time: 0.2151 s/iter total_throughput: 2979.21 samples/s lr: 2.28e-04 [09/23 04:14:26] lb.utils.events INFO: eta: 5:02:30 iteration: 258799/375342 consumed_samples: 265011200 total_loss: 3.269 time: 0.3437 s/iter data_time: 0.2088 s/iter total_throughput: 2979.19 samples/s lr: 2.27e-04 [09/23 04:15:01] lb.utils.events INFO: eta: 5:03:29 iteration: 258899/375342 consumed_samples: 265113600 total_loss: 3.277 time: 0.3437 s/iter data_time: 0.2183 s/iter total_throughput: 2979.18 samples/s lr: 2.27e-04 [09/23 04:15:36] lb.utils.events INFO: eta: 5:03:13 iteration: 258999/375342 consumed_samples: 265216000 total_loss: 3.281 time: 0.3437 s/iter data_time: 0.2070 s/iter total_throughput: 2979.17 samples/s lr: 2.27e-04 [09/23 04:16:10] lb.utils.events INFO: eta: 5:02:04 iteration: 259099/375342 consumed_samples: 265318400 total_loss: 3.276 time: 0.3437 s/iter data_time: 0.2249 s/iter total_throughput: 2979.15 samples/s lr: 2.26e-04 [09/23 04:16:46] lb.utils.events INFO: eta: 5:01:44 iteration: 259199/375342 consumed_samples: 265420800 total_loss: 3.267 time: 0.3437 s/iter data_time: 0.2242 s/iter total_throughput: 2979.12 samples/s lr: 2.26e-04 [09/23 04:17:21] lb.utils.events INFO: eta: 5:01:33 iteration: 259299/375342 consumed_samples: 265523200 total_loss: 3.266 time: 0.3437 s/iter data_time: 0.2215 s/iter total_throughput: 2979.10 samples/s lr: 2.26e-04 [09/23 04:17:56] lb.utils.events INFO: eta: 5:01:10 iteration: 259399/375342 consumed_samples: 265625600 total_loss: 3.253 time: 0.3437 s/iter data_time: 0.2159 s/iter total_throughput: 2979.08 samples/s lr: 2.25e-04 [09/23 04:18:31] lb.utils.events INFO: eta: 5:00:33 iteration: 259499/375342 consumed_samples: 265728000 total_loss: 3.281 time: 0.3437 s/iter data_time: 0.2142 s/iter total_throughput: 2979.06 samples/s lr: 2.25e-04 [09/23 04:19:06] lb.utils.events INFO: eta: 5:00:36 iteration: 259599/375342 consumed_samples: 265830400 total_loss: 3.287 time: 0.3437 s/iter data_time: 0.2243 s/iter total_throughput: 2979.04 samples/s lr: 2.25e-04 [09/23 04:19:41] lb.utils.events INFO: eta: 5:00:10 iteration: 259699/375342 consumed_samples: 265932800 total_loss: 3.27 time: 0.3437 s/iter data_time: 0.2284 s/iter total_throughput: 2979.00 samples/s lr: 2.24e-04 [09/23 04:20:16] lb.utils.events INFO: eta: 4:59:53 iteration: 259799/375342 consumed_samples: 266035200 total_loss: 3.274 time: 0.3437 s/iter data_time: 0.2232 s/iter total_throughput: 2978.99 samples/s lr: 2.24e-04 [09/23 04:20:51] lb.utils.events INFO: eta: 4:58:57 iteration: 259899/375342 consumed_samples: 266137600 total_loss: 3.283 time: 0.3437 s/iter data_time: 0.2281 s/iter total_throughput: 2978.97 samples/s lr: 2.24e-04 [09/23 04:21:26] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0259999 [09/23 04:21:27] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/23 04:21:27] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/23 04:21:31] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.1012 s/iter. Inference: 0.1631 s/iter. Eval: 0.0021 s/iter. Total: 0.2664 s/iter. ETA=0:00:09 [09/23 04:21:36] lb.evaluation.evaluator INFO: Inference done 28672/50000. Dataloading: 0.1198 s/iter. Inference: 0.1663 s/iter. Eval: 0.0020 s/iter. Total: 0.2882 s/iter. ETA=0:00:05 [09/23 04:21:41] lb.evaluation.evaluator INFO: Inference done 46080/50000. Dataloading: 0.1268 s/iter. Inference: 0.1638 s/iter. Eval: 0.0020 s/iter. Total: 0.2926 s/iter. ETA=0:00:00 [09/23 04:21:42] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/23 04:21:42] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.660901 (0.000253 s / iter per device, on 8 devices) [09/23 04:21:42] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000144 s / iter per device, on 8 devices) [09/23 04:21:42] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/23 04:21:42] lb.evaluation.utils INFO: copypaste: Acc@1=77.704 [09/23 04:21:42] lb.evaluation.utils INFO: copypaste: Acc@5=93.72399999999999 [09/23 04:21:42] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 77.70400, better than last best score 77.62200 @ iteration 249999. [09/23 04:21:42] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/23 04:21:43] lb.utils.events INFO: eta: 4:57:50 iteration: 259999/375342 consumed_samples: 266240000 total_loss: 3.295 time: 0.3437 s/iter data_time: 0.2238 s/iter total_throughput: 2978.96 samples/s lr: 2.23e-04 [09/23 04:22:15] lb.utils.events INFO: eta: 4:59:38 iteration: 260099/375342 consumed_samples: 266342400 total_loss: 3.281 time: 0.3437 s/iter data_time: 0.2373 s/iter total_throughput: 2979.02 samples/s lr: 2.23e-04 [09/23 04:22:50] lb.utils.events INFO: eta: 4:59:07 iteration: 260199/375342 consumed_samples: 266444800 total_loss: 3.277 time: 0.3437 s/iter data_time: 0.2229 s/iter total_throughput: 2979.00 samples/s lr: 2.23e-04 [09/23 04:23:25] lb.utils.events INFO: eta: 4:58:33 iteration: 260299/375342 consumed_samples: 266547200 total_loss: 3.276 time: 0.3437 s/iter data_time: 0.2244 s/iter total_throughput: 2979.00 samples/s lr: 2.22e-04 [09/23 04:24:00] lb.utils.events INFO: eta: 4:58:15 iteration: 260399/375342 consumed_samples: 266649600 total_loss: 3.259 time: 0.3437 s/iter data_time: 0.2250 s/iter total_throughput: 2978.97 samples/s lr: 2.22e-04 [09/23 04:24:35] lb.utils.events INFO: eta: 4:58:19 iteration: 260499/375342 consumed_samples: 266752000 total_loss: 3.258 time: 0.3437 s/iter data_time: 0.2238 s/iter total_throughput: 2978.95 samples/s lr: 2.22e-04 [09/23 04:25:09] lb.utils.events INFO: eta: 4:57:55 iteration: 260599/375342 consumed_samples: 266854400 total_loss: 3.266 time: 0.3437 s/iter data_time: 0.2281 s/iter total_throughput: 2978.95 samples/s lr: 2.21e-04 [09/23 04:25:44] lb.utils.events INFO: eta: 4:58:12 iteration: 260699/375342 consumed_samples: 266956800 total_loss: 3.276 time: 0.3437 s/iter data_time: 0.2311 s/iter total_throughput: 2978.94 samples/s lr: 2.21e-04 [09/23 04:26:18] lb.utils.events INFO: eta: 4:57:40 iteration: 260799/375342 consumed_samples: 267059200 total_loss: 3.273 time: 0.3437 s/iter data_time: 0.2162 s/iter total_throughput: 2978.93 samples/s lr: 2.21e-04 [09/23 04:26:53] lb.utils.events INFO: eta: 4:57:30 iteration: 260899/375342 consumed_samples: 267161600 total_loss: 3.271 time: 0.3437 s/iter data_time: 0.2145 s/iter total_throughput: 2978.93 samples/s lr: 2.20e-04 [09/23 04:27:27] lb.utils.events INFO: eta: 4:57:41 iteration: 260999/375342 consumed_samples: 267264000 total_loss: 3.303 time: 0.3437 s/iter data_time: 0.2118 s/iter total_throughput: 2978.92 samples/s lr: 2.20e-04 [09/23 04:28:02] lb.utils.events INFO: eta: 4:56:10 iteration: 261099/375342 consumed_samples: 267366400 total_loss: 3.305 time: 0.3438 s/iter data_time: 0.2236 s/iter total_throughput: 2978.90 samples/s lr: 2.20e-04 [09/23 04:28:37] lb.utils.events INFO: eta: 4:56:18 iteration: 261199/375342 consumed_samples: 267468800 total_loss: 3.285 time: 0.3438 s/iter data_time: 0.2148 s/iter total_throughput: 2978.89 samples/s lr: 2.19e-04 [09/23 04:29:12] lb.utils.events INFO: eta: 4:56:03 iteration: 261299/375342 consumed_samples: 267571200 total_loss: 3.27 time: 0.3438 s/iter data_time: 0.2215 s/iter total_throughput: 2978.87 samples/s lr: 2.19e-04 [09/23 04:29:47] lb.utils.events INFO: eta: 4:55:47 iteration: 261399/375342 consumed_samples: 267673600 total_loss: 3.263 time: 0.3438 s/iter data_time: 0.2160 s/iter total_throughput: 2978.87 samples/s lr: 2.19e-04 [09/23 04:30:22] lb.utils.events INFO: eta: 4:55:31 iteration: 261499/375342 consumed_samples: 267776000 total_loss: 3.273 time: 0.3438 s/iter data_time: 0.2297 s/iter total_throughput: 2978.85 samples/s lr: 2.18e-04 [09/23 04:30:57] lb.utils.events INFO: eta: 4:55:02 iteration: 261599/375342 consumed_samples: 267878400 total_loss: 3.273 time: 0.3438 s/iter data_time: 0.2234 s/iter total_throughput: 2978.83 samples/s lr: 2.18e-04 [09/23 04:31:31] lb.utils.events INFO: eta: 4:54:15 iteration: 261699/375342 consumed_samples: 267980800 total_loss: 3.262 time: 0.3438 s/iter data_time: 0.2183 s/iter total_throughput: 2978.82 samples/s lr: 2.18e-04 [09/23 04:32:06] lb.utils.events INFO: eta: 4:54:03 iteration: 261799/375342 consumed_samples: 268083200 total_loss: 3.263 time: 0.3438 s/iter data_time: 0.2271 s/iter total_throughput: 2978.81 samples/s lr: 2.17e-04 [09/23 04:32:40] lb.utils.events INFO: eta: 4:53:44 iteration: 261899/375342 consumed_samples: 268185600 total_loss: 3.259 time: 0.3438 s/iter data_time: 0.2141 s/iter total_throughput: 2978.81 samples/s lr: 2.17e-04 [09/23 04:33:16] lb.utils.events INFO: eta: 4:52:03 iteration: 261999/375342 consumed_samples: 268288000 total_loss: 3.25 time: 0.3438 s/iter data_time: 0.2321 s/iter total_throughput: 2978.78 samples/s lr: 2.17e-04 [09/23 04:33:50] lb.utils.events INFO: eta: 4:52:23 iteration: 262099/375342 consumed_samples: 268390400 total_loss: 3.253 time: 0.3438 s/iter data_time: 0.2214 s/iter total_throughput: 2978.77 samples/s lr: 2.16e-04 [09/23 04:34:25] lb.utils.events INFO: eta: 4:52:05 iteration: 262199/375342 consumed_samples: 268492800 total_loss: 3.268 time: 0.3438 s/iter data_time: 0.2239 s/iter total_throughput: 2978.74 samples/s lr: 2.16e-04 [09/23 04:35:00] lb.utils.events INFO: eta: 4:52:02 iteration: 262299/375342 consumed_samples: 268595200 total_loss: 3.273 time: 0.3438 s/iter data_time: 0.2210 s/iter total_throughput: 2978.73 samples/s lr: 2.16e-04 [09/23 04:35:35] lb.utils.events INFO: eta: 4:51:31 iteration: 262399/375342 consumed_samples: 268697600 total_loss: 3.265 time: 0.3438 s/iter data_time: 0.2315 s/iter total_throughput: 2978.72 samples/s lr: 2.15e-04 [09/23 04:36:10] lb.utils.events INFO: eta: 4:51:31 iteration: 262499/375342 consumed_samples: 268800000 total_loss: 3.267 time: 0.3438 s/iter data_time: 0.2152 s/iter total_throughput: 2978.71 samples/s lr: 2.15e-04 [09/23 04:36:45] lb.utils.events INFO: eta: 4:50:56 iteration: 262599/375342 consumed_samples: 268902400 total_loss: 3.267 time: 0.3438 s/iter data_time: 0.2230 s/iter total_throughput: 2978.69 samples/s lr: 2.15e-04 [09/23 04:37:20] lb.utils.events INFO: eta: 4:50:47 iteration: 262699/375342 consumed_samples: 269004800 total_loss: 3.24 time: 0.3438 s/iter data_time: 0.2262 s/iter total_throughput: 2978.67 samples/s lr: 2.14e-04 [09/23 04:37:55] lb.utils.events INFO: eta: 4:50:45 iteration: 262799/375342 consumed_samples: 269107200 total_loss: 3.234 time: 0.3438 s/iter data_time: 0.2211 s/iter total_throughput: 2978.65 samples/s lr: 2.14e-04 [09/23 04:38:29] lb.utils.events INFO: eta: 4:50:14 iteration: 262899/375342 consumed_samples: 269209600 total_loss: 3.248 time: 0.3438 s/iter data_time: 0.2240 s/iter total_throughput: 2978.65 samples/s lr: 2.14e-04 [09/23 04:39:04] lb.utils.events INFO: eta: 4:49:50 iteration: 262999/375342 consumed_samples: 269312000 total_loss: 3.273 time: 0.3438 s/iter data_time: 0.2284 s/iter total_throughput: 2978.63 samples/s lr: 2.13e-04 [09/23 04:39:38] lb.utils.events INFO: eta: 4:49:55 iteration: 263099/375342 consumed_samples: 269414400 total_loss: 3.264 time: 0.3438 s/iter data_time: 0.2199 s/iter total_throughput: 2978.64 samples/s lr: 2.13e-04 [09/23 04:40:13] lb.utils.events INFO: eta: 4:49:56 iteration: 263199/375342 consumed_samples: 269516800 total_loss: 3.25 time: 0.3438 s/iter data_time: 0.2201 s/iter total_throughput: 2978.61 samples/s lr: 2.13e-04 [09/23 04:40:47] lb.utils.events INFO: eta: 4:49:53 iteration: 263299/375342 consumed_samples: 269619200 total_loss: 3.251 time: 0.3438 s/iter data_time: 0.2254 s/iter total_throughput: 2978.62 samples/s lr: 2.12e-04 [09/23 04:41:22] lb.utils.events INFO: eta: 4:50:33 iteration: 263399/375342 consumed_samples: 269721600 total_loss: 3.254 time: 0.3438 s/iter data_time: 0.2171 s/iter total_throughput: 2978.61 samples/s lr: 2.12e-04 [09/23 04:41:56] lb.utils.events INFO: eta: 4:50:40 iteration: 263499/375342 consumed_samples: 269824000 total_loss: 3.266 time: 0.3438 s/iter data_time: 0.2093 s/iter total_throughput: 2978.61 samples/s lr: 2.12e-04 [09/23 04:42:31] lb.utils.events INFO: eta: 4:50:33 iteration: 263599/375342 consumed_samples: 269926400 total_loss: 3.275 time: 0.3438 s/iter data_time: 0.2100 s/iter total_throughput: 2978.60 samples/s lr: 2.11e-04 [09/23 04:43:06] lb.utils.events INFO: eta: 4:49:56 iteration: 263699/375342 consumed_samples: 270028800 total_loss: 3.264 time: 0.3438 s/iter data_time: 0.2096 s/iter total_throughput: 2978.60 samples/s lr: 2.11e-04 [09/23 04:43:40] lb.utils.events INFO: eta: 4:50:02 iteration: 263799/375342 consumed_samples: 270131200 total_loss: 3.272 time: 0.3438 s/iter data_time: 0.2188 s/iter total_throughput: 2978.60 samples/s lr: 2.11e-04 [09/23 04:44:15] lb.utils.events INFO: eta: 4:50:03 iteration: 263899/375342 consumed_samples: 270233600 total_loss: 3.283 time: 0.3438 s/iter data_time: 0.2190 s/iter total_throughput: 2978.60 samples/s lr: 2.10e-04 [09/23 04:44:49] lb.utils.events INFO: eta: 4:50:45 iteration: 263999/375342 consumed_samples: 270336000 total_loss: 3.281 time: 0.3438 s/iter data_time: 0.2108 s/iter total_throughput: 2978.60 samples/s lr: 2.10e-04 [09/23 04:45:23] lb.utils.events INFO: eta: 4:49:51 iteration: 264099/375342 consumed_samples: 270438400 total_loss: 3.275 time: 0.3438 s/iter data_time: 0.2193 s/iter total_throughput: 2978.59 samples/s lr: 2.10e-04 [09/23 04:45:58] lb.utils.events INFO: eta: 4:49:43 iteration: 264199/375342 consumed_samples: 270540800 total_loss: 3.254 time: 0.3438 s/iter data_time: 0.2282 s/iter total_throughput: 2978.58 samples/s lr: 2.09e-04 [09/23 04:46:33] lb.utils.events INFO: eta: 4:49:05 iteration: 264299/375342 consumed_samples: 270643200 total_loss: 3.238 time: 0.3438 s/iter data_time: 0.2186 s/iter total_throughput: 2978.57 samples/s lr: 2.09e-04 [09/23 04:47:07] lb.utils.events INFO: eta: 4:48:45 iteration: 264399/375342 consumed_samples: 270745600 total_loss: 3.227 time: 0.3438 s/iter data_time: 0.2220 s/iter total_throughput: 2978.57 samples/s lr: 2.09e-04 [09/23 04:47:42] lb.utils.events INFO: eta: 4:47:45 iteration: 264499/375342 consumed_samples: 270848000 total_loss: 3.241 time: 0.3438 s/iter data_time: 0.2193 s/iter total_throughput: 2978.56 samples/s lr: 2.08e-04 [09/23 04:48:17] lb.utils.events INFO: eta: 4:46:58 iteration: 264599/375342 consumed_samples: 270950400 total_loss: 3.281 time: 0.3438 s/iter data_time: 0.2239 s/iter total_throughput: 2978.54 samples/s lr: 2.08e-04 [09/23 04:48:52] lb.utils.events INFO: eta: 4:47:26 iteration: 264699/375342 consumed_samples: 271052800 total_loss: 3.274 time: 0.3438 s/iter data_time: 0.2187 s/iter total_throughput: 2978.53 samples/s lr: 2.08e-04 [09/23 04:49:26] lb.utils.events INFO: eta: 4:47:11 iteration: 264799/375342 consumed_samples: 271155200 total_loss: 3.261 time: 0.3438 s/iter data_time: 0.2198 s/iter total_throughput: 2978.52 samples/s lr: 2.07e-04 [09/23 04:50:01] lb.utils.events INFO: eta: 4:46:28 iteration: 264899/375342 consumed_samples: 271257600 total_loss: 3.252 time: 0.3438 s/iter data_time: 0.2296 s/iter total_throughput: 2978.51 samples/s lr: 2.07e-04 [09/23 04:50:36] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0264999 [09/23 04:50:36] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/23 04:50:36] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/23 04:50:41] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0893 s/iter. Inference: 0.1611 s/iter. Eval: 0.0020 s/iter. Total: 0.2524 s/iter. ETA=0:00:09 [09/23 04:50:46] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1378 s/iter. Inference: 0.1628 s/iter. Eval: 0.0020 s/iter. Total: 0.3027 s/iter. ETA=0:00:05 [09/23 04:50:52] lb.evaluation.evaluator INFO: Inference done 50000/50000. Dataloading: 0.1223 s/iter. Inference: 0.1625 s/iter. Eval: 0.0020 s/iter. Total: 0.2869 s/iter. ETA=0:00:00 [09/23 04:50:52] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/23 04:50:52] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.628936 (0.000253 s / iter per device, on 8 devices) [09/23 04:50:52] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000143 s / iter per device, on 8 devices) [09/23 04:50:52] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/23 04:50:52] lb.evaluation.utils INFO: copypaste: Acc@1=77.9 [09/23 04:50:52] lb.evaluation.utils INFO: copypaste: Acc@5=93.72399999999999 [09/23 04:50:52] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 77.90000, better than last best score 77.70400 @ iteration 259999. [09/23 04:50:52] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/23 04:50:52] lb.utils.events INFO: eta: 4:45:16 iteration: 264999/375342 consumed_samples: 271360000 total_loss: 3.256 time: 0.3438 s/iter data_time: 0.2297 s/iter total_throughput: 2978.50 samples/s lr: 2.07e-04 [09/23 04:51:25] lb.utils.events INFO: eta: 4:46:53 iteration: 265099/375342 consumed_samples: 271462400 total_loss: 3.273 time: 0.3438 s/iter data_time: 0.2352 s/iter total_throughput: 2978.57 samples/s lr: 2.06e-04 [09/23 04:51:59] lb.utils.events INFO: eta: 4:45:55 iteration: 265199/375342 consumed_samples: 271564800 total_loss: 3.271 time: 0.3438 s/iter data_time: 0.2254 s/iter total_throughput: 2978.56 samples/s lr: 2.06e-04 [09/23 04:52:33] lb.utils.events INFO: eta: 4:45:45 iteration: 265299/375342 consumed_samples: 271667200 total_loss: 3.258 time: 0.3438 s/iter data_time: 0.2080 s/iter total_throughput: 2978.57 samples/s lr: 2.06e-04 [09/23 04:53:08] lb.utils.events INFO: eta: 4:45:42 iteration: 265399/375342 consumed_samples: 271769600 total_loss: 3.234 time: 0.3438 s/iter data_time: 0.2240 s/iter total_throughput: 2978.58 samples/s lr: 2.05e-04 [09/23 04:53:42] lb.utils.events INFO: eta: 4:46:03 iteration: 265499/375342 consumed_samples: 271872000 total_loss: 3.25 time: 0.3438 s/iter data_time: 0.2086 s/iter total_throughput: 2978.59 samples/s lr: 2.05e-04 [09/23 04:54:16] lb.utils.events INFO: eta: 4:46:18 iteration: 265599/375342 consumed_samples: 271974400 total_loss: 3.247 time: 0.3438 s/iter data_time: 0.2178 s/iter total_throughput: 2978.59 samples/s lr: 2.05e-04 [09/23 04:54:51] lb.utils.events INFO: eta: 4:45:30 iteration: 265699/375342 consumed_samples: 272076800 total_loss: 3.24 time: 0.3438 s/iter data_time: 0.2201 s/iter total_throughput: 2978.58 samples/s lr: 2.04e-04 [09/23 04:55:25] lb.utils.events INFO: eta: 4:46:09 iteration: 265799/375342 consumed_samples: 272179200 total_loss: 3.244 time: 0.3438 s/iter data_time: 0.2236 s/iter total_throughput: 2978.60 samples/s lr: 2.04e-04 [09/23 04:55:59] lb.utils.events INFO: eta: 4:46:39 iteration: 265899/375342 consumed_samples: 272281600 total_loss: 3.246 time: 0.3438 s/iter data_time: 0.2257 s/iter total_throughput: 2978.60 samples/s lr: 2.04e-04 [09/23 04:56:33] lb.utils.events INFO: eta: 4:46:36 iteration: 265999/375342 consumed_samples: 272384000 total_loss: 3.25 time: 0.3438 s/iter data_time: 0.2240 s/iter total_throughput: 2978.59 samples/s lr: 2.03e-04 [09/23 04:57:08] lb.utils.events INFO: eta: 4:43:55 iteration: 266099/375342 consumed_samples: 272486400 total_loss: 3.233 time: 0.3438 s/iter data_time: 0.2118 s/iter total_throughput: 2978.58 samples/s lr: 2.03e-04 [09/23 04:57:43] lb.utils.events INFO: eta: 4:44:40 iteration: 266199/375342 consumed_samples: 272588800 total_loss: 3.231 time: 0.3438 s/iter data_time: 0.2256 s/iter total_throughput: 2978.57 samples/s lr: 2.03e-04 [09/23 04:58:18] lb.utils.events INFO: eta: 4:43:43 iteration: 266299/375342 consumed_samples: 272691200 total_loss: 3.247 time: 0.3438 s/iter data_time: 0.2163 s/iter total_throughput: 2978.56 samples/s lr: 2.02e-04 [09/23 04:58:53] lb.utils.events INFO: eta: 4:42:29 iteration: 266399/375342 consumed_samples: 272793600 total_loss: 3.262 time: 0.3438 s/iter data_time: 0.2290 s/iter total_throughput: 2978.52 samples/s lr: 2.02e-04 [09/23 04:59:29] lb.utils.events INFO: eta: 4:41:53 iteration: 266499/375342 consumed_samples: 272896000 total_loss: 3.258 time: 0.3438 s/iter data_time: 0.2323 s/iter total_throughput: 2978.49 samples/s lr: 2.02e-04 [09/23 05:00:04] lb.utils.events INFO: eta: 4:40:56 iteration: 266599/375342 consumed_samples: 272998400 total_loss: 3.265 time: 0.3438 s/iter data_time: 0.2349 s/iter total_throughput: 2978.44 samples/s lr: 2.01e-04 [09/23 05:00:40] lb.utils.events INFO: eta: 4:41:22 iteration: 266699/375342 consumed_samples: 273100800 total_loss: 3.247 time: 0.3438 s/iter data_time: 0.2244 s/iter total_throughput: 2978.41 samples/s lr: 2.01e-04 [09/23 05:01:15] lb.utils.events INFO: eta: 4:39:56 iteration: 266799/375342 consumed_samples: 273203200 total_loss: 3.228 time: 0.3438 s/iter data_time: 0.2306 s/iter total_throughput: 2978.38 samples/s lr: 2.01e-04 [09/23 05:01:50] lb.utils.events INFO: eta: 4:39:05 iteration: 266899/375342 consumed_samples: 273305600 total_loss: 3.243 time: 0.3438 s/iter data_time: 0.2295 s/iter total_throughput: 2978.36 samples/s lr: 2.00e-04 [09/23 05:02:25] lb.utils.events INFO: eta: 4:38:42 iteration: 266999/375342 consumed_samples: 273408000 total_loss: 3.245 time: 0.3438 s/iter data_time: 0.2203 s/iter total_throughput: 2978.33 samples/s lr: 2.00e-04 [09/23 05:03:01] lb.utils.events INFO: eta: 4:38:27 iteration: 267099/375342 consumed_samples: 273510400 total_loss: 3.227 time: 0.3438 s/iter data_time: 0.2223 s/iter total_throughput: 2978.30 samples/s lr: 2.00e-04 [09/23 05:03:36] lb.utils.events INFO: eta: 4:38:10 iteration: 267199/375342 consumed_samples: 273612800 total_loss: 3.233 time: 0.3438 s/iter data_time: 0.2323 s/iter total_throughput: 2978.27 samples/s lr: 1.99e-04 [09/23 05:04:11] lb.utils.events INFO: eta: 4:38:12 iteration: 267299/375342 consumed_samples: 273715200 total_loss: 3.25 time: 0.3438 s/iter data_time: 0.2149 s/iter total_throughput: 2978.27 samples/s lr: 1.99e-04 [09/23 05:04:46] lb.utils.events INFO: eta: 4:38:41 iteration: 267399/375342 consumed_samples: 273817600 total_loss: 3.241 time: 0.3438 s/iter data_time: 0.2138 s/iter total_throughput: 2978.25 samples/s lr: 1.99e-04 [09/23 05:05:21] lb.utils.events INFO: eta: 4:38:26 iteration: 267499/375342 consumed_samples: 273920000 total_loss: 3.24 time: 0.3438 s/iter data_time: 0.2239 s/iter total_throughput: 2978.22 samples/s lr: 1.98e-04 [09/23 05:05:56] lb.utils.events INFO: eta: 4:38:22 iteration: 267599/375342 consumed_samples: 274022400 total_loss: 3.248 time: 0.3438 s/iter data_time: 0.2212 s/iter total_throughput: 2978.19 samples/s lr: 1.98e-04 [09/23 05:06:31] lb.utils.events INFO: eta: 4:37:38 iteration: 267699/375342 consumed_samples: 274124800 total_loss: 3.252 time: 0.3438 s/iter data_time: 0.2152 s/iter total_throughput: 2978.17 samples/s lr: 1.98e-04 [09/23 05:07:06] lb.utils.events INFO: eta: 4:36:44 iteration: 267799/375342 consumed_samples: 274227200 total_loss: 3.255 time: 0.3438 s/iter data_time: 0.2318 s/iter total_throughput: 2978.14 samples/s lr: 1.97e-04 [09/23 05:07:41] lb.utils.events INFO: eta: 4:36:16 iteration: 267899/375342 consumed_samples: 274329600 total_loss: 3.252 time: 0.3438 s/iter data_time: 0.2240 s/iter total_throughput: 2978.13 samples/s lr: 1.97e-04 [09/23 05:08:16] lb.utils.events INFO: eta: 4:36:41 iteration: 267999/375342 consumed_samples: 274432000 total_loss: 3.237 time: 0.3438 s/iter data_time: 0.2192 s/iter total_throughput: 2978.11 samples/s lr: 1.97e-04 [09/23 05:08:51] lb.utils.events INFO: eta: 4:36:40 iteration: 268099/375342 consumed_samples: 274534400 total_loss: 3.233 time: 0.3438 s/iter data_time: 0.2130 s/iter total_throughput: 2978.11 samples/s lr: 1.96e-04 [09/23 05:09:26] lb.utils.events INFO: eta: 4:36:18 iteration: 268199/375342 consumed_samples: 274636800 total_loss: 3.234 time: 0.3438 s/iter data_time: 0.2198 s/iter total_throughput: 2978.09 samples/s lr: 1.96e-04 [09/23 05:10:01] lb.utils.events INFO: eta: 4:35:39 iteration: 268299/375342 consumed_samples: 274739200 total_loss: 3.229 time: 0.3438 s/iter data_time: 0.2185 s/iter total_throughput: 2978.06 samples/s lr: 1.96e-04 [09/23 05:10:36] lb.utils.events INFO: eta: 4:35:10 iteration: 268399/375342 consumed_samples: 274841600 total_loss: 3.245 time: 0.3439 s/iter data_time: 0.2625 s/iter total_throughput: 2978.02 samples/s lr: 1.95e-04 [09/23 05:11:11] lb.utils.events INFO: eta: 4:34:46 iteration: 268499/375342 consumed_samples: 274944000 total_loss: 3.252 time: 0.3439 s/iter data_time: 0.2156 s/iter total_throughput: 2978.01 samples/s lr: 1.95e-04 [09/23 05:11:46] lb.utils.events INFO: eta: 4:35:10 iteration: 268599/375342 consumed_samples: 275046400 total_loss: 3.255 time: 0.3439 s/iter data_time: 0.2127 s/iter total_throughput: 2978.01 samples/s lr: 1.95e-04 [09/23 05:12:20] lb.utils.events INFO: eta: 4:35:52 iteration: 268699/375342 consumed_samples: 275148800 total_loss: 3.248 time: 0.3439 s/iter data_time: 0.2130 s/iter total_throughput: 2978.00 samples/s lr: 1.94e-04 [09/23 05:12:55] lb.utils.events INFO: eta: 4:35:58 iteration: 268799/375342 consumed_samples: 275251200 total_loss: 3.254 time: 0.3439 s/iter data_time: 0.2234 s/iter total_throughput: 2977.98 samples/s lr: 1.94e-04 [09/23 05:13:30] lb.utils.events INFO: eta: 4:35:18 iteration: 268899/375342 consumed_samples: 275353600 total_loss: 3.255 time: 0.3439 s/iter data_time: 0.2350 s/iter total_throughput: 2977.96 samples/s lr: 1.94e-04 [09/23 05:14:05] lb.utils.events INFO: eta: 4:34:37 iteration: 268999/375342 consumed_samples: 275456000 total_loss: 3.242 time: 0.3439 s/iter data_time: 0.2169 s/iter total_throughput: 2977.95 samples/s lr: 1.93e-04 [09/23 05:14:40] lb.utils.events INFO: eta: 4:34:22 iteration: 269099/375342 consumed_samples: 275558400 total_loss: 3.221 time: 0.3439 s/iter data_time: 0.2136 s/iter total_throughput: 2977.93 samples/s lr: 1.93e-04 [09/23 05:15:15] lb.utils.events INFO: eta: 4:34:22 iteration: 269199/375342 consumed_samples: 275660800 total_loss: 3.226 time: 0.3439 s/iter data_time: 0.2133 s/iter total_throughput: 2977.92 samples/s lr: 1.93e-04 [09/23 05:15:50] lb.utils.events INFO: eta: 4:34:27 iteration: 269299/375342 consumed_samples: 275763200 total_loss: 3.248 time: 0.3439 s/iter data_time: 0.2179 s/iter total_throughput: 2977.90 samples/s lr: 1.93e-04 [09/23 05:16:25] lb.utils.events INFO: eta: 4:33:47 iteration: 269399/375342 consumed_samples: 275865600 total_loss: 3.233 time: 0.3439 s/iter data_time: 0.2287 s/iter total_throughput: 2977.86 samples/s lr: 1.92e-04 [09/23 05:17:00] lb.utils.events INFO: eta: 4:33:35 iteration: 269499/375342 consumed_samples: 275968000 total_loss: 3.235 time: 0.3439 s/iter data_time: 0.2182 s/iter total_throughput: 2977.85 samples/s lr: 1.92e-04 [09/23 05:17:35] lb.utils.events INFO: eta: 4:33:03 iteration: 269599/375342 consumed_samples: 276070400 total_loss: 3.259 time: 0.3439 s/iter data_time: 0.2218 s/iter total_throughput: 2977.83 samples/s lr: 1.92e-04 [09/23 05:18:10] lb.utils.events INFO: eta: 4:32:32 iteration: 269699/375342 consumed_samples: 276172800 total_loss: 3.23 time: 0.3439 s/iter data_time: 0.2154 s/iter total_throughput: 2977.82 samples/s lr: 1.91e-04 [09/23 05:18:45] lb.utils.events INFO: eta: 4:32:07 iteration: 269799/375342 consumed_samples: 276275200 total_loss: 3.228 time: 0.3439 s/iter data_time: 0.2311 s/iter total_throughput: 2977.79 samples/s lr: 1.91e-04 [09/23 05:19:21] lb.utils.events INFO: eta: 4:32:41 iteration: 269899/375342 consumed_samples: 276377600 total_loss: 3.24 time: 0.3439 s/iter data_time: 0.2334 s/iter total_throughput: 2977.76 samples/s lr: 1.91e-04 [09/23 05:19:56] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0269999 [09/23 05:19:57] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/23 05:19:57] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/23 05:20:01] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0947 s/iter. Inference: 0.1652 s/iter. Eval: 0.0022 s/iter. Total: 0.2621 s/iter. ETA=0:00:09 [09/23 05:20:06] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1282 s/iter. Inference: 0.1676 s/iter. Eval: 0.0021 s/iter. Total: 0.2980 s/iter. ETA=0:00:05 [09/23 05:20:12] lb.evaluation.evaluator INFO: Inference done 49152/50000. Dataloading: 0.1203 s/iter. Inference: 0.1634 s/iter. Eval: 0.0021 s/iter. Total: 0.2858 s/iter. ETA=0:00:00 [09/23 05:20:12] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/23 05:20:12] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.484810 (0.000250 s / iter per device, on 8 devices) [09/23 05:20:12] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000144 s / iter per device, on 8 devices) [09/23 05:20:12] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/23 05:20:12] lb.evaluation.utils INFO: copypaste: Acc@1=78.164 [09/23 05:20:12] lb.evaluation.utils INFO: copypaste: Acc@5=93.864 [09/23 05:20:12] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 78.16400, better than last best score 77.90000 @ iteration 264999. [09/23 05:20:12] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/23 05:20:13] lb.utils.events INFO: eta: 4:32:17 iteration: 269999/375342 consumed_samples: 276480000 total_loss: 3.246 time: 0.3439 s/iter data_time: 0.2300 s/iter total_throughput: 2977.73 samples/s lr: 1.90e-04 [09/23 05:20:45] lb.utils.events INFO: eta: 4:33:36 iteration: 270099/375342 consumed_samples: 276582400 total_loss: 3.244 time: 0.3439 s/iter data_time: 0.2328 s/iter total_throughput: 2977.78 samples/s lr: 1.90e-04 [09/23 05:21:20] lb.utils.events INFO: eta: 4:33:18 iteration: 270199/375342 consumed_samples: 276684800 total_loss: 3.227 time: 0.3439 s/iter data_time: 0.2289 s/iter total_throughput: 2977.76 samples/s lr: 1.90e-04 [09/23 05:21:55] lb.utils.events INFO: eta: 4:33:40 iteration: 270299/375342 consumed_samples: 276787200 total_loss: 3.239 time: 0.3439 s/iter data_time: 0.2155 s/iter total_throughput: 2977.75 samples/s lr: 1.89e-04 [09/23 05:22:30] lb.utils.events INFO: eta: 4:33:24 iteration: 270399/375342 consumed_samples: 276889600 total_loss: 3.229 time: 0.3439 s/iter data_time: 0.2351 s/iter total_throughput: 2977.73 samples/s lr: 1.89e-04 [09/23 05:23:05] lb.utils.events INFO: eta: 4:33:35 iteration: 270499/375342 consumed_samples: 276992000 total_loss: 3.227 time: 0.3439 s/iter data_time: 0.2144 s/iter total_throughput: 2977.73 samples/s lr: 1.89e-04 [09/23 05:23:39] lb.utils.events INFO: eta: 4:34:05 iteration: 270599/375342 consumed_samples: 277094400 total_loss: 3.251 time: 0.3439 s/iter data_time: 0.2098 s/iter total_throughput: 2977.72 samples/s lr: 1.88e-04 [09/23 05:24:14] lb.utils.events INFO: eta: 4:33:47 iteration: 270699/375342 consumed_samples: 277196800 total_loss: 3.237 time: 0.3439 s/iter data_time: 0.2295 s/iter total_throughput: 2977.69 samples/s lr: 1.88e-04 [09/23 05:24:49] lb.utils.events INFO: eta: 4:34:11 iteration: 270799/375342 consumed_samples: 277299200 total_loss: 3.233 time: 0.3439 s/iter data_time: 0.2205 s/iter total_throughput: 2977.69 samples/s lr: 1.88e-04 [09/23 05:25:24] lb.utils.events INFO: eta: 4:34:49 iteration: 270899/375342 consumed_samples: 277401600 total_loss: 3.248 time: 0.3439 s/iter data_time: 0.2258 s/iter total_throughput: 2977.68 samples/s lr: 1.87e-04 [09/23 05:25:59] lb.utils.events INFO: eta: 4:34:28 iteration: 270999/375342 consumed_samples: 277504000 total_loss: 3.253 time: 0.3439 s/iter data_time: 0.2202 s/iter total_throughput: 2977.66 samples/s lr: 1.87e-04 [09/23 05:26:34] lb.utils.events INFO: eta: 4:32:34 iteration: 271099/375342 consumed_samples: 277606400 total_loss: 3.234 time: 0.3439 s/iter data_time: 0.2227 s/iter total_throughput: 2977.64 samples/s lr: 1.87e-04 [09/23 05:27:09] lb.utils.events INFO: eta: 4:32:07 iteration: 271199/375342 consumed_samples: 277708800 total_loss: 3.229 time: 0.3439 s/iter data_time: 0.2276 s/iter total_throughput: 2977.62 samples/s lr: 1.86e-04 [09/23 05:27:43] lb.utils.events INFO: eta: 4:31:14 iteration: 271299/375342 consumed_samples: 277811200 total_loss: 3.23 time: 0.3439 s/iter data_time: 0.2138 s/iter total_throughput: 2977.62 samples/s lr: 1.86e-04 [09/23 05:28:18] lb.utils.events INFO: eta: 4:30:50 iteration: 271399/375342 consumed_samples: 277913600 total_loss: 3.232 time: 0.3439 s/iter data_time: 0.2219 s/iter total_throughput: 2977.61 samples/s lr: 1.86e-04 [09/23 05:28:53] lb.utils.events INFO: eta: 4:30:03 iteration: 271499/375342 consumed_samples: 278016000 total_loss: 3.23 time: 0.3439 s/iter data_time: 0.2349 s/iter total_throughput: 2977.59 samples/s lr: 1.85e-04 [09/23 05:29:28] lb.utils.events INFO: eta: 4:29:46 iteration: 271599/375342 consumed_samples: 278118400 total_loss: 3.209 time: 0.3439 s/iter data_time: 0.2307 s/iter total_throughput: 2977.57 samples/s lr: 1.85e-04 [09/23 05:30:03] lb.utils.events INFO: eta: 4:29:32 iteration: 271699/375342 consumed_samples: 278220800 total_loss: 3.201 time: 0.3439 s/iter data_time: 0.2148 s/iter total_throughput: 2977.56 samples/s lr: 1.85e-04 [09/23 05:30:37] lb.utils.events INFO: eta: 4:29:48 iteration: 271799/375342 consumed_samples: 278323200 total_loss: 3.23 time: 0.3439 s/iter data_time: 0.2144 s/iter total_throughput: 2977.56 samples/s lr: 1.85e-04 [09/23 05:31:12] lb.utils.events INFO: eta: 4:28:31 iteration: 271899/375342 consumed_samples: 278425600 total_loss: 3.246 time: 0.3439 s/iter data_time: 0.2273 s/iter total_throughput: 2977.53 samples/s lr: 1.84e-04 [09/23 05:31:46] lb.utils.events INFO: eta: 4:28:49 iteration: 271999/375342 consumed_samples: 278528000 total_loss: 3.231 time: 0.3439 s/iter data_time: 0.2153 s/iter total_throughput: 2977.54 samples/s lr: 1.84e-04 [09/23 05:32:21] lb.utils.events INFO: eta: 4:28:33 iteration: 272099/375342 consumed_samples: 278630400 total_loss: 3.229 time: 0.3439 s/iter data_time: 0.2238 s/iter total_throughput: 2977.53 samples/s lr: 1.84e-04 [09/23 05:32:56] lb.utils.events INFO: eta: 4:28:56 iteration: 272199/375342 consumed_samples: 278732800 total_loss: 3.229 time: 0.3439 s/iter data_time: 0.2164 s/iter total_throughput: 2977.52 samples/s lr: 1.83e-04 [09/23 05:33:31] lb.utils.events INFO: eta: 4:28:42 iteration: 272299/375342 consumed_samples: 278835200 total_loss: 3.215 time: 0.3439 s/iter data_time: 0.2171 s/iter total_throughput: 2977.51 samples/s lr: 1.83e-04 [09/23 05:34:05] lb.utils.events INFO: eta: 4:29:11 iteration: 272399/375342 consumed_samples: 278937600 total_loss: 3.227 time: 0.3439 s/iter data_time: 0.2150 s/iter total_throughput: 2977.50 samples/s lr: 1.83e-04 [09/23 05:34:40] lb.utils.events INFO: eta: 4:29:23 iteration: 272499/375342 consumed_samples: 279040000 total_loss: 3.244 time: 0.3439 s/iter data_time: 0.2242 s/iter total_throughput: 2977.48 samples/s lr: 1.82e-04 [09/23 05:35:16] lb.utils.events INFO: eta: 4:28:29 iteration: 272599/375342 consumed_samples: 279142400 total_loss: 3.222 time: 0.3439 s/iter data_time: 0.2144 s/iter total_throughput: 2977.45 samples/s lr: 1.82e-04 [09/23 05:35:50] lb.utils.events INFO: eta: 4:27:44 iteration: 272699/375342 consumed_samples: 279244800 total_loss: 3.234 time: 0.3439 s/iter data_time: 0.2237 s/iter total_throughput: 2977.45 samples/s lr: 1.82e-04 [09/23 05:36:25] lb.utils.events INFO: eta: 4:26:49 iteration: 272799/375342 consumed_samples: 279347200 total_loss: 3.229 time: 0.3439 s/iter data_time: 0.2157 s/iter total_throughput: 2977.43 samples/s lr: 1.81e-04 [09/23 05:37:00] lb.utils.events INFO: eta: 4:26:53 iteration: 272899/375342 consumed_samples: 279449600 total_loss: 3.217 time: 0.3439 s/iter data_time: 0.2202 s/iter total_throughput: 2977.42 samples/s lr: 1.81e-04 [09/23 05:37:34] lb.utils.events INFO: eta: 4:25:43 iteration: 272999/375342 consumed_samples: 279552000 total_loss: 3.236 time: 0.3439 s/iter data_time: 0.2253 s/iter total_throughput: 2977.42 samples/s lr: 1.81e-04 [09/23 05:38:09] lb.utils.events INFO: eta: 4:25:16 iteration: 273099/375342 consumed_samples: 279654400 total_loss: 3.233 time: 0.3439 s/iter data_time: 0.2351 s/iter total_throughput: 2977.40 samples/s lr: 1.80e-04 [09/23 05:38:44] lb.utils.events INFO: eta: 4:24:28 iteration: 273199/375342 consumed_samples: 279756800 total_loss: 3.232 time: 0.3439 s/iter data_time: 0.2108 s/iter total_throughput: 2977.39 samples/s lr: 1.80e-04 [09/23 05:39:18] lb.utils.events INFO: eta: 4:24:20 iteration: 273299/375342 consumed_samples: 279859200 total_loss: 3.231 time: 0.3439 s/iter data_time: 0.2170 s/iter total_throughput: 2977.39 samples/s lr: 1.80e-04 [09/23 05:39:53] lb.utils.events INFO: eta: 4:24:01 iteration: 273399/375342 consumed_samples: 279961600 total_loss: 3.227 time: 0.3439 s/iter data_time: 0.2203 s/iter total_throughput: 2977.38 samples/s lr: 1.80e-04 [09/23 05:40:28] lb.utils.events INFO: eta: 4:24:02 iteration: 273499/375342 consumed_samples: 280064000 total_loss: 3.229 time: 0.3439 s/iter data_time: 0.2329 s/iter total_throughput: 2977.38 samples/s lr: 1.79e-04 [09/23 05:41:03] lb.utils.events INFO: eta: 4:24:02 iteration: 273599/375342 consumed_samples: 280166400 total_loss: 3.229 time: 0.3439 s/iter data_time: 0.2137 s/iter total_throughput: 2977.36 samples/s lr: 1.79e-04 [09/23 05:41:38] lb.utils.events INFO: eta: 4:23:51 iteration: 273699/375342 consumed_samples: 280268800 total_loss: 3.236 time: 0.3439 s/iter data_time: 0.2259 s/iter total_throughput: 2977.33 samples/s lr: 1.79e-04 [09/23 05:42:13] lb.utils.events INFO: eta: 4:23:10 iteration: 273799/375342 consumed_samples: 280371200 total_loss: 3.229 time: 0.3439 s/iter data_time: 0.2358 s/iter total_throughput: 2977.30 samples/s lr: 1.78e-04 [09/23 05:42:48] lb.utils.events INFO: eta: 4:23:28 iteration: 273899/375342 consumed_samples: 280473600 total_loss: 3.216 time: 0.3439 s/iter data_time: 0.2198 s/iter total_throughput: 2977.30 samples/s lr: 1.78e-04 [09/23 05:43:23] lb.utils.events INFO: eta: 4:22:39 iteration: 273999/375342 consumed_samples: 280576000 total_loss: 3.201 time: 0.3439 s/iter data_time: 0.2264 s/iter total_throughput: 2977.28 samples/s lr: 1.78e-04 [09/23 05:43:57] lb.utils.events INFO: eta: 4:22:44 iteration: 274099/375342 consumed_samples: 280678400 total_loss: 3.189 time: 0.3439 s/iter data_time: 0.2114 s/iter total_throughput: 2977.27 samples/s lr: 1.77e-04 [09/23 05:44:32] lb.utils.events INFO: eta: 4:22:32 iteration: 274199/375342 consumed_samples: 280780800 total_loss: 3.209 time: 0.3439 s/iter data_time: 0.2223 s/iter total_throughput: 2977.26 samples/s lr: 1.77e-04 [09/23 05:45:07] lb.utils.events INFO: eta: 4:22:09 iteration: 274299/375342 consumed_samples: 280883200 total_loss: 3.224 time: 0.3439 s/iter data_time: 0.2197 s/iter total_throughput: 2977.25 samples/s lr: 1.77e-04 [09/23 05:45:41] lb.utils.events INFO: eta: 4:21:50 iteration: 274399/375342 consumed_samples: 280985600 total_loss: 3.203 time: 0.3439 s/iter data_time: 0.2139 s/iter total_throughput: 2977.25 samples/s lr: 1.76e-04 [09/23 05:46:16] lb.utils.events INFO: eta: 4:21:04 iteration: 274499/375342 consumed_samples: 281088000 total_loss: 3.206 time: 0.3439 s/iter data_time: 0.2315 s/iter total_throughput: 2977.22 samples/s lr: 1.76e-04 [09/23 05:46:51] lb.utils.events INFO: eta: 4:20:41 iteration: 274599/375342 consumed_samples: 281190400 total_loss: 3.217 time: 0.3439 s/iter data_time: 0.2251 s/iter total_throughput: 2977.21 samples/s lr: 1.76e-04 [09/23 05:47:26] lb.utils.events INFO: eta: 4:21:01 iteration: 274699/375342 consumed_samples: 281292800 total_loss: 3.217 time: 0.3439 s/iter data_time: 0.2172 s/iter total_throughput: 2977.21 samples/s lr: 1.75e-04 [09/23 05:48:01] lb.utils.events INFO: eta: 4:21:00 iteration: 274799/375342 consumed_samples: 281395200 total_loss: 3.219 time: 0.3439 s/iter data_time: 0.2313 s/iter total_throughput: 2977.18 samples/s lr: 1.75e-04 [09/23 05:48:36] lb.utils.events INFO: eta: 4:20:33 iteration: 274899/375342 consumed_samples: 281497600 total_loss: 3.22 time: 0.3440 s/iter data_time: 0.2218 s/iter total_throughput: 2977.16 samples/s lr: 1.75e-04 [09/23 05:49:12] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0274999 [09/23 05:49:12] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/23 05:49:12] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/23 05:49:16] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.1004 s/iter. Inference: 0.1689 s/iter. Eval: 0.0020 s/iter. Total: 0.2714 s/iter. ETA=0:00:10 [09/23 05:49:22] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1386 s/iter. Inference: 0.1643 s/iter. Eval: 0.0020 s/iter. Total: 0.3050 s/iter. ETA=0:00:05 [09/23 05:49:27] lb.evaluation.evaluator INFO: Inference done 49152/50000. Dataloading: 0.1226 s/iter. Inference: 0.1627 s/iter. Eval: 0.0020 s/iter. Total: 0.2874 s/iter. ETA=0:00:00 [09/23 05:49:28] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/23 05:49:28] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.806459 (0.000256 s / iter per device, on 8 devices) [09/23 05:49:28] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000144 s / iter per device, on 8 devices) [09/23 05:49:28] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/23 05:49:28] lb.evaluation.utils INFO: copypaste: Acc@1=78.358 [09/23 05:49:28] lb.evaluation.utils INFO: copypaste: Acc@5=93.978 [09/23 05:49:28] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 78.35800, better than last best score 78.16400 @ iteration 269999. [09/23 05:49:28] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/23 05:49:28] lb.utils.events INFO: eta: 4:20:29 iteration: 274999/375342 consumed_samples: 281600000 total_loss: 3.215 time: 0.3440 s/iter data_time: 0.2311 s/iter total_throughput: 2977.13 samples/s lr: 1.75e-04 [09/23 05:50:01] lb.utils.events INFO: eta: 4:21:01 iteration: 275099/375342 consumed_samples: 281702400 total_loss: 3.203 time: 0.3439 s/iter data_time: 0.2144 s/iter total_throughput: 2977.20 samples/s lr: 1.74e-04 [09/23 05:50:36] lb.utils.events INFO: eta: 4:20:55 iteration: 275199/375342 consumed_samples: 281804800 total_loss: 3.197 time: 0.3439 s/iter data_time: 0.2097 s/iter total_throughput: 2977.18 samples/s lr: 1.74e-04 [09/23 05:51:11] lb.utils.events INFO: eta: 4:20:39 iteration: 275299/375342 consumed_samples: 281907200 total_loss: 3.188 time: 0.3440 s/iter data_time: 0.2270 s/iter total_throughput: 2977.15 samples/s lr: 1.74e-04 [09/23 05:51:45] lb.utils.events INFO: eta: 4:20:42 iteration: 275399/375342 consumed_samples: 282009600 total_loss: 3.193 time: 0.3440 s/iter data_time: 0.2143 s/iter total_throughput: 2977.15 samples/s lr: 1.73e-04 [09/23 05:52:20] lb.utils.events INFO: eta: 4:21:24 iteration: 275499/375342 consumed_samples: 282112000 total_loss: 3.213 time: 0.3440 s/iter data_time: 0.2169 s/iter total_throughput: 2977.14 samples/s lr: 1.73e-04 [09/23 05:52:55] lb.utils.events INFO: eta: 4:21:08 iteration: 275599/375342 consumed_samples: 282214400 total_loss: 3.223 time: 0.3440 s/iter data_time: 0.2216 s/iter total_throughput: 2977.13 samples/s lr: 1.73e-04 [09/23 05:53:30] lb.utils.events INFO: eta: 4:20:30 iteration: 275699/375342 consumed_samples: 282316800 total_loss: 3.208 time: 0.3440 s/iter data_time: 0.2209 s/iter total_throughput: 2977.12 samples/s lr: 1.72e-04 [09/23 05:54:05] lb.utils.events INFO: eta: 4:19:36 iteration: 275799/375342 consumed_samples: 282419200 total_loss: 3.207 time: 0.3440 s/iter data_time: 0.2317 s/iter total_throughput: 2977.09 samples/s lr: 1.72e-04 [09/23 05:54:40] lb.utils.events INFO: eta: 4:18:49 iteration: 275899/375342 consumed_samples: 282521600 total_loss: 3.22 time: 0.3440 s/iter data_time: 0.2325 s/iter total_throughput: 2977.08 samples/s lr: 1.72e-04 [09/23 05:55:15] lb.utils.events INFO: eta: 4:18:40 iteration: 275999/375342 consumed_samples: 282624000 total_loss: 3.229 time: 0.3440 s/iter data_time: 0.2330 s/iter total_throughput: 2977.06 samples/s lr: 1.71e-04 [09/23 05:55:50] lb.utils.events INFO: eta: 4:17:17 iteration: 276099/375342 consumed_samples: 282726400 total_loss: 3.229 time: 0.3440 s/iter data_time: 0.2350 s/iter total_throughput: 2977.04 samples/s lr: 1.71e-04 [09/23 05:56:24] lb.utils.events INFO: eta: 4:16:46 iteration: 276199/375342 consumed_samples: 282828800 total_loss: 3.218 time: 0.3440 s/iter data_time: 0.2261 s/iter total_throughput: 2977.03 samples/s lr: 1.71e-04 [09/23 05:56:59] lb.utils.events INFO: eta: 4:16:14 iteration: 276299/375342 consumed_samples: 282931200 total_loss: 3.204 time: 0.3440 s/iter data_time: 0.2228 s/iter total_throughput: 2977.03 samples/s lr: 1.71e-04 [09/23 05:57:33] lb.utils.events INFO: eta: 4:16:12 iteration: 276399/375342 consumed_samples: 283033600 total_loss: 3.206 time: 0.3440 s/iter data_time: 0.2075 s/iter total_throughput: 2977.04 samples/s lr: 1.70e-04 [09/23 05:58:08] lb.utils.events INFO: eta: 4:15:46 iteration: 276499/375342 consumed_samples: 283136000 total_loss: 3.203 time: 0.3440 s/iter data_time: 0.2363 s/iter total_throughput: 2977.02 samples/s lr: 1.70e-04 [09/23 05:58:43] lb.utils.events INFO: eta: 4:15:28 iteration: 276599/375342 consumed_samples: 283238400 total_loss: 3.209 time: 0.3440 s/iter data_time: 0.2264 s/iter total_throughput: 2977.01 samples/s lr: 1.70e-04 [09/23 05:59:18] lb.utils.events INFO: eta: 4:14:50 iteration: 276699/375342 consumed_samples: 283340800 total_loss: 3.192 time: 0.3440 s/iter data_time: 0.2310 s/iter total_throughput: 2976.99 samples/s lr: 1.69e-04 [09/23 05:59:53] lb.utils.events INFO: eta: 4:15:12 iteration: 276799/375342 consumed_samples: 283443200 total_loss: 3.185 time: 0.3440 s/iter data_time: 0.2240 s/iter total_throughput: 2976.98 samples/s lr: 1.69e-04 [09/23 06:00:27] lb.utils.events INFO: eta: 4:15:26 iteration: 276899/375342 consumed_samples: 283545600 total_loss: 3.202 time: 0.3440 s/iter data_time: 0.2232 s/iter total_throughput: 2976.97 samples/s lr: 1.69e-04 [09/23 06:01:02] lb.utils.events INFO: eta: 4:15:13 iteration: 276999/375342 consumed_samples: 283648000 total_loss: 3.203 time: 0.3440 s/iter data_time: 0.2182 s/iter total_throughput: 2976.96 samples/s lr: 1.68e-04 [09/23 06:01:36] lb.utils.events INFO: eta: 4:15:14 iteration: 277099/375342 consumed_samples: 283750400 total_loss: 3.223 time: 0.3440 s/iter data_time: 0.2156 s/iter total_throughput: 2976.97 samples/s lr: 1.68e-04 [09/23 06:02:10] lb.utils.events INFO: eta: 4:15:18 iteration: 277199/375342 consumed_samples: 283852800 total_loss: 3.222 time: 0.3440 s/iter data_time: 0.2089 s/iter total_throughput: 2976.98 samples/s lr: 1.68e-04 [09/23 06:02:44] lb.utils.events INFO: eta: 4:15:35 iteration: 277299/375342 consumed_samples: 283955200 total_loss: 3.219 time: 0.3440 s/iter data_time: 0.2198 s/iter total_throughput: 2976.98 samples/s lr: 1.68e-04 [09/23 06:03:19] lb.utils.events INFO: eta: 4:15:01 iteration: 277399/375342 consumed_samples: 284057600 total_loss: 3.225 time: 0.3440 s/iter data_time: 0.2430 s/iter total_throughput: 2976.97 samples/s lr: 1.67e-04 [09/23 06:03:54] lb.utils.events INFO: eta: 4:14:58 iteration: 277499/375342 consumed_samples: 284160000 total_loss: 3.21 time: 0.3440 s/iter data_time: 0.2114 s/iter total_throughput: 2976.96 samples/s lr: 1.67e-04 [09/23 06:04:29] lb.utils.events INFO: eta: 4:14:37 iteration: 277599/375342 consumed_samples: 284262400 total_loss: 3.2 time: 0.3440 s/iter data_time: 0.2078 s/iter total_throughput: 2976.95 samples/s lr: 1.67e-04 [09/23 06:05:03] lb.utils.events INFO: eta: 4:14:26 iteration: 277699/375342 consumed_samples: 284364800 total_loss: 3.207 time: 0.3440 s/iter data_time: 0.2250 s/iter total_throughput: 2976.95 samples/s lr: 1.66e-04 [09/23 06:05:38] lb.utils.events INFO: eta: 4:13:51 iteration: 277799/375342 consumed_samples: 284467200 total_loss: 3.22 time: 0.3440 s/iter data_time: 0.2193 s/iter total_throughput: 2976.93 samples/s lr: 1.66e-04 [09/23 06:06:13] lb.utils.events INFO: eta: 4:13:24 iteration: 277899/375342 consumed_samples: 284569600 total_loss: 3.197 time: 0.3440 s/iter data_time: 0.2119 s/iter total_throughput: 2976.92 samples/s lr: 1.66e-04 [09/23 06:06:47] lb.utils.events INFO: eta: 4:13:47 iteration: 277999/375342 consumed_samples: 284672000 total_loss: 3.179 time: 0.3440 s/iter data_time: 0.2168 s/iter total_throughput: 2976.93 samples/s lr: 1.65e-04 [09/23 06:07:22] lb.utils.events INFO: eta: 4:13:17 iteration: 278099/375342 consumed_samples: 284774400 total_loss: 3.196 time: 0.3440 s/iter data_time: 0.2223 s/iter total_throughput: 2976.91 samples/s lr: 1.65e-04 [09/23 06:07:57] lb.utils.events INFO: eta: 4:12:28 iteration: 278199/375342 consumed_samples: 284876800 total_loss: 3.198 time: 0.3440 s/iter data_time: 0.2080 s/iter total_throughput: 2976.91 samples/s lr: 1.65e-04 [09/23 06:08:32] lb.utils.events INFO: eta: 4:11:11 iteration: 278299/375342 consumed_samples: 284979200 total_loss: 3.195 time: 0.3440 s/iter data_time: 0.2425 s/iter total_throughput: 2976.87 samples/s lr: 1.65e-04 [09/23 06:09:07] lb.utils.events INFO: eta: 4:10:47 iteration: 278399/375342 consumed_samples: 285081600 total_loss: 3.206 time: 0.3440 s/iter data_time: 0.2336 s/iter total_throughput: 2976.86 samples/s lr: 1.64e-04 [09/23 06:09:41] lb.utils.events INFO: eta: 4:11:01 iteration: 278499/375342 consumed_samples: 285184000 total_loss: 3.224 time: 0.3440 s/iter data_time: 0.2093 s/iter total_throughput: 2976.87 samples/s lr: 1.64e-04 [09/23 06:10:16] lb.utils.events INFO: eta: 4:11:15 iteration: 278599/375342 consumed_samples: 285286400 total_loss: 3.225 time: 0.3440 s/iter data_time: 0.2161 s/iter total_throughput: 2976.86 samples/s lr: 1.64e-04 [09/23 06:10:50] lb.utils.events INFO: eta: 4:10:55 iteration: 278699/375342 consumed_samples: 285388800 total_loss: 3.214 time: 0.3440 s/iter data_time: 0.2215 s/iter total_throughput: 2976.86 samples/s lr: 1.63e-04 [09/23 06:11:25] lb.utils.events INFO: eta: 4:10:50 iteration: 278799/375342 consumed_samples: 285491200 total_loss: 3.202 time: 0.3440 s/iter data_time: 0.2216 s/iter total_throughput: 2976.85 samples/s lr: 1.63e-04 [09/23 06:11:59] lb.utils.events INFO: eta: 4:11:29 iteration: 278899/375342 consumed_samples: 285593600 total_loss: 3.199 time: 0.3440 s/iter data_time: 0.2174 s/iter total_throughput: 2976.85 samples/s lr: 1.63e-04 [09/23 06:12:33] lb.utils.events INFO: eta: 4:10:19 iteration: 278999/375342 consumed_samples: 285696000 total_loss: 3.203 time: 0.3440 s/iter data_time: 0.2090 s/iter total_throughput: 2976.87 samples/s lr: 1.62e-04 [09/23 06:13:07] lb.utils.events INFO: eta: 4:10:24 iteration: 279099/375342 consumed_samples: 285798400 total_loss: 3.182 time: 0.3440 s/iter data_time: 0.2083 s/iter total_throughput: 2976.88 samples/s lr: 1.62e-04 [09/23 06:13:41] lb.utils.events INFO: eta: 4:10:47 iteration: 279199/375342 consumed_samples: 285900800 total_loss: 3.174 time: 0.3440 s/iter data_time: 0.2216 s/iter total_throughput: 2976.88 samples/s lr: 1.62e-04 [09/23 06:14:16] lb.utils.events INFO: eta: 4:10:16 iteration: 279299/375342 consumed_samples: 286003200 total_loss: 3.186 time: 0.3440 s/iter data_time: 0.2247 s/iter total_throughput: 2976.87 samples/s lr: 1.62e-04 [09/23 06:14:51] lb.utils.events INFO: eta: 4:10:22 iteration: 279399/375342 consumed_samples: 286105600 total_loss: 3.195 time: 0.3440 s/iter data_time: 0.2256 s/iter total_throughput: 2976.87 samples/s lr: 1.61e-04 [09/23 06:15:25] lb.utils.events INFO: eta: 4:10:07 iteration: 279499/375342 consumed_samples: 286208000 total_loss: 3.201 time: 0.3440 s/iter data_time: 0.2147 s/iter total_throughput: 2976.87 samples/s lr: 1.61e-04 [09/23 06:15:59] lb.utils.events INFO: eta: 4:09:49 iteration: 279599/375342 consumed_samples: 286310400 total_loss: 3.199 time: 0.3440 s/iter data_time: 0.2164 s/iter total_throughput: 2976.88 samples/s lr: 1.61e-04 [09/23 06:16:33] lb.utils.events INFO: eta: 4:09:41 iteration: 279699/375342 consumed_samples: 286412800 total_loss: 3.186 time: 0.3440 s/iter data_time: 0.2221 s/iter total_throughput: 2976.89 samples/s lr: 1.60e-04 [09/23 06:17:08] lb.utils.events INFO: eta: 4:09:26 iteration: 279799/375342 consumed_samples: 286515200 total_loss: 3.202 time: 0.3440 s/iter data_time: 0.2210 s/iter total_throughput: 2976.89 samples/s lr: 1.60e-04 [09/23 06:17:42] lb.utils.events INFO: eta: 4:08:42 iteration: 279899/375342 consumed_samples: 286617600 total_loss: 3.202 time: 0.3440 s/iter data_time: 0.2210 s/iter total_throughput: 2976.89 samples/s lr: 1.60e-04 [09/23 06:18:16] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0279999 [09/23 06:18:16] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/23 06:18:16] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/23 06:18:21] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.1043 s/iter. Inference: 0.1641 s/iter. Eval: 0.0021 s/iter. Total: 0.2705 s/iter. ETA=0:00:10 [09/23 06:18:26] lb.evaluation.evaluator INFO: Inference done 28672/50000. Dataloading: 0.1248 s/iter. Inference: 0.1632 s/iter. Eval: 0.0020 s/iter. Total: 0.2901 s/iter. ETA=0:00:05 [09/23 06:18:31] lb.evaluation.evaluator INFO: Inference done 46080/50000. Dataloading: 0.1293 s/iter. Inference: 0.1626 s/iter. Eval: 0.0020 s/iter. Total: 0.2941 s/iter. ETA=0:00:00 [09/23 06:18:32] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/23 06:18:32] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.700771 (0.000254 s / iter per device, on 8 devices) [09/23 06:18:32] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000143 s / iter per device, on 8 devices) [09/23 06:18:32] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/23 06:18:32] lb.evaluation.utils INFO: copypaste: Acc@1=78.35600000000001 [09/23 06:18:32] lb.evaluation.utils INFO: copypaste: Acc@5=94.002 [09/23 06:18:32] lb.engine.hooks INFO: Not saving as latest eval score for Acc@1 is 78.35600, not better than best score 78.35800 @ iteration 274999. [09/23 06:18:32] lb.utils.events INFO: eta: 4:08:34 iteration: 279999/375342 consumed_samples: 286720000 total_loss: 3.194 time: 0.3440 s/iter data_time: 0.2119 s/iter total_throughput: 2976.91 samples/s lr: 1.59e-04 [09/23 06:19:04] lb.utils.events INFO: eta: 4:08:35 iteration: 280099/375342 consumed_samples: 286822400 total_loss: 3.204 time: 0.3440 s/iter data_time: 0.2149 s/iter total_throughput: 2976.97 samples/s lr: 1.59e-04 [09/23 06:19:39] lb.utils.events INFO: eta: 4:08:26 iteration: 280199/375342 consumed_samples: 286924800 total_loss: 3.209 time: 0.3440 s/iter data_time: 0.2192 s/iter total_throughput: 2976.97 samples/s lr: 1.59e-04 [09/23 06:20:13] lb.utils.events INFO: eta: 4:08:14 iteration: 280299/375342 consumed_samples: 287027200 total_loss: 3.204 time: 0.3440 s/iter data_time: 0.2198 s/iter total_throughput: 2976.96 samples/s lr: 1.59e-04 [09/23 06:20:48] lb.utils.events INFO: eta: 4:07:31 iteration: 280399/375342 consumed_samples: 287129600 total_loss: 3.179 time: 0.3440 s/iter data_time: 0.2216 s/iter total_throughput: 2976.94 samples/s lr: 1.58e-04 [09/23 06:21:23] lb.utils.events INFO: eta: 4:07:09 iteration: 280499/375342 consumed_samples: 287232000 total_loss: 3.174 time: 0.3440 s/iter data_time: 0.2161 s/iter total_throughput: 2976.92 samples/s lr: 1.58e-04 [09/23 06:21:59] lb.utils.events INFO: eta: 4:06:52 iteration: 280599/375342 consumed_samples: 287334400 total_loss: 3.17 time: 0.3440 s/iter data_time: 0.2297 s/iter total_throughput: 2976.90 samples/s lr: 1.58e-04 [09/23 06:22:34] lb.utils.events INFO: eta: 4:06:24 iteration: 280699/375342 consumed_samples: 287436800 total_loss: 3.18 time: 0.3440 s/iter data_time: 0.2301 s/iter total_throughput: 2976.87 samples/s lr: 1.57e-04 [09/23 06:23:09] lb.utils.events INFO: eta: 4:05:19 iteration: 280799/375342 consumed_samples: 287539200 total_loss: 3.192 time: 0.3440 s/iter data_time: 0.2304 s/iter total_throughput: 2976.84 samples/s lr: 1.57e-04 [09/23 06:23:44] lb.utils.events INFO: eta: 4:05:42 iteration: 280899/375342 consumed_samples: 287641600 total_loss: 3.198 time: 0.3440 s/iter data_time: 0.2175 s/iter total_throughput: 2976.83 samples/s lr: 1.57e-04 [09/23 06:24:19] lb.utils.events INFO: eta: 4:05:38 iteration: 280999/375342 consumed_samples: 287744000 total_loss: 3.193 time: 0.3440 s/iter data_time: 0.2083 s/iter total_throughput: 2976.83 samples/s lr: 1.56e-04 [09/23 06:24:53] lb.utils.events INFO: eta: 4:05:03 iteration: 281099/375342 consumed_samples: 287846400 total_loss: 3.178 time: 0.3440 s/iter data_time: 0.2136 s/iter total_throughput: 2976.81 samples/s lr: 1.56e-04 [09/23 06:25:28] lb.utils.events INFO: eta: 4:04:21 iteration: 281199/375342 consumed_samples: 287948800 total_loss: 3.178 time: 0.3440 s/iter data_time: 0.2190 s/iter total_throughput: 2976.81 samples/s lr: 1.56e-04 [09/23 06:26:03] lb.utils.events INFO: eta: 4:05:10 iteration: 281299/375342 consumed_samples: 288051200 total_loss: 3.184 time: 0.3440 s/iter data_time: 0.2219 s/iter total_throughput: 2976.79 samples/s lr: 1.56e-04 [09/23 06:26:38] lb.utils.events INFO: eta: 4:05:51 iteration: 281399/375342 consumed_samples: 288153600 total_loss: 3.197 time: 0.3440 s/iter data_time: 0.2162 s/iter total_throughput: 2976.77 samples/s lr: 1.55e-04 [09/23 06:27:14] lb.utils.events INFO: eta: 4:04:55 iteration: 281499/375342 consumed_samples: 288256000 total_loss: 3.202 time: 0.3440 s/iter data_time: 0.2213 s/iter total_throughput: 2976.74 samples/s lr: 1.55e-04 [09/23 06:27:49] lb.utils.events INFO: eta: 4:04:12 iteration: 281599/375342 consumed_samples: 288358400 total_loss: 3.178 time: 0.3440 s/iter data_time: 0.2244 s/iter total_throughput: 2976.72 samples/s lr: 1.55e-04 [09/23 06:28:23] lb.utils.events INFO: eta: 4:04:06 iteration: 281699/375342 consumed_samples: 288460800 total_loss: 3.169 time: 0.3440 s/iter data_time: 0.2150 s/iter total_throughput: 2976.71 samples/s lr: 1.54e-04 [09/23 06:28:59] lb.utils.events INFO: eta: 4:04:52 iteration: 281799/375342 consumed_samples: 288563200 total_loss: 3.178 time: 0.3440 s/iter data_time: 0.2502 s/iter total_throughput: 2976.67 samples/s lr: 1.54e-04 [09/23 06:29:34] lb.utils.events INFO: eta: 4:03:31 iteration: 281899/375342 consumed_samples: 288665600 total_loss: 3.192 time: 0.3440 s/iter data_time: 0.2207 s/iter total_throughput: 2976.66 samples/s lr: 1.54e-04 [09/23 06:30:09] lb.utils.events INFO: eta: 4:02:57 iteration: 281999/375342 consumed_samples: 288768000 total_loss: 3.184 time: 0.3440 s/iter data_time: 0.2236 s/iter total_throughput: 2976.64 samples/s lr: 1.54e-04 [09/23 06:30:43] lb.utils.events INFO: eta: 4:02:44 iteration: 282099/375342 consumed_samples: 288870400 total_loss: 3.179 time: 0.3440 s/iter data_time: 0.2193 s/iter total_throughput: 2976.64 samples/s lr: 1.53e-04 [09/23 06:31:18] lb.utils.events INFO: eta: 4:02:57 iteration: 282199/375342 consumed_samples: 288972800 total_loss: 3.186 time: 0.3440 s/iter data_time: 0.2295 s/iter total_throughput: 2976.62 samples/s lr: 1.53e-04 [09/23 06:31:53] lb.utils.events INFO: eta: 4:01:46 iteration: 282299/375342 consumed_samples: 289075200 total_loss: 3.184 time: 0.3440 s/iter data_time: 0.2228 s/iter total_throughput: 2976.60 samples/s lr: 1.53e-04 [09/23 06:32:28] lb.utils.events INFO: eta: 4:00:30 iteration: 282399/375342 consumed_samples: 289177600 total_loss: 3.199 time: 0.3440 s/iter data_time: 0.2184 s/iter total_throughput: 2976.58 samples/s lr: 1.52e-04 [09/23 06:33:03] lb.utils.events INFO: eta: 4:00:38 iteration: 282499/375342 consumed_samples: 289280000 total_loss: 3.2 time: 0.3440 s/iter data_time: 0.2299 s/iter total_throughput: 2976.56 samples/s lr: 1.52e-04 [09/23 06:33:38] lb.utils.events INFO: eta: 4:00:23 iteration: 282599/375342 consumed_samples: 289382400 total_loss: 3.18 time: 0.3440 s/iter data_time: 0.2136 s/iter total_throughput: 2976.55 samples/s lr: 1.52e-04 [09/23 06:34:13] lb.utils.events INFO: eta: 3:59:58 iteration: 282699/375342 consumed_samples: 289484800 total_loss: 3.18 time: 0.3440 s/iter data_time: 0.2107 s/iter total_throughput: 2976.54 samples/s lr: 1.52e-04 [09/23 06:34:48] lb.utils.events INFO: eta: 4:00:14 iteration: 282799/375342 consumed_samples: 289587200 total_loss: 3.188 time: 0.3440 s/iter data_time: 0.2280 s/iter total_throughput: 2976.53 samples/s lr: 1.51e-04 [09/23 06:35:22] lb.utils.events INFO: eta: 3:59:39 iteration: 282899/375342 consumed_samples: 289689600 total_loss: 3.186 time: 0.3440 s/iter data_time: 0.2225 s/iter total_throughput: 2976.52 samples/s lr: 1.51e-04 [09/23 06:35:57] lb.utils.events INFO: eta: 3:58:56 iteration: 282999/375342 consumed_samples: 289792000 total_loss: 3.177 time: 0.3440 s/iter data_time: 0.2281 s/iter total_throughput: 2976.50 samples/s lr: 1.51e-04 [09/23 06:36:32] lb.utils.events INFO: eta: 3:58:27 iteration: 283099/375342 consumed_samples: 289894400 total_loss: 3.174 time: 0.3440 s/iter data_time: 0.2238 s/iter total_throughput: 2976.48 samples/s lr: 1.50e-04 [09/23 06:37:08] lb.utils.events INFO: eta: 3:58:04 iteration: 283199/375342 consumed_samples: 289996800 total_loss: 3.162 time: 0.3440 s/iter data_time: 0.2309 s/iter total_throughput: 2976.46 samples/s lr: 1.50e-04 [09/23 06:37:42] lb.utils.events INFO: eta: 3:57:56 iteration: 283299/375342 consumed_samples: 290099200 total_loss: 3.165 time: 0.3440 s/iter data_time: 0.2205 s/iter total_throughput: 2976.45 samples/s lr: 1.50e-04 [09/23 06:38:17] lb.utils.events INFO: eta: 3:57:57 iteration: 283399/375342 consumed_samples: 290201600 total_loss: 3.204 time: 0.3440 s/iter data_time: 0.2172 s/iter total_throughput: 2976.43 samples/s lr: 1.49e-04 [09/23 06:38:52] lb.utils.events INFO: eta: 3:57:53 iteration: 283499/375342 consumed_samples: 290304000 total_loss: 3.187 time: 0.3440 s/iter data_time: 0.2207 s/iter total_throughput: 2976.42 samples/s lr: 1.49e-04 [09/23 06:39:27] lb.utils.events INFO: eta: 3:57:22 iteration: 283599/375342 consumed_samples: 290406400 total_loss: 3.169 time: 0.3440 s/iter data_time: 0.2152 s/iter total_throughput: 2976.41 samples/s lr: 1.49e-04 [09/23 06:40:01] lb.utils.events INFO: eta: 3:57:22 iteration: 283699/375342 consumed_samples: 290508800 total_loss: 3.19 time: 0.3440 s/iter data_time: 0.2180 s/iter total_throughput: 2976.40 samples/s lr: 1.49e-04 [09/23 06:40:36] lb.utils.events INFO: eta: 3:57:15 iteration: 283799/375342 consumed_samples: 290611200 total_loss: 3.194 time: 0.3440 s/iter data_time: 0.2175 s/iter total_throughput: 2976.40 samples/s lr: 1.48e-04 [09/23 06:41:11] lb.utils.events INFO: eta: 3:56:57 iteration: 283899/375342 consumed_samples: 290713600 total_loss: 3.186 time: 0.3440 s/iter data_time: 0.2149 s/iter total_throughput: 2976.37 samples/s lr: 1.48e-04 [09/23 06:41:47] lb.utils.events INFO: eta: 3:57:06 iteration: 283999/375342 consumed_samples: 290816000 total_loss: 3.197 time: 0.3440 s/iter data_time: 0.2312 s/iter total_throughput: 2976.34 samples/s lr: 1.48e-04 [09/23 06:42:22] lb.utils.events INFO: eta: 3:57:18 iteration: 284099/375342 consumed_samples: 290918400 total_loss: 3.192 time: 0.3440 s/iter data_time: 0.2253 s/iter total_throughput: 2976.32 samples/s lr: 1.47e-04 [09/23 06:42:57] lb.utils.events INFO: eta: 3:56:48 iteration: 284199/375342 consumed_samples: 291020800 total_loss: 3.174 time: 0.3441 s/iter data_time: 0.2190 s/iter total_throughput: 2976.30 samples/s lr: 1.47e-04 [09/23 06:43:32] lb.utils.events INFO: eta: 3:56:22 iteration: 284299/375342 consumed_samples: 291123200 total_loss: 3.176 time: 0.3441 s/iter data_time: 0.2352 s/iter total_throughput: 2976.28 samples/s lr: 1.47e-04 [09/23 06:44:07] lb.utils.events INFO: eta: 3:56:28 iteration: 284399/375342 consumed_samples: 291225600 total_loss: 3.175 time: 0.3441 s/iter data_time: 0.2353 s/iter total_throughput: 2976.27 samples/s lr: 1.47e-04 [09/23 06:44:42] lb.utils.events INFO: eta: 3:55:39 iteration: 284499/375342 consumed_samples: 291328000 total_loss: 3.182 time: 0.3441 s/iter data_time: 0.2199 s/iter total_throughput: 2976.24 samples/s lr: 1.46e-04 [09/23 06:45:17] lb.utils.events INFO: eta: 3:55:45 iteration: 284599/375342 consumed_samples: 291430400 total_loss: 3.166 time: 0.3441 s/iter data_time: 0.2222 s/iter total_throughput: 2976.23 samples/s lr: 1.46e-04 [09/23 06:45:52] lb.utils.events INFO: eta: 3:55:18 iteration: 284699/375342 consumed_samples: 291532800 total_loss: 3.166 time: 0.3441 s/iter data_time: 0.2177 s/iter total_throughput: 2976.22 samples/s lr: 1.46e-04 [09/23 06:46:26] lb.utils.events INFO: eta: 3:55:06 iteration: 284799/375342 consumed_samples: 291635200 total_loss: 3.195 time: 0.3441 s/iter data_time: 0.2194 s/iter total_throughput: 2976.21 samples/s lr: 1.45e-04 [09/23 06:47:01] lb.utils.events INFO: eta: 3:54:53 iteration: 284899/375342 consumed_samples: 291737600 total_loss: 3.195 time: 0.3441 s/iter data_time: 0.2113 s/iter total_throughput: 2976.20 samples/s lr: 1.45e-04 [09/23 06:47:36] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0284999 [09/23 06:47:37] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/23 06:47:37] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/23 06:47:41] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0811 s/iter. Inference: 0.1619 s/iter. Eval: 0.0020 s/iter. Total: 0.2450 s/iter. ETA=0:00:09 [09/23 06:47:47] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1309 s/iter. Inference: 0.1677 s/iter. Eval: 0.0020 s/iter. Total: 0.3006 s/iter. ETA=0:00:05 [09/23 06:47:52] lb.evaluation.evaluator INFO: Inference done 50000/50000. Dataloading: 0.1193 s/iter. Inference: 0.1644 s/iter. Eval: 0.0020 s/iter. Total: 0.2858 s/iter. ETA=0:00:00 [09/23 06:47:52] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/23 06:47:52] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.578605 (0.000252 s / iter per device, on 8 devices) [09/23 06:47:52] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000145 s / iter per device, on 8 devices) [09/23 06:47:52] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/23 06:47:52] lb.evaluation.utils INFO: copypaste: Acc@1=78.71000000000001 [09/23 06:47:52] lb.evaluation.utils INFO: copypaste: Acc@5=94.184 [09/23 06:47:52] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 78.71000, better than last best score 78.35800 @ iteration 274999. [09/23 06:47:52] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/23 06:47:53] lb.utils.events INFO: eta: 3:54:13 iteration: 284999/375342 consumed_samples: 291840000 total_loss: 3.191 time: 0.3441 s/iter data_time: 0.2424 s/iter total_throughput: 2976.18 samples/s lr: 1.45e-04 [09/23 06:48:26] lb.utils.events INFO: eta: 3:54:36 iteration: 285099/375342 consumed_samples: 291942400 total_loss: 3.186 time: 0.3441 s/iter data_time: 0.2246 s/iter total_throughput: 2976.22 samples/s lr: 1.45e-04 [09/23 06:49:01] lb.utils.events INFO: eta: 3:54:31 iteration: 285199/375342 consumed_samples: 292044800 total_loss: 3.179 time: 0.3441 s/iter data_time: 0.2298 s/iter total_throughput: 2976.20 samples/s lr: 1.44e-04 [09/23 06:49:36] lb.utils.events INFO: eta: 3:54:30 iteration: 285299/375342 consumed_samples: 292147200 total_loss: 3.173 time: 0.3441 s/iter data_time: 0.2220 s/iter total_throughput: 2976.18 samples/s lr: 1.44e-04 [09/23 06:50:11] lb.utils.events INFO: eta: 3:54:10 iteration: 285399/375342 consumed_samples: 292249600 total_loss: 3.164 time: 0.3441 s/iter data_time: 0.2226 s/iter total_throughput: 2976.17 samples/s lr: 1.44e-04 [09/23 06:50:46] lb.utils.events INFO: eta: 3:54:10 iteration: 285499/375342 consumed_samples: 292352000 total_loss: 3.182 time: 0.3441 s/iter data_time: 0.2091 s/iter total_throughput: 2976.16 samples/s lr: 1.43e-04 [09/23 06:51:21] lb.utils.events INFO: eta: 3:53:47 iteration: 285599/375342 consumed_samples: 292454400 total_loss: 3.19 time: 0.3441 s/iter data_time: 0.2204 s/iter total_throughput: 2976.14 samples/s lr: 1.43e-04 [09/23 06:51:55] lb.utils.events INFO: eta: 3:53:47 iteration: 285699/375342 consumed_samples: 292556800 total_loss: 3.178 time: 0.3441 s/iter data_time: 0.2156 s/iter total_throughput: 2976.13 samples/s lr: 1.43e-04 [09/23 06:52:30] lb.utils.events INFO: eta: 3:53:43 iteration: 285799/375342 consumed_samples: 292659200 total_loss: 3.168 time: 0.3441 s/iter data_time: 0.2184 s/iter total_throughput: 2976.13 samples/s lr: 1.43e-04 [09/23 06:53:05] lb.utils.events INFO: eta: 3:53:04 iteration: 285899/375342 consumed_samples: 292761600 total_loss: 3.177 time: 0.3441 s/iter data_time: 0.2281 s/iter total_throughput: 2976.11 samples/s lr: 1.42e-04 [09/23 06:53:40] lb.utils.events INFO: eta: 3:52:27 iteration: 285999/375342 consumed_samples: 292864000 total_loss: 3.185 time: 0.3441 s/iter data_time: 0.2226 s/iter total_throughput: 2976.10 samples/s lr: 1.42e-04 [09/23 06:54:14] lb.utils.events INFO: eta: 3:51:37 iteration: 286099/375342 consumed_samples: 292966400 total_loss: 3.164 time: 0.3441 s/iter data_time: 0.2168 s/iter total_throughput: 2976.09 samples/s lr: 1.42e-04 [09/23 06:54:49] lb.utils.events INFO: eta: 3:51:16 iteration: 286199/375342 consumed_samples: 293068800 total_loss: 3.159 time: 0.3441 s/iter data_time: 0.2329 s/iter total_throughput: 2976.07 samples/s lr: 1.42e-04 [09/23 06:55:25] lb.utils.events INFO: eta: 3:50:43 iteration: 286299/375342 consumed_samples: 293171200 total_loss: 3.177 time: 0.3441 s/iter data_time: 0.2269 s/iter total_throughput: 2976.04 samples/s lr: 1.41e-04 [09/23 06:55:59] lb.utils.events INFO: eta: 3:50:35 iteration: 286399/375342 consumed_samples: 293273600 total_loss: 3.182 time: 0.3441 s/iter data_time: 0.2185 s/iter total_throughput: 2976.04 samples/s lr: 1.41e-04 [09/23 06:56:34] lb.utils.events INFO: eta: 3:50:42 iteration: 286499/375342 consumed_samples: 293376000 total_loss: 3.166 time: 0.3441 s/iter data_time: 0.2160 s/iter total_throughput: 2976.04 samples/s lr: 1.41e-04 [09/23 06:57:09] lb.utils.events INFO: eta: 3:50:32 iteration: 286599/375342 consumed_samples: 293478400 total_loss: 3.148 time: 0.3441 s/iter data_time: 0.2291 s/iter total_throughput: 2976.02 samples/s lr: 1.40e-04 [09/23 06:57:44] lb.utils.events INFO: eta: 3:50:06 iteration: 286699/375342 consumed_samples: 293580800 total_loss: 3.146 time: 0.3441 s/iter data_time: 0.2296 s/iter total_throughput: 2976.01 samples/s lr: 1.40e-04 [09/23 06:58:19] lb.utils.events INFO: eta: 3:49:39 iteration: 286799/375342 consumed_samples: 293683200 total_loss: 3.16 time: 0.3441 s/iter data_time: 0.2317 s/iter total_throughput: 2976.00 samples/s lr: 1.40e-04 [09/23 06:58:53] lb.utils.events INFO: eta: 3:49:30 iteration: 286899/375342 consumed_samples: 293785600 total_loss: 3.167 time: 0.3441 s/iter data_time: 0.2161 s/iter total_throughput: 2975.98 samples/s lr: 1.40e-04 [09/23 06:59:28] lb.utils.events INFO: eta: 3:49:55 iteration: 286999/375342 consumed_samples: 293888000 total_loss: 3.171 time: 0.3441 s/iter data_time: 0.2138 s/iter total_throughput: 2975.98 samples/s lr: 1.39e-04 [09/23 07:00:03] lb.utils.events INFO: eta: 3:49:47 iteration: 287099/375342 consumed_samples: 293990400 total_loss: 3.152 time: 0.3441 s/iter data_time: 0.2222 s/iter total_throughput: 2975.97 samples/s lr: 1.39e-04 [09/23 07:00:38] lb.utils.events INFO: eta: 3:49:37 iteration: 287199/375342 consumed_samples: 294092800 total_loss: 3.157 time: 0.3441 s/iter data_time: 0.2211 s/iter total_throughput: 2975.95 samples/s lr: 1.39e-04 [09/23 07:01:13] lb.utils.events INFO: eta: 3:49:37 iteration: 287299/375342 consumed_samples: 294195200 total_loss: 3.178 time: 0.3441 s/iter data_time: 0.2164 s/iter total_throughput: 2975.93 samples/s lr: 1.38e-04 [09/23 07:01:48] lb.utils.events INFO: eta: 3:49:05 iteration: 287399/375342 consumed_samples: 294297600 total_loss: 3.166 time: 0.3441 s/iter data_time: 0.2197 s/iter total_throughput: 2975.91 samples/s lr: 1.38e-04 [09/23 07:02:23] lb.utils.events INFO: eta: 3:48:42 iteration: 287499/375342 consumed_samples: 294400000 total_loss: 3.151 time: 0.3441 s/iter data_time: 0.2144 s/iter total_throughput: 2975.89 samples/s lr: 1.38e-04 [09/23 07:02:58] lb.utils.events INFO: eta: 3:48:49 iteration: 287599/375342 consumed_samples: 294502400 total_loss: 3.18 time: 0.3441 s/iter data_time: 0.2194 s/iter total_throughput: 2975.87 samples/s lr: 1.38e-04 [09/23 07:03:33] lb.utils.events INFO: eta: 3:48:18 iteration: 287699/375342 consumed_samples: 294604800 total_loss: 3.185 time: 0.3441 s/iter data_time: 0.2219 s/iter total_throughput: 2975.86 samples/s lr: 1.37e-04 [09/23 07:04:08] lb.utils.events INFO: eta: 3:48:03 iteration: 287799/375342 consumed_samples: 294707200 total_loss: 3.172 time: 0.3441 s/iter data_time: 0.2242 s/iter total_throughput: 2975.84 samples/s lr: 1.37e-04 [09/23 07:04:43] lb.utils.events INFO: eta: 3:47:33 iteration: 287899/375342 consumed_samples: 294809600 total_loss: 3.136 time: 0.3441 s/iter data_time: 0.2310 s/iter total_throughput: 2975.83 samples/s lr: 1.37e-04 [09/23 07:05:18] lb.utils.events INFO: eta: 3:46:51 iteration: 287999/375342 consumed_samples: 294912000 total_loss: 3.161 time: 0.3441 s/iter data_time: 0.2190 s/iter total_throughput: 2975.81 samples/s lr: 1.36e-04 [09/23 07:05:53] lb.utils.events INFO: eta: 3:46:34 iteration: 288099/375342 consumed_samples: 295014400 total_loss: 3.181 time: 0.3441 s/iter data_time: 0.2219 s/iter total_throughput: 2975.78 samples/s lr: 1.36e-04 [09/23 07:06:28] lb.utils.events INFO: eta: 3:46:28 iteration: 288199/375342 consumed_samples: 295116800 total_loss: 3.174 time: 0.3441 s/iter data_time: 0.2354 s/iter total_throughput: 2975.77 samples/s lr: 1.36e-04 [09/23 07:07:02] lb.utils.events INFO: eta: 3:46:21 iteration: 288299/375342 consumed_samples: 295219200 total_loss: 3.167 time: 0.3441 s/iter data_time: 0.2186 s/iter total_throughput: 2975.77 samples/s lr: 1.36e-04 [09/23 07:07:37] lb.utils.events INFO: eta: 3:45:51 iteration: 288399/375342 consumed_samples: 295321600 total_loss: 3.171 time: 0.3441 s/iter data_time: 0.2162 s/iter total_throughput: 2975.75 samples/s lr: 1.35e-04 [09/23 07:08:12] lb.utils.events INFO: eta: 3:45:32 iteration: 288499/375342 consumed_samples: 295424000 total_loss: 3.153 time: 0.3441 s/iter data_time: 0.2198 s/iter total_throughput: 2975.73 samples/s lr: 1.35e-04 [09/23 07:08:47] lb.utils.events INFO: eta: 3:45:04 iteration: 288599/375342 consumed_samples: 295526400 total_loss: 3.146 time: 0.3441 s/iter data_time: 0.2188 s/iter total_throughput: 2975.72 samples/s lr: 1.35e-04 [09/23 07:09:22] lb.utils.events INFO: eta: 3:44:44 iteration: 288699/375342 consumed_samples: 295628800 total_loss: 3.151 time: 0.3441 s/iter data_time: 0.2207 s/iter total_throughput: 2975.71 samples/s lr: 1.35e-04 [09/23 07:09:57] lb.utils.events INFO: eta: 3:44:36 iteration: 288799/375342 consumed_samples: 295731200 total_loss: 3.155 time: 0.3441 s/iter data_time: 0.2144 s/iter total_throughput: 2975.69 samples/s lr: 1.34e-04 [09/23 07:10:32] lb.utils.events INFO: eta: 3:44:29 iteration: 288899/375342 consumed_samples: 295833600 total_loss: 3.151 time: 0.3441 s/iter data_time: 0.2358 s/iter total_throughput: 2975.68 samples/s lr: 1.34e-04 [09/23 07:11:06] lb.utils.events INFO: eta: 3:44:27 iteration: 288999/375342 consumed_samples: 295936000 total_loss: 3.164 time: 0.3441 s/iter data_time: 0.2204 s/iter total_throughput: 2975.69 samples/s lr: 1.34e-04 [09/23 07:11:40] lb.utils.events INFO: eta: 3:44:11 iteration: 289099/375342 consumed_samples: 296038400 total_loss: 3.169 time: 0.3441 s/iter data_time: 0.2137 s/iter total_throughput: 2975.69 samples/s lr: 1.33e-04 [09/23 07:12:15] lb.utils.events INFO: eta: 3:43:48 iteration: 289199/375342 consumed_samples: 296140800 total_loss: 3.135 time: 0.3441 s/iter data_time: 0.2334 s/iter total_throughput: 2975.67 samples/s lr: 1.33e-04 [09/23 07:12:50] lb.utils.events INFO: eta: 3:43:26 iteration: 289299/375342 consumed_samples: 296243200 total_loss: 3.152 time: 0.3441 s/iter data_time: 0.2143 s/iter total_throughput: 2975.65 samples/s lr: 1.33e-04 [09/23 07:13:25] lb.utils.events INFO: eta: 3:43:22 iteration: 289399/375342 consumed_samples: 296345600 total_loss: 3.157 time: 0.3441 s/iter data_time: 0.2157 s/iter total_throughput: 2975.65 samples/s lr: 1.33e-04 [09/23 07:14:00] lb.utils.events INFO: eta: 3:43:18 iteration: 289499/375342 consumed_samples: 296448000 total_loss: 3.146 time: 0.3441 s/iter data_time: 0.2091 s/iter total_throughput: 2975.64 samples/s lr: 1.32e-04 [09/23 07:14:34] lb.utils.events INFO: eta: 3:43:26 iteration: 289599/375342 consumed_samples: 296550400 total_loss: 3.137 time: 0.3441 s/iter data_time: 0.2244 s/iter total_throughput: 2975.63 samples/s lr: 1.32e-04 [09/23 07:15:09] lb.utils.events INFO: eta: 3:43:33 iteration: 289699/375342 consumed_samples: 296652800 total_loss: 3.141 time: 0.3441 s/iter data_time: 0.2135 s/iter total_throughput: 2975.64 samples/s lr: 1.32e-04 [09/23 07:15:43] lb.utils.events INFO: eta: 3:42:27 iteration: 289799/375342 consumed_samples: 296755200 total_loss: 3.155 time: 0.3441 s/iter data_time: 0.2083 s/iter total_throughput: 2975.64 samples/s lr: 1.32e-04 [09/23 07:16:18] lb.utils.events INFO: eta: 3:41:45 iteration: 289899/375342 consumed_samples: 296857600 total_loss: 3.147 time: 0.3441 s/iter data_time: 0.2206 s/iter total_throughput: 2975.62 samples/s lr: 1.31e-04 [09/23 07:16:53] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0289999 [09/23 07:16:54] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/23 07:16:54] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/23 07:16:58] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0969 s/iter. Inference: 0.1630 s/iter. Eval: 0.0022 s/iter. Total: 0.2621 s/iter. ETA=0:00:09 [09/23 07:17:04] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1168 s/iter. Inference: 0.1763 s/iter. Eval: 0.0022 s/iter. Total: 0.2954 s/iter. ETA=0:00:05 [09/23 07:17:09] lb.evaluation.evaluator INFO: Inference done 48128/50000. Dataloading: 0.1216 s/iter. Inference: 0.1697 s/iter. Eval: 0.0022 s/iter. Total: 0.2936 s/iter. ETA=0:00:00 [09/23 07:17:09] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/23 07:17:09] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.680098 (0.000254 s / iter per device, on 8 devices) [09/23 07:17:09] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000149 s / iter per device, on 8 devices) [09/23 07:17:09] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/23 07:17:09] lb.evaluation.utils INFO: copypaste: Acc@1=78.922 [09/23 07:17:09] lb.evaluation.utils INFO: copypaste: Acc@5=94.192 [09/23 07:17:09] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 78.92200, better than last best score 78.71000 @ iteration 284999. [09/23 07:17:09] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/23 07:17:10] lb.utils.events INFO: eta: 3:40:56 iteration: 289999/375342 consumed_samples: 296960000 total_loss: 3.146 time: 0.3441 s/iter data_time: 0.2307 s/iter total_throughput: 2975.60 samples/s lr: 1.31e-04 [09/23 07:17:43] lb.utils.events INFO: eta: 3:41:09 iteration: 290099/375342 consumed_samples: 297062400 total_loss: 3.149 time: 0.3441 s/iter data_time: 0.2261 s/iter total_throughput: 2975.63 samples/s lr: 1.31e-04 [09/23 07:18:18] lb.utils.events INFO: eta: 3:40:58 iteration: 290199/375342 consumed_samples: 297164800 total_loss: 3.137 time: 0.3441 s/iter data_time: 0.2123 s/iter total_throughput: 2975.63 samples/s lr: 1.30e-04 [09/23 07:18:52] lb.utils.events INFO: eta: 3:41:16 iteration: 290299/375342 consumed_samples: 297267200 total_loss: 3.119 time: 0.3441 s/iter data_time: 0.2256 s/iter total_throughput: 2975.63 samples/s lr: 1.30e-04 [09/23 07:19:27] lb.utils.events INFO: eta: 3:40:37 iteration: 290399/375342 consumed_samples: 297369600 total_loss: 3.124 time: 0.3441 s/iter data_time: 0.2148 s/iter total_throughput: 2975.62 samples/s lr: 1.30e-04 [09/23 07:20:02] lb.utils.events INFO: eta: 3:39:56 iteration: 290499/375342 consumed_samples: 297472000 total_loss: 3.155 time: 0.3441 s/iter data_time: 0.2168 s/iter total_throughput: 2975.60 samples/s lr: 1.30e-04 [09/23 07:20:37] lb.utils.events INFO: eta: 3:39:40 iteration: 290599/375342 consumed_samples: 297574400 total_loss: 3.155 time: 0.3441 s/iter data_time: 0.2171 s/iter total_throughput: 2975.58 samples/s lr: 1.29e-04 [09/23 07:21:12] lb.utils.events INFO: eta: 3:39:10 iteration: 290699/375342 consumed_samples: 297676800 total_loss: 3.144 time: 0.3441 s/iter data_time: 0.2144 s/iter total_throughput: 2975.57 samples/s lr: 1.29e-04 [09/23 07:21:47] lb.utils.events INFO: eta: 3:39:47 iteration: 290799/375342 consumed_samples: 297779200 total_loss: 3.142 time: 0.3441 s/iter data_time: 0.2172 s/iter total_throughput: 2975.56 samples/s lr: 1.29e-04 [09/23 07:22:21] lb.utils.events INFO: eta: 3:39:21 iteration: 290899/375342 consumed_samples: 297881600 total_loss: 3.157 time: 0.3441 s/iter data_time: 0.2177 s/iter total_throughput: 2975.56 samples/s lr: 1.29e-04 [09/23 07:22:56] lb.utils.events INFO: eta: 3:39:56 iteration: 290999/375342 consumed_samples: 297984000 total_loss: 3.156 time: 0.3441 s/iter data_time: 0.2192 s/iter total_throughput: 2975.56 samples/s lr: 1.28e-04 [09/23 07:23:31] lb.utils.events INFO: eta: 3:38:44 iteration: 291099/375342 consumed_samples: 298086400 total_loss: 3.155 time: 0.3441 s/iter data_time: 0.2146 s/iter total_throughput: 2975.54 samples/s lr: 1.28e-04 [09/23 07:24:06] lb.utils.events INFO: eta: 3:37:57 iteration: 291199/375342 consumed_samples: 298188800 total_loss: 3.158 time: 0.3441 s/iter data_time: 0.2314 s/iter total_throughput: 2975.52 samples/s lr: 1.28e-04 [09/23 07:24:40] lb.utils.events INFO: eta: 3:37:22 iteration: 291299/375342 consumed_samples: 298291200 total_loss: 3.163 time: 0.3441 s/iter data_time: 0.2230 s/iter total_throughput: 2975.52 samples/s lr: 1.28e-04 [09/23 07:25:15] lb.utils.events INFO: eta: 3:37:12 iteration: 291399/375342 consumed_samples: 298393600 total_loss: 3.154 time: 0.3441 s/iter data_time: 0.2224 s/iter total_throughput: 2975.50 samples/s lr: 1.27e-04 [09/23 07:25:50] lb.utils.events INFO: eta: 3:37:42 iteration: 291499/375342 consumed_samples: 298496000 total_loss: 3.151 time: 0.3441 s/iter data_time: 0.2203 s/iter total_throughput: 2975.50 samples/s lr: 1.27e-04 [09/23 07:26:24] lb.utils.events INFO: eta: 3:37:48 iteration: 291599/375342 consumed_samples: 298598400 total_loss: 3.157 time: 0.3441 s/iter data_time: 0.2240 s/iter total_throughput: 2975.49 samples/s lr: 1.27e-04 [09/23 07:26:59] lb.utils.events INFO: eta: 3:37:53 iteration: 291699/375342 consumed_samples: 298700800 total_loss: 3.159 time: 0.3441 s/iter data_time: 0.2161 s/iter total_throughput: 2975.49 samples/s lr: 1.26e-04 [09/23 07:27:33] lb.utils.events INFO: eta: 3:37:12 iteration: 291799/375342 consumed_samples: 298803200 total_loss: 3.155 time: 0.3441 s/iter data_time: 0.2207 s/iter total_throughput: 2975.49 samples/s lr: 1.26e-04 [09/23 07:28:08] lb.utils.events INFO: eta: 3:37:02 iteration: 291899/375342 consumed_samples: 298905600 total_loss: 3.137 time: 0.3441 s/iter data_time: 0.2151 s/iter total_throughput: 2975.49 samples/s lr: 1.26e-04 [09/23 07:28:42] lb.utils.events INFO: eta: 3:37:00 iteration: 291999/375342 consumed_samples: 299008000 total_loss: 3.132 time: 0.3441 s/iter data_time: 0.2014 s/iter total_throughput: 2975.49 samples/s lr: 1.26e-04 [09/23 07:29:17] lb.utils.events INFO: eta: 3:36:41 iteration: 292099/375342 consumed_samples: 299110400 total_loss: 3.152 time: 0.3441 s/iter data_time: 0.2235 s/iter total_throughput: 2975.47 samples/s lr: 1.25e-04 [09/23 07:29:52] lb.utils.events INFO: eta: 3:36:34 iteration: 292199/375342 consumed_samples: 299212800 total_loss: 3.143 time: 0.3441 s/iter data_time: 0.2094 s/iter total_throughput: 2975.46 samples/s lr: 1.25e-04 [09/23 07:30:26] lb.utils.events INFO: eta: 3:36:52 iteration: 292299/375342 consumed_samples: 299315200 total_loss: 3.135 time: 0.3441 s/iter data_time: 0.2138 s/iter total_throughput: 2975.46 samples/s lr: 1.25e-04 [09/23 07:31:00] lb.utils.events INFO: eta: 3:36:46 iteration: 292399/375342 consumed_samples: 299417600 total_loss: 3.146 time: 0.3441 s/iter data_time: 0.2136 s/iter total_throughput: 2975.48 samples/s lr: 1.25e-04 [09/23 07:31:35] lb.utils.events INFO: eta: 3:36:03 iteration: 292499/375342 consumed_samples: 299520000 total_loss: 3.148 time: 0.3441 s/iter data_time: 0.2221 s/iter total_throughput: 2975.47 samples/s lr: 1.24e-04 [09/23 07:32:10] lb.utils.events INFO: eta: 3:35:45 iteration: 292599/375342 consumed_samples: 299622400 total_loss: 3.148 time: 0.3441 s/iter data_time: 0.2291 s/iter total_throughput: 2975.46 samples/s lr: 1.24e-04 [09/23 07:32:44] lb.utils.events INFO: eta: 3:35:06 iteration: 292699/375342 consumed_samples: 299724800 total_loss: 3.145 time: 0.3441 s/iter data_time: 0.2149 s/iter total_throughput: 2975.45 samples/s lr: 1.24e-04 [09/23 07:33:18] lb.utils.events INFO: eta: 3:34:55 iteration: 292799/375342 consumed_samples: 299827200 total_loss: 3.146 time: 0.3441 s/iter data_time: 0.2191 s/iter total_throughput: 2975.46 samples/s lr: 1.24e-04 [09/23 07:33:53] lb.utils.events INFO: eta: 3:34:48 iteration: 292899/375342 consumed_samples: 299929600 total_loss: 3.15 time: 0.3441 s/iter data_time: 0.2237 s/iter total_throughput: 2975.46 samples/s lr: 1.23e-04 [09/23 07:34:28] lb.utils.events INFO: eta: 3:34:15 iteration: 292999/375342 consumed_samples: 300032000 total_loss: 3.15 time: 0.3441 s/iter data_time: 0.2207 s/iter total_throughput: 2975.45 samples/s lr: 1.23e-04 [09/23 07:35:02] lb.utils.events INFO: eta: 3:34:25 iteration: 293099/375342 consumed_samples: 300134400 total_loss: 3.155 time: 0.3441 s/iter data_time: 0.2125 s/iter total_throughput: 2975.45 samples/s lr: 1.23e-04 [09/23 07:35:37] lb.utils.events INFO: eta: 3:34:01 iteration: 293199/375342 consumed_samples: 300236800 total_loss: 3.14 time: 0.3442 s/iter data_time: 0.2247 s/iter total_throughput: 2975.44 samples/s lr: 1.22e-04 [09/23 07:36:11] lb.utils.events INFO: eta: 3:33:23 iteration: 293299/375342 consumed_samples: 300339200 total_loss: 3.141 time: 0.3441 s/iter data_time: 0.2071 s/iter total_throughput: 2975.45 samples/s lr: 1.22e-04 [09/23 07:36:45] lb.utils.events INFO: eta: 3:32:46 iteration: 293399/375342 consumed_samples: 300441600 total_loss: 3.142 time: 0.3441 s/iter data_time: 0.2132 s/iter total_throughput: 2975.46 samples/s lr: 1.22e-04 [09/23 07:37:19] lb.utils.events INFO: eta: 3:32:53 iteration: 293499/375342 consumed_samples: 300544000 total_loss: 3.136 time: 0.3441 s/iter data_time: 0.2066 s/iter total_throughput: 2975.47 samples/s lr: 1.22e-04 [09/23 07:37:53] lb.utils.events INFO: eta: 3:32:54 iteration: 293599/375342 consumed_samples: 300646400 total_loss: 3.142 time: 0.3441 s/iter data_time: 0.2182 s/iter total_throughput: 2975.48 samples/s lr: 1.21e-04 [09/23 07:38:27] lb.utils.events INFO: eta: 3:32:45 iteration: 293699/375342 consumed_samples: 300748800 total_loss: 3.131 time: 0.3441 s/iter data_time: 0.2110 s/iter total_throughput: 2975.50 samples/s lr: 1.21e-04 [09/23 07:39:01] lb.utils.events INFO: eta: 3:32:34 iteration: 293799/375342 consumed_samples: 300851200 total_loss: 3.125 time: 0.3441 s/iter data_time: 0.2168 s/iter total_throughput: 2975.51 samples/s lr: 1.21e-04 [09/23 07:39:35] lb.utils.events INFO: eta: 3:32:20 iteration: 293899/375342 consumed_samples: 300953600 total_loss: 3.137 time: 0.3441 s/iter data_time: 0.2181 s/iter total_throughput: 2975.52 samples/s lr: 1.21e-04 [09/23 07:40:10] lb.utils.events INFO: eta: 3:32:03 iteration: 293999/375342 consumed_samples: 301056000 total_loss: 3.165 time: 0.3441 s/iter data_time: 0.2179 s/iter total_throughput: 2975.51 samples/s lr: 1.20e-04 [09/23 07:40:44] lb.utils.events INFO: eta: 3:31:22 iteration: 294099/375342 consumed_samples: 301158400 total_loss: 3.161 time: 0.3441 s/iter data_time: 0.2255 s/iter total_throughput: 2975.50 samples/s lr: 1.20e-04 [09/23 07:41:19] lb.utils.events INFO: eta: 3:31:09 iteration: 294199/375342 consumed_samples: 301260800 total_loss: 3.136 time: 0.3441 s/iter data_time: 0.2166 s/iter total_throughput: 2975.50 samples/s lr: 1.20e-04 [09/23 07:41:54] lb.utils.events INFO: eta: 3:30:59 iteration: 294299/375342 consumed_samples: 301363200 total_loss: 3.125 time: 0.3441 s/iter data_time: 0.2028 s/iter total_throughput: 2975.50 samples/s lr: 1.20e-04 [09/23 07:42:29] lb.utils.events INFO: eta: 3:30:31 iteration: 294399/375342 consumed_samples: 301465600 total_loss: 3.155 time: 0.3441 s/iter data_time: 0.2233 s/iter total_throughput: 2975.48 samples/s lr: 1.19e-04 [09/23 07:43:04] lb.utils.events INFO: eta: 3:29:51 iteration: 294499/375342 consumed_samples: 301568000 total_loss: 3.156 time: 0.3441 s/iter data_time: 0.2209 s/iter total_throughput: 2975.46 samples/s lr: 1.19e-04 [09/23 07:43:39] lb.utils.events INFO: eta: 3:29:19 iteration: 294599/375342 consumed_samples: 301670400 total_loss: 3.127 time: 0.3442 s/iter data_time: 0.2298 s/iter total_throughput: 2975.43 samples/s lr: 1.19e-04 [09/23 07:44:14] lb.utils.events INFO: eta: 3:28:18 iteration: 294699/375342 consumed_samples: 301772800 total_loss: 3.116 time: 0.3442 s/iter data_time: 0.2300 s/iter total_throughput: 2975.41 samples/s lr: 1.19e-04 [09/23 07:44:50] lb.utils.events INFO: eta: 3:27:14 iteration: 294799/375342 consumed_samples: 301875200 total_loss: 3.112 time: 0.3442 s/iter data_time: 0.2418 s/iter total_throughput: 2975.39 samples/s lr: 1.18e-04 [09/23 07:45:25] lb.utils.events INFO: eta: 3:26:56 iteration: 294899/375342 consumed_samples: 301977600 total_loss: 3.133 time: 0.3442 s/iter data_time: 0.2352 s/iter total_throughput: 2975.36 samples/s lr: 1.18e-04 [09/23 07:46:00] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0294999 [09/23 07:46:01] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/23 07:46:01] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/23 07:46:05] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0993 s/iter. Inference: 0.1637 s/iter. Eval: 0.0023 s/iter. Total: 0.2653 s/iter. ETA=0:00:09 [09/23 07:46:10] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1177 s/iter. Inference: 0.1752 s/iter. Eval: 0.0021 s/iter. Total: 0.2950 s/iter. ETA=0:00:05 [09/23 07:46:16] lb.evaluation.evaluator INFO: Inference done 48128/50000. Dataloading: 0.1192 s/iter. Inference: 0.1685 s/iter. Eval: 0.0021 s/iter. Total: 0.2898 s/iter. ETA=0:00:00 [09/23 07:46:16] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/23 07:46:16] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.534134 (0.000251 s / iter per device, on 8 devices) [09/23 07:46:16] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000148 s / iter per device, on 8 devices) [09/23 07:46:16] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/23 07:46:16] lb.evaluation.utils INFO: copypaste: Acc@1=78.9 [09/23 07:46:16] lb.evaluation.utils INFO: copypaste: Acc@5=94.25 [09/23 07:46:16] lb.engine.hooks INFO: Not saving as latest eval score for Acc@1 is 78.90000, not better than best score 78.92200 @ iteration 289999. [09/23 07:46:16] lb.utils.events INFO: eta: 3:26:43 iteration: 294999/375342 consumed_samples: 302080000 total_loss: 3.141 time: 0.3442 s/iter data_time: 0.2157 s/iter total_throughput: 2975.33 samples/s lr: 1.18e-04 [09/23 07:46:49] lb.utils.events INFO: eta: 3:27:10 iteration: 295099/375342 consumed_samples: 302182400 total_loss: 3.127 time: 0.3442 s/iter data_time: 0.2263 s/iter total_throughput: 2975.38 samples/s lr: 1.18e-04 [09/23 07:47:24] lb.utils.events INFO: eta: 3:26:45 iteration: 295199/375342 consumed_samples: 302284800 total_loss: 3.149 time: 0.3442 s/iter data_time: 0.2157 s/iter total_throughput: 2975.36 samples/s lr: 1.17e-04 [09/23 07:47:59] lb.utils.events INFO: eta: 3:26:15 iteration: 295299/375342 consumed_samples: 302387200 total_loss: 3.157 time: 0.3442 s/iter data_time: 0.2403 s/iter total_throughput: 2975.33 samples/s lr: 1.17e-04 [09/23 07:48:34] lb.utils.events INFO: eta: 3:26:22 iteration: 295399/375342 consumed_samples: 302489600 total_loss: 3.138 time: 0.3442 s/iter data_time: 0.2187 s/iter total_throughput: 2975.31 samples/s lr: 1.17e-04 [09/23 07:49:09] lb.utils.events INFO: eta: 3:27:03 iteration: 295499/375342 consumed_samples: 302592000 total_loss: 3.133 time: 0.3442 s/iter data_time: 0.2090 s/iter total_throughput: 2975.30 samples/s lr: 1.16e-04 [09/23 07:49:44] lb.utils.events INFO: eta: 3:27:01 iteration: 295599/375342 consumed_samples: 302694400 total_loss: 3.114 time: 0.3442 s/iter data_time: 0.2242 s/iter total_throughput: 2975.28 samples/s lr: 1.16e-04 [09/23 07:50:19] lb.utils.events INFO: eta: 3:26:44 iteration: 295699/375342 consumed_samples: 302796800 total_loss: 3.102 time: 0.3442 s/iter data_time: 0.2260 s/iter total_throughput: 2975.26 samples/s lr: 1.16e-04 [09/23 07:50:54] lb.utils.events INFO: eta: 3:26:31 iteration: 295799/375342 consumed_samples: 302899200 total_loss: 3.099 time: 0.3442 s/iter data_time: 0.2349 s/iter total_throughput: 2975.25 samples/s lr: 1.16e-04 [09/23 07:51:29] lb.utils.events INFO: eta: 3:26:39 iteration: 295899/375342 consumed_samples: 303001600 total_loss: 3.116 time: 0.3442 s/iter data_time: 0.2236 s/iter total_throughput: 2975.23 samples/s lr: 1.15e-04 [09/23 07:52:04] lb.utils.events INFO: eta: 3:25:54 iteration: 295999/375342 consumed_samples: 303104000 total_loss: 3.134 time: 0.3442 s/iter data_time: 0.2194 s/iter total_throughput: 2975.21 samples/s lr: 1.15e-04 [09/23 07:52:39] lb.utils.events INFO: eta: 3:25:02 iteration: 296099/375342 consumed_samples: 303206400 total_loss: 3.127 time: 0.3442 s/iter data_time: 0.2168 s/iter total_throughput: 2975.19 samples/s lr: 1.15e-04 [09/23 07:53:14] lb.utils.events INFO: eta: 3:25:12 iteration: 296199/375342 consumed_samples: 303308800 total_loss: 3.154 time: 0.3442 s/iter data_time: 0.2146 s/iter total_throughput: 2975.18 samples/s lr: 1.15e-04 [09/23 07:53:49] lb.utils.events INFO: eta: 3:25:15 iteration: 296299/375342 consumed_samples: 303411200 total_loss: 3.138 time: 0.3442 s/iter data_time: 0.2164 s/iter total_throughput: 2975.16 samples/s lr: 1.14e-04 [09/23 07:54:24] lb.utils.events INFO: eta: 3:24:39 iteration: 296399/375342 consumed_samples: 303513600 total_loss: 3.127 time: 0.3442 s/iter data_time: 0.2242 s/iter total_throughput: 2975.14 samples/s lr: 1.14e-04 [09/23 07:55:00] lb.utils.events INFO: eta: 3:24:02 iteration: 296499/375342 consumed_samples: 303616000 total_loss: 3.127 time: 0.3442 s/iter data_time: 0.2260 s/iter total_throughput: 2975.12 samples/s lr: 1.14e-04 [09/23 07:55:35] lb.utils.events INFO: eta: 3:23:36 iteration: 296599/375342 consumed_samples: 303718400 total_loss: 3.117 time: 0.3442 s/iter data_time: 0.2369 s/iter total_throughput: 2975.09 samples/s lr: 1.14e-04 [09/23 07:56:10] lb.utils.events INFO: eta: 3:23:42 iteration: 296699/375342 consumed_samples: 303820800 total_loss: 3.111 time: 0.3442 s/iter data_time: 0.2158 s/iter total_throughput: 2975.07 samples/s lr: 1.13e-04 [09/23 07:56:45] lb.utils.events INFO: eta: 3:23:26 iteration: 296799/375342 consumed_samples: 303923200 total_loss: 3.119 time: 0.3442 s/iter data_time: 0.2232 s/iter total_throughput: 2975.05 samples/s lr: 1.13e-04 [09/23 07:57:20] lb.utils.events INFO: eta: 3:23:11 iteration: 296899/375342 consumed_samples: 304025600 total_loss: 3.109 time: 0.3442 s/iter data_time: 0.2217 s/iter total_throughput: 2975.04 samples/s lr: 1.13e-04 [09/23 07:57:55] lb.utils.events INFO: eta: 3:22:53 iteration: 296999/375342 consumed_samples: 304128000 total_loss: 3.117 time: 0.3442 s/iter data_time: 0.2252 s/iter total_throughput: 2975.01 samples/s lr: 1.13e-04 [09/23 07:58:30] lb.utils.events INFO: eta: 3:22:35 iteration: 297099/375342 consumed_samples: 304230400 total_loss: 3.144 time: 0.3442 s/iter data_time: 0.2235 s/iter total_throughput: 2975.00 samples/s lr: 1.12e-04 [09/23 07:59:05] lb.utils.events INFO: eta: 3:22:18 iteration: 297199/375342 consumed_samples: 304332800 total_loss: 3.153 time: 0.3442 s/iter data_time: 0.2257 s/iter total_throughput: 2974.99 samples/s lr: 1.12e-04 [09/23 07:59:40] lb.utils.events INFO: eta: 3:21:59 iteration: 297299/375342 consumed_samples: 304435200 total_loss: 3.13 time: 0.3442 s/iter data_time: 0.2198 s/iter total_throughput: 2974.97 samples/s lr: 1.12e-04 [09/23 08:00:16] lb.utils.events INFO: eta: 3:21:40 iteration: 297399/375342 consumed_samples: 304537600 total_loss: 3.123 time: 0.3442 s/iter data_time: 0.2206 s/iter total_throughput: 2974.94 samples/s lr: 1.12e-04 [09/23 08:00:50] lb.utils.events INFO: eta: 3:21:14 iteration: 297499/375342 consumed_samples: 304640000 total_loss: 3.124 time: 0.3442 s/iter data_time: 0.2248 s/iter total_throughput: 2974.93 samples/s lr: 1.11e-04 [09/23 08:01:25] lb.utils.events INFO: eta: 3:21:05 iteration: 297599/375342 consumed_samples: 304742400 total_loss: 3.13 time: 0.3442 s/iter data_time: 0.2201 s/iter total_throughput: 2974.92 samples/s lr: 1.11e-04 [09/23 08:02:01] lb.utils.events INFO: eta: 3:20:41 iteration: 297699/375342 consumed_samples: 304844800 total_loss: 3.113 time: 0.3442 s/iter data_time: 0.2276 s/iter total_throughput: 2974.89 samples/s lr: 1.11e-04 [09/23 08:02:36] lb.utils.events INFO: eta: 3:20:08 iteration: 297799/375342 consumed_samples: 304947200 total_loss: 3.097 time: 0.3442 s/iter data_time: 0.2264 s/iter total_throughput: 2974.87 samples/s lr: 1.11e-04 [09/23 08:03:11] lb.utils.events INFO: eta: 3:19:42 iteration: 297899/375342 consumed_samples: 305049600 total_loss: 3.109 time: 0.3442 s/iter data_time: 0.2262 s/iter total_throughput: 2974.85 samples/s lr: 1.10e-04 [09/23 08:03:46] lb.utils.events INFO: eta: 3:19:45 iteration: 297999/375342 consumed_samples: 305152000 total_loss: 3.114 time: 0.3442 s/iter data_time: 0.2200 s/iter total_throughput: 2974.84 samples/s lr: 1.10e-04 [09/23 08:04:20] lb.utils.events INFO: eta: 3:19:25 iteration: 298099/375342 consumed_samples: 305254400 total_loss: 3.124 time: 0.3442 s/iter data_time: 0.2230 s/iter total_throughput: 2974.83 samples/s lr: 1.10e-04 [09/23 08:04:56] lb.utils.events INFO: eta: 3:18:52 iteration: 298199/375342 consumed_samples: 305356800 total_loss: 3.119 time: 0.3442 s/iter data_time: 0.2271 s/iter total_throughput: 2974.81 samples/s lr: 1.10e-04 [09/23 08:05:30] lb.utils.events INFO: eta: 3:18:37 iteration: 298299/375342 consumed_samples: 305459200 total_loss: 3.107 time: 0.3442 s/iter data_time: 0.2150 s/iter total_throughput: 2974.80 samples/s lr: 1.09e-04 [09/23 08:06:05] lb.utils.events INFO: eta: 3:18:52 iteration: 298399/375342 consumed_samples: 305561600 total_loss: 3.113 time: 0.3442 s/iter data_time: 0.2303 s/iter total_throughput: 2974.78 samples/s lr: 1.09e-04 [09/23 08:06:40] lb.utils.events INFO: eta: 3:18:51 iteration: 298499/375342 consumed_samples: 305664000 total_loss: 3.113 time: 0.3442 s/iter data_time: 0.2218 s/iter total_throughput: 2974.77 samples/s lr: 1.09e-04 [09/23 08:07:15] lb.utils.events INFO: eta: 3:18:34 iteration: 298599/375342 consumed_samples: 305766400 total_loss: 3.122 time: 0.3442 s/iter data_time: 0.2170 s/iter total_throughput: 2974.76 samples/s lr: 1.09e-04 [09/23 08:07:50] lb.utils.events INFO: eta: 3:18:36 iteration: 298699/375342 consumed_samples: 305868800 total_loss: 3.137 time: 0.3442 s/iter data_time: 0.2355 s/iter total_throughput: 2974.74 samples/s lr: 1.08e-04 [09/23 08:08:25] lb.utils.events INFO: eta: 3:18:41 iteration: 298799/375342 consumed_samples: 305971200 total_loss: 3.132 time: 0.3442 s/iter data_time: 0.2340 s/iter total_throughput: 2974.73 samples/s lr: 1.08e-04 [09/23 08:09:00] lb.utils.events INFO: eta: 3:18:50 iteration: 298899/375342 consumed_samples: 306073600 total_loss: 3.104 time: 0.3442 s/iter data_time: 0.2215 s/iter total_throughput: 2974.72 samples/s lr: 1.08e-04 [09/23 08:09:35] lb.utils.events INFO: eta: 3:18:40 iteration: 298999/375342 consumed_samples: 306176000 total_loss: 3.104 time: 0.3442 s/iter data_time: 0.2218 s/iter total_throughput: 2974.71 samples/s lr: 1.08e-04 [09/23 08:10:09] lb.utils.events INFO: eta: 3:18:06 iteration: 299099/375342 consumed_samples: 306278400 total_loss: 3.121 time: 0.3442 s/iter data_time: 0.2217 s/iter total_throughput: 2974.71 samples/s lr: 1.07e-04 [09/23 08:10:44] lb.utils.events INFO: eta: 3:17:48 iteration: 299199/375342 consumed_samples: 306380800 total_loss: 3.123 time: 0.3442 s/iter data_time: 0.2212 s/iter total_throughput: 2974.69 samples/s lr: 1.07e-04 [09/23 08:11:19] lb.utils.events INFO: eta: 3:17:28 iteration: 299299/375342 consumed_samples: 306483200 total_loss: 3.131 time: 0.3442 s/iter data_time: 0.2206 s/iter total_throughput: 2974.67 samples/s lr: 1.07e-04 [09/23 08:11:55] lb.utils.events INFO: eta: 3:16:59 iteration: 299399/375342 consumed_samples: 306585600 total_loss: 3.119 time: 0.3442 s/iter data_time: 0.2252 s/iter total_throughput: 2974.64 samples/s lr: 1.07e-04 [09/23 08:12:30] lb.utils.events INFO: eta: 3:16:44 iteration: 299499/375342 consumed_samples: 306688000 total_loss: 3.123 time: 0.3442 s/iter data_time: 0.2255 s/iter total_throughput: 2974.62 samples/s lr: 1.06e-04 [09/23 08:13:05] lb.utils.events INFO: eta: 3:16:08 iteration: 299599/375342 consumed_samples: 306790400 total_loss: 3.135 time: 0.3442 s/iter data_time: 0.2368 s/iter total_throughput: 2974.60 samples/s lr: 1.06e-04 [09/23 08:13:40] lb.utils.events INFO: eta: 3:15:23 iteration: 299699/375342 consumed_samples: 306892800 total_loss: 3.122 time: 0.3443 s/iter data_time: 0.2250 s/iter total_throughput: 2974.58 samples/s lr: 1.06e-04 [09/23 08:14:15] lb.utils.events INFO: eta: 3:14:45 iteration: 299799/375342 consumed_samples: 306995200 total_loss: 3.127 time: 0.3443 s/iter data_time: 0.2136 s/iter total_throughput: 2974.56 samples/s lr: 1.06e-04 [09/23 08:14:50] lb.utils.events INFO: eta: 3:14:14 iteration: 299899/375342 consumed_samples: 307097600 total_loss: 3.11 time: 0.3443 s/iter data_time: 0.2205 s/iter total_throughput: 2974.55 samples/s lr: 1.05e-04 [09/23 08:15:25] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0299999 [09/23 08:15:26] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/23 08:15:26] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/23 08:15:30] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0996 s/iter. Inference: 0.1665 s/iter. Eval: 0.0020 s/iter. Total: 0.2681 s/iter. ETA=0:00:09 [09/23 08:15:35] lb.evaluation.evaluator INFO: Inference done 28672/50000. Dataloading: 0.1109 s/iter. Inference: 0.1751 s/iter. Eval: 0.0020 s/iter. Total: 0.2881 s/iter. ETA=0:00:05 [09/23 08:15:40] lb.evaluation.evaluator INFO: Inference done 47104/50000. Dataloading: 0.1206 s/iter. Inference: 0.1685 s/iter. Eval: 0.0020 s/iter. Total: 0.2912 s/iter. ETA=0:00:00 [09/23 08:15:41] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/23 08:15:41] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.553595 (0.000251 s / iter per device, on 8 devices) [09/23 08:15:41] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000149 s / iter per device, on 8 devices) [09/23 08:15:41] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/23 08:15:41] lb.evaluation.utils INFO: copypaste: Acc@1=79.158 [09/23 08:15:41] lb.evaluation.utils INFO: copypaste: Acc@5=94.326 [09/23 08:15:41] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 79.15800, better than last best score 78.92200 @ iteration 289999. [09/23 08:15:41] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/23 08:15:42] lb.utils.events INFO: eta: 3:13:42 iteration: 299999/375342 consumed_samples: 307200000 total_loss: 3.104 time: 0.3443 s/iter data_time: 0.2373 s/iter total_throughput: 2974.54 samples/s lr: 1.05e-04 [09/23 08:16:15] lb.utils.events INFO: eta: 3:14:04 iteration: 300099/375342 consumed_samples: 307302400 total_loss: 3.106 time: 0.3443 s/iter data_time: 0.2206 s/iter total_throughput: 2974.57 samples/s lr: 1.05e-04 [09/23 08:16:50] lb.utils.events INFO: eta: 3:13:58 iteration: 300199/375342 consumed_samples: 307404800 total_loss: 3.116 time: 0.3443 s/iter data_time: 0.2239 s/iter total_throughput: 2974.55 samples/s lr: 1.05e-04 [09/23 08:17:25] lb.utils.events INFO: eta: 3:13:43 iteration: 300299/375342 consumed_samples: 307507200 total_loss: 3.125 time: 0.3443 s/iter data_time: 0.2246 s/iter total_throughput: 2974.52 samples/s lr: 1.04e-04 [09/23 08:18:00] lb.utils.events INFO: eta: 3:13:29 iteration: 300399/375342 consumed_samples: 307609600 total_loss: 3.104 time: 0.3443 s/iter data_time: 0.2155 s/iter total_throughput: 2974.51 samples/s lr: 1.04e-04 [09/23 08:18:35] lb.utils.events INFO: eta: 3:13:41 iteration: 300499/375342 consumed_samples: 307712000 total_loss: 3.1 time: 0.3443 s/iter data_time: 0.2228 s/iter total_throughput: 2974.50 samples/s lr: 1.04e-04 [09/23 08:19:10] lb.utils.events INFO: eta: 3:13:22 iteration: 300599/375342 consumed_samples: 307814400 total_loss: 3.119 time: 0.3443 s/iter data_time: 0.2241 s/iter total_throughput: 2974.48 samples/s lr: 1.04e-04 [09/23 08:19:45] lb.utils.events INFO: eta: 3:13:25 iteration: 300699/375342 consumed_samples: 307916800 total_loss: 3.106 time: 0.3443 s/iter data_time: 0.2360 s/iter total_throughput: 2974.47 samples/s lr: 1.04e-04 [09/23 08:20:20] lb.utils.events INFO: eta: 3:12:54 iteration: 300799/375342 consumed_samples: 308019200 total_loss: 3.107 time: 0.3443 s/iter data_time: 0.2192 s/iter total_throughput: 2974.46 samples/s lr: 1.03e-04 [09/23 08:20:55] lb.utils.events INFO: eta: 3:13:02 iteration: 300899/375342 consumed_samples: 308121600 total_loss: 3.12 time: 0.3443 s/iter data_time: 0.2283 s/iter total_throughput: 2974.44 samples/s lr: 1.03e-04 [09/23 08:21:30] lb.utils.events INFO: eta: 3:12:31 iteration: 300999/375342 consumed_samples: 308224000 total_loss: 3.115 time: 0.3443 s/iter data_time: 0.2285 s/iter total_throughput: 2974.42 samples/s lr: 1.03e-04 [09/23 08:22:05] lb.utils.events INFO: eta: 3:11:39 iteration: 301099/375342 consumed_samples: 308326400 total_loss: 3.098 time: 0.3443 s/iter data_time: 0.2255 s/iter total_throughput: 2974.41 samples/s lr: 1.03e-04 [09/23 08:22:40] lb.utils.events INFO: eta: 3:11:13 iteration: 301199/375342 consumed_samples: 308428800 total_loss: 3.104 time: 0.3443 s/iter data_time: 0.2196 s/iter total_throughput: 2974.40 samples/s lr: 1.02e-04 [09/23 08:23:14] lb.utils.events INFO: eta: 3:10:58 iteration: 301299/375342 consumed_samples: 308531200 total_loss: 3.121 time: 0.3443 s/iter data_time: 0.2260 s/iter total_throughput: 2974.39 samples/s lr: 1.02e-04 [09/23 08:23:49] lb.utils.events INFO: eta: 3:11:18 iteration: 301399/375342 consumed_samples: 308633600 total_loss: 3.111 time: 0.3443 s/iter data_time: 0.2189 s/iter total_throughput: 2974.38 samples/s lr: 1.02e-04 [09/23 08:24:24] lb.utils.events INFO: eta: 3:10:36 iteration: 301499/375342 consumed_samples: 308736000 total_loss: 3.105 time: 0.3443 s/iter data_time: 0.2237 s/iter total_throughput: 2974.37 samples/s lr: 1.02e-04 [09/23 08:24:59] lb.utils.events INFO: eta: 3:10:11 iteration: 301599/375342 consumed_samples: 308838400 total_loss: 3.097 time: 0.3443 s/iter data_time: 0.2510 s/iter total_throughput: 2974.34 samples/s lr: 1.01e-04 [09/23 08:25:35] lb.utils.events INFO: eta: 3:09:53 iteration: 301699/375342 consumed_samples: 308940800 total_loss: 3.097 time: 0.3443 s/iter data_time: 0.2178 s/iter total_throughput: 2974.32 samples/s lr: 1.01e-04 [09/23 08:26:10] lb.utils.events INFO: eta: 3:09:46 iteration: 301799/375342 consumed_samples: 309043200 total_loss: 3.107 time: 0.3443 s/iter data_time: 0.2240 s/iter total_throughput: 2974.29 samples/s lr: 1.01e-04 [09/23 08:26:45] lb.utils.events INFO: eta: 3:09:45 iteration: 301899/375342 consumed_samples: 309145600 total_loss: 3.109 time: 0.3443 s/iter data_time: 0.2128 s/iter total_throughput: 2974.29 samples/s lr: 1.01e-04 [09/23 08:27:19] lb.utils.events INFO: eta: 3:10:05 iteration: 301999/375342 consumed_samples: 309248000 total_loss: 3.101 time: 0.3443 s/iter data_time: 0.2294 s/iter total_throughput: 2974.27 samples/s lr: 1.00e-04 [09/23 08:27:54] lb.utils.events INFO: eta: 3:09:58 iteration: 302099/375342 consumed_samples: 309350400 total_loss: 3.097 time: 0.3443 s/iter data_time: 0.2166 s/iter total_throughput: 2974.26 samples/s lr: 1.00e-04 [09/23 08:28:29] lb.utils.events INFO: eta: 3:09:44 iteration: 302199/375342 consumed_samples: 309452800 total_loss: 3.108 time: 0.3443 s/iter data_time: 0.2215 s/iter total_throughput: 2974.25 samples/s lr: 9.99e-05 [09/23 08:29:04] lb.utils.events INFO: eta: 3:09:33 iteration: 302299/375342 consumed_samples: 309555200 total_loss: 3.112 time: 0.3443 s/iter data_time: 0.2353 s/iter total_throughput: 2974.23 samples/s lr: 9.97e-05 [09/23 08:29:39] lb.utils.events INFO: eta: 3:09:03 iteration: 302399/375342 consumed_samples: 309657600 total_loss: 3.1 time: 0.3443 s/iter data_time: 0.2188 s/iter total_throughput: 2974.23 samples/s lr: 9.94e-05 [09/23 08:30:14] lb.utils.events INFO: eta: 3:08:47 iteration: 302499/375342 consumed_samples: 309760000 total_loss: 3.094 time: 0.3443 s/iter data_time: 0.2312 s/iter total_throughput: 2974.22 samples/s lr: 9.92e-05 [09/23 08:30:49] lb.utils.events INFO: eta: 3:08:37 iteration: 302599/375342 consumed_samples: 309862400 total_loss: 3.082 time: 0.3443 s/iter data_time: 0.2159 s/iter total_throughput: 2974.20 samples/s lr: 9.89e-05 [09/23 08:31:24] lb.utils.events INFO: eta: 3:08:36 iteration: 302699/375342 consumed_samples: 309964800 total_loss: 3.095 time: 0.3443 s/iter data_time: 0.2163 s/iter total_throughput: 2974.19 samples/s lr: 9.87e-05 [09/23 08:31:58] lb.utils.events INFO: eta: 3:08:38 iteration: 302799/375342 consumed_samples: 310067200 total_loss: 3.102 time: 0.3443 s/iter data_time: 0.2227 s/iter total_throughput: 2974.18 samples/s lr: 9.85e-05 [09/23 08:32:33] lb.utils.events INFO: eta: 3:07:52 iteration: 302899/375342 consumed_samples: 310169600 total_loss: 3.109 time: 0.3443 s/iter data_time: 0.2229 s/iter total_throughput: 2974.17 samples/s lr: 9.82e-05 [09/23 08:33:08] lb.utils.events INFO: eta: 3:07:31 iteration: 302999/375342 consumed_samples: 310272000 total_loss: 3.118 time: 0.3443 s/iter data_time: 0.2230 s/iter total_throughput: 2974.15 samples/s lr: 9.80e-05 [09/23 08:33:43] lb.utils.events INFO: eta: 3:07:11 iteration: 303099/375342 consumed_samples: 310374400 total_loss: 3.112 time: 0.3443 s/iter data_time: 0.2247 s/iter total_throughput: 2974.14 samples/s lr: 9.78e-05 [09/23 08:34:18] lb.utils.events INFO: eta: 3:06:56 iteration: 303199/375342 consumed_samples: 310476800 total_loss: 3.114 time: 0.3443 s/iter data_time: 0.2250 s/iter total_throughput: 2974.13 samples/s lr: 9.75e-05 [09/23 08:34:52] lb.utils.events INFO: eta: 3:06:40 iteration: 303299/375342 consumed_samples: 310579200 total_loss: 3.109 time: 0.3443 s/iter data_time: 0.2204 s/iter total_throughput: 2974.13 samples/s lr: 9.73e-05 [09/23 08:35:27] lb.utils.events INFO: eta: 3:06:09 iteration: 303399/375342 consumed_samples: 310681600 total_loss: 3.104 time: 0.3443 s/iter data_time: 0.2134 s/iter total_throughput: 2974.13 samples/s lr: 9.71e-05 [09/23 08:36:01] lb.utils.events INFO: eta: 3:06:22 iteration: 303499/375342 consumed_samples: 310784000 total_loss: 3.095 time: 0.3443 s/iter data_time: 0.2113 s/iter total_throughput: 2974.12 samples/s lr: 9.68e-05 [09/23 08:36:36] lb.utils.events INFO: eta: 3:06:13 iteration: 303599/375342 consumed_samples: 310886400 total_loss: 3.09 time: 0.3443 s/iter data_time: 0.2187 s/iter total_throughput: 2974.12 samples/s lr: 9.66e-05 [09/23 08:37:10] lb.utils.events INFO: eta: 3:05:54 iteration: 303699/375342 consumed_samples: 310988800 total_loss: 3.098 time: 0.3443 s/iter data_time: 0.2105 s/iter total_throughput: 2974.12 samples/s lr: 9.64e-05 [09/23 08:37:45] lb.utils.events INFO: eta: 3:05:28 iteration: 303799/375342 consumed_samples: 311091200 total_loss: 3.086 time: 0.3443 s/iter data_time: 0.2287 s/iter total_throughput: 2974.11 samples/s lr: 9.61e-05 [09/23 08:38:19] lb.utils.events INFO: eta: 3:05:08 iteration: 303899/375342 consumed_samples: 311193600 total_loss: 3.092 time: 0.3443 s/iter data_time: 0.2216 s/iter total_throughput: 2974.11 samples/s lr: 9.59e-05 [09/23 08:38:54] lb.utils.events INFO: eta: 3:04:39 iteration: 303999/375342 consumed_samples: 311296000 total_loss: 3.101 time: 0.3443 s/iter data_time: 0.2147 s/iter total_throughput: 2974.11 samples/s lr: 9.57e-05 [09/23 08:39:28] lb.utils.events INFO: eta: 3:04:27 iteration: 304099/375342 consumed_samples: 311398400 total_loss: 3.09 time: 0.3443 s/iter data_time: 0.2093 s/iter total_throughput: 2974.11 samples/s lr: 9.54e-05 [09/23 08:40:03] lb.utils.events INFO: eta: 3:04:27 iteration: 304199/375342 consumed_samples: 311500800 total_loss: 3.102 time: 0.3443 s/iter data_time: 0.2203 s/iter total_throughput: 2974.10 samples/s lr: 9.52e-05 [09/23 08:40:38] lb.utils.events INFO: eta: 3:04:17 iteration: 304299/375342 consumed_samples: 311603200 total_loss: 3.094 time: 0.3443 s/iter data_time: 0.2209 s/iter total_throughput: 2974.10 samples/s lr: 9.50e-05 [09/23 08:41:12] lb.utils.events INFO: eta: 3:04:21 iteration: 304399/375342 consumed_samples: 311705600 total_loss: 3.076 time: 0.3443 s/iter data_time: 0.2264 s/iter total_throughput: 2974.10 samples/s lr: 9.47e-05 [09/23 08:41:47] lb.utils.events INFO: eta: 3:03:30 iteration: 304499/375342 consumed_samples: 311808000 total_loss: 3.088 time: 0.3443 s/iter data_time: 0.2203 s/iter total_throughput: 2974.10 samples/s lr: 9.45e-05 [09/23 08:42:21] lb.utils.events INFO: eta: 3:03:02 iteration: 304599/375342 consumed_samples: 311910400 total_loss: 3.096 time: 0.3443 s/iter data_time: 0.2183 s/iter total_throughput: 2974.08 samples/s lr: 9.43e-05 [09/23 08:42:56] lb.utils.events INFO: eta: 3:02:43 iteration: 304699/375342 consumed_samples: 312012800 total_loss: 3.085 time: 0.3443 s/iter data_time: 0.2171 s/iter total_throughput: 2974.08 samples/s lr: 9.40e-05 [09/23 08:43:30] lb.utils.events INFO: eta: 3:02:25 iteration: 304799/375342 consumed_samples: 312115200 total_loss: 3.098 time: 0.3443 s/iter data_time: 0.2246 s/iter total_throughput: 2974.08 samples/s lr: 9.38e-05 [09/23 08:44:05] lb.utils.events INFO: eta: 3:02:24 iteration: 304899/375342 consumed_samples: 312217600 total_loss: 3.112 time: 0.3443 s/iter data_time: 0.2161 s/iter total_throughput: 2974.08 samples/s lr: 9.36e-05 [09/23 08:44:39] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0304999 [09/23 08:44:40] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/23 08:44:40] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/23 08:44:44] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0853 s/iter. Inference: 0.1618 s/iter. Eval: 0.0024 s/iter. Total: 0.2495 s/iter. ETA=0:00:09 [09/23 08:44:50] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1360 s/iter. Inference: 0.1632 s/iter. Eval: 0.0021 s/iter. Total: 0.3013 s/iter. ETA=0:00:05 [09/23 08:44:55] lb.evaluation.evaluator INFO: Inference done 49152/50000. Dataloading: 0.1197 s/iter. Inference: 0.1626 s/iter. Eval: 0.0021 s/iter. Total: 0.2845 s/iter. ETA=0:00:00 [09/23 08:44:55] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/23 08:44:55] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.608791 (0.000252 s / iter per device, on 8 devices) [09/23 08:44:55] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000144 s / iter per device, on 8 devices) [09/23 08:44:55] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/23 08:44:55] lb.evaluation.utils INFO: copypaste: Acc@1=79.202 [09/23 08:44:55] lb.evaluation.utils INFO: copypaste: Acc@5=94.3 [09/23 08:44:55] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 79.20200, better than last best score 79.15800 @ iteration 299999. [09/23 08:44:55] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/23 08:44:56] lb.utils.events INFO: eta: 3:02:21 iteration: 304999/375342 consumed_samples: 312320000 total_loss: 3.095 time: 0.3443 s/iter data_time: 0.2030 s/iter total_throughput: 2974.09 samples/s lr: 9.33e-05 [09/23 08:45:27] lb.utils.events INFO: eta: 3:03:15 iteration: 305099/375342 consumed_samples: 312422400 total_loss: 3.084 time: 0.3443 s/iter data_time: 0.2155 s/iter total_throughput: 2974.17 samples/s lr: 9.31e-05 [09/23 08:46:02] lb.utils.events INFO: eta: 3:03:11 iteration: 305199/375342 consumed_samples: 312524800 total_loss: 3.083 time: 0.3443 s/iter data_time: 0.2163 s/iter total_throughput: 2974.16 samples/s lr: 9.29e-05 [09/23 08:46:36] lb.utils.events INFO: eta: 3:03:11 iteration: 305299/375342 consumed_samples: 312627200 total_loss: 3.082 time: 0.3443 s/iter data_time: 0.2190 s/iter total_throughput: 2974.17 samples/s lr: 9.27e-05 [09/23 08:47:10] lb.utils.events INFO: eta: 3:03:21 iteration: 305399/375342 consumed_samples: 312729600 total_loss: 3.085 time: 0.3443 s/iter data_time: 0.2088 s/iter total_throughput: 2974.19 samples/s lr: 9.24e-05 [09/23 08:47:44] lb.utils.events INFO: eta: 3:03:15 iteration: 305499/375342 consumed_samples: 312832000 total_loss: 3.089 time: 0.3443 s/iter data_time: 0.2128 s/iter total_throughput: 2974.20 samples/s lr: 9.22e-05 [09/23 08:48:19] lb.utils.events INFO: eta: 3:03:02 iteration: 305599/375342 consumed_samples: 312934400 total_loss: 3.095 time: 0.3443 s/iter data_time: 0.2171 s/iter total_throughput: 2974.19 samples/s lr: 9.20e-05 [09/23 08:48:53] lb.utils.events INFO: eta: 3:02:42 iteration: 305699/375342 consumed_samples: 313036800 total_loss: 3.098 time: 0.3443 s/iter data_time: 0.2222 s/iter total_throughput: 2974.19 samples/s lr: 9.17e-05 [09/23 08:49:28] lb.utils.events INFO: eta: 3:02:37 iteration: 305799/375342 consumed_samples: 313139200 total_loss: 3.081 time: 0.3443 s/iter data_time: 0.2132 s/iter total_throughput: 2974.19 samples/s lr: 9.15e-05 [09/23 08:50:02] lb.utils.events INFO: eta: 3:02:23 iteration: 305899/375342 consumed_samples: 313241600 total_loss: 3.066 time: 0.3443 s/iter data_time: 0.2155 s/iter total_throughput: 2974.21 samples/s lr: 9.13e-05 [09/23 08:50:36] lb.utils.events INFO: eta: 3:02:03 iteration: 305999/375342 consumed_samples: 313344000 total_loss: 3.077 time: 0.3443 s/iter data_time: 0.2193 s/iter total_throughput: 2974.21 samples/s lr: 9.11e-05 [09/23 08:51:10] lb.utils.events INFO: eta: 3:01:02 iteration: 306099/375342 consumed_samples: 313446400 total_loss: 3.117 time: 0.3443 s/iter data_time: 0.2279 s/iter total_throughput: 2974.22 samples/s lr: 9.08e-05 [09/23 08:51:45] lb.utils.events INFO: eta: 3:00:18 iteration: 306199/375342 consumed_samples: 313548800 total_loss: 3.082 time: 0.3443 s/iter data_time: 0.2144 s/iter total_throughput: 2974.21 samples/s lr: 9.06e-05 [09/23 08:52:19] lb.utils.events INFO: eta: 2:59:32 iteration: 306299/375342 consumed_samples: 313651200 total_loss: 3.063 time: 0.3443 s/iter data_time: 0.2063 s/iter total_throughput: 2974.23 samples/s lr: 9.04e-05 [09/23 08:52:53] lb.utils.events INFO: eta: 2:58:58 iteration: 306399/375342 consumed_samples: 313753600 total_loss: 3.066 time: 0.3443 s/iter data_time: 0.2186 s/iter total_throughput: 2974.23 samples/s lr: 9.02e-05 [09/23 08:53:27] lb.utils.events INFO: eta: 2:59:00 iteration: 306499/375342 consumed_samples: 313856000 total_loss: 3.079 time: 0.3443 s/iter data_time: 0.2202 s/iter total_throughput: 2974.24 samples/s lr: 8.99e-05 [09/23 08:54:01] lb.utils.events INFO: eta: 2:58:54 iteration: 306599/375342 consumed_samples: 313958400 total_loss: 3.082 time: 0.3443 s/iter data_time: 0.2039 s/iter total_throughput: 2974.26 samples/s lr: 8.97e-05 [09/23 08:54:34] lb.utils.events INFO: eta: 2:59:15 iteration: 306699/375342 consumed_samples: 314060800 total_loss: 3.062 time: 0.3443 s/iter data_time: 0.2015 s/iter total_throughput: 2974.28 samples/s lr: 8.95e-05 [09/23 08:55:09] lb.utils.events INFO: eta: 2:58:46 iteration: 306799/375342 consumed_samples: 314163200 total_loss: 3.071 time: 0.3443 s/iter data_time: 0.2141 s/iter total_throughput: 2974.29 samples/s lr: 8.93e-05 [09/23 08:55:43] lb.utils.events INFO: eta: 2:58:27 iteration: 306899/375342 consumed_samples: 314265600 total_loss: 3.096 time: 0.3443 s/iter data_time: 0.2158 s/iter total_throughput: 2974.30 samples/s lr: 8.90e-05 [09/23 08:56:16] lb.utils.events INFO: eta: 2:58:38 iteration: 306999/375342 consumed_samples: 314368000 total_loss: 3.085 time: 0.3443 s/iter data_time: 0.2150 s/iter total_throughput: 2974.32 samples/s lr: 8.88e-05 [09/23 08:56:50] lb.utils.events INFO: eta: 2:58:31 iteration: 307099/375342 consumed_samples: 314470400 total_loss: 3.086 time: 0.3443 s/iter data_time: 0.2072 s/iter total_throughput: 2974.34 samples/s lr: 8.86e-05 [09/23 08:57:24] lb.utils.events INFO: eta: 2:58:44 iteration: 307199/375342 consumed_samples: 314572800 total_loss: 3.074 time: 0.3443 s/iter data_time: 0.1995 s/iter total_throughput: 2974.35 samples/s lr: 8.84e-05 [09/23 08:57:58] lb.utils.events INFO: eta: 2:58:40 iteration: 307299/375342 consumed_samples: 314675200 total_loss: 3.074 time: 0.3443 s/iter data_time: 0.2040 s/iter total_throughput: 2974.37 samples/s lr: 8.81e-05 [09/23 08:58:32] lb.utils.events INFO: eta: 2:58:09 iteration: 307399/375342 consumed_samples: 314777600 total_loss: 3.093 time: 0.3443 s/iter data_time: 0.2040 s/iter total_throughput: 2974.39 samples/s lr: 8.79e-05 [09/23 08:59:06] lb.utils.events INFO: eta: 2:57:35 iteration: 307499/375342 consumed_samples: 314880000 total_loss: 3.086 time: 0.3443 s/iter data_time: 0.2202 s/iter total_throughput: 2974.40 samples/s lr: 8.77e-05 [09/23 08:59:39] lb.utils.events INFO: eta: 2:57:37 iteration: 307599/375342 consumed_samples: 314982400 total_loss: 3.098 time: 0.3443 s/iter data_time: 0.2082 s/iter total_throughput: 2974.42 samples/s lr: 8.75e-05 [09/23 09:00:13] lb.utils.events INFO: eta: 2:56:57 iteration: 307699/375342 consumed_samples: 315084800 total_loss: 3.104 time: 0.3443 s/iter data_time: 0.2138 s/iter total_throughput: 2974.44 samples/s lr: 8.72e-05 [09/23 09:00:47] lb.utils.events INFO: eta: 2:56:35 iteration: 307799/375342 consumed_samples: 315187200 total_loss: 3.077 time: 0.3443 s/iter data_time: 0.2110 s/iter total_throughput: 2974.45 samples/s lr: 8.70e-05 [09/23 09:01:21] lb.utils.events INFO: eta: 2:56:20 iteration: 307899/375342 consumed_samples: 315289600 total_loss: 3.055 time: 0.3443 s/iter data_time: 0.2085 s/iter total_throughput: 2974.47 samples/s lr: 8.68e-05 [09/23 09:01:55] lb.utils.events INFO: eta: 2:55:32 iteration: 307999/375342 consumed_samples: 315392000 total_loss: 3.061 time: 0.3443 s/iter data_time: 0.2101 s/iter total_throughput: 2974.48 samples/s lr: 8.66e-05 [09/23 09:02:29] lb.utils.events INFO: eta: 2:54:55 iteration: 308099/375342 consumed_samples: 315494400 total_loss: 3.086 time: 0.3443 s/iter data_time: 0.2218 s/iter total_throughput: 2974.48 samples/s lr: 8.64e-05 [09/23 09:03:04] lb.utils.events INFO: eta: 2:54:09 iteration: 308199/375342 consumed_samples: 315596800 total_loss: 3.086 time: 0.3443 s/iter data_time: 0.2067 s/iter total_throughput: 2974.49 samples/s lr: 8.61e-05 [09/23 09:03:38] lb.utils.events INFO: eta: 2:53:47 iteration: 308299/375342 consumed_samples: 315699200 total_loss: 3.082 time: 0.3443 s/iter data_time: 0.2143 s/iter total_throughput: 2974.48 samples/s lr: 8.59e-05 [09/23 09:04:12] lb.utils.events INFO: eta: 2:53:26 iteration: 308399/375342 consumed_samples: 315801600 total_loss: 3.097 time: 0.3443 s/iter data_time: 0.2353 s/iter total_throughput: 2974.49 samples/s lr: 8.57e-05 [09/23 09:04:47] lb.utils.events INFO: eta: 2:52:54 iteration: 308499/375342 consumed_samples: 315904000 total_loss: 3.1 time: 0.3443 s/iter data_time: 0.2239 s/iter total_throughput: 2974.48 samples/s lr: 8.55e-05 [09/23 09:05:22] lb.utils.events INFO: eta: 2:52:16 iteration: 308599/375342 consumed_samples: 316006400 total_loss: 3.093 time: 0.3443 s/iter data_time: 0.2253 s/iter total_throughput: 2974.47 samples/s lr: 8.53e-05 [09/23 09:05:57] lb.utils.events INFO: eta: 2:51:56 iteration: 308699/375342 consumed_samples: 316108800 total_loss: 3.092 time: 0.3443 s/iter data_time: 0.2170 s/iter total_throughput: 2974.46 samples/s lr: 8.50e-05 [09/23 09:06:32] lb.utils.events INFO: eta: 2:51:47 iteration: 308799/375342 consumed_samples: 316211200 total_loss: 3.07 time: 0.3443 s/iter data_time: 0.2177 s/iter total_throughput: 2974.44 samples/s lr: 8.48e-05 [09/23 09:07:07] lb.utils.events INFO: eta: 2:51:35 iteration: 308899/375342 consumed_samples: 316313600 total_loss: 3.07 time: 0.3443 s/iter data_time: 0.2235 s/iter total_throughput: 2974.43 samples/s lr: 8.46e-05 [09/23 09:07:42] lb.utils.events INFO: eta: 2:51:37 iteration: 308999/375342 consumed_samples: 316416000 total_loss: 3.09 time: 0.3443 s/iter data_time: 0.2254 s/iter total_throughput: 2974.41 samples/s lr: 8.44e-05 [09/23 09:08:16] lb.utils.events INFO: eta: 2:52:14 iteration: 309099/375342 consumed_samples: 316518400 total_loss: 3.088 time: 0.3443 s/iter data_time: 0.2079 s/iter total_throughput: 2974.43 samples/s lr: 8.42e-05 [09/23 09:08:50] lb.utils.events INFO: eta: 2:52:19 iteration: 309199/375342 consumed_samples: 316620800 total_loss: 3.086 time: 0.3443 s/iter data_time: 0.2124 s/iter total_throughput: 2974.43 samples/s lr: 8.39e-05 [09/23 09:09:25] lb.utils.events INFO: eta: 2:52:01 iteration: 309299/375342 consumed_samples: 316723200 total_loss: 3.065 time: 0.3443 s/iter data_time: 0.2310 s/iter total_throughput: 2974.42 samples/s lr: 8.37e-05 [09/23 09:10:00] lb.utils.events INFO: eta: 2:51:48 iteration: 309399/375342 consumed_samples: 316825600 total_loss: 3.062 time: 0.3443 s/iter data_time: 0.2339 s/iter total_throughput: 2974.40 samples/s lr: 8.35e-05 [09/23 09:10:35] lb.utils.events INFO: eta: 2:51:36 iteration: 309499/375342 consumed_samples: 316928000 total_loss: 3.082 time: 0.3443 s/iter data_time: 0.2252 s/iter total_throughput: 2974.39 samples/s lr: 8.33e-05 [09/23 09:11:09] lb.utils.events INFO: eta: 2:51:46 iteration: 309599/375342 consumed_samples: 317030400 total_loss: 3.093 time: 0.3443 s/iter data_time: 0.2090 s/iter total_throughput: 2974.39 samples/s lr: 8.31e-05 [09/23 09:11:44] lb.utils.events INFO: eta: 2:51:31 iteration: 309699/375342 consumed_samples: 317132800 total_loss: 3.07 time: 0.3443 s/iter data_time: 0.2242 s/iter total_throughput: 2974.38 samples/s lr: 8.29e-05 [09/23 09:12:19] lb.utils.events INFO: eta: 2:51:12 iteration: 309799/375342 consumed_samples: 317235200 total_loss: 3.071 time: 0.3443 s/iter data_time: 0.2129 s/iter total_throughput: 2974.37 samples/s lr: 8.26e-05 [09/23 09:12:53] lb.utils.events INFO: eta: 2:50:59 iteration: 309899/375342 consumed_samples: 317337600 total_loss: 3.084 time: 0.3443 s/iter data_time: 0.2172 s/iter total_throughput: 2974.37 samples/s lr: 8.24e-05 [09/23 09:13:28] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0309999 [09/23 09:13:29] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/23 09:13:29] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/23 09:13:33] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0975 s/iter. Inference: 0.1603 s/iter. Eval: 0.0021 s/iter. Total: 0.2599 s/iter. ETA=0:00:09 [09/23 09:13:38] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1214 s/iter. Inference: 0.1724 s/iter. Eval: 0.0020 s/iter. Total: 0.2959 s/iter. ETA=0:00:05 [09/23 09:13:43] lb.evaluation.evaluator INFO: Inference done 49152/50000. Dataloading: 0.1145 s/iter. Inference: 0.1666 s/iter. Eval: 0.0020 s/iter. Total: 0.2832 s/iter. ETA=0:00:00 [09/23 09:13:44] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/23 09:13:44] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.487346 (0.000250 s / iter per device, on 8 devices) [09/23 09:13:44] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000147 s / iter per device, on 8 devices) [09/23 09:13:44] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/23 09:13:44] lb.evaluation.utils INFO: copypaste: Acc@1=79.308 [09/23 09:13:44] lb.evaluation.utils INFO: copypaste: Acc@5=94.274 [09/23 09:13:44] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 79.30800, better than last best score 79.20200 @ iteration 304999. [09/23 09:13:44] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/23 09:13:44] lb.utils.events INFO: eta: 2:50:22 iteration: 309999/375342 consumed_samples: 317440000 total_loss: 3.087 time: 0.3443 s/iter data_time: 0.2160 s/iter total_throughput: 2974.36 samples/s lr: 8.22e-05 [09/23 09:14:17] lb.utils.events INFO: eta: 2:50:52 iteration: 310099/375342 consumed_samples: 317542400 total_loss: 3.076 time: 0.3443 s/iter data_time: 0.2200 s/iter total_throughput: 2974.41 samples/s lr: 8.20e-05 [09/23 09:14:52] lb.utils.events INFO: eta: 2:50:27 iteration: 310199/375342 consumed_samples: 317644800 total_loss: 3.086 time: 0.3443 s/iter data_time: 0.2229 s/iter total_throughput: 2974.40 samples/s lr: 8.18e-05 [09/23 09:15:27] lb.utils.events INFO: eta: 2:50:33 iteration: 310299/375342 consumed_samples: 317747200 total_loss: 3.077 time: 0.3443 s/iter data_time: 0.2224 s/iter total_throughput: 2974.39 samples/s lr: 8.16e-05 [09/23 09:16:01] lb.utils.events INFO: eta: 2:50:26 iteration: 310399/375342 consumed_samples: 317849600 total_loss: 3.069 time: 0.3443 s/iter data_time: 0.2173 s/iter total_throughput: 2974.38 samples/s lr: 8.13e-05 [09/23 09:16:36] lb.utils.events INFO: eta: 2:49:48 iteration: 310499/375342 consumed_samples: 317952000 total_loss: 3.079 time: 0.3443 s/iter data_time: 0.2118 s/iter total_throughput: 2974.38 samples/s lr: 8.11e-05 [09/23 09:17:11] lb.utils.events INFO: eta: 2:49:22 iteration: 310599/375342 consumed_samples: 318054400 total_loss: 3.071 time: 0.3443 s/iter data_time: 0.2188 s/iter total_throughput: 2974.37 samples/s lr: 8.09e-05 [09/23 09:17:46] lb.utils.events INFO: eta: 2:48:39 iteration: 310699/375342 consumed_samples: 318156800 total_loss: 3.061 time: 0.3443 s/iter data_time: 0.2239 s/iter total_throughput: 2974.36 samples/s lr: 8.07e-05 [09/23 09:18:20] lb.utils.events INFO: eta: 2:48:35 iteration: 310799/375342 consumed_samples: 318259200 total_loss: 3.065 time: 0.3443 s/iter data_time: 0.2214 s/iter total_throughput: 2974.35 samples/s lr: 8.05e-05 [09/23 09:18:55] lb.utils.events INFO: eta: 2:48:26 iteration: 310899/375342 consumed_samples: 318361600 total_loss: 3.074 time: 0.3443 s/iter data_time: 0.2211 s/iter total_throughput: 2974.34 samples/s lr: 8.03e-05 [09/23 09:19:30] lb.utils.events INFO: eta: 2:48:17 iteration: 310999/375342 consumed_samples: 318464000 total_loss: 3.068 time: 0.3443 s/iter data_time: 0.2269 s/iter total_throughput: 2974.34 samples/s lr: 8.01e-05 [09/23 09:20:04] lb.utils.events INFO: eta: 2:47:23 iteration: 311099/375342 consumed_samples: 318566400 total_loss: 3.064 time: 0.3443 s/iter data_time: 0.2177 s/iter total_throughput: 2974.35 samples/s lr: 7.99e-05 [09/23 09:20:38] lb.utils.events INFO: eta: 2:47:09 iteration: 311199/375342 consumed_samples: 318668800 total_loss: 3.063 time: 0.3443 s/iter data_time: 0.2115 s/iter total_throughput: 2974.35 samples/s lr: 7.96e-05 [09/23 09:21:13] lb.utils.events INFO: eta: 2:46:55 iteration: 311299/375342 consumed_samples: 318771200 total_loss: 3.071 time: 0.3443 s/iter data_time: 0.2115 s/iter total_throughput: 2974.34 samples/s lr: 7.94e-05 [09/23 09:21:48] lb.utils.events INFO: eta: 2:46:38 iteration: 311399/375342 consumed_samples: 318873600 total_loss: 3.079 time: 0.3443 s/iter data_time: 0.2254 s/iter total_throughput: 2974.32 samples/s lr: 7.92e-05 [09/23 09:22:22] lb.utils.events INFO: eta: 2:46:29 iteration: 311499/375342 consumed_samples: 318976000 total_loss: 3.075 time: 0.3443 s/iter data_time: 0.2169 s/iter total_throughput: 2974.32 samples/s lr: 7.90e-05 [09/23 09:22:57] lb.utils.events INFO: eta: 2:46:27 iteration: 311599/375342 consumed_samples: 319078400 total_loss: 3.068 time: 0.3443 s/iter data_time: 0.2318 s/iter total_throughput: 2974.32 samples/s lr: 7.88e-05 [09/23 09:23:32] lb.utils.events INFO: eta: 2:46:24 iteration: 311699/375342 consumed_samples: 319180800 total_loss: 3.058 time: 0.3443 s/iter data_time: 0.2173 s/iter total_throughput: 2974.30 samples/s lr: 7.86e-05 [09/23 09:24:06] lb.utils.events INFO: eta: 2:46:20 iteration: 311799/375342 consumed_samples: 319283200 total_loss: 3.06 time: 0.3443 s/iter data_time: 0.2118 s/iter total_throughput: 2974.31 samples/s lr: 7.84e-05 [09/23 09:24:41] lb.utils.events INFO: eta: 2:46:08 iteration: 311899/375342 consumed_samples: 319385600 total_loss: 3.059 time: 0.3443 s/iter data_time: 0.2212 s/iter total_throughput: 2974.30 samples/s lr: 7.82e-05 [09/23 09:25:15] lb.utils.events INFO: eta: 2:45:46 iteration: 311999/375342 consumed_samples: 319488000 total_loss: 3.064 time: 0.3443 s/iter data_time: 0.2258 s/iter total_throughput: 2974.30 samples/s lr: 7.80e-05 [09/23 09:25:49] lb.utils.events INFO: eta: 2:45:17 iteration: 312099/375342 consumed_samples: 319590400 total_loss: 3.07 time: 0.3443 s/iter data_time: 0.2131 s/iter total_throughput: 2974.31 samples/s lr: 7.77e-05 [09/23 09:26:24] lb.utils.events INFO: eta: 2:44:54 iteration: 312199/375342 consumed_samples: 319692800 total_loss: 3.06 time: 0.3443 s/iter data_time: 0.2267 s/iter total_throughput: 2974.30 samples/s lr: 7.75e-05 [09/23 09:26:59] lb.utils.events INFO: eta: 2:44:18 iteration: 312299/375342 consumed_samples: 319795200 total_loss: 3.057 time: 0.3443 s/iter data_time: 0.2145 s/iter total_throughput: 2974.29 samples/s lr: 7.73e-05 [09/23 09:27:33] lb.utils.events INFO: eta: 2:44:24 iteration: 312399/375342 consumed_samples: 319897600 total_loss: 3.062 time: 0.3443 s/iter data_time: 0.2129 s/iter total_throughput: 2974.30 samples/s lr: 7.71e-05 [09/23 09:28:08] lb.utils.events INFO: eta: 2:44:07 iteration: 312499/375342 consumed_samples: 320000000 total_loss: 3.061 time: 0.3443 s/iter data_time: 0.2241 s/iter total_throughput: 2974.29 samples/s lr: 7.69e-05 [09/23 09:28:42] lb.utils.events INFO: eta: 2:43:53 iteration: 312599/375342 consumed_samples: 320102400 total_loss: 3.069 time: 0.3443 s/iter data_time: 0.2163 s/iter total_throughput: 2974.29 samples/s lr: 7.67e-05 [09/23 09:29:17] lb.utils.events INFO: eta: 2:43:52 iteration: 312699/375342 consumed_samples: 320204800 total_loss: 3.069 time: 0.3443 s/iter data_time: 0.2169 s/iter total_throughput: 2974.29 samples/s lr: 7.65e-05 [09/23 09:29:51] lb.utils.events INFO: eta: 2:43:44 iteration: 312799/375342 consumed_samples: 320307200 total_loss: 3.056 time: 0.3443 s/iter data_time: 0.2115 s/iter total_throughput: 2974.29 samples/s lr: 7.63e-05 [09/23 09:30:26] lb.utils.events INFO: eta: 2:43:14 iteration: 312899/375342 consumed_samples: 320409600 total_loss: 3.062 time: 0.3443 s/iter data_time: 0.2243 s/iter total_throughput: 2974.29 samples/s lr: 7.61e-05 [09/23 09:31:00] lb.utils.events INFO: eta: 2:42:58 iteration: 312999/375342 consumed_samples: 320512000 total_loss: 3.064 time: 0.3443 s/iter data_time: 0.2100 s/iter total_throughput: 2974.29 samples/s lr: 7.59e-05 [09/23 09:31:35] lb.utils.events INFO: eta: 2:42:47 iteration: 313099/375342 consumed_samples: 320614400 total_loss: 3.066 time: 0.3443 s/iter data_time: 0.2088 s/iter total_throughput: 2974.29 samples/s lr: 7.57e-05 [09/23 09:32:09] lb.utils.events INFO: eta: 2:42:37 iteration: 313199/375342 consumed_samples: 320716800 total_loss: 3.057 time: 0.3443 s/iter data_time: 0.2126 s/iter total_throughput: 2974.29 samples/s lr: 7.55e-05 [09/23 09:32:43] lb.utils.events INFO: eta: 2:42:22 iteration: 313299/375342 consumed_samples: 320819200 total_loss: 3.048 time: 0.3443 s/iter data_time: 0.2127 s/iter total_throughput: 2974.29 samples/s lr: 7.53e-05 [09/23 09:33:18] lb.utils.events INFO: eta: 2:41:59 iteration: 313399/375342 consumed_samples: 320921600 total_loss: 3.044 time: 0.3443 s/iter data_time: 0.2203 s/iter total_throughput: 2974.28 samples/s lr: 7.51e-05 [09/23 09:33:53] lb.utils.events INFO: eta: 2:41:37 iteration: 313499/375342 consumed_samples: 321024000 total_loss: 3.047 time: 0.3443 s/iter data_time: 0.2259 s/iter total_throughput: 2974.27 samples/s lr: 7.48e-05 [09/23 09:34:28] lb.utils.events INFO: eta: 2:40:46 iteration: 313599/375342 consumed_samples: 321126400 total_loss: 3.064 time: 0.3443 s/iter data_time: 0.2251 s/iter total_throughput: 2974.26 samples/s lr: 7.46e-05 [09/23 09:35:03] lb.utils.events INFO: eta: 2:40:00 iteration: 313699/375342 consumed_samples: 321228800 total_loss: 3.072 time: 0.3443 s/iter data_time: 0.2231 s/iter total_throughput: 2974.25 samples/s lr: 7.44e-05 [09/23 09:35:37] lb.utils.events INFO: eta: 2:39:36 iteration: 313799/375342 consumed_samples: 321331200 total_loss: 3.068 time: 0.3443 s/iter data_time: 0.2141 s/iter total_throughput: 2974.24 samples/s lr: 7.42e-05 [09/23 09:36:12] lb.utils.events INFO: eta: 2:39:11 iteration: 313899/375342 consumed_samples: 321433600 total_loss: 3.071 time: 0.3443 s/iter data_time: 0.2198 s/iter total_throughput: 2974.24 samples/s lr: 7.40e-05 [09/23 09:36:46] lb.utils.events INFO: eta: 2:39:11 iteration: 313999/375342 consumed_samples: 321536000 total_loss: 3.059 time: 0.3443 s/iter data_time: 0.2194 s/iter total_throughput: 2974.23 samples/s lr: 7.38e-05 [09/23 09:37:21] lb.utils.events INFO: eta: 2:38:52 iteration: 314099/375342 consumed_samples: 321638400 total_loss: 3.045 time: 0.3443 s/iter data_time: 0.2198 s/iter total_throughput: 2974.22 samples/s lr: 7.36e-05 [09/23 09:37:56] lb.utils.events INFO: eta: 2:38:43 iteration: 314199/375342 consumed_samples: 321740800 total_loss: 3.044 time: 0.3443 s/iter data_time: 0.2236 s/iter total_throughput: 2974.22 samples/s lr: 7.34e-05 [09/23 09:38:30] lb.utils.events INFO: eta: 2:38:24 iteration: 314299/375342 consumed_samples: 321843200 total_loss: 3.041 time: 0.3443 s/iter data_time: 0.2226 s/iter total_throughput: 2974.22 samples/s lr: 7.32e-05 [09/23 09:39:05] lb.utils.events INFO: eta: 2:38:23 iteration: 314399/375342 consumed_samples: 321945600 total_loss: 3.05 time: 0.3443 s/iter data_time: 0.2064 s/iter total_throughput: 2974.22 samples/s lr: 7.30e-05 [09/23 09:39:39] lb.utils.events INFO: eta: 2:37:50 iteration: 314499/375342 consumed_samples: 322048000 total_loss: 3.055 time: 0.3443 s/iter data_time: 0.2380 s/iter total_throughput: 2974.22 samples/s lr: 7.28e-05 [09/23 09:40:14] lb.utils.events INFO: eta: 2:37:56 iteration: 314599/375342 consumed_samples: 322150400 total_loss: 3.056 time: 0.3443 s/iter data_time: 0.2179 s/iter total_throughput: 2974.22 samples/s lr: 7.26e-05 [09/23 09:40:48] lb.utils.events INFO: eta: 2:38:05 iteration: 314699/375342 consumed_samples: 322252800 total_loss: 3.059 time: 0.3443 s/iter data_time: 0.2100 s/iter total_throughput: 2974.22 samples/s lr: 7.24e-05 [09/23 09:41:22] lb.utils.events INFO: eta: 2:37:59 iteration: 314799/375342 consumed_samples: 322355200 total_loss: 3.061 time: 0.3443 s/iter data_time: 0.2128 s/iter total_throughput: 2974.23 samples/s lr: 7.22e-05 [09/23 09:41:56] lb.utils.events INFO: eta: 2:37:56 iteration: 314899/375342 consumed_samples: 322457600 total_loss: 3.051 time: 0.3443 s/iter data_time: 0.2204 s/iter total_throughput: 2974.23 samples/s lr: 7.20e-05 [09/23 09:42:31] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0314999 [09/23 09:42:32] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/23 09:42:32] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/23 09:42:36] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.1129 s/iter. Inference: 0.1611 s/iter. Eval: 0.0021 s/iter. Total: 0.2761 s/iter. ETA=0:00:10 [09/23 09:42:41] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1274 s/iter. Inference: 0.1677 s/iter. Eval: 0.0022 s/iter. Total: 0.2974 s/iter. ETA=0:00:05 [09/23 09:42:46] lb.evaluation.evaluator INFO: Inference done 48128/50000. Dataloading: 0.1244 s/iter. Inference: 0.1645 s/iter. Eval: 0.0021 s/iter. Total: 0.2911 s/iter. ETA=0:00:00 [09/23 09:42:47] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/23 09:42:47] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.578217 (0.000252 s / iter per device, on 8 devices) [09/23 09:42:47] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000145 s / iter per device, on 8 devices) [09/23 09:42:47] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/23 09:42:47] lb.evaluation.utils INFO: copypaste: Acc@1=79.51 [09/23 09:42:47] lb.evaluation.utils INFO: copypaste: Acc@5=94.338 [09/23 09:42:47] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 79.51000, better than last best score 79.30800 @ iteration 309999. [09/23 09:42:47] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/23 09:42:47] lb.utils.events INFO: eta: 2:37:41 iteration: 314999/375342 consumed_samples: 322560000 total_loss: 3.051 time: 0.3443 s/iter data_time: 0.2245 s/iter total_throughput: 2974.23 samples/s lr: 7.18e-05 [09/23 09:43:20] lb.utils.events INFO: eta: 2:38:15 iteration: 315099/375342 consumed_samples: 322662400 total_loss: 3.053 time: 0.3443 s/iter data_time: 0.2313 s/iter total_throughput: 2974.29 samples/s lr: 7.16e-05 [09/23 09:43:54] lb.utils.events INFO: eta: 2:37:44 iteration: 315199/375342 consumed_samples: 322764800 total_loss: 3.043 time: 0.3443 s/iter data_time: 0.2207 s/iter total_throughput: 2974.29 samples/s lr: 7.14e-05 [09/23 09:44:29] lb.utils.events INFO: eta: 2:37:55 iteration: 315299/375342 consumed_samples: 322867200 total_loss: 3.057 time: 0.3443 s/iter data_time: 0.2159 s/iter total_throughput: 2974.28 samples/s lr: 7.12e-05 [09/23 09:45:04] lb.utils.events INFO: eta: 2:37:43 iteration: 315399/375342 consumed_samples: 322969600 total_loss: 3.059 time: 0.3443 s/iter data_time: 0.2129 s/iter total_throughput: 2974.28 samples/s lr: 7.10e-05 [09/23 09:45:38] lb.utils.events INFO: eta: 2:37:39 iteration: 315499/375342 consumed_samples: 323072000 total_loss: 3.048 time: 0.3443 s/iter data_time: 0.2242 s/iter total_throughput: 2974.28 samples/s lr: 7.08e-05 [09/23 09:46:12] lb.utils.events INFO: eta: 2:37:29 iteration: 315599/375342 consumed_samples: 323174400 total_loss: 3.044 time: 0.3443 s/iter data_time: 0.2111 s/iter total_throughput: 2974.29 samples/s lr: 7.06e-05 [09/23 09:46:47] lb.utils.events INFO: eta: 2:37:15 iteration: 315699/375342 consumed_samples: 323276800 total_loss: 3.065 time: 0.3443 s/iter data_time: 0.2153 s/iter total_throughput: 2974.29 samples/s lr: 7.04e-05 [09/23 09:47:21] lb.utils.events INFO: eta: 2:36:47 iteration: 315799/375342 consumed_samples: 323379200 total_loss: 3.049 time: 0.3443 s/iter data_time: 0.2123 s/iter total_throughput: 2974.30 samples/s lr: 7.02e-05 [09/23 09:47:55] lb.utils.events INFO: eta: 2:36:19 iteration: 315899/375342 consumed_samples: 323481600 total_loss: 3.05 time: 0.3443 s/iter data_time: 0.2236 s/iter total_throughput: 2974.29 samples/s lr: 7.00e-05 [09/23 09:48:30] lb.utils.events INFO: eta: 2:35:57 iteration: 315999/375342 consumed_samples: 323584000 total_loss: 3.06 time: 0.3443 s/iter data_time: 0.2287 s/iter total_throughput: 2974.30 samples/s lr: 6.98e-05 [09/23 09:49:05] lb.utils.events INFO: eta: 2:35:08 iteration: 316099/375342 consumed_samples: 323686400 total_loss: 3.047 time: 0.3443 s/iter data_time: 0.2310 s/iter total_throughput: 2974.28 samples/s lr: 6.96e-05 [09/23 09:49:39] lb.utils.events INFO: eta: 2:34:51 iteration: 316199/375342 consumed_samples: 323788800 total_loss: 3.044 time: 0.3443 s/iter data_time: 0.2071 s/iter total_throughput: 2974.28 samples/s lr: 6.94e-05 [09/23 09:50:13] lb.utils.events INFO: eta: 2:34:37 iteration: 316299/375342 consumed_samples: 323891200 total_loss: 3.048 time: 0.3443 s/iter data_time: 0.2051 s/iter total_throughput: 2974.29 samples/s lr: 6.92e-05 [09/23 09:50:47] lb.utils.events INFO: eta: 2:34:16 iteration: 316399/375342 consumed_samples: 323993600 total_loss: 3.062 time: 0.3443 s/iter data_time: 0.2093 s/iter total_throughput: 2974.30 samples/s lr: 6.90e-05 [09/23 09:51:22] lb.utils.events INFO: eta: 2:33:41 iteration: 316499/375342 consumed_samples: 324096000 total_loss: 3.05 time: 0.3443 s/iter data_time: 0.2297 s/iter total_throughput: 2974.28 samples/s lr: 6.88e-05 [09/23 09:51:57] lb.utils.events INFO: eta: 2:33:22 iteration: 316599/375342 consumed_samples: 324198400 total_loss: 3.036 time: 0.3443 s/iter data_time: 0.2196 s/iter total_throughput: 2974.28 samples/s lr: 6.86e-05 [09/23 09:52:31] lb.utils.events INFO: eta: 2:33:10 iteration: 316699/375342 consumed_samples: 324300800 total_loss: 3.038 time: 0.3443 s/iter data_time: 0.2259 s/iter total_throughput: 2974.27 samples/s lr: 6.84e-05 [09/23 09:53:06] lb.utils.events INFO: eta: 2:32:43 iteration: 316799/375342 consumed_samples: 324403200 total_loss: 3.037 time: 0.3443 s/iter data_time: 0.2193 s/iter total_throughput: 2974.28 samples/s lr: 6.82e-05 [09/23 09:53:40] lb.utils.events INFO: eta: 2:32:17 iteration: 316899/375342 consumed_samples: 324505600 total_loss: 3.047 time: 0.3443 s/iter data_time: 0.2104 s/iter total_throughput: 2974.28 samples/s lr: 6.81e-05 [09/23 09:54:14] lb.utils.events INFO: eta: 2:32:12 iteration: 316999/375342 consumed_samples: 324608000 total_loss: 3.049 time: 0.3443 s/iter data_time: 0.2171 s/iter total_throughput: 2974.28 samples/s lr: 6.79e-05 [09/23 09:54:49] lb.utils.events INFO: eta: 2:31:52 iteration: 317099/375342 consumed_samples: 324710400 total_loss: 3.029 time: 0.3443 s/iter data_time: 0.2135 s/iter total_throughput: 2974.29 samples/s lr: 6.77e-05 [09/23 09:55:23] lb.utils.events INFO: eta: 2:31:48 iteration: 317199/375342 consumed_samples: 324812800 total_loss: 3.017 time: 0.3443 s/iter data_time: 0.2235 s/iter total_throughput: 2974.28 samples/s lr: 6.75e-05 [09/23 09:55:57] lb.utils.events INFO: eta: 2:31:30 iteration: 317299/375342 consumed_samples: 324915200 total_loss: 3.033 time: 0.3443 s/iter data_time: 0.2177 s/iter total_throughput: 2974.29 samples/s lr: 6.73e-05 [09/23 09:56:32] lb.utils.events INFO: eta: 2:31:16 iteration: 317399/375342 consumed_samples: 325017600 total_loss: 3.046 time: 0.3443 s/iter data_time: 0.2208 s/iter total_throughput: 2974.29 samples/s lr: 6.71e-05 [09/23 09:57:06] lb.utils.events INFO: eta: 2:31:01 iteration: 317499/375342 consumed_samples: 325120000 total_loss: 3.046 time: 0.3443 s/iter data_time: 0.2175 s/iter total_throughput: 2974.29 samples/s lr: 6.69e-05 [09/23 09:57:41] lb.utils.events INFO: eta: 2:30:26 iteration: 317599/375342 consumed_samples: 325222400 total_loss: 3.051 time: 0.3443 s/iter data_time: 0.2112 s/iter total_throughput: 2974.30 samples/s lr: 6.67e-05 [09/23 09:58:15] lb.utils.events INFO: eta: 2:30:05 iteration: 317699/375342 consumed_samples: 325324800 total_loss: 3.06 time: 0.3443 s/iter data_time: 0.2197 s/iter total_throughput: 2974.31 samples/s lr: 6.65e-05 [09/23 09:58:49] lb.utils.events INFO: eta: 2:30:15 iteration: 317799/375342 consumed_samples: 325427200 total_loss: 3.04 time: 0.3443 s/iter data_time: 0.2058 s/iter total_throughput: 2974.32 samples/s lr: 6.63e-05 [09/23 09:59:23] lb.utils.events INFO: eta: 2:30:32 iteration: 317899/375342 consumed_samples: 325529600 total_loss: 3.021 time: 0.3443 s/iter data_time: 0.2055 s/iter total_throughput: 2974.33 samples/s lr: 6.61e-05 [09/23 09:59:57] lb.utils.events INFO: eta: 2:29:59 iteration: 317999/375342 consumed_samples: 325632000 total_loss: 3.038 time: 0.3443 s/iter data_time: 0.2085 s/iter total_throughput: 2974.33 samples/s lr: 6.59e-05 [09/23 10:00:32] lb.utils.events INFO: eta: 2:29:50 iteration: 318099/375342 consumed_samples: 325734400 total_loss: 3.039 time: 0.3443 s/iter data_time: 0.2161 s/iter total_throughput: 2974.33 samples/s lr: 6.57e-05 [09/23 10:01:06] lb.utils.events INFO: eta: 2:29:28 iteration: 318199/375342 consumed_samples: 325836800 total_loss: 3.047 time: 0.3443 s/iter data_time: 0.2134 s/iter total_throughput: 2974.32 samples/s lr: 6.55e-05 [09/23 10:01:41] lb.utils.events INFO: eta: 2:29:13 iteration: 318299/375342 consumed_samples: 325939200 total_loss: 3.064 time: 0.3443 s/iter data_time: 0.2347 s/iter total_throughput: 2974.32 samples/s lr: 6.54e-05 [09/23 10:02:15] lb.utils.events INFO: eta: 2:29:03 iteration: 318399/375342 consumed_samples: 326041600 total_loss: 3.04 time: 0.3443 s/iter data_time: 0.2283 s/iter total_throughput: 2974.32 samples/s lr: 6.52e-05 [09/23 10:02:50] lb.utils.events INFO: eta: 2:28:36 iteration: 318499/375342 consumed_samples: 326144000 total_loss: 3.032 time: 0.3443 s/iter data_time: 0.2295 s/iter total_throughput: 2974.30 samples/s lr: 6.50e-05 [09/23 10:03:24] lb.utils.events INFO: eta: 2:28:32 iteration: 318599/375342 consumed_samples: 326246400 total_loss: 3.047 time: 0.3443 s/iter data_time: 0.2229 s/iter total_throughput: 2974.31 samples/s lr: 6.48e-05 [09/23 10:03:59] lb.utils.events INFO: eta: 2:28:22 iteration: 318699/375342 consumed_samples: 326348800 total_loss: 3.052 time: 0.3443 s/iter data_time: 0.2094 s/iter total_throughput: 2974.32 samples/s lr: 6.46e-05 [09/23 10:04:33] lb.utils.events INFO: eta: 2:27:48 iteration: 318799/375342 consumed_samples: 326451200 total_loss: 3.054 time: 0.3443 s/iter data_time: 0.2163 s/iter total_throughput: 2974.32 samples/s lr: 6.44e-05 [09/23 10:05:07] lb.utils.events INFO: eta: 2:27:04 iteration: 318899/375342 consumed_samples: 326553600 total_loss: 3.039 time: 0.3443 s/iter data_time: 0.2145 s/iter total_throughput: 2974.32 samples/s lr: 6.42e-05 [09/23 10:05:42] lb.utils.events INFO: eta: 2:27:01 iteration: 318999/375342 consumed_samples: 326656000 total_loss: 3.037 time: 0.3443 s/iter data_time: 0.2177 s/iter total_throughput: 2974.33 samples/s lr: 6.40e-05 [09/23 10:06:16] lb.utils.events INFO: eta: 2:26:34 iteration: 319099/375342 consumed_samples: 326758400 total_loss: 3.04 time: 0.3443 s/iter data_time: 0.2156 s/iter total_throughput: 2974.33 samples/s lr: 6.38e-05 [09/23 10:06:50] lb.utils.events INFO: eta: 2:26:20 iteration: 319199/375342 consumed_samples: 326860800 total_loss: 3.047 time: 0.3443 s/iter data_time: 0.2127 s/iter total_throughput: 2974.34 samples/s lr: 6.37e-05 [09/23 10:07:25] lb.utils.events INFO: eta: 2:25:49 iteration: 319299/375342 consumed_samples: 326963200 total_loss: 3.05 time: 0.3443 s/iter data_time: 0.2196 s/iter total_throughput: 2974.33 samples/s lr: 6.35e-05 [09/23 10:07:59] lb.utils.events INFO: eta: 2:25:09 iteration: 319399/375342 consumed_samples: 327065600 total_loss: 3.036 time: 0.3443 s/iter data_time: 0.2096 s/iter total_throughput: 2974.33 samples/s lr: 6.33e-05 [09/23 10:08:33] lb.utils.events INFO: eta: 2:25:21 iteration: 319499/375342 consumed_samples: 327168000 total_loss: 3.029 time: 0.3443 s/iter data_time: 0.2092 s/iter total_throughput: 2974.34 samples/s lr: 6.31e-05 [09/23 10:09:07] lb.utils.events INFO: eta: 2:24:56 iteration: 319599/375342 consumed_samples: 327270400 total_loss: 3.035 time: 0.3443 s/iter data_time: 0.2094 s/iter total_throughput: 2974.35 samples/s lr: 6.29e-05 [09/23 10:09:42] lb.utils.events INFO: eta: 2:24:25 iteration: 319699/375342 consumed_samples: 327372800 total_loss: 3.062 time: 0.3443 s/iter data_time: 0.2130 s/iter total_throughput: 2974.36 samples/s lr: 6.27e-05 [09/23 10:10:16] lb.utils.events INFO: eta: 2:24:18 iteration: 319799/375342 consumed_samples: 327475200 total_loss: 3.046 time: 0.3443 s/iter data_time: 0.2180 s/iter total_throughput: 2974.36 samples/s lr: 6.25e-05 [09/23 10:10:50] lb.utils.events INFO: eta: 2:23:54 iteration: 319899/375342 consumed_samples: 327577600 total_loss: 3.034 time: 0.3443 s/iter data_time: 0.2162 s/iter total_throughput: 2974.36 samples/s lr: 6.23e-05 [09/23 10:11:25] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0319999 [09/23 10:11:26] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/23 10:11:26] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/23 10:11:30] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.1020 s/iter. Inference: 0.1630 s/iter. Eval: 0.0020 s/iter. Total: 0.2669 s/iter. ETA=0:00:09 [09/23 10:11:35] lb.evaluation.evaluator INFO: Inference done 28672/50000. Dataloading: 0.1099 s/iter. Inference: 0.1773 s/iter. Eval: 0.0020 s/iter. Total: 0.2893 s/iter. ETA=0:00:05 [09/23 10:11:40] lb.evaluation.evaluator INFO: Inference done 47104/50000. Dataloading: 0.1183 s/iter. Inference: 0.1712 s/iter. Eval: 0.0020 s/iter. Total: 0.2916 s/iter. ETA=0:00:00 [09/23 10:11:41] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/23 10:11:41] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.457130 (0.000249 s / iter per device, on 8 devices) [09/23 10:11:41] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000150 s / iter per device, on 8 devices) [09/23 10:11:41] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/23 10:11:41] lb.evaluation.utils INFO: copypaste: Acc@1=79.56400000000001 [09/23 10:11:41] lb.evaluation.utils INFO: copypaste: Acc@5=94.39800000000001 [09/23 10:11:41] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 79.56400, better than last best score 79.51000 @ iteration 314999. [09/23 10:11:41] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/23 10:11:41] lb.utils.events INFO: eta: 2:23:24 iteration: 319999/375342 consumed_samples: 327680000 total_loss: 3.04 time: 0.3443 s/iter data_time: 0.2119 s/iter total_throughput: 2974.36 samples/s lr: 6.22e-05 [09/23 10:12:13] lb.utils.events INFO: eta: 2:24:15 iteration: 320099/375342 consumed_samples: 327782400 total_loss: 3.021 time: 0.3443 s/iter data_time: 0.2071 s/iter total_throughput: 2974.42 samples/s lr: 6.20e-05 [09/23 10:12:48] lb.utils.events INFO: eta: 2:23:52 iteration: 320199/375342 consumed_samples: 327884800 total_loss: 3.027 time: 0.3443 s/iter data_time: 0.2149 s/iter total_throughput: 2974.42 samples/s lr: 6.18e-05 [09/23 10:13:23] lb.utils.events INFO: eta: 2:23:40 iteration: 320299/375342 consumed_samples: 327987200 total_loss: 3.055 time: 0.3443 s/iter data_time: 0.2188 s/iter total_throughput: 2974.41 samples/s lr: 6.16e-05 [09/23 10:13:57] lb.utils.events INFO: eta: 2:23:35 iteration: 320399/375342 consumed_samples: 328089600 total_loss: 3.038 time: 0.3443 s/iter data_time: 0.2071 s/iter total_throughput: 2974.42 samples/s lr: 6.14e-05 [09/23 10:14:31] lb.utils.events INFO: eta: 2:23:15 iteration: 320499/375342 consumed_samples: 328192000 total_loss: 3.032 time: 0.3443 s/iter data_time: 0.2134 s/iter total_throughput: 2974.42 samples/s lr: 6.12e-05 [09/23 10:15:05] lb.utils.events INFO: eta: 2:23:26 iteration: 320599/375342 consumed_samples: 328294400 total_loss: 3.041 time: 0.3443 s/iter data_time: 0.2050 s/iter total_throughput: 2974.43 samples/s lr: 6.11e-05 [09/23 10:15:39] lb.utils.events INFO: eta: 2:22:56 iteration: 320699/375342 consumed_samples: 328396800 total_loss: 3.049 time: 0.3443 s/iter data_time: 0.2169 s/iter total_throughput: 2974.44 samples/s lr: 6.09e-05 [09/23 10:16:14] lb.utils.events INFO: eta: 2:22:16 iteration: 320799/375342 consumed_samples: 328499200 total_loss: 3.027 time: 0.3443 s/iter data_time: 0.2182 s/iter total_throughput: 2974.43 samples/s lr: 6.07e-05 [09/23 10:16:48] lb.utils.events INFO: eta: 2:22:02 iteration: 320899/375342 consumed_samples: 328601600 total_loss: 3.019 time: 0.3443 s/iter data_time: 0.2103 s/iter total_throughput: 2974.45 samples/s lr: 6.05e-05 [09/23 10:17:22] lb.utils.events INFO: eta: 2:22:23 iteration: 320999/375342 consumed_samples: 328704000 total_loss: 3.024 time: 0.3443 s/iter data_time: 0.2116 s/iter total_throughput: 2974.47 samples/s lr: 6.03e-05 [09/23 10:17:56] lb.utils.events INFO: eta: 2:20:57 iteration: 321099/375342 consumed_samples: 328806400 total_loss: 3.031 time: 0.3443 s/iter data_time: 0.2266 s/iter total_throughput: 2974.48 samples/s lr: 6.01e-05 [09/23 10:18:30] lb.utils.events INFO: eta: 2:20:44 iteration: 321199/375342 consumed_samples: 328908800 total_loss: 3.041 time: 0.3443 s/iter data_time: 0.2263 s/iter total_throughput: 2974.48 samples/s lr: 6.00e-05 [09/23 10:19:04] lb.utils.events INFO: eta: 2:20:38 iteration: 321299/375342 consumed_samples: 329011200 total_loss: 3.039 time: 0.3443 s/iter data_time: 0.2090 s/iter total_throughput: 2974.49 samples/s lr: 5.98e-05 [09/23 10:19:38] lb.utils.events INFO: eta: 2:20:13 iteration: 321399/375342 consumed_samples: 329113600 total_loss: 3.04 time: 0.3443 s/iter data_time: 0.2092 s/iter total_throughput: 2974.50 samples/s lr: 5.96e-05 [09/23 10:20:12] lb.utils.events INFO: eta: 2:20:19 iteration: 321499/375342 consumed_samples: 329216000 total_loss: 3.039 time: 0.3443 s/iter data_time: 0.2068 s/iter total_throughput: 2974.52 samples/s lr: 5.94e-05 [09/23 10:20:46] lb.utils.events INFO: eta: 2:20:03 iteration: 321599/375342 consumed_samples: 329318400 total_loss: 3.019 time: 0.3443 s/iter data_time: 0.2112 s/iter total_throughput: 2974.53 samples/s lr: 5.92e-05 [09/23 10:21:20] lb.utils.events INFO: eta: 2:19:50 iteration: 321699/375342 consumed_samples: 329420800 total_loss: 3.008 time: 0.3443 s/iter data_time: 0.2233 s/iter total_throughput: 2974.53 samples/s lr: 5.91e-05 [09/23 10:21:54] lb.utils.events INFO: eta: 2:19:47 iteration: 321799/375342 consumed_samples: 329523200 total_loss: 3.026 time: 0.3443 s/iter data_time: 0.2165 s/iter total_throughput: 2974.55 samples/s lr: 5.89e-05 [09/23 10:22:28] lb.utils.events INFO: eta: 2:19:41 iteration: 321899/375342 consumed_samples: 329625600 total_loss: 3.018 time: 0.3443 s/iter data_time: 0.2057 s/iter total_throughput: 2974.57 samples/s lr: 5.87e-05 [09/23 10:23:02] lb.utils.events INFO: eta: 2:19:18 iteration: 321999/375342 consumed_samples: 329728000 total_loss: 3.029 time: 0.3443 s/iter data_time: 0.2234 s/iter total_throughput: 2974.57 samples/s lr: 5.85e-05 [09/23 10:23:36] lb.utils.events INFO: eta: 2:18:54 iteration: 322099/375342 consumed_samples: 329830400 total_loss: 3.048 time: 0.3443 s/iter data_time: 0.2111 s/iter total_throughput: 2974.58 samples/s lr: 5.83e-05 [09/23 10:24:11] lb.utils.events INFO: eta: 2:18:35 iteration: 322199/375342 consumed_samples: 329932800 total_loss: 3.047 time: 0.3443 s/iter data_time: 0.2167 s/iter total_throughput: 2974.58 samples/s lr: 5.82e-05 [09/23 10:24:45] lb.utils.events INFO: eta: 2:18:24 iteration: 322299/375342 consumed_samples: 330035200 total_loss: 3.06 time: 0.3443 s/iter data_time: 0.2194 s/iter total_throughput: 2974.58 samples/s lr: 5.80e-05 [09/23 10:25:20] lb.utils.events INFO: eta: 2:18:05 iteration: 322399/375342 consumed_samples: 330137600 total_loss: 3.046 time: 0.3443 s/iter data_time: 0.2198 s/iter total_throughput: 2974.58 samples/s lr: 5.78e-05 [09/23 10:25:54] lb.utils.events INFO: eta: 2:17:41 iteration: 322499/375342 consumed_samples: 330240000 total_loss: 3.033 time: 0.3443 s/iter data_time: 0.2245 s/iter total_throughput: 2974.57 samples/s lr: 5.76e-05 [09/23 10:26:30] lb.utils.events INFO: eta: 2:16:59 iteration: 322599/375342 consumed_samples: 330342400 total_loss: 3.035 time: 0.3443 s/iter data_time: 0.2218 s/iter total_throughput: 2974.55 samples/s lr: 5.75e-05 [09/23 10:27:05] lb.utils.events INFO: eta: 2:16:46 iteration: 322699/375342 consumed_samples: 330444800 total_loss: 3.031 time: 0.3443 s/iter data_time: 0.2281 s/iter total_throughput: 2974.53 samples/s lr: 5.73e-05 [09/23 10:27:40] lb.utils.events INFO: eta: 2:16:31 iteration: 322799/375342 consumed_samples: 330547200 total_loss: 3.025 time: 0.3443 s/iter data_time: 0.2217 s/iter total_throughput: 2974.52 samples/s lr: 5.71e-05 [09/23 10:28:14] lb.utils.events INFO: eta: 2:16:11 iteration: 322899/375342 consumed_samples: 330649600 total_loss: 3.025 time: 0.3443 s/iter data_time: 0.2329 s/iter total_throughput: 2974.51 samples/s lr: 5.69e-05 [09/23 10:28:49] lb.utils.events INFO: eta: 2:16:04 iteration: 322999/375342 consumed_samples: 330752000 total_loss: 3.017 time: 0.3443 s/iter data_time: 0.2225 s/iter total_throughput: 2974.51 samples/s lr: 5.67e-05 [09/23 10:29:23] lb.utils.events INFO: eta: 2:16:14 iteration: 323099/375342 consumed_samples: 330854400 total_loss: 3.021 time: 0.3443 s/iter data_time: 0.2148 s/iter total_throughput: 2974.51 samples/s lr: 5.66e-05 [09/23 10:29:58] lb.utils.events INFO: eta: 2:16:07 iteration: 323199/375342 consumed_samples: 330956800 total_loss: 3.028 time: 0.3443 s/iter data_time: 0.2225 s/iter total_throughput: 2974.51 samples/s lr: 5.64e-05 [09/23 10:30:33] lb.utils.events INFO: eta: 2:15:42 iteration: 323299/375342 consumed_samples: 331059200 total_loss: 3.021 time: 0.3443 s/iter data_time: 0.2276 s/iter total_throughput: 2974.50 samples/s lr: 5.62e-05 [09/23 10:31:08] lb.utils.events INFO: eta: 2:15:43 iteration: 323399/375342 consumed_samples: 331161600 total_loss: 3.028 time: 0.3443 s/iter data_time: 0.2278 s/iter total_throughput: 2974.48 samples/s lr: 5.60e-05 [09/23 10:31:42] lb.utils.events INFO: eta: 2:15:21 iteration: 323499/375342 consumed_samples: 331264000 total_loss: 3.026 time: 0.3443 s/iter data_time: 0.2314 s/iter total_throughput: 2974.48 samples/s lr: 5.59e-05 [09/23 10:32:17] lb.utils.events INFO: eta: 2:15:17 iteration: 323599/375342 consumed_samples: 331366400 total_loss: 3.021 time: 0.3443 s/iter data_time: 0.2214 s/iter total_throughput: 2974.48 samples/s lr: 5.57e-05 [09/23 10:32:51] lb.utils.events INFO: eta: 2:15:01 iteration: 323699/375342 consumed_samples: 331468800 total_loss: 3.014 time: 0.3443 s/iter data_time: 0.2215 s/iter total_throughput: 2974.47 samples/s lr: 5.55e-05 [09/23 10:33:26] lb.utils.events INFO: eta: 2:15:07 iteration: 323799/375342 consumed_samples: 331571200 total_loss: 3.008 time: 0.3443 s/iter data_time: 0.2178 s/iter total_throughput: 2974.46 samples/s lr: 5.54e-05 [09/23 10:34:01] lb.utils.events INFO: eta: 2:14:31 iteration: 323899/375342 consumed_samples: 331673600 total_loss: 3.008 time: 0.3443 s/iter data_time: 0.2331 s/iter total_throughput: 2974.45 samples/s lr: 5.52e-05 [09/23 10:34:36] lb.utils.events INFO: eta: 2:14:03 iteration: 323999/375342 consumed_samples: 331776000 total_loss: 3.023 time: 0.3443 s/iter data_time: 0.2204 s/iter total_throughput: 2974.44 samples/s lr: 5.50e-05 [09/23 10:35:10] lb.utils.events INFO: eta: 2:13:28 iteration: 324099/375342 consumed_samples: 331878400 total_loss: 3.041 time: 0.3443 s/iter data_time: 0.2201 s/iter total_throughput: 2974.43 samples/s lr: 5.48e-05 [09/23 10:35:45] lb.utils.events INFO: eta: 2:12:53 iteration: 324199/375342 consumed_samples: 331980800 total_loss: 3.032 time: 0.3443 s/iter data_time: 0.2186 s/iter total_throughput: 2974.42 samples/s lr: 5.47e-05 [09/23 10:36:20] lb.utils.events INFO: eta: 2:12:41 iteration: 324299/375342 consumed_samples: 332083200 total_loss: 3.023 time: 0.3443 s/iter data_time: 0.2243 s/iter total_throughput: 2974.42 samples/s lr: 5.45e-05 [09/23 10:36:55] lb.utils.events INFO: eta: 2:12:11 iteration: 324399/375342 consumed_samples: 332185600 total_loss: 3.011 time: 0.3443 s/iter data_time: 0.2184 s/iter total_throughput: 2974.41 samples/s lr: 5.43e-05 [09/23 10:37:30] lb.utils.events INFO: eta: 2:11:50 iteration: 324499/375342 consumed_samples: 332288000 total_loss: 3.011 time: 0.3443 s/iter data_time: 0.2248 s/iter total_throughput: 2974.39 samples/s lr: 5.41e-05 [09/23 10:38:05] lb.utils.events INFO: eta: 2:11:29 iteration: 324599/375342 consumed_samples: 332390400 total_loss: 3.035 time: 0.3443 s/iter data_time: 0.2245 s/iter total_throughput: 2974.38 samples/s lr: 5.40e-05 [09/23 10:38:39] lb.utils.events INFO: eta: 2:11:20 iteration: 324699/375342 consumed_samples: 332492800 total_loss: 3.017 time: 0.3443 s/iter data_time: 0.2288 s/iter total_throughput: 2974.38 samples/s lr: 5.38e-05 [09/23 10:39:14] lb.utils.events INFO: eta: 2:10:43 iteration: 324799/375342 consumed_samples: 332595200 total_loss: 3.012 time: 0.3443 s/iter data_time: 0.2266 s/iter total_throughput: 2974.36 samples/s lr: 5.36e-05 [09/23 10:39:49] lb.utils.events INFO: eta: 2:10:26 iteration: 324899/375342 consumed_samples: 332697600 total_loss: 3.03 time: 0.3443 s/iter data_time: 0.2277 s/iter total_throughput: 2974.35 samples/s lr: 5.35e-05 [09/23 10:40:24] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0324999 [09/23 10:40:24] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/23 10:40:24] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/23 10:40:28] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0867 s/iter. Inference: 0.1613 s/iter. Eval: 0.0022 s/iter. Total: 0.2502 s/iter. ETA=0:00:09 [09/23 10:40:34] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1354 s/iter. Inference: 0.1666 s/iter. Eval: 0.0022 s/iter. Total: 0.3042 s/iter. ETA=0:00:05 [09/23 10:40:40] lb.evaluation.evaluator INFO: Inference done 50000/50000. Dataloading: 0.1214 s/iter. Inference: 0.1645 s/iter. Eval: 0.0021 s/iter. Total: 0.2881 s/iter. ETA=0:00:00 [09/23 10:40:40] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/23 10:40:40] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.682062 (0.000254 s / iter per device, on 8 devices) [09/23 10:40:40] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000145 s / iter per device, on 8 devices) [09/23 10:40:40] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/23 10:40:40] lb.evaluation.utils INFO: copypaste: Acc@1=79.674 [09/23 10:40:40] lb.evaluation.utils INFO: copypaste: Acc@5=94.498 [09/23 10:40:40] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 79.67400, better than last best score 79.56400 @ iteration 319999. [09/23 10:40:40] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/23 10:40:40] lb.utils.events INFO: eta: 2:10:17 iteration: 324999/375342 consumed_samples: 332800000 total_loss: 3.027 time: 0.3443 s/iter data_time: 0.2150 s/iter total_throughput: 2974.35 samples/s lr: 5.33e-05 [09/23 10:41:13] lb.utils.events INFO: eta: 2:10:26 iteration: 325099/375342 consumed_samples: 332902400 total_loss: 3.021 time: 0.3443 s/iter data_time: 0.2154 s/iter total_throughput: 2974.40 samples/s lr: 5.31e-05 [09/23 10:41:47] lb.utils.events INFO: eta: 2:10:47 iteration: 325199/375342 consumed_samples: 333004800 total_loss: 3.019 time: 0.3443 s/iter data_time: 0.2207 s/iter total_throughput: 2974.40 samples/s lr: 5.30e-05 [09/23 10:42:22] lb.utils.events INFO: eta: 2:10:36 iteration: 325299/375342 consumed_samples: 333107200 total_loss: 3.01 time: 0.3443 s/iter data_time: 0.2171 s/iter total_throughput: 2974.39 samples/s lr: 5.28e-05 [09/23 10:42:57] lb.utils.events INFO: eta: 2:10:11 iteration: 325399/375342 consumed_samples: 333209600 total_loss: 3.002 time: 0.3443 s/iter data_time: 0.2313 s/iter total_throughput: 2974.38 samples/s lr: 5.26e-05 [09/23 10:43:32] lb.utils.events INFO: eta: 2:09:53 iteration: 325499/375342 consumed_samples: 333312000 total_loss: 3.008 time: 0.3443 s/iter data_time: 0.2234 s/iter total_throughput: 2974.37 samples/s lr: 5.25e-05 [09/23 10:44:07] lb.utils.events INFO: eta: 2:09:42 iteration: 325599/375342 consumed_samples: 333414400 total_loss: 3.013 time: 0.3443 s/iter data_time: 0.2258 s/iter total_throughput: 2974.36 samples/s lr: 5.23e-05 [09/23 10:44:41] lb.utils.events INFO: eta: 2:09:25 iteration: 325699/375342 consumed_samples: 333516800 total_loss: 3.026 time: 0.3443 s/iter data_time: 0.2194 s/iter total_throughput: 2974.36 samples/s lr: 5.21e-05 [09/23 10:45:16] lb.utils.events INFO: eta: 2:09:33 iteration: 325799/375342 consumed_samples: 333619200 total_loss: 3.023 time: 0.3443 s/iter data_time: 0.2322 s/iter total_throughput: 2974.34 samples/s lr: 5.20e-05 [09/23 10:45:50] lb.utils.events INFO: eta: 2:09:46 iteration: 325899/375342 consumed_samples: 333721600 total_loss: 3.004 time: 0.3443 s/iter data_time: 0.2224 s/iter total_throughput: 2974.34 samples/s lr: 5.18e-05 [09/23 10:46:25] lb.utils.events INFO: eta: 2:09:27 iteration: 325999/375342 consumed_samples: 333824000 total_loss: 3.014 time: 0.3443 s/iter data_time: 0.2270 s/iter total_throughput: 2974.33 samples/s lr: 5.16e-05 [09/23 10:47:00] lb.utils.events INFO: eta: 2:08:54 iteration: 326099/375342 consumed_samples: 333926400 total_loss: 3.016 time: 0.3443 s/iter data_time: 0.2166 s/iter total_throughput: 2974.34 samples/s lr: 5.15e-05 [09/23 10:47:34] lb.utils.events INFO: eta: 2:08:25 iteration: 326199/375342 consumed_samples: 334028800 total_loss: 3.004 time: 0.3443 s/iter data_time: 0.2207 s/iter total_throughput: 2974.33 samples/s lr: 5.13e-05 [09/23 10:48:09] lb.utils.events INFO: eta: 2:07:52 iteration: 326299/375342 consumed_samples: 334131200 total_loss: 3.021 time: 0.3443 s/iter data_time: 0.2256 s/iter total_throughput: 2974.33 samples/s lr: 5.11e-05 [09/23 10:48:43] lb.utils.events INFO: eta: 2:07:48 iteration: 326399/375342 consumed_samples: 334233600 total_loss: 3.014 time: 0.3443 s/iter data_time: 0.2136 s/iter total_throughput: 2974.33 samples/s lr: 5.10e-05 [09/23 10:49:18] lb.utils.events INFO: eta: 2:07:45 iteration: 326499/375342 consumed_samples: 334336000 total_loss: 3.008 time: 0.3443 s/iter data_time: 0.2197 s/iter total_throughput: 2974.33 samples/s lr: 5.08e-05 [09/23 10:49:52] lb.utils.events INFO: eta: 2:07:37 iteration: 326599/375342 consumed_samples: 334438400 total_loss: 3.003 time: 0.3443 s/iter data_time: 0.2211 s/iter total_throughput: 2974.32 samples/s lr: 5.06e-05 [09/23 10:50:27] lb.utils.events INFO: eta: 2:07:24 iteration: 326699/375342 consumed_samples: 334540800 total_loss: 2.999 time: 0.3443 s/iter data_time: 0.2088 s/iter total_throughput: 2974.31 samples/s lr: 5.05e-05 [09/23 10:51:03] lb.utils.events INFO: eta: 2:07:05 iteration: 326799/375342 consumed_samples: 334643200 total_loss: 3.002 time: 0.3443 s/iter data_time: 0.2263 s/iter total_throughput: 2974.29 samples/s lr: 5.03e-05 [09/23 10:51:37] lb.utils.events INFO: eta: 2:06:29 iteration: 326899/375342 consumed_samples: 334745600 total_loss: 3.002 time: 0.3443 s/iter data_time: 0.2230 s/iter total_throughput: 2974.28 samples/s lr: 5.01e-05 [09/23 10:52:12] lb.utils.events INFO: eta: 2:06:26 iteration: 326999/375342 consumed_samples: 334848000 total_loss: 3.003 time: 0.3443 s/iter data_time: 0.2219 s/iter total_throughput: 2974.28 samples/s lr: 5.00e-05 [09/23 10:52:47] lb.utils.events INFO: eta: 2:05:39 iteration: 327099/375342 consumed_samples: 334950400 total_loss: 2.992 time: 0.3443 s/iter data_time: 0.2177 s/iter total_throughput: 2974.27 samples/s lr: 4.98e-05 [09/23 10:53:21] lb.utils.events INFO: eta: 2:05:24 iteration: 327199/375342 consumed_samples: 335052800 total_loss: 3.009 time: 0.3443 s/iter data_time: 0.2292 s/iter total_throughput: 2974.27 samples/s lr: 4.96e-05 [09/23 10:53:56] lb.utils.events INFO: eta: 2:05:08 iteration: 327299/375342 consumed_samples: 335155200 total_loss: 3.031 time: 0.3443 s/iter data_time: 0.2243 s/iter total_throughput: 2974.25 samples/s lr: 4.95e-05 [09/23 10:54:31] lb.utils.events INFO: eta: 2:04:49 iteration: 327399/375342 consumed_samples: 335257600 total_loss: 3.01 time: 0.3443 s/iter data_time: 0.2229 s/iter total_throughput: 2974.23 samples/s lr: 4.93e-05 [09/23 10:55:06] lb.utils.events INFO: eta: 2:04:30 iteration: 327499/375342 consumed_samples: 335360000 total_loss: 2.995 time: 0.3443 s/iter data_time: 0.2103 s/iter total_throughput: 2974.23 samples/s lr: 4.92e-05 [09/23 10:55:40] lb.utils.events INFO: eta: 2:04:10 iteration: 327599/375342 consumed_samples: 335462400 total_loss: 3.017 time: 0.3443 s/iter data_time: 0.2172 s/iter total_throughput: 2974.23 samples/s lr: 4.90e-05 [09/23 10:56:15] lb.utils.events INFO: eta: 2:03:55 iteration: 327699/375342 consumed_samples: 335564800 total_loss: 3.033 time: 0.3443 s/iter data_time: 0.2141 s/iter total_throughput: 2974.23 samples/s lr: 4.88e-05 [09/23 10:56:49] lb.utils.events INFO: eta: 2:03:46 iteration: 327799/375342 consumed_samples: 335667200 total_loss: 3.024 time: 0.3443 s/iter data_time: 0.2115 s/iter total_throughput: 2974.23 samples/s lr: 4.87e-05 [09/23 10:57:24] lb.utils.events INFO: eta: 2:03:34 iteration: 327899/375342 consumed_samples: 335769600 total_loss: 3.023 time: 0.3443 s/iter data_time: 0.2120 s/iter total_throughput: 2974.23 samples/s lr: 4.85e-05 [09/23 10:57:58] lb.utils.events INFO: eta: 2:03:17 iteration: 327999/375342 consumed_samples: 335872000 total_loss: 3.027 time: 0.3443 s/iter data_time: 0.2274 s/iter total_throughput: 2974.22 samples/s lr: 4.84e-05 [09/23 10:58:34] lb.utils.events INFO: eta: 2:03:06 iteration: 328099/375342 consumed_samples: 335974400 total_loss: 3.002 time: 0.3443 s/iter data_time: 0.2226 s/iter total_throughput: 2974.20 samples/s lr: 4.82e-05 [09/23 10:59:08] lb.utils.events INFO: eta: 2:03:08 iteration: 328199/375342 consumed_samples: 336076800 total_loss: 3.009 time: 0.3443 s/iter data_time: 0.2091 s/iter total_throughput: 2974.20 samples/s lr: 4.80e-05 [09/23 10:59:43] lb.utils.events INFO: eta: 2:02:47 iteration: 328299/375342 consumed_samples: 336179200 total_loss: 3.021 time: 0.3443 s/iter data_time: 0.2280 s/iter total_throughput: 2974.19 samples/s lr: 4.79e-05 [09/23 11:00:18] lb.utils.events INFO: eta: 2:02:37 iteration: 328399/375342 consumed_samples: 336281600 total_loss: 3.011 time: 0.3443 s/iter data_time: 0.2225 s/iter total_throughput: 2974.18 samples/s lr: 4.77e-05 [09/23 11:00:52] lb.utils.events INFO: eta: 2:02:30 iteration: 328499/375342 consumed_samples: 336384000 total_loss: 3.004 time: 0.3443 s/iter data_time: 0.2092 s/iter total_throughput: 2974.18 samples/s lr: 4.76e-05 [09/23 11:01:26] lb.utils.events INFO: eta: 2:02:00 iteration: 328599/375342 consumed_samples: 336486400 total_loss: 3.009 time: 0.3443 s/iter data_time: 0.2115 s/iter total_throughput: 2974.18 samples/s lr: 4.74e-05 [09/23 11:02:01] lb.utils.events INFO: eta: 2:01:42 iteration: 328699/375342 consumed_samples: 336588800 total_loss: 3.025 time: 0.3443 s/iter data_time: 0.2126 s/iter total_throughput: 2974.18 samples/s lr: 4.72e-05 [09/23 11:02:35] lb.utils.events INFO: eta: 2:01:22 iteration: 328799/375342 consumed_samples: 336691200 total_loss: 3.004 time: 0.3443 s/iter data_time: 0.2181 s/iter total_throughput: 2974.18 samples/s lr: 4.71e-05 [09/23 11:03:09] lb.utils.events INFO: eta: 2:01:17 iteration: 328899/375342 consumed_samples: 336793600 total_loss: 2.998 time: 0.3443 s/iter data_time: 0.2038 s/iter total_throughput: 2974.19 samples/s lr: 4.69e-05 [09/23 11:03:44] lb.utils.events INFO: eta: 2:01:11 iteration: 328999/375342 consumed_samples: 336896000 total_loss: 3.014 time: 0.3443 s/iter data_time: 0.2239 s/iter total_throughput: 2974.19 samples/s lr: 4.68e-05 [09/23 11:04:18] lb.utils.events INFO: eta: 2:01:14 iteration: 329099/375342 consumed_samples: 336998400 total_loss: 3.008 time: 0.3443 s/iter data_time: 0.2321 s/iter total_throughput: 2974.19 samples/s lr: 4.66e-05 [09/23 11:04:53] lb.utils.events INFO: eta: 2:00:44 iteration: 329199/375342 consumed_samples: 337100800 total_loss: 3 time: 0.3443 s/iter data_time: 0.2091 s/iter total_throughput: 2974.18 samples/s lr: 4.65e-05 [09/23 11:05:27] lb.utils.events INFO: eta: 2:00:43 iteration: 329299/375342 consumed_samples: 337203200 total_loss: 2.993 time: 0.3443 s/iter data_time: 0.2229 s/iter total_throughput: 2974.19 samples/s lr: 4.63e-05 [09/23 11:06:02] lb.utils.events INFO: eta: 2:00:31 iteration: 329399/375342 consumed_samples: 337305600 total_loss: 3.008 time: 0.3443 s/iter data_time: 0.2151 s/iter total_throughput: 2974.19 samples/s lr: 4.61e-05 [09/23 11:06:36] lb.utils.events INFO: eta: 2:00:03 iteration: 329499/375342 consumed_samples: 337408000 total_loss: 3.01 time: 0.3443 s/iter data_time: 0.2210 s/iter total_throughput: 2974.18 samples/s lr: 4.60e-05 [09/23 11:07:11] lb.utils.events INFO: eta: 1:59:53 iteration: 329599/375342 consumed_samples: 337510400 total_loss: 3.009 time: 0.3443 s/iter data_time: 0.2170 s/iter total_throughput: 2974.18 samples/s lr: 4.58e-05 [09/23 11:07:46] lb.utils.events INFO: eta: 1:59:52 iteration: 329699/375342 consumed_samples: 337612800 total_loss: 2.984 time: 0.3443 s/iter data_time: 0.2125 s/iter total_throughput: 2974.17 samples/s lr: 4.57e-05 [09/23 11:08:20] lb.utils.events INFO: eta: 1:59:35 iteration: 329799/375342 consumed_samples: 337715200 total_loss: 2.991 time: 0.3443 s/iter data_time: 0.2200 s/iter total_throughput: 2974.17 samples/s lr: 4.55e-05 [09/23 11:08:54] lb.utils.events INFO: eta: 1:59:09 iteration: 329899/375342 consumed_samples: 337817600 total_loss: 3.005 time: 0.3443 s/iter data_time: 0.2152 s/iter total_throughput: 2974.18 samples/s lr: 4.54e-05 [09/23 11:09:29] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0329999 [09/23 11:09:29] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/23 11:09:29] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/23 11:09:34] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0922 s/iter. Inference: 0.1673 s/iter. Eval: 0.0023 s/iter. Total: 0.2619 s/iter. ETA=0:00:09 [09/23 11:09:39] lb.evaluation.evaluator INFO: Inference done 28672/50000. Dataloading: 0.0950 s/iter. Inference: 0.1894 s/iter. Eval: 0.0021 s/iter. Total: 0.2866 s/iter. ETA=0:00:05 [09/23 11:09:44] lb.evaluation.evaluator INFO: Inference done 46080/50000. Dataloading: 0.0976 s/iter. Inference: 0.1936 s/iter. Eval: 0.0021 s/iter. Total: 0.2934 s/iter. ETA=0:00:00 [09/23 11:09:45] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/23 11:09:45] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.655811 (0.000253 s / iter per device, on 8 devices) [09/23 11:09:45] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:08 (0.000171 s / iter per device, on 8 devices) [09/23 11:09:45] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/23 11:09:45] lb.evaluation.utils INFO: copypaste: Acc@1=79.744 [09/23 11:09:45] lb.evaluation.utils INFO: copypaste: Acc@5=94.50800000000001 [09/23 11:09:45] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 79.74400, better than last best score 79.67400 @ iteration 324999. [09/23 11:09:45] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/23 11:09:45] lb.utils.events INFO: eta: 1:58:49 iteration: 329999/375342 consumed_samples: 337920000 total_loss: 3.004 time: 0.3443 s/iter data_time: 0.2127 s/iter total_throughput: 2974.18 samples/s lr: 4.52e-05 [09/23 11:10:18] lb.utils.events INFO: eta: 1:59:05 iteration: 330099/375342 consumed_samples: 338022400 total_loss: 3.004 time: 0.3443 s/iter data_time: 0.2249 s/iter total_throughput: 2974.22 samples/s lr: 4.51e-05 [09/23 11:10:52] lb.utils.events INFO: eta: 1:58:48 iteration: 330199/375342 consumed_samples: 338124800 total_loss: 3.004 time: 0.3443 s/iter data_time: 0.2159 s/iter total_throughput: 2974.23 samples/s lr: 4.49e-05 [09/23 11:11:27] lb.utils.events INFO: eta: 1:58:13 iteration: 330299/375342 consumed_samples: 338227200 total_loss: 3.02 time: 0.3443 s/iter data_time: 0.2081 s/iter total_throughput: 2974.22 samples/s lr: 4.48e-05 [09/23 11:12:01] lb.utils.events INFO: eta: 1:58:11 iteration: 330399/375342 consumed_samples: 338329600 total_loss: 3.02 time: 0.3443 s/iter data_time: 0.2200 s/iter total_throughput: 2974.23 samples/s lr: 4.46e-05 [09/23 11:12:36] lb.utils.events INFO: eta: 1:57:51 iteration: 330499/375342 consumed_samples: 338432000 total_loss: 3.001 time: 0.3443 s/iter data_time: 0.2234 s/iter total_throughput: 2974.22 samples/s lr: 4.45e-05 [09/23 11:13:10] lb.utils.events INFO: eta: 1:57:48 iteration: 330599/375342 consumed_samples: 338534400 total_loss: 3.002 time: 0.3443 s/iter data_time: 0.2101 s/iter total_throughput: 2974.22 samples/s lr: 4.43e-05 [09/23 11:13:45] lb.utils.events INFO: eta: 1:57:17 iteration: 330699/375342 consumed_samples: 338636800 total_loss: 3.004 time: 0.3443 s/iter data_time: 0.2162 s/iter total_throughput: 2974.23 samples/s lr: 4.42e-05 [09/23 11:14:19] lb.utils.events INFO: eta: 1:57:13 iteration: 330799/375342 consumed_samples: 338739200 total_loss: 3.004 time: 0.3443 s/iter data_time: 0.2194 s/iter total_throughput: 2974.22 samples/s lr: 4.40e-05 [09/23 11:14:54] lb.utils.events INFO: eta: 1:56:42 iteration: 330899/375342 consumed_samples: 338841600 total_loss: 3.008 time: 0.3443 s/iter data_time: 0.2274 s/iter total_throughput: 2974.21 samples/s lr: 4.39e-05 [09/23 11:15:28] lb.utils.events INFO: eta: 1:56:22 iteration: 330999/375342 consumed_samples: 338944000 total_loss: 3.012 time: 0.3443 s/iter data_time: 0.2251 s/iter total_throughput: 2974.21 samples/s lr: 4.37e-05 [09/23 11:16:03] lb.utils.events INFO: eta: 1:55:43 iteration: 331099/375342 consumed_samples: 339046400 total_loss: 3.011 time: 0.3443 s/iter data_time: 0.2232 s/iter total_throughput: 2974.22 samples/s lr: 4.36e-05 [09/23 11:16:37] lb.utils.events INFO: eta: 1:55:27 iteration: 331199/375342 consumed_samples: 339148800 total_loss: 3.017 time: 0.3443 s/iter data_time: 0.2081 s/iter total_throughput: 2974.22 samples/s lr: 4.34e-05 [09/23 11:17:12] lb.utils.events INFO: eta: 1:55:24 iteration: 331299/375342 consumed_samples: 339251200 total_loss: 3 time: 0.3443 s/iter data_time: 0.2062 s/iter total_throughput: 2974.22 samples/s lr: 4.33e-05 [09/23 11:17:46] lb.utils.events INFO: eta: 1:55:06 iteration: 331399/375342 consumed_samples: 339353600 total_loss: 2.984 time: 0.3443 s/iter data_time: 0.2174 s/iter total_throughput: 2974.21 samples/s lr: 4.31e-05 [09/23 11:18:20] lb.utils.events INFO: eta: 1:54:55 iteration: 331499/375342 consumed_samples: 339456000 total_loss: 2.994 time: 0.3443 s/iter data_time: 0.2204 s/iter total_throughput: 2974.23 samples/s lr: 4.30e-05 [09/23 11:18:55] lb.utils.events INFO: eta: 1:54:26 iteration: 331599/375342 consumed_samples: 339558400 total_loss: 2.997 time: 0.3443 s/iter data_time: 0.2164 s/iter total_throughput: 2974.22 samples/s lr: 4.28e-05 [09/23 11:19:30] lb.utils.events INFO: eta: 1:54:01 iteration: 331699/375342 consumed_samples: 339660800 total_loss: 2.993 time: 0.3443 s/iter data_time: 0.2200 s/iter total_throughput: 2974.21 samples/s lr: 4.27e-05 [09/23 11:20:04] lb.utils.events INFO: eta: 1:53:26 iteration: 331799/375342 consumed_samples: 339763200 total_loss: 2.996 time: 0.3443 s/iter data_time: 0.2292 s/iter total_throughput: 2974.21 samples/s lr: 4.25e-05 [09/23 11:20:39] lb.utils.events INFO: eta: 1:53:29 iteration: 331899/375342 consumed_samples: 339865600 total_loss: 2.991 time: 0.3443 s/iter data_time: 0.2226 s/iter total_throughput: 2974.21 samples/s lr: 4.24e-05 [09/23 11:21:13] lb.utils.events INFO: eta: 1:53:15 iteration: 331999/375342 consumed_samples: 339968000 total_loss: 2.997 time: 0.3443 s/iter data_time: 0.2241 s/iter total_throughput: 2974.21 samples/s lr: 4.22e-05 [09/23 11:21:47] lb.utils.events INFO: eta: 1:52:55 iteration: 332099/375342 consumed_samples: 340070400 total_loss: 3.013 time: 0.3443 s/iter data_time: 0.2166 s/iter total_throughput: 2974.21 samples/s lr: 4.21e-05 [09/23 11:22:22] lb.utils.events INFO: eta: 1:52:41 iteration: 332199/375342 consumed_samples: 340172800 total_loss: 3.005 time: 0.3443 s/iter data_time: 0.2171 s/iter total_throughput: 2974.21 samples/s lr: 4.19e-05 [09/23 11:22:56] lb.utils.events INFO: eta: 1:52:14 iteration: 332299/375342 consumed_samples: 340275200 total_loss: 3.002 time: 0.3443 s/iter data_time: 0.2179 s/iter total_throughput: 2974.22 samples/s lr: 4.18e-05 [09/23 11:23:31] lb.utils.events INFO: eta: 1:51:45 iteration: 332399/375342 consumed_samples: 340377600 total_loss: 3.002 time: 0.3443 s/iter data_time: 0.2125 s/iter total_throughput: 2974.22 samples/s lr: 4.16e-05 [09/23 11:24:04] lb.utils.events INFO: eta: 1:51:37 iteration: 332499/375342 consumed_samples: 340480000 total_loss: 3.006 time: 0.3443 s/iter data_time: 0.2040 s/iter total_throughput: 2974.23 samples/s lr: 4.15e-05 [09/23 11:24:39] lb.utils.events INFO: eta: 1:51:17 iteration: 332599/375342 consumed_samples: 340582400 total_loss: 3.007 time: 0.3443 s/iter data_time: 0.2128 s/iter total_throughput: 2974.23 samples/s lr: 4.13e-05 [09/23 11:25:13] lb.utils.events INFO: eta: 1:51:07 iteration: 332699/375342 consumed_samples: 340684800 total_loss: 3.002 time: 0.3443 s/iter data_time: 0.2136 s/iter total_throughput: 2974.23 samples/s lr: 4.12e-05 [09/23 11:25:48] lb.utils.events INFO: eta: 1:51:08 iteration: 332799/375342 consumed_samples: 340787200 total_loss: 2.977 time: 0.3443 s/iter data_time: 0.2229 s/iter total_throughput: 2974.23 samples/s lr: 4.11e-05 [09/23 11:26:22] lb.utils.events INFO: eta: 1:50:33 iteration: 332899/375342 consumed_samples: 340889600 total_loss: 2.978 time: 0.3443 s/iter data_time: 0.2154 s/iter total_throughput: 2974.23 samples/s lr: 4.09e-05 [09/23 11:26:57] lb.utils.events INFO: eta: 1:50:09 iteration: 332999/375342 consumed_samples: 340992000 total_loss: 2.992 time: 0.3443 s/iter data_time: 0.2189 s/iter total_throughput: 2974.22 samples/s lr: 4.08e-05 [09/23 11:27:32] lb.utils.events INFO: eta: 1:49:56 iteration: 333099/375342 consumed_samples: 341094400 total_loss: 2.985 time: 0.3443 s/iter data_time: 0.2118 s/iter total_throughput: 2974.22 samples/s lr: 4.06e-05 [09/23 11:28:06] lb.utils.events INFO: eta: 1:49:36 iteration: 333199/375342 consumed_samples: 341196800 total_loss: 2.986 time: 0.3443 s/iter data_time: 0.2231 s/iter total_throughput: 2974.22 samples/s lr: 4.05e-05 [09/23 11:28:40] lb.utils.events INFO: eta: 1:49:22 iteration: 333299/375342 consumed_samples: 341299200 total_loss: 2.989 time: 0.3443 s/iter data_time: 0.2197 s/iter total_throughput: 2974.22 samples/s lr: 4.03e-05 [09/23 11:29:15] lb.utils.events INFO: eta: 1:49:40 iteration: 333399/375342 consumed_samples: 341401600 total_loss: 2.992 time: 0.3443 s/iter data_time: 0.2221 s/iter total_throughput: 2974.22 samples/s lr: 4.02e-05 [09/23 11:29:49] lb.utils.events INFO: eta: 1:49:19 iteration: 333499/375342 consumed_samples: 341504000 total_loss: 2.989 time: 0.3443 s/iter data_time: 0.2059 s/iter total_throughput: 2974.23 samples/s lr: 4.00e-05 [09/23 11:30:23] lb.utils.events INFO: eta: 1:49:22 iteration: 333599/375342 consumed_samples: 341606400 total_loss: 2.988 time: 0.3443 s/iter data_time: 0.2077 s/iter total_throughput: 2974.24 samples/s lr: 3.99e-05 [09/23 11:30:57] lb.utils.events INFO: eta: 1:49:08 iteration: 333699/375342 consumed_samples: 341708800 total_loss: 3 time: 0.3443 s/iter data_time: 0.2119 s/iter total_throughput: 2974.24 samples/s lr: 3.98e-05 [09/23 11:31:31] lb.utils.events INFO: eta: 1:49:06 iteration: 333799/375342 consumed_samples: 341811200 total_loss: 3 time: 0.3443 s/iter data_time: 0.2182 s/iter total_throughput: 2974.25 samples/s lr: 3.96e-05 [09/23 11:32:06] lb.utils.events INFO: eta: 1:48:45 iteration: 333899/375342 consumed_samples: 341913600 total_loss: 2.992 time: 0.3443 s/iter data_time: 0.2088 s/iter total_throughput: 2974.24 samples/s lr: 3.95e-05 [09/23 11:32:41] lb.utils.events INFO: eta: 1:48:42 iteration: 333999/375342 consumed_samples: 342016000 total_loss: 2.993 time: 0.3443 s/iter data_time: 0.2157 s/iter total_throughput: 2974.24 samples/s lr: 3.93e-05 [09/23 11:33:15] lb.utils.events INFO: eta: 1:48:20 iteration: 334099/375342 consumed_samples: 342118400 total_loss: 3 time: 0.3443 s/iter data_time: 0.2148 s/iter total_throughput: 2974.24 samples/s lr: 3.92e-05 [09/23 11:33:49] lb.utils.events INFO: eta: 1:48:02 iteration: 334199/375342 consumed_samples: 342220800 total_loss: 2.988 time: 0.3443 s/iter data_time: 0.2153 s/iter total_throughput: 2974.24 samples/s lr: 3.91e-05 [09/23 11:34:24] lb.utils.events INFO: eta: 1:47:43 iteration: 334299/375342 consumed_samples: 342323200 total_loss: 2.982 time: 0.3443 s/iter data_time: 0.2156 s/iter total_throughput: 2974.24 samples/s lr: 3.89e-05 [09/23 11:34:58] lb.utils.events INFO: eta: 1:46:59 iteration: 334399/375342 consumed_samples: 342425600 total_loss: 2.982 time: 0.3443 s/iter data_time: 0.2216 s/iter total_throughput: 2974.25 samples/s lr: 3.88e-05 [09/23 11:35:32] lb.utils.events INFO: eta: 1:46:41 iteration: 334499/375342 consumed_samples: 342528000 total_loss: 2.985 time: 0.3443 s/iter data_time: 0.2177 s/iter total_throughput: 2974.27 samples/s lr: 3.86e-05 [09/23 11:36:06] lb.utils.events INFO: eta: 1:46:20 iteration: 334599/375342 consumed_samples: 342630400 total_loss: 2.978 time: 0.3443 s/iter data_time: 0.2130 s/iter total_throughput: 2974.27 samples/s lr: 3.85e-05 [09/23 11:36:40] lb.utils.events INFO: eta: 1:45:44 iteration: 334699/375342 consumed_samples: 342732800 total_loss: 2.98 time: 0.3443 s/iter data_time: 0.2073 s/iter total_throughput: 2974.27 samples/s lr: 3.84e-05 [09/23 11:37:15] lb.utils.events INFO: eta: 1:45:09 iteration: 334799/375342 consumed_samples: 342835200 total_loss: 2.996 time: 0.3443 s/iter data_time: 0.2186 s/iter total_throughput: 2974.28 samples/s lr: 3.82e-05 [09/23 11:37:49] lb.utils.events INFO: eta: 1:44:53 iteration: 334899/375342 consumed_samples: 342937600 total_loss: 2.987 time: 0.3443 s/iter data_time: 0.2094 s/iter total_throughput: 2974.28 samples/s lr: 3.81e-05 [09/23 11:38:23] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0334999 [09/23 11:38:24] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/23 11:38:24] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/23 11:38:28] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0975 s/iter. Inference: 0.1644 s/iter. Eval: 0.0023 s/iter. Total: 0.2642 s/iter. ETA=0:00:09 [09/23 11:38:33] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1201 s/iter. Inference: 0.1753 s/iter. Eval: 0.0021 s/iter. Total: 0.2976 s/iter. ETA=0:00:05 [09/23 11:38:39] lb.evaluation.evaluator INFO: Inference done 48128/50000. Dataloading: 0.1204 s/iter. Inference: 0.1690 s/iter. Eval: 0.0021 s/iter. Total: 0.2915 s/iter. ETA=0:00:00 [09/23 11:38:39] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/23 11:38:39] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.595559 (0.000252 s / iter per device, on 8 devices) [09/23 11:38:39] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000149 s / iter per device, on 8 devices) [09/23 11:38:39] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/23 11:38:39] lb.evaluation.utils INFO: copypaste: Acc@1=79.738 [09/23 11:38:39] lb.evaluation.utils INFO: copypaste: Acc@5=94.428 [09/23 11:38:39] lb.engine.hooks INFO: Not saving as latest eval score for Acc@1 is 79.73800, not better than best score 79.74400 @ iteration 329999. [09/23 11:38:39] lb.utils.events INFO: eta: 1:44:29 iteration: 334999/375342 consumed_samples: 343040000 total_loss: 2.982 time: 0.3443 s/iter data_time: 0.2177 s/iter total_throughput: 2974.29 samples/s lr: 3.80e-05 [09/23 11:39:11] lb.utils.events INFO: eta: 1:44:15 iteration: 335099/375342 consumed_samples: 343142400 total_loss: 2.999 time: 0.3443 s/iter data_time: 0.2422 s/iter total_throughput: 2974.36 samples/s lr: 3.78e-05 [09/23 11:39:45] lb.utils.events INFO: eta: 1:44:02 iteration: 335199/375342 consumed_samples: 343244800 total_loss: 3.005 time: 0.3443 s/iter data_time: 0.2095 s/iter total_throughput: 2974.37 samples/s lr: 3.77e-05 [09/23 11:40:18] lb.utils.events INFO: eta: 1:44:01 iteration: 335299/375342 consumed_samples: 343347200 total_loss: 2.984 time: 0.3443 s/iter data_time: 0.2108 s/iter total_throughput: 2974.38 samples/s lr: 3.75e-05 [09/23 11:40:53] lb.utils.events INFO: eta: 1:43:56 iteration: 335399/375342 consumed_samples: 343449600 total_loss: 2.963 time: 0.3443 s/iter data_time: 0.2126 s/iter total_throughput: 2974.38 samples/s lr: 3.74e-05 [09/23 11:41:27] lb.utils.events INFO: eta: 1:43:34 iteration: 335499/375342 consumed_samples: 343552000 total_loss: 2.965 time: 0.3443 s/iter data_time: 0.2164 s/iter total_throughput: 2974.39 samples/s lr: 3.73e-05 [09/23 11:42:01] lb.utils.events INFO: eta: 1:43:06 iteration: 335599/375342 consumed_samples: 343654400 total_loss: 2.957 time: 0.3443 s/iter data_time: 0.2107 s/iter total_throughput: 2974.40 samples/s lr: 3.71e-05 [09/23 11:42:35] lb.utils.events INFO: eta: 1:43:02 iteration: 335699/375342 consumed_samples: 343756800 total_loss: 2.96 time: 0.3443 s/iter data_time: 0.2098 s/iter total_throughput: 2974.42 samples/s lr: 3.70e-05 [09/23 11:43:09] lb.utils.events INFO: eta: 1:42:41 iteration: 335799/375342 consumed_samples: 343859200 total_loss: 2.98 time: 0.3443 s/iter data_time: 0.2088 s/iter total_throughput: 2974.43 samples/s lr: 3.69e-05 [09/23 11:43:42] lb.utils.events INFO: eta: 1:42:37 iteration: 335899/375342 consumed_samples: 343961600 total_loss: 2.985 time: 0.3443 s/iter data_time: 0.2071 s/iter total_throughput: 2974.45 samples/s lr: 3.67e-05 [09/23 11:44:17] lb.utils.events INFO: eta: 1:42:29 iteration: 335999/375342 consumed_samples: 344064000 total_loss: 2.977 time: 0.3443 s/iter data_time: 0.2118 s/iter total_throughput: 2974.46 samples/s lr: 3.66e-05 [09/23 11:44:51] lb.utils.events INFO: eta: 1:42:02 iteration: 336099/375342 consumed_samples: 344166400 total_loss: 3.001 time: 0.3443 s/iter data_time: 0.2212 s/iter total_throughput: 2974.46 samples/s lr: 3.65e-05 [09/23 11:45:26] lb.utils.events INFO: eta: 1:41:38 iteration: 336199/375342 consumed_samples: 344268800 total_loss: 3.005 time: 0.3443 s/iter data_time: 0.2205 s/iter total_throughput: 2974.44 samples/s lr: 3.63e-05 [09/23 11:46:01] lb.utils.events INFO: eta: 1:41:18 iteration: 336299/375342 consumed_samples: 344371200 total_loss: 2.993 time: 0.3443 s/iter data_time: 0.2232 s/iter total_throughput: 2974.44 samples/s lr: 3.62e-05 [09/23 11:46:36] lb.utils.events INFO: eta: 1:41:07 iteration: 336399/375342 consumed_samples: 344473600 total_loss: 2.994 time: 0.3443 s/iter data_time: 0.2245 s/iter total_throughput: 2974.43 samples/s lr: 3.61e-05 [09/23 11:47:10] lb.utils.events INFO: eta: 1:40:51 iteration: 336499/375342 consumed_samples: 344576000 total_loss: 3.002 time: 0.3443 s/iter data_time: 0.2251 s/iter total_throughput: 2974.42 samples/s lr: 3.59e-05 [09/23 11:47:46] lb.utils.events INFO: eta: 1:40:40 iteration: 336599/375342 consumed_samples: 344678400 total_loss: 2.986 time: 0.3443 s/iter data_time: 0.2312 s/iter total_throughput: 2974.39 samples/s lr: 3.58e-05 [09/23 11:48:21] lb.utils.events INFO: eta: 1:40:22 iteration: 336699/375342 consumed_samples: 344780800 total_loss: 2.986 time: 0.3443 s/iter data_time: 0.2354 s/iter total_throughput: 2974.37 samples/s lr: 3.57e-05 [09/23 11:48:56] lb.utils.events INFO: eta: 1:40:11 iteration: 336799/375342 consumed_samples: 344883200 total_loss: 2.98 time: 0.3443 s/iter data_time: 0.2220 s/iter total_throughput: 2974.36 samples/s lr: 3.55e-05 [09/23 11:49:31] lb.utils.events INFO: eta: 1:39:55 iteration: 336899/375342 consumed_samples: 344985600 total_loss: 2.973 time: 0.3443 s/iter data_time: 0.2110 s/iter total_throughput: 2974.35 samples/s lr: 3.54e-05 [09/23 11:50:06] lb.utils.events INFO: eta: 1:39:35 iteration: 336999/375342 consumed_samples: 345088000 total_loss: 2.973 time: 0.3443 s/iter data_time: 0.2375 s/iter total_throughput: 2974.33 samples/s lr: 3.53e-05 [09/23 11:50:41] lb.utils.events INFO: eta: 1:39:20 iteration: 337099/375342 consumed_samples: 345190400 total_loss: 2.967 time: 0.3443 s/iter data_time: 0.2267 s/iter total_throughput: 2974.31 samples/s lr: 3.51e-05 [09/23 11:51:16] lb.utils.events INFO: eta: 1:39:09 iteration: 337199/375342 consumed_samples: 345292800 total_loss: 2.97 time: 0.3443 s/iter data_time: 0.2169 s/iter total_throughput: 2974.31 samples/s lr: 3.50e-05 [09/23 11:51:50] lb.utils.events INFO: eta: 1:38:59 iteration: 337299/375342 consumed_samples: 345395200 total_loss: 2.977 time: 0.3443 s/iter data_time: 0.2291 s/iter total_throughput: 2974.31 samples/s lr: 3.49e-05 [09/23 11:52:25] lb.utils.events INFO: eta: 1:38:40 iteration: 337399/375342 consumed_samples: 345497600 total_loss: 2.975 time: 0.3443 s/iter data_time: 0.2185 s/iter total_throughput: 2974.30 samples/s lr: 3.48e-05 [09/23 11:53:00] lb.utils.events INFO: eta: 1:38:26 iteration: 337499/375342 consumed_samples: 345600000 total_loss: 2.985 time: 0.3443 s/iter data_time: 0.2270 s/iter total_throughput: 2974.29 samples/s lr: 3.46e-05 [09/23 11:53:35] lb.utils.events INFO: eta: 1:38:09 iteration: 337599/375342 consumed_samples: 345702400 total_loss: 2.989 time: 0.3443 s/iter data_time: 0.2210 s/iter total_throughput: 2974.26 samples/s lr: 3.45e-05 [09/23 11:54:11] lb.utils.events INFO: eta: 1:37:52 iteration: 337699/375342 consumed_samples: 345804800 total_loss: 2.983 time: 0.3443 s/iter data_time: 0.2215 s/iter total_throughput: 2974.24 samples/s lr: 3.44e-05 [09/23 11:54:45] lb.utils.events INFO: eta: 1:37:44 iteration: 337799/375342 consumed_samples: 345907200 total_loss: 2.973 time: 0.3443 s/iter data_time: 0.2127 s/iter total_throughput: 2974.24 samples/s lr: 3.42e-05 [09/23 11:55:20] lb.utils.events INFO: eta: 1:37:27 iteration: 337899/375342 consumed_samples: 346009600 total_loss: 2.988 time: 0.3443 s/iter data_time: 0.2198 s/iter total_throughput: 2974.24 samples/s lr: 3.41e-05 [09/23 11:55:54] lb.utils.events INFO: eta: 1:37:17 iteration: 337999/375342 consumed_samples: 346112000 total_loss: 2.979 time: 0.3443 s/iter data_time: 0.2240 s/iter total_throughput: 2974.23 samples/s lr: 3.40e-05 [09/23 11:56:29] lb.utils.events INFO: eta: 1:37:14 iteration: 338099/375342 consumed_samples: 346214400 total_loss: 2.952 time: 0.3443 s/iter data_time: 0.2162 s/iter total_throughput: 2974.23 samples/s lr: 3.39e-05 [09/23 11:57:03] lb.utils.events INFO: eta: 1:36:55 iteration: 338199/375342 consumed_samples: 346316800 total_loss: 2.967 time: 0.3443 s/iter data_time: 0.2263 s/iter total_throughput: 2974.22 samples/s lr: 3.37e-05 [09/23 11:57:38] lb.utils.events INFO: eta: 1:36:26 iteration: 338299/375342 consumed_samples: 346419200 total_loss: 2.979 time: 0.3443 s/iter data_time: 0.2174 s/iter total_throughput: 2974.22 samples/s lr: 3.36e-05 [09/23 11:58:12] lb.utils.events INFO: eta: 1:36:13 iteration: 338399/375342 consumed_samples: 346521600 total_loss: 2.965 time: 0.3443 s/iter data_time: 0.2133 s/iter total_throughput: 2974.22 samples/s lr: 3.35e-05 [09/23 11:58:47] lb.utils.events INFO: eta: 1:36:03 iteration: 338499/375342 consumed_samples: 346624000 total_loss: 2.967 time: 0.3443 s/iter data_time: 0.2179 s/iter total_throughput: 2974.22 samples/s lr: 3.33e-05 [09/23 11:59:22] lb.utils.events INFO: eta: 1:35:48 iteration: 338599/375342 consumed_samples: 346726400 total_loss: 2.973 time: 0.3443 s/iter data_time: 0.2239 s/iter total_throughput: 2974.22 samples/s lr: 3.32e-05 [09/23 11:59:56] lb.utils.events INFO: eta: 1:35:34 iteration: 338699/375342 consumed_samples: 346828800 total_loss: 2.975 time: 0.3443 s/iter data_time: 0.2128 s/iter total_throughput: 2974.20 samples/s lr: 3.31e-05 [09/23 12:00:31] lb.utils.events INFO: eta: 1:34:55 iteration: 338799/375342 consumed_samples: 346931200 total_loss: 2.977 time: 0.3443 s/iter data_time: 0.2289 s/iter total_throughput: 2974.19 samples/s lr: 3.30e-05 [09/23 12:01:06] lb.utils.events INFO: eta: 1:34:39 iteration: 338899/375342 consumed_samples: 347033600 total_loss: 2.979 time: 0.3443 s/iter data_time: 0.2142 s/iter total_throughput: 2974.19 samples/s lr: 3.28e-05 [09/23 12:01:41] lb.utils.events INFO: eta: 1:34:24 iteration: 338999/375342 consumed_samples: 347136000 total_loss: 2.985 time: 0.3443 s/iter data_time: 0.2297 s/iter total_throughput: 2974.18 samples/s lr: 3.27e-05 [09/23 12:02:16] lb.utils.events INFO: eta: 1:33:55 iteration: 339099/375342 consumed_samples: 347238400 total_loss: 2.98 time: 0.3443 s/iter data_time: 0.2254 s/iter total_throughput: 2974.16 samples/s lr: 3.26e-05 [09/23 12:02:51] lb.utils.events INFO: eta: 1:33:39 iteration: 339199/375342 consumed_samples: 347340800 total_loss: 2.969 time: 0.3443 s/iter data_time: 0.2242 s/iter total_throughput: 2974.14 samples/s lr: 3.25e-05 [09/23 12:03:26] lb.utils.events INFO: eta: 1:33:17 iteration: 339299/375342 consumed_samples: 347443200 total_loss: 2.958 time: 0.3443 s/iter data_time: 0.2283 s/iter total_throughput: 2974.13 samples/s lr: 3.24e-05 [09/23 12:04:01] lb.utils.events INFO: eta: 1:33:11 iteration: 339399/375342 consumed_samples: 347545600 total_loss: 2.984 time: 0.3443 s/iter data_time: 0.2196 s/iter total_throughput: 2974.11 samples/s lr: 3.22e-05 [09/23 12:04:36] lb.utils.events INFO: eta: 1:32:40 iteration: 339499/375342 consumed_samples: 347648000 total_loss: 2.987 time: 0.3443 s/iter data_time: 0.2121 s/iter total_throughput: 2974.10 samples/s lr: 3.21e-05 [09/23 12:05:11] lb.utils.events INFO: eta: 1:32:24 iteration: 339599/375342 consumed_samples: 347750400 total_loss: 2.978 time: 0.3443 s/iter data_time: 0.2189 s/iter total_throughput: 2974.09 samples/s lr: 3.20e-05 [09/23 12:05:46] lb.utils.events INFO: eta: 1:32:09 iteration: 339699/375342 consumed_samples: 347852800 total_loss: 2.975 time: 0.3443 s/iter data_time: 0.2228 s/iter total_throughput: 2974.08 samples/s lr: 3.19e-05 [09/23 12:06:21] lb.utils.events INFO: eta: 1:31:59 iteration: 339799/375342 consumed_samples: 347955200 total_loss: 2.978 time: 0.3443 s/iter data_time: 0.2160 s/iter total_throughput: 2974.06 samples/s lr: 3.17e-05 [09/23 12:06:56] lb.utils.events INFO: eta: 1:31:38 iteration: 339899/375342 consumed_samples: 348057600 total_loss: 2.995 time: 0.3443 s/iter data_time: 0.2258 s/iter total_throughput: 2974.04 samples/s lr: 3.16e-05 [09/23 12:07:30] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0339999 [09/23 12:07:31] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/23 12:07:31] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/23 12:07:35] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.1016 s/iter. Inference: 0.1646 s/iter. Eval: 0.0022 s/iter. Total: 0.2683 s/iter. ETA=0:00:09 [09/23 12:07:40] lb.evaluation.evaluator INFO: Inference done 28672/50000. Dataloading: 0.0979 s/iter. Inference: 0.1878 s/iter. Eval: 0.0021 s/iter. Total: 0.2879 s/iter. ETA=0:00:05 [09/23 12:07:46] lb.evaluation.evaluator INFO: Inference done 47104/50000. Dataloading: 0.1088 s/iter. Inference: 0.1819 s/iter. Eval: 0.0021 s/iter. Total: 0.2929 s/iter. ETA=0:00:00 [09/23 12:07:46] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/23 12:07:46] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.530537 (0.000251 s / iter per device, on 8 devices) [09/23 12:07:46] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000159 s / iter per device, on 8 devices) [09/23 12:07:46] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/23 12:07:46] lb.evaluation.utils INFO: copypaste: Acc@1=79.838 [09/23 12:07:46] lb.evaluation.utils INFO: copypaste: Acc@5=94.42399999999999 [09/23 12:07:46] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 79.83800, better than last best score 79.74400 @ iteration 329999. [09/23 12:07:46] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/23 12:07:47] lb.utils.events INFO: eta: 1:31:28 iteration: 339999/375342 consumed_samples: 348160000 total_loss: 3 time: 0.3443 s/iter data_time: 0.2158 s/iter total_throughput: 2974.05 samples/s lr: 3.15e-05 [09/23 12:08:19] lb.utils.events INFO: eta: 1:31:57 iteration: 340099/375342 consumed_samples: 348262400 total_loss: 2.979 time: 0.3443 s/iter data_time: 0.2080 s/iter total_throughput: 2974.10 samples/s lr: 3.14e-05 [09/23 12:08:54] lb.utils.events INFO: eta: 1:31:42 iteration: 340199/375342 consumed_samples: 348364800 total_loss: 2.982 time: 0.3443 s/iter data_time: 0.2229 s/iter total_throughput: 2974.08 samples/s lr: 3.13e-05 [09/23 12:09:29] lb.utils.events INFO: eta: 1:31:34 iteration: 340299/375342 consumed_samples: 348467200 total_loss: 2.969 time: 0.3443 s/iter data_time: 0.2276 s/iter total_throughput: 2974.07 samples/s lr: 3.11e-05 [09/23 12:10:04] lb.utils.events INFO: eta: 1:31:11 iteration: 340399/375342 consumed_samples: 348569600 total_loss: 2.956 time: 0.3443 s/iter data_time: 0.2225 s/iter total_throughput: 2974.05 samples/s lr: 3.10e-05 [09/23 12:10:39] lb.utils.events INFO: eta: 1:30:54 iteration: 340499/375342 consumed_samples: 348672000 total_loss: 2.951 time: 0.3443 s/iter data_time: 0.2195 s/iter total_throughput: 2974.05 samples/s lr: 3.09e-05 [09/23 12:11:13] lb.utils.events INFO: eta: 1:30:53 iteration: 340599/375342 consumed_samples: 348774400 total_loss: 2.96 time: 0.3443 s/iter data_time: 0.2172 s/iter total_throughput: 2974.05 samples/s lr: 3.08e-05 [09/23 12:11:48] lb.utils.events INFO: eta: 1:30:42 iteration: 340699/375342 consumed_samples: 348876800 total_loss: 2.974 time: 0.3443 s/iter data_time: 0.2117 s/iter total_throughput: 2974.05 samples/s lr: 3.07e-05 [09/23 12:12:22] lb.utils.events INFO: eta: 1:30:50 iteration: 340799/375342 consumed_samples: 348979200 total_loss: 2.973 time: 0.3443 s/iter data_time: 0.2165 s/iter total_throughput: 2974.05 samples/s lr: 3.05e-05 [09/23 12:12:57] lb.utils.events INFO: eta: 1:30:33 iteration: 340899/375342 consumed_samples: 349081600 total_loss: 2.985 time: 0.3443 s/iter data_time: 0.2166 s/iter total_throughput: 2974.03 samples/s lr: 3.04e-05 [09/23 12:13:32] lb.utils.events INFO: eta: 1:30:14 iteration: 340999/375342 consumed_samples: 349184000 total_loss: 2.991 time: 0.3443 s/iter data_time: 0.2056 s/iter total_throughput: 2974.03 samples/s lr: 3.03e-05 [09/23 12:14:06] lb.utils.events INFO: eta: 1:29:29 iteration: 341099/375342 consumed_samples: 349286400 total_loss: 2.979 time: 0.3443 s/iter data_time: 0.2090 s/iter total_throughput: 2974.03 samples/s lr: 3.02e-05 [09/23 12:14:41] lb.utils.events INFO: eta: 1:29:13 iteration: 341199/375342 consumed_samples: 349388800 total_loss: 2.953 time: 0.3443 s/iter data_time: 0.2073 s/iter total_throughput: 2974.03 samples/s lr: 3.01e-05 [09/23 12:15:16] lb.utils.events INFO: eta: 1:28:43 iteration: 341299/375342 consumed_samples: 349491200 total_loss: 2.955 time: 0.3443 s/iter data_time: 0.2230 s/iter total_throughput: 2974.01 samples/s lr: 3.00e-05 [09/23 12:15:51] lb.utils.events INFO: eta: 1:28:23 iteration: 341399/375342 consumed_samples: 349593600 total_loss: 2.983 time: 0.3443 s/iter data_time: 0.2219 s/iter total_throughput: 2974.01 samples/s lr: 2.98e-05 [09/23 12:16:25] lb.utils.events INFO: eta: 1:28:14 iteration: 341499/375342 consumed_samples: 349696000 total_loss: 2.989 time: 0.3443 s/iter data_time: 0.2200 s/iter total_throughput: 2974.01 samples/s lr: 2.97e-05 [09/23 12:17:00] lb.utils.events INFO: eta: 1:27:45 iteration: 341599/375342 consumed_samples: 349798400 total_loss: 2.957 time: 0.3443 s/iter data_time: 0.2254 s/iter total_throughput: 2974.00 samples/s lr: 2.96e-05 [09/23 12:17:34] lb.utils.events INFO: eta: 1:27:43 iteration: 341699/375342 consumed_samples: 349900800 total_loss: 2.954 time: 0.3443 s/iter data_time: 0.2168 s/iter total_throughput: 2974.00 samples/s lr: 2.95e-05 [09/23 12:18:09] lb.utils.events INFO: eta: 1:27:13 iteration: 341799/375342 consumed_samples: 350003200 total_loss: 2.97 time: 0.3443 s/iter data_time: 0.2210 s/iter total_throughput: 2973.99 samples/s lr: 2.94e-05 [09/23 12:18:44] lb.utils.events INFO: eta: 1:26:57 iteration: 341899/375342 consumed_samples: 350105600 total_loss: 2.968 time: 0.3443 s/iter data_time: 0.2209 s/iter total_throughput: 2973.98 samples/s lr: 2.93e-05 [09/23 12:19:18] lb.utils.events INFO: eta: 1:26:42 iteration: 341999/375342 consumed_samples: 350208000 total_loss: 2.958 time: 0.3443 s/iter data_time: 0.2216 s/iter total_throughput: 2973.98 samples/s lr: 2.92e-05 [09/23 12:19:53] lb.utils.events INFO: eta: 1:26:14 iteration: 342099/375342 consumed_samples: 350310400 total_loss: 2.966 time: 0.3443 s/iter data_time: 0.2150 s/iter total_throughput: 2973.96 samples/s lr: 2.90e-05 [09/23 12:20:28] lb.utils.events INFO: eta: 1:26:00 iteration: 342199/375342 consumed_samples: 350412800 total_loss: 2.977 time: 0.3443 s/iter data_time: 0.2234 s/iter total_throughput: 2973.96 samples/s lr: 2.89e-05 [09/23 12:21:03] lb.utils.events INFO: eta: 1:25:44 iteration: 342299/375342 consumed_samples: 350515200 total_loss: 2.979 time: 0.3443 s/iter data_time: 0.2214 s/iter total_throughput: 2973.95 samples/s lr: 2.88e-05 [09/23 12:21:38] lb.utils.events INFO: eta: 1:25:24 iteration: 342399/375342 consumed_samples: 350617600 total_loss: 2.972 time: 0.3443 s/iter data_time: 0.2191 s/iter total_throughput: 2973.94 samples/s lr: 2.87e-05 [09/23 12:22:12] lb.utils.events INFO: eta: 1:25:09 iteration: 342499/375342 consumed_samples: 350720000 total_loss: 2.995 time: 0.3443 s/iter data_time: 0.2190 s/iter total_throughput: 2973.94 samples/s lr: 2.86e-05 [09/23 12:22:47] lb.utils.events INFO: eta: 1:24:44 iteration: 342599/375342 consumed_samples: 350822400 total_loss: 2.992 time: 0.3443 s/iter data_time: 0.2200 s/iter total_throughput: 2973.92 samples/s lr: 2.85e-05 [09/23 12:23:22] lb.utils.events INFO: eta: 1:24:21 iteration: 342699/375342 consumed_samples: 350924800 total_loss: 2.979 time: 0.3443 s/iter data_time: 0.2129 s/iter total_throughput: 2973.92 samples/s lr: 2.84e-05 [09/23 12:23:56] lb.utils.events INFO: eta: 1:24:06 iteration: 342799/375342 consumed_samples: 351027200 total_loss: 2.973 time: 0.3443 s/iter data_time: 0.2291 s/iter total_throughput: 2973.92 samples/s lr: 2.82e-05 [09/23 12:24:31] lb.utils.events INFO: eta: 1:23:43 iteration: 342899/375342 consumed_samples: 351129600 total_loss: 2.963 time: 0.3443 s/iter data_time: 0.2253 s/iter total_throughput: 2973.91 samples/s lr: 2.81e-05 [09/23 12:25:06] lb.utils.events INFO: eta: 1:23:19 iteration: 342999/375342 consumed_samples: 351232000 total_loss: 2.961 time: 0.3443 s/iter data_time: 0.2118 s/iter total_throughput: 2973.91 samples/s lr: 2.80e-05 [09/23 12:25:40] lb.utils.events INFO: eta: 1:23:27 iteration: 343099/375342 consumed_samples: 351334400 total_loss: 2.98 time: 0.3443 s/iter data_time: 0.2272 s/iter total_throughput: 2973.90 samples/s lr: 2.79e-05 [09/23 12:26:15] lb.utils.events INFO: eta: 1:23:12 iteration: 343199/375342 consumed_samples: 351436800 total_loss: 2.968 time: 0.3443 s/iter data_time: 0.2308 s/iter total_throughput: 2973.89 samples/s lr: 2.78e-05 [09/23 12:26:50] lb.utils.events INFO: eta: 1:23:05 iteration: 343299/375342 consumed_samples: 351539200 total_loss: 2.96 time: 0.3443 s/iter data_time: 0.2193 s/iter total_throughput: 2973.88 samples/s lr: 2.77e-05 [09/23 12:27:25] lb.utils.events INFO: eta: 1:22:52 iteration: 343399/375342 consumed_samples: 351641600 total_loss: 2.965 time: 0.3443 s/iter data_time: 0.2171 s/iter total_throughput: 2973.86 samples/s lr: 2.76e-05 [09/23 12:28:00] lb.utils.events INFO: eta: 1:22:33 iteration: 343499/375342 consumed_samples: 351744000 total_loss: 2.949 time: 0.3443 s/iter data_time: 0.2258 s/iter total_throughput: 2973.86 samples/s lr: 2.75e-05 [09/23 12:28:35] lb.utils.events INFO: eta: 1:22:26 iteration: 343599/375342 consumed_samples: 351846400 total_loss: 2.966 time: 0.3443 s/iter data_time: 0.2218 s/iter total_throughput: 2973.84 samples/s lr: 2.74e-05 [09/23 12:29:09] lb.utils.events INFO: eta: 1:22:21 iteration: 343699/375342 consumed_samples: 351948800 total_loss: 2.975 time: 0.3443 s/iter data_time: 0.2130 s/iter total_throughput: 2973.85 samples/s lr: 2.73e-05 [09/23 12:29:43] lb.utils.events INFO: eta: 1:22:00 iteration: 343799/375342 consumed_samples: 352051200 total_loss: 2.968 time: 0.3443 s/iter data_time: 0.2080 s/iter total_throughput: 2973.85 samples/s lr: 2.72e-05 [09/23 12:30:18] lb.utils.events INFO: eta: 1:21:51 iteration: 343899/375342 consumed_samples: 352153600 total_loss: 2.974 time: 0.3443 s/iter data_time: 0.2178 s/iter total_throughput: 2973.85 samples/s lr: 2.70e-05 [09/23 12:30:52] lb.utils.events INFO: eta: 1:21:35 iteration: 343999/375342 consumed_samples: 352256000 total_loss: 2.968 time: 0.3443 s/iter data_time: 0.2117 s/iter total_throughput: 2973.85 samples/s lr: 2.69e-05 [09/23 12:31:27] lb.utils.events INFO: eta: 1:21:22 iteration: 344099/375342 consumed_samples: 352358400 total_loss: 2.949 time: 0.3443 s/iter data_time: 0.2169 s/iter total_throughput: 2973.84 samples/s lr: 2.68e-05 [09/23 12:32:02] lb.utils.events INFO: eta: 1:21:05 iteration: 344199/375342 consumed_samples: 352460800 total_loss: 2.963 time: 0.3443 s/iter data_time: 0.2262 s/iter total_throughput: 2973.84 samples/s lr: 2.67e-05 [09/23 12:32:36] lb.utils.events INFO: eta: 1:21:02 iteration: 344299/375342 consumed_samples: 352563200 total_loss: 2.969 time: 0.3443 s/iter data_time: 0.2172 s/iter total_throughput: 2973.83 samples/s lr: 2.66e-05 [09/23 12:33:11] lb.utils.events INFO: eta: 1:20:44 iteration: 344399/375342 consumed_samples: 352665600 total_loss: 2.969 time: 0.3443 s/iter data_time: 0.2241 s/iter total_throughput: 2973.84 samples/s lr: 2.65e-05 [09/23 12:33:45] lb.utils.events INFO: eta: 1:20:26 iteration: 344499/375342 consumed_samples: 352768000 total_loss: 2.972 time: 0.3443 s/iter data_time: 0.2171 s/iter total_throughput: 2973.83 samples/s lr: 2.64e-05 [09/23 12:34:19] lb.utils.events INFO: eta: 1:20:02 iteration: 344599/375342 consumed_samples: 352870400 total_loss: 2.968 time: 0.3443 s/iter data_time: 0.2108 s/iter total_throughput: 2973.84 samples/s lr: 2.63e-05 [09/23 12:34:54] lb.utils.events INFO: eta: 1:19:47 iteration: 344699/375342 consumed_samples: 352972800 total_loss: 2.961 time: 0.3443 s/iter data_time: 0.2065 s/iter total_throughput: 2973.84 samples/s lr: 2.62e-05 [09/23 12:35:28] lb.utils.events INFO: eta: 1:19:32 iteration: 344799/375342 consumed_samples: 353075200 total_loss: 2.969 time: 0.3443 s/iter data_time: 0.2238 s/iter total_throughput: 2973.84 samples/s lr: 2.61e-05 [09/23 12:36:02] lb.utils.events INFO: eta: 1:19:30 iteration: 344899/375342 consumed_samples: 353177600 total_loss: 2.965 time: 0.3443 s/iter data_time: 0.2041 s/iter total_throughput: 2973.85 samples/s lr: 2.60e-05 [09/23 12:36:36] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0344999 [09/23 12:36:37] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/23 12:36:37] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/23 12:36:41] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.1003 s/iter. Inference: 0.1610 s/iter. Eval: 0.0021 s/iter. Total: 0.2634 s/iter. ETA=0:00:09 [09/23 12:36:46] lb.evaluation.evaluator INFO: Inference done 28672/50000. Dataloading: 0.1164 s/iter. Inference: 0.1689 s/iter. Eval: 0.0021 s/iter. Total: 0.2875 s/iter. ETA=0:00:05 [09/23 12:36:51] lb.evaluation.evaluator INFO: Inference done 46080/50000. Dataloading: 0.1232 s/iter. Inference: 0.1664 s/iter. Eval: 0.0021 s/iter. Total: 0.2918 s/iter. ETA=0:00:00 [09/23 12:36:52] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/23 12:36:52] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.613401 (0.000252 s / iter per device, on 8 devices) [09/23 12:36:52] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000147 s / iter per device, on 8 devices) [09/23 12:36:52] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/23 12:36:52] lb.evaluation.utils INFO: copypaste: Acc@1=79.93599999999999 [09/23 12:36:52] lb.evaluation.utils INFO: copypaste: Acc@5=94.428 [09/23 12:36:52] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 79.93600, better than last best score 79.83800 @ iteration 339999. [09/23 12:36:52] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/23 12:36:53] lb.utils.events INFO: eta: 1:19:26 iteration: 344999/375342 consumed_samples: 353280000 total_loss: 2.962 time: 0.3443 s/iter data_time: 0.2101 s/iter total_throughput: 2973.86 samples/s lr: 2.59e-05 [09/23 12:37:26] lb.utils.events INFO: eta: 1:19:14 iteration: 345099/375342 consumed_samples: 353382400 total_loss: 2.971 time: 0.3443 s/iter data_time: 0.2284 s/iter total_throughput: 2973.90 samples/s lr: 2.58e-05 [09/23 12:38:00] lb.utils.events INFO: eta: 1:19:00 iteration: 345199/375342 consumed_samples: 353484800 total_loss: 2.96 time: 0.3443 s/iter data_time: 0.2106 s/iter total_throughput: 2973.90 samples/s lr: 2.57e-05 [09/23 12:38:34] lb.utils.events INFO: eta: 1:18:40 iteration: 345299/375342 consumed_samples: 353587200 total_loss: 2.951 time: 0.3443 s/iter data_time: 0.2166 s/iter total_throughput: 2973.91 samples/s lr: 2.56e-05 [09/23 12:39:08] lb.utils.events INFO: eta: 1:18:40 iteration: 345399/375342 consumed_samples: 353689600 total_loss: 2.963 time: 0.3443 s/iter data_time: 0.2146 s/iter total_throughput: 2973.92 samples/s lr: 2.55e-05 [09/23 12:39:43] lb.utils.events INFO: eta: 1:18:36 iteration: 345499/375342 consumed_samples: 353792000 total_loss: 2.963 time: 0.3443 s/iter data_time: 0.2123 s/iter total_throughput: 2973.93 samples/s lr: 2.54e-05 [09/23 12:40:17] lb.utils.events INFO: eta: 1:18:34 iteration: 345599/375342 consumed_samples: 353894400 total_loss: 2.97 time: 0.3443 s/iter data_time: 0.2200 s/iter total_throughput: 2973.93 samples/s lr: 2.53e-05 [09/23 12:40:52] lb.utils.events INFO: eta: 1:18:17 iteration: 345699/375342 consumed_samples: 353996800 total_loss: 2.964 time: 0.3443 s/iter data_time: 0.2188 s/iter total_throughput: 2973.92 samples/s lr: 2.52e-05 [09/23 12:41:26] lb.utils.events INFO: eta: 1:18:04 iteration: 345799/375342 consumed_samples: 354099200 total_loss: 2.955 time: 0.3443 s/iter data_time: 0.2212 s/iter total_throughput: 2973.91 samples/s lr: 2.51e-05 [09/23 12:42:01] lb.utils.events INFO: eta: 1:17:33 iteration: 345899/375342 consumed_samples: 354201600 total_loss: 2.969 time: 0.3443 s/iter data_time: 0.2192 s/iter total_throughput: 2973.92 samples/s lr: 2.50e-05 [09/23 12:42:36] lb.utils.events INFO: eta: 1:17:13 iteration: 345999/375342 consumed_samples: 354304000 total_loss: 2.972 time: 0.3443 s/iter data_time: 0.2249 s/iter total_throughput: 2973.91 samples/s lr: 2.49e-05 [09/23 12:43:10] lb.utils.events INFO: eta: 1:16:38 iteration: 346099/375342 consumed_samples: 354406400 total_loss: 2.97 time: 0.3443 s/iter data_time: 0.2133 s/iter total_throughput: 2973.90 samples/s lr: 2.48e-05 [09/23 12:43:44] lb.utils.events INFO: eta: 1:16:32 iteration: 346199/375342 consumed_samples: 354508800 total_loss: 2.947 time: 0.3443 s/iter data_time: 0.2157 s/iter total_throughput: 2973.91 samples/s lr: 2.47e-05 [09/23 12:44:19] lb.utils.events INFO: eta: 1:16:06 iteration: 346299/375342 consumed_samples: 354611200 total_loss: 2.959 time: 0.3443 s/iter data_time: 0.2127 s/iter total_throughput: 2973.91 samples/s lr: 2.46e-05 [09/23 12:44:53] lb.utils.events INFO: eta: 1:15:41 iteration: 346399/375342 consumed_samples: 354713600 total_loss: 2.98 time: 0.3443 s/iter data_time: 0.2186 s/iter total_throughput: 2973.91 samples/s lr: 2.45e-05 [09/23 12:45:28] lb.utils.events INFO: eta: 1:15:23 iteration: 346499/375342 consumed_samples: 354816000 total_loss: 2.977 time: 0.3443 s/iter data_time: 0.2229 s/iter total_throughput: 2973.91 samples/s lr: 2.44e-05 [09/23 12:46:02] lb.utils.events INFO: eta: 1:14:52 iteration: 346599/375342 consumed_samples: 354918400 total_loss: 2.96 time: 0.3443 s/iter data_time: 0.2042 s/iter total_throughput: 2973.92 samples/s lr: 2.43e-05 [09/23 12:46:37] lb.utils.events INFO: eta: 1:14:47 iteration: 346699/375342 consumed_samples: 355020800 total_loss: 2.971 time: 0.3443 s/iter data_time: 0.2078 s/iter total_throughput: 2973.91 samples/s lr: 2.42e-05 [09/23 12:47:11] lb.utils.events INFO: eta: 1:14:36 iteration: 346799/375342 consumed_samples: 355123200 total_loss: 2.965 time: 0.3443 s/iter data_time: 0.2097 s/iter total_throughput: 2973.92 samples/s lr: 2.41e-05 [09/23 12:47:45] lb.utils.events INFO: eta: 1:14:22 iteration: 346899/375342 consumed_samples: 355225600 total_loss: 2.98 time: 0.3443 s/iter data_time: 0.2116 s/iter total_throughput: 2973.92 samples/s lr: 2.40e-05 [09/23 12:48:20] lb.utils.events INFO: eta: 1:13:46 iteration: 346999/375342 consumed_samples: 355328000 total_loss: 2.988 time: 0.3443 s/iter data_time: 0.2152 s/iter total_throughput: 2973.92 samples/s lr: 2.39e-05 [09/23 12:48:54] lb.utils.events INFO: eta: 1:13:34 iteration: 347099/375342 consumed_samples: 355430400 total_loss: 2.953 time: 0.3443 s/iter data_time: 0.2242 s/iter total_throughput: 2973.92 samples/s lr: 2.38e-05 [09/23 12:49:29] lb.utils.events INFO: eta: 1:13:10 iteration: 347199/375342 consumed_samples: 355532800 total_loss: 2.951 time: 0.3443 s/iter data_time: 0.2190 s/iter total_throughput: 2973.91 samples/s lr: 2.37e-05 [09/23 12:50:03] lb.utils.events INFO: eta: 1:12:57 iteration: 347299/375342 consumed_samples: 355635200 total_loss: 2.964 time: 0.3443 s/iter data_time: 0.2250 s/iter total_throughput: 2973.91 samples/s lr: 2.36e-05 [09/23 12:50:38] lb.utils.events INFO: eta: 1:12:45 iteration: 347399/375342 consumed_samples: 355737600 total_loss: 2.954 time: 0.3443 s/iter data_time: 0.2214 s/iter total_throughput: 2973.91 samples/s lr: 2.35e-05 [09/23 12:51:12] lb.utils.events INFO: eta: 1:12:33 iteration: 347499/375342 consumed_samples: 355840000 total_loss: 2.953 time: 0.3443 s/iter data_time: 0.2153 s/iter total_throughput: 2973.91 samples/s lr: 2.34e-05 [09/23 12:51:47] lb.utils.events INFO: eta: 1:12:18 iteration: 347599/375342 consumed_samples: 355942400 total_loss: 2.964 time: 0.3443 s/iter data_time: 0.2217 s/iter total_throughput: 2973.91 samples/s lr: 2.33e-05 [09/23 12:52:21] lb.utils.events INFO: eta: 1:11:54 iteration: 347699/375342 consumed_samples: 356044800 total_loss: 2.967 time: 0.3443 s/iter data_time: 0.2087 s/iter total_throughput: 2973.91 samples/s lr: 2.32e-05 [09/23 12:52:55] lb.utils.events INFO: eta: 1:11:38 iteration: 347799/375342 consumed_samples: 356147200 total_loss: 2.968 time: 0.3443 s/iter data_time: 0.2156 s/iter total_throughput: 2973.92 samples/s lr: 2.31e-05 [09/23 12:53:30] lb.utils.events INFO: eta: 1:11:14 iteration: 347899/375342 consumed_samples: 356249600 total_loss: 2.952 time: 0.3443 s/iter data_time: 0.2173 s/iter total_throughput: 2973.92 samples/s lr: 2.30e-05 [09/23 12:54:04] lb.utils.events INFO: eta: 1:11:07 iteration: 347999/375342 consumed_samples: 356352000 total_loss: 2.95 time: 0.3443 s/iter data_time: 0.2134 s/iter total_throughput: 2973.92 samples/s lr: 2.29e-05 [09/23 12:54:38] lb.utils.events INFO: eta: 1:11:02 iteration: 348099/375342 consumed_samples: 356454400 total_loss: 2.968 time: 0.3443 s/iter data_time: 0.2221 s/iter total_throughput: 2973.93 samples/s lr: 2.28e-05 [09/23 12:55:12] lb.utils.events INFO: eta: 1:10:50 iteration: 348199/375342 consumed_samples: 356556800 total_loss: 2.953 time: 0.3443 s/iter data_time: 0.2274 s/iter total_throughput: 2973.93 samples/s lr: 2.27e-05 [09/23 12:55:47] lb.utils.events INFO: eta: 1:10:37 iteration: 348299/375342 consumed_samples: 356659200 total_loss: 2.939 time: 0.3443 s/iter data_time: 0.2272 s/iter total_throughput: 2973.93 samples/s lr: 2.26e-05 [09/23 12:56:21] lb.utils.events INFO: eta: 1:10:20 iteration: 348399/375342 consumed_samples: 356761600 total_loss: 2.955 time: 0.3443 s/iter data_time: 0.2184 s/iter total_throughput: 2973.93 samples/s lr: 2.25e-05 [09/23 12:56:55] lb.utils.events INFO: eta: 1:10:01 iteration: 348499/375342 consumed_samples: 356864000 total_loss: 2.941 time: 0.3443 s/iter data_time: 0.2300 s/iter total_throughput: 2973.94 samples/s lr: 2.24e-05 [09/23 12:57:30] lb.utils.events INFO: eta: 1:09:41 iteration: 348599/375342 consumed_samples: 356966400 total_loss: 2.942 time: 0.3443 s/iter data_time: 0.2072 s/iter total_throughput: 2973.95 samples/s lr: 2.23e-05 [09/23 12:58:04] lb.utils.events INFO: eta: 1:09:22 iteration: 348699/375342 consumed_samples: 357068800 total_loss: 2.962 time: 0.3443 s/iter data_time: 0.2109 s/iter total_throughput: 2973.95 samples/s lr: 2.23e-05 [09/23 12:58:38] lb.utils.events INFO: eta: 1:09:02 iteration: 348799/375342 consumed_samples: 357171200 total_loss: 2.973 time: 0.3443 s/iter data_time: 0.2116 s/iter total_throughput: 2973.96 samples/s lr: 2.22e-05 [09/23 12:59:12] lb.utils.events INFO: eta: 1:08:54 iteration: 348899/375342 consumed_samples: 357273600 total_loss: 2.965 time: 0.3443 s/iter data_time: 0.2127 s/iter total_throughput: 2973.98 samples/s lr: 2.21e-05 [09/23 12:59:46] lb.utils.events INFO: eta: 1:08:41 iteration: 348999/375342 consumed_samples: 357376000 total_loss: 2.964 time: 0.3443 s/iter data_time: 0.2059 s/iter total_throughput: 2973.98 samples/s lr: 2.20e-05 [09/23 13:00:19] lb.utils.events INFO: eta: 1:08:25 iteration: 349099/375342 consumed_samples: 357478400 total_loss: 2.973 time: 0.3443 s/iter data_time: 0.2061 s/iter total_throughput: 2974.00 samples/s lr: 2.19e-05 [09/23 13:00:53] lb.utils.events INFO: eta: 1:08:05 iteration: 349199/375342 consumed_samples: 357580800 total_loss: 2.958 time: 0.3443 s/iter data_time: 0.2178 s/iter total_throughput: 2974.02 samples/s lr: 2.18e-05 [09/23 13:01:27] lb.utils.events INFO: eta: 1:07:55 iteration: 349299/375342 consumed_samples: 357683200 total_loss: 2.945 time: 0.3443 s/iter data_time: 0.2037 s/iter total_throughput: 2974.02 samples/s lr: 2.17e-05 [09/23 13:02:01] lb.utils.events INFO: eta: 1:07:42 iteration: 349399/375342 consumed_samples: 357785600 total_loss: 2.956 time: 0.3443 s/iter data_time: 0.2098 s/iter total_throughput: 2974.04 samples/s lr: 2.16e-05 [09/23 13:02:35] lb.utils.events INFO: eta: 1:07:32 iteration: 349499/375342 consumed_samples: 357888000 total_loss: 2.971 time: 0.3443 s/iter data_time: 0.2219 s/iter total_throughput: 2974.05 samples/s lr: 2.15e-05 [09/23 13:03:09] lb.utils.events INFO: eta: 1:07:24 iteration: 349599/375342 consumed_samples: 357990400 total_loss: 2.963 time: 0.3443 s/iter data_time: 0.2095 s/iter total_throughput: 2974.06 samples/s lr: 2.14e-05 [09/23 13:03:43] lb.utils.events INFO: eta: 1:07:09 iteration: 349699/375342 consumed_samples: 358092800 total_loss: 2.957 time: 0.3443 s/iter data_time: 0.2172 s/iter total_throughput: 2974.07 samples/s lr: 2.14e-05 [09/23 13:04:17] lb.utils.events INFO: eta: 1:07:00 iteration: 349799/375342 consumed_samples: 358195200 total_loss: 2.951 time: 0.3443 s/iter data_time: 0.2160 s/iter total_throughput: 2974.09 samples/s lr: 2.13e-05 [09/23 13:04:51] lb.utils.events INFO: eta: 1:06:38 iteration: 349899/375342 consumed_samples: 358297600 total_loss: 2.958 time: 0.3443 s/iter data_time: 0.2161 s/iter total_throughput: 2974.10 samples/s lr: 2.12e-05 [09/23 13:05:25] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0349999 [09/23 13:05:25] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/23 13:05:25] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/23 13:05:30] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0979 s/iter. Inference: 0.1624 s/iter. Eval: 0.0019 s/iter. Total: 0.2622 s/iter. ETA=0:00:09 [09/23 13:05:35] lb.evaluation.evaluator INFO: Inference done 28672/50000. Dataloading: 0.0961 s/iter. Inference: 0.1895 s/iter. Eval: 0.0021 s/iter. Total: 0.2877 s/iter. ETA=0:00:05 [09/23 13:05:40] lb.evaluation.evaluator INFO: Inference done 46080/50000. Dataloading: 0.1073 s/iter. Inference: 0.1836 s/iter. Eval: 0.0021 s/iter. Total: 0.2930 s/iter. ETA=0:00:00 [09/23 13:05:41] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/23 13:05:41] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.655478 (0.000253 s / iter per device, on 8 devices) [09/23 13:05:41] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:08 (0.000161 s / iter per device, on 8 devices) [09/23 13:05:41] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/23 13:05:41] lb.evaluation.utils INFO: copypaste: Acc@1=79.93400000000001 [09/23 13:05:41] lb.evaluation.utils INFO: copypaste: Acc@5=94.484 [09/23 13:05:41] lb.engine.hooks INFO: Not saving as latest eval score for Acc@1 is 79.93400, not better than best score 79.93600 @ iteration 344999. [09/23 13:05:41] lb.utils.events INFO: eta: 1:06:22 iteration: 349999/375342 consumed_samples: 358400000 total_loss: 2.968 time: 0.3443 s/iter data_time: 0.2063 s/iter total_throughput: 2974.12 samples/s lr: 2.11e-05 [09/23 13:06:12] lb.utils.events INFO: eta: 1:06:10 iteration: 350099/375342 consumed_samples: 358502400 total_loss: 2.957 time: 0.3443 s/iter data_time: 0.2113 s/iter total_throughput: 2974.18 samples/s lr: 2.10e-05 [09/23 13:06:48] lb.utils.events INFO: eta: 1:05:51 iteration: 350199/375342 consumed_samples: 358604800 total_loss: 2.942 time: 0.3443 s/iter data_time: 0.2162 s/iter total_throughput: 2974.16 samples/s lr: 2.09e-05 [09/23 13:07:22] lb.utils.events INFO: eta: 1:05:35 iteration: 350299/375342 consumed_samples: 358707200 total_loss: 2.958 time: 0.3443 s/iter data_time: 0.2119 s/iter total_throughput: 2974.16 samples/s lr: 2.08e-05 [09/23 13:07:56] lb.utils.events INFO: eta: 1:05:09 iteration: 350399/375342 consumed_samples: 358809600 total_loss: 2.962 time: 0.3443 s/iter data_time: 0.2047 s/iter total_throughput: 2974.17 samples/s lr: 2.07e-05 [09/23 13:08:31] lb.utils.events INFO: eta: 1:04:44 iteration: 350499/375342 consumed_samples: 358912000 total_loss: 2.949 time: 0.3443 s/iter data_time: 0.2297 s/iter total_throughput: 2974.16 samples/s lr: 2.07e-05 [09/23 13:09:06] lb.utils.events INFO: eta: 1:04:24 iteration: 350599/375342 consumed_samples: 359014400 total_loss: 2.944 time: 0.3443 s/iter data_time: 0.2221 s/iter total_throughput: 2974.15 samples/s lr: 2.06e-05 [09/23 13:09:41] lb.utils.events INFO: eta: 1:04:08 iteration: 350699/375342 consumed_samples: 359116800 total_loss: 2.945 time: 0.3443 s/iter data_time: 0.2196 s/iter total_throughput: 2974.14 samples/s lr: 2.05e-05 [09/23 13:10:16] lb.utils.events INFO: eta: 1:03:59 iteration: 350799/375342 consumed_samples: 359219200 total_loss: 2.961 time: 0.3443 s/iter data_time: 0.2287 s/iter total_throughput: 2974.12 samples/s lr: 2.04e-05 [09/23 13:10:51] lb.utils.events INFO: eta: 1:03:43 iteration: 350899/375342 consumed_samples: 359321600 total_loss: 2.958 time: 0.3443 s/iter data_time: 0.2204 s/iter total_throughput: 2974.11 samples/s lr: 2.03e-05 [09/23 13:11:25] lb.utils.events INFO: eta: 1:03:21 iteration: 350999/375342 consumed_samples: 359424000 total_loss: 2.962 time: 0.3443 s/iter data_time: 0.2257 s/iter total_throughput: 2974.11 samples/s lr: 2.02e-05 [09/23 13:12:01] lb.utils.events INFO: eta: 1:03:01 iteration: 351099/375342 consumed_samples: 359526400 total_loss: 2.953 time: 0.3443 s/iter data_time: 0.2334 s/iter total_throughput: 2974.09 samples/s lr: 2.02e-05 [09/23 13:12:35] lb.utils.events INFO: eta: 1:02:54 iteration: 351199/375342 consumed_samples: 359628800 total_loss: 2.943 time: 0.3443 s/iter data_time: 0.2188 s/iter total_throughput: 2974.08 samples/s lr: 2.01e-05 [09/23 13:13:10] lb.utils.events INFO: eta: 1:02:38 iteration: 351299/375342 consumed_samples: 359731200 total_loss: 2.972 time: 0.3443 s/iter data_time: 0.2277 s/iter total_throughput: 2974.08 samples/s lr: 2.00e-05 [09/23 13:13:44] lb.utils.events INFO: eta: 1:02:21 iteration: 351399/375342 consumed_samples: 359833600 total_loss: 2.966 time: 0.3443 s/iter data_time: 0.2149 s/iter total_throughput: 2974.08 samples/s lr: 1.99e-05 [09/23 13:14:19] lb.utils.events INFO: eta: 1:02:10 iteration: 351499/375342 consumed_samples: 359936000 total_loss: 2.961 time: 0.3443 s/iter data_time: 0.2366 s/iter total_throughput: 2974.07 samples/s lr: 1.98e-05 [09/23 13:14:54] lb.utils.events INFO: eta: 1:02:02 iteration: 351599/375342 consumed_samples: 360038400 total_loss: 2.947 time: 0.3443 s/iter data_time: 0.2111 s/iter total_throughput: 2974.06 samples/s lr: 1.97e-05 [09/23 13:15:29] lb.utils.events INFO: eta: 1:01:42 iteration: 351699/375342 consumed_samples: 360140800 total_loss: 2.944 time: 0.3443 s/iter data_time: 0.2162 s/iter total_throughput: 2974.05 samples/s lr: 1.97e-05 [09/23 13:16:03] lb.utils.events INFO: eta: 1:01:23 iteration: 351799/375342 consumed_samples: 360243200 total_loss: 2.94 time: 0.3443 s/iter data_time: 0.2112 s/iter total_throughput: 2974.05 samples/s lr: 1.96e-05 [09/23 13:16:38] lb.utils.events INFO: eta: 1:01:06 iteration: 351899/375342 consumed_samples: 360345600 total_loss: 2.946 time: 0.3443 s/iter data_time: 0.2262 s/iter total_throughput: 2974.04 samples/s lr: 1.95e-05 [09/23 13:17:13] lb.utils.events INFO: eta: 1:01:06 iteration: 351999/375342 consumed_samples: 360448000 total_loss: 2.947 time: 0.3443 s/iter data_time: 0.2154 s/iter total_throughput: 2974.04 samples/s lr: 1.94e-05 [09/23 13:17:47] lb.utils.events INFO: eta: 1:00:45 iteration: 352099/375342 consumed_samples: 360550400 total_loss: 2.938 time: 0.3443 s/iter data_time: 0.2051 s/iter total_throughput: 2974.04 samples/s lr: 1.93e-05 [09/23 13:18:22] lb.utils.events INFO: eta: 1:00:40 iteration: 352199/375342 consumed_samples: 360652800 total_loss: 2.938 time: 0.3443 s/iter data_time: 0.2091 s/iter total_throughput: 2974.03 samples/s lr: 1.93e-05 [09/23 13:18:56] lb.utils.events INFO: eta: 1:00:25 iteration: 352299/375342 consumed_samples: 360755200 total_loss: 2.952 time: 0.3443 s/iter data_time: 0.2211 s/iter total_throughput: 2974.03 samples/s lr: 1.92e-05 [09/23 13:19:31] lb.utils.events INFO: eta: 1:00:11 iteration: 352399/375342 consumed_samples: 360857600 total_loss: 2.956 time: 0.3443 s/iter data_time: 0.2209 s/iter total_throughput: 2974.02 samples/s lr: 1.91e-05 [09/23 13:20:06] lb.utils.events INFO: eta: 0:59:55 iteration: 352499/375342 consumed_samples: 360960000 total_loss: 2.96 time: 0.3443 s/iter data_time: 0.2161 s/iter total_throughput: 2974.02 samples/s lr: 1.90e-05 [09/23 13:20:40] lb.utils.events INFO: eta: 0:59:40 iteration: 352599/375342 consumed_samples: 361062400 total_loss: 2.96 time: 0.3443 s/iter data_time: 0.2095 s/iter total_throughput: 2974.01 samples/s lr: 1.89e-05 [09/23 13:21:15] lb.utils.events INFO: eta: 0:59:32 iteration: 352699/375342 consumed_samples: 361164800 total_loss: 2.951 time: 0.3443 s/iter data_time: 0.2199 s/iter total_throughput: 2974.01 samples/s lr: 1.89e-05 [09/23 13:21:50] lb.utils.events INFO: eta: 0:59:17 iteration: 352799/375342 consumed_samples: 361267200 total_loss: 2.948 time: 0.3443 s/iter data_time: 0.2222 s/iter total_throughput: 2974.00 samples/s lr: 1.88e-05 [09/23 13:22:24] lb.utils.events INFO: eta: 0:59:06 iteration: 352899/375342 consumed_samples: 361369600 total_loss: 2.943 time: 0.3443 s/iter data_time: 0.2209 s/iter total_throughput: 2973.99 samples/s lr: 1.87e-05 [09/23 13:22:59] lb.utils.events INFO: eta: 0:58:46 iteration: 352999/375342 consumed_samples: 361472000 total_loss: 2.943 time: 0.3443 s/iter data_time: 0.2098 s/iter total_throughput: 2973.99 samples/s lr: 1.86e-05 [09/23 13:23:34] lb.utils.events INFO: eta: 0:58:31 iteration: 353099/375342 consumed_samples: 361574400 total_loss: 2.946 time: 0.3443 s/iter data_time: 0.2221 s/iter total_throughput: 2973.99 samples/s lr: 1.86e-05 [09/23 13:24:08] lb.utils.events INFO: eta: 0:58:17 iteration: 353199/375342 consumed_samples: 361676800 total_loss: 2.963 time: 0.3443 s/iter data_time: 0.2116 s/iter total_throughput: 2973.98 samples/s lr: 1.85e-05 [09/23 13:24:42] lb.utils.events INFO: eta: 0:57:59 iteration: 353299/375342 consumed_samples: 361779200 total_loss: 2.956 time: 0.3443 s/iter data_time: 0.2105 s/iter total_throughput: 2973.99 samples/s lr: 1.84e-05 [09/23 13:25:17] lb.utils.events INFO: eta: 0:57:37 iteration: 353399/375342 consumed_samples: 361881600 total_loss: 2.924 time: 0.3443 s/iter data_time: 0.2210 s/iter total_throughput: 2973.98 samples/s lr: 1.83e-05 [09/23 13:25:52] lb.utils.events INFO: eta: 0:57:24 iteration: 353499/375342 consumed_samples: 361984000 total_loss: 2.933 time: 0.3443 s/iter data_time: 0.2104 s/iter total_throughput: 2973.97 samples/s lr: 1.82e-05 [09/23 13:26:27] lb.utils.events INFO: eta: 0:56:54 iteration: 353599/375342 consumed_samples: 362086400 total_loss: 2.949 time: 0.3443 s/iter data_time: 0.2230 s/iter total_throughput: 2973.96 samples/s lr: 1.82e-05 [09/23 13:27:01] lb.utils.events INFO: eta: 0:56:36 iteration: 353699/375342 consumed_samples: 362188800 total_loss: 2.952 time: 0.3443 s/iter data_time: 0.2096 s/iter total_throughput: 2973.96 samples/s lr: 1.81e-05 [09/23 13:27:36] lb.utils.events INFO: eta: 0:56:21 iteration: 353799/375342 consumed_samples: 362291200 total_loss: 2.973 time: 0.3443 s/iter data_time: 0.2148 s/iter total_throughput: 2973.96 samples/s lr: 1.80e-05 [09/23 13:28:10] lb.utils.events INFO: eta: 0:56:07 iteration: 353899/375342 consumed_samples: 362393600 total_loss: 2.975 time: 0.3443 s/iter data_time: 0.2053 s/iter total_throughput: 2973.96 samples/s lr: 1.80e-05 [09/23 13:28:45] lb.utils.events INFO: eta: 0:55:51 iteration: 353999/375342 consumed_samples: 362496000 total_loss: 2.942 time: 0.3443 s/iter data_time: 0.2105 s/iter total_throughput: 2973.96 samples/s lr: 1.79e-05 [09/23 13:29:19] lb.utils.events INFO: eta: 0:55:36 iteration: 354099/375342 consumed_samples: 362598400 total_loss: 2.943 time: 0.3443 s/iter data_time: 0.2187 s/iter total_throughput: 2973.96 samples/s lr: 1.78e-05 [09/23 13:29:54] lb.utils.events INFO: eta: 0:55:16 iteration: 354199/375342 consumed_samples: 362700800 total_loss: 2.957 time: 0.3443 s/iter data_time: 0.2318 s/iter total_throughput: 2973.95 samples/s lr: 1.77e-05 [09/23 13:30:29] lb.utils.events INFO: eta: 0:54:55 iteration: 354299/375342 consumed_samples: 362803200 total_loss: 2.949 time: 0.3443 s/iter data_time: 0.2151 s/iter total_throughput: 2973.94 samples/s lr: 1.77e-05 [09/23 13:31:04] lb.utils.events INFO: eta: 0:54:46 iteration: 354399/375342 consumed_samples: 362905600 total_loss: 2.953 time: 0.3443 s/iter data_time: 0.2375 s/iter total_throughput: 2973.93 samples/s lr: 1.76e-05 [09/23 13:31:39] lb.utils.events INFO: eta: 0:54:20 iteration: 354499/375342 consumed_samples: 363008000 total_loss: 2.95 time: 0.3443 s/iter data_time: 0.2196 s/iter total_throughput: 2973.92 samples/s lr: 1.75e-05 [09/23 13:32:13] lb.utils.events INFO: eta: 0:54:04 iteration: 354599/375342 consumed_samples: 363110400 total_loss: 2.951 time: 0.3443 s/iter data_time: 0.2274 s/iter total_throughput: 2973.91 samples/s lr: 1.74e-05 [09/23 13:32:48] lb.utils.events INFO: eta: 0:53:47 iteration: 354699/375342 consumed_samples: 363212800 total_loss: 2.955 time: 0.3443 s/iter data_time: 0.2117 s/iter total_throughput: 2973.91 samples/s lr: 1.74e-05 [09/23 13:33:23] lb.utils.events INFO: eta: 0:53:30 iteration: 354799/375342 consumed_samples: 363315200 total_loss: 2.95 time: 0.3443 s/iter data_time: 0.2171 s/iter total_throughput: 2973.90 samples/s lr: 1.73e-05 [09/23 13:33:58] lb.utils.events INFO: eta: 0:53:12 iteration: 354899/375342 consumed_samples: 363417600 total_loss: 2.95 time: 0.3443 s/iter data_time: 0.2195 s/iter total_throughput: 2973.89 samples/s lr: 1.72e-05 [09/23 13:34:32] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0354999 [09/23 13:34:32] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/23 13:34:32] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/23 13:34:37] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.1027 s/iter. Inference: 0.1629 s/iter. Eval: 0.0023 s/iter. Total: 0.2678 s/iter. ETA=0:00:09 [09/23 13:34:42] lb.evaluation.evaluator INFO: Inference done 28672/50000. Dataloading: 0.1108 s/iter. Inference: 0.1756 s/iter. Eval: 0.0020 s/iter. Total: 0.2886 s/iter. ETA=0:00:05 [09/23 13:34:47] lb.evaluation.evaluator INFO: Inference done 47104/50000. Dataloading: 0.1218 s/iter. Inference: 0.1683 s/iter. Eval: 0.0020 s/iter. Total: 0.2921 s/iter. ETA=0:00:00 [09/23 13:34:48] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/23 13:34:48] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.582567 (0.000252 s / iter per device, on 8 devices) [09/23 13:34:48] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000148 s / iter per device, on 8 devices) [09/23 13:34:48] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/23 13:34:48] lb.evaluation.utils INFO: copypaste: Acc@1=79.984 [09/23 13:34:48] lb.evaluation.utils INFO: copypaste: Acc@5=94.464 [09/23 13:34:48] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 79.98400, better than last best score 79.93600 @ iteration 344999. [09/23 13:34:48] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/23 13:34:48] lb.utils.events INFO: eta: 0:52:57 iteration: 354999/375342 consumed_samples: 363520000 total_loss: 2.94 time: 0.3443 s/iter data_time: 0.2109 s/iter total_throughput: 2973.90 samples/s lr: 1.72e-05 [09/23 13:35:21] lb.utils.events INFO: eta: 0:52:48 iteration: 355099/375342 consumed_samples: 363622400 total_loss: 2.947 time: 0.3443 s/iter data_time: 0.2119 s/iter total_throughput: 2973.94 samples/s lr: 1.71e-05 [09/23 13:35:56] lb.utils.events INFO: eta: 0:52:33 iteration: 355199/375342 consumed_samples: 363724800 total_loss: 2.948 time: 0.3443 s/iter data_time: 0.2161 s/iter total_throughput: 2973.94 samples/s lr: 1.70e-05 [09/23 13:36:31] lb.utils.events INFO: eta: 0:52:17 iteration: 355299/375342 consumed_samples: 363827200 total_loss: 2.955 time: 0.3443 s/iter data_time: 0.2143 s/iter total_throughput: 2973.92 samples/s lr: 1.69e-05 [09/23 13:37:05] lb.utils.events INFO: eta: 0:52:03 iteration: 355399/375342 consumed_samples: 363929600 total_loss: 2.962 time: 0.3443 s/iter data_time: 0.2008 s/iter total_throughput: 2973.92 samples/s lr: 1.69e-05 [09/23 13:37:40] lb.utils.events INFO: eta: 0:51:55 iteration: 355499/375342 consumed_samples: 364032000 total_loss: 2.959 time: 0.3443 s/iter data_time: 0.2153 s/iter total_throughput: 2973.91 samples/s lr: 1.68e-05 [09/23 13:38:15] lb.utils.events INFO: eta: 0:51:41 iteration: 355599/375342 consumed_samples: 364134400 total_loss: 2.956 time: 0.3443 s/iter data_time: 0.2308 s/iter total_throughput: 2973.90 samples/s lr: 1.67e-05 [09/23 13:38:49] lb.utils.events INFO: eta: 0:51:20 iteration: 355699/375342 consumed_samples: 364236800 total_loss: 2.949 time: 0.3443 s/iter data_time: 0.2169 s/iter total_throughput: 2973.90 samples/s lr: 1.67e-05 [09/23 13:39:24] lb.utils.events INFO: eta: 0:51:09 iteration: 355799/375342 consumed_samples: 364339200 total_loss: 2.947 time: 0.3443 s/iter data_time: 0.2116 s/iter total_throughput: 2973.90 samples/s lr: 1.66e-05 [09/23 13:39:59] lb.utils.events INFO: eta: 0:50:57 iteration: 355899/375342 consumed_samples: 364441600 total_loss: 2.951 time: 0.3443 s/iter data_time: 0.2191 s/iter total_throughput: 2973.89 samples/s lr: 1.65e-05 [09/23 13:40:33] lb.utils.events INFO: eta: 0:50:38 iteration: 355999/375342 consumed_samples: 364544000 total_loss: 2.972 time: 0.3443 s/iter data_time: 0.2146 s/iter total_throughput: 2973.89 samples/s lr: 1.65e-05 [09/23 13:41:07] lb.utils.events INFO: eta: 0:50:11 iteration: 356099/375342 consumed_samples: 364646400 total_loss: 2.965 time: 0.3443 s/iter data_time: 0.2102 s/iter total_throughput: 2973.89 samples/s lr: 1.64e-05 [09/23 13:41:42] lb.utils.events INFO: eta: 0:49:59 iteration: 356199/375342 consumed_samples: 364748800 total_loss: 2.953 time: 0.3443 s/iter data_time: 0.2078 s/iter total_throughput: 2973.90 samples/s lr: 1.63e-05 [09/23 13:42:16] lb.utils.events INFO: eta: 0:49:51 iteration: 356299/375342 consumed_samples: 364851200 total_loss: 2.955 time: 0.3443 s/iter data_time: 0.2140 s/iter total_throughput: 2973.90 samples/s lr: 1.63e-05 [09/23 13:42:50] lb.utils.events INFO: eta: 0:49:32 iteration: 356399/375342 consumed_samples: 364953600 total_loss: 2.967 time: 0.3443 s/iter data_time: 0.2081 s/iter total_throughput: 2973.90 samples/s lr: 1.62e-05 [09/23 13:43:25] lb.utils.events INFO: eta: 0:49:05 iteration: 356499/375342 consumed_samples: 365056000 total_loss: 2.969 time: 0.3443 s/iter data_time: 0.2257 s/iter total_throughput: 2973.89 samples/s lr: 1.61e-05 [09/23 13:44:00] lb.utils.events INFO: eta: 0:48:57 iteration: 356599/375342 consumed_samples: 365158400 total_loss: 2.957 time: 0.3443 s/iter data_time: 0.2141 s/iter total_throughput: 2973.89 samples/s lr: 1.61e-05 [09/23 13:44:34] lb.utils.events INFO: eta: 0:48:47 iteration: 356699/375342 consumed_samples: 365260800 total_loss: 2.954 time: 0.3443 s/iter data_time: 0.2065 s/iter total_throughput: 2973.90 samples/s lr: 1.60e-05 [09/23 13:45:09] lb.utils.events INFO: eta: 0:48:31 iteration: 356799/375342 consumed_samples: 365363200 total_loss: 2.961 time: 0.3443 s/iter data_time: 0.2232 s/iter total_throughput: 2973.89 samples/s lr: 1.59e-05 [09/23 13:45:43] lb.utils.events INFO: eta: 0:48:12 iteration: 356899/375342 consumed_samples: 365465600 total_loss: 2.974 time: 0.3443 s/iter data_time: 0.2129 s/iter total_throughput: 2973.89 samples/s lr: 1.59e-05 [09/23 13:46:17] lb.utils.events INFO: eta: 0:47:53 iteration: 356999/375342 consumed_samples: 365568000 total_loss: 2.972 time: 0.3443 s/iter data_time: 0.2225 s/iter total_throughput: 2973.89 samples/s lr: 1.58e-05 [09/23 13:46:52] lb.utils.events INFO: eta: 0:47:32 iteration: 357099/375342 consumed_samples: 365670400 total_loss: 2.951 time: 0.3443 s/iter data_time: 0.2336 s/iter total_throughput: 2973.88 samples/s lr: 1.58e-05 [09/23 13:47:27] lb.utils.events INFO: eta: 0:47:17 iteration: 357199/375342 consumed_samples: 365772800 total_loss: 2.951 time: 0.3443 s/iter data_time: 0.2250 s/iter total_throughput: 2973.87 samples/s lr: 1.57e-05 [09/23 13:48:02] lb.utils.events INFO: eta: 0:47:05 iteration: 357299/375342 consumed_samples: 365875200 total_loss: 2.958 time: 0.3443 s/iter data_time: 0.2107 s/iter total_throughput: 2973.87 samples/s lr: 1.56e-05 [09/23 13:48:36] lb.utils.events INFO: eta: 0:46:50 iteration: 357399/375342 consumed_samples: 365977600 total_loss: 2.941 time: 0.3443 s/iter data_time: 0.2103 s/iter total_throughput: 2973.86 samples/s lr: 1.56e-05 [09/23 13:49:11] lb.utils.events INFO: eta: 0:46:46 iteration: 357499/375342 consumed_samples: 366080000 total_loss: 2.949 time: 0.3443 s/iter data_time: 0.2260 s/iter total_throughput: 2973.86 samples/s lr: 1.55e-05 [09/23 13:49:45] lb.utils.events INFO: eta: 0:46:29 iteration: 357599/375342 consumed_samples: 366182400 total_loss: 2.949 time: 0.3443 s/iter data_time: 0.2141 s/iter total_throughput: 2973.87 samples/s lr: 1.54e-05 [09/23 13:50:19] lb.utils.events INFO: eta: 0:46:12 iteration: 357699/375342 consumed_samples: 366284800 total_loss: 2.92 time: 0.3443 s/iter data_time: 0.2126 s/iter total_throughput: 2973.87 samples/s lr: 1.54e-05 [09/23 13:50:53] lb.utils.events INFO: eta: 0:46:03 iteration: 357799/375342 consumed_samples: 366387200 total_loss: 2.939 time: 0.3443 s/iter data_time: 0.2040 s/iter total_throughput: 2973.88 samples/s lr: 1.53e-05 [09/23 13:51:28] lb.utils.events INFO: eta: 0:45:48 iteration: 357899/375342 consumed_samples: 366489600 total_loss: 2.953 time: 0.3443 s/iter data_time: 0.2150 s/iter total_throughput: 2973.88 samples/s lr: 1.53e-05 [09/23 13:52:03] lb.utils.events INFO: eta: 0:45:34 iteration: 357999/375342 consumed_samples: 366592000 total_loss: 2.95 time: 0.3443 s/iter data_time: 0.2254 s/iter total_throughput: 2973.87 samples/s lr: 1.52e-05 [09/23 13:52:38] lb.utils.events INFO: eta: 0:45:16 iteration: 358099/375342 consumed_samples: 366694400 total_loss: 2.948 time: 0.3443 s/iter data_time: 0.2084 s/iter total_throughput: 2973.86 samples/s lr: 1.51e-05 [09/23 13:53:12] lb.utils.events INFO: eta: 0:44:52 iteration: 358199/375342 consumed_samples: 366796800 total_loss: 2.939 time: 0.3443 s/iter data_time: 0.2187 s/iter total_throughput: 2973.85 samples/s lr: 1.51e-05 [09/23 13:53:47] lb.utils.events INFO: eta: 0:44:31 iteration: 358299/375342 consumed_samples: 366899200 total_loss: 2.942 time: 0.3443 s/iter data_time: 0.2296 s/iter total_throughput: 2973.84 samples/s lr: 1.50e-05 [09/23 13:54:22] lb.utils.events INFO: eta: 0:44:16 iteration: 358399/375342 consumed_samples: 367001600 total_loss: 2.938 time: 0.3443 s/iter data_time: 0.2132 s/iter total_throughput: 2973.84 samples/s lr: 1.50e-05 [09/23 13:54:56] lb.utils.events INFO: eta: 0:43:58 iteration: 358499/375342 consumed_samples: 367104000 total_loss: 2.954 time: 0.3443 s/iter data_time: 0.2222 s/iter total_throughput: 2973.84 samples/s lr: 1.49e-05 [09/23 13:55:31] lb.utils.events INFO: eta: 0:43:42 iteration: 358599/375342 consumed_samples: 367206400 total_loss: 2.945 time: 0.3443 s/iter data_time: 0.2284 s/iter total_throughput: 2973.83 samples/s lr: 1.49e-05 [09/23 13:56:06] lb.utils.events INFO: eta: 0:43:30 iteration: 358699/375342 consumed_samples: 367308800 total_loss: 2.928 time: 0.3443 s/iter data_time: 0.2081 s/iter total_throughput: 2973.83 samples/s lr: 1.48e-05 [09/23 13:56:40] lb.utils.events INFO: eta: 0:43:07 iteration: 358799/375342 consumed_samples: 367411200 total_loss: 2.936 time: 0.3443 s/iter data_time: 0.2199 s/iter total_throughput: 2973.82 samples/s lr: 1.47e-05 [09/23 13:57:15] lb.utils.events INFO: eta: 0:42:49 iteration: 358899/375342 consumed_samples: 367513600 total_loss: 2.933 time: 0.3443 s/iter data_time: 0.2110 s/iter total_throughput: 2973.82 samples/s lr: 1.47e-05 [09/23 13:57:49] lb.utils.events INFO: eta: 0:42:35 iteration: 358999/375342 consumed_samples: 367616000 total_loss: 2.937 time: 0.3443 s/iter data_time: 0.2141 s/iter total_throughput: 2973.83 samples/s lr: 1.46e-05 [09/23 13:58:24] lb.utils.events INFO: eta: 0:42:26 iteration: 359099/375342 consumed_samples: 367718400 total_loss: 2.938 time: 0.3443 s/iter data_time: 0.2244 s/iter total_throughput: 2973.82 samples/s lr: 1.46e-05 [09/23 13:58:58] lb.utils.events INFO: eta: 0:42:14 iteration: 359199/375342 consumed_samples: 367820800 total_loss: 2.931 time: 0.3443 s/iter data_time: 0.2221 s/iter total_throughput: 2973.82 samples/s lr: 1.45e-05 [09/23 13:59:33] lb.utils.events INFO: eta: 0:42:05 iteration: 359299/375342 consumed_samples: 367923200 total_loss: 2.938 time: 0.3443 s/iter data_time: 0.2214 s/iter total_throughput: 2973.82 samples/s lr: 1.45e-05 [09/23 14:00:07] lb.utils.events INFO: eta: 0:41:45 iteration: 359399/375342 consumed_samples: 368025600 total_loss: 2.933 time: 0.3443 s/iter data_time: 0.2127 s/iter total_throughput: 2973.82 samples/s lr: 1.44e-05 [09/23 14:00:42] lb.utils.events INFO: eta: 0:41:27 iteration: 359499/375342 consumed_samples: 368128000 total_loss: 2.924 time: 0.3443 s/iter data_time: 0.2176 s/iter total_throughput: 2973.81 samples/s lr: 1.43e-05 [09/23 14:01:17] lb.utils.events INFO: eta: 0:41:12 iteration: 359599/375342 consumed_samples: 368230400 total_loss: 2.932 time: 0.3443 s/iter data_time: 0.2135 s/iter total_throughput: 2973.81 samples/s lr: 1.43e-05 [09/23 14:01:51] lb.utils.events INFO: eta: 0:40:54 iteration: 359699/375342 consumed_samples: 368332800 total_loss: 2.942 time: 0.3443 s/iter data_time: 0.2228 s/iter total_throughput: 2973.81 samples/s lr: 1.42e-05 [09/23 14:02:25] lb.utils.events INFO: eta: 0:40:38 iteration: 359799/375342 consumed_samples: 368435200 total_loss: 2.945 time: 0.3443 s/iter data_time: 0.2150 s/iter total_throughput: 2973.82 samples/s lr: 1.42e-05 [09/23 14:02:59] lb.utils.events INFO: eta: 0:40:23 iteration: 359899/375342 consumed_samples: 368537600 total_loss: 2.932 time: 0.3443 s/iter data_time: 0.2068 s/iter total_throughput: 2973.82 samples/s lr: 1.41e-05 [09/23 14:03:34] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0359999 [09/23 14:03:34] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/23 14:03:34] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/23 14:03:39] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.1118 s/iter. Inference: 0.1694 s/iter. Eval: 0.0019 s/iter. Total: 0.2831 s/iter. ETA=0:00:10 [09/23 14:03:44] lb.evaluation.evaluator INFO: Inference done 28672/50000. Dataloading: 0.1218 s/iter. Inference: 0.1710 s/iter. Eval: 0.0020 s/iter. Total: 0.2948 s/iter. ETA=0:00:05 [09/23 14:03:49] lb.evaluation.evaluator INFO: Inference done 47104/50000. Dataloading: 0.1258 s/iter. Inference: 0.1683 s/iter. Eval: 0.0020 s/iter. Total: 0.2961 s/iter. ETA=0:00:00 [09/23 14:03:50] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/23 14:03:50] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.655341 (0.000253 s / iter per device, on 8 devices) [09/23 14:03:50] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000148 s / iter per device, on 8 devices) [09/23 14:03:50] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/23 14:03:50] lb.evaluation.utils INFO: copypaste: Acc@1=80.048 [09/23 14:03:50] lb.evaluation.utils INFO: copypaste: Acc@5=94.554 [09/23 14:03:50] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 80.04800, better than last best score 79.98400 @ iteration 354999. [09/23 14:03:50] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/23 14:03:50] lb.utils.events INFO: eta: 0:40:07 iteration: 359999/375342 consumed_samples: 368640000 total_loss: 2.931 time: 0.3443 s/iter data_time: 0.2183 s/iter total_throughput: 2973.82 samples/s lr: 1.41e-05 [09/23 14:04:23] lb.utils.events INFO: eta: 0:39:53 iteration: 360099/375342 consumed_samples: 368742400 total_loss: 2.924 time: 0.3443 s/iter data_time: 0.2053 s/iter total_throughput: 2973.88 samples/s lr: 1.40e-05 [09/23 14:04:57] lb.utils.events INFO: eta: 0:39:38 iteration: 360199/375342 consumed_samples: 368844800 total_loss: 2.936 time: 0.3443 s/iter data_time: 0.2066 s/iter total_throughput: 2973.88 samples/s lr: 1.40e-05 [09/23 14:05:31] lb.utils.events INFO: eta: 0:39:23 iteration: 360299/375342 consumed_samples: 368947200 total_loss: 2.944 time: 0.3443 s/iter data_time: 0.2211 s/iter total_throughput: 2973.89 samples/s lr: 1.39e-05 [09/23 14:06:05] lb.utils.events INFO: eta: 0:39:13 iteration: 360399/375342 consumed_samples: 369049600 total_loss: 2.937 time: 0.3443 s/iter data_time: 0.2064 s/iter total_throughput: 2973.90 samples/s lr: 1.39e-05 [09/23 14:06:40] lb.utils.events INFO: eta: 0:39:03 iteration: 360499/375342 consumed_samples: 369152000 total_loss: 2.949 time: 0.3443 s/iter data_time: 0.2120 s/iter total_throughput: 2973.90 samples/s lr: 1.38e-05 [09/23 14:07:14] lb.utils.events INFO: eta: 0:38:47 iteration: 360599/375342 consumed_samples: 369254400 total_loss: 2.953 time: 0.3443 s/iter data_time: 0.2222 s/iter total_throughput: 2973.90 samples/s lr: 1.38e-05 [09/23 14:07:48] lb.utils.events INFO: eta: 0:38:34 iteration: 360699/375342 consumed_samples: 369356800 total_loss: 2.938 time: 0.3443 s/iter data_time: 0.2091 s/iter total_throughput: 2973.90 samples/s lr: 1.37e-05 [09/23 14:08:23] lb.utils.events INFO: eta: 0:38:19 iteration: 360799/375342 consumed_samples: 369459200 total_loss: 2.936 time: 0.3443 s/iter data_time: 0.2135 s/iter total_throughput: 2973.90 samples/s lr: 1.37e-05 [09/23 14:08:57] lb.utils.events INFO: eta: 0:38:06 iteration: 360899/375342 consumed_samples: 369561600 total_loss: 2.918 time: 0.3443 s/iter data_time: 0.2237 s/iter total_throughput: 2973.90 samples/s lr: 1.36e-05 [09/23 14:09:32] lb.utils.events INFO: eta: 0:37:51 iteration: 360999/375342 consumed_samples: 369664000 total_loss: 2.914 time: 0.3443 s/iter data_time: 0.2112 s/iter total_throughput: 2973.90 samples/s lr: 1.36e-05 [09/23 14:10:06] lb.utils.events INFO: eta: 0:37:29 iteration: 361099/375342 consumed_samples: 369766400 total_loss: 2.94 time: 0.3443 s/iter data_time: 0.2156 s/iter total_throughput: 2973.90 samples/s lr: 1.35e-05 [09/23 14:10:40] lb.utils.events INFO: eta: 0:37:11 iteration: 361199/375342 consumed_samples: 369868800 total_loss: 2.936 time: 0.3443 s/iter data_time: 0.2192 s/iter total_throughput: 2973.91 samples/s lr: 1.35e-05 [09/23 14:11:14] lb.utils.events INFO: eta: 0:36:49 iteration: 361299/375342 consumed_samples: 369971200 total_loss: 2.941 time: 0.3443 s/iter data_time: 0.2117 s/iter total_throughput: 2973.92 samples/s lr: 1.34e-05 [09/23 14:11:48] lb.utils.events INFO: eta: 0:36:30 iteration: 361399/375342 consumed_samples: 370073600 total_loss: 2.94 time: 0.3443 s/iter data_time: 0.2067 s/iter total_throughput: 2973.92 samples/s lr: 1.34e-05 [09/23 14:12:23] lb.utils.events INFO: eta: 0:36:13 iteration: 361499/375342 consumed_samples: 370176000 total_loss: 2.922 time: 0.3443 s/iter data_time: 0.2296 s/iter total_throughput: 2973.93 samples/s lr: 1.33e-05 [09/23 14:12:57] lb.utils.events INFO: eta: 0:35:58 iteration: 361599/375342 consumed_samples: 370278400 total_loss: 2.922 time: 0.3443 s/iter data_time: 0.2127 s/iter total_throughput: 2973.93 samples/s lr: 1.33e-05 [09/23 14:13:31] lb.utils.events INFO: eta: 0:35:40 iteration: 361699/375342 consumed_samples: 370380800 total_loss: 2.931 time: 0.3443 s/iter data_time: 0.2039 s/iter total_throughput: 2973.94 samples/s lr: 1.32e-05 [09/23 14:14:06] lb.utils.events INFO: eta: 0:35:27 iteration: 361799/375342 consumed_samples: 370483200 total_loss: 2.931 time: 0.3443 s/iter data_time: 0.2313 s/iter total_throughput: 2973.94 samples/s lr: 1.32e-05 [09/23 14:14:40] lb.utils.events INFO: eta: 0:35:11 iteration: 361899/375342 consumed_samples: 370585600 total_loss: 2.932 time: 0.3443 s/iter data_time: 0.2195 s/iter total_throughput: 2973.93 samples/s lr: 1.31e-05 [09/23 14:15:15] lb.utils.events INFO: eta: 0:34:52 iteration: 361999/375342 consumed_samples: 370688000 total_loss: 2.94 time: 0.3443 s/iter data_time: 0.2090 s/iter total_throughput: 2973.94 samples/s lr: 1.31e-05 [09/23 14:15:49] lb.utils.events INFO: eta: 0:34:34 iteration: 362099/375342 consumed_samples: 370790400 total_loss: 2.93 time: 0.3443 s/iter data_time: 0.2205 s/iter total_throughput: 2973.93 samples/s lr: 1.30e-05 [09/23 14:16:24] lb.utils.events INFO: eta: 0:34:16 iteration: 362199/375342 consumed_samples: 370892800 total_loss: 2.914 time: 0.3443 s/iter data_time: 0.2166 s/iter total_throughput: 2973.93 samples/s lr: 1.30e-05 [09/23 14:16:58] lb.utils.events INFO: eta: 0:34:02 iteration: 362299/375342 consumed_samples: 370995200 total_loss: 2.919 time: 0.3443 s/iter data_time: 0.2184 s/iter total_throughput: 2973.94 samples/s lr: 1.29e-05 [09/23 14:17:31] lb.utils.events INFO: eta: 0:33:46 iteration: 362399/375342 consumed_samples: 371097600 total_loss: 2.927 time: 0.3443 s/iter data_time: 0.2099 s/iter total_throughput: 2973.96 samples/s lr: 1.29e-05 [09/23 14:18:05] lb.utils.events INFO: eta: 0:33:30 iteration: 362499/375342 consumed_samples: 371200000 total_loss: 2.938 time: 0.3443 s/iter data_time: 0.2076 s/iter total_throughput: 2973.97 samples/s lr: 1.29e-05 [09/23 14:18:40] lb.utils.events INFO: eta: 0:33:07 iteration: 362599/375342 consumed_samples: 371302400 total_loss: 2.946 time: 0.3443 s/iter data_time: 0.2179 s/iter total_throughput: 2973.98 samples/s lr: 1.28e-05 [09/23 14:19:14] lb.utils.events INFO: eta: 0:32:50 iteration: 362699/375342 consumed_samples: 371404800 total_loss: 2.945 time: 0.3443 s/iter data_time: 0.2203 s/iter total_throughput: 2973.98 samples/s lr: 1.28e-05 [09/23 14:19:48] lb.utils.events INFO: eta: 0:32:38 iteration: 362799/375342 consumed_samples: 371507200 total_loss: 2.94 time: 0.3443 s/iter data_time: 0.2129 s/iter total_throughput: 2973.99 samples/s lr: 1.27e-05 [09/23 14:20:22] lb.utils.events INFO: eta: 0:32:21 iteration: 362899/375342 consumed_samples: 371609600 total_loss: 2.937 time: 0.3443 s/iter data_time: 0.2126 s/iter total_throughput: 2973.99 samples/s lr: 1.27e-05 [09/23 14:20:57] lb.utils.events INFO: eta: 0:32:07 iteration: 362999/375342 consumed_samples: 371712000 total_loss: 2.926 time: 0.3443 s/iter data_time: 0.2162 s/iter total_throughput: 2973.99 samples/s lr: 1.26e-05 [09/23 14:21:31] lb.utils.events INFO: eta: 0:32:00 iteration: 363099/375342 consumed_samples: 371814400 total_loss: 2.938 time: 0.3443 s/iter data_time: 0.2073 s/iter total_throughput: 2974.00 samples/s lr: 1.26e-05 [09/23 14:22:05] lb.utils.events INFO: eta: 0:31:46 iteration: 363199/375342 consumed_samples: 371916800 total_loss: 2.937 time: 0.3443 s/iter data_time: 0.2132 s/iter total_throughput: 2974.01 samples/s lr: 1.26e-05 [09/23 14:22:39] lb.utils.events INFO: eta: 0:31:26 iteration: 363299/375342 consumed_samples: 372019200 total_loss: 2.932 time: 0.3443 s/iter data_time: 0.2067 s/iter total_throughput: 2974.02 samples/s lr: 1.25e-05 [09/23 14:23:13] lb.utils.events INFO: eta: 0:31:11 iteration: 363399/375342 consumed_samples: 372121600 total_loss: 2.938 time: 0.3443 s/iter data_time: 0.2083 s/iter total_throughput: 2974.03 samples/s lr: 1.25e-05 [09/23 14:23:47] lb.utils.events INFO: eta: 0:30:52 iteration: 363499/375342 consumed_samples: 372224000 total_loss: 2.918 time: 0.3443 s/iter data_time: 0.2239 s/iter total_throughput: 2974.04 samples/s lr: 1.24e-05 [09/23 14:24:21] lb.utils.events INFO: eta: 0:30:38 iteration: 363599/375342 consumed_samples: 372326400 total_loss: 2.914 time: 0.3443 s/iter data_time: 0.2134 s/iter total_throughput: 2974.04 samples/s lr: 1.24e-05 [09/23 14:24:55] lb.utils.events INFO: eta: 0:30:21 iteration: 363699/375342 consumed_samples: 372428800 total_loss: 2.934 time: 0.3443 s/iter data_time: 0.2131 s/iter total_throughput: 2974.06 samples/s lr: 1.23e-05 [09/23 14:25:29] lb.utils.events INFO: eta: 0:30:01 iteration: 363799/375342 consumed_samples: 372531200 total_loss: 2.964 time: 0.3443 s/iter data_time: 0.2233 s/iter total_throughput: 2974.07 samples/s lr: 1.23e-05 [09/23 14:26:02] lb.utils.events INFO: eta: 0:29:47 iteration: 363899/375342 consumed_samples: 372633600 total_loss: 2.953 time: 0.3443 s/iter data_time: 0.2063 s/iter total_throughput: 2974.09 samples/s lr: 1.23e-05 [09/23 14:26:36] lb.utils.events INFO: eta: 0:29:34 iteration: 363999/375342 consumed_samples: 372736000 total_loss: 2.946 time: 0.3443 s/iter data_time: 0.2016 s/iter total_throughput: 2974.10 samples/s lr: 1.22e-05 [09/23 14:27:10] lb.utils.events INFO: eta: 0:29:14 iteration: 364099/375342 consumed_samples: 372838400 total_loss: 2.948 time: 0.3443 s/iter data_time: 0.2076 s/iter total_throughput: 2974.11 samples/s lr: 1.22e-05 [09/23 14:27:45] lb.utils.events INFO: eta: 0:29:00 iteration: 364199/375342 consumed_samples: 372940800 total_loss: 2.936 time: 0.3443 s/iter data_time: 0.2165 s/iter total_throughput: 2974.11 samples/s lr: 1.22e-05 [09/23 14:28:20] lb.utils.events INFO: eta: 0:28:43 iteration: 364299/375342 consumed_samples: 373043200 total_loss: 2.933 time: 0.3443 s/iter data_time: 0.2154 s/iter total_throughput: 2974.11 samples/s lr: 1.21e-05 [09/23 14:28:54] lb.utils.events INFO: eta: 0:28:22 iteration: 364399/375342 consumed_samples: 373145600 total_loss: 2.939 time: 0.3443 s/iter data_time: 0.2164 s/iter total_throughput: 2974.10 samples/s lr: 1.21e-05 [09/23 14:29:28] lb.utils.events INFO: eta: 0:28:04 iteration: 364499/375342 consumed_samples: 373248000 total_loss: 2.932 time: 0.3443 s/iter data_time: 0.2203 s/iter total_throughput: 2974.10 samples/s lr: 1.20e-05 [09/23 14:30:03] lb.utils.events INFO: eta: 0:27:48 iteration: 364599/375342 consumed_samples: 373350400 total_loss: 2.912 time: 0.3443 s/iter data_time: 0.2248 s/iter total_throughput: 2974.09 samples/s lr: 1.20e-05 [09/23 14:30:38] lb.utils.events INFO: eta: 0:27:32 iteration: 364699/375342 consumed_samples: 373452800 total_loss: 2.91 time: 0.3443 s/iter data_time: 0.2280 s/iter total_throughput: 2974.08 samples/s lr: 1.20e-05 [09/23 14:31:14] lb.utils.events INFO: eta: 0:27:15 iteration: 364799/375342 consumed_samples: 373555200 total_loss: 2.928 time: 0.3443 s/iter data_time: 0.2217 s/iter total_throughput: 2974.06 samples/s lr: 1.19e-05 [09/23 14:31:49] lb.utils.events INFO: eta: 0:27:00 iteration: 364899/375342 consumed_samples: 373657600 total_loss: 2.954 time: 0.3443 s/iter data_time: 0.2230 s/iter total_throughput: 2974.05 samples/s lr: 1.19e-05 [09/23 14:32:23] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0364999 [09/23 14:32:24] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/23 14:32:24] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/23 14:32:28] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0918 s/iter. Inference: 0.1608 s/iter. Eval: 0.0022 s/iter. Total: 0.2549 s/iter. ETA=0:00:09 [09/23 14:32:33] lb.evaluation.evaluator INFO: Inference done 28672/50000. Dataloading: 0.1089 s/iter. Inference: 0.1750 s/iter. Eval: 0.0022 s/iter. Total: 0.2862 s/iter. ETA=0:00:05 [09/23 14:32:38] lb.evaluation.evaluator INFO: Inference done 46080/50000. Dataloading: 0.1212 s/iter. Inference: 0.1695 s/iter. Eval: 0.0022 s/iter. Total: 0.2929 s/iter. ETA=0:00:00 [09/23 14:32:39] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/23 14:32:39] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.655772 (0.000253 s / iter per device, on 8 devices) [09/23 14:32:39] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000149 s / iter per device, on 8 devices) [09/23 14:32:39] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/23 14:32:39] lb.evaluation.utils INFO: copypaste: Acc@1=80.048 [09/23 14:32:39] lb.evaluation.utils INFO: copypaste: Acc@5=94.50800000000001 [09/23 14:32:39] lb.engine.hooks INFO: Not saving as latest eval score for Acc@1 is 80.04800, not better than best score 80.04800 @ iteration 359999. [09/23 14:32:39] lb.utils.events INFO: eta: 0:26:46 iteration: 364999/375342 consumed_samples: 373760000 total_loss: 2.957 time: 0.3443 s/iter data_time: 0.2297 s/iter total_throughput: 2974.05 samples/s lr: 1.19e-05 [09/23 14:33:12] lb.utils.events INFO: eta: 0:26:34 iteration: 365099/375342 consumed_samples: 373862400 total_loss: 2.945 time: 0.3443 s/iter data_time: 0.2329 s/iter total_throughput: 2974.09 samples/s lr: 1.18e-05 [09/23 14:33:46] lb.utils.events INFO: eta: 0:26:20 iteration: 365199/375342 consumed_samples: 373964800 total_loss: 2.941 time: 0.3443 s/iter data_time: 0.2156 s/iter total_throughput: 2974.09 samples/s lr: 1.18e-05 [09/23 14:34:21] lb.utils.events INFO: eta: 0:26:06 iteration: 365299/375342 consumed_samples: 374067200 total_loss: 2.936 time: 0.3443 s/iter data_time: 0.2141 s/iter total_throughput: 2974.08 samples/s lr: 1.17e-05 [09/23 14:34:56] lb.utils.events INFO: eta: 0:25:50 iteration: 365399/375342 consumed_samples: 374169600 total_loss: 2.915 time: 0.3443 s/iter data_time: 0.2120 s/iter total_throughput: 2974.07 samples/s lr: 1.17e-05 [09/23 14:35:30] lb.utils.events INFO: eta: 0:25:38 iteration: 365499/375342 consumed_samples: 374272000 total_loss: 2.919 time: 0.3443 s/iter data_time: 0.2146 s/iter total_throughput: 2974.07 samples/s lr: 1.17e-05 [09/23 14:36:05] lb.utils.events INFO: eta: 0:25:25 iteration: 365599/375342 consumed_samples: 374374400 total_loss: 2.936 time: 0.3443 s/iter data_time: 0.2213 s/iter total_throughput: 2974.07 samples/s lr: 1.16e-05 [09/23 14:36:40] lb.utils.events INFO: eta: 0:25:14 iteration: 365699/375342 consumed_samples: 374476800 total_loss: 2.936 time: 0.3443 s/iter data_time: 0.2234 s/iter total_throughput: 2974.06 samples/s lr: 1.16e-05 [09/23 14:37:15] lb.utils.events INFO: eta: 0:25:01 iteration: 365799/375342 consumed_samples: 374579200 total_loss: 2.955 time: 0.3443 s/iter data_time: 0.2222 s/iter total_throughput: 2974.04 samples/s lr: 1.16e-05 [09/23 14:37:49] lb.utils.events INFO: eta: 0:24:46 iteration: 365899/375342 consumed_samples: 374681600 total_loss: 2.93 time: 0.3443 s/iter data_time: 0.2267 s/iter total_throughput: 2974.04 samples/s lr: 1.15e-05 [09/23 14:38:24] lb.utils.events INFO: eta: 0:24:25 iteration: 365999/375342 consumed_samples: 374784000 total_loss: 2.933 time: 0.3443 s/iter data_time: 0.2257 s/iter total_throughput: 2974.03 samples/s lr: 1.15e-05 [09/23 14:38:59] lb.utils.events INFO: eta: 0:24:06 iteration: 366099/375342 consumed_samples: 374886400 total_loss: 2.945 time: 0.3443 s/iter data_time: 0.2203 s/iter total_throughput: 2974.02 samples/s lr: 1.15e-05 [09/23 14:39:34] lb.utils.events INFO: eta: 0:23:43 iteration: 366199/375342 consumed_samples: 374988800 total_loss: 2.941 time: 0.3443 s/iter data_time: 0.2160 s/iter total_throughput: 2974.01 samples/s lr: 1.14e-05 [09/23 14:40:08] lb.utils.events INFO: eta: 0:23:29 iteration: 366299/375342 consumed_samples: 375091200 total_loss: 2.943 time: 0.3443 s/iter data_time: 0.2128 s/iter total_throughput: 2974.01 samples/s lr: 1.14e-05 [09/23 14:40:43] lb.utils.events INFO: eta: 0:23:15 iteration: 366399/375342 consumed_samples: 375193600 total_loss: 2.934 time: 0.3443 s/iter data_time: 0.2216 s/iter total_throughput: 2974.01 samples/s lr: 1.14e-05 [09/23 14:41:18] lb.utils.events INFO: eta: 0:22:58 iteration: 366499/375342 consumed_samples: 375296000 total_loss: 2.934 time: 0.3443 s/iter data_time: 0.2266 s/iter total_throughput: 2973.99 samples/s lr: 1.14e-05 [09/23 14:41:52] lb.utils.events INFO: eta: 0:22:45 iteration: 366599/375342 consumed_samples: 375398400 total_loss: 2.933 time: 0.3443 s/iter data_time: 0.2237 s/iter total_throughput: 2974.00 samples/s lr: 1.13e-05 [09/23 14:42:26] lb.utils.events INFO: eta: 0:22:29 iteration: 366699/375342 consumed_samples: 375500800 total_loss: 2.939 time: 0.3443 s/iter data_time: 0.2147 s/iter total_throughput: 2974.00 samples/s lr: 1.13e-05 [09/23 14:43:01] lb.utils.events INFO: eta: 0:22:13 iteration: 366799/375342 consumed_samples: 375603200 total_loss: 2.943 time: 0.3443 s/iter data_time: 0.2065 s/iter total_throughput: 2974.01 samples/s lr: 1.13e-05 [09/23 14:43:35] lb.utils.events INFO: eta: 0:21:58 iteration: 366899/375342 consumed_samples: 375705600 total_loss: 2.939 time: 0.3443 s/iter data_time: 0.1991 s/iter total_throughput: 2974.02 samples/s lr: 1.12e-05 [09/23 14:44:10] lb.utils.events INFO: eta: 0:21:42 iteration: 366999/375342 consumed_samples: 375808000 total_loss: 2.929 time: 0.3443 s/iter data_time: 0.2189 s/iter total_throughput: 2974.01 samples/s lr: 1.12e-05 [09/23 14:44:44] lb.utils.events INFO: eta: 0:21:26 iteration: 367099/375342 consumed_samples: 375910400 total_loss: 2.927 time: 0.3443 s/iter data_time: 0.2191 s/iter total_throughput: 2974.00 samples/s lr: 1.12e-05 [09/23 14:45:19] lb.utils.events INFO: eta: 0:21:16 iteration: 367199/375342 consumed_samples: 376012800 total_loss: 2.944 time: 0.3443 s/iter data_time: 0.2161 s/iter total_throughput: 2974.00 samples/s lr: 1.11e-05 [09/23 14:45:54] lb.utils.events INFO: eta: 0:21:01 iteration: 367299/375342 consumed_samples: 376115200 total_loss: 2.941 time: 0.3443 s/iter data_time: 0.2212 s/iter total_throughput: 2973.98 samples/s lr: 1.11e-05 [09/23 14:46:29] lb.utils.events INFO: eta: 0:20:47 iteration: 367399/375342 consumed_samples: 376217600 total_loss: 2.933 time: 0.3443 s/iter data_time: 0.2130 s/iter total_throughput: 2973.97 samples/s lr: 1.11e-05 [09/23 14:47:03] lb.utils.events INFO: eta: 0:20:34 iteration: 367499/375342 consumed_samples: 376320000 total_loss: 2.922 time: 0.3443 s/iter data_time: 0.2241 s/iter total_throughput: 2973.97 samples/s lr: 1.11e-05 [09/23 14:47:38] lb.utils.events INFO: eta: 0:20:16 iteration: 367599/375342 consumed_samples: 376422400 total_loss: 2.899 time: 0.3443 s/iter data_time: 0.2204 s/iter total_throughput: 2973.96 samples/s lr: 1.10e-05 [09/23 14:48:13] lb.utils.events INFO: eta: 0:19:59 iteration: 367699/375342 consumed_samples: 376524800 total_loss: 2.902 time: 0.3443 s/iter data_time: 0.2331 s/iter total_throughput: 2973.96 samples/s lr: 1.10e-05 [09/23 14:48:47] lb.utils.events INFO: eta: 0:19:43 iteration: 367799/375342 consumed_samples: 376627200 total_loss: 2.912 time: 0.3443 s/iter data_time: 0.2158 s/iter total_throughput: 2973.96 samples/s lr: 1.10e-05 [09/23 14:49:22] lb.utils.events INFO: eta: 0:19:26 iteration: 367899/375342 consumed_samples: 376729600 total_loss: 2.93 time: 0.3443 s/iter data_time: 0.2154 s/iter total_throughput: 2973.95 samples/s lr: 1.10e-05 [09/23 14:49:56] lb.utils.events INFO: eta: 0:19:13 iteration: 367999/375342 consumed_samples: 376832000 total_loss: 2.932 time: 0.3443 s/iter data_time: 0.2173 s/iter total_throughput: 2973.95 samples/s lr: 1.09e-05 [09/23 14:50:31] lb.utils.events INFO: eta: 0:18:56 iteration: 368099/375342 consumed_samples: 376934400 total_loss: 2.931 time: 0.3443 s/iter data_time: 0.2236 s/iter total_throughput: 2973.95 samples/s lr: 1.09e-05 [09/23 14:51:06] lb.utils.events INFO: eta: 0:18:36 iteration: 368199/375342 consumed_samples: 377036800 total_loss: 2.943 time: 0.3443 s/iter data_time: 0.2201 s/iter total_throughput: 2973.94 samples/s lr: 1.09e-05 [09/23 14:51:40] lb.utils.events INFO: eta: 0:18:23 iteration: 368299/375342 consumed_samples: 377139200 total_loss: 2.941 time: 0.3443 s/iter data_time: 0.2238 s/iter total_throughput: 2973.94 samples/s lr: 1.09e-05 [09/23 14:52:15] lb.utils.events INFO: eta: 0:18:05 iteration: 368399/375342 consumed_samples: 377241600 total_loss: 2.932 time: 0.3443 s/iter data_time: 0.2267 s/iter total_throughput: 2973.94 samples/s lr: 1.08e-05 [09/23 14:52:50] lb.utils.events INFO: eta: 0:17:48 iteration: 368499/375342 consumed_samples: 377344000 total_loss: 2.92 time: 0.3443 s/iter data_time: 0.2158 s/iter total_throughput: 2973.93 samples/s lr: 1.08e-05 [09/23 14:53:25] lb.utils.events INFO: eta: 0:17:31 iteration: 368599/375342 consumed_samples: 377446400 total_loss: 2.914 time: 0.3443 s/iter data_time: 0.2332 s/iter total_throughput: 2973.92 samples/s lr: 1.08e-05 [09/23 14:53:59] lb.utils.events INFO: eta: 0:17:14 iteration: 368699/375342 consumed_samples: 377548800 total_loss: 2.912 time: 0.3443 s/iter data_time: 0.2161 s/iter total_throughput: 2973.91 samples/s lr: 1.08e-05 [09/23 14:54:34] lb.utils.events INFO: eta: 0:17:00 iteration: 368799/375342 consumed_samples: 377651200 total_loss: 2.923 time: 0.3443 s/iter data_time: 0.2204 s/iter total_throughput: 2973.91 samples/s lr: 1.07e-05 [09/23 14:55:09] lb.utils.events INFO: eta: 0:16:43 iteration: 368899/375342 consumed_samples: 377753600 total_loss: 2.933 time: 0.3443 s/iter data_time: 0.2158 s/iter total_throughput: 2973.90 samples/s lr: 1.07e-05 [09/23 14:55:43] lb.utils.events INFO: eta: 0:16:28 iteration: 368999/375342 consumed_samples: 377856000 total_loss: 2.934 time: 0.3443 s/iter data_time: 0.2178 s/iter total_throughput: 2973.90 samples/s lr: 1.07e-05 [09/23 14:56:18] lb.utils.events INFO: eta: 0:16:11 iteration: 369099/375342 consumed_samples: 377958400 total_loss: 2.905 time: 0.3443 s/iter data_time: 0.2169 s/iter total_throughput: 2973.89 samples/s lr: 1.07e-05 [09/23 14:56:52] lb.utils.events INFO: eta: 0:15:56 iteration: 369199/375342 consumed_samples: 378060800 total_loss: 2.91 time: 0.3443 s/iter data_time: 0.2194 s/iter total_throughput: 2973.89 samples/s lr: 1.07e-05 [09/23 14:57:27] lb.utils.events INFO: eta: 0:15:39 iteration: 369299/375342 consumed_samples: 378163200 total_loss: 2.932 time: 0.3443 s/iter data_time: 0.2141 s/iter total_throughput: 2973.89 samples/s lr: 1.06e-05 [09/23 14:58:01] lb.utils.events INFO: eta: 0:15:23 iteration: 369399/375342 consumed_samples: 378265600 total_loss: 2.932 time: 0.3443 s/iter data_time: 0.2231 s/iter total_throughput: 2973.89 samples/s lr: 1.06e-05 [09/23 14:58:36] lb.utils.events INFO: eta: 0:15:07 iteration: 369499/375342 consumed_samples: 378368000 total_loss: 2.922 time: 0.3443 s/iter data_time: 0.2264 s/iter total_throughput: 2973.88 samples/s lr: 1.06e-05 [09/23 14:59:11] lb.utils.events INFO: eta: 0:14:52 iteration: 369599/375342 consumed_samples: 378470400 total_loss: 2.927 time: 0.3443 s/iter data_time: 0.2140 s/iter total_throughput: 2973.88 samples/s lr: 1.06e-05 [09/23 14:59:45] lb.utils.events INFO: eta: 0:14:38 iteration: 369699/375342 consumed_samples: 378572800 total_loss: 2.937 time: 0.3443 s/iter data_time: 0.2317 s/iter total_throughput: 2973.88 samples/s lr: 1.06e-05 [09/23 15:00:19] lb.utils.events INFO: eta: 0:14:22 iteration: 369799/375342 consumed_samples: 378675200 total_loss: 2.933 time: 0.3443 s/iter data_time: 0.2114 s/iter total_throughput: 2973.88 samples/s lr: 1.05e-05 [09/23 15:00:54] lb.utils.events INFO: eta: 0:14:07 iteration: 369899/375342 consumed_samples: 378777600 total_loss: 2.922 time: 0.3443 s/iter data_time: 0.2131 s/iter total_throughput: 2973.87 samples/s lr: 1.05e-05 [09/23 15:01:29] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0369999 [09/23 15:01:30] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/23 15:01:30] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/23 15:01:34] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.1031 s/iter. Inference: 0.1624 s/iter. Eval: 0.0021 s/iter. Total: 0.2677 s/iter. ETA=0:00:09 [09/23 15:01:39] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1263 s/iter. Inference: 0.1691 s/iter. Eval: 0.0020 s/iter. Total: 0.2975 s/iter. ETA=0:00:05 [09/23 15:01:44] lb.evaluation.evaluator INFO: Inference done 48128/50000. Dataloading: 0.1223 s/iter. Inference: 0.1651 s/iter. Eval: 0.0020 s/iter. Total: 0.2895 s/iter. ETA=0:00:00 [09/23 15:01:45] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/23 15:01:45] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.562763 (0.000251 s / iter per device, on 8 devices) [09/23 15:01:45] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000146 s / iter per device, on 8 devices) [09/23 15:01:45] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/23 15:01:45] lb.evaluation.utils INFO: copypaste: Acc@1=80.098 [09/23 15:01:45] lb.evaluation.utils INFO: copypaste: Acc@5=94.46 [09/23 15:01:45] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 80.09800, better than last best score 80.04800 @ iteration 359999. [09/23 15:01:45] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/23 15:01:45] lb.utils.events INFO: eta: 0:13:50 iteration: 369999/375342 consumed_samples: 378880000 total_loss: 2.918 time: 0.3443 s/iter data_time: 0.2354 s/iter total_throughput: 2973.87 samples/s lr: 1.05e-05 [09/23 15:02:18] lb.utils.events INFO: eta: 0:13:39 iteration: 370099/375342 consumed_samples: 378982400 total_loss: 2.932 time: 0.3443 s/iter data_time: 0.2078 s/iter total_throughput: 2973.92 samples/s lr: 1.05e-05 [09/23 15:02:53] lb.utils.events INFO: eta: 0:13:24 iteration: 370199/375342 consumed_samples: 379084800 total_loss: 2.929 time: 0.3443 s/iter data_time: 0.2209 s/iter total_throughput: 2973.91 samples/s lr: 1.05e-05 [09/23 15:03:27] lb.utils.events INFO: eta: 0:13:09 iteration: 370299/375342 consumed_samples: 379187200 total_loss: 2.918 time: 0.3443 s/iter data_time: 0.2207 s/iter total_throughput: 2973.91 samples/s lr: 1.04e-05 [09/23 15:04:02] lb.utils.events INFO: eta: 0:12:54 iteration: 370399/375342 consumed_samples: 379289600 total_loss: 2.926 time: 0.3443 s/iter data_time: 0.2306 s/iter total_throughput: 2973.90 samples/s lr: 1.04e-05 [09/23 15:04:37] lb.utils.events INFO: eta: 0:12:39 iteration: 370499/375342 consumed_samples: 379392000 total_loss: 2.93 time: 0.3443 s/iter data_time: 0.2169 s/iter total_throughput: 2973.89 samples/s lr: 1.04e-05 [09/23 15:05:12] lb.utils.events INFO: eta: 0:12:23 iteration: 370599/375342 consumed_samples: 379494400 total_loss: 2.922 time: 0.3443 s/iter data_time: 0.2235 s/iter total_throughput: 2973.88 samples/s lr: 1.04e-05 [09/23 15:05:46] lb.utils.events INFO: eta: 0:12:05 iteration: 370699/375342 consumed_samples: 379596800 total_loss: 2.927 time: 0.3443 s/iter data_time: 0.2380 s/iter total_throughput: 2973.87 samples/s lr: 1.04e-05 [09/23 15:06:21] lb.utils.events INFO: eta: 0:11:49 iteration: 370799/375342 consumed_samples: 379699200 total_loss: 2.94 time: 0.3443 s/iter data_time: 0.2199 s/iter total_throughput: 2973.86 samples/s lr: 1.04e-05 [09/23 15:06:56] lb.utils.events INFO: eta: 0:11:34 iteration: 370899/375342 consumed_samples: 379801600 total_loss: 2.924 time: 0.3443 s/iter data_time: 0.2184 s/iter total_throughput: 2973.86 samples/s lr: 1.03e-05 [09/23 15:07:31] lb.utils.events INFO: eta: 0:11:18 iteration: 370999/375342 consumed_samples: 379904000 total_loss: 2.921 time: 0.3443 s/iter data_time: 0.2350 s/iter total_throughput: 2973.85 samples/s lr: 1.03e-05 [09/23 15:08:06] lb.utils.events INFO: eta: 0:10:59 iteration: 371099/375342 consumed_samples: 380006400 total_loss: 2.927 time: 0.3443 s/iter data_time: 0.2270 s/iter total_throughput: 2973.84 samples/s lr: 1.03e-05 [09/23 15:08:40] lb.utils.events INFO: eta: 0:10:43 iteration: 371199/375342 consumed_samples: 380108800 total_loss: 2.939 time: 0.3443 s/iter data_time: 0.2253 s/iter total_throughput: 2973.84 samples/s lr: 1.03e-05 [09/23 15:09:15] lb.utils.events INFO: eta: 0:10:27 iteration: 371299/375342 consumed_samples: 380211200 total_loss: 2.938 time: 0.3443 s/iter data_time: 0.2228 s/iter total_throughput: 2973.83 samples/s lr: 1.03e-05 [09/23 15:09:49] lb.utils.events INFO: eta: 0:10:11 iteration: 371399/375342 consumed_samples: 380313600 total_loss: 2.938 time: 0.3443 s/iter data_time: 0.2111 s/iter total_throughput: 2973.83 samples/s lr: 1.03e-05 [09/23 15:10:24] lb.utils.events INFO: eta: 0:09:56 iteration: 371499/375342 consumed_samples: 380416000 total_loss: 2.938 time: 0.3443 s/iter data_time: 0.2145 s/iter total_throughput: 2973.82 samples/s lr: 1.03e-05 [09/23 15:10:59] lb.utils.events INFO: eta: 0:09:42 iteration: 371599/375342 consumed_samples: 380518400 total_loss: 2.928 time: 0.3443 s/iter data_time: 0.2174 s/iter total_throughput: 2973.82 samples/s lr: 1.02e-05 [09/23 15:11:34] lb.utils.events INFO: eta: 0:09:26 iteration: 371699/375342 consumed_samples: 380620800 total_loss: 2.932 time: 0.3443 s/iter data_time: 0.2313 s/iter total_throughput: 2973.80 samples/s lr: 1.02e-05 [09/23 15:12:08] lb.utils.events INFO: eta: 0:09:11 iteration: 371799/375342 consumed_samples: 380723200 total_loss: 2.949 time: 0.3443 s/iter data_time: 0.2100 s/iter total_throughput: 2973.80 samples/s lr: 1.02e-05 [09/23 15:12:43] lb.utils.events INFO: eta: 0:08:56 iteration: 371899/375342 consumed_samples: 380825600 total_loss: 2.937 time: 0.3443 s/iter data_time: 0.2129 s/iter total_throughput: 2973.80 samples/s lr: 1.02e-05 [09/23 15:13:18] lb.utils.events INFO: eta: 0:08:40 iteration: 371999/375342 consumed_samples: 380928000 total_loss: 2.921 time: 0.3443 s/iter data_time: 0.2275 s/iter total_throughput: 2973.79 samples/s lr: 1.02e-05 [09/23 15:13:52] lb.utils.events INFO: eta: 0:08:24 iteration: 372099/375342 consumed_samples: 381030400 total_loss: 2.931 time: 0.3443 s/iter data_time: 0.2222 s/iter total_throughput: 2973.78 samples/s lr: 1.02e-05 [09/23 15:14:27] lb.utils.events INFO: eta: 0:08:09 iteration: 372199/375342 consumed_samples: 381132800 total_loss: 2.942 time: 0.3443 s/iter data_time: 0.2307 s/iter total_throughput: 2973.77 samples/s lr: 1.02e-05 [09/23 15:15:02] lb.utils.events INFO: eta: 0:07:54 iteration: 372299/375342 consumed_samples: 381235200 total_loss: 2.938 time: 0.3443 s/iter data_time: 0.2116 s/iter total_throughput: 2973.77 samples/s lr: 1.02e-05 [09/23 15:15:36] lb.utils.events INFO: eta: 0:07:42 iteration: 372399/375342 consumed_samples: 381337600 total_loss: 2.904 time: 0.3443 s/iter data_time: 0.2155 s/iter total_throughput: 2973.78 samples/s lr: 1.02e-05 [09/23 15:16:10] lb.utils.events INFO: eta: 0:07:24 iteration: 372499/375342 consumed_samples: 381440000 total_loss: 2.928 time: 0.3443 s/iter data_time: 0.2184 s/iter total_throughput: 2973.79 samples/s lr: 1.01e-05 [09/23 15:16:45] lb.utils.events INFO: eta: 0:07:08 iteration: 372599/375342 consumed_samples: 381542400 total_loss: 2.93 time: 0.3443 s/iter data_time: 0.2281 s/iter total_throughput: 2973.78 samples/s lr: 1.01e-05 [09/23 15:17:19] lb.utils.events INFO: eta: 0:06:52 iteration: 372699/375342 consumed_samples: 381644800 total_loss: 2.912 time: 0.3443 s/iter data_time: 0.2070 s/iter total_throughput: 2973.78 samples/s lr: 1.01e-05 [09/23 15:17:54] lb.utils.events INFO: eta: 0:06:37 iteration: 372799/375342 consumed_samples: 381747200 total_loss: 2.916 time: 0.3443 s/iter data_time: 0.2160 s/iter total_throughput: 2973.78 samples/s lr: 1.01e-05 [09/23 15:18:28] lb.utils.events INFO: eta: 0:06:21 iteration: 372899/375342 consumed_samples: 381849600 total_loss: 2.932 time: 0.3443 s/iter data_time: 0.2190 s/iter total_throughput: 2973.78 samples/s lr: 1.01e-05 [09/23 15:19:02] lb.utils.events INFO: eta: 0:06:05 iteration: 372999/375342 consumed_samples: 381952000 total_loss: 2.918 time: 0.3443 s/iter data_time: 0.2391 s/iter total_throughput: 2973.78 samples/s lr: 1.01e-05 [09/23 15:19:37] lb.utils.events INFO: eta: 0:05:49 iteration: 373099/375342 consumed_samples: 382054400 total_loss: 2.918 time: 0.3443 s/iter data_time: 0.2084 s/iter total_throughput: 2973.78 samples/s lr: 1.01e-05 [09/23 15:20:11] lb.utils.events INFO: eta: 0:05:35 iteration: 373199/375342 consumed_samples: 382156800 total_loss: 2.927 time: 0.3443 s/iter data_time: 0.2170 s/iter total_throughput: 2973.80 samples/s lr: 1.01e-05 [09/23 15:20:45] lb.utils.events INFO: eta: 0:05:19 iteration: 373299/375342 consumed_samples: 382259200 total_loss: 2.919 time: 0.3443 s/iter data_time: 0.2266 s/iter total_throughput: 2973.79 samples/s lr: 1.01e-05 [09/23 15:21:20] lb.utils.events INFO: eta: 0:05:02 iteration: 373399/375342 consumed_samples: 382361600 total_loss: 2.935 time: 0.3443 s/iter data_time: 0.2248 s/iter total_throughput: 2973.79 samples/s lr: 1.01e-05 [09/23 15:21:54] lb.utils.events INFO: eta: 0:04:48 iteration: 373499/375342 consumed_samples: 382464000 total_loss: 2.933 time: 0.3443 s/iter data_time: 0.2047 s/iter total_throughput: 2973.80 samples/s lr: 1.01e-05 [09/23 15:22:28] lb.utils.events INFO: eta: 0:04:33 iteration: 373599/375342 consumed_samples: 382566400 total_loss: 2.918 time: 0.3443 s/iter data_time: 0.2113 s/iter total_throughput: 2973.81 samples/s lr: 1.01e-05 [09/23 15:23:02] lb.utils.events INFO: eta: 0:04:17 iteration: 373699/375342 consumed_samples: 382668800 total_loss: 2.913 time: 0.3443 s/iter data_time: 0.2126 s/iter total_throughput: 2973.81 samples/s lr: 1.00e-05 [09/23 15:23:36] lb.utils.events INFO: eta: 0:04:02 iteration: 373799/375342 consumed_samples: 382771200 total_loss: 2.931 time: 0.3443 s/iter data_time: 0.2161 s/iter total_throughput: 2973.83 samples/s lr: 1.00e-05 [09/23 15:24:10] lb.utils.events INFO: eta: 0:03:46 iteration: 373899/375342 consumed_samples: 382873600 total_loss: 2.931 time: 0.3443 s/iter data_time: 0.2278 s/iter total_throughput: 2973.83 samples/s lr: 1.00e-05 [09/23 15:24:45] lb.utils.events INFO: eta: 0:03:30 iteration: 373999/375342 consumed_samples: 382976000 total_loss: 2.929 time: 0.3443 s/iter data_time: 0.2229 s/iter total_throughput: 2973.82 samples/s lr: 1.00e-05 [09/23 15:25:20] lb.utils.events INFO: eta: 0:03:15 iteration: 374099/375342 consumed_samples: 383078400 total_loss: 2.941 time: 0.3443 s/iter data_time: 0.2145 s/iter total_throughput: 2973.82 samples/s lr: 1.00e-05 [09/23 15:25:54] lb.utils.events INFO: eta: 0:02:59 iteration: 374199/375342 consumed_samples: 383180800 total_loss: 2.927 time: 0.3443 s/iter data_time: 0.2173 s/iter total_throughput: 2973.82 samples/s lr: 1.00e-05 [09/23 15:26:28] lb.utils.events INFO: eta: 0:02:43 iteration: 374299/375342 consumed_samples: 383283200 total_loss: 2.916 time: 0.3443 s/iter data_time: 0.2217 s/iter total_throughput: 2973.82 samples/s lr: 1.00e-05 [09/23 15:27:03] lb.utils.events INFO: eta: 0:02:27 iteration: 374399/375342 consumed_samples: 383385600 total_loss: 2.914 time: 0.3443 s/iter data_time: 0.2095 s/iter total_throughput: 2973.82 samples/s lr: 1.00e-05 [09/23 15:27:38] lb.utils.events INFO: eta: 0:02:11 iteration: 374499/375342 consumed_samples: 383488000 total_loss: 2.921 time: 0.3443 s/iter data_time: 0.2131 s/iter total_throughput: 2973.81 samples/s lr: 1.00e-05 [09/23 15:28:12] lb.utils.events INFO: eta: 0:01:56 iteration: 374599/375342 consumed_samples: 383590400 total_loss: 2.906 time: 0.3443 s/iter data_time: 0.2232 s/iter total_throughput: 2973.82 samples/s lr: 1.00e-05 [09/23 15:28:46] lb.utils.events INFO: eta: 0:01:40 iteration: 374699/375342 consumed_samples: 383692800 total_loss: 2.916 time: 0.3443 s/iter data_time: 0.2211 s/iter total_throughput: 2973.82 samples/s lr: 1.00e-05 [09/23 15:29:21] lb.utils.events INFO: eta: 0:01:24 iteration: 374799/375342 consumed_samples: 383795200 total_loss: 2.918 time: 0.3443 s/iter data_time: 0.2204 s/iter total_throughput: 2973.82 samples/s lr: 1.00e-05 [09/23 15:29:55] lb.utils.events INFO: eta: 0:01:09 iteration: 374899/375342 consumed_samples: 383897600 total_loss: 2.915 time: 0.3443 s/iter data_time: 0.2083 s/iter total_throughput: 2973.82 samples/s lr: 1.00e-05 [09/23 15:30:29] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_0374999 [09/23 15:30:30] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/23 15:30:30] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/23 15:30:34] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.1160 s/iter. Inference: 0.1623 s/iter. Eval: 0.0020 s/iter. Total: 0.2804 s/iter. ETA=0:00:10 [09/23 15:30:40] lb.evaluation.evaluator INFO: Inference done 29696/50000. Dataloading: 0.1319 s/iter. Inference: 0.1645 s/iter. Eval: 0.0020 s/iter. Total: 0.2984 s/iter. ETA=0:00:05 [09/23 15:30:45] lb.evaluation.evaluator INFO: Inference done 48128/50000. Dataloading: 0.1291 s/iter. Inference: 0.1628 s/iter. Eval: 0.0020 s/iter. Total: 0.2939 s/iter. ETA=0:00:00 [09/23 15:30:45] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/23 15:30:45] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.692443 (0.000254 s / iter per device, on 8 devices) [09/23 15:30:45] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000144 s / iter per device, on 8 devices) [09/23 15:30:45] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/23 15:30:45] lb.evaluation.utils INFO: copypaste: Acc@1=80.10000000000001 [09/23 15:30:45] lb.evaluation.utils INFO: copypaste: Acc@5=94.47 [09/23 15:30:45] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 80.10000, better than last best score 80.09800 @ iteration 369999. [09/23 15:30:45] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best [09/23 15:30:46] lb.utils.events INFO: eta: 0:00:53 iteration: 374999/375342 consumed_samples: 384000000 total_loss: 2.935 time: 0.3443 s/iter data_time: 0.2229 s/iter total_throughput: 2973.83 samples/s lr: 1.00e-05 [09/23 15:31:19] lb.utils.events INFO: eta: 0:00:37 iteration: 375099/375342 consumed_samples: 384102400 total_loss: 2.914 time: 0.3443 s/iter data_time: 0.2244 s/iter total_throughput: 2973.87 samples/s lr: 1.00e-05 [09/23 15:31:53] lb.utils.events INFO: eta: 0:00:22 iteration: 375199/375342 consumed_samples: 384204800 total_loss: 2.898 time: 0.3443 s/iter data_time: 0.2140 s/iter total_throughput: 2973.87 samples/s lr: 1.00e-05 [09/23 15:32:27] lb.utils.events INFO: eta: 0:00:06 iteration: 375299/375342 consumed_samples: 384307200 total_loss: 2.924 time: 0.3443 s/iter data_time: 0.2054 s/iter total_throughput: 2973.88 samples/s lr: 1.00e-05 [09/23 15:32:41] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_final [09/23 15:32:42] lb.utils.events INFO: eta: 0:00:00 iteration: 375341/375342 consumed_samples: 384350208 total_loss: 2.932 time: 0.3443 s/iter data_time: 0.2227 s/iter total_throughput: 2973.89 samples/s lr: 1.00e-05 [09/23 15:32:42] lb.engine.hooks INFO: Overall training speed: 375340 iterations in 1 day, 11:54:05 (0.3443 s / it) [09/23 15:32:42] lb.engine.hooks INFO: Total training time: 1 day, 12:15:14 (0:21:09 on hooks) [09/23 15:32:42] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000 [09/23 15:32:42] lb.evaluation.evaluator INFO: Start inference on 50000 samples [09/23 15:32:46] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.1042 s/iter. Inference: 0.1617 s/iter. Eval: 0.0020 s/iter. Total: 0.2679 s/iter. ETA=0:00:09 [09/23 15:32:52] lb.evaluation.evaluator INFO: Inference done 28672/50000. Dataloading: 0.1235 s/iter. Inference: 0.1638 s/iter. Eval: 0.0021 s/iter. Total: 0.2895 s/iter. ETA=0:00:05 [09/23 15:32:57] lb.evaluation.evaluator INFO: Inference done 47104/50000. Dataloading: 0.1279 s/iter. Inference: 0.1623 s/iter. Eval: 0.0021 s/iter. Total: 0.2924 s/iter. ETA=0:00:00 [09/23 15:32:57] lb.evaluation.evaluator INFO: Total valid samples: 50000 [09/23 15:32:57] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.498202 (0.000250 s / iter per device, on 8 devices) [09/23 15:32:57] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:07 (0.000143 s / iter per device, on 8 devices) [09/23 15:32:57] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format: [09/23 15:32:57] lb.evaluation.utils INFO: copypaste: Acc@1=80.166 [09/23 15:32:57] lb.evaluation.utils INFO: copypaste: Acc@5=94.536 [09/23 15:32:57] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 80.16600, better than last best score 80.10000 @ iteration 374999. [09/23 15:32:57] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_swin/model_best