[09/15 13:30:31] libai INFO: Rank of current process: 0. World size: 8
[09/15 13:30:31] libai INFO: Command line arguments: Namespace(config_file='configs/swin_imagenet.py', eval_only=False, fast_dev_run=False, opts=[], resume=False)
[09/15 13:30:31] libai INFO: Contents of args.config_file=configs/swin_imagenet.py:
from libai.config import LazyCall
from .common.models.swin.swin_tiny_patch4_window7_224 import model
from .common.models.graph import graph
from .common.train import train
from .common.optim import optim
from .common.data.imagenet import dataloader

from flowvision.data import Mixup
from flowvision.loss.cross_entropy import SoftTargetCrossEntropy

# Refine data path to imagenet
dataloader.train.dataset[0].root = "/data/ImageNet/extract/"
dataloader.test[0].dataset.root ="/data/ImageNet/extract/"

# Add Mixup Func
dataloader.train.mixup_func = LazyCall(Mixup)(
    mixup_alpha=0.8,
    cutmix_alpha=1.0,
    prob=1.0,
    switch_prob=0.5,
    mode="batch",
    num_classes=1000,
)

# Refine model cfg for vit training on imagenet
model.num_classes = 1000
model.loss_func = LazyCall(SoftTargetCrossEntropy)()

# Refine optimizer cfg for vit model
optim.lr = 1e-3
optim.eps = 1e-8
optim.weight_decay = 0.05

# Refine train cfg for vit model
train.train_micro_batch_size = 128
train.test_micro_batch_size = 128
train.train_epoch = 300
train.warmup_ratio = 20 / 300
train.eval_period = 1562
train.log_period = 100
train.output_dir = "./commit_2e56"
# Scheduler
train.scheduler.warmup_factor = 0.001
train.scheduler.alpha = 0.01
train.scheduler.warmup_method = "linear"
graph.enabled = True
train.rdma_enabled = True
# Set fp16 ON
train.amp.enabled = True

[09/15 13:30:31] libai INFO: Full config saved to ./commit_2e56/config.yaml
[09/15 13:30:31] lb.engine.default INFO: > compiling dataset index builder ...
[09/15 13:30:31] lb.engine.default INFO: >>> done with dataset index builder. Compilation time: 0.071 seconds
[09/15 13:30:31] lb.engine.default INFO: >>> done with compiling. Compilation time: 0.072 seconds
[09/15 13:30:31] lb.engine.default INFO: Prepare training, validating, testing set
[09/15 13:30:34] lb.engine.default INFO: Prepare testing set
[09/15 13:30:34] lb.engine.default INFO: Auto-scaling the config to train.train_iter=375342, train.warmup_iter=25023
[09/15 13:30:42] lb.engine.default INFO: Model:
SwinTransformer(
  (patch_embed): PatchEmbed(
    (proj): Conv2d(3, 96, kernel_size=(4, 4), stride=(4, 4))
    (norm): LayerNorm((96,), eps=1e-05, elementwise_affine=True)
  )
  (pos_drop): Dropout(p=0.0, inplace=False)
  (layers): ModuleList(
    (0): BasicLayer(
      (blocks): ModuleList(
        (0): SwinTransformerBlock(
          (norm1): LayerNorm((96,), eps=1e-05, elementwise_affine=True)
          (attn): WindowAttention(
            (qkv): Linear1D(in_features=96, out_features=288, bias=True, parallel=data)
            (attn_drop): Dropout(p=0.0, inplace=False)
            (proj): Linear1D(in_features=96, out_features=96, bias=True, parallel=data)
            (proj_drop): Dropout(p=0.0, inplace=False)
            (softmax): Softmax(dim=-1)
          )
          (drop_path): Identity()
          (norm2): LayerNorm((96,), eps=1e-05, elementwise_affine=True)
          (mlp): MLP(
            bias_gelu_fusion=True, bias_dropout_fusion=True, dropout=0.0
            (dense_h_to_4h): Linear1D(in_features=96, out_features=384, bias=True, parallel=col)
            (dense_4h_to_h): Linear1D(in_features=384, out_features=96, bias=True, parallel=row)
          )
        )
        (1): SwinTransformerBlock(
          (norm1): LayerNorm((96,), eps=1e-05, elementwise_affine=True)
          (attn): WindowAttention(
            (qkv): Linear1D(in_features=96, out_features=288, bias=True, parallel=data)
            (attn_drop): Dropout(p=0.0, inplace=False)
            (proj): Linear1D(in_features=96, out_features=96, bias=True, parallel=data)
            (proj_drop): Dropout(p=0.0, inplace=False)
            (softmax): Softmax(dim=-1)
          )
          (drop_path): DropPath()
          (norm2): LayerNorm((96,), eps=1e-05, elementwise_affine=True)
          (mlp): MLP(
            bias_gelu_fusion=True, bias_dropout_fusion=True, dropout=0.0
            (dense_h_to_4h): Linear1D(in_features=96, out_features=384, bias=True, parallel=col)
            (dense_4h_to_h): Linear1D(in_features=384, out_features=96, bias=True, parallel=row)
          )
        )
      )
      (downsample): PatchMerging(
        (reduction): Linear1D(in_features=384, out_features=192, bias=False, parallel=data)
        (norm): LayerNorm((384,), eps=1e-05, elementwise_affine=True)
      )
    )
    (1): BasicLayer(
      (blocks): ModuleList(
        (0): SwinTransformerBlock(
          (norm1): LayerNorm((192,), eps=1e-05, elementwise_affine=True)
          (attn): WindowAttention(
            (qkv): Linear1D(in_features=192, out_features=576, bias=True, parallel=data)
            (attn_drop): Dropout(p=0.0, inplace=False)
            (proj): Linear1D(in_features=192, out_features=192, bias=True, parallel=data)
            (proj_drop): Dropout(p=0.0, inplace=False)
            (softmax): Softmax(dim=-1)
          )
          (drop_path): DropPath()
          (norm2): LayerNorm((192,), eps=1e-05, elementwise_affine=True)
          (mlp): MLP(
            bias_gelu_fusion=True, bias_dropout_fusion=True, dropout=0.0
            (dense_h_to_4h): Linear1D(in_features=192, out_features=768, bias=True, parallel=col)
            (dense_4h_to_h): Linear1D(in_features=768, out_features=192, bias=True, parallel=row)
          )
        )
        (1): SwinTransformerBlock(
          (norm1): LayerNorm((192,), eps=1e-05, elementwise_affine=True)
          (attn): WindowAttention(
            (qkv): Linear1D(in_features=192, out_features=576, bias=True, parallel=data)
            (attn_drop): Dropout(p=0.0, inplace=False)
            (proj): Linear1D(in_features=192, out_features=192, bias=True, parallel=data)
            (proj_drop): Dropout(p=0.0, inplace=False)
            (softmax): Softmax(dim=-1)
          )
          (drop_path): DropPath()
          (norm2): LayerNorm((192,), eps=1e-05, elementwise_affine=True)
          (mlp): MLP(
            bias_gelu_fusion=True, bias_dropout_fusion=True, dropout=0.0
            (dense_h_to_4h): Linear1D(in_features=192, out_features=768, bias=True, parallel=col)
            (dense_4h_to_h): Linear1D(in_features=768, out_features=192, bias=True, parallel=row)
          )
        )
      )
      (downsample): PatchMerging(
        (reduction): Linear1D(in_features=768, out_features=384, bias=False, parallel=data)
        (norm): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
      )
    )
    (2): BasicLayer(
      (blocks): ModuleList(
        (0): SwinTransformerBlock(
          (norm1): LayerNorm((384,), eps=1e-05, elementwise_affine=True)
          (attn): WindowAttention(
            (qkv): Linear1D(in_features=384, out_features=1152, bias=True, parallel=data)
            (attn_drop): Dropout(p=0.0, inplace=False)
            (proj): Linear1D(in_features=384, out_features=384, bias=True, parallel=data)
            (proj_drop): Dropout(p=0.0, inplace=False)
            (softmax): Softmax(dim=-1)
          )
          (drop_path): DropPath()
          (norm2): LayerNorm((384,), eps=1e-05, elementwise_affine=True)
          (mlp): MLP(
            bias_gelu_fusion=True, bias_dropout_fusion=True, dropout=0.0
            (dense_h_to_4h): Linear1D(in_features=384, out_features=1536, bias=True, parallel=col)
            (dense_4h_to_h): Linear1D(in_features=1536, out_features=384, bias=True, parallel=row)
          )
        )
        (1): SwinTransformerBlock(
          (norm1): LayerNorm((384,), eps=1e-05, elementwise_affine=True)
          (attn): WindowAttention(
            (qkv): Linear1D(in_features=384, out_features=1152, bias=True, parallel=data)
            (attn_drop): Dropout(p=0.0, inplace=False)
            (proj): Linear1D(in_features=384, out_features=384, bias=True, parallel=data)
            (proj_drop): Dropout(p=0.0, inplace=False)
            (softmax): Softmax(dim=-1)
          )
          (drop_path): DropPath()
          (norm2): LayerNorm((384,), eps=1e-05, elementwise_affine=True)
          (mlp): MLP(
            bias_gelu_fusion=True, bias_dropout_fusion=True, dropout=0.0
            (dense_h_to_4h): Linear1D(in_features=384, out_features=1536, bias=True, parallel=col)
            (dense_4h_to_h): Linear1D(in_features=1536, out_features=384, bias=True, parallel=row)
          )
        )
        (2): SwinTransformerBlock(
          (norm1): LayerNorm((384,), eps=1e-05, elementwise_affine=True)
          (attn): WindowAttention(
            (qkv): Linear1D(in_features=384, out_features=1152, bias=True, parallel=data)
            (attn_drop): Dropout(p=0.0, inplace=False)
            (proj): Linear1D(in_features=384, out_features=384, bias=True, parallel=data)
            (proj_drop): Dropout(p=0.0, inplace=False)
            (softmax): Softmax(dim=-1)
          )
          (drop_path): DropPath()
          (norm2): LayerNorm((384,), eps=1e-05, elementwise_affine=True)
          (mlp): MLP(
            bias_gelu_fusion=True, bias_dropout_fusion=True, dropout=0.0
            (dense_h_to_4h): Linear1D(in_features=384, out_features=1536, bias=True, parallel=col)
            (dense_4h_to_h): Linear1D(in_features=1536, out_features=384, bias=True, parallel=row)
          )
        )
        (3): SwinTransformerBlock(
          (norm1): LayerNorm((384,), eps=1e-05, elementwise_affine=True)
          (attn): WindowAttention(
            (qkv): Linear1D(in_features=384, out_features=1152, bias=True, parallel=data)
            (attn_drop): Dropout(p=0.0, inplace=False)
            (proj): Linear1D(in_features=384, out_features=384, bias=True, parallel=data)
            (proj_drop): Dropout(p=0.0, inplace=False)
            (softmax): Softmax(dim=-1)
          )
          (drop_path): DropPath()
          (norm2): LayerNorm((384,), eps=1e-05, elementwise_affine=True)
          (mlp): MLP(
            bias_gelu_fusion=True, bias_dropout_fusion=True, dropout=0.0
            (dense_h_to_4h): Linear1D(in_features=384, out_features=1536, bias=True, parallel=col)
            (dense_4h_to_h): Linear1D(in_features=1536, out_features=384, bias=True, parallel=row)
          )
        )
        (4): SwinTransformerBlock(
          (norm1): LayerNorm((384,), eps=1e-05, elementwise_affine=True)
          (attn): WindowAttention(
            (qkv): Linear1D(in_features=384, out_features=1152, bias=True, parallel=data)
            (attn_drop): Dropout(p=0.0, inplace=False)
            (proj): Linear1D(in_features=384, out_features=384, bias=True, parallel=data)
            (proj_drop): Dropout(p=0.0, inplace=False)
            (softmax): Softmax(dim=-1)
          )
          (drop_path): DropPath()
          (norm2): LayerNorm((384,), eps=1e-05, elementwise_affine=True)
          (mlp): MLP(
            bias_gelu_fusion=True, bias_dropout_fusion=True, dropout=0.0
            (dense_h_to_4h): Linear1D(in_features=384, out_features=1536, bias=True, parallel=col)
            (dense_4h_to_h): Linear1D(in_features=1536, out_features=384, bias=True, parallel=row)
          )
        )
        (5): SwinTransformerBlock(
          (norm1): LayerNorm((384,), eps=1e-05, elementwise_affine=True)
          (attn): WindowAttention(
            (qkv): Linear1D(in_features=384, out_features=1152, bias=True, parallel=data)
            (attn_drop): Dropout(p=0.0, inplace=False)
            (proj): Linear1D(in_features=384, out_features=384, bias=True, parallel=data)
            (proj_drop): Dropout(p=0.0, inplace=False)
            (softmax): Softmax(dim=-1)
          )
          (drop_path): DropPath()
          (norm2): LayerNorm((384,), eps=1e-05, elementwise_affine=True)
          (mlp): MLP(
            bias_gelu_fusion=True, bias_dropout_fusion=True, dropout=0.0
            (dense_h_to_4h): Linear1D(in_features=384, out_features=1536, bias=True, parallel=col)
            (dense_4h_to_h): Linear1D(in_features=1536, out_features=384, bias=True, parallel=row)
          )
        )
      )
      (downsample): PatchMerging(
        (reduction): Linear1D(in_features=1536, out_features=768, bias=False, parallel=data)
        (norm): LayerNorm((1536,), eps=1e-05, elementwise_affine=True)
      )
    )
    (3): BasicLayer(
      (blocks): ModuleList(
        (0): SwinTransformerBlock(
          (norm1): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
          (attn): WindowAttention(
            (qkv): Linear1D(in_features=768, out_features=2304, bias=True, parallel=data)
            (attn_drop): Dropout(p=0.0, inplace=False)
            (proj): Linear1D(in_features=768, out_features=768, bias=True, parallel=data)
            (proj_drop): Dropout(p=0.0, inplace=False)
            (softmax): Softmax(dim=-1)
          )
          (drop_path): DropPath()
          (norm2): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
          (mlp): MLP(
            bias_gelu_fusion=True, bias_dropout_fusion=True, dropout=0.0
            (dense_h_to_4h): Linear1D(in_features=768, out_features=3072, bias=True, parallel=col)
            (dense_4h_to_h): Linear1D(in_features=3072, out_features=768, bias=True, parallel=row)
          )
        )
        (1): SwinTransformerBlock(
          (norm1): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
          (attn): WindowAttention(
            (qkv): Linear1D(in_features=768, out_features=2304, bias=True, parallel=data)
            (attn_drop): Dropout(p=0.0, inplace=False)
            (proj): Linear1D(in_features=768, out_features=768, bias=True, parallel=data)
            (proj_drop): Dropout(p=0.0, inplace=False)
            (softmax): Softmax(dim=-1)
          )
          (drop_path): DropPath()
          (norm2): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
          (mlp): MLP(
            bias_gelu_fusion=True, bias_dropout_fusion=True, dropout=0.0
            (dense_h_to_4h): Linear1D(in_features=768, out_features=3072, bias=True, parallel=col)
            (dense_4h_to_h): Linear1D(in_features=3072, out_features=768, bias=True, parallel=row)
          )
        )
      )
    )
  )
  (norm): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
  (avgpool): AdaptiveAvgPool1d()
  (head): Linear1D(in_features=768, out_features=1000, bias=True, parallel=data)
  (loss_func): SoftTargetCrossEntropy()
)
[09/15 13:30:42] lb.engine.trainer INFO: Starting training from iteration 0
[09/15 13:30:44] lb.models.utils.graph_base INFO: Start compling the train graph which may take some time. Please wait for a moment ...
[09/15 13:31:48] lb.utils.events INFO:  eta: 17:27:14  iteration: 99/375342  consumed_samples: 102400  total_loss: 6.964  time: 0.3816 s/iter  data_time: 0.2585 s/iter total_throughput: 2683.29 samples/s lr: 4.91e-06  
[09/15 13:32:27] lb.utils.events INFO:  eta: 20:29:27  iteration: 199/375342  consumed_samples: 204800  total_loss: 6.939  time: 0.3868 s/iter  data_time: 0.2440 s/iter total_throughput: 2647.03 samples/s lr: 8.86e-06  
[09/15 13:33:07] lb.utils.events INFO:  eta: 22:47:54  iteration: 299/375342  consumed_samples: 307200  total_loss: 6.907  time: 0.3900 s/iter  data_time: 0.2584 s/iter total_throughput: 2625.94 samples/s lr: 1.28e-05  
[09/15 13:33:47] lb.utils.events INFO:  eta: 23:56:59  iteration: 399/375342  consumed_samples: 409600  total_loss: 6.886  time: 0.3925 s/iter  data_time: 0.2467 s/iter total_throughput: 2608.83 samples/s lr: 1.68e-05  
[09/15 13:34:27] lb.utils.events INFO:  eta: 21:55:27  iteration: 499/375342  consumed_samples: 512000  total_loss: 6.864  time: 0.3933 s/iter  data_time: 0.2501 s/iter total_throughput: 2603.33 samples/s lr: 2.07e-05  
[09/15 13:35:06] lb.utils.events INFO:  eta: 21:04:44  iteration: 599/375342  consumed_samples: 614400  total_loss: 6.845  time: 0.3937 s/iter  data_time: 0.2531 s/iter total_throughput: 2600.94 samples/s lr: 2.47e-05  
[09/15 13:35:45] lb.utils.events INFO:  eta: 18:31:36  iteration: 699/375342  consumed_samples: 716800  total_loss: 6.825  time: 0.3933 s/iter  data_time: 0.2578 s/iter total_throughput: 2603.31 samples/s lr: 2.86e-05  
[09/15 13:36:25] lb.utils.events INFO:  eta: 17:03:01  iteration: 799/375342  consumed_samples: 819200  total_loss: 6.799  time: 0.3933 s/iter  data_time: 0.2430 s/iter total_throughput: 2603.57 samples/s lr: 3.26e-05  
[09/15 13:37:05] lb.utils.events INFO:  eta: 16:18:32  iteration: 899/375342  consumed_samples: 921600  total_loss: 6.778  time: 0.3940 s/iter  data_time: 0.2592 s/iter total_throughput: 2598.98 samples/s lr: 3.65e-05  
[09/15 13:37:44] lb.utils.events INFO:  eta: 16:14:26  iteration: 999/375342  consumed_samples: 1024000  total_loss: 6.753  time: 0.3941 s/iter  data_time: 0.2404 s/iter total_throughput: 2598.48 samples/s lr: 4.05e-05  
[09/15 13:38:23] lb.utils.events INFO:  eta: 16:14:10  iteration: 1099/375342  consumed_samples: 1126400  total_loss: 6.731  time: 0.3939 s/iter  data_time: 0.2461 s/iter total_throughput: 2599.42 samples/s lr: 4.44e-05  
[09/15 13:39:03] lb.utils.events INFO:  eta: 16:06:11  iteration: 1199/375342  consumed_samples: 1228800  total_loss: 6.693  time: 0.3940 s/iter  data_time: 0.2528 s/iter total_throughput: 2598.91 samples/s lr: 4.83e-05  
[09/15 13:39:43] lb.utils.events INFO:  eta: 16:02:19  iteration: 1299/375342  consumed_samples: 1331200  total_loss: 6.649  time: 0.3946 s/iter  data_time: 0.2521 s/iter total_throughput: 2595.05 samples/s lr: 5.23e-05  
[09/15 13:40:23] lb.utils.events INFO:  eta: 15:58:08  iteration: 1399/375342  consumed_samples: 1433600  total_loss: 6.617  time: 0.3946 s/iter  data_time: 0.2453 s/iter total_throughput: 2595.16 samples/s lr: 5.62e-05  
[09/15 13:41:03] lb.utils.events INFO:  eta: 15:59:30  iteration: 1499/375342  consumed_samples: 1536000  total_loss: 6.583  time: 0.3950 s/iter  data_time: 0.2602 s/iter total_throughput: 2592.53 samples/s lr: 6.02e-05  
[09/15 13:41:43] lb.utils.events INFO:  eta: 15:56:10  iteration: 1599/375342  consumed_samples: 1638400  total_loss: 6.552  time: 0.3952 s/iter  data_time: 0.2579 s/iter total_throughput: 2590.78 samples/s lr: 6.41e-05  
[09/15 13:42:22] lb.utils.events INFO:  eta: 15:53:50  iteration: 1699/375342  consumed_samples: 1740800  total_loss: 6.523  time: 0.3954 s/iter  data_time: 0.2602 s/iter total_throughput: 2589.84 samples/s lr: 6.81e-05  
[09/15 13:43:02] lb.utils.events INFO:  eta: 15:50:15  iteration: 1799/375342  consumed_samples: 1843200  total_loss: 6.507  time: 0.3955 s/iter  data_time: 0.2464 s/iter total_throughput: 2588.96 samples/s lr: 7.20e-05  
[09/15 13:43:42] lb.utils.events INFO:  eta: 15:48:24  iteration: 1899/375342  consumed_samples: 1945600  total_loss: 6.472  time: 0.3955 s/iter  data_time: 0.2474 s/iter total_throughput: 2589.12 samples/s lr: 7.60e-05  
[09/15 13:44:21] lb.utils.events INFO:  eta: 15:46:45  iteration: 1999/375342  consumed_samples: 2048000  total_loss: 6.441  time: 0.3956 s/iter  data_time: 0.2459 s/iter total_throughput: 2588.54 samples/s lr: 7.99e-05  
[09/15 13:45:01] lb.utils.events INFO:  eta: 15:44:06  iteration: 2099/375342  consumed_samples: 2150400  total_loss: 6.418  time: 0.3957 s/iter  data_time: 0.2530 s/iter total_throughput: 2587.79 samples/s lr: 8.39e-05  
[09/15 13:45:41] lb.utils.events INFO:  eta: 15:42:05  iteration: 2199/375342  consumed_samples: 2252800  total_loss: 6.389  time: 0.3959 s/iter  data_time: 0.2426 s/iter total_throughput: 2586.40 samples/s lr: 8.78e-05  
[09/15 13:46:21] lb.utils.events INFO:  eta: 15:40:21  iteration: 2299/375342  consumed_samples: 2355200  total_loss: 6.35  time: 0.3960 s/iter  data_time: 0.2465 s/iter total_throughput: 2585.77 samples/s lr: 9.18e-05  
[09/15 13:47:01] lb.utils.events INFO:  eta: 15:40:23  iteration: 2399/375342  consumed_samples: 2457600  total_loss: 6.324  time: 0.3960 s/iter  data_time: 0.2461 s/iter total_throughput: 2585.60 samples/s lr: 9.57e-05  
[09/15 13:47:41] lb.utils.events INFO:  eta: 15:39:45  iteration: 2499/375342  consumed_samples: 2560000  total_loss: 6.31  time: 0.3962 s/iter  data_time: 0.2510 s/iter total_throughput: 2584.73 samples/s lr: 9.97e-05  
[09/15 13:48:20] lb.utils.events INFO:  eta: 15:39:35  iteration: 2599/375342  consumed_samples: 2662400  total_loss: 6.277  time: 0.3961 s/iter  data_time: 0.2497 s/iter total_throughput: 2585.42 samples/s lr: 1.04e-04  
[09/15 13:49:00] lb.utils.events INFO:  eta: 15:40:46  iteration: 2699/375342  consumed_samples: 2764800  total_loss: 6.252  time: 0.3961 s/iter  data_time: 0.2452 s/iter total_throughput: 2585.07 samples/s lr: 1.08e-04  
[09/15 13:49:39] lb.utils.events INFO:  eta: 15:42:49  iteration: 2799/375342  consumed_samples: 2867200  total_loss: 6.223  time: 0.3960 s/iter  data_time: 0.2456 s/iter total_throughput: 2585.78 samples/s lr: 1.12e-04  
[09/15 13:50:19] lb.utils.events INFO:  eta: 15:45:38  iteration: 2899/375342  consumed_samples: 2969600  total_loss: 6.198  time: 0.3959 s/iter  data_time: 0.2438 s/iter total_throughput: 2586.26 samples/s lr: 1.15e-04  
[09/15 13:50:58] lb.utils.events INFO:  eta: 15:48:49  iteration: 2999/375342  consumed_samples: 3072000  total_loss: 6.175  time: 0.3958 s/iter  data_time: 0.2448 s/iter total_throughput: 2586.90 samples/s lr: 1.19e-04  
[09/15 13:51:38] lb.utils.events INFO:  eta: 15:50:53  iteration: 3099/375342  consumed_samples: 3174400  total_loss: 6.145  time: 0.3959 s/iter  data_time: 0.2334 s/iter total_throughput: 2586.30 samples/s lr: 1.23e-04  
[09/15 13:52:17] lb.utils.events INFO:  eta: 15:51:11  iteration: 3199/375342  consumed_samples: 3276800  total_loss: 6.12  time: 0.3959 s/iter  data_time: 0.2413 s/iter total_throughput: 2586.60 samples/s lr: 1.27e-04  
[09/15 13:52:57] lb.utils.events INFO:  eta: 15:51:27  iteration: 3299/375342  consumed_samples: 3379200  total_loss: 6.097  time: 0.3958 s/iter  data_time: 0.2366 s/iter total_throughput: 2587.13 samples/s lr: 1.31e-04  
[09/15 13:53:37] lb.utils.events INFO:  eta: 15:52:33  iteration: 3399/375342  consumed_samples: 3481600  total_loss: 6.08  time: 0.3960 s/iter  data_time: 0.2688 s/iter total_throughput: 2585.90 samples/s lr: 1.35e-04  
[09/15 13:54:17] lb.utils.events INFO:  eta: 15:53:55  iteration: 3499/375342  consumed_samples: 3584000  total_loss: 6.039  time: 0.3962 s/iter  data_time: 0.2525 s/iter total_throughput: 2584.52 samples/s lr: 1.39e-04  
[09/15 13:54:57] lb.utils.events INFO:  eta: 15:54:52  iteration: 3599/375342  consumed_samples: 3686400  total_loss: 6.011  time: 0.3962 s/iter  data_time: 0.2544 s/iter total_throughput: 2584.24 samples/s lr: 1.43e-04  
[09/15 13:55:37] lb.utils.events INFO:  eta: 15:55:15  iteration: 3699/375342  consumed_samples: 3788800  total_loss: 5.984  time: 0.3963 s/iter  data_time: 0.2517 s/iter total_throughput: 2583.86 samples/s lr: 1.47e-04  
[09/15 13:56:16] lb.utils.events INFO:  eta: 15:56:15  iteration: 3799/375342  consumed_samples: 3891200  total_loss: 5.968  time: 0.3963 s/iter  data_time: 0.2574 s/iter total_throughput: 2583.83 samples/s lr: 1.51e-04  
[09/15 13:56:56] lb.utils.events INFO:  eta: 15:56:22  iteration: 3899/375342  consumed_samples: 3993600  total_loss: 5.94  time: 0.3963 s/iter  data_time: 0.2440 s/iter total_throughput: 2584.02 samples/s lr: 1.55e-04  
[09/15 13:57:35] lb.utils.events INFO:  eta: 15:54:17  iteration: 3999/375342  consumed_samples: 4096000  total_loss: 5.924  time: 0.3962 s/iter  data_time: 0.2552 s/iter total_throughput: 2584.29 samples/s lr: 1.59e-04  
[09/15 13:58:15] lb.utils.events INFO:  eta: 15:57:20  iteration: 4099/375342  consumed_samples: 4198400  total_loss: 5.904  time: 0.3962 s/iter  data_time: 0.2484 s/iter total_throughput: 2584.87 samples/s lr: 1.63e-04  
[09/15 13:58:55] lb.utils.events INFO:  eta: 16:01:21  iteration: 4199/375342  consumed_samples: 4300800  total_loss: 5.875  time: 0.3963 s/iter  data_time: 0.2544 s/iter total_throughput: 2584.09 samples/s lr: 1.67e-04  
[09/15 13:59:35] lb.utils.events INFO:  eta: 16:08:34  iteration: 4299/375342  consumed_samples: 4403200  total_loss: 5.847  time: 0.3963 s/iter  data_time: 0.2563 s/iter total_throughput: 2583.84 samples/s lr: 1.71e-04  
[09/15 14:00:14] lb.utils.events INFO:  eta: 16:02:49  iteration: 4399/375342  consumed_samples: 4505600  total_loss: 5.823  time: 0.3963 s/iter  data_time: 0.2511 s/iter total_throughput: 2583.90 samples/s lr: 1.75e-04  
[09/15 14:00:53] lb.utils.events INFO:  eta: 16:01:33  iteration: 4499/375342  consumed_samples: 4608000  total_loss: 5.799  time: 0.3962 s/iter  data_time: 0.2541 s/iter total_throughput: 2584.52 samples/s lr: 1.79e-04  
[09/15 14:01:33] lb.utils.events INFO:  eta: 16:00:58  iteration: 4599/375342  consumed_samples: 4710400  total_loss: 5.765  time: 0.3962 s/iter  data_time: 0.2417 s/iter total_throughput: 2584.74 samples/s lr: 1.83e-04  
[09/15 14:02:13] lb.utils.events INFO:  eta: 16:00:03  iteration: 4699/375342  consumed_samples: 4812800  total_loss: 5.744  time: 0.3963 s/iter  data_time: 0.2453 s/iter total_throughput: 2584.17 samples/s lr: 1.87e-04  
[09/15 14:02:53] lb.utils.events INFO:  eta: 15:59:34  iteration: 4799/375342  consumed_samples: 4915200  total_loss: 5.723  time: 0.3964 s/iter  data_time: 0.2523 s/iter total_throughput: 2583.57 samples/s lr: 1.91e-04  
[09/15 14:03:33] lb.utils.events INFO:  eta: 15:57:56  iteration: 4899/375342  consumed_samples: 5017600  total_loss: 5.699  time: 0.3964 s/iter  data_time: 0.2393 s/iter total_throughput: 2583.48 samples/s lr: 1.94e-04  
[09/15 14:04:13] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_2e56/model_0004999
[09/15 14:04:13] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000
[09/15 14:04:13] lb.evaluation.evaluator INFO: Start inference on 50000 samples
[09/15 14:04:15] lb.models.utils.graph_base INFO: Start compling the eval graph which may take some time. Please wait for a moment ...
[09/15 14:04:20] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.0263 s/iter. Inference: 0.1668 s/iter. Eval: 0.0022 s/iter. Total: 0.1953 s/iter. ETA=0:00:07
[09/15 14:04:25] lb.evaluation.evaluator INFO: Inference done 28672/50000. Dataloading: 0.0646 s/iter. Inference: 0.2016 s/iter. Eval: 0.0021 s/iter. Total: 0.2684 s/iter. ETA=0:00:05
[09/15 14:04:30] lb.evaluation.evaluator INFO: Inference done 45056/50000. Dataloading: 0.0417 s/iter. Inference: 0.2431 s/iter. Eval: 0.0021 s/iter. Total: 0.2870 s/iter. ETA=0:00:01
[09/15 14:04:31] lb.evaluation.evaluator INFO: Total valid samples: 50000
[09/15 14:04:31] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.543726 (0.000251 s / iter per device, on 8 devices)
[09/15 14:04:31] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:10 (0.000214 s / iter per device, on 8 devices)
[09/15 14:04:31] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format:
[09/15 14:04:31] lb.evaluation.utils INFO: copypaste: Acc@1=16.828000000000003
[09/15 14:04:31] lb.evaluation.utils INFO: copypaste: Acc@5=36.736000000000004
[09/15 14:04:31] lb.engine.hooks INFO: Saved first model at 16.82800 @ 4999 steps
[09/15 14:04:31] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_2e56/model_best
[09/15 14:04:32] lb.utils.events INFO:  eta: 15:55:14  iteration: 4999/375342  consumed_samples: 5120000  total_loss: 5.685  time: 0.3964 s/iter  data_time: 0.2496 s/iter total_throughput: 2583.18 samples/s lr: 1.98e-04  
[09/15 14:05:10] lb.utils.events INFO:  eta: 15:55:24  iteration: 5099/375342  consumed_samples: 5222400  total_loss: 5.651  time: 0.3961 s/iter  data_time: 0.2535 s/iter total_throughput: 2585.27 samples/s lr: 2.02e-04  
[09/15 14:05:50] lb.utils.events INFO:  eta: 15:49:12  iteration: 5199/375342  consumed_samples: 5324800  total_loss: 5.631  time: 0.3962 s/iter  data_time: 0.2561 s/iter total_throughput: 2584.77 samples/s lr: 2.06e-04  
[09/15 14:06:30] lb.utils.events INFO:  eta: 15:44:42  iteration: 5299/375342  consumed_samples: 5427200  total_loss: 5.614  time: 0.3962 s/iter  data_time: 0.2536 s/iter total_throughput: 2584.69 samples/s lr: 2.10e-04  
[09/15 14:07:09] lb.utils.events INFO:  eta: 15:43:17  iteration: 5399/375342  consumed_samples: 5529600  total_loss: 5.595  time: 0.3961 s/iter  data_time: 0.2430 s/iter total_throughput: 2585.43 samples/s lr: 2.14e-04  
[09/15 14:07:48] lb.utils.events INFO:  eta: 15:43:28  iteration: 5499/375342  consumed_samples: 5632000  total_loss: 5.585  time: 0.3960 s/iter  data_time: 0.2436 s/iter total_throughput: 2585.70 samples/s lr: 2.18e-04  
[09/15 14:08:28] lb.utils.events INFO:  eta: 15:46:33  iteration: 5599/375342  consumed_samples: 5734400  total_loss: 5.56  time: 0.3960 s/iter  data_time: 0.2526 s/iter total_throughput: 2585.63 samples/s lr: 2.22e-04  
[09/15 14:09:08] lb.utils.events INFO:  eta: 15:48:27  iteration: 5699/375342  consumed_samples: 5836800  total_loss: 5.532  time: 0.3961 s/iter  data_time: 0.2480 s/iter total_throughput: 2585.12 samples/s lr: 2.26e-04  
[09/15 14:09:47] lb.utils.events INFO:  eta: 15:46:08  iteration: 5799/375342  consumed_samples: 5939200  total_loss: 5.522  time: 0.3961 s/iter  data_time: 0.2486 s/iter total_throughput: 2585.43 samples/s lr: 2.30e-04  
[09/15 14:10:27] lb.utils.events INFO:  eta: 15:45:31  iteration: 5899/375342  consumed_samples: 6041600  total_loss: 5.503  time: 0.3960 s/iter  data_time: 0.2508 s/iter total_throughput: 2585.76 samples/s lr: 2.34e-04  
[09/15 14:11:07] lb.utils.events INFO:  eta: 15:48:39  iteration: 5999/375342  consumed_samples: 6144000  total_loss: 5.473  time: 0.3961 s/iter  data_time: 0.2515 s/iter total_throughput: 2585.31 samples/s lr: 2.38e-04  
[09/15 14:11:47] lb.utils.events INFO:  eta: 15:46:48  iteration: 6099/375342  consumed_samples: 6246400  total_loss: 5.465  time: 0.3961 s/iter  data_time: 0.2535 s/iter total_throughput: 2584.99 samples/s lr: 2.42e-04  
[09/15 14:12:26] lb.utils.events INFO:  eta: 15:51:01  iteration: 6199/375342  consumed_samples: 6348800  total_loss: 5.457  time: 0.3962 s/iter  data_time: 0.2385 s/iter total_throughput: 2584.83 samples/s lr: 2.46e-04  
[09/15 14:13:06] lb.utils.events INFO:  eta: 15:53:01  iteration: 6299/375342  consumed_samples: 6451200  total_loss: 5.428  time: 0.3961 s/iter  data_time: 0.2358 s/iter total_throughput: 2585.07 samples/s lr: 2.50e-04  
[09/15 14:13:45] lb.utils.events INFO:  eta: 16:00:05  iteration: 6399/375342  consumed_samples: 6553600  total_loss: 5.397  time: 0.3961 s/iter  data_time: 0.2556 s/iter total_throughput: 2584.91 samples/s lr: 2.54e-04  
[09/15 14:14:25] lb.utils.events INFO:  eta: 15:55:10  iteration: 6499/375342  consumed_samples: 6656000  total_loss: 5.386  time: 0.3961 s/iter  data_time: 0.2479 s/iter total_throughput: 2584.90 samples/s lr: 2.58e-04  
[09/15 14:15:05] lb.utils.events INFO:  eta: 15:52:18  iteration: 6599/375342  consumed_samples: 6758400  total_loss: 5.378  time: 0.3961 s/iter  data_time: 0.2465 s/iter total_throughput: 2584.89 samples/s lr: 2.62e-04  
[09/15 14:15:45] lb.utils.events INFO:  eta: 15:53:58  iteration: 6699/375342  consumed_samples: 6860800  total_loss: 5.354  time: 0.3962 s/iter  data_time: 0.2579 s/iter total_throughput: 2584.49 samples/s lr: 2.66e-04  
[09/15 14:16:24] lb.utils.events INFO:  eta: 15:59:28  iteration: 6799/375342  consumed_samples: 6963200  total_loss: 5.314  time: 0.3962 s/iter  data_time: 0.2489 s/iter total_throughput: 2584.52 samples/s lr: 2.69e-04  
[09/15 14:17:04] lb.utils.events INFO:  eta: 16:01:22  iteration: 6899/375342  consumed_samples: 7065600  total_loss: 5.313  time: 0.3963 s/iter  data_time: 0.2497 s/iter total_throughput: 2584.11 samples/s lr: 2.73e-04  
[09/15 14:17:44] lb.utils.events INFO:  eta: 15:56:11  iteration: 6999/375342  consumed_samples: 7168000  total_loss: 5.318  time: 0.3962 s/iter  data_time: 0.2434 s/iter total_throughput: 2584.23 samples/s lr: 2.77e-04  
[09/15 14:18:24] lb.utils.events INFO:  eta: 15:57:15  iteration: 7099/375342  consumed_samples: 7270400  total_loss: 5.302  time: 0.3963 s/iter  data_time: 0.2638 s/iter total_throughput: 2584.20 samples/s lr: 2.81e-04  
[09/15 14:19:03] lb.utils.events INFO:  eta: 15:58:44  iteration: 7199/375342  consumed_samples: 7372800  total_loss: 5.29  time: 0.3962 s/iter  data_time: 0.2515 s/iter total_throughput: 2584.55 samples/s lr: 2.85e-04  
[09/15 14:19:43] lb.utils.events INFO:  eta: 16:02:05  iteration: 7299/375342  consumed_samples: 7475200  total_loss: 5.269  time: 0.3962 s/iter  data_time: 0.2411 s/iter total_throughput: 2584.40 samples/s lr: 2.89e-04  
[09/15 14:20:22] lb.utils.events INFO:  eta: 16:00:39  iteration: 7399/375342  consumed_samples: 7577600  total_loss: 5.249  time: 0.3962 s/iter  data_time: 0.2402 s/iter total_throughput: 2584.47 samples/s lr: 2.93e-04  
[09/15 14:21:02] lb.utils.events INFO:  eta: 16:01:33  iteration: 7499/375342  consumed_samples: 7680000  total_loss: 5.228  time: 0.3963 s/iter  data_time: 0.2646 s/iter total_throughput: 2584.19 samples/s lr: 2.97e-04  
[09/15 14:21:42] lb.utils.events INFO:  eta: 15:57:20  iteration: 7599/375342  consumed_samples: 7782400  total_loss: 5.227  time: 0.3963 s/iter  data_time: 0.2384 s/iter total_throughput: 2584.14 samples/s lr: 3.01e-04  
[09/15 14:22:22] lb.utils.events INFO:  eta: 15:54:36  iteration: 7699/375342  consumed_samples: 7884800  total_loss: 5.211  time: 0.3963 s/iter  data_time: 0.2649 s/iter total_throughput: 2584.09 samples/s lr: 3.05e-04  
[09/15 14:23:01] lb.utils.events INFO:  eta: 15:50:40  iteration: 7799/375342  consumed_samples: 7987200  total_loss: 5.187  time: 0.3963 s/iter  data_time: 0.2488 s/iter total_throughput: 2584.22 samples/s lr: 3.09e-04  
[09/15 14:23:41] lb.utils.events INFO:  eta: 15:49:20  iteration: 7899/375342  consumed_samples: 8089600  total_loss: 5.183  time: 0.3963 s/iter  data_time: 0.2464 s/iter total_throughput: 2584.17 samples/s lr: 3.13e-04  
[09/15 14:24:20] lb.utils.events INFO:  eta: 15:49:33  iteration: 7999/375342  consumed_samples: 8192000  total_loss: 5.195  time: 0.3962 s/iter  data_time: 0.2494 s/iter total_throughput: 2584.45 samples/s lr: 3.17e-04  
[09/15 14:25:00] lb.utils.events INFO:  eta: 15:46:58  iteration: 8099/375342  consumed_samples: 8294400  total_loss: 5.149  time: 0.3963 s/iter  data_time: 0.2437 s/iter total_throughput: 2583.82 samples/s lr: 3.21e-04  
[09/15 14:25:40] lb.utils.events INFO:  eta: 15:44:36  iteration: 8199/375342  consumed_samples: 8396800  total_loss: 5.158  time: 0.3963 s/iter  data_time: 0.2502 s/iter total_throughput: 2583.75 samples/s lr: 3.25e-04  
[09/15 14:26:20] lb.utils.events INFO:  eta: 15:43:56  iteration: 8299/375342  consumed_samples: 8499200  total_loss: 5.153  time: 0.3963 s/iter  data_time: 0.2446 s/iter total_throughput: 2583.97 samples/s lr: 3.29e-04  
[09/15 14:27:00] lb.utils.events INFO:  eta: 15:42:51  iteration: 8399/375342  consumed_samples: 8601600  total_loss: 5.13  time: 0.3964 s/iter  data_time: 0.2529 s/iter total_throughput: 2583.57 samples/s lr: 3.33e-04  
[09/15 14:27:40] lb.utils.events INFO:  eta: 15:41:18  iteration: 8499/375342  consumed_samples: 8704000  total_loss: 5.114  time: 0.3964 s/iter  data_time: 0.2529 s/iter total_throughput: 2583.37 samples/s lr: 3.37e-04  
[09/15 14:28:19] lb.utils.events INFO:  eta: 15:41:02  iteration: 8599/375342  consumed_samples: 8806400  total_loss: 5.102  time: 0.3964 s/iter  data_time: 0.2443 s/iter total_throughput: 2583.47 samples/s lr: 3.41e-04  
[09/15 14:28:59] lb.utils.events INFO:  eta: 15:41:06  iteration: 8699/375342  consumed_samples: 8908800  total_loss: 5.069  time: 0.3964 s/iter  data_time: 0.2509 s/iter total_throughput: 2583.54 samples/s lr: 3.45e-04  
[09/15 14:29:38] lb.utils.events INFO:  eta: 15:41:51  iteration: 8799/375342  consumed_samples: 9011200  total_loss: 5.066  time: 0.3964 s/iter  data_time: 0.2470 s/iter total_throughput: 2583.45 samples/s lr: 3.48e-04  
[09/15 14:30:19] lb.utils.events INFO:  eta: 15:41:24  iteration: 8899/375342  consumed_samples: 9113600  total_loss: 5.059  time: 0.3965 s/iter  data_time: 0.2466 s/iter total_throughput: 2582.84 samples/s lr: 3.52e-04  
[09/15 14:30:59] lb.utils.events INFO:  eta: 15:40:50  iteration: 8999/375342  consumed_samples: 9216000  total_loss: 5.039  time: 0.3965 s/iter  data_time: 0.2506 s/iter total_throughput: 2582.76 samples/s lr: 3.56e-04  
[09/15 14:31:38] lb.utils.events INFO:  eta: 15:39:43  iteration: 9099/375342  consumed_samples: 9318400  total_loss: 5.039  time: 0.3965 s/iter  data_time: 0.2453 s/iter total_throughput: 2582.82 samples/s lr: 3.60e-04  
[09/15 14:32:18] lb.utils.events INFO:  eta: 15:38:49  iteration: 9199/375342  consumed_samples: 9420800  total_loss: 5.04  time: 0.3965 s/iter  data_time: 0.2520 s/iter total_throughput: 2582.86 samples/s lr: 3.64e-04  
[09/15 14:32:57] lb.utils.events INFO:  eta: 15:38:15  iteration: 9299/375342  consumed_samples: 9523200  total_loss: 5.029  time: 0.3965 s/iter  data_time: 0.2470 s/iter total_throughput: 2582.88 samples/s lr: 3.68e-04  
[09/15 14:33:37] lb.utils.events INFO:  eta: 15:37:50  iteration: 9399/375342  consumed_samples: 9625600  total_loss: 5.012  time: 0.3965 s/iter  data_time: 0.2552 s/iter total_throughput: 2582.83 samples/s lr: 3.72e-04  
[09/15 14:34:17] lb.utils.events INFO:  eta: 15:38:11  iteration: 9499/375342  consumed_samples: 9728000  total_loss: 4.99  time: 0.3965 s/iter  data_time: 0.2579 s/iter total_throughput: 2582.58 samples/s lr: 3.76e-04  
[09/15 14:34:57] lb.utils.events INFO:  eta: 15:37:45  iteration: 9599/375342  consumed_samples: 9830400  total_loss: 4.975  time: 0.3965 s/iter  data_time: 0.2488 s/iter total_throughput: 2582.38 samples/s lr: 3.80e-04  
[09/15 14:35:37] lb.utils.events INFO:  eta: 15:37:10  iteration: 9699/375342  consumed_samples: 9932800  total_loss: 4.972  time: 0.3966 s/iter  data_time: 0.2480 s/iter total_throughput: 2582.17 samples/s lr: 3.84e-04  
[09/15 14:36:17] lb.utils.events INFO:  eta: 15:36:20  iteration: 9799/375342  consumed_samples: 10035200  total_loss: 4.96  time: 0.3966 s/iter  data_time: 0.2463 s/iter total_throughput: 2582.22 samples/s lr: 3.88e-04  
[09/15 14:36:56] lb.utils.events INFO:  eta: 15:36:33  iteration: 9899/375342  consumed_samples: 10137600  total_loss: 4.937  time: 0.3965 s/iter  data_time: 0.2427 s/iter total_throughput: 2582.64 samples/s lr: 3.92e-04  
[09/15 14:37:36] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_2e56/model_0009999
[09/15 14:37:36] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000
[09/15 14:37:36] lb.evaluation.evaluator INFO: Start inference on 50000 samples
[09/15 14:37:41] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.1210 s/iter. Inference: 0.1496 s/iter. Eval: 0.0019 s/iter. Total: 0.2725 s/iter. ETA=0:00:10
[09/15 14:37:46] lb.evaluation.evaluator INFO: Inference done 27648/50000. Dataloading: 0.1516 s/iter. Inference: 0.1487 s/iter. Eval: 0.0021 s/iter. Total: 0.3025 s/iter. ETA=0:00:06
[09/15 14:37:51] lb.evaluation.evaluator INFO: Inference done 45056/50000. Dataloading: 0.1471 s/iter. Inference: 0.1495 s/iter. Eval: 0.0021 s/iter. Total: 0.2989 s/iter. ETA=0:00:01
[09/15 14:37:52] lb.evaluation.evaluator INFO: Total valid samples: 50000
[09/15 14:37:52] lb.evaluation.evaluator INFO: Total inference time: 0:00:13.126369 (0.000263 s / iter per device, on 8 devices)
[09/15 14:37:52] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000132 s / iter per device, on 8 devices)
[09/15 14:37:52] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format:
[09/15 14:37:52] lb.evaluation.utils INFO: copypaste: Acc@1=33.282000000000004
[09/15 14:37:52] lb.evaluation.utils INFO: copypaste: Acc@5=57.936
[09/15 14:37:52] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 33.28200, better than last best score 16.82800 @ iteration 4999.
[09/15 14:37:52] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_2e56/model_best
[09/15 14:37:53] lb.utils.events INFO:  eta: 15:37:45  iteration: 9999/375342  consumed_samples: 10240000  total_loss: 4.939  time: 0.3965 s/iter  data_time: 0.2530 s/iter total_throughput: 2582.49 samples/s lr: 3.96e-04  
[09/15 14:38:31] lb.utils.events INFO:  eta: 15:38:20  iteration: 10099/375342  consumed_samples: 10342400  total_loss: 4.916  time: 0.3964 s/iter  data_time: 0.2454 s/iter total_throughput: 2583.50 samples/s lr: 4.00e-04  
[09/15 14:39:11] lb.utils.events INFO:  eta: 15:38:40  iteration: 10199/375342  consumed_samples: 10444800  total_loss: 4.904  time: 0.3964 s/iter  data_time: 0.2419 s/iter total_throughput: 2583.30 samples/s lr: 4.04e-04  
[09/15 14:39:51] lb.utils.events INFO:  eta: 15:38:52  iteration: 10299/375342  consumed_samples: 10547200  total_loss: 4.895  time: 0.3964 s/iter  data_time: 0.2356 s/iter total_throughput: 2582.96 samples/s lr: 4.08e-04  
[09/15 14:40:31] lb.utils.events INFO:  eta: 15:39:07  iteration: 10399/375342  consumed_samples: 10649600  total_loss: 4.875  time: 0.3965 s/iter  data_time: 0.2525 s/iter total_throughput: 2582.81 samples/s lr: 4.12e-04  
[09/15 14:41:10] lb.utils.events INFO:  eta: 15:39:35  iteration: 10499/375342  consumed_samples: 10752000  total_loss: 4.881  time: 0.3964 s/iter  data_time: 0.2429 s/iter total_throughput: 2583.10 samples/s lr: 4.16e-04  
[09/15 14:41:50] lb.utils.events INFO:  eta: 15:40:02  iteration: 10599/375342  consumed_samples: 10854400  total_loss: 4.856  time: 0.3964 s/iter  data_time: 0.2431 s/iter total_throughput: 2583.20 samples/s lr: 4.20e-04  
[09/15 14:42:29] lb.utils.events INFO:  eta: 15:41:04  iteration: 10699/375342  consumed_samples: 10956800  total_loss: 4.838  time: 0.3964 s/iter  data_time: 0.2557 s/iter total_throughput: 2583.23 samples/s lr: 4.24e-04  
[09/15 14:43:09] lb.utils.events INFO:  eta: 15:42:41  iteration: 10799/375342  consumed_samples: 11059200  total_loss: 4.838  time: 0.3964 s/iter  data_time: 0.2431 s/iter total_throughput: 2583.30 samples/s lr: 4.27e-04  
[09/15 14:43:48] lb.utils.events INFO:  eta: 15:47:01  iteration: 10899/375342  consumed_samples: 11161600  total_loss: 4.845  time: 0.3963 s/iter  data_time: 0.2422 s/iter total_throughput: 2583.66 samples/s lr: 4.31e-04  
[09/15 14:44:28] lb.utils.events INFO:  eta: 15:48:20  iteration: 10999/375342  consumed_samples: 11264000  total_loss: 4.855  time: 0.3963 s/iter  data_time: 0.2500 s/iter total_throughput: 2583.58 samples/s lr: 4.35e-04  
[09/15 14:45:08] lb.utils.events INFO:  eta: 15:50:38  iteration: 11099/375342  consumed_samples: 11366400  total_loss: 4.832  time: 0.3964 s/iter  data_time: 0.2421 s/iter total_throughput: 2583.49 samples/s lr: 4.39e-04  
[09/15 14:45:47] lb.utils.events INFO:  eta: 15:58:51  iteration: 11199/375342  consumed_samples: 11468800  total_loss: 4.831  time: 0.3964 s/iter  data_time: 0.2525 s/iter total_throughput: 2583.47 samples/s lr: 4.43e-04  
[09/15 14:46:27] lb.utils.events INFO:  eta: 16:17:29  iteration: 11299/375342  consumed_samples: 11571200  total_loss: 4.809  time: 0.3964 s/iter  data_time: 0.2449 s/iter total_throughput: 2583.32 samples/s lr: 4.47e-04  
[09/15 14:47:07] lb.utils.events INFO:  eta: 17:10:35  iteration: 11399/375342  consumed_samples: 11673600  total_loss: 4.782  time: 0.3964 s/iter  data_time: 0.2529 s/iter total_throughput: 2583.20 samples/s lr: 4.51e-04  
[09/15 14:47:46] lb.utils.events INFO:  eta: 21:20:58  iteration: 11499/375342  consumed_samples: 11776000  total_loss: 4.782  time: 0.3964 s/iter  data_time: 0.2347 s/iter total_throughput: 2583.50 samples/s lr: 4.55e-04  
[09/15 14:48:26] lb.utils.events INFO:  eta: 22:52:05  iteration: 11599/375342  consumed_samples: 11878400  total_loss: 4.778  time: 0.3964 s/iter  data_time: 0.2494 s/iter total_throughput: 2583.30 samples/s lr: 4.59e-04  
[09/15 14:49:05] lb.utils.events INFO:  eta: 1 day, 0:49:17  iteration: 11699/375342  consumed_samples: 11980800  total_loss: 4.772  time: 0.3964 s/iter  data_time: 0.2423 s/iter total_throughput: 2583.55 samples/s lr: 4.63e-04  
[09/15 14:49:46] lb.utils.events INFO:  eta: 1 day, 2:12:09  iteration: 11799/375342  consumed_samples: 12083200  total_loss: 4.76  time: 0.3964 s/iter  data_time: 0.2609 s/iter total_throughput: 2583.28 samples/s lr: 4.67e-04  
[09/15 14:50:25] lb.utils.events INFO:  eta: 1 day, 3:01:35  iteration: 11899/375342  consumed_samples: 12185600  total_loss: 4.752  time: 0.3964 s/iter  data_time: 0.2553 s/iter total_throughput: 2583.12 samples/s lr: 4.71e-04  
[09/15 14:51:05] lb.utils.events INFO:  eta: 1 day, 4:30:49  iteration: 11999/375342  consumed_samples: 12288000  total_loss: 4.748  time: 0.3964 s/iter  data_time: 0.2564 s/iter total_throughput: 2583.06 samples/s lr: 4.75e-04  
[09/15 14:51:45] lb.utils.events INFO:  eta: 1 day, 2:43:50  iteration: 12099/375342  consumed_samples: 12390400  total_loss: 4.739  time: 0.3964 s/iter  data_time: 0.2476 s/iter total_throughput: 2583.02 samples/s lr: 4.79e-04  
[09/15 14:52:24] lb.utils.events INFO:  eta: 23:21:24  iteration: 12199/375342  consumed_samples: 12492800  total_loss: 4.707  time: 0.3964 s/iter  data_time: 0.2487 s/iter total_throughput: 2583.12 samples/s lr: 4.83e-04  
[09/15 14:53:04] lb.utils.events INFO:  eta: 20:47:14  iteration: 12299/375342  consumed_samples: 12595200  total_loss: 4.696  time: 0.3964 s/iter  data_time: 0.2494 s/iter total_throughput: 2583.04 samples/s lr: 4.87e-04  
[09/15 14:53:44] lb.utils.events INFO:  eta: 20:40:27  iteration: 12399/375342  consumed_samples: 12697600  total_loss: 4.713  time: 0.3964 s/iter  data_time: 0.2481 s/iter total_throughput: 2582.98 samples/s lr: 4.91e-04  
[09/15 14:54:24] lb.utils.events INFO:  eta: 21:16:09  iteration: 12499/375342  consumed_samples: 12800000  total_loss: 4.72  time: 0.3964 s/iter  data_time: 0.2527 s/iter total_throughput: 2582.98 samples/s lr: 4.95e-04  
[09/15 14:55:04] lb.utils.events INFO:  eta: 20:56:27  iteration: 12599/375342  consumed_samples: 12902400  total_loss: 4.713  time: 0.3965 s/iter  data_time: 0.2509 s/iter total_throughput: 2582.86 samples/s lr: 4.99e-04  
[09/15 14:55:43] lb.utils.events INFO:  eta: 19:25:52  iteration: 12699/375342  consumed_samples: 13004800  total_loss: 4.687  time: 0.3964 s/iter  data_time: 0.2352 s/iter total_throughput: 2582.93 samples/s lr: 5.02e-04  
[09/15 14:56:23] lb.utils.events INFO:  eta: 16:20:13  iteration: 12799/375342  consumed_samples: 13107200  total_loss: 4.677  time: 0.3965 s/iter  data_time: 0.2560 s/iter total_throughput: 2582.78 samples/s lr: 5.06e-04  
[09/15 14:57:03] lb.utils.events INFO:  eta: 15:50:06  iteration: 12899/375342  consumed_samples: 13209600  total_loss: 4.682  time: 0.3965 s/iter  data_time: 0.2524 s/iter total_throughput: 2582.55 samples/s lr: 5.10e-04  
[09/15 14:57:43] lb.utils.events INFO:  eta: 15:38:40  iteration: 12999/375342  consumed_samples: 13312000  total_loss: 4.704  time: 0.3965 s/iter  data_time: 0.2443 s/iter total_throughput: 2582.65 samples/s lr: 5.14e-04  
[09/15 14:58:22] lb.utils.events INFO:  eta: 15:42:56  iteration: 13099/375342  consumed_samples: 13414400  total_loss: 4.677  time: 0.3965 s/iter  data_time: 0.2459 s/iter total_throughput: 2582.71 samples/s lr: 5.18e-04  
[09/15 14:59:02] lb.utils.events INFO:  eta: 15:39:51  iteration: 13199/375342  consumed_samples: 13516800  total_loss: 4.636  time: 0.3965 s/iter  data_time: 0.2396 s/iter total_throughput: 2582.57 samples/s lr: 5.22e-04  
[09/15 14:59:42] lb.utils.events INFO:  eta: 15:41:08  iteration: 13299/375342  consumed_samples: 13619200  total_loss: 4.629  time: 0.3965 s/iter  data_time: 0.2608 s/iter total_throughput: 2582.34 samples/s lr: 5.26e-04  
[09/15 15:00:22] lb.utils.events INFO:  eta: 15:37:24  iteration: 13399/375342  consumed_samples: 13721600  total_loss: 4.629  time: 0.3966 s/iter  data_time: 0.2507 s/iter total_throughput: 2582.17 samples/s lr: 5.30e-04  
[09/15 15:01:02] lb.utils.events INFO:  eta: 15:34:17  iteration: 13499/375342  consumed_samples: 13824000  total_loss: 4.624  time: 0.3966 s/iter  data_time: 0.2415 s/iter total_throughput: 2582.27 samples/s lr: 5.34e-04  
[09/15 15:01:41] lb.utils.events INFO:  eta: 15:33:45  iteration: 13599/375342  consumed_samples: 13926400  total_loss: 4.602  time: 0.3965 s/iter  data_time: 0.2525 s/iter total_throughput: 2582.42 samples/s lr: 5.38e-04  
[09/15 15:02:21] lb.utils.events INFO:  eta: 15:32:47  iteration: 13699/375342  consumed_samples: 14028800  total_loss: 4.619  time: 0.3965 s/iter  data_time: 0.2423 s/iter total_throughput: 2582.50 samples/s lr: 5.42e-04  
[09/15 15:03:00] lb.utils.events INFO:  eta: 15:32:02  iteration: 13799/375342  consumed_samples: 14131200  total_loss: 4.598  time: 0.3965 s/iter  data_time: 0.2391 s/iter total_throughput: 2582.46 samples/s lr: 5.46e-04  
[09/15 15:03:40] lb.utils.events INFO:  eta: 15:31:01  iteration: 13899/375342  consumed_samples: 14233600  total_loss: 4.591  time: 0.3965 s/iter  data_time: 0.2459 s/iter total_throughput: 2582.43 samples/s lr: 5.50e-04  
[09/15 15:04:20] lb.utils.events INFO:  eta: 15:30:35  iteration: 13999/375342  consumed_samples: 14336000  total_loss: 4.575  time: 0.3966 s/iter  data_time: 0.2505 s/iter total_throughput: 2582.24 samples/s lr: 5.54e-04  
[09/15 15:05:00] lb.utils.events INFO:  eta: 15:29:51  iteration: 14099/375342  consumed_samples: 14438400  total_loss: 4.563  time: 0.3966 s/iter  data_time: 0.2535 s/iter total_throughput: 2582.03 samples/s lr: 5.58e-04  
[09/15 15:05:41] lb.utils.events INFO:  eta: 15:29:21  iteration: 14199/375342  consumed_samples: 14540800  total_loss: 4.556  time: 0.3966 s/iter  data_time: 0.2564 s/iter total_throughput: 2581.71 samples/s lr: 5.62e-04  
[09/15 15:06:21] lb.utils.events INFO:  eta: 15:28:34  iteration: 14299/375342  consumed_samples: 14643200  total_loss: 4.55  time: 0.3967 s/iter  data_time: 0.2502 s/iter total_throughput: 2581.38 samples/s lr: 5.66e-04  
[09/15 15:07:02] lb.utils.events INFO:  eta: 15:28:04  iteration: 14399/375342  consumed_samples: 14745600  total_loss: 4.542  time: 0.3968 s/iter  data_time: 0.2730 s/iter total_throughput: 2580.66 samples/s lr: 5.70e-04  
[09/15 15:07:44] lb.utils.events INFO:  eta: 15:28:11  iteration: 14499/375342  consumed_samples: 14848000  total_loss: 4.544  time: 0.3969 s/iter  data_time: 0.2689 s/iter total_throughput: 2579.82 samples/s lr: 5.74e-04  
[09/15 15:08:26] lb.utils.events INFO:  eta: 15:30:52  iteration: 14599/375342  consumed_samples: 14950400  total_loss: 4.559  time: 0.3971 s/iter  data_time: 0.2702 s/iter total_throughput: 2578.80 samples/s lr: 5.78e-04  
[09/15 15:09:07] lb.utils.events INFO:  eta: 15:31:39  iteration: 14699/375342  consumed_samples: 15052800  total_loss: 4.528  time: 0.3972 s/iter  data_time: 0.2674 s/iter total_throughput: 2577.92 samples/s lr: 5.81e-04  
[09/15 15:09:49] lb.utils.events INFO:  eta: 15:35:45  iteration: 14799/375342  consumed_samples: 15155200  total_loss: 4.518  time: 0.3973 s/iter  data_time: 0.2552 s/iter total_throughput: 2577.31 samples/s lr: 5.85e-04  
[09/15 15:10:29] lb.utils.events INFO:  eta: 15:40:46  iteration: 14899/375342  consumed_samples: 15257600  total_loss: 4.514  time: 0.3974 s/iter  data_time: 0.2538 s/iter total_throughput: 2577.00 samples/s lr: 5.89e-04  
[09/15 15:11:10] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_2e56/model_0014999
[09/15 15:11:10] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000
[09/15 15:11:10] lb.evaluation.evaluator INFO: Start inference on 50000 samples
[09/15 15:11:15] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.1141 s/iter. Inference: 0.1547 s/iter. Eval: 0.0023 s/iter. Total: 0.2712 s/iter. ETA=0:00:10
[09/15 15:11:20] lb.evaluation.evaluator INFO: Inference done 27648/50000. Dataloading: 0.1534 s/iter. Inference: 0.1492 s/iter. Eval: 0.0022 s/iter. Total: 0.3048 s/iter. ETA=0:00:06
[09/15 15:11:25] lb.evaluation.evaluator INFO: Inference done 45056/50000. Dataloading: 0.1503 s/iter. Inference: 0.1490 s/iter. Eval: 0.0022 s/iter. Total: 0.3016 s/iter. ETA=0:00:01
[09/15 15:11:27] lb.evaluation.evaluator INFO: Total valid samples: 50000
[09/15 15:11:27] lb.evaluation.evaluator INFO: Total inference time: 0:00:13.452664 (0.000269 s / iter per device, on 8 devices)
[09/15 15:11:27] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000131 s / iter per device, on 8 devices)
[09/15 15:11:27] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format:
[09/15 15:11:27] lb.evaluation.utils INFO: copypaste: Acc@1=43.248
[09/15 15:11:27] lb.evaluation.utils INFO: copypaste: Acc@5=67.716
[09/15 15:11:27] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 43.24800, better than last best score 33.28200 @ iteration 9999.
[09/15 15:11:27] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_2e56/model_best
[09/15 15:11:28] lb.utils.events INFO:  eta: 15:42:00  iteration: 14999/375342  consumed_samples: 15360000  total_loss: 4.51  time: 0.3974 s/iter  data_time: 0.2680 s/iter total_throughput: 2576.53 samples/s lr: 5.93e-04  
[09/15 15:12:06] lb.utils.events INFO:  eta: 15:57:29  iteration: 15099/375342  consumed_samples: 15462400  total_loss: 4.498  time: 0.3973 s/iter  data_time: 0.2557 s/iter total_throughput: 2577.22 samples/s lr: 5.97e-04  
[09/15 15:12:47] lb.utils.events INFO:  eta: 19:21:33  iteration: 15199/375342  consumed_samples: 15564800  total_loss: 4.491  time: 0.3974 s/iter  data_time: 0.2599 s/iter total_throughput: 2576.75 samples/s lr: 6.01e-04  
[09/15 15:13:28] lb.utils.events INFO:  eta: 22:18:18  iteration: 15299/375342  consumed_samples: 15667200  total_loss: 4.491  time: 0.3975 s/iter  data_time: 0.2720 s/iter total_throughput: 2576.18 samples/s lr: 6.05e-04  
[09/15 15:14:09] lb.utils.events INFO:  eta: 23:00:38  iteration: 15399/375342  consumed_samples: 15769600  total_loss: 4.473  time: 0.3976 s/iter  data_time: 0.2664 s/iter total_throughput: 2575.41 samples/s lr: 6.09e-04  
[09/15 15:14:51] lb.utils.events INFO:  eta: 20:17:46  iteration: 15499/375342  consumed_samples: 15872000  total_loss: 4.491  time: 0.3977 s/iter  data_time: 0.2541 s/iter total_throughput: 2574.74 samples/s lr: 6.13e-04  
[09/15 15:15:31] lb.utils.events INFO:  eta: 15:51:09  iteration: 15599/375342  consumed_samples: 15974400  total_loss: 4.48  time: 0.3978 s/iter  data_time: 0.2505 s/iter total_throughput: 2574.43 samples/s lr: 6.17e-04  
[09/15 15:16:12] lb.utils.events INFO:  eta: 15:43:56  iteration: 15699/375342  consumed_samples: 16076800  total_loss: 4.465  time: 0.3978 s/iter  data_time: 0.2465 s/iter total_throughput: 2574.16 samples/s lr: 6.21e-04  
[09/15 15:16:52] lb.utils.events INFO:  eta: 15:40:04  iteration: 15799/375342  consumed_samples: 16179200  total_loss: 4.469  time: 0.3978 s/iter  data_time: 0.2574 s/iter total_throughput: 2573.90 samples/s lr: 6.25e-04  
[09/15 15:17:33] lb.utils.events INFO:  eta: 15:42:11  iteration: 15899/375342  consumed_samples: 16281600  total_loss: 4.472  time: 0.3979 s/iter  data_time: 0.2538 s/iter total_throughput: 2573.59 samples/s lr: 6.29e-04  
[09/15 15:18:13] lb.utils.events INFO:  eta: 15:42:51  iteration: 15999/375342  consumed_samples: 16384000  total_loss: 4.475  time: 0.3979 s/iter  data_time: 0.2532 s/iter total_throughput: 2573.30 samples/s lr: 6.33e-04  
[09/15 15:18:54] lb.utils.events INFO:  eta: 15:35:13  iteration: 16099/375342  consumed_samples: 16486400  total_loss: 4.486  time: 0.3980 s/iter  data_time: 0.2550 s/iter total_throughput: 2572.92 samples/s lr: 6.37e-04  
[09/15 15:19:35] lb.utils.events INFO:  eta: 15:28:41  iteration: 16199/375342  consumed_samples: 16588800  total_loss: 4.483  time: 0.3981 s/iter  data_time: 0.2573 s/iter total_throughput: 2572.49 samples/s lr: 6.41e-04  
[09/15 15:20:15] lb.utils.events INFO:  eta: 15:25:49  iteration: 16299/375342  consumed_samples: 16691200  total_loss: 4.451  time: 0.3981 s/iter  data_time: 0.2598 s/iter total_throughput: 2572.27 samples/s lr: 6.45e-04  
[09/15 15:20:55] lb.utils.events INFO:  eta: 15:24:54  iteration: 16399/375342  consumed_samples: 16793600  total_loss: 4.414  time: 0.3981 s/iter  data_time: 0.2534 s/iter total_throughput: 2572.09 samples/s lr: 6.49e-04  
[09/15 15:21:36] lb.utils.events INFO:  eta: 15:25:28  iteration: 16499/375342  consumed_samples: 16896000  total_loss: 4.407  time: 0.3981 s/iter  data_time: 0.2516 s/iter total_throughput: 2571.90 samples/s lr: 6.53e-04  
[09/15 15:22:16] lb.utils.events INFO:  eta: 15:25:02  iteration: 16599/375342  consumed_samples: 16998400  total_loss: 4.417  time: 0.3982 s/iter  data_time: 0.2634 s/iter total_throughput: 2571.81 samples/s lr: 6.57e-04  
[09/15 15:22:56] lb.utils.events INFO:  eta: 15:24:17  iteration: 16699/375342  consumed_samples: 17100800  total_loss: 4.411  time: 0.3982 s/iter  data_time: 0.2627 s/iter total_throughput: 2571.69 samples/s lr: 6.60e-04  
[09/15 15:23:36] lb.utils.events INFO:  eta: 15:22:02  iteration: 16799/375342  consumed_samples: 17203200  total_loss: 4.378  time: 0.3982 s/iter  data_time: 0.2541 s/iter total_throughput: 2571.45 samples/s lr: 6.64e-04  
[09/15 15:24:17] lb.utils.events INFO:  eta: 15:21:28  iteration: 16899/375342  consumed_samples: 17305600  total_loss: 4.391  time: 0.3983 s/iter  data_time: 0.2543 s/iter total_throughput: 2571.19 samples/s lr: 6.68e-04  
[09/15 15:24:57] lb.utils.events INFO:  eta: 15:20:43  iteration: 16999/375342  consumed_samples: 17408000  total_loss: 4.414  time: 0.3983 s/iter  data_time: 0.2575 s/iter total_throughput: 2571.11 samples/s lr: 6.72e-04  
[09/15 15:25:37] lb.utils.events INFO:  eta: 15:20:31  iteration: 17099/375342  consumed_samples: 17510400  total_loss: 4.402  time: 0.3983 s/iter  data_time: 0.2464 s/iter total_throughput: 2571.00 samples/s lr: 6.76e-04  
[09/15 15:26:18] lb.utils.events INFO:  eta: 15:20:51  iteration: 17199/375342  consumed_samples: 17612800  total_loss: 4.389  time: 0.3983 s/iter  data_time: 0.2532 s/iter total_throughput: 2570.72 samples/s lr: 6.80e-04  
[09/15 15:26:58] lb.utils.events INFO:  eta: 15:19:57  iteration: 17299/375342  consumed_samples: 17715200  total_loss: 4.387  time: 0.3984 s/iter  data_time: 0.2478 s/iter total_throughput: 2570.55 samples/s lr: 6.84e-04  
[09/15 15:27:38] lb.utils.events INFO:  eta: 15:21:02  iteration: 17399/375342  consumed_samples: 17817600  total_loss: 4.406  time: 0.3984 s/iter  data_time: 0.2525 s/iter total_throughput: 2570.44 samples/s lr: 6.88e-04  
[09/15 15:28:19] lb.utils.events INFO:  eta: 15:20:50  iteration: 17499/375342  consumed_samples: 17920000  total_loss: 4.409  time: 0.3984 s/iter  data_time: 0.2552 s/iter total_throughput: 2570.17 samples/s lr: 6.92e-04  
[09/15 15:28:59] lb.utils.events INFO:  eta: 15:20:42  iteration: 17599/375342  consumed_samples: 18022400  total_loss: 4.364  time: 0.3985 s/iter  data_time: 0.2562 s/iter total_throughput: 2569.94 samples/s lr: 6.96e-04  
[09/15 15:29:39] lb.utils.events INFO:  eta: 15:20:20  iteration: 17699/375342  consumed_samples: 18124800  total_loss: 4.352  time: 0.3985 s/iter  data_time: 0.2504 s/iter total_throughput: 2569.87 samples/s lr: 7.00e-04  
[09/15 15:30:20] lb.utils.events INFO:  eta: 15:20:29  iteration: 17799/375342  consumed_samples: 18227200  total_loss: 4.342  time: 0.3985 s/iter  data_time: 0.2583 s/iter total_throughput: 2569.54 samples/s lr: 7.04e-04  
[09/15 15:31:00] lb.utils.events INFO:  eta: 15:21:18  iteration: 17899/375342  consumed_samples: 18329600  total_loss: 4.339  time: 0.3985 s/iter  data_time: 0.2485 s/iter total_throughput: 2569.43 samples/s lr: 7.08e-04  
[09/15 15:31:41] lb.utils.events INFO:  eta: 15:23:00  iteration: 17999/375342  consumed_samples: 18432000  total_loss: 4.346  time: 0.3986 s/iter  data_time: 0.2672 s/iter total_throughput: 2569.15 samples/s lr: 7.12e-04  
[09/15 15:32:21] lb.utils.events INFO:  eta: 15:22:44  iteration: 18099/375342  consumed_samples: 18534400  total_loss: 4.333  time: 0.3986 s/iter  data_time: 0.2495 s/iter total_throughput: 2568.90 samples/s lr: 7.16e-04  
[09/15 15:33:02] lb.utils.events INFO:  eta: 15:22:22  iteration: 18199/375342  consumed_samples: 18636800  total_loss: 4.333  time: 0.3986 s/iter  data_time: 0.2619 s/iter total_throughput: 2568.76 samples/s lr: 7.20e-04  
[09/15 15:33:42] lb.utils.events INFO:  eta: 15:22:13  iteration: 18299/375342  consumed_samples: 18739200  total_loss: 4.349  time: 0.3987 s/iter  data_time: 0.2543 s/iter total_throughput: 2568.46 samples/s lr: 7.24e-04  
[09/15 15:34:23] lb.utils.events INFO:  eta: 15:21:21  iteration: 18399/375342  consumed_samples: 18841600  total_loss: 4.338  time: 0.3987 s/iter  data_time: 0.2659 s/iter total_throughput: 2568.25 samples/s lr: 7.28e-04  
[09/15 15:35:03] lb.utils.events INFO:  eta: 15:20:31  iteration: 18499/375342  consumed_samples: 18944000  total_loss: 4.321  time: 0.3987 s/iter  data_time: 0.2676 s/iter total_throughput: 2568.05 samples/s lr: 7.32e-04  
[09/15 15:35:44] lb.utils.events INFO:  eta: 15:20:03  iteration: 18599/375342  consumed_samples: 19046400  total_loss: 4.334  time: 0.3988 s/iter  data_time: 0.2563 s/iter total_throughput: 2567.71 samples/s lr: 7.35e-04  
[09/15 15:36:24] lb.utils.events INFO:  eta: 15:18:46  iteration: 18699/375342  consumed_samples: 19148800  total_loss: 4.319  time: 0.3988 s/iter  data_time: 0.2519 s/iter total_throughput: 2567.57 samples/s lr: 7.39e-04  
[09/15 15:37:04] lb.utils.events INFO:  eta: 15:18:15  iteration: 18799/375342  consumed_samples: 19251200  total_loss: 4.311  time: 0.3988 s/iter  data_time: 0.2516 s/iter total_throughput: 2567.55 samples/s lr: 7.43e-04  
[09/15 15:37:45] lb.utils.events INFO:  eta: 15:17:33  iteration: 18899/375342  consumed_samples: 19353600  total_loss: 4.306  time: 0.3989 s/iter  data_time: 0.2562 s/iter total_throughput: 2567.36 samples/s lr: 7.47e-04  
[09/15 15:38:25] lb.utils.events INFO:  eta: 15:15:38  iteration: 18999/375342  consumed_samples: 19456000  total_loss: 4.282  time: 0.3989 s/iter  data_time: 0.2507 s/iter total_throughput: 2567.16 samples/s lr: 7.51e-04  
[09/15 15:39:06] lb.utils.events INFO:  eta: 15:15:23  iteration: 19099/375342  consumed_samples: 19558400  total_loss: 4.307  time: 0.3989 s/iter  data_time: 0.2499 s/iter total_throughput: 2567.01 samples/s lr: 7.55e-04  
[09/15 15:39:46] lb.utils.events INFO:  eta: 15:14:11  iteration: 19199/375342  consumed_samples: 19660800  total_loss: 4.29  time: 0.3990 s/iter  data_time: 0.2681 s/iter total_throughput: 2566.69 samples/s lr: 7.59e-04  
[09/15 15:40:26] lb.utils.events INFO:  eta: 15:14:18  iteration: 19299/375342  consumed_samples: 19763200  total_loss: 4.289  time: 0.3990 s/iter  data_time: 0.2494 s/iter total_throughput: 2566.70 samples/s lr: 7.63e-04  
[09/15 15:41:07] lb.utils.events INFO:  eta: 15:14:35  iteration: 19399/375342  consumed_samples: 19865600  total_loss: 4.296  time: 0.3990 s/iter  data_time: 0.2503 s/iter total_throughput: 2566.54 samples/s lr: 7.67e-04  
[09/15 15:41:48] lb.utils.events INFO:  eta: 15:13:55  iteration: 19499/375342  consumed_samples: 19968000  total_loss: 4.276  time: 0.3990 s/iter  data_time: 0.2533 s/iter total_throughput: 2566.18 samples/s lr: 7.71e-04  
[09/15 15:42:28] lb.utils.events INFO:  eta: 15:13:26  iteration: 19599/375342  consumed_samples: 20070400  total_loss: 4.265  time: 0.3991 s/iter  data_time: 0.2589 s/iter total_throughput: 2565.99 samples/s lr: 7.75e-04  
[09/15 15:43:09] lb.utils.events INFO:  eta: 15:13:22  iteration: 19699/375342  consumed_samples: 20172800  total_loss: 4.264  time: 0.3991 s/iter  data_time: 0.2637 s/iter total_throughput: 2565.84 samples/s lr: 7.79e-04  
[09/15 15:43:49] lb.utils.events INFO:  eta: 15:13:27  iteration: 19799/375342  consumed_samples: 20275200  total_loss: 4.288  time: 0.3991 s/iter  data_time: 0.2518 s/iter total_throughput: 2565.77 samples/s lr: 7.83e-04  
[09/15 15:44:29] lb.utils.events INFO:  eta: 15:13:01  iteration: 19899/375342  consumed_samples: 20377600  total_loss: 4.292  time: 0.3991 s/iter  data_time: 0.2397 s/iter total_throughput: 2565.72 samples/s lr: 7.87e-04  
[09/15 15:45:09] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_2e56/model_0019999
[09/15 15:45:10] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000
[09/15 15:45:10] lb.evaluation.evaluator INFO: Start inference on 50000 samples
[09/15 15:45:14] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.1090 s/iter. Inference: 0.1539 s/iter. Eval: 0.0023 s/iter. Total: 0.2652 s/iter. ETA=0:00:09
[09/15 15:45:19] lb.evaluation.evaluator INFO: Inference done 28672/50000. Dataloading: 0.1381 s/iter. Inference: 0.1489 s/iter. Eval: 0.0022 s/iter. Total: 0.2892 s/iter. ETA=0:00:05
[09/15 15:45:25] lb.evaluation.evaluator INFO: Inference done 46080/50000. Dataloading: 0.1517 s/iter. Inference: 0.1492 s/iter. Eval: 0.0021 s/iter. Total: 0.3031 s/iter. ETA=0:00:00
[09/15 15:45:26] lb.evaluation.evaluator INFO: Total valid samples: 50000
[09/15 15:45:26] lb.evaluation.evaluator INFO: Total inference time: 0:00:13.195733 (0.000264 s / iter per device, on 8 devices)
[09/15 15:45:26] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000132 s / iter per device, on 8 devices)
[09/15 15:45:26] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format:
[09/15 15:45:26] lb.evaluation.utils INFO: copypaste: Acc@1=49.244
[09/15 15:45:26] lb.evaluation.utils INFO: copypaste: Acc@5=73.236
[09/15 15:45:26] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 49.24400, better than last best score 43.24800 @ iteration 14999.
[09/15 15:45:26] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_2e56/model_best
[09/15 15:45:26] lb.utils.events INFO:  eta: 15:14:53  iteration: 19999/375342  consumed_samples: 20480000  total_loss: 4.295  time: 0.3991 s/iter  data_time: 0.2592 s/iter total_throughput: 2565.64 samples/s lr: 7.91e-04  
[09/15 15:46:05] lb.utils.events INFO:  eta: 15:16:39  iteration: 20099/375342  consumed_samples: 20582400  total_loss: 4.284  time: 0.3990 s/iter  data_time: 0.2583 s/iter total_throughput: 2566.12 samples/s lr: 7.95e-04  
[09/15 15:46:45] lb.utils.events INFO:  eta: 15:18:00  iteration: 20199/375342  consumed_samples: 20684800  total_loss: 4.262  time: 0.3991 s/iter  data_time: 0.2547 s/iter total_throughput: 2566.07 samples/s lr: 7.99e-04  
[09/15 15:47:25] lb.utils.events INFO:  eta: 15:16:16  iteration: 20299/375342  consumed_samples: 20787200  total_loss: 4.253  time: 0.3991 s/iter  data_time: 0.2583 s/iter total_throughput: 2565.87 samples/s lr: 8.03e-04  
[09/15 15:48:05] lb.utils.events INFO:  eta: 15:17:10  iteration: 20399/375342  consumed_samples: 20889600  total_loss: 4.24  time: 0.3991 s/iter  data_time: 0.2478 s/iter total_throughput: 2565.89 samples/s lr: 8.07e-04  
[09/15 15:48:46] lb.utils.events INFO:  eta: 15:21:00  iteration: 20499/375342  consumed_samples: 20992000  total_loss: 4.236  time: 0.3991 s/iter  data_time: 0.2510 s/iter total_throughput: 2565.73 samples/s lr: 8.11e-04  
[09/15 15:49:26] lb.utils.events INFO:  eta: 15:26:31  iteration: 20599/375342  consumed_samples: 21094400  total_loss: 4.244  time: 0.3991 s/iter  data_time: 0.2518 s/iter total_throughput: 2565.68 samples/s lr: 8.14e-04  
[09/15 15:50:06] lb.utils.events INFO:  eta: 15:30:46  iteration: 20699/375342  consumed_samples: 21196800  total_loss: 4.25  time: 0.3991 s/iter  data_time: 0.2500 s/iter total_throughput: 2565.55 samples/s lr: 8.18e-04  
[09/15 15:50:46] lb.utils.events INFO:  eta: 15:46:55  iteration: 20799/375342  consumed_samples: 21299200  total_loss: 4.243  time: 0.3991 s/iter  data_time: 0.2478 s/iter total_throughput: 2565.49 samples/s lr: 8.22e-04  
[09/15 15:51:27] lb.utils.events INFO:  eta: 16:24:12  iteration: 20899/375342  consumed_samples: 21401600  total_loss: 4.229  time: 0.3992 s/iter  data_time: 0.2524 s/iter total_throughput: 2565.32 samples/s lr: 8.26e-04  
[09/15 15:52:07] lb.utils.events INFO:  eta: 15:54:40  iteration: 20999/375342  consumed_samples: 21504000  total_loss: 4.228  time: 0.3992 s/iter  data_time: 0.2529 s/iter total_throughput: 2565.26 samples/s lr: 8.30e-04  
[09/15 15:52:47] lb.utils.events INFO:  eta: 15:35:22  iteration: 21099/375342  consumed_samples: 21606400  total_loss: 4.197  time: 0.3992 s/iter  data_time: 0.2484 s/iter total_throughput: 2565.20 samples/s lr: 8.34e-04  
[09/15 15:53:28] lb.utils.events INFO:  eta: 15:37:23  iteration: 21199/375342  consumed_samples: 21708800  total_loss: 4.207  time: 0.3992 s/iter  data_time: 0.2630 s/iter total_throughput: 2565.03 samples/s lr: 8.38e-04  
[09/15 15:54:08] lb.utils.events INFO:  eta: 15:57:06  iteration: 21299/375342  consumed_samples: 21811200  total_loss: 4.189  time: 0.3993 s/iter  data_time: 0.2681 s/iter total_throughput: 2564.74 samples/s lr: 8.42e-04  
[09/15 15:54:48] lb.utils.events INFO:  eta: 16:18:15  iteration: 21399/375342  consumed_samples: 21913600  total_loss: 4.172  time: 0.3993 s/iter  data_time: 0.2446 s/iter total_throughput: 2564.81 samples/s lr: 8.46e-04  
[09/15 15:55:29] lb.utils.events INFO:  eta: 15:36:08  iteration: 21499/375342  consumed_samples: 22016000  total_loss: 4.206  time: 0.3993 s/iter  data_time: 0.2514 s/iter total_throughput: 2564.66 samples/s lr: 8.50e-04  
[09/15 15:56:09] lb.utils.events INFO:  eta: 15:29:45  iteration: 21599/375342  consumed_samples: 22118400  total_loss: 4.204  time: 0.3993 s/iter  data_time: 0.2508 s/iter total_throughput: 2564.64 samples/s lr: 8.54e-04  
[09/15 15:56:49] lb.utils.events INFO:  eta: 15:26:39  iteration: 21699/375342  consumed_samples: 22220800  total_loss: 4.213  time: 0.3993 s/iter  data_time: 0.2547 s/iter total_throughput: 2564.40 samples/s lr: 8.58e-04  
[09/15 15:57:29] lb.utils.events INFO:  eta: 15:20:51  iteration: 21799/375342  consumed_samples: 22323200  total_loss: 4.208  time: 0.3993 s/iter  data_time: 0.2521 s/iter total_throughput: 2564.40 samples/s lr: 8.62e-04  
[09/15 15:58:09] lb.utils.events INFO:  eta: 15:16:25  iteration: 21899/375342  consumed_samples: 22425600  total_loss: 4.186  time: 0.3993 s/iter  data_time: 0.2545 s/iter total_throughput: 2564.32 samples/s lr: 8.66e-04  
[09/15 15:58:50] lb.utils.events INFO:  eta: 15:15:31  iteration: 21999/375342  consumed_samples: 22528000  total_loss: 4.165  time: 0.3993 s/iter  data_time: 0.2524 s/iter total_throughput: 2564.21 samples/s lr: 8.70e-04  
[09/15 15:59:30] lb.utils.events INFO:  eta: 15:14:39  iteration: 22099/375342  consumed_samples: 22630400  total_loss: 4.172  time: 0.3993 s/iter  data_time: 0.2536 s/iter total_throughput: 2564.17 samples/s lr: 8.74e-04  
[09/15 16:00:10] lb.utils.events INFO:  eta: 15:13:35  iteration: 22199/375342  consumed_samples: 22732800  total_loss: 4.18  time: 0.3994 s/iter  data_time: 0.2449 s/iter total_throughput: 2564.04 samples/s lr: 8.78e-04  
[09/15 16:00:51] lb.utils.events INFO:  eta: 15:11:13  iteration: 22299/375342  consumed_samples: 22835200  total_loss: 4.167  time: 0.3994 s/iter  data_time: 0.2479 s/iter total_throughput: 2563.88 samples/s lr: 8.82e-04  
[09/15 16:01:31] lb.utils.events INFO:  eta: 15:09:11  iteration: 22399/375342  consumed_samples: 22937600  total_loss: 4.158  time: 0.3994 s/iter  data_time: 0.2589 s/iter total_throughput: 2563.93 samples/s lr: 8.86e-04  
[09/15 16:02:10] lb.utils.events INFO:  eta: 15:09:43  iteration: 22499/375342  consumed_samples: 23040000  total_loss: 4.169  time: 0.3994 s/iter  data_time: 0.2499 s/iter total_throughput: 2564.02 samples/s lr: 8.90e-04  
[09/15 16:02:51] lb.utils.events INFO:  eta: 15:07:51  iteration: 22599/375342  consumed_samples: 23142400  total_loss: 4.164  time: 0.3994 s/iter  data_time: 0.2488 s/iter total_throughput: 2563.88 samples/s lr: 8.93e-04  
[09/15 16:03:31] lb.utils.events INFO:  eta: 15:07:43  iteration: 22699/375342  consumed_samples: 23244800  total_loss: 4.148  time: 0.3994 s/iter  data_time: 0.2575 s/iter total_throughput: 2563.78 samples/s lr: 8.97e-04  
[09/15 16:04:12] lb.utils.events INFO:  eta: 15:07:27  iteration: 22799/375342  consumed_samples: 23347200  total_loss: 4.133  time: 0.3994 s/iter  data_time: 0.2608 s/iter total_throughput: 2563.58 samples/s lr: 9.01e-04  
[09/15 16:04:52] lb.utils.events INFO:  eta: 15:07:50  iteration: 22899/375342  consumed_samples: 23449600  total_loss: 4.107  time: 0.3995 s/iter  data_time: 0.2646 s/iter total_throughput: 2563.41 samples/s lr: 9.05e-04  
[09/15 16:05:32] lb.utils.events INFO:  eta: 15:09:26  iteration: 22999/375342  consumed_samples: 23552000  total_loss: 4.133  time: 0.3995 s/iter  data_time: 0.2584 s/iter total_throughput: 2563.39 samples/s lr: 9.09e-04  
[09/15 16:06:12] lb.utils.events INFO:  eta: 15:10:08  iteration: 23099/375342  consumed_samples: 23654400  total_loss: 4.144  time: 0.3995 s/iter  data_time: 0.2376 s/iter total_throughput: 2563.32 samples/s lr: 9.13e-04  
[09/15 16:06:54] lb.utils.events INFO:  eta: 15:10:15  iteration: 23199/375342  consumed_samples: 23756800  total_loss: 4.145  time: 0.3995 s/iter  data_time: 0.2662 s/iter total_throughput: 2562.97 samples/s lr: 9.17e-04  
[09/15 16:07:34] lb.utils.events INFO:  eta: 15:12:09  iteration: 23299/375342  consumed_samples: 23859200  total_loss: 4.154  time: 0.3996 s/iter  data_time: 0.2560 s/iter total_throughput: 2562.86 samples/s lr: 9.21e-04  
[09/15 16:08:14] lb.utils.events INFO:  eta: 15:15:18  iteration: 23399/375342  consumed_samples: 23961600  total_loss: 4.15  time: 0.3995 s/iter  data_time: 0.2516 s/iter total_throughput: 2562.90 samples/s lr: 9.25e-04  
[09/15 16:08:54] lb.utils.events INFO:  eta: 15:17:15  iteration: 23499/375342  consumed_samples: 24064000  total_loss: 4.144  time: 0.3996 s/iter  data_time: 0.2516 s/iter total_throughput: 2562.88 samples/s lr: 9.29e-04  
[09/15 16:09:34] lb.utils.events INFO:  eta: 15:27:56  iteration: 23599/375342  consumed_samples: 24166400  total_loss: 4.125  time: 0.3996 s/iter  data_time: 0.2517 s/iter total_throughput: 2562.85 samples/s lr: 9.33e-04  
[09/15 16:10:14] lb.utils.events INFO:  eta: 15:57:09  iteration: 23699/375342  consumed_samples: 24268800  total_loss: 4.13  time: 0.3996 s/iter  data_time: 0.2509 s/iter total_throughput: 2562.73 samples/s lr: 9.37e-04  
[09/15 16:10:55] lb.utils.events INFO:  eta: 20:15:29  iteration: 23799/375342  consumed_samples: 24371200  total_loss: 4.152  time: 0.3996 s/iter  data_time: 0.2594 s/iter total_throughput: 2562.45 samples/s lr: 9.41e-04  
[09/15 16:11:36] lb.utils.events INFO:  eta: 18:48:40  iteration: 23899/375342  consumed_samples: 24473600  total_loss: 4.12  time: 0.3996 s/iter  data_time: 0.2544 s/iter total_throughput: 2562.28 samples/s lr: 9.45e-04  
[09/15 16:12:16] lb.utils.events INFO:  eta: 16:01:04  iteration: 23999/375342  consumed_samples: 24576000  total_loss: 4.101  time: 0.3997 s/iter  data_time: 0.2514 s/iter total_throughput: 2562.20 samples/s lr: 9.49e-04  
[09/15 16:12:57] lb.utils.events INFO:  eta: 15:23:22  iteration: 24099/375342  consumed_samples: 24678400  total_loss: 4.102  time: 0.3997 s/iter  data_time: 0.2534 s/iter total_throughput: 2562.10 samples/s lr: 9.53e-04  
[09/15 16:13:37] lb.utils.events INFO:  eta: 15:17:57  iteration: 24199/375342  consumed_samples: 24780800  total_loss: 4.088  time: 0.3997 s/iter  data_time: 0.2560 s/iter total_throughput: 2561.91 samples/s lr: 9.57e-04  
[09/15 16:14:18] lb.utils.events INFO:  eta: 15:14:12  iteration: 24299/375342  consumed_samples: 24883200  total_loss: 4.105  time: 0.3997 s/iter  data_time: 0.2619 s/iter total_throughput: 2561.76 samples/s lr: 9.61e-04  
[09/15 16:14:57] lb.utils.events INFO:  eta: 15:10:06  iteration: 24399/375342  consumed_samples: 24985600  total_loss: 4.105  time: 0.3997 s/iter  data_time: 0.2466 s/iter total_throughput: 2561.86 samples/s lr: 9.65e-04  
[09/15 16:15:37] lb.utils.events INFO:  eta: 15:07:27  iteration: 24499/375342  consumed_samples: 25088000  total_loss: 4.103  time: 0.3997 s/iter  data_time: 0.2613 s/iter total_throughput: 2561.86 samples/s lr: 9.68e-04  
[09/15 16:16:18] lb.utils.events INFO:  eta: 15:06:46  iteration: 24599/375342  consumed_samples: 25190400  total_loss: 4.107  time: 0.3997 s/iter  data_time: 0.2481 s/iter total_throughput: 2561.73 samples/s lr: 9.72e-04  
[09/15 16:16:58] lb.utils.events INFO:  eta: 15:05:22  iteration: 24699/375342  consumed_samples: 25292800  total_loss: 4.103  time: 0.3997 s/iter  data_time: 0.2508 s/iter total_throughput: 2561.73 samples/s lr: 9.76e-04  
[09/15 16:17:38] lb.utils.events INFO:  eta: 15:04:29  iteration: 24799/375342  consumed_samples: 25395200  total_loss: 4.102  time: 0.3997 s/iter  data_time: 0.2528 s/iter total_throughput: 2561.69 samples/s lr: 9.80e-04  
[09/15 16:18:18] lb.utils.events INFO:  eta: 15:05:52  iteration: 24899/375342  consumed_samples: 25497600  total_loss: 4.097  time: 0.3997 s/iter  data_time: 0.2649 s/iter total_throughput: 2561.71 samples/s lr: 9.84e-04  
[09/15 16:18:58] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_2e56/model_0024999
[09/15 16:18:59] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000
[09/15 16:18:59] lb.evaluation.evaluator INFO: Start inference on 50000 samples
[09/15 16:19:03] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.1050 s/iter. Inference: 0.1477 s/iter. Eval: 0.0018 s/iter. Total: 0.2545 s/iter. ETA=0:00:09
[09/15 16:19:08] lb.evaluation.evaluator INFO: Inference done 27648/50000. Dataloading: 0.0880 s/iter. Inference: 0.2087 s/iter. Eval: 0.0021 s/iter. Total: 0.2989 s/iter. ETA=0:00:06
[09/15 16:19:14] lb.evaluation.evaluator INFO: Inference done 46080/50000. Dataloading: 0.0555 s/iter. Inference: 0.2450 s/iter. Eval: 0.0021 s/iter. Total: 0.3027 s/iter. ETA=0:00:00
[09/15 16:19:15] lb.evaluation.evaluator INFO: Total valid samples: 50000
[09/15 16:19:15] lb.evaluation.evaluator INFO: Total inference time: 0:00:13.136625 (0.000263 s / iter per device, on 8 devices)
[09/15 16:19:15] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:10 (0.000214 s / iter per device, on 8 devices)
[09/15 16:19:15] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format:
[09/15 16:19:15] lb.evaluation.utils INFO: copypaste: Acc@1=52.536
[09/15 16:19:15] lb.evaluation.utils INFO: copypaste: Acc@5=76.574
[09/15 16:19:15] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 52.53600, better than last best score 49.24400 @ iteration 19999.
[09/15 16:19:15] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_2e56/model_best
[09/15 16:19:15] lb.utils.events INFO:  eta: 15:10:24  iteration: 24999/375342  consumed_samples: 25600000  total_loss: 4.093  time: 0.3997 s/iter  data_time: 0.2506 s/iter total_throughput: 2561.63 samples/s lr: 9.88e-04  
[09/15 16:19:54] lb.utils.events INFO:  eta: 15:23:08  iteration: 25099/375342  consumed_samples: 25702400  total_loss: 4.093  time: 0.3997 s/iter  data_time: 0.2515 s/iter total_throughput: 2562.12 samples/s lr: 9.89e-04  
[09/15 16:20:34] lb.utils.events INFO:  eta: 16:14:34  iteration: 25199/375342  consumed_samples: 25804800  total_loss: 4.08  time: 0.3997 s/iter  data_time: 0.2531 s/iter total_throughput: 2562.06 samples/s lr: 9.89e-04  
[09/15 16:21:14] lb.utils.events INFO:  eta: 19:39:26  iteration: 25299/375342  consumed_samples: 25907200  total_loss: 4.063  time: 0.3997 s/iter  data_time: 0.2613 s/iter total_throughput: 2561.95 samples/s lr: 9.89e-04  
[09/15 16:21:54] lb.utils.events INFO:  eta: 23:27:19  iteration: 25399/375342  consumed_samples: 26009600  total_loss: 4.078  time: 0.3997 s/iter  data_time: 0.2515 s/iter total_throughput: 2561.90 samples/s lr: 9.89e-04  
[09/15 16:22:35] lb.utils.events INFO:  eta: 1 day, 1:05:22  iteration: 25499/375342  consumed_samples: 26112000  total_loss: 4.056  time: 0.3997 s/iter  data_time: 0.2667 s/iter total_throughput: 2561.77 samples/s lr: 9.89e-04  
[09/15 16:23:15] lb.utils.events INFO:  eta: 1 day, 1:23:55  iteration: 25599/375342  consumed_samples: 26214400  total_loss: 4.031  time: 0.3997 s/iter  data_time: 0.2691 s/iter total_throughput: 2561.70 samples/s lr: 9.89e-04  
[09/15 16:23:55] lb.utils.events INFO:  eta: 1 day, 1:57:37  iteration: 25699/375342  consumed_samples: 26316800  total_loss: 4.063  time: 0.3997 s/iter  data_time: 0.2536 s/iter total_throughput: 2561.78 samples/s lr: 9.89e-04  
[09/15 16:24:35] lb.utils.events INFO:  eta: 23:13:14  iteration: 25799/375342  consumed_samples: 26419200  total_loss: 4.092  time: 0.3997 s/iter  data_time: 0.2665 s/iter total_throughput: 2561.75 samples/s lr: 9.89e-04  
[09/15 16:25:15] lb.utils.events INFO:  eta: 23:02:27  iteration: 25899/375342  consumed_samples: 26521600  total_loss: 4.077  time: 0.3997 s/iter  data_time: 0.2478 s/iter total_throughput: 2561.76 samples/s lr: 9.88e-04  
[09/15 16:25:55] lb.utils.events INFO:  eta: 18:05:14  iteration: 25999/375342  consumed_samples: 26624000  total_loss: 4.06  time: 0.3997 s/iter  data_time: 0.2461 s/iter total_throughput: 2561.81 samples/s lr: 9.88e-04  
[09/15 16:26:35] lb.utils.events INFO:  eta: 15:30:05  iteration: 26099/375342  consumed_samples: 26726400  total_loss: 4.051  time: 0.3997 s/iter  data_time: 0.2468 s/iter total_throughput: 2561.78 samples/s lr: 9.88e-04  
[09/15 16:27:14] lb.utils.events INFO:  eta: 15:28:02  iteration: 26199/375342  consumed_samples: 26828800  total_loss: 4.017  time: 0.3997 s/iter  data_time: 0.2488 s/iter total_throughput: 2561.84 samples/s lr: 9.88e-04  
[09/15 16:27:54] lb.utils.events INFO:  eta: 17:07:10  iteration: 26299/375342  consumed_samples: 26931200  total_loss: 4.002  time: 0.3997 s/iter  data_time: 0.2506 s/iter total_throughput: 2561.85 samples/s lr: 9.88e-04  
[09/15 16:28:34] lb.utils.events INFO:  eta: 16:51:22  iteration: 26399/375342  consumed_samples: 27033600  total_loss: 4.01  time: 0.3997 s/iter  data_time: 0.2467 s/iter total_throughput: 2561.85 samples/s lr: 9.88e-04  
[09/15 16:29:15] lb.utils.events INFO:  eta: 16:13:43  iteration: 26499/375342  consumed_samples: 27136000  total_loss: 4.016  time: 0.3997 s/iter  data_time: 0.2620 s/iter total_throughput: 2561.65 samples/s lr: 9.88e-04  
[09/15 16:29:55] lb.utils.events INFO:  eta: 15:22:34  iteration: 26599/375342  consumed_samples: 27238400  total_loss: 4.022  time: 0.3997 s/iter  data_time: 0.2518 s/iter total_throughput: 2561.65 samples/s lr: 9.88e-04  
[09/15 16:30:35] lb.utils.events INFO:  eta: 15:07:48  iteration: 26699/375342  consumed_samples: 27340800  total_loss: 4.028  time: 0.3997 s/iter  data_time: 0.2516 s/iter total_throughput: 2561.64 samples/s lr: 9.88e-04  
[09/15 16:31:15] lb.utils.events INFO:  eta: 15:09:51  iteration: 26799/375342  consumed_samples: 27443200  total_loss: 4.014  time: 0.3997 s/iter  data_time: 0.2476 s/iter total_throughput: 2561.66 samples/s lr: 9.88e-04  
[09/15 16:31:56] lb.utils.events INFO:  eta: 15:07:26  iteration: 26899/375342  consumed_samples: 27545600  total_loss: 4  time: 0.3998 s/iter  data_time: 0.2526 s/iter total_throughput: 2561.54 samples/s lr: 9.88e-04  
[09/15 16:32:35] lb.utils.events INFO:  eta: 15:03:55  iteration: 26999/375342  consumed_samples: 27648000  total_loss: 4.006  time: 0.3998 s/iter  data_time: 0.2403 s/iter total_throughput: 2561.55 samples/s lr: 9.87e-04  
[09/15 16:33:15] lb.utils.events INFO:  eta: 15:02:51  iteration: 27099/375342  consumed_samples: 27750400  total_loss: 4.018  time: 0.3997 s/iter  data_time: 0.2394 s/iter total_throughput: 2561.70 samples/s lr: 9.87e-04  
[09/15 16:33:55] lb.utils.events INFO:  eta: 14:57:56  iteration: 27199/375342  consumed_samples: 27852800  total_loss: 3.99  time: 0.3997 s/iter  data_time: 0.2585 s/iter total_throughput: 2561.61 samples/s lr: 9.87e-04  
[09/15 16:34:35] lb.utils.events INFO:  eta: 14:54:11  iteration: 27299/375342  consumed_samples: 27955200  total_loss: 4.002  time: 0.3997 s/iter  data_time: 0.2459 s/iter total_throughput: 2561.73 samples/s lr: 9.87e-04  
[09/15 16:35:14] lb.utils.events INFO:  eta: 14:54:20  iteration: 27399/375342  consumed_samples: 28057600  total_loss: 3.996  time: 0.3997 s/iter  data_time: 0.2542 s/iter total_throughput: 2561.81 samples/s lr: 9.87e-04  
[09/15 16:35:54] lb.utils.events INFO:  eta: 14:55:56  iteration: 27499/375342  consumed_samples: 28160000  total_loss: 3.99  time: 0.3997 s/iter  data_time: 0.2508 s/iter total_throughput: 2561.89 samples/s lr: 9.87e-04  
[09/15 16:36:35] lb.utils.events INFO:  eta: 14:55:41  iteration: 27599/375342  consumed_samples: 28262400  total_loss: 4.007  time: 0.3997 s/iter  data_time: 0.2483 s/iter total_throughput: 2561.72 samples/s lr: 9.87e-04  
[09/15 16:37:15] lb.utils.events INFO:  eta: 14:55:46  iteration: 27699/375342  consumed_samples: 28364800  total_loss: 3.972  time: 0.3997 s/iter  data_time: 0.2468 s/iter total_throughput: 2561.71 samples/s lr: 9.87e-04  
[09/15 16:37:54] lb.utils.events INFO:  eta: 14:55:03  iteration: 27799/375342  consumed_samples: 28467200  total_loss: 3.969  time: 0.3997 s/iter  data_time: 0.2520 s/iter total_throughput: 2561.85 samples/s lr: 9.87e-04  
[09/15 16:38:34] lb.utils.events INFO:  eta: 14:53:06  iteration: 27899/375342  consumed_samples: 28569600  total_loss: 3.971  time: 0.3997 s/iter  data_time: 0.2552 s/iter total_throughput: 2561.83 samples/s lr: 9.87e-04  
[09/15 16:39:14] lb.utils.events INFO:  eta: 14:52:32  iteration: 27999/375342  consumed_samples: 28672000  total_loss: 3.972  time: 0.3997 s/iter  data_time: 0.2581 s/iter total_throughput: 2561.81 samples/s lr: 9.86e-04  
[09/15 16:39:55] lb.utils.events INFO:  eta: 14:51:27  iteration: 28099/375342  consumed_samples: 28774400  total_loss: 3.967  time: 0.3997 s/iter  data_time: 0.2629 s/iter total_throughput: 2561.60 samples/s lr: 9.86e-04  
[09/15 16:40:35] lb.utils.events INFO:  eta: 14:52:12  iteration: 28199/375342  consumed_samples: 28876800  total_loss: 3.964  time: 0.3998 s/iter  data_time: 0.2547 s/iter total_throughput: 2561.54 samples/s lr: 9.86e-04  
[09/15 16:41:16] lb.utils.events INFO:  eta: 14:51:30  iteration: 28299/375342  consumed_samples: 28979200  total_loss: 3.977  time: 0.3998 s/iter  data_time: 0.2476 s/iter total_throughput: 2561.30 samples/s lr: 9.86e-04  
[09/15 16:41:58] lb.utils.events INFO:  eta: 14:49:56  iteration: 28399/375342  consumed_samples: 29081600  total_loss: 3.968  time: 0.3998 s/iter  data_time: 0.2758 s/iter total_throughput: 2561.05 samples/s lr: 9.86e-04  
[09/15 16:42:39] lb.utils.events INFO:  eta: 14:48:38  iteration: 28499/375342  consumed_samples: 29184000  total_loss: 3.957  time: 0.3999 s/iter  data_time: 0.2746 s/iter total_throughput: 2560.63 samples/s lr: 9.86e-04  
[09/15 16:43:21] lb.utils.events INFO:  eta: 14:49:31  iteration: 28599/375342  consumed_samples: 29286400  total_loss: 3.968  time: 0.4000 s/iter  data_time: 0.2609 s/iter total_throughput: 2560.21 samples/s lr: 9.86e-04  
[09/15 16:44:03] lb.utils.events INFO:  eta: 14:50:00  iteration: 28699/375342  consumed_samples: 29388800  total_loss: 3.97  time: 0.4000 s/iter  data_time: 0.2668 s/iter total_throughput: 2559.83 samples/s lr: 9.86e-04  
[09/15 16:44:44] lb.utils.events INFO:  eta: 14:53:45  iteration: 28799/375342  consumed_samples: 29491200  total_loss: 3.936  time: 0.4001 s/iter  data_time: 0.2569 s/iter total_throughput: 2559.63 samples/s lr: 9.86e-04  
[09/15 16:45:25] lb.utils.events INFO:  eta: 14:57:55  iteration: 28899/375342  consumed_samples: 29593600  total_loss: 3.927  time: 0.4001 s/iter  data_time: 0.2607 s/iter total_throughput: 2559.43 samples/s lr: 9.86e-04  
[09/15 16:46:06] lb.utils.events INFO:  eta: 15:04:04  iteration: 28999/375342  consumed_samples: 29696000  total_loss: 3.94  time: 0.4001 s/iter  data_time: 0.2682 s/iter total_throughput: 2559.21 samples/s lr: 9.85e-04  
[09/15 16:46:46] lb.utils.events INFO:  eta: 15:10:19  iteration: 29099/375342  consumed_samples: 29798400  total_loss: 3.934  time: 0.4001 s/iter  data_time: 0.2463 s/iter total_throughput: 2559.09 samples/s lr: 9.85e-04  
[09/15 16:47:27] lb.utils.events INFO:  eta: 15:27:23  iteration: 29199/375342  consumed_samples: 29900800  total_loss: 3.924  time: 0.4002 s/iter  data_time: 0.2597 s/iter total_throughput: 2558.95 samples/s lr: 9.85e-04  
[09/15 16:48:08] lb.utils.events INFO:  eta: 18:39:14  iteration: 29299/375342  consumed_samples: 30003200  total_loss: 3.923  time: 0.4002 s/iter  data_time: 0.2591 s/iter total_throughput: 2558.69 samples/s lr: 9.85e-04  
[09/15 16:48:50] lb.utils.events INFO:  eta: 21:51:43  iteration: 29399/375342  consumed_samples: 30105600  total_loss: 3.937  time: 0.4003 s/iter  data_time: 0.2679 s/iter total_throughput: 2558.37 samples/s lr: 9.85e-04  
[09/15 16:49:31] lb.utils.events INFO:  eta: 19:21:01  iteration: 29499/375342  consumed_samples: 30208000  total_loss: 3.952  time: 0.4003 s/iter  data_time: 0.2603 s/iter total_throughput: 2558.07 samples/s lr: 9.85e-04  
[09/15 16:50:12] lb.utils.events INFO:  eta: 23:53:04  iteration: 29599/375342  consumed_samples: 30310400  total_loss: 3.947  time: 0.4003 s/iter  data_time: 0.2721 s/iter total_throughput: 2557.87 samples/s lr: 9.85e-04  
[09/15 16:50:53] lb.utils.events INFO:  eta: 1 day, 2:18:31  iteration: 29699/375342  consumed_samples: 30412800  total_loss: 3.936  time: 0.4004 s/iter  data_time: 0.2520 s/iter total_throughput: 2557.71 samples/s lr: 9.85e-04  
[09/15 16:51:34] lb.utils.events INFO:  eta: 23:57:16  iteration: 29799/375342  consumed_samples: 30515200  total_loss: 3.912  time: 0.4004 s/iter  data_time: 0.2573 s/iter total_throughput: 2557.58 samples/s lr: 9.85e-04  
[09/15 16:52:14] lb.utils.events INFO:  eta: 20:04:50  iteration: 29899/375342  consumed_samples: 30617600  total_loss: 3.921  time: 0.4004 s/iter  data_time: 0.2621 s/iter total_throughput: 2557.45 samples/s lr: 9.85e-04  
[09/15 16:52:55] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_2e56/model_0029999
[09/15 16:52:56] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000
[09/15 16:52:56] lb.evaluation.evaluator INFO: Start inference on 50000 samples
[09/15 16:53:00] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.1090 s/iter. Inference: 0.1500 s/iter. Eval: 0.0023 s/iter. Total: 0.2613 s/iter. ETA=0:00:09
[09/15 16:53:05] lb.evaluation.evaluator INFO: Inference done 28672/50000. Dataloading: 0.1350 s/iter. Inference: 0.1534 s/iter. Eval: 0.0021 s/iter. Total: 0.2905 s/iter. ETA=0:00:05
[09/15 16:53:10] lb.evaluation.evaluator INFO: Inference done 45056/50000. Dataloading: 0.1465 s/iter. Inference: 0.1527 s/iter. Eval: 0.0021 s/iter. Total: 0.3014 s/iter. ETA=0:00:01
[09/15 16:53:12] lb.evaluation.evaluator INFO: Total valid samples: 50000
[09/15 16:53:12] lb.evaluation.evaluator INFO: Total inference time: 0:00:13.406830 (0.000268 s / iter per device, on 8 devices)
[09/15 16:53:12] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000135 s / iter per device, on 8 devices)
[09/15 16:53:12] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format:
[09/15 16:53:12] lb.evaluation.utils INFO: copypaste: Acc@1=55.728
[09/15 16:53:12] lb.evaluation.utils INFO: copypaste: Acc@5=79.296
[09/15 16:53:12] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 55.72800, better than last best score 52.53600 @ iteration 24999.
[09/15 16:53:12] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_2e56/model_best
[09/15 16:53:13] lb.utils.events INFO:  eta: 15:37:35  iteration: 29999/375342  consumed_samples: 30720000  total_loss: 3.931  time: 0.4004 s/iter  data_time: 0.2566 s/iter total_throughput: 2557.32 samples/s lr: 9.84e-04  
[09/15 16:53:51] lb.utils.events INFO:  eta: 16:55:25  iteration: 30099/375342  consumed_samples: 30822400  total_loss: 3.903  time: 0.4004 s/iter  data_time: 0.2620 s/iter total_throughput: 2557.73 samples/s lr: 9.84e-04  
[09/15 16:54:31] lb.utils.events INFO:  eta: 21:01:25  iteration: 30199/375342  consumed_samples: 30924800  total_loss: 3.911  time: 0.4004 s/iter  data_time: 0.2568 s/iter total_throughput: 2557.60 samples/s lr: 9.84e-04  
[09/15 16:55:12] lb.utils.events INFO:  eta: 20:33:51  iteration: 30299/375342  consumed_samples: 31027200  total_loss: 3.916  time: 0.4004 s/iter  data_time: 0.2509 s/iter total_throughput: 2557.56 samples/s lr: 9.84e-04  
[09/15 16:55:52] lb.utils.events INFO:  eta: 19:03:15  iteration: 30399/375342  consumed_samples: 31129600  total_loss: 3.901  time: 0.4004 s/iter  data_time: 0.2602 s/iter total_throughput: 2557.44 samples/s lr: 9.84e-04  
[09/15 16:56:33] lb.utils.events INFO:  eta: 20:50:20  iteration: 30499/375342  consumed_samples: 31232000  total_loss: 3.907  time: 0.4004 s/iter  data_time: 0.2660 s/iter total_throughput: 2557.24 samples/s lr: 9.84e-04  
[09/15 16:57:14] lb.utils.events INFO:  eta: 21:43:22  iteration: 30599/375342  consumed_samples: 31334400  total_loss: 3.909  time: 0.4004 s/iter  data_time: 0.2598 s/iter total_throughput: 2557.17 samples/s lr: 9.84e-04  
[09/15 16:57:55] lb.utils.events INFO:  eta: 22:46:16  iteration: 30699/375342  consumed_samples: 31436800  total_loss: 3.882  time: 0.4005 s/iter  data_time: 0.2619 s/iter total_throughput: 2556.96 samples/s lr: 9.84e-04  
[09/15 16:58:35] lb.utils.events INFO:  eta: 1 day, 0:27:43  iteration: 30799/375342  consumed_samples: 31539200  total_loss: 3.895  time: 0.4005 s/iter  data_time: 0.2541 s/iter total_throughput: 2556.89 samples/s lr: 9.84e-04  
[09/15 16:59:16] lb.utils.events INFO:  eta: 1 day, 2:29:36  iteration: 30899/375342  consumed_samples: 31641600  total_loss: 3.899  time: 0.4005 s/iter  data_time: 0.2530 s/iter total_throughput: 2556.68 samples/s lr: 9.84e-04  
[09/15 16:59:57] lb.utils.events INFO:  eta: 1 day, 5:05:39  iteration: 30999/375342  consumed_samples: 31744000  total_loss: 3.86  time: 0.4005 s/iter  data_time: 0.2614 s/iter total_throughput: 2556.56 samples/s lr: 9.83e-04  
[09/15 17:00:37] lb.utils.events INFO:  eta: 1 day, 4:05:08  iteration: 31099/375342  consumed_samples: 31846400  total_loss: 3.876  time: 0.4006 s/iter  data_time: 0.2599 s/iter total_throughput: 2556.46 samples/s lr: 9.83e-04  
[09/15 17:01:18] lb.utils.events INFO:  eta: 1 day, 2:15:32  iteration: 31199/375342  consumed_samples: 31948800  total_loss: 3.907  time: 0.4006 s/iter  data_time: 0.2599 s/iter total_throughput: 2556.36 samples/s lr: 9.83e-04  
[09/15 17:01:59] lb.utils.events INFO:  eta: 23:52:21  iteration: 31299/375342  consumed_samples: 32051200  total_loss: 3.903  time: 0.4006 s/iter  data_time: 0.2688 s/iter total_throughput: 2556.19 samples/s lr: 9.83e-04  
[09/15 17:02:40] lb.utils.events INFO:  eta: 1 day, 1:17:34  iteration: 31399/375342  consumed_samples: 32153600  total_loss: 3.868  time: 0.4006 s/iter  data_time: 0.2600 s/iter total_throughput: 2555.99 samples/s lr: 9.83e-04  
[09/15 17:03:20] lb.utils.events INFO:  eta: 1 day, 0:58:59  iteration: 31499/375342  consumed_samples: 32256000  total_loss: 3.876  time: 0.4006 s/iter  data_time: 0.2525 s/iter total_throughput: 2555.90 samples/s lr: 9.83e-04  
[09/15 17:04:01] lb.utils.events INFO:  eta: 20:40:18  iteration: 31599/375342  consumed_samples: 32358400  total_loss: 3.887  time: 0.4007 s/iter  data_time: 0.2485 s/iter total_throughput: 2555.81 samples/s lr: 9.83e-04  
[09/15 17:04:42] lb.utils.events INFO:  eta: 16:20:08  iteration: 31699/375342  consumed_samples: 32460800  total_loss: 3.875  time: 0.4007 s/iter  data_time: 0.2527 s/iter total_throughput: 2555.68 samples/s lr: 9.83e-04  
[09/15 17:05:23] lb.utils.events INFO:  eta: 15:03:03  iteration: 31799/375342  consumed_samples: 32563200  total_loss: 3.889  time: 0.4007 s/iter  data_time: 0.2532 s/iter total_throughput: 2555.50 samples/s lr: 9.83e-04  
[09/15 17:06:03] lb.utils.events INFO:  eta: 14:57:18  iteration: 31899/375342  consumed_samples: 32665600  total_loss: 3.887  time: 0.4007 s/iter  data_time: 0.2656 s/iter total_throughput: 2555.37 samples/s lr: 9.82e-04  
[09/15 17:06:44] lb.utils.events INFO:  eta: 14:52:53  iteration: 31999/375342  consumed_samples: 32768000  total_loss: 3.871  time: 0.4008 s/iter  data_time: 0.2656 s/iter total_throughput: 2555.15 samples/s lr: 9.82e-04  
[09/15 17:07:25] lb.utils.events INFO:  eta: 14:49:14  iteration: 32099/375342  consumed_samples: 32870400  total_loss: 3.857  time: 0.4008 s/iter  data_time: 0.2579 s/iter total_throughput: 2555.09 samples/s lr: 9.82e-04  
[09/15 17:08:06] lb.utils.events INFO:  eta: 14:47:41  iteration: 32199/375342  consumed_samples: 32972800  total_loss: 3.86  time: 0.4008 s/iter  data_time: 0.2553 s/iter total_throughput: 2554.97 samples/s lr: 9.82e-04  
[09/15 17:08:46] lb.utils.events INFO:  eta: 14:47:33  iteration: 32299/375342  consumed_samples: 33075200  total_loss: 3.859  time: 0.4008 s/iter  data_time: 0.2585 s/iter total_throughput: 2554.82 samples/s lr: 9.82e-04  
[09/15 17:09:27] lb.utils.events INFO:  eta: 14:47:10  iteration: 32399/375342  consumed_samples: 33177600  total_loss: 3.835  time: 0.4008 s/iter  data_time: 0.2616 s/iter total_throughput: 2554.75 samples/s lr: 9.82e-04  
[09/15 17:10:08] lb.utils.events INFO:  eta: 14:45:26  iteration: 32499/375342  consumed_samples: 33280000  total_loss: 3.865  time: 0.4009 s/iter  data_time: 0.2579 s/iter total_throughput: 2554.51 samples/s lr: 9.82e-04  
[09/15 17:10:49] lb.utils.events INFO:  eta: 14:44:00  iteration: 32599/375342  consumed_samples: 33382400  total_loss: 3.862  time: 0.4009 s/iter  data_time: 0.2645 s/iter total_throughput: 2554.32 samples/s lr: 9.82e-04  
[09/15 17:11:30] lb.utils.events INFO:  eta: 14:43:47  iteration: 32699/375342  consumed_samples: 33484800  total_loss: 3.852  time: 0.4009 s/iter  data_time: 0.2471 s/iter total_throughput: 2554.20 samples/s lr: 9.82e-04  
[09/15 17:12:11] lb.utils.events INFO:  eta: 14:43:25  iteration: 32799/375342  consumed_samples: 33587200  total_loss: 3.855  time: 0.4009 s/iter  data_time: 0.2587 s/iter total_throughput: 2554.09 samples/s lr: 9.81e-04  
[09/15 17:12:51] lb.utils.events INFO:  eta: 14:44:05  iteration: 32899/375342  consumed_samples: 33689600  total_loss: 3.801  time: 0.4009 s/iter  data_time: 0.2525 s/iter total_throughput: 2554.02 samples/s lr: 9.81e-04  
[09/15 17:13:32] lb.utils.events INFO:  eta: 14:46:08  iteration: 32999/375342  consumed_samples: 33792000  total_loss: 3.824  time: 0.4010 s/iter  data_time: 0.2561 s/iter total_throughput: 2553.90 samples/s lr: 9.81e-04  
[09/15 17:14:13] lb.utils.events INFO:  eta: 14:46:32  iteration: 33099/375342  consumed_samples: 33894400  total_loss: 3.88  time: 0.4010 s/iter  data_time: 0.2584 s/iter total_throughput: 2553.77 samples/s lr: 9.81e-04  
[09/15 17:14:53] lb.utils.events INFO:  eta: 14:46:38  iteration: 33199/375342  consumed_samples: 33996800  total_loss: 3.843  time: 0.4010 s/iter  data_time: 0.2497 s/iter total_throughput: 2553.72 samples/s lr: 9.81e-04  
[09/15 17:15:34] lb.utils.events INFO:  eta: 14:46:20  iteration: 33299/375342  consumed_samples: 34099200  total_loss: 3.832  time: 0.4010 s/iter  data_time: 0.2548 s/iter total_throughput: 2553.62 samples/s lr: 9.81e-04  
[09/15 17:16:15] lb.utils.events INFO:  eta: 14:46:10  iteration: 33399/375342  consumed_samples: 34201600  total_loss: 3.838  time: 0.4010 s/iter  data_time: 0.2671 s/iter total_throughput: 2553.46 samples/s lr: 9.81e-04  
[09/15 17:16:55] lb.utils.events INFO:  eta: 14:46:43  iteration: 33499/375342  consumed_samples: 34304000  total_loss: 3.826  time: 0.4010 s/iter  data_time: 0.2574 s/iter total_throughput: 2553.34 samples/s lr: 9.81e-04  
[09/15 17:17:36] lb.utils.events INFO:  eta: 14:50:38  iteration: 33599/375342  consumed_samples: 34406400  total_loss: 3.832  time: 0.4011 s/iter  data_time: 0.2521 s/iter total_throughput: 2553.25 samples/s lr: 9.81e-04  
[09/15 17:18:17] lb.utils.events INFO:  eta: 14:54:06  iteration: 33699/375342  consumed_samples: 34508800  total_loss: 3.843  time: 0.4011 s/iter  data_time: 0.2519 s/iter total_throughput: 2553.15 samples/s lr: 9.80e-04  
[09/15 17:18:57] lb.utils.events INFO:  eta: 15:09:33  iteration: 33799/375342  consumed_samples: 34611200  total_loss: 3.845  time: 0.4011 s/iter  data_time: 0.2431 s/iter total_throughput: 2553.16 samples/s lr: 9.80e-04  
[09/15 17:19:37] lb.utils.events INFO:  eta: 15:25:09  iteration: 33899/375342  consumed_samples: 34713600  total_loss: 3.807  time: 0.4011 s/iter  data_time: 0.2554 s/iter total_throughput: 2553.02 samples/s lr: 9.80e-04  
[09/15 17:20:18] lb.utils.events INFO:  eta: 15:22:28  iteration: 33999/375342  consumed_samples: 34816000  total_loss: 3.794  time: 0.4011 s/iter  data_time: 0.2571 s/iter total_throughput: 2553.00 samples/s lr: 9.80e-04  
[09/15 17:20:58] lb.utils.events INFO:  eta: 15:10:47  iteration: 34099/375342  consumed_samples: 34918400  total_loss: 3.793  time: 0.4011 s/iter  data_time: 0.2559 s/iter total_throughput: 2552.88 samples/s lr: 9.80e-04  
[09/15 17:21:39] lb.utils.events INFO:  eta: 14:56:40  iteration: 34199/375342  consumed_samples: 35020800  total_loss: 3.803  time: 0.4011 s/iter  data_time: 0.2583 s/iter total_throughput: 2552.76 samples/s lr: 9.80e-04  
[09/15 17:22:20] lb.utils.events INFO:  eta: 14:48:40  iteration: 34299/375342  consumed_samples: 35123200  total_loss: 3.817  time: 0.4012 s/iter  data_time: 0.2579 s/iter total_throughput: 2552.58 samples/s lr: 9.80e-04  
[09/15 17:23:01] lb.utils.events INFO:  eta: 14:45:27  iteration: 34399/375342  consumed_samples: 35225600  total_loss: 3.825  time: 0.4012 s/iter  data_time: 0.2574 s/iter total_throughput: 2552.55 samples/s lr: 9.80e-04  
[09/15 17:23:41] lb.utils.events INFO:  eta: 14:41:55  iteration: 34499/375342  consumed_samples: 35328000  total_loss: 3.812  time: 0.4012 s/iter  data_time: 0.2456 s/iter total_throughput: 2552.53 samples/s lr: 9.80e-04  
[09/15 17:24:22] lb.utils.events INFO:  eta: 14:41:07  iteration: 34599/375342  consumed_samples: 35430400  total_loss: 3.793  time: 0.4012 s/iter  data_time: 0.2669 s/iter total_throughput: 2552.42 samples/s lr: 9.79e-04  
[09/15 17:25:02] lb.utils.events INFO:  eta: 14:40:41  iteration: 34699/375342  consumed_samples: 35532800  total_loss: 3.799  time: 0.4012 s/iter  data_time: 0.2472 s/iter total_throughput: 2552.39 samples/s lr: 9.79e-04  
[09/15 17:25:42] lb.utils.events INFO:  eta: 14:40:33  iteration: 34799/375342  consumed_samples: 35635200  total_loss: 3.823  time: 0.4012 s/iter  data_time: 0.2639 s/iter total_throughput: 2552.30 samples/s lr: 9.79e-04  
[09/15 17:26:23] lb.utils.events INFO:  eta: 14:40:57  iteration: 34899/375342  consumed_samples: 35737600  total_loss: 3.811  time: 0.4012 s/iter  data_time: 0.2734 s/iter total_throughput: 2552.26 samples/s lr: 9.79e-04  
[09/15 17:27:04] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_2e56/model_0034999
[09/15 17:27:04] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000
[09/15 17:27:04] lb.evaluation.evaluator INFO: Start inference on 50000 samples
[09/15 17:27:09] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.1210 s/iter. Inference: 0.1496 s/iter. Eval: 0.0022 s/iter. Total: 0.2727 s/iter. ETA=0:00:10
[09/15 17:27:14] lb.evaluation.evaluator INFO: Inference done 27648/50000. Dataloading: 0.1525 s/iter. Inference: 0.1487 s/iter. Eval: 0.0021 s/iter. Total: 0.3035 s/iter. ETA=0:00:06
[09/15 17:27:19] lb.evaluation.evaluator INFO: Inference done 45056/50000. Dataloading: 0.1482 s/iter. Inference: 0.1493 s/iter. Eval: 0.0022 s/iter. Total: 0.2998 s/iter. ETA=0:00:01
[09/15 17:27:20] lb.evaluation.evaluator INFO: Total valid samples: 50000
[09/15 17:27:20] lb.evaluation.evaluator INFO: Total inference time: 0:00:13.338435 (0.000267 s / iter per device, on 8 devices)
[09/15 17:27:20] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000132 s / iter per device, on 8 devices)
[09/15 17:27:20] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format:
[09/15 17:27:20] lb.evaluation.utils INFO: copypaste: Acc@1=57.824
[09/15 17:27:20] lb.evaluation.utils INFO: copypaste: Acc@5=81.17
[09/15 17:27:20] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 57.82400, better than last best score 55.72800 @ iteration 29999.
[09/15 17:27:20] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_2e56/model_best
[09/15 17:27:21] lb.utils.events INFO:  eta: 14:41:23  iteration: 34999/375342  consumed_samples: 35840000  total_loss: 3.801  time: 0.4012 s/iter  data_time: 0.2559 s/iter total_throughput: 2552.15 samples/s lr: 9.79e-04  
[09/15 17:28:00] lb.utils.events INFO:  eta: 14:43:52  iteration: 35099/375342  consumed_samples: 35942400  total_loss: 3.793  time: 0.4012 s/iter  data_time: 0.2426 s/iter total_throughput: 2552.44 samples/s lr: 9.79e-04  
[09/15 17:28:40] lb.utils.events INFO:  eta: 14:45:56  iteration: 35199/375342  consumed_samples: 36044800  total_loss: 3.79  time: 0.4012 s/iter  data_time: 0.2574 s/iter total_throughput: 2552.33 samples/s lr: 9.79e-04  
[09/15 17:29:21] lb.utils.events INFO:  eta: 14:46:26  iteration: 35299/375342  consumed_samples: 36147200  total_loss: 3.783  time: 0.4012 s/iter  data_time: 0.2509 s/iter total_throughput: 2552.26 samples/s lr: 9.79e-04  
[09/15 17:30:02] lb.utils.events INFO:  eta: 14:46:08  iteration: 35399/375342  consumed_samples: 36249600  total_loss: 3.771  time: 0.4012 s/iter  data_time: 0.2642 s/iter total_throughput: 2552.10 samples/s lr: 9.78e-04  
[09/15 17:30:42] lb.utils.events INFO:  eta: 14:48:08  iteration: 35499/375342  consumed_samples: 36352000  total_loss: 3.758  time: 0.4012 s/iter  data_time: 0.2588 s/iter total_throughput: 2552.07 samples/s lr: 9.78e-04  
[09/15 17:31:23] lb.utils.events INFO:  eta: 14:46:21  iteration: 35599/375342  consumed_samples: 36454400  total_loss: 3.766  time: 0.4012 s/iter  data_time: 0.2507 s/iter total_throughput: 2552.03 samples/s lr: 9.78e-04  
[09/15 17:32:04] lb.utils.events INFO:  eta: 14:44:33  iteration: 35699/375342  consumed_samples: 36556800  total_loss: 3.779  time: 0.4013 s/iter  data_time: 0.2693 s/iter total_throughput: 2551.87 samples/s lr: 9.78e-04  
[09/15 17:32:44] lb.utils.events INFO:  eta: 14:41:16  iteration: 35799/375342  consumed_samples: 36659200  total_loss: 3.785  time: 0.4013 s/iter  data_time: 0.2627 s/iter total_throughput: 2551.78 samples/s lr: 9.78e-04  
[09/15 17:33:25] lb.utils.events INFO:  eta: 14:38:01  iteration: 35899/375342  consumed_samples: 36761600  total_loss: 3.807  time: 0.4013 s/iter  data_time: 0.2499 s/iter total_throughput: 2551.70 samples/s lr: 9.78e-04  
[09/15 17:34:05] lb.utils.events INFO:  eta: 14:36:10  iteration: 35999/375342  consumed_samples: 36864000  total_loss: 3.797  time: 0.4013 s/iter  data_time: 0.2540 s/iter total_throughput: 2551.65 samples/s lr: 9.78e-04  
[09/15 17:34:46] lb.utils.events INFO:  eta: 14:34:15  iteration: 36099/375342  consumed_samples: 36966400  total_loss: 3.772  time: 0.4013 s/iter  data_time: 0.2566 s/iter total_throughput: 2551.57 samples/s lr: 9.78e-04  
[09/15 17:35:26] lb.utils.events INFO:  eta: 14:34:00  iteration: 36199/375342  consumed_samples: 37068800  total_loss: 3.751  time: 0.4013 s/iter  data_time: 0.2566 s/iter total_throughput: 2551.55 samples/s lr: 9.77e-04  
[09/15 17:36:07] lb.utils.events INFO:  eta: 14:33:52  iteration: 36299/375342  consumed_samples: 37171200  total_loss: 3.753  time: 0.4013 s/iter  data_time: 0.2650 s/iter total_throughput: 2551.52 samples/s lr: 9.77e-04  
[09/15 17:36:47] lb.utils.events INFO:  eta: 14:33:58  iteration: 36399/375342  consumed_samples: 37273600  total_loss: 3.771  time: 0.4013 s/iter  data_time: 0.2558 s/iter total_throughput: 2551.44 samples/s lr: 9.77e-04  
[09/15 17:37:27] lb.utils.events INFO:  eta: 14:31:56  iteration: 36499/375342  consumed_samples: 37376000  total_loss: 3.767  time: 0.4013 s/iter  data_time: 0.2555 s/iter total_throughput: 2551.44 samples/s lr: 9.77e-04  
[09/15 17:38:08] lb.utils.events INFO:  eta: 14:32:01  iteration: 36599/375342  consumed_samples: 37478400  total_loss: 3.76  time: 0.4013 s/iter  data_time: 0.2514 s/iter total_throughput: 2551.42 samples/s lr: 9.77e-04  
[09/15 17:38:48] lb.utils.events INFO:  eta: 14:32:58  iteration: 36699/375342  consumed_samples: 37580800  total_loss: 3.753  time: 0.4013 s/iter  data_time: 0.2530 s/iter total_throughput: 2551.43 samples/s lr: 9.77e-04  
[09/15 17:39:28] lb.utils.events INFO:  eta: 14:33:53  iteration: 36799/375342  consumed_samples: 37683200  total_loss: 3.758  time: 0.4013 s/iter  data_time: 0.2524 s/iter total_throughput: 2551.44 samples/s lr: 9.77e-04  
[09/15 17:40:08] lb.utils.events INFO:  eta: 14:33:33  iteration: 36899/375342  consumed_samples: 37785600  total_loss: 3.761  time: 0.4014 s/iter  data_time: 0.2612 s/iter total_throughput: 2551.34 samples/s lr: 9.77e-04  
[09/15 17:40:49] lb.utils.events INFO:  eta: 14:33:05  iteration: 36999/375342  consumed_samples: 37888000  total_loss: 3.75  time: 0.4014 s/iter  data_time: 0.2543 s/iter total_throughput: 2551.23 samples/s lr: 9.76e-04  
[09/15 17:41:30] lb.utils.events INFO:  eta: 14:32:49  iteration: 37099/375342  consumed_samples: 37990400  total_loss: 3.752  time: 0.4014 s/iter  data_time: 0.2581 s/iter total_throughput: 2551.15 samples/s lr: 9.76e-04  
[09/15 17:42:10] lb.utils.events INFO:  eta: 14:31:43  iteration: 37199/375342  consumed_samples: 38092800  total_loss: 3.761  time: 0.4014 s/iter  data_time: 0.2507 s/iter total_throughput: 2551.06 samples/s lr: 9.76e-04  
[09/15 17:42:51] lb.utils.events INFO:  eta: 14:31:29  iteration: 37299/375342  consumed_samples: 38195200  total_loss: 3.748  time: 0.4014 s/iter  data_time: 0.2561 s/iter total_throughput: 2551.05 samples/s lr: 9.76e-04  
[09/15 17:43:31] lb.utils.events INFO:  eta: 14:31:12  iteration: 37399/375342  consumed_samples: 38297600  total_loss: 3.747  time: 0.4014 s/iter  data_time: 0.2628 s/iter total_throughput: 2551.00 samples/s lr: 9.76e-04  
[09/15 17:44:11] lb.utils.events INFO:  eta: 14:31:34  iteration: 37499/375342  consumed_samples: 38400000  total_loss: 3.755  time: 0.4014 s/iter  data_time: 0.2560 s/iter total_throughput: 2550.99 samples/s lr: 9.76e-04  
[09/15 17:44:52] lb.utils.events INFO:  eta: 14:31:08  iteration: 37599/375342  consumed_samples: 38502400  total_loss: 3.747  time: 0.4014 s/iter  data_time: 0.2599 s/iter total_throughput: 2550.99 samples/s lr: 9.76e-04  
[09/15 17:45:32] lb.utils.events INFO:  eta: 14:29:43  iteration: 37699/375342  consumed_samples: 38604800  total_loss: 3.746  time: 0.4014 s/iter  data_time: 0.2567 s/iter total_throughput: 2550.90 samples/s lr: 9.76e-04  
[09/15 17:46:13] lb.utils.events INFO:  eta: 14:28:35  iteration: 37799/375342  consumed_samples: 38707200  total_loss: 3.739  time: 0.4014 s/iter  data_time: 0.2630 s/iter total_throughput: 2550.81 samples/s lr: 9.75e-04  
[09/15 17:46:53] lb.utils.events INFO:  eta: 14:28:36  iteration: 37899/375342  consumed_samples: 38809600  total_loss: 3.733  time: 0.4014 s/iter  data_time: 0.2549 s/iter total_throughput: 2550.81 samples/s lr: 9.75e-04  
[09/15 17:47:33] lb.utils.events INFO:  eta: 14:28:53  iteration: 37999/375342  consumed_samples: 38912000  total_loss: 3.711  time: 0.4014 s/iter  data_time: 0.2441 s/iter total_throughput: 2550.84 samples/s lr: 9.75e-04  
[09/15 17:48:13] lb.utils.events INFO:  eta: 14:30:04  iteration: 38099/375342  consumed_samples: 39014400  total_loss: 3.736  time: 0.4014 s/iter  data_time: 0.2483 s/iter total_throughput: 2550.79 samples/s lr: 9.75e-04  
[09/15 17:48:54] lb.utils.events INFO:  eta: 14:34:07  iteration: 38199/375342  consumed_samples: 39116800  total_loss: 3.745  time: 0.4014 s/iter  data_time: 0.2497 s/iter total_throughput: 2550.78 samples/s lr: 9.75e-04  
[09/15 17:49:34] lb.utils.events INFO:  eta: 14:36:51  iteration: 38299/375342  consumed_samples: 39219200  total_loss: 3.713  time: 0.4014 s/iter  data_time: 0.2549 s/iter total_throughput: 2550.77 samples/s lr: 9.75e-04  
[09/15 17:50:14] lb.utils.events INFO:  eta: 14:44:15  iteration: 38399/375342  consumed_samples: 39321600  total_loss: 3.711  time: 0.4015 s/iter  data_time: 0.2536 s/iter total_throughput: 2550.75 samples/s lr: 9.75e-04  
[09/15 17:50:55] lb.utils.events INFO:  eta: 15:14:54  iteration: 38499/375342  consumed_samples: 39424000  total_loss: 3.73  time: 0.4015 s/iter  data_time: 0.2517 s/iter total_throughput: 2550.71 samples/s lr: 9.75e-04  
[09/15 17:51:35] lb.utils.events INFO:  eta: 17:45:53  iteration: 38599/375342  consumed_samples: 39526400  total_loss: 3.74  time: 0.4015 s/iter  data_time: 0.2393 s/iter total_throughput: 2550.74 samples/s lr: 9.74e-04  
[09/15 17:52:15] lb.utils.events INFO:  eta: 20:45:05  iteration: 38699/375342  consumed_samples: 39628800  total_loss: 3.714  time: 0.4015 s/iter  data_time: 0.2434 s/iter total_throughput: 2550.71 samples/s lr: 9.74e-04  
[09/15 17:52:55] lb.utils.events INFO:  eta: 1 day, 2:33:56  iteration: 38799/375342  consumed_samples: 39731200  total_loss: 3.699  time: 0.4014 s/iter  data_time: 0.2510 s/iter total_throughput: 2550.76 samples/s lr: 9.74e-04  
[09/15 17:53:35] lb.utils.events INFO:  eta: 1 day, 4:34:22  iteration: 38899/375342  consumed_samples: 39833600  total_loss: 3.731  time: 0.4015 s/iter  data_time: 0.2570 s/iter total_throughput: 2550.67 samples/s lr: 9.74e-04  
[09/15 17:54:16] lb.utils.events INFO:  eta: 1 day, 4:27:25  iteration: 38999/375342  consumed_samples: 39936000  total_loss: 3.727  time: 0.4015 s/iter  data_time: 0.2608 s/iter total_throughput: 2550.64 samples/s lr: 9.74e-04  
[09/15 17:54:56] lb.utils.events INFO:  eta: 1 day, 3:04:13  iteration: 39099/375342  consumed_samples: 40038400  total_loss: 3.673  time: 0.4015 s/iter  data_time: 0.2595 s/iter total_throughput: 2550.64 samples/s lr: 9.74e-04  
[09/15 17:55:36] lb.utils.events INFO:  eta: 1 day, 1:05:26  iteration: 39199/375342  consumed_samples: 40140800  total_loss: 3.712  time: 0.4015 s/iter  data_time: 0.2495 s/iter total_throughput: 2550.70 samples/s lr: 9.74e-04  
[09/15 17:56:16] lb.utils.events INFO:  eta: 1 day, 1:37:30  iteration: 39299/375342  consumed_samples: 40243200  total_loss: 3.722  time: 0.4015 s/iter  data_time: 0.2526 s/iter total_throughput: 2550.71 samples/s lr: 9.73e-04  
[09/15 17:56:56] lb.utils.events INFO:  eta: 1 day, 0:58:02  iteration: 39399/375342  consumed_samples: 40345600  total_loss: 3.702  time: 0.4015 s/iter  data_time: 0.2491 s/iter total_throughput: 2550.70 samples/s lr: 9.73e-04  
[09/15 17:57:36] lb.utils.events INFO:  eta: 22:01:27  iteration: 39499/375342  consumed_samples: 40448000  total_loss: 3.712  time: 0.4014 s/iter  data_time: 0.2454 s/iter total_throughput: 2550.76 samples/s lr: 9.73e-04  
[09/15 17:58:16] lb.utils.events INFO:  eta: 22:01:03  iteration: 39599/375342  consumed_samples: 40550400  total_loss: 3.718  time: 0.4015 s/iter  data_time: 0.2559 s/iter total_throughput: 2550.75 samples/s lr: 9.73e-04  
[09/15 17:58:57] lb.utils.events INFO:  eta: 21:20:23  iteration: 39699/375342  consumed_samples: 40652800  total_loss: 3.695  time: 0.4015 s/iter  data_time: 0.2481 s/iter total_throughput: 2550.69 samples/s lr: 9.73e-04  
[09/15 17:59:37] lb.utils.events INFO:  eta: 14:56:21  iteration: 39799/375342  consumed_samples: 40755200  total_loss: 3.703  time: 0.4015 s/iter  data_time: 0.2638 s/iter total_throughput: 2550.61 samples/s lr: 9.73e-04  
[09/15 18:00:18] lb.utils.events INFO:  eta: 14:35:48  iteration: 39899/375342  consumed_samples: 40857600  total_loss: 3.708  time: 0.4015 s/iter  data_time: 0.2453 s/iter total_throughput: 2550.55 samples/s lr: 9.73e-04  
[09/15 18:00:58] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_2e56/model_0039999
[09/15 18:00:59] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000
[09/15 18:00:59] lb.evaluation.evaluator INFO: Start inference on 50000 samples
[09/15 18:01:03] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.1063 s/iter. Inference: 0.1501 s/iter. Eval: 0.0023 s/iter. Total: 0.2587 s/iter. ETA=0:00:09
[09/15 18:01:08] lb.evaluation.evaluator INFO: Inference done 28672/50000. Dataloading: 0.1112 s/iter. Inference: 0.1727 s/iter. Eval: 0.0021 s/iter. Total: 0.2862 s/iter. ETA=0:00:05
[09/15 18:01:13] lb.evaluation.evaluator INFO: Inference done 46080/50000. Dataloading: 0.1049 s/iter. Inference: 0.1909 s/iter. Eval: 0.0021 s/iter. Total: 0.2980 s/iter. ETA=0:00:00
[09/15 18:01:14] lb.evaluation.evaluator INFO: Total valid samples: 50000
[09/15 18:01:14] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.964934 (0.000259 s / iter per device, on 8 devices)
[09/15 18:01:14] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:08 (0.000170 s / iter per device, on 8 devices)
[09/15 18:01:14] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format:
[09/15 18:01:14] lb.evaluation.utils INFO: copypaste: Acc@1=59.836
[09/15 18:01:14] lb.evaluation.utils INFO: copypaste: Acc@5=82.61800000000001
[09/15 18:01:14] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 59.83600, better than last best score 57.82400 @ iteration 34999.
[09/15 18:01:14] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_2e56/model_best
[09/15 18:01:15] lb.utils.events INFO:  eta: 14:34:34  iteration: 39999/375342  consumed_samples: 40960000  total_loss: 3.688  time: 0.4015 s/iter  data_time: 0.2578 s/iter total_throughput: 2550.52 samples/s lr: 9.73e-04  
[09/15 18:01:54] lb.utils.events INFO:  eta: 14:44:47  iteration: 40099/375342  consumed_samples: 41062400  total_loss: 3.672  time: 0.4015 s/iter  data_time: 0.2713 s/iter total_throughput: 2550.71 samples/s lr: 9.72e-04  
[09/15 18:02:34] lb.utils.events INFO:  eta: 14:32:16  iteration: 40199/375342  consumed_samples: 41164800  total_loss: 3.681  time: 0.4015 s/iter  data_time: 0.2594 s/iter total_throughput: 2550.73 samples/s lr: 9.72e-04  
[09/15 18:03:13] lb.utils.events INFO:  eta: 14:26:54  iteration: 40299/375342  consumed_samples: 41267200  total_loss: 3.692  time: 0.4014 s/iter  data_time: 0.2378 s/iter total_throughput: 2550.87 samples/s lr: 9.72e-04  
[09/15 18:03:54] lb.utils.events INFO:  eta: 14:26:31  iteration: 40399/375342  consumed_samples: 41369600  total_loss: 3.689  time: 0.4014 s/iter  data_time: 0.2415 s/iter total_throughput: 2550.84 samples/s lr: 9.72e-04  
[09/15 18:04:34] lb.utils.events INFO:  eta: 14:27:40  iteration: 40499/375342  consumed_samples: 41472000  total_loss: 3.713  time: 0.4014 s/iter  data_time: 0.2483 s/iter total_throughput: 2550.86 samples/s lr: 9.72e-04  
[09/15 18:05:14] lb.utils.events INFO:  eta: 14:29:29  iteration: 40599/375342  consumed_samples: 41574400  total_loss: 3.714  time: 0.4014 s/iter  data_time: 0.2458 s/iter total_throughput: 2550.83 samples/s lr: 9.72e-04  
[09/15 18:05:54] lb.utils.events INFO:  eta: 14:32:59  iteration: 40699/375342  consumed_samples: 41676800  total_loss: 3.705  time: 0.4014 s/iter  data_time: 0.2488 s/iter total_throughput: 2550.83 samples/s lr: 9.72e-04  
[09/15 18:06:34] lb.utils.events INFO:  eta: 14:40:29  iteration: 40799/375342  consumed_samples: 41779200  total_loss: 3.706  time: 0.4014 s/iter  data_time: 0.2452 s/iter total_throughput: 2550.89 samples/s lr: 9.71e-04  
[09/15 18:07:14] lb.utils.events INFO:  eta: 14:38:47  iteration: 40899/375342  consumed_samples: 41881600  total_loss: 3.692  time: 0.4014 s/iter  data_time: 0.2446 s/iter total_throughput: 2550.92 samples/s lr: 9.71e-04  
[09/15 18:07:54] lb.utils.events INFO:  eta: 14:42:16  iteration: 40999/375342  consumed_samples: 41984000  total_loss: 3.705  time: 0.4014 s/iter  data_time: 0.2481 s/iter total_throughput: 2551.00 samples/s lr: 9.71e-04  
[09/15 18:08:34] lb.utils.events INFO:  eta: 14:31:28  iteration: 41099/375342  consumed_samples: 42086400  total_loss: 3.715  time: 0.4014 s/iter  data_time: 0.2469 s/iter total_throughput: 2551.05 samples/s lr: 9.71e-04  
[09/15 18:09:13] lb.utils.events INFO:  eta: 14:32:30  iteration: 41199/375342  consumed_samples: 42188800  total_loss: 3.699  time: 0.4014 s/iter  data_time: 0.2564 s/iter total_throughput: 2551.08 samples/s lr: 9.71e-04  
[09/15 18:09:53] lb.utils.events INFO:  eta: 14:33:48  iteration: 41299/375342  consumed_samples: 42291200  total_loss: 3.675  time: 0.4014 s/iter  data_time: 0.2438 s/iter total_throughput: 2551.11 samples/s lr: 9.71e-04  
[09/15 18:10:33] lb.utils.events INFO:  eta: 14:27:15  iteration: 41399/375342  consumed_samples: 42393600  total_loss: 3.672  time: 0.4014 s/iter  data_time: 0.2569 s/iter total_throughput: 2551.12 samples/s lr: 9.71e-04  
[09/15 18:11:13] lb.utils.events INFO:  eta: 14:25:52  iteration: 41499/375342  consumed_samples: 42496000  total_loss: 3.672  time: 0.4014 s/iter  data_time: 0.2583 s/iter total_throughput: 2551.17 samples/s lr: 9.70e-04  
[09/15 18:11:53] lb.utils.events INFO:  eta: 14:23:15  iteration: 41599/375342  consumed_samples: 42598400  total_loss: 3.674  time: 0.4014 s/iter  data_time: 0.2472 s/iter total_throughput: 2551.19 samples/s lr: 9.70e-04  
[09/15 18:12:33] lb.utils.events INFO:  eta: 14:23:35  iteration: 41699/375342  consumed_samples: 42700800  total_loss: 3.684  time: 0.4014 s/iter  data_time: 0.2572 s/iter total_throughput: 2551.22 samples/s lr: 9.70e-04  
[09/15 18:13:13] lb.utils.events INFO:  eta: 14:22:27  iteration: 41799/375342  consumed_samples: 42803200  total_loss: 3.669  time: 0.4014 s/iter  data_time: 0.2556 s/iter total_throughput: 2551.22 samples/s lr: 9.70e-04  
[09/15 18:13:53] lb.utils.events INFO:  eta: 14:24:52  iteration: 41899/375342  consumed_samples: 42905600  total_loss: 3.67  time: 0.4014 s/iter  data_time: 0.2461 s/iter total_throughput: 2551.30 samples/s lr: 9.70e-04  
[09/15 18:14:33] lb.utils.events INFO:  eta: 14:26:08  iteration: 41999/375342  consumed_samples: 43008000  total_loss: 3.676  time: 0.4014 s/iter  data_time: 0.2468 s/iter total_throughput: 2551.30 samples/s lr: 9.70e-04  
[09/15 18:15:13] lb.utils.events INFO:  eta: 14:27:31  iteration: 42099/375342  consumed_samples: 43110400  total_loss: 3.673  time: 0.4014 s/iter  data_time: 0.2539 s/iter total_throughput: 2551.31 samples/s lr: 9.70e-04  
[09/15 18:15:54] lb.utils.events INFO:  eta: 14:31:58  iteration: 42199/375342  consumed_samples: 43212800  total_loss: 3.678  time: 0.4014 s/iter  data_time: 0.2564 s/iter total_throughput: 2551.28 samples/s lr: 9.69e-04  
[09/15 18:16:34] lb.utils.events INFO:  eta: 14:30:35  iteration: 42299/375342  consumed_samples: 43315200  total_loss: 3.672  time: 0.4014 s/iter  data_time: 0.2538 s/iter total_throughput: 2551.20 samples/s lr: 9.69e-04  
[09/15 18:17:15] lb.utils.events INFO:  eta: 14:35:06  iteration: 42399/375342  consumed_samples: 43417600  total_loss: 3.65  time: 0.4014 s/iter  data_time: 0.2585 s/iter total_throughput: 2551.11 samples/s lr: 9.69e-04  
[09/15 18:17:57] lb.utils.events INFO:  eta: 14:35:19  iteration: 42499/375342  consumed_samples: 43520000  total_loss: 3.646  time: 0.4014 s/iter  data_time: 0.2803 s/iter total_throughput: 2550.84 samples/s lr: 9.69e-04  
[09/15 18:18:39] lb.utils.events INFO:  eta: 14:27:28  iteration: 42599/375342  consumed_samples: 43622400  total_loss: 3.667  time: 0.4015 s/iter  data_time: 0.2701 s/iter total_throughput: 2550.51 samples/s lr: 9.69e-04  
[09/15 18:19:21] lb.utils.events INFO:  eta: 14:24:56  iteration: 42699/375342  consumed_samples: 43724800  total_loss: 3.669  time: 0.4015 s/iter  data_time: 0.2609 s/iter total_throughput: 2550.26 samples/s lr: 9.69e-04  
[09/15 18:20:03] lb.utils.events INFO:  eta: 14:29:34  iteration: 42799/375342  consumed_samples: 43827200  total_loss: 3.645  time: 0.4016 s/iter  data_time: 0.2616 s/iter total_throughput: 2550.08 samples/s lr: 9.69e-04  
[09/15 18:20:43] lb.utils.events INFO:  eta: 14:33:34  iteration: 42899/375342  consumed_samples: 43929600  total_loss: 3.65  time: 0.4016 s/iter  data_time: 0.2602 s/iter total_throughput: 2550.02 samples/s lr: 9.68e-04  
[09/15 18:21:24] lb.utils.events INFO:  eta: 14:29:16  iteration: 42999/375342  consumed_samples: 44032000  total_loss: 3.646  time: 0.4016 s/iter  data_time: 0.2576 s/iter total_throughput: 2549.89 samples/s lr: 9.68e-04  
[09/15 18:22:05] lb.utils.events INFO:  eta: 14:25:33  iteration: 43099/375342  consumed_samples: 44134400  total_loss: 3.643  time: 0.4016 s/iter  data_time: 0.2506 s/iter total_throughput: 2549.83 samples/s lr: 9.68e-04  
[09/15 18:22:45] lb.utils.events INFO:  eta: 14:25:36  iteration: 43199/375342  consumed_samples: 44236800  total_loss: 3.647  time: 0.4016 s/iter  data_time: 0.2620 s/iter total_throughput: 2549.78 samples/s lr: 9.68e-04  
[09/15 18:23:27] lb.utils.events INFO:  eta: 14:24:45  iteration: 43299/375342  consumed_samples: 44339200  total_loss: 3.649  time: 0.4016 s/iter  data_time: 0.2609 s/iter total_throughput: 2549.61 samples/s lr: 9.68e-04  
[09/15 18:24:08] lb.utils.events INFO:  eta: 14:23:16  iteration: 43399/375342  consumed_samples: 44441600  total_loss: 3.666  time: 0.4017 s/iter  data_time: 0.2620 s/iter total_throughput: 2549.46 samples/s lr: 9.68e-04  
[09/15 18:24:50] lb.utils.events INFO:  eta: 14:24:18  iteration: 43499/375342  consumed_samples: 44544000  total_loss: 3.671  time: 0.4017 s/iter  data_time: 0.2739 s/iter total_throughput: 2549.20 samples/s lr: 9.68e-04  
[09/15 18:25:31] lb.utils.events INFO:  eta: 14:26:06  iteration: 43599/375342  consumed_samples: 44646400  total_loss: 3.669  time: 0.4017 s/iter  data_time: 0.2590 s/iter total_throughput: 2548.99 samples/s lr: 9.67e-04  
[09/15 18:26:12] lb.utils.events INFO:  eta: 14:24:38  iteration: 43699/375342  consumed_samples: 44748800  total_loss: 3.647  time: 0.4017 s/iter  data_time: 0.2469 s/iter total_throughput: 2548.90 samples/s lr: 9.67e-04  
[09/15 18:26:52] lb.utils.events INFO:  eta: 14:21:40  iteration: 43799/375342  consumed_samples: 44851200  total_loss: 3.653  time: 0.4017 s/iter  data_time: 0.2522 s/iter total_throughput: 2548.91 samples/s lr: 9.67e-04  
[09/15 18:27:33] lb.utils.events INFO:  eta: 14:17:14  iteration: 43899/375342  consumed_samples: 44953600  total_loss: 3.646  time: 0.4018 s/iter  data_time: 0.2603 s/iter total_throughput: 2548.84 samples/s lr: 9.67e-04  
[09/15 18:28:13] lb.utils.events INFO:  eta: 14:18:10  iteration: 43999/375342  consumed_samples: 45056000  total_loss: 3.625  time: 0.4018 s/iter  data_time: 0.2485 s/iter total_throughput: 2548.81 samples/s lr: 9.67e-04  
[09/15 18:28:54] lb.utils.events INFO:  eta: 14:18:20  iteration: 44099/375342  consumed_samples: 45158400  total_loss: 3.639  time: 0.4018 s/iter  data_time: 0.2498 s/iter total_throughput: 2548.70 samples/s lr: 9.67e-04  
[09/15 18:29:35] lb.utils.events INFO:  eta: 14:18:45  iteration: 44199/375342  consumed_samples: 45260800  total_loss: 3.659  time: 0.4018 s/iter  data_time: 0.2600 s/iter total_throughput: 2548.69 samples/s lr: 9.67e-04  
[09/15 18:30:15] lb.utils.events INFO:  eta: 14:19:35  iteration: 44299/375342  consumed_samples: 45363200  total_loss: 3.652  time: 0.4018 s/iter  data_time: 0.2483 s/iter total_throughput: 2548.61 samples/s lr: 9.66e-04  
[09/15 18:30:56] lb.utils.events INFO:  eta: 14:21:16  iteration: 44399/375342  consumed_samples: 45465600  total_loss: 3.637  time: 0.4018 s/iter  data_time: 0.2605 s/iter total_throughput: 2548.60 samples/s lr: 9.66e-04  
[09/15 18:31:36] lb.utils.events INFO:  eta: 14:23:13  iteration: 44499/375342  consumed_samples: 45568000  total_loss: 3.634  time: 0.4018 s/iter  data_time: 0.2600 s/iter total_throughput: 2548.54 samples/s lr: 9.66e-04  
[09/15 18:32:17] lb.utils.events INFO:  eta: 14:20:44  iteration: 44599/375342  consumed_samples: 45670400  total_loss: 3.635  time: 0.4018 s/iter  data_time: 0.2528 s/iter total_throughput: 2548.47 samples/s lr: 9.66e-04  
[09/15 18:32:58] lb.utils.events INFO:  eta: 14:22:39  iteration: 44699/375342  consumed_samples: 45772800  total_loss: 3.618  time: 0.4018 s/iter  data_time: 0.2571 s/iter total_throughput: 2548.38 samples/s lr: 9.66e-04  
[09/15 18:33:38] lb.utils.events INFO:  eta: 14:24:27  iteration: 44799/375342  consumed_samples: 45875200  total_loss: 3.618  time: 0.4018 s/iter  data_time: 0.2642 s/iter total_throughput: 2548.32 samples/s lr: 9.66e-04  
[09/15 18:34:19] lb.utils.events INFO:  eta: 14:27:01  iteration: 44899/375342  consumed_samples: 45977600  total_loss: 3.617  time: 0.4019 s/iter  data_time: 0.2638 s/iter total_throughput: 2548.21 samples/s lr: 9.65e-04  
[09/15 18:35:00] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_2e56/model_0044999
[09/15 18:35:00] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000
[09/15 18:35:00] lb.evaluation.evaluator INFO: Start inference on 50000 samples
[09/15 18:35:05] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.1067 s/iter. Inference: 0.1513 s/iter. Eval: 0.0020 s/iter. Total: 0.2600 s/iter. ETA=0:00:09
[09/15 18:35:10] lb.evaluation.evaluator INFO: Inference done 28672/50000. Dataloading: 0.1039 s/iter. Inference: 0.1807 s/iter. Eval: 0.0021 s/iter. Total: 0.2867 s/iter. ETA=0:00:05
[09/15 18:35:15] lb.evaluation.evaluator INFO: Inference done 46080/50000. Dataloading: 0.0818 s/iter. Inference: 0.2150 s/iter. Eval: 0.0021 s/iter. Total: 0.2990 s/iter. ETA=0:00:00
[09/15 18:35:16] lb.evaluation.evaluator INFO: Total valid samples: 50000
[09/15 18:35:16] lb.evaluation.evaluator INFO: Total inference time: 0:00:12.995751 (0.000260 s / iter per device, on 8 devices)
[09/15 18:35:16] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:09 (0.000190 s / iter per device, on 8 devices)
[09/15 18:35:16] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format:
[09/15 18:35:16] lb.evaluation.utils INFO: copypaste: Acc@1=61.392
[09/15 18:35:16] lb.evaluation.utils INFO: copypaste: Acc@5=83.31
[09/15 18:35:16] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 61.39200, better than last best score 59.83600 @ iteration 39999.
[09/15 18:35:16] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_2e56/model_best
[09/15 18:35:17] lb.utils.events INFO:  eta: 14:36:45  iteration: 44999/375342  consumed_samples: 46080000  total_loss: 3.65  time: 0.4019 s/iter  data_time: 0.2475 s/iter total_throughput: 2548.18 samples/s lr: 9.65e-04  
[09/15 18:35:55] lb.utils.events INFO:  eta: 14:37:05  iteration: 45099/375342  consumed_samples: 46182400  total_loss: 3.661  time: 0.4018 s/iter  data_time: 0.2437 s/iter total_throughput: 2548.46 samples/s lr: 9.65e-04  
[09/15 18:36:36] lb.utils.events INFO:  eta: 14:41:23  iteration: 45199/375342  consumed_samples: 46284800  total_loss: 3.62  time: 0.4018 s/iter  data_time: 0.2609 s/iter total_throughput: 2548.42 samples/s lr: 9.65e-04  
[09/15 18:37:17] lb.utils.events INFO:  eta: 15:56:04  iteration: 45299/375342  consumed_samples: 46387200  total_loss: 3.607  time: 0.4018 s/iter  data_time: 0.2595 s/iter total_throughput: 2548.29 samples/s lr: 9.65e-04  
[09/15 18:37:58] lb.utils.events INFO:  eta: 17:27:13  iteration: 45399/375342  consumed_samples: 46489600  total_loss: 3.63  time: 0.4019 s/iter  data_time: 0.2661 s/iter total_throughput: 2548.20 samples/s lr: 9.65e-04  
[09/15 18:38:38] lb.utils.events INFO:  eta: 17:31:11  iteration: 45499/375342  consumed_samples: 46592000  total_loss: 3.626  time: 0.4019 s/iter  data_time: 0.2602 s/iter total_throughput: 2548.12 samples/s lr: 9.65e-04  
[09/15 18:39:19] lb.utils.events INFO:  eta: 17:40:24  iteration: 45599/375342  consumed_samples: 46694400  total_loss: 3.633  time: 0.4019 s/iter  data_time: 0.2610 s/iter total_throughput: 2548.04 samples/s lr: 9.64e-04  
[09/15 18:40:01] lb.utils.events INFO:  eta: 17:21:28  iteration: 45699/375342  consumed_samples: 46796800  total_loss: 3.629  time: 0.4019 s/iter  data_time: 0.2599 s/iter total_throughput: 2547.87 samples/s lr: 9.64e-04  
[09/15 18:40:41] lb.utils.events INFO:  eta: 15:38:22  iteration: 45799/375342  consumed_samples: 46899200  total_loss: 3.625  time: 0.4019 s/iter  data_time: 0.2607 s/iter total_throughput: 2547.78 samples/s lr: 9.64e-04  
[09/15 18:41:22] lb.utils.events INFO:  eta: 15:25:48  iteration: 45899/375342  consumed_samples: 47001600  total_loss: 3.638  time: 0.4019 s/iter  data_time: 0.2569 s/iter total_throughput: 2547.76 samples/s lr: 9.64e-04  
[09/15 18:42:02] lb.utils.events INFO:  eta: 14:46:31  iteration: 45999/375342  consumed_samples: 47104000  total_loss: 3.63  time: 0.4019 s/iter  data_time: 0.2638 s/iter total_throughput: 2547.73 samples/s lr: 9.64e-04  
[09/15 18:42:43] lb.utils.events INFO:  eta: 14:41:18  iteration: 46099/375342  consumed_samples: 47206400  total_loss: 3.624  time: 0.4019 s/iter  data_time: 0.2543 s/iter total_throughput: 2547.60 samples/s lr: 9.64e-04  
[09/15 18:43:24] lb.utils.events INFO:  eta: 14:30:29  iteration: 46199/375342  consumed_samples: 47308800  total_loss: 3.639  time: 0.4020 s/iter  data_time: 0.2551 s/iter total_throughput: 2547.57 samples/s lr: 9.63e-04  
[09/15 18:44:04] lb.utils.events INFO:  eta: 14:21:30  iteration: 46299/375342  consumed_samples: 47411200  total_loss: 3.63  time: 0.4020 s/iter  data_time: 0.2675 s/iter total_throughput: 2547.53 samples/s lr: 9.63e-04  
[09/15 18:44:45] lb.utils.events INFO:  eta: 14:15:20  iteration: 46399/375342  consumed_samples: 47513600  total_loss: 3.621  time: 0.4020 s/iter  data_time: 0.2619 s/iter total_throughput: 2547.47 samples/s lr: 9.63e-04  
[09/15 18:45:25] lb.utils.events INFO:  eta: 14:14:24  iteration: 46499/375342  consumed_samples: 47616000  total_loss: 3.628  time: 0.4020 s/iter  data_time: 0.2638 s/iter total_throughput: 2547.44 samples/s lr: 9.63e-04  
[09/15 18:46:06] lb.utils.events INFO:  eta: 14:17:25  iteration: 46599/375342  consumed_samples: 47718400  total_loss: 3.642  time: 0.4020 s/iter  data_time: 0.2506 s/iter total_throughput: 2547.37 samples/s lr: 9.63e-04  
[09/15 18:46:47] lb.utils.events INFO:  eta: 14:34:02  iteration: 46699/375342  consumed_samples: 47820800  total_loss: 3.635  time: 0.4020 s/iter  data_time: 0.2532 s/iter total_throughput: 2547.32 samples/s lr: 9.63e-04  
[09/15 18:47:28] lb.utils.events INFO:  eta: 21:17:18  iteration: 46799/375342  consumed_samples: 47923200  total_loss: 3.629  time: 0.4020 s/iter  data_time: 0.2626 s/iter total_throughput: 2547.22 samples/s lr: 9.63e-04  
[09/15 18:48:08] lb.utils.events INFO:  eta: 23:42:43  iteration: 46899/375342  consumed_samples: 48025600  total_loss: 3.641  time: 0.4020 s/iter  data_time: 0.2622 s/iter total_throughput: 2547.18 samples/s lr: 9.62e-04  
[09/15 18:48:49] lb.utils.events INFO:  eta: 1 day, 1:01:12  iteration: 46999/375342  consumed_samples: 48128000  total_loss: 3.62  time: 0.4020 s/iter  data_time: 0.2552 s/iter total_throughput: 2547.14 samples/s lr: 9.62e-04  
[09/15 18:49:29] lb.utils.events INFO:  eta: 1 day, 2:51:49  iteration: 47099/375342  consumed_samples: 48230400  total_loss: 3.62  time: 0.4020 s/iter  data_time: 0.2601 s/iter total_throughput: 2547.14 samples/s lr: 9.62e-04  
[09/15 18:50:10] lb.utils.events INFO:  eta: 1 day, 3:39:32  iteration: 47199/375342  consumed_samples: 48332800  total_loss: 3.604  time: 0.4020 s/iter  data_time: 0.2706 s/iter total_throughput: 2547.03 samples/s lr: 9.62e-04  
[09/15 18:50:50] lb.utils.events INFO:  eta: 1 day, 4:03:59  iteration: 47299/375342  consumed_samples: 48435200  total_loss: 3.612  time: 0.4020 s/iter  data_time: 0.2506 s/iter total_throughput: 2546.99 samples/s lr: 9.62e-04  
[09/15 18:51:31] lb.utils.events INFO:  eta: 1 day, 5:34:24  iteration: 47399/375342  consumed_samples: 48537600  total_loss: 3.616  time: 0.4020 s/iter  data_time: 0.2643 s/iter total_throughput: 2546.95 samples/s lr: 9.62e-04  
[09/15 18:52:11] lb.utils.events INFO:  eta: 1 day, 6:19:06  iteration: 47499/375342  consumed_samples: 48640000  total_loss: 3.616  time: 0.4020 s/iter  data_time: 0.2551 s/iter total_throughput: 2546.97 samples/s lr: 9.61e-04  
[09/15 18:52:51] lb.utils.events INFO:  eta: 1 day, 4:52:15  iteration: 47599/375342  consumed_samples: 48742400  total_loss: 3.61  time: 0.4021 s/iter  data_time: 0.2572 s/iter total_throughput: 2546.93 samples/s lr: 9.61e-04  
[09/15 18:53:32] lb.utils.events INFO:  eta: 1 day, 4:10:27  iteration: 47699/375342  consumed_samples: 48844800  total_loss: 3.613  time: 0.4021 s/iter  data_time: 0.2598 s/iter total_throughput: 2546.88 samples/s lr: 9.61e-04  
[09/15 18:54:13] lb.utils.events INFO:  eta: 1 day, 0:00:21  iteration: 47799/375342  consumed_samples: 48947200  total_loss: 3.615  time: 0.4021 s/iter  data_time: 0.2522 s/iter total_throughput: 2546.82 samples/s lr: 9.61e-04  
[09/15 18:54:54] lb.utils.events INFO:  eta: 22:00:29  iteration: 47899/375342  consumed_samples: 49049600  total_loss: 3.603  time: 0.4021 s/iter  data_time: 0.2665 s/iter total_throughput: 2546.74 samples/s lr: 9.61e-04  
[09/15 18:55:34] lb.utils.events INFO:  eta: 18:30:18  iteration: 47999/375342  consumed_samples: 49152000  total_loss: 3.605  time: 0.4021 s/iter  data_time: 0.2576 s/iter total_throughput: 2546.69 samples/s lr: 9.61e-04  
[09/15 18:56:15] lb.utils.events INFO:  eta: 16:20:56  iteration: 48099/375342  consumed_samples: 49254400  total_loss: 3.603  time: 0.4021 s/iter  data_time: 0.2515 s/iter total_throughput: 2546.67 samples/s lr: 9.60e-04  
[09/15 18:56:55] lb.utils.events INFO:  eta: 14:59:36  iteration: 48199/375342  consumed_samples: 49356800  total_loss: 3.594  time: 0.4021 s/iter  data_time: 0.2577 s/iter total_throughput: 2546.64 samples/s lr: 9.60e-04  
[09/15 18:57:36] lb.utils.events INFO:  eta: 16:17:24  iteration: 48299/375342  consumed_samples: 49459200  total_loss: 3.593  time: 0.4021 s/iter  data_time: 0.2622 s/iter total_throughput: 2546.52 samples/s lr: 9.60e-04  
[09/15 18:58:17] lb.utils.events INFO:  eta: 16:06:29  iteration: 48399/375342  consumed_samples: 49561600  total_loss: 3.587  time: 0.4021 s/iter  data_time: 0.2519 s/iter total_throughput: 2546.49 samples/s lr: 9.60e-04  
[09/15 18:58:57] lb.utils.events INFO:  eta: 15:38:52  iteration: 48499/375342  consumed_samples: 49664000  total_loss: 3.605  time: 0.4021 s/iter  data_time: 0.2559 s/iter total_throughput: 2546.47 samples/s lr: 9.60e-04  
[09/15 18:59:37] lb.utils.events INFO:  eta: 18:47:24  iteration: 48599/375342  consumed_samples: 49766400  total_loss: 3.623  time: 0.4021 s/iter  data_time: 0.2492 s/iter total_throughput: 2546.43 samples/s lr: 9.60e-04  
[09/15 19:00:18] lb.utils.events INFO:  eta: 21:19:07  iteration: 48699/375342  consumed_samples: 49868800  total_loss: 3.605  time: 0.4021 s/iter  data_time: 0.2500 s/iter total_throughput: 2546.43 samples/s lr: 9.59e-04  
[09/15 19:00:58] lb.utils.events INFO:  eta: 21:55:00  iteration: 48799/375342  consumed_samples: 49971200  total_loss: 3.584  time: 0.4021 s/iter  data_time: 0.2549 s/iter total_throughput: 2546.38 samples/s lr: 9.59e-04  
[09/15 19:01:39] lb.utils.events INFO:  eta: 18:37:52  iteration: 48899/375342  consumed_samples: 50073600  total_loss: 3.583  time: 0.4022 s/iter  data_time: 0.2588 s/iter total_throughput: 2546.30 samples/s lr: 9.59e-04  
[09/15 19:02:20] lb.utils.events INFO:  eta: 20:38:19  iteration: 48999/375342  consumed_samples: 50176000  total_loss: 3.594  time: 0.4022 s/iter  data_time: 0.2553 s/iter total_throughput: 2546.26 samples/s lr: 9.59e-04  
[09/15 19:03:00] lb.utils.events INFO:  eta: 22:55:00  iteration: 49099/375342  consumed_samples: 50278400  total_loss: 3.6  time: 0.4022 s/iter  data_time: 0.2495 s/iter total_throughput: 2546.26 samples/s lr: 9.59e-04  
[09/15 19:03:41] lb.utils.events INFO:  eta: 21:59:34  iteration: 49199/375342  consumed_samples: 50380800  total_loss: 3.578  time: 0.4022 s/iter  data_time: 0.2492 s/iter total_throughput: 2546.17 samples/s lr: 9.59e-04  
[09/15 19:04:21] lb.utils.events INFO:  eta: 16:26:56  iteration: 49299/375342  consumed_samples: 50483200  total_loss: 3.566  time: 0.4022 s/iter  data_time: 0.2490 s/iter total_throughput: 2546.15 samples/s lr: 9.58e-04  
[09/15 19:05:01] lb.utils.events INFO:  eta: 15:10:27  iteration: 49399/375342  consumed_samples: 50585600  total_loss: 3.574  time: 0.4022 s/iter  data_time: 0.2540 s/iter total_throughput: 2546.17 samples/s lr: 9.58e-04  
[09/15 19:05:42] lb.utils.events INFO:  eta: 14:32:23  iteration: 49499/375342  consumed_samples: 50688000  total_loss: 3.579  time: 0.4022 s/iter  data_time: 0.2576 s/iter total_throughput: 2546.10 samples/s lr: 9.58e-04  
[09/15 19:06:23] lb.utils.events INFO:  eta: 14:15:20  iteration: 49599/375342  consumed_samples: 50790400  total_loss: 3.6  time: 0.4022 s/iter  data_time: 0.2774 s/iter total_throughput: 2546.00 samples/s lr: 9.58e-04  
[09/15 19:07:04] lb.utils.events INFO:  eta: 14:07:20  iteration: 49699/375342  consumed_samples: 50892800  total_loss: 3.616  time: 0.4022 s/iter  data_time: 0.2542 s/iter total_throughput: 2545.94 samples/s lr: 9.58e-04  
[09/15 19:07:45] lb.utils.events INFO:  eta: 14:04:52  iteration: 49799/375342  consumed_samples: 50995200  total_loss: 3.577  time: 0.4022 s/iter  data_time: 0.2569 s/iter total_throughput: 2545.85 samples/s lr: 9.58e-04  
[09/15 19:08:25] lb.utils.events INFO:  eta: 14:03:38  iteration: 49899/375342  consumed_samples: 51097600  total_loss: 3.564  time: 0.4022 s/iter  data_time: 0.2542 s/iter total_throughput: 2545.80 samples/s lr: 9.57e-04  
[09/15 19:09:06] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_2e56/model_0049999
[09/15 19:09:06] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000
[09/15 19:09:06] lb.evaluation.evaluator INFO: Start inference on 50000 samples
[09/15 19:09:11] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.1153 s/iter. Inference: 0.1524 s/iter. Eval: 0.0024 s/iter. Total: 0.2701 s/iter. ETA=0:00:09
[09/15 19:09:16] lb.evaluation.evaluator INFO: Inference done 27648/50000. Dataloading: 0.1512 s/iter. Inference: 0.1491 s/iter. Eval: 0.0022 s/iter. Total: 0.3026 s/iter. ETA=0:00:06
[09/15 19:09:21] lb.evaluation.evaluator INFO: Inference done 45056/50000. Dataloading: 0.1480 s/iter. Inference: 0.1491 s/iter. Eval: 0.0021 s/iter. Total: 0.2993 s/iter. ETA=0:00:01
[09/15 19:09:23] lb.evaluation.evaluator INFO: Total valid samples: 50000
[09/15 19:09:23] lb.evaluation.evaluator INFO: Total inference time: 0:00:13.352053 (0.000267 s / iter per device, on 8 devices)
[09/15 19:09:23] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000132 s / iter per device, on 8 devices)
[09/15 19:09:23] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format:
[09/15 19:09:23] lb.evaluation.utils INFO: copypaste: Acc@1=62.370000000000005
[09/15 19:09:23] lb.evaluation.utils INFO: copypaste: Acc@5=84.26
[09/15 19:09:23] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 62.37000, better than last best score 61.39200 @ iteration 44999.
[09/15 19:09:23] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_2e56/model_best
[09/15 19:09:24] lb.utils.events INFO:  eta: 14:02:10  iteration: 49999/375342  consumed_samples: 51200000  total_loss: 3.615  time: 0.4022 s/iter  data_time: 0.2539 s/iter total_throughput: 2545.76 samples/s lr: 9.57e-04  
[09/15 19:10:02] lb.utils.events INFO:  eta: 14:01:26  iteration: 50099/375342  consumed_samples: 51302400  total_loss: 3.623  time: 0.4022 s/iter  data_time: 0.2694 s/iter total_throughput: 2545.98 samples/s lr: 9.57e-04  
[09/15 19:10:43] lb.utils.events INFO:  eta: 14:02:19  iteration: 50199/375342  consumed_samples: 51404800  total_loss: 3.613  time: 0.4022 s/iter  data_time: 0.2618 s/iter total_throughput: 2545.87 samples/s lr: 9.57e-04  
[09/15 19:11:24] lb.utils.events INFO:  eta: 14:01:55  iteration: 50299/375342  consumed_samples: 51507200  total_loss: 3.59  time: 0.4022 s/iter  data_time: 0.2519 s/iter total_throughput: 2545.85 samples/s lr: 9.57e-04  
[09/15 19:12:04] lb.utils.events INFO:  eta: 14:00:55  iteration: 50399/375342  consumed_samples: 51609600  total_loss: 3.552  time: 0.4022 s/iter  data_time: 0.2606 s/iter total_throughput: 2545.80 samples/s lr: 9.57e-04  
[09/15 19:12:45] lb.utils.events INFO:  eta: 13:58:22  iteration: 50499/375342  consumed_samples: 51712000  total_loss: 3.543  time: 0.4022 s/iter  data_time: 0.2557 s/iter total_throughput: 2545.75 samples/s lr: 9.56e-04  
[09/15 19:13:26] lb.utils.events INFO:  eta: 13:57:17  iteration: 50599/375342  consumed_samples: 51814400  total_loss: 3.534  time: 0.4023 s/iter  data_time: 0.2696 s/iter total_throughput: 2545.66 samples/s lr: 9.56e-04  
[09/15 19:14:06] lb.utils.events INFO:  eta: 13:56:28  iteration: 50699/375342  consumed_samples: 51916800  total_loss: 3.543  time: 0.4023 s/iter  data_time: 0.2507 s/iter total_throughput: 2545.62 samples/s lr: 9.56e-04  
[09/15 19:14:46] lb.utils.events INFO:  eta: 13:56:43  iteration: 50799/375342  consumed_samples: 52019200  total_loss: 3.55  time: 0.4023 s/iter  data_time: 0.2479 s/iter total_throughput: 2545.63 samples/s lr: 9.56e-04  
[09/15 19:15:27] lb.utils.events INFO:  eta: 13:57:38  iteration: 50899/375342  consumed_samples: 52121600  total_loss: 3.558  time: 0.4023 s/iter  data_time: 0.2570 s/iter total_throughput: 2545.56 samples/s lr: 9.56e-04  
[09/15 19:16:07] lb.utils.events INFO:  eta: 14:00:49  iteration: 50999/375342  consumed_samples: 52224000  total_loss: 3.585  time: 0.4023 s/iter  data_time: 0.2503 s/iter total_throughput: 2545.60 samples/s lr: 9.56e-04  
[09/15 19:16:48] lb.utils.events INFO:  eta: 14:00:44  iteration: 51099/375342  consumed_samples: 52326400  total_loss: 3.585  time: 0.4023 s/iter  data_time: 0.2609 s/iter total_throughput: 2545.58 samples/s lr: 9.55e-04  
[09/15 19:17:28] lb.utils.events INFO:  eta: 13:59:24  iteration: 51199/375342  consumed_samples: 52428800  total_loss: 3.567  time: 0.4023 s/iter  data_time: 0.2582 s/iter total_throughput: 2545.54 samples/s lr: 9.55e-04  
[09/15 19:18:09] lb.utils.events INFO:  eta: 14:00:36  iteration: 51299/375342  consumed_samples: 52531200  total_loss: 3.568  time: 0.4023 s/iter  data_time: 0.2562 s/iter total_throughput: 2545.52 samples/s lr: 9.55e-04  
[09/15 19:18:49] lb.utils.events INFO:  eta: 14:00:30  iteration: 51399/375342  consumed_samples: 52633600  total_loss: 3.562  time: 0.4023 s/iter  data_time: 0.2520 s/iter total_throughput: 2545.50 samples/s lr: 9.55e-04  
[09/15 19:19:29] lb.utils.events INFO:  eta: 14:01:31  iteration: 51499/375342  consumed_samples: 52736000  total_loss: 3.552  time: 0.4023 s/iter  data_time: 0.2654 s/iter total_throughput: 2545.46 samples/s lr: 9.55e-04  
[09/15 19:20:10] lb.utils.events INFO:  eta: 14:01:40  iteration: 51599/375342  consumed_samples: 52838400  total_loss: 3.562  time: 0.4023 s/iter  data_time: 0.2519 s/iter total_throughput: 2545.48 samples/s lr: 9.55e-04  
[09/15 19:20:50] lb.utils.events INFO:  eta: 14:02:22  iteration: 51699/375342  consumed_samples: 52940800  total_loss: 3.551  time: 0.4023 s/iter  data_time: 0.2539 s/iter total_throughput: 2545.46 samples/s lr: 9.54e-04  
[09/15 19:21:30] lb.utils.events INFO:  eta: 14:04:27  iteration: 51799/375342  consumed_samples: 53043200  total_loss: 3.545  time: 0.4023 s/iter  data_time: 0.2528 s/iter total_throughput: 2545.48 samples/s lr: 9.54e-04  
[09/15 19:22:10] lb.utils.events INFO:  eta: 14:03:46  iteration: 51899/375342  consumed_samples: 53145600  total_loss: 3.552  time: 0.4023 s/iter  data_time: 0.2603 s/iter total_throughput: 2545.48 samples/s lr: 9.54e-04  
[09/15 19:22:50] lb.utils.events INFO:  eta: 13:59:52  iteration: 51999/375342  consumed_samples: 53248000  total_loss: 3.573  time: 0.4023 s/iter  data_time: 0.2672 s/iter total_throughput: 2545.49 samples/s lr: 9.54e-04  
[09/15 19:23:31] lb.utils.events INFO:  eta: 13:59:00  iteration: 52099/375342  consumed_samples: 53350400  total_loss: 3.583  time: 0.4023 s/iter  data_time: 0.2516 s/iter total_throughput: 2545.45 samples/s lr: 9.54e-04  
[09/15 19:24:12] lb.utils.events INFO:  eta: 13:59:03  iteration: 52199/375342  consumed_samples: 53452800  total_loss: 3.57  time: 0.4023 s/iter  data_time: 0.2637 s/iter total_throughput: 2545.42 samples/s lr: 9.54e-04  
[09/15 19:24:52] lb.utils.events INFO:  eta: 13:58:02  iteration: 52299/375342  consumed_samples: 53555200  total_loss: 3.573  time: 0.4023 s/iter  data_time: 0.2480 s/iter total_throughput: 2545.38 samples/s lr: 9.53e-04  
[09/15 19:25:33] lb.utils.events INFO:  eta: 13:55:53  iteration: 52399/375342  consumed_samples: 53657600  total_loss: 3.574  time: 0.4023 s/iter  data_time: 0.2508 s/iter total_throughput: 2545.33 samples/s lr: 9.53e-04  
[09/15 19:26:13] lb.utils.events INFO:  eta: 13:55:48  iteration: 52499/375342  consumed_samples: 53760000  total_loss: 3.571  time: 0.4023 s/iter  data_time: 0.2476 s/iter total_throughput: 2545.33 samples/s lr: 9.53e-04  
[09/15 19:26:53] lb.utils.events INFO:  eta: 13:56:08  iteration: 52599/375342  consumed_samples: 53862400  total_loss: 3.563  time: 0.4023 s/iter  data_time: 0.2524 s/iter total_throughput: 2545.33 samples/s lr: 9.53e-04  
[09/15 19:27:34] lb.utils.events INFO:  eta: 13:54:40  iteration: 52699/375342  consumed_samples: 53964800  total_loss: 3.546  time: 0.4023 s/iter  data_time: 0.2658 s/iter total_throughput: 2545.25 samples/s lr: 9.53e-04  
[09/15 19:28:15] lb.utils.events INFO:  eta: 13:53:11  iteration: 52799/375342  consumed_samples: 54067200  total_loss: 3.551  time: 0.4023 s/iter  data_time: 0.2573 s/iter total_throughput: 2545.20 samples/s lr: 9.52e-04  
[09/15 19:28:55] lb.utils.events INFO:  eta: 13:52:08  iteration: 52899/375342  consumed_samples: 54169600  total_loss: 3.557  time: 0.4023 s/iter  data_time: 0.2482 s/iter total_throughput: 2545.18 samples/s lr: 9.52e-04  
[09/15 19:29:35] lb.utils.events INFO:  eta: 13:51:56  iteration: 52999/375342  consumed_samples: 54272000  total_loss: 3.546  time: 0.4023 s/iter  data_time: 0.2427 s/iter total_throughput: 2545.21 samples/s lr: 9.52e-04  
[09/15 19:30:16] lb.utils.events INFO:  eta: 13:50:51  iteration: 53099/375342  consumed_samples: 54374400  total_loss: 3.557  time: 0.4023 s/iter  data_time: 0.2538 s/iter total_throughput: 2545.19 samples/s lr: 9.52e-04  
[09/15 19:30:56] lb.utils.events INFO:  eta: 13:50:56  iteration: 53199/375342  consumed_samples: 54476800  total_loss: 3.584  time: 0.4023 s/iter  data_time: 0.2484 s/iter total_throughput: 2545.20 samples/s lr: 9.52e-04  
[09/15 19:31:36] lb.utils.events INFO:  eta: 13:52:02  iteration: 53299/375342  consumed_samples: 54579200  total_loss: 3.579  time: 0.4023 s/iter  data_time: 0.2569 s/iter total_throughput: 2545.15 samples/s lr: 9.52e-04  
[09/15 19:32:17] lb.utils.events INFO:  eta: 13:53:46  iteration: 53399/375342  consumed_samples: 54681600  total_loss: 3.565  time: 0.4023 s/iter  data_time: 0.2618 s/iter total_throughput: 2545.13 samples/s lr: 9.51e-04  
[09/15 19:32:57] lb.utils.events INFO:  eta: 13:54:55  iteration: 53499/375342  consumed_samples: 54784000  total_loss: 3.537  time: 0.4023 s/iter  data_time: 0.2682 s/iter total_throughput: 2545.09 samples/s lr: 9.51e-04  
[09/15 19:33:38] lb.utils.events INFO:  eta: 13:54:03  iteration: 53599/375342  consumed_samples: 54886400  total_loss: 3.545  time: 0.4023 s/iter  data_time: 0.2446 s/iter total_throughput: 2545.09 samples/s lr: 9.51e-04  
[09/15 19:34:18] lb.utils.events INFO:  eta: 13:54:27  iteration: 53699/375342  consumed_samples: 54988800  total_loss: 3.545  time: 0.4023 s/iter  data_time: 0.2493 s/iter total_throughput: 2545.07 samples/s lr: 9.51e-04  
[09/15 19:34:59] lb.utils.events INFO:  eta: 13:54:18  iteration: 53799/375342  consumed_samples: 55091200  total_loss: 3.525  time: 0.4024 s/iter  data_time: 0.2596 s/iter total_throughput: 2545.03 samples/s lr: 9.51e-04  
[09/15 19:35:38] lb.utils.events INFO:  eta: 13:53:32  iteration: 53899/375342  consumed_samples: 55193600  total_loss: 3.528  time: 0.4023 s/iter  data_time: 0.2482 s/iter total_throughput: 2545.08 samples/s lr: 9.50e-04  
[09/15 19:36:19] lb.utils.events INFO:  eta: 13:52:17  iteration: 53999/375342  consumed_samples: 55296000  total_loss: 3.54  time: 0.4023 s/iter  data_time: 0.2450 s/iter total_throughput: 2545.08 samples/s lr: 9.50e-04  
[09/15 19:36:59] lb.utils.events INFO:  eta: 13:51:45  iteration: 54099/375342  consumed_samples: 55398400  total_loss: 3.564  time: 0.4023 s/iter  data_time: 0.2552 s/iter total_throughput: 2545.09 samples/s lr: 9.50e-04  
[09/15 19:37:39] lb.utils.events INFO:  eta: 13:55:10  iteration: 54199/375342  consumed_samples: 55500800  total_loss: 3.547  time: 0.4023 s/iter  data_time: 0.2489 s/iter total_throughput: 2545.14 samples/s lr: 9.50e-04  
[09/15 19:38:19] lb.utils.events INFO:  eta: 13:55:57  iteration: 54299/375342  consumed_samples: 55603200  total_loss: 3.525  time: 0.4023 s/iter  data_time: 0.2463 s/iter total_throughput: 2545.17 samples/s lr: 9.50e-04  
[09/15 19:38:59] lb.utils.events INFO:  eta: 13:57:30  iteration: 54399/375342  consumed_samples: 55705600  total_loss: 3.535  time: 0.4023 s/iter  data_time: 0.2600 s/iter total_throughput: 2545.18 samples/s lr: 9.50e-04  
[09/15 19:39:39] lb.utils.events INFO:  eta: 13:55:18  iteration: 54499/375342  consumed_samples: 55808000  total_loss: 3.528  time: 0.4023 s/iter  data_time: 0.2595 s/iter total_throughput: 2545.18 samples/s lr: 9.49e-04  
[09/15 19:40:19] lb.utils.events INFO:  eta: 13:55:31  iteration: 54599/375342  consumed_samples: 55910400  total_loss: 3.523  time: 0.4023 s/iter  data_time: 0.2534 s/iter total_throughput: 2545.20 samples/s lr: 9.49e-04  
[09/15 19:40:59] lb.utils.events INFO:  eta: 13:56:44  iteration: 54699/375342  consumed_samples: 56012800  total_loss: 3.545  time: 0.4023 s/iter  data_time: 0.2499 s/iter total_throughput: 2545.21 samples/s lr: 9.49e-04  
[09/15 19:41:39] lb.utils.events INFO:  eta: 14:07:31  iteration: 54799/375342  consumed_samples: 56115200  total_loss: 3.534  time: 0.4023 s/iter  data_time: 0.2485 s/iter total_throughput: 2545.23 samples/s lr: 9.49e-04  
[09/15 19:42:19] lb.utils.events INFO:  eta: 18:08:25  iteration: 54899/375342  consumed_samples: 56217600  total_loss: 3.542  time: 0.4023 s/iter  data_time: 0.2572 s/iter total_throughput: 2545.24 samples/s lr: 9.49e-04  
[09/15 19:43:00] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_2e56/model_0054999
[09/15 19:43:00] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000
[09/15 19:43:00] lb.evaluation.evaluator INFO: Start inference on 50000 samples
[09/15 19:43:05] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.1222 s/iter. Inference: 0.1540 s/iter. Eval: 0.0022 s/iter. Total: 0.2783 s/iter. ETA=0:00:10
[09/15 19:43:10] lb.evaluation.evaluator INFO: Inference done 27648/50000. Dataloading: 0.1515 s/iter. Inference: 0.1507 s/iter. Eval: 0.0021 s/iter. Total: 0.3044 s/iter. ETA=0:00:06
[09/15 19:43:15] lb.evaluation.evaluator INFO: Inference done 45056/50000. Dataloading: 0.1510 s/iter. Inference: 0.1486 s/iter. Eval: 0.0021 s/iter. Total: 0.3019 s/iter. ETA=0:00:01
[09/15 19:43:17] lb.evaluation.evaluator INFO: Total valid samples: 50000
[09/15 19:43:17] lb.evaluation.evaluator INFO: Total inference time: 0:00:13.244689 (0.000265 s / iter per device, on 8 devices)
[09/15 19:43:17] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000131 s / iter per device, on 8 devices)
[09/15 19:43:17] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format:
[09/15 19:43:17] lb.evaluation.utils INFO: copypaste: Acc@1=62.629999999999995
[09/15 19:43:17] lb.evaluation.utils INFO: copypaste: Acc@5=84.542
[09/15 19:43:17] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 62.63000, better than last best score 62.37000 @ iteration 49999.
[09/15 19:43:17] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_2e56/model_best
[09/15 19:43:17] lb.utils.events INFO:  eta: 21:44:11  iteration: 54999/375342  consumed_samples: 56320000  total_loss: 3.565  time: 0.4023 s/iter  data_time: 0.2440 s/iter total_throughput: 2545.22 samples/s lr: 9.48e-04  
[09/15 19:43:56] lb.utils.events INFO:  eta: 23:06:52  iteration: 55099/375342  consumed_samples: 56422400  total_loss: 3.555  time: 0.4023 s/iter  data_time: 0.2546 s/iter total_throughput: 2545.47 samples/s lr: 9.48e-04  
[09/15 19:44:35] lb.utils.events INFO:  eta: 23:07:05  iteration: 55199/375342  consumed_samples: 56524800  total_loss: 3.547  time: 0.4023 s/iter  data_time: 0.2543 s/iter total_throughput: 2545.50 samples/s lr: 9.48e-04  
[09/15 19:45:16] lb.utils.events INFO:  eta: 21:18:52  iteration: 55299/375342  consumed_samples: 56627200  total_loss: 3.546  time: 0.4023 s/iter  data_time: 0.2522 s/iter total_throughput: 2545.51 samples/s lr: 9.48e-04  
[09/15 19:45:56] lb.utils.events INFO:  eta: 17:22:17  iteration: 55399/375342  consumed_samples: 56729600  total_loss: 3.524  time: 0.4023 s/iter  data_time: 0.2494 s/iter total_throughput: 2545.55 samples/s lr: 9.48e-04  
[09/15 19:46:35] lb.utils.events INFO:  eta: 16:00:41  iteration: 55499/375342  consumed_samples: 56832000  total_loss: 3.519  time: 0.4023 s/iter  data_time: 0.2495 s/iter total_throughput: 2545.60 samples/s lr: 9.48e-04  
[09/15 19:47:15] lb.utils.events INFO:  eta: 17:53:47  iteration: 55599/375342  consumed_samples: 56934400  total_loss: 3.531  time: 0.4023 s/iter  data_time: 0.2510 s/iter total_throughput: 2545.65 samples/s lr: 9.47e-04  
[09/15 19:47:55] lb.utils.events INFO:  eta: 17:53:27  iteration: 55699/375342  consumed_samples: 57036800  total_loss: 3.521  time: 0.4023 s/iter  data_time: 0.2478 s/iter total_throughput: 2545.68 samples/s lr: 9.47e-04  
[09/15 19:48:35] lb.utils.events INFO:  eta: 15:41:21  iteration: 55799/375342  consumed_samples: 57139200  total_loss: 3.52  time: 0.4022 s/iter  data_time: 0.2576 s/iter total_throughput: 2545.73 samples/s lr: 9.47e-04  
[09/15 19:49:15] lb.utils.events INFO:  eta: 14:57:38  iteration: 55899/375342  consumed_samples: 57241600  total_loss: 3.539  time: 0.4022 s/iter  data_time: 0.2445 s/iter total_throughput: 2545.77 samples/s lr: 9.47e-04  
[09/15 19:49:55] lb.utils.events INFO:  eta: 16:49:33  iteration: 55999/375342  consumed_samples: 57344000  total_loss: 3.526  time: 0.4022 s/iter  data_time: 0.2479 s/iter total_throughput: 2545.82 samples/s lr: 9.47e-04  
[09/15 19:50:35] lb.utils.events INFO:  eta: 15:41:32  iteration: 56099/375342  consumed_samples: 57446400  total_loss: 3.553  time: 0.4022 s/iter  data_time: 0.2605 s/iter total_throughput: 2545.77 samples/s lr: 9.46e-04  
[09/15 19:51:16] lb.utils.events INFO:  eta: 14:08:48  iteration: 56199/375342  consumed_samples: 57548800  total_loss: 3.555  time: 0.4022 s/iter  data_time: 0.2527 s/iter total_throughput: 2545.76 samples/s lr: 9.46e-04  
[09/15 19:51:56] lb.utils.events INFO:  eta: 14:26:01  iteration: 56299/375342  consumed_samples: 57651200  total_loss: 3.537  time: 0.4022 s/iter  data_time: 0.2712 s/iter total_throughput: 2545.68 samples/s lr: 9.46e-04  
[09/15 19:52:38] lb.utils.events INFO:  eta: 17:11:54  iteration: 56399/375342  consumed_samples: 57753600  total_loss: 3.508  time: 0.4023 s/iter  data_time: 0.2654 s/iter total_throughput: 2545.55 samples/s lr: 9.46e-04  
[09/15 19:53:20] lb.utils.events INFO:  eta: 17:06:06  iteration: 56499/375342  consumed_samples: 57856000  total_loss: 3.502  time: 0.4023 s/iter  data_time: 0.2758 s/iter total_throughput: 2545.38 samples/s lr: 9.46e-04  
[09/15 19:54:02] lb.utils.events INFO:  eta: 14:35:28  iteration: 56599/375342  consumed_samples: 57958400  total_loss: 3.491  time: 0.4023 s/iter  data_time: 0.2785 s/iter total_throughput: 2545.20 samples/s lr: 9.45e-04  
[09/15 19:54:43] lb.utils.events INFO:  eta: 14:24:40  iteration: 56699/375342  consumed_samples: 58060800  total_loss: 3.491  time: 0.4024 s/iter  data_time: 0.2726 s/iter total_throughput: 2545.04 samples/s lr: 9.45e-04  
[09/15 19:55:25] lb.utils.events INFO:  eta: 14:14:51  iteration: 56799/375342  consumed_samples: 58163200  total_loss: 3.494  time: 0.4024 s/iter  data_time: 0.2668 s/iter total_throughput: 2544.87 samples/s lr: 9.45e-04  
[09/15 19:56:07] lb.utils.events INFO:  eta: 13:59:36  iteration: 56899/375342  consumed_samples: 58265600  total_loss: 3.496  time: 0.4024 s/iter  data_time: 0.2642 s/iter total_throughput: 2544.72 samples/s lr: 9.45e-04  
[09/15 19:56:47] lb.utils.events INFO:  eta: 13:48:28  iteration: 56999/375342  consumed_samples: 58368000  total_loss: 3.528  time: 0.4024 s/iter  data_time: 0.2503 s/iter total_throughput: 2544.68 samples/s lr: 9.45e-04  
[09/15 19:57:27] lb.utils.events INFO:  eta: 13:45:47  iteration: 57099/375342  consumed_samples: 58470400  total_loss: 3.517  time: 0.4024 s/iter  data_time: 0.2603 s/iter total_throughput: 2544.67 samples/s lr: 9.45e-04  
[09/15 19:58:08] lb.utils.events INFO:  eta: 13:46:09  iteration: 57199/375342  consumed_samples: 58572800  total_loss: 3.517  time: 0.4024 s/iter  data_time: 0.2516 s/iter total_throughput: 2544.62 samples/s lr: 9.44e-04  
[09/15 19:58:49] lb.utils.events INFO:  eta: 13:45:28  iteration: 57299/375342  consumed_samples: 58675200  total_loss: 3.516  time: 0.4024 s/iter  data_time: 0.2719 s/iter total_throughput: 2544.53 samples/s lr: 9.44e-04  
[09/15 19:59:30] lb.utils.events INFO:  eta: 13:44:51  iteration: 57399/375342  consumed_samples: 58777600  total_loss: 3.544  time: 0.4024 s/iter  data_time: 0.2662 s/iter total_throughput: 2544.43 samples/s lr: 9.44e-04  
[09/15 20:00:12] lb.utils.events INFO:  eta: 13:48:22  iteration: 57499/375342  consumed_samples: 58880000  total_loss: 3.548  time: 0.4025 s/iter  data_time: 0.2621 s/iter total_throughput: 2544.32 samples/s lr: 9.44e-04  
[09/15 20:00:53] lb.utils.events INFO:  eta: 13:56:49  iteration: 57599/375342  consumed_samples: 58982400  total_loss: 3.529  time: 0.4025 s/iter  data_time: 0.2663 s/iter total_throughput: 2544.22 samples/s lr: 9.44e-04  
[09/15 20:01:34] lb.utils.events INFO:  eta: 14:10:23  iteration: 57699/375342  consumed_samples: 59084800  total_loss: 3.504  time: 0.4025 s/iter  data_time: 0.2634 s/iter total_throughput: 2544.10 samples/s lr: 9.43e-04  
[09/15 20:02:15] lb.utils.events INFO:  eta: 14:52:09  iteration: 57799/375342  consumed_samples: 59187200  total_loss: 3.501  time: 0.4025 s/iter  data_time: 0.2604 s/iter total_throughput: 2544.05 samples/s lr: 9.43e-04  
[09/15 20:02:56] lb.utils.events INFO:  eta: 14:23:34  iteration: 57899/375342  consumed_samples: 59289600  total_loss: 3.489  time: 0.4025 s/iter  data_time: 0.2499 s/iter total_throughput: 2543.98 samples/s lr: 9.43e-04  
[09/15 20:03:36] lb.utils.events INFO:  eta: 15:14:02  iteration: 57999/375342  consumed_samples: 59392000  total_loss: 3.49  time: 0.4025 s/iter  data_time: 0.2512 s/iter total_throughput: 2543.97 samples/s lr: 9.43e-04  
[09/15 20:04:17] lb.utils.events INFO:  eta: 16:34:20  iteration: 58099/375342  consumed_samples: 59494400  total_loss: 3.5  time: 0.4025 s/iter  data_time: 0.2524 s/iter total_throughput: 2543.93 samples/s lr: 9.43e-04  
[09/15 20:04:57] lb.utils.events INFO:  eta: 16:55:45  iteration: 58199/375342  consumed_samples: 59596800  total_loss: 3.51  time: 0.4025 s/iter  data_time: 0.2700 s/iter total_throughput: 2543.91 samples/s lr: 9.42e-04  
[09/15 20:05:38] lb.utils.events INFO:  eta: 15:41:58  iteration: 58299/375342  consumed_samples: 59699200  total_loss: 3.519  time: 0.4025 s/iter  data_time: 0.2612 s/iter total_throughput: 2543.82 samples/s lr: 9.42e-04  
[09/15 20:06:20] lb.utils.events INFO:  eta: 14:24:28  iteration: 58399/375342  consumed_samples: 59801600  total_loss: 3.507  time: 0.4026 s/iter  data_time: 0.2639 s/iter total_throughput: 2543.71 samples/s lr: 9.42e-04  
[09/15 20:07:01] lb.utils.events INFO:  eta: 13:54:35  iteration: 58499/375342  consumed_samples: 59904000  total_loss: 3.51  time: 0.4026 s/iter  data_time: 0.2715 s/iter total_throughput: 2543.62 samples/s lr: 9.42e-04  
[09/15 20:07:41] lb.utils.events INFO:  eta: 13:45:26  iteration: 58599/375342  consumed_samples: 60006400  total_loss: 3.504  time: 0.4026 s/iter  data_time: 0.2579 s/iter total_throughput: 2543.62 samples/s lr: 9.42e-04  
[09/15 20:08:22] lb.utils.events INFO:  eta: 13:41:38  iteration: 58699/375342  consumed_samples: 60108800  total_loss: 3.506  time: 0.4026 s/iter  data_time: 0.2532 s/iter total_throughput: 2543.57 samples/s lr: 9.41e-04  
[09/15 20:09:02] lb.utils.events INFO:  eta: 13:39:53  iteration: 58799/375342  consumed_samples: 60211200  total_loss: 3.513  time: 0.4026 s/iter  data_time: 0.2588 s/iter total_throughput: 2543.55 samples/s lr: 9.41e-04  
[09/15 20:09:43] lb.utils.events INFO:  eta: 13:39:12  iteration: 58899/375342  consumed_samples: 60313600  total_loss: 3.5  time: 0.4026 s/iter  data_time: 0.2642 s/iter total_throughput: 2543.49 samples/s lr: 9.41e-04  
[09/15 20:10:24] lb.utils.events INFO:  eta: 13:37:32  iteration: 58999/375342  consumed_samples: 60416000  total_loss: 3.482  time: 0.4026 s/iter  data_time: 0.2577 s/iter total_throughput: 2543.42 samples/s lr: 9.41e-04  
[09/15 20:11:04] lb.utils.events INFO:  eta: 13:36:12  iteration: 59099/375342  consumed_samples: 60518400  total_loss: 3.474  time: 0.4026 s/iter  data_time: 0.2550 s/iter total_throughput: 2543.39 samples/s lr: 9.41e-04  
[09/15 20:11:46] lb.utils.events INFO:  eta: 13:35:37  iteration: 59199/375342  consumed_samples: 60620800  total_loss: 3.504  time: 0.4026 s/iter  data_time: 0.2651 s/iter total_throughput: 2543.28 samples/s lr: 9.40e-04  
[09/15 20:12:27] lb.utils.events INFO:  eta: 13:35:42  iteration: 59299/375342  consumed_samples: 60723200  total_loss: 3.522  time: 0.4026 s/iter  data_time: 0.2562 s/iter total_throughput: 2543.22 samples/s lr: 9.40e-04  
[09/15 20:13:07] lb.utils.events INFO:  eta: 13:35:18  iteration: 59399/375342  consumed_samples: 60825600  total_loss: 3.523  time: 0.4026 s/iter  data_time: 0.2536 s/iter total_throughput: 2543.16 samples/s lr: 9.40e-04  
[09/15 20:13:49] lb.utils.events INFO:  eta: 13:35:22  iteration: 59499/375342  consumed_samples: 60928000  total_loss: 3.52  time: 0.4027 s/iter  data_time: 0.2638 s/iter total_throughput: 2543.07 samples/s lr: 9.40e-04  
[09/15 20:14:29] lb.utils.events INFO:  eta: 13:35:00  iteration: 59599/375342  consumed_samples: 61030400  total_loss: 3.492  time: 0.4027 s/iter  data_time: 0.2553 s/iter total_throughput: 2543.01 samples/s lr: 9.40e-04  
[09/15 20:15:10] lb.utils.events INFO:  eta: 13:35:11  iteration: 59699/375342  consumed_samples: 61132800  total_loss: 3.483  time: 0.4027 s/iter  data_time: 0.2571 s/iter total_throughput: 2542.97 samples/s lr: 9.39e-04  
[09/15 20:15:51] lb.utils.events INFO:  eta: 13:34:26  iteration: 59799/375342  consumed_samples: 61235200  total_loss: 3.52  time: 0.4027 s/iter  data_time: 0.2637 s/iter total_throughput: 2542.86 samples/s lr: 9.39e-04  
[09/15 20:16:33] lb.utils.events INFO:  eta: 13:33:34  iteration: 59899/375342  consumed_samples: 61337600  total_loss: 3.516  time: 0.4027 s/iter  data_time: 0.2533 s/iter total_throughput: 2542.76 samples/s lr: 9.39e-04  
[09/15 20:17:13] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_2e56/model_0059999
[09/15 20:17:14] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000
[09/15 20:17:14] lb.evaluation.evaluator INFO: Start inference on 50000 samples
[09/15 20:17:18] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.1194 s/iter. Inference: 0.1505 s/iter. Eval: 0.0023 s/iter. Total: 0.2723 s/iter. ETA=0:00:10
[09/15 20:17:23] lb.evaluation.evaluator INFO: Inference done 27648/50000. Dataloading: 0.1511 s/iter. Inference: 0.1497 s/iter. Eval: 0.0022 s/iter. Total: 0.3031 s/iter. ETA=0:00:06
[09/15 20:17:29] lb.evaluation.evaluator INFO: Inference done 46080/50000. Dataloading: 0.1495 s/iter. Inference: 0.1496 s/iter. Eval: 0.0021 s/iter. Total: 0.3013 s/iter. ETA=0:00:00
[09/15 20:17:30] lb.evaluation.evaluator INFO: Total valid samples: 50000
[09/15 20:17:30] lb.evaluation.evaluator INFO: Total inference time: 0:00:13.126011 (0.000263 s / iter per device, on 8 devices)
[09/15 20:17:30] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000132 s / iter per device, on 8 devices)
[09/15 20:17:30] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format:
[09/15 20:17:30] lb.evaluation.utils INFO: copypaste: Acc@1=63.016000000000005
[09/15 20:17:30] lb.evaluation.utils INFO: copypaste: Acc@5=84.566
[09/15 20:17:30] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 63.01600, better than last best score 62.63000 @ iteration 54999.
[09/15 20:17:30] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_2e56/model_best
[09/15 20:17:31] lb.utils.events INFO:  eta: 13:32:46  iteration: 59999/375342  consumed_samples: 61440000  total_loss: 3.513  time: 0.4027 s/iter  data_time: 0.2652 s/iter total_throughput: 2542.73 samples/s lr: 9.39e-04  
[09/15 20:18:10] lb.utils.events INFO:  eta: 13:33:20  iteration: 60099/375342  consumed_samples: 61542400  total_loss: 3.511  time: 0.4027 s/iter  data_time: 0.2575 s/iter total_throughput: 2542.84 samples/s lr: 9.39e-04  
[09/15 20:18:51] lb.utils.events INFO:  eta: 13:31:53  iteration: 60199/375342  consumed_samples: 61644800  total_loss: 3.491  time: 0.4027 s/iter  data_time: 0.2804 s/iter total_throughput: 2542.74 samples/s lr: 9.38e-04  
[09/15 20:19:32] lb.utils.events INFO:  eta: 13:30:55  iteration: 60299/375342  consumed_samples: 61747200  total_loss: 3.503  time: 0.4027 s/iter  data_time: 0.2600 s/iter total_throughput: 2542.67 samples/s lr: 9.38e-04  
[09/15 20:20:13] lb.utils.events INFO:  eta: 13:30:39  iteration: 60399/375342  consumed_samples: 61849600  total_loss: 3.514  time: 0.4027 s/iter  data_time: 0.2748 s/iter total_throughput: 2542.60 samples/s lr: 9.38e-04  
[09/15 20:20:54] lb.utils.events INFO:  eta: 13:29:35  iteration: 60499/375342  consumed_samples: 61952000  total_loss: 3.518  time: 0.4028 s/iter  data_time: 0.2645 s/iter total_throughput: 2542.48 samples/s lr: 9.38e-04  
[09/15 20:21:36] lb.utils.events INFO:  eta: 13:29:32  iteration: 60599/375342  consumed_samples: 62054400  total_loss: 3.494  time: 0.4028 s/iter  data_time: 0.2623 s/iter total_throughput: 2542.38 samples/s lr: 9.38e-04  
[09/15 20:22:16] lb.utils.events INFO:  eta: 13:29:00  iteration: 60699/375342  consumed_samples: 62156800  total_loss: 3.484  time: 0.4028 s/iter  data_time: 0.2562 s/iter total_throughput: 2542.34 samples/s lr: 9.37e-04  
[09/15 20:22:57] lb.utils.events INFO:  eta: 13:29:14  iteration: 60799/375342  consumed_samples: 62259200  total_loss: 3.5  time: 0.4028 s/iter  data_time: 0.2740 s/iter total_throughput: 2542.29 samples/s lr: 9.37e-04  
[09/15 20:23:38] lb.utils.events INFO:  eta: 13:29:22  iteration: 60899/375342  consumed_samples: 62361600  total_loss: 3.488  time: 0.4028 s/iter  data_time: 0.2497 s/iter total_throughput: 2542.25 samples/s lr: 9.37e-04  
[09/15 20:24:18] lb.utils.events INFO:  eta: 13:29:17  iteration: 60999/375342  consumed_samples: 62464000  total_loss: 3.493  time: 0.4028 s/iter  data_time: 0.2663 s/iter total_throughput: 2542.22 samples/s lr: 9.37e-04  
[09/15 20:24:59] lb.utils.events INFO:  eta: 13:28:28  iteration: 61099/375342  consumed_samples: 62566400  total_loss: 3.473  time: 0.4028 s/iter  data_time: 0.2487 s/iter total_throughput: 2542.21 samples/s lr: 9.37e-04  
[09/15 20:25:39] lb.utils.events INFO:  eta: 13:29:39  iteration: 61199/375342  consumed_samples: 62668800  total_loss: 3.476  time: 0.4028 s/iter  data_time: 0.2543 s/iter total_throughput: 2542.17 samples/s lr: 9.36e-04  
[09/15 20:26:20] lb.utils.events INFO:  eta: 13:30:46  iteration: 61299/375342  consumed_samples: 62771200  total_loss: 3.498  time: 0.4028 s/iter  data_time: 0.2546 s/iter total_throughput: 2542.15 samples/s lr: 9.36e-04  
[09/15 20:27:01] lb.utils.events INFO:  eta: 13:31:05  iteration: 61399/375342  consumed_samples: 62873600  total_loss: 3.507  time: 0.4028 s/iter  data_time: 0.2581 s/iter total_throughput: 2542.11 samples/s lr: 9.36e-04  
[09/15 20:27:41] lb.utils.events INFO:  eta: 13:31:09  iteration: 61499/375342  consumed_samples: 62976000  total_loss: 3.499  time: 0.4028 s/iter  data_time: 0.2500 s/iter total_throughput: 2542.09 samples/s lr: 9.36e-04  
[09/15 20:28:21] lb.utils.events INFO:  eta: 13:31:09  iteration: 61599/375342  consumed_samples: 63078400  total_loss: 3.485  time: 0.4028 s/iter  data_time: 0.2606 s/iter total_throughput: 2542.08 samples/s lr: 9.36e-04  
[09/15 20:29:02] lb.utils.events INFO:  eta: 13:32:00  iteration: 61699/375342  consumed_samples: 63180800  total_loss: 3.462  time: 0.4028 s/iter  data_time: 0.2541 s/iter total_throughput: 2542.07 samples/s lr: 9.35e-04  
[09/15 20:29:42] lb.utils.events INFO:  eta: 13:33:22  iteration: 61799/375342  consumed_samples: 63283200  total_loss: 3.463  time: 0.4028 s/iter  data_time: 0.2512 s/iter total_throughput: 2542.06 samples/s lr: 9.35e-04  
[09/15 20:30:23] lb.utils.events INFO:  eta: 13:35:53  iteration: 61899/375342  consumed_samples: 63385600  total_loss: 3.458  time: 0.4028 s/iter  data_time: 0.2412 s/iter total_throughput: 2542.04 samples/s lr: 9.35e-04  
[09/15 20:31:04] lb.utils.events INFO:  eta: 13:36:10  iteration: 61999/375342  consumed_samples: 63488000  total_loss: 3.484  time: 0.4028 s/iter  data_time: 0.2526 s/iter total_throughput: 2541.95 samples/s lr: 9.35e-04  
[09/15 20:31:45] lb.utils.events INFO:  eta: 13:36:00  iteration: 62099/375342  consumed_samples: 63590400  total_loss: 3.506  time: 0.4028 s/iter  data_time: 0.2601 s/iter total_throughput: 2541.89 samples/s lr: 9.35e-04  
[09/15 20:32:25] lb.utils.events INFO:  eta: 13:34:38  iteration: 62199/375342  consumed_samples: 63692800  total_loss: 3.499  time: 0.4029 s/iter  data_time: 0.2521 s/iter total_throughput: 2541.86 samples/s lr: 9.34e-04  
[09/15 20:33:06] lb.utils.events INFO:  eta: 13:32:46  iteration: 62299/375342  consumed_samples: 63795200  total_loss: 3.501  time: 0.4029 s/iter  data_time: 0.2571 s/iter total_throughput: 2541.87 samples/s lr: 9.34e-04  
[09/15 20:33:46] lb.utils.events INFO:  eta: 13:33:53  iteration: 62399/375342  consumed_samples: 63897600  total_loss: 3.502  time: 0.4029 s/iter  data_time: 0.2649 s/iter total_throughput: 2541.81 samples/s lr: 9.34e-04  
[09/15 20:34:27] lb.utils.events INFO:  eta: 13:35:32  iteration: 62499/375342  consumed_samples: 64000000  total_loss: 3.483  time: 0.4029 s/iter  data_time: 0.2631 s/iter total_throughput: 2541.74 samples/s lr: 9.34e-04  
[09/15 20:35:08] lb.utils.events INFO:  eta: 13:35:16  iteration: 62599/375342  consumed_samples: 64102400  total_loss: 3.491  time: 0.4029 s/iter  data_time: 0.2547 s/iter total_throughput: 2541.72 samples/s lr: 9.34e-04  
[09/15 20:35:48] lb.utils.events INFO:  eta: 13:33:54  iteration: 62699/375342  consumed_samples: 64204800  total_loss: 3.486  time: 0.4029 s/iter  data_time: 0.2559 s/iter total_throughput: 2541.70 samples/s lr: 9.33e-04  
[09/15 20:36:29] lb.utils.events INFO:  eta: 13:29:49  iteration: 62799/375342  consumed_samples: 64307200  total_loss: 3.476  time: 0.4029 s/iter  data_time: 0.2648 s/iter total_throughput: 2541.65 samples/s lr: 9.33e-04  
[09/15 20:37:10] lb.utils.events INFO:  eta: 13:30:38  iteration: 62899/375342  consumed_samples: 64409600  total_loss: 3.476  time: 0.4029 s/iter  data_time: 0.2619 s/iter total_throughput: 2541.60 samples/s lr: 9.33e-04  
[09/15 20:37:50] lb.utils.events INFO:  eta: 13:31:26  iteration: 62999/375342  consumed_samples: 64512000  total_loss: 3.445  time: 0.4029 s/iter  data_time: 0.2475 s/iter total_throughput: 2541.59 samples/s lr: 9.33e-04  
[09/15 20:38:31] lb.utils.events INFO:  eta: 13:34:08  iteration: 63099/375342  consumed_samples: 64614400  total_loss: 3.456  time: 0.4029 s/iter  data_time: 0.2406 s/iter total_throughput: 2541.59 samples/s lr: 9.33e-04  
[09/15 20:39:11] lb.utils.events INFO:  eta: 13:34:30  iteration: 63199/375342  consumed_samples: 64716800  total_loss: 3.488  time: 0.4029 s/iter  data_time: 0.2650 s/iter total_throughput: 2541.56 samples/s lr: 9.32e-04  
[09/15 20:39:52] lb.utils.events INFO:  eta: 13:35:35  iteration: 63299/375342  consumed_samples: 64819200  total_loss: 3.472  time: 0.4029 s/iter  data_time: 0.2523 s/iter total_throughput: 2541.56 samples/s lr: 9.32e-04  
[09/15 20:40:32] lb.utils.events INFO:  eta: 13:35:56  iteration: 63399/375342  consumed_samples: 64921600  total_loss: 3.491  time: 0.4029 s/iter  data_time: 0.2517 s/iter total_throughput: 2541.57 samples/s lr: 9.32e-04  
[09/15 20:41:13] lb.utils.events INFO:  eta: 13:41:06  iteration: 63499/375342  consumed_samples: 65024000  total_loss: 3.504  time: 0.4029 s/iter  data_time: 0.2676 s/iter total_throughput: 2541.49 samples/s lr: 9.32e-04  
[09/15 20:41:54] lb.utils.events INFO:  eta: 13:36:37  iteration: 63599/375342  consumed_samples: 65126400  total_loss: 3.461  time: 0.4029 s/iter  data_time: 0.2522 s/iter total_throughput: 2541.45 samples/s lr: 9.32e-04  
[09/15 20:42:34] lb.utils.events INFO:  eta: 13:33:45  iteration: 63699/375342  consumed_samples: 65228800  total_loss: 3.434  time: 0.4029 s/iter  data_time: 0.2527 s/iter total_throughput: 2541.43 samples/s lr: 9.31e-04  
[09/15 20:43:15] lb.utils.events INFO:  eta: 13:39:59  iteration: 63799/375342  consumed_samples: 65331200  total_loss: 3.461  time: 0.4029 s/iter  data_time: 0.2695 s/iter total_throughput: 2541.39 samples/s lr: 9.31e-04  
[09/15 20:43:55] lb.utils.events INFO:  eta: 13:38:40  iteration: 63899/375342  consumed_samples: 65433600  total_loss: 3.464  time: 0.4029 s/iter  data_time: 0.2482 s/iter total_throughput: 2541.39 samples/s lr: 9.31e-04  
[09/15 20:44:36] lb.utils.events INFO:  eta: 13:40:08  iteration: 63999/375342  consumed_samples: 65536000  total_loss: 3.457  time: 0.4029 s/iter  data_time: 0.2450 s/iter total_throughput: 2541.38 samples/s lr: 9.31e-04  
[09/15 20:45:16] lb.utils.events INFO:  eta: 13:38:54  iteration: 64099/375342  consumed_samples: 65638400  total_loss: 3.465  time: 0.4029 s/iter  data_time: 0.2561 s/iter total_throughput: 2541.37 samples/s lr: 9.30e-04  
[09/15 20:45:56] lb.utils.events INFO:  eta: 13:34:20  iteration: 64199/375342  consumed_samples: 65740800  total_loss: 3.492  time: 0.4029 s/iter  data_time: 0.2511 s/iter total_throughput: 2541.35 samples/s lr: 9.30e-04  
[09/15 20:46:37] lb.utils.events INFO:  eta: 13:33:24  iteration: 64299/375342  consumed_samples: 65843200  total_loss: 3.486  time: 0.4029 s/iter  data_time: 0.2522 s/iter total_throughput: 2541.31 samples/s lr: 9.30e-04  
[09/15 20:47:17] lb.utils.events INFO:  eta: 13:36:23  iteration: 64399/375342  consumed_samples: 65945600  total_loss: 3.459  time: 0.4029 s/iter  data_time: 0.2506 s/iter total_throughput: 2541.32 samples/s lr: 9.30e-04  
[09/15 20:47:58] lb.utils.events INFO:  eta: 13:40:15  iteration: 64499/375342  consumed_samples: 66048000  total_loss: 3.463  time: 0.4029 s/iter  data_time: 0.2579 s/iter total_throughput: 2541.30 samples/s lr: 9.30e-04  
[09/15 20:48:38] lb.utils.events INFO:  eta: 14:11:29  iteration: 64599/375342  consumed_samples: 66150400  total_loss: 3.475  time: 0.4029 s/iter  data_time: 0.2547 s/iter total_throughput: 2541.28 samples/s lr: 9.29e-04  
[09/15 20:49:19] lb.utils.events INFO:  eta: 15:33:04  iteration: 64699/375342  consumed_samples: 66252800  total_loss: 3.463  time: 0.4030 s/iter  data_time: 0.2568 s/iter total_throughput: 2541.24 samples/s lr: 9.29e-04  
[09/15 20:50:00] lb.utils.events INFO:  eta: 15:09:46  iteration: 64799/375342  consumed_samples: 66355200  total_loss: 3.461  time: 0.4030 s/iter  data_time: 0.2545 s/iter total_throughput: 2541.21 samples/s lr: 9.29e-04  
[09/15 20:50:40] lb.utils.events INFO:  eta: 16:13:44  iteration: 64899/375342  consumed_samples: 66457600  total_loss: 3.47  time: 0.4030 s/iter  data_time: 0.2521 s/iter total_throughput: 2541.24 samples/s lr: 9.29e-04  
[09/15 20:51:20] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_2e56/model_0064999
[09/15 20:51:21] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000
[09/15 20:51:21] lb.evaluation.evaluator INFO: Start inference on 50000 samples
[09/15 20:51:25] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.1182 s/iter. Inference: 0.1488 s/iter. Eval: 0.0022 s/iter. Total: 0.2691 s/iter. ETA=0:00:09
[09/15 20:51:31] lb.evaluation.evaluator INFO: Inference done 27648/50000. Dataloading: 0.1553 s/iter. Inference: 0.1506 s/iter. Eval: 0.0022 s/iter. Total: 0.3082 s/iter. ETA=0:00:06
[09/15 20:51:36] lb.evaluation.evaluator INFO: Inference done 45056/50000. Dataloading: 0.1505 s/iter. Inference: 0.1507 s/iter. Eval: 0.0021 s/iter. Total: 0.3034 s/iter. ETA=0:00:01
[09/15 20:51:37] lb.evaluation.evaluator INFO: Total valid samples: 50000
[09/15 20:51:37] lb.evaluation.evaluator INFO: Total inference time: 0:00:13.379291 (0.000268 s / iter per device, on 8 devices)
[09/15 20:51:37] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000133 s / iter per device, on 8 devices)
[09/15 20:51:37] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format:
[09/15 20:51:37] lb.evaluation.utils INFO: copypaste: Acc@1=62.442
[09/15 20:51:37] lb.evaluation.utils INFO: copypaste: Acc@5=84.0
[09/15 20:51:37] lb.engine.hooks INFO: Not saving as latest eval score for Acc@1 is 62.44200, not better than best score 63.01600 @ iteration 59999.
[09/15 20:51:37] lb.utils.events INFO:  eta: 16:50:43  iteration: 64999/375342  consumed_samples: 66560000  total_loss: 3.475  time: 0.4030 s/iter  data_time: 0.2600 s/iter total_throughput: 2541.22 samples/s lr: 9.29e-04  
[09/15 20:52:15] lb.utils.events INFO:  eta: 14:31:32  iteration: 65099/375342  consumed_samples: 66662400  total_loss: 3.47  time: 0.4029 s/iter  data_time: 0.2558 s/iter total_throughput: 2541.41 samples/s lr: 9.28e-04  
[09/15 20:52:56] lb.utils.events INFO:  eta: 17:07:10  iteration: 65199/375342  consumed_samples: 66764800  total_loss: 3.457  time: 0.4029 s/iter  data_time: 0.2568 s/iter total_throughput: 2541.38 samples/s lr: 9.28e-04  
[09/15 20:53:37] lb.utils.events INFO:  eta: 16:11:26  iteration: 65299/375342  consumed_samples: 66867200  total_loss: 3.448  time: 0.4029 s/iter  data_time: 0.2692 s/iter total_throughput: 2541.35 samples/s lr: 9.28e-04  
[09/15 20:54:17] lb.utils.events INFO:  eta: 14:04:32  iteration: 65399/375342  consumed_samples: 66969600  total_loss: 3.459  time: 0.4029 s/iter  data_time: 0.2536 s/iter total_throughput: 2541.32 samples/s lr: 9.28e-04  
[09/15 20:54:58] lb.utils.events INFO:  eta: 13:42:38  iteration: 65499/375342  consumed_samples: 67072000  total_loss: 3.47  time: 0.4029 s/iter  data_time: 0.2651 s/iter total_throughput: 2541.30 samples/s lr: 9.27e-04  
[09/15 20:55:38] lb.utils.events INFO:  eta: 13:31:47  iteration: 65599/375342  consumed_samples: 67174400  total_loss: 3.487  time: 0.4029 s/iter  data_time: 0.2450 s/iter total_throughput: 2541.30 samples/s lr: 9.27e-04  
[09/15 20:56:19] lb.utils.events INFO:  eta: 13:30:46  iteration: 65699/375342  consumed_samples: 67276800  total_loss: 3.451  time: 0.4029 s/iter  data_time: 0.2522 s/iter total_throughput: 2541.29 samples/s lr: 9.27e-04  
[09/15 20:56:58] lb.utils.events INFO:  eta: 13:30:27  iteration: 65799/375342  consumed_samples: 67379200  total_loss: 3.461  time: 0.4029 s/iter  data_time: 0.2491 s/iter total_throughput: 2541.34 samples/s lr: 9.27e-04  
[09/15 20:57:39] lb.utils.events INFO:  eta: 13:30:15  iteration: 65899/375342  consumed_samples: 67481600  total_loss: 3.469  time: 0.4029 s/iter  data_time: 0.2501 s/iter total_throughput: 2541.32 samples/s lr: 9.27e-04  
[09/15 20:58:19] lb.utils.events INFO:  eta: 13:34:15  iteration: 65999/375342  consumed_samples: 67584000  total_loss: 3.45  time: 0.4029 s/iter  data_time: 0.2477 s/iter total_throughput: 2541.31 samples/s lr: 9.26e-04  
[09/15 20:59:00] lb.utils.events INFO:  eta: 19:18:55  iteration: 66099/375342  consumed_samples: 67686400  total_loss: 3.45  time: 0.4029 s/iter  data_time: 0.2552 s/iter total_throughput: 2541.28 samples/s lr: 9.26e-04  
[09/15 20:59:41] lb.utils.events INFO:  eta: 20:17:36  iteration: 66199/375342  consumed_samples: 67788800  total_loss: 3.457  time: 0.4030 s/iter  data_time: 0.2476 s/iter total_throughput: 2541.23 samples/s lr: 9.26e-04  
[09/15 21:00:21] lb.utils.events INFO:  eta: 19:46:42  iteration: 66299/375342  consumed_samples: 67891200  total_loss: 3.471  time: 0.4030 s/iter  data_time: 0.2489 s/iter total_throughput: 2541.22 samples/s lr: 9.26e-04  
[09/15 21:01:01] lb.utils.events INFO:  eta: 19:47:33  iteration: 66399/375342  consumed_samples: 67993600  total_loss: 3.455  time: 0.4030 s/iter  data_time: 0.2522 s/iter total_throughput: 2541.25 samples/s lr: 9.26e-04  
[09/15 21:01:42] lb.utils.events INFO:  eta: 18:31:49  iteration: 66499/375342  consumed_samples: 68096000  total_loss: 3.422  time: 0.4030 s/iter  data_time: 0.2542 s/iter total_throughput: 2541.23 samples/s lr: 9.25e-04  
[09/15 21:02:22] lb.utils.events INFO:  eta: 23:21:25  iteration: 66599/375342  consumed_samples: 68198400  total_loss: 3.441  time: 0.4030 s/iter  data_time: 0.2452 s/iter total_throughput: 2541.23 samples/s lr: 9.25e-04  
[09/15 21:03:02] lb.utils.events INFO:  eta: 1 day, 0:09:53  iteration: 66699/375342  consumed_samples: 68300800  total_loss: 3.465  time: 0.4030 s/iter  data_time: 0.2571 s/iter total_throughput: 2541.23 samples/s lr: 9.25e-04  
[09/15 21:03:43] lb.utils.events INFO:  eta: 1 day, 0:12:45  iteration: 66799/375342  consumed_samples: 68403200  total_loss: 3.446  time: 0.4030 s/iter  data_time: 0.2563 s/iter total_throughput: 2541.22 samples/s lr: 9.25e-04  
[09/15 21:04:23] lb.utils.events INFO:  eta: 1 day, 0:12:17  iteration: 66899/375342  consumed_samples: 68505600  total_loss: 3.468  time: 0.4030 s/iter  data_time: 0.2527 s/iter total_throughput: 2541.24 samples/s lr: 9.24e-04  
[09/15 21:05:04] lb.utils.events INFO:  eta: 22:02:53  iteration: 66999/375342  consumed_samples: 68608000  total_loss: 3.471  time: 0.4030 s/iter  data_time: 0.2650 s/iter total_throughput: 2541.19 samples/s lr: 9.24e-04  
[09/15 21:05:44] lb.utils.events INFO:  eta: 14:06:10  iteration: 67099/375342  consumed_samples: 68710400  total_loss: 3.461  time: 0.4030 s/iter  data_time: 0.2616 s/iter total_throughput: 2541.18 samples/s lr: 9.24e-04  
[09/15 21:06:25] lb.utils.events INFO:  eta: 13:38:22  iteration: 67199/375342  consumed_samples: 68812800  total_loss: 3.448  time: 0.4030 s/iter  data_time: 0.2661 s/iter total_throughput: 2541.14 samples/s lr: 9.24e-04  
[09/15 21:07:05] lb.utils.events INFO:  eta: 13:41:20  iteration: 67299/375342  consumed_samples: 68915200  total_loss: 3.452  time: 0.4030 s/iter  data_time: 0.2495 s/iter total_throughput: 2541.16 samples/s lr: 9.24e-04  
[09/15 21:07:45] lb.utils.events INFO:  eta: 13:29:00  iteration: 67399/375342  consumed_samples: 69017600  total_loss: 3.456  time: 0.4030 s/iter  data_time: 0.2687 s/iter total_throughput: 2541.14 samples/s lr: 9.23e-04  
[09/15 21:08:25] lb.utils.events INFO:  eta: 13:35:47  iteration: 67499/375342  consumed_samples: 69120000  total_loss: 3.453  time: 0.4030 s/iter  data_time: 0.2539 s/iter total_throughput: 2541.17 samples/s lr: 9.23e-04  
[09/15 21:09:07] lb.utils.events INFO:  eta: 13:20:31  iteration: 67599/375342  consumed_samples: 69222400  total_loss: 3.469  time: 0.4030 s/iter  data_time: 0.2591 s/iter total_throughput: 2541.09 samples/s lr: 9.23e-04  
[09/15 21:09:47] lb.utils.events INFO:  eta: 13:16:04  iteration: 67699/375342  consumed_samples: 69324800  total_loss: 3.429  time: 0.4030 s/iter  data_time: 0.2491 s/iter total_throughput: 2541.12 samples/s lr: 9.23e-04  
[09/15 21:10:26] lb.utils.events INFO:  eta: 13:14:59  iteration: 67799/375342  consumed_samples: 69427200  total_loss: 3.422  time: 0.4030 s/iter  data_time: 0.2474 s/iter total_throughput: 2541.16 samples/s lr: 9.22e-04  
[09/15 21:11:06] lb.utils.events INFO:  eta: 13:14:10  iteration: 67899/375342  consumed_samples: 69529600  total_loss: 3.462  time: 0.4030 s/iter  data_time: 0.2568 s/iter total_throughput: 2541.18 samples/s lr: 9.22e-04  
[09/15 21:11:47] lb.utils.events INFO:  eta: 13:19:56  iteration: 67999/375342  consumed_samples: 69632000  total_loss: 3.447  time: 0.4030 s/iter  data_time: 0.2567 s/iter total_throughput: 2541.20 samples/s lr: 9.22e-04  
[09/15 21:12:27] lb.utils.events INFO:  eta: 13:24:16  iteration: 68099/375342  consumed_samples: 69734400  total_loss: 3.447  time: 0.4030 s/iter  data_time: 0.2510 s/iter total_throughput: 2541.19 samples/s lr: 9.22e-04  
[09/15 21:13:08] lb.utils.events INFO:  eta: 13:22:19  iteration: 68199/375342  consumed_samples: 69836800  total_loss: 3.443  time: 0.4030 s/iter  data_time: 0.2786 s/iter total_throughput: 2541.15 samples/s lr: 9.22e-04  
[09/15 21:13:48] lb.utils.events INFO:  eta: 13:20:47  iteration: 68299/375342  consumed_samples: 69939200  total_loss: 3.417  time: 0.4030 s/iter  data_time: 0.2460 s/iter total_throughput: 2541.15 samples/s lr: 9.21e-04  
[09/15 21:14:28] lb.utils.events INFO:  eta: 13:18:14  iteration: 68399/375342  consumed_samples: 70041600  total_loss: 3.431  time: 0.4030 s/iter  data_time: 0.2577 s/iter total_throughput: 2541.17 samples/s lr: 9.21e-04  
[09/15 21:15:08] lb.utils.events INFO:  eta: 13:15:54  iteration: 68499/375342  consumed_samples: 70144000  total_loss: 3.456  time: 0.4030 s/iter  data_time: 0.2513 s/iter total_throughput: 2541.22 samples/s lr: 9.21e-04  
[09/15 21:15:48] lb.utils.events INFO:  eta: 13:16:52  iteration: 68599/375342  consumed_samples: 70246400  total_loss: 3.436  time: 0.4030 s/iter  data_time: 0.2467 s/iter total_throughput: 2541.24 samples/s lr: 9.21e-04  
[09/15 21:16:28] lb.utils.events INFO:  eta: 13:21:00  iteration: 68699/375342  consumed_samples: 70348800  total_loss: 3.442  time: 0.4029 s/iter  data_time: 0.2464 s/iter total_throughput: 2541.30 samples/s lr: 9.20e-04  
[09/15 21:17:08] lb.utils.events INFO:  eta: 13:24:31  iteration: 68799/375342  consumed_samples: 70451200  total_loss: 3.449  time: 0.4029 s/iter  data_time: 0.2529 s/iter total_throughput: 2541.33 samples/s lr: 9.20e-04  
[09/15 21:17:48] lb.utils.events INFO:  eta: 13:23:45  iteration: 68899/375342  consumed_samples: 70553600  total_loss: 3.441  time: 0.4029 s/iter  data_time: 0.2506 s/iter total_throughput: 2541.36 samples/s lr: 9.20e-04  
[09/15 21:18:27] lb.utils.events INFO:  eta: 13:20:56  iteration: 68999/375342  consumed_samples: 70656000  total_loss: 3.416  time: 0.4029 s/iter  data_time: 0.2448 s/iter total_throughput: 2541.39 samples/s lr: 9.20e-04  
[09/15 21:19:07] lb.utils.events INFO:  eta: 13:25:51  iteration: 69099/375342  consumed_samples: 70758400  total_loss: 3.415  time: 0.4029 s/iter  data_time: 0.2470 s/iter total_throughput: 2541.44 samples/s lr: 9.19e-04  
[09/15 21:19:48] lb.utils.events INFO:  eta: 13:38:42  iteration: 69199/375342  consumed_samples: 70860800  total_loss: 3.43  time: 0.4029 s/iter  data_time: 0.2560 s/iter total_throughput: 2541.42 samples/s lr: 9.19e-04  
[09/15 21:20:28] lb.utils.events INFO:  eta: 14:04:35  iteration: 69299/375342  consumed_samples: 70963200  total_loss: 3.425  time: 0.4029 s/iter  data_time: 0.2455 s/iter total_throughput: 2541.45 samples/s lr: 9.19e-04  
[09/15 21:21:08] lb.utils.events INFO:  eta: 14:07:14  iteration: 69399/375342  consumed_samples: 71065600  total_loss: 3.424  time: 0.4029 s/iter  data_time: 0.2519 s/iter total_throughput: 2541.45 samples/s lr: 9.19e-04  
[09/15 21:21:48] lb.utils.events INFO:  eta: 13:45:25  iteration: 69499/375342  consumed_samples: 71168000  total_loss: 3.442  time: 0.4029 s/iter  data_time: 0.2496 s/iter total_throughput: 2541.45 samples/s lr: 9.19e-04  
[09/15 21:22:28] lb.utils.events INFO:  eta: 13:31:54  iteration: 69599/375342  consumed_samples: 71270400  total_loss: 3.461  time: 0.4029 s/iter  data_time: 0.2513 s/iter total_throughput: 2541.52 samples/s lr: 9.18e-04  
[09/15 21:23:07] lb.utils.events INFO:  eta: 13:30:26  iteration: 69699/375342  consumed_samples: 71372800  total_loss: 3.448  time: 0.4029 s/iter  data_time: 0.2463 s/iter total_throughput: 2541.59 samples/s lr: 9.18e-04  
[09/15 21:23:48] lb.utils.events INFO:  eta: 13:45:03  iteration: 69799/375342  consumed_samples: 71475200  total_loss: 3.426  time: 0.4029 s/iter  data_time: 0.2582 s/iter total_throughput: 2541.61 samples/s lr: 9.18e-04  
[09/15 21:24:28] lb.utils.events INFO:  eta: 13:32:12  iteration: 69899/375342  consumed_samples: 71577600  total_loss: 3.43  time: 0.4029 s/iter  data_time: 0.2507 s/iter total_throughput: 2541.63 samples/s lr: 9.18e-04  
[09/15 21:25:07] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_2e56/model_0069999
[09/15 21:25:08] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000
[09/15 21:25:08] lb.evaluation.evaluator INFO: Start inference on 50000 samples
[09/15 21:25:12] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.1170 s/iter. Inference: 0.1506 s/iter. Eval: 0.0020 s/iter. Total: 0.2697 s/iter. ETA=0:00:09
[09/15 21:25:17] lb.evaluation.evaluator INFO: Inference done 27648/50000. Dataloading: 0.1505 s/iter. Inference: 0.1500 s/iter. Eval: 0.0021 s/iter. Total: 0.3027 s/iter. ETA=0:00:06
[09/15 21:25:22] lb.evaluation.evaluator INFO: Inference done 45056/50000. Dataloading: 0.1482 s/iter. Inference: 0.1502 s/iter. Eval: 0.0021 s/iter. Total: 0.3005 s/iter. ETA=0:00:01
[09/15 21:25:24] lb.evaluation.evaluator INFO: Total valid samples: 50000
[09/15 21:25:24] lb.evaluation.evaluator INFO: Total inference time: 0:00:13.321255 (0.000266 s / iter per device, on 8 devices)
[09/15 21:25:24] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000132 s / iter per device, on 8 devices)
[09/15 21:25:24] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format:
[09/15 21:25:24] lb.evaluation.utils INFO: copypaste: Acc@1=64.254
[09/15 21:25:24] lb.evaluation.utils INFO: copypaste: Acc@5=85.612
[09/15 21:25:24] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 64.25400, better than last best score 63.01600 @ iteration 59999.
[09/15 21:25:24] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_2e56/model_best
[09/15 21:25:25] lb.utils.events INFO:  eta: 13:22:06  iteration: 69999/375342  consumed_samples: 71680000  total_loss: 3.446  time: 0.4029 s/iter  data_time: 0.2496 s/iter total_throughput: 2541.71 samples/s lr: 9.17e-04  
[09/15 21:26:03] lb.utils.events INFO:  eta: 13:16:37  iteration: 70099/375342  consumed_samples: 71782400  total_loss: 3.447  time: 0.4029 s/iter  data_time: 0.2764 s/iter total_throughput: 2541.85 samples/s lr: 9.17e-04  
[09/15 21:26:44] lb.utils.events INFO:  eta: 13:14:05  iteration: 70199/375342  consumed_samples: 71884800  total_loss: 3.439  time: 0.4029 s/iter  data_time: 0.2543 s/iter total_throughput: 2541.85 samples/s lr: 9.17e-04  
[09/15 21:27:24] lb.utils.events INFO:  eta: 13:11:56  iteration: 70299/375342  consumed_samples: 71987200  total_loss: 3.439  time: 0.4029 s/iter  data_time: 0.2502 s/iter total_throughput: 2541.82 samples/s lr: 9.17e-04  
[09/15 21:28:06] lb.utils.events INFO:  eta: 13:12:06  iteration: 70399/375342  consumed_samples: 72089600  total_loss: 3.431  time: 0.4029 s/iter  data_time: 0.2730 s/iter total_throughput: 2541.69 samples/s lr: 9.17e-04  
[09/15 21:28:48] lb.utils.events INFO:  eta: 13:13:16  iteration: 70499/375342  consumed_samples: 72192000  total_loss: 3.443  time: 0.4029 s/iter  data_time: 0.2507 s/iter total_throughput: 2541.57 samples/s lr: 9.16e-04  
[09/15 21:29:29] lb.utils.events INFO:  eta: 13:17:52  iteration: 70599/375342  consumed_samples: 72294400  total_loss: 3.443  time: 0.4029 s/iter  data_time: 0.2745 s/iter total_throughput: 2541.44 samples/s lr: 9.16e-04  
[09/15 21:30:11] lb.utils.events INFO:  eta: 13:18:21  iteration: 70699/375342  consumed_samples: 72396800  total_loss: 3.419  time: 0.4029 s/iter  data_time: 0.2779 s/iter total_throughput: 2541.28 samples/s lr: 9.16e-04  
[09/15 21:30:53] lb.utils.events INFO:  eta: 13:12:40  iteration: 70799/375342  consumed_samples: 72499200  total_loss: 3.409  time: 0.4030 s/iter  data_time: 0.2686 s/iter total_throughput: 2541.13 samples/s lr: 9.16e-04  
[09/15 21:31:35] lb.utils.events INFO:  eta: 13:10:07  iteration: 70899/375342  consumed_samples: 72601600  total_loss: 3.419  time: 0.4030 s/iter  data_time: 0.2559 s/iter total_throughput: 2541.02 samples/s lr: 9.15e-04  
[09/15 21:32:16] lb.utils.events INFO:  eta: 13:07:03  iteration: 70999/375342  consumed_samples: 72704000  total_loss: 3.432  time: 0.4030 s/iter  data_time: 0.2512 s/iter total_throughput: 2540.96 samples/s lr: 9.15e-04  
[09/15 21:32:56] lb.utils.events INFO:  eta: 13:07:23  iteration: 71099/375342  consumed_samples: 72806400  total_loss: 3.419  time: 0.4030 s/iter  data_time: 0.2499 s/iter total_throughput: 2540.95 samples/s lr: 9.15e-04  
[09/15 21:33:37] lb.utils.events INFO:  eta: 13:10:25  iteration: 71199/375342  consumed_samples: 72908800  total_loss: 3.424  time: 0.4030 s/iter  data_time: 0.2629 s/iter total_throughput: 2540.91 samples/s lr: 9.15e-04  
[09/15 21:34:18] lb.utils.events INFO:  eta: 13:12:46  iteration: 71299/375342  consumed_samples: 73011200  total_loss: 3.443  time: 0.4030 s/iter  data_time: 0.2705 s/iter total_throughput: 2540.84 samples/s lr: 9.14e-04  
[09/15 21:35:00] lb.utils.events INFO:  eta: 13:09:32  iteration: 71399/375342  consumed_samples: 73113600  total_loss: 3.429  time: 0.4030 s/iter  data_time: 0.2705 s/iter total_throughput: 2540.69 samples/s lr: 9.14e-04  
[09/15 21:35:42] lb.utils.events INFO:  eta: 13:07:27  iteration: 71499/375342  consumed_samples: 73216000  total_loss: 3.406  time: 0.4031 s/iter  data_time: 0.2688 s/iter total_throughput: 2540.56 samples/s lr: 9.14e-04  
[09/15 21:36:24] lb.utils.events INFO:  eta: 13:04:23  iteration: 71599/375342  consumed_samples: 73318400  total_loss: 3.411  time: 0.4031 s/iter  data_time: 0.2687 s/iter total_throughput: 2540.40 samples/s lr: 9.14e-04  
[09/15 21:37:05] lb.utils.events INFO:  eta: 13:02:21  iteration: 71699/375342  consumed_samples: 73420800  total_loss: 3.411  time: 0.4031 s/iter  data_time: 0.2605 s/iter total_throughput: 2540.33 samples/s lr: 9.14e-04  
[09/15 21:37:46] lb.utils.events INFO:  eta: 13:02:44  iteration: 71799/375342  consumed_samples: 73523200  total_loss: 3.41  time: 0.4031 s/iter  data_time: 0.2731 s/iter total_throughput: 2540.26 samples/s lr: 9.13e-04  
[09/15 21:38:27] lb.utils.events INFO:  eta: 13:04:03  iteration: 71899/375342  consumed_samples: 73625600  total_loss: 3.419  time: 0.4031 s/iter  data_time: 0.2563 s/iter total_throughput: 2540.20 samples/s lr: 9.13e-04  
[09/15 21:39:08] lb.utils.events INFO:  eta: 13:05:39  iteration: 71999/375342  consumed_samples: 73728000  total_loss: 3.422  time: 0.4031 s/iter  data_time: 0.2526 s/iter total_throughput: 2540.21 samples/s lr: 9.13e-04  
[09/15 21:39:48] lb.utils.events INFO:  eta: 13:03:45  iteration: 72099/375342  consumed_samples: 73830400  total_loss: 3.394  time: 0.4031 s/iter  data_time: 0.2458 s/iter total_throughput: 2540.21 samples/s lr: 9.13e-04  
[09/15 21:40:29] lb.utils.events INFO:  eta: 13:03:37  iteration: 72199/375342  consumed_samples: 73932800  total_loss: 3.389  time: 0.4031 s/iter  data_time: 0.2631 s/iter total_throughput: 2540.15 samples/s lr: 9.12e-04  
[09/15 21:41:10] lb.utils.events INFO:  eta: 13:02:37  iteration: 72299/375342  consumed_samples: 74035200  total_loss: 3.373  time: 0.4031 s/iter  data_time: 0.2571 s/iter total_throughput: 2540.12 samples/s lr: 9.12e-04  
[09/15 21:41:51] lb.utils.events INFO:  eta: 13:04:44  iteration: 72399/375342  consumed_samples: 74137600  total_loss: 3.392  time: 0.4031 s/iter  data_time: 0.2582 s/iter total_throughput: 2540.06 samples/s lr: 9.12e-04  
[09/15 21:42:32] lb.utils.events INFO:  eta: 13:04:30  iteration: 72499/375342  consumed_samples: 74240000  total_loss: 3.42  time: 0.4031 s/iter  data_time: 0.2554 s/iter total_throughput: 2540.01 samples/s lr: 9.12e-04  
[09/15 21:43:12] lb.utils.events INFO:  eta: 13:05:28  iteration: 72599/375342  consumed_samples: 74342400  total_loss: 3.408  time: 0.4032 s/iter  data_time: 0.2544 s/iter total_throughput: 2539.98 samples/s lr: 9.11e-04  
[09/15 21:43:54] lb.utils.events INFO:  eta: 13:05:28  iteration: 72699/375342  consumed_samples: 74444800  total_loss: 3.409  time: 0.4032 s/iter  data_time: 0.2620 s/iter total_throughput: 2539.88 samples/s lr: 9.11e-04  
[09/15 21:44:34] lb.utils.events INFO:  eta: 13:03:17  iteration: 72799/375342  consumed_samples: 74547200  total_loss: 3.396  time: 0.4032 s/iter  data_time: 0.2610 s/iter total_throughput: 2539.85 samples/s lr: 9.11e-04  
[09/15 21:45:15] lb.utils.events INFO:  eta: 13:02:50  iteration: 72899/375342  consumed_samples: 74649600  total_loss: 3.359  time: 0.4032 s/iter  data_time: 0.2551 s/iter total_throughput: 2539.82 samples/s lr: 9.11e-04  
[09/15 21:45:56] lb.utils.events INFO:  eta: 13:02:26  iteration: 72999/375342  consumed_samples: 74752000  total_loss: 3.39  time: 0.4032 s/iter  data_time: 0.2566 s/iter total_throughput: 2539.78 samples/s lr: 9.10e-04  
[09/15 21:46:37] lb.utils.events INFO:  eta: 13:03:32  iteration: 73099/375342  consumed_samples: 74854400  total_loss: 3.404  time: 0.4032 s/iter  data_time: 0.2528 s/iter total_throughput: 2539.74 samples/s lr: 9.10e-04  
[09/15 21:47:17] lb.utils.events INFO:  eta: 13:02:35  iteration: 73199/375342  consumed_samples: 74956800  total_loss: 3.391  time: 0.4032 s/iter  data_time: 0.2543 s/iter total_throughput: 2539.73 samples/s lr: 9.10e-04  
[09/15 21:47:58] lb.utils.events INFO:  eta: 13:02:15  iteration: 73299/375342  consumed_samples: 75059200  total_loss: 3.414  time: 0.4032 s/iter  data_time: 0.2731 s/iter total_throughput: 2539.65 samples/s lr: 9.10e-04  
[09/15 21:48:39] lb.utils.events INFO:  eta: 13:01:24  iteration: 73399/375342  consumed_samples: 75161600  total_loss: 3.412  time: 0.4032 s/iter  data_time: 0.2618 s/iter total_throughput: 2539.62 samples/s lr: 9.09e-04  
[09/15 21:49:20] lb.utils.events INFO:  eta: 13:02:11  iteration: 73499/375342  consumed_samples: 75264000  total_loss: 3.402  time: 0.4032 s/iter  data_time: 0.2627 s/iter total_throughput: 2539.57 samples/s lr: 9.09e-04  
[09/15 21:50:01] lb.utils.events INFO:  eta: 13:02:21  iteration: 73599/375342  consumed_samples: 75366400  total_loss: 3.389  time: 0.4032 s/iter  data_time: 0.2529 s/iter total_throughput: 2539.53 samples/s lr: 9.09e-04  
[09/15 21:50:41] lb.utils.events INFO:  eta: 13:05:16  iteration: 73699/375342  consumed_samples: 75468800  total_loss: 3.399  time: 0.4032 s/iter  data_time: 0.2520 s/iter total_throughput: 2539.54 samples/s lr: 9.09e-04  
[09/15 21:51:21] lb.utils.events INFO:  eta: 13:09:13  iteration: 73799/375342  consumed_samples: 75571200  total_loss: 3.414  time: 0.4032 s/iter  data_time: 0.2563 s/iter total_throughput: 2539.53 samples/s lr: 9.09e-04  
[09/15 21:52:02] lb.utils.events INFO:  eta: 13:13:20  iteration: 73899/375342  consumed_samples: 75673600  total_loss: 3.401  time: 0.4032 s/iter  data_time: 0.2635 s/iter total_throughput: 2539.47 samples/s lr: 9.08e-04  
[09/15 21:52:43] lb.utils.events INFO:  eta: 13:14:42  iteration: 73999/375342  consumed_samples: 75776000  total_loss: 3.377  time: 0.4032 s/iter  data_time: 0.2500 s/iter total_throughput: 2539.44 samples/s lr: 9.08e-04  
[09/15 21:53:24] lb.utils.events INFO:  eta: 13:14:04  iteration: 74099/375342  consumed_samples: 75878400  total_loss: 3.375  time: 0.4032 s/iter  data_time: 0.2561 s/iter total_throughput: 2539.41 samples/s lr: 9.08e-04  
[09/15 21:54:05] lb.utils.events INFO:  eta: 13:10:56  iteration: 74199/375342  consumed_samples: 75980800  total_loss: 3.389  time: 0.4033 s/iter  data_time: 0.2731 s/iter total_throughput: 2539.30 samples/s lr: 9.08e-04  
[09/15 21:54:46] lb.utils.events INFO:  eta: 13:09:44  iteration: 74299/375342  consumed_samples: 76083200  total_loss: 3.387  time: 0.4033 s/iter  data_time: 0.2629 s/iter total_throughput: 2539.25 samples/s lr: 9.07e-04  
[09/15 21:55:27] lb.utils.events INFO:  eta: 13:08:20  iteration: 74399/375342  consumed_samples: 76185600  total_loss: 3.394  time: 0.4033 s/iter  data_time: 0.2553 s/iter total_throughput: 2539.24 samples/s lr: 9.07e-04  
[09/15 21:56:07] lb.utils.events INFO:  eta: 13:04:51  iteration: 74499/375342  consumed_samples: 76288000  total_loss: 3.403  time: 0.4033 s/iter  data_time: 0.2542 s/iter total_throughput: 2539.21 samples/s lr: 9.07e-04  
[09/15 21:56:48] lb.utils.events INFO:  eta: 13:06:11  iteration: 74599/375342  consumed_samples: 76390400  total_loss: 3.409  time: 0.4033 s/iter  data_time: 0.2548 s/iter total_throughput: 2539.22 samples/s lr: 9.07e-04  
[09/15 21:57:28] lb.utils.events INFO:  eta: 13:03:34  iteration: 74699/375342  consumed_samples: 76492800  total_loss: 3.4  time: 0.4033 s/iter  data_time: 0.2616 s/iter total_throughput: 2539.18 samples/s lr: 9.06e-04  
[09/15 21:58:09] lb.utils.events INFO:  eta: 13:02:02  iteration: 74799/375342  consumed_samples: 76595200  total_loss: 3.395  time: 0.4033 s/iter  data_time: 0.2522 s/iter total_throughput: 2539.17 samples/s lr: 9.06e-04  
[09/15 21:58:49] lb.utils.events INFO:  eta: 12:58:08  iteration: 74899/375342  consumed_samples: 76697600  total_loss: 3.407  time: 0.4033 s/iter  data_time: 0.2428 s/iter total_throughput: 2539.17 samples/s lr: 9.06e-04  
[09/15 21:59:30] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_2e56/model_0074999
[09/15 21:59:31] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000
[09/15 21:59:31] lb.evaluation.evaluator INFO: Start inference on 50000 samples
[09/15 21:59:35] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.1175 s/iter. Inference: 0.1493 s/iter. Eval: 0.0023 s/iter. Total: 0.2691 s/iter. ETA=0:00:09
[09/15 21:59:41] lb.evaluation.evaluator INFO: Inference done 27648/50000. Dataloading: 0.1518 s/iter. Inference: 0.1513 s/iter. Eval: 0.0021 s/iter. Total: 0.3053 s/iter. ETA=0:00:06
[09/15 21:59:46] lb.evaluation.evaluator INFO: Inference done 45056/50000. Dataloading: 0.1489 s/iter. Inference: 0.1513 s/iter. Eval: 0.0021 s/iter. Total: 0.3024 s/iter. ETA=0:00:01
[09/15 21:59:47] lb.evaluation.evaluator INFO: Total valid samples: 50000
[09/15 21:59:47] lb.evaluation.evaluator INFO: Total inference time: 0:00:13.524918 (0.000271 s / iter per device, on 8 devices)
[09/15 21:59:47] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000134 s / iter per device, on 8 devices)
[09/15 21:59:47] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format:
[09/15 21:59:47] lb.evaluation.utils INFO: copypaste: Acc@1=64.668
[09/15 21:59:47] lb.evaluation.utils INFO: copypaste: Acc@5=86.13
[09/15 21:59:47] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 64.66800, better than last best score 64.25400 @ iteration 69999.
[09/15 21:59:47] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_2e56/model_best
[09/15 21:59:48] lb.utils.events INFO:  eta: 12:57:32  iteration: 74999/375342  consumed_samples: 76800000  total_loss: 3.407  time: 0.4033 s/iter  data_time: 0.2631 s/iter total_throughput: 2539.11 samples/s lr: 9.06e-04  
[09/15 22:00:27] lb.utils.events INFO:  eta: 12:58:26  iteration: 75099/375342  consumed_samples: 76902400  total_loss: 3.405  time: 0.4033 s/iter  data_time: 0.2525 s/iter total_throughput: 2539.25 samples/s lr: 9.05e-04  
[09/15 22:01:08] lb.utils.events INFO:  eta: 12:59:07  iteration: 75199/375342  consumed_samples: 77004800  total_loss: 3.38  time: 0.4033 s/iter  data_time: 0.2681 s/iter total_throughput: 2539.18 samples/s lr: 9.05e-04  
[09/15 22:01:48] lb.utils.events INFO:  eta: 13:01:47  iteration: 75299/375342  consumed_samples: 77107200  total_loss: 3.369  time: 0.4033 s/iter  data_time: 0.2633 s/iter total_throughput: 2539.17 samples/s lr: 9.05e-04  
[09/15 22:02:29] lb.utils.events INFO:  eta: 13:03:45  iteration: 75399/375342  consumed_samples: 77209600  total_loss: 3.401  time: 0.4033 s/iter  data_time: 0.2643 s/iter total_throughput: 2539.17 samples/s lr: 9.05e-04  
[09/15 22:03:09] lb.utils.events INFO:  eta: 13:05:09  iteration: 75499/375342  consumed_samples: 77312000  total_loss: 3.386  time: 0.4033 s/iter  data_time: 0.2541 s/iter total_throughput: 2539.15 samples/s lr: 9.04e-04  
[09/15 22:03:50] lb.utils.events INFO:  eta: 13:05:39  iteration: 75599/375342  consumed_samples: 77414400  total_loss: 3.368  time: 0.4033 s/iter  data_time: 0.2520 s/iter total_throughput: 2539.16 samples/s lr: 9.04e-04  
[09/15 22:04:30] lb.utils.events INFO:  eta: 13:17:06  iteration: 75699/375342  consumed_samples: 77516800  total_loss: 3.387  time: 0.4033 s/iter  data_time: 0.2517 s/iter total_throughput: 2539.15 samples/s lr: 9.04e-04  
[09/15 22:05:11] lb.utils.events INFO:  eta: 16:14:21  iteration: 75799/375342  consumed_samples: 77619200  total_loss: 3.377  time: 0.4033 s/iter  data_time: 0.2578 s/iter total_throughput: 2539.13 samples/s lr: 9.04e-04  
[09/15 22:05:52] lb.utils.events INFO:  eta: 21:56:32  iteration: 75899/375342  consumed_samples: 77721600  total_loss: 3.378  time: 0.4033 s/iter  data_time: 0.2558 s/iter total_throughput: 2539.06 samples/s lr: 9.03e-04  
[09/15 22:06:32] lb.utils.events INFO:  eta: 22:19:15  iteration: 75999/375342  consumed_samples: 77824000  total_loss: 3.394  time: 0.4033 s/iter  data_time: 0.2533 s/iter total_throughput: 2539.06 samples/s lr: 9.03e-04  
[09/15 22:07:13] lb.utils.events INFO:  eta: 1 day, 1:19:24  iteration: 76099/375342  consumed_samples: 77926400  total_loss: 3.388  time: 0.4033 s/iter  data_time: 0.2575 s/iter total_throughput: 2539.01 samples/s lr: 9.03e-04  
[09/15 22:07:54] lb.utils.events INFO:  eta: 1 day, 3:10:21  iteration: 76199/375342  consumed_samples: 78028800  total_loss: 3.395  time: 0.4033 s/iter  data_time: 0.2671 s/iter total_throughput: 2538.99 samples/s lr: 9.03e-04  
[09/15 22:08:35] lb.utils.events INFO:  eta: 1 day, 0:34:09  iteration: 76299/375342  consumed_samples: 78131200  total_loss: 3.394  time: 0.4033 s/iter  data_time: 0.2587 s/iter total_throughput: 2538.95 samples/s lr: 9.02e-04  
[09/15 22:09:15] lb.utils.events INFO:  eta: 1 day, 0:24:03  iteration: 76399/375342  consumed_samples: 78233600  total_loss: 3.369  time: 0.4033 s/iter  data_time: 0.2506 s/iter total_throughput: 2538.92 samples/s lr: 9.02e-04  
[09/15 22:09:56] lb.utils.events INFO:  eta: 1 day, 1:48:31  iteration: 76499/375342  consumed_samples: 78336000  total_loss: 3.364  time: 0.4033 s/iter  data_time: 0.2518 s/iter total_throughput: 2538.90 samples/s lr: 9.02e-04  
[09/15 22:10:36] lb.utils.events INFO:  eta: 1 day, 0:23:04  iteration: 76599/375342  consumed_samples: 78438400  total_loss: 3.367  time: 0.4033 s/iter  data_time: 0.2588 s/iter total_throughput: 2538.89 samples/s lr: 9.02e-04  
[09/15 22:11:17] lb.utils.events INFO:  eta: 19:24:29  iteration: 76699/375342  consumed_samples: 78540800  total_loss: 3.362  time: 0.4033 s/iter  data_time: 0.2579 s/iter total_throughput: 2538.86 samples/s lr: 9.01e-04  
[09/15 22:11:57] lb.utils.events INFO:  eta: 13:23:44  iteration: 76799/375342  consumed_samples: 78643200  total_loss: 3.359  time: 0.4033 s/iter  data_time: 0.2488 s/iter total_throughput: 2538.87 samples/s lr: 9.01e-04  
[09/15 22:12:38] lb.utils.events INFO:  eta: 12:59:35  iteration: 76899/375342  consumed_samples: 78745600  total_loss: 3.377  time: 0.4033 s/iter  data_time: 0.2553 s/iter total_throughput: 2538.85 samples/s lr: 9.01e-04  
[09/15 22:13:18] lb.utils.events INFO:  eta: 12:56:28  iteration: 76999/375342  consumed_samples: 78848000  total_loss: 3.378  time: 0.4033 s/iter  data_time: 0.2494 s/iter total_throughput: 2538.84 samples/s lr: 9.01e-04  
[09/15 22:13:59] lb.utils.events INFO:  eta: 12:51:48  iteration: 77099/375342  consumed_samples: 78950400  total_loss: 3.37  time: 0.4033 s/iter  data_time: 0.2646 s/iter total_throughput: 2538.79 samples/s lr: 9.00e-04  
[09/15 22:14:41] lb.utils.events INFO:  eta: 12:49:31  iteration: 77199/375342  consumed_samples: 79052800  total_loss: 3.37  time: 0.4034 s/iter  data_time: 0.2737 s/iter total_throughput: 2538.71 samples/s lr: 9.00e-04  
[09/15 22:15:21] lb.utils.events INFO:  eta: 12:48:40  iteration: 77299/375342  consumed_samples: 79155200  total_loss: 3.37  time: 0.4034 s/iter  data_time: 0.2622 s/iter total_throughput: 2538.69 samples/s lr: 9.00e-04  
[09/15 22:16:02] lb.utils.events INFO:  eta: 12:48:12  iteration: 77399/375342  consumed_samples: 79257600  total_loss: 3.368  time: 0.4034 s/iter  data_time: 0.2579 s/iter total_throughput: 2538.65 samples/s lr: 9.00e-04  
[09/15 22:16:43] lb.utils.events INFO:  eta: 12:46:45  iteration: 77499/375342  consumed_samples: 79360000  total_loss: 3.376  time: 0.4034 s/iter  data_time: 0.2567 s/iter total_throughput: 2538.62 samples/s lr: 8.99e-04  
[09/15 22:17:23] lb.utils.events INFO:  eta: 12:46:35  iteration: 77599/375342  consumed_samples: 79462400  total_loss: 3.383  time: 0.4034 s/iter  data_time: 0.2525 s/iter total_throughput: 2538.62 samples/s lr: 8.99e-04  
[09/15 22:18:03] lb.utils.events INFO:  eta: 12:47:30  iteration: 77699/375342  consumed_samples: 79564800  total_loss: 3.364  time: 0.4034 s/iter  data_time: 0.2514 s/iter total_throughput: 2538.62 samples/s lr: 8.99e-04  
[09/15 22:18:44] lb.utils.events INFO:  eta: 12:47:36  iteration: 77799/375342  consumed_samples: 79667200  total_loss: 3.349  time: 0.4034 s/iter  data_time: 0.2578 s/iter total_throughput: 2538.61 samples/s lr: 8.99e-04  
[09/15 22:19:25] lb.utils.events INFO:  eta: 12:47:40  iteration: 77899/375342  consumed_samples: 79769600  total_loss: 3.363  time: 0.4034 s/iter  data_time: 0.2650 s/iter total_throughput: 2538.56 samples/s lr: 8.98e-04  
[09/15 22:20:05] lb.utils.events INFO:  eta: 12:47:18  iteration: 77999/375342  consumed_samples: 79872000  total_loss: 3.363  time: 0.4034 s/iter  data_time: 0.2560 s/iter total_throughput: 2538.54 samples/s lr: 8.98e-04  
[09/15 22:20:46] lb.utils.events INFO:  eta: 12:46:29  iteration: 78099/375342  consumed_samples: 79974400  total_loss: 3.372  time: 0.4034 s/iter  data_time: 0.2681 s/iter total_throughput: 2538.50 samples/s lr: 8.98e-04  
[09/15 22:21:28] lb.utils.events INFO:  eta: 12:45:06  iteration: 78199/375342  consumed_samples: 80076800  total_loss: 3.392  time: 0.4034 s/iter  data_time: 0.2729 s/iter total_throughput: 2538.43 samples/s lr: 8.98e-04  
[09/15 22:22:08] lb.utils.events INFO:  eta: 12:44:26  iteration: 78299/375342  consumed_samples: 80179200  total_loss: 3.369  time: 0.4034 s/iter  data_time: 0.2693 s/iter total_throughput: 2538.43 samples/s lr: 8.97e-04  
[09/15 22:22:49] lb.utils.events INFO:  eta: 12:43:40  iteration: 78399/375342  consumed_samples: 80281600  total_loss: 3.35  time: 0.4034 s/iter  data_time: 0.2566 s/iter total_throughput: 2538.38 samples/s lr: 8.97e-04  
[09/15 22:23:29] lb.utils.events INFO:  eta: 12:43:19  iteration: 78499/375342  consumed_samples: 80384000  total_loss: 3.368  time: 0.4034 s/iter  data_time: 0.2579 s/iter total_throughput: 2538.38 samples/s lr: 8.97e-04  
[09/15 22:24:10] lb.utils.events INFO:  eta: 12:42:29  iteration: 78599/375342  consumed_samples: 80486400  total_loss: 3.376  time: 0.4034 s/iter  data_time: 0.2712 s/iter total_throughput: 2538.36 samples/s lr: 8.97e-04  
[09/15 22:24:50] lb.utils.events INFO:  eta: 12:41:49  iteration: 78699/375342  consumed_samples: 80588800  total_loss: 3.392  time: 0.4034 s/iter  data_time: 0.2492 s/iter total_throughput: 2538.35 samples/s lr: 8.96e-04  
[09/15 22:25:31] lb.utils.events INFO:  eta: 12:41:59  iteration: 78799/375342  consumed_samples: 80691200  total_loss: 3.402  time: 0.4034 s/iter  data_time: 0.2530 s/iter total_throughput: 2538.36 samples/s lr: 8.96e-04  
[09/15 22:26:10] lb.utils.events INFO:  eta: 12:41:44  iteration: 78899/375342  consumed_samples: 80793600  total_loss: 3.389  time: 0.4034 s/iter  data_time: 0.2535 s/iter total_throughput: 2538.41 samples/s lr: 8.96e-04  
[09/15 22:26:50] lb.utils.events INFO:  eta: 12:42:20  iteration: 78999/375342  consumed_samples: 80896000  total_loss: 3.367  time: 0.4034 s/iter  data_time: 0.2614 s/iter total_throughput: 2538.42 samples/s lr: 8.96e-04  
[09/15 22:27:31] lb.utils.events INFO:  eta: 12:43:54  iteration: 79099/375342  consumed_samples: 80998400  total_loss: 3.366  time: 0.4034 s/iter  data_time: 0.2598 s/iter total_throughput: 2538.40 samples/s lr: 8.95e-04  
[09/15 22:28:11] lb.utils.events INFO:  eta: 12:44:54  iteration: 79199/375342  consumed_samples: 81100800  total_loss: 3.364  time: 0.4034 s/iter  data_time: 0.2524 s/iter total_throughput: 2538.41 samples/s lr: 8.95e-04  
[09/15 22:28:51] lb.utils.events INFO:  eta: 12:46:27  iteration: 79299/375342  consumed_samples: 81203200  total_loss: 3.35  time: 0.4034 s/iter  data_time: 0.2543 s/iter total_throughput: 2538.43 samples/s lr: 8.95e-04  
[09/15 22:29:32] lb.utils.events INFO:  eta: 12:47:48  iteration: 79399/375342  consumed_samples: 81305600  total_loss: 3.321  time: 0.4034 s/iter  data_time: 0.2547 s/iter total_throughput: 2538.43 samples/s lr: 8.95e-04  
[09/15 22:30:12] lb.utils.events INFO:  eta: 12:47:05  iteration: 79499/375342  consumed_samples: 81408000  total_loss: 3.321  time: 0.4034 s/iter  data_time: 0.2627 s/iter total_throughput: 2538.41 samples/s lr: 8.94e-04  
[09/15 22:30:53] lb.utils.events INFO:  eta: 12:49:14  iteration: 79599/375342  consumed_samples: 81510400  total_loss: 3.36  time: 0.4034 s/iter  data_time: 0.2655 s/iter total_throughput: 2538.37 samples/s lr: 8.94e-04  
[09/15 22:31:33] lb.utils.events INFO:  eta: 12:51:30  iteration: 79699/375342  consumed_samples: 81612800  total_loss: 3.369  time: 0.4034 s/iter  data_time: 0.2584 s/iter total_throughput: 2538.40 samples/s lr: 8.94e-04  
[09/15 22:32:13] lb.utils.events INFO:  eta: 12:55:50  iteration: 79799/375342  consumed_samples: 81715200  total_loss: 3.368  time: 0.4034 s/iter  data_time: 0.2464 s/iter total_throughput: 2538.41 samples/s lr: 8.94e-04  
[09/15 22:32:54] lb.utils.events INFO:  eta: 14:52:36  iteration: 79899/375342  consumed_samples: 81817600  total_loss: 3.359  time: 0.4034 s/iter  data_time: 0.2609 s/iter total_throughput: 2538.39 samples/s lr: 8.93e-04  
[09/15 22:33:35] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_2e56/model_0079999
[09/15 22:33:35] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000
[09/15 22:33:35] lb.evaluation.evaluator INFO: Start inference on 50000 samples
[09/15 22:33:40] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.1212 s/iter. Inference: 0.1502 s/iter. Eval: 0.0023 s/iter. Total: 0.2738 s/iter. ETA=0:00:10
[09/15 22:33:45] lb.evaluation.evaluator INFO: Inference done 26624/50000. Dataloading: 0.1650 s/iter. Inference: 0.1498 s/iter. Eval: 0.0023 s/iter. Total: 0.3172 s/iter. ETA=0:00:06
[09/15 22:33:50] lb.evaluation.evaluator INFO: Inference done 43008/50000. Dataloading: 0.1652 s/iter. Inference: 0.1499 s/iter. Eval: 0.0023 s/iter. Total: 0.3175 s/iter. ETA=0:00:01
[09/15 22:33:52] lb.evaluation.evaluator INFO: Total valid samples: 50000
[09/15 22:33:52] lb.evaluation.evaluator INFO: Total inference time: 0:00:13.913379 (0.000278 s / iter per device, on 8 devices)
[09/15 22:33:52] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000132 s / iter per device, on 8 devices)
[09/15 22:33:52] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format:
[09/15 22:33:52] lb.evaluation.utils INFO: copypaste: Acc@1=64.23400000000001
[09/15 22:33:52] lb.evaluation.utils INFO: copypaste: Acc@5=85.8
[09/15 22:33:52] lb.engine.hooks INFO: Not saving as latest eval score for Acc@1 is 64.23400, not better than best score 64.66800 @ iteration 74999.
[09/15 22:33:52] lb.utils.events INFO:  eta: 17:03:15  iteration: 79999/375342  consumed_samples: 81920000  total_loss: 3.364  time: 0.4034 s/iter  data_time: 0.2661 s/iter total_throughput: 2538.37 samples/s lr: 8.93e-04  
[09/15 22:34:31] lb.utils.events INFO:  eta: 15:45:09  iteration: 80099/375342  consumed_samples: 82022400  total_loss: 3.365  time: 0.4034 s/iter  data_time: 0.2551 s/iter total_throughput: 2538.51 samples/s lr: 8.93e-04  
[09/15 22:35:11] lb.utils.events INFO:  eta: 13:44:05  iteration: 80199/375342  consumed_samples: 82124800  total_loss: 3.34  time: 0.4034 s/iter  data_time: 0.2653 s/iter total_throughput: 2538.49 samples/s lr: 8.93e-04  
[09/15 22:35:51] lb.utils.events INFO:  eta: 13:04:12  iteration: 80299/375342  consumed_samples: 82227200  total_loss: 3.328  time: 0.4034 s/iter  data_time: 0.2535 s/iter total_throughput: 2538.52 samples/s lr: 8.92e-04  
[09/15 22:36:32] lb.utils.events INFO:  eta: 12:58:19  iteration: 80399/375342  consumed_samples: 82329600  total_loss: 3.34  time: 0.4034 s/iter  data_time: 0.2715 s/iter total_throughput: 2538.50 samples/s lr: 8.92e-04  
[09/15 22:37:12] lb.utils.events INFO:  eta: 13:19:29  iteration: 80499/375342  consumed_samples: 82432000  total_loss: 3.354  time: 0.4034 s/iter  data_time: 0.2535 s/iter total_throughput: 2538.49 samples/s lr: 8.92e-04  
[09/15 22:37:53] lb.utils.events INFO:  eta: 13:20:04  iteration: 80599/375342  consumed_samples: 82534400  total_loss: 3.365  time: 0.4034 s/iter  data_time: 0.2625 s/iter total_throughput: 2538.45 samples/s lr: 8.92e-04  
[09/15 22:38:34] lb.utils.events INFO:  eta: 12:57:32  iteration: 80699/375342  consumed_samples: 82636800  total_loss: 3.357  time: 0.4034 s/iter  data_time: 0.2545 s/iter total_throughput: 2538.41 samples/s lr: 8.91e-04  
[09/15 22:39:14] lb.utils.events INFO:  eta: 12:52:43  iteration: 80799/375342  consumed_samples: 82739200  total_loss: 3.351  time: 0.4034 s/iter  data_time: 0.2520 s/iter total_throughput: 2538.44 samples/s lr: 8.91e-04  
[09/15 22:39:54] lb.utils.events INFO:  eta: 12:46:39  iteration: 80899/375342  consumed_samples: 82841600  total_loss: 3.348  time: 0.4034 s/iter  data_time: 0.2495 s/iter total_throughput: 2538.44 samples/s lr: 8.91e-04  
[09/15 22:40:35] lb.utils.events INFO:  eta: 12:44:19  iteration: 80999/375342  consumed_samples: 82944000  total_loss: 3.335  time: 0.4034 s/iter  data_time: 0.2611 s/iter total_throughput: 2538.41 samples/s lr: 8.91e-04  
[09/15 22:41:15] lb.utils.events INFO:  eta: 12:46:39  iteration: 81099/375342  consumed_samples: 83046400  total_loss: 3.344  time: 0.4034 s/iter  data_time: 0.2451 s/iter total_throughput: 2538.44 samples/s lr: 8.90e-04  
[09/15 22:41:56] lb.utils.events INFO:  eta: 12:53:27  iteration: 81199/375342  consumed_samples: 83148800  total_loss: 3.345  time: 0.4034 s/iter  data_time: 0.2605 s/iter total_throughput: 2538.43 samples/s lr: 8.90e-04  
[09/15 22:42:36] lb.utils.events INFO:  eta: 14:22:12  iteration: 81299/375342  consumed_samples: 83251200  total_loss: 3.365  time: 0.4034 s/iter  data_time: 0.2450 s/iter total_throughput: 2538.41 samples/s lr: 8.90e-04  
[09/15 22:43:17] lb.utils.events INFO:  eta: 22:15:16  iteration: 81399/375342  consumed_samples: 83353600  total_loss: 3.357  time: 0.4034 s/iter  data_time: 0.2608 s/iter total_throughput: 2538.41 samples/s lr: 8.89e-04  
[09/15 22:43:57] lb.utils.events INFO:  eta: 20:47:22  iteration: 81499/375342  consumed_samples: 83456000  total_loss: 3.327  time: 0.4034 s/iter  data_time: 0.2554 s/iter total_throughput: 2538.39 samples/s lr: 8.89e-04  
[09/15 22:44:37] lb.utils.events INFO:  eta: 20:58:57  iteration: 81599/375342  consumed_samples: 83558400  total_loss: 3.33  time: 0.4034 s/iter  data_time: 0.2563 s/iter total_throughput: 2538.40 samples/s lr: 8.89e-04  
[09/15 22:45:17] lb.utils.events INFO:  eta: 23:00:06  iteration: 81699/375342  consumed_samples: 83660800  total_loss: 3.372  time: 0.4034 s/iter  data_time: 0.2493 s/iter total_throughput: 2538.42 samples/s lr: 8.89e-04  
[09/15 22:45:58] lb.utils.events INFO:  eta: 1 day, 1:23:31  iteration: 81799/375342  consumed_samples: 83763200  total_loss: 3.358  time: 0.4034 s/iter  data_time: 0.2523 s/iter total_throughput: 2538.44 samples/s lr: 8.88e-04  
[09/15 22:46:38] lb.utils.events INFO:  eta: 23:21:56  iteration: 81899/375342  consumed_samples: 83865600  total_loss: 3.329  time: 0.4034 s/iter  data_time: 0.2582 s/iter total_throughput: 2538.44 samples/s lr: 8.88e-04  
[09/15 22:47:18] lb.utils.events INFO:  eta: 18:55:14  iteration: 81999/375342  consumed_samples: 83968000  total_loss: 3.332  time: 0.4034 s/iter  data_time: 0.2585 s/iter total_throughput: 2538.44 samples/s lr: 8.88e-04  
[09/15 22:47:59] lb.utils.events INFO:  eta: 15:05:32  iteration: 82099/375342  consumed_samples: 84070400  total_loss: 3.334  time: 0.4034 s/iter  data_time: 0.2533 s/iter total_throughput: 2538.42 samples/s lr: 8.88e-04  
[09/15 22:48:39] lb.utils.events INFO:  eta: 12:54:24  iteration: 82199/375342  consumed_samples: 84172800  total_loss: 3.365  time: 0.4034 s/iter  data_time: 0.2501 s/iter total_throughput: 2538.42 samples/s lr: 8.87e-04  
[09/15 22:49:19] lb.utils.events INFO:  eta: 12:44:49  iteration: 82299/375342  consumed_samples: 84275200  total_loss: 3.357  time: 0.4034 s/iter  data_time: 0.2566 s/iter total_throughput: 2538.44 samples/s lr: 8.87e-04  
[09/15 22:49:59] lb.utils.events INFO:  eta: 12:40:02  iteration: 82399/375342  consumed_samples: 84377600  total_loss: 3.345  time: 0.4034 s/iter  data_time: 0.2462 s/iter total_throughput: 2538.50 samples/s lr: 8.87e-04  
[09/15 22:50:39] lb.utils.events INFO:  eta: 12:42:01  iteration: 82499/375342  consumed_samples: 84480000  total_loss: 3.341  time: 0.4034 s/iter  data_time: 0.2490 s/iter total_throughput: 2538.51 samples/s lr: 8.87e-04  
[09/15 22:51:19] lb.utils.events INFO:  eta: 12:42:23  iteration: 82599/375342  consumed_samples: 84582400  total_loss: 3.334  time: 0.4034 s/iter  data_time: 0.2521 s/iter total_throughput: 2538.56 samples/s lr: 8.86e-04  
[09/15 22:51:59] lb.utils.events INFO:  eta: 12:45:05  iteration: 82699/375342  consumed_samples: 84684800  total_loss: 3.333  time: 0.4034 s/iter  data_time: 0.2507 s/iter total_throughput: 2538.56 samples/s lr: 8.86e-04  
[09/15 22:52:39] lb.utils.events INFO:  eta: 12:45:45  iteration: 82799/375342  consumed_samples: 84787200  total_loss: 3.333  time: 0.4034 s/iter  data_time: 0.2548 s/iter total_throughput: 2538.57 samples/s lr: 8.86e-04  
[09/15 22:53:19] lb.utils.events INFO:  eta: 12:57:26  iteration: 82899/375342  consumed_samples: 84889600  total_loss: 3.361  time: 0.4034 s/iter  data_time: 0.2445 s/iter total_throughput: 2538.61 samples/s lr: 8.86e-04  
[09/15 22:54:00] lb.utils.events INFO:  eta: 13:26:45  iteration: 82999/375342  consumed_samples: 84992000  total_loss: 3.368  time: 0.4034 s/iter  data_time: 0.2687 s/iter total_throughput: 2538.60 samples/s lr: 8.85e-04  
[09/15 22:54:40] lb.utils.events INFO:  eta: 13:12:40  iteration: 83099/375342  consumed_samples: 85094400  total_loss: 3.323  time: 0.4034 s/iter  data_time: 0.2506 s/iter total_throughput: 2538.63 samples/s lr: 8.85e-04  
[09/15 22:55:19] lb.utils.events INFO:  eta: 15:30:16  iteration: 83199/375342  consumed_samples: 85196800  total_loss: 3.316  time: 0.4034 s/iter  data_time: 0.2502 s/iter total_throughput: 2538.69 samples/s lr: 8.85e-04  
[09/15 22:55:59] lb.utils.events INFO:  eta: 17:17:50  iteration: 83299/375342  consumed_samples: 85299200  total_loss: 3.332  time: 0.4034 s/iter  data_time: 0.2565 s/iter total_throughput: 2538.71 samples/s lr: 8.84e-04  
[09/15 22:56:39] lb.utils.events INFO:  eta: 17:24:54  iteration: 83399/375342  consumed_samples: 85401600  total_loss: 3.335  time: 0.4034 s/iter  data_time: 0.2535 s/iter total_throughput: 2538.73 samples/s lr: 8.84e-04  
[09/15 22:57:19] lb.utils.events INFO:  eta: 13:37:50  iteration: 83499/375342  consumed_samples: 85504000  total_loss: 3.341  time: 0.4033 s/iter  data_time: 0.2455 s/iter total_throughput: 2538.77 samples/s lr: 8.84e-04  
[09/15 22:57:59] lb.utils.events INFO:  eta: 13:58:15  iteration: 83599/375342  consumed_samples: 85606400  total_loss: 3.365  time: 0.4033 s/iter  data_time: 0.2518 s/iter total_throughput: 2538.80 samples/s lr: 8.84e-04  
[09/15 22:58:39] lb.utils.events INFO:  eta: 13:49:35  iteration: 83699/375342  consumed_samples: 85708800  total_loss: 3.35  time: 0.4033 s/iter  data_time: 0.2475 s/iter total_throughput: 2538.81 samples/s lr: 8.83e-04  
[09/15 22:59:19] lb.utils.events INFO:  eta: 13:38:21  iteration: 83799/375342  consumed_samples: 85811200  total_loss: 3.354  time: 0.4033 s/iter  data_time: 0.2517 s/iter total_throughput: 2538.86 samples/s lr: 8.83e-04  
[09/15 22:59:59] lb.utils.events INFO:  eta: 12:53:45  iteration: 83899/375342  consumed_samples: 85913600  total_loss: 3.357  time: 0.4033 s/iter  data_time: 0.2533 s/iter total_throughput: 2538.87 samples/s lr: 8.83e-04  
[09/15 23:00:39] lb.utils.events INFO:  eta: 12:49:24  iteration: 83999/375342  consumed_samples: 86016000  total_loss: 3.354  time: 0.4033 s/iter  data_time: 0.2470 s/iter total_throughput: 2538.89 samples/s lr: 8.83e-04  
[09/15 23:01:20] lb.utils.events INFO:  eta: 13:02:05  iteration: 84099/375342  consumed_samples: 86118400  total_loss: 3.339  time: 0.4033 s/iter  data_time: 0.2576 s/iter total_throughput: 2538.88 samples/s lr: 8.82e-04  
[09/15 23:02:01] lb.utils.events INFO:  eta: 13:01:48  iteration: 84199/375342  consumed_samples: 86220800  total_loss: 3.33  time: 0.4033 s/iter  data_time: 0.2673 s/iter total_throughput: 2538.83 samples/s lr: 8.82e-04  
[09/15 23:02:42] lb.utils.events INFO:  eta: 12:41:08  iteration: 84299/375342  consumed_samples: 86323200  total_loss: 3.34  time: 0.4033 s/iter  data_time: 0.2543 s/iter total_throughput: 2538.78 samples/s lr: 8.82e-04  
[09/15 23:03:23] lb.utils.events INFO:  eta: 12:40:21  iteration: 84399/375342  consumed_samples: 86425600  total_loss: 3.327  time: 0.4033 s/iter  data_time: 0.2698 s/iter total_throughput: 2538.75 samples/s lr: 8.82e-04  
[09/15 23:04:05] lb.utils.events INFO:  eta: 12:38:52  iteration: 84499/375342  consumed_samples: 86528000  total_loss: 3.343  time: 0.4034 s/iter  data_time: 0.2753 s/iter total_throughput: 2538.57 samples/s lr: 8.81e-04  
[09/15 23:04:48] lb.utils.events INFO:  eta: 12:33:55  iteration: 84599/375342  consumed_samples: 86630400  total_loss: 3.354  time: 0.4034 s/iter  data_time: 0.2815 s/iter total_throughput: 2538.42 samples/s lr: 8.81e-04  
[09/15 23:05:29] lb.utils.events INFO:  eta: 12:32:02  iteration: 84699/375342  consumed_samples: 86732800  total_loss: 3.347  time: 0.4034 s/iter  data_time: 0.2638 s/iter total_throughput: 2538.31 samples/s lr: 8.81e-04  
[09/15 23:06:11] lb.utils.events INFO:  eta: 12:31:54  iteration: 84799/375342  consumed_samples: 86835200  total_loss: 3.315  time: 0.4034 s/iter  data_time: 0.2701 s/iter total_throughput: 2538.19 samples/s lr: 8.80e-04  
[09/15 23:06:52] lb.utils.events INFO:  eta: 12:33:08  iteration: 84899/375342  consumed_samples: 86937600  total_loss: 3.327  time: 0.4034 s/iter  data_time: 0.2587 s/iter total_throughput: 2538.15 samples/s lr: 8.80e-04  
[09/15 23:07:33] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_2e56/model_0084999
[09/15 23:07:33] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000
[09/15 23:07:33] lb.evaluation.evaluator INFO: Start inference on 50000 samples
[09/15 23:07:38] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.1178 s/iter. Inference: 0.1556 s/iter. Eval: 0.0023 s/iter. Total: 0.2757 s/iter. ETA=0:00:10
[09/15 23:07:43] lb.evaluation.evaluator INFO: Inference done 27648/50000. Dataloading: 0.1520 s/iter. Inference: 0.1532 s/iter. Eval: 0.0023 s/iter. Total: 0.3075 s/iter. ETA=0:00:06
[09/15 23:07:48] lb.evaluation.evaluator INFO: Inference done 43008/50000. Dataloading: 0.1655 s/iter. Inference: 0.1529 s/iter. Eval: 0.0023 s/iter. Total: 0.3208 s/iter. ETA=0:00:01
[09/15 23:07:51] lb.evaluation.evaluator INFO: Total valid samples: 50000
[09/15 23:07:51] lb.evaluation.evaluator INFO: Total inference time: 0:00:13.988761 (0.000280 s / iter per device, on 8 devices)
[09/15 23:07:51] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000134 s / iter per device, on 8 devices)
[09/15 23:07:51] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format:
[09/15 23:07:51] lb.evaluation.utils INFO: copypaste: Acc@1=64.818
[09/15 23:07:51] lb.evaluation.utils INFO: copypaste: Acc@5=86.478
[09/15 23:07:51] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 64.81800, better than last best score 64.66800 @ iteration 74999.
[09/15 23:07:51] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_2e56/model_best
[09/15 23:07:51] lb.utils.events INFO:  eta: 12:33:33  iteration: 84999/375342  consumed_samples: 87040000  total_loss: 3.349  time: 0.4034 s/iter  data_time: 0.2500 s/iter total_throughput: 2538.14 samples/s lr: 8.80e-04  
[09/15 23:08:30] lb.utils.events INFO:  eta: 12:35:07  iteration: 85099/375342  consumed_samples: 87142400  total_loss: 3.341  time: 0.4034 s/iter  data_time: 0.2682 s/iter total_throughput: 2538.29 samples/s lr: 8.80e-04  
[09/15 23:09:11] lb.utils.events INFO:  eta: 12:33:22  iteration: 85199/375342  consumed_samples: 87244800  total_loss: 3.339  time: 0.4034 s/iter  data_time: 0.2507 s/iter total_throughput: 2538.25 samples/s lr: 8.79e-04  
[09/15 23:09:51] lb.utils.events INFO:  eta: 12:34:37  iteration: 85299/375342  consumed_samples: 87347200  total_loss: 3.326  time: 0.4034 s/iter  data_time: 0.2679 s/iter total_throughput: 2538.23 samples/s lr: 8.79e-04  
[09/15 23:10:33] lb.utils.events INFO:  eta: 12:31:38  iteration: 85399/375342  consumed_samples: 87449600  total_loss: 3.319  time: 0.4034 s/iter  data_time: 0.2697 s/iter total_throughput: 2538.14 samples/s lr: 8.79e-04  
[09/15 23:11:14] lb.utils.events INFO:  eta: 12:34:14  iteration: 85499/375342  consumed_samples: 87552000  total_loss: 3.304  time: 0.4035 s/iter  data_time: 0.2635 s/iter total_throughput: 2538.05 samples/s lr: 8.79e-04  
[09/15 23:11:56] lb.utils.events INFO:  eta: 12:37:03  iteration: 85599/375342  consumed_samples: 87654400  total_loss: 3.314  time: 0.4035 s/iter  data_time: 0.2671 s/iter total_throughput: 2537.93 samples/s lr: 8.78e-04  
[09/15 23:12:38] lb.utils.events INFO:  eta: 12:42:09  iteration: 85699/375342  consumed_samples: 87756800  total_loss: 3.321  time: 0.4035 s/iter  data_time: 0.2647 s/iter total_throughput: 2537.87 samples/s lr: 8.78e-04  
[09/15 23:13:19] lb.utils.events INFO:  eta: 12:44:20  iteration: 85799/375342  consumed_samples: 87859200  total_loss: 3.321  time: 0.4035 s/iter  data_time: 0.2689 s/iter total_throughput: 2537.82 samples/s lr: 8.78e-04  
[09/15 23:13:59] lb.utils.events INFO:  eta: 13:00:34  iteration: 85899/375342  consumed_samples: 87961600  total_loss: 3.312  time: 0.4035 s/iter  data_time: 0.2567 s/iter total_throughput: 2537.81 samples/s lr: 8.77e-04  
[09/15 23:14:39] lb.utils.events INFO:  eta: 13:31:31  iteration: 85999/375342  consumed_samples: 88064000  total_loss: 3.341  time: 0.4035 s/iter  data_time: 0.2521 s/iter total_throughput: 2537.81 samples/s lr: 8.77e-04  
[09/15 23:15:21] lb.utils.events INFO:  eta: 14:20:42  iteration: 86099/375342  consumed_samples: 88166400  total_loss: 3.344  time: 0.4035 s/iter  data_time: 0.2612 s/iter total_throughput: 2537.73 samples/s lr: 8.77e-04  
[09/15 23:16:02] lb.utils.events INFO:  eta: 16:27:01  iteration: 86199/375342  consumed_samples: 88268800  total_loss: 3.337  time: 0.4035 s/iter  data_time: 0.2607 s/iter total_throughput: 2537.70 samples/s lr: 8.77e-04  
[09/15 23:16:43] lb.utils.events INFO:  eta: 13:44:23  iteration: 86299/375342  consumed_samples: 88371200  total_loss: 3.341  time: 0.4035 s/iter  data_time: 0.2574 s/iter total_throughput: 2537.66 samples/s lr: 8.76e-04  
[09/15 23:17:23] lb.utils.events INFO:  eta: 13:44:05  iteration: 86399/375342  consumed_samples: 88473600  total_loss: 3.344  time: 0.4035 s/iter  data_time: 0.2615 s/iter total_throughput: 2537.64 samples/s lr: 8.76e-04  
[09/15 23:18:04] lb.utils.events INFO:  eta: 12:54:45  iteration: 86499/375342  consumed_samples: 88576000  total_loss: 3.328  time: 0.4035 s/iter  data_time: 0.2571 s/iter total_throughput: 2537.61 samples/s lr: 8.76e-04  
[09/15 23:18:44] lb.utils.events INFO:  eta: 12:52:36  iteration: 86599/375342  consumed_samples: 88678400  total_loss: 3.302  time: 0.4035 s/iter  data_time: 0.2534 s/iter total_throughput: 2537.61 samples/s lr: 8.76e-04  
[09/15 23:19:26] lb.utils.events INFO:  eta: 12:54:12  iteration: 86699/375342  consumed_samples: 88780800  total_loss: 3.315  time: 0.4035 s/iter  data_time: 0.2665 s/iter total_throughput: 2537.55 samples/s lr: 8.75e-04  
[09/15 23:20:07] lb.utils.events INFO:  eta: 12:34:55  iteration: 86799/375342  consumed_samples: 88883200  total_loss: 3.329  time: 0.4035 s/iter  data_time: 0.2647 s/iter total_throughput: 2537.50 samples/s lr: 8.75e-04  
[09/15 23:20:48] lb.utils.events INFO:  eta: 12:30:48  iteration: 86899/375342  consumed_samples: 88985600  total_loss: 3.31  time: 0.4036 s/iter  data_time: 0.2582 s/iter total_throughput: 2537.45 samples/s lr: 8.75e-04  
[09/15 23:21:29] lb.utils.events INFO:  eta: 12:26:14  iteration: 86999/375342  consumed_samples: 89088000  total_loss: 3.337  time: 0.4036 s/iter  data_time: 0.2695 s/iter total_throughput: 2537.40 samples/s lr: 8.74e-04  
[09/15 23:22:09] lb.utils.events INFO:  eta: 12:23:09  iteration: 87099/375342  consumed_samples: 89190400  total_loss: 3.351  time: 0.4036 s/iter  data_time: 0.2608 s/iter total_throughput: 2537.38 samples/s lr: 8.74e-04  
[09/15 23:22:50] lb.utils.events INFO:  eta: 12:21:41  iteration: 87199/375342  consumed_samples: 89292800  total_loss: 3.326  time: 0.4036 s/iter  data_time: 0.2635 s/iter total_throughput: 2537.34 samples/s lr: 8.74e-04  
[09/15 23:23:31] lb.utils.events INFO:  eta: 12:21:17  iteration: 87299/375342  consumed_samples: 89395200  total_loss: 3.326  time: 0.4036 s/iter  data_time: 0.2554 s/iter total_throughput: 2537.32 samples/s lr: 8.74e-04  
[09/15 23:24:12] lb.utils.events INFO:  eta: 12:21:15  iteration: 87399/375342  consumed_samples: 89497600  total_loss: 3.331  time: 0.4036 s/iter  data_time: 0.2564 s/iter total_throughput: 2537.29 samples/s lr: 8.73e-04  
[09/15 23:24:52] lb.utils.events INFO:  eta: 12:21:15  iteration: 87499/375342  consumed_samples: 89600000  total_loss: 3.341  time: 0.4036 s/iter  data_time: 0.2545 s/iter total_throughput: 2537.28 samples/s lr: 8.73e-04  
[09/15 23:25:33] lb.utils.events INFO:  eta: 12:20:34  iteration: 87599/375342  consumed_samples: 89702400  total_loss: 3.345  time: 0.4036 s/iter  data_time: 0.2544 s/iter total_throughput: 2537.24 samples/s lr: 8.73e-04  
[09/15 23:26:14] lb.utils.events INFO:  eta: 12:20:02  iteration: 87699/375342  consumed_samples: 89804800  total_loss: 3.333  time: 0.4036 s/iter  data_time: 0.2645 s/iter total_throughput: 2537.21 samples/s lr: 8.73e-04  
[09/15 23:26:55] lb.utils.events INFO:  eta: 12:20:37  iteration: 87799/375342  consumed_samples: 89907200  total_loss: 3.322  time: 0.4036 s/iter  data_time: 0.2526 s/iter total_throughput: 2537.18 samples/s lr: 8.72e-04  
[09/15 23:27:35] lb.utils.events INFO:  eta: 12:21:48  iteration: 87899/375342  consumed_samples: 90009600  total_loss: 3.32  time: 0.4036 s/iter  data_time: 0.2481 s/iter total_throughput: 2537.21 samples/s lr: 8.72e-04  
[09/15 23:28:15] lb.utils.events INFO:  eta: 12:23:15  iteration: 87999/375342  consumed_samples: 90112000  total_loss: 3.312  time: 0.4036 s/iter  data_time: 0.2596 s/iter total_throughput: 2537.18 samples/s lr: 8.72e-04  
[09/15 23:28:56] lb.utils.events INFO:  eta: 12:23:48  iteration: 88099/375342  consumed_samples: 90214400  total_loss: 3.289  time: 0.4036 s/iter  data_time: 0.2536 s/iter total_throughput: 2537.14 samples/s lr: 8.71e-04  
[09/15 23:29:37] lb.utils.events INFO:  eta: 12:24:39  iteration: 88199/375342  consumed_samples: 90316800  total_loss: 3.311  time: 0.4036 s/iter  data_time: 0.2571 s/iter total_throughput: 2537.13 samples/s lr: 8.71e-04  
[09/15 23:30:17] lb.utils.events INFO:  eta: 12:27:52  iteration: 88299/375342  consumed_samples: 90419200  total_loss: 3.337  time: 0.4036 s/iter  data_time: 0.2541 s/iter total_throughput: 2537.12 samples/s lr: 8.71e-04  
[09/15 23:30:58] lb.utils.events INFO:  eta: 12:31:25  iteration: 88399/375342  consumed_samples: 90521600  total_loss: 3.329  time: 0.4036 s/iter  data_time: 0.2634 s/iter total_throughput: 2537.11 samples/s lr: 8.71e-04  
[09/15 23:31:39] lb.utils.events INFO:  eta: 12:34:32  iteration: 88499/375342  consumed_samples: 90624000  total_loss: 3.325  time: 0.4036 s/iter  data_time: 0.2513 s/iter total_throughput: 2537.09 samples/s lr: 8.70e-04  
[09/15 23:32:19] lb.utils.events INFO:  eta: 12:44:19  iteration: 88599/375342  consumed_samples: 90726400  total_loss: 3.325  time: 0.4036 s/iter  data_time: 0.2597 s/iter total_throughput: 2537.06 samples/s lr: 8.70e-04  
[09/15 23:33:00] lb.utils.events INFO:  eta: 14:40:48  iteration: 88699/375342  consumed_samples: 90828800  total_loss: 3.321  time: 0.4036 s/iter  data_time: 0.2449 s/iter total_throughput: 2537.08 samples/s lr: 8.70e-04  
[09/15 23:33:40] lb.utils.events INFO:  eta: 16:15:28  iteration: 88799/375342  consumed_samples: 90931200  total_loss: 3.328  time: 0.4036 s/iter  data_time: 0.2598 s/iter total_throughput: 2537.04 samples/s lr: 8.69e-04  
[09/15 23:34:21] lb.utils.events INFO:  eta: 16:29:04  iteration: 88899/375342  consumed_samples: 91033600  total_loss: 3.308  time: 0.4036 s/iter  data_time: 0.2637 s/iter total_throughput: 2537.02 samples/s lr: 8.69e-04  
[09/15 23:35:02] lb.utils.events INFO:  eta: 15:45:43  iteration: 88999/375342  consumed_samples: 91136000  total_loss: 3.292  time: 0.4036 s/iter  data_time: 0.2549 s/iter total_throughput: 2536.98 samples/s lr: 8.69e-04  
[09/15 23:35:43] lb.utils.events INFO:  eta: 18:00:16  iteration: 89099/375342  consumed_samples: 91238400  total_loss: 3.308  time: 0.4036 s/iter  data_time: 0.2587 s/iter total_throughput: 2536.94 samples/s lr: 8.69e-04  
[09/15 23:36:24] lb.utils.events INFO:  eta: 18:42:16  iteration: 89199/375342  consumed_samples: 91340800  total_loss: 3.32  time: 0.4036 s/iter  data_time: 0.2559 s/iter total_throughput: 2536.89 samples/s lr: 8.68e-04  
[09/15 23:37:04] lb.utils.events INFO:  eta: 17:12:28  iteration: 89299/375342  consumed_samples: 91443200  total_loss: 3.314  time: 0.4036 s/iter  data_time: 0.2543 s/iter total_throughput: 2536.90 samples/s lr: 8.68e-04  
[09/15 23:37:45] lb.utils.events INFO:  eta: 18:31:14  iteration: 89399/375342  consumed_samples: 91545600  total_loss: 3.299  time: 0.4036 s/iter  data_time: 0.2570 s/iter total_throughput: 2536.89 samples/s lr: 8.68e-04  
[09/15 23:38:26] lb.utils.events INFO:  eta: 14:18:29  iteration: 89499/375342  consumed_samples: 91648000  total_loss: 3.298  time: 0.4037 s/iter  data_time: 0.2688 s/iter total_throughput: 2536.81 samples/s lr: 8.67e-04  
[09/15 23:39:07] lb.utils.events INFO:  eta: 12:44:14  iteration: 89599/375342  consumed_samples: 91750400  total_loss: 3.316  time: 0.4037 s/iter  data_time: 0.2541 s/iter total_throughput: 2536.78 samples/s lr: 8.67e-04  
[09/15 23:39:48] lb.utils.events INFO:  eta: 12:29:08  iteration: 89699/375342  consumed_samples: 91852800  total_loss: 3.322  time: 0.4037 s/iter  data_time: 0.2618 s/iter total_throughput: 2536.72 samples/s lr: 8.67e-04  
[09/15 23:40:29] lb.utils.events INFO:  eta: 12:23:21  iteration: 89799/375342  consumed_samples: 91955200  total_loss: 3.291  time: 0.4037 s/iter  data_time: 0.2494 s/iter total_throughput: 2536.70 samples/s lr: 8.67e-04  
[09/15 23:41:09] lb.utils.events INFO:  eta: 12:21:48  iteration: 89899/375342  consumed_samples: 92057600  total_loss: 3.295  time: 0.4037 s/iter  data_time: 0.2570 s/iter total_throughput: 2536.71 samples/s lr: 8.66e-04  
[09/15 23:41:50] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_2e56/model_0089999
[09/15 23:41:50] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000
[09/15 23:41:50] lb.evaluation.evaluator INFO: Start inference on 50000 samples
[09/15 23:41:55] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.1181 s/iter. Inference: 0.1498 s/iter. Eval: 0.0024 s/iter. Total: 0.2703 s/iter. ETA=0:00:10
[09/15 23:42:00] lb.evaluation.evaluator INFO: Inference done 27648/50000. Dataloading: 0.1513 s/iter. Inference: 0.1525 s/iter. Eval: 0.0022 s/iter. Total: 0.3062 s/iter. ETA=0:00:06
[09/15 23:42:05] lb.evaluation.evaluator INFO: Inference done 45056/50000. Dataloading: 0.1482 s/iter. Inference: 0.1521 s/iter. Eval: 0.0023 s/iter. Total: 0.3027 s/iter. ETA=0:00:01
[09/15 23:42:07] lb.evaluation.evaluator INFO: Total valid samples: 50000
[09/15 23:42:07] lb.evaluation.evaluator INFO: Total inference time: 0:00:13.598714 (0.000272 s / iter per device, on 8 devices)
[09/15 23:42:07] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000134 s / iter per device, on 8 devices)
[09/15 23:42:07] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format:
[09/15 23:42:07] lb.evaluation.utils INFO: copypaste: Acc@1=65.872
[09/15 23:42:07] lb.evaluation.utils INFO: copypaste: Acc@5=86.758
[09/15 23:42:07] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 65.87200, better than last best score 64.81800 @ iteration 84999.
[09/15 23:42:07] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_2e56/model_best
[09/15 23:42:08] lb.utils.events INFO:  eta: 12:20:29  iteration: 89999/375342  consumed_samples: 92160000  total_loss: 3.318  time: 0.4037 s/iter  data_time: 0.2584 s/iter total_throughput: 2536.69 samples/s lr: 8.66e-04  
[09/15 23:42:47] lb.utils.events INFO:  eta: 12:21:10  iteration: 90099/375342  consumed_samples: 92262400  total_loss: 3.334  time: 0.4037 s/iter  data_time: 0.2549 s/iter total_throughput: 2536.79 samples/s lr: 8.66e-04  
[09/15 23:43:27] lb.utils.events INFO:  eta: 12:19:57  iteration: 90199/375342  consumed_samples: 92364800  total_loss: 3.316  time: 0.4037 s/iter  data_time: 0.2620 s/iter total_throughput: 2536.78 samples/s lr: 8.66e-04  
[09/15 23:44:08] lb.utils.events INFO:  eta: 12:21:14  iteration: 90299/375342  consumed_samples: 92467200  total_loss: 3.295  time: 0.4037 s/iter  data_time: 0.2622 s/iter total_throughput: 2536.76 samples/s lr: 8.65e-04  
[09/15 23:44:49] lb.utils.events INFO:  eta: 12:20:24  iteration: 90399/375342  consumed_samples: 92569600  total_loss: 3.283  time: 0.4037 s/iter  data_time: 0.2643 s/iter total_throughput: 2536.74 samples/s lr: 8.65e-04  
[09/15 23:45:30] lb.utils.events INFO:  eta: 12:22:05  iteration: 90499/375342  consumed_samples: 92672000  total_loss: 3.283  time: 0.4037 s/iter  data_time: 0.2576 s/iter total_throughput: 2536.71 samples/s lr: 8.65e-04  
[09/15 23:46:10] lb.utils.events INFO:  eta: 12:24:55  iteration: 90599/375342  consumed_samples: 92774400  total_loss: 3.31  time: 0.4037 s/iter  data_time: 0.2536 s/iter total_throughput: 2536.67 samples/s lr: 8.64e-04  
[09/15 23:46:51] lb.utils.events INFO:  eta: 12:26:26  iteration: 90699/375342  consumed_samples: 92876800  total_loss: 3.307  time: 0.4037 s/iter  data_time: 0.2626 s/iter total_throughput: 2536.66 samples/s lr: 8.64e-04  
[09/15 23:47:32] lb.utils.events INFO:  eta: 12:34:38  iteration: 90799/375342  consumed_samples: 92979200  total_loss: 3.307  time: 0.4037 s/iter  data_time: 0.2536 s/iter total_throughput: 2536.66 samples/s lr: 8.64e-04  
[09/15 23:48:12] lb.utils.events INFO:  eta: 12:48:45  iteration: 90899/375342  consumed_samples: 93081600  total_loss: 3.32  time: 0.4037 s/iter  data_time: 0.2635 s/iter total_throughput: 2536.65 samples/s lr: 8.64e-04  
[09/15 23:48:52] lb.utils.events INFO:  eta: 12:54:04  iteration: 90999/375342  consumed_samples: 93184000  total_loss: 3.31  time: 0.4037 s/iter  data_time: 0.2542 s/iter total_throughput: 2536.65 samples/s lr: 8.63e-04  
[09/15 23:49:33] lb.utils.events INFO:  eta: 12:33:48  iteration: 91099/375342  consumed_samples: 93286400  total_loss: 3.306  time: 0.4037 s/iter  data_time: 0.2579 s/iter total_throughput: 2536.65 samples/s lr: 8.63e-04  
[09/15 23:50:13] lb.utils.events INFO:  eta: 12:33:32  iteration: 91199/375342  consumed_samples: 93388800  total_loss: 3.312  time: 0.4037 s/iter  data_time: 0.2568 s/iter total_throughput: 2536.63 samples/s lr: 8.63e-04  
[09/15 23:50:54] lb.utils.events INFO:  eta: 12:31:27  iteration: 91299/375342  consumed_samples: 93491200  total_loss: 3.319  time: 0.4037 s/iter  data_time: 0.2534 s/iter total_throughput: 2536.61 samples/s lr: 8.62e-04  
[09/15 23:51:35] lb.utils.events INFO:  eta: 12:37:37  iteration: 91399/375342  consumed_samples: 93593600  total_loss: 3.299  time: 0.4037 s/iter  data_time: 0.2591 s/iter total_throughput: 2536.58 samples/s lr: 8.62e-04  
[09/15 23:52:15] lb.utils.events INFO:  eta: 13:16:30  iteration: 91499/375342  consumed_samples: 93696000  total_loss: 3.284  time: 0.4037 s/iter  data_time: 0.2562 s/iter total_throughput: 2536.57 samples/s lr: 8.62e-04  
[09/15 23:52:56] lb.utils.events INFO:  eta: 12:48:33  iteration: 91599/375342  consumed_samples: 93798400  total_loss: 3.284  time: 0.4037 s/iter  data_time: 0.2481 s/iter total_throughput: 2536.57 samples/s lr: 8.62e-04  
[09/15 23:53:37] lb.utils.events INFO:  eta: 13:00:53  iteration: 91699/375342  consumed_samples: 93900800  total_loss: 3.306  time: 0.4037 s/iter  data_time: 0.2576 s/iter total_throughput: 2536.54 samples/s lr: 8.61e-04  
[09/15 23:54:17] lb.utils.events INFO:  eta: 12:31:14  iteration: 91799/375342  consumed_samples: 94003200  total_loss: 3.312  time: 0.4037 s/iter  data_time: 0.2622 s/iter total_throughput: 2536.53 samples/s lr: 8.61e-04  
[09/15 23:54:58] lb.utils.events INFO:  eta: 12:24:43  iteration: 91899/375342  consumed_samples: 94105600  total_loss: 3.319  time: 0.4037 s/iter  data_time: 0.2555 s/iter total_throughput: 2536.47 samples/s lr: 8.61e-04  
[09/15 23:55:39] lb.utils.events INFO:  eta: 12:23:53  iteration: 91999/375342  consumed_samples: 94208000  total_loss: 3.311  time: 0.4037 s/iter  data_time: 0.2679 s/iter total_throughput: 2536.44 samples/s lr: 8.60e-04  
[09/15 23:56:20] lb.utils.events INFO:  eta: 12:22:07  iteration: 92099/375342  consumed_samples: 94310400  total_loss: 3.278  time: 0.4037 s/iter  data_time: 0.2583 s/iter total_throughput: 2536.42 samples/s lr: 8.60e-04  
[09/15 23:57:01] lb.utils.events INFO:  eta: 12:18:45  iteration: 92199/375342  consumed_samples: 94412800  total_loss: 3.278  time: 0.4037 s/iter  data_time: 0.2606 s/iter total_throughput: 2536.37 samples/s lr: 8.60e-04  
[09/15 23:57:41] lb.utils.events INFO:  eta: 12:16:35  iteration: 92299/375342  consumed_samples: 94515200  total_loss: 3.304  time: 0.4037 s/iter  data_time: 0.2616 s/iter total_throughput: 2536.36 samples/s lr: 8.59e-04  
[09/15 23:58:22] lb.utils.events INFO:  eta: 12:13:43  iteration: 92399/375342  consumed_samples: 94617600  total_loss: 3.327  time: 0.4037 s/iter  data_time: 0.2470 s/iter total_throughput: 2536.39 samples/s lr: 8.59e-04  
[09/15 23:59:02] lb.utils.events INFO:  eta: 12:10:44  iteration: 92499/375342  consumed_samples: 94720000  total_loss: 3.299  time: 0.4037 s/iter  data_time: 0.2554 s/iter total_throughput: 2536.40 samples/s lr: 8.59e-04  
[09/15 23:59:42] lb.utils.events INFO:  eta: 12:09:33  iteration: 92599/375342  consumed_samples: 94822400  total_loss: 3.274  time: 0.4037 s/iter  data_time: 0.2519 s/iter total_throughput: 2536.40 samples/s lr: 8.59e-04  
[09/16 00:00:22] lb.utils.events INFO:  eta: 12:08:51  iteration: 92699/375342  consumed_samples: 94924800  total_loss: 3.303  time: 0.4037 s/iter  data_time: 0.2632 s/iter total_throughput: 2536.40 samples/s lr: 8.58e-04  
[09/16 00:01:04] lb.utils.events INFO:  eta: 12:07:59  iteration: 92799/375342  consumed_samples: 95027200  total_loss: 3.297  time: 0.4037 s/iter  data_time: 0.2524 s/iter total_throughput: 2536.35 samples/s lr: 8.58e-04  
[09/16 00:01:44] lb.utils.events INFO:  eta: 12:07:53  iteration: 92899/375342  consumed_samples: 95129600  total_loss: 3.292  time: 0.4037 s/iter  data_time: 0.2596 s/iter total_throughput: 2536.37 samples/s lr: 8.58e-04  
[09/16 00:02:24] lb.utils.events INFO:  eta: 12:08:36  iteration: 92999/375342  consumed_samples: 95232000  total_loss: 3.294  time: 0.4037 s/iter  data_time: 0.2581 s/iter total_throughput: 2536.36 samples/s lr: 8.57e-04  
[09/16 00:03:05] lb.utils.events INFO:  eta: 12:07:53  iteration: 93099/375342  consumed_samples: 95334400  total_loss: 3.307  time: 0.4037 s/iter  data_time: 0.2559 s/iter total_throughput: 2536.33 samples/s lr: 8.57e-04  
[09/16 00:03:46] lb.utils.events INFO:  eta: 12:08:02  iteration: 93199/375342  consumed_samples: 95436800  total_loss: 3.308  time: 0.4037 s/iter  data_time: 0.2660 s/iter total_throughput: 2536.29 samples/s lr: 8.57e-04  
[09/16 00:04:26] lb.utils.events INFO:  eta: 12:07:50  iteration: 93299/375342  consumed_samples: 95539200  total_loss: 3.302  time: 0.4037 s/iter  data_time: 0.2486 s/iter total_throughput: 2536.30 samples/s lr: 8.57e-04  
[09/16 00:05:07] lb.utils.events INFO:  eta: 12:07:13  iteration: 93399/375342  consumed_samples: 95641600  total_loss: 3.295  time: 0.4037 s/iter  data_time: 0.2639 s/iter total_throughput: 2536.28 samples/s lr: 8.56e-04  
[09/16 00:05:48] lb.utils.events INFO:  eta: 12:06:42  iteration: 93499/375342  consumed_samples: 95744000  total_loss: 3.298  time: 0.4037 s/iter  data_time: 0.2785 s/iter total_throughput: 2536.23 samples/s lr: 8.56e-04  
[09/16 00:06:28] lb.utils.events INFO:  eta: 12:06:33  iteration: 93599/375342  consumed_samples: 95846400  total_loss: 3.293  time: 0.4037 s/iter  data_time: 0.2500 s/iter total_throughput: 2536.23 samples/s lr: 8.56e-04  
[09/16 00:07:08] lb.utils.events INFO:  eta: 12:05:44  iteration: 93699/375342  consumed_samples: 95948800  total_loss: 3.304  time: 0.4037 s/iter  data_time: 0.2464 s/iter total_throughput: 2536.26 samples/s lr: 8.55e-04  
[09/16 00:07:49] lb.utils.events INFO:  eta: 12:05:51  iteration: 93799/375342  consumed_samples: 96051200  total_loss: 3.286  time: 0.4037 s/iter  data_time: 0.2547 s/iter total_throughput: 2536.26 samples/s lr: 8.55e-04  
[09/16 00:08:29] lb.utils.events INFO:  eta: 12:05:59  iteration: 93899/375342  consumed_samples: 96153600  total_loss: 3.275  time: 0.4037 s/iter  data_time: 0.2619 s/iter total_throughput: 2536.26 samples/s lr: 8.55e-04  
[09/16 00:09:09] lb.utils.events INFO:  eta: 12:05:25  iteration: 93999/375342  consumed_samples: 96256000  total_loss: 3.284  time: 0.4037 s/iter  data_time: 0.2461 s/iter total_throughput: 2536.26 samples/s lr: 8.55e-04  
[09/16 00:09:50] lb.utils.events INFO:  eta: 12:06:09  iteration: 94099/375342  consumed_samples: 96358400  total_loss: 3.295  time: 0.4037 s/iter  data_time: 0.2567 s/iter total_throughput: 2536.27 samples/s lr: 8.54e-04  
[09/16 00:10:30] lb.utils.events INFO:  eta: 12:05:57  iteration: 94199/375342  consumed_samples: 96460800  total_loss: 3.291  time: 0.4037 s/iter  data_time: 0.2519 s/iter total_throughput: 2536.26 samples/s lr: 8.54e-04  
[09/16 00:11:11] lb.utils.events INFO:  eta: 12:05:06  iteration: 94299/375342  consumed_samples: 96563200  total_loss: 3.286  time: 0.4037 s/iter  data_time: 0.2678 s/iter total_throughput: 2536.27 samples/s lr: 8.54e-04  
[09/16 00:11:51] lb.utils.events INFO:  eta: 12:04:45  iteration: 94399/375342  consumed_samples: 96665600  total_loss: 3.28  time: 0.4037 s/iter  data_time: 0.2493 s/iter total_throughput: 2536.28 samples/s lr: 8.53e-04  
[09/16 00:12:31] lb.utils.events INFO:  eta: 12:04:30  iteration: 94499/375342  consumed_samples: 96768000  total_loss: 3.29  time: 0.4037 s/iter  data_time: 0.2534 s/iter total_throughput: 2536.27 samples/s lr: 8.53e-04  
[09/16 00:13:12] lb.utils.events INFO:  eta: 12:04:31  iteration: 94599/375342  consumed_samples: 96870400  total_loss: 3.301  time: 0.4037 s/iter  data_time: 0.2626 s/iter total_throughput: 2536.24 samples/s lr: 8.53e-04  
[09/16 00:13:52] lb.utils.events INFO:  eta: 12:04:17  iteration: 94699/375342  consumed_samples: 96972800  total_loss: 3.304  time: 0.4037 s/iter  data_time: 0.2616 s/iter total_throughput: 2536.26 samples/s lr: 8.52e-04  
[09/16 00:14:33] lb.utils.events INFO:  eta: 12:03:30  iteration: 94799/375342  consumed_samples: 97075200  total_loss: 3.323  time: 0.4037 s/iter  data_time: 0.2585 s/iter total_throughput: 2536.24 samples/s lr: 8.52e-04  
[09/16 00:15:13] lb.utils.events INFO:  eta: 12:02:38  iteration: 94899/375342  consumed_samples: 97177600  total_loss: 3.317  time: 0.4037 s/iter  data_time: 0.2570 s/iter total_throughput: 2536.24 samples/s lr: 8.52e-04  
[09/16 00:15:53] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_2e56/model_0094999
[09/16 00:15:54] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000
[09/16 00:15:54] lb.evaluation.evaluator INFO: Start inference on 50000 samples
[09/16 00:15:59] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.1204 s/iter. Inference: 0.1529 s/iter. Eval: 0.0024 s/iter. Total: 0.2758 s/iter. ETA=0:00:10
[09/16 00:16:04] lb.evaluation.evaluator INFO: Inference done 26624/50000. Dataloading: 0.1664 s/iter. Inference: 0.1519 s/iter. Eval: 0.0022 s/iter. Total: 0.3207 s/iter. ETA=0:00:07
[09/16 00:16:09] lb.evaluation.evaluator INFO: Inference done 43008/50000. Dataloading: 0.1657 s/iter. Inference: 0.1522 s/iter. Eval: 0.0022 s/iter. Total: 0.3201 s/iter. ETA=0:00:01
[09/16 00:16:11] lb.evaluation.evaluator INFO: Total valid samples: 50000
[09/16 00:16:11] lb.evaluation.evaluator INFO: Total inference time: 0:00:13.965988 (0.000279 s / iter per device, on 8 devices)
[09/16 00:16:11] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000134 s / iter per device, on 8 devices)
[09/16 00:16:11] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format:
[09/16 00:16:11] lb.evaluation.utils INFO: copypaste: Acc@1=66.026
[09/16 00:16:11] lb.evaluation.utils INFO: copypaste: Acc@5=86.828
[09/16 00:16:11] lb.engine.hooks INFO: Saved best model as latest eval score for Acc@1 is 66.02600, better than last best score 65.87200 @ iteration 89999.
[09/16 00:16:11] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_2e56/model_best
[09/16 00:16:12] lb.utils.events INFO:  eta: 12:02:06  iteration: 94999/375342  consumed_samples: 97280000  total_loss: 3.297  time: 0.4037 s/iter  data_time: 0.2563 s/iter total_throughput: 2536.25 samples/s lr: 8.52e-04  
[09/16 00:16:50] lb.utils.events INFO:  eta: 12:02:28  iteration: 95099/375342  consumed_samples: 97382400  total_loss: 3.31  time: 0.4037 s/iter  data_time: 0.2510 s/iter total_throughput: 2536.38 samples/s lr: 8.51e-04  
[09/16 00:17:31] lb.utils.events INFO:  eta: 12:02:28  iteration: 95199/375342  consumed_samples: 97484800  total_loss: 3.262  time: 0.4037 s/iter  data_time: 0.2479 s/iter total_throughput: 2536.36 samples/s lr: 8.51e-04  
[09/16 00:18:11] lb.utils.events INFO:  eta: 12:03:29  iteration: 95299/375342  consumed_samples: 97587200  total_loss: 3.265  time: 0.4037 s/iter  data_time: 0.2592 s/iter total_throughput: 2536.35 samples/s lr: 8.51e-04  
[09/16 00:18:52] lb.utils.events INFO:  eta: 12:04:43  iteration: 95399/375342  consumed_samples: 97689600  total_loss: 3.302  time: 0.4037 s/iter  data_time: 0.2570 s/iter total_throughput: 2536.36 samples/s lr: 8.50e-04  
[09/16 00:19:32] lb.utils.events INFO:  eta: 12:07:40  iteration: 95499/375342  consumed_samples: 97792000  total_loss: 3.298  time: 0.4037 s/iter  data_time: 0.2525 s/iter total_throughput: 2536.36 samples/s lr: 8.50e-04  
[09/16 00:20:12] lb.utils.events INFO:  eta: 12:10:44  iteration: 95599/375342  consumed_samples: 97894400  total_loss: 3.277  time: 0.4037 s/iter  data_time: 0.2584 s/iter total_throughput: 2536.37 samples/s lr: 8.50e-04  
[09/16 00:20:52] lb.utils.events INFO:  eta: 12:23:21  iteration: 95699/375342  consumed_samples: 97996800  total_loss: 3.281  time: 0.4037 s/iter  data_time: 0.2604 s/iter total_throughput: 2536.40 samples/s lr: 8.50e-04  
[09/16 00:21:33] lb.utils.events INFO:  eta: 12:52:10  iteration: 95799/375342  consumed_samples: 98099200  total_loss: 3.323  time: 0.4037 s/iter  data_time: 0.2581 s/iter total_throughput: 2536.40 samples/s lr: 8.49e-04  
[09/16 00:22:13] lb.utils.events INFO:  eta: 13:50:25  iteration: 95899/375342  consumed_samples: 98201600  total_loss: 3.313  time: 0.4037 s/iter  data_time: 0.2615 s/iter total_throughput: 2536.39 samples/s lr: 8.49e-04  
[09/16 00:22:54] lb.utils.events INFO:  eta: 15:42:07  iteration: 95999/375342  consumed_samples: 98304000  total_loss: 3.287  time: 0.4037 s/iter  data_time: 0.2664 s/iter total_throughput: 2536.37 samples/s lr: 8.49e-04  
[09/16 00:23:34] lb.utils.events INFO:  eta: 14:40:42  iteration: 96099/375342  consumed_samples: 98406400  total_loss: 3.307  time: 0.4037 s/iter  data_time: 0.2423 s/iter total_throughput: 2536.37 samples/s lr: 8.48e-04  
[09/16 00:24:14] lb.utils.events INFO:  eta: 15:50:46  iteration: 96199/375342  consumed_samples: 98508800  total_loss: 3.304  time: 0.4037 s/iter  data_time: 0.2468 s/iter total_throughput: 2536.41 samples/s lr: 8.48e-04  
[09/16 00:24:54] lb.utils.events INFO:  eta: 15:44:23  iteration: 96299/375342  consumed_samples: 98611200  total_loss: 3.29  time: 0.4037 s/iter  data_time: 0.2494 s/iter total_throughput: 2536.40 samples/s lr: 8.48e-04  
[09/16 00:25:35] lb.utils.events INFO:  eta: 15:34:32  iteration: 96399/375342  consumed_samples: 98713600  total_loss: 3.283  time: 0.4037 s/iter  data_time: 0.2426 s/iter total_throughput: 2536.43 samples/s lr: 8.47e-04  
[09/16 00:26:15] lb.utils.events INFO:  eta: 17:40:58  iteration: 96499/375342  consumed_samples: 98816000  total_loss: 3.28  time: 0.4037 s/iter  data_time: 0.2509 s/iter total_throughput: 2536.44 samples/s lr: 8.47e-04  
[09/16 00:26:55] lb.utils.events INFO:  eta: 15:35:29  iteration: 96599/375342  consumed_samples: 98918400  total_loss: 3.262  time: 0.4037 s/iter  data_time: 0.2528 s/iter total_throughput: 2536.44 samples/s lr: 8.47e-04  
[09/16 00:27:35] lb.utils.events INFO:  eta: 12:28:04  iteration: 96699/375342  consumed_samples: 99020800  total_loss: 3.255  time: 0.4037 s/iter  data_time: 0.2506 s/iter total_throughput: 2536.47 samples/s lr: 8.47e-04  
[09/16 00:28:15] lb.utils.events INFO:  eta: 12:08:52  iteration: 96799/375342  consumed_samples: 99123200  total_loss: 3.275  time: 0.4037 s/iter  data_time: 0.2643 s/iter total_throughput: 2536.49 samples/s lr: 8.46e-04  
[09/16 00:28:55] lb.utils.events INFO:  eta: 12:05:38  iteration: 96899/375342  consumed_samples: 99225600  total_loss: 3.272  time: 0.4037 s/iter  data_time: 0.2552 s/iter total_throughput: 2536.52 samples/s lr: 8.46e-04  
[09/16 00:29:35] lb.utils.events INFO:  eta: 12:04:12  iteration: 96999/375342  consumed_samples: 99328000  total_loss: 3.284  time: 0.4037 s/iter  data_time: 0.2392 s/iter total_throughput: 2536.55 samples/s lr: 8.46e-04  
[09/16 00:30:15] lb.utils.events INFO:  eta: 12:02:40  iteration: 97099/375342  consumed_samples: 99430400  total_loss: 3.278  time: 0.4037 s/iter  data_time: 0.2586 s/iter total_throughput: 2536.56 samples/s lr: 8.45e-04  
[09/16 00:30:55] lb.utils.events INFO:  eta: 12:01:30  iteration: 97199/375342  consumed_samples: 99532800  total_loss: 3.28  time: 0.4037 s/iter  data_time: 0.2512 s/iter total_throughput: 2536.60 samples/s lr: 8.45e-04  
[09/16 00:31:35] lb.utils.events INFO:  eta: 12:00:53  iteration: 97299/375342  consumed_samples: 99635200  total_loss: 3.303  time: 0.4037 s/iter  data_time: 0.2484 s/iter total_throughput: 2536.61 samples/s lr: 8.45e-04  
[09/16 00:32:15] lb.utils.events INFO:  eta: 11:59:54  iteration: 97399/375342  consumed_samples: 99737600  total_loss: 3.277  time: 0.4037 s/iter  data_time: 0.2478 s/iter total_throughput: 2536.62 samples/s lr: 8.44e-04  
[09/16 00:32:55] lb.utils.events INFO:  eta: 11:58:35  iteration: 97499/375342  consumed_samples: 99840000  total_loss: 3.271  time: 0.4037 s/iter  data_time: 0.2481 s/iter total_throughput: 2536.67 samples/s lr: 8.44e-04  
[09/16 00:33:35] lb.utils.events INFO:  eta: 11:57:53  iteration: 97599/375342  consumed_samples: 99942400  total_loss: 3.281  time: 0.4037 s/iter  data_time: 0.2449 s/iter total_throughput: 2536.70 samples/s lr: 8.44e-04  
[09/16 00:34:16] lb.utils.events INFO:  eta: 11:58:29  iteration: 97699/375342  consumed_samples: 100044800  total_loss: 3.261  time: 0.4037 s/iter  data_time: 0.2399 s/iter total_throughput: 2536.67 samples/s lr: 8.44e-04  
[09/16 00:34:56] lb.utils.events INFO:  eta: 11:59:04  iteration: 97799/375342  consumed_samples: 100147200  total_loss: 3.261  time: 0.4037 s/iter  data_time: 0.2554 s/iter total_throughput: 2536.68 samples/s lr: 8.43e-04  
[09/16 00:35:36] lb.utils.events INFO:  eta: 11:59:10  iteration: 97899/375342  consumed_samples: 100249600  total_loss: 3.27  time: 0.4037 s/iter  data_time: 0.2480 s/iter total_throughput: 2536.71 samples/s lr: 8.43e-04  
[09/16 00:36:16] lb.utils.events INFO:  eta: 12:00:41  iteration: 97999/375342  consumed_samples: 100352000  total_loss: 3.255  time: 0.4037 s/iter  data_time: 0.2415 s/iter total_throughput: 2536.73 samples/s lr: 8.43e-04  
[09/16 00:36:56] lb.utils.events INFO:  eta: 12:00:43  iteration: 98099/375342  consumed_samples: 100454400  total_loss: 3.26  time: 0.4037 s/iter  data_time: 0.2647 s/iter total_throughput: 2536.73 samples/s lr: 8.42e-04  
[09/16 00:37:37] lb.utils.events INFO:  eta: 12:00:33  iteration: 98199/375342  consumed_samples: 100556800  total_loss: 3.283  time: 0.4037 s/iter  data_time: 0.2568 s/iter total_throughput: 2536.74 samples/s lr: 8.42e-04  
[09/16 00:38:17] lb.utils.events INFO:  eta: 11:58:25  iteration: 98299/375342  consumed_samples: 100659200  total_loss: 3.284  time: 0.4037 s/iter  data_time: 0.2591 s/iter total_throughput: 2536.72 samples/s lr: 8.42e-04  
[09/16 00:38:58] lb.utils.events INFO:  eta: 12:00:26  iteration: 98399/375342  consumed_samples: 100761600  total_loss: 3.292  time: 0.4037 s/iter  data_time: 0.2510 s/iter total_throughput: 2536.71 samples/s lr: 8.41e-04  
[09/16 00:39:39] lb.utils.events INFO:  eta: 12:03:57  iteration: 98499/375342  consumed_samples: 100864000  total_loss: 3.291  time: 0.4037 s/iter  data_time: 0.2638 s/iter total_throughput: 2536.63 samples/s lr: 8.41e-04  
[09/16 00:40:21] lb.utils.events INFO:  eta: 12:04:41  iteration: 98599/375342  consumed_samples: 100966400  total_loss: 3.28  time: 0.4037 s/iter  data_time: 0.2848 s/iter total_throughput: 2536.52 samples/s lr: 8.41e-04  
[09/16 00:41:04] lb.utils.events INFO:  eta: 12:04:25  iteration: 98699/375342  consumed_samples: 101068800  total_loss: 3.285  time: 0.4037 s/iter  data_time: 0.2647 s/iter total_throughput: 2536.40 samples/s lr: 8.40e-04  
[09/16 00:41:45] lb.utils.events INFO:  eta: 12:01:26  iteration: 98799/375342  consumed_samples: 101171200  total_loss: 3.295  time: 0.4037 s/iter  data_time: 0.2545 s/iter total_throughput: 2536.34 samples/s lr: 8.40e-04  
[09/16 00:42:26] lb.utils.events INFO:  eta: 12:03:41  iteration: 98899/375342  consumed_samples: 101273600  total_loss: 3.288  time: 0.4037 s/iter  data_time: 0.2624 s/iter total_throughput: 2536.30 samples/s lr: 8.40e-04  
[09/16 00:43:08] lb.utils.events INFO:  eta: 12:02:07  iteration: 98999/375342  consumed_samples: 101376000  total_loss: 3.258  time: 0.4038 s/iter  data_time: 0.2711 s/iter total_throughput: 2536.21 samples/s lr: 8.40e-04  
[09/16 00:43:48] lb.utils.events INFO:  eta: 12:01:50  iteration: 99099/375342  consumed_samples: 101478400  total_loss: 3.262  time: 0.4038 s/iter  data_time: 0.2614 s/iter total_throughput: 2536.19 samples/s lr: 8.39e-04  
[09/16 00:44:29] lb.utils.events INFO:  eta: 12:02:54  iteration: 99199/375342  consumed_samples: 101580800  total_loss: 3.277  time: 0.4038 s/iter  data_time: 0.2548 s/iter total_throughput: 2536.20 samples/s lr: 8.39e-04  
[09/16 00:45:10] lb.utils.events INFO:  eta: 12:01:19  iteration: 99299/375342  consumed_samples: 101683200  total_loss: 3.283  time: 0.4038 s/iter  data_time: 0.2613 s/iter total_throughput: 2536.16 samples/s lr: 8.39e-04  
[09/16 00:45:52] lb.utils.events INFO:  eta: 11:58:49  iteration: 99399/375342  consumed_samples: 101785600  total_loss: 3.289  time: 0.4038 s/iter  data_time: 0.2637 s/iter total_throughput: 2536.07 samples/s lr: 8.38e-04  
[09/16 00:46:33] lb.utils.events INFO:  eta: 11:54:39  iteration: 99499/375342  consumed_samples: 101888000  total_loss: 3.289  time: 0.4038 s/iter  data_time: 0.2678 s/iter total_throughput: 2536.00 samples/s lr: 8.38e-04  
[09/16 00:47:15] lb.utils.events INFO:  eta: 11:53:36  iteration: 99599/375342  consumed_samples: 101990400  total_loss: 3.307  time: 0.4038 s/iter  data_time: 0.2656 s/iter total_throughput: 2535.93 samples/s lr: 8.38e-04  
[09/16 00:47:56] lb.utils.events INFO:  eta: 11:52:07  iteration: 99699/375342  consumed_samples: 102092800  total_loss: 3.303  time: 0.4038 s/iter  data_time: 0.2558 s/iter total_throughput: 2535.89 samples/s lr: 8.37e-04  
[09/16 00:48:36] lb.utils.events INFO:  eta: 11:51:46  iteration: 99799/375342  consumed_samples: 102195200  total_loss: 3.287  time: 0.4038 s/iter  data_time: 0.2657 s/iter total_throughput: 2535.89 samples/s lr: 8.37e-04  
[09/16 00:49:17] lb.utils.events INFO:  eta: 11:52:08  iteration: 99899/375342  consumed_samples: 102297600  total_loss: 3.3  time: 0.4038 s/iter  data_time: 0.2677 s/iter total_throughput: 2535.85 samples/s lr: 8.37e-04  
[09/16 00:49:58] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_2e56/model_0099999
[09/16 00:49:59] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000
[09/16 00:49:59] lb.evaluation.evaluator INFO: Start inference on 50000 samples
[09/16 00:50:03] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.1260 s/iter. Inference: 0.1526 s/iter. Eval: 0.0022 s/iter. Total: 0.2809 s/iter. ETA=0:00:10
[09/16 00:50:09] lb.evaluation.evaluator INFO: Inference done 26624/50000. Dataloading: 0.1701 s/iter. Inference: 0.1536 s/iter. Eval: 0.0022 s/iter. Total: 0.3261 s/iter. ETA=0:00:07
[09/16 00:50:14] lb.evaluation.evaluator INFO: Inference done 43008/50000. Dataloading: 0.1688 s/iter. Inference: 0.1536 s/iter. Eval: 0.0022 s/iter. Total: 0.3247 s/iter. ETA=0:00:01
[09/16 00:50:16] lb.evaluation.evaluator INFO: Total valid samples: 50000
[09/16 00:50:16] lb.evaluation.evaluator INFO: Total inference time: 0:00:13.809569 (0.000276 s / iter per device, on 8 devices)
[09/16 00:50:16] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000135 s / iter per device, on 8 devices)
[09/16 00:50:16] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format:
[09/16 00:50:16] lb.evaluation.utils INFO: copypaste: Acc@1=64.91799999999999
[09/16 00:50:16] lb.evaluation.utils INFO: copypaste: Acc@5=86.046
[09/16 00:50:16] lb.engine.hooks INFO: Not saving as latest eval score for Acc@1 is 64.91800, not better than best score 66.02600 @ iteration 94999.
[09/16 00:50:16] lb.utils.events INFO:  eta: 11:51:34  iteration: 99999/375342  consumed_samples: 102400000  total_loss: 3.278  time: 0.4038 s/iter  data_time: 0.2534 s/iter total_throughput: 2535.80 samples/s lr: 8.37e-04  
[09/16 00:50:55] lb.utils.events INFO:  eta: 11:51:35  iteration: 100099/375342  consumed_samples: 102502400  total_loss: 3.283  time: 0.4038 s/iter  data_time: 0.2601 s/iter total_throughput: 2535.89 samples/s lr: 8.36e-04  
[09/16 00:51:35] lb.utils.events INFO:  eta: 11:52:05  iteration: 100199/375342  consumed_samples: 102604800  total_loss: 3.282  time: 0.4038 s/iter  data_time: 0.2555 s/iter total_throughput: 2535.86 samples/s lr: 8.36e-04  
[09/16 00:52:16] lb.utils.events INFO:  eta: 11:53:12  iteration: 100299/375342  consumed_samples: 102707200  total_loss: 3.258  time: 0.4038 s/iter  data_time: 0.2580 s/iter total_throughput: 2535.85 samples/s lr: 8.36e-04  
[09/16 00:52:57] lb.utils.events INFO:  eta: 11:55:01  iteration: 100399/375342  consumed_samples: 102809600  total_loss: 3.254  time: 0.4038 s/iter  data_time: 0.2594 s/iter total_throughput: 2535.81 samples/s lr: 8.35e-04  
[09/16 00:53:38] lb.utils.events INFO:  eta: 11:56:53  iteration: 100499/375342  consumed_samples: 102912000  total_loss: 3.249  time: 0.4038 s/iter  data_time: 0.2722 s/iter total_throughput: 2535.75 samples/s lr: 8.35e-04  
[09/16 00:54:19] lb.utils.events INFO:  eta: 11:59:31  iteration: 100599/375342  consumed_samples: 103014400  total_loss: 3.267  time: 0.4038 s/iter  data_time: 0.2512 s/iter total_throughput: 2535.73 samples/s lr: 8.35e-04  
[09/16 00:55:00] lb.utils.events INFO:  eta: 12:00:14  iteration: 100699/375342  consumed_samples: 103116800  total_loss: 3.265  time: 0.4038 s/iter  data_time: 0.2656 s/iter total_throughput: 2535.70 samples/s lr: 8.34e-04  
[09/16 00:55:42] lb.utils.events INFO:  eta: 12:02:12  iteration: 100799/375342  consumed_samples: 103219200  total_loss: 3.279  time: 0.4038 s/iter  data_time: 0.2695 s/iter total_throughput: 2535.63 samples/s lr: 8.34e-04  
[09/16 00:56:22] lb.utils.events INFO:  eta: 11:58:16  iteration: 100899/375342  consumed_samples: 103321600  total_loss: 3.287  time: 0.4038 s/iter  data_time: 0.2602 s/iter total_throughput: 2535.61 samples/s lr: 8.34e-04  
[09/16 00:57:03] lb.utils.events INFO:  eta: 11:56:09  iteration: 100999/375342  consumed_samples: 103424000  total_loss: 3.292  time: 0.4039 s/iter  data_time: 0.2714 s/iter total_throughput: 2535.56 samples/s lr: 8.33e-04  
[09/16 00:57:44] lb.utils.events INFO:  eta: 11:54:13  iteration: 101099/375342  consumed_samples: 103526400  total_loss: 3.286  time: 0.4039 s/iter  data_time: 0.2578 s/iter total_throughput: 2535.57 samples/s lr: 8.33e-04  
[09/16 00:58:24] lb.utils.events INFO:  eta: 11:51:33  iteration: 101199/375342  consumed_samples: 103628800  total_loss: 3.282  time: 0.4039 s/iter  data_time: 0.2488 s/iter total_throughput: 2535.58 samples/s lr: 8.33e-04  
[09/16 00:59:04] lb.utils.events INFO:  eta: 11:51:53  iteration: 101299/375342  consumed_samples: 103731200  total_loss: 3.29  time: 0.4039 s/iter  data_time: 0.2461 s/iter total_throughput: 2535.58 samples/s lr: 8.32e-04  
[09/16 00:59:45] lb.utils.events INFO:  eta: 11:51:07  iteration: 101399/375342  consumed_samples: 103833600  total_loss: 3.281  time: 0.4039 s/iter  data_time: 0.2530 s/iter total_throughput: 2535.56 samples/s lr: 8.32e-04  
[09/16 01:00:26] lb.utils.events INFO:  eta: 11:49:48  iteration: 101499/375342  consumed_samples: 103936000  total_loss: 3.265  time: 0.4039 s/iter  data_time: 0.2549 s/iter total_throughput: 2535.53 samples/s lr: 8.32e-04  
[09/16 01:01:07] lb.utils.events INFO:  eta: 11:49:25  iteration: 101599/375342  consumed_samples: 104038400  total_loss: 3.276  time: 0.4039 s/iter  data_time: 0.2676 s/iter total_throughput: 2535.48 samples/s lr: 8.32e-04  
[09/16 01:01:48] lb.utils.events INFO:  eta: 11:48:52  iteration: 101699/375342  consumed_samples: 104140800  total_loss: 3.275  time: 0.4039 s/iter  data_time: 0.2460 s/iter total_throughput: 2535.42 samples/s lr: 8.31e-04  
[09/16 01:02:29] lb.utils.events INFO:  eta: 11:47:59  iteration: 101799/375342  consumed_samples: 104243200  total_loss: 3.252  time: 0.4039 s/iter  data_time: 0.2649 s/iter total_throughput: 2535.41 samples/s lr: 8.31e-04  
[09/16 01:03:10] lb.utils.events INFO:  eta: 11:47:49  iteration: 101899/375342  consumed_samples: 104345600  total_loss: 3.274  time: 0.4039 s/iter  data_time: 0.2540 s/iter total_throughput: 2535.38 samples/s lr: 8.31e-04  
[09/16 01:03:50] lb.utils.events INFO:  eta: 11:47:36  iteration: 101999/375342  consumed_samples: 104448000  total_loss: 3.284  time: 0.4039 s/iter  data_time: 0.2591 s/iter total_throughput: 2535.37 samples/s lr: 8.30e-04  
[09/16 01:04:31] lb.utils.events INFO:  eta: 11:48:02  iteration: 102099/375342  consumed_samples: 104550400  total_loss: 3.269  time: 0.4039 s/iter  data_time: 0.2615 s/iter total_throughput: 2535.38 samples/s lr: 8.30e-04  
[09/16 01:05:11] lb.utils.events INFO:  eta: 11:50:17  iteration: 102199/375342  consumed_samples: 104652800  total_loss: 3.275  time: 0.4039 s/iter  data_time: 0.2506 s/iter total_throughput: 2535.39 samples/s lr: 8.30e-04  
[09/16 01:05:51] lb.utils.events INFO:  eta: 11:53:40  iteration: 102299/375342  consumed_samples: 104755200  total_loss: 3.278  time: 0.4039 s/iter  data_time: 0.2517 s/iter total_throughput: 2535.40 samples/s lr: 8.29e-04  
[09/16 01:06:32] lb.utils.events INFO:  eta: 11:53:33  iteration: 102399/375342  consumed_samples: 104857600  total_loss: 3.257  time: 0.4039 s/iter  data_time: 0.2561 s/iter total_throughput: 2535.39 samples/s lr: 8.29e-04  
[09/16 01:07:13] lb.utils.events INFO:  eta: 11:53:40  iteration: 102499/375342  consumed_samples: 104960000  total_loss: 3.287  time: 0.4039 s/iter  data_time: 0.2522 s/iter total_throughput: 2535.37 samples/s lr: 8.29e-04  
[09/16 01:07:53] lb.utils.events INFO:  eta: 11:53:24  iteration: 102599/375342  consumed_samples: 105062400  total_loss: 3.293  time: 0.4039 s/iter  data_time: 0.2537 s/iter total_throughput: 2535.36 samples/s lr: 8.28e-04  
[09/16 01:08:34] lb.utils.events INFO:  eta: 11:52:46  iteration: 102699/375342  consumed_samples: 105164800  total_loss: 3.273  time: 0.4039 s/iter  data_time: 0.2559 s/iter total_throughput: 2535.34 samples/s lr: 8.28e-04  
[09/16 01:09:14] lb.utils.events INFO:  eta: 11:52:49  iteration: 102799/375342  consumed_samples: 105267200  total_loss: 3.262  time: 0.4039 s/iter  data_time: 0.2546 s/iter total_throughput: 2535.32 samples/s lr: 8.28e-04  
[09/16 01:09:55] lb.utils.events INFO:  eta: 11:52:36  iteration: 102899/375342  consumed_samples: 105369600  total_loss: 3.259  time: 0.4039 s/iter  data_time: 0.2553 s/iter total_throughput: 2535.32 samples/s lr: 8.27e-04  
[09/16 01:10:35] lb.utils.events INFO:  eta: 11:52:20  iteration: 102999/375342  consumed_samples: 105472000  total_loss: 3.258  time: 0.4039 s/iter  data_time: 0.2592 s/iter total_throughput: 2535.31 samples/s lr: 8.27e-04  
[09/16 01:11:16] lb.utils.events INFO:  eta: 11:49:12  iteration: 103099/375342  consumed_samples: 105574400  total_loss: 3.255  time: 0.4039 s/iter  data_time: 0.2595 s/iter total_throughput: 2535.27 samples/s lr: 8.27e-04  
[09/16 01:11:58] lb.utils.events INFO:  eta: 11:46:36  iteration: 103199/375342  consumed_samples: 105676800  total_loss: 3.257  time: 0.4039 s/iter  data_time: 0.2650 s/iter total_throughput: 2535.23 samples/s lr: 8.27e-04  
[09/16 01:12:38] lb.utils.events INFO:  eta: 11:43:33  iteration: 103299/375342  consumed_samples: 105779200  total_loss: 3.261  time: 0.4039 s/iter  data_time: 0.2553 s/iter total_throughput: 2535.21 samples/s lr: 8.26e-04  
[09/16 01:13:19] lb.utils.events INFO:  eta: 11:41:58  iteration: 103399/375342  consumed_samples: 105881600  total_loss: 3.252  time: 0.4039 s/iter  data_time: 0.2562 s/iter total_throughput: 2535.20 samples/s lr: 8.26e-04  
[09/16 01:13:59] lb.utils.events INFO:  eta: 11:42:09  iteration: 103499/375342  consumed_samples: 105984000  total_loss: 3.243  time: 0.4039 s/iter  data_time: 0.2661 s/iter total_throughput: 2535.19 samples/s lr: 8.26e-04  
[09/16 01:14:40] lb.utils.events INFO:  eta: 11:42:26  iteration: 103599/375342  consumed_samples: 106086400  total_loss: 3.255  time: 0.4039 s/iter  data_time: 0.2486 s/iter total_throughput: 2535.17 samples/s lr: 8.25e-04  
[09/16 01:15:21] lb.utils.events INFO:  eta: 11:43:20  iteration: 103699/375342  consumed_samples: 106188800  total_loss: 3.289  time: 0.4039 s/iter  data_time: 0.2622 s/iter total_throughput: 2535.15 samples/s lr: 8.25e-04  
[09/16 01:16:02] lb.utils.events INFO:  eta: 11:44:20  iteration: 103799/375342  consumed_samples: 106291200  total_loss: 3.296  time: 0.4039 s/iter  data_time: 0.2483 s/iter total_throughput: 2535.13 samples/s lr: 8.25e-04  
[09/16 01:16:42] lb.utils.events INFO:  eta: 11:46:42  iteration: 103899/375342  consumed_samples: 106393600  total_loss: 3.273  time: 0.4039 s/iter  data_time: 0.2545 s/iter total_throughput: 2535.11 samples/s lr: 8.24e-04  
[09/16 01:17:23] lb.utils.events INFO:  eta: 11:49:50  iteration: 103999/375342  consumed_samples: 106496000  total_loss: 3.271  time: 0.4039 s/iter  data_time: 0.2475 s/iter total_throughput: 2535.12 samples/s lr: 8.24e-04  
[09/16 01:18:04] lb.utils.events INFO:  eta: 11:58:42  iteration: 104099/375342  consumed_samples: 106598400  total_loss: 3.266  time: 0.4039 s/iter  data_time: 0.2687 s/iter total_throughput: 2535.08 samples/s lr: 8.24e-04  
[09/16 01:18:45] lb.utils.events INFO:  eta: 12:19:35  iteration: 104199/375342  consumed_samples: 106700800  total_loss: 3.263  time: 0.4039 s/iter  data_time: 0.2621 s/iter total_throughput: 2535.04 samples/s lr: 8.23e-04  
[09/16 01:19:25] lb.utils.events INFO:  eta: 12:43:53  iteration: 104299/375342  consumed_samples: 106803200  total_loss: 3.245  time: 0.4039 s/iter  data_time: 0.2541 s/iter total_throughput: 2535.05 samples/s lr: 8.23e-04  
[09/16 01:20:06] lb.utils.events INFO:  eta: 13:13:43  iteration: 104399/375342  consumed_samples: 106905600  total_loss: 3.245  time: 0.4039 s/iter  data_time: 0.2656 s/iter total_throughput: 2535.00 samples/s lr: 8.23e-04  
[09/16 01:20:47] lb.utils.events INFO:  eta: 13:39:27  iteration: 104499/375342  consumed_samples: 107008000  total_loss: 3.259  time: 0.4039 s/iter  data_time: 0.2504 s/iter total_throughput: 2535.00 samples/s lr: 8.22e-04  
[09/16 01:21:27] lb.utils.events INFO:  eta: 12:18:47  iteration: 104599/375342  consumed_samples: 107110400  total_loss: 3.267  time: 0.4039 s/iter  data_time: 0.2568 s/iter total_throughput: 2534.98 samples/s lr: 8.22e-04  
[09/16 01:22:08] lb.utils.events INFO:  eta: 11:58:42  iteration: 104699/375342  consumed_samples: 107212800  total_loss: 3.252  time: 0.4039 s/iter  data_time: 0.2582 s/iter total_throughput: 2534.97 samples/s lr: 8.22e-04  
[09/16 01:22:49] lb.utils.events INFO:  eta: 11:51:52  iteration: 104799/375342  consumed_samples: 107315200  total_loss: 3.259  time: 0.4040 s/iter  data_time: 0.2532 s/iter total_throughput: 2534.95 samples/s lr: 8.21e-04  
[09/16 01:23:29] lb.utils.events INFO:  eta: 11:46:27  iteration: 104899/375342  consumed_samples: 107417600  total_loss: 3.263  time: 0.4040 s/iter  data_time: 0.2593 s/iter total_throughput: 2534.96 samples/s lr: 8.21e-04  
[09/16 01:24:10] lb.utils.checkpoint INFO: Saving checkpoint to ./commit_2e56/model_0104999
[09/16 01:24:11] lb.evaluation.evaluator INFO: with eval_iter 100000.0, reset total samples 50000 to 50000
[09/16 01:24:11] lb.evaluation.evaluator INFO: Start inference on 50000 samples
[09/16 01:24:15] lb.evaluation.evaluator INFO: Inference done 11264/50000. Dataloading: 0.1131 s/iter. Inference: 0.1542 s/iter. Eval: 0.0021 s/iter. Total: 0.2695 s/iter. ETA=0:00:09
[09/16 01:24:20] lb.evaluation.evaluator INFO: Inference done 26624/50000. Dataloading: 0.1562 s/iter. Inference: 0.1581 s/iter. Eval: 0.0021 s/iter. Total: 0.3164 s/iter. ETA=0:00:06
[09/16 01:24:25] lb.evaluation.evaluator INFO: Inference done 43008/50000. Dataloading: 0.1580 s/iter. Inference: 0.1579 s/iter. Eval: 0.0021 s/iter. Total: 0.3181 s/iter. ETA=0:00:01
[09/16 01:24:27] lb.evaluation.evaluator INFO: Total valid samples: 50000
[09/16 01:24:27] lb.evaluation.evaluator INFO: Total inference time: 0:00:13.821599 (0.000276 s / iter per device, on 8 devices)
[09/16 01:24:27] lb.evaluation.evaluator INFO: Total inference pure compute time: 0:00:06 (0.000139 s / iter per device, on 8 devices)
[09/16 01:24:27] lb.engine.default INFO: Evaluation results for ImageNetDataset in csv format:
[09/16 01:24:27] lb.evaluation.utils INFO: copypaste: Acc@1=65.75999999999999
[09/16 01:24:27] lb.evaluation.utils INFO: copypaste: Acc@5=86.71799999999999
[09/16 01:24:27] lb.engine.hooks INFO: Not saving as latest eval score for Acc@1 is 65.76000, not better than best score 66.02600 @ iteration 94999.
[09/16 01:24:27] lb.utils.events INFO:  eta: 11:42:17  iteration: 104999/375342  consumed_samples: 107520000  total_loss: 3.249  time: 0.4040 s/iter  data_time: 0.2710 s/iter total_throughput: 2534.92 samples/s lr: 8.21e-04  
[09/16 01:25:06] lb.utils.events INFO:  eta: 11:41:52  iteration: 105099/375342  consumed_samples: 107622400  total_loss: 3.258  time: 0.4039 s/iter  data_time: 0.2548 s/iter total_throughput: 2535.01 samples/s lr: 8.21e-04  
[09/16 01:25:47] lb.utils.events INFO:  eta: 11:41:05  iteration: 105199/375342  consumed_samples: 107724800  total_loss: 3.265  time: 0.4039 s/iter  data_time: 0.2553 s/iter total_throughput: 2534.99 samples/s lr: 8.20e-04  
[09/16 01:26:28] lb.utils.events INFO:  eta: 11:41:12  iteration: 105299/375342  consumed_samples: 107827200  total_loss: 3.248  time: 0.4039 s/iter  data_time: 0.2544 s/iter total_throughput: 2534.97 samples/s lr: 8.20e-04  
[09/16 01:27:08] lb.utils.events INFO:  eta: 11:41:54  iteration: 105399/375342  consumed_samples: 107929600  total_loss: 3.233  time: 0.4040 s/iter  data_time: 0.2565 s/iter total_throughput: 2534.96 samples/s lr: 8.20e-04  
[09/16 01:27:49] lb.utils.events INFO:  eta: 11:42:33  iteration: 105499/375342  consumed_samples: 108032000  total_loss: 3.246  time: 0.4040 s/iter  data_time: 0.2624 s/iter total_throughput: 2534.94 samples/s lr: 8.19e-04  
[09/16 01:28:30] lb.utils.events INFO:  eta: 11:43:21  iteration: 105599/375342  consumed_samples: 108134400  total_loss: 3.264  time: 0.4040 s/iter  data_time: 0.2592 s/iter total_throughput: 2534.92 samples/s lr: 8.19e-04  
[09/16 01:29:11] lb.utils.events INFO:  eta: 11:42:10  iteration: 105699/375342  consumed_samples: 108236800  total_loss: 3.259  time: 0.4040 s/iter  data_time: 0.2468 s/iter total_throughput: 2534.90 samples/s lr: 8.19e-04  
[09/16 01:29:51] lb.utils.events INFO:  eta: 11:43:25  iteration: 105799/375342  consumed_samples: 108339200  total_loss: 3.259  time: 0.4040 s/iter  data_time: 0.2538 s/iter total_throughput: 2534.89 samples/s lr: 8.18e-04  
[09/16 01:30:31] lb.utils.events INFO:  eta: 11:46:05  iteration: 105899/375342  consumed_samples: 108441600  total_loss: 3.279  time: 0.4040 s/iter  data_time: 0.2434 s/iter total_throughput: 2534.91 samples/s lr: 8.18e-04  
[09/16 01:31:12] lb.utils.events INFO:  eta: 11:50:02  iteration: 105999/375342  consumed_samples: 108544000  total_loss: 3.274  time: 0.4040 s/iter  data_time: 0.2599 s/iter total_throughput: 2534.91 samples/s lr: 8.18e-04  
[09/16 01:31:52] lb.utils.events INFO:  eta: 11:49:00  iteration: 106099/375342  consumed_samples: 108646400  total_loss: 3.262  time: 0.4040 s/iter  data_time: 0.2669 s/iter total_throughput: 2534.90 samples/s lr: 8.17e-04