add model: regionvit

[kaijieshi7]

Add RegionViT

• build model (F.unfold 算子不支持 Can I use oneflow to efficiently implement methods like torch.nn.functional.unfold() and torch.nn.functional.fold() oneflow#3785)
• update init.py in models
• convert pretrained weight
• inference test on imagenet and update model_zoo
• update docs
• update changelog
• pytorch speed comparison

[rentainhe]

可以先用nn.Unfold绕过去

[kaijieshi7]

测速时，196行加下面的代码

for i, line in enumerate(lines):
    if "self.rel_pos_index = flow.tensor(out, dtype=flow.long)" in line:
        lines = lines[:i] + ["            self.rel_pos_index = flow.tensor(out)",] + lines[i + 1 :]
        break
for i, line in enumerate(lines):
    if "out.append(tmp[dy - y + offset_y, dx - x + offset_x].numpy())" in line:
        lines = lines[:i] + ["                            out.append(tmp[dy - y + offset_y, dx - x + offset_x])",] + lines[i + 1 :]
        break
for i, line in enumerate(lines):
    if "attn = attn.masked_fill(mask == 0, float(np.finfo(attn.numpy().dtype).min))" in line:
        lines = lines[:i] + ["            attn = attn.masked_fill(mask == 0, torch.finfo(attn.dtype).min)",] + lines[i + 1 :]
        break

[kaijieshi7]

OneFlow regionvit_tiny_224 time: 75.4ms (= 7537.5ms / 100, input_shape=[16, 3, 224, 224])
PyTorch regionvit_tiny_224 time: 101.5ms (= 10150.3ms / 100, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.35 (= 101.5ms / 75.4ms)

OneFlow regionvit_small_224 time: 87.5ms (= 8751.0ms / 100, input_shape=[16, 3, 224, 224])
PyTorch regionvit_small_224 time: 126.7ms (= 12673.1ms / 100, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.45 (= 126.7ms / 87.5ms)

OneFlow regionvit_small_w14_224 time: 168.0ms (= 16804.1ms / 100, input_shape=[16, 3, 224, 224])
PyTorch regionvit_small_w14_224 time: 198.2ms (= 19815.2ms / 100, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.18 (= 198.2ms / 168.0ms)

OneFlow regionvit_small_w14_peg_224 time: 178.5ms (= 17849.5ms / 100, input_shape=[16, 3, 224, 224])
PyTorch regionvit_small_w14_peg_224 time: 224.2ms (= 22421.7ms / 100, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.26 (= 224.2ms / 178.5ms)

OneFlow regionvit_medium_224 time: 139.1ms (= 13908.7ms / 100, input_shape=[16, 3, 224, 224])
PyTorch regionvit_medium_224 time: 151.9ms (= 15191.3ms / 100, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.09 (= 151.9ms / 139.1ms)

OneFlow regionvit_base_224 time: 166.2ms (= 16623.4ms / 100, input_shape=[16, 3, 224, 224])
PyTorch regionvit_base_224 time: 201.1ms (= 20114.3ms / 100, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.21 (= 201.1ms / 166.2ms)

OneFlow regionvit_base_w14_224 time: 299.6ms (= 29962.9ms / 100, input_shape=[16, 3, 224, 224])
PyTorch regionvit_base_w14_224 time: 318.6ms (= 31858.9ms / 100, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.06 (= 318.6ms / 299.6ms)

OneFlow regionvit_base_w14_peg_224 time: 305.8ms (= 30583.4ms / 100, input_shape=[16, 3, 224, 224])
PyTorch regionvit_base_w14_peg_224 time: 350.2ms (= 35016.8ms / 100, input_shape=[16, 3, 224, 224])
✔️ Relative speed: 1.14 (= 350.2ms / 305.8ms)

OneFlow regionvit_tiny_224 time: 75.9ms (= 7591.4ms / 100, input_shape=[16, 3, 224, 224], world size=2)
PyTorch regionvit_tiny_224 time: 189.1ms (= 18911.1ms / 100, input_shape=[16, 3, 224, 224], world size=2)
✔️ Relative speed: 2.49 (= 189.1ms / 75.9ms)

OneFlow regionvit_small_224 time: 87.8ms (= 8775.0ms / 100, input_shape=[16, 3, 224, 224], world size=2)
PyTorch regionvit_small_224 time: 265.0ms (= 26503.2ms / 100, input_shape=[16, 3, 224, 224], world size=2)
✔️ Relative speed: 3.02 (= 265.0ms / 87.8ms)

OneFlow regionvit_small_w14_224 time: 168.5ms (= 16848.2ms / 100, input_shape=[16, 3, 224, 224], world size=2)
PyTorch regionvit_small_w14_224 time: 433.3ms (= 43331.0ms / 100, input_shape=[16, 3, 224, 224], world size=2)
✔️ Relative speed: 2.57 (= 433.3ms / 168.5ms)

OneFlow regionvit_small_w14_peg_224 time: 206.7ms (= 20672.3ms / 100, input_shape=[16, 3, 224, 224], world size=2)
PyTorch regionvit_small_w14_peg_224 time: 506.3ms (= 50626.8ms / 100, input_shape=[16, 3, 224, 224], world size=2)
✔️ Relative speed: 2.45 (= 506.3ms / 206.7ms)

OneFlow regionvit_medium_224 time: 141.2ms (= 14120.5ms / 100, input_shape=[16, 3, 224, 224], world size=2)
PyTorch regionvit_medium_224 time: 309.5ms (= 30951.7ms / 100, input_shape=[16, 3, 224, 224], world size=2)
✔️ Relative speed: 2.19 (= 309.5ms / 141.2ms)

OneFlow regionvit_base_224 time: 166.3ms (= 16634.6ms / 100, input_shape=[16, 3, 224, 224], world size=2)
PyTorch regionvit_base_224 time: 442.6ms (= 44260.8ms / 100, input_shape=[16, 3, 224, 224], world size=2)
✔️ Relative speed: 2.66 (= 442.6ms / 166.3ms)

[kaijieshi7]

OneFlow regionvit_base_w14_224 time: 185.0ms (= 18498.5ms / 100, input_shape=[8, 3, 224, 224], world size=2)
PyTorch regionvit_base_w14_224 time: 387.7ms (= 38771.0ms / 100, input_shape=[8, 3, 224, 224], world size=2)
✔️ Relative speed: 2.10 (= 387.7ms / 185.0ms)

OneFlow regionvit_base_w14_peg_224 time: 190.5ms (= 19045.6ms / 100, input_shape=[8, 3, 224, 224], world size=2)
PyTorch regionvit_base_w14_peg_224 time: 420.6ms (= 42059.7ms / 100, input_shape=[8, 3, 224, 224], world size=2)
✔️ Relative speed: 2.21 (= 420.6ms / 190.5ms)

[rentainhe]

这里是不是改成region_vit_tiny更合适一些呢, 我感觉这里的名字用全一点比较好

[kaijieshi7]

改了