[CodeCamp2023-367] Add pp_mobileseg model #3239

Yang-Changhui · 2023-07-31T12:21:06Z

Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers.

Motivation

Please describe the motivation of this PR and the goal you want to achieve through this PR.

Modification

Please briefly describe what modification is made in this PR.

BC-breaking (Optional)

Does the modification introduce changes that break the backward-compatibility of the downstream repos?
If so, please describe how it breaks the compatibility and how the downstream projects should modify their code to keep compatibility with this PR.

Use cases (Optional)

If this PR introduces a new feature, it is better to list some use cases here, and update the documentation.

Checklist

Pre-commit or other linting tools are used to fix the potential lint issues.
The modification is covered by complete unit tests. If not, please add more unit test to ensure the correctness.
If the modification has potential influence on downstream projects, this PR should be tested with downstream projects, like MMDet or MMDet3D.
The documentation has been modified accordingly, like docstring or example tutorials.

mm-assistant · 2023-07-31T12:21:10Z

We recommend using English or English & Chinese for pull requests so that we could have broader discussion.

CLAassistant · 2023-07-31T12:21:14Z

All committers have signed the CLA.

OpenMMLab-Assistant-004 · 2023-08-02T07:02:29Z

Hi @Yang-Changhui,

We'd like to express our appreciation for your valuable contributions to the mmsegmentation. Your efforts have significantly aided in enhancing the project's quality.
It is our pleasure to invite you to join our community thorugh Discord_Special Interest Group (SIG) channel. This is a great place to share your experiences, discuss ideas, and connect with other like-minded people. To become a part of the SIG channel, send a message to the moderator, OpenMMLab, briefly introduce yourself and mention your open-source contributions in the #introductions channel. Our team will gladly facilitate your entry. We eagerly await your presence. Please follow this link to join us: https://discord.gg/UjgXkPWNqA.

If you're on WeChat, we'd also love for you to join our community there. Just add our assistant using the WeChat ID: openmmlabwx. When sending the friend request, remember to include the remark "mmsig + Github ID".

Thanks again for your awesome contribution, and we're excited to have you as part of our community!

xiexinch · 2023-08-02T09:40:42Z

mmseg/models/backbones/strideformer.py

+from mmengine.model import BaseModule
+from mmengine.runner.checkpoint import CheckpointLoader, load_state_dict
+
+from mmseg.models.utils.transformer_utils import DropPath


Can't find this file.
We might modify it like https://github.com/open-mmlab/mmsegmentation/blob/main/mmseg/models/backbones/swin.py#L178

xiexinch · 2023-08-04T09:20:52Z

mmseg/models/backbones/strideformer.py

+                 cfg1,
+                 cfg2,
+                 cfg3,
+                 cfg4,


Generally, we should use meaningful parameter names.

mmseg/models/backbones/strideformer.py

xiexinch · 2023-08-04T09:23:05Z

mmseg/models/backbones/strideformer.py

+class ConvBNAct(nn.Module):
+    def __init__(self,
+                 in_channels,
+                 out_channels,
+                 kernel_size=1,
+                 stride=1,
+                 padding=0,
+                 groups=1,
+                 conv_cfg=dict(type='Conv'),
+                 norm_cfg=dict(type='BN'),
+                 act_cfg=None,
+                 bias_attr=False):
+        super(ConvBNAct, self).__init__()
+
+        self.conv = build_conv_layer(
+            conv_cfg,
+            in_channels=in_channels,
+            out_channels=out_channels,
+            kernel_size=kernel_size,
+            stride=stride,
+            padding=padding,
+            groups=groups,
+            bias=None if bias_attr else False)
+        self.act = build_activation_layer(act_cfg) if act_cfg is not None else nn.Identity()
+        self.bn = build_norm_layer(norm_cfg, out_channels)[1] \
+            if norm_cfg is not None else nn.Identity()
+
+    def forward(self, x):
+        x = self.conv(x)
+        x = self.bn(x)
+        x = self.act(x)
+        return x


Its function is the same as ConvModule, might use ConvModule instead.

xiexinch · 2023-08-04T09:24:00Z

mmseg/models/backbones/strideformer.py

+class Conv2DBN(nn.Module):
+    def __init__(self,
+                 in_channels,
+                 out_channels,
+                 ks=1,
+                 stride=1,
+                 pad=0,
+                 dilation=1,
+                 groups=1,
+                 ):
+        super().__init__()
+        self.conv_norm = ConvModule(in_channels, out_channels, ks, stride, pad, dilation, groups, False,
+                                    norm_cfg=dict(type='BN'), act_cfg=None)
+
+    def forward(self, inputs):
+        out = self.conv_norm(inputs)
+        return out


It's no need to define this class.

xiexinch · 2023-08-04T09:28:08Z

mmseg/models/backbones/strideformer.py

+        self.conv1 = build_conv_layer(
+            conv_cfg,
+            in_channels=channel,
+            out_channels=channel // reduction,
+            kernel_size=1,
+            stride=1,
+            padding=0)
+        self.relu = build_activation_layer(act_cfg)
+        self.conv2 = build_conv_layer(
+            conv_cfg,
+            in_channels=channel // reduction,
+            out_channels=channel,
+            kernel_size=1,
+            stride=1,
+            padding=0)


Might instead with two ConvModule.

xiexinch · 2023-08-04T09:29:43Z

mmseg/models/backbones/strideformer.py

+class Sea_Attention(nn.Module):
+    def __init__(self,


Might rename to SeaAttention.

xiexinch · 2023-08-04T09:31:26Z

mmseg/models/backbones/strideformer.py

+class Fusion_block(nn.Module):
+    def __init__(self, inp, oup, embed_dim, activations=None) -> None:


Might rename to FusionBlock.

And the names of parameters are not readable, such as inp, oup. It suggests changing to names more readable.

xiexinch · 2023-08-04T09:35:45Z

mmseg/models/backbones/strideformer.py

+
+def _create_act(act):
+    if act == "hardswish":
+        return nn.Hardswish()
+    elif act == "relu":
+        return nn.ReLU()
+    elif act is None:
+        return None
+    else:
+        raise RuntimeError(
+            "The activation function is not supported: {}".format(act))


Can we replace it with build_activation_layer?

build_activation_layer好像没有nn.Hardswish()有这个类

那可以单独写一个包装类，然后注册进去吗，这样写法就比较通用了

xiexinch · 2023-08-04T09:37:42Z

mmseg/models/backbones/strideformer.py

+@MODELS.register_module()
+class MobileSeg_Base(StrideFormer):
+    def __init__(self, **kwargs):
+        super().__init__(**kwargs)
+
+
+@MODELS.register_module()
+class MobileSeg_Tiny(StrideFormer):
+    def __init__(self, **kwargs):
+        super().__init__(**kwargs)


We might add StrideFormer to the MODEL register and delete these two classes.

xiexinch · 2023-08-07T02:18:17Z

mmseg/models/backbones/strideformer.py

+                 mobileV3_cfg,
+                 channels,
+                 embed_dims,
+                 key_dims=[16, 24],
+                 depths=[2, 2],
+                 num_heads=8,
+                 attn_ratios=2,
+                 mlp_ratios=[2, 4],
+                 drop_path_rate=0.1,
+                 act_cfg=dict(type='ReLU'),
+                 inj_type='AAM',
+                 out_channels=256,
+                 dims=(128, 160),
+                 out_feat_chs=None,
+                 stride_attention=True,
+                 pretrained=None,
+                 init_cfg=None


Might add type hints

xiexinch · 2023-08-07T02:18:45Z

mmseg/models/backbones/strideformer.py

+class ResidualUnit(nn.Module):
+    def __init__(self,


And type hints are needed.

xiexinch · 2023-08-07T02:22:21Z

mmseg/models/backbones/strideformer.py

+class SqueezeAxialPositionalEmbedding(nn.Module):
+    def __init__(self, dim, shape):
+        super().__init__()


Might add a docstring.

xiexinch · 2023-08-07T02:22:37Z

mmseg/models/backbones/strideformer.py

+class SeaAttention(nn.Module):
+    def __init__(self,
+                 dim,
+                 key_dim,
+                 num_heads,
+                 attn_ratio=4.,
+                 act_cfg=None,
+                 norm_cfg=dict(type='BN'),
+                 stride_attention=False):


Might add docstring and type hints.

xiexinch · 2023-08-07T02:24:43Z

mmseg/models/backbones/strideformer.py

+class HSigmoid(nn.Module):
+    def __init__(self):
+        super().__init__()
+        self.relu = nn.ReLU6()
+
+    def forward(self, x):
+        return self.relu(x + 3) / 6
+
+
+MODELS.register_module(module=HSigmoid, name='HSigmoid')


Suggested change

class HSigmoid(nn.Module):

def __init__(self):

super().__init__()

self.relu = nn.ReLU6()

def forward(self, x):

return self.relu(x + 3) / 6

MODELS.register_module(module=HSigmoid, name='HSigmoid')

@MODELS.register_module()

class HSigmoid(nn.Module):

def __init__(self):

super().__init__()

self.relu = nn.ReLU6()

def forward(self, x):

return self.relu(x + 3) / 6

xiexinch · 2023-08-07T02:26:12Z

mmseg/models/backbones/strideformer.py

+class hardswish(nn.Module):
+    def __init__(self, inplace=False):
+        super().__init__()
+        self.relu = nn.Hardswish(inplace=inplace)
+
+    def forward(self, x):
+        return self.relu(x)
+
+
+MODELS.register_module(module=hardswish, name='hardswish')


Suggested change

class hardswish(nn.Module):

def __init__(self, inplace=False):

super().__init__()

self.relu = nn.Hardswish(inplace=inplace)

def forward(self, x):

return self.relu(x)

MODELS.register_module(module=hardswish, name='hardswish')

@MODELS.register_module()

class Hardswish(nn.Module):

def __init__(self, inplace=False):

super().__init__()

self.relu = nn.Hardswish(inplace=inplace)

def forward(self, x):

return self.relu(x)

Might add a docstring to explain that it's a wrapper to torch.nn.Hardswish.

xiexinch · 2023-08-07T02:28:52Z

mmseg/models/backbones/strideformer.py

+class Hardsigmoid(nn.Module):
+    def __init__(self, slope=0.2, offset=0.5, inplace=False):
+        super().__init__()
+        self.slope = slope
+        self.offset = offset
+
+    def forward(self, x):
+        return x.mul(self.slope).add(self.offset).clamp(0, 1)
+
+
+MODELS.register_module(module=Hardsigmoid, name='Hardsigmoid')


Suggested change

class Hardsigmoid(nn.Module):

def __init__(self, slope=0.2, offset=0.5, inplace=False):

super().__init__()

self.slope = slope

self.offset = offset

def forward(self, x):

return x.mul(self.slope).add(self.offset).clamp(0, 1)

MODELS.register_module(module=Hardsigmoid, name='Hardsigmoid')

@MODELS.register_module()

class Hardsigmoid(nn.Module):

def __init__(self, slope=0.2, offset=0.5, inplace=False):

super().__init__()

self.slope = slope

self.offset = offset

def forward(self, x):

return x.mul(self.slope).add(self.offset).clamp(0, 1)

Also, we should add a docstring.

xiexinch · 2023-08-07T02:30:00Z

mmseg/models/decode_heads/pp_mobileseg_head.py

+    def __init__(self,
+                 num_classes,
+                 in_channels,
+                 use_dw=True,
+                 dropout_ratio=0.1,
+                 align_corners=False,
+                 upsample='intepolate',
+                 out_channels=None,
+                 conv_cfg=dict(type='Conv'),
+                 act_cfg=dict(type='ReLU'),
+                 norm_cfg=dict(type='BN')


We should add a docstring and type hints.

xiexinch · 2023-08-07T02:48:33Z

mmseg/models/backbones/strideformer.py

+            act_cfg(nn.Layer, optional): The activation layer of AAM.
+            inj_type(string, optional): The type of injection/AAM.


Might add a brief introduction to 'AAM'.

xiexinch · 2023-08-07T02:56:59Z

mmseg/models/backbones/strideformer.py

+@MODELS.register_module()
+class HSigmoid(nn.Module):
+
+    def __init__(self):
+        super().__init__()
+        self.relu = nn.ReLU6()
+
+    def forward(self, x):
+        return self.relu(x + 3) / 6
+


Might use the implementation in mmcv https://github.com/open-mmlab/mmcv/blob/main/mmcv/cnn/bricks/hsigmoid.py#L10

xiexinch · 2023-08-07T02:59:17Z

mmseg/models/backbones/strideformer.py

+@MODELS.register_module()
+class Hardswish(nn.Module):
+
+    def __init__(self, inplace=False):
+        super().__init__()
+        self.relu = nn.Hardswish(inplace=inplace)
+
+    def forward(self, x):
+        return self.relu(x)


https://github.com/open-mmlab/mmcv/blob/main/mmcv/cnn/bricks/hswish.py#L9

Yang-Changhui added 5 commits July 31, 2023 20:12

Create pp_mobileseg_mobilenetv3_2x16_80k_ade20k_512x512_base.py

3c4e111

Add files via upload

1fbac52

Add files via upload

6c3b352

Add files via upload

75c97f3

Add files via upload

4ad772c

Yang-Changhui added 2 commits July 31, 2023 20:33

Add files via upload

00971de

Add files via upload

f791dc4

Yang-Changhui added 4 commits August 4, 2023 16:49

Add files via upload

aaf79c9

Add files via upload

b2ae048

Add files via upload

e4825c4

Add files via upload

46f9164

xiexinch reviewed Aug 4, 2023

View reviewed changes

Yang-Changhui added 5 commits August 5, 2023 14:30

Add files via upload

ba2cd27

Add files via upload

8c4a146

Add files via upload

f9e6fe0

Add files via upload

a03ec9c

Add files via upload

1c368d5

xiexinch reviewed Aug 7, 2023

View reviewed changes

xiexinch changed the title ~~Add pp_mobileseg model 添加 pp_mobileseg 模型~~ [CodeCamp2023-367] Add pp_mobileseg model Aug 7, 2023

xiexinch added 2 commits August 7, 2023 10:36

--other=fix lint

6417747

--other=update register

ced4698

xiexinch reviewed Aug 7, 2023

View reviewed changes

Yang-Changhui and others added 3 commits August 7, 2023 19:34

update

c97ceed

Update README.md

5abc66c

fix lint

f3a03e4

Yang-Changhui added 2 commits August 8, 2023 19:57

Update strideformer.py

f5f7c7a

Update strideformer.py

bcf432a

xiexinch approved these changes Aug 9, 2023

View reviewed changes

--other=update model link

73e979b

xiexinch approved these changes Aug 9, 2023

View reviewed changes

xiexinch merged commit 1e93796 into open-mmlab:dev-1.x Aug 9, 2023
7 checks passed

angiecao pushed a commit to angiecao/mmsegmentation that referenced this pull request Aug 31, 2023

[CodeCamp2023-367] Add pp_mobileseg model (open-mmlab#3239)

759bd7e

emily-lin pushed a commit to emily-lin/mmsegmentation that referenced this pull request Nov 18, 2023

[CodeCamp2023-367] Add pp_mobileseg model (open-mmlab#3239)

aae51c6

nahidnazifi87 pushed a commit to nahidnazifi87/mmsegmentation_playground that referenced this pull request Apr 5, 2024

[CodeCamp2023-367] Add pp_mobileseg model (open-mmlab#3239)

0b67c31

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[CodeCamp2023-367] Add pp_mobileseg model #3239

[CodeCamp2023-367] Add pp_mobileseg model #3239

Yang-Changhui commented Jul 31, 2023

mm-assistant bot commented Jul 31, 2023

CLAassistant commented Jul 31, 2023 •

edited

OpenMMLab-Assistant-004 commented Aug 2, 2023

xiexinch Aug 2, 2023

xiexinch Aug 4, 2023

xiexinch Aug 4, 2023

xiexinch Aug 4, 2023

xiexinch Aug 4, 2023

xiexinch Aug 4, 2023

xiexinch Aug 4, 2023

xiexinch Aug 4, 2023

xiexinch Aug 4, 2023

Yang-Changhui Aug 4, 2023

xiexinch Aug 4, 2023

xiexinch Aug 4, 2023

xiexinch Aug 7, 2023

xiexinch Aug 7, 2023

xiexinch Aug 7, 2023

xiexinch Aug 7, 2023

xiexinch Aug 7, 2023

xiexinch Aug 7, 2023

xiexinch Aug 7, 2023

xiexinch Aug 7, 2023

xiexinch Aug 7, 2023

xiexinch Aug 7, 2023

xiexinch Aug 7, 2023

xiexinch Aug 7, 2023

xiexinch Aug 7, 2023

		class Fusion_block(nn.Module):
		def __init__(self, inp, oup, embed_dim, activations=None) -> None:

		act_cfg(nn.Layer, optional): The activation layer of AAM.
		inj_type(string, optional): The type of injection/AAM.

[CodeCamp2023-367] Add pp_mobileseg model #3239

[CodeCamp2023-367] Add pp_mobileseg model #3239

Conversation

Yang-Changhui commented Jul 31, 2023

Motivation

Modification

BC-breaking (Optional)

Use cases (Optional)

Checklist

mm-assistant bot commented Jul 31, 2023

CLAassistant commented Jul 31, 2023 • edited

OpenMMLab-Assistant-004 commented Aug 2, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

CLAassistant commented Jul 31, 2023 •

edited