[Improvement] Update NasMutator to build search_space in NAS #426

gaoyang07 · 2023-01-06T09:27:35Z

Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers.

Motivation

Please describe the motivation of this PR and the goal you want to achieve through this PR.

Modification

unify mutators for NAS algorithms as the NasMutator;
regard ChannelMutator as pruning-specified;
remove value_mutators & module_mutators;
set GroupMixin only for NAS;
add UT for NasMutator;
fix bugs.

BC-breaking (Optional)

Does the modification introduce changes that break the backward compatibility of the downstream repositories?
If so, please describe how it breaks the compatibility and how the downstream projects should modify their code to keep compatibility with this PR.

Use cases (Optional)

If this PR introduces a new feature, it is better to list some use cases here and update the documentation.

Checklist

Before PR:

Pre-commit or other linting tools are used to fix the potential lint issues.
Bug fixes are fully covered by unit tests, the case that causes the bug should be added in the unit tests.
The modification is covered by complete unit tests. If not, please add more unit tests to ensure the correctness.
The documentation has been modified accordingly, like docstring or example tutorials.

After PR:

If the modification has potential influence on downstream or other related projects, this PR should be tested with those projects, like MMDet or MMSeg.
CLA has been signed and all committers have signed the CLA in this PR.

sunnyxiaohu · 2023-01-10T07:26:59Z

mmrazor/engine/hooks/estimate_resources_hook.py

-        fix_mutable = copied_model.search_subnet()
+        copied_model.set_subnet(copied_model.sample_subnet())
+
+        fix_mutable = export_fix_subnet(copied_model)[0]


Unify fix_mutable to subnet_dict.

sunnyxiaohu · 2023-01-10T07:29:01Z

mmrazor/engine/runner/iteprune_val_loop.py

@@ -39,7 +39,7 @@ def run(self):

    def _save_fix_subnet(self):
        """Save model subnet config."""
-        # TO DO: Modify export_fix_subnet's output. Might contain weight return
+        # TODO: Modify export_fix_subnet's output. Might contain weight return


del this line (already done.)

sunnyxiaohu · 2023-01-10T07:37:24Z

mmrazor/models/algorithms/nas/dsnas.py


        copied_model = copy.deepcopy(self)
-        fix_mutable = copied_model.search_subnet()
+        copied_model.set_subnet(copied_model.sample_subnet())


Unify fix_mutable to subnet_dict

sunnyxiaohu · 2023-01-10T08:06:58Z

mmrazor/models/algorithms/space_mixin.py

+        therefore it currently supports NAS/Pruning algorithms with mutator(s).
+    """
+
+    def _build_search_space(self, prefix=''):


Add necessary hint for users that mutator.prepare_from_supernet has to be called before _build_search_space.

sunnyxiaohu · 2023-01-10T08:38:59Z

mmrazor/models/algorithms/space_mixin.py

+    def sample_subnet(self, kind='random') -> Dict:
+        """Random sample subnet by mutator."""
+        subnet = dict()
+        for name, modules in self.search_space.items():


Unify 'name' and 'group_id' when traversing self.search_space for sample_subnet, set_subnet, max_subnet and so on.

1. unify mutators for NAS algorithms as the NasMutator; 2. regard ChannelMutator as pruning-specified; 3. remove value_mutators & module_mutators; 4. set GroupMixin only for NAS; 5. revert all changes in ChannelMutator.

sunnyxiaohu · 2023-01-28T06:02:26Z

configs/pruning/mmcls/dcff/dcff_compact_resnet_8xb32_in1k.py

@@ -6,7 +6,6 @@
    type='sub_model',
    cfg=dict(
        cfg_path='mmcls::resnet/resnet50_8xb32_in1k.py', pretrained=False),
-    fix_subnet='configs/pruning/mmcls/dcff/fix_subnet.json',


sunnyxiaohu · 2023-01-28T06:02:46Z

configs/pruning/mmdet/dcff/dcff_compact_faster_rcnn_resnet50_8xb4_coco.py

@@ -5,7 +5,6 @@
    _scope_='mmrazor',
    type='sub_model',
    cfg=_base_.architecture,
-    fix_subnet='configs/pruning/mmdet/dcff/fix_subnet.json',


sunnyxiaohu · 2023-01-28T06:02:52Z

configs/pruning/mmpose/dcff/dcff_compact_topdown_heatmap_resnet50_coco.py

@@ -5,7 +5,6 @@
    _scope_='mmrazor',
    type='sub_model',
    cfg=_base_.architecture,
-    fix_subnet='configs/pruning/mmpose/dcff/fix_subnet.json',


sunnyxiaohu · 2023-01-28T06:03:11Z

configs/pruning/mmseg/dcff/dcff_compact_pointrend_resnet50_8xb2_cityscapes.py

@@ -5,7 +5,6 @@
    _scope_='mmrazor',
    type='sub_model',
    cfg=_base_.architecture,
-    fix_subnet='configs/pruning/mmseg/dcff/fix_subnet.json',


sunnyxiaohu · 2023-01-28T06:12:45Z

mmrazor/models/algorithms/nas/autoformer.py

 from mmrazor.registry import MODELS
 from mmrazor.utils import ValidFixMutable
 from ..base import BaseAlgorithm, LossResults

+VALID_MUTATOR_TYPE = Union[BaseMutator, Dict]


Use unify NasMutator for autoformer

sunnyxiaohu · 2023-01-28T06:24:18Z

mmrazor/structures/subnet/fix_subnet.py

+        from mmrazor.models.mutators import ChannelMutator
+
+        copied_model = copy.deepcopy(model)
+        if isinstance(model.mutator, ChannelMutator):


It seems that all conditions could be handled by _dynamic_to_static.

sunnyxiaohu · 2023-01-28T06:31:20Z

mmrazor/models/mutators/nas_mutator.py

+    def set_choices(self, choices: Dict) -> None:
+        """Set choices for each mutable in search space."""
+        for name, mutables in self.search_groups.items():
+            if name not in choices:


alias have been already handled in GroupMixin when building search_space.

sunnyxiaohu · 2023-01-28T06:37:51Z

mmrazor/models/architectures/utils/mutable_register.py

+        mutable_expand_ratio2 = copy.deepcopy(mutable_expand_ratio)
+        mutable_expand_ratio2.alias += '_se'
+
+        derived_se_channels = mutable_expand_ratio2 * mutable_in_channels


Changes for what?

To avoid the repeat of alias when it comes to fine_grained_mode.

sunnyxiaohu · 2023-02-01T13:05:50Z

mmrazor/engine/runner/evolution_search_loop.py

@@ -129,8 +130,7 @@ def __init__(self,
        self.predictor_cfg = predictor_cfg
        if self.predictor_cfg is not None:
            self.predictor_cfg['score_key'] = self.score_key
-            self.predictor_cfg['search_groups'] = \
-                self.model.mutator.search_groups
+            self.predictor_cfg['search_groups'] = self.model.search_space


sunnyxiaohu · 2023-02-01T13:29:35Z

mmrazor/structures/subnet/fix_subnet.py

+        copied_model = copy.deepcopy(model)
+        if hasattr(model, 'mutator') and \
+                isinstance(model.mutator, ChannelMutator):
+            _dynamic_to_static(copied_model)


The two conditions could be summarized by _dynamic_to_static?

sunnyxiaohu · 2023-02-01T13:32:33Z

mmrazor/models/algorithms/nas/dsnas.py

+                # NOTE: with `ModuleMutable` as mutable, the keys in
+                # self.mutator.arch_params must contain the prefix `module`.
+                # See `prepare_from_supernet` in `NasMutator` for details.
+                probs = F.softmax(self.mutator.arch_params['module_' + str(k)],


replace fixed string ‘module_’ with mutator.search_groups.items()[0].mutable_prefix

sunnyxiaohu · 2023-02-01T13:32:43Z

mmrazor/models/algorithms/nas/dsnas.py

+                # NOTE: with `ModuleMutable` as mutable, the keys in
+                # self.mutator.arch_params must contain the prefix `module`.
+                # See `prepare_from_supernet` in `NasMutator` for details.
+                self.mutator.arch_params['module_' + str(k)].grad.data.mul_(


replace fixed string ‘module_’ with mutator.search_groups.items()[0].mutable_prefix

codecov · 2023-02-01T14:49:52Z

Codecov Report

Base: 80.24% // Head: 79.97% // Decreases project coverage by -0.27% ⚠️

Coverage data is based on head (9a8b0d8) compared to base (1c47009).
Patch coverage: 88.03% of modified lines in pull request are covered.

Additional details and impacted files

@@             Coverage Diff             @@
##           dev-1.x     #426      +/-   ##
===========================================
- Coverage    80.24%   79.97%   -0.27%     
===========================================
  Files          256      251       -5     
  Lines        12748    12821      +73     
  Branches      1943     1989      +46     
===========================================
+ Hits         10229    10254      +25     
- Misses        2113     2156      +43     
- Partials       406      411       +5

Flag	Coverage Δ
unittests	`79.97% <88.03%> (-0.27%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
mmrazor/engine/hooks/estimate_resources_hook.py	`36.17% <0.00%> (-0.79%)`	⬇️
mmrazor/engine/runner/iteprune_val_loop.py	`34.48% <0.00%> (-1.24%)`	⬇️
mmrazor/engine/runner/subnet_val_loop.py	`23.63% <0.00%> (ø)`
...zor/models/algorithms/pruning/slimmable_network.py	`96.33% <ø> (ø)`
...mrazor/models/distillers/configurable_distiller.py	`93.47% <ø> (ø)`
mmrazor/engine/hooks/dump_subnet_hook.py	`35.00% <25.00%> (-0.30%)`	⬇️
...bles/mutable_channel/units/mutable_channel_unit.py	`91.09% <66.66%> (-0.52%)`	⬇️
mmrazor/models/algorithms/nas/dsnas.py	`85.00% <71.87%> (-0.17%)`	⬇️
mmrazor/models/algorithms/nas/bignas.py	`37.22% <73.91%> (-1.83%)`	⬇️
...r/models/mutables/mutable_module/mutable_module.py	`94.28% <77.77%> (-5.72%)`	⬇️
... and 33 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

* fix ci * add nvidia key * remote torch * recover pytorch

* docs(docs/zh_cn): add doc and link checker * docs(REAME): update * docs(docs/zh_cn): update * docs(benchmark): update table * docs(zh_cn/benchmark): update link * CI(docs): update link check * ci(doc): update checker * docs(zh_cn): update * style(ci): remove useless para * style(ci): update * docs(zh_cn): update * docs(benchmark.md): fix mobilnet link error * docs(docs/zh_cn): add doc and link checker * docs(REAME): update * docs(docs/zh_cn): update * docs(benchmark): update table * docs(zh_cn/benchmark): update link * CI(docs): update link check * ci(doc): update checker * docs(zh_cn): update * style(ci): remove useless para * style(ci): update * docs(zh_cn): update * docs(benchmark.md): fix mobilnet link error * docs(zh_cn/do_regression_test.md): rebase * docs(docs/zh_cn): add doc and link checker * Update README_zh-CN.md * Update README_zh-CN.md * Update index.rst * Update check-doc-link.yml * [Fix] Fix ci (open-mmlab#426) * fix ci * add nvidia key * remote torch * recover pytorch * ci(codecov): ignore ci * docs(zh_cn): add get_started.md * docs(zh_cn): fix review advice * docs(readthedocs): update * docs(zh_CN): update * docs(zh_CN): revert * fix(docs): review advices * fix(docs): review advices * fix(docs): review Co-authored-by: q.yao <streetyao@live.com>

* refactor(onnx2ncnn.cpp): split it to shape_inference, pass and utils * refactor(onnx2ncnn.cpp): split it to shape_inference, pass and utils * refactor(onnx2ncnn.cpp): split code * refactor(net_module.cpp): fix build error * ci(test_onnx2ncnn.py): add generate model adn run * ci(onnx2ncnn): add ncnn backend * ci(test_onnx2ncnn): add converted onnx model` * ci(onnx2ncnn): fix ncnn tar * ci(backed-ncnn): simplify dependency install * ci(onnx2ncnn): fix apt install * Update backend-ncnn.yml * Update backend-ncnn.yml * Update backend-ncnn.yml * Update backend-ncnn.yml * Update backend-ncnn.yml * Update backend-ncnn.yml * Update backend-ncnn.yml * Update backend-ncnn.yml * Update backend-ncnn.yml * Update backend-ncnn.yml * Update backend-ncnn.yml * fix(ci): add include algorithm * Update build.yml * parent aa85760 author q.yao <streetyao@live.com> 1651287879 +0800 committer tpoisonooo <khj.application@aliyun.com> 1652169959 +0800 [Fix] Fix ci (open-mmlab#426) * fix ci * add nvidia key * remote torch * recover pytorch refactor(onnx2ncnn.cpp): split it to shape_inference, pass and utils * fix(onnx2ncnn): review * fix(onnx2ncnn): build error Co-authored-by: q.yao <streetyao@live.com>

gaoyang07 added 2 commits January 6, 2023 16:10

update space_mixin

b8aa9fc

update NAS algorithms with SpaceMixin

e20da2f

sunnyxiaohu self-requested a review January 9, 2023 10:44

gaoyang07 added 2 commits January 9, 2023 18:57

update pruning algorithms with SpaceMixin

a8d6e17

fix ut

675cb37

sunnyxiaohu reviewed Jan 10, 2023

View reviewed changes

gaoyang07 added 5 commits January 10, 2023 20:19

fix comments

255c3df

revert _load_fix_subnet_by_mutator

120faca

fix dcff test

fc8e820

add ut for registry

02de63d

update autoslim_greedy_search

7866460

gaoyang07 force-pushed the gy/search_space branch from 90f2da5 to 7866460 Compare January 13, 2023 07:30

gaoyang07 and others added 6 commits January 16, 2023 16:58

fix repeat-mutables bug

e7ae4e9

fix slice_weight in export_fix_subnet

f7902b6

Update NasMutator:

477cf38

1. unify mutators for NAS algorithms as the NasMutator; 2. regard ChannelMutator as pruning-specified; 3. remove value_mutators & module_mutators; 4. set GroupMixin only for NAS; 5. revert all changes in ChannelMutator.

update NAS algorithms using NasMutator

bba830e

update channel mutator

671aedc

update one_shot_channel_mutator

6b1966b

sunnyxiaohu reviewed Jan 28, 2023

View reviewed changes

gaoyang07 added 3 commits January 29, 2023 17:09

fix comments

645c70e

update UT for NasMutator

bf567f9

fix isort version

8fdcf09

gaoyang07 force-pushed the gy/search_space branch from 4764e25 to 9ba66eb Compare February 1, 2023 10:14

gaoyang07 changed the title ~~[Improvement] Unify search_groups in mutators as search_space~~ [Improvement] Update NasMutator to build search_space in NAS Feb 1, 2023

gaoyang07 force-pushed the gy/search_space branch from 9ba66eb to 8fdcf09 Compare February 1, 2023 10:34

sunnyxiaohu approved these changes Feb 1, 2023

View reviewed changes

fix comments

9a8b0d8

sunnyxiaohu merged commit a27952d into dev-1.x Feb 1, 2023

sunnyxiaohu deleted the gy/search_space branch February 1, 2023 14:51

humu789 pushed a commit to humu789/mmrazor that referenced this pull request Feb 13, 2023

[Fix] Fix ci (open-mmlab#426)

59453dc

* fix ci * add nvidia key * remote torch * recover pytorch

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Improvement] Update NasMutator to build search_space in NAS #426

[Improvement] Update NasMutator to build search_space in NAS #426

gaoyang07 commented Jan 6, 2023 •

edited

Loading

sunnyxiaohu Jan 10, 2023

sunnyxiaohu Jan 10, 2023

sunnyxiaohu Jan 10, 2023

sunnyxiaohu Jan 10, 2023

sunnyxiaohu Jan 10, 2023

sunnyxiaohu Jan 28, 2023

sunnyxiaohu Jan 28, 2023

sunnyxiaohu Jan 28, 2023

sunnyxiaohu Jan 28, 2023

sunnyxiaohu Jan 28, 2023

sunnyxiaohu Jan 28, 2023

sunnyxiaohu Jan 28, 2023

sunnyxiaohu Jan 28, 2023

gaoyang07 Jan 29, 2023

sunnyxiaohu Feb 1, 2023

sunnyxiaohu Feb 1, 2023

sunnyxiaohu Feb 1, 2023

sunnyxiaohu Feb 1, 2023

codecov bot commented Feb 1, 2023

[Improvement] Update NasMutator to build search_space in NAS #426

[Improvement] Update NasMutator to build search_space in NAS #426

Conversation

gaoyang07 commented Jan 6, 2023 • edited Loading

Motivation

Modification

BC-breaking (Optional)

Use cases (Optional)

Checklist

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Feb 1, 2023

Codecov Report

gaoyang07 commented Jan 6, 2023 •

edited

Loading