Support deepcopy for Config #1658

mzr1996 · 2022-01-11T04:32:53Z

Motivation

The original Config doesn't support deep copy because it will call the __deepcopy__ method of ConfigDict.

Modification

Add __deepcopy__ to Config.

BC-breaking (Optional)

No

Use cases (Optional)

Before the PR

>>> from mmcv import Config
>>> from copy import deepcopy
>>> cfg = Config.fromfile("./tests/data/config/a.py")
>>> new_cfg = deepcopy(cfg)
>>> type(cfg) == type(new_cfg)
False
>>> type(cfg), type(new_cfg)
(mmcv.utils.config.Config, mmcv.utils.config.ConfigDict)

After the PR

>>> from mmcv import Config
>>> from copy import deepcopy
>>> cfg = Config.fromfile("./tests/data/config/a.py")
>>> new_cfg = deepcopy(cfg)
>>> type(cfg) == type(new_cfg)
True
>>> type(cfg), type(new_cfg)
(mmcv.utils.config.Config, mmcv.utils.config.Config)
>>> print(cfg._cfg_dict == new_cfg._cfg_dict)
True
>>> print(cfg._cfg_dict is new_cfg._cfg_dict)
False

Checklist

Before PR:

I have read and followed the workflow indicated in the CONTRIBUTING.md to create this PR.
Pre-commit or linting tools indicated in CONTRIBUTING.md are used to fix the potential lint issues.
Bug fixes are covered by unit tests, the case that causes the bug should be added in the unit tests.
New functionalities are covered by complete unit tests. If not, please add more unit test to ensure the correctness.
The documentation has been modified accordingly, including docstring or example tutorials.

After PR:

If the modification has potential influence on downstream or other related projects, this PR should be tested with some of those projects, like MMDet or MMCls.
CLA has been signed and all committers have signed the CLA in this PR.

mmcv/utils/config.py

zhouzaida · 2022-01-27T16:40:24Z

The PR needs to merge the upstream master to fix the CI.

zhouzaida · 2022-01-28T15:24:58Z

Will it cause BC breaking? Take the following example.

>>> from mmcv import Config,  ConfigDict
>>> from copy import deepcopy
>>> cfg = Config.fromfile("./tests/data/config/a.py")
# before the PR
>>> isinstance(cfg, ConfigDict)
True
# after the PR
>>> isinstance(cfg, ConfigDict)
False

mzr1996 · 2022-01-29T06:49:11Z

Will it cause BC breaking? Take the following example.

>>> from mmcv import Config,  ConfigDict
>>> from copy import deepcopy
>>> cfg = Config.fromfile("./tests/data/config/a.py")
# before the PR
>>> isinstance(cfg, ConfigDict)
True
# after the PR
>>> isinstance(cfg, ConfigDict)
False

I can't get your example. Before this PR, the return value of isinstance(cfg, ConfigDict) is also False

mzr1996 · 2022-01-29T06:53:17Z

Will it cause BC breaking? Take the following example.

>>> from mmcv import Config,  ConfigDict
>>> from copy import deepcopy
>>> cfg = Config.fromfile("./tests/data/config/a.py")
# before the PR
>>> isinstance(cfg, ConfigDict)
True
# after the PR
>>> isinstance(cfg, ConfigDict)
False

If your meaning is the value of isinstance(deepcopy(cfg), ConfigDict) is changed, the problem is, do we want to get a ConfigDict after deepcopy it? I don't think it's a intuitive behavior.

zhouzaida · 2022-01-29T07:15:28Z

Will it cause BC breaking? Take the following example.
>>> from mmcv import Config,  ConfigDict
>>> from copy import deepcopy
>>> cfg = Config.fromfile("./tests/data/config/a.py")
# before the PR
>>> isinstance(cfg, ConfigDict)
True
# after the PR
>>> isinstance(cfg, ConfigDict)
False
I can't get your example. Before this PR, the return value of isinstance(cfg, ConfigDict) is also False

Sorry for my negligence. The example should be the following one.

>>> from mmcv import Config,  ConfigDict
>>> from copy import deepcopy
>>> cfg = Config.fromfile("./tests/data/config/a.py")
>>> new_cfg = deepcopy(cfg)
# before the PR
>>> isinstance(new_cfg, ConfigDict)
True
# after the PR
>>> isinstance(new_cfg, ConfigDict)
False

zhouzaida · 2022-01-29T07:19:41Z

Will it cause BC breaking? Take the following example.
>>> from mmcv import Config,  ConfigDict
>>> from copy import deepcopy
>>> cfg = Config.fromfile("./tests/data/config/a.py")
# before the PR
>>> isinstance(cfg, ConfigDict)
True
# after the PR
>>> isinstance(cfg, ConfigDict)
False
If your meaning is the value of isinstance(deepcopy(cfg), ConfigDict) is changed, the problem is, do we want to get a ConfigDict after deepcopy it? I don't think it's a intuitive behavior.

isinstance(deepcopy(cfg), ConfigDict) is True is not a behavior that meets our expectations.

mmcv/utils/config.py

tests/test_utils/test_config.py

zhouzaida · 2022-02-12T06:48:57Z

Should we also support the copy?

ZwwWayne · 2022-02-16T14:51:04Z

Should we also support the copy?

Let‘s do it in a new PR.

* Support deepcopy for Config (#1658) * Support deepcopy for Config * Iterate the `__dict__` of Config directly. * Use __new__ to avoid unnecessary initialization. * Improve according to comments * [Feature] Add spconv ops from mmdet3d (#1581) * add ops (spconv) of mmdet3d * fix typo * refactor code * resolve comments in #1452 * fix compile error * fix bugs * fix bug * transform from 'types.h' to 'extension.h' * fix bug * transform from 'types.h' to 'extension.h' in parrots * add extension.h in pybind.cpp * add unittest * Recover code * (1) Remove prettyprint.h (2) Switch `T` to `scalar_t` (3) Remove useless lines (4) Refine example in docstring of sparse_modules.py * (1) rename from `cu.h` to `cuh` (2) remove useless files (3) move cpu files to `pytorch/cpu` * reorganize files * Add docstring for sparse_functional.py * use dispatcher * remove template * use dispatch in cuda ops * resolve Segmentation fault * remove useless files * fix lint * fix lint * fix lint * fix unittest in test_build_layers.py * add tensorview into include_dirs when compiling * recover all deleted files * fix lint and comments * recover setup.py * replace tv::GPU as tv::TorchGPU & support device guard * fix lint Co-authored-by: hdc <hudingchang.vendor@sensetime.com> Co-authored-by: grimoire <yaoqian@sensetime.com> * Imporve the docstring of imfrombytes and fix a deprecation-warning (#1731) * [Refactor] Refactor the interface for RoIAlignRotated (#1662) * fix interface for RoIAlignRotated * Add a unit test for RoIAlignRotated * Make a unit test for RoIAlignRotated concise * fix interface for RoIAlignRotated * Refactor ext_module.nms_rotated * Lint cpp files * add transforms * add invoking time check for cacheable methods * fix lint * add unittest * fix bug in non-strict input mapping * fix ci * fix ci * fix compatibility with python<3.9 * fix typing compatibility * fix import * fix typing * add alternative for nullcontext * fix import * fix import * add docstrings * add docstrings * fix callable check * resolve comments * fix lint * enrich unittest cases * fix lint * fix unittest Co-authored-by: Ma Zerun <mzr1996@163.com> Co-authored-by: Wenhao Wu <79644370+wHao-Wu@users.noreply.github.com> Co-authored-by: hdc <hudingchang.vendor@sensetime.com> Co-authored-by: grimoire <yaoqian@sensetime.com> Co-authored-by: Jiazhen Wang <47851024+teamwong111@users.noreply.github.com> Co-authored-by: Hakjin Lee <nijkah@gmail.com>

* Support deepcopy for Config (open-mmlab#1658) * Support deepcopy for Config * Iterate the `__dict__` of Config directly. * Use __new__ to avoid unnecessary initialization. * Improve according to comments * [Feature] Add spconv ops from mmdet3d (open-mmlab#1581) * add ops (spconv) of mmdet3d * fix typo * refactor code * resolve comments in open-mmlab#1452 * fix compile error * fix bugs * fix bug * transform from 'types.h' to 'extension.h' * fix bug * transform from 'types.h' to 'extension.h' in parrots * add extension.h in pybind.cpp * add unittest * Recover code * (1) Remove prettyprint.h (2) Switch `T` to `scalar_t` (3) Remove useless lines (4) Refine example in docstring of sparse_modules.py * (1) rename from `cu.h` to `cuh` (2) remove useless files (3) move cpu files to `pytorch/cpu` * reorganize files * Add docstring for sparse_functional.py * use dispatcher * remove template * use dispatch in cuda ops * resolve Segmentation fault * remove useless files * fix lint * fix lint * fix lint * fix unittest in test_build_layers.py * add tensorview into include_dirs when compiling * recover all deleted files * fix lint and comments * recover setup.py * replace tv::GPU as tv::TorchGPU & support device guard * fix lint Co-authored-by: hdc <hudingchang.vendor@sensetime.com> Co-authored-by: grimoire <yaoqian@sensetime.com> * Imporve the docstring of imfrombytes and fix a deprecation-warning (open-mmlab#1731) * [Refactor] Refactor the interface for RoIAlignRotated (open-mmlab#1662) * fix interface for RoIAlignRotated * Add a unit test for RoIAlignRotated * Make a unit test for RoIAlignRotated concise * fix interface for RoIAlignRotated * Refactor ext_module.nms_rotated * Lint cpp files * add transforms * add invoking time check for cacheable methods * fix lint * add unittest * fix bug in non-strict input mapping * fix ci * fix ci * fix compatibility with python<3.9 * fix typing compatibility * fix import * fix typing * add alternative for nullcontext * fix import * fix import * add docstrings * add docstrings * fix callable check * resolve comments * fix lint * enrich unittest cases * fix lint * fix unittest Co-authored-by: Ma Zerun <mzr1996@163.com> Co-authored-by: Wenhao Wu <79644370+wHao-Wu@users.noreply.github.com> Co-authored-by: hdc <hudingchang.vendor@sensetime.com> Co-authored-by: grimoire <yaoqian@sensetime.com> Co-authored-by: Jiazhen Wang <47851024+teamwong111@users.noreply.github.com> Co-authored-by: Hakjin Lee <nijkah@gmail.com>

* Support deepcopy for Config (#1658) * Support deepcopy for Config * Iterate the `__dict__` of Config directly. * Use __new__ to avoid unnecessary initialization. * Improve according to comments * [Feature] Add spconv ops from mmdet3d (#1581) * add ops (spconv) of mmdet3d * fix typo * refactor code * resolve comments in #1452 * fix compile error * fix bugs * fix bug * transform from 'types.h' to 'extension.h' * fix bug * transform from 'types.h' to 'extension.h' in parrots * add extension.h in pybind.cpp * add unittest * Recover code * (1) Remove prettyprint.h (2) Switch `T` to `scalar_t` (3) Remove useless lines (4) Refine example in docstring of sparse_modules.py * (1) rename from `cu.h` to `cuh` (2) remove useless files (3) move cpu files to `pytorch/cpu` * reorganize files * Add docstring for sparse_functional.py * use dispatcher * remove template * use dispatch in cuda ops * resolve Segmentation fault * remove useless files * fix lint * fix lint * fix lint * fix unittest in test_build_layers.py * add tensorview into include_dirs when compiling * recover all deleted files * fix lint and comments * recover setup.py * replace tv::GPU as tv::TorchGPU & support device guard * fix lint Co-authored-by: hdc <hudingchang.vendor@sensetime.com> Co-authored-by: grimoire <yaoqian@sensetime.com> * Imporve the docstring of imfrombytes and fix a deprecation-warning (#1731) * [Refactor] Refactor the interface for RoIAlignRotated (#1662) * fix interface for RoIAlignRotated * Add a unit test for RoIAlignRotated * Make a unit test for RoIAlignRotated concise * fix interface for RoIAlignRotated * Refactor ext_module.nms_rotated * Lint cpp files * add transforms * add invoking time check for cacheable methods * fix lint * add unittest * fix bug in non-strict input mapping * fix ci * fix ci * fix compatibility with python<3.9 * fix typing compatibility * fix import * fix typing * add alternative for nullcontext * fix import * fix import * add docstrings * add docstrings * fix callable check * resolve comments * fix lint * enrich unittest cases * fix lint * fix unittest Co-authored-by: Ma Zerun <mzr1996@163.com> Co-authored-by: Wenhao Wu <79644370+wHao-Wu@users.noreply.github.com> Co-authored-by: hdc <hudingchang.vendor@sensetime.com> Co-authored-by: grimoire <yaoqian@sensetime.com> Co-authored-by: Jiazhen Wang <47851024+teamwong111@users.noreply.github.com> Co-authored-by: Hakjin Lee <nijkah@gmail.com>

* Support deepcopy for Config (open-mmlab#1658) * Support deepcopy for Config * Iterate the `__dict__` of Config directly. * Use __new__ to avoid unnecessary initialization. * Improve according to comments * [Feature] Add spconv ops from mmdet3d (open-mmlab#1581) * add ops (spconv) of mmdet3d * fix typo * refactor code * resolve comments in open-mmlab#1452 * fix compile error * fix bugs * fix bug * transform from 'types.h' to 'extension.h' * fix bug * transform from 'types.h' to 'extension.h' in parrots * add extension.h in pybind.cpp * add unittest * Recover code * (1) Remove prettyprint.h (2) Switch `T` to `scalar_t` (3) Remove useless lines (4) Refine example in docstring of sparse_modules.py * (1) rename from `cu.h` to `cuh` (2) remove useless files (3) move cpu files to `pytorch/cpu` * reorganize files * Add docstring for sparse_functional.py * use dispatcher * remove template * use dispatch in cuda ops * resolve Segmentation fault * remove useless files * fix lint * fix lint * fix lint * fix unittest in test_build_layers.py * add tensorview into include_dirs when compiling * recover all deleted files * fix lint and comments * recover setup.py * replace tv::GPU as tv::TorchGPU & support device guard * fix lint Co-authored-by: hdc <hudingchang.vendor@sensetime.com> Co-authored-by: grimoire <yaoqian@sensetime.com> * Imporve the docstring of imfrombytes and fix a deprecation-warning (open-mmlab#1731) * [Refactor] Refactor the interface for RoIAlignRotated (open-mmlab#1662) * fix interface for RoIAlignRotated * Add a unit test for RoIAlignRotated * Make a unit test for RoIAlignRotated concise * fix interface for RoIAlignRotated * Refactor ext_module.nms_rotated * Lint cpp files * add transforms * add invoking time check for cacheable methods * fix lint * add unittest * fix bug in non-strict input mapping * fix ci * fix ci * fix compatibility with python<3.9 * fix typing compatibility * fix import * fix typing * add alternative for nullcontext * fix import * fix import * add docstrings * add docstrings * fix callable check * resolve comments * fix lint * enrich unittest cases * fix lint * fix unittest Co-authored-by: Ma Zerun <mzr1996@163.com> Co-authored-by: Wenhao Wu <79644370+wHao-Wu@users.noreply.github.com> Co-authored-by: hdc <hudingchang.vendor@sensetime.com> Co-authored-by: grimoire <yaoqian@sensetime.com> Co-authored-by: Jiazhen Wang <47851024+teamwong111@users.noreply.github.com> Co-authored-by: Hakjin Lee <nijkah@gmail.com>

mzr1996 requested a review from zhouzaida January 11, 2022 04:32

zhouzaida reviewed Jan 23, 2022

View reviewed changes

mmcv/utils/config.py Outdated Show resolved Hide resolved

mzr1996 added 2 commits January 28, 2022 11:21

Support deepcopy for Config

9f76486

Iterate the __dict__ of Config directly.

7b81416

mzr1996 force-pushed the copy-config branch from 0c1783d to 7b81416 Compare January 28, 2022 03:22

zhouzaida reviewed Feb 2, 2022

View reviewed changes

mmcv/utils/config.py Outdated Show resolved Hide resolved

Use __new__ to avoid unnecessary initialization.

105a342

zhouzaida reviewed Feb 7, 2022

View reviewed changes

tests/test_utils/test_config.py Show resolved Hide resolved

Improve according to comments

1f123ed

zhouzaida approved these changes Feb 8, 2022

View reviewed changes

zhouzaida requested a review from gaotongxiao February 8, 2022 04:55

MeowZheng approved these changes Feb 16, 2022

View reviewed changes

ZwwWayne merged commit 33c83b5 into open-mmlab:master Feb 16, 2022

imabackstabber mentioned this pull request Mar 12, 2022

[Feature] Support shallow copy for Config #1796

Merged

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support deepcopy for Config #1658

Support deepcopy for Config #1658

mzr1996 commented Jan 11, 2022 •

edited by imabackstabber

zhouzaida commented Jan 27, 2022 •

edited

zhouzaida commented Jan 28, 2022

mzr1996 commented Jan 29, 2022

mzr1996 commented Jan 29, 2022

zhouzaida commented Jan 29, 2022 •

edited

zhouzaida commented Jan 29, 2022

zhouzaida commented Feb 12, 2022

ZwwWayne commented Feb 16, 2022

Support deepcopy for Config #1658

Support deepcopy for Config #1658

Conversation

mzr1996 commented Jan 11, 2022 • edited by imabackstabber

Motivation

Modification

BC-breaking (Optional)

Use cases (Optional)

Checklist

zhouzaida commented Jan 27, 2022 • edited

zhouzaida commented Jan 28, 2022

mzr1996 commented Jan 29, 2022

mzr1996 commented Jan 29, 2022

zhouzaida commented Jan 29, 2022 • edited

zhouzaida commented Jan 29, 2022

zhouzaida commented Feb 12, 2022

ZwwWayne commented Feb 16, 2022

mzr1996 commented Jan 11, 2022 •

edited by imabackstabber

zhouzaida commented Jan 27, 2022 •

edited

zhouzaida commented Jan 29, 2022 •

edited