[1.x] Move block.optimize_for backend_opts to kwargs #19386

Kh4L · 2020-10-20T03:46:26Z

Description

symbol.optimize_for backend options are given as kwargs but the block.optimize_for ones are given as a separate backend_opts dict.

We end up with an API discrepancy, than can be confusing for the user. Here's an exemple:

Symbol

sym.optimize_for(backend, args, reqArgs=True, dedup_subgraph=True)

Block (Gluon)

blk.optimize_for(x, backend="addInputPass", clear=False, backend_opts={'dedup_subgraph':True})

This limitation was due to preserve the kwargs from hybridize.
However this kwargs is virtually only used to forward the static_alloc and static_shape arguments.

Change

This PR factors out static_alloc and static_shape and changes the kwargs, in both optimize_for and hybridize.

Example after the change:

blk.optimize_for(x, backend="addInputPass", clear=False, backend_opts={'dedup_subgraph':True})

will be now called as

blk.optimize_for(x, backend="addInputPass", clear=False, dedup_subgraph=True)

Note: if needed in the future, we can add an extra_hybridize_kwargs.

mxnet-bot · 2020-10-20T03:46:30Z

Hey @Kh4L , Thanks for submitting the PR
All tests are already queued to run once. If tests fail, you can trigger one or more tests again with the following commands:

To trigger all jobs: @mxnet-bot run ci [all]
To trigger specific jobs: @mxnet-bot run ci [job1, job2]

CI supported jobs: [edge, unix-gpu, unix-cpu, windows-gpu, miscellaneous, sanity, centos-cpu, centos-gpu, clang, website, windows-cpu]

Note:
Only following 3 categories can trigger CI :PR Author, MXNet Committer, Jenkins Admin.
All CI tests must pass before the PR can be merged.

Signed-off-by: Serge Panev <spanev@nvidia.com>

Kh4L · 2020-10-30T00:04:00Z

@mxnet-bot run ci [edge, unix-cpu]

mxnet-bot · 2020-10-30T00:04:08Z

Jenkins CI successfully triggered : [edge, unix-cpu]

samskalicky · 2020-11-04T02:45:53Z

python/mxnet/gluon/block.py

@@ -1059,7 +1059,7 @@ def _call_cached_op(self, *args):
            out = [out]
        return _regroup(out, self._out_format)

-    def optimize_for(self, x, *args, backend=None, backend_opts=None, clear=True, **kwargs):
+    def optimize_for(self, x, *args, backend=None, clear=True, static_alloc=False, static_shape=False, **kwargs):


I like this, can we do the same for hybridize and eliminate backend_opts altogether?
Change:

def hybridize(self, active=True, backend=None, backend_opts=None, clear=True, **kwargs):

To:

def hybridize(self, active=True, backend=None, clear=True, static_alloc=False, static_shape=False, **kwargs):

example/extensions/lib_subgraph/README.md

example/extensions/lib_subgraph/test_subgraph.py

python/mxnet/gluon/block.py

Signed-off-by: Serge Panev <spanev@nvidia.com>

samskalicky · 2020-11-14T06:00:58Z

python/mxnet/gluon/block.py

-        self.hybridize(True, backend, backend_opts, clear, **kwargs)
+        if clear or not self._active:
+            self.hybridize(True, backend, clear, static_alloc, static_shape,
+                           inline_limit, forward_bulk_size, backward_bulk_size, kwargs)


we dont need to pass kwargs here since we just set them above

Signed-off-by: Serge Panev <spanev@nvidia.com>

samskalicky

Thanks @Kh4L for all the iterations! This is huge simplification in user experience!

mseth10 · 2020-11-14T06:36:48Z

python/mxnet/gluon/block.py

        if clear:
            self._clear_cached_op()
        if active and self._forward_hooks or self._forward_pre_hooks:
            warnings.warn('"{block}" is being hybridized while still having forward hook/pre-hook. '
                          'If "{block}" is a child of HybridBlock, the hooks will not take effect.'
                          .format(block=self))
-        super(HybridBlock, self).hybridize(active, **kwargs)
+        super(HybridBlock, self).hybridize(active,


Should we explicitly specify args for Block.hybridize as well now that we have done it here?
https://github.com/apache/incubator-mxnet/blob/09d0cc8418cddefddf3f03aeeb1cbeb1fd4cbafa/python/mxnet/gluon/block.py#L658-L662

no we can leave this as-is for the Block class since it doesnt really care about what the kwargs are anyway, its just going to pass them all recursively to its children. It would also be a 2nd place were the args would be duplicated for no benefit, creating a maintenance issue.

example/extensions/lib_subgraph/README.md

mseth10

LGTM. Thanks for simplifying the API @Kh4L

Signed-off-by: Serge Panev <spanev@nvidia.com>

* Move block.optimize_for backend_opts to kwargs Signed-off-by: Serge Panev <spanev@nvidia.com> * Update Hybridize to use kwargs as backend opts Signed-off-by: Serge Panev <spanev@nvidia.com> * Fix lint Signed-off-by: Serge Panev <spanev@nvidia.com> * Change clear default to False and allow hybrize+optimize_for calls Signed-off-by: Serge Panev <spanev@nvidia.com> * Fix nit Signed-off-by: Serge Panev <spanev@nvidia.com> * Adress review comments Signed-off-by: Serge Panev <spanev@nvidia.com> * Adress more review comments Signed-off-by: Serge Panev <spanev@nvidia.com> * Adress more more review comments Signed-off-by: Serge Panev <spanev@nvidia.com> * Fix nit Signed-off-by: Serge Panev <spanev@nvidia.com>

* Update MXNet-TRT doc with the new optimize_for API Signed-off-by: Serge Panev <spanev@nvidia.com> * [1.x] Move block.optimize_for backend_opts to kwargs (#19386) * Move block.optimize_for backend_opts to kwargs Signed-off-by: Serge Panev <spanev@nvidia.com> * Update Hybridize to use kwargs as backend opts Signed-off-by: Serge Panev <spanev@nvidia.com> * Fix lint Signed-off-by: Serge Panev <spanev@nvidia.com> * Change clear default to False and allow hybrize+optimize_for calls Signed-off-by: Serge Panev <spanev@nvidia.com> * Fix nit Signed-off-by: Serge Panev <spanev@nvidia.com> * Adress review comments Signed-off-by: Serge Panev <spanev@nvidia.com> * Adress more review comments Signed-off-by: Serge Panev <spanev@nvidia.com> * Adress more more review comments Signed-off-by: Serge Panev <spanev@nvidia.com> * Fix nit Signed-off-by: Serge Panev <spanev@nvidia.com> * Add 1:many conversions in nnvm_to_onnx and non-flatten GEMM (#19652) Signed-off-by: Serge Panev <spanev@nvidia.com>

Kh4L requested review from aaronmarkham and szha as code owners October 20, 2020 03:46

lanking520 added the pr-work-in-progress PR is still work in progress label Oct 20, 2020

Kh4L changed the title ~~[WIP] Move block.optimize_for backend_opts to kwargs~~ [WIP][1.x] Move block.optimize_for backend_opts to kwargs Oct 21, 2020

Move block.optimize_for backend_opts to kwargs

7373172

Signed-off-by: Serge Panev <spanev@nvidia.com>

Kh4L force-pushed the optimize_for_backend_opts_kwargs branch from aa7a4d1 to 7373172 Compare October 21, 2020 05:33

Kh4L changed the title ~~[WIP][1.x] Move block.optimize_for backend_opts to kwargs~~ [1.x] Move block.optimize_for backend_opts to kwargs Oct 23, 2020

lanking520 added pr-awaiting-review PR is waiting for code review and removed pr-work-in-progress PR is still work in progress labels Oct 30, 2020

samskalicky reviewed Nov 4, 2020

View reviewed changes

lanking520 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress and removed pr-awaiting-review PR is waiting for code review pr-awaiting-testing PR is reviewed and waiting CI build and test labels Nov 5, 2020

Kh4L force-pushed the optimize_for_backend_opts_kwargs branch from 6708b59 to 0bc6c84 Compare November 5, 2020 19:49

lanking520 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress and removed pr-work-in-progress PR is still work in progress pr-awaiting-testing PR is reviewed and waiting CI build and test labels Nov 5, 2020

lanking520 added the pr-awaiting-testing PR is reviewed and waiting CI build and test label Nov 14, 2020

samskalicky reviewed Nov 14, 2020

View reviewed changes

example/extensions/lib_subgraph/README.md Outdated Show resolved Hide resolved

samskalicky reviewed Nov 14, 2020

View reviewed changes

example/extensions/lib_subgraph/test_subgraph.py Outdated Show resolved Hide resolved

samskalicky reviewed Nov 14, 2020

View reviewed changes

python/mxnet/gluon/block.py Outdated Show resolved Hide resolved

Adress more review comments

e2dfc27

Signed-off-by: Serge Panev <spanev@nvidia.com>

samskalicky reviewed Nov 14, 2020

View reviewed changes

Adress more more review comments

09d0cc8

Signed-off-by: Serge Panev <spanev@nvidia.com>

samskalicky approved these changes Nov 14, 2020

View reviewed changes

samskalicky requested a review from mseth10 November 14, 2020 06:09

mseth10 reviewed Nov 14, 2020

View reviewed changes

mseth10 approved these changes Nov 14, 2020

View reviewed changes

lanking520 added pr-work-in-progress PR is still work in progress and removed pr-awaiting-testing PR is reviewed and waiting CI build and test labels Nov 14, 2020

Fix nit

d9c51e5

Signed-off-by: Serge Panev <spanev@nvidia.com>

samskalicky added pr-awaiting-merge Review and CI is complete. Ready to Merge and removed pr-awaiting-review PR is waiting for code review labels Nov 15, 2020

samskalicky merged commit d0710e2 into apache:v1.x Nov 16, 2020

Kh4L mentioned this pull request Nov 16, 2020

Separate backend from hybridize and refactor optimize_for kwargs #19543

Merged

Kh4L mentioned this pull request Mar 5, 2021

Backport TRT fixes to 1.8 #19983

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[1.x] Move block.optimize_for backend_opts to kwargs #19386

[1.x] Move block.optimize_for backend_opts to kwargs #19386

Kh4L commented Oct 20, 2020 •

edited

mxnet-bot commented Oct 20, 2020

Kh4L commented Oct 30, 2020

mxnet-bot commented Oct 30, 2020

samskalicky Nov 4, 2020 •

edited

Kh4L Nov 5, 2020

samskalicky Nov 14, 2020

Kh4L Nov 14, 2020 •

edited

samskalicky left a comment

mseth10 Nov 14, 2020

samskalicky Nov 14, 2020

mseth10 left a comment

[1.x] Move block.optimize_for backend_opts to kwargs #19386

[1.x] Move block.optimize_for backend_opts to kwargs #19386

Conversation

Kh4L commented Oct 20, 2020 • edited

Description

Symbol

Block (Gluon)

Change

Example after the change:

mxnet-bot commented Oct 20, 2020

Kh4L commented Oct 30, 2020

mxnet-bot commented Oct 30, 2020

samskalicky Nov 4, 2020 • edited

Choose a reason for hiding this comment

Kh4L Nov 5, 2020

Choose a reason for hiding this comment

samskalicky Nov 14, 2020

Choose a reason for hiding this comment

Kh4L Nov 14, 2020 • edited

Choose a reason for hiding this comment

samskalicky left a comment

Choose a reason for hiding this comment

mseth10 Nov 14, 2020

Choose a reason for hiding this comment

samskalicky Nov 14, 2020

Choose a reason for hiding this comment

mseth10 left a comment

Choose a reason for hiding this comment

Kh4L commented Oct 20, 2020 •

edited

samskalicky Nov 4, 2020 •

edited

Kh4L Nov 14, 2020 •

edited