Add onnx support for InstanceNorm #4626

houseroad · 2018-01-12T05:25:58Z

Now, we export the InstanceNorm as an InstanceNorm op, not Reshape + BatchNorm + Reshape

houseroad · 2018-01-12T06:02:33Z

Address #4584, it works end 2 end so far. But some unused tensors got exported, too. Will fix it soon.

houseroad · 2018-01-13T00:57:11Z

@ezyang, the unused stuff got removed, ready for review. :-)

@onnxbot retest this please

houseroad · 2018-01-13T05:35:27Z

Please ignore the failed onnx-fb-universe CI, I already updated the expected files in onnxbot/onnx-fb-universe#268

torch/nn/functional.py

+                eps=eps, affine=affine)
+
+
+def instance_norm(input, weight, bias, saved_running_mean, saved_running_var,


torch/csrc/jit/export.cpp

+  // The jit tracer may record some unnecessary information.
+  // We should remove the unused input and initializer.
+  // Gather all the names of useful nodes and filter out unused ones.
+  std::unordered_set<std::string> useful_names;


torch/csrc/jit/export.cpp

  for (auto input : g->inputs()) {
+    input_names.push_back(value_name(input));
+    if (useful_names.find(value_name(input)) == useful_names.end()) {
+      continue;


ezyang · 2018-01-16T17:15:23Z

CC @linkerzhang

dzhulgakov

Cool, when it lands I can also upstream some changes from https://github.com/dzhulgakov/fast-neural-style to pytorch/examples (onnx exporter)

torch/csrc/jit/export.cpp

  for (auto input : g->inputs()) {
+    input_names.push_back(value_name(input));
+    if (useful_names.find(value_name(input)) == useful_names.end()) {
+      continue;


torch/onnx/symbolic.py

@@ -619,3 +619,15 @@ def symbolic(g, input, all_weights, h0, **fkwargs):
        return prev_output, h_outs

    return torch.onnx.symbolic_override(symbolic)
+
+
+def instance_norm_symbolic_builder(*args, **kwargs):


torch/nn/functional.py

+    """
+    import torch
+    func = instance_norm
+    if torch._C._jit_is_tracing(input):


torch/onnx/__init__.py

-            flat_args = tuple(function._iter_variables(args))
-            if not any(map(torch._C._jit_is_tracing, flat_args)):
+            flat_args = tuple(function._iter_None_variables(args))
+            if not any(map(lambda x: False if x is None else torch._C._jit_is_tracing(x), flat_args)):


ezyang · 2018-01-19T20:43:58Z

I'm on the hook for finishing this.

dzhulgakov

For mapping the tensor inputs - I wish we could find a better solution.

torch/onnx/symbolic.py

@@ -619,3 +619,15 @@ def symbolic(g, input, all_weights, h0, **fkwargs):
        return prev_output, h_outs

    return torch.onnx.symbolic_override(symbolic)
+
+
+def instance_norm_symbolic_builder(*args, **kwargs):


torch/csrc/jit/export.cpp

@@ -215,7 +236,10 @@ void encodeGraph(onnx::GraphProto * p_g, const std::shared_ptr<Graph> & g, const
  for (auto & tensor : initializers) {
    // TODO: stop using positions to determine which initializers
    // match to which inputs
-    std::string name = p_g->get_input_name(inputs_count++);
+    std::string name = input_names[inputs_count++];
+    if (useful_names.find(name) == useful_names.end()) {


torch/csrc/jit/export.cpp

@@ -181,7 +181,28 @@ void encodeGraph(onnx::GraphProto * p_g, const std::shared_ptr<Graph> & g, const
  JIT_ASSERT(p_g != nullptr);
  p_g->set_name("torch-jit-export");

+  // The jit tracer may record some unnecessary information.
+  // We should remove the unused input and initializer.
+  // Gather all the names of useful nodes and filter out unused ones.


CynthiaLu1119 · 2018-03-06T22:31:13Z

After reading this post, I tried to install pytorch by source and the onnx does work on instanceNorm2D now. Thanks.

However, I encountered some other problems. I trained some pytorch model using the pytorch version installed through http://pytorch.org/. I was able to save the trained model as .pth file and load it back in successfully by doing the following:

model = myNet(parameters...) model.load_state_dict(torch.load('xxx.pth'))

However, after I install the latest pytorch from source. The load_state_dict is reporting errors:

File "/anaconda2/lib/python2.7/site-packages/torch/nn/modules/module.py", line 528, in load_state_dict
.format(name))
KeyError: 'unexpected key "pre_model.1.running_mean" in state_dict'

I did the following to check which layer was causing the problem
model1 = myNet(parameters...)
model2 = torch.load('xxx.pth')

for name, param in model2.items():
print(name)

for name, param in model1.state_dict().items():
print(name)

I found that for the instanceNorm2D layers, model2 contains xxx.running_mean and xxx.running_var, but not model1. Any idea what I should do?

CynthiaLu1119 · 2018-03-07T01:38:15Z

Another question:
If I use the instanceNorm2D layer with affine=True, nn.InstanceNorm2d(ngf,affine=True), then I get the following problem when doing torch.onnx.export

torch.onnx.export(model, (dummy_input, dummy_input), MODEL_DIR + "eye_G.proto", verbose=True)
File "/anaconda2/lib/python2.7/site-packages/torch/onnx/init.py", line 83, in export
_export(model, args, f, export_params, verbose, training, input_names, output_names)
File "/anaconda2/lib/python2.7/site-packages/torch/onnx/init.py", line 138, in _export
_optimize_trace(trace, aten)
File "/anaconda2/lib/python2.7/site-packages/torch/onnx/init.py", line 94, in _optimize_trace
torch._C._jit_pass_onnx(trace, aten)
File "/anaconda2/lib/python2.7/site-packages/torch/onnx/init.py", line 180, in _run_symbolic_method
return symbolic_fn(*args)
File "/anaconda2/lib/python2.7/site-packages/torch/onnx/init.py", line 396, in symbolic
symbolic_output = symbolic_fn(g, *symbolic_args, **kwargs)
File "/anaconda2/lib/python2.7/site-packages/torch/onnx/symbolic.py", line 552, in instance_norm
if not weight:
RuntimeError: bool value of Tensor with more than one value is ambiguous

If I do nn.InstanceNorm2d(ngf,affine=False) or nn.InstanceNorm2d(ngf), then the export will succeed. Please help. Thanks.

onnxbot-worker-1 mentioned this pull request Jan 12, 2018

[auto] pytorch-pr-4626 onnxbot/onnx-fb-universe#266

Closed

houseroad force-pushed the instancenorm branch from 74e65e9 to d840c65 Compare January 12, 2018 05:34

houseroad mentioned this pull request Jan 12, 2018

Add/Modify Instancenorm Related Test onnxbot/onnx-fb-universe#268

Merged

houseroad force-pushed the instancenorm branch from d840c65 to ea90c81 Compare January 12, 2018 05:50

houseroad force-pushed the instancenorm branch from ea90c81 to dad96d4 Compare January 12, 2018 21:53

houseroad force-pushed the instancenorm branch from c865552 to 72f71e0 Compare January 13, 2018 05:15

ezyang reviewed Jan 16, 2018

View reviewed changes

torch/nn/functional.py Outdated

eps=eps, affine=affine)

def instance_norm(input, weight, bias, saved_running_mean, saved_running_var,

This comment was marked as off-topic.

Sign in to view

This comment was marked as off-topic.

Sign in to view

ezyang reviewed Jan 16, 2018

View reviewed changes

houseroad force-pushed the instancenorm branch from 72f71e0 to 579be41 Compare January 17, 2018 19:19

dzhulgakov reviewed Jan 19, 2018

View reviewed changes

dzhulgakov reviewed Jan 23, 2018

View reviewed changes

torch/onnx/symbolic.py Outdated

@@ -619,3 +619,15 @@ def symbolic(g, input, all_weights, h0, **fkwargs):

return prev_output, h_outs

return torch.onnx.symbolic_override(symbolic)

def instance_norm_symbolic_builder(*args, **kwargs):

This comment was marked as off-topic.

Sign in to view

Add ONNX symbolic for instancenorm

b276294

houseroad force-pushed the instancenorm branch 3 times, most recently from 6f2776d to 60abfe9 Compare February 6, 2018 19:42

ezyang reviewed Feb 6, 2018

View reviewed changes

Fix some bugs

6f870f5

houseroad force-pushed the instancenorm branch from 60abfe9 to 6f870f5 Compare February 6, 2018 22:53

ezyang merged commit c111cdf into pytorch:master Feb 7, 2018

		eps=eps, affine=affine)


		def instance_norm(input, weight, bias, saved_running_mean, saved_running_var,

Add onnx support for InstanceNorm #4626

Add onnx support for InstanceNorm #4626

Uh oh!

Conversation

houseroad commented Jan 12, 2018

Uh oh!

houseroad commented Jan 12, 2018

Uh oh!

houseroad commented Jan 13, 2018

Uh oh!

houseroad commented Jan 13, 2018

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

ezyang commented Jan 16, 2018

Uh oh!

dzhulgakov left a comment

Choose a reason for hiding this comment

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

ezyang commented Jan 19, 2018

Uh oh!

dzhulgakov left a comment

Choose a reason for hiding this comment

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

CynthiaLu1119 commented Mar 6, 2018

Uh oh!

CynthiaLu1119 commented Mar 7, 2018

Uh oh!

Uh oh!