upsample_bilinear2d() got an unexpected keyword argument 'align_corners' (occurred when translating upsample_bilinear2d)

[hongzhenwang]

tensorboardX1.2 may not support upsample module in pytorch0.4

File "/home/wanghongzhen/anaconda2/lib/python2.7/site-packages/tensorboardX/writer.py", line 422, in add_graph
self.file_writer.add_graph(graph(model, input_to_model, verbose))
File "/home/wanghongzhen/anaconda2/lib/python2.7/site-packages/tensorboardX/graph.py", line 94, in graph
torch.onnx._optimize_trace(trace, False)
File "/home/wanghongzhen/anaconda2/lib/python2.7/site-packages/torch/onnx/init.py", line 30, in _optimize_trace
trace.set_graph(utils._optimize_graph(trace.graph(), aten))
File "/home/wanghongzhen/anaconda2/lib/python2.7/site-packages/torch/onnx/utils.py", line 95, in _optimize_graph
graph = torch._C._jit_pass_onnx(graph, aten)
File "/home/wanghongzhen/anaconda2/lib/python2.7/site-packages/torch/onnx/init.py", line 40, in _run_symbolic_function
return utils._run_symbolic_function(*args, **kwargs)
File "/home/wanghongzhen/anaconda2/lib/python2.7/site-packages/torch/onnx/utils.py", line 368, in _run_symbolic_function
return fn(g, *inputs, **attrs)
TypeError: upsample_bilinear2d() got an unexpected keyword argument 'align_corners' (occurred when translating upsample_bilinear2d)

[lanpa]

I don't have time to do a test, but I guess it will be fixed if you update your pytorch/onnx with onnx/onnx#1023

[hongzhenwang]

@lanpa my onnx is already the newest one, onnx1.2.1. Is there any other solutions?

[lanpa]

Have you built newest pytorch from source? Please post a code snippet so I can try it as well.

[hongzhenwang]

pytorch0.4 is installed by conda.
I added upsample layer in module Net1(demo_graph.py), the error is same.

class Net1(nn.Module):
def init(self):
super(Net1, self).init()
self.conv1 = nn.Conv2d(1, 10, kernel_size=5)
self.conv2 = nn.Conv2d(10, 20, kernel_size=5)
self.conv2_drop = nn.Dropout2d()
self.fc1 = nn.Linear(320, 50)
self.fc2 = nn.Linear(50, 10)
self.bn = nn.BatchNorm2d(20)
def forward(self, x):
x = F.max_pool2d(self.conv1(x), 2)
x = F.relu(x) + F.relu(-x)
x = F.relu(F.max_pool2d(self.conv2_drop(self.conv2(x)), 2))
x = F.upsample(x, scale_factor=2, mode='bilinear')
x = self.bn(x)
x = x.view(-1, 320)
x = F.relu(self.fc1(x))
x = F.dropout(x, training=self.training)
x = self.fc2(x)
x = F.softmax(x, dim=1)
return x

error:
File "demo_graph.py", line 56, in
w.add_graph(model, (dummy_input, ))
File "/home/wanghongzhen/torch_project/tensorboard-pytorch/tensorboardX/writer.py", line 419, in add_graph
self.file_writer.add_graph(graph(model, input_to_model, verbose))
File "/home/wanghongzhen/torch_project/tensorboard-pytorch/tensorboardX/graph.py", line 77, in graph
torch.onnx._optimize_trace(trace, False)
File "/home/wanghongzhen/anaconda2/lib/python2.7/site-packages/torch/onnx/init.py", line 30, in _optimize_trace
trace.set_graph(utils._optimize_graph(trace.graph(), aten))
File "/home/wanghongzhen/anaconda2/lib/python2.7/site-packages/torch/onnx/utils.py", line 95, in _optimize_graph
graph = torch._C._jit_pass_onnx(graph, aten)
File "/home/wanghongzhen/anaconda2/lib/python2.7/site-packages/torch/onnx/init.py", line 40, in _run_symbolic_function
return utils._run_symbolic_function(*args, **kwargs)
File "/home/wanghongzhen/anaconda2/lib/python2.7/site-packages/torch/onnx/utils.py", line 368, in _run_symbolic_function
return fn(g, *inputs, **attrs)
TypeError: upsample_bilinear2d() got an unexpected keyword argument 'align_corners' (occurred when translating upsample_bilinear2d)

[lanpa]

I use conda install -c pytorch pytorch-nightly and pip install git+https://github.com/lanpa/tensorboard-pytorch to install latest pytorch and tensorboardX.
And it plots graph (with warnings below)

/home/dexter/anaconda3/lib/python3.6/site-packages/torch/nn/functional.py:1761: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
  "See the documentation of nn.Upsample for details.".format(mode))
/home/dexter/anaconda3/lib/python3.6/site-packages/torch/onnx/symbolic.py:69: UserWarning: ONNX export failed on upsample_bilinear2d because align_corners == True not supported
  warnings.warn("ONNX export failed on " + op + " because " + msg + " not supported")
/home/dexter/tensorboard-pytorch/tensorboardX/graph.py:41: UserWarning: Error getting attributes of node %26 : Dynamic = onnx::Constant[value={2}](), scope: Net1, error is VariableType::ID() not implemented
  warnings.warn("Error getting attributes of node {}, error is {}".format(attrs, e))
/home/dexter/tensorboard-pytorch/tensorboardX/graph.py:41: UserWarning: Error getting attributes of node %31 : Dynamic = onnx::Constant[value={2}](), scope: Net1, error is VariableType::ID() not implemented
  warnings.warn("Error getting attributes of node {}, error is {}".format(attrs, e))

[hongzhenwang]

My python version is 2.7, may be this is the problem?

install pytorch and tensorboardX with
conda install pytorch torchvision -c pytorch
pip install git+https://github.com/lanpa/tensorboard-pytorch
both are the lastest version

[lanpa]

your pytorch is not the latest. use pytorch-nightly

[hongzhenwang]

I installed pytorch-nightly, but the problem still exist....

[lanpa]

Hi, did you do the test with the code you provided?

[hongzhenwang]

yes, the code is unchanged

[lanpa]

Do you have the same warning message as mine? A quick solution might be the same as #145

[hongzhenwang]

No, there is only error message.
I have tried the solution in #145, but still output the same error message.

[lanpa]

conv1's feature map is 10D and your upsample layer expects 20 channel input

[hongzhenwang]

Sorry, I first add unsample behind conv2_drop, then add after conv1. Now, I have modified the mistake, but the error message is the same

[lanpa]

How about test in a conda virtual environment?

[hongzhenwang]

Sorry, what did you mean? A conda virtual environment is the same to the active environment, what to test ?

[lanpa]

It comes to my mind that does your network forwards its input properly?

[hongzhenwang]

Did you mean the dummy_input or the ConvTranspose2d attribute ? I haven't found the wrong setting ......

[lanpa]

dummy_input. I just posted an introduction in chinese. Hope that helps (check the add_graph part). https://zhuanlan.zhihu.com/p/37626738

[hongzhenwang]

Thank you. I have plot the net graph if comment the code behind view, otherwise will prompt error. These is a strange problem.

[lanpa]

After upsample by scale of 2, the number of elements roughly multipled by 4 times. so the following layers need to be rewrite. (those you commented out)

[hongzhenwang]

A stupid mistake.... Thank you