Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tensorboard add_graph error #24904

Open
as754770178 opened this issue Aug 20, 2019 · 28 comments
Open

tensorboard add_graph error #24904

as754770178 opened this issue Aug 20, 2019 · 28 comments
Labels
module: tensorboard triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module

Comments

@as754770178
Copy link

I update torch to 1.2.0 and use torch.utils.tensorboard import SummaryWriter save the graph, report error:

Only tensors or tuples of tensors can be output from traced functions(getOutput at /pytorch/torch/csrc/jit/tracer.cpp:208)
frame #0: c10::Error::Error(c10::SourceLocation, std::string const&) + 0x33 (0x7f67d5059273 in /opt/anaconda3/lib/python3.7/site-packages/lib/libc10.so)
...

When I use torch 1.1.0 + cuda9.0 don't report error.

@gchanan gchanan added module: tensorboard triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module labels Aug 20, 2019
@gchanan
Copy link
Contributor

gchanan commented Aug 20, 2019

CC @orionr

@orionr
Copy link
Contributor

orionr commented Aug 20, 2019

@as754770178 do you have a sample bit of code that reproduces the failure?

cc @sanekmelnikov @lanpa @natalialunova

@orionr
Copy link
Contributor

orionr commented Aug 20, 2019

Also cc @suo

@as754770178
Copy link
Author

error info:

Only tensors or tuples of tensors can be output from traced functions (getOutput at /pytorch/torch/csrc/jit/tracer.cpp:208)
frame #0: c10::Error::Error(c10::SourceLocation, std::string const&) + 0x33 (0x7febc68ca273 in /opt/anaconda3/lib/python3.7/site-packages/torch/lib/libc10.so)
frame #1: torch::jit::tracer::TracingState::getOutput(c10::IValue const&) + 0x3e8 (0x7feb09cf8228 in /opt/anaconda3/lib/python3.7/site-packages/torch/lib/libtorch.so)
frame #2: torch::jit::tracer::exit(std::vector<c10::IValue, std::allocator<c10::IValue> > const&) + 0x3c (0x7feb09cf855c in /opt/anaconda3/lib/python3.7/site-packages/torch/lib/libtorch.so)
frame #3: <unknown function> + 0x4bf042 (0x7febc71da042 in /opt/anaconda3/lib/python3.7/site-packages/torch/lib/libtorch_python.so)
frame #4: <unknown function> + 0x4d30f1 (0x7febc71ee0f1 in /opt/anaconda3/lib/python3.7/site-packages/torch/lib/libtorch_python.so)
frame #5: <unknown function> + 0x1d3ab4 (0x7febc6eeeab4 in /opt/anaconda3/lib/python3.7/site-packages/torch/lib/libtorch_python.so)
frame #6: _PyMethodDef_RawFastCallKeywords + 0x264 (0x7febd410aea4 in /opt/anaconda3/bin/python)
frame #7: _PyCFunction_FastCallKeywords + 0x21 (0x7febd410afc1 in /opt/anaconda3/bin/python)
frame #8: _PyEval_EvalFrameDefault + 0x523e (0x7febd41802ce in /opt/anaconda3/bin/python)
frame #9: _PyEval_EvalCodeWithName + 0x2e8 (0x7febd40b9528 in /opt/anaconda3/bin/python)
frame #10: _PyFunction_FastCallKeywords + 0x325 (0x7febd410a645 in /opt/anaconda3/bin/python)
frame #11: _PyEval_EvalFrameDefault + 0x6d6 (0x7febd417b766 in /opt/anaconda3/bin/python)
frame #12: _PyEval_EvalCodeWithName + 0x2e8 (0x7febd40b9528 in /opt/anaconda3/bin/python)
frame #13: _PyFunction_FastCallKeywords + 0x325 (0x7febd410a645 in /opt/anaconda3/bin/python)
frame #14: _PyEval_EvalFrameDefault + 0x4a9e (0x7febd417fb2e in /opt/anaconda3/bin/python)
frame #15: _PyEval_EvalCodeWithName + 0x2e8 (0x7febd40b9528 in /opt/anaconda3/bin/python)
frame #16: _PyFunction_FastCallKeywords + 0x325 (0x7febd410a645 in /opt/anaconda3/bin/python)
frame #17: _PyEval_EvalFrameDefault + 0x6d6 (0x7febd417b766 in /opt/anaconda3/bin/python)
frame #18: _PyEval_EvalCodeWithName + 0x2e8 (0x7febd40b9528 in /opt/anaconda3/bin/python)
frame #19: _PyFunction_FastCallKeywords + 0x325 (0x7febd410a645 in /opt/anaconda3/bin/python)
frame #20: _PyEval_EvalFrameDefault + 0x520 (0x7febd417b5b0 in /opt/anaconda3/bin/python)
frame #21: _PyFunction_FastCallKeywords + 0xfb (0x7febd410a41b in /opt/anaconda3/bin/python)
frame #22: _PyEval_EvalFrameDefault + 0x520 (0x7febd417b5b0 in /opt/anaconda3/bin/python)
frame #23: _PyFunction_FastCallKeywords + 0xfb (0x7febd410a41b in /opt/anaconda3/bin/python)
frame #24: _PyEval_EvalFrameDefault + 0x520 (0x7febd417b5b0 in /opt/anaconda3/bin/python)
frame #25: _PyFunction_FastCallDict + 0x10b (0x7febd40ba4eb in /opt/anaconda3/bin/python)
frame #26: _PyObject_Call_Prepend + 0x63 (0x7febd40d9703 in /opt/anaconda3/bin/python)
frame #27: <unknown function> + 0x16c1fa (0x7febd41131fa in /opt/anaconda3/bin/python)
frame #28: _PyObject_FastCallKeywords + 0x49b (0x7febd411411b in /opt/anaconda3/bin/python)
frame #29: _PyEval_EvalFrameDefault + 0x4b86 (0x7febd417fc16 in /opt/anaconda3/bin/python)
frame #30: _PyEval_EvalCodeWithName + 0x2e8 (0x7febd40b9528 in /opt/anaconda3/bin/python)
frame #31: PyEval_EvalCodeEx + 0x44 (0x7febd40ba3a4 in /opt/anaconda3/bin/python)
frame #32: PyEval_EvalCode + 0x1c (0x7febd40ba3cc in /opt/anaconda3/bin/python)
frame #33: <unknown function> + 0x22d304 (0x7febd41d4304 in /opt/anaconda3/bin/python)
frame #34: PyRun_FileExFlags + 0xa1 (0x7febd41dc611 in /opt/anaconda3/bin/python)
frame #35: PyRun_SimpleFileExFlags + 0x1c4 (0x7febd41dc804 in /opt/anaconda3/bin/python)
frame #36: <unknown function> + 0x23717d (0x7febd41de17d in /opt/anaconda3/bin/python)
frame #37: _Py_UnixMain + 0x80 (0x7febd41de3f0 in /opt/anaconda3/bin/python)
frame #38: __libc_start_main + 0xf5 (0x7febd37c9b35 in /lib64/libc.so.6)
frame #39: <unknown function> + 0x1e3e32 (0x7febd418ae32 in /opt/anaconda3/bin/python)

Error occurs, No graph saved
Traceback (most recent call last):
  File "/home/110482/code/AIFlow_bak/Trunk/dh_aiflow/dh_aiflow_script/auto_train.py", line 420, in <module>
    AutoTrain(cfg)()
  File "/home/110482/code/AIFlow_bak/Trunk/dh_aiflow/dh_aiflow_script/auto_train.py", line 358, in __call__
    process.run()
  File "/home/110482/code/AIFlow_bak/Trunk/dh_aiflow/dh_aiflow_process/dh_aiflow_process.py", line 220, in run
    self._train(self.gpu_ids)
  File "/home/110482/code/AIFlow_bak/Trunk/dh_aiflow/dh_aiflow_process/dh_aiflow_process.py", line 305, in _train
    torch.rand(self.summary_config.input_size).unsqueeze(0).cuda())
  File "/opt/anaconda3/lib/python3.7/site-packages/torch/utils/tensorboard/writer.py", line 648, in add_graph
    self._get_file_writer().add_graph(graph(model, input_to_model, verbose))
  File "/opt/anaconda3/lib/python3.7/site-packages/torch/utils/tensorboard/_pytorch_graph.py", line 234, in graph
    raise e
  File "/opt/anaconda3/lib/python3.7/site-packages/torch/utils/tensorboard/_pytorch_graph.py", line 229, in graph
    trace = torch.jit.trace(model, args)
  File "/opt/anaconda3/lib/python3.7/site-packages/torch/jit/__init__.py", line 772, in trace
    check_tolerance, _force_outplace, _module_class)
  File "/opt/anaconda3/lib/python3.7/site-packages/torch/jit/__init__.py", line 904, in trace_module
    module._c._create_method_from_trace(method_name, func, example_inputs, var_lookup_fn, _force_outplace)
RuntimeError: Only tensors or tuples of tensors can be output from traced functions (getOutput at /pytorch/torch/csrc/jit/tracer.cpp:208)
frame #0: c10::Error::Error(c10::SourceLocation, std::string const&) + 0x33 (0x7febc68ca273 in /opt/anaconda3/lib/python3.7/site-packages/torch/lib/libc10.so)
frame #1: torch::jit::tracer::TracingState::getOutput(c10::IValue const&) + 0x3e8 (0x7feb09cf8228 in /opt/anaconda3/lib/python3.7/site-packages/torch/lib/libtorch.so)
frame #2: torch::jit::tracer::exit(std::vector<c10::IValue, std::allocator<c10::IValue> > const&) + 0x3c (0x7feb09cf855c in /opt/anaconda3/lib/python3.7/site-packages/torch/lib/libtorch.so)
frame #3: <unknown function> + 0x4bf042 (0x7febc71da042 in /opt/anaconda3/lib/python3.7/site-packages/torch/lib/libtorch_python.so)
frame #4: <unknown function> + 0x4d30f1 (0x7febc71ee0f1 in /opt/anaconda3/lib/python3.7/site-packages/torch/lib/libtorch_python.so)
frame #5: <unknown function> + 0x1d3ab4 (0x7febc6eeeab4 in /opt/anaconda3/lib/python3.7/site-packages/torch/lib/libtorch_python.so)
frame #6: _PyMethodDef_RawFastCallKeywords + 0x264 (0x7febd410aea4 in /opt/anaconda3/bin/python)
frame #7: _PyCFunction_FastCallKeywords + 0x21 (0x7febd410afc1 in /opt/anaconda3/bin/python)
frame #8: _PyEval_EvalFrameDefault + 0x523e (0x7febd41802ce in /opt/anaconda3/bin/python)
frame #9: _PyEval_EvalCodeWithName + 0x2e8 (0x7febd40b9528 in /opt/anaconda3/bin/python)
frame #10: _PyFunction_FastCallKeywords + 0x325 (0x7febd410a645 in /opt/anaconda3/bin/python)
frame #11: _PyEval_EvalFrameDefault + 0x6d6 (0x7febd417b766 in /opt/anaconda3/bin/python)
frame #12: _PyEval_EvalCodeWithName + 0x2e8 (0x7febd40b9528 in /opt/anaconda3/bin/python)
frame #13: _PyFunction_FastCallKeywords + 0x325 (0x7febd410a645 in /opt/anaconda3/bin/python)
frame #14: _PyEval_EvalFrameDefault + 0x4a9e (0x7febd417fb2e in /opt/anaconda3/bin/python)
frame #15: _PyEval_EvalCodeWithName + 0x2e8 (0x7febd40b9528 in /opt/anaconda3/bin/python)
frame #16: _PyFunction_FastCallKeywords + 0x325 (0x7febd410a645 in /opt/anaconda3/bin/python)
frame #17: _PyEval_EvalFrameDefault + 0x6d6 (0x7febd417b766 in /opt/anaconda3/bin/python)
frame #18: _PyEval_EvalCodeWithName + 0x2e8 (0x7febd40b9528 in /opt/anaconda3/bin/python)
frame #19: _PyFunction_FastCallKeywords + 0x325 (0x7febd410a645 in /opt/anaconda3/bin/python)
frame #20: _PyEval_EvalFrameDefault + 0x520 (0x7febd417b5b0 in /opt/anaconda3/bin/python)
frame #21: _PyFunction_FastCallKeywords + 0xfb (0x7febd410a41b in /opt/anaconda3/bin/python)
frame #22: _PyEval_EvalFrameDefault + 0x520 (0x7febd417b5b0 in /opt/anaconda3/bin/python)
frame #23: _PyFunction_FastCallKeywords + 0xfb (0x7febd410a41b in /opt/anaconda3/bin/python)
frame #24: _PyEval_EvalFrameDefault + 0x520 (0x7febd417b5b0 in /opt/anaconda3/bin/python)
frame #25: _PyFunction_FastCallDict + 0x10b (0x7febd40ba4eb in /opt/anaconda3/bin/python)
frame #26: _PyObject_Call_Prepend + 0x63 (0x7febd40d9703 in /opt/anaconda3/bin/python)
frame #27: <unknown function> + 0x16c1fa (0x7febd41131fa in /opt/anaconda3/bin/python)
frame #28: _PyObject_FastCallKeywords + 0x49b (0x7febd411411b in /opt/anaconda3/bin/python)
frame #29: _PyEval_EvalFrameDefault + 0x4b86 (0x7febd417fc16 in /opt/anaconda3/bin/python)
frame #30: _PyEval_EvalCodeWithName + 0x2e8 (0x7febd40b9528 in /opt/anaconda3/bin/python)
frame #31: PyEval_EvalCodeEx + 0x44 (0x7febd40ba3a4 in /opt/anaconda3/bin/python)
frame #32: PyEval_EvalCode + 0x1c (0x7febd40ba3cc in /opt/anaconda3/bin/python)
frame #33: <unknown function> + 0x22d304 (0x7febd41d4304 in /opt/anaconda3/bin/python)
frame #34: PyRun_FileExFlags + 0xa1 (0x7febd41dc611 in /opt/anaconda3/bin/python)
frame #35: PyRun_SimpleFileExFlags + 0x1c4 (0x7febd41dc804 in /opt/anaconda3/bin/python)
frame #36: <unknown function> + 0x23717d (0x7febd41de17d in /opt/anaconda3/bin/python)
frame #37: _Py_UnixMain + 0x80 (0x7febd41de3f0 in /opt/anaconda3/bin/python)
frame #38: __libc_start_main + 0xf5 (0x7febd37c9b35 in /lib64/libc.so.6)
frame #39: <unknown function> + 0x1e3e32 (0x7febd418ae32 in /opt/anaconda3/bin/python)

@as754770178
Copy link
Author

I try. Now I can provide some information.
(1) model: I run yolo model accoding to https://github.com/eriklindernoren/PyTorch-YOLOv3.git.
(2) dataset: my dataset return string, such as the image path and the json path, but don't use in forward.

# forward only use images
def forward(self, *inputs):
    images, *others = inputs

@orionr
Copy link
Contributor

orionr commented Aug 21, 2019

@as754770178, if you can paste in your complete code that would help us debug. Ultimately, though, based on the error, I think you are returning something non-standard from your forward method. You can only return tensors and tuples of tensors. I took a quick look at https://github.com/eriklindernoren/PyTorch-YOLOv3 and nothing obvious jumped out at me there.

@lanpa
Copy link
Collaborator

lanpa commented Aug 22, 2019

RuntimeError: Only tensors or tuples of tensors can be output from traced functions
@as754770178 Can you cut the model in half and see if the graph can be traced without error? With binary search, you can help us locate the exact location of the operator easily.

@as754770178
Copy link
Author

Thanks, I will try it next week

@as754770178
Copy link
Author

    featuremaps = []
    featuremap1 = xxx
    featuremaps.append(featuremap1)
    # stride 16
    featuremap2 = xxx
    featuremaps.append(featuremap2)
    # stride 32
    featuremap3 = xxx
    featuremaps.append(featuremap3)

return featuremaps

So return List, I change to tuple(featuremaps), the program not report error. But the content in Graph is empty.
note: It is not showing No graph definition files were found., but the page is blank.
I download the Graph, when I open it showing:

Failed to load the set of active dashboards.
This can occur if the TensorBoard backend is no longer running. Perhaps this page is cached?

If you think that you’ve fixed the problem, click the reload button in the top-right. We’ll try to reload every 30 seconds as well.
Last reload: Tue Aug 27 2019 15:19:29 GMT+0800 (中国标准时间)
Log directory:

@lanpa
Copy link
Collaborator

lanpa commented Aug 28, 2019

@as754770178 I can't find so-called featuremaps in PyTorch-YOLOv3.

@as754770178
Copy link
Author

sorry, I change backbone.

def conv3x3(in_planes, out_planes, stride=1):
  """3x3 convolution with padding"""
  return nn.Conv2d(
    in_planes,
    out_planes,
    kernel_size=3,
    stride=stride,
    padding=1,
    bias=False)


class BasicBlock(nn.Module):
  expansion = 1

  def __init__(self, inplanes, planes, stride=1, downsample=None):
    super(BasicBlock, self).__init__()
    self.conv1 = conv3x3(inplanes, planes, stride)
    self.bn1 = nn.BatchNorm2d(planes)
    self.relu = nn.ReLU(inplace=True)
    self.conv2 = conv3x3(planes, planes)
    self.bn2 = nn.BatchNorm2d(planes)
    self.downsample = downsample
    self.stride = stride

  def forward(self, x):
    residual = x

    out = self.conv1(x)
    out = self.bn1(out)
    out = self.relu(out)

    out = self.conv2(out)
    out = self.bn2(out)

    if self.downsample is not None:
      residual = self.downsample(x)

    out += residual
    out = self.relu(out)

    return out


class Bottleneck(nn.Module):
  expansion = 4

  def __init__(self, inplanes, planes, stride=1, downsample=None):
    super(Bottleneck, self).__init__()
    self.conv1 = nn.Conv2d(inplanes, planes, kernel_size=1, bias=False)
    self.bn1 = nn.BatchNorm2d(planes)
    self.conv2 = nn.Conv2d(
      planes, planes, kernel_size=3, stride=stride, padding=1, bias=False)
    self.bn2 = nn.BatchNorm2d(planes)
    self.conv3 = nn.Conv2d(planes, planes * 4, kernel_size=1, bias=False)
    self.bn3 = nn.BatchNorm2d(planes * 4)
    self.relu = nn.ReLU(inplace=True)
    self.downsample = downsample
    self.stride = stride

  def forward(self, x):
    residual = x

    out = self.conv1(x)
    out = self.bn1(out)
    out = self.relu(out)

    out = self.conv2(out)
    out = self.bn2(out)
    out = self.relu(out)

    out = self.conv3(out)
    out = self.bn3(out)

    if self.downsample is not None:
      residual = self.downsample(x)

    out += residual
    out = self.relu(out)

    return out


class DetTail(nn.Module):

  def __init__(self, inplanes, classes):
    super(DetTail, self).__init__()
    self.conv1 = nn.Conv2d(inplanes, 512, 3, padding=1, bias=False)
    self.bn1 = nn.BatchNorm2d(512)
    self.conv2 = nn.Conv2d(512, classes, 1)

  def forward(self, x):
    x = self.conv1(x)
    x = self.bn1(x)
    x = F.relu(x)
    x = self.conv2(x)
    return x


class ResNet(nn.Module):

  def __init__(self, block, layers, num_classes, yolo_config):
    self.yolo_config = yolo_config
    self.num_classes = num_classes
    super(ResNet, self).__init__()
    self.inplanes = 64
    self.conv1 = nn.Conv2d(
      3, 64, kernel_size=7, stride=2, padding=3, bias=False)
    self.bn1 = nn.BatchNorm2d(64)
    self.relu = nn.ReLU(inplace=True)
    self.maxpool = nn.MaxPool2d(kernel_size=3, stride=2, padding=1)
    self.layer1 = self._make_layer(block, 64, layers[0])

    self.layer2 = self._make_layer(block, 128, layers[1], stride=2)
    self.det2 = DetTail(128 * block.expansion,
                        (5 + self.num_classes) * self.yolo_config.anchor_num)

    self.layer3 = self._make_layer(block, 256, layers[2], stride=2)
    self.det3 = DetTail(256 * block.expansion,
                        (5 + self.num_classes) * self.yolo_config.anchor_num)

    self.layer4 = self._make_layer(block, 512, layers[3], stride=2)
    self.det4 = DetTail(512 * block.expansion,
                        (5 + self.num_classes) * self.yolo_config.anchor_num)

  def _make_layer(self, block, planes, blocks, stride=1):
    downsample = None
    if stride != 1 or self.inplanes != planes * block.expansion:
      downsample = nn.Sequential(
        nn.Conv2d(
          self.inplanes,
          planes * block.expansion,
          kernel_size=1,
          stride=stride,
          bias=False),
        nn.BatchNorm2d(planes * block.expansion),
      )

    layers = []
    layers.append(block(self.inplanes, planes, stride, downsample))
    self.inplanes = planes * block.expansion
    for i in range(1, blocks):
      layers.append(block(self.inplanes, planes))

    return nn.Sequential(*layers)

  def forward(self, *inputs):
    images, *others = inputs

    x = self.conv1(images)
    x = self.bn1(x)
    x = self.relu(x)
    x = self.maxpool(x)
    x = self.layer1(x)
    # stride 8
    featuremaps = []
    featuremap2 = self.layer2(x)
    featuremaps.append(self.det2(featuremap2))
    # stride 16
    featuremap3 = self.layer3(featuremap2)
    featuremaps.append(self.det3(featuremap3))
    # stride 32
    featuremap4 = self.layer4(featuremap3)
    featuremaps.append(self.det4(featuremap4))

    return tuple(featuremaps)


def resnet18yolo(num_classes, yolo_config):
  """Constructs a ResNet-18 model.

    Args:
        pretrained (bool): If True, returns a model pre-trained on ImageNet
    """
  model = ResNet(BasicBlock, [2, 2, 2, 2], num_classes, yolo_config)
  return model
``
Is it due to the version of tensorboard, my version is 1.14

@as754770178
Copy link
Author

writer.add_graph(self.model,  torch.rand(self.summary_config.input_size).unsqueeze(0).cuda())

When summary_config.input_size is (3, 224, 224), the graph don't save. I change to (8, 3, 224, 224), the graph save.

@hufangjian
Copy link

I get the same issue.

torch.version =1.1.0
code:
#--coding:utf-8--
import torch
import torchvision
from torch.autograd import Variable
from tensorboardX import SummaryWriter

模拟输入数据

input_data = Variable(torch.rand(16, 3, 224, 224))

从torchvision中导入已有模型

net = torchvision.models.resnet18()

声明writer对象,保存的文件夹,异己名称

writer = SummaryWriter(log_dir='.log/', comment='resnet18')
with writer:
writer.add_graph(net, (input_data,))
error:
RuntimeError: r ASSERT FAILED at /pytorch/aten/src/ATen/core/jit_type.h:142, please report a bug to PyTorch. (expect at /pytorch/aten/src/ATen/core/jit_type.h:142)
frame #0: std::function<std::string ()>::operator()() const + 0x11 (0x7f705ad58441 in /root/anaconda3/lib/python3.6/site-packages/torch/lib/libc10.so)
frame #1: c10::Error::Error(c10::SourceLocation, std::string const&) + 0x2a (0x7f705ad57d7a in /root/anaconda3/lib/python3.6/site-packages/torch/lib/libc10.so)
frame #2: std::shared_ptrc10::CompleteTensorType c10::Type::expectc10::CompleteTensorType() + 0xa3 (0x7f705b37e253 in /root/anaconda3/lib/python3.6/site-packages/torch/lib/libtorch_python.so)
frame #3: + 0x421d2b (0x7f705b392d2b in /root/anaconda3/lib/python3.6/site-packages/torch/lib/libtorch_python.so)
frame #4: + 0x130cfc (0x7f705b0a1cfc in /root/anaconda3/lib/python3.6/site-packages/torch/lib/libtorch_python.so)

@tjsongzw
Copy link

tjsongzw commented Nov 6, 2019

hi

RuntimeError: Only tensors or tuples of tensors can be output from traced functions

what if, forward return a dict, any plan to support other return type?

@as754770178
Copy link
Author

I debug my program, function add_graph of class FileWriter in /opt/anaconda3/lib/python3.7/site-packages/torch/utils/tensorboard/writer.py. The parameter graph_profile contain graph info, and the program not report error. But no graph display in tensorboard.

@orionr
Copy link
Contributor

orionr commented Dec 16, 2019

Those that are seeing this issue, can you try nightly? We landed a fix and hopefully it resolves the regression seen in 1.3.

@develooper1994
Copy link

count me too.

@smivv
Copy link

smivv commented Feb 18, 2020

RuntimeError: Only tensors or tuples of tensors can be output from traced functions when my model has dict as output :(

@minygd
Copy link

minygd commented May 29, 2020

Hi, I have fix this by change the LIST(outputs) of last output to tuple(outputs) and can work for the Multi-Stage Prediction Networks. I hope it can help. And please let me know if anyone can fix this in torchsummary Libs :)

@orionr
Copy link
Contributor

orionr commented Jun 8, 2020

Is the latest version of PyTorch working for folks here? Seems like there might still be some outstanding issues, so alerting @lanpa and @J0Nreynolds. If you have detailed failures with the latest version (>=1.5.0) please post below and we'll try and fix. @smivv was this with 1.5.0?

@enginbozaba
Copy link

enginbozaba commented Jun 10, 2020

@orionr thanks

@austinmw
Copy link

austinmw commented Jun 24, 2020

Hi @orionr , would upgrading to 1.5.0 solve the Only tensors or tuples of tensors can be output from traced functions error or is that still a limitation? Also please let me know if there's an easy way to point to where in my code this stems from (I'm currently trying to trace this: https://github.com/xingyizhou/CenterTrack, which uses DCNv2)

@Yonggie
Copy link

Yonggie commented Nov 4, 2020

I'm using torch 1.4.0+cpu.
It says the tensorboard tool does not support dict type output from the network as tensorboard can not track...
Only tensors or tuples of tensors can be output from traced functions (getOutput at ..\torch\csrc\jit\tracer.cpp:212) (no backtrace available)

@orionr
Copy link
Contributor

orionr commented Nov 4, 2020

cc @cryptopic, @nataliakliushkina, @edward-io to track

@VanChilly
Copy link

@as754770178, if you can paste in your complete code that would help us debug. Ultimately, though, based on the error, I think you are returning something non-standard from your forward method. You can only return tensors and tuples of tensors. I took a quick look at https://github.com/eriklindernoren/PyTorch-YOLOv3 and nothing obvious jumped out at me there.

This solution works on my problem

@dheerajvarma24
Copy link

Hi @orionr , would upgrading to 1.5.0 solve the Only tensors or tuples of tensors can be output from traced functions error or is that still a limitation? Also please let me know if there's an easy way to point to where in my code this stems from (I'm currently trying to trace this: https://github.com/xingyizhou/CenterTrack, which uses DCNv2)

Hi @austinmw were you able to visualize the CenterTrack library with tensorboard graphs? I am still facing the same issue using the DCNv2 and torch 1.4 ?

@austinmw
Copy link

@dheerajvarma24 Hi, I'm sorry but it was so long ago that I can't seem to remember

@dheerajvarma24
Copy link

@dheerajvarma24 Hi, I'm sorry but it was so long ago that I can't seem to remember

Hi @austinmw no problem. Thank you v.much for the immediate reply.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
module: tensorboard triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Projects
None yet
Development

No branches or pull requests