Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

a bug when converting the dino detector to tensorrt #265

Open
goldentimecoolk opened this issue Dec 8, 2023 · 1 comment
Open

a bug when converting the dino detector to tensorrt #265

goldentimecoolk opened this issue Dec 8, 2023 · 1 comment

Comments

@goldentimecoolk
Copy link

Does anyone successfully export DINO from pytorch to tensorrt? Please share your experience.

There is the bug (and lots of warnings) in my experience. An error RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! occurs with the command line python deploy.py ./deploy/configs/mmdet/instance-seg/instance-seg_tensorrt_dynamic-320x320-1344x1344.py ./configs/coco/dino_4scale_internimage_l_1x_coco_0.1x_backbone_lr.py /media/shukun/Dataset1/downloads/dino_4scale_internimage_l_1x_coco_0.1x_backbone_lr.pth ./deploy/demo.jpg --work-dir ./work_dirs/mmdet/instance-seg/dino_4scale_internimage_l_1x_coco_0.1x_backbone_lr --device cuda --dump-info. Thanks in advance!
@goldentimecoolk
Copy link
Author 

/home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/mmdet/datasets/utils.py:70: UserWarning: "ImageToTensor" pipeline is replaced by "DefaultFormatBundle" for batch inference. It is recommended to manually replace it in the test data pipeline in your config file.
  'data pipeline in your config file.', UserWarning)
2023-12-08 14:08:55,567 - mmdeploy - INFO - Start pipeline mmdeploy.apis.pytorch2onnx.torch2onnx in subprocess
2023-12-08 14:08:55,649 - mmdet - INFO - using core type: DCNv3_pytorch
2023-12-08 14:08:55,650 - mmdet - INFO - using activation layer: GELU
2023-12-08 14:08:55,650 - mmdet - INFO - using main norm layer: LN
2023-12-08 14:08:55,650 - mmdet - INFO - using dpr: linear, 0.4
2023-12-08 14:08:55,650 - mmdet - INFO - level2_post_norm: False
2023-12-08 14:08:55,650 - mmdet - INFO - level2_post_norm_block_ids: None
2023-12-08 14:08:55,650 - mmdet - INFO - res_post_norm: False
load checkpoint from local path: /media/shukun/Dataset1/downloads/dino_4scale_internimage_l_1x_coco_0.1x_backbone_lr.pth
/home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/mmdet/apis/inference.py:51: UserWarning: Class names are not saved in the checkpoint's meta data, use COCO classes by default.
  warnings.warn('Class names are not saved in the checkpoint\'s '
/home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/mmdet/datasets/utils.py:70: UserWarning: "ImageToTensor" pipeline is replaced by "DefaultFormatBundle" for batch inference. It is recommended to manually replace it in the test data pipeline in your config file.
  'data pipeline in your config file.', UserWarning)
2023-12-08 14:09:05,503 - mmdeploy - WARNING - DeprecationWarning: get_onnx_config will be deprecated in the future. 
2023-12-08 14:09:05,503 - mmdeploy - INFO - Export PyTorch model to ONNX: ./work_dirs/mmdet/instance-seg/dino_4scale_internimage_l_1x_coco_0.1x_backbone_lr/end2end.onnx.
2023-12-08 14:09:05,509 - mmdeploy - WARNING - Can not find torch._C._jit_pass_onnx_autograd_function_process, function rewrite will not be applied
2023-12-08 14:09:05,512 - mmdeploy - WARNING - Can not find torch._C._jit_pass_onnx_deduplicate_initializers, function rewrite will not be applied
/home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/mmdeploy/core/optimizers/function_marker.py:158: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  ys_shape = tuple(int(s) for s in ys.shape)
/media/shukun/新加卷/code/InternImage/detection/ops_dcnv3/functions/dcnv3_func.py:93: UserWarning: __floordiv__ is deprecated, and its behavior will change in a future version of pytorch. It currentrounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor').
  H_out = (H_ - (dilation_h * (kernel_h - 1) + 1)) // stride_h + 1
/media/shukun/新加卷/code/InternImage/detection/ops_dcnv3/functions/dcnv3_func.py:104: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record tdata flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  device=device),
/home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/functional.py:568: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at  ../aten/src/ATen/native/TensorShape.cpp:2228.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
/media/shukun/新加卷/code/InternImage/detection/ops_dcnv3/functions/dcnv3_func.py:164: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warninf you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
  spatial_norm = torch.tensor([W_in, H_in]).reshape(1, 1, 1, 2).\
/home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/nn/functional.py:2498: UserWarning: __floordiv__ is deprecated, and its behavior will change in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor').
  _verify_batch_size([input.size(0) * input.size(1) // num_groups, num_groups] + list(input.size()[2:]))
/home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/nn/functional.py:2386: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if size_prods == 1:
/media/shukun/新加卷/code/InternImage/detection/mmdet_custom/models/dense_heads/dino_head.py:72: TracerWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of difent shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results).
  input_img_h, input_img_w = img_metas[0]['img_shape']
/home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/mmdet/models/utils/positional_encoding.py:81: UserWarning: __floordiv__ is deprecated, and its behavior will change in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor').
  dim_t = self.temperature**(2 * (dim_t // 2) / self.num_feats)
/home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/_tensor.py:638: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
  return torch.tensor(other, dtype=dtype, device=self.device) ** self
/media/shukun/新加卷/code/InternImage/detection/mmdet_custom/models/utils/transformer.py:195: TracerWarning: torch.as_tensor results are registered as constants in the trace. You can safely ignore t warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
  spatial_shapes, dtype=torch.long, device=feat_flatten.device)
/media/shukun/新加卷/code/InternImage/detection/mmdet_custom/models/utils/transformer.py:195: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't rrd the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  spatial_shapes, dtype=torch.long, device=feat_flatten.device)
/home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/mmdet/models/utils/transformer.py:848: TracerWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results).
  for lvl, (H, W) in enumerate(spatial_shapes):
/home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/mmdet/models/utils/transformer.py:852: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  0.5, H - 0.5, H, dtype=torch.float32, device=device),
/home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/mmcv/ops/multi_scale_deform_attn.py:312: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert (spatial_shapes[:, 0] * spatial_shapes[:, 1]).sum() == num_value
/home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/mmdeploy/pytorch/functions/topk.py:58: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if k > size:
/media/shukun/新加卷/code/InternImage/detection/mmdet_custom/models/utils/transformer.py:92: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't red the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if reference_points.shape[-1] == 4:
/media/shukun/新加卷/code/InternImage/detection/mmdet_custom/models/utils/transformer.py:51: UserWarning: __floordiv__ is deprecated, and its behavior will change in a future version of pytorch. It rently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor').
  dim_t = 10000**(2 * (dim_t // 2) / (self.embed_dims//2))
/home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/mmdeploy/pytorch/functions/multi_head_attention_forward.py:23: TracerWarning: Converting a tensor to a Python float might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  q = q / math.sqrt(E)
/media/shukun/新加卷/code/InternImage/detection/mmdet_custom/models/dense_heads/dino_head.py:119: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We canrecord the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if reference.shape[-1] == 4:
/media/shukun/新加卷/code/InternImage/detection/mmdet_custom/models/dense_heads/detr_head.py:769: TracerWarning: Using len to get tensor shape might cause the trace to be incorrect. Recommended usagould be tensor.shape[0]. Passing a tensor of different shape might lead to errors or silently give incorrect results.
  assert len(cls_score) == len(bbox_pred)
/home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/mmdeploy/pytorch/functions/mod.py:19: UserWarning: __floordiv__ is deprecated, and its behavior will change in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor').
  return input - (input // other) * other
/media/shukun/新加卷/code/InternImage/detection/mmdet_custom/models/dense_heads/detr_head.py:776: UserWarning: __floordiv__ is deprecated, and its behavior will change in a future version of pytorcht currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor').
  bbox_index = indexes // self.num_classes
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::ScaledDotProductAttentionTRT type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::ScaledDotProductAttentionTRT type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::ScaledDotProductAttentionTRT type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::ScaledDotProductAttentionTRT type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::ScaledDotProductAttentionTRT type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::ScaledDotProductAttentionTRT type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::ScaledDotProductAttentionTRT type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::ScaledDotProductAttentionTRT type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::ScaledDotProductAttentionTRT type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::ScaledDotProductAttentionTRT type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::ScaledDotProductAttentionTRT type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::ScaledDotProductAttentionTRT type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Warning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied.
WARNING: The shape inference of mmdeploy::grid_sampler type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::TRTInstanceNormalization type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
[W shape_type_inference.cpp:429] Warning: Constant folding in symbolic shape inference fails: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! (when checking argument for argument index in method wrapper__index_select)
Exception raised from common_device_check_failure at ../aten/src/ATen/core/adaption.cpp:10 (most recent call first):
frame #0: c10::Error::Error(c10::SourceLocation, std::string) + 0x42 (0x7f79051087d2 in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libc10.so)
frame #1: c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::string const&) + 0x5b (0x7f7905104e6b in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libc10.so)
frame #2: c10::impl::common_device_check_failure(c10::optional<c10::Device>&, at::Tensor const&, char const*, char const*) + 0x37e (0x7f787626b90e in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_cpu.so)
frame #3: <unknown function> + 0x2c7fa4b (0x7f783f3faa4b in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_cuda_cu.so)
frame #4: <unknown function> + 0x2c7fad2 (0x7f783f3faad2 in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_cuda_cu.so)
frame #5: at::_ops::index_select::redispatch(c10::DispatchKeySet, at::Tensor const&, long, at::Tensor const&) + 0xbd (0x7f787692551d in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_cpu.so)
frame #6: <unknown function> + 0x28a08e7 (0x7f78779938e7 in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_cpu.so)
frame #7: <unknown function> + 0x28a0db5 (0x7f7877993db5 in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_cpu.so)
frame #8: at::_ops::index_select::call(at::Tensor const&, long, at::Tensor const&) + 0x14a (0x7f787699604a in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_cpu.so)
frame #9: torch::jit::onnx_constant_fold::runTorchBackendForOnnx(torch::jit::Node const*, std::vector<at::Tensor, std::allocator<at::Tensor> >&, int) + 0x21a7 (0x7f78f12b7d17 in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_python.so)
frame #10: <unknown function> + 0x5c000f (0x7f78f130300f in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_python.so)
frame #11: torch::jit::ONNXShapeTypeInference(torch::jit::Node*, std::map<std::string, c10::IValue, std::less<std::string>, std::allocator<std::pair<std::string const, c10::IValue> > > const&, int) + 0xabe (0x7f78f1307d8e in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_python.so)
frame #12: <unknown function> + 0x5c6992 (0x7f78f1309992 in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_python.so)
frame #13: torch::jit::ONNXShapeTypeInference(std::shared_ptr<torch::jit::Graph>&, std::map<std::string, c10::IValue, std::less<std::string>, std::allocator<std::pair<std::string const, c10::IValue> > > const&, int) + 0x2f (0x7f78f1309f7f in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_python.so)
frame #14: <unknown function> + 0x52f000 (0x7f78f1272000 in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_python.so)
frame #15: <unknown function> + 0x1f73f1 (0x7f78f0f3a3f1 in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_python.so)
frame #16: _PyMethodDef_RawFastCallKeywords + 0x237 (0x4aef97 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #17: /home/shukun/anaconda3/envs/internimage/bin/python() [0x4ae9f0]
frame #18: _PyEval_EvalFrameDefault + 0x468a (0x4ab36a in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #19: _PyEval_EvalCodeWithName + 0x201 (0x4a5a81 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #20: _PyFunction_FastCallKeywords + 0x29c (0x4b9eac in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #21: /home/shukun/anaconda3/envs/internimage/bin/python() [0x4ae8df]
frame #22: _PyEval_EvalFrameDefault + 0x15d6 (0x4a82b6 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #23: _PyEval_EvalCodeWithName + 0x201 (0x4a5a81 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #24: _PyFunction_FastCallDict + 0x2d7 (0x4c0f57 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #25: _PyEval_EvalFrameDefault + 0x1ea4 (0x4a8b84 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #26: _PyEval_EvalCodeWithName + 0x201 (0x4a5a81 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #27: _PyFunction_FastCallDict + 0x2d7 (0x4c0f57 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #28: _PyEval_EvalFrameDefault + 0x1ea4 (0x4a8b84 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #29: _PyEval_EvalCodeWithName + 0x201 (0x4a5a81 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #30: _PyFunction_FastCallKeywords + 0x29c (0x4b9eac in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #31: /home/shukun/anaconda3/envs/internimage/bin/python() [0x4ae8df]
frame #32: _PyEval_EvalFrameDefault + 0x15d6 (0x4a82b6 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #33: _PyEval_EvalCodeWithName + 0x201 (0x4a5a81 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #34: _PyFunction_FastCallKeywords + 0x29c (0x4b9eac in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #35: /home/shukun/anaconda3/envs/internimage/bin/python() [0x4ae8df]
frame #36: _PyEval_EvalFrameDefault + 0x15d6 (0x4a82b6 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #37: _PyEval_EvalCodeWithName + 0x201 (0x4a5a81 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #38: _PyFunction_FastCallKeywords + 0x29c (0x4b9eac in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #39: /home/shukun/anaconda3/envs/internimage/bin/python() [0x4ae8df]
frame #40: _PyEval_EvalFrameDefault + 0x468a (0x4ab36a in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #41: _PyEval_EvalCodeWithName + 0x201 (0x4a5a81 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #42: _PyFunction_FastCallKeywords + 0x29c (0x4b9eac in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #43: /home/shukun/anaconda3/envs/internimage/bin/python() [0x4ae8df]
frame #44: _PyEval_EvalFrameDefault + 0x15d6 (0x4a82b6 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #45: _PyEval_EvalCodeWithName + 0x201 (0x4a5a81 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #46: _PyFunction_FastCallDict + 0x2d7 (0x4c0f57 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #47: _PyEval_EvalFrameDefault + 0x1ea4 (0x4a8b84 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #48: _PyEval_EvalCodeWithName + 0x201 (0x4a5a81 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #49: _PyFunction_FastCallDict + 0x2d7 (0x4c0f57 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #50: _PyObject_Call_Prepend + 0x6e (0x4c6abe in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #51: /home/shukun/anaconda3/envs/internimage/bin/python() [0x578b67]
frame #52: PyObject_Call + 0x60 (0x4c7170 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #53: _PyEval_EvalFrameDefault + 0x1ea4 (0x4a8b84 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #54: _PyEval_EvalCodeWithName + 0x201 (0x4a5a81 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #55: _PyFunction_FastCallDict + 0x2d7 (0x4c0f57 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #56: /home/shukun/anaconda3/envs/internimage/bin/python() [0x4c9a80]
frame #57: PyObject_Call + 0x60 (0x4c7170 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #58: _PyEval_EvalFrameDefault + 0x1ea4 (0x4a8b84 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #59: _PyEval_EvalCodeWithName + 0x201 (0x4a5a81 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #60: _PyFunction_FastCallDict + 0x2d7 (0x4c0f57 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #61: /home/shukun/anaconda3/envs/internimage/bin/python() [0x4c9a80]
frame #62: PyObject_Call + 0x60 (0x4c7170 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #63: _PyEval_EvalFrameDefault + 0x1ea4 (0x4a8b84 in /home/shukun/anaconda3/envs/internimage/bin/python)
 (function ComputeConstantFolding)
[W shape_type_inference.cpp:429] Warning: Constant folding in symbolic shape inference fails: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! (when checking argument for argument index in method wrapper__index_select)
Exception raised from common_device_check_failure at ../aten/src/ATen/core/adaption.cpp:10 (most recent call first):
frame #0: c10::Error::Error(c10::SourceLocation, std::string) + 0x42 (0x7f79051087d2 in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libc10.so)
frame #1: c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::string const&) + 0x5b (0x7f7905104e6b in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libc10.so)
frame #2: c10::impl::common_device_check_failure(c10::optional<c10::Device>&, at::Tensor const&, char const*, char const*) + 0x37e (0x7f787626b90e in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_cpu.so)
frame #3: <unknown function> + 0x2c7fa4b (0x7f783f3faa4b in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_cuda_cu.so)
frame #4: <unknown function> + 0x2c7fad2 (0x7f783f3faad2 in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_cuda_cu.so)
frame #5: at::_ops::index_select::redispatch(c10::DispatchKeySet, at::Tensor const&, long, at::Tensor const&) + 0xbd (0x7f787692551d in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_cpu.so)
frame #6: <unknown function> + 0x28a08e7 (0x7f78779938e7 in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_cpu.so)
frame #7: <unknown function> + 0x28a0db5 (0x7f7877993db5 in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_cpu.so)
frame #8: at::_ops::index_select::call(at::Tensor const&, long, at::Tensor const&) + 0x14a (0x7f787699604a in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_cpu.so)
frame #9: torch::jit::onnx_constant_fold::runTorchBackendForOnnx(torch::jit::Node const*, std::vector<at::Tensor, std::allocator<at::Tensor> >&, int) + 0x21a7 (0x7f78f12b7d17 in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_python.so)
frame #10: <unknown function> + 0x5c000f (0x7f78f130300f in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_python.so)
frame #11: torch::jit::ONNXShapeTypeInference(torch::jit::Node*, std::map<std::string, c10::IValue, std::less<std::string>, std::allocator<std::pair<std::string const, c10::IValue> > > const&, int) + 0xabe (0x7f78f1307d8e in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_python.so)
frame #12: <unknown function> + 0x5c6992 (0x7f78f1309992 in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_python.so)
frame #13: torch::jit::ONNXShapeTypeInference(std::shared_ptr<torch::jit::Graph>&, std::map<std::string, c10::IValue, std::less<std::string>, std::allocator<std::pair<std::string const, c10::IValue> > > const&, int) + 0x2f (0x7f78f1309f7f in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_python.so)
frame #14: <unknown function> + 0x52f000 (0x7f78f1272000 in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_python.so)
frame #15: <unknown function> + 0x1f73f1 (0x7f78f0f3a3f1 in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_python.so)
frame #16: _PyMethodDef_RawFastCallKeywords + 0x237 (0x4aef97 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #17: /home/shukun/anaconda3/envs/internimage/bin/python() [0x4ae9f0]
frame #18: _PyEval_EvalFrameDefault + 0x468a (0x4ab36a in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #19: _PyEval_EvalCodeWithName + 0x201 (0x4a5a81 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #20: _PyFunction_FastCallKeywords + 0x29c (0x4b9eac in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #21: /home/shukun/anaconda3/envs/internimage/bin/python() [0x4ae8df]
frame #22: _PyEval_EvalFrameDefault + 0x15d6 (0x4a82b6 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #23: _PyEval_EvalCodeWithName + 0x201 (0x4a5a81 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #24: _PyFunction_FastCallDict + 0x2d7 (0x4c0f57 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #25: _PyEval_EvalFrameDefault + 0x1ea4 (0x4a8b84 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #26: _PyEval_EvalCodeWithName + 0x201 (0x4a5a81 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #27: _PyFunction_FastCallDict + 0x2d7 (0x4c0f57 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #28: _PyEval_EvalFrameDefault + 0x1ea4 (0x4a8b84 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #29: _PyEval_EvalCodeWithName + 0x201 (0x4a5a81 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #30: _PyFunction_FastCallKeywords + 0x29c (0x4b9eac in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #31: /home/shukun/anaconda3/envs/internimage/bin/python() [0x4ae8df]
frame #32: _PyEval_EvalFrameDefault + 0x15d6 (0x4a82b6 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #33: _PyEval_EvalCodeWithName + 0x201 (0x4a5a81 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #34: _PyFunction_FastCallKeywords + 0x29c (0x4b9eac in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #35: /home/shukun/anaconda3/envs/internimage/bin/python() [0x4ae8df]
frame #36: _PyEval_EvalFrameDefault + 0x15d6 (0x4a82b6 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #37: _PyEval_EvalCodeWithName + 0x201 (0x4a5a81 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #38: _PyFunction_FastCallKeywords + 0x29c (0x4b9eac in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #39: /home/shukun/anaconda3/envs/internimage/bin/python() [0x4ae8df]
frame #40: _PyEval_EvalFrameDefault + 0x468a (0x4ab36a in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #41: _PyEval_EvalCodeWithName + 0x201 (0x4a5a81 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #42: _PyFunction_FastCallKeywords + 0x29c (0x4b9eac in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #43: /home/shukun/anaconda3/envs/internimage/bin/python() [0x4ae8df]
frame #44: _PyEval_EvalFrameDefault + 0x15d6 (0x4a82b6 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #45: _PyEval_EvalCodeWithName + 0x201 (0x4a5a81 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #46: _PyFunction_FastCallDict + 0x2d7 (0x4c0f57 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #47: _PyEval_EvalFrameDefault + 0x1ea4 (0x4a8b84 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #48: _PyEval_EvalCodeWithName + 0x201 (0x4a5a81 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #49: _PyFunction_FastCallDict + 0x2d7 (0x4c0f57 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #50: _PyObject_Call_Prepend + 0x6e (0x4c6abe in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #51: /home/shukun/anaconda3/envs/internimage/bin/python() [0x578b67]
frame #52: PyObject_Call + 0x60 (0x4c7170 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #53: _PyEval_EvalFrameDefault + 0x1ea4 (0x4a8b84 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #54: _PyEval_EvalCodeWithName + 0x201 (0x4a5a81 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #55: _PyFunction_FastCallDict + 0x2d7 (0x4c0f57 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #56: /home/shukun/anaconda3/envs/internimage/bin/python() [0x4c9a80]
frame #57: PyObject_Call + 0x60 (0x4c7170 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #58: _PyEval_EvalFrameDefault + 0x1ea4 (0x4a8b84 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #59: _PyEval_EvalCodeWithName + 0x201 (0x4a5a81 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #60: _PyFunction_FastCallDict + 0x2d7 (0x4c0f57 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #61: /home/shukun/anaconda3/envs/internimage/bin/python() [0x4c9a80]
frame #62: PyObject_Call + 0x60 (0x4c7170 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #63: _PyEval_EvalFrameDefault + 0x1ea4 (0x4a8b84 in /home/shukun/anaconda3/envs/internimage/bin/python)
 (function ComputeConstantFolding)
[W shape_type_inference.cpp:429] Warning: Constant folding in symbolic shape inference fails: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! (when checking argument for argument index in method wrapper__index_select)
Exception raised from common_device_check_failure at ../aten/src/ATen/core/adaption.cpp:10 (most recent call first):
frame #0: c10::Error::Error(c10::SourceLocation, std::string) + 0x42 (0x7f79051087d2 in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libc10.so)
frame #1: c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::string const&) + 0x5b (0x7f7905104e6b in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libc10.so)
frame #2: c10::impl::common_device_check_failure(c10::optional<c10::Device>&, at::Tensor const&, char const*, char const*) + 0x37e (0x7f787626b90e in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_cpu.so)
frame #3: <unknown function> + 0x2c7fa4b (0x7f783f3faa4b in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_cuda_cu.so)
frame #4: <unknown function> + 0x2c7fad2 (0x7f783f3faad2 in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_cuda_cu.so)
frame #5: at::_ops::index_select::redispatch(c10::DispatchKeySet, at::Tensor const&, long, at::Tensor const&) + 0xbd (0x7f787692551d in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_cpu.so)
frame #6: <unknown function> + 0x28a08e7 (0x7f78779938e7 in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_cpu.so)
frame #7: <unknown function> + 0x28a0db5 (0x7f7877993db5 in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_cpu.so)
frame #8: at::_ops::index_select::call(at::Tensor const&, long, at::Tensor const&) + 0x14a (0x7f787699604a in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_cpu.so)
frame #9: torch::jit::onnx_constant_fold::runTorchBackendForOnnx(torch::jit::Node const*, std::vector<at::Tensor, std::allocator<at::Tensor> >&, int) + 0x21a7 (0x7f78f12b7d17 in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_python.so)
frame #10: <unknown function> + 0x5c000f (0x7f78f130300f in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_python.so)
frame #11: torch::jit::ONNXShapeTypeInference(torch::jit::Node*, std::map<std::string, c10::IValue, std::less<std::string>, std::allocator<std::pair<std::string const, c10::IValue> > > const&, int) + 0xabe (0x7f78f1307d8e in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_python.so)
frame #12: <unknown function> + 0x5c6992 (0x7f78f1309992 in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_python.so)
frame #13: torch::jit::ONNXShapeTypeInference(std::shared_ptr<torch::jit::Graph>&, std::map<std::string, c10::IValue, std::less<std::string>, std::allocator<std::pair<std::string const, c10::IValue> > > const&, int) + 0x2f (0x7f78f1309f7f in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_python.so)
frame #14: <unknown function> + 0x52f000 (0x7f78f1272000 in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_python.so)
frame #15: <unknown function> + 0x1f73f1 (0x7f78f0f3a3f1 in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_python.so)
frame #16: _PyMethodDef_RawFastCallKeywords + 0x237 (0x4aef97 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #17: /home/shukun/anaconda3/envs/internimage/bin/python() [0x4ae9f0]
frame #18: _PyEval_EvalFrameDefault + 0x468a (0x4ab36a in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #19: _PyEval_EvalCodeWithName + 0x201 (0x4a5a81 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #20: _PyFunction_FastCallKeywords + 0x29c (0x4b9eac in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #21: /home/shukun/anaconda3/envs/internimage/bin/python() [0x4ae8df]
frame #22: _PyEval_EvalFrameDefault + 0x15d6 (0x4a82b6 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #23: _PyEval_EvalCodeWithName + 0x201 (0x4a5a81 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #24: _PyFunction_FastCallDict + 0x2d7 (0x4c0f57 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #25: _PyEval_EvalFrameDefault + 0x1ea4 (0x4a8b84 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #26: _PyEval_EvalCodeWithName + 0x201 (0x4a5a81 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #27: _PyFunction_FastCallDict + 0x2d7 (0x4c0f57 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #28: _PyEval_EvalFrameDefault + 0x1ea4 (0x4a8b84 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #29: _PyEval_EvalCodeWithName + 0x201 (0x4a5a81 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #30: _PyFunction_FastCallKeywords + 0x29c (0x4b9eac in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #31: /home/shukun/anaconda3/envs/internimage/bin/python() [0x4ae8df]
frame #32: _PyEval_EvalFrameDefault + 0x15d6 (0x4a82b6 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #33: _PyEval_EvalCodeWithName + 0x201 (0x4a5a81 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #34: _PyFunction_FastCallKeywords + 0x29c (0x4b9eac in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #35: /home/shukun/anaconda3/envs/internimage/bin/python() [0x4ae8df]
frame #36: _PyEval_EvalFrameDefault + 0x15d6 (0x4a82b6 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #37: _PyEval_EvalCodeWithName + 0x201 (0x4a5a81 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #38: _PyFunction_FastCallKeywords + 0x29c (0x4b9eac in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #39: /home/shukun/anaconda3/envs/internimage/bin/python() [0x4ae8df]
frame #40: _PyEval_EvalFrameDefault + 0x468a (0x4ab36a in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #41: _PyEval_EvalCodeWithName + 0x201 (0x4a5a81 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #42: _PyFunction_FastCallKeywords + 0x29c (0x4b9eac in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #43: /home/shukun/anaconda3/envs/internimage/bin/python() [0x4ae8df]
frame #44: _PyEval_EvalFrameDefault + 0x15d6 (0x4a82b6 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #45: _PyEval_EvalCodeWithName + 0x201 (0x4a5a81 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #46: _PyFunction_FastCallDict + 0x2d7 (0x4c0f57 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #47: _PyEval_EvalFrameDefault + 0x1ea4 (0x4a8b84 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #48: _PyEval_EvalCodeWithName + 0x201 (0x4a5a81 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #49: _PyFunction_FastCallDict + 0x2d7 (0x4c0f57 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #50: _PyObject_Call_Prepend + 0x6e (0x4c6abe in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #51: /home/shukun/anaconda3/envs/internimage/bin/python() [0x578b67]
frame #52: PyObject_Call + 0x60 (0x4c7170 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #53: _PyEval_EvalFrameDefault + 0x1ea4 (0x4a8b84 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #54: _PyEval_EvalCodeWithName + 0x201 (0x4a5a81 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #55: _PyFunction_FastCallDict + 0x2d7 (0x4c0f57 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #56: /home/shukun/anaconda3/envs/internimage/bin/python() [0x4c9a80]
frame #57: PyObject_Call + 0x60 (0x4c7170 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #58: _PyEval_EvalFrameDefault + 0x1ea4 (0x4a8b84 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #59: _PyEval_EvalCodeWithName + 0x201 (0x4a5a81 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #60: _PyFunction_FastCallDict + 0x2d7 (0x4c0f57 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #61: /home/shukun/anaconda3/envs/internimage/bin/python() [0x4c9a80]
frame #62: PyObject_Call + 0x60 (0x4c7170 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #63: _PyEval_EvalFrameDefault + 0x1ea4 (0x4a8b84 in /home/shukun/anaconda3/envs/internimage/bin/python)
 (function ComputeConstantFolding)
[W shape_type_inference.cpp:429] Warning: Constant folding in symbolic shape inference fails: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! (when checking argument for argument index in method wrapper__index_select)
Exception raised from common_device_check_failure at ../aten/src/ATen/core/adaption.cpp:10 (most recent call first):
frame #0: c10::Error::Error(c10::SourceLocation, std::string) + 0x42 (0x7f79051087d2 in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libc10.so)
frame #1: c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::string const&) + 0x5b (0x7f7905104e6b in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libc10.so)
frame #2: c10::impl::common_device_check_failure(c10::optional<c10::Device>&, at::Tensor const&, char const*, char const*) + 0x37e (0x7f787626b90e in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_cpu.so)
frame #3: <unknown function> + 0x2c7fa4b (0x7f783f3faa4b in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_cuda_cu.so)
frame #4: <unknown function> + 0x2c7fad2 (0x7f783f3faad2 in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_cuda_cu.so)
frame #5: at::_ops::index_select::redispatch(c10::DispatchKeySet, at::Tensor const&, long, at::Tensor const&) + 0xbd (0x7f787692551d in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_cpu.so)
frame #6: <unknown function> + 0x28a08e7 (0x7f78779938e7 in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_cpu.so)
frame #7: <unknown function> + 0x28a0db5 (0x7f7877993db5 in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_cpu.so)
frame #8: at::_ops::index_select::call(at::Tensor const&, long, at::Tensor const&) + 0x14a (0x7f787699604a in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_cpu.so)
frame #9: torch::jit::onnx_constant_fold::runTorchBackendForOnnx(torch::jit::Node const*, std::vector<at::Tensor, std::allocator<at::Tensor> >&, int) + 0x21a7 (0x7f78f12b7d17 in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_python.so)
frame #10: <unknown function> + 0x5c000f (0x7f78f130300f in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_python.so)
frame #11: torch::jit::ONNXShapeTypeInference(torch::jit::Node*, std::map<std::string, c10::IValue, std::less<std::string>, std::allocator<std::pair<std::string const, c10::IValue> > > const&, int) + 0xabe (0x7f78f1307d8e in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_python.so)
frame #12: <unknown function> + 0x5c6992 (0x7f78f1309992 in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_python.so)
frame #13: torch::jit::ONNXShapeTypeInference(std::shared_ptr<torch::jit::Graph>&, std::map<std::string, c10::IValue, std::less<std::string>, std::allocator<std::pair<std::string const, c10::IValue> > > const&, int) + 0x2f (0x7f78f1309f7f in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_python.so)
frame #14: <unknown function> + 0x52f000 (0x7f78f1272000 in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_python.so)
frame #15: <unknown function> + 0x1f73f1 (0x7f78f0f3a3f1 in /home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/lib/libtorch_python.so)
frame #16: _PyMethodDef_RawFastCallKeywords + 0x237 (0x4aef97 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #17: /home/shukun/anaconda3/envs/internimage/bin/python() [0x4ae9f0]
frame #18: _PyEval_EvalFrameDefault + 0x468a (0x4ab36a in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #19: _PyEval_EvalCodeWithName + 0x201 (0x4a5a81 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #20: _PyFunction_FastCallKeywords + 0x29c (0x4b9eac in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #21: /home/shukun/anaconda3/envs/internimage/bin/python() [0x4ae8df]
frame #22: _PyEval_EvalFrameDefault + 0x15d6 (0x4a82b6 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #23: _PyEval_EvalCodeWithName + 0x201 (0x4a5a81 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #24: _PyFunction_FastCallDict + 0x2d7 (0x4c0f57 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #25: _PyEval_EvalFrameDefault + 0x1ea4 (0x4a8b84 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #26: _PyEval_EvalCodeWithName + 0x201 (0x4a5a81 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #27: _PyFunction_FastCallDict + 0x2d7 (0x4c0f57 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #28: _PyEval_EvalFrameDefault + 0x1ea4 (0x4a8b84 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #29: _PyEval_EvalCodeWithName + 0x201 (0x4a5a81 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #30: _PyFunction_FastCallKeywords + 0x29c (0x4b9eac in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #31: /home/shukun/anaconda3/envs/internimage/bin/python() [0x4ae8df]
frame #32: _PyEval_EvalFrameDefault + 0x15d6 (0x4a82b6 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #33: _PyEval_EvalCodeWithName + 0x201 (0x4a5a81 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #34: _PyFunction_FastCallKeywords + 0x29c (0x4b9eac in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #35: /home/shukun/anaconda3/envs/internimage/bin/python() [0x4ae8df]
frame #36: _PyEval_EvalFrameDefault + 0x15d6 (0x4a82b6 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #37: _PyEval_EvalCodeWithName + 0x201 (0x4a5a81 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #38: _PyFunction_FastCallKeywords + 0x29c (0x4b9eac in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #39: /home/shukun/anaconda3/envs/internimage/bin/python() [0x4ae8df]
frame #40: _PyEval_EvalFrameDefault + 0x468a (0x4ab36a in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #41: _PyEval_EvalCodeWithName + 0x201 (0x4a5a81 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #42: _PyFunction_FastCallKeywords + 0x29c (0x4b9eac in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #43: /home/shukun/anaconda3/envs/internimage/bin/python() [0x4ae8df]
frame #44: _PyEval_EvalFrameDefault + 0x15d6 (0x4a82b6 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #45: _PyEval_EvalCodeWithName + 0x201 (0x4a5a81 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #46: _PyFunction_FastCallDict + 0x2d7 (0x4c0f57 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #47: _PyEval_EvalFrameDefault + 0x1ea4 (0x4a8b84 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #48: _PyEval_EvalCodeWithName + 0x201 (0x4a5a81 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #49: _PyFunction_FastCallDict + 0x2d7 (0x4c0f57 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #50: _PyObject_Call_Prepend + 0x6e (0x4c6abe in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #51: /home/shukun/anaconda3/envs/internimage/bin/python() [0x578b67]
frame #52: PyObject_Call + 0x60 (0x4c7170 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #53: _PyEval_EvalFrameDefault + 0x1ea4 (0x4a8b84 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #54: _PyEval_EvalCodeWithName + 0x201 (0x4a5a81 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #55: _PyFunction_FastCallDict + 0x2d7 (0x4c0f57 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #56: /home/shukun/anaconda3/envs/internimage/bin/python() [0x4c9a80]
frame #57: PyObject_Call + 0x60 (0x4c7170 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #58: _PyEval_EvalFrameDefault + 0x1ea4 (0x4a8b84 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #59: _PyEval_EvalCodeWithName + 0x201 (0x4a5a81 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #60: _PyFunction_FastCallDict + 0x2d7 (0x4c0f57 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #61: /home/shukun/anaconda3/envs/internimage/bin/python() [0x4c9a80]
frame #62: PyObject_Call + 0x60 (0x4c7170 in /home/shukun/anaconda3/envs/internimage/bin/python)
frame #63: _PyEval_EvalFrameDefault + 0x1ea4 (0x4a8b84 in /home/shukun/anaconda3/envs/internimage/bin/python)
 (function ComputeConstantFolding)
WARNING: The shape inference of prim::PythonOp type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::PythonOp type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::PythonOp type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::PythonOp type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::PythonOp type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::PythonOp type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::ScaledDotProductAttentionTRT type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::ScaledDotProductAttentionTRT type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::PythonOp type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::ScaledDotProductAttentionTRT type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::ScaledDotProductAttentionTRT type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::PythonOp type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::ScaledDotProductAttentionTRT type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::ScaledDotProductAttentionTRT type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::PythonOp type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::ScaledDotProductAttentionTRT type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::ScaledDotProductAttentionTRT type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::PythonOp type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::ScaledDotProductAttentionTRT type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::ScaledDotProductAttentionTRT type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::PythonOp type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::ScaledDotProductAttentionTRT type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of mmdeploy::ScaledDotProductAttentionTRT type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::PythonOp type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
Process Process-2:
Traceback (most recent call last):
  File "/home/shukun/anaconda3/envs/internimage/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/home/shukun/anaconda3/envs/internimage/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "/home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/mmdeploy/apis/core/pipeline_manager.py", line 107, in __call__
    ret = func(*args, **kwargs)
  File "/home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/mmdeploy/apis/pytorch2onnx.py", line 111, in torch2onnx
    optimize=optimize)
  File "/home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/mmdeploy/apis/core/pipeline_manager.py", line 356, in _wrap
    return self.call_function(func_name_, *args, **kwargs)
  File "/home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/mmdeploy/apis/core/pipeline_manager.py", line 326, in call_function
    return self.call_function_local(func_name, *args, **kwargs)
  File "/home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/mmdeploy/apis/core/pipeline_manager.py", line 275, in call_function_local
    return pipe_caller(*args, **kwargs)
  File "/home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/mmdeploy/apis/core/pipeline_manager.py", line 107, in __call__
    ret = func(*args, **kwargs)
  File "/home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/mmdeploy/apis/onnx/export.py", line 132, in export
    verbose=verbose)
  File "/home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/onnx/__init__.py", line 309, in export
    export_modules_as_functions)
  File "/home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/onnx/utils.py", line 122, in export
    custom_opsets=custom_opsets, export_modules_as_functions=export_modules_as_functions)
  File "/home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/onnx/utils.py", line 724, in _export
    dynamic_axes=dynamic_axes)
  File "/home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/mmdeploy/core/rewriters/rewriter_utils.py", line 402, in wrapper
    return self.func(self, *args, **kwargs)
  File "/home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/mmdeploy/apis/onnx/optimizer.py", line 10, in model_to_graph__custom_optimizer
    graph, params_dict, torch_out = ctx.origin_func(*args, **kwargs)
  File "/home/shukun/anaconda3/envs/internimage/lib/python3.7/site-packages/torch/onnx/utils.py", line 538, in _model_to_graph
    _export_onnx_opset_version)
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! (when checking argument for argument index in method wrapper__index_select)
2023-12-08 14:13:31,620 - mmdeploy - ERROR - `mmdeploy.apis.pytorch2onnx.torch2onnx` with Call id: 0 failed. exit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@goldentimecoolk