I am trying to export just the base model to onnx but hitting a wall.
Loading pretrain weights
Exporting model to ONNX format
UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at /pytorch/aten/src/ATen/native/TensorShape.cpp:4317.)
PyTorch inference output shapes - Boxes: torch.Size([1, 3900, 4]), Labels: torch.Size([1, 3900, 91])
W1126 16:49:01.465000 97506 venv/lib/python3.10/site-packages/torch/onnx/_internal/exporter/_compat.py:114] Setting ONNX exporter to use operator set version 18 because the requested opset_version 17 is a lower version than we have implementations for. Automatic version conversion will be performed, which may not be successful at converting to the requested version. If version conversion is unsuccessful, the opset version of the exported model will be kept at 18. Please consider setting opset_version >=18 to leverage latest ONNX features
[torch.onnx] Obtain model graph for `LWDETR([...]` with `torch.export.export(..., strict=False)`...
[torch.onnx] Obtain model graph for `LWDETR([...]` with `torch.export.export(..., strict=False)`... ❌
[torch.onnx] Obtain model graph for `LWDETR([...]` with `torch.export.export(..., strict=True)`...
[torch.onnx] Obtain model graph for `LWDETR([...]` with `torch.export.export(..., strict=True)`... ❌
Traceback (most recent call last):
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/onnx/_internal/exporter/_capture_strategies.py", line 118, in __call__
exported_program = self._capture(model, args, kwargs, dynamic_shapes)
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/onnx/_internal/exporter/_capture_strategies.py", line 210, in _capture
return torch.export.export(
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/export/__init__.py", line 311, in export
raise e
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/export/__init__.py", line 277, in export
return _export(
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/export/_trace.py", line 1163, in wrapper
raise e
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/export/_trace.py", line 1129, in wrapper
ep = fn(*args, **kwargs)
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/export/exported_program.py", line 124, in wrapper
return fn(*args, **kwargs)
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/export/_trace.py", line 2255, in _export
ep = _export_for_training(
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/export/_trace.py", line 1163, in wrapper
raise e
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/export/_trace.py", line 1129, in wrapper
ep = fn(*args, **kwargs)
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/export/exported_program.py", line 124, in wrapper
return fn(*args, **kwargs)
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/export/_trace.py", line 2071, in _export_for_training
export_artifact = export_func(
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/export/_trace.py", line 2002, in _non_strict_export
aten_export_artifact = _to_aten_func( # type: ignore[operator]
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/export/_trace.py", line 1793, in _export_to_aten_ir_make_fx
gm, graph_signature = transform(_make_fx_helper)(
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/export/_trace.py", line 1922, in _aot_export_non_strict
gm, sig = aot_export(wrapped_mod, args, kwargs=kwargs, **flags)
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/export/_trace.py", line 1706, in _make_fx_helper
gm = make_fx(
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/fx/experimental/proxy_tensor.py", line 2429, in wrapped
return make_fx_tracer.trace(f, *args)
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/fx/experimental/proxy_tensor.py", line 2356, in trace
return self._trace_inner(f, *args)
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/fx/experimental/proxy_tensor.py", line 2318, in _trace_inner
t = dispatch_trace(
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/_compile.py", line 53, in inner
return disable_fn(*args, **kwargs)
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 1044, in _fn
return fn(*args, **kwargs)
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/fx/experimental/proxy_tensor.py", line 1303, in dispatch_trace
graph = tracer.trace(root, concrete_args) # type: ignore[arg-type]
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/fx/experimental/proxy_tensor.py", line 1908, in trace
res = super().trace(root, concrete_args)
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/fx/_symbolic_trace.py", line 868, in trace
(self.create_arg(fn(*args)),),
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/fx/experimental/proxy_tensor.py", line 1361, in wrapped
out = f(*tensors) # type:ignore[call-arg]
File "<string>", line 1, in <lambda>
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/export/_trace.py", line 1593, in wrapped_fn
return tuple(flat_fn(*args))
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/_functorch/_aot_autograd/utils.py", line 187, in flat_fn
tree_out = fn(*args, **kwargs)
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/_functorch/_aot_autograd/graph_capture_wrappers.py", line 1354, in functional_call
out = mod(*args[params_len:], **kwargs)
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/fx/_symbolic_trace.py", line 843, in module_call_wrapper
return self.call_module(mod, forward, args, kwargs)
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/fx/experimental/proxy_tensor.py", line 1997, in call_module
return Tracer.call_module(self, m, forward, args, kwargs)
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/fx/_symbolic_trace.py", line 560, in call_module
ret_val = forward(*args, **kwargs)
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/fx/_symbolic_trace.py", line 836, in forward
return _orig_module_call(mod, *args, **kwargs)
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1775, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1786, in _call_impl
return forward_call(*args, **kwargs)
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/export/_trace.py", line 1906, in forward
tree_out = mod(*args, **kwargs)
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/fx/_symbolic_trace.py", line 843, in module_call_wrapper
return self.call_module(mod, forward, args, kwargs)
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/fx/experimental/proxy_tensor.py", line 1997, in call_module
return Tracer.call_module(self, m, forward, args, kwargs)
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/fx/_symbolic_trace.py", line 560, in call_module
ret_val = forward(*args, **kwargs)
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/fx/_symbolic_trace.py", line 836, in forward
return _orig_module_call(mod, *args, **kwargs)
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1775, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1786, in _call_impl
return forward_call(*args, **kwargs)
File "/home/magni/rf-detr/rfdetr/models/lwdetr.py", line 222, in forward_export
hs, ref_unsigmoid, hs_enc, ref_enc = self.transformer(
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/fx/_symbolic_trace.py", line 843, in module_call_wrapper
return self.call_module(mod, forward, args, kwargs)
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/fx/experimental/proxy_tensor.py", line 1997, in call_module
return Tracer.call_module(self, m, forward, args, kwargs)
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/fx/_symbolic_trace.py", line 560, in call_module
ret_val = forward(*args, **kwargs)
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/fx/_symbolic_trace.py", line 836, in forward
return _orig_module_call(mod, *args, **kwargs)
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1775, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1786, in _call_impl
return forward_call(*args, **kwargs)
File "/home/magni/rf-detr/rfdetr/models/transformer.py", line 226, in forward
output_memory, output_proposals = gen_encoder_output_proposals(
File "/home/magni/rf-detr/rfdetr/models/transformer.py", line 92, in gen_encoder_output_proposals
valid_H = torch.tensor([H_ for _ in range(N_)], device=memory.device)
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/fx/experimental/proxy_tensor.py", line 1409, in __torch_function__
return func(*args, **kwargs)
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/fx/experimental/proxy_tensor.py", line 1479, in __torch_function__
return func(*args, **kwargs)
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/_export/non_strict_utils.py", line 1066, in __torch_function__
return func(*args, **kwargs)
RuntimeError: The tensor has a non-zero number of elements, but its data is not allocated yet.
If you're using torch.compile/export/fx, it is likely that we are erroneously tracing into a custom kernel. To fix this, please wrap the custom kernel into an opaque custom op. Please see the following for details: https://pytorch.org/tutorials/advanced/custom_ops_landing_page.html
If you're using Caffe2, Caffe2 uses a lazy allocation, so you will need to call mutable_data() or raw_mutable_data() to actually allocate memory.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/magni/rf-detr/export.py", line 5, in <module>
model.export()
File "/home/magni/rf-detr/rfdetr/detr.py", line 124, in export
self.model.export(**kwargs)
File "/home/magni/rf-detr/rfdetr/main.py", line 565, in export
output_file = export_onnx(
File "/home/magni/rf-detr/rfdetr/deploy/export.py", line 78, in export_onnx
torch.onnx.export(
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/onnx/__init__.py", line 296, in export
return _compat.export_compat(
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/onnx/_internal/exporter/_compat.py", line 143, in export_compat
onnx_program = _core.export(
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/onnx/_internal/exporter/_flags.py", line 23, in wrapper
return func(*args, **kwargs)
File "/home/magni/rf-detr/venv/lib/python3.10/site-packages/torch/onnx/_internal/exporter/_core.py", line 1385, in export
raise _errors.TorchExportError(
torch.onnx._internal.exporter._errors.TorchExportError: Failed to export the model with torch.export. This is step 1/3 of exporting the model to ONNX. Next steps:
- Modify the model code for `torch.export.export` to succeed. Refer to https://pytorch.org/docs/stable/generated/exportdb/index.html for more information.
- Debug `torch.export.export` and submit a PR to PyTorch.
- Create an issue in the PyTorch GitHub repository against the *torch.export* component and attach the full error stack as well as reproduction scripts.
## Exception summary
<class 'RuntimeError'>: The tensor has a non-zero number of elements, but its data is not allocated yet.
If you're using torch.compile/export/fx, it is likely that we are erroneously tracing into a custom kernel. To fix this, please wrap the custom kernel into an opaque custom op. Please see the following for details: https://pytorch.org/tutorials/advanced/custom_ops_landing_page.html
If you're using Caffe2, Caffe2 uses a lazy allocation, so you will need to call mutable_data() or raw_mutable_data() to actually allocate memory.
(Refer to the full stack trace above for more information.)
Search before asking
Bug
I am trying to export just the base model to onnx but hitting a wall.
Environment
RF-DETR: latest commit on develop branch
OS: Ubuntu 24.04
Python version: python3.10
PyTorch version: did the following to get the packages
CUDA/cuDNN version: 12.8
GPU/CPU hardware: NVIDIA GeForce RTX 4070 and 13th Gen Intel(R) Core(TM) i9-13900H
Minimal Reproducible Example
Additional
this is the error I get:
Are you willing to submit a PR?