[Caffe2] ONNX Caffe2Backend.prepare() initializes input as float64 #14104
Description
🐛 Bug
When working with an ONNX model, preparing the backend automatically initializes the graph inputs as float64, which then throws out the following warning when specifying a CUDA device:
CUDA operators do not support 64-bit doubles, please use arr.astype(np.float32) or np.int32 for ints. Blob: 0 type: float64
This is because numpy.ones has default datatype numpy.float64.
To Reproduce
Steps to reproduce the behavior:
In my case, I initially converted a pre-trained PyTorch model to ONNX with torch.onnx._export(), but I believe this warning should still appear when working with any ONNX model.
- Convert PyTorch model to ONNX with
torch.onnx._export(torch_model, example_input, 'model.onnx', export_params=True) - Load ONNX model with onnx:
onnx.load('model.onnx') - Prepare the caffe2 backend:
caffe2.python.onnx.backend.prepare(model, device='CUDA:0')
Sample code:
# Prepare backend
print('Preparing backend...')
prep_t0 = time.time()
prepared_backend = caffe2.python.onnx.backend.prepare(model, device='CUDA:0')
print('Backend prepared in {} seconds'.format(time.time() - prep_t0))
# Run the ONNX model with Caffe2
print('Running inference..')
fwd_t0 = time.time()
outputs = prepared_backend.run(img_arr.astype(np.float32))[0]
print('Foward pass time: {} seconds'.format(time.time() - fwd_t0))Outputs:
Preparing backend...
CUDA operators do not support 64-bit doubles, please use arr.astype(np.float32) or np.int32 for ints. Blob: 0 type: float64
Backend prepared in 1.4734389781951904 seconds
Running inference..
Foward pass time: 0.04503035545349121 seconds
Expected behavior
The behavior is as expected apart from the warning. The model still seems to be loaded on GPU and inference is executed on GPU as well. But I was wondering why I was getting a warning for such a simple task.
Environment
- PyTorch Version (e.g., 1.0): 1.0.0.dev20181105
- OS (e.g., Linux): Linux
- How you installed PyTorch (
conda,pip, source): conda - Python version: 3.6.6
- CUDA/cuDNN version: 9.2.148/7104
- GPU models and configuration: GeForce GTX 1080 Ti with drivers 410.73
Additional context
As mentioned above, this error is thrown when preparing the caffe2 backend for CUDA device, and I managed to remove the warning by modifying the source here:
@classmethod
def _direct_initialize_inputs(cls, inputs, initialized, ws, device_option):
for value_info in inputs:
if value_info.name in initialized:
continue
shape = list(d.dim_value for d in value_info.type.tensor_type.shape.dim)
ws.FeedBlob(value_info.name, np.ones(shape, dtype=np.float32), device_option)Unless this is the expected behavior? But I figured since it's just the initialization process for the graph inputs, and they're initialized to ones, I really don't see a particular reason as to why it cannot be float32 instead.
Thanks