ONNX model generation

[ayazhassan]

Can you please convert your model into the ONNX model? I want to test it on tensor rt for inferencing.
I am trying to convert it to the ONNX model but getting the following error:
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0, and cpu! (when checking argument for argument index in method wrapper__index_select)

[ayazhassan]

It seems that some of the model layers is running on CPU even I set the device as GPU. Following is the profiling screenshot. Let me know this is correct. How can we run the complete transformer model (generating all tensors) on GPU?

[sazani]

I have faced the same issue