-
Notifications
You must be signed in to change notification settings - Fork 126
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Convert Model to TensorRT #16
Comments
I'm not sure if the team has any ready solution you could use, but if not, I have a backup suggestion for you. OneFormer is built with Detectron2. I deployed a similar segmentation model in the past, and I used this script to convert, and it worked. |
Hi @abhigoku10, thanks for your interest in our work. We don't have an already written script to convert the model to TensorRT or ONNX format. I think the conversion should be pretty straightforward. You can try following the script shared by @SkalskiP (thanks for sharing!). If you need any assistance from our side, please let us know. |
Feel free to re-open the issue if you need any help. |
i've managed to export the model to onnx, there's a few issues at the moment to get it working
With those changes the export_model.py script worked with a few small changes and I was able to use the model in onnxruntime. |
Hi @tomhog, could you please share the changes you did for exporting to onnx? Thanks a ton! |
Certainly, though as I said it's a bit of a mess at the moment. I will tidy it up, I just needed to prove it can work before moving forward. Also worth noting I'm not sure if the onnx file would work on GPU, not tried yet. hack to ms_deform_attn so it doesn't use the custom op (and why it might not work on gpu) Changes to oneformer model to handle missing task in input dict, and only output one task type script to export the onnx file Export command would look something like
Currently the dimensions of the sample image need to match the dimensions the model was trained at. I think the rest of that commit it just noise (I was having odd issues with pybind11) and a few of my own bash scripts for convenience. If it works for you please let me know and maybe we could work on a cleaner approach to add official support to OneFormer. Tom |
@tomhog have you tested gpu support yet? |
I'm very new to ML, so I apologize if my question is naive. Is it possible to convert this model to PyTorch Lite? I'm curious to see if it's possible to use this model with https://playtorch.dev/. I was hoping to follow this: https://playtorch.dev/docs/tutorials/prepare-custom-model/ But I don't think it's that straightforward. I would appreciate any guidance if possible! Thank you. |
Hi @AAAstorga, thanks for your interest in our work. You should follow the tutorial on using DETR with PlayTorch, as OneFormer and DETR are both built using detectron2. Also, it might be better to create a new issue for PlayTorch. |
Thank you @praeclarumjj3 - I appreciate the response. Do you have any directions on how to load the pretrained model in Python with PyTorch so I can convert it to a mobile friendly version? This might sound like a simple question, but I'm just starting to learn about all of this. Thank you! |
You can refer PyTorch tutorials: https://pytorch.org/tutorials/ |
@tomhog I'm using your code to convert onnx and I'm having this problem is there any way to fix it look forward to your reply! |
@tomhog Hi, sorry to bother you. Have you ever encountered the problem of significant drop in accuracy when using the onnx model to predict? I compared the output with the same input, the difference is in MSDeformAttnFunction.apply and ms_deform_attn_core_pytorch. Obviously the accuracy is slightly worse when using CPU, but it is directly 0 when using CUDA. Have you encountered this problem? Or can you provide some suggestions? |
Was anyone able to run the ONNX model on the GPU and having the same metrics as with Pytorch inference (i.e. AP50) ? |
Thank you very much for providing the ONNX format conversion code! I have successfully obtained the ONNX file and performed inference. Have you tried converting the model to TensorRT format? I encountered an issue where the BiasGelu operator is not supported during this process. How can I resolve this? |
@honghuis @SkalskiP thanks for sharing the source code , Just wante dto knw can we convert this model to Tensorrt or ONNX format ? if so please share the conversion and inference script
Thanks in advance
The text was updated successfully, but these errors were encountered: