-
Notifications
You must be signed in to change notification settings - Fork 141
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
What exactly does "Optimal" mean? #45
Comments
Optimal is the config TensorRT will optimize for, whereas min and max only define the range in which it needs to be operable. |
Thanks for the answer!! So if I start my image at 512 and grow it to 2048, do I need 3 engines with Optimal set to 512, 1024, and 2048? how would you recommend setting Optimal? |
Technically, you could do an engine with min=512, opt=1024, max=2048 that covers all cases. But performance will probably be suboptimal at 512 and 2048. How large the performance impact is going to be is hard to estimate. In that case, I'd recommend exporting multiple engines, using your most commonly used resolutions for optimal. |
TensorRT sometimes is more art than science. It could be that 2048 is so large that, at that point, it doesn't make a difference. For 1024 you can definitely see a performance delta between the opt=512 and opt=1024 engine. From your experience is 1.0x it/s slow? |
The difference in speed between using TensorRT and not using TensorRT is clear, but it doesn't seem to be an optimal value (at low resolutions, the images are generated very quickly, so it may be that can't tell) |
when l export a TensorRT-based Engine, I know exactly what Min/Max Width and Height are for, but I don't know what Optimal is, and I didn't find anything about it in the description on the right. Does anyone know anything about it?
The text was updated successfully, but these errors were encountered: