Infer vs generate for Triton ensemble models #8921
Unanswered
gdeepa2007
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
I wanted to understand what are the implications of generate vs infer endpoints for a Triton ensemble model with Python backend. I tried both the endpoints and see that the data returned is same across both. Generate's request and response are more cleaner to integrate with client application. But I wanted to understand if there are any downsides if I go with generate.
All reactions