feature request: Expose logprob of output tokens

[yunfeng-scale]

Hi, can you help expose logprob of output tokens? This would be helpful for our use cases.

[jdemouth-nvidia]

Hi @yunfeng-scale ,

Thanks for your request, we will take it into account and add it to our roadmap. It's not clear to me, yet, when we will have the bandwidth to add it but we will keep you posted.

Thanks,
Julien

[byshiue]

Update title to highlight "feature reuqest".

[ncomly-nvidia]

Hi @yunfeng-scale, good news!! this work has been completed & is on main! check out return_log_probs in the config.pbtxt