Support non-detached mode for python trtllm backend #639
Description
System Info
tensorrtllm backend doesn't work for us because of this bug: #598. So I have to use python backend. However, it only supports detached model which we don't need.
Can we add support for non-detached mode?
Who can help?
Information
- The official example scripts
- My own modified scripts
Tasks
- An officially supported task in the
examplesfolder (such as GLUE/SQuAD, ...) - My own task or dataset (give details below)
Reproduction
N/A
Expected behavior
N/A
actual behavior
N/A
additional notes
N/A