deepseek-r1-awq overflow

[taishan1994]

System Info

tgi:3.2.3

Information

• Docker
• The CLI directly

Tasks

• An officially supported command
• My own modifications

Reproduction

1. Input a long text, concatenate the contents of user and assistant
2. truncate part of the text. Set the length of the output truncated text.
3. The model directly outputs "", and the finish_reason shows the reason as "length".

Expected behavior

It will also occur.: The model suddenly starts giving random answers during the inference process.