The quality of the generated answers is degraded

[yangzhj53]

I use baichuan2 as llm. When inputting long text, although the speed is improved, the quality of the generated answers has declined. Some people say that it is caused by different sampling during generation. How can I modify it?

[zjuzhfbloodz]

same situation

[viktor-ferenczi]

Please see whether there have been any changes to the settings in SamplingParams. Does reverting to the old settings help? If yes, then which setting caused the issue? If it does not help, then please provide more information, ideally with a minimal test case to reproduce this issue.

[viktor-ferenczi]

Maybe related: #1128

[akaihaoshuai]

可以参考文档 https://zhuanlan.zhihu.com/p/658780653

[Sanster]

You can try the modifications mentioned here.
#712 (comment)

[hmellor]

Closing this issue as stale as there has been no discussion in the past 3 months.

If you are still experiencing the issue you describe, feel free to re-open this issue.