[Bug]: Memory leak in 0.6 and 0.7 when setting "max_tokens=1"

[wangleigmail]

Your current environment

There is memory leak when I set max_tokens=1

I have tested 2 versions:
pip install vllm==0.6.6
or
pip install vllm==0.7.2

If I call generate with SamplingParams(max_tokens=1), there is a memory leak.

🐛 Describe the bug

from vllm import LLM, SamplingParams

llm=LLM(model='qwen/Qwen2.5-0.5B-Instruct', trust_remote_code=True)
sampling_params = SamplingParams(max_tokens=1)
while True:
result = llm.generate("Hello world, ", sampling_params)
pass
pass

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[wangleigmail]

For 0.6.6,
in vllm/core/scheduler.py: _schedule_running:
if max_tokens=1, every time self.cache_id is the same value, so one of the _scheduler_running_outputs_cache is always used and never reset, so the length grows endlessly.

[github-actions]

This issue has been automatically marked as stale because it has not had any activity within 90 days. It will be automatically closed if no further activity occurs within 30 days. Leave a comment if you feel this issue should remain open. Thank you!

[github-actions]

This issue has been automatically closed due to inactivity. Please feel free to reopen if you feel it is still relevant. Thank you!