Skip to content

reset cache for mm models#189

Merged
tastelikefeet merged 2 commits into
modelscope:mainfrom
hjh0119:reset-cache
May 11, 2026
Merged

reset cache for mm models#189
tastelikefeet merged 2 commits into
modelscope:mainfrom
hjh0119:reset-cache

Conversation

@hjh0119
Copy link
Copy Markdown
Collaborator

@hjh0119 hjh0119 commented May 11, 2026

Add two vllm reset cache interfaces for multi-modal models, and update cookbooks

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces methods to reset multimodal and vision encoder caches in the vLLM sampler, ensuring memory is freed after profiling and stale embeddings are cleared after weight updates. The changes are integrated into the GKD and GRPO cookbooks. Feedback includes a suggestion to add a safety check for the reset_mm_cache method to prevent errors on older vLLM versions, a correction for a version number typo in the docstrings, and a recommendation to add the @remote_function decorator to the existing reset_prefix_cache method for consistency in distributed environments.

Comment thread src/twinkle/sampler/vllm_sampler/vllm_engine.py
Comment thread src/twinkle/sampler/vllm_sampler/vllm_engine.py
Comment thread src/twinkle/sampler/vllm_sampler/vllm_sampler.py
@tastelikefeet tastelikefeet merged commit 096cacb into modelscope:main May 11, 2026
1 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants