Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix dlpack memory leak #1344

Merged
merged 1 commit into from
Mar 26, 2024
Merged

Fix dlpack memory leak #1344

merged 1 commit into from
Mar 26, 2024

Conversation

ispobock
Copy link
Contributor

Motivation

In #1334 (comment), there may be some memory leak in dlpack.

For 5000 test prompts:

  • before this change, the memory usage will increase from 3110M -> 3660M
  • after this change, the memory usage will increase from 3110M -> 3460M

There are 200M memory reduced.

Modification

  • Return object instead of the pointer, then Python will take over the lifecycle of this object.

@ispobock
Copy link
Contributor Author

@AllentDan @grimoire @zhulinJulia24 Could you help check?

@ispobock ispobock mentioned this pull request Mar 26, 2024
Closed
2 tasks
@lvhan028 lvhan028 merged commit 9c954ef into InternLM:main Mar 26, 2024
8 of 9 checks passed
RunningLeon pushed a commit to RunningLeon/lmdeploy that referenced this pull request Mar 26, 2024
@ispobock @RunningLeon
fix dlpack memory leak (InternLM#1344)
f4262fc
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@grimoire grimoire grimoire approved these changes

@AllentDan AllentDan AllentDan approved these changes

Assignees
No one assigned
Labels
Bug:P0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@ispobock @grimoire @AllentDan @lvhan028