Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] Leak tracking for Java Scalar instances #8227

Open
jlowe opened this issue May 12, 2021 · 2 comments
Open

[FEA] Leak tracking for Java Scalar instances #8227

jlowe opened this issue May 12, 2021 · 2 comments
Labels
cuDF (Java) Affects Java cuDF API. feature request New feature or request helps: Spark Functionality that helps Spark RAPIDS

Comments

@jlowe
Copy link
Member

jlowe commented May 12, 2021

Is your feature request related to a problem? Please describe.
The Java bindings require explicit close() calls by the caller to prevent GPU OOM errors if garbage collection doesn't kick in often enough to free unreferenced Java objects wrapping GPU resources in a timely manner. We currently have leaked object tracking via weak reference collection events for most GPU resources, but that does not currently cover Scalar instances which wrap cudf::scalar.

Describe the solution you'd like
The same leak tracking and debugging done for ColumnVector should be applied to Scalar. Although Scalar instances are not very large in practice (excluding the corner cases of nested type scalars), they can fragment the GPU memory pool drastically if there are enough of these tiny allocations leaked, causing GPU OOM errors much earlier than one would expect.

@jlowe jlowe added feature request New feature or request cuDF (Java) Affects Java cuDF API. helps: Spark Functionality that helps Spark RAPIDS labels May 12, 2021
@jlowe jlowe added this to Needs prioritizing in Feature Planning via automation May 12, 2021
@github-actions
Copy link

This issue has been labeled inactive-90d due to no recent activity in the past 90 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed.

@jlowe
Copy link
Member Author

jlowe commented Nov 15, 2021

Still desired, low priority.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cuDF (Java) Affects Java cuDF API. feature request New feature or request helps: Spark Functionality that helps Spark RAPIDS
Projects
No open projects
Feature Planning
Needs prioritizing
Development

No branches or pull requests

1 participant