New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Python][C++] Memory leak when converting to arrow dtype #39599
Comments
raulcd
changed the title
Memory leak when converting to arrow dtype
[Python][C++] Memory leak when converting to arrow dtype
Jan 15, 2024
Thanks for the report @seberg . This does look like an issue indeed. |
pitrou
added a commit
to pitrou/arrow
that referenced
this issue
Jan 16, 2024
`PyArray_DescrFromScalar` returns a new reference, so we should be careful to decref it when we don't use it anymore.
jorisvandenbossche
pushed a commit
that referenced
this issue
Jan 17, 2024
### Rationale for this change `PyArray_DescrFromScalar` returns a new reference, so we should be careful to decref it when we don't use it anymore. ### Are these changes tested? No. ### Are there any user-facing changes? No. * Closes: #39599 Authored-by: Antoine Pitrou <antoine@python.org> Signed-off-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
idailylife
pushed a commit
to idailylife/arrow
that referenced
this issue
Jan 18, 2024
…ache#39636) ### Rationale for this change `PyArray_DescrFromScalar` returns a new reference, so we should be careful to decref it when we don't use it anymore. ### Are these changes tested? No. ### Are there any user-facing changes? No. * Closes: apache#39599 Authored-by: Antoine Pitrou <antoine@python.org> Signed-off-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
clayburn
pushed a commit
to clayburn/arrow
that referenced
this issue
Jan 23, 2024
…ache#39636) ### Rationale for this change `PyArray_DescrFromScalar` returns a new reference, so we should be careful to decref it when we don't use it anymore. ### Are these changes tested? No. ### Are there any user-facing changes? No. * Closes: apache#39599 Authored-by: Antoine Pitrou <antoine@python.org> Signed-off-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
dgreiss
pushed a commit
to dgreiss/arrow
that referenced
this issue
Feb 19, 2024
…ache#39636) ### Rationale for this change `PyArray_DescrFromScalar` returns a new reference, so we should be careful to decref it when we don't use it anymore. ### Are these changes tested? No. ### Are there any user-facing changes? No. * Closes: apache#39599 Authored-by: Antoine Pitrou <antoine@python.org> Signed-off-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
raulcd
pushed a commit
that referenced
this issue
Feb 20, 2024
### Rationale for this change `PyArray_DescrFromScalar` returns a new reference, so we should be careful to decref it when we don't use it anymore. ### Are these changes tested? No. ### Are there any user-facing changes? No. * Closes: #39599 Authored-by: Antoine Pitrou <antoine@python.org> Signed-off-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
zanmato1984
pushed a commit
to zanmato1984/arrow
that referenced
this issue
Feb 28, 2024
…ache#39636) ### Rationale for this change `PyArray_DescrFromScalar` returns a new reference, so we should be careful to decref it when we don't use it anymore. ### Are these changes tested? No. ### Are there any user-facing changes? No. * Closes: apache#39599 Authored-by: Antoine Pitrou <antoine@python.org> Signed-off-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
thisisnic
pushed a commit
to thisisnic/arrow
that referenced
this issue
Mar 8, 2024
…ache#39636) ### Rationale for this change `PyArray_DescrFromScalar` returns a new reference, so we should be careful to decref it when we don't use it anymore. ### Are these changes tested? No. ### Are there any user-facing changes? No. * Closes: apache#39599 Authored-by: Antoine Pitrou <antoine@python.org> Signed-off-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the bug, including details regarding any error messages, version, and platform.
I was running valgrind on some code which uses pyarrow as a dependency. This was leaking NumPy dtypes. Now the issue could be downstream, but this line (and similar ones) to me look like they will leak the NumPy descriptors:
arrow/python/pyarrow/src/arrow/python/python_to_arrow.cc
Line 390 in 7e703aa
I don't see how the NumPy descriptor would ever be decref'd. The fix would seem to first assign it to an
ownedRef
? For some dtypes that may just leak a reference, but not for all (reference leak being only a nuisance when debugging references).This like not often a problem deal in practice, although it could imagine it being a nuisance some in some very long running code. I am not planning on running such leaks checks regularly right now, so it shouldn't affect me really.
The reported origin of the leaked memory is:
Component(s)
Python
The text was updated successfully, but these errors were encountered: