New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
BUG, MAINT: Ufunc reduce reference leak #10194
Conversation
Would lead to a reference leak for the case that an invalid axis is passed in.
&axes_in, | ||
PyArray_DescrConverter2, &otype, | ||
PyArray_OutputConverter, &out)) { | ||
Py_XDECREF(otype); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
otype
cannot be defined if this fails.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's not true - this was correct before - it could fail on the OutputConverter
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In fact, this should maybe xdecref out
too.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I looked for other usages, and they always DECREF only stuff before OutputConverter
- so, I guess one does not have to worry about the one that does fail, just what comes before. Anyway, I'll change it back.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Or, rather, change it to goto fail
as it is for the other options.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess one does not have to worry about the one that does fail, just what comes before.
Looking at the implementation of PyArg_ParseTupleAndKeywords
in 3.6, I think I can break this by providing an argument by keyword and position, which happens after the other arguments are parsed. I certainly don't think that cpython guarantees the last object will never need freeing
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You may well be right, but since this is separate from the fix here (in that it is done everywhere OutputConverter
is used), I think is should be a separate issue: #10197
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm wrong, PyArray_OutputConverter
borrows a reference. Would be good to distinguish the Converter
s that lend/create references in the name somehow, but we're way too late to make that decision.
} | ||
/* Special case letting axis={0 or -1} slip through for scalars */ | ||
if (ndim == 0 && (axis == 0 || axis == -1)) { | ||
axis = 0; | ||
} | ||
else if (check_and_adjust_axis(&axis, ndim) < 0) { | ||
return NULL; | ||
goto fail; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is where the DECREF
was missing.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Careless of me, good catch
numpy/core/src/umath/ufunc_object.c
Outdated
@@ -3941,6 +3919,11 @@ PyUFunc_GenericReduction(PyUFuncObject *ufunc, PyObject *args, | |||
} | |||
} | |||
return PyArray_Return(ret); | |||
|
|||
fail: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure we normally indent like this.
2af1692
to
0a2c9a9
Compare
numpy/core/src/umath/ufunc_object.c
Outdated
return NULL; | ||
if (error_converting(axis) || | ||
(check_and_adjust_axis(&axis, ndim) < 0)) { | ||
goto fail; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this would be clearer as two separate if
s, like it was before - it makes it look more like the other uses of check_and_adjust_axis
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One style nit, but generally looks correct. Thanks for correcting my mistake!
This should help avoid reference leaks in future additions.
0a2c9a9
to
2fadede
Compare
OK, nitpick addressed. |
Thanks! |
How'd you find this one? |
I started trying to implement |
While you're looking at reduce, mind reviewing #8952? |
A missing
DECREF
stanza in one of the failure paths, introduced as part of thecheck_and_adjust_axis
PR. In the maintenance commit, I collected all the similar failure paths in a singlefail
.I kept the two commits separate, though probably it is easiest to just backport both.