Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Allow ufunc operand flags to be set #359

Merged
merged 12 commits into from May 10, 2013

Conversation

Projects
None yet
6 participants
Contributor

jayvius commented Jul 15, 2012

This change allows ufunc operand flags to be set in addition to the default operand flags set by the nditer object. For example, after calling PyUFunc_FromFuncAndData() to create a ufunc object, the first operand can be set to be read/write like so:

ufunc->op_flags[0] = NPY_ITER_READWRITE

Setting flags for output operands is not supported at this time and will raise an exception.

This change also allows the global nditer flags to be set, so after setting the read/write flag, the reduce ok flag could be set like so:

ufunc->iter_flags = NPY_ITER_REDUCE_OK

Owner

njsmith commented Jul 15, 2012

What's the motivation?

This pull request fails (merged 1d2111b9 into 4a2295d).

This pull request fails (merged e5ec51ea into 4a2295d).

This pull request fails (merged 638a2ff0 into 4a2295d).

Owner

teoliphant commented Oct 10, 2012

The motivation for this change is to allow some ufunc operands to be in-out and have the ufunc iterator support that intention on the part of the users. At least one user was hit by changes to how the ufunc machinery iterated over the stored kernels and the inability to set operands as READ_WRITE.

Owner

charris commented Mar 2, 2013

ping travisbot.

@charris charris closed this Mar 2, 2013

@charris charris reopened this Mar 2, 2013

Owner

charris commented Mar 2, 2013

@jayvius This needs a rebase.

@charris charris commented on the diff Apr 1, 2013

numpy/core/src/umath/ufunc_object.c
@@ -4254,6 +4293,11 @@ static int get_ufunc_arguments(PyUFuncObject *ufunc,
}
ufunc->doc = doc;
+ ufunc->op_flags = PyArray_malloc(sizeof(npy_uint32)*ufunc->nargs);
@charris

charris Apr 1, 2013

Owner

Success of memory allocation needs to be checked.

@charris

charris May 3, 2013

Owner

Still needs the NULL check. I there an error return available here?

@charris

charris May 9, 2013

Owner

Still missing check for allocation success.

@jayvius

jayvius May 9, 2013

Contributor

Somehow I missed these comments before. The op_flags allocation has a matching PyArray_free in the ufunc_dealloc function, but I do need one a few lines down if _parse_signature fails.

@jayvius

jayvius May 9, 2013

Contributor

Nevermind, the op_flags will still be freed because ufunc_dealloc gets called if _parse_signature fails.

@charris charris commented on the diff Apr 1, 2013

numpy/core/src/umath/umathmodule.c
@@ -123,6 +123,9 @@
self->core_dim_ixs = NULL;
self->core_offsets = NULL;
self->core_signature = NULL;
+ self->op_flags = PyArray_malloc(sizeof(npy_uint32)*self->nargs);
@charris

charris Apr 1, 2013

Owner

Success of memory allocation needs checking.

@charris

charris May 3, 2013

Owner

Still needed.

@charris

charris May 3, 2013

Owner

Notice that there is no matching PyArray_free as for the previous allocation of this type. Is that OK?

@charris

charris May 9, 2013

Owner

Does the allocated memory need to be freed?

@jayvius

jayvius May 9, 2013

Contributor

Missed these comments too somehow. op_flags get freed in ufunc_dealloc(). Adding null check now.

Owner

charris commented Apr 1, 2013

The new functionality needs to be documented and a discussion of its use would be appreciated. Is there anyway it can be tested? How does a user set the flags? Does numpy/core/code_generators/generate_umath.py need some added functionality?

Owner

charris commented May 3, 2013

@jayvius Needs a rebase.

@charris charris commented on an outdated diff May 3, 2013

numpy/core/setup.py
@@ -935,6 +935,13 @@ def get_dotblas_sources(ext, build_dir):
config.add_extension('multiarray_tests',
sources = [join('src', 'multiarray', 'multiarray_tests.c.src')])
+ #######################################################################
+ # operand_flag_tests module #
@charris

charris May 3, 2013

Owner

# is leaving the scene.

@charris charris commented on an outdated diff May 3, 2013

numpy/core/src/umath/operand_flag_tests.c.src
+ TestMethods,
+ NULL,
+ NULL,
+ NULL,
+ NULL
+};
+
+PyObject *PyInit_operand_flag_tests(void)
+{
+#else
+PyMODINIT_FUNC initoperand_flag_tests(void)
+{
+#endif
+ PyObject *m, *ufunc;
+
+#if defined(NPY_PY3K)
@charris

charris May 3, 2013

Owner

Needs the same fix as in the previous PR.

@charris charris commented on an outdated diff May 3, 2013

numpy/core/src/umath/operand_flag_tests.c.src
+ m = PyModule_Create(&moduledef);
+#else
+ m = Py_InitModule("operand_flag_tests", TestMethods);
+#endif
+ if (m == NULL) {
+ return;
+ }
+
+ import_array();
+ import_umath();
+
+ ufunc = PyUFunc_FromFuncAndData(funcs, data, types, 1, 2, 0,
+ PyUFunc_None, "inplace_add",
+ "inplace_add_docstring", 0);
+
+ /* Set flags to turn off buffering for first input operand,
@charris

charris May 3, 2013

Owner

Multiline comment formatting.

@charris charris commented on an outdated diff May 3, 2013

numpy/core/src/umath/ufunc_object.c
@@ -1182,12 +1182,22 @@ static int get_ufunc_arguments(PyUFuncObject *ufunc,
PyArrayObject **op_it;
@charris

charris May 3, 2013

Owner

Extra blank line ?

@charris charris commented on an outdated diff May 3, 2013

numpy/core/src/umath/ufunc_object.c
@@ -1182,12 +1182,22 @@ static int get_ufunc_arguments(PyUFuncObject *ufunc,
PyArrayObject **op_it;
+ npy_uint32 iter_flags;
+
NPY_BEGIN_THREADS_DEF;
/* Set up the flags */
for (i = 0; i < nin; ++i) {
op_flags[i] = NPY_ITER_READONLY|
@charris

charris May 3, 2013

Owner

Space before |.

@charris charris commented on an outdated diff May 3, 2013

numpy/core/src/umath/ufunc_object.c
@@ -1197,18 +1207,21 @@ static int get_ufunc_arguments(PyUFuncObject *ufunc,
NPY_ITER_NO_SUBTYPE;
}
+ iter_flags = ufunc->iter_flags|
@charris

charris May 3, 2013

Owner

Spaces before trailing operators.

@charris charris commented on an outdated diff May 3, 2013

numpy/core/src/umath/ufunc_object.c
@@ -1463,8 +1476,10 @@ static int get_ufunc_arguments(PyUFuncObject *ufunc,
PyArrayObject **op_it;
@charris

charris May 3, 2013

Owner

Extra blank line. Could move npy_uint32 ... up a line.

@charris charris commented on an outdated diff May 3, 2013

numpy/core/src/umath/ufunc_object.c
@@ -1496,17 +1519,20 @@ static int get_ufunc_arguments(PyUFuncObject *ufunc,
NPY_UF_DBG_PRINT("Making iterator\n");
+ iter_flags = ufunc->iter_flags|
@charris

charris May 3, 2013

Owner

Spaces before |.

@charris charris commented on an outdated diff May 3, 2013

numpy/core/src/umath/ufunc_object.c
@@ -1665,6 +1691,8 @@ static int get_ufunc_arguments(PyUFuncObject *ufunc,
npy_intp iter_shape[NPY_MAXARGS];
@charris

charris May 3, 2013

Owner

Don't think the blank lines add anything here.

@charris charris commented on an outdated diff May 3, 2013

numpy/core/src/umath/ufunc_object.c
@@ -1967,6 +1995,14 @@ static int get_ufunc_arguments(PyUFuncObject *ufunc,
op_flags[i] = NPY_ITER_READONLY|
@charris

charris May 3, 2013

Owner

Spaces again.

@charris charris commented on an outdated diff May 3, 2013

numpy/core/include/numpy/ufuncobject.h
@@ -212,6 +212,20 @@ typedef int (PyUFunc_MaskedInnerLoopSelectionFunc)(
* A function which returns a masked inner loop for the ufunc.
*/
PyUFunc_MaskedInnerLoopSelectionFunc *masked_inner_loop_selector;
+
+ /*
+ * List of flags for each operand when ufunc is called by nditer object.
+ * These flags will be used in addition to the default flags for each
+ * operand set by nditer object.
@charris

charris May 3, 2013

Owner

Extra leading space.

jayvius added some commits Jul 12, 2012

This change allows ufunc operand flags to be set in addition to the
default operand flags set by the nditer object. For example, after
calling PyUFunc_FromFuncAndData() to create a ufunc object, the first
operand can be set to be read/write like so:

ufunc->op_flags[0] = NPY_ITER_READWRITE

Setting flags for output operands is not supported at this time and
will raise an exception.

This change also allows the global nditer flags to be set, so after
setting the read/write flag, the reduce ok flag could be set like so:

ufunc->iter_flags = NPY_ITER_REDUCE_OK

@charris charris commented on an outdated diff May 9, 2013

numpy/core/src/umath/ufunc_object.c
NPY_BEGIN_THREADS_DEF;
-
+
@charris

charris May 9, 2013

Owner

Trailing whitespace?

@charris charris commented on the diff May 9, 2013

numpy/core/src/umath/ufunc_object.c
@@ -1976,11 +2009,26 @@ static int get_ufunc_arguments(PyUFuncObject *ufunc,
NPY_ITER_NO_BROADCAST;
}
+ /*
+ * If there are no iteration dimensions, create a fake one
+ * so that the scalar edge case works right.
+ */
@charris

charris May 9, 2013

Owner

@seberg ISTR you may have done something relevant to this.

@seberg

seberg May 9, 2013

Member

Yeah, my guess is, it can probably be removed. But would have to try too...

@charris charris commented on an outdated diff May 9, 2013

numpy/core/src/umath/ufunc_object.c
@@ -1976,11 +2009,26 @@ static int get_ufunc_arguments(PyUFuncObject *ufunc,
NPY_ITER_NO_BROADCAST;
}
+ /*
+ * If there are no iteration dimensions, create a fake one
+ * so that the scalar edge case works right.
+ */
+ if (iter_ndim == 0) {
+ iter_ndim = 1;
+ iter_shape[0] = 1;
+ for (i = 0; i < nop; ++i) {
+ op_axes[i][0] = -1;
+ }
+ }
+
+ iter_flags = ufunc->iter_flags|
@charris

charris May 9, 2013

Owner

Spaces before operators.

Owner

charris commented May 10, 2013

OK, let's put this in. Are there any plans to add some true inplace operators to numpy.

charris added a commit that referenced this pull request May 10, 2013

Merge pull request #359 from jayvius/operand-flags
Allow ufunc operand flags to be set

@charris charris merged commit aac0e1b into numpy:master May 10, 2013

1 check passed

default The Travis CI build passed
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment