Skip to content
This repository

Added inplace_increment function #326

Closed
wants to merge 27 commits into from

7 participants

John Salvatier njsmith Travis E. Oliphant Charles Harris Don't Add Me To Your Organization a.k.a The Travis Bot Henry Gomersall Bradley M. Froehle
John Salvatier

I've added an inplace_increment function which allows you to increment an array inplace using advanced indexing. The key feature is that repeated indexes will repeatedly increment the array whereas

a[idx] +=1

will not do this. More discussion here and here.

I've included, documentation, a couple of tests and supported multiple datatypes.

I'm new to doing Python functions in C so a good once over of the C code is a good idea. I'm also new to submitting code for numpy, so I've probably made some mistakes somewhere.

njsmith
Owner

I have mixed feelings about this. The functionality is often-requested and clearly useful, but the implementation has a number of problems. The basic features numpy provides are indexing and generic operations over generic types. This code uses numpy's indexing, but ignores the generic operation and type support, so it only supports one operation (addition) and a fixed set of types. All that would be fine if it were just implementation limitations that could be fixed later, but the API is not flexible enough to support other operations -- a "proper" version of this functionality would probably be a method on ufuncs, not a standalone function. So the API is kind of a dead-end as well.

I don't want to make the perfect the enemy of the good, but on balance I think there's enough wrong with this that I'm -0.5 on merging it and making it a permanent part of the numpy API. The best compromise might be to extract the core function and release it as a tiny standalone library? It looks like it only uses public numpy API functions, so it should be easy to do. That way people who want the feature have an easy way to get it until numpy grows a cleaner solution.

John Salvatier

I'd be in favor of a stand alone library (it was my original intention to put this into a separate package) except that the MapIter interface is unfortunately not public; mapping.c has a comment that explains:

/*

  • The mapiter object must be created new each time. It does not work
  • to bind to a new array, and continue. *
  • This was the orginal intention, but currently that does not work.
  • Do not expose the MapIter_Type to Python. *
  • It's not very useful anyway, since mapiter(indexobj); mapiter.bind(a);
  • mapiter is equivalent to a[indexobj].flat but the latter gets to use
  • slice syntax. */

If MapIter could be exposed, then I'd be happy to make this a function in a separate package. I don't know whether it would be difficult to fix the reuse issue. I think Mark Wiebe was the person who implemented MapIter.

I'll look into making this a function of binary ufuncs like reduce, but I suspect that's still above my skill level.

John Salvatier

Actually looking at the implementation of PyUFunc_ReduceWrapper, it doesn't look like it would be too difficult to do something similar for this. However, I'm having a hard time picturing how the added generality would be used. Can you think of some useful cases where this would be used on ufuncs besides add? I can imagine it being useful on prod and pow, but I can't think of any concrete uses.

njsmith
Owner

I don't have any concrete cases in mind for using this with prod and pow, but I bet someone will show up with one sooner or later... automatically supporting all numpy dtypes is the more useful part, I think.

If MapIter isn't exposed then that'll be a hurdle, though, because ufuncs (which live in the "umath" library) don't have access to the "multiarray" library internal functions. @mwiebe, is there any reasonable way to do ndarray indexing via the public C API?

Travis E. Oliphant
Owner

While in an ideal world, perhaps this functionality would be on ufuncs. However, that is actually quite a bit of work. The fancy-indexing feature is just not exposed to the ufunc machinery. Thus, I think that currently this patch does a pretty good job as is. I don't think it would be too difficult to add a verison for mul and pow using a similar API at some point.

numpy/core/bscript
@@ -428,6 +428,7 @@ def pre_build(context):
428 428
                 "src/multiarray/arraytypes.c.src",
429 429
                 "src/multiarray/nditer_templ.c.src",
430 430
                 "src/multiarray/lowlevel_strided_loops.c.src",
  431
+				"src/multiarray/mapping.c.src",
1
Travis E. Oliphant Owner
teoliphant added a note July 17, 2012

Indentation problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
numpy/core/setup.py
@@ -688,6 +688,7 @@ def generate_multiarray_templated_sources(ext, build_dir):
688 688
                    join(local_dir, subpath, 'arraytypes.c.src'),
689 689
                    join(local_dir, subpath, 'nditer_templ.c.src'),
690 690
                    join(local_dir, subpath, 'lowlevel_strided_loops.c.src'),
  691
+				   join(local_dir, subpath, 'mapping.c.src'),
1
Travis E. Oliphant Owner
teoliphant added a note July 17, 2012

Indentation problem again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
numpy/core/src/multiarray/mapping.c.src
((135 lines not shown))
  2158
+		if (type_number == type_numbers[i])
  2159
+		{
  2160
+			add_inplace = addition_funcs[i];
  2161
+			break;
  2162
+		}
  2163
+		i++ ;
  2164
+	}
  2165
+	
  2166
+	if (add_inplace == NULL)
  2167
+	{
  2168
+		PyErr_SetString(PyExc_TypeError, "unsupported type for a"); 
  2169
+		return NULL;
  2170
+	}
  2171
+	
  2172
+	
  2173
+	//body
1
Travis E. Oliphant Owner
teoliphant added a note July 17, 2012

Do we support // -style comments in C in NumPy. Last I knew we did not.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
numpy/core/src/multiarray/mapping.c.src
((138 lines not shown))
  2161
+			break;
  2162
+		}
  2163
+		i++ ;
  2164
+	}
  2165
+	
  2166
+	if (add_inplace == NULL)
  2167
+	{
  2168
+		PyErr_SetString(PyExc_TypeError, "unsupported type for a"); 
  2169
+		return NULL;
  2170
+	}
  2171
+	
  2172
+	
  2173
+	//body
  2174
+	PyArrayMapIterObject * mit = (PyArrayMapIterObject *) PyArray_MapIterNew(index, 0, 1);
  2175
+    if (mit == NULL) 
  2176
+	{
1
Travis E. Oliphant Owner
teoliphant added a note July 17, 2012

Indentation issues. Please put the if aligned with the previous line.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
numpy/core/src/multiarray/mapping.c.src
((152 lines not shown))
  2175
+    if (mit == NULL) 
  2176
+	{
  2177
+		goto fail;
  2178
+	}
  2179
+    PyArray_MapIterBind(mit, a);
  2180
+	if (map_increment(mit, inc, add_inplace) != 0)
  2181
+    {
  2182
+    	goto fail;
  2183
+    }
  2184
+    
  2185
+    //endbody
  2186
+	
  2187
+	Py_DECREF(mit);
  2188
+	
  2189
+	Py_INCREF(Py_None);
  2190
+    return Py_None;
1
Travis E. Oliphant Owner
teoliphant added a note July 17, 2012

Align this line with the previous one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
John Salvatier

I think I've fixed those formatting issues.

numpy/core/src/multiarray/mapping.c.src
((114 lines not shown))
  2137
+    
  2138
+    if (PyArray_NDIM(a) == 0) {
  2139
+        PyErr_SetString(PyExc_IndexError, "0-d arrays can't be indexed.");
  2140
+        return NULL; 
  2141
+    }
  2142
+    
  2143
+    if (!PyArray_ISWRITEABLE(a)) {
  2144
+        PyErr_SetString(PyExc_RuntimeError,
  2145
+                        "array is not writeable");
  2146
+        return NULL;
  2147
+    }
  2148
+
  2149
+    inplace_binop add_inplace = NULL; 
  2150
+    int type_number = PyArray_TYPE(a);
  2151
+    int i =0;
  2152
+    while  (type_numbers[i] >= 0 && addition_funcs[i] != NULL)
1
Charles Harris Owner
charris added a note July 18, 2012

wrong curly braces style for while and if.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Charles Harris
Owner

Not quite, there are still lots of style issues. There is a C style guide in doc/C_STYLE_GUIDE.rst.txt. Also check that there are no hard tabs anywhere, some of the indents still look odd.

numpy/core/src/multiarray/mapping.c.src
@@ -2024,6 +2024,169 @@
2024 2024
 }
2025 2025
 
2026 2026
 
  2027
+typedef void (*inplace_binop)(char *, char *);
  2028
+
  2029
+
  2030
+//is something like this already defined somewhere? where?
  2031
+/**begin repeat
  2032
+* #type  = npy_int8, npy_int16, npy_int32, npy_int64, npy_uint8, npy_uint16, npy_uint32, npy_uint64, npy_float16, npy_float32, npy_float64#
  2033
+* #typen = NPY_INT8, NPY_INT16, NPY_INT32, NPY_INT64, NPY_UINT8, NPY_UINT16, NPY_UINT32, NPY_UINT64, NPY_FLOAT16, NPY_FLOAT32, NPY_FLOAT64#
2
Henry Gomersall
hgomersall added a note July 19, 2012

Isn't this missing a few types? What happens for bit lengths bigger than 64? (forgive me if this should be apparent!)

John Salvatier
jsalvatier added a note July 22, 2012

I just chose the ones that I had seen used. How large should this go up to?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
numpy/core/src/multiarray/mapping.c.src
((5 lines not shown))
  2028
+
  2029
+
  2030
+//is something like this already defined somewhere? where?
  2031
+/**begin repeat
  2032
+* #type  = npy_int8, npy_int16, npy_int32, npy_int64, npy_uint8, npy_uint16, npy_uint32, npy_uint64, npy_float16, npy_float32, npy_float64#
  2033
+* #typen = NPY_INT8, NPY_INT16, NPY_INT32, NPY_INT64, NPY_UINT8, NPY_UINT16, NPY_UINT32, NPY_UINT64, NPY_FLOAT16, NPY_FLOAT32, NPY_FLOAT64#
  2034
+*/
  2035
+static void @type@_inplace_add(char * l_op, char* r_op)
  2036
+{
  2037
+    ((@type@*)l_op)[0] = ((@type@*)l_op)[0] + ((@type@ *)r_op)[0];
  2038
+}
  2039
+/**end repeat**/
  2040
+
  2041
+/**begin repeat
  2042
+* #type  = npy_complex64, npy_complex128#
  2043
+* #typen = NPY_COMPLEX64, NPY_COMPLEX128#
1
Henry Gomersall
hgomersall added a note July 19, 2012

As for the reals above, what happens for the longer complex types?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Don't Add Me To Your Organization a.k.a The Travis Bot

This pull request fails (merged 16603b5 into f2f0ac0).

Don't Add Me To Your Organization a.k.a The Travis Bot

This pull request fails (merged 6a80aba into f2f0ac0).

John Salvatier

I read through the C style document and went through and tried to make this patch conform.

I've also added support for all int,uint, float, complex datatype bit-width sizes that are listed at the bottom of here: http://docs.scipy.org/doc/numpy/reference/c-api.dtype.html

numpy/core/src/multiarray/mapping.c.src
((118 lines not shown))
  2145
+
  2146
+static PyObject *
  2147
+inplace_increment(PyObject *dummy, PyObject *args)
  2148
+{
  2149
+    PyObject *arg_a = NULL, *index=NULL, *inc=NULL;
  2150
+    PyArrayObject *a;
  2151
+    inplace_binop add_inplace = NULL; 
  2152
+    int type_number = -1;
  2153
+    int i =0;
  2154
+    PyArrayMapIterObject * mit;
  2155
+    
  2156
+    if (!PyArg_ParseTuple(args, "OOO", &arg_a, &index,
  2157
+            &inc)) { 
  2158
+        return NULL;
  2159
+    }
  2160
+    if (arg_a->ob_type != &PyArray_Type){
2
Bradley M. Froehle
bfroehle added a note July 24, 2012

I think you meant to write if (!PyArray_Check(arg_a)) {.

John Salvatier
jsalvatier added a note July 25, 2012

Thanks, I was looking for that function.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
numpy/core/src/multiarray/mapping.c.src
@@ -2028,6 +2028,189 @@
2028 2028
 }
2029 2029
 
2030 2030
 
  2031
+typedef void (*inplace_binop)(char *, char *);
  2032
+
  2033
+
  2034
+/*is something like this already defined somewhere? where?*/
2
Bradley M. Froehle
bfroehle added a note July 24, 2012

Doubtful, because calling a function just to add two integers or floats together would be exceptionally slow. Instead I would expect that the function you generate here would perform the entire loop...

Edit: I now see PyArray_GetMap and PyArray_SetMap do something similar for the constructions a[b] and a[b] = c, but I must say I'm a bit surprised.

John Salvatier
jsalvatier added a note July 25, 2012

I hadn't understood why this was true before, but now I do. I've changed it the specialized functions to do the entire loop.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
numpy/core/src/multiarray/mapping.c.src
((128 lines not shown))
  2155
+    
  2156
+    if (!PyArg_ParseTuple(args, "OOO", &arg_a, &index,
  2157
+            &inc)) { 
  2158
+        return NULL;
  2159
+    }
  2160
+    if (arg_a->ob_type != &PyArray_Type){
  2161
+         PyErr_SetString(PyExc_ValueError, "needs an ndarray as first argument");
  2162
+         return NULL;
  2163
+    }
  2164
+    a = (PyArrayObject *) arg_a;
  2165
+    
  2166
+    if (!PyArray_ISWRITEABLE(a)) {
  2167
+        PyErr_SetString(PyExc_RuntimeError,
  2168
+                        "array is not writeable");
  2169
+        return NULL;
  2170
+    }
1
Bradley M. Froehle
bfroehle added a note July 24, 2012

I think you meant:

if (PyArray_FailUnlessWriteable(a, "input/output array") < 0) {
    return NULL;
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Don't Add Me To Your Organization a.k.a The Travis Bot

This pull request fails (merged 73b36f4 into f2f0ac0).

njsmith
Owner

It's great that you guys are interested in this, but I'm worried that you're spending a lot of time in ways that aren't really getting the patch closer to merging. The main obstacle is still the points that I raised. IMHO the best way forward would be to work out how to expose an API for doing indexing:
http://mail.scipy.org/pipermail/numpy-discussion/2012-July/063411.html

Of course there may be other options as well, but this approach doesn't seem too difficult and AFAICT everyone seems to like it.

John Salvatier
Travis E. Oliphant
Owner
John Salvatier
njsmith
Owner

There's a script that runs over the source and generates the actual API proper. To mark a function to be exported, you just put a magic comment that says NUMPY_API in front of it. If you grep the source you'll find some examples to follow. Then you should also add it to the list in numpy/core/code_generators/numpy_api.py.

I'm not familiar enough with the details of the MapIterObject interface to comment off-the-cuff on whether it can be exposed as is. If no-one else speaks up, though, then you can always make a first hackish attempt and post it for review. (Note that Travis did have a suggestion about how the interface should look in that comment I posted up above, though.) I think the solution to the resetting problem is to just not expose any functionality that would let people run into the problem, though I don't know what the problem is in detail so I might be missing something.

John Salvatier

I've made a separate pull request, for njsmith's recommendation to expose the MapIter API (#375).

John Salvatier

I've made a cleaned up pull request (#377).

John Salvatier jsalvatier referenced this pull request September 26, 2012
Closed

MapIter API #377

Frédéric Bastien nouiz referenced this pull request in Theano/Theano October 29, 2012
Open

Implement GpuAdvancedIncSubtensor #930

John Salvatier jsalvatier referenced this pull request in Theano/Theano December 13, 2012
Closed

Full advanced Indexing support + gradient #1083

John Salvatier

We decided not to move ahead with this pull request and instead just added API support: #377

John Salvatier jsalvatier closed this February 08, 2013
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Showing 27 unique commits by 1 author.

Jun 25, 2012
John Salvatier added index increment function 2d4fc32
John Salvatier made index_increment return the array; if the input wasn't the right …
…kind, it will copy, otherwise it will just do it inplace
ef1d021
Jun 26, 2012
John Salvatier made index increment require a double array 6af27c5
John Salvatier made index increment require a double array 82587d3
John Salvatier improved error handling e9b0c2b
Jun 27, 2012
John Salvatier added support for multiple datatypes ccc1189
John Salvatier only goto fail if we've initialized mit 745faa8
John Salvatier added initial doc 5aef9c7
John Salvatier renamed index_increment to inplace_increment 86b15d6
John Salvatier made inplace_increment import by default a47e934
John Salvatier updated examples 77d5341
John Salvatier supported complex types e6cecf9
John Salvatier added initial tests 47b1f15
John Salvatier added more tests 27bd77b
John Salvatier fixed tests mostly 420a241
John Salvatier fixed boolean test 3085f98
Jul 06, 2012
John Salvatier added another example 7d03753
Jul 18, 2012
John Salvatier fixed formatting issues that travis pointed out 6f84384
Jul 22, 2012
John Salvatier fixed style issues 68cafc1
John Salvatier added more types and refactored typing generation; it seems kind of c…
…omplex now...
684ac6a
Jul 23, 2012
John Salvatier corrected a segfault issue from reformatting 16603b5
John Salvatier Revert "added more types and refactored typing generation; it seems k…
…ind of complex now..."

This reverts commit 684ac6a.
bd513bf
John Salvatier forgot ; f9314aa
John Salvatier added other datatypes 44a1c49
John Salvatier added check for non-array 6a80aba
Jul 25, 2012
John Salvatier moved the looping into the datatype specific functions and simplified…
… two error checks
73b36f4
Nov 06, 2012
John Salvatier Merge remote-tracking branch 'main/master' d862f52
This page is out of date. Refresh to see the latest.
53  numpy/add_newdocs.py
@@ -2231,6 +2231,59 @@ def luf(lamdaexpr, *args, **kwargs):
2231 2231
     30
2232 2232
 
2233 2233
     """)
  2234
+	
  2235
+add_newdoc('numpy.core.multiarray', 'inplace_increment',
  2236
+    """
  2237
+    inplace_increment(a, index, inc)
  2238
+
  2239
+    Increments `a` inplace by `inc` in the locations specified by the index tuple
  2240
+	`index`. Supports advanced indexing. Repeated indexes are repeatedly
  2241
+	incremented.
  2242
+
  2243
+    Parameters
  2244
+    ----------
  2245
+    a : ndarray
  2246
+		Array to increment.
  2247
+    index : tuple of indexes
  2248
+		Supports advanced indexing.
  2249
+	inc : array_like
  2250
+		How much to increment `a`.
  2251
+
  2252
+    Returns
  2253
+    -------
  2254
+    output : None
  2255
+
  2256
+    Examples
  2257
+    --------
  2258
+	>>> a = arange(12).reshape((3,4)).astype(float64)
  2259
+	>>> index = ([1,1,2,0], [0,0,2,3])
  2260
+	>>> vals = [50,50, 30,16]
  2261
+	>>> a
  2262
+	array([[  0.,   1.,   2.,   3.],
  2263
+		   [  4.,   5.,   6.,   7.],
  2264
+		   [  8.,   9.,  10.,  11.]])
  2265
+	>>> inplace_increment(a, index, vals)
  2266
+	>>> a
  2267
+	array([[   0.,    1.,    2.,   19.],
  2268
+		   [ 104.,    5.,    6.,    7.],
  2269
+		   [   8.,    9.,   40.,   11.]])
  2270
+	
  2271
+	>>> x = zeros((3,3))	   
  2272
+	>>> x
  2273
+	array([[ 0.,  0.,  0.],
  2274
+		   [ 0.,  0.,  0.],
  2275
+		   [ 0.,  0.,  0.]])
  2276
+	>>> y = arange(3*3).reshape((3,3))
  2277
+	>>> y
  2278
+	array([[0, 1, 2],
  2279
+		   [3, 4, 5],
  2280
+		   [6, 7, 8]])
  2281
+	>>> inplace_increment(x, [0,1,1], y)
  2282
+	>>> x
  2283
+	array([[  0.,   1.,   2.],
  2284
+		   [  9.,  11.,  13.],
  2285
+		   [  0.,   0.,   0.]])
  2286
+    """)
2234 2287
 
2235 2288
 
2236 2289
 ##############################################################################
3  numpy/core/bscript
@@ -427,6 +427,7 @@ def pre_build(context):
427 427
                 "src/multiarray/arraytypes.c.src",
428 428
                 "src/multiarray/nditer_templ.c.src",
429 429
                 "src/multiarray/lowlevel_strided_loops.c.src",
  430
+                "src/multiarray/mapping.c.src",
430 431
                 "src/multiarray/einsum.c.src"]
431 432
         bld(target="multiarray_templates", source=multiarray_templates)
432 433
         if ENABLE_SEPARATE_COMPILATION:
@@ -449,7 +450,7 @@ def pre_build(context):
449 450
                 pjoin('src', 'multiarray', 'item_selection.c'),
450 451
                 pjoin('src', 'multiarray', 'iterators.c'),
451 452
                 pjoin('src', 'multiarray', 'lowlevel_strided_loops.c.src'),
452  
-                pjoin('src', 'multiarray', 'mapping.c'),
  453
+                pjoin('src', 'multiarray', 'mapping.c.src'),
453 454
                 pjoin('src', 'multiarray', 'methods.c'),
454 455
                 pjoin('src', 'multiarray', 'multiarraymodule.c'),
455 456
                 pjoin('src', 'multiarray', 'nditer_pywrap.c'),
3  numpy/core/numeric.py
@@ -10,7 +10,7 @@
10 10
            'alterdot', 'restoredot', 'roll', 'rollaxis', 'cross', 'tensordot',
11 11
            'array2string', 'get_printoptions', 'set_printoptions',
12 12
            'array_repr', 'array_str', 'set_string_function',
13  
-           'little_endian', 'require',
  13
+           'little_endian', 'require', 'inplace_increment',
14 14
            'fromiter', 'array_equal', 'array_equiv',
15 15
            'indices', 'fromfunction', 'isclose',
16 16
            'load', 'loads', 'isscalar', 'binary_repr', 'base_repr',
@@ -252,6 +252,7 @@ def extend_all(module):
252 252
 compare_chararrays = multiarray.compare_chararrays
253 253
 putmask = multiarray.putmask
254 254
 einsum = multiarray.einsum
  255
+inplace_increment = multiarray.inplace_increment
255 256
 
256 257
 def asarray(a, dtype=None, order=None):
257 258
     """
3  numpy/core/setup.py
@@ -691,6 +691,7 @@ def generate_multiarray_templated_sources(ext, build_dir):
691 691
                    join(local_dir, subpath, 'arraytypes.c.src'),
692 692
                    join(local_dir, subpath, 'nditer_templ.c.src'),
693 693
                    join(local_dir, subpath, 'lowlevel_strided_loops.c.src'),
  694
+                   join(local_dir, subpath, 'mapping.c.src'),
694 695
                    join(local_dir, subpath, 'einsum.c.src')]
695 696
 
696 697
         # numpy.distutils generate .c from .c.src in weird directories, we have
@@ -776,7 +777,7 @@ def generate_multiarray_templated_sources(ext, build_dir):
776 777
             join('src', 'multiarray', 'item_selection.c'),
777 778
             join('src', 'multiarray', 'iterators.c'),
778 779
             join('src', 'multiarray', 'lowlevel_strided_loops.c.src'),
779  
-            join('src', 'multiarray', 'mapping.c'),
  780
+            join('src', 'multiarray', 'mapping.c.src'),
780 781
             join('src', 'multiarray', 'methods.c'),
781 782
             join('src', 'multiarray', 'multiarraymodule.c'),
782 783
             join('src', 'multiarray', 'nditer_templ.c.src'),
187  numpy/core/src/multiarray/mapping.c → numpy/core/src/multiarray/mapping.c.src
@@ -2056,6 +2056,193 @@ PyArray_MapIterArray(PyArrayObject * a, PyObject * index)
2056 2056
 #undef SOBJ_TOOMANY
2057 2057
 #undef SOBJ_LISTTUP
2058 2058
 
  2059
+typedef void (*inplace_map_binop)(PyArrayMapIterObject *, PyArrayIterObject *);
  2060
+
  2061
+/**begin repeat
  2062
+* #type  = npy_int8, npy_int16, npy_int32, npy_int64, npy_int128, npy_int256,
  2063
+           npy_uint8, npy_uint16, npy_uint32, npy_uint64, npy_uint128, npy_uint256, 
  2064
+           npy_float16, npy_float32, npy_float64, npy_float80, npy_float96, npy_float128, npy_float256#
  2065
+
  2066
+* #typen = NPY_INT8, NPY_INT16, NPY_INT32, NPY_INT64, NPY_INT128, NPY_INT256,
  2067
+           NPY_UINT8, NPY_UINT16, NPY_UINT32, NPY_UINT64, NPY_UINT128, NPY_UINT256, 
  2068
+           NPY_FLOAT16, NPY_FLOAT32, NPY_FLOAT64, NPY_FLOAT80, NPY_FLOAT96, NPY_FLOAT128, NPY_FLOAT256#
  2069
+*/
  2070
+#if defined(@typen@)
  2071
+static void @type@_inplace_add(PyArrayMapIterObject *mit, PyArrayIterObject *it)
  2072
+{
  2073
+    int index = mit->size;
  2074
+    while (index--) {
  2075
+        ((@type@*)mit->dataptr)[0] = ((@type@*)mit->dataptr)[0] + ((@type@ *)it->dataptr)[0];
  2076
+
  2077
+        PyArray_MapIterNext(mit);
  2078
+        PyArray_ITER_NEXT(it);
  2079
+    }
  2080
+}
  2081
+#endif
  2082
+/**end repeat**/
  2083
+
  2084
+/**begin repeat
  2085
+* #type  = npy_complex32, npy_complex64, npy_complex128, npy_complex160, npy_complex192, npy_complex512#
  2086
+* #typen = NPY_COMPLEX32, NPY_COMPLEX64, NPY_COMPLEX128, NPY_COMPLEX160, NPY_COMPLEX192, NPY_COMPLEX512#
  2087
+*/
  2088
+#if defined(@typen@)
  2089
+static void @type@_inplace_add(PyArrayMapIterObject *mit, PyArrayIterObject *it)
  2090
+{
  2091
+    int index = mit->size;
  2092
+    while (index--) {
  2093
+        ((@type@*)mit->dataptr)[0].real = ((@type@*)mit->dataptr)[0].real + ((@type@ *)it->dataptr)[0].real;
  2094
+        ((@type@*)mit->dataptr)[0].imag = ((@type@*)mit->dataptr)[0].imag + ((@type@ *)it->dataptr)[0].imag;
  2095
+
  2096
+        PyArray_MapIterNext(mit);
  2097
+        PyArray_ITER_NEXT(it);
  2098
+    }
  2099
+}
  2100
+#endif
  2101
+/**end repeat**/
  2102
+
  2103
+inplace_map_binop addition_funcs[] = {
  2104
+/**begin repeat
  2105
+* #type  = npy_int8, npy_int16, npy_int32, npy_int64, npy_int128, npy_int256,
  2106
+           npy_uint8, npy_uint16, npy_uint32, npy_uint64, npy_uint128, npy_uint256, 
  2107
+           npy_float16, npy_float32, npy_float64, npy_float80, npy_float96, npy_float128, npy_float256,
  2108
+           npy_complex32, npy_complex64, npy_complex128, npy_complex160, npy_complex192, npy_complex512#
  2109
+
  2110
+* #typen = NPY_INT8, NPY_INT16, NPY_INT32, NPY_INT64, NPY_INT128, NPY_INT256,
  2111
+           NPY_UINT8, NPY_UINT16, NPY_UINT32, NPY_UINT64, NPY_UINT128, NPY_UINT256, 
  2112
+           NPY_FLOAT16, NPY_FLOAT32, NPY_FLOAT64, NPY_FLOAT80, NPY_FLOAT96, NPY_FLOAT128, NPY_FLOAT256,
  2113
+           NPY_COMPLEX32, NPY_COMPLEX64, NPY_COMPLEX128, NPY_COMPLEX160, NPY_COMPLEX192, NPY_COMPLEX512#
  2114
+*/
  2115
+#if defined(@typen@)
  2116
+@type@_inplace_add,
  2117
+#endif
  2118
+/**end repeat**/
  2119
+NULL}; 
  2120
+
  2121
+int type_numbers[] = {
  2122
+/**begin repeat
  2123
+* #typen = NPY_INT8, NPY_INT16, NPY_INT32, NPY_INT64, NPY_INT128, NPY_INT256,
  2124
+           NPY_UINT8, NPY_UINT16, NPY_UINT32, NPY_UINT64, NPY_UINT128, NPY_UINT256, 
  2125
+           NPY_FLOAT16, NPY_FLOAT32, NPY_FLOAT64, NPY_FLOAT80, NPY_FLOAT96, NPY_FLOAT128, NPY_FLOAT256,
  2126
+           NPY_COMPLEX32, NPY_COMPLEX64, NPY_COMPLEX128, NPY_COMPLEX160, NPY_COMPLEX192, NPY_COMPLEX512#
  2127
+*/
  2128
+#if defined(@typen@)
  2129
+@typen@,
  2130
+#endif
  2131
+/**end repeat**/
  2132
+-1000}; 
  2133
+
  2134
+static int
  2135
+map_increment(PyArrayMapIterObject *mit, PyObject *op, inplace_map_binop add_inplace)
  2136
+{
  2137
+    PyArrayObject *arr = NULL;
  2138
+    PyArrayIterObject *it;
  2139
+    PyArray_Descr *descr;
  2140
+
  2141
+    if (mit->ait == NULL) {
  2142
+        return -1;
  2143
+    }
  2144
+    descr = PyArray_DESCR(mit->ait->ao);
  2145
+    Py_INCREF(descr);
  2146
+    arr = (PyArrayObject *)PyArray_FromAny(op, descr,
  2147
+                                0, 0, NPY_ARRAY_FORCECAST, NULL);
  2148
+    if (arr == NULL) {
  2149
+        return -1;
  2150
+    }
  2151
+    
  2152
+    if ((mit->subspace != NULL) && (mit->consec)) {
  2153
+        if (mit->iteraxes[0] > 0) {  
  2154
+            _swap_axes(mit, (PyArrayObject **)&arr, 0);
  2155
+            if (arr == NULL) {
  2156
+                return -1;
  2157
+            }
  2158
+        }
  2159
+    }
  2160
+
  2161
+    if ((it = (PyArrayIterObject *)\
  2162
+            PyArray_BroadcastToShape(arr, mit->dimensions, mit->nd)) == NULL) {
  2163
+        Py_DECREF(arr);	
  2164
+        
  2165
+        return -1;
  2166
+    }
  2167
+
  2168
+    
  2169
+
  2170
+    PyArray_MapIterReset(mit);
  2171
+
  2172
+    (*add_inplace)(mit, it);
  2173
+
  2174
+    Py_DECREF(arr);
  2175
+    Py_DECREF(it);
  2176
+    return 0;
  2177
+}
  2178
+
  2179
+
  2180
+static PyObject *
  2181
+inplace_increment(PyObject *dummy, PyObject *args)
  2182
+{
  2183
+    PyObject *arg_a = NULL, *index=NULL, *inc=NULL;
  2184
+    PyArrayObject *a;
  2185
+    inplace_map_binop add_inplace = NULL; 
  2186
+    int type_number = -1;
  2187
+    int i =0;
  2188
+    PyArrayMapIterObject * mit;
  2189
+    
  2190
+    if (!PyArg_ParseTuple(args, "OOO", &arg_a, &index,
  2191
+            &inc)) { 
  2192
+        return NULL;
  2193
+    }
  2194
+    if (!PyArray_Check(arg_a)) {
  2195
+         PyErr_SetString(PyExc_ValueError, "needs an ndarray as first argument");
  2196
+         return NULL;
  2197
+    }
  2198
+    a = (PyArrayObject *) arg_a;
  2199
+    
  2200
+    if (PyArray_FailUnlessWriteable(a, "input/output array") < 0) {
  2201
+        return NULL;
  2202
+    }   
  2203
+
  2204
+    if (PyArray_NDIM(a) == 0) {
  2205
+        PyErr_SetString(PyExc_IndexError, "0-d arrays can't be indexed.");
  2206
+        return NULL; 
  2207
+    }
  2208
+    type_number = PyArray_TYPE(a);
  2209
+    
  2210
+
  2211
+
  2212
+
  2213
+    while (type_numbers[i] >= 0 && addition_funcs[i] != NULL){    
  2214
+        if (type_number == type_numbers[i]) {
  2215
+            add_inplace = addition_funcs[i];
  2216
+            break;
  2217
+        }
  2218
+        i++ ;
  2219
+    }
  2220
+    
  2221
+    if (add_inplace == NULL) {
  2222
+        PyErr_SetString(PyExc_TypeError, "unsupported type for a"); 
  2223
+        return NULL;
  2224
+    }
  2225
+    
  2226
+    mit = (PyArrayMapIterObject *) PyArray_MapIterNew(index, 0, 1);
  2227
+    if (mit == NULL) {
  2228
+        goto fail;
  2229
+    }
  2230
+    PyArray_MapIterBind(mit, a);
  2231
+    if (map_increment(mit, inc, add_inplace) != 0) {
  2232
+        goto fail;
  2233
+    }
  2234
+    
  2235
+    Py_DECREF(mit);
  2236
+    
  2237
+    Py_INCREF(Py_None);
  2238
+    return Py_None;
  2239
+
  2240
+fail:
  2241
+    Py_XDECREF(mit);
  2242
+
  2243
+    return NULL;
  2244
+}
  2245
+
2059 2246
 static void
2060 2247
 arraymapiter_dealloc(PyArrayMapIterObject *mit)
2061 2248
 {
3  numpy/core/src/multiarray/mapping.h
@@ -59,4 +59,7 @@ PyArray_MapIterBind(PyArrayMapIterObject *, PyArrayObject *);
59 59
 NPY_NO_EXPORT PyObject*
60 60
 PyArray_MapIterNew(PyObject *, int, int);
61 61
 
  62
+static PyObject *
  63
+inplace_increment(PyObject *dummy, PyObject *args);
  64
+
62 65
 #endif
3  numpy/core/src/multiarray/multiarraymodule.c
@@ -3677,6 +3677,9 @@ static struct PyMethodDef array_module_methods[] = {
3677 3677
     {"test_interrupt",
3678 3678
         (PyCFunction)test_interrupt,
3679 3679
         METH_VARARGS, NULL},
  3680
+	{"inplace_increment",
  3681
+        (PyCFunction)inplace_increment,
  3682
+        METH_VARARGS, NULL},	
3680 3683
     {NULL, NULL, 0, NULL}                /* sentinel */
3681 3684
 };
3682 3685
 
49  numpy/core/tests/test_inplace_increment.py
... ...
@@ -0,0 +1,49 @@
  1
+import numpy as np
  2
+from numpy.testing import *
  3
+from numpy.testing.utils import WarningManager
  4
+import warnings
  5
+
  6
+
  7
+class TestInplaceIncrement(TestCase):
  8
+	def test_dtypes(self):
  9
+		def testdt(dt):
  10
+			a = np.arange(12).reshape((3,4)).astype(dt)
  11
+			index = ([1,1,2,0], [0,0,2,3])
  12
+			inc = [50,50, 30,16]
  13
+			
  14
+			np.inplace_increment(a, index, inc)
  15
+			
  16
+			assert_array_almost_equal(a, np.array([[   0,    1,    2,   19],
  17
+											[ 104,    5,    6,    7],
  18
+											[   8,    9,   40,   11]]).astype(dt))
  19
+		
  20
+		testdt(np.float)
  21
+		testdt(np.int)
  22
+		testdt(np.complex)
  23
+		
  24
+	def test_slice(self):
  25
+		a = np.arange(12).reshape((3,4)).astype(float)
  26
+		index = (1, slice(None, None))
  27
+		inc = [50,50, 30,16]
  28
+
  29
+		np.inplace_increment(a, index, inc)
  30
+
  31
+		assert_array_almost_equal(a, np.array([[   0,    1,    2,   3],
  32
+											[ 54,    55,    36,  23],
  33
+											[   8,    9,   10,   11]]).astype(float))
  34
+										
  35
+	def test_boolean(self):
  36
+		a = np.arange(12).reshape((3,4)).astype(float)
  37
+		index = (1, np.array([ False, True, False, True], dtype = bool))
  38
+		inc = [30,16]
  39
+		
  40
+		np.inplace_increment(a, index, inc)
  41
+		
  42
+		assert_array_almost_equal(a, np.array([[   0,    1,    2,   3],
  43
+											[   4,   35,    6,  23],
  44
+											[   8,    9,   10,   11]]).astype(float))
  45
+
  46
+		
  47
+
  48
+if __name__ == "__main__":
  49
+	run_module_suite()
Commit_comment_tip

Tip: You can add notes to lines in a file. Hover to the left of a line to make a note

Something went wrong with that request. Please try again.