ENH: Efficient interpolation on regular grids in arbitrary dimensions #3323

andreas-h · 2014-02-13T09:34:28Z

Based on Johannes Buchner's regulargrid package (see https://github.com/JohannesBuchner/regulargrid), I implemented a new class RegularGridInterpolator for efficient linear and nearest-neighbour interpolation on regular (possibly unevenly spaced) grids in arbitrary dimensions.

I believe this is a significant addition to the scipy.interpolate package because the existing ND-interpolators perform a triangulation of the input points with becomes unfeasible with medium (~8) dimensions. There, by taking advantage of the regular grid structure, the interpolation can be sped up quite a bit.

Another advantage of this class is that it accepts anthing that can be appropriately indexed as values parameter, meaning that it is possible to use this interpolation object for interpolation of disk-based data sets (netCDF, pytables, ...).

I asked for permission about inclusion in Scipy; the (positive) answer is here: JohannesBuchner/regulargrid#2.

pv · 2014-02-13T09:53:48Z

Yes, this is a very commonly needed feature, and currently missing. What is needed is an interpolation method that does not make a copy of the data or do any expensive preprocessing, and this seems to fit the bill.

However, I suggest at the same time also adding a convenience function interpn. This should be similar in spirit to griddata i.e. a front-end to various interpolation methods, but for data on a rectangular grid, and having a method= keyword arg. The other methods (in addition to "nearest" and "linear" you add here) currently available is "splinef2d" (for RectBivariateSpline) and that's restricted to 2D.

The point of interpn would be to guide newbies away from interp2d (which probably should be deprecated at some point, due to its wonkiness and the fact that it's FITPACK-based).

The interface probably should mimic griddata by being interpn((x, y, z, ...), values, xi).

andreas-h · 2014-02-13T10:07:13Z

Which other interpolation methods do you have in mind? Otherwise,
|interpn| would just be a wrapper around |RegularGridInterpolator|, correct?

coveralls · 2014-02-13T10:16:45Z

Changes Unknown when pulling e23aa50 on andreas-h:regulargrid into * on scipy:master*.

coveralls · 2014-02-13T10:21:59Z

Changes Unknown when pulling e23aa50 on andreas-h:regulargrid into * on scipy:master*.

pv · 2014-02-13T10:48:32Z

@andreas-h: RectBivariateSpline, and any other methods that may be eventually added

pv · 2014-02-13T10:50:19Z

scipy/interpolate/interpolate.py

+
+    References
+    ----------
+    - Python package *regulargrid* by Johannes Buchner, see https://pypi.python.org/pypi/regulargrid/


RST reference format

.. [1] Python package ... .. [2] Foo bar. Lorem ipsum.

andreas-h · 2014-02-13T10:52:39Z

@pv: Doesn't |RectBivariateSpline| only support the 2d case?

I should be able to do that this weekend.

Do you think there's any chance this will make it into 0.14?

pv · 2014-02-13T11:01:39Z

Yes, it's 2D only (as I noted above). It needs to be documented and an error raised for other dimensions.

The optional parameters shouldn't be available in interpn, I believe. This is intended to be a simple interface, for interpolation only. People who want more control, can use the interpolators directly.

pv · 2014-02-13T11:03:39Z

We can try to get this (and interpn) to 0.14.0.

andreas-h · 2014-02-13T13:00:41Z

Currently in the nearest neighbor implementation, I'm using yi <= .5 as condition to determine if a given coordinate "belongs" to the next upper or lower gridpoint. Is there any convention on how to handle this? Should I explicitly mention this behaviour in the docstring?

coveralls · 2014-02-13T22:51:43Z

Changes Unknown when pulling a68a922 on andreas-h:regulargrid into * on scipy:master*.

…ND-interpolators

coveralls · 2014-02-14T08:48:28Z

Changes Unknown when pulling 8fbf99b on andreas-h:regulargrid into * on scipy:master*.

andreas-h · 2014-02-14T09:55:54Z

@pv Do you think it's a good idea to add bounds_error and fill_value to the interpn interface?

pv · 2014-02-14T10:25:31Z

scipy/interpolate/interpolate.py

+    if method not in ["linear", "nearest", "splinef2d"]:
+        raise ValueError("interpn only understands the methods 'linear', "
+                         "'nearest', and 'splinef2d'. You provided "
+                         "{}".format(method))


This is not valid in Python 2.6 which we still support. Better use the % string formatting.

pv · 2014-02-14T10:34:25Z

Not sure about bounds_error and fill_value. griddata has fill_value.
RectBivariateSpline doesn't seem to support either.

andreas-h · 2014-02-14T10:37:31Z

The only problem would be extrapolation, which wouldn't be supported for
splinef2d. bounds_error with a given fill_value could be handled in the
wrapper by checking for out_of_bounds before the call to
RectBivariateSpline and filling the return array before returning. I
think I'd be in favor of adding both to interpn. Will do so over the
weekend, if you don't object.

pv · 2014-02-14T10:38:55Z

I don't see problems with that.

coveralls · 2014-02-14T10:39:52Z

Changes Unknown when pulling e5918ef on andreas-h:regulargrid into * on scipy:master*.

ev-br · 2014-02-14T15:36:24Z

Re nearest neighbor implementation (y <= 0.5 etc). While I don't have any intelligent suggestions here, I just note that there's some discussion on data model vs pixel model in the roadmap under ndimage.

pv · 2014-02-14T15:40:33Z

I think we want here nearest-neighbor semantics, ie. voronoi cells. I.e. 0.5 seems fine to me.

andreas-h · 2014-02-19T20:07:58Z

one further question is about the kwarg method or kind. interp1d and interp2d use kind, griddata uses method. I personally prefer method, so would go with this for RegularGridInterpolator and interpn.

What do you think?

pv · 2014-02-19T20:11:31Z

I'd go with method, as it's used a bit more often in scipy than kind.

coveralls · 2014-02-19T22:13:31Z

Coverage remained the same when pulling c5632f7 on andreas-h:regulargrid into 32cd96d on scipy:master.

coveralls · 2014-02-19T22:31:50Z

Coverage remained the same when pulling c5632f7 on andreas-h:regulargrid into 32cd96d on scipy:master.

coveralls · 2014-02-19T22:46:01Z

Coverage remained the same when pulling c5632f7 on andreas-h:regulargrid into 32cd96d on scipy:master.

pv · 2014-02-19T22:47:31Z

scipy/interpolate/interpolate.py

+    grid = tuple([np.asarray(p) for p in points])
+
+    # sanity check requested xi
+    xi = np.atleast_2d(xi)


I do not fully understand your point """griddata expects xi to be of shape (M, D). My intention was to allow 1D xi of shape (D, ) as well""". You mean you want to evaluate at a single point?

However, I think uniform interface with griddata is of high priority here. The xi parameter has exactly the same meaning, so it should behave the same way.

The problem with atleast_2d is that it does not broadcast. If you want to evaluate it on a grid, you need to do a meshgrid+ravel dance on the user side.

The other option is to add an "expected dimension" argument to _ndim_coords_from_arrays, and transpose 1D the array if >1D is expected.
Can be done if it doesn't cause some inputs to be ambiguous. (It probably doesn't cause ambiguity.)

pv · 2014-02-19T22:58:50Z

I think this is almost ready now, and should make it to 0.14.0.

pv · 2014-02-23T16:32:46Z

Some missing stuff is added in: pv/scipy-work@andreas-h:regulargrid...pr-3323

uniform xi argument handling in interpn and griddata (and it does as suggested above for 1D inputs)
support for non-scalar valued values in regular grid interpolation
improve tests and fix up array_like handling at some points
minor doc improvements

andreas-h · 2014-02-23T16:42:48Z

Looks great! I wanted to add the lower-dim xi case last night but couldn't figure out how to do the proper slicing ... So thanks for reading my thoughts and implementing them =)

Do I need to do anything with this PR or do you do the necessary git magic before merging?

andreas-h · 2014-02-23T16:43:33Z

I hope I can still write an example for interpn tonight or tomorrow, but don't want to promise that. So probably you go ahead merging and I can submit the example PR against master then.

pv · 2014-02-23T16:43:40Z

@andreas-h: comments? I'd be ready to merge to 0.14.0 with those changes.

pv · 2014-02-23T16:44:24Z

Ok, I'll merge this and any remaining enhancements can then be made later.

ENH: interpolate: add interpn and RegularGridInterpolator Implemented a new class RegularGridInterpolator for efficient linear and nearest-neighbour interpolation on regular (possibly unevenly spaced) grids in arbitrary dimensions. The existing ND-interpolators perform a triangulation of the input points with becomes unfeasible with medium (~8) dimensions. There, by taking advantage of the regular grid structure, the interpolation can be sped up quite a bit. Another advantage of this class is that it accepts anthing that can be appropriately indexed as values parameter, meaning that it is possible to use this interpolation object for interpolation of disk-based data sets (netCDF, pytables, ...). This PR also adds a simpler function `interpn` for regular grid data interpolation.

pv · 2014-02-23T17:22:03Z

Thanks, merged in a90dc28

ENH: Efficient interpolation on regular grids in arbitrary dimensions

e23aa50

pv reviewed Feb 13, 2014
View reviewed changes

pv added this to the 0.14.0 milestone Feb 13, 2014

ENH: add fill_value and bounds_error to RegularGridInterpolator

a68a922

TST: test for equivalence of RegularGridInterpolator and qhull-based …

8fbf99b

…ND-interpolators

ENH: add new convenience method scipy.interpolate.interpn

e5918ef

pv reviewed Feb 14, 2014
View reviewed changes

use old-style string formatting to support python 2.6

d36dbed

pv added the needs-work label Feb 19, 2014

pv added the PR label Feb 19, 2014

ENH: support fill_value and bounds_error in interpn

3155a09

pv reviewed Feb 19, 2014
View reviewed changes

STY: rename kwarg 'kind' to 'method'

d322a87

pv closed this Feb 23, 2014

andreas-h deleted the regulargrid branch February 24, 2014 10:38

ev-br mentioned this pull request Jan 9, 2023

BUG: interpolate/RGI: upcast float32 to float64 #17726

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ENH: Efficient interpolation on regular grids in arbitrary dimensions #3323

ENH: Efficient interpolation on regular grids in arbitrary dimensions #3323

andreas-h commented Feb 13, 2014

pv commented Feb 13, 2014

andreas-h commented Feb 13, 2014

coveralls commented Feb 13, 2014

coveralls commented Feb 13, 2014

pv commented Feb 13, 2014

pv Feb 13, 2014

andreas-h commented Feb 13, 2014

pv commented Feb 13, 2014

pv commented Feb 13, 2014

andreas-h commented Feb 13, 2014

coveralls commented Feb 13, 2014

coveralls commented Feb 14, 2014

andreas-h commented Feb 14, 2014

pv Feb 14, 2014

pv commented Feb 14, 2014

andreas-h commented Feb 14, 2014

pv commented Feb 14, 2014

coveralls commented Feb 14, 2014

ev-br commented Feb 14, 2014

pv commented Feb 14, 2014

andreas-h commented Feb 19, 2014

pv commented Feb 19, 2014

coveralls commented Feb 19, 2014

coveralls commented Feb 19, 2014

coveralls commented Feb 19, 2014

pv Feb 19, 2014

pv Feb 19, 2014

pv commented Feb 19, 2014

pv commented Feb 23, 2014

andreas-h commented Feb 23, 2014

andreas-h commented Feb 23, 2014

pv commented Feb 23, 2014

pv commented Feb 23, 2014

pv commented Feb 23, 2014

ENH: Efficient interpolation on regular grids in arbitrary dimensions #3323

ENH: Efficient interpolation on regular grids in arbitrary dimensions #3323

Conversation

andreas-h commented Feb 13, 2014

pv commented Feb 13, 2014

andreas-h commented Feb 13, 2014

coveralls commented Feb 13, 2014

coveralls commented Feb 13, 2014

pv commented Feb 13, 2014

pv Feb 13, 2014

Choose a reason for hiding this comment

andreas-h commented Feb 13, 2014

pv commented Feb 13, 2014

pv commented Feb 13, 2014

andreas-h commented Feb 13, 2014

coveralls commented Feb 13, 2014

coveralls commented Feb 14, 2014

andreas-h commented Feb 14, 2014

pv Feb 14, 2014

Choose a reason for hiding this comment

pv commented Feb 14, 2014

andreas-h commented Feb 14, 2014

pv commented Feb 14, 2014

coveralls commented Feb 14, 2014

ev-br commented Feb 14, 2014

pv commented Feb 14, 2014

andreas-h commented Feb 19, 2014

pv commented Feb 19, 2014

coveralls commented Feb 19, 2014

coveralls commented Feb 19, 2014

coveralls commented Feb 19, 2014

pv Feb 19, 2014

Choose a reason for hiding this comment

pv Feb 19, 2014

Choose a reason for hiding this comment

pv commented Feb 19, 2014

pv commented Feb 23, 2014

andreas-h commented Feb 23, 2014

andreas-h commented Feb 23, 2014

pv commented Feb 23, 2014

pv commented Feb 23, 2014

pv commented Feb 23, 2014