regridding

[bpbond]

No description provided.

[bpbond]

There's a brand new package on CRAN that might be useful: http://cran.r-project.org/web/packages/HiClimR/HiClimR.pdf

[ktoddbrown]

No this is not the function we are looking for. Their regridding 'coarsing' algorithm just pulls a subset of coordinates. I'm going to recode this today and try to clean it up and speed up the regridding algorithm.

[ktoddbrown]

I'll start an alternate version of the regridding since it looks like you are already working on revising the current one. This new algorithm might not work, in which case we'll just go with what we got.

[bpbond]

OK.

---

From: Kathe Todd-Brown [notifications@github.com]
Sent: Tuesday, September 02, 2014 10:36 AM
To: ktoddbrown/CMIP5_R
Cc: Bond-Lamberty, Benjamin
Subject: Re: [CMIP5_R] regridding (#16)

I'll start an alternate version of the regridding since it looks like you are already working on revising the current one. This new algorithm might not work, in which case we'll just go with what we got.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/16#issuecomment-54160683.

[bpbond]

We had a meeting here yesterday with everyone who's working with CMIP5 data, and the value of this package came up repeatedly. People were particularly interested in a future regridding capability.

[ktoddbrown]

The problem with regridding is that there are so gosh darn many algorithms that could be used and what should be used is dependent on what variable you are looking at. I agree thought that this is a high priority feature. In my mind there are three algorithms that we might want to support:

1. linear interpolation
2. area weighted averaging
3. maximum category

We may actually want to look at pulling in someone who does this kind of data processing for climate variables and get some feedback here.

[bpbond]

And there aren't existing implementations of these in R, that we've found?

[ktoddbrown]

raster has implementations for interpolation and max category but I haven't seen anything for area weighted averaging (but I have code for this) and we would have to port to that structure which doesn't play nice with some of the non-regular grids used by certain centers. The package you originally linked to has an intelligent sampling algorithm but not the three mentioned.

In theory you are correct, these are all common but I don't know of a package off hand.

[serbinsh]

Fields has a number of gridding options, including splines: http://cran.r-project.org/web/packages/fields/fields.pdf I have used the Fields package a lot in the past and it seems to work well, even with large datasets.

Spatstat has IDW interpolation options (i am sure there are others): http://www.inside-r.org/packages/cran/spatstat/docs/idw

Kriging can be done in GeoR and gstat, etc.

However the concern about gridding different data types is valid. E.g. what works for temp doesn't generally work for precip; plus with precip you need to deal with the "dribble" issue. So care needs to be taken when choosing an option.

[cahartin]

would it be too difficult to have a few gridding options that the user can pick from?

[ktoddbrown]

Would this be better as a vignette with examples of how to use the results from this package instead of adding it as a function in the package itself?

[bpbond]

Hey Sean @serbinsh thanks for the info!

Kathe, that's a good point; does RCMIP5 have to offer regridding? I guess the answer might be 'yes' if we expect big use cases like data -> regrid -> makeXStat(). But if most work flows are data -> makeStat() -> regrid, maybe not.

[ktoddbrown]

Regridding is synced with master branch. However I can't figure out how to include the dependency for 'plyr'. @bpbond could you take a look at the getProjectionMatrix function in regrid and see if there is a smarter way to do the ldply?

[bpbond]

Hi Kathe, I've rewritten that ldply call as a for loop, which should be (haven't run it) about the same speed, and eliminates the plyr dependency. See commit 09b3082.

[ktoddbrown]

Looks like a bug got introduced... working on it.

[bpbond]

Nuts, sorry.

[ktoddbrown]

Done. I think regridding can be considered implemented. ffe53e7

[bpbond]

When I run the regrid test code from the command line, it doesn't work (below). When I run it as part of the test suite, it doesn't terminate (very slow). Any thoughts?

> test_file("tests/testthat/test_regrid.R")
regrid : 1S

1. Error: regrid returns expected values for simple case -----------------------------------------------------------------------------------------------------------------------------------------------
Cholmod error 'X and/or Y have wrong dimensions' at file ../MatrixOps/cholmod_sdmult.c, line 90
1: withCallingHandlers(eval(code, new_test_environment), error = capture_calls)
2: eval(code, new_test_environment)
3: eval(expr, envir, enclos)
4: regrid(orgVar, projLat, projLon, orgArea = orgArea, projArea = projArea) at test_regrid.R:35
5: apply(orgVar$val, c(3, 4), function(myMap) {
       temp <- myMap * orgArea$val[, , 1, 1]
       temp <- temp[TRUE]
       ans <- as.numeric(temp %*% projectionMatrix)
       dim(ans) <- dim(projVar$lat)
       ans <- ans/projArea$val[, , 1, 1]
       return(ans)
   }) at /Users/d3x290/Documents/Work/Code/RCMIP5/R/regrid.R:154
6: FUN(array(newX[, i], d.call, dn.call), ...)
7: temp %*% projectionMatrix at /Users/d3x290/Documents/Work/Code/RCMIP5/R/regrid.R:157
8: temp %*% projectionMatrix

[ktoddbrown]

I think you may have an old version. Sync and re-run.

[bpbond]

Thanks. Closing.