Tech Potential Entry Point for RED-E

[grantbuster]

Tech Potential Tool Outline
We need a lean and mean version of reV aggregation to replace the previous tech pot tool. Basically we only need to apply exclusions to pre-computed cf means.

Requirements

1. New endpoint (Either CLI or class method)
2. Resource data is cf_means (formatting is to be done by interns maybe, currently in raster format)
3. Flexible exclusion layer inputs (need to format into h5 from current geotiffs)
4. Run on docker - we can run reV from CLI within docker - actually will probably call from python so we just develop a class method entry point
5. What they really need as output is a 2D spatial lat/lon array corresponding to exclusion grid.
6. Generation and exclusions are on the same resolution (1-to-1 boolean exclusions)
7. Parallel compute is okay/acceptable.

Timeline
Start on this in 2019, goal is to finish in 1st half of January.

Charge code
Charge code TBD from ricardo/haiku.

[grantbuster]

FYI Evan brought up the issue of potentially having a global dataset to run this tool from in which case we need to consider:

1. A raster layer of the pre-computed pixel areas (kind of an area meta data 2D array in the case the raster projection is not equal area)
2. Steamlined methods for only analyzing a select portion of the full extent (we kind of need this regardless)

[Ricardo-C-Oliveira]

@EvanRosenliebNREL any ideas on this?

[MRossol]

(2) is already implemented into the code. We'll just have to change how we extract data in RED-E.

[EvanRosenliebNREL]

I think that making an array of pixel areas for a global grid for projection 4326 works perfectly fine, the only potential downfall I can think of is this:

As far as I can tell, to the extent that:

Generation and exclusions are on the same resolution (1-to-1 boolean exclusions)

means that one way or the other all of the different data sets end up getting resampled to the lowest common denominator. Currently in the tech pot code that lowest common denominator is 90m pixels, as this is the resolution of our finest resolution input (the SRTM elevation/slope data).

Creating an area array for the global 90m grid would mean that we would essentially be hardcoding the 90m resolution into the process. If we wanted to do an analysis at a different resolution in the future (whether it is smaller or larger), we would first have to create the pixel area array at that resolution.

Does this sound right to you guys, or am I fundamentally misunderstanding something about how you guys were envisioning the process would work?

[grantbuster]

@EvanRosenliebNREL, you comment makes sense but I definitely didnt think that the TechPot tool was presenting results on a 90m grid? I think we can all agree 90m global techpot seems a bit ambitious. Should we maybe resample to the most coarse resolution not the finest?

[Ricardo-C-Oliveira]

I guess we never really looked into what the optimal resolution should be for tech pot. For some countries, like Singapore, a 90m resolution is nice, but becomes a bit pointless for countries like India or Thailand.

[grantbuster]

@Ricardo-C-Oliveira, does this suggest that each country will have its own set of layers with disparate resolutions? If so, I think that's fine, we just need to be aware of it.

[Ricardo-C-Oliveira]

All layers will have the same resolution within the domain extent. During the data prep all input layers are resampled to match the finest res, in the case slope at 90m.

[EvanRosenliebNREL]

@Ricardo-C-Oliveira @MRossol @grantbuster

Sorry guys, I was being a dummy when Grant asked me for my public github account and gave him an account that I don't get notifications for very frequently. I've asked him to invite my nrel.gov account.

In the meantime... I'm afraid I don't necessarily agree that we can up sample to a coarser resolution.

Even going from, say, a 90m grid to a 180m grid, which would be only a reduction of 4x the data, would have drastic affects on the amount of land that would be excluded from the slope parameter. By up sampling slope to a coarser resolution we would essentially be representing everything as being flatter than it is and giving an overly rosy estimate.

I don't want it to make seem like there is anything magic about 90m -- I'm sure that there is some amount of up sampling we could do without drastically different results. I just can't imagine that going from a 90m grid to a 120m grid or whatever is the sort of computational slack we are looking for.

In regards to:

I think we can all agree 90m global techpot seems a bit ambitious.

I hate to say it but -- That is what I was trying to say in that original meeting in the SEAC collaboration room. I was surprised that you guys thought it would be so easy.

At the time I thought, however, that if the new reV is so performant on the continental scale maybe it isn't so crazy? @grantbuster what specifically are you worried most about computationally using a 90m grid?

Or is there something different about the way reV runs that makes that a bad comparison?

[grantbuster]

Action items from 1/30/2020 meeting:

• @MRossol to dump resource array and CF array from data (need multiple output layers)
• @Ricardo-C-Oliveira to produce geotiff layer description json for @EvanRosenliebNREL
• @EvanRosenliebNREL to use reVX exclusions converter to make test data for MRossol (needs CF data and exclusion layers) - for India and a smaller country (Brunei)
• @MRossol to build tests
• @grantbuster merge pull request RED-E Tech Potential Tool #24
• @MRossol to give output sample data to @Ricardo-C-Oliveira or show him how to run the tool