Investment costs

[nina-vincent]

Note: Not all tests are done.

This adds cost calculations for all new (added since base_grid) transmission, generation, and storage capacity.

create_mapping_files.py contains functions that help create shapefiles and map lat/lon points to regions from shapefiles.
investment_costs.py uses these mappings to find the regional cost multipliers (for both generation/storage and ac transmission).

Fundamental calculations

There are 3 separate cost calculations: one for dclines, one for branches, one for plants.

branches: calculate_ac_inv_costs

For all capacity added on a line, the investment cost is:
Cost ($2010) = rateA (MW) * lengthMi (miles) * costMWmi ($2010/MW-mi) * mult (regional cost multiplier, unitless)
Then all lines are summed.

For all capacity added on a transformer, the investment cost is:
Cost (per transformer at a given kV level, $2010)

Now that I'm thinking about this again, this is a flawed method because if we're only slightly scaling a transformer, this is overestimating the cost. Might be better to apply to completely new transformers only? Either ignore this cost or reevaluate its calculation.

dclines: calculate_dc_inv_costs

For all capacity added on a dcline, the investment cost is:
Cost ($2015) = Pmax (MW) * lengthMi (miles) * costMWmi ($2015/MW-mi) + dc_term_cost (DC terminal cost $2015)
Then all line costs are summed.

plant: calculate_gen_inv_costs

For all capacity added on a plant, the investment cost is:
Cost ($2018) = Pmax (MW) * CAPEX ($2018/MW) * reg_cap_cost_mult (regional capital cost multiplier, unitless)
Then all costs are summed by technology (so you can ignore the non-renewables/storage if you want).

Methods

branches: calculate_ac_inv_costs

1. Find new upgrades for each line/transformer (change in MW rating): grid.branch.rateA - base_grid.branch.rateA
2. Drop unchanged branches.
3. Map branches to corresponding kV.
4. Separate lines and transformers (TransformerWindings dropped).
5. Lines: Find closest kV present in the corresponding cost table using select_kv . Labeled "kV_cost."
6. Lines: Find MW closest to the line's rateA that exists within the corresponding kV_cost sub-table. Then use that costMWmi base cost value.
7. (messy) Lines: Import bus to NEEM mapping file. Check that no buses you need are missing. Check that buses have the same lat/lon values as the pre-made mapping file. If any issues, re-map NEEM regions to those points.
8. Lines: Map regional multipliers onto lines' to_bus and from_bus by NEEM region.
9. Lines: Regional multiplier is then the average of the 2 buses' regional multipliers.
10. Lines: final calculations
11. Xfmrs: Find closest kV present in the corresponding cost table using select_kv . Labeled "kV_cost."
12. Xmrs: Map kV_cost to corresponding transformer cost.

Warning: Now that I think about this again, I bet the transformer costs are too high with this method. If we upgrade the rating just slightly, it's going to count this as an entirely new transformer. We could just drop this investment cost altogether, or just include on totally new transformers? Not sure.

dcline: calculate_dc_inv_costs

1. Find new capacity for each dcline (change in capacity): grid.dcline.Pmax - base_grid.dcline.Pmax
2. Drop dcline if no changes.
3. Map using buses to get the from/to lat/lon's of line.
4. Find line length. Find MWmi.
5. Only one costMWmi value, so multiply this by MWmi.
6. Add HVDC terminal cost term.

plant: calculate_gen_inv_costs

1. Find new capacity for each plant (change in generation capacity): grid.plant.Pmax - base_grid.plant.Pmax
2. Drop plants < 0.1 MW.
3. Drop "dfo" and "other" because no ATB cost data available.
4. Load in base CAPEX costs from ATB data and select cost year/cost_case out of "Conservative", "Moderate", and "Advanced."
5. Select (arbitrary) TechDetail (sub-technology) because we can't differentiate between these yet in our plants.
6. Map base costs to plants by "Technology".
7. Map plant location to ReEDS region (for regional cost multipliers).
8. (messy) If a technology is wind or wind_offshore or csp, regions are in rs (wind resource region), so keep rs is kept as the region to map. If technology is another tech, regions are in rb (BA region), so keep rb as the region to map.
9. Map ["Technology","r" (region)] to ReEDS regional capital cost multipliers. Keep (arbitrary) subclasses for renewables.
10. Final calculations.

Warning: there's a lot of messy code here right now because I'm manually dropping a lot of subtypes of technologies so we only have one type.

Mapping functions

sjoin_nearest: joins a geodataframe of Points and Polygons/Multipolygons. Used in points_to_polys.

points_to_polys: joins a dataframe (which includes lat and lon columns) with a shapefile. Used in bus_to_neem_reg and plant_to_reeds_reg.

Functions used for AC regional multiplier mapping

bus_to_neem_reg: maps bus locations to NEEM regions. Used in write_bus_neem_map and (if there are errors in the mapping file produced in write_bus_neem_map), this function is also used in _calculate_ac_inv_costs.

write_bus_neem_map: maps all base_grid bus locations to NEEM regions and produces a csv mapping file: regionsNEEM.shp. This csv is used in _calculate_ac_inv_costs.

Functions used for generation/storage regional multiplier mapping

write_poly_shapefile: Using a csv with specified geometry, creates the shapefile for ReEDS wind resource regions: rs.shp. This shp is used in plant_to_reeds_reg.

plant_to_reeds_reg: maps plant locations to ReEDS regions. Used in _calculate_gen_inv_costs.

Sources (more specifics to follow in README)

branches: calculate_ac_inv_costs

• base costs: EIPC
• regional multipliers: EIPC/NEEM

branches: calculate_dc_inv_costs

• base costs: EIPC

plant: calculate_gen_inv_costs

• base costs: NREL's 2020 ATB
• regional multipliers: ReEDS 2.0 Version 2019

To do:

• requirements.txt not necessarily correct -- will need to fix pipfile.

• Untested on storage scenarios.

• Write tests for calculate_gen_inv_costs.

• $Dollar-years are inconsistent. Need a function to convert to inflate to proper year.

• If we want to have financial information other than the default ATB values, a separate financials module will be useful for CAPEX/other calculations.

• Figure out issue with transformers described above.

• Find correct wind and solar classes (based on wind speed, irradience) to map to ATB costs and ReEDS regional cost multipliers.

• check that not using costs for underground lines.

[danielolsen]

I believe the tests are failing because not all required dependencies are being installed; I see the tests fail because they can't import fiona, even though it's listed in requirements.txt. @jon-hagg can you help us with this? Or is there a wiki page that guides us through regenerating all the files that will get CI working properly?

[danielolsen]

I see several small CSV files that seems like they could be combined into a single module containing a few dicts/constants:

• Data/HVDC.csv
• Data/HVDCTerminal.csv
• Data/LineBase.csv
• Data/Transformers.csv

Storing the data this way would also allow comments discussing things like units, sources, etc. Do you think a refactor like this would be compatible with the calculation workflow?

[danielolsen]

There are a few very large CSV files. Which of these are source data, and which can be calculated using the source data and the scripts in this PR?

[jenhagg]

Yeah I can help - we'll need to add the packages in Pipfile, then run pipenv lock --pre to regenerate Pipfile.lock (actual source of truth for the tests). The requirements.txt is something we maintain for convenience but isn't used in our ci pipeline. Let me know if you want me to do that and push changes to your branch. One question I had - do we need setuptools in the requirements? I'm guessing we can omit that since it's included by default in virtual environments

[rouille]

The Data folder should be rename data (lower case). That is a convention we have.

[rouille]

I see several small CSV files that seems like they could be combined into a single module containing a few dicts/constants:

* Data/HVDC.csv

* Data/HVDCTerminal.csv

* Data/LineBase.csv

* Data/Transformers.csv

Storing the data this way would also allow comments discussing things like units, sources, etc. Do you think a refactor like this would be compatible with the calculation workflow?

That would be nice. You can look at powersimdata.network.usa_tamu.constants.plants or powersimdata.network.usa_tamu.constants.zones for an example.

[rouille]

Yeah I can help - we'll need to add the packages in Pipfile, then run pipenv lock --pre to regenerate Pipfile.lock (actual source of truth for the tests). The requirements.txt is something we maintain for convenience but isn't used in our ci pipeline. Let me know if you want me to do that and push changes to your branch. One question I had - do we need setuptools in the requirements? I'm guessing we can omit that since it's included by default in virtual environments

I don't believe we need setuptools in the requirements.

[nina-vincent]

There are a few very large CSV files. Which of these are source data, and which can be calculated using the source data and the scripts in this PR?

Which CSV files are you referring to?

(Direct) Source data:

• everything in NEEM dir

• everything in mapping dir

• reg_cap_cost_mult_default.csv

Source data pulled from a paper (so formatting can be changed easily):

• HVDC.csv

• HVDCTerminal.csv

• LineBase.csv

• LineRegMult.csv

• Transformers.csv

Calculated using script in PR:

-buses_NEEMregion.csv (this is created from a function using NEEM dir in this PR). It takes some hours to run (I didn't time it exactly). I have a backup function that uses the NEEM dir (and the mapping function) if any buses are excluded or don't match this csv file.

• everything in rs dir

• plant_region.csv can be excluded, it's unused and I left it on accident. I'll delete this in the next commit.

[nina-vincent]

I see several small CSV files that seems like they could be combined into a single module containing a few dicts/constants:

• Data/HVDC.csv
• Data/HVDCTerminal.csv
• Data/LineBase.csv
• Data/Transformers.csv

Storing the data this way would also allow comments discussing things like units, sources, etc. Do you think a refactor like this would be compatible with the calculation workflow?

Yes, this is possible.

[rouille]

should grid be a parameter of the function? In the future, the Gridclass could be instantiated withEurope`, who knows!

[nina-vincent]

For this method of calculating investment costs, we have regional multipliers that only exist for regions in the U.S. (the shapefile has NEEM regions). This whole thing would break for Europe, for sure.

[danielolsen]

There are a few very large CSV files. Which of these are source data, and which can be calculated using the source data and the scripts in this PR?

Which CSV files are you referring to?

Thanks, your list clears things up a bit. I think in general we want to avoid having unnecessary large data files in the repos, but it seems like most of the big ones are source data. Let's document the provenance of these as well, perhaps with README files in the appropriate subdirectories.

[danielolsen]

Attribution is added, so I believe this is ready for any necessary final review. Two new features in the last week:

• feat: allow user to get non-summed inv costs (ac, dc, gen): get the per-line/per-generator upgrade costs, so that we can do more granular analysis (e.g. upgrade costs by location).
• feat: add exclude_branches to calculate_ac_inv_costs: self-explanatory I think. E.g. if we don't want to count the cost of offshore wind connection cables.

[rouille]

Are you done with cleaning the git commit history?

[danielolsen]

Are you done with cleaning the git commit history?

We could always make it cleaner... but I think the vast majority of the commits now are either meaningful or hard-to-squash.