fix: enable AC investment costs to be calculated for branches connected to new buses

[danielolsen]

Purpose

I found a new bug, where if you try to calculate AC investment costs for a scenario where at least one branch is connected to a new bus, we run into an error. Traceback, using Scenario 3828:

c:\python38\lib\site-packages\powersimdata\design\investment\investment_costs.py in calculate_ac_inv_costs(scenario, sum_results, exclude_branches)
     63         grid_new.branch.drop(index=present_exclude_branches, inplace=True)
     64 
---> 65     costs = _calculate_ac_inv_costs(grid_new, sum_results)
     66     return costs
     67 

c:\python38\lib\site-packages\powersimdata\design\investment\investment_costs.py in _calculate_ac_inv_costs(grid_new, sum_results)
    181     branch = grid_new.branch
    182     branch.loc[:, "kV"] = bus.loc[branch.from_bus_id, "baseKV"].tolist()
--> 183     branch.loc[:, "from_region"] = bus_reg.loc[branch.from_bus_id, "name_abbr"].tolist()
    184     branch.loc[:, "to_region"] = bus_reg.loc[branch.to_bus_id, "name_abbr"].tolist()
    185     # separate transformers and lines

c:\python38\lib\site-packages\pandas\core\indexing.py in __getitem__(self, key)
    887                     # AttributeError for IntervalTree get_value
    888                     return self.obj._get_value(*key, takeable=self._takeable)
--> 889             return self._getitem_tuple(key)
    890         else:
    891             # we by definition only have the 0th axis

c:\python38\lib\site-packages\pandas\core\indexing.py in _getitem_tuple(self, tup)
   1058     def _getitem_tuple(self, tup: Tuple):
   1059         with suppress(IndexingError):
-> 1060             return self._getitem_lowerdim(tup)
   1061 
   1062         # no multi-index, so validate all of the indexers

c:\python38\lib\site-packages\pandas\core\indexing.py in _getitem_lowerdim(self, tup)
    829                     return section
    830                 # This is an elided recursive call to iloc/loc
--> 831                 return getattr(section, self.name)[new_key]
    832 
    833         raise IndexingError("not applicable")

c:\python38\lib\site-packages\pandas\core\indexing.py in __getitem__(self, key)
    893 
    894             maybe_callable = com.apply_if_callable(key, self.obj)
--> 895             return self._getitem_axis(maybe_callable, axis=axis)
    896 
    897     def _is_scalar_access(self, key: Tuple):

c:\python38\lib\site-packages\pandas\core\indexing.py in _getitem_axis(self, key, axis)
   1111                     raise ValueError("Cannot index with multidimensional key")
   1112 
-> 1113                 return self._getitem_iterable(key, axis=axis)
   1114 
   1115             # nested tuple slicing

c:\python38\lib\site-packages\pandas\core\indexing.py in _getitem_iterable(self, key, axis)
   1051 
   1052         # A collection of keys
-> 1053         keyarr, indexer = self._get_listlike_indexer(key, axis, raise_missing=False)
   1054         return self.obj._reindex_with_indexers(
   1055             {axis: [keyarr, indexer]}, copy=True, allow_dups=True

c:\python38\lib\site-packages\pandas\core\indexing.py in _get_listlike_indexer(self, key, axis, raise_missing)
   1264             keyarr, indexer, new_indexer = ax._reindex_non_unique(keyarr)
   1265 
-> 1266         self._validate_read_indexer(keyarr, indexer, axis, raise_missing=raise_missing)
   1267         return keyarr, indexer
   1268 

c:\python38\lib\site-packages\pandas\core\indexing.py in _validate_read_indexer(self, key, indexer, axis, raise_missing)
   1319 
   1320             with option_context("display.max_seq_items", 10, "display.width", 80):
-> 1321                 raise KeyError(
   1322                     "Passing list-likes to .loc or [] with any missing labels "
   1323                     "is no longer supported. "

KeyError: "Passing list-likes to .loc or [] with any missing labels is no longer supported. The following labels were missing: Int64Index([70050, 70048, 70051, 70051, 70049], dtype='int64', name='bus_id'). See https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#deprecate-loc-reindex-listlike"

This commit fixes that bug.

What the code is doing

Inside the if len(bus_fix_index_all) > 0: block, we are attempting to fix any buses that are not in the bus-to-region mapping dataframe bus_reg, or whose locations don't match what's in the previous mapping file (e.g. in case we changed the locations of some buses since the last time we generated the cached bus-to-region mappings).

However, in the current code, we are only successfully updating bus-to-region mappings for buses whose locations don't match, but not adding new mappings for new buses. The first commit fixes this issue, by separating the new mappings into corrections (which are modified in-place) and additions (which are appended). The second commit refactors the code slightly for clarity. The third commit makes use of the index-name-preserving append method (from #450). All commits will be squashed down before merging.

Testing

After the first commit, tested successfully on Scenario 3828. For the second and third commit, I manually tested to make sure that the values returned remained the same.

Time estimate

15 minutes. The functional change is very simple, but if you want to trace through the variable renaming and refactoring, you have to take the time to decipher the current code, where the variable naming is not especially intuitive.

[BainanXia]

Make sense.