Calling calculate_congestion_surplus on a partial-year Scenario causes an error

[jithesh82]

🪲

Bug summary

I ran the example scenario demo_texas.py from the website. Calculated the congestion_surplus following the code snippet on the website. Its give the error shown below.

Code for reproduction

    from powersimdata import Scenario

    from postreise.analyze.transmission.congestion import calculate_congestion_surplus

    scenario = Scenario(1)
    congestion_surplus = calculate_congestion_surplus(scenario)

Actual outcome

--> Loading LMP
--> Loading PG
--> Loading demand
-------------------------------------------------------------------
ValueError                        Traceback (most recent call last)
<ipython-input-8-e75c6e90fd5f> in <module>
----> 1 cs = calculate_congestion_surplus(s)

~/breakthrough_energy/PostREISE/postreise/analyze/transmission/congestion.py in calculate_congestion_surplus(scenario)                
     26     bus_pg = summarize_plant_to_bus(pg, grid, all_buses=True)                                                                 
     27 
---> 28     congestion_surplus = (lmp.to_numpy() * (bus_demand - bus_pg)).sum(axis=1)
     29     # Remove any negative values caused by barrier method imprecision                                                         
     30     congestion_surplus = np.clip(congestion_surplus, a_min=0, a_max=None)                                                     

~/.env/lib/python3.9/site-packages/pandas/core/generic.py in __array_ufunc__(self, ufunc, method, *inputs, **kwargs)                  
   2111         self, ufunc: np.ufunc, method: str, *inputs: Any, **kwargs: Any                                                       
   2112     ):
-> 2113         return arraylike.array_ufunc(self, ufunc, method, *inputs, **kwargs)
   2114 
   2115     # ----------------------------------------------------------------------                                                  

~/.env/lib/python3.9/site-packages/pandas/core/arraylike.py in array_ufunc(self, ufunc, method, *inputs, **kwargs)                    
    261 
    262     # for binary ops, use our custom dunder methods
--> 263     result = maybe_dispatch_ufunc_to_dunder_op(self, ufunc, method, *inputs, **kwargs)
    264     if result is not NotImplemented:
    265         return result

~/.env/lib/python3.9/site-packages/pandas/_libs/ops_dispatch.pyx in pandas._libs.ops_dispatch.maybe_dispatch_ufunc_to_dunder_op()

~/.env/lib/python3.9/site-packages/pandas/core/ops/common.py in new_method(self, other)                                               
     68         other = item_from_zerodim(other)
     69 
---> 70         return method(self, other)
     71 
     72     return new_method

~/.env/lib/python3.9/site-packages/pandas/core/arraylike.py in __rsub__(self, other)                                                  
    110     @unpack_zerodim_and_defer("__rsub__")
    111     def __rsub__(self, other):
--> 112         return self._arith_method(other, roperator.rsub)
    113 
    114     @unpack_zerodim_and_defer("__mul__")

~/.env/lib/python3.9/site-packages/pandas/core/frame.py in _arith_method(self, other, op)                                             
   6944         other = ops.maybe_prepare_scalar_for_op(other, (self.shape[axis],))
   6945 
-> 6946         self, other = ops.align_method_FRAME(self, other, axis, flex=True, level=None)
   6947 
   6948         new_data = self._dispatch_frame_op(other, op, axis=axis)

~/.env/lib/python3.9/site-packages/pandas/core/ops/__init__.py in align_method_FRAME(left, right, axis, flex, level)                  
    262 
    263             else:
--> 264                 raise ValueError(
    265                     "Unable to coerce to DataFrame, shape "
    266                     f"must be {left.shape}: given {right.shape}"                                                              

ValueError: Unable to coerce to DataFrame, shape must be (2, 10024): given (8784, 10024)

Analysis

The error is thrown from calculate_congestion_surplus from the line:

   congestion_surplus = (lmp.to_numpy() * (bus_demand - bus_pg)).sum(axis=1)

Here the bus_demand has different shape than bus_pg. The reason is get_bus_demand returns demand for the whole time range, whereas lmp, pg, bus_pg all have the input simulation range. I can make a small modification in calculate_congestion_surplus to overcome this issue:

    bus_demand = get_bus_demand(scenario.info, grid).to_numpy()
    ## hack #######
    # bus demand gives output for whole range of time
    # but pg, lmp, bus_pg is calculated for input range of
    # time hence (bus_demand - bus_pg) throws error
    index_min = pg.index.get_loc(pg.index[0])
    index_max = pg.index.get_loc(pg.index[-1])
    bus_demand = bus_demand[index_min:index_max+1]
    ## hack end ###

Environment

• Operating system: Linux 5.15.0-kali2-amd64 Create proper timer in extract #1 SMP Debian 5.15.5-2kali2 (2021-12-22) x86_64 GNU/Linux
• PostREISE : Latest version
• Python version: 3.9.9
• IPython version (if applicable): 7.31.0

Comments

Please have a look at it. Thanks a lot for your kind help, support and work.
Best regards,
-Jithesh

[danielolsen]

Hi @jithesh82, thanks for bringing this to our attention. This issue has been fixed via changes to both PostREISE and PowerSimData (where the get_bus_demand function was defined). If you update PowerSimData to at least v0.5.0 and PostREISE to at least v0.5.1 (or use the latest version of each), you should no longer see this issue.

[jithesh82]

Hello @danielolsen, thanks a lot all of you, for your prompt response and work.