Why is the battery not charged and pv surplus energy exported to grid instead?

[daniel309]

Look at the second day, Battery is kept around 50% and not charged, but instead energy is exported to grid.

It looks like this has something to do with the low grid price at that time. It still is higher than the 8.4 cent export price including efficiency so I dont understand why the energy is not stored in the battery.

Is something off with the cost model?

Request

{
  "batteries": [
    {
      "c_max": 4000,
      "c_min": 0,
      "charge_from_grid": true,
      "d_max": 4000,
      "p_a": 0.00008473409,
      "s_capacity": 10000,
      "s_initial": 10000,
      "s_max": 10000,
      "s_min": 500
    }
  ],
  "eta_c": 0.9,
  "eta_d": 0.9,
  "grid": {},
  "strategy": {
    "charging_strategy": "charge_before_export",
    "discharging_strategy": "discharge_before_import"
  },
  "time_series": {
    "dt": [873, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900],
    "ft": [1663.503, 1677.1859, 1639.4203, 1582.6047, 1506.739, 1430.8734, 1355.0078, 1269.3063, 1173.7688, 1078.2312, 982.6937, 873, 749.15, 625.3, 501.45, 395.40625, 307.16876, 218.93124, 130.69376, 76.435936, 56.157814, 35.87969, 15.6015625, 4.7796874, 3.4140625, 2.0484376, 0.6828125, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1.1046875, 3.3140626, 5.5234375, 7.7328124, 15.3375, 28.3375, 41.3375, 54.3375, 73.25938, 98.10313, 122.94688, 147.79062, 185.7172, 236.72656, 287.73593, 338.7453, 379.6203, 410.36093, 441.10156, 471.8422, 487.81406, 489.01718, 490.2203, 491.42343, 495.61874, 502.80624, 509.99374, 517.1813, 521.1766, 521.9797, 522.78284, 523.58594, 514.9313, 496.81876, 478.70624, 460.59375, 447.17032, 438.43594, 429.70157, 420.9672, 404.35782, 379.87344, 355.38907, 330.9047, 301.05157, 265.82968, 230.60782, 195.38594, 162.10469, 130.76407, 99.42344, 68.08281, 46.734375, 35.378124, 24.021875, 12.665625, 6.1140623, 4.3671875, 2.6203125, 0.8734375, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
    "gt": [99.7133, 83.95446, 141.42757, 125.38818, 96.393074, 56.777428, 70.6182, 73.94749, 135.98067, 249.56635, 259.76328, 136.22935, 54.260296, 44.621143, 40.059586, 52.12447, 50.48521, 72.41135, 51.86838, 112.78008, 126.30988, 89.188286, 138.34378, 109.52149, 110.41173, 86.99745, 92.37061, 73.96064, 79.158905, 104.04319, 136.59142, 68.794235, 75.81226, 71.388306, 76.304146, 73.00222, 64.53891, 55.902416, 66.63862, 64.380135, 70.62197, 74.201775, 76.23938, 70.07575, 65.69674, 73.87492, 63.323467, 61.856697, 68.29337, 86.45963, 122.472694, 123.58584, 116.246155, 85.14185, 82.74437, 89.90508, 78.26803, 107.64318, 143.08163, 150.30704, 151.60904, 146.1876, 121.01219, 107.95574, 101.866615, 122.72184, 115.492485, 103.31704, 95.47504, 94.426445, 81.74868, 87.1014, 100.152084, 110.707886, 117.08106, 117.45641, 112.61845, 103.367386, 98.68544, 115.07792, 111.90079, 158.499, 167.64091, 144.2579, 116.48126, 114.15665, 243.46617, 465.34216, 539.8518, 469.90332, 575.4158, 303.372, 108.95757, 122.09269, 195.05943, 226.04369, 135.94156, 111.85231, 179.33702, 159.1252, 138.16441, 111.77759, 110.61815, 115.82248, 183.48067, 278.62875, 305.51334, 186.7293, 96.760185, 84.121124, 74.01805, 131.76721, 124.65194, 130.53633, 128.4309, 202.78029, 195.68478, 157.10617, 191.23392, 151.06264, 154.46793, 115.059006, 135.9233, 99.48542, 104.12529, 131.29823, 155.10916, 88.664635, 99.49745, 89.83448, 99.09031, 96.6492, 80.535835, 68.12334, 88.808556],
    "p_E": [0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084, 0.000084],
    "p_N": [0.0000951, 0.0001007, 0.0001021, 0.000115, 0.0001301, 0.0001424, 0.0001543, 0.0001543, 0.0001721, 0.0001801, 0.0001781, 0.0001851, 0.0001864, 0.0001893, 0.0001957, 0.0001928, 0.0002341, 0.0002618, 0.0002791, 0.000271, 0.0002796, 0.0002816, 0.000279, 0.0002925, 0.0002734, 0.0002792, 0.0002708, 0.0002709, 0.0002696, 0.0002649, 0.0002649, 0.0002732, 0.0002745, 0.0002739, 0.0002649, 0.0002733, 0.0002733, 0.0002722, 0.0002677, 0.0002733, 0.0002745, 0.0002794, 0.0002779, 0.0002817, 0.0002772, 0.0002806, 0.0002848, 0.0002821, 0.0002853, 0.000286, 0.0002892, 0.000288, 0.0002918, 0.000291, 0.0002948, 0.0002986, 0.0003007, 0.0003044, 0.0003088, 0.0003037, 0.0003082, 0.0003081, 0.0003065, 0.0003109, 0.0003103, 0.0003055, 0.000303, 0.0003151, 0.0003041, 0.0002961, 0.0002934, 0.0002947, 0.0002867, 0.0002818, 0.000259, 0.0002657, 0.0002578, 0.0002467, 0.0002435, 0.0002491, 0.0002196, 0.0002198, 0.0002152, 0.0002212, 0.0002181, 0.0002147, 0.0002125, 0.0002156, 0.0002093, 0.0002078, 0.0001984, 0.0001966, 0.0001923, 0.0001902, 0.0001901, 0.00019, 0.0001939, 0.0001996, 0.0001984, 0.0001971, 0.0002019, 0.0002105, 0.0002261, 0.0002293, 0.0002644, 0.0002701, 0.0003175, 0.0002664, 0.0003074, 0.000348, 0.0003625, 0.0003363, 0.0003458, 0.0003567, 0.0003654, 0.0003546, 0.0003534, 0.000364, 0.0003768, 0.0003694, 0.0003671, 0.0003649, 0.0003572, 0.0003577, 0.000351, 0.0003533, 0.0003487, 0.0003561, 0.0003503, 0.0003431, 0.0003321, 0.0003507, 0.000332, 0.0003321, 0.0003211]
  }
}

Response

{
  "batteries": [
    {
      "charging_power": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
      "discharging_power": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 36.344143, 70.15207, 53.308598, 122.74222, 104.7418, 106.99767, 84.94901, 91.6878, 73.96064, 79.158905, 104.04319, 136.59142, 68.794235, 75.81226, 71.388306, 76.304146, 73.00222, 64.53891, 55.902416, 66.63862, 64.380135, 70.62197, 74.201775, 76.23938, 70.07575, 65.69674, 73.87492, 63.323467, 61.856697, 68.29337, 86.45963, 122.47269, 123.58584, 116.24616, 85.14185, 82.74437, 89.90508, 78.26803, 107.64318, 143.08163, 150.30704, 151.60904, 146.1876, 121.01219, 106.85105, 98.55255, 117.1984, 107.75967, 87.97954, 67.13754, 53.088943, 27.41118, 13.84202, 2.048954, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 37.04556, 0, 58.2345, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 31.11289, 60.34809, 156.04591, 160.30666, 133.08429, 178.5683, 144.94858, 150.10074, 112.43869, 135.04987, 99.48542, 104.12529, 131.29823, 155.10916, 88.664635, 99.49745, 89.83448, 99.09031, 96.6492, 80.535835, 68.12334, 88.808556],
      "state_of_charge": [10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 10000, 9959.617, 9881.671, 9822.439, 9686.059, 9569.679, 9450.793, 9356.405, 9254.529, 9172.352, 9084.396, 8968.793, 8817.025, 8740.587, 8656.352, 8577.031, 8492.248, 8411.135, 8339.425, 8277.312, 8203.269, 8131.735, 8053.266, 7970.8193, 7886.109, 7808.247, 7735.251, 7653.1675, 7582.808, 7514.0786, 7438.197, 7342.131, 7206.05, 7068.7324, 6939.57, 6844.968, 6753.03, 6653.1353, 6566.1704, 6446.567, 6287.5874, 6120.5796, 5952.125, 5789.6943, 5655.237, 5536.513, 5427.0103, 5296.79, 5177.057, 5079.302, 5004.7046, 4945.717, 4915.26, 4899.88, 4897.6035, 4897.6035, 4897.6035, 4897.6035, 4897.6035, 4897.6035, 4897.6035, 4897.6035, 4897.6035, 4897.6035, 4897.6035, 4897.6035, 4897.6035, 4897.6035, 4897.6035, 4897.6035, 4856.4414, 4856.4414, 4791.737, 4791.737, 4791.737, 4791.737, 4791.737, 4791.737, 4791.737, 4791.737, 4791.737, 4791.737, 4791.737, 4791.737, 4791.737, 4791.737, 4791.737, 4791.737, 4791.737, 4791.737, 4791.737, 4791.737, 4791.737, 4791.737, 4791.737, 4757.167, 4690.1133, 4516.729, 4338.6104, 4190.739, 3992.3298, 3831.276, 3664.4973, 3539.5654, 3389.51, 3278.9707, 3163.276, 3017.389, 2845.0454, 2746.529, 2635.9763, 2536.1604, 2426.06, 2318.672, 2229.1877, 2153.495, 2054.8188]
    }
  ],
  "flow_direction": [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
  "grid_export": [1295.0973, 1593.2314, 1497.9927, 1457.2166, 1410.346, 1374.096, 1284.3896, 1195.3588, 1037.7881, 828.66486, 722.9304, 736.7706, 694.8897, 580.67883, 461.3904, 343.28177, 256.68356, 146.5199, 78.82538, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 12.238994, 30.70956, 68.26079, 124.10811, 184.36855, 240.05986, 264.5424, 298.46014, 282.60257, 304.2013, 343.55615, 372.53592, 376.06366, 247.95726, 30.27658, 0, 40.09042, 0, 217.8046, 413.02213, 400.69016, 328.52652, 288.8876, 360.8772, 366.85394, 281.25674, 288.0451, 300.2715, 317.92398, 310.34906, 288.53534, 196.39278, 76.76032, 25.39136, 114.32227, 169.0695, 146.4867, 121.36789, 30.33748, 6.11213, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
  "grid_import": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
  "limit_violations": {},
  "objective_value": 1.4490716,
  "status": "Optimal"
}

[ekkea]

The optimizer does the right thing. It's our old friend (and usual suspect) p_a (price of the battery stored energy at the end of the time horizon). Our current heuristics is to set this slightly below (considering discharge losses) the lowest import price of the time horizon. This works fine with relatively stable prices but not in this case.

Actually p_a should be a prediction about the import price beyond the time horizon: If the import prices will be low, also the value of the battery stored energy will be low - and vice versa.

Possible solutions / mitigations:

1. as p_a, choose the last import price at the end of the horizon (reduced by the discharge losses). This assumes that the last price is a better prediction that the lowest price.
2. reduce the time horizon by say 12 hours and use the last 12 hours of day-ahead-price-data to set p_a. As this reduces the optimization horizon significantly (typically the day-ahead horizon is limiting our optimization horizon, right?), I don't see that this would improve things.
3. Find a way to predict the beyond-day-ahead prices. Even a rough estimate for the following day would help a lot. If you search the web for "day ahead price forecasting", you will see that others had the idea already. Besides commercial APIs there is an open source solution (https://github.com/b3nn0/EpexPredictor ) which looks quite promising.

@daniel309 , @andig , your thoughts?

[daniel309]

Hmm. How about preferring the battery for this "beyond horizon" case?

Battery energy could be valued at export price (8.4c in my case) and efficiency could be added on top, e.g. 10÷ loss means value is 7,56 cent/kWh. This could be used alternatively as p_a i think.

This would make things independent of long horizon epex predictions which add another dependency and complexity imo.

Also, optimizer runs very often, so once new dayahead values are available this should fix itself. It did so for me a few hours later.

[andig]

Current pa:

// end of horizon Wh value
pa := lo.Min(req.TimeSeries.PN) * eta * 0.99

We can choose anything, might be worth to decide on Slack. Anyway, this needs be changed on evcc side. Closing here since the optimizer is doing fine.

[andig]

Battery energy could be valued at export price (8.4c in my case) and efficiency could be added on top, e.g. 10÷ loss means value is 7,56 cent/kWh. This could be used alternatively as p_a i think.

...and why would that be better in the general case?

This would make things independent of long horizon epex predictions which add another dependency and complexity imo.

Afaiu other countries have dynamic feed in tariffs, too.

Also, optimizer runs very often, so once new dayahead values are available this should fix itself. It did so for me a few hours later.

The same is true for the current approach, so that's not an argument.

[daniel309]

Agreed. My point about fixing itself was meant to say it doesnt really matter too much which value was picked for beyond horizon.

[andig]

I don't understand. The whole issue here is about exactly that value?

[ekkea]

it seems we have to understand the problem better. For the particular case, the p_a threshold between charging and not charging on the second day is at about 9.324 ct/kWh which is very close to the export price by efficiency (8.4 ct / 0.9). So far, just an observation ....
Still trying to make sense of that ...

[daniel309]

It could also help to document that prediction quality is getting worse towards the end of the plan and that it will improve as time moves on.

Such issues like this one are probaly hard to solve generally, so docs could help to avoid same discussions repeating.

[andig]

Isn't that obvious? Same as every other prediction/forecast on earth...

[ekkea]

ok, here we go .... here is the explanation why at p_a = 8.473 ct/kWh the optimizer decides to rather export surplus energy and why it stops doing that when raising p_a above p_ex / 0.9 :

• there is a solar surplus E_s that either gets fed to grid for p_ex (8.4 ct/kWh in this case) or charged to the battery with eta_ch (0.9 in this case).
• The energy that would be charge to the battery is E_b = E_s * eta_ch
• otherwise, in the case of exporting we would get a value of V = E_s * p_ex
• so the threshold for p_a becomes p_a = V / E_b = (E_s * p_ex) / (E_s * eta_ch) = p_ex / eta_ch (i.e. 8.4 ct/kWh / 0.9)

As simple as confusing 😕

[ekkea]

so, what we could consider to do is that we add to the existing rule ...

// end of horizon Wh value
pa := lo.Min(req.TimeSeries.PN) * eta * 0.99

... a statement like (pseudo code):

if(pa <= p_ex / eta)
  pa = p_ex / eta * 1.01