optimisation commands not updating data #15

kcoffau · 2022-08-29T06:41:40Z

Around 26th August, the optimisation process stopped running on my HAOS install.

I run 2 calls, 5.47am Day Ahead, and regularly throughout day the perfect with external solar, price and battery state being sent. its worked fine for 6 months.

when I goto webUI:5000 I run the commands and observe the charts do not update.

I have uninstalled, and reinstalled, and same data set is maintained.

I think the DB is correct in EMHASS. How can I clear this?

Today is 28 August, this shows 26 aug:

davidusb-geek · 2022-08-29T08:44:22Z

Hi. There is maybe another open issue that might be related to this UI not being updated, #16

So I will look at this more in deep.

Could you please confirm that the optimization is not working, for example the logs on the add-on are throwing some error messages? If you call the publish_data action there are any problems with that? Your laods are being controlled correctly?

I just want to identify where is the problem, if it is in the real optimization or just on the UI web page not being updated

kcoffau · 2022-08-29T13:59:30Z

Hi David, my publish is also not working.

Log is showing some errors:
Logger: homeassistant.components.shell_command
Source: /usr/src/homeassistant/homeassistant/components/shell_command/init.py:115
Integration: Shell Command (documentation, issues)
First occurred: 4:25:00 PM (4 occurrences)
Last logged: 4:30:00 PM

Error running command: curl -i -H "Content-Type: application/json" -X POST -d '{"load_cost_forecast":{{( ([states('sensor.amber_general_price')|float(0)] + state_attr('sensor.amber_general_forecast', 'forecasts') |map(attribute='per_kwh')|list)[:48]) }}, "prod_price_forecast":{{( ([states('sensor.amber_feed_in_price')|float(0)] + state_attr('sensor.amber_feed_in_forecast', 'forecasts')|map(attribute='per_kwh')|list)[:48]) }}, "prediction_horizon":{{min(48, (state_attr('sensor.amber_feed_in_forecast', 'forecasts')|map(attribute='per_kwh')|list|length)+1) }},"soc_init":{{states('sensor.victron_battery_state_of_charge')|float(0)/100}},"def_total_hours":[0,0,0,0]}' http://localhost:5000/action/naive-mpc-optim, return code: 7
Error running command: curl -i -H "Content-Type: application/json" -X POST -d '{}' http://localhost:5000/action/publish-data , return code: 7
NoneType: None

EMHASS error log (haven't had time to read this yet, running to work, will try to read later)

self.opt_res = self.perform_optimization(df_input_data, P_PV.values.ravel(), P_load.values.ravel(), File "/usr/local/lib/python3.9/dist-packages/emhass/optimization.py", line 386, in perform_optimization opt_tp["P_grid"] = [P_grid_pos[i].varValue + P_grid_neg[i].varValue for i in set_I] File "/usr/local/lib/python3.9/dist-packages/emhass/optimization.py", line 386, in <listcomp> opt_tp["P_grid"] = [P_grid_pos[i].varValue + P_grid_neg[i].varValue for i in set_I] TypeError: unsupported operand type(s) for +: 'NoneType' and 'NoneType' [2022-08-29 23:50:07,537] INFO in command_line: Setting up needed data [2022-08-29 23:50:07,540] INFO in web_server: >> Publishing data... [2022-08-29 23:50:07,540] INFO in command_line: Publishing data to HASS instance [2022-08-29 23:50:07,547] ERROR in app: Exception on /action/publish-data [POST] Traceback (most recent call last): File "/usr/local/lib/python3.9/dist-packages/pandas/core/arrays/datetimelike.py", line 962, in _validate_frequency raise ValueError ValueError The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 2525, in wsgi_app response = self.full_dispatch_request() File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1822, in full_dispatch_request rv = self.handle_user_exception(e) File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1820, in full_dispatch_request rv = self.dispatch_request() File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1796, in dispatch_request return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) File "/usr/local/lib/python3.9/dist-packages/emhass/web_server.py", line 138, in action_call _ = publish_data(input_data_dict, app.logger) File "/usr/local/lib/python3.9/dist-packages/emhass/command_line.py", line 276, in publish_data opt_res_latest.index.freq = input_data_dict['retrieve_hass_conf']['freq'] File "/usr/local/lib/python3.9/dist-packages/pandas/core/indexes/extension.py", line 78, in fset setattr(self._data, name, value) File "/usr/local/lib/python3.9/dist-packages/pandas/core/arrays/datetimelike.py", line 891, in freq self._validate_frequency(self, value) File "/usr/local/lib/python3.9/dist-packages/pandas/core/arrays/datetimelike.py", line 973, in _validate_frequency raise ValueError( ValueError: Inferred frequency 30T from passed values does not conform to passed frequency 15T [2022-08-29 23:55:00,019] INFO in command_line: Setting up needed data [2022-08-29 23:55:00,023] INFO in retrieve_hass: Retrieve hass get data method initiated... [2022-08-29 23:55:01,093] INFO in forecast: Retrieving weather forecast data using method = scrapper [2022-08-29 23:55:04,341] INFO in forecast: Retrieving data from hass for load forecast using method = naive [2022-08-29 23:55:04,342] INFO in retrieve_hass: Retrieve hass get data method initiated... [2022-08-29 23:55:07,718] INFO in web_server: >> Performing naive MPC optimization... [2022-08-29 23:55:07,718] INFO in command_line: Performing naive MPC optimization [2022-08-29 23:55:07,734] INFO in optimization: Perform an iteration of a naive MPC controller [2022-08-29 23:55:07,841] ERROR in optimization: It was not possible to find a valid solver for Pulp package [2022-08-29 23:55:07,841] INFO in optimization: Status: Not Solved [2022-08-29 23:55:07,841] WARNING in optimization: Cost function cannot be evaluated, probably None [2022-08-29 23:55:07,846] ERROR in app: Exception on /action/naive-mpc-optim [POST] Traceback (most recent call last): File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 2525, in wsgi_app response = self.full_dispatch_request() File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1822, in full_dispatch_request rv = self.handle_user_exception(e) File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1820, in full_dispatch_request rv = self.dispatch_request() File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1796, in dispatch_request return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) File "/usr/local/lib/python3.9/dist-packages/emhass/web_server.py", line 159, in action_call opt_res = naive_mpc_optim(input_data_dict, app.logger) File "/usr/local/lib/python3.9/dist-packages/emhass/command_line.py", line 236, in naive_mpc_optim opt_res_naive_mpc = input_data_dict['opt'].perform_naive_mpc_optim( File "/usr/local/lib/python3.9/dist-packages/emhass/optimization.py", line 543, in perform_naive_mpc_optim self.opt_res = self.perform_optimization(df_input_data, P_PV.values.ravel(), P_load.values.ravel(), File "/usr/local/lib/python3.9/dist-packages/emhass/optimization.py", line 386, in perform_optimization opt_tp["P_grid"] = [P_grid_pos[i].varValue + P_grid_neg[i].varValue for i in set_I] File "/usr/local/lib/python3.9/dist-packages/emhass/optimization.py", line 386, in <listcomp> opt_tp["P_grid"] = [P_grid_pos[i].varValue + P_grid_neg[i].varValue for i in set_I] TypeError: unsupported operand type(s) for +: 'NoneType' and 'NoneType' [2022-08-29 23:55:07,866] INFO in command_line: Setting up needed data [2022-08-29 23:55:07,869] INFO in web_server: >> Publishing data... [2022-08-29 23:55:07,869] INFO in command_line: Publishing data to HASS instance [2022-08-29 23:55:07,877] ERROR in app: Exception on /action/publish-data [POST] Traceback (most recent call last): File "/usr/local/lib/python3.9/dist-packages/pandas/core/arrays/datetimelike.py", line 962, in _validate_frequency raise ValueError ValueError The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 2525, in wsgi_app response = self.full_dispatch_request() File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1822, in full_dispatch_request rv = self.handle_user_exception(e) File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1820, in full_dispatch_request rv = self.dispatch_request() File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1796, in dispatch_request return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) File "/usr/local/lib/python3.9/dist-packages/emhass/web_server.py", line 138, in action_call _ = publish_data(input_data_dict, app.logger) File "/usr/local/lib/python3.9/dist-packages/emhass/command_line.py", line 276, in publish_data opt_res_latest.index.freq = input_data_dict['retrieve_hass_conf']['freq'] File "/usr/local/lib/python3.9/dist-packages/pandas/core/indexes/extension.py", line 78, in fset setattr(self._data, name, value) File "/usr/local/lib/python3.9/dist-packages/pandas/core/arrays/datetimelike.py", line 891, in freq self._validate_frequency(self, value) File "/usr/local/lib/python3.9/dist-packages/pandas/core/arrays/datetimelike.py", line 973, in _validate_frequency raise ValueError( ValueError: Inferred frequency 30T from passed values does not conform to passed frequency 15T

davidusb-geek · 2022-08-29T14:05:04Z

It seems that you are trying to pass 15 min data but you have fixed the time step to 30 min. These should match. Maybe there has been a change on the Amber API?

kcoffau · 2022-08-29T14:15:25Z

adjust the times to match (that could have been me fiddling after it stopped working)

s6-rc: info: service s6rc-oneshot-runner: starting
s6-rc: info: service s6rc-oneshot-runner successfully started
s6-rc: info: service fix-attrs: starting
s6-rc: info: service fix-attrs successfully started
s6-rc: info: service legacy-cont-init: starting
s6-rc: info: service legacy-cont-init successfully started
s6-rc: info: service legacy-services: starting
services-up: info: copying legacy longrun emhass (no readiness notification)
s6-rc: info: service legacy-services successfully started
[2022-08-30 00:11:57,652] INFO in web_server: Launching the emhass webserver at: http://0.0.0.0:5000
[2022-08-30 00:11:57,652] INFO in web_server: Home Assistant data fetch will be performed using url: http://supervisor/core/api
[2022-08-30 00:11:57,652] INFO in web_server: The base path is: /usr/src
[2022-08-30 00:11:57,654] INFO in web_server: Using core emhass version: 0.3.18
[2022-08-30 00:12:34,135] INFO in web_server: EMHASS server online, serving index.html...
[2022-08-30 00:12:42,685] INFO in command_line: Setting up needed data
[2022-08-30 00:12:42,749] INFO in forecast: Retrieving weather forecast data using method = scrapper
[2022-08-30 00:12:44,975] INFO in forecast: Retrieving data from hass for load forecast using method = naive
[2022-08-30 00:12:44,976] INFO in retrieve_hass: Retrieve hass get data method initiated...
[2022-08-30 00:12:47,888] INFO in web_server: EMHASS server online, serving index.html...
[2022-08-30 00:12:48,408] INFO in web_server: >> Performing dayahead optimization...
[2022-08-30 00:12:48,408] INFO in command_line: Performing day-ahead forecast optimization
[2022-08-30 00:12:48,413] INFO in optimization: Perform optimization for the day-ahead
[2022-08-30 00:12:48,526] ERROR in optimization: It was not possible to find a valid solver for Pulp package
[2022-08-30 00:12:48,527] INFO in optimization: Status: Not Solved
[2022-08-30 00:12:48,527] WARNING in optimization: Cost function cannot be evaluated, probably None
[2022-08-30 00:12:48,532] ERROR in app: Exception on /action/dayahead-optim [POST]
Traceback (most recent call last):
File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 2525, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1822, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1820, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1796, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
File "/usr/local/lib/python3.9/dist-packages/emhass/web_server.py", line 151, in action_call
opt_res = dayahead_forecast_optim(input_data_dict, app.logger)
File "/usr/local/lib/python3.9/dist-packages/emhass/command_line.py", line 195, in dayahead_forecast_optim
opt_res_dayahead = input_data_dict['opt'].perform_dayahead_forecast_optim(
File "/usr/local/lib/python3.9/dist-packages/emhass/optimization.py", line 495, in perform_dayahead_forecast_optim
self.opt_res = self.perform_optimization(df_input_data, P_PV.values.ravel(),
File "/usr/local/lib/python3.9/dist-packages/emhass/optimization.py", line 386, in perform_optimization
opt_tp["P_grid"] = [P_grid_pos[i].varValue + P_grid_neg[i].varValue for i in set_I]
File "/usr/local/lib/python3.9/dist-packages/emhass/optimization.py", line 386, in
opt_tp["P_grid"] = [P_grid_pos[i].varValue + P_grid_neg[i].varValue for i in set_I]
TypeError: unsupported operand type(s) for +: 'NoneType' and 'NoneType'
[2022-08-30 00:12:54,790] INFO in command_line: Setting up needed data
[2022-08-30 00:12:54,793] INFO in forecast: Retrieving weather forecast data using method = scrapper
[2022-08-30 00:12:57,002] INFO in forecast: Retrieving data from hass for load forecast using method = naive
[2022-08-30 00:12:57,004] INFO in retrieve_hass: Retrieve hass get data method initiated...
[2022-08-30 00:13:00,295] INFO in web_server: >> Performing dayahead optimization...
[2022-08-30 00:13:00,295] INFO in command_line: Performing day-ahead forecast optimization
[2022-08-30 00:13:00,301] INFO in optimization: Perform optimization for the day-ahead
[2022-08-30 00:13:00,410] ERROR in optimization: It was not possible to find a valid solver for Pulp package
[2022-08-30 00:13:00,410] INFO in optimization: Status: Not Solved
[2022-08-30 00:13:00,411] WARNING in optimization: Cost function cannot be evaluated, probably None
[2022-08-30 00:13:00,415] ERROR in app: Exception on /action/dayahead-optim [POST]
Traceback (most recent call last):
File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 2525, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1822, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1820, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1796, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
File "/usr/local/lib/python3.9/dist-packages/emhass/web_server.py", line 151, in action_call
opt_res = dayahead_forecast_optim(input_data_dict, app.logger)
File "/usr/local/lib/python3.9/dist-packages/emhass/command_line.py", line 195, in dayahead_forecast_optim
opt_res_dayahead = input_data_dict['opt'].perform_dayahead_forecast_optim(
File "/usr/local/lib/python3.9/dist-packages/emhass/optimization.py", line 495, in perform_dayahead_forecast_optim
self.opt_res = self.perform_optimization(df_input_data, P_PV.values.ravel(),
File "/usr/local/lib/python3.9/dist-packages/emhass/optimization.py", line 386, in perform_optimization
opt_tp["P_grid"] = [P_grid_pos[i].varValue + P_grid_neg[i].varValue for i in set_I]
File "/usr/local/lib/python3.9/dist-packages/emhass/optimization.py", line 386, in
opt_tp["P_grid"] = [P_grid_pos[i].varValue + P_grid_neg[i].varValue for i in set_I]
TypeError: unsupported operand type(s) for +: 'NoneType' and 'NoneType'
[2022-08-30 00:13:03,725] INFO in web_server: EMHASS server online, serving index.html...

davidusb-geek · 2022-08-29T14:38:34Z

Ok, you need to check the input data. This is what happens when there are invalid values on the input data, for example missing values and nan's.
Check with Home Assistant templates that you are sending correct values to the EMHASS module.

purcell-lab · 2022-08-29T19:31:47Z

Maybe there has been a change on the Amber API?

I am not having an issue with the Amber API, it is still passing 30 minute values.

"load_cost_forecast":{{(([states('sensor.amber_general_price')|float] + 
                         state_attr('sensor.amber_general_forecast', 'forecasts') |map(attribute='per_kwh')|list)[:48])}}
"prod_price_forecast":{{([states('sensor.amber_feed_in_price')|float] + 
                         state_attr('sensor.amber_feed_in_forecast', 'forecasts')|map(attribute='per_kwh')|list)[:48]}}

"load_cost_forecast":[0.27, 0.26, 0.23, 0.26, 0.27, 0.27, 0.24, 0.23, 0.27, 0.41, 0.26, 0.19, 0.27, 0.27, 0.26, 0.26, 0.19, 0.26, 0.27, 0.41, 0.53, 0.41, 0.2, 0.2, 0.25, 0.26, 0.28, 0.28, 0.28, 0.28, 0.35, 0.35, 0.37, 0.28, 0.36, 0.34, 0.35, 0.31, 0.31, 0.31, 0.27, 0.26, 0.26, 0.26, 0.22, 0.22]
"prod_price_forecast":[0.17, 0.16, 0.13, 0.16, 0.17, 0.17, 0.14, 0.14, 0.17, 0.3, 0.16, 0.1, 0.17, 0.17, 0.16, 0.17, 0.1, 0.16, 0.17, 0.3, 0.41, 0.3, 0.11, 0.11, 0.15, 0.16, 0.18, 0.18, 0.18, 0.18, 0.25, 0.25, 0.26, 0.18, 0.25, 0.23, 0.24, 0.2, 0.21, 0.2, 0.17, 0.16, 0.16, 0.16, 0.12, 0.12]

Runnng @kcoffau curl command here works for me: (with modified soc_init as I don't have victron sensor.

curl -i -H "Content-Type: application/json" -X POST -d '{"load_cost_forecast":{{( ([states('sensor.amber_general_price')|float(0)] + state_attr('sensor.amber_general_forecast', 'forecasts') |map(attribute='per_kwh')|list)[:48]) }}, "prod_price_forecast":{{( ([states('sensor.amber_feed_in_price')|float(0)] + state_attr('sensor.amber_feed_in_forecast', 'forecasts')|map(attribute='per_kwh')|list)[:48]) }}, "prediction_horizon":{{min(48, (state_attr('sensor.amber_feed_in_forecast', 'forecasts')|map(attribute='per_kwh')|list|length)+1) }},"soc_init":{{states('sensor.victron_battery_state_of_charge')|float(0)/100}},"def_total_hours":[0,0,0,0]}' http://localhost:5000/action/naive-mpc-optim

What does the expansion present in developer tools/ template:

$ curl -i -H "Content-Type: application/json" -X POST -d '{"load_cost_forecast":[0.27, 0.26, 0.23, 0.26, 0.27, 0.27, 0.24, 0.23, 0.27, 0.41, 0.26, 0.19, 0.27, 0.27, 0.26, 0.26, 0.19, 0.26, 0.27, 0.41, 0.53, 0.41, 0.2, 0.2, 0.25, 0.26, 0.28, 0.28, 0.28, 0.28, 0.35, 0.35, 0.37, 0.28, 0.36, 0.34, 0.35, 0.31, 0.31, 0.31, 0.27, 0.26, 0.26, 0.26, 0.22, 0.22], "prod_price_forecast":[0.17, 0.16, 0.13, 0.16, 0.17, 0.17, 0.14, 0.14, 0.17, 0.3, 0.16, 0.1, 0.17, 0.17, 0.16, 0.17, 0.1, 0.16, 0.17, 0.3, 0.41, 0.3, 0.11, 0.11, 0.15, 0.16, 0.18, 0.18, 0.18, 0.18, 0.25, 0.25, 0.26, 0.18, 0.25, 0.23, 0.24, 0.2, 0.21, 0.2, 0.17, 0.16, 0.16, 0.16, 0.12, 0.12], "prediction_horizon":46,"soc_init":0.50,"def_total_hours":
[0,0,0,0]}' http://localhost:5000/action/naive-mpc-optim
HTTP/1.1 201 CREATED
Content-Length: 46
Content-Type: text/html; charset=utf-8
Date: Mon, 29 Aug 2022 19:27:14 GMT
Server: waitress

EMHASS >> Action naive-mpc-optim executed...

With correct/ updated optimisation results:

davidusb-geek · 2022-08-29T21:05:34Z

Thanks Mark. I also tested on my side and everything is working fine. In any case I haven't made any notable changes that would've broke anything.
The problem is somewhere else.
@kcoffau are there any changes that you made to your install just before the add-on stopped working correctly?

kcoffau · 2022-08-29T23:12:37Z

I did make some changes a few weeks ago to the MPC script, removing the target charge amount from memory. I am working on the other side of the world atm, so i`ll come back in 12 or so hours with a response, and ill revert to my old MPC command. i am running latest HAos and EMHASS fyi. Just wanted to come back to you both, as Im not ignoring your ideas above. Thank you.

…

________________________________ From: David ***@***.***> Sent: Tuesday, 30 August 2022 07:05 To: davidusb-geek/emhass ***@***.***> Cc: Kelvin Coffey ***@***.***>; Mention ***@***.***> Subject: Re: [davidusb-geek/emhass] optimisation commands not updating data (Issue #15) Thanks Mark. I also tested on my side and everything is working fine. In any case I haven't made any notable changes that would've broke anything. The problem is somewhere else. @kcoffau<https://aus01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fkcoffau&data=05%7C01%7Ckelvin%40coffey.net.au%7C2fc6efd33b79487e0e7c08da8a023ea8%7C61cfce740fac45528d20ebc66386c976%7C0%7C0%7C637974039502802562%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=MFhrakGeu0jQdZw2TgbpkjEP%2FzMWvRvfdtLufG21eR8%3D&reserved=0> are there any changes that you made to your install just before the add-on stopped working correctly? — Reply to this email directly, view it on GitHub<https://aus01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fdavidusb-geek%2Femhass%2Fissues%2F15%23issuecomment-1230860395&data=05%7C01%7Ckelvin%40coffey.net.au%7C2fc6efd33b79487e0e7c08da8a023ea8%7C61cfce740fac45528d20ebc66386c976%7C0%7C0%7C637974039502802562%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=PIcc8nxUD2xsFCG4vYWfe7P9deEBPOVAdIyDvk1629c%3D&reserved=0>, or unsubscribe<https://aus01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAIN7JJRMLS44R5555VKQRQTV3UQ2RANCNFSM574SEHKA&data=05%7C01%7Ckelvin%40coffey.net.au%7C2fc6efd33b79487e0e7c08da8a023ea8%7C61cfce740fac45528d20ebc66386c976%7C0%7C0%7C637974039502802562%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=CmZjzy9zwNLqpyRTYtETdsPYINY4P6QjCmgZC9Uw1z4%3D&reserved=0>. You are receiving this because you were mentioned.Message ID: ***@***.***>

kcoffau · 2022-08-31T06:04:39Z

OK, interesting.
I rolled back to a backup I knew was good, plus another week (13th august), same issue.
On current VM, I cleared out entity values and published optimisation results and it loaded in 26 Apr. So that works.

I cant run optimisation in the webUI, same error, so its not the curl script, but I have also rolled back my curl script to 1 month ago as well for safe measure.

I am dumbfounded that I am running an old VM that was known good and its not good.

MPC Command
curl -i -H "Content-Type: application/json" -X POST -d '{"load_cost_forecast":{{(
([states('sensor.amber_general_price')|float(0)] +
state_attr('sensor.amber_general_forecast', 'forecasts') |map(attribute='per_kwh')|list)[:48])
}}, "prod_price_forecast":{{(
([states('sensor.amber_feed_in_price')|float(0)] +
state_attr('sensor.amber_feed_in_forecast', 'forecasts')|map(attribute='per_kwh')|list)[:48])
}}, "prediction_horizon":{{min(48,
(state_attr('sensor.amber_feed_in_forecast', 'forecasts')|map(attribute='per_kwh')|list|length)+1)
}},"soc_init":{{states('sensor.victron_battery_state_of_charge')|float(0)/100}},"soc_final":0.2,"def_total_hours":[0,0,0,0]}' http://localhost:5000/action/naive-mpc-optim

MPC developer expanded
curl -i -H "Content-Type: application/json" -X POST -d '{"load_cost_forecast":[0.29, 0.2, 0.25, 0.32, 0.43, 0.5, 0.5, 0.47, 0.36, 0.41, 0.4, 0.36, 0.33, 0.33, 0.32, 0.33, 0.33, 0.33, 0.33, 0.33, 0.32, 0.32, 0.31, 0.3, 0.3, 0.33, 0.33, 0.33, 0.32, 0.33, 0.43, 0.41, 0.33, 0.34, 0.3, 0.3, 0.31, 0.31, 0.23, 0.17, 0.17, 0.17, 0.17, 0.18, 0.22, 0.26, 0.28, 0.33], "prod_price_forecast":[0.14, 0.05, 0.1, 0.17, 0.27, 0.32, 0.32, 0.3, 0.2, 0.25, 0.24, 0.2, 0.18, 0.17, 0.17, 0.17, 0.17, 0.18, 0.17, 0.18, 0.17, 0.17, 0.15, 0.14, 0.14, 0.17, 0.17, 0.17, 0.16, 0.17, 0.26, 0.25, 0.18, 0.18, 0.15, 0.15, 0.16, 0.15, 0.09, 0.03, 0.02, 0.02, 0.03, 0.04, 0.07, 0.11, 0.13, 0.17], "prediction_horizon":48,"soc_init":0.142,"soc_final":0.2,"def_total_hours":[0,0,0,0]}' http://localhost:5000/action/naive-mpc-optim

Day ahead script
curl -i -H 'Content-Type: application/json' -X POST -d '{"prod_price_forecast":{{(
state_attr('sensor.amber_feed_in_forecast', 'forecasts')|map(attribute='per_kwh')|list)
}},"load_cost_forecast":{{(
state_attr('sensor.amber_general_forecast', 'forecasts') |map(attribute='per_kwh')|list)
}}}' http://localhost:5000/action/dayahead-optim"

day ahead developer expanded
curl -i -H 'Content-Type: application/json' -X POST -d '{"prod_price_forecast":[0.05, 0.1, 0.17, 0.27, 0.32, 0.32, 0.3, 0.2, 0.25, 0.24, 0.2, 0.18, 0.17, 0.17, 0.17, 0.17, 0.18, 0.17, 0.18, 0.17, 0.17, 0.15, 0.14, 0.14, 0.17, 0.17, 0.17, 0.16, 0.17, 0.26, 0.25, 0.18, 0.18, 0.15, 0.15, 0.16, 0.15, 0.09, 0.03, 0.02, 0.02, 0.03, 0.04, 0.07, 0.11, 0.13, 0.17, 0.31],"load_cost_forecast":[0.2, 0.25, 0.32, 0.43, 0.5, 0.5, 0.47, 0.36, 0.41, 0.4, 0.36, 0.33, 0.33, 0.32, 0.33, 0.33, 0.33, 0.33, 0.33, 0.32, 0.32, 0.31, 0.3, 0.3, 0.33, 0.33, 0.33, 0.32, 0.33, 0.43, 0.41, 0.33, 0.34, 0.3, 0.3, 0.31, 0.31, 0.23, 0.17, 0.17, 0.17, 0.17, 0.18, 0.22, 0.26, 0.28, 0.33, 0.48]}' http://localhost:5000/action/dayahead-optim"

Current error:
[2022-08-31 16:00:15,635] INFO in web_server: >> Performing naive MPC optimization...
[2022-08-31 16:00:15,635] INFO in command_line: Performing naive MPC optimization
[2022-08-31 16:00:15,662] INFO in optimization: Perform an iteration of a naive MPC controller
[2022-08-31 16:00:15,777] ERROR in optimization: It was not possible to find a valid solver for Pulp package
[2022-08-31 16:00:15,778] INFO in optimization: Status: Not Solved
[2022-08-31 16:00:15,778] WARNING in optimization: Cost function cannot be evaluated, probably None
[2022-08-31 16:00:15,787] ERROR in app: Exception on /action/naive-mpc-optim [POST]
Traceback (most recent call last):
File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 2525, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1822, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1820, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1796, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
File "/usr/local/lib/python3.9/dist-packages/emhass/web_server.py", line 159, in action_call
opt_res = naive_mpc_optim(input_data_dict, app.logger)
File "/usr/local/lib/python3.9/dist-packages/emhass/command_line.py", line 236, in naive_mpc_optim
opt_res_naive_mpc = input_data_dict['opt'].perform_naive_mpc_optim(
File "/usr/local/lib/python3.9/dist-packages/emhass/optimization.py", line 543, in perform_naive_mpc_optim
self.opt_res = self.perform_optimization(df_input_data, P_PV.values.ravel(), P_load.values.ravel(),
File "/usr/local/lib/python3.9/dist-packages/emhass/optimization.py", line 386, in perform_optimization
opt_tp["P_grid"] = [P_grid_pos[i].varValue + P_grid_neg[i].varValue for i in set_I]
File "/usr/local/lib/python3.9/dist-packages/emhass/optimization.py", line 386, in
opt_tp["P_grid"] = [P_grid_pos[i].varValue + P_grid_neg[i].varValue for i in set_I]
TypeError: unsupported operand type(s) for +: 'NoneType' and 'NoneType'

kcoffau · 2022-08-31T06:19:24Z

changed my lp_solver from 'PULP_CBC_CMD to COIN_CMD'. and fixes it. might be something wrong with puLP?

davidusb-geek · 2022-08-31T07:12:31Z

Oh yes I did not saw those log lines where it says that it couldn't find any solver!
There is nothing wrong with PuLP, we just have to give the correct name of the solver installed on your system with the correct path to the solver executable

davidusb-geek · 2022-09-02T20:32:39Z

Hi @kcoffau , so you finally solved this issue?

davidusb-geek closed this as completed Sep 6, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

optimisation commands not updating data #15

optimisation commands not updating data #15

kcoffau commented Aug 29, 2022

davidusb-geek commented Aug 29, 2022

kcoffau commented Aug 29, 2022

davidusb-geek commented Aug 29, 2022

kcoffau commented Aug 29, 2022

davidusb-geek commented Aug 29, 2022

purcell-lab commented Aug 29, 2022

davidusb-geek commented Aug 29, 2022

kcoffau commented Aug 29, 2022 via email

kcoffau commented Aug 31, 2022

kcoffau commented Aug 31, 2022

davidusb-geek commented Aug 31, 2022

davidusb-geek commented Sep 2, 2022

optimisation commands not updating data #15

optimisation commands not updating data #15

Comments

kcoffau commented Aug 29, 2022

davidusb-geek commented Aug 29, 2022

kcoffau commented Aug 29, 2022

davidusb-geek commented Aug 29, 2022

kcoffau commented Aug 29, 2022

davidusb-geek commented Aug 29, 2022

purcell-lab commented Aug 29, 2022

davidusb-geek commented Aug 29, 2022

kcoffau commented Aug 29, 2022 via email

kcoffau commented Aug 31, 2022

kcoffau commented Aug 31, 2022

davidusb-geek commented Aug 31, 2022

davidusb-geek commented Sep 2, 2022