Determining the "Power calculation method"

[careyer]

In the CLI output it says: "Using the new method of calculating the power demand" (which overshoots by quite a lot).
Is there a way to use the "standard method"?

I found out, that this is in soyosource_virtual_meter.cpp (L82-L88ff)
Here I can see there is the determination of what method to use.

However both versions do not fit my needs.
I want to implement a "PV-first" functionality in my system and therefore I need the unaltered last MQTT power measurement. This power measurment refelcts the current PV input in Watts and I want to command my inverter to invert exaclty this power so that the PV power gets absorbed right awaybefore charging my batteries.

Thanks for any hints mate!

[syssi]

There are two methods available:

1. DUMB_OEM_BEHAVIOR
2. NEGATIVE_MEASUREMENTS_REQUIRED

You can choose between the methods here:

https://github.com/syssi/esphome-soyosource-gtn-virtual-meter/blob/main/esp8266-example.yaml#L60

I cannot recommend to use the dump OEM behavior because the method is unable to detect overshooting.You should stick to the NEGATIVE_MEASUREMENTS_REQUIRED method instead and add some buffer:

https://github.com/syssi/esphome-soyosource-gtn-virtual-meter/blob/main/esp8266-example.yaml#L65-L67

Another option is smoothing the readings of the smart meter locally by applying additional filters:

https://github.com/syssi/esphome-soyosource-gtn-virtual-meter/blob/main/esp8266-example.yaml#L112-L113

[careyer]

BTW: I felt lucky enough to add a 3rd method called "PASSTHROUGH" (you can observe the changes in my fork: https://github.com/careyer/esphome-soyosource-gtn-virtual-meter -- I am sorry but I had to fork because otherwise the esphome build processs does overwrite my local changes with the status of your online repo while building because it pulls the latest code during buildtime) 🙄

The use-case of this method is the following: I wanted to implement a "Solar/PV-First"-Approach... i.e.: I am measuring what power comes from my solar panels right at the very same moment and I want my SoyoSource Inverter to invert exactly this ammount of power while not charging my batteries and leaving them +-0W charging current.

[syssi]

Feel free to fork this repository (it's how GitHub works). ;-) I would be happy about a contribution / pull request if you want to introduce a third method which fits to your needs.

[careyer]

Thank you! I see you are embodying the open source idea! Kudos!
Sadly I am not very good at coding... I managed to implement my passthough method meanwhile (just to discover that I had fallen short of a missconception) ¯_(ツ)_/¯

However I had ideas on how to improve on your 'NEGATIVE_MEASUREMENTS_REQUIRED' method:

1. From what I can tell you need the negative input values to determine if you are in an overshoot situation (you are inverting more than the actual demand). As long as you are seeing positive values you keep adding them to the previous power demand and hence approaching the actual power demand (you are incremennting towards the power demand). You do that until you see the very first negative value and then you take a step back. Alright!!!! Good approach.... However you can improve that:
   In your situation you will increment as long as you reach an overshoot and try stabilizing arround that point with the chance of hitting more overshoots over time and always stepping back from them.
   --Here goes my idea on how to improve on that: Each time when receiving a new power reading you can just observe "Have I been actively inverting the last time I received a value (i.e. output power >0)?"... if so the current received value needs to be added to the power demand. If not (wasn't inverting on the last power reading) --> Inverting had been stopped due to overshoot and we need to start a new approximation cycle. Bennefit: You achieve pretty much the same buth without the need for negative values being needed/allowed (makes it more versatile). The negative prefix is basically replaced by the observation: Was I inverting before or not?

2. Use a PID controller to approximate the needed power demand without much overshooting (if at all). I believe this might be the most sophisticated method with mimium jitter and less 'approximation cycles'. However coding a PID-controller is way over my head

What are your thoughts?
Best regards!

[syssi]

Let's talk about idea 1 in detail: Could you try to explain / outline your idea using example values?

[careyer]

Hey @syssi,

sorry for answering late: I have been way to busy these days.
Okay- here comes my proposal for another method that does not overshoot and that can also work without nagative values:

Preliminary considerations:
We can determine at any point in time if the inverter is currently active and inverting by taking a look of the currently inverted power (output power >0W). Now lets use that information to approximate our power demand from the lower end without ever overshooting:

Lets take a look at an example:
1.) Given our limiter signals a power demand of 400W
2.) Now start inverting at 90% that power with 400W x 0,9 = 360W
3.) Now upon the next measurement we see that there are 40W of demanded power left -- AND -- we see that our inverter is already inverting (output power 360W>0W). That is: We know we have to add more power to the currently set output power.
4.) Now we add another 90% of the demand that is left, ie: 360W + 40 x 0,9 = 396W
5.) Now upon the next measurement we see that there are 4W of demanded power left -- AND -- we see that our inverter is already inverting (output power 396W>0W). That is: We know we have to add more power to the currently set output power.
6.) Now we add another 90% of the demand that is left, ie: 396W + 4 x 0,9 = 399.6W
7.) Now lets assume our power draw suddenly drops from 400W to only 150W
8.) Out limiter power reading will signal <=0W, i.e. we stop inverting and set the new power output to 0W
9.) On the very next reading the Power limiter value will read 150W and hence we repeat approximating the needed power like before (Back to step 1. but this time with the new value of 150W)

This way we rapidly approach the current power demand with immediatly and effectively stepping back from any overshoot and without the need for negative power limiter values.

[syssi]

I understand your idea. Could you explain why you don't like overshooting and why do you prefer stopping the inverter instead of slowly decrease the limiter signals to find a new "positive sweet spot" (f.e. if the smartweter reports a power consumption of 1W).

[careyer]

Sure! There are two major reasons for this:
a) Every overshoot means loosing energy to the energy grid (for anyone who does not get paid for it)
b) For blackout prevention I have a offgrid-inverter at hand which I can use to simulate the grid for my Grid-Tie inverters. This works well but one has to make sure that in case of sudden powerdemand drops the overshoot is very quickly resolved - otherwise the offgrid-inverter releases its magic smoke. There is no time to slowly approach a new sweetspot. ;-)

[syssi]

Last question before implementing the method: Are you able to measure negative values (overshooting/exporting to the (off)grid) in case of b) or are you looking for a "positive measurements method" because of this reason?

[careyer]

Ohh... yes, I can also measure negative values. Just thought it would be beneficial to have a 3rd method that doesn't rely on this prerequisite. Just to make this whole suite of tools more universal ;-)

[syssi]

Please give it a try:

substitutions:
  name: soyosource-gtn-virtual-meter
  device_description: "Monitor a Soyosource GTN and control the power output on demand both via RS485"
  external_components_source: github://syssi/esphome-soyosource-gtn-virtual-meter@restart-on-crossing-zero-method
  tx_pin: GPIO1
  rx_pin: GPIO3

soyosource_virtual_meter:
  power_id: powermeter
  power_sensor_inactivity_timeout: 20s
  power_demand_calculation: RESTART_ON_CROSSING_ZERO
  min_power_demand: 0
  max_power_demand: 900
  power_demand_divider: 1
  buffer: 10
  operation_mode_id: operation_mode_id
  update_interval: 3s

The implemention isn't exactly your approach but it's pretty similar. The configuration above tries to control the limiter to import 10W from the grid always. If your smartmeter reports a value less than 10W the method tries to reach 10W again. If your smartmeter reports a value <= 0 we stop the inverter and start from scratch.

[careyer]

Hey Sebastian,

thanks! Sorry... I was busy again like crazy. I will try it this weekend and will report back to you!
Thanks in advance!

Best regards
Thomas

[blacklopo]

Hi,
I'm also very concerned of minimising of overshooting. I have question about calculation of power demand - If my PV has at the moment not enough production, script very quickly rises the power 900W. But generally I only need to produce what is detected by clamp.
Is in understable? IMHO we have to compare demand with real production of the invertor.

[syssi]

@blacklopo Do you have a smartmeter which provides negatives measurements of you overshoot / export to the grid?

[blacklopo]

@blacklopo Do you have a smartmeter which provides negatives measurements of you overshoot / export to the grid?

Yes, Shelly 3M. Attaching graph from today. Finally I tried new variant - RESTART_ON_CROSSING_ZERO . But data will be available tomorrow.

[syssi]

Keep me updated. I will try to provide a similar graph for comparison.

[blacklopo]

This is something I don't like to see:

[syssi]

Could you explain what I can see here? I must admit I didn't get the meaning of FVE and DS yet.

[blacklopo]

Now I can see problem with settings - RESTART_ON_CROSSING_ZERO. When negative value is detected, system automatically set Demand power to 0, and iniciate "reset", but for next several seconds the inverter still generates higher power than is expected...

[blacklopo]

• DS: Power from grid. Detected by Shelly 3M
• FVE: "Production" of inverter measured by smart tuya plug.

[kev300]

Hi, I also have some issues with the overshooting of the default method.
My smartmeter is read every second and I update the inverter every 2 seconds (does not like 1s 😉)
Buffer is set to 0 because i want 0W.

See attached screenshot of the smartmeter power measurement. It never settles at 0 but keeps jumping back and forth. My powe consumption before the test runs is at stable 122w, so it should not jump like that.

Any idea besides smoothing the sensor? I want a fast response with less overshooting. 😄

[syssi]

Could you provide your YAML configuration? I would like to know the update interval of the powermeter sensor for sure. It is throttled?

[syssi]

Please try to throttle the powermeter0 sensor step by step. At the moment your parameter set reacts too fast and the inverter cannot respond as fast. This leads to oscillation and the virtual meter is unable to close the feedback loop.

[kev300]

Its just that:

• platform: homeassistant
  id: powermeter0
  name: "smartmeter power consumption"
  entity_id: sensor.stromz_power_power_curr

The sensor itself sends a value as soon as the meter is updating it's measurement, which is about 1s
So not throttled.

I am thinking about changing the calculation to something like
int16_t power_demand = (importing_now - this->last_power_demand_delta_ / 2) + last_power_demand;
to "wait" a little bit for the inverter based on the last demand change.

Throttling the sensor would slowdown the initial reaction on sudden load change. Changing the calculation method would only slow down the following adjustments.

I will try to test both methods and see if my assumption can work. ;-)

[syssi]

If you clone this repository you can make/test all changes locally if you compile the YAML using:

esphome -s external_components_source components run esp8266-example.yaml

This will build the project using the local component directory instead of cloning the repository.

[syssi]

I'm happy about possible improvements. My knowledge of control engineering is very limited.

[ButschFPV]

@kev300 I'm running my inverter with these parameters:

power_demand_calculation: NEGATIVE_MEASUREMENTS_REQUIRED
buffer: 0
update_interval: 1s
#- throttle_average: 15s

I have no oscillation. My smartmeter updates every second. It's working okay but overshoots for 1-2 seconds are normal.
You mentioned in #102 that you have a quiet soyosource. What Version is it 24V/48V?
What error does it show if you send the power demand too frequently (1 second)?

[kev300]

It's 48v 1200w with display; no wifi.
It does not show any errors, but the rs485 logo disappears for 1-2 seconds and it goes to 0w. Then it start working again for 2-3s.

Interesting. I just tested with low power consumption, because my battery is empty and not connected to solar, yet. But mine never stops oscillating. I'm trying a new algorithm at the moment but didn't have enough time yet.

[ButschFPV]

I have hooked up the original RS485 limiter to a logic analyzer and it's transmitting the power demand every 0.5s.
Maybe it's the status request frame that quiet inverter don't like.

Have you bought your inverter with the original limiter?

[syssi]

I would be happy about another experiment to answer these questions:

1. How fast is the Soyosource able to ramp up from 0 to (f.e.) 500W?
2. How long does it take to ramp down (by sending a 0W demand)?
3. How long does it take to ramp down if no demand is transmitted?

[kev300]

What do you mean by "info stuff"? You could simply increase the update interval of the virtual limiter.

soyosource_virtual_meter:
  - id: virtualmeter0
    soyosource_modbus_id: modbus0
    [...]
    update_interval: 500ms

Sorry I meant

Maybe it's the status request frame that quiet inverter don't like.

so the whole information readout, which my inverter does not provide

[syssi]

Yes. If your inverter is a silent one you should use the

https://github.com/syssi/esphome-soyosource-gtn-virtual-meter/blob/main/esp8266-limiter-example.yaml

example instead of

https://github.com/syssi/esphome-soyosource-gtn-virtual-meter/blob/main/esp8266-example.yaml

The soyosource_inverter component which requests the status frame periodically is removed here.

[ButschFPV]

@syssi was about to write the same.
The device_description needs to be updated:

esphome-soyosource-gtn-virtual-meter/esp8266-limiter-example.yaml

Line 3 in 9e91033

device_description: "Monitor a Soyosource GTN and control the power output on demand both via RS485"

Shouldn't "Monitor an control" only be "Control"?

[humpataa]

I have a similar setup running and was curious too about how fast the Soyo is adjusting. I don't have decent measurement equipment, just a Shelly. But I am sure there is quite a bit of latency of one or even two seconds until a new limit takes effect. And it looks like a smooth ramp between highly different values, no sudden impacts. I have disabled reading from the limiter "port" because this makes no sense in my eyes and for many people the inverter is silent anyway. After testing I have found that "hammering" the limiter twice a second is not necessary at all. It works very good when updating once every 3 seconds - and taking into account the inverter's latency in adjusting, 2 seconds seems like a very good setting. Above 4 seconds is certainly to big of delay. Just my two cents. Keep up the good work guys!

[kev300]

Ok, i did a few tests.

1. My inverter works now with 0.5s updates if the information part is removed from the config.

But it's not necessary. ;-)
In my case there is a latency of about 2,5 seconds when the power meter sensor gets to 0W after the inverter demand has been set.
Optimal for me is 1s update for the smart meter and 3s for the inverter.

In addition to that, I can stop the overshoot, if I skip the second demand setting after the initial one.

The fluctuations you see happen, when I turn on or off the lights and it takes a few seconds until it reacts but then its very flat without overshooting.

[kev300]

This seems to work good for me: https://github.com/kev300/esphome-soyosource-gtn-virtual-meter/tree/overshoot-compensation2

But I'm still testing with fast (0.5s) and slow (3s) update interval today and will create a PR if don't find more issues. ;-)

[syssi]

Please keep in mind you can smooth the regulation too if you throttle / throttle_average the input signal:

  - platform: mqtt_subscribe
    id: powermeter_firstfloor
    internal: true
    name: "${inverter0} smartmeter instantaneous power"
    topic: "..."
    accuracy_decimals: 2
    unit_of_measurement: W
    device_class: power
    filters:
      - throttle: 15s

It's not as intelligent as comparing values but it slows things down and let the inverter breathe.

[kev300]

I know, but my goal is, that the inverter reacts as quickly as possible. I think you have to smooth quiet a lot of data, like 15s in your example, to get rid of the overshooting, but thats too slow for me. :D

I'm very happy with 0.5s and if it's working without overshooting: 👍
I noticed that my smartmeter has inconsitent latency. Some times < 1s some times >2s but it's running with tasmota atm. Maybe I can fix it with esphome 😁

You can see in my changes, that i tried running with update_intervall: never and only trigger the update by the sensor reading.
But it was too inconsistant for the inverter, so it went to 0W some times. But the intervall was never > 3. 🤔

By the way: my inverter seems to react somewhere between 0.5s <--> 1s to demand changes. But it's the whole loop demand-setting -> inverter -> smartmeter -> esp01 reader -> Home Assistant -> ESP32 -> inverter, which is taking around 2.5-3s latency. So I can't confirm the slow reaction of humpataa Only for the whole control-loop.

[pfeupfeu]

I also like to see the inverter reacting as quick as possible.
The original limiter outputs the demand in 0.5s intervals, but must also take the total latency into account. How does it do that?
Is there an overshoot with the original limiter?

I think triggering the uptdate by a new power sensor reading, is an good idea, but how can I do this?

[kev300]

I think triggering the uptdate by a new power sensor reading, is an good idea, but how can I do this?

Just set update_interval of soyosource_virtual_meter to never (and use my branch here: https://github.com/kev300/esphome-soyosource-gtn-virtual-meter/tree/overshoot-compensation2)

Or instead of using my branch also add this to your powermeter sensor:

on_value:
  then:
    -component.update: virtualmeter0

virtualmeter0 is the id of the soyosource_virtual_meter

[kev300]

But keep in mind:
If your power sensor reading is inconsistent, the RS485 protocol will fail and reset the inverter to 0W occasionally.
It was the case for me, so I had to go back to update_intervall: 500ms which is reliable.

[pfeupfeu]

Thanks, it is working!

At first I got this Error:

  on_value:
    then:

    [-component.update] is an invalid option for [on_value]. Please check the indentation.
    -component.update: virtualmeter0

So I put the update command at the end of the Shelly 3EM query:

time:
  - platform: sntp
    id: sntp_time
    on_time:
      - seconds: /1
        then:
          - http_request.get:
              url: ${shelly_3em_url}
              headers:
                Content-Type: application/json
              verify_ssl: false
              on_response:
                then:
                  - lambda: |-
                      json::parse_json(id(http_request_data).get_string(), [](JsonObject root) {
                        id(powermeter).publish_state(root["total_power"]);
                      });
          - component.update: virtualmeter0

Now it is working, with 1s repetition rate and without overshoot. I'm using syssis unchanged software.

[kev300]

Cool, seems like your shelly and directly querying it is much faster than my smartmeter setup through home assistant. 😉

[pfeupfeu]

Yersterday I told you, that I would'nt have overshoots. Ok, that was true, but this morning I found one of my 3 inverters was not working yersterday.
With the third inverter working I get little overshoots, independet from the update interval.

Operation with one inverter missing yesterday was similar to Kevin's magic_demand_delta set to 0,67.
--> Ok, we need the demand delta factor for better control characteristics. It is much better than averaging the smart meter readings.
In control engineering, this factor is often referred to as "Kp", perhaps the variable name can be oriented to this.

[kev300]

Yes, Kp fits. 😁

By the way, I found this right now:
https://esphome.io/components/climate/pid.html

Maybe we can use that and just replace °C by Watts (Adapt our sensor + demand control). Not sure if thats realy possible with that component. But I would only go there if my simplified solution does not work good enough.

[Chickenbreast0]

Thanks, it is working!

At first I got this Error:

  on_value:
    then:

    [-component.update] is an invalid option for [on_value]. Please check the indentation.
    -component.update: virtualmeter0

So I put the update command at the end of the Shelly 3EM query:

time:
  - platform: sntp
    id: sntp_time
    on_time:
      - seconds: /1
        then:
          - http_request.get:
              url: ${shelly_3em_url}
              headers:
                Content-Type: application/json
              verify_ssl: false
              on_response:
                then:
                  - lambda: |-
                      json::parse_json(id(http_request_data).get_string(), [](JsonObject root) {
                        id(powermeter).publish_state(root["total_power"]);
                      });
          - component.update: virtualmeter0

Now it is working, with 1s repetition rate and without overshoot. I'm using syssis unchanged software.

Would you please so kind and paste your entire code here?
Currently I have some "timeout" issues with my Soyo. I guess the reason might be the "looong way" in my configuration: Shelly 3EM -> HA -> merge the values it into a sum -> ESPHome -> WT32 -> Soyo

Thanks in advance.

[pfeupfeu]

Wy don't you use the sum of the 3 phases provided by the 3EM http status page?
My code that I now use has changed since March, to have a finer resolution for the repetition of the 3EM query:

interval:
  - interval: 900ms
    then:
      - http_request.get: 
         url: ${shelly_3em_url}
         headers:
            Content-Type: application/json
         verify_ssl: false
         on_response:
            then:
              - lambda: |-
                  std::string response_data = id(http_request_data).get_string();
                  if(!response_data.empty()) {
                  json::parse_json(response_data, [](JsonObject root) {
                  id(powermeter).publish_state(root["total_power"]);
                  });
                  }
              - component.update: virtualmeter0

And at

soyosource_virtual_meter: 
  - id: virtualmeter0
[...]
    update_interval: never

[kev300]

@pfeupfeu : do you have graphs with 1s resolution of you power consumption meter? I can't belive that you don't have overshoots or oscillation with the shelly and 900ms (btw. why 900?).
You're still on syssi's master branch, right?
😬
The shelly must be very quick to react. But the Soyo also takes 0.5-1s to react to demand changes...

[Chickenbreast0]

Wy don't you use the sum of the 3 phases provided by the 3EM http status page? My code that I now use has changed since March, to have a finer resolution for the repetition of the 3EM query:

interval:
  - interval: 900ms
    then:
      - http_request.get: 
         url: ${shelly_3em_url}
         headers:
            Content-Type: application/json
         verify_ssl: false
         on_response:
            then:
              - lambda: |-
                  std::string response_data = id(http_request_data).get_string();
                  if(!response_data.empty()) {
                  json::parse_json(response_data, [](JsonObject root) {
                  id(powermeter).publish_state(root["total_power"]);
                  });
                  }
              - component.update: virtualmeter0

And at

soyosource_virtual_meter: 
  - id: virtualmeter0
[...]
    update_interval: never

Thanks for getting back to me.
The reason is simply that I currently lack the time to do further research and I was previously under the impression that there might be a hardware defect. Also, I'm new to this topic - that should answer your question sufficiently ;)

Maybe you can help me again and tell me where my error is so that your code does not work for me?

[Chickenbreast0]

This is my entire code. No matter what I try, it would be appreciated if you can share me yours. So I can find where my mistake is.

substitutions:
  name: epever-soyosource

esphome:
  name: wt32-eth01
  friendly_name: WT32
  platformio_options:
    build_flags:
      - -DCONFIG_ARDUINO_LOOP_STACK_SIZE=8192

esp32:
  board: esp32dev
  framework:
    type: arduino

external_components:
  - source: github://syssi/esphome-soyosource-gtn-virtual-meter@main
    refresh: 0s

ethernet:
  type: LAN8720
  mdc_pin: GPIO23
  mdio_pin: GPIO18
  clk_mode: GPIO0_IN
  phy_addr: 1
  power_pin: GPIO16

# Optional manual IP
  manual_ip:
    static_ip: 192.168.178.9
    gateway: 192.168.178.1
    subnet: 255.255.255.0

# Enable logging
logger: 
  level: DEBUG
  baud_rate: 0

# Enable Home Assistant API
api:
  reboot_timeout: 0s
  encryption:
    key: "blanked out"

ota:
  password: "blanked out"

uart:
  - id: uart_0
    tx_pin: GPIO1 #TXD0 
    rx_pin: GPIO3 #RXD0
    baud_rate: 115200
    stop_bits: 1

  - id: uart_1
    baud_rate: 4800
    tx_pin: GPIO17 #TXD
    rx_pin: GPIO5 #RXD
    debug:
      direction: BOTH

modbus:
  - id: modbus_epever
    uart_id: uart_0
    send_wait_time: 200ms

modbus_controller:
  - id: epever
    address: 0x1
    modbus_id: modbus_epever
    command_throttle: 200ms
    setup_priority: -10
    update_interval: 2s

soyosource_modbus:
  - id: modbus_soyo
    uart_id: uart_1

soyosource_virtual_meter:
  - id: virtualmeter0
    soyosource_modbus_id: modbus_soyo
    power_id: powermeter0
    power_sensor_inactivity_timeout: 20s
    power_demand_calculation: NEGATIVE_MEASUREMENTS_REQUIRED
    min_power_demand: 0
    max_power_demand: 700
    zero_output_on_min_power_demand: true
    buffer: 5
    update_interval: never

soyosource_inverter:
  - id: inverter0
    soyosource_modbus_id: modbus_soyo

preferences:
  flash_write_interval: 1min

binary_sensor:
  - platform: soyosource_inverter
    soyosource_inverter_id: inverter0
    fan_running:
      name: "soyo fan running"

number:
  - platform: soyosource_virtual_meter
    soyosource_virtual_meter_id: virtualmeter0
    buffer:
      name: "soyo buffer"
      initial_value: 5
      restore_value: true
    manual_power_demand:
      name: "soyo manual power demand"
      max_value: 700
    max_power_demand:
      name: "soyo max power demand"
      initial_value: 50
      max_value: 700
      restore_value: true

interval:
  - interval: 1s
    then:
      - http_request.get:
         url: http://192.168.178.5/status/
         headers:
            Content-Type: application/json
         verify_ssl: false
         on_response:
            then:
              - lambda: |-
                  std::string response_data = id(http_request_data).get_string();
                  if(!response_data.empty()) {
                  json::parse_json(response_data, [](JsonObject root) {
                  id(powermeter).publish_state(root["total_power"]);
                  });
                  }
              - component.update: virtualmeter0


sensor:
  # import smartmeter reading from homeassistant
  #- platform: homeassistant
  #  id: powermeter0
  #  name: "smartmeter instantaneous power"
  #  entity_id: sensor.phases_sum
  #  filters:
  #   - throttle_average: 6s

  - platform: modbus_controller
    modbus_controller_id: epever
    id: max_battery_voltage_today
    name: "EPEVER Maximum battery voltage today"
    address: 0x3302
    register_type: read
    value_type: U_WORD
    accuracy_decimals: 1
    unit_of_measurement: "V"
    filters:
      - multiply: 0.01

  - platform: modbus_controller
    modbus_controller_id: epever
    id: min_battery_today
    name: "EPEVER Minimum battery voltage today"
    address: 0x3303
    register_type: read
    value_type: U_WORD
    accuracy_decimals: 1
    unit_of_measurement: "V"
    filters:
      - multiply: 0.01

  - platform: modbus_controller
    modbus_controller_id: epever
    id: generated_energy_today
    name: "EPEVER Generated energy today"
    address: 0x330C
    register_type: read
    value_type: U_DWORD_R
    accuracy_decimals: 0
    unit_of_measurement: "Wh"
    filters:
      - multiply: 10.0

  - platform: modbus_controller
    modbus_controller_id: epever
    id: battery_voltage
    name: "EPEVER Battery voltage"
    address: 0x331A
    register_type: read
    value_type: U_WORD
    accuracy_decimals: 1
    unit_of_measurement: "V"
    filters:
      - multiply: 0.01

  - platform: modbus_controller
    modbus_controller_id: epever
    id: battery_current
    name: "EPEVER Battery current"
    address: 0x331B
    register_type: read
    value_type: S_DWORD_R
    register_count: 2
    accuracy_decimals: 2
    unit_of_measurement: "A"
    filters:
      - multiply: 0.01

  - platform: modbus_controller
    modbus_controller_id: epever
    id: pv_input_voltage
    name: "EPEVER PV array input voltage"
    address: 0x3100
    unit_of_measurement: "V"
    register_type: read
    value_type: U_WORD
    accuracy_decimals: 1
    filters:
      - multiply: 0.01

  - platform: modbus_controller
    modbus_controller_id: epever
    id: pv_input_current
    name: "EPEVER PV array input current"
    address: 0x3101
    unit_of_measurement: "A"
    register_type: read
    value_type: U_WORD
    accuracy_decimals: 2
    filters:
      - multiply: 0.01

  - platform: modbus_controller
    modbus_controller_id: epever
    id: pv_input_power
    name: "EPEVER PV array input power"
    address: 0x3102
    unit_of_measurement: "W"
    register_type: read
    value_type: U_DWORD_R
    accuracy_decimals: 1
    filters:
      - multiply: 0.01

  - platform: modbus_controller
    modbus_controller_id: epever
    id: charging_voltage
    name: "EPEVER Charging voltage"
    address: 0x3104
    unit_of_measurement: "V"
    register_type: read
    value_type: U_WORD
    accuracy_decimals: 1
    filters:
      - multiply: 0.01

  - platform: modbus_controller
    modbus_controller_id: epever
    id: charging_current
    name: "EPEVER Charging current"
    address: 0x3105
    unit_of_measurement: "A"
    register_type: read
    value_type: U_WORD
    accuracy_decimals: 1
    filters:
      - multiply: 0.01

  - platform: modbus_controller
    modbus_controller_id: epever
    id: Charging_power
    name: "EPEVER Charging power"
    address: 0x3106
    unit_of_measurement: "W"
    register_type: read
    value_type: U_DWORD_R
    accuracy_decimals: 1
    filters:
      - multiply: 0.01

  - platform: modbus_controller
    modbus_controller_id: epever
    id: load_voltage
    name: "EPEVER Load voltage"
    address: 0x310C
    unit_of_measurement: "V"
    register_type: read
    value_type: U_WORD
    accuracy_decimals: 1
    filters:
      - multiply: 0.01

  - platform: modbus_controller
    modbus_controller_id: epever
    id: load_current
    name: "EPEVER Load Current"
    address: 0x310D
    unit_of_measurement: "A"
    register_type: read
    value_type: U_WORD
    accuracy_decimals: 2
    filters:
      - multiply: 0.01

  - platform: modbus_controller
    modbus_controller_id: epever
    id: load_power
    name: "EPEVER Load power"
    address: 0x310E
    unit_of_measurement: "W"
    register_type: read
    value_type: U_DWORD_R
    accuracy_decimals: 1
    filters:
      - multiply: 0.01

  - platform: modbus_controller
    modbus_controller_id: epever
    id: battery_temperature
    name: "EPEVER Battery temperature"
    address: 0x3110
    unit_of_measurement: °C
    register_type: read
    value_type: S_WORD
    accuracy_decimals: 1
    filters:
      - multiply: 0.01

  - platform: modbus_controller
    modbus_controller_id: epever
    id: device_temperature
    name: "EPEVER Device temperature"
    address: 0x3111
    unit_of_measurement: °C
    register_type: read
    value_type: S_WORD
    accuracy_decimals: 1
    filters:
      - multiply: 0.01

  - platform: modbus_controller
    modbus_controller_id: epever
    id: power_components_temperature
    name: "EPEVER Power components temperature"
    address: 0x3112
    unit_of_measurement: °C
    register_type: read
    value_type: S_WORD
    accuracy_decimals: 1
    filters:
      - multiply: 0.01

  - platform: modbus_controller
    modbus_controller_id: epever
    id: battery_soc
    name: "EPEVER Battery SOC"
    address: 0x311A
    unit_of_measurement: "%"
    device_class: battery
    register_type: read
    value_type: U_WORD
    accuracy_decimals: 0

  - platform: soyosource_virtual_meter
    soyosource_virtual_meter_id: virtualmeter0
    power_demand:
      name: "soyo power demand"

  - platform: soyosource_inverter
    soyosource_inverter_id: inverter0
    operation_status_id:
      name: "operation status id"
      id: operation_status_id0
    battery_voltage:
      name: "soyo battery voltage"
    battery_current:
      name: "soyo battery current"
    battery_power:
      name: "soyo battery power"
    ac_voltage:
      name: "soyo ac voltage"
    ac_frequency:
      name: "soyo ac frequency"
    temperature:
      name: "soyo temperature"

switch:
  - platform: modbus_controller
    modbus_controller_id: epever
    id: manual_control_load
    register_type: coil
    address: 2
    name: "EPEVER load switch"
    bitmask: 1

  - platform: modbus_controller
    modbus_controller_id: epever
    id: clear_energy_stats
    register_type: coil
    address: 0x14
    name: "Clear generating  electricity statistic"
    bitmask: 1

  - platform: soyosource_virtual_meter
    soyosource_virtual_meter_id: virtualmeter0
    manual_mode:
      name: "soyo manual mode"
      restore_mode: RESTORE_DEFAULT_ON
    emergency_power_off:
      name: "soyo emergency power off"
      restore_mode: RESTORE_DEFAULT_OFF

text_sensor:      
  - platform: modbus_controller
    modbus_controller_id: epever
    name: "EPEVER Charging Status"
    address: 0x3201
    register_type: read
    # bitmask: 12
    raw_encode: HEXBYTES
    lambda: |-
      uint16_t value = modbus_controller::word_from_hex_str(x, 0);
      value = modbus_controller::mask_and_shift_by_rightbit(value, 12);
      switch (value) {
        case 0: return std::string("Off");
        case 1: return std::string("Float");
        case 2: return std::string("Boost");
        case 3: return std::string("Equalization");
      }
      return str_snprintf("Unknown (0x%04X)", 17, value);

  - platform: modbus_controller
    modbus_controller_id: epever
    name: "EPEVER Solar Status"
    address: 0x3201
    register_type: read
    # bitmask: 2
    raw_encode: HEXBYTES
    lambda: |-
      uint16_t value = modbus_controller::word_from_hex_str(x, 0);
      value = modbus_controller::mask_and_shift_by_rightbit(value, 2);
      switch (value) {
        case 0: return std::string("Cut out");
        case 1: return std::string("Input");
      }
      return str_snprintf("Unknown (0x%04X)", 17, value);

  - platform: modbus_controller
    modbus_controller_id: epever
    name: "EPEVER Device Status"
    address: 0x3201
    register_type: read
    # bitmask: 1
    raw_encode: HEXBYTES
    lambda: |-
      uint16_t value = modbus_controller::word_from_hex_str(x, 0);
      value = modbus_controller::mask_and_shift_by_rightbit(value, 1);
      switch (value) {
        case 0: return std::string("Standby");
        case 1: return std::string("Normal");
      }
      return str_snprintf("Unknown (0x%04X)", 17, value);

  - platform: modbus_controller
    modbus_controller_id: epever
    name: "EPEVER Battery Voltage Status"
    address: 0x3200
    register_type: read
    # bitmask: 7
    raw_encode: HEXBYTES
    lambda: |-
      uint16_t value = modbus_controller::word_from_hex_str(x, 0);
      value = modbus_controller::mask_and_shift_by_rightbit(value, 7);
      switch (value) {
        case 0: return std::string("Normal");
        case 1: return std::string("Overvolt");
        case 2: return std::string("Low voltage disconnect");
        case 3: return std::string("Fault");
      }
      return str_snprintf("Unknown (0x%04X)", 17, value);

  - platform: modbus_controller
    modbus_controller_id: epever
    name: "EPEVER Battery Temperature Status"
    address: 0x3200
    register_type: read
    # bitmask: 56
    raw_encode: HEXBYTES
    lambda: |-
      uint16_t value = modbus_controller::word_from_hex_str(x, 0);
      value = modbus_controller::mask_and_shift_by_rightbit(value, 56);
      switch (value) {
        case 0: return std::string("Normal");
        case 1: return std::string("Overtemperature");
        case 2: return std::string("Undertemperature");
      }
      return str_snprintf("Unknown (0x%04X)", 17, value);
      
  - platform: soyosource_virtual_meter
    soyosource_virtual_meter_id: virtualmeter0
    operation_mode:
      name: "soyo limiter operation mode"

  - platform: soyosource_inverter
    soyosource_inverter_id: inverter0
    operation_status:
      name: "soyo operation status"

[pfeupfeu]

I'm sorry, I forgot you need also this:

http_request:
  id: http_request_data
  useragent: esphome/device
  timeout: 500ms

For reference, and in case I forgot something else, this is my entire code:

substitutions:
  name: soyo-inverter
  device_description: "Monitor and control the Soyosource GTN via the display port and control the power output on demand via RS485"
  external_components_source: github://syssi/esphome-soyosource-gtn-virtual-meter@main
  shelly_3em_url: http://192.168.188.71/status/
  tx_pin_ttl_wifi: GPIO4  #d2
  rx_pin_ttl_wifi: GPIO5  #d1
  tx_pin_rs485: GPIO14  #d5
  rx_pin_rs485: GPIO12  #d6
  
     
esphome:
  name: ${name}
  comment: ${device_description}
  project:
    name: "syssi.esphome-soyosource-gtn-virtual-meter"
    version: 2.1.0
    
esp8266:
  board: d1_mini

external_components:
  - source: ${external_components_source}
    refresh: 0s

wifi: 
  ssid: !secret wifi_ssid
  password: !secret wifi_password
  manual_ip:
    static_ip: 192.168.188.44
    gateway: 192.168.188.1
    subnet: 255.255.255.0
    dns1: 1.1.1.1

  ap:
    ssid: "Fallback-AP-soyo"
    password: ""  
    
captive_portal:

web_server:
  local: true
  version: 2

ota:

api:
  reboot_timeout: 3d


logger:
  level: INFO

http_request:
  id: http_request_data
  useragent: esphome/device
  timeout: 500ms
  
uart:
  - id: uart0
    baud_rate: 9600
    tx_pin: ${tx_pin_ttl_wifi}
    rx_pin: ${rx_pin_ttl_wifi}

  - id: uart1
    baud_rate: 4800
    tx_pin: ${tx_pin_rs485}
    rx_pin: ${rx_pin_rs485}

soyosource_modbus:
  - id: modbus0
    uart_id: uart1
    

soyosource_virtual_meter: # hier wird die die RS485-Anforderung errechnet
  - id: virtualmeter0
    soyosource_modbus_id: modbus0

    # The state of this sensor (instantaneous power in watt) is used as source
    power_id: powermeter

    # Optional settings
    power_sensor_inactivity_timeout: 20s
    power_demand_calculation: NEGATIVE_MEASUREMENTS_REQUIRED
    min_power_demand: 0 
    max_power_demand: 3600
    # Split/distribute the power demand if you have multiple inverters attached to the same RS485 bus
    # power_demand_divider: 2
    # A positive buffer value (10) tries to avoid exporting power to the grid (demand - 10 watts)
    # A negative buffer value (-10) exports power to the grid (demand + 10 watts)
    # buffer: 1
    # The operation_status_id sensor is expected here. Passing the operation_status won't work
    # The state is used to suspend the limiter if the operation status of the inverter isn't 0x0 (normal)
    operation_status_id: operation_status_id0

    # The update interval is important and defaults to 3 seconds. If the demand is sent too frequently
    # or rarely the interverter stops. TODO: Identify and validate the lower and upper update limit
    update_interval: never

soyosource_display:
  - id: display0
    uart_id: uart0
 #   protocol_version: SOYOSOURCE_WIFI_VERSION
    update_interval: 5s

binary_sensor:
  - platform: soyosource_display
    fan_running:
      name: "so1 fan running"
    limiter_connected:
      name: "So1 Limiter"

button:
  - platform: soyosource_display
    restart:
      name: "Soyo1 restart"

number:

  - platform: soyosource_virtual_meter
    soyosource_virtual_meter_id: virtualmeter0
    buffer:
      name: "Power buffer"
      id: buffer0
      mode: "box"
      initial_value: 0
      min_value: -301
      max_value: 100
      step: 1
    power_demand_divider:
      name: "Soyo number"
      mode: "box"
      initial_value: 3
      max_value: 4
      restore_value: true
    manual_power_demand:
      name: "manual P demand RS485"
      mode: "box"
      max_value: 3600
      step: 10
    max_power_demand:
      name: "Max P demand"
      mode: "box"
      initial_value: 3600
      min_value: 0
      max_value: 3600
      

  - platform: soyosource_display
    start_voltage:
      name: "Soy1 Start V"
      min_value: 49
      max_value: 59
    shutdown_voltage:
      name: "Soy1 Shutdwn V"
      min_value: 48
    # Maximum output power in limiter mode / Output power in constant power mode
    output_power_limit:
      name: "So1 P limit"
    start_delay:
      name: "so1 start delay"
      min_value: 1
      
  - platform: template
    name: "Bat full voltage"
    id: bat_full_v
    optimistic: true
    unit_of_measurement: "V"
    mode: "box"
    initial_value: 54.7
    min_value: 54
    max_value: 56
    step: 0.1
    
  - platform: template
    name: "Bat_float_voltage"
    id: bat_float_v
    optimistic: true
    unit_of_measurement: "V"
    mode: "box"
    initial_value: 53.3
    min_value: 52
    max_value: 54
    step: 0.1
    
  - platform: template
    name: "Bat full extra power"
    id: bat_full_power
    optimistic: true
    unit_of_measurement: "W"
    mode: "box"
    initial_value: 250
    min_value: 0
    max_value: 300
    step: 1
     
select:
  - platform: soyosource_display
    operation_mode:
      name: "Soyo1 Op mode select"
      optionsmap:
        1: "PV"
        2: "Battery Constant Power"
        17: "PV Limit"
        18: "Battery Limit"

  
sensor:
  - platform: template
    id: powermeter
    name: " P @shelly3EM"
    lambda: |-
      if (std::isnan(id(powermeter).state)) {
        return 0.0f;
      } else {
        return id(powermeter).state;
      }
    unit_of_measurement: W
    device_class: "power"
    accuracy_decimals: 1
    filters:
      - calibrate_linear:
        - 0.0 -> 0.0
        - 30 -> 25
        - 500-> 490 
        
  - platform: template
    name: "Bat full extra grid P"
    id: batteryfull
    lambda: |-
        static int bat_full = 0;
        static int buffer_temp = 0;
        buffer_temp =id(buffer0).state;
        if ((bat_full == 0) && id(battery_voltage0).state > id(bat_full_v).state)  {
          // Batterie ist jetzt voll, buffer herabsetzen
          bat_full = 1; 
          buffer_temp -= id(bat_full_power).state;
          auto call = id(buffer0).make_call();
          call.set_value(buffer_temp);
          call.perform();
        } 
        if (id(battery_voltage0).state < id(bat_float_v).state && bat_full == 1) {
          // Batterie ist nicht mehr voll, buffer wieder anheben
          bat_full = 0; 
          buffer_temp += id(bat_full_power).state;
          auto call = id(buffer0).make_call();
          call.set_value(buffer_temp);
          call.perform();
        }
        return (bat_full * id(bat_full_power).state);
    unit_of_measurement: W
    device_class: "power"
    accuracy_decimals: 0
    update_interval: 60s
        
        
  - platform: soyosource_virtual_meter
    soyosource_virtual_meter_id: virtualmeter0
    power_demand:
      name: " Soyo P-demand"
  
  - platform: soyosource_display
   # error_bitmask:
   #   name: "error Bitmask"
#    operation_mode_id:
#      name: "op Mode ID"
    operation_status_id:
 #     name: "op Status ID"
      id: operation_status_id0
    battery_voltage:
      name: "Bat Voltage"
      id: battery_voltage0
      filters:  
        - median:
            window_size: 3
    battery_current:
      name: "soyo1 Bat A"
    battery_power:
      name: "soyo1 Bat P"
    ac_voltage:
      name: "AC Voltage"
   # ac_frequency:
   #   name: "AC Frequency"
    temperature:
      name: "soyo1 Temp"
    #total_energy:
     # name: "Total Energy"
    output_power:
      name: "So1 Output P"


  - platform: wifi_signal
    name: "WiFi Signal"
    update_interval: 30s

switch:
  - platform: soyosource_virtual_meter
    soyosource_virtual_meter_id: virtualmeter0
    emergency_power_off:
      name: " Emergency off"
      restore_mode: RESTORE_DEFAULT_OFF    
    manual_mode:
      name: "Manual mode RS485"
      restore_mode: RESTORE_DEFAULT_OFF


text_sensor:
  - platform: soyosource_virtual_meter
    soyosource_virtual_meter_id: virtualmeter0
    operation_mode:
      name: "Limiter mode"

  - platform: soyosource_display
    errors:
      name: "Soyo errors"
    operation_mode:
      name: "Soyo1 Op mode reported"
    operation_status:
      name: "Op status"


  - interval: 900ms
    then:
      - http_request.get: 
         url: ${shelly_3em_url}
         headers:
            Content-Type: application/json
         verify_ssl: false
         on_response:
            then:
              - lambda: |-
                  std::string response_data = id(http_request_data).get_string();
                  if(!response_data.empty()) {
                  json::parse_json(response_data, [](JsonObject root) {
                  id(powermeter).publish_state(root["total_power"]);
                  });
                  }
              - component.update: virtualmeter0

[kev300]

Are you guys all on vacation? :D Or nobody has overshoots anymore?
I'm just wondering why it's soo quiet here. 😄

[syssi]

Vacation. ;-)

[Chickenbreast0]

Are you guys all on vacation? :D Or nobody has overshoots anymore? I'm just wondering why it's soo quiet here. 😄

Currently am busy with my job and I just have some little time windows, in which I need to take care of more important stuff ;)
But how do we say in german language: "Aufgeschoben ist nicht aufgehoben!"(Postponed is not abandoned!)

[mb-software]

Hi,
I am also trying to optimize the zero feed controller with this component.
For easier handling of modifications and quick testing of changes I am following a different approach: I set the component to manual demand mode and calculate/set the demand from within a script/action in the EspHome YAML code. This way I can modify, upload and test the controller algorithm simply through the esphome webinterface,

Anyway, currently I have replicated the algorithm of the original component using my method and it works ok (but not perfect) with a 3s interval action.
Now I want to improve it, maybe using a similar approach like @kev300 , though I have to admit that from looking into the code, I don't completely understand how the demand compensation works. Is it only a fixed delay time you wait until you expect the inverter to produce the power that you demanded? Maybe a short prosa description would help here.
Another approach that I am thinking of could be using the actual output power sensor that I am getting from the soyosource display component. But therefore I would need a short update intervall of the display component, I have no idea how low one can go there. Any experience with that?

[kev300]

Isn't the original behavior already included in this component by syssi or what do you mean by original component?

Did you try to run the branch from my pull request?
I tested it successfully down to 0.2s update interval with my inverter, but each feedback is welcome. 😄

My solution does react immediately on each power demand change but it subtracts the last demand change from the current calculation, because that change has not reach the powermeter, yet. It is subtracted in each update until the targeted zero consumption is reached or crossed.
In case it is not reached in time, there is a fallback timeout, which resets the calculation. This timeout is configurable in the yaml (default 5s)

The 5 s works good if your powermeter updates each 1-2s.
Maybe you need to increase it, if it is slower.

I have run the display update interval with 1s without issues, but did not go lower.

Maybe also a good idea, but the display reading is also very inaccurate and do not reflect the actual ac power.