Total kWh consumption?

[Kev1000000]

How would I go about totaling up the total usage in kWh for a specific device?

[NachtaktiverHalbaffe]

With the Riemann Integral integration. Physically energy usage (kWh) is the integration of power (W) and the integral integration does exactly this. A example configuration can be seen below.

- platform: integration
  source: sensor.lights_power
  name: lights_power_kWh
  unit_prefix: k
  round: 2

Home Assistant updated this integration recently, so it can be used directly with the energy integration. Before that you would have to feed this created sensor into a utility meter. It can be useful though if you make a utility meter, if you want to reset the usage e.g. every month, day or whathever cycle or you want to work with peak and off-peak tariffs.

[ronaldheft]

Seems like this works, if lights’ state changes. The Riemann Integral integration only updates on state change, and since the energy values for these sensors are static, if a light is constantly on, the Riemann Integral integration will never get a value.

I don’t think this is necessary an issue with this library; just trying to figure out a workaround for those lights.

[bramstroker]

I already worked out a solution to update power state at a interval regardless whether the value had changed. This makes it also work for fixed mode sensors. It is merged in the master branch. Will do some final changes in the next days and than make a new release.

[marithpl]

I saw there was an update. Im not sure if it work correctly.
Here is my powercalc group, template sensor with sum of power consumption and the last total in Wh.

Ex. my bulbs in Standby mode consume about 5W of power, so for each hour total consumption should be at least 5Wh.

For whole day I have about 1,40Wh

- platform: template
  sensors:
    all_light_power:
      friendly_name: All light power
      unit_of_measurement: W
      device_class: power
      icon_template: mdi:transmission-tower
      value_template: >
        {{ expand('group.all_light_power') 
        | rejectattr('state', 'in', ['unavailable', 'unknown'])
        | map(attribute='state') | map('float') | sum | round(2) }}

- platform: integration
  source: sensor.all_light_power
  name: Total Light Consumption
  unit_prefix: k
  round: 2

all_light_power:
    name: All lights power
    entities:
      - sensor.spot_1_power
      - sensor.spot_2_power
      - sensor.spot_3_power

[bramstroker]

@marithpl I'm not sure what could be the cause. I must update my component first in my production instance to do some aggregation. Did not have time to do it yet. I only did some basic testing in my development HA instance while developing this feature and checked whether the entity states were updated frequently. Maybe there is some issue with the riemann integral component, but I don't know the exact inner workings of that. Maybe @NachtaktiverHalbaffe has an idea?

[marithpl]

Other example is my AC. In standby mode it consume about 8W, during a night Total Sensor count about 0,07 kWh and it seems be correctly.

In powercalc light I still have below 2 Wh after to days. It’s impossible.

I will really appreciate for fixing this issue.

[NachtaktiverHalbaffe]

@marithpl I did have the same problem when creating a "energy group". The TLDR Version is that you have to create a integration sensor for each powercalc sensor so each of these integration sensors gets triggers every 10 min (or whatever you specified as an scan_interval). If you want to create light groups, then create a template sensor which sums up all integration sensors.
I have 1 template below which helps creating all integration sensors . The template sensor creates integrations sensors for all power sensors (so not just the powercalc ones), you can paste the template in dev tools--> templates and copy the config created by the template into your configuration.yaml (or only the ones you need)

{% for state in states -%}
{%- if state.attributes.unit_of_measurement == "W" and state.attributes.device_class == "power" -%}
- platform: integration
  source: {{ state.entity_id }}
  unit_prefix: k
  round: 2
{% endif -%}
{%- endfor -%}

An "energy group" sensor can then look like this:

- platform: template
  sensors:
    energy_server:
      friendly_name: "Alle Server Energieverbrauch"
      unit_of_measurement: kWh
      value_template: >-
        {{states('sensor.kingkong_power_kwh') | float + states('sensor.kinglouie_power_kwh')|float}}
      attribute_templates:
        last_reset: "1970-01-01T00:00:00+00:00"
        state_class: measurement
        device_class: energy
        icon: mdi:counter

I dont know the cause of the problem exactly, but my suggestion is that the integration integration only updates, if the underlying sensor updates. With the new powercalc update with scan_intervals we fixed that. If we create a template sensor which sums up all powers of an specified category, the template sensors only updates its state when the values from the values template changes which it doesnt if you have a constant power like the standby-usage. So if you do it the way i specified, the integration integration gets regulary triggered by the scaninterval and will calculate the energy unregarding that the power value itself didnt change. The "energy group" sensor gets triggered when the kWh value changes from the underlying sensors which should then happen often cause this is an evergrowing and as a consequence changing value.

@bramstroker Maybe we should include a section in the Readme about creating energy sensors and how to create a "energy group"? I could write it sometime next week and create a PR. It will be my first ever PR though, sop dont mind if my PR isnt state of the art 😄

[bramstroker]

@NachtaktiverHalbaffe thanks for the clear explanation. Yes it's much appreciated if you can make a chapter in the readme how to setup everything for the energy dashboard. Your English is very good, I cannot do it better myself ;-). Looking forward to your PR.

[bramstroker]

Example has been added to the readme by @NachtaktiverHalbaffe
Closing this issue now.

[marithpl]

@marithpl I did have the same problem when creating a "energy group". The TLDR Version is that you have to create a integration sensor for each powercalc sensor so each of these integration sensors gets triggers every 10 min (or whatever you specified as an scan_interval). If you want to create light groups, then create a template sensor which sums up all integration sensors.
I have 1 template below which helps creating all integration sensors . The template sensor creates integrations sensors for all power sensors (so not just the powercalc ones), you can paste the template in dev tools--> templates and copy the config created by the template into your configuration.yaml (or only the ones you need)

{% for state in states -%}
{%- if state.attributes.unit_of_measurement == "W" and state.attributes.device_class == "power" -%}
- platform: integration
  source: {{ state.entity_id }}
  unit_prefix: k
  round: 2
{% endif -%}
{%- endfor -%}

An "energy group" sensor can then look like this:

- platform: template
  sensors:
    energy_server:
      friendly_name: "Alle Server Energieverbrauch"
      unit_of_measurement: kWh
      value_template: >-
        {{states('sensor.kingkong_power_kwh') | float + states('sensor.kinglouie_power_kwh')|float}}
      attribute_templates:
        last_reset: "1970-01-01T00:00:00+00:00"
        state_class: measurement
        device_class: energy
        icon: mdi:counter

I dont know the cause of the problem exactly, but my suggestion is that the integration integration only updates, if the underlying sensor updates. With the new powercalc update with scan_intervals we fixed that. If we create a template sensor which sums up all powers of an specified category, the template sensors only updates its state when the values from the values template changes which it doesnt if you have a constant power like the standby-usage. So if you do it the way i specified, the integration integration gets regulary triggered by the scaninterval and will calculate the energy unregarding that the power value itself didnt change. The "energy group" sensor gets triggered when the kWh value changes from the underlying sensors which should then happen often cause this is an evergrowing and as a consequence changing value.

@bramstroker Maybe we should include a section in the Readme about creating energy sensors and how to create a "energy group"? I could write it sometime next week and create a PR. It will be my first ever PR though, sop dont mind if my PR isnt state of the art 😄

To be honest I'm not sure what should I do.
Have I create a template sensor for each bulb and then sum all bulbs in next template sensor?

[OzGav]

I have just today started looking at the major part of the latest 2021.8 release that being the energy dashboard. In regards to this issue I would echo that it would be great if the integration generated the ENERGY sensor automatically as it does for power. Perhaps a configuration option to enable or disable this?

Also while I am here I just learned that there is a new way (old way used above: https://www.home-assistant.io/integrations/template/#legacy-sensor-configuration-format) to define templates so @NachtaktiverHalbaffe example above I think should now be:

template:
  - sensor:
      - unique_id: "energy_server"
        name: "Alle Server Energieverbrauch"
        unit_of_measurement: "kWh"
        state_class: "measurement"
        device_class: "energy"
        attributes:
          last_reset: "1970-01-01T00:00:00+00:00"
        icon: "mdi:fan-on"
        state: >-
            {{states('sensor.kingkong_power_kwh') | float + states('sensor.kinglouie_power_kwh')|float}}             

I am going to have to do some more research on the Riemann Integral integration because that is a problem I think if it only updates on a state change. The energy dashboard has the ability to display total energy used from whatever devices but it seems if that isn't calculated natively by the device (eg smart plug) then how to sum the total power used unless you do some sort of time triggering template and add it all up yourself?

[NachtaktiverHalbaffe]

@marithpl You should create a Riemann integration integration for each power sensor you have and you want to use in the energy dashboard (so this can be done with all power sensors and not just with powercalc sensors). This sensors are your energy sensors and can be used directly in the energy dashboard. If you want to group them up, then you can sum them up in a template sensor like in my example.

@OzGav My template sensor works and never change a running system 😄. I will check the new way of defining template sensors for the Readme though if this new way is recommended by HA (although I dont like the new syntax on the first impression). The Riemann integral integration triggers on state updates, but the powercalc integration has since release v0.30 force updates specified by scan_intervals built in so it regularly updates the state of the powercalc sensors (by default every 10 min). So the Riemann integral integration should work fine now with the powercalc sensors.
Creating the Riemann integration sensors in the background should be possible because the energy dashboard does it already with the energy cost sensors, but I dont know how this works. @bramstroker mentioned he has already an idea and will maybe implement it in a future release.

[OzGav]

I know what you mean! Since I need the extra config optons I have converted all my power related templates only to the new format otherwise I am guaranteed to break something...

I am wondering if the Reiman Integration is not fit for purpose for slowly changing values. I just knocked up the following and it works well. Why not just set up these automations instead of the Rieman Integration? I would have to use the customize functioanlity to add in the add necessary configuartion variables but you have to do that with the Rieman Integration anyway...

input_number:
  main_pc_power_total:
    step: .001
    unit_of_measurement: kWh
    min: 0
    max: 10000

automation:
  - alias: add_up_main_pc_power
    id: add_up_main_pc_power
    trigger:
      - platform: time_pattern
        minutes: "/10"
    action:
      - service: input_number.set_value
        target:
          entity_id: input_number.main_pc_power_total
        data:
          value: "{{ states('input_number.main_pc_power_total') | float + ((states('sensor.mainpc_power_usage') | float / 6) / 1000 )}}"

Edit: This may not work as I am unable to change the necessary configuartion variables through the UI. Will have to try directly in YAML and see if that works...

[NachtaktiverHalbaffe]

@OzGav Whats the benefit of doing it with an automation? You can setup an Blueprint, but since the powercalc integration needs to be setup by yaml, the user needs yaml-skills anyways. I think the Riemann integration works just fine since release v0.30 of powercalc sensor and its also "the recommended way" of the HA devs. Didn't have any sampling problems or something like that. And you can setup the template sensor completely finished so you don't need the customize functionality

[OzGav]

You are right I am drifting. Since you brought to my attention that the Riemann Integration only updates on state change I was thinking about non-powercalc sensors I have made...

[OzGav]

Back on topic though if you do use a Riemann Integration you say that you don't need to customize but the integration doesn't seem to have the ability to set:

    state_class: "measurement"
    device_class: energy
    attributes:
      last_reset: "1970-01-01T00:00:00+00:00"

[NachtaktiverHalbaffe]

Back on topic though if you do use a Riemann Integration you say that you don't need to customize but the integration doesn't seem to have the ability to set:

    state_class: "measurement"
    device_class: energy
    attributes:
      last_reset: "1970-01-01T00:00:00+00:00"

Since release 2021.8 these attributes are automatically added if the underlying sensor is a power sensor (I think Frenck added it to HA Core). So if you created the integration sensor before 2021.8, you have to customize the sensor, but after 2021.8 these attributes are added automatically by the integration if using a power sensor.

[OzGav]

Not my experience because I am just starting to use this now as of 2021.8. Just to be sure I am about to create another one to test what you have been saying so I am about to confirm or deny!

[OzGav]

OK I take it back all the required variables are there. Thanks for that I must be thinking of something else. I still have the problem that I have a few non-powercalc sensors that I need to do something with. I need to nut out the best way to do that.

[bramstroker]

Just a quick headsup. I did a proof of concept to let the powercalc component automatically create integration sensors for measuring kWh. Was a struggle but I have something working now. Will need to do some cleanups, testing and add some configuration options.

I suggest the following default naming convention.
Let's assume you have a light bulb with the name "Livingroom" and the entity_id is light.livingroom.
The component will create the following two sensors in this example:

• sensor.livingroom_power (name "Livingroom power")
• sensor.livingroom_energy (name "Livingroom energy")

The second one being the integration sensor. Energy is a quite common naming also in zwave sensors and other components. But we can also choose _kwh for example or _energy_consumed.

[chilicheech]

For the sensor.livingroom_energy type sensors I've also seen it named sensor.livingroom_consumption

[OzGav]

I think sensor.livingroom_energy is most appropriate. It sits with the convention you have established for power and it aligns with the HA uses of the words https://www.home-assistant.io/docs/energy/faq/

[marithpl]

Just a quick headsup. I did a proof of concept to let the powercalc component automatically create integration sensors for measuring kWh. Was a struggle but I have something working now. Will need to do some cleanups, testing and add some configuration options.

I suggest the following default naming convention.
Let's assume you have a light bulb with the name "Livingroom" and the entity_id is light.livingroom.
The component will create the following two sensors in this example:

• sensor.livingroom_power (name "Livingroom power")
• sensor.livingroom_energy (name "Livingroom energy")

The second one being the integration sensor. Energy is a quite common naming also in zwave sensors and other components. But we can also choose _kwh for example or _energy_consumed.

Sounds great! It will be much easier. I made a integration sensor for each light and template sensor for sum up and seems to work. But Can you explain why data is saved to energy data every 2 hours like on this chart (The grey bar is a total light consumption):
.

[bramstroker]

Just a quick headsup. I did a proof of concept to let the powercalc component automatically create integration sensors for measuring kWh. Was a struggle but I have something working now. Will need to do some cleanups, testing and add some configuration options.
I suggest the following default naming convention.
Let's assume you have a light bulb with the name "Livingroom" and the entity_id is light.livingroom.
The component will create the following two sensors in this example:

• sensor.livingroom_power (name "Livingroom power")
• sensor.livingroom_energy (name "Livingroom energy")

The second one being the integration sensor. Energy is a quite common naming also in zwave sensors and other components. But we can also choose _kwh for example or _energy_consumed.

Sounds great! It will be much easier. I made a integration sensor for each light and template sensor for sum up and seems to work. But Can you explain why data is saved to energy data every 2 hours like on this chart (The grey bar is a total light consumption):
.

I have no clue why this happens. I didn't find time yet to configure the energy dashboard on my production home assistant instance, so I am not sure whether I will be experiencing the same issue. Maybe it's a bug in the energy dashboard.
Maybe @NachtaktiverHalbaffe has an idea why this is happening.

[bramstroker]

I need the opinion of you guys for two questions.
https://www.strawpoll.me/45587615
https://www.strawpoll.me/45587624

Please leave your answer, than I can finish this feature next weekend.

[Leatherface75]

I am using _kwh on my sensors.

[bramstroker]

I am using _kwh on my sensors.

No worries, I'll make it configurable. Looks like it will be _energy by default (looking at the poll), which is also my own preference.

[NachtaktiverHalbaffe]

Just a quick headsup. I did a proof of concept to let the powercalc component automatically create integration sensors for measuring kWh. Was a struggle but I have something working now. Will need to do some cleanups, testing and add some configuration options.
I suggest the following default naming convention.
Let's assume you have a light bulb with the name "Livingroom" and the entity_id is light.livingroom.
The component will create the following two sensors in this example:

• sensor.livingroom_power (name "Livingroom power")
• sensor.livingroom_energy (name "Livingroom energy")

The second one being the integration sensor. Energy is a quite common naming also in zwave sensors and other components. But we can also choose _kwh for example or _energy_consumed.

Sounds great! It will be much easier. I made a integration sensor for each light and template sensor for sum up and seems to work. But Can you explain why data is saved to energy data every 2 hours like on this chart (The grey bar is a total light consumption):
.

@marithpl Sorry I had some days off. In general the energy dashboards have some delay ( in my case it can 10-15 minutes until creating the energy usage entries in the dashboard for the last hour). But when it sums up the energy only each 2 hours, then it seems to be a bug or a misconfiguration because in my configuration everything works fine. Just from the picture of your dashboard I cant figure anything out. Can you provide me with these information:

• The configuration of your powercalc sensor
• The configuration of your energy sensor (if you don't do want to wait until the v0.4 release which automatically creates it for you)
• The configuration of the template sensors which should be your energy group
• The powercalc version you are currently running and some information about the entity for which you created the powercalc sensor

[OzGav]

Looks like it might be worth having a discussion about or allowing the option to specify the integration method. Some related discussion here https://community.home-assistant.io/t/custom-component-iotawatt-energy-monitor-integration/254110/100

[bramstroker]

When I check the documentation on HA site it states the following:

In case you have an appliance which produces spikey consumption (like an on/off electrical boiler) you should opt for the left method to get accurate readings.

This is definitely not the case for the virtual power sensors powercalc creates, they have very static characteristics and not spikey at all. So trapezoid (which is the default) should be the right one to chose according to the HA docs.

I did have. a short read on Riemann Integral sum workings, but this is all highly mathematical stuff (which is not my cup of coffee ;-)).

I am happy to add configuration options for the integration method, but only when it actually solves a problem.

[OzGav]

I will do some testing and see what I come up with using the three methods.

[marithpl]

I've updated to 4.0 power and energy sensors are created perfectly. My YAML is much shorter.
I've have only one question because now sensors count in Wh, when I consume 1000Wh it will change to kWh?

Is it possible to make a value_template automatically add new {}_energy entities?
I'm dummy in templates.

- platform: template
  sensors:
    total_light_consumption:
      friendly_name: "Total Light Consumption"
      unit_of_measurement: kWh
      value_template: >-
        {{states('sensor.light1_energy') | float + [...] + states('sensor.lightN_energy') | float}}
      attribute_templates:
        last_reset: "1970-01-01T00:00:00+00:00"
        state_class: measurement
        device_class: energy
        icon_template: mdi:transmission-tower

[bramstroker]

I've updated to 4.0 power and energy sensors are created perfectly. My YAML is much shorter.
I've have only one question because now sensors count in Wh, when I consume 1000Wh it will change to kWh?

Is it possible to make a value_template automatically add new {}_energy entities?
I'm dummy in templates.

- platform: template
  sensors:
    total_light_consumption:
      friendly_name: "Total Light Consumption"
      unit_of_measurement: kWh
      value_template: >-
        {{states('sensor.light1_energy') | float + [...] + states('sensor.lightN_energy') | float}}
      attribute_templates:
        last_reset: "1970-01-01T00:00:00+00:00"
        state_class: measurement
        device_class: energy
        icon_template: mdi:transmission-tower

Very strange. The energy sensors created by powercalc should definitely be kWh and not Wh. No need to add any template sensors for it. You are sure you did not create a energy sensor with the same name before yourself (with wrong configuration)? Than powercalc is not able to change the unit as it reads the old states of the sensors.

[Kev1000000]

FYI, 4.0 tracks kWh perfectly for me. Awesome work @bramstroker, thanks so much for adding this. I would close this issue, but I'll have you do it when you're satisfied with the performance.

[marithpl]

I've updated to 4.0 power and energy sensors are created perfectly. My YAML is much shorter.
I've have only one question because now sensors count in Wh, when I consume 1000Wh it will change to kWh?
Is it possible to make a value_template automatically add new {}_energy entities?
I'm dummy in templates.

- platform: template
  sensors:
    total_light_consumption:
      friendly_name: "Total Light Consumption"
      unit_of_measurement: kWh
      value_template: >-
        {{states('sensor.light1_energy') | float + [...] + states('sensor.lightN_energy') | float}}
      attribute_templates:
        last_reset: "1970-01-01T00:00:00+00:00"
        state_class: measurement
        device_class: energy
        icon_template: mdi:transmission-tower

Very strange. The energy sensors created by powercalc should definitely be kWh and not Wh. No need to add any template sensors for it. You are sure you did not create a energy sensor with the same name before yourself (with wrong configuration)? Than powercalc is not able to change the unit as it reads the old states of the sensors.

Nice advice. Perhaps I had the same sensor name in the past.
Know I changed a entity name its it seems to work.

[bramstroker]

@marithpl ok nice. Also see https://community.home-assistant.io/t/powercalc-virtual-power-sensors/318515/77?u=bramski. This user had the same problem and figured out a fix.

@Kev1000000 you are welcome. Great it is also working for you.

Closing this issue now