Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Energy dashboard not all graphs are populating with Envoy system #54455

Closed
anachronox99 opened this issue Aug 11, 2021 · 40 comments
Closed

Energy dashboard not all graphs are populating with Envoy system #54455

anachronox99 opened this issue Aug 11, 2021 · 40 comments
Labels
integration: enphase_envoy

Comments

@anachronox99
Copy link

The problem

I have tried to setup the Energy dashboard to work with my Envoy system. However, when doing the configuration, I only get two options to pick from when setting up the Consumption, Return and Solar Panels. Below is an example of the options I see when setting up Consumption.

image

Here's my configuration. Picked what seemed to make sense to me.

image

So my problem is that the Energy dashboard doesn't populate all parts. Solar Production and self-consumed solar energy are empty.
image

image

Have I configured correctly? I'm not sure what else to try.

Thanks

What is version of Home Assistant Core has the issue?

core-2021.8.4

What was the last working version of Home Assistant Core?

new feature not supported in previous version

What type of installation are you running?

Home Assistant Core

Integration causing the issue

envoy

Link to integration documentation on our website

https://www.home-assistant.io/integrations/enphase_envoy/

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response
@probot-home-assistant
Copy link

enphase_envoy documentation
enphase_envoy source
(message by IssueLinks)

@probot-home-assistant
Copy link

Hey there @gtdiehl, mind taking a look at this issue as it has been labeled with an integration (enphase_envoy) you are listed as a code owner for? Thanks!
(message by CodeOwnersMention)

@frenck
Copy link
Member

frenck commented Aug 11, 2021

image

This looks incorrect, using the same sensor for returning to grid & solar, meaning you don't use your solar? Each of those should be a separate entity.

@anachronox99
Copy link
Author

anachronox99 commented Aug 11, 2021 via email

@frenck
Copy link
Member

frenck commented Aug 11, 2021

In that case, I'm not sure what you are asking for?

@dennis07527
Copy link

I think you configured it incorrect. Grid consumption and return to grid must be from a smart electricity meter entity (P1 port). That's not an envoy problem.

@frenck
Copy link
Member

frenck commented Aug 11, 2021

@dennis07527 It is also listed as solar production for the same sensor, which is not correct. The same sensor cannot be measuring solar production and what is going back to the grid. Those are different things.

@dennis07527
Copy link

@frenck I should have mentioned @anachronox99 in my previous post. The energy dashboard is working fine only the topic starter has not al the required sensors I guess.

@gtdiehl
Copy link
Contributor

gtdiehl commented Aug 11, 2021

@dennis07527 Are you looking for more fine-grained energy production details, such as individual inverter data? I'm not sure what the problem is if the dashboard is working now.

@anachronox99
Copy link
Author

Thanks all for your feedback. I think maybe my expectations were wrong. I was thinking that when I configured the Energy tab, and selected the consumption and return fields, that I would be able to pick the correct envoy sensor that represented this. But I only see 'Lifetime Energy Consumption' and 'Lifetime Energy Production' as options to pick, when in reality, I have several more envoy sensors automatically surfaced for me by the Envoy integration in HA.

Here are the sensors that the Envoy integration surfaces:

image
image

(I excluded all the individual sensors for the inverters to reduce the size of the list).

I now realise that none of these sensors actually measure the energy returned to grid. But I could create a new calculated sensor based on the amount currently being produced and consumed. Is this what I need to do? And if I did, how do I surface this so it shows up in the drop down list in the Energy config tab?

Are the other two fields (Grid Consumption and solar production) correctly configured?

thx

@dennis07527
Copy link

@gtdiehl I am not the one with the problem, just trying to help ;)

@dennis07527
Copy link

@anachronox99 This is my setup. Only lifetime envoy data is needed at solar production. It calculates the solar production based on that sensor. The rest of the sensors are from different integration (dsmr reader in my situation).
3D6006B2-E6E2-4A16-AC28-754A795B89FE

@gtdiehl
Copy link
Contributor

gtdiehl commented Aug 11, 2021 •
edited

Thanks all for your feedback. I think maybe my expectations were wrong. I was thinking that when I configured the Energy tab, and selected the consumption and return fields, that I would be able to pick the correct envoy sensor that represented this. But I only see 'Lifetime Energy Consumption' and 'Lifetime Energy Production' as options to pick, when in reality, I have several more envoy sensors automatically surfaced for me by the Envoy integration in HA.

Here are the sensors that the Envoy integration surfaces:

image
image

(I excluded all the individual sensors for the inverters to reduce the size of the list).

I now realise that none of these sensors actually measure the energy returned to grid. But I could create a new calculated sensor based on the amount currently being produced and consumed. Is this what I need to do? And if I did, how do I surface this so it shows up in the drop down list in the Energy config tab?

Are the other two fields (Grid Consumption and solar production) correctly configured?

thx

Envoy's are installed in two different configurations when using Consumption CTs. Depending on if the Envoy is configured as Net or Total Consumption could impact the values you are seeing in Home Assistant. The Envoy backend should actually have both values, and the API retrieving the data should be getting the Total Consumption data. I only mention this just in case for some reason the Envoy backend is not returning what is expected.

You are correct the sensors don't show directly what is being returned to the Grid but by using Production (under Solar Panels) and Consumption data the difference will give you what is being returned to the Grid, if any.

EDIT: I think I get what you are saying Home Assistant needs a third Envoy sensor to show what is being returned to the Grid?

@anachronox99
Copy link
Author

Thanks all.
Yes, it would be nice to have a 3rd envoy sensor showing what is returned to the grid. I guess I can easily create this myself, will need to work out what to do to make sure it shows up as an option from the config screen.

So some questions on the 3 input sensors that are required:
Grid Consumption - is that meant to be a sensor that shows current grid consumption, or total consumed for the day?
Return to Grid - current return levels, or accumulated for the day?
Solar production - I can see that both my and @dennis07527 are using an existing sensor 'envoy lifetime energy production' which is the amount accumulated over all time. I assume that's the correct way to go.

Thanks

@Ragingfire105
Copy link

Yes, The envoy integration lacks a sensor for showing energy returned to the grid.
Screenshot_20210812-000200

@gtdiehl
Copy link
Contributor

gtdiehl commented Aug 12, 2021 •
edited

I don't have access to an Envoy with Consumption and MyEnlighten app. The envoy_reader API and Home Assistant sensor code could be updated, but probably need to test out what data is correct for the Return to Grid value.

I did create a 3rd sensor in my development environment, but not sure if it is working correctly though since I can only see the current data from the Envoy and not the hourly data from MyEnlighten.

If the below works it could be added

template:
  - sensor:
      - name: "Envoy Lifetime Return to Grid"
        state: "{{ states('sensor.envoy_lifetime_energy_production') | int - states('sensor.envoy_lifetime_energy_consumption') | int }}"
        unique_id: "envoy_return_to_grid_lifetime"
        unit_of_measurement: 'Wh'
        icon: mdi:solar-power
        device_class: 'energy'
        state_class: 'measurement'
        attributes:
          last_reset: '1970-01-01T00:00:00+00:00'

Edit: Looking into it a bit more and I don't think the above will work. Might have to come up with a different way of calculating grid export kWh from the data the Envoy returns.

@anachronox99
Copy link
Author

Thanks - will try it. I was wondering why you used the lifetime rather than the current value sensors?

@anachronox99
Copy link
Author

@gtdiehl happy to debug with you if you'd like to try on my system.

@gtdiehl
Copy link
Contributor

gtdiehl commented Aug 13, 2021

@anachronox99 After looking at the above template I think the state is wrong should be consumption minus production
state: "{{ states('sensor.envoy_lifetime_energy_consumption') | int - states('sensor.envoy_lifetime_energy_production') | int }}"

But this still gives us Net Energy in both directions, so it should be filtered even more to only include a negative value. I don't know still looking into this.

Lifetime is being used so the last_reset attribute doesn't have to be messed with and Home Assistant will take care of the hourly calculations. HA keeps track of the difference every hour, at least for the Solar Production and Total Consumption. For Export to Grid values, this is a challenge given the data coming from the Envoy.

@gtdiehl
Copy link
Contributor

gtdiehl commented Aug 13, 2021 •
edited

@anachronox99 I don't have access to an Envoy system that is exporting much to the grid. The below might work. The template only records a value when the Envoy has a Net Consumption of a negative value. The Envoy resets this counter every day, so the Timezone configured in Home Assistant needs to match your Envoy device.

If you don't mind can you try this out? Replace x.x.x.x with the IP Address of your Envoy

sensor:
  - platform: rest
    resource: http://x.x.x.x/production.json
    name: Net Consumption WhToday
    #scan_interval: 90
    #timeout: 60
    value_template: "{{ value_json['consumption'][1].whToday }}"

template:
  - sensor:
      - name: "Envoy Daily Return to Grid"
        state: >
          {% if states("sensor.net_consumption_whToday") | float < 0 %}
            states("sensor.net_consumption_whtoday") | float | abs
          {%- else -%}
            0
          {%- endif %}
        unique_id: "envoy_return_to_grid_daily"
        availability: "{{ states('sensor.net_consumption_whtoday') not in ['unknown', 'unavailable'] }}"
        unit_of_measurement: 'Wh'
        icon: mdi:solar-power
        device_class: 'energy'
        state_class: 'measurement'
        attributes:
          last_reset: "{{ now().replace(hour=0, minute=0, second=0, microsecond=0).isoformat() }}"

I've commented out the timeout and scan_interval attributes, they should only be needed if for some reason Home Assistant is bombarding your Envoy with requests these can tone it down.

@anachronox99
Copy link
Author

OK gave it a go.

Configuration looks like this:
image

Results:
image

It's mid morning here and I am not using very much so should be showing lots of return to the grid.

Here's enlighten's charts.
image
image

And finally production.json.
{"production":[{"type":"inverters","activeCount":26,"readingTime":1628901046,"wNow":5565,"whLifetime":23084998},{"type":"eim","activeCount":1,"measurementType":"production","readingTime":1628901047,"wNow":5486.015,"whLifetime":22779571.109,"varhLeadLifetime":2.136,"varhLagLifetime":8423439.303,"vahLifetime":28575955.136,"rmsCurrent":24.097,"rmsVoltage":685.86,"reactPwr":384.24,"apprntPwr":5508.007,"pwrFactor":1.0,"whToday":13992.109,"whLastSevenDays":217158.109,"vahToday":17147.136,"varhLeadToday":1.136,"varhLagToday":4325.303}],"consumption":[{"type":"eim","activeCount":1,"measurementType":"total-consumption","readingTime":1628901047,"wNow":1480.2,"whLifetime":29811842.658,"varhLeadLifetime":9346640.228,"varhLagLifetime":25514678.637,"vahLifetime":47843069.645,"rmsCurrent":6.233,"rmsVoltage":685.844,"reactPwr":-581.037,"apprntPwr":4274.973,"pwrFactor":0.35,"whToday":5803.658,"whLastSevenDays":224098.658,"vahToday":15765.645,"varhLeadToday":5744.228,"varhLagToday":4805.637},{"type":"eim","activeCount":1,"measurementType":"net-consumption","readingTime":1628901047,"wNow":-4005.815,"whLifetime":20037538.834,"varhLeadLifetime":9346638.092,"varhLagLifetime":17091239.334,"vahLifetime":47843069.645,"rmsCurrent":17.864,"rmsVoltage":685.828,"reactPwr":-196.796,"apprntPwr":4078.189,"pwrFactor":-0.98,"whToday":0,"whLastSevenDays":0,"vahToday":0,"varhLeadToday":0,"varhLagToday":0}],"storage":[{"type":"acb","activeCount":0,"readingTime":0,"wNow":0,"whNow":0,"state":"idle"}]}

So I don't think quite working yet, but happy to keep experimenting.

@gtdiehl
Copy link
Contributor

gtdiehl commented Aug 14, 2021 •
edited

@anachronox99 Thanks for checking it out. Thats strange looking at the net consumption part of the JSON output I see a negative value for wNow but not for the whToday value, it's still zero. Not sure how often the Envoy updates the whToday value.

Here's the section I'm looking at from the JSON output:
{"type":"eim","activeCount":1,"measurementType":"net-consumption","readingTime":1628901047,"wNow":-4005.815,"whLifetime":20037538.834,"varhLeadLifetime":9346638.092,"varhLagLifetime":17091239.334,"vahLifetime":47843069.645,"rmsCurrent":17.864,"rmsVoltage":685.828,"reactPwr":-196.796,"apprntPwr":4078.189,"pwrFactor":-0.98,"whToday":0,"whLastSevenDays":0,"vahToday":0,"varhLeadToday":0,"varhLagToday":0}

EDIT: I thought Enlighten would display the Export to Grid value? I would expect to see Enlighten showing something being returned to the grid?

EDIT: I found this reference that others having the same issue of whToday in net-consumption being zero. Probably a bug in Envoy's firmware. They used this forumula:
total consumption whToday - production whToday. Let me see if I can modify the template for this.

@gtdiehl
Copy link
Contributor

gtdiehl commented Aug 14, 2021

@anachronox99 Here is another stab at it

sensor:
  - platform: rest
    resource: http://x.x.x.x/production.json
    name: Production JSON
    #scan_interval: 90
    #timeout: 60
    value_template: 1
    json_attributes:
      - production
      - consumption

template:
  - sensor:
      - name: "Envoy Daily Return to Grid"
        state: >
          {% set con = state_attr('sensor.production_json', 'consumption')[0]['whToday'] %}
          {% set prd = state_attr('sensor.production_json', 'production')[1]['whToday'] %}
          {% set diff = con  - prd %}

          {% if diff | float < 0 %}
            {{ diff | float | abs }}
          {%- else -%}
            0
          {%- endif %}
        unique_id: "envoy_production_json"
        availability: "{{ states('sensor.production_json') not in ['unknown', 'unavailable'] }}"
        unit_of_measurement: 'Wh'
        icon: mdi:solar-power
        device_class: 'energy'
        state_class: 'measurement'
        attributes:
          last_reset: "{{ now().replace(hour=0, minute=0, second=0, microsecond=0).isoformat() }}"

@anachronox99
Copy link
Author

Sorry - not looking right yet. Still doesn't show anything being returned to the grid.

image

{"production":[{"type":"inverters","activeCount":26,"readingTime":1628911613,"wNow":4781,"whLifetime":23101628},{"type":"eim","activeCount":1,"measurementType":"production","readingTime":1628911721,"wNow":4677.209,"whLifetime":22795653.046,"varhLeadLifetime":2.136,"varhLagLifetime":8424614.772,"vahLifetime":28592086.294,"rmsCurrent":20.087,"rmsVoltage":699.794,"reactPwr":400.602,"apprntPwr":4693.419,"pwrFactor":1.0,"whToday":30074.046,"whLastSevenDays":217224.046,"vahToday":33278.294,"varhLeadToday":1.136,"varhLagToday":5500.772}],"consumption":[{"type":"eim","activeCount":1,"measurementType":"total-consumption","readingTime":1628911721,"wNow":1367.991,"whLifetime":29816478.905,"varhLeadLifetime":9348014.157,"varhLagLifetime":25516511.517,"vahLifetime":47854893.144,"rmsCurrent":5.46,"rmsVoltage":699.813,"reactPwr":-603.466,"apprntPwr":3820.832,"pwrFactor":0.36,"whToday":10439.905,"whLastSevenDays":209561.905,"vahToday":27589.144,"varhLeadToday":7118.157,"varhLagToday":6638.517},{"type":"eim","activeCount":1,"measurementType":"net-consumption","readingTime":1628911721,"wNow":-3309.218,"whLifetime":20037593.749,"varhLeadLifetime":9348012.021,"varhLagLifetime":17091896.745,"vahLifetime":47854893.144,"rmsCurrent":14.627,"rmsVoltage":699.831,"reactPwr":-202.864,"apprntPwr":3413.159,"pwrFactor":-0.97,"whToday":0,"whLastSevenDays":0,"vahToday":0,"varhLeadToday":0,"varhLagToday":0}],"storage":[{"type":"acb","activeCount":0,"readingTime":0,"wNow":0,"whNow":0,"state":"idle"}]}

@gtdiehl
Copy link
Contributor

gtdiehl commented Aug 14, 2021

Maybe leave it running for a few hours. 🤷

Also you can check the sensor.envoy_daily_return_to_grid in the Developer Tools to see if the value is being saved there. Mine is flat because I'm consuming more than I'm producing. But you are the opposite so this value should be increasing

Screen Shot 2021-08-13 at 8 36 10 PM
Screen Shot 2021-08-13 at 8 36 21 PM

@anachronox99
Copy link
Author

ok will leave it a few hours. Here's the value of the sensor.
image

@anachronox99
Copy link
Author

I think that might do it.
image

Look right to you?

@anachronox99
Copy link
Author

Hmm may not be quite right? The return to grid values are showing on the postiive part of the graph this evening. Defintely no solar being generated, so amount returned to grid should be zero.

image

@gtdiehl
Copy link
Contributor

gtdiehl commented Aug 14, 2021

Ok thank you for trying it out. I'll see what the issue is. Maybe in a few days I'll have an idea 😂

@gtdiehl
Copy link
Contributor

gtdiehl commented Aug 14, 2021

Can I see the Return to Grid graph from the Developers Tool?

@anachronox99
Copy link
Author

image

It goes straight down to zero at 2:35am

@gtdiehl
Copy link
Contributor

gtdiehl commented Aug 15, 2021

Thank you. The template is not working as I expected. The graph should be continuing upward until around Midnight, this is about when it should drop to zero due to the counter on the Envoy being reset. But that downward slope before Midnight should not be happening. That's probably why Home Assistant displayed the graphs because the calculations had negative values. I'll go back to the drawing board on this one. Come back in a few days.

@h00ligan
Copy link

I modified the template above with the following logic to prevent the return to grid values from rendering if there isn't any solar production. Will let you know if this helps in the next 24 hours :)

  - name: "Envoy Daily Return to Grid"
    state: >
      {% set con = states('sensor.envoy_today_s_energy_consumption') | float %}
      {% set prd = states('sensor.envoy_today_s_energy_production') | float %}
      {% set diff = prd  - con %}
      {% if states('sensor.envoy_current_power_production') | float > 0 and diff > 0 %}
        {{ diff | abs }}
      {%- else -%}
        0
      {%- endif %}

@anachronox99
Copy link
Author

Thanks - have put that in will leave it running and see how it goes and will post back tomorrow

@h00ligan
Copy link

Just realized this isn't going to work at all. It won't register return to grid values until the daily power production exceeds the daily power consumption. {sigh} back to the drawing board.

@h00ligan
Copy link

h00ligan commented Aug 25, 2021 •
edited

Found this thread that addresses the problem nicely! :)
Essentially you build sensors for the Grid Import and Grid Export power (W) current values, then use the integration platform to calculate the energy (Wh) over time.

https://community.home-assistant.io/t/enhpase-envoy-on-2021-8-with-new-energy-feature/328668

@anachronox99
Copy link
Author

Nice one - have implemented it and will report back.

@anachronox99
Copy link
Author

This is looking good! Thanks @h00ligan
image

@gtdiehl should I close this out or would you like me to keep it open?

@gtdiehl
Copy link
Contributor

gtdiehl commented Sep 4, 2021

@anachronox99 If the configuration fixes your issue I’d say close it.

@anachronox99
Copy link
Author

Yes confirmed all working. Thanks all.

@github-actions github-actions bot locked and limited conversation to collaborators Oct 4, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
integration: enphase_envoy
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@frenck @h00ligan @anachronox99 @gtdiehl @Ragingfire105 @dennis07527