history-graph: contains entities which were NOT selected for the card #73155

ildar170975 · 2022-06-03T23:28:41Z

Checklist

I have updated to the latest available Home Assistant version.
I have cleared the cache of my browser.
I have tried a different browser to see if it is related to my browser.

Describe the issue you are experiencing

Create a history-graph with TWO entities:

sun.sun (which is supposed to be included in the Recorder)
any other entity (which was excluded from Recorder)

Let's create a sensor w/o a History:

template:
  - sensor:
      - name: service_zero_value
        icon: mdi:null
        state: '{{ "0" }}'

recorder:
  exclude:
    entity_globs:
      - sensor.service_zero_value

The card:

type: history-graph
entities:
  - entity: sun.sun
  - entity: sensor.service_zero_value
hours_to_show: 24

The output:

The card contains TWO same another entities - which were not selected.

Describe the behavior you expected

If some selected entity does not have a history - then it is NOT displayed on the graph.
No any other entities are displayed, except selected entities with a history.

Steps to reproduce the issue

Create a sensor w/o history.
Create a card.

What version of Home Assistant Core has the issue?

HA 2022.6.1

What was the last working version of Home Assistant Core?

No response

In which browser are you experiencing the issue with?

Chrome 102.0.5005.63

Which operating system are you using to run this browser?

Win10x64

State of relevant entities

No response

Problem-relevant frontend configuration

No response

Javascript errors shown in your browser console/inspector

No response

Additional information

No response

The text was updated successfully, but these errors were encountered:

ildar170975 · 2022-06-03T23:41:25Z

I may explain why I find this bug important.
I am using the history-graph with that "zero sensor w/o history" to create a decluttering template for a quick building graphs.
The history-graph in the template has several zero sensors which - if not replaced by non-zero sensors with history - are not displayed on the graph. By default the graph contains ONE non-zero sensor (i.e. "one graph"). You may specify more sensors for the card by input variables to show more sensors on the graph.
Now all my graphs have these 2 excessive sensors - because they are added due to the bug.

bdraco · 2022-06-04T00:22:47Z

Here is what I get with your example card and configuration

ildar170975 · 2022-06-04T00:25:26Z

For me it seems very strange.
I am absolutely sure that right after update to HA 2022.6.1 there were no any excessive entities in the card.
But then - probably after several HA reboots - they appeared.

Here is one more horror picture:

This is my test page. Every card must contain TWO graphs for different binary_sensors (11 cards x 2 sensors).
And almost every card:

either contains ONLY these two same excessive sensors
or contains them additionally to correct sensors.

bdraco · 2022-06-04T00:27:23Z

Can you open the network tab, watch the websocket, and paste the data you get back the from the history query?

ildar170975 · 2022-06-04T00:31:14Z

Can you open the network tab, watch the websocket

Sorry, cannot find this tab.
Here is what I only have:

Using HA in Docker.

bdraco · 2022-06-04T00:34:33Z

Sorry, its the network tab in your browser's inspector. You may have to force reload once its open

ildar170975 · 2022-06-04T00:40:46Z

Cannot find it...

bdraco · 2022-06-04T00:44:43Z

Click the arrows to the right of sources

ildar170975 · 2022-06-04T00:47:17Z

I found some Network view but it does not look like yours (see the post above)

bdraco · 2022-06-04T00:53:47Z

You need to shift+reload to get it to appear

ildar170975 · 2022-06-04T01:03:27Z

While trying to find that Network page I probably made a mistake - I rebooted HA.
After reboot the glitch gone - so there is nothing to trace probably (or may be I am wrong).
I am afraid the glitch may come back.

If we can trace something now - please tell me what to do.
I found the page you described.

bdraco · 2022-06-04T01:15:43Z

If it happens again please get a copy of the websocket response to the history api call

ildar170975 · 2022-06-04T02:02:16Z

Could you tell me how can I do it?

There are many messages.

paste the data you get back the from the history query?

What I did is:

wrote a "history" word into a filter field,
rows are filtered,
some rows are highlighted by green color

Shall I select each found row and save a data below?

bdraco · 2022-06-04T02:04:24Z

You'll see an id in this case 31. If you look through the messages you'll find a response to id 31 which will have the history data.

When you find one that isn't working correctly find the request and then find the response by id

ildar170975 · 2022-06-04T10:49:03Z

Thank you very much for helping!
If the glitch repeats - I will try to post reports.

bdraco · 2022-06-05T00:17:11Z

Can you post the YAML source to the panel you have displayed here #73155

I'm going to try to create similar entities to see if I can make it happen

ildar170975 · 2022-06-05T01:34:11Z

The view is masonry, a bit simplified view is:

- type: vertical-stack
  cards:
    - type: entities
      entities:
        - entity: binary_sensor.test_alert_life360_iphone_5s
          secondary_info: last-changed
        - entity: binary_sensor.test_alert_life360_test_iphone_5s
          secondary_info: last-changed
    - type: history-graph
      hours_to_show: 4
      entities:
        - entity: binary_sensor.test_alert_life360_iphone_5s
        - entity: binary_sensor.test_alert_life360_test_iphone_5s

- type: vertical-stack
  cards:
    - type: entities
      entities:
        - entity: binary_sensor.test_alert_life360_iphone_6
          secondary_info: last-changed
        - entity: binary_sensor.test_alert_life360_test_iphone_6
          secondary_info: last-changed
    - type: history-graph
      hours_to_show: 4
      entities:
        - entity: binary_sensor.test_alert_life360_iphone_6
        - entity: binary_sensor.test_alert_life360_test_iphone_6

- type: vertical-stack
  cards:
    - type: entities
      entities:
        - entity: binary_sensor.test_alert_life360_redmi_7a
          secondary_info: last-changed
        - entity: binary_sensor.test_alert_life360_test_redmi_7a
          secondary_info: last-changed
    - type: history-graph
      hours_to_show: 4
      entities:
        - entity: binary_sensor.test_alert_life360_redmi_7a
        - entity: binary_sensor.test_alert_life360_test_redmi_7a

- type: vertical-stack
  cards:
    - type: entities
      entities:
        - entity: binary_sensor.test_alert_life360_redmi_9a
          secondary_info: last-changed
        - entity: binary_sensor.test_alert_life360_test_redmi_9a
          secondary_info: last-changed
    - type: history-graph
      hours_to_show: 4
      entities:
        - entity: binary_sensor.test_alert_life360_redmi_9a
        - entity: binary_sensor.test_alert_life360_test_redmi_9a

- type: vertical-stack
  cards:
    - type: entities
      entities:
        - entity: binary_sensor.test_alert_life360_phone_galina
          secondary_info: last-changed
        - entity: binary_sensor.test_alert_life360_test_phone_galina
          secondary_info: last-changed
    - type: history-graph
      hours_to_show: 4
      entities:
        - entity: binary_sensor.test_alert_life360_phone_galina
        - entity: binary_sensor.test_alert_life360_test_phone_galina

- type: vertical-stack
  cards:
    - type: entities
      entities:
        - entity: binary_sensor.test_alert_life360_phone_boris
          secondary_info: last-changed
        - entity: binary_sensor.test_alert_life360_test_phone_boris
          secondary_info: last-changed
    - type: history-graph
      hours_to_show: 4
      entities:
        - entity: binary_sensor.test_alert_life360_phone_boris
        - entity: binary_sensor.test_alert_life360_test_phone_boris

- type: vertical-stack
  cards:
    - type: entities
      entities:
        - entity: binary_sensor.test_alert_life360_ipad_air_2
          secondary_info: last-changed
        - entity: binary_sensor.test_alert_life360_test_ipad_air_2
          secondary_info: last-changed
    - type: history-graph
      hours_to_show: 4
      entities:
        - entity: binary_sensor.test_alert_life360_ipad_air_2
        - entity: binary_sensor.test_alert_life360_test_ipad_air_2

- type: vertical-stack
  cards:
    - type: entities
      entities:
        - entity: binary_sensor.test_alert_life360_ipad_mini_2
          secondary_info: last-changed
        - entity: binary_sensor.test_alert_life360_test_ipad_mini_2
          secondary_info: last-changed
    - type: history-graph
      hours_to_show: 4
      entities:
        - entity: binary_sensor.test_alert_life360_ipad_mini_2
        - entity: binary_sensor.test_alert_life360_test_ipad_mini_2

- type: vertical-stack
  cards:
    - type: entities
      entities:
        - entity: binary_sensor.test_alert_life360_samsung_p6800
          secondary_info: last-changed
        - entity: binary_sensor.test_alert_life360_test_samsung_p6800
          secondary_info: last-changed
    - type: history-graph
      hours_to_show: 4
      entities:
        - entity: binary_sensor.test_alert_life360_samsung_p6800
        - entity: binary_sensor.test_alert_life360_test_samsung_p6800

To create dummy sensors, use this code generator (insert it into a "template" HA page, it will generate a code - copy/paste it into some yaml file):

{% set SETTINGS = [
  ("iphone_5s"),
  ("iphone_6"),
  ("redmi_7a"),
  ("redmi_9a"),
  ("phone_galina"),
  ("phone_boris"),
  ("ipad_air_2"),
  ("ipad_mini_2"),
  ("samsung_p6800")
] %}

{% set code_alerts = '
      - name: test_alert_life360_{0}
        state: "on"

      - name: test_alert_life360_test_{0}
        state: "off"
' %}

template:
  - binary_sensor:
{% for object in SETTINGS -%}
  {{ code_alerts.format(object) }}
{%- endfor %}

NO custom cards are needed.

bdraco · 2022-06-05T01:41:08Z

Do you have a minimal replication case that doesn't require custom cards? I can't be sure I'm looking at the right issue unless we can reduce it down to the most minimal reproducible case.

ildar170975 · 2022-06-05T01:53:20Z

Yes, I changed a code.
There are 9 similar cards, paste them into an empty view.

The code generator is edited too since there is no need in sensor entities now.

bdraco · 2022-06-05T05:16:28Z

I've got your test data up and running and I'm going to try it with every supported database

bdraco · 2022-06-05T05:19:47Z

MariaDB
MySQL
PostgreSQL
SQLite

bdraco · 2022-06-05T05:20:43Z

Its working as expected on all 4 DBMs, but I'll leave it turn in a tab in case something goes wrong and I can catch it

bdraco · 2022-06-05T05:22:40Z

It's also possible I may have fixed this in one of the PRs that hasn't made it into a public release yet

ildar170975 · 2022-06-05T09:19:20Z

Thank you for testing! As I said before, the glitch once appeared, then it was occurring stable, then it disappeared after HA restart (w/o HA updates).
I am afraid it could be some erratic thing.

More facts:

The glitch was ALWAYS occurring with these conditions - i.e. when one sensor w/o history is added:

type: history-graph
entities:
  - entity: sun.sun
  - entity: sensor.service_zero_value
hours_to_show: 24

With this code the glitch did NOT occur:

type: history-graph
entities:
  - entity: sun.sun
hours_to_show: 24

That "horror page" did NOT contain sensors w/o history - so it was strange for me how it became "contaminated".

bdraco · 2022-06-05T16:59:44Z

This might be caused by home-assistant/frontend#12878 so let's wait to see if it happens again after that one gets released

lmagyar · 2022-06-07T02:42:49Z

Isn't it related to #72991 ?

One of the "failure modes" when history graph receives wrong data from the database? After restart bug disappears?

bdraco · 2022-06-07T03:30:50Z

This sure looks like it has the same root cause as #72991

probot-home-assistant · 2022-06-07T05:27:49Z

recorder documentation
recorder source
_{^{(message by IssueLinks)}}

probot-home-assistant · 2022-06-07T05:27:52Z

Hey there @home-assistant/core, mind taking a look at this issue as it has been labeled with an integration (recorder) you are listed as a code owner for? Thanks!
_{^{(message by CodeOwnersMention)}}

ildar170975 · 2022-06-07T10:48:09Z

After restart bug disappears?

Yes, I have not observed it so far.
P.S. Made 2-3 restarts since that incident.
HA in Docker, native mysql.

ildar170975 mentioned this issue Jun 4, 2022

Logbook card: what happens when a sensor with UoM is selected #73044

Closed

3 tasks

bdraco transferred this issue from home-assistant/frontend Jun 7, 2022

bdraco added the integration: recorder label Jun 7, 2022

bdraco mentioned this issue Jun 7, 2022

Remove sqlalchemy lambda_stmt usage from history, logbook, and statistics #73191

Merged

22 tasks

balloob closed this as completed in #73191 Jun 7, 2022

github-actions bot locked and limited conversation to collaborators Jul 8, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

history-graph: contains entities which were NOT selected for the card #73155

history-graph: contains entities which were NOT selected for the card #73155

ildar170975 commented Jun 3, 2022 •

edited by bdraco

ildar170975 commented Jun 3, 2022 •

edited

bdraco commented Jun 4, 2022

ildar170975 commented Jun 4, 2022 •

edited

bdraco commented Jun 4, 2022

ildar170975 commented Jun 4, 2022

bdraco commented Jun 4, 2022

ildar170975 commented Jun 4, 2022 •

edited

bdraco commented Jun 4, 2022

ildar170975 commented Jun 4, 2022

bdraco commented Jun 4, 2022

ildar170975 commented Jun 4, 2022 •

edited

bdraco commented Jun 4, 2022

ildar170975 commented Jun 4, 2022

bdraco commented Jun 4, 2022

ildar170975 commented Jun 4, 2022

bdraco commented Jun 5, 2022

ildar170975 commented Jun 5, 2022 •

edited

bdraco commented Jun 5, 2022

ildar170975 commented Jun 5, 2022 •

edited

bdraco commented Jun 5, 2022

bdraco commented Jun 5, 2022 •

edited

bdraco commented Jun 5, 2022

bdraco commented Jun 5, 2022

ildar170975 commented Jun 5, 2022 •

edited

bdraco commented Jun 5, 2022

lmagyar commented Jun 7, 2022

bdraco commented Jun 7, 2022

probot-home-assistant bot commented Jun 7, 2022

probot-home-assistant bot commented Jun 7, 2022

ildar170975 commented Jun 7, 2022

history-graph: contains entities which were NOT selected for the card #73155

history-graph: contains entities which were NOT selected for the card #73155

Comments

ildar170975 commented Jun 3, 2022 • edited by bdraco

Checklist

Describe the issue you are experiencing

Describe the behavior you expected

Steps to reproduce the issue

What version of Home Assistant Core has the issue?

What was the last working version of Home Assistant Core?

In which browser are you experiencing the issue with?

Which operating system are you using to run this browser?

State of relevant entities

Problem-relevant frontend configuration

Javascript errors shown in your browser console/inspector

Additional information

ildar170975 commented Jun 3, 2022 • edited

bdraco commented Jun 4, 2022

ildar170975 commented Jun 4, 2022 • edited

bdraco commented Jun 4, 2022

ildar170975 commented Jun 4, 2022

bdraco commented Jun 4, 2022

ildar170975 commented Jun 4, 2022 • edited

bdraco commented Jun 4, 2022

ildar170975 commented Jun 4, 2022

bdraco commented Jun 4, 2022

ildar170975 commented Jun 4, 2022 • edited

bdraco commented Jun 4, 2022

ildar170975 commented Jun 4, 2022

bdraco commented Jun 4, 2022

ildar170975 commented Jun 4, 2022

bdraco commented Jun 5, 2022

ildar170975 commented Jun 5, 2022 • edited

bdraco commented Jun 5, 2022

ildar170975 commented Jun 5, 2022 • edited

bdraco commented Jun 5, 2022

bdraco commented Jun 5, 2022 • edited

bdraco commented Jun 5, 2022

bdraco commented Jun 5, 2022

ildar170975 commented Jun 5, 2022 • edited

bdraco commented Jun 5, 2022

lmagyar commented Jun 7, 2022

bdraco commented Jun 7, 2022

probot-home-assistant bot commented Jun 7, 2022

probot-home-assistant bot commented Jun 7, 2022

ildar170975 commented Jun 7, 2022

ildar170975 commented Jun 3, 2022 •

edited by bdraco

ildar170975 commented Jun 3, 2022 •

edited

ildar170975 commented Jun 4, 2022 •

edited

ildar170975 commented Jun 4, 2022 •

edited

ildar170975 commented Jun 4, 2022 •

edited

ildar170975 commented Jun 5, 2022 •

edited

ildar170975 commented Jun 5, 2022 •

edited

bdraco commented Jun 5, 2022 •

edited

ildar170975 commented Jun 5, 2022 •

edited