Huawei Kiosk invalid data, positive glitch.

[AndersHoglund]

Last night Huawei Kiosk presented a new type of invalid data, a positive glitch. We have seen negative glitches every night before, this has been handled and filtered out from HA. See attach history graphs where the new glitch is marked in red. Handled glitches are marked green. Another problem is that the fix for negative glitches causes the positive glitch to become the new cumulative energy level, until production catches up. Energy statistics tab looks like crap.

N.B. This was observed with the old kiosk implementation, but it should affect this new implementation as well.
Not sure how to fix/handle this new behaviour. Other than trying Open API or MODBUS/TCP, leaving this crap kiosk data behind. Question is if this is a kiosk only problem, or if it is the same on one or all of the other ways?
/A

[AndersHoglund]

Same thing happened again last night. I am testing a fix where we do not accept any new life time energy data if current power is zero. ie:

                power = self.coordinator.data[self._kioskId][ATTR_DATA_REALKPI][ATTR_REALTIME_POWER]

                if (power) == 0:
                    _LOGGER.debug(f'{self.entity_id}: new value ({new_value}) is discarded if no power is reported ({entity.state}), so not updating.')
                    return float(current_value)

[AndersHoglund]

Seems to work. No bad bars on the energy page this morning.
But then there are no positive glitch anymore when looking at the "Current Year Energy" data. Has Huawei fixed this in their end at last ? Might not need to do any PR for this. Well, just have to wait and see a bit more...
/A

[AndersHoglund]

Nah, that fix does not work. Positive glitch again this night.
But I can not really understand why it does not work? Is entity ATTR_REALTIME_POWER not available when handling life time energy? Puzzled...
/A

[tijsverkoyen]

I think Huawei is trying to minimise the impact of the kiosks on their systems. So probably some caching that is missing while they are recalculating the metrics?

[AndersHoglund]

Yeah, some caching or time shift problem. SolarEdge cloud have similar problems, either NULL data returned or Lifetime energy lower than current year energy. They are doing nothing to fix that. Moved to Solaredge local MODBUS/TCP access instead, works OK. Will try to do the same with Huawei, once installer password is reset so I can get in there.
Until then, I will try my old fix in the api layer, modified to check for zero power and reject any new energy data then. OpenAPI might be a way fwd too, unless those glitches appear there too?
What a mess.....
/A

[tijsverkoyen]

I'm using the integration with an OpenAPI account. I don't see any glitches

[AndersHoglund]

The fix above do work, if correct data type is used.

                if (power) == '0.00':
                    _LOGGER.debug(f'{self.entity_id}: new value ({new_value}) is discarded if no power is reported ({entity.state}), so not updating.')
                    return float(current_value)

/A

[tijsverkoyen]

Will you create a PR?

[AndersHoglund]

I have not yet moved over to this new kiosk implementation, experimenting and testing on the old one. I will PR the old one and if you want you can copy it here.

[tijsverkoyen]

See tijsverkoyen/Home-Assistant-FusionSolar-Kiosk#42

[tallbacksmannen]

The fix above do work, if correct data type is used.

                if (power) == '0.00':
                    _LOGGER.debug(f'{self.entity_id}: new value ({new_value}) is discarded if no power is reported ({entity.state}), so not updating.')
                    return float(current_value)

/A

I've got the same problem as you with value zero glitches. I'm using the kiosk implementation as well. Where do I enter the code?

[AndersHoglund]

The fix above do work, if correct data type is used.

                if (power) == '0.00':
                    _LOGGER.debug(f'{self.entity_id}: new value ({new_value}) is discarded if no power is reported ({entity.state}), so not updating.')
                    return float(current_value)

/A

I've got the same problem as you with value zero glitches. I'm using the kiosk implementation as well. Where do I enter the code?

Use the older Kiosk implementation, https://github.com/tijsverkoyen/Home-Assistant-FusionSolar-Kiosk
Not sure why there are two implementations, one with the fix and one without.
Or convert to OpenAPI supported by this one.
/A

[tijsverkoyen]

You should not use the older Kiosk implementation. I will not provide any support for that on.

[tijsverkoyen]

Should be fixed in #46
I will release a new version in a few moments.

[tallbacksmannen]

You should not use the older Kiosk implementation. I will not provide any support for that on.

Thank you. I've updated it now and hope it works.

[tijsverkoyen]

Let me know. Because I don't experience these glitches. So it is really difficult for me to test

[tijsverkoyen]

Stupid me: see #46 (comment)

[tijsverkoyen]

PR: #48

[tallbacksmannen]

Let me know. Because I don't experience these glitches. So it is really difficult for me to test

The problem remains. This is how both the Energy Tab and the Database looks like when the "glitch" happend. First the sensors become unavalible and then lifetime energy becomes 0. I guess Home Assistant calculates the difference from last hour of lifetime energy and then displays it in the graph. Since the difference is 0 and 10000 the graph will look bad. I looked at the graph the hour before and there was no problem.

[tijsverkoyen]

Yeah, sorry. I has implemented it wrong. I releases 2.3.3 yesterday. Could you recheck with that version

[tallbacksmannen]

Still the same problem. First the sensors become unavalible and then zero value in total_lifetime_energy.

I think that "unavalible" and "0" should not be stored in the database. Or if you only store values in total_lifetime_energy that is the same or bigger than previous value.

[AndersHoglund]

As I said in PR #48, those lines in custom_components/fusion_solar/fusion_solar/energy_sensor.py should be removed:

           if not isfloat(new_value):
                   _LOGGER.warning(f'{self.entity_id}: new value ({new_value}) is not a float, so not updating.')
                   return float(current_value)

               if not isfloat(current_value):
                   _LOGGER.warning(f'{self.entity_id}: current value ({current_value}) is not a float, send 0.')
                   return 0

               if float(new_value) < float(current_value):
                   _LOGGER.debug(
                       f'{self.entity_id}: new value ({new_value}) is smaller then current value ({entity.state}), so not updating.')
                   return float(current_value)

Open in file editor and do it yourself.
/A

[tallbacksmannen]

As I said in PR #48, those lines in custom_components/fusion_solar/fusion_solar/energy_sensor.py should be removed:

           if not isfloat(new_value):
                   _LOGGER.warning(f'{self.entity_id}: new value ({new_value}) is not a float, so not updating.')
                   return float(current_value)

               if not isfloat(current_value):
                   _LOGGER.warning(f'{self.entity_id}: current value ({current_value}) is not a float, send 0.')
                   return 0

               if float(new_value) < float(current_value):
                   _LOGGER.debug(
                       f'{self.entity_id}: new value ({new_value}) is smaller then current value ({entity.state}), so not updating.')
                   return float(current_value)

Open in file editor and do it yourself. /A

I removed the lines and it has been stable since. Is this fix implemented correctly in the newer versions?

[AndersHoglund]

Yes, PR #52 was merged and included in release version 2.3.4
Case closed.
/A