Component error: modbus - Exception importing homeassistant.components.modbus

[cmatte]

The problem

Adding ModBus integration does not succeed and triggers the following error, blocking HA from restarting and loading it.
Component error: modbus - Exception importing homeassistant.components.modbus

Attempted to, with the same result:

1. remove the TCP part and only leave the serial connector
2. remove anything else but the simple modbus: line

Thanks!

What version of Home Assistant Core has the issue?

2023.1.4

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

modbus

Link to integration documentation on our website

https://www.home-assistant.io/integrations/modbus

Diagnostics information

N/A

Example YAML snippet

modbus:
  - type: tcp
    name: ew11
    host: 192.168.4.39
    port: 502

Anything in the logs that might be useful for us?

Logger: homeassistant.loader
Source: components/modbus/modbus.py:10
First occurred: 12:03:35 (17 occurrences)
Last logged: 13:18:33

Unexpected exception importing component homeassistant.components.modbus
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/loader.py", line 725, in get_component
    cache[self.domain] = importlib.import_module(self.pkg_path)
  File "/usr/local/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/usr/src/homeassistant/homeassistant/components/modbus/__init__.py", line 117, in <module>
    from .modbus import ModbusHub, async_modbus_setup
  File "/usr/src/homeassistant/homeassistant/components/modbus/modbus.py", line 10, in <module>
    from pymodbus.client.sync import (
  File "/usr/local/lib/python3.10/site-packages/pymodbus/client/sync.py", line 180, in <module>
    class ModbusTcpClient(BaseModbusClient):
  File "/usr/local/lib/python3.10/site-packages/pymodbus/client/sync.py", line 184, in ModbusTcpClient
    def __init__(self, host='127.0.0.1', port=Defaults.Port,
AttributeError: type object 'Defaults' has no attribute 'Port'

Additional information

Modbus does not load regardless of the configuration parameters, which seems to be the main issue. The connection to the host that is being attempted, shall be working as per the previous experience and settings shared here . I'd assume the integration is not designed to hard fail when unable to connect to the host.

Useful to know, yet not too sure of the interactions (expecting none), is that the Huawei Solar HACS integration is running and is connecting through ModBus too, so the the issue could be:

1. Default.Port not translating to a number port to bind upon 127.0.0.1,
2. the host to connect to does not get fed into modbus initialization (127.0.0.1) and so is for the port,
3. the default port to bind to might, for instance, be used by this third party integration and the official integration could require a setting to listen upon another port (not 502) to function, too, and establish a connection to this second host
4. anything else :)

[home-assistant]

Hey there @adamchengtkc, @janiversen, @vzahradnik, mind taking a look at this issue as it has been labeled with an integration (modbus) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of modbus can trigger bot actions by commenting:

• @home-assistant close Closes the issue.
• @home-assistant rename Awesome new title Change the title of the issue.
• @home-assistant reopen Reopen the issue.
• @home-assistant unassign modbus Removes the current integration label and assignees on the issue, add the integration domain after the command.

_{^{(message by CodeOwnersMention)}}

---

modbus documentation
modbus source
_{^{(message by IssueLinks)}}

[elupus]

It does look like the pymodbus library is some weird version. Or something is breaking that library. I didnt see anything strange with Huawei Solar.

Do you have anything else using modbus?

[cmatte]

I do not have anything else that could be using modbus, in fact though, I did try to disable Huawei Solar, reboot HA, and the modbus integration does operate normally, so I assume that is a clear culprit somehow. Upon attempting to re enable Huawei Solar, the following error triggered.

What is the best next step here? Pinging @wlcrs for any comments you might have here, I imagine the official guidance is to close this and move the issue in the Huawei Solar repository, I'm going to be standing by for guidance, thanks in advance!

`Logger: homeassistant.setup
Source: setup.py:210
First occurred: 19:39:23 (1 occurrences)
Last logged: 19:39:23

Setup failed for custom integration huawei_solar: Unable to import component: cannot import name 'AsyncModbusSerialClient' from 'pymodbus.client' (/usr/local/lib/python3.10/site-packages/pymodbus/client/init.py)`

[elupus]

Yes ut seems it pulls in a later version of the lib. Which is strange since it mentions same version as core as supported.

[elupus]

Found it. Huawei solar dev version uses 3.0.0 version of pymodbus. You must use main version to be compatible with core.

Would be good to bump modbus to 3.0.0 in core and support that version of pymodbus instead.

[wlcrs]

The maintainer of pyModbus @janiversen has already signalled to me that he would upgrade the Modbus integration to pyModbus 3.x.x .

The huawei_solar beta version already uses this new pyModbus version, and will stay in beta until the version conflict with pyModbus in core is resolved.

[janiversen]

This is not a modbus integration problem, you use a custom component that uses another version of pymodbus (which are known to have a couple of serious problems).

[cmatte]

Thanks everyone, moving back to stable for Huawei Solar has enabled both integrations to work as expected, even though that means losing many Huawei Solar great new developments in the dev branch! This seemingly would still allow me to change the max battery grid charge for the Huawei inverter as I was planning to, dynamically with the second older SMA modbus inverter production yields, so no issues my end closing this and waiting for further developments. @janiversen can you help us all understand, though, does the serious problems you mention hinder a realistic possibility for modbus in core to be utilizing 3.1.0 pymodbus anytime soon? What are those issues can you expand, just so @wlcrs can consider them given Huawei Solar has been working well with it since a while, including expecting alignment as per the above, in order to promote a dev branch to stable? Last reflection to share, looking at the repo, only pymodbus 3.x supports Python 3.10 whilst 2.x supports until 3.7, and I see from the logs that HASSOS utilizes Python 3.10. It would seem a requirement that modbus aligns, as we would otherwise have a core integration not compatible officially with the python HASSOS version, whereby not even tests are run against python 3.10 based on pymodbus 2.x repo, in turn meaning even if we are functional now, this can break the whole HASSOS at any time, i.e., from what we understood from this finding here, the whole system might not even start/reboot!

[janiversen]

I happen to maintain the pymodbus library as well, that should be enough reason ! you are not reading the README correctly. v2.5.3 run nicely on python version to at least 3.10, but v3+ only runs on python version 3.8 and higher.

Please do not forget the custom integration do not use e.g. the climate entity, the modbus integration is general and needs to support many different use cases.

As a rule, core do not adapt to custom integrations, it would be an endless loop (one custom integration uses a new version, while another uses an older version). The recommendation to custom integrations is to:

a) Use the integration in core, by importing and calling the methods. That way the custom integration is independent of library changes.
b) use the same version as the core integration, and upgrade when the core integration is upgraded (this might actually be the most efficient way, but will cause conflicts for a while every time the version is updated).

I am planning a series of updates to both pymodbus and the modbus integration over the next months, so unless the custom integration upgrades with the same tact there will be instability.

I have no idea why the Huawai solar integration was updated to use a new (and for our use, untested) version. I have a Huawei SUN2000 Tkl, and it works fine with the modbus integration, but I agree the that the custom integration provides some niceties.

So in short, because this custom integration have chosen not to use the same library version as core, the users face a problem, but please do not make it my problem, the modbus integration runs stable and that is my major goal.

I am in the final rounds of testing a new library version (3.1.0)....but if core starts using that, you will see the opposite conflict, because then there will still be a version conflict.

[janiversen]

In reality I am not sure I understand why the custom integration choose solely to use the new version. I have a version of the modbus integration that works on both library version, which I am using to test. All differences is isolated in one .py file. At least for the mudbus integration there are no functional advantages in using pymodbus 3.1.0 (but when changing to async there will be).

[cmatte]

Thanks Jan for the extensive explanation, much appreciated and in alignment, also it's great to get to know you're indeed maintaining both the library and integration, I didn't spot that and in fact it tells the fuller story on its own ☺️, but also great to know you're already advancing internal tests for the next version implementation in the following months.
Leaving the interesting reflections with you Thijs @wlcrs on whether it would make sense to follow the guidelines above on utilizing for the time being the older pymodbus as Jan says it could be a pretty quick retrofit, or let the pymodbus and modbus development times and steps in core decide when to make the current Huawei Solar dev version stable, if I'm not wrong there's some async use attempts you were doing or already achieved that might be a good reason to wait and progress on the dev branch front only for a while.

[wlcrs]

I clearly misinterpreted the intended timeline when Jan said on October 30th in issue pymodbus-dev/pymodbus#1146 (comment) that "HomeAssistant modbus integration is also due to be bumped to use the new library.".

As stated by Jan in this issue: keeping library versions in sync between integrations is impossible, be it core or non-core integrations. I will not attempt to keep compatibility by doing retrofits etc. The current dev/beta-version of the huawei_solar will be promoted to stable once the pymodbus-dependency of the modbus-component in core is upgraded to 3.x.x .

[janiversen]

In October I planned to use pymodbus v3.0.2, but then we found a serious problem in the serial part, which meant the modbus integration could not use v3.0.2.

I opened a PR in the asyncio-pyserial repo but that was not responded to (and still not is) so we had to go down another route, all of this causing delays, and now finally the modbus integration is using v3.1.0 on dev, but I suspect the custom integration needs some smaller update due to some api differences between v3.0.2 and v3.1.0.

[jfbuggen]

Hello, I wanted to install the Huawei Solar integration for the first time. I'm installing it via HACS but when I try to configure it I get the error "the configuration flow could not be charged {"message":"Invalid handler **specified"}" (note: I have it in French so the exact wording may differ).

Looking at the logfile, this relates to a issue with the pymodbus library.

Logger: homeassistant.config_entries
Source: config_entries.py:852
Error occurred loading configuration flow for integration huawei_solar: cannot import name 'AsyncModbusSerialClient' from 'pymodbus.client' (/usr/local/lib/python3.10/site-packages/pymodbus/client/init.py)

I'm using HA 2023.1.7. I tried the integration version 1.2.0 and (after de-installing and rebooting) version 1.1.6. Both give the error.
I'm not sure to understand the discussion above on the various versions of the pymodbus library.

What should I do to be able to install this integration? Alternatively, instead of going via the configuration UI, is there a sample yaml that I could copy and update manually to get it running?

Many thanks for your help

[janiversen]

We do not support custom integration, they are as the name says “custom” !

Please deinstall the custom integration and see if the core modbus integration works, if not then it is a problem we will look into.

Remark: this issue is closed, for the same reason, so please if you have problems with the core modbus integration please open a new issue.