Sensor SNNP

[I-Kuznetsov]

The problem

Snmp sensors don't work after Python 3.12 update.

What version of Home Assistant Core has the issue?

2024.2.1

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant Core

Integration causing the issue

snmp sensor

Link to integration documentation on our website

https://www.home-assistant.io/integrations/snmp/

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

2024-02-09 15:08:20.932 ERROR (MainThread) [homeassistant.config] Platform error: sensor - No module named 'asyncore'
Traceback (most recent call last):
  File "/srv/homeassistant/lib/python3.12/site-packages/homeassistant/config.py", line 1518, in async_process_component_config
    platform = p_integration.get_platform(domain)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/srv/homeassistant/lib/python3.12/site-packages/homeassistant/loader.py", line 847, in get_platform
    cache[full_name] = self._import_platform(platform_name)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/srv/homeassistant/lib/python3.12/site-packages/homeassistant/loader.py", line 864, in _import_platform
    return importlib.import_module(f"{self.pkg_path}.{platform_name}")
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 995, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/srv/homeassistant/lib/python3.12/site-packages/homeassistant/components/snmp/sensor.py", line 8, in <module>
    import pysnmp.hlapi.asyncio as hlapi
  File "/srv/homeassistant/lib/python3.12/site-packages/pysnmp/hlapi/__init__.py", line 10, in <module>
    from pysnmp.hlapi import auth
  File "/srv/homeassistant/lib/python3.12/site-packages/pysnmp/hlapi/auth.py", line 7, in <module>
    from pysnmp.entity import config
  File "/srv/homeassistant/lib/python3.12/site-packages/pysnmp/entity/config.py", line 8, in <module>
    from pysnmp.carrier.asyncore.dgram import udp, udp6, unix
  File "/srv/homeassistant/lib/python3.12/site-packages/pysnmp/carrier/asyncore/dgram/udp.py", line 9, in <module>
    from pysnmp.carrier.asyncore.dgram.base import DgramSocketTransport
  File "/srv/homeassistant/lib/python3.12/site-packages/pysnmp/carrier/asyncore/dgram/base.py", line 10, in <module>
    from pysnmp.carrier.asyncore.base import AbstractSocketTransport
  File "/srv/homeassistant/lib/python3.12/site-packages/pysnmp/carrier/asyncore/base.py", line 9, in <module>
    import asyncore
ModuleNotFoundError: No module named 'asyncore'

Additional information

Installing pyasyncore
python3 -m pip install pyasyncore
hasen't effect

[home-assistant]

snmp documentation
snmp source

[joostlek]

You should use pysnmp-lextudio==5.0.31 instead

[I-Kuznetsov]

Downgrade pysnmp-lextudio from 5.0.34 to 5.0.31 didn't help, same error:
ERROR (MainThread) [homeassistant.config] Platform error: sensor - No module named 'asyncore'
This does not work on RaspberryPI OS 12 (Bookworm), everything works on the same x64 distribution

[joostlek]

In that case I don't know how to fix this, could be that there just are no wheels available for this or something along those lines

[I-Kuznetsov]

When I run HA 2024.1 under Python 3.11, there are no errors, SNMP sensors work again

[joostlek]

Yes, there is no support for pysnmp in python 3.12, there is a switch in libraries for that.

[jbouwh]

This is interesting. I'm not having issues with snmp on python 3.12. What installation type are you using?

[I-Kuznetsov]

Raspberry Pi OS Lite - Debian 12 (bookworm)
Home Assistant Core in a Python venv

[jbouwh]

Raspberry Pi OS Lite - Debian 12 (bookworm) Home Assistant Core in a Python venv

Okay, I can confirm it works okay with HASS OS 11.5, with pysnmp-lextudio==5.0.31

[I-Kuznetsov]

pysnmp-lextudio 5.0.31 works fine on x86-64 system! Doesn't want on Pi OS (arm64)

[I-Kuznetsov]

2024-02-17 10:29:04.310 ERROR (MainThread) [homeassistant.config] Platform error: sensor - No module named 'asyncore'
Home Assistant Core 2024.2.2 - the error is still there.
Asyncore removed in python 3.12 (https://peps.python.org/pep-0594/#asyncore). Replaced by asyncio.
I haven't found a solution yet.

[erikmouw]

The issue is that pyasyncore doesn't seem to install the asyncore.py file.

Here's a workaround when running Hass core in a virtualenv: copy asyncore.py from a Python 3.11 install (on a Raspberry Pi it can be found in /usr/lib/python3.11) to /path/to/your/python-3.12/virtualenv/lib/python3.12/site-packages. Edit the file, around line 62 change:

warnings._deprecated(__name__, _DEPRECATION_MSG, remove=(3, 12))

into

warnings._deprecated(__name__, _DEPRECATION_MSG, remove=(3, 13))

Restart hass, and you'll have your SNMP device tracker back. Dirty, but it works.

[I-Kuznetsov]

Yes, everything worked. Thank you!

[erikmouw]

Come to think of it: the issue is that there appears to be no wheel available for linux_armv7l or manylinux. That has to be fixed upstream, I guess. In the mean time, here is a proper way to fix it:

• First undo the dirty hack by removing that asyncore.py file from /path/to/your/python-3.12/virtualenv/lib/python3.12/site-packages
• Activate your virtualenv: source /path/to/your/python-3.12/virtualenv/bin/activate
• Uninstall pyasyncore: pip uninstall pyasyncore
• Reinstall pyasyncore and tell pip not to download wheels: pip install --no-binary :all: pyasyncore==1.0.3

That way it also works and you don't mess up pip's idea about what's installed.

[erikmouw]

I opened simonrob/pyasyncore#2 upstream.

[joostlek]

Collecting pyasyncore
  Downloading pyasyncore-1.0.3-py3-none-any.whl.metadata (3.4 kB)
Downloading pyasyncore-1.0.3-py3-none-any.whl (10 kB)
Installing collected packages: pyasyncore
Successfully installed pyasyncore-1.0.3

Works on ARM mac

[erikmouw]

The proof is to import asyncore, is that possible om ARM Mac?

[erikmouw]

See simonrob/pyasyncore#2 (comment) and my comment below.

As this issue (and the related ones) only appears to affect Hass core users on Raspberry Pi, I guess those all still have a file /etc/pip.conf with following lines:

[global]
extra-index-url=https://www.piwheels.org/simple

Which tells pip to download wheels from piwheels.org. Apparently piwheels.org rebuild the wheels with a Python version < 3.12, and the setup code for pyasyncore contains code to prevent building on python < 3.12 to avoid messing up the system installed asyncore.

On my Pi, this file was a remnant of the python-pip-whl package, after purging that package using dpkd --purge python-pip-whl (and purging the pip cache) I can install the pyasyncore package without issue.

So, solution can be:

Check if the file is a left over config file:

dpkg -l $(dpkg -S /etc/pip.conf | awk -F: '{ print $1 }')

The output should look like:

Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name           Version          Architecture Description
+++-==============-================-============-=====================================
rc  python-pip-whl 20.3.4-4+deb11u1 all          Python package installer (pip wheels)

If the output starts with rc you can safely remove the package. If the file is not owned by any package, you can just remove the file.

If the file /etc/pip.conf is owned by a package, or you have more customizations in it, remove the line(s):

[global]
extra-index-url=https://www.piwheels.org/simple

[roumano]

Come to think of it: the issue is that there appears to be no wheel available for linux_armv7l or manylinux. That has to be fixed upstream, I guess. In the mean time, here is a proper way to fix it:

* First undo the dirty hack by removing  that `asyncore.py` file from `/path/to/your/python-3.12/virtualenv/lib/python3.12/site-packages`

* Activate your virtualenv: `source /path/to/your/python-3.12/virtualenv/bin/activate`

* Uninstall pyasyncore: `pip uninstall pyasyncore`

* Reinstall pyasyncore and tell pip not to download wheels: `pip install --no-binary :all: pyasyncore==1.0.3`

That way it also works and you don't mess up pip's idea about what's installed.

This workarround fix the issue, i confirm it's working

[bieniu]

I think we should remove pyasn1 from here since the current pysnmp-lextudio library uses version 0.5.1, and for pysnmplib increase the version to 1000000000.0.0 to prevent it from being installed.

core/homeassistant/package_constraints.txt

Lines 162 to 168 in d12335a

	# pyasn1 0.5.0 has breaking changes which cause pysnmplib to fail
	# until they are resolved, we need to pin pyasn1 to 0.4.8 and
	# pysnmplib to 5.0.21 to avoid the issue.
	# https://github.com/pyasn1/pyasn1/pull/30#issuecomment-1517564335
	# https://github.com/pysnmp/pysnmp/issues/51
	pyasn1==0.4.8
	pysnmplib==5.0.21

[erikmouw]

I think we should remove pyasn1 from here since the current pysnmp-lextudio library uses version 0.5.1, and for pysnmplib increase the version to 1000000000.0.0 to prevent it from being installed.

core/homeassistant/package_constraints.txt

Lines 162 to 168 in d12335a

	# pyasn1 0.5.0 has breaking changes which cause pysnmplib to fail
	# until they are resolved, we need to pin pyasn1 to 0.4.8 and
	# pysnmplib to 5.0.21 to avoid the issue.
	# https://github.com/pyasn1/pyasn1/pull/30#issuecomment-1517564335
	# https://github.com/pysnmp/pysnmp/issues/51
	pyasn1==0.4.8
	pysnmplib==5.0.21

I don't see an issue with PySNMP and hass core 2024.2.2, once I had the proper pyasyncore in place SNMP presence detection started working again.

[lextm]

Ultimately you should rewrite this with asyncio, instead of asyncore.

pysnmp-lextudio release 6.0 has removed asyncore related API, so the rewrite is a must, unless you cap the package to just 5.0 release.

[erikmouw]

Hey @bieniu , could you please have a look at #112548 ? It looks like #111193 only made things worse. Downgrading pysnmp-lextudio to 5.0.34 is a verified workaround for now.

[lextm]

Proposed #112795 to resolve this.

[I-Kuznetsov]

After updating version 2024.3.0, the sensors stopped working again. There are no errors in the log.

[spangingpan]

I found this after searching for the problem, I have the same SNMP failures on 2024.3.0, the logs report that SNMP takes too long to run, but the values are stuck. I've reverted to 2024.2.4 and It has started working again.

[sphen13]

I found this after searching for the problem, I have the same SNMP failures on 2024.3.0, the logs report that SNMP takes too long to run, but the values are stuck. I've reverted to 2024.2.4 and It has started working again.

i think maybe we should open up a new issue as it looks like this one is related to snmp not work ing at all when your and mine is related to stuck entries. this started with 2024.3.0 for me - snmp will work for awhile but then stop. i get a 10 sec timeout in the log and then consecutive errors refreshing after that. reboot brings it all back for a period of time.

[erikmouw]

Proposed #112795 to resolve this.

It might work for the sensor part, but it doesn't solve it for the device_tracker, see my comments in #112548 .

So far only pysnmp-lextudio==5.0.34 works properly in hass. Version 6.0.9 might seem to work, but the device_tracker part fails with real network hardware whereas 5.0.34 works just fine. Please revert #112795 .

(network hardware is an old but very reliable HP ProCurve 1800-24G J9028B gigabit ethernet switch)

[jaydenthorup]

I found this after searching for the problem, I have the same SNMP failures on 2024.3.0, the logs report that SNMP takes too long to run, but the values are stuck. I've reverted to 2024.2.4 and It has started working again.

Problems for me too. Revert to 2024.2.4 and I'm just fine again.

[lextm]

Since this issue was originally opened to track the broken sensor feature only and the issue was partially fixed in 2024.3.1 and updated again in #113463, we suggest it should be closed now. As maintainers of pysnmp-lextudio, we have set up the initial test environments to confirm no more work is needed here.

For other problems (mostly around device tracker feature), we have created new issues to track, like #113457 and #113605. Based on our investigation, the necessary fixes are already available and we just wait for #112815 to be merged.

[Theetjuh]

Yesterday 2024.4.2 has been released with major SNMP changes, maybe check if this is still an issue?

[erikmouw]

SNMP device tracker works for me on hass 2024.4.2.