Installed today the 2.93 Update on CCGX and reinstalled dbus-mqtt-device "without" any issue

[Hofyyy]

2.93 CCGX works

[pos-ei-don]

Updating from 2.92 did not work for me.
I got

/service/dbus-mqtt-devices/run
*** starting dbus-mqtt-devices ***
Traceback (most recent call last):
  File "/data/drivers/dbus-mqtt-devices-0.5.1/dbus_mqtt_devices.py", line 34, in <module>
    from device_manager import MQTTDeviceManager
  File "/data/drivers/dbus-mqtt-devices-0.5.1/device_manager.py", line 17, in <module>
    import yaml
ModuleNotFoundError: No module named 'yaml'

reverting to 2.92 with "boot backup" worked as expected. Did not invest more time into debugging...

[Hofyyy]

you need to install the setup script again which will install the yaml modul

[pos-ei-don]

Thank you.. Will try tomorrow.

[stundenblume]

I get the same error on a relative fresh 2.92 RasPi installation.

root@raspberrypi2:/data/drivers# ./dbus-mqtt-devices-0.5.1/bin/setup.sh

Setup dbus-mqtt-devices in /data/drivers/dbus-mqtt-devices-0.5.1 started

Ensure Python's Pip is installed

pip 20.0.2 from /usr/lib/python3.8/site-packages/pip (python 3.8)

Pip install module dependencies

Collecting PyYAML==6.0

  Using cached PyYAML-6.0.tar.gz (124 kB)

  Installing build dependencies ... done

  Getting requirements to build wheel ... done

ERROR: Exception:

Traceback (most recent call last):

  File "/usr/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 186, in _main

    status = self.run(options, args)

  File "/usr/lib/python3.8/site-packages/pip/_internal/commands/install.py", line 331, in run

    resolver.resolve(requirement_set)

  File "/usr/lib/python3.8/site-packages/pip/_internal/legacy_resolve.py", line 177, in resolve

    discovered_reqs.extend(self._resolve_one(requirement_set, req))

  File "/usr/lib/python3.8/site-packages/pip/_internal/legacy_resolve.py", line 333, in _resolve_one

    abstract_dist = self._get_abstract_dist_for(req_to_install)

  File "/usr/lib/python3.8/site-packages/pip/_internal/legacy_resolve.py", line 282, in _get_abstract_dist_for

    abstract_dist = self.preparer.prepare_linked_requirement(req)

  File "/usr/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 515, in prepare_linked_requirement

    abstract_dist = _get_prepared_distribution(

  File "/usr/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 95, in _get_prepared_distribution

    abstract_dist.prepare_distribution_metadata(finder, build_isolation)

  File "/usr/lib/python3.8/site-packages/pip/_internal/distributions/sdist.py", line 38, in prepare_distribution_metadata

    self._setup_isolation(finder)

  File "/usr/lib/python3.8/site-packages/pip/_internal/distributions/sdist.py", line 96, in _setup_isolation

    reqs = backend.get_requires_for_build_wheel()

  File "/usr/lib/python3.8/site-packages/pip/_vendor/pep517/wrappers.py", line 151, in get_requires_for_build_wheel

    return self._call_hook('get_requires_for_build_wheel', {

  File "/usr/lib/python3.8/site-packages/pip/_vendor/pep517/wrappers.py", line 255, in _call_hook

    raise BackendUnavailable(data.get('traceback', ''))

pip._vendor.pep517.wrappers.BackendUnavailable: Traceback (most recent call last):

  File "/usr/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 63, in _build_backend

    obj = import_module(mod_path)

  File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module

    return _bootstrap._gcd_import(name[level:], package, level)

  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import

  File "<frozen importlib._bootstrap>", line 991, in _find_and_load

  File "<frozen importlib._bootstrap>", line 961, in _find_and_load_unlocked

  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed

  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import

  File "<frozen importlib._bootstrap>", line 991, in _find_and_load

  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked

  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked

  File "<frozen importlib._bootstrap_external>", line 843, in exec_module

  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed

  File "/var/volatile/tmp/pip-build-env-lvoroxgj/overlay/lib/python3.8/site-packages/setuptools/__init__.py", line 18, in <module>

    from setuptools.dist import Distribution

  File "/var/volatile/tmp/pip-build-env-lvoroxgj/overlay/lib/python3.8/site-packages/setuptools/dist.py", line 42, in <module>

    from setuptools.config import setupcfg, pyprojecttoml

  File "/var/volatile/tmp/pip-build-env-lvoroxgj/overlay/lib/python3.8/site-packages/setuptools/config/__init__.py", line 10, in <module>

    from . import setupcfg

  File "/var/volatile/tmp/pip-build-env-lvoroxgj/overlay/lib/python3.8/site-packages/setuptools/config/setupcfg.py", line 18, in <module>

    from setuptools.extern.packaging.requirements import Requirement, InvalidRequirement

  File "/var/volatile/tmp/pip-build-env-lvoroxgj/overlay/lib/python3.8/site-packages/setuptools/_vendor/packaging/requirements.py", line 8, in <module>

    from ._parser import parse_requirement

  File "/var/volatile/tmp/pip-build-env-lvoroxgj/overlay/lib/python3.8/site-packages/setuptools/_vendor/packaging/_parser.py", line 10, in <module>

    from ._tokenizer import DEFAULT_RULES, Tokenizer

  File "/var/volatile/tmp/pip-build-env-lvoroxgj/overlay/lib/python3.8/site-packages/setuptools/_vendor/packaging/_tokenizer.py", line 3, in <module>

    from dataclasses import dataclass

ModuleNotFoundError: No module named 'dataclasses'


Set up Victron module libraries

Set up device service to autorun on restart

Setup dbus-mqtt-devices complete

[freakent]

@stundenblume Please can you try typing
python -m pip install dataclasses
from the ssh command line, then re-run the setup?

[stundenblume]

Yes I did that and also the solution from here. Your drive starts normally, but I don't get a device instance.

Right know I setting up a fresh 2.93 test system. I will see how it goes.

[freakent]

dataclasses is supposed to be a standard feature of Python from Python 3.7. I don't know why it is not in Victron's Venus build.

[freakent]

@stundenblume
I think I have found a much better fix. If you have already installed dataclasses via pip then please do this:
python -m pip uninstall dataclasses
then type:
opkg install python3-modules

That should help a lot.

[stundenblume]

Ok, it works.
RasPi 3+
venus-image-raspberrypi2.wic.gz (2.93)

python -m pip install dataclasses
wget -qO- https://files.pythonhosted.org/packages/36/2b/61d51a2c4f25ef062ae3f74576b01638bebad5e045f747ff12643df63844/PyYAML-6.0.tar.gz | tar xvz -C /data/drivers/
cd PyYAML-6.0/
python setup.py --without-libyaml install
mkdir -p /data/drivers
cd /data/drivers
wget -O dbus-mqtt-devices.zip https://github.com/freakent/dbus-mqtt-devices/archive/refs/tags/v0.5.1.zip
unzip dbus-mqtt-devices.zip
./dbus-mqtt-devices-0.5.1/bin/setup.sh
reboot

Where do I get venusOS 3?

[freakent]

I have created a new temporary python-modules branch with the fix in the setup.sh script. It would be good to get feedback if this helps.
https://github.com/freakent/dbus-mqtt-devices/blob/python-modules/bin/setup.sh

[stundenblume]

Tried your new setup script on a fresh install.
Installs without errors but driver does not start. ModuleNotFoundError: No module named 'yaml'

Last login: Thu Jan  1 00:00:49 1970
root@raspberrypi2:~# mkdir -p /data/drivers
root@raspberrypi2:~# cd /data/drivers
root@raspberrypi2:/data/drivers# wget -O dbus-mqtt-devices.zip https://github.com/freakent/dbus-mqtt-devices/archive/refs/tags/v0.5.1.zip
--2023-02-17 16:36:34--  https://github.com/freakent/dbus-mqtt-devices/archive/refs/tags/v0.5.1.zip
Resolving github.com... 140.82.121.3
Connecting to github.com|140.82.121.3|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://codeload.github.com/freakent/dbus-mqtt-devices/zip/refs/tags/v0.5.1 [following]
--2023-02-17 16:36:35--  https://codeload.github.com/freakent/dbus-mqtt-devices/zip/refs/tags/v0.5.1
Resolving codeload.github.com... 140.82.121.10Last login: Thu Jan  1 00:00:49 1970
root@raspberrypi2:~# mkdir -p /data/drivers
root@raspberrypi2:~# cd /data/drivers
root@raspberrypi2:/data/drivers# wget -O dbus-mqtt-devices.zip https://github.com/freakent/dbus-mqtt-devices/archive/refs/tags/v0.5.1.zip
--2023-02-17 16:36:34--  https://github.com/freakent/dbus-mqtt-devices/archive/refs/tags/v0.5.1.zip
Resolving github.com... 140.82.121.3
Connecting to github.com|140.82.121.3|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://codeload.github.com/freakent/dbus-mqtt-devices/zip/refs/tags/v0.5.1 [following]
--2023-02-17 16:36:35--  https://codeload.github.com/freakent/dbus-mqtt-devices/zip/refs/tags/v0.5.1
Resolving codeload.github.com... 140.82.121.10
Connecting to codeload.github.com|140.82.121.10|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/zip]
Saving to: ‘dbus-mqtt-devices.zip’

dbus-mqtt-devices.z     [ <=>                ]  24.85K  --.-KB/s    in 0.01s   

2023-02-17 16:36:35 (2.02 MB/s) - ‘dbus-mqtt-devices.zip’ saved [25445]

root@raspberrypi2:/data/drivers# unzip dbus-mqtt-devices.zip
Archive:  dbus-mqtt-devices.zip
   creating: dbus-mqtt-devices-0.5.1/
  inflating: dbus-mqtt-devices-0.5.1/.gitignore
  inflating: dbus-mqtt-devices-0.5.1/.gitmodules
  inflating: dbus-mqtt-devices-0.5.1/LICENSE
  inflating: dbus-mqtt-devices-0.5.1/README.md
   creating: dbus-mqtt-devices-0.5.1/bin/
   creating: dbus-mqtt-devices-0.5.1/bin/service/
   creating: dbus-mqtt-devices-0.5.1/bin/service/log/
  inflating: dbus-mqtt-devices-0.5.1/bin/service/log/run
  inflating: dbus-mqtt-devices-0.5.1/bin/service/run.tmpl
  inflating: dbus-mqtt-devices-0.5.1/bin/setup-ccgx.sh
  inflating: dbus-mqtt-devices-0.5.1/bin/setup.sh
  inflating: dbus-mqtt-devices-0.5.1/bin/uninstall.sh
  inflating: dbus-mqtt-devices-0.5.1/dbus_mqtt_devices.py
  inflating: dbus-mqtt-devices-0.5.1/device.py
  inflating: dbus-mqtt-devices-0.5.1/device_manager.py
  inflating: dbus-mqtt-devices-0.5.1/device_service.py
  inflating: dbus-mqtt-devices-0.5.1/device_service_config.py
   creating: dbus-mqtt-devices-0.5.1/ext/
   creating: dbus-mqtt-devices-0.5.1/ext/dbus-mqtt/
   creating: dbus-mqtt-devices-0.5.1/ext/velib_python/
  inflating: dbus-mqtt-devices-0.5.1/requirements.txt
  inflating: dbus-mqtt-devices-0.5.1/services.yml
   creating: dbus-mqtt-devices-0.5.1/test-data/
  inflating: dbus-mqtt-devices-0.5.1/test-data/combo-simulator.py
  inflating: dbus-mqtt-devices-0.5.1/test-data/device-status.json
  inflating: dbus-mqtt-devices-0.5.1/test-data/gps-simulator.py
  inflating: dbus-mqtt-devices-0.5.1/test-data/pvinverter-simulator.py
  inflating: dbus-mqtt-devices-0.5.1/test-data/tank-level.json
  inflating: dbus-mqtt-devices-0.5.1/test-data/tank-simulator.py
  inflating: dbus-mqtt-devices-0.5.1/test-data/temperature.json
   creating: dbus-mqtt-devices-0.5.1/tests/
  inflating: dbus-mqtt-devices-0.5.1/tests/test_validation.py

root@raspberrypi2:/data/drivers# nano dbus-mqtt-devices-0.5.1/bin/setup.sh 
root@raspberrypi2:/data/drivers# ./dbus-mqtt-devices-0.5.1/bin/setup.sh
dbus-mqtt-devices: Setup in /data/drivers/dbus-mqtt-devices-0.5.1 started
dbus-mqtt-devices: Checking to see if Python's Pip is installed
/usr/bin/python: No module named pip
Downloading https://updates.victronenergy.com/feeds/venus/release/packages/dunfell/all/Packages.gz.
Updated source 'all'.
Downloading https://updates.victronenergy.com/feeds/venus/release/packages/dunfell/cortexa7hf-neon-vfpv4/Packages.gz.
Updated source 'cortexa7hf-neon-vfpv4'.
Downloading https://updates.victronenergy.com/feeds/venus/release/packages/dunfell/raspberrypi2/Packages.gz.
Updated source 'raspberrypi2'.
Collected errors:
 * opkg_prepare_url_for_install: Couldn't find anything to satisfy 'python-modules'.
dbus-mqtt-devices: Pip install module dependencies
/usr/bin/python: No module named pip
dbus-mqtt-devices: Set up Victron module libraries
dbus-mqtt-devices: Set up device service to autorun on restart
dbus-mqtt-devices: Adding device service to /data/rc.local
dbus-mqtt-devices: Setup complete
root@raspberrypi2:/data/drivers# reboot

Last login: Fri Feb 17 16:39:44 2023
root@raspberrypi2:~# cat /data/rc.local
ln -s /data/drivers/dbus-mqtt-devices-0.5.1/bin/service /service/dbus-mqtt-devices
root@raspberrypi2:~# more /var/log/dbus-mqtt-devices/current
@4000000063efadca2e84514c *** starting dbus-mqtt-devices ***
@4000000063efadcc000af6a4 Traceback (most recent call last):
@4000000063efadcc000b11fc   File "/data/drivers/dbus-mqtt-devices-0.5.1/dbus_mqtt_devices.py", line 34, in <module>
@4000000063efadcc000b2584     from device_manager import MQTTDeviceManager
@4000000063efadcc000b313c   File "/data/drivers/dbus-mqtt-devices-0.5.1/device_manager.py", line 17, in <module>
@4000000063efadcc000b44c4     import yaml
@4000000063efadcc000b4c94 ModuleNotFoundError: No module named 'yaml'

[freakent]

Sorry, my mistake. I made a typo.... python3-modules not python-modules

the line should be
opkg update && opkg install python3-modules python3-pip

I updated this version of the setup script.

[stundenblume]

After editing setup.sh (without reinstalling venusOS) it works as expected.
Thank you very much!

Now I will try the evcharger PR.

[Hofyyy]

Are their any news on this, and could we get a new release?

[stundenblume]

Hi,
you can install it the normal way. The download will contain the working install script. The changes are done in the main branch without updating the release number.
I installed it on two systems without problems.

[freakent]

Sorry guys, I've been extremely busy with commercial projects and getting my own boat ready for the summer. Things should quieten down in a few weeks. I really want to re-work the services.yaml file but I should probably focus on rolling out the install improvements asap. Watch this space.

[Hofyyy]

No problem. its working find with the last version from yesterday with gcxx

[freakent]

Installing PyYaml still crashes my CCGX. The CCGX looks to be running out of memory during the install and just reboots. I tried many different ways to perform the install without any success. I think my CCGX might already be a bit overloaded.

[Hofyyy]

Hmmm, why does it run on my side without any problems?

[freakent]

I have updated an updated version of the setup.sh that should work for both ccgx and Cerbo. I have tested on my CCGX but I don't have my Raspberry Pi with me. Feel free to test it if you are able.
https://github.com/freakent/dbus-mqtt-devices/blob/single-installer/bin/setup.sh

[stundenblume]

Hmmm, why does it run on my side without any problems?

For me too. Installed on a new easySolar without problems.

have updated an updated version of the setup.sh that should work for both ccgx and Cerbo. I have tested on my CCGX but I don't have my Raspberry Pi with me. Feel free to test it if you are able.

I will do on my test system.

[freakent]

I am pretty sure it is memory related. I now have 5 Ve direct devices, 1 multiplus and 3 Ruuvi tags all connected to my CCGX and there isn't a great deal of RAM left. When the amount of free mem falls below a certain level the device reboots automatically. I think it's time to upgrade to a Cerbo!

[freakent]

I am going to close this issue, please feel free to open another when we get the next big Venus OS update.