Repository navigation
Any experience with installing dsmr6 without cointainer (on plain debian)? Question to users :) #2123
|
Dear Users, Does any of you managed to install dsmr-reader 6 on plain debian - without containers (docker, podman). I'm running dsmr v5.12 on proxmox (n150 cpu, 2 cores, 1 GB ram) and it uses 10% CPU. Thanks a lot! |
Replies: 7 comments 12 replies
|
You can also try to host a v5.12 container as well to compare. Since this weekend there are separate containers tags for the final v5 release ( The v5 container is almost compatible with envvars, but not 100% sure. Easy to revert anyway. Also make sure to check the Python version used, as it may affect comparison. The v6 container has Python 3.14, not sure what the latest v5 has (or your server). |
|
All DSMRs running on proxmox VMs (n150 cpu, 2 cores, 1 GB ram) |
|
It's a bit hard to compare, but on RaspberryPi 4 & 5 I see very low usage in general. These run on Podman. Pi 4
Pi 5
|
|
Maybe @xirixiz has some tips you can use to find any causes. Also, not sure if containers inside VM's suffer from an additional penalty. I don't know enough of this topic. |
|
@sts-ux regarding the comparison without containers, if you still have the old setup running, try to disable the datalogger in Supervisor. You can remove this file: And then reload: It should now be similar to the container without datalogger process. |
|
@dennissiemensma |
|
CPU usage actually decreased in general since the release of v6 as we've spend some effort in reducing CPU usage: #2122 (reply in thread) That fact doesn't solve the problem for you ofcourse, so my feeling is that it has something todo with you host system config. Do you also run other containers with podman? What also could be an issue is using an outdated postgresql backend. Which version does your PG server run on? Also please share your compose file content, or podman run content, maybe I can identify a missing or invalid setting. Also please share the output of this (hit c on your keyboard show it provides more info): Which podman version are you using? Are you running Podman rootless? Maybe some good news on the side: |
|
Dear, Thanks for the reply. My VMs are dedicated for single DSMR (v5, v6). When I close dockers: Compose files: Output of Output of Please let me know which more info you need. |
|
The datalogger seems to cause over 70% CPU usage, which is absurd, unless it hammers the meter without short sleep. Do the logs in the dsmr container indicate anything, errors? Also, If you are not using the built-in datalogger, maybe you can disable it (not sure how in the container though). Do you use the dsmr_datalogger proces in the bare metal setup?
|
|
In the non-container version (v5.12) datalogger is also main CPU "eater". |
|
@sts-ux my complements about the details you've provided, that helps a lot giving insight in what's happening! In your setup I don't believe the datalogger has any function, am I correct @dennissiemensma? If that's the case, then underwater the datalogger is acting like a madmen to find a valid configuration (device) to fetch telegrams, as there is nothing, it starts hammering. The bright news is that we probably have a solution. When you run in Normally this setup is used with a More info on container run modes: https://www.yunta.nl/dsmr-reader-docker-docs/general/run-modes/ Hopefully this will fix it for you 🤞 . If not, we now already know where the painpoint is and we can come up with a solution. @sts-ux, compliments on the detailed logs, that gives us exactly the insight we need! You are completely right. In your setup, the local datalogger process has no function (correct me if I'm wrong, @dennissiemensma?). Because it can't find a valid hardware configuration or serial device to fetch telegrams, it's constantly crashing and restarting behind the scenes. That relentless loop is exactly what's hammering your CPU. The good news is that there's a built-in fix for this. Simply change your environment variable to: This mode starts the web interface and the backend, but explicitly skips starting the local datalogger. Normally, this setup is used when a separate (For context, if someone were to use You can find more info on the different container run modes here: https://www.yunta.nl/dsmr-reader-docker-docs/general/run-modes/ Hopefully, this brings your CPU usage right back down to normal 🤞. If not, we at least know exactly where the pain point is and can work out another solution 🥳 . Updated compose file: Changes:
|
|
Also some improvements made to the plugin. First validate the fixes I`ve proposed. If everything is working as expected, you could try to use this version of the updated plugin.
import logging
import os
import time
from typing import Any
import requests
from django.dispatch import receiver
from dsmr_backend.signals import backend_called
from dsmr_datalogger.services.datalogger import telegram_to_reading
LOGGER = logging.getLogger(__name__)
# Pull configurations from (Docker) environment variables with safe defaults
DSMRLOGGERWS_ENDPOINT = os.environ.get("DSMRLOGGERWS_ENDPOINT", "http://localhost/api/v1/sm/telegram")
# Preserve a low timeout to prevent the entire backend process from hanging too long.
DSMRLOGGERWS_TIMEOUT = int(os.environ.get("DSMRLOGGERWS_TIMEOUT", "5"))
# Minimum interval between requests in seconds
DSMRLOGGERWS_INTERVAL = float(os.environ.get("DSMRLOGGERWS_INTERVAL", "1.0"))
# Track the last time the API was successfully called.
_LAST_CALL_TIME = 0.0
# Keep the TCP connection open (HTTP Keep-Alive).
SESSION = requests.Session()
@receiver(backend_called)
def handle_backend_called(**kwargs: Any) -> None:
"""Fetch telegram data from DSMRloggerWS and forward to server_remote_datalogger."""
global _LAST_CALL_TIME
current_time = time.monotonic()
if (current_time - _LAST_CALL_TIME) < DSMRLOGGERWS_INTERVAL:
return
_LAST_CALL_TIME = current_time
try:
response = SESSION.get(
DSMRLOGGERWS_ENDPOINT,
timeout=DSMRLOGGERWS_TIMEOUT,
)
response.raise_for_status()
except requests.Timeout:
LOGGER.warning("[ WARN ] DSMRloggerWS plugin | Timeout after %s seconds", DSMRLOGGERWS_TIMEOUT)
return
except requests.RequestException as exc:
LOGGER.error("[ ERROR ] DSMRloggerWS plugin | Request failed: %s", exc)
return
try:
telegram_to_reading(data=response.text)
except Exception:
LOGGER.exception("[ ERROR ] DSMRloggerWS plugin | Failed to process DSMR telegram")I wan't able to validate this myself, as I cannot reproduce the usecase in my environment, but I`ve validated the code with AI as well. It seems to be fine. Example compose environment file: DSMRLOGGERWS_ENDPOINT=http://192.168.1.50/api/v1/sm/telegram
DSMRLOGGERWS_TIMEOUT=10 |
|
Yes, the datalogger has no purpose in this setup. It was probably hammering, causing lot's of CPU |
|
@xirixiz thank you very much. 
The CPU peaks on the above graphs are caused by rebooting the VM (after applying changes), so to be neglected. docker stats: Big thank you :) One more question. What would be proper "backups" volume mapping? |
I see the Changed permissions to Issue "worked around". |
|
You can check out this example: https://github.com/xirixiz/dsmr-reader-docker/blob/main/examples/docker-compose.example.yaml Or use something like: |
|
Tried and none of this solves permission denied if dsmr_backup doesn't have o+w permission. That makes some sense - |
|
The folder on the host system should be the same owner as the DUID and DGID in your container. |
|
It is the same. |
@sts-ux my complements about the details you've provided, that helps a lot giving insight in what's happening! In your setup I don't believe the datalogger has any function, am I correct @dennissiemensma? If that's the case, then underwater the datalogger is acting like a madmen to find a valid configuration (device) to fetch telegrams, as there is nothing, it starts hammering.
The bright news is that we probably have a solution. When you run in
CONTAINER_RUN_MODE: server_remote_dataloggerit doesn't start the local datalogger, but it does start all processes. I believe that should work properly for you situation.Normally this setup is used with a
remote_dataloggercontainer. What happen…