-
Notifications
You must be signed in to change notification settings - Fork 70
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
D-Bus ItemsChanged: reduce CPU load caused by D-Bus traffic #789
Comments
vedirect-interface branch is here: velib branches are here: velib_python is here: |
Some results from testing. Test environment
This was tested vs an unmodified setup. First test: Test effect on CPU use on a working systemSystem has:
Unoptimised systemCPU use:
CPU idle: +-30% Optimised systemCPU use:
CPU idle: +-40% (handsome improvement) Second test: DBus throughput on a heavily loaded systemSystem has:
Unoptimised system
Optimised system
Very heavily loaded optimised systemAnother two simulated solarchargers were added, for a total of 7 simulated and 2 real solarchargers (plus a BMV).
Conclusion
Other observations
|
This comment has been minimized.
This comment has been minimized.
This allows tracking dict_updates (multiple paths set on the root path of PropertiesChanged) from other services. Presently there are none, but in future tanks may well produce dict updates. victronenergy/venus#789
This is to allow tracking dict_updates that may be coming from other services. Soon systemcalc will start sending dict_updates. victronenergy/venus#789
Allow PropertiesChanged signals to contain multiple paths This is only on the consumption side for now. Production will follow. victronenergy/venus#789
More improvements to reduce CPU use victronenergy/venus#861 Allow PropertiesChanged signals to contain multiple paths victronenergy/venus#789
This replaces the earlier implementation where PropertiesChanged would carry a dict on the root path. victronenergy/venus#789
The previous PropertiesChanged on root implementation has been replaced with an ItemsChanged signal that does the same job. This time, instead of recording the message as is, we instead unroll it. This will be compensated for during replay in giving the user the option to replay the recording in packed ItemsChanged signals or individual PropertiesChanged signals. victronenergy/venus#789
By default, send all changes that happen at the same time as a single ItemsChanged signal on the root path. This replaces the previous implementation with a PropertiesChanged message on root. The old behaviour where individual PropertiesChanged messages are sent is still possible by passing the --nozip commandline option to play.py. victronenergy/venus#789
Support receiving the ItemsChanged signal. victronenergy/venus#789
This replaces the previous dict_updates implementation with one that instead tracks the ItemsChanged signal. victronenergy/venus#789
This replaces tracking of dict_update tracking by a new and taylor-made ItemsChanged signal. victronenergy/venus#789
refactor the recording script remove playback functionality, since this has moved to play.py a long time ago. Record changes from ItemsChanged signals containing multiple paths and values This replaces an earlier unreleases PropertiesChanged implementation Recorded dicts are no longer stored as-is. Instead they are repacked at replay time. This allows easier editing when manipulating a simulation, as well as more efficiently replaying older recordings. It is now possible to replay recordings in a specific CSV format. A much easier format for making demos. victronenergy/venus#789
ItemsChanged signals can contain data for multiple paths This replaces the previous implementation that piggy-backed on PropertiesChanged victronenergy/venus#789
Support receiving the ItemsChanged signal This replaces the earlier PropertiesChanged-on-root implementation victronenergy/venus#789
Support receiving the ItemsChanged signal. victronenergy/venus#789
Support ItemsChanged / GetItems as consumer. victronenergy/venus#789
Switch to ItemsChanged signal for bulk updates. - Some individual changes still generate PropertiesChanged victronenergy/venus#789
- log failed devices - group dbus updates using ItemsChanged message victronenergy/venus#789
- Fixed name for Solar charger PV power, it was named Total yield (kWh), now its PV power (W). - Add Battery temperature to the System input node. - Add Battery temperature to the VE.Bus input node. - Removed MppOperationMode from the solar-charger control node: thats not a parameter that can be written to. - Add "BMS allows to charge" and "BMS allows to discharge" paths to VE.Bus input node. Note these relate only(!) to the VE.Bus BMS. - Add support for dict-updates on the root path. victronenergy/venus#789 - Refresh npm-shrinkwrap: three small updates of dependencies.
- Fixed name for Solar charger PV power, it was named Total yield (kWh), now its PV power (W). - Add Battery temperature to the System input node. - Add Battery temperature to the VE.Bus input node. - Removed MppOperationMode from the solar-charger control node: thats not a parameter that can be written to. - Add "BMS allows to charge" and "BMS allows to discharge" paths to VE.Bus input node. Note these relate only(!) to the VE.Bus BMS. - Add support for dict-updates on the root path. victronenergy/venus#789 - Refresh npm-shrinkwrap: three small updates of dependencies.
- Fixed name for Solar charger PV power, it was named Total yield (kWh), now its PV power (W). - Add Battery temperature to the System input node. - Add Battery temperature to the VE.Bus input node. - Removed MppOperationMode from the solar-charger control node: thats not a parameter that can be written to. - Add "BMS allows to charge" and "BMS allows to discharge" paths to VE.Bus input node. Note these relate only(!) to the VE.Bus BMS. - Add support for dict-updates on the root path. victronenergy/venus#789 - Refresh npm-shrinkwrap: three small updates of dependencies.
add support for ItemsChanged signal victronenergy/venus#789
- Fixed name for Solar charger PV power, it was named Total yield (kWh), now its PV power (W). - Add Battery temperature to the System input node. - Add Battery temperature to the VE.Bus input node. - Removed MppOperationMode from the solar-charger control node: thats not a parameter that can be written to. - Add "BMS allows to charge" and "BMS allows to discharge" paths to VE.Bus input node. Note these relate only(!) to the VE.Bus BMS. - Add support for dict-updates on the root path. victronenergy/venus#789 - Refresh npm-shrinkwrap: three small updates of dependencies.
- Fixed name for Solar charger PV power, it was named Total yield (kWh), now its PV power (W). - Add Battery temperature to the System input node. - Add Battery temperature to the VE.Bus input node. - Removed MppOperationMode from the solar-charger control node: thats not a parameter that can be written to. - Add "BMS allows to charge" and "BMS allows to discharge" paths to VE.Bus input node. Note these relate only(!) to the VE.Bus BMS. - Add support for dict-updates on the root path. victronenergy/venus#789 - Refresh npm-shrinkwrap: three small updates of dependencies.
node-red-contrib-victron: v1.4.16 -> v1.4.17 1.4.17: - Add registration of the node output-vebus - Remove the paths /SystemReset and /PvInverter/Disable from output-ess. These are also in output-vebus and are not related to the Energy Storage System. - Update documentation to reflect the changes in paths. node-red-contrib-victron upgrade 1.4.12 -> 1.4.16 1.4.16: - Revert of removing output-vebus. The output-vebus node is perhaps used by some users, so reverting the removal of the node for now. The move of the paths to output-ess does not cover each case as there might be older Phoenix models on the VE.Bus. - npm package babel-eslint is now known as @babel/eslint-parser. 1.4.15: - The PropertiesChanged messages ended up returning an array; this fixes that by returning the value instead. - Add the ability to query the user defined name (as part of input-system) in order to make testing easier. It can also be useful when creating a dashboard. 1.4.14: - bump node-fetch from 2.6.6 to 2.6.7 - Alphabetized services.json. First on node name, then on path. - Add several missing nodes: - input-alternator - input-dcsystem - input-evcharger - input-generator - input-genset - output-evcharger - output-multi - Remove node: output-vebus (the paths are part of output-ess) - Update documentation to reflect the changes in order and paths. - Add paths to input nodes that where present in the output nodes, - allowing them to be read too. - Fix typo in README.md 1.4.13: - New nodes: - input-acload - input-multi and output-multi - Moved GridLost alarm from input-system to input-vebus, add /Alarms/PhaseRotation - Waded through the services file, comparing it to the dbus_modbustcp excel sheet. This resulted in a lot of additions, some removals and some renaming and updates of documentation. - Add script for comparing entries with dbus_modbustcp excel sheet - fix markdown syntax, typo and alphabetized input and output entries node-red-contrib-victron: 1.4.11 -> 1.4.12 - In some cases invalid messages end up on the dbus. Now filtering the incoming messages so only valid messages are further processed. node-red-contrib-victron 1.4.9 -> 1.4.11 1.4.11: fix in dbus ItemsChanged - now copying msg.path for dbus ItemsChanged signals 1.4.10: support for dbus ItemsChanged and minor fixes - add support for newly added ItemsChanged signals in Venus OS D-Bus - now using strict type check when comparing two variables - the 'index.html' can be left out of the Victron Energy community url. This solves a too wide string for the help text of 'victron-client' node-red-contrib-victron: 1.4.6 ->1.4.9 - add support for Ruuvi tags: humidity, pressure, batt. voltage and acceleration. The other value, temperature, was already supported. - fix conditionally setting the number type (float values where not accepted) - add relay 3 and 4 to input-relay and output-relay nodes. At the moment only useful on a raspberrypi having additional IOs or relays. - add extra nodes (via @nmbath / pr #109) - add input node for dcload meters (input-dcload) - add input node for dcsource meters (input-dcsource) - add input nodes for alternator and dcdc converters ((input-alternator, input-dcdc) - added ATC (Allow To Charge) and ATD (Allow To Discharge) to input-battery note that not all BMS-es have those paths available. Most probably do not, probably only the Lynx Smart BMS has it. - added various paths to the output-ess node, allowing more control (issue #108): - Feed DC overvoltage into grid : /Hub4/DoNotFeedInOvervoltage - Disable PV inverter : /PvInverter/Disable - Maximum overvoltage feed-in power L1 : /Hub4/L1/MaxFeedInPower - Maximum overvoltage feed-in power L2 : /Hub4/L2/MaxFeedInPower - Maximum overvoltage feed-in power L3: /Hub4/L3/MaxFeedInPower - VE.Bus Reset : /SystemReset - added documentation to each node node-red-contrib-victron v1.4.6 - Fixed name for Solar charger PV power, it was named Total yield (kWh), now its PV power (W). - Add Battery temperature to the System input node. - Add Battery temperature to the VE.Bus input node. - Removed MppOperationMode from the solar-charger control node: thats not a parameter that can be written to. - Add "BMS allows to charge" and "BMS allows to discharge" paths to VE.Bus input node. Note these relate only(!) to the VE.Bus BMS. - Add support for dict-updates on the root path. victronenergy/venus#789 - Refresh npm-shrinkwrap: three small updates of dependencies. node-red-contrib-victron v1.4.2 -> v1.4.5 Changes to nodes: - solar-charger node: add support for multiple trackers (MPPT RS) - solar-charger node: remove PV current and replace with PV Power. - system-node: add lots of measurements - ess-node: add Active SoC limit - many nodes: add various missing enum names as well as update them, this doesn't change actual node-red functionality; it only makes the descriptions better as visible in the Edit Node pane. - many nodes: update the names of the measurements to match how now defined on VRM Other changes: - node-red-contrib-victron has had a major update of its package-lock.json. But, that won't make much of a difference on how its installed in Venus OS, since the npm-shrinkwrap used already had all those newer versions. To prevent issues like this in the future, there is now a procedure written here: https://github.com/victronenergy/node-red-contrib-victron#releasing-a-new-version-developers - though the npm-shrinkwrap has lots of changed lines, most of it is just whitespace, most dependency versions are unchanged.
This somewhat reduces dbus messages, mostly useful for pulsecounter devices that may send updates more frequently. victronenergy/venus#789
send ItemsChanged instead of PropertiesChanged victronenergy/venus#789
- send ItemsChanged instead of PropertiesChanged victronenergy/venus#789 - remove deprecated /Ac/Current - Don't react to uninteresting NameOwnerChanges
Send one ItemsChanged instead of one per phase as earlier. victronenergy/venus#789
Reduce D-bus traffic, send one ItemsChanged instead of one per phase. victronenergy/venus#789
Reducing D-Bus traffic - increasing scalability
For example a solar charger, sends multiple PropertiesChanged signals every second, one for each D-Bus path. Also cgwacs and dbus-fronius do: 9 signals per second per PV inverter. And for reference, on a CCGX, the max signals per second is approx 30.
The idea is to send only one every (for example) second, from root, with a Dict with all paths that have an updated value.
this project started under the name PropertiesChanged, and during the implementation we changed the details and renamed the D-Bus signal to ItemsChanged.
The plan:
Services that consume (and thus need updating first)
Consumers that are not immediately required:
Producers of highest prio:
Producers that can be done some day:
Python:
C:
Cpp:
Notes per language:
Python:
In the majority of cases all that is required is updating velib_python after merging dict_updates. In some cases, a project might need a small change, ie. using the
with
statement notation, to send updates in a batch.The text was updated successfully, but these errors were encountered: