objects.jsonl slowly blowing up with growing debounce hash #2359
Comments
|
In fact that the objects.jsonl is growing is completely fine because this is the idea of this fileformat. There are settings when this file gets "optimized" and rewritten to lower the filesize again. Normally the file gets "compressed" every 23h after ab start and because of other rules. You can manually compress it by running "iob fix". How many Objects you have? (Admin shows this). Are you sure that no adapter creates new objects all the time? There were versions of ble that - whem discovery was turned on and many devices came by - it ended with many objects ... |
|
Ahhhh ok now I see ... very strange ... debounce should just be a humber and nothing else ... I would more move to admin. So please try the following: @foxriver76 any idea? That shpuld be a number and no array |
|
PS: please also post the object json from the object system.adapter.history.0 (thank you) |
Indeed, now that I checked the code here, I'm also quite confused that this should be a completely different datatype, i.e. just a simple integer...
Then it does look fine, i.e.: "custom": {
"history.0": {
"enabled": true,
"aliasId": "",
"debounceTime": 0,
"blockTime": 500,
"changesOnly": true,
"changesRelogInterval": "0",
"changesMinDelta": 0,
"ignoreBelowNumber": "",
"disableSkippedValueLogging": false,
"retention": "0",
"customRetentionDuration": 365,
"maxLength": 10,
"enableDebugLogs": false,
"debounce": 0
}
}(only pasted the interesting
I am using: v6.13.16. One idea: I usually adjust objects in group, i..e. using the wrench after selecting a list of objects by searching for their name. Once I press the wrench icon, if the individual objects had individual settings (e.g. different blocktimes), I get a comma-separated list for that setting:
Creating two data points, setting different "maxLength": [
10,
[
10,
11
],
11
],Maybe something similar happend to me with What I did to get this:
So that's probably a bug in the
Leaving out the "boring" parts (let me know if you need them): "native": {
"maxLength": 5,
"limit": 2000,
"storeDir": "/opt/iobroker/data/",
"debounce": "10000",
"retention": "0",
"storeFrom": false,
"storeAck": true,
"changesRelogInterval": "0",
"changesMinDelta": "0",
"writeNulls": true,
"blockTime": 0,
"debounceTime": 0,
"disableSkippedValueLogging": false,
"enableLogging": false,
"round": "",
"customRetentionDuration": 365
},
"acl": {
"object": 1636,
"ownerGroup": "system.group.administrator"
},
"protectedNative": [],
"encryptedNative": [],
"instanceObjects": [],
"objects": [],
"notifications": [],
"from": "system.adapter.admin.0",
"user": "system.user.admin",
"ts": 1707478179889 |
|
If I am correct, then the func for the hidden will cause this to happen if adjusting |
|
Thanks, I think the info with the wrench might help to reproduce |
|
Yes it is basically reproduceable like filter for two objects, give one e.g. debounce of 100 other debounce of 200, then use wrench to modify both objects at once. Admin will then show a comma separated list because of the different values. If you then save again, admin will save this csv list into both. See e.g. alias id or difference to last value
|
|
While showing the comma-separated list may be a feature (it's not fully clear what else The main problem in my example, though, is that this also happens with So a fix / change in behaviour needs to consider:
|
|
After talking about it it seems with older admin versions we displayed |
|
Will the fix cleanup/alter already "destroyed" objects? |
|
no, there is no automatic cleanup as it is not possible to identify the correct values, however there is now a dropdown which shows you which values are configured if you are in multi edit mode. |
|
Ok, then I have to search for the affected objects in objects.jsonl and manually fix them hopefully finaly. |
|
FYI: Not only debouce is affected: |
|
The fix is a general approach and not specific to one of the values |
|
For the history adapter, the easiest fix is likely to disable the adapter on all datapoints and then re-enable. Since data is not purged (to my knowledge) when disabling, this cleans up the configuration (but of course this may be a lot of work if you have custom retention settings for groups of data points). |
|
You can go in bulk edit mode, check where |
For me no option. I have over 1200 objects with enabled history and many have different retention, debouce, changemindelta ... |
|
@Diginix Then lets say it that way ... IF an object are affectet (at least in my assumption) then it woul fail back to the defaiuukt debounce/retention and such because ar array-data can not be parsed into a number so it will be invalid and so the instance default is used ... at least "should" ... so IF it is affected then your special set data (sorry) are lost there ... |
|
I will find all affected by looking for e.g Found 52 wrong debounce, 25 wrong enabled and 9 wrong changesRelogInterval: |
|
Let me know whether this should be reported as a new issue, but after updating to the new Admin release and trying this out by:
I get an This is with admin v6.13.20. |
|
I will have a look tomorrow @olifre |
|
6.13.21 should fix it. |
|
@foxriver76 Many thanks for the quick fix, I can confirm 👍 . |
Describe the bug
Since a few days, my
objects.jsonlkeeps growing and growing whenever I reconfigure parameters, causing adapters to use hundreds of megabytes of RAM.Checking, I find:
This hash appears to keep growing whenever I reconfigure history settings for the data point.
To Reproduce
Steps to reproduce the behavior:
Sadly, I am not fully sure whether this can be triggered as easily in any configuration... For me, it happens on each config change, i.e.:
maxLength) or debounce settings.object.jsonlseems to grow, hash never shrinks for me.Expected behavior
Hash to be trimmed at some point...
Screenshots & Logfiles
Maybe this is related to some unexpected configuration on my end? I can provide config if asked what to look for.
Versions:
The text was updated successfully, but these errors were encountered: