Add support for Blueline PowerCost Monitor #1580

jbrzozoski · 2020-12-14T05:09:11Z

This is still a work in progress, but it is nominally working and I would like other people's feedback.
Open questions and things to do:

Do decoder callback functions in rtl_433 need to support multi-threaded re-entrancy? I use some function-scoped static variables that could be problematic if so.
Is the ID guessing algorithm too much processing? It will trigger 16000 2-byte CRC checks any time a message comes in with a bad CRC, to help find what the proper ID should be. There is a pre-check that it is looking for a row of the right length with the right first byte before that, so it shouldn't be hit too often, in my opinion.
Why don't any of the temperature scaling equations others have provided match up with what I'm correlating between my radio data and LCD display? I haven't put any in just yet and want to figure it out before I do.
I need to validate the ID message and the battery status flag work as behaved. I just copied them from old code but haven't tested them yet.

zuckschwerdt · 2020-12-14T09:26:29Z

Quick notes: yes, decoders must not be stateful. I need to look into this in detail later, but we really don't want any static state in decoders.

src/devices/blueline.c

merbanan · 2020-12-14T10:37:01Z

src/devices/blueline.c

+// If BLUELINE_ACTIVE_ID_AUTODETECT is set to 1, this code will try to autodetect IDs from
+// incoming messages which otherwise would fail CRC.  While there are a lot of false positives
+// on each individual message, only one ID will continuously match up, and it becomes
+// obvious which one is the right one within a few minutes.


How many matches do you get? Is there a way to validate the payload parameters?

Any individual message will get 2^6 matches. The payload can unfortunately be any value and there's no way to narrow it down by looking for reserved or unused fields.

merbanan · 2020-12-14T10:45:02Z

@jbrzozoski regarding recovering the correct id, if it still is almost real time on most platforms it is ok. But you must be as sure as possible before we try to recover the id. The sparsnas decoder uses something similar.

merbanan · 2020-12-14T10:49:26Z

Regarding the temperature. If you post a set of readings Im sure we can figure it out.

merbanan · 2020-12-14T10:59:52Z

I now remember what I thought initially. Each transmission contains 3 payloads that might be different. I think that if all payloads are used then it should be possible to recover a unique id. But that should not be the case based on comments I've read. So I think we need to pass the id by command line to the decoder. Or accept that this decoder needs a state.

The raw logic is that we got a 14 bit id and only 8 bits of crc to match it against. (And the crc is supposed to validate the payload, not to be used to recover the id, a broken payload will recover a bad id).

jbrzozoski · 2020-12-14T15:22:49Z

Within any given burst, all 3 messages are often the same message repeated, so it's not practical to expect to be able to recover the ID from one burst. Your summary that we need to either pass the ID from the command line or maintain a state was my conclusion as well, but I couldn't find any examples of other devices with custom command line options, so I tried to avoid that.

merbanan · 2020-12-14T16:30:28Z

https://github.com/merbanan/rtl_433/blob/master/src/devices/fineoffset.c#L20

I suggest you add one option that sets the key and another option to brute force the key (maybe you can not reset the transmitter). That option will make the decoder keeping a state. I think that is an ok compromise. By default I prefer things to be stateless. @zuckschwerdt I think this is a good enough compromise.

A user could enable to option and things would just work. If anyone requests support the answer will be that they need to figure out the key for things to work. And for regression tests we can either add the key to the command line or just handle the message with a proper crc.

jbrzozoski · 2020-12-14T17:01:49Z

I like the two options plan. A user can interactively use the option to brute force the key, and within a couple minutes rtl_433 should start reporting data including the ID key that it found. Then the user can stop that interactive session and just pass in the discovered key on the command line for all future use.

Not having the stateful key search running all the time avoids some weird issues that could crop up if there happens to be two Blueline transmitters within range, or a single rtl_433 session running so long that random noise eventually looks like a valid transmitter. The brute force search mechanism has no concept of time. If enough random messages all match a key over several days without restarting rtl_433, it would consider that a match and switch to a bogus ID, which would then switch back quickly if a real transmitter is in range.

jbrzozoski · 2020-12-15T06:28:41Z

Slight tweaking to behavior...

If you don't pass any parameters to this decoder, all it will ever do is spit out the TXID message if it hears someone restart a monitor. And I suppose if you just happened to have a monitor ID of 0 it would work, too, but that's unlikely.

You can pass in the special parameter "auto" which will put it in auto-detect mode until the first time it sees a TXID message or the brute-force passes the threshold, at which point it will switch to using that ID and turn off the autodetect. (To avoid any of the weird issues I mentioned earlier in this thread.)

Finally, any other parameter passed in is assumed to be an ID, and just passed through strtoul.

I like this since it actually can support multiple transmitters in range if you just pass multiple registrations on the command line like -R 176:ABCD -R 176:5432. 👍

All state is setup at creation time except when "auto" mode is enabled, and once that picks an ID it also stops changing state.

This kind of what you were thinking, @merbanan ?

merbanan · 2020-12-15T08:20:48Z

src/devices/blueline.c

+        working_buffer[1] += 1;
+    }
+
+    fprintf(stderr, "Attempting Blueline autodetect: best_hits=%u\n", best_hits);


Needs to be behind verbosity check.

merbanan · 2020-12-15T08:21:24Z

src/devices/blueline.c

+
+// TODO - Add comments for the new parameters and expected usage
+
+#include <stdbool.h>


No stdbool.h please.

merbanan · 2020-12-15T08:22:31Z

src/devices/blueline.c

+#define MAX_POSSIBLE_BLUELINE_IDS (65536/BLUELINE_ID_STEP_SIZE)
+#define BLUELINE_ID_GUESS_THRESHOLD 10
+
+struct blueline_context {


Name it blueline_stateful_context, the it will be very clear what this is.

merbanan · 2020-12-15T08:24:04Z

src/devices/blueline.c

+            decoder_output_data(decoder, data);
+            payloads_decoded++;
+            if (context->searching_for_new_id) {
+                fprintf(stderr,"Switching to received Blueline ID 0x%04X\n", received_sensor_id);


Verbose guard missing.

merbanan · 2020-12-15T08:24:19Z

src/devices/blueline.c

+            if ((context->searching_for_new_id) && (message_type != BLUELINE_TXID_MSG)) {
+                uint16_t id_guess = guess_blueline_id(context, current_row);
+                if (id_guess != 0) {
+                    fprintf(stderr,"Switching to auto-detected Blueline ID 0x%04X\n", id_guess);


Verbose guard missing.

merbanan · 2020-12-15T08:25:34Z

src/devices/blueline.c

+        if (strcmp(arg, "auto") == 0) {
+            // Setup for auto identification
+            context->searching_for_new_id = true;
+            fprintf(stderr, "Blueline decoder will try to autodetect ID.\n");


Verbose guard missing.

I'm not seeing any way to determine if we're in verbose mode inside the creator functions.

If you'd like, I can add a commit into this pull request to add more parameters to the create_fn, either the whole of the r_cfg_t or just the verbosity value from it. I'll adjust fineoffset.c to match.

Remove the output for now. Lets focus on getting this patch in first. Then we can add it later.

Please don't access r_cfg from decoders -- we need some modularity.

merbanan · 2020-12-15T08:25:39Z

src/devices/blueline.c

+        } else {
+            // Assume user is trying to pass in hex ID
+            context->current_sensor_id = strtoul(arg, NULL, 16);
+            fprintf(stderr, "Blueline decoder using ID %04X\n", context->current_sensor_id);


Verbose guard missing.

merbanan · 2020-12-15T09:03:34Z

I like this since it actually can support multiple transmitters in range if you just pass multiple registrations on the command line like -R 176:ABCD -R 176:5432.

All state is setup at creation time except when "auto" mode is enabled, and once that picks an ID it also stops changing state.

So the command -R 176:auto would enable the brute force mode?

bkenobi · 2020-12-16T02:10:40Z

I received confirmation from BLI that the Black & Decker EM100B that looks identical to this unit i functionally identical and should be compatible with this library. When this work is completed, it should be noted that it is also usable.

jbrzozoski · 2020-12-16T22:06:24Z

Just some minor tweaks in the right direction:

added most of the verbosity guards as requested
switched to displaying/expecting the ID in decimal (instead of hex) to be consistent with MQTT payloads/topics
fixed a mismatch for the model displayed on TXID messages

And to answer your question, using -R 176:auto would enabled the brute-force mode.

jbrzozoski · 2020-12-16T22:10:40Z

(Just pushing to fix a style check)

src/devices/blueline.c

merbanan · 2020-12-17T15:06:44Z

src/devices/blueline.c

+
+#ifndef UNUSED
+#define UNUSED(x) (void)(x)
+#endif


This looks unused.

Of course. That's what it is. 🤣
(Yes, I'll remove it.)

merbanan · 2020-12-17T15:08:17Z

src/devices/blueline.c

+        .short_width = 500,
+        .long_width  = 1000,
+        .gap_limit   = 2000,
+        .reset_limit = 200000,


I think the practical limit for this is much smaller.

Maybe 8000 would work?

It's almost a don't-care then, as long as it's bigger than gap_limit. The pauses between rows in one burst is actually closer to 100000, so they always show up in the decoder one row at a time. I'll bring this down to 8000.

merbanan · 2020-12-17T15:10:27Z

Get a good set of sample signals and submit it to the rtl_433_tests repo. Then we try to figure out the temperature encoding and then I think we are set. The code looks fine to me.

@zuckschwerdt ok to merge after those fixes?

Signed-off-by: Justin Brzozoski <justin.brzozoski@gmail.com>

jbrzozoski · 2020-12-17T18:42:34Z

I'll submit some samples later today after I can head outside and hit the button a few times to make sure I have samples of the ID message.

jbrzozoski · 2020-12-18T03:39:16Z

Samples are up:

merbanan/rtl_433_tests#380

Mindavi · 2020-12-19T21:37:39Z

I don't know why, but none of the verbose logging seems to be working. I guess this has something to do with rtl_433 internals? Otherwise this seems to work well with the submitted samples. Have not thoroughly reviewed it, but the code looks good in general.

jbrzozoski · 2020-12-19T22:12:29Z

I noticed the verbose output wasn't working when I was testing the recorded samples, but assumed it was somehow a side affect of the replay mechanism. I just double checked on my receiver and it doesn't work there either. I'll take a closer look in the next 5-7 hours.

jbrzozoski · 2020-12-20T03:15:48Z

Verbosity and registrations are command line order dependent:

Running rtl_433 -vvv -R 23 -R 176 gives both decoders verbosity level 3.
Running rtl_433 -R 23 -vvv -R 176 gives decoder 23 verbosity level 0, and decoder 176 verbosity level 3.
Running rtl_433 -R 23 -R 176 -vvv gives both decoders verbosity level 0.

@Mindavi do you know which order you had your command line setup?

jbrzozoski · 2020-12-20T03:34:08Z

Is there a way to pass a parameter to one decoder without disabling all the others?

I'm seeing that just using -R 176:12345 disables all the other decoders as well. Is there a practical way to use this decoder with an option and the default decoders?

zuckschwerdt · 2020-12-20T09:39:56Z

Not really recommended but -R -1 -R 1 -R 176:12345does the trick.

Mindavi · 2020-12-20T16:04:51Z

Yes, I had the verbose parameter at the end of the command. However, I can't get the verbose output working with the parameters anywhere on the line. Did you get it working?

Edit: whoops, I have to add auto to the line, of course. That works with the -vv in the beginning of the parameters.

jbrzozoski · 2020-12-20T16:17:55Z

Not really the ideal way to test it, buy you can prove the "auto" mode works with the samples I provided. You just have to pass each file multiple times to a single instance, so something like this will exercise the brute force method:
rtl_433 -R 176:auto -r gaps.cu8 -r temp.cu8 -r impulse.cu8 -r gaps.cu8 -r temp.cu8 -r impulse.cu8

I wasn't really checking for it when I made the samples, but the CRCs in those payloads hit enough orthogonal IDs that only the right one matches all of them and passes the threshold even if they're repeated. (i.e. no false positives)

If you put it in "auto" mode and use the ID sample, it will immediately lock onto the ID payload and will decode everything after that, so a command line like this will decode all of the samples:
rtl_433 -R 176:auto -r id.cu8 -r gaps.cu8 -r temp.cu8 -r impulse.cu8

Mindavi · 2020-12-20T16:19:59Z

Yeah, I figured it worked like that. Seems to work fine.

Are there any open comments here?

merbanan · 2020-12-20T16:47:43Z

Config and manual additions are missing but besides that everything is ok by me.

jbrzozoski · 2020-12-23T20:24:20Z

Did you need anything else from me on this?

And my main monitoring station now has these config lines:

protocol -176
protocol 176:12345

Which looks weird, but works fine setting an option on this decoder while leaving all default decoders enabled. 👍

jbrzozoski mentioned this pull request Dec 14, 2020

Added decoding for Blueline/Black and Decker Power Monitor #38

Closed

This comment has been minimized.

Sign in to view

jbrzozoski force-pushed the blueline branch from 9ce1854 to 772d477 Compare December 14, 2020 05:36

merbanan reviewed Dec 14, 2020

View reviewed changes

src/devices/blueline.c Show resolved Hide resolved

merbanan reviewed Dec 14, 2020

View reviewed changes

jbrzozoski force-pushed the blueline branch from 772d477 to 5f0412c Compare December 15, 2020 06:19

merbanan reviewed Dec 15, 2020

View reviewed changes

This comment has been minimized.

Sign in to view

jbrzozoski force-pushed the blueline branch from 5f0412c to cd6110f Compare December 16, 2020 22:02

jbrzozoski force-pushed the blueline branch from cd6110f to 6108a17 Compare December 16, 2020 22:10

merbanan reviewed Dec 17, 2020

View reviewed changes

src/devices/blueline.c Show resolved Hide resolved

merbanan reviewed Dec 17, 2020

View reviewed changes

Add support for Blueline PowerCost Monitor

e04529c

Signed-off-by: Justin Brzozoski <justin.brzozoski@gmail.com>

jbrzozoski force-pushed the blueline branch from 6108a17 to e04529c Compare December 17, 2020 17:11

jbrzozoski changed the title ~~WIP - Add support for Blueline PowerCost Monitor~~ Add support for Blueline PowerCost Monitor Dec 17, 2020

jbrzozoski marked this pull request as ready for review December 17, 2020 17:13

Mindavi mentioned this pull request Dec 19, 2020

Add Blueline PowerCost Monitor samples merbanan/rtl_433_tests#380

Merged

Maintainer update for blueline

8773ec4

Mindavi merged commit a20cd1a into merbanan:master Dec 23, 2020

jbrzozoski mentioned this pull request Dec 24, 2020

Add Blueline PowerCost Monitor #1248

Closed

hertzg mentioned this pull request Feb 13, 2021

Command not working properly hertzg/rtl_433_docker#5

Closed


		// TODO - Add comments for the new parameters and expected usage

		#include <stdbool.h>

Add support for Blueline PowerCost Monitor #1580

Add support for Blueline PowerCost Monitor #1580

Conversation

jbrzozoski commented Dec 14, 2020 • edited Loading

This comment has been minimized.

zuckschwerdt commented Dec 14, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

merbanan commented Dec 14, 2020

merbanan commented Dec 14, 2020

merbanan commented Dec 14, 2020

jbrzozoski commented Dec 14, 2020

merbanan commented Dec 14, 2020

jbrzozoski commented Dec 14, 2020

jbrzozoski commented Dec 15, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

merbanan Dec 15, 2020 • edited by zuckschwerdt Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

merbanan commented Dec 15, 2020

This comment has been minimized.

This comment has been minimized.

bkenobi commented Dec 16, 2020

jbrzozoski commented Dec 16, 2020 • edited Loading

jbrzozoski commented Dec 16, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

merbanan commented Dec 17, 2020

jbrzozoski commented Dec 17, 2020

jbrzozoski commented Dec 18, 2020

Mindavi commented Dec 19, 2020

jbrzozoski commented Dec 19, 2020 via email • edited Loading

jbrzozoski commented Dec 20, 2020 • edited Loading

jbrzozoski commented Dec 20, 2020 • edited Loading

zuckschwerdt commented Dec 20, 2020

Mindavi commented Dec 20, 2020 • edited Loading

jbrzozoski commented Dec 20, 2020

Mindavi commented Dec 20, 2020

merbanan commented Dec 20, 2020

jbrzozoski commented Dec 23, 2020

jbrzozoski commented Dec 14, 2020 •

edited

Loading

jbrzozoski commented Dec 15, 2020 •

edited

Loading

merbanan Dec 15, 2020 •

edited by zuckschwerdt

Loading

jbrzozoski commented Dec 16, 2020 •

edited

Loading

jbrzozoski commented Dec 19, 2020 via email •

edited

Loading

jbrzozoski commented Dec 20, 2020 •

edited

Loading

jbrzozoski commented Dec 20, 2020 •

edited

Loading

Mindavi commented Dec 20, 2020 •

edited

Loading