Assistance in interpreting (temperature? humidity?) signals

[giddyhup]

I seem to be so close to cut the wire and use my Raspberry Pi for temperature logging. All my attempts with GPIO connected 433 MHz receivers failed. Now, with a SDR approach and your solution I seem to be close to the goal.

I have several sensors around the house, one of which also measures humidity. I am not sure if what I am receiving is coming from that sensor. I would already be happy to receive a temperature. Could you please assist me in interpreting the output (http://pastie.org/private/exdyxwgomasmfwcub0pkfw)? The only pattern I see are static zeroes and ones, there is nothing, I think, that could be an identifier.

Your feedback us greatly appreciated.

[merbanan]

The output is not constant, most likely the signal is not captured correctly. Try with -l 3000.

[giddyhup]

Thank you for your feedback. Regrettably, changing the pulse detection level to 3000 made the results worse. Using SDR# I was able to pinpoint the frequency to 433936655 Hz, I recorded one pulse sequence and captured an image and looked at it with Audacity (http://imgur.com/a/fiTgY).
If I interpret the first burst properly I should receive 11110101111111011001100011001110100011001000. How can I proceed?

[merbanan]

Run with rtl_433 -a -t that should save the signal. Sent that signal to me.

[giddyhup]

I figured out the first burst. It is a LaCrosse sensor (cf. http://forum.arduino.cc/index.php?topic=155483.0). Here are my results:

It is also consistent with what the command line output shows.

I guess the second series of bursts is the humidity. I am currently less interested in this but is probably just as simple to decipher. As for submitting the recorded data, Benjamin, am I right to assume that you are not at the North Pole?

[merbanan]

You are correct :) I am in Stockholm. But then it looks like the signal is captured correctly.

[giddyhup]

Thanks again for the feedback. I guess I won't submit the recording.

Now comes the complicated part, decoding the binary bursts and writing a script to parse the output. Should I manage to bash (sic) something working together I will post it here for posterity. Currently I am already failing since the continuous output cannot be grepped.

I have given up to parse the humidity. Raw 11110101000111011000 1011 1001 1111 1011 1001 1010 (11-9-15-11-9-10 as payload) should translate to 46 % but I doubt the formula 11 – 11 = 0; 15 – 9 = 4; 21 – 15 = 6 is correct.

[giddyhup]

Figured out humidity: the preamble is different, it uses the same formula as temperature and adds 50 (what would result in -4 °C is 46 % relative humidity).

I am still at a dead end since the output from rtl_433 -a cannot be grepped. Also, weirdly, the closer the sensor is to the antenna the less reliable the signal gets. The receiver wouldn't accept gain setting changes.

I'll submit six recordings. The 5th and 6th were recorded with the sensor close to the antenna. 1 through 4 contain the following signals.

[merbanan]

Well I want the actual gfiles. Can you upload them somewhere ?

[giddyhup]

Does this link to the files work: https://mega.co.nz/#F!bIlX1IDC!TvnDj3FtAm18LdjPZHt5cQ (edited)

[giddyhup]

I removed two of my previous script entries, they handled negative temperatures wrongly, this one should work:

#!/bin/bash

# bash script to decode 44 bit LaCrossee 433 Mhz temperature/humidity sensors
# as captured by rtl_433 -a
#
# usage:
# script_name "raw rtl_433 -a output"
#
# example:
# lacrossetemp.sh "[00] {44} f5 fd 8a c8 ac 50 : 11110101 11111101 10001010 11001000 10101100 01010000"
#
# script has no real error checking and is not optimized
# (e.g. loops or better variable handling)
#
# 2014-01-13, mg1106

# use command line argument as input
orig=$1

# check length declaration (bits 7 and 8)
length=${orig:6:2}
if [ $length != "44" ]
then
    echo "Length declaration not 44, exiting."
    exit 1
fi

# remove unused characters and spaces
nospace=$(echo $orig | cut -d" " -f 10-15 | tr -d " ")

# determine mode (10th bit); 1 is temperature, 0 is humidity
mode=${nospace:9:1}

# cut out value nibbles
tensbin=${nospace:20:4}
onesbin=${nospace:24:4}
tenthssbin=${nospace:28:4}

# convert bin to dec
tensraw=$(echo "ibase=2;obase=A;$tensbin" | bc)
onesraw=$(echo "ibase=2;obase=A;$onesbin" | bc)
tenthsraw=$(echo "ibase=2;obase=A;$tenthssbin" | bc)

# different formula for depending on mode and value of tens
if [ $mode = "1" ]
then
    tens=$(expr 10 - $tensraw)
    ones=$(expr 15 - $onesraw)
    tenths=$(expr 15 - $tenthsraw)

# echo $tens" "$ones"."$tenths

# if tens > -1 then temperatures are above 0
    if [ $tens -gt -1 ]
        then
            echo $tens$ones"."$tenths"° C"
        else
# for negative temperatures 20 must be added to get actual value
        negtemp=$(echo $tens$ones"."$tenths + 20 | bc)
        echo "- "$negtemp"° C"
    fi
else    
# mode is not 1, we are showing humidity
    tens=$(expr 15 - $tensraw)
    ones=$(expr 15 - $onesraw)
    echo $tens$ones" %"
fi

[rct]

@giddyhup Try the current version to see if your LaCrosse Temperature and Humidity sensors are correctly handled now. I submitted code for this which has been merged into master by now.

[giddyhup]

Thank you for the information. I tried it but with the current version (after applying an include fix which was necessary for my Raspberry Pi) even my Prologue-type sensor stopped being recognized and the Lacrosse-type sensors aren't either. I only get many "AlectoV1 CRC error" error messages.

[01] {44} f5 1d f8 df 8d 30 : 11110101 00011101 11111000 11011111 10001101 00110000

should be interpreted as a humidity of 72 %. And

[01] {44} f5 fd fa 9f a9 d0 : 11110101 11111101 11111010 10011111 10101001 11010000

should be 6.0 °C. I will continue to use my own script, where I capture and decode the -a output.

[rct]

@giddyhup, I will give the latest merge a try and make sure the LaCrosse code is still working. If you get a chance you could try my branch: https://github.com/rct/rtl_433/tree/lacrosse-tx-temp-humidity

I'd like to know that the LaCrosse code works for others too.

[giddyhup]

@rct, I tried your branch (there were some build errors mentioning Alecto, but it built). Prologue is working, my Lacrosse-style sensors aren't. But I am not really sure they are Lacrosse-type, it is just what I found as similar in signal to my sensors.

[rct]

Yes, that looks like a valid LaCrosse message to me. They bare both from Sensor 10. The checksum is valid. I'm not sure about the parity bit, but my code would give you an error message.

If you get a chance to capture signal files (gdataNNN) with -t and post them some place, I'd be interested in trying to figure out why they aren't being recognized.

What brand are they? What countries are they available in?

[giddyhup]

I have only one sensor of that type ("TFA Dostmann V11 A TX 7 N"). Here's a German manual. I only have the sensor.

I will record some raw data and see how I can make it available.

[giddyhup]

@rct You'll find recent samples here. Also, from last year, some samples are also here.

[rct]

@giddyhup -

1. My version parses all but two of your files (gfile005 and gfile006 from the older batch).
2. The new merged code base is not parsing my LaCrosse sensors correctly for me either. I'll have to dig into all the changes to see what's different.

Here's the output from your files (-new are your 2015 samples):

( for i in *;do echo ==== $i ====;./src/rtl_433 -r $i;done  ) 2>/dev/null | egrep ===\|Cross
==== gfile001.data ====
2015-01-18 11:36:18 LaCrosse TX Sensor 13: Temperature 29.9 C / 85.8 F
==== gfile002.data ====
2015-01-18 11:36:18 LaCrosse TX Sensor 13: Humidity 43.0%
==== gfile003.data ====
2015-01-18 11:36:19 LaCrosse TX Sensor 13: Temperature 1.1 C / 34.0 F
==== gfile004.data ====
2015-01-18 11:36:19 LaCrosse TX Sensor 13: Humidity 85.0%
==== gfile005.data ====
==== gfile005-new.data ====
2015-01-18 11:36:20 LaCrosse TX Sensor 10: Temperature 21.1 C / 70.0 F
==== gfile006.data ====
==== gfile006-new.data ====
2015-01-18 11:36:21 LaCrosse TX Sensor 10: Humidity 58.0%

[giddyhup]

@rtc, thanks for the feedback. Let's see if the master gets fixed. Old gfile005 and gfile006 might have been Prologue-style data.

While we're at this, I wish there were are better parseable output. Currently I am actually decoding the -a output (piped to a separate file). In the regular mode sensor data is printed in top-to-bottom text blocks which means this data needs to be transposed to be used. Wouldn't it be nice if the output were in a table or a tsv?

[pclov3r]

No luck here with a LaCrosse TX6U

If you'd like samples I'd be happy to provide them.

[rct]

The current merbanan/master won't work with the LaCrosse because a fix was lost in the big @helgew merge. Hopefully @merbanan will merge the pull requests with the fixes soon.

In the meantime you could use my branch.

https://github.com/rct/rtl_433/tree/lacrosse-tx-temp-humidity

Note: I have both TX6U and TX7U. The protocol is identical other than the TX6U only sending temperature while the TX7U sends both.

[rct]

@giddyhup, This is what I'm currently outputting, I'm not sure how a table/tsv would help much, since this is long running output. rtl_433 needs a bit of restructuring to deal with call backs/web services or what ever sort of output people want.

2015-01-18 11:26:44 LaCrosse TX Sensor 29: Temperature 3.0 C / 37.4 F
2015-01-18 11:26:44 LaCrosse TX Sensor 29: Humidity 90.0%
2015-01-18 11:27:03 LaCrosse TX Sensor 06: Temperature 20.4 C / 68.7 F
2015-01-18 11:27:04 LaCrosse TX Sensor 06: Humidity 30.0%
2015-01-18 11:27:07 LaCrosse TX Sensor 54: Temperature 20.2 C / 68.4 F
2015-01-18 11:27:19 LaCrosse TX Sensor 30: Temperature 20.7 C / 69.3 F
2015-01-18 11:27:19 LaCrosse TX Sensor 30: Humidity 31.0%

[rct]

Ah, @giddyhup, I remember now, I tried your bash script, you expect the input on the command line instead of just reading it from stdin. I was initially confused by that and didn't read enough of your script.

[giddyhup]

@rct, sorry for being quiet. Here's what I do. I have a master script where I run rtl_433 -a -l 10000 -f 433925028 -p 1 and write stdout to a file. After 60 seconds I kill the rtl_433 process and grep for the signatures of my known sensors. The grep output is piped through the bash script of which you see an early version above. The results are written to individual files from which they are picked up by other processes one of which writes the values to a mysql database.

It would be a bit easier if I could use the non-analytics output. I learned later that one of my sensor actually uses the prologue protocol but I already had reverse-engineered the code.

Does your code still include the prologue decoding?

[giddyhup]

@rct, I tried your code. My Prologue sensors' signals are interpreted, my one Lacrosse's isn't:

signal_len = 101990,  pulses = 132
Iteration 1. t: 239    min: 140 (45)    max: 338 (87)    delta 18
Iteration 2. t: 239    min: 140 (45)    max: 338 (87)    delta 0
Pulse coding: Short pulse length 140 - Long pulse length 338

Short distance: 245, long distance: 248, packet distance: 7107

p_limit: 239

[00] {44} f5 fd fa a6 aa 60 : 11110101 11111101 11111010 10100110 10101010 01100000
[01] {44} f5 fd fa a6 aa 60 : 11110101 11111101 11111010 10100110 10101010 01100000
[02] {44} f5 fd fa a6 aa 60 : 11110101 11111101 11111010 10100110 10101010 01100000

*** signal_start = 28859356, signal_end = 28961233
signal_len = 101877,  pulses = 132
Iteration 1. t: 238    min: 139 (45)    max: 338 (87)    delta 17
Iteration 2. t: 238    min: 139 (45)    max: 338 (87)    delta 0
Pulse coding: Short pulse length 139 - Long pulse length 338

Short distance: 246, long distance: 248, packet distance: 7048

p_limit: 238

[00] {44} f5 1d f9 6f 96 70 : 11110101 00011101 11111001 01101111 10010110 01110000
[01] {44} f5 1d f9 6f 96 70 : 11110101 00011101 11111001 01101111 10010110 01110000
[02] {44} f5 1d f9 6f 96 70 : 11110101 00011101 11111001 01101111 10010110 01110000

The current master is broken. Even when I fix the missing bracket.

[pclov3r]

The latest release appears to work with my TX6U correctly however is a missing opening bracket with the Oregon scientific c file so it fails to compile correctly.

I did submit a pull request to include that opening bracket.

#89

[pclov3r]

@giddyhup What error are you getting when trying to build it?

Do you have a output log?

[giddyhup]

@pclov3r, I fixed the bracket but as output I mostly get AlectoV1 CRC error repeatedly. It seems that the Alecto decoder tries to decode the Prologue data. The good news is, now that I let it run longer, my Lacrosse sensor shows up!

[giddyhup]

Where/how would I comment out the Alecto part?

[giddyhup]

@rct, now that I see your Lacrosse output: thank you, it looks very good and usable.

[pclov3r]

@giddyhup Ah I read that as it was failing to build even after fixing the bracket.

What I do to decode the LaCrosse sensor without errors is use a program like SDR# or GQRX calibrate the PPM and narrow the frequency down. Mine was transmitting 433.402 Mhz. Once you do that populate the same info in rtl_433 with -f frequency and -p ppm

Hopefully that helps.

@rct thanks for the code as it works now without doing hacking with a bash script!

[giddyhup]

@pclov3r, thanks, that's what I did to find the best frequency to scan.

[giddyhup]

I commented out the Alecto device in the current master code but regrettably my Prlogue sensors are not coming back.

[rct]

@giddyhup - Glad to hear it is working now.

To quiet down the Alecto if you aren't using it, comment out the register_protocol() in main. e

If you want to limit the amount of CPU rtl_433 uses, comment out the register_protocol() for all protocols that you aren't using. Currently every protocol registered iterates over every sample that is received. Future work on plugins, etc. will allow you to run only the protocols you are interested in.

[rct]

When testing with 5 or so LaCrosse TX sensors, I found the center frequency spread between them could be as much as 100 Khz!. The TX6u's seem to have better oscillators than than the TX7u and are closer in frequency.

Try to find a good stable signal to correct your PPM error for your rtl_sdr stick first. For the US, NOAA weather radio is a pretty good choice.

[rct]

Also, if you lower your detection level, -l (try 8000, 7000, 6000) to see what works, the threshold will be lower allowing a signal that is farther off frequency to be recognized. The more the frequency is off and the more physically distance the lower the signal.

[giddyhup]

@rct, thank you for your helpful tips, especially the one regarding CPU conservation. I think I already found the best frequency, I use the median between all sensors' signals as determined using SDR#'s waterfall diagram.

I use the same command line parameters with your code and with the original. There seems to something else. Problematic for is me is the recent modularization, I would like to modify the Prologue output to be in the same tabular style as you did it for Lacrosse sensors but I don't find the Lacrosse code in your branch. It might be my unfamiliarity with coding and github.

Lastly, with your change the actually issue is solved, so it could be closed. If you are willing to, where could we continue this exchange?

[merbanan]

We have a google group, search for rtl_433 there.

[rct]

@merbanan - Did you list the google group some place, should it be added to the read me?

Also, Maybe I'm misinterpreting but it looks like the google group is private, but anyone can join (or do you have to approve the join requests)?

[rct]

@giddyhup - yes, too much info winds up in issues that get closed.

FYI, right now the LaCrosse call back is the only one sending it's data to STDOUT. All of the of the other output goes to STDERR, which I guess makes some sense given the somewhat fluid/experiment nature.

I made the LaCrosse output be single line, with time stamps to be easy to parse by other code that the output is piped to.

Let's continue this in the google group.

[giddyhup]

I do not know why the Lacrosse plugin is disabled in the current version but I got everything to work. Thank you for all your support and feedback. I changed the Prologue output in my version to be in one line, maybe somebody wants to do the same:

fprintf(stderr, "Prologue (%d bits); ",bits_per_row[1]);
fprintf(stderr, "button: %d; ",bb[1][1]&0x04?1:0);
fprintf(stderr, "battery: %s; ",bb[1][1]&0x08?"Ok":"Low");
fprintf(stderr, "temp: %s%d.%d; ",temp2<0?"-":"",abs((int16_t)temp2/10),abs((int16_t)temp2%10));
fprintf(stderr, "humidity: %d; ", ((bb[1][3]&0x0F)<<4)|(bb[1][4]>>4));
fprintf(stderr, "channel: %d; ",(bb[1][1]&0x03)+1);
fprintf(stderr, "id: %d; ",(bb[1][0]&0xF0)>>4);
rid = ((bb[1][0]&0x0F)<<4)|(bb[1][1]&0xF0)>>4;
fprintf(stderr, "rid: %d; ", rid);
fprintf(stderr, "hrid: %02x; ", rid);

fprintf(stderr, "sig: %02x %02x %02x %02x %02x\n",bb[1][0],bb[1][1],bb[1][2],bb[1][3],bb[1][4]); 

[pclov3r]

So I built rtl_433 on windows and the LaCrosse temp decode doesn't work. It does work fine on linux however. There is just no output from the sesnor. Appears to be fine in -a debugging mode.

Seems to not work as well using the data sample files. Works fine in Linux of course.

Any ideas?

EDIT: It looks like decoding previous recorded samples doesn't work on windows. Suggestions?

[Batilan]

@pclov3r What exact method of compiling did you use (Cygwin or native Visual Studio?). You used the same github master branch for compiling under windows as you did for Linux I assume?

The fact that you get output from "rtl_433 -a" very likely proves that the rtlsdr device is recognized and librtlsdr is OK.

Can you send the output from "rtl_433 -a" including the first lines which go like this:

Using device 0: Generic RTL2832U OEM
Found Rafael Micro R820T tuner
Exact sample rate is: 250000.000414 Hz
Sample rate set to 250000.
Sample rate decimation set to 0. 250000->250000
Bit detection level set to 10000.
Tuner gain set to Auto.
Reading samples in async mode...
Tuned to 433920000 Hz.

If the rtl_433 -a output is OK and the prologue sensor is registered then there shouldn't be an issue unless something weird is going on within librtlsdr (or you are receiving the data from a different place which might need adjusting the level setting with the -l option.)

[pclov3r]

@Batilan upon more testing and messing with bit detection it does work.
What doesn't work is decoding preciously recorded data files. This works
fine on Linux.

Shows all the other output except the expected data.

Also the symbols used before Fahrenheit and Celsius show random garbage for
some reason. Perhaps Windows doesn't handle these correctly?

I'm using Dev-c++ which uses MinGW.

[pclov3r]

To be more specific the random garbage is with a Oregon Scientific sensor. I will try with a LaCorsse sensor later and see what happens

Here is an example of the output windows gives.

Thermo Sensor THN132N, Channel 1, Battery: Ok, Rolling-code 0xA1, Temp: 7.0°
C 44.6°F

[rct]

@pclov3r, You could be seeing a character set issue. I recall that for some of the sensors, someone put a degree symbol in the fprintf using UTF-8. Your windows console is most likely not UTF-8, and/or the compiler doesn't think the source file is UTF-8.

(Personally I would avoid non-ascii unless it's really presentation)

[rct]

@Batilan - may not be an issue you are seeing but just a reminder, #66 - if there isn't long enough silence in the recorded files (here I was talking about the -t separated files, the call back never occurs, because the limit isn't hit. This can be worked around by using lower reset limits.

[rct]

I (and I think others) would be very happy if some very brief Windows compilation instructions were made available. (I mostly care about Linux, but I'd like to test changes and other things on Windows)

[Batilan]

@rct, @pclov3r , Indeed, the garbled characters are caused by the (UTF-8) character code for degrees I think, I would also vote for removing them from the code, as rtl_433 is more about extracting the right values from the signals and less about presenting them nicely.

Also I'm very curious about compiling for Windows, Have downloaded and installed mingw, haven't yet tried to build rtl_433. I also would like to test building and running the sofwtare in a Windows environment to assure portability to the platform.

About the Windows rtl_433 not giving output as oposed to Linux working OK, as I understand now from @pclov3r this is resolved now by adjusting the bit detection level (was previously not working on Windows, so maybe indeed is related to location of receiving the signal?).

About the Windows version not working for recorded data samples: I guess the logic w.r.t. the reset limit is the same for Windows and Linux versions? This shouldn't differ then for different platforms? What type of sample files are we talking about (the earlier mentioned '-a -t'? I' didn't yet dive into the details of the sample formats, but are all samples binary compatible for all platforms?

[pclov3r]

@Batilan It seems to record data samples fine on windows but it doesn't display the output data when you attempt to decode the data from them.

If you build it you for windows i'm sure you'd see the issue.

Oh also it appears I can't build a 32-bit version and only a 64-bit version. Probably some unrelated issue to rtl_433.

I'm going to start a new issue since this is unrelated to LaCrosse now.

See #91 for the windows related issues.

[giddyhup]

Thank you all for working together on getting LaCrosse sensors recognized, especially @rct. I'll close this issue and hope we can continue in the Google group.