Better handling of RS41-SGM Sondes

[darksidelemm]

R-series Vaisala RS41s are being detected, but no GPS data is being extracted.
An audio sample of a sonde that has this issue is here: https://rfhead.net/sondes/brokenrs41.wav
Download and pipe into rs41dm_dft (rs41ecc in auto_rx/) using;
cat brokenrs41.wav | ./rs41ecc
Only the serial number will be shown.

[darksidelemm]

A longer sample is here: https://rfhead.net/sondes/brokenrs41_2.wav
Observations:

• GPS frame CRCs are failing (all 3 of them)
• iTOW and Week number are all over the place.

Did something change in the packet format?! Or are the sondes just broken?

[darksidelemm]

Additional information:
The sonde in the air over Adelaide is now showing a decrementing burst-kill counter. This implies that the sonde has burst, and that the onboard GPS is working correctly.
Additional sample of the sonde on descent here: https://rfhead.net/sondes/brokenrs41_3.wav

[darksidelemm]

Another interesting observation:
On a 'working' radiosonde (sample from a previous launch), we get cal sub-frame 0x32 (which contains the burst/countdown timer values) about once every 50 frames, like so:

[ 7854]  0x32: ec 72 80 00 5b 02 07 00 f9 ff a5 01 1b 79 00 00 [OK] : countdown timer 0x72ec = 29420sec = 490.3min
[ 7905]  0x32: b9 72 80 00 5b 02 07 00 fb 01 a4 01 1b 7c 00 00 [OK] : countdown timer 0x72b9 = 29369sec = 489.5min
[ 7956]  0x32: 86 72 80 00 5b 02 07 00 fd 03 a4 01 1b 7f 00 00 [OK] : countdown timer 0x7286 = 29318sec = 488.6min
[ 8007]  0x32: 53 72 80 00 5b 02 07 00 fe 05 a3 01 1b 82 00 00 [OK] : countdown timer 0x7253 = 29267sec = 487.8min

We usually get the other calibration frames in sequence.

On the above 'broken sonde' sample, we are getting the 0x32 sub-frame with EVERY frame:

[ 6385]  0x32: e5 76 8c 01 5d 02 07 00 f9 f5 bc 01 1b 17 00 00 [OK] : countdown timer 0x76e5 = 30437sec = 507.3min
[ 6387]  0x32: e3 76 8c 01 5d 02 07 00 f9 f5 bc 01 1b 17 00 00 [OK] : countdown timer 0x76e3 = 30435sec = 507.2min
[ 6388]  0x32: e2 76 8c 01 5d 02 07 00 f9 f5 bc 01 1b 17 00 00 [OK] : countdown timer 0x76e2 = 30434sec = 507.2min
[ 6389]  0x32: e1 76 8c 01 5d 02 07 00 f9 f5 bc 01 1b 17 00 00 [OK] : countdown timer 0x76e1 = 30433sec = 507.2min
[ 6390]  0x32: e0 76 8c 01 5d 02 07 00 f9 f5 bc 01 1b 17 00 00 [OK] : countdown timer 0x76e0 = 30432sec = 507.2min
[ 6391]  0x32: df 76 8c 01 5d 02 07 00 f9 f5 bc 01 1b 17 00 00 [OK] : countdown timer 0x76df = 30431sec = 507.2min
[ 6392]  0x32: de 76 8c 01 5d 02 07 00 f9 f5 bc 01 1b 17 00 00 [OK] : countdown timer 0x76de = 30430sec = 507.2min
[ 6393]  0x32: dd 76 8c 01 5d 02 07 00 f9 f5 bc 01 1b 17 00 00 [OK] : countdown timer 0x76dd = 30429sec = 507.1min`

This suggests there may be something very very wrong with the radiosonde.

[darksidelemm]

Further information:
The packets I am seeing contain a 'new' 0x80 frame, containing 167 bytes of data. I haven't seen any reference to this frame type before. This would explain why rs41dm_dft is reporting gibberish GPS data, as it's using hardcoded offsets into the frame, instead of looking for the header and frame length.

Still, this 0x80 frame type appears to be a bit of a mystery!

Got frame type: 79, length: 40, CRC:OK
Got frame type: 80, length: 167, CRC:OK
Got frame type: 76, length: 44, CRC:OK
{'content': '\xf9\x18R0230556\x1a\x00\x00\x03\x00\x03\x13\x00\x00&\x00\x0722\xddv\x8c\x01]\x02\x07\x00\xf9\xf5\xbc\x01\x1b\x17\x00\x00', 'crc': '\x90\t', 'type': '79', 'len': 40}
{'content': '\xe0\xf0\x0bE\x0f\xba\xb8.\xa9\x1fK\xc3\xa9\x90\x9d\xa9\x922\n\xf6\xdb\x00q\xa0\xc5pm\x03\xc4\xdcC\xb5.\x1e\xf1\x11tp#g\xea\xd9T`W\x97@\xbd<\xc0vgo\xb69&)\xc1a\xfe\x1fN\xc7\x81v\xca,\xaf\x1aR1\x1e\xe3\xab\x8e\xc0\x92\xbaA7\xa0\xdf\t\xe3\xa6\xdf\xa6\xcdJ\x14\xd0\xc9\xbd}\xa6\xa5\x96\x8cc\x85\xeb\x9c\x08\xbap\x10qV\xd3\x9c<z\x15G\xc8\x19\x8aG\x1b{\xec!\x08\xd3XS\xd7\xcb\x1d\xac-\xf89C\x01\x9d\xb7$R\xc2,y\x1f\xc4\xce-\x10u\xff)\xda<\xf8/\xafG\xa6\x81\x01H\x97\x0bPd\xbc\xff\xa8', 'crc': 'J,', 'type': '80', 'len': 167}
{'content': '\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', 'crc': 'x\x9a', 'type': '76', 'len': 44}

[rs1729]

Looks like RS41-SGM.

[ 4250] (R0230556) # 7928[0] 80A7[0] 762C[0]
[ 4251] (R0230556) # 7928[0] 80A7[0] 762C[0]
[ 4252] (R0230556) # 7928[0] 80A7[0] 762C[0]
[ 4253] (R0230556) # 7928[0] 80A7[0] 762C[0]
[ 4254] (R0230556) # 7928[0] 80A7[0] 762C[0]
[...]

Encrypted block in 0x80A7. When there is no standard gps-block/pck, the decoder displays only the pck_ids and crc_check (0==OK).

EDIT/remark:
The pck-crc info is only displayed when using the "--crc" option:
./rs41ecc --ecc --crc <audio.wav>

[darksidelemm]

Renamed this issue to reflect the need for better handling of RS41-SGMs. This will be fixed in 2 steps:

• Updating to a newer upstream demod (the new 'mod' decoders) which will indicate the presence of encrypted telemetry, and
• (somehow...) add the frequency of an encrypted sonde to a temporary blacklist, to avoid the scan loop continually re-detecting it.

[darksidelemm]

Resolved in https://github.com/projecthorus/radiosonde_auto_rx/releases/tag/v1.0.3

[rs1729]

Looks like the RS41-SGM can transmit also plain text data without encryption.
https://www.fingers-welt.de/phpBB/viewtopic.php?f=14&t=43&start=2950#p272805
After radio silence it transmitted two data sets per frame, then transitioned to normal (live?) mode, one data set per frame. The data set packets are almost like the standard RS41-SG(P) packets. Only the usual 7A2A-PTU packet is now a reduced 7F1B-packet with temperature and humidity (afaik rs41-sgm has no pressure sensor).
I have a modified version rs41_sgm.c that seems to work in both worlds, don't know yet how to include this (because of crc-handling of frames that could not be corrected). I guess there won't be many of these unencrypted rs41-sgm transmissions anyway.

[darksidelemm]

Probably not worth trying to add it into auto_rx, given their rarity anyway.
However, that's really interesting information, and might give away some plaintext information about what's within that encrypted block.

[rs1729]

The unencrypted version has the packets
(7928) [7F1B 7C1E 7D59 7B15] (7620)
7928 and 76xx are not encrypted also in the encrypted version:
(7928) [80A7] (762C)
Maybe just a coincidence, but the length of the data in [...] is the same:
1B+1E+59+15=A7
But the calibration data in 7928 is missing in the encrypted version, only sub-packet 0x32.
Maybe it is transmitted only at the beginning, or it is only known in the base station (the key should not be in the transmitted data as well ...), or it is in the 80A7-packet (don't know if the whole GPS2-7D59-packet is really needed).

[darksidelemm]

Interestingly I've just been informed that there will be some unencrypted RS41-SGM's flying near here soon... I'll try and grab some data.

[AlexM9999]

I have a modified version rs41_sgm.c

Hi @rs1729
Have you any progress with the investigation? Is it possible to get and test this version?
I've got some recording from RS41-SGM and I'd like to check if I could to decode some unencrypted data.
Auto_rx decoder shows only serial, battery voltage and coundown timer data.

[darksidelemm]

If you are only seeing serial, battery voltage and countdown timer, then the actual positional data is encrypted.

We have long since had support for unencrypted RS41-SGMs, this has been in place since about May 2019.

[rs1729]

@AlexM9999
as @darksidelemm said, RS41-SGM was integrated in the decoder rs41mod.c some time ago.
If you have an audio recording, you can check if it is encrpyted or not.
E.g.

./rs41mod -v --auto --ptu2 rs41fm-1.wav 
[ 4573] (N3610115)   [80A7] (RS41-SGM) 
[ 4574] (N3610115)   [80A7] (RS41-SGM) 
[ 4575] (N3610115)   [80A7] (RS41-SGM) 
[ 4576] (N3610115)   [80A7] (RS41-SGM) 
[ 4577] (N3610115)   [80A7] (RS41-SGM) 
...

If it shows [80A7] it means that GPS and TU data is encrypted.
You can also output battery voltage with option -vv. The --json output that is used by auto_rx should also show "encrypted": true, e.g.

[ 4577] (N3610115) (2.8 V)   [80A7] (RS41-SGM) 
[ 4577]  0x32: ff ff c1 ec 5c 02 07 00 01 01 e2 01 3b 77 00 00  
{ "type": "RS41", "frame": 4577, "id": "N3610115", "datetime": "0000-00-00T00:00:00.000Z", "lat": 0.00000, "lon": 0.00000, "alt": 0.00000, "vel_h": 0.00000, "heading": 0.00000, "vel_v": 0.00000, "sats": 0, "bt": 65535, "batt": 2.80, "subtype": "RS41-SGM", "encrypted": true, "ref_datetime": "GPS", "ref_position": "GPS" }

Maybe you can find it somewhere in the auto_rx log files.

If it is unencrypted, it should show position and eventually temperature/humidity

./rs41mod -v --auto --ptu2 rs41fm-2.wav 
...
[ 1561] (L5130203)  Thu 2019-05-09 18:00:00.001  lat: 46.03186  lon: -77.21042  alt: 4288.00   vH: 26.7  D:  47.3  vV: 4.6   T=-6.6C  RH2=93% 
[ 1562] (L5130203)  Thu 2019-05-09 18:00:01.001  lat: 46.03202  lon: -77.21017  alt: 4292.74   vH: 25.8  D:  45.1  vV: 5.5   T=-6.6C  RH2=93% 
[ 1563] (L5130203)  Thu 2019-05-09 18:00:02.001  lat: 46.03218  lon: -77.20995  alt: 4298.80   vH: 24.8  D:  43.5  vV: 6.4   T=-6.7C  RH2=92% 
[ 1564] (L5130203)  Thu 2019-05-09 18:00:03.001  lat: 46.03234  lon: -77.20972  alt: 4303.95   vH: 24.6  D:  45.3  vV: 4.6   T=-6.7C  RH2=92% : RS41-SGM 
[ 1565] (L5130203)  Thu 2019-05-09 18:00:04.001  lat: 46.03250  lon: -77.20950  alt: 4309.45   vH: 24.2  D:  45.6  vV: 6.8   T=-6.7C  RH2=92% 
[ 1566] (L5130203)  Thu 2019-05-09 18:00:05.001  lat: 46.03265  lon: -77.20928  alt: 4316.07   vH: 23.8  D:  44.7  vV: 5.4   T=-6.8C  RH2=92% 
...

(Temperature/humidity is shown only after enough calibration data is collected.)
If you are not sure if it is something else (new), provide a recording of the signal or raw output.

./rs41mod -r --auto rs41fm.wav > rs41fm_raw.txt

But only if there are some frames showing [OK].

[AlexM9999]

Thanks @rs1729, @darksidelemm

If it shows [80A7] it means that GPS and TU data is encrypted.

Yes, I`ve got the same [80A7] message:

[ 7470] (T1620234) [80A7] (RS41-SGM)
[ 7471] (T1620234) [80A7] (RS41-SGM)

Thanks for clarification.

Maybe you can find it somewhere in the auto_rx log files.

Auto_rx finds out the crypted sonde once and bans the frequency without any additional details.
It would be great if there was an option in the config to continue monitoring such sonde, logging information about the telemetry that is unencrypted(eg. serial, ticks, freq, type, battery, poweroff timer).

Is there any publicly available information about the crypto algorithm used in RS41-SGM?

[darksidelemm]

We know what it is, and it is implemented well enough that trying to break it is probably not worth the effort.
I cannot and will not support any efforts to break the encryption. I do not want to be in any way responsible for giving away the location of personnel launching radiosondes, that don't want that location to be known.

[bazjo]

What was previously discussed though is using trilateration and ToF once the sonde is in the air, but that would require a widespread deployment of specialized hardware, and probably not worth the effort

[darksidelemm]

Yep... not something I would ever consider adding into auto_rx. That's a lot of complexity for little reward. There are sometimes very good reasons why someone is using encryption on a radiosonde launch (and sometimes not so good reasons).