Skip to content
Mark Jessop edited this page Mar 17, 2019 · 62 revisions

Automatic Radiosonde Receiver Utilities

This fork of rs1279's RS codebase provides a set of utilities ('auto_rx') to allow automatic reception and uploading of Radiosonde positions to multiple services, including:

Currently we support the following radiosonde types:

  • Vaisala RS92 (experimental support for the RS92-NGP)
  • Vaisala RS41
  • Graw DFM06/DFM09/DFM17/PS-15
  • Meteomodem M10 (Thanks Viproz!)

Support for other radiosondes may be added as required (please send us sondes to test with!)

A presentation on this software was given at Linux.conf.au 2019 by Mark and Michael, and is available on Youtube here: https://www.youtube.com/watch?v=YBy-bXEWZeM

This software is under regular development. Please update regularly to get bug-fixes and improvements!

The key changes from the RS master codebase are:

  • Addition of the auto_rx directory, containing the auto_rx automatic reception software
  • Modifications to the rs92/rs41/dfm decoders, to provide telemetry output in JSON format.

Improvements from the upstream RS codebase will be merged into this codebase when/where appropriate.

Please consider joining the Google Group to receive updates on new software features: https://groups.google.com/forum/#!forum/radiosonde_auto_rx

Latest Updates

  • 2018-06-28 - Add basic web interface using Flask and Flask-SocketIO. This requires the installation of some additional dependencies, which can be performed by running:
    • $ sudo apt-get install python-pip
    • $ sudo pip install flask flask-socketio
  • 2019-02-27 - Fixed support for Graw DFM-17 and PS-15 sondes, updated APRS Type= field for DFM sondes.
  • 2019-03-13 - Major update improving detection performance by ~5dB for RS41/RS92 and DFM sondes, and fixing a lot of bugs. Please update!
  • 2019-03-16 - Add RS41 Burst Timer support, and fix some whitelist scan issues.

Also, all users please note that in a few weeks time we will be progressively blocking telemetry uploads to sondehub.org from older versions of auto_rx (older than 2019-02-27). We are planning on making changes to the telemetry formats uploaded to sondehub, and need to ensure all users are on a common version before this occurs to avoid issues.

1. General Theory of Operation

This software performs the following steps:

  1. Use rtl_power to scan across a user-defined frequency range, and detect peaks in the spectrum.
  2. For each detected peak frequency, run the rs_detect utility, which determines if a radiosonde signal is present, and what type it is. This detection is performed using correlation, and will detect a sonde many dB below the threshold where a decode is possible.
  3. If a radiosonde signal is found, start demodulating it, and upload data to various internet services.
  4. If no peaks are found, or if no packets are heard from the radiosonde in a given amount of time (3 minutes by default), go back to step 1.

The latest versions can make use of multiple RTLSDRs to allow for tracking of many radiosondes simultaneously. The number of simultaneous radiosondes you can track is limited only by the number of RTLSDRs you have setup!

By running auto_rx continuously, not just at known radiosonde launch times, you may see other radiosonde launches (military or otherwise) that would otherwise go un-noticed. Here in Australia, we discovered the Bureau of Meteorology's Ozone sounding schedule this way (which was later confirmed by the Bureau - thanks guys!).

2. Hardware Requirements

  • Ideally, a standalone single-board computer (Raspberry Pi 2/3, ODroid, etc...) to run the software. Other Linux machines will also work.
  • A RTLSDR receiver.
    • A receiver with a TCXO is mandatory, and you must make sure the PPM offset value is set correctly (see here). The demodulators are quite sensitive to frequency offsets, and the warm-up drift of the non-TCXO RTLSDRs is enough to throw it off. I've had great success with the RTL-SDR.com 'v3' dongles.
  • An antenna suitable for receiving on 400-406 MHz (a basic 1/4 wave monopole is usually good enough). If you have an 70cm amateur-band antenna, that will probably work fine too.
  • For optimal receive performance, a preamplifier and a band-pass filter, like this one. You can still achieve pretty good results without one.

3. Setup & Configuration

This section is intended to assist with setup under a fresh Raspbian installation. The instructions should be usable on other Debian-based systems.

For a good guide on setting up a 'headless' (no display) Raspberry Pi 2/3/Zero/ZeroW, look here.

3.1. Software Dependencies

Update your system, then install the required dependencies using:

$ sudo apt-get update
$ sudo apt-get upgrade
$ sudo apt-get install python-numpy python-setuptools python-crcmod python-requests python-dateutil python-pip sox git build-essential cmake usbutils libusb-1.0-0-dev rng-tools

We then need to grab a few packages using Pip:

$ sudo pip install flask flask-socketio

A few errors may be shown while pip does it's thing. The packages we need should still be installed however.

Note: If you have either gevent or eventlet installed, these will need to be uninstalled so that the Flask-SocketIO does not try to use it. If you didn't understand that sentence then you're probably OK. If you did understand it, then you'll need to uninstall gevent and/or eventlet until we figure out a workaround. Sorry :-(

3.2. RTL-SDR from source

As we may we wish to use the bias-tee option in the newer v3 RTLSDRs, we need a fairly recent version of the rtl-sdr software. Recent ubuntu distributions provide this support in the rtl-sdr package (package version 0.5.3-12 or newer), however at the time of writing (2018-05-29), Raspbian does not.

If your package manager's version of rtl-sdr does not provide bias-tee support (no -T option), then you will need to compile from source:

$ git clone https://github.com/osmocom/rtl-sdr.git
$ cd rtl-sdr
$ mkdir build
$ cd build
$ cmake -DINSTALL_UDEV_RULES=ON -DDETACH_KERNEL_DRIVER=ON ../
$ sudo make install
$ sudo ldconfig

IMPORTANT NOTE: Make sure to remove any installations of rtl-sdr already on your system before compiling/installing from source. (i.e. sudo apt-get remove rtl-sdr librtlsdr0 librtlsdr-dev

3.2.1 RTL-DVB Kernel Module Blacklisting

Because the kernel rtl_dvb driver (yes, these SDRs can be used to watch TV after all) will inhibit our intended use, we need to blacklist a few kernel modules.

If you installed rtl-sdr from source as above, then a blacklist file should have already been installed, otherwise create a new module blacklist file using:

$ sudo nano /etc/modprobe.d/rtlsdr-blacklist.conf

and add the lines:

blacklist dvb_usb_rtl28xxu
blacklist rtl2832
blacklist rtl2830
blacklist dvb_usb_rtl2832u
blacklist dvb_usb_v2
blacklist dvb_core

Save the file (Ctrl+x, y), and reboot the Raspberry Pi.

3.2.2 - Check your RTLSDRs are working!

At this point it is worth checking that you can communicate with your RTLSDR, which can be achieved by running:

$ rtl_test

Found 1 device(s):
  0:  Realtek, RTL2838UHIDIR, SN: 00000002

Using device 0: Generic RTL2832U OEM
Found Rafael Micro R820T tuner
Supported gain values (29): 0.0 0.9 1.4 2.7 3.7 7.7 8.7 12.5 14.4 15.7 16.6 19.7 20.7 22.9 25.4 28.0 29.7 32.8 33.8 36.4 37.2 38.6 40.2 42.1 43.4 43.9 44.5 48.0 49.6
[R82XX] PLL not locked!
Sampling at 2048000 S/s.

Info: This tool will continuously read from the device, and report if
samples get lost. If you observe no further output, everything is fine.

Reading samples in async mode...

Hit Ctrl+C to kill rtl_test. If you see warnings such as lost at least <X> bytes, this indicates USB bandwidth issues, or some other issue with the RTLSDR.

3.3. Cloning & Configuring the Radiosonde Auto RX Repository

From your home directory, run:

$ git clone https://github.com/projecthorus/radiosonde_auto_rx.git
$ cd radiosonde_auto_rx/auto_rx
$ ./build.sh
$ cp station.cfg.example station.cfg

It is normal for a few warnings to appear when running build.sh.

Edit station.cfg (i.e. nano station.cfg) and modify the settings as required. For detailed information on configuration settings, please read through the Configuration Settings page.

3.4. Initial Testing

To check everything is operational, start up the automatic receiver script using:

(from within radiosonde_auto_rx/auto_rx)
$ python auto_rx.py

Depending on how you have configured the software, you should see output similar to the following:

2018-06-01 21:54:18,753 INFO:Reading configuration file...
2018-06-01 21:54:20,611 INFO:Config - Tested SDR #0 OK
2018-06-01 21:54:20,612 INFO:Telemetry Logger - Started Telemetry Logger Thread.
2018-06-01 21:54:20,621 INFO:APRS-IS - APRS Uploader Started.
2018-06-01 21:54:20,622 INFO:OziMux - Started OziMux / Payload Summary Exporter
2018-06-01 21:54:20,622 INFO:SDR #0 has been allocated to Scanner.
2018-06-01 21:54:22,480 INFO:Scanner #0 - Starting Scanner Thread
2018-06-01 21:54:22,486 INFO:Scanner #0 - Running frequency scan.
2018-06-01 21:54:24,883 INFO:Habitat - Listener information uploaded.

This indicates that the software is successfully scanning for radiosondes. The rest of the functionality can only be tested with a signal from a radiosonde. These are launched at around 11:15Z and 23:15Z at most major airports around the world.

When a sonde is found, the following output will be seen:

2018-06-01 21:56:12,077 INFO:Scanner #0 - Detected peaks on 1 frequencies (MHz): [402.5]
2018-06-01 21:56:17,521 INFO:Detected new RS41 sonde on 402.500 MHz!
2018-06-01 21:56:17,521 INFO:Halting Scanner to decode detected radiosonde.
2018-06-01 21:56:17,521 INFO:Scanner #0 - Waiting for current scan to finish...
2018-06-01 21:56:25,686 INFO:Scanner #0 - Scanner Thread Closed.
2018-06-01 21:56:25,687 INFO:SDR #0 has been allocated to Decoder (RS41, 402.500 MHz).
2018-06-01 21:56:27,556 INFO:Decoder #0 RS41 402.500 - Starting decoder subprocess.
2018-06-01 21:56:30,868 INFO:Telemetry Logger - Opening new log file: ./log/20180601-113422_N3740572_RS41_402500_sonde.log
2018-06-01 21:56:37,172 INFO:Habitat - Listener information uploaded.
2018-06-01 21:56:41,747 INFO:APRS-IS - Uploaded to APRS-IS: ;N3740572 *111111z3509.05S/13854.80EO116/046/A=053079 RS41 Radiosonde 402.500 MHz 5.2m/s http://bit.ly/2Bj4Sfk
...

Depending on what exporter options have been enabled (Habitat, APRS, OziPlotter), you will see indications when telemetry has been uploaded. If no exporters are enabled, you will only see telemetry information with the 'verbose' logging option enabled (-v).

A web interface which displays the log information as above, along with a sonde position map and frequency scan output, is available on port 5000, i.e. http://127.0.0.1:5000/

To shut-down the automatic reception script hit Ctrl-C, and the threads will eventually close.

When a Radiosonde is successfully detected, a log-file will be created in the auto_rx/log/ directory, containing the telemetry for that flight.

3.5. Automatic / Continuous Operation

3.5.1. Option #1 - Operation as a systemd Service (RECOMMENDED)

auto_rx can be operated in a 'continuous' mode, running as a systemd service.

To set this up, the auto_rx.service file (located in radiosonde_auto_rx/auto_rx) must be edited to include your username, and the path to this directory.

$ sudo cp auto_rx.service /etc/systemd/system/
$ sudo nano /etc/systemd/system/auto_rx.service

If you are not running auto_rx as the 'pi' user, you will need to edit the auto_rx.service file and modify the ExecStart, WorkingDirectory and User fields. Otherwise, leave all settings at their defaults:

[Unit]
Description=auto_rx
After=syslog.target

[Service]
ExecStart=/usr/bin/python /home/pi/radiosonde_auto_rx/auto_rx/auto_rx.py -t 0
Restart=always
RestartSec=3
WorkingDirectory=/home/pi/radiosonde_auto_rx/auto_rx/
User=pi

[Install]
WantedBy=multi-user.target

Once/if edited, install and start the service using:

$ sudo systemctl enable auto_rx.service
$ sudo systemctl start auto_rx.service

The log output can be viewed buy running:

$ sudo journalctl -u auto_rx.service -f -n

To stop the service, simply run:

$ sudo systemctl stop auto_rx.service

3.5.2. Option #2 - Operation via Crontab (Backup option if systemd not available)

As radiosonde launch times are quite predictable, auto_rx.py can be started around launch-time using a crontab entry.

Note that when a process runs within cron, it doesn't have access to the user's $PATH environment variable, and hence utilities like rtl_power and rtl_fm will not be found. This can be fixed by declaring the PATH environment variable within the crontab. An example crontab is below:

PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/games:/usr/games`
15 11 * * * /home/<your_username>/radiosonde_auto_rx/auto_rx/auto_rx.sh
15 23 * * * /home/<your_username>/radiosonde_auto_rx/auto_rx/auto_rx.sh

You will also need to edit line 11 of auto_rx.sh to have the script change into the correct directory.

You can’t perform that action at this time.