Problem testing my own ADS-B modulator

[krono-i2]

Hello!
Thanks for you work in implementing this wonderful ADS-B demodulator.
I'm testing my own ADS-B modulator using your demodulator in loopback through gnuradio-companion.
On my laptop I obtain good results, but when I try the loopback on different machines I obtain this output:

/usr/local/lib/python2.7/dist-packages/adsb/framer.py:162: RuntimeWarning: divide by zero encountered in float_scalars snr = 10.0*np.log10(in0[pulse_idx]/np.median(in0[(pulse_idx - NUM_NOISE_SAMPLES):pulse_idx])) + 1.6 /usr/local/lib/python2.7/dist-packages/adsb/demod.py:107: RuntimeWarning: divide by zero encountered in divide self.bit_confidence = 10.0*np.log10(bit1_amps/bit0_amps) /usr/local/lib/python2.7/dist-packages/adsb/demod.py:107: RuntimeWarning: divide by zero encountered in log10 self.bit_confidence = 10.0*np.log10(bit1_amps/bit0_amps)

There is a problem with my modulator software?
Thank you.

Ivan

[mhostetter]

Does your modulated signal have zero noise, i.e. perfect 1s and 0s for the PPM bits? If so, that would explain the divide-by-zero error. Try adding some small Gaussian noise and see if this error goes away. I should probably catch/avoid this error regardless, though.

[krono-i2]

Right! I solved warnings adding noise.
Now I can't understand why my modulator works fine on my laptop but not in my virtual machine.
It's a strange phenomenon: when I start loopback on my laptop I obtain an error on the first two received packets, successively information is decoded:

----------------------------------------------------------------------
SNR:            35.82 dB
DF:             17 Extended Squitter
CRC:            Passed
CA:             5 Level 2 or Above Transponder, Can Set CA 7, In Air
AA:             a32dea
TC:             11 Airborne Position
valid_lat_lon False
lat_cpr 62229
lon_cpr 90297
lat nan
lon nan
SS:             0 No Condition Information
Time:           0 Not Synced to 0.2s UTC Epoch
Latitude:       N/A N
Longitude:      N/A E
Altitude:       225 ft
SCHED_EVENT: time=1123000, len=240  


----------------------------------------------------------------------
SNR:            32.97 dB
DF:             17 Extended Squitter
CRC:            Passed
CA:             5 Level 2 or Above Transponder, Can Set CA 7, In Air
AA:             a32dea
TC:             11 Airborne Position
valid_lat_lon False
lat_cpr 74514
lon_cpr 83427
lat -33.1413760428
lon 19.3701856963
SS:             0 No Condition Information
Time:           0 Not Synced to 0.2s UTC Epoch
Latitude:       -33.1413760 N
Longitude:      19.3701857 E
Altitude:       225 ft
SCHED_EVENT: time=2127000, len=240  


----------------------------------------------------------------------
SNR:            33.29 dB
DF:             17 Extended Squitter
CRC:            Passed
CA:             5 Level 2 or Above Transponder, Can Set CA 7, In Air
AA:             a32dea
TC:             11 Airborne Position
SS:             0 No Condition Information
Time:           0 Not Synced to 0.2s UTC Epoch
Latitude:       -33.1413760 N
Longitude:      19.3701857 E
Altitude:       225 ft
SCHED_EVENT: time=3126000, len=240

On the virtual machine I obtain:

----------------------------------------------------------------------
Datetime:       2019-10-25 14:47:30.196821 UTC
SNR:            33.40 dB
DF:             0 Short Air-Air Surveillance (ACAS)
CRC:            Failed (Unrecognized AA from AP)
AA:             000000
SCHED_EVENT: time=1230000, len=240  


----------------------------------------------------------------------
Datetime:       2019-10-25 14:47:30.696321 UTC
SNR:            32.28 dB
DF:             0 Short Air-Air Surveillance (ACAS)
CRC:            Failed (Unrecognized AA from AP)
AA:             5d5222
SCHED_EVENT: time=2233000, len=240  


----------------------------------------------------------------------
Datetime:       2019-10-25 14:47:31.197821 UTC
SNR:            33.25 dB
DF:             0 Short Air-Air Surveillance (ACAS)
CRC:            Failed (Unrecognized AA from AP)
AA:             84faa0
SCHED_EVENT: time=3237000, len=240  


----------------------------------------------------------------------
Datetime:       2019-10-25 14:47:31.699821 UTC
SNR:            33.35 dB
DF:             0 Short Air-Air Surveillance (ACAS)
CRC:            Failed (Unrecognized AA from AP)
AA:             59beb0
SCHED_EVENT: time=4238000, len=240

P.S. Don't consider the "SCHED_EVENT", I'm using a burst scheduler to modulate the signal.

[mhostetter]

Can you attach screenshots of the working GRC flowgraph and the failing flowgraph? Are you sending packets across a network? Or are you running the exact same flowgraph on the VM and it doesn’t work?

…

On Fri, Oct 25, 2019 at 10:48 AM Ivan Iudice ***@***.***> wrote: Right! I solved warnings adding noise. Now I can't understand why my modulator works fine on my laptop but not in my virtual machine. It's a strange phenomenon: when I start loopback on my laptop I obtain an error on the first two received packets, successively information is decoded: `---------------------------------------------------------------------- SNR: 35.82 dB DF: 17 Extended Squitter CRC: Passed CA: 5 Level 2 or Above Transponder, Can Set CA 7, In Air AA: a32dea TC: 11 Airborne Position valid_lat_lon False lat_cpr 62229 lon_cpr 90297 lat nan lon nan SS: 0 No Condition Information Time: 0 Not Synced to 0.2s UTC Epoch Latitude: N/A N Longitude: N/A E Altitude: 225 ft SCHED_EVENT: time=1123000, len=240 ------------------------------ SNR: 32.97 dB DF: 17 Extended Squitter CRC: Passed CA: 5 Level 2 or Above Transponder, Can Set CA 7, In Air AA: a32dea TC: 11 Airborne Position valid_lat_lon False lat_cpr 74514 lon_cpr 83427 lat -33.1413760428 lon 19.3701856963 SS: 0 No Condition Information Time: 0 Not Synced to 0.2s UTC Epoch Latitude: -33.1413760 N Longitude: 19.3701857 E Altitude: 225 ft SCHED_EVENT: time=2127000, len=240 ------------------------------ SNR: 33.29 dB DF: 17 Extended Squitter CRC: Passed CA: 5 Level 2 or Above Transponder, Can Set CA 7, In Air AA: a32dea TC: 11 Airborne Position SS: 0 No Condition Information Time: 0 Not Synced to 0.2s UTC Epoch Latitude: -33.1413760 N Longitude: 19.3701857 E Altitude: 225 ft SCHED_EVENT: time=3126000, len=240On the virtual machine I obtain: ---------------------------------------------------------------------- Datetime: 2019-10-25 14:47:30.196821 UTC SNR: 33.40 dB DF: 0 Short Air-Air Surveillance (ACAS) CRC: Failed (Unrecognized AA from AP) AA: 000000 SCHED_EVENT: time=1230000, len=240 ------------------------------ Datetime: 2019-10-25 14:47:30.696321 UTC SNR: 32.28 dB DF: 0 Short Air-Air Surveillance (ACAS) CRC: Failed (Unrecognized AA from AP) AA: 5d5222 SCHED_EVENT: time=2233000, len=240 ------------------------------ Datetime: 2019-10-25 14:47:31.197821 UTC SNR: 33.25 dB DF: 0 Short Air-Air Surveillance (ACAS) CRC: Failed (Unrecognized AA from AP) AA: 84faa0 SCHED_EVENT: time=3237000, len=240 ------------------------------ Datetime: 2019-10-25 14:47:31.699821 UTC SNR: 33.35 dB DF: 0 Short Air-Air Surveillance (ACAS) CRC: Failed (Unrecognized AA from AP) AA: 59beb0 SCHED_EVENT: time=4238000, len=240` P.S. Don't consider the "SCHED_EVENT", I'm using a burst scheduler to modulate the signal. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#31?email_source=notifications&email_token=AC4NFPPD5I65BDQQH6LYLCLQQMBNPA5CNFSM4JE7QXLKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECISYIQ#issuecomment-546384930>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AC4NFPP3QEDBLD4D72GNCM3QQMBNPANCNFSM4JE7QXLA> .

[krono-i2]

I'm using exactly the same flowgraph on the host and on the VM.
Packets are sent across the network, and I use exactly the same source (on the VM).
Find attached the screenshots.

[mhostetter]

I just realized you’re running two different versions of the code. The VM output is newer. Also, it looks like the laptop code is running “Extended Squitter Only” on the ADS-B Decoder block and the VM is running “All Messages”. The VM output isn’t an error. It’s just saying it can’t verify the CRC on the DF 0 message because the alleged AA hasn’t been seen before, so it assumes the CRC residual is an error and fails the CRC. You need to receive a DF 17 message with that AA before the decoder will accept the CRC. The reason you don’t see the DF 0 bad CRCs on your laptop is because I believe it’s in “Extended Squitter Only” mode (DF 17 only). Keep running the code and you get stuff on the VM. Also, please verify the decoder settings between the two flowgraphs.

…

On Fri, Oct 25, 2019 at 2:09 PM Ivan Iudice ***@***.***> wrote: I'm using exactly the same flowgraph on the host and on the VM. Packets are sent across the network, and I use exactly the same source (on the VM). Find attached the screenshots. [image: Screenshot from 2019-10-25 20-06-08] <https://user-images.githubusercontent.com/121834/67593775-397fa380-f763-11e9-8ee9-9a6108db2c62.png> [image: Screenshot from 2019-10-25 20-06-16] <https://user-images.githubusercontent.com/121834/67593786-3d132a80-f763-11e9-8e27-5d972a4331e2.png> — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#31?email_source=notifications&email_token=AC4NFPIWEZYPNXU53QZCHMTQQMY33A5CNFSM4JE7QXLKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECJEA7Q#issuecomment-546455678>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AC4NFPKAWEQU7GYJJTSEABLQQMY33ANCNFSM4JE7QXLA> .

[krono-i2]

I compiled the module on VM few days ago, on the host I compiled it before the summer, I think.
Both VM and laptop codes are running “Extended Squitter Only”.
My modulator does not generate DF 0 packets, I don’t know why the decoder on VM finds this format...

[mhostetter]

Hmm... How are you encoding the messages? Can you attach a screenshot of the output of the QT Time Sink that connects to the demod from the VM? I’m curious what the burst look like.

…

On Fri, Oct 25, 2019 at 4:00 PM Ivan Iudice ***@***.***> wrote: I compiled the module on VM few days ago, on the host I compiled it before the summer, I think. Both VM and laptop codes are running “Extended Squitter Only”. My modulator does not generate DF 0 packets, I don’t know why the decoder on VM finds this format... — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#31?email_source=notifications&email_token=AC4NFPKW2RKAHFULWDUPVV3QQNF4FA5CNFSM4JE7QXLKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECJM6ZA#issuecomment-546492260>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AC4NFPNQ4FE42OZXBGX2ZCLQQNF4FANCNFSM4JE7QXLA> .

[krono-i2]

It looks correct. Here you can see the VM and the host.

[mhostetter]

Hmmm. Are there any other differences between your laptop and VM, other than the VM code being newer? Could you try to update your laptop code to the latest master branch and retest? If that fails, then maybe there's a bug in master. I doubt it, but that's my best guess.

[krono-i2]

No differences on the flowgraphs.
I fear if I update my laptop code I would not have anymore a working setup...
How could I preserve the possibility to get back my current version?
Thank you.

Ivan

[mhostetter]

Run git log and save the commit hash that you're currently on. Then run git pull and rebuild the OOT module. If things go catastrophically wrong, you can run git reset <hash>, where <hash> is the hexadecimal string of the commit hash you saved from the git log command.

https://www.git-scm.com/docs/git-log
https://git-scm.com/docs/git-reset

[edersonrmachado]

Hi! I've also observed this output on the first two received packets, as krono86 has commented. So looking at the decoder.py script, (in gr-adsb/python folder), we can identify the part of the code that evaluates valid_lat_lon:

Looking at this part of the code, to be considered a valid_lat_lon decoded value, the subtraction value, resulting from the lat (calculated from cpr) and last lat decoded value from the same aircraft (self.plane_dict[self.aa_str]["latitude"]), must be less than 0.1 deg, to prevent that bad lat/lons get published. Then I tried to reproduce the krono86 test sending similar frames: just 1 (lat, lon) pair = 2 frames (even + odd). I sent it only once, (with 'Verbose' and "All messages" decoder options), as result I obtain:

----------------------------------------------------------------------
Datetime:       2020-10-19 20:52:16.589236 UTC
SNR:            17.06 dB
DF:             17 Extended Squitter
CRC:            Passed
CA:             5 Level 2 or Above Transponder, Can Set CA 7, In Air
AA:             a32dea
TC:             11 Airborne Position
valid_lat_lon False
lat_cpr 62448
lon_cpr 90479
lat nan
lon nan
SS:             0 No Condition Information
Time:           0 Not Synced to 0.2s UTC Epoch
Latitude:       N/A N
Longitude:      N/A E
Altitude:       225 ft


----------------------------------------------------------------------
Datetime:       2020-10-19 20:52:16.589756 UTC
SNR:            17.10 dB
DF:             17 Extended Squitter
CRC:            Passed
CA:             5 Level 2 or Above Transponder, Can Set CA 7, In Air
AA:             a32dea
TC:             11 Airborne Position
valid_lat_lon False
lat_cpr 74514
lon_cpr 83427
lat -33.1413760428
lon 19.3701856963
SS:             0 No Condition Information
Time:           0 Not Synced to 0.2s UTC Epoch
Latitude:       -33.1413760 N
Longitude:      19.3701857 E
Altitude:       225 ft

In a second test to see how these lat values vary and are taken into account in the code, I added some "prints" in the decoder.py script , shown below:

Sending the same (lat,lon) pair continuously, ( with 'Verbose' "All messages" and "Extended Squitter Only" decoder options)
results in:

----------------------------------------------------------------------
Datetime:       2020-10-19 22:50:09.095426 UTC
SNR:            18.69 dB
DF:             17 Extended Squitter
CRC:            Passed
CA:             5 Level 2 or Above Transponder, Can Set CA 7, In Air
AA:             a32dea
TC:             11 Airborne Position
valid_lat_lon False
lat_cpr 62448
lon_cpr 90479
lat nan
lon nan

###################### prints added from here ########################
lat = nan
self.plane_dict[self.aa_str]["latitude"] = nan
if(lat-self.plane_dict[self.aa_str]["latitude"])< 0.1 and (lat-self.plane_dict[self.aa_str]["latitude"])<0.1)
nan < 0.1?
no, so valid_lat_lon = False
############################ to here #################################

SS:             0 No Condition Information
Time:           0 Not Synced to 0.2s UTC Epoch
Latitude:       N/A N
Longitude:      N/A E
Altitude:       225 ft


----------------------------------------------------------------------
Datetime:       2020-10-19 22:50:09.095946 UTC
SNR:            18.63 dB
DF:             17 Extended Squitter
CRC:            Passed
CA:             5 Level 2 or Above Transponder, Can Set CA 7, In Air
AA:             a32dea
TC:             11 Airborne Position
valid_lat_lon False
lat_cpr 74514
lon_cpr 83427
lat -33.1413760428
lon 19.3701856963

###################### prints added from here ########################
lat = -33.1413760428
self.plane_dict[self.aa_str]["latitude"] = nan
if(lat-self.plane_dict[self.aa_str]["latitude"])< 0.1 and (lat-self.plane_dict[self.aa_str]["latitude"])<0.1)
nan < 0.1?
no, so valid_lat_lon = False
############################ to here #################################

SS:             0 No Condition Information
Time:           0 Not Synced to 0.2s UTC Epoch
Latitude:       -33.1413760 N
Longitude:      19.3701857 E
Altitude:       225 ft


----------------------------------------------------------------------
Datetime:       2020-10-19 22:50:09.096450 UTC
SNR:            18.77 dB
DF:             17 Extended Squitter
CRC:            Passed
CA:             5 Level 2 or Above Transponder, Can Set CA 7, In Air
AA:             a32dea
TC:             11 Airborne Position

###################### prints added from here ########################
lat = -33.1413760428
self.plane_dict[self.aa_str]["latitude"] = -33.1413760428
if(lat-self.plane_dict[self.aa_str]["latitude"])< 0.1 and (lat-self.plane_dict[self.aa_str]["latitude"])<0.1)
0.0 < 0.1?
yes, so valid_lat_lon = True
############################ to here #################################

SS:             0 No Condition Information
Time:           0 Not Synced to 0.2s UTC Epoch
Latitude:       -33.1413760 N
Longitude:      19.3701857 E
Altitude:       225 ft


----------------------------------------------------------------------

In summary in the if statement of this test we have:

1st frame (unknown AA): nan - nan = nan, valid_lat_lon = False
2nd frame from same AA: -33.14 - nan = nan, valid_lat_lon = False
3rd frame from same AA: -33.14 - (-33.14) = 0, valid_lat_lon = True

So, I not sure but I think that the first (lat,lon) pair received from an unknown AA will have always valid_lat_lon = False (both even and odd frames).

Also in the second component of the if statement it might have been in your interest to use the longitude value and not the latitude value again:

if(lat-self.plane_dict[self.aa_str]["latitude"])< 0.1 and (lat-self.plane_dict[self.aa_str]["latitude"])<0.1)

What do you think?

Here is my Gnuradio flowgraph used in the tests (I've connected directly the Tx channel to Rx channel of USRP2932 with a -30dB cable) :