-
Notifications
You must be signed in to change notification settings - Fork 81
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
some questions #6
Comments
The writing of the individual bursts was disabled [1]. I honestly don't recall why I did that, but I did =\ You can re-enable by uncommenting the linked line. You can also try out the Octave doesn't do nearly as good a job using multiple cores as MATLAB. It's crazy slow. To the extent that I created a C MEX extension to plow through the correlations faster. The main time suck is the normalized cross correlation done in [2]. There is a custom cross correlation function in use that's much faster than MATLAB's [1] https://github.com/proto17/dji_droneid/blob/gr-droneid/gnuradio/gr-droneid/lib/time_sync_impl.cc#L85 |
thanks for the clarification. Are we supposed to feed the shorter burst recordings (the ones that have been disabled) directly to the .m file or should I just stick with the full IQ recording? |
I think the bursts, but in my case none of my bursts ran individual or smashed together produce a result (at least for me) with the octave script. If instead I run the full IQ file that produced the bursts through octave, I hit jackpot. Be great to hear if you get results with just the bursts from your IQ file. |
I guess you are cemaxucuter, right? I watched your youtube vids, and in that you directly used the ~2GB IQ recording, so that was part of my confusion. Maybe @proto17 can clarify this specific issue further. |
I am not 100% sure why the |
That’s me :). I had a 8GB file at first.. that took a long while, so bursts would be great. |
I have done some extra experimentation with I have uncommented Line 85 in
Is this normal expected behavior? Another thing I notice is that when running I also tried running the IQ recording through the octave file. (I recorded a Mavic Air 2 with B210 @ 2.4295GHz with gain set to 0.8) Another thing I noticed is that droneid constellation detects some bursts even when there is no drone within the vicinity of the SDR. Is droneid somehow capturing WiFi/802.11 packets? Should I try to record outdoors far from other 2.4GHz sources? |
I moved away from WiFi and into an empty parking lot for captures once I noticed it was picking up WiFi (or it was just getting in my way) or so it seemed. Initially the flow graph crashed on me to, detailed in a now closed ticket. After some code updates by the developer the flow graph was much more stable and if I recall only stopped on me when I cranked the gain up real high (maybe unrelated). I recorded maybe 3-4 IQs which I’ve used to run back through the flow graph. I get bursts, but basically same results as you. However, running the entire IQ file(s) through octave produce numerous good frames that translate to accurate info. Very neat you have it all running in docker! |
@alphafox02 yeah I have experienced the same, high gain is probably the common denominator in the "crashes" though I don't why. I will check the closed ticket you mentioned. One other problem I have is calling |
The WiFi detection is quite interesting. I am not in a very WiFi heavy area, so maybe I never encountered that issue just due to not having a lot of activity there. I can say that high gain will cause crashes in the The original crashing bug was due to walking off the edge of an array. Make sure that you have the most up to date I suspect that the PATH variable in Octave doesn't know about
And to fix an issue with the
Edit: I'm not sure exactly how Octave gets its environment variables set when using
Which might force the use of Bash and will bring up the environment specified in |
@proto17 , yeah looks like octave's In terms of high gain; I have another question regarding the
but I don't get any hex frames dumped. When I get the hex frames they are always Also this is a trivial suggestion; but for ease of installation you might want to include |
Right now the frequency offset and equalization in GNU Radio isn't working well. In fact the equalization is turned off as it causes more problems than it solves. This means that the constellations are almost certainly going to be rotated too much for demodulation to work properly. I have improvements pending in MATLAB that once tested there will be moved over the the GNU Radio module. The GNU Radio module was not actually meant to do full demod. The original purpose was to just extract bursts so that I didn't have to make a multi-gigabyte recording that takes ages to process. But, that plan has shifted a bit to trying to make the GNU Radio code as robust as the MATLAB. All of that to say that the CRC failure you're seeing is because the frame CRC-24 failed [1]. You definitely should see some hex, so I'm not quite sure what's going on there [2]. So long as there isn't an error in the Turbo decoder/rate matcher you should get some bytes printed to the terminal. I didn't include As for the CMake, I do have Now that I think about it, I might be able to instruct CMake to clone the repo containing [1] https://github.com/proto17/dji_droneid/blob/gr-droneid-update/gnuradio/gr-droneid/lib/demodulation_impl.cc#L461 |
my bad, I didn't see libturbofec in cmake. Instead of just cloning the repo, I thought it might be more straightforward to just include the As I mentioned, terminal usually prints: Btw, have you tried running your extracted bursts through octave/matlab? When I do that I can't get any hex dumps. I either get:
or
|
The issue with running the bursts through MATLAB is that some amount of padding either side of the burst is required in order to do some of the operations. You might be able to get away with increasing the extracted sample count [1] and increasing the delay in the GNU Radio graph [2]. Perhaps setting [1] to something like
And setting [2] to something like
I'm not 100% sure this will work, but I think it will add more padding to either side of the burst and give MATLAB/Octave a better chance. [1] https://github.com/proto17/dji_droneid/blob/gr-droneid-update/gnuradio/gr-droneid/lib/extractor_impl.cc#L47 |
thanks for the answer! another thing I noticed is the flowgraph in |
Hmm, that's not ideal. I don't recall if there was a reason for this 😳 |
explanation for discrepancy:
|
So the older code gave more padding than the newer one. The newer graph doesn't add any real padding from the start of the burst which will likely make MATLAB angry. The symbol being used for correlation is the 4th symbol, so I think that the newer graph doesn't provide any additional padding at the start. I'd have to look at the plots to know for sure. As some background, the delay is all about stalling the complex samples long enough that by the time the correlation filter sees the spike, the delayed samples are some amount before the start of the burst. If the second OFDM symbol is the one that we are interested in, then without the delay the first sample we would see when the correlation spike is found would be the first (or last depending on the correlation) sample of the second OFDM symbol, losing the first symbol entirely. |
@proto17, I had the chance to look more into this project and I have some observations that might be of interest. I am using I have mainly observed two distinct Another thing about Also, this is slightly weird but I can only get the hex frames when the drone is sitting next to the USRP. If I move the drone something like 10 meters away then I can't seem to extract any bursts. I tried increasing the gain but that doesn't seem to do much (except pushing the gain all the way to 1.0 generates spurious bursts). Any recommendations regarding this observation? Should I maybe reduce the Thanks once again for your great work! |
Any intuition/ideas regarding this observation? Can lower correlation bursts still be leveraged? The I have not really put much effort into detections at a longer range. The original intent for the GNU Radio code was to extract bursts without having to record literal minutes of IQ at 30.72 MSPS to then have to run through MATLAB for several more minutes. The real limiting factor for the GNU Radio code is that there's no equalizer. The code is there [4] but when I try to use it things go all screwy. I stopped working on it because it became apparent that integer frequency offset was going to hose everything, so I focused more on trying to solve that problem. After several days of trying things I thought I had something that worked down to just a few dB SNR, but once I tried it on real data everything fell apart [1] https://github.com/proto17/dji_droneid/blob/main/matlab/updated_scripts/process_file.m#L37 |
Thanks for the detailed answer @proto17. I have changed the OOT module and made correlation threshold a parameter that can be controlled from GRC. Once the ZC correlation exceeds the threshold; the bursts are registered properly as you described. However, I did not notice much of an improvement just by dropping the correlation threshold, maybe qualitatively more bursts were getting detected but I am not exactly sure about that. I have tried with a correlation threshold of 0.1 and 0.01 and they both seemed to work fine. Can lower correlation bursts still be leveraged? Another thing I noticed is about frequency offset calculation. I tried pushing the index discovered by |
Apologies for the delay! I suspect that my assumption that the FFT filter in GNU Radio would just magically provide normalized output was false. This would explain why you have to drop the correlation score randomly, and why MATLAB and GNU Radio do not agree. I started work on a normalized cross correlator for GNU Radio, but it's crazy slow. On the order of ~ 2 MSPS max throughput with a filter size of 1024 samples which is the best case for the GNU Radio graph. It will take a while to figure out how to get the throughput up, and to test that everything is working properly. There are a few possible alternatives to the normalized cross correlation:
The frequency offset estimation code might not be perfect. I have an adjustment to it in my local repo that uses all of the cyclic prefixes to compute a (hopefully) better coarse estimate. Moving the starting index to the left (in time) causes the starting index of each OFDM symbol to end up in the cyclic prefix. Starting inside the cyclic prefix is fine as it just causes a cyclic shift in the frequency domain. This is actually something that I have done in the past when I wasn't sure about the true starting sample index. When in doubt, move the starting sample into the cyclic prefix. Now, what's surprising here is that you say you added to the start index which would push the first sample into the data samples. This would cause the 1024 samples that go to the FFT to include cyclic prefix samples of the next OFDM symbol. That's bad and will break things. That's a bit confusing to me. The channel estimation logic is definitely load bearing code. The line you reference for The biggest issue with starting sample estimation right now is that if you have a frequency offset then the starting offset of the ZC sequence will be wrong. It's some strange feature of the ZC sequence that the cross or auto correlation of the sequence in the presence of a frequency offset will correlate well, but the peak of the correlation will be before or after the actual starting index of the sequence. Since I use the peak to determine the starting index, a large enough frequency offset will break everything. This frequency offset comes from a combination of the transmitter and receiver clock inaccuracies. So even if you have a GPS disciplined oscillator on your SDR, the drone might have a worst case offset and throw everything off. Ultimately I think you're right, that the only real option is to back the starting index off into the cyclic prefix. I've tried that in the past and had issues, but it could have just been that I was doing it wrong =\ [1] https://github.com/proto17/dji_droneid/blob/main/matlab/updated_scripts/process_file.m#L166 |
@proto17 , hi there! I just revisited this repo and had a thought. In the grc file, you feed the pseudo-correlation to the extractor block via filtering the input with the zc sequence ( Though for it to be a (pseudo)correlation should not the zc sequence be reversed? For instance, check gnuradio's correlation implementation here. |
You're absolutely right about needing to reverse the filter taps! It just so happens that the ZC sequence is mirrored about the center in time and frequency :) |
Would it be safe to assume that a correlation threshold of 0.1 and 0.01 will be good to enter by default into the gr-drone_update flow graph? I wanted to put a default setting in there to ensure the flow graph runs by default. |
The GNU Radio update branch ( I would worry that as you lower the correlation threshold, the chance of having a timing offset increases. This shouldn't be an issue for the MATLAB/Octave code, but might for the GNU Radio. I don't think the GNU Radio logic makes a second, more targeted pass at the ZC sequence start index. If it doesn't, then a very low threshold would cause the GNU Radio logic to see the start time offset (STO) as being several samples early, which would break demod later on. The other issue with a low threshold is false triggering which will both consume loads of CPU, and cause missed DroneID bursts since the software is busy consuming samples for a phantom burst. It's also worth noting that a large frequency offset will cause you to get low correlation scores. I don't have access to the flow graph right now, but I seem to recall that I do have a low pass filter in the graph so that you can only see ~ 10 MHz in the center of your baseband. If you have a DroneID burst that only partially comes into your passband, then you might get a tiny correlation spike and think that something was wrong. I recommend looking at the In the end, lowering the threshold has the most effect on CPU. The lower you make it, the more false positives, and the more time your CPU spends chugging on samples. Because of this, you should be on the lookout for Hope that helps! |
Think I'm on the right track then, plan was to update the gnuradio with the gnuradio update branch, but then switch back to main after it's installed so that the matlab code was newer. That way if someone wanted to save off the IQ file/bursts they could then use matlab/octave scripts? But.. if they wanted to try and get decoding running in the bottom left window of gnuradio - that ability would also be there if they open the new flowgraph. I just wasn't completely sure what to stick as a default value in the correlation setting in the flow graph. I need to grab another drone and try this out again. I have the b205 and for a sort period, the x310. |
Hi there,
When I am using the gr-droneid branch with a mavic air 2, gnuradio console displays offsets/phase angle etc. and i see some extracted constellation results in the QT constellation diagram which I assume corresponds to gnuradio extractor/demodulator blocks working properly. But when I check the /tmp/bursts folder I only see a zc and channel file.
I have watched the DragonOS youtube videos and in those videos it seems that thee bursts are getting recorded as well (file_1, file_2, etc.). As I mentioned I only got files named
zc
andchannel
getting created in the folder. Does this mean something is not working correctly, or is this normal behavior?My other question is regarding computation time for the
process_file.m
script. I have a fairly decent machine (16 cores, 16GB RAM) and running the octave script on 2GB of fc32 recording takes a very long time. Is this normal, or have I configured something incorrectly?Thanks for all your work, very interesting repo!
The text was updated successfully, but these errors were encountered: