How does your mcHF software DSP work
Clone this wiki locally
[WORK IN PROGRESS, comments welcome]
DSP means "digital signal processing" - this is the basic foundation of the audio processing in the mcHF and all other SDRs (software defined radios) on the market.
This document is meant as a brief description of the functioning of the mcHF software (and part of the hardware) in order to help users to understand the basic principles of their transceiver as well as to give the programmer an overview over the main functioning software DSP blocks in the mcHF.
[parts of this document have been taken from my decription of the Teensy SDR and adapted for the mcHF and inspired by the excellent source code description by Clint KA7OEI, DD4WH 2016_08_11]
Basics of a software radio
As with all IQ SDR radios, the real RF (radio frequency) input signals are transformed by hardware components into complex I & Q signals (see Smith 1998 for a mathematical description of these signals). These analog I & Q signals are then digitized by an ADC (analog-digital-converter) and further processed (filtered, demodulated etc.) in software. At the end, the digital signals are again transformed into analog audio by the DAC (digital-analog-converter) and amplified in the headphone amp for listening. That´s basically all for Rx and in Tx the whole thing works the other way round! ;-)
In general, there are four methods for a receiver to demodulate signals (Youngblood 2003): 1. The filter method, where the signal filtering occurs in crystal/mechanical filters (commonly used in superhet receivers), 2.The phasing method (normally hardware-based in old-style tube phasing receivers or in the R2Pro, see EMRFD), 3. The “ Weaver ” or “ Third ” method (nearly forgotten method and rarely implemented, but see Weaver 1956, Kainka 1983 & the excellent website by Summers 2015), 4. The “ Fourth method ” (van Graas 1990), which uses a quadrature sampling detector (QSD) and which has become popular due to publications and a patent by Tayloe (2001).
mcHF IQ processing
The mcHF uses the standard approach for an IQ SDR radio (also implemented by many of the PC-based or stand-alone commercial SDR radios like the KX2, KX3, Flex-Radio series, Softrock etc.) – i.e. a hardware QSD (fourth method) to produce the I & Q signals and the phasing method (second method) implemented in software to suppress the undesired sideband in various ways.
The hardware used to produce the I & Q signals in the mcHF is a double balanced QSD (based on U15 = 3253) followed by an operation amp OPA 2350 (U16) to amplify (and lowpass filter) the IQ signals together with the sampling caps C68 and C69 which also determine the bandwidth of this QSD. Youngblood (2002) has provided us with an excellent description and explanation why the Quadrature Sampling architecture in RX and TX acts as an additional filter. When trying to calculate this from the specs of the mcHF (double balanced QSD, sampling caps = 22nF, switching resistance of the SN74CBT3253C = 4-5 Ohms), I get: BW = 1/ (PI * 4 * (50+4) * 22nF) = 60.285kHz. We would only need 48kHz here, because that is the displayed bandwidth in spectrum and waterfall display. So, if we used 33nF or 47nF for C68 and C69, it would still be sufficient (before you replace the caps, do the math yourself ! ;-)), but we would have better suppression of non-interesting frequencies and prevent overload of the ADC a bit better.
Local oscillator (LO) is the famous Si570. Famous not only for the low phase noise, but also for its quite high power consumption and the restriction to frequencies above 4MHz. [Best would be to take something like the Si514 as the LO, which has much lower power demand, but comparable phase noise, and also produces frequencies from 100kHz upwards - (vladn has very good experience with this LO: http://theradioboard.com/rb/viewtopic.php?f=4&t=6641) and my Teensy SDR Receiver also now works with the Si514]. The LO runs with four times the Rx or Tx frequency and feeds a Johnson counter (U11). Thus, the phase noise requirements are not that high, because the use of the Johnson counter U11 (7474) with fLO = 4 x fRx reduces the phase noise by a factor of four.
The I & Q signals from the QSD are then input to the Audio codec. They are converted by the ADC with a sample rate of 48kHz. I am not sure whether there is any built-in analog anti-aliasing filter in front of the ADC ?
From this point on, all the processing is in software and is performed by the STM32F4, a 32-bit microprocessor.
mcHF software IQ processing with the STM32F4
The software path starts by stuffing the I & Q audio samples into two buffer variables. The I and Q signals are just sampled amplitude measurements of the input signal measured in 1/48000 sec = 20.1 µsec intervalls. They are however 90 degrees out of phase (I = cos, Q = sin) and that is the key to the flexibility to use them for all kind of demodulation and also other goodies in your mcHF.
What the I & Q signals are used for in your mcHF:
spectrum display and waterfall: these display the amplitude (= magnitude of the signal) at different frequencies. Because the I & Q samples represent magnitudes at different points in time, we need a time to frequency domain conversion for this purpose. The standard method for this is a Fast Fourier Transform = FFT. The spectrum and waterfall display in the mcHF use 256-point complex FFTs, that means we have 256 "bins" = individual calculations of signal magnitude for a division of the 48kHz spectrum. So the resolution of the spectrum / waterfall display is 48000/256 = 187.5Hz. AND: this resolution remains even in magnify mode, so you have to find other tricks to get more resolution for a higher magnification of the spectrum display (eg. a Zoom FFT as sketched by Lyons 2011 --> go ahead and implement it ;-))
very accurate frequency determination from an AM carrier (SNAP): the name "SNAP" was borrowed from vladns implementation of an SDR. [see below: section sophisticated reception]
demodulation: this is of course the core of the mcHF: demodulation with I & Q is very easy ;-):
I + Q = USB
I - Q = LSB
USB + LSB = DSB [called SAM in the mcHF]
SQRT (I * I + Q * Q) = AM
FM = atan(Q * I_old - I * Q_old, I * I_old + Q * Q_old) [taken from the excellent description by Clint, KA7OEI]
That´s it, all demodulation can be done in software by these simple calculations. Of course, in reality, it becomes a little more complex.
I & Q correction
Because of tolerances in the hardware components, the amplitudes and phases of the I and Q signals are not perfectly matched accurately, i.e. they are not exactly 90 degrees out of phase and their amplitudes differ a bit. But this is corrected in software by scaling the I and Q signals: The amplitude of I and Q can be calibrated by simply adjusting the input gains of the signals. Phase calibration is just as simple: A small amount of the I signal can be mixed into the Q channel and vice versa. This is equivalent to a slight phase shift of the signals (because the amplitude of a wave of different phase added to another wave determines the phase of the resulting wave). Ideally, the amplitude- and phase-corrected I and Q signals then have exactly the same amplitude and are exactly 90 degrees out of phase.
Every SDR should use an IF (intermediate frequency) in order to get rid of all the noise present near DC (below 1kHz) caused by power line hum (50/60Hz) etc. The mcHF gives the user maximum flexibility to choose the IF frequency (-6kHz, +6kHz, -12kHz, +12kHz). Because of the usage of the IF (frequency translation) we need to translate our I & Q signals to baseband before the real audio processing begins. This is accomplished using complex frequency translation. A complex software NCO (numerically controlled oscillator) is built and each input sample is multiplied by the real and imaginary parts of the NCO generator (for an explanation and an evaluation of its effectiveness see Lyons 2014, Wheatley 2011). The NCO uses two IF generators with 90 degrees phase shift (cosIF&sinIF) and uses four complex multipliers to multiply the I and Q signals with the NCOs. At the end, the resulting signals are added and substracted depending on whether you use a negative or a positive IF.
After passing the Complex Quadrature NCOs and the complex multiply, the I and Q signals are still I and Q, but now they are in baseband and can be processed by the filters! So never switch off frequency translation in your mcHF, unless you have a very good reason to do so.
What we have achieved up to now, is the downconversion of the RF signal into an audio signal, the sampling of the signal by the ADC, the conversion of the single signal into the phase-shifted I&Q, the correction of amplitude and phase of I & Q and the conversion to baseband. So we now have near to perfect I&Q audio signals in baseband with exactly equal amplitude and exactly 90 degree phase difference.
Audio Rx path
This is the path, that the audio signal now has to go through:
- Hilbert transform with Hilbert FIR filters OR lowpass Filtering with FIR filters (AM & FM)
- Demodulation according to the formulae above
- Decimation filter & Decimation
- Automatic notch filter (if activated)
- Automatic noise reduction (if activated)
- Audio Filter (depending on user choice, see below)
- AGC processing
- Manual notch filter (if activated)
- Manual peak filter (if activated)
- Bass equalizer
- Interpolation & Interpolation filter
- Anti-Alias Filter (only for certain filter combinations)
- Treble Equalizer
All these calculations for the different feature are done in real-time! Hard work for the STM32F4 . . .
Now let´s have a look at these 12 steps in a little more detail, if you like . . .
- & 2. Hilbert FIR filters are very phase linear filters that are used here to perform the "phasing" that serves to suppress the undesired sideband. In the mcHF we use a Hilbert FIR (89 taps) with 0° phase shift in the I path and a -90° phase shift in the Q path. Why do we need a 0° phase shift, isn´t that pretty useless? Well, the phase shift of -90° in the Q path needs processing time, so we have to spend that time in the I path too, alternatively to the Hilbert 0° we could use a delay. So how does this Hilbert filtering work? The desired sideband in I is not shifted. The desired sideband in Q is at 90° and is shifted by the Hilbert -90° to 0°, so it adds to the desired sideband from the I path (+3dB !). The undesired sideband has a 180° shift compared to the desired sideband and is at 180° in the I path. The undesired sideband from the Q path is at 90° and is shifted to 0° by the Hilbert, so the undesired sideband components in I & Q eliminate each other! Well, probably others can explain this a lot better: https://www.dsprelated.com/showarticle/176.php
One other desired feature of the Hilbert filters is the bandpass filtering. Depending on the desired bandwidth that the user has chosen, a suitable Hilbert Filter is used. The Hilbert Filters used in the mcHF have bandwidths of
3.6kHz, 4.5kHz, 5kHz, 6kHz, 7.5kHz or 10kHz
The coefficients have been precalculated with Iowa Hills Hilbert Filter Designer and are stored in memory in the mcHF.
Alternatively, for AM or FM demodulation, we do not need the Hilbert Transform, we use lowpass FIR filters instead. Again, depending on the desired bandwidth that the user has chosen, a suitable FIR lowpass Filter is used. The FIR lowpass Filters used in the mcHF have bandwidths of
2.3kHz, 3.6kHz, 4.5kHz, 5kHz, 6kHz, 7.5kHz, 10kHz
These coefficients have been precalculated with Iowa Hills FIR Filter Designer and are also stored in memory in the mcHF.
´3. Decimation means reducing the sampling rate of a signal. This has the advantage, that less samples have to be processed per time unit and filters can have a steeper stopband transition with the same number of coefficients. To prevent aliasing, we need good lowpass filtering BEFORE the decimation. This lowpass filter function is already fulfilled by the Hilbert/FIR filters, so we do not need to care much about that. The CMSIS decimation functions that are used in the mcHF have built-in lowpass FIR filtering with a minimum no. of 4 taps, so we use these 4 taps in order to be processor efficient. For filters with low bandwidth, we use decimation by 4 (12kHz sample rate), for higher bandwidths we use decimation by two (24kHz sample rate).
´4. & 5. The automatic notch filter and automatic noise reduction have been implemented by Clint KA7OEI using LMS algorithms. For a detailed description of these algorithms see part 1 and part 3 of the excellent -but also math-oriented- article series by Smith (1998).
´6. This is the main audio filter, the brick wall that shapes your audio signal and prevents other signals of disturbing your audio experience ;-). Clint has implemented very efficient IIR filters here. The main advantage of these kind of filters over FIR filters is their high efficiency compared to the processor power that they use. Their main disadvantage is the non-linear phase response (which, at this place of the audio chain, does not matter any more) and the complexity of calculating the coefficients and ensuring that these filters are stable (unstable IIR filters oscillate and have annoying output even when they have no input . . . ). The mcHF uses ARMA structure ladder filters at this place, which are very stable, when the coefficients are calculated with the algorithms implemented in fdatools in MATLAB. We mainly use 10th order IIR filters. We expanded the range of filters that the user can choose at this place of the audio chain to the following (all filters have elliptic response and 60dB stopband):
Lowpass & bandpass version, for some of them several bandpass versions: 300Hz, 500Hz, 1.4kHz, 1.6kHz, 1.8kHz, 2.1kHz, 2.3kHz, 2.5kHz, 2.7kHz, 2.9kHz, 3.2kHz, 3.4kHz, 3.6kHz, 3.8kHz
Here an example of a typical bandpass IIR filter of the mcHF:
Only lowpass version: 4.0kHz, 4.2kHz, 4.4kHz, 4.6kHz, 4.8kHz, 5.0kHz, 5.5kHz, 6.0kHz, 6.5kHz, 7.0kHz, 7.5kHz, 8.0kHz, 8.5kHz, 9.0kHz, 9.5kHz, 10.0kHz
Here an example of a typical lowpass IIR filter of the mcHF:
´7. AGC processor : The mcHF has a very good working AGC, that can be adjusted by the user for virtually every specific purpose.
´8. & 9. Manual notch and peak filter and bass equalizer : These three filters are implemented using cascaded IIR biquad filters. In comparison with the IIR ARMA structure filter, that uses precalculated coefficients, these filters use coefficients that are calculated on-the-fly in the mcHF! Notch and peak frequencies and bass attenuation/enhancement can therefore be adjusted by the user in very fine steps. These filters are very efficient, using very low processor power, nonetheless enabling a notch filter with a Q of 10 leading to a notch bandwidth of a few tens of Hz only! [it could even be much narrower, but you would no longer be able to use it, because you would not find the notch ;-)] The formulae for the calculation of the coefficients are based on the famous DSP EQ cookbook by Robert Bristow-Johnson: http://www.musicdsp.org/files/Audio-EQ-Cookbook.txt
´10. & 11. In order to get to the 48kHz sample rate again, we use the CMSIS interpolation routine. Again, this has a built in interpolation FIR filter. We only use the minimal version with 4 taps. The interpolation filtering is then done with an IIR filter, which is more effective in terms of processor power. For some bandwidths it is better to have a 16 tap FIR interpolation filter, in these cases we do not use an additional IIR interpolation filter. With this strategy we have completely eliminated alias products in the audio range.
`12. This treble equalizer has the same structure as the bass equalizer filter (IIR biquad). The nature of treble audio signals is a higher frequency, so we work at the 48kHz sample rate in order to prevent aliasing.
TX - transmit path
All audio calculations in the TX path are at 48kHz sample rate.
´1. Audio source (microphone, USB, LINE)
´2. IIR Filter: All TX IIR filters are Bandpass filters with ARMA structure IIR filters of 10th order elliptic with 60dB stopband response with precalculated coefficients [fdatools in MATLAB]. There are three TX audio filters the user can choose from:
´3. Bass & Treble Equalizer: Bass & Treble response for TX can also be adjusted by the user (-20dB to +5dB). This is implemented with two cascaded biquad filters similar to the RX case. This filter is specific for the SSB TX case, in all other modes this filter is skipped.
´4. FIR Hilbert Transform with two mighty 201-tap-Filters, I with 0° phase shift, Q with -90° phase shift. The problem with Hilbert transforms is that they perform badly at low audio frequencies (Lyons 2011). So do not expect a superb sideband suppression when you look at a 300Hz tone in SSB mode. The sideband suppression for that tone is about -38dBc. For higher frequencies, eg. a 1000Hz tone, the sideband suppression can be as good as -70dBc or better. I have also tested the use of FIR Hilbert filters at this place with +45° phase shift in the I path and -45° phase shift in the Q path. These two filters had exactly the same sideband suppression and filter response in the low frequency audio band. So we still use the 0° and -90° version.
´6. Frequency translation
´7. I & Q amplitude and phase correction
TO BE CONTINUED . . .
Thanks to all contributors in forums and all SDR radio fans who helped and/or commented: Clint KA7OEI, vladn, Andreas DF8OE, Danilo DB4PLE, Michael DL2FW, Esteban Benito, Joris van Scheindelen PE1KTH, Pete El Supremo, Rich Heslip VE3MKC, Rob PA0RWE, and kpc.
Useful references (some cited in the text):
Hayward, W., R. Campbell & B. Larkin (2003): Experimental Methods in RF Design. – ARRL. [= EMRFD]
Heslip, R. (2015): The Teensy Software Defined Radio. – The QRP Quarterly July 2015: 37-40.
Kainka, B. (1983): Transceiver-Zwischenfrequenzteil nach der dritten Methode. – cq-DL 1/83: 4-9.
Lyons, R.G. (2011): Understanding Digital Processing. – Pearson, 3rd edition.
PA0RWE, R. (2015): Stand-alone SDR ontvanger met de PSoC. - Benelux QRP Club Nieuwsbrief 156: 19-21.[see also: http://pa0rwe.nl]
Smith, D. (1998): Signals, Samples and Stuff: A DSP Tutorial (Part 1). – QEX Mar/Apr 1998: 3-16.[see also the other three parts: http://www.arrl.org/software-defined-radio]
Summers, H. (2015): Weaver article library. – online: http://www.hanssummers.com/weaver/weaverlib [2015-12-06]
van Graas, H. (1990): The Fourth Method: Generatin gand Detecting SSB Signals. – QEX Sep 1990: 7-11.[cited by Youngblood 2003, if anybody has the original paper, I would be glad to receive a pdf]
Weaver, D. (1956): A Third Method of Generation of Single-Sideband Signals. – Proceedings of the IRE, Dec 1956.
Wheatley, M. (2011): CuteSDR Technical Manual Ver. 1.01. - http://sourceforge.net/projects/cutesdr/
Youngblood, G. (2003): A software-defined radio for the masses (Part 4). – QEX Mar/Apr 2003: 20-31. http://www.arrl.org/software-defined-radio