Naming and function of hardware gyro_lpf options

[ctzsnooze]

Currently the list of choices for gyro_lpf include "OFF" and "EXPERIMENTAL". In the code, the list of choices is intended to let the user set the cut point of the hardware LPF in the gyro; it then sends values of DLP_CFG and FCHOICE_B to the gyro to achieve the requested rates.

The situation is complicated by some gyros not doing exactly the same thing as others, and also that the code is rather old and inconsistent. See @etracer65's notes about the MPU9250 : #5306

There are three main things I'm going to discuss here.

First, 'OFF'.

For all Invensense gyros, 'OFF' (does not actually disable the hardware gyro filter, but sets it to 250Hz (256Hz and 260Hz on some versions maybe, but around 250Hz), at 8kHz.

My suggestion is to re-name 'off' to '250'.

Second, Option 7, "EXPERIMENTAL".

This sets DLPF_CFG to 7. When received by MPU6500, MPU9250 and I think all ICM206xx gyros, whether in 32k or 8k mode, the gyro will set its sampling rate to 8khz and the hardware filter to between 3.1 to 3.3 kHz, the exact set-point depending on the gyro.

The same value, if sent to an MPU60x0 gyro, sets the gyro to 8kz sampling rate but as to frequency response, the spec sheet says 'RESERVED'. So the outcome is unknown at this point (aka truly experimental).

This would be very useful for people who seek to avoid the inherent 1ms delay caused by the hardware 250Hz filter, and replace it with a series software filter, thereby achieving essentially the frequency response speed of a gyro in 32kHz mode without the need to clock the gyro at 32k

At present, how the code deals with 'EXPERIMENTAL' is not consistent, in that not all gyros receive the DLPF=7 command.

My suggestion is that when set to 8k mode, the code should send DLPF_CFG = 7 to ALL invensense gyros, including MPU60x0. This way we can bypass the 3k hardware filter on supported gyros, and test the behaviour of the MPU60x0 to see what it does.

Third, behaviour of 32k capable gyros

MPU6500, MPU9250 and I think all ICM206xx gyro's are 32k capable. MPU60x0 are not.

For 32k capable gyros, setting the FCHOICE_B values is how the gyro is forced to enter 32k sampling mode; it will then ignore whatever is set in DLPF_CFG.

However 32k mode also has an internal hardware filter. This cannot be bypassed. Depending on the FCHOICE_B values, it can be either around 3kHz, or around 8kHz, on each of those gyros.

The code is currently not clear about what it does in relation to those two different filter points.

It's obvious that from a flight perspective, we do not need incoming frequencies above 3kHz.

So we have two options. We could only ever initiate 32k gyro mode with the 3k hardware filter enabled. Or, there could be a selection via a user choice in gyro_lpf.

My suggestion is to make it so that when 32k gyro mode is active, if gyro_lpf is set to 'experimental, then the gyro enters 8k hardware filtering mode, if set to anything else, it returns 3k hardware filtering.

Thus a user who uses the normal 'conservative' value of 250 (off) in the gyro_lpf field, would, in 8k mode, get 250Hz hardware filtering; if they switched to 32k mode, they would get 3k hardware filtering. And if they set gyro_lpf to experimental, they would, in 8k gyro mode, get 3k hardware filtering (or an uncertain hardware filter state on MPU60x0 gyros), and 8k hardware filtering in 32k mode.

The main intent of this suggestion, is to make the functionality consistent across all gyros and to sensibly allow access to the low hardware filtering options available in many gyros while still in 8k modes.

[etracer65]

Okay, a few comments... There are two separate (but related) things being discussed here. Note that everything here refers to Invensense gyros (not sure if we're actively supporting any others).

Gyros running in 8K mode:

The first issue centers around revamping how gyro_lpf is used to select the hardware lpf filtering in the gyro. The current enumeration for gyro_lpf is misleading:

static const char * const lookupTableGyroLpf[] = {
    "OFF",
    "188HZ",
    "98HZ",
    "42HZ",
    "20HZ",
    "10HZ",
    "5HZ",
    "EXPERIMENTAL"
};

First it's based on the MPU60x0 gyros and the actual cutoff frequencies vary for each gyro.

Secondly some of the settings are misleading and not representative of the actual function. As @ctzsnooze mentioned, "OFF" is not really off, but instead around 250Hz depending on the gyro. Option "EXPERIMENTAL" selects a high-frequency cutoff with significantly less filtering. On MPU60x0 gyros the "EXPERIMENTAL" (7) value is actually listed as "reserved" and Invensense doesn't publish the lpf cutoffs. It's unclear whether this setting does anything on that gyro (or maybe something bad). However I have a hunch it does change the lpf to reduce filtering. For all other gyros the "EXPERIMENTAL" setting selects a documented high-frequency cutoff. For example, here's the MPU6500. The gyro_lpf value gets mapped to the DLPF_CFG bits in the register.

Thirdly, any selected gyro_lpf value from 1-6 (188Hz through 5Hz) puta the gyro in 1KHz sample mode. I'm not sure that any of these cutoffs are appropriate for use at this point. I can't see how using a 5Hz hardware lpf would be useful (or even flyable!).

So my suggestion is to revamp the gyro_lpf enumeration to make it more representative and possibly drop 1KHz gyro sample rate options. Maybe just shift to two options like "NORMAL" and "ADVANCED" that represent the current "OFF" (250Hz) and "EXPERIMENTAL" (~3000Hz).

Gyros running in 32K mode:

In the MPU6500 table above the first two rows represent the options available when the gyro is running in 32K sampling. Currently the ~3000Hz (row 2) is selected for all gyros. There's an opportunity to make use of the higher ~8K cutoff but we don't have a good way to select it currently. A hack was done a while back that uses the gyro_lpf = EXPERIMENTAL value to select the high bandwidth option - but this was only done for the ICM20689 gyro. It also confusingly uses the gyro_lpf which has nothing to do with 32K mode as a re-purposed control.

So the first question is whether being able to select either 32K lpf is useful.

Secondly, what's the best way to select it. Personally I'm not a fan of re-purposing gyro_lpf for this function. I would rather see a dedicated parameter that selects the desired hardware lpf. Maybe something like 32khz_gyro_lpf. Again values like "NORMAL" and "ADVANCED" would be appropriate.

[etracer65]

I just tested a MPU6000 and captured the raw gyro debug_mode = GYRO_RAW for both gyro_lpf = OFF (256Hz) and gyro_lpf = EXPERIMENTAL (???) and clearly there's a difference. It's unknown whether the DLPF is turned completely off (doubtful) or set to some high cutoff. Log viewing options (smoothing, expo and scale) were the same for both. FC was immobilized on the desk.

[DieHertz]

Looks noisy beyond any usefulness. We need a target with two such gyros on one PCB to try and estimate the relative delay between EXPERIMENTAL and other settings.
P.S. 250Hz was probably considered high enough to have no effect on flight characteristics and be called OFF by someone in the past.

[ctzsnooze]

Thanks @etracer.

PS - what did the spectrums look like? That's where we should see what the frequency response difference is.

@DieHertz. Funny how much thinking changes over time. My first cleanflight quads ran at 1kHz PID loop and 32Hz hardware filter only :-)

The guys flying 32k gyro modes right now are actively using the 'uselessly high' filtering of which you speak, at 8k actually, to reduce delay. Of course, they need a software filter to replace the hardware filter, there is so much noise that it won't work with just a single PT1. Hence RS2k's 'Kalman', basically a first attempt at a series low-pass filter to attenuate the extra noise in 32k gyro modes. With a software replacement for the hardware filter, a lot of guys are getting great results.

But 32k gyro modes are best if the PID loop runs 32k also, to avoid aliasing. And not many people can run 32k PID loop on F4's. So I know a lot of people who would be very excited to have the ability to truly eliminate the 250Hz hardware gyro filter, and replace it with a good software filter with less delay, while running the PID loop at 8k. To make that possible for an F4 and maybe even an F3 would be quite awesome.

The advantage of cleaning up these settings is that users can potentially get the least delay possible in the hardware of their gyro, even when there is only one gyro on the board, and only one CPU on the board. True, to do that will require some robust second series software gyro low-pass filter options, but I anticipate we will see that in RC's of 3.4 RC, hopefully very soon.

[BrenLaw]

First time posting here, but I've been closely following filter development for a while now. Some time hopefully in the next 12 hours, I plan to do a few tests with my MPU6000 equipped flight controller.
Here are the different tests I want to do:

1. Baseline test with classic soft Gyro LPF set to PT1 at 100hz, and gyro_lpf = OFF
   Nothing new here. Just need a spectrum analysis of a typical setup to compare against.
2. Soft lpf PT1 at 100hz, gyro_lpf = EXPERIMENTAL
   This and test 1 will be essentially the same data @etracer65 collected and posted above.
3. Replace PT1 gyro lpf with new BQRCF2 at 100hz
   This should yield a very similar result to test 1. As we know, under about 500hz, PT1 and BQRC2 filters are pretty much identical as far as noise attenuation goes.
4. BQRCF2 at 100hz, gyro_lpf = EXPERIMENTAL
   This is where I think things may get interesting. Although BQRCF2 adds more delay than PT1, it also is better at filtering higher frequency noise. If it's possible that changing gyro_lpf from OFF to EXPERIMENTAL does in fact raise the hard lpf, this might make BQRCF2 (or future gyro lowpass options) very useful for MPU6000 equipped flight controllers.

What I'm really anxious to see is the comparison between steps 2 and 4. I'm very interested in finding out if the extra higher frequency filtering of BQRCF2 can make the gyro_lpf = EXPERIMENTAL data from the gyro actually usable. Of course this requires that setting it to EXPERIMENTAL does actually raise the hard lowpass, which we're not 100% positive it does. But etracer's post clearly shows that it does something. If it does raise the hard lowpass, and if BQRCF2 does effectively filter out the extra noise, this will be good news for people running MPU6000 gyros. Or like @ctzsnooze said, people with F4s who don't want to/can't overclock, and even people with F3s.

If anyone can beat me to these tests (or already has done them), more power to them! I just really want to compare spectrum analyses of the 4 tests I listed; possibly with different debug options. Like gyro_raw to compare gyro_lpf = OFF vs. EXPERIMENTAL. And then debug mode FFT to see the effects of BQRCF2 in the different tests.

[etracer65]

Here's a blackbox log containing about 20 seconds of raw gyro data from a MPU6000 with gyro_lpf = EXPERIMENTAL captured at 8KHz. The board was sitting still on a table.

Not sure if it shows anything meaningful other than lots of noise.

mpu6000 8K gyro_lpf experimental.BBL.zip

[flint723]

I can confirm that gyro_lpf = experimental definitely does either disable or raise the hardware LPF in the MPU6000. This is a 30 second hover of both logged at 8k. I do have BQRC at 450hz in this log.
mpu6000 gyro_lpf = off vs experimental.zip

Edit: DO NOT attempt to fly in experimental mode unless the BQRC (stage2) is turned on, even then it may be unstable.

[BrenLaw]

I'm still collecting data with various debug options (gyro raw, notch, FFT to see the direct impact of BQRCF2 before the dynamic notch).

I don't have all the blackbox logs I want yet, but anecdotally I do have some findings that are pretty interesting. I've done some test idling with a typical setup--classic gyro PT1 at 100hz, dynamic notch enabled, gyro_lpf = Off. Then I did the same test but gyro_lpf Experimental. Motors sounded noticeably less smooth. But then I did test with Gyro PT1 at 100 hz, dynamic notch enabled, stage 2 lowpass 150hz, gyro_lpf Experimental and motors were actually running smoother than the typical PT1 + dynamic notch + gyro_lpf = OFF setup.

What I don't have the ability to test for is the differences in delay. But if using gyro_lpf = EXPERIMENTAL decreases phase delay on an MPU6000, then the dual stage lowpass setup is looking really promising. Because like I mentioned, motors sounded smoother than with a classic filter setup.

[ctzsnooze]

@flint723 - one question - did you have the BQRC_FIR2 enabled in either of those traces, or were they both just PT1's on gyro and D (which are both at 100)

Below I show spectrums of PID_SUM, exactly 30s each, from your hovers.

Interestingly, although the first noise spike from the quad at about 130Hz is about the same magnitude in both, there is a heap more noise in the 0-150 range, i.e. in the range we care about, in the EXPERIMENTAL mode. That doesn't really make sense to me, unless somehow there is a lot of random non-quad-derived noise contaminating that 0-150 range. Just shifting a lowpass up from 250 to 3000 should not cause that. Aliasing artefacts maybe?

At the same time, although this noise exists, the motor traces are only a few percent wider. I'd be surprised if the motors were noticeably hotter. If the BQRC_FIR2 was not enabled on the EXPERIMENTAL trace, it would be great to see how it looked when it was enabled.

PS 90% all the noise above 100Hz arose from D, as to be expected.

[etracer65]

I really don't think we should turn this issue into another huge discussion about filter performance. Instead we should keep it focused on how (or if) gyro_lpf should be refactored and how (or if) we should enable access to the higher cutoff 32K lpf setting.

[flint723]

@ctzsnooze yes BQRC is on at 450hz in that log.

When I tried to fly in experimental with BQRC off, the quad just flew away immediately upon arming (from extreme dterm noise). I could post that log but it's probably pointless.

[ctzsnooze]

Agreed @etracer65.
@BrenLaw testing experimental gyro option itself might better be discussed on RCGroups, either the FKF or the betaflight threads. I suggest trying 250Hz PT1 also and post results there.

[DieHertz]

FKF thread should be left to rs2k cultists, move along to BF thread then.

[ctzsnooze]

@etracer65 - I agree also on simplifying the names.

When the user has selected 8k mode, maybe all we really need is 'NORMAL' and 'HIGH', where normal = 250 and high = 'experimental'/3k. That would be very user-friendly and simple. I suppose we'd have to know if any users actually do fly with the hardware filter on and set below 250, I guess, but I doubt anyone does.

For gyros where the user has selected 32k mode, there are only 3k and 8k options, these could again be 'NORMAL' and 'HIGH'.

That would allow just one CLI line for gyro_harware_filter : NORMAL / HIGH defaulting to NORMAL

I dare say if some people want lower hardware filtering than 250 it could be VERY_LOW / LOW / NORMAL / HIGH :-) Code would try, for different gyros, to do 90 / 180 / 250 / 3k

[BrenLaw]

@ctzsnooze Totally agree. I got so excited with my initial results that I forgot this thread was about the function and naming of gyro_lpf option for the various gyros. But before I migrate to another thread, I just want to say I've now done a full flight (with MPU6000 gyro) with Gyro PT1 at 100hz, Stage 2 lowpass at 150hz, dynamic filter enabled, gyro_lpf EXPERIMENTAL. Motors came down ice cold, flight performance was great, and obviously didn't have any issues with a runaway quad on takeoff. So there's definitely something here and it has great implications for the future of F4 and F3 boards.

I'm going to continue collecting data, but will post it to the appropriate place. Thank you for dealing with my enthusiasm.

[denovich]

It would be great if the gyro options unambiguously named.

LOW, NORMAL, HIGH.... does HIGH refer to the frequency cut-off, or the amount of filtering? I have no way of intuiting that from the naming. Especially when other popular firmwares use similarly named settings that may or may not follow the same convention. The sort of confusion this causes is endless.

And if this is visible in the configurator, please, please, have a tooltip or something that gives the details behind the setting, not just "HIGH is high filtering..." but "HIGH is 32khz gyro 8khz bandwidth" or something.

[etracer65]

Yes, that's the reason I suggested "NORMAL" and "ADVANCED" (even though those may not be the best either). I saw "HIGH" as ambiguous. Does it mean high frequency cutoff or "high filtering" (low frequency or steeper cutoff).

The other undetermined is which (if any) of the 1KHz sample rate options will be retained and how do we name them?

[ctzsnooze]

ha ha - 'HIGH' to me meant a high filter setting, allowing high frequency noise in, not a high level of filtering. And since the name has been gyro_LPF_Hz, I'd have thought 'high' meant a high number in Hz. I appreciate the ambiguity though.

The name has to match the label, e.g,:

noise reduction: strong, normal, minimal
filter strength : strong, normal, weak
filter setpoint Hz : low, normal, high

It's not easy finding an intuitive solution. I'm not at all fussy about the names. The above 'three name' option was to allow one 1k filter option.

Personally I'd be happy to get rid of the 1k options altogether, then this menu can just have two entries, and normal vs advanced are fine by me.

[AndersHoglund]

If HIGH means 32kHz, why hide that? Say "32kHz" instead.
As little encoding and transformation as possible. Anything else would only lead to misunderstandings and support cases.

[etracer65]

@AndersHoglund This discussion has nothing to do with 32KHz mode or 32KHz in general. We're discussing the gyro hardware lowpass filter when running in 8KHz sampling and how gyro_lpf sets the DLPF_CFG bits in the configuration register. Currently the gyro_lpf value maps directly to the DLPF_CFG bits in the register. However the enumeration for gyro_lpf is inaccurate so that needs adressing (the "NORMAL", "ADVANCED", etc. discussion). Also discussing whether some of the lowpass cutoffs are even valid anymore. For example is it even desirable/needed to continue to support 1KHz sampling from the gyro (gyro_lpf/DLPF_CFG 1-6).

[BrenLaw]

@denovich This sort of option should probably stay in CLI for a while. The last thing we need is for inexperienced people to be able to easily change gyro hard lowpass settings in the Configurator. If they don't have dual stage lowpass filtering enabled (which you currently already have do through CLI), or at least perfectly tuned notch filters, they're very likely to smoke a motor due to the increased noise the gyro is passing through to the PID loop.

I like @etracer65 's suggestion of "NORMAL" and "ADVANCED". It's simple and default can be NORMAL. But advanced users who are familiar with CLI and blackbox logging can use the ADVANCED option to sort of "unlock" their gyro and get the increased sensitivity.

I really don't know anyone on Earth who uses the gyro DLPF 1-6 options. I don't want to say they're not out there, but I'm probably correct in assuming that 99% of Betaflight users are running gyro_lpf = OFF. And we now have a couple of us testing EXPERIMENTAL. Perhaps the 1-6 options can just be left as they are currently (188HZ, 98HZ, 42HZ, 20HZ, 10HZ, 5HZ) for the people who may be using those options. But just change OFF and EXPERIMENTAL to NORMAL and ADVANCED, respectively. To be honest though, I personally wouldn't even mind dropping support for 1-6. I can't imagine the benefit to using any of those settings over a good software lowpass. The delay those settings add is hilariously long.