Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Accelerometer value failures on 1.9.2 stable only #12929

Open
ryanjAA opened this issue Sep 9, 2019 · 18 comments
Open

Accelerometer value failures on 1.9.2 stable only #12929

ryanjAA opened this issue Sep 9, 2019 · 18 comments
Labels
bug Drivers 🔧 Sensors, Actuators, etc stale

Comments

@ryanjAA
Copy link
Contributor

ryanjAA commented Sep 9, 2019

Accel 0 and 1 differ greatly when running (1.9.2) stable from 1.8.0 . Can't arm since offsets are so high.

Have done multiple calibrations for good measure, all end up similar, close on 1.8, far on 1.92

Same exact plane used but with firmware 1.8.0 and works fine, same hardware, same everything.

Quick look at the tests sensors and see attached images.

Nothing has moved, even the time when done were right after each other except the firmware.

Screen Shot 2019-09-09 at 12 45 47 PM

Screen Shot 2019-09-09 at 12 37 58 PM

@ryanjAA
Copy link
Contributor Author

ryanjAA commented Sep 10, 2019

So for good measure I went and formatted the SD card (read some old issues about this), flashed 1.8 then back to stable (1.9.2) tried again and all the same. Not same exact numbers but way off.

After calibrating on sensors on 1.8:

CAL_ACC0_EN 1
CAL_ACC0_ID 1246218
CAL_ACC0_XOFF -0.045222282409667969
CAL_ACC0_XSCALE 1.004043698310852051
CAL_ACC0_YOFF -0.370789140462875366
CAL_ACC0_YSCALE 0.999102413654327393
CAL_ACC0_ZOFF -1.115360736846923828
CAL_ACC0_ZSCALE 0.982260644435882568
CAL_ACC1_EN 1
CAL_ACC1_ID 1114634
CAL_ACC1_XOFF 0.636579990386962891
CAL_ACC1_XSCALE 1.061447978019714355
CAL_ACC1_YOFF 1.085980534553527832
CAL_ACC1_YSCALE 1.016108870506286621
CAL_ACC1_ZOFF 1.802345871925354004
CAL_ACC1_ZSCALE 1.044702291488647461

Then on 1.9.2:
CAL_ACC0_EN 1
CAL_ACC0_ID 1246218
CAL_ACC0_XOFF 0.074542999267578125
CAL_ACC0_XSCALE 1.000208139419555664
CAL_ACC0_YOFF -0.259348034858703613
CAL_ACC0_YSCALE 1.004135727882385254
CAL_ACC0_ZOFF -1.201539516448974609
CAL_ACC0_ZSCALE 0.984851241111755371
CAL_ACC1_EN 1
CAL_ACC1_ID 1114634
CAL_ACC1_XOFF 1.785231351852416992
CAL_ACC1_XSCALE 1.057846903800964355
CAL_ACC1_YOFF 2.889719724655151367
CAL_ACC1_YSCALE 1.021885633468627930
CAL_ACC1_ZOFF 3.765102386474609375
CAL_ACC1_ZSCALE 1.047726869583129883

So it seems to be only effecting Accel1 but Accel0 is the one failing the sensor tests... @dagar Weird as just changing firmwares (back) and calibrating fixes it. Also, didnt load old params in a few tests, flashed stable, hit airframe and redid all params from scratch just to try. Still no go.

@mcsauder
Copy link
Contributor

I hate to say this, but the likely reason the are way off now might be less due to the fact that they are wrong now than it was to the fact that they were wrong in the past... The drivers have had sooooooooooooooooooooooooooo many errors, (not anyone's fault, just the way it is), that correcting them is literally never-ending.

Let's do some work to prove the current trustworthiness between now/then and determine which is right, (given all that we understand at this present moment...). But, let me assure you... this is going to be some work on your part!

-Mark

@ryanjAA
Copy link
Contributor Author

ryanjAA commented Sep 10, 2019

The ever present nature of driver errors and changes seems logical at a high level as has been understandably and expectedly been the case with anything going under constant involvement since the inception of the codebase but how something can differ so greatly and on such magnitude (given what that means from an operability standpoint) strikes me as odd if not hardware related.

A mere hardware issue would be more palatable short of just a PR that “broke” the reporting output and this not be situational to my tests. I haven’t looked at what was changed between the version variants but having accel and gyro fail tests on 192 and not on 180 seems like logically (at least from my view) that the problem isn’t on the older firmware. Two different sensors reporting fails only on a newer software variant. That would seem like this is either something in my specific setup or on the newer software only. Or simply something in the test So, to get some further evidence on that I took a Pixhawk 4 sitting here (never flown, never crashed, etc - the last test was on a Pixhawk 1 [with 2mb of flash running fmu_v3]) and did the same tests as before. The same thing occurred. See attached. So now that accel and gyro is failing on a new Pixhawk 4 and Pixhawk 2 under 1.9.2 means I would like to think it is not hardware and is either something I have set inadvertently (although I reset all parameters) or something, like you said is driving this haywire. The big differing thing that is really important here which may make it hardware related is between the two sets of tests, the magnitude of differences in sensor offsets (they are actually quite close on the Pixhawk 4 under both firmware versions, but then still failing the sensor tests only on 1.92).

Pixhawk 4 on 192 stable
CAL_ACC0_EN 1
CAL_ACC0_ID 3866634
CAL_ACC0_XOFF -1.228418350219726563
CAL_ACC0_XSCALE 0.936501502990722656
CAL_ACC0_YOFF 0.091481208801269531
CAL_ACC0_YSCALE 1.007555007934570313
CAL_ACC0_ZOFF 0.078575611114501953
CAL_ACC0_ZSCALE 0.985880851745605469
CAL_ACC1_EN 1
CAL_ACC1_ID 4260618
CAL_ACC1_XOFF 0.592797756195068359
CAL_ACC1_XSCALE 0.987311780452728271
CAL_ACC1_YOFF -0.187482357025146484
CAL_ACC1_YSCALE 0.989881873130798340
CAL_ACC1_ZOFF 0.174437999725341797
CAL_ACC1_ZSCALE 0.988425850868225098

Pixhawk 4 on 180

CAL_ACC0_EN 1
CAL_ACC0_ID 3866634
CAL_ACC0_XOFF -1.229876995086669922
CAL_ACC0_XSCALE 0.936062276363372803
CAL_ACC0_YOFF 0.068903923034667969
CAL_ACC0_YSCALE 1.001949191093444824
CAL_ACC0_ZOFF 0.082245349884033203
CAL_ACC0_ZSCALE 0.982810974121093750
CAL_ACC1_EN 1
CAL_ACC1_ID 4260618
CAL_ACC1_XOFF 0.581225395202636719
CAL_ACC1_XSCALE 0.985325038433074951
CAL_ACC1_YOFF -0.191888332366943359
CAL_ACC1_YSCALE 0.984586596488952637
CAL_ACC1_ZOFF 0.212089061737060547
CAL_ACC1_ZSCALE 0.983702719211578369

Screen Shot 2019-09-10 at 10 28 54 AM

Screen Shot 2019-09-10 at 10 27 48 AM

@dagar
Copy link
Member

dagar commented Sep 10, 2019

Can you check sensors status instead? The "sensors test" within the tests command needs to be updated to use orb or removed entirely.

@dagar
Copy link
Member

dagar commented Sep 10, 2019

#12937

@ryanjAA
Copy link
Contributor Author

ryanjAA commented Sep 13, 2019

Screen Shot 2019-09-12 at 4 23 35 PM

Here is the sensors status on 180

@ryanjAA
Copy link
Contributor Author

ryanjAA commented Sep 13, 2019

Do you need to see them on 192?

@dagar
Copy link
Member

dagar commented Sep 17, 2019

A lot of the output is missing (accel section, etc). Can you try again?

@mcsauder
Copy link
Contributor

@ryanjAA , I'll try to get some time into this tomorrow. Thanks for your work so far!

@ryanjAA
Copy link
Contributor Author

ryanjAA commented Sep 17, 2019

@dagar redid it (same unit). Here:

Screen Shot 2019-09-17 at 2 36 48 PM

Screen Shot 2019-09-17 at 2 36 54 PM

@mcsauder
Copy link
Contributor

Hi @ryanjAA , it somehow looks like the values you have shown in bold above are additive, meaning the value of CAL_ACC1Z_OFF looks like it might if the x offset plus the y offset plus the z offset. I do agree that is not right.

Unfortunately, when I check my flying airframe with v1.8.2 downloaded here, v1.9.2 downloaded here, and with current master branch, I'm not seeing the same issue you are seeing. From my autopilot after calibrating with v1.8.2, then again with v1.9.2, and finally with master, these are my calibration values, (they all seem to agree very well):
image

Could you try loading v1.8.2 from the release binaries, calibrate, then v1.9.2 from the release binaries, and recalibrate and post the values reported back for each from param show CAL_ACC* so I can examine them again? Also, what hardware are you on?

@ryanjAA
Copy link
Contributor Author

ryanjAA commented Sep 19, 2019

So heres a strange thing, I just opened another MRO Pixhawk 1 (wanted to see with a brand new unit). Never put 1.8x on it, just updated to stable 1.9.2 (fmu_v3) and did all the calibration after loading in our used settings for everything. Ran param show CAL_ACC* and get strange offsets:

Screen Shot 2019-09-19 at 12 40 57 PM

So the question then probably becomes if this isn't firmware directly, what setting(s) in a param file can mess with this to such an extent?

@mcsauder
Copy link
Contributor

@ryanjAA , can you try resetting all of your CAL_* values to default, recalibrating, and see if the issue persists? Also, can you look at the param file you are loading onto blank autopilots and see if the CAL_ACC* values are way off zero?

@ryanjAA
Copy link
Contributor Author

ryanjAA commented Sep 21, 2019

Unfortunately, thats actually what i did before doing the calibration, i figured setting them to default would be the best thing.

Interestingly, even though the offset is incredibly different on the two sensors (ACC0 and ACC1 and maybe that is actually within what is acceptable), the error is sporadic. For instance, i was able to arm it just now but there doesn't seem to be a rhyme or reason to it for when and when not.

As to the CAL_ACC* in the canned config file, they are as follows:

CAL_ACC0_XOFF -0.040366649627685547
CAL_ACC0_XSCALE 0.992640852928161621
CAL_ACC0_YOFF -0.150812089443206787
CAL_ACC0_YSCALE 1.005856990814208984
CAL_ACC0_ZOFF 0.734849452972412109
CAL_ACC0_ZSCALE 1.001375198364257813
CAL_ACC1_EN 1
CAL_ACC1_ID 1114634
CAL_ACC1_XOFF 0.320536136627197266
CAL_ACC1_XSCALE 0.997538805007934570
CAL_ACC1_YOFF 1.158816933631896973
CAL_ACC1_YSCALE 1.031461834907531738
CAL_ACC1_ZOFF 1.378924727439880371
CAL_ACC1_ZSCALE 1.056028246879577637
CAL_ACC_PRIME 1246218

@mcsauder
Copy link
Contributor

Well... this is a real pickle... I knew when I told you it would be work for you it would be work for me too... ;) I'll have a bit of time to dig into the code in the next few days. Learn what you can and I will do the same! I will report back as I make progress.

@julianoes julianoes added bug Drivers 🔧 Sensors, Actuators, etc labels Sep 25, 2019
@mcsauder
Copy link
Contributor

@ryanjAA , can you test again to see if this is present in the current master branch? So far I am unable to reproduce... I don't understand what's going on here. :/

@mcsauder
Copy link
Contributor

@ryanjAA , any updates? Nothing to report from my end.

@stale
Copy link

stale bot commented Mar 11, 2020

This issue has been automatically marked as stale because it has not had recent activity. Thank you for your contributions.

@stale stale bot added the stale label Mar 11, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Drivers 🔧 Sensors, Actuators, etc stale
Projects
None yet
Development

No branches or pull requests

4 participants