Internal Error Reporting Page #15916

Hwurzburg · 2020-11-27T14:57:27Z

This page is for anyone to report an Internal Error ("Internal Error:" in GCS or DataFlash messages). This allows the Dev team to be alerted, and to investigate as soon as possible.

Please enter the firmware used, vehicle type, and the Internal Error message. If you have a log, please retain it for 30 days in case a Developer would like to examine it.

Note: follow up timing on a new issue will depend on many factors, so some may not be addressed immediately.

The list of internal error numbers and meaning can be found here.

jini93 · 2020-12-04T07:13:09Z

hi
I'm Korean. Let me ask you a question.
Please understand if I say something weird
My message is like a picture.)

ndotkarkut · 2020-12-07T19:18:57Z

I receive Internal Error 0x1000. Pixhawk1 4.0.5 Stable Firmware Build. HexaCopter.

peterbarker · 2020-12-08T21:47:58Z

hi
I'm Korean. Let me ask you a question.
Please understand if I say something weird
My message is like a picture.)

What version of the firmware was this with? A git hash is prefered.

peterbarker · 2020-12-08T21:49:43Z

I receive Internal Error 0x1000. Pixhawk1 4.0.5 Stable Firmware Build. HexaCopter.

When did this happen? Close to boot time? After doing something with the device (e.g. flashing a bootloader)?

Just after upgrading to 4.0.5?

alvinc103 · 2020-12-10T01:28:01Z

Internal Error message: PREARM: INTERNAL ERRORS (0x4000)
Firmware used: Arduplane V4.0.7
Vehicle type: Wing

Prearm error log: https://www.dropbox.com/s/qnyhvh19c59f4rf/2020-12-09%2009-16-03.bin?dl=0

After I cycled the power I was able to arm, but the plane crashed during takeoff. It banked to the left and crashed. Not sure if it is related to the error but it would be great to find out what caused the crash.

Crash log: https://www.dropbox.com/s/i862unbg7bw45bi/2020-12-09%2009-21-51.bin?dl=0

Thanks

giacomo892 · 2021-01-06T11:43:28Z

Matek F405-STD
I was downloading logs from SDCARD via MAVLink running master as 4 jan

Hwurzburg · 2021-02-04T01:11:03Z

Attempting to re-arm after landing a TVBS using MatekH743 arming was prevented by an internal error.....it has IMU0 disabled since the ICM20602 drifts over time...only IMU1 is used....

PreArm:INTERNAL ERRORS 0X1000000 L:160 IMU_RESET

bnsgeyer · 2021-03-19T04:05:37Z

@peterbarker @rmackay9 @tridge
Reporting the internal error that resulted in a hard fault and watchdog reset. This has occurred twice for this user with a usable watchdog reset code. The first time it occurred was reported in this discuss post and the second time reported in the same thread but here is the post.
In speaking with Peter Barker about this, he seems to think there is an issue with IRQ handling. receiver is FrSky R-XSR, connected via S-BUS to the PIXHAWK 4 mini.

User reported that he had flown master on this particular setup with no issues prior to loading 4.0.6.

User setup details:
500 size Heli:
MSG, 648550, ArduCopter V4.0.6-ATNH-rc1 (780c496)
MSG, 648566, ChibiOS: d4fce84e
MSG, 648590, PH4-mini 003F0027 31385118 35343633
This code is a version of heli autotune built on the stable version of 4.0.6. branch is here

Log files after the watchdog reset are here. ELF and object dump for latest log is in the same link. The previous log used the same branch built on an older compiler version and I didn't keep the elf or object dump.

Here is the decoded WDOG msg
pbarker@bluebottle:~/rc/ardupilot(pr/decode-watch)$ ./Tools/scripts/decode_watchdog.py "WDOG, 2641424, -3, 0, 0, 0, 0, 0, 0, 122, 3, 0, 181, 4196355, 135203219, SPI1"
T Scheduler Task: -3: Waiting for sample
IE Internal Error Mask: 0: ?????
IEC Internal Error Count: 0: 0
IEL Internal Error Line: 0: 0
MM MAVLink Message: 0: [None]
MC MAVLink Command: 0: [None]
SL Semaphore Line: 0: Not waiting on semaphore
FL Fault Line: 122: ?????
FT Fault Type: 3: HardFault
FA Fault Address: 0x0: ?????
FTP Fault Thread Priority: 181: ?????
FICSR Fault ICS Register: 0x4196355: [Below]
VECTACTIVE: 3 (Hard fault)
RESERVED1: 0
RETOBASE: 1 (no (or no more) active exceptions)
VECTPENDING: 0 (Thread mode)
RESERVED2: 0
ISRPENDING: 1 (Interrupt pending)
RESERVED3: 0
PENDSTCLR: 0 (WO clears SysTick exception)
PENDSTSET: 0 (SysTick not pending)
PENDSVCLR: 0 (WO clears pendsv exception)
PENDSVSET: 0 (SysTick not pending)
RESERVED4: 0
NMIPENDSET: 0 (NMI not pending)
FLR Fault Long Return Address: 0x80f0993: ?????
TN Thread name: SPI1: ?????

and here is the portion of code the Fault address points to
80f0982: e7e2 b.n 80f094a <_ZN7ChibiOS9SPIDevice8transferEPKhmPhm+0x66>
80f0984: 44d8 add r8, fp
80f0986: 2100 movs r1, #0
80f0988: 462a mov r2, r5
80f098a: 607b str r3, [r7, #4]
80f098c: 4640 mov r0, r8
80f098e: f022 f959 bl 8112c44
80f0992: 4630 mov r0, r6
80f0994: 4659 mov r1, fp
80f0996: 465a mov r2, fp
80f0998: 687b ldr r3, [r7, #4]
80f099a: f7ff fee5 bl 80f0768 <_ZN7ChibiOS9SPIDevice11do_transferEPKhPhm>
80f099e: 2800 cmp r0, #0
80f09a0: d0d9 beq.n 80f0956 <_ZN7ChibiOS9SPIDevice8transferEPKhmPhm+0x72>

For the second watchdog reset, here is the WDOG message decoded
./Tools/scripts/decode_watchdog.py "WDOG {TimeUS : 183209104, Tsk : -3, IE : 0, IEC : 0, IEL : 0, MvMsg : 0, MvCmd : 0, SmLn : 0, FL : 122, FT : 3, FA : 0, FP : 177, ICSR : 4450307, LR : 135314015, TN : rcin}"
T Scheduler Task: -3: Waiting for sample
IE Internal Error Mask: 0: ?????
IEC Internal Error Count: 0: 0
IEL Internal Error Line: 0: 0
MM MAVLink Message: 0: [None]
MC MAVLink Command: 0: [None]
SL Semaphore Line: 0: Not waiting on semaphore
FL Fault Line: 122: ?????
FT Fault Type: 3: HardFault
FA Fault Address: 0x0: ?????
FTP Fault Thread Priority: 177: ?????
FICSR Fault ICS Register: 0x4450307: [Below]
VECTACTIVE: 3 (Hard fault)
RESERVED1: 0
RETOBASE: 1 (no (or no more) active exceptions)
VECTPENDING: 62 (IRQ46)
RESERVED2: 0
ISRPENDING: 1 (Interrupt pending)
RESERVED3: 0
PENDSTCLR: 0 (WO clears SysTick exception)
PENDSTSET: 0 (SysTick not pending)
PENDSVCLR: 0 (WO clears pendsv exception)
PENDSVSET: 0 (SysTick not pending)
RESERVED4: 0
NMIPENDSET: 0 (NMI not pending)
FLR Fault Long Return Address: 0x135314015: ?????
TN Thread name: rcin: ?????

The HEX value is 0x810ba5f and the section of code is below

if (port_irq_enabled(sts)) {
810ba3e: 4605 mov r5, r0
810ba40: b978 cbnz r0, 810ba62 <chSysRestoreStatusX+0x26>
__ASM volatile ("MRS %0, ipsr" : "=r" (result) );
810ba42: f3ef 8405 mrs r4, IPSR
810ba46: f3c4 0408 ubfx r4, r4, #0, #9
if (port_is_isr_context()) {
810ba4a: b124 cbz r4, 810ba56 <chSysRestoreStatusX+0x1a>
_stats_stop_measure_crit_isr();
810ba4c: f7ff ff2e bl 810b8ac <_stats_stop_measure_crit_isr>
__ASM volatile ("MSR basepri, %0" : : "r" (basePri) : "memory");
810ba50: f385 8811 msr BASEPRI, r5
810ba54: bd38 pop {r3, r4, r5, pc}
chSysUnlockFromISR();
}
else {
chSchRescheduleS();
810ba56: f7ff fe69 bl 810b72c
_stats_stop_measure_crit_thd();
810ba5a: f7ff ff1b bl 810b894 <_stats_stop_measure_crit_thd>
810ba5e: f384 8811 msr BASEPRI, r4
810ba62: bd38 pop {r3, r4, r5, pc}

zhaodelong · 2021-03-23T20:36:25Z

I reported a internalerror 0x800 incident at https://discuss.ardupilot.org/t/arduplane-4-0-9-froze-in-the-middle-of-flight/69140/2

firmware used: arduplane 4.0.9 stable,
vehicle type: plane
the WDG message: decoded message in the forum link

Could you investigate this? logs and videos in the forum

axelisart · 2021-03-24T15:13:43Z

Hi, I (seems randomly but I'm not sure) often got Internal Error 0x1000000 (IMU_RESET). I use a Matek H743 FC in a Tricopter Frame. This started to happen in Arducopter 4.1. Never got this before in 4.0.6 (I migrated to 4.1 based on Matek Support advice as I got lot of ACCELS Inconsistent and Gyro Health problems...).
Connected peripherals:

RX Fli14+ (RX6)
Beitan BN880 GPS + Magnetometer (HMC5883) (Serial1 & i2C1)
Clone Sik Radio (Serial1)
2 Cameras on C1 & C2 (Back Caddx ANT & Front Foxeer Razer)
VTX AKK RaceRanger on Vtx output
3 ESCs Turnigy Multistar BLheli_32 ARM 33A on S1, S2 & S4
1 Servo MG996R for Yaw on S7
No CAN devices

Error Screenshot :

Frame Picture for reference (7 inch props) :

Target0815 · 2021-03-29T07:37:37Z

The combination of ArduPlane 4.1 DEV on a H743-Wing is definitely not ok ...

I am currently building a plane with this combination. There are the following problems:

The setup goes corrupt after a reboot. There are currently 98 backup files (STRG0.bak - STR98.bak) in the APM/STRG_BAK directory of the SD card. Here probably a copy was inserted, in order to find the problem.

When trying to arm via MP (Data / Actions / Arm/Disarm) there is immediately an error: "PreArm: Internal errors 0x800000I:57 stack_ovrflw". You can force arming here, but not disarming anymore ...

I have attached the last 5 backup files.

Matek H743-Wing.zip

rain-er · 2021-04-02T18:50:16Z

Rover 4.1.0-dev "latest" on Matek H743
When arming/disarming:

Prearm: Param storage failed
followed by: InternalError: 0x4000020

giacomo892 · 2021-04-12T07:18:59Z

FIX IS IN MASTER

Copter master 1fe1e53
Matek F405-STD
LOG_DISARM=1 EKF_REPLAY=1

STABILIZE> log list
STABILIZE> Requesting log list
Log 1 numLogs 2 lastLog 2 size 1828703 Tue Jan 1 01:00:00 1980
Log 2 numLogs 2 lastLog 2 size 1568768

STABILIZE> log erase
STABILIZE> APM: PreArm: Throttle below Failsafe
APM: PreArm: Battery 1 low voltage failsafe
APM: PreArm: Internal errors 0x800000 l:59 stack_ovrflw

looks like storage thread stack overflow when deleting the logs on sdcard:

ThreadsV2
ISR PRI=255 sp=0x20000000 STACK=1296/1536
ArduCopter PRI=182 sp=0x20000600 STACK=5120/7168
idle PRI= 1 sp=0x200115E8 STACK=144/352
UART_RX PRI= 60 sp=0x2001DBE8 STACK=768/1104
OTG1 PRI= 60 sp=0x2001D308 STACK=264/656
monitor PRI=183 sp=0x2000CF48 STACK=472/848
timer PRI=181 sp=0x2000E168 STACK=1488/1872
rcout PRI=181 sp=0x2000D858 STACK=488/848
rcin PRI=177 sp=0x2000D2D0 STACK=1072/1360
io PRI= 58 sp=0x2000C5C0 STACK=1416/2384
storage PRI= 59 sp=0x2000DBE0 STACK=848/1360
UART1 PRI= 60 sp=0x2001C9B0 STACK=376/656
UART3 PRI= 60 sp=0x2001C150 STACK=256/656
UART4 PRI= 60 sp=0x2001B308 STACK=256/656
I2C0 PRI=176 sp=0x2001A728 STACK=896/1360
FrSky PRI=178 sp=0x20018E28 STACK=1040/1360
UART5 PRI= 60 sp=0x20018700 STACK=304/656
OSD PRI= 59 sp=0x20014210 STACK=664/1360
log_io PRI= 59 sp=0x20014910 STACK=48/1360
SPI1 PRI=181 sp=0x20013530 STACK=656/1360
FTP PRI= 58 sp=0x100088C0 STACK=1704/2896

juvinski · 2021-04-12T18:19:24Z

Hi guys,

I updated a qplane with ArduPlane V4.1.0dev (31c889f) - I was with 4.0.8 and after that - I'm running on a bbbmini - beaglebone green + bbbmini cape - I got my linux with more than 16.8 of load average and with 100% of cpu and memory usage.
I'm having this error message:
12/04/2021 15:07:38 : PreArm: Internal errors 0x1000000 l:168 imu_reset
12/04/2021 15:07:38 : PreArm: Bad GPS Position
12/04/2021 15:07:38 : PreArm: Compass not healthy
12/04/2021 15:07:28 : PreArm: Internal errors 0x1000000 l:168 imu_reset

I don't know if the load is caused by this imu_reset error or the error is caused by the high consuption.
with the 4.0.8 was working fine and the load was around 2%.
I'm having this message to: MPU: temp reset IMU[0] 7668 9

andyp1per · 2021-04-14T12:45:24Z

Copter master 1fe1e53
Matek F405-STD
LOG_DISARM=1 EKF_REPLAY=1

STABILIZE> log list
STABILIZE> Requesting log list
Log 1 numLogs 2 lastLog 2 size 1828703 Tue Jan 1 01:00:00 1980
Log 2 numLogs 2 lastLog 2 size 1568768

STABILIZE> log erase
STABILIZE> APM: PreArm: Throttle below Failsafe
APM: PreArm: Battery 1 low voltage failsafe
APM: PreArm: Internal errors 0x800000 l:59 stack_ovrflw

looks like storage thread stack overflow when deleting the logs on sdcard:

ThreadsV2
ISR PRI=255 sp=0x20000000 STACK=1296/1536
ArduCopter PRI=182 sp=0x20000600 STACK=5120/7168
idle PRI= 1 sp=0x200115E8 STACK=144/352
UART_RX PRI= 60 sp=0x2001DBE8 STACK=768/1104
OTG1 PRI= 60 sp=0x2001D308 STACK=264/656
monitor PRI=183 sp=0x2000CF48 STACK=472/848
timer PRI=181 sp=0x2000E168 STACK=1488/1872
rcout PRI=181 sp=0x2000D858 STACK=488/848
rcin PRI=177 sp=0x2000D2D0 STACK=1072/1360
io PRI= 58 sp=0x2000C5C0 STACK=1416/2384
storage PRI= 59 sp=0x2000DBE0 STACK=848/1360
UART1 PRI= 60 sp=0x2001C9B0 STACK=376/656
UART3 PRI= 60 sp=0x2001C150 STACK=256/656
UART4 PRI= 60 sp=0x2001B308 STACK=256/656
I2C0 PRI=176 sp=0x2001A728 STACK=896/1360
FrSky PRI=178 sp=0x20018E28 STACK=1040/1360
UART5 PRI= 60 sp=0x20018700 STACK=304/656
OSD PRI= 59 sp=0x20014210 STACK=664/1360
log_io PRI= 59 sp=0x20014910 STACK=48/1360
SPI1 PRI=181 sp=0x20013530 STACK=656/1360
FTP PRI= 58 sp=0x100088C0 STACK=1704/2896

Its not the storage thread its the log_io thread

See ArduPilot#15916 (comment)

tridge · 2021-04-14T20:25:49Z

Its not the storage thread its the log_io thread

thanks Andy, I've opened a PR to fix it
@giacomo892 please test
#17178

peterbarker · 2021-04-15T02:06:36Z

@tridge did you have a callpath in mind when you thought increasing the stack size might be the solution to this problem?

See #15916 (comment)

peterbarker · 2021-04-15T06:26:35Z

Rover 4.1.0-dev "latest" on Matek H743
When arming/disarming:
* Prearm: Param storage failed

* followed by: InternalError: 0x4000020

We believe this is failure to honour the sd-card-slowdown parameter.

A patch is being tested.

rain-er · 2021-04-15T12:24:27Z

Rover 4.1.0-dev "latest" on Matek H743
When arming/disarming:
* Prearm: Param storage failed

* followed by: InternalError: 0x4000020
We believe this is failure to honour the sd-card-slowdown parameter.

A patch is being tested.

This was fixed with:
#17117
https://discuss.ardupilot.org/t/h743-wing-rover-not-saving-changes-after-power-down/69442

SergeyBokhantsev · 2021-05-15T19:54:32Z

PreArm: Internal errors 0x800000 l:57 stack_ovrflw if set VTX_ENABLE 1

5/15/2021 10:49:30 PM : PreArm: Internal errors 0x800000 l:57 stack_ovrflw 5/15/2021 10:49:30 PM : PreArm: Throttle below Failsafe 5/15/2021 10:49:07 PM : u-blox 1 HW: 00080000 SW: ROM CORE 3.01 (107888) 5/15/2021 10:49:05 PM : GPS: u-blox 1 saving config 5/15/2021 10:49:01 PM : EKF3 IMU0 origin set 5/15/2021 10:49:00 PM : PreArm: Internal errors 0x800000 l:57 stack_ovrflw 5/15/2021 10:49:00 PM : PreArm: Throttle below Failsafe 5/15/2021 10:48:51 PM : GPS 1: detected as u-blox at 230400 baud 5/15/2021 10:48:49 PM : Frame: QUAD/X 5/15/2021 10:48:49 PM : IMU0: fast sampling enabled 8.0kHz/2.0kHz 5/15/2021 10:48:49 PM : RCOut: DS150:1-4 NeoP:5 5/15/2021 10:48:49 PM : BeastF7 00480036 3039510B 34323433 5/15/2021 10:48:49 PM : ChibiOS: c82ce90e 5/15/2021 10:48:49 PM : ArduCopter V4.1.0-beta1 (a47823a5)

VTX settings (TBS Unify Pro 5.8G HV - Race V2):

@andyp1per

wazoo22 · 2021-05-31T09:59:06Z

using an early pixhawk 1, delta wing, latest pixhawk 1M firmware, I also get: Pre|Arm: internal error 0x3000 and can't arm.

annoyinganime · 2021-06-02T07:42:59Z

Matek f405wse (MATEK F405 WING firmware), arducopter v4.1.0-dev. After arming and disarming quadcopter will not arm again, mission planner shows 0x100000 l:868 (flow_of_control) error. After rebooting quad will fly again one time, and fall back to that error again after disarming.

Update: after reading some posts here, I tried removing SD card and disabled logging completely: and it worked! I am now able to disarm and re-arm my quad, and get no error messages. But what to do with logging?

Sekilsgs2 · 2021-06-02T08:51:02Z

Mamba f405 mk2 ARDU 4.1 beta 3

02.06.2021 11:34:55 : PreArm: Internal errors 0x800000 l:183 stack_ovrfl
02.06.2021 11:34:55 : PreArm: Radio failsafe on
02.06.2021 11:34:25 : w
02.06.2021 11:34:25 : PreArm: Internal errors 0x800000 l:183 stack_ovrfl
02.06.2021 11:34:25 : PreArm: Radio failsafe on
02.06.2021 11:33:56 : w
02.06.2021 11:33:56 : PreArm: Internal errors 0x800000 l:183 stack_ovrfl
02.06.2021 11:33:56 : PreArm: Radio failsafe on
02.06.2021 11:33:26 : w
02.06.2021 11:33:26 : PreArm: Internal errors 0x800000 l:183 stack_ovrfl
02.06.2021 11:33:26 : PreArm: Radio failsafe on
02.06.2021 11:33:17 : EKF3 IMU0 initial yaw alignment complete
02.06.2021 11:33:17 : EKF3 IMU0 tilt alignment complete
02.06.2021 11:33:16 : EKF3 IMU0 initialised
02.06.2021 11:33:16 : EKF3 IMU0 forced reset
02.06.2021 11:33:11 : EKF3 IMU0 initial yaw alignment complete
02.06.2021 11:33:11 : EKF3 IMU0 tilt alignment complete
02.06.2021 11:32:53 : EKF3 IMU0 initialised
02.06.2021 11:32:53 : EKF3 IMU0 forced reset
02.06.2021 11:32:48 : EKF3 IMU0 initial yaw alignment complete
02.06.2021 11:32:48 : EKF3 IMU0 tilt alignment complete
02.06.2021 11:32:45 : EKF3 IMU0 initialised
02.06.2021 11:32:45 : EKF3 IMU0 forced reset
02.06.2021 11:32:39 : EKF3 IMU0 initial yaw alignment complete
02.06.2021 11:32:39 : EKF3 IMU0 tilt alignment complete
02.06.2021 11:32:26 : EKF3 IMU0 initialised
02.06.2021 11:32:26 : EKF3 IMU0 forced reset
02.06.2021 11:32:21 : EKF3 IMU0 initial yaw alignment complete
02.06.2021 11:32:21 : EKF3 IMU0 tilt alignment complete
02.06.2021 11:32:15 : EKF3 IMU0 initialised
02.06.2021 11:32:15 : EKF3 IMU0 forced reset
02.06.2021 11:31:52 : PreArm: Radio failsafe on
02.06.2021 11:31:22 : PreArm: Radio failsafe on
02.06.2021 11:30:52 : PreArm: Radio failsafe on
02.06.2021 11:30:22 : PreArm: Radio failsafe on
02.06.2021 11:29:52 : PreArm: Radio failsafe on
02.06.2021 11:29:23 : PreArm: Radio failsafe on
02.06.2021 11:28:53 : PreArm: Radio failsafe on
02.06.2021 11:28:23 : PreArm: Radio failsafe on
02.06.2021 11:28:10 : Frame: QUAD/X
02.06.2021 11:28:10 : IMU0: fast sampling enabled 8.0kHz/1.0kHz
02.06.2021 11:28:10 : RCOut: DS300:1-4
02.06.2021 11:28:10 : MambaF405v2 0042003C 424D5004 20393954
02.06.2021 11:28:10 : ChibiOS: 08877972
02.06.2021 11:28:10 : Frame: QUAD/X
02.06.2021 11:28:10 : IMU0: fast sampling enabled 8.0kHz/1.0kHz
02.06.2021 11:28:10 : RCOut: DS300:1-4
02.06.2021 11:28:10 : MambaF405v2 0042003C 424D5004 20393954
02.06.2021 11:28:10 : ChibiOS: 08877972

This is after i'm run save all logs from spi flash

IamPete1 · 2021-06-05T10:52:26Z

I have had a constraining nan internal error on Copter 4.1.beta3 line number 234. During log download over USB, whole system seems to lock up for 1 second in every five and you end up with lots of empty log files on the SD. Sounds the same as #15916 (comment) .

Some greping for constrains on that line suggest the internal error is a symptom of the system locking up and not the root cause.

xcl010 · 2021-06-06T10:43:55Z

This page is for anyone to report an Internal Error ("Internal Error:" in GCS or DataFlash messages). This allows the Dev team to be alerted, and to investigate as soon as possible.

Please enter the firmware used, vehicle type, and the Internal Error message. If you have a log, please retain it for 30 days in case a Developer would like to examine it.

Note: follow up timing on a new issue will depend on many factors, so some may not be addressed immediately.

The list of internal error numbers and meaning can be found here.

STRG_BAK.zip

Matek H743-Wing

Git HEAD detached at 3582b69

The flight controller restarted in the air, the aircraft landed in an emergency, and the flight controller restarted repeatedly after landing. After the power is off, the flight control cannot be started, and it stops at calibrating the barometer. It can only be started by refreshing the firmware with BL using STM32CubeProgrammer, help, please!
h743-wing-cash.zip

geoffreybainathsah · 2021-06-11T10:26:03Z

Hi
i had an issue with mt H743 board using 4.1.0DEV arduplane fw.
my airplane is a glider style airframe using
-BLheli ESC with telemetry
-2x FPV camera using internal relay switcher and internal OSD
-RFD900 telemetry
-Crossfire RX with mavlink connection

The error happend after 131min in the air, while performing AUTO mission.
mission planner showed error message 'Internal error 0x1000000' followed by failsafe message
I then activated RTL mode and safely landed the aircraft
Luckily no reboot observed but i did have a backup 5V regulator connected to the 5V pin on the UART port which consumed 650mah during this flight

Attached is a screenshot of the error in MP and a Tlog file (incident at 91%)

2021-06-05 16-39-09.zip

note: Internal error and Failsafe could not be found on the Internal SD log file

rmackay9 · 2021-06-11T13:16:19Z

@geoffreybainathsah, this is most likely a hardware error because we have seen quite a few reports recently about this board and this error. We've reported it to Mateksys but it is really up to them whether they do about it. Here is a topic in the discuss forums about this: https://discuss.ardupilot.org/t/prearm-internal-errors-0x1000000-l-168-imu-reset/71658

Sekilsgs2 · 2021-07-18T10:16:31Z

18.07.2021 13:12:55 : PreArm: Internal errors 0x400 l:234 cnstring_nan
18.07.2021 13:12:25 : PreArm: Internal errors 0x400 l:234 cnstring_nan
18.07.2021 13:12:23 : EKF3 IMU0 MAG0 initial yaw alignment complete
18.07.2021 13:12:23 : EKF3 IMU0 tilt alignment complete
18.07.2021 13:12:22 : EKF3 IMU0 initialised
18.07.2021 13:12:22 : EKF3 IMU0 forced reset
18.07.2021 13:12:13 : Calibration successful
18.07.2021 13:12:12 : CRSF: RF mode 2, rate is 157Hz
18.07.2021 13:12:09 : Initialising ArduPilot
18.07.2021 13:12:07 : CRSF: RF mode 1, rate is 158Hz
18.07.2021 13:12:05 : Initialising ArduPilot
18.07.2021 13:12:00 : CRSF: RF mode 2, rate is 156Hz
18.07.2021 13:12:00 : Initialising ArduPilot
18.07.2021 13:11:55 : CRSF: RF mode 1, rate is 158Hz
18.07.2021 13:11:55 : Initialising ArduPilot
18.07.2021 13:11:50 : Initialising ArduPilot
18.07.2021 13:11:45 : Initialising ArduPilot
18.07.2021 13:11:40 : Initialising ArduPilot
18.07.2021 13:11:35 : Initialising ArduPilot
18.07.2021 13:11:35 : Disarm to allow calibration
18.07.2021 13:11:35 : Current
18.07.2021 13:11:35 : Starting calibration

After compas/mot calibration..

4.1 have many problems with ineternal errors - any fixes?

Maybe in 4.1 we have very bad optimisations when many threads cant working good with proper latency and maybe some irq's lost or have bad priority - i'm think this is main problem in 4.1 with many internal errors.

See ArduPilot#15916 (comment)

AndersonRayner · 2021-11-24T19:33:05Z

Pixhawk 4 running Copter V4.1.1 (01d1aa1), got error Internal errors 0x100000 I:256 flow_of_ctr while out testing today. Changes from previous configurations were an auto-tune and some playing around with the EFK3 parameter to try and get the altitude estimate to be more precise (and the heading). The upgrade to V4.1.1 was completed several flying trips ago and has flown ~10 flights with the new firmware without issue.

Additionally, during the second autonomous flight of the day (log 58) the tracking started going very poorly, ended up needing to recover in STAB mode (never had this issue before). Vehicle has over 100 flights (about 24 h flight time) with no previous issue.

Logs for today (*.BIN and tlogs), plus some quick flight notes and a photo of the UAV (12" props) can be found here. I have all the *.BIN logs from previous flights so can provide those as required.

Thanks!

rmackay9 · 2021-11-24T23:27:28Z

@AndersonRayner, the "256" means that it must be this line that is producing the error. Is it possible that one of the COMPASS_ORIENT or AHRS_ORIENT parameters has been set to CUSTOM? I'm not sure of the exact issue but I think it is related to this.

AndersonRayner · 2021-11-25T00:00:37Z

@rmackay9 - Yes - I have COMPASS_ORIENT set to CUSTOM (100) and COMPASS_CUS_YAW set to -4 (in degrees, as per the parameter description field in Mission Planner). I have just (as a sanity check) converted my value over to radians and the error persists (even through a full power cycle). Setting the COMPASS_ORIENT back to 0 appears to have fixed the problem, however I can't flight test it (to be sure) for a couple of days.

IamPete1 · 2021-11-25T00:41:55Z

@AndersonRayner Looks like you have a UAVCAN compass, custom rotations do not work for UAVCAN if I recall correctly. Something that #19164 will fix.

AndersonRayner · 2021-11-25T01:45:02Z

@IamPete1 - You are correct, I have a mag in the Here+ puck which is connected via CAN. I'll re-mount my compass a little more carefully so I don't need the custom corrections for now, then do the fine tuning once the PR goes through. Thanks for the quick responses!

timtuxworth · 2022-08-29T15:57:22Z

I'm getting an "Internal Errors 0x8000" on my Zealot H743 running ArduPlane 4.2.3 stable. This is happening on the bench (disarmed no GPS fix), so I'm not sure I have a log. I guess this means "Main loop stuck".
The device seems to recover after reporting this, but I think I've seen it a few times on the field, always before arming, so no log.
There's a screenshot here: https://www.dropbox.com/s/pcggwi3lbudregd/Screen%20Shot%202022-08-29%20at%208.51.53%20AM.png?dl=0

Hwurzburg · 2022-08-29T16:03:35Z

@timtuxworth you can get a log by setting LOG_DISARMED = 1

timtuxworth · 2022-08-29T16:14:54Z

@timtuxworth you can get a log by setting LOG_DISARMED = 1

Thanks Henry - ok enabled. Of course it doesn't happen on the next reboot. But I'll leave it on for a while and hope it pops up again.

timtuxworth · 2022-09-02T04:15:37Z

I just got another one, this time it is on a PixRacer R15 running 4.3.0 dev. It seems to be throwing 0x8000 about once every minute. See the screen shot.

Sorry it's a different FC from the one above, so LOG_DISARMED was off. I set LOG_DISARMED = 1 and rebooted but didn't see the error again.

This was "on the bench" powered by USB, happened while downloading log files.

KTechDev4213 · 2022-09-11T22:54:36Z

Recieved the following on a 3dr solo:

9/2/2022 2:26:45 PM : PreArm: Internal errors (0x800)
9/2/2022 2:26:45 PM : PreArm: Compasses inconsistent
9/2/2022 2:26:43 PM : EKF3 IMU1 is using GPS
9/2/2022 2:26:43 PM : EKF3 IMU0 is using GPS
9/2/2022 2:26:17 PM : EKF3 IMU1 origin set
9/2/2022 2:26:17 PM : EKF3 IMU0 origin set
9/2/2022 2:26:16 PM : u-blox 1 HW: 00070000 SW: 1.00 (59843)
9/2/2022 2:26:15 PM : PreArm: Internal errors (0x800)
9/2/2022 2:26:15 PM : PreArm: EKF starting GPS checks
9/2/2022 2:26:15 PM : PreArm: Compasses inconsistent
9/2/2022 2:26:13 PM : GPS: u-blox 1 saving config
9/2/2022 2:26:10 PM : EKF3 IMU1 tilt alignment complete
9/2/2022 2:26:10 PM : EKF3 IMU0 tilt alignment complete
9/2/2022 2:26:07 PM : EKF3 IMU1 initial yaw alignment complete
9/2/2022 2:26:07 PM : EKF3 IMU0 initial yaw alignment complete
9/2/2022 2:26:07 PM : EKF3 IMU1 initialised
9/2/2022 2:26:07 PM : EKF3 IMU0 initialised
9/2/2022 2:26:06 PM : PreArm: Internal errors (0x800)
9/2/2022 2:26:06 PM : PreArm: EKF starting GPS checks
9/2/2022 2:26:06 PM : PreArm: Compasses inconsistent

JAEBOV · 2022-10-05T11:52:10Z

Currently having an issue with an internal error message on a Arduplane 4.2.2 ( 693c9d9 ) system running on a Cube orange on a ADS-B standard Carrier Board from cubepilot.

Anybody else having a similar issue at the moment ? Or having an idea how to track the cause of it ?

One more thing we saw is that the raw_temp output of this cube does not show 0.0 all the time, but not sure if it is related tot this error message.

hendjoshsr71 · 2022-10-05T17:03:10Z

Please provide a log if possible.

Regarding raw_temp please update Mission Planner to the latest beta that has been fixed.

JAEBOV · 2022-10-06T07:48:27Z

Please provide a log if possible.

Regarding raw_temp please update Mission Planner to the latest beta that has been fixed.

First of all thanks for the fast reply and your time.

Log of the same error:

https://www.dropbox.com/s/1izax4zvzdlh4ev/Upload_00000113.BIN?dl=0

Regarding Mission Planner, we will update it good to know the reason for the raw_temp.

timtuxworth · 2022-10-28T21:01:52Z

I continue to see Internal Errors 8000 while downloading logs on the QioTek Zealot H743. Because I am not armed I don't have a log. I tried turning on log while disarmed and I don't think I have seen the error when I do that.
Happened again today on 4.3.1.

dl2ecc · 2022-11-04T13:18:09Z

I built an X8 Quad with Arducopter 4.2.3
During AUTOTUNE my Copter changed mode into LAND ( failsafe) and landed.
I got an internal error 0X100000 921: flow_of_ctr
I'm looking forward to your anwers.
Many Thanks
Wolfgang

samguru19 · 2022-11-15T06:34:21Z

I faced an internal error . I am having a hexacopter with arducopter 4.2 and here 3 gps.
The setup was working till yesterday but now I am seeing this error.

alariq · 2022-12-03T15:16:28Z

Matek F743 Slim.
Upon boot:

12/3/2022 5:07:49 PM :  FICSR4196355 MM0 MC0 IE0 IEC0 TN:log_
12/3/2022 5:07:49 PM : WDG: T-3 SL0 FL217 FT3 FA30030768 FTP59 FLR80B3691
12/3/2022 5:07:39 PM :  FICSR4196355 MM0 MC0 IE0 IEC0 TN:log_
12/3/2022 5:07:39 PM : WDG: T-3 SL0 FL217 FT3 FA30030768 FTP59 FLR80B3691
12/3/2022 5:07:29 PM :  FICSR4196355 MM0 MC0 IE0 IEC0 TN:log_
12/3/2022 5:07:29 PM : WDG: T-3 SL0 FL217 FT3 FA30030768 FTP59 FLR80B3691
12/3/2022 5:07:28 PM : PreArm: Internal errors 0x800 l:271 watchdog_rst
12/3/2022 5:07:28 PM : PreArm: Battery 1 below minimum arming voltage
12/3/2022 5:07:19 PM :  FICSR4196355 MM0 MC0 IE0 IEC0 TN:log_
12/3/2022 5:07:19 PM : WDG: T-3 SL0 FL217 FT3 FA30030768 FTP59 FLR80B3691
12/3/2022 5:07:09 PM :  FICSR4196355 MM0 MC0 IE0 IEC0 TN:log_
12/3/2022 5:07:09 PM : WDG: T-3 SL0 FL217 FT3 FA30030768 FTP59 FLR80B3691
12/3/2022 5:06:59 PM :  FICSR4196355 MM0 MC0 IE0 IEC0 TN:log_
12/3/2022 5:06:59 PM : WDG: T-3 SL0 FL217 FT3 FA30030768 FTP59 FLR80B3691
12/3/2022 5:06:57 PM : PreArm: Internal errors 0x800 l:271 watchdog_rst
12/3/2022 5:06:57 PM : PreArm: Battery 1 below minimum arming voltage
12/3/2022 5:06:51 PM : u-blox 1 HW: 00080000 SW: ROM CORE 3.01 (107888)
12/3/2022 5:06:49 PM : GPS 1: u-blox saving config
12/3/2022 5:06:48 PM :  FICSR4196355 MM0 MC0 IE0 IEC0 TN:log_
12/3/2022 5:06:48 PM : WDG: T-3 SL0 FL217 FT3 FA30030768 FTP59 FLR80B3691
12/3/2022 5:06:39 PM : Crash Dump incomplete, dumping what we got!
12/3/2022 5:06:39 PM : Crash Dump incomplete, dumping what we got!
12/3/2022 5:06:39 PM : Crash Dump incomplete, dumping what we got!
12/3/2022 5:06:39 PM : Crash Dump incomplete, dumping what we got!
12/3/2022 5:06:39 PM : Crash Dump incomplete, dumping what we got!
12/3/2022 5:06:39 PM : Crash Dump incomplete, dumping what we got!
12/3/2022 5:06:38 PM :  FICSR4196355 MM0 MC0 IE0 IEC0 TN:log_
12/3/2022 5:06:38 PM : WDG: T-3 SL0 FL217 FT3 FA30030768 FTP59 FLR80B3691
12/3/2022 5:06:38 PM : Frame: QUAD/X
12/3/2022 5:06:38 PM : IMU1: fast sampling enabled 2.0kHz
12/3/2022 5:06:38 PM : IMU0: fast sampling enabled 2.0kHz
12/3/2022 5:06:38 PM : RCOut: DS300:1-6 PWM:7-13
12/3/2022 5:06:38 PM : MatekH743 004B0022 31315119 31313534
12/3/2022 5:06:38 PM : ChibiOS: 66e5de0d
12/3/2022 5:06:38 PM : ArduCopter V4.3.0 (93448b71)

These lasst messages (on top) are constantly repeated.
This: PreArm: Battery 1 below minimum arming voltage is fine (battery is not connected)

Looking at this: WDG: T-3 SL0 FL217 FT3 FA30030768 FTP59 FLR80B3691 thread prio is 59, so I downloaded threads.txt
And here is how it looks:

ThreadsV2
ISR           PRI=255 sp=0x30000000 STACK=1312/1536
ArduCopter    PRI=182 sp=0x30000600 STACK=4064/7168
idle          PRI=  1 sp=0x3001B408 STACK=144/352
UART_RX       PRI= 60 sp=0x3003B5F8 STACK=768/1104
OTG1          PRI= 60 sp=0x3003A718 STACK=272/656
monitor       PRI=183 sp=0x30016C20 STACK=1016/1360
timer         PRI=181 sp=0x30018040 STACK=1448/1872
rcout         PRI=181 sp=0x30017730 STACK=472/848
rcin          PRI=177 sp=0x300171A8 STACK=488/1360
io            PRI= 58 sp=0x30016298 STACK=1536/2384
storage       PRI= 59 sp=0x30017AB8 STACK=848/1360
UART7         PRI= 60 sp=0x30039968 STACK=200/656
UART1         PRI= 60 sp=0x30038088 STACK=248/656
UART2         PRI= 60 sp=0x300375A8 STACK=272/656
UART3         PRI= 60 sp=0x30036AC8 STACK=368/656
OTG2          PRI= 60 sp=0x30035BE8 STACK=368/656
I2C0          PRI=176 sp=0x300349A8 STACK=848/1360
UART6         PRI= 60 sp=0x30033420 STACK=264/656
OSD           PRI= 59 sp=0x3002F8F8 STACK=920/1616
log_io        PRI= 59 sp=0x30030100 STACK=952/1912
I2C1          PRI=176 sp=0x30032398 STACK=904/1360
SPI1          PRI=181 sp=0x3002DAA8 STACK=784/1360
SPI4          PRI=181 sp=0x3002D518 STACK=784/1360
Scripting     PRI=  2 sp=0x30027770 STACK=16876/17744
FTP           PRI= 58 sp=0x2001EBE0 STACK=1008/2896

So it could be storage, OSD or log_io thread. Two of them are related to writing data and taking in account that crash dump could not be fully written is seems that something may be wrong with writing data. Though I do not have enough knowledge about ardupilot to make further investigation.
Any thoughts?

vandanadevatkal22 · 2023-03-07T07:13:56Z

Hi,
I have got an error message saying
"PreArm: Internal errors 0x10000000 I:363"

Attaching the image for reference

Any help would be appreciated.
Thanks in advance

awong97 · 2023-03-14T02:21:03Z

Hi,
Firmware: Arduplane 4.3.4
Hardware: Matek H743 WLITE
Airframe: Tricopter tilt rotor

I experienced an inflight error upon transiting from FBWA to QLOITER which gave an “internal error 400” message and let to loss of control of the plane with minimal damage. After disarming, the warning “Prearm : Internal errors 0x400” continued showing until power cycling the aircraft which seemed to fix the issue.
The same issue was replicated in a second flight with the same trigger condition ( FBWA to QLOITER transit)

Any help would be greatly appreciated

timtuxworth · 2023-06-12T20:19:45Z

Just got this with a 4.5.0 test build - as usual while downloading logs. Luckily I had log_disarm on for something else.
https://www.dropbox.com/s/h7mm2i51mz1gzmn/log_159_2023-6-12-13-48-58.bin?dl=0

timtuxworth · 2023-06-17T23:41:00Z

Another one - happened multiple (many, many) times, again while downloading some larger logs. This is the log of the download.
ArduPlane 4.5.0 (master)
Zealot H743

https://www.dropbox.com/s/hbi55d3n1pfhel3/log_193_2023-6-17-17-12-56.bin?dl=0

harelevy1 · 2023-09-29T07:53:18Z

After a flight of about 3 minutes I get this message 'Internal error 0x1000000'
Both in manual flight and automatic flight The flight controller is a cube blue with an AUTOPILOT board

After receiving the message, the drone enters FAIL SAFE and
I cannot continue the mission, it should be noted that it continues to fly normally after receiving the error

I use a cube blue but there is a version of an cube orange on it
Version 4.1.5 copter cube orange

Platform

[ ] Copter

Airframe type
normal copter

Hardware type
Pixhawk, Cube blue

rmackay9 mentioned this issue Jan 6, 2021

PreArm: Internal errors (0x3000) #16231

Closed

rmackay9 pinned this issue Feb 4, 2021

tridge added a commit to tridge/ardupilot that referenced this issue Apr 14, 2021

AP_Logger: raise log_io stack size

1d0186e

See ArduPilot#15916 (comment)

tridge mentioned this issue Apr 14, 2021

AP_Logger: raise log_io stack size #17178

Merged

tridge added a commit that referenced this issue Apr 15, 2021

AP_Logger: raise log_io stack size

7a7ac0c

See #15916 (comment)

This was referenced Jun 6, 2021

EKF3: Aiding and forced reset loop #16844

Closed

Parameters are not persistent on reboot #16949

Closed

LiYuanxing pushed a commit to LiYuanxing/ardupilot that referenced this issue Sep 29, 2021

AP_Logger: raise log_io stack size

ae241a8

See ArduPilot#15916 (comment)

rmackay9 unpinned this issue May 1, 2022

Internal Error Reporting Page #15916

Internal Error Reporting Page #15916

Comments

Hwurzburg commented Nov 27, 2020 • edited by rmackay9 Loading

jini93 commented Dec 4, 2020

ndotkarkut commented Dec 7, 2020

peterbarker commented Dec 8, 2020

peterbarker commented Dec 8, 2020

alvinc103 commented Dec 10, 2020

giacomo892 commented Jan 6, 2021

Hwurzburg commented Feb 4, 2021

bnsgeyer commented Mar 19, 2021 • edited Loading

zhaodelong commented Mar 23, 2021

axelisart commented Mar 24, 2021

Target0815 commented Mar 29, 2021 • edited Loading

rain-er commented Apr 2, 2021

giacomo892 commented Apr 12, 2021 • edited Loading

juvinski commented Apr 12, 2021

andyp1per commented Apr 14, 2021

tridge commented Apr 14, 2021

peterbarker commented Apr 15, 2021

peterbarker commented Apr 15, 2021

rain-er commented Apr 15, 2021

SergeyBokhantsev commented May 15, 2021

wazoo22 commented May 31, 2021 • edited Loading

annoyinganime commented Jun 2, 2021 • edited Loading

Sekilsgs2 commented Jun 2, 2021 • edited Loading

IamPete1 commented Jun 5, 2021

xcl010 commented Jun 6, 2021 • edited Loading

geoffreybainathsah commented Jun 11, 2021

rmackay9 commented Jun 11, 2021

Sekilsgs2 commented Jul 18, 2021 • edited Loading

AndersonRayner commented Nov 24, 2021 • edited Loading

rmackay9 commented Nov 24, 2021

AndersonRayner commented Nov 25, 2021

IamPete1 commented Nov 25, 2021

AndersonRayner commented Nov 25, 2021

timtuxworth commented Aug 29, 2022

Hwurzburg commented Aug 29, 2022

timtuxworth commented Aug 29, 2022

timtuxworth commented Sep 2, 2022 • edited Loading

KTechDev4213 commented Sep 11, 2022

JAEBOV commented Oct 5, 2022

hendjoshsr71 commented Oct 5, 2022

JAEBOV commented Oct 6, 2022

timtuxworth commented Oct 28, 2022

dl2ecc commented Nov 4, 2022

samguru19 commented Nov 15, 2022

alariq commented Dec 3, 2022 • edited Loading

vandanadevatkal22 commented Mar 7, 2023

awong97 commented Mar 14, 2023

timtuxworth commented Jun 12, 2023 • edited Loading

timtuxworth commented Jun 17, 2023 • edited Loading

harelevy1 commented Sep 29, 2023

Hwurzburg commented Nov 27, 2020 •

edited by rmackay9

Loading

bnsgeyer commented Mar 19, 2021 •

edited

Loading

Target0815 commented Mar 29, 2021 •

edited

Loading

giacomo892 commented Apr 12, 2021 •

edited

Loading

wazoo22 commented May 31, 2021 •

edited

Loading

annoyinganime commented Jun 2, 2021 •

edited

Loading

Sekilsgs2 commented Jun 2, 2021 •

edited

Loading

xcl010 commented Jun 6, 2021 •

edited

Loading

Sekilsgs2 commented Jul 18, 2021 •

edited

Loading

AndersonRayner commented Nov 24, 2021 •

edited

Loading

timtuxworth commented Sep 2, 2022 •

edited

Loading

alariq commented Dec 3, 2022 •

edited

Loading

timtuxworth commented Jun 12, 2023 •

edited

Loading

timtuxworth commented Jun 17, 2023 •

edited

Loading