Kernel panic in macOS causing hard system hang #93
Comments
|
Thanks for the report. I was able to correlate two addresses from the call trace with the disassembly of HAXM 7.3.0 KEXT ( Is it possible for us to reproduce the issue? Alternatively, if you have the |
|
Thank you for the fast response! The way I've been debugging this is to add instruction decodes around VM exits1. Because the crash brings my entire system down (along with my logging mechanisms), it is difficult to give a precise answer. However, I've crudely resorted to using my cellphone to capture a video of my console log just before it crashes so I did manage to get some data. The culprit appears to be: test %esi, 0xfec00130The address decodes to an AC97 MMIO register in my QEMU system. 1: I was previously using version 7.0.0 which also has issues with |
When decoding an instruction with an unsupported opcode (indicated by the INSN_NOTIMPL flag), em_decode_insn() does not fail, which can lead to a disaster in em_emulate_insn(), e.g. calling an invalid handler function (soft_handler == NULL) and causing a host kernel panic (#93). 1. As soon as the opcode is decoded, check if it is unsupported. If so, return a fatal error, raise a vCPU panic, and log the raw bytes of the instruction. 2. In em_emulate_insn(), make sure soft_handler is valid before calling it. 3. Before decoding a new instruction, reset the emulation context, so the old context is not accidentally referred to. 4. Add a unit test for the unsupported opcode case. This requires refactoring EmulatorTest::run() first. Signed-off-by: Yu Ning <yu.ning@intel.com>
|
Thanks for providing the crucial piece of information! This a) Any unsupported MMIO instruction crashes the host. I've fixed a) with #95. Attached is a signed
This time, hopefully you'll get a guest crash instead of a host crash, and HAXM should report |
|
I will give it a try. So, your goal is for it to capture the correct crash info for you to further debug? |
|
I tried to install it but got: Untrusted kexts are not allowed |
Ah, not really. It's actually pretty easy to implement opcode It's weird that the KEXT load error you got does not mention |
When decoding an instruction with an unsupported opcode (indicated by the INSN_NOTIMPL flag), em_decode_insn() does not fail, which can lead to a disaster in em_emulate_insn(), e.g. calling an invalid handler function (soft_handler == NULL) and causing a host kernel panic (#93). 1. In em_decode_insn(), check if it is unsupported, i.e. either the INSN_NOTIMPL flag is set or there is no emulation handler. If so, return a fatal error, raise a vCPU panic, and log the raw bytes of the instruction. 2. Before decoding a new instruction, reset the emulation context, so the old context is not accidentally referred to. 3. Add unit tests for some unsupported opcode cases. This requires refactoring EmulatorTest::run() first. Signed-off-by: Yu Ning <yu.ning@intel.com>
Thanks to the scalable emulator design, implementing opcode 0x84 and 0x85 (TEST r/mN, rN) is quite simple. In addition, add a unit test for the TEST instruction, and update one of the unimplemented opcode unit tests to use XCHG instead of TEST. Fixes #93. Signed-off-by: Yu Ning <yu.ning@intel.com>
When decoding an instruction with an unsupported opcode (indicated by the INSN_NOTIMPL flag), em_decode_insn() does not fail, which can lead to a disaster in em_emulate_insn(), e.g. calling an invalid handler function (soft_handler == NULL) and causing a host kernel panic (#93). 1. In em_decode_insn(), check if the opcode is unsupported, i.e. the INSN_NOTIMPL flag is set or there is no emulation handler. If so, return a fatal error, raise a vCPU panic, and log the raw bytes of the instruction. 2. Before decoding a new instruction, reset the emulation context, so the old context is not accidentally referred to. 3. Add unit tests for two unsupported opcode cases. This requires refactoring EmulatorTest::run() first. Signed-off-by: Yu Ning <yu.ning@intel.com>
Thanks to the scalable emulator design, implementing opcode 0x84 and 0x85 (TEST r/mN, rN) is quite simple. In addition, add a unit test for the TEST instruction, and update one of the unimplemented opcode unit tests to use XCHG instead of TEST. Fixes #93. Signed-off-by: Yu Ning <yu.ning@intel.com>
When decoding an instruction with an unsupported opcode (indicated by the INSN_NOTIMPL flag), em_decode_insn() does not fail, which can lead to a disaster in em_emulate_insn(), e.g. calling an invalid handler function (soft_handler == NULL) and causing a host kernel panic (#93). 1. In em_decode_insn(), check if the opcode is unsupported, i.e. the INSN_NOTIMPL flag is set or there is no emulation handler. If so, return a fatal error, raise a vCPU panic, and log the raw bytes of the instruction. 2. Before decoding a new instruction, reset the emulation context, so the old context is not accidentally referred to. 3. Add unit tests for two unsupported opcode cases. This requires refactoring EmulatorTest::run() first. Signed-off-by: Yu Ning <yu.ning@intel.com>
I was able to load the KEXT on another MacBook, so the KEXT and the install instructions I provided should be fine, although I could add a few details. Since I've updated PR #95, let me update the signed KEXT as well: i. haxm-PR95-patch1.tar.gz This only includes (the latest revision of) my first patch. Therefore, it doesn't implement opcode More detailed instructions on testing these KEXTs:
@mborgerson Could you try these steps? @billtlee I'm still not sure why you got those "invalid signature" errors. It's possible that our then the test KEXT is indeed loaded. Note that the |
|
I don't know why I can't load the kernel... Bills-MacBook-Pro:Downloads Bill$ sudo kextutil intelhaxm.kext Bills-MacBook-Pro:Downloads Bill$ kextstat | grep intelhaxm |
OK, so this is the real error. Could you show me the output of:
My guess is that
In particular, |
|
Bills-MacBook-Pro:Downloads Bill$ sudo tar xpf haxm-PR95-patch2.tar.gz |
|
Bills-MacBook-Pro:Downloads Bill$ ls -l intelhaxm.kext |
|
I'm on high sierra, does that matter? I read high sierra blocks 3rd party kernel's |
|
I'm also on High Sierra, so that's not the problem. I think macOS caches the KEXT you've failed to load in |
|
Bills-MacBook-Pro:Downloads Bill$ mkdir temp Does this mean it successfully installed? |
|
Yes it does, congratulations :) If you had started Console.app and prepared it for capturing HAXM logs before the |
|
I only see the following in the Library/Extensions folder though: ACS6x.kext |
|
I believe |
|
I have been experiencing consistent kernel panic crashes and apple support suggested i remove intelhaxm from library/extensions. After I removed it, crashes appear to have stopped. Stumbled upon this post. Wanted to see if the new versions address my crash since I need to use android emulator... Not 100% sure my issue is the same as mborgerson's. |
|
Ah, I just saw your comment on #88. Let's discuss your issue there. |
Currently trying to emulate an x86-based system using QEMU with HAXM on macOS. When using the HAXM accelerator, I'm able to cause a reproducible kernel panic which results in an entire system hard reset (making debugging a significant challenge). Appears to be related to the
testinstruction, or emulation thereof. Thanks!The text was updated successfully, but these errors were encountered: