wsl2: using custom kernel on arm64

[zlim]

• Your Windows build number: (Type ver at a Windows Command Prompt)

Microsoft Windows [Version 10.0.19041.1]

• What you're doing and what's happening: (Copy&paste the full set of specific command-line steps necessary to reproduce the behavior, and their output. Include screen shots if that helps demonstrate the problem.)

Attempting to use a custom kernel, per https://github.com/MicrosoftDocs/WSL/releases/tag/18947.
The following is added to .wslconfig:

[wsl2]
kernel=C:\\Users\\z\\wsl2\\Image

(In PS, confirmed C:\\Users\\z\\wsl2\\Image is a valid path.)

• What's wrong / what should be happening instead:

WSL2 doesn't pick up the custom kernel as configured in .wslconfig.

• Other experiments:

1. Confirmed .wslconfig is being picked up. With following configuration:

[wsl2]
processors=2

confirmed WSL2 only shows two processors.

2. Confirmed custom kernel Image is valid image. WSL2 boots with custom kernel when Image is manually dropped in as replacement of C:\Windows\System32\lxss\tools\kernel.

[benhillis]

Are you by chance using an ARM64 device? Could you ensure you run WSL.exe --shutdown after adding the entry?

[zlim]

Are you by chance using an ARM64 device?

Yup, Surface Pro X to be specific.

Could you ensure you run WSL.exe --shutdown after adding the entry?

Yes, the shutdown command is run on every config update.

[zlim]

Looking at the logs (via https://github.com/microsoft/WSL/blob/master/CONTRIBUTING.md#start-collecting-logs), the only ErrorCode is 15003. What does this mean? What should I be looking for?

In this case, I removed C:\Windows\System32\lxss\tools\kernel so the only available kernel should be the one configured in .wslconfig. WSL2 doesn't startup.

Are there specific events to look for related to the handling of custom kernel?

[zlim]

The closest thing I could find in Event Viewer related to kernel is under Microsoft-Windows-Hyper-V-Worker/Admin/Hyper-V-Chipset.

Both success/fail cases show the exact same "kernel" path in the VmBootSourceResult:

<DevicePath>AcpiEx(VMBus,0,0)/VenHw(...)/\kernel</DevicePath>

I'm guessing there's another mapping that WSL2 would've done before launching the VM.

[benhillis]

@zlim - Unfortunately the custom kernel .wslconfig option is not supported on ARM64 yet. We are looking to add support for this in the future.

[zlim]

@benhillis - thanks for looking into adding support.

[Frankdwu]

Is there any other way to use the custom kernel on ARM64?

[Biswa96]

Is there any other way to use the custom kernel on ARM64?

1. Shutdown any current WSL instances wsl.exe --shutdown.
2. Take ownership C:\Windows\System32\lxss\tools folder to current user.
3. Rename C:\Windows\System32\lxss\tools\kernel to kernel.old.
4. Put the new Linux kernel in that path with kernel name.

[Frankdwu]

Is there any other way to use the custom kernel on ARM64?

Shutdown any current WSL instances wsl.exe --shutdown.
Take ownership C:\Windows\System32\lxss\tools folder to current user.
Rename C:\Windows\System32\lxss\tools\kernel to kernel.old.
Put the new Linux kernel in that path with kernel name.

Thanks for your help.
I have tried this step with my custom kernel, but when I am launching my Ubuntu it failed.
I checked the etl logs, it shows one error but I don't find the exact error message.
Attached the error here.

lxcore_kernel_service_user.zip

[Biswa96]

I don't know what happens in ARM64 but I was doing those steps at first of WSL2 release (i.e. when custom kernel was not officially supported).

[Frankdwu]

I don't know what happens in ARM64 but I was doing those steps at first of WSL2 release (i.e. when custom kernel was not officially supported).

Thanks for your info.
I checked the errors it is "The operation timed out because a response was not received from the virtual machine or container."
lxcore_service.etl error is "onecore\vm\wsl\lxss\lxcmdlineshared\hvsocket.cpp" and error code is "0x80370109"
lxcore_user.etl exception is "onecore\vm\wsl\lxss\wsl\main.cpp" and "0x80370109".

Maybe this error is ARM64 specific

[hmartinez82]

2021 update. The .wslconfig kernel entry is still not being honored, but I was able to run a custom kernel replacing the kernel file under lxss.

I didn't need a newer kernel, I just needed the same kernel, but with extra modules and supported features (from https://github.com/rpasek/usbip-wsl2-instructions) . The trick was to copy the current config and apply my chances on top of it with make menuconfig as described in the steps in the link.

The kernel version I used is 5.4.72 from the output file: /arch/arm64/boot/Image

[benhillis]

Feature added in https://github.com/microsoft/WSL/releases/tag/1.1.5.