Surface Book 2 UEFI firmware malfunction after firmware update in dualboot (Surface - Firmware - 394.651.768.0)

[Ramen-LadyHKG]

(This is not linux-surface problem. Can I get some help here?)

[Description of the bug or feature]

My Surface Book 2 just updated its UEFI firmware due to a Windows automatically update.
After that, my EFI table seems to have broken.

Except for the Windows operating system that I am dual booting with, all Linux systems are no longer bootable. Fedora and Arch.
For more details: https://www.reddit.com/r/SurfaceLinux/comments/146sjo7/my_sb2_stuck_on_logo_when_booting_windows_updates/

Solution:

Anyway, downgrading the firmware is the only viable option that I can think of.
https://github.com/linux-surface/surface-uefi-firmware
I tried to follow the instructions that you provided in this document.
However, Microsoft has not been releasing any newer firmware package since 26th-June,2022.
Surface Book 2 Drivers and Firmware:
Surface Book 2 update history:
But I don't understand why Microsoft is still releasing firmware updates through Windows Updates.

### Questions:

1. Is downgrading still viable?
   I've just read this post, Microsoft seems to have blocked user from doing that?
2. Where can I find newer surface firmware other than Windows Update?

Environment

• Hardware model: Surface Book 2 15"
• Kernel version: 6.3.5.arch-1-1-surface
• Distribution: Arch Linux

[Ramen-LadyHKG]

I've opened Reddit Posts in regard to this issue:
My SB2 stuck on logo when booting. ( Windows Updates might have involved after UEFI firmware updated) ** DON'T UPDATE Surface - Firmware - 394.651.768.0 **

Where do we report Surface firmware bug? WARNING ** SB2 user please DO NOT UPDATE FIRMWARE**

[StollD]

I literally cannot find that version number anywhere on the internet, not even in the Microsoft Update Catalog. Could this be an beta release published through Windows Insiders?

Also, fun fact: According to Microsoft the SB2 went EOL 12 days ago, on May 30th: https://learn.microsoft.com/en-us/surface/surface-driver-firmware-lifecycle-support

[StollD]

Also, to my knowledge, downgrades still work. What doesnt work is downgrading to the version of the firmware that allowed undervolting, because that firmware is deemed unsafe.

In any case, it doesnt hurt to try it.

[Ramen-LadyHKG]

Also, to my knowledge, downgrades still work. What doesnt work is downgrading to the version of the firmware that allowed undervolting, because that firmware is deemed unsafe.

In any case, it doesnt hurt to try it.

Thank you. I am going to do it in the live environment. Is that safe?

Since I couldn't boot into any Linux system that is installed on the SSD, I tried to recreate GRUB, but I am still stuck at the boot sequence.

[Ramen-LadyHKG]

I literally cannot find that version number anywhere on the internet, not even in the Microsoft Update Catalog. Could this be an beta release published through Windows Insiders?

Also, fun fact: According to Microsoft the SB2 went EOL 12 days ago, on May 30th: https://learn.microsoft.com/en-us/surface/surface-driver-firmware-lifecycle-support

Sure, Surface Book 2 was just end of support.

But the update history page ended at August 2022.
https://support.microsoft.com/en-us/surface/surface-book-2-update-history-8903a391-7a88-e6c1-5f61-5863795a7816

[StollD]

I dont know if a live environment will work because fwupd needs to put a binary on the ESP and add a bootentry for it.

Although I believe that the guys who downgraded to get undervolting probably used a live environment. Maybe check out what they did?

But the update history page ended at August 2022.
https://support.microsoft.com/en-us/surface/surface-book-2-update-history-8903a391-7a88-e6c1-5f61-5863795a7816

I wouldnt consider it unreasonable that that actually is the last firmware update they released. You usually dont update that very often, especially for older devices that are already stable and bugfree. And that page most likely doesnt contain beta versions released through Windows Insiders, hence me wondering if that might be the case here.

I am just finding it interesting that they push out some beta update right after EOL-ing the device.

FYI, this is my efibootmgr -v. Doesnt look too different from yours to me tbh.

Do you have secureboot enabled? Have you tried disabling it?

[Ramen-LadyHKG]

I cannot find that version number anywhere on internet as well. I'm not sure if insider preview is the cause that I've received these non-documented firmware.

That's why I'm asking Microsoft to document everything that they're going to release.

Here's a screenshot which includes all the driver updates that I've been received since 24th-Nov,2022

[Ramen-LadyHKG]

FYI, this is my efibootmgr -v. Doesnt look too different from yours to me tbh.

Interesting, then, I've no idea what caused the problem.

Do you have secureboot enabled? Have you tried disabling it?

Yes, I've disabled it.

[Ramen-LadyHKG]

FYI, this is my efibootmgr -v. Doesnt look too different from yours to me tbh.

I will have to remove comments related to that, preventing further misinformation.

But from what I remembered, before that firmware updates, efibootmgr -v didn't have that long lines of dp data

[Ramen-LadyHKG]

I'm curious, does non-insider Windows not receiving these beta firmware?

[Ramen-LadyHKG]

Can you give me your UEFI firmware version number?

I don't know how to rollback to 392.178.768.0, that's the version number before the 394.651.768.0 update

[StollD]

I'm curious, does non-insider Windows not receiving these beta firmware?

I havent used Windows in years, but my understanding is that if its not on the Microsoft Update Catalog, its not on (normal) Windows Update: https://www.catalog.update.microsoft.com/Home.aspx

Can you give me your UEFI firmware version number?

I installed the one from the latest MSI a couple of days ago, so 392.178.768.0.

I don't know how to rollback to 392.178.768.0, that's the version number before the 394.651.768.0 update

Well you already linked to the surface-uefi-firmware repackaging scripts, so follow the steps from its README.

[Ramen-LadyHKG]

I'm curious, does non-insider Windows not receiving these beta firmware?

I havent used Windows in years, but my understanding is that if its not on the Microsoft Update Catalog, its not on (normal) Windows Update: https://www.catalog.update.microsoft.com/Home.aspx

Can you give me your UEFI firmware version number?

I installed the one from the latest MSI a couple of days ago, so 392.178.768.0.

I don't know how to rollback to 392.178.768.0, that's the version number before the 394.651.768.0 update

Well you already linked to the surface-uefi-firmware repackaging scripts, so follow the steps from its README.

Thank god and thank you, looks like 394.651.768.0 is a mistaken by Microsoft, I guess?
394.651.768.0 is probably not made for Surface Book 2.

I just realized 392.178.768.0 is the version, I firstly installed Windows.

Hopefully, I'm able to downgrade it.

[Ramen-LadyHKG]

Sorry, do you mind helping me on this one as well

Book2 (15") - No CPU Turbo Boost (always below 2.0GHz)

Basically, My surface Book 2 does not turbo boost in Linux.

[qzed]

I literally cannot find that version number anywhere on the internet, not even in the Microsoft Update Catalog. Could this be an beta release published through Windows Insiders?

Potentially. I've got the same update and I'm running insiders, but hadn't noticed any issues. rEFInd still works fine, but after testing this just now, Grub gets stuck trying to boot any kernel (regardless of secureboot status).

Thank god and thank you, looks like 394.651.768.0 is a mistaken by Microsoft, I guess?
394.651.768.0 is probably not made for Surface Book 2.

That would have to be a major screw-up and I somehow doubt that. We'd probably see more and different failures. As in: I'd be surprised if it even booted something, because UEFI firmware (what the update seems to be) contains a lot of hardware-specific stuff. Including ACPI, which seems to be the correct one for the SB2.

I'm wondering whether this could be some kind of updated protection mechanism going wrong. Things failed similarly on the SPX and there it was an issue with memory allocated for the kernel not being marked as executable. As far as I remember though, the x86 code of Grub handled that part correctly.

Does the rEFInd failure happen before or after you select a kernel? And can you maybe try to update it and see if that helps (I'm wondering why I am not seeing the issue with rEFInd on my SB2)?

[Ramen-LadyHKG]

Thank you for all the reply.

[Ramen-LadyHKG]

Grub gets stuck trying to boot any kernel (regardless of secureboot status).

My Grub behaved exactly like you said. That's the first time I encounter something like this. Are you able to recover or fix the problem?

Because I'm about to write a support ticket to Microsoft/ Microsoft Surface

[qzed]

I tried updating grub, but that didn't work. So I'm booting via rEFInd right now, which still works for me.

[Ramen-LadyHKG]

That would have to be a major screw-up and I somehow doubt that. We'd probably see more and different failures. As in: I'd be surprised if it even booted something, because UEFI firmware (what the update seems to be) contains a lot of hardware-specific stuff. Including ACPI, which seems to be the correct one for the SB2.

Yeah. I thought, firmware/ UEFI only handles the startup process and load the Boot Manager (Grub/ Refind/systemd), the rest of OS startup are handle by them.

That shock me and I spent hours last 2 days, cannot boot back to my Linux

I'm wondering whether this could be some kind of updated protection mechanism going wrong. Things failed similarly on the SPX and there it was an issue with memory allocated for the kernel not being marked as executable. As far as I remember though, the x86 code of Grub handled that part correctly.

That night, the Windows Update automatically update the firmware and restart the PC, I didn't what exactly happened in-between because I went to sleep.

When I woke up, the SB2 stuck at the Surface Logo, I wait 10 minutes still nothing had happened, then I force restarted it.

Does the rEFInd failure happen before or after you select a kernel? And can you maybe try to update it and see if that helps (I'm wondering why I am not seeing the issue with rEFInd on my SB2)?

For Grub,
(Secure boot on/off) is still give me the boot entry menu. After selecting the kernel executable file, it just freeze in a black screen. Sometimes, left corner shown Starting Fedora 38..... something like that.

If the clicked the On-Screen-Keybaord when the menu is shown, at that period, the OSK is working.
After the kernel was selected, there was about 2 secs, the OSK still work and you can move the cursor as well.
After 2 secs, everything froze.

( Something went wrong in that 2 secs)

For rEFind,

It doesn't even give me the boot entries, just stuck at the Surface Logo

[Ramen-LadyHKG]

I will try and install systemd-boot and see if it works.

[Ramen-LadyHKG]

Normally, my rEFind boot process is like that:

1. Power on
2. Surface Logo Appears
3. Screen Turn black
4. A refresh animation 3times
5. Boot menu show

That's my refind configuration
refind.txt

I set Windows as default in refind because I was afraid Windows Update restart went into Linux.
I use firmware_bootnum 1 instead of the common way /efi/Microsoft/Boot/bootmgfw.efi

[Ramen-LadyHKG]

Normally, my rEFind boot process is like that:

1. Power on

2. Surface Logo Appears

3. Screen Turn black

4. A refresh animation 3times

5. Boot menu show

That's my refind configuration refind.txt

I set Windows as default in refind because I was afraid Windows Update restart went into Linux. I use firmware_bootnum 1 instead of the common way /efi/Microsoft/Boot/bootmgfw.efi

I want to address something else but it could be related:

I hope this could lead to something

There's strange behavior in my SB2 setup, I think SB2 does not have enough bandwidth that create some problem:
(like SD card disconnect, USBC port cannot register USB3.0 Speed (5Gbps )

I've mentioned in this post #1041 (2nd comment). I don't think that's linux kernel fault but mine or Mircosoft.

I bought a USBC Hub that has 11 ports (including a Smart Card slot)
[I don't really use that port but that hub was really cheap when I ordered it from Taiwan]

Sometimes, not always
rFEind boot manager cannot goes from step 3 to step 4. Not until I unplug it.
That's never happen when I set Windows as first boot item in UEFI.

Only happened in non-Microsoft boot (rEFind/Grub)

[Ramen-LadyHKG]

That's the product (I'm not promoting))

https://24h-pchome-com-tw.translate.goog/prod/DCADJ5-A900BQEHN?_x_tr_sl=auto&_x_tr_tl=zh-TW&_x_tr_hl=en&_x_tr_pto=wapp

[Ramen-LadyHKG]

When the Hub is connected to my SB2
There are tons of Error in Device Manager on Windows 11'

These errors fill up the logs that I couldn't track what happened that night (9th-June,2023) when the UEFI 394.651.768.0 was updated.

That USB error has also shown in my Linux dmesg but not as violent as that.

[Ramen-LadyHKG]

The reason why I want to address this issue right now , is because my rEFind stuck similarly at that time.
step 3 ~ step 4

I'm not an expert in System start up process.

I don't know what happened in Step3 ~ Step4. But there's a similar phenomenon in Grub.
Normally, When Surface UEFI start Grub, there's a 2 secs you cannot move menu cursor with keyboard arrowRIGHT?

What exactly happened in that 2secs?

[Ramen-LadyHKG]

[Ramen-LadyHKG]

Sometimes, not always
rFEind boot manager cannot goes from step 3 to step 4. Not until I unplug it.
That's never happen when I set Windows as first boot item in UEFI.

Only happened in non-Microsoft boot (rEFind/Grub)

never mind, my SB2 just stuck when I try to boot USB / enter UEFI page, it stuck at the surface logo when I click volume +/-

After I unplug the HUB, it continue to open the UEFI page.

All that I could say, Microsoft firmware is very buggy....
There are no logs we could trace, really annoying.

[Ramen-LadyHKG]

Sorry, I might have said too much nonsense here.

Under (Surface - Firmware - 394.651.768.0)
I just confirmed, systemd-boot works on my system.

• I create an additional EFI partition, and install systemd-boot there.

• I turned off secure boot

• I've not successfully set up secure boot yet, I've already signed EFI image (systemd-bootx64.efi) but it does not boot.

• Looks like shim-signed is still a must

[Ramen-LadyHKG]

Sorry, I might have said too much nonsense here.

Under (Surface - Firmware - 394.651.768.0) I just confirmed, systemd-boot works on my system.

* I create an additional EFI partition, and install systemd-boot there.

* I turned off secure boot

* I've not successfully set up secure boot yet, I've already signed EFI image (systemd-bootx64.efi) but it does not boot.

* Looks like shim-signed is still a must

Secure Boot on
I use sbctl sign with custom keys, successfully use systemd-boot

Now I am going to test shim-signed by Microsoft

[Ramen-LadyHKG]

I tried updating grub, but that didn't work. So I'm booting via rEFInd right now, which still works for me.

Okay, I figured out rEFInd is not the probelm.

I forgot I used shimx64.efi as an agent for Secure Boot.
Sorry, I was confusing you guys. :(

1. I turned off Secure Boot☑
2. I created an entry directly using refindx64.efi now everything works like it used to be.☑
3. Boot Arch Surface Kernel successfully ☑

However, rEFInd failed to boot my Fedora Surface kernel

there was an error message during boot

Starting initrd-switch-root.service
[14.552943] systemd[1]: Freezing excution
[!!!!!!] Failed to mount API filesystems

But I think that's no big deal. Probably, because of the old surface kernel 6.2.14-1.surface

Last night I updated everything for Fedora in LiveE except surface
dnf update --refresh --exclude *surface*
Because I wanted to wait and skip v6.3.6 (Fedora) due to the new changes.

[Ramen-LadyHKG]

[Update "6" - Surface Business Support meeting ] 17th-July, 2023

1. The issue is discovered --- ✅
2. The issue is confirmed exist on other users --- ✅
3. The issue is reported to Microsoft --- ✅
4. Temporary solution is found --- ✅
5. The issue is confirmed by Microsoft --- 🟩
6. The issue is fixed by Microsoft --- 🟩

Here's some update, I received Microsoft reply to my support ticket.
We're going to have a MS Meet and discuss this problem, I will talk about Surface Pro series as well not just Book

If you have anything related need to tell them or you want to join the meeting.
Or any information that helps us resolve this issue.
Please do let me know

[Ramen-LadyHKG]

BTW, The meeting is going to hold on Friday around 1pm SGT.

[SudoSnoop]

Good thing. Wish you the best of luck, that you're able to tell them what's going on and hope that this issue can be resolved. It's probably a good idea to join the meeting via your smartphone, so that you could show them the boot issues via a live camera feed from it.

[renehub]

I just put ubuntu (22 lts) on a surface 5 pro and all was fine after basic installation. After enabling surface-linux with extrepo and doing the updates and try to run the surface kernel I have the problem described here and now are only able to boot arch-linux.

[longtry]

I got a similar problem in #1194 , but on Mint. Now I've managed to boot, but Linux can't mount the C:\ and D:\ drives any more (it only sees them as "100 GB" and "200 GB"). When I try selecting them, Mint prompts that I need to type a "passphrase". I don't recall ever set a phrase for either drive (BitLocker is off, btw), so I just type the Mint login password, but it says it fails to mount. Does anyone have the same experience?

[Ramen-LadyHKG]

I got a similar problem in #1194 , but on Mint. Now I've managed to boot, but Linux can't mount the C:\ and D:\ drives any more (it only sees them as "100 GB" and "200 GB"). When I try selecting them, Mint prompts that I need to type a "passphrase". I don't recall ever set a phrase for either drive (BitLocker is off, btw), so I just type the Mint login password, but it says it fails to mount. Does anyone have the same experience?

What does blkid tell? It's it a NTFS partition?

Just a reminder, you need root permission to mount partition.

[Ramen-LadyHKG]

I got a similar problem in #1194 , but on Mint. Now I've managed to boot, but Linux can't mount the C:\ and D:\ drives any more (it only sees them as "100 GB" and "200 GB"). When I try selecting them, Mint prompts that I need to type a "passphrase". I don't recall ever set a phrase for either drive (BitLocker is off, btw), so I just type the Mint login password, but it says it fails to mount. Does anyone have the same experience?

Btw
I don't think your #1194 issue is related to ours.

What's your UEFI firmware version number? You can check it in UEFI/BIOS settings (power button + vol up)

[qzed]

Yeah, #1194 is different, looks like it was some secureboot/mok problem.

[Ramen-LadyHKG]

Due to personal reason, the date of my meeting with Microsoft is change.

Would anyone like to join, add some information about the UEFI firmware problem, and linux not bootable?

[Ramen-LadyHKG]

A similar thing has happened to me on a Surface Pro 5, windows updated and it is now running the same UEFI version as @Ramen-LadyHKG 's Pro 5. It will boot to grub and show the menu, but once it tries to start fedora on any of the installed kernels it gets stuck at a cursor which does not blink. Booting from a fedora live usb or a ventoy fedora usb does exactly the same thing.

I can confirm that on my SP6 i5. The firmware was installed some days ago via the windows updater. Since then my Fedora on USB SSD does not work anymore. It just always freezes the Surface when I try to boot it. Even tried reinstalling on the USB SSD via Hyper-V. That doesn't work either. :(

Hello, @ThePinkUnicorn6 @SudoSnoop . Were you on Insider Preview when you received the update from Windows Update?

I'm writing a report to Microsoft, I want to record the timeline when or how it happens.

Much obliged!

[ThePinkUnicorn6]

Hi @Ramen-LadyHKG,
Thank you for your work in getting this brought up with Microsoft, I am very grateful for the effort you have put in.

AFAIK, I am not on the insider preview but I am not sure. Here is what winver tells me:

And here is what the windows insider webpage shows me:

But looking at the insider settings seems to imply that I am part of windows insiders but I will stop being part at the next major release of windows:

Kind regards, TPU

[longtry]

What's your UEFI firmware version number? You can check it in UEFI/BIOS settings (power button + vol up)

It's 394.651.768.0 according to my Win update info.

BTW, it turned out my C & D:\ drives are encrypted without BitLocker turning on (thus no passphrases). I had to manually switch it on in order to set up the phrases, then put it in Linux for mounting. It seems that the firmware update messed with the whole mentioned encrypt business, in addition to the problems you listed on here, because after the update, some warning signs suddenly appeared on my drives in Win Explorer.

[Ramen-LadyHKG]

What's your UEFI firmware version number? You can check it in UEFI/BIOS settings (power button + vol up)

It's 394.651.768.0 according to my Win update info.

BTW, it turned out my C & D:\ drives are encrypted without BitLocker turning on (thus no passphrases). I had to manually switch it on in order to set up the phrases, then put it in Linux for mounting. It seems that the firmware update messed with the whole mentioned encrypt business, in addition to the problems you listed on here, because after the update, some warning signs suddenly appeared on my drives in Win Explorer.

Post some screenshots here.

1. Bitlocker setting page in Windows
2. run sudo blkid in Linux terminal
3. the warning sign you're referring to

Issue A)
I'm not sure that you've mount Windows partition before.
In order to mount an encrypted system drive (specifically C drive), you need to use recovery key.

Issue B)
The 394.651.768.0firmware, AFAIK, it only affects booting.

Since you're using mint, you're most likely unable to boot before downgrade back to 392, unless you're NOT using Grub or the Grub isn't Redhat/Canonical issued.

[Ramen-LadyHKG]

Hi @Ramen-LadyHKG,
Thank you for your work in getting this brought up with Microsoft, I am very grateful for the effort you have put in.

AFAIK, I am not on the insider preview but I am not sure. Here is what winver tells me:

And here is what the windows insider webpage shows me:

But looking at the insider settings seems to imply that I am part of windows insiders but I will stop being part at the next major release of windows:

Kind regards, TPU

Excellent, then, I'll mark your case as the first incident of non-Insider(Surface Pro5) .

[Ramen-LadyHKG]

I'll will share a PDF file here before i send to Microsoft

[SudoSnoop]

A similar thing has happened to me on a Surface Pro 5, windows updated and it is now running the same UEFI version as @Ramen-LadyHKG 's Pro 5. It will boot to grub and show the menu, but once it tries to start fedora on any of the installed kernels it gets stuck at a cursor which does not blink. Booting from a fedora live usb or a ventoy fedora usb does exactly the same thing.

I can confirm that on my SP6 i5. The firmware was installed some days ago via the windows updater. Since then my Fedora on USB SSD does not work anymore. It just always freezes the Surface when I try to boot it. Even tried reinstalling on the USB SSD via Hyper-V. That doesn't work either. :(

Hello, @ThePinkUnicorn6 @SudoSnoop . Were you on Insider Preview when you received the update from Windows Update?

I'm writing a report to Microsoft, I want to record the timeline when or how it happens.

Much obliged!

Hi there. No, I'm not on the insider programme. The only thing I checked on my update settings is that I would receive regular updates as fast as possible.
Thanks for your effort.

[longtry]

Post some screenshots here.

1. Bitlocker setting page in Windows

2. run `sudo blkid` in Linux terminal

3. the warning sign you're referring to

Issue A) I'm not sure that you've mount Windows partition before. In order to mount an encrypted system drive (specifically C drive), you need to use recovery key.

Issue B) The 394.651.768.0firmware, AFAIK, it only affects booting.

Since you're using mint, you're most likely unable to boot before downgrade back to 392, unless you're NOT using Grub or the Grub isn't Redhat/Canonical issued.

OK, here goes.

1. The Bitlocker page is now all lovey-dovey, because I enabled it some days ago. Before that, it was off.
2. blkid here.
3. Exclamation-in-triangle here (not a true scrshot tho, because as said in 1., I enabled it and they're no longer in Win Explorer.

A. Yes, my Mint mounted C & D:\ all the time, and I read & saved files on them w/o probs. It was after the auto-update of Linux Surface on Mint that I encountered the dead-Win screen at bootup, and after the measures taken to get pass that state (described in my #1194 post) that Mint began to fail to mount the drives.

B. idk what grub is for sure, I just installed Mint 2 months ago using mostly the default setup guide. Yet I know for sure that I'm having 394 running right now, and I can boot into LS Mint also quite easily - after that scare a week ago, that is.

[J3RN]

There was some curiosity about whether Debian's patched GRUB would boot. It does, or at least the installer gets past GRUB (I didn't install). I had Secure Boot disabled when I tried Debian (which was required for Arch to even get to GRUB), and I have not tried the Debian installer with Secure Boot enabled.

FWIW, openSUSE MicroOs also appears to get past GRUB, even with Secure Boot enabled. I was originally going to install Fedora Silverblue, but this issue has been such a pain that I might just install MicroOs as "the next best thing" and shake my fists at Red Hat.

Speaking of Red Hat, has this issue been raised to the Fedora team? What would be the proper way to go about doing that?

[Ramen-LadyHKG]

[Update "7" - 1st Surface Business Support meeting ] 27th-July, 2023

1. The issue is discovered --- ✅
2. The issue is confirmed exist on other users --- ✅
3. The issue is reported to Microsoft --- ✅
4. Temporary solution is found --- ✅
5. The issue is confirmed by Microsoft --- 🟩
6. The issue is fixed by Microsoft --- 🟩

I've just finished the meeting with Microsoft.
The experience was great.

As for now, they request some documentations referring to this issue. (like a video footage of the whole process, code or anything that shows what's working and what's not.

They also want to know, what devices are having the problem.
AFAIK, Surface Book2, Pro 5,6 has this problem

Normal:

SB2 - 392.178.768.0
Pro5/6 - 239.645.768.0

Abnormal:

SB2 - 394.651.768.0
Pro 5/6 - 238.167.768.0

They'll send me a link to upload all documents that's related to this problem.
In possible future, we'll have another online meeting.

He said that they are welcome other users to join and tell what problems they're having even if it isn't related to this specific UEFI problem.

[Ramen-LadyHKG]

Surface – Linux not booting after UEFI firmware updates | issue

[Update "8" - 1st Surface Business Support meeting ] 7th-Aug, 2023

1. The issue is discovered --- ✅
2. The issue is confirmed exist on other users --- ✅
3. The issue is reported to Microsoft --- ✅
4. Temporary solution is found --- ✅
5. The issue is confirmed by Microsoft --- 🟩
6. The issue is fixed by Microsoft --- 🟩

I think I've complete the requested document.

Here's the temporary view link:
Surface – Linux not booting after UEFI firmware update | issue - Google Docs

Also this link:
Chat Linux-Surface Kernel Developer - Google Docs

this is another document including groups of screenshots captured from Linux-Surface Support Channel - Element(Matrix) , It's the discuss of the root cause.

However,

This document is not yet finished. I'm still writing some summaries of each group of screenshots.
Otherwise, Microsoft Support will be confused.

Also I'll remove screenshots that included group members personal / privacy information.
I'm planning on to remove the part of AMD discussion on 5th-Aug,2023

Once the document is reviewed by you guys, I'll upload it to Microsoft Support tomorrow(7-Aug,2023).

If you've have anything wants to add or adjust, please inform me. Thank you

[Ramen-LadyHKG]

Surface – Linux not booting after UEFI firmware update | issue - Google Docs

Please take some time in

Page 6 – List of Surface Model and UEFI version

Page 7~8 – Known working/not working OS

If yours is different from the document, please contact me

[Ramen-LadyHKG]

There was some curiosity about whether Debian's patched GRUB would boot. It does, or at least the installer gets past GRUB (I didn't install). I had Secure Boot disabled when I tried Debian (which was required for Arch to even get to GRUB), and I have not tried the Debian installer with Secure Boot enabled.

FWIW, openSUSE MicroOs also appears to get past GRUB, even with Secure Boot enabled. I was originally going to install Fedora Silverblue, but this issue has been such a pain that I might just install MicroOs as "the next best thing" and shake my fists at Red Hat.

Thank you for your informative test.
I'll consider to add it into my document

Speaking of Red Hat, has this issue been raised to the Fedora team? What would be the proper way to go about doing that?
I have not reported to Red Hat / bugzilla of linux kernel yet.
because I'm not familiar with the platform.

If you're available, please help us to open a ticket to RedHat & linux kernel

[Ramen-LadyHKG]

Post some screenshots here.

1. Bitlocker setting page in Windows

2. run `sudo blkid` in Linux terminal

3. the warning sign you're referring to

Issue A) I'm not sure that you've mount Windows partition before. In order to mount an encrypted system drive (specifically C drive), you need to use recovery key.
Issue B) The 394.651.768.0firmware, AFAIK, it only affects booting.
Since you're using mint, you're most likely unable to boot before downgrade back to 392, unless you're NOT using Grub or the Grub isn't Redhat/Canonical issued.

OK, here goes.

1. The Bitlocker page is now all lovey-dovey, because I enabled it some days ago. Before that, it was off.

2. [blkid](https://imgur.com/a/OqdfO4o) here.

@longtry Your partition table is a bit confusing to me
I don't understand why there are 2 bitlocker partitions and 1 ntfs partition.

My guess is your Windows partition is not encrypted but you have 2 encrypted partition to store data?

Please explain what partitions 345 are? or show Disk Management in Windows

3. [Exclamation-in-triangle](https://imgur.com/a/QizWZ2a) here (not a true scrshot tho, because as said in 1., I enabled it and they're no longer in Win Explorer.

A. Yes, my Mint mounted C & D:\ all the time, and I read & saved files on them w/o probs. It was after the auto-update of Linux Surface on Mint that I encountered the dead-Win screen at bootup, and after the measures taken to get pass that state (described in my #1194 post) that Mint began to fail to mount the drives.

B. idk what grub is for sure, I just installed Mint 2 months ago using mostly the default setup guide. Yet I know for sure that I'm having 394 running right now, and I can boot into LS Mint also quite easily - after that scare a week ago, that is.

I'm not sure what the Windows C: drive Screenshot means

On Linux Mint,
Can you try mount the partition in the terminal with verbose?
for example,
sudo mount -v /dev/nvme0n1p4
sudo cryptsetup bitlkOpen /dev/nvme0n1p3 WinM
sudo mount -v /dev/mapper/WinM /mnt

[Ramen-LadyHKG]

Post some screenshots here.

1. Bitlocker setting page in Windows

2. run `sudo blkid` in Linux terminal

3. the warning sign you're referring to

Issue A) I'm not sure that you've mount Windows partition before. In order to mount an encrypted system drive (specifically C drive), you need to use recovery key.
Issue B) The 394.651.768.0firmware, AFAIK, it only affects booting.
Since you're using mint, you're most likely unable to boot before downgrade back to 392, unless you're NOT using Grub or the Grub isn't Redhat/Canonical issued.

OK, here goes.

1. The Bitlocker page is now all lovey-dovey, because I enabled it some days ago. Before that, it was off.

2. [blkid](https://imgur.com/a/OqdfO4o) here.

@longtry Your partition table is a bit confusing to me I don't understand why there are 2 bitlocker partitions and 1 ntfs partition.

My guess is your Windows partition is not encrypted but you have 2 encrypted partition to store data?

Please explain what partitions 345 are? or show Disk Management in Windows

3. [Exclamation-in-triangle](https://imgur.com/a/QizWZ2a) here (not a true scrshot tho, because as said in 1., I enabled it and they're no longer in Win Explorer.

A. Yes, my Mint mounted C & D:\ all the time, and I read & saved files on them w/o probs. It was after the auto-update of Linux Surface on Mint that I encountered the dead-Win screen at bootup, and after the measures taken to get pass that state (described in my #1194 post) that Mint began to fail to mount the drives.
B. idk what grub is for sure, I just installed Mint 2 months ago using mostly the default setup guide. Yet I know for sure that I'm having 394 running right now, and I can boot into LS Mint also quite easily - after that scare a week ago, that is.

I'm not sure what the Windows C: drive Screenshot means

On Linux Mint, Can you try mount the partition in the terminal with verbose? for example, sudo mount -v /dev/nvme0n1p4 sudo cryptsetup bitlkOpen /dev/nvme0n1p3 WinM sudo mount -v /dev/mapper/WinM /mnt

@longtry Anyway, let's continue in your post #1194.
Since they're not related.

[Ramen-LadyHKG]

For latest information, please visit Support (linux-surface)

[Ramen-LadyHKG]

Surface – Linux not booting after UEFI firmware updates | issue

For latest information, please visit Support Channel (linux-surface) on Matrix

[Update "9" - 2nd Surface Business Support meeting ] 12th-Aug, 2023

1. The issue is discovered --- ✅
2. The issue is confirmed exist on other users --- ✅
3. The issue is reported to Microsoft --- ✅
4. Temporary solution is found --- ✅
5. The issue is aware confirmed by Microsoft --- ✅
6. The issue is fixed by Microsoft --- ❎

7. The issue is reported to Red Hat --- 🟩
8. The issue is solved -- 🟩

---

Summary:

Microsoft Support can only solve Windows issue on Surface Devices.

1. I forgot to ask them whether the issue will spread to Newer Surface Devices.
2. Regarding to the assisting linux-surface kernel development

The Surface technical team didn't have any words about that this time. However, they'll look into them as Curie have requested. They'll notify Curie if they've any updates regarding to that.

3. Are they going to stop/ postpone/warn about the firmware update to user?

They don't have control over the firmware update pushes, end users have the control over which updates they're going to get.

We advice user who has issue with non-Windows Operating System, they are better off reach out The developer of the non-Windows Operating System

What can we do now?

Send a issue ticket to Red Hat.

---

I wouldn't say I'm disappointed, It's all expected and understandable.

[J3RN]

Very difficult to find (I know, I just tried), but there is an open bug on Redhat's bugzilla for (what I suspect) is this issue: https://bugzilla.redhat.com/show_bug.cgi?id=2149020

[Ramen-LadyHKG]

Very difficult to find (I know, I just tried), but there is an open bug on Redhat's bugzilla for (what I suspect) is this issue: https://bugzilla.redhat.com/show_bug.cgi?id=2149020

Thank you, That's a great help.
I'm planning to rewrite the post on Reddit and Lemmy. I'll put this link on it.

Hey, are you on Support Channel (linux-surface) on Matrix
We can discuss this over there.

[J3RN]

I'll join! I'm working on trying to downgrade the UEFI presently.

[Ramen-LadyHKG]

I'll join! I'm working on trying to downgrade the UEFI presently.

read my file about downgrade
Surface – Linux not booting after UEFI firmware update issue.pdf