CRITICAL Wrong entry in loader.conf prevents booting #976
Comments
|
@thac0 This seems a bit odd to me. Any ideas why the kernel wouldn't load the initrd? @FrankNL1984 Could tell me what the shasum of the EFI\org.clearlinux\freestanding-00-intel-ucode.cpio file is? |
|
Using patched Intel DH67BL Intel Desktop board along with an Intel i7 2600K CPU.
This is weird, I do not know if this is happened due to an old EFI
specification, on my laptop it boots well with 3 initrd.
Can you verify the hash from the cpio file.
$ sudo sha256sum /boot/EFI/org.clearlinux/freestanding-00-intel-ucode.cpio /usr/lib/initrd.d/00-intel-ucode.cpio
bb8f0512802b82fd3249c28be4a0e1b74f401541cd94e67d918da946fb00ac5f /boot/EFI/org.clearlinux/freestanding-00-intel-ucode.cpio
bb8f0512802b82fd3249c28be4a0e1b74f401541cd94e67d918da946fb00ac5f /usr/lib/initrd.d/00-intel-ucode.cpio
|
|
Hi there,
Thanks again for taking this seriously.
The shasum:
3b6ca55d2691f4b312961be26b46770a4f6622c7 (BOOT_PARTITION/EFI/org.clearlinux/) freestanding-00-intel-ucode.cpio
Kind regards
From: William Douglas <notifications@github.com>
Sent: Wednesday, July 3, 2019 11:18 PM
To: clearlinux/distribution <distribution@noreply.github.com>
Cc: Voorde ter, Frank <f.ter.voorde@tervoorde.net>; Mention <mention@noreply.github.com>
Subject: Re: [clearlinux/distribution] CRITICAL Wrong entry in loader.conf prevents booting (#976)
@thac0<https://github.com/thac0> This seems a bit odd to me. Any ideas why the kernel wouldn't load the initrd?
@FrankNL1984<https://github.com/FrankNL1984> Could tell me what the shasum of the EFI\org.clearlinux\freestanding-00-intel-ucode.cpio file is?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub<#976?email_source=notifications&email_token=AMQTWGVDV76QNK5RXYDDAYLP5UJOZA5CNFSM4H5JFQVKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZFWPLI#issuecomment-508258221>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AMQTWGU2EYXUGEYLOR7XAXTP5UJOZANCNFSM4H5JFQVA>.
|
|
The shasum:
3b6ca55d2691f4b312961be26b46770a4f6622c7 (BOOT_PARTITION/EFI/org.clearlinux/) freestanding-00-intel-ucode.cpio
$ shasum /usr/lib/initrd.d/00-intel-ucode.cpio
3b6ca55d2691f4b312961be26b46770a4f6622c7 /usr/lib/initrd.d/00-intel-ucode.cpio
Not a corrupted file.
|
|
This is really ugly and I am noticing that we don't have a nice way to mask those initrd files currently. I can at least start working on a way so you can update without having problems booting after while we try and figure out what is going on with loading the initrd. |
|
To be complete, I would like to inform both files have the same hash.
BOOT_PARTITION/EFI/org.clearlinux/
3b6ca55d2691f4b312961be26b46770a4f6622c7 freestanding-00-intel-ucode.cpio
ROOT_PARTITION/usr/lib/initrd.d/
3b6ca55d2691f4b312961be26b46770a4f6622c7 00-intel-ucode.cpio
From: Miguel Bernal Marin <notifications@github.com>
Sent: Wednesday, July 3, 2019 11:30 PM
To: clearlinux/distribution <distribution@noreply.github.com>
Cc: Voorde ter, Frank <f.ter.voorde@tervoorde.net>; Mention <mention@noreply.github.com>
Subject: Re: [clearlinux/distribution] CRITICAL Wrong entry in loader.conf prevents booting (#976)
The shasum:
3b6ca55d2691f4b312961be26b46770a4f6622c7 (BOOT_PARTITION/EFI/org.clearlinux/) freestanding-00-intel-ucode.cpio
$ shasum /usr/lib/initrd.d/00-intel-ucode.cpio
3b6ca55d2691f4b312961be26b46770a4f6622c7 /usr/lib/initrd.d/00-intel-ucode.cpio
Not a corrupted file.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub<#976?email_source=notifications&email_token=AMQTWGWZZYR3JUVEO4PDKMLP5UK7HA5CNFSM4H5JFQVKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZFXLYQ#issuecomment-508261858>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AMQTWGUL37QR2OSFV76JWZLP5UK7HANCNFSM4H5JFQVA>.
|
|
@bryteise also we need an stateless for initrd as you do with the firmware ( |
|
@miguelinux We do have stateless for some initrd files via /etc/kernel/initrd-* but not for the freestanding so I need to add that to clr-boot-manager. |
|
Hi there,
That is very nice, but I'm fine just disabling autoupdate and kindly awaiting the progress on this issue.
If I can help you with something (providing information or trying things on bare metal), I would love to.
FIY: the error text appears in the font of the motherboard's POST screen. The the system hangs and doesn't even respond to the power switch, num lock lights or CTRL-ALT-DEL.
Kind regards from The Netherlands
From: William Douglas <notifications@github.com>
Sent: Wednesday, July 3, 2019 11:32 PM
To: clearlinux/distribution <distribution@noreply.github.com>
Cc: Voorde ter, Frank <f.ter.voorde@tervoorde.net>; Mention <mention@noreply.github.com>
Subject: Re: [clearlinux/distribution] CRITICAL Wrong entry in loader.conf prevents booting (#976)
This is really ugly and I am noticing that we don't have a nice way to mask those initrd files currently. I can at least start working on a way so you can update without having problems booting after while we try and figure out what is going on with loading the initrd.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub<#976?email_source=notifications&email_token=AMQTWGQQ3L4GKCREO7XWSFDP5ULGRA5CNFSM4H5JFQVKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZFXQRA#issuecomment-508262468>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AMQTWGSK3M4N6LXYDGF75Y3P5ULGRANCNFSM4H5JFQVA>.
|
|
So here's the kernel code where the error message is being seen: I've not spent much time in this corner of the kernel, but from the code snippet, the kernel will attempt, through low-level EFI calls, to instantiate the ramdisk. If it fails, it will attempt to do it again, but load it at a different (higher) target address. For this case, EFI appears to be reporting failure for both attempts. The code is venerable (more than 5 years old) at this point, and well-exercised by other Linux distributions, so my hypothesis is that @FrankNL1984 's problem may be local to his platform. @FrankNL1984 - Have the BIOS settings for your platform been changed from their factory defaults? If so, what changes were made? |
|
Hi there,
My BIOS is patched with the latest upgrade and is in factory default.
Met vriendelijke groet,Frank ter Voorde
…________________________________
From: Joe Konno <notifications@github.com>
Sent: Monday, July 8, 2019 5:57:29 PM
To: clearlinux/distribution
Cc: Voorde ter, Frank; Mention
Subject: Re: [clearlinux/distribution] CRITICAL Wrong entry in loader.conf prevents booting (#976)
So here's the kernel code where the error message is being seen:
https://github.com/torvalds/linux/blob/e93c9c99a629c61837d5a7fc2120cd2b6c70dbdd/arch/x86/boot/compressed/eboot.c#L469
I've not spent much time in this corner of the kernel, but from the code snippet, the kernel will attempt, through low-level EFI calls, to instantiate the ramdisk. If it fails, it will attempt to do it again, but load it at a different (higher) target address. For this case, EFI appears to be reporting failure for both attempts. The code is venerable (more than 5 years old) at this point, and well-exercised by other Linux distributions, so my hypothesis is that @FrankNL1984<https://github.com/FrankNL1984> 's problem may be local to his platform.
@FrankNL1984<https://github.com/FrankNL1984> - Have the BIOS settings for your platform been changed from their factory defaults? If so, what changes were made?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub<#976?email_source=notifications&email_token=AMQTWGVWVWXUTJIH4F2G24LP6NPWTA5CNFSM4H5JFQVKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZNRIFI#issuecomment-509285397>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AMQTWGSQNJSLF6EZXZ3QRGTP6NPWTANCNFSM4H5JFQVA>.
|
|
@FrankNL1984 - Another question-- your responses are very helpful, by the way! When you omit the I ask because the clr_power binary, executed shortly after rootfs is mounted, triggers a microcode reload event. In that flow, microcode is loaded directly from the rootfs, not through an initrd/ramdisk. Thanks so much for your help here. It's appreciated! |
|
Hi there,
Thanks for the compliments. I will keep helping.
You wrote: "When you omit the initrd= argument from your command line (...)"
I would like to respond to that by saying that I have no choice but to do that. If I don't omit that, the system will not boot.
root@lavendel/home/operator # dmesg | grep -i microcode
[ 0.199441] calling save_microcode_in_initrd+0x0/0x54 @ 1
[ 0.199441] initcall save_microcode_in_initrd+0x0/0x54 returned 0 after 0 usecs
[ 0.317568] calling microcode_init+0x0/0x1f7 @ 1
[ 0.317604] microcode: sig=0x206a7, pf=0x2, revision=0x2e
[ 0.317744] microcode: Microcode Update Driver: v2.2.
[ 0.317745] initcall microcode_init+0x0/0x1f7 returned 0 after 172 usecs
[ 10.879239] microcode: updated to revision 0x2f, date = 2019-02-17
[ 10.880939] x86/CPU: CPU features have changed after loading microcode, but might not take effect.
[ 10.880943] microcode: Reload completed, microcode revision: 0x2f
Kind regards,
Frank ter Voorde
From: Joe Konno <notifications@github.com>
Sent: Monday, July 8, 2019 9:57 PM
To: clearlinux/distribution <distribution@noreply.github.com>
Cc: Voorde ter, Frank <f.ter.voorde@tervoorde.net>; Mention <mention@noreply.github.com>
Subject: Re: [clearlinux/distribution] CRITICAL Wrong entry in loader.conf prevents booting (#976)
@FrankNL1984<https://github.com/FrankNL1984> - Another question-- your responses are very helpful, by the way!
When you omit the initrd= argument from your command line, and successfully boot the platform, do you see any dmesg output about microcode failing to (re)load? Here's a one-liner for you:
# sudo dmesg | grep -i microcode
I ask because the clr_power<https://github.com/clearlinux/clr-power-tweaks/blob/51b1224cac34dc300b25fe6613d4480866acef21/clr-power-tweaks.conf#L93> binary, executed shortly after rootfs is mounted, triggers a microcode reload event. In that flow, microcode is loaded directly from the rootfs, not through an initrd/ramdisk.
Thanks so much for your help here. It's appreciated!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub<#976?email_source=notifications&email_token=AMQTWGWHY535SCQKQKUWWGTP6OLZNA5CNFSM4H5JFQVKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZOFUWY#issuecomment-509368923>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AMQTWGTBSQ3J22LRVXTOI4TP6OLZNANCNFSM4H5JFQVA>.
|
|
All right. So, microcode can be patched, just not in the e(arly)boot path. Next step is to rule out Clear-specific optimizations to the kernel. So, @FrankNL1984 , if you could install the following and attempt to boot mainline-vanilla with and without the ucode $ sudo swupd bundle-add kernel-mainline-vanilla I do not have your platform or I would do this myself, hence all of these requests. It's worth re-iterating that your current work-around-- removing the ucode initrd from your kernel cmdline-- is the best path forward until we find a proper solution. |
|
Ah. Thanks @bryteise . I'll update my earlier comment. |
|
Hi there, I've encountered a similar problem like this one. As requested by Douglas William, I hereby posted my findings below.
B) Base Board Information C) CPU Information Hope this could lead to something. Best Regards |
|
Thanks @FURIOUS-7 . At first blush, looks like the same issue. Would you mind following the steps I put down earlier?
|
|
Well... we're stuck without feedback from the reporters. I do not have a platform that can reproduce this issue, so I've no way to move this forward. |
|
I am currently on holiday and will get back within a couple of days.
Met vriendelijke groet,Frank ter Voorde
…________________________________
From: Joe Konno <notifications@github.com>
Sent: Monday, July 22, 2019 5:15:37 PM
To: clearlinux/distribution <distribution@noreply.github.com>
Cc: Voorde ter, Frank <f.ter.voorde@tervoorde.net>; Mention <mention@noreply.github.com>
Subject: Re: [clearlinux/distribution] CRITICAL Wrong entry in loader.conf prevents booting (#976)
Well... we're stuck without feedback from the reporters. I do not have a platform that can reproduce this issue, so I've no way to move this forward.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub<#976?email_source=notifications&email_token=AMQTWGSIYNPTOCLRFWEWPS3QAXFJTA5CNFSM4H5JFQVKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2QIEFA#issuecomment-513835540>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AMQTWGTC2ZTOGMMAG3VKSKDQAXFJTANCNFSM4H5JFQVA>.
|
|
I am having this same issue after a fresh install from USB. The USB itself boots fine, but after the install finishes I get the issue with the ucode I installed the When I boot the mainline vanilla kernel without the ucode initrd, I get this: root@clearlinuxbox~ # dmesg | grep -i microcode
[ 0.248158] calling save_microcode_in_initrd+0x0/0x4d @ 1
[ 0.248162] initcall save_microcode_in_initrd+0x0/0x4d returned 0 after 0 usecs
[ 0.974948] calling microcode_init+0x0/0x20e @ 1
[ 0.974972] microcode: sig=0x206a7, pf=0x10, revision=0x23
[ 0.975006] microcode: Microcode Update Driver: v2.2.
[ 0.975008] initcall microcode_init+0x0/0x20e returned 0 after 57 usecs
[ 22.159616] microcode: updated to revision 0x2f, date = 2019-02-17
[ 22.160178] core: PEBS enabled due to microcode update
[ 22.160186] x86/CPU: CPU features have changed after loading microcode, but might not take effect.
[ 22.160188] microcode: Reload completed, microcode revision: 0x2f...and when I boot the LTS kernel I get this: root@clearlinuxbox~ # dmesg | grep -i microcode
[ 0.159719] calling save_microcode_in_initrd+0x0/0x54 @ 1
[ 0.159723] initcall save_microcode_in_initrd+0x0/0x54 returned 0 after 0 usecs
[ 1.098373] calling microcode_init+0x0/0x217 @ 1
[ 1.098399] microcode: sig=0x206a7, pf=0x10, revision=0x23
[ 1.098431] microcode: Microcode Update Driver: v2.2.
[ 1.098433] initcall microcode_init+0x0/0x217 returned 0 after 56 usecs
[ 22.154758] microcode: updated to revision 0x2f, date = 2019-02-17
[ 22.155394] core: PEBS enabled due to microcode update
[ 22.155407] x86/CPU: CPU features have changed after loading microcode, but might not take effect.Sorry to just jump into the convo, but I was hoping I could help. |
|
@logavanc Thanks for the info. Is it possible to share your hardware configuration, as @FURIOUS-7 and @FrankNL1984 have done? |
|
@thac0 I am using a Dell Latitude E6420 with BIOS version A08 (in the process of trying to upgrade the BIOS to A25 to see if it helps). root@clearlinuxbox~ # lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
Address sizes: 36 bits physical, 48 bits virtual
CPU(s): 4
On-line CPU(s) list: 0-3
Thread(s) per core: 2
Core(s) per socket: 2
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 42
Model name: Intel(R) Core(TM) i5-2520M CPU @ 2.50GHz
Stepping: 7
CPU MHz: 1293.530
CPU max MHz: 3200.0000
CPU min MHz: 800.0000
BogoMIPS: 4988.55
Virtualization: VT-x
L1d cache: 32K
L1i cache: 32K
L2 cache: 256K
L3 cache: 3072K
NUMA node0 CPU(s): 0-3
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm epb pti tpr_shadow vnmi flexpriority ept vpid xsaveopt dtherm ida arat pln ptsroot@clearlinuxbox~ # dmidecode -t 2
# dmidecode 3.2
Getting SMBIOS data from sysfs.
SMBIOS 2.6 present.
Handle 0x0002, DMI type 2, 15 bytes
Base Board Information
Manufacturer: Dell Inc.
Product Name: 0K0DNP
Version: A02
Serial Number: /G8WGFS1/CN129611CP0992/
Asset Tag: Not Specified
Features:
Board is a hosting board
Board is replaceable
Location In Chassis: Not Specified
Chassis Handle: 0x0003
Type: Motherboard
Contained Object Handles: 0root@clearlinuxbox~ # dmidecode -t 0
# dmidecode 3.2
Getting SMBIOS data from sysfs.
SMBIOS 2.6 present.
Handle 0x0000, DMI type 0, 24 bytes
BIOS Information
Vendor: Dell Inc.
Version: A08
Release Date: 10/18/2011
Address: 0xF0000
Runtime Size: 64 kB
ROM Size: 2048 kB
Characteristics:
PCI is supported
BIOS is upgradeable
BIOS shadowing is allowed
Boot from CD is supported
Selectable boot is supported
EDD is supported
5.25"/1.2 MB floppy services are supported (int 13h)
3.5"/720 kB floppy services are supported (int 13h)
3.5"/2.88 MB floppy services are supported (int 13h)
Print screen service is supported (int 5h)
8042 keyboard services are supported (int 9h)
Serial services are supported (int 14h)
Printer services are supported (int 17h)
ACPI is supported
BIOS boot specification is supported
Targeted content distribution is supported
BIOS Revision: 4.6Hope this is the right info! |
|
Very helpful. Thanks @logavanc ! |
|
@thac0 the BIOS update from A08 to A25 (Dell did not make that easy...) did not help. Let me know if there is anything else that I can do to help. |
|
@logavanc, @FURIOUS-7, @FrankNL1984 What is the output of |
|
@miguelinux yeah, this is what I get (grabbed all LoaderFirmware* vars for fun and added a few missing newlines): root@clearlinuxbox~ # cat /sys/firmware/efi/efivars/LoaderFirmware*
American Megatrends 4.632
UEFI 2.00 |
|
Looks like this issue is expected ;-( From torvalds/linux@47226ad commit body:
and
Unfortunately |
|
@miguelinux - Time to test that theory by building a (much) smaller kernel, such that |
|
Hello, Thanks, |
|
@DrGo - The standing work-around is to remove the |
|
Thanks, I did and it works fine now. Also disabled autoupdates. |
|
@DrGo If you have a moment, could you share your platform particulars? Like in #976 (comment) ? |
|
Sure! |
|
Much obliged, @DrGo . This is very helpful information. |
|
Thanks @thac0 , |
|
I experience the exact same issue after trying to install Clear Linux for the first time. |
|
Same issue here, workaround works. Spec of the system (hope this helps): |
|
Unfortunately we haven't had any luck getting a similar system locally. Thanks for the reminder but we are a bit stuck in being able to further diagnose the issue =(. |
|
I hope you can find a similar system, although it should be not too difficult to source one: https://www.amazon.com/gp/aw/d/B004WKRDA4 If there is any other way I can help out (even if you want to login remotely on my system), let me know. |
|
Any progress on this issue? This is still occurring at every update. |
|
@JSEHV Unfortunately not getting a full fix. But we did add support for a workaround to avoid auto-update needing to be disabled. If you create a symlink to /dev/null for the initrds you don't want to be loaded in /etc/kernel/initrd.d it will cause clr-boot-manager to not add those lines to the systemd-boot loader config. So for example: |
|
Any possibility that this can be added to the clearlinux install on certain systems or mainboards? |
|
@FrankNL1984 I think this probably should stay as a manual work around for those who know they need it given that this involves touching system state and involves disabling other functionality. |
|
https://imgur.com/a/biDHE46 |
|
@chapagaiashim That's really unrelated to this bug. |
|
Sorry, I posted in the wrong section. My mistake. |
|
Same issue on clear install. Workaround worked for me. Waiting for bugfix |
|
Same here, Files: This entry works: This doesn't: dmidecode dmesg hash |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hi there,
As requested by William Douglas, I hereby post this issue on GitHub. I marked this issue as CRITICAL, because it keeps Clear Linux from booting.
Problem occurs when:
A) Installing Clear Linux from ISO version 30130
B) Upgrading to version 30170
C) Maybe more versions, but these ones are for sure
The installer/updater places this line in BOOT_PARTITION/loader/loader:conf: 'default Clear-linux-native-5.1.15.791' (install) or 'default Clear-linux-native-5.1.15.792' (update to 30170).
Looking in the file BOOT_PARTITION/loader/entries/Clear-linux-native-5.1.15-79X.conf, I see the line: 'initrd /EFI/org.clearlinux/freestanding-00-intel-ucode.cpio'.
The file BOOT_PARTITION/EFI/org.clearlinux/freestanding-00-intel-ucode.cpio exists.
However, on boot, my computer stops before booting, stating the following three lines:
Failed to open file: EFI\org.clearlinux\freestanding-00-intel-ucode.cpio
Trying to load files to higher address
Failed to open file: EFI\org.clearlinux\freestanding-00-intel-ucode.cpio
I resolved this issue by removing this intel-ucode line from the loader.conf file and disable swupd auto update, because autoupdate will regenerate that line which keeps my system from booting.
Using patched Intel DH67BL Intel Desktop board along with an Intel i7 2600K CPU.
Kind regards,
Frank ter Voorde
The text was updated successfully, but these errors were encountered: