Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Could not find the ESP! #16

Closed
suconakh opened this issue Aug 25, 2021 · 14 comments
Closed

Could not find the ESP! #16

suconakh opened this issue Aug 25, 2021 · 14 comments
Assignees
@Venom1991
Labels
enhancement New feature or request

Comments

@suconakh
Copy link

suconakh commented Aug 25, 2021 •
edited
Loading

Not sure what i should include here besides this. ESP is mounted for sure

$ sudo /usr/bin/refind-btrfs
Initializing the block devices using lsblk.
Initializing the physical partition table for device '/dev/sda' using lsblk.
Initializing the live partition table for device '/dev/sda' using findmnt.
Initializing the physical partition table for device '/dev/sdb' using lsblk.
Initializing the live partition table for device '/dev/sdb' using findmnt.
ERROR (refind_btrfs.state_management.conditions/conditions.py/check_filtered_block_devices): Could not find the ESP!

Feel free to ask any info about my setup
@Venom1991
Copy link
Owner

I'll need the output of the following commands:

lsblk --json --merge --paths --output NAME,TYPE,MAJ:MIN
lsblk /dev/sda --json --paths --tree --output PTUUID,PTTYPE,PARTUUID,PARTTYPE,PARTLABEL,UUID,NAME,FSTYPE,LABEL,MOUNTPOINT
lsblk /dev/sdb --json --paths --tree --output PTUUID,PTTYPE,PARTUUID,PARTTYPE,PARTLABEL,UUID,NAME,FSTYPE,LABEL,MOUNTPOINT

@Venom1991 Venom1991 self-assigned this Aug 25, 2021
@Venom1991 Venom1991 added the help wanted Extra attention is needed label Aug 25, 2021
@suconakh
Copy link
Author 

$ lsblk --json --merge --paths --output NAME,TYPE,MAJ:MIN
{
   "blockdevices": [
      {
         "name": "/dev/sda",
         "type": "disk",
         "maj:min": "8:0",
         "children": [
            {
               "name": "/dev/sda1",
               "type": "part",
               "maj:min": "8:1"
            },{
               "name": "/dev/sda2",
               "type": "part",
               "maj:min": "8:2"
            },{
               "name": "/dev/sda3",
               "type": "part",
               "maj:min": "8:3"
            },{
               "name": "/dev/sda4",
               "type": "part",
               "maj:min": "8:4"
            }
         ]
      },{
         "name": "/dev/sdb",
         "type": "disk",
         "maj:min": "8:16",
         "children": [
            {
               "name": "/dev/sdb1",
               "type": "part",
               "maj:min": "8:17"
            },{
               "name": "/dev/sdb2",
               "type": "part",
               "maj:min": "8:18"
            }
         ]
      },{
         "name": "/dev/sdc",
         "type": "disk",
         "maj:min": "8:32",
         "children": [
            {
               "name": "/dev/sdc1",
               "type": "part",
               "maj:min": "8:33"
            },{
               "name": "/dev/sdc2",
               "type": "part",
               "maj:min": "8:34"
            }
         ]
      }
   ]
}

This is my separate windows HDD.

$ lsblk /dev/sda --json --paths --tree --output PTUUID,PTTYPE,PARTUUID,PARTTYPE,PARTLABEL,UUID,NAME,FSTYPE,LABEL,MOUNTPOINT
{
   "blockdevices": [
      {
         "ptuuid": "25f3f253-5c55-4675-ac3e-946fa8f4e160",
         "pttype": "gpt",
         "partuuid": null,
         "parttype": null,
         "partlabel": null,
         "uuid": null,
         "name": "/dev/sda",
         "fstype": null,
         "label": null,
         "mountpoint": null,
         "children": [
            {
               "ptuuid": "25f3f253-5c55-4675-ac3e-946fa8f4e160",
               "pttype": "gpt",
               "partuuid": "739b6d80-27aa-4353-8e77-eb34c76fbcc3",
               "parttype": "c12a7328-f81f-11d2-ba4b-00a0c93ec93b",
               "partlabel": "EFI system partition",
               "uuid": "466B-657A",
               "name": "/dev/sda1",
               "fstype": "vfat",
               "label": null,
               "mountpoint": null
            },{
               "ptuuid": "25f3f253-5c55-4675-ac3e-946fa8f4e160",
               "pttype": "gpt",
               "partuuid": "33ff3db0-2649-48fc-ae78-8a06ffe5604e",
               "parttype": "e3c9e316-0b5c-4db8-817d-f92df00215ae",
               "partlabel": "Microsoft reserved partition",
               "uuid": null,
               "name": "/dev/sda2",
               "fstype": null,
               "label": null,
               "mountpoint": null
            },{
               "ptuuid": "25f3f253-5c55-4675-ac3e-946fa8f4e160",
               "pttype": "gpt",
               "partuuid": "0a46d03f-d00c-4dfc-b562-5cf04c489190",
               "parttype": "ebd0a0a2-b9e5-4433-87c0-68b6b72699c7",
               "partlabel": "Basic data partition",
               "uuid": "8C8E81EF8E81D25E",
               "name": "/dev/sda3",
               "fstype": "ntfs",
               "label": "Windows 10",
               "mountpoint": "/mnt/windows"
            },{
               "ptuuid": "25f3f253-5c55-4675-ac3e-946fa8f4e160",
               "pttype": "gpt",
               "partuuid": "1116434c-3cad-4159-a401-2ecd6d3b51a8",
               "parttype": "ebd0a0a2-b9e5-4433-87c0-68b6b72699c7",
               "partlabel": "Basic data partition",
               "uuid": "72A6A287A6A24C03",
               "name": "/dev/sda4",
               "fstype": "ntfs",
               "label": null,
               "mountpoint": "/mnt/storage"
            }
         ]
      }
   ]
}

This is Arch Linux

$ lsblk /dev/sdb --json --paths --tree --output PTUUID,PTTYPE,PARTUUID,PARTTYPE,PARTLABEL,UUID,NAME,FSTYPE,LABEL,MOUNTPOINT
{
   "blockdevices": [
      {
         "ptuuid": "1e80f7ee-7c21-4975-815d-261105538664",
         "pttype": "gpt",
         "partuuid": null,
         "parttype": null,
         "partlabel": null,
         "uuid": null,
         "name": "/dev/sdb",
         "fstype": null,
         "label": null,
         "mountpoint": null,
         "children": [
            {
               "ptuuid": "1e80f7ee-7c21-4975-815d-261105538664",
               "pttype": "gpt",
               "partuuid": "ed31ab35-a36e-8341-b60e-40977b4e66c0",
               "parttype": "0fc63daf-8483-4772-8e79-3d69d8477de4",
               "partlabel": null,
               "uuid": "51B8-51FF",
               "name": "/dev/sdb1",
               "fstype": "vfat",
               "label": "EFI",
               "mountpoint": "/boot"
            },{
               "ptuuid": "1e80f7ee-7c21-4975-815d-261105538664",
               "pttype": "gpt",
               "partuuid": "0bd39b61-63e8-6140-8f9b-9a3dd1528148",
               "parttype": "0fc63daf-8483-4772-8e79-3d69d8477de4",
               "partlabel": null,
               "uuid": "c8b3b189-ea1f-438d-b3ec-e731eb4f8542",
               "name": "/dev/sdb2",
               "fstype": "btrfs",
               "label": "ROOT",
               "mountpoint": "/btrfs"
            }
         ]
      }
   ]
}

@Venom1991
Copy link
Owner

Venom1991 commented Aug 25, 2021 •
edited
Loading

This part of the output presents a problem:

{
   "ptuuid": "25f3f253-5c55-4675-ac3e-946fa8f4e160",
   "pttype": "gpt",
   "partuuid": "739b6d80-27aa-4353-8e77-eb34c76fbcc3",
   "parttype": "c12a7328-f81f-11d2-ba4b-00a0c93ec93b",
   "partlabel": "EFI system partition",
   "uuid": "466B-657A",
   "name": "/dev/sda1",
   "fstype": "vfat",
   "label": null,
   "mountpoint": null
}

The "parttype" property's value is correct (c12a7328-f81f-11d2-ba4b-00a0c93ec93b) as well as the "fstype" property's value (vfat) but the value of the "mountpoint" property is null, apparently.
Did you add the ESP to your fstab file? As far as lsblk is concerned, your ESP is not mounted.

@suconakh
Copy link
Author

suconakh commented Aug 25, 2021 •
edited
Loading

This part is Windows ESP. My refind ESP is on /dev/sdb1 and it is mounted for sure as well as present in fstab. I tried to mount my windows efi partition and here is the output

$ sudo /usr/bin/refind-btrfs
Initializing the block devices using lsblk.
Initializing the physical partition table for device '/dev/sda' using lsblk.
Initializing the live partition table for device '/dev/sda' using findmnt.
Initializing the physical partition table for device '/dev/sdb' using lsblk.
Initializing the live partition table for device '/dev/sdb' using findmnt.
Found the ESP mounted at '/mnt/windows-boot' on '/dev/sda1'.
Found the root partition on '/dev/sdb2'.
ERROR (refind_btrfs/__init__.py/main): An unexpected error happened, exiting...
Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/refind_btrfs/__init__.py", line 80, in main
    exit_code = runner.run()
  File "/usr/lib/python3.9/site-packages/refind_btrfs/console/cli_runner.py", line 49, in run
    if not machine.run():
  File "/usr/lib/python3.9/site-packages/refind_btrfs/state_management/refind_btrfs_machine.py", line 102, in run
    while model.next_state():
  File "/usr/lib/python3.9/site-packages/transitions/core.py", line 393, in trigger
    return self.machine._process(func)
  File "/usr/lib/python3.9/site-packages/transitions/core.py", line 1172, in _process
    return trigger()
  File "/usr/lib/python3.9/site-packages/transitions/core.py", line 411, in _trigger
    return self._process(event_data)
  File "/usr/lib/python3.9/site-packages/transitions/core.py", line 420, in _process
    if trans.execute(event_data):
  File "/usr/lib/python3.9/site-packages/transitions/core.py", line 265, in execute
    if not self._eval_conditions(event_data):
  File "/usr/lib/python3.9/site-packages/transitions/core.py", line 246, in _eval_conditions
    if not cond.check(event_data):
  File "/usr/lib/python3.9/site-packages/transitions/core.py", line 185, in check
    return predicate(*event_data.args, **event_data.kwargs) == self.target
  File "/usr/lib/python3.9/site-packages/refind_btrfs/state_management/conditions.py", line 92, in check_filtered_block_devices
    boot = none_throws(esp_device.boot)
  File "/usr/lib/python3.9/site-packages/refind_btrfs/utility/helpers.py", line 247, in none_throws
    raise AssertionError(message)
AssertionError: Unexpected 'None'

I also tried to swap SATA cables to make Linux drive appear as /dev/sda so /dev/sda1 is a Linux ESP and it didn't help. Result was the same without Windows ESP - Could not find the ESP. And with mounted Windows ESP only these lines changed:

Found the ESP mounted at '/mnt/windows-boot' on '/dev/sdb1'.
Found the root partition on '/dev/sda2'.

Also i tried to umount all Windows NTFS partitions that i had and it still couldn't find the ESP

@Venom1991
Copy link
Owner

Venom1991 commented Aug 26, 2021 •
edited
Loading

Well, I don't know exactly why, in your case, lsblk reports the /dev/sdb ESP's mountpoint as null. That is a problem because I cannot access anything related to rEFInd that way (its config file, primarily).

Having more than one ESP is definitely problematic, imho. I've not dabbled with supporting such a setup, unfortunately. I also have Windows installed alongside Linux (installed after Windows) but only one ESP (with rEFInd installed on it, of course).
The problem really boils down to what I've mentioned here, previously.

Can you access the ESP partition (the one on /dev/sdb/) while your Linux system is running (by using a file manager, for instance)?

EDIT: I see that your /boot and ESP are one and the same partition, judging by this part of the output:

{
   "ptuuid": "1e80f7ee-7c21-4975-815d-261105538664",
   "pttype": "gpt",
   "partuuid": "ed31ab35-a36e-8341-b60e-40977b4e66c0",
   "parttype": "0fc63daf-8483-4772-8e79-3d69d8477de4",
   "partlabel": null,
   "uuid": "51B8-51FF",
   "name": "/dev/sdb1",
   "fstype": "vfat",
   "label": "EFI",
   "mountpoint": "/boot"
}

If so, you technically do not have a separate /boot, from the perspective of this tool as I suppose your kernel and microcode images are located on it. Your / partition is mounted as /btrfs, which is also pretty interesting. You have quite an exotic setup, I have to say.

I'd have to think about exactly how to implement support for this particular scenario - the "fstype" property's value is correct, the partition itself is obviously mounted but the "parttype" property's value (its type's GUID) is not the one I expect, per specification .

@Venom1991 Venom1991 added enhancement New feature or request and removed help wanted Extra attention is needed labels Aug 26, 2021
@Venom1991
Copy link
Owner

Venom1991 commented Aug 26, 2021 •
edited
Loading

Also, I'd like to see findmnt's output, as well:
findmnt --json --mtab --real --nofsroot --output PARTUUID,PARTLABEL,UUID,SOURCE,FSTYPE,LABEL,TARGET,OPTIONS

Thank you in advance.

@suconakh
Copy link
Author

I unplugged my Windows drive SATA cable and problem is still here. So it seems to me that having more than one ESP has nothing to do with this. Gotta be that i messed up my Linux ESP. My install is fresh so it's ok to me to quickly fix this even if it would require to wipe and redo everything. Here is the output

$  findmnt --json --mtab --real --nofsroot --output PARTUUID,PARTLABEL,UUID,SOURCE,FSTYPE,LABEL,TARGET,OPTIONS
{
   "filesystems": [
      {
         "partuuid": "0bd39b61-63e8-6140-8f9b-9a3dd1528148",
         "partlabel": null,
         "uuid": "c8b3b189-ea1f-438d-b3ec-e731eb4f8542",
         "source": "/dev/sdb2",
         "fstype": "btrfs",
         "label": "ROOT",
         "target": "/",
         "options": "rw,noatime,nodiratime,compress=zstd:3,ssd,discard=async,space_cache,autodefrag,commit=120,subvolid=256,subvol=/@"
      },{
         "partuuid": "0bd39b61-63e8-6140-8f9b-9a3dd1528148",
         "partlabel": null,
         "uuid": "c8b3b189-ea1f-438d-b3ec-e731eb4f8542",
         "source": "/dev/sdb2",
         "fstype": "btrfs",
         "label": "ROOT",
         "target": "/.snapshots",
         "options": "rw,noatime,nodiratime,compress=zstd:3,ssd,discard=async,space_cache,autodefrag,commit=120,subvolid=262,subvol=/@snapshots"
      },{
         "partuuid": "0bd39b61-63e8-6140-8f9b-9a3dd1528148",
         "partlabel": null,
         "uuid": "c8b3b189-ea1f-438d-b3ec-e731eb4f8542",
         "source": "/dev/sdb2",
         "fstype": "btrfs",
         "label": "ROOT",
         "target": "/home",
         "options": "rw,noatime,nodiratime,compress=zstd:3,ssd,discard=async,space_cache,autodefrag,commit=120,subvolid=257,subvol=/@home"
      },{
         "partuuid": "0bd39b61-63e8-6140-8f9b-9a3dd1528148",
         "partlabel": null,
         "uuid": "c8b3b189-ea1f-438d-b3ec-e731eb4f8542",
         "source": "/dev/sdb2",
         "fstype": "btrfs",
         "label": "ROOT",
         "target": "/swap",
         "options": "rw,relatime,compress=zstd:3,ssd,discard=async,space_cache,autodefrag,commit=120,subvolid=264,subvol=/@swap"
      },{
         "partuuid": "0bd39b61-63e8-6140-8f9b-9a3dd1528148",
         "partlabel": null,
         "uuid": "c8b3b189-ea1f-438d-b3ec-e731eb4f8542",
         "source": "/dev/sdb2",
         "fstype": "btrfs",
         "label": "ROOT",
         "target": "/srv",
         "options": "rw,noatime,nodiratime,compress=zstd:3,ssd,discard=async,space_cache,autodefrag,commit=120,subvolid=261,subvol=/@srv"
      },{
         "partuuid": "0bd39b61-63e8-6140-8f9b-9a3dd1528148",
         "partlabel": null,
         "uuid": "c8b3b189-ea1f-438d-b3ec-e731eb4f8542",
         "source": "/dev/sdb2",
         "fstype": "btrfs",
         "label": "ROOT",
         "target": "/var/abs",
         "options": "rw,noatime,nodiratime,compress=zstd:3,ssd,discard=async,space_cache,autodefrag,commit=120,subvolid=259,subvol=/@abs"
      },{
         "partuuid": "0bd39b61-63e8-6140-8f9b-9a3dd1528148",
         "partlabel": null,
         "uuid": "c8b3b189-ea1f-438d-b3ec-e731eb4f8542",
         "source": "/dev/sdb2",
         "fstype": "btrfs",
         "label": "ROOT",
         "target": "/var/cache/pacman/pkg",
         "options": "rw,noatime,nodiratime,compress=zstd:3,ssd,discard=async,space_cache,autodefrag,commit=120,subvolid=258,subvol=/@pkg"
      },{
         "partuuid": "0bd39b61-63e8-6140-8f9b-9a3dd1528148",
         "partlabel": null,
         "uuid": "c8b3b189-ea1f-438d-b3ec-e731eb4f8542",
         "source": "/dev/sdb2",
         "fstype": "btrfs",
         "label": "ROOT",
         "target": "/var/log",
         "options": "rw,noatime,nodiratime,compress=zstd:3,ssd,discard=async,space_cache,autodefrag,commit=120,subvolid=292,subvol=/@log"
      },{
         "partuuid": "0bd39b61-63e8-6140-8f9b-9a3dd1528148",
         "partlabel": null,
         "uuid": "c8b3b189-ea1f-438d-b3ec-e731eb4f8542",
         "source": "/dev/sdb2",
         "fstype": "btrfs",
         "label": "ROOT",
         "target": "/var/tmp",
         "options": "rw,noatime,nodiratime,compress=zstd:3,ssd,discard=async,space_cache,autodefrag,commit=120,subvolid=260,subvol=/@tmp"
      },{
         "partuuid": "ed31ab35-a36e-8341-b60e-40977b4e66c0",
         "partlabel": null,
         "uuid": "51B8-51FF",
         "source": "/dev/sdb1",
         "fstype": "vfat",
         "label": "EFI",
         "target": "/boot",
         "options": "rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,utf8,errors=remount-ro"
      },{
         "partuuid": "0bd39b61-63e8-6140-8f9b-9a3dd1528148",
         "partlabel": null,
         "uuid": "c8b3b189-ea1f-438d-b3ec-e731eb4f8542",
         "source": "/dev/sdb2",
         "fstype": "btrfs",
         "label": "ROOT",
         "target": "/btrfs",
         "options": "rw,noatime,nodiratime,compress=zstd:3,ssd,discard=async,space_cache,autodefrag,commit=120,subvolid=5,subvol=/"
      },{
         "partuuid": "0a46d03f-d00c-4dfc-b562-5cf04c489190",
         "partlabel": "Basic data partition",
         "uuid": "8C8E81EF8E81D25E",
         "source": "/dev/sda3",
         "fstype": "fuseblk",
         "label": "Windows 10",
         "target": "/mnt/windows",
         "options": "rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other,blksize=4096"
      },{
         "partuuid": "1116434c-3cad-4159-a401-2ecd6d3b51a8",
         "partlabel": "Basic data partition",
         "uuid": "72A6A287A6A24C03",
         "source": "/dev/sda4",
         "fstype": "fuseblk",
         "label": null,
         "target": "/mnt/storage",
         "options": "rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other,blksize=4096"
      }
   ]
}

@Venom1991
Copy link
Owner

Venom1991 commented Aug 26, 2021 •
edited
Loading

Even this picture on the Wiki article is showing what is, essentially (assuming I'm interpreting it correctly), a two in one kind of partition (both ESP and /boot) so it must be a common enough setup, I guess.

The most sensible approach, at this time, would be to allow the user to define the ESP partition's UUID in the config file. This should be an optional value which would be (if defined) taken for granted, so to speak. I can use that value to try and match the ESP directly from lsblk's output.
If it isn't defined, the current mechanism (trying to identify it programmatically) would be employed.

What I still don't understand is where exactly are your kernel images located at? Do they reside on this ESP/boot hybrid (for lack of a better word) partition or are they somewhere else?

EDIT:
Don't start from scratch just so you can shoehorn your setup into being accepted by this tool, there is no need for you to be doing that if you have a stable, running Arch install. Just let me try implementing support for this scenario.

@suconakh
Copy link
Author

The most sensible approach, at this time, would be to allow the user to define the ESP partition's UUID in the config file.

I though about that too. It would be nice if there was such an option in the config
Here is my boot directory

$ ls /boot
amd-ucode.img*  initramfs-linux-zen-fallback.img*  refind_linux.conf*
EFI/            initramfs-linux-zen.img*           vmlinuz-linux-zen*

$ ls /boot/EFI
refind/  tools/

$ ls /boot/EFI/refind
BOOT.CSV*  icons/  keys/  refind.conf*  refind_x64.efi*  themes/  vars/

@Venom1991
Copy link
Owner

So, it is a separate /boot partition and your kernel and microcode images do not reside on a Btrfs partition?

Sure, I'll try to implement this feature during the following weekend.

@suconakh
Copy link
Author

Yup, it's a vfat partition with microcode, kernel and refind on it mounted as /boot.

@Venom1991
Copy link
Owner

Venom1991 commented Aug 28, 2021 •
edited
Loading

I've just released an implementation of this proposed feature, tried it locally and I've been able to identify the ESP by setting its UUID in the config file. By default this new option (named "esp_uuid", first one listed in the file) is set to an empty UUID and thus ignored.

You can try setting it to "ed31ab35-a36e-8341-b60e-40977b4e66c0" and hope for the best, I guess. In any case, please report back here with the results, when convenient.

@suconakh
Copy link
Author

I set esp_uuid and after that the stanzas started to generate, became bootable and everything works as expected. Thanks for the great work on this must-have tool!

@Venom1991
Copy link
Owner

That's just great, I'm glad it finally worked out for you!

@Venom1991 Venom1991 mentioned this issue Oct 28, 2021
Closed
@UltraBlackLinux UltraBlackLinux mentioned this issue Aug 30, 2022
Closed
@sentakuhm sentakuhm mentioned this issue Nov 21, 2023
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Venom1991 Venom1991

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@Venom1991 @suconakh