No menu was displayed after pxe boot (3.2.0 on el8)

[didier13150]

Describe the bug

No menu was displayed after pxe boot. Install hangs just after download successfully pxelinux.0 via tftp

Steps to reproduce

1. Install cobbler 3.2.0 in fresh vm (rocky 8): https://gist.github.com/didier13150/44b77c74fab2a9047647f3ad1ad89b9b
2. Boot pxe new vm

Expected behavior

Menu displayed on pxe client

Cobbler version

from epel 3.2.0 on el8

Operating system

rocky, but exactly same problem from centos 8 or almalinux

Cobbler log

cobbler report
distros:
==========
Name                           : Centos-7-x86_64
Architecture                   : x86_64
Automatic Installation Template Metadata : {'tree': 'http://@@http_server@@/cblr/links/Centos-7-x86_64'}
TFTP Boot Files                : {}
Boot loader                    : grub
Breed                          : redhat
Comment                        : 
Fetchable Files                : {}
Initrd                         : /var/www/cobbler/distro_mirror/Centos-7/images/pxeboot/initrd.img
Kernel                         : /var/www/cobbler/distro_mirror/Centos-7/images/pxeboot/vmlinuz
Kernel Options                 : {}
Kernel Options (Post Install)  : {}
Management Classes             : []
OS Version                     : rhel7
Owners                         : ['admin']
Redhat Management Key          : 
Remote Boot Initrd             : ~
Remote Boot Kernel             : ~
Template Files                 : {}


profiles:
==========
Name                           : Centos-7-x86_64
Automatic Installation Template : sample.ks
Automatic Installation Metadata : {}
TFTP Boot Files                : {}
Comment                        : 
DHCP Tag                       : default
Distribution                   : Centos-7-x86_64
Enable gPXE?                   : 0
Enable PXE Menu?               : True
Fetchable Files                : {}
DHCP Filename Override         : <<inherit>>
Kernel Options                 : {}
Kernel Options (Post Install)  : {}
Management Classes             : []
Management Parameters          : <<inherit>>
Name Servers                   : []
Name Servers Search Path       : []
Next Server Override           : <<inherit>>
Owners                         : ['admin']
Parent Profile                 : 
Proxy                          : 
Red Hat Management Key         : <<inherit>>
Repos                          : []
Server Override                : <<inherit>>
Template Files                 : {}
Virt Auto Boot                 : 1
Virt Bridge                    : xenbr0
Virt CPUs                      : 1
Virt Disk Driver Type          : raw
Virt File Size(GB)             : 5
Virt Path                      : 
Virt RAM (MB)                  : 512
Virt Type                      : kvm


systems:
==========
Name                           : default
Automatic Installation Template : <<inherit>>
Automatic Installation Template Metadata : {}
TFTP Boot Files                : {}
Boot loader                    : <<inherit>>
Comment                        : 
Enable gPXE?                   : <<inherit>>
Fetchable Files                : {}
DHCP Filename Override         : <<inherit>>
Gateway                        : 
Hostname                       : 
Image                          : 
IPv6 Autoconfiguration         : False
IPv6 Default Device            : 
Kernel Options                 : {}
Kernel Options (Post Install)  : {}
Management Classes             : <<inherit>>
Management Parameters          : <<inherit>>
Name Servers                   : []
Name Servers Search Path       : []
Netboot Enabled                : True
Next Server Override           : <<inherit>>
Owners                         : <<inherit>>
Power Management Address       : 
Power Management ID            : 
Power Identity File            : 
Power Management Options       : 
Power Management Password      : 
Power Management Type          : ipmitool
Power Management Username      : 
Profile                        : Centos-7-x86_64
Internal Proxy                 : <<inherit>>
Redhat Management Key          : <<inherit>>
Repos Enabled                  : False
Serial Baud Rate               : 
Serial Device #                : 
Server Override                : <<inherit>>
Status                         : production
Template Files                 : {}
Virt Auto Boot                 : <<inherit>>
Virt CPUs                      : <<inherit>>
Virt Disk Driver Type          : <<inherit>>
Virt File Size(GB)             : <<inherit>>
Virt Path                      : <<inherit>>
Virt PXE Boot                  : 0
Virt RAM (MB)                  : <<inherit>>
Virt Type                      : <<inherit>>


repos:
==========

images:
==========
Name                           : memtest
Architecture                   : x86_64
Automatic installation file    : 
Breed                          : redhat
Comment                        : 
File                           : /boot/memtest86+-5.01
Image Type                     : direct
Virt NICs                      : 1
OS Version                     : 
Owners                         : ['admin']
Virt Auto Boot                 : 1
Virt Bridge                    : xenbr0
Virt CPUs                      : 1
Virt Disk Driver Type          : raw
Virt File Size (GB)            : 5
Virt Path                      : 
Virt RAM (MB)                  : 512
Virt Type                      : xenpv


mgmtclasses:
==========

packages:
==========

files:
==========

Screenshots

Additional information

Same install process is success on el7 (cobbler 2.8.5)

[SchoolGuy]

@didier13150 Did you run a cobbler sync before or not? In case no, could you do that and try again please? I don't expect magic to happen but this should work and reportedly it does at most installations I know.

[mnys176]

I saw this issue as well. Just a note, in 3.2.0 the cobbler get-loaders command is still implemented, but the URI it pulls the loaders from doesn't exist anymore. Try cat /var/lib/tftpboot/pxelinux.0 after the cobbler sync. If you get a GitHub 404 Not Found HTML page, this is the same problem I ran into a while back. I'm no expert on the implementation of cobbler sync, but as far as the loaders are concerned I think it just copies the loaders that are in /var/lib/cobbler/loaders to /var/lib/tftpboot. In that regard, it does its job well; the files it copies are just useless. All you would have to do in this case is make sure the proper executables are in the loaders directory:

1. Install the syslinux dependency
2. $ cp -f /usr/share/syslinux/{ldlinux.c32,libcom32.c32,libutil.c32,menu.c32,pxelinux.0} /var/lib/cobbler/loaders
3. $ cobbler sync

Now, cobbler sync will copy the actual binaries from /var/lib/cobbler/loaders and they should be usable by the client. Hope that helps! =)

Edit: I am using Cobbler on CentOS 8.

[SchoolGuy]

If that is actually the case then this can be closed as a duplicate of #2660

[SchoolGuy]

@didier13150 Please confirm if this is the case.

[mnys176]

@SchoolGuy One last note, cobbler get-loaders didn't exactly fail for me like in #2660, the files it got were just garbage.

[SchoolGuy]

@mnys176 Well the reason is the same as far as I understand. I removed the outdated upstream bootloaders.

[didier13150]

@SchoolGuy yes cobbler sync is the last line in my install process (I try to restart cobblerd as well and resync). I'm not sure it's a duplicate bug report, because version are mismatch (2.8.5 != 3.2.0) even the workaround was the same, delete the get-loaders target. Many thanks for your fast reply.

@mnys176 I don't have the idea to check type of pxelinux.0. It's a html file (which contains 404 error). You're save my life and my week :-). Thank you very much. Now it's working

[marbaa]

Not sure if I can hijack this issue, or create another. It won't start installation neither RHEL7.x/8.x from cobbler installed, because dhcp.template is looking for filename "grub/grub.0"; which doesn't exists anywhere. Let me know, if I should open new issue or explain it here.

[SchoolGuy]

@marbaa Let's create a different one for this one.

[benipeled]

I see this case and #2660 and wonder - aren't the syslinux files supposed to be provided with cobbler installation?

PS> The workaround suggested by @mnys176 on #2708 (comment) works for me on Fedora34

[SchoolGuy]

@benipeled No. They are not supposed to come with Cobbler. Please take the ones from your distribution. Taking them from Cobbler is bad for multitude of reasons that I already described in more then one place. We introduced cobbler mkloaders in recent versions that takes the bootloaders Cobbler can find from the OS automatically.

[benipeled]

Thanks @SchoolGuy for the quick answer,
What version includes the cobbler mkloaders command? I don't have it 3.2.2 (Fedora34)

btw, unless I missed something - I see no reference to the download/install pxelinux part on cobbler installation/quickstart docs - since is a basic-mandatory part, IMO it is worth mentioning it.

[SchoolGuy]

@benipeled The first version that has it as desired is probably 3.3.1

Feel free to open a PR with your suggestion. I will gladly review it. :)