[Bug]: Linux Guest Additions 7.2.10 fail to build vboxguest on RHEL 9.8 kernel 5.14.0-687.19.1.el9_8 — implicit declaration of function open_with_fake_path

[MichaelRoth-ger]

Version

7.2.10

Host OS Type

Windows

Host OS name + version

Windows 11

Host Architecture

x86

Guest OS Type

Linux

Guest Architecture

x86

Guest OS name + version

RHEL 9.8

Component

Guest Additions/Installer

What happened?

VBoxGuestAdditions 7.2.10 fails to build the vboxguest kernel module on RHEL 9.8 with kernel 5.14.0-687.19.1.el9_8.x86_64.

vboxadd-setup.log

How can we reproduce this?

1. Create RHEL 9.8 guest VM in VirtualBox 7.2.10
2. Ensure kernel-devel matching running kernel is installed
3. Mount VBoxGuestAdditions 7.2.10 ISO
4. Run: sudo sh ./VBoxLinuxAdditions.run
5. Check /var/log/vboxadd-setup.log — vboxguest build fails

Did you upload all of your necessary log files, screenshots, etc.?

• Yes, I've uploaded all pertinent files to this issue.

[angel-lafuente-echeazarra]

Hi all,

I can confirm the issue.

• VirtualBox version: 7.2.10
• Host OS: MS Windows 10 ESU
• Guest OS: Red Hat Linux 9.8
• Guest Linux Kernel version: 5.14.0-687.19.1.el9_8.x86_64

Issue raised after a regular kernel update (look at 'dnf_update_vboxadd_error.log').

After reboot, 'vboxadd' services failed. I attached the Guest Additions ISO to VM and tried to build modules but it failed (look at 'vboxadd-install.log').

dnf list installed kernel*
Updating Subscription Management repositories.
Installed Packages
kernel.x86_64                                                                     5.14.0-687.15.1.el9_8                                                        @rhel-9-for-x86_64-baseos-rpms   
kernel.x86_64                                                                     5.14.0-687.17.1.el9_8                                                        @rhel-9-for-x86_64-baseos-rpms   
kernel.x86_64                                                                     5.14.0-687.19.1.el9_8                                                        @rhel-9-for-x86_64-baseos-rpms   
kernel-core.x86_64                                                                5.14.0-687.15.1.el9_8                                                        @rhel-9-for-x86_64-baseos-rpms   
kernel-core.x86_64                                                                5.14.0-687.17.1.el9_8                                                        @rhel-9-for-x86_64-baseos-rpms   
kernel-core.x86_64                                                                5.14.0-687.19.1.el9_8                                                        @rhel-9-for-x86_64-baseos-rpms   
kernel-devel.x86_64                                                               5.14.0-687.15.1.el9_8                                                        @rhel-9-for-x86_64-appstream-rpms
kernel-devel.x86_64                                                               5.14.0-687.17.1.el9_8                                                        @rhel-9-for-x86_64-appstream-rpms
kernel-devel.x86_64                                                               5.14.0-687.19.1.el9_8                                                        @rhel-9-for-x86_64-appstream-rpms
kernel-devel-matched.x86_64                                                       5.14.0-687.19.1.el9_8                                                        @rhel-9-for-x86_64-appstream-rpms
kernel-headers.x86_64                                                             5.14.0-687.19.1.el9_8                                                        @rhel-9-for-x86_64-appstream-rpms
kernel-modules.x86_64                                                             5.14.0-687.15.1.el9_8                                                        @rhel-9-for-x86_64-baseos-rpms   
kernel-modules.x86_64                                                             5.14.0-687.17.1.el9_8                                                        @rhel-9-for-x86_64-baseos-rpms   
kernel-modules.x86_64                                                             5.14.0-687.19.1.el9_8                                                        @rhel-9-for-x86_64-baseos-rpms   
kernel-modules-core.x86_64                                                        5.14.0-687.15.1.el9_8                                                        @rhel-9-for-x86_64-baseos-rpms   
kernel-modules-core.x86_64                                                        5.14.0-687.17.1.el9_8                                                        @rhel-9-for-x86_64-baseos-rpms   
kernel-modules-core.x86_64                                                        5.14.0-687.19.1.el9_8                                                        @rhel-9-for-x86_64-baseos-rpms   
kernel-srpm-macros.noarch                                                         1.0-14.el9                                                                   @rhel-9-for-x86_64-appstream-rpms
kernel-tools.x86_64                                                               5.14.0-687.19.1.el9_8                                                        @rhel-9-for-x86_64-baseos-rpms   
kernel-tools-libs.x86_64                                                          5.14.0-687.19.1.el9_8                                                        @rhel-9-for-x86_64-baseos-rpms

Regards

vboxadd-install.log
dnf_update_vboxadd_error.log

[Achim-Haag]

Did yum update sunday afternoon in my (actually seldom used) development systems.
Result: no kernel update, stayed on 687.17
Heard of kernel update 687.19 today morning, so I took a snapshot followed by yum update
yum found kernel 687.20 leading to the above error while applying kernel-core.
Let me know if I could help in any way.

[angel-lafuente-echeazarra]

Hi there,

Red Hat has released a new kernel version 5.14.0-687.20 today and the issue persist.

# uname -r
5.14.0-687.20.1.el9_8.x86_64
# dnf list installed kernel*
Updating Subscription Management repositories.
Installed Packages
kernel.x86_64                                                                     5.14.0-687.17.1.el9_8                                                        @rhel-9-for-x86_64-baseos-rpms   
kernel.x86_64                                                                     5.14.0-687.19.1.el9_8                                                        @rhel-9-for-x86_64-baseos-rpms   
kernel.x86_64                                                                     5.14.0-687.20.1.el9_8                                                        @rhel-9-for-x86_64-baseos-rpms   
kernel-core.x86_64                                                                5.14.0-687.17.1.el9_8                                                        @rhel-9-for-x86_64-baseos-rpms   
kernel-core.x86_64                                                                5.14.0-687.19.1.el9_8                                                        @rhel-9-for-x86_64-baseos-rpms   
kernel-core.x86_64                                                                5.14.0-687.20.1.el9_8                                                        @rhel-9-for-x86_64-baseos-rpms   
kernel-devel.x86_64                                                               5.14.0-687.17.1.el9_8                                                        @rhel-9-for-x86_64-appstream-rpms
kernel-devel.x86_64                                                               5.14.0-687.19.1.el9_8                                                        @rhel-9-for-x86_64-appstream-rpms
kernel-devel.x86_64                                                               5.14.0-687.20.1.el9_8                                                        @rhel-9-for-x86_64-appstream-rpms
kernel-devel-matched.x86_64                                                       5.14.0-687.20.1.el9_8                                                        @rhel-9-for-x86_64-appstream-rpms
kernel-headers.x86_64                                                             5.14.0-687.20.1.el9_8                                                        @rhel-9-for-x86_64-appstream-rpms
kernel-modules.x86_64                                                             5.14.0-687.17.1.el9_8                                                        @rhel-9-for-x86_64-baseos-rpms   
kernel-modules.x86_64                                                             5.14.0-687.19.1.el9_8                                                        @rhel-9-for-x86_64-baseos-rpms   
kernel-modules.x86_64                                                             5.14.0-687.20.1.el9_8                                                        @rhel-9-for-x86_64-baseos-rpms   
kernel-modules-core.x86_64                                                        5.14.0-687.17.1.el9_8                                                        @rhel-9-for-x86_64-baseos-rpms   
kernel-modules-core.x86_64                                                        5.14.0-687.19.1.el9_8                                                        @rhel-9-for-x86_64-baseos-rpms   
kernel-modules-core.x86_64                                                        5.14.0-687.20.1.el9_8                                                        @rhel-9-for-x86_64-baseos-rpms   
kernel-srpm-macros.noarch                                                         1.0-14.el9                                                                   @rhel-9-for-x86_64-appstream-rpms
kernel-tools.x86_64                                                               5.14.0-687.20.1.el9_8                                                        @rhel-9-for-x86_64-baseos-rpms   
kernel-tools-libs.x86_64                                                          5.14.0-687.20.1.el9_8                                                        @rhel-9-for-x86_64-baseos-rpms   
# 

Attached, the log of the Guest Additions module build process triggered by updates.

vboxadd-setup-5.14.0-687.20.log

Best regards.

[MichaelRolltideroll]

Hi all,

I can confirm the issue.

• VirtualBox version: 7.2.10
• Host OS: MS Windows 10 ESU
• Guest OS: Red Hat Linux 9.8
• Guest Linux Kernel version: 5.14.0-687.19.1.el9_8.x86_64

Issue raised after a regular kernel update (look at 'dnf_update_vboxadd_error.log').

After reboot, 'vboxadd' services failed. I attached the Guest Additions ISO to VM and tried to build modules but it failed (look at 'vboxadd-install.log').

dnf list installed kernel*
Updating Subscription Management repositories.
Installed Packages
kernel.x86_64                                                                     5.14.0-687.15.1.el9_8                                                        @rhel-9-for-x86_64-baseos-rpms   
kernel.x86_64                                                                     5.14.0-687.17.1.el9_8                                                        @rhel-9-for-x86_64-baseos-rpms   
kernel.x86_64                                                                     5.14.0-687.19.1.el9_8                                                        @rhel-9-for-x86_64-baseos-rpms   
kernel-core.x86_64                                                                5.14.0-687.15.1.el9_8                                                        @rhel-9-for-x86_64-baseos-rpms   
kernel-core.x86_64                                                                5.14.0-687.17.1.el9_8                                                        @rhel-9-for-x86_64-baseos-rpms   
kernel-core.x86_64                                                                5.14.0-687.19.1.el9_8                                                        @rhel-9-for-x86_64-baseos-rpms   
kernel-devel.x86_64                                                               5.14.0-687.15.1.el9_8                                                        @rhel-9-for-x86_64-appstream-rpms
kernel-devel.x86_64                                                               5.14.0-687.17.1.el9_8                                                        @rhel-9-for-x86_64-appstream-rpms
kernel-devel.x86_64                                                               5.14.0-687.19.1.el9_8                                                        @rhel-9-for-x86_64-appstream-rpms
kernel-devel-matched.x86_64                                                       5.14.0-687.19.1.el9_8                                                        @rhel-9-for-x86_64-appstream-rpms
kernel-headers.x86_64                                                             5.14.0-687.19.1.el9_8                                                        @rhel-9-for-x86_64-appstream-rpms
kernel-modules.x86_64                                                             5.14.0-687.15.1.el9_8                                                        @rhel-9-for-x86_64-baseos-rpms   
kernel-modules.x86_64                                                             5.14.0-687.17.1.el9_8                                                        @rhel-9-for-x86_64-baseos-rpms   
kernel-modules.x86_64                                                             5.14.0-687.19.1.el9_8                                                        @rhel-9-for-x86_64-baseos-rpms   
kernel-modules-core.x86_64                                                        5.14.0-687.15.1.el9_8                                                        @rhel-9-for-x86_64-baseos-rpms   
kernel-modules-core.x86_64                                                        5.14.0-687.17.1.el9_8                                                        @rhel-9-for-x86_64-baseos-rpms   
kernel-modules-core.x86_64                                                        5.14.0-687.19.1.el9_8                                                        @rhel-9-for-x86_64-baseos-rpms   
kernel-srpm-macros.noarch                                                         1.0-14.el9                                                                   @rhel-9-for-x86_64-appstream-rpms
kernel-tools.x86_64                                                               5.14.0-687.19.1.el9_8                                                        @rhel-9-for-x86_64-baseos-rpms   
kernel-tools-libs.x86_64                                                          5.14.0-687.19.1.el9_8                                                        @rhel-9-for-x86_64-baseos-rpms

Regards

vboxadd-install.log dnf_update_vboxadd_error.log

[Achim-Haag]

The error message points to a conversion error while assigning the function's return value to the *pFile struct:
/tmp/vbox.0/r0drv/linux/fileio-r0drv-linux.c:162:30: warning: initialization of
‘struct file *’ from ‘int’ makes pointer from integer without a cast
[-Wint-conversion]

As this assignment worked up to kernel 687.17 and I couldn't grep -ri "open_with_fake" * in the VBox GA,
I searched in Kernel 687.17 source directory and found an external definition in
/usr/src/kernels/5.14.0-687.17.1.el9_8.x86_64/include/linux/fs.h:

   2580 extern struct file * dentry_open(const struct path *, int, const struct cred *);
   2581 extern struct file *dentry_create(const struct path *path, int flags,
   2582                                   umode_t mode, const struct cred *cred);
   2583 extern struct file * open_with_fake_path(const struct path *, int,					<----
   2584                                          struct inode*, const struct cred *);
   2585 static inline struct file *file_clone_open(struct file *file)
   2586 {
   2587         return dentry_open(&file->f_path, file->f_flags, file->f_cred);
   2588 }

The same after yum update leading to kernel 687.20, but as
grep -ri "open_with_fake" * didn't find anything, I looked at its fs.h:
less -N /usr/src/kernels/5.14.0-687.17.1.el9_8.x86_64/include/linux/fs.h

   2585 struct file *dentry_open(const struct path *path, int flags,
   2586                          const struct cred *creds);
   2587 struct file *dentry_create(const struct path *path, int flags, umode_t mode,
   2588                            const struct cred *cred);
   2589 struct path *backing_file_user_path(const struct file *f);
   2590
   2591 #ifdef CONFIG_SECURITY
   2592 void *backing_file_security(const struct file *f);
   2593 void backing_file_set_security(struct file *f, void *security);

So the former reference to open_with_fake_path was removed in the 687.20 kernel include.
Searching /usr doesn't find references to open_with_fake_path in directories for kernel 687.20 either (but in 687.17 and 687.19 directories):

# rpm -qa |grep -i "kernel-core"
kernel-core-5.14.0-687.15.1.el9_8.x86_64
kernel-core-5.14.0-687.17.1.el9_8.x86_64
kernel-core-5.14.0-687.20.1.el9_8.x86_64

# grep -ri "open_with_fake_path" *
grep: lib/modules/5.14.0-687.15.1.el9_8.x86_64/misc/vboxguest.ko: binary file matches
lib/modules/5.14.0-687.15.1.el9_8.x86_64/System.map:ffffffff814897b0 T __pfx_open_with_fake_path
lib/modules/5.14.0-687.15.1.el9_8.x86_64/System.map:ffffffff814897c0 T open_with_fake_path
lib/modules/5.14.0-687.15.1.el9_8.x86_64/System.map:ffffffff8284ba2c r __ksymtab_open_with_fake_path
lib/modules/5.14.0-687.15.1.el9_8.x86_64/System.map:ffffffff8287c86f r __kstrtab_open_with_fake_path
lib/modules/5.14.0-687.15.1.el9_8.x86_64/System.map:ffffffff828856d6 r __kstrtabns_open_with_fake_path
grep: lib/modules/5.14.0-687.17.1.el9_8.x86_64/misc/vboxguest.ko: binary file matches
lib/modules/5.14.0-687.17.1.el9_8.x86_64/System.map:ffffffff814897b0 T __pfx_open_with_fake_path
lib/modules/5.14.0-687.17.1.el9_8.x86_64/System.map:ffffffff814897c0 T open_with_fake_path
lib/modules/5.14.0-687.17.1.el9_8.x86_64/System.map:ffffffff8284ba2c r __ksymtab_open_with_fake_path
lib/modules/5.14.0-687.17.1.el9_8.x86_64/System.map:ffffffff8287c86f r __kstrtab_open_with_fake_path
lib/modules/5.14.0-687.17.1.el9_8.x86_64/System.map:ffffffff828856d6 r __kstrtabns_open_with_fake_path
src/kernels/5.14.0-687.15.1.el9_8.x86_64/include/linux/fs.h:extern struct file * open_with_fake_path(const struct path *, int,
src/kernels/5.14.0-687.15.1.el9_8.x86_64/Module.symvers:0x7768e229      open_with_fake_path     vmlinux EXPORT_SYMBOL
src/kernels/5.14.0-687.15.1.el9_8.x86_64/System.map:ffffffff814897b0 T __pfx_open_with_fake_path
src/kernels/5.14.0-687.15.1.el9_8.x86_64/System.map:ffffffff814897c0 T open_with_fake_path
src/kernels/5.14.0-687.15.1.el9_8.x86_64/System.map:ffffffff8284ba2c r __ksymtab_open_with_fake_path
src/kernels/5.14.0-687.15.1.el9_8.x86_64/System.map:ffffffff8287c86f r __kstrtab_open_with_fake_path
src/kernels/5.14.0-687.15.1.el9_8.x86_64/System.map:ffffffff828856d6 r __kstrtabns_open_with_fake_path
src/kernels/5.14.0-687.17.1.el9_8.x86_64/include/linux/fs.h:extern struct file * open_with_fake_path(const struct path *, int,
src/kernels/5.14.0-687.17.1.el9_8.x86_64/Module.symvers:0x7768e229      open_with_fake_path     vmlinux EXPORT_SYMBOL
src/kernels/5.14.0-687.17.1.el9_8.x86_64/System.map:ffffffff814897b0 T __pfx_open_with_fake_path
src/kernels/5.14.0-687.17.1.el9_8.x86_64/System.map:ffffffff814897c0 T open_with_fake_path
src/kernels/5.14.0-687.17.1.el9_8.x86_64/System.map:ffffffff8284ba2c r __ksymtab_open_with_fake_path
src/kernels/5.14.0-687.17.1.el9_8.x86_64/System.map:ffffffff8287c86f r __kstrtab_open_with_fake_path
src/kernels/5.14.0-687.17.1.el9_8.x86_64/System.map:ffffffff828856d6 r __kstrtabns_open_with_fake_path

It seems, Redhat based kernel 687.19 upward backported a kernel change and i found another function exists that I couldn't grep in kernel 687.17 and .15:

# grep -ri "kernel_file_open" *
lib/modules/5.14.0-687.20.1.el9_8.x86_64/System.map:ffffffff8148a770 T __pfx_kernel_file_open
lib/modules/5.14.0-687.20.1.el9_8.x86_64/System.map:ffffffff8148a780 T kernel_file_open
lib/modules/5.14.0-687.20.1.el9_8.x86_64/System.map:ffffffff8285d4d8 r __ksymtab_kernel_file_open
lib/modules/5.14.0-687.20.1.el9_8.x86_64/System.map:ffffffff8287e147 r __kstrtab_kernel_file_open
lib/modules/5.14.0-687.20.1.el9_8.x86_64/System.map:ffffffff828870a4 r __kstrtabns_kernel_file_open

So I think, fileio-r0drv-linux.c has to be changed regarding the version checking lines 157...163 as it seems, RHEL kernel 687.19 upwards replace open_with_fake_path by kernel_file_open, but that's just an opinion of a dumbass 😋 and has to be proved carefully:

    144     /*
    145      * Lookup the parent directory entry.
    146      */
    147 # if RTLNX_VER_MIN(2,6,28)
    148     rc = kern_path(pszFilename, 0, &Path);
    149 # else
    150 #  error "port me"
    151 # endif
    152     if (!rc)
    153     {
    154         /*
    155          * Open it.
    156          */
    157 # if RTLNX_VER_MIN(6,10,0)
    158         struct file *pFile = kernel_file_open(&Path, fOpenMode, current_cred());
    159 # elif RTLNX_VER_MIN(6,5,0)
    160         struct file *pFile = kernel_file_open(&Path, fOpenMode, d_inode(Path.dentry), current_cred());
    161 # elif RTLNX_VER_MIN(4,19,0)
    162         struct file *pFile = open_with_fake_path(&Path, fOpenMode, d_inode(Path.dentry), current_cred());
    163 # elif RTLNX_VER_MIN(3,6,0)
    164         struct file *pFile = dentry_open(&Path, fOpenMode, current_cred());
    165 # else
    166 #  error "port me"
    167 # endif

[Achim-Haag]

Simply changing fileio-r0drv-linux.c to

160 # elif RTLNX_VER_MIN(6,5,0) || RTLNX_RHEL_RANGE(9,8, 9,99)
		161         struct file *pFile = kernel_file_open(&Path, fOpenMode, d_inode(Path.dentry), current_cred());		

fails as kernel_file_open isn't declared as external in fs.h 👍
5.14.0-687.15.1.el9_8.x86_64/include/linux/fs.h:

   2583 **extern struct** file * open_with_fake_path(const struct path *, int,
   2584                                          struct inode*, const struct cred *);

5.14.0-687.20.1.el9_8.x86_64/include/linux/fs.h:

   1839 **struct** file *kernel_file_open(const struct path *path, int flags,
   1840                               struct inode *inode, const struct cred *cred);

As I foundkernel_file_open exported by the kernel (my interpretation):
5.14.0-687.20.1.el9_8.x86_64/Module.symvers:0xdc2ab0cb kernel_file_open vmlinux EXPORT_SYMBOL_GPL
I changed fileio-r0drv-linux.c again (lines 160-162)

		158 # if RTLNX_VER_MIN(6,10,0)
		159         struct file *pFile = kernel_file_open(&Path, fOpenMode, current_cred());
		160 // # elif RTLNX_VER_MIN(6,5,0)
		161 # elif RTLNX_VER_MIN(6,5,0) || RTLNX_RHEL_RANGE(9,8, 9,99)
		162 extern struct file *kernel_file_open(const struct path *path, int flags,struct inode *inode, const struct         cred *cred);
		163         struct file *pFile = kernel_file_open(&Path, fOpenMode, d_inode(Path.dentry), current_cred());
		164 # elif RTLNX_VER_MIN(4,19,0)
		165         struct file *pFile = open_with_fake_path(&Path, fOpenMode, d_inode(Path.dentry), current_cred());
		166 # elif RTLNX_VER_MIN(3,6,0)
		167         struct file *pFile = dentry_open(&Path, fOpenMode, current_cred());
		168 # else
		169 #  error "port me"
		170 # endif

Now I could generate kernel modules:

# rcvboxadd quicksetup 5.14.0-687.20.1.el9_8.x86_64
  VirtualBox Guest Additions: Building the Guest Additions 7.2.10 modules for kernel 5.14.0-687.20.1.el9_8.x86_64.
  VirtualBox Guest Additions: Building the main module.
  VirtualBox Guest Additions: Building the shared folder support module.
  VirtualBox Guest Additions: Building the graphics driver module.
  VirtualBox Guest Additions: Updating initramfs

Rebooted, no errors. But have done things I don't really understand so I could not test
(what's the usage of RTFileOpen? Only VBox logging?).

[Achim-Haag]

Btw, my selection works only for the actual kernel update 687.19 onwards:

# elif RTLNX_VER_MIN(6,5,0) || RTLNX_RHEL_RANGE(9,8, 9,99)

Regenerating VBox GA modules for kernels 687 below 19 leads to errors.
Maybe for RHEL based systems, there should be a more granular option (kernel patch 687, patchlevel 17,19,20 aso.).