Building mesa from source (rebuild package from source)

[MadLittleMods]

Rebuild Arch/Manjaro package from source

The reason I want to build a package from source is because mesa-git currently requires llvm@17 but since I'm using Manjaro stable, it only has llvm@16 and gives unable to satisfy dependency errors (see #50). I need mesa-git because otherwise Halo Infinite won't launch.

This guide is going to be using the Manjaro chrootbuild tool in order to build the Arch/Manjaro packages in a clean environment (install via pamac install manjaro-chrootbuild). It's possible just to use makepkg directly as mentioned in the Arch docs but this will install a bunch of packages that are used just for making the package (makedepends) to your system which is probably fine but also just makes everything messy (plus you could always clean up orphaned packages on your system pretty easily).

(If you're just using Arch, I see clean-chroot-manager recommended in a few places or the official docs on how to build in a clean chroot)

Check what version of mesa you currently have installed

$ glxinfo | grep Mesa
client glx vendor string: Mesa Project and SGI
OpenGL core profile version string: 4.6 (Core Profile) Mesa 24.0.2-manjaro1.1
OpenGL version string: 4.6 (Compatibility Profile) Mesa 24.0.2-manjaro1.1
OpenGL ES profile version string: OpenGL ES 3.2 Mesa 24.0.2-manjaro1.1

Building mesa from source

• mesa source repository: https://gitlab.freedesktop.org/mesa/mesa
• Manjaro package repository source:
  • mesa
  • lib32-mesa
• Arch package repository source:
  • (this is where to find the PKGBUILD for the mesa package in the Arch extra repo)
  • mesa
  • lib32-mesa

pamac install manjaro-chrootbuild

# Run this outside of the folder containing the PKGBUILD run - where 
# pkg-name is the folder containing the actual PKGBUILD file
#
# This will build `*.pkg.tar.zst` packages that you can install with `pacman`
sudo chrootbuild \
  -c `# Use a clean chroot` \
  -b stable `# Use the stable branch of the package repo` \
  -p mesa `# Build the mesa package` \
  -p lib32-mesa `# Build the lib32-mesa package`

# Install your newly built packages
sudo pacman --upgrade mesa-1:24.0.6-1-x86_64.pkg.tar.zst
sudo pacman --upgrade lib32-mesa-1:24.0.6-1-x86_64.pkg.tar.zst

By default, when you run chrootbuild, it uses the /var/lib/chrootbuild/ directory which is useful to check out the file structure and clean-up or add anything needed if something goes wrong during the build process.

Building mesa-git from source

1. On both mesa-git and lib32-mesa-git, use the View Changes option on the right, switch to the Summary view and copy the "Clone" URL to clone the git repo (ex. https://aur.archlinux.org/mesa-git.git/)
   • Or use the Download snapshot option on both mesa-git and lib32-mesa-git to get .tar.gz package archives
2. Extract the archives in a directory you want to build everything in (like ~/Downloads/mesa-scratch-build)
3. Edit the PKGBUILD files and rename the packages so they don't collide with package names in AUR or the main package repositories. This way when you update, pamac won't try to replace your custom build with what's in the repo.
   • In mesa-git/PKGBUILD:
     • pkgname=mesa-git -> pkgname=mesa-custom
     • Add mesa-git to the list of provides=(...)
   • In lib32-mesa-git/PKGBUILD:
     • pkgname=lib32-mesa-git -> pkgname=lib32-mesa-custom
     • Add lib32-mesa-git to the list of provides=(...)
4. More edits to the PKGBUILD files:
   • In the $MESA_WHICH_LLVM case 4 -> remove the llvm and clang version requirements (we're going to assume the llvm/clang versions in the stable branch of the package repo are good enough)
   • In the meson_options, remove --wrap-mode=nofallback ^[1] (try without this change first; you may have the correct versions already)
   • In the meson_options, update -D glvnd=enabled -> -D glvnd=true ^[2]
   • In the meson_options, update -D vulkan-drivers=amd,intel,intel_hasvk,swrast,virtio,nouveau -> -D vulkan-drivers=amd,intel,intel_hasvk,swrast,virtio,nouveau-experimental ^[2]
   • Copy the makedepends from the PKGBUILD in the Manjaro lib32-mesa to the lib32-mesa-git version.

pamac install manjaro-chrootbuild

# Build `mesa-git` first
#
# Run this outside of the folder containing the PKGBUILD run - where 
# pkg-name is the folder containing the actual PKGBUILD file
#
# This will build `*.pkg.tar.zst` packages that you can install with `pacman`
MESA_WHICH_LLVM=4 sudo chrootbuild \
  -c `# Use a clean chroot` \
  -b stable `# Use the stable branch of the package repo` \
  -p mesa-git `# Build the mesa package`

# Build `lib32-mesa-git`
#
# I ran into a couple errors/issues with this so see the [3] reference below
MESA_WHICH_LLVM=4 sudo chrootbuild \
  -c `# Use a clean chroot` \
  -b stable `# Use the stable branch of the package repo` \
  -i /home/eric/Downloads/mesa-git-scratch-build/mesa-git-todo.pkg.tar.zst `# Install the mesa-git package we just built` \
  -p lib32-mesa-git `# Build the lib32-mesa package`



# Install your newly built packages
# See [4] for errors/issues
sudo pacman --upgrade mesa-git-TODO.pkg.tar.zst
sudo pacman --upgrade lib32-mesa-git-TODO.pkg.tar.zst

[1]: Dependency DirectX-Headers found: NO found 1.611.0 but need: '>= 1.613.0' ERROR: Dependency 'DirectX-Headers' is required but not found.

The reason we edit PKGBUILD to remove --wrap-mode=nofallback from the meson_options, is to avoid this directx-headers version mismatch on Manjaro.

The other option is to checkout a previous version of mesa that has a compatible version with your package repository.

[...]
Run-time dependency directx-headers found: NO (tried pkgconfig and cmake)
Dependency DirectX-Headers found: NO found 1.611.0 but need: '>= 1.613.0'
Run-time dependency directx-headers found: NO (tried pkgconfig and cmake)
Not looking for a fallback subproject for the dependency DirectX-Headers because:
Use of fallback dependencies is disabled.

mesa/meson.build:586:20: ERROR: Dependency 'DirectX-Headers' is required but not found.

A full log can be found at /build/mesa-git/src/_build/meson-logs/meson-log.txt
==> ERROR: A failure occurred in build().
    Aborting...
==> ERROR: Building package [mesa-git] failed.
      Cleaning up.

[2]: mesa/meson.build:21:0: ERROR: Value enabled is not boolean (true or false). ERROR: Options "nouveau" are not in allowed choices: "auto, amd, broadcom, freedreno, intel, intel_hasvk, panfrost, swrast, virtio, imagination-experimental, microsoft-experimental, nouveau-experimental, all"

To fix, update -D glvnd=enabled -> -D glvnd=true. I was able to see the difference by looking at the PKGBUILD in the Manjaro/Arch repos.

Relevant error without this fix:

MESA_WHICH_LLVM=4 sudo chrootbuild \
  -c `# Use a clean chroot` \
  -b stable `# Use the stable branch of the package repo` \
  -p mesa-git `# Build the mesa package`

[...]
==> Starting prepare()...
==> Starting pkgver()...
==> Updated version: mesa-git 24.0.7.183753.cc175010c5d.d41d8cd-1
==> Starting build()...
The Meson build system
Version: 1.4.0
Source dir: /build/mesa-git/src/mesa
Build dir: /build/mesa-git/src/_build
Build type: native build

mesa/meson.build:21:0: ERROR: Value enabled is not boolean (true or false).

A full log can be found at /build/mesa-git/src/_build/meson-logs/meson-log.txt
==> ERROR: A failure occurred in build().
    Aborting...
==> ERROR: Building package [mesa-git] failed.
      Cleaning up.

The other error you might run into

[...]
==> Starting prepare()...
==> Starting pkgver()...
==> Updated version: mesa-git 24.0.7.183753.cc175010c5d.d41d8cd-1
==> Starting build()...
The Meson build system
Version: 1.4.0
Source dir: /build/mesa-git/src/mesa
Build dir: /build/mesa-git/src/_build
Build type: native build

mesa/meson.build:21:0: ERROR: Options "nouveau" are not in allowed choices: "auto, amd, broadcom, freedreno, intel, intel_hasvk, panfrost, swrast, virtio, imagination-experimental, microsoft-experimental, nouveau-experimental, all"

A full log can be found at /build/mesa-git/src/_build/meson-logs/meson-log.txt
==> ERROR: A failure occurred in build().
    Aborting...
==> ERROR: Building package [mesa-git] failed.
      Cleaning up.

[3]: conflicting files wayland-protocols: /usr/share/wayland-protocols/stable/[...].xml exists in filesystem (owned by mesa-git)

When building lib32-mesa-git, it requires mesa-git which is an AUR package so pacman doesn't know about it. We can provide the mesa-git-todo.pkg.tar.zst package that we just built to the lib32-mesa-git chrootbuild using the -i flag (-i <pkg> Install local package (specify full path!)).

MESA_WHICH_LLVM=4 sudo chrootbuild \
  -c `# Use a clean chroot` \
  -b stable `# Use the stable branch of the package repo` \
  -i /home/eric/Downloads/mesa-git-scratch-build/mesa-git-todo.pkg.tar.zst `# Install the mesa-git package we just built` \
  -p lib32-mesa-git `# Build the lib32-mesa package`
  
[...]
error: failed to commit transaction (conflicting files)
wayland-protocols: /usr/share/pkgconfig/wayland-protocols.pc exists in filesystem (owned by mesa-git)
wayland-protocols: /usr/share/wayland-protocols/stable/linux-dmabuf/linux-dmabuf-v1.xml exists in filesystem (owned by mesa-git)
wayland-protocols: /usr/share/wayland-protocols/stable/presentation-time/presentation-time.xml exists in filesystem (owned by mesa-git)
[...]
==> ERROR: Building package [lib32-mesa-git] failed.
      Cleaning up.

But we run into error: failed to commit transaction (conflicting files) problem. mesa-git uses wayland-protocols as a makedepends and for some reason is conflicting with lib32-mesa-git which also wants to install wayland-protocols for some reason.

Normally, this situation is resolved with these sort of docs in the Arch wiki but it's harder to interact in the chrootbuild environment. Since we have nothing to lose just trying things in a clean chroot environment, deleting the conflicting files seemed to work just fine.

sudo rm -rf /var/lib/chrootbuild/usr/share/wayland-protocols
sudo rm /var/lib/chrootbuild/usr/share/pkgconfig/wayland-protocols.pc

# Then continue the build
MESA_WHICH_LLVM=4 sudo chrootbuild \
  -b stable `# Use the stable branch of the package repo` \
  -p lib32-mesa-git `# Build the lib32-mesa package`

[4]: conflicting files /usr/share/wayland-protocols/stable/[...].xml exists in both 'mesa-git' and 'lib32-mesa-git'

If you see some conflicting files errors when installing the package in your actual system, you may just need to delete your orphaned packages. For example, I ran into some conflicting files with directx-headers and wayland-protocols. But turns out they are orphans anyway (pamac list --orphans) and can be removed with pamac remove --orphans.

And for comparison, I never had to do anything special when I was able to pamac install mesa-git before the llvm version bump so I'd be hesitant just to go and delete stuff. Here is what the Arch docs suggest doing when running into Failed to commit transaction (conflicting files) error.

sudo pacman --upgrade /home/eric/Downloads/mesa-git-scratch-build/mesa-git-24.1.0_rc3.188419.71f25dc28ca.d41d8cd-1-x86_64.pkg.tar.zst
loading packages...
resolving dependencies...
looking for conflicting packages...
:: mesa-git and mesa are in conflict (mesa-libgl). Remove mesa? [y/N] y

Packages (2) mesa-1:24.0.6-1 [removal]  mesa-git-24.1.0_rc3.188419.71f25dc28ca.d41d8cd-1

Total Installed Size:  301.59 MiB
Net Upgrade Size:      205.39 MiB

:: Proceed with installation? [Y/n]
(1/1) checking keys in keyring                                                                   [#########################################################] 100%
(1/1) checking package integrity                                                                 [#########################################################] 100%
(1/1) loading package files                                                                      [#########################################################] 100%
(1/1) checking for file conflicts                                                                [#########################################################] 100%
error: failed to commit transaction (conflicting files)
mesa-git: /usr/include/directx/D3D12TokenizedProgramFormat.hpp exists in filesystem (owned by directx-headers)
mesa-git: /usr/include/directx/d3d12.h exists in filesystem (owned by directx-headers)
mesa-git: /usr/include/directx/d3d12.idl exists in filesystem (owned by directx-headers)
[...]
mesa-git: /usr/share/wayland-protocols/stable/presentation-time/presentation-time.xml exists in filesystem (owned by wayland-protocols)
mesa-git: /usr/share/wayland-protocols/stable/viewporter/viewporter.xml exists in filesystem (owned by wayland-protocols)
[...]
Errors occurred, no packages were upgraded.

Even after solving the first conflicting files problem, I also see an additional conflicting files problem when trying to install both mesa-git and lib32-mesa-git on my actual system. But this time the conflict is between the two packages we just built, ex. /usr/share/wayland-protocols/stable/linux-dmabuf/linux-dmabuf-v1.xml exists in both 'mesa-git' and 'lib32-mesa-git'.

Not sure how to solve this one yet ❌

sudo pacman --upgrade /home/eric/Downloads/mesa-git-scratch-build/mesa-git-todo.pkg.tar.zst /home/eric/Downloads/mesa-git-scratch-build/lib32-mesa-git-todo.pkg.tar.zst
loading packages...
resolving dependencies...
looking for conflicting packages...
:: mesa-git and mesa are in conflict (mesa-libgl). Remove mesa? [y/N] y
:: lib32-mesa-git and lib32-mesa are in conflict. Remove lib32-mesa? [y/N] y

Packages (4) lib32-mesa-1:24.0.2-1 [removal]  mesa-1:24.0.2-1 [removal]  lib32-mesa-git-24.1.0_rc3.188419.71f25dc28ca.d41d8cd-1
             mesa-git-24.1.0_rc3.188419.71f25dc28ca.d41d8cd-1

Total Installed Size:  469.04 MiB
Net Upgrade Size:      289.25 MiB

:: Proceed with installation? [Y/n]
(2/2) checking keys in keyring                                                                   [#########################################################] 100%
(2/2) checking package integrity                                                                 [#########################################################] 100%
(2/2) loading package files                                                                      [#########################################################] 100%
(2/2) checking for file conflicts                                                                [#########################################################] 100%
error: failed to commit transaction (conflicting files)
/usr/share/pkgconfig/wayland-protocols.pc exists in both 'mesa-git' and 'lib32-mesa-git'
/usr/share/wayland-protocols/stable/linux-dmabuf/linux-dmabuf-v1.xml exists in both 'mesa-git' and 'lib32-mesa-git'
/usr/share/wayland-protocols/stable/presentation-time/presentation-time.xml exists in both 'mesa-git' and 'lib32-mesa-git'
[...]
Errors occurred, no packages were upgraded.

[5]: chrootbuild -n

If we try to use the chrootbuild -n option to Install built pkg to chroot fs (default when building lists), we run into mesa-git and mesa are in conflict (mesa-libgl). Remove mesa? [y/N] and I don't know of a way to say 'Yes' to the conflict.

MESA_WHICH_LLVM=4 sudo chrootbuild \
  -c `# Use a clean chroot` \
  -b stable `# Use the stable branch of the package repo` \
  -n `# Install built packages to the chroot (because lib32-mesa depends on mesa)` \
  -p mesa-git `# Build the mesa package` \
  -p lib32-mesa-git `# Build the lib32-mesa package`
  
[...]
==> Finished making: mesa-git 24.2.0_devel.189025.74ea0d006ef.d41d8cd-1 (Sat 11 May 2024 10:40:41 PM UTC)
==> Installing package mesa-git with pacman -U...
loading packages...
resolving dependencies...
looking for conflicting packages...
:: mesa-git and mesa are in conflict (mesa-libgl). Remove mesa? [y/N]
error: unresolvable package conflicts detected
error: failed to prepare transaction (conflicting dependencies)
:: mesa-git and mesa are in conflict
==> WARNING: Failed to install built package(s).
==> ERROR: Building package [mesa-git] failed.
      Cleaning up.

Use local copy of git repo as source in PKGBUILD

I wasn't able to get this to work. According to the docs, something like 'mesa::git+file:///home/eric/Documents/gitlab/mesa' should work but it always just gave an error that it does not appear to be a git repository. I tried a bunch of variations with no luck.

I'm guessing this is something to do with the chroot jail and not being able to access the file system outside of it but it also seems like the build might bind mount any files/directories inside the chroot (or is supposed to) so I don't know.

Other references

References:

• https://wiki.archlinux.org/title/Arch_build_system
• https://wiki.archlinux.org/title/PKGBUILD
  • Explains the source=('directory::url#fragment?query') syntax, https://man.archlinux.org/man/PKGBUILD.5#USING_VCS_SOURCES
• https://wiki.manjaro.org/index.php/Arch_User_Repository

Other references:

• https://www.reddit.com/r/archlinux/comments/x9off9/comment/inr30rb/
• https://www.reddit.com/r/ManjaroLinux/comments/zr1jhj/manjaroarch_and_radeon_7900_series_a_howto_guide/
• https://forum.manjaro.org/t/how-do-i-apply-this-patch-on-mesa-git-for-installation/65898
• https://cloudcone.com/docs/article/installing-packages-from-source-in-arch-linux/
• https://forum.linuxconfig.org/t/how-to-rebuild-a-package-using-the-arch-linux-build-system-linuxconfig-org/4798
  • https://linuxconfig.org/how-to-rebuild-a-package-using-the-arch-linux-build-system
• Building and using mesa for development and testing, https://gist.github.com/Venemo/a9483106565df3a83fc67a411191edbd

Using manjaro-chrootbuild:

• https://forum.manjaro.org/t/building-aur-packages-using-manjaro-tools/45416/2
• https://forum.manjaro.org/t/pamac-aur-and-whether-packages-are-built-in-a-clean-chroot/104084/2