Skip to content

DXMT: add LLVM build compatibility fixes#748

Open
moreaki wants to merge 1 commit into
VirtualBox:mainfrom
moreaki:agent/dxmt-llvm17-build-compat
Open

DXMT: add LLVM build compatibility fixes#748
moreaki wants to merge 1 commit into
VirtualBox:mainfrom
moreaki:agent/dxmt-llvm17-build-compat

Conversation

@moreaki

@moreaki moreaki commented Jul 8, 2026

Copy link
Copy Markdown

Summary

Add minimal DXMT/LLVM compatibility fixes. These are independent from the earlier GUI repaint and HiDPI work.

The VBoxDxMt link now derives LLVM libraries from the configured llvm-config when available. That avoids hard-coding a Homebrew libLLVM-N.dylib name and supports shared LLVM packages, while preserving the original component-library list as a fallback for Oracle/devtools trees without llvm-config.

The AIR argmemonly attribute handling is version-gated: LLVM 15 and older use the original enum attribute path, while LLVM 16 and newer use the MemoryEffects API required by LLVM 17+.

The AIR/DXBC conversion code now passes explicit element types into GEP construction instead of relying on typed-pointer introspection. For LLVM 15 typed-pointer builds, the vector-array helper preserves the aggregate element type where needed so IRBuilder::CreateGEP does not assert on mismatched source element types. The shuffle-mask sentinel is also kept local so the code is not tied to an LLVM-version-specific mask constant name.

Why

The pointer/API issues are real DXMT/LLVM compatibility bugs:

  • getNonOpaquePointerElementType() is invalid once LLVM opaque pointers are in use.
  • The GEP crash under LLVM 15 is also real: the IR builder was being given the wrong element type for typed-pointer aggregate accesses.
  • ArgMemOnly versus MemoryEffects::argMemOnly() is LLVM API drift.
  • The shuffle-mask sentinel naming is LLVM-version drift.

Compatibility note

This patch intentionally does not hard-code libLLVM-15.dylib. The source should build against LLVM 15 and newer LLVM APIs where possible, while keeping the link flexible through llvm-config and preserving the static component-library fallback.

On the macOS/Arm test host, the installed VirtualBox 7.2.12 VBoxDxMt.dylib does not have a dynamic libLLVM-17.dylib dependency according to otool -L. It contains LLVM symbols directly, and strings reports llvm-mc (based on LLVM 15.0.7). That suggests the shipped binary may still be produced through an LLVM 15-era static component-library toolchain.

For the macOS/Arm Metal/AIR path, LLVM 15.0.7 is the currently verified producer. Local LLVM 17/22 builds can get further through the source build, but newer upstream LLVM bitcode may not be accepted by Apple's Metal compiler without a downgrade/normalization layer or a different shader handoff boundary; this showed up locally as Failed to upgrade function bitcode.

Validation

  • Applied independently to origin/main.
  • git diff --check passes.
  • Configured a clean PR worktree on macOS/Arm with kBuild initialized.
  • Built VBoxDxMt successfully with Homebrew LLVM 15.0.7:
    • VBOX_LLVM_PATH=/opt/homebrew/opt/llvm@15
    • VBOX_LLVM_CONFIG=/opt/homebrew/opt/llvm@15/bin/llvm-config
  • otool -L out/darwin.arm64/release/dist/VirtualBox.app/Contents/MacOS/VBoxDxMt.dylib shows /opt/homebrew/opt/llvm@15/lib/libLLVM.dylib with current version 15.0.7.

Refs #742.

@oracle-contributor-agreement

Copy link
Copy Markdown

Thank you for your pull request and welcome to our community! To contribute, please sign the Oracle Contributor Agreement (OCA).
The following contributors of this PR have not signed the OCA:

To sign the OCA, please create an Oracle account and sign the OCA in Oracle's Contributor Agreement Application.

When signing the OCA, please provide your GitHub username. After signing the OCA and getting an OCA approval from Oracle, this PR will be automatically updated.

If you are an Oracle employee, please make sure that you are a member of the main Oracle GitHub organization, and your membership in this organization is public.

@oracle-contributor-agreement oracle-contributor-agreement Bot added the OCA Required At least one contributor does not have an approved Oracle Contributor Agreement. label Jul 8, 2026
@moreaki moreaki force-pushed the agent/dxmt-llvm17-build-compat branch from 4550410 to edc4e12 Compare July 8, 2026 08:39
@moreaki moreaki changed the title DXMT: add LLVM 17 build compatibility fixes DXMT: add LLVM build compatibility fixes Jul 8, 2026
@moreaki moreaki marked this pull request as ready for review July 8, 2026 08:45
@moreaki moreaki force-pushed the agent/dxmt-llvm17-build-compat branch from edc4e12 to b9b0359 Compare July 8, 2026 08:51
@moreaki moreaki marked this pull request as draft July 8, 2026 08:53
@moreaki moreaki marked this pull request as ready for review July 8, 2026 08:55
Derive VBoxDxMt LLVM link libraries from llvm-config when available, so shared libLLVM installs are not tied to a hard-coded LLVM major version. Keep the existing component-library list as a fallback for devtools trees without llvm-config.

Keep the AIR argmemonly attribute source-compatible with LLVM 15 and LLVM 16+ by using the older enum attribute before the MemoryEffects API.

Remove remaining AIR conversion dependencies on typed pointer introspection by passing explicit element types into GEP construction. For LLVM 15 typed-pointer builds, preserve the aggregate element type where needed so IRBuilder does not assert on mismatched GEP source element types. Use a local shuffle-mask sentinel to avoid depending on LLVM-version-specific names.

Signed-off-by: Roberto Nibali <rnibali@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

OCA Required At least one contributor does not have an approved Oracle Contributor Agreement.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant