Build vs Target build passes

[jwinarske]

I'm working on a Yocto (Linux) recipe for Filament. This enables any Linux platform (NXP, RaspberryPi, Qualcomm, Renesas, etc) to easily use filament.

The recipe is currently part of my Yocto meta-vulkan layer:
https://github.com/jwinarske/meta-vulkan/blob/jw/filament/recipes-graphics/filament/filament-vk_git.bb

In Yocto / OpenEmbedded (OE) there are in general two build passes: Native and Target

Native is used to generate host side tools that are used during the Target build pass. So no intent for display testing in a Native build pass. The expectation is that tests run on the target architecture (cross compiled).

Building sdl2. sdl2 for Wayland expects some libraries that are not present in the Native build environment, nor is it practical they get added.

Enter CMake variable FILAMENT_SKIP_SAMPLES. Using FILAMENT_SKIP_SAMPLES and adding minor CMake changes I can build Filament without sdl2 and things dependent on sdl2 (FilamentApp, tests, etc). Keeping in mind Native build is for creating the tools for Target build pass. Branch that builds for Native is here: https://github.com/meta-flutter/filament/tree/yocto
With this branch and mentioned recipe I get a Native package generated that has the required tools for the Target build pass.

Tests should be independent of samples. So it seems there could be an additional flag FILAMENT_SKIP_TESTS. Then if either FILAMENT_SKIP_SAMPLES or FILAMENT_SKIP_TESTS is set sdl2 is built. If both are set, then sdl2 would be skipped, call it a minimal host build, with tools only. This logic would also gate what gets installed in this state. Effectively only host side tools.

If you're okay with me adding FILAMENT_SKIP_TESTS and associated logic to support the Native/Target model for Yocto I will create a PR.

Beyond this I would also add minimal CMake logic to support a Mobile build pass for Linux.

[romainguy]

You could build only what you need. For instance our Android builds only compile the following targets on the host: matc resgen cmgen filamesh. This means samples and visuals tests are not compiled.

[jwinarske]

I understand your point of building target of interest opposed to all. For this scenario it would save build time, and limit installed files. This is good.

The only caveat is if you set FILAMENT_SUPPORT_WAYLAND (PR waiting on your review) sdl2 will fail if required packages are not present. sdl2 will fail without these, so there needs to be a way to gate this in the first place. So by skipping the build of sdl2 it would enable individual target builds as you suggest.

[romainguy]

sdl2 shouldn't be built if all you're building are host tools. Do you need to build Filament itself in what you call the native build?

[jwinarske]

Filament only for Target build pass (cross compile). For sdl2 deps there are the samples, and a couple tests. I just don't think gating tests with FILAMENT_SKIP_SAMPLES is a good idea, hence addition of FILAMENT_SKIP_TESTS.

[romainguy]

Ok then just build the tools and sdl2 shouldn't be built in the native pass (since you won't build the samples nor Filament and its tests).

[jwinarske]

When CMake is run it will run sdl2 tnt cmake where it fails to find dependent packages. Regardless if sdl2 is actually built.

Two options:

1. Remove required statement in sdl2 tnt cmake package check. Build failure will be far less obvious as to why.
2. Gate external to prevent sdl2 tnt cmake from running.

After that the remainder is cmake logic to enable mobile build pass for linux.

[romainguy]

Ah, see that's exactly why I was commenting in your PR that we don't like using external CMake files and use our own even for dependencies instead.

[jwinarske]

In the case of my current open PR, in order to build/link sdl2 it requires system specific packages. I would agree in case of non-system libraries.

[romainguy]

And that's fine but can't we have those dependencies be checked/required only when sdl2 is actually built?

[jwinarske]

It would involve custom steps with dependencies. In my opinion messy and not fun to maintain.

The change without proposed FILAMENT_SKIP_TESTS is in this commit:
meta-flutter@2351e1a

[romainguy]

FILAMENT_SKIP_SAMPLES is definitely wrong here. It would be better to define a FILAMENT_SKIP_SDL2 at the root level based on FILAMENT_SKIP_SAMPLES and maybe FILAMENT_SKIP_TESTS but even that seems a bit off to me. I'm not against a FILAMENT_SKIP_TESTS but that seems like a poor workaround (and brittle since we could create a visual tool for instance that requires sdl2, and then that would have to be taken into account, etc.).

[jwinarske]

I'm good with renaming proposed FILAMENT_SKIP_TESTS to FILAMENT_SKIP_SDL2. The commit using FILAMENT_SKIP_TESTS was just to see what was required for the build to pass.

[jwinarske]

So I have a working Yocto recipe. It only targets Vulkan. jwinarske/meta-vulkan#2

There are two recipes:

• filament-vk (Target - include files and static libraries)
• filament-vk-native (Host - tools used at compile time)

These artifacts can also be added to a SDK build, which is used for host cross compilation.

Implementing target demos would require quite a bit more work. I may create a separate repo for this later.

Filament changes are here: meta-flutter@a4b3bc9

These are based on the Vulkan Wayland PR. Once that gets merged I will submit new PR for the changes to support Yocto.

[romainguy]

Those changes look reasonable.

[jwinarske]

I went ahead and created a separate filament-samples repo and associated Yocto recipes. With this I can build all the samples with Vulkan Wayland Backend cross-compiled for target (any machine type), and install into Wayland based OS image. Master can be diffed with https://github.com/jwinarske/filament-samples to see the scope. It is a first pass, it builds, and I suspect it could be improved. If there is an appetite to pull this functionality into master I'm up for it.

Yocto recipe for Wayland based image:
https://github.com/jwinarske/meta-vulkan/blob/main/recipes-graphics/filament/filament-samples-vk_git.bb

filament-vk-native -> builds host tools
filament-vk -> builds filament static libs/headers based on Vulkan Wayland and creates -dev package
filament-samples-vk -> builds samples with Vulkan Wayland back end and installs into /usr/bin/filament-samples