Skip to content

Releases: StormBytePP/StormByte-BuildMaster

Version 2.0.1

Choose a tag to compare

@StormBytePP StormBytePP released this 27 Sep 00:00

[Summary]

BuildMaster is a small CMake DSL for a graph of other people’s builds.
You declare each dependency once — how it is produced, what it waits on — and the parent gets one shared prefix instead of a pile of ExternalProject / FetchContent glue.
CMake and Meson trees are first-class.

Typical use: add the submodule, add_subdirectory, declare components, then depend / link.
Declaration order does not matter.
The public surface is ten commands on purpose; everything else is internal.

If you landed here from a release link and have not read the tree:

Fixed

  • Nested project() no longer stacks bm/<id> under the child's -B. Component build dirs stay ${BUILDMASTER_BINDIR}/bm/<id> of the outermost consumer, so Windows CMAKE_OBJECT_PATH_MAX (250) is not blown by Suite BUNDLED graphs (Crypto → Buffer → Logger → String → Base).
  • Windows env runner Add-Type dumped SOURCE_CODE_ERROR under MSVC LIB. Enable-BmAnsiConsole compiled the VT P/Invoke while the process still had the parent job LIB/INCLUDE/LIBPATH (invalid SDK fragments such as 10.0.26100.0//x64). PowerShell 5.1 treats those as compiler errors. The variables are cleared only around Add-Type and restored after, in both runner_windows.ps1.in and runner_windows_silent.ps1.in. Nested LIB prepend for the real compile is unchanged.
  • Nested -G Ninja used a ghost CMAKE_MAKE_PROGRAM. The ninja tool already exported NINJA_EXECUTABLE on the component toolchain. It did not export CMAKE_MAKE_PROGRAM, so a child CMake (including a clang-cl leaf under an msvc parent) ran C:/ProgramData/chocolatey/bin/ninja.exe --version and died when that path was missing. When the ninja tool is loaded, update_toolchain.cmake now also exports CMAKE_MAKE_PROGRAM to the same binary. If the tool was not requested, nothing is written.
  • A later build re-entered the leaf on the unpatched tree. Configure, build and install now hash the component srcdir with CMake only (file(SHA256) per file, string(SHA256) of the sorted paths). No sha256sum and no cat. The other half of the key is options, toolchain, CMAKE_BUILD_TYPE, IPO, mode, produced and host OS/arch, stored in bm-stamp.extra. bm-stamp.key is created when install finishes and refreshed when a later build actually recompiles, and is the file a later install cache will read. A match skips nested cmake/meson and does not touch the prefix. A miss runs that stage. GIT={PATCH} is applied before the hash and the git root is reset when the last live holder finishes, including on a hit, so a later make may re-apply and re-hash. The hit still does not parse upstream CMakeLists.txt. Fixture stamp-patch: upstream cmake_minimum_required(VERSION 99.0), the patch lowers it to 3.20, the second stamp-leaf_install is a stamp hit and the worktree is back on 99.0.
  • buildmaster_group_add required the group to already exist. Membership is stored even when buildmaster_group has not run yet. Finalize (_bm_group_plan) is the only existence check: an id named by buildmaster_group_add that was never created is FATAL, including when that was the only declaration. Cycles and id clashes stay FATAL where they already were. A group is still not a component, a meta, or a link edge. Fixture groups-late: buildmaster_group_add names late-outer and late-inner before either buildmaster_group. The outline is still late-outer → late-inner → late-leaf (indent 2). Negative group-undefined dies at configure with group was never created.
  • Shared-dep skip dropped the consumer's need. First-wins still skips configure and build when links/<id>.cmake already exists (already built by). That skip is not "this consumer has no dependency". The nested process appends the skipped id to bm-reuse-needs.txt. After that configure returns, the parent walks the id and the dests already stored in its links file and hangs the closure on the consumer (LINKS_ATTACHED, so a later rewrite keeps it). <consumer>_configure / <consumer>_build wait on <id>_install when that stage is in this process, otherwise on the parent component that created the links file (<owner>_install). A skipped configure is still not a second build. Fixture links-race: RaceA and RaceB both declare RaceMid, RaceMid declares RaceBase, no parent buildmaster_depend and no hoist. RaceB_build waits on RaceA_install. Both shared libs call a Base-only symbol and link only RaceMid.
  • Transitive buildmaster_link stopped at one recorded hop. buildmaster_link(A B) still names only B. _bm_links_write_one now unions this process's link edges with LINKS_ATTACHED and the dests already stored in links/<id>.cmake, then walks those ids to a fixpoint (seen-set, cycle cut). A parent rewrite no longer replaces links/StormByte-String.cmake with the edges it knows and drops String → Base. A later configure that skips the winner still flattens -l / Stem.lib for the whole chain (--no-allow-shlib-undefined, Apple -undefined,error; Windows already fails unresolved). The caller does not name grandparents. Fixture links-transitive: shared TransBase ← TransMid ← TransUpper, and deferred TransLeaf links only TransUpper after the skip, while calling a Base-only symbol.
  • WHOLE wrap was empty on ELF. _bm_opt_whole_items built -Wl,--whole-archive + produced .a + -Wl,--no-whole-archive, then fragment.cmake flattened the CMake list to spaces and target_link_libraries(<id> INTERFACE …) let CMake classify the -Wl tokens as flags and the archives as libraries. The DSO line became libavutil.a … libavfilter.a -Wl,--whole-archive -Wl,--no-whole-archive. GNU ld.bfd (single pass) then dropped unreferenced avutil objects (av_md5_sum, AES/HMAC, …) while lld still linked. ELF now emits one $<LINK_GROUP:BM_WHOLE,…> and registers CMAKE_{,C_,CXX_}LINK_GROUP_USING_BM_WHOLE (prefix / suffix --whole-archive / --no-whole-archive) so every produced static of that id stays inside the wrap. Apple (-force_load) and MSVC (-WHOLEARCHIVE:) are unchanged. The fragment no longer replaces ; with spaces. WHOLE still means one region around all produced archives of the id, not one wrap per file.

Version 2.0.0

Choose a tag to compare

@StormBytePP StormBytePP released this 04 Sep 22:03

[Summary]

BuildMaster is a small CMake DSL for a graph of other people’s builds.
You declare each dependency once — how it is produced, what it waits on — and the parent gets one shared prefix instead of a pile of ExternalProject / FetchContent glue.
CMake and Meson trees are first-class.

Typical use: add the submodule, add_subdirectory, declare components, then depend / link.
Declaration order does not matter.
The public surface is ten commands on purpose; everything else is internal.

If you landed here from a release link and have not read the tree:

2.x versus 1.0.x is a different product: declarative graph, no generated fragment to include(), no public dependant factories, no public create_cmake_component / create_meson_component.
A 1.x CMakeLists.txt will not configure.
That is the point.

What 1.0.1 already did internally (headers mode, per-component toolchains, nested binutils, env runners) is still there; the caller and the declaration shape changed.

Added

  • Declarative component graph.
    buildmaster_depend(source dest) is order-only (id, stage name, or existing CMake target).
    buildmaster_link(source dest) waits and records the same depend edge, so buildmaster_link(A B) before buildmaster_component(B) still defers A.
    dest may be a component (all produced libs), a library spec (name / subdir/name), a target, or an archive path.
    Spec dests are listed on the source install OUTPUT so Ninja has a production rule.
    A spec or on-disk archive stays link-only.
    Duplicate explicit edges are WARNING + no-op; internal auto-deps do not warn.
    Unresolvable dest at finalize is FATAL.
  • Deferred materialization + eager INTERFACE stub.
    Registration only stores metadata.
    Fragments and stage targets exist at the end of parent configure (cmake_language(DEFER) on CMAKE_SOURCE_DIR).
    Declaration order does not matter.
    Components without edges still configure during parent configure; components with edges configure at build time under <id>_configure.
    add_library(<id> INTERFACE) at registration so a sibling ALIAS / target_link_libraries before DEFER does not see a missing target.
  • buildmaster_component.
    Backend is inferred from srcdir (CMakeLists.txt vs meson.build; neither + headers → none).
    Dual markers are FATAL unless BACKEND=cmake or BACKEND=meson (allowed set: BUILDMASTER_FACTORY_BACKENDS).
    BACKEND= empty or an unknown name is FATAL.
    SOURCE=<rel> is applied before that detect: the value is always under the positional srcdir (a leading / is still a child of srcdir, not an absolute path).
    Escape above the component srcdir (or above the host CMAKE_SOURCE_DIR) is FATAL before any existence probe.
    After the boundary check, a missing directory is FATAL.
    The resolved tree must contain a .git only when GIT={…} is also set; SOURCE= itself is not a git root.
    Arity is id title srcdir options mode produced [optstr].
    Mode is static, shared, headers, or executable.
    options is a backend-agnostic CMake list of KEY=value (a single string is one pair).
    Idioms CFLAGS, CXXFLAGS, CPPFLAGS, LDFLAGS, INCLUDES, DEFINITIONS are rewritten for the nested compile and append to the parent job / toolchain.
    Every other key is forwarded as -DKEY=value to the nested CMake configure or Meson setup (Meson also uses -D).
    A leading -D / -d / /D on the key is stripped.
    Private to that nested step, not INTERFACE.
    none ignores the list.
    none outside headers mode is FATAL unless NOINSTALL is set; a unique backend is still used when present.
  • Mode executable.
    Produced specs are binary stems under BUILDMASTER_INSTALL_BINDIR (<stem> on Unix, <stem>${CMAKE_EXECUTABLE_SUFFIX} on Windows; never .exe.exe).
    The parent stub is still add_library(<id> INTERFACE) — it is not an IMPORTED executable.
    Nested CMake/Meson build the binary; LINK / LINKFLAGS apply to that nested link.
    Linux gcc/clang: nested and parent CMAKE_<LANG>_LINK_EXECUTABLE wrap <LINK_LIBRARIES> with -Wl,--start-group / --end-group so a static exe that lists provider before consumer (ld.bfd single pass) still resolves, including under LTO.
    Darwin ld64 and MSVC/clang-cl do not get those flags.
    SHARED/MODULE recipes are unchanged.
    WHOLE on the executable itself is INFO and ignored.
    A leaf of a WHOLE meta is the same INFO skip: the meta must not emit -WHOLEARCHIVE:<bindir>/<stem>.exe / --whole-archive of the binary.
    STRIPRES does not run.
    PC={…} ENABLED is FATAL.
    REPACK on the executable itself is FATAL.
    A first-level depend/link dest that is executable is not a REPACK member (INFO skip; not FATAL as a “publishing member”).
    A leaf of a REPACK meta that is executable is the same INFO skip and is not an INPUT of the merge.
    Extra buildmaster_link dests that are raw library specs are not folded as IMPORTED archives on an executable.
    Produced exe stems are never BM_LINKS_LIBNAMES and never land on a consumer or meta link line (gzip.exe is not a library).
    Membership in a meta is order-only (*_install).
    Oficios: rename_executable (when RENAME) then outputs.
  • Assigned build directory.
    There is no public builddir argument.
    The graph uses ${CMAKE_CURRENT_BINARY_DIR}/bm/<id> and file(MAKE_DIRECTORY).
    The path is an internal property, not part of the DSL.
  • Outline groups.
    buildmaster_group(id [title]) and buildmaster_group_add(group member…).
    A group is not a component, a meta, or a link: no targets, no edges, no install.
    After the graph is complete it only walks members in addition order and prints configure banners with indent.
    Nested groups are allowed; a member must not contain a group.
    Cycles and id clashes with a component/meta/group are FATAL (caller file:line).
    Configure-time messages inherit the walk indent; compile/install stay flat.
    Eager vs deferred is unchanged — the outline is cosmetic order, not a wait edge.
  • Headers island (mode headers, with or without a backend).
    No backend, or NOINSTALL headers: private.
    Direct consumers get a quoted -I on that id’s nested configure only (CMake CMAKE_{C,CXX}_FLAGS and Meson c_args / cpp_args).
    It does not recurse through further BM components, runners, or INTERFACE.
    Publishing headers (backend + not NOINSTALL) install into the shared prefix; the prefix -I already covers them.
    Several private islands on one consumer accumulate several -I.
  • FILES={URL=…;NAME=…;UNPACK;SOURCE[=rel];FORCE;MD5=|SHA256=|EXPECTED_HASH=…}.
    Declarative download on buildmaster_component (meta + any FILES group is FATAL).
    Always cached under BUILDMASTER_DOWNLOADSDIR (FORCE refetches).
    Unpack is ${BUILDMASTER_BINDIR}/files/<NAME>/, before nested configure (eager components included).
    Inner SOURCE (at most one group, requires UNPACK) is the srcdir after unpack — the positional path is ignored by design (WARNING).
    That key is not the component optstr SOURCE=.
    Other unpacked groups inject a private -I on that id only (same rule as the headers island).
    GIT={…} + FILES SOURCE is FATAL.
    Replaces the 1.x file_download* + dependant-component pattern.
  • LINK= / LINK={…}.
    Raw system linker names on the component or meta INTERFACE.
    They propagate to whoever links that id.
    Not BM nodes.
    Revives 1.x LINK_EXTRA under a shorter name.
  • LINKFLAGS= / LINKFLAGS={…}.
    Raw linker flags (/FORCE:MULTIPLE, -Wl,-Bsymbolic) for the nested cmake/meson link only.
    Groups: WINDOWS, LINUX, MAC, UNIX (UNIX = Linux + macOS).
    A group that does not apply is skipped at INFO.
    Unknown platform key is FATAL.
    Folded into that id’s OPTIONS at finalize (CMAKE_EXE/SHARED/MODULE_LINKER_FLAGS or Meson c_link_args / cpp_link_args).
    Not target_link_options on the INTERFACE — a consumer of this id does not inherit the flags.
    Meta: WARNING + ignore (no nested link).
    Headers: WARNING + ignore.
  • GIT={FETCH;SWITCH=<branch>;RESET;PATCH=<file>;ROOT=<rel>;TITLE=…}.
    Srcdir git work on buildmaster_component.
    ROOT= is always under the component srcdir (same isolation as optstr SOURCE=): escape FATAL before existence, missing tree FATAL, work tree that is the host CMAKE_SOURCE_DIR FATAL.
    Flush order is fixed: FETCH → SWITCH → RESET → PATCH (PATCH order is declaration order).
    Relative PATCH= is from CMAKE_CURRENT_SOURCE_DIR.
    Empty GIT / GIT={} is WARNING.
    Meta + any git op is FATAL.
    FETCH / RESET are inner flags.
    Post-install reset runs only when a PATCH was queued, and only inside that work tree (ROOT=).
  • RENAME (flag, default ON).
    Post-install normalize of variant basenames.
    Libraries: oficio rename_library, worker component/rename/normalize_install_libraries.cmake.
    Executables: oficio rename_executable, worker normalize_install_executables.cmake.
    Headers mode never registers a rename oficio (stamps are not archives).
  • WHOLE.
    Whole-archive link of produced static archives.
    Ignored (INFO) on headers and executable, including when that executable is a leaf of a WHOLE meta.
  • STRIPRES (flag, default ON).
    After RENAME, strip *.res from static MSVC / clang-cl archives.
    Shared / headers / executable never strip.
  • NOINSTALL + REPACK.
    Bare NOINSTALL builds without publishing to the shared prefix (artifacts sta...
Read more

Version 1.0.1

Choose a tag to compare

@StormBytePP StormBytePP released this 26 Aug 08:17

Added

Header-only components

  • New library mode headers in create_component / create_cmake_stages / create_meson_stages
  • Templates component_headers.cmake.in and component_headers_dependant.cmake.in
    • INTERFACE library only (no IMPORTED static/shared archives)
    • target_include_directories(... SYSTEM INTERFACE "${BUILDMASTER_INSTALL_INCLUDEDIR}")
    • Depends on <component>_install like library modes
  • Install OUTPUT is a stamp file ${builddir}/.buildmaster_headers_installed (avoids empty OUTPUT / CMP0175 with header-only trees)
  • install_exec (CMake and Meson) creates missing stamp paths after a successful install
  • Simple API:
    • create_cmake_headers_component
    • create_cmake_headers_dependant_component
    • create_meson_headers_component
    • create_meson_headers_dependant_component
  • Build stage is kept for a uniform graph (header-only projects are typically no-ops)

Nested linker and binutils propagation

  • Nested CMake configures forward the parent toolchain so third-party builds match the top level:
    • Linker: CMAKE_LINKER_TYPE, CMAKE_LINKER, CMAKE_C_COMPILER_LINKER, CMAKE_CXX_COMPILER_LINKER, CMAKE_MT
    • Archiver / nm: CMAKE_AR, CMAKE_C_COMPILER_AR, CMAKE_CXX_COMPILER_AR, CMAKE_RANLIB, CMAKE_C_COMPILER_RANLIB, CMAKE_CXX_COMPILER_RANLIB, CMAKE_NM
    • CMAKE_MODULE_LINKER_FLAGS (in addition to existing EXE/SHARED linker flags)
  • tools/cmake/update_toolchain.cmake registers non-empty linker and archiver cache entries for the BuildMaster toolchain dump (paths normalized to forward slashes)
  • Nested Meson setups:
    • Build _MESON_LINK_ARGS from CMAKE_EXE_LINKER_FLAGS
    • Linker selection via buildmaster_fuse_ld_flag() (driver-safe -fuse-ld= flavors only):
      • CMAKE_LINKER_TYPE=LLD / forced LLD → -fuse-ld=lld-link (Windows) or -fuse-ld=lld (elsewhere)
      • CMAKE_LINKER_TYPE=MSVC → -fuse-ld=link
      • Else map CMAKE_LINKER / BM_TC_LINKER basename (lld, ld.lld, gold, mold, bfd, …); system ld and absolute paths such as /usr/bin/ld emit no -fuse-ld (GCC rejects path-form -fuse-ld=/usr/bin/ld)
    • Pass AR / RANLIB into Meson setup via cmake -E env (from CMAKE_AR / CMAKE_RANLIB or the process environment)
  • Env runners (runner_linux.sh.in / runner_windows.bat.in) export AR, RANLIB, and NM so any command launched through ENV_RUNNER inherits the same binutils as nested CMake/Meson
  • env/init_vars.cmake and update_env_runner() resolve AR / RANLIB / NM from CMAKE_AR / CMAKE_RANLIB / CMAKE_NM or ENV{…} before regenerating the runner scripts

Per-component toolchains

  • Optional trailing TOOLCHAIN argument on the simple component API (create_cmake_* / create_meson_*, including headers and dependant variants) and on the atomic stage helpers (create_cmake_stages / create_meson_stages); the same profile applies to that component’s configure, build and install whether stages are generated via the factory or wired explicitly
  • Named profiles under toolchain/profiles/: gcc, clang, clang-cl, msvc
    • clang: LLD required on Linux; LLD not forced on macOS
    • clang-cl: LLD (lld-link) + llvm-lib (Windows only)
    • msvc: cl + link.exe + lib.exe (Windows only)
    • gcc: system linker/archiver (LLD not forced)
  • New module toolchain/ (init, helpers, profiles): validation, platform guards, profile load
  • Toolchain file registry (single source of truth for parent and component dumps):
    • buildmaster_toolchain_reset / export / export_raw / write
    • Modules register state in */update_toolchain.cmake; the parent toolchain.cmake is written once at the end of the BuildMaster root CMakeLists.txt
    • buildmaster_toolchain_write_component: parent registry snapshot + profile compiler/binutils CACHE FORCE overlay (no hand-maintained variable list in stage generators)
  • buildmaster_clean_ldflags() / buildmaster_clean_cflags() strip known-incoherent tokens by profile:
    • msvc: remove LLD / Clang-LTO switches; other flags preserved
    • clang-cl: remove MSVC LTCG tokens (/GL, /LTCG and variants) that clang-cl ignores or mishandles
  • Component-local env runners (normal + silent) when TOOLCHAIN is set; parent global runners are not rewritten
  • When TOOLCHAIN is set, configure and build status lines (and dependant configure COMMENT) include (with toolchain <name>); omitted when inheriting the parent job
  • IPO/LTO is never enabled by a profile; if the parent already had IPO on, nested stages keep a coherent setting (CMAKE_INTERPROCEDURAL_OPTIMIZATION_* / Meson b_lto) without re-injecting MSVC /GL
  • Fully backward compatible: omitting TOOLCHAIN keeps previous behaviour

Fixed

  • Per-component TOOLCHAIN and nested BuildMaster installs: components with a toolchain override no longer bootstrap a second install tree under the component build dir (e.g. …/buffer/build/thirdparty/buildmaster/install). Nested configures load a component toolchain file built from the parent registry so BUILDMASTER_INSTALL_*, template dirs (BUILDMASTER_TOOLS_CMAKE_SRCDIR, …) and ENV_* stay unified with the parent; only compilers/binutils are overridden
  • Incomplete component toolchain snapshots (missing tool *_SRCDIR / ENV_*) that led to failures such as File /configure.cmake.in does not exist when nested projects created further components
  • Nested add_subdirectory(buildmaster) no longer corrupts the shared toolchain.cmake: when BUILDMASTER_CONFIGURED is already TRUE (host loaded the parent dump as CMAKE_TOOLCHAIN_FILE), the root CMakeLists.txt loads helpers, propagates vars, and returns—without toolchain_reset / module re-export / toolchain_write. Previously a nested bootstrap cleared the registry, re-wrote only root-level keys, and overwrote the parent dump, which then broke deeper components with empty template roots (/configure.cmake.in, /component_shared.cmake.in)
  • Meson nested setups with system ld: no longer pass -fuse-ld=/usr/bin/ld (or other absolute linker paths) into c_link_args / cpp_link_args. GCC rejects path-form -fuse-ld=; buildmaster_fuse_ld_flag() only emits driver flavor names, so PostgreSQL and other Meson components configure correctly under a default Linux linker
  • clang-cl + inherited MSVC LTCG flags: when the parent job uses clang-cl (or a stage selects TOOLCHAIN clang-cl), create_cmake_stages / create_meson_stages strip /GL and /LTCG* from C/CXX and linker flags instead of forwarding them. clang-cl was warning unknown argument ignored for /GL; IPO remains driven by CMAKE_INTERPROCEDURAL_OPTIMIZATION_* and Meson b_lto, not by those MSVC-only switches
  • Meson on Windows (MSVC-like toolchains): do not inject /std:c11 into nested Meson c_args. That flag did not fix clang-cl PostgreSQL C99 probes (UCRT complex/tgmath vs Clang _Complex) and could break real cl builds (e.g. Postgres VA_ARGS_NARGS_ / non-constant initializers). Prefer TOOLCHAIN msvc for PostgreSQL on Windows. Upstream projects set their own C standard; only /Z7 is still appended for CodeView on MSVC-like drivers
  • Meson stages: SCCACHE_DIR path normalization wrote into CCACHE_DIR instead of SCCACHE_DIR, so sccache cache directories could be lost or overwrite the ccache path during nested Meson setup
  • Dependant configure targets (component_*_dependant.cmake.in): under the Ninja generator, long configures (e.g. FFmpeg meson setup) looked hung — the silent env runner swallowed message(STATUS) from the configure -P script. Makefiles still printed progress. Now each dependant configure target sets USES_TERMINAL and a clear COMMENT "Configuring <component>" so Ninja shows the step as soon as it starts
  • Dependant configure progress on Windows + Ninja: cmake -E echo "Configuring …" plus the same COMMENT concatenated on one line (Configuring x265Configuring x265). Dropped the redundant echo; a single COMMENT is enough
  • Dependant components: indent_level is forced to 0 in create_component when a dependency is set. Hierarchical tabs are only meaningful in the parent configure log (message_indented); dependant stages run at build time and must not inherit plugin-level indentation in status lines or nested stage scripts

Version 1.0.0

Choose a tag to compare

@StormBytePP StormBytePP released this 23 Aug 18:06

Initial public release of StormByte-BuildMaster: a CMake DSL to configure, build, install and consume external CMake and Meson projects as first-class parts of a parent tree, with stage-based orchestration, explicit targets, coherent environment propagation, portable static-library bundling, and controlled failure propagation across the dependency graph.

Added

Core orchestration

  • Parent-configure generation of configure / build / install stage scripts for external projects
  • Explicit stage targets: <component>_configure, <component>_build, <component>_install
  • <component>_build depends on <component>_configure
  • Shared install prefix (BUILDMASTER_INSTALL_DIR) and generated script tree across the whole dependency graph
  • Safe recursive nesting via BUILDMASTER_CONFIGURED (single initialization, no prefix fights)
  • IMPORTED targets (static and shared, including MSVC import libraries and DLLs) wired to install stages
  • INTERFACE libraries also depend on <component>_install so parent target_link_libraries waits for a successful install
  • Simple API: create_cmake_component, create_meson_component
  • Dependant variants: create_cmake_dependant_component, create_meson_dependant_component
  • Advanced/explicit API: create_cmake_stages, create_meson_stages
  • Project version exposed as BUILDMASTER_VERSION and shown in the bootstrap status line

Fail-fast and failure propagation

  • Optional BUILDMASTER_FAIL_FAST (env or -D; truthy: 1 / ON / TRUE / YES; default OFF)
  • On stage failure with fail-fast ON: write markers/buildmaster.failed and markers/<component_id>.failed
  • Later stages print Skipped <component title> and exit non-zero when the global marker exists
  • Env runners refuse further work if the global fail marker is present (Skipped due to previous errors)
  • Unique buildmaster_build_init target resets the markers directory at the start of every parent build (Ninja, Make, cmake --build)
  • Markers directory under ${BUILDMASTER_BINDIR}/markers/ (no persistent success stamps)
  • Fail-fast OFF writes no markers so independent components can keep building (cache warming with ccache/sccache)
  • Stage exec scripts (configure_exec / build_exec / install_exec for CMake; setup_exec / compile_exec / install_exec for Meson) centralize exit-code handling and marker writes

Environment and toolchain

  • Platform env runners (Linux/macOS shell, Windows batch) with silent variants
  • Propagation of compilers, flags, PATH, PKG_CONFIG_PATH, LIB, INCLUDE
  • Compiler-cache support (CMAKE_*_COMPILER_LAUNCHER, CCACHE_DIR, SCCACHE_DIR) into child CMake and Meson builds
  • Optional full live output (BUILDMASTER_DEBUG) and verbose compile-only output (BUILDMASTER_VERBOSE)
  • Failure diagnostics: silent runners dump captured logs on non-zero exit

CMake and Meson backends

  • Nested CMake configures with Ninja, toolchain file, PIC, LTO and launcher injection
  • Nested Meson setup/compile/install with matching environment and library type control
  • Parallel builds via NPROC / CMAKE_BUILD_PARALLEL_LEVEL
  • Stage targets depend on buildmaster_build_init when available

File helpers

  • Cache-aware downloads (file_download_cached) with hash verification and retries
  • Force downloads (file_download) with progressive backoff
  • Flexible EXPECTED_HASH (ALGORITHM=digest, including forms such as SHA3_256=…; bare digest defaults to SHA256)
  • Portable archive extraction (file_decompress) via file(ARCHIVE_EXTRACT)
  • Strict path-traversal protection and consistent status messages

Git helpers

  • Generated fragments for fetch, reset/clean, patch apply and branch switch
  • API binds each operation to a component id (same id as create_*_component):
    • create_git_reset_file(out, component_id, title, repo)
    • create_git_patch_file(out, component_id, title, repo, patches)
    • create_git_fetch(out, component_id, title, repo)
    • create_git_switch_branch(out, component_id, title, repo, branch)
  • Registered git scripts run at the start of <component>_configure (before nested CMake/Meson setup), in registration order
  • Call create_git_* before create_*_component / create_*_stages for that component
  • Optional aggregate target buildmaster_clean (BUILDMASTER_CLEAN_RESET_REPOS, default ON)
    • Only components that used create_git_* are affected
    • Per component: git reset --hard + git clean -fd from the git toplevel (rev-parse --show-toplevel)
    • Invalidates that component’s configure (removes Meson build.ninja / meson-private, or CMake CMakeCache.txt / build.ninja under the component build dir)
    • Next cmake --build / ninja / make re-enters <component>_configure → re-applies git ops → nested setup → build
    • Controlled exclusively via environment variable (falsy: 0 / OFF / FALSE / NO)
    • Propagated to nested BuildMaster instances through the toolchain file
    • Not wired to the generator’s native clean target (unreliable with Ninja); use:
      cmake --build <builddir> --target buildmaster_clean
  • Automatic per-component post-install git reset
    • After a successful *_install, runs reset --hard + clean -fd only for that component’s repo
    • Does not invalidate configure (avoids full reconfigure after every install)
    • Removes the need for manual POST_BUILD reset hooks in consumer projects (e.g. VPX, VMAF)

Static library support

  • Portable static archive merging (create_bundle_static_libraries):
    • Linux: GNU ar -M (MRI)
    • macOS: libtool -static
    • Windows: lib /OUT:
  • Optional post-install rename of static libraries to canonical names

Platform support

  • Linux, Windows (MSVC) and macOS (x86_64 and Apple Silicon)
  • Extra-tool registration (e.g. bundled pkgconf)

Notes

  • Requires CMake ≥ 3.20; Meson and Ninja when using the corresponding backends.
  • Stage scripts are generated at parent configure time — change BUILDMASTER_DEBUG / BUILDMASTER_VERBOSE / BUILDMASTER_FAIL_FAST / BUILDMASTER_CLEAN_RESET_REPOS and re-run CMake to regenerate them.
  • Designed as a building block for multi-dependency projects (e.g. FFmpeg plugin graphs, multi-bitdepth codecs, database client bundles).