HMake: More Correct Than Ninja

HMake employs caching extensively to achieve both build speed and correctness. This article describes HMake's caching in detail and contrasts it with CMake+Ninja in the context of building Clang/LLVM.

In contrast to CMake's 2-step process, HMake has a 3-step process. CMake's steps are cmake ../ and ninja. HMake's steps are hhelper (first run), hhelper (second run), and hbuild. The two hhelper invocations do different things: the first run generates a cache.json file that records the selected toolset and the commands that will be used to compile the configure and build executables. The second run compiles those two executables and immediately runs configure, creating the 3 binary cache files described below.

You can consider steps 2 and 3 of the HMake process as analogous to CMake+Ninja: cmake ../ creates a build.ninja file listing all rules, which ninja then consumes to build the project. The second hhelper invocation compiles 2 binaries ? build.exe and configure.exe ? and creates 3 binary cache files: config-cache.bin, build-cache.bin, and nodes.bin.

nodes.bin is a sequence of variable-length records. Each record is a 16-bit path-length prefix followed by that many bytes of file-path data; the file is read until the buffer is exhausted. There is no up-front count header. All file paths are required to be lexically normal and lower-cased on Windows.

nodes.bin is written at configure-time. During later builds, re-configurations, or re-builds, it may be extended with new file-path entries. Existing entries are never deleted and persist even if unused. The nodes file is always written before config-cache or build-cache. Each of the 3 files is written via a full overwrite (the nodes file by appending new entries to the in-memory nodesCacheGlobal buffer before writing).

Node.hpp is the class that manages this. You call various Node::get functions to obtain a Node*. These functions either add a new normalized path to the nodeAllFiles hash-set or return the existing entry. If no entry existed, a new Node is created, the static Node::idCount is assigned to Node::myId and then incremented, and the new Node* is stored in the nodeIndices array at index Node::myId. writeNodesCacheIfNewNodesAdded is called afterward and only writes the file when at least one new node was added.

nodes.bin is read first (if it exists) to populate both nodeAllFiles and nodeIndices. The following is the exact code in initializeCache that does this:

    if (const auto p = path(configureNode->filePath + slashc + getFileNameJsonOrOut("nodes")); exists(p))
    {
        const string str = p.string();
        nodesCacheGlobal = readBufferFromCompressedFile(str);

        // Nodes loaded from cache are inserted into the hash set and nodeIndices. performSystemCheck() is deferred and
        // later run in parallel.

        const uint64_t bufferSize = nodesCacheGlobal.size();
        uint64_t bufferRead = 0;
        while (bufferRead != bufferSize)
        {
            uint16_t nodeFilePathSize;
            memcpy(&nodeFilePathSize, nodesCacheGlobal.data() + bufferRead, sizeof(uint16_t));
            bufferRead += sizeof(uint16_t);
            Node::getHalfNode(string(nodesCacheGlobal.data() + bufferRead, nodeFilePathSize));
            bufferRead += nodeFilePathSize;
        }
        nodesSizeBefore = Node::idCount;
    }

In config-cache.bin and build-cache.bin, file paths are never stored directly. Instead, only a 32-bit node index is stored. For example, to reference /foo/bar in the build-cache, we call Node *n = Node::getNode("/foo/bar"). If n->myId == 3, then 3 is what gets serialized. On a later run, we recover the Node* via nodeIndices[3].

This keeps both config-cache and build-cache very compact. It also enables a useful remote-build property: to replicate the 3 cache files and 2 binaries on a different machine, the remote only needs to rewrite the path prefix up to the project root in nodes.bin, and all references in config-cache and build-cache become automatically correct.

Now, let's look at what goes into config-cache and build-cache.

BTarget is the fundamental building block of HMake. Every BTarget subclass ? CppTarget, LOAT (LinkOrArchiveTarget), CppSrc, CppMod ? gets its own slice in both caches, and the order of those slices is stable across runs. Unlike Ninja, HMake lets each target type store custom cache data rather than forcing a uniform representation.

To manage uniqueness and stable insertion order, two parallel data structures are used: vector<BTargetCache> bTargetCaches (which owns the ordered entries) and flat_hash_map<uint64_t, uint32_t> nameToIndexMap (which provides O(1) lookup by name into that vector). Uniqueness is keyed on BTarget::cacheName. Some BTarget constructors take this value directly; others hash their string name argument.

Here is a hello-world example:

#include "Configure.hpp"

void configurationSpecification(Configuration &config)
{
    config.getCppExeDSC("app").getSourceTarget().sourceFiles("main.cpp");
}

void buildSpecification()
{
    getConfiguration();
    CALL_CONFIGURATION_SPECIFICATION
}

MAIN_FUNCTION

The line config.getCppExeDSC("app").getSourceTarget().sourceFiles("main.cpp"); defines 3 BTargets: a LOAT named app, a CppTarget named app-cpp, and a CppSrc with a unique cache-name. For the first two, a hash of their name is used as the key for nameToIndexMap. For CppSrc, the constructor computes the key as:

// target_->cacheIndex is the index of the CppTarget in the `BTargetCache` array. We can't use the `node->myId` alone as different `CppTarget` can have same node as its source-file. We also encode the `CppModType` in the 64-bit `cacheName`.
static_cast<uint64_t>(node_->myId) << 32 | static_cast<uint64_t>(target_->cacheIndex) << 3 |
static_cast<uint64_t>(cppModType)


enum class CppModType : uint8_t
{
    CPP_SRC = 0,                 ///< Ordinary source file (non-module).
    PRIMARY_EXPORT = 1,          ///< Module interface unit exporting the primary module.
    PARTITION_EXPORT = 2,      ///< Module interface unit for a module partition.
    HEADER_UNIT = 3,             ///< Header unit (`.h` compiled as a BMI producer).
    PRIMARY_IMPLEMENTATION = 4,  ///< Module implementation unit for the primary module.
};

/// Per-target slices of the on-disk caches managed by `initializeCache()` / `configureOrBuild()` in
/// `BuildSystemFunctions.cpp`.
///
/// On disk (under the configure directory, optionally LZ4-compressed):
/// - `config-cache` ? one entry per target: `cacheName`, sized `configCache` blob (written at configure-time).
/// - `build-cache` ? parallel array: inline `depsCache` (round-0 FULL/WAIT `cacheIndex` list), then sized per-target
///   body; process-launching targets append a 16-byte footer (`cumulativeHash`, `launchTime`).
///
/// `readConfigCache()` / `readBuildCache()` fill `bTargetCaches` and `nameToIndexMap` before `buildSpecification()`.
/// Live `BTarget` constructors attach via `initializeBTarget()`. At configure end or after a build,
/// `getConfigCache()` / `getBuildCache()` serialize updates (build mode re-hashes nodes via `checkNodes(false)` when
/// needed, then rewrites only entries with `buildCacheUpdated` / `buildFooterUpdated`).

/// Order of entries is stable across builds; each target has matching config and build slices.
class BTargetCache
{
  public:
    /// Back-pointer to the live `BTarget` owning this cache entry.
    BTarget *bTarget = nullptr;

    /// Same value as `BTarget::cacheName` ? hash key for this entry in `nameToIndexMap`.
    uint64_t name;

    /// Target-specific config-cache payload (written at configure-time, read-only at build-time).
    string_view configCache;

    /// Inline prefix in the on-disk build-cache entry: round-0 FULL/WAIT dependency `cacheIndex` list (see `readBuildCache()`).
    string_view depsCache;

  private:
    /// Full build-cache blob: body followed by a fixed 16-byte footer (`cumulativeHash` + `launchTime`).
    string_view buildCache;

  public:
    /// Build-cache body excluding the trailing 16-byte footer.
    string_view getBuildCache() const
    {
        return string_view{buildCache.data(), buildCache.size() - 16};
    }
    /// Full build-cache blob including the footer.
    string_view getFullBuildCache() const
    {
        return buildCache;
    }
    /// Trailing 16 bytes: `cumulativeHash` (8) + `launchTime` (8).
    string_view getBuildFooter() const
    {
        const uint64_t starting = buildCache.size() - 16;
        return string_view{buildCache.data() + starting, 16};
    }
    void setBuildCache(const string_view buildCache_)
    {
        buildCache = buildCache_;
    }
};

Note that configCache, depsCache, and buildCache are string_views pointing directly into the configCacheGlobal and buildCacheGlobal buffers that are loaded at startup. Those global buffers must therefore outlive all BTargetCache instances.

We read the config-cache and build-cache into the vector and populate the map immediately after the nodes-cache is read. The following is the code that does this:

void readConfigCache()
{
    const uint32_t bufferSize = configCacheGlobal.size();
    uint32_t bufferRead = 0;

    uint32_t count = 0;
    const char *ptr = configCacheGlobal.data();
    while (bufferRead != bufferSize)
    {
        BTargetCache fileCacheTarget;

        fileCacheTarget.name = readUint64(ptr, bufferRead);
        fileCacheTarget.configCache = readStringView(ptr, bufferRead);

        bTargetCaches.emplace_back(fileCacheTarget);
        nameToIndexMap.emplace(fileCacheTarget.name, count);

        ++count;
    }

    if (bufferRead != bufferSize)
    {
        HMAKE_HMAKE_INTERNAL_ERROR
    }
}

void readBuildCache()
{
    const uint32_t bufferSize = buildCacheGlobal.size();
    uint32_t bytesRead = 0;

    const char *ptr = buildCacheGlobal.data();
    for (BTargetCache &fileCacheTarget : bTargetCaches)
    {
        // reading the deps-cache-inline
        const uint32_t offset = bytesRead;
        const uint32_t depsSize = readUint32(ptr, bytesRead);
        bytesRead += 4 * depsSize;

        fileCacheTarget.depsCache = {ptr + offset, bytesRead - offset};
        fileCacheTarget.setBuildCache(readStringView(ptr, bytesRead));
    }

    if (bytesRead != bufferSize)
    {
        HMAKE_HMAKE_INTERNAL_ERROR
    }
}

string getConfigCache()
{
    string configCache;
    for (const BTargetCache &fileCacheTarget : bTargetCaches)
    {
        writeUint64(configCache, fileCacheTarget.name);

        const uint32_t currentSize = configCache.size();
        // Reserve 4 bytes for the serialized size prefix.
        writeUint32(configCache, 0);
        if (fileCacheTarget.bTarget)
        {
            fileCacheTarget.bTarget->writeConfigCacheAtConfigTime(configCache);
        }

        // writing size to the placeholder above.
        const uint32_t size = configCache.size() - (currentSize + 4);
        memcpy(configCache.data() + currentSize, &size, sizeof(size));
    }
    return configCache;
}

string getBuildCache()
{
    string buildCache;
    if constexpr (bsMode == BSMode::CONFIGURE)
    {
        for (const BTargetCache &fileCacheTarget : bTargetCaches)
        {
            if (fileCacheTarget.depsCache.empty())
            {
                writeUint32(buildCache, 0); // no deps yet: write count = 0
            }
            else
            {
                buildCache.append(fileCacheTarget.depsCache.data(), fileCacheTarget.depsCache.size());
            }

            const uint32_t currentSize = buildCache.size();
            // Reserve 4 bytes for the serialized size prefix.
            writeUint32(buildCache, 0);
            if (BTarget *bt = fileCacheTarget.bTarget; bt && fileCacheTarget.bTarget->newlyAdded)
            {
                fileCacheTarget.bTarget->writeBuildCacheAtConfigTime(buildCache);
                if (bt->launchesProcess)
                {
                    writeUint64(buildCache, bt->realBTargets[0].cumulativeHash);
                    writeUint64(buildCache, bt->realBTargets[0].launchTime);
                }
            }
            else
            {
                buildCache.append(fileCacheTarget.getFullBuildCache().begin(),
                                  fileCacheTarget.getFullBuildCache().end());
            }

            // writing size to the placeholder above.
            const uint32_t size = buildCache.size() - (currentSize + 4);
            memcpy(buildCache.data() + currentSize, &size, sizeof(size));

            if (ndeb == NDEB::NO)
            {
                if (BTarget *bt = fileCacheTarget.bTarget; bt && fileCacheTarget.bTarget->newlyAdded)
                {
                    fileCacheTarget.bTarget->verifyBuildCache(string_view{buildCache.data() + currentSize + 4, size});
                }
            }
        }
        return buildCache;
    }

    bool cacheUpdated = false;
    for (const BTargetCache &fileCacheTarget : bTargetCaches)
    {
        if (fileCacheTarget.bTarget)
        {
            if (fileCacheTarget.bTarget->buildCacheUpdated || fileCacheTarget.bTarget->buildFooterUpdated)
            {
                cacheUpdated = true;
                break;
            }
        }
    }
    if (!cacheUpdated)
    {
        return buildCache;
    }

    Builder::checkNodes(false);
    for (const BTargetCache &fileCacheTarget : bTargetCaches)
    {
        if (fileCacheTarget.depsCache.empty())
        {
            writeUint32(buildCache, 0); // no deps yet: write count = 0
        }
        else
        {
            buildCache.append(fileCacheTarget.depsCache.data(), fileCacheTarget.depsCache.size());
        }

        const uint32_t currentSize = buildCache.size();
        // Reserve 4 bytes for the serialized size prefix.
        writeUint32(buildCache, 0);

        if (const BTarget *bt = fileCacheTarget.bTarget; bt && (bt->buildFooterUpdated || bt->buildCacheUpdated))
        {
            if (fileCacheTarget.bTarget->buildCacheUpdated)
            {
                fileCacheTarget.bTarget->writeBuildCacheAtBuildTime(buildCache);
            }
            if (fileCacheTarget.bTarget->buildFooterUpdated)
            {
                fileCacheTarget.bTarget->writeBuildCacheHeaderAtBuildTime(buildCache);
            }
        }
        else
        {
            buildCache.append(fileCacheTarget.getFullBuildCache().begin(), fileCacheTarget.getFullBuildCache().end());
        }

        // writing size to the placeholder above.
        const uint32_t size = buildCache.size() - (currentSize + 4);
        memcpy(buildCache.data() + currentSize, &size, sizeof(size));

        if (ndeb == NDEB::NO)
        {
            if (const BTarget *bt = fileCacheTarget.bTarget; bt && (bt->buildFooterUpdated || bt->buildCacheUpdated))
            {
                const string_view written{buildCache.data() + currentSize + 4, size};
                fileCacheTarget.bTarget->verifyBuildCache(written);
            }
        }
    }
    return buildCache;
}

// please see that we get the config-cache or build-cache and write it only after the nodes-cache
bool configureOrBuild()
{
    builderPtr = new Builder{};

    if constexpr (bsMode == BSMode::CONFIGURE)
    {
        if (!builderPtr->errorHappenedInRoundMode)
        {
            cache.registerCacheVariables();
            const string configCache = getConfigCache();
            const string buildCache = getBuildCache();
            writeNodesCacheIfNewNodesAdded();
            writeBufferToCompressedFile(configureNode->filePath + slashc + getFileNameJsonOrOut("config-cache"),
                                        configCache);
            writeBufferToCompressedFile(configureNode->filePath + slashc + getFileNameJsonOrOut("build-cache"),
                                        buildCache);
        }
    }
    else
    {
        const string buildCache = getBuildCache();
        writeNodesCacheIfNewNodesAdded();
        if (!buildCache.empty())
        {
            writeBufferToCompressedFile(configureNode->filePath + slashc + getFileNameJsonOrOut("build-cache"),
                                        buildCache);
        }
    }

    return builderPtr->errorHappenedInRoundMode;
}

config-cache is read first because only it contains the cacheName of each BTarget. Once readConfigCache() completes, the number of BTargetCache entries is known and the vector is fully populated. readBuildCache() then iterates over that vector to read the corresponding build-cache slice for each entry. An empty entry costs very little: even if a target has been deleted (its bTarget is null), getConfigCache() still writes the 8-byte name and a 4-byte zero-size payload ? 12 bytes per stale entry. The build-cache for such a target is similarly preserved intact: getBuildCache() in CONFIGURE mode copies the existing build-cache bytes for any entry with a null bTarget.

Config-cache is rewritten in full for every target at configure-time. The build-cache persists for deleted targets as long as they remain in the bTargetCaches vector from the last config.

We read the config-cache and build-cache before the buildSpecification function call and before any BTarget construction. The BTarget constructor at configure-time initializes cacheIndex from the passed cacheName using nameToIndexMap. bTargetCaches[cacheIndex] then represents the respective config-cache and build-cache slice for that target.

Static deps (FULL/WAIT dependencies registered between BTargets) are automatically serialized into the build-cache and compared on the next build. If the set of static deps has changed, HMake sets realBTargets[0].updateStatus = UpdateStatus::UPDATE_NEEDED. This makes HMake more correct than Ninja, which cannot detect changes to implicit dependencies that are not covered by the command line. (ninja#1522)

While the static-dep check is automatic, BTarget::setFileStatus must be called explicitly by each target subclass to determine whether the target itself needs to run. The following is its implementation:

/// Determines whether `realBTargets[0]` needs to run. No-op if `updateStatus != UNCHECKED`.
///
/// When `launchesProcess` is true: compares `realBTargets[0].cumulativeHash` against the stored value in the
/// build-cache footer ? a mismatch immediately sets `UPDATE_NEEDED`. Otherwise, `highestTime` starts at
/// `realBTargets[0].launchTime` (restored from the footer by `initializeBTarget()`).
///
/// When `launchesProcess` is false: `highestTime` starts at 0.
///
/// Recurses over FULL/WAIT dependencies (calling `setFileStatus()` on any that are still `UNCHECKED`). For each:
/// - If the dependency is `UPDATE_NEEDED`, this target is also `UPDATE_NEEDED`.
/// - If `launchesProcess` and `depRb->launchTime > highestTime` (a dep was rebuilt after this target last ran),
///   this target is `UPDATE_NEEDED`.
/// - Otherwise `highestTime = max(highestTime, depRb->launchTime)`.
///
/// If no dependency triggers a rebuild, sets `UPDATE_NOT_NEEDED`. When `launchesProcess` is false, propagates
/// `highestTime` into `realBTargets[0].launchTime` so upstream targets can detect downstream rebuilds.
///
/// Subclasses (`CppSrc`, `CppMod`, `LOAT`, `HeaderGen`) compute `cumulativeHash` from content hashes of inputs
/// and call this base implementation via `ObjectFile::setFileStatus()` / `PLOAT::setFileStatus()`.
virtual void setFileStatus();


void BTarget::setFileStatus()
{
    RealBTarget &rb = realBTargets[0];
    if (rb.updateStatus != UpdateStatus::UNCHECKED)
    {
       return;
    }

    uint64_t highestTime;
    if (launchesProcess)
    {
        uint32_t bytesRead = 0;
        const char *ptr = bTargetCaches[cacheIndex].getBuildFooter().data();
        if (const uint64_t compileHash = readUint64(ptr, bytesRead); compileHash != rb.cumulativeHash)
        {
            rb.updateStatus = UpdateStatus::UPDATE_NEEDED;
            return;
        }
        highestTime = rb.launchTime;
    }
    else
    {
        highestTime = 0;
    }

    for (const RBTWithType &rbt : rb.dependencies)
    {
        if (rbt.getRelationType() == RelationType::FULL || rbt.getRelationType() == RelationType::WAIT)
        {
        }
        else
        {
            continue;
        }
        const RealBTarget *depRb = rbt.getPointer();

        if (depRb->updateStatus == UpdateStatus::UNCHECKED)
        {
            depRb->getBTarget()->setFileStatus();
        }

        if (depRb->updateStatus == UpdateStatus::UPDATE_NEEDED)
        {
            rb.updateStatus = UpdateStatus::UPDATE_NEEDED;
            return;
        }

        if (depRb->launchTime > highestTime)
        {
            if (launchesProcess)
            {
                rb.updateStatus = UpdateStatus::UPDATE_NEEDED;
                return;
            }
            highestTime = depRb->launchTime;
        }
    }

    rb.updateStatus = UpdateStatus::UPDATE_NOT_NEEDED;

    if (!launchesProcess)
    {
        // highest time is set as the highest time of one of our dependencies.
        rb.launchTime = highestTime;
    }
}

Notice that we also cache and compare launchTimes. This ensures that if a BTarget was skipped in a previous build (e.g. because it was not in the set of targets requested by the user), it will be updated on the next run that does include it.

The following are the setFileStatus overrides and cache-management functions of LOAT, CppSrc, and CppMod:

void LOAT::setFileStatus()
{
    RealBTarget &rb = realBTargets[0];
    if (rb.updateStatus != UpdateStatus::UNCHECKED)
    {
        return;
    }

    if (outputFileNode->fileType == file_type::not_found)
    {
        rb.updateStatus = UpdateStatus::UPDATE_NEEDED;
        return;
    }

    {
        STACK_PMR_STRING(linkWithTargets, 64 * 1024)
        setLinkOrArchiveCommands(linkWithTargets, true);
        rb.cumulativeHash = rapidhash(linkWithTargets.data(), linkWithTargets.size());
    }

    PLOAT::setFileStatus();
}
void CppSrc::setFileStatus()
{
    RealBTarget &rb = realBTargets[0];
    if (rb.updateStatus != UpdateStatus::UNCHECKED)
    {
       return;
    }

    if (node->fileType == file_type::not_found)
    {
        printErrorMessage(FORMAT("Source-File {}\n of target {}\n not found.\n", node->filePath, target->name));
    }

    if (objectNode->fileType == file_type::not_found)
    {
        rb.updateStatus = UpdateStatus::UPDATE_NEEDED;
        return;
    }

    // command-hash + source-hash + cacheHeaderFiles. 8 for uint64_t
    STACK_PMR_VECTOR(uint64_t, contentHashes, cachedHeaderFiles.size() * 8 + 2)
    contentHashes.emplace_back(commandHash);
    contentHashes.emplace_back(node->contentHash);
    for (const uint32_t nodeIndex : cachedHeaderFiles)
    {
        contentHashes.emplace_back(Node::getHalfNode(nodeIndex)->contentHash);
    }
    rb.cumulativeHash = rapidhash(contentHashes.data(), contentHashes.size() * 8);

    ObjectFile::setFileStatus();
}
void CppMod::setFileStatus()
{
    RealBTarget &rb = realBTargets[0];
    if (rb.updateStatus != UpdateStatus::UNCHECKED)
    {
       return;
    }

    if (node->fileType == file_type::not_found)
    {
        string str;
        if (type == CppModType::HEADER_UNIT)
        {
            str = "C++HeaderUnit";
        }
        else if (type == CppModType::PRIMARY_IMPLEMENTATION)
        {
            str = "C++Module";
        }
        else
        {
            str = "C++InterfaceModule";
        }

        printErrorMessage(FORMAT("{} {}\n of target {}\n not found.\n", str, node->filePath, target->name));
    }

    rb.updateStatus = UpdateStatus::UPDATE_NEEDED;

    if (type == CppModType::HEADER_UNIT)
    {
        if (interfaceNode->fileType == file_type::not_found)
        {
            return;
        }
    }
    else if (type == CppModType::PRIMARY_IMPLEMENTATION)
    {
        if (objectNode->fileType == file_type::not_found)
        {
            return;
        }
    }
    else
    {
        if (interfaceNode->fileType == file_type::not_found || objectNode->fileType == file_type::not_found)
        {
            return;
        }
    }

    for (const uint32_t depIndex : cachedDeps)
    {
        CppMod *cppMod = static_cast<CppMod *>(bTargetCaches[depIndex].bTarget);

        // Can happen because the export-name or the include-name got mapped to a different file in the same target.
        if (!cppMod)
        {
            return;
        }

        const RealBTarget *depRb = &cppMod->realBTargets[0];
        if (depRb->updateStatus == UpdateStatus::UNCHECKED)
        {
            cppMod->setFileStatus();
        }

        if (depRb->updateStatus == UpdateStatus::UPDATE_NEEDED)
        {
            return;
        }

        if (depRb->launchTime > rb.launchTime)
        {
            return;
        }
    }

    rb.updateStatus = UpdateStatus::UNCHECKED;

    // command-hash + source-hash + cachedHeaderFiles. 8 for uint64_t
    STACK_PMR_VECTOR(uint64_t, contentHashes, cachedHeaderFiles.size() * 8 + 2)
    contentHashes.emplace_back(commandHash);
    contentHashes.emplace_back(node->contentHash);
    for (const uint32_t nodeIndex : cachedHeaderFiles)
    {
        contentHashes.emplace_back(Node::getHalfNode(nodeIndex)->contentHash);
    }
    rb.cumulativeHash = rapidhash(contentHashes.data(), contentHashes.size() * 8);

    ObjectFile::setFileStatus();
}


// Cache writing functions. LOAT does not write any build-cache beside the footer.


void CppSrc::writeBuildCacheAtConfigTime(string &buffer)
{
    // sizeof header-files
    writeUint32(buffer, 0);
}

void CppSrc::writeBuildCacheAtBuildTime(string &buffer)
{
    RealBTarget &rb = realBTargets[0];
    // command-hash + source-hash + headerFiles. 8 for uint64_t
    STACK_PMR_VECTOR(uint64_t, contentHashes, headerFiles.size() * 8 + 2)
    contentHashes.emplace_back(commandHash);
    contentHashes.emplace_back(node->contentHash);
    for (const Node *headerNode : headerFiles) // headerFiles, not cachedHeaderFiles
    {
        if (headerNode->lastWriteTime > rb.launchTime)
        {
            // File was modified after process launched ? hash is stale.
            contentHashes.emplace_back(0);
        }
        else
        {
            contentHashes.emplace_back(headerNode->contentHash);
        }
    }
    rb.cumulativeHash = rapidhash(contentHashes.data(), contentHashes.size() * 8);
    writeUint32(buffer, headerFiles.size());
    for (const Node *header : headerFiles)
    {
        writeNode(buffer, header);
    }
}

void CppMod::writeBuildCacheAtConfigTime(string &buffer)
{
    // headerStatusChanged
    writeBool(buffer, true);
    // sizeof header-files
    writeUint32(buffer, 0);
    // sizeof cppMod-deps
    writeUint32(buffer, 0);
}

void CppMod::writeBuildCacheAtBuildTime(string &buffer)
{
    RealBTarget &rb = realBTargets[0];

    // command-hash + source-hash + container-size. 8 for uint64_t
    STACK_PMR_VECTOR(uint64_t, contentHashes, (target->useIPC ? composingHeaders.size() : headerFiles.size()) * 8 + 2)
    contentHashes.emplace_back(commandHash);
    contentHashes.emplace_back(node->contentHash);

    if (target->useIPC)
    {
        for (const auto &[includeName, headerNode] : composingHeaders)
        {
            if (headerNode->lastWriteTime > originalLaunchTime)
            {
                // File was modified after process launched ? hash is stale.
                contentHashes.emplace_back(0);
            }
            else
            {
                contentHashes.emplace_back(headerNode->contentHash);
            }
        }
    }
    else
    {
        for (Node *headerNode : headerFiles)
        {
            if (headerNode->lastWriteTime > originalLaunchTime)
            {
                // File was modified after process launched ? hash is stale.
                contentHashes.emplace_back(0);
            }
            else
            {
                contentHashes.emplace_back(headerNode->contentHash);
            }
        }
    }
    rb.cumulativeHash = rapidhash(contentHashes.data(), contentHashes.size() * 8);

    // headerStatusChanged. directly written as false
    writeBool(buffer, false);
    if (target->useIPC)
    {
        writeUint32(buffer, composingHeaders.size());
        for (const auto &[includeName, headerNode] : composingHeaders)
        {
            writeNode(buffer, headerNode);
        }
    }
    else
    {
        // sizeof header-files
        writeUint32(buffer, headerFiles.size());
        for (const Node *header : headerFiles)
        {
            writeNode(buffer, header);
        }
    }

    const uint32_t currentSize = buffer.size();
    uint32_t count = 0;
    // placeholder for direct-deps count;
    writeUint32(buffer, 0);

    // make this pmr string
    string cacheBuffer;
    for (const CppModWithDirect &cppModDirect : allCppModDeps)
    {
        if (cppModDirect.isDirect())
        {
            ++count;
            writeUint32(buffer, cppModDirect.getPointer()->cacheIndex);
        }
    }

    memcpy(buffer.data() + currentSize, &count, sizeof(count));
}

A few things to notice:

1. All 3 subclasses set cumulativeHash before calling the base setFileStatus (via ObjectFile::setFileStatus() or PLOAT::setFileStatus(), which ultimately delegate to BTarget::setFileStatus()). This is what makes the hash comparison work.
2. LOAT::setFileStatus hashes the link command (not the compile command), and this command intentionally excludes the object files and libraries, because those are BTarget dependencies and are therefore already covered by the automatic static-deps check and BTarget::setFileStatus. LOAT does not write any custom build-cache payload; the cumulativeHash and launchTime are automatically managed in the 16-byte footer.
3. Both CppSrc and CppMod, when they complete successfully, set both buildCacheUpdated and buildFooterUpdated. LOAT only sets buildFooterUpdated. The build-footer ? containing cumulativeHash and launchTime ? is automatically managed by the build system for all process-launching targets.
4. Both CppSrc and CppMod, when they complete successfully, set Node::doHashFile for all discovered header files. Builder::checkNodes(false) is then run again to compute content hashes for any node that was not previously hashed.
5. In CppSrc::writeBuildCacheAtBuildTime, if a header file was modified after the compilation process was launched, a zero hash is stored instead of the actual content hash. This keeps the build-cache valid while forcing a re-build on the next build.
6. CppMod does the same but compares against originalLaunchTime rather than rb.launchTime. rb.launchTime is advanced to system_clock::now() each time the build system forwards a completed provider module to a waiting consumer compiler process. This advancement is deliberate: it guarantees that the consumer's launchTime is always greater than that of every provider it depended on, so setFileStatus's time-comparison logic on the next build will never see a dependency with a higher launchTime and trigger a spurious rebuild. However, it also means rb.launchTime reflects when the last dependency was delivered, not when the consumer's compiler process actually started. originalLaunchTime captures that true start time. Without this distinction, a header file edited early in the compilation window ? after the process started but before rb.launchTime was bumped ? would appear unmodified, causing a stale hash to be cached and the next build to incorrectly skip recompilation.
7. In CppMod::writeBuildCacheAtBuildTime, only direct CppMod deps are cached, not the transitive closure.
8. The loop in CppMod::setFileStatus over cachedDeps is a span<uint32_t> containing indices into the bTargetCaches build-cache. This loop replicates exactly what the build system does for static deps in BTarget::setFileStatus: recurse, check UPDATE_NEEDED, and check launchTime.

What goes in config-cache is a deliberate trade-off. The goal is to minimize zero-target build time. Anything expensive to compute can be stored in config-cache and loaded cheaply at build-time, but that shifts cost to configure-time and increases cache size. Consider Example8:

#include "Configure.hpp"

void configurationSpecification(Configuration &config)
{
    config.getCppExeDSC("app").getSourceTarget().moduleDirs("Mod_Src/");
}

void buildSpecification()
{
    getConfiguration().assign(IsCppMod::YES);
    CALL_CONFIGURATION_SPECIFICATION
}

MAIN_FUNCTION

moduleDirs only runs at configure-time. It walks the directory, registers all files with relevant extensions as interface or module files, and records export names (which for moduleDirs match the file name). At build-time this function is a no-op. The cost of scanning the directory on every build would be too high. The trade-off is that if a user adds a new file to the directory and rebuilds without reconfiguring, the build will not see it. Reconfiguration is required. Most user-facing functions follow this same pattern.

The following functions write the config-cache for CppTarget, CppSrc, and CppMod:

void CppTarget::writeConfigCacheAtConfigTime(string &buffer)
{
    writeUint32(buffer, reqDeps.size());
    for (const CppTarget *r : reqDeps)
    {
        writeUint32(buffer, r->cacheIndex);
    }

    const bool hasObjFiles = !srcFileDeps.empty() || !modFileDeps.empty() || !imodFileDeps.empty();
    writeBool(buffer, hasObjFiles);

    writeUint32(buffer, srcFileDeps.size());
    for (const CppSrc *source : srcFileDeps)
    {
        writeNode(buffer, source->node);
    }

    writeUint32(buffer, modFileDeps.size());
    for (const CppMod *cppMod : modFileDeps)
    {
        writeNode(buffer, cppMod->node);
    }

    writeUint32(buffer, imodFileDeps.size());
    for (const CppMod *cppMod : imodFileDeps)
    {
        writeNode(buffer, cppMod->node);
        writeBool(buffer, cppMod->type == CppModType::PRIMARY_EXPORT);
    }

    writeUint32(buffer, huDeps.size());
    for (const CppMod *hu : huDeps)
    {
        writeNode(buffer, hu->node);
    }

    writeNode(buffer, myBuildDir);

    if (configuration->evaluate(IsCppMod::NO) || !useIPC)
    {
        writeIncDirsAtConfigTime(buffer, reqIncls);
        writeIncDirsAtConfigTime(buffer, useReqIncls);
    }

    if (configuration->evaluate(IsCppMod::YES))
    {
        writeHeaderFilesAtConfigTime(buffer, reqHeaderNameMapping);
        writeHeaderFilesAtConfigTime(buffer, useReqHeaderNameMapping);
    }
}

void CppSrc::writeConfigCacheAtConfigTime(string &buffer)
{
    const string fileNumber = toString(node->myId);
    objectNode =
        Node::getNode(target->myBuildDir->filePath + slashc + node->getFileName() + fileNumber + ".o", true, true);
    writeNode(buffer, objectNode);
}

void CppMod::writeConfigCacheAtConfigTime(string &buffer)
{
    const string fileNumber = toString(node->myId);
    const bool isHU = type == CppModType::HEADER_UNIT;
    const bool isImpl = type == CppModType::PRIMARY_IMPLEMENTATION;

    if (!isImpl)
    {
        interfaceNode = Node::getNode(target->myBuildDir->filePath + slashc + node->getFileName() + fileNumber + ".ifc",
                                      true, true);
        writeNode(buffer, interfaceNode);
        if (!isHU)
        {
            writeStringView(buffer, logicalNames[0]);
        }
    }

    if (!isHU)
    {
        objectNode =
            Node::getNode(target->myBuildDir->filePath + slashc + node->getFileName() + fileNumber + ".o", true, true);
        writeNode(buffer, objectNode);
        writeUint32(buffer, logicalNames.size());
        for (const string &str : logicalNames)
        {
            writeStringView(buffer, str);
        }
    }
    else
    {
        writeBool(buffer, isReqHu);
        writeBool(buffer, isUseReqHu);
        writeUint32(buffer, composingHeaders.size());
        writeUint32(buffer, logicalNames.size());
        if (target->useIPC)
        {
            for (const auto &[headerName, headerNode] : composingHeaders)
            {
                writeStringView(buffer, headerName);
                writeNode(buffer, headerNode);
            }
        }
        else
        {
            for (const auto &[headerName, headerNode] : composingHeaders)
            {
                writeStringView(buffer, headerName);
            }
        }
        for (const string &str : logicalNames)
        {
            writeStringView(buffer, str);
        }
    }
}

Two things worth highlighting:

The node index is added in the object-file name and to the .d file name ? e.g., main.cpp0000000E.d and main.cpp0000000E.o. This disambiguates object files when two source files share the same name but live in different directories. (CMake handles the same problem by mirroring the directory structure relative to the source root inside the build directory.)

The full compile command is not stored in the cache. In HMake, every Configuration shares the same base compile command; targets extend it with additional include directories or compile definitions. The base command lives in build.exe and is completed at build-time. This keeps the total cache small compared to Ninja, which stores every complete command in build.ninja.

For a LLVM Clang's build, Ninja's combined cache (build.ninja + .ninja_log + .ninja_deps) comes to around 70 MB. HMake's combined cache (build.exe + configure.exe + build-cache.bin + config-cache.bin + nodes.bin) comes to around 10 MB (For 2 configs btw). Zero-target build time is 60 ms with HMake versus 260 ms with Ninja.

Bottom-Line

Whatever you are using today, it does not matter. Whether it be PCH, jumbo-build or destributed-build / caching, it does not matter. Switching to HMake would result in 2x-4x build speed-up.

Post-Script

1. HMake does not yet support restat, which Ninja does. The implementation is straightforward for static graphs but slightly difficult when modules are involved. There is a clear path forward; this will be addressed in a later iteration along with comprehensive tests.
2. Added a new test that comprehensively covers workflows where a code-generation tool is compiled as part of the build, run to produce source files, and then those outputs are consumed by subsequent targets. This is straightforward to express in HMake and behaves exactly as expected ? including arbitrarily deep chains where the code-generator itself depends on code produced by another in-build code-gen tool.
3. Further improved the build algorithm so that IPC-based builds now behave correctly even under constrained configurations such as 32 threads with 15 GB of RAM. See the comments on RealBTarget::insertionIndex for details.